Repository navigation
Conversation
leozhad
force-pushed
the
leozh/cron-mixed-refusal-status
branch
from
November 20, 2026 23:30
64a4090 to
795ab9e
Compare
leozhad
force-pushed
the
leozh/cron-mixed-refusal-status
branch
from
November 21, 2026 02:05
795ab9e to
568cb4c
Compare
A cron run whose agent spec has a PreToolUse hook that a policy refuses to run (for example a cron profile that turns script_hooks off) loses every call that hook guards. When the model then ran one other call, the verdict saw an approved call, no security block, and recorded ok, so the job read healthy while its work did not happen. The flag that says a policy refused the hook (ScriptHookResult.refused) now reaches the run's tally: permission_pre_tool_block reports it through an optional callback, the cron spec floor returns it beside the reason, the gate logs that refusal under mechanism spec_hook_refused, and on_tool_gate carries it as a fifth argument. A run with an approved call, no security block and at least one such call records error with its own reason head, banners the result, and moves neither counter. Every other refusal beside an approved call keeps the run ok. A partly security-blocked run that also lost such calls names them after its security-gate part, under the hook head without its "of M", so the reason no longer hides the calls that stopped the job. Closes kirodotdev#39973
leozhad
force-pushed
the
leozh/cron-mixed-refusal-status
branch
from
November 21, 2026 04:08
568cb4c to
89c17cf
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem / Motivation
Goal: a cron run that lost tool calls because a policy refused to run the agent spec's PreToolUse hook shows as Error and names those calls, even when another call ran or another call hit the security gate, instead of reading
okor naming only the security block.A scheduled agent job runs an agent whose spec has a PreToolUse hook on file reads. The profile bound to scheduled jobs turns
script_hooksoff, so the hook is refused and every file read is blocked, as #34927 decided. The model then runs one other call, such as a shellls, and that call is approved. The run recordslast_status: ok, its history row readssuccess, andrecord_success()resets the failure counter. The job reads healthy while its work does not happen. When the same run also has one security-gate block, it readserror, but its reason and itsBlocked:banner name only that block, never the calls the refused hook blocked, as a comment on #39973 shows.Why it matters
A refused hook blocks the same calls on every run, and no adapting brings them back. In #39973, each of the 11 silent scheduled jobs that ran in the three days the audit log covered lost its reads this way, and every one still read
ok. Only the audit log and the model's own prose said otherwise.Not a goal
policy_deny, a hook that ran and exited 2, a hook whose firing raised, a batch-guard deferral, or a rejected or timed-out approval. Those runs stayok.What changed (motivation → approach → change)
last_statusbeforelast_statusnowok, and a failure streak is reseterror, the reason names those callserror, the reason names only the security blockerror, the reason names the security block, then the hook-blocked callsThe cause: the hook's result records that a policy refused to run it (
ScriptHookResult.refused), but that flag stopped atpermission_pre_tool_block, which returns only a reason string. So the gate logged every spec-hook block asmechanism: spec_hook, and the run's tally could not tell a refused hook from a hook's own verdict. With an approved call and no security block,_apply_gate_verdictrecorded a success.The flag now reaches the tally.
permission_pre_tool_blockinhook_runtime/hook_dispatch.pytakes an optionalon_policy_refusedcallback, so its other callers are unchanged. The cron spec floor (CronTurnHooks.refusal) returns(reason, policy_refused)._resolve_permissionaudits that block under a new mechanism,spec_hook_refused.on_tool_gategets a fifth argument,hook_refused._GateTallycounts those calls and keeps the first three by name, and a new arm recordserror, moves neither counter and banners the result. The reason readsN of M tool call(s) blocked because policy refused to run a hook:, then each call with its hook and refusal. It does not match the Partial pattern, so the Schedule page shows Error. Adelete_after_runone-shot is kept and paused, as any run ending inerroris. A partly security-blocked run that also lost such calls keeps its security-gate reason and adds. N tool call(s) blocked because policy refused to run a hook:and those calls; the two parts share the 500-character line so that neither head is cut, and the reason still opens with the Partial wording. Both new reasons reuse the per-call equal share from #40103, so a long title still gives way before its rule. The gate-verdict table inlearn-cron-dashboard.mdgains the row.Backwards compatibility
Breaking:
stream_and_collectnow callson_tool_gatewith five arguments, not four, andCronTurnHooks.refusalreturns a(reason, policy_refused)tuple, not a string. Writer sweep on origin/main @ f8558e3:src/kiro_crew/slack/gateway_runtime/cron_turn.pypasses_GateTally.noteon both cron paths, which takes the new argument with a default;src/kiro_crew/apps/agent_turn_sdk.pyand every otherstream_and_collectcaller pass noon_tool_gate; the one caller ofCronTurnHooks.refusalis_resolve_permission, throughstream_and_collect(spec_hooks=...); the 19 test callbacks intest_llm_helpers_tool_gate.py,test_llm_helpers_spawn_brief_scan.pyandtest_kas_cron_spec_hooks.pytake the fifth argument.permission_pre_tool_blockgains a keyword with a default, so its four other callers (channel, messaging, sub-agent, task runner) fit, andtest_hooks_composition_contract.pyre-pins its shape.The audit row of a spec-hook block that a policy refused to run reads
mechanism: spec_hook_refused; a hook's own block keepsspec_hook. A partly security-blocked run'slast_errorgains the hook part after its unchanged opening.Reader: src/kiro_crew/llm_helpers_parts/turn_stream.py:stream_and_collect -- cron -- reads the mechanism through
_resolve_permission'son_decisionfunnel and mapsspec_hook_refusedtohook_refused(test_a_hook_a_policy_refused_to_run_is_told_apart_from_its_own_verdict); no other src or website reader compares the mechanism valueReader: website/src/utils/cronPartlyBlocked.ts:isPartlyBlocked -- crew page -- matches only the reason's opening words, which are unchanged (test_a_security_block_names_the_hook_blocked_calls_after_it, test_the_schedule_page_tells_a_partial_block_from_a_total_one)
Tests
Proof: PROVEN test/test_cron_refusal_status.py::TestSchedulerContract::test_a_one_shot_whose_hook_policy_refused_to_run_is_kept, test/test_hooks_composition_contract.py::test_the_moved_names_keep_their_base_shapes, test/test_kas_cron_spec_hooks.py::test_a_cron_turn_gates_its_permission_requests_on_the_specs_pre_tool_hooks[2-True], test/test_kas_cron_spec_hooks.py::test_a_cron_turn_gates_its_permission_requests_on_the_specs_pre_tool_hooks[0-False], test/test_kas_cron_spec_hooks.py::test_a_cron_turn_gates_its_permission_requests_on_the_specs_pre_tool_hooks[-1-False] (head 795ab9e)
With the source changes reverted, 42 of the 289 tests in the changed test files fail: 17 in
test_cron_refusal_status.py(the 15 tests of the new class other than its three..._stays_okcontrols, the one-shot test, and the Schedule page pattern test), 6 intest_kas_cron_spec_hooks.py, 18 of the 20 tests intest_llm_helpers_tool_gate.pyandtest_llm_helpers_spawn_brief_scan.pywhose callbacks take the fifth argument (the raising-callback and discarded-retry tests pass either way, since a callback's error is swallowed), and the shape pin. The changed test files passed 10 of 10 runs at-n 4 --dist loadgroup --max-worker-restart=2(289 tests), and the two cron files once in an order shuffled byrandom.Random(39973)(80 of 80).test_kas_cron_spec_hooks.py::test_a_hook_a_policy_refused_to_run_is_told_apart_from_its_own_verdict: throughstream_and_collectwithCronTurnHooks, a hook a policy refused and a hook that exited 2 both block the call; only the first reacheson_tool_gatewithhook_refusedtrue and is audited asspec_hook_refused.test_an_unreadable_spec_blocks_a_cron_turns_tools_as_in_a_dashboard_chatalso pinshook_refusedfalse for a spec that cannot be read.TestRefusedHookBesideAnApprovedCall::test_the_run_records_error,test_it_spends_no_failure_budgetandtest_it_does_not_reset_a_failure_streak: such a run recordserror; five of them leave the job enabled and unpaused; one after two blocked runs keepsconsecutive_failuresat 2.test_the_reason_names_those_calls_with_the_hook,test_another_unresolved_call_is_counted_but_not_namedandtest_the_reason_names_at_most_three: the reason's exact text, with other unresolved calls counted in M but not named, at most three names,and N more, within 500 characters.test_the_notification_names_the_lost_calls,test_it_reaches_the_sequential_pathandtest_the_schedule_page_shows_it_as_error_not_partial: the delivered body opens with the same reason on the single-agent andagent_sequencepaths, and the reason does not match the Partial pattern.test_any_other_refusal_beside_an_approved_call_stays_ok(hook exited 2,policy_deny, timed out),test_with_no_approved_call_the_no_approved_arm_is_unchangedandtest_a_security_block_keeps_the_partial_arm: the unchanged arms.test_a_security_block_names_the_hook_blocked_calls_after_it(with and without an approved call) andtest_both_parts_of_a_long_partial_reason_keep_their_heads: a partly security-blocked run names the security block, then the hook-blocked calls, inlast_errorand the banner; with five long calls of each kind both heads and bothand N moreclauses survive within 500 characters.test_a_hook_part_that_fits_is_not_cut_beside_a_long_gate_part: beside a long security-gate part, a hook part that fits is shown whole, rules included.TestPartialBlock::test_the_schedule_page_tells_a_partial_block_from_a_total_onealso checks that such a mixed reason still matches the Partial pattern.TestSchedulerContract::test_a_one_shot_whose_hook_policy_refused_to_run_is_kept: adelete_after_runone-shot in this arm is kept and paused, with this arm's reason.test_hooks_composition_contract.py::test_the_moved_names_keep_their_base_shapes: the re-pinned shape ofpermission_pre_tool_block.Manual verification
One cron turn driven through the real gate path:
CronTurnHooks.prepareover a spec with a PreToolUse hook onfs_read, thenstream_and_collectwith three permission requests (a file read, a shellls, a second file read), then_annotate_partial_blockand_apply_gate_verdicton a job whose failure streak is 2. In casepolicy-refuses-hookthecapabilities.script_hookscheck is forced to deny withprofile denies: capability disabled, sorun_script_hookreturns its real governance refusal. Casehook-and-security-blockadds a fourth request, a shellrm -rf /, which the security gate refuses. In casehook-exits-2the hook command runs and exits 2. Each run used a throwaway data home withKIROCREW_PROFILE=standalone. On the head, the policy case wrote twospec_hook_refusedaudit rows and the control case twospec_hookrows.Real-machine verification
Machine: own computer, Linux (Amazon Linux 2023, kernel 6.12), x86_64, 96 vCPU / 743 GiB
Before (main f8558e3):
env -i PATH=/usr/bin:/bin HOME=$H KIROCREW_HOME=$H/.kiro/crew KIROCREW_PROFILE=standalone PYTHONPATH=src python repro_39973.py policy-refuses-hook->approved=['r2'] rejected=['r1', 'r3'],last_status=None consecutive_failures=0,last_error=None,banner=False(the scheduler then writesok, and the streak of 2 was reset)After (head 795ab9e):
env -i PATH=/usr/bin:/bin HOME=$H KIROCREW_HOME=$H/.kiro/crew KIROCREW_PROFILE=standalone PYTHONPATH=src python repro_39973.py policy-refuses-hook->approved=['r2'] rejected=['r1', 'r3'],last_status='error' consecutive_failures=2,last_error='2 of 3 tool call(s) blocked because policy refused to run a hook: Read File (a1 spec hook (PreToolUse #2): Blocked by governance policy: profile denies: capability disabled), Read File (…same…)',banner=TrueBefore, same command with
hook-and-security-block:approved=['r2'] rejected=['r1', 'r3', 'r4'],last_status='error' consecutive_failures=2,last_error='1 of 4 tool call(s) blocked by the security gate: Running: rm -rf / (Blocked by security policy: rm -rf /.*)',banner=TrueAfter, same command with
hook-and-security-block:approved=['r2'] rejected=['r1', 'r3', 'r4'],last_status='error' consecutive_failures=2,last_error='1 of 4 tool call(s) blocked by the security gate: Running: rm -rf / (Blocked by security policy: rm -rf /.*). 2 tool call(s) blocked because policy refused to run a hook: Read File (a1 spec hook (PreToolUse #2): Blocked by governance policy: profile denies: capability disabled), Read File (…same…)',banner=TrueControl, same command with
hook-exits-2, on both:approved=['r2'] rejected=['r1', 'r3'],last_status=None consecutive_failures=0,last_error=None,banner=FalseCleanup: own computer; the throwaway data homes and the base worktree were deleted.
repro_39973.py
Related Issues
Closes #39973
Naming the hook-blocked calls in a partly security-blocked run follows the suggestion in a comment on #39973.
Builds on #37604, merged for #35101. #34927 decided that a hook a profile refuses keeps blocking the call. #28603 also touches how a cron run names a lost call; whichever lands second folds onto the other.
Sibling sweep
Swept: git grep -n -e 'permission_pre_tool_block' -e 'spec_floor' -e 'on_tool_gate' -e '_GateTally' -e '_apply_gate_verdict' -- src website/src
Sibling: src/kiro_crew/slack/gateway_runtime/cron_verdict.py -- changed here
Sibling: src/kiro_crew/slack/gateway_runtime/runtime_charge.py -- changed here
Sibling: src/kiro_crew/cron_service/turn_hooks.py -- changed here
Sibling: src/kiro_crew/slack/gateway_runtime/cron_turn.py -- not affected -- both cron agent paths pass
_GateTally.noteand call the shared_annotate_partial_blockand_apply_gate_verdictSibling: src/kiro_crew/channel_spec_hooks.py -- not affected -- a channel turn records no run status; its refusal is audited as
rejected_spec_hookand the turn goes onSibling: src/kiro_crew/messaging/tool_hooks.py -- not affected -- the same channel gate, with no run verdict to read the flag
Sibling: src/kiro_crew/subagent_manager/run.py -- not affected -- a sub-agent's refusal goes through the permission ladder and its own completion, not a cron run status
Sibling: src/kiro_crew/task_executor.py -- not affected -- a task step logs the refusal and has no cron run status
Sibling: website/src/utils/cronPartlyBlocked.ts -- not affected -- the new reason does not match its Partial pattern, so the run shows as Error
Pattern harvest
Not generalizable: one flag stopped at a seam that returns only a string, and the verdict arms it feeds are all listed in one spec table.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement
Contributed under the project's Apache-2.0 license.