Skip to content

fix(cron): record error when no tool call of a cron run was approved - #35337

Open
rohnkum wants to merge 1 commit into
kirodotdev:mainfrom
rohnkum:fix/cron-unapproved-run-records-error
Open

rohnkum wants to merge 1 commit into
kirodotdev:mainfrom
rohnkum:fix/cron-unapproved-run-records-error

Conversation

@rohnkum

@rohnkum rohnkum commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

A cron agent run recorded last_status: ok when no tool call was approved and none was security-blocked.
This happened when every call was refused by a governance profile, by the agent spec's PreToolUse hook, or by a rejected or timed-out approval.
_GateTally.note kept only a count of those calls.
_apply_gate_verdict then called record_success(), because neither all_blocked nor partially_blocked was true.
The Schedule page showed a run that did nothing as a success.

Goal: a cron run with no approved call, no security block and at least one other refusal records error and names the refused calls, without moving either failure counter.

Why it matters

A daily job whose agent spec carries PreToolUse hooks, run under a profile that turns hooks off, had every call refused.
Each run still showed ok, so nothing told the user the job had stopped working.

Not a goal

  • Naming unapproved calls in a run that also had an approved call. That run stays ok.
  • Changing how a security block, a partial block or a tool-free run is recorded.
  • Changing the on_tool_gate callback shape or the failure counter rules.

What changed

  • src/kiro_crew/slack/gateway_runtime/cron_verdict.py: _GateTally keeps each unresolved call's title and first reason line in unresolved_calls. A new none_approved property marks the new case. refusal_summary() names up to three such calls with their reason, or "not approved", under the head none of N tool call(s) ran, all refused or not approved: . _annotate_partial_block prefixes the delivered result for this case too. The class and function docstrings describe the new case.
  • src/kiro_crew/slack/gateway_runtime/runtime_charge.py: _apply_gate_verdict records none_approved runs as error with the summary as last_error, and moves neither counter. Its docstring lists four outcomes.
  • docs/system-specs/modules/learn-cron-dashboard.md: the gate-verdict table gains the no-approved row, the arm count reads four, a paragraph describes the new arm, and the Partial paragraph lists the none of N ... reason among those that keep Error.
  • test/test_cron_refusal_status.py: the harness passes an optional reason to on_tool_gate. test_an_unapproved_run_is_not_a_failure now asserts last_status == "error" and keeps its counter and pause asserts. A new TestUnapprovedRun class and one scheduler test cover the new arm and the unchanged ones.

Tests

Proof: PROVEN test/test_cron_refusal_status.py::test_an_unapproved_run_is_not_a_failure, test/test_cron_refusal_status.py::TestUnapprovedRun::test_a_run_with_no_approved_call_records_error[unapproved], test/test_cron_refusal_status.py::TestUnapprovedRun::test_a_run_with_no_approved_call_records_error[policy-deny], test/test_cron_refusal_status.py::TestUnapprovedRun::test_a_run_with_no_approved_call_records_error[spec-hook], test/test_cron_refusal_status.py::TestUnapprovedRun::test_a_run_with_no_approved_call_records_error[mixed] (head 1fa7eee)

  • test_an_unapproved_run_is_not_a_failure: a timed-out approval run records error and never auto-pauses after five runs.
  • TestUnapprovedRun::test_a_run_with_no_approved_call_records_error: an unapproved, a policy_deny, a spec_hook and a mixed run each record error with no failure counted.
  • TestUnapprovedRun::test_it_spends_no_failure_budget: five such runs leave the job unpaused and enabled.
  • TestUnapprovedRun::test_it_does_not_reset_a_failure_streak: such a run after two blocked runs keeps consecutive_failures at 2.
  • TestUnapprovedRun::test_the_reason_names_each_call_with_its_refusal_text: last_error names each call with the first line of its reason, or "not approved".
  • TestUnapprovedRun::test_the_reason_names_at_most_three: six refused calls are counted, three are named, and the reason stays within 500 characters.
  • TestUnapprovedRun::test_the_schedule_page_shows_it_as_error_not_partial: the reason does not match the Partial pattern in backendPhrases.json.
  • TestUnapprovedRun::test_the_notification_names_the_refused_calls: the delivered body opens with the same reason as last_error and keeps the model's prose.
  • TestUnapprovedRun::test_it_reaches_the_sequential_path: an agent_sequence run gets the same status, reason and banner.
  • TestUnapprovedRun::test_one_approved_call_keeps_the_run_ok: an approved call beside an unapproved one stays ok with no banner.
  • TestUnapprovedRun::test_a_tool_free_run_stays_ok: a run with no tool call stays ok.
  • TestSchedulerContract::test_an_unapproved_one_shot_is_kept_and_auto_paused: a delete_after_run one-shot with only a policy denial is kept, auto-paused and disabled, with no failure counted.

Sibling sweep

Swept: every reader of _GateTally, refusal_summary(), _annotate_partial_block and _apply_gate_verdict, and every place that turns a cron run's last_status into a pause or a badge.
Sibling: src/kiro_crew/slack/gateway_runtime/cron_verdict.py -- changed here
Sibling: src/kiro_crew/slack/gateway_runtime/runtime_charge.py -- changed here
Sibling: src/kiro_crew/slack/gateway_runtime/cron_turn.py -- not affected -- both agent paths call the shared _annotate_partial_block and _apply_gate_verdict, so they pick up the new arm unchanged
Sibling: src/kiro_crew/slack/gateway_runtime/cron_failure.py -- not affected -- it alerts after a counted failure and the new arm counts none
Sibling: src/kiro_crew/cron_service/execution.py -- not affected -- auto_pause_failed_one_shot already keeps and pauses any delete_after_run run whose status is error
Sibling: website/src/utils/cronPartlyBlocked.ts -- not affected -- the new reason does not match its Partial pattern, so the run shows as Error
Sibling: src/kiro_crew/llm_helpers_parts/turn_stream.py -- not affected -- on_tool_gate keeps its four-argument shape and already passes the reason

Pattern harvest

Not generalizable: the gap is one missing branch in a single verdict function whose arms are all listed in one table.

Closes #35101

@rohnkum
rohnkum requested review from a team and KiroCrewCaptain as code owners November 11, 2026 14:53
@rohnkum
rohnkum requested a review from Zedmor November 11, 2026 14:53
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 11, 2026
A cron run in which no tool call was approved and none was blocked by
the security gate recorded last_status ok, even when every call was
refused by a governance profile, by the agent spec's PreToolUse hook,
or by a rejected or timed-out approval. _GateTally kept only a count of
those calls, and _apply_gate_verdict called record_success() for them.

_GateTally now keeps each such call's title and first reason line, and
a new none_approved property marks a run with no approved call, no
security block and at least one other refusal. _apply_gate_verdict
records that run as error without moving either counter, so a
delete_after_run one-shot is kept and auto-paused. refusal_summary()
names up to three of the calls with their refusal text, or "not
approved", and _annotate_partial_block puts the same reason in front
of the delivered result. The reason does not use the partial-block
wording, so the Schedule page shows the run as Error.
@rohnkum
rohnkum force-pushed the fix/cron-unapproved-run-records-error branch from c71f327 to 1fa7eee Compare November 11, 2026 17:45
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 1fa7eee17b019e59cc682b44f525fd469102459c touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 1fa7eee17b019e59cc682b44f525fd469102459c via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix adds the missing arm to the shared verdict, leaves the counters alone, and the doc table changes in the same commit.

Notes

  • The edit to test_an_unapproved_run_is_not_a_failure turns its != "error" assert into == "error". That reads as a gap being closed, not a decision being reversed: the test's docstring argues only about the failure counter, and the counter and pause asserts stay.
  • A delete_after_run one-shot whose approval timed out was consumed before; now it is kept, auto-paused and disabled. That is deliberate and tested, but it is a change users will see, and a revert undoes it cleanly.

[DESIGN-REVIEWED] 1fa7eee

Description read

[DESCRIPTION-READ] a981ae8e670f35206193ae361680a8a60a710332490b270e7f265fd0dcf21dcc

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 1fa7eee17b019e59cc682b44f525fd469102459c via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix is at mechanism level, issue #35101 is its provenance, and the reversed != "error" assert is declared in the description. That pin's own docstring gave a reason only for the counter, and the counter asserts are kept.

Inventory (6 items) — 6 justified

Intent: make a cron run in which nothing ran (every call refused outside the security gate) stop showing as a success. FIX.

  1. A run with no approved call, no security block and at least one other refusal now records error instead of ok — justified
  2. That run's last_error names up to three refused calls with the first line of each reason, or "not approved" — justified
  3. The delivered notification and result for that run now open with a ⛔ none of N… banner — justified
  4. That run neither counts as a failure nor resets a failure streak — justified
  5. A delete_after_run one-shot whose calls were all refused is now kept and auto-paused instead of consumed — justified
  6. The spec table gains the no-approved row and the arm count goes from three to four — justified

[FIRST-PRINCIPLES-REVIEWED] 1fa7eee

Description read

[DESCRIPTION-READ] 9da034103f16f626d85bfe1dffe38e2f2a61d700be2edba0c408c7fec87cb7d6

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 1fa7eee17b019e59cc682b44f525fd469102459c via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 1fa7eee

@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #37604; thanks for the start.

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

The PR adds a none_approved case to the shared cron gate verdict. A run where no call was approved, none was security-blocked, and at least one was refused another way now records error. Its last_error names up to three refused calls and moves neither failure counter, the same way the partial-block case works. I checked it against the code:

  • Executed calls with no gate decision are counted as approved in turn_stream.py, so a run where auto-approved tools ran still does not fall into the new case.
  • Tool-free runs and runs with at least one approved call still record ok.
  • _apply_gate_verdict returns False, so cron_failure's _gate_counted logic is unchanged.
  • Both cron agent paths, single-agent and agent_sequence, call the shared _annotate_partial_block and _apply_gate_verdict.
  • The new none of N ... reason does not match the Schedule page's Partial pattern in backendPhrases.json, so the run shows as Error.
  • auto_pause_failed_one_shot keeps and pauses a delete_after_run one-shot that ends in error. That is a deliberate change, documented in the spec and tested.
  • The spec's statement that only a security block counts toward the failure counter (ar

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 1fa7eee17b · 250 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cron: a run whose every tool call was policy-denied, hook-refused or unapproved records ok

3 participants