Skip to content

fix: fail unfinished turns in channels, cli chat and refine - #19408

Merged
javenciu merged 1 commit into
kirodotdev:mainfrom
javenciu:fix/collectors-timed-out-turn-is-failure
Oct 13, 2026
Merged

javenciu merged 1 commit into
kirodotdev:mainfrom
javenciu:fix/collectors-timed-out-turn-is-failure

Conversation

@javenciu

@javenciu javenciu commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Agent Channels, kirocrew chat and the Task Runner's refine report a turn the backend never finished as unfinished, as stream_and_collect already does, instead of taking its partial text for a reply, a clean exit or a finished spec.

  • When the backend does not answer a prompt within the turn ceiling, the session handle ends the turn with a terminal EVENT_COMPLETE whose stop_reason is timeout (src/kiro_crew/acp/session_handle.py:5887), and a stale turn it could not recover ends stale_recover. unfinished_turn_error (src/kiro_crew/llm_helpers.py:2558, added by PR #19034) turns those two reasons into an AcpTimeoutError that carries the partial text, and stream_and_collect raises it (src/kiro_crew/llm_helpers.py:2814). Slack (PR #19008), the messaging channels (PR #19026) and the auto-improvement runner (PR #19005) already fail such a turn.
  • Three more loops end on EVENT_COMPLETE and never read stop_reason:
    • _stream_task (src/kiro_crew/channel.py:1726 at c4c51aa) posts the partial text as the agent's reply, or posts nothing for an empty turn.
    • _send_and_print (src/kiro_crew/cli_chat.py:1083 at c4c51aa) prints the partial answer and exits 0, where the same function prints "Response timed out." and exits 1 when the prompt timeout is raised.
    • _run_refine (src/kiro_crew/dashboard/handlers/taskrunner.py:1218 at c4c51aa) sets the refine to done with a spec that stops mid-sentence.

Why it matters

Severity: a cut-off result presented as a finished one, on three user-facing surfaces. It needs a turn that ran to the turn ceiling or a stale turn that did not recover, so it is rare, and nothing on these surfaces says so when it happens:

  • A channel shows half a sentence as the agent's answer, or shows nothing, so members cannot tell a cut-off turn from an agent that chose not to reply.
  • A script that runs kirocrew chat -m and checks the exit code takes a cut-off answer for a result.
  • A refine shows a spec that stops mid-sentence under status done, with no sign it was cut off.

Not a goal

  • Changing which stop reasons count as unfinished (still timeout and stale_recover), or the behaviour of stream_and_collect.
  • The webhook runner's loop, which has the same defect and is left for a follow-up (see the note under Change).
  • The other loops that end on EVENT_COMPLETE without reading stop_reason. They are listed for a follow-up and untouched here: src/kiro_crew/task_planner.py:416 (the decomposition parses the partial text), src/kiro_crew/dashboard/handlers/optimizer.py:345, src/kiro_crew/eval/judge.py:114, src/kiro_crew/eval/runner.py:618 and src/kiro_crew/messaging/auto_title.py:419.
  • Merging the existing copies of the stop-reason set: UNFINISHED_STOP_REASONS (src/kiro_crew/messaging/driver.py:165), _UNFINISHED_STOP_REASONS (src/kiro_crew/slack/handler.py:713) and the inline tuple in _raise_if_unfinished (src/kiro_crew/llm_helpers.py:1745) stay as they are. This change adds no copy.
  • A retry of a cut-off channel turn: the error is not transient (src/kiro_crew/llm_helpers.py:2576), and no retry policy changes.

What changed (motivation → approach → change)

In short: 8 of the 15 new cases fail on origin/main 65df088 and pass here; the other 7 are controls that pass on both. Red, 3 of 3 runs: 8 failed, 7 passed, 233 deselected. Green on this branch, 3 of 3 runs: 15 passed, 233 deselected; the three test files in full, 248 passed; the 234 neighbouring test files, 16586 passed. Details are under Tests.

  • Symptom: a turn that ends timeout or stale_recover is posted to a channel as the agent's reply, exits kirocrew chat with 0, and marks a refine done.
  • Root cause: the three loops break on EVENT_COMPLETE without asking the rule stream_and_collect asks.
  • Approach: ask that rule at the break and raise its AcpTimeoutError, so each caller's existing timeout or error path runs. The rule is the existing unfinished_turn_error, renamed from _unfinished_turn_error so three more modules can import it; the set of stop reasons is untouched. Rejected: a further copy of the set in each module (the copies listed above exist already), and a new wrapper around the rule (a new name with no reader beyond these call sites).
  • Change:
    • unfinished_turn_error (src/kiro_crew/llm_helpers.py:2599) is public, and its one caller moves with it (src/kiro_crew/llm_helpers.py:2855). No other module referenced the old name.
    • _stream_task raises at the break (src/kiro_crew/channel.py:1730). Its existing except Exception branch posts the error card, a system message (src/kiro_crew/channel.py:1752), and returns _STREAM_OK.
    • _send_and_print raises with no text (src/kiro_crew/cli_chat.py:1177), because the text was printed as it streamed. Its existing except AcpTimeoutError (src/kiro_crew/cli_chat.py:1182) prints the notice and exits 1.
    • _run_refine raises (src/kiro_crew/dashboard/handlers/taskrunner.py:1290). Its existing except Exception sets status error and the error text (src/kiro_crew/dashboard/handlers/taskrunner.py:1301), and the partial spec is still broadcast with it.
    • Spec: docs/system-specs/modules/persistent-agent-channels.md (new "Unfinished Turns" subsection), docs/system-specs/modules/cli.md (exit codes) and docs/system-specs/modules/taskrunner.md (Dynamic Refine).
  • Left for a follow-up a maintainer can run: the webhook runner's loop has the same defect. _run_hook_inner (src/kiro_crew/dashboard/handlers/hooks.py:1267) breaks on EVENT_COMPLETE (src/kiro_crew/dashboard/handlers/hooks.py:1527), books record_success (src/kiro_crew/dashboard/handlers/hooks.py:1530) and returns the partial text, so the run history records completed. It is not changed here. The same function decides each tool request of the webhook turn through the hook gate (hooks_gate.on_tool_call, src/kiro_crew/dashboard/handlers/hooks.py:1389, which is HookManager.on_tool_call, src/kiro_crew/hooks.py:1051), and src/kiro_crew/hooks.py and src/kiro_crew/hook_runtime/* are under the Sensitive Change Review, which refuses a fork PR (.github/scripts/sensitive_change_review.py:52-53). The fix has the same shape as the three above: ask unfinished_turn_error at the break, skip record_success for such a turn, and give _run_hook_agent an AcpTimeoutError branch that records the failure.
  • Stated behaviour changes:
    • A cut-off channel turn posts the error card in place of the partial text, and an empty cut-off turn posts the card in place of nothing.
    • kirocrew chat exits 1 for a cut-off turn. It exited 0. In the interactive REPL the same exit ends the session after the notice (src/kiro_crew/cli_chat.py:1237 calls _send_and_print with no handler for SystemExit), exactly as a raised prompt timeout already does there.
    • A cut-off refine ends error. It ended done.

Backwards compatibility

Compatible: no config key, wire shape or stored format changes. The private name _unfinished_turn_error becomes unfinished_turn_error; git grep finds no other reference to the old name in src, test or docs. A turn that ends end_turn, cancelled or any other reason behaves as before, and the control cases below pin that on all three surfaces. The behaviour changes listed above are the only ones.
Removes nothing else: no public function, field or file changes.

Tests

Changed in three existing test modules, 15 cases in 6 tests, with no sleeps, no ports and no real backend. Each drives the real function with a fake event stream; a cut-off turn is a terminal event whose stop_reason is timeout or stale_recover.

  • test/test_channel_prompt_busy.py (_stream_task): test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn (4 cases: a partial or an empty turn, each ending timeout or stale_recover; one error card, as a system message) and test_stream_task_posts_the_text_of_a_finished_or_cancelled_turn (control: end_turn and cancelled).
  • test/test_cli_chat.py (_send_and_print): test_send_and_print_reports_an_unfinished_turn_as_timed_out (exit 1, "Response timed out.", the streamed text printed once; its third case is the raised prompt timeout, which passes on main) and test_send_and_print_prints_a_finished_or_cancelled_turn (control).
  • test/test_handlers_taskrunner_coverage.py (_run_refine): test_a_refine_the_backend_never_finished_is_an_error (status error, an error naming the stop reason, the partial spec still sent) and test_a_refine_that_ends_its_turn_is_done (control).

Before the fix, on origin/main 65df088 with the three test files from this branch copied into a detached checkout, pytest test/test_channel_prompt_busy.py test/test_cli_chat.py test/test_handlers_taskrunner_coverage.py -k "unfinished_turn or finished_or_cancelled or never_finished or ends_its_turn" -p no:cacheprovider -q -n 0 --no-cov, RC=1 in 3 of 3 runs, the same 8 failures each time. The 7 passes are the controls:

FAILED test/test_channel_prompt_busy.py::test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn[partial-timeout] - AssertionError: assert 'error occurred' in 'half a pl'
FAILED test/test_channel_prompt_busy.py::test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn[partial-stale_recover] - AssertionError: assert 'error occurred' in 'half a pl'
FAILED test/test_channel_prompt_busy.py::test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn[empty-timeout] - AssertionError: Expected mock to have been awaited once. Awaited 0 times.
FAILED test/test_channel_prompt_busy.py::test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn[empty-stale_recover] - AssertionError: Expected mock to have been awaited once. Awaited 0 times.
FAILED test/test_cli_chat.py::test_send_and_print_reports_an_unfinished_turn_as_timed_out[timeout] - Failed: DID NOT RAISE <class 'SystemExit'>
FAILED test/test_cli_chat.py::test_send_and_print_reports_an_unfinished_turn_as_timed_out[stale_recover] - Failed: DID NOT RAISE <class 'SystemExit'>
FAILED test/test_handlers_taskrunner_coverage.py::TestRunRefine::test_a_refine_the_backend_never_finished_is_an_error[timeout] - AssertionError: assert 'done' == 'error'
FAILED test/test_handlers_taskrunner_coverage.py::TestRunRefine::test_a_refine_the_backend_never_finished_is_an_error[stale_recover] - AssertionError: assert 'done' == 'error'
============ 8 failed, 7 passed, 233 deselected, 1 warning in 3.80s ============

After the fix, on this branch, the same command, RC=0 in 3 of 3 runs:

================ 15 passed, 233 deselected, 1 warning in 2.97s =================

The three test files in full, 3 of 3 runs RC=0: 248 passed.

One mutation per rule, each run against the 15 cases on this branch, fails only its own cases:

Mutation Cases that fail
_stream_task: the raise removed the 4 channel cases
_send_and_print: the raise removed the 2 CLI cases
_run_refine: the raise removed the 2 refine cases
the rule drops stale_recover the 4 stale_recover cases, one per surface and two on the channel
the rule adds cancelled the 3 cancelled controls, one per surface

Neighbours: the 234 test files that mention cli_chat (all 37 of them), kiro_crew.channel, dashboard.handlers.taskrunner, dashboard.handlers.hooks or llm_helpers, read the three spec files this change edits (test_cli.py, test_cli_cron_add_kinds.py, test_cli_version_fastpath.py read cli.md), or are named for them (test_channel*, test_cli_chat*, test_taskrunner*, test_task_runner*, test_llm_helpers*, test_handlers_taskrunner*, test_webhook*, test_api_hook*), together with test/test_wait_discipline_ratchet.py and test/test_flake_pattern_ratchet.py, run serially with -n 0 --no-cov in three batches of 77, 77 and 80 files: 16586 passed, 17 skipped, 0 failed. test/test_cli_doctor_refactor_report.py is in the second batch; its frozen report gains a lib path line when LLAMA_CPP_LIB_PATH is exported in the shell, so two of its tests fail there on origin/main as well, and the batches ran with that variable unset. test/test_wait_discipline_ratchet.py also passes (39 passed) on this branch, which is one commit on current origin/main, as it does on origin/main alone. Also: mypy on the 4 changed source files, flake8, isort and black on the changed files, scripts/check_black_formatting.py, COMMENT_HISTORY_BASE_REF=origin/main scripts/check_comment_history.py, scripts/check_subprocess_encoding.py, scripts/check_sync_io_in_async.py, scripts/check_agent_sdk_boundary.py and ./scripts/docs-lint.sh: all exit 0.

Manual verification

N/A: unit coverage drives the real _stream_task, _send_and_print and _run_refine, with a fake event stream as the only stand-in for the backend. No gateway, model or network was used.

Related Issues

None open. Searched open and closed PRs and issues for unfinished turn, stale_recover, timed-out turn, stop_reason, turn ceiling, partial text, never finished, EVENT_COMPLETE, _stream_task, _send_and_print, _run_refine, _run_hook_inner, refine timeout and webhook run history; no open PR or issue covers these loops, and no comment claims them.

  • Siblings, all merged: PR #19034 (stream_and_collect, where unfinished_turn_error comes from), PR #19008 (Slack), PR #19026 (messaging channels) and PR #19005 (auto-improvement).
  • PR #19259 (merged) named the host refusal on four more deny surfaces and edited src/kiro_crew/cli_chat.py. The rebase onto it conflicted on one line, the neighbouring from kiro_crew.llm_helpers import ... in cli_chat.py, which now imports both names.
  • PR #18953 (open) edits _stream_task for the approval window; a trial merge of this head with it is clean. PR #19117 (merged) edited dashboard/handlers/taskrunner.py, test/test_handlers_taskrunner_coverage.py and docs/system-specs/modules/taskrunner.md for Continue in Chat; the rebase onto it applied without a conflict.
  • Issue #19041 (open) is the Slack mid-turn import, in the Slack handler, none of these loops.

Pattern harvest

Rule candidate: review-prompt
Pattern: a loop that ends on a terminal event and takes the text collected so far as the result, without reading the terminal's stop reason, while a sibling collector already treats that reason as a failure.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 12, 2026 11:36
@javenciu
javenciu requested a review from dodgeblaster October 12, 2026 11:36
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 12, 2026
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 770e288b8ca893817e522da9eab764655c5dde06 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 770e288b8ca893817e522da9eab764655c5dde06 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the channel card is enough: a cut-off channel turn now drops its partial text and posts only the generic "An error occurred" card, so members can't see why.

[DESIGN-REVIEWED] 770e288

Description read

[DESCRIPTION-READ] 17ecc93821f2c86331f3807abf0c532f5acbd84d9956bdce0eab3b0f2ac7b6f5

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 770e288b8ca893817e522da9eab764655c5dde06 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

A cut-off turn now ends an interactive kirocrew chat REPL session through SystemExit. Confirm that is acceptable, because the description notes it but no test covers it.

What this change ships

Inventory (6 items) — 6 justified

Intent: Channels, kirocrew chat and Task Runner refine should report a turn the backend never finished as failed, not as a finished result. This is a FIX.

  1. A cut-off channel turn posts the error card instead of the partial text or nothing. — justified
  2. kirocrew chat -m exits 1 on a cut-off turn, where it used to exit 0. — justified
  3. The interactive REPL ends after the notice for a cut-off turn, the same way a raised prompt timeout already does. — justified
  4. A cut-off refine ends with status error instead of done, and the partial spec is still sent. — justified
  5. unfinished_turn_error is now public. It has 3 new importers plus its existing caller, and no second copy of the stop-reason set is added. — justified
  6. Three spec sections describe the new terminal behaviour (required by AGENTS.md: a spec is updated in the same commit). — justified

[FIRST-PRINCIPLES-REVIEWED] 770e288

Description read

[DESCRIPTION-READ] 27bd7a9c8daa66c8b3f5220d9f70e5ec5da93e533de94e5230731171d0569e71

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 770e288b8ca893817e522da9eab764655c5dde06 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 770e288

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 770e288b8ca893817e522da9eab764655c5dde06 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 770e288

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running labels Oct 12, 2026
Three collectors stop on EVENT_COMPLETE and never read stop_reason, so
a turn the session handle ended because its backend never finished it
("timeout" at the turn ceiling, "stale_recover" for a stale turn that
did not recover) counted as a finished turn:

- channel._stream_task posted the partial text as the agent's reply,
  or nothing for an empty turn;
- cli_chat._send_and_print printed the partial answer and exited 0,
  where the same function exits 1 for a raised prompt timeout;
- the task runner's refine marked the spec done although it stops
  mid-sentence.

All three now ask llm_helpers.unfinished_turn_error, the rule
stream_and_collect already uses, made public so the set of unfinished
stop reasons is decided in one place, and raise its AcpTimeoutError at
the break. The channel turn takes its error branch and posts the error
card. The CLI takes its timeout branch: the text was printed as it
streamed, so the error carries none, and the branch prints the notice
and exits 1. The refine takes its error branch: status "error", with
the partial spec still broadcast. "end_turn", "cancelled" and every
other stop reason behave as before.
@javenciu
javenciu force-pushed the fix/collectors-timed-out-turn-is-failure branch from 7929f29 to 770e288 Compare October 12, 2026 20:29
@javenciu

Copy link
Copy Markdown
Contributor Author

Rebased onto main (65df088). One conflict, the from kiro_crew.llm_helpers import ... line in src/kiro_crew/cli_chat.py: #19259 added _steer_host_deny there, so the line imports both _steer_host_deny and unfinished_turn_error; the rest of the patch is unchanged.
The line refs under Change moved with the rebase and are updated. In the first section, src/kiro_crew/llm_helpers.py:2558, :2814, :1745 and :2576 are now :2599, :2855, :1786 and :2617, src/kiro_crew/dashboard/handlers/optimizer.py:345 is :350, src/kiro_crew/messaging/auto_title.py:419 is :429, and src/kiro_crew/messaging/driver.py:165 is :166.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 12, 2026
@javenciu
javenciu enabled auto-merge (squash) October 12, 2026 23:19

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR makes Agent Channels (_stream_task), kirocrew chat (_send_and_print) and the Task Runner refine (_run_refine) treat a turn that ended timeout or stale_recover as unfinished, the same way stream_and_collect already does, instead of posting the partial text as a reply, exiting 0, or marking a cut-off spec done. The root cause is fixed at each loop's EVENT_COMPLETE break by asking the one existing rule, unfinished_turn_error (renamed from private to public with its single caller), so no new copy of the stop-reason set is added. I followed each raise into its caller's existing handler. The channel's except Exception posts its system error card and returns _STREAM_OK (it is not a prompt-busy or session-lost error, so no session is replaced), and the partial text with any @mentions is no longer routed to other members. The CLI's except AcpTimeoutError prints the notice once, because the error carries no text, and exits 1. The refine's handler sets error and still broadcasts the partial spec. Turns ending end_turn, cancelled or any other reason are unchanged, and control cases pin that on all three surfaces. The webhook loop and the other collectors are listed as follow-ups, which is the right scope. The new cases fail on base and pass here, both AI lanes passed, checks are green, and no change request is open. — crew-pr-reviewer (deep re-check)

@javenciu
javenciu merged commit 9a260da into kirodotdev:main Oct 13, 2026
105 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants