Repository navigation
fix: fail unfinished turns in channels, cli chat and refine - #19408
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the channel card is enough: a cut-off channel turn now drops its partial text and posts only the generic "An error occurred" card, so members can't see why. [DESIGN-REVIEWED] 770e288 Description read[DESCRIPTION-READ] 17ecc93821f2c86331f3807abf0c532f5acbd84d9956bdce0eab3b0f2ac7b6f5 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS A cut-off turn now ends an interactive What this change shipsInventory (6 items) — 6 justifiedIntent: Channels,
[FIRST-PRINCIPLES-REVIEWED] 770e288 Description read[DESCRIPTION-READ] 27bd7a9c8daa66c8b3f5220d9f70e5ec5da93e533de94e5230731171d0569e71 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Three collectors stop on EVENT_COMPLETE and never read stop_reason, so
a turn the session handle ended because its backend never finished it
("timeout" at the turn ceiling, "stale_recover" for a stale turn that
did not recover) counted as a finished turn:
- channel._stream_task posted the partial text as the agent's reply,
or nothing for an empty turn;
- cli_chat._send_and_print printed the partial answer and exited 0,
where the same function exits 1 for a raised prompt timeout;
- the task runner's refine marked the spec done although it stops
mid-sentence.
All three now ask llm_helpers.unfinished_turn_error, the rule
stream_and_collect already uses, made public so the set of unfinished
stop reasons is decided in one place, and raise its AcpTimeoutError at
the break. The channel turn takes its error branch and posts the error
card. The CLI takes its timeout branch: the text was printed as it
streamed, so the error carries none, and the branch prints the notice
and exits 1. The refine takes its error branch: status "error", with
the partial spec still broadcast. "end_turn", "cancelled" and every
other stop reason behave as before.
7929f29 to
770e288
Compare
|
Rebased onto main (65df088). One conflict, the |
iamwhatever
left a comment
There was a problem hiding this comment.
This PR makes Agent Channels (_stream_task), kirocrew chat (_send_and_print) and the Task Runner refine (_run_refine) treat a turn that ended timeout or stale_recover as unfinished, the same way stream_and_collect already does, instead of posting the partial text as a reply, exiting 0, or marking a cut-off spec done. The root cause is fixed at each loop's EVENT_COMPLETE break by asking the one existing rule, unfinished_turn_error (renamed from private to public with its single caller), so no new copy of the stop-reason set is added. I followed each raise into its caller's existing handler. The channel's except Exception posts its system error card and returns _STREAM_OK (it is not a prompt-busy or session-lost error, so no session is replaced), and the partial text with any @mentions is no longer routed to other members. The CLI's except AcpTimeoutError prints the notice once, because the error carries no text, and exits 1. The refine's handler sets error and still broadcasts the partial spec. Turns ending end_turn, cancelled or any other reason are unchanged, and control cases pin that on all three surfaces. The webhook loop and the other collectors are listed as follow-ups, which is the right scope. The new cases fail on base and pass here, both AI lanes passed, checks are green, and no change request is open. — crew-pr-reviewer (deep re-check)
Problem / Motivation
Goal: Agent Channels,
kirocrew chatand the Task Runner's refine report a turn the backend never finished as unfinished, asstream_and_collectalready does, instead of taking its partial text for a reply, a clean exit or a finished spec.EVENT_COMPLETEwhosestop_reasonistimeout(src/kiro_crew/acp/session_handle.py:5887), and a stale turn it could not recover endsstale_recover.unfinished_turn_error(src/kiro_crew/llm_helpers.py:2558, added by PR #19034) turns those two reasons into anAcpTimeoutErrorthat carries the partial text, andstream_and_collectraises it (src/kiro_crew/llm_helpers.py:2814). Slack (PR #19008), the messaging channels (PR #19026) and the auto-improvement runner (PR #19005) already fail such a turn.EVENT_COMPLETEand never readstop_reason:_stream_task(src/kiro_crew/channel.py:1726at c4c51aa) posts the partial text as the agent's reply, or posts nothing for an empty turn._send_and_print(src/kiro_crew/cli_chat.py:1083at c4c51aa) prints the partial answer and exits 0, where the same function prints "Response timed out." and exits 1 when the prompt timeout is raised._run_refine(src/kiro_crew/dashboard/handlers/taskrunner.py:1218at c4c51aa) sets the refine todonewith a spec that stops mid-sentence.Why it matters
Severity: a cut-off result presented as a finished one, on three user-facing surfaces. It needs a turn that ran to the turn ceiling or a stale turn that did not recover, so it is rare, and nothing on these surfaces says so when it happens:
kirocrew chat -mand checks the exit code takes a cut-off answer for a result.done, with no sign it was cut off.Not a goal
timeoutandstale_recover), or the behaviour ofstream_and_collect.EVENT_COMPLETEwithout readingstop_reason. They are listed for a follow-up and untouched here:src/kiro_crew/task_planner.py:416(the decomposition parses the partial text),src/kiro_crew/dashboard/handlers/optimizer.py:345,src/kiro_crew/eval/judge.py:114,src/kiro_crew/eval/runner.py:618andsrc/kiro_crew/messaging/auto_title.py:419.UNFINISHED_STOP_REASONS(src/kiro_crew/messaging/driver.py:165),_UNFINISHED_STOP_REASONS(src/kiro_crew/slack/handler.py:713) and the inline tuple in_raise_if_unfinished(src/kiro_crew/llm_helpers.py:1745) stay as they are. This change adds no copy.src/kiro_crew/llm_helpers.py:2576), and no retry policy changes.What changed (motivation → approach → change)
In short: 8 of the 15 new cases fail on
origin/main65df088 and pass here; the other 7 are controls that pass on both. Red, 3 of 3 runs:8 failed, 7 passed, 233 deselected. Green on this branch, 3 of 3 runs:15 passed, 233 deselected; the three test files in full,248 passed; the 234 neighbouring test files,16586 passed. Details are under Tests.timeoutorstale_recoveris posted to a channel as the agent's reply, exitskirocrew chatwith 0, and marks a refinedone.EVENT_COMPLETEwithout asking the rulestream_and_collectasks.AcpTimeoutError, so each caller's existing timeout or error path runs. The rule is the existingunfinished_turn_error, renamed from_unfinished_turn_errorso three more modules can import it; the set of stop reasons is untouched. Rejected: a further copy of the set in each module (the copies listed above exist already), and a new wrapper around the rule (a new name with no reader beyond these call sites).unfinished_turn_error(src/kiro_crew/llm_helpers.py:2599) is public, and its one caller moves with it (src/kiro_crew/llm_helpers.py:2855). No other module referenced the old name._stream_taskraises at the break (src/kiro_crew/channel.py:1730). Its existingexcept Exceptionbranch posts the error card, asystemmessage (src/kiro_crew/channel.py:1752), and returns_STREAM_OK._send_and_printraises with no text (src/kiro_crew/cli_chat.py:1177), because the text was printed as it streamed. Its existingexcept AcpTimeoutError(src/kiro_crew/cli_chat.py:1182) prints the notice and exits 1._run_refineraises (src/kiro_crew/dashboard/handlers/taskrunner.py:1290). Its existingexcept Exceptionsets statuserrorand the error text (src/kiro_crew/dashboard/handlers/taskrunner.py:1301), and the partial spec is still broadcast with it.docs/system-specs/modules/persistent-agent-channels.md(new "Unfinished Turns" subsection),docs/system-specs/modules/cli.md(exit codes) anddocs/system-specs/modules/taskrunner.md(Dynamic Refine)._run_hook_inner(src/kiro_crew/dashboard/handlers/hooks.py:1267) breaks onEVENT_COMPLETE(src/kiro_crew/dashboard/handlers/hooks.py:1527), booksrecord_success(src/kiro_crew/dashboard/handlers/hooks.py:1530) and returns the partial text, so the run history recordscompleted. It is not changed here. The same function decides each tool request of the webhook turn through the hook gate (hooks_gate.on_tool_call,src/kiro_crew/dashboard/handlers/hooks.py:1389, which isHookManager.on_tool_call,src/kiro_crew/hooks.py:1051), andsrc/kiro_crew/hooks.pyandsrc/kiro_crew/hook_runtime/*are under the Sensitive Change Review, which refuses a fork PR (.github/scripts/sensitive_change_review.py:52-53). The fix has the same shape as the three above: askunfinished_turn_errorat the break, skiprecord_successfor such a turn, and give_run_hook_agentanAcpTimeoutErrorbranch that records the failure.kirocrew chatexits 1 for a cut-off turn. It exited 0. In the interactive REPL the same exit ends the session after the notice (src/kiro_crew/cli_chat.py:1237calls_send_and_printwith no handler forSystemExit), exactly as a raised prompt timeout already does there.error. It endeddone.Backwards compatibility
Compatible: no config key, wire shape or stored format changes. The private name
_unfinished_turn_errorbecomesunfinished_turn_error;git grepfinds no other reference to the old name insrc,testordocs. A turn that endsend_turn,cancelledor any other reason behaves as before, and the control cases below pin that on all three surfaces. The behaviour changes listed above are the only ones.Removes nothing else: no public function, field or file changes.
Tests
Changed in three existing test modules, 15 cases in 6 tests, with no sleeps, no ports and no real backend. Each drives the real function with a fake event stream; a cut-off turn is a terminal event whose
stop_reasonistimeoutorstale_recover.test/test_channel_prompt_busy.py(_stream_task):test_stream_task_posts_a_card_not_the_text_of_an_unfinished_turn(4 cases: a partial or an empty turn, each endingtimeoutorstale_recover; one error card, as asystemmessage) andtest_stream_task_posts_the_text_of_a_finished_or_cancelled_turn(control:end_turnandcancelled).test/test_cli_chat.py(_send_and_print):test_send_and_print_reports_an_unfinished_turn_as_timed_out(exit 1, "Response timed out.", the streamed text printed once; its third case is the raised prompt timeout, which passes on main) andtest_send_and_print_prints_a_finished_or_cancelled_turn(control).test/test_handlers_taskrunner_coverage.py(_run_refine):test_a_refine_the_backend_never_finished_is_an_error(statuserror, an error naming the stop reason, the partial spec still sent) andtest_a_refine_that_ends_its_turn_is_done(control).Before the fix, on
origin/main65df088 with the three test files from this branch copied into a detached checkout,pytest test/test_channel_prompt_busy.py test/test_cli_chat.py test/test_handlers_taskrunner_coverage.py -k "unfinished_turn or finished_or_cancelled or never_finished or ends_its_turn" -p no:cacheprovider -q -n 0 --no-cov, RC=1 in 3 of 3 runs, the same 8 failures each time. The 7 passes are the controls:After the fix, on this branch, the same command, RC=0 in 3 of 3 runs:
The three test files in full, 3 of 3 runs RC=0:
248 passed.One mutation per rule, each run against the 15 cases on this branch, fails only its own cases:
_stream_task: the raise removed_send_and_print: the raise removed_run_refine: the raise removedstale_recoverstale_recovercases, one per surface and two on the channelcancelledcancelledcontrols, one per surfaceNeighbours: the 234 test files that mention
cli_chat(all 37 of them),kiro_crew.channel,dashboard.handlers.taskrunner,dashboard.handlers.hooksorllm_helpers, read the three spec files this change edits (test_cli.py,test_cli_cron_add_kinds.py,test_cli_version_fastpath.pyreadcli.md), or are named for them (test_channel*,test_cli_chat*,test_taskrunner*,test_task_runner*,test_llm_helpers*,test_handlers_taskrunner*,test_webhook*,test_api_hook*), together withtest/test_wait_discipline_ratchet.pyandtest/test_flake_pattern_ratchet.py, run serially with-n 0 --no-covin three batches of 77, 77 and 80 files: 16586 passed, 17 skipped, 0 failed.test/test_cli_doctor_refactor_report.pyis in the second batch; its frozen report gains alib pathline whenLLAMA_CPP_LIB_PATHis exported in the shell, so two of its tests fail there onorigin/mainas well, and the batches ran with that variable unset.test/test_wait_discipline_ratchet.pyalso passes (39 passed) on this branch, which is one commit on currentorigin/main, as it does onorigin/mainalone. Also: mypy on the 4 changed source files, flake8, isort and black on the changed files,scripts/check_black_formatting.py,COMMENT_HISTORY_BASE_REF=origin/main scripts/check_comment_history.py,scripts/check_subprocess_encoding.py,scripts/check_sync_io_in_async.py,scripts/check_agent_sdk_boundary.pyand./scripts/docs-lint.sh: all exit 0.Manual verification
N/A: unit coverage drives the real
_stream_task,_send_and_printand_run_refine, with a fake event stream as the only stand-in for the backend. No gateway, model or network was used.Related Issues
None open. Searched open and closed PRs and issues for
unfinished turn,stale_recover,timed-out turn,stop_reason,turn ceiling,partial text,never finished,EVENT_COMPLETE,_stream_task,_send_and_print,_run_refine,_run_hook_inner,refine timeoutandwebhook run history; no open PR or issue covers these loops, and no comment claims them.stream_and_collect, whereunfinished_turn_errorcomes from), PR #19008 (Slack), PR #19026 (messaging channels) and PR #19005 (auto-improvement).src/kiro_crew/cli_chat.py. The rebase onto it conflicted on one line, the neighbouringfrom kiro_crew.llm_helpers import ...incli_chat.py, which now imports both names._stream_taskfor the approval window; a trial merge of this head with it is clean. PR #19117 (merged) editeddashboard/handlers/taskrunner.py,test/test_handlers_taskrunner_coverage.pyanddocs/system-specs/modules/taskrunner.mdfor Continue in Chat; the rebase onto it applied without a conflict.Pattern harvest
Rule candidate: review-prompt
Pattern: a loop that ends on a terminal event and takes the text collected so far as the result, without reading the terminal's stop reason, while a sibling collector already treats that reason as a failure.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)