Repository navigation
fix(subagent): keep a finished result whole when the restart loses its flag - #15670
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The owning spec still pins the old contract — Watch
[DESIGN-REVIEWED] 142a8f4 Description read[DESCRIPTION-READ] cfe90cc9cd2d3899d834e189637a72180b4e68a859c9a711ba72d50e0ea8c1e9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/subagent_manager/run.py:2660 -- A restart after False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The bodyless marker ships temp-file+rename torn-write machinery a zero-byte file cannot need — Not justified as shipped
What this change shipsInventory (6 items) — 5 justifiedIntent: FIX — a finished subagent result that survived a gateway restart is reported to its parent as a whole answer, not a mid-turn fragment (linked issue #15221).
[FIRST-PRINCIPLES-REVIEWED] 142a8f4 Description read[DESCRIPTION-READ] 8b36d5c85b522aedf756e818c007626df046acc03d4cf2e1b57a40dfac91b643 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNothing blocks; the new completion-marker drain is the one off-loop writer in this module that is neither bounded nor repeated-cancel-safe. FINDING — src/kiro_crew/subagent_manager/run.py:2677 — the drain FINDING — src/kiro_crew/subagent_manager/monitoring.py:140 — the classifier now accepts two proofs of completeness, but every statement of that contract still says one: the calling function's own docstring at monitoring.py:424 ("only a run that recorded [OPUS-REVIEWED] 142a8f4 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
1e7ab32 to
970624b
Compare
970624b to
7e4ee10
Compare
…s flag A subagent run finalizes result.txt in one step and records result_complete in state.json in a later step. A gateway restart landing between the two left a whole answer on disk with the flag never written, so the orphan reconciler read the completed answer as a fragment and told the parent it was cut off mid-turn. Drop a durable completion marker (result.complete) in the same completion step that caps result.txt, before the separate state.json write. The reconciler's classifier treats the marker or the state flag as proof the answer is whole, so the crash window between the two writes no longer under-claims a finished run. A run that never completed writes neither signal, which stays the safe direction. The marker is atomic (temp file + rename) and transient runs that never touch disk write no marker.
7e4ee10 to
142a8f4
Compare
|
Follow-up opened as #16185, with @chenmingwei23 credited as Co-authored-by. This branch has #16185 fixes #15221 with a different design. This PR adds a |
iamwhatever
left a comment
There was a problem hiding this comment.
Reviewed: result.complete is written atomically right after cap_result_file and only on a successful complete event, so a genuinely interrupted run is still reported as a fragment; continuations mint fresh ids so an old marker cannot vouch for later output. No deletion beyond the marker's own temp file, no kills. Nits: mark_result_complete's docstring says it never overwrites but Path.replace does; the cancel branch in run.py swallows non-OSError from the marker writer silently, a debug log would help.
Problem / Motivation
Goal: A subagent run that finished and wrote its whole answer to disk is reported to its parent as a finished result after a gateway restart, not as a fragment cut off mid-turn.
At a gateway restart, a subagent run that finished and wrote its whole result to disk minutes earlier can be reconciled as
partial_resultand announced to its parent as "cut off mid-turn by gateway restart ... read it as an unfinished fragment, not as the agent's answer". The label is driven entirely by whether one flag,result_complete, was persisted — and that flag lives instate.json, written in a step after the one that finalizesresult.txt. A restart falling in that gap leaves a complete answer on disk with the flag never written, and the reconciler then demotes a whole finding to an opening sentence.Why it matters
The parent agent acts on the notification's wording. A completed subagent answer relabeled a fragment is read as an incomplete opening sentence, so the parent discards or re-runs work that was already done — silently losing a finished result whenever a restart happens to land in a sub-second write window. The failure is invisible: nothing is corrupt, the bytes are all present, only the label is wrong.
Not a goal
result.txtorstate.jsonare written, only adds a second, atomic completeness signal beside them.What changed (motivation → approach → change)
Observed symptom: a finished subagent run announced to its parent as "cut off mid-turn" after a gateway restart, its complete answer read as a fragment.
Root cause: completeness and the bytes it describes live in two files written at two times.
write_result_chunkstreamsresult.txtin chunk by chunk and the completion path caps it;result_completeis recorded only later, in a separatestate.jsonwrite.tombstone_recovery_actionread completeness offstate.jsonalone, so a restart between the cap and the flag write classified a whole answerpartial_result.The specific change makes the completeness signal land atomically with the file it describes. The completion path drops a durable marker (
result.complete) in the same step that capsresult.txt, before the separatestate.jsonwrite, via temp-file + rename so a torn write is never read as half-present. The classifier treats the marker or the state flag as proof the answer is whole:subagent_persistence.py:mark_result_complete(agent_id)writes the atomic marker (skipped for transient in-memory runs, matchingwrite_result_chunk);result_marked_complete(agent_id)reads it.subagent_manager/run.py: the completion path writes the marker right aftercap_result_fileand before the_write_state_off_loopflag write, under the same_complete_event is not None and _stop.is_successcondition. The marker write goes off the event loop viaasyncio.to_thread(the filesystem work must not block the gateway's single loop on a network-backed data home), and is shielded and drained on cancellation like the sibling_write_state_off_loop: a cancel during the rename holds until the worker settles, so the marker lands before any terminal tombstone a cancel arm writes and the two never disagree on disk.subagent_manager/monitoring.py:tombstone_recovery_actionreturnspartial_resultonly when neitherstate["result_complete"]norresult_marked_complete(agent_id)is set.subagent.py: the new helper is threaded onto the two component-binding surfaces sorun.pyresolves it at runtime.Backwards compatibility
Compatible: adds a new signal (
result.complete) and widens the completion test to an OR. A run with the oldstate.jsonflag and no marker is still classifiedresult_available; a run with neither is stillpartial_result. No existing field is rejected, renamed, or removed, and the new marker file is additive beside the agent directory's existing files.Tests
Added to
test/test_runtime_reconcile.py, each registered in the module's mutation-target docstring:test_the_durable_marker_rescues_a_finished_result_whose_flag_never_wrote— a whole result with the marker present but nostate.jsonflag classifiesresult_available.test_a_result_without_either_signal_still_under_claims_as_a_fragment— the complement: no flag and no marker stayspartial_result, preserving the safe direction.test_the_reconcile_calls_a_marker_only_finished_run_orphaned_not_cut_off— end to end over the real orphan reconcile: the notification reads "orphaned by gateway restart" (not "cut off mid-turn") and the tombstone'srecovery_actionisresult_available.Added to
test/test_subagent_persistence.py(TestResultCompleteMarker), covering every branch of the new helpers: the write→read round trip, a transient (incognito/temporary) run writing no marker, a failed atomic rename swallowed with its temp file cleaned up, and the reader answering False for an unusable agent id.Added to
test/test_subagent_stop_reason_consistency.py, driving the realSubagentManager.spawn→_run→_run_inner_implpath (only the ACP session faked): a cleanend_turncompletion drops the durable marker on disk, and a stream that dies with no complete event drops none — pinning the production invariant that the marker is created by successful completion, not only by a direct helper call.Each was confirmed to fail without the fix (reverting the classifier's marker clause reds the first and third) and pass with it. Full
test/test_runtime_reconcile.py(127 tests) and the subagent persistence / component-globals / continuable / result-view suites pass.Manual verification
N/A — unit coverage sufficient: the defect is a disk-state classification reproduced directly by seeding
result.txt+ the marker with nostate.jsonflag, both at the classifier and end to end over the reconcile.Related Issues
Fixes #15221
Pattern harvest
Rule candidate: review-prompt
Pattern: a completeness/validity flag persisted in a different file (or a later write) than the bytes it describes can diverge across a crash — the signal must land atomically with, or no later than, the data it certifies.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)