Repository navigation
fix(subagent): claim a whole answer's ending and keep its result whole - #16185
Conversation
First Principles Review (Opus 5.5) — ✅ human override accepted@bolichen97 overrode this lane for |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The completed-ending claim is a second "ended" token next to Watch
[DESIGN-REVIEWED] 95e5a5d Description read[DESCRIPTION-READ] 819ebbb240469aef665f0dbd4957af7a363d563292e312c1ff1bd8391f5526a3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS One thing to check: the reader found a "Completed" card that says "without writing any text" odd. Confirm parents and users treat a textless finish as a success. [UX-REVIEWED] 95e5a5d Description read[DESCRIPTION-READ] 60e20bbede28dbf90a306f26b7e51dbe330f6904565b48867cf9c291cc17759d That sha256 names everything this verdict read: the PR description, plus the 7 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
fb94df3 to
96edcc9
Compare
|
mechanism: write_result_chunk(fresh=True) stages through atomic_write (temp file + rename), refusing to recreate a removed run folder
|
|
|
96edcc9 to
d644f78
Compare
|
mechanism: a whole answer a reap got to first is left unrecorded by the tail, and _run writes the reap's ending (the reap-echo arm's record) and leaves the report to the reap
|
|
d644f78 to
4b7d742
Compare
|
mechanism: result_complete records one fact, "result.txt is whole", and a refused write leaves it False; the run's outcome is the tombstone's, not this flag's
|
4b7d742 to
6360ec7
Compare
|
mechanism: until an attempt's first result.txt write succeeds, each chunk starts the file with the attempt's accumulated text, so a refused first write leaves no head-missing fragment
|
|
|
|
|
A finished subagent result could read as a fragment after a gateway restart, and a stop landing while a finished run recorded its ending could leave the run returning with no ending recorded. A whole answer now claims its completed ending once, synchronously, unless a stop got there first (an error a failed child or an expired wait stamped counts as one), and every stop path treats the claim as done. The claimed run rewrites result.txt whole from memory, capped and fsynced, before recording result_complete; an empty answer leaves no file. That write is bounded, since nothing else can end it. Every other ending, including those that never reach the run's tail, caps the streamed file in place from the run's finally, and a whole rewrite that fails falls back to the same cap. An attempt's first chunk starts result.txt and the live partial afresh, so a respawn never glues two attempts; that first write is a temp file and rename, so a failed write keeps the earlier partial. Until a write succeeds, and again after a refused append, each chunk starts the file with everything the attempt streamed, so a refused write leaves no hole for a restart to find; that start write grows with the answer, so it runs off the loop in a worker drained like a state write, after the live partial takes the chunk. The usage row runs on a task cancel_all drains, so no ending waits for it. The restart reconcile, its notice and the task queue's boot probe read one completeness fact, so the result.complete marker is retired: the flag is now written right after the whole rewrite it describes. A run that finished before a restart is announced as "finished before gateway restart". Co-authored-by: Raymond Chen <45910466+chenmingwei23@users.noreply.github.com>
A successful complete event arriving while a reap tears the run down was refused the ending claim, but the tail still set done and counted a success. The reap's record guard then skipped, so a deadline-stopped run was reported completed and cleared its session's failure count. The tail now records nothing under a reap in flight; _run writes the reap's ending with the reap-echo arm's own record and leaves the report to the reap. That record says the answer finished only when a complete event ended it; a stream that just stopped names the teardown. A user stop no reap has reached yet records the neutral stop, and a run another path already ended counts no success. Each of the three is pinned by its own test. A start_queue_saturated reap, a tombstone cause main added since, settles failed on boot like startup_timeout, so the boot probe keeps its rule that every cause but gateway_restart proves an ending. _run's finally caps an unclaimed ending after the run is done, so _conversation_busy does not count it, and a release landing while the cap's worker runs wrote keep=False for that worker's whole-file update_state to roll back. The cap now holds the conversation in _abandoned_state_writers until its worker lands. The hold counts a run's writers, not the run: a state write left detached past its drain and the final cap can both be live for one id, and the earlier one landing first released the conversation while the cap was still writing. The hold now lasts until the run's last writer lands, and a writer registered twice is held once.
d0252de to
95e5a5d
Compare
|
Pushed 95e5a5d: rebased onto origin/main 3f960a1. It is still two commits, with authorship and the Co-authored-by trailer kept. The rebase had no conflicts.
Local checks: black, subprocess-encoding, isort, flake8, mypy, comment-history and harness-parity gates all pass. The PR's 15 test files plus the continuable, composition-contract, skill-budget and install-aware files ran with 1923 passed. |
|
|
|
|
|
|
|
|
/ai-review override first-principles 95e5a5d: the hard-kill gap exists on main too. There the result.complete marker is a separate write in a later step than the bytes it certifies. Here the rename, fsync_dir and the flag run back to back in one function, so the gap is narrower, and it fails safe: it under-claims (partial) and never over-claims. Closing it fully needs one atomic write of the bytes and the flag together, which is a design change; the gap is recorded in the Gateway Restart Reconciliation spec. |
Human judgment recorded@bolichen97 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
|
|
|
Problem / Motivation
Goal: A subagent run that finished with its whole answer is reported to its parent as a finished result, whatever lands around its ending: a gateway restart, a Stop, a cancel or the deadline.
After a gateway restart, a subagent that finished minutes earlier can be announced to its parent as "cut off mid-turn by gateway restart ... read it as an unfinished fragment". The only completeness signal is
result_completeinstate.json, written in a later step than the one that finishesresult.txt, so a restart in between demotes a whole answer to a fragment (#15221). The same window has a live-run variant: a Stop, a cancel or the deadline landing while a finished run records its ending can overwrite that ending or leave the run with none recorded. Separately,result.txtis appended per chunk across attempts, so a respawned run can glue two attempts' text together.Why it matters
The parent agent acts on the notice's wording. A completed answer labelled a fragment gets discarded or re-run, so finished work is silently lost whenever a restart or a stop happens to land in a short write window. Nothing looks corrupt: the bytes are on disk and only the label is wrong, so nobody notices.
Not a goal
result.txtandstate.jsonlive, or to the result byte budget.What changed (motivation → approach → change)
Symptom: a finished run reported as a fragment after a restart, and a finished run whose ending a late stop could overwrite. Root cause: "finished" was never one fact. It was a flag written after the bytes it describes, and each stop path decided for itself whether the run had already ended.
Approach: make the ending ONE synchronous claim, and make the completeness fact describe a file written from memory.
SubagentInfo._ending_claimed), unless a stop got there first (done, a reap, a user stop, or an error a failed child or an expired wait stamped). From the claim on, Stop, a parent end, the reaper,fail_parentand wait expiry treat the run as done._run's timeout, cancel and exception arms leave a recorded ending alone.settle_before_deleteanswerspendingwhile a claimed run is still writing its result.subagent_persistence.py). The claimed run rewritesresult.txtwhole from the streamed text, capped (cap_result_bytes, UTF-8 safe), fsynced with its folder, and only then recordsresult_complete=True. Both happen in one drained worker (write_finished_result), which is bounded even when nothing cancels it. An empty answer leaves noresult.txt._run'sfinallycaps the streamed file in place (context_management.py'scap_result_file, reading only the two ends) and recordsresult_complete=False. That covers the endings that never reach the run's tail, and it never raises, so the report still spawns. A whole rewrite that fails falls back to the same cap.result.txt(write_result_chunk(fresh=True), a temp file and rename, LF on every platform) and the live partial afresh, on separate signals, so a failed write never costs the partial or the earlier attempt's file.cancel_alldrains.result_is_whole(state), so fix(subagent): keep a finished result whole when the restart loses its flag #15670'sresult.completemarker is retired. A run that finished before the restart is announced✅ finished before gateway restart. The live settle and the boot probe share one outcome-to-task-state table (outcome_task_state).tombstone_terminal_state) mapsreaped,startup_timeoutandstart_queue_saturated(a cause main added in fix(acp): give interactive chat starts priority over subagent starts #16020) to failed, as the live settle does, so every cause butgateway_restartproves an ending.test_taskq_reconcile.py::test_every_tombstone_cause_has_a_terminal_statepins that no written cause is unmapped.subagent.pybinds the new helpers.update_statereads viaread_bytes_with_retry, and_atomic_writedelegates toatomic_write(fsync=True, mode=0o600).result.txtleft uncapped_run'sfinallyresult.txt🟩 added · 🟨 changed · 🟥 removed · 🟦 unchanged
Follow-up to #15670 (@chenmingwei23, credited as Co-authored-by), merged with a
result.completemarker. Here the flag lands right after the whole rewrite, in one worker job, so the marker covers nothing more.Specs updated in the same commit:
docs/system-specs/modules/subagent.md(lifecycle, ending claim, terminal-state contract, folder layout, write-site census, reconcile),docs/system-specs/modules/taskq.md(boot probe),docs/system-specs/common/injected-messages.md(orphan notice shapes),docs/request-for-change/rfc-resumable-subagent-sessions.md(cap helper names).Backwards compatibility
Compatible:
result_completekeeps its meaning and its key. A state written by an older build without the flag still reads as not whole, which is the same safe direction as today. Nothing that main accepts is rejected or renamed, except the two marker helpers #15670 added (mark_result_complete,result_marked_complete). One narrow case changes: a run that an earlier build on main finished, dropped the marker for, and lost the flag write for, before upgrading to this one, now reads as a partial (the pre-#15670 behaviour) instead of whole. A staleresult.completefile is never read and goes away with its folder. The orphan notice gains a ✅ shape, and the card renders it from the glyph.Reader: src/kiro_crew/subagent_manager/monitoring.py:tombstone_recovery_action -- subagent -- reads result_is_whole(state), the flag the claimed ending writes after the whole rewrite; test_runtime_reconcile.py::test_a_flagged_result_is_the_agents_answer
Reader: src/kiro_crew/subagent_manager/monitoring.py:_notify_orphan_impl -- subagent -- same fact, end to end over the reconcile; test_runtime_reconcile.py::test_the_reconcile_announces_a_finished_run_as_finished
Reader: src/kiro_crew/subagent_manager/admission/taskq_bridge.py:outcome_task_state -- subagent -- the boot probe reads the tombstone outcome then result_is_whole; test_taskq_reconcile.py
Reader: website/src/pages/chat/subagentCompletion.ts:parseSubagentCompletion -- chat -- the outcome is keyed on the glyph, so the reworded note parses as before; SubagentCompletionCard.test.tsx
Tests
test_subagent_stop_reason_consistency.py: the claimed ending is written after post-processing, whole and capped from memory. Every other ending caps, including endings that never reach the tail. A stamped error counts as a stop that got there first. A hung claimed write is bounded. Whatever lands after the claim, the run completes once. Every other stop path treats a claimed run as done. The usage row never holds an ending. A respawn never glues, and a cut-off respawn delivers the partial it left on disk. A failed append still delivers the whole partial. A textless whole answer leaves no file. The file is LF. A transient run touches no disk.test_subagent_persistence.py: a fresh chunk the disk refuses keeps the earlier partial, the orphan notice builder (4 cases), the restart tombstone outcome,write_finished_result(including a refused rename that still caps), andupdate_stateWindows contention on read and rename, plus the 0o600 mode.test_runtime_reconcile.py: fix(subagent): keep a finished result whole when the restart loses its flag #15670's three marker tests now pin the flag rule (flagged reads whole, unflagged under-claims, the reconcile announces a finished run as finished).test_session_start_gate.py: reads_running_countonce the run task returned, since_run'sfinallynow awaits the off-loop cap before releasing the slot.test_taskq_reconcile.py: the boot probe reads the tombstone outcome, then the cause, thenresult_is_whole. A test pins that every outcome maps.test_context_management.py:cap_result_bytesand the in-placecap_result_file(UTF-8 seams, shrink only).test_session,test_subagent_dependency_mark,test_subagent_panel_durable_replay,test_turn_duration_subagent, and others for the new record fields.SubagentCompletionCard.test.tsxfor the ✅ orphan shape.Manual verification
N/A. Unit coverage is sufficient: the restart reconcile, stop paths and file writes run against real
SubagentManagerinstances and a temp data home. A live gateway restart cannot be scripted deterministically inside the write window this closes.Screenshots / video
The three gateway-restart cards, seeded with the exact text⚠️ , Interrupted). Collapsed and expanded, light and dark.
_notify_orphan_implcomposes and rendered by the built SPA (a scratch copy ofwebsite/scripts/capture-subagent-completion-card.mjs): a whole answer (✅finished before gateway restart, Completed), a textless whole answer, and a run the restart cut off (Related Issues
Fixes #15221
Follow-up to #15670
Pattern harvest
Rule candidate: review-prompt
Pattern: a completeness flag written in a later step than the bytes it certifies, or a terminal state that each stop path decides for itself instead of one synchronous first-arrival claim.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement