Repository navigation
fix: leave a busy parent running when a completion delivery times out - #33184
Conversation
A sub-agent completion that waited out the 1200 s delivery cap reset the parent session unconditionally, so a cron, channel or webhook parent still busy in its own long turn was killed mid-turn. By the time the timeout arm runs, the cancelled delivery route has released any lease it held, so a turn that still holds the session is the parent's own. The arm now resets only when no turn holds the session (sessions.turn_holder) and logs the reset it declined. A stuck injection is still reset, including when its release has just handed the lease to a queued sibling delivery, which then gets a fresh provider. Builds on #26275. Co-authored-by: Robert Noack <rnoack@amazon.com>
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The one behaviour change targets the cause: the timeout arm reset the parent without checking whether the parent's own turn held the lease. Everything else is a log line and a spec update. Not checked: whether What this change shipsInventory (3 items) — 3 justifiedIntent: let an unattended run keep working when a finished child's result cannot reach it within 20 minutes. This is a FIX (#22157).
[FIRST-PRINCIPLES-REVIEWED] ab85266 Description read[DESCRIPTION-READ] 672bca7fb94af5fc9d0d2b0dc3f3368c039fabf923d2ff6e422c00a73aecc5e4 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the reset now depends on who holds the lease, which is the property that actually separates a stuck injection from a busy parent. Notes
[DESIGN-REVIEWED] ab85266 Description read[DESCRIPTION-READ] dbd7fb9e23511df6038e1d0c42c5eeb75f4fd22c7b33a181485a0613c4e4cbde That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (5 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: a completion injection that times out no longer resets the parent session when the parent's own turn holds it; the reset still runs when no turn holds the session. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/architecture/resource-protection.md, docs/system-specs/modules/subagent.md.
…#33184) A sub-agent completion that waited out the 1200 s delivery cap reset the parent session unconditionally, so a cron, channel or webhook parent still busy in its own long turn was killed mid-turn. By the time the timeout arm runs, the cancelled delivery route has released any lease it held, so a turn that still holds the session is the parent's own. The arm now resets only when no turn holds the session (sessions.turn_holder) and logs the reset it declined. A stuck injection is still reset, including when its release has just handed the lease to a queued sibling delivery, which then gets a fresh provider. Builds on #26275. Original author: Bolin Chen (cherry picked from commit f216ae2)
…#33184) A sub-agent completion that waited out the 1200 s delivery cap reset the parent session unconditionally, so a cron, channel or webhook parent still busy in its own long turn was killed mid-turn. By the time the timeout arm runs, the cancelled delivery route has released any lease it held, so a turn that still holds the session is the parent's own. The arm now resets only when no turn holds the session (sessions.turn_holder) and logs the reset it declined. A stuck injection is still reset, including when its release has just handed the lease to a queued sibling delivery, which then gets a fresh provider. Builds on #26275. Original author: Bolin Chen (cherry picked from commit f216ae2)
Problem / Motivation
Goal: A sub-agent completion that cannot be delivered within 20 minutes no longer kills a parent that is still busy in its own long turn.
When a finished sub-agent's result waits longer than
_ON_DONE_TIMEOUT(1200 s) to reach its parent, the delivery is given up and the parent session is reset. A cron job, channel or webhook run whose own turn keeps working for more than 20 minutes after a child finishes is killed mid-turn by that reset: the gateway logscompletion injection timed out ... after 1200s, thenruntime kill pid=<parent> ... reason=provider shutdown, and the run ends with an error.Why it matters
Any unattended run that starts a short child and then keeps working (a long build, a code change and a push) loses its live turn 20 minutes after the child ends, and each later child's delivery kills the replacement process the same way. The reset was meant to clear a stuck injection prompt; a busy parent is not one.
Not a goal
spawn_sub_agentsalready returned inline (fix(subagent): mark returned results only after the reply is written #17684, Cron and channel parents get a redundant completion turn for each child spawn_sub_agents already returned #23434).INJECTION_TIMEOUTresets, which act only on an attempt that holds the parent's session itself.What changed (motivation → approach → change)
A parent that is busy in its own turn now keeps running when a completion's delivery times out; a delivery that is itself stuck still gets the parent's session reset.
The timeout arm in
subagent_manager/terminal.pyreset the parent unconditionally. The cron and channel routes wait for the parent's session lease inside the 1200 s cap, so a long, healthy turn reached the same arm as a stuck injection. At that moment the two differ in who holds the lease: the timeout cancels the route first, and the route'sfinallycancels its own prompt and releases any lease it held before the arm runs. A turn still holding the lease is then the parent's own turn, or a channel member's lifetime lease. A permit that release handed to another queued delivery names no holder until that delivery runs.The arm now asks
sessions.turn_holder(parent_key). No holder: the session is reset as before, so a stuck provider is cleared and a queued delivery validates onto a fresh one. A holder: the parent is left running and the arm logs that. Either way the delivery is still reported failed throughnotify_injection_failed. The_ON_DONE_TIMEOUTrow and the Two-Level Timeout section ofdocs/system-specs/modules/subagent.md, and the outer-cap row ofdocs/architecture/resource-protection.md, state the rule.Backwards compatibility
Compatible: the reset is declined only while a turn holds the parent's session; every timeout with no holder resets exactly as before.
Removes nothing: no path, record, key or function is taken away; one reset now waits for no turn to be holding the session.
Tests
test/test_subagent_injection_timeout_reset.pydrives the realSessionManagerlease under the real terminal report, with only the provider process mocked.test_a_parent_busy_with_its_own_turn_is_left_running[turn-lease|lifecycle-lease]runs the real cron route and locks in that the parent's session, provider and turn survive the timeout.test_a_stuck_injection_is_reset_while_a_sibling_waits_for_the_parentlocks in that a delivery stuck holding the lease is still reset when its release hands the permit to a queued sibling, and that the sibling gets a fresh provider. It passes onmaintoo, which resets unconditionally: it guards the reset this change keeps.test_a_stuck_injection_prompt_is_still_resetruns the same through the real cron route with no sibling.test/test_subagent.py::TestOnDoneTimeout::test_injection_timeout_resets_parent_session: with no turn holding the parent, the arm still resets it.-n0(80 passed); 10 repeats at-n 4 --dist loadgroup --max-worker-restart=2besidetest_subagent_injection_live_turn.py,test_subagent_injection_session_reload.pyandTestOnDoneTimeout, one of which lost every test at setup when a concurrent session on the shared temp root removed its pytest base directory and passed again with a privateTMPDIR; one shuffled order (seed 25715, the same 14 tests). All green.Proof: PROVEN test/test_subagent_injection_timeout_reset.py::test_a_parent_busy_with_its_own_turn_is_left_running[turn-lease], test/test_subagent_injection_timeout_reset.py::test_a_parent_busy_with_its_own_turn_is_left_running[lifecycle-lease] (head d949bdf8dd40)
Manual verification
N/A — unit coverage sufficient: the tests drive the real session lease, cron route and terminal report, and only the kiro-cli process is mocked.
Screenshots / video
N/A — no frontend change.
Related Issues
Closes #22157
Builds on #26275 by @rnoack1.
Sibling sweep
Swept: git grep -nE "sessions.reset(" -- src/kiro_crew/subagent_manager src/kiro_crew/slack/gateway_runtime/subagent_injection.py src/kiro_crew/slack/gateway_runtime/runtime_charge.py
Sibling: src/kiro_crew/slack/gateway_runtime/subagent_injection.py:1207 -- not affected -- every parent reset in this file runs under the route's own claim (the channel attempt timeout at 1207, the cron route's new-session reset at 1340 and its injection timeout at 1392) or already passes skip_if_busy=True (the idle cleanup at 1503)
Sibling: src/kiro_crew/slack/gateway_runtime/runtime_charge.py:293 -- not affected -- the prompt-busy and process-death resets in
_inject_with_retry(293, 329, 368) run under the delivery's own claimSibling: src/kiro_crew/subagent_manager/terminal.py:1051 -- not affected -- resets the child run's own session during a reap, never the parent's
Sibling: src/kiro_crew/subagent_manager/run_lifecycle.py:176 -- not affected -- resets the child run's own session at its teardown, never the parent's
Pattern harvest
Not generalizable: the defect is one timeout arm resetting a session after its caller had already given up the lease; every other parent reset either runs under the route's own claim or already passes skip_if_busy.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)