Repository navigation
fix(chat): re-load a chat whose live backend lost its session (release/0.8.0) - #16412
Conversation
A backend process that is still alive can answer every prompt with "Session not found" for the backend session id the chat is bound to. The process is up, so nothing marks it dead and the claim-time dead-provider eviction never runs; the error was not retry-eligible either, so the turn ended on a bare error card and every later prompt (Continue included) hit the same binding. Only a gateway restart re-attached the chat. Treat that answer as a lost session: reset the slot's session (which keeps the session-map entry, so the next claim spawns a fresh runtime that session/loads the same backend id, falling back to a new session with history replay only when the load fails), show a short reconnect notice, and retry the turn once. A one-shot guard, re-armed only by a landed turn, ends a repeat loss on a clear error instead of looping. Co-authored-by: Joe Guo <zejiangg@amazon.com> (cherry picked from commit cf32ce7)
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm no further follow-up fixes to #16221 landed on main after c692a07 — the backport should carry the complete set. [DESIGN-REVIEWED] 290725c |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify that the ~4,500 diff lines outside the four declared files are base commits #16385/#16284/#16396 swept in by the merge diff, not undeclared branch work. Not justified as shipped
What this change shipsInventory (6 items) — 5 justifiedIntent: FIX — a chat bound to a backend session the live process no longer holds recovers on its own instead of erroring until a gateway restart (backport of #16221 + #16548 to release/0.8.0).
[FIRST-PRINCIPLES-REVIEWED] 290725c |
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @bolichen97 overrides the GPT 5.6 finding for [GPT-OVERRIDE] 290725c This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNothing blocks — one advisory: the new lost-session give-up arm is a no-requeue exit that skips the per-turn retry-budget refresh every other terminal arm performs. FINDING — src/kiro_crew/dashboard/chat_runner.py:19275 — the [OPUS-REVIEWED] 290725c Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
The session-not-found arm queued its one reconnect replay unless _should_suppress_requeue(slot) was true, which only sees a Stop still in flight. A Stop that already resolved back to idle moved only the Stop counters, and the replay snapshots are taken after it, so the drain-side veto read 'not stopped' and the cancelled prompt ran again. Pair the gate with _stop_pressed(), as every sibling requeue gate does. Fixes #16547 (cherry picked from commit c692a07)
|
mechanism: requeue gate now pairs _should_suppress_requeue with _stop_pressed (new regression test)
|
UX Review (Fable 5) — ✅ PASSUX-level review of UX-Verdict: PASS Verify "ℹ️ Session reconnect cancelled — nothing was run." doesn't read as the chat staying broken — the reset already happened and the next prompt reconnects. Suggestions
[UX-REVIEWED] 290725c |
|
/ai-review override gpt 290725c: The only blocking finding is at src/kiro_crew/config/loader.py:6466 (the default-agent healer from #16385), a file this PR does not change (its diff is chat_runner.py, state.py, llm_helpers.py and one test), so it is out of scope here and is tracked in #16555 against the code that owns it. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
Problem / Motivation
Goal: On release/0.8.0, a chat whose backend still runs but has lost the chat's session recovers on its own, instead of failing every prompt until the gateway restarts.
A live backend process can answer every
session/promptwithSession not foundfor the backend session id the chat is bound to. The process is up, so the claim-time dead-provider eviction never runs, and the error is not retry-eligible, so the turn ends on a bare error card. Every later prompt, Continue included, hits the same binding. Only a gateway restart re-attaches the chat.Why it matters
The chat looks healthy but is unusable, and the only remedy users found was restarting the whole gateway, which interrupts every other chat, cron and subagent on it. The reports are from the 0.8 line.
Not a goal
What changed
One commit, a
git cherry-pick -xof #16221 (cf32ce7d27). It applied cleanly: every helper it calls (_session_stop_generation_for,_has_user_queued_followup,_should_suppress_requeue,_queue_recovery,build_recovery_requeue,ResetCause.CONNECTION_LOST) exists on release/0.8.0 with the same shape, so no adaptation was needed.src/kiro_crew/llm_helpers.py:acp_error_is_session_not_found(exc), scoped toAcpError.src/kiro_crew/dashboard/chat_runner.py: a new AcpError arm setsneeds_session_reset, so the turn'sfinallyresets the session. That drops the live binding and keeps the session-map entry, so the next claim spawns a fresh runtime thatsession/loads the same backend id. The turn is retried once behind "⟳ The agent lost this chat's session — reconnecting…". The queued replay is dropped at the drain and at the consume seam when a Stop, a follow-up or steer, or a rebind landed meanwhile.src/kiro_crew/dashboard/state.py: the one-shot guard_session_not_found_retry_used, re-armed only by a landed turn, plus the replay's queue id, Stop counters and binding. A repeat loss ends on a clear error, never a loop._should_suppress_requeue(slot) or _stop_pressed(), so a Stop that already resolved to idle vetoes the replay. New testtest_stop_resolved_before_the_error_never_queues_a_replayfails without it (1 failed / 8 passed) and passes with it (9/9).Backwards compatibility
Compatible. Only an
AcpErrorwhose text says the session was not found takes the new path; every other error is handled as before.Tests
test/test_chat_session_not_found_recovery.py(from fix(chat): re-load a chat whose live backend lost its session #16221): 8 passed.test_acp_error_surface,test_acp_provider_retry_hint,test_dashboard_chatand the other files that cover the sibling one-shot recoveries): 976 passed.Manual verification
N/A — unit coverage drives the turn through a fake provider that answers
Session not found.Related Issues
Backport of #16221 to release/0.8.0.
Pattern harvest
Rule candidate: review-prompt
Pattern: a recovery that requeues a turn at the queue head across an awaited teardown must snapshot its queue id, Stop counters and session binding, and re-check them at the drain and the consume seam.
Checklist