Repository navigation
fix(session): re-load a lost backend session on Slack, channel, cron and subagent turns - #16428
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Before merge, confirm against a real Claude adapter that a fresh runtime's [DESIGN-REVIEWED] f5cd141 Description read[DESCRIPTION-READ] ee55a7ea9fc3edb662dd9d0dad14c761e3f9873ea46b0cca12e0aaa23e2a71ad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS A seventh per-surface copy of one reset, while Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: a Slack thread, channel agent, cron job or subagent whose backend lost its session recovers without a gateway restart. FIX.
[FIRST-PRINCIPLES-REVIEWED] f5cd141 Description read[DESCRIPTION-READ] 15882d541ba4c48384f866984b8567b8d56e45323287a792baec835faaa1a800 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/subagent_manager/cancellation.py:43 -- Shared subagent recovery calls [GPT-REVIEWED] f5cd141 False positive or not applicable? A repository writer can comment: |
5eb1734 to
e92286b
Compare
|
self-added: yes
|
|
self-added: yes
|
|
Heads-up from the chat surface: #16548 (Fixes #16547) changes the lost-session requeue gate in |
e92286b to
b5ef251
Compare
|
|
|
|
|
|
|
|
7aa8a14 to
ca0977a
Compare
|
Rebased onto current Conflict in
Conflict in Verification:
Please re-check the |
|
Self-review (crew-pr-reviewer, read-only): CHANGES NEEDED I checked all five armed surfaces on head ca0977a: Slack thread, channel agent, cron single and sequence, subagent-completion injection, and subagent run. Each resets once, re-loads the same backend id and replays at most once. A second loss gives up on every surface, and none re-sends after a tool ran, because the tool-activity tag and the CONTINUE mark close the replay-after-side-effect path. The tests pin each of those branches, and the revert check in the body is per surface. Two current-head CONCERNS have no |
|
Moved to draft by crew-pr-reviewer (self-review gate). The self-review comment above lists a point this PR still needs. GitHub does not let an author request changes on their own PR, and PR Readiness does not read human reviews, so a draft is the signal: it keeps readiness at To resume: fix it, push, then run |
ca0977a to
7461ed0
Compare
…and subagent turns "Session not found" for the mapped backend session id. The other surfaces that run a turn on a mapped session still ended on a generic error with no reset, so every later message hit the same stale binding until the gateway restarted. Each surface already owns one "replace the session and run it again" seam; route acp_error_is_session_not_found into it: - Slack thread: the nested replay the compaction failure uses, with a reconnect notice; a second loss ends on a clear error. - channel agent: _stream_task returns a verdict, and a lost session goes through _recover_busy_agent like a wedged one. - cron: the single-agent ACP-death arm, and a one-shot re-run per sequence step. - subagent completion injected into a parent: reset and re-claim once. - subagent run: one respawn on a fresh runtime when no tool ran. The reset keeps the session-map entry, so the next claim session/loads the same backend id. A turn that already emitted output or ran a tool resets but does not replay. A turn that already ran a tool resets but is not re-sent: stream_and_collect tags the AcpError it raises with turn_tool_activity. The after-tool "not replayed" notice is the single shared llm_helpers.SESSION_NOT_FOUND_NOT_REPLAYED_TEXT on every surface, including the subagent run arm (which previously wrote it inline), and a lost sequence step whose sibling completion is still injecting defers its reset to _subagent_done instead of tearing the session down. Refs #16225 Co-authored-by: cixuuz <26782336+cixuuz@users.noreply.github.com>
7461ed0 to
f5cd141
Compare
|
|
Addressed on head Point by point against the self-review above:
Checks: black / isort / flake8 / mypy clean on the changed files; 117 backend tests pass locally (incl. the 9 cron-sequence tests). All machine checks on this head are green; the four review lanes re-ran with no BLOCK (Design PASS, First Principles advisory CONCERNS dispositioned, Opus & GPT no blocking findings). Marking ready. |
|
Self-review (crew-pr-reviewer, read-only): RECOMMEND APPROVE Re-reviewed head |
bolichen97
left a comment
There was a problem hiding this comment.
Approving head f5cd141.
- Read the diff and its call paths (session-not-found recovery per surface; slack handler replay; cron step retry; subagent injection/run respawn).
- PR Readiness is success from github-actions[bot] on f5cd141.
- Read review threads and AI-review comments on this head: no open BLOCK, no unanswered CONCERNS, no CHANGES_REQUESTED in force.
- Compat/UX check: compat risk none, UX risk low; a turn without tools is re-sent once automatically; merge-tree is clean against main 6765454 with no semantic overlap.
Reviewed by Kiro (agent) on behalf of bolichen97 as part of a repo-wide PR sweep.
|
Landed as f92855b (squash of head f5cd141) through the auto-merge already armed on this PR, right after the sweep's approval. Checked before approving: PR Readiness success on f5cd141, no blocking review left, clean against current main. Kiro (agent), on behalf of bolichen97, repo-wide PR sweep. |
Problem / Motivation
Goal: a Slack thread, channel agent, cron job or subagent whose live backend lost its session recovers on its own, the way the dashboard chat already does.
A backend process that is alive can answer every prompt with
Session not foundfor the backend session id a conversation is bound to. #16221 fixed this for the dashboard chat only. On the other turn surfaces the error still ends the turn with no reset, so every later message hits the same stale binding until the gateway restarts.Why it matters
The conversation looks healthy but fails every message. The only remedy users found is restarting the whole gateway, which interrupts every other chat, cron and subagent on it.
Not a goal
What changed (motivation → approach → change)
Session not found; Slack, channel, cron and subagent turns fail on every message.acp_error_is_session_not_foundis routed into that seam, not copied into a new one.resetkeeps the session-map entry, so the next claimsession/loads the same backend id; a new session plus history replay only when that load fails):slack/handler.py): the compaction failure's nested replay moves into one_replay_abandonedclosure. A lost session resets, posts "⟳ The agent lost its session — reconnecting…", and replays once. A second loss, or a loss after output or a tool card, ends on a clear error.channel.py):_stream_taskreturns a verdict (busy,lost,lost_after_tool). A lost session goes through_recover_busy_agentwith the same notice. After a tool ran it is replaced but not replayed.slack/gateway.py): added to the existing ACP-death reset-and-retry arm. The failure alert names the outcome: the give-up after a retry, or "not re-run" when a tool ran.slack/gateway.py): a one-shot re-run of the step; a second loss raises the give-up error.slack/gateway.py_inject_with_retry): reset the parent and claim a fresh client once, without spending an injection attempt.stream_and_collecttags theAcpErrorit raises withturn_tool_activity(acp_error_after_tool_activityreads it), and the cron post-token CONTINUE (slack/gateway_runtime/cron_dispatch.py) marks any loss after its first prompt as activity.subagent_manager/run.py,cancellation.py, one-shot flag insrc/kiro_crew/subagent.py): one respawn through_schedule_cancel_recoverywhen no tool ran; otherwise the run ends with its partial kept and a clear error.docs/system-specs/modules/session.mdgains a section with the rule, the per-surface seam table, the no-replay-after-a-tool invariant and this list.llm_helpers.pynext to the predicate:SESSION_NOT_FOUND_NOT_REPLAYED_TEXTis the one source every surface's after-tool arm uses, including the subagent run arm (subagent_manager/run.py), which no longer writes it inline. A lost cron sequence step whose sibling completion is still injecting defers its reset to_subagent_doneinstead of resetting the session out from under the pending completion. (The dashboard chat path's ownchat_runner.pynotice constants are a separate copy; folding them is left to the dashboard, which is a Not-a-goal here.)Backport.
release/0.8.0does not have #16221's predicate or dashboard arm, and the reports come from 0.8. A 0.8 backport is needed, and it should take #16221 and this PR together.Backwards compatibility
Compatible: each surface gains an arm for an error that used to end the turn; every other error takes the same path as before.
_stream_tasknow returns a string verdict instead of a bool; its only caller and its tests are updated in this diff.Removes nothing: the diff only adds arms, a tool-activity tag and notice text.
Tests
test/test_session_not_found_surfaces.py, a fake ACP raisingAcpError("Session not found: …"):_stream_taskclassifies the loss; recovery replays once with the notice; a loss after a tool replaces without replay; a second loss returnsNone.stream_and_collecttags tool activity on a lost-session error.Revert check: with the predicate disabled per surface, that surface's tests fail (Slack 3/3, channel 3/4, subagent 3/3, gateway cron+injection 6/6). With
acp_error_after_tool_activityforced False, 4 tool-activity tests fail; with the CONTINUE tag removed, its test fails. Restored, all 21 pass.Neighbours pass:
test_channel_prompt_busy.py,test_cron_acp_retry.py,test_gateway_pipe_death.py,test_slack_handler.py,test_subagent_turn_resilience.py,test_chat_session_not_found_recovery.py.Manual verification
N/A — the
Session not foundanswer is reproduced with a fake provider on each surface; the Claude adapter that sends it was not installed on the test host.Related Issues
Refs #16225: the guard-record fold it also asks for is deferred to a follow-up, so the issue stays open. Follows #16221.
Pattern harvest
Rule candidate: review-prompt
Pattern: a recovery wired into one turn surface must be checked against every other surface that runs a turn on the same session map.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)