Skip to content

fix: keep a resume prefetch whose runtime start has begun - #33898

Open
neilieva wants to merge 1 commit into
kirodotdev:mainfrom
neilieva:fix/33711-prefetch-cancel-kill
Open

neilieva wants to merge 1 commit into
kirodotdev:mainfrom
neilieva:fix/33711-prefetch-cancel-kill

Conversation

@neilieva

@neilieva neilieva commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Clicking through resumable chats no longer throws away resume prefetches that already started their runtime.

Focusing a resumable chat schedules a resume prefetch. The focus handler cancels that prefetch on the next focus change, on blur and when the socket closes. That cancel is meant for the 1.5 s debounce, but it also lands after the debounce, once the runtime has spawned and session/load has run. The CancelledError then goes through the start's except BaseException cleanup, and the runtime is killed as failed session setup cleanup or failed init handshake cleanup. #33711 has logs of runtimes killed 2 to 17 s after spawn, a second after session_load and set_mode logged outcome=ready.

Why it matters

Every runtime thrown away this way costs a full start: the agent process, its MCP stub processes and the stdio MCP servers. That's about twenty process launches on the host in the report. It hits anyone with session.eager_spawn on who moves between chats after a gateway restart, and the chat they come back to still cold-starts on its first message.

Not a goal

  • Changing the init-handshake cleanup's kill reason. That cleanup runs in its own shielded task and never sees the exception, so it would need the exception passed in. Test doubles in test_acp_runtime_rederive_retry.py match on its exact reason string.
  • Changing the unclaimed-prefetch TTL, the population cap or the debounce.
  • Changing the other cancels of a slot's eager spawn task (slot delete, agent/project re-arm, session environment switch, app ownership refusal, shutdown). Each one tears the session down anyway.

What changed (motivation → approach → change)

A cancel can't safely interrupt a start that already has a process, so the focus path stops cancelling once the start begins. _eager_spawn adds its task to _starting_spawns (held in chat_runner next to _armed_prefetches) for the length of _spawn_admitted_prefetch, and eager_spawn_is_starting reads it. _handle_slot_focused leaves a starting task running, and the socket close in _serve_ws goes through the same handler as a blur. Refocusing a slot whose prefetch is starting returns that task instead of arming a new one, which would cancel it through schedule_eager_spawn. A reload's respawn is checked first and stays with the reload, so a focus returns None for it even while it is starting. The finished session goes to the existing TTL and population cap, as the issue suggests. A cancel during the debounce or before admission still lands. An admitted prefetch is kept even while it waits in the start queue, before its process exists. The ws.py entry in learn-cron-dashboard.md now says so. A cancel that still reaches the session setup cleanup in acp_runtime_start.py kills the runtime as cancelled session setup cleanup, and any other setup exception kills it as failed session setup cleanup. That change and its test come from #33768.

Backwards compatibility

Compatible: nothing is rejected or renamed. A focus change, blur or socket close now skips one cancel it used to issue.

Removes nothing: no path, record, key or function is taken away. The predicate and the set are new.

Tests

test/test_eager_spawn_slot_frames.py:

  • test_focus_change_after_start_keeps_the_load[t2] and [None]: a focus change to another chat, or a blur, while the prefetch is inside get_or_create doesn't cancel it. The load finishes and the TTL is armed.
  • test_refocus_while_starting_keeps_the_load: leaving the chat and coming back while its prefetch is starting keeps the same task and runs one load.
  • test_a_starting_reload_respawn_is_not_adopted_by_a_focus: a focus on a slot whose reload respawn is starting returns None and leaves the respawn running. It fails if the reload check and the starting check swap order.
  • test_socket_close_after_start_keeps_the_load: drives api_ws with a socket that sends one slot_focused frame and then disconnects while get_or_create is blocked. The starting prefetch isn't cancelled, the load finishes and the TTL is armed.
  • test_a_failed_start_is_not_kept_by_the_next_focus: a start that raises leaves _starting_spawns, and the next focus of its slot doesn't adopt the finished task. _finish_load also checks the task left the set after a successful load.
  • The existing test_focus_change_cancels_previous_prefetch and test_blur_cancels_and_schedules_nothing still pass, so a cancel during the debounce still lands.

test/test_acp_provider_setup_cleanup_reason.py:

  • test_a_setup_cleanup_kill_names_why_the_setup_ended is parametrized over both branches. A CancelledError from create_session kills the runtime once, as cancelled session setup cleanup. A RuntimeError kills it as failed session setup cleanup. With the change reverted, the cancelled case sees failed session setup cleanup and fails. The checks live in their own file because test/test_acp_provider.py is already at the 2,000-line cap on main.

Proof: PROVEN test/test_acp_provider_setup_cleanup_reason.py::test_a_setup_cleanup_kill_names_why_the_setup_ended[error1-cancelled session setup cleanup], test/test_eager_spawn_slot_frames.py::TestSlotFocusedFrame::test_focus_change_after_start_keeps_the_load[t2], test/test_eager_spawn_slot_frames.py::TestSlotFocusedFrame::test_focus_change_after_start_keeps_the_load[None], test/test_eager_spawn_slot_frames.py::TestSlotFocusedFrame::test_refocus_while_starting_keeps_the_load, test/test_eager_spawn_slot_frames.py::TestSocketCloseKeepsStartingPrefetch::test_socket_close_after_start_keeps_the_load (head a81c406)

The related suites (2,513 tests: test_acp_provider.py, the new cleanup-reason file, test_eager_spawn_slot_frames.py, the websocket suites and the suites #33875 and #35739 touched) pass on this head. A full scripts/local-gate.py run on an earlier base passed 18,767 tests. Its 13 failures fail the same way on unmodified main on this host (test_cli.py::TestDoctor cannot read a home-directory file, and two test_mcp_gateway_pool_integ.py tests cannot read a stub's start-time identity).

Manual verification

N/A: unit coverage drives the real _handle_slot_focused, api_ws and _eager_spawn with a session manager that blocks inside get_or_create. I haven't repeated the issue's click-through against a live gateway.

Related Issues

Closes #33711

Sibling sweep

Swept: git grep -nE "(_eager_spawn_task|_focus_task|prev_task|prev).cancel()|task.cancel()" -- src/kiro_crew/dashboard
Sibling: src/kiro_crew/dashboard/ws.py -- changed here
Sibling: src/kiro_crew/dashboard/chat_turn/eager_spawn.py -- not affected -- 503 re-arms on new bindings, so the old runtime is torn down either way, and 1156 is gateway shutdown, which has to stop every start
Sibling: src/kiro_crew/dashboard/chat_api/slot_lifecycle.py:785 -- not affected -- the slot is being deleted
Sibling: src/kiro_crew/dashboard/chat_turn/app_agent_binding.py:56 -- not affected -- an app-owned turn was refused and its session must not serve
Sibling: src/kiro_crew/dashboard/handlers/session_environments.py:725 -- not affected -- the session moves to another environment

Pattern harvest

Not generalizable: the defect is one handler cancelling a task past the point where cancelling is cheap. The other cancels of this task are deliberate teardowns.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@neilieva
neilieva requested review from a team and KiroCrewCaptain as code owners November 9, 2026 14:12
@neilieva
neilieva requested a review from Zedmor November 9, 2026 14:12
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 9, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 10, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from 7af2f38 to b1f3fb6 Compare November 10, 2026 05:23
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 10, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from b1f3fb6 to 12ee8d4 Compare November 11, 2026 14:10
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 11, 2026
@KiroCrewCaptain

KiroCrewCaptain commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider thinks the change is probably fine but estimates an 18% chance that this change introduces a defect, above the 12% limit for approving on its own.

The fix does what the PR says. It marks an eager-spawn task as "starting" around _spawn_admitted_prefetch (after admission; removed in a finally), and _handle_slot_focused no longer cancels a starting task on focus change, blur or socket close. Socket close now goes through the same handler as a blur. Refocusing a slot whose prefetch is starting returns that same task. A reload's respawn is still checked first, and a cancel during the debounce still lands. compose() binds the eager-spawn functions to chat_runner's globals, so the new set resolves correctly. The cleanup kill now reports "cancelled session setup cleanup" for a CancelledError; nothing in src/ matches on the old string. The deliberate teardown cancels (shutdown, slot delete, agent/project re-arm, environment switch) are unchanged. The tests wait on events, drive the real handlers and pin each new branch.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 47fdca3a16 · 254 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 11, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from 12ee8d4 to 47fdca3 Compare November 11, 2026 19:22
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 11, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from 47fdca3 to f47c009 Compare November 12, 2026 14:20
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 12, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from f47c009 to 5d166b7 Compare November 13, 2026 14:25
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 13, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 14, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from 5d166b7 to f7583f8 Compare November 14, 2026 12:12
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 14, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from f7583f8 to a1344e7 Compare November 16, 2026 16:38
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 16, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from a1344e7 to a81c406 Compare November 17, 2026 08:24
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 19, 2026
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from a81c406 to 73f6766 Compare November 19, 2026 22:04
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 19, 2026
A focus change, blur or socket close cancelled the slot's resume
prefetch even after its runtime had spawned, often after session/load
had already run. The cancel reached the start's BaseException cleanup,
which killed the runtime as a failed session setup or a failed init
handshake. Clicking through resumable chats paid a full process-tree
start each time and kept none of them.

A prefetch is marked as starting while _spawn_admitted_prefetch runs.
The slot_focused handler leaves a starting prefetch running, and the
socket close takes the same path as a blur. A later focus of its slot
keeps it instead of arming a new one. The finished session falls to the
existing unclaimed-prefetch TTL and population cap. A cancel during the
debounce still lands. A cancel that still reaches the session
setup cleanup kills the runtime with the reason "cancelled session setup
cleanup". Other setup exceptions use "failed session setup cleanup".

Closes kirodotdev#33711

Co-authored-by: jeeshofone <167437+jeeshofone@users.noreply.github.com>
@neilieva
neilieva force-pushed the fix/33711-prefetch-cancel-kill branch from 73f6766 to eb1e059 Compare November 20, 2026 20:03
@github-actions github-actions Bot removed the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cancelling a resume prefetch after its runtime is up kills the runtime as a failed setup

2 participants