Repository navigation
fix(dashboard): drop a cancelled threshold request's pending commit - #36630
Conversation
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
[operator: bolichen97]
This PR changes only the dashboard slot settings and their tests, and the same node id is red on other unrelated heads today. |
ff80c35 to
64cce26
Compare
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns. The cleanup now waits for the shielded worker to finish before it drops the record, so the worker can't write the record after the drop. The transcript lock is held for that wait, so the drop can only remove this request's own record. Notes
[DESIGN-REVIEWED] d1b9e62 Description read[DESCRIPTION-READ] 2dd3b3d1641a1ac6c4b3847027bc82b4d207074a95942a5f6915d1e57d94aab7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: both routes' cancellation leak is fixed in one shared helper that has two real callers, and four new tests fail on base for the reported reason. What this change shipsInventory (5 items) — 5 justifiedIntent: cancelling a threshold or turn-limit save should not leave a stale value that every later save of that transcript writes. FIX.
[FIRST-PRINCIPLES-REVIEWED] d1b9e62 Description read[DESCRIPTION-READ] feeca0045a80e241dc999ca6a3f0a23f563bf7b7aa5a1efab95c649655468a17 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
The threshold and turn-time-limit routes record a committed value for the transcript from the save's commit hook, and every later save of that transcript writes it until the route publishes it and drops the record. The save shields its executor future, so when the request is cancelled during the save the route unwinds at once while the worker runs on and may record the value after the route is gone. Nothing then publishes or drops it, and every later save writes a value the tabs and the live session do not hold. Each route now runs the save as its own task and awaits it shielded. On cancellation it waits for that save to settle, still inside the transcript's transaction lock so no later request's record can be the one dropped, drops the record and re-raises. One helper serves both routes. Part of #32569. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
64cce26 to
d1b9e62
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
I reviewed head d1b9e62. Both the threshold route and the turn time limit route now await their save through one shared helper. If the request is cancelled, the helper waits for the shielded save to settle, still holding the transcript lock, then drops the pending commit record. A late commit therefore cannot outlive its request or remove a later request's record. The four new cancellation tests fail on the base for the reported reason and pass here, and both affected test files pass in full locally. I found no behaviour, crash or data-loss issue that blocks this change.
Problem / Motivation
Goal: A threshold or turn-limit request cancelled during its save leaves no pending commit behind for later saves of that transcript to write.
The compaction-threshold route (
POST /api/chat/slots/{slot}/autocompact) and the turn-time-limit route (POST /api/chat/slots/{slot}/turn-timeout) record the value they committed in a process-wide dict (autocompact_commits/turn_timeout_commits) from the save's commit hook. Every save of that transcript writes the recorded value until the route publishes it to the slot and drops the record. Only the route's normal path dropped it.save_slot_off_loopshields its worker, so when the request task is cancelled during the save, the route unwinds at once while the worker runs on and may record the value after the route has gone. Nothing then publishes or drops it.Why it matters
After such a cancellation, every later save of that transcript writes a value the tabs and the live session do not hold, until the next request for the transcript. A handler task is cancelled when the server's runner shuts down, and a test process builds many
DashboardStateinstances that share the dict, so a stale record from one request can leak into another state's saves. The fix is also what keeps the commit window correct if handler cancellation on client disconnect is ever turned on.Not a goal
save_slot_off_loop, the save folds (autocompact_fold/turn_timeout_fold), or the normal-path publish and drop.test_a_queued_alias_save_writes_a_merges_commit. The intermittent failure the issue reports did not reproduce in 60 runs (see Manual verification), so that test stays as it is and the issue stays open for it.What changed (motivation → approach → change)
A cancelled threshold or turn-limit request could leave its committed value recorded, and every later save of that transcript then wrote it.
The record is written by a hook that runs in the shielded save worker, which can run after the cancelled route has unwound. So a plain
finallythat drops the record is not enough: it runs before such a late write.Both routes now await the save through one helper in
slot_settings.py,_save_dropping_commit_on_cancel. It starts the save as its own task, eagerly, so the save still dispatches in the route's own tick, and awaits it shielded. On cancellation it waits for that save to settle, drops the record, and re-raises. The route still holds the transcript's transaction lock while it waits, so no later request's record can be the one dropped.chat_handlers.pyre-exports the helper because the route functions run on that facade's globals. The spec paragraph that documents the commit record (docs/system-specs/modules/learn-cron-dashboard.md) states the cancellation rule for both routes.Backwards compatibility
Compatible: responses and the normal path are unchanged; only a request that is already being cancelled now waits for its own save before it finishes unwinding.
Removes nothing: the only thing dropped is an in-memory record that nothing publishes once its request is gone.
Tests
test/test_session_autocompact_override.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[before-cancel|after-cancel]test/test_session_turn_timeout.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[before-cancel|after-cancel]Each runs the handler as a task it holds. The save stand-in follows the real contract, awaiting
asyncio.shield(worker), so cancelling the route does not cancel the worker. The worker runs the commit hook either before the request is cancelled or after it has unwound. Each test asserts the dict holds no record for the transcript, the slot keeps its old value, and (threshold route) the live override was never set. No sleeps; every await is bounded.Proof: PROVEN test/test_session_autocompact_override.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[before-cancel], test/test_session_autocompact_override.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[after-cancel], test/test_session_turn_timeout.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[before-cancel], test/test_session_turn_timeout.py::TestACancelledRequestLeavesNoCommit::test_a_cancelled_request_drops_its_commit[after-cancel] (head e93d146d682d)
prove.pyran one93d146d682d, which has this head's parent and the same code and tests; only one spec sentence differs.Manual verification
Run on Ubuntu Linux (x86_64, Python 3.12.3).
On the base (
c22557cd8) with only the two test files added, all four cases fail for the reported reason:On this change:
Static gates (black, isort, flake8, mypy, comment history, brand name, file line cap, harness parity, sync I/O in async, loop-bound locks, docs lint, flake-pattern and wait ratchets, semgrep on the diff) all exit 0.
scripts/local-gate.py's backend related set: 12951 passed, 2 failed intest/test_source_link_github_case.py::TestOneChipPerObject::test_case_variants_collapse_to_one_chip; the same two fail on the base whentest/test_session_card_source_links_knob.pyruns first on the worker, so they are not from this change.The intermittent alias-save failure the issue reports was not reproduced. The issue's seven-file
pytestcommand ran 60 times with every write to the transcript's metadata line and every set and drop of its commit record logged:Every fold in that test wrote 60.0, each record was set in the save worker before the queued alias save and dropped on the loop after it, and no other test touched that record. The host reached about 2x CPU oversubscription, against about 3.6x in the issue's run.
Related Issues
Part of #32569
Sibling sweep
Swept:
git grep -nE "_commits\[[^]]*\] *=" -- src/Sibling: src/kiro_crew/dashboard/chat_api/slot_settings.py:195 -- changed here
Sibling: src/kiro_crew/dashboard/chat_api/slot_settings.py:476 -- changed here
Sibling: src/kiro_crew/dashboard/guide_runs.py:946 -- not affected -- a token map written on the event loop, not from a shielded save worker's hook
Pattern harvest
Rule candidate: review-prompt
Pattern: state recorded from inside shielded or executor-run work and dropped only on the awaiting caller's normal path; on cancellation the caller must wait for that work to settle before it cleans up, because a plain
finallyruns before the worker's late write.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)