Repository navigation
fix(subagent): requeue a row whose claimer is cancelled under its claim - #21570
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] ea889da Description read[DESCRIPTION-READ] 72f90d17ad20332e530373483f98daca3455dad0cf6513b2e979e52f71e10895 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check before merge: a claim that commits and then reports What this change shipsInventory (5 items) — 5 justifiedIntent: when a sub-agent spawn's claimer is cancelled mid-claim, the spawn goes straight back to the queue instead of waiting for a gateway restart. This is a FIX (Closes #20391; the new test fails on base).
[FIRST-PRINCIPLES-REVIEWED] ea889da Description read[DESCRIPTION-READ] 9fd437ede8fa034bf39d5950e664f600fdd67854c2ecd0f7f572e43191941cea That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
[operator: bolichen97] Frontend Tests (2) is red on |
|
[operator: bolichen97] Disposition: rebutted (the PASS punchline's reuse check, answered)
|
7177e4f to
6625b3c
Compare
A claimer cancelled while its claim ran on the store's writer thread left the row admitted under this incarnation's lease with nothing carrying it on, so accepted work waited for the next boot's reconcile. The claim job is now shielded and awaited to its answer; a claim that took the row is released by one generation-fenced admitted -> queued write behind it, and the pump is asked for another pass. Closes #20391
6625b3c to
ea889da
Compare
|
[operator: bolichen97] Disposition: rebutted (the PASS punchline's check, answered: not a defect)
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (6 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: bug fix; a claimer cancelled under its claim await now gives the admitted row back to queued under a generation fence instead of leaving it for the boot reconcile, with tests. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/taskq.md.
Problem / Motivation
Goal: A sub-agent spawn whose claimer is cancelled mid-claim goes back to the queue at once instead of waiting for a gateway restart.
claim_and_startawaitstaskq_claimon the store's writer thread. When the claimer's task is cancelled (a shutdown, a caller whose client went away) after that job already took the row, the row isadmittedunder this incarnation's lease, the generation never reached the claimer, and nothing carries it on. It sitsadmitteduntil the next boot's reconcile requeues it.Why it matters
Accepted work silently stalls: the user's spawn shows as queued and never starts until someone restarts the gateway.
Not a goal
_starting_claims(Subagent claim: pin every loop reader that must consult _starting_claims #20572).What changed (motivation → approach → change)
Symptom: a cancelled claimer leaves its row
admittedwith no owner. Cause: the claim await was a plainawait store.run(...), so the cancel dropped the claim's answer, and with it the generation needed to fence a release.Approach: the same shape #20319 uses one await later.
claim_and_startnow calls_claim_or_release_on_cancelinsubagent_manager/admission/pump.py. It shields the claim job and, on a cancel, awaits it to its answer. If the claim took the row, it posts one generation-fencedadmitted -> queuedwrite (_release_untaken_claiminsubagent_manager/admission/taskq_bridge.py) behind it on the same writer, asks the pump for another pass, then re-raises the cancel. No start write is posted. A claim that did not take the row, or a row a stop recorded meanwhile, writes nothing. A claim that fails with the store unavailable took nothing, so it is logged and the cancel itself propagates, never the outage. A release the store cannot take is logged and the row is left to the boot reconcile, as before.docs/system-specs/modules/taskq.mdreplaces the sentence that said this case was left to the reconcile.Backwards compatibility
Compatible: only the cancelled-claimer path changes; a claim that is not cancelled returns the same tuple as before.
Removes nothing: the boot reconcile still requeues any row this release could not write.
Tests
test/test_taskq_claim_start.py::test_a_claimer_cancelled_under_its_claim_await_requeues_the_row(new): cancel the claimer after the claim job took the row; the row endsqueuedat the claim's generation, unleased, no start write posted, reservation returned; the release requests exactly one pump pass (recorded, not run, so no pass reclaims the row while its released state is read); the test's own pump pass then starts it under a new generation. Fails on main (row staysadmitted), passes here.test/test_taskq_claim_start.py::test_a_claimer_cancelled_under_a_failing_claim_still_propagates_its_cancel(new): the claim fails withTaskStoreUnavailableafter the cancel; the claimer raisesCancelledError, the row is unchanged and nothing registers. Fails without theexcept TaskStoreUnavailablebranch (the outage replaced the cancel), passes here.test/test_taskq_bridge_bindings.py: inventory row for the new static method.test/test_subagent_scale.py::TestQueuedDepthReachesZero::test_a_popped_but_unclaimed_row_still_counts_as_pending_work: the shielded claim adds loop hops, so the test now waits for the registration itself (async_wait_until) instead of a fixed count of store barriers.MarkdownRenderer.domGoldenin the frontend related set) andtest:electron, which fails the same way on a PR with no desktop change on this host.Manual verification
N/A — unit coverage sufficient: the path is a cancel between two writer jobs, which the new test drives with a real
TaskStore.Related Issues
Closes #20391
Pattern harvest
Rule candidate: review-prompt
Pattern: an
await store.run(...)whose job mutates durable state must be shielded and its answer settled when the awaiting task is cancelled.Checklist