Repository navigation
test(subagents): pin the pre-first-turn reaper coverage gap - #10902
iamwhatever merged 1 commit into
Conversation
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed |
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The "must edit these tests deliberately" guarantee fails for the wall-clock test: it asserts against a hand-copied mirror of the reaper, which a real fix bypasses silently. Watch
Suggestions
[DESIGN-REVIEWED] a7fb865 |
Record two states a subagent can sit in while shown as `starting` that neither the fast startup watchdog nor the stuck-wave sweep reaps: a wave member stranded in `_queue` (never in `_agents`, so no per-agent reaper sees it, and the sweep skips any wave holding one), and a registered run parked before `_run_inner` with `_exec_started is None` (the startup watchdog keys on `_exec_started`, so its predicate is False). In both, the only backstop is the wall-clock reaper. The tests are named for the gap and each defect-encoding assertion is marked in its docstring as currently-observed behaviour a fix is expected to change, so closing the gap must edit these tests deliberately rather than pass them silently. The wall-clock assertion pins the absence of any shorter bound, not the timeout value. This change pins the gap; it does not fix it. Whether a queued or pre-execution run should carry an independent deadline measured from registration, and what expiry should do, is left for a maintainer. ## Pattern harvest Not generalizable: a repo-specific characterization test for one manager's reaper coverage; no reusable rule. Refs kirodotdev#6303
1ba40ef to
a7fb865
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: adds one new characterization test file under test/ pinning two uncovered pre-first-turn subagent reaper states (queued wave member absent from _agents, registered run with _exec_started None); no production file touched. CodeQL is not applicable on this fork PR (default-setup emits no check-run); SAST coverage is Semgrep only, latest run success with 0 annotations.
…ev#10902) Record two states a subagent can sit in while shown as `starting` that neither the fast startup watchdog nor the stuck-wave sweep reaps: a wave member stranded in `_queue` (never in `_agents`, so no per-agent reaper sees it, and the sweep skips any wave holding one), and a registered run parked before `_run_inner` with `_exec_started is None` (the startup watchdog keys on `_exec_started`, so its predicate is False). In both, the only backstop is the wall-clock reaper. The tests are named for the gap and each defect-encoding assertion is marked in its docstring as currently-observed behaviour a fix is expected to change, so closing the gap must edit these tests deliberately rather than pass them silently. The wall-clock assertion pins the absence of any shorter bound, not the timeout value. This change pins the gap; it does not fix it. Whether a queued or pre-execution run should carry an independent deadline measured from registration, and what expiry should do, is left for a maintainer. ## Pattern harvest Not generalizable: a repo-specific characterization test for one manager's reaper coverage; no reusable rule. Refs kirodotdev#6303
Problem / Motivation
A subagent can sit in the
startingphase for over an hour with nothing flagging it, and a blocking orchestrator waiting on such children looks stuck too (issue #6303). "starting" means a run is registered but has not advanced past turn 0. Two states in that phase are reaped by no fast watchdog and end only at the wall-clock reaper, whose default_default_timeoutis 3 hours:_queueand never inserts it into_agents. The reaper's per-agent loop iterates_agentsonly;_is_startup_stalledreturnsFalsewithout_exec_started; and_sweep_stuck_wavescontinues past any wave that still holds a queued member. The only exit is_drain_queue, gated on a free slot, so a fan-out wider than the cap parks its tail instartingbehind runners that are themselves only reaped at the wall clock._run_inner(_exec_started is None, e.g. awaiting a spawn approval that, per the code's own comment, has "a wait with no deadline of its own") is invisible to the 120s startup watchdog, and its stall badge is suppressed while awaiting approval. It is backstopped only by the wall clock.This PR records those two states as executable characterization tests. It does not fix them: whether a queued or pre-execution run should carry an independent deadline measured from registration, and what expiry should do (reap, re-queue, or notify), is a policy decision about what
startingandqueuedmean, and a wrong value would kill legitimate long-queued fan-out. That decision is left for a maintainer and is posted as a question on issue #6303.Why it matters
A run that shows
startingfor an hour-plus with no stalled indicator is indistinguishable, to a user, from a hang. The orchestrator that blocks on it inherits the same appearance. The startup watchdog added for the post-_exec_startedwindow does not reach either state above, so the observable symptom in #6303 persists on currentmain. A related issue, #6484 ("runs stay queued without starting a child ACP process"), was closed by #7299, which fixed only the reporting of approval-parked runs and not the queued-member no-deadline gap; without a recorded test, a reader concludes the class is handled when half of it is not.What changed
One test file,
test/test_subagent_startup_prefirstturn_gap.py, five tests, no production change:_queueentry is absent from_agents;_sweep_stuck_wavesleaves a queued-member wave untouched andbatch_members_pendingstaysTrue;_is_startup_stalledreturnsFalsewhen_exec_started is None; and, for a pre-execution run, the reaper's per-agent decision does not terminate it at any instant short of_default_timeout. The wall-clock test pins the absence of a shorter bound, not the 3-hour value._exec_startedon turn 0 with no pid is reaped fast) so the boundary between covered and uncovered is legible.Three mechanisms ruled out (recorded so they are not re-walked):
_agentsreleases its slot on a terminal outcome through a one-shot release token that also drains the queue, so the queue does drain -- just as slowly as the wall-clock-reaped runners ahead of it free slots._exec_started(its first statement in_run_inner), so the 120s startup watchdog covers it._agentsloop that flags or badges at all.Tests
test/test_subagent_startup_prefirstturn_gap.py, 5 passed, run single-file with a fixed single worker (-n0) against a scratchKIROCREW_HOME. Comment History Gate passes on the added lines (diff-scoped againstorigin/main). No production code changed, so no behavioural test needed re-running.Refs #6303