Skip to content

test(subagent): pin the startup-progress wake edge - #14367

Merged
iamwhatever merged 1 commit into
mainfrom
test/subagent-startup-progress-wake-edge
Sep 27, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
test/subagent-startup-progress-wake-edge

Conversation

@pepmach

@pepmach pepmach commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The suite notices when one of the three lines of production code that wake a spawn the in-startup bound is holding is gone.

Three lines of production code wake a spawn that the in-startup bound is holding, and the suite did not notice when one of them was gone. _note_startup_progress is called at the dedicated-process PID record in _run_inner, at the shared-runtime PID record in _bind_shared_handle (both in subagent_manager/run.py), and on the first frame of a run's own turn in SubagentManager._leave_startup (subagent.py). Deleting the dedicated PID-record call left every test green. The other two were pinned only by mocks that assert the call happens; nothing drove the queued spawn to the start the call exists to cause.

Why it matters

That call is the only edge that announces a non-terminal exit from the in-startup population: the slot-release drain fires on a terminal and the pump does not poll. Lose one call site and a fan-out wider than _startup_cap() (4 at the stock agent.session_start_concurrency of 2) sits queued with free running slots until an unrelated terminal or a new arrival happens. That is a silent hang, and no test would go red.

Not a goal

  • Any production code, default or document change: the change is test only, and the manual verification is N/A for the same reason.
  • Touching startup_full = False in _should_stagger_queue_impl or the _gate_wait_started stamp: both are left alone on purpose, already load-bearing per the issue's own verification.
  • Replacing the existing mock-based pins on the shared-runtime and first-frame call sites: the new cases run beside them, and the Tests table counts both.

What changed (motivation → approach → change)

Test only. One new class in test/test_subagent_startup_pressure.py, TestLeavingStartupStartsTheHeldSpawn, with one case per call site.

Case Real path driven What must happen
test_the_dedicated_pid_record_starts_the_held_spawn _run_inner, get_pid answers a PID, the stream is held open and silent the held spawn starts before any frame arrives
test_the_first_frame_of_its_own_turn_starts_the_held_spawn _run_inner, get_pid answers nothing (a child created lazily from stream()), one roster frame, then the stream is held open nothing starts on the open silent stream; the held spawn starts on the first frame
test_the_shared_runtime_pid_record_starts_the_held_spawn _bind_shared_handle with runtime.pid set the held spawn starts at bind

The shape is the same in every case. Gate width 1 gives a bound of 2; a peer and the driven start fill it; one more spawn goes through the real spawn() and queues with six free running slots. The test never calls _drain_queue() or _note_startup_progress() itself. It watches the queued spawn start while the driven start is still mid-execution, so a terminal's drain cannot be what moved it. Every wait is asyncio.wait_for with a ceiling or the file's own _until on observable state; every hold is released in finally. No production code, default or document changes.

Backwards compatibility

Compatible: tests only, nothing that runs today changes.

Tests

Each call site deleted alone (the line replaced by pass), then test/test_subagent_startup_pressure.py test/test_subagent_startup_watchdog.py test/test_session_start_gate.py run with -n0:

Call site deleted Before this change With this change
dedicated-process PID record (_run_inner, run.py) 92 passed, 0 failed: the gap test_the_dedicated_pid_record_starts_the_held_spawn red; the other two new cases green
shared-runtime PID record (_bind_shared_handle, run.py) 4 red, all asserting on a mocked _note_startup_progress: test_shared_start_is_out_of_startup_at_bind, the three test_a_co_tenants_fanned_out_frame_neither_starts_nor_marks_the_run cases those 4 plus test_the_shared_runtime_pid_record_starts_the_held_spawn red; the other two new cases green
first frame (_leave_startup, subagent.py) 3 red, all asserting on a mocked _note_startup_progress: test_the_first_event_received_leaves_startup, test_a_completion_withheld_for_recovery_is_an_answer, test_a_routed_roster_frame_is_the_sessions_own_answer those 3 plus test_the_first_frame_of_its_own_turn_starts_the_held_spawn red; the other two new cases green

Unmodified code: the three files pass (95), the pressure file alone passes under -n 4 --dist loadgroup, and the new class passes 8 consecutive isolated runs. Gates run from the worktree: black ratchet, subprocess-encoding check, isort, flake8, mypy (1734 files, no issues), scripts/local-gate.py (backend related set 59 passed; frontend related set 302 files, 9016 tests passed).

Left alone on purpose, already load-bearing per the issue's own verification: startup_full = False in _should_stagger_queue_impl and the _gate_wait_started stamp.

Manual verification

N/A — unit coverage sufficient: test-only change with no product code and no UI.

Screenshots / video

Why no screenshot: test-only backend change; no rendered surface is touched.

Related Issues

Fixes #14023

Pattern harvest

Rule candidate: review-prompt
Pattern: a mock that asserts a wake or pump call was made pins the call, not the wake; the test must let the pump run for real, with the queue populated, and observe the consumer start before the driven run's terminal can stand in for the edge under test.

Local review

One local round, one lane at a time, each briefed from the contract CI reads on the base commit. The lanes judged the diff at a7c4e00a2d; the pushed head f3a9f54116 is that single commit replayed onto the moved base e4e6ed7c60, byte-identical (git diff --stat $(git merge-tree --write-tree a7c4e00a2d e4e6ed7c60) HEAD is empty; the base's movement touched neither this file nor the code under test), with the static gates, the three test files, mypy and the local gate re-run on it.

  • GPT (gpt-5.6-sol, codex-review contract): no findings, PASS.
  • Opus (claude-opus-5, claude-review contract): discovery raised six candidates, validation kept none. It measured each call site deleted in its own process and saw exactly its own case go red, confirmed the AcpSessionProvider built at bind creates no task, thread or timer, and read CapacityView.any_slot as a real property. PASS.
  • First Principles (claude-fable-5): no CONCERNS, PASS. It counted the three production call sites, confirmed no existing test drove a queued spawn to a start through a real call site, and asked that the red/green table be verified by execution, which the runs above and the Opus lane's own per-process measurement are.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable) — N/A, the spec already names these edges as pinned by this file
  • No secrets, credentials, or internal references in the diff

_note_startup_progress is the only edge that announces a non-terminal exit
from the in-startup population, and none of its three call sites was driven
to the start it exists to cause: the dedicated-process PID record in
_run_inner could be deleted with the suite fully green, and the shared-runtime
PID record and the first-frame exit in _leave_startup were pinned only by
mocks asserting the call happens.

One harness, three cases, each driving one transition through its real code
path with a spawn queued behind a full startup bound: the queued spawn must
start with no _drain_queue() call of the test's own, observed while the
driven start is still mid-execution so a terminal's drain cannot be what moved
it. Deleting any one call site turns exactly its own case red while the other
two stay green.
@pepmach
pepmach requested review from a team and smeyffret September 27, 2026 04:50
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the per-call-site deletion table was exercised at the pushed head, not only the pre-rebase commit — it is the PR's entire value claim.

[DESIGN-REVIEWED] f3a9f54

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] f3a9f54

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] f3a9f54

Verdict parsed from the review's SHA-scoped output markers for commit f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f3a9f541164ec4c4cea86cf6b9f0c2a6c612dece: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision labels Sep 27, 2026
@pepmach

pepmach commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

Description gains the **Goal:** line and ## Not a goal section the PR template requires since #14181 -- a first write of the frozen sections from this PR's opening intent, not an edit of a frozen goal. Head unchanged.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 27, 2026 17:31
@iamwhatever
iamwhatever merged commit 80c2fb8 into main Sep 27, 2026
104 checks passed
@iamwhatever
iamwhatever deleted the test/subagent-startup-progress-wake-edge branch September 27, 2026 17:31
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: pin the startup-progress wake edge that keeps the in-startup bound from wedging

2 participants