Repository navigation
feat(subagent): wake memory-queued spawns on events with per-lane shares - #17030
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check one case before merge: if the waiting lane's start can never fit, [DESIGN-REVIEWED] 680a2e3 Description read[DESCRIPTION-READ] 243df252b968a584151981fa1591184eccaebdb55975fedf8b349e6404124170 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ human override accepted@bolichen97 overrode this lane for |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Heads-up on an overlap: #17106 (fixing #17104) changes the unit of |
98ac6ea to
034a4ae
Compare
|
|
|
|
|
|
Rebased onto main after #17017 merged: the U4 commit is dropped and this PR is one commit (034a4ae), marked ready. The rebase also follows two changes main made under it: Red on 034a4ae: Comment History Gate (three "no longer" phrasings), plus Await Fast Gate and Coverage Gate, which fail because of it. GPT's fixed-sleep finding and the two First Principles items are fixed in the next push, which is already committed locally as 60073fdbbd and not pushed yet (one push per round). The dispositions above record each one. |
034a4ae to
9c9160c
Compare
|
|
|
9c9160c to
260271a
Compare
|
|
With count caps gone, memory is the pool every chat's subagents share. A start the floor deferred was re-checked only when its 30 s admit wait passed, and one chat's wave could still take every start that fit while another chat waited for memory. - A memory wait is recorded from its first deferral until it fits, starts or ends. A child's terminal (run end or reap), a wait ending and a sampler tick each run one fit pass: one off-loop host reading, the bar rebuilt as the gate builds it (so a warming row that settled is seen at the next tick), and only the waits that now fit are brought forward (a durable row through TaskStore.expedite, no event; an in-memory park cut on the floor's monotonic_ns clock). A wait that still does not fit is left parked with nothing written. The sampler runs only while a wait exists; the admit wait stays as the restart-safe backstop. - Per-lane memory share: while another lane waits below its share, a root start of a lane at or above its share of running dedicated children waits too (low_memory, no figures; audited with cause lane_share). The share follows agent.lane_weights; the starved lane is never held, so it cannot deadlock. Nested starts and claim re-entries are exempt. No count cap; WRR ordering stays. - A nested child that shares its parent's runtime is admitted at ~0 against the floor while its row keeps the shared price, so the floor gives way by one shared start, not a fan-out. The 2 GiB floor, the macOS pressure hold, the explicit-pin floor of 3 and the 1800 s max wait are unchanged. Specs: subagent.md (Memory waits: event wake and the per-lane share, the floor guarantee), adaptive-concurrency.md, taskq.md, dynamic-subagent-sizing.md, subagents.md, configuration.md; rfc-overload-resilience.md Q12 records the event wake, the share and the shared nested child (D4). Part of #16480
260271a to
680a2e3
Compare
|
|
|
|
|
/ai-review override first-principles 680a2e3: Maintainer decision. RFC Q12 records the owner-directed fix for #16480 symptom b, where one chat's wave takes every memory admission. D4 comes from that redesign's critique: a yielded parent keeps its RSS, so a nested child sharing its runtime must be admissible. AGENTS.md requires the spec in the same commit as the code. Q12 restates Q9 openly: the floor yields by at most one shared start, and a dedicated child still waits. |
Human judgment recorded@bolichen97 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
Problem / Motivation
Goal: A subagent start waiting for memory starts as soon as memory allows it, and one chat's subagent wave cannot take every start that fits while another chat waits for memory.
Since U4 (#17017, merged) removed the count caps, free memory alone bounds concurrency. Two gaps remain:
A nested child that shares its parent's runtime also waited at the floor, although its parent, often waiting on exactly that child, already holds the memory. Only the 30-minute max wait could end that.
Why it matters
Anyone who runs subagents from more than one chat on a host near its memory floor sees starts that sit for up to 30 s after memory frees up. One chat's fan-out also takes every admission while the other chat shows "Waiting for memory". A parent waiting on its own shared child at the floor can end only at the max-wait terminal.
Not a goal
What changed (motivation → approach → change)
Root cause: nothing tied a memory wait to the events that move memory, and memory admission had no notion of a lane's share. RFC Q12 (
rfc-overload-resilience.md) records the decisions, extending §6 Fairness to memory.admission/memory_wake.py,types.MemoryWait): one entry per start, from its first deferral until it fits, starts or ends.run.py, and the reap interminal.py), a wait ending while another lane waits, and a 5 s sampler armed only while a wait exists each run one fit pass: ONE off-loop host reading, each bar rebuilt as the gate builds it (so a settled row is seen at the next tick), and only the waits that fit and the share does not hold are brought forward. A durable row goes through the newTaskStore.expediteintaskq/store.py(no event); an in-memory one has its_floor_waitspark cut on themonotonic_nsclock. A wait that still does not fit is left parked, with nothing written. The admit wait stays as the backstop.lane_share_holds, ingate.py). While another lane waits below its share, a root start of a lane at or above its share of running dedicated children waits too:low_memorywith no figures, audited with causelane_share. Shares followagent.lane_weights; the starved lane is never held, so it cannot deadlock; nested starts and claim re-entries are exempt. Not a count cap.subagent.pyholds the record's state and_holds_dedicated_runtime; shutdown (cancellation.py) cancels the sampler and any fit pass.Specs:
subagent.md,adaptive-concurrency.md,taskq.md,dynamic-subagent-sizing.md,subagents.md,configuration.md, RFC Q12; help text for three keys inconfig/sections.py, withconfig-baseline.jsonregenerated.Judgement calls
_floor_waitsin nanoseconds (fix(subagent): end a floor wait on the park that reaches the bound #17106) andTaskStore.defer(wait=)(fix(taskq): take a memory park's span from one clock read #17217).deferredevent per admit wait: U5a's max wait sums them, so dropping them needs a schema change. The gate audits each deferral, as main does.Backwards compatibility
Compatible: no config key, schema field or API changes.
agent.lane_weightskeeps its meaning for picks, and now also weights the memory share.TaskStore.expediteis new and writes no event. Thedeferred_low_memorySEL row keeps its fields. A lane-share row leaves outavailable_gb, the same way a reading that did not answer already does.Tests
test/test_subagent_memory_wake.pydrives the realSubagentManagerpump and task store on a virtual clock; only_run_inneris stubbed. A sampler test fires its tick itself. Every barrier is a signal (store flushes, depth-emit tasks, the fit and pump tasks, a per-start event), never a timed sleep._floor_waits' nanosecond unit (this fails with a float-seconds cut).lane_weightschange moves the share.deferredevent, no depth frame and no audit row. A share hold is audited with causelane_shareand no figures.test/test_taskq_store.py:expediteends a deferral early with no event, and a row parked again is charged only the gap.Each fails with the line it pins removed. Determinism: 10 runs at
-n 4and one shuffled run, all green.Local runs: the lint, type, comment-history and Fast Gate scripts and docs-lint pass; 4771 tests (subagent, taskq, adaptive, config clusters) pass, and
test_subagent_memory_wake.pypasses 3 times at-n 4.Manual verification
N/A: scheduling paths inside the manager, driven by the tests above.
Related Issues
Part of #16480. Implements U5 (b) of its subagent admission redesign; it builds on U4 (#17017) and U5a (#16347), both merged.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement