Skip to content

fix(subagent): veto macOS starts under kernel memory pressure - #15876

Merged
bolichen97 merged 3 commits into
mainfrom
fix/macos-pressure-veto
Oct 3, 2026
Merged

bolichen97 merged 3 commits into
mainfrom
fix/macos-pressure-veto

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On macOS, a root subagent start waits while the kernel reports memory pressure and one of this gateway's own dedicated runtimes is running. New starts then stop pushing a Mac that is already compressing and swapping further into swap.

On macOS the memory floor (agent.spawn_min_memory_gb) reads only the Mach reclaimable page count (free + inactive + speculative + purgeable). That count does not show the compressor. A Mac can report gigabytes "reclaimable" while the kernel already rates its memory pressure as WARN, and the gate then admits another kiro-cli runtime into that state.

This matters now that #15890 lowered the floor to 2 GiB. The bar a dedicated start must clear is about 3 GiB, and the only WARN reading on record (2.59-2.77 GB) sits right at it.

Why it matters

A 16 GB Mac is the common laptop case. Without a backstop, a lower floor admits starts into compressor pressure and swap thrash. The whole machine slows, not just Kiro Crew. The kernel's pressure verdict is the one signal macOS gives for "short of memory whatever the counters say". This PR makes the floor honour it for the starts we control, as the floor's second input rather than a second rule.

Not a goal

What changed (motivation → approach → change)

Motivation → approach. The level is the kernel's lagging verdict, not a measure of free memory (#15244 read NORMAL with 4.8 of 6.0 GB swap in use), so it backs the figure up and never replaces it. It is read fresh on every check. The hold is part of the floor rule: off with the floor (0), and it applies only while a dedicated runtime of ours is running or warming; otherwise the pressure is not ours to relieve. The rule's full statement is docs/system-specs/modules/subagent.md (macOS: the kernel memory-pressure hold).

Commit 1: fix(subagent): veto macOS starts under kernel memory pressure

  • platform_compat.memory_pressure_level() reads kern.memorystatus_vm_pressure_level via sysctlbyname (1/2/4 = NORMAL/WARN/CRITICAL; None off macOS or on failure, which fails open).
  • The hold (admission/gate.py) keeps a ROOT start, shared-priced included (a shared start still launches its own MCP servers; the owner's scope decision, and the maintainer's ruling that only dedicated runtimes count as ours: comment). Nested children are never held.
  • It is a capacity-style wait: the row stays in the window labelled memory_pressure; slot releases and a 15 s timer re-check it.
  • It is bounded: 1800 s per row (_PRESSURE_HOLD_MAX_WAIT_SECS), and an episode that has lasted that long ends the starts it would hold at once (owner decision; failing open after the bound was weighed and rejected, because the start would then launch into the very pressure it waited on). Either way the start ends "never started: waiting for memory", with a WARNING and a never_started_memory_pressure SEL row whose expired_by names the bound. agent.subagent_queue_max_wait_secs is a separate PR; whichever lands second re-points the constant.
  • Agent validation now runs before the memory gates, and its directory scans (incl. app ownership) run off the event loop (AgentCheck): the reordered re-validation runs from the pump on the event loop, and the repo's blocking rule no-blocking-call-on-event-loop was upheld against the on-loop version on this PR.
  • Reporting: the level in resource_status, the [RESOURCES] line and diagnostics; prewarm admission skips while the hold applies.
  • UI: the memory_pressure queued kind, a card header and counted ErrorNotice for starts that never started, in all 12 catalogs.

Commit 2: fix(subagent): keep a waiting row's approval and answer refusals (three pre-existing defects on paths the hold now takes routinely)

  • A window refill dropped approval_mode, so a held App Kit spawn drained into an unanswerable prompt; the refill now restores it (in-process).
  • finish_parked_defer announced a refusal on TaskStoreUnavailable for a row still QUEUED; it now answers queued.
  • The channel spawn reply said "Spawned" for a refused record; it now says it was not started, and why.

Backwards compatibility

Compatible: nothing that works today is rejected or renamed.

  • Off macOS, and with spawn_min_memory_gb = 0, every new branch is inert.
  • Reader: older dashboards. The memory_pressure reason is additive; they fall back to the default queued text.
  • Reader: callers naming an unknown agent. They are refused agent_not_found before any memory or capacity wait instead of after it.
  • Reader: channel users of spawn. A refused spawn's reply changes from "Spawned ..." to "not started: ".
  • Reader: App Kit spawns that waited. A refill keeps approval_mode="auto" instead of raising a prompt.

Tests

  • test/test_admission_gate.py: the hold queues a start in the window (WARN and CRITICAL) with no store defer; five cases let a start through (no runtime of ours, level unreadable, NORMAL, floor 0, nested child); posture-critical wins; an unknown agent is refused first; a restricted spawn waits in memory; a held member keeps its asking turn; a shared wave never holds itself while a shared-priced start is held; WARNING once per level and again after an episode ends; the per-row bound ends a row never started (gate, pump and approval release), and a spent episode ends new starts at once; a held wave starts on the slot release with the pick never re-entering the gate meanwhile; a dedicated wave is released one runtime at a time with clocks kept; the recheck timer notices pressure easing and is cancelled at shutdown; relabelling at the pick and at the end of the hold; the label over a full cap; approval release held, and a held root never blocks a child behind it; the unreadable macOS figure; the real readers end to end; the coroutine pump keeps the row without a defer; a direct table for _owns_dedicated_runtime; queued-stop cleanup; the user docs state the bounds the code uses. Commit 2 adds the App Kit refill and the defer-outage tests.
  • test/test_spawn_floor_defaults.py: the top-up waits on the hold for a root and never for a nested child.
  • test/test_eager_spawn.py: the hold blocks a new pre-warm.
  • test/test_platform_compat.py::TestMemoryPressureLevel: each level through a real ctypes function pointer, failure shapes, a concurrent first build, and a macOS-only read of the real sysctl.
  • test/test_resource_status.py, test/test_messaging_commands.py, website/src/test/ActivityViewer.test.tsx.
  • Mutation-checked locally: the pick filter, the per-entry release, the kept clocks, both relabels, the timer and its shutdown cancel, the shared-price exclusion, the label precedence and each commit-2 fix each turn a test red when removed.
  • Red from main's cross-merge, not from this PR: Frontend Tests (1) and Frontend Tests (2) fail on src/test/SessionActionsMenu.duplicate.test.tsx and src/test/hoverVariantPolicy.test.ts, which fix(website): gate hover so iOS taps land on the first try #14807's hover gate broke. Both reproduce on a clean main worktree, both fixes are in flight (fix: repair two frontend tests broken by the hover-gate cross-merge #16210, with fix(website): repair hover policy and SessionActionsMenu tests after #14807 #16223 overlapping it), and this PR touches neither file.
  • Locally red on BOTH this tree and clean main (host-only, not touched here): TestPeakRssIsThisProcesss::test_a_child_of_a_bloated_parent_reports_its_own_small_peak, test_platform_compat_refactor_facade::test_every_owner_is_loaded_with_the_facade, test_telegram_transport_dispatch_composition_contract::test_a_fresh_import_loads_every_owner.

Review rebuttals

  • "Count running shared sessions as runtimes of ours." That would serialize every shared wave behind its own first member. The hold waits only on dedicated runtimes because their end is what returns memory; a shared start's MCP cost is priced by the floor.
  • "assert events == [] is vacuous in the restricted-spawn test." Verified it can fail under the inline pump conftest pins; the test is rewritten anyway, since restricted spawns now queue.
  • "Add an autouse reset of resource_status._cached_decision." The cached verdict has no reader in the suite (healthy_host_memory and the spawn-pin ratchet cover every spawning test), and the hold reads the level fresh, so the conftest level pin covers it.
  • "Filter the WARNING counts by logger name." The message filter plus the logger-scoped caplog.at_level already isolate the counts.
  • "The advisory 'may queue' note shows with the floor at 0." It says "may": the probe cannot see the gate's other conditions (a dedicated runtime running) either. Gating the note on the floor is listed as a follow-up.
  • "Agent validation now scans the agents directory on first entry for a queued spawn." That is the cost of refusing an unknown agent before any wait; list_agents is cached on the directory's stat signature.

Follow-ups (held)

  • Feed the level into AdmissionDecision.admitted (cron, runner lane, workflow ctx.agent).
  • Derive the posture tier from the level (notifier, pipeline-conductor table, the resource_status Posture line).
  • Adaptive-controller integration (SIGNAL_MEMORY on WARN).
  • Count runner-lane sessions as ours.
  • Parent teardown, continuation busy checks and the crew-log turn on the low_memory / posture_critical STORE-deferral path (the pressure hold avoids them by not deferring; approval is now covered for all three kinds by commit 2).
  • An episode notifier for the pressure hold; gate the advisory note on spawn_min_memory_gb > 0.

Manual verification

Not run on a real Mac. No macOS host was available. What was verified instead:

  • The sysctl values come from Apple's XNU source. bsd/kern/kern_memorystatus_notify.c's sysctl_memorystatus_vm_pressure_level answers convert_internal_pressure_level_to_dispatch_level(memorystatus_vm_pressure_level), which maps Normal / Warning+Urgent / Critical to NOTE_MEMORYSTATUS_PRESSURE_NORMAL / _WARN / _CRITICAL (0x1 / 0x2 / 0x4 in bsd/sys/event_private.h), the same values as libdispatch's public DISPATCH_MEMORYPRESSURE_*.
  • The handler checks PRIV_VM_PRESSURE only under !XNU_TARGET_OS_OSX, so an ordinary macOS process can read it. The new macOS-only test reads the real sysctl on the macOS CI runners.
  • The UI sentence was rendered in the real built SPA (screenshots below).

Still worth doing on a Mac: sysctl kern.memorystatus_vm_pressure_level should print 1 on an idle machine, and memory_pressure -S -l warn (Apple's simulator, needs sudo) should move it to 2 and show the chip sentence on a queued spawn.

Screenshots / video

Captured from the real built SPA (npm run build) with every /api/** call and the websocket stubbed, using website/scripts/capture-subagent-visibility.mjs. The subagent_queued frames carry the gate's figure-free reason: "memory_pressure"; the timed-out shots push subagent_done frames with outcome: "failed" and the gate's own never-started error. No gateway was running. All 12 catalogs carry these strings. Every image below was captured at this PR's current head.

Held, run card (dark and light):

Run card: 3 agents queued, the starts resume on their own once it eases or the running agents finish; starts give up once the pressure has lasted 30 minutes (dark)

Run card: the same held wave on the light theme

Every member ended at the bound, run card (dark and light): the header says what happened, and a counted error notice directly under the card (the shared ErrorNotice, outside the card button) gives the cause and names the control that retries.

Run card: 3 agents never started; 3 never started, macOS memory pressure did not ease in time; once it eases or a running agent finishes, use Retry failed in the Subagents panel (dark)

Run card: 3 agents never started (light)

A mixed wave (two ran, one was ended at the bound) says how many ran, and the counted notice binds the cause to the one that did not:

Run card: 2 of 3 agents ran, 2 done and 1 failed; 1 never started, macOS memory pressure did not ease in time, Activity Monitor shows it (dark)

Every member ended at the bound, Subagents panel (dark)

Subagents panel: Error, never started: waiting for memory (macOS memory pressure did not ease in time); retry once it eases or a running agent finishes

Related Issues

Refs #15244 #15270 #15890

No linked issue: this is the macOS backstop for the 2 GiB floor (#15890); #15244 stays open because there the kernel read NORMAL, which this change does not touch.

Pattern harvest

Rule candidate: review-prompt
Pattern: an OS memory signal (a kernel pressure level, a PSI reading) used as a measure of free memory, or a NORMAL reading taken as proof of headroom. Such signals lag, so they back up a measured figure; they never replace it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@bolichen97
bolichen97 requested a review from a team October 1, 2026 13:20
@bolichen97
bolichen97 requested a review from a team as a code owner October 1, 2026 13:20
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of 97e31197db58314db244ba76c36ad870b7994c6a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

A spent episode refuses new root starts outright: on a Mac that sits at WARN, every new root spawn ends "never started" while a long dedicated run continues.

Watch

  • Cause: "an episode that has lasted that long ends the starts it would hold at once" (_pressure_episode_spent). Mechanism: macOS WARN lags and can stay chronic on a 16 GB laptop. While the episode lasts and one of our dedicated runtimes keeps running, each new root spawn is ended with waited_secs≈0, and nothing retries it. Consequence: unattended cron, chat and channel work that fans out loses its subagents for good. The only remedy is spawn_min_memory_gb = 0, which turns off the floor itself. The owner chose this; humans should confirm they accept it as the steady state for chronic WARN.
    Clears when: a macOS measurement shows how long WARN episodes typically last with one kiro-cli runtime running, or the spent-episode refusal is replaced by holding the start until that runtime ends.
  • The hold covers only subagent-manager starts. Cron, the runner lane and workflow ctx.agent still admit into WARN (listed under "Not a goal"). So the same Mac now gives two different admission answers until that follow-up lands.
    Clears when: the follow-up issue is linked in the PR, or the spec's hold section names the paths the hold does not cover.

[DESIGN-REVIEWED] 97e3119

Description read

[DESCRIPTION-READ] feee671bede179d121b78f52ae3157166b29234df7c5be617b972f27f2fee52b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 97e31197db58314db244ba76c36ad870b7994c6a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 97e3119

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 97e31197db58314db244ba76c36ad870b7994c6a: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 97e31197db58314db244ba76c36ad870b7994c6a — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The macOS hold is aimed at a real gap, but its by-default 1800 s "never started" ending has no recorded decision, and the [RESOURCES] line widening is outside the Goal.

Not justified as shipped

  • Item 2 — inherited: on macOS, with the default floor of 2.0, a root spawn the user asked for is now ended without running. That is a new default outcome; floor waits never expired before. Its only support is PR comments; rfc-status.txt lists no RFC on spawn admission or pressure (rfc-solo-spawn-gate.md covers something else).
    Clears when: an RFC covering the pressure-hold expiry is merged non-draft on base (then push or rebase), or /ai-review override first-principles 97e31197db58314db244ba76c36ad870b7994c6a: <decision>.
  • Item 5 — oversized: the Goal is to gate starts. "On macOS the line also fires while the kernel reports memory pressure" changes what the model sees. Held starts wait whatever the model is told.
    Subtraction: drop the memory_pressure_held branches from the [RESOURCES] line in resource_status.py, turn.py and the resource_pressure_gb help text.
    Clears when: the branch is removed, or a human amends the Goal.

What this change ships

Inventory (9 items) — 7 justified

Intent: stop new subagent starts pushing a Mac already under kernel memory pressure further into swap — FIX.

  1. A macOS root spawn waits, labelled "memory pressure", while the kernel reads WARN or worse and a dedicated runtime of ours runs — justified
  2. A start held 1800 s, or held after the episode passes that long, ends "never started" — inherited
  3. Pre-warm skips a new runtime while the hold applies — justified
  4. An unknown agent is refused before any memory wait; validation runs off the loop — rides along
  5. The [RESOURCES] advisory line and the "heavy work is fine" guidance change under macOS pressure — oversized
  6. The level is shown in resource_status and diagnostics — justified
  7. A held App Kit spawn keeps auto-approval after a window refill — rides along
  8. A channel spawn reply says "not started: " instead of "Spawned" — rides along
  9. An unreadable memory figure on macOS now logs a WARNING — undeclared

[FIRST-PRINCIPLES-REVIEWED] 97e3119

Description read

[DESCRIPTION-READ] 084381e362406f73382c01ab4696f0ba0fe539492ae0df8a4428d8bf420e7acc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — 🟡 CONCERNS

UX-level review of 97e31197db58314db244ba76c36ad870b7994c6a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

The queued copy is a three-clause run-on, and its bound, "starts give up once the pressure has lasted 30 minutes", confused the blind reader.

Watch

  • Queued text buries the deadline. The memory_pressure string chains three clauses with semicolons. The blind reader understood the wait as "only a guess" and said "'starts give up' reads oddly; I'm not told what I should do while I wait." This shows on every held wave on a Mac under pressure, and it causes friction rather than failure. Fix: shorten it to two plain sentences that put the outcome first.
  • The reason is stated three times. The card notice, the red line below it and each Subagents-panel row (never started: waiting for memory (...)) all explain the memory problem. The reader noted "the memory explanation appears three times." Each repeat pushes the one action, "Retry failed", further from view, on every ended wave. Fix: keep the full explanation in the card's ErrorNotice and cut the panel row down to its short form.
  • A mixed wave reads as an alarm. Shot-05 shows a red "!" header icon over "2 of 3 agents ran". The reader said "the red alarm icon feels strong when most of it worked." Fix: use the neutral or partial header icon when done > 0, and leave red to the "1" chip and the notice.

Suggestions

  • memory_pressure: "Waiting for macOS memory pressure to ease. Starts resume on their own, or give up after 30 minutes."
  • never_started_memory_pressure: start with the action, e.g. "{{count}} never started (macOS memory pressure). Retry from the Subagents panel once it eases."

[UX-REVIEWED] 97e3119

Description read

[DESCRIPTION-READ] 4614ddc5ff77d232301a865b172fdb7887b9da345919e786252bcbdac0133fb5

That sha256 names everything this verdict read: the PR description, plus the 9 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 97e31197db58314db244ba76c36ad870b7994c6a — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 97e3119

Verdict parsed from the review's SHA-scoped output markers for commit 97e31197db58314db244ba76c36ad870b7994c6a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 97e31197db58314db244ba76c36ad870b7994c6a: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the fix/macos-pressure-veto branch from 3f47aaf to 6785435 Compare October 1, 2026 13:54
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=a61c5a0e98fc fixed: the pressure note no longer promises a queue the gate may not hold

Outcome: fixed in 6785435.
PRESSURE_QUEUE_NOTE now reads "new dedicated subagents may queue". The tool guidance, summary_lines and the [RESOURCES] clause all render it from that one constant (resource_status.py).
"May" is exact: the gate holds only while the floor is on, a dedicated runtime of ours runs, and the start is not a nested child (gate.py). The report reads the probe, which does not know those gate-side conditions, so it cannot say "will".
Pinned by the three test_resource_status.py assertions on the new wording.

@bolichen97

bolichen97 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=e20707424d7b fixed: a nested child is never held behind the parent waiting on it

Outcome: fixed in 6785435.
The hold now also requires not _is_child. That is the same taskq_parent_id_for(parent_session_key) test the child reserve uses, computed once before the hold and reused there (gate.py).
A child's parent is a live dedicated runtime of ours waiting on it. Holding the child would hold the parent on an episode only the child can end.
Pinned by test_admission_gate.py::TestSpawnAdmissionGate::test_the_hold_lets_a_start_through[nested-child]. A mutation dropping the condition turns it red.
Spec: subagent.md lists it as the fifth hold condition.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: no WARN episode was observed on a real Mac for this PR

Outcome: rebutted (the premise): the screenshots come from the real built SPA with /api and the websocket stubbed. The gate's memory_pressure frame was pushed by hand; no Mac and no gateway were involved. The description's Manual verification says "Not run on a real Mac".
The sysctl values come from XNU source: sysctl_memorystatus_vm_pressure_level answers convert_internal_pressure_level_to_dispatch_level, i.e. NOTE_MEMORYSTATUS_PRESSURE_* = 1/2/4. The tests drive the ctypes call through a real function pointer.
The WARN reading cited (2.59-2.77 GB) and the NORMAL-while-swapping reading are prior field reports (#15270, #15244), not this PR's runs.
A real-Mac check is listed as still worth doing: sysctl kern.memorystatus_vm_pressure_level, then memory_pressure -S -l warn.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: merge order is stated and held by the maintainer, not by code

Outcome: rebutted (no change): the description states that this PR merges after the 2 GiB floor PR (fix/subagent-2gib-floor), as its safety net.
At today's 4.0 GB floor the hold rarely decides an outcome, and says so. It is still correct there: it only ever holds a start the kernel rates WARN or worse while a dedicated runtime of ours runs. So landing it first would be safe, just low-value.
The PR carries no code dependency on the floor PR. Ordering is a merge decision for the maintainer.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 1, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=6a3a78cb8f4d fixed: a row parked at the spawn approval no longer counts as a runtime of ours

Outcome: fixed in 7d47a0a.
_owns_dedicated_runtime (subagent.py) now skips a live row with _awaiting_approval and _exec_started is None. That is the pre-execution spawn-approval park, which has no process yet. A row on a mid-run tool prompt has a live runtime and still counts.
The shared _live_dedicated and the warming reserve are unchanged: only the hold's "is a runtime of ours running" question excludes the park.
Pinned by test_admission_gate.py::TestSpawnAdmissionGate::test_an_approval_wait_counts_only_with_a_runtime, two cases. Reverting the predicate turns the parked case red.

@bolichen97
bolichen97 force-pushed the fix/macos-pressure-veto branch from 6785435 to 7d47a0a Compare October 1, 2026 14:40
@bolichen97

bolichen97 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=8f01e917c86d fixed: the owning spec now names four deferred kinds and the pressure SEL outcomes

Outcome: fixed in 7d47a0a.
subagent.md: POST /api/spawn answers "the four DEFERRED kinds (DEFERRED_QUEUED_REASONS)". The "Memory pressure defers" SEL list now includes deferred_memory_pressure / refused_memory_pressure beside the low-memory and memory-critical outcomes.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=ba4e0217c34c needs-a-decision: merge this only after the 2 GiB floor PR

Outcome: needs-a-decision. The item's own clear condition, "the 2 GiB floor PR merges first", is a merge-order decision, not a code change, so it is put to the maintainer: please merge #15890 (fix/subagent-2gib-floor) before this PR.
The description states the order and concedes the low value at today's 4.0 GB floor. The hold stays correct there: it only holds a start the kernel rates WARN or worse while a dedicated runtime of ours runs, and never a nested child.
Unbounded wait is a declared Not-goal; the queued-spawn maximum wait will bound this hold too.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: the screenshots are from the stubbed SPA, not a real Mac

Outcome: rebutted (the premise): no Mac run exists for this PR. The four images render the real built SPA with /api and the websocket stubbed; the gate's memory_pressure frame was pushed by hand. Manual verification says "Not run on a real Mac" and lists the two checks still worth doing (sysctl kern.memorystatus_vm_pressure_level, then memory_pressure -S -l warn).
The values are cited to XNU source (convert_internal_pressure_level_to_dispatch_level, NOTE_MEMORYSTATUS_PRESSURE_* = 1/2/4). The ctypes call runs through a real function pointer in the tests.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: "memory pressure" is the term macOS itself shows the user

Outcome: rebutted (no change): "Memory Pressure" is the label of Activity Monitor's own memory graph, the one place a Mac user meets this signal, so the sentence uses the platform's words rather than inventing ours.
The sentence also states the consequence and that nothing is needed: "starts resume on their own once it eases". The blind reader understood the behaviour correctly.
Rewording would change 12 catalogs for an advisory PASS item.

@bolichen97
bolichen97 force-pushed the fix/macos-pressure-veto branch from 7d47a0a to 77934d1 Compare October 1, 2026 15:39
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=148aabcee2e1 rebutted: zeroing prewarm is the hold's other half, not a ride-along

Outcome: rebutted (kept). A prewarm session is a speculative dedicated kiro-cli runtime, exactly what the hold refuses to start for user-requested work at WARN. Without this, the gateway would hold the user's subagents while starting idle runtimes of its own into the same pressure.
It reads the same probe level, is declared in the description, and is pinned by test_prewarm_holds_nothing_under_pressure. Cron, runner and the adaptive controller are left untouched, so the scope stays the starts this gateway can choose not to make.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=8ba58aa868ef rebutted: the child exemption stays whole, because the gate cannot know a parent will not wait

Outcome: rebutted (no change). Narrowing the exemption to "children of a parent blocked in spawn_sub_agents" would bring back the stall fixed last round.
A spawn_run parent can wait on its child at any later point (wait, spawn_status polling, a dependency park). Whether it will is unknowable at admission, so a held child whose parent waits later holds that parent on an episode only the child can end.
Cost of keeping it: while WARN lasts, a running subagent's own nested children still start, and they often share that parent's runtime. Root starts, which are the bulk of new dedicated runtimes, stay held.
Pinned by test_the_hold_lets_a_start_through[nested-child].

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=8ba58aa868ef fixed: the held-start copy no longer says the pressure must ease

Outcome: fixed in 77934d1. Detail: "macOS reports memory pressure; waiting until it eases or a running subagent finishes". Refusal: "spawn refused: macOS reports memory pressure; retry later" (gate.py).
The chip drops "once it eases" in all 12 catalogs, and the pseudolocale is regenerated: "macOS reports memory pressure; starts resume on their own". That is true on both release paths. The screenshots are recaptured.
Pinned by TestSpawnAdmissionGate._PRESSURE_DETAIL / _PRESSURE_REFUSAL and ActivityViewer.test.tsx. subagent.md names both ways the hold ends.

@bolichen97

bolichen97 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=744ce9a19522 fixed: both [RESOURCES] trigger rows name the kernel pressure trigger

Outcome: fixed in 77934d1. docs/system-specs/common/injected-messages.md: the row adds "or the macOS kernel reports memory pressure of WARN or worse (ResourceStatus.memory_pressure_held) while the figure reads ample". docs/architecture/context-management.md: the matching row adds "or a macOS kernel memory-pressure level of WARN or worse".

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: the release path on a real Mac is not verified here; it is the gate's ordinary re-check

Outcome: rebutted (the premise): no Mac run exists for this PR. Manual verification says so and lists memory_pressure -S -l warn, then back to normal, as the check still worth doing.
Release needs no new mechanism. A held row is a durable deferral re-checked every admit_wait_secs by the same pump that releases low-memory deferrals. Each re-check reads a fresh cached probe (5 s TTL), so a level back below WARN lets the next pass admit it. test_the_drain_re_check_parks_the_pressure_label drives that re-check.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: a wrong value premise is no longer silent

Outcome: fixed in 77934d1. On macOS an unreadable or unexpected level (None) now logs one WARNING per process: "macOS kernel memory-pressure level is unreadable (kern.memorystatus_vm_pressure_level); the subagent pressure hold is off" (resource_status._read_memory_pressure_level). Pinned by test_an_unreadable_level_on_macos_is_reported_once and test_an_unknown_level_off_macos_says_nothing.
The values themselves are cited to XNU source (NOTE_MEMORYSTATUS_PRESSURE_* = 1/2/4). A real-Mac read is still listed under Manual verification.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • answered: merge order stays as described

Outcome: acknowledged (no change). #15890 (fix/subagent-2gib-floor) merges first. The predicted conflicts are one line each, in src/kiro_crew/docs/configuration.md (the spawn_min_memory_gb row) and possibly src/kiro_crew/config/sections.py (the help text two lines below the default). They are resolved by keeping both edits when this PR is rebased onto the merged floor.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=9454427c9371 rebutted: the chronic-episode immediate end is not licensed

The clears-when condition is met: the owner's decision covers it explicitly, posted on this PR (#15876 (comment)): "Once an episode has held starts for the full bound without easing, new root starts end at once until the pressure level eases." The SEL row records expired_by=episode, so these ends stay distinguishable in the audit.

On macOS the memory floor now has a second input: the kernel's
kern.memorystatus_vm_pressure_level. At WARN or worse a root start waits
while a dedicated runtime of this gateway is running, even when the Mach
reclaimable figure clears the floor. The level is the kernel's lagging
verdict (it has read NORMAL with 4.9 GB compressed and 4.8/6.0 GB of
swap in use), so it backs the figure up; it does not replace it.

- platform_compat.memory_pressure_level() reads the sysctl through
  sysctlbyname on the cached libc handle, now built once under a lock.
  None off macOS and on any failure, and None fails open.
- The gate reads the level fresh (resource_status
  .read_memory_pressure_level). The hold is off with the floor (0),
  never applies to a nested child or a claim re-entry, and only counts
  dedicated-priced runtimes of ours, so a shared wave never holds
  itself. A start priced shared is held too: it still launches the
  agent's MCP servers.
- It is a capacity-style wait, not a store deferral: the row stays in
  the window as memory_pressure (restricted rows too), the pump pick
  passes over held roots, every slot release and a 15 s timer re-check
  it. Past 1800 s a row is ended, never started ("never started: waiting
  for memory"), and an episode that outlives that bound ends new starts
  at once until it eases. The same rule is re-checked at approval
  release (per entry, so a held root never blocks a child) and at the
  shared-to-dedicated top-up.
- A critical posture wins over the hold, and agent validation now runs
  before the memory gates, so an unknown agent is refused at once; the
  event-loop callers run its directory scan off the loop.
- An unreadable Mach figure is reported on macOS as on Linux
  (memory_check_unavailable). The level is shown in resource_status,
  the [RESOURCES] line and the diagnostics bundle; the eager-spawn
  admission takes no pre-warm while the hold applies.
Three defects on the paths a held or deferred start now takes
routinely. Each predates the pressure hold.

- A window refill rebuilds an entry from the store, which never carries
  approval_mode, so a held or deferred App Kit spawn (approval_mode
  "auto") drained into a spawn prompt nobody can answer. The accepting
  process now keeps that row's mode in _held_approval_modes and the
  refill restores it; a restart still replays the row without it.
- finish_parked_defer announced a refusal on TaskStoreUnavailable while
  the row stayed QUEUED and due, so one id could get a refusal and a
  later completion. It now answers still-queued and re-pumps.
- The channel spawn reply said "Spawned" for a refused terminal record.
  It now says the subagent was not started, and why.
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=3a9e3192314e fixed: "retry once it eases" gave no signal to act on

Fixed in 1022a31: the notice now names where the level is shown: "...once it eases (Activity Monitor’s Memory tab shows it) or a running agent finishes, use “Retry failed” in the Subagents panel". All 12 catalogs name Activity Monitor's Memory tab, which graphs the same kernel level the hold reads.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=16767fcb2ac9 fixed: a mixed wave's header read as finished

Fixed in 1022a31: a settled wave where only some members never started now reads "2 of 3 agents ran" (new plural key agent_ran_of_total in all 12 catalogs plus the plural registry), with the counted notice below it. The mixed-wave test asserts the new header and that "3 agents finished" is gone; the PR body's screenshot shows it.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=92cb1acf7440 rebutted: cut the queued sentence and replace "memory pressure"

Kept: "memory pressure" is the name macOS itself gives this level, in Activity Monitor, which the never-started notice now points to. Renaming it would leave the user with no term to look up, and the earlier review round asked for the bound to stay in the sentence.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=0c5ed6bbc5f3 rebutted: word the card notice and the panel error the same

Kept, deliberately. The panel row is the run's own record and opens with the owner's fixed words, "never started: waiting for memory". The card notice summarizes the wave: it is counted, and it names the Retry failed control and Activity Monitor. Both state the same cause, "macOS memory pressure did not ease in time", word for word.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=5679a13c8b37 rebutted: ongoing foreign pressure stops root spawns

The owner's decision (#15876 (comment)) covers this exactly: once an episode has held starts for the full bound, new root starts end at once until the level eases. The hold applies only while a dedicated runtime of ours runs (maintainer ruling, comment 5963482796), so ending that runtime releases it without touching the floor.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=20265f68d33f rebutted: the cause: every root start held, then ended

Same owner decision (#15876 (comment)). The wording quoted is the rule as signed off.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=5ed4c52ae037 rebutted: the hold cannot tell its own load from foreign load

Stated in the spec on purpose and accepted by the maintainer (comment 5963482796): with nothing of ours running the hold never applies, so foreign pressure alone never starves our first start.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=6d6f288ec5d6 rebutted: the only off switch also removes the floor

Turning off the hold without the floor is a new knob this PR was asked not to add (a review round earlier: a config key main does not have). The way out without config is ending our own dedicated runtime; the never-started notice names Retry failed for afterwards.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=ad3d8e954ccb rebutted: the bound is a hardcoded constant until the max-wait key lands

Agreed and planned: the agent.subagent_queue_max_wait_secs PR re-points the one constant, whichever of the two lands second. The owner accepted the fixed constant in the meantime (#15876 (comment)).

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=fd077f7d8563 rebutted: split validation order, AgentCheck and the commit-2 fixes out

Each is needed by the hold: an unknown agent must not wait 30 minutes in the hold before failing; the pump's re-validation on the event loop drew an upheld blocking finding without AgentCheck; and the commit-2 defects are hit routinely by held rows. They sit in their own commit, so commit 2 reverts alone, within the repo's two-commit limit.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=a612047379f3 rebutted: the episode end refuses a start the Goal says waits

The clears-when is met: the owner decision is now linked in the description (#15876 (comment)). Failing open after the bound was weighed there too, and rejected, because the start would launch into the pressure it waited on; the description says so.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=ca9f1e7a6a3f rebutted: AgentCheck names no measured loop stall

The repo rule outranks a measurement: GPT's finding on this PR was upheld under the blocking rule no-blocking-call-on-event-loop (AUTOSDE.yaml names filesystem walks), because the reordered re-validation runs from the coroutine pump. Deleting AgentCheck reinstates that upheld blocking finding.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=992e19e84b27 rebutted: the queued sentence is three clauses with jargon first

Kept: "memory pressure" is macOS's own name for this level (Activity Monitor uses it), the clause "resume on their own" is the reassurance the reader asked for, and an earlier round asked for the bound to stay in the sentence. Rephrasing it again round by round churns 12 catalogs without a settled target.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=59ac536afe28 rebutted: no in-product signal for when it is safe to retry

A live pressure indicator in the dashboard is a new surface, not a fix to this one; Activity Monitor is where macOS shows the same level the hold reads. Retrying early costs nothing more than another bounded wait.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=14243c572ae6 rebutted: a mixed wave wears the red failure icon

Pre-existing and shared: the card shows the failure icon whenever any member failed, for every failure kind, and this PR does not change that rule. A partial-outcome icon is a card-wide design change for every mixed wave, not one this PR should make for its one failure kind.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=b5cd9d8a8df0 rebutted: reword memory_pressure as short on memory

"Short on memory" would be inaccurate: the hold applies when the free-memory figure is above the floor and only the kernel's pressure level is high. Naming the level as macOS names it is what lets the user find it in Activity Monitor.

@bolichen97

Copy link
Copy Markdown
Contributor Author
  • span=c55c029dad83 rebutted: reword the never-started notice

Kept for the same reason as the queued sentence: "short on memory" misstates what was measured, and the notice must keep naming the Retry failed control, which an earlier round asked for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants