Skip to content

fix: show unknown memory headroom on an unreadable cgroup wait - #21126

Merged
chenmingwei23 merged 1 commit into
mainfrom
fix/15956-memory-headroom-unknown
Oct 16, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
fix/15956-memory-headroom-unknown

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: When a subagent waits because its cgroup memory usage can't be read, the dashboard's queued chip says free memory is unknown instead of "0 GB free now".

A spawn deferred on a finite cgroup memory limit whose usage file (memory.current / memory.usage_in_bytes) is unreadable has unknown headroom. PR #15929 named that in the deferral reason, refusal text, gateway warning and audit row, but the chip rebuilds its line from the structured subagent_queued fields. The probe contributes 0.0 for that level, so the chip read "needs 4.5 GB of free memory, 0 GB free now".

Why it matters

The chip tells the person to "free up memory" and reports a figure that was never measured. On a host with plenty of memory but a locked-down cgroup, that sends them after the wrong fix. The real remedy, restoring read access, only showed in the gateway log.

Not a goal

What changed (motivation → approach → change)

Symptom: "0 GB free now" on an unknown-headroom wait. Root cause: the gate's wait= label for low_memory included available_gb whenever the reading was >= 0, and an unreadable cgroup usage reads as 0.0. The cause from pop_memory_check_cause() never reached the label.

  • src/kiro_crew/subagent_manager/admission/gate.py: when the memory cause is cgroup_usage_unreadable, the low_memory wait carries cause: "cgroup_usage_unreadable" and no available_gb. required_gb stays.
  • website/src/pages/chat/subagentQueuedReason.ts: parses cause (kept only for low_memory and only that value). queuedWaitText renders the unknown-headroom sentence ahead of any figures, with the same three wait-bound tails as the other memory sentences. The sentence says which file can't be read and who needs access: "the gateway can't read its cgroup memory usage (memory.current / memory.usage_in_bytes); give the gateway read access to that file to continue". The deferred start resumes by itself once the file is readable, because the admit-wait re-check and the fit pass re-read the host on their own.
  • New keys pages.chat.subagentQueued.low_memory_unknown{,_limit,_no_limit} in en.json and all 11 translated catalogs, plus a regenerated en-XA.json.
  • docs/system-specs/modules/subagent.md and the _emit_queue_depth_impl docstring now describe the new field.
  • website/scripts/capture-15956-memory-headroom-unknown.mjs: an asserting screenshot harness, like the existing capture-parked-subagent-chip.mjs.

Backwards compatibility

Compatible: cause is an additive optional field. An older frontend ignores it and shows the figure-less low_memory_no_figures sentence, because available_gb is now absent. That is still more accurate than "0 GB free". A newer frontend on an older gateway gets no cause and behaves as before.

Removes nothing: the low_memory label only stops carrying an available_gb that was never measured. Readers of _queue_wait were checked on origin/main: gate_pressure.py reads only reason, run.py spreads the dict into the subagent_queued frame, and the frontend's parseSubagentQueuedReason treats available_gb as optional.

Tests

  • test/test_spawn_memory_cause.py::test_unreadable_usage_wait_label_names_the_cause_not_zero_available: real cgroup fixture with the usage file unreadable. The wait label carries the cause and required_gb, and has no available_gb.
  • test/test_spawn_memory_cause.py::test_spawn_deferral_for_measured_low_memory_is_unchanged: now also checks that a measured wait keeps available_gb == 1.0 and gets no cause.
  • website/src/test/subagentQueuedReason.unknownHeadroom.test.ts: covers parsing (cause kept, unknown causes and non-low_memory kinds dropped), the unknown sentence for each wait-bound tail with no "free now" or "0 GB", and an unchanged measured sentence.
  • website/src/test/SubagentProgressBar.test.tsx: the rendered chip line says "free memory is unknown" for the unreadable-usage frame.

Run locally:

  • pytest test/test_spawn_memory_cause.py test/test_spawn_floor_defaults.py::test_only_a_read_that_answered_unmeasurable_proceeds_unchecked: 14 passed.
  • vitest on the two files above plus subagentActivity, ActivityViewer, SubagentRunCard, chatSlice.publicSurface, UseWebSocketCoverage.activity and ChatSidebar.boardStateLanes: all passed.
  • I18N_BASE_REF=origin/main vitest run src/i18n: 50 files, 1022 passed. npm run i18n:check, tsc -b, eslint: clean.
  • black, isort, flake8, mypy, comment-history, brand, harness-parity, file-line-cap, docs-lint, flake ratchet: clean.
  • scripts/local-gate.py: backend batch 1 shows 20 failures in test_acp_spawn_offload.py::TestRuntimeProjectedLaunchDocuments, test_agent_backend_recheck_e2e.py::TestRecheckOverHttp and code_review_sage/.../test_store_ancestor_chain.py. The same 20 fail with this diff reverted, on clean origin/main, on this host, so they come from the host environment. The rest of the oversized related set (3,346 backend and 1,275 frontend files, mostly because the locale catalogs are referenced everywhere) is left to CI.

Manual verification

npm run build, then node scripts/capture-15956-memory-headroom-unknown.mjs (scripted Playwright against the built SPA). It pushes real subagent_queued frames and asserts the chip text. All 4 scenes passed: dark and light, unknown headroom and measured low memory.

Screenshots / video

Unreadable cgroup usage (after):

dark, unknown headroom
light, unknown headroom

Measured low memory (unchanged):

dark, measured low memory

Related Issues

Closes #15956
Follow-up from #15929.

Pattern harvest

Not generalizable: the wait label leaked one specific sentinel (0.0 for unknown cgroup usage), and a cause field on the label now carries it. Other sentinel paths already pass a cause (host_read_unanswered) or use -1.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the new low_memory_unknown* text reads correctly in the 11 translated catalogs, since it names the file paths memory.current / memory.usage_in_bytes literally.

[DESIGN-REVIEWED] df6ba85

Description read

[DESCRIPTION-READ] 88dd6223b8c17f14f58f0288e7d5aa4be7d3bc7d0c3eabb813fa0aa1df603ff2

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Check one thing: the SEL audit row still records available_gb: 0.0 beside cause. The chip no longer shows that sentinel, but the audit row keeps it.

What this change ships

Inventory (6 items) — 6 justified

Intent: when a subagent is waiting because its cgroup memory usage file can't be read, the person should see that free memory is unknown and that read access needs restoring, not "0 GB free now". This is a FIX, closes #15956.

  1. The queued chip says free memory is unknown and names the unreadable file and the remedy, instead of "0 GB free now" — justified
  2. A low_memory subagent_queued frame drops available_gb and carries cause: cgroup_usage_unreadable when usage is unreadable — justified
  3. The unknown-headroom sentence keeps the same three wait-bound tails (limit, no limit, bare) — justified
  4. Three new i18n keys in en, en-XA and the 11 translated catalogs — justified
  5. The subagent spec and the _emit_queue_depth_impl docstring describe the new field — justified
  6. A Playwright screenshot harness capture-15956-memory-headroom-unknown.mjs, following the existing repo convention of more than 760 capture-*.mjs scripts — justified

[FIRST-PRINCIPLES-REVIEWED] df6ba85

Description read

[DESCRIPTION-READ] 737a5f5fe165d68d06c3fb8441534e69b962328238c413128ccd4285312e5815

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — 🟡 CONCERNS

UX-level review of df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

The new chip sentence gives an instruction nobody can follow: the blind reader called it "sure it's a warning, no idea what it means" and couldn't tell "which file, where it is, or how".

Watch

  • The remedy is written in implementation terms. low_memory_unknown* puts "gateway", "cgroup" and "(memory.current / memory.usage_in_bytes)" into the dashboard chip. It then says "give the gateway read access to that file", but two filenames were just listed and no path is given. The reader said: "I don't know which file, where it is, or how to do that." Unlike the measured sentence (shot-03, "much clearer… I should close other programs"), this one names a problem without a step the person can take. Severity: rare state, but every time it appears it blocks the only action on offer. Fix: lead with the task and a pointer, e.g. "waiting to start — can't check free memory (no read access to the memory usage file); see the gateway log for the path and fix". Or name the one resolved path the gateway actually tried.
  • The warning is harder to read in light mode. In shot-02 the reader said the yellow/olive text is "harder to read in light mode". The sentence is now about twice as long, which makes the low contrast worse. Severity: affects every light-theme user who sees this state. Fix: check the warning token's contrast against the light background.

Suggestions

  • low_memory_unknown_limit: "they never start if it stays unknown for {{limit}}". Replace "it" with "free memory" so the sentence doesn't depend on context.

[UX-REVIEWED] df6ba85

Description read

[DESCRIPTION-READ] 241d8173958b23f48544292702aef1ed21a94d77063a70ab118d1564de9f9067

That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] df6ba85

Verdict parsed from the review's SHA-scoped output markers for commit df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b: <one-sentence reason>

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 16, 2026
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] df6ba85

False positive or not applicable? A repository writer can comment:
/ai-review override gpt df6ba858c1f9c8ff7bcc841fa7d5466dc451d52b: <one-sentence reason>

A spawn deferred because a finite cgroup memory limit has no readable
usage file waits with unknown headroom, but the queued chip read the 0
the probe contributes as a measurement and said "0 GB free now".

The gate's low-memory wait label now carries cause
cgroup_usage_unreadable and drops available_gb in that case. The chip
parses the cause and renders a new "free memory is unknown" sentence
(three wait-bound tails, all locales). Admission is unchanged.

Closes #15956
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Pushed df6ba85 (rebased on main). It answers the UX Review concern on f419627: the unknown-headroom sentence now names the unreadable file and who needs access. It reads "the gateway can't read its cgroup memory usage (memory.current / memory.usage_in_bytes); give the gateway read access to that file to continue", updated in all 12 catalogs. On the Design Review check: the start resumes by itself once the file is readable. The fit pass and the admit-wait re-check re-read the host, and a reading without the cause clears the wait (docs/system-specs/modules/subagent.md, the fit-pass paragraph). Screenshots in the description were re-captured on this head.

@iamwhatever
iamwhatever force-pushed the fix/15956-memory-headroom-unknown branch from f419627 to df6ba85 Compare October 16, 2026 07:44
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Reply to UX Review on df6ba85. The verdict is advisory, and I'm leaving the wording to the maintainer instead of starting another edit round:

  • Implementation terms vs. naming the file: round 1 on f419627 asked the chip to name what can't be read and who needs access, and round 2 asks to drop those terms again. The two asks pull in opposite directions. The current text keeps the file names because only someone who controls the cgroup (container or systemd slice) can fix this state, and they need the exact file. A person who can't change cgroup permissions can't act on any wording. Pointing to the gateway log instead, as round 2 suggests, is a fine alternative if the maintainer prefers it.
  • "it stays unknown": in the same sentence "it" refers to "free memory is unknown". Not changed.
  • Light-theme warning contrast: that comes from the shared text-warn token, which every existing wait sentence on this chip uses. This PR doesn't change it, so it's out of scope here.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 16, 2026

@chenmingwei23 chenmingwei23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: readiness: passed, no conflicts, checks green.

@chenmingwei23
chenmingwei23 merged commit 2505c05 into main Oct 16, 2026
102 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/15956-memory-headroom-unknown branch October 16, 2026 15:40
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dashboard queued chip: show unknown cgroup memory headroom

2 participants