Skip to content

fix(acp): judge the macOS runtime ceilings by phys_footprint, not ps RSS - #15766

Merged
bolichen97 merged 1 commit into
release/0.8.0from
backport/0.8.0-macos-phys-footprint
Oct 1, 2026
Merged

bolichen97 merged 1 commit into
release/0.8.0from
backport/0.8.0-macos-phys-footprint

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: on macOS, the background runtime's memory ceiling judges the memory the OS actually charges the runtime, so a runtime that has grown past it is recycled.

On macOS the 500 MiB ceiling (AcpRuntime._is_stale) never fires (#15757). From one 16 GB Mac:

  • the gateway pid read 124 MB through ps RSS against 1983 MB of phys_footprint (16x);
  • two idle background runtimes held 462 MB of footprint for hours;
  • there were nine jetsam kills in eight days.

Why it matters

The ceiling is the only bound on a background runtime that keeps getting reused, and on macOS it is effectively disabled. Memory the runtime holds compressed or swapped is invisible to it, and that is exactly the memory jetsam counts when it kills processes on a full Mac. The Sessions panel's macOS memory figure, which reads the same tree sum, under-reports the same way.

Not a goal

  • Reaping a zero-session background runtime after a fixed idle time, regardless of size. test_bg_runtime_idle_recycle.py::test_a_fresh_idle_runtime_is_left_alone pins the opposite ("idle is not a reason to recycle, staleness is"). Changing that is a maintainer decision, recorded on macOS: background runtime memory ceiling reads ps RSS, which omits compressed pages, so it never fires #15757.
  • Changing the 500 MiB / 6 h ceilings or their defaults.
  • The session watchdog's per-session RSS recycle, which is Linux-only (get_session_rss_mb returns 0 on macOS).
  • Linux and Windows measurement, which is unchanged.

What changed (motivation → approach → change)

Symptom: a macOS runtime holding gigabytes is never judged stale.

Root cause: on darwin, _get_rss_tree_mb in src/kiro_crew/acp/runtime.py summed ps -Ao pid=,ppid=,rss=, and _get_rss_mb read ps -o rss=.

  • ps RSS counts only resident pages. macOS compresses and swaps idle memory aggressively, which is what an idle grown runtime mostly consists of.
  • jetsam and Activity Monitor's "Memory" column read phys_footprint, which includes those pages.

Change:

  • Footprint reader. src/kiro_crew/platform_compat.py gains proc_phys_footprint_bytes_for_pid(pid), which reads ri_phys_footprint via proc_pid_rusage(pid, RUSAGE_INFO_V2). It needs no subprocess and no entitlement for a same-uid pid, and it answers None off macOS or when unreadable. The struct layout (16-byte uuid, then 18 uint64s, footprint the 8th) is pinned by a test. proc_pid_rusage is configured separately on the shared libproc handle, so a libproc without it still serves every proc_pidinfo probe.
  • Tree sum. On macOS the tree sum measures each pid by its footprint, falling back to that pid's ps RSS only when the footprint cannot be read. The ps snapshot still provides the tree shape in one memoized walk, and the depth bound is unchanged. _get_rss_mb prefers the footprint too.
  • Specs. docs/system-specs/common/platform-compat.md gets a new helper row; docs/system-specs/modules/session.md documents the staleness measurement.

Backwards compatibility

Compatible: same function signatures and units (MiB). On macOS the figure is now the footprint, so the ceiling can fire where it could not before. That is the intended fix, and a runtime with sessions is still only parked, never killed under a co-tenant.

Tests

test/test_macos_phys_footprint.py (13 tests):

  • Reader:
    • reads the footprint field and not the resident one;
    • a failed call, and a missing libproc, both read as None;
    • a libproc without proc_pid_rusage still serves the shared handle;
    • the reader is macOS-only;
    • the struct layout is pinned.
  • Tree sum:
    • it sums footprints;
    • a pid with no readable footprint falls back to its RSS;
    • with no footprint at all it equals the old RSS sum;
    • the depth bound still holds;
    • the single-pid read prefers the footprint.
  • Ceiling: 1983 MiB of footprint behind 120 MiB of RSS makes _is_stale() return "rss", and a small footprint does not.

Existing fixtures that pin the ps path now pin the footprint off. Otherwise a real Mac would answer for a same-uid pid the fake table names. On this branch the only such fixture is in test/test_macos_process_table.py.

Regression proof: with runtime_process_tree.py reverted and the tests kept, 5 fail. The related files (session memory, bg-runtime recycle, platform_compat, process tree, facade, spawn audit) pass: 1180 passed.

Manual verification

Not run on a real Mac: this host is Linux, so the libproc call is exercised through a fake library that fills a real ctypes buffer, which is the established pattern for the other _darwin_* probes. A reviewer on macOS can compare python3 -c "from kiro_crew import platform_compat as p, os; print(p.proc_phys_footprint_bytes_for_pid(os.getpid()))" with Activity Monitor's Memory column for that python process.

Gates:

Related Issues

Backport of #15760 to release/0.8.0. Refs #15757.

Conflict resolved: on this branch the process-tree helpers still live in src/kiro_crew/acp/runtime.py (acp/runtime_process_tree.py and the refactor facade tests arrived on main later). So the same two hunks (_get_rss_mb and the macOS tree sum) are applied there, and test/test_acp_refactor_* are not carried. The new tests, together with the macOS process table, platform_compat, bg-runtime recycle, session-memory and spawn-audit suites, pass here (877). test/test_acp_runtime.py has 3 reds under -n 6 that fail identically on pristine release/0.8.0 @ f8d6039 and pass serially.

Pattern harvest

Rule candidate: review-prompt
Pattern: a macOS memory ceiling that reads ps RSS; on macOS the figure the OS acts on is phys_footprint, and RSS can understate it by an order of magnitude.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Oct 1, 2026
On macOS the background runtime's 500 MiB ceiling never fired. The tree
sum behind AcpRuntime._is_stale read `ps -Ao pid=,ppid=,rss=`, and ps
RSS counts only pages resident right now. macOS compresses and swaps
idle memory aggressively, and an idle runtime that has grown consists
mostly of exactly those pages. One Mac measured the gateway at 124 MB
of RSS against 1983 MB of footprint (16x), with two idle runtimes
holding 462 MB of footprint for hours.

Add platform_compat.proc_phys_footprint_bytes_for_pid. It reads
ri_phys_footprint through proc_pid_rusage(RUSAGE_INFO_V2): no
subprocess, no entitlement for a same-uid pid. It is configured
separately on the shared libproc handle, so a libproc without that
symbol still serves the proc_pidinfo probes. On macOS the tree sum, and
the single-pid read, measure each pid by its footprint, falling back to
that pid's ps RSS only when the footprint cannot be read. The ps
snapshot still provides the tree shape, in one walk.

Backport of #15760 to release/0.8.0.
@bolichen97
bolichen97 force-pushed the backport/0.8.0-macos-phys-footprint branch from ad8c296 to b562e2c Compare October 1, 2026 08:18
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of b562e2c085bb26c3e108ede90110ff95ea460e77 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

The rusage_info_v2 transcription was never run on a real Mac — trigger the ci:macos lane so TestTheRealDarwinAbi actually executes before merge.

What this change ships

Inventory (7 items) — 7 justified

Intent: FIX — on macOS the background runtime's 500 MiB ceiling never fires because ps RSS omits compressed/swapped pages (#15757, measured 16x understatement).

  1. The macOS runtime ceiling now judges phys_footprint, so a grown runtime is recycled — justified
  2. The Sessions panel's macOS memory figure now matches Activity Monitor — justified
  3. Single-pid macOS read (_get_rss_mb) prefers the footprint too — justified
  4. New platform_compat.proc_phys_footprint_bytes_for_pid helper — justified
  5. Per-pid fallback to ps RSS when the footprint is unreadable — justified
  6. Spec rows updated in the same commit — justified
  7. Existing macOS process-table fixture pins the footprint off — justified

Counts run: proc_phys_footprint_bytes_for_pid has 2 real consumers (acp/runtime.py:1295, :1569). Sibling grep for macOS ps -o rss= readers: cli_doctor.py:1258 reports a figure explicitly labelled RSS, and _macos_current_rss_bytes (platform_compat.py:8790) records footprint-vs-RSS as "a separate accounting question" in its own comment — both are prior decisions, not unfixed siblings. The ceiling itself and its 500 MiB/6 h values are unchanged, so this restores behaviour the issue already pins, not product shape.

[FIRST-PRINCIPLES-REVIEWED] b562e2c

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for b562e2c085bb26c3e108ede90110ff95ea460e77; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b562e2c085bb26c3e108ede90110ff95ea460e77: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🔴 BLOCK (blocking)

Design-level review of b562e2c085bb26c3e108ede90110ff95ea460e77 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: BLOCK

A darwin-only ABI read ships to release/0.8.0 unverified: no macOS lane runs on release-branch PRs, and no real Mac ran it.

Blockers

Hand-pinned rusage_info_v2 offsets reach the release branch with zero real-Darwin execution.
The description says "Not run on a real Mac: this host is Linux", and every unit test here, by the test file's own admission, "hands the reader a buffer built from the SAME offset constants it reads with, so none of them can notice the constants disagreeing with the kernel's struct". The one real-ABI check, TestTheRealDarwinAbi, is skipif(sys.platform != "darwin"), and every macOS CI lane (macos-on-demand.yml → pull_request: branches: [main]; ci.yml's macos-15 leg "on the push-to-main path only"; nightly on schedule) is out of reach of a release/0.8.0 PR — and the original #15760, whose main-targeted CI would run the darwin lane on these same hunks, is still open and unverified. If an offset or call convention is wrong on a shipped macOS version, the reader returns garbage: a garbage-high read fires _is_stale() → "rss" on every check and recycles the background runtime continuously on every macOS install of the 0.8.0 release; a garbage-low read silently reproduces the bug this PR claims to fix while the Sessions panel shows the wrong figure — and no lane the release branch runs can catch either.
Fix: get one real-Darwin execution of these hunks on record before merge.
Clears when: #15760's macOS on-demand lane is green on the identical hunks and is cited on this PR, or a maintainer posts the output of the description's own one-liner (proc_phys_footprint_bytes_for_pid(os.getpid()) vs Activity Monitor) from a real Mac.

[DESIGN-REVIEWED] b562e2c

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed b562e2c085bb26c3e108ede90110ff95ea460e77 — this comment is updated in place on each push.

Review details

No findings worth blocking; one advisory below.

FINDING — docs/system-specs/common/platform-compat.md:71 — the new row names `acp/runtime_process_tree` as the footprint's consumer, but no such module exists (grep -rn runtime_process_tree matches only this line); the real callers are acp/runtime.py::_get_rss_tree_mb / _get_rss_mb, so the one doc AGENTS.md makes mandatory before a POSIX change sends the next reader to a nonexistent file → Fix: replace acp/runtime_process_tree with acp/runtime.py's _get_rss_tree_mb (and _get_rss_mb).

[OPUS-REVIEWED] b562e2c

Verdict parsed from the review's SHA-scoped output markers for commit b562e2c085bb26c3e108ede90110ff95ea460e77.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b562e2c085bb26c3e108ede90110ff95ea460e77: <one-sentence reason>

@bolichen97

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt b562e2c: maintainer decision (bolichen): GPT lane incomplete (runner/quota); the backport is byte-identical to #15760, reviewed there.

@bolichen97

Copy link
Copy Markdown
Contributor Author

/ai-review override design b562e2c: maintainer decision (bolichen): accept. Release-branch PRs have no macOS lane, but the darwin ABI read is byte-identical to #15760, whose ci:macos run executed TestTheRealDarwinAbi green on a real Mac; the insider release run builds and signs macOS artifacts before any promotion.

@bolichen97
bolichen97 marked this pull request as ready for review October 1, 2026 21:20
@bolichen97
bolichen97 requested a review from a team as a code owner October 1, 2026 21:20
@bolichen97
bolichen97 requested review from iamwhatever and removed request for a team October 1, 2026 21:20
@bolichen97
bolichen97 merged commit ddaede0 into release/0.8.0 Oct 1, 2026
20 of 29 checks passed
@bolichen97
bolichen97 deleted the backport/0.8.0-macos-phys-footprint branch October 1, 2026 21:20
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for b562e2c085bb26c3e108ede90110ff95ea460e77.

maintainer decision (bolichen): GPT lane incomplete (runner/quota); the backport is byte-identical to #15760, reviewed there.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for b562e2c085bb26c3e108ede90110ff95ea460e77 manually from the Actions tab.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 marked the design AI finding as false positive, not applicable, or explicitly accepted for b562e2c085bb26c3e108ede90110ff95ea460e77.

maintainer decision (bolichen): accept. Release-branch PRs have no macOS lane, but the darwin ABI read is byte-identical to #15760, whose ci:macos run executed TestTheRealDarwinAbi green on a real Mac; the insider release run builds and signs macOS artifacts before any promotion.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: Design Review. Re-run the lane's latest workflow run for b562e2c085bb26c3e108ede90110ff95ea460e77 manually from the Actions tab.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant