Skip to content

docs(rfc): record measured executor CPU, GIL holders and anon memory - #22065

Merged
iamwhatever merged 1 commit into
mainfrom
zedmor/rfc-python-rust-hybrid-measurements
Oct 20, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
zedmor/rfc-python-rust-hybrid-measurements

Conversation

@Zedmor

@Zedmor Zedmor commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Record measured answers to the three unknowns the Python+Rust hybrid RFC left open in §2.3, so the decision on how much Rust is worth its packaging cost rests on data.

rfc-python-rust-hybrid-backend.md (accepted 2026-10-13) lists three things only an owner outside the agent sandbox could measure: which functions burn the 69% executor-thread CPU, who holds the GIL during event-loop stalls, and what the 1.3-2.3 GB of unattributed anonymous memory is. §7 makes "any Rust beyond the stub" wait on them. Nobody had measured them.

Why it matters

Phase 3 adds a native package to every platform, and the maintainer asked that packaging be settled before the Rust PRs (#19627, #19628) move. Without measurements, the size of the Rust payoff is a guess. With them, follow-up work goes to the language that actually removes each cost.

Not a goal

  • Changing any phase, default, decision or status in the RFC. It stays accepted, and §1 and §4 are untouched.
  • The fixes themselves. Each is its own PR, linked from §2.4.4.
  • Re-measuring the RFC's original 48-core host.
  • Per-call-site attribution of the remaining ~1 GB Python/glibc heap, which needs a gateway restart with PYTHONTRACEMALLOC=25.

What changed (motivation → approach → change)

  • Approach. I measured a 16-core Linux host that had run upstream b4cf148542 for four days (RSS 3.08 GB + 0.70 GB swapped). Every method was read-only and needed no sudo: py-spy record/dump --nonblocking, a 10 Hz /api/health probe that dumps stacks during a stall, per-thread CPU from /proc/<pid>/task/*/stat, an allocator-header scan of anonymous memory (smaps + pagemap + /proc/<pid>/mem, present pages only), the gateway's own 98-hour diag recorder, and an embedder replay at 8fd2af91d9 that pins each llama.cpp buffer byte for byte.
  • rfc-python-rust-hybrid-backend.md:
    • A new §2.4 records the method and the three answers.
    • Executor CPU: the mount-namespace sweep (83% kernel time), the /proc/net/tcp peer-PID scan, and transcript framing.
    • GIL during stalls: the event-loop thread itself, serializing GET /api/chat/slots for 186-463 ms once a minute. No executor thread held it.
    • Anonymous memory: 62% is the embedder. A per-token logits+embedding output buffer reaches 1,193 MiB after one max-length input and is never shrunk; the KV cache and compute buffer make up the rest.
    • Transcript framing: measured directly, it runs at 77 MB/s (slower than the JSON parse after it), and 78% of that time is a per-record [\r\n] regex. bytes.find frames the same bytes at 4.9 GB/s.
    • §2.4.4 maps each finding to the language that fixes it and links the follow-up PRs. None is a compute-heavy leaf Rust would remove, so the stub stays the only measured Rust win; its saving is re-measured at 34 MB PSS per stub.
    • §2.3 gains a "Now" column pointing at the answers, plus two corrections: py-spy sees the Python thread names, and sudo is unnecessary at ptrace_scope=0.
    • The §2.1 rows and the §7 question point to §2.4. last-audited/audited-at are bumped after every code reference was re-checked at 207ed5ae4c.
  • Front matter and status line: implementation-prs now lists perf(mcp-gateway): Rust MCP stub, used by default when built #19627, the open draft that implements Phase 3's stub. The status line says nothing is on main, and that Phases 1 and 2 have no implementation PR.
  • docs/request-for-change/README.md: the index row's verified commit and the perf(mcp-gateway): Rust MCP stub, used by default when built #19627 draft, plus the dated re-audit note other rows carry.

Backwards compatibility

Compatible: documentation only.

Removes nothing: no phase, decision or reference is removed; §2.3's original commands stay as written.

Tests

N/A — documentation only. scripts/docs-lint.sh and scripts/check_brand_name.py pass.

Manual verification

  • Re-checked every code reference in the RFC at 207ed5ae4c. The §2.4 paths still run there: the slot-list endpoint still serializes on the loop; the mount sweep, the transcript framing and the vendored embed loop are unchanged since the measured build; _linux_tcp_peer_pid now lives in platform_compat_ports.py.
  • Re-ran the embedder replay on main's code. Load: 490 MiB. After one 6,000-character input: 1,869 MiB. Buffer growth is 610,772 B per token = (151,669 + 1,024) × 4 B.

Related Issues

Follow-ups opened from these measurements, each linked from §2.4.4:

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@Zedmor
Zedmor requested review from a team and KiroCrewCaptain as code owners October 19, 2026 04:34
@Zedmor
Zedmor requested a review from cruisercohen October 19, 2026 04:34
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 5416533003ade48ad145c7ac1d084fb0129a1fb4 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 5416533003ade48ad145c7ac1d084fb0129a1fb4 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Advisory — does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 5416533003ade48ad145c7ac1d084fb0129a1fb4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that §2.4.4's line "the stub stays the only measured Rust win" stays a finding and doesn't amend §1 or §7. The RFC keeps its accepted status, so changing a phase needs its own decision.

[DESIGN-REVIEWED] 5416533

Description read

[DESCRIPTION-READ] 8bdc85e27bf38298d78da2f7726bea2bdfe83b4bcc7c9001d4a98ee2dfece213

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 5416533003ade48ad145c7ac1d084fb0129a1fb4 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 5416533

Verdict parsed from the review's SHA-scoped output markers for commit 5416533003ade48ad145c7ac1d084fb0129a1fb4.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 5416533003ade48ad145c7ac1d084fb0129a1fb4: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 5416533003ade48ad145c7ac1d084fb0129a1fb4 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 5416533

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 5416533003ade48ad145c7ac1d084fb0129a1fb4: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 19, 2026
@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from c04dd30 to 551fab9 Compare October 19, 2026 05:41
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Oct 19, 2026
@Zedmor

Zedmor commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Stacked follow-up: #22120 answers the two §7 packaging questions that block Phase 3 (Windows signing of an .exe inside a wheel; macOS re-signing of a suffix-less Mach-O). It edits the same §7 table, so it is based on this branch. Merge it after this one.

@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from 551fab9 to ffd0d5b Compare October 19, 2026 08:09
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Oct 19, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 19, 2026
@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from ffd0d5b to f1475cb Compare October 19, 2026 14:20
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 19, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Oct 19, 2026
@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from 79c39a3 to a0e1008 Compare October 19, 2026 21:39
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Oct 19, 2026
@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from a0e1008 to 5943f11 Compare October 20, 2026 00:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Oct 20, 2026
@Zedmor
Zedmor force-pushed the zedmor/rfc-python-rust-hybrid-measurements branch from 5943f11 to 611fbbf Compare October 20, 2026 02:47
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 611fbbf.

The code reads as sound and I would approve it, but the branch now conflicts with main. Once that clears, a fresh look can approve it.

— crew-pr-reviewer (r114)

@Zedmor

Zedmor commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Conflict with main cleared at e3a3fae: rebased, with main's newer RFC index notes kept. Docs-only, docs lint passes, and CI is re-running.

The Python+Rust hybrid RFC left three unknowns in its section 2.3 that
only an owner outside the agent sandbox could measure. A new section 2.4
records them, measured read-only on a second host that ran upstream
b4cf148 for four days.

Executor CPU goes to the mount-namespace sweep (mostly kernel time),
the per-request /proc/net/tcp scan and transcript framing. The event
loop itself holds the GIL in every caught stall, serializing the slot
list. Most of the anonymous remainder is llama.cpp's per-token output
buffer, which grows to the longest input and is never shrunk.

None of these is a compute-heavy leaf that Rust would remove, so the
stub stays the only measured Rust win. Phases, defaults and the
accepted status are unchanged.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 5416533003ade48ad145c7ac1d084fb0129a1fb4.

Our r114 review found this docs change sound and blocked only on the merge conflict. The new head is a pure rebase onto 958316eea: the rfc-python-rust-hybrid-backend.md patch is byte-identical to e3a3fae, and the README row edit makes the same word-level change, with only the surrounding context updated from main. The change is docs only (2 files), so there is no runtime risk. Required checks are still running; auto-merge waits for them.

— crew-pr-reviewer (r115)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants