Skip to content

perf: cut the fixed ~100ms guest process-launch charge to ~36ms - #1919

Open
WyvernMonarch wants to merge 2 commits into
rivet-dev:mainfrom
WyvernMonarch:perf/thin-exec-startup-cost
Open

perf: cut the fixed ~100ms guest process-launch charge to ~36ms#1919
WyvernMonarch wants to merge 2 commits into
rivet-dev:mainfrom
WyvernMonarch:perf/thin-exec-startup-cost

Conversation

@WyvernMonarch

Copy link
Copy Markdown

Summary

Every guest process launch (each vm.process.exec, i.e. every bash tool call in an exec-style workload) paid a fixed ~100 ms charge. Stage-by-stage instrumentation of the warm path (92% coverage of the measured p50) attributed most of it to two avoidable costs, fixed here:

  • perf(wasm): stop copying the guest module through a JS Array. The memory-limit enforcement path copied the module bytes element-wise through a JS Array on every exec — ~29 ms per launch for the shipped coreutils payload. Replaced with subarray views + a single Buffer.concat; parsing is unchanged.
  • perf(v8): memoize the snapshot cache key instead of re-digesting per exec. The snapshot cache key re-digested the bridge + userland bytes on every exec. The key is now memoized keyed by content (unit tests assert the memo agrees with a fresh digest and that the key is dep-keyed over both bridge and userland, so no cross-content reuse).

Measured effect

Same-session sandwich (base → fixed → base restored), release sidecar, darwin arm64:

  • direct vm.process.exec (n=25): p50 98.95 → 36.51 ms (2.71×), p95 100.03 → 38.20; restored baseline 99.16 confirms attribution;
  • through pi's shipped bash tool (n=20): p50 99.5 → 36.4 ms (2.73×); read/edit/write and local-bash controls unmoved;
  • guest stage attribution: memory-limit enforcement 29 → 0 ms, wasi.start 22 → 17 ms.

Equivalence

enfml_equiv.mjs over the shipped payloads: 271 checks, 0 divergences across 120 real wasm modules (module parse results byte-equivalent pre/post). New unit tests: snapshot_cache_key_memo_agrees_with_a_fresh_digest, snapshot_cache_key_is_dep_keyed_over_bridge_and_userland. agentos-v8-runtime --lib: 156 passed.

Notes

  • Measurements are darwin arm64; the ratios were stable across two sessions but Linux numbers were not taken.
  • While profiling we observed AGENTOS_WASM_SNAPSHOT_RUNNER=off measuring faster than auto on the pre-fix baseline — unexplained and possibly worth its own look; this PR does not change that logic.
  • The snapshot-key memo is a small FIFO; behavior under many distinct userland bundles or high VM concurrency was not stress-measured.

🤖 Generated with Claude Code

Yuriy Butenko and others added 2 commits August 28, 2026 11:22
`enforceMemoryLimit` rewrites the memory section's page cap, which is tens of
bytes, but it materialised the whole module by pushing every byte into a JS
number `Array` and calling `Buffer.from` on it. That costs ~9.5 ms per MB, so
launching the 3 MB shell — what every guest `bash` call loads — spent ~29 ms
there, ten times the cost of the `WebAssembly.Module` compile it precedes.

Carry the untouched sections as `subarray` views and join once with
`Buffer.concat`. The parse, the validation errors and the output bytes are
unchanged; only the copy strategy is. Verified byte-identical against the
previous implementation over all 120 wasm binaries in the coreutils package
(including the 3.08 MB `sh`) and over synthetic modules covering absent,
unbounded, capped, oversized, multiple and malformed memory sections.

Warm `sh -c "echo x"` in a thin VM: the stage goes 29 ms -> 0 ms, and
`wasi.start` drops 22 -> 16 ms as well, because the guest module is no longer a
buffer materialised from three million boxed numbers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…exec

`snapshot_cache_key` SHA-256s the bridge bundle (~2.5 MB) plus the userland
bundle. Both are process-lifetime constants, and every execution derives the key
four times: the snapshot-cache lookup, the warm-worker pool key on the pre-warm
path, and again on the claim path inside session creation. Each digest measured
~7 ms, so a warm guest exec spent ~28 ms hashing two strings that never change —
the flat ~14 ms "snapshot-ready/pre-warm handshake" and the flat ~14 ms
"JS execution dispatch" on the launch path were almost entirely this.

Memoize by content: a bounded process-wide table keyed on full equality of the
bridge and userland text. A memcmp over the same bytes is ~35x cheaper than the
digest, and content equality means a lookup can only return the key of a bundle
byte-identical to the one asked for, so the memo is indistinguishable from
recomputing and cannot surface another caller's bundle. Also hash the two inputs
as streaming updates rather than concatenating them into a fresh 2.8 MB buffer
first; the digest is unchanged.

Sidecar phases for a warm `sh -c "echo x"`: snapshot-ready + pre-warm handshake
14.0 -> 0.3 ms, JS execution dispatch 14.3 -> 0.5 ms, execution finish
28.5 -> 1.1 ms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant