perf: cut the fixed ~100ms guest process-launch charge to ~36ms - #1919
Open
WyvernMonarch wants to merge 2 commits into
Open
perf: cut the fixed ~100ms guest process-launch charge to ~36ms#1919WyvernMonarch wants to merge 2 commits into
WyvernMonarch wants to merge 2 commits into
Conversation
`enforceMemoryLimit` rewrites the memory section's page cap, which is tens of bytes, but it materialised the whole module by pushing every byte into a JS number `Array` and calling `Buffer.from` on it. That costs ~9.5 ms per MB, so launching the 3 MB shell — what every guest `bash` call loads — spent ~29 ms there, ten times the cost of the `WebAssembly.Module` compile it precedes. Carry the untouched sections as `subarray` views and join once with `Buffer.concat`. The parse, the validation errors and the output bytes are unchanged; only the copy strategy is. Verified byte-identical against the previous implementation over all 120 wasm binaries in the coreutils package (including the 3.08 MB `sh`) and over synthetic modules covering absent, unbounded, capped, oversized, multiple and malformed memory sections. Warm `sh -c "echo x"` in a thin VM: the stage goes 29 ms -> 0 ms, and `wasi.start` drops 22 -> 16 ms as well, because the guest module is no longer a buffer materialised from three million boxed numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…exec `snapshot_cache_key` SHA-256s the bridge bundle (~2.5 MB) plus the userland bundle. Both are process-lifetime constants, and every execution derives the key four times: the snapshot-cache lookup, the warm-worker pool key on the pre-warm path, and again on the claim path inside session creation. Each digest measured ~7 ms, so a warm guest exec spent ~28 ms hashing two strings that never change — the flat ~14 ms "snapshot-ready/pre-warm handshake" and the flat ~14 ms "JS execution dispatch" on the launch path were almost entirely this. Memoize by content: a bounded process-wide table keyed on full equality of the bridge and userland text. A memcmp over the same bytes is ~35x cheaper than the digest, and content equality means a lookup can only return the key of a bundle byte-identical to the one asked for, so the memo is indistinguishable from recomputing and cannot surface another caller's bundle. Also hash the two inputs as streaming updates rather than concatenating them into a fresh 2.8 MB buffer first; the digest is unchanged. Sidecar phases for a warm `sh -c "echo x"`: snapshot-ready + pre-warm handshake 14.0 -> 0.3 ms, JS execution dispatch 14.3 -> 0.5 ms, execution finish 28.5 -> 1.1 ms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every guest process launch (each
vm.process.exec, i.e. every bash tool call in an exec-style workload) paid a fixed ~100 ms charge. Stage-by-stage instrumentation of the warm path (92% coverage of the measured p50) attributed most of it to two avoidable costs, fixed here:perf(wasm): stop copying the guest module through a JS Array. The memory-limit enforcement path copied the module bytes element-wise through a JS Array on every exec — ~29 ms per launch for the shipped coreutils payload. Replaced with subarray views + a singleBuffer.concat; parsing is unchanged.perf(v8): memoize the snapshot cache key instead of re-digesting per exec. The snapshot cache key re-digested the bridge + userland bytes on every exec. The key is now memoized keyed by content (unit tests assert the memo agrees with a fresh digest and that the key is dep-keyed over both bridge and userland, so no cross-content reuse).Measured effect
Same-session sandwich (base → fixed → base restored), release sidecar, darwin arm64:
vm.process.exec(n=25): p50 98.95 → 36.51 ms (2.71×), p95 100.03 → 38.20; restored baseline 99.16 confirms attribution;Equivalence
enfml_equiv.mjsover the shipped payloads: 271 checks, 0 divergences across 120 real wasm modules (module parse results byte-equivalent pre/post). New unit tests:snapshot_cache_key_memo_agrees_with_a_fresh_digest,snapshot_cache_key_is_dep_keyed_over_bridge_and_userland.agentos-v8-runtime --lib: 156 passed.Notes
AGENTOS_WASM_SNAPSHOT_RUNNER=offmeasuring faster thanautoon the pre-fix baseline — unexplained and possibly worth its own look; this PR does not change that logic.🤖 Generated with Claude Code