Repository navigation
fix(ts): run each bench route alone in a worker - #116
Conversation
Quest-Claim: c70fa28a30e9fb78
Routes in one realm shared state: Firefox's handoff `translated` shared its module with `spawned`, JavaScriptCore merged the type profiles of identical bundles, and V8 ran a route slower loaded after its identical copy. Each route now loads its own file in a worker of its own, which blocks in `Atomics.wait` between samples instead of going back to its event loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: da50acf
No actionable bugs found. Reviewed worker startup/error handling and cleanup, the single-outstanding-request shared-control handshake, output checks, warmup/sample accounting, and the microtask-only handoff workloads.
Direction looks sound: per-route workers remove shared-realm profiling effects while retaining interleaved rounds; the small Atomics protocol is justified by the reported idle-time distortion rather than expanding into a general worker RPC layer. Independent JIT-instance variation remains a measurement limitation, as the PR notes; this does not establish that a single A/A pair captures all benchmark uncertainty.
Verification: static source review only. I did not run tests or benchmarks, reproduce browser/JIT measurements, or independently validate the reported before/after results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Problem
just ts benchran every JS route in one realm, so routes shared state and the A/A route did not measure only noise. I reproduced each effect on main and in in-realm variants of the bench:translatedandspawnedshared one module, so the samedrainand channel call sites saw both routes' wakers.A/Ahad a module of its own and ran 20% faster thantranslatedat depth 1 and 11-15% faster at depth 4, in both low-load runs on main. Givingspawnedits own module brought the gap to -6.7%, +1.1%, and +0.4%. feat(ts): smaller output from MIR #95's finding was this same effect: whichever build sat in thetranslatedslot shared a module withspawned.spawnedin a module of its own but with the same text, Bun'stranslatedran depth 16 in 587 ns. Once each bundle's text differed (an extra export naming the route), it ran in 454 ns. Sospawned's types also slowed downtranslatedandA/Ain Bun.A/Aagreed withtranslatedeven when that state was slow. For example, Chrome'sbtree/256ran at 122 ns in both copies on main. Run alone, each copy runs at 88-90 ns.Approach
measuredrives the workers one sample at a time, so rounds stay interleaved in a random order across all routes. Browsers use module workers, and Node and Bun useworker_threads.task-spawned.jsis new. No cache keyed by URL can serve one route's code to another.Atomics.waiton a shared control word between samples. It never goes back to its event loop, and eachawaitreturns only to the microtask queue, which is all an async body needs. A first version that sent each request bypostMessagelet the engine do idle-time work between samples. That spread Firefox's samples of one body up to 2.5x apart (±10-30% per row), against ±1-2% now.ts/README.md, and the comments intime.tsandentry.ts. Deleted the quest.Measurements
All runs went through
quest gateon an AMD Ryzen 9 5900XT (32 threads). The 1-minute load average was 1.2-2.4 during the compared runs. "Before" is main (two runs) and "after" is this branch (three runs; the last is the pushed head). Codec cells are |A/A ÷ translated - 1| at the median and maximum workload. Handoff cells are A/A ÷ translated - 1 at depths 1, 4, and 16.translatedhandoff no longer carriesspawned's types: depth 16 dropped from 552-618 ns to 431-469 ns.translatedcopy ran varints up to 1.45x slower than its A/A copy. One realm hid this by putting both copies in the same state, so a--routebuild with different text was then compared against a correlated pair.translatedcodec rows is about the same in Chrome (0.7-0.8% before, 0.8-0.9% after) and Firefox (3.6-4.1% before, 3.2-5.5% after). It rose in Node (0.5% to 1.3-1.5%) and Bun (1.2% to 1.4-1.8%).bench/main.ts: 292-293 s before, 293-295 s after. CPU cost is unchanged: same samples, plus a few workers' startup.Impact
Alternatives
postMessagerequests. Firefox's per-sample noise rose 3-10x, as above.Follow-ups
/quest-plan: per-instance JIT states now dominate the A/A maximum in Chrome and Bun. Timing more than one instance per route, or reporting the gap between instances, would tell a real change from a route that settled into a slow state.main.ts. Whichever lands second merges the other.Checks
just check,just test, andquest check --base origin/mainpass. Ran 5 fulljust ts benchruns of this design (2 with thepostMessagetransport, 3 withAtomics.wait) and 4 of main throughquest gate, plus in-realm and Chrome-only diagnostic variants. Every route's output matched native Rust's on every engine.(written by Opus 5.5)
🤖 Generated with Claude Code
https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn