Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,12 +409,13 @@ Crashes, wrong bytes, missing frames, a window that presents nothing, or an
unavailable CPU clock fail the run; timing never does. The nightly runs
`--smoke` and keeps its run directory as the `bench-browser` artifact.

The in-memory frame, retained-history, and gap ledger paths also run directly in Chromium:
The in-memory frame, retained-history, gap ledger, and resume-history paths also run directly in Chromium:

```bash
nix develop --command bun bench/browser/micro.ts --bench frames --out .tmp/frames.json base=/path/to/release current=. aa=.
nix develop --command bun bench/browser/micro.ts --bench drift --out .tmp/drift.json before=/path/to/main current=. aa=.
nix develop --command bun bench/browser/micro.ts --bench ledger --out .tmp/ledger.json before=/path/to/main current=. aa=.
nix develop --command bun bench/browser/micro.ts --bench resume-history --out .tmp/resume-history.json before=/path/to/main current=. aa=.
```

Each variant uses its own workspace source and a persistent page, with one full
Expand All @@ -432,7 +433,22 @@ Its `time_mode` column is 0 for untimed, 1 for millisecond timestamps, and 2
for mixed millisecond/microsecond timestamps, which exercise exact cross-scale
comparison.

The browser smoke lane preserves one measured round of all three fixtures, and drift
`resume-history` moves long-lived logical fronts across lite sessions over the
in-memory transport, make-before-break as a session draining after GOAWAY does.
It sweeps fronts, names read per route, and route count, with `stable` names
(the same ones on every route) against `churn` (new names first answered by each
route), each read by info lookup, fetch, or subscription and then closed. Once a
manual clock ages out every cache, full collections from Chromium's exposed `gc`
let weak references count the retired routes' sessions, transports, broadcast
handles, and delivered payload buffers still reachable. These counts are exact,
so any nonzero one fails the run once every sample is written, leaving a retaining
`before` revision's counts in the JSON. A concluded control must release everything,
or the fixture fails. `size_bytes` is the JS heap the fronts and their serving
route hold, from Chromium's precise memory info; `serve_us` (clamped to the
page's coarse timer) and `read_us` include no setup. Bun's collector scans its
stack conservatively, so the fixture refuses to run there.

The browser smoke lane preserves one measured round of all four fixtures, and drift
and ledger also run in the nightly JS benchmark leg. Timing is
observational, with the A/A spread showing the host's noise floor.

Expand Down
42 changes: 32 additions & 10 deletions bench/browser/micro.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ const { values, positionals } = parseArgs({
},
});
const benchmark = values.bench;
if (benchmark !== "frames" && benchmark !== "drift" && benchmark !== "ledger")
throw new Error("--bench must be frames, drift, or ledger");
const expectedRows = ({ frames: 26, drift: 36, ledger: 12, "resume-history": 72 } as Record<string, number>)[benchmark];
if (expectedRows === undefined) throw new Error("--bench must be frames, drift, ledger, or resume-history");
const rounds = Number(values.rounds);
if (!Number.isSafeInteger(rounds) || rounds < 1) throw new Error("--rounds must be a positive integer");
const roots = new Map<string, string>();
Expand All @@ -39,11 +39,13 @@ for (const [name, root] of roots) {
{
name: "repeat-microbenchmark",
setup(build) {
// The ledger is internal, so select it explicitly instead of a package export.
build.onResolve({ filter: /^\.\.\/src\/ledger\.ts$/ }, () => ({
path: resolve(root, "js/net/src/ledger.ts"),
}));
build.onLoad({ filter: /\/bench\/(frames|drift|ledger)\.ts$/ }, async ({ path }) => {
// Internal modules have no package export, so select each variant's own explicitly.
build.onResolve({ filter: /^\.\.\/src\// }, ({ path, importer }) =>
importer.endsWith(`/js/net/bench/${benchmark}.ts`)
? { path: resolve(root, "js/net/bench", path) }
: undefined,
);
build.onLoad({ filter: /\/bench\/(frames|drift|ledger|resume-history)\.ts$/ }, async ({ path }) => {
const source = await Bun.file(path).text();
if (benchmark !== "frames")
return { loader: "ts", contents: `${source}\nglobalThis.measure = run;` };
Expand Down Expand Up @@ -81,7 +83,12 @@ const server = Bun.serve({
});
},
});
const browser = await chromium.launch({ headless: true, executablePath: process.env.BROWSER_BIN });
const browser = await chromium.launch({
headless: true,
executablePath: process.env.BROWSER_BIN,
// Retention is counted after full collections the page asks for, against an unrounded heap size.
args: benchmark === "resume-history" ? ["--js-flags=--expose-gc", "--enable-precise-memory-info"] : [],
});
const samples: { round: number; variant: string; rows: string[] }[] = [];
try {
const pages = new Map<string, Page>();
Expand Down Expand Up @@ -109,8 +116,7 @@ try {
rows.set(variant, []);
await pages.get(variant)?.evaluate("globalThis.measure()");
const measured = rows.get(variant);
if (measured?.length !== (benchmark === "frames" ? 26 : benchmark === "drift" ? 36 : 12))
throw new Error(`incomplete rows: ${variant}`);
if (measured?.length !== expectedRows) throw new Error(`incomplete rows: ${variant}`);
samples.push({ round, variant, rows: measured });
await Bun.write(
values.out,
Expand All @@ -123,6 +129,22 @@ try {
console.log(`measured ${round} ${variant}`);
}
}
// Reachability is exact here, so a retired route still reachable fails the run, once every
// variant's samples are written for comparison.
if (benchmark === "resume-history") {
const retaining = samples.filter(({ rows }) =>
rows.some((row) =>
row
.split(",")
.slice(5, 9)
.some((retired) => retired !== "0"),
),
);
if (retaining.length)
throw new Error(
`retired routes stayed reachable: ${[...new Set(retaining.map((s) => s.variant))].join(", ")}`,
);
}
} finally {
await browser.close();
server.stop(true);
Expand Down
2 changes: 1 addition & 1 deletion bench/browser/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ bun install --frozen-lockfile

cd "$HERE"
# The in-memory paths share Chromium installation and artifact retention with this lane.
for benchmark in frames drift ledger; do
for benchmark in frames drift ledger resume-history; do
bun micro.ts --bench "$benchmark" --rounds 1 --out "$HARNESS_RUN/$benchmark.json" \
current="$WORKSPACE" aa="$WORKSPACE"
done
Expand Down
Loading