Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions js/hang/bench/slow-group.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
// Run with `bun js/hang/bench/slow-group.ts` to measure warning traffic per catch-up.
// Measure warning traffic and catch-up time over the groups a slow consumer skips.

import { Milli } from "@moq/clock";
import { Group, Time, Track, Varint } from "@moq/net";
import { Signal } from "@moq/signals";
import { Consumer } from "../src/container/consumer.ts";
import { Format } from "../src/container/legacy.ts";

console.log("groups,warnings,bytes,catchup_ms");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Record warning traffic as measurements

When the number or size of warnings regresses, the nightly report will not record either value as a measurement: javascript_file recognizes only timing headers, so warnings and bytes become dimensions in the series name, such as groups=16/warnings=15/bytes=300/catchup_ms. A warning-traffic change therefore creates a new catch-up series instead of producing a comparable warning or byte data point, defeating this benchmark's stated purpose and preventing the trend report from flagging that regression. Extend the report parser/unit convention and name these columns as measurements rather than dimensions.

AGENTS.md reference: AGENTS.md:L55-L55

Useful? React with 👍 / 👎.

for (const count of [1, 16, 256, 1024]) {
const track = new Track.Producer("tone");
const maxDelay = new Signal(Milli(30_000));
Expand Down Expand Up @@ -35,7 +36,7 @@ for (const count of [1, 16, 256, 1024]) {
const started = performance.now();
write(count);
await buffered(count);
console.log(JSON.stringify({ groups: count, warnings, bytes, catchupMs: performance.now() - started }));
console.log(`${count},${warnings},${bytes},${(performance.now() - started).toFixed(3)}`);
} finally {
consumer.close();
track.close();
Expand Down
4 changes: 2 additions & 2 deletions js/json/bench/codec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Decoder } from "../src/snapshot/decoder.ts";
import { Encoder } from "../src/snapshot/encoder.ts";

const updates = 256;
console.log("keys\tchanged\tcompression\tms/update");
console.log("keys,changed,compression,ms_per_update");
for (const keys of [1, 10, 100, 1000]) {
for (const changed of new Set([1, Math.min(10, keys)])) {
for (const compression of ["none", "deflate"] as const) {
Expand All @@ -28,7 +28,7 @@ for (const keys of [1, 10, 100, 1000]) {
samples.push((performance.now() - start) / updates);
}
samples.sort((a, b) => a - b);
console.log(`${keys}\t${changed}\t${compression}\t${samples[1]?.toFixed(6)}`);
console.log(`${keys},${changed},${compression},${samples[1]?.toFixed(6)}`);
}
}
}
6 changes: 2 additions & 4 deletions js/json/bench/parse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,9 @@ function time(bytes: Uint8Array, decode: (bytes: Uint8Array) => unknown): number
return samples[2] ?? 0;
}

console.log("frame\tbytes\tJSON.parse µs\tparse µs\tratio");
console.log("frame,bytes,json_parse_us,parse_us");
for (const [name, bytes] of cases) {
const baseline = time(bytes, (bytes) => JSON.parse(decoder.decode(bytes)));
const strict = time(bytes, parse);
console.log(
`${name}\t${bytes.length}\t${baseline.toFixed(2)}\t${strict.toFixed(2)}\t${(strict / baseline).toFixed(2)}`,
);
console.log(`${name},${bytes.length},${baseline.toFixed(2)},${strict.toFixed(2)}`);
}
17 changes: 17 additions & 0 deletions js/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,23 @@ json-stream-bench:
bunx tsc --noEmit --project json/bench/tsconfig.json
bun json/bench/stream.ts

# Sweep snapshot codec cost over document size and changed keys.
json-codec-bench:
bun install --frozen-lockfile
bunx tsc --noEmit --project json/bench/tsconfig.json
bun json/bench/codec.ts

# Compare the duplicate-refusing JSON frame parse against bare JSON.parse.
json-parse-bench:
bun install --frozen-lockfile
bunx tsc --noEmit --project json/bench/tsconfig.json
bun json/bench/parse.ts

# Sweep a slow hang consumer's skipped groups against its warning traffic and catch-up time.
slow-group-bench:
bun install --frozen-lockfile
bun hang/bench/slow-group.ts

# Sweep decoder count against PCM callback size during audio warmup.
audio-warmup-bench:
bun install --frozen-lockfile
Expand Down
3 changes: 3 additions & 0 deletions js/net/bench/broadcasts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ import { Producer } from "../src/origin.ts";
import * as Path from "../src/path.ts";
import { Stream, Writer } from "../src/stream.ts";

// Keep protocol diagnostics out of the measured CSV on stdout.
console.debug = console.error;

const routeCounts = [8, 32, 128];
const observerCounts = [1, 8, 32];
const updates = 32;
Expand Down
3 changes: 2 additions & 1 deletion js/net/bench/requests.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { ALPN, Version } from "../src/ietf/version";
import { createMockTransportPair } from "../src/mock";
import { Producer } from "../src/origin";
import * as Path from "../src/path";
import { readsFrom } from "../src/resume.test-util";
import { Stream } from "../src/stream";
import { wireOf } from "../src/wire";

Expand Down Expand Up @@ -149,7 +150,7 @@ for (const failovers of [0, 1, 16, 128])
sources.push(source);
pending[i].accept(source);
await settle();
if (entries[i].held.active.peek()?.closed !== source.closed) throw new Error("lost failover");
if (!readsFrom(entries[i].held.active.peek(), source)) throw new Error("lost failover");
}
if (round > 1) samples.push(active / 128);
} finally {
Expand Down
5 changes: 3 additions & 2 deletions js/watch/bench/warmup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ import { Terminal } from "../src/audio/terminal";
// Decoder count × callback size checks that trimming costs a span calculation,
// independent of the PCM payload size. Periodic gaps exercise the priming path.
const CALLBACKS = 4096;
console.log("decoders,frames,warmup_us,ns_per_callback");
// No timing suffix on warmup: the trend report would read that dimension as a measurement.
console.log("decoders,frames,warmup,ns_per_callback");
let retained = 0;
for (const count of [1, 16, 64]) {
for (const frames of [120, 960, 2880]) {
Expand All @@ -30,7 +31,7 @@ for (const count of [1, 16, 64]) {
timings.push(((performance.now() - start) * 1e6) / (CALLBACKS * count));
}
timings.sort((a, b) => a - b);
console.log(`${count},${frames},${warmup},${timings[1]?.toFixed(1)}`);
console.log(`${count},${frames},${warmup / 1000}ms,${timings[1]?.toFixed(1)}`);
}
}
}
Expand Down
1 change: 0 additions & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ Each lands its fixes in its own PR and plans only what needs a decision.
- [qmux 0.5.3](/quest/m0/qmux-0.5.3.md) - crates.io publishes qmux 0.5.3 with the builder session config the release backport of #5609 needs
- [Release shutdown backport](/quest/m0/ffi-shutdown-backport.md) - `release` publishes a non-breaking moq-ffi with an async `drain()` and qmux sessions on the QUIC stream limit
- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - every first-party publisher that can restart mints a fresh route epoch, the newest wins a path, and only routes with the same epoch resume a subscription
- [Performance audit](/quest/m0/audit-perf.md) - benchmarks that regressed since the last run are explained, fixed, or planned, and unbenched hot paths get coverage
- [Simplification audit](/quest/m0/audit-simplify.md) - overbuilt code and APIs are collapsed, merged, or narrowed into a net simpler whole
- [Lint override and deprecation audit](/quest/m0/audit-lints.md) - every lint suppression is still needed and says why, nothing calls a deprecated API, and stale Rust overrides fail the build
- [Quest tree audit](/quest/m0/audit-quests.md) - the tree agrees with itself and with main
110 changes: 0 additions & 110 deletions quest/m0/audit-perf.md

This file was deleted.

3 changes: 3 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ lands before the cut.

- [Playwright browsers from Nix](/quest/m1/playwright-nix.md) - every Playwright check runs on Nix's Chromium locally and in CI, with npm pinned to match

- [Allocation-free single-range subscriber window](/quest/m1/window-alloc.md) - a one-range subscribe resolves its window without a heap allocation, recovering the join cost per-subscriber ranges added

- [moq-net compiles once](/quest/m1/net-erase-transport.md) - the session drivers stop monomorphizing per transport and per crate, saving several MiB of the relay, FFI library, `libmoq` and CLI
- [moq-net internals](/quest/m1/net-internals.md) - no per-group session clones, no transport type parameter in lite and IETF internals, and the `Local` check measured

Expand All @@ -129,6 +131,7 @@ lands before the cut.
- [Documentation audit](/quest/m1/audit-docs.md) - stale pages, examples, links, and undocumented exports are fixed or planned
- [Artifact size audit](/quest/m1/audit-size.md) - binary, FFI, WASM, and bundle growth since the m0 run is explained, fixed, or planned
- [Compile time audit](/quest/m1/audit-compile-time.md) - build-time hotspots are measured against the m0 baselines and the worst are fixed or planned
- [Performance audit](/quest/m1/audit-perf.md) - benchmarks that regressed since the m0 run are explained, fixed, or planned, and unbenched hot paths get coverage

## Related

Expand Down
Loading
Loading