From 72dc44d229e45178f2d0491e611cb9e8175219e6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 5 Nov 2026 22:28:44 -0800 Subject: [PATCH 1/8] chore(quest): claim quest/m1/push-to-talk-latency.md Quest-Claim: 6542ec2ef61286cb From cc2adf88603ea5a21317e45f410ab0d8af4847a5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 5 Nov 2026 22:32:44 -0800 Subject: [PATCH 2/8] test(audio-quality): measure push-to-talk on the default delay and codec Drop the fixed 300 ms delay and 60 ms Opus from just test push-to-talk, so it measures what a listener gets by default. Record the resolved playout delay over time, since "auto" moves it, and split each press's jitter buffer at the target the ring held when its PCM landed. Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 2 +- test/audio-quality/README.md | 14 ++++---- .../clients/js/ptt-analyze.test.ts | 16 +++++++++- test/audio-quality/clients/js/ptt-analyze.ts | 32 +++++++++++-------- test/audio-quality/clients/js/src/ptt.ts | 6 ++-- test/justfile | 2 +- 6 files changed, 46 insertions(+), 26 deletions(-) diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 848f10e8500..3037528e068 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -43,7 +43,7 @@ Decided 2026-10-28 (maintainer): same idea special-cased. The epic's own work: the acceptance proof `just test push-to-talk` (22 -presses, 60 ms Opus, 300 ms delay). A listener's unmute pays no catalog +presses on the default Opus and `delay: "auto"`). A listener's unmute pays no catalog round trip and no resubscribe, and every frame of each utterance is encoded, sent, received, and decoded from its first frame, with no catalog flip and no underrun or stall across the presses. diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 5a8cc36489e..f11100b70da 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -129,10 +129,10 @@ just test push-to-talk ``` This separate, roughly nine-minute run drives the shipped publish and watch elements in one Chromium -page. A constant tone loops through the fake microphone, encoded as 128 kbit/s Opus with 60 ms -frames. A release mutes the publisher, which sends nothing until the next press while its rendition -stays enabled, so the listener keeps its subscription and plays comfort noise in between. The -listener holds a fixed 300 ms delay. After warmup and a mute, it measures 22 presses, +page with their defaults: Opus at 20 ms frames, and `delay: "auto"`. A constant tone loops through +the fake microphone. A release mutes the publisher, which sends nothing until the next press while +its rendition stays enabled, so the listener keeps its subscription and plays comfort noise in +between. After warmup and a mute, it measures 22 presses, alternating two and three seconds: the first after the warmup's five-second mute, seven with no intervening wait, seven after five seconds, and seven after sixty seconds. Both elements reach a local relay directly over WebTransport, with no shaping, loss, or queueing, so every millisecond @@ -165,13 +165,13 @@ it: `capture` (worklet delivery and framing, after the media the frame still nee `packetization` (that media, placed on the media clock by the newest chunk captured before the press), `encode` (codec and container write), `network` (write to arrival, through the relay), `decode` (arrival to decoded PCM), `jitter_buffer` (the head's PCM to the first PCM that fills the -playout delay), `render` (that PCM to the render onset: the worklet hand-off and the context's +playout delay in effect as it lands, which each press reports as `delay`), `render` (that PCM to the render onset: the worklet hand-off and the context's render callback), and `device`, as the stages below name them. `spread` is how far an utterance's frames stray from a steady arrival: the latest arrival relative to its media timestamp minus the earliest, which on this ideal path is the capture graph's cadence and what `"auto"`'s jitter estimate measures. -`latency` and `path` give each one's min, median and max. +`latency`, `path` and `delay` give each one's min, median and max. -The command prints `latency`, `path`, the completeness counts, and `playback`'s underruns and +The command prints `latency`, `path`, `delay`, the completeness counts, and `playback`'s underruns and stalls, and keeps its run directory even when it passes (set `HARNESS_KEEP=0` to drop it). Nightly runs it and keeps the raw evidence alongside the audio quality rows. diff --git a/test/audio-quality/clients/js/ptt-analyze.test.ts b/test/audio-quality/clients/js/ptt-analyze.test.ts index 6e8fa9bf4ef..70602ad1a15 100644 --- a/test/audio-quality/clients/js/ptt-analyze.test.ts +++ b/test/audio-quality/clients/js/ptt-analyze.test.ts @@ -25,7 +25,7 @@ function trace() { playback: { underruns: 0, stalls: 0 }, errors: [] as string[], arrivals: [] as [number, number, number, number][], - state: { delay: 300 }, + delays: [{ at: 0, delay: 300 }], }; for (let i = 0; i < 22; i++) { const press = 10_000 + i * 10_000; @@ -85,6 +85,20 @@ test("a complete run passes, reports its latency, and splits each press exactly" }); const parts = Object.values(press.path as Record).reduce((sum, part) => sum + part, 0); expect(parts).toBe(press.audible); + expect(summary.delay).toEqual({ min: 300, median: 300, max: 300 }); +}); + +test("a target that rises before the delay fills is filled by the frame that reaches it", async () => { + // "auto" raises the target a frame as the press starts, so the sixth frame fills it. + const result = trace(); + result.delays.push({ at: result.presses[0].at, delay: 300 + FRAME }); + result.audible[0].rendered += FRAME; + result.audible[0].at += FRAME; + const { exitCode, summary } = await analyze(result); + expect(exitCode).toBe(0); + expect(summary.presses[0].delay).toBe(300 + FRAME); + expect(summary.presses[0].path.jitter_buffer).toBe(5 * FRAME); + expect(summary.presses[0].path.render).toBe(10); }); test("a frame held back by a capture burst widens its press's spread", async () => { diff --git a/test/audio-quality/clients/js/ptt-analyze.ts b/test/audio-quality/clients/js/ptt-analyze.ts index 967254b1fe2..e130a23deac 100644 --- a/test/audio-quality/clients/js/ptt-analyze.ts +++ b/test/audio-quality/clients/js/ptt-analyze.ts @@ -14,6 +14,9 @@ if (result.captureError || result.errors.length) throw new Error("invalid captur const encoded = new Map(result.encoded.map((frame) => [frame.timestamp, frame])); const sent = new Map(result.sent.map((frame) => [frame.timestamp, frame])); const arrivals = new Map(result.arrivals.map((frame) => [Math.round(frame[1] * 1000), frame])); +// The playout target the ring held at `at`: it checks each write against the delay of that moment, +// which "auto" moves with its estimate. +const delayAt = (at: number) => result.delays.findLast((entry) => entry.at <= at)?.delay; // The first decode of each timestamp; a later one would be a re-decode, not the frame's first. const decoded = new Map(); for (const frame of result.decoded) if (!decoded.has(frame.timestamp)) decoded.set(frame.timestamp, frame); @@ -52,26 +55,25 @@ const presses = result.presses // from their media timestamps, slowest minus fastest. An ideal path leaves only capture cadence. const offsets = received.map(([at, media]) => at - media); const spread = offsets.length ? Math.max(...offsets) - Math.min(...offsets) : null; + const first = input[0]; + // The first PCM that reached the delay from the head: the ring leaves its stall on it, and a + // later frame decoded before the onset only adds to a ring already playing. + const filled = input + .flatMap((frame) => decoded.get(frame.timestamp) ?? []) + .toSorted((a, b) => a.at - b.at) + .find((frame) => { + const delay = delayAt(frame.at); + return delay !== undefined && frame.timestamp + (frame.duration ?? 0) - first.timestamp >= delay * 1000; + }); // The head frame's path from the press to projected output, one boundary after another, so // the parts sum to `audible`. Capture and packetization split at the head frame's end on the // media clock, placed against the newest captured chunk delivered before the press. - const first = input[0]; const path = (() => { - if (!audible || !first || !press.capture) return null; + if (!audible || !first || !press.capture || !filled) return null; const write = sent.get(first.timestamp); const arrival = arrivals.get(first.timestamp); const output = decoded.get(first.timestamp); - // The first PCM that reached the delay from the head: the ring leaves its stall on it, and a - // later frame decoded before the onset only adds to a ring already playing. - const delay = result.state?.delay; - const filled = - delay === undefined - ? undefined - : input - .flatMap((frame) => decoded.get(frame.timestamp) ?? []) - .toSorted((a, b) => a.at - b.at) - .find((frame) => frame.timestamp + (frame.duration ?? 0) - first.timestamp >= delay * 1000); - if (!write || !arrival || !output || !filled) return null; + if (!write || !arrival || !output) return null; const pressed = press.capture.end / 1000 + (press.at - press.capture.at); const packetization = (first.timestamp + (first.duration ?? 0)) / 1000 - pressed; return { @@ -100,6 +102,7 @@ const presses = result.presses audible: audible ? audible.at - press.at : null, delivery, spread, + delay: filled ? (delayAt(filled.at) ?? null) : null, playout: audible && delivery !== null ? audible.rendered - press.at - delivery : null, output: audible ? audible.at - audible.rendered : null, path, @@ -124,7 +127,8 @@ const summary = { path: Object.fromEntries( parts.map((part) => [part, summarize(presses.map((press) => press.path?.[part] ?? null))]), ), - delay: result.state?.delay, + // The playout target each head filled. + delay: summarize(presses.map((press) => press.delay)), presses, captures: result.captures, encoders: result.encoders, diff --git a/test/audio-quality/clients/js/src/ptt.ts b/test/audio-quality/clients/js/src/ptt.ts index 3741759998f..cf58f168aaf 100644 --- a/test/audio-quality/clients/js/src/ptt.ts +++ b/test/audio-quality/clients/js/src/ptt.ts @@ -27,6 +27,8 @@ const records = { encoders: [] as number[], // Each time the published rendition became unselectable (disabled or dropped) or selectable again. catalog: [] as { at: number; selectable: boolean }[], + // The resolved playout delay since the measurement's start: "auto" moves it with the estimate. + delays: [] as { at: number; delay: number }[], errors: [] as string[], }; @@ -114,7 +116,6 @@ const publish = document.createElement("moq-publish") as MoqPublish; publish.url = required("relay"); publish.name = "ptt.hang"; publish.invisible = true; -publish.audio.codec.set({ mime: "opus", frameDuration: Milli(60), bitrate: 128_000 }); publish.source = "camera"; document.body.append(publish); @@ -126,7 +127,6 @@ const listener = new URL(required("relay")); listener.searchParams.set("role", "listener"); watch.url = listener.toString(); watch.name = publish.name; -watch.setAttribute("delay", "300ms"); watch.setAttribute("visible", "always"); watch.volume = 1; watch.muted = false; @@ -208,6 +208,7 @@ publish.audio.out.catalog.subscribe((config) => { if (now !== selectable) records.catalog.push({ at: performance.now(), selectable: now }); selectable = now; }); +watch.player.sync.out.delay.subscribe((delay) => records.delays.push({ at: performance.now(), delay })); // Playback counts from the measurement's start, so the warmup's tune-in is not charged to it. let baseline = { underruns: 0, stalls: 0 }; const playback = () => { @@ -241,6 +242,7 @@ export const measurement = { }, reset() { for (const list of Object.values(records)) list.length = 0; + records.delays.push({ at: performance.now(), delay: watch.player.sync.out.delay.peek() }); capture.drain(); arrivals.length = 0; const stats = audio.out.stats.peek(); diff --git a/test/justfile b/test/justfile index bd9db4d79e4..926489062ce 100644 --- a/test/justfile +++ b/test/justfile @@ -164,7 +164,7 @@ native-play: audio-quality *args: ./audio-quality/run.sh "$@" -# Measure 22 push-to-talk presses at 300 ms on an ideal local path. +# Measure 22 push-to-talk presses with the default delay and codec on an ideal local path. push-to-talk: ./audio-quality/ptt.sh From f2b6daae47ea0504c5a56864b910aa96057408c6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 5 Nov 2026 22:58:06 -0800 Subject: [PATCH 3/8] quest: record the push-to-talk default-path breakdown Measured on kixtop (macOS) and Linux: the defaults reach the render onset in about 70 ms, but "auto" underruns push-to-talk on both, so the budget waits on a maintainer decision about Watch. Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 128 +++++++++++++++---------------- quest/m1/push-to-talk/README.md | 5 +- test/audio-quality/README.md | 4 +- 3 files changed, 67 insertions(+), 70 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 1a7b2f50cb7..a13dd3221f6 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -10,77 +10,13 @@ playback is out of scope. ## Plan -Measured 2026-11-04 by -[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760), for -[Push-to-talk](/quest/m1/push-to-talk/README.md). The setup was `just test -push-to-talk` through a local relay, publisher and listener on separate -sessions, with no shaping or loss, 60 ms Opus at 128 kbit/s, and a fixed 300 -ms `delay`. Chromium 156 ran headless on Linux. There were five runs of 22 -presses each, 15 of them separated, so 75 separated presses in all; one press -lost its head (see Push-to-talk) and has no breakdown, leaving 74. -`summary.json` `path` splits each press at page-clock boundaries that sum -exactly to press → projected output. - -| Part | Boundaries | Median ms | Max ms | -| --- | --- | --- | --- | -| capture | worklet delivery and framing | 29 | 110 | -| packetization | media the head frame still needed after the press | 39 | 60 | -| encode | encoder input to container write | 0.8 | 14 | -| network | container write to arrival, through the relay | 3.2 | 15 | -| decode | arrival to decoded PCM | 0.7 | 13 | -| jitter buffer | head's PCM to the first PCM that filled the delay | 252 | 268 | -| render | that PCM to the render onset | 21 | 54 | -| device | render onset to projected output | 115 | 119 | -| total | press to projected output | 469 | 576 | - -The capture and packetization split is approximate, but their 66 ms median sum -is exact. Playout on top of one-way delivery was 261 ms median and 294 ms max -over all 75, inside the 300 ms the maintainer set. - -Biggest contributors and their knobs: - -- **Playout delay, about 300 ms.** The ring starts once it holds the delay in - decoded PCM, counted from the head's first sample. That is why the jitter - buffer plus packetization sum to about 291 ms. Knob: `` `delay`. - This test fixes it at 300 ms. In `"auto"`, the default, the target is - `max(measured, advertised) + frame + catalog delay` - (`js/watch/src/audio/latency.ts`). -- **Device output, about 115 ms.** This matches Chromium's declared - `baseLatency + outputLatency` on this headless Linux host. Knob: the watch - AudioContext's `latencyHint` (`"interactive"` in - `js/watch/src/audio/decoder.ts`), otherwise the platform. A real device, - such as `kixtop`'s macOS output, should be measured before acting on it. -- **Capture and render, about 52 ms together.** Both followed the - AudioContext callback cadence on this host: PipeWire gives Chromium 2048 - samples (43 ms) per callback even at `"interactive"`. Capture no longer - does, since its context renders to no device - ([moq-dev/moq#7881](https://github.com/moq-dev/moq/pull/7881)): on this host - its median dropped from 29 ms to 1-9 ms, and the median per-press arrival - spread from 45 ms to 12 ms. Render still does. Knob: the watch AudioContext's - `latencyHint`, otherwise the platform. -- **Opus frame, 60 ms.** This test chooses it; the publisher defaults to - 20 ms. Under a fixed delay it is absorbed by the delay. In `"auto"` it adds - one frame to the target. Knob: the encoder's `frameDuration`. - -No part looked like a bug. On an ideal path, encode and decode take about 1 ms -and the relay hop about 3 ms, and the head is not trimmed as decoder warmup. -The one lost head is an ordering defect, owned by -[Push-to-talk](/quest/m1/push-to-talk/README.md), not a latency part. The -fixed delay bypasses the jitter estimator. In `"auto"`, the 46 ms median -per-press arrival-lateness spread, from capture bursts rather than the -network, would make the measured term nonzero; that is the estimator working -as specified in [Audio jitter -target](/quest/m1/audio-jitter-target/README.md). With capture fixed, the -spread left on Chromium 155+ is its 10 ms capture chunks; a release no longer -flushes the codec's lookahead chunk out ahead of its media. -Older Chromium still paces capture at the device's callback size. - Decided 2026-11-04 (maintainer, planning in [moq-dev/moq#7816](https://github.com/moq-dev/moq/pull/7816)): - ✅ Budget the default path with realistic settings, not the fixed 300 ms / - 60 ms setup above; drop the fixed-delay row from `just test push-to-talk`. - The maintainer is fine assuming the constants are the problem. + 60 ms setup [moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) + measured; drop the fixed-delay row from `just test push-to-talk`. The + maintainer is fine assuming the constants are the problem. - ✅ Measure capture and output latency on real macOS hardware (`kixtop`) first; set the budget from those numbers, and keep output-side knobs until then. @@ -89,6 +25,64 @@ Decided 2026-11-04 (maintainer, planning in budget is measured on silence suppression, which [moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942) landed. +[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) runs +`just test push-to-talk` on the defaults and splits each press's jitter buffer +at the target `"auto"` held when its PCM landed. Measured 2026-11-05, one run +of 22 presses each, Chromium 156 headless through a local relay. `kixtop` is a +MacBook Air on macOS 26 with its built-in speakers as the default output. +Linux is the agent host under heavy load, with PipeWire output. +The table gives the median and max in ms over the separated presses with a +breakdown: 15 on `kixtop`, 13 on Linux, which lost two heads. + +| Part | `kixtop` | Linux | +| --- | --- | --- | +| capture | 9 / 12 | 10 / 11 | +| packetization | 16 / 20 | 16 / 20 | +| encode | 0.5 / 0.6 | 0.5 / 8 | +| network | 2 / 12 | 3 / 5 | +| decode | 0.5 / 3 | 0.6 / 17 | +| jitter buffer | 21 / 59 | 40 / 43 | +| render | 23 / 34 | -1 / 24 | +| press to render onset | 73 / 110 | 71 / 95 | +| device | 54 / 57 | 105 / 113 | +| `"auto"` target | 40 (max 80) | 60 | +| underruns / stalls | 10 / 9 | 64 / 33 | + +Both runs fail the run's no-underrun check. `"auto"` targets the 0.95 +quantile of arrival lateness and accepts the re-stalls past it by design +until [Watch](/quest/m1/audio-jitter-target/watch.md) time-stretches the +ring. Push-to-talk's capture spread on this ideal path reached 60 ms on +`kixtop` and 78 ms on Linux, past a 40 or 60 ms target. Linux's much higher +count is not separated from its host load (load average above 20) or its +43 ms render callbacks draining a 60 ms target. Watch changes the jitter +buffer part too, so a budget set before it lands would move. + +Measurement limits: the harness's headless Chromium mutes its output and reads +a fake microphone, so `device` is Chromium's projected output for the default +device and `capture` excludes the microphone's own input latency. Linux's +render callbacks are 43 ms, so its `render` can be negative by up to one +callback. The lost heads are the ordering defect +[Push-to-talk](/quest/m1/push-to-talk/README.md) owns. + +Knobs: the target's frame term (`frameDuration`, 20 ms by default) and its +measured term (`js/watch/src/audio/latency.ts`); device and render follow the +watch AudioContext's `latencyHint` (`"interactive"` in +`js/watch/src/audio/decoder.ts`), otherwise the platform. + +## Waiting + +- Maintainer: decide how `just test push-to-talk` runs until + [Watch](/quest/m1/audio-jitter-target/watch.md) lands, since its + no-underrun check fails on the defaults by design. Options: (a) require + Watch here, keep the fixed 300 ms delay until then, and budget the + defaults after it lands (recommended: the check stays green, and the budget + is set on the jitter buffer users will get); (b) run the defaults now and + stop failing on underruns while `"auto"` is on, which also drops the + [Push-to-talk](/quest/m1/push-to-talk/README.md) proof's check that a + declared silence is never an underrun; (c) run both rows, roughly doubling + the nine-minute nightly step. Answer on + [moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009). + ## Related - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the `"auto"` target this latency feeds diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 3037528e068..40300ea85b9 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -59,8 +59,9 @@ Decided with the maintainer on (2026-11-03): "We can't ever expect 300ms latency if it takes more than 300ms to deliver a packet." No clocked live start or other startup machinery. Five ideal runs on 2026-11-04 held it on all 75 separated presses - (median 261 ms, max 294 ms). The breakdown and its knobs are in - [Push-to-talk latency](/quest/m1/push-to-talk-latency.md). + (median 261 ms, max 294 ms) at a fixed 300 ms delay, broken down on + [moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760). The defaults' + breakdown is in [Push-to-talk latency](/quest/m1/push-to-talk-latency.md). - ✅ **No encoder recovery on re-enable** (2026-11-03). Asked first why a fatal encoder error is expected at all: it is not. No push-to-talk run reported one, and no issue reports a publisher encoder failure; the latch diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index f11100b70da..33639f25d1b 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -166,7 +166,9 @@ it: `capture` (worklet delivery and framing, after the media the frame still nee press), `encode` (codec and container write), `network` (write to arrival, through the relay), `decode` (arrival to decoded PCM), `jitter_buffer` (the head's PCM to the first PCM that fills the playout delay in effect as it lands, which each press reports as `delay`), `render` (that PCM to the render onset: the worklet hand-off and the context's -render callback), and `device`, as the stages below name them. `spread` is how far an utterance's frames +render callback), and `device`, as the stages below name them. The onset is placed on the render +clock, which a context rendering in large callbacks runs up to one callback ahead of the page, so +there `render` can be negative and `device` larger by as much. `spread` is how far an utterance's frames stray from a steady arrival: the latest arrival relative to its media timestamp minus the earliest, which on this ideal path is the capture graph's cadence and what `"auto"`'s jitter estimate measures. `latency`, `path` and `delay` give each one's min, median and max. From 0b6936e9a2116415c6c847a53176c12e04a5721d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 08:33:42 -0800 Subject: [PATCH 4/8] test(audio-quality): hold push-to-talk at a fixed 300 ms until Watch lands Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 2 +- test/audio-quality/README.md | 5 +++-- test/audio-quality/clients/js/src/ptt.ts | 3 +++ test/justfile | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 40300ea85b9..98b64b674fb 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -43,7 +43,7 @@ Decided 2026-10-28 (maintainer): same idea special-cased. The epic's own work: the acceptance proof `just test push-to-talk` (22 -presses on the default Opus and `delay: "auto"`). A listener's unmute pays no catalog +presses, default Opus, 300 ms delay). A listener's unmute pays no catalog round trip and no resubscribe, and every frame of each utterance is encoded, sent, received, and decoded from its first frame, with no catalog flip and no underrun or stall across the presses. diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 33639f25d1b..e098c6611ee 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -129,8 +129,9 @@ just test push-to-talk ``` This separate, roughly nine-minute run drives the shipped publish and watch elements in one Chromium -page with their defaults: Opus at 20 ms frames, and `delay: "auto"`. A constant tone loops through -the fake microphone. A release mutes the publisher, which sends nothing until the next press while +page. The publisher keeps its default Opus at 20 ms frames. The listener holds a fixed 300 ms delay, +since `"auto"` still underruns on push-to-talk's capture spread and the run fails on any underrun. +A constant tone loops through the fake microphone. A release mutes the publisher, which sends nothing until the next press while its rendition stays enabled, so the listener keeps its subscription and plays comfort noise in between. After warmup and a mute, it measures 22 presses, alternating two and three seconds: the first after the warmup's five-second mute, seven with no diff --git a/test/audio-quality/clients/js/src/ptt.ts b/test/audio-quality/clients/js/src/ptt.ts index cf58f168aaf..ae77ae98675 100644 --- a/test/audio-quality/clients/js/src/ptt.ts +++ b/test/audio-quality/clients/js/src/ptt.ts @@ -127,6 +127,9 @@ const listener = new URL(required("relay")); listener.searchParams.set("role", "listener"); watch.url = listener.toString(); watch.name = publish.name; +// Fixed until "auto" stops underrunning on push-to-talk's capture spread +// (quest/m1/audio-jitter-target/watch.md): the run fails on any underrun. +watch.setAttribute("delay", "300ms"); watch.setAttribute("visible", "always"); watch.volume = 1; watch.muted = false; diff --git a/test/justfile b/test/justfile index 926489062ce..bd9db4d79e4 100644 --- a/test/justfile +++ b/test/justfile @@ -164,7 +164,7 @@ native-play: audio-quality *args: ./audio-quality/run.sh "$@" -# Measure 22 push-to-talk presses with the default delay and codec on an ideal local path. +# Measure 22 push-to-talk presses at 300 ms on an ideal local path. push-to-talk: ./audio-quality/ptt.sh From bd10e445cb8fd7aa72965af7a5503bdf73afeeae Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 08:34:21 -0800 Subject: [PATCH 5/8] quest: require Watch before budgeting push-to-talk on the defaults Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 68 +++++++++++++++++--------------- 1 file changed, 37 insertions(+), 31 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index a13dd3221f6..f691ad0bb86 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -15,8 +15,8 @@ Decided 2026-11-04 (maintainer, planning in - ✅ Budget the default path with realistic settings, not the fixed 300 ms / 60 ms setup [moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) - measured; drop the fixed-delay row from `just test push-to-talk`. The - maintainer is fine assuming the constants are the problem. + measured; drop the fixed-delay row from `just test push-to-talk` (once Watch + lands, per the 2026-11-06 decision below). The maintainer is fine assuming the constants are the problem. - ✅ Measure capture and output latency on real macOS hardware (`kixtop`) first; set the budget from those numbers, and keep output-side knobs until then. @@ -25,14 +25,32 @@ Decided 2026-11-04 (maintainer, planning in budget is measured on silence suppression, which [moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942) landed. -[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) runs -`just test push-to-talk` on the defaults and splits each press's jitter buffer -at the target `"auto"` held when its PCM landed. Measured 2026-11-05, one run -of 22 presses each, Chromium 156 headless through a local relay. `kixtop` is a -MacBook Air on macOS 26 with its built-in speakers as the default output. -Linux is the agent host under heavy load, with PipeWire output. -The table gives the median and max in ms over the separated presses with a -breakdown: 15 on `kixtop`, 13 on Linux, which lost two heads. +Decided 2026-11-06 (maintainer, on +[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009)): how should +`just test push-to-talk` run until Watch lands, since its no-underrun check +fails on the defaults by design? + +- ✅ (a) Require [Watch](/quest/m1/audio-jitter-target/watch.md) for this + quest. The run holds a fixed 300 ms delay until then, and the defaults are + budgeted after Watch lands, on the jitter buffer users will get. +- (b) Run the defaults now and stop failing on underruns while `"auto"` is + on. +- (c) Run both rows, roughly doubling the nine-minute nightly step. + +What did not depend on Watch landed in +[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009): the run uses the publisher's +default Opus (20 ms frames, 64 kbit/s), and the page records the resolved +delay over time, so each press's `jitter_buffer` ends at the first PCM that +reaches the target the ring held when that PCM landed. What remains is to +switch the page to `delay: "auto"`, measure on `kixtop` again, and set and +enforce the budget. + +Measured 2026-11-05 on the defaults, before Watch, one run of 22 presses +each, Chromium 156 headless through a local relay. `kixtop` is a MacBook Air +on macOS 26 with its built-in speakers as the default output. Linux is the +agent host under heavy load, with PipeWire output. The table gives the median +and max in ms over the separated presses with a breakdown: 15 on `kixtop`, 13 +on Linux, which lost two heads. | Part | `kixtop` | Linux | | --- | --- | --- | @@ -48,14 +66,12 @@ breakdown: 15 on `kixtop`, 13 on Linux, which lost two heads. | `"auto"` target | 40 (max 80) | 60 | | underruns / stalls | 10 / 9 | 64 / 33 | -Both runs fail the run's no-underrun check. `"auto"` targets the 0.95 -quantile of arrival lateness and accepts the re-stalls past it by design -until [Watch](/quest/m1/audio-jitter-target/watch.md) time-stretches the -ring. Push-to-talk's capture spread on this ideal path reached 60 ms on -`kixtop` and 78 ms on Linux, past a 40 or 60 ms target. Linux's much higher -count is not separated from its host load (load average above 20) or its -43 ms render callbacks draining a 60 ms target. Watch changes the jitter -buffer part too, so a budget set before it lands would move. +Both runs failed the no-underrun check. `"auto"` targets the 0.95 quantile of +arrival lateness and accepts the re-stalls past it by design until Watch +time-stretches the ring. Push-to-talk's capture spread on this ideal path +reached 60 ms on `kixtop` and 78 ms on Linux, past a 40 or 60 ms target. +Linux's much higher count is not separated from its host load (load average +above 20) or its 43 ms render callbacks draining a 60 ms target. Measurement limits: the harness's headless Chromium mutes its output and reads a fake microphone, so `device` is Chromium's projected output for the default @@ -69,19 +85,9 @@ measured term (`js/watch/src/audio/latency.ts`); device and render follow the watch AudioContext's `latencyHint` (`"interactive"` in `js/watch/src/audio/decoder.ts`), otherwise the platform. -## Waiting - -- Maintainer: decide how `just test push-to-talk` runs until - [Watch](/quest/m1/audio-jitter-target/watch.md) lands, since its - no-underrun check fails on the defaults by design. Options: (a) require - Watch here, keep the fixed 300 ms delay until then, and budget the - defaults after it lands (recommended: the check stays green, and the budget - is set on the jitter buffer users will get); (b) run the defaults now and - stop failing on underruns while `"auto"` is on, which also drops the - [Push-to-talk](/quest/m1/push-to-talk/README.md) proof's check that a - declared silence is never an underrun; (c) run both rows, roughly doubling - the nine-minute nightly step. Answer on - [moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009). +## Required + +- [Watch](/quest/m1/audio-jitter-target/watch.md) - `"auto"` stops underrunning by time-stretching the ring, so the run can hold the default delay ## Related From 0d6cc67d702740a32db0bea08048b601d022abeb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 08:45:24 -0800 Subject: [PATCH 6/8] test(audio-quality): keep push-to-talk's 60 ms Opus until the defaults are budgeted Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 2 +- test/audio-quality/README.md | 9 ++++----- test/audio-quality/clients/js/src/ptt.ts | 3 +-- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 22489116c78..d9dcc1915a6 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -43,7 +43,7 @@ Decided 2026-10-28 (maintainer): same idea special-cased. The epic's own work: the acceptance proof `just test push-to-talk` (22 -presses, default Opus, 300 ms delay). A listener's unmute pays no catalog +presses, 60 ms Opus, 300 ms delay). A listener's unmute pays no catalog round trip and no resubscribe, and every frame of each utterance is encoded, sent, received, and decoded from its first frame, with no catalog flip and no underrun or stall across the presses. diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index e098c6611ee..b4a4a2f6f58 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -129,11 +129,10 @@ just test push-to-talk ``` This separate, roughly nine-minute run drives the shipped publish and watch elements in one Chromium -page. The publisher keeps its default Opus at 20 ms frames. The listener holds a fixed 300 ms delay, -since `"auto"` still underruns on push-to-talk's capture spread and the run fails on any underrun. -A constant tone loops through the fake microphone. A release mutes the publisher, which sends nothing until the next press while -its rendition stays enabled, so the listener keeps its subscription and plays comfort noise in -between. After warmup and a mute, it measures 22 presses, +page. A constant tone loops through the fake microphone, encoded as 128 kbit/s Opus with 60 ms +frames. A release mutes the publisher, which sends nothing until the next press while its rendition +stays enabled, so the listener keeps its subscription and plays comfort noise in between. The +listener holds a fixed 300 ms delay. After warmup and a mute, it measures 22 presses, alternating two and three seconds: the first after the warmup's five-second mute, seven with no intervening wait, seven after five seconds, and seven after sixty seconds. Both elements reach a local relay directly over WebTransport, with no shaping, loss, or queueing, so every millisecond diff --git a/test/audio-quality/clients/js/src/ptt.ts b/test/audio-quality/clients/js/src/ptt.ts index ae77ae98675..a664c7ff499 100644 --- a/test/audio-quality/clients/js/src/ptt.ts +++ b/test/audio-quality/clients/js/src/ptt.ts @@ -116,6 +116,7 @@ const publish = document.createElement("moq-publish") as MoqPublish; publish.url = required("relay"); publish.name = "ptt.hang"; publish.invisible = true; +publish.audio.codec.set({ mime: "opus", frameDuration: Milli(60), bitrate: 128_000 }); publish.source = "camera"; document.body.append(publish); @@ -127,8 +128,6 @@ const listener = new URL(required("relay")); listener.searchParams.set("role", "listener"); watch.url = listener.toString(); watch.name = publish.name; -// Fixed until "auto" stops underrunning on push-to-talk's capture spread -// (quest/m1/audio-jitter-target/watch.md): the run fails on any underrun. watch.setAttribute("delay", "300ms"); watch.setAttribute("visible", "always"); watch.volume = 1; From 6c055e1c441353d84d2ae13c82f87a2a444b5299 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 08:45:50 -0800 Subject: [PATCH 7/8] quest: record why push-to-talk keeps 60 ms Opus until the defaults are budgeted Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index f691ad0bb86..76251def779 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -37,13 +37,16 @@ fails on the defaults by design? on. - (c) Run both rows, roughly doubling the nine-minute nightly step. -What did not depend on Watch landed in -[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009): the run uses the publisher's -default Opus (20 ms frames, 64 kbit/s), and the page records the resolved -delay over time, so each press's `jitter_buffer` ends at the first PCM that -reaches the target the ring held when that PCM landed. What remains is to -switch the page to `delay: "auto"`, measure on `kixtop` again, and set and -enforce the budget. +[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) landed the +analyzer's delay timeline: the page records the resolved delay over time, so +each press's `jitter_buffer` ends at the first PCM that reaches the target the +ring held when that PCM landed. The run keeps 60 ms / 128 kbit/s Opus too: a +2026-11-06 Linux run on the default 20 ms Opus at the fixed 300 ms delay had no +underrun or stall but lost 2 of 22 heads ("utterance head was not decoded"), +as the defaults' Linux run below did, while the nightly at 60 ms passes. That +is the ordering defect [Push-to-talk](/quest/m1/push-to-talk/README.md) owns. +What remains is to switch the page to the defaults, measure on `kixtop` again, +and set and enforce the budget. Measured 2026-11-05 on the defaults, before Watch, one run of 22 presses each, Chromium 156 headless through a local relay. `kixtop` is a MacBook Air From 9e32a43372dcebe7dd3c54d4d60911685223a521 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 08:55:13 -0800 Subject: [PATCH 8/8] quest: note push-to-talk lost heads on both frame sizes locally Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 76251def779..a25d6e5208c 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -40,12 +40,13 @@ fails on the defaults by design? [moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) landed the analyzer's delay timeline: the page records the resolved delay over time, so each press's `jitter_buffer` ends at the first PCM that reaches the target the -ring held when that PCM landed. The run keeps 60 ms / 128 kbit/s Opus too: a -2026-11-06 Linux run on the default 20 ms Opus at the fixed 300 ms delay had no -underrun or stall but lost 2 of 22 heads ("utterance head was not decoded"), -as the defaults' Linux run below did, while the nightly at 60 ms passes. That -is the ordering defect [Push-to-talk](/quest/m1/push-to-talk/README.md) owns. -What remains is to switch the page to the defaults, measure on `kixtop` again, +ring held when that PCM landed. The run also keeps 60 ms / 128 kbit/s Opus, +the setup the nightly passes on. On the loaded agent host, 2026-11-06 runs at +the fixed 300 ms delay had no underrun or stall but lost heads ("utterance +head was not decoded"): 2 of 22 on the default 20 ms Opus and 1 on 60 ms. +Whether 20 ms frames hit that ordering defect, which +[Push-to-talk](/quest/m1/push-to-talk/README.md) owns, more often is not +settled. What remains is to switch the page to the defaults, measure on `kixtop` again, and set and enforce the budget. Measured 2026-11-05 on the defaults, before Watch, one run of 22 presses