From a33ec406bf54327c55349ce4d5efdae83ee6df60 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 21:19:00 -0800 Subject: [PATCH 1/7] quest: plan push-to-talk DTX, capture smoothing, and the DTX route spike Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 11 +++++ quest/m1/push-to-talk/capture-smoothing.md | 27 +++++++++++++ quest/m1/push-to-talk/dtx-route.md | 47 ++++++++++++++++++++++ quest/m1/push-to-talk/dtx.md | 43 ++++++++++++++++++++ 4 files changed, 128 insertions(+) create mode 100644 quest/m1/push-to-talk/capture-smoothing.md create mode 100644 quest/m1/push-to-talk/dtx-route.md create mode 100644 quest/m1/push-to-talk/dtx.md diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index f7bd47307cc..ca42a72c495 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -22,6 +22,11 @@ Decided 2026-10-28 (maintainer): - **Hold, not DTX.** DTX is for silence the app can't predict. Push-to-talk knows when it is silent, and a held subscription costs no media bytes and no encoder CPU. DTX is off on main anyway ([#4919](https://github.com/moq-dev/moq/pull/4919)). + **Reversed 2026-11-04 (maintainer):** a catalog toggle makes every listener + wait out the `enabled` flip, so the jitter buffer must hold at least a round + trip. Push-to-talk moves to DTX ([Push-to-talk suppresses + silence](/quest/m1/push-to-talk/dtx.md)), and hold retires when that lands; + until then hold stays on main. - **Players hold a disabled rendition**, audio and video, by default, in every first-party player. Exports keep dropping it, since a held track sends nothing and an interleaving muxer or HLS player may wait on it. The @@ -53,6 +58,12 @@ encoder. Capture now stays open, so only a device swap, a settings change or a new `source` recovers one. Decide whether re-enabling should retry a failed encoder, and record why. +## Required + +- [Pick how the browser publisher suppresses silence](/quest/m1/push-to-talk/dtx-route.md) - measures the DTX routes +- [Push-to-talk suppresses silence instead of toggling the catalog](/quest/m1/push-to-talk/dtx.md) - replaces hold with DTX +- [Browser capture delivers audio frames on a steady cadence](/quest/m1/push-to-talk/capture-smoothing.md) - so `"auto"` measures the network + ## Closes - [#7101](https://github.com/moq-dev/moq/issues/7101) - a disabled audio rendition is deselected, so the start of the next utterance is lost diff --git a/quest/m1/push-to-talk/capture-smoothing.md b/quest/m1/push-to-talk/capture-smoothing.md new file mode 100644 index 00000000000..162d3925396 --- /dev/null +++ b/quest/m1/push-to-talk/capture-smoothing.md @@ -0,0 +1,27 @@ +# [S] Browser capture delivers audio frames on a steady cadence + +## Goal + +`@moq/publish`'s capture graph hands encoded audio frames to the network at +the frame rate, not in bursts, so the listener's `"auto"` jitter target +measures the network instead of the capture graph. On an ideal path the +per-press arrival spread drops well below the ~46 ms measured on +[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760); the PR records +before and after on Chromium and on kixtop. + +## Plan + +Planned 2026-11-04 with the maintainer. Measured on #7760: the capture graph +delivered 60 ms frames 25-109 ms apart, and capture plus framing took 29 ms +median, 110 ms max. In `"auto"` that spread became about 46 ms of measured +jitter per press, padding the target. + +Decided 2026-11-04 (maintainer): reduce it, not just report it. Start by +finding where the bursts form: the AudioWorklet's posting cadence, the capture +AudioContext's callback size (already `latencyHint: "interactive"` in +`js/publish/src/audio/capture.ts`), or batching between the worklet and the +encoder. Fix at the source; don't re-pace on a timer. + +## Related + +- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the estimator that reads this spread diff --git a/quest/m1/push-to-talk/dtx-route.md b/quest/m1/push-to-talk/dtx-route.md new file mode 100644 index 00000000000..078ef8ced1e --- /dev/null +++ b/quest/m1/push-to-talk/dtx-route.md @@ -0,0 +1,47 @@ +# [S] Pick how the browser publisher suppresses silence + +## Goal + +A measured comparison picks the route `@moq/publish` uses to stop sending +silent Opus frames while keeping every chunk on the capture timeline, and the +maintainer approves the pick in the PR. Done when the PR records each route's +numbers and the chosen route with ✅. + +## Plan + +Planned 2026-11-04 with the maintainer, from the +[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) latency +breakdown. Push-to-talk moves from the catalog toggle to DTX (see +[Push-to-talk uses DTX](/quest/m1/push-to-talk/dtx.md)), so the publisher needs +a silence suppression that works in every browser. + +Why WebCodecs DTX alone can't be used today: with `usedtx: true`, Chromium +stamps each Opus chunk as the first input timestamp plus the samples it +emitted, so every suppressed frame pulls later audio earlier (14 s of input +ended at 8.92 s), and no WebCodecs output maps a chunk back to its input +([moq-dev/moq#4783](https://github.com/moq-dev/moq/issues/4783), +[moq-dev/moq#4902](https://github.com/moq-dev/moq/pull/4902)). Firefox kept +the timeline; Safari wasn't measured. `usedtx` was removed in +[moq-dev/moq#4919](https://github.com/moq-dev/moq/pull/4919). + +Decided 2026-11-04 (maintainer): measure before choosing. The routes: + +1. **Gate before the encoder.** The publisher decides silence itself (energy + threshold or VAD) and neither encodes nor sends silent frames, so it keeps + stamping from the input timestamps it already controls. Works with any + encoder. +2. **WASM libopus.** Ship an Opus encoder with real DTX and exact timestamps. + Overlaps the m2 [Opus backend](/quest/m2/audio-opus-backend.md) quest. +3. **Native WebCodecs DTX** in the browsers that keep the timeline, if the + WebCodecs DTX repro (see the epic) shows any do, with route 1 or 2 as the + fallback elsewhere. + +Measure each on Chromium, Firefox and Safari (kixtop): timeline drift over a +tone/silence/noise pattern, bytes sent during silence, onset latency of the +first word after silence (no clipped head), CPU, bundle size, and a listening +check. Record the numbers in the PR and put the pick to the maintainer. + +## Related + +- [Push-to-talk uses DTX](/quest/m1/push-to-talk/dtx.md) - implements the route this picks +- [Opus backend](/quest/m2/audio-opus-backend.md) - the native side's Opus and DTX measurements diff --git a/quest/m1/push-to-talk/dtx.md b/quest/m1/push-to-talk/dtx.md new file mode 100644 index 00000000000..ca235a62ff2 --- /dev/null +++ b/quest/m1/push-to-talk/dtx.md @@ -0,0 +1,43 @@ +# [M] Push-to-talk suppresses silence instead of toggling the catalog + +## Goal + +Muting a push-to-talk publisher keeps its audio rendition enabled and its +subscription live, and sends no media while silent. The next utterance reaches +a listener with no catalog flip and no round trip, and listeners play comfort +noise across the silence without counting it as an underrun. The hold design, +listeners staying subscribed to a disabled rendition, is retired in the same +change. `just test push-to-talk` proves it. + +## Plan + +Planned 2026-11-04 with the maintainer, from the +[moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) latency +breakdown. + +Decided 2026-11-04 (maintainer), reversing the epic's 2026-10-28 "Hold, not +DTX": a catalog toggle makes every listener wait out the `enabled` flip, so +the jitter buffer must hold at least a round trip for push-to-talk to work, +and the maintainer would rather fix DTX and avoid the catalog entirely. + +- **Route:** whichever [Pick how the browser publisher suppresses + silence](/quest/m1/push-to-talk/dtx-route.md) chooses. +- **Hold retires when this lands ✅**, not before, so push-to-talk keeps + working on main meanwhile. Remove the players' default hold of a disabled + rendition and restore the hang spec's "must not select" if nothing else + relies on "should not select"; check video and exports before restoring. +- **Gaps play comfort noise ✅**, not plain silence. A suppressed span is + intentional silence: it counts as no underrun, does not grow the buffer, and + the jitter estimator ignores it + ([Audio jitter target](/quest/m1/audio-jitter-target/README.md) specifies + the estimator; record the exception there too). +- The publisher's mute becomes "suppress", not `enabled: false`. Keep + `enabled` for a real stop (leaving the call), where a catalog change is + correct. + +Update the epic's acceptance and `doc/` pages that describe hold. + +## Required + +- [Pick how the browser publisher suppresses silence](/quest/m1/push-to-talk/dtx-route.md) - the route this implements + From 82da4b628e526c24cc0139f925d02e9b8c29232f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 21:44:33 -0800 Subject: [PATCH 2/7] quest: name the gate-and-restamp workaround, track the Chromium Opus timestamp bug Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/dtx-route.md | 29 +++++++++++++++++++------ quest/m2/README.md | 1 + quest/m2/chromium-opus-timestamps.md | 32 ++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 7 deletions(-) create mode 100644 quest/m2/chromium-opus-timestamps.md diff --git a/quest/m1/push-to-talk/dtx-route.md b/quest/m1/push-to-talk/dtx-route.md index 078ef8ced1e..bd2293c82cf 100644 --- a/quest/m1/push-to-talk/dtx-route.md +++ b/quest/m1/push-to-talk/dtx-route.md @@ -24,17 +24,32 @@ ended at 8.92 s), and no WebCodecs output maps a chunk back to its input the timeline; Safari wasn't measured. `usedtx` was removed in [moq-dev/moq#4919](https://github.com/moq-dev/moq/pull/4919). +Measured 2026-11-04 (repro attached to +[w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986)): with +`usedtx: true`, Chrome 156 (Linux and macOS) ends 14 s of input at 8.92 s, and it +also ignores a jump in input timestamps. Firefox 157 keeps the timeline under +DTX but ignores input jumps. Safari 26.6.2 follows input jumps but never +applies `usedtx`. The spec requires the input's timestamp (WebCodecs §5.6) and +says nothing about DTX; the spec issue asks for both, and the maintainer +submits the Chromium bug. + Decided 2026-11-04 (maintainer): measure before choosing. The routes: -1. **Gate before the encoder.** The publisher decides silence itself (energy - threshold or VAD) and neither encodes nor sends silent frames, so it keeps - stamping from the input timestamps it already controls. Works with any - encoder. +1. **Gate before the encoder, stamp from our own inputs ✅ as the workaround.** + The publisher decides silence itself (energy threshold or VAD) and neither + encodes nor sends silent frames. Inputs are already cut to one codec frame + each (`js/publish/src/audio/encoder.ts`, "Align those inputs to codec + frames"), and without DTX every browser emits one chunk per input in order, + so each chunk is stamped from a queue of the input timestamps fed, not from + `chunk.timestamp`. That sidesteps both Chromium bugs with no encoder + restart (today's code restarts the encoder on any input gap to re-base, + which also works but costs a re-prime per utterance). Works in every + browser. 2. **WASM libopus.** Ship an Opus encoder with real DTX and exact timestamps. Overlaps the m2 [Opus backend](/quest/m2/audio-opus-backend.md) quest. -3. **Native WebCodecs DTX** in the browsers that keep the timeline, if the - WebCodecs DTX repro (see the epic) shows any do, with route 1 or 2 as the - fallback elsewhere. +3. **Native WebCodecs DTX** where it keeps the timeline: only Firefox today, + so at best a later optimization once Chromium is fixed, with route 1 or 2 + as the fallback. Measure each on Chromium, Firefox and Safari (kixtop): timeline drift over a tone/silence/noise pattern, bytes sent during silence, onset latency of the diff --git a/quest/m2/README.md b/quest/m2/README.md index b0aea4795ba..2cd76747794 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -288,3 +288,4 @@ lite-07; bitrate enforcement is not part of this release. - [The dev shell builds Kotlin with Gradle 9](/quest/m2/kt-gradle-9.md) - `gradle_9` in the dev shell, so the Android target builds there - [The publish support panel lists only codecs publishing accepts](/quest/m2/publish-support-probe.md) - the settings panel runs the publisher's Annex B encoder probe - [A hung Python test fails instead of hanging the suite](/quest/m2/py-test-timeouts.md) - one pytest-timeout bound covers every `py/` test +- [Chromium's Opus encoder stamps chunks from their input](/quest/m2/chromium-opus-timestamps.md) - waiting on Chromium; unlocks native `usedtx` for push-to-talk diff --git a/quest/m2/chromium-opus-timestamps.md b/quest/m2/chromium-opus-timestamps.md new file mode 100644 index 00000000000..566abd79a74 --- /dev/null +++ b/quest/m2/chromium-opus-timestamps.md @@ -0,0 +1,32 @@ +# [XS] Chromium's Opus encoder stamps chunks from their input + +## Goal + +Chromium's WebCodecs `AudioEncoder` stamps each Opus chunk with the timestamp +of the input audio it encodes, both across `usedtx` silence and across a jump +in input timestamps. Check by running the repro attached to +[w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986) in a released +Chrome: scenario A ends near 13.98 s with steps across suppressed spans, and +scenario C follows the +1 s jump. Delete this quest once a stable Chrome +passes, and tell [Pick how the browser publisher suppresses +silence](/quest/m1/push-to-talk/dtx-route.md)'s successor that native +`usedtx` (its route 3) is usable there. + +## Plan + +Waiting on Chromium. Measured 2026-11-04: Chrome 156 ends 14 s of DTX input +at 8.92 s and ignores input timestamp jumps. In +`media/audio/audio_opus_encoder.cc`, the timestamp tracker is seeded once from +the first input and advances only when a packet is emitted, so DTX's +non-transmitted frames and input gaps are both lost. WebCodecs §5.6 requires +the input's timestamp; the spec issue asks it to also define DTX. + +Decided 2026-11-04 (maintainer): file the Chromium bug (Blink>Media>WebCodecs, +submitted by the maintainer) and the spec issue; track it here in m2, since +push-to-talk works around it by gating silence before the encoder and stamping +chunks from its own inputs, so nothing depends on this. Advance it by pinging +the Chromium bug when a release passes the repro or a fix lands. + +## Related + +- [Pick how the browser publisher suppresses silence](/quest/m1/push-to-talk/dtx-route.md) - the workaround and the native-DTX route this unlocks From eb27ed94cb8ef4dadbdd18a93f9ef2f12e34ae1f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 21:54:30 -0800 Subject: [PATCH 3/7] quest: the DTX timestamp bug is Chromium's, not the spec's Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/dtx-route.md | 10 +++++----- quest/m2/chromium-opus-timestamps.md | 15 ++++++++------- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/quest/m1/push-to-talk/dtx-route.md b/quest/m1/push-to-talk/dtx-route.md index bd2293c82cf..6d8dd103e14 100644 --- a/quest/m1/push-to-talk/dtx-route.md +++ b/quest/m1/push-to-talk/dtx-route.md @@ -24,14 +24,14 @@ ended at 8.92 s), and no WebCodecs output maps a chunk back to its input the timeline; Safari wasn't measured. `usedtx` was removed in [moq-dev/moq#4919](https://github.com/moq-dev/moq/pull/4919). -Measured 2026-11-04 (repro attached to -[w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986)): with +Measured 2026-11-04 (repro in the Chromium bug tracked by +[Chromium's Opus encoder stamps chunks from their +input](/quest/m2/chromium-opus-timestamps.md)): with `usedtx: true`, Chrome 156 (Linux and macOS) ends 14 s of input at 8.92 s, and it also ignores a jump in input timestamps. Firefox 157 keeps the timeline under DTX but ignores input jumps. Safari 26.6.2 follows input jumps but never -applies `usedtx`. The spec requires the input's timestamp (WebCodecs §5.6) and -says nothing about DTX; the spec issue asks for both, and the maintainer -submits the Chromium bug. +applies `usedtx`. WebCodecs §5.6 already requires the input's timestamp, so +this is a Chromium bug, not a spec gap; the maintainer files it. Decided 2026-11-04 (maintainer): measure before choosing. The routes: diff --git a/quest/m2/chromium-opus-timestamps.md b/quest/m2/chromium-opus-timestamps.md index 566abd79a74..50e548facd2 100644 --- a/quest/m2/chromium-opus-timestamps.md +++ b/quest/m2/chromium-opus-timestamps.md @@ -4,10 +4,9 @@ Chromium's WebCodecs `AudioEncoder` stamps each Opus chunk with the timestamp of the input audio it encodes, both across `usedtx` silence and across a jump -in input timestamps. Check by running the repro attached to -[w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986) in a released -Chrome: scenario A ends near 13.98 s with steps across suppressed spans, and -scenario C follows the +1 s jump. Delete this quest once a stable Chrome +in input timestamps. Check by running the repro attached to the Chromium bug +in a released Chrome: scenario A ends near 13.98 s with steps across +suppressed spans, and scenario C follows the +1 s jump. Delete this quest once a stable Chrome passes, and tell [Pick how the browser publisher suppresses silence](/quest/m1/push-to-talk/dtx-route.md)'s successor that native `usedtx` (its route 3) is usable there. @@ -18,11 +17,13 @@ Waiting on Chromium. Measured 2026-11-04: Chrome 156 ends 14 s of DTX input at 8.92 s and ignores input timestamp jumps. In `media/audio/audio_opus_encoder.cc`, the timestamp tracker is seeded once from the first input and advances only when a packet is emitted, so DTX's -non-transmitted frames and input gaps are both lost. WebCodecs §5.6 requires -the input's timestamp; the spec issue asks it to also define DTX. +non-transmitted frames and input gaps are both lost. WebCodecs §5.6 already +requires the input's timestamp, so this is a Chromium bug; a spec issue +([w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986)) was opened +and closed by the maintainer for that reason. Decided 2026-11-04 (maintainer): file the Chromium bug (Blink>Media>WebCodecs, -submitted by the maintainer) and the spec issue; track it here in m2, since +submitted by the maintainer) and no spec issue; track it here in m2, since push-to-talk works around it by gating silence before the encoder and stamping chunks from its own inputs, so nothing depends on this. Advance it by pinging the Chromium bug when a release passes the repro or a fix lands. From ceffde6cde550438b7d7ea2186467fcfe47c1c1f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 22:46:18 -0800 Subject: [PATCH 4/7] quest: link the Chromium Opus timestamp bug Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/dtx-route.md | 5 +++-- quest/m2/chromium-opus-timestamps.md | 8 +++++--- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/quest/m1/push-to-talk/dtx-route.md b/quest/m1/push-to-talk/dtx-route.md index 6d8dd103e14..f41b91c5a10 100644 --- a/quest/m1/push-to-talk/dtx-route.md +++ b/quest/m1/push-to-talk/dtx-route.md @@ -24,14 +24,15 @@ ended at 8.92 s), and no WebCodecs output maps a chunk back to its input the timeline; Safari wasn't measured. `usedtx` was removed in [moq-dev/moq#4919](https://github.com/moq-dev/moq/pull/4919). -Measured 2026-11-04 (repro in the Chromium bug tracked by +Measured 2026-11-04 (repro in +[chromium#581458361](https://issues.chromium.org/issues/581458361), tracked by [Chromium's Opus encoder stamps chunks from their input](/quest/m2/chromium-opus-timestamps.md)): with `usedtx: true`, Chrome 156 (Linux and macOS) ends 14 s of input at 8.92 s, and it also ignores a jump in input timestamps. Firefox 157 keeps the timeline under DTX but ignores input jumps. Safari 26.6.2 follows input jumps but never applies `usedtx`. WebCodecs §5.6 already requires the input's timestamp, so -this is a Chromium bug, not a spec gap; the maintainer files it. +this is a Chromium bug, not a spec gap. Decided 2026-11-04 (maintainer): measure before choosing. The routes: diff --git a/quest/m2/chromium-opus-timestamps.md b/quest/m2/chromium-opus-timestamps.md index 50e548facd2..d13f3e4d75c 100644 --- a/quest/m2/chromium-opus-timestamps.md +++ b/quest/m2/chromium-opus-timestamps.md @@ -4,8 +4,9 @@ Chromium's WebCodecs `AudioEncoder` stamps each Opus chunk with the timestamp of the input audio it encodes, both across `usedtx` silence and across a jump -in input timestamps. Check by running the repro attached to the Chromium bug -in a released Chrome: scenario A ends near 13.98 s with steps across +in input timestamps. Check by running the repro attached to +[chromium#581458361](https://issues.chromium.org/issues/581458361) in a released +Chrome: scenario A ends near 13.98 s with steps across suppressed spans, and scenario C follows the +1 s jump. Delete this quest once a stable Chrome passes, and tell [Pick how the browser publisher suppresses silence](/quest/m1/push-to-talk/dtx-route.md)'s successor that native @@ -22,7 +23,8 @@ requires the input's timestamp, so this is a Chromium bug; a spec issue ([w3c/webcodecs#986](https://github.com/w3c/webcodecs/issues/986)) was opened and closed by the maintainer for that reason. -Decided 2026-11-04 (maintainer): file the Chromium bug (Blink>Media>WebCodecs, +Decided 2026-11-04 (maintainer): file the Chromium bug +([chromium#581458361](https://issues.chromium.org/issues/581458361), Blink>Media>WebCodecs, submitted by the maintainer) and no spec issue; track it here in m2, since push-to-talk works around it by gating silence before the encoder and stamping chunks from its own inputs, so nothing depends on this. Advance it by pinging From 6b1df2021a2300500d5cdafb1d0d3ca7bbf736da Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 23:10:32 -0800 Subject: [PATCH 5/7] quest: budget push-to-talk latency on the defaults, after DTX and steady capture Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index c46e3b1edad..7b8a1554137 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -1,11 +1,12 @@ -# [S] Push-to-talk's press-to-audible latency is chosen, not inherited +# [M] Push-to-talk's press-to-audible latency fits a budget on the defaults ## Goal -Each part of a push-to-talk press's latency on an ideal path, from capture to -device output, is either reduced through the knob that controls it or kept -with a recorded reason. `just test push-to-talk` reports the result with the -same breakdown. +On an ideal path with the browser defaults (`` `delay: "auto"`, +20 ms Opus), push-to-talk's press-to-audible latency, excluding device output, +stays under a budget set from a real-device measurement on `kixtop`, and +`just test push-to-talk` enforces it with the per-part breakdown. Native +playback is out of scope. ## Plan @@ -67,8 +68,24 @@ network, would make the measured term nonzero; that is the estimator working as specified in [Audio jitter target](/quest/m1/audio-jitter-target/README.md). -Open: decide which knob to turn first, and whether to measure on a real -device before deciding. +Decided 2026-11-04 (maintainer, planning in +[moq-dev/moq#7816](https://github.com/moq-dev/moq/pull/7816)): + +- ✅ Budget the default path with realistic settings, not the fixed 300 ms / + 60 ms setup above; drop the fixed-delay row from `just test push-to-talk`. + The maintainer is fine assuming the constants are the problem. +- ✅ Measure capture and output latency on real macOS hardware (`kixtop`) + first; set the budget from those numbers, and keep output-side knobs until + then. +- ✅ Browser only. +- The catalog toggle forces at least a round trip of jitter buffer, so the + budget is measured once push-to-talk runs on DTX (Required below), and once + capture delivers frames steadily. + +## Required + +- [Push-to-talk suppresses silence instead of toggling the catalog](/quest/m1/push-to-talk/dtx.md) - removes the catalog round trip the budget would otherwise carry +- [Browser capture delivers audio frames on a steady cadence](/quest/m1/push-to-talk/capture-smoothing.md) - so `"auto"` measures the network, not capture bursts ## Related From dc10c562269777fb9913ae39939f5495c99e2f57 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 06:13:27 -0800 Subject: [PATCH 6/7] quest: route 1 is a confirmed workaround, not the pick; refresh m1 index MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address review: dtx-route.md no longer marks the gate-and-restamp route with the pick's ✅, dates Safari's first measurement, and the m1 index describes the reworked push-to-talk epic and latency quest. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 4 ++-- quest/m1/push-to-talk/dtx-route.md | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 9136e7589c9..3f94db8b1a8 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -165,8 +165,8 @@ their end report, so an embedder's drain alert pages on real cuts. - [A/V clock](/quest/m1/av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor -- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - consumers hold a disabled rendition, and an unmute encodes at once with the mic still open; a reordered head after a long mute is still lost at the listener -- [Push-to-talk's press-to-audible latency is chosen, not inherited](/quest/m1/push-to-talk-latency.md) - each part of an ideal-path press's 469 ms, chiefly the 300 ms delay and 115 ms device output, is reduced through its knob or kept with a reason +- [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - an unmute is heard from its first frame with no catalog round trip: mute moves from holding a disabled rendition to suppressed silence with comfort noise, after a route spike; capture delivers frames on a steady cadence +- [Push-to-talk's press-to-audible latency fits a budget on the defaults](/quest/m1/push-to-talk-latency.md) - press to audible on the browser defaults, excluding device output, stays under a kixtop-measured budget that `just test push-to-talk` enforces - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [CI housekeeping](/quest/m1/ci-housekeeping.md) - remove the actionlint queue ignore once a release after 1.7.12 reaches the dev shell diff --git a/quest/m1/push-to-talk/dtx-route.md b/quest/m1/push-to-talk/dtx-route.md index f41b91c5a10..7120d97583c 100644 --- a/quest/m1/push-to-talk/dtx-route.md +++ b/quest/m1/push-to-talk/dtx-route.md @@ -21,7 +21,8 @@ emitted, so every suppressed frame pulls later audio earlier (14 s of input ended at 8.92 s), and no WebCodecs output maps a chunk back to its input ([moq-dev/moq#4783](https://github.com/moq-dev/moq/issues/4783), [moq-dev/moq#4902](https://github.com/moq-dev/moq/pull/4902)). Firefox kept -the timeline; Safari wasn't measured. `usedtx` was removed in +the timeline; Safari wasn't measured until the 2026-11-04 repro below. +`usedtx` was removed in [moq-dev/moq#4919](https://github.com/moq-dev/moq/pull/4919). Measured 2026-11-04 (repro in @@ -36,9 +37,10 @@ this is a Chromium bug, not a spec gap. Decided 2026-11-04 (maintainer): measure before choosing. The routes: -1. **Gate before the encoder, stamp from our own inputs ✅ as the workaround.** - The publisher decides silence itself (energy threshold or VAD) and neither - encodes nor sends silent frames. Inputs are already cut to one codec frame +1. **Gate before the encoder, stamp from our own inputs.** Confirmed + 2026-11-04 as a workaround for the Chromium bug; it is measured like the + others and is not yet the pick. The publisher decides silence itself + (energy threshold or VAD) and neither encodes nor sends silent frames. Inputs are already cut to one codec frame each (`js/publish/src/audio/encoder.ts`, "Align those inputs to codec frames"), and without DTX every browser emits one chunk per input in order, so each chunk is stamped from a queue of the input timestamps fed, not from From 8ba6a58719f66dd1097f9fca311b075f25530037 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 06:15:22 -0800 Subject: [PATCH 7/7] quest: list the Chromium Opus timestamp fix under Waiting Co-Authored-By: Claude Opus 5.5 --- quest/m2/chromium-opus-timestamps.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/quest/m2/chromium-opus-timestamps.md b/quest/m2/chromium-opus-timestamps.md index d13f3e4d75c..75ad61d0555 100644 --- a/quest/m2/chromium-opus-timestamps.md +++ b/quest/m2/chromium-opus-timestamps.md @@ -14,7 +14,7 @@ silence](/quest/m1/push-to-talk/dtx-route.md)'s successor that native ## Plan -Waiting on Chromium. Measured 2026-11-04: Chrome 156 ends 14 s of DTX input +Measured 2026-11-04: Chrome 156 ends 14 s of DTX input at 8.92 s and ignores input timestamp jumps. In `media/audio/audio_opus_encoder.cc`, the timestamp tracker is seeded once from the first input and advances only when a packet is emitted, so DTX's @@ -27,8 +27,15 @@ Decided 2026-11-04 (maintainer): file the Chromium bug ([chromium#581458361](https://issues.chromium.org/issues/581458361), Blink>Media>WebCodecs, submitted by the maintainer) and no spec issue; track it here in m2, since push-to-talk works around it by gating silence before the encoder and stamping -chunks from its own inputs, so nothing depends on this. Advance it by pinging -the Chromium bug when a release passes the repro or a fix lands. +chunks from its own inputs, so nothing depends on this. + +## Waiting + +- Chromium: fix [chromium#581458361](https://issues.chromium.org/issues/581458361) + so Opus chunks carry their input's timestamp across `usedtx` silence and + input jumps. Check by running the attached repro in the latest stable Chrome + (scenario A ends near 13.98 s, scenario C follows the +1 s jump); advance it + by commenting on the bug with each stable release that still fails. ## Related