From 02d58349898c5a8c96aab13f507a44660eee2d6b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 15:22:28 -0800 Subject: [PATCH] quest: plan the audio stretch model and its grading Audio slows down when its buffer runs low and speeds up once it refills, in every player, replacing the rule that the browser only plays faster. The harness is regraded first so no gate rewards zero underruns or zero lag. Co-Authored-By: Claude --- quest/m1/README.md | 2 +- quest/m1/audio-jitter-target/README.md | 71 ++++++-- .../audio-jitter-target/browser-ceilings.md | 56 ------ .../audio-jitter-target/buffered-converge.md | 65 ------- quest/m1/audio-jitter-target/calibrate.md | 59 +++++++ quest/m1/audio-jitter-target/grading.md | 83 +++++++++ quest/m1/audio-jitter-target/native-av-lag.md | 73 +++----- .../m1/audio-jitter-target/native-stretch.md | 163 ------------------ .../audio-jitter-target/opus-stalled-share.md | 25 ++- .../m1/audio-jitter-target/stretch-setting.md | 61 +++++++ quest/m1/audio-jitter-target/stretch.md | 141 +++++++++++++++ quest/m1/push-to-talk-latency.md | 9 +- quest/m4/audio-bufferbloat-av.md | 5 +- 13 files changed, 451 insertions(+), 362 deletions(-) delete mode 100644 quest/m1/audio-jitter-target/browser-ceilings.md delete mode 100644 quest/m1/audio-jitter-target/buffered-converge.md create mode 100644 quest/m1/audio-jitter-target/calibrate.md create mode 100644 quest/m1/audio-jitter-target/grading.md delete mode 100644 quest/m1/audio-jitter-target/native-stretch.md create mode 100644 quest/m1/audio-jitter-target/stretch-setting.md create mode 100644 quest/m1/audio-jitter-target/stretch.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 0a7773560f9..0ea10e549fc 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -38,7 +38,7 @@ moves what remains of them once their claims merge. - [Group errors are cached by kind](/quest/m1/group-error-cache.md) - permanent group errors bind every reader, retryable ones are kept only for a bounded time, and a live SUBSCRIBE_DROP or FETCH answered `Old` no longer caches it for good - [The overcommitted receive-window cells run again](/quest/m1/receive-windows-evicted.md) - `benchmark_receive_windows` measures its 256-stream and 4 MiB-frame cells again, reporting delivered frames and evictions instead of stalling - [Stream credit by priority](/quest/m1/stream-credit/README.md) - every stream opens through one priority entry point, and a higher-priority group takes over or resets a lower one when credit runs out -- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; time-stretched browser playback without steady-state underruns and a native trace replay remain +- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; audio that slows down and catches up around it in every player, graded in that direction, and a native trace replay remain - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor - [Push-to-talk's press-to-audible latency fits a budget on the defaults](/quest/m1/push-to-talk-latency.md) - press to audible on the browser defaults, excluding device output, stays under a kixtop-measured budget that `just test push-to-talk` enforces - [Lifecycles are explicit state](/quest/m1/explicit-state/README.md) - every lifecycle the 2026-11-07 audit found implicit is one enum or union, fixing the bugs in the 11 lifecycles it redesigns, with review and agent rules against new ones diff --git a/quest/m1/audio-jitter-target/README.md b/quest/m1/audio-jitter-target/README.md index b025813de04..ecf0dde57ea 100644 --- a/quest/m1/audio-jitter-target/README.md +++ b/quest/m1/audio-jitter-target/README.md @@ -11,12 +11,19 @@ at once got a target far too shallow to play through the next flush. One written algorithm, cited by both implementations, produces the same target from the same arrival trace. -Boundaries: both players converge on the target with NetEq's time stretch: -native playback by playing slightly faster or slower, and the browser only -faster, since its ring starts on the target and slowing down only settles audio -behind the shared clock. Underruns past the target are accepted, not covered -with latency. No packet loss concealment. Video keeps its own target; the -shared clock stays on local time, and the audio ring's stretch converges on it. +Every player follows one rule around that target, with NetEq's time stretch. +In steady state audio plays on the shared clock at the playout delay, which is +the expected resting point for audio and video alike. When the audio received +ahead of the playhead runs low, playback slows down and audio falls behind the +clock; once the buffer refills past the target, playback speeds up until audio +is back on it. Congestion then costs a gap and a catch-up instead of the audio +that arrived late. + +Boundaries: zero underruns is not a goal, and neither the target nor a fixed +delay is raised to reach it. No packet loss concealment. Video stays on the +shared clock and shows a late frame when it arrives, so it recovers on its +own. Video measures no network term: the shared delay is the largest +requirement among the subscribed renditions, so video waits for audio's. ## Plan @@ -35,14 +42,39 @@ target, in both players from one algorithm and its vectors: [moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856) natively and [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) in the browser's ring. -Decided 2026-11-08 (maintainer, on -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058)): never add -latency to chase zero underruns. Audio and video keep separate clocks, which -align at steady state once the delay is right, so the browser's stretch only -steers audio back onto the clock and never settles it later to absorb jitter. -Some underruns during congestion are expected. The replay rows budget -`av_lag_ms`, how far audio trails the shared clock, exactly as measured, so a -change that buys underruns with latency fails them. +Decided 2026-11-09 (maintainer, /quest-plan): the rule in the Goal. It +replaces the 2026-11-08 decision on +[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) that the browser +only plays faster and audio never settles later. Video gets the effect for +free, since a late frame shows at once and the picture is live again; audio +cannot, so without a slow-down and a catch-up a stall drops whatever audio +arrives late. What carries over from 2026-11-08: latency is never added to +reach zero underruns, and some underruns during congestion are expected. + +- Players: ✅ `moq play`, a live ``, and a buffered one, on one rule + (rejected: live only, with buffered mode keeping its own). +- Target: ✅ unchanged. The shared delay is + `max over subscribed renditions of (delay + target)`, where audio's target + is `max(measured, jitter) + frame` and video's is its catalog span + (rejected: a network term on every rendition; the plain sum + `delay + jitter + frame + measured`, which buffers a batched publisher's + batch twice; a catalog-only target). +- Video: ✅ on the same clock as audio, and audio alone speeds up and slows + down with the network (rejected: video at its catalog target, with audio + trailing it by the measured lateness). +- Target moves: ✅ the estimator stays as it is, and audio glides to each new + target by stretching (rejected: a slew limit on the measured term, which + the spec already declined as a second knob; deleting the estimator and + letting the lag be the only network term, which underruns a 20 ms Opus + target until the lag builds). +- Bound: ✅ configurable, as + [the `stretch` setting](/quest/m1/audio-jitter-target/stretch-setting.md) + records. +- Tests: no gate may reward moving away from the rule, so + [the grading quest](/quest/m1/audio-jitter-target/grading.md) lands before + [the model](/quest/m1/audio-jitter-target/stretch.md). +- Milestone: ✅ every new child in m1, here (rejected: the model and grading + in m0; the setting in m2). Decided 2026-11-04 (maintainer), for push-to-talk's silence suppression ([moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942)): a silence the publisher declares @@ -57,13 +89,14 @@ native playback against its budgets. ## Required -- [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - the chromium rows' ceilings re-measured from seven Nightlies with the browser's time stretch, budgeting `player_skips_per_min` -- [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - `moq play` copies NetEq's accelerate and preemptive expand, sharing the browser's seams and vectors, so its native rows play with no audible glitch once the target settles -- [Buffered converge](/quest/m1/audio-jitter-target/buffered-converge.md) - a buffered `` time-stretches its audio back onto the shared clock after a stall or drift, so lip sync returns and no audio group is dropped +- [Grading](/quest/m1/audio-jitter-target/grading.md) - the audio harness grades steady-state lag, dropped audio, and recovery from a stall, with no gate fixed at zero underruns or zero lag +- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - every player slows down when its buffer runs low and speeds up once it refills, by one rule and one set of vectors, buffered mode included +- [The `stretch` setting](/quest/m1/audio-jitter-target/stretch-setting.md) - a player sets how far audio may trail before it skips, and zero turns the stretch off +- [Calibrate](/quest/m1/audio-jitter-target/calibrate.md) - every browser and native ceiling re-measured from seven Nightlies with the model - [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the AAC browser budgets are re-measured from nightlies after the TS importer groups each PES -- [Native A/V lag](/quest/m1/audio-jitter-target/native-av-lag.md) - `moq play` reports how far audio trails its presentation clock, and native stops expanding into the gap between arrivals if it settles audio late +- [Native A/V lag](/quest/m1/audio-jitter-target/native-av-lag.md) - `moq play --metrics` reports how far audio trails its presentation clock, so native rows grade `av_lag_ms` - [Native rendition delay](/quest/m1/audio-jitter-target/native-rendition-delay.md) - `moq play` holds each subscribed rendition's catalog `delay` plus its target, as the browser does -- [Chromium Opus stalled share](/quest/m1/audio-jitter-target/opus-stalled-share.md) - the Chromium Opus browser rows stay under the 0.01 stalled-share budget every night +- [Chromium Opus stalled share](/quest/m1/audio-jitter-target/opus-stalled-share.md) - the Chromium Opus browser rows' stalled-share ceiling is measured with the stretch, and passes ## Closes diff --git a/quest/m1/audio-jitter-target/browser-ceilings.md b/quest/m1/audio-jitter-target/browser-ceilings.md deleted file mode 100644 index 5a12c4ab6b2..00000000000 --- a/quest/m1/audio-jitter-target/browser-ceilings.md +++ /dev/null @@ -1,56 +0,0 @@ -# [S] Re-measure the chromium audio ceilings with the browser's time stretch - -## Goal - -`test/audio-quality/budgets.json` carries every chromium row's ceilings -re-measured, by the rule in its note, from seven scheduled Nightly runs on -`main` that all ran with the browser's NetEq time stretch, and budgets -`player_skips_per_min` and `av_lag_ms` on those rows beside the tap metrics. No chromium row -skips buffered audio in steady state: its `player_skips_per_min` stays at the -startup re-anchors the estimator's hold releases, as on the replay rows. What -still runs dry is what NetEq would conceal, a receive stall longer than the -target, and its underrun ceilings are measured rather than fixed at zero. - -## Waiting - -- moq-dev/moq: seven scheduled Nightly runs on `main` after - [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) merges. Find them - with `gh run list --workflow nightly.yml --event schedule` and download each - with `gh run download -n audio-quality-run`; never dispatch the nightly by - hand. The artifacts expire after 7 days, so download each run's as it lands - rather than waiting for the seventh. - -## Plan - -Decided 2026-11-07 (maintainer, /quest-complete on -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058)): ✅ land the -stretch, then measure (rejected: hold the PR until the Nightly ceilings are in). -The chromium rows can only be measured once the stretch is on `main`, so this -quest carries that measurement, and its Goal is judged by it and the replay -rows alone, with no maintainer listening run (decided 2026-11-07, -/quest-unblock). - -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) budgeted every -replay row exactly as measured, A/V lag included, and left the browser -ceilings unchanged. The browser's stretch never slows down, so underruns stay -where the trim had them (`relay-mic-rtt40` 10.6 `player_underruns_per_min` -against 8.2) while the audio the trim dropped plays faster instead, and the -A/V lag rests about one frame ahead of the clock. Expect the chromium rows -alike: re-stalls about as before, `player_skips` near zero, and -`skip_aheads` and `discarded_ms`, which infer a skip from a step in the -playhead's lag, reading a fast accelerate after a receive stall as a skip. `player_skips` counts the ring's real -re-anchors, so a rise there is a regression worth reporting before budgeting -it, as is an `av_lag_ms` p50 above zero, which is audio settling behind the -clock. Rewrite the note's browser sentences to the new nights and say the stretch -is in them. - -[moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308) adds the -player's own `trimmed_ms` and `accelerated_ms` to every row. The native -calibration budgets `trimmed_ms_per_min` beside `player_skips_per_min`, as the -maintainer decided there; budgeting it here too keeps the two runtimes graded -alike. - -## Related - -- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - re-measures the AAC browser rows from nightlies too; the same seven nights can serve both -- [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - the native rows' re-measure after the same stretch diff --git a/quest/m1/audio-jitter-target/buffered-converge.md b/quest/m1/audio-jitter-target/buffered-converge.md deleted file mode 100644 index b418840c46c..00000000000 --- a/quest/m1/audio-jitter-target/buffered-converge.md +++ /dev/null @@ -1,65 +0,0 @@ -# [M] Buffered audio converges on the shared clock - -## Goal - -A `` with a nonzero `lookahead` keeps lip sync in steady state: once -a stall, a rise of the `"auto"` delay, or publisher clock drift has left the -audio playhead behind the shared clock, the ring time-stretches back onto it. -It never skips, so a TTS utterance still plays every word. Today buffered -mode turns off the ring's catch-up (`playing = !this.#stalled && -!this.#buffered` in `js/watch/src/audio/ring-buffer.ts`), so any lag is -permanent. Audio trails video for the rest of the stream, and once the lag -passes `delay + lookahead` the container consumer drops whole audio groups -("skipping slow groups"), which is the cut-out reported in -[moq-dev/moq#8031](https://github.com/moq-dev/moq/issues/8031). - -Boundaries: video already follows the shared clock and is unchanged. The -clock is not driven by audio, and a slow audio track never holds video. Live -mode (`lookahead` of zero) keeps its own catch-up. Native playback has no -buffered mode. - -## Plan - -Planned 2026-11-06 (maintainer, /quest-import of -[moq-dev/moq#8031](https://github.com/moq-dev/moq/issues/8031)). A -simulation on main with the real `Sync`, container `Consumer`, `Backpressure` -and `AudioRingBuffer` (a copy of `js/watch/src/audio/replay.ts` with buffered -mode on) measured audio behind video: about -12 ms live versus +105 ms -buffered under `"auto"` with jittery then calm phases; 0 versus +300 ms after -400 ms stalls with `delay=100ms lookahead=500ms`; and a publisher clock 1% fast -climbing to 1.4 s of lag with 2926 slow-group skips. The reporter's other -claim, that a delay above 500 ms breaks lip sync, did not reproduce (300, 800 -and 1500 ms all held about -22 ms). - -Decisions: - -- Direction: ✅ converge buffered audio by time-stretching / won't fix, - reply and close / sharpen docs and warn only. Steady-state lip sync comes - from each track converging on the shared clock, and buffered mode broke - that even for TTS with video. -- Milestone: ✅ m1, in this epic / m3. - -Recommendations: - -- Reuse the ring's stretch (`js/watch/src/audio/stretch.ts`) rather - than a second mechanism: buffered mode measures the playhead's lag behind - the shared clock and accelerates until it is back within the floor. It - never stretches ahead of the clock, since future-dated frames are - buffered mode's purpose. -- Extend the replay harness with buffered rows (the stall and the drift - scenarios above), budgeting audio-behind-video and slow-group skips, and - keep a TTS row asserting no frame is dropped. -- Update the `lookahead` docs in `js/watch/src/element.ts`, - `js/watch/src/sync.ts` and `doc/lib/js/watch.md` to say that a buffered - player catches up after a stall by playing slightly faster. - -Public API: none expected. Wire: none. - -## Closes - -- [#8031](https://github.com/moq-dev/moq/issues/8031) - buffered playback loses lip sync and cuts audio out on a live stream - -## Related - -- [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - measures on the nightly runner the ring's time stretch this reuses, which landed in [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) -- [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - the other buffered-mode change to the audio path diff --git a/quest/m1/audio-jitter-target/calibrate.md b/quest/m1/audio-jitter-target/calibrate.md new file mode 100644 index 00000000000..f68022a45df --- /dev/null +++ b/quest/m1/audio-jitter-target/calibrate.md @@ -0,0 +1,59 @@ +# [S] Re-measure every audio ceiling after the stretch model lands + +## Goal + +`test/audio-quality/budgets.json` carries every browser and native row's +ceilings re-measured, by the rule in its note, from seven scheduled Nightly +runs on a `main` that includes +[the stretch model](/quest/m1/audio-jitter-target/stretch.md). The rows budget +steady-state lag, the player's own trims and skips, and underruns as +measured, and the note says the model is in them. + +## Plan + +Planned 2026-11-09 (maintainer, /quest-plan). This replaces two waits that +were calibrating a player the model changes: the chromium re-measure after +[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) and the native +one after [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308). Nights +before the model merges do not count. + +Carried decisions: + +- 2026-11-07 (maintainer, on #8058): ✅ land the player change, then measure + (rejected: hold it until Nightly ceilings are in). The goal is judged by + the rows alone, with no listening run. +- 2026-11-09 (maintainer, on + [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308#issuecomment-6730322286)): + ✅ budget `trimmed_ms_per_min` and `player_skips_per_min` (rejected: + budgeting accelerates as discarded audio), and ✅ the nightly runner's + `bench-playback` counts as the quiet-host run (rejected: naming another + host). + +The rule: the worst of seven clean nights times 1.25, plus the documented +runner hiccup, rounded to two significant figures. A developer run is not +that calibration. Keep each night's rows in the PR body. + +What to report rather than budget: a steady-state `av_lag_ms` p50 more than a +frame above zero on any row but `step`, or a nonzero `player_skips_per_min` +on a row whose stalls fit inside the `stretch` bound. Either means the model +is not doing its job there. + +Native rows stay report-only; enforcing them belongs to +[Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md). + +## Waiting + +- moq-dev/moq: seven scheduled Nightly runs on `main` after the stretch model + merges. Find them with `gh run list --workflow nightly.yml --event + schedule` and download each with `gh run download -n + audio-quality-run` as it lands, since artifacts expire after 7 days. Never + dispatch the nightly by hand. + +## Required + +- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - the player these ceilings measure + +## Related + +- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - re-measures the AAC browser rows after the TS importer change; if it has not finished when the model lands, these seven nights serve both +- [Chromium Opus stalled share](/quest/m1/audio-jitter-target/opus-stalled-share.md) - the one ceiling failing nightly today diff --git a/quest/m1/audio-jitter-target/grading.md b/quest/m1/audio-jitter-target/grading.md new file mode 100644 index 00000000000..a92db0201bf --- /dev/null +++ b/quest/m1/audio-jitter-target/grading.md @@ -0,0 +1,83 @@ +# [M] The audio harness grades the stretch model + +## Goal + +No audio test, budget, or benchmark rewards a player for moving away from the +[epic's model](/quest/m1/audio-jitter-target/README.md). On today's player, +before the model lands, `test/audio-quality` grades: + +- Steady-state lag: every row, native and browser included, carries + `av_lag_ms` p50, so latency can no longer buy fewer underruns unnoticed. +- Temporary lag passes: p95 lag is a ceiling with measured headroom on the + browser and native rows, and no note calls audio trailing through a stall a + regression. +- Dropped audio is the player's own count (`trimmed_ms`, `player_skips`). + Speeding up is not a skip. +- No gate is fixed at zero underruns or zero stalls. +- A congestion row, where the path stalls for longer than the target and + then delivers the backlog, reports the audio dropped and how long audio + takes to return to the clock. + +Boundaries: no player behavior changes here. Ceilings are measured, never +chosen. + +## Plan + +Planned 2026-11-09 (maintainer, /quest-plan: "make sure there are no tests or +benchmarks that cause us to optimize in the wrong direction"). ✅ m1, in this +epic, and before [the model](/quest/m1/audio-jitter-target/stretch.md), so +its implementer is never graded against the old rule. + +The gates the audit of `main` found pushing the wrong way (2026-11-09, +`10772522a5`): + +- `test/audio-quality/budgets.json` budgets the replay rows' `av_lag_ms` p50 + and p95 exactly, at -19 to -22 ms, and its note says "any change that + settles audio later to buy fewer underruns fails them". That forbids any + temporary lag and treats audio leading the clock as the resting point. +- `skip_aheads_per_min` and `discarded_ms_per_min`, on all 29 rows, infer a + skip from a step in the playhead's lag, so a catch-up reads as dropped + audio. Decided 2026-11-09 (maintainer, on + [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308#issuecomment-6730322286)): + ✅ budget the player's own trims (rejected: keep budgeting accelerates as + `discarded_ms`). +- The 24 browser and native rows cap underruns with no ceiling on lag. Native + settled 60 to 88 ms behind its clock without failing one. +- `test/audio-quality/clients/js/ptt-analyze.ts` fails on any underrun or + stall. [Push-to-talk latency](/quest/m1/push-to-talk-latency.md) owns that + change in + [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364); leave it there. +- The unit tests that pin the old rule belong to the model quest, which + rewrites them with the behavior. + +Neutral, and left alone: `just rs bench-playback` and +`js/watch/bench/ring.ts` measure CPU per callback, and the estimator's +conformance corpus measures the target. + +Recommendations: + +- Replay rows stay exact. Rewrite the note to say which direction is a + regression: a steady-state p50 lag that rises, or audio the player + trimmed. A p95 lag that rises through a stall, or an underrun count that + moves, is not one by itself. +- Add the congestion case twice: a shaper profile that queues until packets + drop, per the repository's impairment rule, and a replay trace where media + trickles in late after a stall, which the audit of #8360 asked for. +- New ceilings on the browser and native rows are provisional until + [the re-measure](/quest/m1/audio-jitter-target/calibrate.md), as the native + rows already are. +- The `step` rows' lag is budgeted as measured, with a sentence in the note: + the shared clock's reference only moves earlier, so a lasting path-delay + step leaves it stale. The follow-up is deferred until the model's numbers + exist. +- Update `test/audio-quality/README.md` for every metric added or dropped. + +Public API: none. Wire: none. + +## Required + +- [Native A/V lag](/quest/m1/audio-jitter-target/native-av-lag.md) - `moq play --metrics` reports the presentation clock, so native rows can grade `av_lag_ms` + +## Related + +- [Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md) - adds audio and video rows through a rate-limited link to the same harness diff --git a/quest/m1/audio-jitter-target/native-av-lag.md b/quest/m1/audio-jitter-target/native-av-lag.md index e7b0af6ea4d..0f583476ee9 100644 --- a/quest/m1/audio-jitter-target/native-av-lag.md +++ b/quest/m1/audio-jitter-target/native-av-lag.md @@ -1,56 +1,33 @@ -# [M] Native audio never settles behind its presentation clock +# [S] moq play reports how far audio trails its presentation clock ## Goal `moq play --metrics` reports how far the native audio playhead trails the presentation clock video plays from, so every native row in -`test/audio-quality` grades `av_lag_ms` (p50, p95) as the replay rows do. If -native's time stretch settles audio behind that clock, as the browser's did -before [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058), native -stops spending latency on it: at steady state its p50 lag is at or below zero -on every native row, and its ceilings guard that. - -Boundaries: the jitter estimator and its target stay as -`doc/concept/audio-jitter.md` specifies them. Underruns during congestion are -accepted, never bought down with latency. Native keeps compensating clock -drift between publisher and device, which the browser's ring cannot. +`test/audio-quality` can grade `av_lag_ms` as the replay rows do. Native's +playback decision does not change here. ## Plan -Decided 2026-11-08 (maintainer, on -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058)): never add -latency to chase zero underruns; audio and video keep separate clocks that -align at steady state once the delay is right. The browser's ring now decides -on its depth right after each arrival and never expands, because expanding on -the depth between arrivals settled audio a burst behind the shared clock -(`bursty-aac` lag p50 1 to 289 ms). Native's `Stretch::decide` -(`rs/moq-audio/src/stretch.rs`) still decides on `buffered` at each callback, -so it may do the same. - -1. Report the lag. The metrics writer (`rs/moq-cli/src/play/metrics.rs`) - already holds the presentation `moq_mux::sync::Consumer` and the - `Playhead`. Write each sample's `clock`, the presentation clock's media - time at the instant the playhead is read, with the field and meaning - `test/audio-quality/clients/js/src/schema.ts` gives the browser's, so - `analyze.ts` reduces native samples to `av_lag_ms` unchanged. Leave it out - while nothing is mapped, as other unreadable fields are. -2. Measure. Run the native rows locally against `main`'s decision, then read - the scheduled Nightlies. Lag p50 near or below zero means native already - aligns: budget `av_lag_ms` by the native rule and finish. -3. If p50 sits above zero on any row, decide on the depth right after each - arrival, less what the stretch removed since, as the browser's - `AudioRingBuffer` does, and expand only to follow a publisher clock - slower than the device's, not to fill the gap between arrivals. Hold the - listened drift case (media per render second about 0.998 against a 0.998 - publisher) and the shared stretch corpus, and update the spec's native - decision in the same PR. - -[Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) is -calibrating the native ceilings from seven Nightlies of today's decision. -Step 3 moves those ceilings, so a decision change re-measures them by the -same rule rather than stacking on the old calibration. - -## Related - -- [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - the native ceilings this lag joins, calibrated from the current decision -- [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - budgets `av_lag_ms` on the chromium rows the same way +Decided 2026-11-09 (maintainer, /quest-complete on +[moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361)): ✅ keep the +PR's metrics half and revert its playback changes (rejected: land its stretch +keyed to the lag at arrival). That stretch sped up into a thin buffer, taking +AAC `step` from 0 to 614 underruns a minute, and removed native's only +slow-down. It was built on the rule the +[epic](/quest/m1/audio-jitter-target/README.md) replaced the same day, so the +PR's recorded decisions "accept AAC step underruns" and "p50 within one pitch +period" go with it. + +What stays: `sync::Consumer::position` and the `clock` field. The metrics +writer (`rs/moq-cli/src/play/metrics.rs`) writes each sample's `clock`, the +presentation clock's media time at the instant the playhead is read, with the +field and meaning `test/audio-quality/clients/js/src/schema.ts` gives the +browser's, so `analyze.ts` reduces native samples to `av_lag_ms` unchanged. +Leave it out while nothing is mapped, as other unreadable fields are. + +Budgeting the lag belongs to +[the grading quest](/quest/m1/audio-jitter-target/grading.md), and native's +decision to [the stretch model](/quest/m1/audio-jitter-target/stretch.md). + +Public API: an additive `sync::Consumer::position`. Wire: none. diff --git a/quest/m1/audio-jitter-target/native-stretch.md b/quest/m1/audio-jitter-target/native-stretch.md deleted file mode 100644 index 4fdd9cff0e8..00000000000 --- a/quest/m1/audio-jitter-target/native-stretch.md +++ /dev/null @@ -1,163 +0,0 @@ -# [L] moq play converges inaudibly by time-stretching - -## Goal - -`rs/moq-audio`'s playback converges on its playout target by time-stretching -received audio, as WebRTC NetEq does, instead of running its ring dry and -padding with silence or trimming buffered audio. `moq play`'s native rows in -`test/audio-quality` play with no audible glitch once the target settles, -with underrun ceilings measured by `budgets.json`'s rule rather than fixed at -zero, as decided 2026-11-06 on -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058). The -stretch shares one algorithm and one set of test vectors with the browser's. - -Boundaries: no packet loss concealment, so a frame that never arrives is -still missing audio; only received audio is stretched. The estimator -(`rs/moq-audio/src/jitter.rs`) stays as `doc/concept/audio-jitter.md` -specifies it. - -## Plan - -Decided 2026-11-03 (maintainer, on -[moq-dev/moq#7679](https://github.com/moq-dev/moq/pull/7679)): copy NetEq in -m1. The implementation, merged 2026-11-04 in -[moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856), ports its -received-audio accelerate and preemptive expand into the native playback -callback. The estimator is unchanged. `doc/concept/audio-jitter.md` specifies -the shared floating-point algorithm; -`doc/concept/audio-jitter/stretch/corpus.json` supplies literal PCM vectors -for both languages. The browser half landed in -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058). - -The port retains WebRTC's BSD notice and patent grant, 4 kHz analysis, -50-sample correlation at lags 10 through 59, 14746/16384 normal and -8192/16384 fast admission thresholds, passive-speech VAD, 20 ms decision -band, 4x fast threshold, and 60 ms startup / 50 ms successful-stretch -cooldowns. -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) drains a declared -end's tail unstretched, as finishing does. Its browser half decides on the -ring's depth right after each arrival and never expands, since expanding on -the depth between arrivals settled audio a burst behind the shared clock -(decided 2026-11-08). Native playback still decides on `buffered`, so -whether it settles behind its clock the same way is worth measuring once its -rows report A/V lag. -Analysis samples NetEq's 48 kHz FIR on a virtual clock for other -device rates and uses floating-point peak refinement and fades. Those -arithmetic adaptations are documented, not claimed bit-identical to int16 -WebRTC. No threshold was tuned. - -Callback state is preallocated and uses the existing SPSC ring. Startup -silence and short/finished input bypass stretching. A genuine underrun -emits emergency silence until received PCM refills the target; it never -reanalyses padding or repeats history after input ends. Only the existing -capacity watermark and an incoming excess at the fast threshold trim. - -Remaining verification: - -- Native `test/audio-quality` rows must meet their measured underrun - ceilings once their targets settle. Keep a failed or void row explicit and - fix any implementation cause. -- Re-measure the provisional native budgets from seven scheduled nightly - artifacts that carry `moq play`'s own `trimmed` and `accelerated` - counters, using `budgets.json`'s rule: worst clean run times 1.25, plus - its documented native hiccup, rounded to two significant figures. A - developer run is not that calibration. Native rows then budget the - player's own `player_skips_per_min` and `trimmed_ms_per_min` in place of - the analyzer's `skip_aheads_per_min` and `discarded_ms_per_min`, which - count the accelerate too. - -Graded 2026-11-08: the scheduled Nightlies of 2026-11-05 to 11-08 on -`main` (42050683248, 42236107954, 42377867523, 42487375545) are the first -four with this implementation; 11-04's ran at 14:56 UTC, before it merged -at 17:31. Every night has all 12 native rows and none voided. Eleven rows -read zero underruns every night. Opus `step` reads one 22 to 44 ms gap on -three nights, at 29.6 s, when the shaper's one-way delay steps from 5 to -60 ms while the settled target is 60 ms: a late arrival no stretch can -cover without concealment, played clean once the target rises to 100 ms. -The scheduled Nightlies of 11-02 to 11-04, before this implementation, -read up to 23.9 underruns a minute on AAC `wide` and 8.6 on Opus `wide`. - -The stretch adds skip-aheads the harness did not see before: up to 3.3 a -minute and 232 ms discarded on Opus `bursty`, 2.2 and 181 ms on AAC -`near-zero` and `step` on 11-08. Each is the accelerate shedding a lowered -target or a startup overfill at about 8% speed for 1 to 4 s, with no single -sample's media advancing more than 33 ms past real time. The harness counts -any shed of more than 40 ms across nine samples as a skip, which its README -now says. Two rows failed their provisional ceilings that way: Opus -`bursty` on 11-07 (232 > 170 ms discarded, and a 380 ms startup target hold -> 350) and AAC `step` on 11-08 (157.5 > 110). The rest of the job's red is -outside this work: `raw-audio-bench` on 11-06, `qmux-bench` on 11-08, and -browser and replay rows. - -The nightly `rust-test` benchmark job ran `just rs bench-playback` and -passed on all four nights, alone on a GitHub `ubuntu-24.04-arm` runner. The -stretch costs 16 to 43% per callback, stable across nights; its worst cell, -64 sinks at 2048 frames, is 1.02 ms of a 42.7 ms period. -[moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308)'s body keeps -every graded native row's metrics. - -Decided 2026-11-09 (maintainer, on -[moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308#issuecomment-6730322286)): - -- ✅ `moq play --metrics` reports its own trimmed and accelerated audio, - and the calibration budgets trims alone (rejected: keep budgeting - accelerates as `discarded_ms`). An accelerate is inaudible, so it is not - the discarded audio these rows guard. #8308 adds the counters to both - players, beside the browser's skip count - [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) added. -- ✅ The nightly runner's `bench-playback` counts as the quiet-host run - (rejected: naming another host). That settles the timing check. - -Decided 2026-11-04 (maintainer, on -[moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856)): accept the -additive `playback::Sink::set_latency(Duration)` and the shared -floating-point NetEq adaptation, with its documented arithmetic differences -from bit-exact int16 WebRTC. The setter passes the moving target from -`moq play` to the callback and preserves it across device rebuilds; existing -fixed-latency callers need no change. Wire: none. Options shown: - -- ✅ The additive setter and the shared float adaptation -- An int16 port of WebRTC's DSP -- A different target handoff without the setter - -The native Opus `bursty` target p95 of 360 ms that one developer run of this -implementation reported, over the 350 ms provisional ceiling, is a startup -interval rather than the stretch. Its target jumped from the 120 ms prior to -360 ms 2 s in and held until the 20 s half-life brought it to 180 ms at -23 s; the run's other two attempts read 180 ms. The reported target is the -estimator's (`consumer.delay()` in `rs/moq-cli/src/play/media.rs`), timed -when the receive loop reads each frame, and this work changes neither. -Seven interleaved runs each of `main` (`635dd54f3c`) and this branch on one -loaded 24-core host (load 13 to 55, seed 7, 60 s) gave target p95s of -140, 180, 180, 180, 160, 240, and 160 ms on `main` and 180, 200, 200, 180, -180, 180, and 180 ms here, both with a median of 180 ms. The worst was on -`main`, a second late interval at 7 s. Every run of this branch had zero -underruns; `main` had up to 14.1 a minute. The 2026-10-29 to 11-03 scheduled -nightlies on `main` read 160 to 280 ms. So the ceiling stays as it is, -as `test/audio-quality/budgets.json`'s note budgets startup holds as measured, -until the seven-nightly calibration below. - -Listened 2026-11-05 (maintainer, /quest-unblock): the stretch is inaudible -in all four runs on `kixtop` at `main` `7e621623a8`, two minutes each of -`say` speech from `LICENSE-APACHE` over Opus 128 kb/s, the relay at -`test/audio-quality/relay.toml`, and the shaper's `bursty` profile, seed 7. -The player's metrics confirm the stretch was active: media per render second -was 1.0020 (fast, direct) and 1.0018 (fast, shaped) against a 1.002 -publisher, and 0.9981 and 0.9979 against 0.998; the shaped runs raised the -target delay from 120 to 260 ms. Each direct run stalled once at startup, -none in steady state. This settles the perceptual check; the calibration and -bench waits below remain. - -## Waiting - -- moq-dev/moq: seven scheduled Nightlies on a `main` that includes - [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308), whose - native rows report `player_skips` and `trimmed_ms`. Find them with `gh run list --workflow - nightly.yml --event schedule`, download each with `gh run download - -n audio-quality-run` within its 7 days, keep its native rows in the - calibration PR's body, and apply the calibration rule once seven exist. - Never dispatch the nightly by hand. - -## Related - -- [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - re-measures the browser rows with the stretch sharing this algorithm and its vectors diff --git a/quest/m1/audio-jitter-target/opus-stalled-share.md b/quest/m1/audio-jitter-target/opus-stalled-share.md index f51a17e7339..f1cc94fcdd3 100644 --- a/quest/m1/audio-jitter-target/opus-stalled-share.md +++ b/quest/m1/audio-jitter-target/opus-stalled-share.md @@ -1,11 +1,11 @@ -# [S] Chromium Opus rows stay under the stalled-share budget +# [S] Chromium Opus rows pass a measured stalled-share budget ## Goal -Nightly's Audio quality browser rows for Chromium Opus pass their -`stalled_share` budget (0.01) every night. Today a different row fails each -night at 0.014: `opus-48000-bursty` on 2026-11-08 and `opus-48000-wide` on -2026-11-07. +Nightly's Audio quality browser rows for Chromium Opus pass a `stalled_share` +budget measured with the browser's time stretch every night. Today a +different row fails each night at 0.014 against 0.01: `opus-48000-bursty` on +2026-11-08 and `opus-48000-wide` on 2026-11-07. ## Plan @@ -18,8 +18,17 @@ grading). [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058), which time-stretches the browser's audio ring, merged on 2026-11-09 and may already remove these stalls. Grade the scheduled nightlies after it (never dispatch -one). If the rows pass, delete this quest. Otherwise find what stalls Opus -that the stretch does not cover and fix it there; do not loosen the budget to -pass. +one). If the rows pass, delete this quest. + +Decided 2026-11-09 (maintainer, /quest-plan of the +[epic's model](/quest/m1/audio-jitter-target/README.md)): zero stalls is not +a goal, so a row over 0.01 is not by itself a defect to hunt. This replaces +"do not loosen the budget to pass". The 0.01 ceiling was measured before the +stretch, so if the rows still exceed it, re-measure `stalled_share` for the +Chromium Opus rows by the rule in `budgets.json`'s note from seven nights +after #8058. Fix a cause only when a stall is one the player should have +played through: audio that had arrived, or a refill longer than the model's +cushion. [Calibrate](/quest/m1/audio-jitter-target/calibrate.md) re-measures +every ceiling again once the model lands. Public API: none expected. Wire: none. diff --git a/quest/m1/audio-jitter-target/stretch-setting.md b/quest/m1/audio-jitter-target/stretch-setting.md new file mode 100644 index 00000000000..9610c8eafb5 --- /dev/null +++ b/quest/m1/audio-jitter-target/stretch-setting.md @@ -0,0 +1,61 @@ +# [M] A stretch setting bounds how far audio trails before it skips + +## Goal + +A player chooses how much lateness its audio forgives. `` and `moq play --stretch 2s` set how far live audio may trail +the shared clock before it skips ahead. Zero turns the time stretch off: the +player never alters received audio, late audio is skipped, an underrun is +silence, and drift between the publisher's clock and the device's is +corrected by a small skip or pad. The default is 2 s whether `delay` is +`auto` or fixed, so a fixed-delay live player also keeps late audio and +catches up. + +## Plan + +Planned 2026-11-09 (maintainer, /quest-plan). Asked how far audio may trail, +the maintainer answered that it is a product decision and not everybody will +want stretching. + +- Shape: ✅ one duration, where zero turns it off (rejected: an on/off switch + with a fixed bound; no new setting, leaving `auto` at 2 s and a fixed + delay at none; NetEq's four times the target as the bound). A voice app + can ask for 200 ms while a sports stream takes 2 s. +- Name: ✅ `stretch` (maintainer; rejected: `catchup`). +- At zero: ✅ no stretching at all (rejected: only falling behind is off, + with small corrections still stretched). A pitch-period splice is more + audible on music than on speech, so one value gives unaltered audio. +- Default: ✅ 2 s for `auto` and fixed delays alike (rejected: a fixed delay + keeping max delay equal to the delay). +- Milestone: ✅ m1 (rejected: m2). The attribute and flag are additive. + +The audio subscription's max delay is what makes late audio arrive at all: +the relay and the container consumer drop media older than it. Today it is 2 s +under `auto` (`AUTO_MAX_DELAY` in `js/watch/src/audio/latency.ts` and +`rs/moq-cli/src/play/args.rs`) and the delay itself when fixed, which leaves a +fixed-delay live player nothing to catch up on. The setting feeds that bound: +audio asks for `delay + stretch`. + +Recommendations: + +- Mirror the name in `js/watch` (`Sync`, the element attribute and property), + `moq play`, and whatever `moq-audio` type carries the bound, so a Rust + application gets it without the CLI. `rs/moq-ffi` already takes the decode + consumer's `delay` min and max and has no playback ring, so no binding + changes. +- The attribute needs a unit, and an invalid value throws, as `delay` and + `lookahead` do. +- Say how it composes with `lookahead`: a buffered player already keeps + everything inside its lookahead, so `stretch` bounds a live player's lag + and its zero still turns the stretch off in both modes. +- The zero path is the player before the stretch existed. Keep it tested, not + only reachable. +- Docs: `doc/lib/js/watch.md`, the element's attribute docs, and + `moq play --help`. + +Public API: an additive attribute, property, and flag. Wire: none; a +fixed-delay player's audio subscription asks for a longer max delay. + +## Required + +- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - the slow-down and catch-up this setting bounds and turns off diff --git a/quest/m1/audio-jitter-target/stretch.md b/quest/m1/audio-jitter-target/stretch.md new file mode 100644 index 00000000000..05f10a3c15f --- /dev/null +++ b/quest/m1/audio-jitter-target/stretch.md @@ -0,0 +1,141 @@ +# [XL] Audio slows down and speeds up around its target in every player + +## Goal + +`moq play`, a live ``, and a buffered one (`lookahead` above zero) +follow one decision rule, written once in `doc/concept/audio-jitter.md` and +held by one set of shared vectors: + +- In steady state audio plays on the shared clock at the playout delay. +- When the audio received ahead of the playhead runs low, playback slows down + and audio falls behind the clock. +- Once that buffer is back above the target, playback speeds up until audio is + on the clock again. +- After an underrun, audio resumes on a small cushion and slows down until + the buffer is back at the target. +- When the target moves, audio follows by stretching. It neither parks in + silence on a rise nor skips buffered audio on a fall. +- Audio skips ahead only once it trails by more than the subscription's max + delay still delivers. + +So a congestion stall costs a gap and a catch-up, not the audio that arrived +late. A buffered player follows the same rule and never skips inside its +lookahead, which fixes the permanent lag and dropped audio groups of +[moq-dev/moq#8031](https://github.com/moq-dev/moq/issues/8031). + +Boundaries: the estimator, the target formula, and video are unchanged. No +packet loss concealment. Zero underruns is not a goal. No new public setting: +the bound stays the max delay each player subscribes with today, until +[the `stretch` setting](/quest/m1/audio-jitter-target/stretch-setting.md). + +## Plan + +Planned 2026-11-09 (maintainer, /quest-plan). The +[epic](/quest/m1/audio-jitter-target/README.md) records the model and why. +Decisions for this quest: + +- Refill after an underrun: ✅ resume on a small cushion, about one stretch + window plus the render slack, then slow down to the target (rejected: + refill to the full target, as the browser does today; measure both and pick + in the PR). Refilling adds up to one target of silence and leaves audio + that much further behind. +- Speed: ✅ NetEq's thresholds and rates, untuned, in both directions, with + fast accelerate at four times the high limit (rejected: a few percent both + ways; a capped slow-down with NetEq's catch-up). A few percent recovers a + 2 s lag in over a minute and saves about 1 ms near empty. The browser's + fast accelerate at 20 ms of excess goes. +- Buffered mode: ✅ folded in here, and + [moq-dev/moq#8360](https://github.com/moq-dev/moq/pull/8360) is closed + (rejected: merge it as is and unify after; fix its four audit items first). + Its branch `quest/m1/audio-jitter-target/buffered-converge` at `7683e4e076` + anchors buffered audio to the shared clock and adds buffered replay rows; + both are worth taking. +- Order: ✅ after native backpressure (rejected: this first; one quest for + both). The level this rule reads is everything received ahead of the + playhead, decoded or still encoded, and the backlog while audio trails + sits encoded upstream in the browser. One mirrored rule needs both players + to split the buffer the same way. +- Clock step: ✅ deferred (rejected: plan it now; drop it). The shared + clock's reference only moves earlier, so after a lasting path-delay step + audio would settle behind a stale clock. Report what the `step` rows' lag + does in the PR body so the follow-up is decided with numbers. + +What the audit of `main` found (2026-11-09, `10772522a5`): + +- Native (`rs/moq-audio/src/stretch.rs`, `decide`) expands whenever the raw + depth is below the target, which is true between every arrival, so audio + settled 60 to 88 ms behind. NetEq's `DecisionLogic` compares a filtered + level against `low = max(3/4 target, target - 85 ms)` and + `high = max(target, low + 20 ms)`; the port kept neither the filter nor + the low limit. +- The browser (`js/watch/src/audio/ring-buffer.ts`, `stretch.ts`) never + expands, fast-accelerates at 20 ms of excess, stays silent after an + underrun until the whole target refills, parks on a target rise, and turns + the stretch off in buffered mode. +- [moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361) sped up into a + thin buffer and took AAC `step` from 0 to 614 underruns a minute. A + speed-up is admitted only while the buffer stays above the target by the + lag it removes. + +Recommendations: + +- NetEq assumes evenly paced packets. A batched publisher such as AAC in TS + delivers a whole PES at once, so the depth drains by a burst between + arrivals and its mean sits well under the target on a perfect path. + NetEq's limits on that mean would settle audio behind, as native does + today. Keep NetEq's limits, filter, and rates, and choose a level with the + publisher's cadence removed: the depth right after each arrival less what + the stretch changed since (the browser's `peak`), or the playhead's lag + behind the shared clock. Keep a check on the raw depth too, since a stall + delivers no arrival to decide on. The `bursty-aac` replay row holds this: + its steady-state lag should match the Opus rows'. +- A target rise leaves audio ahead of the clock and a fall leaves it behind, + so both become ordinary slow-down and speed-up. `AudioRingBuffer.park` and + the skip on a lowered target have no caller left. +- Rewrite the unit tests that pin the old rule rather than work around them: + `stretch.test.ts` ("never to slow down"), `ring-buffer.test.ts` ("re-stalls + and refills to the target", "never expands", "underruns every burst ... + rather than settling later", the 20 ms fast case), `replay.test.ts` ("never + behind it", "rather than settling behind the clock"), and + `stretch_tracks_both_clock_directions_and_target_changes` in + `rs/moq-audio/src/ring.rs`. The shared corpus + (`doc/concept/audio-jitter/stretch/corpus.json`) gains decision vectors for + the new inputs. +- Re-measure every replay row in this PR; the rule changes all of them. +- Docs in the same PR: one decision section in `doc/concept/audio-jitter.md` + replaces the native order and "Browser decision", its "Across renditions" + sentence on parking and skipping goes, and the `lookahead` docs in + `js/watch/src/element.ts`, `js/watch/src/sync.ts`, and + `doc/lib/js/watch.md` say a buffered player catches up by playing faster. + +Carried from the native port +([moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856)): the +maintainer listened on 2026-11-05 and found the stretch inaudible on speech, +and it costs 16 to 43% of a callback, 1.02 ms of a 42.7 ms period at worst. +Rerun `just rs bench-playback` and `bun watch/bench/ring.ts`, since the +decision now runs a filter. + +Public API: none expected. Wire: none. + +## Waiting + +- moq-dev/moq: [moq-dev/moq#8380](https://github.com/moq-dev/moq/pull/8380) + merges, planning native backpressure + (`/quest/m1/audio-jitter-target/native-backpressure.md`). Then list that + quest under Required and delete this item. It asks native to mirror the + browser's stalled refill to the full target; this quest then changes the + refill in both. + +## Required + +- [The audio harness grades the stretch model](/quest/m1/audio-jitter-target/grading.md) - the gates that pushed toward zero underruns and zero lag are gone before this is measured against them + +## Closes + +- [#8031](https://github.com/moq-dev/moq/issues/8031) - buffered playback loses lip sync and cuts audio out on a live stream + +## Related + +- [The `stretch` setting](/quest/m1/audio-jitter-target/stretch-setting.md) - makes this quest's skip bound configurable, and zero turns the stretch off +- [Native rendition delay](/quest/m1/audio-jitter-target/native-rendition-delay.md) - the PCM queue native backpressure replaces +- [Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md) - the audio and video congestion rows this behavior is asserted on diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 0e08084622d..fdc38bedc14 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -37,6 +37,13 @@ fails on the defaults by design? on. - (c) Run both rows, roughly doubling the nine-minute nightly step. +Decided 2026-11-09 (maintainer, /quest-plan of the +[audio jitter target](/quest/m1/audio-jitter-target/README.md)'s model): zero +underruns is not a goal of that epic either, so the run gates underruns and +stalls on a measured ceiling for good, not until the epic reaches zero. +[moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364)'s wording +"leaving zero to the epic" goes. + [moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) landed the analyzer's delay timeline: the page records the resolved delay over time, so each press's `jitter_buffer` ends at the first PCM that reaches the target the @@ -93,5 +100,5 @@ watch AudioContext's `latencyHint` (`"interactive"` in ## Related -- [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - `"auto"` underruns less by time-stretching the ring, so the run can hold the default delay; the stretch landed in [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058), and its nightly ceilings are re-measured there +- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - `"auto"` underruns less once the ring slows down near empty and resumes on a cushion, which moves this run's measured ceiling - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the `"auto"` target this latency feeds diff --git a/quest/m4/audio-bufferbloat-av.md b/quest/m4/audio-bufferbloat-av.md index e246bfc7635..3c1be80564f 100644 --- a/quest/m4/audio-bufferbloat-av.md +++ b/quest/m4/audio-bufferbloat-av.md @@ -24,7 +24,10 @@ Decided in planning (2026-10-20): audio. Video freezes, skips, and lag are reported, not gated: the point is that audio does not pay for them. Since 2026-11-05 the shared clock never follows audio, so congested video lags it by design, and steady-state lip - sync belongs to the audio ring's time stretch. + sync belongs to the audio ring's time stretch. Since 2026-11-09 audio may + trail the clock through the congestion and catch up after it + ([stretch model](/quest/m1/audio-jitter-target/stretch.md)), so these rows + gate audio dropped, not audio late. - Folded in: the native `pulse` rows become enforced. They report only because an intermittent two-second device stall voids about one AAC native row in seven (see `test/audio-quality/README.md`). Reproduce and fix that