From df4c521444c3d446a5e5008144c0029e3d23c658 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 11:27:44 -0800 Subject: [PATCH 1/5] quest: own the 11-13 Nightly failures nothing covered Co-Authored-By: Claude Opus 5.5 --- quest/m0/nightly/README.md | 12 ++++++ quest/m0/nightly/bundle-size.md | 29 ++++++++++++++ quest/m0/nightly/gate-cost-perf.md | 38 +++++++++++++++++++ quest/m0/nightly/release-builds.md | 30 +++++++++++++++ quest/m0/nightly/send-depth-gate.md | 31 +++++++++++++++ quest/m1/audio-jitter-target/calibrate.md | 11 +++++- .../congestion-catch-up.md | 34 +++++++++++++++++ 7 files changed, 184 insertions(+), 1 deletion(-) create mode 100644 quest/m0/nightly/bundle-size.md create mode 100644 quest/m0/nightly/gate-cost-perf.md create mode 100644 quest/m0/nightly/release-builds.md create mode 100644 quest/m0/nightly/send-depth-gate.md create mode 100644 quest/m1/audio-jitter-target/congestion-catch-up.md diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index ce2512dce37..49d77c9c431 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -47,3 +47,15 @@ more children for the causes nothing owned. Dropped: one-night bench moves near the paired threshold (re-check if one repeats), a one-off `remote error: 52` in the browser bench smoke, and failures main or [moq-dev/moq#8290](https://github.com/moq-dev/moq/pull/8290) already fixes. + +Decided 2026-11-13 (maintainer, /quest-plan), from the triage of run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289): four +more children, all in m0. Owned elsewhere: the fuzz build +([moq-dev/moq#8657](https://github.com/moq-dev/moq/pull/8657)), the Python +audit ([moq-dev/moq#8648](https://github.com/moq-dev/moq/pull/8648)), the size +report's output ([moq-dev/moq#8654](https://github.com/moq-dev/moq/pull/8654)), +the firefox resume ([resume sync](/quest/m0/interop-green/resume-sync.md)), +and the chromium `av_lag` rows +([render callback](/quest/m1/audio-jitter-target/render-callback.md) and +[congestion catch-up](/quest/m1/audio-jitter-target/congestion-catch-up.md)). + diff --git a/quest/m0/nightly/bundle-size.md b/quest/m0/nightly/bundle-size.md new file mode 100644 index 00000000000..74dabd4f831 --- /dev/null +++ b/quest/m0/nightly/bundle-size.md @@ -0,0 +1,29 @@ +# [M] The JS bundles shed what isn't worth its weight + +## Goal + +Every shipped JS bundle drops what does not earn its bytes, and the rest of +the growth since 2026-10-22 is justified in the PR as feature growth, so the +30-day size check passes once its window moves past the growth. + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13) and [moq-dev/moq#8654](https://github.com/moq-dev/moq/pull/8654): +every JS bundle grew 11 to 15% over about 96 js/net, qmux, and clock +commits, with no single culprit (`js/net` brotli 136,880 to 157,282 bytes). +The 30-day check (10% limit) compares against the 10-22 report from 11-21 and +fails every night until about 12-13 unless the bundles shrink. + +Decided 2026-11-13 (maintainer, /quest-plan): ✅ audit and shrink, then accept +the rest as feature growth (rejected: cut until every bundle is within 10%, +which may cost features; accept without an audit). Do not loosen the check. + +Known item: `@moq/clock`'s test-only `Manual` clock ships in every bundle +(about 2.3 KB) because the package index re-exports it. Attribute the rest +with `sh/size/bundles.ts`. + +## Related + +- [Audit cost](/quest/m3/audit-cost.md) - the recurring audit that also covers bundle size diff --git a/quest/m0/nightly/gate-cost-perf.md b/quest/m0/nightly/gate-cost-perf.md new file mode 100644 index 00000000000..be8046d50d7 --- /dev/null +++ b/quest/m0/nightly/gate-cost-perf.md @@ -0,0 +1,38 @@ +# [XS] Gate cost can count instructions on the merge-queue runners + +## Goal + +Nightly's `Gate cost` job (`just gh gate-cost`) counts user-space instructions +with perf on the `nixos` merge-queue runners and passes. + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13). Gate cost first ran on `nixos-moq-ci` after +[moq-dev/moq#8515](https://github.com/moq-dev/moq/pull/8515) and failed with +`perf cannot count user-space instructions`. The sysctl in the hint is not +the cause: the runners are systemd-nspawn containers, and nspawn's default +seccomp filter denies `perf_event_open` (the host's +`kernel.perf_event_paranoid` is already -1). +[moq-dev/moq#8655](https://github.com/moq-dev/moq/pull/8655) corrects the +hint; the fix is host config in kixel-ai/dotfiles +`nixos/modules/ci-runners.nix`. + +Decided 2026-11-13 (maintainer, /quest-plan): + +- Containers: ✅ only the `moq-queue` ones get + `--system-call-filter=perf_event_open` in `extraFlags` (rejected: all + runners, which would let PR-branch code open perf events). +- Host: ✅ set `kernel.perf_event_paranoid=2` in the same change (rejected: + keep -1, which lets those containers see CPU-wide and kernel events). + Profiling on `nixos` then needs sudo. + +Open the dotfiles PR, noting that open dotfiles PR #125 also edits that file. +Check with `perf stat -e instructions:u true` as the runner user inside a +queue container, then the next scheduled Nightly's Gate cost job; never +dispatch Nightly. + +## Waiting + +- Maintainer: merge the dotfiles PR and rebuild `nixos`, which restarts the CI runner containers; an agent does not deploy host config. Check with the `perf stat` command above. diff --git a/quest/m0/nightly/release-builds.md b/quest/m0/nightly/release-builds.md new file mode 100644 index 00000000000..522537d2893 --- /dev/null +++ b/quest/m0/nightly/release-builds.md @@ -0,0 +1,30 @@ +# [S] Nightly's release builds pass + +## Goal + +The release builds Nightly runs pass every night, so the jobs that need them, +such as `Shipping sizes`, run instead of being skipped. + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13): `Shipping sizes` was skipped 11-08 to 11-12 because release +builds it depends on failed, so six days of size growth surfaced at once. +Failures seen, each with its own cause: + +- 11-12, `moq-cli / release / Build (armv7-unknown-linux-gnueabihf)`: + `error[E0463]: can't find crate for core`, so the toolchain lacks that + target's standard library. +- 11-11, `Release FFI / Shared moq-ffi / Build moq-ffi (aarch64-unknown-linux-gnu)`: + apt failed with `Temporary failure resolving 'security.ubuntu.com'`, and + `Release C`/`Release Cpp` aarch64 then failed on a missing + `cpp-native-aarch64-unknown-linux-gnu` artifact. +- 11-11/11-12: the macOS `moq-cli` tarball, the Go FFI bindings and package, + the Kotlin FFI package, and the aarch64 `moq-gst` builds, not yet triaged. + +Fix each at its root. A network fetch at build time that can fail on a DNS +blip is a defect to remove, for example by taking the cross toolchain from +Nix, not to retry. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/send-depth-gate.md b/quest/m0/nightly/send-depth-gate.md new file mode 100644 index 00000000000..ff4936a422a --- /dev/null +++ b/quest/m0/nightly/send-depth-gate.md @@ -0,0 +1,31 @@ +# [S] The merge gate checks the session driver's Send depth + +## Goal + +A PR that pushes the session driver's `Send` proof past the compiler's default +recursion limit fails the merge gate, instead of landing and breaking every +Fuzz job in the next Nightly. The check is the one Nightly already runs +(`just rs send-depth` on the pinned nightly toolchain, and the Fuzz build's +`-D recursion_depth_exceeding_limit`). + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13): every Fuzz job failed with `overflow evaluating the requirement +driver::Driver: Send`. [moq-dev/moq#8545](https://github.com/moq-dev/moq/pull/8545) +deepened the proof past the limit, and +[moq-dev/moq#8657](https://github.com/moq-dev/moq/pull/8657) fixed it with 12 +levels to spare, so the next addition to session state can break it again. + +Decided 2026-11-13 (maintainer, /quest-plan): ✅ add it to the merge gate +(rejected: keep it Nightly-only, which lets the break land on `main`; drop +the quest). Measure what it adds to the gate and say so in the PR, per the +gate-cost rule. Reuse the existing recipe; pick the cheapest gate job that +already has the pinned nightly toolchain. + +Public API: none. Wire: none. + +## Waiting + +- moq-dev/moq: [moq-dev/moq#8657](https://github.com/moq-dev/moq/pull/8657) merges, so the gate starts from a passing proof; check with `gh pr view 8657 --json state` diff --git a/quest/m1/audio-jitter-target/calibrate.md b/quest/m1/audio-jitter-target/calibrate.md index 43a10e789c4..6d5cf6f8ca2 100644 --- a/quest/m1/audio-jitter-target/calibrate.md +++ b/quest/m1/audio-jitter-target/calibrate.md @@ -46,13 +46,22 @@ host noise for these nightlies to settle (rejected: re-measure on a quiet host first; investigate #8507's review fixes first). If the nightlies reproduce it, report it rather than budget it. +Decided 2026-11-13 (maintainer, /quest-plan): ✅ count only nights after +[the congestion fix](/quest/m1/audio-jitter-target/congestion-catch-up.md) +lands too, so the congestion rows budget the fixed player (rejected: +calibrate the other rows now and re-measure congestion later). + Native rows stay report-only; enforcing them belongs to [Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md). ## Waiting - moq-dev/moq: seven scheduled Nightly runs on `main` after the stretch model - merges. Find them with `gh run list --workflow nightly.yml --event + and the congestion fix merge. Find them with `gh run list --workflow nightly.yml --event schedule` and download each with `gh run download -n audio-quality-run` as it lands, since artifacts expire after 7 days. Never dispatch the nightly by hand. + +## Required + +- [Congestion catch-up](/quest/m1/audio-jitter-target/congestion-catch-up.md) - the congestion behavior the ceilings budget diff --git a/quest/m1/audio-jitter-target/congestion-catch-up.md b/quest/m1/audio-jitter-target/congestion-catch-up.md new file mode 100644 index 00000000000..2abeb7a5ba7 --- /dev/null +++ b/quest/m1/audio-jitter-target/congestion-catch-up.md @@ -0,0 +1,34 @@ +# [M] Audio recovers from a congestion stall quickly + +## Goal + +After a congestion stall, audio returns to the shared clock within a few +seconds instead of trailing by up to 1.4 s for 17 to 28 s, so the audio +quality harness's congestion rows no longer show a second of lip-sync error. +Congestion still costs a gap and a catch-up, as the +[epic's rule](/quest/m1/audio-jitter-target/README.md) says. + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13). Caused by [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507): +the browser ring no longer trims audio that trails the clock. It keeps up to +the `stretch` bound (2 s by default, `js/watch/src/audio/latency.ts` `bound`) +and accelerates at about 48 ms/s. Chromium `opus-48000-congestion` +`av_lag_ms` p50 rose from 35 to 73 ms locally and p95 is 1434 ms against a +32 ms ceiling; trims fell from about 1000 ms/min to 0. + +Decided 2026-11-13 (maintainer, /quest-plan): + +- ✅ a regression to fix (rejected: accept it and let calibrate budget the + congestion rows, which budgets the defect `calibrate.md` says to report). +- ✅ the implementer measures and picks (rejected: committing now to a lower + default bound alone, or a faster catch-up alone). Try a lower default bound, + past which audio skips onto the clock, and NetEq's fast accelerate for large + lag (`rs/moq-audio/src/stretch.rs` already ports both operations). Pick by + congestion `av_lag`, trims, and audible artifacts, and keep the browser and + native players on one rule from one algorithm. + +Put the before and after rows of `just test audio-quality` for chromium and +native in the PR. From 5fe94faa0b13f752c629b065d2c9f279df22c7c6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 12:11:51 -0800 Subject: [PATCH 2/5] quest: plan one lag-driven audio playout rule before the cut Move the audio jitter target epic to m2 as audio-playout, re-scoped to a rule driven by audio's lag behind the shared clock: stretch up to a 200 ms threshold at no more than 10% with a continuous time stretch, snap past it, measure lag where audio is heard, and let the target fall back soon after a one-off spike. Calibration moves to m3 behind the epic. Co-Authored-By: Claude Opus 5.5 --- quest/m0/nightly/README.md | 6 +- quest/m1/audio-jitter-target/README.md | 92 --------------- quest/m1/audio-jitter-target/calibrate.md | 67 ----------- quest/m1/audio-jitter-target/clock-step.md | 33 ------ .../congestion-catch-up.md | 34 ------ .../m1/audio-jitter-target/render-callback.md | 32 ----- quest/m2/audio-playout/README.md | 111 ++++++++++++++++++ quest/m2/audio-playout/clock-step.md | 42 +++++++ quest/m2/audio-playout/congestion-catch-up.md | 58 +++++++++ quest/m2/audio-playout/lag-rule.md | 69 +++++++++++ quest/m2/audio-playout/output-latency.md | 45 +++++++ quest/m2/audio-playout/target-fall.md | 36 ++++++ quest/m3/audio-calibrate.md | 62 ++++++++++ quest/m3/vendor-sync/webrtc.md | 4 +- quest/m4/audio-bufferbloat-av.md | 11 +- 15 files changed, 435 insertions(+), 267 deletions(-) delete mode 100644 quest/m1/audio-jitter-target/README.md delete mode 100644 quest/m1/audio-jitter-target/calibrate.md delete mode 100644 quest/m1/audio-jitter-target/clock-step.md delete mode 100644 quest/m1/audio-jitter-target/congestion-catch-up.md delete mode 100644 quest/m1/audio-jitter-target/render-callback.md create mode 100644 quest/m2/audio-playout/README.md create mode 100644 quest/m2/audio-playout/clock-step.md create mode 100644 quest/m2/audio-playout/congestion-catch-up.md create mode 100644 quest/m2/audio-playout/lag-rule.md create mode 100644 quest/m2/audio-playout/output-latency.md create mode 100644 quest/m2/audio-playout/target-fall.md create mode 100644 quest/m3/audio-calibrate.md diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index 49d77c9c431..42a03707b87 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -24,7 +24,7 @@ This README's own work, after the children: - Confirm a scheduled run on `main` passes; never dispatch Nightly. Failures already owned elsewhere stay there: the audio near-zero p95 row belongs to - [audio jitter target](/quest/m1/audio-jitter-target/README.md), the + [audio playout](/quest/m2/audio-playout/README.md), the macOS `receive_credit` timeout has no owner since its quest was deleted in [moq-dev/moq#6732](https://github.com/moq-dev/moq/pull/6732), so plan one under [test flakes](/quest/m0/test-flakes-2/README.md) if it recurs, and the paired bench @@ -56,6 +56,6 @@ audit ([moq-dev/moq#8648](https://github.com/moq-dev/moq/pull/8648)), the size report's output ([moq-dev/moq#8654](https://github.com/moq-dev/moq/pull/8654)), the firefox resume ([resume sync](/quest/m0/interop-green/resume-sync.md)), and the chromium `av_lag` rows -([render callback](/quest/m1/audio-jitter-target/render-callback.md) and -[congestion catch-up](/quest/m1/audio-jitter-target/congestion-catch-up.md)). +([output latency](/quest/m2/audio-playout/output-latency.md) and +[congestion catch-up](/quest/m2/audio-playout/congestion-catch-up.md)). diff --git a/quest/m1/audio-jitter-target/README.md b/quest/m1/audio-jitter-target/README.md deleted file mode 100644 index 0d80fdee610..00000000000 --- a/quest/m1/audio-jitter-target/README.md +++ /dev/null @@ -1,92 +0,0 @@ -# Audio jitter target - -## Goal - -The audio playout target is a measured estimate of arrival timing, ported from -a known-good implementation, in the browser and natively alike. The round-trip -formula is gone. `max(20ms, 1.25 x minRtt)` sized the buffer for a single -retransmit, which describes how long the network takes to recover a loss and -not how unevenly a publisher emits frames, so a sender flushing 100 ms of media -at once got a target far too shallow to play through the next flush. One -written algorithm, cited by both implementations, produces the same target from -the same arrival trace. - -Every player follows one rule around that target, with NetEq's time stretch. -In steady state audio plays on the shared clock at the playout delay, which is -the expected resting point for audio and video alike. When the audio received -ahead of the playhead runs low, playback slows down and audio falls behind the -clock; once the buffer refills past the target, playback speeds up until audio -is back on it. Congestion then costs a gap and a catch-up instead of the audio -that arrived late. - -Boundaries: zero underruns is not a goal, and neither the target nor a fixed -delay is raised to reach it. No packet loss concealment. Video stays on the -shared clock and shows a late frame when it arrives, so it recovers on its -own. Video measures no network term: the shared delay is the largest -requirement among the subscribed renditions, so video waits for audio's. - -## Plan - -Both implementations are on `main` (#4162): `js/hang/src/container/jitter.ts` -and `rs/moq-audio`'s `jitter.rs` follow `doc/concept/audio-jitter.md` and pass -its conformance corpus, `js/watch` `Sync` holds the deepest track's target in -`"auto"`, and `moq play --delay` defaults to `auto`. Both are the default for -every viewer, so `"auto"` shipped ahead of its browser proof; the maintainer -accepted that, and the re-measured -replay budgets, on 2026-10-04. Decided 2026-10-13: retain the remaining proof in m1 because the A/V -clock and worker release contracts depend on it. Native quality expansion -remains separately scoped in m2. The 2026-11-03 proof run found the 0.95 -quantile leaves about six re-stalls a minute on real paths by design; decided -2026-11-03 (maintainer), time stretch closes that rather than a deeper -target, in both players from one algorithm and its vectors: -[moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856) natively and -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) in the browser's ring. - -Decided 2026-11-09 (maintainer, /quest-plan): the rule in the Goal. It -replaces the 2026-11-08 decision on -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) that the browser -only plays faster and audio never settles later. Video gets the effect for -free, since a late frame shows at once and the picture is live again; audio -cannot, so without a slow-down and a catch-up a stall drops whatever audio -arrives late. What carries over from 2026-11-08: latency is never added to -reach zero underruns, and some underruns during congestion are expected. - -- Players: ✅ `moq play`, a live ``, and a buffered one, on one rule - (rejected: live only, with buffered mode keeping its own). -- Target: ✅ unchanged. The shared delay is - `max over subscribed renditions of (delay + target)`, where audio's target - is `max(measured, jitter) + frame` and video's is its catalog span - (rejected: a network term on every rendition; the plain sum - `delay + jitter + frame + measured`, which buffers a batched publisher's - batch twice; a catalog-only target). -- Video: ✅ on the same clock as audio, and audio alone speeds up and slows - down with the network (rejected: video at its catalog target, with audio - trailing it by the measured lateness). -- Target moves: ✅ the estimator stays as it is, and audio glides to each new - target by stretching (rejected: a slew limit on the measured term, which - the spec already declined as a second knob; deleting the estimator and - letting the lag be the only network term, which underruns a 20 ms Opus - target until the lag builds). -- Bound: ✅ configurable: the `stretch` setting (``, - `moq play --stretch`), 2 s by default, where zero turns the stretch off. -- Tests: no gate may reward moving away from the rule, so the audio harness - grades steady-state lag, the player's own trims, and recovery from a stall - before [the model](https://github.com/moq-dev/moq/pull/8507) lands. -- Milestone: ✅ every new child in m1, here (rejected: the model and grading - in m0; the setting in m2). - -Decided 2026-11-04 (maintainer), for push-to-talk's silence suppression -([moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942)): a silence the publisher declares -with an end marker is not idle, so the estimator folds no idle intervals for -it and keeps its estimate, where an undeclared gap still relaxes toward the -prior. The document's `silence` corpus case holds both implementations to it. - -What the line still owes: its recorded trace replayed through -the native decode path too, asserting the same target series the browser's -`playout.test.ts` does. The audio quality harness's `pulse` rows already grade -native playback against its budgets. - -## Closes - -- [#2812](https://github.com/moq-dev/moq/issues/2812) - the iOS stutter report the same estimator fixes - diff --git a/quest/m1/audio-jitter-target/calibrate.md b/quest/m1/audio-jitter-target/calibrate.md deleted file mode 100644 index 6d5cf6f8ca2..00000000000 --- a/quest/m1/audio-jitter-target/calibrate.md +++ /dev/null @@ -1,67 +0,0 @@ -# [S] Re-measure every audio ceiling after the stretch model lands - -## Goal - -`test/audio-quality/budgets.json` carries every browser and native row's -ceilings re-measured, by the rule in its note, from seven scheduled Nightly -runs on a `main` that includes -[the stretch model](https://github.com/moq-dev/moq/pull/8507). The rows budget -steady-state lag, the player's own trims and skips, and underruns as -measured, and the note says the model is in them. - -## Plan - -Planned 2026-11-09 (maintainer, /quest-plan). This replaces two waits that -were calibrating a player the model changes: the chromium re-measure after -[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) and the native -one after [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308). Nights -before the model merges do not count. - -Carried decisions: - -- 2026-11-07 (maintainer, on #8058): ✅ land the player change, then measure - (rejected: hold it until Nightly ceilings are in). The goal is judged by - the rows alone, with no listening run. -- 2026-11-09 (maintainer, on - [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308#issuecomment-6730322286)): - ✅ budget `trimmed_ms_per_min` and `player_skips_per_min` (rejected: - budgeting accelerates as discarded audio), and ✅ the nightly runner's - playback benchmarks count as the quiet-host run (rejected: naming another - host). - -The rule: the worst of seven clean nights times 1.25, plus the documented -runner hiccup, rounded to two significant figures. A developer run is not -that calibration. Keep each night's rows in the PR body. - -What to report rather than budget: a steady-state `av_lag_ms` p50 more than a -frame above zero on any row but `step`, or a nonzero `player_skips_per_min` -on a row whose stalls fit inside the `stretch` bound. Either means the model -is not doing its job there. - -Watch native-opus near-zero: at [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507)'s -head a developer host at load average about 43 filed 64 to 77 tap underruns -a minute against its ceiling of 42, where two earlier runs filed 0. Decided -2026-11-11 (maintainer, /quest-merge pick of #8507): ✅ merge and treat it as -host noise for these nightlies to settle (rejected: re-measure on a quiet -host first; investigate #8507's review fixes first). If the nightlies -reproduce it, report it rather than budget it. - -Decided 2026-11-13 (maintainer, /quest-plan): ✅ count only nights after -[the congestion fix](/quest/m1/audio-jitter-target/congestion-catch-up.md) -lands too, so the congestion rows budget the fixed player (rejected: -calibrate the other rows now and re-measure congestion later). - -Native rows stay report-only; enforcing them belongs to -[Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md). - -## Waiting - -- moq-dev/moq: seven scheduled Nightly runs on `main` after the stretch model - and the congestion fix merge. Find them with `gh run list --workflow nightly.yml --event - schedule` and download each with `gh run download -n - audio-quality-run` as it lands, since artifacts expire after 7 days. Never - dispatch the nightly by hand. - -## Required - -- [Congestion catch-up](/quest/m1/audio-jitter-target/congestion-catch-up.md) - the congestion behavior the ceilings budget diff --git a/quest/m1/audio-jitter-target/clock-step.md b/quest/m1/audio-jitter-target/clock-step.md deleted file mode 100644 index 2668f1760b8..00000000000 --- a/quest/m1/audio-jitter-target/clock-step.md +++ /dev/null @@ -1,33 +0,0 @@ -# [M] The shared clock follows a lasting path-delay step - -## Goal - -After a lasting increase in path delay, audio and video settle on the shared -clock at the playout delay again, in the browser and natively, instead of -audio resting behind a clock whose reference never moves later. The `step` -rows of the audio-quality harness show a steady-state `av_lag_ms` p50 within -a frame of zero. - -## Plan - -Decided 2026-11-11 (maintainer, /quest-merge pick of -[moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507)): keep. - -The stretch model deferred this so it would be decided with numbers. At -#8507's head the live `step` rows rested 35.2 and 41.8 ms behind in the -browser (AAC, Opus) and 5.7 and 17.3 ms natively, p95 37.5 to 65.1 ms, with -native AAC at 29.3 underruns a minute (614 under -[moq-dev/moq#8361](https://github.com/moq-dev/moq/pull/8361)). - -Recommendations: - -- Let the reference move later only on sustained evidence, such as the - minimum offset over a window well past the target, and treat the move like - a target rise: audio glides to it by stretching, video holds a frame. -- Keep one rule in `js/watch` `Sync` and `moq_mux::sync`, with a shared test - vector, as the stretch model's decisions corpus does. -- Do not add latency to reach zero underruns on the step. - -## Required - -- [Calibrate](/quest/m1/audio-jitter-target/calibrate.md) - the step rows' ceilings to measure against diff --git a/quest/m1/audio-jitter-target/congestion-catch-up.md b/quest/m1/audio-jitter-target/congestion-catch-up.md deleted file mode 100644 index 2abeb7a5ba7..00000000000 --- a/quest/m1/audio-jitter-target/congestion-catch-up.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] Audio recovers from a congestion stall quickly - -## Goal - -After a congestion stall, audio returns to the shared clock within a few -seconds instead of trailing by up to 1.4 s for 17 to 28 s, so the audio -quality harness's congestion rows no longer show a second of lip-sync error. -Congestion still costs a gap and a catch-up, as the -[epic's rule](/quest/m1/audio-jitter-target/README.md) says. - -## Plan - -Found by the triage of Nightly run -[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) -(2026-11-13). Caused by [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507): -the browser ring no longer trims audio that trails the clock. It keeps up to -the `stretch` bound (2 s by default, `js/watch/src/audio/latency.ts` `bound`) -and accelerates at about 48 ms/s. Chromium `opus-48000-congestion` -`av_lag_ms` p50 rose from 35 to 73 ms locally and p95 is 1434 ms against a -32 ms ceiling; trims fell from about 1000 ms/min to 0. - -Decided 2026-11-13 (maintainer, /quest-plan): - -- ✅ a regression to fix (rejected: accept it and let calibrate budget the - congestion rows, which budgets the defect `calibrate.md` says to report). -- ✅ the implementer measures and picks (rejected: committing now to a lower - default bound alone, or a faster catch-up alone). Try a lower default bound, - past which audio skips onto the clock, and NetEq's fast accelerate for large - lag (`rs/moq-audio/src/stretch.rs` already ports both operations). Pick by - congestion `av_lag`, trims, and audible artifacts, and keep the browser and - native players on one rule from one algorithm. - -Put the before and after rows of `just test audio-quality` for chromium and -native in the PR. diff --git a/quest/m1/audio-jitter-target/render-callback.md b/quest/m1/audio-jitter-target/render-callback.md deleted file mode 100644 index e283a045a8e..00000000000 --- a/quest/m1/audio-jitter-target/render-callback.md +++ /dev/null @@ -1,32 +0,0 @@ -# [M] The browser's target counts its render callback - -## Goal - -The browser's audio target includes the audio its render callback pulls at -once, and its clock is paired on the wall clock, so the browser rests on the -shared clock like native does, without the underruns that pairing alone -caused. - -## Plan - -Decided 2026-11-11 (maintainer, /quest-merge pick of -[moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507)): keep. - -Headless Chromium renders 2048-frame callbacks (`baseLatency` 42.7 ms), and -the clock is paired with `currentTime` just after each callback, so the -browser rests 28 to 35 ms behind the clock where native rests within a few -ms. A max-filtered wall-clock pairing put audio on the clock (p50 -1.3 to -4.5 ms) but underran near-zero Opus 45 times a minute, since the target did -not count the callback. - -Recommendations, in order and in one PR: - -1. Add the render quantum (`baseLatency`, or the measured callback size) to - the browser ring's target. -2. Then pair the clock on the wall clock with the max filter. - -Measure the browser live rows before and after; native is unchanged. - -## Required - -- [Calibrate](/quest/m1/audio-jitter-target/calibrate.md) - the browser ceilings this is measured against diff --git a/quest/m2/audio-playout/README.md b/quest/m2/audio-playout/README.md new file mode 100644 index 00000000000..461999a07a1 --- /dev/null +++ b/quest/m2/audio-playout/README.md @@ -0,0 +1,111 @@ +# Audio playout + +## Goal + +Every player (`moq play`, a live ``, and a buffered one) plays +audio on the shared clock by one rule driven by audio's lag behind that +clock, measured where the audio is heard. In steady state audio and video +are in sync, and neither waits on the other or on the other's settings. A +discrepancy up to the `stretch` threshold, 200 ms by default, is corrected by +time stretch at no more than 10%, so audio is back on the clock within about +2 s; a larger one snaps the way video does, skipping late audio or padding +silence when audio is ahead. + +The playout target is a measured estimate of arrival timing, from one written +algorithm both players implement, and it falls back soon after a one-off +spike instead of holding audio and video seconds behind for 10 to 20 s. + +Boundaries: zero underruns is not a goal, and no latency is added to reach +it. No packet loss concealment. Video stays on the shared clock and shows a +late frame when it arrives. Audio and video may desync during congestion, not +in steady state. + +## Plan + +The estimator is on `main` (#4162): `js/hang/src/container/jitter.ts` and +`rs/moq-audio`'s `jitter.rs` follow `doc/concept/audio-jitter.md` and pass its +conformance corpus, `js/watch` `Sync` holds the deepest track's target in +`"auto"`, and `moq play --delay` defaults to `auto`. Time stretch followed in +[moq-dev/moq#7856](https://github.com/moq-dev/moq/pull/7856), +[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058), and the +2026-11-09 rule [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507) +and [moq-dev/moq#8532](https://github.com/moq-dev/moq/pull/8532) put in every +player. None of it is on `release` yet, so the cut publishes whatever this +epic leaves. + +Decided 2026-11-13 (maintainer, /quest-plan, from an audit of `main`'s +audio): the rule in the Goal replaces the 2026-11-09 one. Measured on `main` +by two simulations of the real ring and stretch code and by the Nightlies of +11-12 and 11-13, that rule: + +- Rests off the clock. NetEq's `DecisionLogic` is a buffer-level controller, + and here the clock decides where audio belongs. With the newest frame J + late, `level = target + frame - J`, so audio rests up to J behind: browser + `wide` 32 to 41 ms, AAC `bursty` 37 to 47 ms, natively 70 to 80 ms after + video's arrivals move the reference earlier. +- Catches up at the talker's pitch, 4 to 20%, and not at all on music or + noise that fails correlation, which held a 1.4 s offset in simulation. +- Lets audio trail 2 s: congestion rows trail video by 1.1 to 1.4 s at p95 and + take 17 to 28 s to return, against 2 to 3.3 s before #8507. +- Makes native video wait on audio's `--stretch` for a missing group. +- Ignores output latency in both players, and natively floors the shared + clock at 50 ms instead. + +Decisions, with the prompts as asked: + +- Goal: ✅ one lag-driven rule that snaps past a small threshold, no audio + setting makes video wait, lag measured where audio is heard, and the target + falls back faster after a one-off spike (rejected: only shrinking the bound + and speeding the catch-up; the same without the target fall). +- Threshold: ✅ 200 ms both ways, the rate capped at 10% and proportional to + the lag below that (rejected: 200 ms behind and 100 ms ahead; 100 ms both + ways; a fraction of the target). Every move of the clock, a target rise or + fall or a reference move, is lag like any other, so a large fall snaps as + video skips. +- Stretch: ✅ a continuous WSOLA time stretch for any content (rejected: + NetEq's pitch-period splices driven by lag; resampling, which shifts pitch). +- Gaps: ✅ audio slows down when the audio received ahead of the playhead is + about to run out, never trailing past the threshold (rejected: lag only, + an underrun being silence). +- Target fall: ✅ the implementer measures and picks (rejected: a shorter + half-life now; one observation per outage now). +- Setting: ✅ `stretch` stays a duration and is the threshold, zero turning + the stretch off; it sizes nothing else (rejected: an on/off switch; + removing it). +- Buffered: ✅ a buffered player snaps past the larger of its lookahead and + the threshold (rejected: the threshold alone). +- Milestone: ✅ this epic in m2 and its calibration in m3 (rejected: the whole + epic with calibration in m2; only the lag rule in m2). + +Overturned: from 2026-11-09, the 2 s bound, audio gliding to each new target +by stretching, NetEq's thresholds and rates untuned, and the estimator +staying as it is; from 2026-11-11, the bound alone deciding a lag skip. Kept: +lip sync in steady state with desync allowed under congestion, and a shared +clock that never follows audio (2026-11-05); latency never added to chase +zero underruns (2026-11-08); the target formula and the playout as the +largest `delay + target` among subscribed renditions; one algorithm with +shared vectors for both players. + +Order: [congestion catch-up](/quest/m2/audio-playout/congestion-catch-up.md) +gives the stall its snap on today's splices, the +[lag rule](/quest/m2/audio-playout/lag-rule.md) replaces the controller +under it, and the [clock step](/quest/m2/audio-playout/clock-step.md) builds +on the lag rule. [Target fall](/quest/m2/audio-playout/target-fall.md) and +[output latency](/quest/m2/audio-playout/output-latency.md) are independent. +The [calibration](/quest/m3/audio-calibrate.md) re-measures every ceiling +after this epic. + +Decided 2026-11-04 (maintainer), for push-to-talk's silence suppression +([moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942)): a silence the +publisher declares with an end marker is not idle, so the estimator folds no +idle intervals for it and keeps its estimate, where an undeclared gap still +relaxes toward the prior. The document's `silence` corpus case holds both +implementations to it. + +What the line still owes: its recorded trace replayed through the native +decode path too, asserting the same target series the browser's +`playout.test.ts` does. + +## Closes + +- [#2812](https://github.com/moq-dev/moq/issues/2812) - the iOS stutter report the same estimator fixes diff --git a/quest/m2/audio-playout/clock-step.md b/quest/m2/audio-playout/clock-step.md new file mode 100644 index 00000000000..00ab8aaceb2 --- /dev/null +++ b/quest/m2/audio-playout/clock-step.md @@ -0,0 +1,42 @@ +# [M] The shared clock follows a lasting path-delay step + +## Goal + +After a lasting increase in path delay, audio and video settle on the shared +clock at the playout delay again, in the browser and natively, instead of +playing against a reference that never moves later. The `step` rows of the +audio-quality harness rest within a frame of the clock after the step, with +no more underruns than before it. + +## Plan + +Decided 2026-11-11 (maintainer, /quest-merge pick of +[moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507)): keep. +Re-scoped 2026-11-13 (maintainer, /quest-plan) to the +[audio playout](/quest/m2/audio-playout/README.md) rule, under which a move +of the clock is lag like any other. + +The reference is the earliest lateness any track has shown, and arrivals only +move it earlier (`js/watch/src/sync.ts`, `rs/moq-mux/src/sync.rs`), while the +estimator measures each frame against the fastest of the last 2 s. After a +lasting step the estimate forgets the step within its window, but the clock +keeps the old reference, so every frame is late for it. On the 2026-11-09 +rule audio rested behind: at #8507's head the `step` rows rested 35.2 and +41.8 ms behind in the browser (AAC, Opus) and 5.7 and 17.3 ms natively; a +simulated +200 ms step left native audio 150 to 160 ms behind. Under the +[lag rule](/quest/m2/audio-playout/lag-rule.md) audio follows the clock +instead, so the same stale reference shows as underruns. + +Recommendations: + +- Let the reference move later only on sustained evidence, such as the + least lateness over a window well past the target staying past it. The + move is lag like any other: audio slows down within the threshold or pads + past it, and video holds a frame. +- Keep one rule in `js/watch` `Sync` and `moq_mux::sync`, with a shared test + vector, as the estimator's corpus does. +- Do not add latency to reach zero underruns on the step. + +## Required + +- [Lag rule](/quest/m2/audio-playout/lag-rule.md) - the controller that treats a reference move as lag diff --git a/quest/m2/audio-playout/congestion-catch-up.md b/quest/m2/audio-playout/congestion-catch-up.md new file mode 100644 index 00000000000..ed6a80dd554 --- /dev/null +++ b/quest/m2/audio-playout/congestion-catch-up.md @@ -0,0 +1,58 @@ +# [M] Audio recovers from a congestion stall quickly + +## Goal + +After a congestion stall, audio is back on the shared clock within about 2 s +of arrivals resuming, in the browser and natively, instead of trailing by up +to 1.4 s for 17 to 28 s. Audio that trails the clock by more than the +`stretch` threshold, now 200 ms by default, skips onto the clock, and the +`stretch` setting sizes nothing but that threshold. + +## Plan + +Found by the triage of Nightly run +[43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) +(2026-11-13). Caused by [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507), +which stopped trimming audio that trails the clock, and +[moq-dev/moq#8532](https://github.com/moq-dev/moq/pull/8532), whose `stretch` +bound defaults to 2 s. Chromium `opus-48000-congestion` `av_lag_ms` p95 is +1434 ms against a 32 ms ceiling; trims fell from about 1000 ms/min to 0. Both +players catch up at about 82 ms of lag a second, and each later fall of the +target adds lag back. + +Decided 2026-11-13 (maintainer, /quest-plan): ✅ a regression to fix +(rejected: accept it and let calibrate budget the congestion rows). Then the +same day, in the [audio playout](/quest/m2/audio-playout/README.md) plan: +✅ a 200 ms default threshold past which audio skips onto the clock +(rejected: the implementer measuring a lower bound against NetEq's fast +accelerate). This is the first step of that rule, on today's splices; the +[lag rule](/quest/m2/audio-playout/lag-rule.md) replaces the controller +under it. + +Scope: + +- `stretch` defaults to 200 ms (`js/watch/src/sync.ts` `STRETCH`, `moq play + --stretch`); zero still turns the stretch off. +- A lag past the threshold skips onto the clock, not back to the bound, in + `js/watch/src/audio/ring-buffer.ts` and `rs/moq-audio/src/ring.rs`. +- The setting sizes nothing else. The audio subscription keeps the delay + plus the estimator's 2 s ceiling, so late frames are still observed. + Natively, the decode cap, the hole policy (`fill_max`), and video's wait on + a missing group (`args.rs` `video_max_delay`) stop following it, so video + waits its own delay. That also fixes `--delay auto --stretch 0`, where any + hole over 50 ms drops the sink and its queued audio (`play/media.rs` + `cap`). +- The audio-quality harness reports, per row, how long after a disturbance + audio is back within a frame of where it rested before it. Today's + `recover_ms` measures against the window's p50 and is not tied to the + disturbance. +- `doc/concept/audio-jitter.md`'s observation point and skip rules say the + same. + +Put the before and after rows of `just test audio-quality` for chromium and +native in the PR. + +## Related + +- [Resume sync](/quest/m0/interop-green/resume-sync.md) - snaps resumed audio to the clock, the same rule for a return from absence +- [Python to C++ audio gap](/quest/m0/test-flakes-2/interop-python-cpp-audio-gap.md) - also changes how long a missing group is waited on diff --git a/quest/m2/audio-playout/lag-rule.md b/quest/m2/audio-playout/lag-rule.md new file mode 100644 index 00000000000..18946c89f9a --- /dev/null +++ b/quest/m2/audio-playout/lag-rule.md @@ -0,0 +1,69 @@ +# [XL] Audio follows the clock by its lag alone + +## Goal + +Both players decide how to play audio from its lag behind the shared clock +and stretch any content, speech, music, or noise, with a continuous time +stretch at a rate proportional to that lag, capped at 10%. Within the +`stretch` threshold audio speeds up or slows down onto the clock; past it, it +snaps; when the audio received ahead of the playhead is about to run out, it +slows down, never trailing past the threshold. In steady state audio rests +within a frame of the clock on every audio-quality row, including `wide` and +`bursty`. + +## Plan + +Decided 2026-11-13 (maintainer, /quest-plan); the decisions and their prompts +are in the [epic](/quest/m2/audio-playout/README.md). + +Why the controller goes: `decide` in `js/watch/src/audio/stretch.ts` and +`rs/moq-audio/src/stretch.rs` is NetEq's `DecisionLogic`. It accelerates only +while the filtered level reaches the target and slows down below three +quarters of it. In NetEq the buffer level is the latency; here the clock is, +and the two agree only when the newest frame arrives on time. Its splices +remove one pitch period per 50 ms cooldown, so the rate follows the talker, +and music or noise that fails admission never moves. + +Settled: + +- Input: the lag, with the received depth (decoded or still encoded) used + only to bridge a gap. A target rise or fall and a reference move are lag + like any other, so the `jumped` accumulation and its one-window rule go, + and `jumped` leaves `Sink::due` and `bus::Positioned`. +- Rate: proportional to the lag and capped at 10%. The implementer picks the + gain and reports the time from 200 ms back to within a frame. +- Snap: past the threshold, or for a buffered player past the larger of its + lookahead and the threshold, skip onto the clock when behind and pad + silence when ahead. A join snaps. +- Bridge: slow down while the received depth is below a cushion sized from + the stretch's window and the device read, and the lag is under the + threshold. The depth, not NetEq's level at arrival, so a batched publisher + between batches does not trigger it while its next batch is on time. +- Delete NetEq's decision, level filter, cooldown, fast mode, correlation + admission, pitch-period splices, and their corpus, and the WebRTC license + files if nothing else ported from WebRTC remains. + +Recommendations: + +- Chromium's `media/filters/audio_renderer_algorithm.cc` (BSD-3) is a WSOLA + for arbitrary rates and any content. Implement from a description, or port + it and carry its license, as the NetEq port did. +- One decision corpus and one stretch corpus both languages consume, as + today; the spec's "Received-audio time stretch" section is rewritten. +- Fix what the rewrite touches: the worklet drops its clock on every latency + message (`render-worklet.ts`), so each target change decides unclocked for + a few quanta; the browser ring starts a frame ahead of the clock (its floor + is the delay); native's `pad` and `trim` sink options are dead. +- Measure `just test audio-quality` before and after in both players, and + listen to speech and music at the 10% cap. + +Public API: `Sink::due` and `bus::Positioned` lose `jumped`; both are +`main`-only, so nothing published migrates. Wire: none. + +## Required + +- [Congestion catch-up](/quest/m2/audio-playout/congestion-catch-up.md) - the 200 ms threshold and the snap this controller keeps + +## Related + +- [WebRTC vendor sync](/quest/m3/vendor-sync/webrtc.md) - tracks the NetEq port this deletes diff --git a/quest/m2/audio-playout/output-latency.md b/quest/m2/audio-playout/output-latency.md new file mode 100644 index 00000000000..4c971d79564 --- /dev/null +++ b/quest/m2/audio-playout/output-latency.md @@ -0,0 +1,45 @@ +# [M] Audio is placed on the clock where it is heard + +## Goal + +Each player compares audio with the shared clock at the moment the audio +leaves the speaker, so audio and video stay in sync on any output device, and +the audio target counts that output latency. The browser rests on the clock +as native does, without the underruns a clock pairing alone caused, and +native's fixed 50 ms floor, which raises the shared delay for video too, is +replaced by its device's actual latency. + +## Plan + +Planned 2026-11-11 (maintainer, /quest-merge pick of +[moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507)) for the +browser's render callback. Widened 2026-11-13 (maintainer, /quest-plan): +✅ lag is measured where audio is heard, in both players; the +[epic](/quest/m2/audio-playout/README.md) records the decision. + +Neither player reads its output latency today. The browser never reads +`baseLatency` or `outputLatency`. Natively `playback/driver.rs` places audio +by the host clock at the callback and hands the device's reported latency +only to the playback ledger, and `moq play` floors the audio target at 50 ms +(`AUDIO_BUFFER_MIN` in `play/media.rs`) instead. So audio is +heard later than video by the device latency: tens of milliseconds on a +desktop sink, 150 to 250 ms over Bluetooth. + +The browser evidence: headless Chromium renders 2048-frame callbacks +(`baseLatency` 42.7 ms) and pairs the clock with `currentTime` just after +each one, so it rests 28 to 35 ms behind where native rests within a few ms; +Nightly's near-zero rows show +12 to +20 ms since #8507. A max-filtered +wall-clock pairing put audio on the clock (p50 -1.3 to 4.5 ms) but underran +near-zero Opus 45 times a minute, since the target did not count the +callback. + +Recommendations, in one PR: + +1. Add the output latency to each player's audio target: the render quantum + or measured callback size plus `outputLatency` where the browser reports + it, and natively the device period plus the latency the host reports. The + shared delay then covers it, so video waits that much longer, which lip + sync on that device needs. +2. Read the clock at output time: in the browser, pair it on the wall clock + with the max filter; natively, use the callback's playback timestamp. +3. Measure the live rows before and after in both players. diff --git a/quest/m2/audio-playout/target-fall.md b/quest/m2/audio-playout/target-fall.md new file mode 100644 index 00000000000..297ddd7b08b --- /dev/null +++ b/quest/m2/audio-playout/target-fall.md @@ -0,0 +1,36 @@ +# [M] The target falls back soon after a one-off spike + +## Goal + +After a one-off arrival spike, such as the audio-quality `congestion` row's +1.5 s outage, the playout target, and with it video's delay, returns to its +level before the spike sooner than today's 10 to 23 s, without more underruns +on the `wide`, `bursty`, and `step` rows. One rule in +`doc/concept/audio-jitter.md`, held by its corpus, in both estimators. + +## Plan + +Decided 2026-11-13 (maintainer, /quest-plan): ✅ the implementer prototypes +and picks by the Nightly rows (rejected: a shorter half-life overall now; +counting an outage as one observation now). This reopens the 2026-11-09 +decision that the estimator stays as it is; the +[epic](/quest/m2/audio-playout/README.md) records the rest. + +Why it holds: each 500 ms interval of the outage folds its own peak, so three +or four folds put about 7% of the weight past the 5% the 0.95 quantile +needs, and the 20 s half-life takes 10 to 23 s to decay it. The 11-13 Opus +row's target fell 1020, 720, 340, then 140 ms, holding the whole broadcast +seconds behind live after the link had recovered. + +Candidates: + +- Count one outage, the empty intervals and the burst that ends them, as one + observation. +- Decay the weight above the current target faster than the rest. +- A shorter half-life overall, which re-stalls more on periodic bursts. + +Judge each by how long the congestion row's target stays raised and by +underruns on the other rows, in both players, and add corpus cases for the +pick. A fall is lag like any other under the +[lag rule](/quest/m2/audio-playout/lag-rule.md), so a fall past the +threshold costs one snap rather than a slow catch-up. diff --git a/quest/m3/audio-calibrate.md b/quest/m3/audio-calibrate.md new file mode 100644 index 00000000000..c1c9c3cbe62 --- /dev/null +++ b/quest/m3/audio-calibrate.md @@ -0,0 +1,62 @@ +# [S] Re-measure every audio ceiling after the playout rule lands + +## Goal + +`test/audio-quality/budgets.json` carries every browser and native row's +ceilings re-measured, by the rule in its note, from seven scheduled Nightly +runs on a `main` that includes the whole +[audio playout](/quest/m2/audio-playout/README.md) epic. The rows budget +steady-state lag, the time audio takes to return to the clock after a +disturbance, the player's own trims and skips, and underruns as measured, +and the note says the rule is in them. + +## Plan + +Planned 2026-11-09 (maintainer, /quest-plan) for the stretch model. This +replaced two waits that were calibrating a player the model changes: the +chromium re-measure after +[moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058) and the native +one after [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308). + +Moved 2026-11-13 (maintainer, /quest-plan) to m3 behind the epic: ✅ the cut +waits on the playout rule, not on a week of Nightlies to re-budget ceilings +that ship nothing (rejected: calibrating in m2 with the epic). Nights before +the epic's last child merges do not count; this supersedes the same day's +decision to wait only for the congestion fix. + +Carried decisions: + +- 2026-11-07 (maintainer, on #8058): ✅ land the player change, then measure + (rejected: hold it until Nightly ceilings are in). The goal is judged by + the rows alone, with no listening run. +- 2026-11-09 (maintainer, on + [moq-dev/moq#8308](https://github.com/moq-dev/moq/pull/8308#issuecomment-6730322286)): + ✅ budget `trimmed_ms_per_min` and `player_skips_per_min` (rejected: + budgeting accelerates as discarded audio), and ✅ the nightly runner's + playback benchmarks count as the quiet-host run (rejected: naming another + host). + +The rule: the worst of seven clean nights times 1.25, plus the documented +runner hiccup, rounded to two significant figures. A developer run is not +that calibration. Keep each night's rows in the PR body. + +What to report rather than budget, since each means the rule is not doing its +job there: a steady-state `av_lag_ms` p50 more than a frame off zero on any +row; audio not back on the clock within about 2 s of a disturbance; native +Opus `step` still filing the 26 to 27 underruns a minute it has since #8507, +against 0 before. + +Native rows stay report-only; enforcing them belongs to +[Audio outlives video under bufferbloat](/quest/m4/audio-bufferbloat-av.md). + +## Waiting + +- moq-dev/moq: seven scheduled Nightly runs on `main` after the epic's last + child merges. Find them with `gh run list --workflow nightly.yml --event + schedule` and download each with `gh run download -n + audio-quality-run` as it lands, since artifacts expire after 7 days. Never + dispatch the nightly by hand. + +## Required + +- [Audio playout](/quest/m2/audio-playout/README.md) - the rule whose rows these ceilings budget diff --git a/quest/m3/vendor-sync/webrtc.md b/quest/m3/vendor-sync/webrtc.md index 95256e2bc68..8ae3b6ea1b1 100644 --- a/quest/m3/vendor-sync/webrtc.md +++ b/quest/m3/vendor-sync/webrtc.md @@ -34,7 +34,9 @@ a revision. Start from the last WebRTC `main` commit on 2026-11-04, and record the revision checked in `stretch.rs`'s module doc. The port is a deliberate floating-point adaptation (maintainer, 2026-11-04), so an upstream change to Q15 arithmetic alone does not apply; `doc/concept/audio-jitter.md` and its PCM -corpus define the algorithm, so update both with any ported change. +corpus define the algorithm, so update both with any ported change. The +[lag rule](/quest/m2/audio-playout/lag-rule.md) replaces the port with a +continuous time stretch before the cut; once it lands, drop that half. WebRTC's gitiles history pages need a Google sign-in; clone `https://webrtc.googlesource.com/src` instead. diff --git a/quest/m4/audio-bufferbloat-av.md b/quest/m4/audio-bufferbloat-av.md index cbb7b123053..7fdb85e42e4 100644 --- a/quest/m4/audio-bufferbloat-av.md +++ b/quest/m4/audio-bufferbloat-av.md @@ -24,10 +24,11 @@ Decided in planning (2026-10-20): audio. Video freezes, skips, and lag are reported, not gated: the point is that audio does not pay for them. Since 2026-11-05 the shared clock never follows audio, so congested video lags it by design, and steady-state lip - sync belongs to the audio ring's time stretch. Since 2026-11-09 audio may - trail the clock through the congestion and catch up after it - ([stretch model](https://github.com/moq-dev/moq/pull/8507)), so these rows - gate audio dropped, not audio late. + sync belongs to the audio ring's time stretch. Since 2026-11-13 audio + trails the clock by at most its 200 ms `stretch` threshold and snaps onto + it past that ([audio playout](/quest/m2/audio-playout/README.md)), so these + rows gate how soon audio is back on the clock and its underruns, and budget + the audio it skips rather than forbid it. - Folded in: the native `pulse` rows become enforced. They report only because an intermittent two-second device stall voids about one AAC native row in seven (see `test/audio-quality/README.md`). Reproduce and fix that @@ -57,5 +58,5 @@ Public API: none. Wire: none. ## Related -- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the estimator whose budgets these rows extend +- [Audio playout](/quest/m2/audio-playout/README.md) - the playout rule and estimator whose budgets these rows extend - [ViSQOL grading](/quest/m4/audio-visqol.md) - adds a quality score to the same harness From ec92d2cfa923ff54dbddd914b121edd836bc9118 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 12:13:56 -0800 Subject: [PATCH 3/5] quest(m2): log the audio playout epic's move Co-Authored-By: Claude Opus 5.5 --- quest/m2/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/quest/m2/README.md b/quest/m2/README.md index 38f1ba60207..53addd4a318 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -191,3 +191,11 @@ quests that require them. Decided 2026-11-13 (maintainer, /quest-run): ✅ m2 no longer requires the m1 stream-credit epic (rejected: keep the gate). It was an ordering rule, not a code dependency, and with takeover moved to m3 the epic changes no wire. + +Decided 2026-11-13 (maintainer, /quest-plan from an audit of `main`'s audio): +✅ the [audio playout](/quest/m2/audio-playout/README.md) epic moves here from +m1, with its calibration in m3 (rejected: calibration here too; only the lag +rule here). Its lag rule reshapes `main`-only API the cut would publish (the +`stretch` setting, moq-audio's `Sink::due` and `bus::Positioned`), and the +cut should not ship the 2026-11-09 stretch model, which lets audio trail +video by up to 2 s. From bf248274b8cac64d51704aebaa4c8020523c84ea Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 12:26:05 -0800 Subject: [PATCH 4/5] quest: address review of the audio playout plan Buffered players keep their lookahead in the goal and the congestion fix, the browser target counts baseLatency, and the send-depth gate builds what failed. Co-Authored-By: Claude Opus 5.5 --- quest/m0/nightly/send-depth-gate.md | 20 ++++++++++++------- quest/m2/audio-playout/README.md | 3 ++- quest/m2/audio-playout/congestion-catch-up.md | 7 +++++-- quest/m2/audio-playout/output-latency.md | 7 ++++--- 4 files changed, 24 insertions(+), 13 deletions(-) diff --git a/quest/m0/nightly/send-depth-gate.md b/quest/m0/nightly/send-depth-gate.md index ff4936a422a..c6ba7ff3a07 100644 --- a/quest/m0/nightly/send-depth-gate.md +++ b/quest/m0/nightly/send-depth-gate.md @@ -4,16 +4,17 @@ A PR that pushes the session driver's `Send` proof past the compiler's default recursion limit fails the merge gate, instead of landing and breaking every -Fuzz job in the next Nightly. The check is the one Nightly already runs -(`just rs send-depth` on the pinned nightly toolchain, and the Fuzz build's -`-D recursion_depth_exceeding_limit`). +Fuzz job that builds moq-net in the next Nightly. The gate builds what +failed: in run 43447849289 `Tests (rs send-depth)`, which checks only +moq-net's `send` test, passed while the moq-net and moq-rtmp fuzz builds +failed under the same `-D recursion_depth_exceeding_limit`. ## Plan Found by the triage of Nightly run [43447849289](https://github.com/moq-dev/moq/actions/runs/43447849289) -(2026-11-13): every Fuzz job failed with `overflow evaluating the requirement -driver::Driver: Send`. [moq-dev/moq#8545](https://github.com/moq-dev/moq/pull/8545) +(2026-11-13): the 11 moq-net and moq-rtmp Fuzz jobs failed with `overflow +evaluating the requirement driver::Driver: Send`. [moq-dev/moq#8545](https://github.com/moq-dev/moq/pull/8545) deepened the proof past the limit, and [moq-dev/moq#8657](https://github.com/moq-dev/moq/pull/8657) fixed it with 12 levels to spare, so the next addition to session state can break it again. @@ -21,8 +22,13 @@ levels to spare, so the next addition to session state can break it again. Decided 2026-11-13 (maintainer, /quest-plan): ✅ add it to the merge gate (rejected: keep it Nightly-only, which lets the break land on `main`; drop the quest). Measure what it adds to the gate and say so in the PR, per the -gate-cost rule. Reuse the existing recipe; pick the cheapest gate job that -already has the pinned nightly toolchain. +gate-cost rule. Pick the cheapest gate job that already has the pinned +nightly toolchain. Corrected 2026-11-13 (review of +[moq-dev/moq#8663](https://github.com/moq-dev/moq/pull/8663)): reusing `just +rs send-depth` as it stands would not have caught this break, so first find +why the fuzz build overflows where the `send` test does not (its features, +profile, or sanitizer flags), then make `send-depth` check that or gate on a +fuzz target's build. Public API: none. Wire: none. diff --git a/quest/m2/audio-playout/README.md b/quest/m2/audio-playout/README.md index 461999a07a1..e944482a3e3 100644 --- a/quest/m2/audio-playout/README.md +++ b/quest/m2/audio-playout/README.md @@ -9,7 +9,8 @@ are in sync, and neither waits on the other or on the other's settings. A discrepancy up to the `stretch` threshold, 200 ms by default, is corrected by time stretch at no more than 10%, so audio is back on the clock within about 2 s; a larger one snaps the way video does, skipping late audio or padding -silence when audio is ahead. +silence when audio is ahead. A buffered player keeps what it holds: it snaps +only past the larger of its lookahead and the threshold. The playout target is a measured estimate of arrival timing, from one written algorithm both players implement, and it falls back soon after a one-off diff --git a/quest/m2/audio-playout/congestion-catch-up.md b/quest/m2/audio-playout/congestion-catch-up.md index ed6a80dd554..da81ef61ccd 100644 --- a/quest/m2/audio-playout/congestion-catch-up.md +++ b/quest/m2/audio-playout/congestion-catch-up.md @@ -34,9 +34,12 @@ Scope: - `stretch` defaults to 200 ms (`js/watch/src/sync.ts` `STRETCH`, `moq play --stretch`); zero still turns the stretch off. - A lag past the threshold skips onto the clock, not back to the bound, in - `js/watch/src/audio/ring-buffer.ts` and `rs/moq-audio/src/ring.rs`. + `js/watch/src/audio/ring-buffer.ts` and `rs/moq-audio/src/ring.rs`. A + buffered player skips only past the larger of its lookahead and the + threshold, as today. - The setting sizes nothing else. The audio subscription keeps the delay - plus the estimator's 2 s ceiling, so late frames are still observed. + plus the larger of the lookahead and the estimator's 2 s ceiling, so late + frames are still observed and a buffered player keeps its lookahead. Natively, the decode cap, the hole policy (`fill_max`), and video's wait on a missing group (`args.rs` `video_max_delay`) stop following it, so video waits its own delay. That also fixes `--delay auto --stretch 0`, where any diff --git a/quest/m2/audio-playout/output-latency.md b/quest/m2/audio-playout/output-latency.md index 4c971d79564..c53bf9e2047 100644 --- a/quest/m2/audio-playout/output-latency.md +++ b/quest/m2/audio-playout/output-latency.md @@ -35,9 +35,10 @@ callback. Recommendations, in one PR: -1. Add the output latency to each player's audio target: the render quantum - or measured callback size plus `outputLatency` where the browser reports - it, and natively the device period plus the latency the host reports. The +1. Add the output latency to each player's audio target: in the browser, + `baseLatency` (or the measured callback size where it is missing) plus + `outputLatency` where reported, and natively the device period plus the + latency the host reports. The shared delay then covers it, so video waits that much longer, which lip sync on that device needs. 2. Read the clock at output time: in the browser, pair it on the wall clock From a91c412a5b3ee0777c9a7e28f63c481b9c4eb446 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 12:28:28 -0800 Subject: [PATCH 5/5] quest(m2/audio-playout): congestion catch-up keeps the max delay #8611 skips holes at Co-Authored-By: Claude Opus 5.5 --- quest/m2/audio-playout/congestion-catch-up.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m2/audio-playout/congestion-catch-up.md b/quest/m2/audio-playout/congestion-catch-up.md index da81ef61ccd..e6f45192dcc 100644 --- a/quest/m2/audio-playout/congestion-catch-up.md +++ b/quest/m2/audio-playout/congestion-catch-up.md @@ -40,7 +40,9 @@ Scope: - The setting sizes nothing else. The audio subscription keeps the delay plus the larger of the lookahead and the estimator's 2 s ceiling, so late frames are still observed and a buffered player keeps its lookahead. - Natively, the decode cap, the hole policy (`fill_max`), and video's wait on + Since [moq-dev/moq#8611](https://github.com/moq-dev/moq/pull/8611) that max + delay is also where buffered media past a missing group skips it, so the + threshold must not shrink it. Natively, the decode cap, the hole policy (`fill_max`), and video's wait on a missing group (`args.rs` `video_max_delay`) stop following it, so video waits its own delay. That also fixes `--delay auto --stretch 0`, where any hole over 50 ms drops the sink and its queued audio (`play/media.rs` @@ -58,4 +60,3 @@ native in the PR. ## Related - [Resume sync](/quest/m0/interop-green/resume-sync.md) - snaps resumed audio to the clock, the same rule for a return from absence -- [Python to C++ audio gap](/quest/m0/test-flakes-2/interop-python-cpp-audio-gap.md) - also changes how long a missing group is waited on