diff --git a/quest/m1/audio-jitter-target/README.md b/quest/m1/audio-jitter-target/README.md index be98187f643..d2d108b062b 100644 --- a/quest/m1/audio-jitter-target/README.md +++ b/quest/m1/audio-jitter-target/README.md @@ -42,7 +42,7 @@ native playback against its budgets. - [Watch](/quest/m1/audio-jitter-target/watch.md) - the browser plays the measured real-microphone rows with zero steady-state underruns by time-stretching the ring - [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - `moq play` copies NetEq's accelerate and preemptive expand, sharing the browser's algorithm and vectors, so its native rows reach zero steady-state underruns -- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - post-cap nightlies confirm AAC bursty discards about as much audio as Opus bursty and the first row no longer stalls, and every browser row's budgets are re-measured from them +- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the AAC browser budgets are re-measured from nightlies after the TS importer groups each PES ## Closes diff --git a/quest/m1/audio-jitter-target/capped-budgets.md b/quest/m1/audio-jitter-target/capped-budgets.md index a42edf0ea51..586b61f9f80 100644 --- a/quest/m1/audio-jitter-target/capped-budgets.md +++ b/quest/m1/audio-jitter-target/capped-budgets.md @@ -1,118 +1,52 @@ -# [S] Re-measure the browser budgets after the estimator capped one observation's weight, and group AAC bursts +# [XS] Re-measure the AAC browser budgets after the TS importer groups each PES ## Goal -Seven scheduled nightlies on `main` from 2026-10-29 on, after the estimator -began capping how much weight one observation takes, the player changes -that followed, and the relay history-fill fix, show the AAC `bursty` browser rows discarding -about as much audio a minute as the Opus `bursty` rows, and the Opus -`near-zero` row holding a 60 ms target with no underrun episodes. `test/audio-quality/budgets.json` -carries every browser row's ceilings re-measured from them. The AAC -publisher stops sending each seven-frame flush as seven groups newest -first, so AAC `wide` stops running dry. +`test/audio-quality/budgets.json` carries the six AAC browser rows' ceilings +re-measured, by the rule in its note, from seven scheduled nightlies on `main` +that all ran after the TS importer began publishing each PES as one group, and +the note's sentence that the AAC rows predate the grouping is dropped. Nearly +every AAC frame on those nights arrives in order. ## Waiting -- moq-dev/moq: seven scheduled Nightly runs on `main` from 2026-10-29 on, the - seventh due about 2026-11-04. Check with `gh run download -n - audio-quality-run` on each; never dispatch the nightly by hand. The artifacts - expire after 7 days, so download each run's as it lands rather than waiting - for the seventh. +- moq-dev/moq: seven scheduled Nightly runs on `main` after + [moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) merges. Find them + with `gh run list --workflow nightly.yml --event schedule` and download each + with `gh run download -n audio-quality-run`; never dispatch the nightly by + hand. The artifacts expire after 7 days, so download each run's as it lands + rather than waiting for the seventh. ## Plan -Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs): plan the AAC publisher fix here. On -[moq-dev/moq#7679](https://github.com/moq-dev/moq/pull/7679)'s runs AAC -`wide` ran dry 1 to 4 times a minute at a 323 ms target. The publisher sends -each seven-frame PES flush as seven one-frame groups, delivered newest first, -so the oldest frame waits behind every newer group (the cause predicted -below). Send a flush so its oldest frame goes first, for example as one -group or in ascending order, and re-measure the AAC rows after it. Options -shown: - -- m0 -- ✅ m1, in this quest -- m2 - -Decided 2026-10-30 (maintainer): the count restarts at the 2026-10-29 -nightly, the first after -[moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168), because the -2026-10-25 to 10-28 nightlies measured the relay history-fill bug it fixed -(below). One has run (2026-10-29), with the seventh due about 2026-11-04. -This replaces the 2026-10-27 restart at the 2026-10-25 nightly, which -followed the post-cap estimator and player changes -([moq-dev/moq#6359](https://github.com/moq-dev/moq/pull/6359), -[moq-dev/moq#6525](https://github.com/moq-dev/moq/pull/6525), -[moq-dev/moq#6484](https://github.com/moq-dev/moq/pull/6484), -[moq-dev/moq#6609](https://github.com/moq-dev/moq/pull/6609)). No earlier -run counts toward the seven. Re-measure by the rule in the budgets note, and drop the note's sentence -that budgets the AAC bursty discards against Opus's as measured. Nightlies -from before the one-ring removal also carry `-isolated` rows; grade only the -`-plain` rows, which are the browser rows now. - -Decided 2026-10-22 (maintainer): the Opus near-zero budgets quest folded in -here, so one pass over the same seven nightlies covers both. The cap -([#6164](https://github.com/moq-dev/moq/pull/6164)) merged 2026-10-22T00:10Z, -after the build flush, so post-cap nightlies are also post-flush. - -The first post-cap scheduled run, [2026-10-22 nightly](https://github.com/moq-dev/moq/actions/runs/39779729461), -failed only the browser Opus `wide` target ceilings: `isolated` p95 was -380 ms against 270 ms, and `plain` was 400 ms against 300 ms. In the -`audio-quality-run` artifact, both `chromium-opus-48000-wide-*.ndjson` -series hold their startup target until about 21 s, then settle at 200 to -220 ms; both matching `.longtasks.json` arrays are empty. The isolated -row has zero underruns, stalls, and silence; plain has 2.2 underruns a -minute, 0 stalls, and 0 silence. It predates the restarted count, and is -not a reason to change the estimator or recalibrate early. - -The near-zero stall was never the isolated ring. That row is simply the -matrix's first, and a cold `cargo build` left about 800 MB dirty when it -started. On the ARM runner each periodic writeback burst froze the page's main -thread for 100 to 320 ms until the backlog drained, and the harness now -flushes its build before the first row. If the first row still shows -long-task clusters, compare `.longtasks.json` against the runner's disk -activity before changing the player. - -The cap changed every row's startup, not only AAC bursty's: the prior is -replaced in about five seconds instead of two, and a late interval in those -first seconds holds the target for up to one 20 s half-life instead of falling -back within seconds. Replays of local captures held the clean rows' targets -where they were, but one live Opus `near-zero` `plain` run held a startup -hiccup at 160 ms for 20 s, over that row's 95 ms ceiling. If a nightly breaches -a startup target ceiling that way, budget it as measured rather than reopening -the estimator. - -On replays of the 2026-10-18 nightly capture and of two local captures, the -cap cut AAC bursty discards from 540 to 860 ms a minute to 0 to 170, and its -target p95 from 663 ms to 343 to 363. If the nightlies still show AAC bursty -discarding several times Opus's, the remaining source is the publisher: each -seven-frame PES flush is seven one-frame groups, delivered newest first, so the -oldest frame of a flush waits behind every newer group on a path that cannot -carry them together. - -The 2026-10-25 to 10-28 nightlies -([40192013303](https://github.com/moq-dev/moq/actions/runs/40192013303), -[40332938121](https://github.com/moq-dev/moq/actions/runs/40332938121), -[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673), -[40698329026](https://github.com/moq-dev/moq/actions/runs/40698329026)) -measured a relay bug rather than the player: 2 to 4.5 s into every row except -`fixed-250` and Opus `near-zero`, the subscriber received 15 to 25 s of the -publisher's cached history, newest first. On the `wide` rows the estimator -took that as a late interval and held a 2020 ms target for about 20 s, -then discarded about 1.8 to 1.9 s of audio a minute falling back, every night; -the 10-28 Opus `bursty` 540 ms target is the same arrival. No `wide` or -`bursty` row logged more than one long task, of at most 58 ms. -[moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168) stopped a fresh -subscriber filling history below its start, merged 2026-10-29T00:52Z, and the -[2026-10-29 nightly](https://github.com/moq-dev/moq/actions/runs/40885347517) -is the first after it and the first that counts: the oldest frame any row -received is about 2.6 s behind live, every browser row is within budget, and -AAC `wide` discards 205.6 ms/min at a 323.2 ms target p95, Opus `wide` 243.4 -at 460. On it AAC `bursty` discards 162.9 ms a minute against 0 for Opus -`bursty`, and Opus `near-zero` has no underrun episodes at a 60 ms target -p95. Across the four earlier nights Opus `near-zero` held 60 ms twice and -80 twice: the 120 ms prior decays to 60 only at 21.4 s, after the 5 s -warmup. Each night's browser metrics are kept -in [moq-dev/moq#7286](https://github.com/moq-dev/moq/pull/7286), since the -artifacts expire. The separate publisher fairness timeout was -excess per-object work, fixed by [moq-dev/moq#7133](https://github.com/moq-dev/moq/pull/7133). +Decided 2026-11-04 (maintainer, on +[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898)): land one group +per PES as is. The extra AAC `wide` re-stalls are for Watch's time stretch to +absorb, and this quest no longer expects AAC `wide` to stop running dry. +Options shown: + +- ✅ Land as is +- Land only the budgets, holding the importer change until time stretch lands +- Keep the change and deepen AAC's target + +[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) re-measured every +browser row from the 2026-10-29 to 2026-11-04 nightlies (its body keeps each +night's metrics) and changed the publisher: `moq import ts` used to send each +seven-frame AAC PES as seven one-frame groups, which the transport delivers +newest first, so 82 to 86% of AAC frames arrived behind a newer one. Each PES is +now one group, its frames in presentation order. + +On those nights, before the change, AAC `bursty` discarded up to 368.3 ms a +minute against Opus `bursty`'s 117.4. Replaying their captures with each PES's +arrival slots handed to its frames oldest first, as one group, halved the AAC +`bursty` and `wide` discards (`bursty` summed over the seven nights 1156 to 584 +ms a minute) and lowered both rows' target by 20 to 60 ms. That shallower target doubled AAC +`wide`'s player re-stalls (mean 3.0 to 6.6 a minute), the 0.95 quantile's trade +that [Watch](/quest/m1/audio-jitter-target/watch.md)'s time stretch absorbs. + +AAC `bursty` still discards about three times what Opus `bursty` does (584 +against 174 summed), and parity is not expected: the harness keeps ffmpeg's +default PES packing on purpose, so AAC frames reach the viewer a median 210 ms +behind their media time on `bursty` against Opus's 41. Grade the same +`chromium-aac-*` rows, and check the in-order claim with the share of each row's +`.arrivals.ndjson` frames that arrive behind a newer one. diff --git a/quest/m1/audio-jitter-target/native-stretch.md b/quest/m1/audio-jitter-target/native-stretch.md index 50f37c7de61..f14c509dc67 100644 --- a/quest/m1/audio-jitter-target/native-stretch.md +++ b/quest/m1/audio-jitter-target/native-stretch.md @@ -83,8 +83,8 @@ loaded 24-core host (load 13 to 55, seed 7, 60 s) gave target p95s of `main`, a second late interval at 7 s. Every run of this branch had zero underruns; `main` had up to 14.1 a minute. The 2026-10-29 to 11-03 scheduled nightlies on `main` read 160 to 280 ms. So the ceiling stays as it is, -as [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) directs -for a startup breach, until the seven-nightly calibration below. +as `test/audio-quality/budgets.json`'s note budgets startup holds as measured, +until the seven-nightly calibration below. ## Waiting diff --git a/quest/m1/audio-jitter-target/watch.md b/quest/m1/audio-jitter-target/watch.md index 774c39821e6..74f5dc24c53 100644 --- a/quest/m1/audio-jitter-target/watch.md +++ b/quest/m1/audio-jitter-target/watch.md @@ -165,6 +165,6 @@ A/V clock work. ## Related -- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - confirms the build-flush fix on the nightlies, no gate on this proof +- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - re-measures the AAC browser budgets after the publisher change, no gate on this proof - [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - the native time-stretch sharing one NetEq algorithm and its test vectors with this one - [Watch worker](/quest/m2/watch-worker.md) - moves the worklet this stretch changes diff --git a/quest/m1/av-clock.md b/quest/m1/av-clock.md index 4996641baf4..7a7726ec1fd 100644 --- a/quest/m1/av-clock.md +++ b/quest/m1/av-clock.md @@ -58,14 +58,13 @@ Recommendations for the implementation: catalog schema (`js/msf/src/catalog.ts`) accepts a negative `delay` and folds it into absent; refuse it on decode instead. -Decided 2026-10-27 (maintainer): this waits only on -[capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md), not the -whole [audio jitter target](/quest/m1/audio-jitter-target/README.md) epic, -whose remaining manual browser proof does not gate reshaping the clock. - -## Required - -- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the estimator's browser budgets, re-measured, before this reshapes the clock around them +Decided 2026-10-27 (maintainer): this waits only on the estimator's browser +budgets being re-measured, not the whole +[audio jitter target](/quest/m1/audio-jitter-target/README.md) epic, whose +remaining manual browser proof does not gate reshaping the clock. +[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) re-measured every +browser row. What [capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) +still owes, the AAC rows after a publisher change, does not gate it either. ## Related diff --git a/rs/moq-mux/src/codec/legacy.rs b/rs/moq-mux/src/codec/legacy.rs index 41f291a95e1..f6ccd5b96a9 100644 --- a/rs/moq-mux/src/codec/legacy.rs +++ b/rs/moq-mux/src/codec/legacy.rs @@ -115,9 +115,9 @@ pub(crate) struct Config { /// Legacy audio importer. /// -/// Publishes each whole frame as one hang frame in its own group, so the relay -/// forwards it immediately. The audio is never decoded; the catalog carries the -/// codec, sample rate and channel count read from the frame header. +/// Publishes each whole frame as one hang frame, into the current group until the +/// caller cuts it. The audio is never decoded; the catalog carries the codec, +/// sample rate and channel count read from the frame header. pub(crate) struct Import { track: crate::container::Producer, } diff --git a/rs/moq-mux/src/container/ts/import.rs b/rs/moq-mux/src/container/ts/import.rs index f12c34b4398..f66e80b617f 100644 --- a/rs/moq-mux/src/container/ts/import.rs +++ b/rs/moq-mux/src/container/ts/import.rs @@ -995,6 +995,7 @@ impl Import { tail_pts: None, resync: Resync::new(pid.as_u16(), ".aac"), burst: std::time::Duration::ZERO, + last: None, })), // Legacy broadcast audio, carried verbatim. Both MP2 stream types // (0x03 MPEG-1, 0x04 MPEG-2 half rate) share one parser; sample rate and @@ -1018,6 +1019,7 @@ impl Import { config.container = self.container.clone(); Stream::Opus(Box::new(OpusStream { import: opus::Import::new(track, self.reserve(), config)?, + last: None, })) } Err(err) => self.uncarried( @@ -1087,6 +1089,7 @@ impl Import { tail: Vec::new(), tail_pts: None, resync: Resync::new(pid.as_u16(), descriptor.track_suffix), + last: None, })) } @@ -3731,10 +3734,28 @@ struct AacStream { resync: Resync, // Completed audio duration since the last video PES, measured independently for each PID. burst: std::time::Duration, + /// The PTS of the last frame `import` published: see [`forward`]. + last: Option, } impl AacStream { fn write(&mut self, pending: Pending, batched: bool) -> anyhow::Result { + // The frames a PES completes share one group, closed once the PES is spent, even by an + // error. The transport sends a track's newest group first, so a group per frame would + // deliver a multi-frame PES newest first, its oldest frame waiting behind the rest. + let mut open = None; + let result = self.split(pending, batched, &mut open); + // A failed cut is a publishing failure, which outranks a damaged unit the caller recovers from. + if let (Some(end), Some(import)) = (open, &mut self.import) { + import.cut(Some(end))?; + } + result + } + + /// Publish the frames `pending` completes into one group, leaving `open` at where its last frame + /// stops while it is open. A frame without a PTS is stamped by the producer, which checks only + /// where groups start, so it keeps a group of its own, as every frame once did. + fn split(&mut self, pending: Pending, batched: bool, open: &mut Option) -> anyhow::Result { let pes_base = on_clock(pending.pts, pending.offset)?; // Prepend the partial frame left by the previous PES, if any. @@ -3871,8 +3892,12 @@ impl AacStream { name = import.name(), "AAC format changed mid-stream, starting a new track" ); + if let Some(end) = open.take() { + import.cut(Some(end))?; + } import.finish()?; self.import = None; + self.last = None; } let import = match &mut self.import { Some(import) => { @@ -3926,14 +3951,20 @@ impl AacStream { } }; + forward(&mut self.last, pts)?; + if pts.is_none() + && let Some(end) = open.take() + { + import.cut(Some(end))?; + } import.decode(block, pts)?; // Count only completed frames; input gaps and unfinished tails are not a media burst. burst += std::time::Duration::from_nanos((1024_u64 * 1_000_000_000).div_ceil(header.sample_rate as u64)); - // The importer accumulates; cut each ADTS frame into its own group (one QUIC stream) - // so the relay forwards it without waiting for the next. Its known endpoint also - // bounds the final HLS segment, since a one-frame group has no measured cadence. pts = advance_pts(pts, 1024, header.sample_rate)?; - import.cut(pts)?; + match pts { + Some(end) => *open = Some(end), + None => import.cut(None)?, + } self.resync.published(unvouched); published += 1; // Offsets behind the published frame are spent; carrying one would republish it. @@ -4027,6 +4058,8 @@ impl AacStream { /// more Opus packets, each prefixed by the Opus-in-TS control header. struct OpusStream { import: opus::Import, + /// The PTS of the last packet `import` published: see [`forward`]. + last: Option, } impl OpusStream { @@ -4042,13 +4075,21 @@ impl OpusStream { // Default to 20 ms (960 samples) if the TOC can't be read, so a malformed packet // doesn't stall the timeline for the rest of the PES. elapsed += opus::packet_samples(packet).unwrap_or(960) as u64; - let end = advance_pts(base, elapsed, 48_000)?; + forward(&mut self.last, pts)?; self.import.decode(packet, pts)?; - // The importer accumulates; cut each Opus packet into its own group (one QUIC stream) - // so the relay forwards it without waiting for the next. Its known endpoint also - // bounds the final HLS segment, since a one-packet group has no measured cadence. - self.import.cut(end)?; published += 1; + // The producer stamps a packet without a PTS and checks only where groups start, so it + // keeps a group of its own. + if pts.is_none() { + self.import.cut(None)?; + } + } + // One group per PES, at its known end: see `AacStream::write`. A failed decode ends the + // import, so there is no open group to close on that path. + if published > 0 + && let Some(end) = advance_pts(base, elapsed, 48_000)? + { + self.import.cut(Some(end))?; } Ok(published) } @@ -4314,10 +4355,26 @@ struct LegacyStream { /// only covers frames that begin in that PES. tail_pts: Option, resync: Resync, + /// The PTS of the last frame `import` published: see [`forward`]. + last: Option, } impl LegacyStream { fn write(&mut self, pending: Pending) -> anyhow::Result { + // One group per PES, closed even by an error: see `AacStream::write`. + let mut open = None; + let result = self.split(pending, &mut open); + // A failed cut is a publishing failure, which outranks a damaged unit the caller recovers from. + if let (Some(end), Some(import)) = (open, &mut self.import) { + import.cut(Some(end))?; + } + result + } + + /// Publish the frames `pending` completes into one group, leaving `open` at where its last frame + /// stops while it is open. A frame without a PTS is stamped by the producer, which checks only + /// where groups start, so it keeps a group of its own, as every frame once did. + fn split(&mut self, pending: Pending, open: &mut Option) -> anyhow::Result { let mut published = 0; let pes_base = on_clock(pending.pts, pending.offset)?; @@ -4464,8 +4521,12 @@ impl LegacyStream { name = import.name(), "audio format changed mid-stream, starting a new track" ); + if let Some(end) = open.take() { + import.cut(Some(end))?; + } import.finish()?; self.import = None; + self.last = None; } let import = match &mut self.import { Some(import) => import, @@ -4489,12 +4550,18 @@ impl LegacyStream { } }; + forward(&mut self.last, pts)?; + if pts.is_none() + && let Some(end) = open.take() + { + import.cut(Some(end))?; + } import.decode(&data[offset..end], pts)?; - // The importer accumulates; cut each frame into its own group (one QUIC stream) - // so the relay forwards it without waiting for the next. Its known endpoint also - // bounds the final HLS segment, since a one-frame group has no measured cadence. pts = advance_pts(pts, header.samples, header.sample_rate)?; - import.cut(pts)?; + match pts { + Some(end) => *open = Some(end), + None => import.cut(None)?, + } self.resync.published(unvouched); published += 1; // Offsets behind the published frame are spent; carrying one would republish it. @@ -4592,6 +4659,22 @@ fn skip_missing_keyframe(result: crate::Result<()>) -> anyhow::Result { } } +/// Refuse an audio frame that starts below the last one its track published: a restart, which is +/// a new broadcast. Audio never reorders, and the producer only checks where groups start, which +/// is once a PES. A frame without a PTS is left to the producer, in a group of its own. +fn forward(last: &mut Option, pts: Option) -> crate::Result<()> { + let Some(pts) = pts else { + return Ok(()); + }; + if let Some(floor) = *last + && pts < floor + { + return Err(crate::container::TimestampRewind { timestamp: pts, floor }.into()); + } + *last = Some(pts); + Ok(()) +} + /// Advance a PES-derived PTS past one frame of `samples` at `sample_rate`. /// /// Per frame rather than `index * constant`: E-AC-3 varies the samples per frame, and a @@ -5870,8 +5953,8 @@ pub(super) mod test { p } - /// ffmpeg packs several ADTS frames into one PES, and the importer cuts every one of them into - /// its own group in a single synchronous pass. The catalog has to describe that burst, not the + /// ffmpeg packs several ADTS frames into one PES, and the importer publishes them as one group + /// in a single synchronous pass. The catalog has to describe that burst, not the /// 21 ms frame the estimator sees between writes, and a later bitrate refinement must not walk /// it back to one frame. #[test] @@ -6679,9 +6762,8 @@ pub(super) mod test { ); } - // #3533: a content join landed a PES PTS a millisecond below the last frame published. Each - // audio frame is its own group, so that group starts before the previous one: a restart, - // which ends the import rather than shifting it forward. + // #3533: a content join landed a PES PTS a millisecond below the last frame published. Audio + // never reorders, so that is a restart, which ends the import rather than shifting it forward. #[tokio::test(start_paused = true)] async fn legacy_pes_below_the_live_edge_is_refused() { const MP2_PID: u16 = 0x0061; @@ -9917,6 +9999,110 @@ pub(super) mod test { import_refused(&mux.out); } + /// A join a millisecond below the start of the last frame published is a restart, as it was + /// when each frame was its own group, even though it lands inside the last PES's group. + #[tokio::test(start_paused = true)] + async fn aac_join_inside_the_last_pes_is_refused() { + const AAC_PID: u16 = 0x0060; + // 1024 samples at 48 kHz, in 90 kHz ticks. + const AAC_FRAME: u64 = 1024 * 90_000 / 48_000; + let mut mux = Mux { + out: synth_pmt(&[(StreamType::AdtsAac, AAC_PID)], false), + ..Default::default() + }; + for (pts, fill) in [(90_000, 0xA0), (90_000 + 2 * AAC_FRAME - 90, 0xB0)] { + let mut payload = Vec::new(); + for i in 0..3 { + payload.extend_from_slice(&adts_frame(17, fill | i)); + } + let cc = mux.cc(AAC_PID); + mux.out.extend_from_slice(&audio_pes_packet(AAC_PID, cc, pts, &payload)); + } + import_refused(&mux.out); + } + + /// Each PES is one group in presentation order, so the transport, which sends a track's + /// newest group first, cannot deliver a PES's frames newest first. + #[tokio::test(start_paused = true)] + async fn aac_pes_is_one_group() { + const AAC_PID: u16 = 0x0060; + // 1024 samples at 48 kHz, in 90 kHz ticks. + const AAC_FRAME: u64 = 1024 * 90_000 / 48_000; + let mut mux = Mux { + out: synth_pmt(&[(StreamType::AdtsAac, AAC_PID)], false), + ..Default::default() + }; + for pes in 0..2u8 { + let mut payload = Vec::new(); + for i in 0..3 { + payload.extend_from_slice(&adts_frame(17, 0xA0 | (pes << 4) | i)); + } + let cc = mux.cc(AAC_PID); + let pts = 90_000 + u64::from(pes) * 3 * AAC_FRAME; + mux.out.extend_from_slice(&audio_pes_packet(AAC_PID, cc, pts, &payload)); + } + let (consumer, catalog, _import) = import_all(&mux.out).unwrap(); + + let name = catalog.snapshot().audio.renditions.keys().next().unwrap().clone(); + let subscription = moq_net::track::Subscription::default() + .with_delay(moq_net::track::Delay::default().with_max(Duration::from_secs(3600))); + let mut track = consumer.track(&name).unwrap().subscribe(subscription).await.unwrap(); + let mut groups = Vec::new(); + while let Ok(Ok(Some(mut group))) = kio::clock::timeout(Duration::from_millis(50), track.recv_group()).await { + let mut fills = Vec::new(); + while let Some(mut frame) = group.next_frame().await.unwrap() { + let frame = hang::container::Frame::decode(frame.read_all().await.unwrap()).unwrap(); + fills.extend(frame.payload.first().copied()); + } + groups.push(fills); + } + assert_eq!(groups, [vec![0xA0, 0xA1, 0xA2], vec![0xB0, 0xB1, 0xB2]]); + } + + /// A frame without a PTS is stamped by the producer, which checks only where groups start, so + /// each keeps a group of its own, unlike the PES that carries a PTS after them. + #[tokio::test(start_paused = true)] + async fn aac_untimed_frames_keep_a_group_each() { + const AAC_PID: u16 = 0x0060; + let mut mux = Mux { + out: synth_pmt(&[(StreamType::AdtsAac, AAC_PID)], false), + ..Default::default() + }; + let mut payload = Vec::new(); + for i in 0..3 { + payload.extend_from_slice(&adts_frame(17, 0xA0 | i)); + } + let cc = mux.cc(AAC_PID); + // The same PES with its PTS flag cleared: header length 0, five fewer bytes of PES. + let mut untimed = audio_pes_packet(AAC_PID, cc, 0, &payload); + let pes = untimed.iter().position(|&b| b == 0xc0).unwrap() - 3; + let len = u16::from_be_bytes([untimed[pes + 4], untimed[pes + 5]]) - 5; + untimed[pes + 4..pes + 6].copy_from_slice(&len.to_be_bytes()); + untimed[pes + 7] = 0x00; + untimed[pes + 8] = 0x00; + untimed.drain(pes + 9..pes + 14); + // Refill the adaptation field so the packet stays 188 bytes. + untimed[4] += 5; + untimed.splice(6..6, std::iter::repeat_n(0xff, 5)); + mux.out.extend_from_slice(&untimed); + let (consumer, catalog, _import) = import_all(&mux.out).unwrap(); + + let name = catalog.snapshot().audio.renditions.keys().next().unwrap().clone(); + let subscription = moq_net::track::Subscription::default() + .with_delay(moq_net::track::Delay::default().with_max(Duration::from_secs(3600))); + let mut track = consumer.track(&name).unwrap().subscribe(subscription).await.unwrap(); + let mut groups = Vec::new(); + while let Ok(Ok(Some(mut group))) = kio::clock::timeout(Duration::from_millis(50), track.recv_group()).await { + let mut fills = Vec::new(); + while let Some(mut frame) = group.next_frame().await.unwrap() { + let frame = hang::container::Frame::decode(frame.read_all().await.unwrap()).unwrap(); + fills.extend(frame.payload.first().copied()); + } + groups.push(fills); + } + assert_eq!(groups, [vec![0xA0], vec![0xA1], vec![0xA2]]); + } + /// PCR placement advances independently of the video's B-frame presentation reorder. #[tokio::test(start_paused = true)] async fn section_after_a_b_frame_publishes_forward() { diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 84571e61922..de33803e74a 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -180,10 +180,10 @@ The shaper's profiles are synthetic. [`traces/`](traces) holds real arrivals, ea | Trace | Recorded from | | --- | --- | -| `relay-bbb` | the public relay's `bbb.hang`: one AAC frame per group, flushed in bursts of about seven, the flush-span shape #3477 measured | -| `local-aac` | this harness's AAC publisher through a local relay: the shallow control | +| `relay-bbb` | the public relay's `bbb.hang`: one AAC frame per group, flushed in bursts of about seven, the flush-span shape #3477 measured, from before `moq import ts` grouped each PES | +| `local-aac` | this harness's AAC publisher through a local relay, from before it grouped each PES: the shallow control | | `relay-mic` | `` through the public relay: a real microphone with `--mic-broadcast` (the checked-in one is Chrome on a MacBook Air, its built-in microphone), otherwise headless Chromium's fake capture device | -| `bursty-aac` | a nightly `bursty` AAC row's capture: each flush delivered newest first, which kept the player re-stalling until the estimator measured reordered frames | +| `bursty-aac` | a nightly `bursty` AAC row's capture, from when the publisher sent each frame as its own group: each flush delivered newest first, which kept the player re-stalling until the estimator measured reordered frames | | `relay-mic-rtt40` | a page playing a real microphone (Chrome on a MacBook Air) through the public relay with 40 ms of added round trip: the steady-state re-stalls time stretch must remove | A trace is a [`Trace`](clients/js/src/schema.ts): the catalog's audio rendition, the smallest diff --git a/test/audio-quality/budgets.json b/test/audio-quality/budgets.json index 61dbf5a7595..e419b11f5e2 100644 --- a/test/audio-quality/budgets.json +++ b/test/audio-quality/budgets.json @@ -1,5 +1,5 @@ { - "note": "The browser rows are measured on the nightly runner (ubuntu-24.04-arm, headless Chromium, seed 7, 60 s a row), from the audio-quality-run artifacts of seven scheduled Nightly runs on main each. The AAC rows come from the seven after the estimator began measuring a reordered frame against the frames before it, 2026-10-14 to 2026-10-20 (38564590314, 38733214352, 38897397428, 39027965381, 39124936449, 39266265648, 39421743227); the Opus playback-loss ceilings come from the seven before it, 2026-10-07 to 2026-10-13 (37633358782, 37790146216, 37940662573, 38054566400, 38143901976, 38276133321, 38415278161). The Opus target ceilings use 2026-10-17 to 2026-10-23 (39027965381, 39124936449, 39266265648, 39421743227, 39600361153, 39779729461, 39951870847), after the reordered-frame correction; the wide target reached 440 ms, which replaying its captured arrivals reproduces. Each ceiling is the worst of its seven nights times 1.25, plus one runner hiccup, rounded up to two significant figures. The hiccup is sized from the single gaps the 2026-10-08 night put on otherwise clean rows, the fixed-250 control among them: 250 underruns/min, 700 gap ms/min, 3.3 skips/min, 280 discarded ms/min, 0.01 stalled, 0.015 silence, and one 20 ms bucket of target. Budgeted this way, every night passes when it is held out of its row's measurement except one, and that row's own ceiling includes it: 0.75 s of main-thread long tasks on the 2026-10-17 AAC step row, which stalled it and lifted its target to 803 ms. These ceilings record today's player as the runner hears it, not target audio quality. The AAC bursty rows stall for at most 0.041 of the window, against 0.018 for Opus bursty on the same nights, but still discard up to 870 ms a minute against its 210. To re-measure, download the same artifact from the latest scheduled nightlies (`gh run download -n audio-quality-run`), which keep every run, and apply the same rule. The replay rows are deterministic, so their ceilings are exactly what they measured, through the end of each recording, with no headroom: any regression fails them, and an improvement should be tightened in. They were last re-measured on 2026-10-22 using version 3 traces with encoded frame durations: AAC-LC uses its fixed 1024 samples, and relay-mic retains Opus packet TOC durations. relay-mic was re-recorded on 2026-11-03 from a real microphone, Chrome 156 on a MacBook Air publishing its built-in microphone through the public relay, recorded by headless Chromium on a Linux host (35 s, minimum RTT 2 ms), which raised its target ceiling from 60 to 80 ms and its silence from 0 to 0.001 as measured. The recording path itself has no added RTT. relay-mic-rtt40 is the 2026-11-03 Chrome row at 40 ms of added RTT on that laptop (180 s, minimum RTT 47 ms), kept as the baseline time stretch must bring to zero. player_underruns_per_min, the player's own counter that its stats panel shows, is budgeted on the replay rows only, exactly as measured on 2026-11-03; it counts the run-dry re-stalls the tap files as stalled time. The local-aac replay's stalled ceiling rises from 0.008 to 0.017 and silence from 0.015 to 0.020 because replay no longer stretches each frame to nearby timestamp gaps; the production player is unchanged. All other changed replay ceilings tighten to the exact measurement. That replay calibration leaves browser and native ceilings unchanged. bursty-aac is the 2026-10-18 AAC bursty isolated capture, recorded with the observer subscribed at the decoder's 2 s budget, so it holds every group the player saw. The native rows are provisional: measured on a loaded developer host (x86_64 Linux, the private PulseAudio null sink, seed 7, 60 s a row) on 2026-10-17, three runs of every row from the debug build nightly uses plus three more of the AAC rows from a release build, which played alike. Each ceiling is the worst clean run times 1.25, plus one native hiccup sized from the unimpaired Opus row's worst run (20 underruns/min, 50 gap ms/min, 2.2 skips/min, 110 discarded ms/min, 0.01 stalled, 0.005 silence, one 20 ms bucket of target), rounded up to two significant figures. On that host the null sink's clock ran about 150 ppm slow against the monotonic clock, read within 5 ms at p95 per sample, and its output latency sat between 14 and 28 ms. Five of the 36 AAC rows (none of the 18 Opus rows) voided on the clock: the device stopped pulling for about two seconds mid-row, sometimes after a backend error and a stream restart, which is a native playback defect rather than noise, and the ceilings exclude those runs. Re-measure from the nightly runner's artifacts once it has a week of native rows.", + "note": "The browser rows are measured on the nightly runner (ubuntu-24.04-arm, headless Chromium, seed 7, 60 s a row), from the audio-quality-run artifacts of the seven scheduled Nightly runs on main from 2026-10-29 to 2026-11-04 (40885347517, 41065371094, 41204380716, 41313466339, 41483654426, 41656613559, 41852517544), the first after a fresh subscriber stopped receiving the relay's cached history. The AAC rows predate the TS importer publishing each PES as one group. Each ceiling is the worst of its seven nights times 1.25, plus one runner hiccup, rounded up to two significant figures. The hiccup is sized from the single gaps the 2026-10-08 night put on otherwise clean rows, the fixed-250 control among them: 250 underruns/min, 700 gap ms/min, 3.3 skips/min, 280 discarded ms/min, 0.01 stalled, 0.015 silence, and one 20 ms bucket of target. Budgeted this way, every night passes when it is held out of its row's measurement except one, and that row's own ceiling includes it: the 2026-11-03 Opus bursty row raised its target to 300 ms 2.6 s in and held it for one 20 s half-life before settling at 180. Such startup holds, Opus near-zero's 100 ms on 2026-10-30 and 10-31 among them, are budgeted as measured. These ceilings record today's player as the runner hears it, not target audio quality. To re-measure, download the same artifact from the latest scheduled nightlies (`gh run download -n audio-quality-run`), which keep every run, and apply the same rule. The replay rows are deterministic, so their ceilings are exactly what they measured, through the end of each recording, with no headroom: any regression fails them, and an improvement should be tightened in. They were last re-measured on 2026-10-22 using version 3 traces with encoded frame durations: AAC-LC uses its fixed 1024 samples, and relay-mic retains Opus packet TOC durations. relay-mic was re-recorded on 2026-11-03 from a real microphone, Chrome 156 on a MacBook Air publishing its built-in microphone through the public relay, recorded by headless Chromium on a Linux host (35 s, minimum RTT 2 ms), which raised its target ceiling from 60 to 80 ms and its silence from 0 to 0.001 as measured. The recording path itself has no added RTT. relay-mic-rtt40 is the 2026-11-03 Chrome row at 40 ms of added RTT on that laptop (180 s, minimum RTT 47 ms), kept as the baseline time stretch must bring to zero. player_underruns_per_min, the player's own counter that its stats panel shows, is budgeted on the replay rows only, exactly as measured on 2026-11-03; it counts the run-dry re-stalls the tap files as stalled time. The local-aac replay's stalled ceiling rises from 0.008 to 0.017 and silence from 0.015 to 0.020 because replay no longer stretches each frame to nearby timestamp gaps; the production player is unchanged. All other changed replay ceilings tighten to the exact measurement. That replay calibration leaves browser and native ceilings unchanged. bursty-aac is the 2026-10-18 AAC bursty isolated capture, recorded with the observer subscribed at the decoder's 2 s budget, so it holds every group the player saw. The native rows are provisional: measured on a loaded developer host (x86_64 Linux, the private PulseAudio null sink, seed 7, 60 s a row) on 2026-10-17, three runs of every row from the debug build nightly uses plus three more of the AAC rows from a release build, which played alike. Each ceiling is the worst clean run times 1.25, plus one native hiccup sized from the unimpaired Opus row's worst run (20 underruns/min, 50 gap ms/min, 2.2 skips/min, 110 discarded ms/min, 0.01 stalled, 0.005 silence, one 20 ms bucket of target), rounded up to two significant figures. On that host the null sink's clock ran about 150 ppm slow against the monotonic clock, read within 5 ms at p95 per sample, and its output latency sat between 14 and 28 ms. Five of the 36 AAC rows (none of the 18 Opus rows) voided on the clock: the device stopped pulling for about two seconds mid-row, sometimes after a backend error and a stream restart, which is a native playback defect rather than noise, and the ceilings exclude those runs. Re-measure from the nightly runner's artifacts once it has a week of native rows.", "rows": [ { "runtime": "chromium", @@ -7,12 +7,12 @@ "rate": 44100, "profile": "bursty", "underruns_per_min": 260, - "underrun_ms_per_min": 720, - "skip_aheads_per_min": 13, - "discarded_ms_per_min": 1200, - "stalled_share": 0.062, - "silence_share": 0.059, - "target_ms_p95": 650 + "underrun_ms_per_min": 710, + "skip_aheads_per_min": 6.1, + "discarded_ms_per_min": 750, + "stalled_share": 0.022, + "silence_share": 0.03, + "target_ms_p95": 450 }, { "runtime": "chromium", @@ -24,7 +24,7 @@ "skip_aheads_per_min": 3.3, "discarded_ms_per_min": 280, "stalled_share": 0.01, - "silence_share": 0.017, + "silence_share": 0.015, "target_ms_p95": 340 }, { @@ -34,11 +34,11 @@ "profile": "mild", "underruns_per_min": 250, "underrun_ms_per_min": 700, - "skip_aheads_per_min": 4.7, - "discarded_ms_per_min": 340, - "stalled_share": 0.022, - "silence_share": 0.022, - "target_ms_p95": 350 + "skip_aheads_per_min": 3.3, + "discarded_ms_per_min": 280, + "stalled_share": 0.01, + "silence_share": 0.015, + "target_ms_p95": 290 }, { "runtime": "chromium", @@ -58,13 +58,13 @@ "codec": "aac", "rate": 44100, "profile": "step", - "underruns_per_min": 620, - "underrun_ms_per_min": 1800, - "skip_aheads_per_min": 7.5, - "discarded_ms_per_min": 950, + "underruns_per_min": 260, + "underrun_ms_per_min": 710, + "skip_aheads_per_min": 3.3, + "discarded_ms_per_min": 280, "stalled_share": 0.022, - "silence_share": 0.04, - "target_ms_p95": 730 + "silence_share": 0.023, + "target_ms_p95": 300 }, { "runtime": "chromium", @@ -73,36 +73,36 @@ "profile": "wide", "underruns_per_min": 260, "underrun_ms_per_min": 710, - "skip_aheads_per_min": 7.5, - "discarded_ms_per_min": 460, + "skip_aheads_per_min": 8.8, + "discarded_ms_per_min": 540, "stalled_share": 0.033, - "silence_share": 0.035, - "target_ms_p95": 450 + "silence_share": 0.034, + "target_ms_p95": 430 }, { "runtime": "chromium", "codec": "opus", "rate": 48000, "profile": "bursty", - "underruns_per_min": 260, - "underrun_ms_per_min": 710, - "skip_aheads_per_min": 8.8, - "discarded_ms_per_min": 770, - "stalled_share": 0.044, - "silence_share": 0.048, - "target_ms_p95": 370 + "underruns_per_min": 250, + "underrun_ms_per_min": 700, + "skip_aheads_per_min": 4.7, + "discarded_ms_per_min": 430, + "stalled_share": 0.01, + "silence_share": 0.015, + "target_ms_p95": 400 }, { "runtime": "chromium", "codec": "opus", "rate": 48000, "profile": "fixed-250", - "underruns_per_min": 470, - "underrun_ms_per_min": 1300, + "underruns_per_min": 250, + "underrun_ms_per_min": 700, "skip_aheads_per_min": 3.3, "discarded_ms_per_min": 280, "stalled_share": 0.01, - "silence_share": 0.025, + "silence_share": 0.015, "target_ms_p95": 340 }, { @@ -110,39 +110,39 @@ "codec": "opus", "rate": 48000, "profile": "mild", - "underruns_per_min": 330, - "underrun_ms_per_min": 900, - "skip_aheads_per_min": 3.3, - "discarded_ms_per_min": 280, + "underruns_per_min": 260, + "underrun_ms_per_min": 710, + "skip_aheads_per_min": 4.7, + "discarded_ms_per_min": 360, "stalled_share": 0.01, - "silence_share": 0.019, - "target_ms_p95": 120 + "silence_share": 0.017, + "target_ms_p95": 200 }, { "runtime": "chromium", "codec": "opus", "rate": 48000, "profile": "near-zero", - "underruns_per_min": 270, - "underrun_ms_per_min": 730, + "underruns_per_min": 250, + "underrun_ms_per_min": 700, "skip_aheads_per_min": 3.3, "discarded_ms_per_min": 280, "stalled_share": 0.01, - "silence_share": 0.018, - "target_ms_p95": 95 + "silence_share": 0.015, + "target_ms_p95": 150 }, { "runtime": "chromium", "codec": "opus", "rate": 48000, "profile": "step", - "underruns_per_min": 260, - "underrun_ms_per_min": 710, - "skip_aheads_per_min": 4.7, - "discarded_ms_per_min": 360, + "underruns_per_min": 250, + "underrun_ms_per_min": 700, + "skip_aheads_per_min": 6.1, + "discarded_ms_per_min": 430, "stalled_share": 0.022, "silence_share": 0.019, - "target_ms_p95": 150 + "target_ms_p95": 200 }, { "runtime": "chromium", @@ -151,11 +151,11 @@ "profile": "wide", "underruns_per_min": 260, "underrun_ms_per_min": 710, - "skip_aheads_per_min": 6.1, - "discarded_ms_per_min": 420, - "stalled_share": 0.028, - "silence_share": 0.03, - "target_ms_p95": 570 + "skip_aheads_per_min": 7.5, + "discarded_ms_per_min": 630, + "stalled_share": 0.01, + "silence_share": 0.018, + "target_ms_p95": 620 }, { "runtime": "native",