Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m1/audio-jitter-target/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ native playback against its budgets.

- [Watch](/quest/m1/audio-jitter-target/watch.md) - the browser plays the measured real-microphone rows with zero steady-state underruns by time-stretching the ring
- [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - `moq play` copies NetEq's accelerate and preemptive expand, sharing the browser's algorithm and vectors, so its native rows reach zero steady-state underruns
- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - post-cap nightlies confirm AAC bursty discards about as much audio as Opus bursty and the first row no longer stalls, and every browser row's budgets are re-measured from them
- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the AAC browser budgets are re-measured from nightlies after the TS importer groups each PES

## Closes

Expand Down
152 changes: 43 additions & 109 deletions quest/m1/audio-jitter-target/capped-budgets.md
Original file line number Diff line number Diff line change
@@ -1,118 +1,52 @@
# [S] Re-measure the browser budgets after the estimator capped one observation's weight, and group AAC bursts
# [XS] Re-measure the AAC browser budgets after the TS importer groups each PES

## Goal

Seven scheduled nightlies on `main` from 2026-10-29 on, after the estimator
began capping how much weight one observation takes, the player changes
that followed, and the relay history-fill fix, show the AAC `bursty` browser rows discarding
about as much audio a minute as the Opus `bursty` rows, and the Opus
`near-zero` row holding a 60 ms target with no underrun episodes. `test/audio-quality/budgets.json`
carries every browser row's ceilings re-measured from them. The AAC
publisher stops sending each seven-frame flush as seven groups newest
first, so AAC `wide` stops running dry.
`test/audio-quality/budgets.json` carries the six AAC browser rows' ceilings
re-measured, by the rule in its note, from seven scheduled nightlies on `main`
that all ran after the TS importer began publishing each PES as one group, and
the note's sentence that the AAC rows predate the grouping is dropped. Nearly
every AAC frame on those nights arrives in order.

## Waiting

- moq-dev/moq: seven scheduled Nightly runs on `main` from 2026-10-29 on, the
seventh due about 2026-11-04. Check with `gh run download <id> -n
audio-quality-run` on each; never dispatch the nightly by hand. The artifacts
expire after 7 days, so download each run's as it lands rather than waiting
for the seventh.
- moq-dev/moq: seven scheduled Nightly runs on `main` after
[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) merges. Find them
with `gh run list --workflow nightly.yml --event schedule` and download each
with `gh run download <id> -n audio-quality-run`; never dispatch the nightly by
hand. The artifacts expire after 7 days, so download each run's as it lands
rather than waiting for the seventh.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs): plan the AAC publisher fix here. On
[moq-dev/moq#7679](https://github.com/moq-dev/moq/pull/7679)'s runs AAC
`wide` ran dry 1 to 4 times a minute at a 323 ms target. The publisher sends
each seven-frame PES flush as seven one-frame groups, delivered newest first,
so the oldest frame waits behind every newer group (the cause predicted
below). Send a flush so its oldest frame goes first, for example as one
group or in ascending order, and re-measure the AAC rows after it. Options
shown:

- m0
- ✅ m1, in this quest
- m2

Decided 2026-10-30 (maintainer): the count restarts at the 2026-10-29
nightly, the first after
[moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168), because the
2026-10-25 to 10-28 nightlies measured the relay history-fill bug it fixed
(below). One has run (2026-10-29), with the seventh due about 2026-11-04.
This replaces the 2026-10-27 restart at the 2026-10-25 nightly, which
followed the post-cap estimator and player changes
([moq-dev/moq#6359](https://github.com/moq-dev/moq/pull/6359),
[moq-dev/moq#6525](https://github.com/moq-dev/moq/pull/6525),
[moq-dev/moq#6484](https://github.com/moq-dev/moq/pull/6484),
[moq-dev/moq#6609](https://github.com/moq-dev/moq/pull/6609)). No earlier
run counts toward the seven. Re-measure by the rule in the budgets note, and drop the note's sentence
that budgets the AAC bursty discards against Opus's as measured. Nightlies
from before the one-ring removal also carry `-isolated` rows; grade only the
`-plain` rows, which are the browser rows now.

Decided 2026-10-22 (maintainer): the Opus near-zero budgets quest folded in
here, so one pass over the same seven nightlies covers both. The cap
([#6164](https://github.com/moq-dev/moq/pull/6164)) merged 2026-10-22T00:10Z,
after the build flush, so post-cap nightlies are also post-flush.

The first post-cap scheduled run, [2026-10-22 nightly](https://github.com/moq-dev/moq/actions/runs/39779729461),
failed only the browser Opus `wide` target ceilings: `isolated` p95 was
380 ms against 270 ms, and `plain` was 400 ms against 300 ms. In the
`audio-quality-run` artifact, both `chromium-opus-48000-wide-*.ndjson`
series hold their startup target until about 21 s, then settle at 200 to
220 ms; both matching `.longtasks.json` arrays are empty. The isolated
row has zero underruns, stalls, and silence; plain has 2.2 underruns a
minute, 0 stalls, and 0 silence. It predates the restarted count, and is
not a reason to change the estimator or recalibrate early.

The near-zero stall was never the isolated ring. That row is simply the
matrix's first, and a cold `cargo build` left about 800 MB dirty when it
started. On the ARM runner each periodic writeback burst froze the page's main
thread for 100 to 320 ms until the backlog drained, and the harness now
flushes its build before the first row. If the first row still shows
long-task clusters, compare `<tag>.longtasks.json` against the runner's disk
activity before changing the player.

The cap changed every row's startup, not only AAC bursty's: the prior is
replaced in about five seconds instead of two, and a late interval in those
first seconds holds the target for up to one 20 s half-life instead of falling
back within seconds. Replays of local captures held the clean rows' targets
where they were, but one live Opus `near-zero` `plain` run held a startup
hiccup at 160 ms for 20 s, over that row's 95 ms ceiling. If a nightly breaches
a startup target ceiling that way, budget it as measured rather than reopening
the estimator.

On replays of the 2026-10-18 nightly capture and of two local captures, the
cap cut AAC bursty discards from 540 to 860 ms a minute to 0 to 170, and its
target p95 from 663 ms to 343 to 363. If the nightlies still show AAC bursty
discarding several times Opus's, the remaining source is the publisher: each
seven-frame PES flush is seven one-frame groups, delivered newest first, so the
oldest frame of a flush waits behind every newer group on a path that cannot
carry them together.

The 2026-10-25 to 10-28 nightlies
([40192013303](https://github.com/moq-dev/moq/actions/runs/40192013303),
[40332938121](https://github.com/moq-dev/moq/actions/runs/40332938121),
[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673),
[40698329026](https://github.com/moq-dev/moq/actions/runs/40698329026))
measured a relay bug rather than the player: 2 to 4.5 s into every row except
`fixed-250` and Opus `near-zero`, the subscriber received 15 to 25 s of the
publisher's cached history, newest first. On the `wide` rows the estimator
took that as a late interval and held a 2020 ms target for about 20 s,
then discarded about 1.8 to 1.9 s of audio a minute falling back, every night;
the 10-28 Opus `bursty` 540 ms target is the same arrival. No `wide` or
`bursty` row logged more than one long task, of at most 58 ms.
[moq-dev/moq#7168](https://github.com/moq-dev/moq/pull/7168) stopped a fresh
subscriber filling history below its start, merged 2026-10-29T00:52Z, and the
[2026-10-29 nightly](https://github.com/moq-dev/moq/actions/runs/40885347517)
is the first after it and the first that counts: the oldest frame any row
received is about 2.6 s behind live, every browser row is within budget, and
AAC `wide` discards 205.6 ms/min at a 323.2 ms target p95, Opus `wide` 243.4
at 460. On it AAC `bursty` discards 162.9 ms a minute against 0 for Opus
`bursty`, and Opus `near-zero` has no underrun episodes at a 60 ms target
p95. Across the four earlier nights Opus `near-zero` held 60 ms twice and
80 twice: the 120 ms prior decays to 60 only at 21.4 s, after the 5 s
warmup. Each night's browser metrics are kept
in [moq-dev/moq#7286](https://github.com/moq-dev/moq/pull/7286), since the
artifacts expire. The separate publisher fairness timeout was
excess per-object work, fixed by [moq-dev/moq#7133](https://github.com/moq-dev/moq/pull/7133).
Decided 2026-11-04 (maintainer, on
[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898)): land one group
per PES as is. The extra AAC `wide` re-stalls are for Watch's time stretch to
absorb, and this quest no longer expects AAC `wide` to stop running dry.
Options shown:

- ✅ Land as is
- Land only the budgets, holding the importer change until time stretch lands
- Keep the change and deepen AAC's target

[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) re-measured every
browser row from the 2026-10-29 to 2026-11-04 nightlies (its body keeps each
night's metrics) and changed the publisher: `moq import ts` used to send each
seven-frame AAC PES as seven one-frame groups, which the transport delivers
newest first, so 82 to 86% of AAC frames arrived behind a newer one. Each PES is
now one group, its frames in presentation order.

On those nights, before the change, AAC `bursty` discarded up to 368.3 ms a
minute against Opus `bursty`'s 117.4. Replaying their captures with each PES's
arrival slots handed to its frames oldest first, as one group, halved the AAC
`bursty` and `wide` discards (`bursty` summed over the seven nights 1156 to 584
ms a minute) and lowered both rows' target by 20 to 60 ms. That shallower target doubled AAC
`wide`'s player re-stalls (mean 3.0 to 6.6 a minute), the 0.95 quantile's trade
that [Watch](/quest/m1/audio-jitter-target/watch.md)'s time stretch absorbs.

AAC `bursty` still discards about three times what Opus `bursty` does (584
against 174 summed), and parity is not expected: the harness keeps ffmpeg's
default PES packing on purpose, so AAC frames reach the viewer a median 210 ms
behind their media time on `bursty` against Opus's 41. Grade the same
`chromium-aac-*` rows, and check the in-order claim with the share of each row's
`.arrivals.ndjson` frames that arrive behind a newer one.
4 changes: 2 additions & 2 deletions quest/m1/audio-jitter-target/native-stretch.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ loaded 24-core host (load 13 to 55, seed 7, 60 s) gave target p95s of
`main`, a second late interval at 7 s. Every run of this branch had zero
underruns; `main` had up to 14.1 a minute. The 2026-10-29 to 11-03 scheduled
nightlies on `main` read 160 to 280 ms. So the ceiling stays as it is,
as [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) directs
for a startup breach, until the seven-nightly calibration below.
as `test/audio-quality/budgets.json`'s note budgets startup holds as measured,
until the seven-nightly calibration below.

## Waiting

Expand Down
2 changes: 1 addition & 1 deletion quest/m1/audio-jitter-target/watch.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,6 @@ A/V clock work.

## Related

- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - confirms the build-flush fix on the nightlies, no gate on this proof
- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - re-measures the AAC browser budgets after the publisher change, no gate on this proof
- [Native stretch](/quest/m1/audio-jitter-target/native-stretch.md) - the native time-stretch sharing one NetEq algorithm and its test vectors with this one
- [Watch worker](/quest/m2/watch-worker.md) - moves the worklet this stretch changes
15 changes: 7 additions & 8 deletions quest/m1/av-clock.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,13 @@ Recommendations for the implementation:
catalog schema (`js/msf/src/catalog.ts`) accepts a negative `delay` and
folds it into absent; refuse it on decode instead.

Decided 2026-10-27 (maintainer): this waits only on
[capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md), not the
whole [audio jitter target](/quest/m1/audio-jitter-target/README.md) epic,
whose remaining manual browser proof does not gate reshaping the clock.

## Required

- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the estimator's browser budgets, re-measured, before this reshapes the clock around them
Decided 2026-10-27 (maintainer): this waits only on the estimator's browser
budgets being re-measured, not the whole
[audio jitter target](/quest/m1/audio-jitter-target/README.md) epic, whose
remaining manual browser proof does not gate reshaping the clock.
[moq-dev/moq#7898](https://github.com/moq-dev/moq/pull/7898) re-measured every
browser row. What [capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md)
still owes, the AAC rows after a publisher change, does not gate it either.

## Related

Expand Down
6 changes: 3 additions & 3 deletions rs/moq-mux/src/codec/legacy.rs
Original file line number Diff line number Diff line change
Expand Up @@ -115,9 +115,9 @@ pub(crate) struct Config {

/// Legacy audio importer.
///
/// Publishes each whole frame as one hang frame in its own group, so the relay
/// forwards it immediately. The audio is never decoded; the catalog carries the
/// codec, sample rate and channel count read from the frame header.
/// Publishes each whole frame as one hang frame, into the current group until the
/// caller cuts it. The audio is never decoded; the catalog carries the codec,
/// sample rate and channel count read from the frame header.
pub(crate) struct Import {
track: crate::container::Producer<crate::catalog::hang::Container, hang::catalog::AudioConfig>,
}
Expand Down
Loading
Loading