Repository navigation
quest(m0): a browser publisher stall ratchets viewer latency #6149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| # [M] A browser publisher's stall never ratchets its viewers' latency | ||
|
|
||
| ## Goal | ||
|
|
||
| A live camera and microphone publish from `moq.dev/publish` plays at the | ||
| viewer's Auto latency target for the life of the broadcast, the way | ||
| `bbb.hang` does: the catalog's `jitter` and `delay` settle after startup | ||
| instead of climbing, so a viewer's buffered media stops growing. Reported on | ||
| Brave on Fedora 44 (Wayland) and reproduced on macOS: the viewer's total | ||
| buffer climbed 8.8 s, 11 s, 12.7 s, then 42 s within minutes of going live. | ||
|
|
||
| The fix lands at whatever publisher-local stall feeds the estimate. The | ||
| catalog rule that `jitter` and `delay` never fall stays as specified. | ||
|
|
||
| ## Plan | ||
|
|
||
| Mechanism, from reading the code on 2026-10-20 (not yet reproduced): | ||
|
|
||
| - `Estimator` (`js/publish/src/jitter.ts`) measures each frame's flush | ||
| lateness (`now - timestamp`) and advertises its lifetime maximum spread as | ||
| `jitter` and its trail behind the broadcast's earliest rendition as | ||
| `delay`. `CatalogProducer` (`js/publish/src/catalog.ts`) refuses a decrease, | ||
| as the hang draft requires. One stall on the publisher is therefore | ||
| advertised for the rest of the broadcast. | ||
| - The watcher's Auto target adds both catalog terms with no bound | ||
| (`js/watch/src/video/decoder.ts`, `js/watch/src/audio/decoder.ts`), so the | ||
| playhead holds that much behind the live edge and media piles up ahead of | ||
| it. The reported numbers match exactly: the "+Xs" label is the total buffer | ||
| minus the timeline's 4 s width. | ||
| - The file source behind `bbb.hang` stamps both tracks from one clock and | ||
| never stalls, which is why it stays near 0.4 s. | ||
|
|
||
| Candidate triggers, to confirm by reproduction before fixing: | ||
|
|
||
| 1. Video encoder backlog. The encode loop (`js/publish/src/video/encoder.ts`) | ||
| never reads `encodeQueueSize`, so a software H.264/VP8 encoder slower than | ||
| real time queues without bound and lateness rises linearly. This fits the | ||
| growth rate best. | ||
| 2. Audio and video on different clocks. Audio timestamps count samples on the | ||
| AudioContext render clock (`js/publish/src/audio/capture-worklet.ts`); | ||
| video follows the camera clock re-based once on its first frame | ||
| (`js/publish/src/video/processor.ts`). Rate drift between them becomes | ||
| `delay`. | ||
| 3. A throttled or backgrounded publisher tab. | ||
|
|
||
| Reproduce first: publish from Brave or Chromium on `moq.dev/publish` (or a | ||
| local `just dev`), log `encodeQueueSize` and each flush's lateness on the | ||
| publisher, and record the catalog's per-rendition `jitter` and `delay` on a | ||
| viewer over time. Land a regression test with mocked time that reproduces the | ||
| trigger, such as a slow mock encoder. | ||
|
|
||
| Settled decisions: | ||
|
|
||
| - Fix the stall at its source and keep the never-fall rule. Letting the | ||
| estimates decay was rejected: it changes wire semantics in the hang draft, | ||
| Rust `moq-mux`, `js/publish`, and the player for a problem that starts on | ||
| the publisher. If the trigger turns out to be one the publisher cannot | ||
| prevent (a throttled tab, a GC pause), stop and propose a course change | ||
| rather than patching around it. | ||
| - No player-side bound on the advertised terms. The player keeps trusting | ||
| the catalog's `jitter` and `delay`. | ||
| - If the trigger is the encoder backlog, a real-time encoder drops input | ||
| frames instead of queueing; a pending keyframe request survives the drop. | ||
| - If the fix touches `Estimator` itself, mirror it in Rust's | ||
| `moq_mux::catalog::Estimator` (`rs/moq-mux/src/catalog/estimate.rs`). | ||
| - The fixed-latency half of the report is already fixed on `main` ([#4162](https://github.com/moq-dev/moq/pull/4162)); | ||
| [Release fixed delay](/quest/m0/release-fixed-delay.md) ports it to | ||
| `release`, which is what `moq.dev` runs. | ||
|
|
||
| ## Related | ||
|
|
||
| - [Release fixed delay](/quest/m0/release-fixed-delay.md) - the deployed player also adds the catalog terms to a fixed latency | ||
| - [Subscribe update churn](/quest/m1/subscribe-update-churn.md) - the alternating `subscribe update` flood in the same report | ||
| - [A/V clock](/quest/m1/av-clock.md) - the player-side clock; the publisher's two capture clocks are this quest's | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # [XS] A fixed latency on release ignores the catalog jitter and delay | ||
|
|
||
| ## Goal | ||
|
|
||
| On `release`, which `moq.dev` runs, a viewer who picks a fixed latency | ||
| (100 ms, 250 ms, ...) plays at that delay. Today `Sync.#runDelay` | ||
| (`js/watch/src/sync.ts` on `release`) adds every rendition's catalog | ||
| `delay + jitter` on top of the fixed value, so a publisher whose estimates | ||
| ratchet up drags a "100 ms" viewer tens of seconds behind; a user reported a | ||
| 42 s total buffer with 100 ms selected. | ||
|
|
||
| ## Plan | ||
|
|
||
| `main` already behaves this way since | ||
| [#4162](https://github.com/moq-dev/moq/pull/4162), but that change is a | ||
| breaking `@moq/watch` rework that won't be backported. Port only the behavior: | ||
| when the configured delay is a number, the output delay is that number; | ||
| "auto" and "instant" are unchanged. | ||
|
|
||
| - The PR targets `release` directly, with no `main` change, and keeps the | ||
| published API so it ships as a patch (see `CONTRIBUTING.md`). | ||
| - Add a regression test in `js/watch/src/sync.test.ts` on `release`: a | ||
| registered rendition jitter does not raise a fixed delay. | ||
| - The fix reaches `moq.dev` with the next release deploy. | ||
|
|
||
| ## Related | ||
|
|
||
| - [Publisher stall estimate](/quest/m0/publish-stall-estimate.md) - why the catalog terms grow in the first place |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # [S] A viewer's subscription updates stop chattering | ||
|
|
||
| ## Goal | ||
|
|
||
| A `js/watch` viewer sends a SUBSCRIBE_UPDATE only when its subscription | ||
| meaningfully changes, not several times a second per track. In a reported | ||
| `moq.dev/publish` session, the publisher logged an endless alternating | ||
| `subscribe update` for audio and video. | ||
|
|
||
| ## Plan | ||
|
|
||
| Cause, from reading the code on 2026-10-20: `js/watch/src/media.ts` pushes | ||
| `maxDelay` into `subscriber.update` on every change, and the subscriber loop | ||
| (`js/net/src/lite/subscriber.ts`, `#sameSubscription`) compares it exactly. | ||
| Video's `maxDelay` follows `sync.out.maxDelay`, which moves with the per-frame | ||
| arrival spread (20 ms steps, at most every 500 ms) and every catalog estimate | ||
| republish. Audio only joins once `maxDelay` passes `AUTO_MAX_DELAY`. In the | ||
| report, the ratcheting estimate fixed by [Publisher stall | ||
| estimate](/quest/m0/publish-stall-estimate.md) drove most of it. | ||
|
|
||
| Settled: measure first. Once that quest lands, count updates per track for a | ||
| live camera publish and `bbb.hang`, then pick a cut with the numbers, and | ||
| close this as unneeded if they are already quiet. Options weighed when | ||
| planning: | ||
|
|
||
| - Round the subscription's `maxDelay` up to the next power of two ms in | ||
| `js/watch`, so it changes only when the target crosses a doubling. | ||
| - Rate-limit sends to one per smoothed RTT per track in `js/net` and | ||
| `rs/moq-net`, latest value wins. | ||
|
|
||
| Coalescing to one update "in flight" was ruled out: moq-lite's | ||
| SUBSCRIBE_UPDATE has no acknowledgement, and the sender already sends only the | ||
| latest value whenever the stream applies backpressure. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Publisher stall estimate](/quest/m0/publish-stall-estimate.md) - the runaway estimate dominates the update rate until it is fixed |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When an Auto viewer opens a broadcast whose audio rendition advertises a very large
jitterordelay, this decision lets the valid U53 value flow throughjs/watch/src/audio/latency.ts:24-26intoringSamples, andSharedAudioBuffer.setLatencythen attempts to resize its shared ring to roughly twice that sample count (js/watch/src/audio/buffer.ts:220-233). A hostile publisher can therefore use a valid catalog to trigger a huge allocation, aRangeError, or an out-of-memory tab crash; retain adaptive behavior if desired, but clamp or reject values above a resource-safe ceiling before they size buffers.AGENTS.md reference: AGENTS.md:L18-L18
Useful? React with 👍 / 👎.