From 71fe8e2e12a8d19bce874b39f1690e432573cc49 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 13:13:20 -0700 Subject: [PATCH 1/3] chore(quest): plan the native playout clock `moq play` has no playout offset anywhere in its path, and two clocks that disagree. With audio the clock is re-derived per frame from `sink.buffered()`, so the offset is whatever the speaker holds under a hardcoded 1s cap. Video-only, `video_clock` is anchored once on the first frame and never re-anchored, so a late first frame appears to leave playback behind live for the session. The plan extends `moq_mux::pace::Pacer` rather than adding a second primitive: its `anchor` is already `js/watch/src/sync.ts`'s `reference`, it re-anchors forward only, and `with_lead` is nearly moq-watch's `buffer`. It needs a delay offset and the crate's usual `poll_*`/`async` pair, and a rename to `moq_mux::Clock` now that it serves playout as well as export pacing. Also de-stales two quests that PR #3396 moved under: 2981 said `latency_max` where the field and flag are now `max_age`, and 2278 quoted the `Bound` / `Latency` types it replaced with `Delay` plus a separate `buffer`, including a naming note that break partly resolves. Co-Authored-By: Claude Opus 5 --- ...in-the-decode-or-playback-path-models-a.md | 4 +- quest/m1/README.md | 1 + quest/m1/playout-clock.md | 72 +++++++++++++++++++ ...l-clock-latency-target-for-synchronized.md | 14 ++-- 4 files changed, 84 insertions(+), 7 deletions(-) create mode 100644 quest/m1/playout-clock.md diff --git a/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md b/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md index 9b00ca786c2..4e26143d61d 100644 --- a/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md +++ b/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md @@ -9,7 +9,7 @@ timestamps say where its samples actually belong. ## Plan -Gaps are routine: `latency_max` skipping a stalled group, `moq play` dropping +Gaps are routine: `max_age` skipping a stalled group, `moq play` dropping an undecodable packet, an ingest that resyncs. Below the catalog they are all treated as contiguous audio. @@ -44,7 +44,7 @@ reads as discontinuous under a strict rule. belong before the hole) or drop them, reset it, and stamp the next output from the new packet rather than by rewinding. - `play_audio` writes silence for the missing duration, which also covers - `latency_max` skips. + `max_age` skips. - Tests: frames at 0 and 2048 samples across a gap produce a hole; a 2.5 ms Opus packet lost after a 20 ms one is a gap; millisecond-stamped 1024-sample AAC packets detect no gap; the encoder's first PTS matches its first input. diff --git a/quest/m1/README.md b/quest/m1/README.md index 0e51adba618..5d93703ca44 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -30,6 +30,7 @@ with the current dev tree before starting. - [#3086](/quest/m1/3086-refactor-net-make-group-delivery-order-a-handle-and-move.md) - refactor(net): make group delivery order a handle, and move timestamp-based skipping into moq-net - [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - Retention should reclaim idle open groups now that expiry is timestamp-only - [Group overflow](/quest/m1/group-overflow-abort.md) - an oversized open group aborts for every reader instead of shedding its head +- [Playout clock](/quest/m1/playout-clock.md) - moq play presents on a clock it controls, with a `--delay` offset and forward re-anchoring - [#2895](/quest/m1/2895-add-an-atomic-readiness-gate-for-origin-broadcasts.md) - Add an atomic readiness gate for Origin broadcasts - [#2985](/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md) - js/net: path-keyed publisher state goes stale when a broadcast is replaced - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - net: coalesce dynamic tracks and preserve sequences across replacements diff --git a/quest/m1/playout-clock.md b/quest/m1/playout-clock.md new file mode 100644 index 00000000000..b417b04119e --- /dev/null +++ b/quest/m1/playout-clock.md @@ -0,0 +1,72 @@ +# [M] play: a real playout clock, with a --delay offset + +## Goal + +`moq play` presents against a clock it controls. `--delay ` (default +`100ms`) is how far playback trails the live edge, and a frame that arrives +earlier re-anchors playback forward instead of leaving it wherever the first +frame landed. + +Today there is no offset anywhere in the path, and two clocks disagree about +what time it is. With audio, `rs/moq-cli/src/play.rs` re-derives `Clock { media: +end - sink.buffered(), wall: now }` on every audio frame, so the offset is +whatever the speaker happens to hold, bounded by a hardcoded `AUDIO_BUFFER_MAX` +of one second. Video-only, `video_clock` is anchored once on the first frame and +never re-anchored (its sole write is gated on `is_none()`), so a first frame that +arrives late appears to leave playback behind live for the rest of the session. +`js/watch/src/sync.ts` handles exactly this by lowering its reference whenever a +frame arrives earlier. Reproduce that case before treating it as a defect. + +Not in scope: the `"auto"` (RTT-derived) and `"instant"` modes of `moq-watch`'s +`Delay`, and any FFI surface. `libmoq` and `moq-ffi` hand decoded frames to a +caller callback and have no playout stage, so there is nothing there to delay +yet. + +## Plan + +Branch from dev: renaming a published `moq-mux` item is a semver break. + +### One clock, not two + +`moq_mux::pace::Pacer` is already most of `Sync`. Its `anchor: Option<(Instant, +u128)>` is `Sync`'s `reference`, it re-anchors forward only, and `with_lead` is +nearly `moq-watch`'s `buffer`. The only missing piece is a delay offset, so +extend the primitive rather than adding a parallel one beside it. + +- Rename `Pacer` to `Clock` (`moq_mux::Clock`), since it now serves playout as + well as export pacing. Update the three call sites, which all keep `delay = 0`: + `rs/moq-srt/src/server.rs` (two) and `rs/moq-cli/src/subscribe.rs`. +- Add the offset: `send_at = anchor + (ts - base) + delay`. +- Add the pair the rest of the crate uses: `poll_wait(&kio::Waiter, ts) -> + Poll<()>` with an `async fn wait(ts)` wrapping it, per the Async / poll + plumbing section of `rs/CLAUDE.md`. + +### play.rs + +- Delete the local `Clock` struct and `video_clock`, and route both audio and + video through the one `moq_mux::Clock`. The reference anchor becomes the single + authority; the speaker-derived clock goes away. +- Size the audio sink to `delay` rather than the hardcoded `AUDIO_BUFFER_MAX`, + mirroring `ringSamples(rate, delay)` in `js/watch/src/audio/latency.ts`. Keep a + floor the way `ringSamples` does: a zero-depth ring can never be read from. +- `--delay ` replaces `--max-age` on `play`, with no alias. The two are + one number here, since nothing older than `delay` is worth presenting, so + `delay` is what goes on the wire as the subscription's max age. `--max-age` + stays unchanged on `import`, the stdout containers, and `rtmp export`. + +### Verification + +- Unit tests on `Clock` with `tokio::time::pause()`: a late first frame followed + by an earlier one re-anchors forward; a frame that merely arrives late does + not; the delay is honored; `delay = 0` paces exactly as `Pacer` did, which is + what keeps the export path unchanged. +- Manually: `moq play` against the demo relay, with audio and video-only, + checking that a late start catches up to live rather than staying behind it. + +## Required + +- [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - settles `play_audio`'s gap and silence handling first, so the clock rework is not rebased onto it + +## Related + +- [#2278](/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md) - the same anchoring model in js/watch, gaining an absolute wall-clock target diff --git a/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md b/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md index ea6a9630f2d..f3b88550ac7 100644 --- a/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md +++ b/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md @@ -48,8 +48,8 @@ The codebase actively fights wall-clock interpretation of frame timestamps. `Tim **Latency is relative (buffer depth), never absolute.** `js/watch/src/sync.ts`: ```ts -export type Bound = "real-time" | Time.Milli; -export type Latency = Bound | { min?: Bound; max?: Bound }; +export type Delay = "instant" | "auto" | Time.Milli; +// plus a separate `buffer: Getter` input on Sync ``` `Sync.received(timestamp)` computes `ref = Time.Milli.now() - timestamp`, and `Time.Milli.now()` is `performance.now()` (`js/net/src/time.ts:68`) - monotonic since page load, explicitly "not wall-clock time". So `reference` is an arbitrary local-clock↔PTS offset anchored by whichever frame happened to arrive first. Two viewers who tune in 3 seconds apart are 3 seconds apart forever, and nothing in the model can tell. @@ -59,7 +59,7 @@ export type Latency = Bound | { min?: Bound; max?: Bound }; #### Proposed shape 1. **A JS `Timeline.Consumer`** in `js/hang` (needed by the DVR work too). -2. **A third `Bound` variant, or a new absolute mode**: something like `latency: { absolute: Time.Milli }` meaning "render pts P at wall time `wall + P + absolute`". `Sync` then anchors on `Timeline.wall` instead of first-frame arrival, and `now()` becomes a wall-clock computation rather than an offset. +2. **A third mode alongside `delay` and `buffer`**: something like `delay: { absolute: Time.Milli }` meaning "render pts P at wall time `wall + P + absolute`". `Sync` then anchors on `Timeline.wall` instead of first-frame arrival, and `now()` becomes a wall-clock computation rather than an offset. 3. **Client clock sync.** This is the real work. `Timeline.wall` is the *publisher's* wall clock; the viewer's `Date.now()` can be off by seconds. Options, roughly in order of cost: - trust `Date.now()` (fine for a watch party, useless for betting) - estimate offset from the session RTT (we already track min RTT via PROBE in `#runJitter` for exactly this kind of anchoring) @@ -71,13 +71,13 @@ export type Latency = Bound | { min?: Bound; max?: Bound }; #### Open questions -- Absolute latency and the existing `min`/`max` bounds interact awkwardly: an absolute target is a *point*, the bounds are a *range*. Does absolute override the range, or clamp within it? +- An absolute target is a *point* while `delay` and `buffer` are two spans either side of the live edge. Does absolute replace `delay`, or sit alongside it with `buffer` still capping the lookahead? - `Timeline.wall` is per-rendition. Are audio and video guaranteed consistent? A rendition switch must not re-anchor. - Does this need anything on the publisher, or is `set_wall` sufficient? Today nothing in `moq-video`/`js/publish` calls `set_wall` as far as I can tell - so step 0 might be "actually populate `wall`". #### Naming note -`Latency` is already doubly overloaded: the `Bound | {min,max}` type (`js/watch/src/sync.ts:16`) and `class Latency` (`js/hang/src/util/latency.ts:21`, jitter+buffer). Different packages, both exported. Worth resolving before adding a third meaning. +Mostly resolved: `js/watch`'s `Latency` / `Bound` types are gone, replaced by `Delay` plus a separate `buffer`, leaving `class Latency` (`js/hang/src/util/latency.ts`, jitter+buffer) as the only remaining use. Pick a name for the absolute mode that does not reintroduce the collision. #### Branch @@ -90,3 +90,7 @@ export type Latency = Bound | { min?: Bound; max?: Bound }; ## Closes - [#2278](https://github.com/moq-dev/moq/issues/2278) - close this issue when the quest finishes + +## Related + +- [Playout clock](/quest/m1/playout-clock.md) - ports the same anchoring model to Rust as `moq_mux::Clock` From b271c7f2b6bdb818075ef25d3aa4c35262a4f38a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 16:04:30 -0700 Subject: [PATCH 2/3] chore(quest): keep Pacer's name, correct the plan, and move it to m0 Codex found the plan wrong in four places, three of them load-bearing. `moq_mux::Clock` is already taken by the capture-side shared epoch, used across moq-audio's capture path and moq-cli publish. The rename cannot happen without renaming that too, and its only justification was the clarity that name would have bought, so `Pacer` keeps its name. That also keeps the work additive, which drops the moq-srt and moq-cli call-site churn, moves the branch to main, and moves the quest to m0 beside the defect it fixes. Three hazards the plan had glossed: - `pace` re-anchors when `send_at` leads `now` by more than `lead`, which defaults to zero, so a delay folded into `send_at` makes the first delayed frame overshoot and discard the offset. `hurry` returns `now` undelayed and drops it again on every later re-anchor. - `Sync.#setReference` wakes every parked wait; `Pacer` has no notification at all, so a wait would sleep to a stale deadline when the other stream moves the anchor. - Audio waiting for the delayed instant and then filling a sink that also holds a delay's worth trails video by roughly another delay. And one factual correction: `AUDIO_BUFFER_MAX` does not size the sink, it gates writes against `sink.buffered()`. The ring is sized by the private `LATENCY` and `CAPACITY` constants in `rs/moq-audio/src/playback/sink.rs`, so honoring a delay means configuring the sink rather than the throttle. Also adds the `doc/bin/cli.md` reconciliation the flag rename needs. Co-Authored-By: Claude Opus 5 --- quest/m0/README.md | 1 + quest/m0/playout-clock.md | 99 +++++++++++++++++++ quest/m1/README.md | 1 - quest/m1/playout-clock.md | 72 -------------- ...l-clock-latency-target-for-synchronized.md | 2 +- 5 files changed, 101 insertions(+), 74 deletions(-) create mode 100644 quest/m0/playout-clock.md delete mode 100644 quest/m1/playout-clock.md diff --git a/quest/m0/README.md b/quest/m0/README.md index d8952b30738..e68819938e5 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -19,6 +19,7 @@ regression test per Root Cause First. - [Rust log redaction](/quest/m0/rust-log-redaction.md) - moq-native logs relay URLs with their query and moq-rtmp logs stream keys - [#3139](/quest/m0/3139-moqsrc-a-rendition-nobody-answers-keeps-the-session-alive.md) - moqsrc: a rendition nobody answers keeps the session alive after the catalog closes - [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - moq-audio: a media gap is a hole in the output, not a splice +- [Playout clock](/quest/m0/playout-clock.md) - moq play presents on a clock it controls, with a `--delay` offset and forward re-anchoring - [#3080](/quest/m0/3080-fix-watch-audio-ring-truncate-can-race-the-worklet-reader.md) - watch: an audio ring truncate can race the worklet reader for one quantum - [#2833](/quest/m0/2833-moq-export-ts-a-rewound-timeline-stalls-the-si-table.md) - moq export ts: a rewound timeline stalls SI tables, PCR, and pacing until the media clock catches up - [#2806](/quest/m0/2806-js-net-the-draft-14-15-adapter-keeps-one-request-per.md) - js/net: a duplicate PUBLISH_NAMESPACE on draft-14/15 strands the first request diff --git a/quest/m0/playout-clock.md b/quest/m0/playout-clock.md new file mode 100644 index 00000000000..459d62a006c --- /dev/null +++ b/quest/m0/playout-clock.md @@ -0,0 +1,99 @@ +# [M] play: a real playout clock, with a --delay offset + +## Goal + +`moq play` presents against a clock it controls. `--delay ` (default +`100ms`) is how far playback trails the live edge, and a frame that arrives +earlier re-anchors playback forward instead of leaving it wherever the first +frame landed. + +Today there is no offset anywhere in the path, and two clocks disagree about +what time it is. With audio, `rs/moq-cli/src/play.rs` re-derives `Clock { media: +end - sink.buffered(), wall: now }` on every audio frame, so the offset is +whatever the speaker happens to hold, bounded by a hardcoded `AUDIO_BUFFER_MAX` +of one second. Video-only, `video_clock` is anchored once on the first frame and +never re-anchored (its sole write is gated on `is_none()`), so a first frame that +arrives late appears to leave playback behind live for the rest of the session. +`js/watch/src/sync.ts` handles exactly this by lowering its reference whenever a +frame arrives earlier. Reproduce that case before treating it as a defect. + +Not in scope: the `"auto"` (RTT-derived) and `"instant"` modes of `moq-watch`'s +`Delay`, and any FFI surface. `libmoq` and `moq-ffi` hand decoded frames to a +caller callback and have no playout stage, so there is nothing there to delay +yet. + +## Plan + +Branch from main: nothing here breaks a published API. + +### One primitive, not two + +`moq_mux::pace::Pacer` is already most of `Sync`. Its `anchor: Option<(Instant, +u128)>` is `Sync`'s `reference`, it re-anchors forward only, and `with_lead` is +nearly `moq-watch`'s `buffer`. The only missing piece is a delay offset, so +extend the primitive rather than adding a parallel one beside it. + +It keeps the name `Pacer`. `moq_mux::Clock` is taken by the capture-side shared +epoch (`rs/moq-mux/src/clock.rs`, used across `moq-audio`'s capture path and +`moq-cli publish`), which is a different concept, and the rename's only +justification was the clarity that name would have bought. Staying put also keeps +this additive, so `rs/moq-srt/src/server.rs` and `rs/moq-cli/src/subscribe.rs` +need no change at all. + +Three things the offset has to get right, none of which fall out of adding it to +`send_at`: + +- **The lead comparison must exclude the delay.** `pace` re-anchors when + `send_at` leads `now` by more than `lead`, and `lead` defaults to zero, so a + delay folded into `send_at` makes the very first delayed frame overshoot and + re-anchor, discarding the offset. `hurry` has the same problem: it returns + `now`, undelayed, so every later re-anchor drops the delay again. Compare and + re-anchor on the undelayed instant and apply the delay last. +- **A moved anchor has to wake pending waits.** With audio and video sharing one + `Pacer`, a task can already be parked in `wait(ts)` when an earlier frame on + the other stream moves the anchor. `Sync.#setReference` resolves `#update` to + wake every parked wait so it recomputes; `Pacer` has no notification at all. + The `poll_wait(&kio::Waiter, ts)` pair is where that lives. +- **Audio must not take the delay twice.** If audio waits for the delayed + instant and then hands samples to a sink that also holds a delay's worth, it + trails video by roughly another delay. Either feed the ring ahead of the + deadline and let it only update the shared anchor, or subtract the sink depth + from the audio write deadline. + +### play.rs + +- Delete the local `Clock` struct and `video_clock`, and route both audio and + video through the one `Pacer`. The reference anchor becomes the single + authority; the speaker-derived clock goes away. +- Honoring a delay in audio means configuring the sink, not the throttle. + `AUDIO_BUFFER_MAX` in `play.rs` only gates writes against `sink.buffered()`; + the ring itself is sized by the private `LATENCY` (50ms) and `CAPACITY` (3s) + constants in `rs/moq-audio/src/playback/sink.rs`, so one of them has to become + configurable. Keep a floor the way `ringSamples` does in + `js/watch/src/audio/latency.ts`: a zero-depth ring can never be read from. +- `--delay ` replaces `--max-age` on `play`, with no alias. The two are + one number here, since nothing older than `delay` is worth presenting, so + `delay` is what goes on the wire as the subscription's max age. `--max-age` + stays unchanged on `import`, the stdout containers, and `rtmp export`. +- Reconcile `doc/bin/cli.md`, which documents `play --max-age` and carries an + example invocation using it, plus any other `moq play` example in the tree. + +### Verification + +- Unit tests on `Pacer` with `tokio::time::pause()`: a late first frame followed + by an earlier one re-anchors forward; a frame that merely arrives late does + not; the delay survives a re-anchor through both `pace` and `hurry`; a wait + parked on one stream shortens when another stream moves the anchor; and + `delay = 0` paces exactly as today, which is what keeps the export path + unchanged. +- Manually: `moq play` against the demo relay, with audio and video-only, + checking that a late start catches up to live rather than staying behind it, + and that audio and video stay aligned at a non-zero delay. + +## Required + +- [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - settles `play_audio`'s gap and silence handling first, so the clock rework is not rebased onto it + +## Related + +- [#2278](/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md) - the same anchoring model in js/watch, gaining an absolute wall-clock target diff --git a/quest/m1/README.md b/quest/m1/README.md index 5d93703ca44..0e51adba618 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -30,7 +30,6 @@ with the current dev tree before starting. - [#3086](/quest/m1/3086-refactor-net-make-group-delivery-order-a-handle-and-move.md) - refactor(net): make group delivery order a handle, and move timestamp-based skipping into moq-net - [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - Retention should reclaim idle open groups now that expiry is timestamp-only - [Group overflow](/quest/m1/group-overflow-abort.md) - an oversized open group aborts for every reader instead of shedding its head -- [Playout clock](/quest/m1/playout-clock.md) - moq play presents on a clock it controls, with a `--delay` offset and forward re-anchoring - [#2895](/quest/m1/2895-add-an-atomic-readiness-gate-for-origin-broadcasts.md) - Add an atomic readiness gate for Origin broadcasts - [#2985](/quest/m1/2985-js-net-path-keyed-publisher-state-goes-stale-when-a.md) - js/net: path-keyed publisher state goes stale when a broadcast is replaced - [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - net: coalesce dynamic tracks and preserve sequences across replacements diff --git a/quest/m1/playout-clock.md b/quest/m1/playout-clock.md deleted file mode 100644 index b417b04119e..00000000000 --- a/quest/m1/playout-clock.md +++ /dev/null @@ -1,72 +0,0 @@ -# [M] play: a real playout clock, with a --delay offset - -## Goal - -`moq play` presents against a clock it controls. `--delay ` (default -`100ms`) is how far playback trails the live edge, and a frame that arrives -earlier re-anchors playback forward instead of leaving it wherever the first -frame landed. - -Today there is no offset anywhere in the path, and two clocks disagree about -what time it is. With audio, `rs/moq-cli/src/play.rs` re-derives `Clock { media: -end - sink.buffered(), wall: now }` on every audio frame, so the offset is -whatever the speaker happens to hold, bounded by a hardcoded `AUDIO_BUFFER_MAX` -of one second. Video-only, `video_clock` is anchored once on the first frame and -never re-anchored (its sole write is gated on `is_none()`), so a first frame that -arrives late appears to leave playback behind live for the rest of the session. -`js/watch/src/sync.ts` handles exactly this by lowering its reference whenever a -frame arrives earlier. Reproduce that case before treating it as a defect. - -Not in scope: the `"auto"` (RTT-derived) and `"instant"` modes of `moq-watch`'s -`Delay`, and any FFI surface. `libmoq` and `moq-ffi` hand decoded frames to a -caller callback and have no playout stage, so there is nothing there to delay -yet. - -## Plan - -Branch from dev: renaming a published `moq-mux` item is a semver break. - -### One clock, not two - -`moq_mux::pace::Pacer` is already most of `Sync`. Its `anchor: Option<(Instant, -u128)>` is `Sync`'s `reference`, it re-anchors forward only, and `with_lead` is -nearly `moq-watch`'s `buffer`. The only missing piece is a delay offset, so -extend the primitive rather than adding a parallel one beside it. - -- Rename `Pacer` to `Clock` (`moq_mux::Clock`), since it now serves playout as - well as export pacing. Update the three call sites, which all keep `delay = 0`: - `rs/moq-srt/src/server.rs` (two) and `rs/moq-cli/src/subscribe.rs`. -- Add the offset: `send_at = anchor + (ts - base) + delay`. -- Add the pair the rest of the crate uses: `poll_wait(&kio::Waiter, ts) -> - Poll<()>` with an `async fn wait(ts)` wrapping it, per the Async / poll - plumbing section of `rs/CLAUDE.md`. - -### play.rs - -- Delete the local `Clock` struct and `video_clock`, and route both audio and - video through the one `moq_mux::Clock`. The reference anchor becomes the single - authority; the speaker-derived clock goes away. -- Size the audio sink to `delay` rather than the hardcoded `AUDIO_BUFFER_MAX`, - mirroring `ringSamples(rate, delay)` in `js/watch/src/audio/latency.ts`. Keep a - floor the way `ringSamples` does: a zero-depth ring can never be read from. -- `--delay ` replaces `--max-age` on `play`, with no alias. The two are - one number here, since nothing older than `delay` is worth presenting, so - `delay` is what goes on the wire as the subscription's max age. `--max-age` - stays unchanged on `import`, the stdout containers, and `rtmp export`. - -### Verification - -- Unit tests on `Clock` with `tokio::time::pause()`: a late first frame followed - by an earlier one re-anchors forward; a frame that merely arrives late does - not; the delay is honored; `delay = 0` paces exactly as `Pacer` did, which is - what keeps the export path unchanged. -- Manually: `moq play` against the demo relay, with audio and video-only, - checking that a late start catches up to live rather than staying behind it. - -## Required - -- [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - settles `play_audio`'s gap and silence handling first, so the clock rework is not rebased onto it - -## Related - -- [#2278](/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md) - the same anchoring model in js/watch, gaining an absolute wall-clock target diff --git a/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md b/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md index f3b88550ac7..3bfd3331d91 100644 --- a/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md +++ b/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md @@ -93,4 +93,4 @@ Mostly resolved: `js/watch`'s `Latency` / `Bound` types are gone, replaced by `D ## Related -- [Playout clock](/quest/m1/playout-clock.md) - ports the same anchoring model to Rust as `moq_mux::Clock` +- [Playout clock](/quest/m0/playout-clock.md) - ports the same anchoring model to Rust as `moq_mux::Clock` From acdc608292cd0104c899babd962399bda3d64f16 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 16:15:19 -0700 Subject: [PATCH 3/3] chore(quest): keep the sleep in the caller, and record the A/V questions `kio` has no time module, so a `poll_wait` that had to fire at a deadline would park forever with nothing to arm it, and taking a `moq_net::Timers` the way `origin::Driver::run` does would drag a runtime into a type that has never needed one. `pace` keeps returning the instant and the caller keeps sleeping, which is the contract the export path already relies on; the `poll_*` pair reports only the anchor moving. Adds an Open questions section for the A/V policy Codex kept circling: which stream owns the anchor when both exist, what the speaker position contributes given every result is shifted by the delay, and whether a per-write bound is needed when play_audio can hand the sink a one-second chunk against a 100ms delay. Those want the code in front of them, and the answers change what the tests assert, so they belong in the quest rather than settled blind. Co-Authored-By: Claude Opus 5 --- quest/m0/playout-clock.md | 36 ++++++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/quest/m0/playout-clock.md b/quest/m0/playout-clock.md index 459d62a006c..79e0de5b9b9 100644 --- a/quest/m0/playout-clock.md +++ b/quest/m0/playout-clock.md @@ -50,10 +50,17 @@ Three things the offset has to get right, none of which fall out of adding it to `now`, undelayed, so every later re-anchor drops the delay again. Compare and re-anchor on the undelayed instant and apply the delay last. - **A moved anchor has to wake pending waits.** With audio and video sharing one - `Pacer`, a task can already be parked in `wait(ts)` when an earlier frame on - the other stream moves the anchor. `Sync.#setReference` resolves `#update` to - wake every parked wait so it recomputes; `Pacer` has no notification at all. - The `poll_wait(&kio::Waiter, ts)` pair is where that lives. + `Pacer`, a task can already be parked waiting on a frame when an earlier frame + on the other stream moves the anchor. `Sync.#setReference` resolves `#update` + to wake every parked wait so it recomputes; `Pacer` has no notification at all. + + This is a re-anchor notification, not a timed wait: `kio` has no time module + (`rs/CLAUDE.md`), so a `poll_*` that had to fire at a deadline would park + forever with nothing to arm it, and taking a `moq_net::Timers` the way + `origin::Driver::run` does would drag a runtime into a type that has never + needed one. Keep `pace` returning the instant and let the caller sleep, which + is the contract the export path already relies on, and add only the + `poll_*`/`kio::Waiter` pair that reports the anchor moving. - **Audio must not take the delay twice.** If audio waits for the delayed instant and then hands samples to a sink that also holds a delay's worth, it trails video by roughly another delay. Either feed the ring ahead of the @@ -78,6 +85,27 @@ Three things the offset has to get right, none of which fall out of adding it to - Reconcile `doc/bin/cli.md`, which documents `play --max-age` and carries an example invocation using it, plus any other `moq play` example in the tree. +### Open questions + +Left open deliberately: they are A/V policy that wants the code in front of it, +and the answer changes what the tests should assert. + +- **Which stream owns the anchor when both exist.** Video taking a tune-in burst + can `hurry` the shared anchor past what the speaker is still draining, and + because the anchor only moves forward, later audio cannot pull it back. Making + the speaker the sole re-anchor source while audio exists is one answer; + discarding audio to the same edge is another. +- **What the speaker contributes.** `last_timestamp - buffered()` is the sample + sounding now, so feeding it in as the anchor while every result is shifted by + `delay` would schedule video a full delay behind it. If the speaker position + stays in the calculation it has to enter at the live edge, not the playing + edge. +- **Whether a per-write bound is needed.** `play_audio` splits PCM into chunks of + up to a second and checks `sink.buffered()` only between whole writes, so a + first chunk can overshoot a 100ms delay outright. Sizing each part to the + remaining headroom is the obvious fix; whether the sink should grow an + operation for it is not. + ### Verification - Unit tests on `Pacer` with `tokio::time::pause()`: a late first frame followed