Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ timestamps say where its samples actually belong.

## Plan

Gaps are routine: `latency_max` skipping a stalled group, `moq play` dropping
Gaps are routine: `max_age` skipping a stalled group, `moq play` dropping
an undecodable packet, an ingest that resyncs. Below the catalog they are all
treated as contiguous audio.

Expand Down Expand Up @@ -44,7 +44,7 @@ reads as discontinuous under a strict rule.
belong before the hole) or drop them, reset it, and stamp the next output
from the new packet rather than by rewinding.
- `play_audio` writes silence for the missing duration, which also covers
`latency_max` skips.
`max_age` skips.
- Tests: frames at 0 and 2048 samples across a gap produce a hole; a 2.5 ms
Opus packet lost after a 20 ms one is a gap; millisecond-stamped 1024-sample
AAC packets detect no gap; the encoder's first PTS matches its first input.
Expand Down
1 change: 1 addition & 0 deletions quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ regression test per Root Cause First.
- [Rust log redaction](/quest/m0/rust-log-redaction.md) - moq-native logs relay URLs with their query and moq-rtmp logs stream keys
- [#3139](/quest/m0/3139-moqsrc-a-rendition-nobody-answers-keeps-the-session-alive.md) - moqsrc: a rendition nobody answers keeps the session alive after the catalog closes
- [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - moq-audio: a media gap is a hole in the output, not a splice
- [Playout clock](/quest/m0/playout-clock.md) - moq play presents on a clock it controls, with a `--delay` offset and forward re-anchoring
- [#3080](/quest/m0/3080-fix-watch-audio-ring-truncate-can-race-the-worklet-reader.md) - watch: an audio ring truncate can race the worklet reader for one quantum
- [#2833](/quest/m0/2833-moq-export-ts-a-rewound-timeline-stalls-the-si-table.md) - moq export ts: a rewound timeline stalls SI tables, PCR, and pacing until the media clock catches up
- [#2806](/quest/m0/2806-js-net-the-draft-14-15-adapter-keeps-one-request-per.md) - js/net: a duplicate PUBLISH_NAMESPACE on draft-14/15 strands the first request
Expand Down
127 changes: 127 additions & 0 deletions quest/m0/playout-clock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# [M] play: a real playout clock, with a --delay offset

## Goal

`moq play` presents against a clock it controls. `--delay <duration>` (default
`100ms`) is how far playback trails the live edge, and a frame that arrives
earlier re-anchors playback forward instead of leaving it wherever the first
frame landed.

Today there is no offset anywhere in the path, and two clocks disagree about
what time it is. With audio, `rs/moq-cli/src/play.rs` re-derives `Clock { media:
end - sink.buffered(), wall: now }` on every audio frame, so the offset is
whatever the speaker happens to hold, bounded by a hardcoded `AUDIO_BUFFER_MAX`
of one second. Video-only, `video_clock` is anchored once on the first frame and
never re-anchored (its sole write is gated on `is_none()`), so a first frame that
arrives late appears to leave playback behind live for the rest of the session.
`js/watch/src/sync.ts` handles exactly this by lowering its reference whenever a
frame arrives earlier. Reproduce that case before treating it as a defect.

Not in scope: the `"auto"` (RTT-derived) and `"instant"` modes of `moq-watch`'s
`Delay`, and any FFI surface. `libmoq` and `moq-ffi` hand decoded frames to a
caller callback and have no playout stage, so there is nothing there to delay
yet.

## Plan

Branch from main: nothing here breaks a published API.

### One primitive, not two

`moq_mux::pace::Pacer` is already most of `Sync`. Its `anchor: Option<(Instant,
u128)>` is `Sync`'s `reference`, it re-anchors forward only, and `with_lead` is
nearly `moq-watch`'s `buffer`. The only missing piece is a delay offset, so
extend the primitive rather than adding a parallel one beside it.

It keeps the name `Pacer`. `moq_mux::Clock` is taken by the capture-side shared
epoch (`rs/moq-mux/src/clock.rs`, used across `moq-audio`'s capture path and
`moq-cli publish`), which is a different concept, and the rename's only
justification was the clarity that name would have bought. Staying put also keeps
this additive, so `rs/moq-srt/src/server.rs` and `rs/moq-cli/src/subscribe.rs`
need no change at all.

Three things the offset has to get right, none of which fall out of adding it to
`send_at`:

- **The lead comparison must exclude the delay.** `pace` re-anchors when
`send_at` leads `now` by more than `lead`, and `lead` defaults to zero, so a
delay folded into `send_at` makes the very first delayed frame overshoot and
re-anchor, discarding the offset. `hurry` has the same problem: it returns
`now`, undelayed, so every later re-anchor drops the delay again. Compare and
re-anchor on the undelayed instant and apply the delay last.
- **A moved anchor has to wake pending waits.** With audio and video sharing one
`Pacer`, a task can already be parked waiting on a frame when an earlier frame
on the other stream moves the anchor. `Sync.#setReference` resolves `#update`
to wake every parked wait so it recomputes; `Pacer` has no notification at all.

This is a re-anchor notification, not a timed wait: `kio` has no time module
(`rs/CLAUDE.md`), so a `poll_*` that had to fire at a deadline would park
forever with nothing to arm it, and taking a `moq_net::Timers` the way
`origin::Driver::run` does would drag a runtime into a type that has never
needed one. Keep `pace` returning the instant and let the caller sleep, which
is the contract the export path already relies on, and add only the
`poll_*`/`kio::Waiter` pair that reports the anchor moving.
- **Audio must not take the delay twice.** If audio waits for the delayed
instant and then hands samples to a sink that also holds a delay's worth, it
trails video by roughly another delay. Either feed the ring ahead of the
deadline and let it only update the shared anchor, or subtract the sink depth
from the audio write deadline.

### play.rs

- Delete the local `Clock` struct and `video_clock`, and route both audio and
video through the one `Pacer`. The reference anchor becomes the single
authority; the speaker-derived clock goes away.
- Honoring a delay in audio means configuring the sink, not the throttle.
`AUDIO_BUFFER_MAX` in `play.rs` only gates writes against `sink.buffered()`;
the ring itself is sized by the private `LATENCY` (50ms) and `CAPACITY` (3s)
constants in `rs/moq-audio/src/playback/sink.rs`, so one of them has to become
configurable. Keep a floor the way `ringSamples` does in
`js/watch/src/audio/latency.ts`: a zero-depth ring can never be read from.
- `--delay <duration>` replaces `--max-age` on `play`, with no alias. The two are
one number here, since nothing older than `delay` is worth presenting, so
`delay` is what goes on the wire as the subscription's max age. `--max-age`
stays unchanged on `import`, the stdout containers, and `rtmp export`.
- Reconcile `doc/bin/cli.md`, which documents `play --max-age` and carries an
example invocation using it, plus any other `moq play` example in the tree.

### Open questions

Left open deliberately: they are A/V policy that wants the code in front of it,
and the answer changes what the tests should assert.

- **Which stream owns the anchor when both exist.** Video taking a tune-in burst
can `hurry` the shared anchor past what the speaker is still draining, and
because the anchor only moves forward, later audio cannot pull it back. Making
the speaker the sole re-anchor source while audio exists is one answer;
discarding audio to the same edge is another.
- **What the speaker contributes.** `last_timestamp - buffered()` is the sample
sounding now, so feeding it in as the anchor while every result is shifted by
`delay` would schedule video a full delay behind it. If the speaker position
stays in the calculation it has to enter at the live edge, not the playing
edge.
- **Whether a per-write bound is needed.** `play_audio` splits PCM into chunks of
up to a second and checks `sink.buffered()` only between whole writes, so a
first chunk can overshoot a 100ms delay outright. Sizing each part to the
remaining headroom is the obvious fix; whether the sink should grow an
operation for it is not.

### Verification

- Unit tests on `Pacer` with `tokio::time::pause()`: a late first frame followed

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Test the player integration rather than only Pacer

The listed unit tests do not protect the reported moq play regression: the parent tree's rs/moq-mux/src/pace.rs::re_anchors_to_live_edge already verifies that Pacer re-anchors, so these tests can pass even if the renderer remains wired to the one-shot video_clock or otherwise never uses the shared pacer. Add a deterministic play-level test showing that a late first video frame followed by the live edge changes the presentation deadline and fails before the wiring change; the manual check is not a regression gate. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L156-L156

Useful? React with 👍 / 👎.

by an earlier one re-anchors forward; a frame that merely arrives late does
not; the delay survives a re-anchor through both `pace` and `hurry`; a wait
parked on one stream shortens when another stream moves the anchor; and
`delay = 0` paces exactly as today, which is what keeps the export path
unchanged.
- Manually: `moq play` against the demo relay, with audio and video-only,
checking that a late start catches up to live rather than staying behind it,
and that audio and video stay aligned at a non-zero delay.

## Required

- [#2981](/quest/m0/2981-moq-audio-nothing-in-the-decode-or-playback-path-models-a.md) - settles `play_audio`'s gap and silence handling first, so the clock rework is not rebased onto it

## Related

- [#2278](/quest/m2/2278-watch-absolute-wall-clock-latency-target-for-synchronized.md) - the same anchoring model in js/watch, gaining an absolute wall-clock target
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ The codebase actively fights wall-clock interpretation of frame timestamps. `Tim
**Latency is relative (buffer depth), never absolute.** `js/watch/src/sync.ts`:

```ts
export type Bound = "real-time" | Time.Milli;
export type Latency = Bound | { min?: Bound; max?: Bound };
export type Delay = "instant" | "auto" | Time.Milli;
// plus a separate `buffer: Getter<Time.Milli>` input on Sync
```

`Sync.received(timestamp)` computes `ref = Time.Milli.now() - timestamp`, and `Time.Milli.now()` is `performance.now()` (`js/net/src/time.ts:68`) - monotonic since page load, explicitly "not wall-clock time". So `reference` is an arbitrary local-clock↔PTS offset anchored by whichever frame happened to arrive first. Two viewers who tune in 3 seconds apart are 3 seconds apart forever, and nothing in the model can tell.
Expand All @@ -59,7 +59,7 @@ export type Latency = Bound | { min?: Bound; max?: Bound };
#### Proposed shape

1. **A JS `Timeline.Consumer`** in `js/hang` (needed by the DVR work too).
2. **A third `Bound` variant, or a new absolute mode**: something like `latency: { absolute: Time.Milli }` meaning "render pts P at wall time `wall + P + absolute`". `Sync` then anchors on `Timeline.wall` instead of first-frame arrival, and `now()` becomes a wall-clock computation rather than an offset.
2. **A third mode alongside `delay` and `buffer`**: something like `delay: { absolute: Time.Milli }` meaning "render pts P at wall time `wall + P + absolute`". `Sync` then anchors on `Timeline.wall` instead of first-frame arrival, and `now()` becomes a wall-clock computation rather than an offset.
3. **Client clock sync.** This is the real work. `Timeline.wall` is the *publisher's* wall clock; the viewer's `Date.now()` can be off by seconds. Options, roughly in order of cost:
- trust `Date.now()` (fine for a watch party, useless for betting)
- estimate offset from the session RTT (we already track min RTT via PROBE in `#runJitter` for exactly this kind of anchoring)
Expand All @@ -71,13 +71,13 @@ export type Latency = Bound | { min?: Bound; max?: Bound };

#### Open questions

- Absolute latency and the existing `min`/`max` bounds interact awkwardly: an absolute target is a *point*, the bounds are a *range*. Does absolute override the range, or clamp within it?
- An absolute target is a *point* while `delay` and `buffer` are two spans either side of the live edge. Does absolute replace `delay`, or sit alongside it with `buffer` still capping the lookahead?
- `Timeline.wall` is per-rendition. Are audio and video guaranteed consistent? A rendition switch must not re-anchor.
- Does this need anything on the publisher, or is `set_wall` sufficient? Today nothing in `moq-video`/`js/publish` calls `set_wall` as far as I can tell - so step 0 might be "actually populate `wall`".

#### Naming note

`Latency` is already doubly overloaded: the `Bound | {min,max}` type (`js/watch/src/sync.ts:16`) and `class Latency` (`js/hang/src/util/latency.ts:21`, jitter+buffer). Different packages, both exported. Worth resolving before adding a third meaning.
Mostly resolved: `js/watch`'s `Latency` / `Bound` types are gone, replaced by `Delay` plus a separate `buffer`, leaving `class Latency` (`js/hang/src/util/latency.ts`, jitter+buffer) as the only remaining use. Pick a name for the absolute mode that does not reintroduce the collision.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Declare the absent Delay refactor as a blocker

In the reviewed tree, js/watch/src/sync.ts still exports Bound and Latency, and SyncInput still has a latency getter; neither the quoted Delay type nor a separate buffer input exists. Rewriting this main-targeted quest around that absent API, while declaring the collision resolved and adding no dependency on its refactor, leaves the plan unusable if selected now. Keep the current model in the plan or add the refactor as an explicit prerequisite. (Written by GPT-5.6 Sol)

AGENTS.md reference: quest/AGENTS.md:L92-L92

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove the nonexistent Latency collision

In the reviewed tree, js/hang/src/util/latency.ts does not exist, and a repository-wide search finds no class Latency; the remaining Latency declaration is an unrelated interface in the internal js/watch/src/audio/render.ts. This naming note therefore directs the quest executor toward a missing implementation and imposes a collision constraint that no longer exists. Update the plan to state that the old public collision is fully resolved, or identify the actual conflicting symbol. (Written by GPT-5.6 Sol)

AGENTS.md reference: quest/AGENTS.md:L43-L45

Useful? React with 👍 / 👎.


#### Branch

Expand All @@ -90,3 +90,7 @@ export type Latency = Bound | { min?: Bound; max?: Bound };
## Closes

- [#2278](https://github.com/moq-dev/moq/issues/2278) - close this issue when the quest finishes

## Related

- [Playout clock](/quest/m0/playout-clock.md) - ports the same anchoring model to Rust as `moq_mux::Clock`
Loading