Repository navigation
quest(m1): plan per-frame arrivals in @moq/watch #5123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # [S] Per-frame arrivals in @moq/watch | ||
|
|
||
| ## Goal | ||
|
|
||
| The audio and video decoders in `@moq/watch` each expose an `arrivals` | ||
| signal: a bounded window of recent frames as `{timestamp, arrival, late, | ||
| skipped}`, where `arrival` is when the frame came off the transport, `late` | ||
| means it arrived after its sync deadline (not that it rendered late), and | ||
| `skipped` marks content the consumer dropped to stay within the latency | ||
| budget. A page can draw per-frame arrival against playout without patching | ||
| `Sync.received`. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-10-09: | ||
|
|
||
| - **A signal, not a callback.** A signal holding only the newest arrival | ||
| would coalesce and lose frames between reads, so it holds a window that | ||
| updates per frame. It must reach back at least as far as the span the | ||
| first consumer draws (a transcript word); time or count bound is chosen | ||
| when implementing. | ||
| - **Feed it from where the data already exists.** `Sync.received()` | ||
| computes lateness per frame and only logs it today; the container consumer | ||
| knows which groups it skipped and only warns. Neither has a signal. | ||
|
Comment on lines
+22
to
+24
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In the inspected JS path, AGENTS.md reference: AGENTS.md:L16-L16 Useful? React with 👍 / 👎. |
||
| - **`arrival` is the ingress time.** `Container.Consumer` stamps each frame | ||
| as `readFrame()` returns, but the `Frame` it returns drops that stamp, and | ||
| `Sync.received()` samples the clock later, after reordering and any | ||
| decoder backpressure. Carry the ingress stamp through to the decoder and | ||
| compute lateness from it. | ||
| - **Skipped content never reaches the decoder.** A latency-skipped group's | ||
| frames are not decoded, so they cannot appear as ordinary entries. Decide | ||
| how the window represents the gap (a mark on the first frame after it, or | ||
| a separate entry), keeping it distinct from other `continuous: false` | ||
| causes such as markers. | ||
| - The first consumer is moq.dev's Voice AI demo, which draws each word's | ||
| frames and marks the ones a WebRTC jitter buffer would have dropped | ||
| ([moq.dev's condition quest](https://github.com/moq-dev/moq.dev/blob/main/quest/m1/watch-arrivals-release.md)). | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| - Test with recorded arrival traces through `js/watch/src/audio/replay.ts`, | ||
| including a case where delivery is delayed but the recorded arrival is | ||
| unchanged. | ||
|
|
||
| ## Related | ||
|
|
||
| - [Browser stats](/quest/m1/stats/js.md) - its late-frame and skip counters share these sources, but count at the producer since a reader of an evicting window can miss entries | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 42004
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 41582
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 41670
🏁 Script executed:
Repository: moq-dev/moq
Length of output: 42353
Propagate the consumer’s arrival timestamp.
Container.ConsumerrecordsMoq.Time.Milli.now()afterreadFrame()returns and uses it for jitter estimation. The returnedFramedoes not carry that timestamp, so the decoder’sSync.received()call samples the clock later, afternextMedia()returns. Carry the consumer timestamp with each frame and definearrivalas that consumer-observed time. Do not recompute it in the decoder.🤖 Prompt for AI Agents