Skip to content

feat(watch)!: expose per-frame arrivals through decoder handles - #5311

Merged
kixelated merged 30 commits into
mainfrom
quest/m1/watch-arrivals
Oct 13, 2026
Merged

kixelated merged 30 commits into
mainfrom
quest/m1/watch-arrivals

Conversation

@kixelated

@kixelated kixelated commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The container reader observed transport ingress but discarded its timestamp. Decoder backpressure and group reordering therefore changed the time used to classify late frames, and a page could not draw a complete per-frame arrival history (moq.dev's Voice AI demo draws each word's frames against playout) without patching Sync.received.

Approach

Built on #5288 (merged), which makes the container read yield frame | gap | end items in delivery order.

Carry local ingress on the read's frame (Container.ReceivedFrame) and classify it in Sync.received. Each decoder fans what it receives out to caller-owned handles: decoder.arrivals() returns an Arrivals handle that yields, while it is open:

type Arrival =
	| { kind: "frame"; timestamp: Time.Milli; arrival: Time.Milli; late: boolean }
	| { kind: "gap"; timestamp: Time.Milli; duration: Time.Milli };
for await (const entry of player.audio.arrivals()) {
	if (entry.kind === "gap") { line.break(entry.timestamp, entry.duration); continue; } // a latency skip, a hole
	line.push(entry.timestamp, entry.arrival, entry.late);
}
  • arrival is stamped when readFrame() returns, before reordering or decoder backpressure.
  • late is computed from ingress, not delivery.
  • A gap is the container's gap item passed through, in milliseconds: media after an earlier frame will never arrive. Joining is not a gap. A declared end is not a gap and is not reported. Retiring a fully delivered group, or one holding only its trailing endpoint, is not a gap either (fixed in feat(mux): read gaps and declared ends as items in delivery order #5288, with the two regression tests that started here).
  • The container read's gap now carries its span in both languages (JS { kind: "gap"; timestamp; duration } in Time.Micro, Rust Read::Gap { timestamp, duration } as moq_net::Timestamp in the next item's scale). timestamp is the furthest presentation point already read (max of timestamp + duration, endpoints included); duration runs to the item it precedes, clamped at zero. It is an upper bound: without a per-frame duration it starts one frame early, and it covers any declared pause in between. No gap is read before the first item, in either language.
  • Video reports only the promoted rendition (the one on screen). A rendition preparing to replace it reports nothing until promotion, so two renditions never interleave.
  • Closing a handle releases its queue; a decoder with no open handle retains nothing. Closing the decoder ends its handles once they drain.

Decisions

Read shape shared with #5288 (maintainer, resolved there):

  • ✅ The arrivals stream mirrors the container read: a frame item or a gap item, in delivery order. The previous skipped flag and the consumer's separate skip bookkeeping are deleted.
  • A per-frame skipped boolean (previous revision): a second notion of loss beside the read's gap, and one more field every fake read result had to supply.

Gap span (maintainer): every gap but the join knows its span, from the delivered edge to the item it precedes; the join has no start.

  • ✅ No join gap; required timestamp + duration on every gap, documented as an upper bound. The decoders that react to a gap (JS Terminal, Rust audio restart(), CLI play's video_restarted(), replay.ts) start in the same state a gap resets them to, so dropping the join gap changes nothing for them.
  • Optional pair, absent only at the join: keeps feat(mux): read gaps and declared ends as items in delivery order #5288's leading gap, but every reader handles the missing case.
  • Resume point required, duration optional: always filled in, but timestamp would mean the gap's end, unlike Frame.timestamp.

Handle vs. retained snapshot (maintainer):

  • ✅ A caller-owned observation handle replaces the retained out.arrivals snapshots.
  • Fixed 30 s / 4,096-frame snapshot window on each decoder (earlier revision).

Handle shape (follows the repo's Consumer convention, as in Hang.Timeline.Consumer):

  • ✅ decoder.arrivals(): Arrivals with next(), async iteration, and close().
  • A handle exposing a growing Signal<readonly Arrival[]> snapshot: convenient to bind, but copying on every frame scales with retained history.

Handle buffer bound (settled on the recommendation):

  • ✅ Unbounded unread queue; the caller reads continuously and keeps its own window. No arbitrary constant.
  • A fixed backlog cap that fails loud, or a caller-chosen arrivals({ backlog }).

Late frames under group reordering (Codex P1 declined; settled on the recommendation):

  • ✅ Classify late in delivery order, against the playout reference Sync actually runs.
  • Classify in ingress order, which grades frames against a reference playback never used.

Video rendition switch:

  • ✅ Report only the promoted rendition; drop what a pending rendition received before promotion.
  • Hold pending arrivals and flush them at promotion, which overlaps timestamps from two renditions.

Impact

Public API (on top of #5288; no wire, catalog, or encoded container change):

  • @moq/hang: new Container.ReceivedFrame extends Frame with a required local arrival; the frame item of Container.Read carries it. Producer Frame inputs are unchanged. Fake read results must supply arrival.
  • @moq/watch: Sync.received(timestamp, label?, arrival?) takes recorded ingress and returns whether the frame was late; an omitted arrival samples the current time as before.
  • @moq/watch: new Audio.Decoder.arrivals() and Video.Decoder.arrivals(); exports types Watch.Arrival (the frame | gap union, the gap carrying timestamp and duration in Time.Milli) and Watch.Arrivals. No out.arrivals signal.
  • @moq/hang (breaking): Container.Read's gap is { kind: "gap"; timestamp: Time.Micro; duration: Time.Micro }, and next() no longer yields a gap before the first item.
  • moq-mux (breaking): container::Read::Gap { timestamp, duration } (both moq_net::Timestamp), and Consumer::read no longer yields a gap before the first item. In-repo matches (moq-audio, moq-video, moq-cli, moq-c, moq-ffi, moq-gst, moq-rtc, moq-srt, the hang example) only change their patterns; no FFI, C ABI, or binding surface changes.
  • @moq/hang: Container.Consumer.close() drops an item deferred behind a returned gap, so reads after close return undefined (a feat(mux): read gaps and declared ends as items in delivery order #5288 follow-up from review).
  • Demo consumers need no migration.

Verification

just check passes. Container tests in both languages pin each gap's span (legacy without durations, CMAF with them, a hole after a declared end, the overlapping-edge clamp, cross-scale conversion in Rust) and that no gap precedes the first item. Watch tests cover handle fan-out, release on close (including leaving a for await), drain on decoder close, concurrent next(), the gap item reaching an audio handle before the frame it precedes, audio and legacy/CMAF video decoder output, and recorded-trace replay where held delivery keeps the recorded arrival and lateness (no gap before the first frame). A nightly benchmark (just js arrivals-bench) sweeps 1/4/16 handles against 0/256/4,096 unread entries; per-handle write+read cost stays flat with no slope in the backlog.

The completed quest quest/m1/watch-arrivals.md is removed.

Follow-ups

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 6 commits October 10, 2026 14:04
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Co-Authored-By: GPT-6 Astra <noreply@openai.com>
kixelated and others added 2 commits October 11, 2026 18:02
…andles

Decoders no longer keep an `out.arrivals` snapshot. `arrivals()` opens a
handle that queues every frame received while it is open; closing it
releases the queue, so players that don't chart retain nothing. Video
reports only the rendition on screen. The quest is complete and removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title feat(watch): expose bounded ingress diagnostics feat(watch): expose per-frame arrivals through decoder handles Oct 12, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T20:38:40.188979Z ba99ad9 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c4ca61a51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/watch/src/arrivals.ts Outdated
Comment on lines +53 to +57
async *[Symbol.asyncIterator](): AsyncIterator<Arrival> {
for (;;) {
const entry = await this.next();
if (entry === undefined) return;
yield entry;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Close the handle when async iteration stops

When a caller exits for await early via break, return, or an exception, JavaScript closes this generator but never calls Arrivals.close(). The state therefore remains in Producer.#open and its unbounded queue retains every subsequent frame until the decoder closes, so repeated short-lived iterations can leak increasing amounts of memory. Wrap the generator loop in a try/finally that closes the handle.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 76eb7a6: the iterator closes the handle in a finally, with a regression test for break.

(Written by Claude Opus 5.5)

Comment thread js/watch/src/arrivals.ts Outdated
Comment on lines +41 to +44
const entry = state.queue.shift();
if (entry) return entry;
if (state.closed) return undefined;
if (state.wake) throw new Error("multiple calls to next not supported");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject a second next() before dequeuing

When one next() is pending, a write queues an entry and wakes it, but its continuation does not run until a microtask. A second next() issued in that interval shifts the queued entry before checking state.wake, leaving the original call waiting for another frame instead of rejecting the unsupported concurrent read; this can reorder or indefinitely stall callers. Check for an in-flight read before consuming the queue.

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 76eb7a6: next() checks an in-flight flag before touching the queue, with a regression test for a write landing between wake and resume.

(Written by Claude Opus 5.5)

Leaving a for-await loop now closes the handle, so it stops queueing frames.
A second next() is refused before it can take a frame queued for the
pending read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76eb7a66c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/hang/src/container/consumer.ts Outdated
if (marker || !ptsContiguous(first.end ?? this.#presentedEnd, nextStart)) {
hole = true;
}
if (!marker && (!first.empty || !first.done)) this.#skipped = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not mark fully delivered groups as skipped

When decoder backpressure has consumed a completed group's last media frame but has not yet requested its group-done result, that group remains in #groups with an empty frame queue. If later arrivals push the buffered span over maxDelay, this branch removes the completed group and sets #skipped because first.empty records historical content, even though no unread payload was discarded. The next arrival is therefore falsely reported as following a latency skip; gate this marker on unread or still-pending media instead.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, reproduced. Fixed in a0c16a6: the skip mark now requires unread frames or a group still arriving, with a regression test that failed before the change.

(Written by Claude Opus 5.5)

A delivered group waiting on its done result has nothing unread, so
retiring it for the max delay no longer marks the next frame skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0c16a620f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/watch/src/arrivals.ts Outdated
const state = this.#state;
try {
for (;;) {
const entry = state.queue.shift();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid linear-time dequeue for unbounded arrival queues

When a handle accumulates a large unread backlog, every next() calls Array.shift(), which reindexes the remaining array and makes catch-up cost grow linearly with the backlog. Because these queues are intentionally unbounded, a temporarily stalled reader can incur quadratic total work while draining and block the browser thread; the newly added backlog sweep is specifically the scenario that exposes this slope. Use a head index with occasional compaction or a deque instead. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L37-L37

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 26a5c65: reads advance a head index and compact the read prefix once it outweighs the rest, so draining is amortized O(1). The nightly sweep stays flat across 0/256/4,096 standing backlog.

(Written by Claude Opus 5.5)

Advance a head index and compact the read prefix instead of shifting,
which can reindex a large backlog on every read in some engines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26a5c65523

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/hang/src/container/consumer.ts Outdated
hole = true;
}
// Only unread or still-arriving content is a skip; a group already delivered discards nothing.
if (!marker && (first.frames.length > 0 || !first.done)) this.#skipped = true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude queued endpoint markers from skip detection

The earlier fully-delivered-group fix still misses a queued endpoint: legacy video appends an empty-payload duration marker to an otherwise normal media group (legacy.ts lines 20-28). If the media frames have been delivered but that marker remains queued when later groups exceed maxDelay, first.media makes marker false and first.frames.length > 0 sets #skipped, so the next actual media frame falsely reports that content was discarded. Check whether the unread entries contain media rather than counting the endpoint marker, and add coverage for this state. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, reproduced with a legacy video group holding only its endpoint. Fixed in 4a227bf: the skip mark counts unread media only, with a regression test that failed before the change.

(Written by Claude Opus 5.5)

A legacy video group whose media was delivered but whose endpoint marker is
still queued discards no content when the max delay retires it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a227bf3cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Classify ingress before decoder backpressure changes the delivery clock.
const timestamp = Time.Milli.fromMicro(frame.timestamp as Time.Micro);
this.sync.received(timestamp, "audio");
const late = this.sync.received(timestamp, "audio", frame.arrival);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Classify reordered frames in ingress order

When group streams arrive out of order but Container.Consumer later emits them in presentation order, these calls still reach Sync.received in delivery order. received is order-dependent because its first call establishes the reference and subsequent calls may re-anchor it, so merely passing the recorded timestamp does not preserve ingress semantics: for example, a frame at PTS 100 arriving at 100 ms followed by PTS 0 arriving at 200 ms can be delivered as PTS 0 then 100, causing the actually late PTS 0 frame to establish the clock and be reported as late: false. This defeats the feature's recorded-ingress guarantee for both audio and video whenever group reordering occurs; classification must be performed in ingress order or made independent of call order.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for this PR. late means the frame arrived after the deadline of the playout clock Sync actually runs, and that clock is built in delivery order: in your example PTS 0 anchors the reference, so against the real playout it was on time. Classifying in ingress order would grade frames against a clock that never existed, and would split late from the reference that schedules playback. What the ingress stamp fixes is the distortion the quest named: backpressure and held delivery moving the sample time. Re-ordering the anchor itself is a Sync design change; I've flagged it to the maintainer as a possible follow-up.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 3 commits October 12, 2026 09:58
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… continuity

Rename container::Sample to container::Delivery so it does not collide
with PCM or MP4 samples. Complete the Rust continuity quest for container
and audio reads and move video propagation to quest/m1/video-continuous.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uous

An aborted current group skipped to the next buffered group without
advancing the playhead, so the next frame read as continuous although
the rest of the evicted group was lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 9 commits October 12, 2026 19:42
Replace the container::Delivery wrapper and the polled discontinuity()/end()
accessors with one read stream in both languages:

- Rust: container::Read::{Frame { group, frame }, Gap, End { group, end }}
- JS: Consumer.next() yields { kind: "frame", group, frame } | { kind: "gap" }
  | { kind: "end", group, end }; the continuous/discontinuity fields, the
  group-done results, and the discontinuity getter are gone.

A gap is media this reader will never get (the join, a skip, an eviction, a
truncation, an unproven hole). An end is the publisher declaring its media
stops: a marker group or an audio terminal. The frames that follow an end
from its own group are the stopped encoder's drain; the next group is fresh.
Timestamps cannot tell those apart (drain packets are stamped past the end),
so the group rides along, as the hang draft scopes the endpoint.

Trailing video duration markers stay internal. moq-cli play now applies a
gap or end before the frame after it instead of polling a counter after the
read. Drops quest/m1/video-continuous.md: decoders read the gap directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	quest/m2/intra-refresh/consumer-warmup.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retiring a fully delivered group, or one holding only a trailing endpoint, for the max delay
read as a gap. Only unread media, a shed marker group, or a broken timeline is a gap, matching
the Rust skip path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge the gap-item read from quest/m1/rust-continuous. An arrivals handle now yields
`{ kind: "frame", timestamp, arrival, late }` or `{ kind: "gap" }` in delivery order, written
from the container's own gap items, so the `skipped` flag and the consumer's separate skip
tracking are gone. The container read's frame keeps its transport arrival stamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tired groups

Rust startup now adopts the first group carrying any wire frame, so joining a paused track whose
only served group is a marker reads the gap and end instead of waiting for media. JS retirement
for the max delay only treats an adjacent successor as continuous once the retired group
finished cleanly; a truncated or still-open group may have lost its tail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2fef1881d2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +458 to +461
if (this.#next) {
const next = this.#next;
this.#next = undefined;
return next;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the deferred read when closing the consumer

When a caller reads a gap, #emit has already removed the following frame or end from its group and stored it in #next. If the caller then invokes close() before reading again, this fast path returns that deferred item without observing the closed state, even though cleanup discarded every other buffered item and the existing close contract expects subsequent reads to return undefined. Clear #next during close or check the terminal state before returning it. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L29-L30

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, reproduced. Fixed in 0cb0cbc: close() drops the deferred item, with a regression test that failed before the change.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 13, 2026 11:04
…5311

# Conflicts:
#	js/hang/src/container/consumer.ts
#	js/watch/src/audio/decoder.test.ts
#	js/watch/src/audio/decoder.ts
#	js/watch/src/audio/replay.ts
#	js/watch/src/video/decoder.test.ts
#	js/watch/src/video/decoder.ts
#	rs/moq-cli/src/play/video.rs
…d ahead of

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 13, 2026 18:08
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged main (#5288 is in, so the diff is this PR's own work) and fixed the open Codex finding. Holding the merge on the gap shape, since it is an API choice for both languages.

What a gap can know. JS Container.Consumer and Rust container::Consumer emit one gap lazily, ahead of the next item, after any loss (join, latency skip, eviction of unread media, truncated group, unproven sequence hole).

  • Where media resumes: always known, the timestamp of the item the gap precedes (frame timestamp, or an end's end).
  • Where it starts: the furthest presentation point already delivered (JS #presentedEnd, Rust presented_end / live_edge). Known for every gap except the join, where nothing was delivered.
  • The span is a bound, not exact: with no per-frame duration (Legacy, streaming LOC) the start is the last frame's timestamp, so it over-counts one frame, and after a declared end a later loss spans the declared pause too.

Options (each mirrored as Rust Read::Gap { .. }, 9 match sites):

  1. No join gap; required timestamp + duration (recommended). { kind: "gap"; timestamp: Time.Micro; duration: Time.Micro }. A gap always means lost media with a known span, so nothing is optional. Changes feat(mux): read gaps and declared ends as items in delivery order #5288's "a gap before the first frame" contract; the decoders (JS Terminal, Rust audio restart(), CLI play) start fresh anyway, so it should be behavior-neutral, but each needs a check.
  2. Optional pair, absent only at the join. { kind: "gap"; timestamp?: Time.Micro; duration?: Time.Micro } (Rust Option<..> on each, or one Option<Span>). Keeps feat(mux): read gaps and declared ends as items in delivery order #5288 as is; every reader handles the missing case.
  3. Resume point required, duration optional. { kind: "gap"; timestamp: Time.Micro; duration?: Time.Micro }, where timestamp is where media resumes. Always populated, but timestamp would mean the gap's end, unlike Frame.timestamp.

Watch's Arrival gap would carry the same fields in milliseconds (Time.Milli), like the frame entry.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0cb0cbcf38

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Upgrade the pending track to active.
// #runActive will be in charge of it now.
if (current) current.arrivals = undefined;
pending.arrivals = this.#arrivals;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the frame that promotes a video rendition

When switching renditions within the same broadcast, this assignment happens only after pending.timestamp proves the pending track has caught up. That timestamp is produced by decoding at least one frame, but the arrival write occurs before decoding while pending.arrivals is still undefined, so the frame that becomes the first displayed picture, plus its initial join gap, is omitted from every open arrival handle. If no subsequent frame arrives, observers see nothing for the newly displayed rendition; buffer the pending diagnostics and publish them on promotion, or otherwise retain the promotion frame without reporting candidates that never become active.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining; this is the settled rendition-switch decision in the description. Promotion runs in an effect after the pending track's decoded timestamp catches up, so several pending frames may have been received by then and no single one is "the promotion frame". Flushing what the pending rendition received would interleave timestamps from two renditions, which is what the decision rules out. Every frame after promotion is reported, so the only loss is a track that ends at the instant it is promoted. No gap is written at promotion because the switch is seamless: no media is missing.

(Written by Claude Opus 5.5)

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 400da7c5-a2c4-4d09-8f32-ef80d56258cc
📥 Commits

Reviewing files that changed from the base of the PR and between 014e230 and 0cb0cbc.

📒 Files selected for processing (20)
  • .github/workflows/nightly.yml
  • doc/lib/js/watch.md
  • js/hang/src/container/consumer.test.ts
  • js/hang/src/container/consumer.ts
  • js/hang/src/container/types.ts
  • js/justfile
  • js/watch/bench/arrivals.ts
  • js/watch/src/arrivals.test.ts
  • js/watch/src/arrivals.ts
  • js/watch/src/audio/decoder.test.ts
  • js/watch/src/audio/decoder.ts
  • js/watch/src/audio/replay.test.ts
  • js/watch/src/audio/replay.ts
  • js/watch/src/index.ts
  • js/watch/src/sync.test.ts
  • js/watch/src/sync.ts
  • js/watch/src/video/decoder.test.ts
  • js/watch/src/video/decoder.ts
  • quest/m1/README.md
  • quest/m1/watch-arrivals.md
💤 Files with no reviewable changes (2)
  • quest/m1/README.md
  • quest/m1/watch-arrivals.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The change adds ingress timestamps to received frames and introduces asynchronous arrival handles for frame and gap events. Audio and video decoders publish arrival and lateness data, and replay records arrival diagnostics in each quantum. Tests cover handle behavior, decoder output, recorded ingress, and replay timing. Documentation and a nightly benchmark are also added.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 0cb0c

The change adds per-frame arrival diagnostics to the watch package. No concrete merge-blocking problem was found. The author is still deciding the gap API shape, so that remains open for the author.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 15 files. (3 skipped: 3… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly and concisely describes the main change: exposing per-frame arrivals through decoder handles.
Description check Passed The description directly explains the problem, implementation, public API changes, testing, and impact of the arrival diagnostics work.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 15 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The container read's gap now says where the missing media starts (the
furthest point already read) and how long until media resumes, as an
upper bound, in both languages. Nothing read before the first item means
no gap there. Watch's arrival gap carries the same span in milliseconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Implemented option 1 in 1b38d59: no join gap in either language, and every gap carries a required timestamp (furthest point read) and duration (to the item it precedes, clamped at zero), documented as an upper bound. Rust Read::Gap { timestamp, duration } uses moq_net::Timestamp in the next item's scale; watch's arrival gap carries the same in Time.Milli. The decoders that reset on a gap start in that reset state already, so dropping the join gap changes nothing for them.

(Written by Claude Opus 5.5)

…5311

# Conflicts:
#	quest/m1/README.md
#	quest/m2/watch-arrivals.md
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: ba99ad955d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging. Summary since the last review round:

  • Merged main twice; the diff is this PR's own work. The completed quest (moved to quest/m2/watch-arrivals.md on main) is deleted.
  • Gap span (maintainer, option 1): no join gap, and every gap carries a required timestamp and duration as an upper bound, mirrored in JS Container.Read, Rust container::Read::Gap, and watch's Arrival. Quests keyed on the join gap now key on the first frame read too.
  • Container.Consumer.close() drops an item deferred behind a gap (Codex finding, with a regression test).
  • Declined the rendition-promotion finding per the settled decision.

Codex reviewed ba99ad9 with no findings; CI is green.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 13, 2026
@kixelated kixelated changed the title feat(watch): expose per-frame arrivals through decoder handles feat(watch)!: expose per-frame arrivals through decoder handles Oct 13, 2026
Merged via the queue into main with commit eb93bf4 Oct 13, 2026
17 checks passed
@kixelated
kixelated deleted the quest/m1/watch-arrivals branch October 13, 2026 21:21
kixelated added a commit to moq-dev/moq.dev that referenced this pull request Oct 19, 2026
moq.pro admitted moq.dev on staging (moq-dev/moq.pro#2301), so the
voice-api-moqdev condition quest is deleted. @moq/watch 0.6.3 came from
moq's release branch without arrivals() (moq-dev/moq#5311), so that
condition still holds open. voice.json merged in moq.pro but wtf is not
redeployed; its quest link moves to m2. The page starts from
voice-demo.tsx on claude/voice-post-components, and the buffer goal calls
red frames an estimate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant