From f8d920f84cac9f6b07ec6dde26b6d459cb12f986 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 09:12:52 -0800 Subject: [PATCH 01/23] chore(quest): claim quest/m2/unused-api/net.md Quest-Claim: dab5b0a95de7342e From 2c207550ebca67ca140c76481d2db7231723de18 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 09:22:09 -0800 Subject: [PATCH 02/23] quest(unused-api/net): design the container reader's move onto Ordered Co-Authored-By: Claude Opus 5.5 --- quest/m2/unused-api/README.md | 2 +- quest/m2/unused-api/net.md | 75 +++++++++++++++++++++++++++-------- 2 files changed, 59 insertions(+), 18 deletions(-) diff --git a/quest/m2/unused-api/README.md b/quest/m2/unused-api/README.md index fa57dcee0b..cfc08b9c07 100644 --- a/quest/m2/unused-api/README.md +++ b/quest/m2/unused-api/README.md @@ -22,7 +22,7 @@ check Kixel and moq.pro for a caller, and keep what one uses. Both pin a lists its items with the grep that found no caller here, so its PR re-runs the grep on its own head and adds the downstream check. -Removal is the default. Hide an item (`pub(crate)`, `#[doc(hidden)]`, +Removal is the default. Hide an item (`pub(crate)`, `@internal`, a `test-support` or `lab` feature) only when code in this repository still needs it across a crate or package boundary. diff --git a/quest/m2/unused-api/net.md b/quest/m2/unused-api/net.md index fafd832ccc..04e8d580c5 100644 --- a/quest/m2/unused-api/net.md +++ b/quest/m2/unused-api/net.md @@ -3,8 +3,8 @@ ## Goal `track::Subscriber::next_in_range`, `is_skipped`, and `poll_start` are -crate-private, and so is `Subscriber::with_range` if its last production -caller can leave. Their one consumer is moq-mux's container reader, which +crate-private, and their js/net mirrors leave its published surface. Their +one consumer is the container reader (moq-mux, and `@moq/hang` in JS), which orders groups itself; `track::Ordered` reports a `Gap`, so the reader can move onto it. @@ -51,23 +51,59 @@ Decided 2026-11-17 (maintainer, `/quest-run m0/m1/m2`): documented surface; C, shrink it to making `with_range` crate-private alone). +Hiding is no middle ground: `#[doc(hidden)]` is banned +([moq-dev/moq#8392](https://github.com/moq-dev/moq/pull/8392)), and js/net +keeps `@internal` for deprecated exports. + +Step 1's design (2026-11-19, from reading `container/consumer.rs` and +`Ordered`), rule by rule for a hole at the reader's cursor: + +- `Ordered` already covers a finished track, keys the ranges exclude + (`next_in_range`), declared skips (`is_skipped`), and `Delay::max` aged on + the reach against the newest frame, though only on a track whose moq-net + groups carry timestamps; on an untimed one it waits until the hole + resolves, where the reader ages on decoded container timestamps. +- The reader keeps 1 ms continuity (on a `Gap`, it calls the loss only once + the next group's first frame fails to continue the edge) and the slow-group + skip, since `Ordered` hands out the next group while the reader still reads + the current one. A full max buffer past a hole is moot: `Ordered` leaves + those groups in the transport cache. +- `Ordered` cannot run two rules, since it never hands out the groups past a + hole: the `Delay::min` wall-time wait since a group past the hole landed, + and the playout clock reaching where that group starts. + +Options for those two, waiting on the maintainer (see Waiting): + +- A (recommended): `Ordered` lets the reader see the group held past the + hole it waits at and give that hole up as a `Gap` now: two public methods. + Every reader rule and its hole tests carry over, untimed tracks included, + and moq-net takes no playout policy. The reader's aged rule sees that one + group, not the newest content past it; `Ordered`'s own `Delay::max` still + ages the hole on a timed track. +- B: `Ordered` takes the policy: `Delay::min` becomes a wall-time hole wait + for every ordered reader (a behavior change for moq-json, moq-hls, and the + bindings, which read `Ordered` too), plus an input for the clock's position + as a group key, which only a time-keyed track has: a sequence-keyed legacy + track loses the clock rule. + +Either way the net public API shrinks by one method (three leave, A adds two, +B one input plus a behavior change), so keeping the three public, rejected +above, stays the cheaper path if the rewrite's playback risk outweighs that. + +`@moq/hang`'s container reader calls the js/net mirrors (`start`, +`nextInRange`, `isSkipped`), so the same move applies there in the same PR. + +`with_range` stays public (recommended, from the same reading): moq-json, +moq-audio, and moq-mux's `container/source.rs` call it as a local filter on +an established subscription, which `Subscription::ranges`, a wire request, +does not replace. + Steps: -1. Design the container reader's hole policy on `track::Ordered` first: what - moves into moq-net (the wait, the clock, the buffer cap, continuity) and - what stays in the reader. Decide there whether `Ordered` takes the - wall-time wait that `Delay::min` now means, and how the reader's hole - tests in `container/consumer.rs` carry over. Added `Ordered` API needs the - maintainer's say. +1. Add the picked `Ordered` API. 2. Move the reader onto it, and the `is_skipped` callers onto `Ordered::is_skipped`. -3. Make `next_in_range`, `is_skipped`, and `poll_start` crate-private. Make - `Subscriber::with_range` crate-private too if the callers listed above can - leave: `moq-json`, `moq-audio`, and the benches move to - `Subscription::ranges`; `container/source.rs`'s resubscription needs - "after group N" with a time start, which a `Range` cannot say, so it - needs the reader's own cursor to drop that group. Otherwise `with_range` - stays and this quest records why. +3. Make `next_in_range`, `is_skipped`, and `poll_start` crate-private. - `origin::Consumer::covering` stays `pub`: moq.pro and moq-net's own `benches/origin.rs` call it. @@ -76,8 +112,13 @@ The reader reads gaps through `Ordered::next_item` once `next_group` then yields groups alone (decided 2026-11-18, maintainer, /quest-plan); whichever lands second makes the switch. -Public API: breaking, removals only unless step 1 adds `Ordered` inputs. -Wire: none. +Public API: breaking; A adds two `Ordered` methods, B one input. Wire: none. + +## Waiting + +- Maintainer: pick how the container reader's hole wait and playout clock + reach `Ordered`, A or B above; recommended A. An agent cannot pick: this + quest reserves new `Ordered` API for the maintainer's say. ## Related From 384500f86d975d7c8d529aa3e2d63bb7edeafcfc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 12:05:51 -0800 Subject: [PATCH 03/23] quest(unused-api/net): record B and the two decisions it leaves open Co-Authored-By: Claude Opus 5.5 --- quest/m2/unused-api/net.md | 77 +++++++++++++++++++++++++++----------- 1 file changed, 55 insertions(+), 22 deletions(-) diff --git a/quest/m2/unused-api/net.md b/quest/m2/unused-api/net.md index 04e8d580c5..64a94ffb2e 100644 --- a/quest/m2/unused-api/net.md +++ b/quest/m2/unused-api/net.md @@ -72,23 +72,36 @@ Step 1's design (2026-11-19, from reading `container/consumer.rs` and hole: the `Delay::min` wall-time wait since a group past the hole landed, and the playout clock reaching where that group starts. -Options for those two, waiting on the maintainer (see Waiting): - -- A (recommended): `Ordered` lets the reader see the group held past the - hole it waits at and give that hole up as a `Gap` now: two public methods. - Every reader rule and its hole tests carry over, untimed tracks included, - and moq-net takes no playout policy. The reader's aged rule sees that one - group, not the newest content past it; `Ordered`'s own `Delay::max` still - ages the hole on a timed track. -- B: `Ordered` takes the policy: `Delay::min` becomes a wall-time hole wait - for every ordered reader (a behavior change for moq-json, moq-hls, and the - bindings, which read `Ordered` too), plus an input for the clock's position - as a group key, which only a time-keyed track has: a sequence-keyed legacy - track loses the clock rule. - -Either way the net public API shrinks by one method (three leave, A adds two, -B one input plus a behavior change), so keeping the three public, rejected -above, stays the cheaper path if the rewrite's playback risk outweighs that. +Decided 2026-11-19 (maintainer, `/quest-complete`): ✅ B, `Ordered` owns +the hole policy (rejected: A, two peek/give-up methods for the reader; C, keep +`next_in_range`, `poll_start` and `is_skipped` public). `Ordered` gives up a +hole once `Delay::min` of wall time has passed since a later group landed (an +unstated minimum keeps waiting for `Delay::max`, as today), and takes a +playout-position input that gives up a hole once playback reaches where the +next group starts. Reason: these are player rules any `Ordered` reader that +states a minimum wants, and `Delay::min`'s docs already promise the wait. + +Findings while starting B (2026-11-19, from reading its callers): + +- `Delay::default()` states `min: Some(0)`, so every reader built from + `Delay::default().with_max(..)` states a zero minimum. moq-archive's ordered + store (`producer/track.rs`) does, and under B it would give up a reordered + group the moment a later one lands, where it waits up to its budget today. + The bindings' defaults (`moq_subscription_defaults`, libmoq's zeroed + `moq_delay`) state it too, so a binding user who raises only `max` changes + as well. +- `Ordered` never hands the reader a group it skips as stale or gives up as + late, so the reader can no longer report when that backlog landed: + `container::Consumer::backlog()`, which moq-audio feeds its join estimate + from, loses most of its signal. +- A local track owes nothing, so `Ordered` steps over its missing keys at + once and without a gap. The reader then keeps only its 1 ms continuity + rule there (and calls an undeclared, discontinuous run a gap without + waiting); its hole tests carry over on a received copy + (`container::test_util::Link`), not on a local track. +- `Ordered`'s wall-clock deadlines (the new `min` wait and playout position, + like `Delay::max` today) fire only on a track whose cache pool a driver + collects, as an origin's does; the reader's own timer goes away. `@moq/hang`'s container reader calls the js/net mirrors (`start`, `nextInRange`, `isSkipped`), so the same move applies there in the same PR. @@ -100,7 +113,9 @@ does not replace. Steps: -1. Add the picked `Ordered` API. +1. Add B's `Ordered` policy: the `Delay::min` hole wait and a playout-position + input (Rust `Ordered::set_playhead`, js/net `Ordered.setPlayhead`), shaped + by the decisions under Waiting. 2. Move the reader onto it, and the `is_skipped` callers onto `Ordered::is_skipped`. 3. Make `next_in_range`, `is_skipped`, and `poll_start` crate-private. @@ -112,13 +127,31 @@ The reader reads gaps through `Ordered::next_item` once `next_group` then yields groups alone (decided 2026-11-18, maintainer, /quest-plan); whichever lands second makes the switch. -Public API: breaking; A adds two `Ordered` methods, B one input. Wire: none. +Public API: breaking; three `Subscriber` methods go crate-private, `Ordered` +gains one input, and `Delay::min` becomes a hole wait for every ordered +reader. Wire: none. ## Waiting -- Maintainer: pick how the container reader's hole wait and playout clock - reach `Ordered`, A or B above; recommended A. An agent cannot pick: this - quest reserves new `Ordered` API for the maintainer's say. +- Maintainer: which ordered readers get B's `Delay::min` hole wait, since + `Delay::default()` states a zero minimum (see Findings while starting B): + - 1a (recommended): every reader that states a minimum, zero included, as + decided; in-tree readers that tolerate reordering (moq-archive, and any + other `Ordered` reader the audit finds) state `with_min(None)`, and the + bindings document that a zero minimum now gives a hole up at once. + - 1b: `Delay::default().min` becomes `None`, so only a stated minimum waits; + a media reader built from `Delay::default().with_max(x)` then waits `x` + for a hole instead of zero, and the bindings' defaults change with it. +- Maintainer: how moq-audio keeps learning when a join's backlog landed once + the reader reads through `Ordered` (see Findings while starting B): + - 2a (recommended): moq-audio observes every arrival through a second, + arrival-order subscription, as `@moq/hang`'s reader already does + (`#observe`), and `container::Consumer::backlog()` is removed (breaking + in moq-mux, one caller). + - 2b: `track::Item::Gap` gains when the latest group it gives up had + landed, which the reader reports as `backlog()`; mirrored in js/net. + - 2c: `backlog()` reports only what the reader itself discards; moq-audio's + join estimate can then end before its backlog finished arriving. ## Related From ffb0957119ecf87bfdffd299d0bd47b314b11ada Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 12:47:26 -0800 Subject: [PATCH 04/23] feat(moq-net)!: an ordered gap says when what it gave up landed, and the container-reader methods go crate-private Co-Authored-By: Claude Opus 5.5 --- js/net/src/track.ts | 6 +- rs/moq-net/src/model/resume.rs | 13 ++- rs/moq-net/src/model/resume_model.rs | 2 +- rs/moq-net/src/model/subscription.rs | 6 +- rs/moq-net/src/model/track.rs | 104 ++++++++++++++++-- rs/moq-net/tests/integration/range_anchors.rs | 24 ++-- rs/moq-stats/src/produce.rs | 2 +- 7 files changed, 121 insertions(+), 36 deletions(-) diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 37ecd48b37..8181bad256 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -363,9 +363,9 @@ export interface Delay { /** * The target latency, the playout or jitter-buffer delay held behind the live edge. Omitted * means `0`; `null` means not stated, which a peer whose wire cannot carry it (moq-lite - * before 07, moq-transport) sends. A media consumer also waits this long for a missing group, - * since past it the group would play late, and starts an unstated minimum as `0` but waits a - * missing group for {@link Delay.max} instead. + * before 07, moq-transport) sends. It only sets where a reader starts: a missing group is given + * up by {@link Delay.max} alone, whatever the minimum. A media consumer starts an unstated + * minimum as `0`. * * A live join, `{ start: { relative: 0n } }` with no end, starts this far behind the live edge: * at the newest group that began at or before the live edge minus this, so a player's buffer diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 013b8bb34d..de83b190dd 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -1054,6 +1054,13 @@ impl Subscriber { serving.previous_start(start) } + /// When the latest group any copy caches at `keys` landed: a group a replaced copy + /// delivered landed all the same. + pub(super) fn landed(&self, keys: std::ops::Range) -> Option { + let copies: Vec<_> = self.reader().copies.iter().map(|copy| copy.track.clone()).collect(); + copies.iter().filter_map(|track| track.landed(keys.clone())).max() + } + /// Whether the serving copy holds the group at `key`; see [`track::Subscriber::holds`]. pub(super) fn holds(&self, key: u64) -> bool { let serving = self.reader().copies.last().map(|copy| copy.track.clone()); @@ -5677,7 +5684,7 @@ mod test { while let Poll::Ready(Ok(Some(item))) = sub.poll_next_group(&waiter) { items.push(match item { track::Item::Group(group) => format!("group {}", group.start), - track::Item::Gap { range, error } => format!("gap {range:?} ({error})"), + track::Item::Gap { range, error, .. } => format!("gap {range:?} ({error})"), }); } items @@ -5708,7 +5715,7 @@ mod test { while let Poll::Ready(Ok(Some(item))) = sub.poll_next_group(&waiter) { items.push(match item { track::Item::Group(group) => format!("group {}", group.start), - track::Item::Gap { range, error } => format!("gap {range:?} ({error})"), + track::Item::Gap { range, error, .. } => format!("gap {range:?} ({error})"), }); } items @@ -5735,7 +5742,7 @@ mod test { while let Poll::Ready(Ok(Some(item))) = sub.poll_next_group(&waiter) { items.push(match item { track::Item::Group(group) => format!("group {}", group.start), - track::Item::Gap { range, error } => format!("gap {range:?} ({error})"), + track::Item::Gap { range, error, .. } => format!("gap {range:?} ({error})"), }); } items diff --git a/rs/moq-net/src/model/resume_model.rs b/rs/moq-net/src/model/resume_model.rs index 3fb5a1327b..bfe37e1597 100644 --- a/rs/moq-net/src/model/resume_model.rs +++ b/rs/moq-net/src/model/resume_model.rs @@ -669,7 +669,7 @@ impl Model { Cursor::Arrival(sub) => sub.poll_recv_group(&waiter), Cursor::Ordered(sub) => match sub.poll_next_group(&waiter) { // An ordered read reports its drops in turn. - Poll::Ready(Ok(Some(track::Item::Gap { range, error }))) => { + Poll::Ready(Ok(Some(track::Item::Gap { range, error, .. }))) => { reader.report_drop(index, range, &format!("{error:?}"))?; continue; } diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index d4a8d9c041..bcda523ee9 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -67,9 +67,9 @@ pub struct Delay { /// included, and a resumed subscription ignore it, and changing it later moves nothing /// already started. /// - /// A media consumer also waits this long for a missing group, since past it the group would - /// play late, and starts an unstated minimum as zero but waits a missing group for - /// [`Self::max`] instead. The aggregate across subscribers is + /// It only sets where a reader starts: a missing group is given up by [`Self::max`] alone, + /// whatever the minimum. A media consumer starts an unstated minimum as zero. The aggregate + /// across subscribers is /// the largest stated one, which a relay forwards upstream, so the copy it fills reaches /// back for the reader that buffers most; each reader's own minimum then bounds what it /// reads of that copy. moq-lite before 07 and moq-transport cannot carry it. diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 7e240bf9f6..cb8f6460f1 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -1540,6 +1540,11 @@ impl TrackState { } } + /// When the latest group cached at `keys` landed here, `None` when none is. + fn landed(&self, keys: ops::Range) -> Option { + self.lookup.range(keys).map(|(_, slot)| slot.arrived).max() + } + /// Whether the live feed's copy of the group at `key` is cached and still serves it. fn holds_group(&self, key: u64) -> bool { self.lookup @@ -6917,6 +6922,14 @@ impl Consumer { self.with_state(|state| state?.read().previous_start(start)) } + /// When the latest group cached at `keys` landed; see [`Subscriber::landed`]. + pub(super) fn landed(&self, keys: ops::Range) -> Option { + if let Some(serving) = self.serving() { + return serving.landed(keys); + } + self.with_state(|state| state.and_then(|state| state.read().landed(keys))) + } + /// Whether the cache holds the group at `key`; see [`Subscriber::holds`]. pub(super) fn holds(&self, key: u64) -> bool { if let Some(serving) = self.serving() { @@ -10767,8 +10780,7 @@ impl Subscriber { /// /// Pending too while a time start waits on the groups held to place it, since where its /// range starts is unknown until then. - #[cfg_attr(feature = "async", kio::wrapper)] - pub fn poll_start(&mut self, waiter: &kio::Waiter) -> Poll> { + pub(crate) fn poll_start(&mut self, waiter: &kio::Waiter) -> Poll> { let cursor = match &mut self.inner { Inner::Plain(cursor) => cursor, Inner::Resume(resume, _) => return resume.poll_start(waiter), @@ -10841,7 +10853,7 @@ impl Subscriber { /// the groups between are never read, so a reader waiting on each group in key need not /// wait for them. A wire before lite-07 forwards a skip as such a drop. `Some` from there /// while a relative anchor waits on the first group, which can't say yet. - pub fn next_in_range(&self, from: u64) -> Option { + pub(crate) fn next_in_range(&self, from: u64) -> Option { let from = from.max(self.bounds().start); match &self.inner { Inner::Plain(cursor) => { @@ -10861,7 +10873,7 @@ impl Subscriber { /// It knows only the skips recorded so far, so `false` means "not known skipped": a skip /// still in flight reads `false`, as does one a local producer's cache expired below its /// edge. A reader that asks once treats such a group as one with something before it. - pub fn is_skipped(&self, groups: ops::Range) -> bool { + pub(crate) fn is_skipped(&self, groups: ops::Range) -> bool { let Some(last) = groups.end.checked_sub(1).filter(|last| *last >= groups.start) else { return true; }; @@ -10887,6 +10899,15 @@ impl Subscriber { } } + /// When the latest group the track's cache holds at `keys` landed here, `None` when it + /// holds none: a group evicted since leaves no trace. + fn landed(&self, keys: ops::Range) -> Option { + match &self.inner { + Inner::Plain(cursor) => cursor.state.read().landed(keys), + Inner::Resume(resume, _) => resume.landed(keys), + } + } + /// Whether the track's cache holds the live feed's group at `key`, still serving it. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] pub(crate) fn holds(&self, key: u64) -> bool { @@ -11585,6 +11606,10 @@ pub enum Item { range: ops::Range, /// Why: what a read of those groups would have ended with. error: Error, + /// When the latest of those groups landed here, `None` when none did or the cache + /// has since evicted it: a reader timing arrivals learns when what it never read came + /// in, such as a join's backlog skipped as stale. + landed: Option, }, } @@ -11809,7 +11834,8 @@ impl Ordered { self.at = self.at.max(range.end); // Told once: no later drop of these keys is a gap again. self.inner.settle_range(range.clone()); - Item::Gap { range, error } + let landed = self.inner.landed(range.clone()); + Item::Gap { range, error, landed } } /// Narrow subsequent reads to these group keys, as [`Subscriber::with_range`] does: a @@ -11869,8 +11895,15 @@ impl Ordered { self.inner.poll_largest(waiter) } - /// Whether the publisher declared every start in `groups` skipped; see - /// [`Subscriber::is_skipped`]. + /// Whether the publisher declared every start in `groups` skipped, never to be produced + /// (see [`Producer::skip_groups`], which appending does between groups), so no group can + /// arrive there: true for an empty range. Unlike the gaps, this ignores the ranges. A copy + /// knows only the skips its wire forwarded. + /// + /// What tells a reader that nothing lies between two groups, or before a track's first. + /// It knows only the skips recorded so far, so `false` means "not known skipped": a skip + /// still in flight reads `false`, as does one a local producer's cache expired below its + /// edge. A reader that asks once treats such a group as one with something before it. pub fn is_skipped(&self, groups: ops::Range) -> bool { self.inner.is_skipped(groups) } @@ -12921,7 +12954,7 @@ mod test { while let Some(next) = ordered.next_group().now_or_never() { match next { Ok(Some(Item::Group(group))) => items.push(format!("group {}", group.start)), - Ok(Some(Item::Gap { range, error })) => items.push(format!("gap {range:?} ({error})")), + Ok(Some(Item::Gap { range, error, .. })) => items.push(format!("gap {range:?} ({error})")), Ok(None) => return items, Err(err) => { items.push(format!("error ({err})")); @@ -18115,7 +18148,7 @@ mod test { } // The group given up is reported, then the read ends. if given_up { - let Poll::Ready(Ok(Some(Item::Gap { range, error }))) = ordered.poll_next_group(&kio::Waiter::noop()) + let Poll::Ready(Ok(Some(Item::Gap { range, error, .. }))) = ordered.poll_next_group(&kio::Waiter::noop()) else { panic!("the given up group is reported") }; @@ -18549,7 +18582,7 @@ mod test { matches!(next, Err(Error::Timeout)), "{:?}", next.map(|i| i.map(|i| match i { - Item::Gap { range, error } => format!("{range:?} {error}"), + Item::Gap { range, error, .. } => format!("{range:?} {error}"), _ => "group".into(), })) ); @@ -20303,6 +20336,57 @@ mod test { assert_eq!(items(&mut ordered), ["gap 0..1 (old)", "group 1", "group 2", "pending"]); } + /// A gap says when the latest group it gives up landed: one the read passed as stale or + /// that landed behind it, and none for keys that never came. + #[test] + #[cfg(feature = "async")] + fn a_gap_reports_when_what_it_gives_up_landed() { + fn landed(ordered: &mut Ordered) -> Vec<(String, Option)> { + let mut items = Vec::new(); + while let Some(next) = ordered.next_group().now_or_never() { + match next.unwrap() { + Some(Item::Group(group)) => items.push((format!("group {}", group.start), None)), + Some(Item::Gap { range, landed, .. }) => items.push((format!("gap {range:?}"), landed)), + None => break, + } + } + items + } + + let producer = received_producer(None); + let first = producer.broadcast.pool.now_wall(); + put(&producer, 0, None); + put(&producer, 1, None); + elapse(&producer, Duration::from_millis(100)); + put(&producer, 2, None); + let mut ordered = producer + .subscribe(Subscription::default().with_delay(Delay::default().with_max(Duration::from_millis(50)))) + .ordered(); + producer.drop_groups(3..4, StreamError::DeliveryTimeout); + put(&producer, 4, None); + assert_eq!( + landed(&mut ordered), + [ + ("gap 0..1".to_string(), Some(first)), + ("group 1".to_string(), None), + ("group 2".to_string(), None), + ("gap 3..4".to_string(), None), + ("group 4".to_string(), None), + ], + "group 0 landed before it went stale, and group 3 never did" + ); + + let producer = track_producer("test", None); + let mut ordered = producer.subscribe(replay()).ordered(); + put(&producer, 0, None); + put(&producer, 5, None); + assert_eq!(landed(&mut ordered).len(), 2); + elapse(&producer, Duration::from_millis(10)); + let late = producer.broadcast.pool.now_wall(); + put(&producer, 3, None); + assert_eq!(landed(&mut ordered), [("gap 3..4".to_string(), Some(late))]); + } + /// A final drop replaces the part of a stale gap it covers while that gap still waits, so /// the buffered gaps never overlap: the stale run keeps what lies either side of it. #[test] diff --git a/rs/moq-net/tests/integration/range_anchors.rs b/rs/moq-net/tests/integration/range_anchors.rs index fda4f0e699..9b6cd972a7 100644 --- a/rs/moq-net/tests/integration/range_anchors.rs +++ b/rs/moq-net/tests/integration/range_anchors.rs @@ -184,10 +184,10 @@ async fn every_wire_carries_the_live_and_oldest_starts() { /// A publisher counts a relative start back from its newest group, which may hold no frame yet /// and so counts neither in the largest it reports nor where the copy resolves the start. The -/// copy reports the publisher's start instead, where a reader waiting on each group in key -/// starts, rather than one below that never arrives. +/// copy keeps the publisher's start instead, so a reader waiting on each group in key starts +/// there rather than waiting below it for groups that never arrive. #[kio::test] -async fn a_relative_start_reports_the_publishers_start_over_a_frameless_newest_group() { +async fn an_ordered_reader_starts_at_the_publishers_start_over_a_frameless_newest_group() { let publisher = produce_origin(1); let broadcast = publisher .route("room", moq_net::origin::RouteOptions::hidden()) @@ -219,26 +219,20 @@ async fn a_relative_start_reports_the_publishers_start_over_a_frameless_newest_g let mut reader = kio::clock::timeout(TIMEOUT, remote.track("data").unwrap().subscribe(subscription)) .await .expect("subscribe timeout") - .unwrap(); - - let start = kio::clock::timeout(TIMEOUT, reader.start()) - .await - .expect("start timeout"); - assert_eq!( - start, - Some(5), - "the publisher's resolved start, counting its frameless newest group" - ); + .unwrap() + .ordered(); newest .write_frame(moq_net::Timestamp::unscaled(newest.start), &b"x"[..]) .unwrap(); newest.finish().unwrap(); - let group = kio::clock::timeout(TIMEOUT, reader.recv_group()) + // Waiting on each group in key, the reader starts where the publisher resolved the start, + // counting its frameless newest group, with nothing below it owed or given up. + let item = kio::clock::timeout(TIMEOUT, reader.next_group()) .await .expect("group timeout") .unwrap(); - assert_eq!(group.map(|group| group.start), Some(5)); + assert_eq!(item.and_then(moq_net::track::Item::group).map(|group| group.start), Some(5)); let _track = track.finish().unwrap(); } diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index 1e89e76ae7..c700c0eb38 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -1721,7 +1721,7 @@ mod tests { ); match ordered.next_group().await.unwrap().unwrap() { track::Item::Group(group) => assert_eq!(group.start, second.start, "{version}"), - track::Item::Gap { range, error } => panic!("{version}: groups {range:?} given up: {error}"), + track::Item::Gap { range, error, .. } => panic!("{version}: groups {range:?} given up: {error}"), } } } From 92aa113d8381d4d8f9486b07354d8426fd4f1b85 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:17:35 -0800 Subject: [PATCH 05/23] feat(moq-mux)!: the container reader reads through Ordered, which gives holes up by the max delay alone The reader's min-based hole wait, playout-clock rule, and own start wait go away: the ordered reader decides every hole and the start the feed owes. A gap's landing feeds backlog(). Co-Authored-By: Claude Opus 5.5 --- dart/moq/lib/src/aliases.dart | 11 +- go/wrapper/records.go | 13 +- .../kotlin/dev/moq/Aliases.kt | 10 +- py/moq-rs/moq/_records.py | 13 +- rs/libmoq/src/api.rs | 14 +- rs/moq-audio/src/decode/consumer.rs | 190 +-- rs/moq-cli/src/play/video.rs | 3 - rs/moq-ffi/src/consumer.rs | 20 +- rs/moq-mux/src/container/consumer.rs | 1307 +++++++---------- rs/moq-mux/src/container/source.rs | 14 +- .../src/container/ts/passthrough_test.rs | 8 +- rs/moq-mux/src/container/ts/recovery_test.rs | 8 +- rs/moq-mux/src/container/ts/release_test.rs | 8 +- swift/Sources/Moq/Aliases.swift | 10 +- 14 files changed, 598 insertions(+), 1031 deletions(-) diff --git a/dart/moq/lib/src/aliases.dart b/dart/moq/lib/src/aliases.dart index 503094fb57..17a6292dfe 100644 --- a/dart/moq/lib/src/aliases.dart +++ b/dart/moq/lib/src/aliases.dart @@ -158,13 +158,12 @@ extension type Timestamp._(MoqTimestamp _timestamp) implements MoqTimestamp { /// How far behind the live edge a subscriber or decoder plays: where it starts, and how late it may fall. extension type Delay._(MoqDelay _delay) implements MoqDelay { - /// `min` is the target latency, where a cold join starts, and how long a decoder waits for a - /// missing group, and `max` is how far a non-latest group may fall behind before it is skipped. + /// `min` is the target latency, where a cold join starts, and `max` is how far a non-latest + /// group may fall behind before it is skipped, and how long a decoder waits for a missing group. /// Both default to zero, moq-net's default. An explicit null `min` leaves it unstated, which a - /// decoder starts as zero, waiting a missing group for `max` instead (an audio decoder floors it - /// at zero and waits only its playout estimate, which can be larger; a catalog warmup or an audio - /// decoder's playout delay still states a `min` to the publisher), and an - /// explicit null `max` sets no subscriber limit. A `min` past `max` starts at `max`. + /// decoder starts as zero (a catalog warmup or an audio decoder's playout delay still states a + /// `min` to the publisher), and an explicit null `max` sets no subscriber limit. A `min` past + /// `max` starts at `max`. Delay({Duration? min = Duration.zero, Duration? max = Duration.zero}) : this._(MoqDelay(min: min, max: max)); } diff --git a/go/wrapper/records.go b/go/wrapper/records.go index b9ed4d77ea..ef1416daaf 100644 --- a/go/wrapper/records.go +++ b/go/wrapper/records.go @@ -76,15 +76,12 @@ type DatagramInfo = ffi.MoqDatagramInfo // starts at the newest group and skips a superseded group at once. type Delay struct { // Min is the target latency: the playout or jitter-buffer delay held behind the live edge, - // where a cold join starts, and how long a decoder waits for a missing group. A Min past Max - // starts and waits at Max, since the publisher serves nothing older. moq-lite-07 sends it - // upstream. + // where a cold join starts. A missing group is waited for Max alone. A Min past Max starts at + // Max, since the publisher serves nothing older. moq-lite-07 sends it upstream. Min time.Duration - // MinUnstated leaves Min unstated, ignoring it. A decoder starts it as zero and waits a - // missing group for Max (an audio decoder floors it at zero and waits only its playout - // estimate, which can be larger), while the subscription keeps it unstated, so it never - // pulls the aggregate across subscribers to zero, unless a catalog warmup or an audio - // decoder's playout delay states one. + // MinUnstated leaves Min unstated, ignoring it. A decoder starts it as zero, while the + // subscription keeps it unstated, so it never pulls the aggregate across subscribers to + // zero, unless a catalog warmup or an audio decoder's playout delay states one. MinUnstated bool // Max is how far a non-latest group may fall behind the live edge before it is skipped. Zero // skips immediately; a larger value tolerates that much reordering. It caps every range, diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 83a3153180..29794e2af2 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -204,12 +204,10 @@ typealias Subscription = uniffi.moq.MoqSubscription /** * How far behind the live edge a subscriber or decoder plays: `min` is the target latency, where a - * cold join starts, and how long a decoder waits for a missing group, null to leave it unstated (a - * decoder starts it as zero and waits a missing group for `max`, except an audio decoder, which - * floors it at zero and waits only its playout estimate, which can be larger; a catalog warmup or - * an audio decoder's playout delay still states a `min` to the publisher), and `max`, null for no - * subscriber limit, is how far a non-latest group may fall behind before it is skipped. A `min` - * past `max` starts at `max`. + * cold join starts, null to leave it unstated (a decoder starts it as zero; a catalog warmup or an + * audio decoder's playout delay still states a `min` to the publisher), and `max`, null for no + * subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how + * long a decoder waits for a missing group. A `min` past `max` starts at `max`. */ typealias Delay = uniffi.moq.MoqDelay diff --git a/py/moq-rs/moq/_records.py b/py/moq-rs/moq/_records.py index 96d2dfaa2c..fbb7dfd0e3 100644 --- a/py/moq-rs/moq/_records.py +++ b/py/moq-rs/moq/_records.py @@ -229,13 +229,12 @@ class Delay: """How far behind the live edge a subscriber or decoder plays, mirroring moq-net's ``track::Delay``. ``min`` is the target latency: the playout or jitter-buffer delay held behind the - live edge, where a cold join starts, and how long a decoder waits for a missing group. - ``None`` leaves it unstated, which a decoder starts as zero, waiting a missing group for - ``max`` instead (an audio decoder floors it at zero and waits only its playout estimate, - which can be larger), while the subscription keeps it unstated, so it never pulls the - aggregate across subscribers to zero, unless a catalog warmup or an audio decoder's - playout delay states one. A ``min`` past ``max`` starts at ``max``. - ``max`` is how far a non-latest group may fall behind before it is skipped; zero + live edge, where a cold join starts. ``None`` leaves it unstated, which a decoder starts + as zero, while the subscription keeps it unstated, so it never pulls the aggregate across + subscribers to zero, unless a catalog warmup or an audio decoder's playout delay states + one. A ``min`` past ``max`` starts at ``max``. + ``max`` is how far a non-latest group may fall behind before it is skipped, and how long a + decoder waits for a missing group; zero skips at once, ``None`` sets no subscriber limit and leaves the publisher's max age alone to bound it, and it caps every range. diff --git a/rs/libmoq/src/api.rs b/rs/libmoq/src/api.rs index 667ea320dc..6e947922a5 100644 --- a/rs/libmoq/src/api.rs +++ b/rs/libmoq/src/api.rs @@ -826,9 +826,9 @@ pub enum moq_refresh { #[derive(Clone, Copy, Default)] pub struct moq_delay { /// The target latency in microseconds: the playout or jitter-buffer delay held behind - /// the live edge, where a cold join starts, and how long a media consumer waits for a - /// missing group. moq-lite-07 sends it upstream. A `min_us` past the max starts and waits - /// at the max, since the publisher serves nothing older. Ignored when `min_unstated` is set. + /// the live edge, where a cold join starts. A missing group is waited for the max alone. + /// moq-lite-07 sends it upstream. A `min_us` past the max starts at the max, since the + /// publisher serves nothing older. Ignored when `min_unstated` is set. pub min_us: u64, /// How far a non-latest group may fall behind the live edge before it is skipped, in /// microseconds. Zero skips immediately. Enforced by the publisher's cache and by any @@ -837,11 +837,9 @@ pub struct moq_delay { pub max_us: u64, /// No subscriber max: wait on a stalled group as long as the publisher's max age allows. pub max_unlimited: bool, - /// Leave the minimum unstated. A media consumer starts it as zero and waits a missing group - /// for the max instead (an audio decoder floors it at zero and waits only its playout - /// estimate, which can be larger), while the subscription keeps it unstated, so it never - /// pulls the aggregate across subscribers to zero, unless a catalog warmup or an audio - /// decoder's playout delay states one. + /// Leave the minimum unstated. A media consumer starts it as zero, while the subscription + /// keeps it unstated, so it never pulls the aggregate across subscribers to zero, unless a + /// catalog warmup or an audio decoder's playout delay states one. pub min_unstated: bool, } diff --git a/rs/moq-audio/src/decode/consumer.rs b/rs/moq-audio/src/decode/consumer.rs index b23f484e21..be27340a2e 100644 --- a/rs/moq-audio/src/decode/consumer.rs +++ b/rs/moq-audio/src/decode/consumer.rs @@ -84,9 +84,10 @@ pub struct Options { /// [`Delay::min`] is the playout delay: the jitter buffer a player holds ahead of the /// playhead, reported by [`Consumer::delay`], and where a cold join starts. With /// [`estimate`](Self::estimate) it is a floor under the measured delay instead. The - /// subscription states whichever delay the player holds as its min, following the estimate, - /// so a missing group is waited for that long. A `min` beyond the track's max age is refused. [`Delay::max`] is how far a group may fall - /// behind the live edge before the subscription skips it; it adds no buffering. + /// subscription states whichever delay the player holds as its min, following the estimate. + /// A `min` beyond the track's max age is refused. [`Delay::max`] is how far a group may fall + /// behind the live edge before the subscription skips it, and how long a missing group is + /// waited for; it adds no buffering. pub delay: Delay, /// Measure the playout delay from arrival timing, never below [`Delay::min`] (default: /// `true`). `false` holds exactly `delay.min`. @@ -234,8 +235,7 @@ impl Consumer { let warmup = catalog.warmup.unwrap_or_default().min(moq_mux::container::DELAY_MAX); let target = Target::new(options.delay.min.unwrap_or_default(), options.estimate, catalog.jitter); // The subscription states the playout delay as its min from the start, the estimate's cold - // start included: the container places an unfloored start and waits a missing group out - // by it (see `next`). + // start included, so a live join reaches back by it (see `next`). let stated = from_millis(target.millis()); let playout = moq_mux::container::Playout::new(options.delay.with_min(stated), warmup); let track = broadcast.track(hang::catalog::Reference::track(catalog, &name))?; @@ -626,11 +626,10 @@ impl Consumer { /// that arrives meanwhile, or `None` once the track ends. async fn next(&mut self) -> Result, Error> { loop { - // The container waits a missing group out for the subscription's min, so it states the - // playout delay: past it the group would play late, and a reordered backlog's holes - // waited out to the max delay would read as jitter. Late groups that did arrive still - // play up to the max. Checked before every poll, since each arrival and each decoded - // frame can move it; the estimate moves in whole buckets, so each move is one update. + // The subscription states the playout delay as its min, where a join starts; the + // container gives a missing group up by the max delay alone. Checked before every poll, + // since each arrival and each decoded frame can move it; the estimate moves in whole + // buckets, so each move is one update. if self.stated != self.delay() { self.stated = self.delay(); self.track.set_delay(self.stated()); @@ -3021,177 +3020,6 @@ mod tests { ); } - /// A missing packet is waited out for the playout estimate, the delay the subscription states - /// once reading started: not skipped at once, and not held to the max delay, which would hold - /// every packet behind it that long. - #[tokio::test(start_paused = true)] - async fn a_hole_is_waited_out_for_the_playout_estimate() { - use moq_mux::container::Container as _; - - let catalog = hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Pcm, 8_000, 1); - let broadcast = moq_net::broadcast::Info::new().produce(); - let track = broadcast - .create_track("audio", hang::container::track_info(hang::catalog::PRIORITY.audio)) - .unwrap(); - let subscriber = broadcast.consume(); - let max = Duration::from_secs(2); - let options = Options { - delay: Delay { - max: Some(max), - ..Delay::default() - }, - estimate: true, - ..Options::new() - }; - let mut consumer = Consumer::new(&subscriber, &catalog, "audio", options).await.unwrap(); - let container = moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Audio); - let payload: Bytes = vec![0u8; 160 * size_of::()].into(); - let write = |sequence: u64| { - let mut group = track.create_group(moq_net::group::Info { start: sequence }).unwrap(); - let frame = moq_mux::container::Frame { - timestamp: Timestamp::from_micros(sequence * 20_000).unwrap(), - duration: None, - payload: payload.clone(), - keyframe: true, - }; - container.write(&mut group, &[frame]).unwrap(); - group.finish().unwrap(); - }; - - write(0); - next_frame(&mut consumer).await.unwrap().expect("the first packet"); - let estimate = consumer.delay(); - assert!(!estimate.is_zero() && estimate < max, "estimate {estimate:?}"); - - // Packet 1 never arrives; packet 2 lands 20 ms later, as it would have anyway. - tokio::time::advance(Duration::from_millis(20)).await; - write(2); - let since = kio::clock::now(); - assert!(matches!(consumer.read().await.unwrap(), Some(Read::Gap { .. }))); - assert_eq!( - since.elapsed(), - estimate, - "the hole was not waited out for the estimate" - ); - next_frame(&mut consumer) - .await - .unwrap() - .expect("the packet past the hole"); - } - - /// Backpressure holding the first packet encoded doesn't keep the estimate from the hole wait: - /// a hole right after it waits out the estimate, not the configured zero min. - #[tokio::test(start_paused = true)] - async fn a_hole_behind_a_held_first_packet_waits_out_the_estimate() { - use moq_mux::container::Container as _; - - let catalog = hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Pcm, 8_000, 1); - let broadcast = moq_net::broadcast::Info::new().produce(); - let track = broadcast - .create_track("audio", hang::container::track_info(hang::catalog::PRIORITY.audio)) - .unwrap(); - let options = Options { - delay: Delay { - max: Some(Duration::from_secs(2)), - ..Delay::default() - }, - ..Options::new() - }; - let mut consumer = Consumer::new(&broadcast.consume(), &catalog, "audio", options) - .await - .unwrap(); - let backpressure = consumer.backpressure(); - backpressure.set_latency(Duration::from_millis(100), Duration::from_secs(2)); - let container = moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Audio); - let payload: Bytes = vec![0u8; 160 * size_of::()].into(); - let write = |sequence: u64| { - let mut group = track.create_group(moq_net::group::Info { start: sequence }).unwrap(); - let frame = moq_mux::container::Frame { - timestamp: Timestamp::from_micros(sequence * 20_000).unwrap(), - duration: None, - payload: payload.clone(), - keyframe: true, - }; - container.write(&mut group, &[frame]).unwrap(); - group.finish().unwrap(); - }; - - write(0); - assert!(kio::clock::timeout(Duration::ZERO, consumer.read()).await.is_err()); - assert_eq!(consumer.pending.len(), 1, "the first packet is held encoded"); - write(2); - assert!( - kio::clock::timeout(Duration::from_millis(10), consumer.read()) - .await - .is_err() - ); - write(1); - drop(backpressure); - let mut read = Vec::new(); - for _ in 0..3 { - read.push(match consumer.read().await.unwrap().unwrap() { - Read::Frame(frame) => format!("frame {}", Duration::try_from(frame.timestamp).unwrap().as_millis()), - Read::Gap { .. } => "gap".to_string(), - Read::End { .. } => "end".to_string(), - }); - } - assert_eq!(read, ["frame 0", "frame 20", "frame 40"]); - } - - /// The group the publisher says its feed starts at, landing just behind the one after it, is - /// waited out for the estimate, not skipped as a hole at the configured zero min. - #[tokio::test(start_paused = true)] - async fn the_publishers_start_landing_late_waits_out_the_estimate() { - use moq_mux::container::Container as _; - - let catalog = hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Pcm, 8_000, 1); - let broadcast = moq_net::broadcast::Info::new().produce(); - let mut track = broadcast - .create_track("audio", hang::container::track_info(hang::catalog::PRIORITY.audio)) - .unwrap(); - track.start_at(5).unwrap(); - let options = Options { - delay: Delay { - max: Some(Duration::from_secs(2)), - ..Delay::default() - }, - ..Options::new() - }; - let mut consumer = Consumer::new(&broadcast.consume(), &catalog, "audio", options) - .await - .unwrap(); - let container = moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Audio); - let payload: Bytes = vec![0u8; 160 * size_of::()].into(); - let write = |sequence: u64| { - let mut group = track.create_group(moq_net::group::Info { start: sequence }).unwrap(); - let frame = moq_mux::container::Frame { - timestamp: Timestamp::from_micros(sequence * 20_000).unwrap(), - duration: None, - payload: payload.clone(), - keyframe: true, - }; - container.write(&mut group, &[frame]).unwrap(); - group.finish().unwrap(); - }; - - write(6); - assert!( - kio::clock::timeout(Duration::from_millis(10), consumer.read()) - .await - .is_err() - ); - write(5); - let mut read = Vec::new(); - for _ in 0..2 { - read.push(match consumer.read().await.unwrap().unwrap() { - Read::Frame(frame) => format!("frame {}", Duration::try_from(frame.timestamp).unwrap().as_millis()), - Read::Gap { .. } => "gap".to_string(), - Read::End { .. } => "end".to_string(), - }); - } - assert_eq!(read, ["frame 100", "frame 120"]); - } - /// A path that delivers every fifth packet 300 ms late from the start: each lands after /// the cursor passed it and is dropped unread, but was produced after the subscription /// began, so it is no backlog and holds no join open. The estimator measures the path diff --git a/rs/moq-cli/src/play/video.rs b/rs/moq-cli/src/play/video.rs index fa53d483dc..55584dc3be 100644 --- a/rs/moq-cli/src/play/video.rs +++ b/rs/moq-cli/src/play/video.rs @@ -153,9 +153,6 @@ impl Video { // Signalled when a segment behind the front becomes the front. let advanced = tokio::sync::Notify::new(); let role = self.presentation.track(moq_mux::container::Kind::Video); - // A missing group stops being waited for once the clock reaches the group after it, - // whose pictures would replace any late one it held. - track.set_playhead(self.presentation.consume()); let receive = async { // Video has no jitter estimate, so its target is the encoder delay alone, // which holds audio back with it. diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index ab8b8f2c05..41c58db736 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -25,18 +25,16 @@ fn raw_frame(frame: moq_net::frame::Frame) -> MoqFrame { #[derive(Clone, Copy, Debug, uniffi::Record)] pub struct MoqDelay { /// The target latency: the playout or jitter-buffer delay held behind the live edge, where - /// a cold join starts, and how long a media consumer waits for a missing group, or null to - /// leave it unstated. A media consumer starts an unstated minimum as zero and waits a - /// missing group for `max` instead (an audio decoder floors it at zero and waits only its - /// playout estimate, which can be larger), while the subscription keeps it unstated, so it - /// never pulls the aggregate across subscribers to zero, unless a catalog warmup or an - /// audio decoder's playout delay states one. moq-lite-07 sends it upstream. A `min` - /// past `max` starts and waits at `max`, since the publisher serves nothing older. + /// a cold join starts, or null to leave it unstated. A missing group is waited for `max` + /// alone. A media consumer starts an unstated minimum as zero, while the subscription keeps + /// it unstated, so it never pulls the aggregate across subscribers to zero, unless a catalog + /// warmup or an audio decoder's playout delay states one. moq-lite-07 sends it upstream. A + /// `min` past `max` starts at `max`, since the publisher serves nothing older. pub min: Option, - /// How far a non-latest group may fall behind the live edge before it is skipped, or null - /// for no subscriber limit, leaving the publisher's max age alone to bound it. `0` skips - /// immediately; a larger value tolerates that much reordering. It bounds lateness only, - /// never the start. + /// How far a non-latest group may fall behind the live edge before it is skipped, and how + /// long a missing group is waited for, or null for no subscriber limit, leaving the + /// publisher's max age alone to bound it. `0` skips immediately; a larger value tolerates + /// that much reordering. It bounds lateness only, never the start. /// /// Enforced both by the publisher's cache (sent on the wire) and by any local buffering, /// such as [`MoqMediaTrackConsumer`]'s jitter buffer. It caps every range. diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 5520913267..8b2bacb1db 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -52,51 +52,35 @@ pub(crate) enum Item { /// Decode a moq-lite track into a stream of media [`Frame`]s in age-bounded /// presentation order. /// -/// `Consumer` wraps a [`moq_net::track::Subscriber`] and a [`Container`] -/// format implementation, typically -/// [`catalog::hang::Container`](crate::catalog::hang::Container). Yields -/// decoded frames via [`poll_read`](Self::poll_read). +/// `Consumer` reads a [`moq_net::track::Subscriber`] through its +/// [`Ordered`](moq_net::track::Ordered) reader and decodes it with a [`Container`] format +/// implementation, typically [`catalog::hang::Container`](crate::catalog::hang::Container). +/// Yields decoded frames via [`poll_read`](Self::poll_read). /// /// ## Ordering & age skipping /// -/// Groups can arrive on the wire out of order. The consumer always reads frames *within* -/// a group in arrival order, but across groups it advances by start, skipping -/// a stalled or missing group once everything it could still present (bounded by where -/// the next group begins) falls a full budget behind the newest content, the same reach -/// [`Delay::max`] measures. With the default max delay of zero, -/// the consumer skips aggressively: any group that has a newer alternative is dropped. -/// With a non-zero max delay, slow groups are tolerated up to that budget before being -/// skipped. Starts are timestamps, so keys lie between any two groups: a run the publisher -/// declared skipped ([`skip_groups`](moq_net::track::Producer::skip_groups), which appending -/// does) holds no group, and the consumer steps over it at once. Any other run may hold a -/// missing group, which gets the same tolerance: there is no way to tell a stream that lost the -/// delivery race from one the cache evicted, and a reader starved of CPU may take the missing -/// group in the same burst as the ones past it. A missing group is also skipped once waited out -/// for the reader's [`Delay::min`], or the max delay when no min is stated, since past its own -/// buffer a reader would play it late. That wait runs in monotonic time from when the first group -/// past it arrived, so a -/// marker behind a hole is delivered even when the track is paused, and the holes a stalled -/// source left behind expire together rather than one wait each. A reader feeding a playout clock -/// ([`set_playhead`](Self::set_playhead)) also skips a hole once that clock reaches where the -/// first group past it begins. +/// Groups can arrive on the wire out of order. The [`Ordered`](moq_net::track::Ordered) reader +/// hands them out by start and decides every missing one: it waits for a group the track still +/// owes until everything it could still present (bounded by where the next group begins) falls +/// [`Delay::max`] behind the newest content, and reports what it gives up as a gap. With the +/// default max delay of zero it gives up at once: any group that has a newer alternative is +/// dropped. [`Delay::min`] only sets where the reader starts. A run the publisher declared +/// skipped ([`skip_groups`](moq_net::track::Producer::skip_groups), which appending does) holds no +/// group, and the consumer steps over it at once, as it does over keys the subscription's ranges +/// leave out. /// -/// Delivery starts at the floor the -/// [`Subscription::ranges`](moq_net::track::Subscription::ranges) name, raised to where the -/// publisher reports its feed starts ([`poll_start`](moq_net::track::Subscriber::poll_start)) once -/// that resolves, and waits for that group under the budget. The publisher resolves a relative -/// start against a newest group that may hold no frame yet, which a floor resolved here misses. -/// A group that arrives below the start is never delivered: groups arrive newest first, so a -/// cursor taken from the first arrival would drop the older groups the publisher serves. Without -/// either it starts at the lowest group that arrived, waiting for one covering the live edge less -/// [`Delay::min`] plus any [`Playout::warmup`]. Both default to zero, so it waits -/// for nothing. Until the first item, the start also moves past each group whose first frame -/// trails the newest content by more than the max delay plus the warmup, as `@moq/hang`'s -/// consumer measures, unless that group covers the live edge less the min delay and the warmup: -/// a backlog that landed whole would otherwise play from its oldest group, since nothing -/// blocks, and stay that far behind. From there -/// the same budget is the one rule that catches the consumer up to the live edge, and -/// history the publisher no longer serves expires like any other gap, since its reach -/// sits a full budget behind the newest content. +/// Within a group the consumer reads frames in arrival order. A group already handed out that +/// stalls behind newer ones is skipped by the same budget, measured from where the next group +/// begins. +/// +/// Delivery starts at the first group the ordered reader hands out: the publisher resolved a +/// live join [`Delay::min`] plus any [`Playout::warmup`] behind the live edge, and the ordered +/// reader waits for the groups below it that the feed owes. Until the first item, the start also +/// moves past each group whose first frame trails the newest content by more than the max delay +/// plus the warmup, as `@moq/hang`'s consumer measures, unless that group covers the live edge +/// less the min delay and the warmup: a backlog that landed whole would otherwise play from its +/// oldest group, since nothing blocks, and stay that far behind. From there the same budget is +/// the one rule that catches the consumer up to the live edge. /// /// A stalled group is also skipped early, regardless of the max delay budget, once it has /// presented up to where the next group begins. CMAF frames carry a per-sample duration, @@ -112,8 +96,9 @@ pub(crate) enum Item { /// ## Gaps and ends /// /// [`poll_read`](Self::poll_read) yields [`Read::Gap`] wherever media after an item already read -/// will never arrive: a latency skip, an evicted current group, or a hole the publisher did not -/// declare skipped whose boundary is not contiguous within 1 ms. Whatever is lost before the first item is no +/// will never arrive: a latency skip, an evicted current group, or keys between two groups the +/// publisher did not declare skipped whose boundary is not contiguous within 1 ms. A local track +/// owes nothing, so there a discontinuous run is a gap the moment the group past it lands. Whatever is lost before the first item is no /// gap, since nothing was read before it. A marker group (no decodable frames: one empty /// payload, or on a data track one object with no bytes) is the publisher declaring a stop, /// read as [`Read::End`]; nothing is missing. A data track's marker carries no timestamp, so @@ -162,30 +147,19 @@ pub struct Consumer { // How the cursor chooses its first group. startup: Startup, - // How far behind the live edge the reader plays, before the warmup: see `Delay::min`. An unfloored - // start reaches back by it, reading `None` as zero, and a missing group is waited for it, or for - // the max delay when `None`. + // How far behind the live edge the reader plays, before the warmup: see `Delay::min`. The start + // skip keeps the group covering the live edge less it, reading `None` as zero. min_delay: Option, - // How much further than the min delay an unfloored start reaches: see `Playout`. + // How much further than the min delay the start reaches: see `Playout`. warmup: std::time::Duration, - // The point an unfloored start reached back to, the live edge less the reach, once it chose. - target: Option, - - // Where the warming ends: the warmup past the first frame delivered, or past the start's target - // when that is later. `None` before the first frame. + // Where the warming ends: the warmup past the first frame delivered. `None` before it. warmed: Option, // How far we may drift from the live edge before skipping a group. max_delay: std::time::Duration, - // Wakes the reader once the hole at the cursor, or the start's wait, expires. - hole_timer: Option>>, - - // The playout clock this reader feeds, which ends a hole once it reaches the group past it. - playhead: Option, - // The furthest presentation point delivered so far: max(timestamp + duration), endpoints // included. `None` until the first item, so a loss before it is no gap. edge: Option, @@ -206,6 +180,10 @@ pub struct Consumer { // Media went missing since the last item read, so the next one is preceded by a gap. gap: bool, + // The cursor stepped over keys nothing declared skipped, so whether they held media is judged + // once the group it landed on shows where it starts. + unproven: bool, + // The item a gap was returned ahead of. next: Option, @@ -229,10 +207,10 @@ pub struct Consumer { /// Whether a [`Consumer`] still reads its subscription. enum Reader { /// Receiving groups. - Reading(moq_net::track::Subscriber), + Reading(moq_net::track::Ordered), /// The track failed, so no group can arrive. The error is returned once the groups already here /// drain. - Failed(moq_net::track::Subscriber, E), + Failed(moq_net::track::Ordered, E), /// An error ended the reader, which reads as finished from then on. The subscription is /// released, and the publisher's demand with it. Ended, @@ -243,10 +221,8 @@ enum Reader { /// How a [`Consumer`]'s cursor chooses its first group. #[derive(Clone, Copy)] enum Startup { - /// The floor once resolved, raised to the publisher's resolved start, or else the first - /// arrivals. `anchor` is when the newest group had landed as the first one arrived, once such a - /// start waits for its reach. - Earliest { anchor: Option }, + /// The first group the ordered reader hands out. + Earliest, /// The newest group, discarding what arrived below it. Newest, /// The cursor is on its first group or past it. @@ -333,19 +309,10 @@ fn gap(edge: std::time::Duration, resume: Timestamp) -> Result Consumer { /// publisher preserves the same replay window. pub fn new(track: moq_net::track::Subscriber, format: F) -> Self { let subscription = track.subscription(); - // Delivery starts at the subscription floor raised to the publisher's resolved start, - // once that resolves, since a floor resolved here may count below a frameless newest - // group the publisher counted; without either it starts at the first arrivals (see - // poll_read). Either way the age budget is the one rule that catches the cursor up to the live - // edge from there. The budget is clamped to the track's max age, since a group - // the publisher already marks stale can't be waited for and would otherwise - // stall the catch-up by the excess. - let start = track.floor(); + // The budget is clamped to the track's max age, since a group the publisher already marks + // stale can't be waited for and would otherwise stall the catch-up by the excess. let max_delay = subscription .delay .max @@ -430,27 +391,25 @@ impl Consumer { .min(track.info().max_age.unwrap_or(std::time::Duration::MAX)); Self { name: track.name().into(), - reader: Reader::Reading(track), + reader: Reader::Reading(track.ordered()), format, - current: start.unwrap_or(0), + current: 0, pending: VecDeque::new(), received: None, collect_at: 1, memory: Memory::new(DEFAULT_MAX_BUFFER), gated: false, - startup: Startup::Earliest { anchor: None }, + startup: Startup::Earliest, min_delay: subscription.delay.min, warmup: std::time::Duration::ZERO, - target: None, warmed: None, max_delay, - hole_timer: None, - playhead: None, edge: None, start: None, floor: None, presented_end: None, gap: false, + unproven: false, next: None, group: 0, arrived: None, @@ -497,8 +456,8 @@ impl Consumer { /// Backlog is a group at or below the publisher's largest when the subscription began, so it /// was produced before. Its frames were never read, so a caller timing /// [`arrival`](Self::arrival)s learns when a join's backlog finished arriving even when the - /// reader skipped it: one that arrived below the cursor, or below the start, or that a skip - /// passed over. + /// reader skipped it: one the ordered reader gave up as stale or late (see + /// [`Item::Gap`](moq_net::track::Item::Gap)'s `landed`), or one a skip here passed over. pub fn backlog(&self) -> Option { self.backlog } @@ -613,90 +572,45 @@ impl Consumer { } fn poll_event_once(&mut self, waiter: &kio::Waiter) -> Poll, F::Error>> { - // Where the publisher stood as the subscription began, read as soon as its answer resolves - // the start: groups landing later raise the largest past it. A start decided first, or a - // largest not yet in range, takes the first one that is. - if self.joined.is_none() - && (matches!(self.startup, Startup::Started) || self.subscriber().poll_start(waiter).is_ready()) - { - self.join(waiter); - } // Grab any new groups from the track, recording whether the track is finished. let finished = self.poll_read_finish(waiter).is_ready(); - // The floor is raised to where the publisher says its feed starts: it serves nothing - // below, and resolved a relative start against its own latest group, which may hold no - // frame yet and so count neither in the largest it reports nor in a floor resolved here. Waiting below would be waiting for a - // group that won't come. Without a reported start the first arrivals stand in for it - // below, since timestamps can't reveal it: a gap below the served window looks like - // in-flight groups until the budget expires, or forever on a quiet track. Skipping - // stays the budget's job. - if let Startup::Earliest { .. } = self.startup { - let floor = self.subscriber().floor(); - let served = match self.subscriber().poll_start(waiter) { - Poll::Ready(served) => served, - // A finished track resolves no start any more. - Poll::Pending if finished => None, - Poll::Pending => return Poll::Pending, - }; - let start = match (floor, served) { - (Some(floor), Some(served)) => Some(floor.max(served)), - (floor, served) => floor.or(served), - }; - if let Some(start) = start { - self.join(waiter); - let (joined, backlog) = (self.joined, &mut self.backlog); - self.pending.retain(|group| { - let keep = group.start >= start; - if !keep { - Self::skip(joined, backlog, group); - } - keep - }); - self.current = start; - self.startup = Startup::Started; + match self.startup { + // The ordered reader already waited for the groups below its first that the feed owes. + Startup::Earliest => { + if let Some(front) = self.pending.front() { + self.current = front.start; + self.join(waiter); + self.startup = Startup::Started; + } } - } - if !matches!(self.startup, Startup::Started) { - // NOTE: poll_min_timestamp buffers at least one frame per group and - // registers the waiter on the ones still empty. Any wire frame counts, so a - // paused track whose only group is a marker still reads its end. The oldest - // buffers past a full max buffer, so a start is always found. - let any_frame = self.pending.iter_mut().enumerate().any(|(i, group)| { - let _ = group.poll_min_timestamp(waiter, &self.format, i == 0); - !group.is_empty() - }); - if any_frame { - // A live reader takes the newest group, though its frames may still be on their - // way, and discards the older ones the budget also delivers. The publisher's - // largest names it before its stream lands: the backlog's oldest group can arrive - // first, and starting there loses the groups after it once the live edge moves a - // budget past them. - let start = match matches!(self.startup, Startup::Newest) { - true => { - self.poll_largest(waiter); - let back = self.pending.back().expect("a group has a frame").start; - back.max(self.largest.unwrap_or(back)) - } - // The start may still wait for a group earlier than any that arrived. - false => match self.reach_start(waiter, finished) { - Some(start) => start, - None => return Poll::Pending, - }, - }; - self.join(waiter); - // Nothing has been read, so discarding what lies below the start is no gap. - self.discard(|group| group.start < start)?; - self.current = start; - self.startup = Startup::Started; + Startup::Newest => { + // NOTE: poll_min_timestamp buffers at least one frame per group and registers the + // waiter on the ones still empty. Any wire frame counts, so a paused track whose only + // group is a marker still reads its end. The oldest buffers past a full max buffer, so + // a start is always found. + let any_frame = self.pending.iter_mut().enumerate().any(|(i, group)| { + let _ = group.poll_min_timestamp(waiter, &self.format, i == 0); + !group.is_empty() + }); + if any_frame { + // A live reader takes the newest group, though its frames may still be on their + // way, and discards the older ones the budget also delivers. The publisher's + // largest names it before its stream lands. + self.poll_largest(waiter); + let back = self.pending.back().expect("a group has a frame").start; + let start = back.max(self.largest.unwrap_or(back)); + self.join(waiter); + // Nothing has been read, so discarding what lies below the start is no gap. + self.discard(|group| group.start < start)?; + self.current = start; + self.startup = Startup::Started; + } } + Startup::Started => {} } loop { - if self.pending.front().is_none_or(|group| group.start <= self.current) { - self.hole_timer = None; - } - // Until the first item, a backlog that landed whole starts within the budget: nothing // blocks, so the skip below would play it from its oldest group and stay that far behind // for good. The start moves past each group whose first frame trails the newest content @@ -724,9 +638,12 @@ impl Consumer { if self.start.is_none_or(|(start, _)| start != seq) { self.start = Some((seq, ts)); } - // A group covering the target began before it, so the reader warms until the target too. - self.warmed - .get_or_insert_with(|| first.max(self.target.unwrap_or(first)).saturating_add(self.warmup)); + self.warmed.get_or_insert_with(|| first.saturating_add(self.warmup)); + // Keys stepped over below this group lost media unless it continues where the + // content read so far ended. + if std::mem::take(&mut self.unproven) && !pts_contiguous(self.presented_end, first) { + self.lose(); + } let end = first + frame.duration.and_then(time).unwrap_or_default(); self.advance(end); self.group = seq; @@ -734,7 +651,11 @@ impl Consumer { } Poll::Ready(Ok(Some(Event::FrameEnd { end, opens }))) => { self.group = group.group.start; - self.advance(time(end).ok_or(moq_net::Error::TimestampMismatch)?); + let at = time(end).ok_or(moq_net::Error::TimestampMismatch)?; + if std::mem::take(&mut self.unproven) && !pts_contiguous(self.presented_end, at) { + self.lose(); + } + self.advance(at); return Poll::Ready(Ok(Some(Event::FrameEnd { end, opens }))); } Poll::Ready(Ok(Some(event))) => return Poll::Ready(Ok(Some(event))), @@ -787,6 +708,10 @@ impl Consumer { .and_then(|micros| Timestamp::from_micros(micros).ok()) .ok_or(moq_net::Error::Decode(moq_net::DecodeError::BoundsExceeded))?; self.group = group.group.start; + // It has no timestamp to prove the keys stepped over below it held nothing. + if std::mem::take(&mut self.unproven) { + self.lose(); + } return Poll::Ready(Ok(Some(Event::FrameEnd { end, opens: true }))); } // A group its producer finished whole presents up to its end, which can pass @@ -808,22 +733,31 @@ impl Consumer { } } - // Step to the next group at once when nothing lies between: every group ends this - // way, so it comes before the scans below, which walk every pending group. + // The ordered reader hands out a group only once it resolved every key below it: a + // declared skip, one the ranges leave out, or one it gave up on. So the cursor steps to + // the next group at once, past one aborted before anything of it was read. if let Some(front) = self.pending.front_mut() && front.start > self.current - && !front.reapable(waiter) - && self.poll_continues(waiter).0 { - self.current = self.pending.front().unwrap().start; + if front.reapable(waiter) { + self.discard_front(1)?; + continue; + } + let next = front.start; + // Keys the publisher declared skipped hold no group. Any other run lost media unless + // the group past it continues the timeline, which its first frame shows. + if !self.is_skipped(self.current..next) { + self.unproven = true; + } + self.current = next; self.note_group_edge(); continue; } - // Reap later aborts only when delivery blocks. An aborted group beyond a - // sequence gap must not park the reader forever. Buffered frames and faults - // stay in delivery order; the current group has its eviction path above. - // Live groups register the waiter here so a later abort wakes the reader. + // Reap later aborts only when delivery blocks. An aborted group past the current one + // must not hold its buffer. Buffered frames and faults stay in delivery order; the + // current group has its eviction path above. Live groups register the waiter here so a + // later abort wakes the reader. let current = self.current; self.discard(|group| group.start > current && group.reapable(waiter))?; @@ -850,97 +784,6 @@ impl Consumer { let next_group = self.poll_next_group(waiter); let max_timestamp = self.poll_newest(waiter); - // Walk the cursor over missing keys below the first arrived group. - // Groups race on independent QUIC streams (newer ones at higher priority), - // so a buffered higher key proves nothing: the missing one may be - // merely late, and there is no way to tell that from an eviction. The wait, - // in wall time since a later group arrived, is the gate. Media buffered past - // the hole proves it stale only past the max delay, not the shorter wait: a - // reader starved of CPU takes in a burst of groups at once, and the missing - // one may land in the same burst. A finished track closes - // the gap outright, since no new group can arrive. That proof covers only - // keys that never arrived: finishing the track ends new groups, not - // the frames still flowing on ones already open, so arrived groups are - // settled below by their own FIN, abort, or the budget. Keys the - // subscriber's own ranges exclude never arrive either, so a gap between two - // ranges closes at once. - if let Some(front) = self.pending.front() - && front.start > self.current - { - let sequence = front.start; - // Late since a group past it arrived, not since the cursor got here: once the edge - // stops moving, a backlog with every other group missing would otherwise drain one - // budget per hole, which reads downstream as a source far slower than real time. - let since = front.late; - let wait = self.hole_wait(); - let elapsed = since.elapsed() >= wait; - let aged = next_group.is_some_and(|(_, start)| max_timestamp.saturating_sub(start) >= self.max_delay); - // Once the playout clock reaches where the first group past it begins, that group's - // media is due, and it replaces whatever the missing one held. - let schedule = self.playhead.as_ref().map(|playhead| playhead.schedule()); - let reached = schedule - .zip(next_group) - .and_then(|(schedule, (_, start))| schedule.due(Timestamp::try_from(start).ok()?)); - let passed = reached.is_some_and(|at| at <= kio::clock::now()); - let excluded = match &self.reader { - Reader::Reading(track) | Reader::Failed(track, _) => { - track.next_in_range(self.current).is_none_or(|next| next >= sequence) - } - Reader::Ended | Reader::Lost => false, - }; - // The groups past the hole already fill the max buffer, so it can hold none of what - // would arrive into it. - let full = self.memory.full(); - let (continues, first) = self.poll_continues(waiter); - // A run nothing will fill, such as one dropped over a wire that can't say it was - // skipped, still waits for the group past it to show where it starts: stepping - // before that frame lands would call a contiguous timeline a loss. - let excluded = excluded && first.is_ready(); - if finished || elapsed || aged || passed || excluded || full || continues { - let _ = self.pending.front_mut().unwrap().look_all(waiter, &self.format); - let front = self.pending.front_mut().unwrap(); - let empty = front.is_empty() && matches!(front.group.poll_finished(waiter), Poll::Ready(Ok(_))); - let front = self.pending.front().unwrap(); - let next_start = front.min_timestamp.and_then(time).or(front.max_end); - // A data marker has no timestamp to prove continuity across the hole. - if !empty && !continues && next_start.is_none_or(|start| !pts_contiguous(self.presented_end, start)) - { - self.lose(); - } - tracing::debug!( - old = self.current, - new = sequence, - finished, - elapsed, - aged, - passed, - excluded, - full, - continues, - "skipping missing groups" - ); - self.current = sequence; - self.note_group_edge(); - continue; - } - // The wait expires, or the clock reaches the group past the hole, whichever is first. - let expires = since.checked_add(wait); - if let Some(wake) = expires.into_iter().chain(reached).min() { - let timer = self - .hole_timer - .get_or_insert_with(|| Box::pin(kio::clock::sleep_until(wake))); - timer.as_mut().reset(wake); - let _ = waiter.poll_future(timer.as_mut()); - } - // A clock that jumps, re-anchoring or changing its delay, moves when it gets there. - if let Some((playhead, schedule)) = self.playhead.as_ref().zip(schedule) - && next_group.is_some() - && playhead.poll_changed(waiter, schedule).is_ready() - { - continue; - } - } - // The current group is blocking. Everything it could still present ends // where the next group begins, so skip once that reach falls a full budget // behind the newest content, the same measure as a missing group. Its own @@ -990,12 +833,6 @@ impl Consumer { } } - // How long a missing group is waited for: the min delay, the reader's own buffer, since past it the - // group would play late, or the max delay when the reader states no min. Never past the max. - fn hole_wait(&self) -> std::time::Duration { - self.min_delay.map_or(self.max_delay, |min| min.min(self.max_delay)) - } - // Before the first item, move the cursor past its group when that group's first frame trails the // newest content by more than the budget and the next group still starts at or before the live // edge less the reach, so it is not what the start covers, and repeat for the group it lands on. @@ -1112,78 +949,14 @@ impl Consumer { Ok(()) } - // Where an unfloored start begins, the lowest group that arrived, or `None` while it waits for - // one covering the reach (see `Playout`), registering the waiter on whatever would end the wait. - fn reach_start(&mut self, waiter: &kio::Waiter, finished: bool) -> Option { - let lowest = self.pending.front().expect("a group has a frame").start; - // The publisher serves nothing older than the max delay, so a min delay past it reaches no - // further. - let reach = self - .min_delay - .unwrap_or_default() - .min(self.max_delay) - .saturating_add(self.warmup); - // The newest group covers a zero reach. - if reach.is_zero() { - return Some(lowest); - } - - // The live edge, the newest content, read once looking ahead has decoded it. - let mut edge = None; - for group in self.pending.iter_mut().rev() { - if let Poll::Ready(Some(ts)) = group.poll_max_timestamp(waiter, &self.format) { - edge = time(ts); - break; - } - } - // Untimed content, such as a marker, has no edge to reach back from. - let Some(edge) = edge else { - return Some(lowest); - }; - let target = edge.saturating_sub(reach); - self.target = Some(target); - let covered = self - .pending - .iter() - .any(|group| group.min_timestamp.and_then(time).is_some_and(|start| start <= target)); - if covered { - return Some(lowest); - } - - // Nothing earlier can arrive or be held. Looking ahead may have filled the max buffer, - // which then registers no waiter on more frames. - let floor = self.subscriber().next_in_range(0); - if finished || self.memory.full() || floor.is_none_or(|floor| lowest <= floor) { - return Some(lowest); - } - - // Wall time: the newest group landed a whole reach ago. - let landed = self.pending.back().expect("a group has a frame").group.arrived(); - let arrived = match &mut self.startup { - Startup::Earliest { anchor } => *anchor.get_or_insert(landed), - Startup::Newest | Startup::Started => landed, - }; - if arrived.elapsed() >= reach { - return Some(lowest); - } - if let Some(wake) = arrived.checked_add(reach) { - let timer = self - .hole_timer - .get_or_insert_with(|| Box::pin(kio::clock::sleep_until(wake))); - timer.as_mut().reset(wake); - let _ = waiter.poll_future(timer.as_mut()); - } - None - } - - // The reader has not yet read past the start's target and the warmup past its first frame. + // The reader has not yet read the warmup past its first frame. fn warming(&self) -> bool { self.warmed .is_none_or(|warmed| self.edge.is_none_or(|edge| edge < warmed)) } // The subscription, which only an error ending the reader drops. - fn subscriber(&mut self) -> &mut moq_net::track::Subscriber { + fn subscriber(&mut self) -> &mut moq_net::track::Ordered { match &mut self.reader { Reader::Reading(track) | Reader::Failed(track, _) => track, Reader::Ended | Reader::Lost => unreachable!("read only until an error ends the reader"), @@ -1193,6 +966,7 @@ impl Consumer { // Media this reader will never deliver: the next item is preceded by a gap. fn lose(&mut self) { self.gap = true; + self.unproven = false; } fn advance(&mut self, end: std::time::Duration) { @@ -1201,23 +975,7 @@ impl Consumer { } } - // Whether the cursor may step to the first pending group past it without waiting or losing - // anything, and that group's first-timestamp poll. Group starts are timestamps, so the key - // past the cursor is rarely the next group's. The publisher declares the starts it never - // produced, so a run it skipped holds no group to wait for or lose. Nor does one ending at a - // group whose first frame, or a marker's bound, continues where the content read so far ended. - fn poll_continues(&mut self, waiter: &kio::Waiter) -> (bool, Poll>) { - let next = self.pending.front().expect("a group past the cursor").start; - let skipped = self.is_skipped(self.current..next); - let front = self.pending.front_mut().unwrap(); - let first = front.poll_min_timestamp(waiter, &self.format, false); - let start = front.min_timestamp.and_then(time).or(front.max_end); - let continues = skipped || start.is_some_and(|start| pts_contiguous(self.presented_end, start)); - (continues, first) - } - fn note_group_edge(&mut self) { - self.hole_timer = None; if let Some((_, ts)) = self.start { self.floor = Some(ts); } @@ -1254,13 +1012,12 @@ impl Consumer { return Poll::Ready(()); } loop { - // Past the max buffer, later groups wait in the transport's cache. A missing group at the - // cursor then expires like one past the budget: see the hole below. + // Past the max buffer, later groups wait in the transport's cache. if self.pull_gated() { self.gated = true; return Poll::Pending; } - let next = match ready!(self.subscriber().poll_recv_group(waiter)) { + let next = match ready!(self.subscriber().poll_next_group(waiter)) { Ok(next) => next, Err(err) => { if let Reader::Reading(track) = std::mem::replace(&mut self.reader, Reader::Ended) { @@ -1269,34 +1026,33 @@ impl Consumer { return Poll::Ready(()); } }; + // Any item shows the subscription answered, so the publisher's largest now says where it + // stood as it began. + self.join(waiter); - let Some(group) = next else { + let group = match next { + Some(moq_net::track::Item::Group(group)) => group, + // What the ordered reader gave up was never read, so it is backlog when produced before + // the subscription. + Some(moq_net::track::Item::Gap { range, landed, .. }) => { + if self.joined.is_some_and(|joined| range.start <= joined) { + self.backlog = self.backlog.max(landed); + } + continue; + } // Track is finished. - return Poll::Ready(()); + None => return Poll::Ready(()), }; - let mut reader = GroupBuffer::new(group, self.memory.clone()); - let sequence = reader.group.start; - - if sequence < self.current { - tracing::debug!(old = ?sequence, current = ?self.current, "skipping old group"); + let reader = GroupBuffer::new(group, self.memory.clone()); + // Groups come in key order, so only a live start, which jumped to the newest, leaves any + // below the cursor. + if reader.start < self.current { + tracing::debug!(old = ?reader.start, current = ?self.current, "skipping old group"); Self::skip(self.joined, &mut self.backlog, &reader); continue; } - - let idx = self.pending.partition_point(|g| g.group.start < reader.group.start); - // Each group is late from the earliest arrival among it and the groups past it. A drained - // backlog comes in sequence order, not arrival order, so an arrival can lower those below it. - if let Some(next) = self.pending.get(idx) { - reader.late = reader.late.min(next.late); - } - for below in self.pending.range_mut(..idx).rev() { - if below.late <= reader.late { - break; - } - below.late = reader.late; - } - self.pending.insert(idx, reader); + self.pending.push_back(reader); self.collect_pending(waiter); } } @@ -1406,7 +1162,7 @@ impl Consumer { } /// Whether the publisher declared every group start in `groups` skipped, so no group lies - /// there; see [`moq_net::track::Subscriber::is_skipped`]. A group with nothing possible + /// there; see [`moq_net::track::Ordered::is_skipped`]. A group with nothing possible /// below it is its track's first. False once an error ended the reader. pub fn is_skipped(&self, groups: std::ops::Range) -> bool { match &self.reader { @@ -1419,9 +1175,9 @@ impl Consumer { /// /// [`Delay::max`] is clamped to the track's max age like [`new`](Self::new). The /// subscription keeps the requested delay verbatim, plus any [`Playout::warmup`]. A - /// [`Delay::min`] moves a start still to come, reading an unstated one as zero, and how long - /// a missing group is waited for from now on. A min past the max starts at the max, since the - /// publisher serves nothing older. + /// [`Delay::min`] moves a start still to come, reading an unstated one as zero. A min past the + /// max starts at the max, since the publisher serves nothing older. A missing group is given + /// up by the subscription's max alone. pub fn set_delay(&mut self, delay: Delay) { self.set_skip(delay); // An ended reader has no subscription left to tune. @@ -1458,15 +1214,6 @@ impl Consumer { } } - /// Skip a missing group once `playhead`, the playout clock this reader feeds, reaches where the - /// first group past it begins (default: no clock), rather than waiting out the whole - /// [`Delay::min`]. Suits media where a late item is worthless once the next is due, such as - /// video, whose next due picture replaces it. Audio that still plays late, catching up by time - /// stretch, keeps the wait. - pub fn set_playhead(&mut self, playhead: crate::sync::Consumer) { - self.playhead = Some(playhead); - } - /// Cap the bytes of decoded frames held ahead of the reader, counting the wire payload they share /// and each frame's and group's own overhead (default [`DEFAULT_MAX_BUFFER`]). Past it the consumer stops decoding ahead /// and pulling new groups; see the type docs. @@ -1497,10 +1244,6 @@ enum Failure { struct GroupBuffer { group: moq_net::group::Consumer, - // From then on, any sequence missing below this group is late: the earliest arrival among it - // and the groups past it, kept on insert so a hole reads it without scanning the backlog. - late: kio::clock::Instant, - // The current frame index within the group. index: usize, @@ -1547,8 +1290,6 @@ enum Content { impl GroupBuffer { fn new(group: moq_net::group::Consumer, memory: Memory) -> Self { let mut this = Self { - // When the group landed, not when it was pulled: a full max buffer delays pulls. - late: group.arrived(), group, index: 0, content: Content::Empty, @@ -1831,7 +1572,7 @@ mod tests { use crate::catalog::hang::Container; /// The subscription a consumer still reads. - fn subscribed(consumer: &Consumer) -> &moq_net::track::Subscriber { + fn subscribed(consumer: &Consumer) -> &moq_net::track::Ordered { match &consumer.reader { Reader::Reading(track) | Reader::Failed(track, _) => track, Reader::Ended | Reader::Lost => panic!("the reader ended"), @@ -1841,6 +1582,78 @@ mod tests { use bytes::Bytes; + /// A track published at one origin and read from another over a lite-07 session: a received + /// copy, whose ordered reader waits for a group the feed still owes, as a player's does. A + /// locally produced track owes nothing, so a hole there is stepped over at once. + struct Received { + track: moq_net::track::Producer, + remote: moq_net::broadcast::Consumer, + _route: moq_net::origin::Route, + _link: crate::container::test_util::Link, + } + + impl Received { + async fn new() -> Self { + Self::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await + } + + async fn with_info(info: moq_net::track::Info) -> Self { + use crate::container::test_util::{Link, origin}; + + let (publisher, subscriber) = (origin(1), origin(2)); + let link = Link::new(&publisher, &subscriber).await; + let route = publisher + .route("room", moq_net::origin::RouteOptions::hidden()) + .unwrap(); + let track = route.broadcast().create_track("data", info).unwrap(); + route.announce().unwrap(); + let remote = subscriber + .consume() + .follow("room", None) + .unwrap() + .available() + .await + .unwrap() + .unwrap(); + Self { + track, + remote, + _route: route, + _link: link, + } + } + + /// Subscribe to the copy at `delay`. + async fn subscribe(&self, delay: Delay) -> moq_net::track::Subscriber { + self.subscribe_with(moq_net::track::Subscription::default().with_delay(delay)) + .await + } + + async fn subscribe_with(&self, subscription: moq_net::track::Subscription) -> moq_net::track::Subscriber { + self.remote + .track("data") + .unwrap() + .subscribe(subscription) + .await + .unwrap() + } + + /// A reader of the copy at `delay`. + async fn reader(&self, delay: Delay) -> Consumer { + Consumer::new( + self.subscribe(delay).await, + Container::Legacy(crate::container::Kind::Data), + ) + } + } + + /// Let the session carry what was written to the copy, without moving the clock. + async fn carried() { + for _ in 0..64 { + tokio::task::yield_now().await; + } + } + /// A millisecond track: the frames written here carry timestamps. It sets no publisher max /// age, so the test's delay budget alone decides which superseded groups are read. fn timed() -> moq_net::track::Info { @@ -2124,19 +1937,19 @@ mod tests { assert_eq!(items(&mut reader).await, ["0", "10000", "gap 10000+0", "end 10000"]); } + /// A missing group before a marker on a paused track is given up by the max delay's wall clock, + /// since no newer media will age it out, and wakes the reader when it is. #[tokio::test(start_paused = true)] async fn a_sequence_hole_before_a_paused_data_marker_expires() { for budget in [Duration::ZERO, Duration::from_millis(100)] { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( - moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), - ); - let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 0, &[ts(10_000)]); + let mut copy = Received::new().await; + let mut reader = copy.reader(Delay::default().with_min(None).with_max(budget)).await; + write_group(&mut copy.track, 0, &[ts(10_000)]); assert!(matches!(reader.read().await.unwrap(), Some(Read::Frame { .. }))); - let mut marker = track.create_group(moq_net::group::Info { start: 2 }).unwrap(); + let mut marker = copy.track.create_group(moq_net::group::Info { start: 2 }).unwrap(); marker.write_frame(ts(20_000), Bytes::new()).unwrap(); marker.finish().unwrap(); + carried().await; if !budget.is_zero() { assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); @@ -2151,38 +1964,40 @@ mod tests { assert_eq!( before.elapsed(), Duration::from_millis(1), - "the hole timer wakes the reader" + "the budget wakes the reader" ); } else { - assert!(matches!( - reader.poll_read(&kio::Waiter::noop()), - Poll::Ready(Ok(Some(Read::Gap { .. }))) - )); + let read = kio::clock::timeout(Duration::ZERO, reader.read()).await.unwrap().unwrap(); + assert!(matches!(read, Some(Read::Gap { .. }))); } assert!( matches!(reader.poll_read(&kio::Waiter::noop()), Poll::Ready(Ok(Some(Read::End { group: 2, end }))) if end == ts(10_000)) ); - let _track = track.finish().unwrap(); + let _track = copy.track.finish().unwrap(); } } #[tokio::test(start_paused = true)] async fn a_filled_sequence_hole_gives_the_next_hole_a_fresh_budget() { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default() .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(100))), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 0, .. }) )); - write_group(&mut track, 2, &[ts(20_000)]); + write_group(&mut copy.track, 2, &[ts(20_000)]); + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(Duration::from_millis(50)).await; - write_group(&mut track, 1, &[ts(10_000)]); + carried().await; + write_group(&mut copy.track, 1, &[ts(10_000)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 1, .. }) @@ -2191,19 +2006,22 @@ mod tests { reader.read().await.unwrap(), Some(Read::Frame { group: 2, .. }) )); - let mut marker = track.create_group(moq_net::group::Info { start: 4 }).unwrap(); + let mut marker = copy.track.create_group(moq_net::group::Info { start: 4 }).unwrap(); marker.write_frame(ts(30_000), Bytes::new()).unwrap(); marker.finish().unwrap(); + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(Duration::from_millis(99)).await; + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(Duration::from_millis(1)).await; + carried().await; assert!(matches!( reader.poll_read(&kio::Waiter::noop()), Poll::Ready(Ok(Some(Read::Gap { .. }))) )); assert!(matches!(reader.read().await.unwrap(), Some(Read::End { group: 4, .. }))); - let _track = track.finish().unwrap(); + let _track = copy.track.finish().unwrap(); } /// The arrival is when a frame landed, not when it was read, and a break keeps the last one. @@ -2324,6 +2142,27 @@ mod tests { assert_eq!(reader.backlog(), Some(landed)); } + /// A join's backlog the ordered reader gives up as stale, never handing it to this reader, still + /// reports when it landed: its gap carries the landing. + #[tokio::test(start_paused = true)] + async fn a_stale_backlog_the_ordered_reader_gives_up_reports_when_it_landed() { + let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let landed = kio::clock::now(); + write_group(&mut track, 0, &[ts(0)]); + write_group(&mut track, 1, &[ts(20_000)]); + tokio::time::advance(Duration::from_millis(100)).await; + write_group(&mut track, 2, &[ts(40_000)]); + let subscriber = track.subscribe( + moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_millis(50))), + ); + let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Audio)); + assert!(matches!( + reader.read().await.unwrap(), + Some(Read::Frame { group: 1, .. }) + )); + assert_eq!(reader.backlog(), Some(landed), "group 0 went stale unread"); + } + /// The join's boundary is where the publisher stood as it answered, not the newer largest by the /// time a live start found a frame to start on. #[tokio::test(start_paused = true)] @@ -2364,25 +2203,30 @@ mod tests { } /// The holes a source left behind when it stopped expire together, a budget after the groups - /// past them arrived, rather than one budget each as the cursor reaches them. + /// past them arrived, rather than one budget each as the cursor reaches them. The groups that + /// waited behind them that long are a budget late too, superseded that long ago, so only the + /// newest is read. #[tokio::test(start_paused = true)] async fn holes_behind_a_stalled_source_share_one_budget() { let budget = Duration::from_millis(100); - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 0, .. }) )); for sequence in [2, 4, 6] { - write_group(&mut track, sequence, &[ts(sequence * 10_000)]); + write_group(&mut copy.track, sequence, &[ts(sequence * 10_000)]); + carried().await; } assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(budget).await; + carried().await; let mut read = Vec::new(); while let Poll::Ready(Ok(Some(item))) = reader.poll_read(&kio::Waiter::noop()) { read.push(match item { @@ -2391,61 +2235,74 @@ mod tests { Read::End { .. } => "end".to_string(), }); } - assert_eq!(read, ["gap", "frame 2", "gap", "frame 4", "gap", "frame 6"]); + assert_eq!(read, ["gap", "frame 6"]); } - /// A missing group is waited for the stated min delay, capped at the max, or the max delay when - /// no min is stated. + /// A missing group is waited for the max delay, whatever the min delay: the min only sets + /// where the reader starts. #[tokio::test(start_paused = true)] - async fn a_hole_is_waited_for_the_min_delay_or_the_max_when_none_is_stated() { + async fn a_hole_is_waited_for_the_max_delay_whatever_the_min() { let max = Duration::from_secs(1); - for (min, waited) in [ - (Some(Duration::ZERO), Duration::ZERO), - (Some(Duration::from_millis(100)), Duration::from_millis(100)), - (Some(Duration::from_secs(5)), max), - (None, max), + for min in [ + Some(Duration::ZERO), + Some(Duration::from_millis(100)), + Some(Duration::from_secs(5)), + None, ] { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default().with_delay(Delay::default().with_min(min).with_max(max)), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 0, .. }) )); - // Group 1 never arrives, and group 2 starts right after it, so only the wait ends it. - write_group(&mut track, 2, &[ts(40_000)]); + // Group 1 never arrives, and group 2 starts right after it, so only the wall clock ends it. + write_group(&mut copy.track, 2, &[ts(40_000)]); + carried().await; let since = kio::clock::now(); assert!(matches!(reader.read().await.unwrap(), Some(Read::Gap { .. }))); - assert_eq!(since.elapsed(), waited, "min {min:?}"); + assert_eq!(since.elapsed(), max, "min {min:?}"); } } - /// A burst of later groups spanning past the wait doesn't age out a missing group that lands in - /// the same burst: a reader starved of CPU takes in every stream at once. + /// A burst of later groups spanning past the min delay doesn't age out a missing group that + /// lands in the same burst, even once the min delay has passed: a reader starved of CPU takes in + /// every stream at once, and only the max delay gives a hole up. #[tokio::test(start_paused = true)] - async fn a_burst_past_the_wait_does_not_age_out_a_hole() { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + async fn a_burst_past_the_min_delay_does_not_age_out_a_hole() { + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); reader.set_delay( Delay::default() .with_min(Duration::from_millis(100)) .with_max(Duration::from_secs(1)), ); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 0, .. }) )); - write_group(&mut track, 2, &[ts(40_000), ts(200_000)]); - write_group(&mut track, 3, &[ts(300_000)]); + write_group(&mut copy.track, 2, &[ts(40_000), ts(200_000)]); + carried().await; + write_group(&mut copy.track, 3, &[ts(300_000)]); + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); - write_group(&mut track, 1, &[ts(20_000)]); + tokio::time::advance(Duration::from_millis(100)).await; + carried().await; + assert!( + reader.poll_read(&kio::Waiter::noop()).is_pending(), + "the min delay gives nothing up" + ); + write_group(&mut copy.track, 1, &[ts(20_000)]); + carried().await; let mut read = Vec::new(); while let Poll::Ready(Ok(Some(item))) = reader.poll_read(&kio::Waiter::noop()) { read.push(match item { @@ -2457,27 +2314,32 @@ mod tests { assert_eq!(read, ["frame 1", "frame 2", "frame 2", "frame 3"]); } - /// A group that arrives late below an earlier one inherits that earlier arrival: the holes - /// around it were already observable then. + /// A group that arrives late below an earlier one is late from that earlier arrival, as are the + /// holes around it, so they all expire a budget after it. #[tokio::test(start_paused = true)] async fn a_hole_is_late_from_the_earliest_arrival_past_it() { let budget = Duration::from_millis(100); - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; assert!(matches!( reader.read().await.unwrap(), Some(Read::Frame { group: 0, .. }) )); - write_group(&mut track, 4, &[ts(40_000)]); + write_group(&mut copy.track, 4, &[ts(40_000)]); + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(budget / 2).await; - write_group(&mut track, 2, &[ts(20_000)]); + carried().await; + write_group(&mut copy.track, 2, &[ts(20_000)]); + carried().await; assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); tokio::time::advance(budget / 2).await; + carried().await; let mut read = Vec::new(); while let Poll::Ready(Ok(Some(item))) = reader.poll_read(&kio::Waiter::noop()) { read.push(match item { @@ -2486,17 +2348,17 @@ mod tests { Read::End { .. } => "end".to_string(), }); } - assert_eq!(read, ["gap", "frame 2", "gap", "frame 4"]); + assert_eq!(read, ["gap", "frame 4"]); } - /// A player waits on a hole for its playout delay, while groups that did arrive play until the - /// max delay. + /// A player waits on a hole for the max delay, as it does on a group that did arrive, whatever + /// its playout delay. #[tokio::test(start_paused = true)] - async fn a_hole_is_waited_for_the_min_delay_while_an_arrived_group_plays_up_to_the_max_delay() { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let subscriber = track.subscribe( + async fn a_hole_and_an_arrived_group_both_wait_up_to_the_max_delay() { + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; + let subscriber = copy.subscribe_with( moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), - ); + ).await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); reader.set_delay( Delay::default() @@ -2516,7 +2378,7 @@ mod tests { }; // A stalled group 500 ms behind the newest content is still within the max delay. - let mut stalled = track.create_group(moq_net::group::Info { start: 0 }).unwrap(); + let mut stalled = copy.track.create_group(moq_net::group::Info { start: 0 }).unwrap(); let frame = Frame { timestamp: ts(0), payload: Bytes::from_static(&[0xDE, 0xAD]), @@ -2526,29 +2388,43 @@ mod tests { Container::Legacy(crate::container::Kind::Data) .write(&mut stalled, &[frame]) .unwrap(); - write_group(&mut track, 1, &[ts(500_000)]); + write_group(&mut copy.track, 1, &[ts(500_000)]); + carried().await; tokio::time::advance(Duration::from_millis(500)).await; + carried().await; assert_eq!(drain(&mut reader), ["frame 0"]); stalled.finish().unwrap(); + carried().await; assert_eq!(drain(&mut reader), ["frame 1"]); - // A missing group expires the wait after the first group past it arrived. - write_group(&mut track, 3, &[ts(520_000)]); - tokio::time::advance(Duration::from_millis(99)).await; + // A missing group is given up the max delay after the first group past it arrived, not the + // min delay. + write_group(&mut copy.track, 3, &[ts(520_000)]); + carried().await; + tokio::time::advance(Duration::from_millis(999)).await; + carried().await; assert!(drain(&mut reader).is_empty()); tokio::time::advance(Duration::from_millis(1)).await; + carried().await; assert_eq!(drain(&mut reader), ["gap", "frame 3"]); - // Content buffered past the wait doesn't skip it early: a burst may still carry the group. - write_group(&mut track, 5, &[ts(700_000)]); - write_group(&mut track, 6, &[ts(800_000)]); - assert!(drain(&mut reader).is_empty()); + // Content buffered past the hole doesn't skip it early: a burst may still carry the group. + write_group(&mut copy.track, 5, &[ts(700_000)]); + carried().await; + write_group(&mut copy.track, 6, &[ts(800_000)]); + carried().await; tokio::time::advance(Duration::from_millis(100)).await; - assert_eq!(drain(&mut reader), ["gap", "frame 5", "frame 6"]); + carried().await; + assert!(drain(&mut reader).is_empty()); + write_group(&mut copy.track, 4, &[ts(600_000)]); + carried().await; + assert_eq!(drain(&mut reader), ["frame 4", "frame 5", "frame 6"]); // It skips at once when the newest content is already the max delay past the group after it. - write_group(&mut track, 8, &[ts(900_000)]); - write_group(&mut track, 9, &[ts(1_900_000)]); + write_group(&mut copy.track, 8, &[ts(900_000)]); + carried().await; + write_group(&mut copy.track, 9, &[ts(1_900_000)]); + carried().await; assert_eq!(drain(&mut reader), ["gap", "frame 8", "frame 9"]); } @@ -2565,133 +2441,38 @@ mod tests { read } - /// A waker that records whether it was woken. - #[derive(Default)] - struct Woken(std::sync::atomic::AtomicBool); - - impl std::task::Wake for Woken { - fn wake(self: Arc) { - self.0.store(true, Ordering::SeqCst); - } - } - - impl Woken { - fn waiter(self: &Arc) -> kio::Waiter { - kio::Waiter::new(std::task::Waker::from(self.clone())) - } - - fn take(&self) -> bool { - self.0.swap(false, Ordering::SeqCst) - } - } - - /// A reader that delivered group 0, waiting 500 ms on a hole within a 2 s max delay, and fed - /// to `playhead` when given. - fn holed(track: &mut moq_net::track::Producer, playhead: Option) -> Consumer { - let subscriber = track.subscribe( - moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(2))), - ); - let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - reader.set_delay( - Delay::default() - .with_min(Duration::from_millis(500)) - .with_max(Duration::from_secs(2)), - ); - if let Some(playhead) = playhead { - reader.set_playhead(playhead); - } - write_group(track, 0, &[ts(0)]); - assert_eq!(drain_with(&mut reader, &kio::Waiter::noop()), ["frame 0"]); - reader - } - - /// A hole ends once the playout clock reaches where the group past it begins, rather than at - /// the wait, which still ends one the clock trails or a reader without a clock. - #[tokio::test(start_paused = true)] - async fn a_hole_ends_once_the_playout_clock_reaches_the_group_past_it() { - // The media presenting as the group past the hole arrives, and how long until the hole ends. - for (presenting, ends) in [(Some(900), 100), (Some(0), 500), (None, 500)] { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let clock = crate::sync::Producer::new(Duration::ZERO); - // Another track drives the clock, which presents `presenting` ms of media now. - let video = clock.track(crate::container::Kind::Video); - if let Some(millis) = presenting { - video.received(ts(millis * 1_000), kio::clock::now()); - } - let mut reader = holed(&mut track, presenting.map(|_| clock.consume())); - // A resume: the group past the hole starts a second of media later, inside the max delay. - write_group(&mut track, 50, &[ts(1_000_000)]); - let woken = Arc::new(Woken::default()); - let waiter = woken.waiter(); - assert!(drain_with(&mut reader, &waiter).is_empty()); - tokio::time::advance(Duration::from_millis(ends - 1)).await; - assert!(drain_with(&mut reader, &waiter).is_empty(), "{presenting:?}"); - woken.take(); - tokio::time::advance(Duration::from_millis(1)).await; - assert!(woken.take(), "{presenting:?}: the hole's end did not wake the reader"); - assert_eq!(drain_with(&mut reader, &waiter), ["gap", "frame 50"], "{presenting:?}"); - } - } - - /// A hole the playout clock trails ends as soon as the clock jumps past the group after it, which - /// wakes the reader: a re-anchor here, as a raised or lowered delay would. - #[tokio::test(start_paused = true)] - async fn a_hole_ends_once_the_playout_clock_jumps_past_the_group_past_it() { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let clock = crate::sync::Producer::new(Duration::ZERO); - let video = clock.track(crate::container::Kind::Video); - video.received(ts(0), kio::clock::now()); - let mut reader = holed(&mut track, Some(clock.consume())); - write_group(&mut track, 50, &[ts(1_000_000)]); - let woken = Arc::new(Woken::default()); - let waiter = woken.waiter(); - assert!(drain_with(&mut reader, &waiter).is_empty()); - tokio::time::advance(Duration::from_millis(10)).await; - assert!(drain_with(&mut reader, &waiter).is_empty()); - woken.take(); - // Video arrives a second of media ahead, which re-pins the clock past the group. - assert!(video.received(ts(1_000_000), kio::clock::now())); - assert!(woken.take(), "the clock's jump did not wake the reader"); - assert_eq!(drain_with(&mut reader, &waiter), ["gap", "frame 50"]); - } - /// The browser's resume: a reader that delivered a tail before a pause, then a replayed backlog - /// arriving newest first behind a hole. The clock kept running through the pause, so the reader - /// moves on to the oldest group that arrived as soon as the clock passes it, rather than - /// waiting for the backlog's oldest group, which lands last. + /// arriving newest first behind a hole. Nothing gives a hole up before the max delay, so the + /// reader waits for the backlog's oldest group, which lands last, and plays the replay whole. #[tokio::test(start_paused = true)] - async fn a_backlog_replayed_newest_first_plays_from_where_the_clock_passed() { - let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); - let clock = crate::sync::Producer::new(Duration::from_millis(100)); - let audio = clock.track(crate::container::Kind::Audio); - let subscriber = track.subscribe( - moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(2))), - ); - let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - reader.set_delay( - Delay::default() - .with_min(Duration::from_millis(500)) - .with_max(Duration::from_secs(2)), - ); - reader.set_playhead(clock.consume()); + async fn a_backlog_replayed_newest_first_plays_whole_within_the_max_delay() { + let mut copy = Received::new().await; + let mut reader = copy + .reader( + Delay::default() + .with_min(Duration::from_millis(500)) + .with_max(Duration::from_secs(2)), + ) + .await; let waiter = kio::Waiter::noop(); // The tail before the pause, a group per 100 ms of media and of wall time. for sequence in 0..5 { - write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + write_group(&mut copy.track, sequence, &[ts(sequence * 100_000)]); + carried().await; assert_eq!(drain_with(&mut reader, &waiter), [format!("frame {sequence}")]); - audio.received(ts(sequence * 100_000), kio::clock::now()); tokio::time::advance(Duration::from_millis(100)).await; } // Paused until 1.4 s, when the publisher replays groups 14 down to 5, one every 50 ms. tokio::time::advance(Duration::from_millis(900)).await; let mut read = Vec::new(); for sequence in (5..15).rev() { - write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + write_group(&mut copy.track, sequence, &[ts(sequence * 100_000)]); + carried().await; read.extend(drain_with(&mut reader, &waiter)); tokio::time::advance(Duration::from_millis(50)).await; } - // At 1.45 s the clock presents 1.35 s, past group 13, which landed then. - assert_eq!(read, ["gap", "frame 13", "frame 14"]); + let expected: Vec = (5..15).map(|sequence| format!("frame {sequence}")).collect(); + assert_eq!(read, expected); } /// A malformed data marker group found while looking ahead fails the reader, rather than @@ -3543,18 +3324,19 @@ mod tests { /// one after it, and is waited for like any hole rather than skipped as too old. #[tokio::test(start_paused = true)] async fn the_head_after_a_declared_end_is_waited_for() { - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; let mut consumer = Consumer::new( - track.subscribe( + copy.subscribe_with( moq_net::track::Subscription::default() .with_delay(Delay::default().with_min(None).with_max(Duration::from_secs(30))), - ), + ).await, Container::Legacy(crate::container::Kind::Audio), ); consumer.max_delay = Duration::from_millis(300); - write_group(&mut track, 0, &[ts(0)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; // The end, then the stopped encoder's final packet, as `moq_audio`'s producer writes them. - let mut group = track.create_group(moq_net::group::Info { start: 1 }).unwrap(); + let mut group = copy.track.create_group(moq_net::group::Info { start: 1 }).unwrap(); let frame = |timestamp, payload| Frame { timestamp, payload, @@ -3571,17 +3353,21 @@ mod tests { ) .unwrap(); group.finish().unwrap(); + carried().await; assert_eq!(consumer.read_frame().await.unwrap().unwrap().timestamp, ts(0)); assert!(matches!( consumer.read().await.unwrap(), Some(Read::End { group: 1, .. }) )); assert_eq!(consumer.read_frame().await.unwrap().unwrap().timestamp, ts(40_000)); - write_group(&mut track, 3, &[ts(60_060_000)]); + write_group(&mut copy.track, 3, &[ts(60_060_000)]); + carried().await; tokio::time::advance(Duration::from_millis(299)).await; + carried().await; assert!(consumer.poll_read(&kio::Waiter::noop()).is_pending()); - write_group(&mut track, 2, &[ts(60_000_000)]); - let _track = track.finish().unwrap(); + write_group(&mut copy.track, 2, &[ts(60_000_000)]); + carried().await; + let _track = copy.track.finish().unwrap(); assert_eq!(items(&mut consumer).await, ["60000000", "60060000"]); } @@ -4137,16 +3923,18 @@ mod tests { #[tokio::test] async fn gap_keeps_late_arriving_group_within_max_delay() { tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let consumer_track = track.subscribe( + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + let consumer_track = copy.subscribe_with( moq_net::track::Subscription::default() .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(500))), - ); + ).await; let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); // Group 2's stream beats group 1's, which hasn't arrived at all yet. - write_group(&mut track, 0, &[ts(0)]); - write_group(&mut track, 2, &[ts(200_000)]); + write_group(&mut copy.track, 0, &[ts(0)]); + carried().await; + write_group(&mut copy.track, 2, &[ts(200_000)]); + carried().await; let first = consumer.read_frame().await.unwrap().expect("group 0 frame"); assert_eq!(first.timestamp, ts(0)); @@ -4161,8 +3949,9 @@ mod tests { ); // Group 1's stream opens moments later, well within the 500ms budget. - write_group(&mut track, 1, &[ts(100_000)]); - let _track = track.finish().unwrap(); + write_group(&mut copy.track, 1, &[ts(100_000)]); + carried().await; + let _track = copy.track.finish().unwrap(); let frames = read_all(&mut consumer).await.unwrap(); let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); @@ -4231,28 +4020,36 @@ mod tests { playout.consumer(consumer_track, Container::Legacy(crate::container::Kind::Data)) } - /// The live edge's stream wins the race, yet the start waits for a group covering the reach. - #[tokio::test] + /// A reader of `copy` that starts `min` plus `warmup` behind the live edge and skips at `max`. + async fn reach_reader(copy: &Received, min: Duration, max: Duration, warmup: Duration) -> Consumer { + let playout = Playout::new( + Delay { + min: Some(min), + max: Some(max), + }, + warmup, + ); + let subscriber = copy + .subscribe_with(playout.subscription(moq_net::track::Subscription::default())) + .await; + playout.consumer(subscriber, Container::Legacy(crate::container::Kind::Data)) + } + + /// The publisher sends the live edge first, newest first, yet the reader starts at the earlier + /// group its join owes, which the ordered reader waits for. + #[tokio::test(start_paused = true)] async fn the_start_waits_for_an_earlier_group_when_the_live_edge_arrives_first() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let mut consumer = reach_consumer( - &mut track, + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + write_group(&mut copy.track, 4, &[ts(4_800_000)]); + write_group(&mut copy.track, 5, &[ts(5_000_000)]); + let mut consumer = reach_reader( + ©, Duration::from_millis(500), Duration::from_millis(500), Duration::ZERO, - ); - - write_group(&mut track, 5, &[ts(5_000_000)]); - assert!( - kio::clock::timeout(Duration::from_millis(50), consumer.read()) - .await - .is_err(), - "an earlier group within the budget must be waited for" - ); - - write_group(&mut track, 4, &[ts(4_800_000)]); - let _track = track.finish().unwrap(); + ) + .await; + let _track = copy.track.finish().unwrap(); let frames = read_all(&mut consumer).await.unwrap(); let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); @@ -4298,33 +4095,30 @@ mod tests { assert_eq!(warmed(unstated.with_min(Duration::ZERO), warmup).min, Some(warmup)); } - /// The warmup reaches past the min delay for the start, waiting for the group covering the live - /// edge less both. - #[tokio::test] + /// The warmup reaches past the min delay for the start: the subscription widens the max by it, + /// so the publisher serves the group covering the live edge less both. + #[tokio::test(start_paused = true)] async fn the_warmup_reaches_further_back_for_the_start() { - tokio::time::pause(); for (warmup, expected) in [ (Duration::ZERO, vec![4_700_000, 5_000_000]), (Duration::from_millis(300), vec![4_400_000, 4_700_000, 5_000_000]), ] { - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let mut consumer = reach_consumer( - &mut track, + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + // Group 4 covers the min delay alone, but not the warmup too. + write_group(&mut copy.track, 3, &[ts(4_400_000)]); + write_group(&mut copy.track, 4, &[ts(4_700_000)]); + write_group(&mut copy.track, 5, &[ts(5_000_000)]); + let mut consumer = reach_reader( + ©, Duration::from_millis(200), Duration::from_millis(200), warmup, - ); - - // Newest first: group 4 covers the min delay alone, but not the warmup too. - write_group(&mut track, 5, &[ts(5_000_000)]); - write_group(&mut track, 4, &[ts(4_700_000)]); - let first = kio::clock::timeout(Duration::from_millis(50), consumer.read_frame()).await; - write_group(&mut track, 3, &[ts(4_400_000)]); - let _track = track.finish().unwrap(); + ) + .await; + let _track = copy.track.finish().unwrap(); - let frames = first.ok().map(|frame| frame.unwrap().unwrap()); - let frames = frames.into_iter().chain(read_all(&mut consumer).await.unwrap()); - let micros: Vec = frames.map(|f| f.timestamp.as_micros().unwrap()).collect(); + let frames = read_all(&mut consumer).await.unwrap(); + let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); assert_eq!(micros, expected, "warmup {warmup:?}"); } } @@ -4445,28 +4239,22 @@ mod tests { assert_eq!(consumer.max_delay, Duration::from_secs(1)); } - /// A quiet track with no earlier group starts once its first group arrived a whole reach ago. - #[tokio::test] - async fn a_quiet_track_starts_within_one_reach() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let mut consumer = reach_consumer( - &mut track, + /// A quiet copy whose publisher holds nothing earlier starts at once: the ordered reader knows + /// no earlier group is owed, so nothing waits a reach for one. + #[tokio::test(start_paused = true)] + async fn a_quiet_track_starts_at_once_when_nothing_earlier_is_owed() { + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + write_group(&mut copy.track, 5, &[ts(5_000_000)]); + let mut consumer = reach_reader( + ©, Duration::from_millis(200), Duration::from_millis(200), Duration::from_millis(300), - ); - - write_group(&mut track, 5, &[ts(5_000_000)]); - assert!( - kio::clock::timeout(Duration::from_millis(450), consumer.read()) - .await - .is_err(), - "the wait lasts while an earlier group could still cover the reach" - ); - let frame = kio::clock::timeout(Duration::from_millis(100), consumer.read_frame()) + ) + .await; + let frame = kio::clock::timeout(Duration::from_millis(50), consumer.read_frame()) .await - .expect("the wait ends a reach after the first group arrived") + .expect("nothing earlier is owed") .unwrap() .unwrap(); assert_eq!(frame.timestamp, ts(5_000_000)); @@ -4566,31 +4354,24 @@ mod tests { } /// A min delay past the max delay reaches back only the max delay, since the publisher serves - /// nothing older: a quiet track starts once its first group arrived the max delay ago. - #[tokio::test] + /// nothing older. + #[tokio::test(start_paused = true)] async fn a_min_delay_past_the_max_delay_reaches_the_max_delay() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let mut consumer = reach_consumer( - &mut track, + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + write_group(&mut copy.track, 3, &[ts(4_400_000)]); + write_group(&mut copy.track, 4, &[ts(4_700_000)]); + write_group(&mut copy.track, 5, &[ts(5_000_000)]); + let mut consumer = reach_reader( + ©, Duration::from_secs(2), Duration::from_millis(200), Duration::ZERO, - ); - - write_group(&mut track, 5, &[ts(5_000_000)]); - assert!( - kio::clock::timeout(Duration::from_millis(150), consumer.read()) - .await - .is_err(), - "the wait lasts the max delay" - ); - let frame = kio::clock::timeout(Duration::from_millis(100), consumer.read_frame()) - .await - .expect("the wait ends at the max delay, not the min delay") - .unwrap() - .unwrap(); - assert_eq!(frame.timestamp, ts(5_000_000)); + ) + .await; + let _track = copy.track.finish().unwrap(); + let frames = read_all(&mut consumer).await.unwrap(); + let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); + assert_eq!(micros, vec![4_700_000, 5_000_000], "group 3 is past the max delay"); } /// The subscription carries the min and max delay, each plus the warmup, so a live join @@ -4646,14 +4427,16 @@ mod tests { assert_eq!(consumer.warmup, DELAY_MAX); } - /// An unlimited max delay keeps the subscription unlimited, and the min delay still sets the - /// start's reach. - #[tokio::test] + /// An unlimited max delay keeps the subscription unlimited, and the start still reaches the + /// earlier group the join owes. + #[tokio::test(start_paused = true)] async fn an_unlimited_max_delay_still_reaches_the_min_delay() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - let consumer_track = - track.subscribe(moq_net::track::Subscription::default().with_delay(Delay::default().with_max(None))); + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + write_group(&mut copy.track, 4, &[ts(4_500_000)]); + write_group(&mut copy.track, 5, &[ts(5_000_000)]); + let subscriber = copy + .subscribe_with(moq_net::track::Subscription::default().with_delay(Delay::default().with_max(None))) + .await; let mut consumer = Playout::new( Delay { min: Some(Duration::from_millis(500)), @@ -4661,18 +4444,9 @@ mod tests { }, Duration::ZERO, ) - .consumer(consumer_track, Container::Legacy(crate::container::Kind::Data)); + .consumer(subscriber, Container::Legacy(crate::container::Kind::Data)); assert_eq!(subscribed(&consumer).subscription().delay.max, None); - - write_group(&mut track, 5, &[ts(5_000_000)]); - assert!( - kio::clock::timeout(Duration::from_millis(50), consumer.read()) - .await - .is_err(), - "an earlier group within the min delay must be waited for" - ); - write_group(&mut track, 4, &[ts(4_500_000)]); - let _track = track.finish().unwrap(); + let _track = copy.track.finish().unwrap(); let frames = read_all(&mut consumer).await.unwrap(); let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); assert_eq!(micros, vec![4_500_000, 5_000_000]); @@ -5477,97 +5251,60 @@ mod tests { assert_eq!(micros, vec![700_000, 800_000, 900_000]); } - /// Groups arrive newest first, so the head of what the publisher serves lands after the - /// live edge: the cursor starts where the publisher says its feed starts, rather than at - /// the first arrival, and a group that arrives below it is never delivered, with no gap. - #[tokio::test] + /// The reader starts where the publisher resolved the join, though the newest group lands first. + #[tokio::test(start_paused = true)] async fn startup_at_the_publishers_start_over_a_newer_first_arrival() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - track.start_at(8).unwrap(); - let mut consumer = Consumer::new( - track.subscribe( - moq_net::track::Subscription::default() - .with_delay(Delay::default().with_min(None).with_max(Duration::from_secs(10))), - ), - Container::Legacy(crate::container::Kind::Data), - ); - - write_group(&mut track, 10, &[ts(1_000_000)]); - assert!( - kio::clock::timeout(Duration::from_millis(50), consumer.read()) - .await - .is_err(), - "the publisher's start is still on its way" - ); - for sequence in [7, 8, 9] { - write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + for sequence in [7, 8, 9, 10] { + write_group(&mut copy.track, sequence, &[ts(sequence * 100_000)]); } - let _track = track.finish().unwrap(); + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default() + .with_delay(Delay::default().with_min(None).with_max(Duration::from_secs(10))) + .with_ranges([moq_net::track::Range { + start: moq_net::track::Anchor::Relative(2), + end: None, + }]), + ) + .await; + let mut consumer = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); + let _track = copy.track.finish().unwrap(); assert_eq!(items(&mut consumer).await, ["800000", "900000", "1000000"]); } - /// A relative start still waiting on the feed takes the start the publisher resolved for it - /// when that is later: the publisher counts back from a newest group that may hold no frame - /// yet, which neither the largest it reports nor this copy counts, so a floor counted here - /// would wait on a group that never comes. - #[tokio::test] + /// A live start takes the start the publisher resolved for it: the publisher counts back from a + /// newest group that may hold no frame yet, which neither the largest it reports nor this copy + /// counts, so the reader waits for no group below it. + #[tokio::test(start_paused = true)] async fn a_waiting_relative_start_takes_the_publishers_resolved_start() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); - track.start_at(5).unwrap(); - let consumer_track = track.subscribe( - moq_net::track::Subscription::default() - .with_delay(Delay::default().with_max(Duration::from_secs(10))) - .with_ranges([moq_net::track::Range::live()]), - ); - assert_eq!( - consumer_track.floor(), - None, - "the relative start waits on a first group" - ); - let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); - - write_group(&mut track, 4, &[ts(400_000)]); - write_group(&mut track, 5, &[ts(500_000)]); - let _track = track.finish().unwrap(); - - assert_eq!(items(&mut consumer).await, ["500000"]); - } - - /// A relative floor already resolved here still rises to the start the publisher resolved, - /// which counts a newest group that held no frame yet, rather than wait on a group below it - /// that never comes. - #[tokio::test] - async fn a_resolved_relative_floor_rises_to_the_publishers_start() { - tokio::time::pause(); - let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; for sequence in 0..5 { - write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + write_group(&mut copy.track, sequence, &[ts(sequence * 100_000)]); } - track.start_at(5).unwrap(); - let consumer_track = track.subscribe( - moq_net::track::Subscription::default() - .with_delay(Delay::default().with_max(Duration::from_secs(10))) - .with_ranges([moq_net::track::Range::live()]), - ); - assert_eq!( - consumer_track.floor(), - Some(4), - "resolved here from the newest group with a frame" - ); - let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); + let mut newest = copy.track.create_group(moq_net::group::Info { start: 5 }).unwrap(); + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default() + .with_delay(Delay::default().with_max(Duration::from_secs(10))) + .with_ranges([moq_net::track::Range::live()]), + ) + .await; + let mut consumer = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); - write_group(&mut track, 5, &[ts(500_000)]); - let read = kio::clock::timeout(Duration::from_millis(50), consumer.read()) - .await - .expect("group 5 reads without waiting on group 4") + let frame = Frame { + timestamp: ts(500_000), + payload: Bytes::from_static(&[0xDE, 0xAD]), + keyframe: false, + duration: None, + }; + Container::Legacy(crate::container::Kind::Data) + .write(&mut newest, &[frame]) .unwrap(); - assert!( - matches!(read, Some(crate::container::Read::Frame { group: 5, .. })), - "{read:?}" - ); + newest.finish().unwrap(); + let _track = copy.track.finish().unwrap(); + assert_eq!(items(&mut consumer).await, ["500000"]); } #[kio::test] diff --git a/rs/moq-mux/src/container/source.rs b/rs/moq-mux/src/container/source.rs index 037b826207..ff1e52c6b9 100644 --- a/rs/moq-mux/src/container/source.rs +++ b/rs/moq-mux/src/container/source.rs @@ -1335,8 +1335,7 @@ mod tests { /// A live start begins at the publisher's newest group, even when an older group of the /// backlog the budget serves arrives first. Starting at that older group reads it, then /// skips the groups after it once the next PES moves the live edge a budget past them: a - /// gap right after the first frame. Behind a relay holding only that first group, the - /// relay passes on the newest group its source named. + /// gap right after the first frame. #[tokio::test(start_paused = true)] async fn a_live_start_behind_a_backlog_starts_at_the_newest_group() { use std::time::Duration; @@ -1424,8 +1423,15 @@ mod tests { timestamps.push(timestamp); } - // Group 84 was the newest when the export joined. - let expected: Vec = (84..95).map(|sequence| sequence * 24_000).collect(); + // Group 84 was the newest when the export joined. Behind the relay the ordered reader + // hands out nothing until its start is placed, which the held streams delay past the + // next two PES, so the newest group by then is the start: either way nothing after it is + // skipped. + let start = match relay { + false => 84, + true => 94, + }; + let expected: Vec = (start..95).map(|sequence| sequence * 24_000).collect(); assert_eq!(timestamps, expected, "relay: {relay}"); assert_eq!(export.skips(), 0, "relay: {relay}"); } diff --git a/rs/moq-mux/src/container/ts/passthrough_test.rs b/rs/moq-mux/src/container/ts/passthrough_test.rs index e68c192f48..37fb701de3 100644 --- a/rs/moq-mux/src/container/ts/passthrough_test.rs +++ b/rs/moq-mux/src/container/ts/passthrough_test.rs @@ -106,14 +106,18 @@ async fn import_onto( moq_net::track::Subscription::default() .with_delay(moq_net::track::Delay::default().with_max(RECORDING_MAX_DELAY)), ) - .await?; + .await? + .ordered(); let mut objects = Vec::new(); let mut markers = Vec::new(); let mut starts = Vec::new(); - while let Some(mut group) = kio::clock::timeout(Duration::from_secs(1), track.recv_group()) + while let Some(item) = kio::clock::timeout(Duration::from_secs(1), track.next_group()) .await .expect("the track stalled")? { + let moq_net::track::Item::Group(mut group) = item else { + continue; + }; starts.push(group.start); while let Some(frame) = group.read_frame().await? { if frame.payload.is_empty() { diff --git a/rs/moq-mux/src/container/ts/recovery_test.rs b/rs/moq-mux/src/container/ts/recovery_test.rs index 9a3da4fded..07a594bab7 100644 --- a/rs/moq-mux/src/container/ts/recovery_test.rs +++ b/rs/moq-mux/src/container/ts/recovery_test.rs @@ -482,14 +482,18 @@ async fn record(profile: Profile) -> Recording { .with_delay(moq_net::track::Delay::default().with_max(Duration::from_secs(300))), ) .await - .unwrap(); + .unwrap() + .ordered(); let mut groups = Vec::new(); let mut ends = Vec::new(); - while let Some(mut group) = kio::clock::timeout(Duration::from_secs(1), track.recv_group()) + while let Some(item) = kio::clock::timeout(Duration::from_secs(1), track.next_group()) .await .unwrap() .unwrap() { + let moq_net::track::Item::Group(mut group) = item else { + continue; + }; let mut frames = Vec::new(); while let Some(frame) = group.read_frame().await.unwrap() { frames.push(frame); diff --git a/rs/moq-mux/src/container/ts/release_test.rs b/rs/moq-mux/src/container/ts/release_test.rs index 03cb4cdef0..fa46b52e39 100644 --- a/rs/moq-mux/src/container/ts/release_test.rs +++ b/rs/moq-mux/src/container/ts/release_test.rs @@ -55,13 +55,17 @@ async fn import(data: &[u8]) -> Imported { moq_net::track::Subscription::default().with_delay(moq_net::track::Delay::default().with_max(RECORDING)), ) .await - .unwrap(); + .unwrap() + .ordered(); let mut objects = Vec::new(); - while let Some(mut group) = kio::clock::timeout(Duration::from_secs(1), track.recv_group()) + while let Some(item) = kio::clock::timeout(Duration::from_secs(1), track.next_group()) .await .unwrap() .unwrap() { + let moq_net::track::Item::Group(mut group) = item else { + continue; + }; while let Some(frame) = group.read_frame().await.unwrap() { if frame.payload.is_empty() { let micros = objects.last().map_or(0, |object: &Object| object.micros); diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index 04dae30e24..e16e7e513a 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -71,12 +71,10 @@ public typealias AnnounceEvent = MoqFFI.MoqAnnounceEvent /// deliver, and group ordering. public typealias Subscription = MoqFFI.MoqSubscription /// How far behind the live edge a subscriber or decoder plays: `min` is the target latency, where a -/// cold join starts, and how long a decoder waits for a missing group, `nil` to leave it unstated -/// (a decoder starts it as zero and waits a missing group for `max`, except an audio decoder, which -/// floors it at zero and waits only its playout estimate, which can be larger; a catalog warmup or -/// an audio decoder's playout delay still states a `min` to the publisher), and `max`, -/// `nil` for no subscriber limit, is how far a non-latest group may fall behind before it is -/// skipped. A `min` past `max` starts at `max`. +/// cold join starts, `nil` to leave it unstated (a decoder starts it as zero; a catalog warmup or +/// an audio decoder's playout delay still states a `min` to the publisher), and `max`, `nil` for no +/// subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how +/// long a decoder waits for a missing group. A `min` past `max` starts at `max`. public typealias Delay = MoqFFI.MoqDelay /// Options for fetching one complete group by start. public typealias FetchGroupOptions = MoqFFI.MoqFetchGroupOptions From 2f78356fd650e318f9fed7d787189f19b496e14d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:18:53 -0800 Subject: [PATCH 06/23] quest(unused-api/net): the sweep finishes; follow up on moq-audio's estimate timing reads Co-Authored-By: Claude Opus 5.5 --- quest/m2/README.md | 5 +- .../audio-playout/browser-audio-hole-skip.md | 4 - quest/m2/audio-playout/estimate-landing.md | 32 ++++ quest/m2/outcome-enums/ordered-gaps.md | 13 +- quest/m2/unused-api/README.md | 42 ----- quest/m2/unused-api/net.md | 159 ------------------ quest/m3/control-stream-recv.md | 4 - quest/m3/release/api-review.md | 2 +- 8 files changed, 41 insertions(+), 220 deletions(-) create mode 100644 quest/m2/audio-playout/estimate-landing.md delete mode 100644 quest/m2/unused-api/README.md delete mode 100644 quest/m2/unused-api/net.md diff --git a/quest/m2/README.md b/quest/m2/README.md index 4f61f38fbb..fb796e34e0 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -39,8 +39,9 @@ removal, a rename, or a wire, catalog, or format change that is free until what the binding sends. - **moq-net API.** [Socket endpoints](https://github.com/moq-dev/moq/pull/9029) leave for Kixel, and the demos return to their earlier format. -- **Unused API.** [One sweep per area](/quest/m2/unused-api/README.md), each - item re-checked for a downstream caller first. +- **Unused API.** One sweep per area (done: the last, + [moq-dev/moq#9196](https://github.com/moq-dev/moq/pull/9196)), each item + re-checked for a downstream caller first. - **Auth and crypto.** End-to-end encryption is its own unpublished crate again ([moq-dev/moq#8947](https://github.com/moq-dev/moq/pull/8947)); its [finished format](/quest/m3/e2ee-format.md) follows in m3. diff --git a/quest/m2/audio-playout/browser-audio-hole-skip.md b/quest/m2/audio-playout/browser-audio-hole-skip.md index 55d07dfecc..17d2eb2423 100644 --- a/quest/m2/audio-playout/browser-audio-hole-skip.md +++ b/quest/m2/audio-playout/browser-audio-hole-skip.md @@ -42,7 +42,3 @@ Work: max buffer fills, which #8799's playhead covered. A trace recorded with `--trace` holds only the frames that arrived, so replaying such a group also needs the trace to record when each group opens. - -## Related - -- [moq-net's last pub items with a single container-reader caller go private](/quest/m2/unused-api/net.md) - the Rust container reader moving onto `Gap`, this quest's native counterpart diff --git a/quest/m2/audio-playout/estimate-landing.md b/quest/m2/audio-playout/estimate-landing.md new file mode 100644 index 0000000000..fcbe25a925 --- /dev/null +++ b/quest/m2/audio-playout/estimate-landing.md @@ -0,0 +1,32 @@ +# [M] moq-audio's playout estimate times when a packet landed, not when it was read + +## Goal + +A missing group the container reader waits out no longer reads as jitter in +`moq-audio`'s playout estimate, so a lost packet doesn't raise `moq play`'s +audio delay toward the max delay. + +## Plan + +Found by [moq-dev/moq#9196](https://github.com/moq-dev/moq/pull/9196), which +moved the container reader onto `Ordered`: a missing group is now given up by +`Delay::max` alone, where the reader used to give it up after the +subscription's `Delay::min`, which `moq-audio` set to its playout estimate. +`decode::Consumer::next` observes each packet at `now()`, when it is read, so +every packet queued behind a hole the reader waited out reads as late by up to +the max delay. `@moq/hang` doesn't have this: its estimate observes a second +subscription as frames land (`#observe`). + +Recommended: observe each packet at `container::Consumer::arrival()`, when its +wire frame landed in this node's cache, which a hole wait doesn't move. That +needs the jitter corpus test (`the_decode_path_conforms_to_the_corpus`) to +drive moq-net's clock, since its replay sets a thread-local read clock that +moq-net's landing times don't follow: run it on tokio's paused clock and +advance it to each arrival. #9196 tried the switch and that test failed only +for this reason (`buildup: packet 75 delays 120 ms, expected 100`). + +Public API: none. Wire: none. + +## Related + +- [Audio playout](/quest/m2/audio-playout/README.md) - the estimator this feeds diff --git a/quest/m2/outcome-enums/ordered-gaps.md b/quest/m2/outcome-enums/ordered-gaps.md index 6c016aabab..3bdd119ae7 100644 --- a/quest/m2/outcome-enums/ordered-gaps.md +++ b/quest/m2/outcome-enums/ordered-gaps.md @@ -11,8 +11,8 @@ that needs gaps in order: moq-mux's container reader. Decided 2026-11-18 (maintainer, /quest-plan): ✅ two methods (rejected: keeping `Item` as the only return; replacing gaps with a counter, which -[unused-api/net](/quest/m2/unused-api/net.md)'s reader cannot use, since it -needs each gap's range in order). +moq-mux's container reader cannot use, since it needs each gap's range in +order and when what it gave up landed). Every production caller skips `Item::Gap` today: libmoq, moq-ffi, moq-json snapshot and window, moq-hls, moq-mux's TS export and MSF catalog, @@ -21,14 +21,11 @@ moq-archive, moq-stats, and moq-net's benches `.and_then(Item::group)` read a gap as the end of the track; they move to `next_group`. `Item::group` goes if nothing needs it after. -Whichever lands second of this and [unused-api/net](/quest/m2/unused-api/net.md) -points the container reader at `next_item`. +moq-mux's container reader (`container/consumer.rs`, which reads through +`Ordered` since [moq-dev/moq#9196](https://github.com/moq-dev/moq/pull/9196)) +moves onto `next_item`, the one production caller that reads gaps. Public API: breaking in moq-net; callers of `next_group` that matched `Item` drop the match, and the gap-reading one calls `next_item`. No JS or binding change: js/net's `Ordered` already has `nextGroup` and `nextItem`, and moq-ffi hides gaps. Wire: none. - -## Related - -- [moq-net's last pub items](/quest/m2/unused-api/net.md) - moves the container reader onto `Ordered`'s gaps diff --git a/quest/m2/unused-api/README.md b/quest/m2/unused-api/README.md deleted file mode 100644 index cfc08b9c07..0000000000 --- a/quest/m2/unused-api/README.md +++ /dev/null @@ -1,42 +0,0 @@ -# The cut publishes no API without a caller - -## Goal - -Every public item the 0.22 cut would publish for the first time has a caller -outside its own crate or package and its tests, or it is private, hidden, or -gone before the cut. The sweep runs once per area, one child each, and has -no work of its own: it completes when every child merges. - -## Plan - -Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit): ✅ remove -or hide every unused public item the audit found, one sweep quest per area -(rejected: keeping them as harmless; one sweep across the whole tree). An -unused item costs nothing to delete now, and costs a breaking release once -0.22 has published it. A feature returns as additive work when a consumer -needs it. - -The maintainer's condition applies to every child: before removing an item, -check Kixel and moq.pro for a caller, and keep what one uses. Both pin a -`main` commit and call API that nothing in this repository does. Each child -lists its items with the grep that found no caller here, so its PR re-runs -the grep on its own head and adds the downstream check. - -Removal is the default. Hide an item (`pub(crate)`, -`@internal`, a `test-support` or `lab` feature) only when code in this -repository still needs it across a crate or package boundary. - -The same rule now covers the bindings (AGENTS.md, 2026-11-16): a feature -reaches them when a binding user needs it, so the sweep mirrors nothing new -into moq-ffi. - -Not part of the sweep, because each has its own quest: socket endpoints -([removed for Kixel](https://github.com/moq-dev/moq/pull/9029)), the public -route API (made crate-private by -[moq-dev/moq#9053](https://github.com/moq-dev/moq/pull/9053)). - -Public API: each child lists what leaves. Wire: none. - -## Related - -- [API shape review](/quest/m3/release/api-review.md) - the recurring review that should have caught these; its Plan records what this run learned diff --git a/quest/m2/unused-api/net.md b/quest/m2/unused-api/net.md deleted file mode 100644 index 64a94ffb2e..0000000000 --- a/quest/m2/unused-api/net.md +++ /dev/null @@ -1,159 +0,0 @@ -# [L] moq-net's last pub items with a single container-reader caller go private - -## Goal - -`track::Subscriber::next_in_range`, `is_skipped`, and `poll_start` are -crate-private, and their js/net mirrors leave its published surface. Their -one consumer is the container reader (moq-mux, and `@moq/hang` in JS), which -orders groups itself; `track::Ordered` reports a `Gap`, so the reader can -move onto it. - -## Plan - -What is left of the moq-net sweep, which removed or hid every other item -without a caller. - -Findings (2026-11-17, from reading the code when this quest was claimed as -`[S]`): moving the reader onto `Gap` is not an `Ordered` swap, so the quest -is `[L]`. - -- `moq-mux`'s `container::Consumer` reads in arrival order - (`poll_recv_group`) into its own `pending` queue and judges a hole at - `current` while the groups past it are already buffered: the wall-time wait - since a later group arrived (`hole_wait`, now - `Delay::min`), media-timestamp `aged` against the newest buffered content, - the playout clock `passed`, a full max buffer, and container continuity - (`poll_continues`, 1 ms contiguity). -- `track::Ordered` holds at most one group back behind a hole and judges it by - its own reach-versus-newest-frame budget. It never delivers the groups past - a hole, so the reader could not run the wait, clock, buffer-full, or - continuity rules above on it. `next_in_range` is what tells the reader - "nothing will fill this hole" (the `excluded` case) and `poll_start` - resolves its start; `Gap` covers the first and `Ordered`'s start the - second, only if the reader drops everything else above. -- The three methods have more callers than the one reader. - `Subscriber::is_skipped` is also called from `moq-audio` (through - `container::Consumer::is_skipped`), `moq-stats`, and the ts import tests; - `Ordered::is_skipped`, already public, would serve those. - `Subscriber::with_range` has production callers in `moq-mux` - `container/source.rs`, `moq-json` `window`, and `moq-audio` - `decode/consumer.rs`, and in `moq-net/benches/origin.rs`. -- The reader's code is being changed by - [#8984](https://github.com/moq-dev/moq/pull/8984) (hole waiting), - [#9021](https://github.com/moq-dev/moq/pull/9021), and - [#8951](https://github.com/moq-dev/moq/pull/8951), so this starts after - the hole wait lands. - -Decided 2026-11-17 (maintainer, `/quest-run m0/m1/m2`): - -- ✅ B: keep the quest, rewritten as `[L]` with the hole wait as `Required` - (rejected: A, delete it and keep the methods public as the reader's - documented surface; C, shrink it to making `with_range` crate-private - alone). - -Hiding is no middle ground: `#[doc(hidden)]` is banned -([moq-dev/moq#8392](https://github.com/moq-dev/moq/pull/8392)), and js/net -keeps `@internal` for deprecated exports. - -Step 1's design (2026-11-19, from reading `container/consumer.rs` and -`Ordered`), rule by rule for a hole at the reader's cursor: - -- `Ordered` already covers a finished track, keys the ranges exclude - (`next_in_range`), declared skips (`is_skipped`), and `Delay::max` aged on - the reach against the newest frame, though only on a track whose moq-net - groups carry timestamps; on an untimed one it waits until the hole - resolves, where the reader ages on decoded container timestamps. -- The reader keeps 1 ms continuity (on a `Gap`, it calls the loss only once - the next group's first frame fails to continue the edge) and the slow-group - skip, since `Ordered` hands out the next group while the reader still reads - the current one. A full max buffer past a hole is moot: `Ordered` leaves - those groups in the transport cache. -- `Ordered` cannot run two rules, since it never hands out the groups past a - hole: the `Delay::min` wall-time wait since a group past the hole landed, - and the playout clock reaching where that group starts. - -Decided 2026-11-19 (maintainer, `/quest-complete`): ✅ B, `Ordered` owns -the hole policy (rejected: A, two peek/give-up methods for the reader; C, keep -`next_in_range`, `poll_start` and `is_skipped` public). `Ordered` gives up a -hole once `Delay::min` of wall time has passed since a later group landed (an -unstated minimum keeps waiting for `Delay::max`, as today), and takes a -playout-position input that gives up a hole once playback reaches where the -next group starts. Reason: these are player rules any `Ordered` reader that -states a minimum wants, and `Delay::min`'s docs already promise the wait. - -Findings while starting B (2026-11-19, from reading its callers): - -- `Delay::default()` states `min: Some(0)`, so every reader built from - `Delay::default().with_max(..)` states a zero minimum. moq-archive's ordered - store (`producer/track.rs`) does, and under B it would give up a reordered - group the moment a later one lands, where it waits up to its budget today. - The bindings' defaults (`moq_subscription_defaults`, libmoq's zeroed - `moq_delay`) state it too, so a binding user who raises only `max` changes - as well. -- `Ordered` never hands the reader a group it skips as stale or gives up as - late, so the reader can no longer report when that backlog landed: - `container::Consumer::backlog()`, which moq-audio feeds its join estimate - from, loses most of its signal. -- A local track owes nothing, so `Ordered` steps over its missing keys at - once and without a gap. The reader then keeps only its 1 ms continuity - rule there (and calls an undeclared, discontinuous run a gap without - waiting); its hole tests carry over on a received copy - (`container::test_util::Link`), not on a local track. -- `Ordered`'s wall-clock deadlines (the new `min` wait and playout position, - like `Delay::max` today) fire only on a track whose cache pool a driver - collects, as an origin's does; the reader's own timer goes away. - -`@moq/hang`'s container reader calls the js/net mirrors (`start`, -`nextInRange`, `isSkipped`), so the same move applies there in the same PR. - -`with_range` stays public (recommended, from the same reading): moq-json, -moq-audio, and moq-mux's `container/source.rs` call it as a local filter on -an established subscription, which `Subscription::ranges`, a wire request, -does not replace. - -Steps: - -1. Add B's `Ordered` policy: the `Delay::min` hole wait and a playout-position - input (Rust `Ordered::set_playhead`, js/net `Ordered.setPlayhead`), shaped - by the decisions under Waiting. -2. Move the reader onto it, and the `is_skipped` callers onto - `Ordered::is_skipped`. -3. Make `next_in_range`, `is_skipped`, and `poll_start` crate-private. -- `origin::Consumer::covering` stays `pub`: moq.pro and moq-net's own - `benches/origin.rs` call it. - -The reader reads gaps through `Ordered::next_item` once -[ordered-gaps](/quest/m2/outcome-enums/ordered-gaps.md) lands, since -`next_group` then yields groups alone (decided 2026-11-18, maintainer, -/quest-plan); whichever lands second makes the switch. - -Public API: breaking; three `Subscriber` methods go crate-private, `Ordered` -gains one input, and `Delay::min` becomes a hole wait for every ordered -reader. Wire: none. - -## Waiting - -- Maintainer: which ordered readers get B's `Delay::min` hole wait, since - `Delay::default()` states a zero minimum (see Findings while starting B): - - 1a (recommended): every reader that states a minimum, zero included, as - decided; in-tree readers that tolerate reordering (moq-archive, and any - other `Ordered` reader the audit finds) state `with_min(None)`, and the - bindings document that a zero minimum now gives a hole up at once. - - 1b: `Delay::default().min` becomes `None`, so only a stated minimum waits; - a media reader built from `Delay::default().with_max(x)` then waits `x` - for a hole instead of zero, and the bindings' defaults change with it. -- Maintainer: how moq-audio keeps learning when a join's backlog landed once - the reader reads through `Ordered` (see Findings while starting B): - - 2a (recommended): moq-audio observes every arrival through a second, - arrival-order subscription, as `@moq/hang`'s reader already does - (`#observe`), and `container::Consumer::backlog()` is removed (breaking - in moq-mux, one caller). - - 2b: `track::Item::Gap` gains when the latest group it gives up had - landed, which the reader reports as `backlog()`; mirrored in js/net. - - 2c: `backlog()` reports only what the reader itself discards; moq-audio's - join estimate can then end before its backlog finished arriving. - -## Related - -- [Ordered gaps](/quest/m2/outcome-enums/ordered-gaps.md) - splits `Ordered` into `next_group` and the `next_item` this reader uses -- [Browser audio holes](/quest/m2/audio-playout/browser-audio-hole-skip.md) - grading the browser counterpart, which already reads `Gap`s diff --git a/quest/m3/control-stream-recv.md b/quest/m3/control-stream-recv.md index 089a67b462..cdf3715e8f 100644 --- a/quest/m3/control-stream-recv.md +++ b/quest/m3/control-stream-recv.md @@ -36,7 +36,3 @@ same pattern and leave it out only if it cannot share the reader. Public API: none. Wire: none. Cover each reader with a test of the three outcomes, as `a_read_tells_a_cut_stream_from_malformed_bytes` does for the reader. - -## Related - -- [moq-net's last pub items with a single container-reader caller go private](/quest/m2/unused-api/net.md) - the rest of the moq-net sweep diff --git a/quest/m3/release/api-review.md b/quest/m3/release/api-review.md index 97325d95ed..b7a0e31330 100644 --- a/quest/m3/release/api-review.md +++ b/quest/m3/release/api-review.md @@ -60,7 +60,7 @@ up. Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit): ✅ every unused public item is removed or hidden before the cut, in -[one sweep per area](/quest/m2/unused-api/README.md), each re-checked for a +one sweep per area (done in m2), each re-checked for a downstream caller first (rejected: judging them after m2 lands). That audit re-read this list, so it is settled: From ba3bd08c2ff891d3d24b0ba968148bcf413aee6d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:24:32 -0800 Subject: [PATCH 07/23] feat(net,hang)!: an ordered gap says when what it gave up landed, and @moq/hang reads through Ordered js/net mirrors moq-net: `Item` gaps carry `landed`, and Subscriber's `start()`, `nextInRange()` and `isSkipped()` leave the published surface, with `Ordered.isSkipped` taking the docs. @moq/hang's Consumer always reads through an ordered read, which alone decides holes by `delay.max`; its `playhead` prop and `Playhead` type go, as does `Sync.anchor` in @moq/watch, whose only user was that playhead. Co-Authored-By: Claude Opus 5.5 --- .../src/container/consumer.backlog.test.ts | 154 +++--- js/hang/src/container/consumer.holes.test.ts | 147 ++---- .../container/consumer.nonsequential.test.ts | 21 +- .../src/container/consumer.outoforder.test.ts | 49 +- js/hang/src/container/consumer.skips.test.ts | 2 + .../src/container/consumer.startup.test.ts | 15 +- js/hang/src/container/consumer.test.ts | 25 +- js/hang/src/container/consumer.ts | 466 +++++------------- js/hang/src/container/consumer.warmup.test.ts | 93 +--- js/hang/src/container/index.ts | 2 +- js/net/src/fill.test.ts | 2 +- js/net/src/integration.test.ts | 15 +- js/net/src/internal.ts | 10 + js/net/src/lite/subscriber.test.ts | 2 +- js/net/src/ordered.test.ts | 32 +- js/net/src/resume.ts | 10 + js/net/src/track.ts | 78 +-- js/watch/src/sync.ts | 6 - js/watch/src/video/decoder.ts | 2 - 19 files changed, 417 insertions(+), 714 deletions(-) diff --git a/js/hang/src/container/consumer.backlog.test.ts b/js/hang/src/container/consumer.backlog.test.ts index 6202f74237..9adf8829b5 100644 --- a/js/hang/src/container/consumer.backlog.test.ts +++ b/js/hang/src/container/consumer.backlog.test.ts @@ -1,6 +1,6 @@ -// Pairs `rs/moq-audio`'s decode-path tests of the join's backlog: a group the consumer closes below -// its cursor unread still tells the estimator when it arrived, but only when the publisher's largest -// at the subscription's start shows it was produced before. +// Pairs `rs/moq-audio`'s decode-path tests of the join's backlog: a group the ordered read gives up +// unread still tells the estimator when it landed, but only when the publisher's largest at the +// subscription's start shows it was produced before. import { afterEach, beforeEach, expect, spyOn, test } from "bun:test"; import { Duration, type Installed, Instant, now } from "@moq/clock"; @@ -29,11 +29,17 @@ type Event = [number, number, number]; * consumer's measured term once they all have, beside the estimator fed the same arrivals with each * group `backlog` names reported rather than observed. */ -async function play(events: Event[], backlog: (group: number) => boolean): Promise<[number, number]> { +async function play( + events: Event[], + backlog: (group: number) => boolean, + observing = false, +): Promise<[number, number]> { const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(2000) } }), { + const subscription = { delay: { max: Duration.millis(2000) } }; + const consumer = new Consumer(track.subscribe(subscription), { format: new Format("audio"), delay: { min: Duration.millis(100), max: Duration.millis(2000) }, + ordered: observing ? track.subscribe(subscription).ordered() : undefined, }); const oracle = new Jitter(); const base = now().since(Instant.ZERO).asMillis(); @@ -81,7 +87,7 @@ const order = (a: Event, b: Event) => a[0] - b[0] || a[1] - b[1]; // over the first 400 ms and 15 more 1.0 to 1.3 s in, with the live frames produced meanwhile queued // behind them. Without `delivered`, that backlog never reaches this node, though the live frames still // queue behind it. -async function join(delivered: boolean): Promise { +async function join(delivered: boolean, observing: boolean): Promise { const events = live((produced) => { const at = produced + 40; if (at <= 400) return 400 + produced / 10; @@ -94,14 +100,18 @@ async function join(delivered: boolean): Promise { for (let s = 0; s < 15; s++) events.push([1000 + 20 * s, 141 + s, 99_700 + 20 * s]); } events.sort(order); - const [measured, oracle] = await play(events, (group) => group < 200); + const [measured, oracle] = await play(events, (group) => group < 200, observing); expect(measured).toBe(oracle); return measured; } +// Read through a second, ordered subscription, as the browser's audio decoder does, so the estimate +// observes the backlog as it lands. Without one, the consumer's own ordered read would say nothing of +// it: the backlog lands below where the read already is, and a local track's late arrival there is no +// gap in JS (Rust reports it Old, `a_late_arrival_is_a_gap`), so its landing never reaches the estimate. test("a join's backlog skipped below the cursor folds nothing", async () => { - const skipped = await join(true); - const unseen = await join(false); + const skipped = await join(true, true); + const unseen = await join(false, true); expect(skipped).toBeLessThanOrEqual(80); expect(unseen).toBeGreaterThanOrEqual(280); }); @@ -123,53 +133,67 @@ test("late groups dropped from the start hold no join open", async () => { expect(dropped).toBe(lost); }); -test("the join's boundary is the publisher's answer, not a group that raced it", async () => { - // Group 150 lands from a cache before the publisher answers that it stood at group 200. Group 160 - // then arrives below the cursor: produced before the subscription, so it is backlog. +/** + * A track whose ordered read's info resolves once `answer` is called, and whose largest is what + * `largest` says when asked. Its feed owes every key from `from`, as a session's would. + */ +function joining(from: bigint, largest: () => bigint) { const track = new Track.Producer("audio"); + track.requests(); const subscriber = track.subscribe({ delay: { max: Duration.millis(2000) } }); - let answered = false; + subscriber.withRange({ start: { included: from } }); let answer!: () => void; - const started = new Promise((resolve) => { + const answered = new Promise((resolve) => { answer = resolve; }); - const raced = new Proxy(subscriber, { - get(target, key) { - if (key === "start") return () => started.then(() => undefined); - if (key === "info") return () => started.then(() => ({})); - if (key === "largest") return async () => ({ group: answered ? 200n : 150n, frame: 0 }); - const value = Reflect.get(target, key, target); - return typeof value === "function" ? value.bind(target) : value; - }, - }); + const proxy = (target: T, overrides: Record): T => + new Proxy(target, { + get(target, key) { + if (typeof key === "string" && key in overrides) return overrides[key]; + const value = Reflect.get(target, key, target); + return typeof value === "function" ? value.bind(target) : value; + }, + }); + const ordered = () => + proxy(subscriber.ordered(), { + info: () => answered.then(() => ({})), + largest: async () => ({ group: largest(), frame: 0 }), + }); + const write = (sequence: number, media: number) => { + const group = track.createGroup(BigInt(sequence)); + const payload = new Uint8Array([...Varint.encode(media * 1000), 1]); + group.writeFrame({ payload, timestamp: Time.Timestamp.fromMicros(media * 1000) }); + group.finish(); + }; + return { track, subscriber: proxy(subscriber, { ordered }), answer, write }; +} + +// The ordered read gives up group 151 that never came with group 152 that landed behind it, once 153 +// landed a budget ago. The publisher stood at group 200 as the subscription began, so the gap is +// backlog: when 152 landed reaches the estimate, once. The read hands out its first group only once +// its start is known, so a cached group can no longer race the publisher's answer. +test("a gap given up at or below the join tells the estimate when it landed", async () => { + const { track, subscriber, answer, write } = joining(150n, () => 200n); const reported: number[] = []; const backlog = spyOn(Jitter.prototype, "backlog").mockImplementation((arrival: number) => { reported.push(arrival); }); - const consumer = new Consumer(raced, { + const consumer = new Consumer(subscriber, { format: new Format("audio"), delay: { min: Duration.millis(100), max: Duration.millis(2000) }, }); - const write = (sequence: number, media: number) => { - const group = track.createGroup(BigInt(sequence)); - const payload = new Uint8Array([...Varint.encode(media * 1000), 1]); - group.writeFrame({ payload, timestamp: Time.Timestamp.fromMicros(media * 1000) }); - group.finish(); - }; try { - write(150, 0); - await clock.advance(Duration.ZERO); - answered = true; answer(); + write(150, 0); expect(await consumer.next()).toMatchObject({ kind: "frame", group: 150n }); - write(201, 1020); - await clock.advance(Duration.millis(200)); + await clock.advance(Duration.millis(10)); + const landed = now().since(Instant.ZERO).asMillis(); + write(152, 40); + write(153, 60); + await clock.advance(Duration.millis(2001)); expect(await consumer.next()).toMatchObject({ kind: "gap" }); - expect(await consumer.next()).toMatchObject({ kind: "frame", group: 201n }); - write(160, 200); - write(202, 1040); - await clock.advance(Duration.ZERO); - expect(reported).toHaveLength(1); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 153n }); + expect(reported).toEqual([landed]); } finally { backlog.mockRestore(); consumer.close(); @@ -178,56 +202,32 @@ test("the join's boundary is the publisher's answer, not a group that raced it", } }); +// The publisher answers that it stood at group 200, then its largest moves to 210 before the first +// group reaches this consumer. Group 206, given up with hole 205, was produced after the subscription +// began, so it is no backlog. test("the join's boundary is the publisher's answer, not the largest by the first group", async () => { - // The start resolves before the subscription is established, while the largest still names cached - // group 150. The publisher answers that it stood at group 200, then groups up to 210 land before - // the first reaches this consumer. Group 205 then arrives below the cursor: produced after the - // subscription began, so it is no backlog, unlike group 160. - const track = new Track.Producer("audio"); - const subscriber = track.subscribe({ delay: { max: Duration.millis(2000) } }); - let answered = false; let advanced = false; - let answer!: () => void; - const started = new Promise((resolve) => { - answer = resolve; - }); - const raced = new Proxy(subscriber, { - get(target, key) { - if (key === "start") return async () => undefined; - if (key === "info") return () => started.then(() => ({})); - if (key === "largest") return async () => ({ group: advanced ? 210n : answered ? 200n : 150n, frame: 0 }); - const value = Reflect.get(target, key, target); - return typeof value === "function" ? value.bind(target) : value; - }, - }); + const { track, subscriber, answer, write } = joining(204n, () => (advanced ? 210n : 200n)); const reported: number[] = []; const backlog = spyOn(Jitter.prototype, "backlog").mockImplementation((arrival: number) => { reported.push(arrival); }); - const consumer = new Consumer(raced, { + const consumer = new Consumer(subscriber, { format: new Format("audio"), delay: { max: Duration.millis(2000) }, }); - const write = (sequence: number, media: number) => { - const group = track.createGroup(BigInt(sequence)); - const payload = new Uint8Array([...Varint.encode(media * 1000), 1]); - group.writeFrame({ payload, timestamp: Time.Timestamp.fromMicros(media * 1000) }); - group.finish(); - }; try { - await clock.advance(Duration.ZERO); - answered = true; answer(); await clock.advance(Duration.ZERO); advanced = true; - write(210, 200); - expect(await consumer.next()).toMatchObject({ kind: "frame", group: 210n }); - write(205, 100); - await clock.advance(Duration.ZERO); - expect(reported).toHaveLength(0); - write(160, 0); - await clock.advance(Duration.ZERO); - expect(reported).toHaveLength(1); + write(204, 0); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 204n }); + write(206, 40); + write(207, 60); + await clock.advance(Duration.millis(2001)); + expect(await consumer.next()).toMatchObject({ kind: "gap" }); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 207n }); + expect(reported).toEqual([]); } finally { backlog.mockRestore(); consumer.close(); diff --git a/js/hang/src/container/consumer.holes.test.ts b/js/hang/src/container/consumer.holes.test.ts index b6e8bd0e40..cca2ceb877 100644 --- a/js/hang/src/container/consumer.holes.test.ts +++ b/js/hang/src/container/consumer.holes.test.ts @@ -1,18 +1,24 @@ import { expect, test } from "bun:test"; -import { Duration, now } from "@moq/clock"; +import { Duration } from "@moq/clock"; import { Manual } from "@moq/clock/test"; import { Time, Track } from "@moq/net"; -import { Signal } from "@moq/signals"; import { Consumer, type Read } from "./consumer.ts"; import { encodeFrame, Format } from "./legacy.ts"; const flush = () => new Promise((resolve) => setImmediate(resolve)); +// A subscription to `track` whose missing groups an unserved fill owes, as a session's feed would, so +// its ordered read waits `max` for each hole before giving it up. +function owed(track: Track.Producer, max: Duration): Track.Subscriber { + track.requests(); + return track.subscribe({ delay: { max } }); +} + test.each([0, 100])("a sequence hole before a paused data marker expires at %sms", async (budget) => { const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("data"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(budget)), { format: new Format("data"), delay: { min: null, max: Duration.millis(budget) }, }); @@ -64,7 +70,7 @@ test("a filled hole cancels its timer and the next hole gets a fresh budget", as const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("data"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(100)), { format: new Format("data"), delay: { min: null, max: Duration.millis(100) }, }); @@ -118,11 +124,13 @@ test("a filled hole cancels its timer and the next hole gets a fresh budget", as } }); +// The ordered read gives every hole up at once a budget after the first group past it landed, and the +// groups buffered between the holes with them: by then each one's successor landed a budget ago. test("the holes a stalled source left behind share one budget", async () => { const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("data"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(100)), { format: new Format("data"), delay: { min: null, max: Duration.millis(100) }, }); @@ -148,8 +156,10 @@ test("the holes a stalled source left behind share one budget", async () => { for (const sequence of [2n, 4n, 6n]) write(sequence); await flush(); expect(results).toHaveLength(1); - await clock.advance(Duration.millis(100)); - expect(results.map((read) => read.kind)).toEqual(["frame", "gap", "frame", "gap", "frame", "gap", "frame"]); + await clock.advance(Duration.millis(99)); + expect(results).toHaveLength(1); + await clock.advance(Duration.millis(1)); + expect(results.map((read) => read.kind)).toEqual(["frame", "gap", "frame"]); } finally { consumer.close(); track.finish(); @@ -162,11 +172,12 @@ test("the holes a stalled source left behind share one budget", async () => { } }); +// Group 2, landing inside the wait, is given up with the hole: its successor landed a budget ago. test("a hole is late from the earliest arrival past it", async () => { const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("data"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(100)), { format: new Format("data"), delay: { min: null, max: Duration.millis(100) }, }); @@ -194,9 +205,10 @@ test("a hole is late from the earliest arrival past it", async () => { await clock.advance(Duration.millis(50)); write(2n); await flush(); + await clock.advance(Duration.millis(49)); expect(results).toHaveLength(1); - await clock.advance(Duration.millis(50)); - expect(results.map((read) => read.kind)).toEqual(["frame", "gap", "frame", "gap", "frame"]); + await clock.advance(Duration.millis(1)); + expect(results.map((read) => read.kind)).toEqual(["frame", "gap", "frame"]); } finally { consumer.close(); track.finish(); @@ -233,7 +245,7 @@ test("a hole in a drained backlog is late from the earliest arrival past it, not await clock.advance(Duration.millis(100)); // Joined now, the backlog drains in sequence order: group 3 before group 5, which landed first. - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(500)), { format: new Format("data"), delay: { max: Duration.millis(500) }, }); @@ -247,12 +259,11 @@ test("a hole in a drained backlog is late from the earliest arrival past it, not })(); try { await flush(); - // Group 5 landed a second ago, so the holes before groups 3 and 5 both expired. + // Group 5 landed a second ago, so the holes before groups 3 and 5 both expired, and group 3 + // with them: its successor landed a budget ago. expect(results.map((read) => (read.kind === "end" ? `end ${read.group}` : read.kind))).toEqual([ "frame", "gap", - "end 3", - "gap", "end 5", ]); } finally { @@ -273,7 +284,7 @@ test.each([true, false])("the head after a declared end is waited for (arrives: const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(300)), { format: new Format("audio"), delay: { min: null, max: Duration.millis(300) }, }); @@ -353,7 +364,7 @@ test("a newest-first flush past a hole delivers the groups landing behind its ne const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(80)), { format: new Format("audio"), delay: { min: null, max: Duration.millis(80) }, }); @@ -401,12 +412,13 @@ test("a newest-first flush past a hole delivers the groups landing behind its ne } }); -// A player waits on a hole for its playout delay, while groups that did arrive play until the max. -test("a hole is waited for the wait while an arrived group plays up to the max delay", async () => { +// A hole and a group that did arrive both wait out the max delay: the min delay only sets where the +// reader starts. +test("a hole is waited for the max delay, as an arrived group plays up to it", async () => { const clock = new Manual(); const installed = clock.install(); const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(1000)), { format: new Format("audio"), delay: { min: Duration.millis(100), max: Duration.millis(1000) }, }); @@ -439,35 +451,27 @@ test("a hole is waited for the wait while an arrived group plays up to the max d await flush(); expect(reads()).toEqual([0, 500]); - // A missing group expires the wait after the first group past it arrived. + // A missing group is waited for the max delay after the first group past it arrived, not the min. write(3n, 520); await flush(); - await clock.advance(Duration.millis(99)); + await clock.advance(Duration.millis(999)); expect(reads()).toEqual([0, 500]); await clock.advance(Duration.millis(1)); expect(reads()).toEqual([0, 500, "gap", 520]); - // Content buffered past the wait doesn't skip it early: a burst may still carry the group. - write(5n, 700); - write(6n, 800); - await flush(); - expect(reads()).toEqual([0, 500, "gap", 520]); - await clock.advance(Duration.millis(100)); - expect(reads()).toEqual([0, 500, "gap", 520, "gap", 700, 800]); - // It skips at once when the newest content is already the max delay past the group after it. - write(8n, 900); - write(9n, 1900); + write(5n, 900); + write(6n, 1900); await flush(); - expect(reads()).toEqual([0, 500, "gap", 520, "gap", 700, 800, "gap", 900, 1900]); + expect(reads()).toEqual([0, 500, "gap", 520, "gap", 900, 1900]); - // A burst past the wait that carries the missing group reads with no gap. - write(11n, 2000); - write(12n, 2200); + // A burst past the hole that carries the missing group reads with no gap. + write(8n, 2000); + write(9n, 2200); await flush(); - write(10n, 1950); + write(7n, 1950); await flush(); - expect(reads()).toEqual([0, 500, "gap", 520, "gap", 700, 800, "gap", 900, 1900, 1950, 2000, 2200]); + expect(reads()).toEqual([0, 500, "gap", 520, "gap", 900, 1900, 1950, 2000, 2200]); } finally { consumer.close(); track.finish(); @@ -480,24 +484,14 @@ test("a hole is waited for the wait while an arrived group plays up to the max d } }); -test.each([ - ["the playout clock reaching the group past it", 900, 100], - ["the wait, while the clock trails", 0, 500], - ["the wait, without a clock", undefined, 500], -])("a hole ends at %s", async (_, start, after) => { +// The min delay sets only where the reader starts: a hole waits out the max delay alone. +test("a hole ends at the max delay, not the min delay", async () => { const clock = new Manual(); const installed = clock.install(); - const origin = now(); - // A playout clock presenting `start` ms of media now, advancing with the manual clock. - const playhead = - start === undefined - ? undefined - : { now: () => Duration.millis(start).add(now().since(origin)), anchor: new Signal(0) }; const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(owed(track, Duration.millis(2_000)), { format: new Format("audio"), delay: { min: Duration.millis(500), max: Duration.millis(2_000) }, - playhead, }); const results: Read[] = []; const reading = (async () => { @@ -522,7 +516,7 @@ test.each([ // A resume: the groups past the hole start a second of media later, inside the max delay. write(50n, 1_000); await flush(); - await clock.advance(Duration.millis(after - 1)); + await clock.advance(Duration.millis(1_999)); expect(results).toHaveLength(1); await clock.advance(Duration.millis(1)); expect(results.slice(1).map((read) => read.kind)).toEqual(["gap", "frame"]); @@ -537,54 +531,3 @@ test.each([ } } }); - -test("a hole ends once the playout clock jumps past the group past it", async () => { - const clock = new Manual(); - const installed = clock.install(); - // A clock trailing the group past the hole until it re-anchors a second later. - let position = Duration.ZERO; - const anchor = new Signal(0); - const track = new Track.Producer("audio"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { - format: new Format("audio"), - delay: { min: Duration.millis(500), max: Duration.millis(2_000) }, - playhead: { now: () => position, anchor }, - }); - const results: Read[] = []; - const reading = (async () => { - for (;;) { - const read = await consumer.next(); - if (!read) return; - results.push(read); - } - })(); - const write = (sequence: bigint, millis: number) => { - const group = track.createGroup(sequence); - group.writeFrame({ - payload: encodeFrame(new Uint8Array([1]), Duration.millis(millis)), - timestamp: Time.Timestamp.fromMicros(millis * 1000), - }); - group.finish(); - }; - try { - write(0n, 0); - await flush(); - write(50n, 1_000); - await flush(); - await clock.advance(Duration.millis(10)); - expect(results).toHaveLength(1); - position = Duration.millis(1_000); - anchor.set(1); - await flush(); - expect(results.slice(1).map((read) => read.kind)).toEqual(["gap", "frame"]); - } finally { - consumer.close(); - track.finish(); - track.close(); - try { - await reading; - } finally { - installed.close(); - } - } -}); diff --git a/js/hang/src/container/consumer.nonsequential.test.ts b/js/hang/src/container/consumer.nonsequential.test.ts index cce3350b1d..7a90aaa3cc 100644 --- a/js/hang/src/container/consumer.nonsequential.test.ts +++ b/js/hang/src/container/consumer.nonsequential.test.ts @@ -57,8 +57,14 @@ test("Consumer waits on a non-sequential hang group because it has no per-frame // A hang group can only promote early once its end is known, e.g. via an end-of-group signal such // as a trailing frame carrying the group's end timestamp. Until the container carries that, expect // the wait. The CMAF path (per-sample duration in the moof) is covered in consumer.test.ts. + // An unserved fill owes the missing groups, as a session's feed would, so the ordered read waits + // for them up to the max delay. const track = new Track.Producer("test"); - const consumer = new Consumer(track.subscribe(), { + track.requests(); + const subscriber = track.subscribe({ delay: { max: Duration.millis(5000) } }); + // Read from group A on, so nothing below it is owed. + subscriber.withRange({ start: { included: 1_000_000n } }); + const consumer = new Consumer(subscriber, { format: new LegacyFormat("data"), delay: { min: null, max: Duration.millis(5000) }, }); @@ -140,21 +146,25 @@ test("Consumer reads on without a gap past a group it skipped", async () => { // groups sequenced by their starts, as a Rust publisher numbers them, then run back to back without // waiting on the sequences between them, while a missing one is still a gap. test("Consumer reads one-packet Opus groups at sparse sequences back to back", async () => { + // An unserved fill owes the missing groups, as a session's feed would. const track = new Track.Producer("test"); + track.requests(); const config = AudioConfigSchema.parse({ codec: "opus", container: { kind: "legacy" }, sampleRate: 48_000, numberOfChannels: 2, }); - const consumer = new Consumer(track.subscribe(), { + const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(5000) } }), { format: new LegacyFormat(config), delay: { min: null, max: Duration.millis(5000) }, }); // One 20 ms CELT packet per group, each read before the next lands. The group starting at 40 ms // never arrives. const packet = new Uint8Array([31 << 3, 0]); - const publish = (start: number) => { + // Appending declares the starts between groups skipped; `after` is the start of the group before. + const publish = (start: number, after?: number) => { + if (after !== undefined) track.skipGroups(BigInt(after + 1), BigInt(start)); const group = track.createGroup(BigInt(start)); group.writeFrame({ payload: encodeLegacyFrame(start, packet), timestamp: Time.Timestamp.fromMicros(start) }); group.finish(); @@ -163,11 +173,12 @@ test("Consumer reads one-packet Opus groups at sparse sequences back to back", a publish(0); expect(await poll(consumer.next())).toMatchObject({ kind: "frame", group: 0n }); const next = consumer.next(); - publish(20_000); + publish(20_000, 0); expect(await poll(next)).toMatchObject({ kind: "frame", group: 20_000n }); // Nothing proves the timeline unbroken past 40 ms, so the reader waits on the missing group. const after = consumer.next(); - publish(60_000); + // The publisher appended a group at 40 ms, which never arrives. + publish(60_000, 40_000); expect(await poll(after)).toBe("pending"); consumer.close(); diff --git a/js/hang/src/container/consumer.outoforder.test.ts b/js/hang/src/container/consumer.outoforder.test.ts index e20ab70d2d..630d3fbc22 100644 --- a/js/hang/src/container/consumer.outoforder.test.ts +++ b/js/hang/src/container/consumer.outoforder.test.ts @@ -71,19 +71,17 @@ async function drain(consumer: Consumer): Promise<[bigint, number | undefined][] } // Groups go out newest first, so the head of a subscription's window arrives after the live edge -// it was served alongside. A cursor taken from the first arrival would drop that head, so it starts -// at the floor the subscription names, or the start the publisher reports, and waits for it under -// the budget. A group below that floor is never delivered and adds no gap. Mirrors Rust's -// `startup_at_the_publishers_start_over_a_newer_first_arrival`. +// it was served alongside. The ordered read starts at the floor the subscription names and waits for +// it under the budget. A group below that floor is never delivered and adds no gap. test("the head of the window arriving after the live edge is delivered first", async () => { const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI, maxAge: Duration.millis(30_000), }); - // A local track holding no group at or before the anchor resolves no floor from it, so the - // publisher's start names the head, as a session declares it. - const subscriber = track.subscribe({ delay: { max: Duration.millis(5000) }, ranges: [{ start: { time: 10n } }] }); - subscriber.start = async () => 10n; + // An unserved fill owes the window's head, as a session's feed would. + track.requests(); + const subscriber = track.subscribe({ delay: { max: Duration.millis(5000) } }); + subscriber.withRange({ start: { included: 10n } }); const consumer = new Consumer(subscriber, { format: new LegacyFormat("data"), delay: { min: null, max: Duration.millis(5000) }, @@ -101,11 +99,10 @@ test("the head of the window arriving after the live edge is delivered first", a publish(track, 11n, 11); track.finish(); - const reads: string[] = []; - for (let read = await waiting; read; read = await consumer.next()) { - reads.push(read.kind === "frame" ? `${read.group}` : read.kind); - } - expect(reads).toEqual(["10", "11", "12"]); + const head = await waiting; + const reads = [head?.kind === "frame" ? head.group : head?.kind]; + for (const [group] of await drain(consumer)) reads.push(group); + expect(reads).toEqual([10n, 11n, 12n]); consumer.close(); }); @@ -171,29 +168,3 @@ test("an excluded gap between ranges does not stall", async () => { ]); consumer.close(); }); - -// A relative floor already resolved here still rises to where the publisher says its feed starts, -// which counts a newest group that held no frame yet, rather than wait under the budget for a group -// below it that never comes. Mirrors Rust's `a_resolved_relative_floor_rises_to_the_publishers_start`. -test("a resolved relative floor rises to the publisher's start", async () => { - const track = new Track.Producer("test").accept({ - timescale: Time.Timescale.MILLI, - maxAge: Duration.millis(30_000), - }); - for (const sequence of [0n, 1n, 2n, 3n, 4n]) publish(track, sequence, Number(sequence) * 100); - const subscriber = track.subscribe({ - delay: { max: Duration.millis(10_000) }, - ranges: [{ start: { relative: 0n } }], - }); - expect(subscriber.floor()).toBe(4n); - // A session declares the publisher's start; a local track has none, so stand in for it. - subscriber.start = async () => 5n; - const consumer = new Consumer(subscriber, { - format: new LegacyFormat("data"), - delay: { max: Duration.millis(10_000) }, - }); - - publish(track, 5n, 500); - expect(await drain(consumer)).toEqual([[5n, 5]]); - consumer.close(); -}); diff --git a/js/hang/src/container/consumer.skips.test.ts b/js/hang/src/container/consumer.skips.test.ts index b4bf2c25c9..79d3e170e5 100644 --- a/js/hang/src/container/consumer.skips.test.ts +++ b/js/hang/src/container/consumer.skips.test.ts @@ -193,6 +193,8 @@ test("Consumer measures each skip's span over every group left, not just the new test("a blocking group is kept until its successor starts a budget behind the newest", async () => { const warn = spyOn(console, "warn").mockImplementation(() => {}); const track = new Track.Producer("tone"); + // An unserved fill owes the groups in flight, as a session's feed would. + track.requests(); const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { format: new Format("data"), delay: { min: null, max: Duration.millis(150) }, diff --git a/js/hang/src/container/consumer.startup.test.ts b/js/hang/src/container/consumer.startup.test.ts index 024d1d8f5b..b30a8704c9 100644 --- a/js/hang/src/container/consumer.startup.test.ts +++ b/js/hang/src/container/consumer.startup.test.ts @@ -5,13 +5,14 @@ import { type Group, Error as NetError, StreamCode, Time, Track } from "@moq/net import { Consumer, type Read } from "./consumer.ts"; import { encodeFrame, Format } from "./legacy.ts"; +// A consumer at `maxDelay` on a local track, which owes no missing group until a fill handler opens. function setup(maxDelay = 100) { const clock = new Manual(); const installed = clock.install(); // Drain the in-memory readers and their continuations without moving time. const flush = () => clock.advance(Duration.ZERO); const track = new Track.Producer("video"); - const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(30_000) } }), { + const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(maxDelay) } }), { format: new Format("video"), delay: { min: null, max: Duration.millis(maxDelay) }, }); @@ -111,13 +112,16 @@ test("Consumer preserves a declared marker before its first media across a hole" expect(warn).not.toHaveBeenCalled(); }); -test.each([false, true])("Consumer still waits for a gap after media (playhead reset: %s)", async (reset) => { +test.each([false, true])("Consumer still waits for a gap after media (playhead reset: %p)", async (reset) => { const { track, consumer, results, warn, flush } = setup(1000); const first = track.createGroup(16n); write(first, 0); first.finish(); await flush(); expect(items(results)).toEqual(["16@0"]); + // From here its missing groups an unserved fill owes, as a session's feed would, so the ordered + // read waits out the max delay for each hole. + track.requests(); if (reset) { const marker = track.createGroup(17n); @@ -130,10 +134,13 @@ test.each([false, true])("Consumer still waits for a gap after media (playhead r const later = track.createGroup(sequence); write(later, 50_000); await flush(); - expect(consumer.buffered.peek().length).toBeGreaterThan(0); + // The ordered read holds the group past the hole until the hole resolves, so none of it is buffered. + expect(consumer.buffered.peek()).toEqual([]); expect(frames(results)).toHaveLength(1); - // Supply the missing, timestamp-contiguous bridge and prove the queued frame survives. + // Supply the missing, timestamp-contiguous bridge and prove the queued frame survives. The key + // below it was never produced, as an appending publisher declares. + track.skipGroups(sequence - 2n, sequence - 1n); const bridge = track.createGroup(sequence - 1n); write(bridge, 0); bridge.finish(); diff --git a/js/hang/src/container/consumer.test.ts b/js/hang/src/container/consumer.test.ts index c89a62a179..f410707cdb 100644 --- a/js/hang/src/container/consumer.test.ts +++ b/js/hang/src/container/consumer.test.ts @@ -107,6 +107,14 @@ async function poll(read: Promise): Promise { function replay(track: Track.Producer): Track.Subscriber { return track.subscribe({ delay: { max: Duration.millis(30_000) } }); } +// A subscription from group `from` on whose missing groups an unserved fill owes, as a session's +// feed would, so its ordered read waits `max` for each hole before giving it up. +function owed(track: Track.Producer, max: number, from: bigint): Track.Subscriber { + track.requests(); + const subscriber = track.subscribe({ delay: { max: Duration.millis(max) } }); + subscriber.withRange({ start: { included: from } }); + return subscriber; +} // A track at hang's microsecond timescale, as `trackInfo` pins it: the Legacy producer opens each // keyframe's group at its timestamp. No publisher max age, so a reader reaches superseded groups. function micro(name: string): Track.Producer { @@ -1779,7 +1787,7 @@ test("Consumer delivers a PTS-contiguous next group whose sequence jumped (Legac // Delivery resumes only once the missing, timeline-continuous group arrives. test("Consumer waits on a PTS gap instead of skipping to a later buffered group (Legacy)", async () => { const track = new Track.Producer("test"); - const consumer = new Consumer(replay(track), { + const consumer = new Consumer(owed(track, 10_000, 1_000_000n), { format: timedLegacy, delay: { min: null, max: Duration.millis(10_000) }, }); @@ -1797,7 +1805,10 @@ test("Consumer waits on a PTS gap instead of skipping to a later buffered group const pending = consumer.next(); // Group C (seq 1_090_000) starts at 90_000 ticks (1_000_000µs) -- far past A.end (33_333µs), a real - // gap. An intermediate group may still be in transit, so C must NOT be delivered yet. + // gap. An intermediate group may still be in transit, so C must NOT be delivered yet. Appending + // B and C declared the starts between the groups skipped, so B's alone is owed. + track.skipGroups(1_000_001n, 1_045_000n); + track.skipGroups(1_045_001n, 1_090_000n); const c = track.createGroup(1_090_000n); c.writeFrame({ payload: encodeLegacyFrame(Math.round((90_000 / TIMESCALE) * 1_000_000) as number, new Uint8Array([0x03])), @@ -1871,7 +1882,7 @@ test("Consumer delivers a contiguous group after one that completed out of order // old to play, so the head is the group to play next and must not be dropped as "slow". test("Consumer plays the head once a waited-out gap exceeds the budget (Legacy)", async () => { const track = new Track.Producer("test"); - const consumer = new Consumer(replay(track), { + const consumer = new Consumer(owed(track, 100, 1000n), { format: timedLegacy, delay: { min: null, max: Duration.millis(100) }, }); @@ -1906,7 +1917,7 @@ test("Consumer plays the head once a waited-out gap exceeds the budget (Legacy)" // A gap still inside the budget is waited out: the missing group may yet arrive in time. test("Consumer waits on a gap within the budget, then plays the head once it exceeds it (Legacy)", async () => { const track = new Track.Producer("test"); - const consumer = new Consumer(replay(track), { + const consumer = new Consumer(owed(track, 100, 1000n), { format: timedLegacy, delay: { min: null, max: Duration.millis(100) }, }); @@ -1926,12 +1937,16 @@ test("Consumer waits on a gap within the budget, then plays the head once it exc await clock.advance(Duration.millis(99)); expect(await poll(pending)).toBe("pending"); - // B alone grows past the budget (up to 12_000 ticks, 100ms past A's end and beyond). + // B alone grows past the budget (up to 12_000 ticks, 100ms past A's end and beyond). The ordered + // read measures the hole against the newest group's start, so that proves nothing yet. for (let i = 1; i <= 3; i++) { b.writeFrame(timedFrame(0x02, 6000 + i * 3000, 1 + i)); await flush(); } + expect(await poll(pending)).toBe("pending"); + // B landed a whole budget ago, so the wall clock gives the hole up. + await clock.advance(Duration.millis(1)); const result = await poll(pending); expect(result).toEqual({ kind: "gap", timestamp: Duration.micros(33_333), duration: Duration.micros(33_334) }); expect(frameOf(await consumer.next()).payload).toEqual(new Uint8Array([0x02])); diff --git a/js/hang/src/container/consumer.ts b/js/hang/src/container/consumer.ts index 3da434e5de..9b6b992bb6 100644 --- a/js/hang/src/container/consumer.ts +++ b/js/hang/src/container/consumer.ts @@ -18,20 +18,17 @@ export interface ConsumerProps { /** * How far behind the live edge to play (default: zero for both bounds). See `Moq.Track.Delay`. * - * The start is the group covering the live edge less `min`, plus any {@link ConsumerProps.warmup}, waiting - * for it when the newest group arrives first; an unstated (`null`) `min` reads as zero. A `min` - * past `max` starts at `max`, since the publisher serves nothing older. A missing group is waited - * for `min`, the reader's own buffer, since past it the group would play late, or for `max` when - * `min` is `null`, never longer than `max`. The wait expires this much monotonic time after the - * first group past it arrived, even when that group is a marker on a paused track, or sooner once - * where that group begins falls `max` behind the newest content. `max` is measured as - * the span from the oldest buffered frame to the newest, so it bounds how long a late group that - * did arrive is waited for, letting a player catch up on late media its holes skipped past. It is the local - * half of the subscription's `delay.max`; both measure the same budget, one on the wire and one - * as frames are read. It bounds a join's lateness too: a backlog that landed before the first - * read starts at the first group whose first frame is within it, plus any warmup, of the - * newest content, though the start keeps the group covering the live edge less `min` and the - * warmup. + * `min` is how far behind the live edge the subscription starts, plus any + * {@link ConsumerProps.warmup}: the publisher resolves a live join that far back, and the reader + * starts at the first group its ordered read hands out. An unstated (`null`) `min` reads as zero. + * `max` alone gives up a missing group: the ordered read waits for one until where the group past + * it begins falls `max` behind the newest content. `max` is also measured as the span from the + * oldest buffered frame to the newest, so it bounds how long a late group that did arrive is + * waited for. It is the local half of the subscription's `delay.max`; both measure the same + * budget, one on the wire and one as frames are read. It bounds a join's lateness too: a backlog + * that landed before the first read starts at the first group whose first frame is within it, + * plus any warmup, of the newest content, though the start keeps the group covering the live edge + * less `min` and the warmup. */ // Read-only: a Getter (e.g. another component's output) is accepted directly. delay?: GetterInit; @@ -55,16 +52,11 @@ export interface ConsumerProps { * Opus decoder. Pass the catalog audio rendition's `warmup`. Mirrors Rust's `container::Playout`. * * Put the min and max delay plus the warmup on the subscription, since the publisher starts a - * live join at the min and skips what falls outside the max. The start is no later than the group covering the live edge less - * `delay.min` and the warmup: one that starts at or before that point. Both default to zero, - * which the newest group covers, so the start waits for nothing. Otherwise groups can arrive - * newest first, so when none that arrived covers that point the start waits for an earlier one, - * until one covers it, group 0 arrived, the track finishes, the max buffer fills, or the newest - * group's arrival is that whole reach ago. The wall clock bounds a quiet or paused track, which - * never ages out in media time. Delivery then starts at the lowest group that arrived. Until the - * reader is the warmup past both its first frame and that point, the age skip keeps a group that - * covers the reach and tolerates the warmup, so the media leading up to the start is not dropped - * as stale. After that, the max delay alone governs it. + * live join at the min and skips what falls outside the max, so the first group the ordered read + * hands out covers the live edge less `delay.min` and the warmup. Until the reader is the warmup + * past its first frame, the age skip keeps a group that covers the reach and tolerates the + * warmup, so the media leading up to the start is not dropped as stale. After that, the max delay + * alone governs it. */ warmup?: Duration; /** @@ -75,32 +67,16 @@ export interface ConsumerProps { */ predecessor?: Consumer; /** - * Read the media in key order from this reader of the same track instead of `track`'s arrival - * order, so it decides every hole. It must be a second subscription, never `track`'s own, which - * keeps reading in arrival order to feed the estimate: a missing group is a hole once the reader reports it as a gap, - * and never waited for here. Its `delay.max` is then how long a hole is waited for, so size it to - * what the player can still play late. `track` only feeds the arrival estimate, read frame by frame - * as it lands, so give it a `delay.max` reaching as far as lateness can move the estimate. + * Read the media from this ordered reader of the same track rather than from `track` itself. It + * must be a second subscription, never `track`'s own, which then keeps reading in arrival order + * to feed the estimate: every frame as it lands, late media the ordered read gives up included. + * Without it the consumer reads `track.ordered()`, and the estimate learns from the frames that + * read hands out and from when each group it gives up as a join's backlog landed. Either way the + * ordered read decides every hole: a missing group is a hole once it reports a gap, and never + * waited for here. Its `delay.max` is how long a hole is waited for, so size it to what the player + * can still play late. */ ordered?: Moq.Track.Ordered; - /** - * The playout clock this reader feeds, such as the player's `Sync` (default: none). A missing - * group whose successor starts at or before the oldest media the clock still plays could only - * arrive too late to play, so it is skipped then rather than waited for: a reader resuming behind - * the live edge moves on once the clock passes the hole, not a whole `delay.min` later. - */ - playhead?: Playhead; -} - -/** A playout clock, which a {@link Consumer} reads to tell when a missing group could no longer play. */ -export interface Playhead { - /** - * The oldest media time the player still plays, or undefined while nothing presents: the media - * presenting now, less however far a player that catches up late media lets it trail. - */ - now(): Duration | undefined; - /** Changes whenever {@link Playhead.now} jumps rather than advancing with the wall clock: a re-anchor, a new delay, or a pause. */ - readonly anchor: Getter; } /** One item of a {@link Consumer.next} read, in delivery order. Mirrors Rust's `container::Read`. */ @@ -134,11 +110,8 @@ const GROUP_COST = 256; interface Group { consumer: Moq.Group.Consumer; - // From then on, any sequence missing below this group is late: the earliest arrival among it - // and the groups past it, kept on insert so a hole reads it without scanning the backlog. - late: Instant; frames: ReceivedFrame[]; // decode order - gap?: boolean; // An ordered reader gave up on groups just below this one. + gap?: boolean; // The ordered read gave up on groups just below this one. costs: number[]; // What each of frames holds against the max buffer empty: boolean; // no wire frame was published; empty groups mean nothing media: boolean; // a decodable (non-marker) frame was buffered @@ -200,31 +173,27 @@ function maxMicros(delay: Moq.Track.Delay): number { } /** - * Reads frames from a MoQ track in group order, buffering groups and skipping ones that fall the max - * delay behind. Delivery starts at the floor the subscription's ranges name, raised to where the - * publisher reports its feed starts, or with neither at the first group to arrive; a group that arrives - * below the cursor is never delivered. Read in arrival order from the track itself to keep every group. + * Reads frames from a MoQ track in group order through its ordered read, buffering groups and skipping + * ones that fall the max delay behind. The ordered read decides every hole and where delivery starts: + * at the first group it hands out. */ export class Consumer { #track: Moq.Track.Subscriber; - // The reader that decides every hole, when one does; `#track` then only feeds the estimate. - #ordered?: Moq.Track.Ordered; - // The ordered reader gave up on groups since the last one it returned. + // The read that decides every hole and delivers the media. + #ordered: Moq.Track.Ordered; + // `#track` is a second subscription read in arrival order, which alone feeds the estimate. + #observing: boolean; + // The ordered read gave up on groups since the last one it returned. #gap = false; + // When each gap given up before the join's boundary was known landed, by its first key: told to + // the estimate once the boundary is known, or dropped once the first group is handed out. + #unjoined?: { start: bigint; landed: Instant }[] = []; #format: Format; #delay: Getter; - #playhead?: Playhead; #maxBuffer: number; // How much further than the min delay the start reaches, and the age skip tolerates while warming. #warmup: Duration; - // The start may still wait for a group covering the reach, until that wait ends for good. - #starting = true; - // When the newest group had landed as the first one arrived, once the start waits. - #anchor?: Instant; - // The point the start reached back to, the live edge less the reach, once its wait ended. - #target?: number; - // Where the warming ends: the warmup past the first frame read, or past the start's target when - // that is later. Undefined before the first frame. + // Where the warming ends: the warmup past the first frame read. Undefined before the first frame. #warmed?: number; // What the buffered groups hold, against the max buffer. #used = 0; @@ -232,11 +201,6 @@ export class Consumer { #room: (() => void)[] = []; #groups: Group[] = []; #active?: bigint; // the active group sequence number - // The cursor started at the floor or the publisher's resolved start, rather than at the first - // arrival, so a hole at it is waited for like any other. - #floored = false; - // When the missing sequence at the cursor became late, while one is. - #hole?: Instant; // Presentation end (max PTS + duration) of the group we most recently advanced past, so next()'s // promotion guard can tell a timeline-continuous next group from one sitting after a gap. // Written only by #pass, which every move past a group goes through. @@ -301,36 +265,31 @@ export class Consumer { /** Start consuming the given track, decoding frames with `props.format`. */ constructor(track: Moq.Track.Subscriber, props: ConsumerProps) { this.#track = track; - this.#ordered = props.ordered; + this.#observing = props.ordered !== undefined; + const ordered = props.ordered ?? track.ordered(); + this.#ordered = ordered; this.#format = props.format; this.#delay = getter(props.delay ?? {}); - this.#playhead = props.playhead; this.#maxBuffer = props.maxBuffer ?? DEFAULT_MAX_BUFFER; this.#warmup = (props.warmup ?? Duration.ZERO).min(DELAY_MAX); this.#jitter = props.predecessor ? props.predecessor.#jitter.fork() : new Jitter(); this.#spread = new Signal(Duration.millis(this.#jitter.measured)); this.spread = this.#spread; - // Where the publisher stood as the subscription began, read as soon as its answer resolves the - // start: after it rather than from a cached group that raced it, and before live groups landing - // later raise the largest past it. The start says only once the info or a first group shows the - // subscription established. Off the delivery path: a range capped below the largest never - // reports one, which reports no backlog rather than stalling delivery. - const ordered = this.#ordered; - if (ordered) { - this.#signals.spawn(this.#runOrdered.bind(this, ordered)); - this.#signals.spawn(this.#observe.bind(this)); - } else { - this.#signals.spawn(async () => { - await race([this.#track.info().catch(() => undefined), this.#tuned]); - await this.#track.start(); - this.#joined = (await this.#track.largest())?.group; - }); - this.#signals.spawn(this.#run.bind(this)); - } + // Where the publisher stood as the subscription began, read once its answer shows the + // subscription established: after it rather than from a cached group that raced it, and before + // live groups landing later raise the largest past it. Off the delivery path: a range capped + // below the largest never reports one, which reports no backlog rather than stalling delivery. + this.#signals.spawn(async () => { + await race([ordered.info().catch(() => undefined), this.#tuned]); + this.#joined = (await ordered.largest())?.group; + this.#joinBacklog(); + }); + this.#signals.spawn(this.#run.bind(this, ordered)); + if (this.#observing) this.#signals.spawn(this.#observe.bind(this)); this.#signals.cleanup(() => { - this.#track.close(); - this.#ordered?.close(); + if (this.#observing) this.#track.close(); + this.#ordered.close(); for (const group of this.#groups) { group.consumer.close(); } @@ -346,66 +305,11 @@ export class Consumer { this.#notify = undefined; } - async #run() { - // Start fetching groups in the background - try { - for (;;) { - // Past the max buffer, later groups wait in the transport's cache, once the head has a successor - // to move on to: however small the cap, a stalled head can always be skipped. A missing group - // at the cursor then expires like one past the budget: see next(). - while (this.#full() && this.#groups.length >= 2 && !this.#signals.abort.aborted) await this.#park(); - if (this.#signals.abort.aborted) break; - - const consumer = await this.#track.recvGroup(); - if (!consumer) break; - // Closed while the read waited: cleanup already ran, so nothing would release it. - if (this.#signals.abort.aborted) { - consumer.close(); - break; - } - - // Delivery starts at the floor the ranges name, raised to where the publisher says its feed - // starts: it serves nothing below, and resolved a relative start against its own latest - // group, which may hold no frame yet and so count in no floor resolved here. Groups go out - // newest first, so the first arrival is not the oldest served; without either it stands - // in. Mirrors Rust's `Consumer::poll_read`. - if (this.#active === undefined) { - this.#tuned.set(true); - const floor = this.#track.floor(); - const served = await this.#track.start(); - const start = served !== undefined && (floor === undefined || served > floor) ? served : floor; - this.#active = this.#startAt(start) ?? consumer.start; - } - - // A group below the cursor is never delivered: what lies below is already covered by the - // gap that moved the cursor, or was never read. Mirrors Rust's consumer. While the start - // still waits for a group covering its reach, it starts at the lowest that arrived. - if (!this.#starting && consumer.start < this.#active) { - // Its frames go unobserved, so the estimator learns when backlog arrived: a join's - // intervals count only once it has. One produced since, such as an outage's, is no backlog. - if (this.#joined !== undefined && consumer.start <= this.#joined) { - this.#jitter.backlog(consumer.arrived.since(Instant.ZERO).asMillis()); - } - consumer.close(); - continue; - } - this.#add(consumer); - } - if (!this.#signals.abort.aborted) { - this.#delivered = true; - this.#settle(); - } - this.#finish(null); - } catch (err) { - this.#finish(err instanceof Error ? err : new Error(String(err))); - } - } - // Read groups in key order from `ordered`, noting each gap it reports on the group after it. - async #runOrdered(ordered: Moq.Track.Ordered) { + async #run(ordered: Moq.Track.Ordered) { try { for (;;) { - // As #run: past the max buffer, later groups wait in the transport's cache. + // Past the max buffer, later groups wait in the transport's cache. while (this.#full() && this.#groups.length >= 2 && !this.#signals.abort.aborted) await this.#park(); if (this.#signals.abort.aborted) break; @@ -416,16 +320,18 @@ export class Consumer { item.group?.close(); break; } + if (this.#tuned.peek() === undefined) this.#tuned.set(true); if (item.kind === "gap") { this.#gap = true; + if (item.landed) this.#backlog(item.start, item.landed); continue; } // Delivery starts at the first group: the reader already skipped what is too stale to // play, and waited for the groups below it that its feed owed. if (this.#active === undefined) { - this.#tuned.set(true); this.#active = item.group.start; - this.#starting = false; + // What the join's boundary has not judged by now it never judges. + this.#unjoined = undefined; } this.#add(item.group, this.#gap); this.#gap = false; @@ -440,12 +346,31 @@ export class Consumer { } } + // Tell the estimate when groups the ordered read gave up from `start` on landed, if the publisher's + // largest as the subscription began reached them: they were produced before it, a join's backlog, + // so the estimate learns when the backlog finished arriving though nothing of it was read. One + // produced since, such as an outage's, is no backlog. The estimate observes every arrival itself + // when `#track` feeds it. + #backlog(start: bigint, landed: Instant): void { + if (this.#observing) return; + if (this.#joined === undefined) { + this.#unjoined?.push({ start, landed }); + return; + } + if (start <= this.#joined) this.#jitter.backlog(landed.since(Instant.ZERO).asMillis()); + } + + // The join's boundary is known: judge the gaps given up before it was. + #joinBacklog(): void { + const unjoined = this.#unjoined ?? []; + this.#unjoined = undefined; + for (const { start, landed } of unjoined) this.#backlog(start, landed); + } + // Buffer a group in sequence order and watch it for frames. #add(consumer: Moq.Group.Consumer, gap = false): void { const group: Group = { consumer, - // When the group landed, not when it was pulled: a full max buffer delays pulls. - late: consumer.arrived, frames: [], costs: [], gap, @@ -456,16 +381,8 @@ export class Consumer { }; this.#charge(group, GROUP_COST); - // Insert into #groups based on the group sequence number (ascending). - // This is used to cancel old groups. - let index = this.#groups.findIndex((other) => other.consumer.start > consumer.start); - if (index === -1) index = this.#groups.length; - // Each group is late from the earliest arrival among it and the groups past it. A drained - // backlog comes in sequence order, not arrival order, so an arrival can lower those below it. - const next = this.#groups.at(index); - if (next?.late.lt(group.late)) group.late = next.late; - for (let i = index - 1; i >= 0 && this.#groups[i].late.gt(group.late); i--) this.#groups[i].late = group.late; - this.#groups.splice(index, 0, group); + // The ordered read hands groups out in key order, so each goes last. + this.#groups.push(group); // Group overhead alone can fill the max buffer, past which next() moves on from a stalled head. if (this.#full()) { this.#notify?.(); @@ -475,8 +392,8 @@ export class Consumer { this.#signals.spawn(this.#watch.bind(this, group)); } - // Feed the arrival estimate from every frame of `#track` as it lands, while an ordered reader - // delivers: late media that reader gives up on must still move the estimate, or it could never rise. + // Feed the arrival estimate from every frame of `#track` as it lands, while the ordered read + // delivers: late media that read gives up on must still move the estimate, or it could never rise. async #observe() { try { for (;;) { @@ -612,16 +529,6 @@ export class Consumer { } } - // Start the cursor at `start`, when there is one, rather than at the first arrival: the start no - // longer waits for a group covering the reach, since nothing below the start arrives. - #startAt(start: bigint | undefined): bigint | undefined { - if (start === undefined) return undefined; - this.#active = start; - this.#floored = true; - this.#starting = false; - return start; - } - // Note an end the publisher declared in group `sequence`, which a later group's first frame resumes from. #declare(sequence: bigint): void { if (this.#mediaSequence !== undefined && sequence < this.#mediaSequence) return; @@ -652,7 +559,7 @@ export class Consumer { return; } group.bare = decoded.length === 0; - if (group.bare && !this.#ordered) this.#declare(group.consumer.start); + if (group.bare && !this.#observing) this.#declare(group.consumer.start); } // Decoded payloads are views of the wire payload, which stays allocated until the last of them is @@ -661,7 +568,7 @@ export class Consumer { for (const sample of decoded) { const marker = this.#format.end?.(sample) !== undefined; // Observed here, as it comes off the transport, unless #observe sees every arrival instead. - if (!this.#ordered) this.#note(group.consumer.start, marker, group.media, sample.timestamp, arrived); + if (!this.#observing) this.#note(group.consumer.start, marker, group.media, sample.timestamp, arrived); const frame: ReceivedFrame = { arrival, payload: sample.payload, @@ -814,12 +721,6 @@ export class Consumer { } if (skipped) { - // What the start would wait for is older than the reach, so it is past. The reader still - // warms until the start's target, as it would had the wait ended on its own. - if (this.#starting) { - this.#starting = false; - this.#target = this.#startTarget(this.#reach(this.#delay.peek())); - } console.warn( `skipping slow groups: track=${this.#track.name} ${start} -> ${this.#active} count=${skipped}`, ); @@ -978,8 +879,8 @@ export class Consumer { if (this.#error) return; this.#error = error; this.#finish(error); - this.#track.close(error); - this.#ordered?.close(error); + if (this.#observing) this.#track.close(error); + this.#ordered.close(error); this.#notify?.(); this.#notify = undefined; // Nothing more is read, so stop the group watchers and release their groups rather than let a @@ -1010,9 +911,6 @@ export class Consumer { if (this.#error) throw this.#error; const ended = this.#closed.peek(); - if (this.#groups[0]?.consumer.start === this.#active || this.#groups.length === 0) { - this.#hole = undefined; - } if (this.#groups.length === 0) { if (ended !== undefined) { if (ended instanceof Error) throw ended; @@ -1020,88 +918,40 @@ export class Consumer { } } - // The start may wait for a group covering the reach, the min delay plus the warmup. - const starting = this.#reachStart(ended !== undefined); - - // If #active points below all buffered groups -- e.g. a finished group's `+ 1` - // fallback fired because no later group was buffered yet, and the real (large-gap, - // non-sequential) next group has since arrived -- promote #active to the first real - // group so delivery resumes instead of stalling on a nonexistent sequence. - // Before any media is delivered there is no timeline hole to wait for, so this exception - // only applies at startup. - // Otherwise promote #active when the first buffered group continues the timeline we left off at, - // when a completed empty group can be walked (empty groups mean nothing), or when the hole - // is proven: everything a missing group could still present ends where the first group past - // it begins, so once that start falls the max delay behind the newest content, all of it - // would arrive too late to play. Mirrors Rust's `aged`. Where presentation left off proves - // only the oldest of it stale, and groups go out newest first, so a flush's newest group lands - // just ahead of the groups behind it, which would then arrive below the cursor and never play. - // Proving it here matters: #checkMaxDelay would instead drop the head, the very group to play - // next. - // After track termination no missing group can arrive, so drain across any remaining gap. - // A missing sequence also expires on elapsed monotonic time, so timestamp-free markers - // behind a hole can end a paused track without more arrivals. It is late since a group - // past it arrived, not since the cursor got here: once the edge stops moving, a backlog - // with every other group missing would otherwise drain one budget per hole. And once the - // playout clock reaches where the first group past it begins, nothing missing can still play. - if (starting === undefined && this.#active !== undefined && this.#groups.length > 0) { + // The ordered read hands a group out only once it resolved every key below it, so the cursor + // steps onto the next buffered group once it can tell whether the step lost media. Nothing is + // lost where the publisher declared the keys between skipped, where the group continues the + // timeline within 1 ms, or onto an empty group. Anything else is a loss, whether the ordered + // read gave the keys up as a gap or stepped over them as a local track does, which owes + // nothing and reports none: without a gap, the step waits for the group's first frame or its + // end to show where it starts. The cursor below every buffered group is also where a finished + // group's `+ 1` fallback leaves it when the next group numbers non-sequentially. + if (this.#active !== undefined && this.#groups.length > 0) { const head = this.#groups[0]; if (head.consumer.start > this.#active) { - this.#hole = head.late; - // Joining at the first arrival leaves no hole to wait for; a floor does. - const startup = this.#start === undefined && !this.#floored; - // The publisher declares the starts it never produced, so a run it skipped holds no group - // to wait for or lose, as in Rust's `poll_continues`. + const first = head.frames.at(0)?.timestamp.asMicros(); const contiguous = - this.#track.isSkipped(this.#active, head.consumer.start) || - ptsContiguous(this.#presentedEnd, head.frames.at(0)?.timestamp.asMicros()); + this.#ordered.isSkipped(this.#active, head.consumer.start) || + ptsContiguous(this.#presentedEnd, first); const empty = head.empty && head.consumer.done; - const wait = this.#holeWait(this.#delay.peek()); - const bound = this.#bound(); - const edge = this.#liveEdge(); - // Measured against the max delay, not the wait: a reader starved of CPU takes in a burst of - // groups at once, and the missing one may land in the same burst. - const aged = - bound !== undefined && edge !== undefined && edge - bound >= maxMicros(this.#delay.peek()); - const elapsed = now().since(this.#hole).asMicros() >= wait; - const passed = this.#passed(bound) === 0; - // The ranges ask for nothing in between, so nothing there is owed. - const next = this.#track.nextInRange(this.#active); - const excluded = next === undefined || next >= head.consumer.start; - // The groups past the hole already fill the max buffer, so it can hold none of what would arrive into it. - const full = this.#full(); - // An ordered reader returns a group only once it resolved every key below it. - const decided = this.#ordered !== undefined; - if ( - decided || - startup || - empty || - contiguous || - aged || - elapsed || - passed || - excluded || - full || - ended !== undefined - ) { - // What an ordered reader gives up it reports as a gap; a key it skips without one was never a - // group. A gap an empty group carries still marks the next frame, which follows the loss. - if (!contiguous && (decided ? head.gap : !empty)) { + const known = + contiguous || head.gap || first !== undefined || head.consumer.done || ended !== undefined; + if (known) { + if (!contiguous && (head.gap || !empty)) { this.#lost = true; - if (!startup) { + // Before the first media, nothing was read to lose. + if (this.#start !== undefined) console.warn( `skipping missing groups: track=${this.#track.name} ${this.#active} -> ${head.consumer.start}`, ); - } } this.#active = head.consumer.start; - this.#hole = undefined; } } } const head = this.#groups.at(0); - if (starting === undefined && head && this.#active !== undefined && head.consumer.start <= this.#active) { + if (head && this.#active !== undefined && head.consumer.start <= this.#active) { const seq = head.consumer.start; const frame = head.frames.shift(); if (frame) { @@ -1118,9 +968,8 @@ export class Consumer { } this.#mediaGroup = seq; - // A group covering the target began before it, so the reader warms until the target too. const at = frame.timestamp.asMicros(); - this.#warmed ??= Math.max(at, this.#target ?? at) + this.#warmup.asMicros(); + this.#warmed ??= at + this.#warmup.asMicros(); if (this.#start === undefined || seq > this.#start.group) { this.#floor = this.#start?.timestamp; this.#start = { group: seq, timestamp: at }; @@ -1184,27 +1033,6 @@ export class Consumer { const abort = this.#signals.abort; if (abort.aborted) return undefined; - let timer: Effect | undefined; - const since = starting ?? this.#hole; - if (since !== undefined) { - timer = new Effect(); - timer.run((effect) => { - const delay = effect.get(this.#delay); - // A clock that jumps moves when it reaches the group past a hole. - if (this.#playhead) effect.get(this.#playhead.anchor); - // The start waits out its reach, and a hole the wait, which an unlimited one never does. - const budget = starting === undefined ? this.#holeWait(delay) : this.#reach(delay); - let left = Number.isFinite(budget) - ? Duration.micros(budget).saturatingSub(now().since(since)) - : undefined; - // A hole also ends once the playout clock reaches the first group past it. - const passing = starting === undefined ? this.#passed(this.#bound()) : undefined; - if (passing !== undefined) left = Duration.micros(Math.min(passing, left?.asMicros() ?? passing)); - if (left === undefined) return; - effect.timer(() => this.#notify?.(), left); - }); - } - const aborted = await new Promise((resolve) => { const onAbort = () => resolve(true); abort.addEventListener("abort", onAbort, { once: true }); @@ -1213,93 +1041,23 @@ export class Consumer { resolve(false); }; }); - timer?.close(); - this.#notify = undefined; if (aborted) return undefined; } } - // Where the first stamped buffered group begins, which bounds everything a missing group before it presents. - #bound(): number | undefined { - return this.#groups.find((group) => group.start !== undefined)?.start; - } - - // How long until the playout clock reaches `bound`, in microseconds: zero once it has, undefined - // without a clock that is presenting or without a bound. - #passed(bound: number | undefined): number | undefined { - const playhead = this.#playhead?.now()?.asMicros(); - if (bound === undefined || playhead === undefined) return undefined; - return Math.max(0, bound - playhead); - } - - // How long a missing group is waited for, in microseconds: the min delay, the reader's own buffer, - // or the max delay when the min is unstated, at most the max delay. Mirrors Rust's `hole_wait`. - #holeWait(delay: Moq.Track.Delay): number { - const max = maxMicros(delay); - return delay.min === null ? max : Math.min(delay.min?.asMicros() ?? 0, max); - } - // How far behind the live edge the start reaches, in microseconds: the min delay plus the warmup. // The publisher serves nothing older than the max delay, so a min delay past it reaches no further. #reach(delay: Moq.Track.Delay): number { return Math.min(delay.min?.asMicros() ?? 0, maxMicros(delay)) + this.#warmup.asMicros(); } - // While the start still waits for a group covering the live edge less the min delay and the - // warmup, when the newest group had landed as the first one arrived: the wait ends a reach after - // it. Undefined once the start is free to go, for good. - #reachStart(ended: boolean): Instant | undefined { - if (!this.#starting) return undefined; - // Nothing has a wire frame yet, so nothing can start either. - if (!this.#groups.some((group) => !group.empty)) return undefined; - - const reach = this.#reach(this.#delay.peek()); - // The newest group covers a zero reach. - if (reach === 0) { - this.#started(); - return undefined; - } - this.#anchor ??= this.#groups[this.#groups.length - 1].consumer.arrived; - const target = this.#startTarget(reach); - const covered = - target !== undefined && this.#groups.some((group) => group.start !== undefined && group.start <= target); - // Nothing earlier can arrive or be held. - const lowest = this.#groups[0].consumer.start === 0n; - // Wall time: the newest group landed a whole reach ago. - const elapsed = now().since(this.#anchor).asMicros() >= reach; - if (ended || this.#full() || target === undefined || covered || lowest || elapsed) { - this.#started(); - this.#target = target; - return undefined; - } - return this.#anchor; - } - - // The start's wait ended: delivery starts at the lowest group that arrived, as Rust's. - #started(): void { - this.#starting = false; - this.#active = this.#groups[0]?.consumer.start ?? this.#active; - } - // The point the start reaches back to: the live edge (the newest content) less `reach`. Undefined - // for a zero reach, which the newest group covers, and before any timed content, since untimed - // content such as a marker has no edge to reach back from. - // A caller that already knows the live edge passes it, sparing a scan of every group. - #startTarget(reach: number, known?: number): number | undefined { + // for a zero reach, which the newest group covers. + #startTarget(reach: number, edge: number): number | undefined { if (reach === 0) return undefined; - const edge = known ?? this.#liveEdge(); // Media starts at zero, so a reach past it is covered there. - return edge === undefined ? undefined : Math.max(0, edge - reach); - } - - // The newest known frame timestamp across the buffered groups. - #liveEdge(): number | undefined { - let edge: number | undefined; - for (const group of this.#groups) { - if (group.latest !== undefined && (edge === undefined || group.latest > edge)) edge = group.latest; - } - return edge; + return Math.max(0, edge - reach); } // How stale the buffered span may get before the age skip drops its oldest group, in microseconds: @@ -1309,7 +1067,7 @@ export class Consumer { return this.#warming() ? maxDelay + this.#warmup.asMicros() : maxDelay; } - // The reader has not yet read past the start's target and the warmup past its first frame. + // The reader has not yet read the warmup past its first frame. #warming(): boolean { return this.#warmed === undefined || (this.#edge ?? 0) < this.#warmed; } diff --git a/js/hang/src/container/consumer.warmup.test.ts b/js/hang/src/container/consumer.warmup.test.ts index 5436cc9eb5..d57b05e4e2 100644 --- a/js/hang/src/container/consumer.warmup.test.ts +++ b/js/hang/src/container/consumer.warmup.test.ts @@ -8,21 +8,23 @@ import { encodeFrame, Format } from "./legacy.ts"; const BUDGET = 500; // A consumer that starts `min` plus `warmup` behind the live edge and skips at `max`, `null` for no limit. -function setup(delay = BUDGET, warmup = 0, min = delay, max: number | null = delay) { +// With `owed`, the subscription reads from that group, which an unserved fill owes as a session's feed +// would, so the ordered read waits for it. +function setup(delay = BUDGET, warmup = 0, min = delay, max: number | null = delay, owed?: bigint) { const clock = new Manual(); const installed = clock.install(); // Drain the in-memory readers and their continuations without moving time. const flush = () => clock.advance(Duration.ZERO); const track = new Track.Producer("video"); + if (owed !== undefined) track.requests(); // A null max delay sets no subscriber limit; an omitted one is zero. - const consumer = new Consumer( - track.subscribe({ delay: { max: max === null ? null : Duration.millis(max + warmup) } }), - { - format: new Format("video"), - delay: { min: Duration.millis(min), max: max === null ? null : Duration.millis(max) }, - warmup: Duration.millis(warmup), - }, - ); + const subscriber = track.subscribe({ delay: { max: max === null ? null : Duration.millis(max + warmup) } }); + if (owed !== undefined) subscriber.withRange({ start: { included: owed } }); + const consumer = new Consumer(subscriber, { + format: new Format("video"), + delay: { min: Duration.millis(min), max: max === null ? null : Duration.millis(max) }, + warmup: Duration.millis(warmup), + }); const results: Read[] = []; const warn = spyOn(console, "warn").mockImplementation(() => {}); const reading = (async () => { @@ -66,8 +68,9 @@ function group(track: Track.Producer, sequence: bigint, timestamps: number[]): G return group; } +// The ordered read decides the start: a group its feed owes below the live edge is waited for. test("the start waits for an earlier group when the live edge arrives first", async () => { - const { track, results, flush } = setup(); + const { track, results, flush } = setup(BUDGET, 0, BUDGET, BUDGET, 4n); group(track, 5n, [5_000_000]); await flush(); expect(frames(results)).toEqual([]); @@ -90,28 +93,17 @@ test("the warmup reaches further back for the start", async () => { [300, "3@4500000"], ] as const) { const { track, results, flush } = setup(200, warmup); - // Newest first: group 4 covers the max delay alone, but not the warmup too. - group(track, 5n, [5_000_000]).finish(); - group(track, 4n, [4_800_000]).finish(); - await flush(); + // Group 4 covers the max delay alone, but not the warmup too. The ordered read hands them out + // in key order, so the start skip picks the start among them. group(track, 3n, [4_500_000]).finish(); + group(track, 4n, [4_800_000]).finish(); + group(track, 5n, [5_000_000]).finish(); await flush(); expect(frames(results)[0]).toEqual(start); results.length = 0; } }); -test("a quiet track starts within one reach", async () => { - const { clock, track, results, flush } = setup(200, 300); - group(track, 5n, [5_000_000]); - await flush(); - await clock.advance(Duration.millis(BUDGET - 50)); - expect(frames(results)).toEqual([]); - - await clock.advance(Duration.millis(50)); - expect(frames(results)).toEqual(["5@5000000"]); -}); - test("the start needs no wait once a group covers the reach", async () => { const { track, results, flush } = setup(); group(track, 5n, [5_000_000, 5_000_000 + BUDGET * 1000]); @@ -121,19 +113,17 @@ test("the start needs no wait once a group covers the reach", async () => { test("the start keeps a group that began before the reach but covers it", async () => { const { track, results, flush } = setup(200, 300); - group(track, 5n, [5_000_000]); - await flush(); // Group 4 begins 600ms before the newest content, past the 500ms reach, yet covers its 4.5s target. group(track, 4n, [4_400_000, 4_600_000, 4_900_000]).finish(); + group(track, 5n, [5_000_000]); await flush(); expect(frames(results)[0]).toEqual("4@4400000"); }); test("the age skip still drops a group its successor covers", async () => { const { track, results, flush } = setup(); - group(track, 5n, [5_000_000]); - await flush(); // Group 6 starts a whole budget behind group 7's content, so group 5 cannot cover the reach. + group(track, 5n, [5_000_000]); group(track, 6n, [5_000_000 + (BUDGET + 100) * 1000]); group(track, 7n, [5_000_000 + (2 * BUDGET + 200) * 1000]); await flush(); @@ -147,19 +137,8 @@ test("a large max delay alone starts at a live edge that arrives first", async ( expect(frames(results)).toEqual(["5@5000000"]); }); -test("a min delay past the max delay reaches only the max delay", async () => { - const { clock, track, results, flush } = setup(200, 0, 2_000); - group(track, 5n, [5_000_000]); - await flush(); - await clock.advance(Duration.millis(150)); - expect(frames(results)).toEqual([]); - - await clock.advance(Duration.millis(50)); - expect(frames(results)).toEqual(["5@5000000"]); -}); - test("an unlimited max delay still reaches the min delay", async () => { - const { track, results, flush } = setup(BUDGET, 0, BUDGET, null); + const { track, results, flush } = setup(BUDGET, 0, BUDGET, null, 4n); group(track, 5n, [5_000_000]); await flush(); expect(frames(results)).toEqual([]); @@ -171,10 +150,9 @@ test("an unlimited max delay still reaches the min delay", async () => { test("the covering group is kept while its warmup arrives after the first read", async () => { const { track, results, flush } = setup(200, 300); - group(track, 5n, [5_000_000]); - await flush(); // Group 4 covers the 4.5s target, and its first frame is read before the rest of it lands. const covering = group(track, 4n, [4_400_000]); + group(track, 5n, [5_000_000]); await flush(); expect(frames(results)).toEqual(["4@4400000"]); @@ -189,10 +167,9 @@ test("the covering group is kept while its warmup arrives after the first read", test("the covering group is kept while it arrives after the first read, without a warmup", async () => { const { track, results, flush } = setup(200); - group(track, 5n, [5_000_000]); - await flush(); // Group 4 covers the 4.8s target, so its media before that point is older than the max delay. const covering = group(track, 4n, [4_400_000]); + group(track, 5n, [5_000_000]); await flush(); expect(frames(results)).toEqual(["4@4400000"]); @@ -204,29 +181,3 @@ test("the covering group is kept while it arrives after the first read, without await flush(); expect(frames(results)).toEqual(["4@4400000", "4@4433000", "4@4700000", "4@4900000", "5@5000000"]); }); - -test("an age skip that ends the start still warms until the start's target", async () => { - const { track, results, flush } = setup(200, 300); - // Group 4 alone does not cover its 4.5s target, so the start waits. - group(track, 4n, [5_000_000]); - await flush(); - // Groups 5 and 6 decode in one pass: group 6's 6.2s edge puts the 500ms reach's target at 5.7s, - // and the age skip drops group 4, whose successor starts past the reach, before the start sees it. - const covering = group(track, 5n, []); - const newest = group(track, 6n, []); - await flush(); - write(covering, 5_600_000); - write(covering, 5_900_000); - write(newest, 6_200_000); - await flush(); - expect(frames(results)).toEqual(["5@5600000", "5@5900000"]); - - // The reader warms until 6.0s (the target plus the warmup), not 5.9s (the first frame plus the - // warmup), so newer content does not yet drop the covering group. - write(newest, 6_210_000); - await flush(); - write(covering, 5_950_000); - covering.finish(); - await flush(); - expect(frames(results)).toEqual(["5@5600000", "5@5900000", "5@5950000", "6@6200000", "6@6210000"]); -}); diff --git a/js/hang/src/container/index.ts b/js/hang/src/container/index.ts index a019d23e24..770f680b90 100644 --- a/js/hang/src/container/index.ts +++ b/js/hang/src/container/index.ts @@ -7,7 +7,7 @@ /** Re-export of {@link https://www.npmjs.com/package/@moq/loc | @moq/loc}, the LOC container. */ export * as Loc from "@moq/loc"; -export { Consumer, type ConsumerProps, DELAY_MAX, type Playhead, type Read } from "./consumer.ts"; +export { Consumer, type ConsumerProps, DELAY_MAX, type Read } from "./consumer.ts"; export type { Format } from "./format.ts"; /** The legacy varint container. */ export * as Legacy from "./legacy.ts"; diff --git a/js/net/src/fill.test.ts b/js/net/src/fill.test.ts index ae181a96bf..5f4bf560bb 100644 --- a/js/net/src/fill.test.ts +++ b/js/net/src/fill.test.ts @@ -417,7 +417,7 @@ test("fill: a gap below a group that fails after a probe is asked", async () => failed.abort(new StreamError(StreamCode.Cancel)); await yieldNow(); // Judged stale against group 4, so not asked. - await reader.start(); + hooks.sourceStart(reader); group.abort(new StreamError(StreamCode.Cancel)); await yieldNow(); const next = ordered.nextGroup(); diff --git a/js/net/src/integration.test.ts b/js/net/src/integration.test.ts index c7f230bdb4..ae7fec03fd 100644 --- a/js/net/src/integration.test.ts +++ b/js/net/src/integration.test.ts @@ -484,12 +484,13 @@ test("integration: lite-07 serves sparse and relative ranges", async () => { }); // A publisher counts a relative start back from its newest group, which may hold no frame yet and -// so counts neither in the largest it reports nor where the copy resolves the start. The copy reports -// the publisher's start, where a reader waiting on each group in sequence starts. Mirrors the Rust -// `a_relative_start_reports_the_publishers_start_over_a_frameless_newest_group`. +// so counts neither in the largest it reports nor where the copy resolves the start. The copy keeps +// the publisher's start, so a reader waiting on each group in sequence starts there rather than +// waiting below it for groups that never arrive. Mirrors the Rust +// `an_ordered_reader_starts_at_the_publishers_start_over_a_frameless_newest_group`. // The newest group lands before the reader asks, as it can under load: a sibling reads it, so the // reader's own window has yet to settle on it when it asks where it starts. -test("integration: lite-07 reports a relative start over a frameless newest group", async () => { +test("integration: lite-07 starts an ordered read at a relative start over a frameless newest group", async () => { const pair = createMockTransportPair(Lite.ALPN_07_WIP); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -508,16 +509,16 @@ test("integration: lite-07 reports a relative start over a frameless newest grou const remote = wireOf(client).consume(Path.from("test")); const subscription = { delay: { max: null }, ranges: [{ start: { relative: 0n } }] }; - const reader = remote.track("video").subscribe(subscription); + const reader = remote.track("video").subscribe(subscription).ordered(); const sibling = remote.track("video").subscribe(subscription); try { // Accepting the track subscribes it upstream, whose answer resolves the start. await reader.info(); expect((await sibling.recvGroup())?.start).toBe(5n); - expect(await reader.start()).toBe(5n); newest.writeFrame({ payload: new Uint8Array([5]), timestamp: Timestamp.unscaled(newest.start) }); newest.finish(); - expect((await reader.recvGroup())?.start).toBe(5n); + const item = await reader.nextItem(); + expect(item?.kind === "group" ? item.group.start : item).toBe(5n); } finally { sibling.close(); reader.close(); diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index eb4718e351..0db80d4e7a 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -550,6 +550,8 @@ export interface ResumeSource { cachedGroup(key: bigint, from?: number): GroupConsumer | undefined; /** As {@link cachedGroup} from its first frame, judged by the budget of the copy it comes from. */ judgedGroup(key: bigint): GroupConsumer | undefined; + /** When the latest group any copy caches in `[start, end)` landed; see `hooks.landed`. */ + landed(start: bigint, end: bigint): Instant | undefined; /** Let go of every copy. */ close(abort?: Error): void; } @@ -888,6 +890,11 @@ export const hooks: { judgedGroup: (subscriber: Subscriber, key: bigint) => GroupConsumer | undefined; /** Whether {@link cachedGroup} would find the group, without making a read handle. */ holdsGroup: (subscriber: Subscriber, key: bigint) => boolean; + /** + * When the latest group the subscriber's track caches in `[start, end)` landed, undefined when + * it caches none; a routed reader asks every copy. Mirrors the Rust `Subscriber::landed`. + */ + landed: (subscriber: Subscriber, start: bigint, end: bigint) => Instant | undefined; /** * The start of the group before `start` on the subscriber's track, as a lite-07 GROUP's * Previous Start names it: just below the run of starts declared skipped directly under @@ -1273,6 +1280,9 @@ export const hooks: { holdsGroup: () => { throw new Error("track.ts not loaded"); }, + landed: () => { + throw new Error("track.ts not loaded"); + }, previousStart: () => { throw new Error("track.ts not loaded"); }, diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index b4012c01a1..642c1dd069 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -2875,7 +2875,7 @@ test("a widened range's SUBSCRIBE_STARTED resolves the feed's start", async () = expect(update.ranges).toHaveLength(1); await respond(request, version, [{ started: new SubscribeStarted(1n, 1n) }]); await drainReactions(); - expect(await track.start()).toBe(1n); + expect(hooks.sourceStart(track)).toBe(1n); } finally { track.close(); broadcast.close(); diff --git a/js/net/src/ordered.test.ts b/js/net/src/ordered.test.ts index b29bdcf5b2..a4b8341ac2 100644 --- a/js/net/src/ordered.test.ts +++ b/js/net/src/ordered.test.ts @@ -1,7 +1,7 @@ // What an ordered read returns from nextItem: groups in key order, and a gap for every group it gives up on. // Ports the Rust `track::Ordered` item tests, so both read the same. import { expect, test } from "bun:test"; -import { Duration, sleep } from "@moq/clock"; +import { Duration, now, sleep } from "@moq/clock"; import { Manual } from "@moq/clock/test"; import { StreamCode, StreamError } from "./error.ts"; import { hooks, staleHole } from "./internal.ts"; @@ -203,6 +203,36 @@ test("a cached group the seek passes as stale is a gap", () => expect(items(ordered)).toEqual(["gap 0..1 (old)", "group 1", "group 2"]); })); +// A gap says when the latest group it gives up landed: one the read passed as stale, and none for +// keys that never came. Mirrors the Rust `a_gap_reports_when_what_it_gives_up_landed`. +test("a gap reports when what it gives up landed", () => + withClock((clock) => { + const landed = (ordered: Ordered): string[] => { + const out: string[] = []; + for (let item = ordered.tryNextItem(); item; item = ordered.tryNextItem()) { + if (item.kind === "group") { + out.push(`group ${item.group.start}`); + item.group.close(); + } else { + out.push(`gap ${item.start}..${item.end} ${item.landed?.since(first).asMillis() ?? "never"}`); + } + } + return out; + }; + const producer = receivedTrack(); + const first = now(); + clock.jump(Duration.millis(5)); + put(producer, 0n, false); + put(producer, 1n, false); + clock.jump(Duration.millis(100)); + put(producer, 2n, false); + const ordered = producer.subscribe({ delay: { max: Duration.millis(50) } }).ordered(); + drop(producer, 3n, 4n); + put(producer, 4n, false); + // Group 0 landed 5 ms in, before it went stale, and group 3 never did. + expect(landed(ordered)).toEqual(["gap 0..1 5", "group 1", "group 2", "gap 3..4 never", "group 4"]); + })); + // Rust reports it Old; JS keeps no record below the read, so its memory stays bounded. test("a local track's late arrival is no gap", () => { const producer = new TrackProducer("test"); diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index 67bf441b5b..6204efedbd 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -815,6 +815,16 @@ class Reader implements ResumeSource { return undefined; } + // A group a replaced copy delivered landed all the same, so every copy counts. + landed(start: bigint, end: bigint): Instant | undefined { + let landed: Instant | undefined; + for (const copy of this.#copies) { + const arrived = hooks.landed(copy.sub, start, end); + if (arrived && (!landed || arrived.gt(landed))) landed = arrived; + } + return landed; + } + judgedGroup(key: bigint): GroupConsumer | undefined { for (let index = this.#copies.length - 1; index >= 0; index--) { const copy = this.#copies[index]; diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 8181bad256..227830ea8d 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -92,8 +92,11 @@ export type Item = /** * Groups `[start, end)` this reader gave up on, which it will not return, with what a read of * them would have ended with. `group` is absent, so `item?.group` reads a group or nothing. + * `landed` is when the latest of those groups landed here, undefined when none did or the cache + * has since evicted it: a reader timing arrivals learns when what it never read came in, such + * as a join's backlog skipped as stale. */ - | { kind: "gap"; start: bigint; end: bigint; error: Error; group?: never }; + | { kind: "gap"; start: bigint; end: bigint; error: Error; landed?: Instant; group?: never }; // What an ordered read found: an item, or the same idle, end, and error as a group read. type ItemRecv = Item | { kind: "idle" } | { kind: "done" } | { kind: "error"; error: Error }; @@ -1017,6 +1020,9 @@ let receivedCopy: (producer: Producer) => boolean; // The keys in `[start, end)` the producer caches, unsorted, costing the smaller of the span // and the cache; assigned in its static block. let cachedIn: (producer: Producer, start: bigint, end: bigint) => bigint[]; +// When the latest group the producer caches in `[start, end)` landed, undefined when it caches +// none; assigned in its static block. +let cachedLanded: (producer: Producer, start: bigint, end: bigint) => Instant | undefined; // Mirror the cached groups in `[start, end)` that `offer` accepts into a sink again, cut ones // included as a fresh sink gets them, for a reader whose floor moved below them; assigned in // the producer's static block. @@ -1142,6 +1148,7 @@ let ordered_: { readFrame(subscriber: Subscriber): Promise<({ group: bigint; frame: number } & Frame) | undefined>; readString(subscriber: Subscriber): Promise; readJson(subscriber: Subscriber): Promise; + isSkipped(subscriber: Subscriber, start: bigint, end: bigint): boolean; }; // A pending request for a group past every key written. Mirrors the Rust `demand::Pending`. @@ -1620,6 +1627,14 @@ export class Producer { } return [...producer.#cached.keys()].filter((key) => key >= start && key < end); }; + cachedLanded = (producer, start, end) => { + let landed: Instant | undefined; + for (const key of cachedIn(producer, start, end)) { + const arrived = producer.#cached.get(key)?.arrived; + if (arrived && (!landed || arrived.gt(landed))) landed = arrived; + } + return landed; + }; cachedMirror = (producer, key, from) => { const entry = producer.#cached.get(key); if (!entry || entry.group.closed.peek() instanceof Error) return undefined; @@ -4369,6 +4384,11 @@ export class Subscriber { if (group) hooks.expireGroup(group, hooks.staleAt(subscriber, key)); return group; }; + hooks.landed = (subscriber, start, end) => { + if (subscriber.#resume) return subscriber.#resume.landed(start, end); + const producer = subscriber.#state.producer; + return producer && cachedLanded(producer, start, end); + }; hooks.holdsGroup = (subscriber, key) => { const producer = subscriber.#state.producer; return producer !== undefined && cachedHolds(producer, key); @@ -4499,6 +4519,7 @@ export class Subscriber { readFrame: (subscriber) => subscriber.#readFrame(), readString: (subscriber) => subscriber.#readString(), readJson: (subscriber) => subscriber.#readJson(), + isSkipped: (subscriber, start, end) => subscriber.#isSkipped(start, end), }; } @@ -4572,25 +4593,6 @@ export class Subscriber { return start === undefined ? undefined : max(start, this.#local.start); } - /** - * Where the source's feed starts once its publisher resolved it, raised to this reader's own - * floor: the first group a publisher tells its peer it serves, or lower where a fill this - * reader waits on starts. Undefined when the source declares none and no fill is out. - * - * Where a reader that waits on each group in key starts when {@link floor} names none: - * groups arrive newest first, so the first to arrive is not the oldest served. A copy says - * only once subscribed upstream, so ask once {@link info} resolves or a group arrives. Mirrors - * the Rust `Subscriber::poll_start`. - */ - async start(): Promise { - for (;;) { - const start = hooks.sourceStart(this); - if (start !== "pending") return start; - if (this.#state.closed.peek() !== undefined) return undefined; - await this.#wait(); - } - } - /** * The lowest group at or above `from` this subscriber's ranges and {@link withRange} floor * cover, past the starts the publisher declared skipped (see {@link Producer.appendGroup}) and @@ -4599,7 +4601,7 @@ export class Subscriber { * a skip as such a drop. `from` itself while a relative anchor waits on the first group, which * can't say yet. Mirrors the Rust `Subscriber::next_in_range`. */ - nextInRange(from: bigint): bigint | undefined { + #nextInRange(from: bigint): bigint | undefined { from = max(from, this.#local.start); if (this.#resume) { // Without a serving copy nothing says yet. @@ -4612,18 +4614,8 @@ export class Subscriber { return firstOpen(this.#window.runs(from), (key) => holeEnd(drops, key, true)); } - /** - * Whether the publisher declared every start in `[start, end)` skipped, never to be produced - * (which {@link Producer.appendGroup} does between groups), so no group can arrive there: true - * for an empty range. Unlike {@link nextInRange}, this ignores the subscriber's ranges. A copy - * knows only the skips its wire forwarded. - * - * What tells a reader that nothing lies between two groups, or before a track's first. It - * knows only the skips recorded so far, so `false` means "not known skipped": a skip still in - * flight reads `false`, as does one a local producer's cache expired below its edge. Mirrors - * the Rust `Subscriber::is_skipped`. - */ - isSkipped(start: bigint, end: bigint): boolean { + // Whether the publisher declared every start in `[start, end)` skipped; see Ordered.isSkipped. + #isSkipped(start: bigint, end: bigint): boolean { if (end <= start) return true; const run = [{ start, end }]; if (this.#resume) { @@ -5240,7 +5232,7 @@ export class Subscriber { // What the read still owes lies above the gap, so the gap goes first. const lowest = this.#lowestGap(items); if (!lowest) return { kind: "idle" }; - const owed = this.nextInRange(items.at); + const owed = this.#nextInRange(items.at); const ready = lowest.span.start <= items.at || owed === undefined || owed >= lowest.span.start; return ready ? this.#takeGap(items, lowest.index, lowest.span, lowest.span.end) : { kind: "idle" }; } @@ -5327,7 +5319,8 @@ export class Subscriber { if (forgot > floor) this.#stoppedReading(floor, forgot); if (!this.#resume) this.#nextKey = max(this.#nextKey, given.end); this.#fold(forgot); - return { kind: "gap", start: given.start, end: given.end, error }; + const landed = hooks.landed(this, given.start, given.end); + return { kind: "gap", start: given.start, end: given.end, error, landed }; } // Remove `[from, resume)` from the gap at `index`, returning its error. @@ -5741,10 +5734,19 @@ export class Ordered { return this.#subscriber.finished(); } - /** Whether the publisher declared every start in `[start, end)` skipped; see {@link Subscriber.isSkipped}. */ - // fallow-ignore-next-line unused-class-member + /** + * Whether the publisher declared every start in `[start, end)` skipped, never to be produced + * (which {@link Producer.appendGroup} does between groups), so no group can arrive there: true + * for an empty range. Unlike the gaps, this ignores the ranges. A copy knows only the skips its + * wire forwarded. + * + * What tells a reader that nothing lies between two groups, or before a track's first. It + * knows only the skips recorded so far, so `false` means "not known skipped": a skip still in + * flight reads `false`, as does one a local producer's cache expired below its edge. Mirrors + * the Rust `Ordered::is_skipped`. + */ isSkipped(start: bigint, end: bigint): boolean { - return this.#subscriber.isSkipped(start, end); + return ordered_.isSkipped(this.#subscriber, start, end); } /** diff --git a/js/watch/src/sync.ts b/js/watch/src/sync.ts index 0fc3d55450..1854c89543 100644 --- a/js/watch/src/sync.ts +++ b/js/watch/src/sync.ts @@ -471,8 +471,6 @@ export class Sync { readonly #clock: Clock; /** Whether buffered startup has released the media consumers. @internal */ readonly playing: Getter; - /** Changes whenever {@link Sync.now} jumps rather than advancing with the wall clock: a re-anchor, a new delay, or a pause. */ - readonly anchor: Getter; #signals = new Effect(); @@ -481,10 +479,6 @@ export class Sync { this.in = this.#clock.in; this.out = readonlys(this.#clock.out); this.playing = this.#clock.playing; - this.anchor = new Derived( - [this.#clock.out.reference, this.#clock.out.delay, this.#clock.playing], - (reference, delay, playing) => ({ reference, delay, playing }), - ); this.#signals.run(this.#runInstant.bind(this)); this.#signals.run(this.#runDelay.bind(this)); diff --git a/js/watch/src/video/decoder.ts b/js/watch/src/video/decoder.ts index 6c4e2a25e5..df67c98b62 100644 --- a/js/watch/src/video/decoder.ts +++ b/js/watch/src/video/decoder.ts @@ -692,8 +692,6 @@ class Rendition { format, delay: this.#delay, predecessor: this.consumer ?? this.#predecessor, - // The one decoder drives the clock, so it skips a hole once the clock passed it. - playhead: this.sync, }); effect.cleanup(() => consumer.close()); this.consumer = consumer; From c5f9a9fc75ae2dd0ba3ec00b50a82e26a33e144b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:37:29 -0800 Subject: [PATCH 08/23] fix: the audio-quality JS client reads the join without Subscriber.start, and the API reports and formatting follow Co-Authored-By: Claude Opus 5.5 --- js/hang/api/container.api.md | 6 -- js/net/api/index.api.md | 4 +- js/watch/api/index.api.md | 1 - rs/moq-mux/src/container/consumer.rs | 79 +++++++++++-------- rs/moq-net/src/model/track.rs | 3 +- rs/moq-net/tests/integration/range_anchors.rs | 5 +- test/audio-quality/clients/js/src/capture.ts | 5 +- 7 files changed, 55 insertions(+), 48 deletions(-) diff --git a/js/hang/api/container.api.md b/js/hang/api/container.api.md index 88108b5dab..66ccc5996e 100644 --- a/js/hang/api/container.api.md +++ b/js/hang/api/container.api.md @@ -27,7 +27,6 @@ export interface ConsumerProps { delay?: Signals.GetterInit; format: Format; ordered?: Net.Track.Ordered; - playhead?: Playhead; predecessor?: Consumer; warmup?: Net.Clock.Duration; } @@ -80,11 +79,6 @@ export * as Loc from "@moq/loc"; export const MAX_AGE: Net.Clock.Duration; -export interface Playhead { - readonly anchor: Signals.Getter; - now(): Net.Clock.Duration | undefined; -} - export type Read = { frame: ReceivedFrame; group: bigint; diff --git a/js/net/api/index.api.md b/js/net/api/index.api.md index 386c31fc7a..80ba3ec772 100644 --- a/js/net/api/index.api.md +++ b/js/net/api/index.api.md @@ -758,6 +758,7 @@ export namespace Track { error: Error; group?: never; kind: "gap"; + landed?: Clock.Instant; start: bigint; }; export type Order = "ascending" | "descending"; @@ -834,14 +835,11 @@ export namespace Track { finished(): Promise; floor(): bigint | undefined; info(): Promise; - isSkipped(start: bigint, end: bigint): boolean; largest(): Promise; readonly name: string; - nextInRange(from: bigint): bigint | undefined; ordered(): Track.Ordered; recvGroup(): Promise; refresh(options?: Track.RefreshOptions): Promise; - start(): Promise; get subscription(): Signals.Getter; update(options: Track.Subscription): void; withRange(groups: Track.Groups): this; diff --git a/js/watch/api/index.api.md b/js/watch/api/index.api.md index ed170e6cef..88d8226dd7 100644 --- a/js/watch/api/index.api.md +++ b/js/watch/api/index.api.md @@ -177,7 +177,6 @@ export * as Signals from "@moq/signals"; export class Sync { constructor(props?: Signals.Inputs); - readonly anchor: Signals.Getter; close(): void; readonly in: Signals.Readonlys; now(): Clock.Duration | undefined; diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 8b2bacb1db..296c7d8d02 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -1967,7 +1967,10 @@ mod tests { "the budget wakes the reader" ); } else { - let read = kio::clock::timeout(Duration::ZERO, reader.read()).await.unwrap().unwrap(); + let read = kio::clock::timeout(Duration::ZERO, reader.read()) + .await + .unwrap() + .unwrap(); assert!(matches!(read, Some(Read::Gap { .. }))); } assert!( @@ -1980,10 +1983,12 @@ mod tests { #[tokio::test(start_paused = true)] async fn a_filled_sequence_hole_gives_the_next_hole_a_fresh_budget() { let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default() - .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(100))), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default() + .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(100))), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); write_group(&mut copy.track, 0, &[ts(0)]); carried().await; @@ -2210,9 +2215,11 @@ mod tests { async fn holes_behind_a_stalled_source_share_one_budget() { let budget = Duration::from_millis(100); let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); write_group(&mut copy.track, 0, &[ts(0)]); carried().await; @@ -2250,9 +2257,11 @@ mod tests { None, ] { let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default().with_delay(Delay::default().with_min(min).with_max(max)), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default().with_delay(Delay::default().with_min(min).with_max(max)), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); write_group(&mut copy.track, 0, &[ts(0)]); carried().await; @@ -2275,9 +2284,11 @@ mod tests { #[tokio::test(start_paused = true)] async fn a_burst_past_the_min_delay_does_not_age_out_a_hole() { let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); reader.set_delay( Delay::default() @@ -2320,9 +2331,11 @@ mod tests { async fn a_hole_is_late_from_the_earliest_arrival_past_it() { let budget = Duration::from_millis(100); let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default().with_delay(Delay::default().with_min(None).with_max(budget)), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); write_group(&mut copy.track, 0, &[ts(0)]); carried().await; @@ -2356,9 +2369,11 @@ mod tests { #[tokio::test(start_paused = true)] async fn a_hole_and_an_arrived_group_both_wait_up_to_the_max_delay() { let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.audio)).await; - let subscriber = copy.subscribe_with( - moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), - ).await; + let subscriber = copy + .subscribe_with( + moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), + ) + .await; let mut reader = Consumer::new(subscriber, Container::Legacy(crate::container::Kind::Data)); reader.set_delay( Delay::default() @@ -3329,7 +3344,8 @@ mod tests { copy.subscribe_with( moq_net::track::Subscription::default() .with_delay(Delay::default().with_min(None).with_max(Duration::from_secs(30))), - ).await, + ) + .await, Container::Legacy(crate::container::Kind::Audio), ); consumer.max_delay = Duration::from_millis(300); @@ -3924,10 +3940,12 @@ mod tests { async fn gap_keeps_late_arriving_group_within_max_delay() { tokio::time::pause(); let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; - let consumer_track = copy.subscribe_with( - moq_net::track::Subscription::default() - .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(500))), - ).await; + let consumer_track = copy + .subscribe_with( + moq_net::track::Subscription::default() + .with_delay(Delay::default().with_min(None).with_max(Duration::from_millis(500))), + ) + .await; let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); // Group 2's stream beats group 1's, which hasn't arrived at all yet. @@ -4108,13 +4126,8 @@ mod tests { write_group(&mut copy.track, 3, &[ts(4_400_000)]); write_group(&mut copy.track, 4, &[ts(4_700_000)]); write_group(&mut copy.track, 5, &[ts(5_000_000)]); - let mut consumer = reach_reader( - ©, - Duration::from_millis(200), - Duration::from_millis(200), - warmup, - ) - .await; + let mut consumer = + reach_reader(©, Duration::from_millis(200), Duration::from_millis(200), warmup).await; let _track = copy.track.finish().unwrap(); let frames = read_all(&mut consumer).await.unwrap(); @@ -4323,7 +4336,7 @@ mod tests { ) .consumer(consumer_track, Container::Legacy(crate::container::Kind::Data)); assert!( - matches!(consumer.startup, Startup::Earliest { .. }), + matches!(consumer.startup, Startup::Earliest), "a local range names no start, so the start still reaches back" ); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index cb8f6460f1..9a7c4f48b9 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -18148,7 +18148,8 @@ mod test { } // The group given up is reported, then the read ends. if given_up { - let Poll::Ready(Ok(Some(Item::Gap { range, error, .. }))) = ordered.poll_next_group(&kio::Waiter::noop()) + let Poll::Ready(Ok(Some(Item::Gap { range, error, .. }))) = + ordered.poll_next_group(&kio::Waiter::noop()) else { panic!("the given up group is reported") }; diff --git a/rs/moq-net/tests/integration/range_anchors.rs b/rs/moq-net/tests/integration/range_anchors.rs index 9b6cd972a7..92158b9fac 100644 --- a/rs/moq-net/tests/integration/range_anchors.rs +++ b/rs/moq-net/tests/integration/range_anchors.rs @@ -232,7 +232,10 @@ async fn an_ordered_reader_starts_at_the_publishers_start_over_a_frameless_newes .await .expect("group timeout") .unwrap(); - assert_eq!(item.and_then(moq_net::track::Item::group).map(|group| group.start), Some(5)); + assert_eq!( + item.and_then(moq_net::track::Item::group).map(|group| group.start), + Some(5) + ); let _track = track.finish().unwrap(); } diff --git a/test/audio-quality/clients/js/src/capture.ts b/test/audio-quality/clients/js/src/capture.ts index 275533cbe8..1edc6064e9 100644 --- a/test/audio-quality/clients/js/src/capture.ts +++ b/test/audio-quality/clients/js/src/capture.ts @@ -141,12 +141,11 @@ export class Capture { /** * The publisher's largest group as `track`'s subscription began, read the way the container consumer - * reads it: once the track's info or `tuned`, its first group, shows the subscription established and - * its start resolved, so a cached group racing the answer, or live groups landing after it, don't set it. + * reads it: once the track's info or `tuned`, its first group, shows the subscription established, so + * live groups landing after it don't set it. */ export async function joined(track: Track.Subscriber, tuned: Promise): Promise { await Promise.race([track.info().catch(() => undefined), tuned]); - await track.start(); const largest = await track.largest(); return largest === undefined ? undefined : Number(largest.group); } From b4dcb3dbada8e1422518e85000e0ca071d8099c7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:44:23 -0800 Subject: [PATCH 09/23] test(libmoq): the warmup keeps the history a newest-first backlog already delivered The reader no longer waits a reach for history a local track never owes, so the test delivers the backlog before the first read. Co-Authored-By: Claude Opus 5.5 --- rs/libmoq/src/consume.rs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/rs/libmoq/src/consume.rs b/rs/libmoq/src/consume.rs index 138ebba945..4023f5c67f 100644 --- a/rs/libmoq/src/consume.rs +++ b/rs/libmoq/src/consume.rs @@ -1721,7 +1721,9 @@ mod tests { } } - /// Newest-first delivery must leave time for history covering min plus warmup. + /// A newest-first backlog delivered before the first read starts where min plus warmup reach + /// back to: the warmup keeps the history it needs. A local track owes nothing, so the reader + /// waits for no group that has not landed (a received copy's owed groups are moq-mux's tests). #[tokio::test(start_paused = true)] async fn media_warmup_reaches_history() { for warmup in [None, Some(Duration::ZERO), Some(Duration::from_millis(300))] { @@ -1732,12 +1734,10 @@ mod tests { let mut consumer = MediaConsumer::new(warmup, delay).await; consumer.write(5, 5_000_000); consumer.write(4, 4_700_000); - let first = kio::clock::timeout(Duration::from_millis(50), consumer.frame()).await; - let warming = warmup.is_some_and(|warmup| !warmup.is_zero()); - assert_eq!(first.is_err(), warming, "warmup={warmup:?}"); consumer.write(3, 4_400_000); drop(consumer.track.clone().finish().unwrap()); - let mut stamps: Vec<_> = first.ok().flatten().into_iter().collect(); + let warming = warmup.is_some_and(|warmup| !warmup.is_zero()); + let mut stamps = Vec::new(); while let Some(stamp) = consumer.frame().await { stamps.push(stamp); } From e942a91e5c29835515ccd221f417d372e7ea5314 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:51:23 -0800 Subject: [PATCH 10/23] test(moq-cli): a press after a lost group plays one delay out, since a local track owes the lost group nothing Co-Authored-By: Claude Opus 5.5 --- rs/moq-cli/src/play/media.rs | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/rs/moq-cli/src/play/media.rs b/rs/moq-cli/src/play/media.rs index b514d66dcb..cfdd74140f 100644 --- a/rs/moq-cli/src/play/media.rs +++ b/rs/moq-cli/src/play/media.rs @@ -1466,13 +1466,8 @@ mod tests { let played = recorder.played(); let first = played.iter().find(|part| part.sample == 0.5).unwrap(); assert_eq!(first.utterance, 1); - let waited = first.from - pressed; - if lost { - // The ordered read first waits within its age budget for the lost group. - assert!(waited > delay && waited < 2 * delay, "{played:?}"); - } else { - assert_eq!(waited, delay, "{played:?}"); - } + // A local track owes nothing, so the ordered read waits for no lost group. + assert_eq!(first.from - pressed, delay, "{played:?}"); let new: Vec<_> = played.iter().filter(|part| part.sample == 0.5).collect(); assert_eq!( new.iter().map(|part| part.to - part.from).sum::(), From d1e0b484cd44f0dd836ce00d43c19bc562d5efda Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 13:56:40 -0800 Subject: [PATCH 11/23] quest(audio-playout): the browser's estimate observes landings through the decoder's second subscription Co-Authored-By: Claude Opus 5.5 --- quest/m2/audio-playout/estimate-landing.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m2/audio-playout/estimate-landing.md b/quest/m2/audio-playout/estimate-landing.md index fcbe25a925..7b3501130a 100644 --- a/quest/m2/audio-playout/estimate-landing.md +++ b/quest/m2/audio-playout/estimate-landing.md @@ -14,8 +14,9 @@ moved the container reader onto `Ordered`: a missing group is now given up by subscription's `Delay::min`, which `moq-audio` set to its playout estimate. `decode::Consumer::next` observes each packet at `now()`, when it is read, so every packet queued behind a hole the reader waited out reads as late by up to -the max delay. `@moq/hang` doesn't have this: its estimate observes a second -subscription as frames land (`#observe`). +the max delay. The browser doesn't have this: `@moq/watch`'s audio decoder hands +`@moq/hang`'s consumer a second, ordered subscription, so its estimate observes +the track's own frames as they land (`#observe`). Recommended: observe each packet at `container::Consumer::arrival()`, when its wire frame landed in this node's cache, which a hole wait doesn't move. That From 9d0ac975a8c52be5b415d918bb3560f89423bc36 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 14:09:08 -0800 Subject: [PATCH 12/23] fix(moq-audio,hang): the playout estimate times packets by landing, and a decode fault settles a step past a hole moq-audio observes each packet at `container::Consumer::arrival()`, so a hole the container waits out up to the max delay no longer reads as jitter; the corpus replay runs on tokio's paused clock, which moq-net's landing times follow. The estimate-landing follow-up quest is done here. @moq/hang's step onto a group past a hole also settles on the group's decode fault, which gives no timestamp, so next() raises it instead of waiting on the group's other frames. Co-Authored-By: Claude Opus 5.5 --- js/hang/src/container/consumer.test.ts | 32 ++++++ js/hang/src/container/consumer.ts | 9 +- quest/m2/audio-playout/estimate-landing.md | 33 ------ rs/moq-audio/src/decode/consumer.rs | 119 +++++++++++++++++---- 4 files changed, 137 insertions(+), 56 deletions(-) delete mode 100644 quest/m2/audio-playout/estimate-landing.md diff --git a/js/hang/src/container/consumer.test.ts b/js/hang/src/container/consumer.test.ts index f410707cdb..d3eddcfaaf 100644 --- a/js/hang/src/container/consumer.test.ts +++ b/js/hang/src/container/consumer.test.ts @@ -733,6 +733,38 @@ for (const [where, payloads] of [ }); } +// A step onto a group whose first payload failed to decode raises the error rather than waiting for a +// timestamp that frame will never give, while the group's other frames are still unread. +test("Consumer raises a decode error at the head of an open group past a hole", async () => { + const failingFormat: ContainerFormat = { + decode(frame: Uint8Array): Frame[] { + if (frame[0] === 0xff) throw new Error("malformed"); + return [{ payload: frame, timestamp: Duration.micros(frame[0]), keyframe: false }]; + }, + }; + const track = new Track.Producer("test"); + const consumer = new Consumer(replay(track), { format: failingFormat, delay: { max: Duration.millis(500) } }); + + const g0 = track.createGroup(0n); + g0.writeFrame({ payload: new Uint8Array([0x01]), timestamp: Time.Timestamp.fromMicros(1) }); + g0.finish(); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 0n }); + + // A sparse key the live local track never declared the keys below skipped, left open. + const g10 = track.createGroup(10n); + g10.writeFrame({ payload: new Uint8Array([0xff]), timestamp: Time.Timestamp.fromMicros(2) }); + g10.writeFrame({ payload: new Uint8Array([0x03]), timestamp: Time.Timestamp.fromMicros(3) }); + + const timeout = new Promise((resolve) => setTimeout(() => resolve("hung"), 1000)); + const read = consumer.next().then( + () => "read", + (err: Error) => err.message, + ); + expect(await Promise.race([read, timeout])).toBe("malformed"); + consumer.close(); + track.close(); +}); + // Errors come in delivery order, so a decode error waits behind the items ahead of it, a gap included. test("Consumer delivers everything ahead of a decode error", async () => { const track = new Track.Producer("test"); diff --git a/js/hang/src/container/consumer.ts b/js/hang/src/container/consumer.ts index 9b6b992bb6..6edef1061b 100644 --- a/js/hang/src/container/consumer.ts +++ b/js/hang/src/container/consumer.ts @@ -934,8 +934,15 @@ export class Consumer { this.#ordered.isSkipped(this.#active, head.consumer.start) || ptsContiguous(this.#presentedEnd, first); const empty = head.empty && head.consumer.done; + // A payload that failed to decode gives no timestamp, so its fault settles the step, + // for the read below to raise. const known = - contiguous || head.gap || first !== undefined || head.consumer.done || ended !== undefined; + contiguous || + head.gap || + first !== undefined || + head.fault !== undefined || + head.consumer.done || + ended !== undefined; if (known) { if (!contiguous && (head.gap || !empty)) { this.#lost = true; diff --git a/quest/m2/audio-playout/estimate-landing.md b/quest/m2/audio-playout/estimate-landing.md deleted file mode 100644 index 7b3501130a..0000000000 --- a/quest/m2/audio-playout/estimate-landing.md +++ /dev/null @@ -1,33 +0,0 @@ -# [M] moq-audio's playout estimate times when a packet landed, not when it was read - -## Goal - -A missing group the container reader waits out no longer reads as jitter in -`moq-audio`'s playout estimate, so a lost packet doesn't raise `moq play`'s -audio delay toward the max delay. - -## Plan - -Found by [moq-dev/moq#9196](https://github.com/moq-dev/moq/pull/9196), which -moved the container reader onto `Ordered`: a missing group is now given up by -`Delay::max` alone, where the reader used to give it up after the -subscription's `Delay::min`, which `moq-audio` set to its playout estimate. -`decode::Consumer::next` observes each packet at `now()`, when it is read, so -every packet queued behind a hole the reader waited out reads as late by up to -the max delay. The browser doesn't have this: `@moq/watch`'s audio decoder hands -`@moq/hang`'s consumer a second, ordered subscription, so its estimate observes -the track's own frames as they land (`#observe`). - -Recommended: observe each packet at `container::Consumer::arrival()`, when its -wire frame landed in this node's cache, which a hole wait doesn't move. That -needs the jitter corpus test (`the_decode_path_conforms_to_the_corpus`) to -drive moq-net's clock, since its replay sets a thread-local read clock that -moq-net's landing times don't follow: run it on tokio's paused clock and -advance it to each arrival. #9196 tried the switch and that test failed only -for this reason (`buildup: packet 75 delays 120 ms, expected 100`). - -Public API: none. Wire: none. - -## Related - -- [Audio playout](/quest/m2/audio-playout/README.md) - the estimator this feeds diff --git a/rs/moq-audio/src/decode/consumer.rs b/rs/moq-audio/src/decode/consumer.rs index be27340a2e..72ff4b3513 100644 --- a/rs/moq-audio/src/decode/consumer.rs +++ b/rs/moq-audio/src/decode/consumer.rs @@ -351,10 +351,9 @@ impl Consumer { /// The playout delay to hold ahead of the playhead right now: the fixed /// [`Delay::min`], or with [`Options::estimate`] the estimate from every frame read so far. /// - /// The estimate measures when [`read`](Self::read) hands each packet over, so - /// it is only as honest as the caller is prompt: read as soon as the previous - /// frame is handled and let the playback buffer absorb the jitter, rather than - /// pacing reads to the speaker. + /// The estimate measures when each packet landed in this node's cache + /// ([`moq_mux::container::Consumer::arrival`]), so neither a hole the container waits out nor + /// a late read holds the packets behind it as jitter. pub fn delay(&self) -> Duration { from_millis(self.target.millis()) } @@ -709,7 +708,10 @@ impl Consumer { match &read { Encoded::Frame { frame, .. } => { if let Some(scale) = frame.timestamp.scale() { - let arrival = jitter::millis(now().duration_since(self.origin)); + // When the packet landed, not when it was read: a hole the container + // waits out holds the packets behind it, which arrived on time. + let arrived = self.track.arrival().unwrap_or_else(now); + let arrival = jitter::millis(arrived.duration_since(self.origin)); let media = frame.timestamp.value() as f64 * 1000.0 / scale.as_u64() as f64; self.target.observe(arrival, media); } @@ -913,13 +915,8 @@ impl Consumer { } } -/// The receiver's monotonic clock, which a test can pin to replay a recorded -/// arrival trace without waiting it out. +/// The receiver's monotonic clock. fn now() -> Instant { - #[cfg(test)] - if let Some(now) = tests::CLOCK.get() { - return now; - } Instant::now() } @@ -2781,12 +2778,6 @@ mod tests { after } - thread_local! { - /// A pinned receiver clock for [`now`], so a test replays an arrival trace - /// instantly. Thread-local because a `tokio::test` runs on one thread. - pub(super) static CLOCK: std::cell::Cell> = const { std::cell::Cell::new(None) }; - } - /// Replay a corpus trace through the whole decode path, one PCM packet per group /// written at its recorded arrival and read straight back, and return the /// consumer's delay after each packet in milliseconds. @@ -2817,7 +2808,6 @@ mod tests { ); let origin = Instant::now(); - CLOCK.set(Some(origin)); let mut consumer = Consumer::new( &subscriber, &catalog, @@ -2853,7 +2843,8 @@ mod tests { producer.seek(micros).unwrap(); } } - CLOCK.set(Some(origin.saturating_add(duration(arrival)))); + // On tokio's paused clock, which moq-net's landing times follow. + tokio::time::advance(duration(arrival).saturating_sub(origin.elapsed())).await; producer .write(moq_mux::container::Frame { timestamp: Timestamp::from_micros(micros).unwrap(), @@ -2870,7 +2861,6 @@ mod tests { .expect("one decoded frame per packet"); series.push(crate::jitter::millis(consumer.delay())); } - CLOCK.set(None); series } @@ -2880,7 +2870,7 @@ mod tests { /// estimator's own test, since the container refuses a timestamp that runs /// backwards. `silence` publishes a discontinuity marker ahead of each frame its /// `ends` lists, so the consumer has to find the declared end on its own. - #[tokio::test] + #[tokio::test(start_paused = true)] async fn the_decode_path_conforms_to_the_corpus() { for name in [ "advertised", @@ -3043,9 +3033,94 @@ mod tests { assert_eq!(dropped, lost); } + /// A hole the container waits out up to the max delay holds the packets behind it, which + /// landed on time: the estimate observes when they landed, not when they were read, so the + /// wait reads as no jitter. + #[tokio::test(start_paused = true)] + async fn a_hole_waited_out_to_the_max_delay_leaves_the_estimate() { + use moq_mux::container::Container as _; + + let publisher = origin(1); + let route = publisher + .route("bcast", moq_net::origin::RouteOptions::hidden()) + .unwrap(); + let mut track = route + .broadcast() + .create_track("audio", hang::container::track_info(hang::catalog::PRIORITY.audio)) + .unwrap(); + route.announce().unwrap(); + let container = moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Audio); + let payload: Bytes = vec![0u8; 160 * size_of::()].into(); + let mut write = |sequence: u64| { + let mut group = track.create_group(moq_net::group::Info { start: sequence }).unwrap(); + let frame = moq_mux::container::Frame { + timestamp: Timestamp::from_micros(sequence * 20_000).unwrap(), + duration: None, + payload: payload.clone(), + keyframe: true, + }; + container.write(&mut group, &[frame]).unwrap(); + group.finish().unwrap(); + }; + + // A received copy: its feed owes the missing group, so the reader waits for it. + let relay = origin(2); + let _link = link(&publisher, &relay).await; + let remote = relay + .consume() + .follow("bcast", None) + .unwrap() + .available() + .await + .unwrap() + .unwrap(); + let catalog = hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Pcm, 8_000, 1); + let max = Duration::from_secs(2); + let options = Options { + delay: Delay { + max: Some(max), + ..Delay::default() + }, + estimate: true, + ..Options::new() + }; + let mut consumer = Consumer::new(&remote, &catalog, "audio", options).await.unwrap(); + + write(0); + next_frame(&mut consumer).await.unwrap().expect("the first packet"); + let estimate = consumer.delay(); + + // Packet 1 never arrives; the ones after it land on time, 20 ms apart. + for sequence in 2..6 { + tokio::time::advance(Duration::from_millis(if sequence == 2 { 40 } else { 20 })).await; + write(sequence); + } + let since = kio::clock::now(); + assert!(matches!(consumer.read().await.unwrap(), Some(Read::Gap { .. }))); + assert!(since.elapsed() >= max / 2, "the hole was not waited: {:?}", since.elapsed()); + for _ in 2..6 { + next_frame(&mut consumer) + .await + .unwrap() + .expect("a packet past the hole"); + } + // Steady arrivals after it, read as they land, close the interval that measured the wait. + let resumed = 6 + (since.elapsed().as_millis() / 20) as u64; + for sequence in resumed..resumed + 100 { + write(sequence); + next_frame(&mut consumer).await.unwrap().expect("a steady packet"); + tokio::time::advance(Duration::from_millis(20)).await; + } + assert!( + consumer.delay() <= estimate, + "the wait read as jitter: {:?} after, {estimate:?} before", + consumer.delay() + ); + } + /// A group lost between a declared end and what resumes leaves the end declared, as the /// browser's consumer does: the estimator measures the same as when nothing was lost. - #[tokio::test] + #[tokio::test(start_paused = true)] async fn a_declared_end_survives_a_lost_group() { let case = crate::jitter::tests::case("silence"); assert!(!case.ends.is_empty()); From 9dc6c97f7177858e123d1ed0e6ea79e0d77e1c27 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 14:15:46 -0800 Subject: [PATCH 13/23] style: format Co-Authored-By: Claude Opus 5.5 --- rs/moq-audio/src/decode/consumer.rs | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/rs/moq-audio/src/decode/consumer.rs b/rs/moq-audio/src/decode/consumer.rs index 72ff4b3513..2e9618c7b7 100644 --- a/rs/moq-audio/src/decode/consumer.rs +++ b/rs/moq-audio/src/decode/consumer.rs @@ -3044,14 +3044,14 @@ mod tests { let route = publisher .route("bcast", moq_net::origin::RouteOptions::hidden()) .unwrap(); - let mut track = route + let track = route .broadcast() .create_track("audio", hang::container::track_info(hang::catalog::PRIORITY.audio)) .unwrap(); route.announce().unwrap(); let container = moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Audio); let payload: Bytes = vec![0u8; 160 * size_of::()].into(); - let mut write = |sequence: u64| { + let write = |sequence: u64| { let mut group = track.create_group(moq_net::group::Info { start: sequence }).unwrap(); let frame = moq_mux::container::Frame { timestamp: Timestamp::from_micros(sequence * 20_000).unwrap(), @@ -3097,7 +3097,11 @@ mod tests { } let since = kio::clock::now(); assert!(matches!(consumer.read().await.unwrap(), Some(Read::Gap { .. }))); - assert!(since.elapsed() >= max / 2, "the hole was not waited: {:?}", since.elapsed()); + assert!( + since.elapsed() >= max / 2, + "the hole was not waited: {:?}", + since.elapsed() + ); for _ in 2..6 { next_frame(&mut consumer) .await From 911f8a044e97a799037e9a877d82c30864724b0c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 14:21:53 -0800 Subject: [PATCH 14/23] test(hang): the open-group decode fault test runs on the Manual clock Co-Authored-By: Claude Opus 5.5 --- js/hang/src/container/consumer.test.ts | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/js/hang/src/container/consumer.test.ts b/js/hang/src/container/consumer.test.ts index d3eddcfaaf..d62b3354f6 100644 --- a/js/hang/src/container/consumer.test.ts +++ b/js/hang/src/container/consumer.test.ts @@ -755,12 +755,17 @@ test("Consumer raises a decode error at the head of an open group past a hole", g10.writeFrame({ payload: new Uint8Array([0xff]), timestamp: Time.Timestamp.fromMicros(2) }); g10.writeFrame({ payload: new Uint8Array([0x03]), timestamp: Time.Timestamp.fromMicros(3) }); - const timeout = new Promise((resolve) => setTimeout(() => resolve("hung"), 1000)); - const read = consumer.next().then( - () => "read", - (err: Error) => err.message, + let settled = "hung"; + consumer.next().then( + () => { + settled = "read"; + }, + (err: Error) => { + settled = err.message; + }, ); - expect(await Promise.race([read, timeout])).toBe("malformed"); + await clock.advance(Duration.millis(100)); + expect(settled).toBe("malformed"); consumer.close(); track.close(); }); From 134c68d7e8778a9d7af59bc0ea415276b2435575 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 17:08:57 -0800 Subject: [PATCH 15/23] fix(hang,mux): a gap reaching past the join folds no backlog; the buffered bench reads through Ordered A gap's landing is its latest group's, so one starting at or below the join but reaching past it could report a group produced after the subscription began as the join's backlog. Only a gap wholly at or below the join folds now, in moq-mux and @moq/hang alike. The buffered bench's sparse case wrote shuffled groups to a local track, whose ordered read drops every group landing below one it already handed out, so it waited forever. It now writes its sparse groups in key order, and its received hook wraps the ordered read the consumer uses. Co-Authored-By: Claude Opus 5.5 --- js/hang/bench/buffered.ts | 35 ++++++-------- .../src/container/consumer.backlog.test.ts | 31 +++++++++++++ js/hang/src/container/consumer.ts | 24 +++++----- rs/moq-mux/src/container/consumer.rs | 46 +++++++++++++++++-- 4 files changed, 101 insertions(+), 35 deletions(-) diff --git a/js/hang/bench/buffered.ts b/js/hang/bench/buffered.ts index 63bd1a9d1e..fb01cff322 100644 --- a/js/hang/bench/buffered.ts +++ b/js/hang/bench/buffered.ts @@ -42,27 +42,21 @@ function wire(group: number, frame: number): Group.Frame { function open(track: Track.Producer, delay: Signal): Player { const subscriber = track.subscribe({ delay: { max: HOLD } }); const received: Group.Consumer[] = []; - const recv = subscriber.recvGroup.bind(subscriber); - subscriber.recvGroup = async () => { - const group = await recv(); - if (group) received.push(group); - return group; + // The consumer reads through the subscriber's ordered read, so that is where groups arrive. + const ordered = subscriber.ordered.bind(subscriber); + subscriber.ordered = () => { + const reader = ordered(); + const next = reader.nextItem.bind(reader); + reader.nextItem = async () => { + const item = await next(); + if (item?.group) received.push(item.group); + return item; + }; + return reader; }; return { track, consumer: new Consumer(subscriber, { format: new Format("video"), delay }), received }; } -// The deterministic arrival order of the sparse case: every group exactly once, shuffled. -function shuffled(count: number): number[] { - const order = Array.from({ length: count }, (_, i) => i); - let seed = 42; - for (let i = count - 1; i > 0; i--) { - seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0; - const j = seed % (i + 1); - [order[i], order[j]] = [order[j], order[i]]; - } - return order; -} - // Write one frame to every publisher, then wait until every player decoded it, as each arrival would // be decoded before the next lands. A decoded frame always changes some buffered range. async function arrive(groups: Group.Producer[], players: Player[], index: number, frame: number): Promise { @@ -120,8 +114,10 @@ async function measure( warnings++; }; try { + // Sparse groups skip every other key. They land in key order: the ordered read hands groups out + // in key order whatever order they land in, so a reordered arrival never reaches the consumer. const sequence = (i: number) => (scenario === "sparse" ? i * 2 : i); - const order = scenario === "sparse" ? shuffled(groups) : Array.from({ length: groups }, (_, i) => i); + const order = Array.from({ length: groups }, (_, i) => i); const expected = Array.from({ length: groups * frames }, (_, i) => timestamp(sequence(Math.floor(i / frames)), i % frames), ); @@ -149,8 +145,6 @@ async function measure( for (let frame = 0; frame < frames; frame++) await arrive(sources, players, sequence(i), frame); for (const source of sources) source.finish(); } - // A finished track proves no missing group can still arrive. - if (scenario === "sparse") for (const track of tracks) track.finish(); await Promise.resolve(); for (const player of players) { @@ -176,6 +170,7 @@ async function measure( for (const player of players) { if (ranges(player) !== JSON.stringify([{ start: newest, end: newest }])) throw new Error(`skip left ${ranges(player)}`); + if (player.received.length < groups) throw new Error("groups not received"); if (player.received.slice(0, groups).some((group) => group.closed.peek() === undefined)) throw new Error("skipped group left open"); const read = await player.consumer.next(); diff --git a/js/hang/src/container/consumer.backlog.test.ts b/js/hang/src/container/consumer.backlog.test.ts index 9adf8829b5..1a84674d64 100644 --- a/js/hang/src/container/consumer.backlog.test.ts +++ b/js/hang/src/container/consumer.backlog.test.ts @@ -202,6 +202,37 @@ test("a gap given up at or below the join tells the estimate when it landed", as } }); +// The same gap when the publisher stood at group 151: its landing is 152's, produced since the +// subscription began, so a gap reaching past the join is no backlog. +test("a gap reaching past the join folds nothing", async () => { + const { track, subscriber, answer, write } = joining(150n, () => 151n); + const reported: number[] = []; + const backlog = spyOn(Jitter.prototype, "backlog").mockImplementation((arrival: number) => { + reported.push(arrival); + }); + const consumer = new Consumer(subscriber, { + format: new Format("audio"), + delay: { min: Duration.millis(100), max: Duration.millis(2000) }, + }); + try { + answer(); + write(150, 0); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 150n }); + await clock.advance(Duration.millis(10)); + write(152, 40); + write(153, 60); + await clock.advance(Duration.millis(2001)); + expect(await consumer.next()).toMatchObject({ kind: "gap" }); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 153n }); + expect(reported).toEqual([]); + } finally { + backlog.mockRestore(); + consumer.close(); + track.finish(); + track.close(); + } +}); + // The publisher answers that it stood at group 200, then its largest moves to 210 before the first // group reaches this consumer. Group 206, given up with hole 205, was produced after the subscription // began, so it is no backlog. diff --git a/js/hang/src/container/consumer.ts b/js/hang/src/container/consumer.ts index cf80e95078..557288f499 100644 --- a/js/hang/src/container/consumer.ts +++ b/js/hang/src/container/consumer.ts @@ -185,9 +185,9 @@ export class Consumer { #observing: boolean; // The ordered read gave up on groups since the last one it returned. #gap = false; - // When each gap given up before the join's boundary was known landed, by its first key: told to + // When each gap given up before the join's boundary was known landed, by the key it ends at: told to // the estimate once the boundary is known, or dropped once the first group is handed out. - #unjoined?: { start: bigint; landed: Instant }[] = []; + #unjoined?: { end: bigint; landed: Instant }[] = []; #format: Format; #delay: Getter; #maxBuffer: number; @@ -326,7 +326,7 @@ export class Consumer { if (this.#tuned.peek() === undefined) this.#tuned.set(true); if (item.kind === "gap") { this.#gap = true; - if (item.landed) this.#backlog(item.start, item.landed); + if (item.landed) this.#backlog(item.end, item.landed); continue; } // Delivery starts at the first group: the reader already skipped what is too stale to @@ -349,25 +349,25 @@ export class Consumer { } } - // Tell the estimate when groups the ordered read gave up from `start` on landed, if the publisher's - // largest as the subscription began reached them: they were produced before it, a join's backlog, - // so the estimate learns when the backlog finished arriving though nothing of it was read. One - // produced since, such as an outage's, is no backlog. The estimate observes every arrival itself - // when `#track` feeds it. - #backlog(start: bigint, landed: Instant): void { + // Tell the estimate when the groups the ordered read gave up below `end` landed, if the publisher's + // largest as the subscription began reached them all: they were produced before it, a join's + // backlog, so the estimate learns when the backlog finished arriving though nothing of it was read. + // One produced since, such as an outage's, is no backlog, and a gap reaching past the join may + // land as one, so it folds nothing. The estimate observes every arrival itself when `#track` feeds it. + #backlog(end: bigint, landed: Instant): void { if (this.#observing) return; if (this.#joined === undefined) { - this.#unjoined?.push({ start, landed }); + this.#unjoined?.push({ end, landed }); return; } - if (start <= this.#joined) this.#jitter.backlog(landed.since(Instant.ZERO).asMillis()); + if (end - 1n <= this.#joined) this.#jitter.backlog(landed.since(Instant.ZERO).asMillis()); } // The join's boundary is known: judge the gaps given up before it was. #joinBacklog(): void { const unjoined = this.#unjoined ?? []; this.#unjoined = undefined; - for (const { start, landed } of unjoined) this.#backlog(start, landed); + for (const { end, landed } of unjoined) this.#backlog(end, landed); } // Buffer a group in sequence order and watch it for frames. diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 296c7d8d02..abe2f0d194 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -457,7 +457,8 @@ impl Consumer { /// was produced before. Its frames were never read, so a caller timing /// [`arrival`](Self::arrival)s learns when a join's backlog finished arriving even when the /// reader skipped it: one the ordered reader gave up as stale or late (see - /// [`Item::Gap`](moq_net::track::Item::Gap)'s `landed`), or one a skip here passed over. + /// [`Item::Gap`](moq_net::track::Item::Gap)'s `landed`), or one a skip here passed over. A gap + /// reaching past the join counts for none of it, since its landing may be a later group's. pub fn backlog(&self) -> Option { self.backlog } @@ -1033,9 +1034,10 @@ impl Consumer { let group = match next { Some(moq_net::track::Item::Group(group)) => group, // What the ordered reader gave up was never read, so it is backlog when produced before - // the subscription. + // the subscription. Its landing is the latest group's, so a gap reaching past the join + // may report one produced since, and folds nothing. Some(moq_net::track::Item::Gap { range, landed, .. }) => { - if self.joined.is_some_and(|joined| range.start <= joined) { + if self.joined.is_some_and(|joined| range.end.saturating_sub(1) <= joined) { self.backlog = self.backlog.max(landed); } continue; @@ -2168,6 +2170,44 @@ mod tests { assert_eq!(reader.backlog(), Some(landed), "group 0 went stale unread"); } + /// A gap's landing is when the latest group it gives up landed, so only a gap wholly at or below + /// the join is backlog: one reaching past it may report a group produced after the subscription. + #[tokio::test(start_paused = true)] + async fn a_gap_reaching_past_the_join_folds_no_backlog() { + let budget = Duration::from_secs(2); + for (joined, backlog) in [(151, false), (152, true)] { + let mut copy = Received::new().await; + let mut reader = copy + .reader(Delay::default().with_min(Duration::from_millis(100)).with_max(budget)) + .await; + write_group(&mut copy.track, 150, &[ts(0)]); + carried().await; + assert!(matches!( + reader.read().await.unwrap(), + Some(Read::Frame { group: 150, .. }) + )); + // Where the publisher stood as the subscription began. + reader.joined = Some(joined); + write_group(&mut copy.track, 151, &[ts(20_000)]); + tokio::time::advance(Duration::from_millis(10)).await; + let landed = kio::clock::now(); + write_group(&mut copy.track, 152, &[ts(40_000)]); + write_group(&mut copy.track, 153, &[ts(60_000)]); + carried().await; + // Groups 151 and 152 wait unread until their successors landed a budget ago, so both are + // given up as one stale gap, landing when 152 did. + tokio::time::advance(budget + Duration::from_millis(1)).await; + write_group(&mut copy.track, 154, &[ts(80_000)]); + carried().await; + assert!(matches!(reader.read().await.unwrap(), Some(Read::Gap { .. }))); + assert!(matches!( + reader.read().await.unwrap(), + Some(Read::Frame { group: 153, .. }) + )); + assert_eq!(reader.backlog(), backlog.then_some(landed), "joined at {joined}"); + } + } + /// The join's boundary is where the publisher stood as it answered, not the newer largest by the /// time a live start found a frame to start on. #[tokio::test(start_paused = true)] From 491bc8f31a9e4f42cfe789455296b75aab4a5481 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 17:10:20 -0800 Subject: [PATCH 16/23] docs(bindings): max is how far newer media runs past a missing group, and a min past max starts at the group covering max Co-Authored-By: Claude Opus 5.5 --- dart/moq/lib/src/aliases.dart | 5 +++-- go/wrapper/records.go | 4 ++-- kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt | 3 ++- py/moq-rs/moq/_records.py | 7 ++++--- rs/libmoq/src/api.rs | 6 +++--- rs/moq-ffi/src/consumer.rs | 8 +++++--- swift/Sources/Moq/Aliases.swift | 3 ++- 7 files changed, 21 insertions(+), 15 deletions(-) diff --git a/dart/moq/lib/src/aliases.dart b/dart/moq/lib/src/aliases.dart index a986895204..e72a0235d2 100644 --- a/dart/moq/lib/src/aliases.dart +++ b/dart/moq/lib/src/aliases.dart @@ -159,11 +159,12 @@ extension type Timestamp._(MoqTimestamp _timestamp) implements MoqTimestamp { /// How far behind the live edge a subscriber or decoder plays: where it starts, and how late it may fall. extension type Delay._(MoqDelay _delay) implements MoqDelay { /// `min` is the target latency, where a cold join starts, and `max` is how far a non-latest - /// group may fall behind before it is skipped, and how long a decoder waits for a missing group. + /// group may fall behind before it is skipped, and how far newer media may run past a missing + /// group before a decoder gives it up. /// Both default to zero, moq-net's default. An explicit null `min` leaves it unstated, which a /// decoder starts as zero (a catalog warmup or an audio decoder's playout delay still states a /// `min` to the publisher), and an explicit null `max` sets no subscriber limit. A `min` past - /// `max` starts at `max`. + /// `max` starts at the group covering the point `max` behind the live edge. Delay({Duration? min = Duration.zero, Duration? max = Duration.zero}) : this._(MoqDelay(min: min, max: max)); } diff --git a/go/wrapper/records.go b/go/wrapper/records.go index ef1416daaf..15fb0b852e 100644 --- a/go/wrapper/records.go +++ b/go/wrapper/records.go @@ -76,8 +76,8 @@ type DatagramInfo = ffi.MoqDatagramInfo // starts at the newest group and skips a superseded group at once. type Delay struct { // Min is the target latency: the playout or jitter-buffer delay held behind the live edge, - // where a cold join starts. A missing group is waited for Max alone. A Min past Max starts at - // Max, since the publisher serves nothing older. moq-lite-07 sends it upstream. + // where a cold join starts. A missing group is given up by Max alone. A Min past Max starts at + // the group covering the point Max behind the live edge, since the publisher serves no older group. moq-lite-07 sends it upstream. Min time.Duration // MinUnstated leaves Min unstated, ignoring it. A decoder starts it as zero, while the // subscription keeps it unstated, so it never pulls the aggregate across subscribers to diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 02a1836081..e857201054 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -207,7 +207,8 @@ typealias Subscription = uniffi.moq.MoqSubscription * cold join starts, null to leave it unstated (a decoder starts it as zero; a catalog warmup or an * audio decoder's playout delay still states a `min` to the publisher), and `max`, null for no * subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how - * long a decoder waits for a missing group. A `min` past `max` starts at `max`. + * far newer media may run past a missing group before a decoder gives it up. A `min` past `max` + * starts at the group covering the point `max` behind the live edge. */ typealias Delay = uniffi.moq.MoqDelay diff --git a/py/moq-rs/moq/_records.py b/py/moq-rs/moq/_records.py index 66a225959f..2e12ebbd74 100644 --- a/py/moq-rs/moq/_records.py +++ b/py/moq-rs/moq/_records.py @@ -220,9 +220,10 @@ class Delay: live edge, where a cold join starts. ``None`` leaves it unstated, which a decoder starts as zero, while the subscription keeps it unstated, so it never pulls the aggregate across subscribers to zero, unless a catalog warmup or an audio decoder's playout delay states - one. A ``min`` past ``max`` starts at ``max``. - ``max`` is how far a non-latest group may fall behind before it is skipped, and how long a - decoder waits for a missing group; zero + one. A ``min`` past ``max`` starts at the group covering the point + ``max`` behind the live edge. + ``max`` is how far a non-latest group may fall behind before it is skipped, and how far newer + media may run past a missing group before a decoder gives it up; zero skips at once, ``None`` sets no subscriber limit and leaves the publisher's max age alone to bound it, and it caps every range. diff --git a/rs/libmoq/src/api.rs b/rs/libmoq/src/api.rs index cbc4da16d0..b0febcdc36 100644 --- a/rs/libmoq/src/api.rs +++ b/rs/libmoq/src/api.rs @@ -826,9 +826,9 @@ pub enum moq_refresh { #[derive(Clone, Copy, Default)] pub struct moq_delay { /// The target latency in microseconds: the playout or jitter-buffer delay held behind - /// the live edge, where a cold join starts. A missing group is waited for the max alone. - /// moq-lite-07 sends it upstream. A `min_us` past the max starts at the max, since the - /// publisher serves nothing older. Ignored when `min_unstated` is set. + /// the live edge, where a cold join starts. A missing group is given up by the max alone. + /// moq-lite-07 sends it upstream. A `min_us` past the max starts at the group covering the + /// point the max behind the live edge, since the publisher serves no older group. Ignored when `min_unstated` is set. pub min_us: u64, /// How far a non-latest group may fall behind the live edge before it is skipped, in /// microseconds. Zero skips immediately. Enforced by the publisher's cache and by any diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 8c41aebeaf..c8094fe37c 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -25,14 +25,16 @@ fn raw_frame(frame: moq_net::frame::Frame) -> MoqFrame { #[derive(Clone, Copy, Debug, uniffi::Record)] pub struct MoqDelay { /// The target latency: the playout or jitter-buffer delay held behind the live edge, where - /// a cold join starts, or null to leave it unstated. A missing group is waited for `max` + /// a cold join starts, or null to leave it unstated. A missing group is given up by `max` /// alone. A media consumer starts an unstated minimum as zero, while the subscription keeps /// it unstated, so it never pulls the aggregate across subscribers to zero, unless a catalog /// warmup or an audio decoder's playout delay states one. moq-lite-07 sends it upstream. A - /// `min` past `max` starts at `max`, since the publisher serves nothing older. + /// `min` past `max` starts at the group covering the point `max` behind the live edge, since + /// the publisher serves no older group. An audio decoder refuses a `min` beyond the track's + /// max age instead. pub min: Option, /// How far a non-latest group may fall behind the live edge before it is skipped, and how - /// long a missing group is waited for, or null for no subscriber limit, leaving the + /// far newer media may run past a missing group before it is given up, or null for no subscriber limit, leaving the /// publisher's max age alone to bound it. `0` skips immediately; a larger value tolerates /// that much reordering. It bounds lateness only, never the start. /// diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index d75c7f05d7..e4f667f52a 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -74,7 +74,8 @@ public typealias Subscription = MoqFFI.MoqSubscription /// cold join starts, `nil` to leave it unstated (a decoder starts it as zero; a catalog warmup or /// an audio decoder's playout delay still states a `min` to the publisher), and `max`, `nil` for no /// subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how -/// long a decoder waits for a missing group. A `min` past `max` starts at `max`. +/// far newer media may run past a missing group before a decoder gives it up. A `min` past `max` +/// starts at the group covering the point `max` behind the live edge. public typealias Delay = MoqFFI.MoqDelay /// Options for fetching one complete group by start. public typealias FetchGroupOptions = MoqFFI.MoqFetchGroupOptions From 0e36d3586b0cce1d93052bf1e194bd7df52ecc9e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 17:13:21 -0800 Subject: [PATCH 17/23] fix(mux): an aborted group keeps an endpoint it already decoded Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/container/consumer.rs | 37 ++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index abe2f0d194..d510f82917 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -1516,9 +1516,9 @@ impl GroupBuffer { } /// True if this group holds nothing to deliver and never will: aborted with nothing - /// buffered and no fault to report. + /// buffered, no endpoint decoded, and no fault to report. fn reapable(&mut self, waiter: &kio::Waiter) -> bool { - self.buffered.is_empty() && self.fault.is_none() && self.poll_aborted(waiter) + self.buffered.is_empty() && self.markers.is_empty() && self.fault.is_none() && self.poll_aborted(waiter) } fn note_end(&mut self, end: std::time::Duration) { @@ -1860,6 +1860,39 @@ mod tests { items } + /// An endpoint looked ahead in the next group is delivered though that group then aborts: it is + /// decoded, so only a group holding nothing at all is reaped. + #[tokio::test(start_paused = true)] + async fn a_looked_ahead_endpoint_outlives_its_groups_abort() { + let mut track = track_producer("gaps", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let subscription = track.subscribe( + moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(10))), + ); + let mut consumer = Consumer::new(subscription, Container::Legacy(crate::container::Kind::Audio)); + let mut head = track.create_group(moq_net::group::Info { start: 0 }).unwrap(); + Container::Legacy(crate::container::Kind::Audio) + .write( + &mut head, + &[Frame { + timestamp: ts(0), + payload: Bytes::from_static(&[0xDE, 0xAD]), + keyframe: true, + duration: None, + }], + ) + .unwrap(); + assert!(matches!(consumer.read().await.unwrap(), Some(Read::Frame { group: 0, .. }))); + let mut next = track.create_group(moq_net::group::Info { start: 1 }).unwrap(); + write_marker(&mut next, ts(20_000)); + // Group 0 is still open, so the read blocks and looks ahead into group 1. + assert!(consumer.poll_read(&kio::Waiter::noop()).is_pending()); + next.abort(moq_net::Error::Cancel).unwrap(); + assert!(consumer.poll_read(&kio::Waiter::noop()).is_pending()); + head.finish().unwrap(); + let _track = track.finish().unwrap(); + assert_eq!(items(&mut consumer).await, ["end 20000"]); + } + /// Joining a paused track, whose only served group is a marker, reads the end without /// waiting for media to resume. #[tokio::test(start_paused = true)] From 1fda6e816612ae5b5505f04f056e73a4c10cbb3a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 17:28:30 -0800 Subject: [PATCH 18/23] style: format the new test and Kotlin doc Co-Authored-By: Claude Opus 5.5 --- kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt | 6 +++--- rs/moq-mux/src/container/consumer.rs | 7 +++++-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index e857201054..13fdd9a8c9 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -206,9 +206,9 @@ typealias Subscription = uniffi.moq.MoqSubscription * How far behind the live edge a subscriber or decoder plays: `min` is the target latency, where a * cold join starts, null to leave it unstated (a decoder starts it as zero; a catalog warmup or an * audio decoder's playout delay still states a `min` to the publisher), and `max`, null for no - * subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how - * far newer media may run past a missing group before a decoder gives it up. A `min` past `max` - * starts at the group covering the point `max` behind the live edge. + * subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how far + * newer media may run past a missing group before a decoder gives it up. A `min` past `max` starts + * at the group covering the point `max` behind the live edge. */ typealias Delay = uniffi.moq.MoqDelay diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index d510f82917..27e01ac5cb 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -1864,7 +1864,7 @@ mod tests { /// decoded, so only a group holding nothing at all is reaped. #[tokio::test(start_paused = true)] async fn a_looked_ahead_endpoint_outlives_its_groups_abort() { - let mut track = track_producer("gaps", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let track = track_producer("gaps", hang::container::track_info(hang::catalog::PRIORITY.audio)); let subscription = track.subscribe( moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(10))), ); @@ -1881,7 +1881,10 @@ mod tests { }], ) .unwrap(); - assert!(matches!(consumer.read().await.unwrap(), Some(Read::Frame { group: 0, .. }))); + assert!(matches!( + consumer.read().await.unwrap(), + Some(Read::Frame { group: 0, .. }) + )); let mut next = track.create_group(moq_net::group::Info { start: 1 }).unwrap(); write_marker(&mut next, ts(20_000)); // Group 0 is still open, so the read blocks and looks ahead into group 1. From 1cb4d722ab27055d3f4946ee1738c525098fec14 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 17:49:33 -0800 Subject: [PATCH 19/23] fix(mux): the join is taken from the subscription's answer, not the first item A group the ordered reader still waits on let later groups raise the largest before the first item, so the join landed past where the publisher stood and a later gap could fold as backlog. Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/container/consumer.rs | 55 ++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 27e01ac5cb..0ac9f0f48e 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -573,6 +573,12 @@ impl Consumer { } fn poll_event_once(&mut self, waiter: &kio::Waiter) -> Poll, F::Error>> { + // Where the publisher stood as the subscription began, read as soon as its answer reports a + // largest, before a group the reader waits on lets later ones raise it. A largest not yet in + // range takes the first one that is. + if self.joined.is_none() { + self.join(waiter); + } // Grab any new groups from the track, recording whether the track is finished. let finished = self.poll_read_finish(waiter).is_ready(); @@ -2244,6 +2250,55 @@ mod tests { } } + /// The join's boundary is where the publisher stood as it answered, though the groups it owes + /// are held up while the publisher moves on: the first item the ordered reader hands out comes + /// after the largest has passed it. + #[tokio::test(start_paused = true)] + async fn the_join_is_the_answer_though_the_first_group_is_held() { + use crate::container::test_util::{Link, origin}; + + let (publisher, subscriber) = (origin(1), origin(2)); + // SETUP and the Announce Stream take the first two uni streams, so every group waits. + let link = Link::held(&publisher, &subscriber, 2).await; + let route = publisher + .route("room", moq_net::origin::RouteOptions::hidden()) + .unwrap(); + let mut track = route + .broadcast() + .create_track("data", hang::container::track_info(hang::catalog::PRIORITY.audio)) + .unwrap(); + route.announce().unwrap(); + let remote = subscriber + .consume() + .follow("room", None) + .unwrap() + .available() + .await + .unwrap() + .unwrap(); + write_group(&mut track, 0, &[ts(0)]); + let subscription = remote + .track("data") + .unwrap() + .subscribe( + moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))), + ) + .await + .unwrap(); + let mut reader = Consumer::new(subscription, Container::Legacy(crate::container::Kind::Audio)); + carried().await; + assert!(reader.poll_read(&kio::Waiter::noop()).is_pending()); + write_group(&mut track, 1, &[ts(20_000)]); + write_group(&mut track, 2, &[ts(40_000)]); + link.open(); + carried().await; + assert!(matches!( + reader.read().await.unwrap(), + Some(Read::Frame { group: 0, .. }) + )); + assert_eq!(reader.joined, Some(0)); + } + /// The join's boundary is where the publisher stood as it answered, not the newer largest by the /// time a live start found a frame to start on. #[tokio::test(start_paused = true)] From 35f634b98c3f0746be9b2b01ccac2300f896e5d1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 20:07:59 -0800 Subject: [PATCH 20/23] fix(hang,audio): a step onto an open head waits for its first frame, and a backlog given up before the first observation still extends the join - @moq/hang: the ordered read finishing no longer settles a step onto a frameless open group as a loss; its first frame or end decides it. - moq-audio and @moq/hang's estimator hold a backlog reported before the first observation, since observations now carry landing times and a newest-first replay can land the given-up group after the first one read. - moq-mux: two local-track tests run on kio's simulated clock. - Bindings: a min past max bounds a cold join's start, not every range. Co-Authored-By: Claude Opus 5.5 --- dart/moq/lib/src/aliases.dart | 3 +- doc/concept/audio-jitter.md | 4 +- go/wrapper/records.go | 4 +- .../container/consumer.nonsequential.test.ts | 35 +++++++++++++++++ js/hang/src/container/consumer.ts | 10 ++--- js/hang/src/container/jitter.test.ts | 19 ++++++++++ js/hang/src/container/jitter.ts | 10 +++-- .../kotlin/dev/moq/Aliases.kt | 2 +- py/moq-rs/moq/_records.py | 4 +- rs/libmoq/src/api.rs | 4 +- rs/moq-audio/src/jitter.rs | 38 ++++++++++++++++--- rs/moq-ffi/src/consumer.rs | 4 +- rs/moq-mux/src/container/consumer.rs | 6 +-- swift/Sources/Moq/Aliases.swift | 2 +- 14 files changed, 113 insertions(+), 32 deletions(-) diff --git a/dart/moq/lib/src/aliases.dart b/dart/moq/lib/src/aliases.dart index e72a0235d2..66bf14e5c5 100644 --- a/dart/moq/lib/src/aliases.dart +++ b/dart/moq/lib/src/aliases.dart @@ -164,7 +164,8 @@ extension type Delay._(MoqDelay _delay) implements MoqDelay { /// Both default to zero, moq-net's default. An explicit null `min` leaves it unstated, which a /// decoder starts as zero (a catalog warmup or an audio decoder's playout delay still states a /// `min` to the publisher), and an explicit null `max` sets no subscriber limit. A `min` past - /// `max` starts at the group covering the point `max` behind the live edge. + /// `max` starts a cold join no further back than the group covering the point `max` behind the + /// live edge. Delay({Duration? min = Duration.zero, Duration? max = Duration.zero}) : this._(MoqDelay(min: min, max: max)); } diff --git a/doc/concept/audio-jitter.md b/doc/concept/audio-jitter.md index f884106e5a..a2c25be2e7 100644 --- a/doc/concept/audio-jitter.md +++ b/doc/concept/audio-jitter.md @@ -115,7 +115,7 @@ observe(arrival, media): push (arrival, media) onto history fastest = min over history of (entry.arrival - entry.media) - if first observation: start = arrival; joined = arrival + if first observation: start = arrival; joined = max(joined, arrival) elif media + fastest < start: # produced before the subscription: backlog joined = max(joined, arrival) @@ -165,7 +165,7 @@ settle(): # a stall is one observation fold(stall.bucket); fold(nothing) min(stall.span, IDLE_MAX) times; close it backlog(arrival): # a backlog group skipped unread arrived - if not first observation and open: + if open: # before the first observation too: it may land later joined = max(joined, arrival) end(): # a declared end precedes the next observation diff --git a/go/wrapper/records.go b/go/wrapper/records.go index 15fb0b852e..1bd60c941a 100644 --- a/go/wrapper/records.go +++ b/go/wrapper/records.go @@ -76,8 +76,8 @@ type DatagramInfo = ffi.MoqDatagramInfo // starts at the newest group and skips a superseded group at once. type Delay struct { // Min is the target latency: the playout or jitter-buffer delay held behind the live edge, - // where a cold join starts. A missing group is given up by Max alone. A Min past Max starts at - // the group covering the point Max behind the live edge, since the publisher serves no older group. moq-lite-07 sends it upstream. + // where a cold join starts. A missing group is given up by Max alone. A Min past Max starts a + // cold join no further back than the group covering the point Max behind the live edge, since the publisher serves no older group. moq-lite-07 sends it upstream. Min time.Duration // MinUnstated leaves Min unstated, ignoring it. A decoder starts it as zero, while the // subscription keeps it unstated, so it never pulls the aggregate across subscribers to diff --git a/js/hang/src/container/consumer.nonsequential.test.ts b/js/hang/src/container/consumer.nonsequential.test.ts index 7a90aaa3cc..c977e0f656 100644 --- a/js/hang/src/container/consumer.nonsequential.test.ts +++ b/js/hang/src/container/consumer.nonsequential.test.ts @@ -185,3 +185,38 @@ test("Consumer reads one-packet Opus groups at sparse sequences back to back", a track.finish(); track.close(); }); + +// The ordered read can finish while its last group is still open, before that group's first frame +// lands. The finish says nothing of where that frame starts, so a step onto the group waits for it, +// and a frame continuing the timeline reads on without a gap. +test("Consumer judges a step onto a frameless group by its first frame, not the track's finish", async () => { + const track = new Track.Producer("test"); + const config = AudioConfigSchema.parse({ + codec: "opus", + container: { kind: "legacy" }, + sampleRate: 48_000, + numberOfChannels: 2, + }); + const consumer = new Consumer(track.subscribe({ delay: { max: Duration.millis(5000) } }), { + format: new LegacyFormat(config), + delay: { min: null, max: Duration.millis(5000) }, + }); + // One 20 ms CELT packet per group. + const packet = new Uint8Array([31 << 3, 0]); + const first = track.createGroup(0n); + first.writeFrame({ payload: encodeLegacyFrame(0, packet), timestamp: Time.Timestamp.fromMicros(0) }); + first.finish(); + expect(await poll(consumer.next())).toMatchObject({ kind: "frame", group: 0n }); + + // Key 1 is never written, and the track finishes while group 2 is open and still empty. + const last = track.createGroup(2n); + track.finish(); + const next = consumer.next(); + expect(await poll(next)).toBe("pending"); + last.writeFrame({ payload: encodeLegacyFrame(20_000, packet), timestamp: Time.Timestamp.fromMicros(20_000) }); + last.finish(); + expect(await poll(next)).toMatchObject({ kind: "frame", group: 2n }); + + consumer.close(); + track.close(); +}); diff --git a/js/hang/src/container/consumer.ts b/js/hang/src/container/consumer.ts index 557288f499..2c23851c47 100644 --- a/js/hang/src/container/consumer.ts +++ b/js/hang/src/container/consumer.ts @@ -939,14 +939,10 @@ export class Consumer { ptsContiguous(this.#presentedEnd, first); const empty = head.empty && head.consumer.done; // A payload that failed to decode gives no timestamp, so its fault settles the step, - // for the read below to raise. + // for the read below to raise. The read's own end says nothing of where an open head + // starts, and reading waits on the head either way. const known = - contiguous || - head.gap || - first !== undefined || - head.fault !== undefined || - head.consumer.done || - ended !== undefined; + contiguous || head.gap || first !== undefined || head.fault !== undefined || head.consumer.done; if (known) { // A gap an empty group carries still marks the next frame, which follows the loss. if (head.gap && !contiguous) { diff --git a/js/hang/src/container/jitter.test.ts b/js/hang/src/container/jitter.test.ts index 77cb424226..674789407f 100644 --- a/js/hang/src/container/jitter.test.ts +++ b/js/hang/src/container/jitter.test.ts @@ -123,6 +123,25 @@ describe("Jitter", () => { } }); + it("holds a backlog skipped before the first observation until the join starts", () => { + // A newest-first replay lands the newer group first, so the backlog given up before anything was + // observed can land after the first observed frame. The join lasts until it did either way, so + // the frames held behind it fold nothing. + const traces = [undefined, 0, 1].map((at) => { + const jitter = new Jitter(); + const trace: number[] = []; + for (let i = 0; i < 100; i++) { + if (i === at) jitter.backlog(1000); + jitter.observe(500 + i * 20 + (i >= 5 && i < 10 ? 200 : 0), i * 20); + trace.push(jitter.measured); + } + return trace; + }); + const [plain, early, late] = traces; + expect(early).toEqual(late); + expect(late).not.toEqual(plain); + }); + it("does not follow the absolute delay", () => { // Only the spread of arrivals matters, not how far away the sender is. const near = new Jitter(); diff --git a/js/hang/src/container/jitter.ts b/js/hang/src/container/jitter.ts index 63161c9f74..94f8e3da49 100644 --- a/js/hang/src/container/jitter.ts +++ b/js/hang/src/container/jitter.ts @@ -68,7 +68,7 @@ export class Jitter { #oldest = 0; /** The first arrival, when the subscription began, unset until the first observation. */ #start?: number; - /** The latest a frame produced before the subscription arrived, starting at `#start`. */ + /** The latest a frame produced before the subscription arrived, no earlier than `#start`. */ #joined = 0; /** Every interval closed so far measured the join, so a skipped backlog can still extend it. */ #open = true; @@ -106,9 +106,11 @@ export class Jitter { // A frame due before the first arrival by the window's fastest path was produced before the // subscription began: it is the backlog a join delivers, and the join lasts until it has all // arrived. + // A backlog skipped before this may have landed after it, as when a newest-first replay lands + // the newer group first. if (this.#start === undefined) { this.#start = arrival; - this.#joined = arrival; + this.#joined = Math.max(this.#joined, arrival); } else if (media + this.#history.fastest() < this.#start) { this.#joined = Math.max(this.#joined, arrival); } @@ -226,9 +228,9 @@ export class Jitter { * milliseconds and was skipped unread, so the join lasts until then as if it had been observed. */ backlog(arrival: number): void { - // Before any observation the join has not started, and starts later than this. + // Before any observation the join has not started, and lasts at least until this once it does. // Once an interval measured the path the join is over, and a skip is an outage's. - if (this.#start !== undefined && this.#open) this.#joined = Math.max(this.#joined, arrival); + if (this.#open) this.#joined = Math.max(this.#joined, arrival); } /** diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 13fdd9a8c9..2b8deb1c5a 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -208,7 +208,7 @@ typealias Subscription = uniffi.moq.MoqSubscription * audio decoder's playout delay still states a `min` to the publisher), and `max`, null for no * subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how far * newer media may run past a missing group before a decoder gives it up. A `min` past `max` starts - * at the group covering the point `max` behind the live edge. + * a cold join no further back than the group covering the point `max` behind the live edge. */ typealias Delay = uniffi.moq.MoqDelay diff --git a/py/moq-rs/moq/_records.py b/py/moq-rs/moq/_records.py index 2e12ebbd74..1043e77843 100644 --- a/py/moq-rs/moq/_records.py +++ b/py/moq-rs/moq/_records.py @@ -220,8 +220,8 @@ class Delay: live edge, where a cold join starts. ``None`` leaves it unstated, which a decoder starts as zero, while the subscription keeps it unstated, so it never pulls the aggregate across subscribers to zero, unless a catalog warmup or an audio decoder's playout delay states - one. A ``min`` past ``max`` starts at the group covering the point - ``max`` behind the live edge. + one. A ``min`` past ``max`` starts a cold join no further back than the + group covering the point ``max`` behind the live edge. ``max`` is how far a non-latest group may fall behind before it is skipped, and how far newer media may run past a missing group before a decoder gives it up; zero skips at once, ``None`` sets no subscriber limit and leaves the publisher's max age diff --git a/rs/libmoq/src/api.rs b/rs/libmoq/src/api.rs index b0febcdc36..22773ef1fe 100644 --- a/rs/libmoq/src/api.rs +++ b/rs/libmoq/src/api.rs @@ -827,8 +827,8 @@ pub enum moq_refresh { pub struct moq_delay { /// The target latency in microseconds: the playout or jitter-buffer delay held behind /// the live edge, where a cold join starts. A missing group is given up by the max alone. - /// moq-lite-07 sends it upstream. A `min_us` past the max starts at the group covering the - /// point the max behind the live edge, since the publisher serves no older group. Ignored when `min_unstated` is set. + /// moq-lite-07 sends it upstream. A `min_us` past the max starts a cold join no further back than the + /// group covering the point the max behind the live edge, since the publisher serves no older group. Ignored when `min_unstated` is set. pub min_us: u64, /// How far a non-latest group may fall behind the live edge before it is skipped, in /// microseconds. Zero skips immediately. Enforced by the publisher's cache and by any diff --git a/rs/moq-audio/src/jitter.rs b/rs/moq-audio/src/jitter.rs index dcd3c48210..6b7e71c576 100644 --- a/rs/moq-audio/src/jitter.rs +++ b/rs/moq-audio/src/jitter.rs @@ -296,7 +296,7 @@ pub(crate) struct Estimator { oldest: f64, /// The first arrival, when the subscription began, once one has been seen. start: Option, - /// The latest a frame produced before the subscription arrived, starting at `start`. + /// The latest a frame produced before the subscription arrived, no earlier than `start`. joined: f64, /// Every interval closed so far measured the join, so a skipped backlog can still extend it. open: bool, @@ -353,9 +353,11 @@ impl Estimator { // before the subscription began: it is the backlog a join delivers, and the join // lasts until it has all arrived. match self.start { + // A backlog skipped before this may have landed after it, as when a newest-first + // replay lands the newer group first. None => { self.start = Some(arrival); - self.joined = arrival; + self.joined = self.joined.max(arrival); } Some(start) => { if media + self.history.fastest() < start { @@ -485,9 +487,9 @@ impl Estimator { /// A frame produced before the subscription, skipped unread, so the join lasts /// until it arrived as if it had been observed. fn backlog(&mut self, arrival: f64) { - // Before any observation the join has not started, and starts later than this. Once - // an interval measured the path the join is over, and a skip is an outage's. - if self.start.is_some() && self.open { + // Before any observation the join has not started, and lasts at least until this once it + // does. Once an interval measured the path the join is over, and a skip is an outage's. + if self.open { self.joined = self.joined.max(arrival); } } @@ -666,6 +668,32 @@ pub(crate) mod tests { } } + /// A newest-first replay lands the newer group first, so a backlog given up before + /// anything was observed can land after the first observed frame. The join lasts until + /// it did either way, so the frames held behind it fold nothing. + #[test] + fn holds_a_backlog_skipped_before_the_first_observation_until_the_join_starts() { + let traces: Vec> = [None, Some(0), Some(1)] + .into_iter() + .map(|at| { + let mut target = Target::new(Duration::ZERO, true, None); + (0..100) + .map(|i| { + if at == Some(i) { + target.backlog(1000.0); + } + let late = if (5..10).contains(&i) { 200.0 } else { 0.0 }; + target.observe(500.0 + i as f64 * 20.0 + late, i as f64 * 20.0); + target.millis() + }) + .collect() + }) + .collect(); + let [plain, early, late] = &traces[..] else { unreachable!() }; + assert_eq!(early, late); + assert_ne!(late, plain); + } + /// The #3517 regression, without the corpus: a stale frame then paced audio far /// ahead of it must not carry the media-time gap into the target. #[test] diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index c8094fe37c..9543aed9fe 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -29,8 +29,8 @@ pub struct MoqDelay { /// alone. A media consumer starts an unstated minimum as zero, while the subscription keeps /// it unstated, so it never pulls the aggregate across subscribers to zero, unless a catalog /// warmup or an audio decoder's playout delay states one. moq-lite-07 sends it upstream. A - /// `min` past `max` starts at the group covering the point `max` behind the live edge, since - /// the publisher serves no older group. An audio decoder refuses a `min` beyond the track's + /// `min` past `max` starts a cold join no further back than the group covering the point `max` + /// behind the live edge, since the publisher serves no older group. An audio decoder refuses a `min` beyond the track's /// max age instead. pub min: Option, /// How far a non-latest group may fall behind the live edge before it is skipped, and how diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 0ac9f0f48e..231069a15e 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -1868,7 +1868,7 @@ mod tests { /// An endpoint looked ahead in the next group is delivered though that group then aborts: it is /// decoded, so only a group holding nothing at all is reaped. - #[tokio::test(start_paused = true)] + #[kio::test] async fn a_looked_ahead_endpoint_outlives_its_groups_abort() { let track = track_producer("gaps", hang::container::track_info(hang::catalog::PRIORITY.audio)); let subscription = track.subscribe( @@ -2193,13 +2193,13 @@ mod tests { /// A join's backlog the ordered reader gives up as stale, never handing it to this reader, still /// reports when it landed: its gap carries the landing. - #[tokio::test(start_paused = true)] + #[kio::test] async fn a_stale_backlog_the_ordered_reader_gives_up_reports_when_it_landed() { let mut track = track_producer("data", hang::container::track_info(hang::catalog::PRIORITY.audio)); let landed = kio::clock::now(); write_group(&mut track, 0, &[ts(0)]); write_group(&mut track, 1, &[ts(20_000)]); - tokio::time::advance(Duration::from_millis(100)).await; + kio::clock::sim::advance(Duration::from_millis(100)).await; write_group(&mut track, 2, &[ts(40_000)]); let subscriber = track.subscribe( moq_net::track::Subscription::default().with_delay(Delay::default().with_max(Duration::from_millis(50))), diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index e4f667f52a..d44207842a 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -75,7 +75,7 @@ public typealias Subscription = MoqFFI.MoqSubscription /// an audio decoder's playout delay still states a `min` to the publisher), and `max`, `nil` for no /// subscriber limit, is how far a non-latest group may fall behind before it is skipped, and how /// far newer media may run past a missing group before a decoder gives it up. A `min` past `max` -/// starts at the group covering the point `max` behind the live edge. +/// starts a cold join no further back than the group covering the point `max` behind the live edge. public typealias Delay = MoqFFI.MoqDelay /// Options for fetching one complete group by start. public typealias FetchGroupOptions = MoqFFI.MoqFetchGroupOptions From daadb9b2f01b5f3d01db993cc6da714e4ac0fa44 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 20:15:27 -0800 Subject: [PATCH 21/23] style(audio): destructure the jitter traces with array map Co-Authored-By: Claude Opus 5.5 --- rs/moq-audio/src/jitter.rs | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/rs/moq-audio/src/jitter.rs b/rs/moq-audio/src/jitter.rs index 6b7e71c576..4d003c4245 100644 --- a/rs/moq-audio/src/jitter.rs +++ b/rs/moq-audio/src/jitter.rs @@ -673,23 +673,19 @@ pub(crate) mod tests { /// it did either way, so the frames held behind it fold nothing. #[test] fn holds_a_backlog_skipped_before_the_first_observation_until_the_join_starts() { - let traces: Vec> = [None, Some(0), Some(1)] - .into_iter() - .map(|at| { - let mut target = Target::new(Duration::ZERO, true, None); - (0..100) - .map(|i| { - if at == Some(i) { - target.backlog(1000.0); - } - let late = if (5..10).contains(&i) { 200.0 } else { 0.0 }; - target.observe(500.0 + i as f64 * 20.0 + late, i as f64 * 20.0); - target.millis() - }) - .collect() - }) - .collect(); - let [plain, early, late] = &traces[..] else { unreachable!() }; + let [plain, early, late] = [None, Some(0), Some(1)].map(|at| { + let mut target = Target::new(Duration::ZERO, true, None); + (0..100) + .map(|i| { + if at == Some(i) { + target.backlog(1000.0); + } + let late = if (5..10).contains(&i) { 200.0 } else { 0.0 }; + target.observe(500.0 + i as f64 * 20.0 + late, i as f64 * 20.0); + target.millis() + }) + .collect::>() + }); assert_eq!(early, late); assert_ne!(late, plain); } From 749af2646b22f4f5921825cc5b90ac1fd2d777e7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 19 Nov 2026 20:31:40 -0800 Subject: [PATCH 22/23] fix(hang): a gap given up ahead of the first group waits for the join's boundary rather than being dropped Co-Authored-By: Claude Opus 5.5 --- .../src/container/consumer.backlog.test.ts | 41 ++++++++++++++++++- js/hang/src/container/consumer.ts | 11 ++--- 2 files changed, 43 insertions(+), 9 deletions(-) diff --git a/js/hang/src/container/consumer.backlog.test.ts b/js/hang/src/container/consumer.backlog.test.ts index 1a84674d64..c111a4eee4 100644 --- a/js/hang/src/container/consumer.backlog.test.ts +++ b/js/hang/src/container/consumer.backlog.test.ts @@ -137,7 +137,7 @@ test("late groups dropped from the start hold no join open", async () => { * A track whose ordered read's info resolves once `answer` is called, and whose largest is what * `largest` says when asked. Its feed owes every key from `from`, as a session's would. */ -function joining(from: bigint, largest: () => bigint) { +function joining(from: bigint, largest: () => bigint | Promise) { const track = new Track.Producer("audio"); track.requests(); const subscriber = track.subscribe({ delay: { max: Duration.millis(2000) } }); @@ -157,7 +157,7 @@ function joining(from: bigint, largest: () => bigint) { const ordered = () => proxy(subscriber.ordered(), { info: () => answered.then(() => ({})), - largest: async () => ({ group: largest(), frame: 0 }), + largest: async () => ({ group: await largest(), frame: 0 }), }); const write = (sequence: number, media: number) => { const group = track.createGroup(BigInt(sequence)); @@ -267,6 +267,43 @@ test("the join's boundary is the publisher's answer, not the largest by the firs } }); +// A cached backlog: group 150 went stale behind 151 before the subscription, so the read gives it up and +// hands out 151 at once, both before the publisher's largest is known. The gap still waits for that +// boundary rather than being dropped with the first group. +test("a gap given up before the join's boundary is known still tells the estimate", async () => { + let settle!: (largest: bigint) => void; + const known = new Promise((resolve) => { + settle = resolve; + }); + const { track, subscriber, answer, write } = joining(150n, () => known); + const reported: number[] = []; + const backlog = spyOn(Jitter.prototype, "backlog").mockImplementation((arrival: number) => { + reported.push(arrival); + }); + const landed = now().since(Instant.ZERO).asMillis(); + write(150, 0); + await clock.advance(Duration.millis(10)); + write(151, 20); + await clock.advance(Duration.millis(2001)); + const consumer = new Consumer(subscriber, { + format: new Format("audio"), + delay: { min: Duration.millis(100), max: Duration.millis(2000) }, + }); + try { + answer(); + expect(await consumer.next()).toMatchObject({ kind: "frame", group: 151n }); + expect(reported).toEqual([]); + settle(200n); + await clock.advance(Duration.ZERO); + expect(reported).toEqual([landed]); + } finally { + backlog.mockRestore(); + consumer.close(); + track.finish(); + track.close(); + } +}); + test("a range capped below the largest still delivers", async () => { // The track holds groups 0 to 5, and the subscriber's range ends before 3, so it never reports a // largest: the consumer reports no backlog, but delivers what it can read. diff --git a/js/hang/src/container/consumer.ts b/js/hang/src/container/consumer.ts index 2c23851c47..5bed15fa19 100644 --- a/js/hang/src/container/consumer.ts +++ b/js/hang/src/container/consumer.ts @@ -186,7 +186,7 @@ export class Consumer { // The ordered read gave up on groups since the last one it returned. #gap = false; // When each gap given up before the join's boundary was known landed, by the key it ends at: told to - // the estimate once the boundary is known, or dropped once the first group is handed out. + // the estimate once the boundary is known. Only gaps ahead of the first group are kept, which bounds it. #unjoined?: { end: bigint; landed: Instant }[] = []; #format: Format; #delay: Getter; @@ -331,11 +331,7 @@ export class Consumer { } // Delivery starts at the first group: the reader already skipped what is too stale to // play, and waited for the groups below it that its feed owed. - if (this.#active === undefined) { - this.#active = item.group.start; - // What the join's boundary has not judged by now it never judges. - this.#unjoined = undefined; - } + if (this.#active === undefined) this.#active = item.group.start; this.#add(item.group, this.#gap); this.#gap = false; } @@ -357,7 +353,8 @@ export class Consumer { #backlog(end: bigint, landed: Instant): void { if (this.#observing) return; if (this.#joined === undefined) { - this.#unjoined?.push({ end, landed }); + // Only gaps ahead of the first group wait for it, which bounds the list a peer can grow. + if (this.#active === undefined) this.#unjoined?.push({ end, landed }); return; } if (end - 1n <= this.#joined) this.#jitter.backlog(landed.since(Instant.ZERO).asMillis()); From 576021a735f9c47b7e0c7e81d61e4d41ab38b984 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 20 Nov 2026 01:04:59 -0800 Subject: [PATCH 23/23] fix(audio): a packet held behind a hole keeps its landing time The decode consumer queued each packet at its read time, so a packet a hole held back got the gate's whole hold again and `arrival()` reported the late read. It now queues the container's landing time, the same instant the estimator and the video decoder use. Also: the audio-jitter page states the max-only hole wait, and the unlimited-max startup test is renamed to what it checks, with older history so it can fail. Co-Authored-By: Claude Opus 5.5 --- doc/concept/audio-jitter.md | 8 ++++---- rs/moq-audio/src/decode/consumer.rs | 26 +++++++++++++++++--------- rs/moq-mux/src/container/consumer.rs | 21 +++++++++++---------- 3 files changed, 32 insertions(+), 23 deletions(-) diff --git a/doc/concept/audio-jitter.md b/doc/concept/audio-jitter.md index a2c25be2e7..01ce39cf0b 100644 --- a/doc/concept/audio-jitter.md +++ b/doc/concept/audio-jitter.md @@ -46,10 +46,10 @@ Natively that holds only as far as the caller lets it. The container consumer reads groups in sequence order and applies its age budget inside `read()`, so a group skipped for age is never observed, and a frame queued behind a stalled group is observed when the stall clears. A missing group, one that has delivered -no frame, is waited for only the playout delay, the subscription's `delay.min`, -since past that it would play late: a reordered backlog's holes waited out to the whole budget -would delay the frames behind them, which the estimator would measure as -jitter. The browser player leaves the hole to the transport: it reads its audio +no frame, is given up by the subscription's `delay.max` alone, like any stale +group; `delay.min` only sets where a join starts. Each frame is timed by when it +landed in the node's cache, not when the read reached it, so the frames a hole +holds back read as on time rather than as jitter. The browser player leaves the hole to the transport: it reads its audio through an ordered track reader whose own `delay.max` is what the ring holds of the delay plus the stretch threshold, past which the ring would skip the hole's audio anyway, and plays each gap that reader reports as the hole. The reader diff --git a/rs/moq-audio/src/decode/consumer.rs b/rs/moq-audio/src/decode/consumer.rs index c7e8c9c5f8..dc01d7e2c7 100644 --- a/rs/moq-audio/src/decode/consumer.rs +++ b/rs/moq-audio/src/decode/consumer.rs @@ -385,8 +385,9 @@ impl Consumer { backpressure } - /// When the packet behind the latest decoded frame arrived, which a gate held - /// encoded may have kept well before [`read`](Self::read) handed the frame over. + /// When the packet behind the latest decoded frame landed in this node's cache + /// ([`moq_mux::container::Consumer::arrival`]), which a hole or a gate holding it encoded + /// may have kept well before [`read`](Self::read) handed the frame over. pub fn arrival(&self) -> Option { self.arrived } @@ -707,20 +708,21 @@ impl Consumer { } // Every packet the container hands over is an arrival, including one the // decoder will refuse, so it is observed before anything can fail. + // When the packet landed, not when it was read: a hole the container waits out + // holds the packets behind it, which arrived on time. The estimator and the + // gate's hold both count from it. + let landed = self.track.arrival().unwrap_or_else(now); match &read { Encoded::Frame { frame, .. } => { if let Some(scale) = frame.timestamp.scale() { - // When the packet landed, not when it was read: a hole the container - // waits out holds the packets behind it, which arrived on time. - let arrived = self.track.arrival().unwrap_or_else(now); - let arrival = jitter::millis(arrived.duration_since(self.origin)); + let arrival = jitter::millis(landed.duration_since(self.origin)); let media = frame.timestamp.value() as f64 * 1000.0 / scale.as_u64() as f64; self.target.observe(arrival, media); } } _ => self.breaks += 1, } - self.pending.push_back((read, kio::clock::Instant::now())); + self.pending.push_back((read, landed)); } Some(Ok(None)) => self.ended = Some(None), Some(Err(err)) => self.ended = Some(Some(err.into())), @@ -3061,8 +3063,8 @@ mod tests { } /// A hole the container waits out up to the max delay holds the packets behind it, which - /// landed on time: the estimate observes when they landed, not when they were read, so the - /// wait reads as no jitter. + /// landed on time: the estimate and [`Consumer::arrival`] use when they landed, not when + /// they were read, so the wait reads as no jitter. #[tokio::test(start_paused = true)] async fn a_hole_waited_out_to_the_max_delay_leaves_the_estimate() { use moq_mux::container::Container as _; @@ -3135,6 +3137,12 @@ mod tests { .unwrap() .expect("a packet past the hole"); } + // They report when they landed too, which a gate's hold counts from. + assert!( + consumer.arrival().is_some_and(|arrived| arrived <= since), + "a packet past the hole reports when it was read: {:?}, read from {since:?}", + consumer.arrival() + ); // Steady arrivals after it, read as they land, close the interval that measured the wait. let resumed = 6 + (since.elapsed().as_millis() / 20) as u64; for sequence in resumed..resumed + 100 { diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 71e714022f..c94e02aec3 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -4571,29 +4571,30 @@ mod tests { assert_eq!(consumer.warmup, DELAY_MAX); } - /// An unlimited max delay keeps the subscription unlimited, and the start still reaches the - /// earlier group the join owes. + /// An unlimited max delay keeps the subscription unlimited and skips nothing: a reader of the + /// oldest group reads all of the history the copy holds, even past the min delay's reach. #[tokio::test(start_paused = true)] - async fn an_unlimited_max_delay_still_reaches_the_min_delay() { + async fn an_unlimited_max_delay_reads_all_history() { let mut copy = Received::with_info(hang::container::track_info(hang::catalog::PRIORITY.video)).await; + write_group(&mut copy.track, 3, &[ts(3_000_000)]); write_group(&mut copy.track, 4, &[ts(4_500_000)]); write_group(&mut copy.track, 5, &[ts(5_000_000)]); - let subscriber = copy - .subscribe_with(moq_net::track::Subscription::default().with_delay(Delay::default().with_max(None))) - .await; - let mut consumer = Playout::new( + let playout = Playout::new( Delay { min: Some(Duration::from_millis(500)), max: None, }, Duration::ZERO, - ) - .consumer(subscriber, Container::Legacy(crate::container::Kind::Data)); + ); + let subscriber = copy + .subscribe_with(playout.subscription(moq_net::track::Subscription::default())) + .await; + let mut consumer = playout.consumer(subscriber, Container::Legacy(crate::container::Kind::Data)); assert_eq!(subscribed(&consumer).subscription().delay.max, None); let _track = copy.track.finish().unwrap(); let frames = read_all(&mut consumer).await.unwrap(); let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros().unwrap()).collect(); - assert_eq!(micros, vec![4_500_000, 5_000_000]); + assert_eq!(micros, vec![3_000_000, 4_500_000, 5_000_000]); } /// A subscription without a max delay reaches back for history, so it starts at once.