From 6894f638b50799d50c8952a6d51a2566302a3bf8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 04:15:04 -0800 Subject: [PATCH 01/24] chore(quest): claim quest/m1/ordered-owed-holes.md Quest-Claim: a79d167ebe40227f From c018da98f7e3e25fd8da3446e495a6d37e7f7ed1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 04:19:18 -0800 Subject: [PATCH 02/24] docs(net): record Ordered latest-read decision and stall evidence Co-Authored-By: GPT-6.1 Sol --- quest/m1/ordered-owed-holes.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/quest/m1/ordered-owed-holes.md b/quest/m1/ordered-owed-holes.md index 994c6a7139d..762cb90a37c 100644 --- a/quest/m1/ordered-owed-holes.md +++ b/quest/m1/ordered-owed-holes.md @@ -10,6 +10,23 @@ buffered snapshot at once instead of waiting for an owed lower group. The `poll_latest` caveat goes. `@moq/net`'s `Ordered` and `@moq/json`'s `latest()` do the same. +## Waiting + +- Maintainer: choose the new public Ordered read's shape and name before + implementation. Recommend Rust `poll_latest_group` with its async + `latest_group`, mirrored by JS `tryLatestGroup` and `latestGroup`: select + the highest eligible buffered sequence across all resume copies in one + read, pass lower owed holes permanently, and retain the same range, + delay budget, returned-group guard, and delivered watermark as ordinary + ordered reads. With nothing eligible, wait for a new candidate or end; + `next_group` / `nextGroup` keep waiting on owed holes. Alternative: + `poll_next_available_group` / `next_available_group` and + `tryNextAvailableGroup` / `nextAvailableGroup` yield the lowest eligible + group without waiting on holes, leaving snapshot consumers to drain + them to reach the newest. The direct newest read better expresses the + goal and avoids handing out every superseded group. Record the choice + here and remove this wait before implementing. + ## Plan Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 @@ -40,3 +57,15 @@ Milestone: ✅ m1 (moved from m2). Not chosen: m2. Public API: adds an `Ordered` read (Rust and JS); `snapshot::Consumer::latest()` and `@moq/json`'s `latest()` stop waiting at owed holes. Wire: none. + +Reproduced 2026-11-03 on the base of +[moq-dev/moq#7736](https://github.com/moq-dev/moq/pull/7736): after reading +snapshot group 0, buffer group 2 while the received track still owes group +1. JS snapshot `latest()` remains pending with `delay.max` of 1,000 ms or +`null`; advancing a manual clock one hour does not release the unlimited +read. Delivering group 1 releases it with state 1, despite group 2 already +being buffered. A two-case scratch probe passed with all pending reads +settled during cleanup. Rust's existing +`next_group_waits_on_an_owed_group_within_the_budget` test also passed, +confirming finite and unlimited owed-hole waits in the Ordered read that +the Rust snapshot consumer drains. No production change is implemented. From 0e7489d81c009c280d695cbac1ea700576f78095 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 12:58:43 -0800 Subject: [PATCH 03/24] wip: snapshot consumers read at zero delay --- doc/lib/js/json.md | 11 ++- doc/lib/rs/moq-json.md | 8 +- js/json/MIGRATE.md | 6 ++ js/json/src/snapshot/consumer.ts | 20 +++-- js/json/src/snapshot/snapshot.test.ts | 87 ++++++++++++------- rs/moq-json/MIGRATE.md | 6 ++ rs/moq-json/src/lib.rs | 4 +- rs/moq-json/src/snapshot/consumer.rs | 38 ++++---- rs/moq-json/src/snapshot/mod.rs | 11 +-- rs/moq-mux/src/json.rs | 3 +- rs/moq-net-tests/tests/integration/main.rs | 3 +- .../tests/integration/snapshot.rs | 44 ++++++++++ 12 files changed, 162 insertions(+), 79 deletions(-) create mode 100644 rs/moq-net-tests/tests/integration/snapshot.rs diff --git a/doc/lib/js/json.md b/doc/lib/js/json.md index 1fd5044a997..3ec744712f6 100644 --- a/doc/lib/js/json.md +++ b/doc/lib/js/json.md @@ -27,18 +27,17 @@ an untimed track (no `timescale`) takes none and reads back without one. ```ts import { Snapshot } from "@moq/json"; -// Hold each state back until the video playhead reaches it. +// Schedule each state at its capture time. const consumer = new Snapshot.Consumer({ track }); for await (const { value, at } of consumer) { schedule(value, at); } ``` -`next()` buffers every state it has not yielded and skips a group once the -subscription's `delay.max` proves it stale. The budget counts media time and -wall time, or wall time alone for an untimed track. The default subscription's -zero keeps only the newest group, so subscribe with a `delay.max` covering how -far the playhead trails the live edge. Use `latest()` to skip straight to the head. +`next()` yields every buffered state of the group it is reading and skips a +group once a newer one supersedes it: the consumer always reads at a zero +`delay.max`, whatever the subscription asked for. Use `latest()` to skip +straight to the head. A stream rides one group with no record-count or cumulative byte limit. The group retains a 32 MiB tail; a reader that falls behind gets `Evicted` rather diff --git a/doc/lib/rs/moq-json.md b/doc/lib/rs/moq-json.md index dfb51393929..d0615b9dddc 100644 --- a/doc/lib/rs/moq-json.md +++ b/doc/lib/rs/moq-json.md @@ -26,10 +26,10 @@ same optional capture time when opened, including when they publish on drop. Consumers return `Timed`: read `.value` for the decoded value and `.at` for its optional frame timestamp, with the track's exact scale preserved. Snapshot -`next()` yields states in order inside the subscription's `delay.max` budget; -set that budget when following a playhead behind live. Use `latest()` to skip -to the newest buffered state. Skipped states are gone, and a later `next()` -continues after them. +`next()` yields states in order, skipping every group a newer one supersedes: +the consumer always reads at a zero `delay.max`, whatever the subscription +asked for. Use `latest()` to skip to the newest buffered state. Skipped states +are gone, and a later `next()` continues after them. Upgrading from an older release: see moq-json's [MIGRATE.md](https://github.com/moq-dev/moq/blob/main/rs/moq-json/MIGRATE.md). diff --git a/js/json/MIGRATE.md b/js/json/MIGRATE.md index d32c2324c0a..e9ed0db8c5b 100644 --- a/js/json/MIGRATE.md +++ b/js/json/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay + +`Snapshot.Consumer` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. +`next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. +A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `Snapshot.Decoder`, or publish it as a `Window` or `Stream` track when every state must survive. + ### Compression moves to the track The `Compression` type, every `compression` option, `Desync`, `Stream.Decoder.reset`, and `Window.DecoderConfig` are gone, along with the `@moq/flate` dependency. diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 6914da66761..73a2d06f087 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -20,6 +20,10 @@ const GAPS: Moq.StreamCode[] = [ * and {@link latest} skips to the newest state, for a caller that only wants the current value. * What {@link latest} skips is gone, so a later {@link next} resumes after it. * When something else already owns the track, use the {@link Decoder} directly. + * + * A snapshot is superseded the moment a newer one exists, so the consumer always reads at a zero + * `delay.max`, whatever the subscription asked for: a group is skipped once a newer one arrives, + * and a lower group the subscription still owes never holds up a newer one. */ export class Consumer { #track: Moq.Track.Ordered; @@ -29,6 +33,12 @@ export class Consumer { #framesRead = 0; constructor(config: Consumer.Config) { + // The default subscription's budget is zero. + const zero = Moq.Track.Subscription.default().delay?.max; + const current = config.track.subscription.peek(); + if (current?.delay?.max !== zero) { + config.track.update({ ...current, delay: { ...current?.delay, max: zero } }); + } this.#track = config.track.ordered(); this.#decoder = new Decoder(config); } @@ -36,12 +46,9 @@ export class Consumer { /** * Get the next state in order, or `undefined` once the track ends. * - * Yields one state per frame and finishes a group before starting the next, buffering every - * state the reader has not reached yet. A group the subscription's `delay.max` proves too old - * is skipped, counting media time and wall time, or wall time alone for an untimed track. The - * default subscription's zero keeps only the newest group, so a playhead reader sets `delay.max` - * to how far it trails the live edge. Use {@link latest} to skip straight to the head. A group lost to a - * gap resyncs from the next one. + * Yields one state per frame from the group it is reading, and skips every group a newer one + * supersedes. Use {@link latest} to skip straight to the head. A group lost to a gap resyncs + * from the next one. */ next(): Promise | undefined> { return this.#read(false); @@ -138,6 +145,7 @@ export class Consumer { export namespace Consumer { /** Snapshot consumer options, including the source track. */ export type Config = Pick, "schema"> & { + /** The track to read, whose `delay.max` the consumer sets to zero. */ track: Moq.Track.Subscriber; }; } diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index 4448b20971a..a3449d8c0d9 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -1,14 +1,15 @@ import { expect, test } from "bun:test"; import { Milli } from "@moq/clock"; import { Error as NetError, StreamCode, Time, Track } from "@moq/net"; +import { hooks } from "../../../net/src/internal.ts"; import { Consumer } from "./consumer.ts"; import { Encoder } from "./encoder.ts"; import { Producer } from "./producer.ts"; type Value = Record; -// These tests inspect complete finished timelines, so request a replay window -// instead of the transport's live-edge default. +// These tests inspect complete finished timelines through raw ordered reads, so request a +// replay window instead of the transport's live-edge default. A snapshot consumer ignores it. const REPLAY_LATENCY = Milli(30_000); // Reconstruct every state a consumer yields from `next()`, in order. @@ -53,7 +54,7 @@ test("a cut makes the next update a snapshot group", async () => { // The ratio would have kept every update in one group; the cut rolled it anyway. The replacement // opens with a full snapshot, so it is one frame rather than a delta appended to the first group. expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([2, 1]); - expect((await drain(track.subscribe({ delay: { max: REPLAY_LATENCY } }))).at(-1)).toEqual({ a: 1, b: 3 }); + expect(await drain(track.subscribe())).toEqual([{ a: 1, b: 3 }]); }); test("a cut republishes an unchanged value", async () => { @@ -115,15 +116,18 @@ test("a cut is inert without deltas", async () => { test("deltas off: a snapshot group per change", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: false }); + const consumer = new Consumer({ track: track.subscribe() }); producer.update({ value: { a: 1 } }); + expect((await consumer.next())?.value).toEqual({ a: 1 }); producer.update({ value: { a: 2 } }); + expect((await consumer.next())?.value).toEqual({ a: 2 }); producer.finish(); - // Two changes => two single-frame snapshot groups. `next()` reads both in order, while `latest()` - // collapses the backlog to the newest value: older groups only hold superseded state (mirrors the - // Rust consumer). The layout itself is asserted via structure() below. - expect(await drain(track.subscribe({ delay: { max: REPLAY_LATENCY } }))).toEqual([{ a: 1 }, { a: 2 }]); - expect(await drainLatest(track.subscribe({ delay: { max: REPLAY_LATENCY } }))).toEqual([{ a: 2 }]); + // Two changes => two single-frame snapshot groups. A reader arriving after both reads only the + // newest: the older group holds superseded state (mirrors the Rust consumer). + expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([1, 1]); + expect(await drain(track.subscribe())).toEqual([{ a: 2 }]); + expect(await drainLatest(track.subscribe())).toEqual([{ a: 2 }]); }); // `latest()` holding a group must jump when a newer snapshot group rolls: the @@ -133,7 +137,7 @@ test("deltas off: a snapshot group per change", async () => { test("latest abandons a held group when a newer snapshot group rolls", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); - const consumer = new Consumer({ track: track.subscribe({ delay: { max: REPLAY_LATENCY } }) }); + const consumer = new Consumer({ track: track.subscribe() }); producer.update({ value: { a: 1 } }); expect((await consumer.latest())?.value).toEqual({ a: 1 }); @@ -153,7 +157,7 @@ test("latest abandons a held group when a newer snapshot group rolls", async () test("latest stays inside the reader's range", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: false }); - const subscriber = track.subscribe({ delay: { max: REPLAY_LATENCY } }).withRange({ end: { excluded: 2n } }); + const subscriber = track.subscribe().withRange({ end: { excluded: 2n } }); const consumer = new Consumer({ track: subscriber }); for (let n = 0; n < 4; n++) producer.update({ value: { a: n } }); @@ -162,11 +166,11 @@ test("latest stays inside the reader's range", async () => { producer.finish(); }); -// `next()` finishes the held group before the newer one, so a playhead sees every state. +// `next()` reads the held group's buffered frames before the newer one. test("next reads a held group out before a newer snapshot group", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); - const consumer = new Consumer({ track: track.subscribe({ delay: { max: REPLAY_LATENCY } }) }); + const consumer = new Consumer({ track: track.subscribe() }); producer.update({ value: { a: 1 } }); expect((await consumer.next())?.value).toEqual({ a: 1 }); @@ -181,18 +185,46 @@ test("next reads a held group out before a newer snapshot group", async () => { expect(await consumer.next()).toBeUndefined(); }); -// Reading in order is still bounded: a group the subscription's max delay proves stale is skipped, -// as on any ordered track, so a reader that falls behind jumps ahead instead of replaying it. -test("next skips the groups the max delay abandons", async () => { +// A snapshot is superseded the moment a newer one exists, so the consumer reads at a zero budget +// whatever the subscription asked for: a reader that falls behind jumps to the newest group. +test("next skips every superseded group, whatever the subscription's max delay", async () => { const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI }); const producer = new Producer({ track, deltas: false }); for (let n = 1; n <= 3; n++) producer.update({ value: { n }, at: Time.Timestamp.fromMillis(n * 1_000) }); producer.finish(); - expect(await drain(track.subscribe({ delay: { max: REPLAY_LATENCY } }))).toEqual([{ n: 1 }, { n: 2 }, { n: 3 }]); - expect(await drain(track.subscribe({ delay: { max: Milli(500) } }))).toEqual([{ n: 2 }, { n: 3 }]); - // The default of zero keeps only the newest group. - expect(await drain(track.subscribe())).toEqual([{ n: 3 }]); + for (const max of [Milli.zero, Milli(500), REPLAY_LATENCY, null]) { + const subscriber = track.subscribe({ delay: { max } }); + expect(await drain(subscriber)).toEqual([{ n: 3 }]); + expect(subscriber.subscription.peek()?.delay?.max).toBe(Milli.zero); + } +}); + +// A session-fed track owes the groups between its arrivals. A snapshot reader never waits on one: +// group 2 arriving while group 1 is still owed supersedes it at once. +test("latest skips a group the subscription still owes", async () => { + for (const max of [Milli.zero, Milli(1_000), null]) { + const track = new Track.Producer("test").accept({}); + hooks.receiveTrack(track); + const consumer = new Consumer({ track: track.subscribe({ delay: { max } }) }); + const put = (sequence: bigint) => { + const group = track.createGroup(sequence); + group.writeFrame({ payload: new TextEncoder().encode(`{"n":${sequence}}`) }); + group.finish(); + }; + + put(0n); + expect((await consumer.latest())?.value).toEqual({ n: 0 }); + put(2n); + expect((await consumer.latest())?.value).toEqual({ n: 2 }); + + // The owed group landing late is superseded, not delivered. + put(1n); + const next = consumer.next(); + track.finish(); + expect(await next).toBeUndefined(); + track.close(); + } }); test("live consumer sees each update", async () => { @@ -424,14 +456,7 @@ test("a refresh after a rejected update restates the last written value", async joiner.close(); expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([4, 2]); - expect(await drain(track.subscribe({ delay: { max: REPLAY_LATENCY } }))).toEqual([ - { n: 0 }, - { n: 1 }, - { n: 2 }, - { n: 3 }, - { n: 3 }, - { n: 4 }, - ]); + expect(await drain(track.subscribe())).toEqual([{ n: 3 }, { n: 4 }]); }); test("no ratio never refreshes", async () => { @@ -548,8 +573,7 @@ test("a delta that would overflow the snapshot rolls a new one instead", async ( expect(await subscriber.nextGroup()).toBeUndefined(); // The newest value is readable with no earlier frame to apply it to. - const values = await drain(track.subscribe({ delay: { max: REPLAY_LATENCY } })); - expect(values[values.length - 1]).toEqual({ v: "y".repeat(size) }); + expect(await drain(track.subscribe())).toEqual([{ v: "y".repeat(size) }]); }); test("a delta that would push the snapshot out of the group's tail rolls a new group", () => { @@ -647,8 +671,7 @@ test("each state reads back with its frame's timestamp", async () => { } producer.finish(); - const subscription = { delay: { max: REPLAY_LATENCY } }; - const consumer = new Consumer({ track: track.subscribe(subscription) }); + const consumer = new Consumer({ track: track.subscribe() }); const states: [unknown, number | undefined][] = []; for await (const { value, at } of consumer) states.push([value.n, at?.as(Time.Timescale.MILLI)]); expect(states).toEqual([ @@ -657,7 +680,7 @@ test("each state reads back with its frame's timestamp", async () => { [3, 3_000], ]); - const latest = new Consumer({ track: track.subscribe(subscription) }); + const latest = new Consumer({ track: track.subscribe() }); const head = await latest.latest(); expect(head?.value.n).toBe(3); expect(head?.at?.as(Time.Timescale.MILLI)).toBe(3_000); diff --git a/rs/moq-json/MIGRATE.md b/rs/moq-json/MIGRATE.md index fd6250edbb9..6bec1be38a9 100644 --- a/rs/moq-json/MIGRATE.md +++ b/rs/moq-json/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay + +`snapshot::Consumer::new` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. +`next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. +A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `snapshot::Decoder`, or publish it as a `window` or `stream` track when every state must survive. + ### Compression moves to the track `Compression`, every `Config::compression` field, the `with_compression` setters, `Error::Flate`, `Error::Desync`, and `stream::Decoder::reset` are gone, along with the moq-flate dependency. diff --git a/rs/moq-json/src/lib.rs b/rs/moq-json/src/lib.rs index 8839c132c44..64dcbd4de26 100644 --- a/rs/moq-json/src/lib.rs +++ b/rs/moq-json/src/lib.rs @@ -1,7 +1,7 @@ //! JSON publishing over [`moq-net`](moq_net) tracks, in three modes: //! -//! - [`snapshot`]: **lossy**. One JSON value updated over time; `next` yields states inside the -//! drift budget, while `latest` skips to the newest state. Superseded groups may be lost. +//! - [`snapshot`]: **lossy**. One JSON value updated over time; `next` yields states in order, +//! while `latest` skips to the newest state. Superseded groups are lost. //! - [`stream`]: **lossless**. An ordered append-log of self-contained records; every record is //! preserved and delivered in order, nothing is ever superseded. //! - [`window`]: **bounded**. An ordered run of records appended to the back and dropped from the diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index a59fc8deb74..7ff6662f1d7 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -29,6 +29,10 @@ pub struct Config { /// yields each reconstructed state and its frame timestamp, or skips to the newest with `latest`. /// When something else already owns the track, use the [`Decoder`] /// directly. +/// +/// A snapshot is superseded the moment a newer one exists, so the consumer always reads at a +/// zero [`max`](moq_net::track::Delay::max) delay: a group is skipped once a newer one arrives, +/// and a lower group the subscription still owes never holds up a newer one. pub struct Consumer { track: moq_net::track::Ordered, group: Option, @@ -39,10 +43,19 @@ pub struct Consumer { impl Consumer { /// Create a consumer reading from the given track subscriber. /// + /// Sets the subscription's [`max`](moq_net::track::Delay::max) delay to zero, whatever the + /// caller asked for; its other preferences, such as ranges and priority, stay. /// A [`compress`](moq_net::track::Info::compress)ed track reads the same: moq-net inflates it. pub fn new(track: moq_net::track::Subscriber, config: Config) -> Self { + let mut track = track.ordered(); + let mut subscription = track.subscription(); + if subscription.delay.max != Some(std::time::Duration::ZERO) { + subscription.delay.max = Some(std::time::Duration::ZERO); + // Only an ended track refuses, and then there is nothing left to bound. + let _ = track.update(subscription); + } Self { - track: track.ordered(), + track, group: None, decoder: Decoder::new(config), frames_read: 0, @@ -57,18 +70,15 @@ impl Consumer { kio::wait(|waiter| self.poll_next(waiter)).await } - /// Poll for each state in order, retaining the subscription's drift budget. + /// Poll for each state in order, skipping every group a newer one has superseded. /// - /// Finish the current group before taking the next. Lost groups resynchronize from a - /// later snapshot. A timed playhead reader sets its subscription's `max_delay` to its - /// distance behind live; the default zero budget keeps only groups not yet superseded. + /// Finish the current group's buffered frames before taking the next. Lost groups + /// resynchronize from a later snapshot. pub fn poll_next(&mut self, waiter: &kio::Waiter) -> Poll>>> { self.poll_read(waiter, false) } /// Skip to the newest buffered state and its timestamp, or wait for a new one. - /// - /// See [`Self::poll_latest`] for how a nonzero `max_delay` delays it. pub async fn latest(&mut self) -> Result>> where T: Unpin, @@ -80,11 +90,6 @@ impl Consumer { /// /// Frames inside the newest group are still decoded in order. What this skips is gone; /// a subsequent `next` resumes after it. An untimed frame keeps `at: None`. - /// - /// It reads groups in order, so on a received track with a nonzero `max_delay` it first - /// waits for a lower group the subscription still owes, up to that budget, and with no - /// limit until the gap resolves, even when a newer group is buffered. A reader that only - /// wants the newest state keeps the default zero budget. pub fn poll_latest(&mut self, waiter: &kio::Waiter) -> Poll>>> { self.poll_read(waiter, true) } @@ -185,10 +190,7 @@ mod timestamps { .produce() .create_track("data", info) .unwrap(); - let mut subscriber = track.subscribe(None); - let mut subscription = subscriber.subscription(); - subscription.delay.max = None; - subscriber.update(subscription).unwrap(); + let subscriber = track.subscribe(None); let mut producer = crate::snapshot::Producer::new(track, crate::snapshot::Config::default()); let mut consumer = Consumer::::new(subscriber, Default::default()); let mut newest = Consumer::::new(producer.consume(), Default::default()); @@ -204,9 +206,6 @@ mod timestamps { if n == 1 { producer.cut().unwrap(); } - } - producer.finish().unwrap(); - for (n, ticks) in times.into_iter().enumerate() { let Poll::Ready(Ok(Some(value))) = consumer.poll_next(&kio::Waiter::noop()) else { panic!("missing state {n}") }; @@ -215,6 +214,7 @@ mod timestamps { assert_eq!(at.value(), ticks); assert_eq!(at.scale(), scale); } + producer.finish().unwrap(); assert!(matches!( consumer.poll_next(&kio::Waiter::noop()), Poll::Ready(Ok(None)) diff --git a/rs/moq-json/src/snapshot/mod.rs b/rs/moq-json/src/snapshot/mod.rs index 625d2dfc96b..5aecfa6da1d 100644 --- a/rs/moq-json/src/snapshot/mod.rs +++ b/rs/moq-json/src/snapshot/mod.rs @@ -1,9 +1,9 @@ //! Lossy latest-value JSON publishing over [`moq-net`](moq_net) tracks. //! //! One value updated over time. `Consumer::next` yields each state with its frame timestamp, -//! in order within the subscription's drift budget; `Consumer::latest` skips to the newest -//! state. Superseded groups may be lost, so use [`stream`](crate::stream) when every record -//! must survive. A playhead reader sets `max_delay` to its distance behind the live edge. +//! in order, skipping every group a newer one supersedes; `Consumer::latest` skips to the +//! newest state. Superseded groups are lost, so use [`stream`](crate::stream) or +//! [`window`](crate::window) when every record must survive. //! //! On the wire the value is published as a series of groups, where each group is //! self-contained: its first frame is a full snapshot and any following frames are @@ -225,10 +225,7 @@ mod test { #[test] fn next_preserves_intermediate_states() { - let (mut producer, mut track) = producer(cfg(true)); - let mut subscription = track.subscription(); - subscription.delay.max = None; - track.update(subscription).unwrap(); + let (mut producer, track) = producer(cfg(true)); for n in [1, 2, 3] { producer .update(&json!({ "padding": "same-long-value-so-deltas-stay-in-one-group", "n": n })) diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index caee3b03c60..735a36788c8 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -384,8 +384,7 @@ impl Consumer { /// The mode the publisher chose. /// - /// [`Mode::Snapshot`] is lossy: groups may be skipped once superseded beyond the subscription - /// drift budget. [`Mode::Stream`] preserves every record a reader keeps up with; one that falls + /// [`Mode::Snapshot`] is lossy: a group is skipped once a newer one supersedes it. [`Mode::Stream`] preserves every record a reader keeps up with; one that falls /// behind a group's retained tail sees `Evicted`. Check this when the application can only /// work with one of them. pub fn mode(&self) -> &Mode { diff --git a/rs/moq-net-tests/tests/integration/main.rs b/rs/moq-net-tests/tests/integration/main.rs index c8a31acfb55..0c89f7d4d85 100644 --- a/rs/moq-net-tests/tests/integration/main.rs +++ b/rs/moq-net-tests/tests/integration/main.rs @@ -1,8 +1,9 @@ -//! Session tests that need JSON consumers or the hidden peer-claim hooks. +//! Tests that need JSON consumers or moq-net's hidden fuzz hooks. mod ietf_conformance; mod refresh; mod restart; +mod snapshot; #[path = "../../../moq-net/tests/integration/support/mod.rs"] mod support; diff --git a/rs/moq-net-tests/tests/integration/snapshot.rs b/rs/moq-net-tests/tests/integration/snapshot.rs new file mode 100644 index 00000000000..05e69a40270 --- /dev/null +++ b/rs/moq-net-tests/tests/integration/snapshot.rs @@ -0,0 +1,44 @@ +//! A snapshot reader on a received track never waits on a hole the subscription still owes. +use moq_json::snapshot; +use moq_net::{fuzz, group, kio, track}; +use serde_json::{Value, json}; +use std::{task::Poll, time::Duration}; + +/// Group 2 arrives while group 1 is still owed: `latest()` returns it at once, whatever +/// budget the caller subscribed with, since a snapshot consumer always reads at zero. +#[test] +fn latest_skips_an_owed_hole() { + for max in [Some(Duration::ZERO), Some(Duration::from_secs(1)), None] { + let track = moq_net::broadcast::Info::new() + .produce() + .create_track("state", track::Info::default()) + .unwrap(); + fuzz::received(&track); + let subscription = track::Subscription::default().with_delay(track::Delay::default().with_max(max)); + let mut consumer = snapshot::Consumer::::new(track.subscribe(subscription), Default::default()); + let waiter = kio::Waiter::noop(); + let put = |sequence: u64| { + let mut group = track.create_group(group::Info { sequence }).unwrap(); + group + .write_frame(None, serde_json::to_vec(&json!({ "n": sequence })).unwrap()) + .unwrap(); + group.finish().unwrap(); + }; + + put(0); + let Poll::Ready(Ok(Some(state))) = consumer.poll_latest(&waiter) else { + panic!("missing group 0 with max {max:?}") + }; + assert_eq!(state.value, json!({ "n": 0 })); + + put(2); + let Poll::Ready(Ok(Some(state))) = consumer.poll_latest(&waiter) else { + panic!("group 2 waited on owed group 1 with max {max:?}") + }; + assert_eq!(state.value, json!({ "n": 2 })); + + // The owed group landing late is superseded, not delivered. + put(1); + assert!(consumer.poll_next(&waiter).is_pending(), "max {max:?}"); + } +} From 0d34abf44db015fba3832a54156452edc91c150c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 12:59:04 -0800 Subject: [PATCH 04/24] quest: snapshot consumers always read at zero delay Decided 2026-11-03 (maintainer, /quest-unblock): a snapshot is always max-delay 0; no new Ordered read. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 2 +- quest/m1/ordered-owed-holes.md | 94 +++++++++++----------------------- 2 files changed, 30 insertions(+), 66 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 2a4ca041d2c..0cdd4e1cd7a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -188,7 +188,7 @@ per-frame CPU growth since #5926. - [A live keyframe interval change never outruns the declared HLS duration](/quest/m1/js-hls-keyframe-duration.md) - raising a live keyframe interval, even to infinite, never pushes a segment past the declared HLS target duration, and the browser interop cell asserts the catalog `hls` field - [One Ed25519 public key type with one z-base-32 codec](/quest/m1/mesh-key-codec.md) - `moq_auth::mesh::EndpointId` is merged into `moq_sign::PublicKey` - [moqsink reports track demand only once the track is in the catalog](/quest/m1/moqsink-demand-catalog.md) - A `moqsink` request pad's `track-demand` stays null until its track is listed in the published catalog, and only then reports `wanted` -- [Ordered reads can skip owed holes, and snapshot latest() does](/quest/m1/ordered-owed-holes.md) - moq-net's `Ordered` gains a read that skips past an owed hole to the newest buffered group, while keeping the reader's bounds +- [Snapshot consumers always read at zero delay](/quest/m1/ordered-owed-holes.md) - Rust and JS snapshot consumers force a zero `delay.max`, so `latest()` never waits on an owed hole - [A Rust relay never replays from group 0 to a lite-03/04 subscriber](/quest/m1/relay-inflight-replay.md) - A Rust relay or publisher serving a lite-03 or lite-04 SUBSCRIBE from a track copy that is still resolving starts at the newest group once the copy resolves - [The resume model passes its long run again](/quest/m1/resume-model-7542.md) - fix #7542's regression that drops groups a long-budget reader can still get - [Stats name the route instance's epoch with a type](/quest/m1/stats-epoch-type.md) - `GroupFrame::epoch` gets a dedicated type and tests stop mislabelling the instance diff --git a/quest/m1/ordered-owed-holes.md b/quest/m1/ordered-owed-holes.md index 762cb90a37c..e39e6979b8f 100644 --- a/quest/m1/ordered-owed-holes.md +++ b/quest/m1/ordered-owed-holes.md @@ -1,71 +1,35 @@ -# [M] Ordered reads can skip owed holes, and snapshot latest() does +# [S] Snapshot consumers always read at zero delay ## Goal -moq-net's `Ordered` gains a read that skips past an owed hole to the newest -buffered group, while keeping the reader's bounds (range, budget, delivered -set). `moq_json::snapshot::Consumer::latest()` moves onto it, so on a -received track with a nonzero or unlimited `max_delay` it returns the newest -buffered snapshot at once instead of waiting for an owed lower group. The -`poll_latest` caveat goes. `@moq/net`'s `Ordered` and `@moq/json`'s -`latest()` do the same. - -## Waiting - -- Maintainer: choose the new public Ordered read's shape and name before - implementation. Recommend Rust `poll_latest_group` with its async - `latest_group`, mirrored by JS `tryLatestGroup` and `latestGroup`: select - the highest eligible buffered sequence across all resume copies in one - read, pass lower owed holes permanently, and retain the same range, - delay budget, returned-group guard, and delivered watermark as ordinary - ordered reads. With nothing eligible, wait for a new candidate or end; - `next_group` / `nextGroup` keep waiting on owed holes. Alternative: - `poll_next_available_group` / `next_available_group` and - `tryNextAvailableGroup` / `nextAvailableGroup` yield the lowest eligible - group without waiting on holes, leaving snapshot consumers to drain - them to reach the newest. The direct newest read better expresses the - goal and avoids handing out every superseded group. Record the choice - here and remove this wait before implementing. +`moq_json::snapshot::Consumer` and `@moq/json`'s snapshot consumer always +subscribe and read at a zero `delay.max`, with no way for a caller to override +it. At budget 0 an owed hole is stale as soon as a newer group arrives, so +`latest()` on a received track returns the newest buffered snapshot at once +instead of waiting for an owed lower group. The `poll_latest` caveat goes. +No new `Ordered` read. ## Plan -Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 -/quest-complete run), from -[moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403). - -Milestone: ✅ m1 (moved from m2). Not chosen: m2. - -- [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403) decision 7, taken overnight on the maintainer's standing - authorization (2026-10-31): ✅ land with the documented caveat (the - `poll_latest` rustdoc, commit - [1aad05c565](https://github.com/moq-dev/moq/commit/1aad05c565)) and plan - this follow-up. Not chosen: fix it in [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403), threading new public API - through `Cursor::poll_next_with`, `Subscriber::poll_group`, and the resume - layer's multi-copy peek (`rs/moq-net/src/model/resume.rs`). -- Today `latest()` drains `Ordered::poll_next_group`, which since [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403) parks - above an owed hole until it lands, the track says it won't, or the budget - gives up. Every in-repo snapshot reader uses the zero default and is - unaffected; a reader with a budget is not. -- The new read is that threading: skip owed holes but not the reader's range - or budget, and never hand out a group twice. The caveat sits at - `rs/moq-json/src/snapshot/consumer.rs:84` at be39188000. -- [moq-dev/moq#7609](https://github.com/moq-dev/moq/pull/7609) (merged) made - JS ordered reads wait on owed groups too, so `@moq/json`'s `latest()` - (`js/json/src/snapshot/consumer.ts:59`, draining `Ordered.tryNextGroup`) - now parks the same way. Mirror the new read in `@moq/net`'s `Ordered` - (`js/net/src/track.ts:3111`) and move `latest()` onto it. - -Public API: adds an `Ordered` read (Rust and JS); `snapshot::Consumer::latest()` -and `@moq/json`'s `latest()` stop waiting at owed holes. Wire: none. - -Reproduced 2026-11-03 on the base of -[moq-dev/moq#7736](https://github.com/moq-dev/moq/pull/7736): after reading -snapshot group 0, buffer group 2 while the received track still owes group -1. JS snapshot `latest()` remains pending with `delay.max` of 1,000 ms or -`null`; advancing a manual clock one hour does not release the unlimited -read. Delivering group 1 releases it with state 1, despite group 2 already -being buffered. A two-case scratch probe passed with all pending reads -settled during cleanup. Rust's existing -`next_group_waits_on_an_owed_group_within_the_budget` test also passed, -confirming finite and unlimited owed-hole waits in the Ordered read that -the Rust snapshot consumer drains. No production change is implemented. +Decided 2026-11-03 (maintainer, /quest-unblock): ✅ a snapshot is always +max-delay 0; no new `Ordered` read. It replaces the 2026-10-31 plan of an +`Ordered` read that skips owed holes. Not chosen: `poll_latest_group` / +`latest_group`; `next_available_group`; forcing 0 and also adding the read. + +- Both consumers set their subscription's `delay.max` to zero at + construction, keeping the caller's other preferences. Neither config has a + delay setting to remove. Add migration notes: a caller that set a budget to + replay history or trail a playhead now reads only the newest group. +- Delete the `poll_latest` caveat (`rs/moq-json/src/snapshot/consumer.rs`) + and the budget guidance in both consumers' docs. +- Rust and JS tests: a snapshot reader sees group 2 while group 1 is still + owed, and `latest()` returns group 2 at once, whatever budget the caller + subscribed with. +- `Ordered` at budget 0 already passes an owed hole in both languages + (`next_group_waits_on_an_owed_group_within_the_budget` in + `rs/moq-net/src/model/track.rs`, and the zero case of + `ordered owed holes follow zero, narrowed, unlimited, and publisher budgets` + in `js/net/src/track.test.ts`); fix `Ordered` only if that stops holding. + +Public API: snapshot consumers ignore the subscription's `delay.max` (Rust and +JS). Wire: none. From 1f8b2ee7619d15d9d22a8a2951a88a7955338f9a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 13:04:22 -0800 Subject: [PATCH 05/24] feat(json)!: snapshot consumers always read at zero delay Rust and JS snapshot consumers set their subscription's delay.max to zero, so latest() never waits on a group a received track still owes. Bindings docs and migration notes follow; the quest is finished. Co-Authored-By: Claude Opus 5.5 --- go/wrapper/MIGRATE.md | 4 +++ go/wrapper/json/json.go | 5 ++- js/json/MIGRATE.md | 2 +- js/json/tsconfig.build.json | 3 ++ js/json/tsconfig.json | 4 ++- kt/moq/MIGRATE.md | 4 +++ .../kotlin/dev/moq/json/Tracks.kt | 4 +-- py/moq-rs/MIGRATE.md | 4 +++ py/moq-rs/moq/json.py | 6 ++-- quest/m1/README.md | 1 - quest/m1/ordered-owed-holes.md | 35 ------------------- rs/moq-ffi/src/json.rs | 3 +- rs/moq-json/MIGRATE.md | 2 +- swift/MIGRATE.md | 4 +++ swift/Sources/Moq/Json.swift | 5 +-- 15 files changed, 35 insertions(+), 51 deletions(-) delete mode 100644 quest/m1/ordered-owed-holes.md diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index ab7c8e208af..3e6f594f151 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`json.SnapshotConsumer` reads at a zero `Delay.Max`, whatever the subscription asked for: `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. + ### Subscriptions take a `moq.Delay` ([#7697](https://github.com/moq-dev/moq/pull/7697)) `Subscription.MaxDelay` is `Delay moq.Delay`, and the zero `Subscription{}` is moq-net's defaults, the same as `DefaultSubscription()`: set `Delay.Max` to the old `*MaxDelay`, and `Delay.MaxUnlimited` where `MaxDelay` was nil. diff --git a/go/wrapper/json/json.go b/go/wrapper/json/json.go index 79c9d3977b1..e87e2e08d5e 100644 --- a/go/wrapper/json/json.go +++ b/go/wrapper/json/json.go @@ -160,9 +160,8 @@ func NewSnapshotConsumer(track *moq.TrackConsumer) (*SnapshotConsumer, error) { return &SnapshotConsumer{inner: inner}, nil } -// Next returns the next state in order, or nil when the track ends. Groups older -// than the subscription's Delay.Max are skipped, so a playhead reader sets that -// budget to its distance behind live. +// Next returns the next state in order, or nil when the track ends. It skips +// every group a newer one supersedes, whatever the subscription's Delay.Max. func (c *SnapshotConsumer) Next(ctx context.Context) (*Timed, error) { value, err := bridge.Call(ctx, c.inner.Cancel, c.inner.Next) return timed(value), err diff --git a/js/json/MIGRATE.md b/js/json/MIGRATE.md index e9ed0db8c5b..f282ed8de5f 100644 --- a/js/json/MIGRATE.md +++ b/js/json/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) `Snapshot.Consumer` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. `next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. diff --git a/js/json/tsconfig.build.json b/js/json/tsconfig.build.json index b0d79fdc2cb..b58733f6162 100644 --- a/js/json/tsconfig.build.json +++ b/js/json/tsconfig.build.json @@ -3,5 +3,8 @@ // are neither published nor discovered by `bun test` at the package root. // `tsc --noEmit` still uses tsconfig.json, so tests stay type-checked. "extends": "./tsconfig.json", + "compilerOptions": { + "rootDir": "./src" + }, "exclude": ["src/**/*.test.ts", "src/**/*.test.tsx"] } diff --git a/js/json/tsconfig.json b/js/json/tsconfig.json index bb55d7c43fc..a1bf54c031e 100644 --- a/js/json/tsconfig.json +++ b/js/json/tsconfig.json @@ -2,7 +2,9 @@ "extends": "../tsconfig.json", "compilerOptions": { "outDir": "dist", - "rootDir": "./src", + // The snapshot test borrows `@moq/net`'s internal hooks by relative path, so the + // type-check spans the workspace. The build pins `rootDir` to src and skips tests. + "rootDir": "..", "types": ["bun"] }, "include": ["src"] diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 98257b6dad4..33d4f665cf5 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. + ### Subscriptions take a `Delay` ([#7697](https://github.com/moq-dev/moq/pull/7697)) `Subscription.maxDelay` is `delay: Delay`: pass `delay = Delay(min = Duration.ZERO, max = maxDelay)`, where a null `max` sets no subscriber limit, or `copy` `Subscription.default()`. diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt index 7db3dbee300..d8884b5b2c6 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt @@ -96,8 +96,8 @@ inline fun WindowEventPush.decode(): T = /** * Stream of each JSON state from a snapshot track, in order. * - * Groups older than the subscription's `delay.max` are skipped, so a playhead reader sets that - * budget to its distance behind live. Call [SnapshotConsumer.latest] to skip to the newest state. + * Skips every group a newer one supersedes, whatever the subscription's `delay.max`. Call + * [SnapshotConsumer.latest] to skip to the newest state. */ fun SnapshotConsumer.values(): Flow = pull({ cancel() }) { next() } diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index aad98da3ea3..c74365e342e 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`moq.json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. + ### Subscriptions take a `Delay` ([#7697](https://github.com/moq-dev/moq/pull/7697)) `Subscription.max_delay` is `delay`, a `moq.Delay`: pass `delay=moq.Delay(max=...)` for the old `max_delay=...`, and `moq.Delay(max=None)` for no subscriber limit. diff --git a/py/moq-rs/moq/json.py b/py/moq-rs/moq/json.py index 0eed010c2cb..40728494213 100644 --- a/py/moq-rs/moq/json.py +++ b/py/moq-rs/moq/json.py @@ -126,9 +126,9 @@ def finish(self) -> None: class SnapshotConsumer(_Stream[Timed]): """Async iterator over a JSON snapshot track, yielding each state in order (lossy). - Each item is a :class:`Timed`. Groups older than the subscription's ``delay.max`` are skipped, - so a playhead reader sets that budget to its distance behind live; :meth:`latest` skips to the - newest state instead. Usable as an async context manager that cancels on exit. + Each item is a :class:`Timed`. Every group a newer one supersedes is skipped, whatever the + subscription's ``delay.max``; :meth:`latest` skips to the newest state instead. Usable as an + async context manager that cancels on exit. """ def __init__(self, track: subscribe.TrackConsumer) -> None: diff --git a/quest/m1/README.md b/quest/m1/README.md index 0cdd4e1cd7a..bcda80c0b7c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -188,7 +188,6 @@ per-frame CPU growth since #5926. - [A live keyframe interval change never outruns the declared HLS duration](/quest/m1/js-hls-keyframe-duration.md) - raising a live keyframe interval, even to infinite, never pushes a segment past the declared HLS target duration, and the browser interop cell asserts the catalog `hls` field - [One Ed25519 public key type with one z-base-32 codec](/quest/m1/mesh-key-codec.md) - `moq_auth::mesh::EndpointId` is merged into `moq_sign::PublicKey` - [moqsink reports track demand only once the track is in the catalog](/quest/m1/moqsink-demand-catalog.md) - A `moqsink` request pad's `track-demand` stays null until its track is listed in the published catalog, and only then reports `wanted` -- [Snapshot consumers always read at zero delay](/quest/m1/ordered-owed-holes.md) - Rust and JS snapshot consumers force a zero `delay.max`, so `latest()` never waits on an owed hole - [A Rust relay never replays from group 0 to a lite-03/04 subscriber](/quest/m1/relay-inflight-replay.md) - A Rust relay or publisher serving a lite-03 or lite-04 SUBSCRIBE from a track copy that is still resolving starts at the newest group once the copy resolves - [The resume model passes its long run again](/quest/m1/resume-model-7542.md) - fix #7542's regression that drops groups a long-budget reader can still get - [Stats name the route instance's epoch with a type](/quest/m1/stats-epoch-type.md) - `GroupFrame::epoch` gets a dedicated type and tests stop mislabelling the instance diff --git a/quest/m1/ordered-owed-holes.md b/quest/m1/ordered-owed-holes.md deleted file mode 100644 index e39e6979b8f..00000000000 --- a/quest/m1/ordered-owed-holes.md +++ /dev/null @@ -1,35 +0,0 @@ -# [S] Snapshot consumers always read at zero delay - -## Goal - -`moq_json::snapshot::Consumer` and `@moq/json`'s snapshot consumer always -subscribe and read at a zero `delay.max`, with no way for a caller to override -it. At budget 0 an owed hole is stale as soon as a newer group arrives, so -`latest()` on a received track returns the newest buffered snapshot at once -instead of waiting for an owed lower group. The `poll_latest` caveat goes. -No new `Ordered` read. - -## Plan - -Decided 2026-11-03 (maintainer, /quest-unblock): ✅ a snapshot is always -max-delay 0; no new `Ordered` read. It replaces the 2026-10-31 plan of an -`Ordered` read that skips owed holes. Not chosen: `poll_latest_group` / -`latest_group`; `next_available_group`; forcing 0 and also adding the read. - -- Both consumers set their subscription's `delay.max` to zero at - construction, keeping the caller's other preferences. Neither config has a - delay setting to remove. Add migration notes: a caller that set a budget to - replay history or trail a playhead now reads only the newest group. -- Delete the `poll_latest` caveat (`rs/moq-json/src/snapshot/consumer.rs`) - and the budget guidance in both consumers' docs. -- Rust and JS tests: a snapshot reader sees group 2 while group 1 is still - owed, and `latest()` returns group 2 at once, whatever budget the caller - subscribed with. -- `Ordered` at budget 0 already passes an owed hole in both languages - (`next_group_waits_on_an_owed_group_within_the_budget` in - `rs/moq-net/src/model/track.rs`, and the zero case of - `ordered owed holes follow zero, narrowed, unlimited, and publisher budgets` - in `js/net/src/track.test.ts`); fix `Ordered` only if that stops holding. - -Public API: snapshot consumers ignore the subscription's `delay.max` (Rust and -JS). Wire: none. diff --git a/rs/moq-ffi/src/json.rs b/rs/moq-ffi/src/json.rs index 9ad9378c11c..f808a6ed8e7 100644 --- a/rs/moq-ffi/src/json.rs +++ b/rs/moq-ffi/src/json.rs @@ -246,8 +246,7 @@ impl MoqJsonSnapshotConsumer { /// Get the next state in order. Returns `None` once the track ends. /// - /// Groups older than the subscription's `delay.max` are skipped, so a playhead - /// reader sets that budget to its distance behind live. + /// Skips every group a newer one supersedes, whatever the subscription's `delay.max`. pub async fn next(&self) -> Result, MoqError> { let value = self .reads diff --git a/rs/moq-json/MIGRATE.md b/rs/moq-json/MIGRATE.md index 6bec1be38a9..f5669fdb36f 100644 --- a/rs/moq-json/MIGRATE.md +++ b/rs/moq-json/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) `snapshot::Consumer::new` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. `next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index cc7bd5e5c43..605234eecb7 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`Json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. + ### Subscriptions take a `Delay` ([#7697](https://github.com/moq-dev/moq/pull/7697)) `Subscription.maxDelay` is `delay: Delay`: pass `delay: Delay(min: 0, max: maxDelay)`, where a `nil` `max` sets no subscriber limit, or start from `Subscription.default`. diff --git a/swift/Sources/Moq/Json.swift b/swift/Sources/Moq/Json.swift index 431f1864696..fd3372809c9 100644 --- a/swift/Sources/Moq/Json.swift +++ b/swift/Sources/Moq/Json.swift @@ -77,8 +77,9 @@ extension Json { } /// Read side of a JSON snapshot track (lossy latest-value). Iterating yields each state in - /// order with its timestamp: `for try await state in json { ... }`. Groups older than the - /// subscription's `delay.max` are skipped; `latest()` skips to the newest state instead. + /// order with its timestamp: `for try await state in json { ... }`. Every group a newer one + /// supersedes is skipped, whatever the subscription's `delay.max`; `latest()` skips to the + /// newest state instead. public final class SnapshotConsumer: AsyncSequence, Sendable { /// The decoded snapshot state emitted by this sequence. public typealias Element = Timed From 7735d5274865dde0adfcd47c782b10a5139b3995 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 13:16:36 -0800 Subject: [PATCH 06/24] chore(dart): refresh the snapshot next() checksum Co-Authored-By: Claude Opus 5.5 --- dart/moq_ffi/lib/src/moq.dart | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index b704af4ce14..2509b02bc3e 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -21940,7 +21940,7 @@ void _checkApiChecksums() { 60932) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } - if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotconsumer_next() != 5806) { + if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotconsumer_next() != 12715) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotproducer_demand() != From fa1154bc2ce9a6b82f8b8284f71886c04a981c05 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 13:48:51 -0800 Subject: [PATCH 07/24] fix(json): hold snapshot consumers at zero delay across later updates A retained control, subscriber, or FFI track handle could restore a nonzero budget after construction. Rust re-pins on every poll; JS re-pins on every read and on each subscription change. Tests cover next() and a restored budget, and the docs stop calling window lossless. Co-Authored-By: Claude Opus 5.5 --- go/wrapper/MIGRATE.md | 2 +- js/json/MIGRATE.md | 2 +- js/json/src/snapshot/consumer.ts | 23 +++++++----- js/json/src/snapshot/snapshot.test.ts | 25 +++++++++---- kt/moq/MIGRATE.md | 2 +- py/moq-rs/MIGRATE.md | 2 +- rs/moq-json/MIGRATE.md | 2 +- rs/moq-json/src/snapshot/consumer.rs | 30 ++++++++++------ rs/moq-json/src/snapshot/mod.rs | 4 +-- .../tests/integration/snapshot.rs | 36 ++++++++++++++----- swift/MIGRATE.md | 2 +- 11 files changed, 88 insertions(+), 42 deletions(-) diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index eb7cd9121f0..65fd5b86dc2 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -6,7 +6,7 @@ How to update your code for each breaking change, newest first. ### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`json.SnapshotConsumer` reads at a zero `Delay.Max`, whatever the subscription asked for: `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. +`json.SnapshotConsumer` reads at a zero `Delay.Max`, whatever the subscription asked for: `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### JSON writes take a broadcast-clock timestamp ([#7691](https://github.com/moq-dev/moq/pull/7691)) diff --git a/js/json/MIGRATE.md b/js/json/MIGRATE.md index f282ed8de5f..ad5b68aaf63 100644 --- a/js/json/MIGRATE.md +++ b/js/json/MIGRATE.md @@ -8,7 +8,7 @@ How to update your code for each breaking change, newest first. `Snapshot.Consumer` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. `next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. -A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `Snapshot.Decoder`, or publish it as a `Window` or `Stream` track when every state must survive. +A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `Snapshot.Decoder`, or publish a `Stream` track when every state must survive, or a `Window` track for a bounded recent history. ### Compression moves to the track diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 73a2d06f087..a0bff89c39b 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -22,8 +22,8 @@ const GAPS: Moq.StreamCode[] = [ * When something else already owns the track, use the {@link Decoder} directly. * * A snapshot is superseded the moment a newer one exists, so the consumer always reads at a zero - * `delay.max`, whatever the subscription asked for: a group is skipped once a newer one arrives, - * and a lower group the subscription still owes never holds up a newer one. + * `delay.max`, whatever the subscription asks for, then or later: a group is skipped once a newer + * one arrives, and a lower group the subscription still owes never holds up a newer one. */ export class Consumer { #track: Moq.Track.Ordered; @@ -33,16 +33,22 @@ export class Consumer { #framesRead = 0; constructor(config: Consumer.Config) { - // The default subscription's budget is zero. - const zero = Moq.Track.Subscription.default().delay?.max; - const current = config.track.subscription.peek(); - if (current?.delay?.max !== zero) { - config.track.update({ ...current, delay: { ...current?.delay, max: zero } }); - } this.#track = config.track.ordered(); + this.#pin(); + // A retained handle can update the subscription while a read waits; put the zero back at + // once, which also wakes a read that update left holding at an owed group. Writes coalesce + // per microtask, so an update undone in the same one never notifies: #read pins as well. + this.#track.subscription.subscribe(() => this.#pin()); this.#decoder = new Decoder(config); } + // Hold the subscription's max delay at zero, the default subscription's, keeping the rest of it. + #pin(): void { + const zero = Moq.Track.Subscription.default().delay?.max; + const current = this.#track.subscription.peek(); + if (current?.delay?.max !== zero) this.#track.update({ ...current, delay: { ...current?.delay, max: zero } }); + } + /** * Get the next state in order, or `undefined` once the track ends. * @@ -69,6 +75,7 @@ export class Consumer { async #read(skip: boolean): Promise | undefined> { for (;;) { + this.#pin(); if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index a3449d8c0d9..6d1c52c79cc 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -201,12 +201,18 @@ test("next skips every superseded group, whatever the subscription's max delay", }); // A session-fed track owes the groups between its arrivals. A snapshot reader never waits on one: -// group 2 arriving while group 1 is still owed supersedes it at once. -test("latest skips a group the subscription still owes", async () => { +// group 2 arriving while group 1 is still owed supersedes it at once, whatever budget the caller +// subscribed with or later restores through the subscriber it kept. +test("reads skip a group the subscription still owes", async () => { for (const max of [Milli.zero, Milli(1_000), null]) { const track = new Track.Producer("test").accept({}); hooks.receiveTrack(track); - const consumer = new Consumer({ track: track.subscribe({ delay: { max } }) }); + const subscribers = [track.subscribe({ delay: { max } }), track.subscribe({ delay: { max } })]; + const [latest, next] = subscribers.map((subscriber) => new Consumer({ track: subscriber })); + const restore = () => { + for (const subscriber of subscribers) subscriber.update({ delay: { max } }); + }; + restore(); const put = (sequence: bigint) => { const group = track.createGroup(sequence); group.writeFrame({ payload: new TextEncoder().encode(`{"n":${sequence}}`) }); @@ -214,15 +220,20 @@ test("latest skips a group the subscription still owes", async () => { }; put(0n); - expect((await consumer.latest())?.value).toEqual({ n: 0 }); + expect((await latest.latest())?.value).toEqual({ n: 0 }); + expect((await next.next())?.value).toEqual({ n: 0 }); + // Restored again while both reads wait, so the pin must come from the update itself. + const reads = [latest.latest(), next.next()]; + await new Promise((resolve) => setTimeout(resolve, 0)); + restore(); put(2n); - expect((await consumer.latest())?.value).toEqual({ n: 2 }); + expect((await Promise.all(reads)).map((state) => state?.value)).toEqual([{ n: 2 }, { n: 2 }]); // The owed group landing late is superseded, not delivered. put(1n); - const next = consumer.next(); + const ends = [latest.latest(), next.next()]; track.finish(); - expect(await next).toBeUndefined(); + expect(await Promise.all(ends)).toEqual([undefined, undefined]); track.close(); } }); diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 89303e05593..fb5bb4e2d18 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -6,7 +6,7 @@ How to update your code for each breaking change, newest first. ### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. +`SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### JSON writes take a broadcast-clock timestamp ([#7691](https://github.com/moq-dev/moq/pull/7691)) diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 0ec1f33239e..6f07ffb39e9 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -6,7 +6,7 @@ How to update your code for each breaking change, newest first. ### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`moq.json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. +`moq.json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### JSON writes take a broadcast-clock timestamp ([#7691](https://github.com/moq-dev/moq/pull/7691)) diff --git a/rs/moq-json/MIGRATE.md b/rs/moq-json/MIGRATE.md index f5669fdb36f..2c85bec9226 100644 --- a/rs/moq-json/MIGRATE.md +++ b/rs/moq-json/MIGRATE.md @@ -8,7 +8,7 @@ How to update your code for each breaking change, newest first. `snapshot::Consumer::new` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. `next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. -A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `snapshot::Decoder`, or publish it as a `window` or `stream` track when every state must survive. +A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `snapshot::Decoder`, or publish a `stream` track when every state must survive, or a `window` track for a bounded recent history. ### Compression moves to the track diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index 7ff6662f1d7..83f415bfdbf 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -44,21 +44,28 @@ impl Consumer { /// Create a consumer reading from the given track subscriber. /// /// Sets the subscription's [`max`](moq_net::track::Delay::max) delay to zero, whatever the - /// caller asked for; its other preferences, such as ranges and priority, stay. + /// caller asked for, and again on every read if a retained handle changed it; its other + /// preferences, such as ranges and priority, stay. /// A [`compress`](moq_net::track::Info::compress)ed track reads the same: moq-net inflates it. pub fn new(track: moq_net::track::Subscriber, config: Config) -> Self { - let mut track = track.ordered(); - let mut subscription = track.subscription(); - if subscription.delay.max != Some(std::time::Duration::ZERO) { - subscription.delay.max = Some(std::time::Duration::ZERO); - // Only an ended track refuses, and then there is nothing left to bound. - let _ = track.update(subscription); - } - Self { - track, + let mut consumer = Self { + track: track.ordered(), group: None, decoder: Decoder::new(config), frames_read: 0, + }; + consumer.pin_delay(); + consumer + } + + /// Hold the subscription's max delay at zero, keeping the rest of it. + fn pin_delay(&mut self) { + let mut subscription = self.track.subscription(); + if subscription.delay.max != Some(std::time::Duration::ZERO) { + subscription.delay.max = Some(std::time::Duration::ZERO); + // Refused only when the track has ended or its ranges name time on an untimed + // track; neither is this reader's to fix. + let _ = self.track.update(subscription); } } @@ -95,6 +102,9 @@ impl Consumer { } fn poll_read(&mut self, waiter: &kio::Waiter, skip: bool) -> Poll>>> { + // Every wait ends in another poll, so pinning here undoes a retained handle's update + // before a read could hold at an owed group under it. + self.pin_delay(); if skip { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share diff --git a/rs/moq-json/src/snapshot/mod.rs b/rs/moq-json/src/snapshot/mod.rs index 5aecfa6da1d..e761d09ffd3 100644 --- a/rs/moq-json/src/snapshot/mod.rs +++ b/rs/moq-json/src/snapshot/mod.rs @@ -2,8 +2,8 @@ //! //! One value updated over time. `Consumer::next` yields each state with its frame timestamp, //! in order, skipping every group a newer one supersedes; `Consumer::latest` skips to the -//! newest state. Superseded groups are lost, so use [`stream`](crate::stream) or -//! [`window`](crate::window) when every record must survive. +//! newest state. Superseded groups are lost, so use [`stream`](crate::stream) when every +//! record must survive, or [`window`](crate::window) for a bounded recent history. //! //! On the wire the value is published as a series of groups, where each group is //! self-contained: its first frame is a full snapshot and any following frames are diff --git a/rs/moq-net-tests/tests/integration/snapshot.rs b/rs/moq-net-tests/tests/integration/snapshot.rs index 05e69a40270..91c238be7f7 100644 --- a/rs/moq-net-tests/tests/integration/snapshot.rs +++ b/rs/moq-net-tests/tests/integration/snapshot.rs @@ -4,10 +4,11 @@ use moq_net::{fuzz, group, kio, track}; use serde_json::{Value, json}; use std::{task::Poll, time::Duration}; -/// Group 2 arrives while group 1 is still owed: `latest()` returns it at once, whatever -/// budget the caller subscribed with, since a snapshot consumer always reads at zero. +/// Group 2 arrives while group 1 is still owed: `latest()` and `next()` return it at once, +/// whatever budget the caller subscribed with or later restores through a retained handle, +/// since a snapshot consumer always reads at zero. #[test] -fn latest_skips_an_owed_hole() { +fn reads_skip_an_owed_hole() { for max in [Some(Duration::ZERO), Some(Duration::from_secs(1)), None] { let track = moq_net::broadcast::Info::new() .produce() @@ -15,7 +16,15 @@ fn latest_skips_an_owed_hole() { .unwrap(); fuzz::received(&track); let subscription = track::Subscription::default().with_delay(track::Delay::default().with_max(max)); - let mut consumer = snapshot::Consumer::::new(track.subscribe(subscription), Default::default()); + let reader = || { + let subscriber = track.subscribe(subscription.clone()); + let control = subscriber.control(); + let consumer = snapshot::Consumer::::new(subscriber, Default::default()); + control.update(subscription.clone()).unwrap(); + consumer + }; + let mut latest = reader(); + let mut next = reader(); let waiter = kio::Waiter::noop(); let put = |sequence: u64| { let mut group = track.create_group(group::Info { sequence }).unwrap(); @@ -26,19 +35,28 @@ fn latest_skips_an_owed_hole() { }; put(0); - let Poll::Ready(Ok(Some(state))) = consumer.poll_latest(&waiter) else { - panic!("missing group 0 with max {max:?}") + let Poll::Ready(Ok(Some(state))) = latest.poll_latest(&waiter) else { + panic!("latest missed group 0 with max {max:?}") + }; + assert_eq!(state.value, json!({ "n": 0 })); + let Poll::Ready(Ok(Some(state))) = next.poll_next(&waiter) else { + panic!("next missed group 0 with max {max:?}") }; assert_eq!(state.value, json!({ "n": 0 })); put(2); - let Poll::Ready(Ok(Some(state))) = consumer.poll_latest(&waiter) else { - panic!("group 2 waited on owed group 1 with max {max:?}") + let Poll::Ready(Ok(Some(state))) = latest.poll_latest(&waiter) else { + panic!("latest waited on owed group 1 with max {max:?}") + }; + assert_eq!(state.value, json!({ "n": 2 })); + let Poll::Ready(Ok(Some(state))) = next.poll_next(&waiter) else { + panic!("next waited on owed group 1 with max {max:?}") }; assert_eq!(state.value, json!({ "n": 2 })); // The owed group landing late is superseded, not delivered. put(1); - assert!(consumer.poll_next(&waiter).is_pending(), "max {max:?}"); + assert!(latest.poll_latest(&waiter).is_pending(), "max {max:?}"); + assert!(next.poll_next(&waiter).is_pending(), "max {max:?}"); } } diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 1f34076db91..4bb9c133ae5 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -6,7 +6,7 @@ How to update your code for each breaking change, newest first. ### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`Json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a window or stream track when every state must survive. +`Json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### JSON writes take a broadcast-clock timestamp ([#7691](https://github.com/moq-dev/moq/pull/7691)) From 22266bd1b1d1bbf274c5c7dce558f983879e32a1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 13:54:13 -0800 Subject: [PATCH 08/24] test(json): park the reads with the clock's sleep Co-Authored-By: Claude Opus 5.5 --- js/json/src/snapshot/snapshot.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index 6d1c52c79cc..97892384e15 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -1,5 +1,5 @@ import { expect, test } from "bun:test"; -import { Milli } from "@moq/clock"; +import { Milli, Nano, sleep } from "@moq/clock"; import { Error as NetError, StreamCode, Time, Track } from "@moq/net"; import { hooks } from "../../../net/src/internal.ts"; import { Consumer } from "./consumer.ts"; @@ -224,7 +224,7 @@ test("reads skip a group the subscription still owes", async () => { expect((await next.next())?.value).toEqual({ n: 0 }); // Restored again while both reads wait, so the pin must come from the update itself. const reads = [latest.latest(), next.next()]; - await new Promise((resolve) => setTimeout(resolve, 0)); + await sleep(Nano.fromMilli(Milli(1))); restore(); put(2n); expect((await Promise.all(reads)).map((state) => state?.value)).toEqual([{ n: 2 }, { n: 2 }]); From 90d87ad47c09a1760c09fce8cc26f79a4e5bd207 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 15:36:12 -0800 Subject: [PATCH 09/24] fix(json): keep the zero delay across coalesced and concurrent updates JS: a signal notifies only a microtask's net change, so an update that undid the pin within the same microtask notified nobody and a parked read kept the restored budget. Every pin that writes now re-checks a microtask later, which also makes the per-read pin redundant. Rust: the pin copied the whole subscription and wrote it back, losing a retained Control's concurrent range or priority change. moq-net gains Subscriber/Ordered::set_max_delay, which changes only delay.max under the preferences' write lock. Co-Authored-By: Claude Opus 5.5 --- js/json/src/snapshot/consumer.ts | 13 ++++--- js/json/src/snapshot/snapshot.test.ts | 27 +++++++++++++++ rs/moq-json/src/snapshot/consumer.rs | 11 +++--- rs/moq-net/src/model/track.rs | 49 ++++++++++++++++++++++++++- rs/moq-net/src/model/track/demand.rs | 8 +++++ 5 files changed, 97 insertions(+), 11 deletions(-) diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index a0bff89c39b..a7f29dce774 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -36,17 +36,23 @@ export class Consumer { this.#track = config.track.ordered(); this.#pin(); // A retained handle can update the subscription while a read waits; put the zero back at - // once, which also wakes a read that update left holding at an owed group. Writes coalesce - // per microtask, so an update undone in the same one never notifies: #read pins as well. + // once, which also wakes a read that update left holding at an owed group. this.#track.subscription.subscribe(() => this.#pin()); this.#decoder = new Decoder(config); } // Hold the subscription's max delay at zero, the default subscription's, keeping the rest of it. + // + // A signal notifies only the net change across a microtask, so an update that undoes a pin + // before that microtask ends notifies nobody: not the subscriber above, nor a read waiting on + // the subscription inside `@moq/net`. Every pin that writes checks again a microtask later, + // after that update could have run. #pin(): void { const zero = Moq.Track.Subscription.default().delay?.max; const current = this.#track.subscription.peek(); - if (current?.delay?.max !== zero) this.#track.update({ ...current, delay: { ...current?.delay, max: zero } }); + if (current?.delay?.max === zero) return; + this.#track.update({ ...current, delay: { ...current?.delay, max: zero } }); + queueMicrotask(() => this.#pin()); } /** @@ -75,7 +81,6 @@ export class Consumer { async #read(skip: boolean): Promise | undefined> { for (;;) { - this.#pin(); if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index 97892384e15..b6fbe5f4145 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -238,6 +238,33 @@ test("reads skip a group the subscription still owes", async () => { } }); +// An update that undoes the consumer's pin before the microtask ends nets no change, so the signal +// notifies nobody, not even a read already waiting on the subscription. The pin still holds. +test("reads skip an owed group when an update undoes the pin in the same microtask", async () => { + for (const max of [Milli(1_000), null]) { + const track = new Track.Producer("test").accept({}); + hooks.receiveTrack(track); + const put = (sequence: bigint) => { + const group = track.createGroup(sequence); + group.writeFrame({ payload: new TextEncoder().encode(`{"n":${sequence}}`) }); + group.finish(); + }; + put(0n); + + // Starting past group 0, both reads wait on group 1 from the first poll. + const subscription = { delay: { max }, ranges: [{ start: { absolute: 1n } }] }; + const subscribers = [track.subscribe(subscription), track.subscribe(subscription)]; + const [latest, next] = subscribers.map((subscriber) => new Consumer({ track: subscriber })); + const reads = [latest.latest(), next.next()]; + for (const subscriber of subscribers) subscriber.update(subscription); + + await sleep(Nano.fromMilli(Milli(1))); + put(2n); + expect((await Promise.all(reads)).map((state) => state?.value)).toEqual([{ n: 2 }, { n: 2 }]); + track.close(); + } +}); + test("live consumer sees each update", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index 83f415bfdbf..26598783c4c 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -60,12 +60,11 @@ impl Consumer { /// Hold the subscription's max delay at zero, keeping the rest of it. fn pin_delay(&mut self) { - let mut subscription = self.track.subscription(); - if subscription.delay.max != Some(std::time::Duration::ZERO) { - subscription.delay.max = Some(std::time::Duration::ZERO); - // Refused only when the track has ended or its ranges name time on an untimed - // track; neither is this reader's to fix. - let _ = self.track.update(subscription); + const ZERO: Option = Some(std::time::Duration::ZERO); + if self.track.subscription().delay.max != ZERO { + // In place, so a retained handle's concurrent change to the ranges or priority stays. + // Refused only once the track has ended, which leaves nothing to read anyway. + let _ = self.track.set_max_delay(ZERO); } } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 951c6e65c2f..33793c3c265 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -9078,6 +9078,27 @@ impl Subscriber { self.consume().refuse_subscription(&subscription)?; } channel.update(subscription)?; + self.synced(); + Ok(()) + } + + /// Set this subscriber's [`max`](Delay::max) delay, keeping the rest of its preferences. + /// + /// Unlike passing [`Self::subscription`] back to [`Self::update`], the change applies in + /// place, so a [`Control`] updating another preference meanwhile keeps its change. + /// Returns [`Error::Closed`] if the track already ended. + pub fn set_max_delay(&mut self, max: Option) -> Result<()> { + let channel = match &self.inner { + Inner::Plain(cursor) => &cursor.subscription, + Inner::Resume(resume, _) => resume.subscription(), + }; + channel.update_with(|subscription| subscription.delay.max = max)?; + self.synced(); + Ok(()) + } + + /// Bring the read side up to date with preferences this handle just changed. + fn synced(&mut self) { match &mut self.inner { Inner::Plain(cursor) => { cursor.sync(); @@ -9086,7 +9107,6 @@ impl Subscriber { // sources even while nobody reads. Inner::Resume(resume, _) => resume.mirror().sync(), } - Ok(()) } /// Poll for the track's largest location within [`Self::with_range`]: the current one, then each newer one, and `None` once the track ends. @@ -9314,6 +9334,12 @@ impl Ordered { self.inner.update(subscription) } + /// Set this subscriber's [`max`](Delay::max) delay, keeping the rest of its preferences; + /// see [`Subscriber::set_max_delay`]. + pub fn set_max_delay(&mut self, max: Option) -> Result<()> { + self.inner.set_max_delay(max) + } + /// Block until the track declares its end, returning the exclusive end position. /// /// See [`Subscriber::finished`]: this reports the declared end, not that every group @@ -14772,6 +14798,27 @@ mod test { assert_eq!(aggregate.priority, 7); } + #[test] + fn set_max_delay_keeps_a_controls_change() { + let producer = track_producer("test", Info::timed()); + let asked = Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))); + let mut subscriber = producer.subscribe(asked.clone()).ordered(); + let control = subscriber.control(); + + // Another handle changes the priority after this side last read its preferences. + let stale = subscriber.subscription(); + control.update(asked.with_priority(7)).unwrap(); + subscriber.set_max_delay(Some(Duration::ZERO)).unwrap(); + + let expected = Subscription::default() + .with_priority(7) + .with_delay(Delay::default().with_max(Duration::ZERO)); + assert_eq!(subscriber.subscription(), expected); + assert_eq!(producer.subscription(), Some(expected)); + // Writing back the copy read before the change would have lost it. + assert_eq!(stale.priority, 0); + } + #[test] fn dropped_subscriber_leaves_no_ghost_in_aggregate() { // Regression (#2351): a departed subscriber must not keep contributing its diff --git a/rs/moq-net/src/model/track/demand.rs b/rs/moq-net/src/model/track/demand.rs index d7ed553ead5..7b5217690c7 100644 --- a/rs/moq-net/src/model/track/demand.rs +++ b/rs/moq-net/src/model/track/demand.rs @@ -227,7 +227,15 @@ impl Preferences { } pub(super) fn update(&self, next: Subscription) -> crate::Result<()> { + self.update_with(|current| *current = next) + } + + /// Change the preferences in place under the write lock, so an update another handle + /// makes meanwhile is kept rather than overwritten with a stale copy. + pub(super) fn update_with(&self, change: impl FnOnce(&mut Subscription)) -> crate::Result<()> { let mut value = self.0.value.write().map_err(|_| crate::Error::Closed)?; + let mut next = value.clone(); + change(&mut next); if *value == next { return Ok(()); } From 2edaac8b832f241c6a9c6e8ee05d9b4e30e9f7e0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 16:07:09 -0800 Subject: [PATCH 10/24] fix(json): pin before each JS read as well An update notifies a microtask late, so a read starting right after one picked its group under the restored budget. Restore the per-read pin, with a test that reads a superseded group without it. Co-Authored-By: Claude Opus 5.5 --- js/json/src/snapshot/consumer.ts | 4 ++++ js/json/src/snapshot/snapshot.test.ts | 14 ++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index a7f29dce774..5088000a3f4 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -47,6 +47,8 @@ export class Consumer { // before that microtask ends notifies nobody: not the subscriber above, nor a read waiting on // the subscription inside `@moq/net`. Every pin that writes checks again a microtask later, // after that update could have run. + // + // Notifications come a microtask late, so each read also pins before it picks a group. #pin(): void { const zero = Moq.Track.Subscription.default().delay?.max; const current = this.#track.subscription.peek(); @@ -81,6 +83,8 @@ export class Consumer { async #read(skip: boolean): Promise | undefined> { for (;;) { + // An update just before this read hasn't notified yet, and the read picks its group now. + this.#pin(); if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index b6fbe5f4145..af9b299732b 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -200,6 +200,20 @@ test("next skips every superseded group, whatever the subscription's max delay", } }); +// An update restores its budget a microtask before anything is notified, so a read starting right +// after it still skips to the newest group. +test("a read right after a restored budget skips superseded groups", async () => { + const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI }); + const producer = new Producer({ track, deltas: false }); + for (let n = 0; n <= 2; n++) producer.update({ value: { n }, at: Time.Timestamp.fromMillis(n * 1_000) }); + producer.finish(); + + const subscriber = track.subscribe(); + const consumer = new Consumer({ track: subscriber }); + subscriber.update({ delay: { max: null } }); + expect((await consumer.next())?.value).toEqual({ n: 2 }); +}); + // A session-fed track owes the groups between its arrivals. A snapshot reader never waits on one: // group 2 arriving while group 1 is still owed supersedes it at once, whatever budget the caller // subscribed with or later restores through the subscriber it kept. From 9b341eef05c7dbde918393624ba658dc675d3497 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 20:07:16 -0800 Subject: [PATCH 11/24] fix(net): pin a subscriber's max delay where every update applies The snapshot consumers re-pinned a zero max delay around each read, which left a window: a retained Control could restore the budget between the pin and the ordered poll, and the JS consumer's signal listener outlived it. moq-net now holds the pin itself. `Subscriber::pin_max_delay` (Rust) and `Subscriber.pinMaxDelay` (JS) set it once, and every later update, through any handle, keeps it under the same write. Both consumers pin once at construction and drop the per-read pins, the JS listener, and the microtask re-check. Co-Authored-By: Claude Opus 5.5 --- bun.lock | 2 +- js/json/package.json | 2 +- js/json/src/snapshot/consumer.ts | 32 +++----------- js/json/src/snapshot/snapshot.test.ts | 12 +++--- js/net/src/track.test.ts | 20 +++++++++ js/net/src/track.ts | 19 ++++++++ rs/moq-json/src/snapshot/consumer.rs | 28 ++++-------- .../tests/integration/snapshot.rs | 13 ++++-- rs/moq-net/src/model/track.rs | 43 +++++++++++-------- rs/moq-net/src/model/track/demand.rs | 16 ++++++- 10 files changed, 111 insertions(+), 76 deletions(-) diff --git a/bun.lock b/bun.lock index 17307876368..a48e5bc164d 100644 --- a/bun.lock +++ b/bun.lock @@ -197,11 +197,11 @@ "name": "@moq/json", "version": "0.4.2", "dependencies": { + "@moq/clock": "workspace:^", "@moq/net": "workspace:^", "@moq/signals": "workspace:^", }, "devDependencies": { - "@moq/clock": "workspace:^", "@types/bun": "^1.4.4", "fast-check": "^4.10.2", "typescript": "7.0.2", diff --git a/js/json/package.json b/js/json/package.json index 1a27c1a37d9..b7596dd7cae 100644 --- a/js/json/package.json +++ b/js/json/package.json @@ -20,6 +20,7 @@ "release": "bun ../common/release.ts" }, "dependencies": { + "@moq/clock": "workspace:^", "@moq/net": "workspace:^", "@moq/signals": "workspace:^" }, @@ -27,7 +28,6 @@ "zod": "^4.5.0" }, "devDependencies": { - "@moq/clock": "workspace:^", "@types/bun": "^1.4.4", "fast-check": "^4.10.2", "typescript": "7.0.2" diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 5088000a3f4..13591174611 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -1,3 +1,4 @@ +import { Milli } from "@moq/clock"; import type { Timed } from "@moq/net"; import * as Moq from "@moq/net"; @@ -21,9 +22,9 @@ const GAPS: Moq.StreamCode[] = [ * What {@link latest} skips is gone, so a later {@link next} resumes after it. * When something else already owns the track, use the {@link Decoder} directly. * - * A snapshot is superseded the moment a newer one exists, so the consumer always reads at a zero - * `delay.max`, whatever the subscription asks for, then or later: a group is skipped once a newer - * one arrives, and a lower group the subscription still owes never holds up a newer one. + * A snapshot is superseded the moment a newer one exists, so the consumer pins its subscription's + * `delay.max` at zero, whatever the subscription asks for, then or later: a group is skipped once + * a newer one arrives, and a lower group the subscription still owes never holds up a newer one. */ export class Consumer { #track: Moq.Track.Ordered; @@ -34,29 +35,10 @@ export class Consumer { constructor(config: Consumer.Config) { this.#track = config.track.ordered(); - this.#pin(); - // A retained handle can update the subscription while a read waits; put the zero back at - // once, which also wakes a read that update left holding at an owed group. - this.#track.subscription.subscribe(() => this.#pin()); + this.#track.pinMaxDelay(Milli.zero); this.#decoder = new Decoder(config); } - // Hold the subscription's max delay at zero, the default subscription's, keeping the rest of it. - // - // A signal notifies only the net change across a microtask, so an update that undoes a pin - // before that microtask ends notifies nobody: not the subscriber above, nor a read waiting on - // the subscription inside `@moq/net`. Every pin that writes checks again a microtask later, - // after that update could have run. - // - // Notifications come a microtask late, so each read also pins before it picks a group. - #pin(): void { - const zero = Moq.Track.Subscription.default().delay?.max; - const current = this.#track.subscription.peek(); - if (current?.delay?.max === zero) return; - this.#track.update({ ...current, delay: { ...current?.delay, max: zero } }); - queueMicrotask(() => this.#pin()); - } - /** * Get the next state in order, or `undefined` once the track ends. * @@ -83,8 +65,6 @@ export class Consumer { async #read(skip: boolean): Promise | undefined> { for (;;) { - // An update just before this read hasn't notified yet, and the read picks its group now. - this.#pin(); if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share @@ -161,7 +141,7 @@ export class Consumer { export namespace Consumer { /** Snapshot consumer options, including the source track. */ export type Config = Pick, "schema"> & { - /** The track to read, whose `delay.max` the consumer sets to zero. */ + /** The track to read, whose `delay.max` the consumer pins at zero. */ track: Moq.Track.Subscriber; }; } diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index af9b299732b..44e54d6f497 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -200,8 +200,7 @@ test("next skips every superseded group, whatever the subscription's max delay", } }); -// An update restores its budget a microtask before anything is notified, so a read starting right -// after it still skips to the newest group. +// An update right before a read keeps the pinned zero, so the read still skips to the newest group. test("a read right after a restored budget skips superseded groups", async () => { const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI }); const producer = new Producer({ track, deltas: false }); @@ -211,6 +210,7 @@ test("a read right after a restored budget skips superseded groups", async () => const subscriber = track.subscribe(); const consumer = new Consumer({ track: subscriber }); subscriber.update({ delay: { max: null } }); + expect(subscriber.subscription.peek()?.delay?.max).toBe(Milli.zero); expect((await consumer.next())?.value).toEqual({ n: 2 }); }); @@ -236,7 +236,7 @@ test("reads skip a group the subscription still owes", async () => { put(0n); expect((await latest.latest())?.value).toEqual({ n: 0 }); expect((await next.next())?.value).toEqual({ n: 0 }); - // Restored again while both reads wait, so the pin must come from the update itself. + // Restored again while both reads wait: every update keeps the pinned zero. const reads = [latest.latest(), next.next()]; await sleep(Nano.fromMilli(Milli(1))); restore(); @@ -252,9 +252,9 @@ test("reads skip a group the subscription still owes", async () => { } }); -// An update that undoes the consumer's pin before the microtask ends nets no change, so the signal -// notifies nobody, not even a read already waiting on the subscription. The pin still holds. -test("reads skip an owed group when an update undoes the pin in the same microtask", async () => { +// An update in the same microtask as the consumer's pin keeps it too, so a read already waiting +// never sees the restored budget. +test("reads skip an owed group when an update follows the pin in the same microtask", async () => { for (const max of [Milli(1_000), null]) { const track = new Track.Producer("test").accept({}); hooks.receiveTrack(track); diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index abad37e2ff9..c3665096bdb 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -755,6 +755,26 @@ test("subscriber options and updates are forwarded to the producer's aggregate", }); }); +test("a pinned max delay survives every later update and reaches the aggregate", async () => { + const producer = new TrackProducer("test"); + const track = producer.subscribe({ priority: 2, delay: { max: Milli(1_000) } }); + const ordered = track.ordered(); + ordered.pinMaxDelay(Milli.zero); + expect(track.subscription.peek()?.delay?.max).toBe(Milli.zero); + expect(track.subscription.peek()?.priority).toBe(2); + + // Either handle restoring the budget keeps the pin and applies the rest. + track.update({ priority: 5, delay: { max: null } }); + ordered.update({ priority: 6, delay: { max: Milli(250) } }); + const expected = { priority: 6, delay: { min: Milli(0), max: Milli(0) }, order: "descending", ranges: [{}] }; + expect(track.subscription.peek()).toEqual(expected); + await Promise.resolve(); + expect(producer.subscription.peek()).toEqual(expected); + track.close(); + producer.finish(); + producer.close(); +}); + test("subscriber maxAge and maxDelay are refused before defaults or updates take effect", () => { const producer = new TrackProducer("test"); const track = producer.subscribe({ delay: { max: Milli(250) } }); diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 74c246c66f7..f5eab000300 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -2066,6 +2066,8 @@ export class Subscriber { // A FETCH's own budget in place of the subscription's; unlike SUBSCRIBE's zero // default, an omitted one is unbounded, leaving publisher retention alone. #fetchBudget?: number; + // The max delay every {@link update} keeps, once {@link pinMaxDelay} sets one. + #maxDelayPin?: Milli; // Which cursor owns this subscription. Both cursors draw from one buffer, so the // first group read commits the track to arrival order and {@link ordered} commits // it to sequence order; whichever wins is the only one allowed from here on. @@ -3694,6 +3696,7 @@ export class Subscriber { */ update(options: Subscription) { const subscription = subscriptionDefaults(options); + if (this.#maxDelayPin !== undefined) subscription.delay = { ...subscription.delay, max: this.#maxDelayPin }; const replaced = !sameRanges(subscription.ranges ?? [], this.#window.ranges); const reordered = subscription.order !== this.#state.update.peek()?.order; this.#state.update.set(subscription); @@ -3701,6 +3704,17 @@ export class Subscriber { // A fill's order follows its readers, so its handler wakes to serve the new first group. else if (reordered) this.#fillsMoved(); } + + /** + * Hold this subscription's `delay.max` at `max`, keeping its other options, for as long as + * the subscription lives: every later {@link update}, through this handle or the + * {@link Ordered} one, keeps the pinned max delay and applies the rest. Mirrors the Rust + * `Subscriber::pin_max_delay`. + */ + pinMaxDelay(max: Milli) { + this.#maxDelayPin = wireMillis("delay.max", max); + this.update(this.#state.update.peek() ?? Subscription.default()); + } } /** @@ -3782,6 +3796,11 @@ export class Ordered { this.#subscriber.update(options); } + /** Hold this subscription's `delay.max` at `max` through every later update; see {@link Subscriber.pinMaxDelay}. */ + pinMaxDelay(max: Milli) { + this.#subscriber.pinMaxDelay(max); + } + /** Close the track (optionally with an error), closing any pending groups. Idempotent. */ close(abort?: Error) { this.#subscriber.close(abort); diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index 26598783c4c..efdb507b020 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -43,28 +43,19 @@ pub struct Consumer { impl Consumer { /// Create a consumer reading from the given track subscriber. /// - /// Sets the subscription's [`max`](moq_net::track::Delay::max) delay to zero, whatever the - /// caller asked for, and again on every read if a retained handle changed it; its other - /// preferences, such as ranges and priority, stay. + /// Pins the subscription's [`max`](moq_net::track::Delay::max) delay at zero, whatever the + /// caller asked for, then or later through a retained handle; its other preferences, such + /// as ranges and priority, stay and still update. /// A [`compress`](moq_net::track::Info::compress)ed track reads the same: moq-net inflates it. pub fn new(track: moq_net::track::Subscriber, config: Config) -> Self { - let mut consumer = Self { - track: track.ordered(), + let mut track = track.ordered(); + // Refused only once the track has ended, which leaves nothing to read anyway. + let _ = track.pin_max_delay(std::time::Duration::ZERO); + Self { + track, group: None, decoder: Decoder::new(config), frames_read: 0, - }; - consumer.pin_delay(); - consumer - } - - /// Hold the subscription's max delay at zero, keeping the rest of it. - fn pin_delay(&mut self) { - const ZERO: Option = Some(std::time::Duration::ZERO); - if self.track.subscription().delay.max != ZERO { - // In place, so a retained handle's concurrent change to the ranges or priority stays. - // Refused only once the track has ended, which leaves nothing to read anyway. - let _ = self.track.set_max_delay(ZERO); } } @@ -101,9 +92,6 @@ impl Consumer { } fn poll_read(&mut self, waiter: &kio::Waiter, skip: bool) -> Poll>>> { - // Every wait ends in another poll, so pinning here undoes a retained handle's update - // before a read could hold at an owed group under it. - self.pin_delay(); if skip { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share diff --git a/rs/moq-net-tests/tests/integration/snapshot.rs b/rs/moq-net-tests/tests/integration/snapshot.rs index 91c238be7f7..b688152f572 100644 --- a/rs/moq-net-tests/tests/integration/snapshot.rs +++ b/rs/moq-net-tests/tests/integration/snapshot.rs @@ -6,7 +6,7 @@ use std::{task::Poll, time::Duration}; /// Group 2 arrives while group 1 is still owed: `latest()` and `next()` return it at once, /// whatever budget the caller subscribed with or later restores through a retained handle, -/// since a snapshot consumer always reads at zero. +/// since a snapshot consumer pins its max delay at zero. #[test] fn reads_skip_an_owed_hole() { for max in [Some(Duration::ZERO), Some(Duration::from_secs(1)), None] { @@ -21,10 +21,10 @@ fn reads_skip_an_owed_hole() { let control = subscriber.control(); let consumer = snapshot::Consumer::::new(subscriber, Default::default()); control.update(subscription.clone()).unwrap(); - consumer + (consumer, control) }; - let mut latest = reader(); - let mut next = reader(); + let (mut latest, latest_control) = reader(); + let (mut next, next_control) = reader(); let waiter = kio::Waiter::noop(); let put = |sequence: u64| { let mut group = track.create_group(group::Info { sequence }).unwrap(); @@ -44,6 +44,11 @@ fn reads_skip_an_owed_hole() { }; assert_eq!(state.value, json!({ "n": 0 })); + // Restored again while both reads wait: every update keeps the pinned zero. + for control in [&latest_control, &next_control] { + control.update(subscription.clone()).unwrap(); + assert_eq!(control.subscription().delay.max, Some(Duration::ZERO)); + } put(2); let Poll::Ready(Ok(Some(state))) = latest.poll_latest(&waiter) else { panic!("latest waited on owed group 1 with max {max:?}") diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 41bb9e7bdbc..0537cabfb5e 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -9101,17 +9101,19 @@ impl Subscriber { Ok(()) } - /// Set this subscriber's [`max`](Delay::max) delay, keeping the rest of its preferences. + /// Hold this subscriber's [`max`](Delay::max) delay at `max`, keeping the rest of its + /// preferences, for as long as the subscription lives. /// - /// Unlike passing [`Self::subscription`] back to [`Self::update`], the change applies in - /// place, so a [`Control`] updating another preference meanwhile keeps its change. - /// Returns [`Error::Closed`] if the track already ended. - pub fn set_max_delay(&mut self, max: Option) -> Result<()> { + /// Every later [`Self::update`], or one through a [`Control`] or a clone, keeps the pinned + /// max delay and applies the rest. The pin applies in place, under the same lock as those + /// updates, so no update lands in between and none made meanwhile is lost. Returns + /// [`Error::Closed`] if the track already ended. + pub fn pin_max_delay(&mut self, max: Duration) -> Result<()> { let channel = match &self.inner { Inner::Plain(cursor) => &cursor.subscription, Inner::Resume(resume, _) => resume.subscription(), }; - channel.update_with(|subscription| subscription.delay.max = max)?; + channel.pin_max_delay(max)?; self.synced(); Ok(()) } @@ -9353,10 +9355,10 @@ impl Ordered { self.inner.update(subscription) } - /// Set this subscriber's [`max`](Delay::max) delay, keeping the rest of its preferences; - /// see [`Subscriber::set_max_delay`]. - pub fn set_max_delay(&mut self, max: Option) -> Result<()> { - self.inner.set_max_delay(max) + /// Hold this subscriber's [`max`](Delay::max) delay at `max` through every later update; + /// see [`Subscriber::pin_max_delay`]. + pub fn pin_max_delay(&mut self, max: Duration) -> Result<()> { + self.inner.pin_max_delay(max) } /// Block until the track declares its end, returning the exclusive end position. @@ -14818,7 +14820,7 @@ mod test { } #[test] - fn set_max_delay_keeps_a_controls_change() { + fn pinned_max_delay_survives_every_update() { let producer = track_producer("test", Info::timed()); let asked = Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))); let mut subscriber = producer.subscribe(asked.clone()).ordered(); @@ -14826,16 +14828,23 @@ mod test { // Another handle changes the priority after this side last read its preferences. let stale = subscriber.subscription(); - control.update(asked.with_priority(7)).unwrap(); - subscriber.set_max_delay(Some(Duration::ZERO)).unwrap(); - - let expected = Subscription::default() + control.update(asked.clone().with_priority(7)).unwrap(); + subscriber.pin_max_delay(Duration::ZERO).unwrap(); + let pinned = Subscription::default() .with_priority(7) .with_delay(Delay::default().with_max(Duration::ZERO)); - assert_eq!(subscriber.subscription(), expected); - assert_eq!(producer.subscription(), Some(expected)); + assert_eq!(subscriber.subscription(), pinned); // Writing back the copy read before the change would have lost it. assert_eq!(stale.priority, 0); + + // Restoring the budget through either handle keeps the pin and applies the rest. + control.update(asked.clone().with_priority(3)).unwrap(); + assert_eq!(subscriber.subscription(), pinned.clone().with_priority(3)); + let unlimited = Subscription::default().with_delay(Delay::default().with_max(None)); + subscriber.update(unlimited).unwrap(); + let expected = Subscription::default().with_delay(Delay::default().with_max(Duration::ZERO)); + assert_eq!(subscriber.subscription(), expected); + assert_eq!(producer.subscription(), Some(expected)); } #[test] diff --git a/rs/moq-net/src/model/track/demand.rs b/rs/moq-net/src/model/track/demand.rs index 7b5217690c7..cee33758edf 100644 --- a/rs/moq-net/src/model/track/demand.rs +++ b/rs/moq-net/src/model/track/demand.rs @@ -2,7 +2,7 @@ use std::{ collections::BTreeMap, sync::{ - Arc, + Arc, Mutex, PoisonError, atomic::{self, AtomicU64}, }, task::Poll, @@ -209,6 +209,9 @@ struct Preference { subscriptions: Option>, /// Bumped under the write whenever the ranges change. ranges_revision: AtomicU64, + /// The max delay every write keeps, once [`Preferences::pin_max_delay`] sets one. Set + /// and read only under `value`'s write lock, so no write lands between a pin and its use. + max_delay_pin: Mutex>, } impl Preferences { @@ -223,6 +226,7 @@ impl Preferences { value: kio::Producer::new(value), subscriptions, ranges_revision: AtomicU64::new(0), + max_delay_pin: Mutex::new(None), })) } @@ -236,6 +240,9 @@ impl Preferences { let mut value = self.0.value.write().map_err(|_| crate::Error::Closed)?; let mut next = value.clone(); change(&mut next); + if let Some(max) = *self.0.max_delay_pin.lock().unwrap_or_else(PoisonError::into_inner) { + next.delay.max = Some(max); + } if *value == next { return Ok(()); } @@ -254,6 +261,12 @@ impl Preferences { } Ok(()) } + + /// Hold the max delay at `max` from now on, whatever a later write asks for. + pub(super) fn pin_max_delay(&self, max: Duration) -> crate::Result<()> { + // Pinned inside the write, which then applies it. + self.update_with(|_| *self.0.max_delay_pin.lock().unwrap_or_else(PoisonError::into_inner) = Some(max)) + } } impl Preferences { @@ -299,6 +312,7 @@ impl From> for Preferences { value, subscriptions: None, ranges_revision: AtomicU64::new(0), + max_delay_pin: Mutex::new(None), })) } } From e7bccb6a71d1740aa7510f3b580b4b086ad490e3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 20:15:15 -0800 Subject: [PATCH 12/24] test(net): type the pinned subscription the test expects Co-Authored-By: Claude Opus 5.5 --- js/net/src/track.test.ts | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index c3665096bdb..201d7470903 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -766,7 +766,12 @@ test("a pinned max delay survives every later update and reaches the aggregate", // Either handle restoring the budget keeps the pin and applies the rest. track.update({ priority: 5, delay: { max: null } }); ordered.update({ priority: 6, delay: { max: Milli(250) } }); - const expected = { priority: 6, delay: { min: Milli(0), max: Milli(0) }, order: "descending", ranges: [{}] }; + const expected: Subscription = { + priority: 6, + delay: { min: Milli(0), max: Milli(0) }, + order: "descending", + ranges: [{}], + }; expect(track.subscription.peek()).toEqual(expected); await Promise.resolve(); expect(producer.subscription.peek()).toEqual(expected); From 0bbb5423b4c5ac6c0f576ac8c0180219e1b59272 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 20:18:14 -0800 Subject: [PATCH 13/24] chore(net): record pinMaxDelay in the API report Co-Authored-By: Claude Opus 5.5 --- js/net/api/index.api.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/js/net/api/index.api.md b/js/net/api/index.api.md index 1f88caaaaa9..429dbeff975 100644 --- a/js/net/api/index.api.md +++ b/js/net/api/index.api.md @@ -666,6 +666,7 @@ export namespace Track { largest(): Promise; readonly name: string; nextGroup(): Promise; + pinMaxDelay(max: Milli): void; readBool(): Promise; readFrame(): Promise<{ frame: number; @@ -731,6 +732,7 @@ export namespace Track { largest(): Promise; readonly name: string; ordered(): Track.Ordered; + pinMaxDelay(max: Milli): void; recvDatagram(): Promise; recvGroup(): Promise; get subscription(): Signals.Getter; From 053b2012b13aa2c46aa4d2d0f4cd3cbb3f9145e5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 21:31:21 -0800 Subject: [PATCH 14/24] fix(net): mirror a routed reader's preferences into its copies before each read The subscription signal notifies a microtask after an update, so a read made right after one (such as a snapshot consumer's pinned zero delay) picked its group from a copy still holding the old budget. Every read now mirrors changed preferences first, as the Rust Reader::sync does. Also drop the stale "every state in order" snapshot notes left in the unreleased migration guides and JS docs. Co-Authored-By: Claude Opus 5.5 --- doc/lib/js/json.md | 2 +- go/wrapper/MIGRATE.md | 2 +- js/json/MIGRATE.md | 2 +- js/json/src/snapshot/consumer.ts | 2 +- js/json/src/snapshot/index.ts | 4 ++-- js/net/src/resume.test.ts | 23 +++++++++++++++++++++++ js/net/src/resume.ts | 26 ++++++++++++++++++++++---- kt/moq/MIGRATE.md | 2 +- py/moq-rs/MIGRATE.md | 2 +- rs/moq-json/MIGRATE.md | 2 +- swift/MIGRATE.md | 2 +- 11 files changed, 55 insertions(+), 14 deletions(-) diff --git a/doc/lib/js/json.md b/doc/lib/js/json.md index 3ec744712f6..d0c49c002c1 100644 --- a/doc/lib/js/json.md +++ b/doc/lib/js/json.md @@ -13,7 +13,7 @@ are the catalog modes on the [hang](/concept/hang#data-tracks) page. Window is a third framing for a raw track: both ends opt into it, and a generic catalog reader will not pick it up. -- **Snapshot**: lossy latest value, with RFC 7396 merge-patch deltas. A delta before any snapshot is an error. `next()` yields every state in order; `latest()` skips to the newest. Deltas ride one group until a joiner wants a fresh one: give the track a `refresh` ratio (8 is typical) and the producer restates the value in a new group. Without a ratio a group rolls only at its caps. +- **Snapshot**: lossy latest value, with RFC 7396 merge-patch deltas. A delta before any snapshot is an error. `next()` yields each state in order, skipping every group a newer one supersedes; `latest()` skips to the newest. Deltas ride one group until a joiner wants a fresh one: give the track a `refresh` ratio (8 is typical) and the producer restates the value in a new group. Without a ratio a group rolls only at its caps. - **Stream**: lossless append-log in a single group. A reader that falls behind fails the read rather than resuming mid-log. - **Window**: a bounded run of records a reader can join at any point. A new group restates what it keeps explicitly, so a reader that was keeping up is not handed a record twice. Its ops roll on the track's `refresh` ratio the same way. diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index d4c5d1cb465..fa0ffa97ef4 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -183,7 +183,7 @@ Application calls are unchanged. ### JSON consumers return `*json.Timed` (#5677) `SnapshotConsumer.Next`, `StreamConsumer.Next`, and `All` return `*Timed{Value, Timestamp}` instead of the raw message; `Timestamp` is a `*time.Duration`, nil when untimed. -A snapshot consumer's `Next` now yields every state in order; call `Latest(ctx)` for the newest. +A snapshot consumer's `Next` now yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); call `Latest(ctx)` for the newest. Timestamps are rounded down to a whole microsecond. ### Ranges filter reads (#5929) diff --git a/js/json/MIGRATE.md b/js/json/MIGRATE.md index ad5b68aaf63..c2505c58b01 100644 --- a/js/json/MIGRATE.md +++ b/js/json/MIGRATE.md @@ -36,7 +36,7 @@ Fix any publisher that writes a repeated key. The snapshot, stream, and window consumers' `next()` and async iterator yield `{ value, at }`, where `at` is the frame's timestamp and absent on an untimed track. A window event's `value` is the `Event` that `next()` used to return; a record restated by a group header carries the header frame's time. -A snapshot consumer's `next()` yields every state in order; call `latest()` to skip to the newest, as `next()` used to. +A snapshot consumer's `next()` yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); call `latest()` to skip to the newest, as `next()` used to. Where you awaited `next()` for the newest value, read `(await consumer.latest())?.value`. A caller that passes the result on as `any` or `unknown` still compiles, so search for every `next()` and `for await` rather than relying on type errors. diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 13591174611..65388405b2d 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -17,7 +17,7 @@ const GAPS: Moq.StreamCode[] = [ * * A {@link Decoder} that owns its track: it reads groups, routes each frame by its position, and * yields each reconstructed value with the timestamp of the frame that produced it. Two reads: - * {@link next} yields every state in order, for a caller that picks the state matching a playhead, + * {@link next} yields each state in order, skipping every group a newer one supersedes, * and {@link latest} skips to the newest state, for a caller that only wants the current value. * What {@link latest} skips is gone, so a later {@link next} resumes after it. * When something else already owns the track, use the {@link Decoder} directly. diff --git a/js/json/src/snapshot/index.ts b/js/json/src/snapshot/index.ts index 3bb074e92f8..311ecbfdd2d 100644 --- a/js/json/src/snapshot/index.ts +++ b/js/json/src/snapshot/index.ts @@ -3,8 +3,8 @@ * * One JSON value updated over time, for consumers that care about the state rather than every * record (a catalog, a status document). This mode is **lossy** by design: a new group supersedes - * the older ones, which are dropped. {@link Consumer.next} yields every state it still receives, - * in order, each with its frame's timestamp, for a caller that picks the state at a playhead. + * the older ones, which are dropped. {@link Consumer.next} yields each state in order, each with + * its frame's timestamp, skipping every group a newer one supersedes. * {@link Consumer.latest} jumps straight to the newest group and collapses any buffered backlog * into a single yield, for a caller that only wants the current value. For an ordered log where * every record is preserved, use the `Stream` module instead. diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index 6b23ae0951e..88de708aefa 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -1114,6 +1114,29 @@ test("an ordered reader takes a replaced copy's buffered group before the new co b.close(); }); +test("a routed read right after an update picks its group under the new preferences", async () => { + const front = new Front(); + const a = new BroadcastProducer(); + const ta = copy(a); + front.serve(a.consume()); + const ordered = front.consumer + .track("seq") + .subscribe({ delay: { max: null } }) + .ordered(); + try { + ta.write(0n); + ta.write(1n); + await settle(); + // The copy learns of the pin a microtask later; the read must not pick under its old budget. + ordered.pinMaxDelay(Milli.zero); + expect(ordered.tryNextGroup()?.sequence).toBe(1n); + } finally { + ordered.close(); + front.consumer.close(); + a.close(); + } +}); + for (const resolution of ["arrival", "drop", "other copy", "budget"] as const) { test(`an ordered resumed reader keeps an older copy's owed hole until ${resolution}`, async () => { const clock = new Manual(); diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index 5eeb0f98130..82a0de05570 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -300,6 +300,8 @@ class Reader implements ResumeSource { #shell: ResumeShell; /** The route generation last read. */ #generation?: number; + /** The reader's preferences as last mirrored into its copies; see {@link #mirror}. */ + #mirrored?: track.Subscription; /** * Oldest first, except that a backfill goes before the copy of the route it backfills: the last * one is the serving route's own, when it is subscribed. @@ -359,11 +361,10 @@ class Reader implements ResumeSource { this.#shell = shell; this.#record = record; this.#start = shell.cursor.peek().start; + this.#mirrored = shell.subscription.peek(); route.readers.add(this); this.#dispose.push( - shell.subscription.subscribe(() => { - for (const copy of this.#copies) this.#update(copy); - }), + shell.subscription.subscribe(() => this.#mirror()), shell.cursor.subscribe(() => { for (const copy of this.#copies) this.#bound(copy); const passed = this.#start; @@ -390,8 +391,12 @@ class Reader implements ResumeSource { this.sync(); } - /** Follow the route: subscribe to a new copy and mark the ones it replaced. */ + /** + * Follow the route: mirror the reader's preferences into its copies, subscribe to a new copy, + * and mark the ones it replaced. + */ sync(): void { + this.#mirror(); if (this.#closed || this.#ended) return; const route = this.route; if (this.#generation === route.generation) return; @@ -497,6 +502,19 @@ class Reader implements ResumeSource { if (copy.sub.closed.peek() === undefined) copy.sub.update(this.#wire(copy)); } + /** + * Ask every copy for the reader's preferences once they changed. The subscription signal + * notifies a microtask after an update, so every read mirrors first too: a group a copy picks + * then follows the preferences the reader holds, such as a pinned `delay.max`, not its last + * ones. Mirrors the Rust `Reader::sync`. + */ + #mirror(): void { + const subscription = this.#shell.subscription.peek(); + if (subscription === this.#mirrored) return; + this.#mirrored = subscription; + for (const copy of this.#copies) this.#update(copy); + } + /** * Apply the reader's group cursor to `copy`, never below the copy's own resume point: groups * under it were handed out by an earlier route, or never owed. diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index d002ec02224..ce334268d26 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -175,7 +175,7 @@ Application calls are unchanged. ### JSON consumers emit timestamps (#5677) `values()` emits `json.Value` (`value`, plus `timestamp` as a `Duration?`) instead of a string, and `valuesAs()` emits `Timed(value, timestamp)`. -A snapshot consumer's `next()` now yields every state in order; call `latest()` for the newest. +A snapshot consumer's `next()` now yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); call `latest()` for the newest. Timestamps are rounded down to a whole microsecond. ### Ranges filter reads (#5929) diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 9868a46c6ea..01a6374686c 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -171,7 +171,7 @@ and native library together; application calls are unchanged. ### JSON consumers yield `json.Timed` (#5677) `json.SnapshotConsumer` and `json.StreamConsumer` yield `Timed(value, timestamp)` instead of the bare value; `timestamp` is a `timedelta`, or `None` when untimed. -Iterating a snapshot consumer now yields every state in order; `await consumer.latest()` returns the newest. +Iterating a snapshot consumer now yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); `await consumer.latest()` returns the newest. Timestamps are rounded down to a whole microsecond. ### Ranges filter reads (#5929) diff --git a/rs/moq-json/MIGRATE.md b/rs/moq-json/MIGRATE.md index 2c85bec9226..871062f96e4 100644 --- a/rs/moq-json/MIGRATE.md +++ b/rs/moq-json/MIGRATE.md @@ -32,7 +32,7 @@ An `Encoder` user restates with `Encoder::snapshot()` once `Demand::wanted` pass Snapshot, stream, and window `next` / `poll_next` return `moq_net::Timed`: read `.value` for the value and `.at` for the frame's exact timestamp, `None` when untimed. A window event's `.value` is the `Event` that `next` used to return; a record restated by a group header carries the header frame's time. -Snapshot `next` now yields every state in order inside the subscription's `max_delay`; call `latest` / `poll_latest` for the old collapse-to-newest read. +Snapshot `next` now yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); call `latest` / `poll_latest` for the old collapse-to-newest read. ### Duplicate keys are refused diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index d7890dc961f..0c34eaf4e27 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -167,7 +167,7 @@ Application calls are unchanged. ### JSON consumers yield `Json.Timed` (#5677) `Json.SnapshotConsumer` and `Json.StreamConsumer` yield `Json.Timed` (`value`, `timestampUs`) instead of the bare value; `timestampUs` is `nil` when untimed. -A snapshot consumer's `next()` now yields every state in order; call `latest()` for the newest. +A snapshot consumer's `next()` now yields each state in order, skipping every group a newer one supersedes (see the zero-delay entry above); call `latest()` for the newest. Timestamps are rounded down to a whole microsecond. ### Ranges filter reads (#5929) From acb5d1d203ce33580cc61582525c027c2decf013 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 23:18:14 -0800 Subject: [PATCH 15/24] feat(json)!: snapshot consumers subscribe themselves at zero delay Replace the moq-net max-delay pin with a snapshot consumer that owns its subscription: Rust snapshot::Consumer::new takes a track::Consumer, the JS Snapshot.Consumer takes a Track.Consumer, and moq-ffi's constructor takes the broadcast and track name. Catalog wrappers in hang and moq-mux follow. Co-Authored-By: Claude Opus 5.5 --- dart/moq/MIGRATE.md | 6 +- dart/moq/lib/json.dart | 4 +- dart/moq/test/moq_test.dart | 9 +- doc/lib/js/json.md | 7 +- doc/lib/js/watch.md | 8 +- doc/lib/rs/moq-json.md | 5 +- go/wrapper/MIGRATE.md | 4 +- go/wrapper/README.md | 5 +- go/wrapper/json/json.go | 11 +- go/wrapper/json/json_test.go | 4 +- js/hang/src/catalog/root.ts | 11 +- js/json/MIGRATE.md | 9 +- js/json/README.md | 5 +- js/json/src/snapshot/consumer.ts | 31 +++- js/json/src/snapshot/snapshot.test.ts | 145 +++++++----------- js/json/src/types.test.ts | 15 +- js/moq-boy/src/game.ts | 8 +- js/net/api/index.api.md | 2 - js/net/src/resume.test.ts | 4 +- js/net/src/resume.ts | 4 +- js/net/src/track.test.ts | 25 --- js/net/src/track.ts | 19 --- js/publish/bench/catalog.ts | 6 +- js/publish/src/announce.test.ts | 5 +- js/publish/src/audio/encoder.test.ts | 21 ++- js/publish/src/broadcast.test.ts | 25 ++- js/publish/src/catalog.test.ts | 12 +- js/publish/src/video/encoder.test.ts | 28 ++-- js/room/src/metadata.test.ts | 5 +- js/room/src/metadata.ts | 5 +- js/watch/src/broadcast.ts | 21 +-- js/watch/src/echo.test.ts | 5 +- kt/moq/MIGRATE.md | 6 +- .../kotlin/dev/moq/json/Tracks.kt | 6 +- .../kotlin/dev/moq/SmokeTest.kt | 8 +- py/moq-rs/MIGRATE.md | 4 +- py/moq-rs/moq/json.py | 11 +- py/moq-rs/tests/test_local.py | 6 +- quest/m2/net-wrap/json.md | 7 +- rs/hang/MIGRATE.md | 6 + rs/hang/examples/subscribe.rs | 6 +- rs/hang/src/catalog/consumer.rs | 13 +- rs/hang/src/catalog/root.rs | 6 +- rs/moq-boy/src/input.rs | 2 +- rs/moq-c/src/consume.rs | 17 +- rs/moq-cli/src/report.rs | 14 +- rs/moq-ffi/MIGRATE.md | 4 + rs/moq-ffi/src/json.rs | 18 +-- rs/moq-ffi/src/media.rs | 18 +-- rs/moq-ffi/src/test.rs | 6 +- rs/moq-gst/src/source/imp.rs | 18 +-- rs/moq-json/MIGRATE.md | 10 +- rs/moq-json/examples/telemetry.rs | 6 +- rs/moq-json/examples/teleop.rs | 2 +- rs/moq-json/src/snapshot/consumer.rs | 101 ++++++------ rs/moq-json/src/snapshot/mod.rs | 40 ++--- rs/moq-mux/MIGRATE.md | 6 + rs/moq-mux/src/catalog/consumer.rs | 8 +- rs/moq-mux/src/catalog/hang/consumer.rs | 10 +- rs/moq-mux/src/container/test_util.rs | 3 - rs/moq-mux/src/fuzz.rs | 5 +- rs/moq-mux/src/json.rs | 58 +++---- rs/moq-net-tests/tests/integration/refresh.rs | 4 +- .../tests/integration/snapshot.rs | 23 +-- rs/moq-net/src/model/track.rs | 58 +------ rs/moq-net/src/model/track/demand.rs | 24 +-- rs/moq-rtc/src/lib.rs | 5 +- rs/moq-stats/src/aggregate.rs | 31 ++-- rs/moq-stats/src/consume.rs | 10 +- rs/moq-stats/src/produce.rs | 71 +++------ rs/moq-transcode/src/lib.rs | 33 ++-- swift/MIGRATE.md | 4 +- swift/README.md | 3 +- swift/Sources/Moq/Json.swift | 12 +- swift/Tests/MoqTests/SmokeTests.swift | 4 +- 75 files changed, 483 insertions(+), 698 deletions(-) diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 455dd1ca3a3..e7b77a9ea6a 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`SnapshotConsumer(broadcast: ..., name: ...)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. + ### `VideoInit.lower` takes either producer kind ([#7699](https://github.com/moq-dev/moq/pull/7699)) `VideoInit.lower` is a `Producer` (`MoqMediaProducer`) instead of a `TrackProducer`. Pass `TrackMoqMediaProducer(producer: ...)`, or `StreamMoqMediaProducer(producer: ...)` for a byte-stream rendition. @@ -40,7 +44,7 @@ A fixed audio playout delay `d` is `delay: Delay(min: d, max: ...)` with `estima `package:moq/flate.dart` is removed: publish opaque bytes on a `TrackProducer`. `SnapshotConfig` and `WindowConfig` lose `compression`, and `StreamConfig` is removed. Publish the track with `MoqTrackInfo(compress: true)` instead. -`StreamProducer` drops its `config`, and `SnapshotConsumer`, `StreamConsumer`, and `WindowConsumer` take only the track. +`StreamProducer` drops its `config`, and `StreamConsumer` and `WindowConsumer` take only the track. `SnapshotConsumer` takes the broadcast and track name; see the entry above. ### Paths with an empty segment are refused ([#6648](https://github.com/moq-dev/moq/pull/6648)) diff --git a/dart/moq/lib/json.dart b/dart/moq/lib/json.dart index 2ede41f88b8..cda9004ac77 100644 --- a/dart/moq/lib/json.dart +++ b/dart/moq/lib/json.dart @@ -34,8 +34,8 @@ import 'package:moq_ffi/moq_ffi.dart'; /// Publishes lossy latest-value JSON snapshots on a track it takes over. typedef SnapshotProducer = MoqJsonSnapshotProducer; -/// Consumes each JSON snapshot state from a track it takes over, or skips to -/// the newest with `latest()`. +/// Consumes each JSON snapshot state from a track it subscribes to itself, or +/// skips to the newest with `latest()`. typedef SnapshotConsumer = MoqJsonSnapshotConsumer; /// Configures a lossy latest-value JSON producer. diff --git a/dart/moq/test/moq_test.dart b/dart/moq/test/moq_test.dart index 0098a57d29c..1b9a6cb002c 100644 --- a/dart/moq/test/moq_test.dart +++ b/dart/moq/test/moq_test.dart @@ -584,11 +584,10 @@ void main() { ), config: moq_json.SnapshotConfig(), ); - final track = await broadcast - .consume() - .subscribeTrack(name: 'status', subscription: null) - .timeout(timeout); - final consumer = moq_json.SnapshotConsumer(track: track); + final consumer = moq_json.SnapshotConsumer( + broadcast: broadcast.consume(), + name: 'status', + ); for (var n = 1; n <= 4; n++) { producer.update( diff --git a/doc/lib/js/json.md b/doc/lib/js/json.md index d0c49c002c1..2c105c612f8 100644 --- a/doc/lib/js/json.md +++ b/doc/lib/js/json.md @@ -35,9 +35,10 @@ for await (const { value, at } of consumer) { ``` `next()` yields every buffered state of the group it is reading and skips a -group once a newer one supersedes it: the consumer always reads at a zero -`delay.max`, whatever the subscription asked for. Use `latest()` to skip -straight to the head. +group once a newer one supersedes it: the consumer takes the track, a +`Track.Consumer` such as `broadcast.track(name)`, and subscribes to it itself at +a zero `delay.max`, so nothing can raise its budget. Use `latest()` to skip +straight to the head, and `close()` to end the subscription. A stream rides one group with no record-count or cumulative byte limit. The group retains a 32 MiB tail; a reader that falls behind gets `Evicted` rather diff --git a/doc/lib/js/watch.md b/doc/lib/js/watch.md index 4bfd2702924..12b7250b591 100644 --- a/doc/lib/js/watch.md +++ b/doc/lib/js/watch.md @@ -145,13 +145,11 @@ const dispose = el.signals.run((effect) => { const name = metadata?.[0]; if (!active || !name) return; - const track = active.track(name).subscribe({ - ...Moq.Track.Subscription.default(), + const consumer = new Json.Snapshot.Consumer({ + track: active.track(name), priority: Hang.Catalog.PRIORITY.catalog, }); - effect.cleanup(() => track.close()); - - const consumer = new Json.Snapshot.Consumer({ track }); + effect.cleanup(() => consumer.close()); effect.spawn(async () => { for (;;) { const state = await effect.race(consumer.latest()); diff --git a/doc/lib/rs/moq-json.md b/doc/lib/rs/moq-json.md index d0615b9dddc..84e0611a307 100644 --- a/doc/lib/rs/moq-json.md +++ b/doc/lib/rs/moq-json.md @@ -27,8 +27,9 @@ same optional capture time when opened, including when they publish on drop. Consumers return `Timed`: read `.value` for the decoded value and `.at` for its optional frame timestamp, with the track's exact scale preserved. Snapshot `next()` yields states in order, skipping every group a newer one supersedes: -the consumer always reads at a zero `delay.max`, whatever the subscription -asked for. Use `latest()` to skip to the newest buffered state. Skipped states +`snapshot::Consumer::new` takes the track and subscribes to it itself at a zero +`delay.max`, so nothing can raise its budget. Use `latest()` to skip to the +newest buffered state. Skipped states are gone, and a later `next()` continues after them. Upgrading from an older release: see moq-json's diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index fa0ffa97ef4..89cfe668d6d 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,9 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`json.SnapshotConsumer` reads at a zero `Delay.Max`, whatever the subscription asked for: `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`json.NewSnapshotConsumer(broadcast, name)` takes the `*BroadcastConsumer` and the track name instead of a `*TrackConsumer`, and subscribes to the track itself at a zero `Delay.Max`. Drop the `SubscribeTrack` call; a refused subscription now fails the first read. `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### `media.VideoInit.Lower` is a `media.Producer` ([#7699](https://github.com/moq-dev/moq/pull/7699)) diff --git a/go/wrapper/README.md b/go/wrapper/README.md index 4ef501f19bc..00b00df522c 100644 --- a/go/wrapper/README.md +++ b/go/wrapper/README.md @@ -108,8 +108,9 @@ so the media importer selects the timescale. JSON tracks live in the `moq.dev/moq/json` subpackage, mirroring the `moq-json` crate. Import it under an alias next to `encoding/json`. Each type wraps a track: `NewSnapshotProducer` / `NewStreamProducer` take over a `TrackProducer` and advertise it -in the broadcast's catalog, and `NewSnapshotConsumer` / `NewStreamConsumer` take over a -`TrackConsumer` that has not read a group yet. Snapshots carry lossy latest state; streams +in the broadcast's catalog, `NewStreamConsumer` takes over a `TrackConsumer` that has not +read a group yet, and `NewSnapshotConsumer` takes the `BroadcastConsumer` and track name and +subscribes itself at a zero `Delay.Max`. Snapshots carry lossy latest state; streams carry every record in order. `NewWindowProducer` / `NewWindowConsumer` carry a sliding window of records a reader can join at any point; the catalog has no window mode, so the producer takes only the track. Producers accept any `encoding/json` value; consumers return diff --git a/go/wrapper/json/json.go b/go/wrapper/json/json.go index 973b7cf3a75..9c8a9577985 100644 --- a/go/wrapper/json/json.go +++ b/go/wrapper/json/json.go @@ -155,10 +155,11 @@ type SnapshotConsumer struct { inner *ffi.MoqJsonSnapshotConsumer } -// NewSnapshotConsumer reads track as a lossy latest-value JSON track. It takes over -// track, whose handle is closed afterward, and fails if track has already read a group. -func NewSnapshotConsumer(track *moq.TrackConsumer) (*SnapshotConsumer, error) { - inner, err := ffi.NewMoqJsonSnapshotConsumer(bridge.TrackConsumer(track)) +// NewSnapshotConsumer subscribes to the track named name on broadcast and reads it as a +// lossy latest-value JSON track. The consumer owns its subscription, at a zero Delay.Max; +// a read waits for it to resolve and fails if it cannot. +func NewSnapshotConsumer(broadcast *moq.BroadcastConsumer, name string) (*SnapshotConsumer, error) { + inner, err := ffi.NewMoqJsonSnapshotConsumer(bridge.BroadcastConsumer(broadcast), name) if err != nil { return nil, err } @@ -166,7 +167,7 @@ func NewSnapshotConsumer(track *moq.TrackConsumer) (*SnapshotConsumer, error) { } // Next returns the next state in order, or nil when the track ends. It skips -// every group a newer one supersedes, whatever the subscription's Delay.Max. +// every group a newer one supersedes. func (c *SnapshotConsumer) Next(ctx context.Context) (*Timed, error) { value, err := bridge.Call(ctx, c.inner.Cancel, c.inner.Next) return timed(value), err diff --git a/go/wrapper/json/json_test.go b/go/wrapper/json/json_test.go index df60fb57ee6..82caedceb6a 100644 --- a/go/wrapper/json/json_test.go +++ b/go/wrapper/json/json_test.go @@ -65,7 +65,7 @@ func TestTracks(t *testing.T) { if err != nil { t.Fatal(err) } - snapshotConsumer, err := moqjson.NewSnapshotConsumer(subscribeTrack(ctx, t, consumer, "status")) + snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status") if err != nil { t.Fatal(err) } @@ -246,7 +246,7 @@ func TestDemand(t *testing.T) { t.Fatalf("Wanted = %d before any subscriber", *wanted) } - snapshotConsumer, err := moqjson.NewSnapshotConsumer(subscribeTrack(ctx, t, consumer, "status")) + snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status") if err != nil { t.Fatal(err) } diff --git a/js/hang/src/catalog/root.ts b/js/hang/src/catalog/root.ts index 9dfbba8f5cd..9d64d6e36e5 100644 --- a/js/hang/src/catalog/root.ts +++ b/js/hang/src/catalog/root.ts @@ -1,6 +1,6 @@ import * as Json from "@moq/json"; import type * as Moq from "@moq/net"; -import { Path, Track } from "@moq/net"; +import { Path } from "@moq/net"; import * as z from "zod/mini"; import { AudioSchema } from "./audio"; import { BinarySchema } from "./binary"; @@ -186,15 +186,18 @@ export function checkResolvable(root: Root, base: Moq.Path.Valid): Root { * yielded unresolved. */ export async function* watch(broadcast: Moq.Broadcast.Consumer): AsyncIterable { - const track = broadcast.track(TRACK).subscribe({ ...Track.Subscription.default(), priority: PRIORITY.catalog }); + const consumer = new Json.Snapshot.Consumer({ + track: broadcast.track(TRACK), + schema: RootSchema, + priority: PRIORITY.catalog, + }); try { - const consumer = new Json.Snapshot.Consumer({ track, schema: RootSchema }); for (;;) { const root = await consumer.latest(); if (!root) return; yield checkResolvable(checkRenditions(root.value), broadcast.path); } } finally { - track.close(); + consumer.close(); } } diff --git a/js/json/MIGRATE.md b/js/json/MIGRATE.md index c2505c58b01..c18dd854690 100644 --- a/js/json/MIGRATE.md +++ b/js/json/MIGRATE.md @@ -4,10 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`Snapshot.Consumer` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. -`next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. +`Snapshot.Consumer`'s `track` option is the track itself, a `Track.Consumer` (`broadcast.track(name)`) or an in-process `Track.Producer`, instead of a `Track.Subscriber`. The consumer subscribes to it at a zero `delay.max` and default options otherwise; the new `priority` option sets the subscription's priority. +Call the consumer's new `close()` to end that subscription, where you closed your subscriber before. +No handle to the subscription leaves the consumer, so ranges, order, and `delay.max` can no longer be set on a snapshot reader. + +`next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes, whatever another reader of the track asks for. A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `Snapshot.Decoder`, or publish a `Stream` track when every state must survive, or a `Window` track for a bounded recent history. ### Compression moves to the track diff --git a/js/json/README.md b/js/json/README.md index 52130324737..b15aa79ac80 100644 --- a/js/json/README.md +++ b/js/json/README.md @@ -32,8 +32,9 @@ import { Time } from "@moq/net"; const producer = new Snapshot.Producer({ track }); producer.update({ value: { hello: "world" }, at: Time.Timestamp.now() }); -// Consume: each state in order, with the timestamp of the frame that produced it. -const consumer = new Snapshot.Consumer({ track: track.subscribe() }); +// Consume: each state in order, with the timestamp of the frame that produced it. The consumer +// subscribes to the track itself; pass `broadcast.track(name)` for a remote one. +const consumer = new Snapshot.Consumer({ track }); for await (const { value, at } of consumer) { console.log(value, at); } diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 65388405b2d..e0b5f84f0ab 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -22,9 +22,10 @@ const GAPS: Moq.StreamCode[] = [ * What {@link latest} skips is gone, so a later {@link next} resumes after it. * When something else already owns the track, use the {@link Decoder} directly. * - * A snapshot is superseded the moment a newer one exists, so the consumer pins its subscription's - * `delay.max` at zero, whatever the subscription asks for, then or later: a group is skipped once - * a newer one arrives, and a lower group the subscription still owes never holds up a newer one. + * A snapshot is superseded the moment a newer one exists, so the consumer subscribes itself at a + * zero `delay.max` and keeps that subscription to itself: a group is skipped once a newer one + * arrives, and a lower group the subscription still owes never holds up a newer one. + * {@link close} ends the subscription. */ export class Consumer { #track: Moq.Track.Ordered; @@ -34,11 +35,19 @@ export class Consumer { #framesRead = 0; constructor(config: Consumer.Config) { - this.#track = config.track.ordered(); - this.#track.pinMaxDelay(Milli.zero); + this.#track = config.track + .subscribe({ ...Moq.Track.Subscription.default(), priority: config.priority ?? 0, delay: { max: Milli.zero } }) + .ordered(); this.#decoder = new Decoder(config); } + /** End the consumer's subscription, closing any pending read. Idempotent. */ + close(): void { + this.#group?.close(); + this.#group = undefined; + this.#track.close(); + } + /** * Get the next state in order, or `undefined` once the track ends. * @@ -68,7 +77,7 @@ export class Consumer { if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share - // the sequence space) and stays inside any range the caller set on the reader. + // the sequence space). // Mirrors the Rust consumer. A newer group restarts from a full snapshot, so switching mid-group loses // nothing but stale state. for (let group = this.#track.tryNextGroup(); group; group = this.#track.tryNextGroup()) { @@ -141,7 +150,13 @@ export class Consumer { export namespace Consumer { /** Snapshot consumer options, including the source track. */ export type Config = Pick, "schema"> & { - /** The track to read, whose `delay.max` the consumer pins at zero. */ - track: Moq.Track.Subscriber; + /** + * The track to read: a {@link Moq.Track.Consumer}, or a {@link Moq.Track.Producer} to read + * in-process. The consumer subscribes to it at a zero `delay.max`, with default options + * otherwise. + */ + track: Pick; + /** Delivery priority of the consumer's subscription, higher first; defaults to 0. */ + priority?: number; }; } diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index 44e54d6f497..4d638d96a37 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -12,21 +12,25 @@ type Value = Record; // replay window instead of the transport's live-edge default. A snapshot consumer ignores it. const REPLAY_LATENCY = Milli(30_000); -// Reconstruct every state a consumer yields from `next()`, in order. -async function drain(track: Track.Subscriber): Promise { +// Reconstruct every state a consumer reading `track` yields from `next()`, in order. +async function drain(track: Track.Producer | Consumer): Promise { const out: Value[] = []; - for await (const { value } of new Consumer({ track })) out.push(value); + for await (const { value } of reader(track)) out.push(value); return out; } -// Reconstruct every state a consumer yields from `latest()`, in order. -async function drainLatest(track: Track.Subscriber): Promise { - const consumer = new Consumer({ track }); +// Reconstruct every state a consumer reading `track` yields from `latest()`, in order. +async function drainLatest(track: Track.Producer | Consumer): Promise { + const consumer = reader(track); const out: Value[] = []; for (let state = await consumer.latest(); state; state = await consumer.latest()) out.push(state.value); return out; } +function reader(track: Track.Producer | Consumer): Consumer { + return track instanceof Consumer ? track : new Consumer({ track }); +} + // Inspect the published layout via the public API: the frame count of each group, in order. // The track must be finished first so group/frame reads terminate. async function structure(track: Track.Ordered): Promise { @@ -54,7 +58,7 @@ test("a cut makes the next update a snapshot group", async () => { // The ratio would have kept every update in one group; the cut rolled it anyway. The replacement // opens with a full snapshot, so it is one frame rather than a delta appended to the first group. expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([2, 1]); - expect(await drain(track.subscribe())).toEqual([{ a: 1, b: 3 }]); + expect(await drain(track)).toEqual([{ a: 1, b: 3 }]); }); test("a cut republishes an unchanged value", async () => { @@ -116,7 +120,7 @@ test("a cut is inert without deltas", async () => { test("deltas off: a snapshot group per change", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: false }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.update({ value: { a: 1 } }); expect((await consumer.next())?.value).toEqual({ a: 1 }); producer.update({ value: { a: 2 } }); @@ -126,8 +130,8 @@ test("deltas off: a snapshot group per change", async () => { // Two changes => two single-frame snapshot groups. A reader arriving after both reads only the // newest: the older group holds superseded state (mirrors the Rust consumer). expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([1, 1]); - expect(await drain(track.subscribe())).toEqual([{ a: 2 }]); - expect(await drainLatest(track.subscribe())).toEqual([{ a: 2 }]); + expect(await drain(track)).toEqual([{ a: 2 }]); + expect(await drainLatest(track)).toEqual([{ a: 2 }]); }); // `latest()` holding a group must jump when a newer snapshot group rolls: the @@ -137,7 +141,7 @@ test("deltas off: a snapshot group per change", async () => { test("latest abandons a held group when a newer snapshot group rolls", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.update({ value: { a: 1 } }); expect((await consumer.latest())?.value).toEqual({ a: 1 }); @@ -152,17 +156,16 @@ test("latest abandons a held group when a newer snapshot group rolls", async () expect(await consumer.latest()).toBeUndefined(); }); -// Skipping to the head never widens the reader's range: a capped reader reads the newest state -// below its cap, and a datagram past every group does not strand it. -test("latest stays inside the reader's range", async () => { +// Skipping to the head takes groups, not the track's largest position, so a datagram past every +// group does not strand it. +test("latest ignores a datagram past every group", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: false }); - const subscriber = track.subscribe().withRange({ end: { excluded: 2n } }); - const consumer = new Consumer({ track: subscriber }); + const consumer = new Consumer({ track }); for (let n = 0; n < 4; n++) producer.update({ value: { a: n } }); track.appendDatagram(Time.Timestamp.now(), new Uint8Array([0])); - expect((await consumer.latest())?.value).toEqual({ a: 1 }); + expect((await consumer.latest())?.value).toEqual({ a: 3 }); producer.finish(); }); @@ -170,7 +173,7 @@ test("latest stays inside the reader's range", async () => { test("next reads a held group out before a newer snapshot group", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.update({ value: { a: 1 } }); expect((await consumer.next())?.value).toEqual({ a: 1 }); @@ -185,48 +188,32 @@ test("next reads a held group out before a newer snapshot group", async () => { expect(await consumer.next()).toBeUndefined(); }); -// A snapshot is superseded the moment a newer one exists, so the consumer reads at a zero budget -// whatever the subscription asked for: a reader that falls behind jumps to the newest group. -test("next skips every superseded group, whatever the subscription's max delay", async () => { +// A snapshot is superseded the moment a newer one exists, so the consumer subscribes at a zero +// budget whatever another reader of the track asks for: a reader that falls behind jumps to the +// newest group. +test("next skips every superseded group, whatever budget another reader asks for", async () => { const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI }); const producer = new Producer({ track, deltas: false }); for (let n = 1; n <= 3; n++) producer.update({ value: { n }, at: Time.Timestamp.fromMillis(n * 1_000) }); producer.finish(); for (const max of [Milli.zero, Milli(500), REPLAY_LATENCY, null]) { - const subscriber = track.subscribe({ delay: { max } }); - expect(await drain(subscriber)).toEqual([{ n: 3 }]); - expect(subscriber.subscription.peek()?.delay?.max).toBe(Milli.zero); + const other = track.subscribe({ delay: { max } }); + expect(await drain(track)).toEqual([{ n: 3 }]); + other.close(); } }); -// An update right before a read keeps the pinned zero, so the read still skips to the newest group. -test("a read right after a restored budget skips superseded groups", async () => { - const track = new Track.Producer("test").accept({ timescale: Time.Timescale.MILLI }); - const producer = new Producer({ track, deltas: false }); - for (let n = 0; n <= 2; n++) producer.update({ value: { n }, at: Time.Timestamp.fromMillis(n * 1_000) }); - producer.finish(); - - const subscriber = track.subscribe(); - const consumer = new Consumer({ track: subscriber }); - subscriber.update({ delay: { max: null } }); - expect(subscriber.subscription.peek()?.delay?.max).toBe(Milli.zero); - expect((await consumer.next())?.value).toEqual({ n: 2 }); -}); - // A session-fed track owes the groups between its arrivals. A snapshot reader never waits on one: -// group 2 arriving while group 1 is still owed supersedes it at once, whatever budget the caller -// subscribed with or later restores through the subscriber it kept. +// group 2 arriving while group 1 is still owed supersedes it at once, whatever budget another +// reader of the track, such as the caller's own, asks for. test("reads skip a group the subscription still owes", async () => { for (const max of [Milli.zero, Milli(1_000), null]) { const track = new Track.Producer("test").accept({}); hooks.receiveTrack(track); - const subscribers = [track.subscribe({ delay: { max } }), track.subscribe({ delay: { max } })]; - const [latest, next] = subscribers.map((subscriber) => new Consumer({ track: subscriber })); - const restore = () => { - for (const subscriber of subscribers) subscriber.update({ delay: { max } }); - }; - restore(); + const other = track.subscribe({ delay: { max } }); + const latest = new Consumer({ track }); + const next = new Consumer({ track }); const put = (sequence: bigint) => { const group = track.createGroup(sequence); group.writeFrame({ payload: new TextEncoder().encode(`{"n":${sequence}}`) }); @@ -236,10 +223,8 @@ test("reads skip a group the subscription still owes", async () => { put(0n); expect((await latest.latest())?.value).toEqual({ n: 0 }); expect((await next.next())?.value).toEqual({ n: 0 }); - // Restored again while both reads wait: every update keeps the pinned zero. const reads = [latest.latest(), next.next()]; await sleep(Nano.fromMilli(Milli(1))); - restore(); put(2n); expect((await Promise.all(reads)).map((state) => state?.value)).toEqual([{ n: 2 }, { n: 2 }]); @@ -248,33 +233,7 @@ test("reads skip a group the subscription still owes", async () => { const ends = [latest.latest(), next.next()]; track.finish(); expect(await Promise.all(ends)).toEqual([undefined, undefined]); - track.close(); - } -}); - -// An update in the same microtask as the consumer's pin keeps it too, so a read already waiting -// never sees the restored budget. -test("reads skip an owed group when an update follows the pin in the same microtask", async () => { - for (const max of [Milli(1_000), null]) { - const track = new Track.Producer("test").accept({}); - hooks.receiveTrack(track); - const put = (sequence: bigint) => { - const group = track.createGroup(sequence); - group.writeFrame({ payload: new TextEncoder().encode(`{"n":${sequence}}`) }); - group.finish(); - }; - put(0n); - - // Starting past group 0, both reads wait on group 1 from the first poll. - const subscription = { delay: { max }, ranges: [{ start: { absolute: 1n } }] }; - const subscribers = [track.subscribe(subscription), track.subscribe(subscription)]; - const [latest, next] = subscribers.map((subscriber) => new Consumer({ track: subscriber })); - const reads = [latest.latest(), next.next()]; - for (const subscriber of subscribers) subscriber.update(subscription); - - await sleep(Nano.fromMilli(Milli(1))); - put(2n); - expect((await Promise.all(reads)).map((state) => state?.value)).toEqual([{ n: 2 }, { n: 2 }]); + other.close(); track.close(); } }); @@ -282,7 +241,7 @@ test("reads skip an owed group when an update follows the pin in the same microt test("live consumer sees each update", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); for (let n = 1; n <= 3; n++) { producer.update({ value: { a: n } }); @@ -320,7 +279,7 @@ test("deltas reconstruct to the final value", async () => { producer.update({ value: { a: 5, b: 2 } }); producer.finish(); - expect((await drain(track.subscribe())).at(-1)).toEqual({ a: 5, b: 2 }); + expect((await drain(track)).at(-1)).toEqual({ a: 5, b: 2 }); }); // `mutate()` edits the shared document: multiple owners edit one producer, each touching its own @@ -329,7 +288,7 @@ test("deltas reconstruct to the final value", async () => { test("mutate composes independent owners", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, initial: {} }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.mutate((v) => { v.video = "v1"; @@ -346,7 +305,7 @@ test("mutate composes independent owners", async () => { test("mutate starts from the configured initial value", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, initial: {} }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.mutate((v) => { v.a = 1; @@ -375,7 +334,7 @@ test("a rejected mutate leaves the value at the last one written", async () => { v.scte35 = 42; }); producer.finish(); - expect(await drainLatest(track.subscribe())).toEqual([{ video: "v1", scte35: 42 }]); + expect(await drainLatest(track)).toEqual([{ video: "v1", scte35: 42 }]); }); test("mutate without a prior value or initial throws", () => { @@ -388,7 +347,7 @@ test("mutate without a prior value or initial throws", () => { test("mutate removes a section", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: true, initial: {} }); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); producer.mutate((v) => { v.a = 1; @@ -438,7 +397,7 @@ test("a joiner over the ratio gets a fresh snapshot", async () => { joiner.close(); expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([...layout]); - expect(await drainLatest(track.subscribe())).toEqual([{ n: 3 }]); + expect(await drainLatest(track)).toEqual([{ n: 3 }]); } }); @@ -508,7 +467,7 @@ test("a refresh after a rejected update restates the last written value", async joiner.close(); expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([4, 2]); - expect(await drain(track.subscribe())).toEqual([{ n: 3 }, { n: 4 }]); + expect(await drain(track)).toEqual([{ n: 3 }, { n: 4 }]); }); test("no ratio never refreshes", async () => { @@ -538,7 +497,7 @@ test("a compressed refresh opens a fresh group", async () => { joiner.close(); expect(await structure(track.subscribe({ delay: { max: REPLAY_LATENCY } }).ordered())).toEqual([2, 2]); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); const values: Value[] = []; for await (const { value } of consumer) values.push(value); expect(values).toEqual([ @@ -561,8 +520,8 @@ test("array change is a wholesale delta", async () => { test("a late joiner reads a buffered backlog state by state, or collapses it to the latest", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track, deltas: true }); - const subscriber = track.subscribe(); - const collapsed = track.subscribe(); + const subscriber = new Consumer({ track }); + const collapsed = new Consumer({ track }); for (let n = 0; n <= 20; n++) { producer.update({ value: { n } }); } @@ -600,7 +559,7 @@ test("a rejected update leaves the previous value readable", async () => { producer.finish(); // A reader arriving now still finds the last good value, not an empty superseding group. - expect(await drain(track.subscribe())).toEqual([{ keep: true }]); + expect(await drain(track)).toEqual([{ keep: true }]); }); test("a delta that would overflow the snapshot rolls a new one instead", async () => { @@ -625,7 +584,7 @@ test("a delta that would overflow the snapshot rolls a new one instead", async ( expect(await subscriber.nextGroup()).toBeUndefined(); // The newest value is readable with no earlier frame to apply it to. - expect(await drain(track.subscribe())).toEqual([{ v: "y".repeat(size) }]); + expect(await drain(track)).toEqual([{ v: "y".repeat(size) }]); }); test("a delta that would push the snapshot out of the group's tail rolls a new group", () => { @@ -652,7 +611,7 @@ test("a snapshot past the budget is refused", () => { // moq-lite version reports as OLD. The reader skips to the newer snapshot instead of failing. test("snapshot consumer skips a group the publisher dropped mid-read", async () => { const track = new Track.Producer("test"); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); const first = track.appendGroup(); first.writeFrame({ payload: new TextEncoder().encode('{"v":0}'), timestamp: Time.Timestamp.now() }); expect((await consumer.latest())?.value).toEqual({ v: 0 }); @@ -667,7 +626,7 @@ test("snapshot consumer skips a group the publisher dropped mid-read", async () // A malformed or failed group must reach the caller; only an explicit retention gap is resumable. test("snapshot consumer propagates a non-gap frame failure", async () => { const track = new Track.Producer("test"); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); const group = track.appendGroup(); group.writeFrame({ payload: new TextEncoder().encode('{"ok":true}'), timestamp: Time.Timestamp.now() }); expect((await consumer.next())?.value).toEqual({ ok: true }); @@ -708,7 +667,7 @@ test("an untimed state reads back untimed", async () => { producer.update({ value: { a: 2, b: "x".repeat(64) } }); producer.finish(); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); expect(await consumer.next()).toEqual({ value: { a: 1, b: "x".repeat(64) }, at: undefined }); expect(await consumer.next()).toEqual({ value: { a: 2, b: "x".repeat(64) }, at: undefined }); }); @@ -723,7 +682,7 @@ test("each state reads back with its frame's timestamp", async () => { } producer.finish(); - const consumer = new Consumer({ track: track.subscribe() }); + const consumer = new Consumer({ track }); const states: [unknown, number | undefined][] = []; for await (const { value, at } of consumer) states.push([value.n, at?.as(Time.Timescale.MILLI)]); expect(states).toEqual([ @@ -732,7 +691,7 @@ test("each state reads back with its frame's timestamp", async () => { [3, 3_000], ]); - const latest = new Consumer({ track: track.subscribe() }); + const latest = new Consumer({ track }); const head = await latest.latest(); expect(head?.value.n).toBe(3); expect(head?.at?.as(Time.Timescale.MILLI)).toBe(3_000); diff --git a/js/json/src/types.test.ts b/js/json/src/types.test.ts index 1da9d1c861d..68b72b0ae1f 100644 --- a/js/json/src/types.test.ts +++ b/js/json/src/types.test.ts @@ -23,7 +23,7 @@ test("track-owning wrappers take one options object", () => { expect(arities).toEqual([1, 1, 1, 1, 1, 1]); const producerTrack: SnapshotProducerInit["track"] = new Track.Producer("test"); - const consumerTrack: SnapshotConsumerInit["track"] = producerTrack.subscribe(); + const consumerTrack: SnapshotConsumerInit["track"] = producerTrack; expect(producerTrack).toBeDefined(); expect(consumerTrack).toBeDefined(); }); @@ -51,16 +51,17 @@ test("consumer options omit producer-only knobs", () => { }); test("object literals reject producer knobs and positional tracks", () => { - const track = new Track.Producer("test").subscribe(); + const source = new Track.Producer("test"); + const track = source.subscribe(); const reject = () => { // @ts-expect-error deltas is producer-only - new Snapshot.Consumer({ track, deltas: true }); + new Snapshot.Consumer({ track: source, deltas: true }); // @ts-expect-error initial is producer-only - new Snapshot.Consumer({ track, initial: {} }); + new Snapshot.Consumer({ track: source, initial: {} }); // @ts-expect-error a positional track is not an options object - new Snapshot.Consumer(track); - // @ts-expect-error a producer track is not a subscriber - new Snapshot.Consumer({ track: new Track.Producer("test") }); + new Snapshot.Consumer(source); + // @ts-expect-error a snapshot consumer subscribes itself, so it takes the track, not a subscriber + new Snapshot.Consumer({ track }); // @ts-expect-error ops is producer-only new Window.Consumer({ track, ops: true }); // @ts-expect-error checkpointRecords is producer-only diff --git a/js/moq-boy/src/game.ts b/js/moq-boy/src/game.ts index 0a043641b3a..432bff4c4e1 100644 --- a/js/moq-boy/src/game.ts +++ b/js/moq-boy/src/game.ts @@ -269,13 +269,11 @@ export class Game { const active = effect.get(this.broadcast.out.active); if (!active) return; - const statusTrack = active.track("status").subscribe({ ...Moq.Track.Subscription.default(), priority: 10 }); - effect.cleanup(() => statusTrack.close()); - // Reconstruct each status from snapshots and deltas, validated against the schema. - const consumer = new Json.Snapshot.Consumer({ track: statusTrack, schema: GameStatusSchema }); + const consumer = new Json.Snapshot.Consumer({ track: active.track("status"), schema: GameStatusSchema, priority: 10 }); + effect.cleanup(() => consumer.close()); - // Closing the track on cleanup unblocks a pending latest() (it returns undefined), so the loop + // Closing the consumer on cleanup unblocks a pending latest() (it returns undefined), so the loop // ends without racing the teardown. effect.spawn(async () => { for (;;) { diff --git a/js/net/api/index.api.md b/js/net/api/index.api.md index 429dbeff975..1f88caaaaa9 100644 --- a/js/net/api/index.api.md +++ b/js/net/api/index.api.md @@ -666,7 +666,6 @@ export namespace Track { largest(): Promise; readonly name: string; nextGroup(): Promise; - pinMaxDelay(max: Milli): void; readBool(): Promise; readFrame(): Promise<{ frame: number; @@ -732,7 +731,6 @@ export namespace Track { largest(): Promise; readonly name: string; ordered(): Track.Ordered; - pinMaxDelay(max: Milli): void; recvDatagram(): Promise; recvGroup(): Promise; get subscription(): Signals.Getter; diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index 88de708aefa..1b495caae46 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -1127,8 +1127,8 @@ test("a routed read right after an update picks its group under the new preferen ta.write(0n); ta.write(1n); await settle(); - // The copy learns of the pin a microtask later; the read must not pick under its old budget. - ordered.pinMaxDelay(Milli.zero); + // The copy learns of the update a microtask later; the read must not pick under its old budget. + ordered.update({ delay: { max: Milli.zero } }); expect(ordered.tryNextGroup()?.sequence).toBe(1n); } finally { ordered.close(); diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index 82a0de05570..4360aa9bd1c 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -505,8 +505,8 @@ class Reader implements ResumeSource { /** * Ask every copy for the reader's preferences once they changed. The subscription signal * notifies a microtask after an update, so every read mirrors first too: a group a copy picks - * then follows the preferences the reader holds, such as a pinned `delay.max`, not its last - * ones. Mirrors the Rust `Reader::sync`. + * then follows the preferences the reader holds, such as a just-lowered `delay.max`, not its + * last ones. Mirrors the Rust `Reader::sync`. */ #mirror(): void { const subscription = this.#shell.subscription.peek(); diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index 201d7470903..abad37e2ff9 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -755,31 +755,6 @@ test("subscriber options and updates are forwarded to the producer's aggregate", }); }); -test("a pinned max delay survives every later update and reaches the aggregate", async () => { - const producer = new TrackProducer("test"); - const track = producer.subscribe({ priority: 2, delay: { max: Milli(1_000) } }); - const ordered = track.ordered(); - ordered.pinMaxDelay(Milli.zero); - expect(track.subscription.peek()?.delay?.max).toBe(Milli.zero); - expect(track.subscription.peek()?.priority).toBe(2); - - // Either handle restoring the budget keeps the pin and applies the rest. - track.update({ priority: 5, delay: { max: null } }); - ordered.update({ priority: 6, delay: { max: Milli(250) } }); - const expected: Subscription = { - priority: 6, - delay: { min: Milli(0), max: Milli(0) }, - order: "descending", - ranges: [{}], - }; - expect(track.subscription.peek()).toEqual(expected); - await Promise.resolve(); - expect(producer.subscription.peek()).toEqual(expected); - track.close(); - producer.finish(); - producer.close(); -}); - test("subscriber maxAge and maxDelay are refused before defaults or updates take effect", () => { const producer = new TrackProducer("test"); const track = producer.subscribe({ delay: { max: Milli(250) } }); diff --git a/js/net/src/track.ts b/js/net/src/track.ts index f5eab000300..74c246c66f7 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -2066,8 +2066,6 @@ export class Subscriber { // A FETCH's own budget in place of the subscription's; unlike SUBSCRIBE's zero // default, an omitted one is unbounded, leaving publisher retention alone. #fetchBudget?: number; - // The max delay every {@link update} keeps, once {@link pinMaxDelay} sets one. - #maxDelayPin?: Milli; // Which cursor owns this subscription. Both cursors draw from one buffer, so the // first group read commits the track to arrival order and {@link ordered} commits // it to sequence order; whichever wins is the only one allowed from here on. @@ -3696,7 +3694,6 @@ export class Subscriber { */ update(options: Subscription) { const subscription = subscriptionDefaults(options); - if (this.#maxDelayPin !== undefined) subscription.delay = { ...subscription.delay, max: this.#maxDelayPin }; const replaced = !sameRanges(subscription.ranges ?? [], this.#window.ranges); const reordered = subscription.order !== this.#state.update.peek()?.order; this.#state.update.set(subscription); @@ -3704,17 +3701,6 @@ export class Subscriber { // A fill's order follows its readers, so its handler wakes to serve the new first group. else if (reordered) this.#fillsMoved(); } - - /** - * Hold this subscription's `delay.max` at `max`, keeping its other options, for as long as - * the subscription lives: every later {@link update}, through this handle or the - * {@link Ordered} one, keeps the pinned max delay and applies the rest. Mirrors the Rust - * `Subscriber::pin_max_delay`. - */ - pinMaxDelay(max: Milli) { - this.#maxDelayPin = wireMillis("delay.max", max); - this.update(this.#state.update.peek() ?? Subscription.default()); - } } /** @@ -3796,11 +3782,6 @@ export class Ordered { this.#subscriber.update(options); } - /** Hold this subscription's `delay.max` at `max` through every later update; see {@link Subscriber.pinMaxDelay}. */ - pinMaxDelay(max: Milli) { - this.#subscriber.pinMaxDelay(max); - } - /** Close the track (optionally with an error), closing any pending groups. Idempotent. */ close(abort?: Error) { this.#subscriber.close(abort); diff --git a/js/publish/bench/catalog.ts b/js/publish/bench/catalog.ts index a3b47134380..d05f37d7c05 100644 --- a/js/publish/bench/catalog.ts +++ b/js/publish/bench/catalog.ts @@ -27,10 +27,8 @@ async function measure(tracks: number, viewers: number, clock: Manual) { const output = new Track.Producer("catalog.json"); catalog.serve(output, scope); const subscribers = Array.from({ length: viewers }, () => { - const subscriber = output.subscribe(); return { - subscriber, - catalog: new Json.Snapshot.Consumer({ track: subscriber }), + catalog: new Json.Snapshot.Consumer({ track: output }), media: media.map((track) => track.subscribe()), }; }); @@ -81,7 +79,7 @@ async function measure(tracks: number, viewers: number, clock: Manual) { publishing.close(); output.close(); for (const viewer of subscribers) { - viewer.subscriber.close(); + viewer.catalog.close(); for (const track of viewer.media) track.close(); } for (const track of media) track.close(); diff --git a/js/publish/src/announce.test.ts b/js/publish/src/announce.test.ts index d943d4c0187..e27e66923be 100644 --- a/js/publish/src/announce.test.ts +++ b/js/publish/src/announce.test.ts @@ -176,8 +176,7 @@ test("capture gaps keep the broadcast instance and its track sequences", async ( const active: boolean[] = []; const stop = request.active.subscribe((value) => active.push(value !== undefined)); const media = net.track("audio").subscribe(); - const catalogTrack = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: catalogTrack }); + const catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); try { await catalog.latest(); await settle(); @@ -212,7 +211,7 @@ test("capture gaps keep the broadcast instance and its track sequences", async ( stop(); request.close(); media.close(); - catalogTrack.close(); + catalog.close(); } }); diff --git a/js/publish/src/audio/encoder.test.ts b/js/publish/src/audio/encoder.test.ts index 36557809e97..c0ab33bf602 100644 --- a/js/publish/src/audio/encoder.test.ts +++ b/js/publish/src/audio/encoder.test.ts @@ -798,7 +798,7 @@ test("audio failure stays local through demand changes and capture format re-arm const encoder = new Encoder("audio", { broadcast, capture: capture as never, bandwidth, enabled }); let subscriber: Moq.Track.Subscriber | undefined; let siblingSubscriber: Moq.Track.Subscriber | undefined; - let catalogSubscriber: Moq.Track.Subscriber | undefined; + let catalog: Json.Snapshot.Consumer | undefined; let recovered: Moq.Track.Subscriber | undefined; const push = async (first: number, channels: number) => { for (let i = 0; i < 3; i++) { @@ -828,8 +828,7 @@ test("audio failure stays local through demand changes and capture format re-arm expect(siblingTrack.appendGroup().sequence).toBe(0n); await push(0, 1); expect((await subscriber.largest())?.group).toBe(0n); - catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: catalogSubscriber }); + catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); expect((await catalog.latest())?.value.audio?.renditions.audio?.enabled).not.toBe(false); const live = LaggingAudioEncoder.instances.at(-1); if (!live) throw new Error("expected live audio encoder"); @@ -879,7 +878,7 @@ test("audio failure stays local through demand changes and capture format re-arm } finally { subscriber?.close(); siblingSubscriber?.close(); - catalogSubscriber?.close(); + catalog?.close(); recovered?.close(); encoder.close(); bandwidth.close(); @@ -894,7 +893,7 @@ test("an audio failure without demand still recovers under a fresh track name", const broadcast = new Broadcast({ origin: new Moq.Origin.Producer(), name: Moq.Path.from("idle.hang") }); const configuring = configured(); const encoder = new Encoder("audio", { broadcast, capture: fakeCapture(new Feed()) as never }); - let catalogSubscriber: Moq.Track.Subscriber | undefined; + let catalog: Json.Snapshot.Consumer | undefined; try { await configuring; await settle(); @@ -914,15 +913,14 @@ test("an audio failure without demand still recovers under a fresh track name", encoder.codec.set({ mime: "opus", bitrate: 64_000 }); await reconfiguring; await settle(); - catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: catalogSubscriber }); + catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); const config = (await catalog.latest())?.value.audio?.renditions.audio; expect(config?.track).toBeDefined(); expect(config?.track).not.toBe("audio"); expect(encoder.out.error.peek()).toBeUndefined(); expect(broadcast.net.peek()).toBe(net); } finally { - catalogSubscriber?.close(); + catalog?.close(); encoder.close(); broadcast.close(); } @@ -940,7 +938,7 @@ test.each(["failure first", "config first"])( }); const configuring = configured(); const encoder = new Encoder("audio", { broadcast, capture: fakeCapture(new Feed()) as never }); - let catalogSubscriber: Moq.Track.Subscriber | undefined; + let catalog: Json.Snapshot.Consumer | undefined; try { await configuring; await settle(); @@ -954,8 +952,7 @@ test.each(["failure first", "config first"])( if (order === "failure first") encoder.codec.set({ mime: "opus", bitrate: 64_000 }); await reconfiguring; await settle(); - catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: catalogSubscriber }); + catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); const config = (await catalog.latest())?.value.audio?.renditions.audio; expect(config?.track).toBeDefined(); expect(config?.track).not.toBe("audio"); @@ -964,7 +961,7 @@ test.each(["failure first", "config first"])( expect(encoder.out.active.peek()).toBe(true); expect(broadcast.net.peek()).toBe(net); } finally { - catalogSubscriber?.close(); + catalog?.close(); encoder.close(); broadcast.close(); } diff --git a/js/publish/src/broadcast.test.ts b/js/publish/src/broadcast.test.ts index 76a47471b93..01f3728a2e5 100644 --- a/js/publish/src/broadcast.test.ts +++ b/js/publish/src/broadcast.test.ts @@ -20,7 +20,7 @@ async function readCatalog(broadcast: Broadcast): Promise({ track: track.subscribe() }).latest())?.value; + const catalog = (await new Json.Snapshot.Consumer({ track }).latest())?.value; effect.close(); return catalog; } @@ -214,12 +214,12 @@ test("serves the catalog through a shared static track", async () => { const net = broadcast.net.peek(); if (!net) throw new Error("expected a network producer once connected"); - const subscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = (await new Json.Snapshot.Consumer({ track: subscriber }).latest())?.value; + const reader = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); + const catalog = (await reader.latest())?.value; expect(catalog?.video?.renditions.video?.codec).toBe("avc1.640028"); // Dropping the subscriber closes the served track; the broadcast keeps running for the next viewer. - subscriber.close(); + reader.close(); await settle(); expect(broadcast.net.peek()).toBe(net); @@ -439,15 +439,15 @@ test("keeps the current catalog snapshot for a reconnecting viewer", async () => const net = broadcast.net.peek(); if (!net) throw new Error("expected a network producer once connected"); - const first = net.track(Broadcast.CATALOG_TRACK).subscribe(); - expect((await new Json.Snapshot.Consumer({ track: first }).latest())?.value.video).toBeDefined(); + const first = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); + expect((await first.latest())?.value.video).toBeDefined(); first.close(); // A reconnect past the idle cache window must still receive the live track's newest // snapshot instead of waiting forever for an edit that may never come. await clock.advance(Nano.fromMilli(Milli(60_000))); - const second = net.track(Broadcast.CATALOG_TRACK).subscribe(); - expect((await new Json.Snapshot.Consumer({ track: second }).latest())?.value.video).toBeDefined(); + const second = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); + expect((await second.latest())?.value.video).toBeDefined(); second.close(); broadcast.close(); @@ -500,7 +500,7 @@ test("a served decoder identity change moves the rendition to a fresh track name const announced = broadcast.in.origin.peek()?.follow(broadcast.in.name.peek()); const instance = announced?.active.peek()?.instance; const catalog = new Json.Snapshot.Consumer({ - track: net.track(Broadcast.CATALOG_TRACK).subscribe(), + track: net.track(Broadcast.CATALOG_TRACK), }); const first = net.track("video").subscribe(); try { @@ -749,7 +749,7 @@ test("identity changes before the first announce take a fresh track name", async const net = broadcast.net.peek(); if (!net) throw new Error("expected a network producer once connected"); const catalog = new Json.Snapshot.Consumer({ - track: net.track(Broadcast.CATALOG_TRACK).subscribe(), + track: net.track(Broadcast.CATALOG_TRACK), }); const config = (await catalog.latest())?.value.video?.renditions.video; expect(config?.codec).toBe("vp8"); @@ -791,9 +791,8 @@ test("stats names a track in the catalog and reports each listed rendition while const net = broadcast.net.peek(); if (!net) throw new Error("expected a network producer once connected"); - const subscriber = net.track(Broadcast.STATS_TRACK).subscribe(); const consumer = new Json.Snapshot.Consumer({ - track: subscriber, + track: net.track(Broadcast.STATS_TRACK), schema: Stats.SnapshotSchema, }); await settle(); @@ -803,7 +802,7 @@ test("stats names a track in the catalog and reports each listed rendition while transport: { rtt: 12 }, renditions: { video: { frames: 3, bytes: 300, keyframes: 1, skipped: 2, bitrate: 1_000_000 } }, }); - subscriber.close(); + consumer.close(); stats.set(false); await settle(); diff --git a/js/publish/src/catalog.test.ts b/js/publish/src/catalog.test.ts index 4a3b2965df5..62f4b320495 100644 --- a/js/publish/src/catalog.test.ts +++ b/js/publish/src/catalog.test.ts @@ -17,7 +17,7 @@ test("catalog producer seeds subscribers and fans out edits", async () => { const effect = new Effect(); const track = new Track.Producer("catalog.json"); catalog.serve(track, effect); - const consumer = new Json.Snapshot.Consumer({ track: track.subscribe() }); + const consumer = new Json.Snapshot.Consumer({ track }); // A new subscriber is seeded with the current catalog. expect((await consumer.latest())?.value.video).toEqual({ renditions: {} }); @@ -57,7 +57,7 @@ test("catalog producer computes the version from the fields it uses", async () = const effect = new Effect(); const track = new Track.Producer("catalog.json"); catalog.serve(track, effect); - const consumer = new Json.Snapshot.Consumer({ track: track.subscribe() }); + const consumer = new Json.Snapshot.Consumer({ track }); const video = { codec: "avc1.64001f", container: { kind: "legacy" as const } }; // A caller's own version is replaced by the computed one. @@ -116,7 +116,7 @@ test("catalog producer advertises the page clock from the first snapshot", async const effect = new Effect(); const track = new Track.Producer("catalog.json"); catalog.serve(track, effect); - const consumer = new Json.Snapshot.Consumer({ track: track.subscribe() }); + const consumer = new Json.Snapshot.Consumer({ track }); // Before any rendition: a live-only publisher exposes its clock. const first = Catalog.RootSchema.parse((await consumer.latest())?.value); @@ -179,7 +179,7 @@ test("a reconnecting subscriber is seeded with the full current catalog", async const effect = new Effect(); const track = new Track.Producer("catalog.json"); catalog.serve(track, effect); - const seeded = (await new Json.Snapshot.Consumer({ track: track.subscribe() }).latest())?.value; + const seeded = (await new Json.Snapshot.Consumer({ track }).latest())?.value; expect(seeded?.video).toEqual({ renditions: {} }); expect(seeded?.scte35).toEqual({ splices: [] }); @@ -351,7 +351,7 @@ describe("estimate publication window", () => { scopes.push(effect); const track = new Track.Producer("catalog.json"); producer.serve(track, effect); - const consumer = new Json.Snapshot.Consumer({ track: track.subscribe() }); + const consumer = new Json.Snapshot.Consumer({ track }); const estimate = (jitter: number) => producer.mutate((catalog) => { if (!catalog.video) throw new Error("video missing"); @@ -440,7 +440,7 @@ describe("estimate publication window", () => { scopes.push(again); const track = new Track.Producer("catalog.json"); producer.serve(track, again); - const reader = new Json.Snapshot.Consumer({ track: track.subscribe() }); + const reader = new Json.Snapshot.Consumer({ track }); await reader.latest(); estimate(3); expect((await reader.latest())?.value.video?.renditions.v.jitter).toBe(Catalog.u53(3)); diff --git a/js/publish/src/video/encoder.test.ts b/js/publish/src/video/encoder.test.ts index 078ce712714..f4871e20eea 100644 --- a/js/publish/src/video/encoder.test.ts +++ b/js/publish/src/video/encoder.test.ts @@ -1331,16 +1331,16 @@ test("an on-demand all-keyframe rendition advertises its records, whose subscrib capture: recording.capture as never, config: { keyframeInterval: Milli(0) }, }); - const subscribers: Moq.Track.Subscriber[] = []; + const subscribers: { close(): void }[] = []; try { await settle(); const net = broadcast.net.peek(); if (!net) throw new Error("expected network broadcast"); expect(encoder.out.active.peek()).toBe(false); - const catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - subscribers.push(catalogSubscriber); - const catalog = await new Json.Snapshot.Consumer({ track: catalogSubscriber }).latest(); + const reader = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); + subscribers.push(reader); + const catalog = await reader.latest(); expect(catalog?.value.video?.renditions.video?.hls).toEqual({ track: "video.hls", duration: Catalog.u53(4000), @@ -1367,7 +1367,7 @@ async function liveInterval(raised: Milli): Promise<{ keys: number[]; hls?: Cata }); const config = new Signal({ keyframeInterval: Milli(2000) }); const encoder = new Encoder("video", { enabled: true, broadcast, capture: recording.capture as never, config }); - const subscribers: Moq.Track.Subscriber[] = []; + const subscribers: { close(): void }[] = []; try { await settle(); for (let ms = 0; ms < 16000; ms += 100) { @@ -1376,9 +1376,9 @@ async function liveInterval(raised: Milli): Promise<{ keys: number[]; hls?: Cata } const net = broadcast.net.peek(); if (!net) throw new Error("expected network broadcast"); - const catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - subscribers.push(catalogSubscriber); - const catalog = await new Json.Snapshot.Consumer({ track: catalogSubscriber }).latest(); + const reader = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); + subscribers.push(reader); + const catalog = await reader.latest(); return { keys: [...recording.keys], hls: catalog?.value.video?.renditions.video?.hls }; } finally { for (const subscriber of subscribers) subscriber.close(); @@ -1456,7 +1456,7 @@ test("a failed video encoder disables only its rendition and recovers on a fresh const encoder = new Encoder("video/hd", { broadcast, capture: capture as never, bandwidth, enabled, config: {} }); let subscriber: Moq.Track.Subscriber | undefined; let siblingSubscriber: Moq.Track.Subscriber | undefined; - let catalogSubscriber: Moq.Track.Subscriber | undefined; + let catalog: Json.Snapshot.Consumer | undefined; let recovered: Moq.Track.Subscriber | undefined; let announced: Moq.Origin.Following | undefined; try { @@ -1476,8 +1476,7 @@ test("a failed video encoder disables only its rendition and recovers on a fresh const siblingTrack = sibling.track.peek(); if (!siblingTrack) throw new Error("expected demanded sibling track"); expect(siblingTrack.appendGroup().sequence).toBe(0n); - catalogSubscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: catalogSubscriber }); + catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); expect((await catalog.latest())?.value.video?.renditions["video/hd"]?.enabled).not.toBe(false); const live = FakeVideoEncoder.instances.find((instance) => instance.state === "configured"); if (!live) throw new Error("expected live video encoder"); @@ -1539,7 +1538,7 @@ test("a failed video encoder disables only its rendition and recovers on a fresh } finally { subscriber?.close(); siblingSubscriber?.close(); - catalogSubscriber?.close(); + catalog?.close(); recovered?.close(); encoder.close(); bandwidth.close(); @@ -2035,8 +2034,7 @@ test("ceiling growth updates the served track in place", async () => { const encoder = new Encoder("video", { broadcast, capture: capture as never, config: { frameRate: 60 } }); const net = broadcast.net.peek() ?? (await broadcast.net.changed()); if (!net) throw new Error("expected a network broadcast"); - const subscriber = net.track(Broadcast.CATALOG_TRACK).subscribe(); - const catalog = new Json.Snapshot.Consumer({ track: subscriber }); + const catalog = new Json.Snapshot.Consumer({ track: net.track(Broadcast.CATALOG_TRACK) }); const next = async (done: (config: Catalog.VideoConfig) => boolean) => { for (;;) { const config = (await catalog.latest())?.value.video?.renditions.video; @@ -2060,7 +2058,7 @@ test("ceiling growth updates the served track in place", async () => { expect(grown.track).toBeUndefined(); expect(broadcast.net.peek()).toBe(net); } finally { - subscriber.close(); + catalog.close(); encoder.close(); broadcast.close(); } diff --git a/js/room/src/metadata.test.ts b/js/room/src/metadata.test.ts index 040db2017bd..6fbf32ef956 100644 --- a/js/room/src/metadata.test.ts +++ b/js/room/src/metadata.test.ts @@ -30,13 +30,12 @@ test("metadata updates keep the same track and reach an existing subscriber", as try { serve(broadcast, user, new Signal({}), effect); await flush(); - const track = net.consume().track(TRACK.user).subscribe(); - const consumer = new Json.Snapshot.Consumer<{ name: string }>({ track }); + const consumer = new Json.Snapshot.Consumer<{ name: string }>({ track: net.consume().track(TRACK.user) }); expect((await consumer.latest())?.value.name).toBe("Alice"); user.name.set("Bob"); await flush(); expect((await consumer.latest())?.value.name).toBe("Bob"); - track.close(); + consumer.close(); } finally { effect.close(); net.close(); diff --git a/js/room/src/metadata.ts b/js/room/src/metadata.ts index 9ad2a3a4c42..d688abfd547 100644 --- a/js/room/src/metadata.ts +++ b/js/room/src/metadata.ts @@ -267,10 +267,9 @@ function subscribeJson( effect: Effect, update: (value: T) => void, ): void { - const track = broadcast.track(name).subscribe({ ...Moq.Track.Subscription.default(), priority: PRIORITY }); - effect.cleanup(() => track.close()); + const consumer = new Json.Snapshot.Consumer({ track: broadcast.track(name), schema, priority: PRIORITY }); + effect.cleanup(() => consumer.close()); - const consumer = new Json.Snapshot.Consumer({ track, schema }); effect.spawn(async () => { for (;;) { let state: Moq.Timed | undefined; diff --git a/js/watch/src/broadcast.ts b/js/watch/src/broadcast.ts index 7afc89bd87e..986e4de0626 100644 --- a/js/watch/src/broadcast.ts +++ b/js/watch/src/broadcast.ts @@ -383,20 +383,23 @@ export class Broadcast { this.#out.status.set("loading"); - const trackName = format === "hang" ? Catalog.TRACK : "catalog"; - const track = broadcast.track(trackName).subscribe({ - ...Track.Subscription.default(), - priority: Catalog.PRIORITY.catalog, - }); - effect.cleanup(() => track.close()); - // The hang catalog is reconstructed from snapshots (and future deltas) via @moq/json, which reads - // it compressed or not; MSF stays on its own one-blob-per-group fetch. + // it compressed or not and subscribes itself; MSF stays on its own one-blob-per-group fetch. let fetchNext: () => Promise; if (format === "hang") { - const consumer = new Json.Snapshot.Consumer({ track, schema: Catalog.RootSchema }); + const consumer = new Json.Snapshot.Consumer({ + track: broadcast.track(Catalog.TRACK), + schema: Catalog.RootSchema, + priority: Catalog.PRIORITY.catalog, + }); + effect.cleanup(() => consumer.close()); fetchNext = async () => (await consumer.latest())?.value; } else { + const track = broadcast.track("catalog").subscribe({ + ...Track.Subscription.default(), + priority: Catalog.PRIORITY.catalog, + }); + effect.cleanup(() => track.close()); const ordered = track.ordered(); fetchNext = async () => { const update = await Msf.fetch(ordered); diff --git a/js/watch/src/echo.test.ts b/js/watch/src/echo.test.ts index 9a8b4df402b..cfde2bd9a68 100644 --- a/js/watch/src/echo.test.ts +++ b/js/watch/src/echo.test.ts @@ -44,12 +44,11 @@ async function read(origin: Origin.Producer, path: string, clock: Manual): Promi await flush(); const broadcast = request.active.peek(); if (!broadcast) throw new Error(`no feedback at ${path}`); - const track = broadcast.track(Schema.TRACK).subscribe(); - const consumer = new Json.Snapshot.Consumer({ track, schema: Schema.SnapshotSchema }); + const consumer = new Json.Snapshot.Consumer({ track: broadcast.track(Schema.TRACK), schema: Schema.SnapshotSchema }); await flush(); await clock.advance(Nano.fromMilli(Milli(1000))); const value = (await consumer.latest())?.value; - track.close(); + consumer.close(); request.close(); return value; } diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index ce334268d26..dcd8fcea5c4 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,9 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`SnapshotConsumer(broadcast, name)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### `VideoInit.lower` takes either producer kind ([#7699](https://github.com/moq-dev/moq/pull/7699)) @@ -40,7 +40,7 @@ A fixed audio playout delay `d` is `delay = Delay(min = d, max = ...)` with `est `dev.moq.flate` is removed: publish opaque bytes on a `TrackProducer`. `SnapshotConfig` and `WindowConfig` lose `compression`, and `StreamConfig` is removed. Publish the track with `TrackInfo(compress = true)` instead. -`StreamProducer` drops its config argument, and `SnapshotConsumer`, `StreamConsumer`, and `WindowConsumer` take only the track. +`StreamProducer` drops its config argument, and `StreamConsumer` and `WindowConsumer` take only the track. `SnapshotConsumer` takes the broadcast and track name; see the entry above. ### Paths with an empty segment are refused ([#6648](https://github.com/moq-dev/moq/pull/6648)) diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt index b0e55d28c4e..24a6f4c8164 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt @@ -12,7 +12,7 @@ package dev.moq.json // val track = broadcast.publishTrack("status", TrackInfo(timescale = 1_000uL, compress = true)) // val status = SnapshotProducer(broadcast, track, SnapshotConfig()) // status.update(Status(state = "live"), CatalogProducer(broadcast).now()) -// val reader = SnapshotConsumer(consumer.subscribeTrack("status", null)) +// val reader = SnapshotConsumer(consumer, "status") import dev.moq.MoqJson import dev.moq.Timestamp @@ -24,7 +24,7 @@ import kotlinx.serialization.serializer /** Publishes lossy latest-value JSON snapshots on a track it takes over. */ typealias SnapshotProducer = uniffi.moq.MoqJsonSnapshotProducer -/** Consumes reconstructed latest-value JSON snapshots from a track it takes over. */ +/** Consumes reconstructed latest-value JSON snapshots from a track it subscribes to itself. */ typealias SnapshotConsumer = uniffi.moq.MoqJsonSnapshotConsumer /** Publishes a lossless stream of JSON records on a track it takes over. */ @@ -100,7 +100,7 @@ inline fun WindowEventPush.decode(): T = /** * Stream of each JSON state from a snapshot track, in order. * - * Skips every group a newer one supersedes, whatever the subscription's `delay.max`. Call + * Skips every group a newer one supersedes, at the zero `delay.max` the consumer subscribes with. Call * [SnapshotConsumer.latest] to skip to the newest state. */ fun SnapshotConsumer.values(): Flow = pull({ cancel() }) { next() } diff --git a/kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt b/kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt index 4bf978bed2d..871c3ef2084 100644 --- a/kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt +++ b/kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt @@ -514,7 +514,7 @@ class SmokeTest { val config = SnapshotConfig(deltas = true) val track = broadcast.publishTrack("status", TrackInfo(timescale = 1_000_000uL)) SnapshotProducer(broadcast, track, config).use { producer -> - val consumer = SnapshotConsumer(broadcast.consume().subscribeTrack("status", null)) + val consumer = SnapshotConsumer(broadcast.consume(), "status") for (n in 1..4) producer.update( Status(state = "live$n"), timestamp = n.milliseconds.toTimestamp(), @@ -584,7 +584,7 @@ class SmokeTest { assertEquals("status", demand.name()) assertEquals(null, demand.wanted()) - val consumer = SnapshotConsumer(broadcast.consume().subscribeTrack("status", null)) + val consumer = SnapshotConsumer(broadcast.consume(), "status") while (demand.wantedChanged() == null) {} consumer.cancel() while (demand.wantedChanged() != null) {} @@ -602,7 +602,7 @@ class SmokeTest { val demand: TrackDemand = producer.demand() producer.update(Status(state = "live"), Timestamp(1uL, 1_000uL)) - val consumer = SnapshotConsumer(broadcast.consume().subscribeTrack("status", null)) + val consumer = SnapshotConsumer(broadcast.consume(), "status") while (demand.wantedChanged() == null) {} // `first` cancels the collection, which must release the subscription without an // explicit cancel. @@ -624,7 +624,7 @@ class SmokeTest { SnapshotProducer(broadcast, track, config).use { producer -> producer.update("""{"state":"raw"}""", Timestamp(1uL, 1_000uL)) - val consumer = SnapshotConsumer(broadcast.consume().subscribeTrack("status", null)) + val consumer = SnapshotConsumer(broadcast.consume(), "status") assertEquals( Timed(Status(state = "raw"), Timestamp(1uL, 1_000uL)), consumer.valuesAs().first(), diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 01a6374686c..73635f21fd2 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,9 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`moq.json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`moq.json.SnapshotConsumer(broadcast, name)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribe_track` call; a refused subscription now raises from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### `VideoInit.lower` takes either producer kind ([#7699](https://github.com/moq-dev/moq/pull/7699)) diff --git a/py/moq-rs/moq/json.py b/py/moq-rs/moq/json.py index cceae016820..0b4c157b6ea 100644 --- a/py/moq-rs/moq/json.py +++ b/py/moq-rs/moq/json.py @@ -134,13 +134,14 @@ def finish(self) -> None: class SnapshotConsumer(_Stream[Timed]): """Async iterator over a JSON snapshot track, yielding each state in order (lossy). - Each item is a :class:`Timed`. Every group a newer one supersedes is skipped, whatever the - subscription's ``delay.max``; :meth:`latest` skips to the newest state instead. Usable as an - async context manager that cancels on exit. + Each item is a :class:`Timed`. Every group a newer one supersedes is skipped; :meth:`latest` + skips to the newest state instead. The consumer subscribes to the track ``name`` on + ``broadcast`` itself, at a zero ``delay.max``; a read waits for that subscription and raises if + it fails. Usable as an async context manager that cancels on exit. """ - def __init__(self, track: subscribe.TrackConsumer) -> None: - self._inner = MoqJsonSnapshotConsumer(track._inner) + def __init__(self, broadcast: subscribe.BroadcastConsumer, name: str) -> None: + self._inner = MoqJsonSnapshotConsumer(broadcast._inner, name) async def _next(self) -> Timed | None: return Timed._from_ffi(await self._inner.next()) diff --git a/py/moq-rs/tests/test_local.py b/py/moq-rs/tests/test_local.py index bcf801290ca..560f229adfe 100644 --- a/py/moq-rs/tests/test_local.py +++ b/py/moq-rs/tests/test_local.py @@ -537,7 +537,7 @@ async def test_json_snapshot_roundtrip(): broadcast = moq.BroadcastProducer() track = broadcast.publish_track("status", moq.TrackInfo(timescale=1_000_000, compress=True)) producer = moq.json.SnapshotProducer(broadcast, track) - consumer = moq.json.SnapshotConsumer(await broadcast.consume().subscribe_track("status")) + consumer = moq.json.SnapshotConsumer(broadcast.consume(), "status") for viewers in range(1, 5): producer.update({"viewers": viewers}, moq.Timestamp(viewers, 1_000)) @@ -664,7 +664,7 @@ async def test_json_producers_report_demand(): assert snapshot_demand.wanted() is None consumer = broadcast.consume() - snapshot_consumer = moq.json.SnapshotConsumer(await consumer.subscribe_track("status")) + snapshot_consumer = moq.json.SnapshotConsumer(consumer, "status") stream_consumer = moq.json.StreamConsumer(await consumer.subscribe_track("events")) await asyncio.wait_for(until_wanted(snapshot_demand), timeout=5.0) await asyncio.wait_for(until_wanted(stream_demand), timeout=5.0) @@ -1783,7 +1783,7 @@ async def assert_cancelled(awaitable) -> None: await assert_cancelled(track_dynamic.requested_fill()) snapshot = moq.json.SnapshotProducer(broadcast, broadcast.publish_track("state", moq.TrackInfo(timescale=1_000))) - async with moq.json.SnapshotConsumer(await broadcast.consume().subscribe_track("state")) as snapshot_consumer: + async with moq.json.SnapshotConsumer(broadcast.consume(), "state") as snapshot_consumer: pass await assert_cancelled(anext(snapshot_consumer)) diff --git a/quest/m2/net-wrap/json.md b/quest/m2/net-wrap/json.md index b0190226c9b..1381100d68a 100644 --- a/quest/m2/net-wrap/json.md +++ b/quest/m2/net-wrap/json.md @@ -12,9 +12,10 @@ types, so each publishes into and reads from a wrapped track unchanged. - moq-json takes the trait, not a wrapper role (decided 2026-10-20, maintainer): its `Encoder`/`Decoder` codec layer stays as it is, and only - `Producer::new(track::Producer)` and `Consumer::new(track::Subscriber)` in - `snapshot`, `stream`, and `window` become generic. -- hang: `catalog::Consumer::new(track::Subscriber)`, + `Producer::new(track::Producer)` in `snapshot`, `stream`, and `window`, + `snapshot::Consumer::new(track::Consumer)`, which subscribes itself, and + `Consumer::new(track::Subscriber)` in `stream` and `window` become generic. +- hang: `catalog::Consumer::new(track::Consumer)`, `Root::subscribe(&broadcast::Consumer)`, and `container::Frame::write_to(&mut group::Producer)` are the entry points. - moq-stats: the produce and consume paths that hold moq-net broadcasts and diff --git a/rs/hang/MIGRATE.md b/rs/hang/MIGRATE.md index 9a00754039f..52ae69c5146 100644 --- a/rs/hang/MIGRATE.md +++ b/rs/hang/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`catalog::Consumer::new` takes the catalog track, a `moq_net::track::Consumer`, instead of a `track::Subscriber`, and subscribes to it itself at `Catalog::default_subscription()`'s priority and a zero `delay.max`, so `latest` never waits on a catalog group the track still owes. +Pass `broadcast.track(Catalog::DEFAULT_NAME)?` rather than subscribing first; `Catalog::subscribe` is unchanged. +A refused subscription now surfaces from the first read. + ### Compression moves to the track `catalog::Compression`, the `compression` field on `JsonConfig` and `BinaryConfig`, `Catalog::COMPRESSED_NAME`, and `catalog::Consumer::compressed` are gone. diff --git a/rs/hang/examples/subscribe.rs b/rs/hang/examples/subscribe.rs index 9718326d08f..5445d1863b2 100644 --- a/rs/hang/examples/subscribe.rs +++ b/rs/hang/examples/subscribe.rs @@ -80,11 +80,7 @@ async fn play(consumer: moq_net::origin::Consumer, announce: moq_net::announce:: let broadcast = consumer.request(&announce.prefix, announce.route.instance).await?; // Read the catalog to discover available tracks. - let catalog_track = broadcast - .track(hang::Catalog::DEFAULT_NAME)? - .subscribe(hang::Catalog::default_subscription()) - .await?; - let mut catalog = moq_mux::catalog::hang::Consumer::<()>::new(catalog_track); + let mut catalog = moq_mux::catalog::hang::Consumer::<()>::new(broadcast.track(hang::Catalog::DEFAULT_NAME)?); let info = catalog.next().await?.ok_or_else(|| anyhow::anyhow!("no catalog"))?; diff --git a/rs/hang/src/catalog/consumer.rs b/rs/hang/src/catalog/consumer.rs index 67c9c722cc5..eebd5b48726 100644 --- a/rs/hang/src/catalog/consumer.rs +++ b/rs/hang/src/catalog/consumer.rs @@ -8,7 +8,7 @@ use serde::de::DeserializeOwned; use super::Catalog; use crate::Result; -/// Reads typed catalog roots from an already subscribed catalog track. +/// Reads typed catalog roots from a catalog track, subscribing to it at catalog priority. pub struct Consumer { // Decoded in two steps, so a catalog needing a newer reader fails with a typed error. inner: moq_json::snapshot::Consumer, @@ -17,11 +17,16 @@ pub struct Consumer { } impl Consumer { - /// Wrap a catalog track subscription, compressed or not. - pub fn new(track: moq_net::track::Subscriber) -> Self { + /// Read a catalog track, compressed or not. + /// + /// Subscribes to it at [`Catalog::default_subscription`]'s priority, reading only the newest + /// catalog; see [`moq_json::snapshot::Consumer::new`]. + pub fn new(track: moq_net::track::Consumer) -> Self { + let mut config = moq_json::snapshot::consumer::Config::default(); + config.priority = Catalog::default_subscription().priority; Self { base: track.broadcast().path.clone(), - inner: moq_json::snapshot::Consumer::new(track, Default::default()), + inner: moq_json::snapshot::Consumer::new(track, config), _ext: std::marker::PhantomData, } } diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 205620600e7..b71f3c31701 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -343,11 +343,7 @@ impl FromStr for Catalog { impl Catalog { /// Subscribe to catalog updates on a broadcast. pub async fn subscribe(broadcast: &moq_net::broadcast::Consumer) -> Result> { - let track = broadcast - .track(Catalog::DEFAULT_NAME)? - .subscribe(Catalog::default_subscription()) - .await?; - Ok(super::Consumer::new(track)) + Ok(super::Consumer::new(broadcast.track(Catalog::DEFAULT_NAME)?)) } /// Parse a catalog from a slice of bytes. diff --git a/rs/moq-boy/src/input.rs b/rs/moq-boy/src/input.rs index b7c954e7acc..606a2359174 100644 --- a/rs/moq-boy/src/input.rs +++ b/rs/moq-boy/src/input.rs @@ -127,7 +127,7 @@ async fn handle_viewer_commands( broadcast: moq_net::broadcast::Consumer, cmd_tx: &tokio::sync::mpsc::Sender, ) -> anyhow::Result<()> { - let track = broadcast.track("command")?.subscribe(None).await?; + let track = broadcast.track("command")?; let mut commands = moq_json::snapshot::Consumer::::new(track, moq_json::snapshot::consumer::Config::default()); diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index 2d628601c1d..b858066ba96 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -925,20 +925,19 @@ impl Consume { let broadcast = context.broadcast.clone(); let name = name.to_string(); - // `subscribe` blocks on SUBSCRIBE_OK, so run it inside the task. + // Reads wait on SUBSCRIBE_OK, so run them inside the task. self.json_task.spawn( on_value, |state| &mut state.consume.json_task, - |mut close| async move { + |close| async move { driver .watch(async move { - let Some(track) = - Self::until_closed(&mut close, Self::subscribe(&broadcast, &name, None)).await? - else { - return Ok(()); - }; - let consumer = - moq_json::snapshot::Consumer::::new(track, Default::default()); + // The consumer subscribes itself; `run_json_snapshot` keeps a close ahead of + // a SUBSCRIBE_OK that never comes. + let consumer = moq_json::snapshot::Consumer::::new( + broadcast.track(&name)?, + Default::default(), + ); Self::run_json_snapshot(on_value, consumer, close).await }) .await diff --git a/rs/moq-cli/src/report.rs b/rs/moq-cli/src/report.rs index f9ff59ff663..310d3ee519c 100644 --- a/rs/moq-cli/src/report.rs +++ b/rs/moq-cli/src/report.rs @@ -393,10 +393,10 @@ async fn next(catalog: &mut Option) -> moq_mux::Result moq_json::snapshot::Consumer { +fn reader(track: moq_net::track::Consumer) -> moq_json::snapshot::Consumer { let mut config = moq_json::snapshot::consumer::Config::default(); config.max_size = Some(MAX_REPORT); - moq_json::snapshot::Consumer::new(subscriber, config) + moq_json::snapshot::Consumer::new(track, config) } /// The next snapshot of a report track, or never while there is none. @@ -437,10 +437,7 @@ pub async fn print_report(origin: moq_net::origin::Consumer, path: String) -> an let stats = update.stats.map(|stats| stats.track); if stats != named { track = match &stats { - Some(name) => { - let subscriber = broadcast.track(name)?.subscribe(None).await?; - Some(reader(subscriber)) - } + Some(name) => Some(reader(broadcast.track(name)?)), None => None, }; named = stats; @@ -567,8 +564,7 @@ impl Readers { async fn read_echo(origin: moq_net::origin::Consumer, path: moq_net::Path, lines: tokio::sync::mpsc::Sender) { let read = async { let broadcast = origin.request(&path, None).await?; - let subscriber = broadcast.track(hang::echo::TRACK)?.subscribe(None).await?; - let mut track = reader(subscriber); + let mut track = reader(broadcast.track(hang::echo::TRACK)?); while let Some(moq_net::Timed { value: feedback, .. }) = track.latest().await? { let line = EchoLine { broadcast: path.as_str(), @@ -601,7 +597,7 @@ mod tests { .produce() .create_track(STATS_TRACK, None) .unwrap(); - let mut read = reader(track.subscribe(None)); + let mut read = reader(track.consume()); let mut report = moq_json::snapshot::Producer::::new(track, Default::default()); report .update(&serde_json::json!({ "pad": "x".repeat(MAX_REPORT) })) diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index 16c160d0c05..8b035ef5f9e 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,6 +5,10 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`MoqJsonSnapshotConsumer::new(broadcast, name)` takes a `MoqBroadcastConsumer` and the track name instead of a `MoqTrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribe_track` call; a refused subscription now fails the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. + ### `MoqVideoInit.lower` takes either producer kind ([#7699](https://github.com/moq-dev/moq/pull/7699)) `MoqVideoInit.lower` is `Option` instead of `Option`. Wrap a frame producer as `MoqMediaProducer::Track { producer }`; a `MoqMediaTrackStreamProducer` goes in as `MoqMediaProducer::Stream { producer }`. diff --git a/rs/moq-ffi/src/json.rs b/rs/moq-ffi/src/json.rs index cf3d26e058f..86ca7d7438c 100644 --- a/rs/moq-ffi/src/json.rs +++ b/rs/moq-ffi/src/json.rs @@ -12,7 +12,7 @@ use std::task::{Poll, ready}; use serde_json::Value; -use crate::consumer::MoqTrackConsumer; +use crate::consumer::{MoqBroadcastConsumer, MoqTrackConsumer}; use crate::demand::MoqTrackDemand; use crate::error::MoqError; use crate::ffi::Polled; @@ -228,23 +228,23 @@ pub struct MoqJsonSnapshotConsumer { #[uniffi::export] impl MoqJsonSnapshotConsumer { - /// Read `track` as a JSON snapshot track (lossy latest-value). + /// Subscribe to the track `name` on `broadcast` and read it as a JSON snapshot track (lossy + /// latest-value). /// - /// Reads a compressed track the same as a plain one. Takes over `track`, whose handle is - /// closed afterward; errors if it has already read a group. + /// The consumer owns its subscription, at a zero `delay.max`: a read waits for it to resolve and + /// fails if it cannot. Reads a compressed track the same as a plain one. #[uniffi::constructor] - pub fn new(track: &MoqTrackConsumer) -> Result, MoqError> { + pub fn new(broadcast: &MoqBroadcastConsumer, name: String) -> Result, MoqError> { let _guard = crate::ffi::enter(); - let (track, driver) = track.take()?; - let consumer = moq_json::snapshot::Consumer::::new(track, Default::default()); + let consumer = moq_json::snapshot::Consumer::::new(broadcast.inner.track(&name)?, Default::default()); Ok(Arc::new(Self { - reads: Polled::new(consumer).supervised(driver), + reads: Polled::new(consumer).supervised(broadcast.driver.clone()), })) } /// Get the next state in order. Returns `None` once the track ends. /// - /// Skips every group a newer one supersedes, whatever the subscription's `delay.max`. + /// Skips every group a newer one supersedes. pub async fn next(&self) -> Result, MoqError> { let value = self .reads diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index c279aa5e878..07ccd6fcb9a 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -1724,17 +1724,7 @@ impl MoqMediaCatalogConsumer { /// Subscribe to this broadcast's catalog snapshots. #[uniffi::constructor] pub async fn subscribe(broadcast: &MoqBroadcastConsumer) -> Result, MoqError> { - let track = broadcast - .driver - .watch(async { - Ok(broadcast - .inner - .track(hang::catalog::Catalog::DEFAULT_NAME)? - .subscribe(hang::catalog::Catalog::default_subscription()) - .await?) - }) - .await?; - let consumer = moq_mux::catalog::hang::Consumer::from(track); + let consumer = moq_mux::catalog::hang::Consumer::new(broadcast.inner.track(hang::catalog::Catalog::DEFAULT_NAME)?); Ok(Arc::new(MoqMediaCatalogConsumer { reads: Polled::new(consumer).supervised(broadcast.driver.clone()), })) @@ -2076,9 +2066,6 @@ mod timebase_tests { .consume_inner() .unwrap() .track(hang::catalog::Catalog::DEFAULT_NAME) - .unwrap() - .subscribe(moq_net::track::Subscription::default()) - .await .unwrap(); let mut catalog = moq_mux::catalog::hang::Consumer::::new(track); assert!(catalog.poll_next(&kio::Waiter::noop()).is_pending()); @@ -2104,9 +2091,6 @@ mod timebase_tests { .consume_inner() .unwrap() .track(hang::catalog::Catalog::DEFAULT_NAME) - .unwrap() - .subscribe(moq_net::track::Subscription::default()) - .await .unwrap(); let mut reader = moq_mux::catalog::hang::Consumer::::new(track); let catalog = MoqMediaCatalogProducer::new(owner.clone()).unwrap(); diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index f4fc78b0371..a4a65aab6b0 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -1464,7 +1464,7 @@ async fn json_snapshot_roundtrip() { matches!(track.demand(), Err(MoqError::Closed)), "the producer takes over the track" ); - let consumer = MoqJsonSnapshotConsumer::new(&*subscribe(&broadcast, "meta").await).unwrap(); + let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into()).unwrap(); for n in 1..=4 { producer.update(format!(r#"{{"a":{n}}}"#), us(n * 1000)).unwrap(); @@ -1553,7 +1553,7 @@ async fn json_snapshot_refreshes_for_a_joiner() { .await .expect("timed out waiting for the refresh"); assert_eq!(refreshed.sequence(), 1); - let consumer = MoqJsonSnapshotConsumer::new(&*subscribe(&broadcast, "meta").await).unwrap(); + let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into()).unwrap(); let value = kio::clock::timeout(TIMEOUT, consumer.next()) .await .expect("timed out waiting for the restated value") @@ -1582,7 +1582,7 @@ async fn json_demand() { assert_eq!(stream_demand.name(), "events"); assert!(!snapshot_demand.wanted().is_some()); - let snapshot_consumer = MoqJsonSnapshotConsumer::new(&*subscribe(&broadcast, "status").await).unwrap(); + let snapshot_consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "status".into()).unwrap(); let stream_consumer = MoqJsonStreamConsumer::new(&*subscribe(&broadcast, "events").await).unwrap(); kio::clock::timeout(TIMEOUT, snapshot_demand.until_wanted()) diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index a05d8a9676f..0af4cabb21f 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -900,20 +900,10 @@ impl Session { /// with the session, except those of renditions the catalog retired. async fn follow_catalog(&self, source: moq_mux::Source, cancel: &mut watch::Receiver) -> Result<()> { let broadcast = source.broadcast(); - let catalog_track = broadcast.track(hang::catalog::Catalog::DEFAULT_NAME)?; - // A publisher that never answers would otherwise hold the run until it is cancelled. - let catalog_track = tokio::select! { - track = catalog_track.subscribe(hang::catalog::Catalog::default_subscription()) => match track { - Ok(track) => track, - Err(err) if source_lost(&err) => { - tracing::warn!(%err, "catalog unavailable, holding pads"); - return Ok(()); - } - Err(err) => return Err(anyhow::Error::from(err).context("catalog refused")), - }, - _ = cancel.changed() => return Ok(()), - }; - let mut catalog_consumer = moq_mux::catalog::hang::Consumer::new(catalog_track); + // The consumer subscribes itself: a refused or lost subscription surfaces from its first read + // below, and a publisher that never answers holds the run only until it is cancelled. + let mut catalog_consumer = + moq_mux::catalog::hang::Consumer::new(broadcast.track(hang::catalog::Catalog::DEFAULT_NAME)?); // Follow the catalog for the whole run and reconcile against every update, rather than // building the renditions once from the first frame. This covers reactive publishers (the diff --git a/rs/moq-json/MIGRATE.md b/rs/moq-json/MIGRATE.md index 871062f96e4..9a3c7527ec2 100644 --- a/rs/moq-json/MIGRATE.md +++ b/rs/moq-json/MIGRATE.md @@ -4,10 +4,14 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`snapshot::Consumer::new` sets its subscription's `delay.max` to zero, replacing whatever the subscriber asked for. -`next()` now skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes. +`snapshot::Consumer::new` takes the track, a `moq_net::track::Consumer`, instead of a `track::Subscriber`, and subscribes to it itself at a zero `delay.max` and default preferences otherwise; the new `Config::priority` sets the subscription's priority. +Pass `broadcast.track(name)?` rather than `broadcast.track(name)?.subscribe(..).await?`, or `producer.consume()` for an in-process `track::Producer`. +A refused or failed subscription now surfaces from the first read instead of before construction. +No handle to that subscription leaves the consumer, so ranges, order, and `delay.max` can no longer be set on a snapshot reader. + +`next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group a received track still owes, whatever another reader of the track asks for. A reader that set `delay.max` to replay a snapshot track's history or trail a playhead gets only the newest group: read the track's groups yourself and feed them to a `snapshot::Decoder`, or publish a `stream` track when every state must survive, or a `window` track for a bounded recent history. ### Compression moves to the track diff --git a/rs/moq-json/examples/telemetry.rs b/rs/moq-json/examples/telemetry.rs index 3c2aded71a6..d319456742e 100644 --- a/rs/moq-json/examples/telemetry.rs +++ b/rs/moq-json/examples/telemetry.rs @@ -115,9 +115,8 @@ fn frame_bytes(config: snapshot::Config, ticks: u64) -> usize { /// exact input value after delta application (not just the final one). fn verify(producer_config: snapshot::Config, compress: bool, ticks: u64) { let track = track(compress); - let consumer = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), Default::default()); let mut producer = Producer::::new(track, producer_config); - let mut consumer = Consumer::::new(consumer, Default::default()); let waiter = kio::Waiter::noop(); for tick in 0..ticks { @@ -149,7 +148,7 @@ fn verify(producer_config: snapshot::Config, compress: bool, ticks: u64) { /// is far below `ticks`, since stale intermediate reconstructions are applied internally but skipped. fn verify_late_joiner(producer_config: snapshot::Config, compress: bool, ticks: u64) -> usize { let track = track(compress); - let consumer = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), Default::default()); let mut producer = Producer::::new(track, producer_config); for tick in 0..ticks { producer @@ -158,7 +157,6 @@ fn verify_late_joiner(producer_config: snapshot::Config, compress: bool, ticks: } producer.finish().unwrap(); - let mut consumer = Consumer::::new(consumer, Default::default()); let waiter = kio::Waiter::noop(); let mut last = None; let mut yielded = 0; diff --git a/rs/moq-json/examples/teleop.rs b/rs/moq-json/examples/teleop.rs index 0d9499a0bc0..6f2e8f4b8a1 100644 --- a/rs/moq-json/examples/teleop.rs +++ b/rs/moq-json/examples/teleop.rs @@ -26,7 +26,7 @@ fn main() -> Result<(), Box> { let info = track::Info::default().with_timescale(Timescale::MILLI); let state = broadcast.create_track("control.json", info.clone())?; let commands = broadcast.create_track("commands.json", info)?; - let mut state_reader = snapshot::Consumer::::new(state.subscribe(None), Default::default()); + let mut state_reader = snapshot::Consumer::::new(state.consume(), Default::default()); let mut command_reader = stream::Consumer::::new(commands.subscribe(None), Default::default()); let mut state = snapshot::Producer::new(state, Default::default()); let mut commands = stream::Producer::new(commands, Default::default()); diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index efdb507b020..a07971c1390 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -21,6 +21,10 @@ pub struct Config { /// reconstructed value costs no more than that. Set it to hold a smaller document, at the cost /// of re-measuring the value after every delta. pub max_size: Option, + + /// Delivery priority of the consumer's subscription, higher first, as + /// [`Subscription::priority`](moq_net::track::Subscription::priority). + pub priority: i8, } /// Consumes a JSON value from a track, reconstructing it from snapshots and deltas. @@ -30,29 +34,33 @@ pub struct Config { /// When something else already owns the track, use the [`Decoder`] /// directly. /// -/// A snapshot is superseded the moment a newer one exists, so the consumer always reads at a -/// zero [`max`](moq_net::track::Delay::max) delay: a group is skipped once a newer one arrives, -/// and a lower group the subscription still owes never holds up a newer one. +/// A snapshot is superseded the moment a newer one exists, so the consumer subscribes itself at a +/// zero [`max`](moq_net::track::Delay::max) delay and keeps that subscription to itself: a group +/// is skipped once a newer one arrives, and a lower group the subscription still owes never holds +/// up a newer one. pub struct Consumer { - track: moq_net::track::Ordered, + track: Track, + name: String, group: Option, decoder: Decoder, frames_read: usize, } impl Consumer { - /// Create a consumer reading from the given track subscriber. + /// Create a consumer reading from the given track. /// - /// Pins the subscription's [`max`](moq_net::track::Delay::max) delay at zero, whatever the - /// caller asked for, then or later through a retained handle; its other preferences, such - /// as ranges and priority, stay and still update. - /// A [`compress`](moq_net::track::Info::compress)ed track reads the same: moq-net inflates it. - pub fn new(track: moq_net::track::Subscriber, config: Config) -> Self { - let mut track = track.ordered(); - // Refused only once the track has ended, which leaves nothing to read anyway. - let _ = track.pin_max_delay(std::time::Duration::ZERO); + /// Subscribes right away at [`Config::priority`] and a zero + /// [`max`](moq_net::track::Delay::max) delay, with default preferences otherwise; no handle to that subscription leaves the + /// consumer, so nothing can change its budget. A read waits until the subscription resolves, + /// and fails if it cannot. A [`compress`](moq_net::track::Info::compress)ed track reads the + /// same: moq-net inflates it. + pub fn new(track: moq_net::track::Consumer, config: Config) -> Self { + let subscription = moq_net::track::Subscription::default() + .with_priority(config.priority) + .with_delay(moq_net::track::Delay::default().with_max(std::time::Duration::ZERO)); Self { - track, + name: track.name().to_string(), + track: Track::Subscribing(track.subscribe(subscription).into_inner()), group: None, decoder: Decoder::new(config), frames_read: 0, @@ -95,7 +103,7 @@ impl Consumer { if skip { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share - // the sequence space) and stays inside any cap the caller set on the reader. + // the sequence space). while let Poll::Ready(Some(group)) = self.track.poll_next_group(waiter)? { self.group = Some(group); self.frames_read = 0; @@ -144,7 +152,7 @@ impl Consumer { ) => { // A later group starts a fresh snapshot. - tracing::warn!(track = self.track.name(), group = group.sequence, error = ?err, + tracing::warn!(track = self.name, group = group.sequence, error = ?err, "snapshot group lost; waiting for a newer one"); self.group = None; } @@ -172,6 +180,24 @@ impl Consumer { } } +/// The consumer's own subscription, resolved by the first read. +enum Track { + Subscribing(moq_net::track::Subscribing), + Ordered(moq_net::track::Ordered), +} + +impl Track { + fn poll_next_group(&mut self, waiter: &kio::Waiter) -> Poll>> { + if let Self::Subscribing(pending) = self { + *self = Self::Ordered(std::task::ready!(pending.poll_ok(waiter))?.ordered()); + } + match self { + Self::Ordered(track) => track.poll_next_group(waiter), + Self::Subscribing(_) => unreachable!("resolved above"), + } + } +} + #[cfg(test)] mod timestamps { use super::*; @@ -187,10 +213,9 @@ mod timestamps { .produce() .create_track("data", info) .unwrap(); - let subscriber = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), Default::default()); + let mut newest = Consumer::::new(track.consume(), Default::default()); let mut producer = crate::snapshot::Producer::new(track, crate::snapshot::Config::default()); - let mut consumer = Consumer::::new(subscriber, Default::default()); - let mut newest = Consumer::::new(producer.consume(), Default::default()); let times = [0, 810_001, 990_007]; for (n, ticks) in times.into_iter().enumerate() { let at = moq_net::Timestamp::new(ticks, scale).unwrap(); @@ -231,9 +256,8 @@ mod timestamps { .produce() .create_track("data", moq_net::track::Info::default().with_compress(true)) .unwrap(); - let subscriber = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), Default::default()); let mut producer = crate::snapshot::Producer::new(track, crate::snapshot::Config::default()); - let mut consumer = Consumer::::new(subscriber, Default::default()); let waiter = kio::Waiter::noop(); for n in 0..3 { producer @@ -265,38 +289,6 @@ mod timestamps { assert!(matches!(consumer.poll_latest(&waiter), Poll::Ready(Ok(None)))); } - #[test] - fn latest_keeps_a_cap_set_before_the_consumer() { - let track = moq_net::broadcast::Info::new() - .produce() - .create_track("data", moq_net::track::Info::default()) - .unwrap(); - let mut subscriber = track.subscribe(None); - subscriber = subscriber.with_range(..5); - let mut producer = crate::snapshot::Producer::new(track, crate::snapshot::Config::default()); - let mut consumer = Consumer::::new(subscriber, Default::default()); - let waiter = kio::Waiter::noop(); - let mut write = |n: u64| { - producer.update(&serde_json::json!({ "n": n })).unwrap(); - producer.cut().unwrap(); - }; - for n in 0..4 { - write(n); - } - let Poll::Ready(Ok(Some(value))) = consumer.poll_latest(&waiter) else { - panic!("missing latest state") - }; - assert_eq!(value.value["n"], 3); - for n in 4..7 { - write(n); - } - let Poll::Ready(Ok(Some(value))) = consumer.poll_latest(&waiter) else { - panic!("missing capped state") - }; - assert_eq!(value.value["n"], 4, "latest crossed the cap"); - assert!(consumer.poll_next(&waiter).is_pending()); - } - #[test] fn latest_ignores_datagrams() { let track = moq_net::broadcast::Info::new() @@ -304,9 +296,8 @@ mod timestamps { .create_track("data", moq_net::track::Info::default()) .unwrap(); let mut raw = track.clone(); - let subscriber = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), Default::default()); let mut producer = crate::snapshot::Producer::new(track, crate::snapshot::Config::default()); - let mut consumer = Consumer::::new(subscriber, Default::default()); let waiter = kio::Waiter::noop(); for n in 0..2 { producer.update(&serde_json::json!({ "n": n })).unwrap(); diff --git a/rs/moq-json/src/snapshot/mod.rs b/rs/moq-json/src/snapshot/mod.rs index e761d09ffd3..7c4656b1f4f 100644 --- a/rs/moq-json/src/snapshot/mod.rs +++ b/rs/moq-json/src/snapshot/mod.rs @@ -181,27 +181,24 @@ mod test { moq_net::track::Info::default().with_compress(true) } - fn producer(config: Config) -> (Producer, moq_net::track::Subscriber) { + fn producer(config: Config) -> (Producer, moq_net::track::Consumer) { producer_on(config, moq_net::track::Info::default()) } - fn producer_on(config: Config, info: moq_net::track::Info) -> (Producer, moq_net::track::Subscriber) { + fn producer_on(config: Config, info: moq_net::track::Info) -> (Producer, moq_net::track::Consumer) { let track = moq_net::broadcast::Info::new() .produce() .create_track("test", info) .unwrap(); - let consumer = track.subscribe(None); + let consumer = track.consume(); (Producer::new(track, config), consumer) } /// Demand follows the track's subscribers, so a producer can idle while nobody is watching. #[test] fn demand_follows_subscribers() { - let (producer, consumer) = producer(Config::default()); + let (producer, _track) = producer(Config::default()); let demand = producer.demand(); - assert_eq!(demand.wanted(), Some(0)); - - drop(consumer); assert_eq!(demand.wanted(), None); let _consumer = producer.consume(); @@ -209,7 +206,7 @@ mod test { } /// Drain newest states currently available from a plaintext consumer without blocking. - fn drain(track: moq_net::track::Subscriber) -> Vec { + fn drain(track: moq_net::track::Consumer) -> Vec { drain_with(Consumer::::new(track, consumer::Config::default())) } @@ -254,7 +251,7 @@ mod test { .produce() .create_track("test", timed()) .unwrap(); - let mut consumer = Consumer::::new(track.subscribe(None), consumer::Config::default()); + let mut consumer = Consumer::::new(track.consume(), consumer::Config::default()); let waiter = kio::Waiter::noop(); // Group 0 delivers a value, then stays open with the reader parked on its next frame. @@ -289,7 +286,7 @@ mod test { .produce() .create_track("test", timed()) .unwrap(); - let mut consumer = Consumer::::new(track.subscribe(None), consumer::Config::default()); + let mut consumer = Consumer::::new(track.consume(), consumer::Config::default()); let waiter = kio::Waiter::noop(); let mut group = track.create_group(moq_net::group::Info { sequence: 0 }).unwrap(); @@ -717,7 +714,7 @@ mod test { .produce() .create_track("test", moq_net::track::Info::default()) .unwrap(); - let consumer = track.subscribe(None); + let consumer = Consumer::::new(track.consume(), consumer::Config::default()); let mut producer = Producer::::new(track, Config::default()); producer.update(&Doc { count: 1 }).unwrap(); @@ -725,7 +722,7 @@ mod test { // The refused edit published nothing after the original value. producer.finish().unwrap(); - assert_eq!(drain(consumer), vec![json!({ "count": "1", "other": 1 })]); + assert_eq!(drain_with(consumer), vec![json!({ "count": "1", "other": 1 })]); } /// An end declared ahead of the live edge still admits the edit below it, and refuses the one @@ -762,7 +759,7 @@ mod test { .produce() .create_track("test", moq_net::track::Info::default()) .unwrap(); - let consumer = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), consumer::Config::default()); let mut producer = Producer::::new(track, Config::default()); // First owner sets its field. @@ -776,7 +773,6 @@ mod test { producer.finish().unwrap(); - let mut consumer = Consumer::::new(consumer, consumer::Config::default()); let waiter = kio::Waiter::noop(); let mut last = None; while let Poll::Ready(Ok(Some(value))) = consumer.poll_latest(&waiter) { @@ -806,7 +802,8 @@ mod test { .produce() .create_track("test", moq_net::track::Info::default()) .unwrap(); - let mut consumer = track.subscribe(None); + let mut raw = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), consumer::Config::default()); let mut producer = Producer::::new(track, cfg(false)); producer.mutate(|doc| doc.video = Some("v1".to_string()), None).unwrap(); @@ -818,7 +815,7 @@ mod test { // otherwise open a group of its own. producer.mutate(|_| {}, None).unwrap(); assert_eq!( - consumer + raw .largest() .now_or_never() .flatten() @@ -828,7 +825,6 @@ mod test { producer.finish().unwrap(); - let mut consumer = Consumer::::new(consumer, consumer::Config::default()); let waiter = kio::Waiter::noop(); let mut last = None; while let Poll::Ready(Ok(Some(value))) = consumer.poll_latest(&waiter) { @@ -893,7 +889,7 @@ mod test { fn a_dropped_failed_edit_aborts_the_track() { let (mut producer, track) = producer(cfg(false)); producer.update(&json!({ "keep": true })).unwrap(); - let mut subscriber = track.ordered(); + let mut subscriber = track.subscribe(None).now_or_never().unwrap().unwrap().ordered(); *producer.modify(None).unwrap() = json!({ "big": "x".repeat(moq_net::group::MAX_CACHE_BYTES as usize + 1) }); @@ -1112,10 +1108,8 @@ mod test { .create_track("test", timed()) .unwrap(); let mut group = track.append_group().unwrap(); - let consumer_track = track.subscribe(None); + let mut consumer = Consumer::::new(track.consume(), consumer::Config::default()); track.finish().unwrap(); - - let mut consumer = Consumer::::new(consumer_track, consumer::Config::default()); let waiter = kio::Waiter::noop(); // Track is finished but the open group is empty: pending, not end-of-stream. @@ -1203,7 +1197,7 @@ mod test { for n in 0..=40 { producer.update(&json!({ "n": n })).unwrap(); } - let late = producer.consume(); + let late = Consumer::::new(track.clone(), consumer::Config::default()); assert!(matches!( producer.poll_refresh(&kio::Waiter::noop()), Poll::Ready(Ok(())) @@ -1215,7 +1209,7 @@ mod test { track.largest().now_or_never().flatten().map(|position| position.group), Some(1) ); - assert_eq!(drain(late).last().unwrap(), &json!({ "n": 41 })); + assert_eq!(drain_with(late).last().unwrap(), &json!({ "n": 41 })); } #[test] diff --git a/rs/moq-mux/MIGRATE.md b/rs/moq-mux/MIGRATE.md index 2ddcae1a714..f59213efa91 100644 --- a/rs/moq-mux/MIGRATE.md +++ b/rs/moq-mux/MIGRATE.md @@ -4,6 +4,12 @@ How to adopt new APIs and update for breaking changes, newest first. ## Unreleased +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) + +`catalog::hang::Consumer::new` and its `From` impl take the catalog track, a `moq_net::track::Consumer`, instead of a `track::Subscriber`; the consumer subscribes itself, as `hang::catalog::Consumer::new` does. Pass `broadcast.track(hang::Catalog::DEFAULT_NAME)?` rather than subscribing first. +`json::Consumer::from_track` takes a `moq_net::track::Consumer` and is now `async`: a snapshot track's reader subscribes itself, and a stream track's subscription resolves before it returns. +A refused catalog or snapshot subscription now surfaces from the first read. + ### Shared playout timing Native players can use `timeline::Producer` and its read-only `Consumer` instead diff --git a/rs/moq-mux/src/catalog/consumer.rs b/rs/moq-mux/src/catalog/consumer.rs index 5ed2bc00d14..6dfccf4b527 100644 --- a/rs/moq-mux/src/catalog/consumer.rs +++ b/rs/moq-mux/src/catalog/consumer.rs @@ -51,13 +51,7 @@ impl Consumer { pub async fn new(broadcast: &moq_net::broadcast::Consumer, format: CatalogFormat) -> Result { Ok(Self { inner: match format { - CatalogFormat::Hang => { - let track = broadcast - .track(hang::Catalog::DEFAULT_NAME)? - .subscribe(hang::Catalog::default_subscription()) - .await?; - State::Hang(super::hang::Consumer::new(track)) - } + CatalogFormat::Hang => State::Hang(super::hang::Consumer::new(broadcast.track(hang::Catalog::DEFAULT_NAME)?)), CatalogFormat::Msf => { let track = broadcast.track(moq_msf::DEFAULT_NAME)?.subscribe(None).await?; State::Msf(super::msf::Consumer::new(track)) diff --git a/rs/moq-mux/src/catalog/hang/consumer.rs b/rs/moq-mux/src/catalog/hang/consumer.rs index ceaf6e69f0b..be0245ecb46 100644 --- a/rs/moq-mux/src/catalog/hang/consumer.rs +++ b/rs/moq-mux/src/catalog/hang/consumer.rs @@ -20,8 +20,10 @@ pub struct Consumer { } impl Consumer { - /// Create a new catalog consumer from a `catalog.json` track subscriber, compressed or not. - pub fn new(track: moq_net::track::Subscriber) -> Self { + /// Create a new catalog consumer reading a `catalog.json` track, compressed or not. + /// + /// Subscribes to the track itself; see [`hang::catalog::Consumer::new`]. + pub fn new(track: moq_net::track::Consumer) -> Self { Self { inner: hang::catalog::Consumer::new(track), } @@ -54,8 +56,8 @@ impl Consumer { } } -impl From for Consumer { - fn from(inner: moq_net::track::Subscriber) -> Self { +impl From for Consumer { + fn from(inner: moq_net::track::Consumer) -> Self { Self::new(inner) } } diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs index 381ea0367be..67faf4b2b87 100644 --- a/rs/moq-mux/src/container/test_util.rs +++ b/rs/moq-mux/src/container/test_util.rs @@ -278,9 +278,6 @@ impl Clocks { pub(crate) async fn subscribe(consumer: &moq_net::broadcast::Consumer) -> Self { let track = consumer .track(hang::Catalog::DEFAULT_NAME) - .unwrap() - .subscribe(None) - .await .unwrap(); Self(crate::catalog::hang::Consumer::new(track)) } diff --git a/rs/moq-mux/src/fuzz.rs b/rs/moq-mux/src/fuzz.rs index e4e4cd38786..00d6e8c3bee 100644 --- a/rs/moq-mux/src/fuzz.rs +++ b/rs/moq-mux/src/fuzz.rs @@ -198,12 +198,11 @@ pub fn catalog(data: &[u8]) -> bool { let track = broadcast .create_track(name, info) .expect("a fresh broadcast has no catalog"); - let subscriber = track.subscribe(None); let mut next: Box Poll> = if selector & 1 == 0 { - let mut consumer = crate::catalog::hang::Consumer::<()>::new(subscriber); + let mut consumer = crate::catalog::hang::Consumer::<()>::new(track.consume()); Box::new(move || consumer.poll_next(&waiter)) } else { - let mut consumer = crate::catalog::msf::Consumer::<()>::new(subscriber); + let mut consumer = crate::catalog::msf::Consumer::<()>::new(track.subscribe(None)); Box::new(move || consumer.poll_next(&waiter)) }; diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index f717048ae89..4b64a50e36e 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -357,18 +357,23 @@ enum Inner { } impl Consumer { - /// Read an already-subscribed track as its catalog entry declares. + /// Subscribe to a track and read it as its catalog entry declares. /// - /// The escape hatch for a caller that resolved the subscription itself. Prefer + /// The escape hatch for a caller that resolved the track itself. Prefer /// [`Entry::subscribe`](crate::catalog::Entry::subscribe), which resolves the track (including a /// cross-broadcast [`broadcast`](JsonConfig::broadcast) reference) from the catalog for you. + /// A snapshot reader subscribes itself (see [`moq_json::snapshot::Consumer::new`]); a stream + /// waits for its subscription here. /// /// Errors if the entry declares a mode this build doesn't implement, which is a track a consumer /// must skip rather than guess at. - pub fn from_track(track: moq_net::track::Subscriber, config: &JsonConfig) -> crate::Result { + pub async fn from_track(track: moq_net::track::Consumer, config: &JsonConfig) -> crate::Result { let inner = match &config.mode { Mode::Snapshot => Inner::Snapshot(moq_json::snapshot::Consumer::new(track, Default::default())), - Mode::Stream => Inner::Stream(moq_json::stream::Consumer::new(track, Default::default())), + Mode::Stream => Inner::Stream(moq_json::stream::Consumer::new( + track.subscribe(None).await?, + Default::default(), + )), other => return Err(crate::Error::UnsupportedMode(other.to_string())), }; @@ -427,14 +432,11 @@ impl crate::catalog::Entry<'_, JsonConfig> { /// /// Errors if the entry declares a mode this build doesn't implement. pub async fn subscribe(&self, source: &crate::Source) -> crate::Result> { - let track = source - .subscribe_track( - self.config().broadcast.as_ref(), - self.config().instance.as_ref(), - self.config().track(self.name()), - ) + let broadcast = source + .bind(self.config().broadcast.as_ref(), self.config().instance.as_ref())? + .broadcast() .await?; - Consumer::from_track(track, self.config()) + Consumer::from_track(broadcast.track(self.config().track(self.name()))?, self.config()).await } } @@ -442,6 +444,7 @@ impl crate::catalog::Entry<'_, JsonConfig> { mod test { use std::task::Poll; + use futures::FutureExt; use serde_json::{Value, json}; use super::*; @@ -475,6 +478,11 @@ mod test { /// Drain every value a consumer currently has, without blocking. /// /// One loop whichever mode the publisher chose, which is the point of the single `Consumer`. + /// [`Consumer::from_track`] on an in-process track, whose subscription resolves at once. + fn read(track: moq_net::track::Consumer, config: &JsonConfig) -> crate::Result> { + Consumer::from_track(track, config).now_or_never().expect("in-process subscription") + } + fn drain(mut consumer: Consumer) -> Vec { let waiter = kio::Waiter::noop(); let mut out = Vec::new(); @@ -495,17 +503,16 @@ mod test { .json_stream::(track(&mut broadcast, "chat"), Config::default()) .unwrap(); - let track = chat.consume(); + // The catalog is the only thing the reader is told; the mode comes from it. + let entry = entry(&catalog, "chat"); + let track = broadcast.consume().track("chat").unwrap(); + let consumer = read(track, &entry).unwrap(); let expected: Vec = (0..3).map(|n| json!({ "n": n })).collect(); for value in &expected { chat.append(value, catalog.clock().now()).unwrap(); } - // `finish` retires the entry, so read both off the live track first. - let entry = entry(&catalog, "chat"); chat.finish().unwrap(); - // The catalog is the only thing the reader is told; the mode comes from it. - let consumer = Consumer::from_track(track, &entry).unwrap(); assert_eq!(consumer.mode(), &Mode::Stream); assert_eq!(drain(consumer), expected); } @@ -517,14 +524,14 @@ mod test { .json_snapshot::(track(&mut broadcast, "status"), Config::default(), Options::default()) .unwrap(); - let track = status.consume(); + let entry = entry(&catalog, "status"); + let track = broadcast.consume().track("status").unwrap(); + let consumer = read(track, &entry).unwrap(); status.update(&json!({ "live": false }), catalog.clock().now()).unwrap(); status.update(&json!({ "live": true }), catalog.clock().now()).unwrap(); - let entry = entry(&catalog, "status"); status.finish().unwrap(); - // A late reader only sees the newest value, which is the point of snapshot mode. - let consumer = Consumer::from_track(track, &entry).unwrap(); + // The reader only sees the newest value, which is the point of snapshot mode. assert_eq!(consumer.mode(), &Mode::Snapshot); assert_eq!(drain(consumer), vec![json!({ "live": true })]); } @@ -847,7 +854,8 @@ mod test { let mut chat = catalog .json_stream::(track(&mut broadcast, "chat"), Config::default()) .unwrap(); - let track = chat.consume(); + let entry = entry(&catalog, "chat"); + let consumer = read(broadcast.consume().track("chat").unwrap(), &entry).unwrap(); let oversized = Value::String("x".repeat(moq_net::group::MAX_CACHE_BYTES as usize)); assert!(matches!( @@ -857,12 +865,8 @@ mod test { assert!(catalog.snapshot().json.tracks.contains_key("chat")); chat.append(&json!({ "n": 1 }), catalog.clock().now()).unwrap(); - let entry = entry(&catalog, "chat"); chat.finish().unwrap(); - assert_eq!( - drain(Consumer::from_track(track, &entry).unwrap()), - vec![json!({ "n": 1 })] - ); + assert_eq!(drain(consumer), vec![json!({ "n": 1 })]); } /// Every write is stamped, so a track without a timescale, which refuses a timestamp, is refused @@ -896,7 +900,7 @@ mod test { let config = JsonConfig::new(Mode::Unknown("windowed".to_string())); assert!(matches!( - Consumer::::from_track(track.subscribe(None), &config), + read(track.consume(), &config), Err(crate::Error::UnsupportedMode(_)) )); } diff --git a/rs/moq-net-tests/tests/integration/refresh.rs b/rs/moq-net-tests/tests/integration/refresh.rs index 5b5df917977..886bc7df6c1 100644 --- a/rs/moq-net-tests/tests/integration/refresh.rs +++ b/rs/moq-net-tests/tests/integration/refresh.rs @@ -76,7 +76,7 @@ async fn json_snapshot_refresh_crosses_a_two_relay_cluster() { ); if n == 1 || n == 2 * u64::from(ratio) { let mut join = - snapshot::Consumer::::new(remote.subscribe(None).await.unwrap(), Default::default()); + snapshot::Consumer::::new(remote.clone(), Default::default()); let received = join.latest().await.unwrap().unwrap(); assert_eq!((received.value, received.at), (value(n), Some(at(n)))); assert!(producer.poll_refresh(&kio::Waiter::noop()).is_pending()); @@ -138,7 +138,7 @@ async fn json_snapshot_refresh_crosses_a_two_relay_cluster() { serde_json::from_slice::(&delta.payload).unwrap(), json!({ "n": n + 1 }) ); - let mut current = snapshot::Consumer::::new(remote.subscribe(None).await.unwrap(), Default::default()); + let mut current = snapshot::Consumer::::new(remote.clone(), Default::default()); let received = current.latest().await.unwrap().unwrap(); assert_eq!((received.value, received.at), (value(n + 1), Some(at(n + 1)))); assert!(producer.poll_refresh(&kio::Waiter::noop()).is_pending()); diff --git a/rs/moq-net-tests/tests/integration/snapshot.rs b/rs/moq-net-tests/tests/integration/snapshot.rs index b688152f572..85177a91ce1 100644 --- a/rs/moq-net-tests/tests/integration/snapshot.rs +++ b/rs/moq-net-tests/tests/integration/snapshot.rs @@ -5,8 +5,8 @@ use serde_json::{Value, json}; use std::{task::Poll, time::Duration}; /// Group 2 arrives while group 1 is still owed: `latest()` and `next()` return it at once, -/// whatever budget the caller subscribed with or later restores through a retained handle, -/// since a snapshot consumer pins its max delay at zero. +/// whatever budget another reader of the track asks for, since a snapshot consumer subscribes +/// itself at a zero max delay. #[test] fn reads_skip_an_owed_hole() { for max in [Some(Duration::ZERO), Some(Duration::from_secs(1)), None] { @@ -15,16 +15,10 @@ fn reads_skip_an_owed_hole() { .create_track("state", track::Info::default()) .unwrap(); fuzz::received(&track); - let subscription = track::Subscription::default().with_delay(track::Delay::default().with_max(max)); - let reader = || { - let subscriber = track.subscribe(subscription.clone()); - let control = subscriber.control(); - let consumer = snapshot::Consumer::::new(subscriber, Default::default()); - control.update(subscription.clone()).unwrap(); - (consumer, control) - }; - let (mut latest, latest_control) = reader(); - let (mut next, next_control) = reader(); + // The caller's own reader of the track, at the budget it might have wanted. + let _other = track.subscribe(track::Subscription::default().with_delay(track::Delay::default().with_max(max))); + let mut latest = snapshot::Consumer::::new(track.consume(), Default::default()); + let mut next = snapshot::Consumer::::new(track.consume(), Default::default()); let waiter = kio::Waiter::noop(); let put = |sequence: u64| { let mut group = track.create_group(group::Info { sequence }).unwrap(); @@ -44,11 +38,6 @@ fn reads_skip_an_owed_hole() { }; assert_eq!(state.value, json!({ "n": 0 })); - // Restored again while both reads wait: every update keeps the pinned zero. - for control in [&latest_control, &next_control] { - control.update(subscription.clone()).unwrap(); - assert_eq!(control.subscription().delay.max, Some(Duration::ZERO)); - } put(2); let Poll::Ready(Ok(Some(state))) = latest.poll_latest(&waiter) else { panic!("latest waited on owed group 1 with max {max:?}") diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 0537cabfb5e..36ca52ffa7b 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -9097,29 +9097,6 @@ impl Subscriber { self.consume().refuse_subscription(&subscription)?; } channel.update(subscription)?; - self.synced(); - Ok(()) - } - - /// Hold this subscriber's [`max`](Delay::max) delay at `max`, keeping the rest of its - /// preferences, for as long as the subscription lives. - /// - /// Every later [`Self::update`], or one through a [`Control`] or a clone, keeps the pinned - /// max delay and applies the rest. The pin applies in place, under the same lock as those - /// updates, so no update lands in between and none made meanwhile is lost. Returns - /// [`Error::Closed`] if the track already ended. - pub fn pin_max_delay(&mut self, max: Duration) -> Result<()> { - let channel = match &self.inner { - Inner::Plain(cursor) => &cursor.subscription, - Inner::Resume(resume, _) => resume.subscription(), - }; - channel.pin_max_delay(max)?; - self.synced(); - Ok(()) - } - - /// Bring the read side up to date with preferences this handle just changed. - fn synced(&mut self) { match &mut self.inner { Inner::Plain(cursor) => { cursor.sync(); @@ -9128,6 +9105,7 @@ impl Subscriber { // sources even while nobody reads. Inner::Resume(resume, _) => resume.mirror().sync(), } + Ok(()) } /// Poll for the track's largest location within [`Self::with_range`]: the current one, then each newer one, and `None` once the track ends. @@ -9355,12 +9333,6 @@ impl Ordered { self.inner.update(subscription) } - /// Hold this subscriber's [`max`](Delay::max) delay at `max` through every later update; - /// see [`Subscriber::pin_max_delay`]. - pub fn pin_max_delay(&mut self, max: Duration) -> Result<()> { - self.inner.pin_max_delay(max) - } - /// Block until the track declares its end, returning the exclusive end position. /// /// See [`Subscriber::finished`]: this reports the declared end, not that every group @@ -14819,34 +14791,6 @@ mod test { assert_eq!(aggregate.priority, 7); } - #[test] - fn pinned_max_delay_survives_every_update() { - let producer = track_producer("test", Info::timed()); - let asked = Subscription::default().with_delay(Delay::default().with_max(Duration::from_secs(1))); - let mut subscriber = producer.subscribe(asked.clone()).ordered(); - let control = subscriber.control(); - - // Another handle changes the priority after this side last read its preferences. - let stale = subscriber.subscription(); - control.update(asked.clone().with_priority(7)).unwrap(); - subscriber.pin_max_delay(Duration::ZERO).unwrap(); - let pinned = Subscription::default() - .with_priority(7) - .with_delay(Delay::default().with_max(Duration::ZERO)); - assert_eq!(subscriber.subscription(), pinned); - // Writing back the copy read before the change would have lost it. - assert_eq!(stale.priority, 0); - - // Restoring the budget through either handle keeps the pin and applies the rest. - control.update(asked.clone().with_priority(3)).unwrap(); - assert_eq!(subscriber.subscription(), pinned.clone().with_priority(3)); - let unlimited = Subscription::default().with_delay(Delay::default().with_max(None)); - subscriber.update(unlimited).unwrap(); - let expected = Subscription::default().with_delay(Delay::default().with_max(Duration::ZERO)); - assert_eq!(subscriber.subscription(), expected); - assert_eq!(producer.subscription(), Some(expected)); - } - #[test] fn dropped_subscriber_leaves_no_ghost_in_aggregate() { // Regression (#2351): a departed subscriber must not keep contributing its diff --git a/rs/moq-net/src/model/track/demand.rs b/rs/moq-net/src/model/track/demand.rs index cee33758edf..d7ed553ead5 100644 --- a/rs/moq-net/src/model/track/demand.rs +++ b/rs/moq-net/src/model/track/demand.rs @@ -2,7 +2,7 @@ use std::{ collections::BTreeMap, sync::{ - Arc, Mutex, PoisonError, + Arc, atomic::{self, AtomicU64}, }, task::Poll, @@ -209,9 +209,6 @@ struct Preference { subscriptions: Option>, /// Bumped under the write whenever the ranges change. ranges_revision: AtomicU64, - /// The max delay every write keeps, once [`Preferences::pin_max_delay`] sets one. Set - /// and read only under `value`'s write lock, so no write lands between a pin and its use. - max_delay_pin: Mutex>, } impl Preferences { @@ -226,23 +223,11 @@ impl Preferences { value: kio::Producer::new(value), subscriptions, ranges_revision: AtomicU64::new(0), - max_delay_pin: Mutex::new(None), })) } pub(super) fn update(&self, next: Subscription) -> crate::Result<()> { - self.update_with(|current| *current = next) - } - - /// Change the preferences in place under the write lock, so an update another handle - /// makes meanwhile is kept rather than overwritten with a stale copy. - pub(super) fn update_with(&self, change: impl FnOnce(&mut Subscription)) -> crate::Result<()> { let mut value = self.0.value.write().map_err(|_| crate::Error::Closed)?; - let mut next = value.clone(); - change(&mut next); - if let Some(max) = *self.0.max_delay_pin.lock().unwrap_or_else(PoisonError::into_inner) { - next.delay.max = Some(max); - } if *value == next { return Ok(()); } @@ -261,12 +246,6 @@ impl Preferences { } Ok(()) } - - /// Hold the max delay at `max` from now on, whatever a later write asks for. - pub(super) fn pin_max_delay(&self, max: Duration) -> crate::Result<()> { - // Pinned inside the write, which then applies it. - self.update_with(|_| *self.0.max_delay_pin.lock().unwrap_or_else(PoisonError::into_inner) = Some(max)) - } } impl Preferences { @@ -312,7 +291,6 @@ impl From> for Preferences { value, subscriptions: None, ranges_revision: AtomicU64::new(0), - max_delay_pin: Mutex::new(None), })) } } diff --git a/rs/moq-rtc/src/lib.rs b/rs/moq-rtc/src/lib.rs index 19895f301d6..6b305030fd4 100644 --- a/rs/moq-rtc/src/lib.rs +++ b/rs/moq-rtc/src/lib.rs @@ -110,10 +110,7 @@ mod tests { async fn opus_output(broadcast: &moq_net::broadcast::Consumer) -> Track { let catalog_track = broadcast .track(hang::Catalog::DEFAULT_NAME) - .expect("output catalog track") - .subscribe(hang::Catalog::default_subscription()) - .await - .expect("subscribe to output catalog"); + .expect("output catalog track"); let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog_track); let catalog = kio::clock::timeout(TIMEOUT, catalogs.next()) .await diff --git a/rs/moq-stats/src/aggregate.rs b/rs/moq-stats/src/aggregate.rs index b1ab9a7c032..d155ecd1273 100644 --- a/rs/moq-stats/src/aggregate.rs +++ b/rs/moq-stats/src/aggregate.rs @@ -14,7 +14,7 @@ use std::time::Duration; use kio::clock::Instant; use moq_net::kio::{self, Pending, Waiter}; use moq_net::stats::{Presence, Traffic}; -use moq_net::track::Subscribing; +use moq_net::track::Resolving; use moq_net::{Instance, Path, broadcast, origin}; use crate::{ @@ -697,8 +697,9 @@ enum Reader { Resolving(Box>), /// Waiting for the node's listing to offer the track. Waiting, - /// Awaiting the subscription handshake. - Subscribing(Pending), + /// Awaiting the subscription handshake: the reader has subscribed, and the + /// track's info resolving shows whether the track exists. + Subscribing(Box>, Pending), /// Reading frames. Boxed: the snapshot consumer dwarfs the other variants, /// and one lives per node in a map. Active(Box>), @@ -1001,7 +1002,7 @@ fn advance( } }, Poll::Ready(Ok(broadcast)) => match broadcast.track(name) { - Ok(track) => node.reader = Reader::Subscribing(track.subscribe(None)), + Ok(track) => node.reader = subscribing(track, config), Err(err) => { tracing::debug!(?err, name, "stats: node missing track"); return changed | node.depart(); @@ -1030,17 +1031,19 @@ fn advance( } node.offered = false; match listing.broadcast().track(name) { - Ok(track) => node.reader = Reader::Subscribing(track.subscribe(None)), + Ok(track) => node.reader = subscribing(track, config), Err(err) => { tracing::debug!(?err, name, "stats: node missing track"); return changed | node.depart(); } } } - Reader::Subscribing(pending) => match pending.poll_ok(waiter) { - Poll::Ready(Ok(subscriber)) => { - node.reader = - Reader::Active(Box::new(moq_json::snapshot::Consumer::new(subscriber, config.clone()))); + Reader::Subscribing(_, pending) => match pending.poll_ok(waiter) { + Poll::Ready(Ok(_)) => { + let Reader::Subscribing(reader, _) = std::mem::replace(&mut node.reader, Reader::Waiting) else { + unreachable!("matched above"); + }; + node.reader = Reader::Active(reader); } // The listed instance ended before the subscription landed: wait for // the next one. @@ -1100,6 +1103,16 @@ fn resolve(origin: &origin::Consumer, path: &Path, instance: Instance) -> Rea Reader::Resolving(Box::new(pending)) } +/// Subscribe a node's reader to `track`, awaiting the handshake. +fn subscribing( + track: moq_net::track::Consumer, + config: &moq_json::snapshot::consumer::Config, +) -> Reader { + // The reader subscribes first, so resolving the info rides on its subscription. + let reader = Box::new(moq_json::snapshot::Consumer::new(track.clone(), config.clone())); + Reader::Subscribing(reader, track.info()) +} + #[cfg(test)] mod tests { /// A millisecond track: the frames written here carry timestamps. diff --git a/rs/moq-stats/src/consume.rs b/rs/moq-stats/src/consume.rs index 4b22bb8f1fc..28bda8fa857 100644 --- a/rs/moq-stats/src/consume.rs +++ b/rs/moq-stats/src/consume.rs @@ -71,15 +71,17 @@ impl Consumer { /// Fails with [`NotFound`](moq_net::Error::NotFound) unless the group is /// listed. pub async fn group(&self, group: impl Into) -> Result { - let track = self.broadcast.track(&group_track(group))?.subscribe(None).await?; Ok(GroupConsumer { - inner: moq_json::snapshot::Consumer::new(track, Default::default()), + inner: self.subscribe(&group_track(group)).await?, }) } async fn subscribe(&self, name: &str) -> Result> { - let track = self.broadcast.track(name)?.subscribe(None).await?; - Ok(moq_json::snapshot::Consumer::new(track, Default::default())) + let track = self.broadcast.track(name)?; + // The reader subscribes itself; resolving the info rides on that subscription. + let reader = moq_json::snapshot::Consumer::new(track.clone(), Default::default()); + track.info().await?; + Ok(reader) } } diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index 7f5538c7ad2..da09553e5ce 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -1485,9 +1485,9 @@ mod tests { let mut previous = None; for (reading, delta) in [(40, 11), (1_000, 7)] { store(reading); - let subscriber = pumping(&mut drain, broadcast.track(&name).unwrap().subscribe(None)) - .await - .unwrap(); + let track = broadcast.track(&name).unwrap(); + let mut reader = moq_json::snapshot::Consumer::::new(track.clone(), Default::default()); + pumping(&mut drain, track.info()).await.unwrap(); let mut cursor = pumping(&mut drain, broadcast.track(&name).unwrap().subscribe(None)) .await .unwrap(); @@ -1497,7 +1497,6 @@ mod tests { } drop(group); drop(cursor); - let mut reader = moq_json::snapshot::Consumer::::new(subscriber, Default::default()); let initial = pumping(&mut drain, reader.next()).await.unwrap().unwrap().value; assert_eq!( (initial[""].wire_tx, initial[""].wire_rx), @@ -1577,10 +1576,6 @@ mod tests { let track = consumer .track("t.json") - .unwrap() - .subscribe(None) - .now_or_never() - .unwrap() .unwrap(); let mut reader = moq_json::snapshot::Consumer::::new(track, Default::default()); let mut next = || { @@ -1745,10 +1740,7 @@ mod tests { async fn read_group(broadcast: &moq_net::broadcast::Consumer, group: &str) -> crate::GroupFrame { let track = broadcast .track(&crate::group_track(moq_net::Path::new(group).unwrap())) - .expect("track") - .subscribe(None) - .await - .expect("subscribe"); + .expect("track"); let mut reader = moq_json::snapshot::Consumer::::new(track, Default::default()); reader.latest().await.expect("read").expect("frame").value } @@ -1768,7 +1760,7 @@ mod tests { struct Reader(moq_json::snapshot::Consumer); impl Reader { - fn new(track: track::Subscriber) -> Self { + fn new(track: track::Consumer) -> Self { Self(moq_json::snapshot::Consumer::new(track, Default::default())) } @@ -1795,24 +1787,17 @@ mod tests { broadcast: &moq_net::broadcast::Consumer, name: &str, ) -> Reader { - Reader::new( - broadcast - .track(name) - .expect("track") - .subscribe(None) - .await - .expect("subscribe"), - ) + let track = broadcast.track(name).expect("track"); + let reader = Reader::new(track.clone()); + track.info().await.expect("subscribe"); + reader } /// The latest buffered totals frame. async fn read_totals(broadcast: &moq_net::broadcast::Consumer) -> TotalsFrame { let track = broadcast .track(crate::TOTALS) - .expect("track") - .subscribe(None) - .await - .expect("subscribe"); + .expect("track"); let mut reader = moq_json::snapshot::Consumer::::new(track, Default::default()); reader.latest().await.expect("read").expect("frame").value } @@ -2281,13 +2266,11 @@ mod tests { .request(moq_net::Path::from_static("acme")); drive_tick().await; let (_, broadcast) = announced(&origin).await; - let track = broadcast + let handle = broadcast .track(&crate::group_track(moq_net::Path::from_static(""))) - .expect("track") - .subscribe(None) - .await - .expect("subscribe"); - let mut track = moq_json::snapshot::Consumer::::new(track, Default::default()); + .expect("track"); + let mut track = moq_json::snapshot::Consumer::::new(handle.clone(), Default::default()); + handle.info().await.expect("subscribe"); let drain = |track: &mut moq_json::snapshot::Consumer| { use futures::FutureExt; let mut frames = Vec::new(); @@ -2358,12 +2341,9 @@ mod tests { /// Subscribe to `label`'s presence track on `broadcast`, driving a tick so /// the producer adopts the request. async fn hold_presence(broadcast: &moq_net::broadcast::Consumer, label: &str) -> Reader { - let subscribing = broadcast - .track(&presence_track(moq_net::Path::new(label).unwrap())) - .expect("track") - .subscribe(None); + let reader = Reader::new(broadcast.track(&presence_track(moq_net::Path::new(label).unwrap())).expect("track")); drive_tick().await; - Reader::new(subscribing.await.expect("held")) + reader } /// A held presence track is cumulative from when it was first held: it @@ -2442,15 +2422,11 @@ mod tests { let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); // Let the request reach the stats broadcast's queue before the drain. - let subscribing = broadcast - .track(&presence_track(moq_net::Path::from_static(""))) - .expect("track") - .subscribe(None); + let mut track = Reader::new(broadcast.track(&presence_track(moq_net::Path::from_static(""))).expect("track")); for _ in 0..8 { tokio::task::yield_now().await; } assert_eq!(tick(), 1, "held"); - let mut track = Reader::new(subscribing.await.expect("held")); assert_eq!(try_presence(&mut track), Some(PresenceFrame::new()), "held at zero"); // The label records while held: the same subscription carries it. @@ -2498,12 +2474,9 @@ mod tests { /// Subscribe to `path`'s prefix track on `broadcast`, driving a tick so the /// producer adopts the request. async fn hold_prefix(broadcast: &moq_net::broadcast::Consumer, path: &str) -> Reader { - let subscribing = broadcast - .track(&prefix_track(moq_net::Path::new(path).unwrap())) - .expect("track") - .subscribe(None); + let reader = Reader::new(broadcast.track(&prefix_track(moq_net::Path::new(path).unwrap())).expect("track")); drive_tick().await; - Reader::new(subscribing.await.expect("held")) + reader } /// The latest buffered prefix frame, `None` when nothing is buffered. @@ -2696,15 +2669,11 @@ mod tests { }; let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); - let subscribing = broadcast - .track(&prefix_track(moq_net::Path::from_static("acme"))) - .expect("track") - .subscribe(None); + let mut track = Reader::new(broadcast.track(&prefix_track(moq_net::Path::from_static("acme"))).expect("track")); for _ in 0..8 { tokio::task::yield_now().await; } assert_eq!(tick(), 1, "held"); - let mut track = Reader::new(subscribing.await.expect("held")); assert_eq!(last_prefix(&mut track), Some(PrefixFrame::new()), "held at zero"); let _feed = feed(®istry, Tier::default(), "acme/live", true, 1, 7).await; diff --git a/rs/moq-transcode/src/lib.rs b/rs/moq-transcode/src/lib.rs index 0807dd170f3..e548d459920 100644 --- a/rs/moq-transcode/src/lib.rs +++ b/rs/moq-transcode/src/lib.rs @@ -126,11 +126,7 @@ impl Transcoder { // The source catalog drives everything; wait for a snapshot with a usable // video rendition (the first may precede the source publishing its video). - let track = source - .track(hang::Catalog::DEFAULT_NAME)? - .subscribe(hang::Catalog::default_subscription()) - .await?; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(source.track(hang::Catalog::DEFAULT_NAME)?); let mut decoders = catalog::Decoders::new(config.feed_decoder()); let (source_name, source_config, snapshot) = loop { let Some(snapshot) = catalogs.next().await? else { @@ -826,7 +822,6 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let track = track.subscribe(None).await.unwrap(); let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); // The catalog track exists from the start but may open empty; the rung // appears once the transcoder has read the source catalog. @@ -933,7 +928,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1055,7 +1050,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); let derived = await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") @@ -1133,7 +1128,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); let derived = await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/360p") @@ -1234,7 +1229,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1301,7 +1296,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1373,7 +1368,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1458,7 +1453,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1569,7 +1564,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1663,7 +1658,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1749,7 +1744,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1799,7 +1794,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); await_catalog(&mut catalogs, |snapshot| { snapshot.video.renditions.contains_key("video/120p") }) @@ -1918,7 +1913,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); let derived = await_catalog(&mut catalogs, |snapshot| !snapshot.video.renditions.is_empty()).await; // Sized against the 640x360 H.264 rendition: no room for 720p, and 120p is @@ -1992,7 +1987,7 @@ mod tests { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); catalogs.next().await.unwrap().unwrap(); // Drop the source: the catalog track ends and the broadcast closes, so diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 0c34eaf4e27..c27cb924a7c 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,9 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Snapshot consumers always read at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) +### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`Json.SnapshotConsumer` reads at a zero `delay.max`, whatever the subscription asked for: iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`Json.SnapshotConsumer(broadcast:name:)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### `lower` is a `Media.Producer` ([#7699](https://github.com/moq-dev/moq/pull/7699)) diff --git a/swift/README.md b/swift/README.md index ed144aef582..5293cb8d8e1 100644 --- a/swift/README.md +++ b/swift/README.md @@ -88,7 +88,8 @@ pre-lite-05, WebSocket, and TCP paths do not deliver them, and there is no strea JSON tracks live under the `Json` namespace, mirroring the `moq-json` crate, and carry your own `Codable` types with the framing handled for you. Each type wraps a track: a producer takes over a `TrackProducer` and advertises it in the broadcast's catalog, and a consumer takes over a -`TrackConsumer`. You opt into one of three distinct modes: `Json.SnapshotProducer`/`Json.SnapshotConsumer` +`TrackConsumer`, except `Json.SnapshotConsumer(broadcast:name:)`, which subscribes itself at a zero +`delay.max`. You opt into one of three distinct modes: `Json.SnapshotProducer`/`Json.SnapshotConsumer` (lossy latest-value, a subscriber only sees the newest value), `Json.StreamProducer`/`Json.StreamConsumer` (lossless append-log, every record preserved in order), or `Json.WindowProducer`/`Json.WindowConsumer` (a sliding window of records a reader can join at any point, not advertised in the catalog). Values encode/decode at the boundary with diff --git a/swift/Sources/Moq/Json.swift b/swift/Sources/Moq/Json.swift index ede0a2295e9..f5772063cfc 100644 --- a/swift/Sources/Moq/Json.swift +++ b/swift/Sources/Moq/Json.swift @@ -83,18 +83,18 @@ extension Json { /// Read side of a JSON snapshot track (lossy latest-value). Iterating yields each state in /// order with its timestamp: `for try await state in json { ... }`. Every group a newer one - /// supersedes is skipped, whatever the subscription's `delay.max`; `latest()` skips to the - /// newest state instead. + /// supersedes is skipped; `latest()` skips to the newest state instead. public final class SnapshotConsumer: AsyncSequence, Sendable { /// The decoded snapshot state emitted by this sequence. public typealias Element = Timed let ffi: MoqJsonSnapshotConsumer - /// Read `track` as a JSON snapshot track. Takes over `track`, whose handle is closed - /// afterward; throws if it has already read a group. - public init(track: TrackConsumer) throws { - ffi = try MoqJsonSnapshotConsumer(track: track.ffi) + /// Subscribe to the track `name` on `broadcast` and read it as a JSON snapshot track. + /// The consumer owns its subscription, at a zero `delay.max`; a read waits for it to + /// resolve and throws if it cannot. + public init(broadcast: BroadcastConsumer, name: String) throws { + ffi = try MoqJsonSnapshotConsumer(broadcast: broadcast.ffi, name: name) } /// The next state in order, or `nil` once the track ends or is closed. diff --git a/swift/Tests/MoqTests/SmokeTests.swift b/swift/Tests/MoqTests/SmokeTests.swift index d64681acc35..fc7a4a6448e 100644 --- a/swift/Tests/MoqTests/SmokeTests.swift +++ b/swift/Tests/MoqTests/SmokeTests.swift @@ -418,7 +418,7 @@ final class SmokeTests: XCTestCase { broadcast: broadcast, track: try broadcast.publishTrack(name: "status", info: TrackInfo(timescale: 1_000_000, compress: true))) let consumer = try Json.SnapshotConsumer( - track: try await broadcast.consume().subscribeTrack(name: "status")) + broadcast: try broadcast.consume(), name: "status") for viewers in 1...4 { try producer.update(Status(state: "live", viewers: viewers), timestamp: micros(UInt64(viewers) * 1000)) @@ -511,7 +511,7 @@ final class SmokeTests: XCTestCase { let consumer = try broadcast.consume() let snapshotConsumer = try Json.SnapshotConsumer<[String: Int]>( - track: try await consumer.subscribeTrack(name: "status")) + broadcast: consumer, name: "status") let streamConsumer = try Json.StreamConsumer<[String: Int]>( track: try await consumer.subscribeTrack(name: "events")) while try await snapshotDemand.wantedChanged() == nil {} From b175944e31bc7a65af65288aa80e66575b2672aa Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 23:36:02 -0800 Subject: [PATCH 16/24] fix: port remaining snapshot and catalog callers; regenerate Dart bindings Co-Authored-By: Claude Opus 5.5 --- dart/moq_ffi/lib/src/moq.dart | 32 +++++++++++-------- demo/carla/web/src/truck.test.ts | 4 +-- demo/carla/web/src/truck.ts | 7 ++-- demo/web/src/index.ts | 8 +++-- demo/web/src/stats.ts | 5 ++- js/json/api/index.api.md | 4 ++- js/json/src/snapshot/consumer.ts | 6 +++- js/moq-boy/src/game.ts | 6 +++- js/watch/src/echo.test.ts | 5 ++- rs/moq-c/src/consume.rs | 15 ++++----- rs/moq-ffi/src/media.rs | 3 +- rs/moq-json/src/snapshot/consumer.rs | 18 +++++++++-- rs/moq-json/src/snapshot/mod.rs | 6 +--- rs/moq-mux/src/catalog/consumer.rs | 4 ++- rs/moq-mux/src/catalog/hang/consumer.rs | 27 +++++++--------- rs/moq-mux/src/catalog/producer.rs | 26 +++++++-------- rs/moq-mux/src/container/test_util.rs | 4 +-- rs/moq-mux/src/json.rs | 4 ++- rs/moq-net-tests/tests/integration/refresh.rs | 3 +- rs/moq-stats/src/produce.rs | 32 +++++++++++++------ rs/moq-transcode/benches/ladder.rs | 2 +- test/interop/clients/js-native/catalog.ts | 10 ++++-- test/interop/clients/js-native/subscribe.ts | 7 ++-- 23 files changed, 140 insertions(+), 98 deletions(-) diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index e4c7b89485c..e358db7ad55 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -10749,14 +10749,17 @@ class MoqJsonSnapshotConsumer implements MoqJsonSnapshotConsumerInterface { MoqJsonSnapshotConsumer._(this._ptr) { _MoqJsonSnapshotConsumerFinalizer.attach(this, _ptr, detach: this); } - MoqJsonSnapshotConsumer({required MoqTrackConsumer track}) - : _ptr = rustCall( - (status) => uniffi_moq_ffi_fn_constructor_moqjsonsnapshotconsumer_new( - FfiConverterMoqTrackConsumer.lower(track), - status, - ), - moqExceptionErrorHandler, - ) { + MoqJsonSnapshotConsumer({ + required MoqBroadcastConsumer broadcast, + required String name, + }) : _ptr = rustCall( + (status) => uniffi_moq_ffi_fn_constructor_moqjsonsnapshotconsumer_new( + FfiConverterMoqBroadcastConsumer.lower(broadcast), + FfiConverterString.lower(name), + status, + ), + moqExceptionErrorHandler, + ) { _MoqJsonSnapshotConsumerFinalizer.attach(this, _ptr, detach: this); } factory MoqJsonSnapshotConsumer.lift(Pointer ptr) { @@ -18863,12 +18866,13 @@ external void uniffi_moq_ffi_fn_free_moqjsonsnapshotconsumer( Pointer uniffiStatus, ); -@Native Function(Pointer, Pointer)>( - assetId: _uniffiAssetId, -) +@Native< + Pointer Function(Pointer, RustBuffer, Pointer) +>(assetId: _uniffiAssetId) external Pointer uniffi_moq_ffi_fn_constructor_moqjsonsnapshotconsumer_new( - Pointer track, + Pointer broadcast, + RustBuffer name, Pointer uniffiStatus, ); @@ -22150,7 +22154,7 @@ void _checkApiChecksums() { 60932) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } - if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotconsumer_next() != 12715) { + if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotconsumer_next() != 15547) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqjsonsnapshotproducer_demand() != @@ -22653,7 +22657,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqjsonsnapshotconsumer_new() != - 11929) { + 49887) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqjsonsnapshotproducer_new() != diff --git a/demo/carla/web/src/truck.test.ts b/demo/carla/web/src/truck.test.ts index 358d03867d3..dbda81b5858 100644 --- a/demo/carla/web/src/truck.test.ts +++ b/demo/carla/web/src/truck.test.ts @@ -328,10 +328,10 @@ describe("Truck safety controls", () => { timestamps: [], }); const front = new Json.Snapshot.Consumer>({ - track: safety.broadcast.track("control").subscribe(), + track: safety.broadcast.track("control"), }); const rear = new Json.Snapshot.Consumer>({ - track: safety.broadcast.track("control").subscribe(), + track: safety.broadcast.track("control"), }); try { diff --git a/demo/carla/web/src/truck.ts b/demo/carla/web/src/truck.ts index cfa29b3d4c7..66cd81a8612 100644 --- a/demo/carla/web/src/truck.ts +++ b/demo/carla/web/src/truck.ts @@ -365,13 +365,12 @@ export class Truck { this.#sawStatusSource = true; this.statusState.set("pending"); - const statusTrack = active.track("status").subscribe({ priority: 10 }); - effect.cleanup(() => statusTrack.close()); - const consumer = new Json.Snapshot.Consumer({ - track: statusTrack, + track: active.track("status"), schema: TruckStatusSchema, + priority: 10, }); + effect.cleanup(() => consumer.close()); effect.spawn(async () => { for (;;) { diff --git a/demo/web/src/index.ts b/demo/web/src/index.ts index 55958c73391..e7342608394 100644 --- a/demo/web/src/index.ts +++ b/demo/web/src/index.ts @@ -440,9 +440,11 @@ ui.run((effect) => { return; } - const track = broadcast.track(trackName).subscribe({ priority: Hang.Catalog.PRIORITY.catalog }); - effect.cleanup(() => track.close()); - const consumer = new Json.Snapshot.Consumer({ track }); + const consumer = new Json.Snapshot.Consumer({ + track: broadcast.track(trackName), + priority: Hang.Catalog.PRIORITY.catalog, + }); + effect.cleanup(() => consumer.close()); effect.spawn(async () => { try { diff --git a/demo/web/src/stats.ts b/demo/web/src/stats.ts index 95f981a762f..421940ad407 100644 --- a/demo/web/src/stats.ts +++ b/demo/web/src/stats.ts @@ -200,9 +200,8 @@ function subscribeNode(effect: Signals.Effect, origin: Net.Origin.Reader, path: const consumer = request.active.peek(); if (!consumer) return; - const track = consumer.track("group.json").subscribe(); - effect.cleanup(() => track.close()); - const group = new Json.Snapshot.Consumer({ track }); + const group = new Json.Snapshot.Consumer({ track: consumer.track("group.json") }); + effect.cleanup(() => group.close()); effect.spawn(async () => { for (;;) { const state = await effect.race(group.latest()); diff --git a/js/json/api/index.api.md b/js/json/api/index.api.md index 771fe2414ae..31d28825535 100644 --- a/js/json/api/index.api.md +++ b/js/json/api/index.api.md @@ -28,12 +28,14 @@ export namespace Snapshot { } export namespace Consumer { export type Config = Pick, "schema"> & { - track: Track.Subscriber; + priority?: number; + track: Pick; }; } export class Consumer { constructor(config: Snapshot.Consumer.Config); [Symbol.asyncIterator](): AsyncIterator>; + close(): void; latest(): Promise | undefined>; next(): Promise | undefined>; } diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index e0b5f84f0ab..95718a5ae91 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -36,7 +36,11 @@ export class Consumer { constructor(config: Consumer.Config) { this.#track = config.track - .subscribe({ ...Moq.Track.Subscription.default(), priority: config.priority ?? 0, delay: { max: Milli.zero } }) + .subscribe({ + ...Moq.Track.Subscription.default(), + priority: config.priority ?? 0, + delay: { max: Milli.zero }, + }) .ordered(); this.#decoder = new Decoder(config); } diff --git a/js/moq-boy/src/game.ts b/js/moq-boy/src/game.ts index 432bff4c4e1..bbfff3d1d06 100644 --- a/js/moq-boy/src/game.ts +++ b/js/moq-boy/src/game.ts @@ -270,7 +270,11 @@ export class Game { if (!active) return; // Reconstruct each status from snapshots and deltas, validated against the schema. - const consumer = new Json.Snapshot.Consumer({ track: active.track("status"), schema: GameStatusSchema, priority: 10 }); + const consumer = new Json.Snapshot.Consumer({ + track: active.track("status"), + schema: GameStatusSchema, + priority: 10, + }); effect.cleanup(() => consumer.close()); // Closing the consumer on cleanup unblocks a pending latest() (it returns undefined), so the loop diff --git a/js/watch/src/echo.test.ts b/js/watch/src/echo.test.ts index cfde2bd9a68..3eea05958dd 100644 --- a/js/watch/src/echo.test.ts +++ b/js/watch/src/echo.test.ts @@ -44,7 +44,10 @@ async function read(origin: Origin.Producer, path: string, clock: Manual): Promi await flush(); const broadcast = request.active.peek(); if (!broadcast) throw new Error(`no feedback at ${path}`); - const consumer = new Json.Snapshot.Consumer({ track: broadcast.track(Schema.TRACK), schema: Schema.SnapshotSchema }); + const consumer = new Json.Snapshot.Consumer({ + track: broadcast.track(Schema.TRACK), + schema: Schema.SnapshotSchema, + }); await flush(); await clock.advance(Nano.fromMilli(Milli(1000))); const value = (await consumer.latest())?.value; diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index b858066ba96..65f19f3a3e6 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -278,20 +278,17 @@ impl Consume { let driver = origin.as_ref().map(|origin| origin.driver.clone()).unwrap_or_default(); - // `subscribe` blocks on SUBSCRIBE_OK, so run it inside the task - // to keep this entrypoint non-blocking. + // Reads wait on SUBSCRIBE_OK, so run them inside the task to keep this entrypoint + // non-blocking. self.catalog_task.spawn( on_catalog, |state| &mut state.consume.catalog_task, - |mut close| async move { + |close| async move { driver .watch(async move { - let subscription = hang::catalog::Catalog::default_subscription(); - let subscribe = Self::subscribe(&broadcast, hang::catalog::Catalog::DEFAULT_NAME, subscription); - let Some(catalog) = Self::until_closed(&mut close, subscribe).await? else { - return Ok(()); - }; - Self::run_catalog(on_catalog, broadcast.clone(), origin, catalog.into(), close).await + // The consumer subscribes itself at catalog priority. + let catalog = broadcast.track(hang::catalog::Catalog::DEFAULT_NAME)?.into(); + Self::run_catalog(on_catalog, broadcast.clone(), origin, catalog, close).await }) .await }, diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index 07ccd6fcb9a..c1a2f134385 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -1724,7 +1724,8 @@ impl MoqMediaCatalogConsumer { /// Subscribe to this broadcast's catalog snapshots. #[uniffi::constructor] pub async fn subscribe(broadcast: &MoqBroadcastConsumer) -> Result, MoqError> { - let consumer = moq_mux::catalog::hang::Consumer::new(broadcast.inner.track(hang::catalog::Catalog::DEFAULT_NAME)?); + let consumer = + moq_mux::catalog::hang::Consumer::new(broadcast.inner.track(hang::catalog::Catalog::DEFAULT_NAME)?); Ok(Arc::new(MoqMediaCatalogConsumer { reads: Polled::new(consumer).supervised(broadcast.driver.clone()), })) diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index a07971c1390..12626a7a7b3 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -58,9 +58,14 @@ impl Consumer { let subscription = moq_net::track::Subscription::default() .with_priority(config.priority) .with_delay(moq_net::track::Delay::default().with_max(std::time::Duration::ZERO)); + let name = track.name().to_string(); + let mut track = Track::Subscribing(track.subscribe(subscription).into_inner()); + // Join now when the track already resolves, as an in-process one does, so the publisher + // sees the reader before its first read; otherwise the first read resolves it. + track.resolve(&kio::Waiter::noop()); Self { - name: track.name().to_string(), - track: Track::Subscribing(track.subscribe(subscription).into_inner()), + name, + track, group: None, decoder: Decoder::new(config), frames_read: 0, @@ -187,6 +192,15 @@ enum Track { } impl Track { + /// Resolve the subscription once it is ready; an error stays for the next read to report. + fn resolve(&mut self, waiter: &kio::Waiter) { + if let Self::Subscribing(pending) = self + && let Poll::Ready(Ok(subscriber)) = pending.poll_ok(waiter) + { + *self = Self::Ordered(subscriber.ordered()); + } + } + fn poll_next_group(&mut self, waiter: &kio::Waiter) -> Poll>> { if let Self::Subscribing(pending) = self { *self = Self::Ordered(std::task::ready!(pending.poll_ok(waiter))?.ordered()); diff --git a/rs/moq-json/src/snapshot/mod.rs b/rs/moq-json/src/snapshot/mod.rs index 7c4656b1f4f..b0943dbebb0 100644 --- a/rs/moq-json/src/snapshot/mod.rs +++ b/rs/moq-json/src/snapshot/mod.rs @@ -815,11 +815,7 @@ mod test { // otherwise open a group of its own. producer.mutate(|_| {}, None).unwrap(); assert_eq!( - raw - .largest() - .now_or_never() - .flatten() - .map(|position| position.group), + raw.largest().now_or_never().flatten().map(|position| position.group), Some(1) ); diff --git a/rs/moq-mux/src/catalog/consumer.rs b/rs/moq-mux/src/catalog/consumer.rs index 6dfccf4b527..1867091bb16 100644 --- a/rs/moq-mux/src/catalog/consumer.rs +++ b/rs/moq-mux/src/catalog/consumer.rs @@ -51,7 +51,9 @@ impl Consumer { pub async fn new(broadcast: &moq_net::broadcast::Consumer, format: CatalogFormat) -> Result { Ok(Self { inner: match format { - CatalogFormat::Hang => State::Hang(super::hang::Consumer::new(broadcast.track(hang::Catalog::DEFAULT_NAME)?)), + CatalogFormat::Hang => State::Hang(super::hang::Consumer::new( + broadcast.track(hang::Catalog::DEFAULT_NAME)?, + )), CatalogFormat::Msf => { let track = broadcast.track(moq_msf::DEFAULT_NAME)?.subscribe(None).await?; State::Msf(super::msf::Consumer::new(track)) diff --git a/rs/moq-mux/src/catalog/hang/consumer.rs b/rs/moq-mux/src/catalog/hang/consumer.rs index be0245ecb46..b1a5233c109 100644 --- a/rs/moq-mux/src/catalog/hang/consumer.rs +++ b/rs/moq-mux/src/catalog/hang/consumer.rs @@ -119,7 +119,7 @@ mod test { let track = broadcast .create_track(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()) .expect("catalog track should create"); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let mut group = track.append_group().expect("catalog group should append"); let payload = serde_json::to_string(&published).expect("catalog should serialize"); @@ -169,14 +169,11 @@ mod test { let resolved = requesting.await; let (_dynamic, _broadcast, track) = handler.await.expect("handler task"); - let subscriber = resolved + let catalog_track = resolved .expect("the handler served it") .track(hang::Catalog::DEFAULT_NAME) - .expect("catalog track should resolve") - .subscribe(None) - .await - .expect("catalog track should subscribe"); - let mut consumer = Consumer::new(subscriber); + .expect("catalog track should resolve"); + let mut consumer = Consumer::new(catalog_track); let mut group = track.append_group().expect("catalog group should append"); let payload = serde_json::to_string(&published).expect("catalog should serialize"); @@ -356,7 +353,7 @@ mod test { #[test] fn waits_for_pending_catalog_group_payload() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let mut group = track.append_group().expect("catalog group should append"); let waiter = kio::Waiter::noop(); @@ -374,7 +371,7 @@ mod test { #[test] fn a_newer_version_is_a_typed_refusal() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer: Consumer = Consumer::new(track.subscribe(None)); + let mut consumer: Consumer = Consumer::new(track.consume()); let mut group = track.append_group().expect("catalog group should append"); group .write_frame(moq_net::Timestamp::ZERO, r#"{"version":5}"#) @@ -390,7 +387,7 @@ mod test { #[test] fn waits_for_pending_catalog_group_payload_after_track_finish() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let mut group = track.append_group().expect("catalog group should append"); track.finish().expect("catalog track should finish"); @@ -410,7 +407,7 @@ mod test { #[test] fn returns_latest_complete_catalog_group() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let waiter = kio::Waiter::noop(); let (_old, old_payload) = catalog_payload("old"); @@ -436,7 +433,7 @@ mod test { #[test] fn waits_for_newer_pending_group_instead_of_returning_older_ready_group() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let waiter = kio::Waiter::noop(); let (_old, old_payload) = catalog_payload("old"); @@ -463,7 +460,7 @@ mod test { #[test] fn retained_pending_group_is_superseded_by_newer_group() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer = Consumer::new(track.subscribe(None)); + let mut consumer = Consumer::new(track.consume()); let waiter = kio::Waiter::noop(); let (_old, old_payload) = catalog_payload("old"); @@ -495,7 +492,7 @@ mod test { #[test] fn a_lost_track_is_a_transport_error() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer: Consumer = Consumer::new(track.subscribe(None)); + let mut consumer: Consumer = Consumer::new(track.consume()); track.abort(moq_net::Error::Cancel).expect("catalog track should abort"); assert!(matches!( @@ -507,7 +504,7 @@ mod test { #[test] fn returns_none_when_empty_track_finishes() { let track = track_producer(hang::Catalog::DEFAULT_NAME, hang::Catalog::default_track_info()); - let mut consumer: Consumer = Consumer::new(track.subscribe(None)); + let mut consumer: Consumer = Consumer::new(track.consume()); let waiter = kio::Waiter::noop(); track.finish().expect("catalog track should finish"); diff --git a/rs/moq-mux/src/catalog/producer.rs b/rs/moq-mux/src/catalog/producer.rs index 7679e7a3137..5b544ac7b97 100644 --- a/rs/moq-mux/src/catalog/producer.rs +++ b/rs/moq-mux/src/catalog/producer.rs @@ -1134,7 +1134,7 @@ impl Producer { /// Create a consumer for this catalog, receiving updates as they're published. pub fn consume(&self) -> Result, moq_net::Error> { - Ok(Consumer::new(self.outputs.hang.consume())) + Ok(Consumer::new(self.outputs.hang_track.consume())) } /// Abort every catalog track with `err`, preventing further edits. @@ -1888,7 +1888,7 @@ mod test { let mut broadcast = moq_net::broadcast::Info::new().produce(); let mut catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let track = catalog.outputs.hang.consume(); + let track = catalog.outputs.hang_track.consume(); assert!(catalog.outputs.hang_track.info().compress, "catalog.json is compressed"); assert!(!catalog.outputs.msf_track.info().compress, "the MSF catalog is not"); let mut consumer = Consumer::new(track); @@ -1914,7 +1914,7 @@ mod test { fn mutate_composes_independent_owners() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let mut catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut plain = Consumer::new(catalog.outputs.hang.consume()); + let mut plain = Consumer::new(catalog.outputs.hang_track.consume()); catalog .mutate(|c| { @@ -2848,7 +2848,7 @@ mod test { fn reservation_gates_until_all_renditions_resolve() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let reserved = catalog.reserve(); @@ -2881,7 +2881,7 @@ mod test { fn live_rendition_owns_its_name_without_gating_the_catalog() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let _unresolved = catalog.rendition::("video0").unwrap(); @@ -2905,7 +2905,7 @@ mod test { let mut broadcast = moq_net::broadcast::Info::new().produce(); let config = Config::default().with_catalog(Catalog::::default()); let mut catalog = Producer::new(&mut broadcast, config).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let reserved = catalog.reserve(); @@ -2943,7 +2943,7 @@ mod test { fn m2ts_counts_as_a_servable_track() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let mut catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let reserved = catalog.reserve(); @@ -2965,7 +2965,7 @@ mod test { let mut broadcast = moq_net::broadcast::Info::new().produce(); let config = Config::default().with_catalog(Catalog::::default()); let mut catalog = Producer::new(&mut broadcast, config).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let reserved = catalog.reserve(); @@ -3004,7 +3004,7 @@ mod test { let mut broadcast = moq_net::broadcast::Info::new().produce(); let config = Config::default().with_catalog(Catalog::::default()); let mut catalog = Producer::new(&mut broadcast, config).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let net = broadcast @@ -3043,7 +3043,7 @@ mod test { let mut broadcast = moq_net::broadcast::Info::new().produce(); let config = Config::default().with_catalog(Catalog::::default()); let catalog = Producer::new(&mut broadcast, config).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let net = broadcast @@ -3081,7 +3081,7 @@ mod test { fn reservation_gate_opens_when_unresolved_reservation_is_dropped() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); let reserved = catalog.reserve(); @@ -3109,7 +3109,7 @@ mod test { fn staged_change_waits_for_a_held_reservation() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); let waiter = kio::Waiter::noop(); // A deferred importer grabs a reservation up front and holds it until it can resolve. @@ -3455,7 +3455,7 @@ mod test { /// The catalog a viewer joining now receives. fn wire(catalog: &Producer) -> Catalog<()> { - let mut consumer: Consumer = Consumer::new(catalog.outputs.hang.consume()); + let mut consumer: Consumer = Consumer::new(catalog.outputs.hang_track.consume()); match consumer.poll_next(&kio::Waiter::noop()) { Poll::Ready(Ok(Some(catalog))) => catalog, _ => panic!("a published catalog"), diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs index 67faf4b2b87..8cf8df70ee1 100644 --- a/rs/moq-mux/src/container/test_util.rs +++ b/rs/moq-mux/src/container/test_util.rs @@ -276,9 +276,7 @@ pub(crate) struct Clocks(crate::catalog::hang::Consumer); impl Clocks { /// Subscribe before importing, so the first snapshot is not missed. pub(crate) async fn subscribe(consumer: &moq_net::broadcast::Consumer) -> Self { - let track = consumer - .track(hang::Catalog::DEFAULT_NAME) - .unwrap(); + let track = consumer.track(hang::Catalog::DEFAULT_NAME).unwrap(); Self(crate::catalog::hang::Consumer::new(track)) } diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index 4b64a50e36e..c70075740e2 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -480,7 +480,9 @@ mod test { /// One loop whichever mode the publisher chose, which is the point of the single `Consumer`. /// [`Consumer::from_track`] on an in-process track, whose subscription resolves at once. fn read(track: moq_net::track::Consumer, config: &JsonConfig) -> crate::Result> { - Consumer::from_track(track, config).now_or_never().expect("in-process subscription") + Consumer::from_track(track, config) + .now_or_never() + .expect("in-process subscription") } fn drain(mut consumer: Consumer) -> Vec { diff --git a/rs/moq-net-tests/tests/integration/refresh.rs b/rs/moq-net-tests/tests/integration/refresh.rs index 886bc7df6c1..45659e063d7 100644 --- a/rs/moq-net-tests/tests/integration/refresh.rs +++ b/rs/moq-net-tests/tests/integration/refresh.rs @@ -75,8 +75,7 @@ async fn json_snapshot_refresh_crosses_a_two_relay_cluster() { json!({ "n": n }) ); if n == 1 || n == 2 * u64::from(ratio) { - let mut join = - snapshot::Consumer::::new(remote.clone(), Default::default()); + let mut join = snapshot::Consumer::::new(remote.clone(), Default::default()); let received = join.latest().await.unwrap().unwrap(); assert_eq!((received.value, received.at), (value(n), Some(at(n)))); assert!(producer.poll_refresh(&kio::Waiter::noop()).is_pending()); diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index da09553e5ce..7dd0a2a377d 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -1574,9 +1574,7 @@ mod tests { }; publish(vec![Path::from_static("small")], 1); - let track = consumer - .track("t.json") - .unwrap(); + let track = consumer.track("t.json").unwrap(); let mut reader = moq_json::snapshot::Consumer::::new(track, Default::default()); let mut next = || { reader @@ -1795,9 +1793,7 @@ mod tests { /// The latest buffered totals frame. async fn read_totals(broadcast: &moq_net::broadcast::Consumer) -> TotalsFrame { - let track = broadcast - .track(crate::TOTALS) - .expect("track"); + let track = broadcast.track(crate::TOTALS).expect("track"); let mut reader = moq_json::snapshot::Consumer::::new(track, Default::default()); reader.latest().await.expect("read").expect("frame").value } @@ -2341,7 +2337,11 @@ mod tests { /// Subscribe to `label`'s presence track on `broadcast`, driving a tick so /// the producer adopts the request. async fn hold_presence(broadcast: &moq_net::broadcast::Consumer, label: &str) -> Reader { - let reader = Reader::new(broadcast.track(&presence_track(moq_net::Path::new(label).unwrap())).expect("track")); + let reader = Reader::new( + broadcast + .track(&presence_track(moq_net::Path::new(label).unwrap())) + .expect("track"), + ); drive_tick().await; reader } @@ -2422,7 +2422,11 @@ mod tests { let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); // Let the request reach the stats broadcast's queue before the drain. - let mut track = Reader::new(broadcast.track(&presence_track(moq_net::Path::from_static(""))).expect("track")); + let mut track = Reader::new( + broadcast + .track(&presence_track(moq_net::Path::from_static(""))) + .expect("track"), + ); for _ in 0..8 { tokio::task::yield_now().await; } @@ -2474,7 +2478,11 @@ mod tests { /// Subscribe to `path`'s prefix track on `broadcast`, driving a tick so the /// producer adopts the request. async fn hold_prefix(broadcast: &moq_net::broadcast::Consumer, path: &str) -> Reader { - let reader = Reader::new(broadcast.track(&prefix_track(moq_net::Path::new(path).unwrap())).expect("track")); + let reader = Reader::new( + broadcast + .track(&prefix_track(moq_net::Path::new(path).unwrap())) + .expect("track"), + ); drive_tick().await; reader } @@ -2669,7 +2677,11 @@ mod tests { }; let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); - let mut track = Reader::new(broadcast.track(&prefix_track(moq_net::Path::from_static("acme"))).expect("track")); + let mut track = Reader::new( + broadcast + .track(&prefix_track(moq_net::Path::from_static("acme"))) + .expect("track"), + ); for _ in 0..8 { tokio::task::yield_now().await; } diff --git a/rs/moq-transcode/benches/ladder.rs b/rs/moq-transcode/benches/ladder.rs index 73ae3973e55..0075767710e 100644 --- a/rs/moq-transcode/benches/ladder.rs +++ b/rs/moq-transcode/benches/ladder.rs @@ -223,7 +223,7 @@ fn flapping(c: &mut Criterion) { Err(err) => panic!("catalog track: {err}"), } }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track.subscribe(None).await.unwrap()); + let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(track); let current = loop { let snapshot = catalogs.next().await.unwrap().unwrap(); if !snapshot.video.renditions.is_empty() { diff --git a/test/interop/clients/js-native/catalog.ts b/test/interop/clients/js-native/catalog.ts index 23a09ba5105..b5185b88f7c 100644 --- a/test/interop/clients/js-native/catalog.ts +++ b/test/interop/clients/js-native/catalog.ts @@ -34,18 +34,22 @@ try { await request.active.changed(); broadcast = request.active.peek(); } - const track = broadcast.track(Catalog.TRACK).subscribe({ priority: Catalog.PRIORITY.catalog }); + const track = broadcast.track(Catalog.TRACK); + const consumer = new Json.Snapshot.Consumer({ + track, + schema: Catalog.RootSchema, + priority: Catalog.PRIORITY.catalog, + }); try { const { compress } = await track.info(); assert.equal(compress, version === "moq-lite-07-wip", "wrong compression metadata"); - const consumer = new Json.Snapshot.Consumer({ track, schema: Catalog.RootSchema }); const root = (await consumer.latest())?.value; assert(root, "catalog ended before a snapshot"); const video = Object.keys(root.video?.renditions ?? {}).sort(); const audio = Object.keys(root.audio?.renditions ?? {}).sort(); console.error(`catalog compress=${compress} video=${video.join(",")} audio=${audio.join(",")}`); } finally { - track.close(); + consumer.close(); } } finally { request.close(); diff --git a/test/interop/clients/js-native/subscribe.ts b/test/interop/clients/js-native/subscribe.ts index 58b385c1f62..c8d23fb92f8 100644 --- a/test/interop/clients/js-native/subscribe.ts +++ b/test/interop/clients/js-native/subscribe.ts @@ -93,8 +93,11 @@ if ( async function catalog( bc: Moq.Broadcast.Consumer, ): Promise<{ video: [string, Catalog.VideoConfig]; audio: [string, Catalog.AudioConfig][] }> { - const track = bc.track("catalog.json").subscribe({ priority: Catalog.PRIORITY.catalog }); - const catalog = new Json.Snapshot.Consumer({ track, schema: Catalog.RootSchema }); + const catalog = new Json.Snapshot.Consumer({ + track: bc.track("catalog.json"), + schema: Catalog.RootSchema, + priority: Catalog.PRIORITY.catalog, + }); for (;;) { const root = (await catalog.latest())?.value; if (!root) throw new Error("catalog ended without a video and an audio track"); From fef07d9a50a15f923d630dc895b5e671eccdeb8d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 23:48:45 -0800 Subject: [PATCH 17/24] fix: box the snapshot reader; format Kotlin Co-Authored-By: Claude Opus 5.5 --- kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt | 4 ++-- rs/moq-json/src/snapshot/consumer.rs | 7 ++++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt index 24a6f4c8164..8804be16c2c 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/json/Tracks.kt @@ -100,8 +100,8 @@ inline fun WindowEventPush.decode(): T = /** * Stream of each JSON state from a snapshot track, in order. * - * Skips every group a newer one supersedes, at the zero `delay.max` the consumer subscribes with. Call - * [SnapshotConsumer.latest] to skip to the newest state. + * Skips every group a newer one supersedes, at the zero `delay.max` the consumer subscribes with. + * Call [SnapshotConsumer.latest] to skip to the newest state. */ fun SnapshotConsumer.values(): Flow = pull({ cancel() }) { next() } diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index 3f4f8a5402f..16a910f8ee7 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -186,7 +186,8 @@ impl Consumer { /// The consumer's own subscription, resolved by the first read. enum Track { Subscribing(moq_net::track::Subscribing), - Ordered(moq_net::track::Ordered), + // Boxed: the reader dwarfs the pending subscription. + Ordered(Box), } impl Track { @@ -195,13 +196,13 @@ impl Track { if let Self::Subscribing(pending) = self && let Poll::Ready(Ok(subscriber)) = pending.poll_ok(waiter) { - *self = Self::Ordered(subscriber.ordered()); + *self = Self::Ordered(Box::new(subscriber.ordered())); } } fn poll_next_group(&mut self, waiter: &kio::Waiter) -> Poll>> { if let Self::Subscribing(pending) = self { - *self = Self::Ordered(std::task::ready!(pending.poll_ok(waiter))?.ordered()); + *self = Self::Ordered(Box::new(std::task::ready!(pending.poll_ok(waiter))?.ordered())); } match self { Self::Ordered(track) => track.poll_next_group(waiter), From 7ee45dd8f5bc76a9b52359484a1d9ad77d1a37b8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 23:56:57 -0800 Subject: [PATCH 18/24] fix(mux): box both JSON readers Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/json.rs | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index a54a7cefd71..fe4711723ab 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -352,8 +352,9 @@ pub struct Consumer { /// Which moq-json consumer is doing the reading. Private: the caller sees one `Consumer`. enum Inner { - Snapshot(moq_json::snapshot::Consumer), - Stream(moq_json::stream::Consumer), + // Boxed: the two readers differ in size by hundreds of bytes. + Snapshot(Box>), + Stream(Box>), } impl Consumer { @@ -369,11 +370,11 @@ impl Consumer { /// must skip rather than guess at. pub async fn from_track(track: moq_net::track::Consumer, config: &JsonConfig) -> crate::Result { let inner = match &config.mode { - Mode::Snapshot => Inner::Snapshot(moq_json::snapshot::Consumer::new(track, Default::default())), - Mode::Stream => Inner::Stream(moq_json::stream::Consumer::new( + Mode::Snapshot => Inner::Snapshot(Box::new(moq_json::snapshot::Consumer::new(track, Default::default()))), + Mode::Stream => Inner::Stream(Box::new(moq_json::stream::Consumer::new( track.subscribe(None).await?, Default::default(), - )), + ))), other => return Err(crate::Error::UnsupportedMode(other.to_string())), }; From 6b170a4a845a9bbf40889f6f4dc9ea6e2320193f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 00:04:27 -0800 Subject: [PATCH 19/24] test(gst): a refused catalog surfaces from the catalog read Co-Authored-By: Claude Opus 5.5 --- rs/moq-gst/src/source/imp.rs | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index 0af4cabb21f..13a18a2033f 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -4989,8 +4989,12 @@ mod session_tests { .expect("the refusal held the session") .unwrap() .expect_err("the refusal did not fail the session"); + // The catalog consumer subscribes itself, so the refusal surfaces from its first read. assert!( - matches!(err.downcast_ref::(), Some(moq_net::Error::NotFound)), + matches!( + err.downcast_ref::(), + Some(moq_mux::Error::Moq(moq_net::Error::NotFound)) + ), "{err:?}" ); } From d5db3d268e40faf7d56c68c29f79bcfc5c072831 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 00:22:31 -0800 Subject: [PATCH 20/24] fix(mux): a refused hang catalog still fails catalog::Consumer::new Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/catalog/consumer.rs | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/rs/moq-mux/src/catalog/consumer.rs b/rs/moq-mux/src/catalog/consumer.rs index 1867091bb16..e31c4288384 100644 --- a/rs/moq-mux/src/catalog/consumer.rs +++ b/rs/moq-mux/src/catalog/consumer.rs @@ -51,9 +51,14 @@ impl Consumer { pub async fn new(broadcast: &moq_net::broadcast::Consumer, format: CatalogFormat) -> Result { Ok(Self { inner: match format { - CatalogFormat::Hang => State::Hang(super::hang::Consumer::new( - broadcast.track(hang::Catalog::DEFAULT_NAME)?, - )), + CatalogFormat::Hang => { + let track = broadcast.track(hang::Catalog::DEFAULT_NAME)?; + let consumer = super::hang::Consumer::new(track.clone()); + // The consumer subscribes itself; resolving the info rides on that subscription, + // so a refused catalog fails here as it does for MSF. + track.info().await?; + State::Hang(consumer) + } CatalogFormat::Msf => { let track = broadcast.track(moq_msf::DEFAULT_NAME)?.subscribe(None).await?; State::Msf(super::msf::Consumer::new(track)) From 9ff94dcf7e2edafeef961840d59b6337c8a1799e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 00:27:36 -0800 Subject: [PATCH 21/24] test(stats): wait for the held track's subscription before reading Co-Authored-By: Claude Opus 5.5 --- rs/moq-stats/src/produce.rs | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/rs/moq-stats/src/produce.rs b/rs/moq-stats/src/produce.rs index 7dd0a2a377d..f61d5a6f7d4 100644 --- a/rs/moq-stats/src/produce.rs +++ b/rs/moq-stats/src/produce.rs @@ -2422,15 +2422,15 @@ mod tests { let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); // Let the request reach the stats broadcast's queue before the drain. - let mut track = Reader::new( - broadcast - .track(&presence_track(moq_net::Path::from_static(""))) - .expect("track"), - ); + let handle = broadcast + .track(&presence_track(moq_net::Path::from_static(""))) + .expect("track"); + let mut track = Reader::new(handle.clone()); for _ in 0..8 { tokio::task::yield_now().await; } assert_eq!(tick(), 1, "held"); + handle.info().await.expect("held"); assert_eq!(try_presence(&mut track), Some(PresenceFrame::new()), "held at zero"); // The label records while held: the same subscription carries it. @@ -2677,15 +2677,15 @@ mod tests { }; let broadcast = origin.consume().request(".stats/node", None).await.expect("resolve"); - let mut track = Reader::new( - broadcast - .track(&prefix_track(moq_net::Path::from_static("acme"))) - .expect("track"), - ); + let handle = broadcast + .track(&prefix_track(moq_net::Path::from_static("acme"))) + .expect("track"); + let mut track = Reader::new(handle.clone()); for _ in 0..8 { tokio::task::yield_now().await; } assert_eq!(tick(), 1, "held"); + handle.info().await.expect("held"); assert_eq!(last_prefix(&mut track), Some(PrefixFrame::new()), "held at zero"); let _feed = feed(®istry, Tier::default(), "acme/live", true, 1, 7).await; From e92330c11b2a4fdfa25a6a2855c92d38c99feb6c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 00:36:47 -0800 Subject: [PATCH 22/24] fix(json): a read in flight when the snapshot consumer closes ends cleanly Co-Authored-By: Claude Opus 5.5 --- js/json/src/snapshot/consumer.ts | 28 +++++++++++++++++++-------- js/json/src/snapshot/snapshot.test.ts | 17 ++++++++++++++++ 2 files changed, 37 insertions(+), 8 deletions(-) diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 95718a5ae91..59956016719 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -33,6 +33,7 @@ export class Consumer { #group?: Moq.Group.Consumer; #framesRead = 0; + #closed = false; constructor(config: Consumer.Config) { this.#track = config.track @@ -45,10 +46,11 @@ export class Consumer { this.#decoder = new Decoder(config); } - /** End the consumer's subscription, closing any pending read. Idempotent. */ + /** End the consumer's subscription; a pending or later read returns `undefined`. Idempotent. */ close(): void { + if (this.#closed) return; + this.#closed = true; this.#group?.close(); - this.#group = undefined; this.#track.close(); } @@ -78,6 +80,8 @@ export class Consumer { async #read(skip: boolean): Promise | undefined> { for (;;) { + // Checked again after every wait: a read that resumes after close ends, not reads on. + if (this.#closed) return undefined; if (skip) { // Jump to the newest group the reader already has; a newer one supersedes the rest. // Taking groups, not the track's largest position, ignores datagrams (which share @@ -93,33 +97,41 @@ export class Consumer { if (!this.#group) { // Advance to the next group with a higher sequence number (skipping late arrivals). - this.#group = await this.#track.nextGroup(); - if (!this.#group) return undefined; + const next = await this.#track.nextGroup(); + if (this.#closed) { + next?.close(); + return undefined; + } + if (!next) return undefined; + this.#group = next; // The next frame is the new group's snapshot, which also restarts the decoder's window. this.#framesRead = 0; } + const group = this.#group; let frame: Moq.Group.Frame | undefined; try { - frame = await this.#group.readFrame(); + frame = await group.readFrame(); } catch (err) { + if (this.#closed) return undefined; if (!(err instanceof Moq.Error.Stream && GAPS.includes(err.code))) throw err; // The group was reset or we fell behind its eviction window. Resync from the next // group, which begins with a fresh snapshot (frame 0), so no partial state is presented. - this.#group = undefined; + if (this.#group === group) this.#group = undefined; continue; } + if (this.#closed) return undefined; if (!frame) { // The group is exhausted; advance to the next one. - this.#group = undefined; + if (this.#group === group) this.#group = undefined; continue; } this.#apply(frame.payload); if (skip) { // Apply the rest of the backlog but yield only the head. - for (let more = this.#group.tryReadFrame(); more; more = this.#group.tryReadFrame()) { + for (let more = group.tryReadFrame(); more; more = group.tryReadFrame()) { this.#apply(more.payload); frame = more; } diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index 4d638d96a37..66463200674 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -238,6 +238,23 @@ test("reads skip a group the subscription still owes", async () => { } }); +// Closing the consumer ends a read already waiting, and every later one, without touching the +// group it was reading. +test("close ends a pending read", async () => { + const track = new Track.Producer("test"); + const producer = new Producer({ track }); + const consumer = new Consumer({ track }); + producer.update({ value: { a: 1 } }); + expect((await consumer.latest())?.value).toEqual({ a: 1 }); + + const pending = [consumer.latest(), consumer.next()]; + producer.update({ value: { a: 2 } }); + consumer.close(); + expect(await Promise.all(pending)).toEqual([undefined, undefined]); + expect(await consumer.latest()).toBeUndefined(); + producer.finish(); +}); + test("live consumer sees each update", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); From 685638c3d6e2261b7d8ae30a8452066bf224590d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 21:28:09 -0800 Subject: [PATCH 23/24] fix: snapshot priority through the FFI; serialize JS snapshot reads - moq-ffi's MoqJsonSnapshotConsumer::new takes a priority (default 0), forwarded to snapshot::consumer::Config::priority; Go gains SnapshotConsumerOptions, Python/Swift/Kotlin/Dart a defaulted priority. - JS Snapshot.Consumer runs overlapping reads in call order: a read parked on a group no longer resumes after a later read moved to a newer one. - Tests: the Rust consumer subscribes at its priority and a zero delay; overlapping JS reads; close releases the subscription's demand. - Go README no longer says every consumer takes only the track. Co-Authored-By: Claude Opus 5.5 --- dart/moq/MIGRATE.md | 2 +- dart/moq_ffi/lib/src/moq.dart | 12 ++++++++++-- go/wrapper/MIGRATE.md | 2 +- go/wrapper/README.md | 5 +++-- go/wrapper/json/json.go | 15 +++++++++++---- go/wrapper/json/json_test.go | 4 ++-- js/json/src/snapshot/consumer.ts | 19 ++++++++++++++----- js/json/src/snapshot/snapshot.test.ts | 21 +++++++++++++++++++++ kt/moq/MIGRATE.md | 2 +- py/moq-rs/MIGRATE.md | 2 +- py/moq-rs/moq/json.py | 9 +++++---- rs/moq-ffi/MIGRATE.md | 2 +- rs/moq-ffi/src/json.rs | 13 ++++++++----- rs/moq-ffi/src/test.rs | 6 +++--- rs/moq-json/src/snapshot/consumer.rs | 19 +++++++++++++++++++ swift/MIGRATE.md | 2 +- swift/README.md | 4 ++-- swift/Sources/Moq/Json.swift | 8 ++++---- 18 files changed, 108 insertions(+), 39 deletions(-) diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 0461ca95a49..bde762ab762 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -18,7 +18,7 @@ A null `TrackInfo.maxAge`, the default, and a null info now advertise a zero max ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`SnapshotConsumer(broadcast: ..., name: ...)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`SnapshotConsumer(broadcast: ..., name: ..., priority: ...)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max` and `priority` (default 0). Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index a7c2307e62e..ef6628b69ac 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -11599,10 +11599,12 @@ class MoqJsonSnapshotConsumer implements MoqJsonSnapshotConsumerInterface { MoqJsonSnapshotConsumer({ required MoqBroadcastConsumer broadcast, required String name, + int priority = 0, }) : _ptr = rustCall( (status) => uniffi_moq_ffi_fn_constructor_moqjsonsnapshotconsumer_new( FfiConverterMoqBroadcastConsumer.lower(broadcast), FfiConverterString.lower(name), + FfiConverterInt8.lower(priority), status, ), moqExceptionErrorHandler, @@ -20133,12 +20135,18 @@ external void uniffi_moq_ffi_fn_free_moqjsonsnapshotconsumer( ); @Native< - Pointer Function(Pointer, RustBuffer, Pointer) + Pointer Function( + Pointer, + RustBuffer, + Int8, + Pointer, + ) >(assetId: _uniffiAssetId) external Pointer uniffi_moq_ffi_fn_constructor_moqjsonsnapshotconsumer_new( Pointer broadcast, RustBuffer name, + int priority, Pointer uniffiStatus, ); @@ -24096,7 +24104,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqjsonsnapshotconsumer_new() != - 49887) { + 47962) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqjsonsnapshotproducer_new() != diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index 82bfb7a5689..bea18463d46 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -18,7 +18,7 @@ Replace stream-track `AppendDatagram` / `InsertDatagram` / `RecvDatagram` with ` ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`json.NewSnapshotConsumer(broadcast, name)` takes the `*BroadcastConsumer` and the track name instead of a `*TrackConsumer`, and subscribes to the track itself at a zero `Delay.Max`. Drop the `SubscribeTrack` call; a refused subscription now fails the first read. `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`json.NewSnapshotConsumer(broadcast, name, options)` takes the `*BroadcastConsumer`, the track name, and `SnapshotConsumerOptions` instead of a `*TrackConsumer`, and subscribes to the track itself at a zero `Delay.Max` and the options' `Priority`. Drop the `SubscribeTrack` call; a refused subscription now fails the first read. `Next` skips every group a newer one supersedes, and `Latest` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/go/wrapper/README.md b/go/wrapper/README.md index 16eed7c14c4..c459b043a99 100644 --- a/go/wrapper/README.md +++ b/go/wrapper/README.md @@ -119,7 +119,7 @@ Import it under an alias next to `encoding/json`. Each type wraps a track: `NewSnapshotProducer` / `NewStreamProducer` take over a `TrackProducer` and advertise it in the broadcast's catalog, `NewStreamConsumer` takes over a `TrackConsumer` that has not read a group yet, and `NewSnapshotConsumer` takes the `BroadcastConsumer` and track name and -subscribes itself at a zero `Delay.Max`. Snapshots carry lossy latest state; streams +subscribes itself at a zero `Delay.Max` and `SnapshotConsumerOptions.Priority`. Snapshots carry lossy latest state; streams carry every record in order. `NewWindowProducer` / `NewWindowConsumer` carry a sliding window of records a reader can join at any point; the catalog has no window mode, so the producer takes only the track. Producers accept any `encoding/json` value; consumers return @@ -149,7 +149,8 @@ _ = publication.Announce(moq.AnnounceRoute{}) ``` Snapshot and window producers take `SnapshotOptions` or `WindowOptions`; the stream producer and -every consumer take only the track, and a consumer reads a compressed track like a plain one. +the stream and window consumers take only the track. Every consumer reads a compressed track like a +plain one. ## Errors diff --git a/go/wrapper/json/json.go b/go/wrapper/json/json.go index 9c8a9577985..1af07d5f31f 100644 --- a/go/wrapper/json/json.go +++ b/go/wrapper/json/json.go @@ -155,11 +155,18 @@ type SnapshotConsumer struct { inner *ffi.MoqJsonSnapshotConsumer } +// SnapshotConsumerOptions configures a snapshot consumer's subscription. +type SnapshotConsumerOptions struct { + // Priority is the subscription's delivery priority, higher first, as + // moq.Subscription.Priority. + Priority int8 +} + // NewSnapshotConsumer subscribes to the track named name on broadcast and reads it as a -// lossy latest-value JSON track. The consumer owns its subscription, at a zero Delay.Max; -// a read waits for it to resolve and fails if it cannot. -func NewSnapshotConsumer(broadcast *moq.BroadcastConsumer, name string) (*SnapshotConsumer, error) { - inner, err := ffi.NewMoqJsonSnapshotConsumer(bridge.BroadcastConsumer(broadcast), name) +// lossy latest-value JSON track. The consumer owns its subscription, at a zero Delay.Max +// and the options' priority; a read waits for it to resolve and fails if it cannot. +func NewSnapshotConsumer(broadcast *moq.BroadcastConsumer, name string, options SnapshotConsumerOptions) (*SnapshotConsumer, error) { + inner, err := ffi.NewMoqJsonSnapshotConsumer(bridge.BroadcastConsumer(broadcast), name, options.Priority) if err != nil { return nil, err } diff --git a/go/wrapper/json/json_test.go b/go/wrapper/json/json_test.go index 82caedceb6a..d05c34f82ea 100644 --- a/go/wrapper/json/json_test.go +++ b/go/wrapper/json/json_test.go @@ -65,7 +65,7 @@ func TestTracks(t *testing.T) { if err != nil { t.Fatal(err) } - snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status") + snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status", moqjson.SnapshotConsumerOptions{}) if err != nil { t.Fatal(err) } @@ -246,7 +246,7 @@ func TestDemand(t *testing.T) { t.Fatalf("Wanted = %d before any subscriber", *wanted) } - snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status") + snapshotConsumer, err := moqjson.NewSnapshotConsumer(consumer, "status", moqjson.SnapshotConsumerOptions{}) if err != nil { t.Fatal(err) } diff --git a/js/json/src/snapshot/consumer.ts b/js/json/src/snapshot/consumer.ts index 59956016719..492c2f5339a 100644 --- a/js/json/src/snapshot/consumer.ts +++ b/js/json/src/snapshot/consumer.ts @@ -19,7 +19,8 @@ const GAPS: Moq.StreamCode[] = [ * yields each reconstructed value with the timestamp of the frame that produced it. Two reads: * {@link next} yields each state in order, skipping every group a newer one supersedes, * and {@link latest} skips to the newest state, for a caller that only wants the current value. - * What {@link latest} skips is gone, so a later {@link next} resumes after it. + * What {@link latest} skips is gone, so a later {@link next} resumes after it. Overlapping reads + * take turns in call order. * When something else already owns the track, use the {@link Decoder} directly. * * A snapshot is superseded the moment a newer one exists, so the consumer subscribes itself at a @@ -34,6 +35,8 @@ export class Consumer { #group?: Moq.Group.Consumer; #framesRead = 0; #closed = false; + // The read in flight: reads share the group and decoder, so each starts once the last settles. + #reading: Promise = Promise.resolve(); constructor(config: Consumer.Config) { this.#track = config.track @@ -62,7 +65,7 @@ export class Consumer { * from the next one. */ next(): Promise | undefined> { - return this.#read(false); + return this.#queue(false); } /** @@ -75,7 +78,13 @@ export class Consumer { * when nothing newer than the last yield has arrived. */ latest(): Promise | undefined> { - return this.#read(true); + return this.#queue(true); + } + + #queue(skip: boolean): Promise | undefined> { + const read = this.#reading.then(() => this.#read(skip)); + this.#reading = read.catch(() => undefined); + return read; } async #read(skip: boolean): Promise | undefined> { @@ -117,14 +126,14 @@ export class Consumer { if (!(err instanceof Moq.Error.Stream && GAPS.includes(err.code))) throw err; // The group was reset or we fell behind its eviction window. Resync from the next // group, which begins with a fresh snapshot (frame 0), so no partial state is presented. - if (this.#group === group) this.#group = undefined; + this.#group = undefined; continue; } if (this.#closed) return undefined; if (!frame) { // The group is exhausted; advance to the next one. - if (this.#group === group) this.#group = undefined; + this.#group = undefined; continue; } diff --git a/js/json/src/snapshot/snapshot.test.ts b/js/json/src/snapshot/snapshot.test.ts index f390da0e1cb..2d037330da0 100644 --- a/js/json/src/snapshot/snapshot.test.ts +++ b/js/json/src/snapshot/snapshot.test.ts @@ -251,10 +251,31 @@ test("close ends a pending read", async () => { producer.update({ value: { a: 2 } }); consumer.close(); expect(await Promise.all(pending)).toEqual([undefined, undefined]); + // The subscription the consumer owned is gone. + expect(track.demand().wanted.peek()).toBeUndefined(); expect(await consumer.latest()).toBeUndefined(); producer.finish(); }); +// Reads share the consumer's group and decoder, so a read waiting on a group must finish before a +// later read moves on: otherwise the old group's delta lands on the newer group's snapshot. +test("overlapping reads take turns", async () => { + const track = new Track.Producer("test"); + const producer = new Producer({ track }); + const consumer = new Consumer({ track }); + producer.update({ value: { a: 1, keep: true } }); + expect((await consumer.next())?.value).toEqual({ a: 1, keep: true }); + + const next = consumer.next(); + producer.update({ value: { a: 2, keep: true } }); + producer.cut(); + producer.update({ value: { b: 3 } }); + const latest = consumer.latest(); + expect((await next)?.value).toEqual({ a: 2, keep: true }); + expect((await latest)?.value).toEqual({ b: 3 }); + producer.finish(); +}); + test("live consumer sees each update", async () => { const track = new Track.Producer("test"); const producer = new Producer({ track }); diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 855d2aed1d5..92423abd12b 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -18,7 +18,7 @@ A null `TrackInfo.maxAge`, the default, and a null info now advertise a zero max ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`SnapshotConsumer(broadcast, name)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`SnapshotConsumer(broadcast, name, priority)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max` and `priority` (default 0). Drop the `subscribeTrack` call; a refused subscription now throws from the first read. `values()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 8b15dc602a2..9fdbbcff993 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -18,7 +18,7 @@ Replace stream-track `append_datagram` / `insert_datagram` / `recv_datagram` wit ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`moq.json.SnapshotConsumer(broadcast, name)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribe_track` call; a refused subscription now raises from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`moq.json.SnapshotConsumer(broadcast, name, *, priority=0)` takes the `BroadcastConsumer` and the track name instead of a `TrackConsumer`, and subscribes to the track itself at a zero `delay.max` and `priority`. Drop the `subscribe_track` call; a refused subscription now raises from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/py/moq-rs/moq/json.py b/py/moq-rs/moq/json.py index 0b4c157b6ea..eab20bfbca4 100644 --- a/py/moq-rs/moq/json.py +++ b/py/moq-rs/moq/json.py @@ -136,12 +136,13 @@ class SnapshotConsumer(_Stream[Timed]): Each item is a :class:`Timed`. Every group a newer one supersedes is skipped; :meth:`latest` skips to the newest state instead. The consumer subscribes to the track ``name`` on - ``broadcast`` itself, at a zero ``delay.max``; a read waits for that subscription and raises if - it fails. Usable as an async context manager that cancels on exit. + ``broadcast`` itself, at a zero ``delay.max`` and ``priority`` (higher first); a read waits for + that subscription and raises if it fails. Usable as an async context manager that cancels on + exit. """ - def __init__(self, broadcast: subscribe.BroadcastConsumer, name: str) -> None: - self._inner = MoqJsonSnapshotConsumer(broadcast._inner, name) + def __init__(self, broadcast: subscribe.BroadcastConsumer, name: str, *, priority: int = 0) -> None: + self._inner = MoqJsonSnapshotConsumer(broadcast._inner, name, priority) async def _next(self) -> Timed | None: return Timed._from_ffi(await self._inner.next()) diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index 295d074a3ba..d5927e2007d 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -21,7 +21,7 @@ A received track's `info()` sets `max_age_unlimited` when its publisher advertis ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`MoqJsonSnapshotConsumer::new(broadcast, name)` takes a `MoqBroadcastConsumer` and the track name instead of a `MoqTrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribe_track` call; a refused subscription now fails the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. +`MoqJsonSnapshotConsumer::new(broadcast, name, priority)` takes a `MoqBroadcastConsumer`, the track name, and the subscription's priority (default 0) instead of a `MoqTrackConsumer`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribe_track` call; a refused subscription now fails the first read. `next()` skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/rs/moq-ffi/src/json.rs b/rs/moq-ffi/src/json.rs index 86ca7d7438c..9bbe24a2319 100644 --- a/rs/moq-ffi/src/json.rs +++ b/rs/moq-ffi/src/json.rs @@ -231,12 +231,15 @@ impl MoqJsonSnapshotConsumer { /// Subscribe to the track `name` on `broadcast` and read it as a JSON snapshot track (lossy /// latest-value). /// - /// The consumer owns its subscription, at a zero `delay.max`: a read waits for it to resolve and - /// fails if it cannot. Reads a compressed track the same as a plain one. - #[uniffi::constructor] - pub fn new(broadcast: &MoqBroadcastConsumer, name: String) -> Result, MoqError> { + /// The consumer owns its subscription, at a zero `delay.max` and `priority` (higher first, as + /// [`MoqSubscription::priority`](crate::consumer::MoqSubscription::priority)): a read waits for + /// it to resolve and fails if it cannot. Reads a compressed track the same as a plain one. + #[uniffi::constructor(default(priority = 0))] + pub fn new(broadcast: &MoqBroadcastConsumer, name: String, priority: i8) -> Result, MoqError> { let _guard = crate::ffi::enter(); - let consumer = moq_json::snapshot::Consumer::::new(broadcast.inner.track(&name)?, Default::default()); + let mut config = moq_json::snapshot::consumer::Config::default(); + config.priority = priority; + let consumer = moq_json::snapshot::Consumer::::new(broadcast.inner.track(&name)?, config); Ok(Arc::new(Self { reads: Polled::new(consumer).supervised(broadcast.driver.clone()), })) diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index dc6a6b93a54..6bc3be5e637 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -1473,7 +1473,7 @@ async fn json_snapshot_roundtrip() { matches!(track.demand(), Err(MoqError::Closed)), "the producer takes over the track" ); - let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into()).unwrap(); + let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into(), 0).unwrap(); for n in 1..=4 { producer.update(format!(r#"{{"a":{n}}}"#), us(n * 1000)).unwrap(); @@ -1563,7 +1563,7 @@ async fn json_snapshot_refreshes_for_a_joiner() { .await .expect("timed out waiting for the refresh"); assert_eq!(refreshed.sequence(), 1); - let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into()).unwrap(); + let consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "meta".into(), 0).unwrap(); let value = kio::clock::timeout(TIMEOUT, consumer.next()) .await .expect("timed out waiting for the restated value") @@ -1592,7 +1592,7 @@ async fn json_demand() { assert_eq!(stream_demand.name(), "events"); assert!(!snapshot_demand.wanted().is_some()); - let snapshot_consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "status".into()).unwrap(); + let snapshot_consumer = MoqJsonSnapshotConsumer::new(&broadcast.consume().unwrap(), "status".into(), 0).unwrap(); let stream_consumer = MoqJsonStreamConsumer::new(&*subscribe(&broadcast, "events").await).unwrap(); kio::clock::timeout(TIMEOUT, snapshot_demand.until_wanted()) diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index d5cfa002b69..9ecef6e1ed6 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -211,6 +211,25 @@ impl Track { } } +#[cfg(test)] +mod subscription { + use super::*; + + #[test] + fn subscribes_at_the_configured_priority_and_zero_delay() { + let track = moq_net::broadcast::Info::new() + .produce() + .create_track("data", moq_net::track::Info::default()) + .unwrap(); + let mut config = Config::default(); + config.priority = 5; + let _consumer = Consumer::::new(track.consume(), config); + let subscription = track.subscription().expect("the consumer subscribed"); + assert_eq!(subscription.priority, 5); + assert_eq!(subscription.delay.max, Some(std::time::Duration::ZERO)); + } +} + #[cfg(test)] mod timestamps { use super::*; diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 06291b4a74b..a0d98402281 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -18,7 +18,7 @@ A `nil` `TrackInfo.maxAge`, the default, and a `nil` info now advertise a zero m ### Snapshot consumers subscribe themselves at a zero delay ([#7736](https://github.com/moq-dev/moq/pull/7736)) -`Json.SnapshotConsumer(broadcast:name:)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max`. Drop the `subscribeTrack` call; a refused subscription now throws from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. +`Json.SnapshotConsumer(broadcast:name:priority:)` takes the `BroadcastConsumer` and the track name instead of `track:`, and subscribes to the track itself at a zero `delay.max` and `priority` (default 0). Drop the `subscribeTrack` call; a refused subscription now throws from the first read. Iteration skips every group a newer one supersedes, and `latest()` no longer waits for a lower group the track still owes. Publish a stream track when every state must survive, or a window track for a bounded recent history. ### Viewer targets are bounded ([#7090](https://github.com/moq-dev/moq/pull/7090)) diff --git a/swift/README.md b/swift/README.md index 24dc5563605..ce520f854d4 100644 --- a/swift/README.md +++ b/swift/README.md @@ -94,8 +94,8 @@ not found. lite-07 uses independent requests. JSON tracks live under the `Json` namespace, mirroring the `moq-json` crate, and carry your own `Codable` types with the framing handled for you. Each type wraps a track: a producer takes over a `TrackProducer` and advertises it in the broadcast's catalog, and a consumer takes over a -`TrackConsumer`, except `Json.SnapshotConsumer(broadcast:name:)`, which subscribes itself at a zero -`delay.max`. You opt into one of three distinct modes: `Json.SnapshotProducer`/`Json.SnapshotConsumer` +`TrackConsumer`, except `Json.SnapshotConsumer(broadcast:name:priority:)`, which subscribes itself at a +zero `delay.max`. You opt into one of three distinct modes: `Json.SnapshotProducer`/`Json.SnapshotConsumer` (lossy latest-value, a subscriber only sees the newest value), `Json.StreamProducer`/`Json.StreamConsumer` (lossless append-log, every record preserved in order), or `Json.WindowProducer`/`Json.WindowConsumer` (a sliding window of records a reader can join at any point, not advertised in the catalog). Values encode/decode at the boundary with diff --git a/swift/Sources/Moq/Json.swift b/swift/Sources/Moq/Json.swift index f5772063cfc..c9e8eb3bcab 100644 --- a/swift/Sources/Moq/Json.swift +++ b/swift/Sources/Moq/Json.swift @@ -91,10 +91,10 @@ extension Json { let ffi: MoqJsonSnapshotConsumer /// Subscribe to the track `name` on `broadcast` and read it as a JSON snapshot track. - /// The consumer owns its subscription, at a zero `delay.max`; a read waits for it to - /// resolve and throws if it cannot. - public init(broadcast: BroadcastConsumer, name: String) throws { - ffi = try MoqJsonSnapshotConsumer(broadcast: broadcast.ffi, name: name) + /// The consumer owns its subscription, at a zero `delay.max` and `priority` (higher + /// first); a read waits for it to resolve and throws if it cannot. + public init(broadcast: BroadcastConsumer, name: String, priority: Int8 = 0) throws { + ffi = try MoqJsonSnapshotConsumer(broadcast: broadcast.ffi, name: name, priority: priority) } /// The next state in order, or `nil` once the track ends or is closed. From 440363336847eb7138207c699388d5175b8c8a29 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 21:33:43 -0800 Subject: [PATCH 24/24] fix: port main's new stats test to the self-subscribing consumer; clippy Co-Authored-By: Claude Opus 5.5 --- js/publish/src/broadcast.test.ts | 11 +++++------ rs/moq-json/src/snapshot/consumer.rs | 6 ++++-- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/js/publish/src/broadcast.test.ts b/js/publish/src/broadcast.test.ts index 9075f8b43dc..958711b2a03 100644 --- a/js/publish/src/broadcast.test.ts +++ b/js/publish/src/broadcast.test.ts @@ -117,19 +117,18 @@ test("shared Rust stats names survive the publisher's report builder", async () await settle(); const net = broadcast.net.peek(); if (!net) throw new Error("expected a network producer"); - const subscriber = net.track(Broadcast.STATS_TRACK).subscribe(); + const consumer = new Json.Snapshot.Consumer({ + track: net.track(Broadcast.STATS_TRACK), + schema: Stats.SnapshotSchema, + }); try { - const consumer = new Json.Snapshot.Consumer({ - track: subscriber, - schema: Stats.SnapshotSchema, - }); await settle(); await clock.advance(Nano.fromMilli(Milli(1000))); const reported = (await consumer.latest())?.value.renditions ?? {}; expect(Object.keys(reported).sort()).toEqual(Object.keys(fixture.renditions ?? {}).sort()); expect(reported).toEqual(fixture.renditions ?? {}); } finally { - subscriber.close(); + consumer.close(); } } finally { broadcast.close(); diff --git a/rs/moq-json/src/snapshot/consumer.rs b/rs/moq-json/src/snapshot/consumer.rs index 9ecef6e1ed6..909ba7e9408 100644 --- a/rs/moq-json/src/snapshot/consumer.rs +++ b/rs/moq-json/src/snapshot/consumer.rs @@ -221,8 +221,10 @@ mod subscription { .produce() .create_track("data", moq_net::track::Info::default()) .unwrap(); - let mut config = Config::default(); - config.priority = 5; + let config = Config { + priority: 5, + ..Default::default() + }; let _consumer = Consumer::::new(track.consume(), config); let subscription = track.subscription().expect("the consumer subscribed"); assert_eq!(subscription.priority, 5);