diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index c2121e9f3ff..8db354e6654 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -38,6 +38,7 @@ broadcast.announce(); - **One connection per URL, unless you opt out.** `new Connection({ url })` pools and reconnects with backoff, which the elements use. Supplying your own transport options, discovery, or origin selects a private loop. - **GOAWAY moves the session.** The connection dials the replacement at once while the old session keeps serving its groups, up to `goaway.handover` (default 10s, or the relay's deadline when that is sooner). New requests keep opening on the old session until the replacement's route outranks it. An empty GOAWAY redials the same URL. A redirect stays on the same host unless `goaway.redirect` is `"follow"`. - **One send estimate per connection.** `Bandwidth.Allocator` divides it by track priority, max-min fair within a tier. An idle track claims nothing. Publishers reserve against it so their targets sum to the estimate. +- **One track per name.** Concurrent subscriptions share one request and producer. An on-demand producer that replaces an ended one continues the name's group and datagram sequences; only a new broadcast restarts them. `createTrack` and `insertTrack` start at 0. - **Timing is per track, with no default.** A track that declares a `timescale` carries a `timestamp` on every frame; one without is [untimed](/concept/moq-lite#subscriptions) and its frames carry none. Unlike Rust, omitting it means untimed. - **Subscriber staleness is `maxDelay`.** It is media time. Passing the old `maxAge` key throws a `TypeError` naming `maxDelay`. Publisher retention is the separate `Track.Info.maxAge`. - **Hidden paths** stay out of discovery unless the announce request opts in. See [hidden broadcasts](/concept/moq-lite#hidden-broadcasts). diff --git a/doc/lib/rs/moq-net.md b/doc/lib/rs/moq-net.md index 57029117b23..b06d663b8f0 100644 --- a/doc/lib/rs/moq-net.md +++ b/doc/lib/rs/moq-net.md @@ -17,6 +17,7 @@ What you use it for, beyond what the concept page already describes: - **An origin outlives the session.** Broadcasts are created on the origin, and a reconnect announces them again. Closing the session does not delete them. - **Requests can pin an epoch.** `consumer.request_broadcast(path, Some(epoch))` resolves only through a route announcing that [publisher epoch](/concept/moq-lite#publisher-epochs); `None` takes whichever route wins. The resolved consumer names its epoch in `info().epoch`, pinned or not. +- **One track per name.** Concurrent subscriptions share one request and producer. A producer that replaces an ended one continues the name's group and datagram sequences; only a new broadcast restarts them. - **Publish only while someone is watching.** `demand()` on a track, group, or broadcast says whether a subscriber is attached, which is how capture and transcode skip work nobody asked for. Holding a broadcast consumer is not demand. A shared fetch stays up until its last reader leaves. - **You drive the session, or `moq-tokio` does.** `connect` and `accept` return a session plus a driver that never reads the clock itself. `moq_net::time::run` polls it on tokio or in the browser. `moq-tokio` and `moq-wasm` hide that. A custom transport implements `moq_net::transport::poll`. - **A session caps what its peer can hold.** `session::Limits` (via `Client::with_limits` and `Server::with_limits`) bounds announces and subscriptions per session; the defaults suit a relay mesh, so lower them for untrusted peers. Past either, the session closes with `TOO_MANY_REQUESTS`. On moq-transport drafts 14 to 16 they also size the request-ID window, which every request counts against, so very low limits can starve it. Peer-declared lengths are capped before they are buffered. diff --git a/js/net/src/broadcast.test.ts b/js/net/src/broadcast.test.ts index cfeac3e8c3c..a50fdf4cfd0 100644 --- a/js/net/src/broadcast.test.ts +++ b/js/net/src/broadcast.test.ts @@ -92,102 +92,70 @@ test("dynamic track sequences continue across producer replacements", async () = nextGeneration.close(); }); -test("concurrent dynamic producers share a sequence namespace", async () => { +test("publishing-side subscriptions coalesce onto one request", async () => { const broadcast = new BroadcastProducer(); const pulled = wireOf(broadcast).requested(); - const firstSubscriber = broadcast.track("media").subscribe(); - const secondSubscriber = broadcast.track("media").subscribe(); - const firstRequest = await pulled; - const secondRequest = await wireOf(broadcast).requested(); - if (!firstRequest || !secondRequest) throw new Error("expected requests"); - const firstProducer = firstRequest.accept({ timescale: Timescale.MILLI }); - const secondProducer = secondRequest.accept({ timescale: Timescale.MILLI }); + const info = broadcast.track("media").info(); + const a = broadcast.track("media").subscribe().ordered(); + const b = broadcast.track("media").subscribe().ordered(); - expect(firstProducer.appendGroup().sequence).toBe(0); - expect(secondProducer.appendGroup().sequence).toBe(1); - expect(firstProducer.appendGroup().sequence).toBe(2); - - firstSubscriber.close(); - secondSubscriber.close(); - firstProducer.close(); - secondProducer.close(); - broadcast.close(); -}); - -test("a sibling producer's groups do not settle an aborted end", async () => { - const broadcast = new BroadcastProducer(); - const pulled = wireOf(broadcast).requested(); - const firstSubscriber = broadcast.track("media").subscribe(); - const secondSubscriber = broadcast.track("media").subscribe(); - const firstRequest = await pulled; - const secondRequest = await wireOf(broadcast).requested(); - if (!firstRequest || !secondRequest) throw new Error("expected requests"); - const firstProducer = firstRequest.accept({ timescale: Timescale.MILLI }); - const secondProducer = secondRequest.accept({ timescale: Timescale.MILLI }); - - const reader = firstProducer.subscribe(); - firstProducer.finishAt(3); - for (let i = 0; i < 3; i++) secondProducer.appendGroup().close(); - const boom = new Error("boom"); - firstProducer.close(boom); + // One request serves the info lookup and every subscriber. + const request = await pulled; + expect(request?.name).toBe("media"); + expect(await pendingRequest(broadcast)).toBeUndefined(); - // The first producer never received the groups its end promised, so it was cut off. - expect(firstProducer.closed.peek()).toBe(boom); - await expect(reader.recvGroup()).rejects.toBe(boom); + if (!request) throw new Error("expected request"); + const producer = request.accept({ timescale: Timescale.MILLI }); + producer.writeString("hello"); + expect(await a.readString()).toBe("hello"); + expect(await b.readString()).toBe("hello"); + await info; + // The lookup leaves the shared producer to its subscribers, and stops counting as demand. + expect(producer.closed.peek()).toBeUndefined(); + a.close(); + b.close(); + await broadcast.demand().unused(); + expect(producer.closed.peek()).toBeUndefined(); - firstSubscriber.close(); - secondSubscriber.close(); - secondProducer.close(); + producer.close(); broadcast.close(); }); -test("a clean close ends at the track's own last group, not a sibling's", async () => { +test("an info lookup joining a subscription's request stays demand after the subscriber leaves", async () => { const broadcast = new BroadcastProducer(); + const demand = broadcast.demand(); const pulled = wireOf(broadcast).requested(); - const firstSubscriber = broadcast.track("media").subscribe(); - const secondSubscriber = broadcast.track("media").subscribe(); - const firstRequest = await pulled; - const secondRequest = await wireOf(broadcast).requested(); - if (!firstRequest || !secondRequest) throw new Error("expected requests"); - const firstProducer = firstRequest.accept({ timescale: Timescale.MILLI }); - const secondProducer = secondRequest.accept({ timescale: Timescale.MILLI }); - - const reader = firstProducer.subscribe(); - firstProducer.appendGroup().close(); - for (let i = 0; i < 3; i++) secondProducer.appendGroup().close(); - firstProducer.close(); + const subscriber = broadcast.track("media").subscribe(); + const info = broadcast.track("media").info(); + const request = await pulled; + if (!request) throw new Error("expected request"); - // The sibling took 1..3, which the first producer will never send. - expect(reader.final()).toBe(1); + // Flush every pending notification, so only the lookup is left holding demand. + subscriber.close(); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(request.demand().used.peek()).toBe(false); + expect(demand.used.peek()).toBe(true); - firstSubscriber.close(); - secondSubscriber.close(); - secondProducer.close(); + request.accept({ timescale: Timescale.MILLI }); + await info; + await demand.unused(); broadcast.close(); }); -test("finishAt accepts an end below a sibling's groups", async () => { +test("an info lookup releases a request nobody subscribed to", async () => { const broadcast = new BroadcastProducer(); const pulled = wireOf(broadcast).requested(); - const firstSubscriber = broadcast.track("media").subscribe(); - const secondSubscriber = broadcast.track("media").subscribe(); - const firstRequest = await pulled; - const secondRequest = await wireOf(broadcast).requested(); - if (!firstRequest || !secondRequest) throw new Error("expected requests"); - const firstProducer = firstRequest.accept({ timescale: Timescale.MILLI }); - const secondProducer = secondRequest.accept({ timescale: Timescale.MILLI }); - - const reader = firstProducer.subscribe(); - firstProducer.appendGroup().close(); - for (let i = 0; i < 3; i++) secondProducer.appendGroup().close(); - expect(() => firstProducer.finishAt(0)).toThrow("track end 0 is below the next sequence 1"); - firstProducer.finishAt(2); - expect(reader.final()).toBe(2); + const info = broadcast.track("media").info(); + const request = await pulled; + if (!request) throw new Error("expected request"); + const producer = request.accept(); + await info; + expect(producer.closed.peek()).toBeDefined(); - firstSubscriber.close(); - secondSubscriber.close(); - firstProducer.close(); - secondProducer.close(); + // The next subscription asks again. + const subscriber = broadcast.track("media").subscribe(); + expect((await pendingRequest(broadcast))?.name).toBe("media"); + subscriber.close(); broadcast.close(); }); diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index 51f78e2cbcb..29b05a6bba7 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -12,7 +12,7 @@ import { hooks, type TrackSequence } from "./internal.ts"; import * as Path from "./path.ts"; import * as track from "./track.ts"; import { untilAborted } from "./util/abort.ts"; -import { registerWire, trackOf, type Broadcast as Wire } from "./wire.ts"; +import { registerWire, trackOf } from "./wire.ts"; /** The origin callback a created broadcast uses to advertise its exact path. @internal */ export interface Announcer { @@ -27,6 +27,19 @@ export interface Announcer { let attachAnnouncer: (producer: Producer, announcer: Announcer) => void; let stampProducer: (producer: Producer, path: Path.Valid) => void; +// Info lookups pending on a requested track. Each is demand, as a pending track request is in Rust, +// though nobody subscribes. +class Lookups { + pending = 0; + // Re-evaluates the track's demand at once; set by watchDemand. + changed = () => {}; + + add(delta: number): void { + this.pending += delta; + this.changed(); + } +} + /** Reactive backing state shared by broadcast producers and consumers. */ class BroadcastState { requested = new Signal([]); @@ -35,8 +48,9 @@ class BroadcastState { // The tracks the application inserted. A finished one stays and keeps serving its cache; only an // abort or removeTrack drops it, the way only a dropped producer leaves a Rust broadcast. tracks = new Map(); - // Upstream subscriptions a consumer opened, shared by repeat subscribes until they close. - upstream = new Map(); + // The on-demand producer for each name, shared by every subscription and info lookup until it closes. + requests = new Map(); + lookups = new WeakMap(); // Whether something answers on-demand track requests: a wire layer's consumed broadcast, or a // publisher that has started pulling them. The counterpart of a live Rust `broadcast::Dynamic`; // without one, a track nobody publishes is `NotFound` rather than a request nobody will answer. @@ -52,8 +66,8 @@ class BroadcastState { } // Each track updates the aggregate on an edge, so a demand change touches only its track. -// A `pinned` track counts as demand until it closes, subscribed or not. -function watchDemand(state: BroadcastState, producer: track.Producer, pinned = false): void { +// A track counts as demand while any `lookups` are pending, subscribed or not. +function watchDemand(state: BroadcastState, producer: track.Producer, lookups?: Lookups): void { if (state.demands.has(producer)) return; const demand = producer.demand(); // A closed track is never demand, and its close already fired, so nothing would dispose a watcher. @@ -61,13 +75,16 @@ function watchDemand(state: BroadcastState, producer: track.Producer, pinned = f let active = false; const update = () => { const used = - state.closed.peek() === undefined && demand.closed.peek() === undefined && (pinned || demand.used.peek()); + state.closed.peek() === undefined && + demand.closed.peek() === undefined && + ((lookups?.pending ?? 0) > 0 || demand.used.peek()); if (active === used) return; state.active += used ? 1 : -1; active = used; state.used.set(state.active > 0); }; const disposeUsed = demand.used.subscribe(update); + if (lookups) lookups.changed = update; const disposeClosed = demand.closed.subscribe(() => { if (demand.closed.peek() === undefined) return; cleanup(); @@ -133,82 +150,68 @@ function lookup(state: BroadcastState, name: string): track.Producer | undefined state.tracks.delete(name); } - const upstream = state.upstream.get(name); - if (upstream) { - if (upstream.closed.peek() === undefined) return upstream; - state.upstream.delete(name); + const requested = state.requests.get(name); + if (requested) { + if (requested.closed.peek() === undefined) return requested; + state.requests.delete(name); } return undefined; } -// `register` is set on the subscribing (consumer) side: the fresh producer is cached in -// `state.upstream` so repeat subscriptions to the same track fan out from one upstream subscription -// instead of opening a new one, mirroring the Rust `broadcast::Consumer::track` weak-dedup. The -// consumer wire watches the producer's demand ({@link track.Demand.used}) and tears the upstream -// down once its last subscriber leaves, closing the producer, which evicts the cache entry below. -// The publishing side leaves `register` false, so a dynamic serve stays one request per peer -// subscription. -function subscribe( - state: BroadcastState, - name: string, - options: track.Subscription = {}, - register = false, -): track.Subscriber { +// The producer serving `name`, or a new one with a request queued for it. +// +// A broadcast has one logical track per name, mirroring the Rust `broadcast::Consumer::track` +// weak-dedup: a requested producer is cached in `state.requests` so every subscription and info +// lookup fans out from one request instead of opening another. Whoever serves it closes it once +// unused (the consuming wire tears the upstream down when its last subscriber leaves), which evicts +// the entry so a later subscribe requests the track again, continuing its sequences. +function logical(state: BroadcastState, name: string): { producer: track.Producer; requested: boolean } { if (state.closed.peek() !== undefined) { throw new Error("broadcast is closed"); } const existing = lookup(state, name); - if (existing) return existing.subscribe(options); + if (existing) return { producer: existing, requested: false }; const producer = new track.Producer(name); if (!state.served) { - // Answer through the subscriber rather than throwing, so a peer's subscribe is refused with + // Answer through the track rather than throwing, so a peer's subscribe is refused with // the code instead of an unhandled error. producer.close(new NotFound(`track ${name}`)); - return producer.subscribe(options); + return { producer, requested: false }; } - watchDemand(state, producer); - const subscriber = producer.subscribe(options); - - if (register) { - state.upstream.set(name, producer); - // Drop the cache entry once the upstream closes (the wire tears it down when its last - // subscriber leaves), so a later subscribe re-opens it. - void producer.closed.then(() => { - if (state.upstream.get(name) === producer) state.upstream.delete(name); - }); - } + const lookups = new Lookups(); + state.lookups.set(producer, lookups); + watchDemand(state, producer, lookups); + state.requests.set(name, producer); + void producer.closed.then(() => { + if (state.requests.get(name) === producer) state.requests.delete(name); + }); state.requested.mutate((requested) => { requested.push(hooks.makeRequest({ name, producer, sequences: state.sequences, pending: state.pending })); }); - return subscriber; + return { producer, requested: true }; } -async function resolveTrackInfo(state: BroadcastState, name: string): Promise { - const existing = lookup(state, name); - if (existing) return existing.info(); - - if (state.closed.peek() !== undefined) { - return Promise.reject(new Error("broadcast is closed")); - } - if (!state.served) return Promise.reject(new NotFound(`track ${name}`)); - - // A pending query is demand, as a pending track request is in Rust, though nobody subscribes. - const producer = new track.Producer(name); - watchDemand(state, producer, true); - state.requested.mutate((requested) => { - requested.push(hooks.makeRequest({ name, producer, sequences: state.sequences, pending: state.pending })); - }); +function subscribe(state: BroadcastState, name: string, options: track.Subscription = {}): track.Subscriber { + return logical(state, name).producer.subscribe(options); +} +async function resolveTrackInfo(state: BroadcastState, name: string): Promise { + const { producer, requested } = logical(state, name); + const lookups = state.lookups.get(producer); + lookups?.add(1); try { return await producer.info(); } finally { - producer.close(); + lookups?.add(-1); + // A finished lookup is no demand: let go of a request it opened that nobody subscribed to + // meanwhile. + if (requested && !producer.demand().used.peek()) producer.close(); } } @@ -287,7 +290,12 @@ export class Producer { #path = Path.empty(); constructor() { - registerWire(this, this.#wire(false)); + registerWire(this, { + subscribe: (name, options) => subscribe(this.#state, name, options), + resolveTrackInfo: (name) => resolveTrackInfo(this.#state, name), + fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), + requested: () => requested(this.#state), + }); } static { @@ -324,8 +332,9 @@ export class Producer { throw new Error("broadcast is closed"); } - const existing = this.#state.tracks.get(track.name); - if (existing && existing.closed.peek() === undefined) { + // One logical track per name: an open inserted track or a live request already serves it. + const live = [this.#state.tracks.get(track.name), this.#state.requests.get(track.name)]; + if (live.some((existing) => existing && existing.closed.peek() === undefined)) { throw new Error(`duplicate track: ${track.name}`); } @@ -360,15 +369,6 @@ export class Producer { return trackOf(name, this); } - #wire(register: boolean): Wire { - return { - subscribe: (name, options) => subscribe(this.#state, name, options, register), - resolveTrackInfo: (name) => resolveTrackInfo(this.#state, name), - fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), - requested: () => requested(this.#state), - }; - } - /** The route this broadcast is announced with, or undefined while it is not announced. */ get route(): Route | undefined { return this.#announcer?.route(); @@ -452,7 +452,7 @@ export class Consumer { } this.#state.consumers++; registerWire(this, { - subscribe: (name, options) => subscribe(this.#state, name, options, true), + subscribe: (name, options) => subscribe(this.#state, name, options), resolveTrackInfo: (name) => resolveTrackInfo(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index 480796fbe5f..0fa1f7f7378 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -114,7 +114,7 @@ export interface ReadGroupFrame { complete(): void; } -/** The next group or datagram sequence shared by dynamic producers of one broadcast track. */ +/** The next group or datagram sequence, shared by every dynamic producer that serves one broadcast track in turn. */ export interface TrackSequence { next: number; } diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 5f1d9275b80..fcb950141e4 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -510,8 +510,8 @@ export class Producer { #state = new TrackState(); #sequence: TrackSequence = { next: 0 }; // One past the highest group or datagram this producer received, like the Rust - // `max_sequence`. The shared counter above can run ahead of it: sibling producers of - // the same track advance it too. + // `max_sequence`. The shared counter above can run ahead of it: earlier producers of + // the same broadcast track advanced it too. #received = 0; // Recently written source groups, retained for replay to late subscribers and diff --git a/quest/m0/largest-regression.md b/quest/m0/largest-regression.md index 73aa85d5d17..93e74c88351 100644 --- a/quest/m0/largest-regression.md +++ b/quest/m0/largest-regression.md @@ -44,4 +44,3 @@ Public API: none. Wire: none. ## Related - [Claim-served epochs](/quest/m0/broadcast-epoch/claim-epochs.md) - prevents the splice on lite-07 by naming the instance -- [Coalesce dynamic tracks](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - keeps sequences going when a dynamic track's producer is replaced within one broadcast diff --git a/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md b/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md deleted file mode 100644 index 4c2d00c09fd..00000000000 --- a/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md +++ /dev/null @@ -1,88 +0,0 @@ -# [M] net: coalesce dynamic tracks and preserve sequences across replacements - -## Goal - -A broadcast has one logical dynamic track per name in both languages: at most -one pending or live producer, one on-demand request that every subscriber fans -out from, and a group and datagram sequence namespace that survives the -producer being replaced. Behavior only, so it ships on main. - -## Plan - -Dynamic tracks should have one logical identity per broadcast and track name, -but the Rust and JavaScript models violate different parts of that invariant. - -### Rust resets sequences when a dynamic producer is replaced - -A closed dynamic track is removed from the broadcast's weak cache. The next -subscription creates a fresh `track::Request` (`rs/moq-net/src/model/track.rs`), -and `Request::new` creates a fresh `TrackState`. Because its `max_sequence` -is empty, both `append_group` and `append_datagram` restart at sequence 0. - -That conflicts with the relay's logical track. The pump this was written -against is gone: #4741 resumes route changes by reading the routes' copies -(`rs/moq-net/src/model/front.rs`, `resume.rs`, and `origin.rs`). Re-check -first that a replacement restarting at sequence 0 still stalls under -copy-based resume, the way groups from a restarted producer were skipped -until its counter caught up (the playback stall fixed for JavaScript in -#2953), and rewrite this section against what the code does now. With #4741 this is the general hazard of restarting at 0 under one -name, which [broadcast epochs](/quest/m0/broadcast-epoch/README.md) fix for a -restarted publisher. This quest covers what an epoch does not: one dynamic -track replaced inside a live broadcast. - -The route-change tests (`rs/moq-net/tests/route_change.rs`) are the ones to -extend: none replaces a producer through `append_group()`, which would -create group 0 and, if the stall still holds, leave the subscriber stalled. Explicit group or datagram -writes can raise the old producer's shared sequence edge further, making the -catch-up window longer. - -### JavaScript permits concurrent same-name dynamic producers - -`BroadcastProducer.subscribe()` calls the internal `subscribe` with -`register = false` (`js/net/src/broadcast.ts`). Multiple publishing-side -subscriptions for the same name therefore enqueue independent requests and -create independent `track.Producer` instances. - -#2953 made those concurrent producers share a sequence allocator. That -prevents duplicate sequence allocation, but concurrent producers are the wrong -model. Publishing-side subscriptions should coalesce like -`BroadcastConsumer.subscribe()` and Rust's `broadcast::Consumer::track`: one -pending or live producer per broadcast and name, one on-demand request, and -multiple subscribers fanning out from it. - -### Desired behavior - -- A broadcast has at most one pending or live dynamic track producer per track - name. -- Concurrent JavaScript publishing-side subscriptions for the same name emit - one request and share its accepted producer. -- Subscription options from all subscribers remain aggregated on that request. - `track::Request` already does this in Rust: it carries `prev_subscription` - and re-combines the aggregate whenever a subscriber changes. -- After that producer closes, a later request creates a new producer but - continues the group and datagram sequence namespace for that broadcast and - name. -- Explicit group and datagram writes advance the shared allocator. -- A new broadcast generation starts each track at sequence 0. -- Rust and JavaScript expose the same lifecycle and sequencing behavior. - -The sequence allocator is shared across producer incarnations without sharing -the closed producer's cache or terminal state. - -### Regression coverage - -- JavaScript: two `BroadcastProducer.subscribe()` calls for one name produce - one request and both subscribers receive from the accepted producer. -- JavaScript: remove or replace #2953's concurrent-producer test, since - concurrent same-name producers should not be representable. -- Rust and JavaScript: close a dynamic producer after group and datagram - sequences have advanced, re-request the same name, and verify the - replacement appends at the next sequence. -- Rust relay model: extend the route-change tests above so a replacement - produced with `append_group()` is delivered immediately rather than skipped - until catch-up. -- Both implementations: verify a separate broadcast generation starts at 0. - -## Closes - -- [#2991](https://github.com/moq-dev/moq/issues/2991) - close this issue when the quest finishes diff --git a/quest/m1/README.md b/quest/m1/README.md index ab753ae3837..1e618664eb5 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -49,7 +49,6 @@ blocks. The quests that gated m0 lines moved under them. - [Early video demand](/quest/m1/catalog-early-demand.md) - a regression guards demand before the first keyframe after the detector removal - [Hang changelog](/quest/m1/hang-changelog-04.md) - the hang draft lists under -03 only what -03 published - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it -- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic producer per track name in both languages, with the sequence namespace surviving a replacement - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges in either order and replaces FETCH; relays fill misses by range, including over moq-transport - [Cold relay Largest](/quest/m1/ietf-cold-largest.md) - a relay reports the larger of its upstream's Largest and the largest object it received, even with nothing cached, so a d14-19 joining FETCH through it gets the current group's head - [Live media time](/quest/m1/subscribe-live-time.md) - re-scoped against `set_live`: a lite-07 SUBSCRIBE_OK carries the publisher's current media time only if a reader still needs it diff --git a/quest/m1/js-track-takeover.md b/quest/m1/js-track-takeover.md index 80ce587a3b7..4db7d467ff0 100644 --- a/quest/m1/js-track-takeover.md +++ b/quest/m1/js-track-takeover.md @@ -24,7 +24,3 @@ takeover, explicit sequence advancement, replacement continuity, and bounded name admission. Update the JS net docs for the behavior change. Public API: behavior change in `@moq/net` only. Wire: none. - -## Required - -- [Dynamic track identity](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - #4929 provides coalesced requests and the shared sequence namespace diff --git a/quest/m1/subscribe-ranges/js.md b/quest/m1/subscribe-ranges/js.md index e404e0a471c..20df7d429f8 100644 --- a/quest/m1/subscribe-ranges/js.md +++ b/quest/m1/subscribe-ranges/js.md @@ -32,4 +32,3 @@ IETF FETCH dispatch. Resized from [M] to [L] for it. ## Required - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the wire this speaks -- [Dynamic track identity](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - settles shared producer identity before adding on-demand requests diff --git a/quest/m1/subscribe-ranges/model.md b/quest/m1/subscribe-ranges/model.md index 26fdb7afb57..6cbf17070bc 100644 --- a/quest/m1/subscribe-ranges/model.md +++ b/quest/m1/subscribe-ranges/model.md @@ -24,7 +24,3 @@ Guidance: `moq_relay::fetch_group` (added in the 2026-10-05 audit). - `max_delay` caps every range. - Benchmark range count and span as separate axes (AGENTS.md fan-out rule). - -## Required - -- [Coalesce dynamic tracks](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - one dynamic track per name with sequences kept across replacements, which range requests build on (#4929) diff --git a/rs/moq-net/src/model/broadcast.rs b/rs/moq-net/src/model/broadcast.rs index f808e8b0044..ad821359052 100644 --- a/rs/moq-net/src/model/broadcast.rs +++ b/rs/moq-net/src/model/broadcast.rs @@ -8,6 +8,7 @@ //! [Info] is the broadcast's static metadata, fixed for its lifetime. use crate::{cache, stats, track}; use std::{ + collections::HashMap, sync::Arc, task::{Poll, ready}, }; @@ -97,6 +98,12 @@ struct BroadcastState { // coalescing onto it there). requests: Requests, track::Request>, + // Each name's sequence namespace, outliving the producers that serve it: a name means + // the same content for the broadcast's whole life, so a replacement producer continues + // where the last one stopped instead of reusing its sequences. Entries nothing holds or + // wrote are swept as the map grows, so requests for names never served stay bounded. + sequences: HashMap, track::Sequence>, + // Set once the broadcast ends: `Producer::close()` or the last producer-side // handle dropping. Every lookup after it answers `Unroutable`. closing: bool, @@ -112,6 +119,14 @@ impl BroadcastState { } } + /// The sequence namespace for `name`, shared with every earlier track of the name. + fn sequence(&mut self, name: &Arc) -> track::Sequence { + if self.sequences.len() == self.sequences.capacity() { + self.sequences.retain(|_, sequence| !sequence.is_unused()); + } + self.sequences.entry(name.clone()).or_default().clone() + } + /// Resolve every name the broadcast never filled, so subscribers waiting on a /// [`track::Info`] that can no longer arrive fail with `err` instead of parking. /// @@ -281,7 +296,10 @@ impl Producer { return Ok(track); } - let track = track::Producer::new(self.info.clone(), name, info).with_stats(self.stats.clone()); + let sequence = state.sequence(&name); + let track = track::Producer::new(self.info.clone(), name, info) + .with_stats(self.stats.clone()) + .with_sequence(sequence); state.insert_track(track.weak())?; Ok(track) } @@ -298,8 +316,12 @@ impl Producer { /// ends up never filling has to be dropped or rejected. Ending the broadcast /// resolves whatever is left. pub fn reserve_track(&self, name: impl Into>) -> Result { - let request = track::Request::new(self.info.clone(), name).with_stats(self.stats.clone()); - self.state.lock().insert_track(request.weak())?; + let name = name.into(); + let mut state = self.state.lock(); + let request = track::Request::new(self.info.clone(), name.clone()) + .with_stats(self.stats.clone()) + .with_sequence(state.sequence(&name)); + state.insert_track(request.weak())?; Ok(request) } @@ -730,7 +752,7 @@ impl Consumer { // requests map, and the FIFO order. The request inherits the broadcast's // cache pool through its `Arc`, same as a producer-created track. let name: Arc = name.into(); - let request = track::Request::new(self.info.clone(), name.clone()); + let request = track::Request::new(self.info.clone(), name.clone()).with_sequence(state.sequence(&name)); let consumer = request.consume(); // With no handler alive to serve it, the request is dropped: `NotFound` beats @@ -1205,6 +1227,61 @@ mod test { track2.assert_group(); } + /// A name keeps one sequence namespace for the broadcast's life: a replacement producer + /// appends past everything an earlier one wrote, explicit writes included, while a new + /// broadcast starts over. + #[test] + fn replacement_continues_sequences() { + let producer = Info::new().produce(); + let mut dynamic = producer.dynamic(); + let consumer = producer.consume(); + + let track = consumer.track("media").unwrap(); + let mut first = dynamic.assert_request().accept(None); + assert_eq!(first.append_group().unwrap().sequence, 0); + assert_eq!(first.append_datagram(crate::Timestamp::ZERO, b"x").unwrap(), 1); + first.create_group(crate::group::Info { sequence: 8 }).unwrap(); + first.insert_datagram(12, crate::Timestamp::ZERO, b"x").unwrap(); + first.finish().unwrap(); + drop((first, track)); + + let track = consumer.track("media").unwrap(); + let mut second = dynamic.assert_request().accept(None); + assert_eq!(second.append_group().unwrap().sequence, 13); + assert_eq!(second.append_datagram(crate::Timestamp::ZERO, b"x").unwrap(), 14); + second.finish().unwrap(); + drop((second, track)); + + // A track the producer creates itself continues the same namespace. + let third = producer.create_track("media", None).unwrap(); + assert_eq!(third.append_group().unwrap().sequence, 15); + + let next = Info::new().produce(); + assert_eq!( + next.create_track("media", None) + .unwrap() + .append_group() + .unwrap() + .sequence, + 0 + ); + } + + /// Names requested but never served leave nothing behind, so a peer asking for + /// arbitrary names cannot grow the broadcast. + #[test] + fn unserved_names_are_forgotten() { + let producer = Info::new().produce(); + let mut dynamic = producer.dynamic(); + let consumer = producer.consume(); + for i in 0..1000 { + let track = consumer.track(&i.to_string()).unwrap(); + drop(dynamic.assert_request()); + drop(track); + } + assert!(producer.state.lock().sequences.len() < 16); + } + #[moq_net_sim::test] async fn requested_unused() { let mut broadcast = Info::new().produce().dynamic(); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 30bcb2a697c..782941d2463 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -26,7 +26,7 @@ use std::{ ops::{Bound, RangeBounds}, sync::Arc, sync::OnceLock, - sync::atomic::{AtomicBool, Ordering}, + sync::atomic::{AtomicBool, AtomicU64, Ordering}, task::{Poll, ready}, time::Duration, }; @@ -161,6 +161,28 @@ impl Info { } } +/// The sequence namespace a track name keeps within its broadcast, shared by every +/// producer that serves the name, so a replacement appends after whatever an earlier one +/// wrote. Holds one past the highest sequence written. +#[derive(Clone, Default)] +pub(crate) struct Sequence(Arc); + +impl Sequence { + /// The next sequence an append takes. + fn next(&self) -> u64 { + self.0.load(Ordering::Relaxed) + } + + fn advance(&self, sequence: u64) { + self.0.fetch_max(sequence.saturating_add(1), Ordering::Relaxed); + } + + /// Whether nothing holds it and nothing was written, so forgetting it changes nothing. + pub(crate) fn is_unused(&self) -> bool { + Arc::strong_count(&self.0) == 1 && self.next() == 0 + } +} + /// Whether a track's cache reflects its live feed; see [`TrackState::set_idle`]. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] enum Feed { @@ -241,6 +263,10 @@ pub(crate) struct TrackState { // datagrams, so it can run ahead of any cached group. max_sequence: Option, + // The name's sequence namespace within the broadcast, which an append continues + // even when an earlier producer of the name wrote past this one's `max_sequence`. + sequence: Sequence, + // The sequence of the newest cached group: the live edge, protected from // eviction by never entering the eviction order until the track is `closed`. // Tracked separately from `max_sequence` because datagrams advance that shared @@ -1034,6 +1060,22 @@ impl TrackState { state } + /// Record a written sequence, here and in the name's shared namespace. + fn advance(&mut self, sequence: u64) { + self.max_sequence = Some(self.max_sequence.map_or(sequence, |max| max.max(sequence))); + self.sequence.advance(sequence); + } + + /// The sequence the next append takes: past this track's own writes and past every + /// earlier producer's of the same name. + fn next_sequence(&self) -> Result { + let own = match self.max_sequence { + Some(max) => max.checked_add(1).ok_or(coding::BoundsExceeded)?, + None => 0, + }; + Ok(own.max(self.sequence.next())) + } + /// Reject a sequence that is still cached; a dead (aborted or evicted) /// incarnation is removed so a fresh group can serve the sequence again. /// @@ -1083,7 +1125,7 @@ impl TrackState { self.evict.push_back((sequence, stamp)); } - self.max_sequence = Some(self.max_sequence.map_or(sequence, |max| max.max(sequence))); + self.advance(sequence); self.lookup.insert( sequence, Slot { @@ -1585,6 +1627,13 @@ impl Producer { self } + /// Continue the name's sequence namespace within its broadcast. Set by the broadcast + /// before the track is handed out. + pub(crate) fn with_sequence(self, sequence: Sequence) -> Self { + set_sequence(&self.state, sequence); + self + } + /// The track's name, unique within its broadcast. pub fn name(&self) -> &str { &self.name @@ -1640,10 +1689,7 @@ impl Producer { /// Create a new group with the next sequence number. pub fn append_group(&self) -> Result { let mut state = self.modify()?; - let sequence = match state.max_sequence { - Some(s) => s.checked_add(1).ok_or(coding::BoundsExceeded)?, - None => 0, - }; + let sequence = state.next_sequence()?; if let Some(fin) = state.final_sequence && sequence >= fin { @@ -1688,16 +1734,13 @@ impl Producer { // Normalize into the track's timescale, like frames (see `group::Producer::create_frame`). let timescale = state.info.as_ref().unwrap().timescale; let timestamp = group::on_track(timestamp.into(), timescale)?; - let sequence = match state.max_sequence { - Some(s) => s.checked_add(1).ok_or(coding::BoundsExceeded)?, - None => 0, - }; + let sequence = state.next_sequence()?; if let Some(fin) = state.final_sequence && sequence >= fin { return Err(Error::Closed); } - state.max_sequence = Some(sequence); + state.advance(sequence); meter.datagram(payload.len() as u64); state.push_datagram(Datagram { sequence, @@ -1736,7 +1779,7 @@ impl Producer { { return Err(Error::Closed); } - state.max_sequence = Some(state.max_sequence.unwrap_or(0).max(sequence)); + state.advance(sequence); meter.datagram(payload.len() as u64); state.push_datagram(Datagram { sequence, @@ -2233,6 +2276,16 @@ impl Drop for Dynamic { } } +fn set_sequence(state: &kio::Producer, sequence: Sequence) { + if let Ok(mut state) = state.write() { + debug_assert!( + state.max_sequence.is_none(), + "a track joins its namespace before writing" + ); + state.sequence = sequence; + } +} + /// Ends the track when the last [`Producer`] or [`Dynamic`] drops. /// /// A refcount rather than a "am I the last one?" check inside `Drop`: that answer is a @@ -4439,6 +4492,13 @@ impl Request { self } + /// Continue the name's sequence namespace within its broadcast. Set by the broadcast + /// before the request is visible to anyone. + pub(crate) fn with_sequence(self, sequence: Sequence) -> Self { + set_sequence(&self.state, sequence); + self + } + /// The requested track name. pub fn name(&self) -> &str { &self.name diff --git a/rs/moq-net/tests/route_change.rs b/rs/moq-net/tests/route_change.rs index 08b9a83eebe..d70a6c93ef1 100644 --- a/rs/moq-net/tests/route_change.rs +++ b/rs/moq-net/tests/route_change.rs @@ -661,3 +661,93 @@ async fn redundant_pair_fails_over_when_the_incumbent_ends() { .await .expect("timed out"); } + +/// `P` serves `video` on demand and replaces its producer partway through the broadcast, +/// the way a publisher restarts an encoder. `R` reads through `A`, whose route stays up, +/// so the logical track resumes onto the replacement. The replacement continues the +/// track's sequence namespace, so its first group reaches `R` at once instead of being +/// skipped until a fresh counter caught up with what `R` already read. +async fn producer_replaced(version: &str) { + let version: Version = version.parse().unwrap(); + let publisher = produce_origin(1); + let relay = produce_origin(2); + let subscriber = produce_origin(3); + + let broadcast = publisher.create_broadcast("live").unwrap(); + let mut dynamic = broadcast.dynamic(); + broadcast.announce(Default::default()).unwrap(); + let (tx, mut producers) = mpsc::unbounded(); + moq_net_sim::spawn(async move { + while let Ok(request) = dynamic.requested_track().await { + if tx.unbounded_send(request.accept(None)).is_err() { + return; + } + } + }); + + let _p_to_a = link(version, &publisher, &relay).await; + let _a_to_r = link(version, &relay, &subscriber).await; + + let consumer = subscriber.consume(); + consumer.routed("live").await.unwrap(); + let remote = consumer.request_broadcast("live", None).await.unwrap(); + let prefs = track::Subscription::default().with_max_delay(Duration::from_secs(60)); + let sub = remote.track("video").unwrap().subscribe(prefs).await.unwrap(); + let mut rx = read(sub); + + let first = producers.next().await.unwrap(); + for sequence in 0..2 { + let mut group = first.append_group().unwrap(); + assert_eq!(group.sequence, sequence, "{version}"); + for frame in 0..FRAMES { + group.write_frame(Timestamp::ZERO, payload(sequence, frame)).unwrap(); + assert_eq!(next(&mut rx).await, (sequence, payload(sequence, frame)), "{version}"); + } + group.finish().unwrap(); + } + + // The encoder restarts: the old producer goes away mid-broadcast, and the resumed + // subscription asks for the track again. + first.abort(Error::Cancel).unwrap(); + let second = moq_net_sim::timeout(Duration::from_secs(10), producers.next()) + .await + .expect("the track was not requested again") + .unwrap(); + + let mut group = second.append_group().unwrap(); + assert_eq!(group.sequence, 2, "{version}: the replacement restarted its sequences"); + for frame in 0..FRAMES { + group.write_frame(Timestamp::ZERO, payload(2, frame)).unwrap(); + assert_eq!(next(&mut rx).await, (2, payload(2, frame)), "{version}"); + } + group.finish().unwrap(); + + settle().await; + assert!( + matches!(rx.try_recv(), Err(mpsc::TryRecvError::Empty)), + "{version}: trailing delivery, or the subscription ended" + ); + drop((second, broadcast)); +} + +macro_rules! producer_replaced_tests { + ($($name:ident: $version:literal,)*) => { + $( + #[moq_net_sim::test] + async fn $name() { + moq_net_sim::timeout(TEST_TIMEOUT, producer_replaced($version)) + .await + .expect("timed out"); + } + )* + }; +} + +producer_replaced_tests! { + replaced_lite_07: "moq-lite-07-wip", + replaced_lite_06: "moq-lite-06", + replaced_lite_05: "moq-lite-05", + replaced_lite_04: "moq-lite-04", + replaced_ietf_19: "moq-transport-19", + replaced_ietf_22: "moq-transport-22", +}