diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index 631035e4d67..ee43da035d3 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -56,8 +56,8 @@ always a prefix. A service that serves only some of the paths beneath it refuses the rest as they are requested. Announcements are hints; the request is what decides. -Inside a broadcast, a reader can list its tracks (moq-lite-07, Rust only so -far): each track with its fixed properties, a synced mark, then tracks as they +Inside a broadcast, a reader can list its tracks (moq-lite-07): each track +with its fixed properties, a synced mark, then tracks as they come and go. Listing is discovery, not delivery: subscribe to each track you want, and its groups still cached upstream reach you late. A broadcast lists the tracks its publisher creates, never one served only on request, and a path diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index a1d1da51a71..f0c964bc2d7 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -42,11 +42,12 @@ broadcast.announce(); - **Connections share by URL and versions.** `new Connection({ url })` pools and reconnects with backoff, which the elements use. An ordered `versions` list, spelled like `moq-lite-06` as in Rust, applies to both WebTransport and WebSocket; equivalent lists share, while a different order or an omitted list gets its own pool entry. Supplying your own transport options or origin selects a private loop. Receive estimates are on demand: observe `connection.probe` reactively or subscribe while you need them; `peek()` alone opens no PROBE stream. - **GOAWAY moves the session.** The connection dials the replacement at once while the old session keeps serving its groups, up to `goaway.handover` (default 10s, or the relay's deadline when that is sooner). New requests keep opening on the old session until the replacement's route outranks it, which is a restart without an epoch: an unpinned announced request follows the replacement, while a blind or epoch-pinned request stays on its accepted instance. With an epoch, which travels only when `moq-lite-07-wip` is offered, the replacement serves the same broadcast, so open subscriptions move across without missing a group. An empty GOAWAY redials the same URL. A redirect stays on the same host unless `goaway.redirect` is `"follow"`. - **One send estimate per connection.** `Bandwidth.Allocator` divides it by track priority, max-min fair within a tier. An idle track claims nothing. Publishers reserve against it so their targets sum to the estimate. +- **List a broadcast's tracks.** `for await (const event of broadcast.tracks())` reports each track the broadcast lists (`added`, with its `Track.Info`), one `synced`, then `added` and `removed` as tracks come and go, until the broadcast ends; leaving the loop stops listing. Over a session it needs `moq-lite-07-wip`, syncs at once with what this side holds, and the peer's tracks follow as `added`. A track served only on request is never listed. See [discovery](/concept/moq-lite#discovery). - **One track per name.** Concurrent subscriptions share one request and producer. `createTrack` on a name with a request still queued answers it. The application owns sequence numbers: a track that replaces an ended one, created or requested, throws on an append that would reuse a sequence the name already wrote, so it writes its first group with an explicit sequence. Only a new broadcast starts over. - **Timing is per track, with no default.** A track that declares a `timescale` carries a `timestamp` on every frame; one without is [untimed](/concept/moq-lite#subscriptions) and its frames carry none. Omitting it means untimed, as in Rust. Nothing stamps a frame for you: the publisher passes the timestamp, and the `Timed` type (`{ value, at }`) carries one into the `@moq/json` and `@moq/flate` producers. - **Subscriber staleness is `maxDelay`.** It counts from a group's successor, by media time and by wall clock, whichever reaches it first. Passing the old `maxAge` key throws a `TypeError` naming `maxDelay`. Publisher retention is the separate `Track.Info.maxAge`. - **Hidden paths** stay out of discovery unless the announce request opts in. See [hidden broadcasts](/concept/moq-lite#hidden-broadcasts). -- **Authorization is in band.** On moq-lite 07 (opt in with `versions: ["moq-lite-07-wip", "moq-lite-06"]`, listing every fallback you still want) and on moq-transport draft-17+ with the [MoQ Auth extension](/draft/moq-auth), `auth.grant` watches what the relay lets this side publish and subscribe to, and `auth.add(token)` presents another token without reconnecting. Publishing outside the grant closes the connection with `SessionCode.Unauthorized`, naming the path. On moq-lite, a subscription the grant stops covering resets with `StreamCode.Unauthorized` and the session stays up. On moq-transport, `auth.add(token)` keeps the token even without the extension, and a broadcast announced with the `Auth.Auth` it returns (`broadcast.announce({ auth })`) carries it on that announcement's `PUBLISH_NAMESPACE`; announcing again with another renews the request. An `Auth.Auth` belongs to the session that added it, so after a reconnect, add the token again and re-announce. +- **Authorization is in band.** On moq-lite 07 (opt in with `versions: ["moq-lite-07-wip", "moq-lite-06"]`, listing every fallback you still want) and on moq-transport draft-17+ with the [MoQ Auth extension](/draft/moq-auth), `auth.grant` watches what the relay lets this side publish and subscribe to, and `auth.add(token)` presents another token without reconnecting. Publishing outside the grant closes the connection with `SessionCode.Unauthorized`, naming the path. On moq-lite, a subscription or track listing the grant stops covering resets with `StreamCode.Unauthorized` and the session stays up. On moq-transport, `auth.add(token)` keeps the token even without the extension, and a broadcast announced with the `Auth.Auth` it returns (`broadcast.announce({ auth })`) carries it on that announcement's `PUBLISH_NAMESPACE`; announcing again with another renews the request. An `Auth.Auth` belongs to the session that added it, so after a reconnect, add the token again and re-announce. - **A graceful close waits.** `await connection.close()` withdraws announcements and drains for up to one second. Application delivery requires explicit `moq-lite-07-wip`; older Lite versions wait only for transport ACKs. IETF media streams are not drained. `abort()` ends immediately. Examples: diff --git a/js/net/bench/broadcasts.ts b/js/net/bench/broadcasts.ts index 3d8ad958fb2..e491fe7e48b 100644 --- a/js/net/bench/broadcasts.ts +++ b/js/net/bench/broadcasts.ts @@ -105,6 +105,36 @@ for (const trackCount of [8, 32, 128]) { } } +// One created and removed track reaches every listing, however many tracks the broadcast holds. +const listed = 256; +console.log("tracks,listings,create_remove_and_read_us"); +for (const trackCount of [8, 128, 1024]) { + for (const listingCount of [1, 8, 32]) { + const broadcast = new BroadcastProducer(); + for (let index = 0; index < trackCount; index++) broadcast.createTrack(`held-${index}`); + const listings = Array.from({ length: listingCount }, () => broadcast.consume().tracks()); + // Past each listing's initial set and SYNCED. + for (const listing of listings) { + for (let index = 0; index <= trackCount; index++) await listing.next(); + } + const start = performance.now(); + for (let index = 0; index < listed; index++) { + broadcast.createTrack(`new-${index}`); + for (const listing of listings) { + const event = await listing.next(); + if (event?.kind !== "added" || event.name !== `new-${index}`) throw new Error("listing missed a track"); + } + broadcast.removeTrack(`new-${index}`); + for (const listing of listings) { + if ((await listing.next())?.kind !== "removed") throw new Error("listing missed a removal"); + checksum++; + } + } + console.log(`${trackCount},${listingCount},${(((performance.now() - start) * 1000) / listed).toFixed(1)}`); + broadcast.close(); + } +} + // Resolve one touched path while both the table and that path's readers grow. console.log("request,routes,readers,update_and_read_us"); const epoch = Epoch.parse("01900000-0000-7000-8000-000000000001"); diff --git a/js/net/src/auth.test.ts b/js/net/src/auth.test.ts index 0914bd856b4..ead76be878e 100644 --- a/js/net/src/auth.test.ts +++ b/js/net/src/auth.test.ts @@ -623,6 +623,40 @@ test("a fetch abandoned while its token waits never reaches the wire", async () } }); +// The peer refuses a listing its token has not yet authorized, so the request waits for it. +test("a listing waits for its token before reaching the wire", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room/x")); + broadcast.announce(); + broadcast.createTrack("video"); + const { client, server, transport } = await connect({ serverPublish: origin, protocol: Lite.ALPN_07_WIP }); + const requests = server.auth.requests(); + const setup = await requests.next(); + if (!setup) throw new Error("missing setup token"); + const initial = setup.accept(grant([], [])); + await waitFor(client.auth.grant, (g) => g !== undefined); + const adding = client.auth.add("first"); + try { + const unanswered = await requests.next(); + if (!unanswered) throw new Error("missing token"); + const opened = transport.sendStreams.bidi.length; + const tracks = wireOf(client).consume(Path.from("room/x")).tracks(); + for (let i = 0; i < 10; i++) await nextTask(); + expect(transport.sendStreams.bidi.length).toBe(opened); + const issued = unanswered.accept(grant([], ["room"])); + await adding; + expect(await tracks.next()).toEqual({ kind: "synced" }); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "video" }); + tracks.close(); + issued.close(); + } finally { + initial.close(); + client.abort(); + server.abort(); + await adding.catch(() => void 0); + } +}); + // The window between the wait seeing the caller leave and acting on it spans a few // microtasks, so sweep where the new caller lands. test.each(Array.from({ length: 17 }, (_, gap) => gap))( diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index ec326eb650e..77a17e4cc2c 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -10,11 +10,14 @@ import { NotFound } from "./error.ts"; import type { Consumer as GroupConsumer } from "./group.ts"; import { Route } from "./hop.ts"; import { hooks, TrackSequences } from "./internal.ts"; +import { Lister, Listing, listTracks, type Tracks } from "./listing.ts"; import * as Path from "./path.ts"; import * as track from "./track.ts"; import { untilAborted } from "./util/abort.ts"; import { registerWire, trackOf } from "./wire.ts"; +export { Tracks, type TracksEvent } from "./listing.ts"; + /** The origin callback a created broadcast uses to advertise its exact path. @internal */ export interface Announcer { /** Advertise or re-price this broadcast's path. */ @@ -48,6 +51,8 @@ class BroadcastState { requested = new Signal([]); pending = new Set(); closed = new Once(); + // The tracks the broadcast lists, and the readers listing them. + listing = new Listing(this.closed); // The tracks the application inserted. A finished one stays and keeps serving its cache; only an // abort or removeTrack drops it, the way only a dropped producer leaves a Rust broadcast. tracks = new Map(); @@ -141,6 +146,7 @@ async function requested(state: BroadcastState): Promise { @@ -297,18 +303,19 @@ function takeQueued(state: BroadcastState, name: string): track.Request | undefi return queued; } -// Serve `producer` as the broadcast's track for its name. -function adopt(state: BroadcastState, producer: track.Producer): void { +// Serve `producer` as the broadcast's track for its name, listed with `info`. +function adopt(state: BroadcastState, producer: track.Producer, info: track.Info): void { const lookups = state.lookups.get(producer) ?? new Lookups(); state.lookups.set(producer, lookups); watchDemand(state, producer, lookups); state.tracks.set(producer.name, producer); + state.listing.listOwn(producer.name, producer, info); // A finished track keeps serving its cache, so only an abort evicts it. void producer.closed.then((closed) => { - if (closed instanceof Error && state.tracks.get(producer.name) === producer) { - state.tracks.delete(producer.name); - } + if (!(closed instanceof Error)) return; + if (state.tracks.get(producer.name) === producer) state.tracks.delete(producer.name); + state.listing.unlistOwn(producer.name, producer); }); } @@ -362,6 +369,8 @@ export class Producer { largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), + tracks: (owner) => listTracks(this.#state.listing, owner), + lister: () => new Lister(this.#state.listing), }); } @@ -420,7 +429,7 @@ export class Producer { const lookups = this.#state.lookups.get(producer); if (lookups) lookups.opened = false; this.#state.requests.delete(name); - adopt(this.#state, producer); + adopt(this.#state, producer, track.infoDefaults(info)); return producer; } @@ -433,13 +442,13 @@ export class Producer { const producer = new track.Producer(name).accept(info); hooks.bindSequence(producer, this.#state.sequences.get(name)); - adopt(this.#state, producer); + adopt(this.#state, producer, track.infoDefaults(info)); return producer; } /** - * Remove a statically inserted track, including a finished cached one, and stop counting it - * toward {@link demand} at once. Throws once the broadcast is finished. + * Remove a statically inserted track, including a finished cached one: stop counting it + * toward {@link demand} and listing it at once. Throws once the broadcast is finished. */ removeTrack(name: string): void { this.#assertUnfinished(); @@ -447,6 +456,7 @@ export class Producer { if (!track) return; this.#state.tracks.delete(name); this.#state.demands.get(track)?.(); + this.#state.listing.unlistOwn(name, track); } /** A lazy read handle for a track on this broadcast. */ @@ -588,6 +598,7 @@ interface Shared { state: BroadcastState; path: Path.Valid; epoch?: Epoch.Valid; + unlisted?: boolean; } // Constructs a Consumer from within this module without exposing a public constructor @@ -606,6 +617,8 @@ export class Consumer { #state: BroadcastState; #path: Path.Valid; #epoch?: Epoch.Valid; + // Served on request through a route claiming a shorter prefix, which names no broadcast to list. + #unlisted: boolean; // Guards against a double close() on this handle over-decrementing the consumer count. #closed = false; @@ -614,6 +627,7 @@ export class Consumer { protected constructor(shared?: Shared) { this.#path = shared?.path ?? Path.empty(); this.#epoch = shared?.epoch; + this.#unlisted = shared?.unlisted ?? false; if (shared) { this.#state = shared.state; } else { @@ -628,6 +642,8 @@ export class Consumer { largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), + tracks: (owner) => this.#tracks(owner), + lister: () => new Lister(this.#state.listing), }); } @@ -640,6 +656,9 @@ export class Consumer { hooks.stampEpoch = (target, epoch) => { target.#epoch = epoch; }; + hooks.stampUnlisted = (target) => { + target.#unlisted = true; + }; } /** @@ -691,7 +710,12 @@ export class Consumer { // Hand this consumer's backing state and path to a clone. Opaque (`never`) so the state type // stays unexported; a subclass passes it straight back into its own `super(...)`. protected shareState(): never { - return { state: this.#state, path: this.#path, epoch: this.#epoch } satisfies Shared as never; + return { + state: this.#state, + path: this.#path, + epoch: this.#epoch, + unlisted: this.#unlisted, + } satisfies Shared as never; } /** Get a lazy handle for a track on this broadcast. Repeat subscriptions dedupe onto one upstream subscription. */ @@ -699,6 +723,26 @@ export class Consumer { return trackOf(name, this); } + /** + * List the broadcast's tracks: each one with its info, then `synced`, then changes as they + * happen, until the broadcast ends. + * + * Lists the tracks its producer created (until they abort or are removed) and, for a + * broadcast consumed from a session, the tracks the peer lists. Through a session the + * listing syncs at once, with what this side holds, and the peer's tracks follow as live + * adds; every listing of one broadcast shares one request to the peer. A track served only + * on request is never listed, and listing never requests one. Throws once the broadcast + * has closed. + */ + tracks(): Tracks { + return this.#tracks(); + } + + #tracks(owner?: object): Tracks { + if (this.#unlisted) throw new NotFound("a route claiming a prefix lists no tracks"); + return listTracks(this.#state.listing, owner); + } + /** * Release this handle. The broadcast is closed once this was the last live handle; * while other {@link clone}s remain open it stays live. diff --git a/js/net/src/ietf/subscriber.test.ts b/js/net/src/ietf/subscriber.test.ts index c9f960a3adc..e9441b33b24 100644 --- a/js/net/src/ietf/subscriber.test.ts +++ b/js/net/src/ietf/subscriber.test.ts @@ -867,6 +867,18 @@ test("a PUBLISH_NAMESPACE update that changes the publisher applies in place", a expect(await announced.next()).toMatchObject({ prefix: Path.from("theirs"), kind: "end" }); }); +test("a consumed broadcast refuses to list its tracks rather than claim it has none", async () => { + const pair = createMockTransportPair(ALPN.DRAFT_19); + const session = new NativeSession(pair.server, VERSION, true); + const subscriber = new Subscriber({ session, cluster: { self: SELF, peer: PEER } }); + const consumer = subscriber.consume(Path.from("theirs")); + await expect(consumer.tracks().next()).rejects.toThrow("track listing requires moq-lite-07"); + // A later reader asks again, and is refused again. + await expect(consumer.tracks().next()).rejects.toThrow("track listing requires moq-lite-07"); + consumer.close(); + subscriber.close(); +}); + /** * A second PUBLISH_NAMESPACE on the stream that already carries one is no longer an * update: it is the base draft's duplicate request, a protocol violation. diff --git a/js/net/src/ietf/subscriber.ts b/js/net/src/ietf/subscriber.ts index 4f74d0f8fc6..6f2d304f743 100644 --- a/js/net/src/ietf/subscriber.ts +++ b/js/net/src/ietf/subscriber.ts @@ -29,6 +29,7 @@ import { scopeHead, scopeOverlaps, } from "../internal.ts"; +import type { Lister } from "../listing.ts"; import * as Path from "../path.ts"; import { type Cursor, Reader, type Stream } from "../stream.ts"; import { Tail } from "../tail.ts"; @@ -726,6 +727,8 @@ export class Subscriber { #createConsume(path: Path.Valid, instance: ReturnType): broadcast.Consumer { // moq-transport resolves metadata through SUBSCRIBE and has no one-shot group fetch here. const consumer = new ConsumeBroadcast(); + // Registered before anyone can list the broadcast, so no reader syncs on an empty answer. + void this.#refuseListing(wireOf(consumer).lister()); void (async () => { for (;;) { const request = await wireOf(consumer).requested(); @@ -738,6 +741,22 @@ export class Subscriber { return this.#consumes.insert(path, instance, consumer); } + // moq-transport lists no tracks here: refuse each listing until the broadcast closes, since an + // empty answer would claim the peer lists nothing. + async #refuseListing(lister: Lister): Promise { + try { + for (;;) { + while (!lister.used.peek()) { + if (lister.ended.peek() !== undefined) return; + await race([Signal.race(lister.used, lister.ended)]); + } + lister.abort(new Error("track listing requires moq-lite-07 or newer")); + } + } finally { + lister.close(); + } + } + // The adapter is gone. If the transport has already closed, that close is the // error. A still-open transport, such as a GOAWAY drain, has no peer code yet, // so this does not wait for it. Either way the route failed, so a resumed track moves on. diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index b387e40894a..9a82687e71c 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -580,6 +580,8 @@ export const hooks: { stampPath: (target: BroadcastProducer | BroadcastConsumer, path: Path.Valid) => void; /** Name the epoch of the route an origin resolved a broadcast handle through. */ stampEpoch: (target: BroadcastConsumer, epoch: Epoch.Valid | undefined) => void; + /** Mark a broadcast handle served through a route claiming a shorter prefix: it lists nothing. */ + stampUnlisted: (target: BroadcastConsumer) => void; } = { endTrack: () => { throw new Error("track.ts not loaded"); @@ -755,6 +757,9 @@ export const hooks: { stampEpoch: () => { throw new Error("broadcast.ts not loaded"); }, + stampUnlisted: () => { + throw new Error("broadcast.ts not loaded"); + }, }; /** diff --git a/js/net/src/listing.test.ts b/js/net/src/listing.test.ts new file mode 100644 index 00000000000..d3eb31325c9 --- /dev/null +++ b/js/net/src/listing.test.ts @@ -0,0 +1,326 @@ +import { expect, test } from "bun:test"; +import { Producer as BroadcastProducer } from "./broadcast.ts"; +import { NotFound, StreamCode, StreamError } from "./error.ts"; +import type { Tracks } from "./listing.ts"; +import { wireOf } from "./wire.ts"; + +/** Lets queued microtasks and the listing's signal notifications run. */ +function settle(): Promise { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** + * Reads a listing as `+name`, `-name`, `synced`, `end`, or `error: ..`, keeping one pending read + * across calls so a read that loses to {@link drain}'s settle never drops an event. + */ +class Events { + readonly tracks: Tracks; + #pending?: Promise; + + constructor(tracks: Tracks) { + this.tracks = tracks; + } + + /** Every event ready now, through the end of the listing. */ + async drain(): Promise { + const events: string[] = []; + for (;;) { + this.#pending ??= this.tracks.next().then( + (event) => { + if (!event) return "end"; + if (event.kind === "synced") return "synced"; + return `${event.kind === "added" ? "+" : "-"}${event.name}`; + }, + (err: unknown) => `error: ${err instanceof StreamError ? err.code : String(err)}`, + ); + const event = await Promise.race([this.#pending, settle().then(() => undefined)]); + if (event === undefined) return events; + this.#pending = undefined; + events.push(event); + if (event === "end" || event.startsWith("error")) return events; + } + } +} + +function list(broadcast: BroadcastProducer): Events { + return new Events(broadcast.consume().tracks()); +} + +test("tracks present arrive before synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + expect(await list(broadcast).drain()).toEqual(["+a", "+b", "synced"]); +}); + +test("an empty listing syncs at once", async () => { + const broadcast = new BroadcastProducer(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["synced"]); + broadcast.createTrack("a"); + expect(await tracks.drain()).toEqual(["+a"]); +}); + +test("a listed track reports its info", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a", { priority: 9, refresh: 4 }); + const event = await broadcast.consume().tracks().next(); + expect(event?.kind === "added" && event.info).toMatchObject({ priority: 9, refresh: 4 }); +}); + +test("a track leaving the table is removed", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const b = broadcast.createTrack("b"); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "+b", "synced"]); + broadcast.removeTrack("a"); + b.close(new Error("aborted")); + expect(await tracks.drain()).toEqual(["-a", "-b"]); +}); + +test("a finished track is listed while the table holds it", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a").close(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual(["-a"]); + broadcast.close(); + expect(await tracks.drain()).toEqual(["end"]); +}); + +test("a recreated track is removed before it is added again", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a").close(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + // The finished track still holds the name; its replacement takes it over. + broadcast.createTrack("a"); + expect(await tracks.drain()).toEqual(["-a", "+a"]); +}); + +test("a track served on request is never listed", async () => { + const broadcast = new BroadcastProducer(); + const consumer = broadcast.consume(); + const tracks = new Events(consumer.tracks()); + const pulled = wireOf(broadcast).requested(); + expect(await tracks.drain()).toEqual(["synced"]); + + consumer.track("x").subscribe(); + const request = await pulled; + request?.accept({}); + expect(await tracks.drain()).toEqual([]); + + // A queued request answered by createTrack is the producer's track, so it is listed. + consumer.track("z").subscribe(); + broadcast.createTrack("z"); + expect(await tracks.drain()).toEqual(["+z"]); + broadcast.close(); +}); + +test("an initial track removed before it is read is omitted", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + const tracks = list(broadcast); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual(["+b", "synced"]); +}); + +test("churn never postpones synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + const tracks = broadcast.consume().tracks(); + // New names keep arriving faster than the reader drains them. + broadcast.createTrack("c"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + broadcast.createTrack("d"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "b" }); + broadcast.createTrack("e"); + expect(await new Events(tracks).drain()).toEqual(["synced", "+c", "+d", "+e"]); +}); + +test("a removal of a seen track may precede synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + broadcast.createTrack("c"); + const tracks = broadcast.consume().tracks(); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + broadcast.removeTrack("a"); + broadcast.removeTrack("c"); + expect(await new Events(tracks).drain()).toEqual(["+b", "-a", "synced"]); +}); + +test("a stalled reader holds one change per name", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("seen"); + broadcast.createTrack("stay"); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+seen", "+stay", "synced"]); + broadcast.removeTrack("seen"); + for (let round = 0; round < 1000; round++) { + broadcast.createTrack(`churn${round}`); + broadcast.createTrack("seen"); + broadcast.removeTrack(`churn${round}`); + broadcast.removeTrack("seen"); + } + expect(await tracks.drain()).toEqual(["-seen"]); +}); + +test("the broadcast ending ends the listing", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const consumer = broadcast.consume(); + const tracks = new Events(consumer.tracks()); + broadcast.close(); + expect(await tracks.drain()).toEqual(["end"]); + expect(() => consumer.tracks()).toThrow("broadcast is closed"); +}); + +test("a closed listing reads nothing more, and leaving a loop early closes it", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const tracks = broadcast.consume().tracks(); + for await (const event of tracks) { + expect(event).toMatchObject({ kind: "added", name: "a" }); + break; + } + broadcast.createTrack("b"); + expect(await tracks.next()).toBeUndefined(); +}); + +test("readers wait for the lister to sync", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("own"); + const lister = wireOf(broadcast).lister(); + expect(lister.used.peek()).toBe(false); + + const tracks = list(broadcast); + expect(lister.used.peek()).toBe(true); + expect(await tracks.drain()).toEqual(["+own"]); + lister.add("up", { priority: 0 }); + expect(() => lister.add("up", { priority: 0 })).toThrow("duplicate"); + lister.synced(); + expect(await tracks.drain()).toEqual(["+up", "synced"]); + + lister.remove("up"); + expect(() => lister.remove("up")).toThrow("not listed"); + expect(await tracks.drain()).toEqual(["-up"]); + + tracks.tracks.close(); + expect(lister.used.peek()).toBe(false); +}); + +test("a name listed by both is reported once", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const lister = wireOf(broadcast).lister(); + lister.add("a", { priority: 0 }); + lister.synced(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual([]); + lister.remove("a"); + expect(await tracks.drain()).toEqual(["-a"]); +}); + +test("an aborted answer fails its readers and a later reader asks again", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + const tracks = list(broadcast); + lister.add("up", { priority: 0 }); + lister.abort(new NotFound("broadcast")); + expect(await tracks.drain()).toEqual([`error: ${StreamCode.NotFound}`]); + expect(lister.used.peek()).toBe(false); + + const again = list(broadcast); + expect(lister.used.peek()).toBe(true); + expect(await again.drain()).toEqual([]); + lister.synced(); + expect(await again.drain()).toEqual(["synced"]); + // The failed reader leaving releases nothing. + tracks.tracks.close(); + expect(lister.used.peek()).toBe(true); +}); + +test("a closed lister withdraws its answer and syncs", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + lister.add("up", { priority: 0 }); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+up"]); + lister.close(); + expect(await tracks.drain()).toEqual(["-up", "synced"]); +}); + +test("an owner lists each broadcast once", async () => { + const broadcast = new BroadcastProducer(); + const other = new BroadcastProducer(); + broadcast.createTrack("a"); + const owner = {}; + const older = new Events(wireOf(broadcast).tracks(owner)); + const elsewhere = new Events(wireOf(other).tracks(owner)); + const stranger = new Events(wireOf(broadcast).tracks({})); + const newer = new Events(wireOf(broadcast).tracks(owner)); + expect(await older.drain()).toEqual([`error: ${StreamCode.Cancel}`]); + expect(await newer.drain()).toEqual(["+a", "synced"]); + expect(await elsewhere.drain()).toEqual(["synced"]); + expect(await stranger.drain()).toEqual(["+a", "synced"]); + + // The replaced reader leaving releases nothing of the newer one's. + older.tracks.close(); + broadcast.createTrack("b"); + expect(await newer.drain()).toEqual(["+b"]); + newer.tracks.close(); + stranger.tracks.close(); + expect(wireOf(broadcast).lister().used.peek()).toBe(false); +}); + +test("a broadcast has one lister", () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + expect(() => wireOf(broadcast).lister()).toThrow("already has a lister"); + lister.close(); + wireOf(broadcast).lister(); +}); + +test("a finished answer ends its readers and a later reader asks again", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + const tracks = list(broadcast); + lister.add("up", { priority: 0 }); + lister.synced(); + expect(await tracks.drain()).toEqual(["+up", "synced"]); + lister.finish(); + expect(await tracks.drain()).toEqual(["end"]); + expect(lister.used.peek()).toBe(false); + + const again = list(broadcast); + expect(await again.drain()).toEqual([]); +}); + +test("a cleared answer unsyncs a reader still waiting", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + lister.synced(); + const tracks = broadcast.consume().tracks(); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + lister.clear(); + const events = new Events(tracks); + expect(await events.drain()).toEqual(["+b"]); + lister.synced(); + expect(await events.drain()).toEqual(["synced"]); +}); + +test("a lister sees the broadcast end", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + broadcast.close(); + expect(lister.ended.peek()).toBeNull(); +}); diff --git a/js/net/src/listing.ts b/js/net/src/listing.ts new file mode 100644 index 00000000000..71e631e5a9d --- /dev/null +++ b/js/net/src/listing.ts @@ -0,0 +1,439 @@ +/** + * A broadcast's track listing: the tracks it lists, reported to each reader in order. + * + * A track is listed while it is in the broadcast's table: created by its producer until it + * aborts or is removed, or answered by a {@link Lister}, as a session lists its peer's tracks. + * A track served only on request is never listed, and listing never requests one. + * + * Each {@link Tracks} reader keeps its own queue, keyed by name in insertion order, so a + * stalled reader holds at most one pending change per name: an add and a removal it has not + * reached cancel, and a removal and re-add of a name it saw collapse in place. Its state is + * bounded by the names it saw plus the live set, however long it stalls. + * + * @module + */ +import { type GetPromise, type Getter, Signal } from "@moq/signals"; +import { StreamCode, StreamError } from "./error.ts"; +import type * as track from "./track.ts"; + +/** + * What a {@link Tracks} listing reports, in order. + * + * `added`: a track the broadcast lists, with its fixed properties. `synced`: every track + * listed when the listing began, and still listed, was reported; sent once, and later events + * are live. `removed`: a track an earlier `added` reported is no longer listed, though a + * handler may still serve its name on request. + * + * @public + */ +export type TracksEvent = + | { kind: "added"; name: string; info: track.Info } + | { kind: "synced" } + | { kind: "removed"; name: string }; + +/** One listed track and who lists it. */ +interface Entry { + info: track.Info; + /** The producer whose track lists it, so a stale unlist never removes a later track of the name. */ + own?: object; + /** Whether the lister's answer lists it. */ + answered: boolean; +} + +/** A change a reader has yet to read. `readd` is a removal of a name it saw, then the name listed again. */ +type Pending = { kind: "add"; info: track.Info } | { kind: "remove" } | { kind: "readd"; info: track.Info }; + +/** One reader's pending changes, keyed by name in the order they happened. */ +class Queue { + pending = new Map(); + /** Events left to send before SYNCED, once the listing has synced. */ + bound?: number; + /** SYNCED was sent. */ + synced = false; + /** `null` once the listing ended, or the error that ended it. */ + end?: Error | null; + /** The reader left the listing, or a newer reader of its owner replaced it. */ + detached = false; + changed = new Signal(0); + + wake() { + this.changed.update((n) => n + 1); + } + + add(name: string, info: track.Info) { + // A name is listed once at a time, so only its removal can be pending. + if (this.pending.get(name)?.kind === "remove") this.pending.set(name, { kind: "readd", info }); + else this.pending.set(name, { kind: "add", info }); + this.wake(); + } + + remove(name: string) { + // Never read: the two cancel. Otherwise the removal replaces a pending re-add in place. + if (this.pending.get(name)?.kind === "add") this.pending.delete(name); + else this.pending.set(name, { kind: "remove" }); + this.wake(); + } + + /** The events the pending changes take to read. */ + events(): number { + let events = 0; + for (const pending of this.pending.values()) events += pending.kind === "readd" ? 2 : 1; + return events; + } + + /** The listing synced: SYNCED goes out once what is queued now was read, or earlier if the queue drains first. */ + sync() { + if (this.synced || this.bound !== undefined) return; + this.bound = this.events(); + this.wake(); + } + + /** The listing's answer restarted before this reader was told it synced. */ + unsync() { + if (!this.synced) this.bound = undefined; + } + + finish(end: Error | null) { + this.end ??= end; + this.detached = true; + this.wake(); + } + + ready(): boolean { + return this.end !== undefined || this.pending.size > 0 || (!this.synced && this.bound !== undefined); + } + + next(): TracksEvent | undefined { + if (this.end !== undefined) { + if (this.end) throw this.end; + return undefined; + } + if (!this.synced && this.bound !== undefined && (this.bound === 0 || this.pending.size === 0)) { + this.synced = true; + return { kind: "synced" }; + } + const first = this.pending.entries().next(); + if (first.done) throw new Error("a ready queue has a pending change"); + const [name, pending] = first.value; + if (!this.synced && this.bound !== undefined) this.bound -= 1; + switch (pending.kind) { + case "add": + this.pending.delete(name); + return { kind: "added", name, info: pending.info }; + case "remove": + this.pending.delete(name); + return { kind: "removed", name }; + case "readd": + // Setting an existing key keeps its place, so the add is read next. + this.pending.set(name, { kind: "add", info: pending.info }); + return { kind: "removed", name }; + } + } +} + +/** + * The listing half of a broadcast's state. + * + * @internal + */ +export class Listing { + /** Every listed track, in the order it was listed. */ + #live = new Map(); + /** The queue of every reader of the current answer. */ + #readers = new Set(); + /** Each owner's reader. */ + #owners = new Map(); + /** Whether a {@link Lister} exists. While one does, readers wait for its answer to sync. */ + #lister = false; + /** Whether the lister's answer has synced. */ + #answered = false; + #used = new Signal(false); + #closed: GetPromise; + + constructor(closed: GetPromise) { + this.#closed = closed; + } + + /** Whether anyone lists the broadcast. */ + get used(): Getter { + return this.#used; + } + + /** Settles once the broadcast ended. */ + get closed(): GetPromise { + return this.#closed; + } + + #synced(): boolean { + return !this.#lister || this.#answered; + } + + #counted() { + this.#used.set(this.#readers.size > 0); + } + + /** Re-evaluate whether the listing has synced, telling readers when it changed. */ + #resync(was: boolean) { + const now = this.#synced(); + if (!was && now) for (const queue of this.#readers) queue.sync(); + if (was && !now) for (const queue of this.#readers) queue.unsync(); + } + + #insert(name: string, entry: Entry) { + this.#live.set(name, entry); + for (const queue of this.#readers) queue.add(name, entry.info); + } + + #remove(name: string) { + if (!this.#live.delete(name)) return; + for (const queue of this.#readers) queue.remove(name); + } + + /** + * A reader with every listed track queued, ending `owner`'s older one with CANCELLED so an + * owner's listings cost one queue per broadcast however many requests reach it. + */ + read(owner?: object): Queue { + if (this.#closed.peek() !== undefined) { + throw new StreamError(StreamCode.Unroutable, { message: "broadcast is closed" }); + } + const queue = new Queue(); + for (const [name, entry] of this.#live) queue.pending.set(name, { kind: "add", info: entry.info }); + if (this.#synced()) queue.bound = queue.pending.size; + this.#readers.add(queue); + if (owner) { + const older = this.#owners.get(owner); + this.#owners.set(owner, queue); + if (older) { + this.#readers.delete(older); + older.finish(new StreamError(StreamCode.Cancel, { message: "replaced by a newer listing" })); + } + } + this.#counted(); + return queue; + } + + /** A reader left. A replaced or ended one already did. */ + release(queue: Queue, owner?: object) { + if (queue.detached) return; + queue.finish(null); + this.#readers.delete(queue); + if (owner && this.#owners.get(owner) === queue) this.#owners.delete(owner); + this.#counted(); + } + + /** List a track `own` created. */ + listOwn(name: string, own: object, info: track.Info) { + const entry = this.#live.get(name); + if (!entry) { + this.#insert(name, { info, own, answered: false }); + } else if (entry.own && !entry.answered) { + // The earlier track of the name has not unlisted yet, but it left the table for this + // one: readers see it go and the new one arrive. + this.#remove(name); + this.#insert(name, { info, own, answered: false }); + } else { + // Listed already by the answer, and it never stopped being listed: only the owner moves. + entry.own = own; + } + } + + /** Unlist a track `own` created, unless a later one took its name. */ + unlistOwn(name: string, own: object) { + const entry = this.#live.get(name); + if (entry?.own !== own) return; + entry.own = undefined; + if (!entry.answered) this.#remove(name); + } + + /** Lister support; see {@link Lister}. */ + attach() { + if (this.#lister) throw new Error("the broadcast already has a lister"); + const was = this.#synced(); + this.#lister = true; + this.#resync(was); + } + + detachLister() { + const was = this.#synced(); + this.#lister = false; + // Synced from here on, with only the producer's tracks. + this.#unanswer(); + this.#resync(was); + } + + answerAdd(name: string, info: track.Info) { + const entry = this.#live.get(name); + if (entry?.answered) throw new Error(`duplicate listed track: ${name}`); + if (entry) entry.answered = true; + else this.#insert(name, { info, answered: true }); + } + + answerRemove(name: string) { + const entry = this.#live.get(name); + if (!entry?.answered) throw new Error(`track not listed: ${name}`); + entry.answered = false; + if (!entry.own) this.#remove(name); + } + + answerSynced() { + const was = this.#synced(); + this.#answered = true; + this.#resync(was); + } + + /** Drop the answer: every track it listed, and its sync. */ + answerClear() { + const was = this.#synced(); + this.#unanswer(); + this.#resync(was); + } + + #unanswer() { + for (const [name, entry] of [...this.#live]) { + if (entry.answered) this.answerRemove(name); + } + this.#answered = false; + } + + /** End every reader with `end` and detach them. */ + detach(end: Error | null) { + for (const queue of this.#readers) queue.finish(end); + this.#readers.clear(); + this.#owners.clear(); + this.#counted(); + } +} + +let makeTracks: (listing: Listing, owner?: object) => Tracks; + +/** + * List a broadcast's tracks, ending `owner`'s older listing of it with CANCELLED. + * + * @internal + */ +export function listTracks(listing: Listing, owner?: object): Tracks { + return makeTracks(listing, owner); +} + +/** + * The tracks a broadcast lists, from its consumer's `tracks()`. + * + * Reports every track listed when it was created, then `synced`, then each change as it + * happens, until the broadcast ends. Listing a broadcast never requests a track: subscribe to + * each listed one through the consumer's `track(name)`. + * + * @public + */ +export class Tracks { + #listing: Listing; + #queue: Queue; + #owner?: object; + + private constructor(listing: Listing, owner?: object) { + this.#listing = listing; + this.#queue = listing.read(owner); + this.#owner = owner; + } + + static { + makeTracks = (listing, owner) => new Tracks(listing, owner); + } + + /** + * The next event, or undefined once the broadcast ended or this listing closed. Throws the + * error that ended the listing, such as the peer refusing it. + */ + async next(): Promise { + for (;;) { + if (this.#queue.ready()) return this.#queue.next(); + await Signal.race(this.#queue.changed); + } + } + + /** The events as they arrive; leaving the loop early closes the listing. */ + async *[Symbol.asyncIterator](): AsyncGenerator { + try { + for (;;) { + const event = await this.next(); + if (!event) return; + yield event; + } + } finally { + this.close(); + } + } + + /** Stop listing. Idempotent. */ + close(): void { + this.#listing.release(this.#queue, this.#owner); + } +} + +// pending: /quest/m1/public-model/answer.md +/** + * Answers a broadcast's listings for a handler that knows the tracks it would serve, as a + * session lists its peer's. A broadcast has at most one. + * + * While it exists, readers wait for it to {@link synced} before reporting `synced`. Answer + * while {@link used}: list what the publisher behind it holds and keep it current, then + * {@link clear} once unused. {@link close} withdraws what it listed. + */ +export class Lister { + #listing: Listing; + #closed = false; + + constructor(listing: Listing) { + listing.attach(); + this.#listing = listing; + } + + /** Whether anyone lists the broadcast. */ + get used(): Getter { + return this.#listing.used; + } + + /** Settles once the broadcast ended. */ + get ended(): GetPromise { + return this.#listing.closed; + } + + /** List a track. Throws if this answer already lists it. */ + add(name: string, info: track.Info): void { + this.#listing.answerAdd(name, info); + } + + /** Stop listing a track. Throws if this answer does not list it. */ + remove(name: string): void { + this.#listing.answerRemove(name); + } + + /** The answer holds everything this hop knows now; later changes are live. */ + synced(): void { + this.#listing.answerSynced(); + } + + /** Withdraw the answer: every track it listed, and its sync. */ + clear(): void { + this.#listing.answerClear(); + } + + /** End every current reader as if the broadcast ended, and withdraw the answer. A later reader asks again. */ + finish(): void { + this.#listing.detach(null); + this.#listing.answerClear(); + } + + /** Fail every current reader with `err`, and withdraw the answer. A later reader asks again. */ + abort(err: Error): void { + this.#listing.detach(err); + this.#listing.answerClear(); + } + + /** Stop answering: the broadcast lists only its producer's tracks from here on. Idempotent. */ + close(): void { + if (this.#closed) return; + this.#closed = true; + this.#listing.detachLister(); + } +} diff --git a/js/net/src/lite/connection.ts b/js/net/src/lite/connection.ts index ea0609a3845..f1ed8a754ac 100644 --- a/js/net/src/lite/connection.ts +++ b/js/net/src/lite/connection.ts @@ -28,6 +28,7 @@ import { DataType, StreamId } from "./stream.ts"; import { Subscribe } from "./subscribe.ts"; import { Subscriber } from "./subscriber.ts"; import { Track as TrackMessage } from "./track.ts"; +import { Tracks as TracksMessage } from "./tracks.ts"; import { hasAuth, hasDatagrams, @@ -36,6 +37,7 @@ import { hasProbeRtt, hasSetupStream, hasTrackStream, + hasTracks, Version, versionName, } from "./version.ts"; @@ -363,6 +365,10 @@ export class Connection implements Established { await this.#publisher.runTrackInfo(msg, stream); })(), ); + } else if (typ === StreamId.Tracks) { + if (!hasTracks(this.#version)) throw new Error("TRACKS is not supported on this version"); + const msg = await TracksMessage.decode(stream.reader, this.#version); + await this.#publisher.runTracks(msg, stream); } else if (typ === StreamId.Probe) { if (this.#version === Version.DRAFT_01 || this.#version === Version.DRAFT_02) throw new Error("PROBE is not supported on this version"); diff --git a/js/net/src/lite/index.ts b/js/net/src/lite/index.ts index 278e7b731e9..8e588403e91 100644 --- a/js/net/src/lite/index.ts +++ b/js/net/src/lite/index.ts @@ -10,4 +10,5 @@ export * from "./setup.ts"; export * from "./stream.ts"; export * from "./subscribe.ts"; export * from "./track.ts"; +export * from "./tracks.ts"; export * from "./version.ts"; diff --git a/js/net/src/lite/malformed.test.ts b/js/net/src/lite/malformed.test.ts index abc5967e37c..f81e1e460ef 100644 --- a/js/net/src/lite/malformed.test.ts +++ b/js/net/src/lite/malformed.test.ts @@ -131,8 +131,8 @@ for (const [alpn, version] of versions) { } for (const [alpn, version] of [[ALPN_04, Version.DRAFT_04], ...versions] as const) { - // An unassigned type, and the Auth Stream on versions that predate it. - for (const typ of version === Version.DRAFT_07 ? [8] : [8, StreamId.Auth]) { + // An unassigned type, and the Tracks and Auth Streams on versions that predate them. + for (const typ of version === Version.DRAFT_07 ? [9] : [9, StreamId.Tracks, StreamId.Auth]) { test(`${alpn}: unknown stream type ${typ} resets only its stream`, async () => { const f = fixture(alpn, version); try { diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index cb53ef66521..63015bf351a 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -40,6 +40,7 @@ import { SubscribeUpdate, } from "./subscribe.ts"; import { TrackInfo as TrackInfoMessage, type Track as TrackMessage } from "./track.ts"; +import { encodeTracksEvent, type Tracks as TracksMessage } from "./tracks.ts"; import { hasAnnounceId, hasAnnounceOk, @@ -97,6 +98,22 @@ function acknowledged(writer: Writer): Promise { return writer.closed.catch(() => {}); } +// The TRACK_INFO a TRACK stream, or a listing's TRACKS_ADD, answers for a track. +function trackInfoMessage(info: track.Info, version: Version): TrackInfoMessage { + return new TrackInfoMessage({ + priority: info.priority, + refresh: info.refresh, + // Publisher Max Age: the publisher's retention bound, advertised so relays re-serve + // with the same window. + maxAge: info.maxAge, + // `#serveGroup` emits each frame converted to it. Unchecked timestamps never enter + // moq-lite, whose receivers check them. + timescale: liteTimescale(info), + // An older peer is served the frames inflated. + compress: hasCompression(version) && !!info.compress, + }); +} + /** A track's cached TRACK_INFO, and the request behind it while requesters hold it. */ interface TrackInfoEntry { info: Promise; @@ -1467,6 +1484,77 @@ export class Publisher { } } + /** + * Answers a TRACKS stream (0x8): the broadcast's listed tracks, SYNCED, then each change, + * until the broadcast ends (FIN) or a newer listing of it on this session replaces this one. + * + * Each listing queues the broadcast's whole listed set, so the session lists each broadcast + * once: a newer listing replaces the older rather than being refused, since a subscriber + * that stops listing and lists again can open the new stream before we see the old one + * close. The requester's FIN or reset cancels it. Not owed: it lasts as long as the broadcast. + * + * @internal + */ + async runTracks(msg: TracksMessage, stream: Stream) { + // Charged before anything else and held until the listing ends, as in Rust. A peer past + // the cap loses the session. + const interest = this.#interests.acquire(msg.broadcast, "", "tracks"); + const cancel = new AbortController(); + const watch = this.#gate.watch(msg.broadcast, () => { + console.debug(`tracks revoked: broadcast=${msg.broadcast}`); + cancel.abort(unauthorized(msg.broadcast)); + }); + // TRACKS is the requester's only message: its FIN or reset ends the listing. + void stream.reader.done().then( + (fin) => { + if (fin) { + cancel.abort(new StreamError(StreamCode.Cancel, { message: "listing cancelled" })); + return; + } + const violation = new ProtocolViolation("data after TRACKS"); + closeViolation(this.#quic, violation); + cancel.abort(violation); + }, + (err: unknown) => cancel.abort(error(err)), + ); + let tracks: broadcast.Tracks | undefined; + try { + if (watch.revoked()) throw unauthorized(msg.broadcast); + const origin = this.#publish && wireOf(this.#publish); + const front = + origin && + (origin.local(msg.broadcast, msg.epoch) ?? + (await origin.demand(msg.broadcast, msg.epoch, cancel.signal))); + if (!front) throw new NotFound(`broadcast ${msg.broadcast}`); + + tracks = wireOf(front).tracks(this); + console.debug(`tracks: broadcast=${msg.broadcast}`); + for (;;) { + // Pull an event only once the last is written, so a stalled requester's changes + // collapse in the listing's queue rather than pile up here. + const event = await untilAborted(tracks.next(), cancel.signal); + if (!event) break; + await untilAborted( + encodeTracksEvent( + stream.writer, + event.kind === "added" ? { ...event, info: trackInfoMessage(event.info, this.version) } : event, + this.version, + ), + cancel.signal, + ); + } + stream.writer.close(); + } catch (err) { + const e = cancel.signal.aborted ? error(cancel.signal.reason) : error(err); + console.debug(`tracks end: broadcast=${msg.broadcast} error=${reason(e)}`); + stream.writer.reset(e); + } finally { + tracks?.close(); + watch.dispose(); + interest.release(); + } + } + // Resolve (and cache) a track's immutable TRACK_INFO by asking the application. // `resolveTrackInfo` triggers a TrackRequest the app answers with accept(TrackInfo); // only the immutable properties are needed (not the groups). Cached because they're @@ -1492,20 +1580,8 @@ export class Publisher { if (reuse && entry?.answered) return entry.info; if (!entry || entry.release.signal.aborted) { const release = new AbortController(); - const info = (async () => { - const info = await wireOf(front).resolveTrackInfo(track, release.signal); - return new TrackInfoMessage({ - priority: info.priority, - refresh: info.refresh, - // Publisher Max Age: the publisher's retention bound, advertised so - // relays re-serve with the same window. - maxAge: info.maxAge, - // `#serveGroup` emits each frame converted to it. Unchecked timestamps never - // enter moq-lite, whose receivers check them. - timescale: liteTimescale(info), - compress: hasCompression(this.version) && !!info.compress, - }); - })(); + const info = (async () => + trackInfoMessage(await wireOf(front).resolveTrackInfo(track, release.signal), this.version))(); const created: TrackInfoEntry = { info, holders: 0, answered: false, release }; info.then( diff --git a/js/net/src/lite/stream.ts b/js/net/src/lite/stream.ts index 6917174c756..7c95e5ca558 100644 --- a/js/net/src/lite/stream.ts +++ b/js/net/src/lite/stream.ts @@ -7,6 +7,7 @@ export const StreamId = { Goaway: 5, Track: 6, Auth: 7, + Tracks: 8, ClientCompat: 0x20, ServerCompat: 0x21, } as const; diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 8be85d8d4fe..a5739bdc165 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -2,6 +2,7 @@ import { expect, jest, spyOn, test } from "bun:test"; import { Manual, Milli, Nano } from "@moq/clock"; import { Signal } from "@moq/signals"; import type { Grant } from "../auth.ts"; +import type { AuthSession } from "../auth_session.ts"; import type { Probe as ProbeStats } from "../connection/stats.ts"; import * as Epoch from "../epoch.ts"; import { @@ -1448,6 +1449,104 @@ test.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])( }, ); +// A TRACKS request carries no epoch for an epochless route, so it would list the replacement. +test("a listing of an epochless handle retired while its token waits never reaches the wire", async () => { + const { quic, streams, opened } = fakeSession(); + let release!: () => void; + const released = new Promise((resolve) => (release = resolve)); + const auth = { + ready: (_path: Path.Valid, _direction: string, opts?: { signal?: AbortSignal }) => + new Promise((resolve, reject) => { + void released.then(() => resolve(true)); + opts?.signal?.addEventListener("abort", () => reject(opts.signal?.reason), { once: true }); + }), + } as unknown as AuthSession; + const version = Version.DRAFT_07; + const subscriber = new Subscriber( + quic, + version, + HopSchema.parse(1n), + undefined, + undefined, + undefined, + undefined, + undefined, + auth, + ); + const announced = subscriber.announced(); + await opened(0); + const send = async (encode: (writer: Writer) => Promise) => { + const writer = new Writer( + new WritableStream({ + write: (chunk) => streams[0].inbound.enqueue(new Uint8Array(chunk)), + }), + version, + ); + await encode(writer); + }; + const path = Path.from("pool"); + await send((writer) => new AnnounceOk(PEER, 0).encode(writer, version)); + await send((writer) => + encodeAnnounceBroadcast(writer, { status: "active", suffix: path, hops: [PUBLISHER_A] }, version), + ); + await announced.next(); + const held = subscriber.consume(Path.from("pool/job")); + const tracks = held.tracks(); + const event = tracks.next().catch((err: unknown) => err); + + await send((writer) => encodeAnnounceBroadcast(writer, { status: "endedId", id: 0n }, version)); + await announced.next(); + await send((writer) => + encodeAnnounceBroadcast(writer, { status: "active", suffix: path, hops: [PUBLISHER_B] }, version), + ); + await announced.next(); + release(); + + const err = await event; + expect(err).toBeInstanceOf(StreamError); + expect((err as StreamError).code).toBe(StreamCode.Unroutable); + expect(streams.length).toBe(1); + tracks.close(); + held.close(); + announced.close(); + subscriber.close(); +}); + +test("a listing whose readers leave as its token settles never reaches the wire", async () => { + const { quic, streams } = fakeSession(); + let release!: () => void; + const released = new Promise((resolve) => (release = resolve)); + let ask!: () => void; + const asked = new Promise((resolve) => (ask = resolve)); + // Ignores `until`, so the wait settles admitted after the readers left. + const auth = { + ready: () => { + ask(); + return released.then(() => true); + }, + } as unknown as AuthSession; + const subscriber = new Subscriber( + quic, + Version.DRAFT_07, + HopSchema.parse(1n), + undefined, + undefined, + undefined, + undefined, + undefined, + auth, + ); + const held = subscriber.consume(Path.from("room")); + const tracks = held.tracks(); + await asked; + release(); + tracks.close(); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(streams.length).toBe(0); + held.close(); + subscriber.close(); +}); + test.each(["track", "fetch", "open"] as const)( "retirement settles coalesced fetches during %s setup", async (stage) => { diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index e0427a8b5e4..cdffbf67c95 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -9,11 +9,22 @@ import type { Probe as ProbeStats } from "../connection/stats.ts"; import { BroadcastCache } from "../consume.ts"; import * as DatagramStream from "../datagram_stream.ts"; import type * as Epoch from "../epoch.ts"; -import { error, ProtocolViolation, reason, StreamCode, StreamError, sessionCause, unauthorized } from "../error.ts"; +import { + error, + ProtocolViolation, + reason, + SessionCode, + SessionError, + StreamCode, + StreamError, + sessionCause, + unauthorized, +} from "../error.ts"; import * as netGroup from "../group.ts"; import { Cost, type Hop, MAX_HOPS, type Route, routesEqual, UNKNOWN_HOP } from "../hop.ts"; import { groupBounds, hiddenBelow, hooks, scopeCaptures, scopeHead, scopeOverlaps } from "../internal.ts"; import { grace } from "../ledger.ts"; +import type { Lister } from "../listing.ts"; import * as Path from "../path.ts"; import { sendOrder } from "../priority.ts"; import { type OpenOptions, type Reader, Stream, Writer } from "../stream.ts"; @@ -24,6 +35,7 @@ import { untilAborted } from "../util/abort.ts"; import { Deadline, REQUEST_TIMEOUT_MS, requestOptions } from "../util/deadline.ts"; import { abandoned, isAbandoned, unwanted } from "../util/demand.ts"; import { LIMITS, type Limits, SlotMap, Slots } from "../util/limits.ts"; +import { Budget } from "../util/yield.ts"; import { overrideBroadcastWire, wireOf } from "../wire.ts"; import { AnnounceHistory, @@ -50,6 +62,7 @@ import { SubscribeUpdate, } from "./subscribe.ts"; import { TrackInfo, Track as TrackMessage } from "./track.ts"; +import { decodeTracksEventMaybe, Tracks as TracksMessage } from "./tracks.ts"; import { hasAnnounceId, hasAnnounceOk, @@ -61,12 +74,17 @@ import { hasRefresh, hasSynced, hasTrackStream, + hasTracks, resolvesStart, updateSupported, Version, waitsForSubscriberFin, } from "./version.ts"; +// A peer's buffered TRACKS events decode without reaching a task boundary, and a flood of adds +// and removals of one name never reaches the announces cap: yield so it cannot starve the host. +const listingBudget = new Budget(); + // Request streams refuse exhausted peer credit, with one deadline covering create and answer. export const SUBSCRIBE_SETUP_TIMEOUT_MS = REQUEST_TIMEOUT_MS; @@ -629,6 +647,8 @@ export class Subscriber { }, }; const consumer = new ConsumeBroadcast(this, path, consumed); + // Registered before anyone can list the broadcast, so no reader syncs without the peer. + void this.#runListing(path, consumed, wireOf(consumer).lister()); void (async () => { for (;;) { @@ -641,6 +661,148 @@ export class Subscriber { return this.#consumes.insert(path, instance, consumer); } + // Answer listings of a consumed broadcast from the peer's TRACKS while anyone lists it, until + // the broadcast closes. Every listing of the broadcast shares one TRACKS stream. + async #runListing(broadcast: Path.Valid, consumed: Consumed, lister: Lister) { + try { + for (;;) { + if (lister.ended.peek() !== undefined || this.#closed.signal.aborted) return; + if (!lister.used.peek()) { + await untilAborted(race([Signal.race(lister.used, lister.ended)]), this.#closed.signal).catch( + () => {}, + ); + continue; + } + if (!hasTracks(this.version)) { + // An empty answer would claim the peer lists nothing. + lister.abort(new Error("track listing requires moq-lite-07 or newer")); + continue; + } + await this.#listTracks(broadcast, consumed, lister); + } + } finally { + lister.close(); + } + } + + // Forward the peer's TRACKS listing of `broadcast` into `lister` until the peer ends it, it + // fails, our grant stops covering the broadcast, or nobody lists the broadcast any more. + // Returns without answering when setup was cut short, for the caller to look again. + async #listTracks(broadcast: Path.Valid, consumed: Consumed, lister: Lister): Promise { + // Until the request is out, the consumed instance must still be the selected one: a TRACKS + // request without an epoch would list its replacement. + const valid = consumed.valid; + if (valid?.aborted) { + lister.abort(error(valid.reason)); + return; + } + const setup = valid ? AbortSignal.any([this.#closed.signal, valid]) : this.#closed.signal; + // A listing is a subscribe: one opened before a token that covers it is answered would be + // refused by the peer. + const ready = await this.#auth + ?.ready(broadcast, "subscribe", { + signal: setup, + until: async () => { + while (lister.used.peek() && lister.ended.peek() === undefined) { + await Signal.race(lister.used, lister.ended); + } + }, + }) + .catch(() => false); + // The readers may have left while the wait settled, before the unused watch below is armed. + if (ready === false || setup.aborted || !lister.used.peek()) return; + + // Armed before the first check and held to the end, so a shrink is never missed. + const refused = unauthorized(broadcast); + const revoked = new AbortController(); + const disposeGrant = this.#grant?.subscribe(() => { + if (this.#denied(broadcast)) revoked.abort(refused); + }); + if (this.#denied(broadcast)) { + disposeGrant?.(); + lister.abort(refused); + return; + } + // Synced with this hop, which holds nothing yet: a relay never waits on its upstream, + // so the peer's tracks follow as live adds. + lister.synced(); + + // Each listed track holds an announces slot: discovery state the peer makes us hold. + const listed = new Set(); + const unused = new AbortController(); + const stop = AbortSignal.any([unused.signal, revoked.signal, this.#closed.signal]); + const disposeUsed = lister.used.subscribe((used) => { + if (!used) unused.abort(new StreamError(StreamCode.Cancel, { message: "listing unused" })); + }); + let stream: Stream | undefined; + try { + const opening = Stream.open(this.#quic, { version: this.version, ...requestOptions }); + const cut = valid ? AbortSignal.any([stop, valid]) : stop; + void opening.then( + (late) => { + if (cut.aborted) late.abort(error(cut.reason)); + }, + () => {}, + ); + stream = await untilAborted(opening, cut); + await stream.writer.u53(StreamId.Tracks); + await new TracksMessage(broadcast, consumed.epoch).encode(stream.writer, this.version); + let synced = false; + for (;;) { + const pause = listingBudget.poll(); + if (pause !== undefined) await untilAborted(pause, stop); + const event = await untilAborted(decodeTracksEventMaybe(stream.reader, this.version), stop); + if (!event) { + // The peer's broadcast ended, whether or not its retraction has arrived: end the + // listing and let go of what it listed. FIN our half too, so the stream is done. + stream.writer.close(); + lister.finish(); + return; + } + if (event.kind === "added") { + if (listed.has(event.name)) throw new ProtocolViolation(`track listed twice: ${event.name}`); + if (!this.#announces.tryAcquire()) { + throw new SessionError(SessionCode.TooManyRequests, { reason: "too many listed tracks" }); + } + listed.add(event.name); + lister.add(event.name, this.#toModelInfo(event.info)); + } else if (event.kind === "removed") { + if (!listed.delete(event.name)) + throw new ProtocolViolation(`unlisted track removed: ${event.name}`); + this.#announces.release(); + lister.remove(event.name); + } else { + // This hop synced its readers when the listing began; only the order matters here. + if (synced) throw new ProtocolViolation("duplicate SYNCED"); + synced = true; + } + } + } catch (err) { + if (revoked.signal.aborted) { + // Reset rather than cancel, so the peer reads a revocation instead of a routine end. + console.debug(`tracks revoked: broadcast=${broadcast}`); + stream?.abort(refused); + lister.abort(refused); + return; + } + if (unused.signal.aborted) { + // Nobody lists the broadcast any more: cancel the peer's listing. + stream?.close(); + lister.clear(); + return; + } + closeViolation(this.#quic, err); + const e = await sessionCause(this.#quic, err); + console.debug(`tracks failed: broadcast=${broadcast} error=${reason(e)}`); + stream?.abort(e); + lister.abort(e); + } finally { + disposeGrant?.(); + disposeUsed(); + this.#announces.release(listed.size); + } + } + async #runSubscribe(broadcast: Path.Valid, consumed: Consumed, request: track.Request) { const id = this.#subscribeNext++; const subscription = request.subscription; diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts new file mode 100644 index 00000000000..36a08cbe7bc --- /dev/null +++ b/js/net/src/lite/tracks.test.ts @@ -0,0 +1,558 @@ +import { afterEach, expect, mock, spyOn, test } from "bun:test"; +import { Milli } from "@moq/clock"; +import { type Getter, Signal } from "@moq/signals"; +import type { Grant } from "../auth.ts"; +import { Producer as BroadcastProducer, type Tracks } from "../broadcast.ts"; +import { accept, connect } from "../connection/index.ts"; +import * as Epoch from "../epoch.ts"; +import { fromTransport, NotFound, SessionCode, StreamCode, StreamError } from "../error.ts"; +import { randomHop } from "../hop.ts"; +import { createMockTransportPair } from "../mock.ts"; +import { Producer as OriginProducer } from "../origin.ts"; +import * as Path from "../path.ts"; +import { Reader, Stream, Writer } from "../stream.ts"; +import { Timescale } from "../time.ts"; +import type { Limits } from "../util/limits.ts"; +import { wireOf } from "../wire.ts"; +import { Publisher } from "./publisher.ts"; +import { StreamId } from "./stream.ts"; +import { Subscriber } from "./subscriber.ts"; +import { TrackInfo } from "./track.ts"; +import { decodeTracksEventMaybe, encodeTracksEvent, type TracksEvent, Tracks as TracksMessage } from "./tracks.ts"; +import { ALPN_06, ALPN_07_WIP, Version } from "./version.ts"; + +const url = new URL("https://localhost:4443/test"); +const version = Version.DRAFT_07; + +afterEach(() => mock.restore()); + +/** Lets queued microtasks, signal notifications, and mock stream deliveries run. */ +function settle(): Promise { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +async function bytes(f: (w: Writer) => Promise): Promise { + const chunks: number[] = []; + const writer = new Writer( + new WritableStream({ write: (chunk) => void chunks.push(...chunk) }), + version, + ); + await f(writer); + writer.close(); + await writer.closed; + return Uint8Array.from(chunks); +} + +function format(event: Awaited>): string { + if (!event) return "end"; + if (event.kind === "synced") return "synced"; + return `${event.kind === "added" ? "+" : "-"}${event.name}`; +} + +/** The next event of a listing, formatted like the model tests, or the error that ended it. */ +async function next(tracks: Tracks): Promise { + try { + return format(await tracks.next()); + } catch (err) { + return `error: ${err instanceof StreamError ? err.code : String(err)}`; + } +} + +test("SYNCED is an empty type four, and an unknown event is refused", async () => { + expect(await bytes((w) => encodeTracksEvent(w, { kind: "synced" }, version))).toEqual(new Uint8Array([4, 0])); + await expect( + decodeTracksEventMaybe(new Reader(undefined, new Uint8Array([2, 0]), version), version), + ).rejects.toThrow("unknown tracks event"); + await expect( + decodeTracksEventMaybe(new Reader(undefined, new Uint8Array([4, 1, 0]), version), version), + ).rejects.toThrow("too few bytes"); +}); + +test("events and the request round-trip", async () => { + const events: TracksEvent[] = [ + { + kind: "added", + name: "video", + info: new TrackInfo({ priority: 3, refresh: 4, maxAge: 2000, timescale: 1_000_000 }), + }, + { kind: "removed", name: "video" }, + { kind: "synced" }, + ]; + const encoded = await bytes(async (w) => { + for (const event of events) await encodeTracksEvent(w, event, version); + }); + const reader = new Reader(undefined, encoded, version); + const added = await decodeTracksEventMaybe(reader, version); + expect(added).toMatchObject({ kind: "added", name: "video", info: { priority: 3, refresh: 4, maxAge: 2000 } }); + expect(added?.kind === "added" && added.info.timescale).toBe(Timescale(1_000_000)); + expect(await decodeTracksEventMaybe(reader, version)).toEqual({ kind: "removed", name: "video" }); + expect(await decodeTracksEventMaybe(reader, version)).toEqual({ kind: "synced" }); + expect(await decodeTracksEventMaybe(reader, version)).toBeUndefined(); + + const epoch = Epoch.parse("01900000-0000-7000-8000-000000000001"); + const request = await bytes((w) => new TracksMessage(Path.from("room/alice"), epoch).encode(w, version)); + const got = await TracksMessage.decode(new Reader(undefined, request, version), version); + expect(got.broadcast).toBe(Path.from("room/alice")); + expect(got.epoch).toBe(epoch); +}); + +test("older versions have no tracks stream", async () => { + await expect(bytes((w) => new TracksMessage(Path.from("room")).encode(w, Version.DRAFT_06))).rejects.toThrow( + "not supported", + ); + await expect(bytes((w) => encodeTracksEvent(w, { kind: "synced" }, Version.DRAFT_06))).rejects.toThrow( + "not supported", + ); +}); + +/** A lite-07 subscriber whose publisher the test plays by hand. */ +function handPlayed({ limits, grant }: { limits?: Limits; grant?: Getter } = {}) { + const pair = createMockTransportPair(ALPN_07_WIP); + const closing = spyOn(pair.client, "close"); + const subscriber = new Subscriber( + pair.client, + version, + randomHop(), + undefined, + undefined, + undefined, + grant, + limits, + ); + // The TRACKS request the subscriber opens next. + const listed = async () => { + const stream = await Stream.accept(pair.server, version); + if (!stream) throw new Error("the subscriber never listed"); + expect(await stream.reader.u53()).toBe(StreamId.Tracks); + const msg = await TracksMessage.decode(stream.reader, version); + return { + msg, + stream, + send: (event: TracksEvent) => encodeTracksEvent(stream.writer, event, version), + }; + }; + return { pair, closing, subscriber, listed }; +} + +const info = new TrackInfo({ priority: 2, timescale: 1000 }); + +test("a session syncs at once and forwards the peer's tracks as live adds", async () => { + const { subscriber, listed } = handPlayed(); + const remote = subscriber.consume(Path.from("room")); + const tracks = remote.tracks(); + expect(await next(tracks)).toBe("synced"); + + const peer = await listed(); + expect(peer.msg.broadcast).toBe(Path.from("room")); + await peer.send({ kind: "added", name: "a", info }); + await peer.send({ kind: "synced" }); + const added = await tracks.next(); + expect(added).toMatchObject({ kind: "added", name: "a", info: { priority: 2, timescale: Timescale.MILLI } }); + + await peer.send({ kind: "removed", name: "a" }); + expect(await next(tracks)).toBe("-a"); + + // The peer's broadcast ended: so does the listing, whether or not its retraction arrived. + peer.stream.writer.close(); + expect(await next(tracks)).toBe("end"); + expect(remote.closed.peek()).toBeUndefined(); + // The requester FINs its half too, so the stream does not linger half-open. + expect(await peer.stream.reader.done()).toBe(true); +}); + +test("every listing of a broadcast shares one TRACKS stream, cancelled once unused", async () => { + const { subscriber, listed, pair } = handPlayed(); + const remote = subscriber.consume(Path.from("room")); + const first = remote.tracks(); + const peer = await listed(); + await peer.send({ kind: "added", name: "a", info }); + expect(await next(first)).toBe("synced"); + expect(await next(first)).toBe("+a"); + + // A second reader, through a shared handle too, joins the same stream with what it holds. + const second = subscriber.consume(Path.from("room")).tracks(); + expect(await next(second)).toBe("+a"); + expect(await next(second)).toBe("synced"); + let opened = false; + const another = Stream.accept(pair.server, version).then((stream) => { + opened = true; + return stream; + }); + await settle(); + expect(opened).toBe(false); + + let cancelled = false; + void peer.stream.reader.done().then( + () => (cancelled = true), + () => (cancelled = true), + ); + first.close(); + await settle(); + expect(cancelled).toBe(false); + // The last reader leaving ends the requester's side. + second.close(); + await settle(); + expect(cancelled).toBe(true); + + // A later reader asks again. + const third = remote.tracks(); + await another; + expect(opened).toBe(true); + third.close(); +}); + +test("a peer's malformed listing closes the session", async () => { + for (const events of [ + [{ kind: "added", name: "a", info } as const, { kind: "added", name: "a", info } as const], + [{ kind: "removed", name: "a" } as const], + [{ kind: "synced" } as const, { kind: "synced" } as const], + ]) { + const { subscriber, listed, closing } = handPlayed(); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + for (const event of events) await peer.send(event); + let event: string; + do event = await next(tracks); + while (event === "synced" || event.startsWith("+")); + expect(event).toMatch(/^error: /); + expect(closing).toHaveBeenCalledWith(expect.objectContaining({ closeCode: SessionCode.ProtocolViolation })); + } +}); + +test("a listing past the announces cap is reset, and the session stays", async () => { + const cap = 3; + const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap, subscriptions: 10, fetches: 10 } }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + for (let i = 0; i <= cap; i++) await peer.send({ kind: "added", name: `t${i}`, info }); + let added = 0; + let end: string; + for (;;) { + end = await next(tracks); + if (end.startsWith("+")) added++; + else if (end !== "synced") break; + } + expect(added).toBe(cap); + expect(end).toMatch(/^error: /); + await settle(); + expect(closing).not.toHaveBeenCalled(); + // The reset released every slot: another listing lists the whole cap again. + const again = subscriber.consume(Path.from("room")).tracks(); + const retry = await listed(); + for (let i = 0; i < cap; i++) await retry.send({ kind: "added", name: `t${i}`, info }); + expect(await next(again)).toBe("synced"); + for (let i = 0; i < cap; i++) expect(await next(again)).toBe(`+t${i}`); + again.close(); +}); + +test("a flood of a peer's listing churn yields to the host's tasks", async () => { + const { subscriber, listed } = handPlayed(); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + // Churn of one name never reaches the announces cap, so only the budget bounds it. + const flood = await bytes(async (w) => { + await encodeTracksEvent(w, { kind: "added", name: "first", info }, version); + for (let i = 0; i < 20_000; i++) { + await encodeTracksEvent(w, { kind: "added", name: "churn", info }, version); + await encodeTracksEvent(w, { kind: "removed", name: "churn" }, version); + } + await encodeTracksEvent(w, { kind: "added", name: "last", info }, version); + }); + await peer.stream.writer.write(flood); + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+first"); + // The receiver is mid-flood: a task queued now runs before it reaches the end. + let last = false; + let ranFirst: boolean | undefined; + setTimeout(() => { + ranFirst = !last; + }, 0); + let event: string; + do event = await next(tracks); + while (event !== "+last"); + last = true; + await settle(); + expect(ranFirst).toBe(true); + tracks.close(); +}); + +test("a version without the Tracks Stream refuses the listing", async () => { + const pair = createMockTransportPair(); + const subscriber = new Subscriber(pair.client, Version.DRAFT_06, randomHop()); + const tracks = subscriber.consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe("error: Error: track listing requires moq-lite-07 or newer"); +}); + +function scoped(prefix: string): Grant { + return { publish: new Path.Patterns(), subscribe: new Path.Patterns([Path.Pattern.subtree(prefix)]) }; +} + +test("a listing our grant does not cover is refused, and never reaches the wire", async () => { + const { subscriber, pair } = handPlayed({ grant: new Signal(scoped("other")) }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe(`error: ${StreamCode.Unauthorized}`); + expect(pair.client.sendStreams.bidi).toHaveLength(0); +}); + +test("a grant that stops covering a listing resets it with UNAUTHORIZED, and the session stays", async () => { + const grant = new Signal(scoped("room")); + const { subscriber, listed, closing } = handPlayed({ grant }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + await peer.send({ kind: "added", name: "a", info }); + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+a"); + + grant.set(scoped("other")); + expect(await next(tracks)).toBe(`error: ${StreamCode.Unauthorized}`); + const reset = await peer.stream.reader.done().then( + () => undefined, + (err: unknown) => fromTransport(err), + ); + expect((reset as StreamError).code).toBe(StreamCode.Unauthorized); + expect(closing).not.toHaveBeenCalled(); +}); + +/** A lite-07 publisher serving `origin`, answering TRACKS streams the test writes by hand. */ +function publisher(origin: OriginProducer) { + const pair = createMockTransportPair(ALPN_07_WIP); + const closing = spyOn(pair.server, "close"); + const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); + const list = async (broadcast: string) => { + const requester = await Stream.open(pair.client, { version }); + void (async () => { + const stream = await Stream.accept(pair.server, version); + if (!stream) throw new Error("no stream"); + expect(await stream.reader.u53()).toBe(StreamId.Tracks); + await publisher.runTracks(await TracksMessage.decode(stream.reader, version), stream); + })(); + await requester.writer.u53(StreamId.Tracks); + await new TracksMessage(Path.from(broadcast)).encode(requester.writer, version); + return { + requester, + next: async () => { + try { + const event = await decodeTracksEventMaybe(requester.reader, version); + return event?.kind === "added" + ? `+${event.name}` + : event?.kind === "removed" + ? `-${event.name}` + : (event?.kind ?? "end"); + } catch (err) { + const reset = fromTransport(err); + return `reset: ${reset instanceof StreamError ? reset.code : reset.message}`; + } + }, + }; + }; + return { list, closing }; +} + +test("data after a TRACKS request closes the session", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a"); + const { list, closing } = publisher(origin); + const listing = await list("room"); + expect(await listing.next()).toBe("+a"); + expect(await listing.next()).toBe("synced"); + await listing.requester.writer.u53(0); + expect(await listing.next()).toMatch(/^reset: /); + expect(closing).toHaveBeenCalledWith(expect.objectContaining({ closeCode: SessionCode.ProtocolViolation })); + broadcast.close(); +}); + +test("a publisher lists its broadcast's own tracks, never one served on request", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { priority: 5 }); + // Served on request: answered when asked, never listed. + void wireOf(broadcast) + .requested() + .then((request) => request?.accept({})); + const { list } = publisher(origin); + const listing = await list("room"); + expect(await listing.next()).toBe("+a"); + expect(await listing.next()).toBe("synced"); + + broadcast.consume().track("x").subscribe(); + broadcast.createTrack("b"); + expect(await listing.next()).toBe("+b"); + + broadcast.close(); + expect(await listing.next()).toBe("end"); +}); + +test("a newer listing of a broadcast on one session replaces the older", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a"); + const { list } = publisher(origin); + const older = await list("room"); + expect(await older.next()).toBe("+a"); + expect(await older.next()).toBe("synced"); + const newer = await list("room"); + expect(await newer.next()).toBe("+a"); + expect(await newer.next()).toBe("synced"); + expect(await older.next()).toBe(`reset: ${StreamCode.Cancel}`); + broadcast.createTrack("b"); + expect(await newer.next()).toBe("+b"); + broadcast.close(); +}); + +test("a path served through a covering prefix route is NOT_FOUND", async () => { + const origin = new OriginProducer(); + const dynamic = origin.dynamicRoute(Path.from("room")); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + const { list } = publisher(origin); + expect(await (await list("room/alice")).next()).toBe(`reset: ${StreamCode.NotFound}`); + expect(await (await list("absent")).next()).toBe(`reset: ${StreamCode.NotFound}`); + dynamic.close(); +}); + +test("a local read through a covering prefix route is NOT_FOUND, and through its exact route lists", async () => { + const origin = new OriginProducer(); + for (const prefix of ["room", "room/bob"]) { + const dynamic = origin.dynamicRoute(Path.from(prefix)); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + } + const consumer = origin.consume(); + const alice = consumer.request(Path.from("room/alice")); + const bob = consumer.request(Path.from("room/bob")); + await settle(); + + const covered = alice.active.peek(); + if (!covered) throw new Error("expected the covering route to serve"); + expect(() => covered.tracks()).toThrow(NotFound); + // Every handle cloned from it refuses too. + expect(() => covered.clone().tracks()).toThrow(NotFound); + + const exact = bob.active.peek(); + if (!exact) throw new Error("expected the exact route to serve"); + const tracks = exact.tracks(); + expect(await next(tracks)).toBe("+video"); + expect(await next(tracks)).toBe("synced"); + tracks.close(); + origin.close(); +}); + +test("a listing over a session carries each track's info, end to end", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const origin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ transport: pair.server, url, publish: origin.consume() }), + ]); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { + timescale: Timescale.MILLI, + maxAge: Milli(3000), + priority: 9, + refresh: 4, + compress: true, + }); + + const tracks = wireOf(client).consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe("synced"); + expect(await tracks.next()).toEqual({ + kind: "added", + name: "a", + info: { timescale: Timescale.MILLI, maxAge: Milli(3000), priority: 9, refresh: 4, compress: true }, + }); + broadcast.createTrack("b"); + expect(await next(tracks)).toBe("+b"); + broadcast.removeTrack("a"); + expect(await next(tracks)).toBe("-a"); + broadcast.close(); + expect(await next(tracks)).toBe("end"); + client.abort(); + server.abort(); +}); + +test("an epoch-pinned front lists its serving route's tracks", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const serverOrigin = new OriginProducer(); + const clientOrigin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client, consume: clientOrigin }), + accept({ transport: pair.server, url, publish: serverOrigin.consume() }), + ]); + const broadcast = serverOrigin.createBroadcast(Path.from("room")); + broadcast.createTrack("a"); + broadcast.announce({ epoch: Epoch.parse("01900000-0000-7000-8000-000000000001") }); + + const reader = clientOrigin.consume(); + const announced = reader.announced(); + expect(await announced.next()).toMatchObject({ prefix: Path.from("room"), kind: "start" }); + const request = reader.request(Path.from("room")); + await settle(); + const front = request.active.peek(); + if (!front) throw new Error("expected the origin to route the broadcast"); + const tracks = front.tracks(); + // The front syncs with its source's listing, which syncs at once with what the session holds. + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+a"); + broadcast.createTrack("b"); + expect(await next(tracks)).toBe("+b"); + tracks.close(); + request.close(); + announced.close(); + broadcast.close(); + client.abort(); + server.abort(); +}); + +test("an epoch-pinned front served through a covering prefix route is NOT_FOUND", async () => { + const origin = new OriginProducer(); + const dynamic = origin.dynamicRoute(Path.from("room"), { + epoch: Epoch.parse("01900000-0000-7000-8000-000000000001"), + }); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + const request = origin.consume().request(Path.from("room/alice")); + await settle(); + const front = request.active.peek(); + if (!front) throw new Error("expected the covering route to serve"); + expect(await next(front.tracks())).toBe(`error: ${StreamCode.NotFound}`); + // A later reader asks again, and is refused again. + expect(await next(front.tracks())).toBe(`error: ${StreamCode.NotFound}`); + request.close(); + origin.close(); +}); + +test("lite-06 cannot list over a session", async () => { + const pair = createMockTransportPair(ALPN_06); + const origin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ transport: pair.server, url, publish: origin.consume() }), + ]); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + const tracks = wireOf(client).consume(Path.from("room")).tracks(); + expect(await next(tracks)).toMatch(/requires moq-lite-07/); + broadcast.close(); + client.abort(); + server.abort(); +}); diff --git a/js/net/src/lite/tracks.ts b/js/net/src/lite/tracks.ts new file mode 100644 index 00000000000..6bb2e121f27 --- /dev/null +++ b/js/net/src/lite/tracks.ts @@ -0,0 +1,100 @@ +import type * as Epoch from "../epoch.ts"; +import { ProtocolViolation } from "../error.ts"; +import * as Path from "../path.ts"; +import type { Reader, Writer } from "../stream.ts"; +import { decodeError } from "../util/decode_error.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; +import * as Message from "./message.ts"; +import { TrackInfo } from "./track.ts"; +import { hasTracks, type Version } from "./version.ts"; + +const TRACKS_ADD = 0x0; +const TRACKS_REMOVE = 0x1; +// The same event the announce stream marks its initial set with. +const SYNCED = 0x4; + +function guardTracks(version: Version) { + if (!hasTracks(version)) throw new Error("tracks stream not supported for this version"); +} + +/** + * TRACKS request: the subscriber's only message on a Tracks Stream (0x8), listing one + * broadcast's tracks. Lite-07+. + */ +export class Tracks { + broadcast: Path.Valid; + /** The publisher instance the subscriber expects. */ + epoch?: Epoch.Valid; + + constructor(broadcast: Path.Valid, epoch?: Epoch.Valid) { + this.broadcast = broadcast; + this.epoch = epoch; + } + + async encode(w: Writer, version: Version): Promise { + guardTracks(version); + return Message.encode(w, async (w) => { + await w.string(Path.encode(this.broadcast)); + await encodeEpoch(w, version, this.epoch); + }); + } + + static async decode(r: Reader, version: Version): Promise { + guardTracks(version); + return Message.decode(r, async (r) => { + const broadcast = Path.decode(await r.string()); + const epoch = await decodeEpoch(r, version); + return new Tracks(broadcast, epoch); + }); + } +} + +/** + * One event the publisher sends on a Tracks Stream: TRACKS_ADD with the TRACK_INFO a Track + * Stream would answer, TRACKS_REMOVE, or SYNCED once the listing's initial set was sent. + */ +export type TracksEvent = + | { kind: "added"; name: string; info: TrackInfo } + | { kind: "removed"; name: string } + | { kind: "synced" }; + +/** Write one {@link TracksEvent}: its type, then its size-prefixed body. */ +export async function encodeTracksEvent(w: Writer, event: TracksEvent, version: Version): Promise { + guardTracks(version); + switch (event.kind) { + case "added": + return Message.encode( + w, + async (w) => { + await w.string(event.name); + await event.info.encode(w, version); + }, + { id: TRACKS_ADD }, + ); + case "removed": + return Message.encode(w, (w) => w.string(event.name), { id: TRACKS_REMOVE }); + case "synced": + return Message.encode(w, async () => {}, { id: SYNCED }); + } +} + +/** Read the next {@link TracksEvent}, or undefined at the publisher's FIN. */ +export async function decodeTracksEventMaybe(r: Reader, version: Version): Promise { + guardTracks(version); + if (await r.done()) return undefined; + const typ = await r.u53(); + switch (typ) { + case TRACKS_ADD: + return Message.decode(r, async (r) => { + const name = await r.string(); + return { kind: "added", name, info: await TrackInfo.decode(r, version) }; + }); + case TRACKS_REMOVE: + return Message.decode(r, async (r) => ({ kind: "removed", name: await r.string() })); + case SYNCED: + return Message.decode(r, async () => ({ kind: "synced" })); + default: + // A new stream with no earlier builds to tolerate. + throw decodeError(new ProtocolViolation(`unknown tracks event: ${typ}`)); + } +} diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 43580e38cfc..b500d818d8b 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -118,6 +118,25 @@ export function hasAuth(version: Version): boolean { } } +/** + * Whether a subscriber may open a Tracks Stream (0x8) to list a broadcast's tracks. + * Added in lite-07. + */ +export function hasTracks(version: Version): boolean { + // Explicitly list older versions so future versions default to having the stream. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /** Whether either endpoint may open a Padding Stream (0x2). Added in lite-07. */ export function hasPadding(version: Version): boolean { switch (version) { diff --git a/js/net/src/origin.ts b/js/net/src/origin.ts index 5ebfaa20587..c42bbc876bb 100644 --- a/js/net/src/origin.ts +++ b/js/net/src/origin.ts @@ -376,11 +376,14 @@ function unroutable(): StreamError { /** A served route from {@link Producer.dynamic}: the queue a handler drains. */ class ServeState { readonly root: Path.Valid; + /** The prefix the route claims; every served path is at or beneath it. */ + readonly prefix: Path.Valid; /** False when a session announced the route: it may serve that session's broadcast. */ readonly originated: boolean; - constructor(root: Path.Valid, originated: boolean) { + constructor(root: Path.Valid, prefix: Path.Valid, originated: boolean) { this.root = root; + this.prefix = prefix; this.originated = originated; } @@ -437,6 +440,9 @@ class ServeState { this.settled.update((n) => n + 1); return; } + // A route claiming a prefix names no broadcast, so a path it serves on request has no + // inventory to list. + if (path !== this.prefix) hooks.stampUnlisted(front); this.served.set(path, front); void front.closed.then(() => { if (this.served.get(path) !== front) return; @@ -1338,7 +1344,7 @@ export class Producer implements Table { #insertRoute(prefix: Path.Valid, route: Route, originated: boolean, publishes: boolean): Dynamic { prefix = this.#scope.prefix(prefix); - const server = new ServeState(this.#scope.root, originated); + const server = new ServeState(this.#scope.root, prefix, originated); server.onChange = (path) => this.#state.refresh(path); const entry: RouteEntry = { identity: {}, diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index 7505f36c9dd..90062b70346 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -8,6 +8,7 @@ import { Route } from "./hop.ts"; import type { Session as IetfSession } from "./ietf/adapter.ts"; import { Subscriber as IetfSubscriber } from "./ietf/subscriber.ts"; import { hooks, MAX_DATAGRAMS } from "./internal.ts"; +import type { Lister } from "./listing.ts"; import { Producer } from "./origin.ts"; import * as Path from "./path.ts"; import { Front, routeFailed } from "./resume.ts"; @@ -1755,6 +1756,44 @@ test.each([Milli(100), Milli(3_600_000)])( }, ); +test.each([ + ["fails", (answer: Lister) => answer.abort(new Error("old route failed"))], + ["ends", (answer: Lister) => answer.finish()], + ["changes", (answer: Lister) => answer.add("gone", { priority: 0 })], +])("a listing follows the source that took over as the old one's listing %s", async (_, end) => { + const front = new Front(); + const a = new BroadcastProducer(); + const b = new BroadcastProducer(); + const from = a.consume(); + // The old source's listing is answered, as a session answers its peer's. + const answer = wireOf(from).lister(); + answer.synced(); + b.createTrack("seq"); + front.serve(from); + const tracks = front.consumer.tracks(); + try { + expect(await tracks.next()).toMatchObject({ kind: "synced" }); + + // The new source takes over just as the old source's listing settles, which a later + // reader of it sees at the same moment the front's reader does. + const witness = from.tracks(); + expect(await witness.next()).toMatchObject({ kind: "synced" }); + const takeover = () => front.serve(b.consume()); + const settled = witness.next().then(takeover, takeover); + end(answer); + await settled; + expect(await tracks.next()).toMatchObject({ kind: "added", name: "seq" }); + b.createTrack("late"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "late" }); + } finally { + tracks.close(); + answer.close(); + front.consumer.close(); + a.close(); + b.close(); + } +}); + test("only delivery codes are a route failing", () => { // Mirrors `only_delivery_codes_are_a_route_failing` in rs/moq-net/src/model/resume.rs. const delivery = [0x0, 0x1, 0x3, 0x4, 0x31]; diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index b449a206af3..19ee4167c58 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -49,6 +49,7 @@ import { type ResumeSource, } from "./internal.ts"; import { grace } from "./ledger.ts"; +import type { Lister, Tracks, TracksEvent } from "./listing.ts"; import * as track from "./track.ts"; import { type Broadcast as BroadcastWire, registerWire, wireOf } from "./wire.ts"; @@ -1138,6 +1139,120 @@ export class Front { constructor() { this.consumer = makeFrontConsumer(this); void this.consumer.closed.then(() => this.conclude()); + void this.#list(wireOf(this.consumer).lister()); + } + + /** + * Answer listings of the front's broadcast with its serving source's listing, while anyone + * lists it. A source of the same instance lists the same tracks, so a takeover shows readers + * only what differs, once the new source's listing synced. + */ + async #list(lister: Lister): Promise { + const closed = this.consumer.closed; + // What the front lists, the source being read, and its pending next event. + let listing: + | { + listed: Set; + /** Names an earlier source listed that the current one has yet to list. */ + stale: Set; + source?: Source; + reader?: Tracks; + next?: Promise<{ event?: TracksEvent; err?: Error }>; + } + | undefined; + const wait = () => race([Signal.race(lister.used, this.#changed), closed]); + const stop = () => { + listing?.reader?.close(); + listing = undefined; + }; + try { + for (;;) { + if (closed.peek() !== undefined) return; + if (!lister.used.peek()) { + if (listing) { + stop(); + lister.clear(); + } + await wait(); + continue; + } + if (this.#concluded) { + // No route will serve it again, so nothing more will be listed. + stop(); + lister.finish(); + continue; + } + listing ??= { listed: new Set(), stale: new Set() }; + if (listing.source !== this.#source) { + listing.reader?.close(); + listing.source = this.#source; + listing.stale = new Set(listing.listed); + listing.next = undefined; + try { + listing.reader = this.#source?.wire.tracks(); + } catch (err) { + // A closing source lists nothing more; the front moves off it soon. + listing.reader = undefined; + // A live one refuses: served through a route claiming a prefix, it names + // nothing to list. A later reader asks again. + if (listing.source && listing.source.from.closed.peek() === undefined) { + stop(); + lister.abort(err instanceof Error ? err : new Error(String(err))); + continue; + } + } + } + const reader = listing.reader; + if (!reader) { + await wait(); + continue; + } + listing.next ??= reader.next().then( + (event) => ({ event }), + (err: unknown) => ({ err: err instanceof Error ? err : new Error(String(err)) }), + ); + const result = await race([listing.next, wait().then(() => undefined)]); + if (!result) continue; + listing.next = undefined; + // The source was replaced as its answer arrived: the new source's listing supersedes it. + if (listing.source !== this.#source) continue; + const { event, err } = result; + if (err) { + // Its readers fail, and a later reader asks again. + stop(); + lister.abort(err); + } else if (!event) { + // A closing source ended its listing: the front moves to another source, or ends + // with it. A live one's handler ended it, as a session does once the peer's + // broadcast ended, so the front's listing ends too. + if (listing.source?.from.closed.peek() !== undefined) { + listing.reader = undefined; + } else { + stop(); + lister.finish(); + } + } else if (event.kind === "added") { + if (!listing.stale.delete(event.name) && !listing.listed.has(event.name)) { + listing.listed.add(event.name); + lister.add(event.name, event.info); + } + } else if (event.kind === "removed") { + listing.stale.delete(event.name); + if (listing.listed.delete(event.name)) lister.remove(event.name); + } else { + // What the new source still lacks at its SYNCED is gone. + for (const name of listing.stale) { + listing.listed.delete(name); + lister.remove(name); + } + listing.stale.clear(); + lister.synced(); + } + } + } finally { + stop(); + lister.close(); + } } /** Serve every track from `from` from now on, once each copy proves compatible. */ @@ -1443,6 +1558,8 @@ class FrontConsumer extends broadcast.Consumer { const view = wireOf(this); registerWire(this, { requested: view.requested, + tracks: view.tracks, + lister: view.lister, subscribe: (name, options) => front.subscribe(name, options), resolveTrackInfo: (name, hold) => front.resolveTrackInfo(name, hold), largest: (name) => front.largest(name), diff --git a/js/net/src/util/limits.test.ts b/js/net/src/util/limits.test.ts index da1cdda264f..e338b7a9913 100644 --- a/js/net/src/util/limits.test.ts +++ b/js/net/src/util/limits.test.ts @@ -61,3 +61,26 @@ test("duplicate interests count individually, and a failed acquisition leaves no for (const held of [...tracks, ...subscribes, other]) held.release(); interests.acquire("path", "track", "subscribe").release(); }); + +test("each listing is charged on top of the interest under the same name", () => { + const interests = new Interests(new Slots(3, { reason: "full" })); + const track = interests.acquire("room", "", "track"); + const subscribe = interests.acquire("room", "", "subscribe"); + const listing = interests.acquire("room", "", "tracks"); + // A replacement listing must fit under the cap too, until the older is gone. + const replacement = interests.acquire("room", "", "tracks"); + expect(() => interests.acquire("room", "", "tracks")).toThrow(); + listing.release(); + replacement.release(); + track.release(); + subscribe.release(); + for (const held of [0, 1, 2].map(() => interests.acquire("room", "", "tracks"))) held.release(); +}); + +test("tryAcquire refuses at the cap without closing the session", () => { + const slots = new Slots(1, { reason: "full" }); + expect(slots.tryAcquire()).toBe(true); + expect(slots.tryAcquire()).toBe(false); + slots.release(); + expect(slots.tryAcquire()).toBe(true); +}); diff --git a/js/net/src/util/limits.ts b/js/net/src/util/limits.ts index 6259bc94b67..7c6cb6ec52b 100644 --- a/js/net/src/util/limits.ts +++ b/js/net/src/util/limits.ts @@ -28,12 +28,17 @@ export class Slots { /** Take one slot, or close the session with TOO_MANY_REQUESTS and throw at the cap. */ acquire() { - if (this.#live >= this.#max) { - const err = new Session(SessionCode.TooManyRequests, { reason: this.#reason }); - this.#quic?.close({ closeCode: err.code, reason: this.#reason }); - throw err; - } + if (this.tryAcquire()) return; + const err = new Session(SessionCode.TooManyRequests, { reason: this.#reason }); + this.#quic?.close({ closeCode: err.code, reason: this.#reason }); + throw err; + } + + /** Take one slot, or leave the session alone and return false at the cap. */ + tryAcquire(): boolean { + if (this.#live >= this.#max) return false; this.#live += 1; + return true; } /** Give back `count` slots. */ @@ -68,15 +73,21 @@ export class SlotMap extends Map { } } -type Kind = "track" | "subscribe"; -type Counts = { track: number; subscribe: number }; +type Kind = "track" | "subscribe" | "tracks"; +type Counts = Record; + +// A TRACK stream and its SUBSCRIBE are one interest; each listing is charged on top. +function cost(counts: Counts): number { + return Math.max(counts.track, counts.subscribe) + counts.tracks; +} /** * A session's subscription cap, charged per track as moq-net does. * * A TRACK stream held open and the SUBSCRIBE that follows it are one interest in the track, * so a track costs the larger of its two counts: a peer at the cap can still subscribe to - * the tracks it holds. + * the tracks it holds. A TRACKS listing is charged under the empty track name on top of + * that, each until it is gone, replaced or not. * * @internal */ @@ -91,9 +102,17 @@ export class Interests { /** Charge one request for `broadcast`'s `track` until the returned handle is released. */ acquire(broadcast: string, track: string, kind: Kind): Interest { let tracks = this.#tracks.get(broadcast); - const counts = tracks?.get(track) ?? { track: 0, subscribe: 0 }; - if (counts[kind] >= counts[kind === "track" ? "subscribe" : "track"]) this.#slots.acquire(); + const counts = tracks?.get(track) ?? { track: 0, subscribe: 0, tracks: 0 }; + const before = cost(counts); counts[kind] += 1; + if (cost(counts) > before) { + try { + this.#slots.acquire(); + } catch (err) { + counts[kind] -= 1; + throw err; + } + } if (!tracks) { tracks = new Map(); this.#tracks.set(broadcast, tracks); @@ -106,9 +125,10 @@ export class Interests { const tracks = this.#tracks.get(broadcast); const counts = tracks?.get(track); if (!tracks || !counts) throw new Error("interest already released"); - if (counts[kind] > counts[kind === "track" ? "subscribe" : "track"]) this.#slots.release(); + const before = cost(counts); counts[kind] -= 1; - if (counts.track === 0 && counts.subscribe === 0) { + if (cost(counts) < before) this.#slots.release(); + if (cost(counts) === 0) { tracks.delete(track); if (tracks.size === 0) this.#tracks.delete(broadcast); } diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts index 1ae5b386c53..709302ca558 100644 --- a/js/net/src/wire.ts +++ b/js/net/src/wire.ts @@ -13,6 +13,7 @@ import type { Drain } from "./connection/goaway.ts"; import type * as Epoch from "./epoch.ts"; import type { Consumer as GroupConsumer } from "./group.ts"; import type { Route } from "./hop.ts"; +import type { Lister, Tracks } from "./listing.ts"; import type * as origin from "./origin.ts"; import type * as Path from "./path.ts"; import type * as track from "./track.ts"; @@ -31,6 +32,13 @@ export interface Broadcast { fetchGroup(name: string, sequence: number, options?: track.FetchGroupOptions): Promise; // pending: /quest/m1/public-model/answer.md requested(): Promise; + // private: capability gap + // Sessions dedup a peer's listings per broadcast; a custom session has no public owner yet. + /** List the broadcast's tracks for `owner`, ending its older listing of this broadcast with CANCELLED. */ + tracks(owner?: object): Tracks; + // pending: /quest/m1/public-model/answer.md + /** Answer the broadcast's listings, as a session does with its peer's; throws while another answers. */ + lister(): Lister; // pending: /quest/m1/public-model/answer.md /** * A session delivered this broadcast. An originated route refuses to serve it: diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index f8ddd743bc0..2498ef03596 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -28,8 +28,8 @@ archive (per-track `.timeline.z` records named by the catalog's root - **Discovery is pull**, decided 2026-10-19 in place of SUBSCRIBE_ALL's push: ANNOUNCE finds the broadcasts under a prefix, a moq-lite TRACKS request lists each broadcast's tracks as they appear, and the archive SUBSCRIBEs per - track, gating the lite-07 cut (`broadcast::Consumer::tracks` in Rust; - [TRACKS in js/net](/quest/m1/archive/tracks-js.md)). + track, gating the lite-07 cut (`broadcast::Consumer::tracks` in Rust, + `broadcast.Consumer.tracks()` in JS). moq-transport answers SUBSCRIBE_TRACKS through the same hook (Rust now, [js/net](/quest/m1/archive/subscribe-tracks-js.md) next) and learns an IETF peer's tracks with a FORWARD=0 SUBSCRIBE_TRACKS @@ -85,7 +85,6 @@ from a host. Each child fixes the stale docs it touches. ## Required -- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - `@moq/net` lists a broadcast's tracks over lite-07's Tracks Stream and answers listings of its own broadcasts, mirroring Rust - [Archive consumer](/quest/m1/archive/consumer.md) - `archive::Consumer` serves a stored prefix as a broadcast from its key listing, and `moq export/import archive` use the new pair - [Catalog HLS field](/quest/m1/archive/catalog-hls.md) - the root `archive` section is gone; an opt-in per-rendition `hls` field marks each aligned HLS segment's first group and pts and declares its nominal duration - [Outbound SUBSCRIBE_TRACKS](/quest/m1/archive/subscribe-tracks-outbound.md) - an IETF peer's tracks are listed through the TRACKS hook by a FORWARD=0 SUBSCRIBE_TRACKS, whose held PUBLISH resumes for data diff --git a/quest/m1/archive/subscribe-tracks-js.md b/quest/m1/archive/subscribe-tracks-js.md index 32fd6e0b86a..afec9102ea6 100644 --- a/quest/m1/archive/subscribe-tracks-js.md +++ b/quest/m1/archive/subscribe-tracks-js.md @@ -13,10 +13,6 @@ a PUBLISHed track. ## Plan Mirror `rs/moq-net/src/ietf/publisher/tracks.rs`, reading tracks through the -TRACKS listing that [TRACKS in js/net](/quest/m1/archive/tracks-js.md) adds. +TRACKS listing (`broadcast.Consumer.tracks()`). Mirror its tests, and drop the JavaScript caveat from `doc/concept/standard.md`. - -## Required - -- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - `@moq/net` lists a broadcast's tracks, which this listing reads diff --git a/quest/m1/archive/tracks-js.md b/quest/m1/archive/tracks-js.md deleted file mode 100644 index aaf1a14b5e6..00000000000 --- a/quest/m1/archive/tracks-js.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] TRACKS in js/net - -## Goal - -`@moq/net` speaks the moq-lite-07 Tracks Stream (0x8) that Rust already -speaks, with the same behavior and mirrored names: - -- **Listing.** A broadcast consumer lists its tracks (Rust - `broadcast::Consumer::tracks`): each listed track with its info, one - `synced` event, then adds and removals until the broadcast ends. Over a - session it opens one TRACKS per broadcast however many readers list it, - syncs its readers at once, and forwards the peer's tracks as live adds. -- **Answering.** A session answers a peer's TRACKS for a broadcast it - publishes with the tracks its producer created (never one served only on - request), using the per-reader queue and SYNCED rule the draft specifies. -- **Refusals.** Older versions reset the stream type, as now. A covering - prefix route answers NOT_FOUND. A listing past the session's announce cap - is reset. - -## Plan - -The wire and the Rust model landed first; read the Tracks section of -`drafts/draft-lcurley-moq-lite.md` and mirror `rs/moq-net/src/model/listing.rs` -(the queue keyed by name in insertion order, where an unread add and removal -cancel and a removal then re-add of a seen name collapses in place) and -`rs/moq-net/src/lite/tracks.rs` (the encoding: TRACKS_ADD 0x0 with an -embedded TRACK_INFO, TRACKS_REMOVE 0x1, SYNCED 0x4). - -`js/net/src/lite/malformed.test.ts` still treats type 8 as unknown on -lite-07; it becomes a real stream here. - -Test the same cases as Rust: snapshot then SYNCED, SYNCED after N events -under churn, a stalled reader's bounded queue, removal before SYNCED, a -dynamic-only track never listed, the broadcast ending the listing, and -`just test interop --all` against the Rust side in both directions. - -Public API: a track-listing method on the JS broadcast consumer, named like -Rust's. Wire: none new. diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 154eb783df0..3ec28c55922 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -100,7 +100,6 @@ Timestamp provenance landed in - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - SUBSCRIBE carries ranges and an order and lite FETCH is gone, in Rust and JS - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route - [lite-06 max sequence](/quest/m1/lite06-max-sequence.md) - the refusal boundary for an uncarriable sequence is settled before lite-07 freezes -- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - JS speaks lite-07's Tracks Stream, which Rust already does ## Related diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index cde26ff2d7e..2f5bc62050c 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -1882,11 +1882,31 @@ struct TracksRecv { /// Each event fans out to every reader, so a peer streaming add/remove churn must not /// hold the thread for as long as its bytes keep arriving. budget: kio::coop::Budget, + /// A listing is a subscribe: it waits for a token that may cover the broadcast, and + /// ends once the session stops allowing it. + gate: crate::auth::Gate, + /// The gate let the listing through, and this hop synced its readers. + admitted: bool, + /// Until the request is out, the source must still be the selected instance: a TRACKS + /// request without an epoch would list its replacement. + validity: crate::model::broadcast::RequestValidity, } impl TracksRecv { /// Apply the peer's events until it ends the listing, or the error that ends it. fn poll(&mut self, source: &SourceServe, waiter: &kio::Waiter) -> Poll> { + if self.stream.is_none() { + self.validity.check(waiter)?; + } + ready!(self.gate.poll_ready(waiter)); + if self.gate.poll_denied(waiter).is_ready() { + return Poll::Ready(Err(Error::Unauthorized)); + } + if !std::mem::replace(&mut self.admitted, true) { + // Synced with this hop, which holds nothing yet: a relay never waits on its + // upstream, so the peer's tracks follow as live adds. + source.lister.synced(); + } let mut cx = waiter.context(); let version = source.subscriber.version; let stream = match &mut self.stream { @@ -1971,15 +1991,19 @@ impl SourceServe { self.lister.abort(Error::Unsupported); continue; } - // Synced with this hop, which holds nothing yet: a relay never waits on its - // upstream, so the peer's tracks follow as live adds. - self.lister.synced(); listing = Some(TracksRecv { session: self.subscriber.session.clone(), stream: None, listed: HashMap::new(), synced: false, budget: kio::coop::Budget::new(32), + gate: crate::auth::Gate::new( + self.subscriber.auth.clone(), + self.path.clone(), + crate::auth::Direction::Subscribe, + ), + admitted: false, + validity: self.validity.start(), }); } Some(_) if self.lister.poll_unused(waiter).is_ready() => { @@ -3751,6 +3775,130 @@ mod tests { } } + /// A lite-07 source of `room` over `session`, a consumer to list it through, and the + /// route that keeps it served. + fn listed_source( + session: SinkSession, + auth: crate::auth::Handle, + validity: crate::model::broadcast::RequestValidity, + ) -> (SourceServe, crate::broadcast::Consumer, kio::Producer<()>) { + let subscriber = Subscriber::new(SubscriberConfig { + runtime: crate::time::sim(), + session, + origin: origin::Config::new(crate::Hop::new(1).unwrap()).produce(), + recv_bandwidth: None, + version: Version::Lite07, + peer_setup: crate::lite::PeerSetup::new(false, crate::transport::Binding::InBand), + peer_hop: None, + cost: None, + going_away: Default::default(), + auth, + }); + let broadcast = crate::broadcast::Info::new().produce(); + let consumer = broadcast.consume(); + let dynamic = broadcast.dynamic(); + let route = kio::Producer::new(()); + let serve = SourceServe::new( + subscriber, + MintedSource { + validity, + path: Path::new("room").to_owned(), + epoch: None, + source: crate::model::broadcast::SourceGuard::new(broadcast), + dynamic, + route: route.consume(), + }, + ); + (serve, consumer, route) + } + + /// A listing is a subscribe: it waits for a token that may cover the broadcast before + /// syncing its readers or asking the peer, which would refuse it until then. + #[moq_clock::test] + async fn a_listing_waits_for_its_token() { + let auth = crate::auth::Handle::new(true); + let token = auth.present(bytes::Bytes::from_static(b"new"), false).unwrap(); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, Default::default()); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 0, "asked the peer before the token settled"); + assert!( + tracks.next().now_or_never().is_none(), + "synced before the token settled" + ); + + drop(token); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 1); + assert!(matches!( + tracks.next().now_or_never(), + Some(Ok(Some(crate::broadcast::TracksEvent::Synced))) + )); + } + + /// An instance retired while its listing waits for a token lists nothing: a TRACKS + /// request without an epoch would list its replacement. + #[moq_clock::test] + async fn a_listing_of_a_retired_instance_never_reaches_the_wire() { + let auth = crate::auth::Handle::new(true); + let token = auth.present(bytes::Bytes::from_static(b"new"), false).unwrap(); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let selected = kio::Producer::new(()); + let validity = crate::model::broadcast::RequestValidity::Route(selected.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, validity); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + drop(selected); + drop(token); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unroutable)))); + assert_eq!(session.log.bi_opens(), 0); + } + + /// A listing outside the session's scope is refused before it reaches the wire. + #[moq_clock::test] + async fn a_listing_outside_the_scope_is_refused() { + let auth = crate::auth::Handle::new(false); + auth.authorize(&crate::auth::Grant::default()); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, Default::default()); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unauthorized)))); + assert_eq!(session.log.bi_opens(), 0); + } + + /// A narrower scope resets a live listing with UNAUTHORIZED rather than keep listing + /// what the session no longer allows, and leaves the session up. + #[moq_clock::test] + async fn a_narrower_limit_resets_a_listing() { + let auth = crate::auth::Handle::new(false); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth.clone(), Default::default()); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 1); + assert!(matches!( + tracks.next().now_or_never(), + Some(Ok(Some(crate::broadcast::TracksEvent::Synced))) + )); + + auth.authorize(&crate::auth::Grant::default()); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unauthorized)))); + assert_eq!(session.log.resets(), vec![crate::StreamError::Unauthorized.to_code()]); + assert!(session.log.closes().is_empty()); + } + /// A track waiting on an unanswered token ends once nobody wants it, rather than /// holding its task and track until a token that may never be answered. #[moq_clock::test] diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index 5bf9412188f..e84d2c3ac49 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -373,6 +373,67 @@ fn lite_auth_unsupported_interop() { assert!(reader.is_empty()); } +/// moq-lite-07 TRACKS across the Rust/JS boundary: the JS publisher answers a Rust TRACKS request +/// with a listing Rust decodes, and JS decodes Rust's listing events and re-encodes them byte for +/// byte. +#[test] +#[ignore = "requires Bun; run by just test interop"] +fn lite_tracks_interop() { + use crate::coding::Decode; + + let version = lite::Version::Lite07; + let request = lite::Tracks { + broadcast: crate::Path::new("room"), + epoch: None, + } + .encode_bytes(version) + .unwrap(); + let input = serde_json::json!({ "mode": "answer", "bytes": request.to_vec() }); + let answer: Vec = serde_json::from_slice(&bun("tracks.ts", input)).expect("JS returned its listing"); + let mut r = Decoder::new(&answer, version.into()); + let mut events = Vec::new(); + while !r.is_empty() { + events.push(match lite::TracksEvent::decode(&mut r, version).unwrap() { + lite::TracksEvent::Added { name, info } => { + if name == "a" { + assert_eq!(info.timescale, Some(crate::Timescale::MILLI)); + assert_eq!(info.max_age, Some(std::time::Duration::from_secs(3))); + assert_eq!(info.priority, 9); + assert_eq!(info.refresh, Some(4)); + } + format!("+{name}") + } + lite::TracksEvent::Removed { name } => format!("-{name}"), + lite::TracksEvent::Synced => "synced".to_string(), + }); + } + // The broadcast's end is the JS publisher's FIN, after the last event. + assert_eq!(events, ["+a", "+b", "synced", "-a"]); + + let info = lite::TrackInfo { + refresh: Some(4), + priority: 3, + max_age: Some(std::time::Duration::from_secs(2)), + timescale: Some(crate::Timescale::MICRO), + compress: true, + }; + let mut bytes = Vec::new(); + let mut w = Encoder::new(&mut bytes, version.into()); + for event in [ + lite::TracksEvent::Added { + name: "video".into(), + info, + }, + lite::TracksEvent::Removed { name: "video".into() }, + lite::TracksEvent::Synced, + ] { + event.encode(&mut w, version).unwrap(); + } + let input = serde_json::json!({ "mode": "echo", "bytes": bytes }); + let echo: Vec = serde_json::from_slice(&bun("tracks.ts", input)).expect("JS echoed the events"); + assert_eq!(echo, bytes); +} + /// TRACK_STATUS on every draft: the JS publisher decodes what Rust encodes and answers with a /// TRACK_STATUS_OK the Rust subscriber reads, carrying what a SUBSCRIBE_OK would as far as the /// draft has room, or refuses a missing broadcast DOES_NOT_EXIST. diff --git a/test/interop/README.md b/test/interop/README.md index f53383f9c0b..2405da75e56 100644 --- a/test/interop/README.md +++ b/test/interop/README.md @@ -451,6 +451,12 @@ match byte for byte and decode back to the same value. NOT\_SUPPORTED across Rust and JS in both directions, checking that the later refusal retains its meaning and both bindings emit the same bytes. +`lite_tracks_interop` checks moq-lite-07's Tracks Stream both ways: `tracks.ts` +answers a Rust TRACKS request with js/net's publisher, whose listing (each +TRACKS\_ADD with its TRACK\_INFO, SYNCED, a TRACKS\_REMOVE, then the FIN when +the broadcast ends) Rust decodes, and it re-encodes Rust's listing events byte for +byte. + `lite_varint_interop` runs next to it and does the same through moq-lite's version dispatch: `lite-varint.ts` decodes Rust's lite-06 (QUIC) and lite-07 (leading-ones) varints, a SETUP carrying a 62-bit Hop ID, a datagram, and a diff --git a/test/interop/tracks.ts b/test/interop/tracks.ts new file mode 100644 index 00000000000..bc32f3d4170 --- /dev/null +++ b/test/interop/tracks.ts @@ -0,0 +1,63 @@ +// moq-lite-07 TRACKS both ways. `answer`: the JS publisher answers a Rust-encoded TRACKS request +// with its listing, through a removal and the broadcast's end. `echo`: JS decodes Rust-encoded +// listing events and encodes them back. +import { Milli } from "@moq/clock"; +import { randomHop } from "../../js/net/src/hop.ts"; +import { Publisher } from "../../js/net/src/lite/publisher.ts"; +import { decodeTracksEventMaybe, encodeTracksEvent, Tracks } from "../../js/net/src/lite/tracks.ts"; +import { Version } from "../../js/net/src/lite/version.ts"; +import { createMockTransportPair } from "../../js/net/src/mock.ts"; +import { Producer as OriginProducer } from "../../js/net/src/origin.ts"; +import * as Path from "../../js/net/src/path.ts"; +import { Reader, Stream, Writer } from "../../js/net/src/stream.ts"; +import { Timescale } from "../../js/net/src/time.ts"; + +// Stdout is the byte channel back to Rust. +console.debug = console.error; +console.info = console.error; +const input: { mode: "answer" | "echo"; bytes: number[] } = JSON.parse(process.argv[2]); +const version = Version.DRAFT_07; +const bytes = Uint8Array.from(input.bytes); +const written: number[] = []; +const settle = () => new Promise((resolve) => setTimeout(resolve, 0)); + +if (input.mode === "answer") { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { timescale: Timescale.MILLI, maxAge: Milli(3000), priority: 9, refresh: 4 }); + broadcast.createTrack("b"); + + const publisher = new Publisher(createMockTransportPair().server, version, randomHop(), origin.consume()); + const stream = new Stream({ + // The requester holds its side open while it lists: a FIN would cancel the listing. + readable: new ReadableStream({ start: (controller) => controller.enqueue(bytes) }), + writable: new WritableStream({ write: (chunk) => void written.push(...chunk) }), + version, + }); + const msg = await Tracks.decode(stream.reader, version); + if (msg.broadcast !== Path.from("room")) throw new Error(`unexpected broadcast: ${msg.broadcast}`); + const listing = publisher.runTracks(msg, stream); + // Past SYNCED before the removal, so the removal is not folded into the initial set. + await settle(); + broadcast.removeTrack("a"); + await settle(); + broadcast.close(); + await listing; + origin.close(); +} else { + const reader = new Reader(undefined, bytes, version); + const writer = new Writer( + new WritableStream({ write: (chunk) => void written.push(...chunk) }), + version, + ); + for (;;) { + const event = await decodeTracksEventMaybe(reader, version); + if (!event) break; + await encodeTracksEvent(writer, event, version); + } + writer.close(); + await writer.closed; +} + +process.stdout.write(JSON.stringify(written)); diff --git a/test/justfile b/test/justfile index ddfcf35e534..b30261a9cc1 100644 --- a/test/justfile +++ b/test/justfile @@ -42,11 +42,12 @@ harness: # Both runs first check js/net's varints against moq-net's encoder at every size boundary, # and on lite-06/07 messages (the filter also matches `lite_varint_interop`), then # moq-transport OBJECT_DATAGRAMs both ways and the JS answer to TRACK_STATUS on every -# draft, Lite AUTH_ERROR NOT_SUPPORTED after a grant, and Lite padding in both directions. +# draft, Lite AUTH_ERROR NOT_SUPPORTED after a grant, Lite padding, and lite-07 TRACKS +# listings, each in both directions. [positional-arguments] interop *args: cd .. && bun install --frozen-lockfile - just rs test -p moq-net --lib varint_interop ietf_datagram_interop ietf_track_status_interop ietf_compression_fallback_interop auth_unsupported_interop padding_interop --run-ignored all + just rs test -p moq-net --lib varint_interop ietf_datagram_interop ietf_track_status_interop ietf_compression_fallback_interop auth_unsupported_interop padding_interop tracks_interop --run-ignored all ./interop/interop.sh "$@" # The harnesses append one record per case to MOQ_TEST_RESULTS; flags link the page to its