diff --git a/doc/concept/standard.md b/doc/concept/standard.md index 94bfa150a88..a2431d426a2 100644 --- a/doc/concept/standard.md +++ b/doc/concept/standard.md @@ -26,7 +26,7 @@ maps everything else to "not supported" or a harmless equivalent. The subset drops. What a peer actually observes against this implementation: - **Pull.** Subscribers ask. Single-track `PUBLISH` offers are declined, unless they answer our `SUBSCRIBE_TRACKS` (below); announce a namespace and serve the resulting subscriptions. Announcements go out unsolicited. A client asks only for the prefixes its readers watch, and a server or relay link for every prefix it may discover. Rust never holds two overlapping requests at a peer that did not declare solicit: a broader one replaces the narrower ones once they are closed. The solicit `SETUP` option makes us wait to be asked. Without it, a draft 16+ peer also gets each match as a `NAMESPACE` on its `SUBSCRIBE_NAMESPACE` stream, so it hears the namespace twice. We send `PUBLISH` only to answer a `SUBSCRIBE_TRACKS` (below): on drafts 16 and 17 a `SUBSCRIBE_NAMESPACE` asking only for `PUBLISH` is refused, one asking for both gets only `NAMESPACE`, and any other Subscribe Options value closes the session. -- **`SUBSCRIBE_TRACKS` lists, it never pushes.** On draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Rust opens each paused `PUBLISH` only as the peer's stream credit admits it, so a peer that grants none holds none; JavaScript caps them, and past the cap a track gets `PUBLISH_SKIPPED` (`PUBLISH_BLOCKED` on draft 18) instead. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. Rust sends it upstream for a listing (below); JavaScript does not. +- **`SUBSCRIBE_TRACKS` lists, it never pushes.** On draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Each paused `PUBLISH` opens only as the peer's stream credit admits it, so a peer that grants none holds none. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. Rust sends it upstream for a listing (below); JavaScript does not. - **A Rust TRACKS listing sends `SUBSCRIBE_TRACKS` upstream, only on demand.** Listing a broadcast from a moq-transport peer on draft 18 and later sends `SUBSCRIBE_TRACKS` with `FORWARD=0` while someone lists it, never on connect, announce, or subscribe, so a peer that refuses, resets, or ignores it affects only that listing. Before draft 18 the listing is refused as unsupported. Each listed track's paused `PUBLISH` is its upstream copy, which a reader resumes rather than subscribing again. Only a `PUBLISH_DONE` with `TRACK_ENDED` removes a listed track, and it lists as finished; any other end, including the error our own answerer sends for a track that left unfinished, fails the listing, and readers list again. - **History is one group.** In Rust, Object ID gaps must be marked by Prior Object ID Gap properties for a group to be fetchable. A `FETCH` returns one group from the cache, or on drafts 14 through 19 the saved prefix of the group a new subscription just joined. A range of groups is refused, as is a draft 20+ `LOCATION_FILTER` that ends at Largest Object rather than an absolute End Location. On drafts 14 through 19, `FETCH_OK` End Object 0 covers the entire End Group, matching the request; Rust also accepts the legacy answer naming the next group with Object 0. On draft 20 and later, `FETCH_OK`'s End Location is the requested end, capped at Largest Object: a range holding no objects gets an empty fetch stream, and only a start past Largest Object is refused, with `INVALID_RANGE`. A relay's copy with no upstream subscription can't know Largest Object short of the track's end, so it echoes the requested end instead. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed, and so does a reader whose publisher refuses `TRACK_STATUS` as `NOT_SUPPORTED`. - **Compression.** Rust peers on draft 17 and later exchange compressed tracks when both offer the [Flate extension](/draft/moq-flate). Older peers receive one shared inflated copy. A publisher that seals the compression declaration makes that fallback unavailable: an incompatible peer is refused, preserving the sealed metadata. JavaScript does not offer the IETF extension. diff --git a/js/net/src/ietf/publisher.ts b/js/net/src/ietf/publisher.ts index a9bc2846ec3..31186b82a53 100644 --- a/js/net/src/ietf/publisher.ts +++ b/js/net/src/ietf/publisher.ts @@ -34,7 +34,6 @@ import { untilAborted } from "../util/abort.ts"; import { nextChange } from "../util/change.ts"; import { Deadline, REQUEST_TIMEOUT_MS, requestOptions } from "../util/deadline.ts"; import { untilGivenUp } from "../util/given_up.ts"; -import { LIMITS, type Limits, Slots } from "../util/limits.ts"; import { TimeoutError, withTimeout } from "../util/timeout.ts"; import * as Varint from "../varint.ts"; import { type Advertised, type Advertisements, sameInstance, wireOf } from "../wire.ts"; @@ -273,6 +272,21 @@ interface Published { /** How a listed track left the listing: it finished, or it left unfinished. */ type Unlisted = "finished" | "removed"; +/** The next listed track to PUBLISH, ready but for its stream. */ +interface Staged { + name: string; + info: Info; + /** On draft-18, claims the track's one subscription from now on. */ + claim?: string; +} + +/** What a broadcast's listing woke for. */ +type Woken = + | { kind: "track"; event: broadcast.TracksEvent | undefined } + | { kind: "opened"; stream: Stream } + | { kind: "failed"; err: unknown } + | { kind: "changed" }; + /** Stop forwarding without ending the subscription. */ class Paused extends Error { /** The update to answer once forwarding stopped, if one asked; its own Request ID on drafts 15 and 16. */ @@ -332,8 +346,6 @@ export class Publisher { #quic: WebTransport; #session: Session; #requiresSolicitation: boolean; - // Subscriptions the peer holds, PUBLISHes for its listings capped by `Limits.requests`. - #subscriptions: Slots; #listings = new Set(); #trackClaims = new Map(); @@ -396,7 +408,6 @@ export class Publisher { grant = new Signal(undefined), auth, ready = Promise.resolve(), - limits = LIMITS, }: { /** The WebTransport session, for uni streams. */ quic: WebTransport; @@ -420,11 +431,8 @@ export class Publisher { auth?: Carrier; /** Resolves once the setup tokens are answered (MoQ Auth). */ ready?: Promise; - /** How many live requests the peer may hold. */ - limits?: Limits; }) { this.#quic = quic; - this.#subscriptions = new Slots(limits.requests, { reason: "too many requests" }); this.#grant = grant; this.#gate = new Gate(grant); this.#auth = auth; @@ -480,15 +488,10 @@ export class Publisher { ); return; } - // Only a subscription we would serve holds a slot, as in Rust: a refused one ends at - // once. Stream credit bounds them, so none is refused here. - const charged = !msg.rangeFilters && this.#gate.allows(msg.trackNamespace); - if (charged) this.#subscriptions.hold(); if (claimed) this.#trackClaims.set(key, ((this.#trackClaims.get(key) as number | undefined) ?? 0) + 1); try { await this.#serveSubscribe(msg, request); } finally { - if (charged) this.#subscriptions.release(); if (claimed) { const count = (this.#trackClaims.get(key) as number) - 1; if (count) this.#trackClaims.set(key, count); @@ -2370,6 +2373,8 @@ export class Publisher { continue; } skips.delete(first[0]); + // A FORWARD=1 listing waits for this to read its next track. + changed.update((n) => n + 1); await Message.write(request.writer, PublishSkipped.id, first[1], version); } if (refusal) await request.refuse("subscribe_namespace", refusal); @@ -2436,6 +2441,16 @@ export class Publisher { } } + /** + * List one broadcast under a SUBSCRIBE_TRACKS: PUBLISH or skip each of its tracks until its + * route goes or the broadcast ends, then let its PUBLISHes run to their end. + * + * A PUBLISH starts only once the peer's stream credit opens its stream, one staged at a time, + * so the PUBLISHes a peer holds are bounded by the credit it grants. The listing is still read + * meanwhile, since an unlisted track's PUBLISH_DONE is what returns that credit; the tracks + * read but not yet PUBLISHed wait in `pending`. A FORWARD=1 listing has nothing to unlist, so + * it reads its next track only once the request stream has taken the last skip. + */ async #listBroadcast({ msg, suffix, @@ -2453,12 +2468,38 @@ export class Publisher { skips: Map; changed: Signal; }) { + const forward = msg.parameters.forward !== false; const namespace = Path.join(msg.prefix, suffix); + // The tracks waiting their turn for a PUBLISH, in listing order. + const pending = new Map(); + // The next track to PUBLISH, waiting on the peer's stream credit. + let staged: Staged | undefined; + // The stream the next PUBLISH goes out on, once the peer's credit admits it. + let opening: Promise | undefined; + // The listing's next event, read while anything else is awaited. + let reading: Promise | undefined; + // Each sent PUBLISH's entry: setting it says the track is no longer listed. const listed = new Map>(); + // The tracks with a skip queued, so the broadcast's end cancels only its own. const skipped = new Map(); const tasks = new Set>(); + const skip = (name: string) => { + const key = Symbol(); + skipped.set(name, key); + skips.set(key, new PublishSkipped(suffix, name)); + changed.update((n) => n + 1); + }; + const unstage = () => { + if (staged?.claim) this.#trackClaims.delete(staged.claim); + staged = undefined; + }; + // The PUBLISHes already sent still end when their track is unlisted, so the listing is + // still read once cancelled, adding nothing. const cancel = adding.subscribe((active) => { - if (!active && !tasks.size) hold.abort(); + if (active) return; + pending.clear(); + unstage(); + if (!tasks.size) hold.abort(); }); let listing: broadcast.Tracks | undefined; try { @@ -2466,10 +2507,84 @@ export class Publisher { const source = await this.#resolve(namespace, hold.signal, epoch); listing = source.tracks(); for (;;) { - const event = await untilAborted(listing.next(), hold.signal); + // Stage the next pending track that will be PUBLISHed, skipping those that will + // not, so a stream is opened only for a PUBLISH that goes out. + for (const [name, info] of pending) { + if (staged || !adding.peek()) break; + pending.delete(name); + // Draft-18 allows one subscription per track (section 5.1), so a track the peer + // is subscribed to, by its SUBSCRIBE or a PUBLISH still alive, is skipped. + // Claimed from now, so a SUBSCRIBE cannot slip in before the PUBLISH goes out. + const claim = + this.#session.version === Version.DRAFT_18 ? JSON.stringify([namespace, name]) : undefined; + if (claim !== undefined) { + if (this.#trackClaims.has(claim)) { + skip(name); + continue; + } + this.#trackClaims.set(claim, "publish"); + } + staged = { name, info, claim }; + } + if (staged && !opening) + opening = Promise.resolve(this.#session.openBi({ timeout: null })).then( + (stream): Woken => ({ kind: "opened", stream }), + (err: unknown): Woken => ({ kind: "failed", err }), + ); + const waits: PromiseLike[] = []; + if (opening) waits.push(opening); + // A FORWARD=1 listing has nothing to unlist, so it reads its next track only once + // the request stream has taken the last skip. + if (forward && skips.size) waits.push(Signal.race(changed).then((): Woken => ({ kind: "changed" }))); + else { + reading ??= listing.next().then((event): Woken => ({ kind: "track", event })); + waits.push(reading); + } + const woken = await untilAborted(race(waits), hold.signal); + if (woken.kind === "changed") continue; + if (woken.kind === "failed") { + console.debug(`listed broadcast=${namespace}: PUBLISH stream did not open`, woken.err); + break; + } + if (woken.kind === "opened") { + opening = undefined; + const next = staged; + if (!next || !adding.peek()) { + woken.stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + continue; + } + staged = undefined; + const unlisted = new Once(); + listed.set(next.name, unlisted); + const task = this.#runPublish({ + msg, + namespace, + source, + stream: woken.stream, + name: next.name, + info: next.info, + unlisted, + adding, + }) + .catch((err) => { + if (err instanceof ProtocolViolation) throw err; + }) + .finally(() => { + if (next.claim) this.#trackClaims.delete(next.claim); + tasks.delete(task); + if (!adding.peek() && !tasks.size) hold.abort(); + }); + tasks.add(task); + void task.catch(() => hold.abort()); + continue; + } + reading = undefined; + const event = woken.event; if (!event) break; if (event.kind === "synced") continue; if (event.kind === "removed" || event.kind === "finished") { + if (staged?.name === event.name) unstage(); + pending.delete(event.name); listed.get(event.name)?.set(event.kind); listed.delete(event.name); const key = skipped.get(event.name); @@ -2478,44 +2593,19 @@ export class Publisher { continue; } if (!adding.peek()) continue; - const duplicate = - this.#session.version === Version.DRAFT_18 && - this.#trackClaims.has(JSON.stringify([namespace, event.name])); - if (msg.parameters.forward !== false || duplicate || !this.#subscriptions.tryAcquire()) { - const key = Symbol(); - skipped.set(event.name, key); - skips.set(key, new PublishSkipped(suffix, event.name)); - changed.update((n) => n + 1); - continue; - } - const unlisted = new Once(); - listed.set(event.name, unlisted); - const key = JSON.stringify([namespace, event.name]); - if (this.#session.version === Version.DRAFT_18) this.#trackClaims.set(key, "publish"); - const task = this.#runPublish({ - msg, - namespace, - source, - name: event.name, - info: event.info, - unlisted, - adding, - }) - .catch((err) => { - if (err instanceof ProtocolViolation) throw err; - }) - .finally(() => { - this.#subscriptions.release(); - if (this.#session.version === Version.DRAFT_18) this.#trackClaims.delete(key); - tasks.delete(task); - if (!adding.peek() && !tasks.size) hold.abort(); - }); - tasks.add(task); - void task.catch(() => hold.abort()); + if (forward) skip(event.name); + else pending.set(event.name, event.info); } } finally { cancel(); listing?.close(); + pending.clear(); + unstage(); + // A stream that opens from now on carries nothing. + void opening?.then((woken) => { + if (woken.kind === "opened") + woken.stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + }); for (const entry of listed.values()) entry.set("removed"); for (const key of skipped.values()) skips.delete(key); await Promise.all(tasks); @@ -2526,6 +2616,7 @@ export class Publisher { msg, namespace, source, + stream, name, info, unlisted, @@ -2534,6 +2625,8 @@ export class Publisher { msg: SubscribeTracks; namespace: Path.Valid; source: broadcast.Consumer; + /** The PUBLISH's stream, opened once the peer's credit admitted it. */ + stream: Stream; name: string; info: Info; unlisted: Once; @@ -2551,21 +2644,10 @@ export class Publisher { const removed = unlisted.subscribe((how) => { if (how === "removed" || (how === "finished" && !publishing)) hold.abort(); }); - const cancelled = () => request?.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + const request = new RequestStream(stream, this.#session, this.#quic); + const cancelled = () => request.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); hold.signal.addEventListener("abort", cancelled); - let request: RequestStream | undefined; try { - if (!adding.peek() || unlisted.peek() !== undefined) return; - const opening = Promise.resolve(this.#session.openBi({ timeout: null })); - void opening.then( - (stream) => { - if (hold.signal.aborted) - stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); - }, - () => undefined, - ); - const stream = await untilAborted(opening, hold.signal); - request = new RequestStream(stream, this.#session, this.#quic); let peerEnded = false; const ended = request.ended.then(() => { peerEnded = true; @@ -2762,14 +2844,14 @@ export class Publisher { } } } catch (err) { - request?.abort(err); + request.abort(err); throw err; } finally { stop(); removed(); hold.signal.removeEventListener("abort", cancelled); hold.abort(); - request?.close(); + request.close(); } } diff --git a/js/net/src/ietf/subscribe_tracks.test.ts b/js/net/src/ietf/subscribe_tracks.test.ts index 011cf85942a..49a2967e053 100644 --- a/js/net/src/ietf/subscribe_tracks.test.ts +++ b/js/net/src/ietf/subscribe_tracks.test.ts @@ -1,7 +1,9 @@ import { expect, onTestFinished, spyOn, test } from "bun:test"; import { Manual, Nano } from "@moq/clock"; +import Qmux from "@moq/qmux"; import { Signal } from "@moq/signals"; import type { Grant } from "../auth.ts"; +import { Tracks } from "../broadcast.ts"; import { StreamCode, StreamError } from "../error.ts"; import { createMockTransportPair } from "../mock.ts"; import { Producer as Origin } from "../origin.ts"; @@ -20,8 +22,43 @@ import { ALPN, type IetfVersion, Version } from "./version.ts"; const drafts = [Version.DRAFT_18, Version.DRAFT_19, Version.DRAFT_20, Version.DRAFT_21, Version.DRAFT_22]; -function fixture(version: IetfVersion, requests = 10, grant = new Signal(undefined)) { - const pair = createMockTransportPair(ALPN.DRAFT_22); +/** A client and server WebTransport, connected to each other. */ +type Pair = { client: WebTransport; server: WebTransport }; + +/** + * A qmux session pair over an in-memory socket, which, unlike the mock transport, holds each + * bidi stream against the receiver's stream credit until both its halves end. The client + * grants the server `credit` such streams at once. + */ +function qmuxPair(credit: bigint): Pair { + const socket = (readable: ReadableStream, writable: WritableStream) => { + const closed = Promise.withResolvers<{ closeCode: number; reason: string }>(); + return { + url: "", + opened: Promise.resolve({ readable, writable, extensions: "", protocol: "qmux-01" }), + closed: closed.promise, + close: () => closed.resolve({ closeCode: 1000, reason: "" }), + }; + }; + // Copied, since a sender may reuse its buffer once a write resolves. + const pipe = () => + new TransformStream({ transform: (chunk, c) => c.enqueue(chunk.slice()) }); + const up = pipe(); + const down = pipe(); + const client = new Qmux(socket(down.readable, up.writable), { + config: { idleTimeout: null, maxStreamsBidi: credit }, + }); + const server = Qmux.accept(socket(up.readable, down.writable), { config: { idleTimeout: null } }); + return { client, server }; +} + +function fixture( + version: IetfVersion, + { + grant = new Signal(undefined), + pair = createMockTransportPair(ALPN.DRAFT_22), + }: { grant?: Signal; pair?: Pair } = {}, +) { const origin = new Origin(); const session = new NativeSession(pair.server, version, false); const publisher = new Publisher({ @@ -30,7 +67,6 @@ function fixture(version: IetfVersion, requests = 10, grant = new Signal[] = []; const streams: Stream[] = []; @@ -38,6 +74,7 @@ function fixture(version: IetfVersion, requests = 10, grant = new Signal { - const fx = fixture(version, 1); + test(`track listing rejects overlapping prefixes (${version})`, async () => { + const fx = fixture(version); const broadcast = fx.origin.createBroadcast(Path.from("room/live")); broadcast.announce(); broadcast.createTrack("video"); - broadcast.createTrack("audio"); const listing = await fx.list(); await ok(listing, version); await fx.offered(); - expect(await listing.reader.u53()).toBe(PublishSkipped.id); - const skipped = await PublishSkipped.decode(listing.reader, version); - expect(skipped.suffix).toBe(Path.from("live")); - expect(skipped.trackName).toBe("audio"); for (const prefix of ["room/live", "", "room"]) { const overlap = await fx.list(prefix); expect(await overlap.reader.u53()).toBe(RequestError.id); @@ -245,7 +285,9 @@ for (const version of drafts) { } else { await ok(listing, version); expect(await listing.reader.u53()).toBe(PublishSkipped.id); - expect((await PublishSkipped.decode(listing.reader, version)).trackName).toBe("video"); + const skipped = await PublishSkipped.decode(listing.reader, version); + expect(skipped.suffix).toBe(Path.from("live")); + expect(skipped.trackName).toBe("video"); } }); } @@ -309,7 +351,7 @@ test("grants hide tracks, revoke PUBLISHes and relist when broadened", async () const clock = new Manual(); const installed = clock.install(); using _installed = { [Symbol.dispose]: () => installed.close() }; - const fx = fixture(version, 10, grant); + const fx = fixture(version, { grant }); const listing = await fx.list(); await ok(listing, version); const broadcast = fx.origin.createBroadcast(Path.from("room/live")); @@ -381,47 +423,6 @@ test("an update ends only its listing and frees its prefix", async () => { await ok(replacement, version); }); -test("paused updates keep one slot and release it when the PUBLISH ends", async () => { - const version = Version.DRAFT_22; - const fx = fixture(version, 1); - const clock = new Manual(); - const installed = clock.install(); - using _installed = { [Symbol.dispose]: () => installed.close() }; - const broadcast = fx.origin.createBroadcast(Path.from("room/live")); - broadcast.announce(); - broadcast.createTrack("video"); - const listing = await fx.list(); - await ok(listing, version); - const offered = await fx.offered(); - for (let i = 0; i < 32; i++) await update(offered.stream, version, BigInt(2 * (i + 1)), false); - broadcast.createTrack("skipped"); - expect(await listing.reader.u53()).toBe(PublishSkipped.id); - expect((await PublishSkipped.decode(listing.reader, version)).trackName).toBe("skipped"); - offered.stream.abort(new StreamError(StreamCode.Cancel)); - await clock.advance(Nano.zero); - broadcast.createTrack("available"); - expect((await fx.offered()).publish.trackName).toBe("available"); -}); - -test("a failed PUBLISH stream open returns its subscription slot", async () => { - const version = Version.DRAFT_22; - const fx = fixture(version, 1); - const clock = new Manual(); - const installed = clock.install(); - using _installed = { [Symbol.dispose]: () => installed.close() }; - const open = spyOn(fx.session, "openBi").mockRejectedValue(new StreamError(StreamCode.Cancel)); - const broadcast = fx.origin.createBroadcast(Path.from("room/live")); - broadcast.announce(); - broadcast.createTrack("video"); - const listing = await fx.list(); - await ok(listing, version); - await clock.advance(Nano.zero); - expect(open).toHaveBeenCalledTimes(1); - open.mockRestore(); - broadcast.createTrack("available"); - expect((await fx.offered()).publish.trackName).toBe("available"); -}); - test("session shutdown releases listing tasks while the local origin stays open", async () => { const version = Version.DRAFT_22; const fx = fixture(version); @@ -483,3 +484,113 @@ for (const version of [Version.DRAFT_20, Version.DRAFT_22]) { uni.releaseLock(); }); } + +/** Listed broadcast sizes to sweep: what a listing holds must not grow with them. */ +const CATALOGS = [4, 64, 512]; + +/** A broadcast under `room` listing `catalog` tracks. */ +function catalogOf(fx: ReturnType, catalog: number) { + const broadcast = fx.origin.createBroadcast(Path.from("room/live")); + broadcast.announce(); + for (let i = 0; i < catalog; i++) broadcast.createTrack(`t${i}`); +} + +// Each PUBLISH opens only as the peer's stream credit admits it: a peer that withholds credit and +// reads nothing leaves one stream open waiting, however many tracks are listed, and each track is +// PUBLISHed once credit returns. +for (const version of drafts) { + test(`a listing publishes only as credit admits (${version})`, async () => { + const credit = 2; + const clock = new Manual(); + const installed = clock.install(); + using _installed = { [Symbol.dispose]: () => installed.close() }; + for (const catalog of CATALOGS) { + const pair = qmuxPair(BigInt(credit)); + const open = pair.server.createBidirectionalStream.bind(pair.server); + let waiting = 0; + spyOn(pair.server, "createBidirectionalStream").mockImplementation((options) => { + waiting += 1; + const opened = open(options); + const done = () => { + waiting -= 1; + }; + opened.then(done, done); + return opened; + }); + const fx = fixture(version, { pair }); + catalogOf(fx, catalog); + const listing = await fx.list(); + await ok(listing, version); + const held = []; + for (let i = 0; i < credit; i++) held.push(await fx.offered(false)); + await clock.advance(Nano.zero); + expect(waiting, `catalog=${catalog}`).toBeLessThanOrEqual(1); + + // Refusing each PUBLISH returns its credit, which opens the next. + const published = new Set(held.map(({ publish }) => publish.trackName)); + for (const { stream } of held) stream.abort(new StreamError(StreamCode.Cancel)); + while (published.size < catalog) { + const { stream, publish } = await fx.offered(false); + expect(published.has(publish.trackName), "PUBLISHed a track twice").toBe(false); + published.add(publish.trackName); + stream.abort(new StreamError(StreamCode.Cancel)); + expect(waiting, `catalog=${catalog}`).toBeLessThanOrEqual(1); + } + } + }); +} + +// A FORWARD=1 listing reads its next track only once the request stream has taken the last +// skip, so a peer that stops reading leaves one skip queued, however many tracks are listed. +for (const version of [Version.DRAFT_19, Version.DRAFT_22]) { + test(`a forwarding listing reads only as skips go out (${version})`, async () => { + const clock = new Manual(); + const installed = clock.install(); + using _installed = { [Symbol.dispose]: () => installed.close() }; + // Hold each skip's write until the test lets it out, as a peer that stops reading would. + const gates: (() => void)[] = []; + let arrived = Promise.withResolvers(); + const write = Message.write; + const writes = spyOn(Message, "write").mockImplementation(async (writer, id, msg, version) => { + if (id === PublishSkipped.id) + await new Promise((resolve) => { + gates.push(resolve); + arrived.resolve(); + }); + return write(writer, id, msg, version); + }); + const gate = async () => { + while (!gates.length) { + await arrived.promise; + arrived = Promise.withResolvers(); + } + return gates.shift() as () => void; + }; + const reads = spyOn(Tracks.prototype, "next"); + onTestFinished(() => { + writes.mockRestore(); + reads.mockRestore(); + }); + for (const catalog of CATALOGS) { + reads.mockClear(); + const fx = fixture(version); + catalogOf(fx, catalog); + const listing = await fx.list("room", true); + await ok(listing, version); + const skipped = new Set(); + while (skipped.size < catalog) { + const release = await gate(); + await clock.advance(Nano.zero); + // The skip being written, one queued, and the listing's SYNCED or next read. + expect(reads.mock.calls.length, `catalog=${catalog}`).toBeLessThanOrEqual(skipped.size + 3); + release(); + expect(await listing.reader.u53()).toBe(PublishSkipped.id); + const { trackName } = await PublishSkipped.decode(listing.reader, version); + expect(skipped.has(trackName), "skipped a track twice").toBe(false); + skipped.add(trackName); + } + await clock.advance(Nano.zero); + expect(gates.length, "skipped past the listing").toBe(0); + } + }); +} diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index dd7c6a09c28..362180d38cb 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -223,7 +223,7 @@ test("a peer's malformed listing closes the session", async () => { test("a listing past the announces cap is reset, and the session stays", async () => { const cap = 3; - const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap, requests: 10 } }); + const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap } }); const tracks = subscriber.consume(Path.from("room")).tracks(); const peer = await listed(); for (let i = 0; i <= cap; i++) await peer.send({ kind: "added", name: `t${i}`, info }); diff --git a/js/net/src/session-limits.test.ts b/js/net/src/session-limits.test.ts index eed9170df27..e2b6a455602 100644 --- a/js/net/src/session-limits.test.ts +++ b/js/net/src/session-limits.test.ts @@ -14,7 +14,7 @@ import { Stream, Writer } from "./stream.ts"; import type { Limits } from "./util/limits.ts"; // Small caps keep each case fast. -const limits: Limits = { announces: 8, requests: 4 }; +const limits: Limits = { announces: 8 }; test.each(["lite", "ietf"] as const)("%s closes on the first advertisement beyond its cap", async (protocol) => { const pair = createMockTransportPair(); diff --git a/js/net/src/util/abort.ts b/js/net/src/util/abort.ts index ed1f32ba8b2..6acbcfe1380 100644 --- a/js/net/src/util/abort.ts +++ b/js/net/src/util/abort.ts @@ -5,7 +5,11 @@ import { race } from "@moq/signals"; // whatever it holds when this rejects. export async function untilAborted(promise: Promise, signal?: AbortSignal): Promise { if (!signal) return promise; - signal.throwIfAborted(); + if (signal.aborted) { + // Observed, since the caller drops it: a later rejection is not unhandled. + promise.catch(() => undefined); + throw signal.reason; + } const { promise: aborted, reject } = Promise.withResolvers(); const onAbort = () => reject(signal.reason); diff --git a/js/net/src/util/limits.test.ts b/js/net/src/util/limits.test.ts index 2f5c7ba0172..32df97bcf71 100644 --- a/js/net/src/util/limits.test.ts +++ b/js/net/src/util/limits.test.ts @@ -34,10 +34,4 @@ test("tryAcquire refuses at the cap without closing the session", () => { expect(slots.tryAcquire()).toBe(false); slots.release(); expect(slots.tryAcquire()).toBe(true); - // A held entry counts past the cap, so tryAcquire waits for it too. - slots.hold(); - slots.release(); - expect(slots.tryAcquire()).toBe(false); - slots.release(); - expect(slots.tryAcquire()).toBe(true); }); diff --git a/js/net/src/util/limits.ts b/js/net/src/util/limits.ts index e8c412db895..9fe30eedf72 100644 --- a/js/net/src/util/limits.ts +++ b/js/net/src/util/limits.ts @@ -1,19 +1,18 @@ import { Session, SessionCode } from "../error.ts"; /** - * How many live requests a peer may hold on one session, mirroring moq-net's `session::Limits`. - * Subscriptions and FETCHes have no cap: each holds a bidi stream, so stream credit bounds them. + * How many live requests a peer may hold on one session, as moq-net's `session::Limits`. + * Subscriptions, FETCHes, and listing PUBLISHes have no cap: each holds a bidi stream, so stream + * credit bounds them. * @internal */ export interface Limits { /** Broadcasts (moq-lite) or namespaces (moq-transport) the peer may have announced to us. */ announces: number; - /** Subscriptions and listing PUBLISHes a moq-transport peer may hold before listed tracks are skipped. */ - requests: number; } /** The default limits, matching moq-net's. @internal */ -export const LIMITS: Readonly = Object.freeze({ announces: 100_000, requests: 20_000 }); +export const LIMITS: Readonly = Object.freeze({ announces: 100_000 }); /** A session's count of live peer requests, which closes the session past its cap. @internal */ export class Slots { @@ -43,11 +42,6 @@ export class Slots { return true; } - /** Count one entry something else already bounds, past the cap too, so `tryAcquire` sees it. */ - hold() { - this.#live += 1; - } - /** Give back `count` slots. */ release(count = 1) { this.#live -= count; diff --git a/quest/m1/README.md b/quest/m1/README.md index b355b87a40a..f3546cae1d2 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -93,7 +93,6 @@ Binding durations wait on Duration support in the generated C. - [moq-congestion](/quest/m1/congestion/README.md) - one QUIC congestion-control crate with a transport-independent trait, run by `moq-quic` and published for noq - [Release audit findings](/quest/m1/release-audit/README.md) - resolve newly identified resource, parser, playback, and interoperability risks before the cut -- [JS SUBSCRIBE_TRACKS follows credit](/quest/m1/js-listing-credit.md) - a `@moq/net` listing opens PUBLISHes only as stream credit admits, like moq-net - [Cancelled moq-transport requests stop](/quest/m1/ietf-request-cancel.md) - a cancelled request releases its state even when the peer withholds credit - [Empty segments refused](/quest/m1/empty-segments.md) - `room//cam` fails loud instead of naming `room/cam` - [In-band auth](/quest/m1/auth/README.md) - a session tells its peer what it may publish and subscribe to, unions tokens presented in band, and fails loud on an out-of-scope publish diff --git a/quest/m1/js-listing-credit.md b/quest/m1/js-listing-credit.md deleted file mode 100644 index d4334d6784b..00000000000 --- a/quest/m1/js-listing-credit.md +++ /dev/null @@ -1,26 +0,0 @@ -# [S] JS SUBSCRIBE_TRACKS listings follow stream credit - -## Goal - -A `@moq/net` draft-18+ SUBSCRIBE_TRACKS listing holds state in proportion to -what the peer reads, not to the catalog, as moq-net's does. Each listing -PUBLISH opens only as the peer's stream credit admits it, a FORWARD=1 listing -reads its next track only as its skips go out, and the listing no longer uses -the `Limits` request cap. - -## Plan - -`#listBroadcast` in `js/net/src/ietf/publisher.ts` starts a PUBLISH for every -listed track up to the `Limits` cap and skips the rest, and queues every -FORWARD=1 skip at once. Mirror Rust's `run_listed_broadcast` in -`rs/moq-net/src/ietf/publisher/tracks.rs`: stage one track per broadcast, open -its stream only as credit admits, keep reading the listing for removals -meanwhile, since an unlisted track's PUBLISH_DONE is what returns credit, and -skip a track before its stream opens. Stream credit bounds open streams only -once `@moq/qmux` returns it when both halves close -([moq-dev/moq#6644](https://github.com/moq-dev/moq/pull/6644)). - -Test with a peer that withholds credit and reads nothing, swept over catalog -size. - -Public API: none expected. Wire: none.