From 87db4783098ce65f70c89113028381422cddfdec Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 10:45:27 -0700 Subject: [PATCH 1/2] chore(quest): claim quest/m1/js-listing-credit.md Quest-Claim: f12d211316a21f24 From 105ac529219271366a43f8c8e106e367c84a8836 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 11:20:21 -0700 Subject: [PATCH 2/2] fix(net): open JS SUBSCRIBE_TRACKS PUBLISHes only as stream credit admits Mirror moq-net's run_listed_broadcast in @moq/net's #listBroadcast: stage one track per broadcast and open its PUBLISH stream only as the peer's credit admits, keep reading the listing for removals meanwhile, and have a FORWARD=1 listing read its next track only once the request stream took the last skip. The listing no longer uses the Limits request cap, so Limits.requests and Slots.hold are gone. untilAborted now observes the promise it drops when its signal is already aborted, which a peer resetting a PUBLISH during its write surfaced as an unhandled rejection. Co-Authored-By: Claude Opus 5.5 --- doc/concept/standard.md | 2 +- js/net/src/ietf/publisher.ts | 208 ++++++++++++++------- js/net/src/ietf/subscribe_tracks.test.ts | 223 +++++++++++++++++------ js/net/src/lite/tracks.test.ts | 2 +- js/net/src/session-limits.test.ts | 2 +- js/net/src/util/abort.ts | 6 +- js/net/src/util/limits.test.ts | 6 - js/net/src/util/limits.ts | 14 +- quest/m1/README.md | 1 - quest/m1/js-listing-credit.md | 26 --- 10 files changed, 324 insertions(+), 166 deletions(-) delete mode 100644 quest/m1/js-listing-credit.md diff --git a/doc/concept/standard.md b/doc/concept/standard.md index 24c56defda0..d8eef08a4e3 100644 --- a/doc/concept/standard.md +++ b/doc/concept/standard.md @@ -26,7 +26,7 @@ maps everything else to "not supported" or a harmless equivalent. The subset drops. What a peer actually observes against this implementation: - **Pull.** Subscribers ask. Single-track `PUBLISH` offers are declined, unless they answer our `SUBSCRIBE_TRACKS` (below); announce a namespace and serve the resulting subscriptions. Announcements go out unsolicited, and we also ask for every prefix we may discover. The solicit `SETUP` option makes us wait to be asked. Without it, a draft 16+ peer also gets each match as a `NAMESPACE` on its `SUBSCRIBE_NAMESPACE` stream, so it hears the namespace twice. We send `PUBLISH` only to answer a `SUBSCRIBE_TRACKS` (below): on drafts 16 and 17 a `SUBSCRIBE_NAMESPACE` asking only for `PUBLISH` is refused, one asking for both gets only `NAMESPACE`, and any other Subscribe Options value closes the session. -- **`SUBSCRIBE_TRACKS` lists, it never pushes.** On draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Rust opens each paused `PUBLISH` only as the peer's stream credit admits it, so a peer that grants none holds none; JavaScript caps them, and past the cap a track gets `PUBLISH_SKIPPED` (`PUBLISH_BLOCKED` on draft 18) instead. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. Rust sends it upstream for a listing (below); JavaScript does not. +- **`SUBSCRIBE_TRACKS` lists, it never pushes.** On draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Each paused `PUBLISH` opens only as the peer's stream credit admits it, so a peer that grants none holds none. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. Rust sends it upstream for a listing (below); JavaScript does not. - **A Rust TRACKS listing sends `SUBSCRIBE_TRACKS` upstream, only on demand.** Listing a broadcast from a moq-transport peer on draft 18 and later sends `SUBSCRIBE_TRACKS` with `FORWARD=0` while someone lists it, never on connect, announce, or subscribe, so a peer that refuses, resets, or ignores it affects only that listing. Before draft 18 the listing is refused as unsupported. Each listed track's paused `PUBLISH` is its upstream copy, which a reader resumes rather than subscribing again. Only a `PUBLISH_DONE` with `TRACK_ENDED` removes a listed track, and it lists as finished; any other end, including the error our own answerer sends for a track that left unfinished, fails the listing, and readers list again. - **History is one group.** In Rust, Object ID gaps must be marked by Prior Object ID Gap properties for a group to be fetchable. A `FETCH` returns one group from the cache, or on drafts 14 through 19 the saved prefix of the group a new subscription just joined. A range of groups is refused, as is a draft 20+ `LOCATION_FILTER` that ends at Largest Object rather than an absolute End Location. On drafts 14 through 19, `FETCH_OK` End Object 0 covers the entire End Group, matching the request; Rust also accepts the legacy answer naming the next group with Object 0. On draft 20 and later, `FETCH_OK`'s End Location is the requested end, capped at Largest Object: a range holding no objects gets an empty fetch stream, and only a start past Largest Object is refused, with `INVALID_RANGE`. A relay's copy with no upstream subscription can't know Largest Object short of the track's end, so it echoes the requested end instead. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed, and so does a reader whose publisher refuses `TRACK_STATUS` as `NOT_SUPPORTED`. - **Compression.** Rust peers on draft 17 and later exchange compressed tracks when both offer the [Flate extension](/draft/moq-flate). Older peers receive one shared inflated copy. A publisher that seals the compression declaration makes that fallback unavailable: an incompatible peer is refused, preserving the sealed metadata. JavaScript does not offer the IETF extension. diff --git a/js/net/src/ietf/publisher.ts b/js/net/src/ietf/publisher.ts index 4455b6e2b47..86173d8723b 100644 --- a/js/net/src/ietf/publisher.ts +++ b/js/net/src/ietf/publisher.ts @@ -33,7 +33,6 @@ import type { Groups, Info, Position as TrackPosition, Subscriber as TrackSubscr import { untilAborted } from "../util/abort.ts"; import { nextChange } from "../util/change.ts"; import { Deadline, REQUEST_TIMEOUT_MS, requestOptions } from "../util/deadline.ts"; -import { LIMITS, type Limits, Slots } from "../util/limits.ts"; import { TimeoutError, withTimeout } from "../util/timeout.ts"; import * as Varint from "../varint.ts"; import { type Advertised, type Advertisements, sameInstance, wireOf } from "../wire.ts"; @@ -272,6 +271,21 @@ interface Published { /** How a listed track left the listing: it finished, or it left unfinished. */ type Unlisted = "finished" | "removed"; +/** The next listed track to PUBLISH, ready but for its stream. */ +interface Staged { + name: string; + info: Info; + /** On draft-18, claims the track's one subscription from now on. */ + claim?: string; +} + +/** What a broadcast's listing woke for. */ +type Woken = + | { kind: "track"; event: broadcast.TracksEvent | undefined } + | { kind: "opened"; stream: Stream } + | { kind: "failed"; err: unknown } + | { kind: "changed" }; + /** Stop forwarding without ending the subscription. */ class Paused extends Error { /** The update to answer once forwarding stopped, if one asked; its own Request ID on drafts 15 and 16. */ @@ -331,8 +345,6 @@ export class Publisher { #quic: WebTransport; #session: Session; #requiresSolicitation: boolean; - // Subscriptions the peer holds, PUBLISHes for its listings capped by `Limits.requests`. - #subscriptions: Slots; #listings = new Set(); #trackClaims = new Map(); @@ -395,7 +407,6 @@ export class Publisher { grant = new Signal(undefined), auth, ready = Promise.resolve(), - limits = LIMITS, }: { /** The WebTransport session, for uni streams. */ quic: WebTransport; @@ -419,11 +430,8 @@ export class Publisher { auth?: Carrier; /** Resolves once the setup tokens are answered (MoQ Auth). */ ready?: Promise; - /** How many live requests the peer may hold. */ - limits?: Limits; }) { this.#quic = quic; - this.#subscriptions = new Slots(limits.requests, { reason: "too many requests" }); this.#grant = grant; this.#gate = new Gate(grant); this.#auth = auth; @@ -479,15 +487,10 @@ export class Publisher { ); return; } - // Only a subscription we would serve holds a slot, as in Rust: a refused one ends at - // once. Stream credit bounds them, so none is refused here. - const charged = !msg.rangeFilters && this.#gate.allows(msg.trackNamespace); - if (charged) this.#subscriptions.hold(); if (claimed) this.#trackClaims.set(key, ((this.#trackClaims.get(key) as number | undefined) ?? 0) + 1); try { await this.#serveSubscribe(msg, request); } finally { - if (charged) this.#subscriptions.release(); if (claimed) { const count = (this.#trackClaims.get(key) as number) - 1; if (count) this.#trackClaims.set(key, count); @@ -2370,6 +2373,8 @@ export class Publisher { continue; } skips.delete(first[0]); + // A FORWARD=1 listing waits for this to read its next track. + changed.update((n) => n + 1); await Message.write(request.writer, PublishSkipped.id, first[1], version); } if (refusal) await request.refuse("subscribe_namespace", refusal); @@ -2436,6 +2441,16 @@ export class Publisher { } } + /** + * List one broadcast under a SUBSCRIBE_TRACKS: PUBLISH or skip each of its tracks until its + * route goes or the broadcast ends, then let its PUBLISHes run to their end. + * + * A PUBLISH starts only once the peer's stream credit opens its stream, one staged at a time, + * so the PUBLISHes a peer holds are bounded by the credit it grants. The listing is still read + * meanwhile, since an unlisted track's PUBLISH_DONE is what returns that credit; the tracks + * read but not yet PUBLISHed wait in `pending`. A FORWARD=1 listing has nothing to unlist, so + * it reads its next track only once the request stream has taken the last skip. + */ async #listBroadcast({ msg, suffix, @@ -2453,12 +2468,38 @@ export class Publisher { skips: Map; changed: Signal; }) { + const forward = msg.parameters.forward !== false; const namespace = Path.join(msg.prefix, suffix); + // The tracks waiting their turn for a PUBLISH, in listing order. + const pending = new Map(); + // The next track to PUBLISH, waiting on the peer's stream credit. + let staged: Staged | undefined; + // The stream the next PUBLISH goes out on, once the peer's credit admits it. + let opening: Promise | undefined; + // The listing's next event, read while anything else is awaited. + let reading: Promise | undefined; + // Each sent PUBLISH's entry: setting it says the track is no longer listed. const listed = new Map>(); + // The tracks with a skip queued, so the broadcast's end cancels only its own. const skipped = new Map(); const tasks = new Set>(); + const skip = (name: string) => { + const key = Symbol(); + skipped.set(name, key); + skips.set(key, new PublishSkipped(suffix, name)); + changed.update((n) => n + 1); + }; + const unstage = () => { + if (staged?.claim) this.#trackClaims.delete(staged.claim); + staged = undefined; + }; + // The PUBLISHes already sent still end when their track is unlisted, so the listing is + // still read once cancelled, adding nothing. const cancel = adding.subscribe((active) => { - if (!active && !tasks.size) hold.abort(); + if (active) return; + pending.clear(); + unstage(); + if (!tasks.size) hold.abort(); }); let listing: broadcast.Tracks | undefined; try { @@ -2466,10 +2507,84 @@ export class Publisher { const source = await this.#resolve(namespace, hold.signal, epoch); listing = source.tracks(); for (;;) { - const event = await untilAborted(listing.next(), hold.signal); + // Stage the next pending track that will be PUBLISHed, skipping those that will + // not, so a stream is opened only for a PUBLISH that goes out. + for (const [name, info] of pending) { + if (staged || !adding.peek()) break; + pending.delete(name); + // Draft-18 allows one subscription per track (section 5.1), so a track the peer + // is subscribed to, by its SUBSCRIBE or a PUBLISH still alive, is skipped. + // Claimed from now, so a SUBSCRIBE cannot slip in before the PUBLISH goes out. + const claim = + this.#session.version === Version.DRAFT_18 ? JSON.stringify([namespace, name]) : undefined; + if (claim !== undefined) { + if (this.#trackClaims.has(claim)) { + skip(name); + continue; + } + this.#trackClaims.set(claim, "publish"); + } + staged = { name, info, claim }; + } + if (staged && !opening) + opening = Promise.resolve(this.#session.openBi({ timeout: null })).then( + (stream): Woken => ({ kind: "opened", stream }), + (err: unknown): Woken => ({ kind: "failed", err }), + ); + const waits: PromiseLike[] = []; + if (opening) waits.push(opening); + // A FORWARD=1 listing has nothing to unlist, so it reads its next track only once + // the request stream has taken the last skip. + if (forward && skips.size) waits.push(Signal.race(changed).then((): Woken => ({ kind: "changed" }))); + else { + reading ??= listing.next().then((event): Woken => ({ kind: "track", event })); + waits.push(reading); + } + const woken = await untilAborted(race(waits), hold.signal); + if (woken.kind === "changed") continue; + if (woken.kind === "failed") { + console.debug(`listed broadcast=${namespace}: PUBLISH stream did not open`, woken.err); + break; + } + if (woken.kind === "opened") { + opening = undefined; + const next = staged; + if (!next || !adding.peek()) { + woken.stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + continue; + } + staged = undefined; + const unlisted = new Once(); + listed.set(next.name, unlisted); + const task = this.#runPublish({ + msg, + namespace, + source, + stream: woken.stream, + name: next.name, + info: next.info, + unlisted, + adding, + }) + .catch((err) => { + if (err instanceof ProtocolViolation) throw err; + }) + .finally(() => { + if (next.claim) this.#trackClaims.delete(next.claim); + tasks.delete(task); + if (!adding.peek() && !tasks.size) hold.abort(); + }); + tasks.add(task); + void task.catch(() => hold.abort()); + continue; + } + reading = undefined; + const event = woken.event; if (!event) break; if (event.kind === "synced") continue; if (event.kind === "removed" || event.kind === "finished") { + if (staged?.name === event.name) unstage(); + pending.delete(event.name); listed.get(event.name)?.set(event.kind); listed.delete(event.name); const key = skipped.get(event.name); @@ -2478,44 +2593,19 @@ export class Publisher { continue; } if (!adding.peek()) continue; - const duplicate = - this.#session.version === Version.DRAFT_18 && - this.#trackClaims.has(JSON.stringify([namespace, event.name])); - if (msg.parameters.forward !== false || duplicate || !this.#subscriptions.tryAcquire()) { - const key = Symbol(); - skipped.set(event.name, key); - skips.set(key, new PublishSkipped(suffix, event.name)); - changed.update((n) => n + 1); - continue; - } - const unlisted = new Once(); - listed.set(event.name, unlisted); - const key = JSON.stringify([namespace, event.name]); - if (this.#session.version === Version.DRAFT_18) this.#trackClaims.set(key, "publish"); - const task = this.#runPublish({ - msg, - namespace, - source, - name: event.name, - info: event.info, - unlisted, - adding, - }) - .catch((err) => { - if (err instanceof ProtocolViolation) throw err; - }) - .finally(() => { - this.#subscriptions.release(); - if (this.#session.version === Version.DRAFT_18) this.#trackClaims.delete(key); - tasks.delete(task); - if (!adding.peek() && !tasks.size) hold.abort(); - }); - tasks.add(task); - void task.catch(() => hold.abort()); + if (forward) skip(event.name); + else pending.set(event.name, event.info); } } finally { cancel(); listing?.close(); + pending.clear(); + unstage(); + // A stream that opens from now on carries nothing. + void opening?.then((woken) => { + if (woken.kind === "opened") + woken.stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + }); for (const entry of listed.values()) entry.set("removed"); for (const key of skipped.values()) skips.delete(key); await Promise.all(tasks); @@ -2526,6 +2616,7 @@ export class Publisher { msg, namespace, source, + stream, name, info, unlisted, @@ -2534,6 +2625,8 @@ export class Publisher { msg: SubscribeTracks; namespace: Path.Valid; source: broadcast.Consumer; + /** The PUBLISH's stream, opened once the peer's credit admitted it. */ + stream: Stream; name: string; info: Info; unlisted: Once; @@ -2551,21 +2644,10 @@ export class Publisher { const removed = unlisted.subscribe((how) => { if (how === "removed" || (how === "finished" && !publishing)) hold.abort(); }); - const cancelled = () => request?.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); + const request = new RequestStream(stream, this.#session, this.#quic); + const cancelled = () => request.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); hold.signal.addEventListener("abort", cancelled); - let request: RequestStream | undefined; try { - if (!adding.peek() || unlisted.peek() !== undefined) return; - const opening = Promise.resolve(this.#session.openBi({ timeout: null })); - void opening.then( - (stream) => { - if (hold.signal.aborted) - stream.abort(new StreamError(StreamCode.Cancel, { message: "listing ended" })); - }, - () => undefined, - ); - const stream = await untilAborted(opening, hold.signal); - request = new RequestStream(stream, this.#session, this.#quic); let peerEnded = false; const ended = request.ended.then(() => { peerEnded = true; @@ -2762,14 +2844,14 @@ export class Publisher { } } } catch (err) { - request?.abort(err); + request.abort(err); throw err; } finally { stop(); removed(); hold.signal.removeEventListener("abort", cancelled); hold.abort(); - request?.close(); + request.close(); } } diff --git a/js/net/src/ietf/subscribe_tracks.test.ts b/js/net/src/ietf/subscribe_tracks.test.ts index 011cf85942a..49a2967e053 100644 --- a/js/net/src/ietf/subscribe_tracks.test.ts +++ b/js/net/src/ietf/subscribe_tracks.test.ts @@ -1,7 +1,9 @@ import { expect, onTestFinished, spyOn, test } from "bun:test"; import { Manual, Nano } from "@moq/clock"; +import Qmux from "@moq/qmux"; import { Signal } from "@moq/signals"; import type { Grant } from "../auth.ts"; +import { Tracks } from "../broadcast.ts"; import { StreamCode, StreamError } from "../error.ts"; import { createMockTransportPair } from "../mock.ts"; import { Producer as Origin } from "../origin.ts"; @@ -20,8 +22,43 @@ import { ALPN, type IetfVersion, Version } from "./version.ts"; const drafts = [Version.DRAFT_18, Version.DRAFT_19, Version.DRAFT_20, Version.DRAFT_21, Version.DRAFT_22]; -function fixture(version: IetfVersion, requests = 10, grant = new Signal(undefined)) { - const pair = createMockTransportPair(ALPN.DRAFT_22); +/** A client and server WebTransport, connected to each other. */ +type Pair = { client: WebTransport; server: WebTransport }; + +/** + * A qmux session pair over an in-memory socket, which, unlike the mock transport, holds each + * bidi stream against the receiver's stream credit until both its halves end. The client + * grants the server `credit` such streams at once. + */ +function qmuxPair(credit: bigint): Pair { + const socket = (readable: ReadableStream, writable: WritableStream) => { + const closed = Promise.withResolvers<{ closeCode: number; reason: string }>(); + return { + url: "", + opened: Promise.resolve({ readable, writable, extensions: "", protocol: "qmux-01" }), + closed: closed.promise, + close: () => closed.resolve({ closeCode: 1000, reason: "" }), + }; + }; + // Copied, since a sender may reuse its buffer once a write resolves. + const pipe = () => + new TransformStream({ transform: (chunk, c) => c.enqueue(chunk.slice()) }); + const up = pipe(); + const down = pipe(); + const client = new Qmux(socket(down.readable, up.writable), { + config: { idleTimeout: null, maxStreamsBidi: credit }, + }); + const server = Qmux.accept(socket(up.readable, down.writable), { config: { idleTimeout: null } }); + return { client, server }; +} + +function fixture( + version: IetfVersion, + { + grant = new Signal(undefined), + pair = createMockTransportPair(ALPN.DRAFT_22), + }: { grant?: Signal; pair?: Pair } = {}, +) { const origin = new Origin(); const session = new NativeSession(pair.server, version, false); const publisher = new Publisher({ @@ -30,7 +67,6 @@ function fixture(version: IetfVersion, requests = 10, grant = new Signal[] = []; const streams: Stream[] = []; @@ -38,6 +74,7 @@ function fixture(version: IetfVersion, requests = 10, grant = new Signal { - const fx = fixture(version, 1); + test(`track listing rejects overlapping prefixes (${version})`, async () => { + const fx = fixture(version); const broadcast = fx.origin.createBroadcast(Path.from("room/live")); broadcast.announce(); broadcast.createTrack("video"); - broadcast.createTrack("audio"); const listing = await fx.list(); await ok(listing, version); await fx.offered(); - expect(await listing.reader.u53()).toBe(PublishSkipped.id); - const skipped = await PublishSkipped.decode(listing.reader, version); - expect(skipped.suffix).toBe(Path.from("live")); - expect(skipped.trackName).toBe("audio"); for (const prefix of ["room/live", "", "room"]) { const overlap = await fx.list(prefix); expect(await overlap.reader.u53()).toBe(RequestError.id); @@ -245,7 +285,9 @@ for (const version of drafts) { } else { await ok(listing, version); expect(await listing.reader.u53()).toBe(PublishSkipped.id); - expect((await PublishSkipped.decode(listing.reader, version)).trackName).toBe("video"); + const skipped = await PublishSkipped.decode(listing.reader, version); + expect(skipped.suffix).toBe(Path.from("live")); + expect(skipped.trackName).toBe("video"); } }); } @@ -309,7 +351,7 @@ test("grants hide tracks, revoke PUBLISHes and relist when broadened", async () const clock = new Manual(); const installed = clock.install(); using _installed = { [Symbol.dispose]: () => installed.close() }; - const fx = fixture(version, 10, grant); + const fx = fixture(version, { grant }); const listing = await fx.list(); await ok(listing, version); const broadcast = fx.origin.createBroadcast(Path.from("room/live")); @@ -381,47 +423,6 @@ test("an update ends only its listing and frees its prefix", async () => { await ok(replacement, version); }); -test("paused updates keep one slot and release it when the PUBLISH ends", async () => { - const version = Version.DRAFT_22; - const fx = fixture(version, 1); - const clock = new Manual(); - const installed = clock.install(); - using _installed = { [Symbol.dispose]: () => installed.close() }; - const broadcast = fx.origin.createBroadcast(Path.from("room/live")); - broadcast.announce(); - broadcast.createTrack("video"); - const listing = await fx.list(); - await ok(listing, version); - const offered = await fx.offered(); - for (let i = 0; i < 32; i++) await update(offered.stream, version, BigInt(2 * (i + 1)), false); - broadcast.createTrack("skipped"); - expect(await listing.reader.u53()).toBe(PublishSkipped.id); - expect((await PublishSkipped.decode(listing.reader, version)).trackName).toBe("skipped"); - offered.stream.abort(new StreamError(StreamCode.Cancel)); - await clock.advance(Nano.zero); - broadcast.createTrack("available"); - expect((await fx.offered()).publish.trackName).toBe("available"); -}); - -test("a failed PUBLISH stream open returns its subscription slot", async () => { - const version = Version.DRAFT_22; - const fx = fixture(version, 1); - const clock = new Manual(); - const installed = clock.install(); - using _installed = { [Symbol.dispose]: () => installed.close() }; - const open = spyOn(fx.session, "openBi").mockRejectedValue(new StreamError(StreamCode.Cancel)); - const broadcast = fx.origin.createBroadcast(Path.from("room/live")); - broadcast.announce(); - broadcast.createTrack("video"); - const listing = await fx.list(); - await ok(listing, version); - await clock.advance(Nano.zero); - expect(open).toHaveBeenCalledTimes(1); - open.mockRestore(); - broadcast.createTrack("available"); - expect((await fx.offered()).publish.trackName).toBe("available"); -}); - test("session shutdown releases listing tasks while the local origin stays open", async () => { const version = Version.DRAFT_22; const fx = fixture(version); @@ -483,3 +484,113 @@ for (const version of [Version.DRAFT_20, Version.DRAFT_22]) { uni.releaseLock(); }); } + +/** Listed broadcast sizes to sweep: what a listing holds must not grow with them. */ +const CATALOGS = [4, 64, 512]; + +/** A broadcast under `room` listing `catalog` tracks. */ +function catalogOf(fx: ReturnType, catalog: number) { + const broadcast = fx.origin.createBroadcast(Path.from("room/live")); + broadcast.announce(); + for (let i = 0; i < catalog; i++) broadcast.createTrack(`t${i}`); +} + +// Each PUBLISH opens only as the peer's stream credit admits it: a peer that withholds credit and +// reads nothing leaves one stream open waiting, however many tracks are listed, and each track is +// PUBLISHed once credit returns. +for (const version of drafts) { + test(`a listing publishes only as credit admits (${version})`, async () => { + const credit = 2; + const clock = new Manual(); + const installed = clock.install(); + using _installed = { [Symbol.dispose]: () => installed.close() }; + for (const catalog of CATALOGS) { + const pair = qmuxPair(BigInt(credit)); + const open = pair.server.createBidirectionalStream.bind(pair.server); + let waiting = 0; + spyOn(pair.server, "createBidirectionalStream").mockImplementation((options) => { + waiting += 1; + const opened = open(options); + const done = () => { + waiting -= 1; + }; + opened.then(done, done); + return opened; + }); + const fx = fixture(version, { pair }); + catalogOf(fx, catalog); + const listing = await fx.list(); + await ok(listing, version); + const held = []; + for (let i = 0; i < credit; i++) held.push(await fx.offered(false)); + await clock.advance(Nano.zero); + expect(waiting, `catalog=${catalog}`).toBeLessThanOrEqual(1); + + // Refusing each PUBLISH returns its credit, which opens the next. + const published = new Set(held.map(({ publish }) => publish.trackName)); + for (const { stream } of held) stream.abort(new StreamError(StreamCode.Cancel)); + while (published.size < catalog) { + const { stream, publish } = await fx.offered(false); + expect(published.has(publish.trackName), "PUBLISHed a track twice").toBe(false); + published.add(publish.trackName); + stream.abort(new StreamError(StreamCode.Cancel)); + expect(waiting, `catalog=${catalog}`).toBeLessThanOrEqual(1); + } + } + }); +} + +// A FORWARD=1 listing reads its next track only once the request stream has taken the last +// skip, so a peer that stops reading leaves one skip queued, however many tracks are listed. +for (const version of [Version.DRAFT_19, Version.DRAFT_22]) { + test(`a forwarding listing reads only as skips go out (${version})`, async () => { + const clock = new Manual(); + const installed = clock.install(); + using _installed = { [Symbol.dispose]: () => installed.close() }; + // Hold each skip's write until the test lets it out, as a peer that stops reading would. + const gates: (() => void)[] = []; + let arrived = Promise.withResolvers(); + const write = Message.write; + const writes = spyOn(Message, "write").mockImplementation(async (writer, id, msg, version) => { + if (id === PublishSkipped.id) + await new Promise((resolve) => { + gates.push(resolve); + arrived.resolve(); + }); + return write(writer, id, msg, version); + }); + const gate = async () => { + while (!gates.length) { + await arrived.promise; + arrived = Promise.withResolvers(); + } + return gates.shift() as () => void; + }; + const reads = spyOn(Tracks.prototype, "next"); + onTestFinished(() => { + writes.mockRestore(); + reads.mockRestore(); + }); + for (const catalog of CATALOGS) { + reads.mockClear(); + const fx = fixture(version); + catalogOf(fx, catalog); + const listing = await fx.list("room", true); + await ok(listing, version); + const skipped = new Set(); + while (skipped.size < catalog) { + const release = await gate(); + await clock.advance(Nano.zero); + // The skip being written, one queued, and the listing's SYNCED or next read. + expect(reads.mock.calls.length, `catalog=${catalog}`).toBeLessThanOrEqual(skipped.size + 3); + release(); + expect(await listing.reader.u53()).toBe(PublishSkipped.id); + const { trackName } = await PublishSkipped.decode(listing.reader, version); + expect(skipped.has(trackName), "skipped a track twice").toBe(false); + skipped.add(trackName); + } + await clock.advance(Nano.zero); + expect(gates.length, "skipped past the listing").toBe(0); + } + }); +} diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index dd7c6a09c28..362180d38cb 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -223,7 +223,7 @@ test("a peer's malformed listing closes the session", async () => { test("a listing past the announces cap is reset, and the session stays", async () => { const cap = 3; - const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap, requests: 10 } }); + const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap } }); const tracks = subscriber.consume(Path.from("room")).tracks(); const peer = await listed(); for (let i = 0; i <= cap; i++) await peer.send({ kind: "added", name: `t${i}`, info }); diff --git a/js/net/src/session-limits.test.ts b/js/net/src/session-limits.test.ts index eed9170df27..e2b6a455602 100644 --- a/js/net/src/session-limits.test.ts +++ b/js/net/src/session-limits.test.ts @@ -14,7 +14,7 @@ import { Stream, Writer } from "./stream.ts"; import type { Limits } from "./util/limits.ts"; // Small caps keep each case fast. -const limits: Limits = { announces: 8, requests: 4 }; +const limits: Limits = { announces: 8 }; test.each(["lite", "ietf"] as const)("%s closes on the first advertisement beyond its cap", async (protocol) => { const pair = createMockTransportPair(); diff --git a/js/net/src/util/abort.ts b/js/net/src/util/abort.ts index ed1f32ba8b2..6acbcfe1380 100644 --- a/js/net/src/util/abort.ts +++ b/js/net/src/util/abort.ts @@ -5,7 +5,11 @@ import { race } from "@moq/signals"; // whatever it holds when this rejects. export async function untilAborted(promise: Promise, signal?: AbortSignal): Promise { if (!signal) return promise; - signal.throwIfAborted(); + if (signal.aborted) { + // Observed, since the caller drops it: a later rejection is not unhandled. + promise.catch(() => undefined); + throw signal.reason; + } const { promise: aborted, reject } = Promise.withResolvers(); const onAbort = () => reject(signal.reason); diff --git a/js/net/src/util/limits.test.ts b/js/net/src/util/limits.test.ts index 2f5c7ba0172..32df97bcf71 100644 --- a/js/net/src/util/limits.test.ts +++ b/js/net/src/util/limits.test.ts @@ -34,10 +34,4 @@ test("tryAcquire refuses at the cap without closing the session", () => { expect(slots.tryAcquire()).toBe(false); slots.release(); expect(slots.tryAcquire()).toBe(true); - // A held entry counts past the cap, so tryAcquire waits for it too. - slots.hold(); - slots.release(); - expect(slots.tryAcquire()).toBe(false); - slots.release(); - expect(slots.tryAcquire()).toBe(true); }); diff --git a/js/net/src/util/limits.ts b/js/net/src/util/limits.ts index e8c412db895..9fe30eedf72 100644 --- a/js/net/src/util/limits.ts +++ b/js/net/src/util/limits.ts @@ -1,19 +1,18 @@ import { Session, SessionCode } from "../error.ts"; /** - * How many live requests a peer may hold on one session, mirroring moq-net's `session::Limits`. - * Subscriptions and FETCHes have no cap: each holds a bidi stream, so stream credit bounds them. + * How many live requests a peer may hold on one session, as moq-net's `session::Limits`. + * Subscriptions, FETCHes, and listing PUBLISHes have no cap: each holds a bidi stream, so stream + * credit bounds them. * @internal */ export interface Limits { /** Broadcasts (moq-lite) or namespaces (moq-transport) the peer may have announced to us. */ announces: number; - /** Subscriptions and listing PUBLISHes a moq-transport peer may hold before listed tracks are skipped. */ - requests: number; } /** The default limits, matching moq-net's. @internal */ -export const LIMITS: Readonly = Object.freeze({ announces: 100_000, requests: 20_000 }); +export const LIMITS: Readonly = Object.freeze({ announces: 100_000 }); /** A session's count of live peer requests, which closes the session past its cap. @internal */ export class Slots { @@ -43,11 +42,6 @@ export class Slots { return true; } - /** Count one entry something else already bounds, past the cap too, so `tryAcquire` sees it. */ - hold() { - this.#live += 1; - } - /** Give back `count` slots. */ release(count = 1) { this.#live -= count; diff --git a/quest/m1/README.md b/quest/m1/README.md index 3bec9ba3912..d8d7d673293 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -86,7 +86,6 @@ audit's receive-memory, priority, and lease bugs rank ahead of that reshape. - [moq-congestion](/quest/m1/congestion/README.md) - one QUIC congestion-control crate with a transport-independent trait, run by `moq-quic` and published for noq - [Release audit findings](/quest/m1/release-audit/README.md) - resolve newly identified resource, parser, playback, and interoperability risks before the cut -- [JS SUBSCRIBE_TRACKS follows credit](/quest/m1/js-listing-credit.md) - a `@moq/net` listing opens PUBLISHes only as stream credit admits, like moq-net - [Cancelled moq-transport requests stop](/quest/m1/ietf-request-cancel.md) - a cancelled request releases its state even when the peer withholds credit - [SUBSCRIBE setup sees a cancel](/quest/m1/ietf-setup-cancel.md) - a subscriber that cancels mid-setup releases upstream demand and its slot at once - [Every close records its cause](/quest/m1/close-cause.md) - protocol-violation closes and abandoned SETUP replies record why diff --git a/quest/m1/js-listing-credit.md b/quest/m1/js-listing-credit.md deleted file mode 100644 index d4334d6784b..00000000000 --- a/quest/m1/js-listing-credit.md +++ /dev/null @@ -1,26 +0,0 @@ -# [S] JS SUBSCRIBE_TRACKS listings follow stream credit - -## Goal - -A `@moq/net` draft-18+ SUBSCRIBE_TRACKS listing holds state in proportion to -what the peer reads, not to the catalog, as moq-net's does. Each listing -PUBLISH opens only as the peer's stream credit admits it, a FORWARD=1 listing -reads its next track only as its skips go out, and the listing no longer uses -the `Limits` request cap. - -## Plan - -`#listBroadcast` in `js/net/src/ietf/publisher.ts` starts a PUBLISH for every -listed track up to the `Limits` cap and skips the rest, and queues every -FORWARD=1 skip at once. Mirror Rust's `run_listed_broadcast` in -`rs/moq-net/src/ietf/publisher/tracks.rs`: stage one track per broadcast, open -its stream only as credit admits, keep reading the listing for removals -meanwhile, since an unlisted track's PUBLISH_DONE is what returns credit, and -skip a track before its stream opens. Stream credit bounds open streams only -once `@moq/qmux` returns it when both halves close -([moq-dev/moq#6644](https://github.com/moq-dev/moq/pull/6644)). - -Test with a peer that withholds credit and reads nothing, swept over catalog -size. - -Public API: none expected. Wire: none.