diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index 898cae6ea35..1a412abba2f 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -223,6 +223,10 @@ Each subscription carries the knobs that decide behavior under congestion: | **Max delay** | How long a non-latest group stays worth delivering before it is skipped. Zero means "live edge only", and raising it is also what asks for history. | | **Ranges** | Which groups to ask for: from a group, or a number of groups back from the latest, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | +On moq-lite 05 and later, a publisher explicitly reports groups that will not arrive. +This lets a finished subscription settle without waiting for missing groups. +Datagram delivery stays best effort. + Ranges are a request, not a filter. A subscriber reads from where its lowest range starts, and may also get groups another subscriber of the same track asked for. A relay merges its subscribers' ranges into one upstream request, diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 941072fbbe6..41a27fb3448 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -512,9 +512,9 @@ A route claims capability rather than inventory, so a subscription for a covered The track's immutable publisher properties are not carried here; they are fetched once via a [Track Stream](#track-stream). The subscriber needs the track's TRACK_INFO (notably its timescale) to interpret FRAME messages, and MAY open the Track and Subscribe streams concurrently, buffering frames until it arrives. -The publisher sends SUBSCRIBE_OK once the absolute start position is resolved, and SUBSCRIBE_END once no further groups will be produced and every Group Stream it opens for the subscription has been opened (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). -The publisher closes its side of the stream (FIN) after SUBSCRIBE_END, once every counted Group Stream has finished or been reset. -After reading the publisher's FIN, the subscriber MUST close its side (FIN) once its tail accounting has settled: every received Group Stream has been read to its FIN, reset, or dropped, and every remaining counted stream has arrived or exhausted the subscriber's grace period. +The publisher sends SUBSCRIBE_OK once the absolute start position is resolved, and SUBSCRIBE_END as soon as the exclusive final group is known (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). +The publisher closes its side of the stream (FIN) after SUBSCRIBE_END, once every Group Stream has finished or been reset. +After reading the publisher's FIN, the subscriber MUST close its side (FIN) once its tail accounting has settled: every received Group Stream has been read to its FIN, reset, or dropped, and every sequence in range has arrived, been named by SUBSCRIBE_DROP, or exhausted the subscriber's grace period. A publisher closing its session gracefully MUST wait for the subscriber's FIN or reset on every Subscribe Stream before closing the connection; a transport acknowledgement alone does not confirm the subscriber read the tail. A subscriber MAY cancel by closing its side (FIN); the publisher then closes its side (FIN) without owing SUBSCRIBE_END. Unbounded subscriptions stay open until SUBSCRIBE_END, and either endpoint MAY reset the stream at any time. @@ -1321,7 +1321,6 @@ SUBSCRIBE_END Message { Type (i) = 0x1 Message Length (i) Group (i) - Stream Count (i) } ~~~ @@ -1333,14 +1332,30 @@ The exclusive end of the range: the absolute sequence number of the first group A value of 0 means the track ended before producing any groups. The subscriber MUST NOT wait for any group at or after this sequence. -**Stream Count**: -The number of Group Streams the publisher opened for this subscription, whether they finished or were reset. -A group that was skipped, never produced, delivered only as a datagram, or given up before its stream opened is not counted. -A relay counts the Group Streams it opened itself, never the count it received upstream. +The publisher SHOULD send SUBSCRIBE_END as soon as the exclusive final group is known, even while lower groups are still in flight. -The publisher MUST NOT send SUBSCRIBE_END until every Group Stream it will open for the subscription has been opened, so the count is final; it does not wait for them to finish. -The subscriber has received every Group Stream once it has read the header of `Stream Count` of them, which MAY happen after SUBSCRIBE_END or the FIN since streams are not ordered. -A Group Stream reset before its header arrived is never seen, so a subscriber SHOULD bound how long it waits for the rest, for example by `Subscriber Max Age`. +## SUBSCRIBE_DROP {#subscribe-drop} + +A publisher names groups it will not deliver with SUBSCRIBE_DROP on the Subscribe Stream. + +~~~ +SUBSCRIBE_DROP Message { + Type (i) = 0x2 + Message Length (i) + Group Start (i) + Group End (i) + Error Code (i) +} +~~~ + +**Group Start** and **Group End** are the inclusive range of group sequences the publisher gave up. +**Error Code** is an application-specific error code; 0 means the groups are simply unavailable. + +For every sequence in the subscription's range, the publisher MUST send a Group Stream, send a datagram, or name it in SUBSCRIBE_DROP. +This includes sequences it skipped and streams reset before their headers were delivered. +A reliably delivered reset retaining the Group Stream header accounts for that group instead. +The publisher SHOULD send SUBSCRIBE_DROP as soon as it gives a sequence up, without waiting for SUBSCRIBE_END. +A datagram counts as sent even if it is lost; a subscriber SHOULD bound its wait for unaccounted sequences, for example by Subscriber Max Age. ## FETCH FETCH is sent by a subscriber to request a single group from a track. @@ -1618,9 +1633,8 @@ The `Message Length` describes the payload size on the wire. - Assigned `moq-lite-07-wip` as this draft's protocol identifier until it is finalized as `moq-lite-07`. - Switched every variable-length integer, including SETUP parameter values, from QUIC's two-bit length prefix to moq-transport's leading-ones encoding, widening the range to 64 bits. - Hid routes with a `.`-prefixed segment below the requested prefix from announce discovery, and added the ANNOUNCE_REQUEST `Hidden` field to opt in. -- Added `Stream Count` to SUBSCRIBE_END: the number of Group Streams opened for the subscription. SUBSCRIBE_END is now sent once every counted Group Stream has opened, rather than as soon as the final group is known. -- Removed SUBSCRIBE_DROP and its type 0x2; a group without a Group Stream is not counted. -- The Subscribe Stream FIN now follows once every counted Group Stream has finished or been reset. +- Restored SUBSCRIBE_DROP (type 0x2), replacing Stream Count on SUBSCRIBE_END. Every sequence is accounted for by a Group Stream, a datagram, or a drop; a publisher SHOULD send a drop as soon as it gives a sequence up. +- SUBSCRIBE_END declares the final boundary while lower groups can still be in flight; the Subscribe Stream FIN follows once every Group Stream has finished or been reset. - Added the Spread Hash tie-break after the shortest path: a hash of the requested path and the route's Hop IDs, so equal-cost advertisers of one prefix share its paths. - Added a local link preference to route selection, after the newest Epoch and before the 0 Hop ID check: the higher preference a subscriber assigns to the arriving link wins, and an unassigned link is neutral. Not carried on the wire. - Added announce compression: ANNOUNCE_START gains `Path Base` and `Path Keep` to copy the head of a live advertisement's suffix, and ANNOUNCE_START and ANNOUNCE_UPDATE gain `Hop Base` and `Hop Keep` to copy the tail of a live advertisement's Hop ID list. diff --git a/js/justfile b/js/justfile index 451854d81d3..2ea94f5d0ee 100644 --- a/js/justfile +++ b/js/justfile @@ -55,7 +55,7 @@ test-changed list: coverage package="": python3 ../sh/js/coverage.py '{{ package }}' -# Run one net benchmark: broadcasts, forward, requests, reader, frames, track, resume, interests, observed, varint, lite-varint, priority, or request-ids. +# Run one net benchmark: broadcasts, forward, requests, reader, frames, track, resume, interests, observed, varint, lite-varint, priority, request-ids, or subscribe-drop. bench name: bun install --frozen-lockfile bun net/bench/{{ name }}.ts diff --git a/js/net/MIGRATE.md b/js/net/MIGRATE.md index 5b81d137fa1..fbcb210ee2d 100644 --- a/js/net/MIGRATE.md +++ b/js/net/MIGRATE.md @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Lite-07 accounts for groups with `SUBSCRIBE_DROP` + +Construct `SubscribeEnd(group)` without a stream count; remove reads of `.streams` and calls to `hasStreamCount()`. +The unpublished `moq-lite-07-wip` wire restores `SUBSCRIBE_DROP` and removes Stream Count from `SUBSCRIBE_END`. Update both ends together; published lite versions keep their framing. + ### `webtransport.protocols` is `versions` (#5979) Move the list to the top-level `versions` of `connect` or `Connection`, spelled as version names: `moq-transport-22`, not the ALPN `moqt-22`; lite names are unchanged. diff --git a/js/net/bench/subscribe-drop.ts b/js/net/bench/subscribe-drop.ts new file mode 100644 index 00000000000..b7bf27d3294 --- /dev/null +++ b/js/net/bench/subscribe-drop.ts @@ -0,0 +1,102 @@ +/** Sweep failed group sends over independent publishers and subscribers. */ +import { Milli } from "@moq/clock"; +import { Producer as GroupProducer } from "../src/group.ts"; +import { randomHop } from "../src/hop.ts"; +import { Publisher } from "../src/lite/publisher.ts"; +import { decodeSubscribeResponseMaybe, Subscribe } from "../src/lite/subscribe.ts"; +import { ALPN_07_WIP, Version } from "../src/lite/version.ts"; +import { createMockTransportPair, textFrame } from "../src/mock.ts"; +import { Producer as OriginProducer } from "../src/origin.ts"; +import * as Path from "../src/path.ts"; +import { Stream } from "../src/stream.ts"; +import { Timescale } from "../src/time.ts"; + +const groups = 128; +const counts = [1, 4, 16]; +const reps = 3; +const debug = console.debug; +console.debug = () => {}; +try { + console.log("publishers,subscribers,groups,drop_us"); + for (const publishers of counts) { + for (const subscribers of counts) { + const times: number[] = []; + for (let rep = 0; rep < reps; rep++) { + const origins = Array.from({ length: publishers }, () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("bench")); + broadcast.announce(); + const track = broadcast.createTrack("video", { timescale: Timescale.MILLI }); + for (let sequence = 0; sequence < groups; sequence++) { + const group = new GroupProducer(sequence); + group.writeFrame(textFrame("frame")); + group.close(); + track.writeGroup(group); + } + track.close(); + return { origin, broadcast }; + }); + // Include connection setup, excluding group creation and cache preparation. + const start = performance.now(); + await Promise.all( + origins.flatMap(({ origin }) => + Array.from({ length: subscribers }, async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + pair.server.createUnidirectionalStream = async () => { + throw new Error("no stream slot"); + }; + const publisher = new Publisher( + pair.server, + Version.DRAFT_07, + randomHop(), + origin.consume(), + ); + const client = await Stream.open(pair.client, { version: Version.DRAFT_07 }); + const server = await Stream.accept(pair.server, Version.DRAFT_07); + if (!server) throw new Error("missing subscribe stream"); + const serving = publisher.runSubscribe( + new Subscribe({ + id: 0n, + broadcast: Path.from("bench"), + track: "video", + priority: 0, + maxDelay: Milli(30_000), + startGroup: 0, + }), + server, + ); + try { + let dropped = 0; + for (;;) { + const response = await decodeSubscribeResponseMaybe( + client.reader, + Version.DRAFT_07, + ); + if (!response) break; + if ("drop" in response) dropped += response.drop.end - response.drop.start + 1; + } + if (dropped !== groups) throw new Error(`dropped ${dropped}, expected ${groups}`); + client.writer.close(); + await serving; + } finally { + publisher.close(); + client.close(); + pair.client.close(); + pair.server.close(); + } + }), + ), + ); + times.push(((performance.now() - start) * 1000) / (publishers * subscribers * groups)); + for (const { origin, broadcast } of origins) { + broadcast.close(); + origin.close(); + } + } + times.sort((a, b) => a - b); + console.log(`${publishers},${subscribers},${groups},${times[Math.floor(reps / 2)].toFixed(2)}`); + } + } +} finally { + console.debug = debug; +} diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index 662c4da20da..0e1ac7edad0 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -740,6 +740,21 @@ test("lite draft-06: a cut group is dropped early, once", async () => { } }); +// A sent datagram is accounted for, so the end sweep does not drop it, as the Rust publisher does. +test("lite draft-06: the end sweep does not drop a sent datagram", async () => { + const sub = await servedSubscription({ version: Version.DRAFT_06, startGroup: 0 }); + try { + sub.serve(0); + expect(await sub.servedSequence()).toBe(0); + sub.track.insertDatagram(1, Timestamp.fromMillis(1), new Uint8Array([1])); + await flush(); + sub.track.close(); + expect(await drops(sub.client.reader, Version.DRAFT_06)).toEqual([]); + } finally { + await sub.close(); + } +}); + // A cut this subscription already dropped stays dropped: the subscriber accounted for it, so // a group the publisher re-creates under it afterwards is not served. test("lite draft-06: a cut dropped then re-created is not served", async () => { @@ -1256,21 +1271,19 @@ test("lite draft-05: subscribe end is 0 when no groups were produced", async () expect((await subscribeEnd([])).group).toBe(0); }); -// The count is of group streams opened, not of groups below the end: a group the track -// never produced has no stream and is not counted. -test("lite draft-07: subscribe end counts the group streams opened", async () => { +// The boundary clears every produced group even when a sequence was skipped. +test("lite draft-07: subscribe end names the final boundary", async () => { const end = await subscribeEnd([0, 2], Version.DRAFT_07); - expect([end.group, end.streams]).toEqual([3, 2]); + expect(end.group).toBe(3); }); -test("lite draft-07: subscribe end counts zero streams when no groups were produced", async () => { +test("lite draft-07: an empty track ends at zero", async () => { const end = await subscribeEnd([], Version.DRAFT_07); - expect([end.group, end.streams]).toEqual([0, 0]); + expect(end.group).toBe(0); }); -// finishAt names the end while groups below it are still being produced. The count -// cannot include a stream that has not opened, so SUBSCRIBE_END waits for them. -test("lite draft-07: subscribe end waits for groups below a declared finish", async () => { +// finishAt declares the boundary while lower groups are still being produced. +test("lite draft-07: subscribe end precedes groups below a declared finish", async () => { const pair = createMockTransportPair(ALPN_07_WIP); const origin = new OriginProducer(); const publisher = new Publisher(pair.server, Version.DRAFT_07, randomHop(), origin.consume()); @@ -1295,8 +1308,8 @@ test("lite draft-07: subscribe end waits for groups below a declared finish", as const start = await decodeSubscribeResponse(client.reader, Version.DRAFT_07); expect("start" in start).toBe(true); const pending = decodeSubscribeResponse(client.reader, Version.DRAFT_07); - const early = await Promise.race([pending, new Promise((resolve) => setTimeout(resolve, IDLE_MS))]); - expect(early).toBeUndefined(); + const early = await pending; + expect("end" in early).toBe(true); const second = new GroupProducer(1); second.writeFrame(textFrame("hello")); @@ -1306,7 +1319,7 @@ test("lite draft-07: subscribe end waits for groups below a declared finish", as const resp = await pending; if (!("end" in resp)) throw new Error("expected SUBSCRIBE_END"); - expect([resp.end.group, resp.end.streams]).toEqual([2, 2]); + expect(resp.end.group).toBe(2); } finally { publisher.close(); client.close(); @@ -1353,6 +1366,7 @@ async function heldOpenEnd() { return { end, open, + next: () => decodeSubscribeResponse(client.reader, Version.DRAFT_07), close() { publisher.close(); client.close(); @@ -1360,36 +1374,157 @@ async function heldOpenEnd() { }; } -// The count is final only once no served group is still waiting for its stream, so -// SUBSCRIBE_END waits for the open. -test("lite draft-07: subscribe end waits for every group stream to open", async () => { +// The end is known even when a group is waiting for stream credit. +test("lite draft-07: subscribe end precedes a blocked stream open", async () => { const held = await heldOpenEnd(); try { - const early = await Promise.race([held.end, new Promise((resolve) => setTimeout(resolve, IDLE_MS))]); - expect(early).toBeUndefined(); + const early = await held.end; + expect("end" in early).toBe(true); held.open(true); const resp = await held.end; if (!("end" in resp)) throw new Error("expected SUBSCRIBE_END"); - expect([resp.end.group, resp.end.streams]).toEqual([1, 1]); + expect(resp.end.group).toBe(1); } finally { held.close(); } }); -// A group that never gets a stream owes the subscriber nothing, so it is not counted. -test("lite draft-07: a group whose stream never opened is not counted", async () => { +// A failed open does not prevent the declared boundary from reaching the subscriber. +test("lite draft-07: subscribe end precedes a failed stream open", async () => { const held = await heldOpenEnd(); try { held.open(false); const resp = await held.end; if (!("end" in resp)) throw new Error("expected SUBSCRIBE_END"); - expect([resp.end.group, resp.end.streams]).toEqual([1, 0]); + expect(resp.end.group).toBe(1); + const dropped = await held.next(); + if (!("drop" in dropped)) throw new Error("expected SUBSCRIBE_DROP"); + expect([dropped.drop.start, dropped.drop.end, dropped.drop.error]).toEqual([0, 0, 0]); } finally { held.close(); } }); +// A stream that fails after its FIN may have lost its header, so the group is still dropped. +test("lite draft-07: a group whose close fails is dropped", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const origin = new OriginProducer(); + const publisher = new Publisher(pair.server, Version.DRAFT_07, randomHop(), origin.consume()); + const broadcast = publish(origin, Path.from("test")); + const track = broadcast.createTrack("video", { timescale: Timescale.MILLI }); + + const createUni = pair.server.createUnidirectionalStream.bind(pair.server); + spyOn(pair.server, "createUnidirectionalStream").mockImplementation(async (options) => { + const writer = (await createUni(options)).getWriter(); + return new WritableStream({ + write: (chunk) => writer.write(chunk), + close: () => { + writer.abort(new Error("connection lost")).catch(() => void 0); + throw new Error("connection lost"); + }, + }); + }); + + const client = await Stream.open(pair.client, { version: Version.DRAFT_07 }); + const server = await Stream.accept(pair.server, Version.DRAFT_07); + if (!server) throw new Error("publisher never accepted the subscribe stream"); + void publisher.runSubscribe( + new Subscribe({ id: 0n, broadcast: Path.from("test"), track: "video", priority: 0 }), + server, + ); + + try { + const group = new GroupProducer(0); + group.writeFrame(textFrame("hello")); + group.close(); + track.writeGroup(group); + track.close(); + + const start = await decodeSubscribeResponse(client.reader, Version.DRAFT_07); + expect("start" in start).toBe(true); + const end = await decodeSubscribeResponse(client.reader, Version.DRAFT_07); + if (!("end" in end)) throw new Error("expected SUBSCRIBE_END"); + const dropped = await decodeSubscribeResponse(client.reader, Version.DRAFT_07); + if (!("drop" in dropped)) throw new Error("expected SUBSCRIBE_DROP"); + expect([dropped.drop.start, dropped.drop.end, dropped.drop.error]).toEqual([0, 0, 0]); + } finally { + publisher.close(); + client.close(); + } +}); + +test("blocked DROP responses stop opening later groups", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const origin = new OriginProducer(); + const publisher = new Publisher(pair.server, Version.DRAFT_07, randomHop(), origin.consume()); + const broadcast = publish(origin, Path.from("test")); + const track = broadcast.createTrack("video", { timescale: Timescale.MILLI }); + const client = await Stream.open(pair.client, { version: Version.DRAFT_07 }); + const server = await Stream.accept(pair.server, Version.DRAFT_07); + if (!server) throw new Error("missing subscribe stream"); + const opens = spyOn(pair.server, "createUnidirectionalStream").mockImplementation(async () => { + throw new Error("no stream credit"); + }); + let release!: () => void; + const gate = new Promise((resolve) => { + release = resolve; + }); + let blocked!: () => void; + const waiting = new Promise((resolve) => { + blocked = resolve; + }); + const write = server.writer.write.bind(server.writer); + let held = false; + const writes = spyOn(server.writer, "write").mockImplementation(async (chunk) => { + if (!held && chunk.length === 1 && chunk[0] === 2) { + held = true; + blocked(); + await gate; + } + await write(chunk); + }); + const append = (sequence: number) => { + const group = new GroupProducer(sequence); + group.writeFrame(textFrame("frame")); + group.close(); + track.writeGroup(group); + }; + for (let sequence = 0; sequence < 8; sequence++) append(sequence); + const serving = publisher.runSubscribe( + replaySubscribe({ id: 0n, broadcast: Path.from("test"), track: "video", priority: 0 }), + server, + ); + try { + expect("start" in (await decodeSubscribeResponse(client.reader, Version.DRAFT_07))).toBe(true); + await waiting; + expect(opens).toHaveBeenCalledTimes(8); + for (let sequence = 8; sequence < 72; sequence++) append(sequence); + // One executor turn drains pending signal notifications without advancing time. + await new Promise((resolve) => setImmediate(resolve)); + expect(opens).toHaveBeenCalledTimes(8); + release(); + track.close(); + let dropped = 0; + for (;;) { + const response = await decodeSubscribeResponseMaybe(client.reader, Version.DRAFT_07); + if (!response) break; + if ("drop" in response) dropped += response.drop.end - response.drop.start + 1; + } + expect(dropped).toBe(72); + expect(opens).toHaveBeenCalledTimes(72); + client.writer.close(); + await serving; + } finally { + release(); + writes.mockRestore(); + opens.mockRestore(); + publisher.close(); + client.close(); + broadcast.close(); + } +}); + /** One group stream the publisher put on the wire. */ type Served = { sequence: number; frameStart: number; payloads: string[] }; diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index f4ec22d7f59..2ad161e3eda 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -45,7 +45,6 @@ import { hasLargest, hasProbeRtt, hasRouteCost, - hasStreamCount, resolvesStart, Version, waitsForSubscriberFin, @@ -313,7 +312,11 @@ function newer(a: track.Location | undefined, b: track.Location | undefined): tr // Register both readiness sources in the same turn after the caller observed neither ready. // The winner disposes both registrations, so an idle subscription accumulates nothing no // matter how many times it wakes. -function waitForSubscription(controls: SubscriptionControls, subscriber: track.Subscriber): Promise { +function waitForSubscription( + controls: SubscriptionControls, + subscriber: track.Subscriber, + drops: Signal, +): Promise { return new Promise((resolve) => { let settled = false; const dispose: Dispose[] = []; @@ -324,6 +327,7 @@ function waitForSubscription(controls: SubscriptionControls, subscriber: track.S resolve(); }; dispose.push(controls.changed(wake), hooks.groupChanged(subscriber, wake)); + dispose.push(drops.changed(wake)); }); } @@ -764,6 +768,12 @@ export class Publisher { // subscription; awaited during teardown so it doesn't outlive the subscription. let datagrams = Promise.resolve(); let controls: SubscriptionControls | undefined; + // Lite-05 and later name every sequence below the end that this subscription never + // got with SUBSCRIBE_DROP, so the subscriber settles without waiting out its grace + // for a group that will not come. The sequences served (as a group or a datagram) or + // dropped. A gap that never arrived stays until the end sweep names it: this side + // cannot tell when the subscriber stopped waiting for it. Undefined on other versions. + const served: Run[] | undefined = supportsTrackStream(this.version) ? [] : undefined; try { // Older drafts carry no timestamps. @@ -797,7 +807,7 @@ export class Publisher { // Serve datagrams concurrently with groups whenever the transport carries them // (the writer exists iff so). No group fallback: otherwise they simply aren't sent. if (this.#datagramWriter) { - datagrams = this.#runDatagrams(msg.id, track, timescale); + datagrams = this.#runDatagrams(msg.id, track, timescale, served); } controls = new SubscriptionControls({ @@ -822,6 +832,7 @@ export class Publisher { sub: msg.id, broadcast: msg.broadcast, timescale, + served, bounds: { startGroup: msg.startGroup, startFrame: msg.startFrame, @@ -985,9 +996,15 @@ export class Publisher { track: track.Subscriber, stream: Writer, controls: SubscriptionControls, - serving: { sub: bigint; broadcast: Path.Valid; timescale: Timescale | undefined; bounds: FrameBounds }, + serving: { + sub: bigint; + broadcast: Path.Valid; + timescale: Timescale | undefined; + served: Run[] | undefined; + bounds: FrameBounds; + }, ): Promise { - const { sub, broadcast, timescale, bounds } = serving; + const { sub, broadcast, timescale, served, bounds } = serving; // Lite-05+ resolves the range on the subscribe stream: SUBSCRIBE_START once the // first group is known, SUBSCRIBE_END when the track finishes. const emitRange = supportsTrackStream(this.version); @@ -996,17 +1013,6 @@ export class Publisher { let startSequence: number | undefined; let endSent = false; - // Lite-07+ counts the group streams in SUBSCRIBE_END, so it goes out only once every - // served group has opened its stream or given up, and a cap holding groups back - // delays it until they are released. - const countStreams = hasStreamCount(this.version); - - // Lite-05 and lite-06 name every sequence below the end that this subscription never - // got with SUBSCRIBE_DROP, so the subscriber settles without waiting out its grace - // for a group that will not come. The sequences served or dropped. A gap that never - // arrived stays until the end sweep names it: this side cannot tell when the - // subscriber stopped waiting for it. Undefined on other versions. - const served: Run[] | undefined = emitRange && !countStreams ? [] : undefined; // Drop whatever part of `[start, end)` this subscription has not been served or dropped yet. const drop = async (start: number, end: number): Promise => { if (!served) return true; @@ -1018,8 +1024,6 @@ export class Publisher { } return true; }; - let streams = 0; - const opening = new Set>(); // The track's exclusive final boundary. A Rust subscriber feeds SUBSCRIBE_END // straight into finish_at, so it must name the track's boundary (which counts @@ -1033,19 +1037,12 @@ export class Publisher { return end ? finalSequence(end) : (hooks.latestSequence(track) ?? -1) + 1; }; - // Before lite-07, SUBSCRIBE_END names that boundary, which a cap can hold groups back - // from, so it goes out as soon as the producer finishes and the subscription keeps - // serving whatever a later cap raise releases (see the Rust publisher's Recv::Boundary). + // Declare the boundary immediately; groups below it can still be in flight. const sendEnd = async (): Promise => { endSent = true; if (!emitRange) return true; return controls.response( - (async () => { - // A group that gives up before its stream opens is never counted. - if (countStreams) while (opening.size > 0) await Promise.all(opening); - const end = new SubscribeEnd(boundary(), streams); - await encodeSubscribeResponse(stream, { end }, this.version); - })(), + encodeSubscribeResponse(stream, { end: new SubscribeEnd(boundary()) }, this.version), ); }; @@ -1054,6 +1051,8 @@ export class Publisher { // Every group this subscription started serving, until its stream finishes or resets. const groups = new Set>(); + const drops = new Signal(0); + const pendingDrops: SubscribeDrop[] = []; // Cancels groups still queued for a stream slot. Only the subscriber leaving counts: // a track that ran out of groups still has to flush the ones already queued, and the @@ -1065,6 +1064,14 @@ export class Publisher { }); try { for (;;) { + if (pendingDrops.length > 0) { + for (const dropped of pendingDrops.splice(0)) { + if ( + !(await controls.response(encodeSubscribeResponse(stream, { drop: dropped }, this.version))) + ) + return false; + } + } // Control before data, matching the Rust publisher: every control decoded while // the loop was parked applies to the next pop, never to one already made. This // drain is synchronous, so nothing the decoder holds can land between the pop @@ -1161,23 +1168,21 @@ export class Publisher { continue; } } - // Before lite-07, an end declared ahead of the live edge goes out as - // soon as it is known, while the remaining groups are still being - // produced. The lite-07 count is not final until those groups open. - if (!endSent && !countStreams && track.end() !== undefined) { + // A declared end goes out while lower groups are still being produced. + if (!endSent && track.end() !== undefined) { if (!(await sendEnd())) return false; continue; } - await waitForSubscription(controls, track); + await waitForSubscription(controls, track, drops); continue; case "boundary": // The producer finished but is still holding groups above the cap. // Declare the boundary, then wait for an update to release them. - if (!endSent && !countStreams) { + if (!endSent) { if (!(await sendEnd())) return false; continue; } - await waitForSubscription(controls, track); + await waitForSubscription(controls, track, drops); continue; case "done": { // Name every sequence from the start to the end this subscription never @@ -1198,6 +1203,7 @@ export class Publisher { const end = await Promise.race([Promise.all(groups).then(() => drained), controls.ended]); if (end instanceof Error) throw end; if (end !== drained) return false; + if (pendingDrops.length > 0) continue; finished = true; return true; } @@ -1242,19 +1248,17 @@ export class Publisher { start: range.start, end: range.end, }; - // `opening` settles when the stream opens so the lite-07 count can be sent. - // `groups` covers the serve too, so the FIN still waits for every stream to - // finish or reset, including one that has not opened yet. + // The FIN waits for every stream to finish or reset, including one that has not opened yet. const opened = this.#openGroup(options); const task = opened.then(async (writer) => { - if (!writer) return; - streams += 1; - await this.#serveGroup(writer, options); + const failed = writer instanceof Error ? writer : await this.#serveGroup(writer, options); + if (emitRange && failed instanceof Error) { + pendingDrops.push(new SubscribeDrop({ start: group.sequence, end: group.sequence, error: 0 })); + drops.update((revision) => revision + 1); + } }); groups.add(task); void task.finally(() => groups.delete(task)); - opening.add(opened); - void opened.finally(() => opening.delete(opened)); } } finally { if (!finished) unsubscribe(); @@ -1394,7 +1398,12 @@ export class Publisher { * * @internal */ - async #runDatagrams(sub: bigint, track: track.Subscriber, timescale: Timescale | undefined) { + async #runDatagrams( + sub: bigint, + track: track.Subscriber, + timescale: Timescale | undefined, + served: Run[] | undefined, + ) { const writer = this.#datagramWriter; if (!writer) return; // Only reached with a writer (see the #datagramWriter gate). const maxSize = DatagramStream.maxDatagramSize(this.#quic); @@ -1414,6 +1423,8 @@ export class Publisher { continue; } + // A sent datagram counts as delivered even if it is lost, so the end sweep skips it. + if (served) insertRun(served, datagram.sequence, datagram.sequence + 1); await writer.ready; await writer.write(body); } @@ -1457,11 +1468,11 @@ export class Publisher { /** * Opens the unidirectional stream for one group, or closes the group and resolves - * `undefined` when it cannot get one. + * an error when it cannot get one. * * @internal */ - async #openGroup(options: RunGroup): Promise { + async #openGroup(options: RunGroup): Promise { const { group, priority, unsubscribed, start } = options; try { // The transport drains streams by send order, so this is what makes a high-priority @@ -1477,11 +1488,16 @@ export class Publisher { cancel: unsubscribed, waitUntilAvailable: false, }); - if (!stream) group.close(new Error("no stream slot")); + if (!stream) { + const failed = new Error("no stream slot"); + group.close(failed); + return failed; + } return stream; } catch (err: unknown) { - group.close(error(err)); - return undefined; + const failed = error(err); + group.close(failed); + return failed; } } @@ -1547,11 +1563,13 @@ export class Publisher { stream.close(); group.close(); - await acknowledged(stream); + // A stream that fails after its FIN may have lost its header, so report it for a DROP. + return await stream.closed.then(() => undefined, error); } catch (err: unknown) { const e = error(err); stream.reset(e); group.close(e); + return e; } finally { priority.remove(stream); } diff --git a/js/net/src/lite/subscribe.test.ts b/js/net/src/lite/subscribe.test.ts index 16f77f46796..9c3c5a56385 100644 --- a/js/net/src/lite/subscribe.test.ts +++ b/js/net/src/lite/subscribe.test.ts @@ -190,30 +190,25 @@ test("SubscribeStart carries the largest position on draft-07", async () => { }); test("SubscribeEnd round-trips on draft-05", async () => { - // Type, length, group: no stream count before draft-07. - expect(await encode(Version.DRAFT_05, { end: new SubscribeEnd(7, 3) })).toEqual(new Uint8Array([1, 1, 7])); - const got = await responseRoundtrip(Version.DRAFT_05, { end: new SubscribeEnd(7, 3) }); + // Type, length, exclusive final group. + expect(await encode(Version.DRAFT_05, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 1, 7])); + const got = await responseRoundtrip(Version.DRAFT_05, { end: new SubscribeEnd(7) }); expect("end" in got).toBe(true); if (!("end" in got)) throw new Error("expected end"); - expect([got.end.group, got.end.streams]).toEqual([7, 0]); + expect(got.end.group).toBe(7); }); -test("SubscribeEnd carries the stream count on draft-07", async () => { - expect(await encode(Version.DRAFT_07, { end: new SubscribeEnd(7, 3) })).toEqual(new Uint8Array([1, 2, 7, 3])); - const got = await responseRoundtrip(Version.DRAFT_07, { end: new SubscribeEnd(7, 3) }); - if (!("end" in got)) throw new Error("expected end"); - expect([got.end.group, got.end.streams]).toEqual([7, 3]); +test("SubscribeEnd has no stream count on draft-07", async () => { + expect(await encode(Version.DRAFT_07, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 1, 7])); }); -test("SubscribeDrop is gone on draft-07", async () => { - const drop: SubscribeResponse = { drop: new SubscribeDrop({ start: 1, end: 3, error: 0 }) }; - await expect(encode(Version.DRAFT_07, drop)).rejects.toThrow(); - - // A draft-06 DROP is an unknown response type on draft-07. - const wire06 = await encode(Version.DRAFT_06, drop); - await expect( - decodeSubscribeResponse(new Reader(undefined, wire06, Version.DRAFT_07), Version.DRAFT_07), - ).rejects.toThrow("unknown subscribe response type: 2"); +test("SubscribeDrop is restored on draft-07", async () => { + const drop: SubscribeResponse = { drop: new SubscribeDrop({ start: 1, end: 3, error: 7 }) }; + const wire = await encode(Version.DRAFT_07, drop); + expect(wire).toEqual(new Uint8Array([2, 3, 1, 3, 7])); + const got = await responseRoundtrip(Version.DRAFT_07, drop); + if (!("drop" in got)) throw new Error("expected drop"); + expect([got.drop.start, got.drop.end, got.drop.error]).toEqual([1, 3, 7]); }); test("SubscribeDrop is type 0x2 on draft-05 and 0x1 on draft-04", async () => { diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index f9be30b4785..d04671f960a 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -5,15 +5,7 @@ import type { Reader, Writer } from "../stream.ts"; import type { Location, Order } from "../track.ts"; import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; -import { - hasFrameBounds, - hasGroupOrder, - hasLargest, - hasOrder, - hasStreamCount, - resolvesStart, - Version, -} from "./version.ts"; +import { hasFrameBounds, hasGroupOrder, hasLargest, hasOrder, resolvesStart, Version } from "./version.ts"; /** * Encode the `Group Start` field shared by SUBSCRIBE and SUBSCRIBE_UPDATE. @@ -540,35 +532,24 @@ export class SubscribeEnd { /** The exclusive final group sequence: the first sequence that will never be produced. */ group: number; - /** - * The number of group streams the publisher opened for this subscription. - * Draft-07+ only; not on the wire before, where it decodes as 0. - */ - streams: number; - - constructor(group: number, streams = 0) { + constructor(group: number) { this.group = group; - this.streams = streams; } - async encode(w: Writer, version: Version): Promise { + async encode(w: Writer, _version: Version): Promise { return Message.encode(w, async (w) => { await w.u53(this.group); - if (hasStreamCount(version)) await w.u53(this.streams); }); } - static async decode(r: Reader, version: Version): Promise { - return Message.decode( - r, - async (r) => new SubscribeEnd(await r.u53(), hasStreamCount(version) ? await r.u53() : 0), - ); + static async decode(r: Reader, _version: Version): Promise { + return Message.decode(r, async (r) => new SubscribeEnd(await r.u53())); } } /// Indicates that one or more groups have been dropped. /// -/// Draft-03 to Draft-06 only: Draft-07 counts group streams in SUBSCRIBE_END instead. +/// Draft-03+; publishers send it on Draft-05 and later. export class SubscribeDrop { start: number; end: number; @@ -605,9 +586,8 @@ export class SubscribeDrop { * * The discriminator is version-dependent: * - Draft-03/04: `0x0` SUBSCRIBE_OK, `0x1` SUBSCRIBE_DROP. - * - Draft-05/06: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END, `0x2` SUBSCRIBE_DROP + * - Draft-05+: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END, `0x2` SUBSCRIBE_DROP * (SUBSCRIBE_OK was removed; acceptance is implicit). - * - Draft-07+: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END (SUBSCRIBE_DROP was removed). */ export type SubscribeResponse = | { ok: SubscribeOk } @@ -645,7 +625,7 @@ export async function encodeSubscribeResponse(w: Writer, resp: SubscribeResponse } else if ("end" in resp) { await w.u53(0x1); await resp.end.encode(w, version); - } else if ("drop" in resp && !hasStreamCount(version)) { + } else if ("drop" in resp) { await w.u53(0x2); await resp.drop.encode(w); } else { @@ -680,7 +660,6 @@ export async function decodeSubscribeResponse(r: Reader, version: Version): Prom case 0x1: return { end: await SubscribeEnd.decode(r, version) }; case 0x2: - if (hasStreamCount(version)) throw new ProtocolViolation(`unknown subscribe response type: ${typ}`); return { drop: await SubscribeDrop.decode(r) }; default: throw new ProtocolViolation(`unknown subscribe response type: ${typ}`); diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 0dfd4519dcc..c98388f23ea 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -1851,7 +1851,7 @@ test("an update whose bound the wire can't carry fails the track", async () => { } }); -test("a retired subscribe response type closes the session", async () => { +test("an unknown subscribe response type closes the session", async () => { const { quic, streams, opened, closes } = fakeSession(); const subscriber = new Subscriber(quic, Version.DRAFT_07, HopSchema.parse(1n)); const broadcast = subscriber.consume(Path.from("room")); @@ -1860,8 +1860,8 @@ test("a retired subscribe response type closes the session", async () => { await opened(0); await answerTrackInfo(streams[0], Version.DRAFT_07); await (await opened(1)).reading; - // lite-07 retired SUBSCRIBE_DROP (0x2), so it is no longer a response type. - streams[1].inbound.enqueue(new Uint8Array([2, 0])); + // An unassigned response type is a session protocol violation. + streams[1].inbound.enqueue(new Uint8Array([3, 0])); expect(await track.closed).toBeInstanceOf(ProtocolViolation); await drainReactions(); expect(closes.map((info) => info?.closeCode)).toEqual([SessionCode.ProtocolViolation]); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 08732d797f2..6c45dd89064 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -56,7 +56,6 @@ import { hasFetchMaxDelay, hasOrder, hasProbeRtt, - hasStreamCount, resolvesStart, updateSupported, Version, @@ -126,8 +125,6 @@ interface SubscribeEntry { // (SUBSCRIBE_END), once it declares them. start?: number; end?: number; - // Group streams opened by the publisher, when SUBSCRIBE_END carries the count. - streams?: number; // The TRACK stream that answered (lite-05+), held open as interest in the track until // the SUBSCRIBE has its first response, so the publisher's demand never lapses between // the two. @@ -1213,7 +1210,6 @@ export class Subscriber { } else if ("end" in resp) { if (entry.end !== undefined) throw new ProtocolViolation("duplicate SUBSCRIBE_END"); entry.end = resp.end.group; - if (hasStreamCount(this.version)) entry.streams = resp.end.streams; // A local close can win the race with the response; there is nothing left to end. if (entry.track.closed.peek() !== undefined) continue; try { @@ -1238,15 +1234,13 @@ export class Subscriber { // Wait for the group streams the publisher still owes once it has ended the subscription. // - // lite-07 counts streams, so skipped sequences owe nothing. Older drafts account for - // the range using received headers and SUBSCRIBE_DROP. A counted stream reset before - // its header leaves no trace, so the grace still bounds that wait. Streams whose + // Received headers and SUBSCRIBE_DROP account for the range. An unaccounted + // sequence, such as a lost datagram, still waits out the grace. Streams whose // headers arrived keep reading until their own FIN or reset. #settleTail(entry: SubscribeEntry): Promise { const { tail, track } = entry; const complete = () => { - if (entry.streams !== undefined) return tail.streams >= entry.streams; // Without SUBSCRIBE_END (older drafts) nothing says which groups are owed. if (entry.end === undefined) return false; // Without SUBSCRIBE_START the publisher served no group at all. diff --git a/js/net/src/lite/tail.test.ts b/js/net/src/lite/tail.test.ts index 23bd18884ab..7c5fd92b2fe 100644 --- a/js/net/src/lite/tail.test.ts +++ b/js/net/src/lite/tail.test.ts @@ -114,7 +114,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const first = groupStream(subscriber, 0); first.write("0.0"); first.finish(); - await respond({ end: new SubscribeEnd(2, 2) }); + await respond({ end: new SubscribeEnd(2) }); await fin(); // The end is known before the last group arrives. @@ -138,7 +138,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const group = groupStream(subscriber, 0); group.write("0.0"); - await respond({ end: new SubscribeEnd(1, 1) }); + await respond({ end: new SubscribeEnd(1) }); await fin(); const received = await reader.recvGroup(); expect(await received?.readString()).toBe("0.0"); @@ -156,7 +156,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver await respond({ start: new SubscribeStart(0) }); const group = groupStream(subscriber, 0); group.write("0.0"); - await respond({ end: new SubscribeEnd(1, 1) }); + await respond({ end: new SubscribeEnd(1) }); await fin(); const received = await reader.recvGroup(); @@ -196,7 +196,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver group.write("1.0"); group.finish(); await group.handled; - await respond({ end: new SubscribeEnd(2, 2) }); + await respond({ end: new SubscribeEnd(2) }); await fin(); await graceArmed; @@ -234,7 +234,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const high = groupStream(subscriber, 1); high.finish(); await high.handled; - await respond({ end: new SubscribeEnd(2, 2) }); + await respond({ end: new SubscribeEnd(2) }); expect(await reader.finished()).toEqual({ group: 2, frame: 0 }); expect((await reader.recvGroup())?.sequence).toBe(1); @@ -255,24 +255,21 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver subscriber.close(); }); - test.skipIf(version === Version.DRAFT_07)( - "a subscription ends without waiting once every group is accounted for", - async () => { - // A max delay far past the test's patience: only the accounting may end it. - const { subscriber, reader, respond, fin } = await subscribed(version, Milli(60_000)); - await respond({ start: new SubscribeStart(0) }); - await respond({ drop: new SubscribeDrop({ start: 0, end: 0, error: 0 }) }); - const group = groupStream(subscriber, 1); - group.write("1.0"); - group.finish(); - await respond({ end: new SubscribeEnd(2, 2) }); - await fin(); + test("a subscription ends without waiting once every group is accounted for", async () => { + // A max delay far past the test's patience: only the accounting may end it. + const { subscriber, reader, respond, fin } = await subscribed(version, Milli(60_000)); + await respond({ start: new SubscribeStart(0) }); + await respond({ drop: new SubscribeDrop({ start: 0, end: 0, error: 0 }) }); + const group = groupStream(subscriber, 1); + group.write("1.0"); + group.finish(); + await respond({ end: new SubscribeEnd(2) }); + await fin(); - expect((await reader.recvGroup())?.sequence).toBe(1); - expect(await settlesWithin(reader.recvGroup(), 1000)).toBe(true); - expect(await reader.closed).toBeNull(); - }, - ); + expect((await reader.recvGroup())?.sequence).toBe(1); + expect(await settlesWithin(reader.recvGroup(), 1000)).toBe(true); + expect(await reader.closed).toBeNull(); + }); test("a subscription that served nothing ends at SUBSCRIBE_END", async () => { const { reader, respond, fin } = await subscribed(version, Milli(60_000)); @@ -288,7 +285,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver test(`a group at or past SUBSCRIBE_END ${version === Version.DRAFT_05 ? "is dropped" : "aborts the track"}`, async () => { const { subscriber, reader, respond } = await subscribed(version); await respond({ start: new SubscribeStart(0) }); - await respond({ end: new SubscribeEnd(2, 2) }); + await respond({ end: new SubscribeEnd(2) }); expect(await reader.finished()).toEqual({ group: 2, frame: 0 }); await groupStream(subscriber, 2).handled; @@ -307,7 +304,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const group = groupStream(subscriber, 3); group.finish(); await group.handled; - await respond({ end: new SubscribeEnd(4, 1) }); + await respond({ end: new SubscribeEnd(4) }); await fin(); expect((await reader.recvGroup())?.sequence).toBe(3); @@ -315,7 +312,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver expect(await reader.closed).toBeNull(); }); - test.skipIf(version === Version.DRAFT_07)("a lowered floor owes the groups it newly asked for", async () => { + test("a lowered floor owes the groups it newly asked for", async () => { const maxDelay = Milli(60_000); const { subscriber, respond, fin } = await subscribed(version, maxDelay, { start: { included: 3 } }); await respond({ start: new SubscribeStart(3) }); @@ -329,7 +326,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver .consume(Path.from("room")) .track("video") .subscribe({ maxDelay, groups: { start: { included: 1 } } }); - await respond({ end: new SubscribeEnd(4, 1) }); + await respond({ end: new SubscribeEnd(4) }); await fin(); // Long enough for a subscription that owed nothing more to have ended. await new Promise((resolve) => setTimeout(resolve, 50)); @@ -361,7 +358,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver const group = groupStream(subscriber, 3); group.finish(); await group.handled; - await respond({ end: new SubscribeEnd(2, 2) }); + await respond({ end: new SubscribeEnd(2) }); if (version === Version.DRAFT_05) { await fin(); @@ -372,7 +369,7 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver }); }); -test("lite-07 ends without grace when every counted stream arrived despite a skipped group", async () => { +test("lite-07 settles a skipped group with SUBSCRIBE_DROP", async () => { const { subscriber, reader, respond, fin } = await subscribed(Version.DRAFT_07, Milli(60_000)); await respond({ start: new SubscribeStart(0) }); for (const sequence of [0, 2]) { @@ -381,7 +378,8 @@ test("lite-07 ends without grace when every counted stream arrived despite a ski group.finish(); await group.handled; } - await respond({ end: new SubscribeEnd(3, 2) }); + await respond({ end: new SubscribeEnd(3) }); + await respond({ drop: new SubscribeDrop({ start: 1, end: 1, error: 0 }) }); await fin(); expect((await reader.recvGroup())?.sequence).toBe(0); @@ -390,10 +388,11 @@ test("lite-07 ends without grace when every counted stream arrived despite a ski expect(await reader.closed).toBeNull(); }); -test("lite-07 zero count ends without grace even when the announced range is nonempty", async () => { +test("lite-07 settles a dropped range without grace", async () => { const { reader, respond, fin } = await subscribed(Version.DRAFT_07, Milli(60_000)); await respond({ start: new SubscribeStart(0) }); - await respond({ end: new SubscribeEnd(3, 0) }); + await respond({ end: new SubscribeEnd(3) }); + await respond({ drop: new SubscribeDrop({ start: 0, end: 2, error: 0 }) }); await fin(); expect(await settlesWithin(reader.recvGroup(), 1000)).toBe(true); @@ -428,7 +427,8 @@ test("lite-07 FIN waits for the skipped and reset final range to settle", async await first.handled; const last = groupStream(subscriber, 2); last.write("tail"); - await respond({ end: new SubscribeEnd(3, 2) }); + await respond({ end: new SubscribeEnd(3) }); + await respond({ drop: new SubscribeDrop({ start: 1, end: 1, error: 0 }) }); await fin(); let finished = false; const ack = subscriberFin().then(() => { diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index f70c8e5c624..a101718c189 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -284,25 +284,6 @@ export function hasHidden(version: Version): boolean { } } -/** - * Whether SUBSCRIBE_END carries the subscription's group stream count, sent once every - * counted stream is open, in place of SUBSCRIBE_DROP. Added in lite-07. - */ -export function hasStreamCount(version: Version): boolean { - // Explicitly list older versions so future versions keep the lite-07+ behavior. - switch (version) { - case Version.DRAFT_01: - case Version.DRAFT_02: - case Version.DRAFT_03: - case Version.DRAFT_04: - case Version.DRAFT_05: - case Version.DRAFT_06: - return false; - default: - return true; - } -} - /** * Whether a served SUBSCRIBE completes only once the subscriber FINs or resets its half of the * Subscribe Stream. Added in lite-07, where a subscriber FINs once its tail accounting settles, diff --git a/quest/m1/README.md b/quest/m1/README.md index 383519dbdcc..192327a9510 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -63,7 +63,7 @@ lands before the cut. - [Compressed tracks](/quest/m1/compress/README.md) - a track flagged `compress` is DEFLATEd per group by moq-net, forwarded compressed, and inflated once on the relay for peers that cannot signal it; `catalog.json.z` and moq-flate go away - [Moxygen minimal interop](/quest/m1/smoke-moxygen.md) - waiting on Moxygen to send a registered filter type, then the smoke lane stops hiding its failure - [Claim answer epochs](/quest/m1/claim-answer-epochs.md) - a claim answers each output by publishing it at its exact path under its own epoch, so a restarted output never reaches a returning reader as the old one's cache -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - a dropped or cut group ends its reader with an error, never a clean end, across cache reclaim, cap changes, and recovery - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Announce SYNCED](/quest/m1/announce-synced.md) - lite-07's Announce stream marks its initial set with the same ordered SYNCED as TRACKS, replacing `Active Count` - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause diff --git a/quest/m1/close-tail-drain.md b/quest/m1/close-tail-drain.md index 95dfe3fba26..d9762cfa8e0 100644 --- a/quest/m1/close-tail-drain.md +++ b/quest/m1/close-tail-drain.md @@ -78,5 +78,5 @@ Public API: none. Wire: none. ## Related -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - groups cut short by a session close +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - groups cut short by a session close - [QUIC ack hook](/quest/m2/quic-ack-hook.md) - a FIN/ack hook in the QUIC layer diff --git a/quest/m1/drop-set.md b/quest/m1/drop-set.md index 5c14778d898..a8263306a1a 100644 --- a/quest/m1/drop-set.md +++ b/quest/m1/drop-set.md @@ -110,10 +110,9 @@ which keeps drops final for now. - **Draft wording.** `drafts/draft-lcurley-moq-lite.md` states, for lite-07's restored SUBSCRIBE_DROP, that a drop holds only under the terms in effect when it was sent, that a widening re-offers held groups and re-drops the - rest, and that a later group, when sent, wins over its drop. Whichever of this and - [SUBSCRIBE_DROP accounts for every group](/quest/m1/subscribe-drop.md) - lands second writes it, including the `Max Age` field above, which is - free to add because lite-07 is unpublished. + rest, and that a later group, when sent, wins over its drop. This quest + writes it, including the `Max Age` field above, which is free to add + because lite-07 is unpublished. Code it replaces, in Rust and `js/net` alike: @@ -217,9 +216,9 @@ unchanged. ## Required - [A widened max age re-offers cached groups](/quest/m1/resolved-demand.md) - the re-offer path this extends to reported groups -- [SUBSCRIBE_DROP accounts for every group](/quest/m1/subscribe-drop.md) - restores the lite-07 drop whose wire and behavior this extends ## Related +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - settles what the same message's Error Code means on lite-07 - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - fills the cache miss an evicted run leaves - [JS unbounded groups](/quest/m2/unbounded-groups/js.md) - the JS byte bound for groups and the runs between them; not required, since JS runs never outgrow the JS cache diff --git a/quest/m1/egress-held-state.md b/quest/m1/egress-held-state.md index d77f3b60928..d8ec6f65399 100644 --- a/quest/m1/egress-held-state.md +++ b/quest/m1/egress-held-state.md @@ -68,7 +68,7 @@ Recommended directions (the implementer settles the shape): decide who gets the next stream. - A parked group that can no longer be served (aborted, evicted, expired) leaves at once and is named to the subscriber where the version can, through - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md). + SUBSCRIBE_DROP. - The serving budget never exceeds what this hop would retain anyway: clamp the requested or defaulted max delay by the local retention (the `cache_duration` ceiling, the track's `max_age`, and possibly the pool's idle @@ -113,7 +113,6 @@ none on the wire; possibly a relay default). - [Bound incomplete messages](/quest/m1/quic/receive-memory.md) - the ingress counterpart: a receiver withholds QUIC credit to bound what it holds, and this quest bounds what the sender holds while it does - [Rust request credit](/quest/m1/rs-request-credit.md) - requests fail fast without stream credit; publisher group streams are left to this quest -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - how a publisher names a group it gave up on - [Scope track priority](/quest/m1/track-priority-scope.md) - who gets the next stream across subscriptions and tenants - [Public serving](/quest/m1/public-model/serve.md) - makes group expiry while serving public; coordinate the publisher serve loops with it - [Treat network input as untrusted](https://github.com/moq-dev/moq/pull/5961) - the rule this quest applies diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 6e3cca9d4f0..9def8517eeb 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -84,7 +84,7 @@ is published; older versions are unchanged. - [In-band auth](/quest/m1/auth/README.md) - lite-07 carries the Auth Stream and UNAUTHORIZED (0x3B) - [Identity proof](/quest/m1/cluster-routing/identity.md) - lite-07 carries the routing key and its per-session proof - [Routes and announces](/quest/m1/cluster-routing/routes.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the hop list gone -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - lite-07 restores SUBSCRIBE_DROP in place of `Stream Count` +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - settles what lite-07's DROP Error Code means - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - SUBSCRIBE carries ranges and an order and lite FETCH is gone, in Rust and JS - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route - [Refresh on the lite wire](/quest/m1/refresh/lite.md) - TRACK_INFO carries the refresh ratio and SUBSCRIBE_UPDATE the wanted group diff --git a/quest/m1/quic/group-reset.md b/quest/m1/quic/group-reset.md index 2754f19bdea..f24cb454b9d 100644 --- a/quest/m1/quic/group-reset.md +++ b/quest/m1/quic/group-reset.md @@ -56,8 +56,5 @@ half did not reach. ## Related -- [SUBSCRIBE_DROP accounts for every group](/quest/m1/subscribe-drop.md) - a - DROP covers a reset whose header was lost; a reset that keeps its header acts - as a one-group drop - [qmux on the QUIC stream state machine](/quest/m2/quic-qmux.md) - qmux reuses `moq-quic`'s reset state diff --git a/quest/m1/resume-reorder.md b/quest/m1/resume-reorder.md index 4603af6efd4..1ef8fd62219 100644 --- a/quest/m1/resume-reorder.md +++ b/quest/m1/resume-reorder.md @@ -30,4 +30,4 @@ Public API: behavior only. Wire: none. ## Related -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - explicit terminal dispositions end recovery without waiting for a newer group +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - explicit terminal dispositions end recovery without waiting for a newer group diff --git a/quest/m1/subscribe-drop-errors.md b/quest/m1/subscribe-drop-errors.md new file mode 100644 index 00000000000..4fbcd9449b4 --- /dev/null +++ b/quest/m1/subscribe-drop-errors.md @@ -0,0 +1,73 @@ +# [L] Readers see SUBSCRIBE_DROP + +## Goal + +A group the publisher drops or cuts ends its reader with an error, never a +clean end or a silent skip. On both Rust and `@moq/net`, a lite SUBSCRIBE_DROP +settles the dropped group's reader as soon as it arrives, before the track's +FIN, and a cut stays visible across cache reclaim, cap changes, and route +recovery. Datagram groups stay best effort. + +## Plan + +The wire already accounts for every stream group: lite-05 and later name each +sequence a publisher gives up with SUBSCRIBE_DROP, and lite-07 restored it in +place of `Stream Count` (#5983). Today that accounting only settles the +subscription tail; it is not a terminal disposition for live readers or +recovery. + +Open, decide first: what `SubscribeDrop.error` means to a reader. +lite-05/06 define it as an application-specific code, with 0 meaning +unavailable without an error, so those values cannot be silently +reinterpreted. + +1. (recommended) lite-07 carries the stream-reset error codes, so a DROP ends + the reader with the same error a reset keeping its header would. + lite-05/06 keep the application-specific meaning, surfaced as one fixed + error. Changes lite-07 semantics, so settle it before + [Finalize moq-lite-07](/quest/m1/lite07-finalize.md). +2. The field stays application-specific on every version, and every DROP ends + the reader with one fixed error carrying the code. + +Decided: + +- A resumed group that is the new copy's latest ends with the DROP's error + when the copy drops it. Audit clarification (2026-10-10): test this while + the subscription is still open, with no successor and no later writes. + Both Rust and JS must settle the group wait on DROP, not only mark the + tail accounted at SUBSCRIBE_END. A pending recovery fetch cannot hide + that disposition or restart an unbounded route-retry loop. +- A dropped or aborted group is visible to readers, not silently skipped. + #4533 found the Rust model releases an aborted group's sequence and skips it, + so a truncated first object is indistinguishable from a group never sent. + +Learned from the shelved [#4998](https://github.com/moq-dev/moq/pull/4998) +(maintainer, 2026-10-07; its approach is on `wip/4998-cut-siblings`): + +- A group a stream cancel or session close cut below a finished end must be + named (a SUBSCRIBE_DROP) and surfaced to readers, never a clean end. +- Carry the cut per group. A publisher that resets the whole subscription for + one cut group makes the downstream relay resubscribe in a loop. +- The front's resume layer (`model/resume.rs`) treats any error from a copy as + a dead route and waits for a replacement, so a reader error for a cut hangs a + relay. Surface a cut on a complete copy to the reader instead. +- Cut tracking must survive the cache reclaiming the aborted slot (expiry, + eviction, and the pool's drain sweep), and the ordered reader's + sealed/closed check must not return a clean end before looking for a cut. +- Fetched backfill that is abandoned, and a `DeliveryTimeout` reset, are + deliberate holes, not cuts, like old, evicted, and lagged groups. +- A cut above a reader's group cap is skipped for good, so a reader whose cap + `set_groups` later raises over it must still see the cut, not a clean end. +- moq-tokio's + `subscription_end_integrity::a_subscription_cut_by_the_publisher_disconnecting_does_not_end_clean` + flakes under load (`Ok(None)` with 10 of 20 frames, #4332): the code can + still end it clean until this lands, and it should pass reliably after. + +Regression tests: an early DROP settles the dropped group for its reader +before the track's FIN, on both stacks, including a resumed latest group. +Update `drafts/draft-lcurley-moq-lite.md` if option 1 changes what Error Code +means on lite-07, then run `just drafts check` and `just test interop --all`. + +## Related + +- [Resume reorder](/quest/m1/resume-reorder.md) - late stream headers are not proof that a group was dropped diff --git a/quest/m1/subscribe-drop.md b/quest/m1/subscribe-drop.md deleted file mode 100644 index e7d8b0fe76f..00000000000 --- a/quest/m1/subscribe-drop.md +++ /dev/null @@ -1,100 +0,0 @@ -# [L] SUBSCRIBE_DROP accounts for every group - -## Goal - -A lite subscriber can tell "not yet" from "never" for every stream group in -its subscription. Each sequence in range either arrives on a Group Stream, is -sent as a datagram, or is named by a SUBSCRIBE_DROP, including sequences the -publisher skipped. Only a lost datagram stays unaccounted. Rust and -`@moq/net` publishers send it on every lite version that has it, and moq-lite-07 -brings it back in place of `Stream Count`. - -## Plan - -Today SUBSCRIBE_DROP is defined on the wire for lite-03 through lite-06. -The Rust and `@moq/net` lite-05/06 response paths account for it in the -subscription tail. That accounting is not a terminal group disposition for -live readers or recovery; the JS lite-03/04 path does not run that response -loop. Once a track ends, the Rust -publisher on lite-05 and lite-06 names every sequence from its -SUBSCRIBE_START to the end that the subscription never got (skipped, -stale, or missing its head), just before its FIN. Still missing: groups -reset before their header. Rust and JS publishers now send drops as soon as a -group is given up, including gaps the publisher aged out of its grace. -lite-07 (still `moq-lite-07-wip`, -unpublished) removed SUBSCRIBE_DROP for a `Stream Count` on SUBSCRIBE_END -(#4224). - -Decided: - -- lite-07 restores SUBSCRIBE_DROP and removes `Stream Count`. With every - sequence accounted for, the count is redundant; one mechanism instead of two. - A reset whose header may be lost is covered by a DROP. -- A reliable reset ([A reset group keeps its header](/quest/m1/quic/group-reset.md)) - that keeps the stream header acts as a one-group drop, an optimization over - sending the DROP. -- Publishers send SUBSCRIBE_DROP on lite-05 and lite-06 too: for every group - in range they won't deliver (expired, deprioritized, or reset without its - header delivered) and for every explicit gap. Publishers that skip sequences - (`cut` and group discontinuities in the media layers) must mark the gap so - the net layer can drop it. -- Not on lite-03 or lite-04 (maintainer, 2026-10-09): they declare no - SUBSCRIBE_END, so no subscriber has an owed range to settle against, and a - drop would only add traffic to legacy peers. -- Datagram groups stay best effort. A publisher counts a datagram as - delivered, so a lost one leaves an uncovered hole that waits out the tail - grace, as today. -- A resumed group that is the new copy's latest ends with the DROP's error - when the copy drops it. Audit clarification (2026-10-10): test this while - the subscription is still open, with no successor and no later writes. - Both Rust and JS must settle the group wait on DROP, not only mark the - tail accounted at SUBSCRIBE_END. A pending recovery fetch cannot hide - that disposition or restart an unbounded route-retry loop. -- A dropped or aborted group is visible to readers, not silently skipped. - #4533 found the Rust model releases an aborted group's sequence and skips it, - so a truncated first object is indistinguishable from a group never sent. - -Learned from the shelved [#4998](https://github.com/moq-dev/moq/pull/4998) -(maintainer, 2026-10-07; its approach is on `wip/4998-cut-siblings`): - -- A group a stream cancel or session close cut below a finished end must be - named (a SUBSCRIBE_DROP) and surfaced to readers, never a clean end. -- Carry the cut per group. A publisher that resets the whole subscription for - one cut group makes the downstream relay resubscribe in a loop. -- The front's resume layer (`model/resume.rs`) treats any error from a copy as - a dead route and waits for a replacement, so a reader error for a cut hangs a - relay. Surface a cut on a complete copy to the reader instead. -- Cut tracking must survive the cache reclaiming the aborted slot (expiry, - eviction, and the pool's drain sweep), and the ordered reader's - sealed/closed check must not return a clean end before looking for a cut. -- Fetched backfill that is abandoned, and a `DeliveryTimeout` reset, are - deliberate holes, not cuts, like old, evicted, and lagged groups. -- A cut above a reader's group cap is skipped for good, so a reader whose cap - `set_groups` later raises over it must still see the cut, not a clean end. -- moq-tokio's - `subscription_end_integrity::a_subscription_cut_by_the_publisher_disconnecting_does_not_end_clean` - flakes under load (`Ok(None)` with 10 of 20 frames, #4332): the code can - still end it clean until this lands, and it should pass reliably after. - -Update `drafts/draft-lcurley-moq-lite.md` (SUBSCRIBE_DROP, including that a -publisher SHOULD send it as soon as it gives a sequence up; SUBSCRIBE_END; the -lite-07 changelog), `doc/concept/moq-lite.md`, and the Rust and JS lite -publishers, subscribers, and tail accounting. Run `just drafts check` and -`just test interop --all`. - -Regression tests: a publisher that expires a group or resets a stream before -its header; on each version the subscriber settles without waiting out the -grace. An early DROP settles the dropped group for its reader before the -track's FIN, on both stacks. A skipped sequence at the end is covered on lite-05 through lite-07 -(`track_tail::skipped_groups_end_without_the_grace`). - -Add the lite-07 drop case to the tail interop harness -(`just test interop --tail`, `test/interop/clients/*/tail.*`): Rust and JS -subscribers both settle on SUBSCRIBE_DROP through the relay, so a group the -publisher skipped or never opened ends the track without waiting out the -grace. Decided in the 2026-09-30 audit: the case moved here so the basic -tail interop could land first. - -## Related - -- [Resume reorder](/quest/m1/resume-reorder.md) - late stream headers are not proof that a group was dropped diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 6897d975607..7a74ce43995 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -28,8 +28,7 @@ relay bounds a sparse FETCH only if the model can request ranges: not accidentally give it the live SUBSCRIBE zero-delay default. SUBSCRIBE_UPDATE replaces the list. Lite FETCH is removed from lite-07; published versions keep it, answered through the same model. -- **Holes.** SUBSCRIBE_DROP (restored in lite-07 by - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md)) names every sequence in the +- **Holes.** SUBSCRIBE_DROP (restored in lite-07) names every sequence in the requested ranges the publisher won't deliver: never existed, expired, or refused upstream. Datagram groups are always dropped in a past range (#4982). The publisher delivers what it has and fills the rest as upstream answers. @@ -87,7 +86,6 @@ every sequence delivered or dropped. ## Required - [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, a subscriber reads only its own ranges, and a range end may be a time -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity diff --git a/quest/m1/subscribe-ranges/dynamic.md b/quest/m1/subscribe-ranges/dynamic.md index ad5fd0d5672..ba6dfd30fda 100644 --- a/quest/m1/subscribe-ranges/dynamic.md +++ b/quest/m1/subscribe-ranges/dynamic.md @@ -34,7 +34,7 @@ cache becomes a range request on `Dynamic` rather than a group request. `moq_relay::fetch_group` (added in the 2026-10-05 audit). - Every sequence in range either arrives or is named as dropped, so a reader never waits on a miss. Share the accounting with - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) rather than adding a second one. + SUBSCRIBE_DROP rather than adding a second one. - Decided in the 2026-10-10 audit: `max_delay` caps every range, including historical FETCH replacements, and publisher `max_age` always clamps it. A FETCH with no reader delay uses publisher max-age alone, not the live @@ -54,7 +54,3 @@ cache becomes a range request on `Dynamic` rather than a group request. the older groups the range asked for. - Benchmark serving and filling over range count and span as separate axes, beside `track_subscription_ranges`, which covers aggregation. - -## Related - -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver diff --git a/quest/m1/subscribe-ranges/lite.md b/quest/m1/subscribe-ranges/lite.md index 6aa5ae0c793..ea10d101460 100644 --- a/quest/m1/subscribe-ranges/lite.md +++ b/quest/m1/subscribe-ranges/lite.md @@ -56,4 +56,3 @@ FETCH, answered through the model's ranges. Run `just drafts check` and ## Required - [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - the misses a relay fills upstream by range -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - lite-07's DROP for holes diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index e9c11d41c55..1241652fb1d 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -29,7 +29,7 @@ merge batch, ranked by how likely each hides a product bug. This README's own work, after the children: run `just check --all` several times on a loaded machine, as the first round did. moq-tokio's `a_subscription_cut_by_the_publisher_disconnecting_does_not_end_clean` is a -known exception, tracked by [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md). +known exception, tracked by [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md). Added 2026-10-19 (maintainer): three more, the redial pair first since CI hit it. diff --git a/quest/m2/ordered-owed-wait.md b/quest/m2/ordered-owed-wait.md index d7a1798ce80..e4727de4ea3 100644 --- a/quest/m2/ordered-owed-wait.md +++ b/quest/m2/ordered-owed-wait.md @@ -22,7 +22,7 @@ sequence until the budget elapsed. The model cannot tell "not yet" from ✅ Decided 2026-10-14 (maintainer): - Wait only on sequences the track still owes, read from the dispositions - that [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) and the track's + that [SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) and the track's arrival record (`TrackState::arrivals`) keep. Not chosen: an opt-in waiting reader (new public API no consumer needs, and opted-in readers hit the same holes); fixing #5475 forward (discontinuity markers still stall, @@ -70,7 +70,7 @@ Public API: behavior only. Wire: none. ## Required -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a subscription arrives or is named dropped, including publisher skips +- [Readers see SUBSCRIBE_DROP](/quest/m1/subscribe-drop-errors.md) - a dropped group is a terminal disposition the model can read ## Related diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index d6f35a95dc5..d2611b9b980 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Lite-07 accounts for groups with `SUBSCRIBE_DROP` + +`lite::SubscribeEnd` now contains only `group`; remove `streams` from struct literals and stop calling `Version::has_stream_count()`. +The unpublished `moq-lite-07-wip` wire restores `SUBSCRIBE_DROP` and removes Stream Count from `SUBSCRIBE_END`. Update both ends together; published lite versions keep their framing. + ### Track demand reports the wanted group (#5930) `track::Demand::used`, `unused`, `is_used`, `poll_used`, and `poll_unused` are gone. diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index adc4197fbe8..0e1ba4f002c 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -1,3 +1,6 @@ +#[cfg(test)] +use std::sync::atomic::AtomicU64; + use crate::{SessionError, announce, frame, group, origin, track}; use rangemap::RangeSet; use std::{ @@ -5,7 +8,7 @@ use std::{ ops::{Bound, ControlFlow, Range}, sync::{ Arc, Mutex, - atomic::{AtomicU64, AtomicUsize, Ordering}, + atomic::{AtomicUsize, Ordering}, }, task::{Context, Poll, ready}, time::Duration, @@ -2026,7 +2029,7 @@ impl Request for SubscribeServe { version: shared.version, timescale, runtime: shared.runtime.clone(), - opens: Default::default(), + outcomes: Default::default(), }; let bounds = Bounds::from(msg.as_ref()); @@ -3199,23 +3202,17 @@ struct Subscription { timescale: Option, /// The clock an untimed track's Lite05/06 send times are read from. runtime: crate::time::Deadlines, - /// The group streams this subscription opened, shared by every group it serves. - opens: Arc, + /// Outcomes shared by the subscription and every group it serves. + outcomes: Arc, } -/// Counts a subscription's group streams for lite-07's SUBSCRIBE_END. -/// -/// A group is pending from the moment it is queued until its stream opens, or it gives -/// up first (expired, or the open failed) and is never counted. -/// -/// `unencodable` rides along because every group machine and the run loop already share -/// this handle. A header that cannot be encoded writes nothing, so the group-stream -/// reset is not a stream the subscriber can pin to this subscription. +/// Group outcomes the subscription must report even if no header reached the peer. #[derive(Default)] -struct Opens { - pending: AtomicU64, - opened: AtomicU64, +struct GroupOutcomes { + #[cfg(test)] + opened: std::sync::atomic::AtomicU64, unencodable: Mutex>, + dropped: Mutex>, } impl Subscription { @@ -3277,7 +3274,7 @@ impl Subscription { /// /// The first one wins. The run loop resets the subscribe stream with it. fn note_unencodable(&self, err: &Error) { - let mut slot = self.opens.unencodable.lock().unwrap(); + let mut slot = self.outcomes.unencodable.lock().unwrap(); if slot.is_none() { *slot = Some(err.clone()); } @@ -3285,7 +3282,7 @@ impl Subscription { /// Take the encode failure, if a group recorded one. fn take_unencodable(&self) -> Option { - self.opens.unencodable.lock().unwrap().take() + self.outcomes.unencodable.lock().unwrap().take() } /// Test shim: drive one group stream like the old `serve_group`, surfacing the @@ -3331,10 +3328,7 @@ struct TrackRun { // Groups skipped for a missing head before the start resolved, which it must not name. skipped: BTreeSet, end_sent: bool, - // Lite07+ sends SUBSCRIBE_END with the stream count instead of as soon as the - // boundary is known, once every group below it has opened its stream. - count_streams: bool, - // Lite05 and Lite06 name every sequence below the end that this subscription never + // Lite05+ names every sequence below the end that this subscription never // got with SUBSCRIBE_DROP, so the subscriber settles without waiting out its grace // for a group that will not come. The sequences served, sent as a datagram, or // dropped. A gap that never arrived stays until the end sweep names it: this side @@ -3355,10 +3349,9 @@ impl TrackRun { track.end_at(bounds.end_group.map_or(Bound::Unbounded, Bound::Included)); let emit_range = ctx.version.has_track_stream(); - let count_streams = ctx.version.has_stream_count(); let datagrams = ctx.version.has_datagrams() && ctx.session.max_datagram_size() > 0; // Lite03 and Lite04 declare no end, so neither side could settle on a drop. - let served = (emit_range && !count_streams).then(RangeSet::new); + let served = emit_range.then(RangeSet::new); Self { start_frame: bounds.start_frame(), @@ -3371,7 +3364,6 @@ impl TrackRun { first: None, skipped: BTreeSet::new(), end_sent: false, - count_streams, served, datagrams, children: kio::Tasks::new(), @@ -3450,7 +3442,7 @@ impl TrackRun { #[cfg(test)] fn poll(&mut self, writer: &mut Writer, waiter: &kio::Waiter) -> Poll> { while ready!(self.poll_step(writer, waiter))?.is_continue() {} - Poll::Ready(Ok(())) + writer.poll_flush(&mut waiter.context()) } /// Serve one ready group or datagram (`Continue`), or `Break` once the live edge @@ -3474,6 +3466,14 @@ impl TrackRun { return Poll::Ready(Err(err)); } + let dropped = std::mem::take(&mut *self.ctx.outcomes.dropped.lock().unwrap()); + if !dropped.is_empty() { + for dropped in dropped { + writer.buffer(&lite::SubscribeResponse::Drop(dropped))?; + } + return Poll::Ready(Ok(ControlFlow::Continue(()))); + } + if self.poll_forward_drop(writer, waiter)? { return Poll::Ready(Ok(ControlFlow::Continue(()))); } @@ -3523,7 +3523,7 @@ impl TrackRun { // group and, when enabled, the next best-effort datagram. Groups are polled // first so a datagram burst can't starve them; datagrams flow whenever no // group is ready (including while groups are parked above the cap). - let emit_boundary = self.emit_range && !self.end_sent && !self.count_streams; + let emit_boundary = self.emit_range && !self.end_sent; if let Poll::Ready(res) = poll_recv_next(&mut self.track, self.datagrams, emit_boundary, waiter) { match res? { Recv::Group(mut group) => { @@ -3547,7 +3547,7 @@ impl TrackRun { } else if self.start.is_some() && let Some(end) = group.sequence.checked_add(1) { - // Lite-05/06 name it now; a no-op on other versions. + // Lite-05+ names it now; a no-op on other versions. self.drop_range(group.sequence..end, writer)?; } return Poll::Ready(Ok(ControlFlow::Continue(()))); @@ -3563,22 +3563,12 @@ impl TrackRun { // even if trailing groups (below `group`) are still in flight, then // keep serving them until the live edge reaches the boundary. self.end_sent = true; - writer.buffer(&lite::SubscribeResponse::End(lite::SubscribeEnd { group, streams: 0 }))?; + writer.buffer(&lite::SubscribeResponse::End(lite::SubscribeEnd { group }))?; } - Recv::Finished if self.count_streams && !self.end_sent => { - // The count is final only once no served group is still waiting to - // open its stream. A group that gives up first is never counted, so - // the subscriber is not left waiting for it. The child's wake - // re-polls this loop. - if self.ctx.opens.pending.load(Ordering::Relaxed) > 0 { + Recv::Finished => { + if !self.children.is_empty() { return Poll::Pending; } - let group = final_sequence(ready!(self.track.poll_finished(waiter))?); - let streams = self.ctx.opens.opened.load(Ordering::Relaxed); - self.end_sent = true; - writer.buffer(&lite::SubscribeResponse::End(lite::SubscribeEnd { group, streams }))?; - } - Recv::Finished => { self.drop_unserved(writer)?; return Poll::Ready(Ok(ControlFlow::Break(()))); } @@ -3593,7 +3583,7 @@ impl TrackRun { Poll::Pending } - /// Lite-05/06 name a sequence as soon as this cursor gives it up; whether one went out. + /// Lite-05+ names a sequence as soon as this cursor gives it up; whether one went out. /// Before SUBSCRIBE_START there is no owed range yet, so the eventual start accounts /// for it instead. fn poll_forward_drop( @@ -3802,7 +3792,6 @@ impl GroupServe { rank: Rank>, group: group::Consumer, ) -> Self { - ctx.opens.pending.fetch_add(1, Ordering::Relaxed); Self { ctx, rank, @@ -3815,11 +3804,11 @@ impl GroupServe { } } - /// Leave [`GroupState::Open`], counting the stream if it opened. - fn settle_open(&mut self, opened: bool) { - self.ctx.opens.pending.fetch_sub(1, Ordering::Relaxed); - if opened { - self.ctx.opens.opened.fetch_add(1, Ordering::Relaxed); + /// Count opened streams for the scheduling tests. + fn settle_open(&mut self, _opened: bool) { + #[cfg(test)] + if _opened { + self.ctx.outcomes.opened.fetch_add(1, Ordering::Relaxed); } } @@ -4079,9 +4068,14 @@ impl kio::Task for GroupServe { type Output = (); fn poll(&mut self, waiter: &kio::Waiter) -> Poll<()> { - // The machine owns its outcome: the stream was aborted with the reason (or - // reset by the writer's Drop), which is all the subscriber sees. - ready!(self.poll_serve(waiter)).map(|()| ()).unwrap_or(()); + // A reset can lose its header, so the control stream also accounts for the group. + if ready!(self.poll_serve(waiter)).is_err() && self.ctx.version.has_track_stream() { + self.ctx.outcomes.dropped.lock().unwrap().push(lite::SubscribeDrop { + start: self.sequence, + end: self.sequence, + error: 0, + }); + } Poll::Ready(()) } } @@ -4229,7 +4223,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4274,7 +4268,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4323,7 +4317,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4372,7 +4366,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4439,7 +4433,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4515,7 +4509,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4563,7 +4557,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4610,7 +4604,7 @@ mod serve_group_test { version: Version::Lite06, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let track = track::Producer::new( @@ -4669,7 +4663,7 @@ mod serve_group_test { version, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: Default::default(), + outcomes: Default::default(), }; let bounds = Bounds { start_group: Some(0), @@ -4878,10 +4872,9 @@ mod serve_group_test { .await; } - /// SUBSCRIBE_END counts the group streams opened, not the groups below the end: a - /// group the track never produced has no stream and is not counted. + /// A skipped sequence is accounted for by DROP rather than a stream count. #[moq_clock::test] - async fn lite07_end_counts_the_streams_opened() { + async fn lite07_end_drops_skipped_sequences() { let mut track = track::Producer::new( Arc::new(broadcast::Info::default()), "test", @@ -4897,8 +4890,8 @@ mod serve_group_test { track.finish().unwrap(); run.await.unwrap(); - // SUBSCRIBE_START at 0 (largest 0.0), then SUBSCRIBE_END at 3 with 2 streams. - assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 1, 2, 3, 2]); + // SUBSCRIBE_START at 0, SUBSCRIBE_END at 3, and a DROP for the skipped group 1. + assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 1, 1, 3, 2, 3, 1, 1, 0]); } /// A relay that holds only the first group of a backlog answers with the largest its @@ -4950,7 +4943,7 @@ mod serve_group_test { drops } - /// Lite-05 and lite-06 name every sequence below the end that the subscription never + /// Lite-05 and later name every sequence below the end that the subscription never /// got once the track ends: one skipped between groups, and one past the last. #[moq_clock::test] async fn the_end_drops_the_sequences_never_served() { @@ -5152,7 +5145,11 @@ mod serve_group_test { drop(track); kio::wait(|waiter| run.poll(&mut writer, waiter)).await.unwrap(); assert_eq!(drops(&mut writer, &log, version).await, [(1, 1)]); - assert_eq!(run.ctx.opens.opened.load(Ordering::Relaxed), 2, "groups 0 and 2 only"); + assert_eq!( + run.ctx.outcomes.opened.load(Ordering::Relaxed), + 2, + "groups 0 and 2 only" + ); } /// A late group the cursor skips as stale is dropped at once, even when it is the only @@ -5177,7 +5174,11 @@ mod serve_group_test { moq_clock::sim::advance(Duration::from_secs(2)).await; write_group(&mut track, 1, 1); while run.poll_step(&mut writer, &kio::Waiter::noop()).is_ready() {} - assert_eq!(run.ctx.opens.opened.load(Ordering::Relaxed), 2, "group 1 is not served"); + assert_eq!( + run.ctx.outcomes.opened.load(Ordering::Relaxed), + 2, + "group 1 is not served" + ); assert_eq!(drops(&mut writer, &log, version).await, [(1, 1)]); } @@ -5210,7 +5211,7 @@ mod serve_group_test { write_group(&mut track, 1, 1); while run.poll_step(&mut writer, &kio::Waiter::noop()).is_ready() {} assert_eq!( - run.ctx.opens.opened.load(Ordering::Relaxed), + run.ctx.outcomes.opened.load(Ordering::Relaxed), 1, "only group 0 is served" ); @@ -5415,7 +5416,7 @@ mod serve_group_test { let mut settle = |run: &mut TrackRun| { let res = kio::wait(|waiter| run.poll(&mut writer, waiter)).now_or_never(); assert!(res.is_none(), "the run ended"); - run.ctx.opens.opened.load(Ordering::Relaxed) + run.ctx.outcomes.opened.load(Ordering::Relaxed) }; write_group(&mut track, 7, 7); @@ -5466,7 +5467,7 @@ mod serve_group_test { let res = kio::wait(|waiter| run.poll(&mut writer, waiter)).now_or_never(); assert!(res.is_none(), "the run ended"); assert_eq!( - run.ctx.opens.opened.load(Ordering::Relaxed), + run.ctx.outcomes.opened.load(Ordering::Relaxed), 1, "group 0 was not served" ); @@ -5507,7 +5508,7 @@ mod serve_group_test { run: TrackRun, writer: Writer, session: ScriptedSession, - opens: Arc, + outcomes: Arc, } impl RelayRun { @@ -5522,7 +5523,7 @@ mod serve_group_test { let session = ScriptedSession::new(Vec::new()); let writer = Writer::new(SinkSend::new(session.log.clone()), Version::Lite07); let steer = kio::Producer::new(Steer::default()); - let opens = Arc::::default(); + let outcomes = Arc::::default(); let ctx = Subscription { session: session.clone(), id: 0, @@ -5534,7 +5535,7 @@ mod serve_group_test { version: Version::Lite07, timescale: Some(crate::Timescale::MILLI), runtime: crate::time::sim(), - opens: opens.clone(), + outcomes: outcomes.clone(), }; let bounds = Bounds { start_group: Some(start_group), @@ -5546,7 +5547,7 @@ mod serve_group_test { run: TrackRun::new(ctx, subscriber, bounds, steer), writer, session, - opens, + outcomes, } } @@ -5559,7 +5560,7 @@ mod serve_group_test { /// How many group streams the run opened. fn opened(&self) -> u64 { - self.opens.opened.load(Ordering::Relaxed) + self.outcomes.opened.load(Ordering::Relaxed) } /// Whether the first thing written was SUBSCRIBE_START at `group`. @@ -5721,7 +5722,7 @@ mod serve_group_test { /// A track that ends without a group still ends the subscription, with no stream owed. #[moq_clock::test] - async fn lite07_end_counts_zero_streams() { + async fn lite07_empty_track_ends_without_a_start() { let track = track::Producer::new( Arc::new(broadcast::Info::default()), "test", @@ -5732,14 +5733,12 @@ mod serve_group_test { let (mut run, mut writer, log) = lite07_run(SinkSession::new(Log::default()), subscriber); kio::wait(|waiter| run.poll(&mut writer, waiter)).await.unwrap(); - assert_eq!(*log.writes.lock().unwrap(), [1, 2, 0, 0]); + assert_eq!(*log.writes.lock().unwrap(), [1, 1, 0]); } - /// The count is sent once every served group has opened its stream or given up, so a - /// group still waiting for stream credit holds SUBSCRIBE_END back, and one that expires - /// first is never counted. + /// A declared end is sent before stream credit, and a group that expires first is dropped. #[moq_clock::test] - async fn lite07_end_waits_for_every_stream_to_open() { + async fn lite07_end_precedes_blocked_streams_and_reports_expiry() { let gate = kio::Producer::new(false); let mut track = track::Producer::new( Arc::new(broadcast::Info::default()), @@ -5760,8 +5759,8 @@ mod serve_group_test { assert!(futures::poll!(run.as_mut()).is_pending(), "group 1 is still opening"); assert_eq!( *log.writes.lock().unwrap(), - [0, 3, 0, 1, 0], - "only SUBSCRIBE_START so far" + [0, 3, 0, 1, 0, 2, 3, 0, 0, 0, 1, 1, 2], + "the expired group is dropped and the end precedes stream credit" ); let Ok(mut open) = gate.write() else { @@ -5770,7 +5769,7 @@ mod serve_group_test { *open = true; drop(open); run.await.unwrap(); - assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 1, 2, 2, 1]); + assert_eq!(*log.writes.lock().unwrap(), [0, 3, 0, 1, 0, 2, 3, 0, 0, 0, 1, 1, 2]); } } @@ -6650,7 +6649,7 @@ mod tests { let RequestState::Serve(SubscribeServe::Run(run)) = &serve.state else { panic!("the subscription is not running"); }; - let opens = run.ctx.opens.clone(); + let outcomes = run.ctx.outcomes.clone(); let mut tasks: kio::Tasks = kio::Tasks::new(); @@ -6689,13 +6688,13 @@ mod tests { let mut polls = 0; kio::wait(|owner| { - if opens.opened.load(Ordering::Relaxed) >= GROUPS { + if outcomes.opened.load(Ordering::Relaxed) >= GROUPS { return Poll::Ready(()); } - let before = opens.opened.load(Ordering::Relaxed); + let before = outcomes.opened.load(Ordering::Relaxed); assert!(tasks.poll(owner).is_pending(), "the serve never ends"); polls += 1; - let served = opens.opened.load(Ordering::Relaxed) - before; + let served = outcomes.opened.load(Ordering::Relaxed) - before; assert!(served <= BUDGET, "one poll served {served} groups"); assert_eq!(turns.load(Ordering::Relaxed), polls, "the sibling missed a turn"); assert!(polls <= 2 * GROUPS, "the serve stalled"); diff --git a/rs/moq-net/src/lite/subscribe.rs b/rs/moq-net/src/lite/subscribe.rs index 78883ffb0fa..79c7c44b8c8 100644 --- a/rs/moq-net/src/lite/subscribe.rs +++ b/rs/moq-net/src/lite/subscribe.rs @@ -376,9 +376,6 @@ impl Message for SubscribeStart { #[derive(Clone, Debug)] pub struct SubscribeEnd { pub group: u64, - /// The number of group streams the publisher opened for this subscription. - /// Lite07+ only; not on the wire before, where it decodes as 0. - pub streams: u64, } impl Message for SubscribeEnd { @@ -387,11 +384,7 @@ impl Message for SubscribeEnd { return Err(DecodeError::Version); } let group = r.varint()?; - let streams = match version.has_stream_count() { - true => r.varint()?, - false => 0, - }; - Ok(Self { group, streams }) + Ok(Self { group }) } fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { @@ -399,9 +392,6 @@ impl Message for SubscribeEnd { return Err(EncodeError::Version); } w.varint(self.group)?; - if version.has_stream_count() { - w.varint(self.streams)?; - } Ok(()) } } @@ -489,7 +479,7 @@ impl Message for SubscribeUpdate { /// The range `[start, end]` is inclusive on both ends. For example, /// `start = 5, end = 7` means groups 5, 6, and 7 were dropped. /// -/// Lite03 to Lite06 only: Lite07 counts group streams in [`SubscribeEnd`] instead. +/// Lite03+; publishers send it on Lite05 and later. #[derive(Clone, Debug)] pub struct SubscribeDrop { /// The first absolute group sequence in the dropped range. @@ -509,7 +499,6 @@ impl Message for SubscribeDrop { Version::Lite01 | Version::Lite02 => { return Err(DecodeError::Version); } - _ if version.has_stream_count() => return Err(DecodeError::Version), _ => {} } @@ -525,7 +514,6 @@ impl Message for SubscribeDrop { Version::Lite01 | Version::Lite02 => { return Err(EncodeError::Version); } - _ if version.has_stream_count() => return Err(EncodeError::Version), _ => {} } @@ -542,9 +530,8 @@ impl Message for SubscribeDrop { /// /// The discriminator is version-dependent: /// - Lite03/04: `0x0` SUBSCRIBE_OK, `0x1` SUBSCRIBE_DROP. -/// - Lite05/06: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END, `0x2` SUBSCRIBE_DROP +/// - Lite05+: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END, `0x2` SUBSCRIBE_DROP /// (SUBSCRIBE_OK was removed; acceptance is implicit). -/// - Lite07+: `0x0` SUBSCRIBE_START, `0x1` SUBSCRIBE_END (SUBSCRIBE_DROP was removed). #[derive(Clone, Debug)] pub enum SubscribeResponse { Ok(SubscribeOk), @@ -575,8 +562,8 @@ impl Encode for SubscribeResponse { _ => match self { Self::Start(start) => encode_typed(w, 0, start, version)?, Self::End(end) => encode_typed(w, 1, end, version)?, - Self::Drop(drop) if !version.has_stream_count() => encode_typed(w, 2, drop, version)?, - Self::Drop(_) | Self::Ok(_) => return Err(EncodeError::Version), + Self::Drop(drop) => encode_typed(w, 2, drop, version)?, + Self::Ok(_) => return Err(EncodeError::Version), }, } @@ -601,7 +588,7 @@ impl Decode for SubscribeResponse { match typ { 0 => Ok(Self::Start(SubscribeStart::decode(buf, version)?)), 1 => Ok(Self::End(SubscribeEnd::decode(buf, version)?)), - 2 if !version.has_stream_count() => Ok(Self::Drop(SubscribeDrop::decode(buf, version)?)), + 2 => Ok(Self::Drop(SubscribeDrop::decode(buf, version)?)), _ => Err(DecodeError::InvalidMessage(typ)), } } @@ -661,54 +648,48 @@ mod test { #[test] fn subscribe_end_roundtrips_on_lite05() { - let resp = SubscribeResponse::End(SubscribeEnd { group: 7, streams: 3 }); + let resp = SubscribeResponse::End(SubscribeEnd { group: 7 }); let mut buf = Vec::new(); resp.encode(&mut Encoder::new(&mut buf, Version::Lite05.into()), Version::Lite05) .unwrap(); - // Type, length, group: no stream count before lite-07. + // Type, length, exclusive final group. assert_eq!(buf, [1, 1, 7]); let mut slice = buf.as_slice(); match crate::coding::decode_buf(&mut slice, Version::Lite05, SubscribeResponse::decode).unwrap() { - SubscribeResponse::End(end) => assert_eq!((end.group, end.streams), (7, 0)), + SubscribeResponse::End(end) => assert_eq!(end.group, 7), other => panic!("expected End, got {other:?}"), } } #[test] - fn subscribe_end_carries_the_stream_count_on_lite07() { - let resp = SubscribeResponse::End(SubscribeEnd { group: 7, streams: 3 }); + fn subscribe_end_has_no_stream_count_on_lite07() { + let resp = SubscribeResponse::End(SubscribeEnd { group: 7 }); let mut buf = Vec::new(); resp.encode(&mut Encoder::new(&mut buf, Version::Lite07.into()), Version::Lite07) .unwrap(); - assert_eq!(buf, [1, 2, 7, 3]); + assert_eq!(buf, [1, 1, 7]); let mut slice = buf.as_slice(); match crate::coding::decode_buf(&mut slice, Version::Lite07, SubscribeResponse::decode).unwrap() { - SubscribeResponse::End(end) => assert_eq!((end.group, end.streams), (7, 3)), + SubscribeResponse::End(end) => assert_eq!(end.group, 7), other => panic!("expected End, got {other:?}"), } } #[test] - fn subscribe_drop_is_gone_on_lite07() { + fn subscribe_drop_is_restored_on_lite07() { let resp = SubscribeResponse::Drop(SubscribeDrop { start: 1, end: 3, - error: 0, + error: 7, }); let mut buf = Vec::new(); - assert!(matches!( - resp.encode(&mut Encoder::new(&mut buf, Version::Lite07.into()), Version::Lite07), - Err(EncodeError::Version) - )); - - // A lite-06 DROP is an unknown response type on lite-07. - let mut buf = Vec::new(); - resp.encode(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06) + resp.encode(&mut Encoder::new(&mut buf, Version::Lite07.into()), Version::Lite07) .unwrap(); - assert!(matches!( - crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite07, SubscribeResponse::decode), - Err(DecodeError::InvalidMessage(2)) - )); + assert_eq!(buf, [2, 3, 1, 3, 7]); + match crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite07, SubscribeResponse::decode).unwrap() { + SubscribeResponse::Drop(drop) => assert_eq!((drop.start, drop.end, drop.error), (1, 3, 7)), + other => panic!("expected Drop, got {other:?}"), + } } #[test] diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index b7b55d2ed42..ab25615c560 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2088,7 +2088,7 @@ mod tests { requested: Some(Position::group(2)), tail: Default::default(), served: None, - end: Some(lite::SubscribeEnd { group: 10, streams: 0 }), + end: Some(lite::SubscribeEnd { group: 10 }), }; assert_eq!( sub.owed(None), @@ -2410,7 +2410,7 @@ mod tests { version, ) .unwrap(); - lite::SubscribeResponse::End(lite::SubscribeEnd { group: 2, streams: 1 }) + lite::SubscribeResponse::End(lite::SubscribeEnd { group: 2 }) .encode( &mut crate::coding::Encoder::new(&mut responses, version.into()), version, @@ -2597,7 +2597,7 @@ mod tests { .unwrap(); } if clean { - lite::SubscribeResponse::End(lite::SubscribeEnd { group: 0, streams: 0 }) + lite::SubscribeResponse::End(lite::SubscribeEnd { group: 0 }) .encode( &mut crate::coding::Encoder::new(&mut responses, version.into()), version, @@ -3056,7 +3056,6 @@ mod tests { serving.mode = ServeMode::Tail { settle: Settle::new(&serve.subscriber.runtime, tail.consume()), owed: Some(0..1), - streams: None, }; assert!(serving.poll(&serve, &kio::Waiter::noop()).is_pending()); peer.close(crate::SessionError::Unauthorized.to_code(), "revoked"); @@ -3113,7 +3112,7 @@ mod tests { send.reset(crate::StreamError::NotFound.to_code()); } else { let mut end = Vec::new(); - lite::SubscribeResponse::End(lite::SubscribeEnd { group: 0, streams: 0 }) + lite::SubscribeResponse::End(lite::SubscribeEnd { group: 0 }) .encode(&mut crate::coding::Encoder::new(&mut end, version.into()), version) .unwrap(); std::future::poll_fn(|cx| send.poll_write(cx, &end)).await.unwrap(); @@ -4653,7 +4652,7 @@ struct SubStream { tail: kio::Producer, /// The first group the publisher serves (SUBSCRIBE_START), once declared. served: Option, - /// The track's exclusive end and stream count (SUBSCRIBE_END), once declared. + /// The track's exclusive end (SUBSCRIBE_END), once declared. end: Option, } @@ -5746,13 +5745,11 @@ enum ServeMode { /// exactly like the old inline await. Establish(Establish), /// The upstream FIN'd, so the track is over, but QUIC does not order streams: keep - /// the subscription routable until the counted headers arrive (lite-07), or every - /// owed group is accounted for (older drafts). The grace bounds a stream reset - /// before its header arrived. + /// the subscription routable until every owed group is accounted for. The grace bounds + /// an unaccounted sequence, such as a lost datagram. Tail { settle: Settle, owed: Option>, - streams: Option, }, } @@ -5965,14 +5962,13 @@ impl ServeLoop { } } } - ServeMode::Tail { settle, owed, streams } => { + ServeMode::Tail { settle, owed } => { let _ = self.fetches.poll(waiter); self.feed_fetches .poll(&self.sub, serve.subscriber.runtime.now(), waiter); if settle - .poll(waiter, |tail| match streams { - Some(streams) => tail.streams() >= *streams, - None => owed.clone().is_some_and(|owed| tail.covers(owed)), + .poll(waiter, |tail| { + owed.as_ref().is_some_and(|owed| tail.covers(owed.clone())) }) .is_ready() { @@ -6197,11 +6193,6 @@ impl ServeLoop { self.mode = ServeMode::Tail { settle: Settle::new(&serve.subscriber.runtime, active.tail.consume()), owed: active.owed(requested_end), - streams: active - .end - .as_ref() - .filter(|_| serve.subscriber.version.has_stream_count()) - .map(|end| end.streams), }; continue; } diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index 9749b2ea0aa..1aab80781be 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -18,8 +18,8 @@ pub enum Version { /// as `moq-lite-06` and preferred by the default version sets. Lite06, /// Lite-07. Adds the hidden opt-in to ANNOUNCE_REQUEST: without it, a route with - /// a `.`-prefixed segment below the requested prefix is left out. SUBSCRIBE_END - /// carries the number of group streams opened, replacing SUBSCRIBE_DROP. + /// a `.`-prefixed segment below the requested prefix is left out. SUBSCRIBE_DROP + /// accounts for every group the subscription will not deliver. /// ANNOUNCE_START and ANNOUNCE_UPDATE may copy a path head or hop-chain tail from a /// live announcement on the same stream. Adds the Auth Stream. The wire format is /// still work-in-progress, so it is advertised over ALPN as `moq-lite-07-wip` and @@ -28,17 +28,6 @@ pub enum Version { } impl Version { - /// Whether SUBSCRIBE_END carries the subscription's group stream count, sent once - /// every counted stream is open, in place of SUBSCRIBE_DROP. Added in lite-07. - #[allow(clippy::match_like_matches_macro)] - pub fn has_stream_count(self) -> bool { - // Match form so future versions default forward (AGENTS.md convention). - match self { - Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 => false, - _ => true, - } - } - /// Whether SUBSCRIBE_OK carries the publisher's largest (group, frame), which a /// subscriber takes as where the live feed is. Added in lite-07; an earlier answer /// says nothing about the live edge. diff --git a/rs/moq-net/tests/integration/track_tail.rs b/rs/moq-net/tests/integration/track_tail.rs index 5f148b8d29a..cb191abe486 100644 --- a/rs/moq-net/tests/integration/track_tail.rs +++ b/rs/moq-net/tests/integration/track_tail.rs @@ -229,8 +229,7 @@ async fn a_lost_group_ends_the_track_after_the_grace() { } } -/// Skipped sequences have no stream, so they must not hold the tail open: lite-07 does not -/// count them, and lite-05 and lite-06 publishers name them with SUBSCRIBE_DROP. +/// Skipped sequences have no stream, so lite-05+ publishers name them with SUBSCRIBE_DROP. #[moq_clock::test] async fn skipped_groups_end_without_the_grace() { for version in ["moq-lite-05", "moq-lite-06", "moq-lite-07-wip"] { @@ -245,9 +244,9 @@ async fn skipped_groups_end_without_the_grace() { } } -/// A zero stream count leaves no tail to wait for. +/// An empty track leaves no tail to wait for. #[moq_clock::test] -async fn lite07_zero_streams_end_without_the_grace() { +async fn lite07_empty_track_ends_without_the_grace() { let outcome = round("moq-lite-07-wip", Late::Delivered, 0).await; assert!(outcome.err.is_none()); assert!(outcome.frames.is_empty()); @@ -318,13 +317,10 @@ async fn ietf_leaving_cancels_a_blocked_end_of_track() { } } -/// A lost datagram is not owed, but nothing tells a lite-05 or lite-06 subscriber its -/// sequence was a datagram rather than a group stream still on its way. Its hole holds -/// the subscription, and its readers, for the grace, like a stream reset before its -/// header: ending at once would drop a reordered group. Lite-07 counts the streams, so a -/// lost datagram never delays its end. +/// A lost datagram is indistinguishable from a reordered stream header, so its hole +/// waits out the tail grace on every lite version. #[moq_clock::test] -async fn a_lost_datagram_delays_the_end_only_without_a_stream_count() { +async fn a_lost_datagram_delays_the_end() { for version in ["moq-lite-05", "moq-lite-07-wip"] { let Pair { pair, @@ -371,13 +367,10 @@ async fn a_lost_datagram_delays_the_end_only_without_a_stream_count() { .expect("reader panicked"); assert_eq!(groups, [0, 2], "{version}"); let elapsed = ended - finished; - match version { - "moq-lite-07-wip" => assert!(elapsed < GRACE / 10, "{version}: ended after {elapsed:?}"), - _ => assert!( - (GRACE / 2..GRACE * 2).contains(&elapsed), - "{version}: ended after {elapsed:?}" - ), - } + assert!( + (GRACE / 2..GRACE * 2).contains(&elapsed), + "{version}: ended after {elapsed:?}" + ); drop((pair, _keep)); } } diff --git a/sh/gh/nightly.sh b/sh/gh/nightly.sh index 7ee65793ab0..d21edd4ce79 100755 --- a/sh/gh/nightly.sh +++ b/sh/gh/nightly.sh @@ -102,7 +102,7 @@ case "$1" in done ;; js) - for benchmark in broadcasts request-ids requests forward reader frames varint lite-varint track resume interests priority; do + for benchmark in broadcasts request-ids requests forward reader frames varint lite-varint track resume interests priority subscribe-drop; do javascript "$benchmark" just js bench "$benchmark" done javascript catalog just js catalog-bench diff --git a/test/interop/interop.sh b/test/interop/interop.sh index 73f675b515f..efd67d555f4 100755 --- a/test/interop/interop.sh +++ b/test/interop/interop.sh @@ -1243,7 +1243,7 @@ if [[ "$NEGATIVE" -eq 0 && "$MEDIA" -eq 0 && "$IN_TREE" -eq 1 ]]; then fi done if needs js-native-node; then - for version in moq-lite-05 moq-lite-06; do + for version in moq-lite-05 moq-lite-06 moq-lite-07-wip; do run_drop_pair rust js-native-node "$version" run_drop_pair js-native-node rust "$version" done