Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions doc/concept/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,10 @@ and waits up to one second for them to be delivered. An abort skips that.

A broadcast has no finished state. Online means announced, and offline means
no route is left. The end of content is a track FIN: the publisher finishes
each track, and a hang broadcast also finishes its catalog. A finished track
stays servable to the subscribers it already has, including through a relay
each track, and a hang broadcast also finishes its catalog. A track can also
finish with an error, such as a replayed recording whose source failed: readers
get its groups, then the error. Only moq-lite-07 carries that error across a
hop; older versions reset instead. A finished track stays servable to the subscribers it already has, including through a relay
whose upstream session has since gone. How long a finished broadcast stays
announced for late readers is the application's retention policy, such as the
`--linger` of [`moq import`](/bin/cli), not a protocol state.
Expand Down
6 changes: 6 additions & 0 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -1503,6 +1503,7 @@ SUBSCRIBE_END Message {
Type (i) = 0x1
Message Length (i)
Group (i)
Error Code (i)
}
~~~

Expand All @@ -1514,6 +1515,10 @@ The exclusive end of the range: the absolute sequence number of the first group
A value of 0 means the track ended before producing any groups.
The subscriber MUST NOT wait for any group at or after this sequence.

**Error Code**:
0 if the track ended cleanly, otherwise the stream error code it ended with plus 1.
Groups below the end stay deliverable either way; a subscriber reaching the end surfaces the error instead of a clean end, and a relay forwards it with the end.

The publisher SHOULD send SUBSCRIBE_END as soon as the exclusive final group is known, even while lower groups are still in flight.

## SUBSCRIBE_DROP {#subscribe-drop}
Expand Down Expand Up @@ -1853,6 +1858,7 @@ The `Message Length` describes the payload size on the wire.
- Added the client-only Token Setup Parameter (0x7), carrying the connection credential in SETUP instead of only the request URI.
- Switched every variable-length integer, including SETUP parameter values, from QUIC's two-bit length prefix to moq-transport's leading-ones encoding, widening the range to 64 bits.
- Hid routes with a `.`-prefixed segment below the requested prefix from announce discovery, and added the ANNOUNCE_REQUEST `Hidden` field to opt in.
- Added `Error Code` to SUBSCRIBE_END: a track can end with an error while keeping the groups below its end.
- Restored SUBSCRIBE_DROP (type 0x2), replacing Stream Count on SUBSCRIBE_END. Every sequence is accounted for by a Group Stream, a datagram, or a drop; a publisher SHOULD send a drop as soon as it gives a sequence up.
- SUBSCRIBE_DROP's Error Code is a stream error code, which ends the dropped groups' readers, including one still arriving.
- SUBSCRIBE_END declares the final boundary while lower groups can still be in flight; the Subscribe Stream FIN follows once every Group Stream has finished or been reset.
Expand Down
2 changes: 1 addition & 1 deletion js/net/api/index.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -622,7 +622,7 @@ export namespace Track {
createGroup(sequence: number): Group.Producer;
demand(): Track.Demand;
finish(): void;
finishAt(end: Track.Position): void;
finishAt(end: Track.Position, error?: Error): void;
info(): Promise<Track.Info>;
insertDatagram(sequence: number, timestamp: Time.Timestamp | undefined, payload: Uint8Array): void;
largest(): Track.Position | undefined;
Expand Down
9 changes: 8 additions & 1 deletion js/net/src/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,8 @@ export interface ResumeSource {
start(): number | "pending" | undefined;
/** The logical track's declared exclusive end, as the first copy to know it declared it. */
end(): Position | undefined;
/** The error the logical track's end carries, if it does not end cleanly. */
endError(): Error | undefined;
/** Resolve with the logical track's exclusive end once known. */
finished(): Promise<Position>;
/** The serving copy's newest buffered frame, which a wire answer names as its edge. */
Expand Down Expand Up @@ -372,7 +374,7 @@ export const hooks: {
* past the grace, or the track closes, since QUIC may deliver a lower group's stream after
* a higher one. Mirrors the Rust `finish_at` on a `received()` copy; assigned by `track.ts`.
*/
endTrack: (producer: Producer, end: Position) => void;
endTrack: (producer: Producer, end: Position, error?: Error) => void;
/**
* End a wire subscription's copy cleanly without declaring an end, for a session closed
* locally: a dead route, not finished content, so a resumed reader's replacement route
Expand Down Expand Up @@ -407,6 +409,8 @@ export const hooks: {
* forwarding it. Readers await `finished()` instead. Assigned by `track.ts`.
*/
subscriberEnd: (subscriber: Subscriber) => Position | undefined;
/** The error that end carries, if the track does not end cleanly. Assigned by `track.ts`. */
subscriberEndError: (subscriber: Subscriber) => Error | undefined;
joinSubscriber: (subscriber: Subscriber) => void;
/** Re-evaluate a live-edge join at a producer. */
joinProducer: (producer: Producer) => void;
Expand Down Expand Up @@ -626,6 +630,9 @@ export const hooks: {
subscriberEnd: () => {
throw new Error("track.ts not loaded");
},
subscriberEndError: () => {
throw new Error("track.ts not loaded");
},
joinSubscriber: () => {
throw new Error("track.ts not loaded");
},
Expand Down
82 changes: 78 additions & 4 deletions js/net/src/lite/publisher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,9 +190,13 @@ test.each([Version.DRAFT_06, Version.DRAFT_07])(
},
);

// Delivers `sequences` in the given order, finishes the track, and returns the
// SUBSCRIBE_END the publisher put on the wire.
async function subscribeEnd(sequences: number[], version: Version = Version.DRAFT_05): Promise<SubscribeEnd> {
// Delivers `sequences` in the given order, finishes the track (with `error`, if given), and
// returns the SUBSCRIBE_END the publisher put on the wire.
async function subscribeEnd(
sequences: number[],
version: Version = Version.DRAFT_05,
error?: Error,
): Promise<SubscribeEnd> {
const pair = createMockTransportPair(version === Version.DRAFT_07 ? ALPN_07_WIP : ALPN_05);
const origin = new OriginProducer();
const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
Expand All @@ -219,7 +223,8 @@ async function subscribeEnd(sequences: number[], version: Version = Version.DRAF
// arrives late opens no stream at all, so there is nothing firmer to wait on.
await new Promise((resolve) => setTimeout(resolve, 5));
}
track.finish();
if (error) track.finishAt({ group: Math.max(-1, ...sequences) + 1, frame: 0 }, error);
else track.finish();
track.close();

try {
Expand Down Expand Up @@ -1729,6 +1734,13 @@ test("lite draft-05: subscribe end is 0 when no groups were produced", async ()
expect((await subscribeEnd([])).group).toBe(0);
});

// Lite-07 forwards an errored end's error with its boundary, so the subscriber keeps the history.
test("lite draft-07: subscribe end carries the track's error", async () => {
const end = await subscribeEnd([0, 1], Version.DRAFT_07, new StreamError(StreamCode(71)));
expect([end.group, end.error]).toEqual([2, 71]);
expect((await subscribeEnd([0, 1], Version.DRAFT_07)).error).toBeUndefined();
});

// The boundary clears every produced group even when a sequence was skipped.
test("lite draft-07: subscribe end names the final boundary", async () => {
const end = await subscribeEnd([0, 2], Version.DRAFT_07);
Expand Down Expand Up @@ -1783,6 +1795,68 @@ test("lite draft-07: subscribe end precedes groups below a declared finish", asy
}
});

// Before lite-07 an errored end goes out only as the reset at that end: a clean SUBSCRIBE_END
// sent early could let the subscriber settle before the reset arrives.
test("lite draft-06: an errored end is not declared early", async () => {
const pair = createMockTransportPair(ALPN_06);
const origin = new OriginProducer();
const publisher = new Publisher(pair.server, Version.DRAFT_06, randomHop(), origin.consume());
const broadcast = publish(origin, Path.from("test"));
const track = broadcast.createTrack("video", { timescale: Timescale.MILLI });

const client = await Stream.open(pair.client, { version: Version.DRAFT_06 });
const server = await Stream.accept(pair.server, Version.DRAFT_06);
if (!server) throw new Error("publisher never accepted the subscribe stream");
void publisher.runSubscribe(
new Subscribe({ id: 0n, broadcast: Path.from("test"), track: "video", priority: 0 }),
server,
);

try {
const first = track.createGroup(0);
first.writeFrame(textFrame("hello"));
first.finish();
track.finishAt({ group: 2, frame: 0 }, new StreamError(StreamCode(71)));

const start = await decodeSubscribeResponse(client.reader, Version.DRAFT_06);
expect("start" in start).toBe(true);
const pending = decodeSubscribeResponse(client.reader, Version.DRAFT_06);
const early = await Promise.race([
pending.then(
() => "response",
() => "reset",
),
new Promise((resolve) => setTimeout(() => resolve("none"), 20)),
]);
expect(early).toBe("none");

const second = track.createGroup(1);
second.writeFrame(textFrame("hello"));
second.finish();
await expect(pending).rejects.toBeDefined();
} finally {
publisher.close();
client.close();
}
});

// A subscription capped short of an errored end is owed no error, so its boundary still goes out.
test("lite draft-06: a subscription capped short of an errored end gets its boundary", async () => {
const sub = await servedSubscription({ version: Version.DRAFT_06, startGroup: 0, endGroup: 1 });
try {
for (const sequence of [0, 1, 2, 3]) sub.serve(sequence);
sub.track.finishAt({ group: 4, frame: 0 }, new StreamError(StreamCode(71)));
for (;;) {
const resp = await decodeSubscribeResponse(sub.client.reader, Version.DRAFT_06);
if (!("end" in resp)) continue;
expect(resp.end.group).toBe(4);
break;
}
} finally {
await sub.close();
}
});

// Serves one group with its stream open held until `open(ok)`, and returns the pending
// SUBSCRIBE_END plus the call that lets the open succeed or fail.
async function heldOpenEnd() {
Expand Down
38 changes: 33 additions & 5 deletions js/net/src/lite/publisher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,16 @@ import { enforceGrant } from "../auth_session.ts";
import type * as broadcast from "../broadcast.ts";
import { Withdrawal } from "../connection/withdrawal.ts";
import * as DatagramStream from "../datagram_stream.ts";
import { error, NotFound, ProtocolViolation, reason, StreamCode, StreamError, unauthorized } from "../error.ts";
import {
error,
NotFound,
ProtocolViolation,
reason,
StreamCode,
StreamError,
toStreamCode,
unauthorized,
} from "../error.ts";
import { stored } from "../flate/group.ts";
import type * as group from "../group.ts";
import { Cost, type Hop, type Route, routesEqual } from "../hop.ts";
Expand Down Expand Up @@ -47,6 +56,7 @@ import {
hasAnnounceRestart,
hasCompression,
hasDatagrams,
hasEndError,
hasLargest,
hasProbeRtt,
hasRouteCost,
Expand Down Expand Up @@ -1216,15 +1226,26 @@ export class Publisher {
return end ? finalSequence(end) : (hooks.latestSequence(track) ?? -1) + 1;
};

// Declare the boundary immediately; groups below it can still be in flight.
// Declare the boundary immediately; groups below it can still be in flight. Lite-07 carries
// an errored end's error with it, which older versions reset with once the end is reached.
const sendEnd = async (): Promise<boolean> => {
endSent = true;
if (!emitRange) return true;
const endError = hooks.subscriberEndError(track);
const code = endError && hasEndError(this.version) ? toStreamCode(endError) : undefined;
return controls.response(
encodeSubscribeResponse(stream, { end: new SubscribeEnd(boundary()) }, this.version),
encodeSubscribeResponse(stream, { end: new SubscribeEnd(boundary(), code) }, this.version),
);
};

// Before lite-07 an errored end is only the reset at that end: a clean SUBSCRIBE_END sent
// early could let the subscriber settle before the reset arrives. A subscription capped
// short of the end is owed no error, so its boundary goes out.
const declareEarly = () =>
hasEndError(this.version) ||
!hooks.subscriberEndError(track) ||
(bounds.endGroup !== undefined && bounds.endGroup + 1 < boundary());

// One ranking for the whole subscription, shared by every group it serves.
const priority = new Priority(track, publisher);

Expand Down Expand Up @@ -1360,7 +1381,7 @@ export class Publisher {
}
}
// A declared end goes out while lower groups are still being produced.
if (!endSent && hooks.subscriberEnd(track) !== undefined) {
if (!endSent && hooks.subscriberEnd(track) !== undefined && declareEarly()) {
if (!(await sendEnd())) return false;
continue;
}
Expand All @@ -1369,13 +1390,20 @@ export class Publisher {
case "boundary":
// The producer finished but is still holding groups above the cap.
// Declare the boundary, then wait for an update to release them.
if (!endSent) {
if (!endSent && declareEarly()) {
if (!(await sendEnd())) return false;
continue;
}
await waitForSubscription(controls, track, drops);
continue;
case "done": {
const endError = hooks.subscriberEndError(track);
if (
endError &&
!hasEndError(this.version) &&
(bounds.endGroup === undefined || bounds.endGroup + 1 >= boundary())
)
throw endError;
// Name every sequence from the start to the end this subscription never
// got. A cap below the end bounds what the subscriber is owed, as on its side.
if (startSequence !== undefined) {
Expand Down
11 changes: 9 additions & 2 deletions js/net/src/lite/subscribe.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,15 @@ test("SubscribeEnd round-trips on draft-05", async () => {
expect(got.end.group).toBe(7);
});

test("SubscribeEnd has no stream count on draft-07", async () => {
expect(await encode(Version.DRAFT_07, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 1, 7]));
test("SubscribeEnd carries its error on draft-07", async () => {
// Type, length, exclusive final group, then the error code + 1, 0 for a clean end.
expect(await encode(Version.DRAFT_07, { end: new SubscribeEnd(7) })).toEqual(new Uint8Array([1, 2, 7, 0]));
expect(await encode(Version.DRAFT_07, { end: new SubscribeEnd(7, 71) })).toEqual(new Uint8Array([1, 2, 7, 72]));
const got = await responseRoundtrip(Version.DRAFT_07, { end: new SubscribeEnd(7, 71) });
if (!("end" in got)) throw new Error("expected end");
expect([got.end.group, got.end.error]).toEqual([7, 71]);
// An older version has nowhere to carry it.
await expect(encode(Version.DRAFT_06, { end: new SubscribeEnd(7, 71) })).rejects.toThrow();
});

test("SubscribeDrop is restored on draft-07", async () => {
Expand Down
28 changes: 24 additions & 4 deletions js/net/src/lite/subscribe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import * as Message from "./message.ts";
import { fromWire, toWire } from "./priority.ts";
import {
hasDropCode,
hasEndError,
hasFrameBounds,
hasGroupOrder,
hasOrder,
Expand Down Expand Up @@ -707,19 +708,38 @@ export class SubscribeStart {
export class SubscribeEnd {
/** The exclusive final group sequence: the first sequence that will never be produced. */
group: number;
/** The stream error code the track ended with, if it did not end cleanly. Draft-07+ only. */
error?: number;

constructor(group: number) {
constructor(group: number, error?: number) {
this.group = group;
this.error = error;
}

async encode(w: Writer, _version: Version): Promise<void> {
/** The error the track ended with, or `undefined` for a clean end. */
cause(): Error | undefined {
if (this.error === undefined) return undefined;
if (this.error > 0xffff_ffff)
return new StreamError(StreamCode.Internal, { message: `invalid end code: ${this.error}` });
return fromTransport({ source: "stream", streamErrorCode: this.error });
}

async encode(w: Writer, version: Version): Promise<void> {
return Message.encode(w, async (w) => {
await w.u53(this.group);
// The code + 1, so 0 is a clean end.
if (hasEndError(version)) await w.u53(this.error === undefined ? 0 : this.error + 1);
else if (this.error !== undefined) throw new Error("an errored end needs draft-07");
});
}

static async decode(r: Reader, _version: Version): Promise<SubscribeEnd> {
return Message.decode(r, async (r) => new SubscribeEnd(await r.u53()));
static async decode(r: Reader, version: Version): Promise<SubscribeEnd> {
return Message.decode(r, async (r) => {
const group = await r.u53();
if (!hasEndError(version)) return new SubscribeEnd(group);
const code = await r.u53();
return new SubscribeEnd(group, code === 0 ? undefined : code - 1);
});
}
}

Expand Down
2 changes: 1 addition & 1 deletion js/net/src/lite/subscriber.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1513,7 +1513,7 @@ export class Subscriber {
try {
// Lower groups may still be on the wire behind a higher one, so readers hold
// at a hole below the end until the tail settles.
hooks.endTrack(entry.track, { group: entry.end, frame: 0 });
hooks.endTrack(entry.track, { group: entry.end, frame: 0 }, resp.end.cause());
} catch (err) {
// lite-05 specified an inclusive end, and @moq/net 0.1.3 to 0.1.9 sent one, so
// there an end below a received group only costs the early boundary: the FIN
Expand Down
19 changes: 19 additions & 0 deletions js/net/src/lite/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -572,6 +572,25 @@ export function hasDropCode(version: Version): boolean {
}
}

/**
* Whether SUBSCRIBE_END carries the error the track ended with, so an errored end keeps its
* history across a hop. Added in lite-07; earlier versions can only reset.
*/
export function hasEndError(version: Version): boolean {
// Explicitly list older versions so future versions keep the lite-07+ behavior.
switch (version) {
case Version.DRAFT_01:
case Version.DRAFT_02:
case Version.DRAFT_03:
case Version.DRAFT_04:
case Version.DRAFT_05:
case Version.DRAFT_06:
return false;
default:
return true;
}
}

/**
* The timescale the wire carries for a track with `timescale`, or `undefined` when its frames
* and datagrams carry no timestamp. Lite-05 and lite-06 always carry one, so an untimed track
Expand Down
Loading