Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/concept/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,10 @@ Each subscription carries the knobs that decide behavior under congestion:
| **Max delay** | How long a non-latest group stays worth delivering before it is skipped. Zero means "live edge only", and raising it is also what asks for history. |
| **Ranges** | Which groups to ask for: from a group, or a number of groups back from the latest, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. |

On moq-lite 05 and later, a publisher explicitly reports groups that will not arrive.
This lets a finished subscription settle without waiting for missing groups.
Datagram delivery stays best effort.

Ranges are a request, not a filter. A subscriber reads from where its lowest
range starts, and may also get groups another subscriber of the same track
asked for. A relay merges its subscribers' ranges into one upstream request,
Expand Down
42 changes: 28 additions & 14 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,9 +512,9 @@ A route claims capability rather than inventory, so a subscription for a covered
The track's immutable publisher properties are not carried here; they are fetched once via a [Track Stream](#track-stream).
The subscriber needs the track's TRACK_INFO (notably its timescale) to interpret FRAME messages, and MAY open the Track and Subscribe streams concurrently, buffering frames until it arrives.

The publisher sends SUBSCRIBE_OK once the absolute start position is resolved, and SUBSCRIBE_END once no further groups will be produced and every Group Stream it opens for the subscription has been opened (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)).
The publisher closes its side of the stream (FIN) after SUBSCRIBE_END, once every counted Group Stream has finished or been reset.
After reading the publisher's FIN, the subscriber MUST close its side (FIN) once its tail accounting has settled: every received Group Stream has been read to its FIN, reset, or dropped, and every remaining counted stream has arrived or exhausted the subscriber's grace period.
The publisher sends SUBSCRIBE_OK once the absolute start position is resolved, and SUBSCRIBE_END as soon as the exclusive final group is known (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)).
The publisher closes its side of the stream (FIN) after SUBSCRIBE_END, once every Group Stream has finished or been reset.
After reading the publisher's FIN, the subscriber MUST close its side (FIN) once its tail accounting has settled: every received Group Stream has been read to its FIN, reset, or dropped, and every sequence in range has arrived, been named by SUBSCRIBE_DROP, or exhausted the subscriber's grace period.
A publisher closing its session gracefully MUST wait for the subscriber's FIN or reset on every Subscribe Stream before closing the connection; a transport acknowledgement alone does not confirm the subscriber read the tail.
A subscriber MAY cancel by closing its side (FIN); the publisher then closes its side (FIN) without owing SUBSCRIBE_END.
Unbounded subscriptions stay open until SUBSCRIBE_END, and either endpoint MAY reset the stream at any time.
Expand Down Expand Up @@ -1321,7 +1321,6 @@ SUBSCRIBE_END Message {
Type (i) = 0x1
Message Length (i)
Group (i)
Stream Count (i)
}
~~~

Expand All @@ -1333,14 +1332,30 @@ The exclusive end of the range: the absolute sequence number of the first group
A value of 0 means the track ended before producing any groups.
The subscriber MUST NOT wait for any group at or after this sequence.

**Stream Count**:
The number of Group Streams the publisher opened for this subscription, whether they finished or were reset.
A group that was skipped, never produced, delivered only as a datagram, or given up before its stream opened is not counted.
A relay counts the Group Streams it opened itself, never the count it received upstream.
The publisher SHOULD send SUBSCRIBE_END as soon as the exclusive final group is known, even while lower groups are still in flight.

The publisher MUST NOT send SUBSCRIBE_END until every Group Stream it will open for the subscription has been opened, so the count is final; it does not wait for them to finish.
The subscriber has received every Group Stream once it has read the header of `Stream Count` of them, which MAY happen after SUBSCRIBE_END or the FIN since streams are not ordered.
A Group Stream reset before its header arrived is never seen, so a subscriber SHOULD bound how long it waits for the rest, for example by `Subscriber Max Age`.
## SUBSCRIBE_DROP {#subscribe-drop}

A publisher names groups it will not deliver with SUBSCRIBE_DROP on the Subscribe Stream.

~~~
SUBSCRIBE_DROP Message {
Type (i) = 0x2
Message Length (i)
Group Start (i)
Group End (i)
Error Code (i)
}
~~~

**Group Start** and **Group End** are the inclusive range of group sequences the publisher gave up.
**Error Code** is an application-specific error code; 0 means the groups are simply unavailable.

For every sequence in the subscription's range, the publisher MUST send a Group Stream, send a datagram, or name it in SUBSCRIBE_DROP.
This includes sequences it skipped and streams reset before their headers were delivered.
A reliably delivered reset retaining the Group Stream header accounts for that group instead.
The publisher SHOULD send SUBSCRIBE_DROP as soon as it gives a sequence up, without waiting for SUBSCRIBE_END.
A datagram counts as sent even if it is lost; a subscriber SHOULD bound its wait for unaccounted sequences, for example by Subscriber Max Age.

## FETCH
FETCH is sent by a subscriber to request a single group from a track.
Expand Down Expand Up @@ -1618,9 +1633,8 @@ The `Message Length` describes the payload size on the wire.
- Assigned `moq-lite-07-wip` as this draft's protocol identifier until it is finalized as `moq-lite-07`.
- Switched every variable-length integer, including SETUP parameter values, from QUIC's two-bit length prefix to moq-transport's leading-ones encoding, widening the range to 64 bits.
- Hid routes with a `.`-prefixed segment below the requested prefix from announce discovery, and added the ANNOUNCE_REQUEST `Hidden` field to opt in.
- Added `Stream Count` to SUBSCRIBE_END: the number of Group Streams opened for the subscription. SUBSCRIBE_END is now sent once every counted Group Stream has opened, rather than as soon as the final group is known.
- Removed SUBSCRIBE_DROP and its type 0x2; a group without a Group Stream is not counted.
- The Subscribe Stream FIN now follows once every counted Group Stream has finished or been reset.
- Restored SUBSCRIBE_DROP (type 0x2), replacing Stream Count on SUBSCRIBE_END. Every sequence is accounted for by a Group Stream, a datagram, or a drop; a publisher SHOULD send a drop as soon as it gives a sequence up.
- SUBSCRIBE_END declares the final boundary while lower groups can still be in flight; the Subscribe Stream FIN follows once every Group Stream has finished or been reset.
- Added the Spread Hash tie-break after the shortest path: a hash of the requested path and the route's Hop IDs, so equal-cost advertisers of one prefix share its paths.
- Added a local link preference to route selection, after the newest Epoch and before the 0 Hop ID check: the higher preference a subscriber assigns to the arriving link wins, and an unassigned link is neutral. Not carried on the wire.
- Added announce compression: ANNOUNCE_START gains `Path Base` and `Path Keep` to copy the head of a live advertisement's suffix, and ANNOUNCE_START and ANNOUNCE_UPDATE gain `Hop Base` and `Hop Keep` to copy the tail of a live advertisement's Hop ID list.
Expand Down
2 changes: 1 addition & 1 deletion js/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ test-changed list:
coverage package="":
python3 ../sh/js/coverage.py '{{ package }}'

# Run one net benchmark: broadcasts, forward, requests, reader, frames, track, resume, interests, observed, varint, lite-varint, priority, or request-ids.
# Run one net benchmark: broadcasts, forward, requests, reader, frames, track, resume, interests, observed, varint, lite-varint, priority, request-ids, or subscribe-drop.
bench name:
bun install --frozen-lockfile
bun net/bench/{{ name }}.ts
Expand Down
5 changes: 5 additions & 0 deletions js/net/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first.

## Unreleased

### Lite-07 accounts for groups with `SUBSCRIBE_DROP`

Construct `SubscribeEnd(group)` without a stream count; remove reads of `.streams` and calls to `hasStreamCount()`.
The unpublished `moq-lite-07-wip` wire restores `SUBSCRIBE_DROP` and removes Stream Count from `SUBSCRIBE_END`. Update both ends together; published lite versions keep their framing.

### `webtransport.protocols` is `versions` (#5979)

Move the list to the top-level `versions` of `connect` or `Connection`, spelled as version names: `moq-transport-22`, not the ALPN `moqt-22`; lite names are unchanged.
Expand Down
102 changes: 102 additions & 0 deletions js/net/bench/subscribe-drop.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
/** Sweep failed group sends over independent publishers and subscribers. */
import { Milli } from "@moq/clock";
import { Producer as GroupProducer } from "../src/group.ts";
import { randomHop } from "../src/hop.ts";
import { Publisher } from "../src/lite/publisher.ts";
import { decodeSubscribeResponseMaybe, Subscribe } from "../src/lite/subscribe.ts";
import { ALPN_07_WIP, Version } from "../src/lite/version.ts";
import { createMockTransportPair, textFrame } from "../src/mock.ts";
import { Producer as OriginProducer } from "../src/origin.ts";
import * as Path from "../src/path.ts";
import { Stream } from "../src/stream.ts";
import { Timescale } from "../src/time.ts";

const groups = 128;
const counts = [1, 4, 16];
const reps = 3;
const debug = console.debug;
console.debug = () => {};
try {
console.log("publishers,subscribers,groups,drop_us");
for (const publishers of counts) {
for (const subscribers of counts) {
const times: number[] = [];
for (let rep = 0; rep < reps; rep++) {
const origins = Array.from({ length: publishers }, () => {
const origin = new OriginProducer();
const broadcast = origin.createBroadcast(Path.from("bench"));
broadcast.announce();
const track = broadcast.createTrack("video", { timescale: Timescale.MILLI });
for (let sequence = 0; sequence < groups; sequence++) {
const group = new GroupProducer(sequence);
group.writeFrame(textFrame("frame"));
group.close();
track.writeGroup(group);
}
track.close();
return { origin, broadcast };
});
// Include connection setup, excluding group creation and cache preparation.
const start = performance.now();
await Promise.all(
origins.flatMap(({ origin }) =>
Array.from({ length: subscribers }, async () => {
const pair = createMockTransportPair(ALPN_07_WIP);
pair.server.createUnidirectionalStream = async () => {
throw new Error("no stream slot");
};
const publisher = new Publisher(
pair.server,
Version.DRAFT_07,
randomHop(),
origin.consume(),
);
const client = await Stream.open(pair.client, { version: Version.DRAFT_07 });
const server = await Stream.accept(pair.server, Version.DRAFT_07);
if (!server) throw new Error("missing subscribe stream");
const serving = publisher.runSubscribe(
new Subscribe({
id: 0n,
broadcast: Path.from("bench"),
track: "video",
priority: 0,
maxDelay: Milli(30_000),
startGroup: 0,
}),
server,
);
try {
let dropped = 0;
for (;;) {
const response = await decodeSubscribeResponseMaybe(
client.reader,
Version.DRAFT_07,
);
if (!response) break;
if ("drop" in response) dropped += response.drop.end - response.drop.start + 1;
}
if (dropped !== groups) throw new Error(`dropped ${dropped}, expected ${groups}`);
client.writer.close();
await serving;
} finally {
publisher.close();
client.close();
pair.client.close();
pair.server.close();
}
}),
),
);
times.push(((performance.now() - start) * 1000) / (publishers * subscribers * groups));
for (const { origin, broadcast } of origins) {
broadcast.close();
origin.close();
}
}
times.sort((a, b) => a - b);
console.log(`${publishers},${subscribers},${groups},${times[Math.floor(reps / 2)].toFixed(2)}`);
}
}
} finally {
console.debug = debug;
}
Loading
Loading