Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions doc/concept/layer/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ If the peer doesn't have the broadcast/track, they will get an error.
Otherwise, the subscription is active and will stay open until closed by the publisher (possibly with an error).

A track is broken into **groups**, each with an increasing ID.
Conceptually, these are join points, and new subscriptions will always start at the latest group.
Conceptually, these are join points. A new subscription starts at the oldest cached group
within its Subscriber Max Latency budget; the default zero budget starts at the latest group.
Groups are delivered independently and potentially out of order, so you should have some logic to reorder or skip during congestion.
A group is closed when finished or aborted with an error (ex. during congestion).

Expand Down Expand Up @@ -111,6 +112,7 @@ Each Subscription consists of a few properties:
- **Track Priority**: A value between 0 and 255. Tracks with higher priority will be delivered first.
- **Group Order**: The order in which groups are delivered. Defaults to descending; higher IDs are delivered first.
- **Subscriber Max Latency**: The maximum age of a non-latest group before it is skipped. Defaults to zero, so stale groups are skipped immediately.
- **Group Start**: An optional absolute floor on the publisher's choice. When omitted, the publisher chooses the starting group using Subscriber Max Latency with an implicit floor of group 0. A supplied floor restricts that choice but does not request history by itself.

The publisher also keeps old groups around for a best-effort **Publisher Max Latency** cache window so relays and late subscribers can still fetch them. This defaults to 5 seconds.
The subscriber's maximum latency is bounded by this window: a group can't be waited for longer than it's actually kept around.
Expand Down Expand Up @@ -168,7 +170,7 @@ But if a publisher needs a feature, then the subscriber needs it too, so you can
- **No Request IDs**: A bidirectional stream for each request to avoid HoLB. (NOTE: likely to be upstreamed into moq-transport)
- **No Push**: A subscriber must explicitly subscribe to each track.
- **Single-group FETCH only (lite-05+)**: Fetch one complete group by sequence. Ranges and joining fetches are not supported.
- **No Joining Fetch**: Subscriptions start at the latest group, not the latest frame.
- **No Joining Fetch**: Subscriptions start at a group boundary selected by Subscriber Max Latency, never in the middle of a group.
- **No sub-groups**: SVC layers should be separate tracks.
- **No gaps**: Makes life much easier for the relay and every application.
- **No object properties**: Encode your metadata into the frame payload.
Expand Down
9 changes: 5 additions & 4 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -844,16 +844,16 @@ This is a delivery-time preference, not a retention rule: the publisher MAY stil
See the [Expiration](#expiration) section for more information.

**Group Start**:
The first group to deliver.
A value of 0 means the latest group (default).
An optional lower bound on the publisher-selected starting group.
A value of 0 means absent (default): the publisher chooses using `Subscriber Max Latency` with an implicit floor of absolute group 0.
A non-zero value is the absolute group sequence + 1.
Comment thread
kixelated marked this conversation as resolved.
Resolved

**Group End**:
The last group to deliver (inclusive).
A value of 0 means unbounded (default).
A non-zero value is the absolute group sequence + 1.

`Group Start` and `Group End` are offset by 1 only so 0 can mean "absent"; every other group field in this document is a plain absolute sequence.
`Group Start` and `Group End` are offset by 1 only so 0 can mean "absent"; every other group field in this document is a plain absolute sequence. An absent `Group Start` leaves the starting position to the publisher, bounded below by absolute group 0. With the default `Subscriber Max Latency` of 0 the publisher chooses the latest group, because every older group is immediately stale.


## SUBSCRIBE_UPDATE
Expand Down Expand Up @@ -965,7 +965,7 @@ Set to 0x0 to indicate a SUBSCRIBE_OK message.
**Group**:
The absolute sequence number of the first group that will be delivered.
It MUST be greater than or equal to the requested start group; any groups in between are unavailable and implicitly dropped, with no separate SUBSCRIBE_DROP required.
A subscriber that requested the latest group learns the resolved sequence here.
A subscriber with no explicit floor learns the age-resolved sequence here.

## SUBSCRIBE_END {#subscribe-end}
A SUBSCRIBE_END message is sent by the publisher to signal that no group at or after a given sequence will be produced.
Expand Down Expand Up @@ -1138,6 +1138,7 @@ The `Message Length` describes the payload size on the wire.
# Appendix A: Changelog

## moq-lite-06
- Made an absent `Group Start` publisher-selected with an implicit floor of group 0; `Subscriber Max Latency` determines the starting group.
- Moved the Qmux-over-WebSocket binding details to draft-lcurley-qmux-websocket; the binding itself is unchanged.
- Extended the SETUP `Path` parameter to carry the URI query: a client appends `?` and the query component after the path, matching moq-transport's PATH option. The credential a deployment puts in the query was previously unrepresentable on a binding with no request URI.
- Allowed an empty SETUP `Path` parameter, equivalent to omitting it; both request the server's default path. Previously an empty value was a protocol violation, which made the two ways of asking for the default disagree.
Expand Down
3 changes: 2 additions & 1 deletion js/json/src/snapshot/compression.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,13 @@ async function groupCount(track: Track.Subscriber): Promise<number> {
test("compressed snapshot per group round-trips", async () => {
const track = new Track.Producer("test");
const producer = new Producer<Value>({ track, deltaRatio: 0, compression: true });
const subscriber = track.subscribe();
producer.update({ a: 1 });
producer.update({ a: 2 });
producer.finish();

// Deltas off: one compressed snapshot per group, reconstructed in order.
expect(await drainCompressed(track.subscribe())).toEqual([{ a: 1 }, { a: 2 }]);
expect(await drainCompressed(subscriber)).toEqual([{ a: 1 }, { a: 2 }]);
});

test("compressed live consumer sees each update in order", async () => {
Expand Down
18 changes: 12 additions & 6 deletions js/json/src/snapshot/snapshot.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ async function drain(track: Track.Subscriber): Promise<Value[]> {
}

// Inspect the published layout via the public API: the frame count of each group, in order.
// The track must be finished first so group/frame reads terminate.
// The track must be finished first so group/frame reads terminate, and the subscriber has to
// exist before the first update: a fresh one starts at the live edge, not at group 0.
async function structure(track: Track.Subscriber): Promise<number[]> {
const counts: number[] = [];
for (;;) {
Expand All @@ -30,24 +31,26 @@ async function structure(track: Track.Subscriber): Promise<number[]> {
test("deltas off: a snapshot group per change", async () => {
const track = new Track.Producer("test");
const producer = new Producer<Value>({ track, deltaRatio: 0 });
const subscriber = track.subscribe();
producer.update({ a: 1 });
producer.update({ a: 2 });
producer.finish();

// Two changes => two single-frame snapshot groups, reconstructed in order.
expect(await drain(track.subscribe())).toEqual([{ a: 1 }, { a: 2 }]);
expect(await drain(subscriber)).toEqual([{ a: 1 }, { a: 2 }]);
});

test("deltaRatio 0 disables deltas, like off", async () => {
const track = new Track.Producer("test");
const producer = new Producer<Value>({ track, deltaRatio: 0 });
const subscriber = track.subscribe();
producer.update({ a: 1 });
producer.update({ a: 2 });
producer.finish();

// `0` is treated as off, not a degenerate "enabled" value that keeps the group open: each change
// is its own single-frame snapshot group.
expect(await structure(track.subscribe())).toEqual([1, 1]);
expect(await structure(subscriber)).toEqual([1, 1]);
});

test("live consumer sees each update", async () => {
Expand Down Expand Up @@ -155,13 +158,14 @@ test("tight ratio rolls snapshots", async () => {
// the deltas already written, so the delta that tips the group over budget still lands (a one-frame
// overshoot): group 0 takes two deltas (14 bytes) before the fourth update rolls group 1.
const producer = new Producer<Value>({ track, deltaRatio: 1 });
const subscriber = track.subscribe();
producer.update({ a: 1 }); // snapshot, group 0
producer.update({ a: 2 }); // delta, group 0 (deltas = 7)
producer.update({ a: 3 }); // delta, group 0 (deltas = 14, now over budget)
producer.update({ a: 4 }); // budget already exceeded, rolls group 1
producer.finish();

expect(await structure(track.subscribe())).toEqual([3, 1]);
expect(await structure(subscriber)).toEqual([3, 1]);
});

test("deltas stay within ratio times snapshot", async () => {
Expand All @@ -172,10 +176,11 @@ test("deltas stay within ratio times snapshot", async () => {
// until they first exceed 56 (nine deltas = 63 bytes) and the next update rolls (a one-frame
// overshoot past the 56-byte budget).
const producer = new Producer<Value>({ track, deltaRatio: 8 });
const subscriber = track.subscribe();
for (let n = 0; n <= 10; n++) producer.update({ n });
producer.finish();

expect(await structure(track.subscribe())).toEqual([10, 1]);
expect(await structure(subscriber)).toEqual([10, 1]);
});

test("array change is a wholesale delta", async () => {
Expand Down Expand Up @@ -206,11 +211,12 @@ test("late joiner collapses a buffered backlog to the latest value", async () =>
test("frame cap rolls snapshot", async () => {
const track = new Track.Producer("test");
const producer = new Producer<Value>({ track, deltaRatio: 1_000_000 });
const subscriber = track.subscribe();
// First update is the snapshot; deltas fill the group until the frame cap forces a roll.
for (let i = 0; i <= 256; i++) {
producer.update({ n: i });
}
producer.finish();

expect(await structure(track.subscribe())).toEqual([256, 1]);
expect(await structure(subscriber)).toEqual([256, 1]);
});
4 changes: 4 additions & 0 deletions js/net/src/broadcast.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ async function fetchGroup(
options: track.FetchGroupOptions = {},
): Promise<GroupConsumer> {
const subscriber = subscribe(state, name, { priority: options.priority });
// FETCH names an exact cached group, so it must not inherit the live
// subscription cursor selected from max latency (zero selects the latest).
subscriber.startAt(sequence);
subscriber.endAt(sequence);
try {
for (;;) {
const group = await subscriber.recvGroup();
Expand Down
2 changes: 1 addition & 1 deletion js/net/src/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ test("integration: lite applies initial and updated group bounds", async () => {
const remote = client.consume(Path.from("test"));
const subscriber = remote
.track("video")
.subscribe({ startGroup: INITIAL_START_GROUP, endGroup: INITIAL_END_GROUP });
.subscribe({ latencyMax: 1000, startGroup: INITIAL_START_GROUP, endGroup: INITIAL_END_GROUP });
try {
expect((await subscriber.nextGroup())?.sequence).toBe(INITIAL_START_GROUP);
expect((await subscriber.nextGroup())?.sequence).toBe(INITIAL_END_GROUP);
Expand Down
7 changes: 6 additions & 1 deletion js/net/src/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
*
* @module
*/
import type { Producer, Request } from "./track.ts";
import type { Producer, Request, Subscriber } from "./track.ts";

/** The next group or datagram sequence shared by dynamic producers of one broadcast track. */
export interface TrackSequence {
Expand All @@ -29,8 +29,13 @@ export interface TrackRequestOptions {
export const hooks: {
/** Mint a track {@link Request}; assigned by `track.ts`. */
makeRequest: (options: TrackRequestOptions) => Request;
/** Re-resolve a subscriber's publisher-selected start, preserving an announced floor. */
positionSubscriber: (subscriber: Subscriber, announcedStart?: number) => void;
} = {
makeRequest: () => {
throw new Error("track.ts not loaded");
},
positionSubscriber: () => {
throw new Error("track.ts not loaded");
},
};
56 changes: 51 additions & 5 deletions js/net/src/lite/publisher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -358,13 +358,22 @@ test("lite draft-05: the fetch response ranks the publisher's own writes", async

// Opens a served subscription and returns the machinery to write groups and observe
// which of them the publisher put on the wire (each group gets its own uni stream).
async function servedSubscription(options: { startGroup?: number } = {}) {
async function servedSubscription(
options: { startGroup?: number; maxLatency?: number; version?: Version; initialGroups?: number[] } = {},
) {
const version = options.version ?? Version.DRAFT_05;
const pair = createMockTransportPair(ALPN_05);
const publisher = new Publisher(pair.server, Version.DRAFT_05, randomOrigin());
const publisher = new Publisher(pair.server, version, randomOrigin());

const broadcast = new BroadcastProducer();
const track = broadcast.createTrack("video");
publisher.publish(Path.from("test"), broadcast);
for (const sequence of options.initialGroups ?? []) {
const group = new GroupProducer(sequence);
group.writeString("cached");
group.close();
track.writeGroup(group);
}

const client = await Stream.open(pair.client);
const server = await Stream.accept(pair.server);
Expand All @@ -375,6 +384,7 @@ async function servedSubscription(options: { startGroup?: number } = {}) {
broadcast: Path.from("test"),
track: "video",
priority: 0,
maxLatency: options.maxLatency,
startGroup: options.startGroup,
});
void publisher.runSubscribe(msg, server);
Expand All @@ -383,6 +393,19 @@ async function servedSubscription(options: { startGroup?: number } = {}) {

return {
client,
async update(options: { maxLatency?: number; startGroup?: number; endGroup?: number }) {
await new SubscribeUpdate({ priority: 0, ...options }).encode(client.writer, version);
for (;;) {
const current = track.subscription.peek();
if (
current?.latencyMax === (options.maxLatency ?? 0) &&
current.startGroup === options.startGroup &&
current.endGroup === options.endGroup
)
return;
await track.subscription.changed();
}
},
serve(sequence: number) {
const group = new GroupProducer(sequence);
group.writeString("hello");
Expand All @@ -405,11 +428,32 @@ async function servedSubscription(options: { startGroup?: number } = {}) {
};
}

test("every lite version resolves an omitted start from the zero max-latency budget", async () => {
for (const version of Object.values(Version)) {
const sub = await servedSubscription({ version, initialGroups: [0, 1] });
try {
expect(await sub.servedSequence()).toBe(1);
} finally {
sub.close();
}
}
});

test("lite draft-05: an update re-resolves an absent start from max latency", async () => {
const sub = await servedSubscription({ startGroup: 10, initialGroups: [0, 1] });
try {
await sub.update({ maxLatency: 0 });
expect(await sub.servedSequence()).toBe(1);
} finally {
sub.close();
}
});

// A relay can ingest back-to-back groups micro-reordered (the upstream leg sends
// newest-first). The older group is cached and in demand, so serving must still
// deliver it; a sequence cursor would skip it permanently.
test("lite draft-05: a late-arriving older group is still served", async () => {
const sub = await servedSubscription({ startGroup: 1 });
const sub = await servedSubscription({ startGroup: 1, maxLatency: 1 });
try {
// Serve one at a time so arrival order at the publisher is the order given.
sub.serve(1);
Expand Down Expand Up @@ -439,9 +483,11 @@ test("lite draft-05: a straggler below the announced start group is not served",
if (!("start" in resp)) throw new Error("expected SUBSCRIBE_START");
expect(resp.start.group).toBe(2);

// A straggler below the announced start never reaches the wire: the next
// stream the publisher opens is group 3's.
// Re-resolving an update with a cap below the announced start must not
// lower the promise already sent to the peer.
await sub.update({ maxLatency: 60_000, endGroup: 1 });
sub.serve(1);
await sub.update({ maxLatency: 0 });
sub.serve(3);
expect(await sub.servedSequence()).toBe(3);
} finally {
Expand Down
25 changes: 20 additions & 5 deletions js/net/src/lite/publisher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { type Dispose, Signal } from "@moq/signals";
import type * as broadcast from "../broadcast.ts";
import { error, reason } from "../error.ts";
import type * as group from "../group.ts";
import { hooks } from "../internal.ts";
import type { Origin } from "../origin.ts";
import * as Path from "../path.ts";
import { type Stream, Writer } from "../stream.ts";
Expand Down Expand Up @@ -278,8 +279,6 @@ export class Publisher {
startGroup: msg.startGroup,
endGroup: msg.endGroup,
});
Comment on lines 279 to 281

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve legacy start semantics before lite-06

When a lite-03 through lite-05 peer sends a nonzero maxLatency and omits startGroup, passing those options directly into the new model makes the publisher select cached history. Those negotiated drafts defined an absent Group Start as the latest group, while the new age-resolved meaning is recorded only for moq-lite-06, so existing clients can unexpectedly restart behind live and replay media. Gate initial and update-time repositioning on lite-06 in both publishers, retaining the previous latest-group behavior for older versions. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L207-L207

Useful? React with 👍 / 👎.

const startGroup = msg.startGroup ?? track.latest();
if (startGroup !== undefined) track.startAt(startGroup);
track.endAt(msg.endGroup);

// The best-effort datagram loop, started once serving begins. It parks when the
Expand Down Expand Up @@ -316,7 +315,8 @@ export class Publisher {

console.debug(`publish ok: broadcast=${msg.broadcast} track=${track.name}`);

const serving = this.#runTrack(msg.id, msg.broadcast, track, stream.writer, timescale);
const range: { start?: number } = {};
const serving = this.#runTrack(msg.id, msg.broadcast, track, stream.writer, timescale, range);

// Serve datagrams concurrently with groups whenever the transport carries them
// (the writer exists iff so). No group fallback: otherwise they simply aren't sent.
Expand All @@ -332,14 +332,21 @@ export class Publisher {

if (result instanceof SubscribeUpdate) {
console.debug(`subscribe update: broadcast=${msg.broadcast} track=${track.name}`);
const previous = track.subscription.peek();
track.update({
priority: result.priority,
ordered: result.ordered,
latencyMax: result.maxLatency,
startGroup: result.startGroup,
endGroup: result.endGroup,
});
if (result.startGroup !== undefined) track.startAt(result.startGroup);
if (
previous?.latencyMax !== result.maxLatency ||
previous.startGroup !== result.startGroup ||
previous.endGroup !== result.endGroup
) {
hooks.positionSubscriber(track, range.start);
}
track.endAt(result.endGroup);
}
}
Expand Down Expand Up @@ -408,7 +415,14 @@ export class Publisher {
*
* @internal
*/
async #runTrack(sub: bigint, broadcast: Path.Valid, track: track.Subscriber, stream: Writer, timescale: Timescale) {
async #runTrack(
sub: bigint,
broadcast: Path.Valid,
track: track.Subscriber,
stream: Writer,
timescale: Timescale,
range: { start?: number },
) {
// Lite-05+ resolves the range on the subscribe stream: SUBSCRIBE_START once the
// first group is known, SUBSCRIBE_END when the track finishes.
const emitRange = supportsTrackStream(this.version);
Expand Down Expand Up @@ -453,6 +467,7 @@ export class Publisher {

if (emitRange && !startSent) {
startSent = true;
range.start = group.sequence;
// SUBSCRIBE_START promises nothing below this sequence will be delivered.
// Arrival-order serving could later surface a straggler below the first
// group, so pin the floor to what was announced.
Expand Down
Loading
Loading