Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
77e3b49
chore(quest): claim quest/m1/transport-frame-offsets.md
kixelated Oct 27, 2026
74130d9
fix(net): preserve transport offsets and bounded fetch provenance
kixelated Oct 27, 2026
5796ab6
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
2420a86
Share canonical group content across transport deliveries
kixelated Oct 27, 2026
d73e28b
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
1bef496
Preserve canonical storage and inflater delivery lifetimes
kixelated Oct 27, 2026
43b3e4d
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
368a2f4
Bound canonical ownership discovery and final cleanup
kixelated Oct 27, 2026
4a4b6c1
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
d16f597
Complete transport frame offset quest
kixelated Oct 27, 2026
21a6496
Align dependent plans with transport frame ranges
kixelated Oct 27, 2026
761c198
Preserve transport storage controls across feature shapes
kixelated Oct 27, 2026
c1dea63
Keep validation checkpoint within quest plan
kixelated Oct 27, 2026
8d4d527
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
1957697
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
340c911
quest: finish transport frame offsets
kixelated Oct 27, 2026
ba6160b
Restore the caller's frame range on a routed fetch
kixelated Oct 27, 2026
088d7fe
FIN an empty received tail served from its start
kixelated Oct 27, 2026
ba6bd8e
Merge remote-tracking branch 'origin/main' into worktree-agent-ac4929…
kixelated Oct 27, 2026
7dda6cf
fix(net): keep bounded copies off whole-group readers
kixelated Oct 27, 2026
5bc8e0b
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
8691256
fix(moq-net): end uncached copies on track abort
kixelated Oct 27, 2026
980c59c
fix(moq-net): keep finished bounded copies on track abort
kixelated Oct 27, 2026
5165e4d
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 27, 2026
65eaf83
fix(moq-net): let a finished bounded copy settle the track end
kixelated Oct 28, 2026
12d3836
Merge origin/main into quest/m1/transport-frame-offsets
kixelated Oct 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ futures = "0.3"
getrandom = { version = "0.4", features = ["wasm_js"] }
h2 = "0.4"
hang = { version = "0.21.11", path = "rs/hang" }
hashbrown = { version = "0.17.1", default-features = false }
hashlink = "0.12.2"
hex = "0.4"
# HMAC-SHA256 for the mDNS membership proofs (moq-tokio's `mdns` feature).
Expand Down
13 changes: 6 additions & 7 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,11 +213,10 @@ A partial Group is only ever delivered to a subscriber that asked for one.
A publisher that cannot serve a Group from the frame the subscription names MUST skip that Group entirely and resolve the subscription to a later one, rather than deliver the part it holds.
The resolved start is therefore always either exactly the requested Position or the beginning of a later Group, never some third Position the subscriber did not choose.

The asymmetry with Groups is deliberate.
A Group is the unit of decodability: dropping leading *Groups* leaves a stream the application can still decode, whereas dropping leading *Frames* often does not, whether because the Group opens with a keyframe the rest depends on or because its compression state lives in the frames that went missing.
A subscription therefore starts at a Group's first frame, and a subscriber asks for a partial Group only to finish one whose earlier frames it already holds, from this publisher or another.

Frame indices are only meaningful relative to a Group the subscriber has already begun receiving, so a subscriber MUST start a Range at frame 0 unless it already holds every frame of that group below its start.
A Group is the unit of decodability: its earlier Frames may carry a keyframe or compression state the later Frames need.
Applications request whole Groups, while a transport may request a partial Group for route continuation, hole fills, or front-eviction refills.
A subscriber MAY start a Range at any frame index, whether or not it retains the earlier Frames.
A relay that holds none of a requested Group SHOULD request it upstream from frame 0 and apply the downstream bounds locally, retaining a copy usable by whole-Group readers and compression.

# Flow
This section outlines the flow of messages within a moq-lite session.
Expand Down Expand Up @@ -1308,7 +1307,7 @@ A floor above the latest group simply waits there: that is a resumed subscriptio
Reaching back is best-effort, not a guarantee that the groups still exist; see `Publisher Max Age` in [TRACK_INFO](#track-info).

An Absolute `Start` past frame 0 delivers its group from that frame; a start resolved at a later group is always delivered from frame 0.
A subscriber MUST start at frame 0 unless it already holds every frame below it, from this publisher or another, since a Group is the unit of decodability (see [Positions](#positions)).
A subscriber MAY name any start frame, as described in [Positions](#positions).
An Absolute `End` past frame 0 delivers its group up to that frame.

## SUBSCRIBE_UPDATE
Expand Down Expand Up @@ -1845,7 +1844,7 @@ The `Message Length` describes the payload size on the wire.
- Added TRACK_INFO `Refresh` and SUBSCRIBE_UPDATE `Wanted Group`, carrying ratio-based refresh demand through relays.

- Added `Compression` to TRACK_INFO: none (0x0) or DEFLATE (0x1), whose Groups are DEFLATE streams per draft-lcurley-moq-flate. FRAME and datagrams carry an `Inflated Size` when it is not none. Earlier versions receive the Track inflated.
- A subscription starts at a Group's first frame: SUBSCRIBE starts past frame 0 only when the subscriber holds every frame below it, compressed Track or not.
- SUBSCRIBE may start at any frame, including when the subscriber no longer holds the prefix. Relays widen a cold Group request upstream to frame 0 and cut locally.

- Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery.

Expand Down
30 changes: 30 additions & 0 deletions js/net/src/group.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,36 @@ import { Timestamp } from "./time.ts";

const dec = new TextDecoder();

test("a transport offset costs no frame storage and proves no group start", async () => {
const offset = MAX_GROUP_FRAMES + 7;
const producer = new Producer(0);
hooks.startGroup(producer, offset);
const early = producer.mirror();
expect(hooks.groupStarted(early)).toBe(false);
expect(hooks.groupTimestamp(early)).toBeUndefined();
for (let i = 0; i < MAX_GROUP_FRAMES; i++)
producer.writeFrame({ payload: new Uint8Array([i % 256]), timestamp: Timestamp.fromMillis(i) });
const late = producer.mirror();
for (const reader of [early, late]) {
expect(hooks.groupStarted(reader)).toBe(true);
expect(hooks.groupTimestamp(reader)).toBeUndefined();
for (let i = 0; i < MAX_GROUP_FRAMES; i++) {
const frame = await reader.readFrameSequence();
expect(frame?.sequence).toBe(offset + i);
expect(frame?.payload[0]).toBe(i % 256);
}
reader.close();
}
expect(() => producer.writeFrame(textFrame("overflow"))).toThrow(GroupTooLarge);
});

test("a compressed group cannot start without its dictionary prefix", () => {
const producer = new Producer(0);
hooks.startGroup(producer, 5);
expect(() => hooks.bindGroupCompression(producer, true)).toThrow("compressed group lacks its prefix");
producer.finish();
});

test("reordered frames keep the highest group PTS in live and later mirrors", () => {
const installed = new Manual().install();
try {
Expand Down
24 changes: 21 additions & 3 deletions js/net/src/group.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ class GroupState {
// Absolute sequence of the frame at the front of `frames`, advanced by a read: the
// sequence the next read returns.
start = 0;
// The wire offset this copy began at; unlike `start`, reads never advance it.
offset = 0;

cacheBytes = 0;
compressed?: boolean;
Expand Down Expand Up @@ -217,6 +219,16 @@ export class Producer {
}
if (state.closed.peek() !== undefined) flate.finish();
};
hooks.groupOffset = (group) => group.#state.offset;
hooks.startGroup = (group, start) => {
if (!Number.isSafeInteger(start) || start < 0) throw new RangeError("invalid group frame start");
const state = group.#state;
if (state.total.peek() !== 0 || state.closed.peek() !== undefined || group.#mirrors?.size)
throw new Error("group already started");
if (start !== 0 && state.compressed) throw new ProtocolViolation("compressed group lacks its prefix");
state.start = state.offset = start;
state.total.set(start);
};
hooks.writeGroupFrame = (group, frame, inflated) => group.#writeFrame(frame, inflated);
hooks.groupCacheBytes = (group) => group.#state.flate?.bytes ?? group.#state.cacheBytes;
hooks.groupFrameCount = (group) => group.#state.total.peek();
Expand Down Expand Up @@ -294,6 +306,7 @@ export class Producer {
dst.compressed = this.#state.compressed;
for (const frame of this.#state.frames.peek()) appendFrame(dst, frame);
dst.start = this.#state.start;
dst.offset = this.#state.offset;
// The replay only covers what is still buffered, so the count has to come from the
// source: it is what names a frame's absolute sequence, and what a publisher reads
// to resolve the live edge.
Expand Down Expand Up @@ -346,6 +359,8 @@ export class Producer {
throw new FrameTooLarge();
if (inflated !== undefined && (!this.#state.flate || !Number.isSafeInteger(inflated) || inflated < 0))
throw new ProtocolViolation("unexpected inflated frame size");
if (this.#state.total.peek() === Number.MAX_SAFE_INTEGER)
throw new ProtocolViolation("group frame index exceeds the safe integer range");
checkTimed(this.#state.timed, frame);
if (this.#limit !== undefined && this.#state.total.peek() >= this.#limit) {
throw new Error(`frame ${this.#state.total.peek()} is past the track's end`);
Expand All @@ -358,7 +373,7 @@ export class Producer {
commits = this.#admits.map((admit) => admit(start));
}
if (
this.#state.total.peek() >= MAX_GROUP_FRAMES ||
this.#state.total.peek() - this.#state.offset >= MAX_GROUP_FRAMES ||
((!this.#state.flate || inflated !== undefined) &&
this.#state.cacheBytes + frame.payload.byteLength > MAX_GROUP_CACHE_BYTES) ||
(this.#state.flate?.declared ?? this.#state.cacheBytes) + (inflated ?? frame.payload.length) >
Expand Down Expand Up @@ -687,13 +702,16 @@ export class Consumer {
static {
makeConsumer = (state) => new Consumer(state);
hooks.releaseGroupReader = (group, abort) => group.#release(abort);
hooks.groupTimestamp = (group) => (group.#state.unverified ? undefined : group.#state.timestamp);
hooks.groupTimestamp = (group) =>
group.#state.unverified || group.#state.offset !== 0 ? undefined : group.#state.timestamp;
hooks.groupLatest = (group) => (group.#state.unverified ? undefined : group.#state.latest);
hooks.unverifyGroup = (group) => {
group.#state.unverified = true;
};
hooks.groupArrived = (group) => group.#state.arrived;
hooks.groupStarted = (group) => group.#state.total.peek() > 0 || group.#state.closed.peek() !== undefined;
hooks.groupIndex = (group) => group.#index();
hooks.groupStarted = (group) =>
group.#state.total.peek() > group.#state.offset || group.#state.closed.peek() !== undefined;
hooks.groupFate = (group) => {
const state = group.#state;
return () => {
Expand Down
24 changes: 22 additions & 2 deletions js/net/src/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ export interface ResumeSource {
/** An independent subscriber to the serving copy. */
fork(options?: Subscription): Subscriber;
/** A fresh reader for a group still cached by one of this reader's copies. */
cachedGroup(sequence: number): GroupConsumer | undefined;
cachedGroup(sequence: number, from?: number): GroupConsumer | undefined;
/** Let go of every copy. */
close(abort?: Error): void;
}
Expand Down Expand Up @@ -386,6 +386,8 @@ export const hooks: {
* the drop. Mirrors the Rust `drop_groups`; assigned by `track.ts`.
*/
dropTrackGroups: (producer: Producer, start: number, end: number, error: Error) => void;
/** Create a received group at its wire offset before publishing it to readers. Assigned by `track.ts`. */
receiveGroup: (producer: Producer, group: { sequence: number; start: number }) => GroupProducer;
/** The producer's largest reported location and its subsequent changes. */
producerLargest: (producer: Producer) => Getter<Position | undefined>;

Expand Down Expand Up @@ -481,7 +483,7 @@ export const hooks: {
/** Whether this subscriber's budget gives up on `sequence`, judged by its successor; see {@link expireGroup}. */
staleAt: (subscriber: Subscriber, sequence: number) => Expiry;
/** A group the subscriber's track still caches, as a fresh read handle. */
cachedGroup: (subscriber: Subscriber, sequence: number) => GroupConsumer | undefined;
cachedGroup: (subscriber: Subscriber, sequence: number, from?: number) => GroupConsumer | undefined;
/** Whether {@link cachedGroup} would find the group, without making a read handle. */
holdsGroup: (subscriber: Subscriber, sequence: number) => boolean;
/** The error the subscriber's track gave `sequence` up with, if it did, and what changes it. */
Expand Down Expand Up @@ -512,6 +514,10 @@ export const hooks: {
/** Bind the group to its immutable track codec. */
bindGroupCompression: (group: GroupProducer, compressed: boolean) => void;
/** Admit a received stored slice without recompressing it. */
/** Set a received group's first absolute frame before publishing it. Assigned by `group.ts`. */
startGroup: (group: GroupProducer, start: number) => void;
/** The first absolute frame this source copy began at. Assigned by `group.ts`. */
groupOffset: (group: GroupProducer) => number;
writeGroupFrame: (group: GroupProducer, frame: Frame, inflated?: number) => void;
/** Payload footprint including a shared inflated copy, for measurements. */
groupCacheBytes: (group: GroupProducer) => number;
Expand Down Expand Up @@ -543,6 +549,8 @@ export const hooks: {
/** Return when a group joined its track at this hop, on the installed clock. */
groupArrived: (group: GroupConsumer) => Instant;
/** Whether a group counts as content: its first frame landed, or it ended. */
/** The next absolute frame this reader can serve. Assigned by `group.ts`. */
groupIndex: (group: GroupConsumer) => number;
groupStarted: (group: GroupConsumer) => boolean;
/** Watch the error a group's copy closes with, ignoring readers' verdicts, without holding the consumer. */
groupFate: (group: GroupConsumer) => () => Error | undefined;
Expand Down Expand Up @@ -714,6 +722,15 @@ export const hooks: {
bindGroupCompression: () => {
throw new Error("group.ts not loaded");
},
receiveGroup: () => {
throw new Error("track.ts not loaded");
},
groupOffset: () => {
throw new Error("group.ts not loaded");
},
startGroup: () => {
throw new Error("group.ts not loaded");
},
writeGroupFrame: () => {
throw new Error("group.ts not loaded");
},
Expand Down Expand Up @@ -753,6 +770,9 @@ export const hooks: {
groupArrived: () => {
throw new Error("group.ts not loaded");
},
groupIndex: () => {
throw new Error("group.ts not loaded");
},
groupStarted: () => {
throw new Error("group.ts not loaded");
},
Expand Down
110 changes: 67 additions & 43 deletions js/net/src/lite/connection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -618,50 +618,74 @@ for (const [alpn, version] of [
});
}

// A group stream that breaks the protocol ends the session, so a hostile peer cannot repeat
// it on every stream.
// A transport tail keeps its absolute frame numbers and the session open. A group past
// the declared end still ends the session, so a hostile peer cannot repeat that violation.
for (const violation of ["a partial group", "a group past the declared end"] as const) {
test(`${violation} closes the session with PROTOCOL_VIOLATION`, async () => {
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
const connection = new Connection({ url: new URL("https://relay.example/"), quic: pair.client, version });
try {
const track = wireOf(connection).consume(Path.from("room")).track("video").subscribe();

// The publisher, played by hand: answer TRACK_INFO, start the subscription, and
// leave any other stream the session opens (such as PROBE) open.
let sub: Stream | undefined;
while (!sub) {
const stream = await Stream.accept(pair.server, version);
if (!stream) throw new Error("the subscriber never subscribed");
const id = await stream.reader.u53();
if (id === StreamId.Track) {
await TrackMessage.decode(stream.reader, version);
await new TrackInfo({ maxAge: 60_000, timescale: Timescale.MILLI }).encode(stream.writer, version);
stream.close();
} else if (id === StreamId.Subscribe) {
await Subscribe.decode(stream.reader, version);
sub = stream;
test(
violation === "a partial group"
? "a partial group keeps its indices and the session open"
: "a group past the declared end closes the session with PROTOCOL_VIOLATION",
async () => {
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
const connection = new Connection({ url: new URL("https://relay.example/"), quic: pair.client, version });
try {
const track = wireOf(connection).consume(Path.from("room")).track("video").subscribe();

// The publisher, played by hand: answer TRACK_INFO, start the subscription, and
// leave any other stream the session opens (such as PROBE) open.
let sub: Stream | undefined;
while (!sub) {
const stream = await Stream.accept(pair.server, version);
if (!stream) throw new Error("the subscriber never subscribed");
const id = await stream.reader.u53();
if (id === StreamId.Track) {
await TrackMessage.decode(stream.reader, version);
await new TrackInfo({ maxAge: 60_000, timescale: Timescale.MILLI }).encode(
stream.writer,
version,
);
stream.close();
} else if (id === StreamId.Subscribe) {
await Subscribe.decode(stream.reader, version);
sub = stream;
}
}
await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0) }, version);

let group: GroupMessage;
if (violation === "a partial group") {
// A received tail uses its wire position even without a locally retained prefix.
group = new GroupMessage({ subscribe: 0n, sequence: 0, frameStart: 5 });
} else {
await encodeSubscribeResponse(sub.writer, { end: new SubscribeEnd(2) }, version);
group = new GroupMessage({ subscribe: 0n, sequence: 2 });
}
const writer = await Writer.open(pair.server, { version });
await writer.u53(DataType.Group);
await group.encode(writer, version);

if (violation === "a partial group") {
await writer.u62(2000n); // timestamp 1000, zigzag delta from 0
await writer.u53(1);
await writer.write(new Uint8Array([7]));
writer.close();
const received = await track.recvGroup();
expect((await received?.readFrameSequence())?.sequence).toBe(5);
expect(await received?.readFrameSequence()).toBeUndefined();
received?.close();
const goaway = await Stream.open(pair.server, { version });
await goaway.writer.u53(StreamId.Goaway);
await new Goaway("").encode(goaway.writer, version);
goaway.writer.close();
expect((await wireOf(connection).goaway).uri).toBe("");
} else {
expect((await pair.client.closed).closeCode).toBe(SessionCode.ProtocolViolation);
}
track.close();
} finally {
connection.abort();
}
await encodeSubscribeResponse(sub.writer, { start: new SubscribeStart(0) }, version);

let group: GroupMessage;
if (violation === "a partial group") {
// The subscriber never asked to start mid-group.
group = new GroupMessage({ subscribe: 0n, sequence: 0, frameStart: 5 });
} else {
await encodeSubscribeResponse(sub.writer, { end: new SubscribeEnd(2) }, version);
group = new GroupMessage({ subscribe: 0n, sequence: 2 });
}
const writer = await Writer.open(pair.server, { version });
await writer.u53(DataType.Group);
await group.encode(writer, version);

expect((await pair.client.closed).closeCode).toBe(SessionCode.ProtocolViolation);
track.close();
} finally {
connection.abort();
}
});
},
);
}
Loading
Loading