Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/concept/standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ subset drops. What a peer actually observes against this implementation:
- **One `SETUP` credential**, forwarded to the [auth server](/bin/relay/auth#the-contract) unverified. A second `SETUP` token closes the session. We advertise no token cache, so an alias reference in `SETUP` closes it with `PROTOCOL_VIOLATION`, one on a request with `UNKNOWN_AUTH_TOKEN_ALIAS`, and a registration on a request with `AUTH_TOKEN_CACHE_OVERFLOW`. Where the Rust side authorizes the peer or verifies tokens, a token on a `SUBSCRIBE` or `PUBLISH_NAMESPACE` is verified like an in-band session token and authorizes that one request alone, even where that authorization already covers it; it never widens the session. It is refused with `NOT_SUPPORTED` when nothing verifies it, when it is empty, or when its Token Type is anything but 0 (out of band). A side that does neither, JavaScript included, ignores the token. Rust refuses a request carrying more than one distinct token with `NOT_SUPPORTED`, since it verifies one; JavaScript, which verifies none, ignores them all. A token-authorized `SUBSCRIBE` still stays within the grant the peer gave us. A token-authorized `PUBLISH_NAMESPACE` needs a grant covering everything beneath the namespace, and the tracks we subscribe under it rely on that announce instead of our limit, ending with it. An update renews the token, except on a drafts 14 and 15 `PUBLISH_NAMESPACE`, which has no update, and a refused renewal ends only that request. Rust and JavaScript send the token a route names (`Route::auth`) on its `PUBLISH_NAMESPACE`, including the one drafts 14 and 15 answer a solicited `SUBSCRIBE_NAMESPACE` with, and swapping it renews the request with `REQUEST_UPDATE` from draft 16, or ends and re-sends it before; the inline `NAMESPACE` that answers a solicited `SUBSCRIBE_NAMESPACE` from draft 16 carries no token. The token belongs to the session that added it, so a reconnect means adding it again and re-announcing. A token on any other request is decoded and grants nothing, and a request repeating one is refused with `MALFORMED_AUTH_TOKEN`. `SUBSCRIBE_TRACKS` reads its namespace before refusal; its token is not decoded. On draft 17 and later, the negotiated [MoQ Auth extension](/concept/moq-lite#authorization) accepts additional session tokens in band.
- **Refused, not fatal.** A legal request this stack does not serve is rejected on its own and the session stays up: `FORWARD=0`, `SUBSCRIBE_TRACKS`, and the fetch forms above get `NOT_SUPPORTED`. Range Filters get `INVALID_FILTER` on draft 19 and later because the advertised filter limit is zero. On draft 21 and later, namespaces whose first tuple field is exactly `.` or `.session` get `DOES_NOT_EXIST` before application lookup; other dot prefixes still reach the application. Subscription updates accept priority, negotiated subscriber max age, and delivery timeouts; Rust also renews request tokens and judges `NEW_GROUP_REQUEST` as a join (see dynamic groups below). Other supported-but-unimplemented preferences end that subscription; a `TRACK_PROPERTY_FILTER` there closes the session with `PROTOCOL_VIOLATION`, since it belongs only to `SUBSCRIBE_TRACKS`. On draft 18 and later a namespace update may change its prefix; suffixes after the acknowledgement are relative to that new prefix.
- **Objects and datagrams.** Only subgroup 0 is accepted: a stream for any other subgroup, whether its header gives the ID or takes it from the first Object ID, is stopped and the session stays up. Rust relays preserve Object IDs, gaps, object properties, FIRST\_OBJECT, and END\_OF\_GROUP status objects between moq-transport peers for subgroup 0. Datagrams retain their IDs, properties, priority, and status. Plain Rust readers and moq-lite peers see consecutive frames and normal datagram payloads; JavaScript supports only normal datagrams at Object ID 0. A malformed datagram closes the session; a relayed one the downstream draft cannot carry is dropped. A relayed Timestamp leaves in the units the downstream response declares, and is dropped where it declares none (drafts 14 through 16), as for a plain frame; an immutable one that would change or vanish is refused. Only property 0x10 is read as a Timestamp. Property 0x06 stays opaque and keeps its value regardless of timestamp units; it is dropped across the draft-19 boundary, or refused when immutable, because draft 19 assigns it to SUBGROUP\_DELIVERY\_TIMEOUT. From draft 15, a status object carrying properties is refused, and one relayed from draft 14 leaves without them. Draft 14 allows them, but an END\_OF\_TRACK carrying properties ends the subscription as unsupported, since a track end has nowhere to keep them. A `FETCH` carries the END\_OF\_GROUP object on drafts 14 and 15; later drafts have no FETCH status, so their response ends at the last frame. A group cannot resume across route changes by frame ordinal once Object IDs skip ahead of the ordinals on either route; recovery refuses it instead of splicing different objects.
- **Dynamic groups.** A track advertises `DYNAMIC_GROUPS` when its publisher sets a refresh ratio, and a track received with `DYNAMIC_GROUPS` gets a ratio of 0. A `NEW_GROUP_REQUEST` (draft 15 and later) counts as one more subscriber joining, which a `SUBSCRIBE` already is: it asks for a new group only when the newest group's bytes after its first frame exceed the ratio times the first frame. Joins coalesce until the asked-for group arrives. A Rust relay sends `NEW_GROUP_REQUEST` upstream, on the `SUBSCRIBE` or in a `REQUEST_UPDATE`, once a join wants a group past the newest it holds; a relay holding nothing yet asks for none. On a track without a ratio, an update carrying one ends the subscription with `UPDATE_FAILED`. On drafts 15 and 16, whose `TRACK_STATUS` answer has no room for it, a relay copy first learned from `TRACK_STATUS` (a fetch-only reader) does not advertise it to later subscribers. moq-lite hops do not carry requests yet.
- **Dynamic groups.** A track advertises `DYNAMIC_GROUPS` when its publisher sets a refresh ratio, and a track received with `DYNAMIC_GROUPS` gets a ratio of 0. A `NEW_GROUP_REQUEST` (draft 15 and later) counts as one more subscriber joining, which a `SUBSCRIBE` already is: it asks for a new group only when the newest group's bytes after its first frame exceed the ratio times the first frame. Joins coalesce until the asked-for group arrives. A Rust relay sends `NEW_GROUP_REQUEST` upstream, on the `SUBSCRIBE` or in a `REQUEST_UPDATE`, once a join wants a group past the newest it holds; a relay holding nothing yet asks for none. On a track without a ratio, an update carrying one ends the subscription with `UPDATE_FAILED`. On drafts 15 and 16, whose `TRACK_STATUS` answer has no room for it, a relay copy first learned from `TRACK_STATUS` (a fetch-only reader) does not advertise it to later subscribers. A moq-lite-07 hop (`moq-lite-07-wip`, opt-in) carries the same demand natively: the ratio in `TRACK_INFO` and the asked-for group in `SUBSCRIBE_UPDATE`. Earlier moq-lite versions carry neither, so a track received over one never refreshes.
- **Priority.** Higher is served first. The IETF default of 128 is this stack's 127, and a track that never sets one is 127. A relay's upstream subscription carries the highest priority its subscribers ask for and updates it as they come and go; its group order stays as the `SUBSCRIBE` asked, since no draft lets an update change it.
- **Size.** An object extension block larger than 64 KiB ends that subgroup stream. The session stays up. This cap is ours, not the draft's.

Expand Down
19 changes: 19 additions & 0 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -1219,13 +1219,22 @@ SUBSCRIBE_UPDATE Message {
Group End (i)
Frame Start (i)
Frame End (i)
Wanted Group (i)
}
~~~

See [SUBSCRIBE](#subscribe) for information about each field.
Moving `Frame Start` forward within a group that is already being delivered does not retract frames the publisher has already sent; like `Group Start`, it only bounds what the publisher sends from here on.


**Wanted Group**:
The newest group the subscriber's live-edge joins want, encoded as the sequence plus one; `0` means none.
Like the other fields, it is subscription state: every SUBSCRIBE_UPDATE restates it, and the latest one applies.
A relay sends a SUBSCRIBE_UPDATE when local demand wants a group beyond its largest that differs from the value it last sent, and keeps restating that value in later updates, including after the group arrives.
A publisher MUST ignore a value at or below its largest group, and otherwise evaluate it as a new SUBSCRIBE under the Track's Refresh ratio.
Restating a value is harmless: a join while a request is pending asks for the same group.
A new SUBSCRIBE itself counts as a join and needs no Wanted Group field.

## TRACK
TRACK is sent by a subscriber to request a Track's immutable publisher properties.
It is the first message on a Track Stream (0x6).
Expand Down Expand Up @@ -1260,6 +1269,7 @@ TRACK_INFO Message {
Publisher Priority (8)
Publisher Max Age (i)
Timescale (i)
Refresh (i)
}
~~~

Expand Down Expand Up @@ -1292,6 +1302,13 @@ The number of timestamp units per second for frame timestamps on this Track, or
lite-05 and lite-06 have no untimed encoding: there it MUST be non-zero, and a subscriber that receives 0 MUST reset the stream with a protocol violation.
Common values include `1000` (milliseconds), `1000000` (microseconds), `48000` (audio sample rate), and `90000` (RTP video clock).

**Refresh**:
An optional unsigned 32-bit byte ratio, encoded as the ratio plus one; `0` disables refresh.
On a live-edge join, a publisher asks for a group beyond every sequence already produced when the newest group's bytes after its first frame exceed the ratio times that first frame's bytes.
Joins coalesce until the requested group arrives.
A relay preserves the ratio and evaluates joins against its own cache, forwarding demand in SUBSCRIBE_UPDATE.
Versions before lite-07 omit Refresh and Wanted Group; a Track received on those versions never refreshes.

## SUBSCRIBE_OK {#subscribe-ok}
A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start position.
It is the first message the publisher sends on the Subscribe Stream, once the start position is known.
Expand Down Expand Up @@ -1622,6 +1639,8 @@ The `Message Length` describes the payload size on the wire.

## moq-lite-07

- Added TRACK_INFO `Refresh` and SUBSCRIBE_UPDATE `Wanted Group`, carrying ratio-based refresh demand through relays.

- Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery.

- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Receivers do not check it, since a received timestamp may be a send time.
Expand Down
86 changes: 86 additions & 0 deletions js/net/bench/refresh.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/** Sweep retained groups, unrelated tracks, and joined readers for a refresh crossing one lite hop. */
import { Manual, Nano } from "@moq/clock";
import { accept } from "../src/connection/accept.ts";
import { connect } from "../src/connection/connect.ts";
import { ALPN_07_WIP } from "../src/lite/version.ts";
import { createMockTransportPair } from "../src/mock.ts";
import { Producer } from "../src/origin.ts";
import * as Path from "../src/path.ts";

console.debug = () => {};
console.log("retained,tracks,joined,refresh_us");
for (const retained of [1, 64, 1024])
for (const tracks of [1, 8, 32])
for (const joined of [1, 8, 32]) {
const clock = new Manual();
const restore = clock.install();
const source = new Producer();
const destination = new Producer();
const broadcast = source.createBroadcast(Path.from("room"));
broadcast.announce();
const producers = Array.from({ length: tracks }, (_, i) =>
broadcast.createTrack(`data${i}`, { refresh: 1 }),
);
const pair = createMockTransportPair(ALPN_07_WIP);
const url = new URL("https://localhost/refresh");
const peers = await Promise.all([
connect({ url, transport: pair.client, consume: destination }),
accept({ url, transport: pair.server, publish: source.consume() }),
]);
try {
const announcements = destination.announced();
while ((await announcements.next())?.kind !== "start") {}
const request = destination.request(Path.from("room"));
while (!request.active.peek()) await request.active.changed();
const remote = request.active.peek()!;
const consumers = producers.map((_, i) => remote.track(`data${i}`));
const baseline = consumers.map((track) => track.subscribe());
for (let i = 0; i < tracks; i++) {
const group = producers[i].appendGroup();
group.writeFrame({ payload: new Uint8Array(3) });
group.close();
const received = (await baseline[i].recvGroup())!;
await received.readFrame();
received.close();
}
// Keep this track's historical groups in its cache. Only the touched
// track grows, so scanning that cache appears independently of fanout.
for (let old = 1; old < retained; old++) {
const group = producers[0].appendGroup();
group.writeFrame({ payload: new Uint8Array(3) });
group.close();
const received = await baseline[0].recvGroup();
if (!received) throw new Error("historical group missing");
await received.readFrame();
received.close();
}

const samples: number[] = [];
for (let round = 0; round < 9; round++) {
const group = producers[0].appendGroup();
group.writeFrame({ payload: new Uint8Array(3) });
group.writeFrame({ payload: new Uint8Array(4) });
group.close();
const received = (await baseline[0].recvGroup())!;
await received.readFrame();
await received.readFrame();
received.close();
const start = performance.now();
const joins = Array.from({ length: joined }, () => consumers[0].subscribe());
const wanted = producers[0].demand().wanted;
while (wanted.peek() !== group.sequence + 1) await wanted.changed();
if (round > 0) samples.push((performance.now() - start) * 1000);
for (const join of joins) join.close();
}
samples.sort((a, b) => a - b);
console.log(`${retained},${tracks},${joined},${samples[4].toFixed(2)}`);
for (const sub of baseline) sub.close();
request.close();
} finally {
destination.close();
source.close();
for (const peer of peers) peer.abort();
await clock.advanceAsync(Nano(0));
restore();
}
}
21 changes: 21 additions & 0 deletions js/net/src/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,8 @@ export type RecvDatagram =
* buffer of its own. Implemented by `resume.ts`, the counterpart of the Rust `Inner::Resume`.
*/
export interface ResumeSource {
/** Re-evaluate a live-edge join at the serving copy. */
join(): void;
/** The next group not in `delivered`, in arrival or sequence order, without waiting. */
tryGroup(ordered: boolean, delivered: Delivered): Recv;
/**
Expand Down Expand Up @@ -345,6 +347,16 @@ export const hooks: {
* a higher one. Mirrors the Rust `finish_at_pending`; assigned by `track.ts`.
*/
endTrack: (producer: Producer, end: Location) => void;
// private: capability gap
/** The producer's largest reported location and its subsequent changes. */
producerLargest: (producer: Producer) => Getter<Location | undefined>;

// private: passthrough
/** Re-evaluate a live-edge join at this subscriber's copy. */
joinSubscriber: (subscriber: Subscriber) => void;
/** Re-evaluate a live-edge join at a producer. */
joinProducer: (producer: Producer) => void;

// pending: /quest/m1/public-model/deferred.md
/**
* Declare the first group a wire subscription's live feed serves (lite SUBSCRIBE_START, or
Expand Down Expand Up @@ -491,6 +503,15 @@ export const hooks: {
endTrack: () => {
throw new Error("track.ts not loaded");
},
producerLargest: () => {
throw new Error("track.ts not loaded");
},
joinProducer: () => {
throw new Error("track.ts not loaded");
},
joinSubscriber: () => {
throw new Error("track.ts not loaded");
},
startTrack: () => {
throw new Error("track.ts not loaded");
},
Expand Down
6 changes: 6 additions & 0 deletions js/net/src/lite/publisher.ts
Original file line number Diff line number Diff line change
Expand Up @@ -893,6 +893,11 @@ export class Publisher {
writer: stream.writer,
version: this.version,
apply: (update) => {
// Every update restates the wanted group. Past our largest it is a join, which
// repeats the pending ask; at or below it the group already exists.
const largest = hooks.peekLargest(track).largest?.group;
if (update.wantedGroup !== undefined && (largest === undefined || update.wantedGroup > largest))
hooks.joinSubscriber(track);
const end = exclusiveGroupEnd(update.endGroup);
track.update({
priority: update.priority,
Expand Down Expand Up @@ -1431,6 +1436,7 @@ export class Publisher {
const info = await wireOf(front).resolveTrackInfo(track, release.signal);
return new TrackInfoMessage({
priority: info.priority,
refresh: info.refresh,
// Publisher Max Age: the publisher's retention bound, advertised so
// relays re-serve with the same window.
maxAge: info.maxAge,
Expand Down
Loading