Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/concept/standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ subset drops. What a peer actually observes against this implementation:
- **Refused, not fatal.** A legal request this stack does not serve is rejected on its own and the session stays up: `FORWARD=0`, `SUBSCRIBE_TRACKS`, and the fetch forms above get `NOT_SUPPORTED`. Range Filters get `INVALID_FILTER` on draft 19 and later because the advertised filter limit is zero. On draft 21 and later, namespaces whose first tuple field is exactly `.` or `.session` get `DOES_NOT_EXIST` before application lookup; other dot prefixes still reach the application. Subscription updates accept priority and delivery timeouts; Rust also renews request tokens and relays `NEW_GROUP_REQUEST` (see dynamic groups below). Other supported-but-unimplemented preferences end that subscription; a `TRACK_PROPERTY_FILTER` there closes the session with `PROTOCOL_VIOLATION`, since it belongs only to `SUBSCRIBE_TRACKS`. On draft 18 and later a namespace update may change its prefix; suffixes after the acknowledgement are relative to that new prefix.
- **Objects and datagrams.** Only subgroup 0 is accepted: a stream for any other subgroup, whether its header gives the ID or takes it from the first Object ID, is stopped and the session stays up. Rust relays preserve Object IDs, gaps, object properties, FIRST\_OBJECT, and END\_OF\_GROUP status objects between moq-transport peers for subgroup 0. Datagrams retain their IDs, properties, priority, and status. Plain Rust readers and moq-lite peers see consecutive frames and normal datagram payloads; JavaScript supports only normal datagrams at Object ID 0. A malformed datagram closes the session; a relayed one the downstream draft cannot carry is dropped. A relayed Timestamp leaves in the units the downstream response declares, and is dropped where it declares none (drafts 14 through 16), as for a plain frame; an immutable one that would change or vanish is refused. Only property 0x10 is read as a Timestamp. Property 0x06 stays opaque and keeps its value regardless of timestamp units; it is dropped across the draft-19 boundary, or refused when immutable, because draft 19 assigns it to SUBGROUP\_DELIVERY\_TIMEOUT. From draft 15, a status object carrying properties is refused, and one relayed from draft 14 leaves without them. Draft 14 allows them, but an END\_OF\_TRACK carrying properties ends the subscription as unsupported, since a track end has nowhere to keep them. A `FETCH` carries the END\_OF\_GROUP object on drafts 14 and 15; later drafts have no FETCH status, so their response ends at the last frame. A group cannot resume across route changes by frame ordinal once Object IDs skip ahead of the ordinals on either route; recovery refuses it instead of splicing different objects.
- **Dynamic groups.** A Rust relay advertises `DYNAMIC_GROUPS` only where the original publisher did, and carries `NEW_GROUP_REQUEST` (draft 15 and later) back to it: on the `SUBSCRIBE` that opens the upstream subscription, or in a `REQUEST_UPDATE` once it is established. Requests from every subscriber of a track coalesce, so one that names an existing group or that an outstanding request already covers goes no further. A request for 0 goes upstream as 0, unless another subscriber's request past the newest group held answers it too. The next group the relay receives answers every request it held, so new groups never send one again. On drafts 15 and 16, whose `TRACK_STATUS` answer has no room for it, a relay copy first learned from `TRACK_STATUS` (a fetch-only reader) does not advertise it to later subscribers. Local publishers never advertise `DYNAMIC_GROUPS`; on their tracks a `SUBSCRIBE`'s request is ignored and an update carrying one ends the subscription with `UPDATE_FAILED`. moq-lite hops do not carry requests yet.
- **Priority.** Higher is served first. The IETF default of 128 is this stack's 127, and a track that never sets one is 127.
- **Priority.** Higher is served first. The IETF default of 128 is this stack's 127, and a track that never sets one is 127. A relay's upstream subscription carries the highest priority its subscribers ask for and updates it as they come and go; its group order stays as the `SUBSCRIBE` asked, since no draft lets an update change it.
- **Size.** An object extension block larger than 64 KiB ends that subgroup stream. The session stays up. This cap is ours, not the draft's.

Several project drafts extend the IETF wire without breaking it, since `SETUP`
Expand Down
7 changes: 7 additions & 0 deletions js/net/src/ietf/adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,10 @@ export class ControlStreamAdapter implements Session {
// SubscribeNamespace requestIds — for routing 0x08/0x0E entries that lack requestId (v14/v15)
#subscribeNamespaces = new Set<bigint>();

// Our updates' own request IDs on drafts 15 and 16, whose answer names that ID rather than
// the request it updated. Nothing waits on it, so it is consumed here.
#updates = new Set<bigint>();

// Incoming stream queue (for acceptBi)
#incomingQueue: Stream[] = [];
#incomingWaiters: ((stream: Stream | undefined) => void)[] = [];
Expand Down Expand Up @@ -393,6 +397,7 @@ export class ControlStreamAdapter implements Session {
}
} else {
spent = this.#updateId(toFlush);
if (spent !== undefined && this.version !== Version.DRAFT_14) this.#updates.add(spent);
}

await this.#send(toFlush, spent);
Expand Down Expand Up @@ -499,6 +504,8 @@ export class ControlStreamAdapter implements Session {
}

const { route, requestId } = classified;
if ((route === Route.Response || route === Route.ErrorResponse) && this.#updates.delete(requestId))
continue;

switch (route) {
case Route.NewRequest:
Expand Down
2 changes: 2 additions & 0 deletions js/net/src/ietf/subscribe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,8 @@ export class SubscribeUpdate {
await w.u62(this.ownRequestId);
await w.u62(this.requestId);
if (version === Version.DRAFT_14) {
// Every field is mandatory and the start must not move. Our subscriptions start at
// {0, 0} open ended (`joinFilter`), so zeroes repeat their bounds.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
await w.u62(0n); // start_group
await w.u62(0n); // start_object
await w.u62(0n); // end_group
Expand Down
112 changes: 104 additions & 8 deletions js/net/src/ietf/subscriber.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ import {
hasFirstObjectBit,
ObjectIdGap,
} from "./object.ts";
import { Parameters } from "./parameters.ts";
import { fromWire, toWire } from "./priority.ts";
import { type Publish, PublishDone, PublishDoneStatus, PublishError, publishDoneClean } from "./publish.ts";
import {
Expand Down Expand Up @@ -743,6 +744,8 @@ export class Subscriber {

let stream: Stream;
let trackAlias: bigint;
// The SUBSCRIBER_PRIORITY the request went out with.
let priority: number;
const abandoned = new Error("subscribe abandoned before it was accepted");
try {
// Returning demand keeps the same setup and its original deadline.
Expand All @@ -752,6 +755,7 @@ export class Subscriber {
if (result !== null) {
stream = result.stream;
trackAlias = result.alias;
priority = result.priority;
break;
}
if (demand.closed.peek() === undefined && demand.used.peek()) continue;
Expand Down Expand Up @@ -830,11 +834,13 @@ export class Subscriber {
// Terminal conditions settle at most once (PublishDone, track close = local
// unsubscribe, a revoked grant); race them once so the demand loop doesn't
// re-subscribe each pass.
const unanswered = new Signal(0);
const done = race([
this.#runPublishDone(stream, subscription, broadcast).then(() => publisherEnded),
this.#runPublishDone(stream, subscription, broadcast, unanswered).then(() => publisherEnded),
producer.closed.then(() => localEnded),
revoked,
]);
const updates = this.#runUpdates(stream, requestId, producer, priority, unanswered, done);

// Serve until a terminal condition fires or the last local subscriber leaves. The unused
// wake is level-triggered: re-check demand so a subscriber that returns before we tear
Expand All @@ -859,6 +865,11 @@ export class Subscriber {
producer.close();
}

// Closing settled `done`, so an update still being written finishes and nothing starts
// another. Its bytes must not interleave with the cancellation, and on drafts 14 and 15
// its spent request ID must reach the wire before any later one can.
await updates;

// The publisher already ended the request, so there is nothing to cancel. Sending
// UNSUBSCRIBE here would name a request it has already torn down.
if (terminal !== publisherEnded) await this.#cancelSubscribe(stream, requestId);
Expand Down Expand Up @@ -889,12 +900,30 @@ export class Subscriber {
* is only a hint: a peer may send 0 regardless, so 0 waits out the grace. A request
* stream that FINs without PUBLISH_DONE is a protocol violation.
*/
async #runPublishDone(stream: Stream, subscription: Subscription, broadcast: Path.Valid): Promise<void> {
async #runPublishDone(
stream: Stream,
subscription: Subscription,
broadcast: Path.Valid,
unanswered: Signal<number>,
): Promise<void> {
const version = this.#session.version;
if (await stream.reader.done()) throw new ProtocolViolation("subscribe stream ended without PUBLISH_DONE");
const typeId = await stream.reader.u53();
if (typeId !== PublishDone.id) {
throw new ProtocolViolation(`unexpected message on a subscription: 0x${typeId.toString(16)}`);
// A refused update is followed by PUBLISH_DONE, which ends the subscription, so no
// further answer is owed.
let refused = false;
for (;;) {
if (await stream.reader.done()) throw new ProtocolViolation("subscribe stream ended without PUBLISH_DONE");
const typeId = await stream.reader.u53();
if (typeId === PublishDone.id) break;
const answer = (typeId === RequestOk.id || typeId === RequestError.id) && !refused && unanswered.peek() > 0;
if (!answer) throw new ProtocolViolation(`unexpected message on a subscription: 0x${typeId.toString(16)}`);
if (typeId === RequestOk.id) {
await RequestOk.decode(stream.reader, version);
unanswered.update((count) => count - 1);
} else {
const err = await RequestError.decode(stream.reader, version);
console.debug(`subscription update refused: ${err.reasonPhrase}`);
refused = true;
}
}
const done = await PublishDone.decode(stream.reader, version);
// A revocation reads the same as one from a moq-lite peer.
Expand All @@ -909,6 +938,72 @@ export class Subscriber {
await tail.settle(complete, track.closed);
}

/**
* Follow the aggregate subscriber priority upstream until `done` settles: SUBSCRIBE_UPDATE
* through draft 16, REQUEST_UPDATE after, each under a Request ID of its own. From draft 17
* the answer comes back on this stream, so one update is in flight at a time and changes
* meanwhile coalesce. No draft lets an update change the group order, so it stays as asked.
*/
async #runUpdates(
stream: Stream,
requestId: bigint,
producer: track.Producer,
sent: number,
unanswered: Signal<number>,
done: Promise<unknown>,
): Promise<void> {
const version = this.#session.version;
const legacy = version === Version.DRAFT_14 || version === Version.DRAFT_15 || version === Version.DRAFT_16;
const stopped = done.then(
() => null,
() => null,
);
const stop = new AbortController();
void stopped.then(() => stop.abort());

for (;;) {
const wanted = toWire(producer.subscription.peek()?.priority ?? 0);
if (wanted === sent || unanswered.peek() > 0) {
const disposes: Dispose[] = [];
const wake = new Promise<void>((resolve) => {
disposes.push(
producer.subscription.changed(() => resolve()),
unanswered.changed(() => resolve()),
);
});
await race([wake, stopped]);
for (const dispose of disposes) dispose();
if (stop.signal.aborted) return;
continue;
}

let own: bigint | undefined;
try {
own = await this.#session.nextRequestId(stop.signal);
} catch {
return;
}
if (own === undefined) return;

const parameters = new Parameters();
parameters.subscriberPriority = wanted;
const update = legacy
? new SubscribeUpdate({ requestId, ownRequestId: own, parameters })
: new SubscribeUpdate({ requestId: own, parameters });
// Counted before writing: the answer can be read before the write's promise resolves.
if (!legacy) unanswered.update((count) => count + 1);
try {
await stream.writer.u53(SubscribeUpdate.id);
await update.encode(stream.writer, version);
} catch {
// The publisher stopped reading; its responses decide how the track ends.
return;
}
sent = wanted;
console.debug(`subscribe update: id=${requestId} priority=${wanted}`);
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}

/**
* Tell the publisher to stop serving a subscription we are walking away from.
*
Expand Down Expand Up @@ -941,7 +1036,7 @@ export class Subscriber {
subscription: Subscription,
requestId: bigint,
deadline: Deadline,
): Promise<{ stream: Stream; alias: bigint }> {
): Promise<{ stream: Stream; alias: bigint; priority: number }> {
const version = this.#session.version;

const legacy = version === Version.DRAFT_14 || version === Version.DRAFT_15 || version === Version.DRAFT_16;
Expand All @@ -964,6 +1059,7 @@ export class Subscriber {
requestId,
trackNamespace: broadcast,
trackName: request.name,
// Read now, since a subscriber can join while the stream opened. Updates follow it.
subscriberPriority: toWire(request.priority),
// Fixed for the subscription's life: no REQUEST_UPDATE parameter carries it.
groupOrder: request.subscription.order,
Expand Down Expand Up @@ -1046,7 +1142,7 @@ export class Subscriber {
throw err;
}
state.registeredAlias = ok.trackAlias;
return { stream: state.stream, alias: ok.trackAlias };
return { stream: state.stream, alias: ok.trackAlias, priority: msg.subscriberPriority };
}

/**
Expand Down
91 changes: 91 additions & 0 deletions js/net/src/subscribe_priority.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// One upstream subscription carries the aggregate priority of the subscribers it serves, and
// follows it as they come and go. The JS origin never serves a received route onward, so the
// relay case lives with the Rust relay (rs/moq-net/tests/integration/subscribe_priority.rs).

import { expect, test } from "bun:test";
import { accept, connect, type Established } from "./connection/index.ts";
import * as Ietf from "./ietf/index.ts";
import * as Lite from "./lite/index.ts";
import { createMockTransportPair, type MockTransport } from "./mock.ts";
import { Producer as OriginProducer } from "./origin.ts";
import * as Path from "./path.ts";
import { Timescale } from "./time.ts";
import type { Producer as TrackProducer } from "./track.ts";
import { wireOf } from "./wire.ts";

const url = new URL("https://localhost:4443/test");
const path = Path.from("test");

const LOW = 1;
const HIGH = 200;

// Dial a session that consumes what `origin` serves. Draft-14's ALPN is the generic one, so
// the server is told the version instead.
async function session(protocol: string, origin: OriginProducer): Promise<[Established, Established]> {
const draft14 = protocol === Ietf.ALPN.DRAFT_14;
const pair = createMockTransportPair(draft14 ? "" : protocol);
lateAcks(pair.client);
return Promise.all([
connect({ url, transport: pair.client }),
accept({
transport: pair.server,
url,
publish: origin.consume(),
version: draft14 ? Ietf.Version.DRAFT_14 : undefined,
}),
]);
}

// Acknowledge each write on the client's bidirectional streams a task late, after the peer
// can already have answered it, as a real send stream may.
function lateAcks(transport: MockTransport) {
const create = transport.createBidirectionalStream.bind(transport);
transport.createBidirectionalStream = async (options) => {
const stream = await create(options);
const inner = stream.writable.getWriter();
const writable = new WritableStream<Uint8Array>({
async write(chunk) {
await inner.write(chunk);
await new Promise((resolve) => setTimeout(resolve, 0));
},
close: () => inner.close(),
abort: (reason) => inner.abort(reason),
});
return { readable: stream.readable, writable };
};
}

// Resolve once the subscription `track` serves asks for `want`.
async function priority(track: TrackProducer, want: number): Promise<void> {
while (track.subscription.peek()?.priority !== want) await track.subscription.changed();
}

test.each([Lite.ALPN_07_WIP, ...Object.values(Ietf.ALPN)])(
"%s follows the aggregate priority upstream",
async (protocol) => {
const origin = new OriginProducer();
const broadcast = origin.createBroadcast(path);
broadcast.announce();
const video = broadcast.createTrack("video", { timescale: Timescale.MILLI });
const [client, server] = await session(protocol, origin);

const remote = wireOf(client).consume(path);
const low = remote.track("video").subscribe({ priority: LOW });
await priority(video, LOW);

// A second subscriber at a higher priority raises the one upstream request.
const high = remote.track("video").subscribe({ priority: HIGH });
await priority(video, HIGH);

// Leaving lowers it again.
high.close();
await priority(video, LOW);
expect(video.subscription.peek()?.priority).toBe(LOW);

low.close();
remote.close();
broadcast.close();
client.abort();
server.abort();
},
);
1 change: 0 additions & 1 deletion quest/m1/subscribe-ranges/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ every sequence delivered or dropped.
- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, a subscriber reads only its own ranges, and a range end may be a time
- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver
- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft
- [moq-transport upstream updates](/quest/m1/subscribe-ranges/ietf-update.md) - the upstream request follows the aggregate priority; its order stays fixed, as every draft requires
- [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream
- [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity

Expand Down
Loading
Loading