diff --git a/doc/concept/standard.md b/doc/concept/standard.md index b41ec2c525d..6add841448c 100644 --- a/doc/concept/standard.md +++ b/doc/concept/standard.md @@ -36,7 +36,7 @@ subset drops. What a peer actually observes against this implementation: - **One `SETUP` credential**, forwarded to the [auth server](/bin/relay/auth#the-contract) unverified. A second `SETUP` token closes the session. We advertise no token cache, so an alias reference in `SETUP` closes it with `PROTOCOL_VIOLATION`, one on a request with `UNKNOWN_AUTH_TOKEN_ALIAS`, and a registration on a request with `AUTH_TOKEN_CACHE_OVERFLOW`. Where the Rust side authorizes the peer or verifies tokens, each distinct request token is verified once while any open request on that session names it, and its grant authorizes only requests naming it, even where that authorization already covers it; it never widens the session. It is refused with `NOT_SUPPORTED` when nothing verifies it, when it is empty, or when its Token Type is anything but 0 (out of band). A side that does neither, JavaScript included, ignores the token. Rust refuses a request carrying more than one distinct token with `NOT_SUPPORTED`, since it verifies one; JavaScript, which verifies none, ignores them all. A token-authorized `SUBSCRIBE` still stays within the grant the peer gave us. A token-authorized `PUBLISH_NAMESPACE` needs a grant covering everything beneath the namespace, and the tracks we subscribe under it rely on that announce instead of our limit, ending with it. An update on a token-authorized request moves it to the token it carries; the old token ends with its last naming request. A request is admitted by the session or by a token for good, so an update naming a token on one the session admitted is refused with `NOT_SUPPORTED`. A token-less `PUBLISH_NAMESPACE` our limit does not cover is refused `UNAUTHORIZED`, and one the limit stops covering ends; the publisher re-announces once it widens. Expiry or revocation ends every request naming that token. The relay reports the lease end to its auth server, and a later request naming the token verifies it again. A refused renewal ends only that request. Rust and JavaScript send the token a route names (`Route::auth`) on its `PUBLISH_NAMESPACE`, and swapping it renews the request with `REQUEST_UPDATE`; the inline `NAMESPACE` that answers a solicited `SUBSCRIBE_NAMESPACE` carries no token. A token added to a reconnecting connection's auth handle rides every session it makes, so the announcement keeps it across a reconnect; one added to a single session's handle ends with that session. Rust honors a token on any supported request; discovery reveals only paths it grants, and a joining `FETCH` keeps following its subscription. A request repeating a token is refused with `MALFORMED_AUTH_TOKEN`. The negotiated [MoQ Auth extension](/concept/moq-lite#authorization) accepts additional session tokens in band. - **Pause delivery.** `FORWARD=0` on a `SUBSCRIBE` or its update drops the subscription, closing its data streams and holding nothing upstream, while the request and its slot stay. `FORWARD=1` subscribes again as a new `SUBSCRIBE` with the same filter would, so nothing carries across the pause: its `REQUEST_OK` names the live edge it joined at. A paused update may change only priority and max age. - **Refused, not fatal.** A legal request this stack does not serve is rejected on its own and the session stays up: `SUBSCRIBE_TRACKS`, multi-group `FETCH`, and joining requests reaching before the joined group get `NOT_SUPPORTED`. Range Filters get `INVALID_FILTER` from draft 20 because the advertised filter limit is zero. From draft 20, namespaces whose first tuple field is exactly `.` or `.session` get `DOES_NOT_EXIST` before application lookup; other dot prefixes still reach the application. Subscription updates accept priority, negotiated subscriber max age, and delivery timeouts. Rust also carries range changes, keeps bounded subscriptions updatable, renews request tokens, and judges `NEW_GROUP_REQUEST` as a join (see dynamic groups below). From draft 20, a wider or historical fill fails on its separate fetch stream while its subscription stays open; a largest-group fill hands its head to the live subgroup tail. Other supported-but-unimplemented preferences end that subscription; a `TRACK_PROPERTY_FILTER` there closes the session with `PROTOCOL_VIOLATION`, since it belongs only to `SUBSCRIBE_TRACKS`. A namespace update may change its prefix; suffixes after the acknowledgement are relative to that new prefix. -- **Objects and datagrams.** Only subgroup 0 is accepted: a stream for any other subgroup, whether its header gives the ID or takes it from the first Object ID, is stopped and the session stays up. Rust relays preserve Object IDs, gaps, object properties, FIRST\_OBJECT, and END\_OF\_GROUP status objects between moq-transport peers for subgroup 0, including across a moq-lite-07 hop between two relays that negotiate the [ietf](/draft/moq-ietf) extension. Plain Rust readers and moq-lite peers without the extension see consecutive frames. Rust and JavaScript drop every OBJECT\_DATAGRAM until they map a [datagram track](/concept/moq-lite#datagrams) onto moq-transport. A malformed datagram closes the session. A relayed Timestamp leaves in the units the downstream response declares, and is dropped where it declares none, as for a plain frame; an immutable one that would change or vanish is refused. On a timed track the frame holds each object's time, so a mutable Timestamp that only repeats the immutable one is not restated; a status object has no frame, so it keeps its Timestamp among its properties. Only property 0x10 is read as a Timestamp. Property 0x06 stays opaque and keeps its value regardless of timestamp units; it is dropped between draft 18 and later drafts, or refused when immutable, because draft 20 assigns it to SUBGROUP\_DELIVERY\_TIMEOUT. A status object carrying properties is refused. A `FETCH` has no status objects, so its response ends at the last frame. A group cannot resume across route changes by frame ordinal once Object IDs skip ahead of the ordinals on either route; recovery refuses it instead of splicing different objects. +- **Objects and datagrams.** Only subgroup 0 is accepted: a stream for any other subgroup, whether its header gives the ID or takes it from the first Object ID, is stopped and the session stays up. Rust relays preserve Object IDs, gaps, object properties, FIRST\_OBJECT, and END\_OF\_GROUP status objects between moq-transport peers for subgroup 0, including across a moq-lite-07 hop between two relays that negotiate the [ietf](/draft/moq-ietf) extension. Plain Rust readers and moq-lite peers without the extension see consecutive frames. moq-transport maps a [datagram track](/concept/moq-lite#datagrams) to `SUBSCRIBE` and `OBJECT_DATAGRAM`, with its sequence as the group ID and object ID 0. One upstream subscription serves both kinds of a name. Stream consumers receive only stream objects, and datagram consumers receive only object 0 datagrams, including through relays. An undefined Object Status or malformed datagram framing closes the session. Unreadable object properties drop only that datagram. A relayed Timestamp leaves in the units the downstream response declares, and is dropped where it declares none, as for a plain frame; an immutable one that would change or vanish is refused. On a timed track the frame holds each object's time, so a mutable Timestamp that only repeats the immutable one is not restated; a status object has no frame, so it keeps its Timestamp among its properties. Only property 0x10 is read as a Timestamp. Property 0x06 stays opaque and keeps its value regardless of timestamp units; it is dropped between draft 18 and later drafts, or refused when immutable, because draft 20 assigns it to SUBGROUP\_DELIVERY\_TIMEOUT. A status object carrying properties is refused. A `FETCH` has no status objects, so its response ends at the last frame. A group cannot resume across route changes by frame ordinal once Object IDs skip ahead of the ordinals on either route; recovery refuses it instead of splicing different objects. - **Dynamic groups.** A track advertises `DYNAMIC_GROUPS` when its publisher sets a refresh ratio, and a track received with `DYNAMIC_GROUPS` gets a ratio of 0. A `NEW_GROUP_REQUEST` counts as one more subscriber joining, which a `SUBSCRIBE` already is: it asks for a new group only when the newest group's bytes after its first frame exceed the ratio times the first frame. Joins coalesce until the asked-for group arrives. A Rust relay sends `NEW_GROUP_REQUEST` upstream, on the `SUBSCRIBE` or in a `REQUEST_UPDATE`, once a join wants a group past the newest it holds; a relay holding nothing yet asks for none. On a track without a ratio, an update carrying one ends the subscription with `UPDATE_FAILED`. A moq-lite-07 hop (`moq-lite-07-wip`, opt-in) carries the same demand natively: the ratio in `TRACK_INFO` and the asked-for group in `SUBSCRIBE_UPDATE`. Earlier moq-lite versions carry neither, so a track received over one never refreshes. - **Priority.** Higher is served first. This stack's priorities run from -128 to 127 with 0 unset, and go out on the IETF wire, which ranks lower first, as `127 - priority`: an unset priority is IETF 127. An omitted subscriber or publisher priority reads as the drafts' default of IETF 128, one step below unset, so a relay forwards it unchanged. A relay's upstream subscription carries the highest priority its subscribers ask for and updates it as they come and go; its group order stays as the `SUBSCRIBE` asked, since no draft lets an update change it. - **Size.** An object extension block larger than 64 KiB ends that subgroup stream. The session stays up. This cap is ours, not the draft's. diff --git a/js/net/MIGRATE.md b/js/net/MIGRATE.md index 5d288408d44..d43665e8430 100644 --- a/js/net/MIGRATE.md +++ b/js/net/MIGRATE.md @@ -44,6 +44,14 @@ The fallback now grants 4 MiB per connection and 1 MiB per stream, up from or retain the old limits. Both accept positive safe integers, like `websocket.maxStreams`. `@moq/qmux`'s standalone defaults are unchanged. +### moq-transport datagram tracks + +Datagram tracks now work over moq-transport using SUBSCRIBE and OBJECT\_DATAGRAM. +The sequence is the group ID and the object ID is zero. One upstream subscription +serves both kinds of a name, including through relays. Stream and datagram +consumers keep only their requested kind; nonzero object IDs are dropped by +datagram consumers. No application API or wire encoding changes are required. + ### Datagrams use separate track types ([#7846](https://github.com/moq-dev/moq/pull/7846)) Create a datagram track with `broadcast.createDatagrams(name, info)` and read it with @@ -64,8 +72,7 @@ stream resume state and does not move to a replacement route. Lite-07-wip uses DATAGRAMS on stream type 0x9 and carries no Inflated Size. Lite-05/06 keep their wire format and carry both kinds of a name on one SUBSCRIBE, without deduplication. Create a datagram track before or with its same-name stream track on those versions. -Lite-01..04 and transports without datagrams refuse datagram requests. moq-transport -currently refuses them while its separate datagram-track adapter is implemented. +Lite-01..04 and transports without datagrams refuse datagram requests. moq-transport carries them as OBJECT\_DATAGRAM with object ID zero. ### Announcements are a listing ([#7678](https://github.com/moq-dev/moq/pull/7678)) diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index e562959b4de..39cd0104c7f 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -57,14 +57,14 @@ class BroadcastState { } // Datagram tracks never enter the stream listing or cache. -function liveDatagrams(state: BroadcastState, name: string): datagram.Consumer | undefined { +function liveDatagrams(state: BroadcastState, name: string, passive = false): datagram.Consumer | undefined { const producer = state.datagrams.get(name); if (!producer) return; if (producer.closed.peek() !== undefined) { state.datagrams.delete(name); return; } - return producer.consume(); + return passive ? hooks.observeDatagrams(producer) : producer.consume(); } function createDatagrams(state: BroadcastState, name: string, info: Partial): datagram.Producer { @@ -378,13 +378,14 @@ export class Producer { constructor() { registerWire(this, { + localDatagrams: !this.#state.unheld, subscribe: (name, options) => subscribe(this.#state, name, options), resolveTrackInfo: (name, hold) => resolveTrackInfo(this.#state, name, hold), largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), datagrams: (name) => datagrams(this.#state, name), - liveDatagrams: (name) => liveDatagrams(this.#state, name), + liveDatagrams: (name) => liveDatagrams(this.#state, name, true), createDatagrams: (name, info) => createDatagrams(this.#state, name, info), requestedDatagrams: () => requestedDatagrams(this.#state), tracks: (owner) => listTracks(this.#state.listing, owner), @@ -539,13 +540,14 @@ export class Consumer { } this.#state.consumers++; registerWire(this, { + localDatagrams: !this.#state.unheld, subscribe: (name, options) => subscribe(this.#state, name, options), resolveTrackInfo: (name, hold) => resolveTrackInfo(this.#state, name, hold), largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), datagrams: (name) => datagrams(this.#state, name), - liveDatagrams: (name) => liveDatagrams(this.#state, name), + liveDatagrams: (name) => liveDatagrams(this.#state, name, true), createDatagrams: (name, info) => createDatagrams(this.#state, name, info), requestedDatagrams: () => requestedDatagrams(this.#state), tracks: (owner) => this.#tracks(owner), diff --git a/js/net/src/datagram.test.ts b/js/net/src/datagram.test.ts index a2d6ab801cf..0a1af9eba18 100644 --- a/js/net/src/datagram.test.ts +++ b/js/net/src/datagram.test.ts @@ -185,3 +185,26 @@ test("closing one waiting datagram consumer releases its info wait while another second.close(); track.finish(); }); + +test("passive forwarding drains a finished track without adding demand", async () => { + const producer = new Datagram.Producer("audio").accept({}); + const changes: (number | undefined)[] = []; + const dispose = producer.priority.subscribe((value) => changes.push(value)); + const observer = hooks.observeDatagrams(producer); + const copy = observer.clone(); + expect(changes).toEqual([]); + dispose(); + expect(producer.priority.peek()).toBeUndefined(); + const reader = producer.consume(); + reader.setPriority(-7); + expect(producer.priority.peek()).toBe(-7); + reader.close(); + expect(producer.priority.peek()).toBeUndefined(); + producer.append(undefined, Uint8Array.of(1)); + producer.finish(); + for (const consumer of [observer, copy]) { + expect((await consumer.recv())?.payload).toEqual(Uint8Array.of(1)); + expect(await consumer.recv()).toBeUndefined(); + consumer.close(); + } +}); diff --git a/js/net/src/datagram.ts b/js/net/src/datagram.ts index 2e111d3847d..ee570c3cf48 100644 --- a/js/net/src/datagram.ts +++ b/js/net/src/datagram.ts @@ -45,7 +45,7 @@ class State { closed = new Once(); changed = new Signal(0); priority = new Signal(undefined); - consumers = new Map(); + consumers = new Map(); buffer: Datagram[] = []; bytes = 0; offset = 0n; @@ -65,9 +65,10 @@ class State { updatePriority(): void { let highest: number | undefined; - for (const value of this.consumers.values()) highest = Math.max(highest ?? -128, value); + for (const value of this.consumers.values()) + if (value !== undefined) highest = Math.max(highest ?? -128, value); this.priority.set(highest); - if (highest === undefined) { + if (this.consumers.size === 0) { if (this.info.peek() === undefined) this.end(new StreamError(StreamCode.Cancel)); if (this.closed.peek() !== undefined) this.release(); } @@ -88,7 +89,7 @@ async function info(state: State, local?: GetPromise): Promise Consumer; +let consume: (name: string, state: State, cursor?: bigint, priority?: number | null) => Consumer; /** Writes a live datagram track, independently of any stream track of the same name. */ export class Producer { @@ -101,6 +102,7 @@ export class Producer { } static { + hooks.observeDatagrams = (producer) => consume(producer.name, producer.#state, undefined, null); hooks.closeUnusedDatagrams = (producer) => { if (producer.#state.consumers.size > 0) return false; producer.abort(new StreamError(StreamCode.Cancel)); @@ -230,13 +232,13 @@ export class Consumer { #closed = new Once(); #priority: Signal; - private constructor(name: string, state: State, cursor?: bigint, value = 0) { + private constructor(name: string, state: State, cursor?: bigint, value: number | null = 0) { this.name = name; this.#state = state; this.#cursor = cursor ?? state.offset + BigInt(state.buffer.length); - this.#priority = new Signal(value); + this.#priority = new Signal(value ?? 0); if (state.closed.peek() === undefined || cursor !== undefined) { - state.consumers.set(this, value); + state.consumers.set(this, value ?? undefined); state.updatePriority(); } } @@ -267,7 +269,8 @@ export class Consumer { /** Another consumer at the same position and priority. */ clone(): Consumer { if (this.#closed.peek() !== undefined) throw new Error("datagram consumer is closed"); - return consume(this.name, this.#state, this.#cursor, this.#priority.peek()); + const passive = this.#state.consumers.has(this) && this.#state.consumers.get(this) === undefined; + return consume(this.name, this.#state, this.#cursor, passive ? null : this.#priority.peek()); } /** Read the next datagram as a copy this consumer owns, skipping any dropped by the bounded shared buffer. */ diff --git a/js/net/src/ietf/datagram.test.ts b/js/net/src/ietf/datagram.test.ts index 3def32fb43d..978c00e816d 100644 --- a/js/net/src/ietf/datagram.test.ts +++ b/js/net/src/ietf/datagram.test.ts @@ -1,8 +1,17 @@ -import { expect, test } from "bun:test"; -import { ProtocolViolation } from "../error.ts"; -import { encodeVarint } from "../stream.ts"; +import { expect, spyOn, test } from "bun:test"; +import { yieldNow } from "@moq/clock"; +import { accept, connect } from "../connection/index.ts"; +import { ProtocolViolation, SessionCode } from "../error.ts"; +import { createMockTransportPair, type MockTransport } from "../mock.ts"; +import { Producer as Origin } from "../origin.ts"; +import * as Path from "../path.ts"; +import { encodeVarint, Stream } from "../stream.ts"; +import { Timescale, Timestamp } from "../time.ts"; +import { Budget } from "../util/yield.ts"; +import { wireOf } from "../wire.ts"; +import { Connection } from "./connection.ts"; import { ObjectDatagram } from "./datagram.ts"; -import { type IetfVersion, Version } from "./version.ts"; +import { ALPN, type IetfVersion, Version } from "./version.ts"; const ALL: IetfVersion[] = [Version.DRAFT_18, Version.DRAFT_20, Version.DRAFT_22]; @@ -59,3 +68,297 @@ test("rejects invalid types", async () => { } } }); + +/** A server session, so a datagram written by the client is received. */ +async function session() { + const version = Version.DRAFT_22; + const pair = createMockTransportPair(ALPN.DRAFT_22); + const control = await Stream.open(pair.server, { version }); + const connection = new Connection({ + url: new URL("https://example.com"), + quic: pair.server, + control, + version, + client: false, + }); + const writable = pair.client.datagrams.writable; + if (!writable) throw new Error("mock datagrams have no writable"); + return { pair, connection, version, writable }; +} + +test("an undefined object status closes the session", async () => { + const { pair, connection, version, writable } = await session(); + try { + const bytes = new ObjectDatagram({ + trackAlias: 1n, + groupId: 4n, + endOfGroup: false, + body: { status: 1 }, + }).encode(version); + const writer = writable.getWriter(); + await writer.write(bytes); + const closed = await pair.server.closed; + expect(closed.closeCode).toBe(SessionCode.ProtocolViolation); + } finally { + connection.abort(); + } +}); + +test("a defined object status does not close the session", async () => { + const { pair, connection, version, writable } = await session(); + try { + const bytes = new ObjectDatagram({ + trackAlias: 1n, + groupId: 4n, + endOfGroup: false, + body: { status: 0 }, + }).encode(version); + const writer = writable.getWriter(); + await writer.write(bytes); + const open = Symbol("open"); + const result = await Promise.race([pair.server.closed, Bun.sleep(50).then(() => open)]); + expect(result).toBe(open); + } finally { + connection.abort(); + } +}); + +test("a datagram track round-trips over moq-transport", async () => { + const source = new Origin(); + const path = Path.from("live"); + const publication = source.route(path, { hidden: true }); + const datagrams = publication.broadcast.createDatagrams("audio", { + timescale: Timescale.MILLI, + priority: 3, + }); + publication.announce(); + const pair = createMockTransportPair(ALPN.DRAFT_22); + const url = new URL("https://localhost/datagrams"); + const peers = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ url, transport: pair.server, publish: source.consume() }), + ]); + const remote = wireOf(peers[0]).consume(path); + const reader = remote.datagrams("audio"); + reader.setPriority(1); + try { + const info = await reader.info(); + while (datagrams.priority.peek() !== 1) await datagrams.priority.changed(); + expect(info.priority).toBe(3); + expect(info.timescale).toBe(Timescale.MILLI); + // A peer's unreadable properties lose one best-effort payload, not the session. + const encode = ObjectDatagram.prototype.encode; + const corrupt = spyOn(ObjectDatagram.prototype, "encode").mockImplementation(function ( + this: ObjectDatagram, + version, + ) { + corrupt.mockRestore(); + return encode.call(new ObjectDatagram({ ...this, properties: Uint8Array.of(0x10) }), version); + }); + datagrams.insert({ sequence: 3n, timestamp: Timestamp.fromMillis(8), payload: Uint8Array.of(0) }); + + datagrams.insert({ sequence: 4n, timestamp: Timestamp.fromMillis(9), payload: new Uint8Array([7]) }); + const received = await reader.recv(); + expect(received?.sequence).toBe(4n); + expect(received?.timestamp?.asMillis()).toBe(9); + expect(received?.payload).toEqual(new Uint8Array([7])); + datagrams.finish(); + expect(await reader.recv()).toBeUndefined(); + } finally { + reader.close(); + await peers[0].close(); + await peers[1].close(); + source.close(); + } +}); + +// Either model kind can arrive first; the relay must retain one upstream request and +// route each payload only to readers of its kind, even after one reader leaves. +test("a relay shares one IETF subscription across stream and datagram readers", async () => { + for (const datagramFirst of [false, true]) { + const source = new Origin(); + const relay = new Origin(); + const path = Path.from("shared"); + const publication = source.route(path, { hidden: true }); + const track = publication.broadcast.createTrack("audio", { timescale: Timescale.MILLI }); + const datagrams = publication.broadcast.createDatagrams("audio", { timescale: Timescale.MICRO }); + publication.announce(); + const upstream = createMockTransportPair(ALPN.DRAFT_18); + const url = new URL("https://localhost/shared"); + const peers = await Promise.all([ + connect({ url, transport: upstream.client }), + accept({ url, transport: upstream.server, publish: source.consume() }), + ]); + const original = wireOf(peers[0]).consume(path); + const requests = relay.routePrefix(path); + const cursor = requests.requested(); + const routing = (async () => { + for await (const request of cursor) request.serve(original); + })(); + const downstream = createMockTransportPair(ALPN.DRAFT_22); + const viewers = await Promise.all([ + connect({ url, transport: downstream.client }), + accept({ url, transport: downstream.server, publish: relay.consume() }), + ]); + const remote = wireOf(viewers[0]).consume(path); + const stream = datagramFirst ? undefined : remote.track("audio").subscribe(); + if (stream) await stream.info(); + const streamCount = stream && upstream.client.sendStreams.bidi.length; + let reader = remote.datagrams("audio"); + await reader.info(); + const before = streamCount ?? upstream.client.sendStreams.bidi.length; + const subscriber = stream ?? remote.track("audio").subscribe(); + await subscriber.info(); + expect(upstream.client.sendStreams.bidi.length).toBe(before); + try { + datagrams.insert({ sequence: 4n, timestamp: Timestamp.fromMillis(9), payload: Uint8Array.of(7) }); + const group = track.appendGroup(); + group.writeFrame({ timestamp: Timestamp.fromMillis(10), payload: Uint8Array.of(8) }); + group.finish(); + const received = await reader.recv(); + expect(received?.payload).toEqual(Uint8Array.of(7)); + expect(received?.timestamp?.asMillis()).toBe(9); + const receivedGroup = await subscriber.recvGroup(); + expect((await receivedGroup?.readFrame())?.payload).toEqual(Uint8Array.of(8)); + receivedGroup?.close(); + subscriber.close(); + datagrams.insert({ sequence: 5n, timestamp: Timestamp.fromMillis(11), payload: Uint8Array.of(9) }); + expect((await reader.recv())?.payload).toEqual(Uint8Array.of(9)); + reader.close(); + while (track.subscription.peek() !== undefined) await track.subscription.changed(); + reader = remote.datagrams("audio"); + await reader.info(); + datagrams.insert({ sequence: 6n, timestamp: Timestamp.fromMillis(12), payload: Uint8Array.of(10) }); + expect((await reader.recv())?.payload).toEqual(Uint8Array.of(10)); + } finally { + subscriber.close(); + reader.close(); + for (const peer of viewers) await peer.close(); + requests.close(); + await routing; + for (const peer of peers) await peer.close(); + relay.close(); + source.close(); + } + } +}); + +function lagAnswers(transport: MockTransport, lag: number): void { + if (lag === 0) return; + const lagging = (writable: WritableStream) => { + const writer = writable.getWriter(); + let chain = Promise.resolve(); + const later = (op: () => Promise) => { + // A write the session's close fails is lost, as it would be on the wire. + chain = chain + .then(async () => { + for (let i = 0; i < lag; i++) await Promise.resolve(); + await op(); + }) + .catch(() => {}); + return chain; + }; + return new WritableStream({ + // Copied: the write resolves now, after which the stream may reuse its buffer. + write: (chunk) => { + const copy = chunk.slice(); + void later(() => writer.write(copy)); + }, + close: () => later(() => writer.close()), + abort: (reason) => writer.abort(reason), + }); + }; + const incoming = transport.incomingBidirectionalStreams.pipeThrough( + new TransformStream({ + transform: (bidi, controller) => + controller.enqueue({ + readable: bidi.readable, + writable: lagging(bidi.writable), + } as WebTransportBidirectionalStream), + }), + ); + Object.defineProperty(transport, "incomingBidirectionalStreams", { value: incoming }); +} + +for (const alpn of [ALPN.DRAFT_18, ALPN.DRAFT_20, ALPN.DRAFT_22]) { + for (const [lag, finish] of [ + [0, false], + [100, false], + [0, true], + [100, true], + ] as const) { + test(`${alpn}: a datagram sent right behind its answer arrives (answer ${lag} microtasks late${finish ? ", then the end" : ""})`, async () => { + const source = new Origin(); + const publication = source.route(Path.from("live"), { hidden: true }); + const datagrams = publication.broadcast.createDatagrams("audio"); + publication.announce(); + const pair = createMockTransportPair(alpn); + lagAnswers(pair.server, lag); + const [client, server] = await Promise.all([ + connect({ url: new URL("https://localhost/datagrams"), transport: pair.client }), + accept({ + url: new URL("https://localhost/datagrams"), + transport: pair.server, + publish: source.consume(), + }), + ]); + const remote = wireOf(client).consume(Path.from("live")); + const reader = remote.datagrams("audio"); + // Only the subscription carries this priority: a lite-06 TRACK lookup asks for none. + reader.setPriority(5); + try { + while (datagrams.priority.peek() !== 5) await datagrams.priority.changed(); + datagrams.append(undefined, Uint8Array.of(1)); + if (finish) datagrams.finish(); + await reader.info(); + if (!finish) datagrams.append(undefined, Uint8Array.of(2)); + expect((await reader.recv())?.payload).toEqual(Uint8Array.of(1)); + if (finish) expect(await reader.recv()).toBeUndefined(); + else expect((await reader.recv())?.payload).toEqual(Uint8Array.of(2)); + } finally { + reader.close(); + remote.close(); + client.abort(); + server.abort(); + source.close(); + } + }); + } +} + +for (const alpn of [ALPN.DRAFT_18, ALPN.DRAFT_20, ALPN.DRAFT_22]) { + test(`${alpn}: a datagram burst reaches its reader before the end across a receive yield`, async () => { + const source = new Origin(); + const publication = source.route(Path.from("live"), { hidden: true }); + const datagrams = publication.broadcast.createDatagrams("audio"); + publication.announce(); + const pair = createMockTransportPair(alpn); + const url = new URL("https://localhost/datagrams"); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ url, transport: pair.server, publish: source.consume() }), + ]); + const remote = wireOf(client).consume(Path.from("live")); + const reader = remote.datagrams("audio"); + reader.setPriority(5); + let polls = 0; + const budget = spyOn(Budget.prototype, "poll").mockImplementation(() => + ++polls % 32 === 0 ? yieldNow() : undefined, + ); + try { + await reader.info(); + while (datagrams.priority.peek() !== 5) await datagrams.priority.changed(); + for (let i = 0; i < 64; i++) datagrams.append(undefined, Uint8Array.of(i)); + datagrams.finish(); + for (let i = 0; i < 64; i++) expect((await reader.recv())?.payload).toEqual(Uint8Array.of(i)); + expect(await reader.recv()).toBeUndefined(); + } finally { + budget.mockRestore(); + reader.close(); + remote.close(); + client.abort(); + server.abort(); + source.close(); + } + }); +} diff --git a/js/net/src/ietf/object.ts b/js/net/src/ietf/object.ts index 1abeca2840e..0ccfcee6544 100644 --- a/js/net/src/ietf/object.ts +++ b/js/net/src/ietf/object.ts @@ -14,6 +14,11 @@ const MAX_OBJECT_EXTENSIONS = 64 * 1024; const GROUP_END = 0x03; const END_OF_TRACK = 0x04; +/** Whether `status` is an Object Status the drafts define: Normal, End of Group, or End of Track. */ +export function statusDefined(status: number): boolean { + return status === 0 || status === GROUP_END || status === END_OF_TRACK; +} + // MOQ Object Property ids, shared with draft-ietf-moq-loc-04. An object-scope Timescale // (0x08) is never applied: a track's timeline is its TIMESCALE Track Property alone. const PROP_TIMESTAMP = 0x10n; diff --git a/js/net/src/ietf/publisher.test.ts b/js/net/src/ietf/publisher.test.ts index 256cee7b9f4..bce722051da 100644 --- a/js/net/src/ietf/publisher.test.ts +++ b/js/net/src/ietf/publisher.test.ts @@ -3,6 +3,7 @@ import { Elapsed, Manual, Milli, Nano, sleep, timeout, yieldNow } from "@moq/clo import { Signal } from "@moq/signals"; import type { Credential, Grant } from "../auth.ts"; import { Producer as BroadcastProducer } from "../broadcast.ts"; +import { accept, connect } from "../connection/index.ts"; import { error, fromTransport, @@ -25,6 +26,7 @@ import type { Producer as TrackProducer } from "../track.ts"; import { unwanted } from "../util/demand.ts"; import { wireOf } from "../wire.ts"; import type * as Cluster from "./cluster.ts"; +import { ObjectDatagram } from "./datagram.ts"; import { toRequestCode } from "./error.ts"; import { Fetch, FetchHeader } from "./fetch.ts"; import * as Filter from "./filter.ts"; @@ -2197,6 +2199,7 @@ const ALPNS: Record = { function fixture( version: IetfVersion = V22, grant?: Signal, + pair = createMockTransportPair(ALPNS[version]), ): { pair: ReturnType; pub: Publisher; @@ -2205,7 +2208,6 @@ function fixture( version: IetfVersion; close: () => void; } { - const pair = createMockTransportPair(ALPNS[version]); const session = new NativeSession(pair.server, version, false); const { pub, origin } = publisher(pair.server, { session, grant }); const broadcast = publish(origin, Path.from("test")); @@ -3625,6 +3627,205 @@ test("a SUBSCRIBE pauses without demand and resumes as a new subscriber", async } }); +test("a paused relay subscription reacquires its upstream datagrams", async () => { + const source = new OriginProducer(); + const route = source.route(Path.from("test")); + const track = route.broadcast.createTrack("audio", {}); + const datagrams = route.broadcast.createDatagrams("audio", {}); + const upstream = createMockTransportPair(ALPN.DRAFT_18); + const url = new URL("https://localhost/resume"); + const peers = await Promise.all([ + connect({ url, transport: upstream.client }), + accept({ url, transport: upstream.server, publish: source.consume() }), + ]); + const original = wireOf(peers[0]).consume(Path.from("test")); + const relay = new OriginProducer(); + const requests = relay.routePrefix(Path.from("test")); + const cursor = requests.requested(); + const routing = (async () => { + for await (const request of cursor) request.serve(original); + })(); + const fx = fixture(V22); + fx.pub.close(); + fx.pub = new Publisher({ + quic: fx.pair.server, + session: new NativeSession(fx.pair.server, V22, false), + publish: relay.consume(), + requiresSolicitation: false, + }); + const received = fx.pair.client.datagrams.readable.getReader(); + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: 0n, + trackNamespace: Path.from("test"), + trackName: "audio", + subscriberPriority: 128, + }), + ); + const forward = async (id: bigint, on: boolean) => { + const parameters = new Parameters(); + parameters.forward = on; + await client.writer.u53(SubscribeUpdate.id); + await new SubscribeUpdate({ requestId: id, parameters }).encode(client.writer, V22); + expect(await client.reader.u53()).toBe(RequestOk.id); + await RequestOk.decode(client.reader, V22); + }; + try { + datagrams.append(undefined, Uint8Array.of(1)); + expect((await received.read()).done).toBe(false); + await forward(2n, false); + while (track.subscription.peek() !== undefined) await track.subscription.changed(); + const before = upstream.client.sendStreams.bidi.length; + await forward(4n, true); + expect(upstream.client.sendStreams.bidi.length).toBe(before + 1); + datagrams.append(undefined, Uint8Array.of(2)); + const next = await received.read(); + if (!next.value) throw new Error("no resumed relay datagram"); + expect((await ObjectDatagram.decode(next.value, V22)).body).toEqual({ payload: Uint8Array.of(2) }); + } finally { + client.close(); + fx.pub.close(); + received.releaseLock(); + fx.close(); + requests.close(); + await routing; + for (const peer of peers) peer.abort(); + original.close(); + relay.close(); + source.close(); + } +}); + +test.each(["writable", "createWritable"] as const)( + "concurrent IETF subscriptions share the %s datagram transport", + async (datagramWritable) => { + const fx = fixture(V22, undefined, createMockTransportPair(ALPN.DRAFT_22, { datagramWritable })); + const producers = ["audio", "video"].map((name) => fx.broadcast.createDatagrams(name, {})); + const received = fx.pair.client.datagrams.readable.getReader(); + const clients: Stream[] = []; + try { + for (const [i, name] of ["audio", "video"].entries()) { + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: BigInt(i * 2), + trackNamespace: Path.from("test"), + trackName: name, + subscriberPriority: 128, + }), + ); + clients.push(client); + } + for (const [i, producer] of producers.entries()) { + producer.append(undefined, Uint8Array.of(i)); + const next = await received.read(); + if (!next.value) throw new Error("missing concurrent datagram"); + const decoded = await ObjectDatagram.decode(next.value, V22); + expect(decoded.trackAlias).toBe(BigInt(i * 2)); + expect(decoded.body).toEqual({ payload: Uint8Array.of(i) }); + } + } finally { + for (const client of clients) client.close(); + for (const producer of producers) producer.finish(); + received.releaseLock(); + fx.pub.close(); + fx.close(); + } + }, +); + +test("a paused IETF subscription leaves a blocked datagram write", async () => { + const pair = createMockTransportPair(ALPN.DRAFT_22); + const started = Promise.withResolvers(); + const release = Promise.withResolvers(); + Object.defineProperty(pair.server.datagrams, "writable", { + value: new WritableStream({ + write: async () => { + started.resolve(); + await release.promise; + }, + }), + }); + const fx = fixture(V22, undefined, pair); + const datagrams = fx.broadcast.createDatagrams("audio", {}); + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: 0n, + trackNamespace: Path.from("test"), + trackName: "audio", + subscriberPriority: 128, + }), + ); + try { + datagrams.append(undefined, Uint8Array.of(1)); + await started.promise; + const parameters = new Parameters(); + parameters.forward = false; + await client.writer.u53(SubscribeUpdate.id); + await new SubscribeUpdate({ requestId: 2n, parameters }).encode(client.writer, V22); + expect(await client.reader.u53()).toBe(RequestOk.id); + await RequestOk.decode(client.reader, V22); + expect(datagrams.priority.peek()).toBeUndefined(); + } finally { + release.resolve(); + client.close(); + datagrams.finish(); + fx.pub.close(); + fx.close(); + } +}); + +test("a datagram-only SUBSCRIBE releases demand on pause and resumes", async () => { + for (const startPaused of [false, true]) { + const fx = fixture(V22); + const datagrams = fx.broadcast.createDatagrams("audio", {}); + const received = fx.pair.client.datagrams.readable.getReader(); + const { client } = await runSubscribe( + fx, + new Subscribe({ + requestId: 0n, + trackNamespace: Path.from("test"), + trackName: "audio", + subscriberPriority: 128, + forward: !startPaused, + }), + ); + const forward = async (id: bigint, on: boolean) => { + const parameters = new Parameters(); + parameters.forward = on; + await client.writer.u53(SubscribeUpdate.id); + await new SubscribeUpdate({ requestId: id, parameters }).encode(client.writer, V22); + expect(await client.reader.u53()).toBe(RequestOk.id); + await RequestOk.decode(client.reader, V22); + }; + try { + if (!startPaused) { + datagrams.append(undefined, Uint8Array.of(1)); + expect((await received.read()).done).toBe(false); + await forward(2n, false); + } + while (datagrams.priority.peek() !== undefined) await datagrams.priority.changed(); + datagrams.append(undefined, Uint8Array.of(2)); + await forward(4n, true); + datagrams.append(undefined, Uint8Array.of(3)); + const next = await received.read(); + if (!next.value) throw new Error("no resumed datagram"); + const decoded = await ObjectDatagram.decode(next.value, V22); + expect(decoded.body).toEqual({ payload: Uint8Array.of(3) }); + datagrams.finish(); + expect(await client.reader.u53()).toBe(PublishDone.id); + expect((await PublishDone.decode(client.reader, V22)).streamCount).toBe(0n); + } finally { + received.releaseLock(); + client.close(); + datagrams.finish(); + fx.close(); + } + } +}); + test("a paused SUBSCRIBE ends once its grant is revoked", async () => { const grant = new Signal(undefined); const fx = fixture(V22, grant); diff --git a/js/net/src/ietf/publisher.ts b/js/net/src/ietf/publisher.ts index 7bb17df7053..96aa319d289 100644 --- a/js/net/src/ietf/publisher.ts +++ b/js/net/src/ietf/publisher.ts @@ -6,6 +6,8 @@ import type { AuthSession } from "../auth_session.ts"; import { enforceGrant } from "../auth_session.ts"; import type * as broadcast from "../broadcast.ts"; import { Withdrawal } from "../connection/withdrawal.ts"; +import type * as datagram from "../datagram.ts"; +import * as DatagramStream from "../datagram_stream.ts"; import { controlTimeout, error, @@ -27,7 +29,14 @@ import { sendPayload } from "../payload.ts"; import { Priority, sendOrder } from "../priority.ts"; import { type Stream, Writer } from "../stream.ts"; import { Timescale } from "../time.ts"; -import type { Groups, Info, Range, Position as TrackPosition, Subscriber as TrackSubscriber } from "../track.ts"; +import { + type Groups, + type Info, + type Range, + type Position as TrackPosition, + Producer as TrackProducer, + type Subscriber as TrackSubscriber, +} from "../track.ts"; import { untilAborted } from "../util/abort.ts"; import { toNumber } from "../util/bigint.ts"; import { nextChange } from "../util/change.ts"; @@ -38,6 +47,7 @@ import { TimeoutError, withTimeout } from "../util/timeout.ts"; import * as Varint from "../varint.ts"; import { type Advertised, type Advertisements, sameInstance, wireOf } from "../wire.ts"; import * as Cluster from "./cluster.ts"; +import { encodeDatagram } from "./datagram.ts"; import { requestReason } from "./error.ts"; import { type Fetch, FetchHeader, FetchOk, type FetchType } from "./fetch.ts"; import * as Filter from "./filter.ts"; @@ -314,6 +324,7 @@ export class Publisher { return this.#withdrawal.close(); } #quic: WebTransport; + #datagramWriter?: ReturnType; #session: Session; #requiresSolicitation: boolean; // What a joining FETCH needs from each draft-18 SUBSCRIBE, by its request ID, while it lasts: @@ -396,6 +407,7 @@ export class Publisher { ready?: Promise; }) { this.#quic = quic; + this.#datagramWriter = DatagramStream.datagramWriter(quic); this.#grant = grant; this.#gate = new Gate(grant); this.#auth = auth; @@ -411,7 +423,10 @@ export class Publisher { } /** The origin owns broadcasts; transport teardown ends this publisher's served requests. @internal */ - close(): void {} + close(): void { + this.#datagramWriter?.close(); + this.#datagramWriter = undefined; + } /** * Handles an incoming SUBSCRIBE request on a bidi stream. @@ -498,45 +513,80 @@ export class Publisher { }), ); }; + // Before any await, so a subscriber waiting on this priority is not stuck behind setup. + // A transport without datagrams never asks: the lookup would open an upstream + // subscription nothing here can carry. + const datagramOf = () => { + if (DatagramStream.maxDatagramSize(this.#quic) === 0) return; + const local = wireOf(broadcast).localDatagrams; + const consumer = local + ? broadcast.datagrams(msg.trackName) + : wireOf(broadcast).liveDatagrams(msg.trackName); + if (local) consumer?.setPriority(fromWire(msg.subscriberPriority)); + return consumer; + }; + let datagrams = datagramOf(); let track = subscribe(broadcast); - try { - let info: Awaited>; - try { - info = await untilAborted(track.info(), left.signal); - } catch (err) { - if (left.signal.aborted) { - request.abort(cancel); - return; - } - const failed = error(err); - if ( - !(failed instanceof StreamError) || - (failed.code !== StreamCode.NotFound && failed.code !== StreamCode.Unroutable) - ) - throw err; - // The broadcast resolved, so only a vanished route still means a missing publisher; - // a missing track is the publisher's answer. - await request.refuse( - "subscribe", - this.#failed(err, rendezvous && failed.code === StreamCode.Unroutable), - ); - return; + let datagramTrack: TrackProducer | undefined; + const resolve = async () => { + // A relay learns both halves from the same stream response. Only local + // application handlers can answer either kind independently. + const streamInfo = track.info().then((info) => ({ kind: "stream" as const, info })); + const consumer = datagrams; + const selected = await untilAborted( + consumer && wireOf(broadcast).localDatagrams + ? Promise.any([ + streamInfo, + consumer.info().then((info) => ({ kind: "datagram" as const, info })), + ]).catch(() => streamInfo) + : streamInfo, + left.signal, + ); + if (selected.kind === "datagram") { + track.close(); + datagramTrack = new TrackProducer(msg.trackName).accept(selected.info); + track = hooks.parked(datagramTrack.subscribe({ priority: fromWire(msg.subscriberPriority) })); } + return selected.info; + }; + try { + const info = await resolve().catch(async (err) => { + if (left.signal.aborted) request.abort(cancel); + else await request.refuse("subscribe", this.#failed(err, rendezvous, true)); + return undefined; + }); + if (!info) return; + datagrams ??= datagramOf(); // Every subscription a resume makes counts toward the same PUBLISH_DONE. const streams: StreamCount = { opened: 0 }; - let paused = await this.#forward({ msg, request, track, info, streams }); + let paused = await this.#forward({ + msg, + request, + track, + info, + streams, + datagrams, + datagramDemand: wireOf(broadcast).localDatagrams, + datagramTrack, + }); // A pause drops the subscription, and its demand with it; a resume subscribes again as // a new SUBSCRIBE would, so nothing carries across but the request. while (paused) { track.close(); + datagramTrack?.abort(cancel); + datagramTrack = undefined; + datagrams?.close(); + datagrams = undefined; if (paused.answer) await Message.write(request.writer, RequestOk.id, new RequestOk(), version); const answer = await this.#pausedSubscribe(msg, request, streams); if (!answer) return; track = subscribe(broadcast); + datagrams = datagramOf(); let resumed: Info; try { - resumed = await untilAborted(track.info(), left.signal); + resumed = await resolve(); + datagrams ??= datagramOf(); // A resume answers with the live edge, which only a granted subscriber learns. if (!this.#gate.allows(name)) throw unauthorized(name); } catch (err) { @@ -546,10 +596,22 @@ export class Publisher { request.close(); return; } - paused = await this.#forward({ msg, request, track, info: resumed, streams, answer }); + paused = await this.#forward({ + msg, + request, + track, + info: resumed, + streams, + answer, + datagrams, + datagramDemand: wireOf(broadcast).localDatagrams, + datagramTrack, + }); } } finally { track.close(); + datagramTrack?.abort(cancel); + datagrams?.close(); } } @@ -651,6 +713,9 @@ export class Publisher { info, streams = { opened: 0 }, answer, + datagrams, + datagramDemand, + datagramTrack, }: { msg: Subscribe; request: RequestStream; @@ -660,13 +725,22 @@ export class Publisher { streams?: StreamCount; /** The update a resumed SUBSCRIBE answers with REQUEST_OK instead of SUBSCRIBE_OK. */ answer?: { requestId: bigint }; + /** A datagram track of the same name, forwarded as OBJECT_DATAGRAM beside the groups. */ + datagrams?: datagram.Consumer; + datagramDemand: boolean; + /** Metadata-only track whose lifetime follows the datagram sender. */ + datagramTrack?: TrackProducer; }): Promise { const version = this.#session.version; const name = msg.trackNamespace; const priority = fromWire(msg.subscriberPriority); + const datagramPriority = new Signal(priority); const order = msg.groupOrder; let cache: TrackSubscriber | undefined; let ranking: Priority | undefined; + // Assigned once forwarding starts. An early pause returns before that, and the finally + // still waits so a task begun on another path cannot outlive the subscription. + let datagramTask: Promise = Promise.resolve(); try { // A Timestamp goes out only when this SUBSCRIBE_OK actually carries TIMESCALE. const timescale = msg.propertiesWanted ? (info.timescale ?? undefined) : undefined; @@ -766,6 +840,22 @@ export class Publisher { } console.debug(`publish ok: broadcast=${name} track=${track.name}`); + // Groups are served below; datagrams ride beside them and must not fail this subscription. + // A burst stays behind the group's send order, so it cannot starve the groups. + datagramTask = datagrams + ? this.#sendDatagrams(datagrams, { + alias: msg.requestId, + timescale, + priority: datagramPriority, + }).then( + () => datagramTrack?.finish(), + (err: unknown) => { + if (datagramTrack) datagramTrack.abort(error(err)); + else console.debug(`stopping datagrams on this subscription: ${reason(err)}`); + }, + ) + : Promise.resolve(); + // Cancels groups still queued for a stream slot. Only the subscriber leaving counts: // a track that ran out of groups still has to flush the ones already queued, and we // close the stream ourselves below to say so. @@ -828,8 +918,12 @@ export class Publisher { if (framed === undefined) return request.ended; const { params, requestId: ownRequestId } = await this.#decodeUpdate(request, framed); // update() replaces every option, so carry the rest over as Rust does. - if (params.subscriberPriority !== undefined) - track.update({ ...track.subscription.peek(), priority: fromWire(params.subscriberPriority) }); + if (params.subscriberPriority !== undefined) { + const priority = fromWire(params.subscriberPriority); + track.update({ ...track.subscription.peek(), priority }); + datagramPriority.set(priority); + if (datagramDemand) datagrams?.setPriority(priority); + } const maxAge = params.maxAge; if (maxAge !== undefined) { const current = track.subscription.peek(); @@ -990,7 +1084,7 @@ export class Publisher { // PUBLISH_DONE has no end location: an END_OF_TRACK object is what tells the // subscriber where the track ended. const final = hooks.subscriberEnd(track); - if (ended && !publishError && final !== undefined && !streams.terminalSent) { + if (!datagramTrack && ended && !publishError && final !== undefined && !streams.terminalSent) { await this.#runEndOfTrack({ requestId: msg.requestId, final, @@ -1059,6 +1153,8 @@ export class Publisher { } finally { // Groups still draining keep being ranked until they finish. ranking?.close(); + datagrams?.close(); + await datagramTask; track.close(); // Idempotent, and the backstop for a throw between the fork and the fill starting: // #runFill closes it itself on every path it reaches. @@ -1067,6 +1163,74 @@ export class Publisher { return undefined; } + /** + * Forward one datagram track as OBJECT_DATAGRAM beside a stream subscription. + * + * Best-effort: a missing track, an oversized payload, or a datagram the draft cannot + * encode is dropped. The datagram track ending does not end the stream subscription. + */ + async #sendDatagrams( + datagrams: datagram.Consumer, + options: { alias: bigint; timescale?: Timescale; priority: Signal }, + ): Promise { + let info: datagram.Info; + try { + info = await datagrams.info(); + } catch (err) { + console.debug(`serving the stream track without datagrams: ${reason(err)}`); + return; + } + const opened = this.#datagramWriter; + if (!opened) return; + const publisher = info.priority; + const timescale = options.timescale; + const rank = () => sendOrder({ priority: options.priority.peek(), publisher, position: 2 ** 36 - 1 }); + const sender = opened.open(rank()); + const dispose = options.priority.subscribe(() => sender.setPriority(rank())); + const version = this.#session.version; + const max = DatagramStream.maxDatagramSize(this.#quic); + const left = hooks.datagramClosed(datagrams); + try { + for (;;) { + const datagram = await datagrams.recv(); + if (!datagram) return; + if (timescale !== undefined && datagram.timestamp === undefined) { + console.debug(`dropping a datagram without a timestamp: sequence=${datagram.sequence}`); + continue; + } + let body: Uint8Array; + try { + body = await encodeDatagram( + datagram, + { + trackAlias: options.alias, + publisherPriority: toWire(publisher), + timescale, + }, + version, + ); + } catch (err) { + console.debug(`dropping a datagram this draft cannot encode: ${reason(err)}`); + continue; + } + if (body.byteLength > max) { + console.debug( + `dropping datagram larger than the transport limit: size=${body.byteLength} max=${max}`, + ); + continue; + } + // A paused subscription must release its consumer even when transport writes stall. + await race([sender.writer.ready, left]); + if (left.peek() !== undefined) return; + await race([sender.writer.write(body), left]); + if (left.peek() !== undefined) return; + } + } finally { + dispose(); + await race([sender.close(), left]); + } + } + /** * Runs a group and sends its frames using ObjectStream (Subgroup delivery mode). */ @@ -2545,11 +2709,11 @@ export class Publisher { } /** What a request that failed to resolve is refused with. */ - #failed(err: unknown, wait = false): Refusal { + #failed(err: unknown, wait = false, resolved = false): Refusal { const e = error(err); const condition = e instanceof StreamError && (e.code === StreamCode.NotFound || e.code === StreamCode.Unroutable) - ? wait + ? wait && (!resolved || e.code === StreamCode.Unroutable) ? "timeout" : "does_not_exist" : "internal"; diff --git a/js/net/src/ietf/subscriber.ts b/js/net/src/ietf/subscriber.ts index 7dd400a79e4..9ac6792beaa 100644 --- a/js/net/src/ietf/subscriber.ts +++ b/js/net/src/ietf/subscriber.ts @@ -1,11 +1,12 @@ -import { Milli } from "@moq/clock"; -import { type Dispose, type GetPromise, type Getter, race, Signal } from "@moq/signals"; +import { Milli, yieldNow } from "@moq/clock"; +import { type Dispose, type GetPromise, type Getter, Once, race, Signal } from "@moq/signals"; import * as announce from "../announced.ts"; import type { Grant } from "../auth.ts"; import type { AuthSession } from "../auth_session.ts"; import * as broadcast from "../broadcast.ts"; import type { Drain } from "../connection/goaway.ts"; import { BroadcastCache } from "../consume.ts"; +import type * as datagram from "../datagram.ts"; import * as DatagramStream from "../datagram_stream.ts"; import { closeError, @@ -27,9 +28,9 @@ import { finalSequence, hiddenBelow, hooks, maxDelayOf, scopeCaptures, scopeHead import type { Lister } from "../listing.ts"; import * as Path from "../path.ts"; import { receiveFrame, skipPayload } from "../payload.ts"; -import type { Cursor, Reader, Stream } from "../stream.ts"; +import { type Cursor, Reader, type Stream } from "../stream.ts"; import { Tail } from "../tail.ts"; -import type { Timescale } from "../time.ts"; +import { Timescale, type Timestamp } from "../time.ts"; import type * as track from "../track.ts"; import { untilAborted } from "../util/abort.ts"; import { closeUnwanted } from "../util/announce.ts"; @@ -37,6 +38,7 @@ import { Deadline } from "../util/deadline.ts"; import { abandoned as demandAbandoned, isAbandoned, unrequested, unwanted } from "../util/demand.ts"; import { LIMITS, type Limits, SlotMap, Slots } from "../util/limits.ts"; import { min } from "../util/sequence.ts"; +import { Budget } from "../util/yield.ts"; import { bounded, wireSpan } from "../window.ts"; import { wireOf } from "../wire.ts"; import { DuplicateTrackAlias, RetiredTrackAlias, TrackAliases } from "./aliases.ts"; @@ -48,7 +50,15 @@ import { isDraft20 } from "./filter.ts"; import { Join } from "./join.ts"; import * as Message from "./message.ts"; import * as Namespace from "./namespace.ts"; -import { decodeFetchObject, EndOfTrackProperties, Frame, type Group as GroupMessage, ObjectIdGap } from "./object.ts"; +import { + decodeFetchObject, + decodeObjectTime, + EndOfTrackProperties, + Frame, + type Group as GroupMessage, + ObjectIdGap, + statusDefined, +} from "./object.ts"; import { Parameters } from "./parameters.ts"; import { DEFAULT_WIRE, fromWire, toWire } from "./priority.ts"; import { type Publish, PublishDone, PublishDoneStatus, publishDoneClean } from "./publish.ts"; @@ -77,6 +87,7 @@ class JoinUnsupported extends Error { // A live subscription, as the track alias its data streams name resolves to. type Subscription = { + datagrams?: { producer: datagram.Producer; dispose: Dispose }; // The write side incoming group streams are routed into. track: track.Producer; // The group streams received, so the subscription can wait for the ones PUBLISH_DONE @@ -93,6 +104,9 @@ type Subscription = { fetchIds: bigint[]; }; +/** Both model kinds share this alias; group and datagram receive paths route to their own producer. */ +type Binding = Subscription; + // Out-parameter for #openSubscribe: lets the caller observe partial progress // (stream opened, trackAlias registered) so it can clean up a failed or abandoned setup // even before the setup promise settles. @@ -101,6 +115,8 @@ type SubscribeSetupState = { // The accepted subscription, which a registered alias names. subscription?: Subscription; registeredAlias?: bigint; + /** The alias binding {@link registeredAlias} names, so cleanup retires that same object. */ + binding?: Binding; requests?: Requests; /** * Aborts once the caller has given up. An open still waiting leaves the line, and setup @@ -142,6 +158,11 @@ function sees(filter: Filter, path: Path.Valid): boolean { * @internal */ export class Subscriber { + #streamSources = new WeakMap(); + #datagramRequests = new WeakMap< + broadcast.Consumer, + Map }> + >(); #session: Session; // The transport, so a request cut off by the session's close ends with the session's @@ -154,7 +175,8 @@ export class Subscriber { #cluster?: Cluster.Hops; // Publisher-chosen aliases used by incoming group streams. - #aliases = new TrackAliases(); + #aliases = new TrackAliases(); + #receiveYield?: Promise; // Units for each track's object Timestamps, from the TIMESCALE Track Property in // SUBSCRIBE_OK. A track missing from this map declared no timeline, so its frames @@ -693,6 +715,7 @@ export class Subscriber { for (;;) { const request = await wireOf(consumer).requested(); if (!request) break; + this.#streamSources.set(request, consumer); void this.#runSubscribe(path, request, instance.validity?.signal); } })(); @@ -700,7 +723,7 @@ export class Subscriber { for (;;) { const request = await wireOf(consumer).requestedDatagrams(); if (!request) return; - request.reject(new NotFound("datagram tracks over moq-transport")); + void this.#runDatagram(consumer, request, instance.validity?.signal); } })(); return this.#consumes.insert(path, instance, consumer); @@ -804,6 +827,7 @@ export class Subscriber { let stream: RequestStream; let subscription: Subscription; + let binding: Binding; let trackAlias: bigint; // The SUBSCRIBER_PRIORITY the request went out with. let priority: number; @@ -817,6 +841,7 @@ export class Subscriber { if (result !== null) { stream = result.stream; subscription = result.subscription; + binding = result.binding; trackAlias = result.alias; priority = result.priority; maxAge = result.maxAge; @@ -850,8 +875,8 @@ export class Subscriber { if ( state.registeredAlias !== undefined && - state.subscription && - this.#aliases.retire(state.registeredAlias, state.subscription) + state.binding && + this.#aliases.retire(state.registeredAlias, state.binding) ) { this.#timescales.delete(state.registeredAlias); } @@ -974,6 +999,14 @@ export class Subscriber { ); } finally { joining.abort(); + if (subscription.datagrams) { + const shared = subscription.datagrams; + shared.dispose(); + const closed = hooks.trackReleased(producer).peek(); + if (closed instanceof Error) shared.producer.abort(closed); + else if (hooks.trackFinished(producer)) shared.producer.finish(); + else shared.producer.abort(new StreamError(StreamCode.Cancel)); + } if (known) hooks.cacheTrack(producer); // A track end already settled the head; anything else leaves it for a later join. subscription.join?.cancel(new Error("subscription ended")); @@ -981,7 +1014,7 @@ export class Subscriber { disposeGrant?.(); // Only the owner tears down the alias metadata: a later subscription may have // reclaimed the alias and installed its own timescale. - if (this.#aliases.retire(trackAlias, subscription)) this.#timescales.delete(trackAlias); + if (this.#aliases.retire(trackAlias, binding)) this.#timescales.delete(trackAlias); } } @@ -1341,6 +1374,10 @@ export class Subscriber { if (done.statusCode !== PublishDoneStatus.TRACK_ENDED) { throw new Error(`publish done: subscription ended reason=${done.reasonPhrase}`); } + if (subscription.datagrams) { + await this.#datagramsRouted(); + subscription.datagrams.producer.finish(); + } // The track ended and every stream is accounted for, so no tail is coming: a head nothing // claimed is its whole group. A draft-18 joining FETCH is not in the Stream Count; its // head held the settle. @@ -1521,6 +1558,7 @@ export class Subscriber { ): Promise<{ stream: RequestStream; subscription: Subscription; + binding: Binding; alias: bigint; priority: number; maxAge?: number; @@ -1604,6 +1642,8 @@ export class Subscriber { if (!known) state.requests = producer.requests(); const tail = new Tail(); const subscription: Subscription = { track: producer, tail, timescale: ok.properties.timescale, fetchIds: [] }; + const binding = subscription; + state.binding = binding; // A live join starts at Largest's group: a filterless one from its first object, and Next // Object after Largest, so the joining FETCH or the fill brings that group's head. With no // Largest the track is empty and every group whole. A draft-22 fill is one of the @@ -1631,11 +1671,17 @@ export class Subscriber { start: live ? (ok.largest?.groupId ?? 0n) : (bounds.start ?? 0n), requested: bounds.start, }); + const source = this.#streamSources.get(request); + if (source) + subscription.datagrams = this.#bindDatagrams(source, producer, { + priority: fromWire(ok.properties.priority ?? DEFAULT_WIRE), + timescale: ok.properties.timescale, + }); staged?.commit(); state.fill?.(subscription); try { - this.#aliases.set(ok.trackAlias, subscription, { broadcast, name: request.name }); + this.#aliases.set(ok.trackAlias, binding, { broadcast, name: request.name }); const timescale = ok.properties.timescale; if (timescale !== undefined) { this.#timescales.set(ok.trackAlias, timescale); @@ -1651,6 +1697,7 @@ export class Subscriber { return { stream: state.stream, subscription, + binding, alias: ok.trackAlias, priority: msg.subscriberPriority, maxAge: msg.maxAge, @@ -1820,10 +1867,10 @@ export class Subscriber { throw new Error("subgroups are not supported"); } - let subscription: Subscription; + let binding: Binding; try { // The control message establishing this alias can arrive after the data stream. - subscription = await this.#aliases.get(group.trackAlias); + binding = await this.#aliases.get(group.trackAlias); } catch (err: unknown) { if (err instanceof ProtocolViolation) this.#fail(err.code, err.message); const e = await sessionCause(this.#quic, err); @@ -1837,6 +1884,8 @@ export class Subscriber { return; } + const subscription = binding; + const { track, tail } = subscription; // Every data stream counts toward PUBLISH_DONE's Stream Count, even one dropped below. const read = tail.open(group.groupId); @@ -2167,8 +2216,11 @@ export class Subscriber { const reader = DatagramStream.datagramReader(this.#quic); if (!reader) return; + const budget = new Budget(); try { for (;;) { + const yielded = budget.poll(); + if (yielded !== undefined) await this.#yieldDatagrams(yielded); // The stream errors once the session closes, which ends this loop like any other. const next = await reader.read().catch(() => undefined); if (!next || next.done) return; @@ -2179,10 +2231,161 @@ export class Subscriber { } } - // Stream subscriptions never admit datagrams; the separate IETF datagram-track adapter - // owns mapping DATAGRAMS demand onto SUBSCRIBE/OBJECT_DATAGRAM. + async #yieldDatagrams(yielded: Promise): Promise { + this.#receiveYield = yielded; + await yielded; + this.#receiveYield = undefined; + } + + // A clean end waits for datagrams already received with it. A budget yield or an + // unanswered alias can interrupt that drain; each next host task lets it continue. + async #datagramsRouted(): Promise { + await yieldNow(); + while (this.#receiveYield !== undefined) { + await this.#receiveYield; + await yieldNow(); + } + } + + /** + * Route one OBJECT_DATAGRAM. + * + * An undefined Object Status is a protocol violation on every subscription kind. + * A defined status is not a payload and does not end the track. Only object ID 0 + * reaches the datagram model; the stream model never receives datagrams. + */ async #recvDatagram(data: Uint8Array): Promise { - await ObjectDatagram.decode(data, this.#session.version); + const datagram = await ObjectDatagram.decode(data, this.#session.version); + if ("status" in datagram.body && !statusDefined(datagram.body.status)) { + throw new ProtocolViolation(`undefined object status ${datagram.body.status}`); + } + + let alias = this.#aliases.peek(datagram.trackAlias); + if (!alias) { + // The answer can arrive in the same packet, behind its datagram. Keep only + // this receive slot while the control stream decodes, then drop an unknown alias. + await this.#yieldDatagrams(yieldNow()); + alias = this.#aliases.peek(datagram.trackAlias); + } + const binding = alias?.datagrams && { + producer: alias.datagrams.producer, + timescale: alias.timescale, + }; + if (!binding) { + console.debug(`dropping a datagram for an unknown alias: alias=${datagram.trackAlias}`); + return; + } + + if ("status" in datagram.body) { + console.debug(`dropping a datagram status: alias=${datagram.trackAlias} group=${datagram.groupId}`); + return; + } + const objectId = datagram.objectId ?? 0n; + if (objectId !== 0n) { + console.debug( + `dropping a datagram with a nonzero object id: alias=${datagram.trackAlias} group=${datagram.groupId} object=${objectId}`, + ); + return; + } + + let timestamp: Timestamp | undefined; + if (datagram.properties) { + try { + const decoded = await new Reader(undefined, datagram.properties, this.#session.version).decode( + (cursor) => decodeObjectTime(cursor, binding.timescale ?? Timescale.SECOND), + ); + if (binding.timescale !== undefined) timestamp = decoded; + } catch (err) { + console.debug(`dropping a datagram with unreadable object properties: ${reason(err)}`); + return; + } + } + if (binding.timescale !== undefined && timestamp === undefined) { + console.debug( + `dropping a datagram without a timestamp on a timed track: alias=${datagram.trackAlias} group=${datagram.groupId}`, + ); + return; + } + try { + binding.producer.insert({ + sequence: datagram.groupId, + timestamp, + payload: datagram.body.payload, + }); + } catch (err) { + console.debug(`dropping a datagram the track refused: ${reason(err)}`); + } + } + + /** Bind both kinds to the same upstream response, without adding stream demand for passive forwarding. */ + #bindDatagrams(source: broadcast.Consumer, track: track.Producer, info: Partial) { + const pending = this.#datagramRequests.get(source); + const entry = pending?.get(track.name); + let producer: datagram.Producer; + try { + producer = entry ? entry.request.accept(info) : wireOf(source).createDatagrams(track.name, info); + } catch (err) { + entry?.request.reject(error(err)); + console.debug(`serving the stream track without datagrams: ${reason(err)}`); + return; + } + pending?.delete(track.name); + entry?.accepted.set(true); + let held: track.Subscriber | undefined; + const sync = () => { + const priority = producer.priority.peek(); + if (priority === undefined) { + held?.close(); + held = undefined; + } else if (held) held.update({ priority }); + else held = hooks.parked(track.subscribe({ priority })); + }; + const dispose = producer.priority.subscribe(sync); + sync(); + return { + producer, + dispose: () => { + dispose(); + held?.close(); + }, + }; + } + + /** Ask through the stream model's shared lookup, so either kind opens the same SUBSCRIBE. */ + async #runDatagram(source: broadcast.Consumer, request: datagram.Request, valid?: AbortSignal): Promise { + if (!this.#quic || DatagramStream.maxDatagramSize(this.#quic) === 0) { + request.reject(new Error("transport does not support datagrams")); + return; + } + let pending = this.#datagramRequests.get(source); + if (!pending) { + pending = new Map(); + this.#datagramRequests.set(source, pending); + } + const entry = { request, accepted: new Once() }; + pending.set(request.name, entry); + const subscriber = wireOf(source).subscribe(request.name, { priority: request.priority.peek() ?? 0 }); + const dispose = request.priority.subscribe((priority) => { + if (priority !== undefined) subscriber.update({ priority }); + }); + try { + await untilAborted( + race([ + // Cached stream metadata alone does not accept a returning datagram request. + Promise.all([subscriber.info(), entry.accepted]), + Promise.resolve(request.closed).then((): never => { + throw new StreamError(StreamCode.Cancel); + }), + ]), + valid, + ); + } catch (err) { + if (pending.get(request.name) === entry) request.reject(error(err)); + } finally { + dispose(); + subscriber.close(); + if (pending.get(request.name) === entry) pending.delete(request.name); + } } } diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index 1f068aa70e5..5eaed2ae609 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -455,6 +455,8 @@ export const hooks: { closeUnusedDatagrams: (producer: datagram.Producer) => boolean; /** Follow this consumer's subscriber priority for transport scheduling. */ datagramPriority: (consumer: datagram.Consumer) => Getter; + /** Forward data carried by an existing stream without contributing another subscription's demand. */ + observeDatagrams: (producer: datagram.Producer) => datagram.Consumer; /** Settles once this consumer is closed, the end of whatever request it serves. */ datagramClosed: (consumer: datagram.Consumer) => GetPromise; /** @@ -770,6 +772,9 @@ export const hooks: { closeUnusedDatagrams: () => { throw new Error("datagram module not loaded"); }, + observeDatagrams: () => { + throw new Error("datagram module not loaded"); + }, datagramPriority: () => { throw new Error("datagram module not loaded"); }, diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 4e75c4430c3..b78da9de6fc 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -1010,6 +1010,7 @@ export class Publisher { endFrame: msg.endFrame, }; let legacy: datagram.Consumer | undefined; + const legacyPriority = new Signal(msg.priority); let controls: SubscriptionControls | undefined; // Lite-05 and later name every sequence below the end that this subscription never // got with SUBSCRIBE_DROP, so the subscriber settles without waiting out its grace @@ -1087,8 +1088,8 @@ export class Publisher { if (this.#datagramWriter && !hasDatagramTracks(this.version)) { legacy = wireOf(front).liveDatagrams(msg.track); if (legacy) { - legacy.setPriority(msg.priority); - datagrams = this.#runDatagrams(legacy, { sub: msg.id, stamp }) + if (wireOf(front).localDatagrams) legacy.setPriority(msg.priority); + datagrams = this.#runDatagrams(legacy, { sub: msg.id, stamp, priority: legacyPriority }) .then(() => {}) .catch((err) => console.debug(`datagram send stopped: ${reason(err)}`)); } @@ -1105,7 +1106,8 @@ export class Publisher { version: this.version, live: new LiveRanges(this.version, asked), apply: (update, live) => { - legacy?.setPriority(update.priority); + legacyPriority.set(update.priority); + if (wireOf(front).localDatagrams) legacy?.setPriority(update.priority); if (hasRanges(this.version)) { const resolved = hooks.servingResolved(track); const entries = live.map(({ id, entry }) => { @@ -2060,12 +2062,16 @@ export class Publisher { } // Shared by dedicated lite-07 requests and the old SUBSCRIBE carry-both mapping. - async #runDatagrams(track: datagram.Consumer, options: { sub: bigint; stamp?: Stamp }): Promise { + async #runDatagrams( + track: datagram.Consumer, + options: { sub: bigint; stamp?: Stamp; priority?: Getter }, + ): Promise { if (!this.#datagramWriter) throw new Error("transport does not support datagrams"); const info = await track.info(); - const rank = () => sendOrder({ priority: track.priority, publisher: info.priority }); + const priority = options.priority ?? hooks.datagramPriority(track); + const rank = () => sendOrder({ priority: priority.peek(), publisher: info.priority }); const sender = this.#datagramWriter.open(rank()); - const dispose = hooks.datagramPriority(track).subscribe(() => sender.setPriority(rank())); + const dispose = priority.subscribe(() => sender.setPriority(rank())); const left = hooks.datagramClosed(track); const { writer } = sender; const maxSize = Math.min(DatagramMessage.MAX_SIZE, DatagramStream.maxDatagramSize(this.#quic)); diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts index c39b99474b1..f20374076f5 100644 --- a/js/net/src/wire.ts +++ b/js/net/src/wire.ts @@ -20,6 +20,8 @@ import type * as track from "./track.ts"; /** The protocol-facing operations behind a broadcast handle. */ export interface Broadcast { + /** Whether datagram metadata comes from an application rather than a peer's kindless SUBSCRIBE. */ + localDatagrams: boolean; datagrams(name: string): datagram.Consumer; liveDatagrams(name: string): datagram.Consumer | undefined; createDatagrams(name: string, info: Partial): datagram.Producer; diff --git a/quest/m1/datagram-tracks/README.md b/quest/m1/datagram-tracks/README.md index ffde2a0114f..3b4e7c8f90b 100644 --- a/quest/m1/datagram-tracks/README.md +++ b/quest/m1/datagram-tracks/README.md @@ -1,4 +1,4 @@ -# Datagram tracks +# [S] Datagram tracks ## Goal @@ -54,10 +54,11 @@ Decided 2026-11-01 (maintainer, in `/quest-plan`): counter. The datagram types have no group, frame, range, fetch, or cache API, so the group terminology disappears for datagrams. - **moq-transport mixing.** ✅ Keep the requested kind. - - A datagram-track subscription maps to IETF SUBSCRIBE and keeps only - datagrams (object 0). Stream objects arriving on it are dropped and - logged. - - A stream-track subscription drops datagrams. + - One upstream IETF SUBSCRIBE serves both kinds of a name, as on lite-05/06. + Decided 2026-11-05 (maintainer), on + [moq-dev/moq#7959](https://github.com/moq-dev/moq/pull/7959). + - Stream consumers receive only stream objects; datagram consumers receive + only datagrams with object ID 0. A relay forwards both kinds independently. - Outbound, a datagram track sends only OBJECT_DATAGRAM with object ID 0. - **Archive.** ✅ moq-archive skips datagram tracks. A datagram track is live-only, and an archive's playback doesn't include it. @@ -118,10 +119,6 @@ not the maintainer, so open to revisiting: This README owns the work no child owns: the Rust-JS interop test of a datagram track on lite-07, direct and through a relay. -## Required - -- [moq-transport datagram tracks](/quest/m1/datagram-tracks/ietf.md) - Rust and JS map a datagram track to SUBSCRIBE and OBJECT_DATAGRAM with object ID 0, keeping the requested kind - ## Related - [Subscribe ranges](/quest/m2/subscribe-ranges/README.md) - ranges no longer cover datagrams diff --git a/quest/m1/datagram-tracks/ietf.md b/quest/m1/datagram-tracks/ietf.md deleted file mode 100644 index 1b797ce6182..00000000000 --- a/quest/m1/datagram-tracks/ietf.md +++ /dev/null @@ -1,53 +0,0 @@ -# [S] moq-transport datagram tracks - -## Goal - -Over moq-transport, Rust and JS map a datagram track to IETF SUBSCRIBE and -OBJECT_DATAGRAM, with its sequence as the group ID and object ID 0, and -keep the requested kind when a peer mixes datagrams and streams on one track. - -## Plan - -Decided 2026-11-01 (maintainer), in the -[epic](/quest/m1/datagram-tracks/README.md): - -- A datagram-track subscription keeps only datagrams with object ID 0. - Stream objects arriving on it are dropped and logged once per - subscription. -- A stream-track subscription drops datagrams. -- Outbound, a datagram track sends only OBJECT_DATAGRAM with object ID 0. - A stream track never sends OBJECT_DATAGRAM. -- A datagram with a nonzero object ID on a datagram-track subscription is - dropped and logged. - -Touches `rs/moq-net/src/ietf/datagram.rs` and the JS IETF adapter. Test -against moxygen in the interop lanes where it sends datagrams. - -Rust datagram tracks ([moq-dev/moq#7752](https://github.com/moq-dev/moq/pull/7752)) -left moq-transport without datagrams in Rust until this lands: the IETF -subscriber drops every OBJECT_DATAGRAM, `datagrams()` on an IETF source is -NOT_FOUND, and a stream track never sends one. Restore what it removed: - -- the Rust-JS `datagram` tail lanes in `test/interop/interop.sh`, the - `datagram` mode of `test/interop/clients/js-native/tail.ts`, and their - README bullet; -- moq-e2ee's `datagrams_over_ietf` transport test; -- the moq-transport rows of moq-tokio's `datagram_delivery` bench and - moq-net's `session_delivery_datagram`; -- the OBJECT_DATAGRAM sentence in `doc/concept/standard.md`. - -An OBJECT_DATAGRAM whose Object Status is undefined is a protocol violation -in JS too, as it is in Rust, on every subscription kind. Planned 2026-11-04 -with the maintainer, from -[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846). Rust's -`recv_datagram` (`rs/moq-net/src/ietf/subscriber.rs`, about line 4839) -closes the session with `ProtocolViolation` when `status_defined` -(`rs/moq-net/src/ietf/object.rs`, about line 21: Normal, END_OF_GROUP and -END_OF_TRACK) rejects the status, even while it drops the datagram. JS's -`#recvDatagram` (`js/net/src/ietf/subscriber.ts`) decodes and drops it -silently, on main and after #7846. Check the shared status set once for -both datagram paths, so a datagram-track subscription keeps the check -when it starts accepting datagrams, and add a JS test that an undefined -status closes the session. - -Public API: none beyond the model. Wire: no format change. diff --git a/quest/m2/recut-held.md b/quest/m2/recut-held.md index 59b0ee3575d..d2ffa84ac4c 100644 --- a/quest/m2/recut-held.md +++ b/quest/m2/recut-held.md @@ -30,7 +30,6 @@ padding probes in [lite07-finalize](/quest/m2/lite07-finalize.md). ## Required -- [IETF datagram tracks](/quest/m1/datagram-tracks/ietf.md) - claim held datagram-tracks - [rind CPU error](/quest/m1/rind/cpu-error.md) - claim held rind - [Connect policy](/quest/m1/session-policy/connect.md) - claim held session-policy - [Release on leave](/quest/m1/stream-credit/release-on-leave.md) - claim held stream-credit diff --git a/quest/m5/net-wrap/e2ee.md b/quest/m5/net-wrap/e2ee.md index 642c380f1d4..db276b01bdb 100644 --- a/quest/m5/net-wrap/e2ee.md +++ b/quest/m5/net-wrap/e2ee.md @@ -63,10 +63,9 @@ the reverse (decided 2026-10-20, maintainer): Subscribe ID, Sequence, and Timestamp, since datagram tracks dropped the relayed `ietf` fields and the Inflated Size from it, so `MAX_DATAGRAM_HEADER` (28, `rs/moq-crypto/src/e2ee/limits.rs` and - `js/crypto/src/e2ee/limits.ts`) covers every version. Re-check it if - moq-transport datagram tracks - ([moq-transport datagram tracks](/quest/m1/datagram-tracks/ietf.md)) add - header fields. + `js/crypto/src/e2ee/limits.ts`) covers moq-lite. [moq-dev/moq#7959](https://github.com/moq-dev/moq/pull/7959) + restores moq-transport datagrams using existing OBJECT_DATAGRAM framing, + without adding header fields. Re-check the allowance if that framing changes. - Ranges: every moq-net range starts at a group's first frame ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)), so the wrapper reads each physical group whole and inflates from frame 0. diff --git a/rs/moq-crypto/tests/transport.rs b/rs/moq-crypto/tests/transport.rs index 489d8fd49d6..8735d7efe5d 100644 --- a/rs/moq-crypto/tests/transport.rs +++ b/rs/moq-crypto/tests/transport.rs @@ -1,4 +1,4 @@ -//! Grouped tracks on moq-lite and MoQ Transport, and datagram tracks on moq-lite. +//! Grouped tracks and datagrams on moq-lite and MoQ Transport. mod support; @@ -156,3 +156,8 @@ async fn datagrams_over_lite_05() { async fn max_datagram_over_lite_07() { datagram_roundtrip("moq-lite-07-wip", &[7; moq_crypto::e2ee::MAX_DATAGRAM_PLAINTEXT]).await; } + +#[tokio::test] +async fn datagrams_over_ietf() { + datagram_roundtrip("moq-transport-22", b"opus").await; +} diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 13e37ae6884..15cbf34f3cc 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -21,6 +21,14 @@ live, so third-party readers and relay caches see no change. `origin::Config::route_key` takes an `origin::RouteKey`, which every lite-07 session on the origin proves to its peer in SETUP. Origins sharing a key fail over as one to a peer that scopes them; `None`, the default, proves nothing. A proof binds to the session through the new `poll::Session::export_keying_material`, which defaults to `None`: a transport over TLS returns its exporter, and a wrapper around another transport forwards it. +### moq-transport datagram tracks + +Datagram tracks now work over moq-transport using SUBSCRIBE and OBJECT\_DATAGRAM. +The sequence is the group ID and the object ID is zero. One upstream subscription +serves both kinds of a name, including through relays. Stream and datagram +consumers keep only their requested kind; nonzero object IDs are dropped by +datagram consumers. No application API or wire encoding changes are required. + ### Group batch operations and `poll_next_frame` are crate-private ([#7874](https://github.com/moq-dev/moq/pull/7874)) Read a frame in chunks with the async `group::Consumer::next_frame`, or read whole frames with `poll_read_frame` from a poll loop. diff --git a/rs/moq-net/benches/session/mod.rs b/rs/moq-net/benches/session/mod.rs index 67cda4335e3..b4e37e67dbe 100644 --- a/rs/moq-net/benches/session/mod.rs +++ b/rs/moq-net/benches/session/mod.rs @@ -32,8 +32,8 @@ //! keeps, so a per-group cost that grows with them shows as a slope over `follow`. //! `session_delivery_datagram` writes to a datagram track per broadcast instead of groups: //! one datagram per track per round across publisher and viewer axes, then larger payloads -//! and cadences, on the lite versions (moq-transport carries no datagram tracks). Throughput -//! counts delivered datagrams, each checked for its sequence and payload, so a lost one fails. +//! and cadences, on moq-lite and moq-transport-22. Throughput counts delivered datagrams, +//! each checked for its sequence and payload, so a lost one fails. //! Delivery uses expiry-only pools: history expires between rounds outside timing, //! so byte pressure cannot shed unread frames during a batch. //! `SESSION_EXPIRY_ONLY=1` advances the cache clock by a second instead of the 30s @@ -209,9 +209,10 @@ impl Shape { self.viewers * self.watch * self.burst } - /// Whether `version` carries this shape: moq-transport has no datagram tracks. + /// Whether `version` carries this shape. Datagram tracks run on moq-lite and on + /// moq-transport-22, which maps one to OBJECT_DATAGRAM at object ID 0. fn runs_on(&self, version: &str) -> bool { - !self.datagrams || version.parse::().unwrap().is_lite() + !self.datagrams || version.parse::().unwrap().is_lite() || version == "moq-transport-22" } /// The broadcasts viewer `viewer` watches: a contiguous window starting at @@ -1039,9 +1040,10 @@ impl Room { /// Line every viewer up on the same next sequence before the strict rounds. /// /// A consumer is live once its track's info arrives, but on lite-05/06 the SUBSCRIBE - /// that carries the datagrams follows that answer, so a datagram may still miss it. Write - /// until every viewer has read one, then a last one each viewer reads through: the memory - /// transport keeps a subscription's datagrams in order, so nothing earlier arrives after. + /// that carries the datagrams follows that answer, so a datagram may still miss it. + /// moq-transport's SUBSCRIBE is that answer. Write until every viewer has read one, then + /// a last one each viewer reads through: the memory transport keeps a subscription's + /// datagrams in order, so nothing earlier arrives after. async fn sync_datagrams(&mut self) { let waiter = kio::Waiter::noop(); let mut reached = vec![false; self.shape.viewers * self.shape.watch]; diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index d5394ffbe45..1a727e60b44 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -17,7 +17,7 @@ use crate::transport::poll::{GroupOf, SendOptions}; use crate::{ DecodeError, Error, Timescale, Timestamp, - coding::{Encoder, Stream, Writer}, + coding::{Encode as _, Encoder, Stream, Writer}, ietf::{self, Control, EndLocation, FetchHeader, FetchType, Filter, Location, RequestId}, priority::{Priority, PriorityHandle, PriorityQueue}, track::{Order, Subscription}, @@ -39,6 +39,25 @@ fn serving_subscription(subscriber_priority: u8) -> Subscription { } } +/// Resolve the name's datagram half, holding demand only on a local publication. +fn serving_datagrams( + request: &SubscribeRequest, + broadcast: &crate::broadcast::Consumer, +) -> Option { + if request.publisher.session.max_datagram_size() == 0 { + return None; + } + let mut datagrams = if broadcast.has_local_datagrams() { + broadcast.datagrams(&request.msg.track_name).ok() + } else { + broadcast.datagrams_live(&request.msg.track_name) + }?; + if broadcast.has_local_datagrams() { + datagrams.set_priority(super::priority::from_wire(request.msg.subscriber_priority)); + } + Some(datagrams) +} + /// The Track Properties an answer describing the track carries: SUBSCRIBE_OK, /// TRACK_STATUS_OK, and FETCH_OK. Empty when the requester opted out with /// INCLUDE_PROPERTIES=0, which keeps the block present but empty. @@ -1369,6 +1388,9 @@ impl SubscribeRequest { /// An admitted SUBSCRIBE's serve, which ends however its authority does: dropping it /// releases everything it holds upstream. struct Serving { + resuming: bool, + datagrams: Option, + stream_error: Option, /// The filter the SUBSCRIBE asked for, which every saved join boundary follows. /// /// Draft-22 replaced joining FETCH with subscription fills. Older drafts save the same @@ -1390,7 +1412,7 @@ enum ServingStep { Resolve(Resolve), /// Subscribing to the track. Subscribe { - track: track::Consumer, + track: Result, subscribe: SubscribeTrack, }, /// Waiting on the peer's SETUP for whether SUBSCRIBE_OK's properties may compress. @@ -1411,11 +1433,6 @@ enum ServingStep { Pausing, /// Paused, answering updates until one resumes it. Paused(WaitResume), - /// Subscribing again for the update resuming it. - Resume { - track: track::Consumer, - subscribe: SubscribeTrack, - }, /// Flushing the error that fails the resuming update. ResumeFailed(Error), /// Flushing the REQUEST_OK answering the resuming update. @@ -1435,6 +1452,9 @@ impl Serving { fn new(publisher: &Publisher, msg: &ietf::Subscribe<'_>) -> Self { let fin_cancels = super::request_stream::fin_cancels(publisher.version); Self { + resuming: false, + datagrams: None, + stream_error: None, filter: msg.filter, broadcast: None, track: None, @@ -1468,10 +1488,8 @@ impl Serving { Err(err) => return Poll::Ready(Ok(refused(err, request.msg.rendezvous, false))), Ok(None) => return Poll::Ready(Ok(Subscribed::Withdrawn)), }; - let track = match broadcast.track(&request.msg.track_name) { - Ok(track) => track, - Err(err) => return Poll::Ready(Ok(refused(err, request.msg.rendezvous, true))), - }; + self.datagrams = serving_datagrams(request, &broadcast); + let track = broadcast.track(&request.msg.track_name); self.broadcast = Some(broadcast); self.step = ServingStep::Subscribe { track, @@ -1479,20 +1497,100 @@ impl Serving { }; } ServingStep::Subscribe { track, subscribe } => { - let subscribed = - ready!(subscribe.poll(waiter, &request.publisher, stream, track, &request.msg, &request.lapse)); - let (serve, edge) = match subscribed { - Some(Ok(subscribed)) => subscribed, - Some(Err(err)) => return Poll::Ready(Ok(refused(err, request.msg.rendezvous, true))), - None => return Poll::Ready(Ok(Subscribed::Withdrawn)), + let result = if self.stream_error.is_some() { + Poll::Pending + } else { + match track { + Ok(track) => { + subscribe.poll(waiter, &request.publisher, stream, track, &request.msg, &request.lapse) + } + Err(err) => Poll::Ready(Some(Err(err.clone()))), + } }; - let ServingStep::Subscribe { track: cache, .. } = - std::mem::replace(&mut self.step, ServingStep::Done) - else { - unreachable!("matched above"); + let mut selected = match result { + Poll::Ready(Some(Ok(found))) => Some(found), + Poll::Ready(Some(Err(err))) => { + self.stream_error = Some(err); + None + } + Poll::Ready(None) => { + return Poll::Ready(Ok(if self.resuming { + Subscribed::Served(Served::Track(Ok(())), request.opened()) + } else { + Subscribed::Withdrawn + })); + } + Poll::Pending => None, }; - self.track = Some(serve); - self.step = ServingStep::Properties { cache, edge }; + let broadcast = self.broadcast.as_ref().expect("resolved"); + let datagram_only = + broadcast.has_local_datagrams() && !broadcast.publishes_track(&request.msg.track_name); + if !datagram_only && self.stream_error.is_some() { + self.datagrams = None; + } + if selected.is_none() + && datagram_only && let Some(datagrams) = &self.datagrams + { + match datagrams.poll_info(waiter) { + Poll::Ready(Ok(info)) => { + let local = crate::broadcast::Producer::new(crate::broadcast::Info::default()); + let producer = local.create_track( + request.msg.track_name.as_ref(), + track::Info::default() + .with_timescale(info.timescale) + .with_priority(info.priority), + )?; + let cache = producer.consume(); + let mut serve = TrackServe::new( + request.publisher.session.clone(), + producer.subscribe(serving_subscription(request.msg.subscriber_priority)), + request.msg.request_id, + version, + ServeRange::default(), + info.timescale.filter(|_| request.msg.properties_wanted), + ); + serve.datagram_track = Some(producer); + *track = Ok(cache); + selected = Some((serve, LiveEdge::default())); + } + Poll::Ready(Err(_)) => self.datagrams = None, + Poll::Pending => {} + } + } + let Some((mut serve, edge)) = selected else { + if self.datagrams.is_none() + && let Some(err) = &self.stream_error + { + if self.resuming { + self.step = Self::resume_failed(request, err.clone())?; + continue; + } + return Poll::Ready(Ok(refused(err.clone(), request.msg.rendezvous, true))); + } + return Poll::Pending; + }; + let cache = track.as_ref().expect("stream or datagram kind resolved").clone(); + let datagrams = self + .datagrams + .take() + .or_else(|| broadcast.datagrams_live(&request.msg.track_name)); + if let Some(datagrams) = datagrams { + serve = serve.with_datagrams( + datagrams, + request.msg.properties_wanted, + broadcast.has_local_datagrams(), + ); + } + if self.resuming { + serve.opened = request.acked.clone().expect("acked before serving"); + Publisher::buffer_request_ok(&mut stream.writer, edge.largest)?; + request.save_join(self.filter, edge.largest, &cache, &serve.ranking); + self.track = Some(serve); + self.step = ServingStep::Resumed(cache); + } else { + self.track = Some(serve); + self.step = ServingStep::Properties { cache, edge }; + } } ServingStep::Properties { edge, .. } => { let flate = ready!(request.publisher.peer_setup.poll_get(waiter)).flate; @@ -1597,35 +1695,12 @@ impl Serving { return Poll::Ready(Ok(Subscribed::Served(served, request.opened()))); } let broadcast = self.broadcast.as_ref().expect("resolved before serving"); - self.step = match broadcast.track(&request.msg.track_name) { - Ok(track) => ServingStep::Resume { - track, - subscribe: SubscribeTrack::new(), - }, - Err(err) => Self::resume_failed(request, err)?, - }; - } - ServingStep::Resume { track, subscribe } => { - let subscribed = - ready!(subscribe.poll(waiter, &request.publisher, stream, track, &request.msg, &request.lapse)); - let ServingStep::Resume { track, .. } = std::mem::replace(&mut self.step, ServingStep::Done) else { - unreachable!("matched above"); - }; - self.step = match subscribed { - // Withdrawn mid-setup, it ends as it would have while paused. - None => { - return Poll::Ready(Ok(Subscribed::Served(Served::Track(Ok(())), request.opened()))); - } - // The resuming update is owed an answer, so it fails. - Some(Err(err)) => Self::resume_failed(request, err)?, - Some(Ok((mut serve, edge))) => { - serve.opened = request.acked.clone().expect("acked before serving"); - request.save_join(self.filter, edge.largest, &track, &serve.ranking); - let stream = request.stream.as_mut().expect("open until PUBLISH_DONE"); - Publisher::buffer_request_ok(&mut stream.writer, edge.largest)?; - self.track = Some(serve); - ServingStep::Resumed(track) - } + self.resuming = true; + self.stream_error = None; + self.datagrams = serving_datagrams(request, broadcast); + self.step = ServingStep::Subscribe { + track: broadcast.track(&request.msg.track_name), + subscribe: SubscribeTrack::new(), }; } ServingStep::ResumeFailed(_) => { @@ -6302,6 +6377,8 @@ impl NamespaceServe { /// the rest wait in the cache, so what a peer withholding credit makes this hold is bounded /// by the credit it grants, not by how long it withholds it. struct TrackServe { + datagrams: Option, + datagram_track: Option, session: Transport, track: track::Subscriber, request_id: RequestId, @@ -6343,6 +6420,15 @@ struct TrackServe { dynamic: bool, } +struct DatagramForward { + consumer: crate::datagram::Consumer, + ready: bool, + ended: bool, + timescale: Option, + publisher: i8, + demand: bool, +} + impl TrackServe { fn new( session: Transport, @@ -6373,6 +6459,8 @@ impl TrackServe { }; Self { + datagrams: None, + datagram_track: None, compression: false, session, track, @@ -6399,6 +6487,22 @@ impl TrackServe { } } + /// Also forward `datagrams` as OBJECT_DATAGRAM with object ID 0. + fn with_datagrams(mut self, mut datagrams: crate::datagram::Consumer, stamp: bool, demand: bool) -> Self { + if demand { + datagrams.set_priority(self.track.subscription().priority); + } + self.datagrams = Some(DatagramForward { + consumer: datagrams, + demand, + ready: false, + ended: false, + timescale: self.timescale.filter(|_| stamp), + publisher: 0, + }); + self + } + fn compression(mut self, negotiated: bool) -> Self { self.compression = negotiated && self.track.info().compress; self @@ -6506,6 +6610,11 @@ impl TrackServe { if let Ok(mut current) = self.priority.write() { *current = priority; } + if let Some(datagrams) = &mut self.datagrams + && datagrams.demand + { + datagrams.consumer.set_priority(priority); + } // One move for the whole subscription; its streams keep their place within it. if let Some(group) = &mut self.ranking.group { group.set_priority(crate::priority::group(priority, self.ranking.publisher)); @@ -6534,6 +6643,7 @@ impl TrackServe { self.tail = None; self.opening = None; self.children = kio::Tasks::new(); + self.datagrams = None; } /// Group machines held: the one waiting for stream credit, and those with a stream. @@ -6545,6 +6655,9 @@ impl TrackServe { /// Where the track ends, when the subscription ran to that end rather than stopping at /// its own range first. fn end(&self) -> Option { + if self.datagram_track.is_some() { + return None; + } let end = self.end?; if self.terminal.sent.load(Ordering::Relaxed) { return None; @@ -6619,6 +6732,17 @@ impl TrackServe { } fn poll(&mut self, waiter: &kio::Waiter) -> Poll> { + if self.datagram_track.is_some() { + self.poll_datagrams(waiter); + } + let result = self.poll_groups(waiter); + if self.datagram_track.is_none() { + self.poll_datagrams(waiter); + } + result + } + + fn poll_groups(&mut self, waiter: &kio::Waiter) -> Poll> { // Before the groups, so one finishing on this poll sees an end declared beside it. if self.terminal.end.get().is_none() && let Poll::Ready(Ok(end)) = self.track.poll_finished(waiter) @@ -6706,6 +6830,129 @@ impl TrackServe { let _ = self.children.poll(waiter); Poll::Pending } + /// Send every buffered datagram as an OBJECT_DATAGRAM with object ID 0. + /// + /// Best-effort: an oversized or unencodable datagram is dropped. The datagram track + /// ending or failing stops this and leaves the stream track alone. + fn poll_datagrams(&mut self, waiter: &kio::Waiter) { + loop { + if self.budget.poll_yield(waiter).is_pending() { + return; + } + let datagram = { + let Some(forward) = self.datagrams.as_mut() else { + return; + }; + if forward.ended { + return; + } + if !forward.ready { + match forward.consumer.poll_info(waiter) { + Poll::Pending => return, + Poll::Ready(Ok(info)) => { + // Object times use the stream subscription's advertised timescale. + forward.publisher = info.priority; + forward.ready = true; + } + Poll::Ready(Err(err)) => { + tracing::debug!(%err, "datagram track ended before it started"); + forward.ended = true; + return; + } + } + } + match forward.consumer.poll_recv(waiter) { + Poll::Pending => return, + Poll::Ready(Ok(None)) => { + forward.ended = true; + if let Some(track) = &mut self.datagram_track { + let _ = track.finish(); + } + return; + } + Poll::Ready(Err(err)) => { + tracing::debug!(%err, "stopping datagrams on this subscription"); + if let Some(track) = &self.datagram_track { + let _ = track.clone().abort(err); + } + forward.ended = true; + return; + } + Poll::Ready(Ok(Some(datagram))) => { + let timescale = forward.timescale; + let publisher = forward.publisher; + (datagram, timescale, publisher) + } + } + }; + let (datagram, timescale, publisher) = datagram; + emit_datagram( + &self.session, + self.version, + self.request_id.0, + SendOptions { + group: self.ranking.group.as_ref(), + priority: crate::priority::in_group(datagram.sequence, 0, self.ranking.order), + }, + timescale, + publisher, + &datagram, + ); + } + } +} + +/// Send one datagram track payload as OBJECT_DATAGRAM, object ID 0, best-effort. +fn emit_datagram( + session: &Transport, + version: Version, + alias: u64, + options: SendOptions<'_, GroupOf>, + timescale: Option, + publisher: i8, + datagram: &crate::datagram::Datagram, +) { + let mut properties = Vec::new(); + if let Some(timescale) = timescale { + let Some(timestamp) = datagram.timestamp else { + tracing::debug!(sequence = datagram.sequence, "dropping a datagram without a timestamp"); + return; + }; + if ietf::encode_object_time(&mut Encoder::new(&mut properties, version.into()), timestamp, timescale).is_err() { + tracing::debug!( + sequence = datagram.sequence, + "dropping a datagram whose timestamp does not encode" + ); + return; + } + } + let body = ietf::ObjectDatagram { + track_alias: alias, + group_id: datagram.sequence, + object_id: None, + publisher_priority: Some(super::priority::to_wire(publisher)), + end_of_group: true, + properties: (!properties.is_empty()).then_some(properties), + body: ietf::DatagramBody::Payload(datagram.payload.clone()), + }; + let Ok(body) = body.encode_bytes(version) else { + tracing::debug!( + sequence = datagram.sequence, + "dropping a datagram this draft cannot encode" + ); + return; + }; + let max = session.max_datagram_size(); + if body.len() > max { + tracing::debug!( + sequence = datagram.sequence, + size = body.len(), + max, + "dropping datagram larger than the transport limit" + ); + return; + } + let _ = session.send_datagram(body, options); } /// Where one request's streams are ranked: the session's queue, keyed by the request's id @@ -8795,7 +9042,7 @@ mod subscribe_cursor_test { #[cfg(test)] mod serve_tests { use super::*; - use crate::coding::{Decode, Decoder, Encode as _}; + use crate::coding::{Decode, Decoder}; use crate::ietf::GroupOrder; use crate::lite::test_transport::{Close, Log, ScriptedSession, Sent, SinkSession}; use crate::model::ProduceTest; @@ -9732,6 +9979,95 @@ mod serve_tests { } } + #[kio::test] + async fn datagrams_continue_after_a_stream_range_is_exhausted() { + let broadcast = crate::broadcast::Producer::new(Default::default()); + let track = broadcast.create_track("audio", track::Info::default()).unwrap(); + let subscriber = track.subscribe(Subscription::default().with_start(0).with_end(1)); + let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); + group.write_frame(None, &b"bounded"[..]).unwrap(); + group.finish().unwrap(); + let mut datagrams = broadcast.create_datagrams("audio", Default::default()).unwrap(); + let consumer = broadcast.consume().datagrams("audio").unwrap(); + let session = ScriptedSession::new(Vec::new()).with_datagrams(1200); + let mut serving = TrackServe::new( + crate::driver::Transport::new(session.clone()), + subscriber, + RequestId(0), + Version::Draft22, + ServeRange { + end: Some(EndLocation { group: 0, object: None }), + ..Default::default() + }, + None, + ) + .with_datagrams(consumer, false, true); + let _group = kio::wait(|waiter| serving.track.poll_recv_group(waiter)) + .await + .unwrap() + .expect("bounded group"); + assert!(matches!( + serving.track.poll_recv_group(&kio::Waiter::noop()), + Poll::Ready(Ok(None)) + )); + datagrams.append(None, &b"live"[..]).unwrap(); + assert!(serving.poll(&kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.datagrams.lock().unwrap().len(), 1); + } + + /// A datagram-only name resumes through the same kind selection as its first request. + #[kio::test] + async fn a_datagram_only_subscription_releases_demand_on_pause_and_resumes() { + for version in [Version::Draft18, Version::Draft22] { + for start_paused in [false, true] { + let h = serve_session( + version, + crate::track::Info::timed(), + ScriptedSession::new(Vec::new()).with_datagrams(1200), + ); + let mut datagrams = h + ._broadcast + .broadcast() + .create_datagrams("audio", crate::datagram::Info::default()) + .unwrap(); + let request = h.session.clone(); + let stream = Stream::open(&crate::driver::Transport::new(request.clone()), version) + .await + .unwrap(); + let mut msg = subscribe(Filter::NextObject, None); + msg.track_name = "audio".into(); + msg.forward = !start_paused; + let mut serve = std::pin::pin!(h.publisher.clone().run_subscribe_stream(stream, msg)); + serve_until(serve.as_mut(), "SUBSCRIBE_OK", || !h.log.trail().is_empty()).await; + if !start_paused { + datagrams.append(None, &b"before"[..]).unwrap(); + serve_until(serve.as_mut(), "the datagram", || { + h.log.datagrams.lock().unwrap().len() == 1 + }) + .await; + request.push(&forward_update(version, 2, false)); + } + serve_until(serve.as_mut(), "demand released", || datagrams.priority().is_none()).await; + let sent = h.log.datagrams.lock().unwrap().len(); + datagrams.append(None, &b"paused"[..]).unwrap(); + request.push(&forward_update(version, 4, true)); + serve_until(serve.as_mut(), "resumed demand", || datagrams.priority().is_some()).await; + datagrams.append(None, &b"after"[..]).unwrap(); + serve_until(serve.as_mut(), "the resumed datagram", || { + h.log.datagrams.lock().unwrap().len() == sent + 1 + }) + .await; + datagrams.finish(); + kio::clock::timeout(std::time::Duration::from_secs(10), serve) + .await + .expect("datagram end") + .unwrap(); + let wire = Wire::read(&h.log, version); + assert_eq!(wire.publish_done, Some((ietf::PublishDoneStatus::TrackEnded.code(), 0))); + } + } + } + /// A resume joins as a new SUBSCRIBE would, so a joining FETCH after it is served from where /// the resume joined rather than refused as the pause left it. #[kio::test] @@ -15480,6 +15816,10 @@ mod serve_tests { .await .unwrap_or_else(|_| panic!("{version}: outlived its withdrawal")) .unwrap(); + assert!( + Wire::read(&h.log, version).publish_done.is_some(), + "{version}: missing PUBLISH_DONE" + ); settle().await; assert!(h.track.subscription().is_none(), "{version}: held its demand"); } @@ -20805,7 +21145,6 @@ mod fill_range_tests { #[cfg(test)] mod compression_tests { use super::*; - use crate::coding::Encode as _; #[test] fn sealed_compression_refuses_incompatible_egress() { for version in [Version::Draft18, Version::Draft20, Version::Draft22] { diff --git a/rs/moq-net/src/ietf/session.rs b/rs/moq-net/src/ietf/session.rs index 7ab881730c1..f0256e50404 100644 --- a/rs/moq-net/src/ietf/session.rs +++ b/rs/moq-net/src/ietf/session.rs @@ -462,8 +462,8 @@ impl Launch { let datagrams = (session.max_datagram_size() != 0).then(|| Datagrams { session: session.clone(), subscriber: subscriber.clone(), - budget: kio::coop::Budget::new(32), - charged: false, + held: Vec::new(), + caught_up: true, }); // Keep the session's SUBSCRIBE_NAMESPACE requests open as discovery decides, each on // its own stream; with no subscribe half, ask nothing. @@ -518,6 +518,15 @@ impl Running { } // Whichever side closed it, read what already arrived. ready!(self.race.unis.poll_drain(waiter)); + let _ = self.race.tasks.poll(waiter); + if let Some(datagrams) = &mut self.race.datagrams { + let _ = datagrams.poll(waiter, false); + if !datagrams.caught_up { + return Poll::Pending; + } + } + self.race.subscriber.datagrams_routed(); + let _ = self.race.tasks.poll(waiter); let res = self.ending.take().expect("draining an ended session"); // Closed cleanly by dropping or draining: readers see what the peer's clean // close would have shown them. @@ -581,7 +590,7 @@ impl Race { if let Poll::Ready(err) = poll_failed(&mut self.dispatch, |dispatch| dispatch.poll(waiter).map(Err)) { return Poll::Ready(Err(err)); } - if let Poll::Ready(err) = poll_failed(&mut self.datagrams, |datagrams| datagrams.poll(waiter)) { + if let Poll::Ready(err) = poll_failed(&mut self.datagrams, |datagrams| datagrams.poll(waiter, true)) { return Poll::Ready(Err(err)); } if let Poll::Ready(err) = poll_failed(&mut self.goaway, |goaway| { @@ -608,6 +617,12 @@ impl Race { if let Poll::Ready(err) = poll_failed(&mut self.publish_namespaces, |publish| publish.poll(waiter)) { return Poll::Ready(Err(err)); } + if let Poll::Ready(err) = poll_failed(&mut self.datagrams, |datagrams| datagrams.poll(waiter, false)) { + return Poll::Ready(Err(err)); + } + if self.datagrams.as_ref().is_none_or(|datagrams| datagrams.caught_up) { + self.subscriber.datagrams_routed(); + } Poll::Pending } } @@ -1358,23 +1373,46 @@ fn died_before_header(err: &Error) -> bool { struct Datagrams { session: Transport, subscriber: Subscriber, - /// A peer keeping datagrams ready must not hold the session's other tasks. - budget: kio::coop::Budget, - /// This pass of the budget is spent, waiting for the next datagram. - charged: bool, + /// Unknown aliases held before the source machines run: at most one receive budget. + held: Vec, + /// Whether the receive pass read the queue empty rather than yielding to its budget. + caught_up: bool, } impl Datagrams { - fn poll(&mut self, waiter: &kio::Waiter) -> Poll> { + fn poll(&mut self, waiter: &kio::Waiter, hold: bool) -> Poll> { + if !hold { + for payload in std::mem::take(&mut self.held) { + self.route(payload, false)?; + } + } + let mut budget = kio::coop::Budget::new(32); loop { - if !self.charged { - ready!(self.budget.poll_yield(waiter)); - self.charged = true; + self.caught_up = false; + ready!(budget.poll_yield(waiter)); + self.caught_up = true; + let payload = match ready!(self.session.poll_recv_datagram(waiter)) { + Ok(payload) => payload, + Err(err) => { + for payload in std::mem::take(&mut self.held) { + self.route(payload, false)?; + } + return Poll::Ready(Err(Error::from_transport(err))); + } + }; + self.route(payload, hold)?; + } + } + + fn route(&mut self, payload: bytes::Bytes, hold: bool) -> Result<(), Error> { + if let Some(alias) = self.subscriber.recv_datagram(payload.clone())? { + if hold { + self.held.push(payload); + } else { + tracing::debug!(alias, "dropping a datagram for an unknown alias"); } - let payload = ready!(self.session.poll_recv_datagram(waiter)).map_err(Error::from_transport)?; - self.charged = false; - self.subscriber.recv_datagram(payload)?; } + Ok(()) } } diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 71a51e3b941..09063a00e51 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -23,7 +23,9 @@ use kio::Lock; mod namespaces; pub(super) use namespaces::Namespaces; +mod datagrams; mod tracks; +use datagrams::{Requested, Shared}; const TRACK_ALIAS_TIMEOUT: Duration = Duration::from_secs(1); @@ -190,6 +192,9 @@ fn retire_track_alias(aliases: &TrackAliases, alias: u64, request_id: RequestId) #[derive(Default)] struct State { + /// Clean ends keep their request streams until the driver drains datagrams received with them. + datagram_ends: Option>, + datagram_requests: HashMap, crate::datagram::Request)>, // Each active subscription subscribes: HashMap, @@ -221,10 +226,17 @@ impl State { /// Active receive tasks abort their own groups. Abort any head waiting for its /// tail here, along with the track. Ordinary unsubscribe removes its entry. fn abort(&mut self, err: &Error) { + let _ = self.datagram_ends.take(); + for (_, (_, request)) in self.datagram_requests.drain() { + request.reject(err.clone()); + } for (_, mut track) in self.subscribes.drain() { if let Some(request) = track.pending.take() { request.reject(err.clone()); } + if let Some(datagrams) = track.datagram { + datagrams.abort(err.clone()); + } if let Some(producer) = track.producer { let _ = producer.abort(err.clone()); } @@ -781,6 +793,7 @@ struct Idle { } struct TrackState { + datagram: Option, producer: Option, /// The origin request, until SUBSCRIBE_OK accepts it. Abort rejects this: the /// producer does not exist yet, and dropping the setup task would be `Dropped`. @@ -845,6 +858,7 @@ impl TrackState { fn pending(name: String, broadcast: Path, fill: kio::Producer) -> Self { Self { + datagram: None, producer: None, pending: None, name, @@ -3238,6 +3252,7 @@ impl kio::Task for RunRoute { /// until it closes: its namespace is retracted (`route` closes), the peer stops /// selecting its instance, nothing holds it any more, or the session dies. struct RunBroadcast { + datagrams: broadcast::DatagramRequests, this: Subscriber, path: Path, source: crate::model::broadcast::SourceGuard, @@ -3268,6 +3283,7 @@ impl RunBroadcast { this, path, source, + datagrams: broadcast.datagrams(), broadcast, route, subscribes: TaskSet::owned(), @@ -3285,6 +3301,21 @@ impl RunBroadcast { self.this .poll_listing(&self.path, &self.lister, &mut self.listening, waiter); loop { + while let Poll::Ready(Ok(request)) = self.datagrams.poll_requested(waiter) { + if self.this.session.max_datagram_size() == 0 { + request.reject(Error::Unsupported); + continue; + } + let key = (self.path.clone(), request.name().into()); + let consumer = self.source.consume(); + match consumer.track(request.name()) { + Ok(track) => { + self.subscribes + .push(Requested::new(self.this.clone(), key, track, request)); + } + Err(err) => request.reject(err), + } + } if let Poll::Ready(next) = self.broadcast.poll_requested_track(waiter) { return Poll::Ready(Some(next)); } @@ -3323,13 +3354,17 @@ impl kio::Task for RunBroadcast { } }; match next { - Some(Ok(request)) => self.subscribes.push(RunSubscribe::new( - self.this.clone(), - self.path.clone(), - self.broadcast.clone(), - self.route.clone(), - request, - )), + Some(Ok(request)) => { + let mut run = RunSubscribe::new( + self.this.clone(), + self.path.clone(), + self.broadcast.clone(), + self.route.clone(), + request, + ); + run.source = Some((*self.source).clone()); + self.subscribes.push(run); + } Some(Err(err)) => { tracing::debug!(%err, "broadcast closed"); drop(self.listening.take()); @@ -3346,6 +3381,7 @@ impl kio::Task for RunBroadcast { /// subscribes to it, keeping the copy lingering between subscriptions until nobody /// wants it. struct RunSubscribe { + source: Option, this: Subscriber, broadcast_path: Path, track_name: String, @@ -3397,6 +3433,7 @@ impl RunSubscribe { None => crate::auth::Gate::new(auth, path, direction), }; Self { + source: None, this, broadcast_path, track_name: request.name().to_owned(), @@ -3502,6 +3539,7 @@ impl kio::Task for RunSubscribe { )); } RunStep::Subscribe(once) => { + once.source = self.source.clone(); let Some(idle) = ready!(once.poll( waiter, &self.this, @@ -3555,6 +3593,8 @@ enum ServeEnd { /// subscribes any more or the track ended, with the copy kept for [`Linger`], `None` once /// it failed. struct SubscribeOnce { + source: Option, + shared: Option, key: tracks::TrackKey, /// This request's own view of the track's gate, which a resumed subscription's token /// binds to. @@ -3637,6 +3677,8 @@ enum OnceStep { count: u64, end: Result<(), Error>, }, + /// The stream tail settled; keep the alias until datagrams received with the end route. + Datagrams(kio::Consumer<()>), /// Walking away from it: cancelling the request while the fetches it drives go on. Cancel { cancelling: Cancelling, aborted: bool }, } @@ -3697,6 +3739,8 @@ impl SubscribeOnce { gate: &crate::auth::Gate, ) -> Self { Self { + source: None, + shared: None, key: (broadcast_path.clone(), track_name.into()), gate: gate.unnamed(), target: Some(target), @@ -3992,6 +4036,9 @@ impl SubscribeOnce { return Poll::Ready(None); } OnceStep::Serve => { + if let Some(shared) = &mut self.shared { + shared.poll(waiter); + } let end = ready!(self.poll_end(waiter, this, group_fetches)); if let Some(idle) = self.ended(this, broadcast_path, track_name, group_fetches, end) && let Some(step) = self.finish(this, idle) @@ -4024,12 +4071,26 @@ impl SubscribeOnce { Err(err) => fill.abort(err.clone()), } } + let datagrams = end.is_ok() && self.shared.is_some(); live.publisher_ended(end); + if datagrams { + let mut state = this.state.lock(); + self.step = + OnceStep::Datagrams(state.datagram_ends.get_or_insert_with(Default::default).consume()); + continue; + } // The publisher already ended the request, so there is nothing to cancel. if let Some(step) = self.finish(this, false) { return step; } } + OnceStep::Datagrams(routed) => { + let _ = group_fetches.poll(waiter); + ready!(routed.poll_closed(waiter)); + if let Some(step) = self.finish(this, false) { + return step; + } + } OnceStep::Cancel { cancelling, aborted } => { let _ = group_fetches.poll(waiter); ready!(cancelling.poll(waiter)); @@ -4350,6 +4411,11 @@ impl SubscribeOnce { true => track.start_at(largest.map_or(0, |largest| largest.group)), false => track.start_at(self.start), }; + if this.session.max_datagram_size() > 0 + && let Some(source) = &self.source + { + self.shared = Shared::new(this, source, self.key.clone(), &track); + } if let Some(staged) = staged { staged.commit(); } @@ -4365,6 +4431,7 @@ impl SubscribeOnce { let mut state = this.state.lock(); if let Some(held) = state.subscribes.get_mut(&request_id) { held.producer = Some(track.clone()); + held.datagram = self.shared.as_ref().map(|shared| shared.producer.clone()); held.timescale = timescale; held.fill_ok(largest); // A group-0 start joins filterless with no fill for the Largest group's head, @@ -4817,6 +4884,10 @@ impl SubscribeOnce { fn finish(&mut self, this: &Subscriber, idle: bool) -> Option>> { let request_id = self.request_id; let live = self.established.as_mut().expect("established"); + if let Some(mut shared) = self.shared.take() { + let end = live.ended.clone().unwrap_or(Err(Error::Cancel)); + shared.end(end); + } // Release an unallocated join now, or let the session cancel its committed request. live.fetching = None; @@ -5671,22 +5742,95 @@ impl Subscriber { kio::wait(|waiter| recv.poll(waiter, self, stream)).await } - /// Drop an OBJECT_DATAGRAM: a subscription is a stream track's, which takes no datagrams. + /// Release clean ends after the driver read the transport's datagram queue empty. + pub(super) fn datagrams_routed(&self) { + let _ = self.state.lock().datagram_ends.take(); + } + + /// Route OBJECT_DATAGRAM to the shared subscription's datagram model half. /// + /// Return an unknown alias so the driver can try after decoding the answers received with it. /// A malformed datagram is still a protocol error. - pub fn recv_datagram(&self, payload: bytes::Bytes) -> Result<(), Error> { + pub fn recv_datagram(&self, payload: bytes::Bytes) -> Result, Error> { let (datagram, _) = ietf::ObjectDatagram::decode_slice(&payload, self.version)?; - if let ietf::DatagramBody::Status(status) = datagram.body - && !super::object::status_defined(status) + if let ietf::DatagramBody::Status(status) = &datagram.body + && !super::object::status_defined(*status) { return Err(Error::Session(SessionError::ProtocolViolation)); } - tracing::debug!( - alias = datagram.track_alias, - group = datagram.group_id, - "dropping a datagram on a stream track's subscription" - ); - Ok(()) + + let (mut producer, timescale) = { + let state = self.state.lock(); + let aliases = state.aliases.read(); + let Some(request_id) = aliases.map.get(&datagram.track_alias).and_then(|alias| alias.holder()) else { + return Ok(Some(datagram.track_alias)); + }; + let Some(track) = state.subscribes.get(&request_id) else { + return Ok(None); + }; + match &track.datagram { + Some(producer) => (producer.clone(), track.timescale), + None => { + tracing::debug!( + alias = datagram.track_alias, + group = datagram.group_id, + "dropping a datagram on a stream track's subscription" + ); + return Ok(None); + } + } + }; + + // A defined status names no payload, and it does not end the datagram track. + // PUBLISH_DONE is that end. + let ietf::DatagramBody::Payload(payload) = datagram.body else { + tracing::debug!( + alias = datagram.track_alias, + group = datagram.group_id, + "dropping a datagram status" + ); + return Ok(None); + }; + let object_id = datagram.object_id.unwrap_or(0); + if object_id != 0 { + tracing::debug!( + alias = datagram.track_alias, + group = datagram.group_id, + object_id, + "dropping a datagram with a nonzero object id" + ); + return Ok(None); + } + + let timestamp = match &datagram.properties { + Some(raw) => { + let bytes = bytes::Bytes::copy_from_slice(raw); + match super::object::read(&bytes, self.version, timescale) { + Ok(object) => object.timestamp, + Err(err) => { + tracing::debug!(%err, "dropping a datagram with unreadable object properties"); + return Ok(None); + } + } + } + None => None, + }; + if timescale.is_some() && timestamp.is_none() { + tracing::debug!( + alias = datagram.track_alias, + group = datagram.group_id, + "dropping a datagram without a timestamp on a timed track" + ); + return Ok(None); + } + if let Err(err) = producer.insert(crate::datagram::Datagram { + sequence: datagram.group_id, + timestamp, + payload, + }) { + tracing::debug!(%err, "dropping a datagram the track refused"); + } + Ok(None) } } @@ -8878,6 +9022,110 @@ mod tests { assert!(is_protocol_violation(&subscriber.recv_datagram(malformed).unwrap_err())); } + /// A datagram subscription keeps object 0 and drops everything else. An undefined + /// status still closes the session, including on this subscription. + #[kio::test] + async fn a_datagram_subscription_keeps_object_zero() { + use crate::coding::Encode as _; + + let info = crate::datagram::Info::default().with_timescale(Timescale::MILLI); + let producer = info.produce("audio"); + let mut consumer = producer.consume(); + let subscriber = subscriber_with_tracks(&[(RequestId(11), "cam", "audio")]); + subscriber.register_alias(RequestId(11), 7).unwrap(); + { + let mut state = subscriber.state.lock(); + let track = state.subscribes.get_mut(&RequestId(11)).unwrap(); + // Only the datagram model half is present in this receive fixture. + track.producer = None; + track.datagram = Some(producer); + track.timescale = Some(Timescale::MILLI); + } + + let mut properties = Vec::new(); + ietf::encode_object_time( + &mut crate::coding::Encoder::new(&mut properties, Version::Draft18.into()), + crate::Timestamp::from_millis(9).unwrap(), + Timescale::MILLI, + ) + .unwrap(); + let encode = |object_id, body| { + ietf::ObjectDatagram { + track_alias: 7, + group_id: 4, + object_id, + publisher_priority: None, + end_of_group: !matches!(body, ietf::DatagramBody::Status(_)), + properties: Some(properties.clone()), + body, + } + .encode_bytes(Version::Draft18) + .unwrap() + }; + + let unreadable = ietf::ObjectDatagram { + track_alias: 7, + group_id: 3, + object_id: None, + publisher_priority: None, + end_of_group: false, + properties: Some(vec![0x10]), + body: ietf::DatagramBody::Payload(bytes::Bytes::from_static(b"bad")), + } + .encode_bytes(Version::Draft18) + .unwrap(); + subscriber.recv_datagram(unreadable).unwrap(); + assert!(consumer.poll_recv(&kio::Waiter::noop()).is_pending()); + + subscriber + .recv_datagram(encode( + None, + ietf::DatagramBody::Payload(bytes::Bytes::from_static(b"d")), + )) + .unwrap(); + match consumer.poll_recv(&kio::Waiter::noop()) { + Poll::Ready(Ok(Some(datagram))) => { + assert_eq!(datagram.sequence, 4); + assert_eq!(datagram.payload.as_ref(), b"d"); + } + other => panic!("object 0 was not delivered: {other:?}"), + } + + subscriber + .recv_datagram(encode( + Some(3), + ietf::DatagramBody::Payload(bytes::Bytes::from_static(b"x")), + )) + .unwrap(); + assert!( + matches!(consumer.poll_recv(&kio::Waiter::noop()), Poll::Pending), + "a nonzero object id must be dropped" + ); + + // Status 0 is defined, so it is not a payload and not a violation. + subscriber + .recv_datagram(encode(None, ietf::DatagramBody::Status(0))) + .unwrap(); + assert!(matches!(consumer.poll_recv(&kio::Waiter::noop()), Poll::Pending)); + + // Status 1 is undefined on every subscription kind. + let undefined = ietf::ObjectDatagram { + track_alias: 7, + group_id: 4, + object_id: None, + publisher_priority: None, + end_of_group: false, + properties: None, + body: ietf::DatagramBody::Status(1), + } + .encode_bytes(Version::Draft18) + .unwrap(); + assert!(matches!( + subscriber.recv_datagram(undefined), + Err(Error::Session(SessionError::ProtocolViolation)) + )); + } + /// One alias naming two different tracks is the collision section 11.1 makes fatal. #[kio::test] async fn an_alias_reused_for_another_track_is_fatal() { diff --git a/rs/moq-net/src/ietf/subscriber/datagrams.rs b/rs/moq-net/src/ietf/subscriber/datagrams.rs new file mode 100644 index 00000000000..e0c9e41ba66 --- /dev/null +++ b/rs/moq-net/src/ietf/subscriber/datagrams.rs @@ -0,0 +1,154 @@ +//! The two model kinds share one kindless IETF subscription, as on lite-05/06. +use super::*; +use crate::datagram; + +/// Keeps the initial stream interest until its answer also accepts the datagram request. +pub(super) struct Requested { + this: Subscriber, + key: tracks::TrackKey, + id: std::sync::Arc<()>, + subscribing: track::Subscribing, + priority: i8, +} + +impl Requested { + pub(super) fn new( + this: Subscriber, + key: tracks::TrackKey, + track: track::Consumer, + request: datagram::Request, + ) -> Self { + let priority = request.priority().unwrap_or(0); + let id = std::sync::Arc::new(()); + this.state + .lock() + .datagram_requests + .insert(key.clone(), (id.clone(), request)); + Self { + id, + priority, + this, + key, + subscribing: track + .subscribe(track::Subscription::default().with_priority(priority)) + .into_inner(), + } + } + fn reject(&self, err: Error) { + let mut state = self.this.state.lock(); + if let Entry::Occupied(entry) = state.datagram_requests.entry(self.key.clone()) + && std::sync::Arc::ptr_eq(&entry.get().0, &self.id) + { + entry.remove().1.reject(err); + } + } +} +impl kio::Task for Requested { + type Output = (); + fn poll(&mut self, waiter: &kio::Waiter) -> Poll<()> { + let priority = { + let state = self.this.state.lock(); + let Some((id, request)) = state.datagram_requests.get(&self.key) else { + return Poll::Ready(()); + }; + if !std::sync::Arc::ptr_eq(id, &self.id) { + return Poll::Ready(()); + } + if request.poll_unused(waiter).is_ready() && request.close_unused() { + drop(state); + self.reject(Error::Cancel); + return Poll::Ready(()); + } + request.priority() + }; + if let Some(priority) = priority + && priority != self.priority + { + self.priority = priority; + if let Err(err) = self + .subscribing + .update(track::Subscription::default().with_priority(priority)) + { + self.reject(err); + return Poll::Ready(()); + } + } + if let Poll::Ready(Err(err)) = self.subscribing.poll_ok(waiter) { + self.reject(err); + return Poll::Ready(()); + } + // A lingering track already knows its metadata. Only Shared accepting this request + // confirms that its new upstream subscription also owns the datagram half. + Poll::Pending + } +} +impl Drop for Requested { + fn drop(&mut self) { + self.reject(Error::Cancel); + } +} + +/// The datagram half of a stream subscription, holding demand only for actual readers. +pub(super) struct Shared { + pub(super) producer: datagram::Producer, + track: track::Producer, + held: Option, + priority: Option, +} +impl Shared { + pub(super) fn new( + this: &Subscriber, + source: &broadcast::Producer, + key: tracks::TrackKey, + track: &track::Producer, + ) -> Option { + let info = datagram::Info::default() + .with_timescale(track.info().timescale) + .with_priority(track.info().priority); + let pending = this.state.lock().datagram_requests.remove(&key); + let producer = match pending { + Some((_, request)) => request.accept(info), + None => source.create_datagrams(key.1, info).ok()?, + }; + Some(Self { + producer, + track: track.clone(), + held: None, + priority: None, + }) + } + pub(super) fn poll(&mut self, waiter: &kio::Waiter) { + while let Poll::Ready(priority) = self.producer.poll_priority(waiter, self.priority) { + self.priority = priority; + match priority { + None => self.held = None, + Some(priority) => match &mut self.held { + Some(held) => { + if let Err(err) = held.update(held.subscription().with_priority(priority)) { + self.producer.clone().abort(err); + } + } + None => { + self.held = Some( + self.track + .subscribe(track::Subscription::default().with_priority(priority)) + .parked(), + ) + } + }, + } + } + } + pub(super) fn end(&mut self, end: Result<(), Error>) { + self.held = None; + match end { + Ok(()) => self.producer.clone().finish(), + Err(err) => self.producer.clone().abort(err), + } + } +} +impl Drop for Shared { + fn drop(&mut self) { + self.end(Err(Error::Cancel)); + } +} diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 352701606d9..b4ff91c4a1d 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -2706,6 +2706,7 @@ impl Legacy { /// Lite-05/06: a name's datagram track carried on the subscription of its stream track, as /// one subscription carries both kinds there. A subscriber keeps the kind it asked for. struct LegacyDatagrams { + demand: bool, consumer: datagram::Consumer, id: u64, priority: i8, @@ -2733,8 +2734,12 @@ impl LegacyDatagrams { .flatten() .filter(|asked| matches!(asked.poll_info(&kio::Waiter::noop()), Poll::Ready(Ok(_)))); let mut consumer = answered.or_else(|| broadcast.datagrams_live(&name))?; - consumer.set_priority(priority); + let demand = broadcast.has_local_datagrams(); + if demand { + consumer.set_priority(priority); + } Some(Self { + demand, consumer, id, priority, @@ -2746,7 +2751,9 @@ impl LegacyDatagrams { /// Move the datagrams to the subscriber's new `priority`. fn steer(&mut self, priority: i8) { self.priority = priority; - self.consumer.set_priority(priority); + if self.demand { + self.consumer.set_priority(priority); + } if let Some(send) = &mut self.send { send.steer(priority); } diff --git a/rs/moq-net/src/lite/test_transport.rs b/rs/moq-net/src/lite/test_transport.rs index 304c7123dd4..f4977e9e85e 100644 --- a/rs/moq-net/src/lite/test_transport.rs +++ b/rs/moq-net/src/lite/test_transport.rs @@ -936,6 +936,7 @@ impl poll::Chunks for Script { #[derive(Clone)] pub struct ScriptedSession { pub log: Log, + max_datagram_size: usize, /// Optional notifications for tests that drive per-stream readiness queues. wake: Option>, stats: SinkStats, @@ -966,6 +967,11 @@ pub struct ScriptedSession { } impl ScriptedSession { + pub fn with_datagrams(mut self, max_size: usize) -> Self { + self.max_datagram_size = max_size; + self + } + pub fn without_credit(self) -> Self { *self.credit.write().ok().expect("credit is live") = false; self @@ -983,6 +989,7 @@ impl ScriptedSession { pub fn new(script: Vec) -> Self { Self { log: Log::default(), + max_datagram_size: 0, wake: None, stats: SinkStats::default(), close: Default::default(), @@ -1243,7 +1250,7 @@ impl poll::Session for ScriptedSession { } fn max_datagram_size(&self) -> usize { - 0 + self.max_datagram_size } fn address(&self) -> crate::transport::Address { diff --git a/rs/moq-net/src/model/broadcast.rs b/rs/moq-net/src/model/broadcast.rs index 3f2e59427f3..2c7eff33f71 100644 --- a/rs/moq-net/src/model/broadcast.rs +++ b/rs/moq-net/src/model/broadcast.rs @@ -1076,7 +1076,8 @@ impl Consumer { } /// Join the datagram track `name` if one is published or already asked for, asking no - /// handler for it otherwise. + /// handler for it otherwise. The reader adds no demand until `set_priority` is called: + /// an existing stream subscription already carries this forwarding work. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] pub(crate) fn datagrams_live(&self, name: &str) -> Option { let state = self.state.lock(); @@ -1088,7 +1089,7 @@ impl Consumer { drop(state); return Some(source.datagrams_live(name)?.with_stats(self.stats.clone())); } - state.datagrams.get(name)?.try_consume(self.stats.clone()) + state.datagrams.get(name)?.observe(self.stats.clone()) } /// Whether datagram lookups reach a local producer or handler, rather than a session that diff --git a/rs/moq-net/src/model/datagram.rs b/rs/moq-net/src/model/datagram.rs index e5f53496471..8b44002e0f4 100644 --- a/rs/moq-net/src/model/datagram.rs +++ b/rs/moq-net/src/model/datagram.rs @@ -13,8 +13,9 @@ //! //! Delivery is best-effort per hop: a session drops (with a debug log) any datagram whose //! encoded body exceeds the transport's datagram size, and sessions that can't carry -//! datagrams at all (moq-lite before 05, moq-transport for now, or stream-only transports -//! like WebSocket) never deliver them. +//! datagrams at all (moq-lite before 05, or stream-only transports like WebSocket) never +//! deliver them. Moq-transport maps a datagram track onto SUBSCRIBE and OBJECT_DATAGRAM, +//! with the sequence as the group ID and object ID 0. //! //! Moq-lite 07 asks for a datagram track on a request of its own. Moq-lite 05 and 06 carry a //! name's groups and datagrams on one subscription, as moq-transport does: a session serves @@ -139,8 +140,8 @@ struct State { end: Option>, // Live producer-side handles: producers and an unanswered request. producers: usize, - // Each live consumer's subscriber priority. - priorities: slab::Slab, + // Each live consumer's priority, or None for passive forwarding over an existing stream. + priorities: slab::Slab>, } impl State { @@ -194,7 +195,7 @@ impl State { } fn priority(&self) -> Option { - self.priorities.iter().map(|(_, priority)| *priority).max() + self.priorities.iter().filter_map(|(_, priority)| *priority).max() } } @@ -208,7 +209,12 @@ pub(super) struct Weak { impl Weak { /// Join the track as a new consumer, unless it already ended. pub(super) fn try_consume(&self, stats: stats::Scope) -> Option { - Consumer::join(self.name.clone(), &self.state, stats) + Consumer::join(self.name.clone(), &self.state, stats, Some(DEFAULT_PRIORITY)) + } + + /// Observe a live track without adding demand; its stream subscription already holds that. + pub(super) fn observe(&self, stats: stats::Scope) -> Option { + Consumer::join(self.name.clone(), &self.state, stats, None) } } @@ -313,8 +319,13 @@ impl Producer { /// A new consumer of the track, starting with the next datagram written. pub fn consume(&self) -> Consumer { - Consumer::join(self.name.clone(), &self.state, stats::Scope::default()) - .unwrap_or_else(|| Consumer::ended(self.name.clone(), &self.state)) + Consumer::join( + self.name.clone(), + &self.state, + stats::Scope::default(), + Some(DEFAULT_PRIORITY), + ) + .unwrap_or_else(|| Consumer::ended(self.name.clone(), &self.state)) } /// End the track cleanly: consumers read `None` once they have read what is buffered. @@ -476,12 +487,12 @@ pub struct Consumer { impl Consumer { /// Join a live track, or `None` once it ended. - fn join(name: Arc, state: &kio::Shared, stats: stats::Scope) -> Option { + fn join(name: Arc, state: &kio::Shared, stats: stats::Scope, priority: Option) -> Option { let mut guard = state.lock(); if guard.end.is_some() { return None; } - let slot = guard.priorities.insert(DEFAULT_PRIORITY); + let slot = guard.priorities.insert(priority); let cursor = guard.offset + guard.buffer.len() as u64; drop(guard); Some(Self { @@ -564,14 +575,14 @@ impl Consumer { /// This consumer's subscriber priority. pub fn priority(&self) -> i8 { self.slot - .and_then(|slot| self.state.read().priorities.get(slot).copied()) + .and_then(|slot| self.state.read().priorities.get(slot).copied().flatten()) .unwrap_or(DEFAULT_PRIORITY) } /// Set this consumer's subscriber priority, higher first. pub fn set_priority(&mut self, priority: i8) { if let Some(slot) = self.slot { - self.state.lock().priorities[slot] = priority; + self.state.lock().priorities[slot] = Some(priority); } } @@ -631,6 +642,31 @@ mod tests { } } + #[test] + fn passive_forwarding_keeps_the_tail_without_adding_demand() { + let mut producer = Info::default().produce("audio"); + let mut observer = producer.weak().observe(stats::Scope::default()).unwrap(); + let mut copy = observer.clone(); + assert_eq!(producer.priority(), None); + let mut reader = producer.consume(); + reader.set_priority(-7); + assert_eq!(producer.priority(), Some(-7)); + drop(reader); + assert_eq!(producer.priority(), None); + producer.append(None, &b"tail"[..]).unwrap(); + producer.finish(); + for consumer in [&mut observer, &mut copy] { + let Poll::Ready(Ok(Some(datagram))) = consumer.poll_recv(&kio::Waiter::noop()) else { + panic!("tail lost"); + }; + assert_eq!(datagram.payload.as_ref(), b"tail"); + assert!(matches!( + consumer.poll_recv(&kio::Waiter::noop()), + Poll::Ready(Ok(None)) + )); + } + } + #[test] fn append_counts_its_own_sequence_past_inserts() { let mut producer = Info::default().produce("dg"); diff --git a/rs/moq-net/tests/integration/datagram.rs b/rs/moq-net/tests/integration/datagram.rs index 5d5451f68e6..40a6c6b269f 100644 --- a/rs/moq-net/tests/integration/datagram.rs +++ b/rs/moq-net/tests/integration/datagram.rs @@ -256,12 +256,18 @@ async fn a_lite07_stream_subscription_never_takes_datagrams() { ); } -/// Lite-05/06 carry a datagram track as a SUBSCRIBE of its name, keeping only the group +/// Lite-05/06 and moq-transport carry a datagram track as a SUBSCRIBE of its name, keeping only the group /// datagrams. Both ends run this crate, so this covers asking an old publisher for a /// datagram track and serving an old subscriber's SUBSCRIBE from one. #[kio::test] -async fn lite05_and_lite06_map_a_datagram_track_onto_a_subscription() { - for version in ["moq-lite-05", "moq-lite-06"] { +async fn legacy_versions_map_a_datagram_track_onto_a_subscription() { + for version in [ + "moq-lite-05", + "moq-lite-06", + "moq-transport-18", + "moq-transport-20", + "moq-transport-22", + ] { let chain = Chain::new(&[version]).await; let mut producer = chain.publication.broadcast().create_datagrams(TRACK, timed()).unwrap(); producer.append(millis(1), &b"early"[..]).unwrap(); @@ -290,13 +296,20 @@ async fn lite05_and_lite06_map_a_datagram_track_onto_a_subscription() { } } -/// Lite-05/06 carry a name's stream track and datagram track on one subscription, as -/// moq-transport does, so a name with both delivers each to its own reader, directly and +/// Lite-05/06 and moq-transport carry a name's stream track and datagram track on one +/// subscription, so a name with both delivers each to its own reader, directly and /// through a relay whose hops are both old. The datagrams ride the subscription, at its /// priority. #[kio::test] -async fn lite05_and_lite06_carry_both_kinds_of_a_name() { - for hops in [&["moq-lite-05"][..], &["moq-lite-06"], &["moq-lite-06", "moq-lite-06"]] { +async fn legacy_versions_carry_both_kinds_of_a_name() { + for hops in [ + &["moq-lite-05"][..], + &["moq-lite-06"], + &["moq-lite-06", "moq-lite-06"], + &["moq-transport-18"], + &["moq-transport-22"], + &["moq-transport-18", "moq-transport-22"], + ] { let chain = Chain::new(hops).await; let mut track = chain .publication @@ -523,7 +536,12 @@ async fn an_old_subscriber_keeps_a_stream_track_across_mixed_hops() { /// A datagram track can't be asked for over a version or transport without datagrams. #[kio::test] async fn a_datagram_track_needs_datagrams() { - for (version, transport) in [("moq-lite-03", true), ("moq-lite-04", true), ("moq-lite-07-wip", false)] { + for (version, transport) in [ + ("moq-lite-03", true), + ("moq-lite-04", true), + ("moq-lite-07-wip", false), + ("moq-transport-22", false), + ] { let chain = Chain::new(&[version]).await; if !transport { // The second node subscribes through the session it accepted. @@ -536,35 +554,34 @@ async fn a_datagram_track_needs_datagrams() { } } -/// moq-transport carries no datagram tracks: asking for one finds nothing, and the stream -/// track of the same name is served as usual. +/// Keeping the broadcast handle across an idle subscription must not leave its datagram +/// readers attached to the old shared producer or accept stale stream metadata as an answer. #[kio::test] -async fn moq_transport_has_no_datagram_tracks() { - let chain = Chain::new(&["moq-transport-22"]).await; - let mut track = chain - .publication - .broadcast() - .create_track(TRACK, moq_net::track::Info::default().with_timescale(Timescale::MILLI)) - .unwrap(); - let _datagrams = chain.publication.broadcast().create_datagrams(TRACK, timed()).unwrap(); +async fn ietf_datagram_readers_rejoin_a_lingering_broadcast() { + let chain = Chain::new(&["moq-transport-18", "moq-transport-22"]).await; + let mut producer = chain.publication.broadcast().create_datagrams(TRACK, timed()).unwrap(); let remote = chain.remote().await; - assert!(matches!(remote.datagrams(TRACK), Err(Error::NotFound))); - - let mut subscriber = kio::clock::timeout(TIMEOUT, remote.track(TRACK).unwrap().subscribe(None)) - .await - .expect("subscribe timeout") - .expect("subscribe"); - support::harness::subscribed(&track).await; - track.write_frame(millis(1), &b"frame"[..]).unwrap(); - let mut group = kio::clock::timeout(TIMEOUT, subscriber.recv_group()) - .await - .expect("group timeout") - .unwrap() - .expect("a group"); - assert_eq!( - &group.read_frame().await.unwrap().expect("a frame").payload[..], - b"frame" - ); + for priority in [2, 3] { + let mut consumer = remote.datagrams(TRACK).unwrap(); + consumer.set_priority(priority); + kio::clock::timeout(TIMEOUT, datagram_priority(&producer, Some(priority))) + .await + .expect("the rejoined request never reached the publisher"); + kio::clock::timeout(TIMEOUT, consumer.info()) + .await + .expect("metadata timeout") + .unwrap(); + producer.append(millis(priority as u64), &b"live"[..]).unwrap(); + assert_eq!(recv(&mut consumer).await.unwrap().unwrap().payload.as_ref(), b"live"); + consumer.set_priority(-3); + kio::clock::timeout(TIMEOUT, datagram_priority(&producer, Some(-3))) + .await + .expect("the rejoined reader's lowered priority never arrived"); + drop(consumer); + kio::clock::timeout(TIMEOUT, datagram_priority(&producer, None)) + .await + .expect("the shared subscription stayed wanted"); + } } /// A publisher that answers a datagram request and writes at once sends the datagram right @@ -573,7 +590,14 @@ async fn moq_transport_has_no_datagram_tracks() { /// datagram, and routes the datagram before it ends the track. #[kio::test] async fn a_datagram_sent_right_behind_its_answer_arrives() { - for version in ["moq-lite-05", "moq-lite-06", "moq-lite-07-wip"] { + for version in [ + "moq-lite-05", + "moq-lite-06", + "moq-lite-07-wip", + "moq-transport-18", + "moq-transport-20", + "moq-transport-22", + ] { for finish in [false, true] { let chain = Chain::new(&[version]).await; let requests = chain.publication.broadcast().requests(); @@ -600,7 +624,7 @@ async fn a_datagram_sent_right_behind_its_answer_arrives() { // The second node subscribes through the session it accepted. assert_eq!( chain.pairs[0].server.stats().datagrams_unrouted, - Some(0), + version.parse::().unwrap().is_lite().then_some(0), "{version} finish={finish}" ); } @@ -714,3 +738,22 @@ async fn a_track_ended_under_a_datagram_flood_keeps_its_stream() { .unwrap_or_else(|_| panic!("{version}: the track never released its stream")); } } + +/// Ending a track must preserve its queued datagrams across the receiver's budget. +#[kio::test] +async fn an_ietf_datagram_burst_sent_with_its_end_arrives() { + for version in ["moq-transport-18", "moq-transport-20", "moq-transport-22"] { + let chain = Chain::new(&[version]).await; + let mut producer = chain.publication.broadcast().create_datagrams(TRACK, timed()).unwrap(); + let mut consumer = chain.consume(&producer, 5).await; + for i in 0..64 { + producer.append(millis(i), bytes::Bytes::from(vec![i as u8])).unwrap(); + } + producer.finish(); + for i in 0..64 { + let datagram = recv(&mut consumer).await.unwrap().expect("queued datagram"); + assert_eq!(datagram.sequence, i, "{version}"); + } + assert!(recv(&mut consumer).await.unwrap().is_none(), "{version}"); + } +} diff --git a/rs/moq-tokio/tests/integration/datagram_delivery.rs b/rs/moq-tokio/tests/integration/datagram_delivery.rs index ce881e6dcc7..a8a141c0c4d 100644 --- a/rs/moq-tokio/tests/integration/datagram_delivery.rs +++ b/rs/moq-tokio/tests/integration/datagram_delivery.rs @@ -322,8 +322,7 @@ async fn benchmark_datagram_delivery() { println!( "version,cell,viewer,offered,lost_packets,overflowed_packets,undelivered_frame_packets,reordered_packets,p50_ms,p99_ms,max_ms" ); - // moq-transport has no datagram tracks. - for version in ["moq-lite-06", "moq-lite-07-wip"] { + for version in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"] { for cell in cells { measure(version, cell).await; } diff --git a/test/interop/README.md b/test/interop/README.md index 3c0733c558f..1939304a3b6 100644 --- a/test/interop/README.md +++ b/test/interop/README.md @@ -239,6 +239,11 @@ one cell. The tails check their own connections, so a fallback fails the lane. The in-tree control lanes also run with `--tail`: +- Four untimed 900-byte datagrams cross the relay in both directions between Rust + and each native JS runtime. The subscriber acknowledges resolved metadata before + sending begins, then acknowledges each packet, so this checks framing and bytes + rather than best-effort burst capacity. The publisher stays open until the read is + acknowledged, keeping a reliable track end from overtaking the datagrams. - A JS wire requester gets the Rust publisher's exact largest location through TRACK\_STATUS on every draft. Rust also queries the JS publisher's non-default priority before subscribing, when the answer can describe a track. @@ -252,7 +257,7 @@ The in-tree control lanes also run with `--tail`: The standalone control endpoints use OS-assigned ports and production QMux, SETUP codecs, and request sessions. They verify their negotiated version directly. -The query lanes through moq-relay use their usual version-log check. +The datagram and query lanes through moq-relay use their usual version-log check. Some cases change shape on moq-transport: diff --git a/test/interop/clients/js-native/tail.ts b/test/interop/clients/js-native/tail.ts index 0b40c72c2f3..c7f3d7af6ad 100644 --- a/test/interop/clients/js-native/tail.ts +++ b/test/interop/clients/js-native/tail.ts @@ -6,9 +6,11 @@ * it must read a contiguous run of groups through the last one. `compress` publishes a compressed * track with several frames, one empty, in each group. `cached` publishes every group before * announcing, for a FETCH of a past group. `history` writes only the last group live and serves the - * ones below it, never cached, from an in-memory responder on `track.requests()`. `late` writes the head of group 0 before announcing and - * the rest only once the harness relays that the subscriber read it, so a join that skips the head - * stalls rather than racing the publisher. + * ones below it, never cached, from an in-memory responder on `track.requests()`. `datagram` publishes + * an untimed datagram track and sends each 900-byte datagram only after the harness says the subscriber + * read the last one. `late` writes the head of group 0 before announcing and the rest only once the + * harness relays that the subscriber read it, so a join that skips the head stalls rather than racing + * the publisher. * * @module */ @@ -30,6 +32,7 @@ const compress = mode === "compress"; console.debug = console.error; console.info = console.error; const join = mode === "join" || mode === "status"; +const datagram = mode === "datagram"; const status = mode === "status"; const cached = mode === "cached"; const history = mode === "history"; @@ -39,10 +42,10 @@ if ( !url || !path || !["publish", "subscribe"].includes(role) || - ![undefined, "drop", "join", "compress", "status", "cached", "history", "late", "fetch"].includes(mode) + ![undefined, "drop", "join", "compress", "datagram", "status", "cached", "history", "late", "fetch"].includes(mode) ) { throw new Error( - "usage: tail.ts publish|subscribe URL BROADCAST [VERSION [drop|join|compress|status|cached|history|late|fetch]]", + "usage: tail.ts publish|subscribe URL BROADCAST [VERSION [drop|join|compress|datagram|status|cached|history|late|fetch]]", ); } @@ -117,46 +120,99 @@ const connection = await Moq.Connection.connect({ }); try { if (role === "publish") { - const broadcastRoute = origin.route(Moq.Path.from(path), { hidden: true }); - const broadcast = broadcastRoute.broadcast; - // `cached` and `late` read groups a newer one supersedes, which the default zero max age - // would leave stale. - const track = broadcast.createTrack("tail", { compress, maxAge: null, ...(status ? { priority: 2 } : {}) }); - if (cached) write(track); - if (history) serveHistory(track); - let head: Moq.Group.Producer | undefined; - if (late) { - head = track.appendGroup(); - for (const payload of frames(0).slice(0, LATE_HEAD)) head.writeFrame({ payload }); - } - broadcastRoute.announce(); - const lines = createInterface({ input: process.stdin }); - const commands = lines[Symbol.asyncIterator](); - if (head) { - assert.equal((await commands.next()).value, "late tail", "subscriber did not read the head"); - for (const payload of frames(0).slice(LATE_HEAD)) head.writeFrame({ payload }); - head.finish(); - for (let sequence = 1; sequence < LATE_GROUPS; sequence++) { - const group = track.appendGroup(); - for (const payload of frames(sequence)) group.writeFrame({ payload }); - group.finish(); + const broadcastPublication = origin.route(Moq.Path.from(path), { hidden: true }); + const broadcast = broadcastPublication.broadcast; + if (datagram) { + // Untimed: the OBJECT_DATAGRAM carries no timestamp, so the subscriber checks the payload alone. + const track = broadcast.createDatagrams("tail"); + broadcastPublication.announce(); + const lines = createInterface({ input: process.stdin }); + const commands = lines[Symbol.asyncIterator](); + try { + while (track.priority.peek() === undefined) await track.priority.changed(); + for (let sequence = 0; sequence < GROUPS; sequence++) { + assert.equal((await commands.next()).value, `datagram ${sequence}`, "missing datagram command"); + track.append(undefined, new Uint8Array(900).fill(sequence)); + } + const ack = await commands.next(); + assert.equal(ack.value, "clean end", "reader did not acknowledge a clean end"); + } finally { + lines.close(); } track.finish(); - track.close(); - } else if (!status && !cached && !history) { - const demand = track.demand(); - while (demand.wanted.peek() === undefined) await demand.wanted.changed(); - write(track); - track.close(); + broadcast.close(); + console.error("tail acknowledged"); + } else { + // `cached` and `late` read groups a newer one supersedes, which the default zero max age + // would leave stale. + const track = broadcast.createTrack("tail", { compress, maxAge: null, ...(status ? { priority: 2 } : {}) }); + if (cached) write(track); + if (history) serveHistory(track); + let head: Moq.Group.Producer | undefined; + if (late) { + head = track.appendGroup(); + for (const payload of frames(0).slice(0, LATE_HEAD)) head.writeFrame({ payload }); + } + broadcastPublication.announce(); + const lines = createInterface({ input: process.stdin }); + const commands = lines[Symbol.asyncIterator](); + if (head) { + assert.equal((await commands.next()).value, "late tail", "subscriber did not read the head"); + for (const payload of frames(0).slice(LATE_HEAD)) head.writeFrame({ payload }); + head.finish(); + for (let sequence = 1; sequence < LATE_GROUPS; sequence++) { + const group = track.appendGroup(); + for (const payload of frames(sequence)) group.writeFrame({ payload }); + group.finish(); + } + track.finish(); + track.close(); + } else if (!status && !cached && !history) { + const demand = track.demand(); + while (demand.wanted.peek() === undefined) await demand.wanted.changed(); + write(track); + track.close(); + } + try { + const ack = await commands.next(); + assert.equal(ack.value, "clean end", "reader did not acknowledge a clean end"); + } finally { + lines.close(); + } + broadcast.close(); + console.error("tail acknowledged"); } + } else if (datagram) { + const request = origin.follow(Moq.Path.from(path)); try { - const ack = await commands.next(); - assert.equal(ack.value, "clean end", "reader did not acknowledge a clean end"); + let broadcast = request.active.peek(); + while (!broadcast) { + await request.active.changed(); + broadcast = request.active.peek(); + } + const track = broadcast.datagrams("tail"); + try { + await track.info(); + process.stdout.write("datagram ready\n"); + for (let sequence = 0; sequence < GROUPS; sequence++) { + const frame = await track.recv(); + assert(frame, "missing datagram"); + assert.equal(frame.sequence, BigInt(sequence), "wrong datagram sequence"); + assert.equal(frame.payload.byteLength, 900, "truncated datagram"); + assert( + frame.payload.every((byte) => byte === sequence), + "corrupt datagram", + ); + assert.equal(frame.timestamp, undefined, "unexpected IETF timestamp"); + process.stdout.write(`datagram ${sequence}\n`); + } + console.error("datagram clean groups=0,1,2,3 bytes=3600"); + } finally { + track.close(); + } } finally { - lines.close(); + request.close(); } - broadcast.close(); - console.error("tail acknowledged"); } else { const request = origin.follow(Moq.Path.from(path)); try { diff --git a/test/interop/clients/rust/tail.rs b/test/interop/clients/rust/tail.rs index d17f1e272e8..199aefa9e61 100644 --- a/test/interop/clients/rust/tail.rs +++ b/test/interop/clients/rust/tail.rs @@ -5,9 +5,11 @@ //! so it must read a contiguous run of groups through the last one. `cached` publishes every //! group before connecting, so a FETCH of a past group never races its write: moq-transport //! refuses a FETCH that starts past the largest group rather than waiting for it. `compress` -//! publishes a compressed track with several frames, one empty, in each group. `late` writes -//! the head of group 0 before connecting and the rest only once the harness relays that the -//! subscriber read it, so a join that skips the head stalls rather than racing the publisher. +//! publishes a compressed track with several frames, one empty, in each group. `datagram` +//! publishes an untimed datagram track and sends each 900-byte datagram only after the harness +//! says the subscriber read the last one. `late` writes the head of group 0 before connecting +//! and the rest only once the harness relays that the subscriber read it, so a join that skips +//! the head stalls rather than racing the publisher. use std::time::Duration; use anyhow::{Context, ensure}; @@ -76,14 +78,14 @@ async fn main() -> anyhow::Result<()> { let args: Vec<_> = std::env::args().collect(); ensure!( (4..=6).contains(&args.len()), - "usage: interop-tail publish|subscribe URL BROADCAST [VERSION [drop|join|cached|compress|status|late]]" + "usage: interop-tail publish|subscribe URL BROADCAST [VERSION [drop|join|cached|compress|datagram|status|late]]" ); let url: url::Url = args[2].parse()?; let mode = args.get(5).map(String::as_str); ensure!( matches!( mode, - None | Some("drop" | "join" | "cached" | "compress" | "status" | "late") + None | Some("drop" | "join" | "cached" | "compress" | "datagram" | "status" | "late") ), "unknown mode: {mode:?}" ); @@ -91,6 +93,7 @@ async fn main() -> anyhow::Result<()> { let join = matches!(mode, Some("join" | "status")); let cached = mode == Some("cached"); let compress = mode == Some("compress"); + let datagram = mode == Some("datagram"); let status = mode == Some("status"); let late = mode == Some("late"); let mut commands = tokio::io::BufReader::new(tokio::io::stdin()).lines(); @@ -106,64 +109,97 @@ async fn main() -> anyhow::Result<()> { match args[1].as_str() { "publish" => { let route = origin.route(&args[3], moq_net::origin::RouteOptions::hidden())?; - let mut track = route.broadcast().create_track( - "tail", - moq_net::track::Info::default() - .with_timescale(moq_net::Timescale::MILLI) - .with_compress(compress) - // `cached` and `late` read groups a newer one supersedes, which the default - // zero max age would leave stale. - .with_max_age(None), - )?; - route.announce()?; - if cached { - write(&mut track, mode)?; - } - let mut head = None; - if late { - let mut group = track.append_group()?; - for frame in frames(0, mode).into_iter().take(LATE_HEAD) { - group.write_frame(moq_net::Timestamp::ZERO, frame)?; + let broadcast = route.broadcast(); + if datagram { + // Untimed: the OBJECT_DATAGRAM carries no timestamp, so the subscriber can + // check the payload alone. Stay open until each one is acknowledged. + let mut track = broadcast.create_datagrams("tail", moq_net::datagram::Info::default())?; + route.announce()?; + let session = client + .clone() + .with_publisher(&origin) + .connect(url) + .established() + .await?; + let mut priority = track.priority(); + while priority.is_none() { + priority = kio::wait(|waiter| track.poll_priority(waiter, priority)).await; + } + for sequence in 0..GROUPS { + ensure!( + commands.next_line().await?.as_deref() == Some(&format!("datagram {sequence}")), + "missing datagram command" + ); + track.append(None, vec![sequence as u8; 900])?; } - head = Some(group); - } - let session = client - .clone() - .with_publisher(&origin) - .connect(url) - .established() - .await?; - if let Some(mut group) = head { ensure!( - commands.next_line().await?.as_deref() == Some("late tail"), - "subscriber did not read the head" + commands.next_line().await?.as_deref() == Some("clean end"), + "reader did not acknowledge a clean end" ); - for frame in frames(0, mode).into_iter().skip(LATE_HEAD) { - group.write_frame(moq_net::Timestamp::ZERO, frame)?; + track.finish(); + broadcast.close(); + session.close().await?; + eprintln!("tail acknowledged"); + } else { + let mut track = broadcast.create_track( + "tail", + moq_net::track::Info::default() + .with_timescale(moq_net::Timescale::MILLI) + .with_compress(compress) + // `cached` and `late` read groups a newer one supersedes, which the default + // zero max age would leave stale. + .with_max_age(None), + )?; + route.announce()?; + if cached { + write(&mut track, mode)?; } - group.finish()?; - for sequence in 1..LATE_GROUPS { + let mut head = None; + if late { let mut group = track.append_group()?; - for frame in frames(sequence, mode) { + for frame in frames(0, mode).into_iter().take(LATE_HEAD) { + group.write_frame(moq_net::Timestamp::ZERO, frame)?; + } + head = Some(group); + } + let session = client + .clone() + .with_publisher(&origin) + .connect(url) + .established() + .await?; + if let Some(mut group) = head { + ensure!( + commands.next_line().await?.as_deref() == Some("late tail"), + "subscriber did not read the head" + ); + for frame in frames(0, mode).into_iter().skip(LATE_HEAD) { group.write_frame(moq_net::Timestamp::ZERO, frame)?; } group.finish()?; + for sequence in 1..LATE_GROUPS { + let mut group = track.append_group()?; + for frame in frames(sequence, mode) { + group.write_frame(moq_net::Timestamp::ZERO, frame)?; + } + group.finish()?; + } + track.finish()?; + } else if !cached { + let mut demand = track.demand(); + while demand.wanted_changed().await?.is_none() {} + write(&mut track, mode)?; } - track.finish()?; - } else if !cached { - let mut demand = track.demand(); - while demand.wanted_changed().await?.is_none() {} - write(&mut track, mode)?; + // A finished track is not a drained transport. Keep the session alive + // until the harness confirms the subscriber checked every byte and EOF. + ensure!( + commands.next_line().await?.as_deref() == Some("clean end"), + "reader did not acknowledge a clean end" + ); + broadcast.close(); + session.close().await?; + eprintln!("tail acknowledged"); } - // A finished track is not a drained transport. Keep the session alive - // until the harness confirms the subscriber checked every byte and EOF. - ensure!( - commands.next_line().await?.as_deref() == Some("clean end"), - "reader did not acknowledge a clean end" - ); - route.broadcast().close(); - session.close().await?; - eprintln!("tail acknowledged"); } "subscribe" => { let consumer = origin.consume(); @@ -178,6 +214,26 @@ async fn main() -> anyhow::Result<()> { .available() .await? .context("went offline before serving")?; + if datagram { + let mut track = broadcast.datagrams("tail")?; + track.info().await?; + println!("datagram ready"); + for sequence in 0..GROUPS { + let frame = track.recv().await?.context("missing datagram")?; + ensure!(frame.sequence == sequence, "wrong datagram sequence"); + ensure!(frame.payload.len() == 900, "truncated datagram"); + ensure!( + frame.payload.iter().all(|byte| *byte == sequence as u8), + "corrupt datagram" + ); + ensure!(frame.timestamp.is_none(), "unexpected IETF timestamp"); + println!("datagram {sequence}"); + } + eprintln!("datagram clean groups=0,1,2,3 bytes=3600"); + session.close().await?; + client.close().await; + return Ok(()); + } let remote = broadcast.track("tail")?; if status { // Query before subscribing so draft-18+ must use TRACK_STATUS, diff --git a/test/interop/interop.sh b/test/interop/interop.sh index a306a9cf400..a2e22c223c5 100755 --- a/test/interop/interop.sh +++ b/test/interop/interop.sh @@ -539,6 +539,8 @@ plan_pass() { record "not run" case=tail publisher=rust subscriber="$runtime" record "not run" case=tail publisher="$runtime" subscriber=rust if [[ "$VERSION" == moq-transport-* ]]; then + record "not run" case=datagram publisher=rust subscriber="$runtime" + record "not run" case=datagram publisher="$runtime" subscriber=rust record "not run" case="wire track status" publisher=rust subscriber="$runtime" record "not run" case=goaway publisher=rust subscriber="$runtime" [[ "$runtime" != js-native-bun ]] || record "not run" case=goaway publisher="$runtime" subscriber=rust @@ -1962,7 +1964,10 @@ run_tail_pair() { mode="join" want='tail join clean end=4 groups=[0-3](,[0-3])*' fi - if [[ "$kind" == "track status" ]]; then + if [[ "$kind" == datagram ]]; then + mode=datagram + want='datagram clean groups=0,1,2,3 bytes=3600' + elif [[ "$kind" == "track status" ]]; then mode=status want="track status answered" elif [[ "$kind" == compress ]]; then @@ -1983,8 +1988,26 @@ run_tail_pair() { # Each side's token grants this broadcast alone, like a media round's. harness_spawn "$name-pub" "$LOGS/$name-pub.log" run_tail_publisher "$fifo" "$pub" publish "$broadcast" "$(token_url --publish "$(publish_grant "$broadcast")")" "$version" "$mode" pid="$HARNESS_PID" - harness_spawn "$name-sub" "$LOGS/$name-sub.log" run_tail_client "$sub" subscribe "$broadcast" "$(token_url --subscribe "$(subscribe_grant "$broadcast")")" "$version" "$mode" - subscriber_pid="$HARNESS_PID" + if [[ "$mode" == datagram ]]; then + # Subscription metadata confirms the alias is bound. An acknowledgement per datagram + # avoids testing a best-effort receive buffer's burst capacity. + mkfifo "$LOGS/datagram-ack" + exec 8<>"$LOGS/datagram-ack" + harness_spawn "$name-sub" "$LOGS/$name-sub.log" run_tail_subscriber "$LOGS/datagram-ack" "$sub" subscribe "$broadcast" "$(token_url --subscribe "$(subscribe_grant "$broadcast")")" "$version" "$mode" + subscriber_pid="$HARNESS_PID" + local ack sequence + if read -r -t "$TIMEOUT" ack <&8 && [[ "$ack" == "datagram ready" ]]; then + for sequence in 0 1 2 3; do + printf 'datagram %s\n' "$sequence" >&9 + read -r -t "$TIMEOUT" ack <&8 && [[ "$ack" == "datagram $sequence" ]] || break + done + fi + exec 8>&- + rm "$LOGS/datagram-ack" + else + harness_spawn "$name-sub" "$LOGS/$name-sub.log" run_tail_client "$sub" subscribe "$broadcast" "$(token_url --subscribe "$(subscribe_grant "$broadcast")")" "$version" "$mode" + subscriber_pid="$HARNESS_PID" + fi harness_wait "$subscriber_pid" || true # Like run_native, a verified data result survives the NAPI addon's exit crash. if read=$(grep -xE "$want" "$LOGS/$name-sub.log") && { [[ "$mode" != status ]] || grep -qx "track status answered" "$LOGS/$name-sub.log"; }; then @@ -2566,6 +2589,8 @@ run_tails() { run_tail_pair rust "$runtime" run_tail_pair "$runtime" rust if [[ "$VERSION" == moq-transport-* ]]; then + run_tail_pair rust "$runtime" datagram + run_tail_pair "$runtime" rust datagram run_control_case "$runtime" status run_control_case "$runtime" goaway [[ "$runtime" != js-native-bun ]] || run_control_case "$runtime" reverse-goaway