Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ flate2 = { version = "1.1", default-features = false }
futures = "0.3"
getrandom = { version = "0.4", features = ["wasm_js"] }
hang = { version = "0.21.11", path = "rs/hang" }
hashlink = "0.12.2"
hex = "0.4"
# HMAC-SHA256 for the mDNS membership proofs (moq-tokio's `mdns` feature).
hmac = "0.13"
Expand Down
29 changes: 22 additions & 7 deletions drafts/draft-lcurley-moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,8 +408,15 @@ The publisher replies with a single ANNOUNCE_OK message followed by announcement
- ANNOUNCE_END: a previously started route is no longer advertised.
- ANNOUNCE_UPDATE: a previously started advertisement was atomically updated (new hops or cost).
- ANNOUNCE_RESTART: another publisher instance now serves a previously started advertisement.
- SYNCED: the initial set is complete.

ANNOUNCE_OK carries metadata that applies to every announcement on the stream: the publisher's own `Hop ID` (the implicit trailing entry of every announcement's path) and the number of initial announcements, which lets the subscriber deliver the initial set as a batch (see [ANNOUNCE_OK](#announce-ok)).
ANNOUNCE_OK carries metadata that applies to every announcement on the stream: the publisher's own `Hop ID` (the implicit trailing entry of every announcement's path) (see [ANNOUNCE_OK](#announce-ok)).
The publisher MUST send [SYNCED](#synced) exactly once, after the initial set.
The subscriber MAY batch announcements until SYNCED; subsequent announcements are live updates.
The initial set is relative to this hop: a relay answers with what it holds without waiting for upstreams.
Each reader's pending queue MUST collapse updates by route in insertion order, rewriting existing entries in place and omitting an unseen route that ends.
The publisher MUST send SYNCED at the first drain of that queue or after sending N events, whichever comes first, where N is the queue size when the reader subscribed.
A retraction of a route already sent MAY precede SYNCED.

Each ANNOUNCE_START implicitly assigns the next Announce ID on the stream: a counter starting at 0 that increments by 1 per advertisement.
The id never appears on the wire; both endpoints derive it from the message order on the (reliable, ordered) stream.
Expand Down Expand Up @@ -949,7 +956,6 @@ It carries metadata that is constant for the lifetime of the stream and applies
ANNOUNCE_OK Message {
Message Length (i)
Hop ID (i)
Active Count (i)
}
~~~

Expand All @@ -960,11 +966,18 @@ The value 0 is reserved to mean "unknown": either no Hop ID was assigned (e.g. w
A publisher that assigns a Hop ID MUST choose a non-zero value, and SHOULD assign itself one (a fresh random value per session suffices), so downstream receivers can detect loops through it.
Receivers reconstruct the full path as `Hop IDs ++ [ANNOUNCE_OK.Hop ID]`.

**Active Count**:
The number of ANNOUNCE_START messages that the publisher will send immediately as the initial set.
The subscriber MAY block reporting any announcement to the application until all `Active Count` initial announcements have arrived, then deliver the initial set as a batch.
Any announcements beyond `Active Count` are live updates and SHOULD be reported as they arrive.
A value of `0` is valid and means the publisher is offering no initial available broadcasts; all subsequent announcements (if any) are live updates.
## SYNCED {#synced}
The initial set on a discovery stream has been delivered.

~~~
SYNCED Message {
Type (i) = 0x4
Message Length (i) = 0
}
~~~

SYNCED has no body and assigns no Announce ID.
A second SYNCED on the same stream, or a stream ending before SYNCED, is a PROTOCOL_VIOLATION.


## ANNOUNCE_START {#announce-start}
Expand Down Expand Up @@ -1609,6 +1622,8 @@ The `Message Length` describes the payload size on the wire.

## moq-lite-07

- Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery.

- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Receivers do not check it, since a received timestamp may be a send time.
- Added the reliable ordered RTCDataChannel binding, with one Qmux Record per binary message and no datagrams.

Expand Down
63 changes: 61 additions & 2 deletions js/net/bench/broadcasts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,15 @@ import { Producer as BroadcastProducer } from "../src/broadcast.ts";
import { BroadcastCache } from "../src/consume.ts";
import * as Epoch from "../src/epoch.ts";
import { HopSchema, Route } from "../src/hop.ts";
import { AnnounceRequest } from "../src/lite/announce.ts";
import { Publisher } from "../src/lite/publisher.ts";
import { Subscriber } from "../src/lite/subscriber.ts";
import { TrackInfo } from "../src/lite/track.ts";
import { Version } from "../src/lite/version.ts";
import { ALPN_07_WIP, Version } from "../src/lite/version.ts";
import { createMockTransportPair } from "../src/mock.ts";
import { Producer } from "../src/origin.ts";
import * as Path from "../src/path.ts";
import { Writer } from "../src/stream.ts";
import { Stream, Writer } from "../src/stream.ts";

const routeCounts = [8, 32, 128];
const observerCounts = [1, 8, 32];
Expand Down Expand Up @@ -369,3 +372,59 @@ for (const routeCount of routeCounts) {
console.log(`retirement,${routeCount},${consumerCount},${((elapsed * 1000) / 256).toFixed(2)},${refused}`);
}
}

// Exercise each sender's keyed queue while its first wire write is stalled.
console.log("scope,routes,observers,update_us");
for (const routeCount of routeCounts) {
for (const observerCount of observerCounts) {
const origin = new Producer();
const routes = Array.from({ length: routeCount }, (_, index) => {
const broadcast = origin.createBroadcast(Path.from(`z/${index}`));
broadcast.announce();
return broadcast;
});
const senders = Array.from({ length: observerCount }, () => {
const pair = createMockTransportPair(ALPN_07_WIP);
const publisher = new Publisher(pair.server, Version.DRAFT_07, HopSchema.parse(1n), origin.consume());
const entered = Promise.withResolvers<void>();
const release = Promise.withResolvers<void>();
const stream = new Stream({
version: Version.DRAFT_07,
readable: new ReadableStream<Uint8Array>(),
writable: new WritableStream<Uint8Array>({
async write() {
entered.resolve();
await release.promise;
},
}),
});
const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
return { pair, publisher, entered, release, running };
});
await Promise.all(senders.map((sender) => sender.entered.promise));
const start = performance.now();
for (let index = 0; index < updates; index++) {
const broadcast = origin.createBroadcast(Path.from(`a/${index}`));
broadcast.announce();
await new Promise<void>((resolve) => setImmediate(resolve));
broadcast.close();
await new Promise<void>((resolve) => setImmediate(resolve));
}
console.log(
`stalled-announce,${routeCount},${observerCount},${(((performance.now() - start) * 1000) / updates).toFixed(1)}`,
);
await Promise.all(
senders.map(async ({ pair, publisher, release, running }) => {
const drained = publisher.drain();
release.resolve();
await drained;
await running;
publisher.close();
pair.client.close();
pair.server.close();
}),
);
for (const route of routes) route.close();
origin.close();
}
}
15 changes: 15 additions & 0 deletions js/net/src/lite/announce.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -469,3 +469,18 @@ test("lite06 ignores a legacy cold price and writes its saturation ceiling", asy
expect(await r.u62()).toBe(12n);
expect(await r.u62()).toBe(2n ** 62n - 1n);
});

test("SYNCED is empty and belongs only to lite-07", async () => {
expect(
await bytes((w) => encodeAnnounceBroadcast(w, { status: "synced" }, Version.DRAFT_07), Version.DRAFT_07),
).toEqual(new Uint8Array([4, 0]));
expect(await roundTrip({ status: "synced" }, Version.DRAFT_07)).toEqual({ status: "synced" });
await expect(roundTrip({ status: "synced" }, Version.DRAFT_06)).rejects.toThrow();
});

test("ANNOUNCE_OK omits Active Count only on lite-07", async () => {
for (const version of [Version.DRAFT_06, Version.DRAFT_07]) {
const wire = await bytes((w) => new AnnounceOk(HopSchema.parse(1n), 7).encode(w, version), version);
expect(wire).toEqual(new Uint8Array(version === Version.DRAFT_06 ? [2, 1, 7] : [1, 1]));
}
});
31 changes: 22 additions & 9 deletions js/net/src/lite/announce.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
hasExcludeHop,
hasHidden,
hasRouteCost,
hasSynced,
Version,
} from "./version.ts";

Expand All @@ -30,6 +31,7 @@ const ANNOUNCE_END = 1;
const ANNOUNCE_UPDATE = 2;
// lite-07: another publisher instance replaces the advertisement.
const ANNOUNCE_RESTART = 3;
const SYNCED = 4;

/**
* Lite-07: copy `keep` path segments (from the head) or hops (from the tail) of the
Expand Down Expand Up @@ -76,7 +78,9 @@ export type AnnounceBroadcast =
* instance. The id stays live. */
| { status: "restart"; id: bigint; epoch?: Epoch.Valid; hops: Hop[]; cost?: Cost; hopBase?: Base }
/** An unknown lite-06+ announce type, skipped by length. Does not assign an id. */
| { status: "skipped" };
| { status: "skipped" }
/** Lite07+: the initial set has been delivered. */
| { status: "synced" };

// Both wire rules on a hop chain, applied to what we send and to what we receive: a
// chain that revisits a hop looped, so neither forwarding it nor subscribing through it
Expand Down Expand Up @@ -234,6 +238,8 @@ async function encodeAnnounce06Body(w: Writer, msg: AnnounceBroadcast, version:
await encodeHopsBlock(w, version, msg.hops, msg.hopBase);
await encodeRouteCost(w, version, msg.cost);
break;
case "synced":
break;
case "ended":
// The pre-lite-06 path-form retraction has no place on lite-06.
throw new Error("ended-by-path not supported for this version");
Expand All @@ -255,6 +261,9 @@ function announce06Type(msg: AnnounceBroadcast, version: Version): number {
// Older versions send an end and a start instead.
if (!hasAnnounceRestart(version)) throw new Error("announce restart not supported for this version");
return ANNOUNCE_RESTART;
case "synced":
if (!hasSynced(version)) throw new Error("synced not supported for this version");
return SYNCED;
case "ended":
throw new Error("ended-by-path not supported for this version");
case "skipped":
Expand All @@ -270,6 +279,9 @@ async function decodeAnnounce06Body(r: Reader, typ: number, version: Version): P
const hops = await decodeHopsBlock(r, version);
return { status: "active", ...path, epoch, ...hops, cost: await decodeRouteCost(r, version) };
}
case SYNCED:
if (!hasSynced(version)) break;
return { status: "synced" };
case ANNOUNCE_END:
return { status: "endedId", id: await r.u62() };
case ANNOUNCE_UPDATE: {
Expand Down Expand Up @@ -306,6 +318,7 @@ async function encodeLegacyBody(w: Writer, msg: AnnounceBroadcast, version: Vers
await w.string(Path.encode(msg.suffix));
await encodeHops(w, version, []);
break;
case "synced":
case "endedId":
case "update":
case "restart":
Expand Down Expand Up @@ -526,8 +539,8 @@ export class AnnounceInit {
///
/// `origin` is the responder's Hop ID, which the subscriber stamps onto each
/// announce's hop chain (the publisher no longer stamps itself), or the reserved
/// {@link UNKNOWN_HOP} when the responder has no identity to give. `active` is
/// the number of initial Announce messages that follow immediately.
/// {@link UNKNOWN_HOP} when the responder has no identity to give. Lite05/06
/// carry `active`, the initial start count; lite07 uses SYNCED instead.
export class AnnounceOk {
hop: Hop;
active: number;
Expand All @@ -543,27 +556,27 @@ export class AnnounceOk {
}
}

async #encode(w: Writer) {
async #encode(w: Writer, version: Version) {
await w.u62(this.hop);
await w.u53(this.active);
if (!hasSynced(version)) await w.u53(this.active);
}

static async #decode(r: Reader): Promise<AnnounceOk> {
static async #decode(r: Reader, version: Version): Promise<AnnounceOk> {
// The draft reserves 0 for "unknown": the responder was never assigned an id, or
// withholds it to obscure its routing. It names nobody, so callers must not stamp
// it onto a hop chain, but it is a legal message and not grounds to drop the stream.
const origin = HopSchema.parse(await r.u62());
const active = await r.u53();
const active = hasSynced(version) ? 0 : await r.u53();
return new AnnounceOk(origin, active);
}

async encode(w: Writer, version: Version): Promise<void> {
AnnounceOk.#guard(version);
return Message.encode(w, this.#encode.bind(this));
return Message.encode(w, (w) => this.#encode(w, version));
}

static async decode(r: Reader, version: Version): Promise<AnnounceOk> {
AnnounceOk.#guard(version);
return Message.decode(r, AnnounceOk.#decode);
return Message.decode(r, (r) => AnnounceOk.#decode(r, version));
}
}
69 changes: 68 additions & 1 deletion js/net/src/lite/publisher.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import * as Path from "../path.ts";
import { sendOrder } from "../priority.ts";
import { Reader, Stream, Writer } from "../stream.ts";
import { Timescale, Timestamp } from "../time.ts";
import { type AnnounceBroadcast, AnnounceRequest, decodeAnnounceBroadcast } from "./announce.ts";
import { type AnnounceBroadcast, AnnounceOk, AnnounceRequest, decodeAnnounceBroadcast } from "./announce.ts";
import { Fetch } from "./fetch.ts";
import { Group as GroupMessage } from "./group.ts";
import { Probe as ProbeMessage } from "./probe.ts";
Expand Down Expand Up @@ -2553,3 +2553,70 @@ for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) {
}
});
}

test("lite-07 SYNCED survives an initial removal and continuous earlier-prefix churn", async () => {
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
const origin = new OriginProducer();
const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
const removed = publish(origin, Path.from("z/removed"));
const retained = publish(origin, Path.from("z/retained"));
const entered = Promise.withResolvers<void>();
const release = Promise.withResolvers<void>();
let output!: ReadableStreamDefaultController<Uint8Array>;
const reader = new Reader(
new ReadableStream<Uint8Array>({
start(controller) {
output = controller;
},
}),
undefined,
version,
);
let writes = 0;
let churnEnabled = true;
const churn: BroadcastProducer[] = [];
const stream = new Stream({
version,
readable: new ReadableStream<Uint8Array>(),
writable: new WritableStream<Uint8Array>({
async write(chunk) {
if (writes++ === 0) {
entered.resolve();
await release.promise;
}
output.enqueue(new Uint8Array(chunk));
if (churnEnabled) {
churn.push(publish(origin, Path.from(`a/${writes}`)));
if (churn.length > 2) churn.shift()?.close();
}
},
}),
});
const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
try {
await entered.promise;
removed.close();
release.resolve();
expect((await AnnounceOk.decode(reader, version)).active).toBe(0);
const events: AnnounceBroadcast[] = [];
for (let i = 0; i < 3; i++) {
const event = await decodeAnnounceBroadcast(reader, version);
events.push(event);
if (event.status === "synced") break;
}
expect(events[0]).toMatchObject({ status: "active", suffix: "z/retained" });
expect(events.at(-1)).toEqual({ status: "synced" });
expect(events).not.toContainEqual(expect.objectContaining({ suffix: "z/removed" }));
} finally {
churnEnabled = false;
await publisher.drain();
await running;
publisher.close();
retained.close();
for (const broadcast of churn) broadcast.close();
origin.close();
pair.client.close();
pair.server.close();
}
});
Loading