Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/lib/js/net.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ publication.announce();
- **Answer claimed paths with `using`.** `requests(prefix)` claims a prefix and yields each requested path. Use `using prepared = request.prepare(epoch)`, fill its broadcast, then `prepared.publish()` to answer with a new publisher instance and keep the `Publication` it returns. Leaving the scope without publishing, including on a throw, refuses the request as unroutable. Explicit `prepared.close()` or closing its unpublished broadcast does the same; disposal after publishing leaves the returned publication and its fill broadcast live. Reject before preparing with `request.reject(err)`. A retained, unclosed prepared answer keeps the request pending; JavaScript garbage collection provides no prompt cleanup guarantee.
- **End content by finishing tracks.** A broadcast has no finished state. When the input ends, `finish()` each track, then `close()` it; closing a track without finishing aborts it. The broadcast stays announced and keeps serving them until you `close()` it. How long to keep it up for late readers is yours to decide.
- **Requests can pin an epoch.** `consumer.request(path, { epoch })` resolves only through a route announcing that [publisher epoch](/concept/moq-lite#publisher-epochs). The resolved broadcast's `epoch` names the route it came through, pinned or not.
- **Follow restarts.** `origin.follow(path)` follows whichever publisher instance serves the path: its `active` changes to a fresh broadcast on a restart, clears on an end, and resolves again on the next start, so read it reactively and close it when done. Instances are never stitched, so subscribe again to each new broadcast. Another route of the same epoch taking over (epochs travel only on `moq-lite-07-wip`) keeps `active` as is, and each track subscription resumes there from the first group it lacks. A refusal ends it. Pass `{ epoch }` to pin one instance, which it waits for again after an end. `request(path)` instead keeps its accepted instance and ends once that instance stops serving; while pending, it fails `Unroutable` once discovery of its path has synced without a route for it. To list what is live, iterate `announced(scope)`.
- **Follow restarts.** `origin.follow(path)` follows whichever publisher instance serves the path: its `active` changes to a fresh broadcast on a restart, clears on an end, and resolves again on the next start, so read it reactively and close it when done. Instances are never stitched, so subscribe again to each new broadcast. Another route of the same epoch taking over (epochs travel only on `moq-lite-07-wip`) keeps `active` as is, and each track subscription resumes there from the first group it lacks. A refusal ends it. Pass `{ epoch }` to pin one instance, which it waits for again after an end. `request(path)` instead keeps its accepted instance and ends once that instance stops serving; while pending, it fails `Unroutable` once discovery of its path has synced without a route for it. A moq-transport peer marks no end to its initial set unless both sides declare MoQ Active Count, so a request subscribes through it directly instead. To list what is live, iterate `announced(scope)`.
- **Discovery follows what you watch.** A `consume` origin asks the relay for announcements only under the prefixes its readers watch with `announced(scope)`, `request(path)`, or `follow(path)`, stops asking about a prefix 2s after nothing at or beneath it is watched, and asks again on reconnect. Every request and follower watches its path from creation until closed, so it outlives the listing that found it. A relay that refuses an announce request downgrades that session: requests subscribe blind through it, and followers keep waiting for a session that can discover.
- **Connections share by URL and versions.** `new Connection({ url })` pools and reconnects with backoff, which the elements use. An ordered `versions` list, spelled like `moq-lite-06` as in Rust, applies to both WebTransport and WebSocket; equivalent lists share, while a different order or an omitted list gets its own pool entry. Supplying your own transport options or origin selects a private loop. Receive estimates are on demand: observe `connection.probe` reactively or subscribe while you need them; `peek()` alone opens no PROBE stream.
- **GOAWAY moves the session.** The connection dials the replacement at once while the old session keeps serving its groups, up to `goaway.handover` (default 10s, or the relay's deadline when that is sooner). New requests keep opening on the old session until the replacement's route outranks it, which is a restart without an epoch: an unpinned announced request follows the replacement, while a blind or epoch-pinned request stays on its accepted instance. With an epoch, which travels only when `moq-lite-07-wip` is offered, the replacement serves the same broadcast, so open subscriptions move across without missing a group. An empty GOAWAY redials the same URL. A redirect stays on the same host unless `goaway.redirect` is `"follow"`, which also presents the tokens on the connection's `auth` handle to the new host.
Expand Down
1 change: 1 addition & 0 deletions js/net/bench/forward.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ class FakeSession {
consume: () => new BroadcastProducer().consume(),
goaway: new Once(),
exclusive: false,
provesAbsence: true,
auth: new AuthSession({ peerGrant: { publish: new Path.Patterns(), subscribe: new Path.Patterns() } }),
});
this.closed = new Promise((resolve) => {
Expand Down
41 changes: 39 additions & 2 deletions js/net/src/connection/forward.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,16 @@ class FakeSession {
closed: Promise<Error | null>;
#die!: (reason: Error | null) => void;

/** `exclusive` stands in for a peer that refuses overlapping requests. */
constructor({ exclusive = false } = {}) {
/**
* `exclusive` stands in for a peer that refuses overlapping requests, and `provesAbsence`
* false for one whose initial set has no end, as moq-transport's without MoQ Active Count.
*/
constructor({ exclusive = false, provesAbsence = true } = {}) {
registerWire(this, {
consume: (path) => this.consume(path),
goaway: new Once(),
exclusive,
provesAbsence,
auth: new AuthSession({ peerGrant: { publish: new Path.Patterns(), subscribe: new Path.Patterns() } }),
});
this.closed = new Promise((resolve) => {
Expand Down Expand Up @@ -764,3 +768,36 @@ test("a scoped session refuses blind requests outside its allowed paths", async
session.die();
origin.close();
});

test("a plain request answers blind through discovery that cannot prove absence", async () => {
const origin = new OriginProducer();
const session = new FakeSession({ provesAbsence: false });
forwardAnnounced(session.session, origin);
const path = Path.from("room");
const request = origin.request(path);
const follower = origin.follow(path);
await settle();
expect(session.open()).toEqual(["room/**"]);
session.stream("room/**").synced();
await settle();
expect(origin.discovery.peek()).toBe(true);
expect(request.closed.peek()).toBeUndefined();
const blind = request.active.peek();
expect(blind).toBeDefined();
expect(follower.active.peek()).toBeUndefined();
expect(session.consumes(path)).toBe(1);

// An announcement arriving later is a route, which a request prefers over a blind answer.
start(session.stream("room/**"), "room");
await settle();
expect(session.consumes(path)).toBe(2);
expect(request.active.peek()).toBeDefined();
expect(request.active.peek()).not.toBe(blind);
expect(follower.active.peek()).toBeDefined();
expect(request.closed.peek()).toBeUndefined();

request.close();
follower.close();
session.die();
origin.close();
});
4 changes: 2 additions & 2 deletions js/net/src/connection/forward.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ interface Reported {
*/
export function forwardAnnounced(conn: Established, origin: OriginProducer): void {
const originWire = wireOf(origin);
const exclusive = wireOf(conn).exclusive;
const { exclusive, provesAbsence } = wireOf(conn);
// Reassigned on a downgrade, so the origin stops counting this session as discovering.
// Called through a closure so the session's death always detaches the current one.
let detach = originWire.attach(true);
let detach = originWire.attach(true, provesAbsence);
let dead = false;
let blind = false;

Expand Down
2 changes: 2 additions & 0 deletions js/net/src/ietf/connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,8 @@ export class Connection implements Established {
// The drafts refuse overlapping SUBSCRIBE_NAMESPACEs (PREFIX_OVERLAP). A peer that
// declared MoQ Solicit is ours and holds both; any other may enforce it.
exclusive: solicit === undefined,
// moq-transport ends no initial set, so only MoQ Active Count says one is complete.
provesAbsence: activeCount,
auth: this.#auth,
});

Expand Down
147 changes: 147 additions & 0 deletions js/net/src/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import {
import * as Epoch from "./epoch.ts";
import { SessionCode, SessionError, StreamCode, StreamError, TooFarBehind } from "./error.ts";
import * as Ietf from "./ietf/index.ts";
import * as Message from "./ietf/message.ts";
import { Parameters } from "./ietf/parameters.ts";
import { Publisher as IetfPublisher } from "./ietf/publisher.ts";
import { RequestOk } from "./ietf/request.ts";
Expand Down Expand Up @@ -2017,6 +2018,152 @@ test("integration: ietf blind handle picks up a publisher that arrives late", as
clientOrigin.close();
});

/**
* Have every moq-transport peer accept SUBSCRIBE_NAMESPACE and announce nothing, the way a
* relay that serves by origin pull does, while still serving SUBSCRIBE. Without
* `activeCount` neither side declares MoQ Active Count, so nothing marks the initial set.
*/
function withoutAnnouncements(version: Ietf.Version, { activeCount = false } = {}) {
if (!activeCount) {
const set = Ietf.SetupOptions.prototype.setVarint;
spyOn(Ietf.SetupOptions.prototype, "setVarint").mockImplementation(function (
this: Ietf.SetupOptions,
id,
value,
) {
if (id !== Ietf.SetupOption.ActiveCount) set.call(this, id, value);
});
}
const counts = { answered: 0, subscribes: 0 };
spyOn(IetfPublisher.prototype, "runSubscribeNamespace").mockImplementation(async (_msg, stream) => {
const parameters = new Parameters();
if (activeCount) parameters.activeCount = 0;
await Message.write(stream.writer, RequestOk.id, new RequestOk({ parameters }), version);
counts.answered++;
await stream.reader.closed.catch(() => {});
});
const run = IetfPublisher.prototype.runSubscribe;
spyOn(IetfPublisher.prototype, "runSubscribe").mockImplementation(function (this: IetfPublisher, msg, stream) {
counts.subscribes++;
return run.call(this, msg, stream);
});
return counts;
}

const IETF_VERSIONS = [
[Ietf.ALPN.DRAFT_18, Ietf.Version.DRAFT_18],
[Ietf.ALPN.DRAFT_22, Ietf.Version.DRAFT_22],
] as const;

test.each(IETF_VERSIONS)(
"%s: a plain request subscribes through a peer that announces nothing",
async (alpn, version) => {
const counts = withoutAnnouncements(version);
const pair = createMockTransportPair(alpn);
const origin = new OriginProducer();
const clientOrigin = new OriginProducer();
const [client, server] = await Promise.all([
connect(url, { transport: pair.client, consume: clientOrigin }),
accept(pair.server, url, { publish: origin.consume() }),
]);
const producer = publish(origin, Path.from("pulled"));
const serving = (async () => {
for (;;) {
const req = await wireOf(producer).requested();
if (!req) break;
req.accept({ timescale: Timescale.MILLI }).writeFrame(textFrame("pulled"));
}
})();

const request = clientOrigin.request(Path.from("pulled"));
await until(() => counts.answered === 1);
await sleep(10);
// Accepted discovery that lists nothing proves nothing absent here.
expect(request.closed.peek()).toBeUndefined();
const active = await waitFor(request.active, (b) => b !== undefined);
expect(await active?.track("video").subscribe().ordered().readString()).toBe("pulled");
expect(counts.subscribes).toBe(1);

request.close();
producer.close();
await serving;
client.abort();
server.abort();
clientOrigin.close();
},
);

test.each(IETF_VERSIONS)("%s: a plain request ends its track with the peer's refusal", async (alpn, version) => {
const counts = withoutAnnouncements(version);
const pair = createMockTransportPair(alpn);
const clientOrigin = new OriginProducer();
const [client, server] = await Promise.all([
connect(url, { transport: pair.client, consume: clientOrigin }),
accept(pair.server, url, { publish: new OriginProducer().consume() }),
]);

const request = clientOrigin.request(Path.from("missing"));
await until(() => counts.answered === 1);
await until(() => request.active.peek() !== undefined);
const track = request.active.peek()?.track("video").subscribe();
const err = await withTimeout(Promise.resolve(track?.closed), Nano.fromMilli(Milli(1000)), "track never closed");
expect((err as Error).message).toContain("does_not_exist");
expect(counts.subscribes).toBe(1);

request.close();
client.abort();
server.abort();
clientOrigin.close();
});

test.each(IETF_VERSIONS)(
"%s: a follower never subscribes through a peer that announces nothing",
async (alpn, version) => {
const counts = withoutAnnouncements(version);
const pair = createMockTransportPair(alpn);
const origin = new OriginProducer();
const clientOrigin = new OriginProducer();
const [client, server] = await Promise.all([
connect(url, { transport: pair.client, consume: clientOrigin }),
accept(pair.server, url, { publish: origin.consume() }),
]);
const producer = publish(origin, Path.from("pulled"));

const follower = clientOrigin.follow(Path.from("pulled"));
await until(() => counts.answered === 1);
await sleep(10);
expect(follower.active.peek()).toBeUndefined();
expect(follower.closed.peek()).toBeUndefined();
expect(counts.subscribes).toBe(0);

follower.close();
producer.close();
client.abort();
server.abort();
clientOrigin.close();
},
);

test.each(IETF_VERSIONS)("%s: a counted empty initial set leaves a plain request unroutable", async (alpn, version) => {
const counts = withoutAnnouncements(version, { activeCount: true });
const pair = createMockTransportPair(alpn);
const clientOrigin = new OriginProducer();
const [client, server] = await Promise.all([
connect(url, { transport: pair.client, consume: clientOrigin }),
accept(pair.server, url, { publish: new OriginProducer().consume() }),
]);

const request = clientOrigin.request(Path.from("pulled"));
const err = await withTimeout(Promise.resolve(request.closed), Nano.fromMilli(Milli(1000)), "request never closed");
expect(err).toBeInstanceOf(StreamError);
expect((err as StreamError).code).toBe(StreamCode.Unroutable);
expect(counts).toEqual({ answered: 1, subscribes: 0 });

client.abort();
server.abort();
clientOrigin.close();
});

// ---------------------------------------------------------------------------
// Origin-fed sessions: the `consume` option end to end.
// ---------------------------------------------------------------------------
Expand Down
2 changes: 2 additions & 0 deletions js/net/src/lite/connection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,8 @@ export class Connection implements Established {
consume: (path) => this.#subscriber.consume(path),
goaway: this.#goaway,
exclusive: false,
// A moq-lite peer announces every broadcast it serves.
provesAbsence: true,
auth: this.#auth,
});

Expand Down
39 changes: 21 additions & 18 deletions js/net/src/origin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ class CoveringRoot {
* @internal
*/
export interface RequestSlot {
/** Open plain handles that can use a peer without discovery. */
/** Open plain handles that can use a peer whose discovery cannot prove them absent. */
blind: number;
answer?: broadcast.Consumer;
readonly handles: Set<Once<Error | null>>;
Expand Down Expand Up @@ -600,9 +600,9 @@ class OriginState {
// retain their old broadcast without participating in routing or same-epoch failover.
retained = new WeakSet<RequestSlot>();

// How many sessions are attached, and how many of those support broadcast discovery.
// What backs the public `discovery` getter.
sessions = new Signal({ total: 0, discovery: 0 });
// How many sessions are attached, how many of those support broadcast discovery, and how
// many of those can prove a path absent. The first two back the public `discovery` getter.
sessions = new Signal({ total: 0, discovery: 0, proven: 0 });

// The prefixes whose initial announcements a session is still delivering, counted. An
// announcement stream overlapping one has not synced yet.
Expand Down Expand Up @@ -659,10 +659,10 @@ class OriginState {
this.#unsettled.clear();
}

/** Whether every attached session discovers and none owes an initial set covering `path`. */
/** Whether every attached session proves absence and none owes an initial set covering `path`. */
synced(path: Path.Valid): boolean {
const { total, discovery } = this.sessions.peek();
if (total === 0 || discovery < total) return false;
const { total, proven } = this.sessions.peek();
if (total === 0 || proven < total) return false;
const snapshots = this.snapshots.peek();
for (const prefix of coveringPrefixes(path)) if (snapshots.has(prefix)) return false;
return true;
Expand Down Expand Up @@ -923,11 +923,11 @@ class OriginState {
}
}

/** Whether a plain request needs the non-discovery session's blind answer. */
/** Whether a plain request needs the blind answer of a session that cannot prove it absent. */
blind(slot: RequestSlot): boolean {
if (slot.blind === 0) return false;
const { total, discovery } = this.sessions.peek();
return discovery < total;
const { total, proven } = this.sessions.peek();
return proven < total;
}

/**
Expand Down Expand Up @@ -1197,7 +1197,7 @@ export class Producer implements Table {
});
},
accepts: (prefix) => this.#scope.overlaps(Path.join(this.#scope.root, prefix)),
attach: (discovery) => this.#attach(discovery),
attach: (discovery, provesAbsence = discovery) => this.#attach(discovery, discovery && provesAbsence),
snapshot: (prefix) =>
this.#borrowed?.closed ? () => {} : this.#hold(this.#state.addSnapshot(this.#scope.prefix(prefix))),
get requests() {
Expand Down Expand Up @@ -1426,18 +1426,19 @@ export class Producer implements Table {
*
* @internal
*/
#attach(discovery: boolean): Dispose {
#attach(discovery: boolean, proven: boolean): Dispose {
if (this.#borrowed?.closed) return () => {};
this.#sessions(1, discovery);
return this.#hold(() => this.#sessions(-1, discovery));
this.#sessions(1, discovery, proven);
return this.#hold(() => this.#sessions(-1, discovery, proven));
}

#sessions(delta: number, discovery: boolean): void {
this.#state.sessions.update(({ total, discovery: d }) => ({
#sessions(delta: number, discovery: boolean, proven: boolean): void {
this.#state.sessions.update(({ total, discovery: d, proven: p }) => ({
total: total + delta,
discovery: d + (discovery ? delta : 0),
proven: p + (proven ? delta : 0),
}));
// Losing a session without discovery can leave every pending request synced.
// Losing a session that cannot prove absence can leave every pending request synced.
this.#state.settle(Path.empty());
}

Expand Down Expand Up @@ -1748,7 +1749,9 @@ export class Consumer {
*
* The request owns discovery of its path until closed and retries route churn while
* pending. It fails `Unroutable` once every session's discovery has synced without a
* route for it, and can use a session without discovery instead. Once resolved it keeps
* route for it. A session whose discovery cannot prove that answers it blind instead: one
* without discovery, or a moq-transport peer, whose initial set has no end unless both
* sides declare MoQ Active Count. Once resolved it keeps
* that instance, ending when the instance stops serving. Use {@link follow} to move on
* to the next one.
*/
Expand Down
Loading