diff --git a/.claude/skills/verify-route/SKILL.md b/.claude/skills/verify-route/SKILL.md
index 2500c1baa5d..f1fa0bb834b 100644
--- a/.claude/skills/verify-route/SKILL.md
+++ b/.claude/skills/verify-route/SKILL.md
@@ -109,7 +109,7 @@ Follow Reading a failure in `rs/moq-net/verify/README.md`. Replay the path on th
## Coverage
-Sync commit: `cbb742cecae437ee94fbf25848521ac11b145fbc`, where [moq-dev/moq#8734](https://github.com/moq-dev/moq/pull/8734), which made doubt spare the node a route begins at, took in [the switch](https://github.com/moq-dev/moq/pull/8731). Run `git log cbb742cec.. -- rs/moq-net/src/route/path.rs rs/moq-net/src/route/reader.rs` to see what changed since. Prefix and reader indexes make the router's updates incremental; test-only full recomputation checks their result against the complete view. The model recomputes every prefix atomically and leaves that optimization out. `Rank` orders routes as `Offer::rank` does: drain, carried with each offer or known from a DRAIN (`Drains`), then the lowest link priority on the path (`Preferred`), then cost; `GoAway` marks a neighbour's session draining. `SelectKey` sets a route's down-only flag as `Router::usable_offer` does, and `Want0` ends a live down-only route toward an upstream link. `SrcKey` mirrors the router's preference for the best-ranked covering live path that does not name its receiver, leaving out down-only routes toward an upstream link and falling back to a held one, with the narrower scope breaking ties; `Select` plans live routes and their reflected sources before deciding holds; a held route keeps its last live path, so `SelectKey` holds under a next hop only on the path it was built on and `Want` ends a held stream whose source now has another path; `Advertise1` forwards a newer floor even when the claims did not change. The router keeps doubt until its one-second timer fires (`Lift::Timer`, `Doubt::DEFAULT`); the model's `Collect` leaves it for `Lift`. The comparison-only `Rule::EARLY` lifts doubt once no heard route names its node and is outside this timer-only model; the comparison-only `Rule::ORIGIN` lets doubt block a route that begins at the doubted node too, which `Flap.cfg` fails with `Doubted` defined as it was, and `Rule::SPARED` lets it block none, even a returning link. Follow Coverage in `rs/moq-net/verify/README.md` after a change to a covered function.
+Sync commit: `cbb742cecae437ee94fbf25848521ac11b145fbc`, where [moq-dev/moq#8734](https://github.com/moq-dev/moq/pull/8734), which made doubt spare the node a route begins at, took in [the switch](https://github.com/moq-dev/moq/pull/8731). Run `git log cbb742cec.. -- rs/moq-net/src/route/path.rs rs/moq-net/src/route/reader.rs` to see what changed since. Prefix and reader indexes make the router's updates incremental; test-only full recomputation checks their result against the complete view. The model recomputes every prefix atomically and leaves that optimization out. `Rank` orders routes as `Offer::rank` does: drain, carried with each offer or known from a DRAIN (`Drains`), then the lowest link priority on the path (`Preferred`), then cost; `GoAway` marks a neighbour's session draining. `SelectKey` sets a route's down-only flag as `Router::usable_offer` does, and `Want0` ends a live down-only route toward an upstream link. `SrcKey` mirrors the router's preference for the best-ranked covering live path that does not name its receiver, leaving out down-only routes toward an upstream link and falling back to a held one, with the narrower scope breaking ties; `Select` plans live routes and their reflected sources before deciding holds; a held route keeps its last live path, so `SelectKey` holds under a next hop only on the path it was built on and `Want` ends a held stream whose source now has another path; `Advertise1` forwards a newer floor even when the claims did not change. `Asks` keeps one edit in flight per neighbour, as `Router::ask` does, and the marker answering it asks again for what changed meanwhile, as `Router::edited` does. The router keeps doubt until its one-second timer fires (`Lift::Timer`, `Doubt::DEFAULT`); the model's `Collect` leaves it for `Lift`. The comparison-only `Rule::EARLY` lifts doubt once no heard route names its node and is outside this timer-only model; the comparison-only `Rule::ORIGIN` lets doubt block a route that begins at the doubted node too, which `Flap.cfg` fails with `Doubted` defined as it was, and `Rule::SPARED` lets it block none, even a returning link. Follow Coverage in `rs/moq-net/verify/README.md` after a change to a covered function.
| Code | Model |
| --- | --- |
@@ -131,4 +131,4 @@ Sync commit: `cbb742cecae437ee94fbf25848521ac11b145fbc`, where [moq-dev/moq#8734
SYNCED is left out: the router only records it per prefix (`Router::synced`, `Router::is_synced`), and nothing it decides reads it. So are scope regions: a relay asks for what it watches, and `interest.rs`'s `a_scope_with_several_regions_asks_for_each` and the sweeps cover a scope met with the interest.
-One publisher, so routes for several publishers are left out, and so are the copies and copy rank of cache-aware routing, which `rs/moq-net/src/route/tests/cache.rs` sweeps; the DEAD message's per-publisher batching reduces to one route per prefix. Two prefixes, one nested in the other, so an edit adds, removes, widens, or narrows one prefix at a time: no wildcards, swaps of several prefixes for one, stream cancellation, grants, qualified keys, or hidden claims. Leaves out two sessions between one pair of nodes at once, as a GOAWAY migration has: the model keeps one per pair, and among equal routes the router stays on the next hop's node and takes its latest session there (`a_later_session_to_the_next_hop_takes_over_among_equals`). Leaves out restarts, priorities other than 0 and 1, a priority that differs between a link's two ends, costs other than 1 and 2, and the hop limit. Down-only routes appear only where every relay's best route is also its only exportable one, so the model never shows a down-only best route hiding an exportable alternative, a cost the routes epic's contract accepts, and a relay that accepted a request from an upstream link never ends it when its route turns down-only. The hold deadline (`HOLD`), the stalled-writer close, and a scoped session's DEAD (`fail_heard`, which doubts and remembers nothing) are left out: a hold that ends early only lets go sooner, and the model's sessions are all unscoped. So are the node's per-stream keys: lite-07 files each announce stream as its own router key, whose one edit is the prefix the stream opened with (`route/node.rs`), where the model has one neighbour per session and edits it in place; [the route stream](/quest/m2/cluster-routes/route-stream.md) joins them. Messages come only from honest routers: validation of an unrelated DEAD or its far node, and empty or overlong replacement paths, is outside the model's input domain. Rust fixtures cover those guards and exportable covering-source selection, including keeping a covering hold when every live source is unexportable. Interest-edit fixtures also cover a cancelled stream retaining its pending child replies; the model has no stream cancellation. A claims message carries the sender's whole view for its prefix, equal to applying its changes on a FIFO stream. A route is sent on every stream whether or not the publisher has a claim there. The publisher ignores what it hears. `Overlap.cfg` also simulates five relays with every session up. No exhaustive pass has more than four relays or two session changes; `Sim`'s sweeps on the path rule (`sweep_handoff`, `sweep_scoped_handoff`, `sweep_scoped_meshes_show_what_the_rule_says`, and `compare.rs`'s churn) cover larger meshes, grants, qualified keys, interest edits, restarts, and churn.
+One publisher, so routes for several publishers are left out, and so are the copies and copy rank of cache-aware routing, which `rs/moq-net/src/route/tests/cache.rs` sweeps; the DEAD message's per-publisher batching reduces to one route per prefix. Two prefixes, one nested in the other, so an edit adds, removes, widens, or narrows one prefix at a time: no wildcards, swaps of several prefixes for one, stream cancellation, grants, qualified keys, or hidden claims. Leaves out two sessions between one pair of nodes at once, as a GOAWAY migration has: the model keeps one per pair, and among equal routes the router stays on the next hop's node and takes its latest session there (`a_later_session_to_the_next_hop_takes_over_among_equals`). Leaves out restarts, priorities other than 0 and 1, a priority that differs between a link's two ends, costs other than 1 and 2, and the hop limit. Down-only routes appear only where every relay's best route is also its only exportable one, so the model never shows a down-only best route hiding an exportable alternative, a cost the routes epic's contract accepts, and a relay that accepted a request from an upstream link never ends it when its route turns down-only. The hold deadline (`HOLD`), the stalled-writer close, and a scoped session's DEAD (`fail_heard`, which doubts and remembers nothing) are left out: a hold that ends early only lets go sooner, and the model's sessions are all unscoped. So are a session's mount keys: the node files each lite-07 session as one router key for both directions (`route/node.rs`), the model's one neighbour per session, edited in place, and a handle that reads through a mount answers under a further key of that session, which only sends. With no mounts, the model has no route its ROUTE_START marks `exact`, which only a mount at its mount point sends and which `rekey` keeps at its own prefix through a widening; `lite::rekey`'s and `lite::subscriber::routed`'s tests cover it. Messages come only from honest routers: validation of an unrelated DEAD or its far node, and empty or overlong replacement paths, is outside the model's input domain. Rust fixtures cover those guards and exportable covering-source selection, including keeping a covering hold when every live source is unexportable. Interest-edit fixtures also cover a cancelled stream retaining its pending child replies; the model has no stream cancellation. A claims message carries the sender's whole view for its prefix, equal to applying its changes on a FIFO stream. A route is sent on every stream whether or not the publisher has a claim there. The publisher ignores what it hears. `Overlap.cfg` also simulates five relays with every session up. No exhaustive pass has more than four relays or two session changes; `Sim`'s sweeps on the path rule (`sweep_handoff`, `sweep_scoped_handoff`, `sweep_scoped_meshes_show_what_the_rule_says`, and `compare.rs`'s churn) cover larger meshes, grants, qualified keys, interest edits, restarts, and churn.
diff --git a/doc/.vitepress/drafts.test.ts b/doc/.vitepress/drafts.test.ts
index 32f7fc3500f..8aa1912c8c1 100644
--- a/doc/.vitepress/drafts.test.ts
+++ b/doc/.vitepress/drafts.test.ts
@@ -128,30 +128,20 @@ describe("tables render as tables", () => {
const rendered = html(page as DraftPage);
- // The Bidirectional Streams table: one header, nine body rows, each
+ // The Bidirectional Streams table: one header, eight body rows, each
// separated by a kramdown rule in the source. Cells carry the source's
// alignment, so match around the style attribute.
const header = /
`));
}
- // One header row plus nine body rows, in a single table.
+ // One header row plus eight body rows, in a single table.
const start = rendered.lastIndexOf("
/g) ?? []).toHaveLength(9);
});
});
diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md
index 48a1e21e76f..1ec3f00120e 100644
--- a/drafts/draft-lcurley-moq-lite.md
+++ b/drafts/draft-lcurley-moq-lite.md
@@ -230,8 +230,9 @@ See [Transports](#transports) for the bindings.
How the underlying connection is authenticated is out-of-scope for this draft.
-A subscriber that holds an Auth Stream and an Announce Stream needs credit for at least 4 concurrent bidirectional streams to establish a subscription.
-AUTH and ANNOUNCE occupy two streams for the session lifetime, and TRACK is briefly held alongside SUBSCRIBE, requiring two more.
+A subscriber that holds an Auth Stream needs credit for at least 3 concurrent bidirectional streams to establish a subscription.
+AUTH occupies one stream for the session lifetime, and TRACK is briefly held alongside SUBSCRIBE, requiring two more.
+The Setup and Announce Streams need credit for 2 unidirectional streams beside the Group Streams.
## Transports {#transports}
moq-lite defines five transport bindings.
@@ -370,41 +371,26 @@ A relay terminating a stream because the upstream session serving it ended SHOUL
# Streams
moq-lite uses a bidirectional stream for each transaction.
If the stream is closed, potentially with an error, the transaction is terminated.
+Discovery and routing instead share one unidirectional [Announce Stream](#announce) per endpoint.
-## Bidirectional Streams
-Bidirectional streams are used for control streams.
-There's a 1-byte STREAM_TYPE at the beginning of each stream.
-
-|---------|--------------|-------------|
-| ID | Stream | Creator |
-|--------:|:-------------|:------------|
-| 0x1 | Announce | Subscriber |
-| ------- | ------------ | ----------- |
-| 0x2 | Subscribe | Subscriber |
-| ------- | ------------- | ---------- |
-| 0x3 | Fetch | Subscriber |
-| ------- | ------------- | ---------- |
-| 0x4 | Probe | Subscriber |
-| ------- | ------------- | ----------- |
-| 0x5 | Goaway | Either |
-| ------- | ------------- | ----------- |
-| 0x6 | Track | Subscriber |
-| ------- | ------------- | ----------- |
-| 0x7 | Auth | Either |
-| ------- | ------------- | ----------- |
-| 0x8 | Tracks | Subscriber |
-| ------- | ------------- | ----------- |
-| 0x9 | Datagrams | Subscriber |
-| ------- | ------------- | ----------- |
-
-### Announce
-A subscriber can open an Announce Stream to discover claims matching a prefix.
+## Announce Stream {#announce}
+Each endpoint MAY open one unidirectional Announce Stream (0x3), on which it sends everything it says about discovery and routing, in order: as a subscriber, the prefixes it wants claims for, and as a publisher, the routes and claims answering the prefixes its peer wants.
A claim is an announcement that paths under its prefix can be served by one origin node; it claims capability, not inventory, so it never asserts that any specific broadcast exists.
A route is how the publisher reaches an origin node for the paths under a prefix, and every claim on the stream travels under a route to its origin covering it (see [Routing](#routing)).
-The subscriber creates the stream with an ANNOUNCE_REQUEST message.
-The publisher replies with a single ANNOUNCE_OK message followed by the matching routes and claims and any future changes:
+An endpoint that opens the stream keeps it for the session, so no message it sends about routing overtakes another.
+An endpoint MUST close the session with a PROTOCOL_VIOLATION if its peer opens a second Announce Stream or resets one.
+An endpoint finishes its stream only when it is leaving the session, after a ROUTE_END for each route it sent.
+Its peer keeps answering the prefixes the endpoint asked for, which it can no longer edit.
+
+A subscriber edits the set of prefixes it wants claims for with ANNOUNCE_REQUEST messages.
+An edit too large for one message continues over several: every ANNOUNCE_REQUEST of an edit but the last sets `More`, and nothing else comes between them on the stream.
+The set starts empty and MAY be empty, no prefix in it is at or below another, and encoded as the Adds of one ANNOUNCE_REQUEST it fits the [Message Length](#message-length) limit.
+A subscriber starts an edit only once the publisher has answered its last.
+
+A publisher answers each edit with one ANNOUNCE_OK, after the edit's last ANNOUNCE_REQUEST, where the edit takes effect on its stream (see [Edits](#edits)), and sends the matching routes and claims and any future changes:
+- ANNOUNCE_OK: the peer's last edit takes effect here.
- ROUTE_START: the publisher reaches an origin node for a prefix.
- ROUTE_UPDATE: a route's path, drain, priority, metric, or down-only bit changed.
- ROUTE_HOLD: the publisher has no usable route but holds it while it recovers.
@@ -415,75 +401,106 @@ The publisher replies with a single ANNOUNCE_OK message followed by the matching
- ANNOUNCE_UPDATE: a claim's cost changed.
- ANNOUNCE_RESTART: a claim now serves another Instance, or a higher Epoch of the same one.
- ANNOUNCE_FLOOR: a batch of a route's claims is complete.
-- SYNCED: the initial set is complete.
+- SYNCED: the initial set of one prefix is complete.
-The subscriber sends nothing on the stream after its ANNOUNCE_REQUEST.
-
-The publisher MUST send [SYNCED](#synced) exactly once, after the initial set.
-The subscriber MAY batch messages until SYNCED; subsequent ones are live updates.
+The publisher MUST send [SYNCED](#synced) exactly once for each prefix an edit adds, after the ANNOUNCE_OK answering it and the prefix's initial set, unless a later edit removes the prefix first.
+The subscriber MAY batch a prefix's messages until its SYNCED; subsequent ones are live updates.
The initial set is relative to this hop: a relay answers with what it holds without waiting for upstreams.
The publisher MAY coalesce changes to a claim it has not yet sent, and sends no more claims before SYNCED than the initial set holds.
A retraction of a claim or route already sent MAY precede SYNCED.
+A prefix outside the subscriber's [scope](#auth-stream) lists nothing and is SYNCED at once.
-Each ROUTE_START implicitly assigns the next Route ID on the stream and each ANNOUNCE_START the next Announce ID: two counters starting at 0 that increment by 1 per assignment.
-No id appears on the wire where it is assigned; both endpoints derive it from the message order on the (reliable, ordered) stream, and other messages reference it instead of repeating the origin node and the route's prefix, or the claim's prefix.
+Each Add of an ANNOUNCE_REQUEST implicitly assigns the next Prefix ID on its sender's stream, each ROUTE_START the next Route ID, and each ANNOUNCE_START the next Announce ID: three counters per stream starting at 0 that increment by 1 per assignment.
+No id appears on the wire where it is assigned; both endpoints derive it from the message order on the (reliable, ordered) stream, and other messages reference it instead of repeating the prefix, the origin node and the route's prefix, or the claim's prefix.
+A ROUTE_START and a SYNCED name a Prefix ID the peer assigned; every other id a message names is its sender's.
Ids are never reused within a stream.
-A ROUTE_END retires its Route ID and the Announce ID of every claim under the route; an ANNOUNCE_END retires one Announce ID.
+A ROUTE_END retires its Route ID and the Announce ID of every claim under the route; an ANNOUNCE_END retires one Announce ID, and an edit the Prefix IDs it removes.
A stream has at most one live route per origin node and prefix, and each route at most one live claim per prefix, at or below the route's prefix.
Another origin claiming the same prefix is another claim, under its own route.
-An ANNOUNCE_UPDATE or ANNOUNCE_RESTART atomically replaces a claim while keeping its id live; a claim never moves to another route.
+An ANNOUNCE_UPDATE or ANNOUNCE_RESTART atomically replaces a claim while keeping its id live; a claim moves to another route only at an [edit](#edits).
+
+An endpoint MUST close the session with a PROTOCOL_VIOLATION if it receives:
+
+- a message referencing a Prefix ID, Route ID, or Announce ID that was never assigned or already retired;
+- an edit before its last was answered, any other message between the ANNOUNCE_REQUESTs of one edit, or an edit that leaves a prefix of the set at or below another or the set past the limit;
+- an ANNOUNCE_OK with no edit unanswered, or between a route's claim message and the ANNOUNCE_FLOOR closing its batch;
+- a ROUTE_START for an origin node and prefix that already have a live route on the stream;
+- an ANNOUNCE_START for a prefix its route already claims or outside the route's prefix; or
+- a second SYNCED for a prefix, or one before the ANNOUNCE_OK answering the edit that added it.
-The subscriber MUST close the session with a PROTOCOL_VIOLATION if it receives a message referencing a Route ID or Announce ID that was never assigned or already retired, a ROUTE_START for an origin node and prefix that already have a live route on the stream, an ANNOUNCE_START for a prefix its route already claims or outside the route's prefix, or any message before ANNOUNCE_OK.
-When the stream is closed, the subscriber MUST assume that every route and claim on it is gone.
+When the session closes, an endpoint MUST assume that every route and claim on the peer's stream is gone.
A broadcast path has at most 32 segments and at most 4,096 bytes; a receiver MUST close the session with a PROTOCOL_VIOLATION on a longer one, except in a claim, which it refuses instead (see [Refused Content](#refused-content)), including one resolved by [Compression](#announce-compression).
A claim covers a path when its prefix is a leading run of the path's segments; matching is per path segment, so a prefix never matches half a segment, and equality is byte-by-byte within each segment.
-A publisher answering a request stream presents, per route, each claim inside the requested prefix and only the most specific claim at or above it, at the requested prefix itself (an empty suffix), which is exactly the covered set the subscriber may see.
-A claim presented above its own prefix carries how many segments broader it is than the requested prefix, its Above, so the subscriber can tell it from a claim exactly at the prefix.
-When that most specific covering claim changes, such as when it is withdrawn and a broader one still covers the request, the publisher moves the presented claim to the new one with an ANNOUNCE_UPDATE or ANNOUNCE_RESTART, and ends it only when no claim of the route covers the requested prefix.
-There MAY be multiple Announce Streams, potentially containing overlapping prefixes, that get their own ANNOUNCE_OK and messages.
-
-#### Floors {#floors}
+A publisher presents, per route, each claim inside the requested prefix the route lies in and only the most specific claim at or above it, at that prefix itself, which is exactly the covered set the subscriber may see.
+A claim presented above its own prefix carries how many segments broader it is than the path it is presented at, its Above, so the subscriber can tell it from a claim exactly at that path.
+Every path on the stream is relative to the session, whichever requested prefix it lies in, so carrying a route over to another prefix (see [Edits](#edits)) changes no claim's path.
+When that most specific covering claim changes, such as when it is withdrawn and a broader one still covers the prefix, the publisher moves the presented claim to the new one with an ANNOUNCE_UPDATE or ANNOUNCE_RESTART, and ends it only when no claim of the route covers the prefix.
+
+### Edits {#edits}
+An edit takes effect on the publisher's stream at the ANNOUNCE_OK answering it.
+Until then the publisher's messages answer the set before it, so they may still name a prefix the edit removes, and never after.
+
+At the ANNOUNCE_OK both endpoints carry the stream's live routes over to the new set alike, so the publisher sends nothing again that both sets cover.
+Each route lies in one prefix of the old set, and in Route ID order:
+
+- It stays when the new set covers its prefix, and the old prefix it lies in is in the new set, the route lies strictly below that old prefix, or its ROUTE_START set `Exact`.
+- Otherwise, when an added prefix is above the old prefix it lies in, it widens: it becomes its origin node's route at that added prefix, without the claim at its own prefix, and certifies nothing until its next ANNOUNCE_FLOOR.
+- Otherwise, when it lies at the old prefix itself, it narrows: for each added prefix below it, in the order the ANNOUNCE_REQUEST added them, it becomes a route of its origin node at that prefix, with its claims strictly below that prefix and, at that prefix itself, its most specific claim at or above it. Each keeps its last floor.
+- Otherwise it ends, as on ROUTE_END.
+
+A stream still has one route per origin node and prefix.
+Of the routes that would share one, the one that remains is the first of: one that is not held, one whose Path does not name the Node ID the subscriber's SETUP declared, the best ranked (see [Routing](#routing)), one that stayed, and the lowest Route ID.
+A widened route that remains takes the claims of the widened routes that do not, and every other route that does not remain ends.
+
+A route that stays or widens keeps its Route ID, and its claims keep their Announce IDs.
+A route that stays keeps its `Exact`, and a widened or narrowed route carries on as one whose `Exact` is 0.
+Each narrowed route that remains is assigned the next Route ID, in the order above; its claims keep their Announce IDs, except that its claim at its prefix is assigned the next Announce ID when the claim it comes from lies above it, and that claim's Above grows by the segments its prefix lies below the claim it comes from.
+Whether a Route Suffix, a Route Offer, or a claim's path is refused (see [Refused Content](#refused-content)) depends only on what the publisher sent and the requested prefixes, so both endpoints carry the routes over alike and assign the same Route IDs and Announce IDs: a route whose Route Suffix is refused is carried as one at its requested prefix, a refused Route Offer ranks after every other, and a claim whose path is refused is not among its route's claims.
+A path the subscriber cannot hold for a limit of its own, such as one it joins beneath a local root, is carried as the stream names it.
+The publisher then sends whatever the carried routes lack, and an ANNOUNCE_FLOOR for each widened route.
+
+### Floors {#floors}
Each origin node numbers the changes to its own claims with a Revision, starting at 0 for each Node ID.
A stream carries a route's claims in batches: the ANNOUNCE messages under the route since its last ANNOUNCE_FLOOR, closed by the next one.
-The publisher sends a ROUTE_START before any claim under it, and SYNCED only once every route it sent has a floor.
+The publisher sends a ROUTE_START before any claim under it, and a prefix's SYNCED only once every route it sent in that prefix has a floor.
A floor names a Revision and certifies that the stream's claims for the route, across the route's whole prefix, are the origin's claims at that Revision as the request presents them (see [Announce](#announce)).
-It certifies only what its own request shows: a stream whose ANNOUNCE_REQUEST did not set `Hidden` certifies nothing about hidden claims.
+It certifies only what its requested prefix shows: one added without `Hidden` certifies nothing about hidden claims.
A publisher that does not know the route's claims across its whole prefix at one Revision, such as a relay that knows them only beneath part of it, still closes each batch, with a floor that certifies nothing.
-A subscriber that may receive claims only beneath one prefix, because its [scope](#auth-stream) or the publisher's confines the stream there, SHOULD request that prefix rather than one above it, so the publisher can certify the request. When that prefix changes, it SHOULD open a request for the new one and end the old one once the new one is SYNCED.
+A subscriber that may receive claims only beneath some prefixes, because its [scope](#auth-stream) or the publisher's confines it there, SHOULD request those prefixes rather than one above them, so the publisher can certify each, and edits its set when they change.
The subscriber applies a batch whole, once its floor arrives.
A receiver keeps one view of an origin's claims for each prefix it holds a route to the origin at: the claims of the floor with the highest Revision that a route it believes, at that prefix or one covering it, gave it.
-It believes a route while the route is usable, its Path does not name the receiver, and its stream answers the receiver's current request; a held route, or one the receiver refused, certifies nothing.
+It believes a route while the route is usable, its Path does not name the receiver, and the receiver wants the prefix it lies in, with no edit unanswered that removes it; a held route, or one the receiver refused, certifies nothing.
An older floor never revives a claim whose withdrawal a newer one delivered.
While no route it believes has given a floor that certifies the prefix, the view holds the claims its selected route carries there, uncertified: a floor that certifies nothing proves no claim absent.
The view stays whole while the receiver holds a route covering the prefix, usable or held, and the receiver shows each claim from the view at the narrowest prefix covering it, available while a usable route covers it.
Once the receiver holds no route covering the prefix, the view's claims go offline, which is not an ANNOUNCE_END.
-#### Compression {#announce-compression}
+### Compression {#announce-compression}
An ANNOUNCE_START MAY copy the head of its path from a live claim on the same stream, its base.
`Path Base` names it by distance: the base's Announce ID is the next unassigned Announce ID minus `Path Base`, so 1 names the latest ANNOUNCE_START, and 0 names no base.
The receiver resolves the base against the claims live when the message arrives, and the result is identical to the literal encoding: a base ending later changes nothing.
-The base need not share the new claim's route.
+The base need not share the new claim's route, nor the requested prefix that route lies in.
A publisher MAY always send a `Path Base` of 0.
The subscriber MUST close the session with a PROTOCOL_VIOLATION if a non-zero `Path Base` names an Announce ID that was never assigned or already retired, or a `Path Keep` is non-zero with a `Path Base` of 0 or exceeds the base's segments.
A resolved path that breaks its own rules, or a base whose path the subscriber refused, refuses the claim (see [Refused Content](#refused-content)).
-#### Refused Content {#refused-content}
+### Refused Content {#refused-content}
Route messages carry content the publisher may only have forwarded: a route's Node ID, Route Suffix, and [Route Offer](#route-offer), and a claim's path, Instance, and Lite-06 Chain.
A newer version or release may send such content that an older receiver cannot use, so a receiver that refuses it withdraws only the route or claim it describes and keeps the session, as BGP treats a malformed attribute as a withdrawal ({{?RFC7606}}).
This specification marks each such value "refused"; the receiver skips the rest of the message's body, and SHOULD log and count each refusal so a refused route is never silently missing.
-- A refused ROUTE_START still assigns its Route ID, and a refused ROUTE_UPDATE, or a refused Route Offer among a [DEAD](#dead)'s moved routes, keeps it; each leaves the receiver no route, so its claims go offline as on ROUTE_END. The messages naming the Route ID stay valid, and a later ROUTE_UPDATE the receiver can use makes the route usable again, with the claims its last floor certified. A route to Node ID 0, or whose Route Suffix was refused, stays refused. A refused Route Offer's Path still counts, so a later DEAD naming a hop on it applies to the route as to any other.
+- A refused ROUTE_START still assigns its Route ID, and a refused ROUTE_UPDATE, one following a [DEAD](#dead) included, keeps it; each leaves the receiver no route, so its claims go offline as on ROUTE_END. The messages naming the Route ID stay valid, and a later ROUTE_UPDATE the receiver can use makes the route usable again, with the claims its last floor certified. A route to Node ID 0, or whose Route Suffix was refused, stays refused. A refused Route Offer's Path still counts, so a later DEAD naming a hop on it applies to the route as to any other.
- A refused ANNOUNCE_START still assigns its Announce ID, and a refused ANNOUNCE_UPDATE or ANNOUNCE_RESTART keeps it; the claim goes offline at its route's next floor, which is not an ANNOUNCE_END. A claim whose path was refused stays refused; any other comes back with a later ANNOUNCE_RESTART the receiver can use. Its origin withdrew nothing, so while a route has a live claim the receiver refused, the receiver takes the route's floors as certifying nothing, and a relay forwards no withdrawal for the claim.
Only the publisher's own framing and stream state close the session: a message that does not decode within its Message Length, and a message naming an id that was never assigned or already retired, or breaking the stream's order, as above.
A relay never forwards content a peer would refuse for the relay's own part in it: a route that arrives with 32 hops is unusable, so the relay's own hop never takes one past 32.
-#### Hidden Paths {#hidden}
+### Hidden Paths {#hidden}
A claim is hidden from a request when a segment of its path below the requested prefix starts with `.` (0x2E).
A segment inside the prefix never hides anything, so a request that names the hidden segment itself (`.stats`) lists what is under it, and a claim at or above the prefix has no segment below it.
Only the first byte counts: `catalog.v2` is not hidden.
@@ -492,7 +509,7 @@ A publisher SHOULD NOT announce a hidden claim unless the ANNOUNCE_REQUEST set `
Hiding is a convenience for discovery, not access control: a publisher MAY treat a subscriber it trusts, such as another relay in its own cluster, as opted in, and MUST authorize hidden paths like any other.
SUBSCRIBE, FETCH, and TRACK resolve a hidden path exactly as any other.
-#### Routing {#routing}
+### Routing {#routing}
Each node selects, per origin node and prefix, one route by its path: the nodes it crossed, each with the session it was learned on.
A node never uses a route whose path names it, and a node that loses a session names that hop to its neighbours, so every route through it is dropped wherever it went.
There is no distance, sequence number, route request, or echo.
@@ -505,7 +522,7 @@ A relay MUST ignore a qualified Node ID received from such a peer, and claims un
A Node ID is otherwise declared, not proven.
A route covers a prefix: the one its receiver requested, or a narrower one that a node on its path requested.
-A stream answering a request carries the sender's route to each origin node at every prefix inside the request, under that prefix, and at the requested prefix itself the best-ranked usable route covering it.
+For each prefix its peer requested, a publisher's stream carries its route to each origin node at every prefix inside it, under that prefix, and at the requested prefix itself the best-ranked usable route covering it.
A node keeps, ranks, and selects its routes per origin node and prefix.
An origin advertises its own route with a Metric of 0.
@@ -530,8 +547,8 @@ A route is unusable while its Path names the node, it is held, a hop on its Path
A node MAY mark a session upstream.
A route is down-only when it arrived on an upstream session or with its down-only flag set; a node sends a down-only route with the flag set, and never on an upstream session.
-A node sends its route to an origin node at a prefix on an Announce Stream it answers only while the route has a claim the stream's request presents (see [Announce](#announce)): it sends ROUTE_START before the first such claim, and once the last one is withdrawn, the batch carrying that withdrawal, closed by its floor (which certifies the withdrawal where the node can certify the request), then ROUTE_END.
-A subscriber therefore holds no route for an origin node its request does not meet.
+A node sends a peer its route to an origin node at a prefix only while the route has a claim the peer's requested prefix presents (see [Announce](#announce)): it sends ROUTE_START before the first such claim, and once the last one is withdrawn, the batch carrying that withdrawal, closed by its floor (which certifies the withdrawal where the node can certify the request), then ROUTE_END.
+A subscriber therefore holds no route for an origin node its requests do not meet.
A node that does not forward, such as a client, sends only its own route.
An origin never advertises a Revision of 2^64-1: a change that would reach it instead rotates its Node ID, ending the old route everywhere and announcing its claims again under the new one.
@@ -546,11 +563,10 @@ A receiver of an ANNOUNCE_RESTART (or of the ANNOUNCE_END and ANNOUNCE_START sta
A version before this one carries no Instance, so a receiver of such a session mints one for each ANNOUNCE_START it receives, with Epoch 0, and forwards it as the claim's Instance with `Minted` set, so a reader can tell it names an announcement rather than the publisher's stream.
A minted Instance is new for every announcement, so a publisher reconnecting or re-announcing through such a hop is another stream.
-#### Dead Hops {#dead-hops}
+### Dead Hops {#dead-hops}
A node that loses a session names the hop it lost: its own Hop ID and that session's Session Number, with the Hop ID of the node at the far end.
It sends one [DEAD](#dead) for it to each neighbour it sent a route whose Path holds the hop, covering every origin node, and each neighbour passes the hop on in the same way, once per neighbour while it has sent that neighbour a route through it.
-The DEAD lists the routes on the stream that moved to another Path, each with its new Route Offer.
-A sender whose moved routes do not fit one message lists them over consecutive DEADs naming the same hop, each but the last with `More` set, and the receiver applies them as one DEAD listing all of them once the last arrives.
+The DEAD counts the routes on the stream that moved to another Path, and a ROUTE_UPDATE for each follows it with its new Route Offer; the receiver applies the DEAD and those updates as one.
The sender's other routes on the stream through the hop end if `Gone` is set.
Otherwise each is held, except one the sender ends with a ROUTE_END after the DEAD, and one the sender learned from the receiver itself, which the sender holds only by a ROUTE_HOLD after the DEAD: until then the receiver waits for its answer (see [Holds](#holds)).
@@ -573,7 +589,7 @@ It sends a DEAD naming the draining node to each neighbour it sent a route throu
A receiver applies it only for a node on a route that peer sent it, ranks every route through that node draining, and doubts the node in the same way.
A draining route stays usable.
-#### Holds {#holds}
+### Holds {#holds}
A node with no usable route to an origin node at a prefix holds the route, unavailable, while it may recover, and sends ROUTE_HOLD wherever it sent the route:
- while the neighbour it learned the route from holds it, on the Path this node's route extended, or has yet to say what became of it after a DEAD;
@@ -586,10 +602,10 @@ A hold protects a claim's presence, not loop freedom, so ending it early is safe
A held route's claims stay shown but unavailable until a ROUTE_UPDATE makes it usable again.
Otherwise the node lets go, sending ROUTE_END wherever it sent the route, and its claims go offline.
Neither proves that the origin withdrew its claims.
-A node closes the session when an Announce Stream it answers has made no progress for 10 seconds with messages left to send, so a peer that stops reading never holds a route.
-It MAY also close it when the stream falls behind repeated failures of the routes it carries, such as when DEADs it has yet to send outnumber the hops on those routes, so a peer that reads too slowly never holds every failure.
+A node closes the session when its Announce Stream has made no progress for 10 seconds with messages left to send, so a peer that stops reading never holds a route.
+It MAY also close it when the stream falls behind repeated failures of the routes it carries, such as when DEADs and ANNOUNCE_OKs it has yet to send outnumber the hops on those routes, so a peer that reads too slowly never holds every failure.
-#### Resolution {#resolution}
+### Resolution {#resolution}
A SUBSCRIBE, FETCH, or TRACK request naming no origin node names a path, and the receiver resolves it against the claims covering that path whose route it selected and is usable.
Only the most specific covering claims are consulted: those with the longest prefix, which for a claim shown below its own prefix is the path it is shown at less its Above (see [ANNOUNCE_START](#announce-start)).
@@ -611,6 +627,30 @@ An advertiser that will not serve a resolved request resets the request stream w
Every refusal is terminal and propagates without re-resolution, so probing unserved paths costs one round trip per path.
An advertiser signals capacity through its claim alone, withdrawing or re-pricing it before it runs out.
+## Bidirectional Streams
+Bidirectional streams are used for control streams.
+There's a 1-byte STREAM_TYPE at the beginning of each stream.
+
+|---------|--------------|-------------|
+| ID | Stream | Creator |
+|--------:|:-------------|:------------|
+| 0x2 | Subscribe | Subscriber |
+| ------- | ------------- | ---------- |
+| 0x3 | Fetch | Subscriber |
+| ------- | ------------- | ---------- |
+| 0x4 | Probe | Subscriber |
+| ------- | ------------- | ----------- |
+| 0x5 | Goaway | Either |
+| ------- | ------------- | ----------- |
+| 0x6 | Track | Subscriber |
+| ------- | ------------- | ----------- |
+| 0x7 | Auth | Either |
+| ------- | ------------- | ----------- |
+| 0x8 | Tracks | Subscriber |
+| ------- | ------------- | ----------- |
+| 0x9 | Datagrams | Subscriber |
+| ------- | ------------- | ----------- |
+
### Subscribe
A subscriber opens Subscribe Streams to request a Track.
@@ -858,7 +898,7 @@ An expired group SHOULD be reset at the QUIC level to avoid consuming flow contr
A publisher MUST NOT finish (FIN) the Group Stream of an expired group that has not finished at this hop, even after sending every frame it holds; it resets with OLD instead, since a FIN claims the group is whole.
## Unidirectional Streams
-Unidirectional streams are used for data transmission.
+Unidirectional streams are used for data transmission, the session's setup, and discovery and routing.
|--------|----------|-------------|
| ID | Stream | Creator |
@@ -869,6 +909,8 @@ Unidirectional streams are used for data transmission.
| ------ | -------- | ----------- |
| 0x2 | Padding | Either |
| ------ | -------- | ----------- |
+| 0x3 | Announce | Either |
+| ------ | -------- | ----------- |
### Setup {#setup-stream}
Each endpoint MUST open a Setup Stream (0x1) at the start of the session to advertise the optional capabilities and extensions it supports.
@@ -883,6 +925,9 @@ The receiver MUST read and discard those bytes incrementally, returning stream a
Receiving padding does not change session or subscription state.
This allows bandwidth probes to measure delivery without requiring a media subscription.
+### Announce {#announce-stream}
+Each endpoint MAY open one Announce Stream (0x3) for the session; see [Announce Stream](#announce).
+
### Group
A publisher creates Group Streams in response to a Subscribe Stream.
@@ -970,9 +1015,9 @@ moq-lite-07 uses the leading-ones encoding of [moqt]: the number of leading 1 bi
Most messages are prefixed with a variable-length integer indicating the number of bytes in the message payload that follows.
This length field does not include the length of the varint length itself.
-A message on the Setup Stream or any bidirectional stream that fails to decode, such as one whose fields disagree with its Message Length or carry a value this specification forbids, MUST close the session with a PROTOCOL_VIOLATION, except content a route message forwards, which the receiver refuses (see [Refused Content](#refused-content)).
+A message on the Setup Stream, an Announce Stream, or any bidirectional stream that fails to decode, such as one whose fields disagree with its Message Length or carry a value this specification forbids, MUST close the session with a PROTOCOL_VIOLATION, except content a route message forwards, which the receiver refuses (see [Refused Content](#refused-content)).
Resetting only the stream would let the peer repeat the violation on the next one.
-A stream that ends or is reset before its message completes only terminates that stream, except on the Setup and Auth Streams.
+A stream that ends or is reset before its message completes only terminates that stream, except on the Setup, Announce, and Auth Streams.
A GROUP or FRAME that fails to decode resets only its Group or Fetch Stream, and a STREAM_TYPE the negotiated version does not define resets only its stream (see [STREAM_TYPE](#stream_type)).
Except in FRAME, a Message Length over 65,535 bytes is unexpected: a sender MUST NOT exceed it, and a receiver MAY reject it based on the length prefix alone.
The version and extensions should be used to support new fields, not the message length.
@@ -1157,16 +1202,42 @@ It is a temporary extension for relays that also peer over moq-lite-06, and the
The Parameter Value MUST be empty; a receiver MUST close the session with a PROTOCOL_VIOLATION otherwise.
## ANNOUNCE_REQUEST {#announce-request}
-A subscriber sends an ANNOUNCE_REQUEST message to indicate it wants to receive the claims, and the routes they travel under, for any broadcasts with a path that starts with the requested prefix.
+A subscriber sends an ANNOUNCE_REQUEST message on its Announce Stream to edit the set of prefixes it wants claims for, and the routes they travel under: every broadcast with a path that starts with a prefix of the set.
~~~
ANNOUNCE_REQUEST Message {
+ Type (i) = 0x10
Message Length (i)
+ More (8),
+ Remove Count (i),
+ Prefix ID (i) ...,
+ Add Count (i),
+ Add (..) ...,
+}
+
+Add {
Broadcast Path Prefix (s),
Hidden (8),
}
~~~
+**Type**:
+Set to 0x10 to indicate an ANNOUNCE_REQUEST message.
+
+**More**:
+1 when the next message on the stream is an ANNOUNCE_REQUEST continuing this edit, 0 on an edit's last.
+Any other value is a PROTOCOL_VIOLATION.
+The publisher applies an edit's ANNOUNCE_REQUESTs together, at its last: each removes a prefix the set held before the edit, and each Add assigns the next Prefix ID in order across them.
+Parts whose Adds alone pass the set's limit, or that remove more prefixes than the set holds, are a PROTOCOL_VIOLATION as soon as they arrive.
+
+**Remove Count** and **Prefix ID**:
+The prefixes to remove from the set, each by its Prefix ID.
+One that was never assigned, was already removed, or is listed twice in an edit is a PROTOCOL_VIOLATION.
+
+**Add Count** and **Add**:
+The prefixes to add once those are removed.
+Each Add implicitly assigns the next Prefix ID on the stream (see [Announce](#announce)).
+
**Broadcast Path Prefix**:
Indicate interest for any broadcasts with a path that starts with this prefix.
@@ -1174,20 +1245,24 @@ Indicate interest for any broadcasts with a path that starts with this prefix.
1 to also receive hidden claims (see [Hidden Paths](#hidden)), 0 otherwise.
Any other value is a PROTOCOL_VIOLATION.
-The publisher MUST respond with an ANNOUNCE_OK message followed by the matching routes and claims and any future changes, subject to [Routing](#routing).
-The subscriber then sends nothing more on the stream; a publisher MUST close the session with a PROTOCOL_VIOLATION if it receives more, and treats the subscriber closing its side as the end of its interest.
+One edit MAY swap prefixes for one above or below them, or re-add a prefix it removes to change its `Hidden`.
+The publisher MUST respond to the edit's last ANNOUNCE_REQUEST with an ANNOUNCE_OK on its own Announce Stream, followed by the routes and claims the added prefixes newly match and any future changes, subject to [Routing](#routing).
Implementations SHOULD consider reasonable limits on the number of matching claims and routes to prevent resource exhaustion.
## ANNOUNCE_OK {#announce-ok}
-A publisher sends an ANNOUNCE_OK message exactly once, as the first message on the response side of an Announce Stream.
+A publisher sends one ANNOUNCE_OK message on its Announce Stream for each edit it receives, after the edit's last ANNOUNCE_REQUEST, where the edit takes effect (see [Edits](#edits)).
~~~
ANNOUNCE_OK Message {
+ Type (i) = 0x11
Message Length (i) = 0
}
~~~
+**Type**:
+Set to 0x11 to indicate an ANNOUNCE_OK message.
+
ANNOUNCE_OK has no body and reserves no fields for a route-key proof.
## SYNCED {#synced}
@@ -1196,13 +1271,18 @@ The initial set on a discovery stream has been delivered.
~~~
SYNCED Message {
Type (i) = 0x4
- Message Length (i) = 0
+ Message Length (i)
+ [Prefix ID (i)]
}
~~~
-SYNCED has no body and assigns no id.
-A second SYNCED on the same stream is a PROTOCOL_VIOLATION.
-An Announce Stream ending before SYNCED is a PROTOCOL_VIOLATION; a Tracks Stream FIN before SYNCED means the Broadcast ended (see [Tracks](#tracks-stream)).
+**Prefix ID**:
+On an Announce Stream, the prefix whose initial set is complete: the Prefix ID an Add of the peer's ANNOUNCE_REQUEST assigned.
+Absent on a Tracks Stream, which has one initial set.
+
+SYNCED assigns no id.
+A second SYNCED for the same prefix, or on the same Tracks Stream, is a PROTOCOL_VIOLATION.
+A Tracks Stream FIN before SYNCED means the Broadcast ended (see [Tracks](#tracks-stream)).
## ROUTE_START {#route-start}
@@ -1214,7 +1294,9 @@ ROUTE_START Message {
Type (i) = 0x6
Message Length (i)
Node ID (i),
+ Prefix ID (i),
Route Suffix (s),
+ Exact (8),
Route Offer (..),
}
~~~
@@ -1226,16 +1308,25 @@ Set to 0x6 to indicate a ROUTE_START message.
The origin node, as the publisher names it on this session.
0 is refused (see [Refused Content](#refused-content)).
+**Prefix ID**:
+The requested prefix the route lies in: the Prefix ID an Add of the peer's ANNOUNCE_REQUEST assigned.
+
**Route Suffix**:
-The route's prefix, relative to the requested prefix: the route covers the requested prefix joined with it, and is empty for a route at the requested prefix itself.
+The route's prefix, relative to that requested prefix: the route covers the requested prefix joined with it, and is empty for a route at the requested prefix itself.
A suffix that is not a valid path, or that joins the requested prefix past the limits of a broadcast path, is refused.
+**Exact**:
+1 when the publisher keeps the route at its own prefix whatever prefix the request lies at, such as a route a relay presents at a mount point, so an edit that widens the request past it leaves it there (see [Edits](#edits)); 0 when the publisher may present it elsewhere, such as a route covering the requested prefix from a broader prefix of its own, which a widening moves up.
+A route strictly below its requested prefix stays through an edit that covers it either way; `Exact` also keeps it at its own prefix through a widening once an edit has made that prefix a requested prefix.
+A publisher MAY send 0 for a route it keeps at its own prefix: a widening then moves it up, and the publisher sends whatever the carried routes lack, such as the route again under its own prefix.
+Any value other than 0 or 1 is a PROTOCOL_VIOLATION.
+
**Route Offer**:
The route, as below.
### Route Offer {#route-offer}
What a route says: its Path, then its attributes.
-ROUTE_START, ROUTE_UPDATE, and a DEAD's moved routes carry it.
+ROUTE_START and ROUTE_UPDATE carry it.
~~~
Route Offer {
@@ -1352,15 +1443,7 @@ Closed {
Hop (..),
Far Hop ID (i),
Gone (8),
- More (8),
Moved Count (i),
- Moved (..) ...,
-}
-
-Moved {
- Route ID (i),
- Offer Length (i),
- Route Offer (..),
}
Drain {
@@ -1384,20 +1467,14 @@ The Hop ID of the node at the session's other end, which the receiver doubts.
0 is a PROTOCOL_VIOLATION.
**Gone**:
-1 when every route on the stream through the hop that Moved does not list has ended, 0 when the rest are held (see [Dead Hops](#dead-hops)).
-Any other value is a PROTOCOL_VIOLATION.
-
-**More**:
-1 when the next message on the stream is a DEAD continuing this one's Moved list, which the receiver applies with it, and 0 for the last or only part.
-A part with More set followed by any other message, or by a DEAD whose Kind, Hop, Far Hop ID, or Gone differs, is a PROTOCOL_VIOLATION.
+1 when every route on the stream through the hop that did not move has ended, 0 when the rest are held (see [Dead Hops](#dead-hops)).
Any other value is a PROTOCOL_VIOLATION.
**Moved Count**:
-The number of Moved entries that follow.
-
-**Moved**:
-A route on the stream through the hop that moved to another Path: its Route ID, which MUST name a live route, and its new [Route Offer](#route-offer), preceded by the offer's length in bytes.
-A refused offer leaves the receiver no route, as a refused ROUTE_UPDATE does, and the length lets it read the next entry; an offer that does not fill its length exactly is a PROTOCOL_VIOLATION.
+The number of routes on the stream through the hop that moved to another Path.
+The next Moved Count messages on the stream are [ROUTE_UPDATE](#route-update)s, one for each with its new Route Offer, and the receiver applies the DEAD and all of them as one.
+A refused one leaves the receiver no route, as any refused ROUTE_UPDATE does.
+Any other message before the last of them, one naming a route that is not live or whose Path does not hold the hop, a second one for the same route, and a count above the stream's routes through the hop are each a PROTOCOL_VIOLATION.
**Hop ID** (Drain):
The Hop ID of the node that sent the GOAWAY.
@@ -1408,7 +1485,7 @@ The Hop ID of the node that sent the GOAWAY.
A publisher sends an ANNOUNCE_START message to advertise a claim of a route's origin node: that paths under a prefix can be served.
Each ANNOUNCE_START implicitly assigns the next Announce ID on the stream, later referenced by ANNOUNCE_END, ANNOUNCE_UPDATE, and ANNOUNCE_RESTART (see [Announce](#announce)).
-Only the suffix is encoded on the wire, as the full claim prefix can be constructed by prepending the requested prefix.
+The claim's prefix is encoded relative to the session.
~~~
ANNOUNCE_START Message {
@@ -1431,11 +1508,11 @@ ANNOUNCE_START Message {
Set to 0x0 to indicate an ANNOUNCE_START message.
**Path Base** and **Path Keep**:
-The suffix begins with the first `Path Keep` segments of the suffix claimed by the base that `Path Base` names (see [Compression](#announce-compression)).
+The claim's prefix begins with the first `Path Keep` segments of the prefix claimed by the base that `Path Base` names (see [Compression](#announce-compression)).
**Claim Prefix Suffix**:
-The remaining segments of the suffix, which is combined with the requested prefix to form the claim's full prefix.
-An empty suffix claims the requested prefix itself, which is how a claim covering more than the request presents (see [Announce](#announce)).
+The remaining segments of the claim's prefix.
+A claim covering more than the requested prefix its route lies in presents at that requested prefix (see [Announce](#announce)).
**Route ID**:
The route the claim travels under: its origin node is the claim's.
@@ -2364,7 +2441,7 @@ An application-specific payload.
- Switched every variable-length integer, including SETUP parameter values, from QUIC's two-bit length prefix to moq-transport's leading-ones encoding, widening the range to 64 bits.
- Capped the Message Length of every message except FRAME at 65,535 bytes. SETUP parameter counts are bounded by Message Length, without a separate count cap.
-- Any Setup Stream or bidirectional stream message that fails to decode closes the session with PROTOCOL_VIOLATION; previously only an unexpected Message Length did, as a SHOULD. This includes a frame bound without its group bound, which previously reset only the stream. A stream ending mid-message outside the Setup and Auth Streams, a malformed GROUP or FRAME on a Group or Fetch Stream, and an unknown stream type still affect only that stream.
+- Any Setup Stream, Announce Stream, or bidirectional stream message that fails to decode closes the session with PROTOCOL_VIOLATION; previously only an unexpected Message Length did, as a SHOULD. This includes a frame bound without its group bound, which previously reset only the stream. A stream ending mid-message outside the Setup and Auth Streams, a malformed GROUP or FRAME on a Group or Fetch Stream, and an unknown stream type still affect only that stream.
- A session without a negotiated subprotocol negotiates the version in a draft-14 CLIENT_SETUP and SERVER_SETUP exchange.
- Added the reliable ordered RTCDataChannel binding, with one Qmux Record per binary message and no datagrams.
- Added the SETUP `Implementation` parameter (0x6): the sender's software as `name/version` product tokens.
@@ -2398,20 +2475,22 @@ An application-specific payload.
- A path never has an empty segment, so a leading, trailing, or doubled `/` is a PROTOCOL_VIOLATION rather than a second spelling of the same path.
- A `\` in a path escapes only `\` or `/`; any other `\` is a PROTOCOL_VIOLATION rather than a second spelling of the same namespace field.
- Hid claims with a `.`-prefixed segment below the requested prefix from announce discovery, and added the ANNOUNCE_REQUEST `Hidden` field to opt in.
-- Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining claims that end before delivery.
+- Replaced ANNOUNCE_OK Active Count with an ordered SYNCED event (0x4), marking the initial set of the prefix it names without retaining claims that end before delivery.
- Added `Instance` and `Epoch` to ANNOUNCE_START and ANNOUNCE_RESTART, and `Instance` to TRACK, TRACKS, SUBSCRIBE, and FETCH. The Instance is non-empty UTF-8 text naming one continuous stream, and a path and an Instance name one Broadcast; the Epoch is a counter within one Instance. A relay MAY move a subscription between claims with the same Instance, continuing from the first frame the subscriber lacks instead of at a group boundary, and never between claims with different Instances. Within one Instance only its highest Epoch is eligible. A request's Instance is empty when it pins none. Replaces the first-hop identity.
- Added ANNOUNCE_RESTART (0x3): a live advertisement now serves another Instance, or a higher Epoch of the same one.
- An advertiser MAY announce a path it answers through a claim under the claim's Instance, so subscriptions resolved through the claim continue, and a claim still serving that Instance when the announcement ends restarts under a new one.
- A receiver mints an Instance for each announcement a session of an earlier version delivers, and a `Minted` field beside every Instance (ANNOUNCE_START, ANNOUNCE_RESTART, TRACK, TRACKS, SUBSCRIBE, FETCH, and DATAGRAMS) says whether a receiver minted it, so no Instance text is reserved for minting.
- Added announce compression: ANNOUNCE_START gains `Path Base` and `Path Keep` to copy the head of a live claim's suffix.
+- Replaced the bidirectional Announce Stream (0x1) with one unidirectional Announce Stream (0x3) per endpoint, carrying everything it says about discovery and routing in order, so none of it overtakes the rest. A second Announce Stream, or one that is reset, closes the session.
+- ANNOUNCE_REQUEST (0x10) edits a set of prefixes, none at or below another, one edit at a time, and ANNOUNCE_OK (0x11) marks where each edit takes effect. An edit too large for one message continues over several ANNOUNCE_REQUESTs, each but the last with `More` set, and takes effect whole. Both endpoints carry the routes and claims that both sets cover over to the new set, so none is sent again. ROUTE_START and SYNCED name the requested prefix by its Prefix ID, and every claim's path is relative to the session rather than to its requested prefix. A finished Announce Stream means its endpoint is leaving.
- Split routing into two layers on the Announce Stream. ROUTE_START (0x6), ROUTE_UPDATE (0x7), and ROUTE_END (0x8) advertise how the publisher reaches an origin node, named by a Node ID, for the paths under a prefix, with an additive metric and a down-only bit. Every ANNOUNCE_START names its route's Route ID in place of the Hop ID list, so a link change sends one ROUTE_UPDATE per origin and prefix rather than an update per claim. Removed the Hop ID list and ANNOUNCE_OK's Hop ID, and replaced the SETUP Hop parameter (0x5) with a [Node ID parameter](#node-id-parameter), so an endpoint that reaches itself detects it at setup.
-- ROUTE_START and ROUTE_UPDATE carry a Route Offer that leads with the route's Path: one hop per node the route crossed, each a Hop ID and the Session Number that node learned the route on, at most 32. A node never uses a route whose Path names it. ROUTE_START also carries a Route Suffix naming the prefix the route covers, and the offer carries the route's drain (Flags 0x2) and priority, which rank before its metric.
+- ROUTE_START and ROUTE_UPDATE carry a Route Offer that leads with the route's Path: one hop per node the route crossed, each a Hop ID and the Session Number that node learned the route on, at most 32. A node never uses a route whose Path names it. ROUTE_START also carries a Prefix ID and a Route Suffix naming the prefix the route covers, and `Exact`, marking a route its publisher keeps at its own prefix, which an edit widening the request past it leaves in place. The offer carries the route's drain (Flags 0x2) and priority, which rank before its metric.
- Added ROUTE_HOLD (0xB), which holds a route while it may recover, for at most 10 seconds; ROUTE_END lets it go. Neither withdraws the origin's claims.
-- Added DEAD (0xF): a node that loses a session names that hop once to each neighbour it sent a route through it, listing the routes that moved, and every node drops every route through the hop and doubts the node at the far end for a second, keeping nodes off routes through it but not off one it originates, except over a session to it opened since; a DRAIN names a node that sent a GOAWAY, whose routes rank draining. The path, the dead hop, holds, and the doubt keep routes free of stale copies, and of loops except briefly when an origin on three or more relays leaves them all at once, with no distance, sequence number, route request, or echo. On a scoped session a DEAD applies only to the routes heard there.
-- A node closes a session whose peer leaves an Announce Stream undrained for 10 seconds, and the subscriber sends nothing on the stream after ANNOUNCE_REQUEST.
+- Added DEAD (0xF): a node that loses a session names that hop once to each neighbour it sent a route through it, counting the routes that moved, each a ROUTE_UPDATE right behind it, and every node drops every route through the hop and doubts the node at the far end for a second, keeping nodes off routes through it but not off one it originates, except over a session to it opened since; a DRAIN names a node that sent a GOAWAY, whose routes rank draining. The path, the dead hop, holds, and the doubt keep routes free of stale copies, and of loops except briefly when an origin on three or more relays leaves them all at once, with no distance, sequence number, route request, or echo. On a scoped session a DEAD applies only to the routes heard there.
+- A node closes a session whose peer leaves its Announce Stream undrained for 10 seconds.
- Added ANNOUNCE_FLOOR (0x9), closing each batch of a route's claims with the origin Revision at which the stream's claims across the route's prefix are certified plus one, or 0 to certify nothing, so a stale copy never revives a claim whose withdrawal a newer one delivered. A floor certifies only the claims its own request shows.
- A route presents only its most specific claim covering the requested prefix, moved to the next covering claim when it is withdrawn rather than ended.
-- A subscriber whose scope confines a stream to one prefix SHOULD request that prefix, and replace the request when the prefix changes.
+- A subscriber whose scope confines it to some prefixes SHOULD request those, and edit its set when they change.
- A route goes on a stream only while it has a claim the request presents, and ends after the batch withdrawing its last claim.
- Added `Node ID` and `Route Depth` to TRACK, TRACKS, SUBSCRIBE, FETCH, and DATAGRAMS: the origin node the first hop selected and the prefix of the route it follows, which every later hop follows.
- Replaced Warm and Cold Route Cost with the origin's Announce Cost, unchanged in transit. ANNOUNCE_UPDATE carries only the Announce Cost.
diff --git a/js/net/bench/broadcasts.ts b/js/net/bench/broadcasts.ts
index 3a3c9dd5837..c439ac65e37 100644
--- a/js/net/bench/broadcasts.ts
+++ b/js/net/bench/broadcasts.ts
@@ -7,7 +7,6 @@ import { Producer as BroadcastProducer } from "../src/broadcast.ts";
import { BroadcastCache } from "../src/consume.ts";
import { toHop, WireRoute } from "../src/hop.ts";
import * as Instance from "../src/instance.ts";
-import { AnnounceRequest } from "../src/lite/announce.ts";
import { Publisher } from "../src/lite/publisher.ts";
import { Subscriber } from "../src/lite/subscriber.ts";
import { TrackInfo } from "../src/lite/track.ts";
@@ -15,7 +14,7 @@ import { ALPN_07_WIP, Version } from "../src/lite/version.ts";
import { createMockTransportPair } from "../src/mock.ts";
import { Producer } from "../src/origin.ts";
import * as Path from "../src/path.ts";
-import { Stream, Writer } from "../src/stream.ts";
+import { Writer } from "../src/stream.ts";
import { wireOf } from "../src/wire.ts";
// Keep protocol diagnostics out of the measured CSV on stdout.
@@ -401,21 +400,25 @@ for (const routeCount of routeCounts) {
});
const senders = Array.from({ length: observerCount }, () => {
const pair = createMockTransportPair(ALPN_07_WIP);
- const publisher = new Publisher(pair.server, Version.DRAFT_07, toHop(1n), origin.consume());
const entered = Promise.withResolvers();
const release = Promise.withResolvers();
- const stream = new Stream({
- version: Version.DRAFT_07,
- readable: new ReadableStream(),
- writable: new WritableStream({
- async write() {
+ // The session's Announce Stream, whose first write stalls.
+ const create = pair.server.createUnidirectionalStream.bind(pair.server);
+ pair.server.createUnidirectionalStream = async (options) => {
+ const writer = (await create(options)).getWriter();
+ return new WritableStream({
+ async write(chunk) {
entered.resolve();
await release.promise;
+ await writer.write(chunk);
},
- }),
- });
- const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
- return { pair, publisher, entered, release, running };
+ close: () => writer.close(),
+ abort: (reason) => writer.abort(reason),
+ }) as Awaited>;
+ };
+ const publisher = new Publisher(pair.server, Version.DRAFT_07, toHop(1n), origin.consume());
+ publisher.request(false, [], [{ prefix: Path.empty(), hidden: false }]);
+ return { pair, publisher, entered, release };
});
await Promise.all(senders.map((sender) => sender.entered.promise));
const start = performance.now();
@@ -430,11 +433,10 @@ for (const routeCount of routeCounts) {
`stalled-announce,${routeCount},${observerCount},${(((performance.now() - start) * 1000) / updates).toFixed(1)}`,
);
await Promise.all(
- senders.map(async ({ pair, publisher, release, running }) => {
+ senders.map(async ({ pair, publisher, release }) => {
const drained = publisher.drain();
release.resolve();
await drained;
- await running;
publisher.close();
pair.client.close();
pair.server.close();
diff --git a/js/net/src/connection/close.test.ts b/js/net/src/connection/close.test.ts
index 2f998961ab0..97d715bc52f 100644
--- a/js/net/src/connection/close.test.ts
+++ b/js/net/src/connection/close.test.ts
@@ -14,25 +14,28 @@ function holdFin(transport: MockTransport) {
const reached = Promise.withResolvers();
const release = Promise.withResolvers();
let enabled = false;
- const wrap = (stream: WebTransportBidirectionalStream): WebTransportBidirectionalStream => {
- const writer = stream.writable.getWriter();
- return {
- readable: stream.readable,
- writable: new WritableStream({
- write: (bytes) => writer.write(bytes),
- abort: (reason) => writer.abort(reason),
- async close() {
- await writer.close();
- if (enabled) {
- reached.resolve();
- await release.promise;
- }
- },
- }),
- } as WebTransportBidirectionalStream;
+ const hold = (writable: WritableStream): WritableStream => {
+ const writer = writable.getWriter();
+ return new WritableStream({
+ write: (bytes) => writer.write(bytes),
+ abort: (reason) => writer.abort(reason),
+ async close() {
+ await writer.close();
+ if (enabled) {
+ reached.resolve();
+ await release.promise;
+ }
+ },
+ });
};
+ const wrap = (stream: WebTransportBidirectionalStream): WebTransportBidirectionalStream =>
+ ({ readable: stream.readable, writable: hold(stream.writable) }) as WebTransportBidirectionalStream;
const create = transport.createBidirectionalStream.bind(transport);
transport.createBidirectionalStream = async (options) => wrap(await create(options));
+ // Lite-07 withdraws on its unidirectional Announce Stream.
+ const createUni = transport.createUnidirectionalStream.bind(transport);
+ transport.createUnidirectionalStream = async (options) =>
+ hold(await createUni(options)) as Awaited>;
Object.defineProperty(transport, "incomingBidirectionalStreams", {
value: transport.incomingBidirectionalStreams.pipeThrough(
new TransformStream({
diff --git a/js/net/src/connection/forward.test.ts b/js/net/src/connection/forward.test.ts
index 2c13e3d9156..8bd23ffe80e 100644
--- a/js/net/src/connection/forward.test.ts
+++ b/js/net/src/connection/forward.test.ts
@@ -1,6 +1,7 @@
import { afterEach, beforeEach, expect, jest, spyOn, test } from "bun:test";
import { Duration, type Installed } from "@moq/clock";
import { Manual } from "@moq/clock/test";
+import { Signal } from "@moq/signals";
import * as Announce from "../announced.ts";
import { AuthSession } from "../auth_session.ts";
import { type Consumer as BroadcastConsumer, Producer as BroadcastProducer } from "../broadcast.ts";
@@ -72,6 +73,9 @@ class FakeSession {
/** Whether a lite-07 claim of `node` at `at` serves `path` beneath it: the session's floors. */
covering?: (node: bigint, at: Path.Valid, path: Path.Valid) => boolean;
+ /** Bumped when what `covering` answers changed with no route reporting it. */
+ readonly coverage = new Signal(0);
+
closed: Promise;
#die!: (reason: Error | null) => void;
@@ -80,6 +84,7 @@ class FakeSession {
registerWire(this, {
consume: (path, instance) => this.consume(path, instance),
covers: (node, at, path) => this.covering?.(node, at, path) ?? true,
+ coverage: this.coverage,
exclusive,
leave: () => {},
auth: new AuthSession({ peerGrant: { publish: new Path.Patterns(), subscribe: new Path.Patterns() } }),
@@ -253,6 +258,48 @@ test("the origin skips a claim above a newer floor's region", async () => {
origin.close();
});
+// A floor moves what a claim covers while both routes stay as they were: the request already
+// open beneath it is admitted anew and ends, and the next one resolves through the claim that
+// still covers its path.
+test("a floor that moves what a claim covers admits the open requests beneath it anew", async () => {
+ const origin = new OriginProducer();
+ const session = new FakeSession();
+ forwardAnnounced(session.session, origin);
+ const a = Instance.parse("a");
+ const b = Instance.parse("b");
+ const NODE_A = 0x7an;
+ const NODE_B = 0x8bn;
+ const live = origin.routes(Path.Pattern.subtree(Path.from("live")));
+ await settle();
+ start(session.stream("live/**"), "live", { ...WireRoute.default, instance: a, node: NODE_A });
+ start(session.stream("live/**"), "live", { ...WireRoute.default, instance: b, node: NODE_B, cost: 5n });
+ session.stream("live/**").synced();
+ await settle();
+
+ const x = Path.from("live/x");
+ const request = origin.request(x);
+ await settle();
+ expect(request.active.peek()).toBeDefined();
+ expect(session.pins).toEqual([a]);
+
+ session.covering = (node, at, path) => !(node === NODE_A && at === Path.from("live") && Path.hasPrefix(x, path));
+ session.coverage.update((n) => n + 1);
+ await settle();
+ // The claim it resolved through no longer serves its path, so it ends unroutable.
+ expect(request.closed.peek()).toBeInstanceOf(Error);
+ expect(session.pins).toEqual([a]);
+ const again = origin.request(x);
+ await settle();
+ expect(again.active.peek()).toBeDefined();
+ expect(session.pins).toEqual([a, b]);
+
+ request.close();
+ again.close();
+ live.close();
+ session.die();
+ origin.close();
+});
+
// A route handed to another request's report of another instance restarts, so its requests
// consume that instance rather than the one the first report named.
test("a route handed over to another instance consumes that instance", async () => {
diff --git a/js/net/src/connection/forward.ts b/js/net/src/connection/forward.ts
index 7afe4762801..bd26329ecdf 100644
--- a/js/net/src/connection/forward.ts
+++ b/js/net/src/connection/forward.ts
@@ -74,6 +74,10 @@ interface Reported {
* left. The routes a closed request carried go with it, and come back once the broader request
* reports them.
*
+ * A lite-07 claim serves only the paths the session's floors still certify beneath it, asked
+ * per request, and asked again for every open request when a floor moves that with no route
+ * changing.
+ *
* Each active announcement inserts a route covering the announced prefix, served lazily
* through this connection, so nothing touches the wire until somebody requests a path
* under it. A retraction removes the entry, and so does the session dying, which is what
@@ -301,9 +305,20 @@ export function forwardAnnounced(conn: Session, origin: OriginProducer): void {
if (want && !covered && !blind) open(prefix);
});
+ // A floor can move what a claim covers while every route stays as it was, so the open
+ // requests beneath each claim are admitted anew: advertising a route as it stands reroutes them.
+ const uncovered = session.coverage?.subscribe(() => {
+ if (dead) return;
+ for (const { handle } of routes.values()) {
+ const wire = wireOf(handle);
+ if (wire.info?.node !== undefined) wire.update(wire.info);
+ }
+ });
+
void conn.closed.then(() => {
dead = true;
unsubscribe();
+ uncovered?.();
// End every request the moment the session closes rather than waiting for the wire to
// error it, so the retractions land promptly.
for (const interest of interests.values()) interest.announced.close();
diff --git a/js/net/src/consume.ts b/js/net/src/consume.ts
index d781e92003d..a47e41c48bf 100644
--- a/js/net/src/consume.ts
+++ b/js/net/src/consume.ts
@@ -69,6 +69,8 @@ export class BroadcastCache {
#blindPrefixes = new Map>();
#paths = new Map>();
readonly #covering?: Covering;
+ // What to re-check once a batch is applied: the prefixes it changed, or `all` of them.
+ #deferred?: { prefixes: Set; all: boolean };
constructor(covering?: Covering) {
this.#covering = covering;
@@ -102,8 +104,32 @@ export class BroadcastCache {
this.#update(path);
}
+ /**
+ * Apply the changes `apply` makes as one: each consume re-checks its selection once, after all
+ * of them, so none switches to a route only the middle of the change preferred.
+ */
+ batch(apply: () => void): void {
+ if (this.#deferred) {
+ apply();
+ return;
+ }
+ const deferred = { prefixes: new Set(), all: false };
+ this.#deferred = deferred;
+ try {
+ apply();
+ } finally {
+ this.#deferred = undefined;
+ if (deferred.all) this.revalidate();
+ else for (const prefix of deferred.prefixes) this.#update(prefix);
+ }
+ }
+
/** Re-check every consume's selection, once what covers a path changed without an announcement. */
revalidate(): void {
+ if (this.#deferred) {
+ this.#deferred.all = true;
+ return;
+ }
for (const path of this.#cache.keys()) this.#check(path);
}
@@ -248,6 +274,10 @@ export class BroadcastCache {
}
#update(prefix: Path.Valid): void {
+ if (this.#deferred) {
+ this.#deferred.prefixes.add(prefix);
+ return;
+ }
for (const path of this.#paths.get(prefix) ?? []) this.#check(path);
}
diff --git a/js/net/src/integration.test.ts b/js/net/src/integration.test.ts
index b1ae87bc172..203624b090d 100644
--- a/js/net/src/integration.test.ts
+++ b/js/net/src/integration.test.ts
@@ -2220,6 +2220,12 @@ test.each([Lite.ALPN_03, Lite.ALPN_05, Lite.ALPN_06, Lite.ALPN_07_WIP])(
spyOn(LitePublisher.prototype, "runAnnounce").mockImplementation(function (this: LitePublisher, msg, stream) {
return run.call(this, msg, stream).finally(() => ended++);
});
+ // Lite-07 has one Announce Stream, and gives a prefix back with the edit removing it.
+ const request = LitePublisher.prototype.request;
+ spyOn(LitePublisher.prototype, "request").mockImplementation(function (this: LitePublisher, more, remove, add) {
+ ended += remove.length;
+ request.call(this, more, remove, add);
+ });
const pair = createMockTransportPair(alpn);
const serverOrigin = new OriginProducer();
const [client, server] = await Promise.all([
@@ -2243,7 +2249,39 @@ test.each([Lite.ALPN_03, Lite.ALPN_05, Lite.ALPN_06, Lite.ALPN_07_WIP])(
},
);
-test.each([Lite.ALPN_05, Lite.ALPN_06, Lite.ALPN_07_WIP])(
+// Lite-07 sends one edit at a time: a consumer that leaves before the peer answered the edit
+// adding its prefix has it removed by the next edit, once that answer is in.
+test("moq-lite-07-wip: closing an announce consumer before ANNOUNCE_OK removes its prefix once answered", async () => {
+ const request = LitePublisher.prototype.request;
+ const edits: { remove: bigint[]; add: string[] }[] = [];
+ const answer = Promise.withResolvers();
+ spyOn(LitePublisher.prototype, "request").mockImplementation(function (this: LitePublisher, more, remove, add) {
+ edits.push({ remove, add: add.map(({ prefix }) => prefix) });
+ // The first edit reaches the publisher only once the test lets it.
+ if (edits.length === 1) void answer.promise.then(() => request.call(this, more, remove, add));
+ else request.call(this, more, remove, add);
+ });
+ const pair = createMockTransportPair(Lite.ALPN_07_WIP);
+ const [client, server] = await Promise.all([
+ connect(url, { transport: pair.client }),
+ accept(pair.server, url, { publish: new OriginProducer().consume() }),
+ ]);
+
+ const announced = client.routes(Path.Pattern.subtree(Path.from("room")));
+ await until(() => edits.length === 1);
+ announced.close();
+ for (let i = 0; i < 20; i++) await yieldNow();
+ // Nothing more is sent while the first edit is unanswered.
+ expect(edits).toEqual([{ remove: [], add: ["room"] }]);
+ answer.resolve();
+ await until(() => edits.length === 2);
+ expect(edits[1]).toEqual({ remove: [0n], add: [] });
+
+ client.abort();
+ server.abort();
+});
+
+test.each([Lite.ALPN_05, Lite.ALPN_06])(
"%s: closing an announce consumer before ANNOUNCE_OK ends the peer's stream",
async (alpn) => {
// The peer never answers, and returns only once the request stream closes.
@@ -2287,6 +2325,18 @@ for (const alpn of [Lite.ALPN_06, Lite.ALPN_07_WIP]) {
started++;
return run.call(this, msg, stream).finally(() => ended++);
});
+ // Lite-07 asks and gives back each prefix with an edit on its one Announce Stream.
+ const edit = LitePublisher.prototype.request;
+ spyOn(LitePublisher.prototype, "request").mockImplementation(function (
+ this: LitePublisher,
+ more,
+ remove,
+ add,
+ ) {
+ started += add.length;
+ ended += remove.length;
+ edit.call(this, more, remove, add);
+ });
const pair = createMockTransportPair(alpn);
const serverOrigin = new OriginProducer();
const clientOrigin = new OriginProducer();
diff --git a/js/net/src/lite/announce.test.ts b/js/net/src/lite/announce.test.ts
index 63812833e13..e3dadfec9fe 100644
--- a/js/net/src/lite/announce.test.ts
+++ b/js/net/src/lite/announce.test.ts
@@ -14,7 +14,10 @@ import {
encodeAnnounceBroadcast,
type Offer,
ROUTE_MAX_HOPS,
+ requestFits,
+ requestParts,
} from "./announce.ts";
+import { MAX_MESSAGE_SIZE } from "./message.ts";
import { Version } from "./version.ts";
function concat(chunks: Uint8Array[]): Uint8Array {
@@ -197,13 +200,75 @@ test("AnnounceRequest drops excludeHop on draft-06", async () => {
expect(with06.byteLength).toBeLessThan(with05.byteLength);
});
-// Draft07 carries the hidden opt-in; every earlier version decodes as not opted in.
-test("AnnounceRequest carries hidden from draft-07", async () => {
- for (const hidden of [false, true]) {
- const msg = new AnnounceRequest(Path.from("room"), UNKNOWN_HOP, hidden);
- expect((await requestRoundTrip(msg, Version.DRAFT_07)).hidden).toBe(hidden);
- expect((await requestRoundTrip(msg, Version.DRAFT_06)).hidden).toBe(false);
+// Lite-07 asks on its Announce Stream: whether the edit continues, the Prefix IDs to remove, then
+// each prefix to add with its hidden opt-in. The stream-opening request is every earlier version's.
+test("ANNOUNCE_REQUEST edits the requested prefixes on lite-07", async () => {
+ const v = Version.DRAFT_07;
+ const msg: AnnounceBroadcast = {
+ status: "request",
+ more: true,
+ remove: [1n, 2n],
+ add: [
+ { prefix: Path.from("room"), hidden: true },
+ { prefix: Path.empty(), hidden: false },
+ ],
+ };
+ const wire = await bytes((w) => encodeAnnounceBroadcast(w, msg, v), v);
+ expect(wire).toEqual(new Uint8Array([16, 13, 1, 2, 1, 2, 2, 4, 0x72, 0x6f, 0x6f, 0x6d, 1, 0, 0]));
+ expect(await roundTrip(msg, v)).toEqual(msg);
+ const empty: AnnounceBroadcast = { status: "request", more: false, remove: [], add: [] };
+ expect(await bytes((w) => encodeAnnounceBroadcast(w, empty, v), v)).toEqual(new Uint8Array([16, 3, 0, 0, 0]));
+ await expect(roundTrip(msg, Version.DRAFT_06)).rejects.toThrow();
+ await expect(bytes((w) => new AnnounceRequest(Path.from("room"), UNKNOWN_HOP).encode(w, v), v)).rejects.toThrow();
+
+ // A count the body cannot hold, and a More or a Hidden that is no boolean, are malformed.
+ const decode = (body: number[]) =>
+ decodeAnnounceBroadcast(new Reader(undefined, new Uint8Array([16, body.length, ...body]), v), v);
+ await expect(decode([0, 3, 1, 0])).rejects.toThrow();
+ await expect(decode([0, 0, 1, 0, 2])).rejects.toThrow();
+ await expect(decode([2, 0, 0])).rejects.toThrow();
+});
+
+// The requested set is its prefixes as an ANNOUNCE_REQUEST adding them all would carry them,
+// which fits one message.
+test("a requested set fits one message", () => {
+ const v = Version.DRAFT_07;
+ const long = (i: number) => Path.from(`${String(i).padStart(4, "0")}/${"x".repeat(1000)}`);
+ const fits = Array.from({ length: 64 }, (_, i) => long(i));
+ expect(requestFits(fits, v)).toBe(true);
+ expect(requestFits([...fits, long(64), long(65)], v)).toBe(false);
+ expect(requestFits([], v)).toBe(true);
+});
+
+// An edit too large for one message goes in as many as it takes, each fitting one, every one but
+// the last continuing it, and together holding the whole edit in order, as `request_parts` does.
+test("an edit too large for one message goes in parts", async () => {
+ const v = Version.DRAFT_07;
+ // Prefix IDs from 2^28 take five bytes.
+ const remove = Array.from({ length: 20_000 }, (_, i) => (1n << 28n) + BigInt(i));
+ const add = Array.from({ length: 3000 }, (_, i) => ({
+ prefix: Path.from(String(i).padStart(20, "0")),
+ hidden: i % 2 === 0,
+ }));
+ const parts = requestParts(remove, add, v);
+ expect(parts.length).toBeGreaterThan(2);
+ const removed: bigint[] = [];
+ const added: { prefix: Path.Valid; hidden: boolean }[] = [];
+ for (const [i, part] of parts.entries()) {
+ expect(part.more).toBe(i + 1 < parts.length);
+ const wire = await bytes((w) => encodeAnnounceBroadcast(w, part, v), v);
+ // The type and the length take at most 4 bytes.
+ expect(wire.byteLength).toBeLessThanOrEqual(MAX_MESSAGE_SIZE + 4);
+ expect(await roundTrip(part, v)).toEqual(part);
+ removed.push(...part.remove);
+ added.push(...part.add);
}
+ expect(removed).toEqual(remove);
+ expect(added).toEqual(add);
+
+ // An edit that fits goes whole, and one with nothing in it still goes.
+ expect(requestParts([1n], [{ prefix: Path.from("a"), hidden: false }], v).map(({ more }) => more)).toEqual([false]);
+ expect(requestParts([], [], v)).toEqual([{ status: "request", more: false, remove: [], add: [] }]);
});
// The draft reserves Hop ID 0 for a responder that was never assigned an id, or that
@@ -387,8 +452,13 @@ test("the route layer round-trips on draft-07", async () => {
const cases: [AnnounceBroadcast, number[]][] = [
// The path first, the publisher first, then flags, priority (offset by 128), and metric.
[
- { status: "routeStart", node: 0x1234n, suffix: Path.empty(), offer },
- [6, 11, 0x92, 0x34, 0, 2, 7, 0, 9, 2, 1, 128, 5],
+ { status: "routeStart", node: 0x1234n, prefix: 3n, suffix: Path.empty(), exact: false, offer },
+ [6, 13, 0x92, 0x34, 3, 0, 0, 2, 7, 0, 9, 2, 1, 128, 5],
+ ],
+ // Exact: a route at the requested prefix whose own prefix it is.
+ [
+ { status: "routeStart", node: 7n, prefix: 0n, suffix: Path.empty(), exact: true, offer: one },
+ [6, 10, 7, 0, 0, 1, 1, 7, 0, 0, 128, 0],
],
// Flag 0x2: a node on the path drains.
[
@@ -438,19 +508,19 @@ test("the route layer round-trips on draft-07", async () => {
hop: { node: 7n, session: 3n },
far: 9n,
gone: false,
- more: false,
- moved: [{ id: 1n, offer: one }],
+ moved: 1n,
},
},
- [15, 15, 0, 7, 3, 9, 0, 0, 1, 1, 6, 1, 7, 0, 0, 128, 0],
+ [15, 6, 0, 7, 3, 9, 0, 1],
],
- // More: the next DEAD continues this one's moved routes.
+ // The routes it counts follow as ROUTE_UPDATEs of their own.
+ [{ status: "routeUpdate", id: 1n, offer: one }, [7, 7, 1, 1, 7, 0, 0, 128, 0]],
[
{
status: "dead",
- dead: { kind: "closed", hop: { node: 7n, session: 3n }, far: 9n, gone: true, more: true, moved: [] },
+ dead: { kind: "closed", hop: { node: 7n, session: 3n }, far: 9n, gone: true, moved: 0n },
},
- [15, 7, 0, 7, 3, 9, 1, 1, 0],
+ [15, 6, 0, 7, 3, 9, 1, 0],
],
// Revision 4 travels as 5: 0 certifies nothing.
[{ status: "floor", id: 1n, revision: 4n }, [9, 2, 1, 5]],
@@ -484,7 +554,9 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
};
const encode = (msg: AnnounceBroadcast) => bytes((w) => encodeAnnounceBroadcast(w, msg, v), v);
// No node sends an offer it would refuse.
- await expect(encode({ status: "routeStart", node: 0n, suffix: Path.empty(), offer })).rejects.toThrow();
+ await expect(
+ encode({ status: "routeStart", node: 0n, prefix: 0n, suffix: Path.empty(), exact: false, offer }),
+ ).rejects.toThrow();
await expect(encode({ status: "routeUpdate", id: 0n, offer: { ...offer, path: [] } })).rejects.toThrow();
await expect(
encode({ status: "routeUpdate", id: 0n, offer: { ...offer, path: [{ node: 0n, session: 0n }] } }),
@@ -496,11 +568,10 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
status: "dead",
dead: {
kind: "closed",
- hop: { node: 1n, session: 0n },
+ hop: { node: 0n, session: 0n },
far: 2n,
gone: false,
- more: false,
- moved: [{ id: 0n, offer: { refused: "flags", path: offer.path } }],
+ moved: 0n,
},
}),
).rejects.toThrow();
@@ -518,7 +589,10 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
const start = (node: bigint, path: bigint[], flags: number) =>
forged(6, async (w) => {
await w.u62(node);
+ await w.u62(5n);
await w.u53(0);
+ // Not exact.
+ await w.u8(0);
await w.u62(BigInt(path.length));
for (const hop of path) {
await w.u62(hop);
@@ -534,7 +608,7 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
const path = [{ node: 1n, session: 0n }];
expect(await decode(await start(0n, [1n], 0))).toEqual({
status: "refused",
- message: { kind: "routeStart", node: 0n, suffix: Path.empty(), path },
+ message: { kind: "routeStart", node: 0n, prefix: 5n, suffix: Path.empty(), exact: false, path },
reason: "node",
});
expect(await decode(await start(1n, [0n], 0))).toMatchObject({ status: "refused", reason: "node" });
@@ -542,7 +616,7 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
for (const flags of [4, 8]) {
expect(await decode(await start(1n, [1n], flags))).toEqual({
status: "refused",
- message: { kind: "routeStart", node: 1n, suffix: Path.empty(), path },
+ message: { kind: "routeStart", node: 1n, prefix: 5n, suffix: Path.empty(), exact: false, path },
reason: "flags",
});
}
@@ -557,44 +631,64 @@ test("the route layer refuses Node ID 0, unknown flags, and too many hops", asyn
expect(kept.message.path).toHaveLength(ROUTE_MAX_HOPS + 1);
// An empty path is the sender's own error: it names itself last.
await expect(decode(await start(1n, [], 0))).rejects.toThrow(ProtocolViolation);
+ // Exact is a boolean.
+ const notBoolean = await forged(6, async (w) => {
+ await w.u62(1n);
+ await w.u62(0n);
+ await w.string("");
+ await w.u8(2);
+ await w.u62(1n);
+ await w.u62(1n);
+ await w.u62(0n);
+ await w.u8(0);
+ await w.u8(128);
+ await w.u62(0n);
+ });
+ await expect(decode(notBoolean)).rejects.toThrow();
- // A DEAD refuses each moved offer on its own, and still reads the next.
+ // A DEAD counts the ROUTE_UPDATEs that follow, each refused on its own.
const dead = await forged(15, async (w) => {
await w.u62(0n);
await w.u62(1n);
await w.u62(0n);
await w.u62(2n);
await w.u8(0);
- await w.u8(0);
await w.u62(2n);
- for (const flags of [8, 0]) {
- await w.u62(flags === 8 ? 3n : 4n);
- await w.u53(6);
+ });
+ expect(await decode(dead)).toEqual({
+ status: "dead",
+ dead: { kind: "closed", hop: { node: 1n, session: 0n }, far: 2n, gone: false, moved: 2n },
+ });
+ const update = (flags: number) =>
+ forged(7, async (w) => {
+ await w.u62(3n);
await w.u62(1n);
await w.u62(1n);
await w.u62(0n);
await w.u8(flags);
await w.u8(128);
await w.u62(0n);
- }
+ });
+ expect(await decode(await update(8))).toEqual({
+ status: "refused",
+ message: { kind: "routeUpdate", id: 3n, path },
+ reason: "flags",
});
- expect(await decode(dead)).toEqual({
- status: "dead",
- dead: {
- kind: "closed",
- hop: { node: 1n, session: 0n },
- far: 2n,
- gone: false,
- more: false,
- moved: [
- { id: 3n, offer: { refused: "flags", path } },
- {
- id: 4n,
- offer: { path, draining: false, priority: 0, metric: 0n, downOnly: false },
- },
- ],
- },
+ expect(await decode(await update(0))).toEqual({
+ status: "routeUpdate",
+ id: 3n,
+ offer: { path, draining: false, priority: 0, metric: 0n, downOnly: false },
});
+ // A Gone that is no boolean is malformed.
+ const gone = await forged(15, async (w) => {
+ await w.u62(0n);
+ await w.u62(1n);
+ await w.u62(0n);
+ await w.u62(2n);
+ await w.u8(2);
+ await w.u62(0n);
+ });
+ await expect(decode(gone)).rejects.toThrow();
});
test("AnnounceHistory rejects a base that is not live", () => {
@@ -762,18 +856,23 @@ test("lite06 ignores a legacy cold price and writes its saturation ceiling", asy
expect(await r.u62()).toBe(2n ** 62n - 1n);
});
-test("SYNCED is empty and belongs only to lite-07", async () => {
- expect(
- await bytes((w) => encodeAnnounceBroadcast(w, { status: "synced" }, Version.DRAFT_07), Version.DRAFT_07),
- ).toEqual(new Uint8Array([4, 0]));
- expect(await roundTrip({ status: "synced" }, Version.DRAFT_07)).toEqual({ status: "synced" });
- await expect(roundTrip({ status: "synced" }, Version.DRAFT_06)).rejects.toThrow();
+test("SYNCED names its Prefix ID and belongs only to lite-07", async () => {
+ const synced: AnnounceBroadcast = { status: "synced", prefix: 5n };
+ expect(await bytes((w) => encodeAnnounceBroadcast(w, synced, Version.DRAFT_07), Version.DRAFT_07)).toEqual(
+ new Uint8Array([4, 1, 5]),
+ );
+ expect(await roundTrip(synced, Version.DRAFT_07)).toEqual(synced);
+ await expect(roundTrip(synced, Version.DRAFT_06)).rejects.toThrow();
});
-// Lite-07 drops both: the count for SYNCED, and the Hop ID for the route layer.
-test("ANNOUNCE_OK is empty on lite-07", async () => {
- for (const version of [Version.DRAFT_06, Version.DRAFT_07]) {
- const wire = await bytes((w) => new AnnounceOk(toHop(1n), 7).encode(w, version), version);
- expect(wire).toEqual(new Uint8Array(version === Version.DRAFT_06 ? [2, 1, 7] : [0]));
- }
+// Lite-07 answers an edit with an empty message on its Announce Stream: SYNCED carries the
+// count's role, and the route layer the Hop ID's.
+test("ANNOUNCE_OK is an empty Announce Stream message on lite-07", async () => {
+ const v = Version.DRAFT_07;
+ const wire = await bytes((w) => new AnnounceOk(toHop(1n), 7).encode(w, Version.DRAFT_06), Version.DRAFT_06);
+ expect(wire).toEqual(new Uint8Array([2, 1, 7]));
+ await expect(bytes((w) => new AnnounceOk(toHop(1n), 7).encode(w, v), v)).rejects.toThrow();
+ expect(await bytes((w) => encodeAnnounceBroadcast(w, { status: "ok" }, v), v)).toEqual(new Uint8Array([17, 0]));
+ expect(await roundTrip({ status: "ok" }, v)).toEqual({ status: "ok" });
+ await expect(roundTrip({ status: "ok" }, Version.DRAFT_06)).rejects.toThrow();
});
diff --git a/js/net/src/lite/announce.ts b/js/net/src/lite/announce.ts
index c86e3eba181..d192c3d2aea 100644
--- a/js/net/src/lite/announce.ts
+++ b/js/net/src/lite/announce.ts
@@ -2,10 +2,11 @@ import { ProtocolViolation } from "../error.ts";
import { type Cost, type Hop, MAX_HOPS, toHop, UNKNOWN_HOP, wireHop } from "../hop.ts";
import type * as Instance from "../instance.ts";
import * as Path from "../path.ts";
-import type { Reader, Writer } from "../stream.ts";
+import { encodeVarint, type Reader, type Writer } from "../stream.ts";
import { SEGMENT_BYTES } from "../util/limits.ts";
import { decodeForwarded, decodeRoute, encodeRoute, utf8 } from "./instance.ts";
import * as Message from "./message.ts";
+import { MAX_MESSAGE_SIZE } from "./message.ts";
import { fromWire, toWire } from "./priority.ts";
import {
hasAnnounceCompression,
@@ -13,7 +14,6 @@ import {
hasAnnounceOk,
hasAnnounceRestart,
hasExcludeHop,
- hasHidden,
hasRouteCost,
hasRoutes,
hasSynced,
@@ -42,6 +42,9 @@ const ROUTE_END = 8;
const ANNOUNCE_FLOOR = 9;
const ROUTE_HOLD = 11;
const DEAD = 15;
+// lite-07: an edit of the prefixes the sender wants, and the marker answering one.
+const ANNOUNCE_REQUEST = 16;
+const ANNOUNCE_OK = 17;
// DEAD kinds.
const DEAD_CLOSED = 0n;
@@ -82,21 +85,13 @@ export interface Unusable {
}
/** News on DEAD about the routes the sender offered. `closed`: the session `hop` names closed, at
- * the node `far` at its other end. Each route on the stream through that hop that `moved` names by
- * Route ID continues on its offer there; the rest end if `gone`, and are held if not, unless the
- * receiver is the node just before the sender on the route's path. With `more`, the next message is
- * a DEAD naming the same hop, far node, and `gone` whose `moved` continues this one's, and the
- * receiver applies them as one. `drain`: `node` sent GOAWAY, so every path through it ranks as
- * draining. */
+ * the node `far` at its other end. The next `moved` messages on the stream are ROUTE_UPDATEs, one
+ * per route on the stream through that hop that continues on another path, and the receiver
+ * applies the DEAD and all of them as one. The rest end if `gone`, and are held if not, unless the
+ * receiver is the node just before the sender on the route's path. `drain`: `node` sent GOAWAY, so
+ * every path through it ranks as draining. */
export type Dead =
- | {
- kind: "closed";
- hop: RouteHop;
- far: bigint;
- gone: boolean;
- more: boolean;
- moved: { id: bigint; offer: Offer | Unusable }[];
- }
+ | { kind: "closed"; hop: RouteHop; far: bigint; gone: boolean; moved: bigint }
| { kind: "drain"; node: bigint };
/** The most Hops a route may carry. One that arrives with this many is unusable. */
@@ -107,7 +102,11 @@ const DOWN_ONLY = 0x1;
const DRAINING = 0x2;
/**
- * A message on the publisher's side of the Announce Stream.
+ * A message on the Announce Stream.
+ *
+ * Lite-07 carries everything one side says about routing on its one unidirectional stream: the
+ * edits of the prefixes it wants (`request`), and the markers (`ok`), routes, and claims answering
+ * the prefixes its peer wants.
*
* On lite-06+ these are independently-typed messages (`ANNOUNCE_START`, `ANNOUNCE_END`,
* `ANNOUNCE_UPDATE`, and on lite-07 `ANNOUNCE_RESTART`, `SYNCED`, the ROUTE messages, and
@@ -120,7 +119,8 @@ const DRAINING = 0x2;
* empty there; every earlier version carries the chain and no route.
*/
export type AnnounceBroadcast =
- /** A broadcast is now available, carrying the path suffix, and (lite-06+) the cost. An absent
+ /** A broadcast is now available, carrying the path suffix, relative to the requested prefix
+ * before lite-07 and to the session from it, and (lite-06+) the cost. An absent
* cost encodes as zero; it decodes as `undefined` on a wire with no room for one. Before
* lite-07 it carries the hop chain; on lite-07 the Route ID, the instance and epoch, which
* change only with a `restart`, and a `pathBase` the suffix follows. */
@@ -150,11 +150,21 @@ export type AnnounceBroadcast =
| { status: "restart"; id: bigint; instance?: Instance.Valid; epoch?: bigint; cost?: Cost; above?: bigint }
/** An unknown lite-06+ announce type, skipped by length. Does not assign an id. */
| { status: "skipped" }
- /** Lite07+: the initial set has been delivered. */
- | { status: "synced" }
+ /** Lite07+ SYNCED: the initial set of the prefix the receiver assigned this Prefix ID has been
+ * delivered. */
+ | { status: "synced"; prefix: bigint }
+ /** Lite07+ ANNOUNCE_REQUEST: the sender edits the prefixes it wants claims for, removing those
+ * with the Prefix IDs in `remove` and then adding each of `add`, which assigns it the next
+ * Prefix ID. With `more`, the edit continues in the next message, and takes effect whole with
+ * its last (see `requestParts`). */
+ | { status: "request"; more: boolean; remove: bigint[]; add: Add[] }
+ /** Lite07+ ANNOUNCE_OK: the receiver's last edit takes effect here. */
+ | { status: "ok" }
/** Lite07+ ROUTE_START: the publisher reaches the publisher node `node` along `offer`, covering
- * the requested prefix joined with `suffix`, and assigns the route the next Route ID. */
- | { status: "routeStart"; node: bigint; suffix: Path.Valid; offer: Offer }
+ * the prefix the receiver assigned the Prefix ID `prefix` joined with `suffix`, and assigns the
+ * route the next Route ID. `exact` marks a route the publisher keeps at its own prefix whatever
+ * prefix the request lies at, so an edit widening the request past it leaves it there. */
+ | { status: "routeStart"; node: bigint; prefix: bigint; suffix: Path.Valid; exact: boolean; offer: Offer }
/** Lite07+ ROUTE_UPDATE: the route with this id changed, and is usable. */
| { status: "routeUpdate"; id: bigint; offer: Offer }
/** Lite07+ ROUTE_HOLD: the publisher has no usable route with this id, but holds it and its
@@ -173,11 +183,19 @@ export type AnnounceBroadcast =
* stream's ids in step; never sent. */
| { status: "refused"; message: Refusable; reason: Refusal };
+/** A prefix an ANNOUNCE_REQUEST adds, relative to the session, and whether it also wants the
+ * hidden claims beneath it. */
+export interface Add {
+ prefix: Path.Valid;
+ hidden: boolean;
+}
+
/** What a refused route message still tells the receiver, so the ids that follow it resolve. */
export type Refusable =
- /** ROUTE_START: assigns the next Route ID to the publisher `node`, covering the requested prefix
- * joined with `suffix`, or no prefix when the suffix itself was refused, along `path`. */
- | { kind: "routeStart"; node: bigint; suffix?: Path.Valid; path: RouteHop[] }
+ /** ROUTE_START: assigns the next Route ID to the publisher `node`, covering the prefix with the
+ * Prefix ID `prefix` joined with `suffix`, or no prefix when the suffix itself was refused,
+ * along `path`, at its own prefix when `exact`. */
+ | { kind: "routeStart"; node: bigint; prefix: bigint; suffix?: Path.Valid; exact: boolean; path: RouteHop[] }
/** ROUTE_UPDATE of the route with this Route ID, now along `path`. */
| { kind: "routeUpdate"; id: bigint; path: RouteHop[] }
/** ANNOUNCE_START under Route ID `route`: assigns the next Announce ID, to `path`, or to no path
@@ -375,14 +393,7 @@ async function encodeDead(w: Writer, dead: Dead) {
await w.u62(dead.hop.session);
await w.u62(dead.far);
await w.u8(dead.gone ? 1 : 0);
- await w.u8(dead.more ? 1 : 0);
- await w.u62(BigInt(dead.moved.length));
- for (const { id, offer } of dead.moved) {
- if ("refused" in offer) throw new Error("a refused offer is never sent");
- await w.u62(id);
- // Sized, so a receiver refusing one offer still reads the next.
- await Message.encode(w, (w) => encodeOffer(w, offer));
- }
+ await w.u62(dead.moved);
}
async function decodeDead(r: Reader): Promise {
@@ -397,13 +408,105 @@ async function decodeDead(r: Reader): Promise {
const far = nonzero(await r.u62());
const gone = await r.u8();
if (gone > 1) throw new ProtocolViolation(`DEAD Gone ${gone}`);
- const more = await r.u8();
- if (more > 1) throw new ProtocolViolation(`DEAD More ${more}`);
- const count = await r.u62();
- const moved: { id: bigint; offer: Offer | Unusable }[] = [];
- // Each entry takes bytes, so the body bounds the count before this loop runs out.
- for (let i = 0n; i < count; i++) moved.push({ id: await r.u62(), offer: await Message.decode(r, decodeOffer) });
- return { kind: "closed", hop, far, gone: gone === 1, more: more === 1, moved };
+ return { kind: "closed", hop, far, gone: gone === 1, moved: await r.u62() };
+}
+
+/** Encode an ANNOUNCE_REQUEST body: `More`, the Prefix IDs to remove, then the prefixes to add,
+ * each with its `Hidden`. */
+async function encodeRequest(w: Writer, more: boolean, remove: bigint[], add: Add[]) {
+ await w.bool(more);
+ await w.u53(remove.length);
+ for (const id of remove) await w.u62(id);
+ await w.u53(add.length);
+ for (const { prefix, hidden } of add) {
+ await w.string(Path.encode(prefix));
+ await w.bool(hidden);
+ }
+}
+
+/** Decode an ANNOUNCE_REQUEST body. The prefixes are the sender's own, so one that is no path
+ * fails the message rather than being refused. Each entry takes bytes, so the body bounds both
+ * counts before either loop runs out. */
+async function decodeRequest(r: Reader): Promise {
+ const more = await r.bool();
+ const remove: bigint[] = [];
+ for (let left = await r.u62(); left > 0n; left--) remove.push(await r.u62());
+ const add: Add[] = [];
+ for (let left = await r.u62(); left > 0n; left--) {
+ add.push({ prefix: Path.decode(await r.string()), hidden: await r.bool() });
+ }
+ return { status: "request", more, remove, add };
+}
+
+/** The bytes one Add of `prefix` takes: its length, its bytes, and its flag. */
+function addSize(prefix: Path.Valid, version: Version): number {
+ const length = new TextEncoder().encode(Path.encode(prefix)).byteLength;
+ return encodeVarint(length, version).byteLength + length + 1;
+}
+
+/** The bytes an ANNOUNCE_REQUEST body takes with `removes` Prefix IDs totalling `ids` bytes and
+ * `adds` Adds totalling `added` bytes. */
+function requestSize(removes: number, ids: number, adds: number, added: number, version: Version): number {
+ return 1 + encodeVarint(removes, version).byteLength + encodeVarint(adds, version).byteLength + ids + added;
+}
+
+/** Whether the prefixes `set` fit the Adds of one ANNOUNCE_REQUEST, as a subscriber's whole set
+ * must. */
+export function requestFits(set: Iterable, version: Version): boolean {
+ let adds = 0;
+ let added = 0;
+ for (const prefix of set) {
+ adds++;
+ added += addSize(prefix, version);
+ }
+ return requestSize(0, 0, adds, added, version) <= MAX_MESSAGE_SIZE;
+}
+
+/**
+ * The ANNOUNCE_REQUESTs of one edit removing the Prefix IDs `remove` and adding `add`: as many as
+ * it takes for each to fit one message, every one but the last with `more`. The receiver applies
+ * them together at the last, so how they split changes nothing. Mirrors `request_parts` in
+ * rs/moq-net.
+ *
+ * @internal
+ */
+export function requestParts(
+ remove: bigint[],
+ add: Add[],
+ version: Version,
+): { status: "request"; more: boolean; remove: bigint[]; add: Add[] }[] {
+ const parts: { remove: bigint[]; add: Add[] }[] = [];
+ let part: { remove: bigint[]; add: Add[] } = { remove: [], add: [] };
+ let ids = 0;
+ let added = 0;
+ const fits = (id: number, adding: number) =>
+ requestSize(
+ part.remove.length + (id > 0 ? 1 : 0),
+ ids + id,
+ part.add.length + (adding > 0 ? 1 : 0),
+ added + adding,
+ version,
+ ) <= MAX_MESSAGE_SIZE;
+ const next = () => {
+ parts.push(part);
+ part = { remove: [], add: [] };
+ ids = 0;
+ added = 0;
+ };
+ for (const id of remove) {
+ const bytes = encodeVarint(id, version).byteLength;
+ if (!fits(bytes, 0)) next();
+ part.remove.push(id);
+ ids += bytes;
+ }
+ for (const one of add) {
+ const bytes = addSize(one.prefix, version);
+ if (!fits(0, bytes)) next();
+ part.add.push(one);
+ added += bytes;
+ }
+ parts.push(part);
+ return parts.map((part, i) => ({ status: "request", more: i + 1 < parts.length, ...part }));
}
/** Decode a lite-07 suffix: its base, which the sender picked, then the path it forwarded, or
@@ -480,7 +583,9 @@ async function encodeAnnounce06Body(w: Writer, msg: AnnounceBroadcast, version:
case "routeStart":
if (msg.node === 0n) throw new Error("Node ID 0 names no origin");
await w.u62(msg.node);
+ await w.u62(msg.prefix);
await w.string(Path.encode(msg.suffix));
+ await w.bool(msg.exact);
await encodeOffer(w, msg.offer);
break;
case "routeUpdate":
@@ -501,6 +606,12 @@ async function encodeAnnounce06Body(w: Writer, msg: AnnounceBroadcast, version:
await w.u62(msg.revision === undefined ? 0n : msg.revision + 1n);
break;
case "synced":
+ await w.u62(msg.prefix);
+ break;
+ case "request":
+ await encodeRequest(w, msg.more, msg.remove, msg.add);
+ break;
+ case "ok":
break;
case "ended":
// The pre-lite-06 path-form retraction has no place on lite-06.
@@ -527,6 +638,8 @@ function announce06Type(msg: AnnounceBroadcast, version: Version): number {
case "synced":
if (!hasSynced(version)) throw new Error("synced not supported for this version");
return SYNCED;
+ case "request":
+ case "ok":
case "routeStart":
case "routeUpdate":
case "routeHold":
@@ -535,6 +648,8 @@ function announce06Type(msg: AnnounceBroadcast, version: Version): number {
case "floor":
if (!hasRoutes(version)) throw new Error("routes not supported for this version");
return {
+ request: ANNOUNCE_REQUEST,
+ ok: ANNOUNCE_OK,
routeStart: ROUTE_START,
routeUpdate: ROUTE_UPDATE,
routeHold: ROUTE_HOLD,
@@ -563,7 +678,13 @@ async function decodeAnnounce06Body(r: Reader, typ: number, version: Version): P
}
case SYNCED:
if (!hasSynced(version)) break;
- return { status: "synced" };
+ return { status: "synced", prefix: await r.u62() };
+ case ANNOUNCE_REQUEST:
+ if (!hasRoutes(version)) break;
+ return decodeRequest(r);
+ case ANNOUNCE_OK:
+ if (!hasRoutes(version)) break;
+ return { status: "ok" };
case ANNOUNCE_END:
return { status: "endedId", id: await r.u62() };
case ANNOUNCE_UPDATE: {
@@ -583,14 +704,20 @@ async function decodeAnnounce06Body(r: Reader, typ: number, version: Version): P
case ROUTE_START: {
if (!hasRoutes(version)) break;
const node = await r.u62();
+ const prefix = await r.u62();
// The suffix is sized, so the offer's path reads even when it is refused.
const text = utf8(await r.read(await r.u53()));
const suffix = text === undefined ? undefined : Path.tryDecode(text);
+ const exact = await r.bool();
const offer = await decodeOffer(r);
if (node !== 0n && suffix !== undefined && !("refused" in offer))
- return { status: "routeStart", node, suffix, offer };
+ return { status: "routeStart", node, prefix, suffix, exact, offer };
const reason = node === 0n ? "node" : suffix === undefined ? "path" : (offer as Unusable).refused;
- return { status: "refused", message: { kind: "routeStart", node, suffix, path: offer.path }, reason };
+ return {
+ status: "refused",
+ message: { kind: "routeStart", node, prefix, suffix, exact, path: offer.path },
+ reason,
+ };
}
case ROUTE_UPDATE: {
if (!hasRoutes(version)) break;
@@ -741,6 +868,14 @@ export class AnnounceHistory {
return live;
}
+ /** Assign the next id to `path` with no ANNOUNCE_START, as an edit does for a claim it presents
+ * at a prefix below its own. */
+ insert(path: Path.Valid): bigint {
+ const id = this.#next++;
+ this.#live.set(id, path);
+ return id;
+ }
+
/** An ANNOUNCE_END: retire the id, returning its suffix, or undefined if it was refused. */
end(id: bigint): Path.Valid | undefined {
if (this.#refused.delete(id)) return undefined;
@@ -751,8 +886,9 @@ export class AnnounceHistory {
}
/**
- * ANNOUNCE_REQUEST: sent by the subscriber to request ANNOUNCE_BROADCAST messages
- * for a path prefix. Renamed from `AnnounceInterest` in lite-05.
+ * ANNOUNCE_REQUEST before lite-07: sent by the subscriber to request ANNOUNCE_BROADCAST messages
+ * for a path prefix, opening a bidirectional Announce Stream. Renamed from `AnnounceInterest` in
+ * lite-05. Lite-07 edits a set of prefixes on its one stream instead (the `request` message).
*/
export class AnnounceRequest {
prefix: Path.Valid;
@@ -761,15 +897,14 @@ export class AnnounceRequest {
* "no exclusion". Not on the wire elsewhere, so a value set here is ignored when
* encoding for another version and decodes as unknown. */
excludeHop: Hop;
- /** Lite07+: also announce routes with a `.`-prefixed segment below the prefix. Not on
- * the wire earlier, so a value set here is ignored when encoding for an older version
- * and decodes as false. */
- hidden: boolean;
- constructor(prefix: Path.Valid, excludeHop: Hop = UNKNOWN_HOP, hidden = false) {
+ constructor(prefix: Path.Valid, excludeHop: Hop = UNKNOWN_HOP) {
this.prefix = prefix;
this.excludeHop = excludeHop;
- this.hidden = hidden;
+ }
+
+ static #guard(version: Version) {
+ if (hasRoutes(version)) throw new Error("announce request streams not supported for this version");
}
async #encode(w: Writer, version: Version) {
@@ -777,23 +912,21 @@ export class AnnounceRequest {
if (hasExcludeHop(version)) {
await w.u62(wireHop(this.excludeHop, version));
}
- if (hasHidden(version)) {
- await w.bool(this.hidden);
- }
}
static async #decode(r: Reader, version: Version): Promise {
const prefix = Path.decode(await r.string());
const excludeHop = hasExcludeHop(version) ? toHop(await r.u62()) : UNKNOWN_HOP;
- const hidden = hasHidden(version) ? await r.bool() : false;
- return new AnnounceRequest(prefix, excludeHop, hidden);
+ return new AnnounceRequest(prefix, excludeHop);
}
async encode(w: Writer, version: Version): Promise {
+ AnnounceRequest.#guard(version);
return Message.encode(w, (w) => this.#encode(w, version));
}
static async decode(r: Reader, version: Version): Promise {
+ AnnounceRequest.#guard(version);
return Message.decode(r, (r) => AnnounceRequest.#decode(r, version));
}
}
@@ -853,13 +986,13 @@ export class AnnounceInit {
}
/// Sent by the publisher as the first message on an announce stream, before any
-/// individual Announce messages. Lite05+ only; the successor to AnnounceInit.
+/// individual Announce messages. Lite05/06 only; the successor to AnnounceInit. Lite-07 answers
+/// on its one stream instead (the `ok` message).
///
/// `origin` is the responder's Hop ID, which the subscriber stamps onto each
/// announce's hop chain (the publisher no longer stamps itself), or the reserved
-/// {@link UNKNOWN_HOP} when the responder has no identity to give. Lite05/06
-/// carry `active`, the initial start count. Lite-07's is empty: claims name their route
-/// instead of a hop chain, and SYNCED marks the initial set, so both decode as unknown.
+/// {@link UNKNOWN_HOP} when the responder has no identity to give. `active` is the
+/// initial start count.
export class AnnounceOk {
hop: Hop;
active: number;
@@ -870,25 +1003,22 @@ export class AnnounceOk {
}
static #guard(version: Version) {
- if (!hasAnnounceOk(version)) {
+ if (!hasAnnounceOk(version) || hasRoutes(version)) {
throw new Error("announce ok not supported for this version");
}
}
async #encode(w: Writer, version: Version) {
- if (hasRoutes(version)) return;
await w.u62(wireHop(this.hop, version));
- if (!hasSynced(version)) await w.u53(this.active);
+ await w.u53(this.active);
}
- static async #decode(r: Reader, version: Version): Promise {
- if (hasRoutes(version)) return new AnnounceOk(UNKNOWN_HOP, 0);
+ static async #decode(r: Reader): Promise {
// The draft reserves 0 for "unknown": the responder was never assigned an id, or
// withholds it to obscure its routing. It names nobody, so callers must not stamp
// it onto a hop chain, but it is a legal message and not grounds to drop the stream.
const origin = toHop(await r.u62());
- const active = hasSynced(version) ? 0 : await r.u53();
- return new AnnounceOk(origin, active);
+ return new AnnounceOk(origin, await r.u53());
}
async encode(w: Writer, version: Version): Promise {
@@ -898,6 +1028,6 @@ export class AnnounceOk {
static async decode(r: Reader, version: Version): Promise {
AnnounceOk.#guard(version);
- return Message.decode(r, (r) => AnnounceOk.#decode(r, version));
+ return Message.decode(r, AnnounceOk.#decode);
}
}
diff --git a/js/net/src/lite/announce_stream.test.ts b/js/net/src/lite/announce_stream.test.ts
new file mode 100644
index 00000000000..2cb882345d0
--- /dev/null
+++ b/js/net/src/lite/announce_stream.test.ts
@@ -0,0 +1,53 @@
+import { expect, test } from "bun:test";
+import { createMockTransportPair } from "../mock.ts";
+import { AnnounceStream } from "./announce_stream.ts";
+import { Version } from "./version.ts";
+
+const version = Version.DRAFT_07;
+
+// Finishing right behind a send, before that send opened the stream, still finishes the stream
+// the send opened, so the peer sees the FIN.
+test("finishing behind an unwritten send finishes the stream it opens", async () => {
+ const { client, server } = createMockTransportPair();
+ try {
+ const stream = AnnounceStream.of(client, version);
+ const sent = stream.send({ status: "ok" });
+ const finished = stream.finish();
+ const incoming = server.incomingUnidirectionalStreams.getReader();
+ const { value } = await incoming.read();
+ if (!value) throw new Error("no stream");
+ const reader = value.getReader();
+ for (;;) {
+ const { done } = await reader.read();
+ if (done) break;
+ }
+ await sent;
+ await finished;
+ } finally {
+ client.close();
+ server.close();
+ }
+});
+
+// A send that fails loses what this endpoint said: `broken` reports it, and a stream that never
+// failed never settles it.
+test("a failed send breaks the stream", async () => {
+ const { client, server } = createMockTransportPair();
+ try {
+ const stream = AnnounceStream.of(client, version);
+ let broken = false;
+ void stream.broken.then(() => {
+ broken = true;
+ });
+ await stream.send({ status: "ok" });
+ await Promise.resolve();
+ expect(broken).toBe(false);
+
+ client.close();
+ await expect(stream.send({ status: "ok" })).rejects.toThrow();
+ await stream.broken;
+ expect(broken).toBe(true);
+ } finally {
+ server.close();
+ }
+});
diff --git a/js/net/src/lite/announce_stream.ts b/js/net/src/lite/announce_stream.ts
new file mode 100644
index 00000000000..c651801b45c
--- /dev/null
+++ b/js/net/src/lite/announce_stream.ts
@@ -0,0 +1,86 @@
+// Lite-07: this endpoint's one unidirectional Announce Stream, on which it sends everything it
+// says about discovery and routing in order: as a subscriber the edits of the prefixes it asks
+// for, and as a publisher the routes and claims answering what its peer asks for.
+
+import { Writer } from "../stream.ts";
+import { type AnnounceBroadcast, encodeAnnounceBroadcast } from "./announce.ts";
+import { DataType } from "./stream.ts";
+import type { Version } from "./version.ts";
+
+const streams = new WeakMap();
+
+/** A session's outgoing Announce Stream, opened with the first message it sends. @internal */
+export class AnnounceStream {
+ #quic: WebTransport;
+ #version: Version;
+ #writer?: Promise;
+ // The last write queued, which the next one follows.
+ #tail: Promise = Promise.resolve();
+ #finished = false;
+ #break!: (err: unknown) => void;
+
+ /**
+ * Settles with the first send that failed: what this endpoint said is lost, and the session
+ * with it.
+ */
+ readonly broken: Promise;
+
+ private constructor(quic: WebTransport, version: Version) {
+ this.#quic = quic;
+ this.#version = version;
+ this.broken = new Promise((resolve) => {
+ this.#break = resolve;
+ });
+ }
+
+ /** The session's one stream, shared by the half that asks and the half that answers. */
+ static of(quic: WebTransport, version: Version): AnnounceStream {
+ let stream = streams.get(quic);
+ if (!stream) {
+ stream = new AnnounceStream(quic, version);
+ streams.set(quic, stream);
+ }
+ return stream;
+ }
+
+ #open(): Promise {
+ this.#writer ??= (async () => {
+ const writer = await Writer.open(this.#quic, { version: this.#version });
+ await writer.u53(DataType.Announce);
+ return writer;
+ })();
+ return this.#writer;
+ }
+
+ /**
+ * Send `msgs` back to back, after everything sent before them, so nothing lands between a
+ * route's claims and the floor closing their batch. Rejects once the stream finished or failed.
+ */
+ send(...msgs: AnnounceBroadcast[]): Promise {
+ if (this.#finished) return Promise.reject(new Error("the Announce Stream finished"));
+ const sent = this.#tail.then(async () => {
+ const writer = await this.#open();
+ for (const msg of msgs) await encodeAnnounceBroadcast(writer, msg, this.#version);
+ });
+ this.#tail = sent.catch((err: unknown) => this.#break(err));
+ return sent;
+ }
+
+ /**
+ * This side is leaving the session: finish the stream once everything sent is written, and
+ * wait for the peer to take it. A stream that never opened stays unopened.
+ */
+ finish(): Promise {
+ this.#finished = true;
+ const done = this.#tail.then(async () => {
+ // Read once the sends queued before finishing are written, which may have opened it.
+ const opened = this.#writer;
+ if (!opened) return;
+ const writer = await opened;
+ writer.close();
+ await writer.closed;
+ });
+ this.#tail = done.catch(() => {});
+ return done;
+ }
+}
diff --git a/js/net/src/lite/answer.ts b/js/net/src/lite/answer.ts
new file mode 100644
index 00000000000..722188f4105
--- /dev/null
+++ b/js/net/src/lite/answer.ts
@@ -0,0 +1,387 @@
+// Lite-07: what this session sends on its Announce Stream as a publisher, answering the prefixes
+// the peer asks for from this origin's own route.
+//
+// A browser forwards nothing, so each requested prefix carries at most one route, this origin's,
+// and only while it shows the peer one of its claims. The peer edits its prefixes in place: each
+// edit takes effect at the ANNOUNCE_OK answering it, where both ends carry the stream's routes
+// over alike (`rekey.plan`), and the answer then sends only what the carried routes lack. Mirrors
+// `publisher/routed.rs` in rs/moq-net for a node that does not proxy.
+
+import { type GetPromise, type Getter, race } from "@moq/signals";
+import type { Grant } from "../auth.ts";
+import { ProtocolViolation } from "../error.ts";
+import { routesEqual, type WireRoute } from "../hop.ts";
+import { coveringPrefixes, hiddenBelow, presented } from "../internal.ts";
+import * as Path from "../path.ts";
+import { type Advertised, type Advertisements, type OriginConsumer, sameInstance } from "../wire.ts";
+import { type Add, type AnnounceBroadcast, type Offer, requestFits } from "./announce.ts";
+import type { AnnounceStream } from "./announce_stream.ts";
+import * as rekey from "./rekey.ts";
+import type { Version } from "./version.ts";
+
+/** What the session answers its peer from. */
+export interface Publish {
+ version: Version;
+ /** The origin's claims, or undefined once it closed. */
+ advertised: Getter;
+ /** The origin's route identity and scope, when the session publishes one. */
+ origin?: Pick;
+ /** Our grant: only what it lets us publish is announced. */
+ grant?: Getter;
+ /** Resolves once the tokens this session presented at setup are answered, so nothing is
+ * announced before the grant it would be checked against. */
+ ready: Promise;
+ /** Settles when the session starts leaving. */
+ closing: GetPromise;
+ stream: AnnounceStream;
+}
+
+/** A prefix the peer asked for. */
+interface Prefix {
+ path: Path.Valid;
+ hidden: boolean;
+}
+
+/** The route the peer holds for a requested prefix: from the first claim shown until the last one goes. */
+interface Sent {
+ id: bigint;
+ node: bigint;
+ /** Whether its last floor certified, so a grant change alone floors it again. Undefined until
+ * its first floor, and once an edit widened it. */
+ certified?: boolean;
+ /** The claims the peer holds, by path. */
+ live: Map;
+}
+
+// What the peer decodes for a claim: lite-07 carries its instance, epoch, and cost.
+const onWire = (route: WireRoute): WireRoute => ({ ...route, auth: undefined });
+const restarted = (prev: Advertised, next: Advertised): boolean =>
+ !sameInstance(prev, next) || (prev.route.epoch ?? 0n) !== (next.route.epoch ?? 0n);
+
+/** Answers the prefixes a lite-07 peer asks for. @internal */
+export class Answer {
+ #publish: Publish;
+ /** The prefixes in effect, by Prefix ID. */
+ #prefixes = new Map();
+ #nextPrefix = 0n;
+ /** The peer's edit not put in effect yet. */
+ #request?: { remove: bigint[]; add: [bigint, Prefix][] };
+ /** The ANNOUNCE_REQUESTs of an edit that continues: its removes and its adds so far. */
+ #parts?: { remove: bigint[]; add: Add[] };
+ /** The route the peer holds for each requested prefix. */
+ #routes = new Map();
+ #nextRoute = 0n;
+ #nextAnnounce = 0n;
+ /** The session is leaving: every route was ended, and nothing more is sent. */
+ #withdrawn = false;
+ #closed = false;
+
+ // Anything that may change what the peer is owed marks the answer dirty and wakes the loop,
+ // which alone writes. A change landing mid-sync leaves it dirty for the next pass.
+ #dirty = true;
+ #notify?: () => void;
+ #running?: Promise;
+
+ constructor(publish: Publish) {
+ this.#publish = publish;
+ }
+
+ #poke() {
+ this.#dirty = true;
+ this.#notify?.();
+ }
+
+ /** Whether the peer's edit continues in its next message, which must be its next ANNOUNCE_REQUEST. */
+ continuing(): boolean {
+ return this.#parts !== undefined;
+ }
+
+ /**
+ * ANNOUNCE_REQUEST: the peer edits the prefixes it asks for, over as many messages as `more`
+ * says. Returns the loop answering it, started by the first request and running until the
+ * session leaves.
+ */
+ request(more: boolean, remove: bigint[], add: Add[]): Promise {
+ this.#running ??= this.#run();
+ if (this.#withdrawn || this.#closed) return this.#running;
+ // The peer sends one edit at a time.
+ if (this.#request) throw new ProtocolViolation("ANNOUNCE_REQUEST before the last was answered");
+ if (this.#parts) {
+ remove = [...this.#parts.remove, ...remove];
+ add = [...this.#parts.add, ...add];
+ this.#parts = undefined;
+ }
+ // Parts that could never make a valid edit fail before more of them pile up: their Adds
+ // alone already pass the set's limit, or they remove more than the set holds.
+ if (
+ remove.length > this.#prefixes.size ||
+ !requestFits(
+ add.map(({ prefix }) => prefix),
+ this.#publish.version,
+ )
+ )
+ throw new ProtocolViolation("ANNOUNCE_REQUEST parts past the limit");
+ if (more) {
+ this.#parts = { remove, add };
+ return this.#running;
+ }
+ const left = new Map(this.#prefixes);
+ for (const id of remove) {
+ if (!left.delete(id)) throw new ProtocolViolation(`ANNOUNCE_REQUEST removes unknown Prefix ID ${id}`);
+ }
+ const set = new Set();
+ for (const prefix of left.values()) set.add(prefix.path);
+ for (const { prefix } of add) {
+ if (set.has(prefix)) throw new ProtocolViolation(`ANNOUNCE_REQUEST adds ${prefix} twice`);
+ set.add(prefix);
+ }
+ // No prefix in the set lies below another, and the whole set fits one message.
+ for (const prefix of set) {
+ for (const above of coveringPrefixes(prefix)) {
+ if (above !== prefix && set.has(above))
+ throw new ProtocolViolation(`ANNOUNCE_REQUEST nests ${prefix} below ${above}`);
+ }
+ }
+ if (!requestFits(set, this.#publish.version)) throw new ProtocolViolation("ANNOUNCE_REQUEST set too large");
+ // Each added prefix takes the next Prefix ID, in the order the edit names them.
+ const added = add.map(({ prefix, hidden }): [bigint, Prefix] => [this.#nextPrefix++, { path: prefix, hidden }]);
+ this.#request = { remove, add: added };
+ this.#notify?.();
+ return this.#running;
+ }
+
+ /** The session ended: stop answering. */
+ close() {
+ this.#closed = true;
+ this.#notify?.();
+ }
+
+ async #run(): Promise {
+ const { advertised, grant, origin, closing, ready } = this.#publish;
+ // Woken by the origin's claims, our grant, or the identity they are originated under.
+ const stops = [advertised, grant, origin?.own.identity].map((source) => source?.subscribe(() => this.#poke()));
+ try {
+ const settled = ready.then(() => "ready" as const);
+ if ((await race([settled, closing])) !== "ready") return;
+ for (;;) {
+ if (this.#closed) return;
+ if (closing.peek()) return await this.#withdraw();
+ if (this.#request) {
+ await this.#edit(this.#request);
+ continue;
+ }
+ if (this.#dirty) {
+ await this.#syncAll();
+ continue;
+ }
+ const woke = new Promise((resolve) => {
+ this.#notify = resolve;
+ });
+ await race([woke, closing]);
+ this.#notify = undefined;
+ }
+ } finally {
+ for (const stop of stops) stop?.();
+ }
+ }
+
+ // The session is leaving: end every route, so the peer lets go of them before the stream finishes.
+ async #withdraw() {
+ this.#withdrawn = true;
+ const ids = [...this.#routes.values()].map((sent) => sent.id).sort((a, b) => (a < b ? -1 : 1));
+ this.#routes.clear();
+ if (ids.length > 0) await this.#publish.stream.send(...ids.map((id) => ({ status: "routeEnd", id }) as const));
+ }
+
+ // Put the peer's request in effect: answer it, carry the routes it holds over, send what they
+ // lack, and close each added prefix's initial set.
+ async #edit(request: { remove: bigint[]; add: [bigint, Prefix][] }) {
+ const old = new Set();
+ for (const prefix of this.#prefixes.values()) old.add(prefix.path);
+ for (const id of request.remove) this.#prefixes.delete(id);
+ const added: Path.Valid[] = [];
+ for (const [id, prefix] of request.add) {
+ this.#prefixes.set(id, prefix);
+ added.push(prefix.path);
+ }
+ const next = new Set();
+ for (const prefix of this.#prefixes.values()) next.add(prefix.path);
+ this.#rekey(old, next, added);
+
+ // The peer may send its next edit once this is on its way, and not before.
+ const answered = this.#publish.stream.send({ status: "ok" });
+ this.#request = undefined;
+ await answered;
+ await this.#syncAll();
+ // Every route the peer holds in an added prefix has its floor by now.
+ const synced = request.add.map(([prefix]) => ({ status: "synced", prefix }) as const);
+ if (synced.length > 0) await this.#publish.stream.send(...synced);
+ }
+
+ // Carry the routes and claims the peer holds over an edit of the prefixes from `old` to
+ // `next`, as the peer does at the ANNOUNCE_OK.
+ #rekey(old: Set, next: Set, added: Path.Valid[]) {
+ const table = new Map();
+ const sources = new Map();
+ const snaps = new Map();
+ // Every route here is this origin's own along one offer, so they rank alike whoever the peer is.
+ const order = rekey.order(false, this.#offer(), this.#offer().path, 0n);
+ for (const [prefix, sent] of this.#routes) {
+ const claims = new Map();
+ for (const [path, { id, snap }] of sent.live) {
+ claims.set(path, id);
+ snaps.set(id, snap);
+ }
+ table.set(sent.id, { node: sent.node, prefix, region: prefix, exact: false, order, claims });
+ sources.set(sent.id, sent);
+ }
+ const plan = rekey.plan(old, next, added, table, this.#nextRoute, this.#nextAnnounce);
+ this.#routes = new Map();
+ for (const carried of plan.routes) {
+ const source = sources.get(carried.from);
+ if (!source) throw new Error("a carried route's source");
+ const live = new Map();
+ for (const [path, claim] of carried.claims) {
+ const from = snaps.get(claim.from);
+ if (!from) throw new Error("a carried claim's state");
+ // A claim now covering a narrower prefix from above is that much more above it.
+ const snap =
+ claim.lowered > 0n
+ ? { ...from, route: { ...from.route, above: (from.route.above ?? 0n) + claim.lowered } }
+ : from;
+ live.set(path, { id: claim.id, snap });
+ }
+ this.#routes.set(carried.prefix, {
+ id: carried.id,
+ node: source.node,
+ // A widened route certifies nothing until its next floor.
+ certified: carried.kind === "widens" ? undefined : source.certified,
+ live,
+ });
+ }
+ this.#nextRoute = plan.nextRoute;
+ this.#nextAnnounce = plan.nextClaim;
+ }
+
+ // The origin's own route: a path of its own hop alone, learned on no session, at the top
+ // priority and no cost. It forwards no other. Mirrors the origin's offer in rs/moq-net.
+ #offer(): Offer {
+ const origin = this.#publish.origin;
+ return {
+ path: origin ? [{ node: origin.own.hop, session: 0n }] : [],
+ draining: false,
+ priority: 127,
+ metric: 0n,
+ downOnly: false,
+ };
+ }
+
+ async #syncAll() {
+ this.#dirty = false;
+ const ids = [...this.#prefixes.keys()].sort((a, b) => (a < b ? -1 : 1));
+ for (const id of ids) {
+ const prefix = this.#prefixes.get(id);
+ if (!prefix) continue;
+ const msgs = this.#sync(id, prefix);
+ if (msgs.length > 0) await this.#publish.stream.send(...msgs);
+ }
+ }
+
+ // Bring the peer's view of one requested prefix in line with the origin's: one batch of claim
+ // changes, closed by its floor, ending the route once it shows nothing.
+ #sync(id: bigint, prefix: Prefix): AnnounceBroadcast[] {
+ const { advertised, origin } = this.#publish;
+ const msgs: AnnounceBroadcast[] = [];
+ // Read together, so the floor certifies exactly the claims it closes.
+ const table = advertised.peek();
+ const identity = origin?.own.identity.peek();
+ const revision = origin?.own.revision();
+ const grant = this.#publish.grant?.peek();
+ // The claims the peer is shown, by their path from the session's root: the table under the
+ // prefix, less the hidden ones it did not ask for and whatever our grant does not let us
+ // publish.
+ const latest = new Map();
+ if (table && identity) {
+ const carries = (covered: Path.Valid) => prefix.hidden || !hiddenBelow(prefix.path, covered);
+ // A claim covering the prefix from above keeps how many segments broader than it it is.
+ for (const [suffix, snap] of presented(prefix.path, table, carries, true)) {
+ const path = Path.join(prefix.path, suffix);
+ if (!grant || grant.publish.matches(path)) latest.set(path, snap);
+ }
+ }
+ // Whether a floor may certify the whole requested prefix: nothing the handle's scope or our
+ // grant leaves out lies beneath it, since the peer cannot tell what was left out from what
+ // is gone. A hidden claim left out does not count: the peer did not ask to see it.
+ const certifies =
+ (!grant || grant.publish.contains(Path.Pattern.subtree(prefix.path))) &&
+ (origin?.covers(prefix.path) ?? true);
+
+ // End the route on the wire: it retires every claim under it.
+ const end = () => {
+ const sent = this.#routes.get(prefix.path);
+ if (!sent) return;
+ this.#routes.delete(prefix.path);
+ msgs.push({ status: "routeEnd", id: sent.id });
+ };
+
+ // A rotated identity ends the old route with every claim under it.
+ if (this.#routes.get(prefix.path)?.node !== identity?.node) end();
+ if (!identity) return msgs;
+
+ let route = this.#routes.get(prefix.path);
+ let changed = false;
+ for (const [path, live] of [...(route?.live ?? [])]) {
+ if (latest.has(path)) continue;
+ route?.live.delete(path);
+ changed = true;
+ console.debug(`announce: broadcast=${path} active=false`);
+ msgs.push({ status: "endedId", id: live.id });
+ }
+ for (const [path, snap] of latest) {
+ const prev = route?.live.get(path);
+ if (prev && !restarted(prev.snap, snap) && routesEqual(onWire(prev.snap.route), onWire(snap.route))) {
+ continue;
+ }
+ changed = true;
+ if (!route) {
+ route = { id: this.#nextRoute++, node: identity.node, live: new Map() };
+ this.#routes.set(prefix.path, route);
+ // The origin's own route covers every path, so the requested prefix too, from above
+ // it: not exact.
+ msgs.push({
+ status: "routeStart",
+ node: identity.node,
+ prefix: id,
+ suffix: Path.empty(),
+ exact: false,
+ offer: this.#offer(),
+ });
+ }
+ const { instance, epoch, cost } = snap.route;
+ // How much broader the claim is than the path it is presented at.
+ const above = snap.route.above ?? 0n;
+ if (!prev) {
+ const claim = this.#nextAnnounce++;
+ route.live.set(path, { id: claim, snap });
+ console.debug(`announce: broadcast=${path} active=true instance=${instance}`);
+ // Every claim's path is relative to the session, whichever prefix it answers.
+ msgs.push({ status: "active", suffix: path, route: route.id, instance, epoch, cost, above });
+ } else if (restarted(prev.snap, snap)) {
+ route.live.set(path, { id: prev.id, snap });
+ console.debug(`announce: broadcast=${path} restart=true instance=${instance} epoch=${epoch}`);
+ msgs.push({ status: "restart", id: prev.id, instance, epoch, cost, above });
+ } else {
+ route.live.set(path, { id: prev.id, snap });
+ msgs.push({ status: "update", id: prev.id, cost, above });
+ }
+ }
+ if (!route) return msgs;
+ if (changed || route.certified !== certifies) {
+ route.certified = certifies;
+ msgs.push({ status: "floor", id: route.id, revision: certifies ? revision : undefined });
+ }
+ // A route an edit carried over with nothing to show ends like one whose last claim went.
+ if (route.live.size === 0) end();
+ return msgs;
+ }
+}
diff --git a/js/net/src/lite/connection.test.ts b/js/net/src/lite/connection.test.ts
index 88e89d64336..a7327806dc8 100644
--- a/js/net/src/lite/connection.test.ts
+++ b/js/net/src/lite/connection.test.ts
@@ -12,6 +12,8 @@ import * as Path from "../path.ts";
import { acceptBudget, Stream, Writer } from "../stream.ts";
import { Timescale } from "../time.ts";
import { wireOf } from "../wire.ts";
+import { type AnnounceBroadcast, encodeAnnounceBroadcast } from "./announce.ts";
+import { AnnounceStream } from "./announce_stream.ts";
import { Connection, probeLevel } from "./connection.ts";
import { Fetch } from "./fetch.ts";
import { Goaway } from "./goaway.ts";
@@ -776,7 +778,8 @@ test("close waits for a request served while withdrawals are in flight", async (
const deadline = mockDeadline();
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
- const fin = holdFins(pair.server, "incoming-bidi");
+ // Lite-07 withdraws on its unidirectional Announce Stream.
+ const fin = holdFins(pair.server, "uni");
const source = new Producer();
const destination = new Producer();
const broadcastRoute = source.route(Path.from("room"));
@@ -1030,3 +1033,201 @@ for (const violation of ["a partial group", "a group past the declared end"] as
},
);
}
+
+// Lite-07: each endpoint sends everything it says about discovery and routing on one
+// unidirectional Announce Stream, which the peer may only finish.
+async function announceStream(pair: { server: MockTransport }) {
+ const writer = await Writer.open(pair.server, { version: Version.DRAFT_07 });
+ await writer.u53(DataType.Announce);
+ const send = (msg: AnnounceBroadcast) => encodeAnnounceBroadcast(writer, msg, Version.DRAFT_07);
+ return { writer, send };
+}
+
+const ROOM = Path.from("room");
+
+for (const [name, violate] of [
+ [
+ "a second Announce Stream",
+ async (pair) => {
+ await announceStream(pair);
+ await announceStream(pair);
+ },
+ ],
+ [
+ "a reset Announce Stream",
+ async (pair) => {
+ const { writer, send } = await announceStream(pair);
+ await send({ status: "request", more: false, remove: [], add: [{ prefix: ROOM, hidden: false }] });
+ writer.reset(new Error("gone"));
+ },
+ ],
+ [
+ "an Announce Stream finished inside a message",
+ async (pair) => {
+ const { writer } = await announceStream(pair);
+ // An ANNOUNCE_REQUEST of five bytes, of which one arrives.
+ await writer.write(new Uint8Array([16, 5, 0]));
+ writer.close();
+ },
+ ],
+ [
+ "an ANNOUNCE_REQUEST nesting one prefix below another",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({
+ status: "request",
+ more: false,
+ remove: [],
+ add: [
+ { prefix: ROOM, hidden: false },
+ { prefix: Path.from("room/cam"), hidden: false },
+ ],
+ });
+ },
+ ],
+ [
+ "an ANNOUNCE_REQUEST adding a prefix below one already asked for",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({ status: "request", more: false, remove: [], add: [{ prefix: ROOM, hidden: false }] });
+ await send({
+ status: "request",
+ more: false,
+ remove: [],
+ add: [{ prefix: Path.from("room/cam"), hidden: false }],
+ });
+ },
+ ],
+ [
+ "an ANNOUNCE_REQUEST removing an unknown Prefix ID",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({ status: "request", more: false, remove: [3n], add: [] });
+ },
+ ],
+ [
+ "a message inside an edit",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({ status: "request", more: true, remove: [], add: [{ prefix: ROOM, hidden: false }] });
+ await send({ status: "ok" });
+ },
+ ],
+ [
+ "the parts of an edit nesting one prefix below another",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({ status: "request", more: true, remove: [], add: [{ prefix: ROOM, hidden: false }] });
+ await send({
+ status: "request",
+ more: false,
+ remove: [],
+ add: [{ prefix: Path.from("room/cam"), hidden: false }],
+ });
+ },
+ ],
+ [
+ "a DEAD moving a route the stream never started",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({
+ status: "dead",
+ dead: { kind: "closed", hop: { node: 2n, session: 1n }, far: 3n, gone: false, moved: 1n },
+ });
+ },
+ ],
+ [
+ "an ANNOUNCE_OK answering no ANNOUNCE_REQUEST",
+ async (pair) => {
+ const { send } = await announceStream(pair);
+ await send({ status: "ok" });
+ },
+ ],
+] as [string, (pair: { server: MockTransport }) => Promise][]) {
+ test(`${name} closes moq-lite-07 with PROTOCOL_VIOLATION`, async () => {
+ const pair = createMockTransportPair(ALPN_07_WIP);
+ const origin = new Producer();
+ const connection = new Connection({
+ url: new URL("https://relay.example/"),
+ quic: pair.client,
+ version: Version.DRAFT_07,
+ publish: origin.consume(),
+ });
+ try {
+ await violate(pair);
+ expect((await pair.client.closed).closeCode).toBe(SessionCode.ProtocolViolation);
+ } finally {
+ connection.abort();
+ origin.close();
+ }
+ });
+}
+
+// The peer finishing its Announce Stream is leaving: its routes end, the session stays, and the
+// prefixes it asked for are still answered.
+test("a finished Announce Stream ends the peer's routes and keeps moq-lite-07 answering", async () => {
+ const version = Version.DRAFT_07;
+ const pair = createMockTransportPair(ALPN_07_WIP);
+ const up = new Producer();
+ const down = new Producer();
+ const fromClient = new Producer();
+ const fromServer = new Producer();
+ const url = new URL("https://localhost/test");
+ const [client, server] = await Promise.all([
+ connect({ url, transport: pair.client, publish: up.consume(), consume: fromServer }),
+ accept({ url, transport: pair.server, publish: down.consume(), consume: fromClient }),
+ ]);
+ const cam = up.route(Path.from("up/cam"));
+ const room = down.route(Path.from("down/room"));
+ const atServer = fromClient.routes();
+ const atClient = fromServer.routes();
+ let closed = false;
+ void pair.server.closed.then(() => {
+ closed = true;
+ });
+ try {
+ expect(await atServer.next()).toMatchObject({ prefix: Path.from("up/cam"), kind: "start" });
+ expect(await atClient.next()).toMatchObject({ prefix: Path.from("down/room"), kind: "start" });
+
+ // The client finishes its stream, as it does when it leaves.
+ await AnnounceStream.of(pair.client, version).finish();
+ expect(await atServer.next()).toMatchObject({ prefix: Path.from("up/cam"), kind: "offline" });
+ expect(closed).toBe(false);
+
+ // What the client asked for is still answered.
+ const late = down.route(Path.from("down/late"));
+ expect(await atClient.next()).toMatchObject({ prefix: Path.from("down/late"), kind: "start" });
+ expect(closed).toBe(false);
+ late.broadcast.close();
+ } finally {
+ atServer.close();
+ atClient.close();
+ client.abort();
+ server.abort();
+ cam.broadcast.close();
+ room.broadcast.close();
+ for (const origin of [up, down, fromClient, fromServer]) origin.close();
+ }
+});
+
+// Lite-07 has no bidirectional Announce stream: its type is one the version does not know,
+// which resets the stream and keeps the session.
+test("a bidirectional stream of the Announce type is unknown to moq-lite-07", async () => {
+ const version = Version.DRAFT_07;
+ const pair = createMockTransportPair(ALPN_07_WIP);
+ const connection = new Connection({ url: new URL("https://relay.example/"), quic: pair.client, version });
+ let closed = false;
+ void pair.client.closed.then(() => {
+ closed = true;
+ });
+ try {
+ const stream = await Stream.open(pair.server, { version });
+ await stream.writer.u53(StreamId.Announce);
+ await stream.writer.write(new Uint8Array([1, 0]));
+ await expect(stream.reader.readAll()).rejects.toThrow();
+ await settle();
+ expect(closed).toBe(false);
+ } finally {
+ connection.abort();
+ }
+});
diff --git a/js/net/src/lite/connection.ts b/js/net/src/lite/connection.ts
index 3e3ed288aed..bd8fb35f268 100644
--- a/js/net/src/lite/connection.ts
+++ b/js/net/src/lite/connection.ts
@@ -19,7 +19,8 @@ import { type Reader, Readers, Stream, Writer } from "../stream.ts";
import { Observed } from "../util/observed.ts";
import { withTimeout } from "../util/timeout.ts";
import { registerWire, wireOf } from "../wire.ts";
-import { AnnounceRequest } from "./announce.ts";
+import { type AnnounceBroadcast, AnnounceRequest, decodeAnnounceBroadcastMaybe } from "./announce.ts";
+import { AnnounceStream } from "./announce_stream.ts";
import { LiteAuthWire } from "./auth.ts";
import { Datagrams } from "./datagram.ts";
import { closeViolation } from "./error.ts";
@@ -139,6 +140,10 @@ export class Connection implements Session {
// until the peer's SETUP arrives; stays undefined forever on older drafts.
#peerSetup = new Signal(undefined);
#setupSeen = false;
+ // Lite-07: the peer opened its one Announce Stream.
+ #announceSeen = false;
+ // The transport closed, so a stream breaking says nothing about the peer.
+ #transportClosed = false;
// Mirrors the role out of #peerSetup, so the public surface exposes the peer's declared
// direction without handing out the whole SETUP (whose probe level gates our own streams).
@@ -227,8 +232,20 @@ export class Connection implements Session {
});
// Left before anything observes the close, so a successor on the same handle never
// finds this session still active.
- const leave = () => this.#auth.close();
+ const leave = () => {
+ this.#transportClosed = true;
+ this.#auth.close();
+ };
void quic.closed.then(leave, leave);
+ // Lite-07: a send on this endpoint's Announce Stream failed while the session lives, so
+ // what it said is lost and the session cannot go on, as rs/moq-net closes it. An ending
+ // session's own end says why.
+ if (hasRoutes(version)) {
+ void AnnounceStream.of(quic, version).broken.then((err) => {
+ if (this.#transportClosed || this.#phase.kind === "closed") return;
+ this.#fail(error(err));
+ });
+ }
// Before lite-07 the peer can't ask for hidden routes, so a client publishes none of
// them, and a socket endpoint behind it is unreachable. A server leaves it to the
// client, whose origin holds that endpoint.
@@ -265,6 +282,7 @@ export class Connection implements Session {
registerWire(this, {
consume: (path, instance) => this.#subscriber.consume(path, instance),
covers: (node, at, path) => this.#subscriber.covers(node, at, path),
+ coverage: this.#subscriber.coverage,
exclusive: false,
leave: () => this.#leave(),
auth: this.#auth,
@@ -305,6 +323,8 @@ export class Connection implements Session {
switch (this.#phase.kind) {
case "open":
case "leaving": {
+ // Leaving finishes our Announce Stream, which asks the peer for nothing more.
+ this.#subscriber.announceSilenced();
const done = withTimeout(
this.#publisher.drain(),
Duration.millis(1000),
@@ -478,7 +498,8 @@ export class Connection implements Session {
// opened without negotiating it: a plain error resets the stream and keeps the session.
if (typ === StreamId.Session) {
throw new ProtocolViolation("duplicate session stream");
- } else if (typ === StreamId.Announce) {
+ } else if (typ === StreamId.Announce && !hasRoutes(this.#version)) {
+ // Lite-07 asks on its Announce Stream instead, and knows no such stream type.
const msg = await AnnounceRequest.decode(stream.reader, this.#version);
await this.#publisher.runAnnounce(msg, stream);
} else if (typ === StreamId.Subscribe) {
@@ -566,6 +587,11 @@ export class Connection implements Session {
await this.#subscriber.runGroup(msg, stream);
} else if (typ === DataType.Padding && hasPadding(this.#version)) {
await stream.discard();
+ } else if (typ === DataType.Announce && hasRoutes(this.#version)) {
+ // Claim the stream before reading it, so two that say nothing yet are duplicates too.
+ if (this.#announceSeen) throw new ProtocolViolation("duplicate Announce Stream");
+ this.#announceSeen = true;
+ await this.#runAnnounce(stream);
} else if (typ === DataType.Setup) {
// Claim the stream before decoding, so two incomplete SETUPs are duplicates too.
if (this.#setupSeen) throw new ProtocolViolation("duplicate SETUP");
@@ -589,6 +615,31 @@ export class Connection implements Session {
}
}
+ // Lite-07: everything the peer says about discovery and routing, in order, until it leaves.
+ // The peer may only finish the stream: a reset or a message cut short breaks the session.
+ async #runAnnounce(stream: Reader) {
+ for (;;) {
+ let msg: AnnounceBroadcast | undefined;
+ try {
+ msg = await decodeAnnounceBroadcastMaybe(stream, this.#version);
+ } catch (err: unknown) {
+ // While the session lives, whatever broke the stream, a reset, a message cut
+ // short, or one that does not decode, lost what the peer said. An ending
+ // session's own end says why.
+ if (this.#transportClosed || this.#phase.kind === "closed") throw err;
+ throw new ProtocolViolation("Announce Stream broke", { cause: err });
+ }
+ if (!msg) break;
+ // Nothing comes between the ANNOUNCE_REQUESTs of one edit.
+ if (msg.status !== "request" && this.#publisher.continuing())
+ throw new ProtocolViolation("a message inside an edit");
+ await this.#subscriber.announce(msg);
+ if (msg.status === "request") this.#publisher.request(msg.more, msg.remove, msg.add);
+ }
+ // The peer is leaving: its routes end, and what it asked for is still answered.
+ this.#subscriber.announceFinished();
+ }
+
/** Snapshot the transport's counters; see {@link Session.stats}. */
async stats(): Promise {
const stats = await transportStats(this.#quic);
diff --git a/js/net/src/lite/corpus.test.ts b/js/net/src/lite/corpus.test.ts
index 2abb51b6abb..0196bacb115 100644
--- a/js/net/src/lite/corpus.test.ts
+++ b/js/net/src/lite/corpus.test.ts
@@ -37,13 +37,6 @@ async function decode(c: Case, r: Reader, v: Version): Promise<((w: Writer) => P
case "announce": {
const msg = await decodeAnnounceBroadcast(r, v);
if (msg.status === "refused") return "withdrawn";
- // A DEAD withdraws each route a refused moved offer names.
- if (
- msg.status === "dead" &&
- msg.dead.kind === "closed" &&
- msg.dead.moved.some(({ offer }) => "refused" in offer)
- )
- return "withdrawn";
return (w) => encodeAnnounceBroadcast(w, msg, v);
}
case "announce_init": {
diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts
index a4f33c9dc93..27ccc00ef39 100644
--- a/js/net/src/lite/publisher.test.ts
+++ b/js/net/src/lite/publisher.test.ts
@@ -14,15 +14,16 @@ import { Producer as OriginProducer } from "../origin.ts";
import * as Path from "../path.ts";
import { sendOrder } from "../priority.ts";
import { Front } from "../resume.ts";
-import { Reader, Stream, Writer, writeBudget } from "../stream.ts";
+import { Reader, Readers, Stream, Writer, writeBudget } from "../stream.ts";
import { Timescale, Timestamp } from "../time.ts";
import { Subscriber as TrackSubscriber } from "../track.ts";
import { wireOf } from "../wire.ts";
-import { type AnnounceBroadcast, AnnounceOk, AnnounceRequest, decodeAnnounceBroadcast } from "./announce.ts";
+import { type AnnounceBroadcast, AnnounceRequest, decodeAnnounceBroadcast } from "./announce.ts";
import { Fetch } from "./fetch.ts";
import { Group as GroupMessage } from "./group.ts";
import { Probe as ProbeMessage } from "./probe.ts";
import { Publisher } from "./publisher.ts";
+import { DataType } from "./stream.ts";
import {
decodeSubscribeResponse,
decodeSubscribeResponseMaybe,
@@ -150,60 +151,49 @@ test.each([
pair.server.close();
});
-test.each([Version.DRAFT_06, Version.DRAFT_07])(
- "a republish restarts the advertisement (version %s)",
- async (version) => {
- using clock = manual();
- const pair = createMockTransportPair(ALPN_05);
- const origin = new OriginProducer();
- const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
- const old = publish(origin, Path.from("cam"));
+test.each([Version.DRAFT_06])("a republish restarts the advertisement (version %s)", async (version) => {
+ using clock = manual();
+ const pair = createMockTransportPair(ALPN_05);
+ const origin = new OriginProducer();
+ const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
+ const old = publish(origin, Path.from("cam"));
- const written: Uint8Array[] = [];
- const stream = new Stream({
- version: version,
- readable: new ReadableStream(),
- writable: new WritableStream({
- write(chunk) {
- written.push(new Uint8Array(chunk));
- },
- }),
- });
- const settle = () => clock.advance(Duration.ZERO);
- const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
- await settle();
- const initial = written.length;
+ const written: Uint8Array[] = [];
+ const stream = new Stream({
+ version: version,
+ readable: new ReadableStream(),
+ writable: new WritableStream({
+ write(chunk) {
+ written.push(new Uint8Array(chunk));
+ },
+ }),
+ });
+ const settle = () => clock.advance(Duration.ZERO);
+ const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
+ await settle();
+ const initial = written.length;
- // Another broadcast at the path, without an instance: another instance.
- const republished = publish(origin, Path.from("cam"));
- await settle();
- const bytes = Uint8Array.from(written.slice(initial).flatMap((chunk) => [...chunk]));
- const reader = new Reader(undefined, bytes, version);
- const messages: AnnounceBroadcast[] = [];
- while (!(await reader.done())) messages.push(await decodeAnnounceBroadcast(reader, version));
- if (version === Version.DRAFT_07) {
- // One batch under the route, closed by its floor.
- expect(messages).toMatchObject([
- { status: "restart", id: 0n },
- { status: "floor", id: 0n },
- ]);
- } else {
- expect(messages).toMatchObject([
- { status: "endedId", id: 0n },
- { status: "active", suffix: "cam" },
- ]);
- }
+ // Another broadcast at the path, without an instance: another instance.
+ const republished = publish(origin, Path.from("cam"));
+ await settle();
+ const bytes = Uint8Array.from(written.slice(initial).flatMap((chunk) => [...chunk]));
+ const reader = new Reader(undefined, bytes, version);
+ const messages: AnnounceBroadcast[] = [];
+ while (!(await reader.done())) messages.push(await decodeAnnounceBroadcast(reader, version));
+ expect(messages).toMatchObject([
+ { status: "endedId", id: 0n },
+ { status: "active", suffix: "cam" },
+ ]);
- stream.close();
- await running;
- publisher.close();
- republished.close();
- old.close();
- origin.close();
- pair.client.close();
- pair.server.close();
- },
-);
+ stream.close();
+ await running;
+ publisher.close();
+ republished.close();
+ old.close();
+ origin.close();
+ pair.client.close();
+ pair.server.close();
+});
// Delivers `sequences` in the given order, finishes the track (with `error`, if given), and
// returns the SUBSCRIBE_FINISH the publisher put on the wire.
@@ -3736,63 +3726,58 @@ test("lite-07 SYNCED survives an initial removal and continuous earlier-prefix c
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
const origin = new OriginProducer();
- const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
const removed = publish(origin, Path.from("z/removed"));
const retained = publish(origin, Path.from("z/retained"));
const entered = Promise.withResolvers();
const release = Promise.withResolvers();
- let output!: ReadableStreamDefaultController;
- const reader = new Reader(
- new ReadableStream({
- start(controller) {
- output = controller;
- },
- }),
- undefined,
- version,
- );
let writes = 0;
let churnEnabled = true;
const churn: BroadcastProducer[] = [];
- const stream = new Stream({
- version,
- readable: new ReadableStream(),
- writable: new WritableStream({
+ // The Announce Stream stalls on its first write, and every write churns an earlier prefix.
+ const create = pair.server.createUnidirectionalStream.bind(pair.server);
+ pair.server.createUnidirectionalStream = async (options) => {
+ const writer = (await create(options)).getWriter();
+ return new WritableStream({
async write(chunk) {
if (writes++ === 0) {
entered.resolve();
await release.promise;
}
- output.enqueue(new Uint8Array(chunk));
+ await writer.write(chunk);
if (churnEnabled) {
churn.push(publish(origin, Path.from(`a/${writes}`)));
if (churn.length > 2) churn.shift()?.close();
}
},
- }),
- });
- const running = publisher.runAnnounce(new AnnounceRequest(Path.empty()), stream);
+ close: () => writer.close(),
+ abort: (reason) => writer.abort(reason),
+ }) as Awaited>;
+ };
+ const publisher = new Publisher(pair.server, version, randomHop(), origin.consume());
+ const readers = new Readers(pair.client, version);
+ publisher.request(false, [], [{ prefix: Path.empty(), hidden: false }]);
try {
await entered.promise;
removed.close();
release.resolve();
- expect((await AnnounceOk.decode(reader, version)).active).toBe(0);
+ const reader = await readers.next();
+ if (!reader) throw new Error("the publisher never opened its Announce Stream");
+ expect(await reader.u53()).toBe(DataType.Announce);
+ expect(await decodeAnnounceBroadcast(reader, version)).toEqual({ status: "ok" });
const events: AnnounceBroadcast[] = [];
- for (let i = 0; i < 5; i++) {
+ for (let i = 0; i < 8; i++) {
const event = await decodeAnnounceBroadcast(reader, version);
events.push(event);
if (event.status === "synced") break;
}
// The route, its claims, and their floor, then SYNCED.
expect(events[0]).toMatchObject({ status: "routeStart" });
- expect(events[1]).toMatchObject({ status: "active", suffix: "z/retained" });
+ expect(events).toContainEqual(expect.objectContaining({ status: "active", suffix: "z/retained" }));
expect(events.at(-2)).toMatchObject({ status: "floor", id: 0n });
- expect(events.at(-1)).toEqual({ status: "synced" });
+ expect(events.at(-1)).toEqual({ status: "synced", prefix: 0n });
expect(events).not.toContainEqual(expect.objectContaining({ suffix: "z/removed" }));
} finally {
churnEnabled = false;
- await publisher.drain();
- await running;
publisher.close();
retained.close();
for (const broadcast of churn) broadcast.close();
@@ -3966,31 +3951,37 @@ for (const version of [Version.DRAFT_05, Version.DRAFT_06]) {
});
}
-// A lite-07 announce stream the test reads as the subscriber, and answers.
+// A lite-07 publisher whose Announce Stream the test reads as the subscriber, having asked for
+// `prefix` with the first ANNOUNCE_REQUEST.
function routedAnswer(origin: OriginProducer, grant?: Signal, prefix = Path.empty()) {
const version = Version.DRAFT_07;
const pair = createMockTransportPair(ALPN_07_WIP);
const auth = grant && { grant, ready: Promise.resolve() };
const publisher = new Publisher(pair.server, version, randomHop(), origin.consume(), auth);
- let replies!: ReadableStreamDefaultController;
- let output!: ReadableStreamDefaultController;
- const reader = new Reader(new ReadableStream({ start: (c) => (output = c) }), undefined, version);
- const stream = new Stream({
- version,
- readable: new ReadableStream({ start: (c) => (replies = c) }),
- writable: new WritableStream({ write: (chunk) => output.enqueue(new Uint8Array(chunk)) }),
- });
- const running = publisher.runAnnounce(new AnnounceRequest(prefix), stream);
+ const readers = new Readers(pair.client, version);
+ const reader = (async () => {
+ const stream = await readers.next();
+ if (!stream) throw new Error("the publisher never opened its Announce Stream");
+ expect(await stream.u53()).toBe(DataType.Announce);
+ return stream;
+ })();
+ publisher.request(false, [], [{ prefix, hidden: false }]);
+ const next = async () => decodeAnnounceBroadcast(await reader, version);
return {
node: wireOf(origin.consume()).own.node,
hop: wireOf(origin.consume()).own.hop,
- running,
- next: () => decodeAnnounceBroadcast(reader, version),
- // The peer sends bytes after its request, which no message is.
- send: (bytes: Uint8Array) => replies.enqueue(bytes),
- ok: () => AnnounceOk.decode(reader, version),
- // The peer finishes its replies, which ends the stream.
- finish: () => replies.close(),
+ publisher,
+ next,
+ // The next ANNOUNCE_REQUEST, as the session hands the publisher each one the peer sends.
+ request: (remove: bigint[], add: Path.Valid[]) =>
+ publisher.request(
+ false,
+ remove,
+ add.map((prefix) => ({ prefix, hidden: false })),
+ ),
+ ok: async () => expect(await next()).toEqual({ status: "ok" }),
+ // Whether the stream finished: the publisher left the session.
+ finished: async () => (await reader).done(),
close: () => {
publisher.close();
pair.client.close();
@@ -4002,7 +3993,7 @@ function routedAnswer(origin: OriginProducer, grant?: Signal,
test("a lite-07 answer carries this origin's claims under its own route", async () => {
const origin = new OriginProducer();
const cam = publish(origin, Path.from("cam"));
- const { node, hop, running, next, ok, close } = routedAnswer(origin);
+ const { node, hop, next, ok, close } = routedAnswer(origin);
// The origin's own route covers every path, along its own hop alone, at the top priority.
const offer = {
path: [{ node: hop, session: 0n }],
@@ -4013,10 +4004,17 @@ test("a lite-07 answer carries this origin's claims under its own route", async
};
try {
await ok();
- expect(await next()).toEqual({ status: "routeStart", node, suffix: Path.empty(), offer });
+ expect(await next()).toEqual({
+ status: "routeStart",
+ node,
+ prefix: 0n,
+ suffix: Path.empty(),
+ exact: false,
+ offer,
+ });
expect(await next()).toMatchObject({ status: "active", suffix: "cam", route: 0n });
expect(await next()).toEqual({ status: "floor", id: 0n, revision: 0n });
- expect(await next()).toEqual({ status: "synced" });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
// Each change is a batch closed by the next revision.
const mic = publish(origin, Path.from("mic"));
@@ -4034,45 +4032,180 @@ test("a lite-07 answer carries this origin's claims under its own route", async
// The next claim starts the route again at once, under the next Route ID: no echo waits.
const again = publish(origin, Path.from("again"));
- expect(await next()).toEqual({ status: "routeStart", node, suffix: Path.empty(), offer });
+ expect(await next()).toEqual({
+ status: "routeStart",
+ node,
+ prefix: 0n,
+ suffix: Path.empty(),
+ exact: false,
+ offer,
+ });
expect(await next()).toMatchObject({ status: "active", suffix: "again", route: 1n });
expect(await next()).toMatchObject({ status: "floor", id: 1n });
again.close();
} finally {
close();
origin.close();
- await running.catch(() => {});
}
});
-// The subscriber sends nothing after its request: its FIN ends the answer cleanly, and any byte,
-// such as a lite-07-wip ROUTE_REQUEST from an older build, is a violation.
-test("a lite-07 answer ends on the subscriber's FIN and closes on any reply", async () => {
+// Another broadcast at the path, without an instance, is another instance: one batch under the
+// route, closed by its floor.
+test("a lite-07 republish restarts the claim", async () => {
const origin = new OriginProducer();
- const cam = publish(origin, Path.from("cam"));
- const first = routedAnswer(origin);
- const second = routedAnswer(origin);
+ const old = publish(origin, Path.from("cam"));
+ const { next, ok, close } = routedAnswer(origin);
try {
- for (const { ok, next } of [first, second]) {
- await ok();
- expect(await next()).toMatchObject({ status: "routeStart" });
- expect(await next()).toMatchObject({ status: "active", suffix: "cam" });
- expect(await next()).toMatchObject({ status: "floor" });
- expect(await next()).toEqual({ status: "synced" });
- }
+ await ok();
+ expect(await next()).toMatchObject({ status: "routeStart" });
+ expect(await next()).toMatchObject({ status: "active", suffix: "cam", route: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 0n });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
- first.finish();
- await first.running;
- // The other stream goes on as it was: no session's close changes the origin's own route.
- const mic = publish(origin, Path.from("mic"));
- expect(await second.next()).toMatchObject({ status: "active", suffix: "mic" });
+ const republished = publish(origin, Path.from("cam"));
+ expect(await next()).toMatchObject({ status: "restart", id: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 0n });
+ republished.close();
+ } finally {
+ old.close();
+ close();
+ origin.close();
+ }
+});
+
+// The peer edits the prefixes it asks for one ANNOUNCE_REQUEST at a time. Each added prefix takes
+// the next Prefix ID, its route names it, and its paths are the session's, whatever the prefix.
+test("a lite-07 edit answers each added prefix under its own Prefix ID", async () => {
+ const origin = new OriginProducer();
+ const cam = publish(origin, Path.from("a/cam"));
+ const mic = publish(origin, Path.from("b/mic"));
+ const { node, next, ok, request, close } = routedAnswer(origin, undefined, Path.from("a"));
+ try {
+ await ok();
+ expect(await next()).toMatchObject({ status: "routeStart", node, prefix: 0n, suffix: "" });
+ expect(await next()).toMatchObject({ status: "active", suffix: "a/cam", route: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 0n });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
+
+ // Adding `b` leaves the route for `a` as it stands: nothing of it is sent again.
+ request([], [Path.from("b")]);
+ await ok();
+ expect(await next()).toMatchObject({ status: "routeStart", node, prefix: 1n, suffix: "" });
+ expect(await next()).toMatchObject({ status: "active", suffix: "b/mic", route: 1n });
+ expect(await next()).toMatchObject({ status: "floor", id: 1n });
+ expect(await next()).toEqual({ status: "synced", prefix: 1n });
+
+ // Removing `a` ends its route with the answer, which names nothing: both ends drop it there.
+ request([0n], []);
+ await ok();
+ cam.close();
+ const late = publish(origin, Path.from("b/late"));
+ expect(await next()).toMatchObject({ status: "active", suffix: "b/late", route: 1n });
+ expect(await next()).toMatchObject({ status: "floor", id: 1n });
+ late.close();
+ } finally {
+ cam.close();
mic.close();
+ close();
+ origin.close();
+ }
+});
+
+// An edit swapping two prefixes for the one covering both re-sends nothing both sets cover: the
+// routes widen at the answer on both ends, each keeping its Route ID and its claims but the one
+// at its own old prefix, and the publisher sends only a floor for each.
+test("a lite-07 swap edit re-sends nothing both sets cover", async () => {
+ const origin = new OriginProducer();
+ const cam = publish(origin, Path.from("room/a/cam"));
+ const mic = publish(origin, Path.from("room/b/mic"));
+ const { next, ok, request, close } = routedAnswer(origin, undefined, Path.from("room/a"));
+ try {
+ await ok();
+ expect(await next()).toMatchObject({ status: "routeStart", prefix: 0n });
+ expect(await next()).toMatchObject({ status: "active", suffix: "room/a/cam", route: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 0n });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
- second.send(Uint8Array.of(0xa, 2, 7, 2));
- await expect(second.running).rejects.toThrow(ProtocolViolation);
+ // `room` covers `room/a`: the route widens, and only what it lacks follows the answer.
+ request([0n], [Path.from("room")]);
+ await ok();
+ const widened: AnnounceBroadcast[] = [];
+ for (;;) {
+ const msg = await next();
+ widened.push(msg);
+ if (msg.status === "synced") break;
+ }
+ expect(widened).toMatchObject([
+ { status: "active", suffix: "room/b/mic", route: 0n },
+ { status: "floor", id: 0n },
+ { status: "synced", prefix: 1n },
+ ]);
+
+ // Back to `room/a`: the route narrows under the next Route ID, its claim there carried
+ // over with what its floor certified, and the claim outside it dropped on both ends.
+ request([1n], [Path.from("room/a")]);
+ await ok();
+ const narrowed: AnnounceBroadcast[] = [];
+ for (;;) {
+ const msg = await next();
+ narrowed.push(msg);
+ if (msg.status === "synced") break;
+ }
+ expect(narrowed).toEqual([{ status: "synced", prefix: 2n }]);
+ // The carried claim kept its Announce ID, under the route's new one.
+ cam.close();
+ expect(await next()).toEqual({ status: "endedId", id: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 1n });
+ expect(await next()).toEqual({ status: "routeEnd", id: 1n });
} finally {
cam.close();
- for (const answer of [first, second]) answer.close();
+ mic.close();
+ close();
+ origin.close();
+ }
+});
+
+// Each edit the peer may not send is a violation, which the session closes on.
+test("a lite-07 edit the peer may not send is a violation", async () => {
+ const origin = new OriginProducer();
+ const answers: ReturnType[] = [];
+ const answer = async (prefix: string) => {
+ const made = routedAnswer(origin, undefined, Path.from(prefix));
+ answers.push(made);
+ await made.ok();
+ expect(await made.next()).toEqual({ status: "synced", prefix: 0n });
+ return made;
+ };
+ try {
+ // A Prefix ID never assigned, and one removed twice.
+ const unknown = await answer("a");
+ expect(() => unknown.request([1n], [])).toThrow(ProtocolViolation);
+ expect(() => unknown.request([0n, 0n], [])).toThrow(ProtocolViolation);
+ // A prefix at or below another in the set the edit leaves, in either order.
+ const nested = await answer("a");
+ expect(() => nested.request([], [Path.from("a/b")])).toThrow(ProtocolViolation);
+ expect(() => nested.request([], [Path.empty()])).toThrow(ProtocolViolation);
+ expect(() => nested.request([], [Path.from("a")])).toThrow(ProtocolViolation);
+ expect(() => nested.request([], [Path.from("b"), Path.from("b/c")])).toThrow(ProtocolViolation);
+ // Removed and added again in one edit is no nesting.
+ nested.request([0n], [Path.from("a")]);
+ await nested.ok();
+ expect(await nested.next()).toEqual({ status: "synced", prefix: 1n });
+ // A second edit before the first is answered.
+ const early = await answer("a");
+ early.request([], [Path.from("b")]);
+ expect(() => early.request([], [Path.from("c")])).toThrow(ProtocolViolation);
+ // A set past what one message holds.
+ const large = await answer("a");
+ const long = (i: number) => Path.from(`${String(i).padStart(4, "0")}/${"x".repeat(1000)}`);
+ expect(() =>
+ large.request(
+ [],
+ Array.from({ length: 66 }, (_, i) => long(i)),
+ ),
+ ).toThrow(ProtocolViolation);
+ } finally {
+ for (const made of answers) made.close();
origin.close();
}
});
@@ -4082,14 +4215,14 @@ test("a lite-07 answer ends on the subscriber's FIN and closes on any reply", as
test("a lite-07 rotated identity ends the route and starts the next", async () => {
const origin = new OriginProducer();
const cam = publish(origin, Path.from("cam"));
- const { node, running, next, ok, close } = routedAnswer(origin);
+ const { node, next, ok, close } = routedAnswer(origin);
const own = wireOf(origin.consume()).own;
try {
await ok();
expect(await next()).toMatchObject({ status: "routeStart", node });
expect(await next()).toMatchObject({ status: "active", suffix: "cam", route: 0n });
expect(await next()).toMatchObject({ status: "floor", id: 0n });
- expect(await next()).toEqual({ status: "synced" });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
own.rotate();
expect(own.node).not.toBe(node);
@@ -4101,7 +4234,6 @@ test("a lite-07 rotated identity ends the route and starts the next", async () =
cam.close();
close();
origin.close();
- await running.catch(() => {});
}
});
@@ -4111,13 +4243,13 @@ test("a lite-07 grant change alone re-floors the route", async () => {
const origin = new OriginProducer();
const cam = publish(origin, Path.from("a/x"));
const grant = new Signal(grantOf("a/x"));
- const { running, next, ok, close } = routedAnswer(origin, grant);
+ const { next, ok, close } = routedAnswer(origin, grant);
try {
await ok();
expect(await next()).toMatchObject({ status: "routeStart" });
expect(await next()).toMatchObject({ status: "active", suffix: "a/x", route: 0n });
expect(await next()).toEqual({ status: "floor", id: 0n, revision: undefined });
- expect(await next()).toEqual({ status: "synced" });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
grant.set({ ...grantOf("a/x"), publish: new Path.Patterns([Path.Pattern.subtree(Path.empty())]) });
expect(await next()).toEqual({ status: "floor", id: 0n, revision: 0n });
@@ -4127,56 +4259,79 @@ test("a lite-07 grant change alone re-floors the route", async () => {
cam.close();
close();
origin.close();
- await running.catch(() => {});
}
});
// A request prefix that fits the wire alone may overflow once joined to the handle's root:
-// nothing the scope allows lies there, so the floor certifies nothing rather than failing.
+// nothing the scope allows lies there, so the floor certifies nothing rather than failing. The
+// claim at room covers more than was asked for, so it is presented at the requested prefix.
test("a lite-07 request beyond the scope's path limits floors uncertified", async () => {
const origin = new OriginProducer();
const scoped = origin.scope(Path.from("tenant"), new Path.Patterns([Path.Pattern.parse("room/**")]));
// The claim at room covers the request, so the answer floors it.
const room = publish(scoped, Path.from("room"));
const prefix = Path.from(["room", ...Array.from({ length: Path.MAX_PARTS - 1 }, () => "a")].join("/"));
- const { running, next, ok, close } = routedAnswer(scoped, undefined, prefix);
+ const { next, ok, close } = routedAnswer(scoped, undefined, prefix);
try {
await ok();
expect(await next()).toMatchObject({ status: "routeStart" });
- expect(await next()).toMatchObject({ status: "active", suffix: "", route: 0n });
+ expect(await next()).toMatchObject({ status: "active", suffix: prefix, route: 0n });
expect(await next()).toEqual({ status: "floor", id: 0n, revision: undefined });
- expect(await next()).toEqual({ status: "synced" });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
} finally {
room.close();
close();
origin.close();
- await running.catch(() => {});
}
});
-// The publishing origin closing ends its claims and route, and finishes the stream.
-test("a lite-07 answer finishes once the publishing origin closes", async () => {
+// The publishing origin closing ends its claims and its route. The stream stays: only this side
+// leaving the session finishes it.
+test("a lite-07 answer ends its route once the publishing origin closes", async () => {
const origin = new OriginProducer();
const cam = publish(origin, Path.from("cam"));
- const { running, next, ok, close } = routedAnswer(origin);
+ const { next, ok, close } = routedAnswer(origin);
try {
await ok();
expect(await next()).toMatchObject({ status: "routeStart" });
expect(await next()).toMatchObject({ status: "active", suffix: "cam", route: 0n });
expect(await next()).toMatchObject({ status: "floor", id: 0n });
- expect(await next()).toEqual({ status: "synced" });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
origin.close();
expect(await next()).toEqual({ status: "endedId", id: 0n });
expect(await next()).toMatchObject({ status: "floor", id: 0n });
expect(await next()).toEqual({ status: "routeEnd", id: 0n });
- await running;
} finally {
cam.close();
close();
}
});
+// Leaving the session ends every route and then finishes the Announce Stream, which is how the
+// peer learns this side is going.
+test("a lite-07 answer ends its routes and finishes the stream as the session leaves", async () => {
+ const origin = new OriginProducer();
+ const cam = publish(origin, Path.from("cam"));
+ const { publisher, next, ok, finished, close } = routedAnswer(origin);
+ try {
+ await ok();
+ expect(await next()).toMatchObject({ status: "routeStart" });
+ expect(await next()).toMatchObject({ status: "active", suffix: "cam", route: 0n });
+ expect(await next()).toMatchObject({ status: "floor", id: 0n });
+ expect(await next()).toEqual({ status: "synced", prefix: 0n });
+
+ const drained = publisher.drain();
+ expect(await next()).toEqual({ status: "routeEnd", id: 0n });
+ expect(await finished()).toBe(true);
+ await drained;
+ } finally {
+ cam.close();
+ close();
+ origin.close();
+ }
+});
+
test("a lite-07 request naming another origin node is unroutable", async () => {
const pair = createMockTransportPair(ALPN_07_WIP);
const origin = new OriginProducer();
diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts
index 2bcc23536ed..013721698e0 100644
--- a/js/net/src/lite/publisher.ts
+++ b/js/net/src/lite/publisher.ts
@@ -20,7 +20,7 @@ import {
unauthorized,
} from "../error.ts";
import type * as group from "../group.ts";
-import { Cost, type Hop, routesEqual, UNKNOWN_HOP, type WireRoute } from "../hop.ts";
+import { Cost, type Hop, routesEqual, type WireRoute } from "../hop.ts";
import type * as Instance from "../instance.ts";
import {
type DropRun,
@@ -50,14 +50,9 @@ import { max, min } from "../util/sequence.ts";
import { Budget } from "../util/yield.ts";
import type { Resolved } from "../window.ts";
import { type Advertised, type Advertisements, sameInstance, wireOf } from "../wire.ts";
-import {
- type AnnounceBroadcast,
- AnnounceInit,
- AnnounceOk,
- type AnnounceRequest,
- encodeAnnounceBroadcast,
- type Offer,
-} from "./announce.ts";
+import { type Add, AnnounceInit, AnnounceOk, type AnnounceRequest, encodeAnnounceBroadcast } from "./announce.ts";
+import { AnnounceStream } from "./announce_stream.ts";
+import { Answer } from "./answer.ts";
import { Datagram as DatagramMessage, type Datagrams, DatagramsInfo, DatagramsUpdate } from "./datagram.ts";
import { closeViolation } from "./error.ts";
import type { Fetch } from "./fetch.ts";
@@ -560,6 +555,10 @@ export class Publisher {
// announced before the grant it would be checked against.
#ready: Promise;
+ // Lite-07: answers the prefixes the peer asks for, and its loop once the peer asked.
+ #answer?: Answer;
+ #answering?: Promise;
+
// TRACK_INFO is immutable per track, so resolve it from the application once
// (via a throwaway subscribe whose info() resolves when the app calls accept)
// and reuse it for every later TRACK request of the same track. Keyed by the
@@ -606,6 +605,17 @@ export class Publisher {
const origin = publish && wireOf(publish);
this.#advertised = origin?.advertised ?? new Signal(new Map());
this.#publish = publish;
+ if (hasRoutes(version)) {
+ this.#answer = new Answer({
+ version,
+ advertised: this.#advertised,
+ origin,
+ grant: this.#grant,
+ ready: this.#ready,
+ closing: this.#withdrawal.closing,
+ stream: AnnounceStream.of(quic, version),
+ });
+ }
// Grab the datagram writer up front when the transport carries datagrams (no group
// fallback, so it stays undefined otherwise). One writer for all subscriptions.
@@ -615,7 +625,33 @@ export class Publisher {
}
/**
- * Handles an announce interest message.
+ * Whether the peer's edit continues in its next message, which must be its next ANNOUNCE_REQUEST.
+ *
+ * @internal
+ */
+ continuing(): boolean {
+ return this.#answer?.continuing() ?? false;
+ }
+
+ /**
+ * Lite-07 ANNOUNCE_REQUEST: the peer edits the prefixes it asks for, answered on this
+ * session's Announce Stream. Throws a {@link ProtocolViolation} for an edit the peer may not send.
+ *
+ * @internal
+ */
+ request(more: boolean, remove: bigint[], add: Add[]): void {
+ if (!this.#answer) throw new ProtocolViolation("ANNOUNCE_REQUEST before lite-07");
+ const started = this.#answering === undefined;
+ const answering = this.#answer.request(more, remove, add);
+ if (!started) return;
+ this.#answering = this.#withdrawal.track(answering);
+ // The session is ending under it, or its Announce Stream broke, which closes it: its close
+ // says why.
+ this.#answering.catch((err: unknown) => console.debug("announce stream failed:", err));
+ }
+
+ /**
+ * Handles a pre-lite-07 announce interest message.
* @param msg - The announce interest message
* @param stream - The stream to write announcements to
*
@@ -628,7 +664,6 @@ export class Publisher {
async #runAnnounce(msg: AnnounceRequest, stream: Stream) {
if (this.#withdrawal.closing.peek()) return;
console.debug(`announce: prefix=${msg.prefix}`);
- if (hasRoutes(this.version)) return this.#runRouted(msg, stream);
// Keyed by suffix, valued by identity plus route, so a republish diffs as a restart
// and a re-price as an update.
@@ -706,8 +741,8 @@ export class Publisher {
await encodeAnnounceBroadcast(stream.writer, { status: "endedId", id }, this.version);
};
- // A hidden route stays off the wire unless the request opted in.
- const carries = (covered: Path.Valid) => msg.hidden || !hiddenBelow(msg.prefix, covered);
+ // A hidden route stays off the wire: only a lite-07 request can opt in.
+ const carries = (covered: Path.Valid) => !hiddenBelow(msg.prefix, covered);
// What the peer currently sees: the table under the prefix, less whatever our grant
// does not let us publish.
@@ -815,190 +850,6 @@ export class Publisher {
}
}
- // Lite-07: answer the peer's request from this origin's own route. A browser forwards nothing,
- // so the stream carries at most one route, this origin's, and only while it shows the peer one
- // of its claims. Mirrors `publisher/routed.rs` in rs/moq-net for a node that does not proxy.
- async #runRouted(msg: AnnounceRequest, stream: Stream) {
- const origin = this.#publish && wireOf(this.#publish);
-
- // The route while the peer holds it: from the first claim shown until the last one goes.
- // `certified` is whether its last floor certified, so a grant change alone re-floors it.
- let route: { id: bigint; node: bigint; certified?: boolean } | undefined;
- let nextRoute = 0n;
- let nextAnnounce = 0n;
- // The claims the peer holds, by suffix.
- const live = new Map();
-
- // A hidden claim stays off the wire unless the request opted in.
- const carries = (covered: Path.Valid) => msg.hidden || !hiddenBelow(msg.prefix, covered);
- const visible = (table: Advertisements): Map => {
- const out = presented(msg.prefix, table, carries, true);
- const grant = this.#grant?.peek();
- if (grant) {
- for (const suffix of [...out.keys()]) {
- if (!grant.publish.matches(Path.join(msg.prefix, suffix))) out.delete(suffix);
- }
- }
- return out;
- };
- // Whether a floor may certify the whole requested prefix: nothing the handle's scope or our
- // grant leaves out lies beneath it, since the peer cannot tell what was left out from what
- // is gone. A hidden claim left out does not count: the peer did not ask to see it.
- const certified = () => {
- const grant = this.#grant?.peek();
- if (grant && !grant.publish.contains(Path.Pattern.subtree(msg.prefix))) return false;
- return origin?.covers(msg.prefix) ?? true;
- };
-
- const write = (out: AnnounceBroadcast) => encodeAnnounceBroadcast(stream.writer, out, this.version);
-
- // End the route on the wire: it retires every claim under it.
- const end = async () => {
- if (!route) return;
- const { id } = route;
- route = undefined;
- live.clear();
- await write({ status: "routeEnd", id });
- };
-
- // Bring the peer's view in line with the origin's: one batch of claim changes, closed by its
- // floor, ending the route once it shows nothing.
- const sync = async () => {
- // Read together, so the floor certifies exactly the claims it closes.
- const table = this.#advertised.peek();
- const identity = origin?.own.identity.peek();
- const revision = origin?.own.revision();
- const latest = table && identity ? visible(table) : new Map();
- const certifies = certified();
-
- // A rotated identity ends the old route with every claim under it.
- if (route && route.node !== identity?.node) await end();
- if (!identity) return;
-
- let changed = false;
- for (const [suffix, { id }] of [...live]) {
- if (latest.has(suffix)) continue;
- live.delete(suffix);
- changed = true;
- console.debug(`announce: broadcast=${suffix} active=false`);
- await write({ status: "endedId", id });
- }
- for (const [suffix, snap] of latest) {
- const prev = live.get(suffix);
- if (prev && !restarted(prev.snap, snap) && routesEqual(onWire(prev.snap.route), onWire(snap.route)))
- continue;
- changed = true;
- if (!route) {
- route = { id: nextRoute++, node: identity.node };
- // The origin's own route covers every path, so the requested prefix too.
- await write({ status: "routeStart", node: identity.node, suffix: Path.empty(), offer: offer() });
- }
- const { instance, epoch, cost } = snap.route;
- // How much broader the claim is than the prefix it is presented at.
- const above = snap.route.above ?? 0n;
- if (!prev) {
- const id = nextAnnounce++;
- live.set(suffix, { id, snap });
- console.debug(`announce: broadcast=${suffix} active=true instance=${instance}`);
- await write({ status: "active", suffix, route: route.id, instance, epoch, cost, above });
- } else if (restarted(prev.snap, snap)) {
- live.set(suffix, { id: prev.id, snap });
- console.debug(`announce: broadcast=${suffix} restart=true instance=${instance} epoch=${epoch}`);
- await write({ status: "restart", id: prev.id, instance, epoch, cost, above });
- } else {
- live.set(suffix, { id: prev.id, snap });
- await write({ status: "update", id: prev.id, cost, above });
- }
- }
- if (!route || (!changed && route.certified === certifies)) return;
- route.certified = certifies;
- await write({ status: "floor", id: route.id, revision: certifies ? revision : undefined });
- if (live.size === 0) await end();
- };
-
- // The origin's own route: a path of its own hop alone, learned on no session, at the top
- // priority and no cost. It forwards no other. Mirrors the origin's offer in rs/moq-net.
- const offer = (): Offer => ({
- path: origin ? [{ node: origin.own.hop, session: 0n }] : [],
- draining: false,
- priority: 127,
- metric: 0n,
- downOnly: false,
- });
- // What the peer decodes for a claim: lite-07 carries its instance, epoch, cost, and how far above its path it is.
- const onWire = (route: WireRoute): WireRoute => ({ ...route, auth: undefined });
- const restarted = (prev: Advertised, next: Advertised): boolean =>
- !sameInstance(prev, next) || (prev.route.epoch ?? 0n) !== (next.route.epoch ?? 0n);
-
- // Anything that may change what the peer is owed marks the stream dirty and wakes the loop,
- // which alone writes. A change landing mid-sync leaves it dirty for the next pass.
- let dirty = true;
- let notify: (() => void) | undefined;
- const poke = () => {
- dirty = true;
- notify?.();
- };
-
- // The subscriber sends nothing after its ANNOUNCE_REQUEST: a clean FIN ends the stream, and
- // anything else is a violation. Mirrors rs/moq-net.
- let departed: Error | null | undefined;
- void (async () => {
- try {
- if (!(await stream.reader.done())) throw new ProtocolViolation("a message after ANNOUNCE_REQUEST");
- departed = null;
- } catch (err) {
- departed = error(err);
- }
- notify?.();
- })();
-
- // Woken by the origin's claims, our grant, or the identity they are originated under.
- const sources = [this.#advertised, this.#grant, origin?.own.identity];
- const stops = sources.map((source) => source?.subscribe(poke));
- try {
- // Nothing is announced before the grant it would be checked against. A local close
- // meanwhile finishes the stream with nothing announced.
- const ready = this.#ready.then(() => "ready" as const);
- if ((await race([ready, stream.reader.closed, this.#withdrawal.closing])) !== "ready") {
- if (this.#withdrawal.closing.peek()) {
- stream.close();
- await stream.writer.closed;
- }
- return;
- }
-
- await new AnnounceOk(UNKNOWN_HOP, 0).encode(stream.writer, this.version);
- dirty = false;
- await sync();
- // Every route the peer holds has its floor by now.
- await write({ status: "synced" });
-
- for (;;) {
- if (departed instanceof Error) throw departed;
- if (departed === null || this.#withdrawal.closing.peek()) break;
- if (dirty) {
- dirty = false;
- await sync();
- continue;
- }
- // The origin closed: the last sync ended its claims and route, and the stream finishes.
- if (this.#advertised.peek() === undefined) break;
- const woke = new Promise((resolve) => {
- notify = resolve;
- });
- await race([woke, this.#withdrawal.closing]);
- notify = undefined;
- }
- if (this.#withdrawal.closing.peek()) {
- await end();
- stream.close();
- await stream.writer.closed;
- }
- } finally {
- for (const stop of stops) stop?.();
- }
- }
-
/**
* The broadcast a request names, from this session's origin. A request naming an origin node
* other than this one is not ours to serve: a browser forwards nothing.
@@ -2745,7 +2596,12 @@ export class Publisher {
* @internal
*/
async drain(): Promise {
- const [withdrawn] = await Promise.allSettled([this.#withdrawal.close()]);
+ const [withdrawn] = await Promise.allSettled([
+ // Lite-07: the stream finishes once every route on it is ended, which says this side is leaving.
+ this.#withdrawal.close().then(async () => {
+ if (hasRoutes(this.version)) await AnnounceStream.of(this.#quic, this.version).finish();
+ }),
+ ]);
// Only after the withdrawals, so a request that arrived while they were in flight is
// waited for too. Owed requests run on their own meanwhile, so this adds no delay.
while (this.#owed.size > 0) await Promise.allSettled(this.#owed);
@@ -2755,6 +2611,7 @@ export class Publisher {
close() {
// The broadcasts belong to the origin, which outlives this session; closing here
// only drops the borrow. The peer sees the unannounce when the streams die.
+ this.#answer?.close();
// Release the datagram writer's lock so the stream can be torn down.
this.#datagramWriter?.close();
diff --git a/js/net/src/lite/rekey.test.ts b/js/net/src/lite/rekey.test.ts
new file mode 100644
index 00000000000..739b2d9c3b4
--- /dev/null
+++ b/js/net/src/lite/rekey.test.ts
@@ -0,0 +1,183 @@
+// The same cases as `lite/rekey.rs` in rs/moq-net: both endpoints must carry routes alike.
+
+import { expect, test } from "bun:test";
+import * as Path from "../path.ts";
+import type { Offer } from "./announce.ts";
+import { compare, type Order, order, type Plan, plan, type Route } from "./rekey.ts";
+
+const BEST: Order = { held: false, names: false, draining: false, priority: 0, metric: 1n };
+const HELD: Order = { ...BEST, held: true };
+
+const set = (...paths: string[]) => new Set(paths.map((path) => Path.from(path)));
+
+function route(node: bigint, prefix: string, region: string, rank: Order, claims: [string, bigint][]): Route {
+ return {
+ node,
+ prefix: Path.from(prefix),
+ region: Path.from(region),
+ exact: false,
+ order: rank,
+ claims: new Map(claims.map(([path, id]) => [Path.from(path), id])),
+ };
+}
+
+const ids = (carried: Plan) => carried.routes.map((r) => [r.id, r.from, r.kind, r.prefix as string]);
+const claims = (carried: Plan, index: number) =>
+ [...carried.routes[index].claims].map(([path, claim]) => [path as string, claim.id, claim.from]).sort();
+
+test("widening merges and keeps ids", () => {
+ const routes = new Map([
+ [
+ 0n,
+ route(7n, "a/b", "a/b", HELD, [
+ ["a/b", 0n],
+ ["a/b/x", 1n],
+ ]),
+ ],
+ [1n, route(7n, "a/c", "a/c", BEST, [["a/c/y", 2n]])],
+ [2n, route(7n, "a/c/deep", "a/c", BEST, [["a/c/deep/z", 3n]])],
+ [3n, route(9n, "q", "q", BEST, [["q/k", 4n]])],
+ ]);
+ const carried = plan(set("a/b", "a/c", "q"), set("a"), [Path.from("a")], routes, 4n, 5n);
+ expect(ids(carried)).toEqual([
+ [1n, 1n, "widens", "a"],
+ [2n, 2n, "stays", "a/c/deep"],
+ ]);
+ // The claim at the route's own prefix goes; the loser's claims move in.
+ expect(claims(carried, 0)).toEqual([
+ ["a/b/x", 1n, 1n],
+ ["a/c/y", 2n, 2n],
+ ]);
+ expect(carried.routes[1].region).toBe(Path.from("a"));
+ expect([carried.nextRoute, carried.nextClaim]).toEqual([4n, 5n]);
+});
+
+test("narrowing presents the covering claim", () => {
+ const routes = new Map([
+ [
+ 0n,
+ route(7n, "a", "a", BEST, [
+ ["a", 0n],
+ ["a/b/x", 1n],
+ ["a/c", 2n],
+ ["a/d", 3n],
+ ]),
+ ],
+ [1n, route(7n, "a/c", "a", HELD, [["a/c/y", 4n]])],
+ ]);
+ const added = [Path.from("a/c"), Path.from("a/b")];
+ const carried = plan(set("a"), set("a/b", "a/c"), added, routes, 2n, 5n);
+ // The narrowed route is preferred to the held one already at `a/c`.
+ expect(ids(carried)).toEqual([
+ [2n, 0n, "narrows", "a/c"],
+ [3n, 0n, "narrows", "a/b"],
+ ]);
+ expect(claims(carried, 0)).toEqual([["a/c", 2n, 2n]]);
+ expect(claims(carried, 1)).toEqual([
+ ["a/b", 5n, 0n],
+ ["a/b/x", 1n, 1n],
+ ]);
+ // Only the claim now presented below its path is further above it, by the segments between.
+ const lowered = (index: number) =>
+ [...carried.routes[index].claims].map(([path, claim]) => [path as string, claim.lowered]).sort();
+ expect(lowered(0)).toEqual([["a/c", 0n]]);
+ expect(lowered(1)).toEqual([
+ ["a/b", 1n],
+ ["a/b/x", 0n],
+ ]);
+ expect([carried.nextRoute, carried.nextClaim]).toEqual([4n, 6n]);
+});
+
+test("a staying route keeps its key against an equal", () => {
+ const routes = new Map([
+ [0n, route(7n, "a", "a", BEST, [["a/c/y", 1n]])],
+ [1n, route(7n, "a/c", "a", BEST, [["a/c/z", 2n]])],
+ [2n, route(7n, "b", "b", BEST, [])],
+ ]);
+ const carried = plan(set("a", "b"), set("a/c", "b"), [Path.from("a/c")], routes, 3n, 3n);
+ expect(ids(carried)).toEqual([
+ [1n, 1n, "stays", "a/c"],
+ [2n, 2n, "stays", "b"],
+ ]);
+});
+
+test("a re-added prefix keeps its routes, and an uncovered one ends", () => {
+ const routes = new Map([
+ [0n, route(7n, "a", "a", BEST, [["a/x", 0n]])],
+ [1n, route(7n, "b", "b", BEST, [["b/y", 1n]])],
+ ]);
+ const carried = plan(set("a", "b"), set("a"), [Path.from("a")], routes, 2n, 2n);
+ expect(ids(carried)).toEqual([[0n, 0n, "stays", "a"]]);
+ expect([carried.nextRoute, carried.nextClaim]).toEqual([2n, 2n]);
+});
+
+test("a route ranks unheld, then off the subscriber's path, then by its offer", () => {
+ const offer = (extra: Partial): Offer => ({
+ path: [{ node: 7n, session: 0n }],
+ draining: false,
+ priority: 0,
+ metric: 1n,
+ downOnly: false,
+ ...extra,
+ });
+ const ranked = [
+ order(false, offer({ priority: 5, metric: 9n }), [{ node: 7n, session: 0n }], 3n),
+ order(false, offer({}), [{ node: 7n, session: 0n }], 3n),
+ order(false, offer({ metric: 2n }), [{ node: 7n, session: 0n }], 3n),
+ order(false, offer({ draining: true }), [{ node: 7n, session: 0n }], 3n),
+ // A refused offer ranks behind every usable one.
+ order(false, undefined, [{ node: 7n, session: 0n }], 3n),
+ order(false, offer({}), [{ node: 3n, session: 0n }], 3n),
+ order(true, offer({}), [{ node: 7n, session: 0n }], 3n),
+ ];
+ for (let i = 0; i < ranked.length; i++) {
+ for (let j = 0; j < ranked.length; j++) {
+ expect(Math.sign(compare(ranked[i], ranked[j]))).toBe(Math.sign(i - j));
+ }
+ }
+});
+
+test("an exact route stays at its own prefix", () => {
+ const exact = (r: Route): Route => ({ ...r, exact: true });
+ let routes = new Map([
+ [
+ 0n,
+ exact(
+ route(7n, "a/b", "a/b", BEST, [
+ ["a/b", 0n],
+ ["a/b/x", 1n],
+ ]),
+ ),
+ ],
+ [
+ 1n,
+ route(8n, "a/b", "a/b", BEST, [
+ ["a/b", 2n],
+ ["a/b/y", 3n],
+ ]),
+ ],
+ ]);
+ let carried = plan(set("a/b"), set("a"), [Path.from("a")], routes, 2n, 4n);
+ expect(ids(carried)).toEqual([
+ [0n, 0n, "stays", "a/b"],
+ [1n, 1n, "widens", "a"],
+ ]);
+ expect([carried.routes[0].region, carried.routes[0].exact]).toEqual([Path.from("a"), true]);
+ expect(claims(carried, 0)).toEqual([
+ ["a/b", 0n, 0n],
+ ["a/b/x", 1n, 1n],
+ ]);
+ expect(carried.routes[1].exact).toBe(false);
+
+ // Asked at its own prefix again, it lies there exactly once more.
+ routes = new Map([[0n, exact(route(7n, "a/b", "a", BEST, []))]]);
+ carried = plan(set("a"), set("a/b"), [Path.from("a/b")], routes, 2n, 4n);
+ expect(ids(carried)).toEqual([[0n, 0n, "stays", "a/b"]]);
+ expect([carried.routes[0].region, carried.routes[0].exact]).toEqual([Path.from("a/b"), true]);
+
+ // Narrowed, it covers the narrower prefix from its own.
+ routes = new Map([[0n, exact(route(7n, "a/b", "a/b", BEST, [["a/b", 0n]]))]]);
+ carried = plan(set("a/b"), set("a/b/c"), [Path.from("a/b/c")], routes, 1n, 1n);
+ expect(ids(carried)).toEqual([[1n, 0n, "narrows", "a/b/c"]]);
+ expect(carried.routes[0].exact).toBe(false);
+});
diff --git a/js/net/src/lite/rekey.ts b/js/net/src/lite/rekey.ts
new file mode 100644
index 00000000000..5dc6e61ce3e
--- /dev/null
+++ b/js/net/src/lite/rekey.ts
@@ -0,0 +1,197 @@
+// Lite-07: how an Announce Stream's live routes carry over an edit of its requested prefixes, at
+// the ANNOUNCE_OK answering it.
+//
+// Both endpoints run `plan` over what the stream carries, so they agree on which routes and claims
+// live on, and under which ids, without a message for any of it. Mirrors `lite/rekey.rs` in
+// rs/moq-net.
+
+import { coveringPrefixes } from "../internal.ts";
+import * as Path from "../path.ts";
+import type { Offer, RouteHop } from "./announce.ts";
+
+/**
+ * How a route ranks among the routes that would share an origin node and prefix: one that is not
+ * held first, then one whose path does not name the subscriber, then the better rank (not
+ * draining, the higher priority, the lower metric).
+ */
+export interface Order {
+ held: boolean;
+ names: boolean;
+ draining: boolean;
+ priority: number;
+ metric: bigint;
+}
+
+/** The {@link Order} of a route offering `offer`, or `path` alone for an offer the receiver refused,
+ * to the subscriber `receiver`. */
+export function order(held: boolean, offer: Offer | undefined, path: RouteHop[], receiver: bigint): Order {
+ const names = path.some((hop) => hop.node === receiver);
+ if (offer) return { held, names, draining: offer.draining, priority: offer.priority, metric: offer.metric };
+ return { held, names, draining: true, priority: -128, metric: 2n ** 64n - 1n };
+}
+
+/** Negative when `a` is preferred to `b`, positive when `b` is, and 0 when they rank alike. */
+export function compare(a: Order, b: Order): number {
+ if (a.held !== b.held) return a.held ? 1 : -1;
+ if (a.names !== b.names) return a.names ? 1 : -1;
+ if (a.draining !== b.draining) return a.draining ? 1 : -1;
+ if (a.priority !== b.priority) return b.priority - a.priority;
+ return a.metric < b.metric ? -1 : a.metric > b.metric ? 1 : 0;
+}
+
+/** A live route on the stream. */
+export interface Route {
+ node: bigint;
+ /** The prefix it covers, or the requested prefix it lies in when its suffix was refused. */
+ prefix: Path.Valid;
+ /** The requested prefix it lies in. */
+ region: Path.Valid;
+ /** Its ROUTE_START said its sender keeps it at its own prefix, so a widening leaves it there
+ * rather than moving it up. */
+ exact: boolean;
+ order: Order;
+ /** The Announce ID of each of its live claims with a path. */
+ claims: ReadonlyMap;
+}
+
+/**
+ * What became of a route that lives on. `stays`: unchanged. `widens`: moved to a prefix above the
+ * one it was requested under, so it certifies nothing until its next floor. `narrows`: carried to
+ * a prefix below the one it covered, with its floor.
+ */
+export type Kind = "stays" | "widens" | "narrows";
+
+/** A claim of a route that lives on: its Announce ID from here on, and the one it had, the same
+ * unless it is now presented at a prefix below its path. `lowered` is how many segments below where
+ * it was presented it is presented now, which its Above grows by: 0 unless it now covers a narrower
+ * prefix from above. */
+export interface Claim {
+ id: bigint;
+ from: bigint;
+ lowered: bigint;
+}
+
+/** A route that lives on: its Route ID from here on, and the route whose offer and state it keeps. */
+export interface Carried {
+ id: bigint;
+ from: bigint;
+ kind: Kind;
+ prefix: Path.Valid;
+ region: Path.Valid;
+ /** It lies at its own prefix: one that stays keeps it, and one that widens or narrows is
+ * presented at a prefix that is not its own. */
+ exact: boolean;
+ claims: Map;
+}
+
+/** The routes that live on, by their new Route ID. Every other route ends, and every Announce ID
+ * no claim here keeps retires. */
+export interface Plan {
+ routes: Carried[];
+ nextRoute: bigint;
+ nextClaim: bigint;
+}
+
+/**
+ * Carry `routes` over an edit from the prefixes `old` to `next`, which added `added` in that
+ * order. `nextRoute` and `nextClaim` are the next unassigned Route ID and Announce ID.
+ */
+export function plan(
+ old: ReadonlySet,
+ next: ReadonlySet,
+ added: readonly Path.Valid[],
+ routes: ReadonlyMap,
+ nextRoute: bigint,
+ nextClaim: bigint,
+): Plan {
+ const region = (prefix: Path.Valid): Path.Valid | undefined => {
+ for (const above of coveringPrefixes(prefix)) if (next.has(above)) return above;
+ return undefined;
+ };
+ const key = (node: bigint, prefix: Path.Valid) => `${node} ${prefix}`;
+ const ids = [...routes.keys()].sort((a, b) => (a < b ? -1 : a > b ? 1 : 0));
+ const route = (id: bigint): Route => {
+ const found = routes.get(id);
+ if (!found) throw new Error(`no route ${id}`);
+ return found;
+ };
+
+ const stays = new Map();
+ const widening = new Map();
+ const narrowing: { from: bigint; inside: Path.Valid }[] = [];
+ for (const id of ids) {
+ const { node, prefix, region: lies, exact } = route(id);
+ if (!old.has(lies)) throw new Error("a route lies in a requested prefix");
+ const covered = region(prefix) !== undefined;
+ const wider = region(lies);
+ // A route at its own prefix stays there, as one strictly inside the region does.
+ if (covered && (next.has(lies) || prefix !== lies || exact)) {
+ stays.set(key(node, prefix), id);
+ } else if (wider !== undefined) {
+ const entry = widening.get(key(node, wider));
+ if (entry) entry.sources.push(id);
+ else widening.set(key(node, wider), { wider, sources: [id] });
+ } else if (prefix === lies) {
+ for (const inside of added) {
+ if (inside !== lies && Path.hasPrefix(lies, inside)) narrowing.push({ from: id, inside });
+ }
+ }
+ }
+
+ const carried = new Map();
+ const kept = (claims: ReadonlyMap): [Path.Valid, Claim][] =>
+ [...claims].map(([path, id]) => [path, { id, from: id, lowered: 0n }]);
+ for (const { wider, sources } of widening.values()) {
+ let winner = sources[0];
+ for (const id of sources) if (compare(route(id).order, route(winner).order) < 0) winner = id;
+ const claims = new Map();
+ for (const id of sources) {
+ const source = route(id);
+ for (const [path, claim] of kept(source.claims)) if (path !== source.prefix) claims.set(path, claim);
+ }
+ carried.set(winner, {
+ id: winner,
+ from: winner,
+ kind: "widens",
+ prefix: wider,
+ region: wider,
+ exact: false,
+ claims,
+ });
+ }
+ for (const { from, inside } of narrowing) {
+ const source = route(from);
+ const at = key(source.node, inside);
+ // A route already there stays unless this one would be preferred.
+ const staying = stays.get(at);
+ if (staying !== undefined) {
+ if (compare(route(staying).order, source.order) <= 0) continue;
+ stays.delete(at);
+ }
+ const claims = new Map();
+ // Of the claims covering the whole prefix alike, only the most specific shows.
+ let above: [Path.Valid, bigint] | undefined;
+ for (const [path, id] of source.claims) {
+ if (path !== inside && Path.hasPrefix(inside, path)) claims.set(path, { id, from: id, lowered: 0n });
+ if (Path.hasPrefix(path, inside) && (!above || path.length > above[0].length)) above = [path, id];
+ }
+ if (above) {
+ const id = above[0] === inside ? above[1] : nextClaim++;
+ const lowered = BigInt(Path.parts(inside).length - Path.parts(above[0]).length);
+ claims.set(inside, { id, from: above[1], lowered });
+ }
+ const id = nextRoute++;
+ carried.set(id, { id, from, kind: "narrows", prefix: inside, region: inside, exact: false, claims });
+ }
+ for (const id of stays.values()) {
+ const { prefix, exact, claims } = route(id);
+ const lies = region(prefix);
+ if (lies === undefined) throw new Error("a route that stays is covered");
+ carried.set(id, { id, from: id, kind: "stays", prefix, region: lies, exact, claims: new Map(kept(claims)) });
+ }
+ return {
+ routes: [...carried.values()].sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0)),
+ nextRoute,
+ nextClaim,
+ };
+}
diff --git a/js/net/src/lite/routed.ts b/js/net/src/lite/routed.ts
new file mode 100644
index 00000000000..ec22cf68edd
--- /dev/null
+++ b/js/net/src/lite/routed.ts
@@ -0,0 +1,1181 @@
+// Lite-07: what the peer sends on its Announce Stream as a publisher, and the prefixes this
+// session asks it for on its own.
+//
+// Every `routes()` reader of the session shares the one stream. The session asks the peer only
+// for the covering set of what its readers want, no prefix at or below another, edited in place
+// one ANNOUNCE_REQUEST at a time, and each reader is fed from the routes and claims the stream
+// carries, presented under the reader's own prefix as the peer would present them to a request
+// for it. Mirrors `subscriber/routed.rs` in rs/moq-net for a node that forwards nothing: an
+// endpoint applies the peer's withdrawals and holds, and casts no doubt.
+
+import { type Dispose, type GetPromise, race } from "@moq/signals";
+import type * as announce from "../announced.ts";
+import type { Drain } from "../connection/goaway.ts";
+import type { BroadcastCache } from "../consume.ts";
+import { error, ProtocolViolation } from "../error.ts";
+import { Cost, type Hop, routesEqual, WireRoute } from "../hop.ts";
+import { bestCandidate, coveringPrefixes, hiddenBelow, scopeCaptures, scopeOverlaps } from "../internal.ts";
+import * as Path from "../path.ts";
+import { SlotMap, type Slots } from "../util/limits.ts";
+import {
+ type Add,
+ type AnnounceBroadcast,
+ AnnounceHistory,
+ type Refusal as AnnounceRefusal,
+ type Base,
+ type Dead,
+ type Offer,
+ type Refusable,
+ ROUTE_MAX_HOPS,
+ type RouteHop,
+ requestFits,
+ requestParts,
+ type Unusable,
+} from "./announce.ts";
+import type { Carrier, Claim, Floors } from "./floors.ts";
+import * as rekey from "./rekey.ts";
+import type { Version } from "./version.ts";
+
+/** What a `routes()` reader asked for. */
+export interface Request {
+ /** The prefix the reader's scope lies under. */
+ prefix: Path.Valid;
+ scope: Path.Pattern;
+ /** Also report claims with a `.`-prefixed segment below the prefix. */
+ hidden: boolean;
+ /** Also report claims whose subtree overlaps the scope without matching it. */
+ covering: boolean;
+ /** Report each origin node's claim at a path rather than the best one. */
+ perNode: boolean;
+}
+
+/** What the session's route layer is built on. */
+export interface Session {
+ quic: WebTransport;
+ version: Version;
+ /** This node's hop: a route whose path names it came back to its origin. */
+ hop: Hop;
+ floors: Floors;
+ consumes: BroadcastCache;
+ /** The session's announce cap: one slot per live route and claim. */
+ slots: Slots;
+ /** Rank a route last once the peer sent GOAWAY. */
+ drained(route: WireRoute): WireRoute;
+ goaway?: GetPromise;
+ /** What covers a path changed with no claim reporting it: every selection is checked anew. */
+ revalidate(): void;
+ /** Send one message on this session's Announce Stream. */
+ send(msg: AnnounceBroadcast): Promise;
+}
+
+/** One reader, and what it was last told of each path. */
+interface Reader extends Request {
+ announced: announce.Producer;
+ /** What each path reports, by origin node with `perNode` and otherwise under `undefined`. */
+ reported: Map>;
+ synced: boolean;
+}
+
+/** A prefix this session asked the peer for, by its Prefix ID. */
+interface Asked {
+ path: Path.Valid;
+ hidden: boolean;
+ /** The peer answered the ANNOUNCE_REQUEST adding it, so its stream may name it. */
+ live: boolean;
+ /** The peer sent its initial set. */
+ synced: boolean;
+}
+
+/** One route the peer sent on the stream, by its Route ID. */
+interface Route {
+ id: bigint;
+ node: bigint;
+ /** The requested prefix it lies in. */
+ region: Path.Valid;
+ /** The prefix it covers, or undefined when this node refused the suffix the peer forwarded,
+ * which leaves the route refused for good. */
+ prefix: Path.Valid | undefined;
+ /** Its ROUTE_START said it lies at its own prefix, so a widening leaves it there. */
+ exact: boolean;
+ /** The path the peer last named, kept while this node refuses the rest, so a DEAD finds it. */
+ path: RouteHop[];
+ /** Undefined while this node refuses what the peer forwarded: no route at all, its claims
+ * offline, until an offer it can use. */
+ offer: Offer | undefined;
+ /** A DRAIN named a node on its path since its last offer: it ranks last. */
+ drain: boolean;
+ /** ROUTE_HOLD, or a DEAD that held it: no usable path to the publisher, until its next offer. */
+ held: boolean;
+ /** The Announce ID of each live claim, by path: one per path, checked by lookup. */
+ claims: Map;
+ /** The Announce IDs of its live claims whose path this node refused. */
+ unnamed: Set;
+ /** The Announce IDs of its live claims at a path whose content this node refuses. */
+ withheld: Set;
+ /** How the claims differ from the view's, applied whole at the next floor. */
+ pending: Map;
+ /** The prefixes whose claims an edit that widened the route took from the routes it merged,
+ * which the view keeps until the route's next floor says what covers each now. */
+ stale: Set;
+ /** The claims as of the last floor, and the revision it certified them at, if any. */
+ view: { prefix: Path.Valid; hidden: boolean; revision: bigint | undefined; claims: Map };
+ /** The view's claims no other route's newer floor withdrew or replaced, each as the newest
+ * floor certifying it has it, cost included. */
+ shown: Map;
+ carrier: Carrier;
+ /** Who the route's claims are filed under in the consume cache, which keeps every route's
+ * claim so a consume ranks them for its own path. */
+ owner: object;
+}
+
+type Joined = Route & { offer: Offer; prefix: Path.Valid };
+
+/** A DEAD whose ROUTE_UPDATEs are still arriving: it applies with its last. */
+interface Moving {
+ hop: RouteHop;
+ gone: boolean;
+ /** How many ROUTE_UPDATEs are still to come. */
+ left: bigint;
+ /** The new offer of each route it moved, by Route ID. */
+ offers: Map;
+}
+
+const same = (a: Claim | undefined, b: Claim | undefined) =>
+ a === b ||
+ (!!a && !!b && a.instance === b.instance && a.epoch === b.epoch && a.cost === b.cost && a.above === b.above);
+
+// The peer's `above` for the claim at `path`, if it can be: a claim is no more than its path's
+// segments broader than the path it is shown at. Mirrors rs/moq-net's `Routed::above`.
+const aboveOf = (path: Path.Valid | undefined, above: bigint | undefined): bigint => {
+ const value = above ?? 0n;
+ if (path === undefined || value === 0n) return value;
+ if (value > BigInt(Path.parts(path).length))
+ throw new ProtocolViolation(`a claim at ${path} cannot be ${value} segments above it`);
+ return value;
+};
+
+const through = (wire: Route, hop: RouteHop) =>
+ wire.path.some((at) => at.node === hop.node && at.session === hop.session);
+
+/** The route layer a lite-07 session's peer feeds, shared by every reader of the session. */
+export class Routed {
+ #session: Session;
+ #readers = new Set();
+ /** The prefixes asked of the peer. */
+ #asked = new Map();
+ #nextPrefix = 0n;
+ /** The ANNOUNCE_REQUEST the peer has not answered yet. */
+ #editing?: { remove: bigint[]; add: bigint[] };
+ /** The peer finished its stream: it answers nothing more. */
+ #finished = false;
+ /** This side finished its stream: it asks for nothing more. */
+ #silent = false;
+
+ // Each live route and claim holds one of the session's announce slots.
+ #routes: SlotMap;
+ // A claim refused has no `claim` until a restart names all of it, and one refused before its
+ // path has no `path` either, and never will.
+ #claims: SlotMap;
+ // The live route to each publisher node and prefix: at most one each.
+ #named = new Map();
+ // The routes whose committed claims include each path.
+ #byPath = new Map>();
+ #history = new AnnounceHistory();
+ #nextRoute = 0n;
+ #moving?: Moving;
+ // While a DEAD applies: each reader's presented paths it touched, and whether any touch there
+ // lost a claim, each refreshed once it is done.
+ #deferred?: Map>;
+
+ // Set once the session lets go of its routes, which then show nothing anew.
+ #closing = false;
+ #closed = Promise.withResolvers();
+ #stopDrain?: Dispose;
+
+ constructor(session: Session) {
+ this.#session = session;
+ this.#routes = new SlotMap(session.slots);
+ this.#claims = new SlotMap(session.slots);
+ // A draining peer ranks its routes last from the GOAWAY itself.
+ this.#stopDrain = session.goaway?.changed(() => {
+ if (this.#closing) return;
+ for (const wire of this.#routes.values()) this.#refreshRoute(wire, false);
+ });
+ }
+
+ /** Feed `announced` what the stream carries for `request`, until its last consumer leaves. */
+ attach(announced: announce.Producer, request: Request): void {
+ if (this.#closing) {
+ announced.close();
+ return;
+ }
+ const reader: Reader = { ...request, announced, reported: new Map(), synced: false };
+ this.#readers.add(reader);
+ // What the stream already carries shows before the reader counts as synced.
+ const paths = new Set();
+ for (const wire of this.#routes.values()) {
+ for (const path of wire.view.claims.keys()) {
+ const at = presentedAt(reader, path);
+ if (at !== undefined) paths.add(at);
+ }
+ }
+ for (const path of paths) this.#refresh(reader, path, false);
+ this.#ask();
+ this.#release();
+ void race([announced.closed]).then(() => {
+ if (!this.#readers.delete(reader)) return;
+ this.#ask();
+ this.#release();
+ });
+ }
+
+ // The covering set of what the readers want: no prefix at or below another. A prefix asks for
+ // hidden claims when a reader beneath it does, or sits behind a hidden segment itself.
+ #wanted(): Map {
+ const heads = new Set();
+ for (const reader of this.#readers) heads.add(reader.prefix);
+ const covering = (head: Path.Valid): Path.Valid => {
+ let top = head;
+ for (const above of coveringPrefixes(head)) if (heads.has(above)) top = above;
+ return top;
+ };
+ const wanted = new Map();
+ for (const reader of this.#readers) {
+ const top = covering(reader.prefix);
+ const hidden = reader.hidden || hiddenBelow(top, reader.prefix);
+ wanted.set(top, (wanted.get(top) ?? false) || hidden);
+ }
+ // The whole set has to fit one request: past that, everything is asked for at the root.
+ if (!requestFits(wanted.keys(), this.#session.version)) {
+ const hidden = [...this.#readers].some(
+ (reader) => reader.hidden || hiddenBelow(Path.empty(), reader.prefix),
+ );
+ return new Map([[Path.empty(), hidden]]);
+ }
+ return wanted;
+ }
+
+ // The edit that brings what is asked of the peer in line with what the readers want.
+ #edit(): { remove: bigint[]; add: Add[] } {
+ const wanted = this.#wanted();
+ const remove: bigint[] = [];
+ for (const [id, asked] of this.#asked) {
+ if (wanted.get(asked.path) === asked.hidden) wanted.delete(asked.path);
+ else remove.push(id);
+ }
+ return { remove, add: [...wanted].map(([prefix, hidden]) => ({ prefix, hidden })) };
+ }
+
+ // Ask the peer for what the readers now want, one edit at a time.
+ #ask(): void {
+ if (this.#editing || this.#finished || this.#silent || this.#closing) return;
+ const { remove, add } = this.#edit();
+ if (remove.length === 0 && add.length === 0) return;
+ const added: bigint[] = [];
+ for (const { prefix, hidden } of add) {
+ const id = this.#nextPrefix++;
+ this.#asked.set(id, { path: prefix, hidden, live: false, synced: false });
+ added.push(id);
+ }
+ this.#editing = { remove, add: added };
+ console.debug(`announced: remove=${remove.join(",")} add=${add.map(({ prefix }) => prefix).join(",")}`);
+ // However large, the edit goes whole, in as many messages as it takes.
+ for (const part of requestParts(remove, add, this.#session.version)) {
+ this.#session.send(part).catch((err: unknown) => this.close(error(err)));
+ }
+ }
+
+ // Let each reader see its prefix synced once its answer is in: the peer answered every
+ // request, and sent the initial set of each prefix asked for it.
+ #release(): void {
+ const done = this.#finished || this.#silent;
+ if (!done) {
+ if (this.#editing) return;
+ const { remove, add } = this.#edit();
+ if (remove.length > 0 || add.length > 0) return;
+ }
+ for (const reader of this.#readers) {
+ if (reader.synced) continue;
+ const answered = [...this.#asked.values()].every(
+ (asked) =>
+ asked.synced ||
+ !(Path.hasPrefix(asked.path, reader.prefix) || Path.hasPrefix(reader.prefix, asked.path)),
+ );
+ if (!done && !answered) continue;
+ reader.synced = true;
+ reader.announced.synced();
+ }
+ }
+
+ /** This side finished its stream: the prefixes asked of the peer stay as they are. */
+ silence(): void {
+ this.#silent = true;
+ this.#release();
+ }
+
+ /** The peer finished its stream: it is leaving, its routes go, and it answers nothing more. */
+ finish(): void {
+ if (this.#moving) throw new ProtocolViolation("Announce Stream finished inside a DEAD");
+ this.#finished = true;
+ for (const wire of [...this.#routes.values()]) this.#endRoute(wire);
+ this.#release();
+ }
+
+ /** The session ended: every reader ends with `err`, and every route and claim goes. */
+ close(err?: Error): void {
+ if (this.#closing) return;
+ this.#closing = true;
+ this.#closed.resolve();
+ this.#stopDrain?.();
+ for (const wire of this.#routes.values()) {
+ this.#session.floors.leave(wire.node, wire.carrier);
+ for (const path of wire.view.claims.keys()) this.#session.consumes.withdraw(path, wire.owner);
+ }
+ if (this.#routes.size > 0) this.#session.revalidate();
+ this.#claims.clear();
+ this.#routes.clear();
+ this.#named.clear();
+ this.#byPath.clear();
+ const readers = [...this.#readers];
+ this.#readers.clear();
+ for (const reader of readers) reader.announced.close(err);
+ }
+
+ #route(id: bigint): Route {
+ const found = this.#routes.get(id);
+ if (!found) throw new ProtocolViolation(`unknown route ${id}`);
+ return found;
+ }
+
+ #claim(id: bigint) {
+ const found = this.#claims.get(id);
+ if (!found) throw new ProtocolViolation(`unknown announce id ${id}`);
+ return found;
+ }
+
+ // The claim at `path` under `id` is now `claim`, pending the route's next floor.
+ #change(id: bigint, path: Path.Valid, claim: Claim | undefined) {
+ const wire = this.#route(id);
+ if (same(wire.view.claims.get(path), claim) && !wire.stale.has(path)) wire.pending.delete(path);
+ else wire.pending.set(path, claim);
+ }
+
+ // Whether this node takes what the peer offers for the route at all: not a refused offer or
+ // suffix, nor a route to the reserved node. Only such a route joins the node's floors.
+ #joined(wire: Route): wire is Joined {
+ return wire.offer !== undefined && wire.prefix !== undefined && wire.node !== 0n;
+ }
+
+ // The same, without narrowing, to compare a route before and after it changes.
+ #filed(wire: Route): boolean {
+ return this.#joined(wire);
+ }
+
+ // A route is no route at all once it crossed as many nodes as one may, or crossed us: a copy
+ // of our own route, come back.
+ #usable(wire: Route): wire is Joined {
+ return (
+ this.#joined(wire) &&
+ wire.offer.path.length < ROUTE_MAX_HOPS &&
+ !wire.offer.path.some((hop) => hop.node === this.#session.hop)
+ );
+ }
+
+ // A claim through this session: its cost plus the route's metric, past this session's link
+ // cost of 1 as Rust's default, ranked last while a node on its path drains, then by the
+ // lowest priority on its path, past this link's 0 as Rust's default, and unavailable while
+ // the route is held. Requests through it name its publisher and its prefix's depth.
+ #candidate(wire: Joined, claim: Claim): WireRoute {
+ return this.#session.drained(
+ WireRoute.normalize({
+ instance: claim.instance,
+ epoch: claim.epoch,
+ cost: claim.cost + wire.offer.metric + 1n,
+ node: wire.node,
+ depth: Path.parts(wire.prefix).length,
+ priority: Math.min(0, wire.offer.priority),
+ above: claim.above,
+ draining: wire.offer.draining || wire.drain || undefined,
+ available: !wire.held,
+ }),
+ );
+ }
+
+ // File the route's claim at `path` in the consume cache, or take it out. Every route's claim
+ // stays there, so each consume ranks them for its own path and its epochs. `restart` marks
+ // another instance or epoch, whose consumes subscribe fresh. An unusable route's claim is
+ // parked, keeping its place in a tie as `byPath` does, unless it just moved to the newest.
+ #serve(wire: Route, path: Path.Valid, restart: boolean) {
+ const consumes = this.#session.consumes;
+ const claim = wire.shown.get(path);
+ if (!claim || (!this.#usable(wire) && restart)) consumes.withdraw(path, wire.owner);
+ else if (!this.#usable(wire)) consumes.park(path, wire.owner);
+ else consumes.announce(path, wire.owner, this.#candidate(wire, claim), restart);
+ }
+
+ // The usable claims `reader` is shown at `path`: those at the path itself, and at the
+ // reader's own prefix each route's most specific claim at or above it, as the peer presents
+ // a claim covering more than a requested prefix: that many segments more above the path, so a
+ // claim exactly there still outranks it, as `covering` in rs/moq-net's `route/view.rs`.
+ #candidates(reader: Reader, path: Path.Valid): WireRoute[] {
+ const candidates: WireRoute[] = [];
+ const seen = new Set();
+ for (const at of path === reader.prefix ? coveringPrefixes(path) : [path]) {
+ const by = BigInt(Path.parts(path).length - Path.parts(at).length);
+ for (const id of this.#byPath.get(at) ?? []) {
+ const wire = this.#route(id);
+ const claim = wire.shown.get(at);
+ if (!claim || seen.has(id)) continue;
+ seen.add(id);
+ // A claim above the path covers it only where the newest floor certifying the path
+ // leaves it, as a consume of the path picks.
+ if (!this.#session.floors.covers(wire.node, at, path)) continue;
+ const shown = by === 0n ? claim : { ...claim, above: claim.above + by };
+ if (this.#usable(wire)) candidates.push(this.#candidate(wire, shown));
+ }
+ }
+ return candidates;
+ }
+
+ // Report to `reader` the best claim at `path`, or with `perNode` each origin node's, and each
+ // one's end. `lost` marks the routes leaving as let go rather than withdrawn, so they go
+ // offline rather than ending.
+ #refresh(reader: Reader, path: Path.Valid, lost: boolean) {
+ const { announced, scope } = reader;
+ if (announced.closed.peek() !== undefined) return;
+ // Keyed by origin node with `perNode`, and otherwise the one key `undefined`.
+ const now = new Map();
+ if (visible(reader, path)) {
+ const candidates = this.#candidates(reader, path);
+ if (reader.perNode) {
+ for (const next of candidates) now.set(next.node, next);
+ } else {
+ // The same pick a consume of `path` makes from the cache.
+ const best = bestCandidate(path, path, candidates, (next) => next);
+ if (best) now.set(undefined, best);
+ }
+ }
+
+ const captures = scopeCaptures(scope, path);
+ const prev = reader.reported.get(path) ?? new Map();
+ // Arrivals first, so a reader sees a claim replaced rather than the path gone between.
+ for (const [key, next] of now) {
+ const was = prev.get(key);
+ if (was && routesEqual(was, next)) continue;
+ const restart = !!was && (was.instance !== next.instance || was.epoch !== next.epoch);
+ const kind = !was ? "start" : restart ? "restart" : "update";
+ console.debug(`announced: broadcast=${path} ${kind} instance=${next.instance} node=${next.node}`);
+ announced.append({ prefix: path, captures, kind, route: next });
+ }
+ for (const [key, was] of prev) {
+ if (now.has(key)) continue;
+ console.debug(`announced: broadcast=${path} active=false lost=${lost} node=${was.node}`);
+ announced.append({ prefix: path, captures, kind: lost ? "offline" : "end", route: was });
+ }
+ if (now.size > 0) reader.reported.set(path, now);
+ else reader.reported.delete(path);
+ }
+
+ // Report the claims at `path` to every reader it shows under: at the path itself for a reader
+ // above it, and at the reader's own prefix for one below.
+ // What covers a path changed: every consume picks anew, and each reader is shown again
+ // what covers its own prefix from above it.
+ #revalidate() {
+ this.#session.revalidate();
+ for (const reader of this.#readers) this.#touch(reader.prefix, false);
+ }
+
+ #touch(path: Path.Valid, lost: boolean) {
+ for (const reader of this.#readers) {
+ const at = presentedAt(reader, path);
+ if (at === undefined) continue;
+ if (!this.#deferred) {
+ this.#refresh(reader, at, lost);
+ continue;
+ }
+ // A path touched again keeps a loss: a later touch only rechecks it.
+ const paths = this.#deferred.get(reader) ?? new Map();
+ paths.set(at, (paths.get(at) ?? false) || lost);
+ this.#deferred.set(reader, paths);
+ }
+ }
+
+ // Show or hide the route's claims at `touched`, each marked whether it restarted, or with
+ // `all` every claim it holds, once another route's floor may have moved what is current.
+ #show(wire: Route, touched: ReadonlyMap, all: boolean) {
+ const floors = this.#session.floors;
+ const paths = all
+ ? new Set([...touched.keys(), ...wire.shown.keys(), ...wire.view.claims.keys()])
+ : touched.keys();
+ for (const path of paths) {
+ const now = floors.shows(wire.node, wire.view, path);
+ const was = wire.shown.get(path);
+ const restart = touched.get(path);
+ if (same(now, was) && restart === undefined) continue;
+ if (now) wire.shown.set(path, now);
+ else wire.shown.delete(path);
+ // A claim shown anew or restarted iterates last, as the newest; a re-price keeps its
+ // place, as the consume cache does. See bestCandidate.
+ const ids = this.#byPath.get(path);
+ if (now && ids && (!was || restart)) {
+ ids.delete(wire.id);
+ ids.add(wire.id);
+ }
+ this.#serve(wire, path, restart ?? false);
+ // A claim gone was let go, not withdrawn by its origin, where the newest floor certifying it
+ // still presents it, or where this node refuses one of the route's claims and no floor
+ // certifies it gone. An uncertified withdrawal otherwise still ends: a relay certifies
+ // nothing to a request wider than its scope's region, as this one may be.
+ const said = now ? undefined : floors.certified(wire.node, wire.view, path);
+ const refusing = wire.unnamed.size > 0 || wire.withheld.size > 0;
+ this.#touch(path, !now && (said === "present" || (refusing && said === undefined)));
+ }
+ }
+
+ #refreshRoute(wire: Route, lost: boolean) {
+ for (const path of wire.view.claims.keys()) {
+ this.#serve(wire, path, false);
+ this.#touch(path, lost);
+ }
+ }
+
+ #key(node: bigint, at: Path.Valid | undefined): string | undefined {
+ return at === undefined || node === 0n ? undefined : `${node} ${at}`;
+ }
+
+ // A route with nothing committed, whose claims another route's floor may hide.
+ #blank(
+ id: bigint,
+ from: Pick,
+ at: {
+ region: Path.Valid;
+ prefix: Path.Valid | undefined;
+ exact: boolean;
+ hidden: boolean;
+ revision: bigint | undefined;
+ },
+ ): Route {
+ const view: Route["view"] = {
+ prefix: at.prefix ?? at.region,
+ hidden: at.hidden,
+ revision: at.revision,
+ claims: new Map(),
+ };
+ const wire: Route = {
+ id,
+ node: from.node,
+ region: at.region,
+ prefix: at.prefix,
+ exact: at.exact,
+ path: from.path,
+ offer: from.offer,
+ drain: from.drain,
+ held: from.held,
+ claims: new Map(),
+ unnamed: new Set(),
+ withheld: new Set(),
+ pending: new Map(),
+ stale: new Set(),
+ view,
+ shown: new Map(),
+ carrier: {
+ view,
+ // Another route's floor or end runs this: a closing session is about to let go.
+ reshow: () => {
+ if (!this.#closing) this.#show(wire, new Map(), true);
+ },
+ },
+ owner: {},
+ };
+ return wire;
+ }
+
+ // The peer started a route to `node` with the next Route ID, covering the requested prefix
+ // with the Prefix ID `asked` joined with `suffix`, at its own prefix when `exact`, along
+ // `offer`, or along `path` when this node refuses the offer.
+ #startRoute(
+ node: bigint,
+ asked: bigint,
+ suffix: Path.Valid | undefined,
+ exact: boolean,
+ offer: Offer | undefined,
+ path: RouteHop[],
+ ) {
+ // The peer names a prefix in effect on its stream.
+ const region = this.#asked.get(asked);
+ if (!region?.live) throw new ProtocolViolation(`unknown Prefix ID ${asked}`);
+ // Fields valid alone can still join the prefix past the path limits: refused too.
+ const joined = suffix === undefined ? undefined : Path.join(region.path, suffix);
+ const at = joined !== undefined && Path.check(joined) === undefined ? joined : undefined;
+ const named = this.#key(node, at);
+ // The peer keeps one route per publisher and prefix.
+ if (named !== undefined && this.#named.has(named))
+ throw new ProtocolViolation(`second route to node ${node} at ${at}`);
+ const id = this.#nextRoute++;
+ const wire = this.#blank(
+ id,
+ { node, path, offer, drain: false, held: false },
+ { region: region.path, prefix: at, exact, hidden: region.hidden, revision: undefined },
+ );
+ this.#routes.set(id, wire);
+ if (named !== undefined) this.#named.set(named, id);
+ // A refused route's floors stay its own, as Rust's node never holds the route.
+ if (this.#joined(wire)) this.#session.floors.join(node, wire.carrier);
+ }
+
+ // The peer let go of the route and every claim under it: offline, not withdrawn.
+ #endRoute(wire: Route) {
+ for (const id of [...wire.claims.values(), ...wire.unnamed]) {
+ this.#history.end(id);
+ this.#claims.delete(id);
+ }
+ this.#routes.delete(wire.id);
+ const named = this.#key(wire.node, wire.prefix);
+ if (named !== undefined && this.#named.get(named) === wire.id) this.#named.delete(named);
+ this.#drop(wire);
+ this.#revalidate();
+ }
+
+ // Take a route that no longer lives out of what the session shows: offline, not withdrawn.
+ #drop(wire: Route) {
+ // Unindexed first: leaving its floors reshows another carrier at once, which refreshes
+ // these paths and so looks up every route indexed there.
+ for (const path of wire.view.claims.keys()) this.#unindex(path, wire.id);
+ // What its last floor certified stays, so no other route's lagging claim revives.
+ this.#session.floors.leave(wire.node, wire.carrier);
+ wire.shown.clear();
+ this.#refreshRoute(wire, true);
+ }
+
+ #index(path: Path.Valid, id: bigint) {
+ let ids = this.#byPath.get(path);
+ if (!ids) {
+ ids = new Set();
+ this.#byPath.set(path, ids);
+ }
+ ids.add(id);
+ }
+
+ #unindex(path: Path.Valid, id: bigint) {
+ const ids = this.#byPath.get(path);
+ ids?.delete(id);
+ if (ids?.size === 0) this.#byPath.delete(path);
+ }
+
+ // A claim the peer started under `routeId` with the next Announce ID, at `path` unless this
+ // node refused it, and `claim` unless it refuses what the peer forwarded.
+ #startClaim(routeId: bigint, path: { suffix: Path.Valid; pathBase?: Base } | undefined, claim: Claim | undefined) {
+ const wire = this.#route(routeId);
+ // Every claim's path is relative to the session, whichever requested prefix it lies in.
+ const named = this.#history.start(path);
+ const id = this.#history.next - 1n;
+ if (claim) claim.above = aboveOf(named, claim.above);
+ if (named === undefined) {
+ if (claim) console.warn(`refusing a forwarded claim: route=${routeId} claim=${id} reason=path`);
+ this.#claims.set(id, { route: routeId });
+ wire.unnamed.add(id);
+ return;
+ }
+ if (wire.claims.has(named)) throw new ProtocolViolation(`route ${routeId} claims ${named} twice`);
+ // A route covers only the claims at or below its prefix.
+ if (wire.prefix !== undefined && !Path.hasPrefix(wire.prefix, named))
+ throw new ProtocolViolation(`route ${routeId} at ${wire.prefix} claims ${named}`);
+ this.#claims.set(id, { route: routeId, path: named, claim });
+ wire.claims.set(named, id);
+ if (claim) this.#change(routeId, named, claim);
+ else wire.withheld.add(id);
+ }
+
+ // The route now reaches its node by `offer`, or along `path` while this node refuses what the
+ // peer forwarded. Either way the peer no longer holds it, nor drains a node on it but as the
+ // offer says. A refused route leaves the node's floors, as an ended one does, and joins again
+ // once usable, with what its last floor certified, as rs/moq-net files it again.
+ #reach(wire: Route, offer: Offer | undefined, path: RouteHop[]) {
+ const floors = this.#session.floors;
+ const was = this.#filed(wire);
+ wire.offer = offer;
+ wire.path = path;
+ wire.held = false;
+ wire.drain = false;
+ const now = this.#filed(wire);
+ if (was && !now) {
+ floors.leave(wire.node, wire.carrier);
+ // What its floor certified stays as it was, while the route goes on reading claims.
+ wire.view.claims = new Map(wire.view.claims);
+ }
+ if (was || !now) return;
+ floors.join(wire.node, wire.carrier);
+ const others = floors.floor(wire.node, wire.carrier);
+ this.#show(wire, new Map(), true);
+ if (others) this.#revalidate();
+ }
+
+ // The route moved to `offer`, which this node refuses with its path when it is `Unusable`.
+ #move(wire: Route, offer: Offer | Unusable) {
+ const was = this.#filed(wire);
+ if ("refused" in offer) {
+ console.warn(`refusing a forwarded route: node=${wire.node} route=${wire.id} reason=${offer.refused}`);
+ this.#reach(wire, undefined, offer.path);
+ } else {
+ this.#reach(wire, offer, offer.path);
+ }
+ this.#refreshRoute(wire, !this.#usable(wire));
+ // Its floors no longer cover anything, as on ROUTE_END.
+ if (was && !this.#filed(wire)) this.#revalidate();
+ }
+
+ // A DEAD: the stream's routes through the hop it names carry on as it says, once the
+ // ROUTE_UPDATEs it counts are in. A route this node refuses still follows it, so both ends
+ // agree on which routes live. An endpoint casts no doubt: it never forwards a route, so no
+ // detour waits on one.
+ #dead(news: Dead) {
+ if (news.kind === "drain") {
+ for (const wire of this.#routes.values()) {
+ if (!wire.path.some((hop) => hop.node === news.node)) continue;
+ wire.drain = true;
+ this.#refreshRoute(wire, false);
+ }
+ return;
+ }
+ // It moves only routes through the hop, so what its updates hold stays within the stream's.
+ let crossing = 0n;
+ for (const wire of this.#routes.values()) if (through(wire, news.hop)) crossing++;
+ if (news.moved > crossing) throw new ProtocolViolation(`DEAD moves ${news.moved} of ${crossing} routes`);
+ const moving = { hop: news.hop, gone: news.gone, left: news.moved, offers: new Map() };
+ if (moving.left === 0n) this.#applyDead(moving);
+ else this.#moving = moving;
+ }
+
+ // One of the ROUTE_UPDATEs a DEAD counted: it names a live route through the hop, once.
+ #moved(moving: Moving, msg: AnnounceBroadcast) {
+ let id: bigint;
+ let offer: Offer | Unusable;
+ if (msg.status === "routeUpdate") {
+ id = msg.id;
+ offer = msg.offer;
+ } else if (msg.status === "refused" && msg.message.kind === "routeUpdate") {
+ id = msg.message.id;
+ offer = { refused: msg.reason, path: msg.message.path };
+ } else {
+ throw new ProtocolViolation(`${msg.status} inside a DEAD's ROUTE_UPDATEs`);
+ }
+ const wire = this.#routes.get(id);
+ if (!wire || !through(wire, moving.hop)) throw new ProtocolViolation(`DEAD moves route ${id} off its hop`);
+ if (moving.offers.has(id)) throw new ProtocolViolation(`DEAD moves route ${id} twice`);
+ moving.offers.set(id, offer);
+ moving.left -= 1n;
+ if (moving.left > 0n) return;
+ this.#moving = undefined;
+ this.#applyDead(moving);
+ }
+
+ // Apply a DEAD and the offers its ROUTE_UPDATEs gave as one: readers and consumes see only
+ // where every route it moves ends up.
+ #applyDead(moving: Moving) {
+ const touched = new Map>();
+ this.#deferred = touched;
+ try {
+ this.#session.consumes.batch(() => this.#deadRoutes(moving));
+ } finally {
+ this.#deferred = undefined;
+ }
+ for (const [reader, paths] of touched) {
+ for (const [at, lost] of paths) this.#refresh(reader, at, lost);
+ }
+ }
+
+ #deadRoutes({ hop, gone, offers }: Moving) {
+ const crossing = [...this.#routes.values()].filter((wire) => through(wire, hop));
+ for (const wire of crossing) {
+ const moved = offers.get(wire.id);
+ const before = wire.path.at(-2);
+ if (moved !== undefined) this.#move(wire, moved);
+ else if (gone) this.#endRoute(wire);
+ // The peer learned it from this node, which it holds under: unchanged.
+ else if (before?.node === this.#session.hop) continue;
+ else {
+ wire.held = true;
+ this.#refreshRoute(wire, false);
+ }
+ }
+ }
+
+ // Apply a message whose content, which the peer only forwarded, this node refuses: withdraw
+ // the route or claim it names, and keep the stream. Mirrors rs/moq-net.
+ #refused(message: Refusable, reason: AnnounceRefusal) {
+ switch (message.kind) {
+ case "routeStart":
+ console.warn(`refusing a forwarded route: node=${message.node} reason=${reason}`);
+ return this.#startRoute(
+ message.node,
+ message.prefix,
+ message.suffix,
+ message.exact,
+ undefined,
+ message.path,
+ );
+ case "routeUpdate":
+ return this.#move(this.#route(message.id), { refused: reason, path: message.path });
+ case "start":
+ console.warn(
+ `refusing a forwarded claim: route=${message.route} claim=${this.#history.next} reason=${reason}`,
+ );
+ return this.#startClaim(message.route, message.path, undefined);
+ case "change": {
+ const live = this.#claim(message.id);
+ console.warn(`refusing a forwarded claim: route=${live.route} claim=${message.id} reason=${reason}`);
+ live.claim = undefined;
+ if (live.path === undefined) return;
+ this.#route(live.route).withheld.add(message.id);
+ this.#change(live.route, live.path, undefined);
+ return;
+ }
+ }
+ }
+
+ // ANNOUNCE_FLOOR: the route's batch applies whole, at the revision its floor certifies: none
+ // while this node refuses one of the route's claims, which its origin never withdrew.
+ #floor(id: bigint, floor: bigint | undefined) {
+ const floors = this.#session.floors;
+ const wire = this.#route(id);
+ const revision = wire.unnamed.size === 0 && wire.withheld.size === 0 ? floor : undefined;
+ // A floor that certifies less keeps what the route's last one certified.
+ const certified = wire.view.revision;
+ const filed = this.#filed(wire);
+ if (filed && certified !== undefined && (revision === undefined || revision < certified))
+ floors.regress(wire.node, wire.carrier);
+ // The floor says what covers the prefix a widening edit took the route's claim from.
+ for (const path of wire.stale) if (!wire.pending.has(path)) wire.pending.set(path, undefined);
+ wire.stale.clear();
+ const touched = new Map();
+ for (const [path, claim] of wire.pending) {
+ // Another instance or epoch: what was consumed through the claim may be stale.
+ const was = wire.view.claims.get(path);
+ const restart = !!was && !!claim && (was.instance !== claim.instance || was.epoch !== claim.epoch);
+ touched.set(path, restart);
+ if (claim) {
+ wire.view.claims.set(path, claim);
+ this.#index(path, id);
+ } else {
+ wire.view.claims.delete(path);
+ this.#unindex(path, id);
+ }
+ }
+ wire.pending.clear();
+ wire.view.revision = revision;
+ // The floor answers the requested prefix as it stands, hidden claims included or not.
+ wire.view.hidden = this.#hidden(wire.region);
+ // Every other route to the node is shown anew first; with one, this route's unchanged
+ // claims may change state too.
+ const others = filed && floors.floor(wire.node, wire.carrier);
+ this.#show(wire, touched, others);
+ // Another view of the node may now cover a consumed path differently.
+ if (others) this.#revalidate();
+ }
+
+ // Whether the requested prefix `region` asks for hidden claims.
+ #hidden(region: Path.Valid): boolean {
+ for (const asked of this.#asked.values()) if (asked.live && asked.path === region) return asked.hidden;
+ return false;
+ }
+
+ // ANNOUNCE_OK: the last edit takes effect on the peer's stream. Its live routes carry over as
+ // `rekey.plan` says, which the peer applies alike, so nothing both sets cover is sent again.
+ #ok() {
+ const editing = this.#editing;
+ if (!editing) throw new ProtocolViolation("ANNOUNCE_OK with no ANNOUNCE_REQUEST unanswered");
+ // The marker lies between batches.
+ for (const wire of this.#routes.values()) {
+ if (wire.pending.size > 0) throw new ProtocolViolation("ANNOUNCE_OK inside a batch");
+ }
+ this.#editing = undefined;
+ const old = new Set();
+ for (const asked of this.#asked.values()) if (asked.live) old.add(asked.path);
+ for (const id of editing.remove) this.#asked.delete(id);
+ const added: Path.Valid[] = [];
+ for (const id of editing.add) {
+ const asked = this.#asked.get(id);
+ if (!asked) throw new Error("an added prefix");
+ asked.live = true;
+ added.push(asked.path);
+ }
+ const next = new Set();
+ for (const asked of this.#asked.values()) next.add(asked.path);
+
+ const table = new Map();
+ for (const [id, wire] of this.#routes) {
+ table.set(id, {
+ node: wire.node,
+ prefix: wire.prefix ?? wire.region,
+ region: wire.region,
+ exact: wire.exact,
+ order: rekey.order(wire.held, wire.offer, wire.path, this.#session.hop),
+ claims: wire.claims,
+ });
+ }
+ this.#carry(rekey.plan(old, next, added, table, this.#nextRoute, this.#history.next));
+
+ // What the readers want may have moved while the peer answered.
+ this.#ask();
+ this.#release();
+ }
+
+ // Carry the stream's routes over an edit as `plan` says: a route that stays keeps everything,
+ // one that widens or narrows is filed again under its new prefix, and every other route ends.
+ #carry(plan: rekey.Plan) {
+ const { floors, consumes } = this.#session;
+ const sources = new Map(this.#routes);
+ const records = new Map(this.#claims);
+ // The routes filed again, the ones they replace or the edit ends, and the claims that live on.
+ const fresh: Route[] = [];
+ const ended: Route[] = [];
+ const stays = new Set();
+ const placed = new Map();
+ // The claims an edit presents below their own path, in the order it assigned their ids.
+ const inserted: { id: bigint; path: Path.Valid }[] = [];
+ // The routes the plan carries on, each of which keeps its own claims.
+ const carriedFrom = new Set(plan.routes.map((carried) => carried.from));
+
+ for (const carried of plan.routes) {
+ const source = sources.get(carried.from);
+ if (!source) throw new Error("a carried route's source");
+ if (carried.kind === "stays") {
+ stays.add(source.id);
+ for (const id of [...source.claims.values(), ...source.unnamed]) {
+ const record = records.get(id);
+ if (record) placed.set(id, record);
+ }
+ source.region = carried.region;
+ // A prefix that no longer asks for hidden claims certifies nothing about them.
+ if (!this.#hidden(carried.region)) source.view.hidden = false;
+ continue;
+ }
+ const wire = this.#blank(carried.id, source, {
+ region: carried.region,
+ prefix: source.prefix === undefined ? undefined : carried.prefix,
+ exact: carried.exact,
+ hidden: source.view.hidden && this.#hidden(carried.region),
+ // A widened route's floors certified only the narrower prefix.
+ revision: carried.kind === "widens" ? undefined : source.view.revision,
+ });
+ for (const [path, claim] of carried.claims) {
+ const record = records.get(claim.from);
+ if (!record) throw new Error("a carried claim's record");
+ // A claim now covering a narrower prefix from above is that much more above it.
+ const held =
+ record.claim && claim.lowered > 0n
+ ? { ...record.claim, above: record.claim.above + claim.lowered }
+ : record.claim;
+ wire.claims.set(path, claim.id);
+ if (held) wire.view.claims.set(path, held);
+ else wire.withheld.add(claim.id);
+ placed.set(claim.id, { route: wire.id, path, claim: held });
+ if (claim.id !== claim.from) inserted.push({ id: claim.id, path });
+ }
+ if (carried.kind === "widens") {
+ // A claim without a path stays with the route it was sent under.
+ for (const id of source.unnamed) {
+ wire.unnamed.add(id);
+ placed.set(id, { route: wire.id });
+ }
+ // The claim at each merged route's old prefix shows until the next floor says what
+ // covers it now: the routes the edit merged into this one end without a successor.
+ const merged = [...sources.values()].filter(
+ (other) =>
+ other.id === source.id ||
+ (!carriedFrom.has(other.id) &&
+ other.node === source.node &&
+ Path.hasPrefix(carried.region, other.region)),
+ );
+ for (const other of merged) {
+ const own = other.prefix === undefined ? undefined : other.view.claims.get(other.prefix);
+ if (!own || other.prefix === undefined || wire.view.claims.has(other.prefix)) continue;
+ wire.view.claims.set(other.prefix, own);
+ wire.stale.add(other.prefix);
+ }
+ }
+ fresh.push(wire);
+ }
+
+ // Let go of what the edit ends before taking the slots of what it starts.
+ for (const id of [...this.#claims.keys()]) {
+ if (placed.has(id)) continue;
+ this.#history.end(id);
+ this.#claims.delete(id);
+ }
+ const replaced = new Set(fresh.map((wire) => wire.id));
+ for (const [id, wire] of sources) {
+ if (stays.has(id)) continue;
+ ended.push(wire);
+ if (!replaced.has(id)) this.#routes.delete(id);
+ }
+ for (const { id, path } of inserted) {
+ if (this.#history.insert(path) !== id) throw new Error("the edit assigned another Announce ID");
+ }
+ if (this.#history.next !== plan.nextClaim) throw new Error("the edit assigned other Announce IDs");
+ for (const [id, record] of placed) this.#claims.set(id, record);
+ for (const wire of fresh) this.#routes.set(wire.id, wire);
+ this.#nextRoute = plan.nextRoute;
+ this.#named.clear();
+ for (const [id, wire] of this.#routes) {
+ const named = this.#key(wire.node, wire.prefix);
+ if (named !== undefined) this.#named.set(named, id);
+ }
+
+ // Every path a reader may be shown differently, once the routes are filed again.
+ const touched = new Set();
+ const shows = (wire: Route) => {
+ for (const path of wire.view.claims.keys()) {
+ const claim = floors.shows(wire.node, wire.view, path);
+ if (claim) wire.shown.set(path, claim);
+ else wire.shown.delete(path);
+ }
+ };
+ // The routes filed again show before the ones they replace go, so neither a reader nor a
+ // consume of a claim both carry finds it gone between.
+ for (const wire of fresh) {
+ for (const path of wire.view.claims.keys()) {
+ this.#index(path, wire.id);
+ touched.add(path);
+ }
+ if (this.#joined(wire)) floors.join(wire.node, wire.carrier);
+ shows(wire);
+ }
+ for (const wire of ended) {
+ // A widened route keeps its Route ID, and with it the entry of each claim it carried over.
+ const kept = replaced.has(wire.id) ? this.#routes.get(wire.id) : undefined;
+ for (const path of wire.view.claims.keys()) {
+ if (!kept?.view.claims.has(path)) this.#unindex(path, wire.id);
+ touched.add(path);
+ }
+ }
+ for (const wire of ended) floors.leave(wire.node, wire.carrier);
+ for (const wire of fresh) {
+ shows(wire);
+ for (const path of wire.view.claims.keys()) this.#serve(wire, path, false);
+ }
+ for (const wire of ended) {
+ for (const path of wire.view.claims.keys()) consumes.withdraw(path, wire.owner);
+ }
+ if (fresh.length > 0 || ended.length > 0) this.#revalidate();
+ // A claim no route carries any more was let go, not withdrawn by its origin.
+ for (const path of touched) this.#touch(path, true);
+ }
+
+ /** Apply one message the peer sent on its Announce Stream as a publisher. */
+ async receive(msg: AnnounceBroadcast): Promise {
+ if (this.#closing) return;
+ // A DEAD's ROUTE_UPDATEs arrive back to back.
+ if (this.#moving) return this.#moved(this.#moving, msg);
+
+ switch (msg.status) {
+ case "ok":
+ this.#ok();
+ break;
+ case "synced": {
+ const asked = this.#asked.get(msg.prefix);
+ if (!asked?.live) throw new ProtocolViolation(`SYNCED for unknown Prefix ID ${msg.prefix}`);
+ if (asked.synced) throw new ProtocolViolation(`duplicate SYNCED for Prefix ID ${msg.prefix}`);
+ asked.synced = true;
+ this.#release();
+ break;
+ }
+ case "request":
+ // The peer's own request is answered by the publisher half: here it only has to
+ // arrive outside a DEAD's updates.
+ break;
+ case "routeStart":
+ this.#startRoute(msg.node, msg.prefix, msg.suffix, msg.exact, msg.offer, msg.offer.path);
+ break;
+ case "routeUpdate":
+ this.#move(this.#route(msg.id), msg.offer);
+ break;
+ case "routeHold": {
+ const wire = this.#route(msg.id);
+ wire.held = true;
+ this.#refreshRoute(wire, false);
+ break;
+ }
+ case "routeEnd":
+ this.#endRoute(this.#route(msg.id));
+ break;
+ case "dead":
+ this.#dead(msg.dead);
+ break;
+ case "floor":
+ this.#floor(msg.id, msg.revision);
+ break;
+ case "active": {
+ if (msg.route === undefined) throw new ProtocolViolation("lite-07 claim without a route");
+ if (msg.instance === undefined) throw new ProtocolViolation("lite-07 claim without an instance");
+ const claim = {
+ instance: msg.instance,
+ epoch: msg.epoch ?? 0n,
+ cost: msg.cost ?? Cost.zero,
+ above: msg.above ?? 0n,
+ };
+ this.#startClaim(msg.route, msg, claim);
+ break;
+ }
+ case "update": {
+ const live = this.#claim(msg.id);
+ const above = aboveOf(live.path, msg.above);
+ // An update names none of what a refused claim lacks.
+ if (live.claim === undefined || live.path === undefined) break;
+ live.claim = { ...live.claim, cost: msg.cost ?? Cost.zero, above };
+ this.#change(live.route, live.path, live.claim);
+ break;
+ }
+ case "restart": {
+ const live = this.#claim(msg.id);
+ if (msg.instance === undefined) throw new ProtocolViolation("lite-07 restart without an instance");
+ // A claim without a path stays refused, whatever the peer sends for it.
+ if (live.path === undefined) break;
+ live.claim = {
+ instance: msg.instance,
+ epoch: msg.epoch ?? 0n,
+ cost: msg.cost ?? Cost.zero,
+ above: aboveOf(live.path, msg.above),
+ };
+ this.#route(live.route).withheld.delete(msg.id);
+ this.#change(live.route, live.path, live.claim);
+ break;
+ }
+ case "refused":
+ this.#refused(msg.message, msg.reason);
+ break;
+ case "endedId": {
+ const live = this.#claim(msg.id);
+ this.#history.end(msg.id);
+ this.#claims.delete(msg.id);
+ const wire = this.#route(live.route);
+ if (live.path === undefined) {
+ wire.unnamed.delete(msg.id);
+ break;
+ }
+ wire.claims.delete(live.path);
+ wire.withheld.delete(msg.id);
+ this.#change(live.route, live.path, undefined);
+ break;
+ }
+ case "skipped":
+ break;
+ case "ended":
+ // Lite-07 ends a claim by its Announce ID only.
+ throw new ProtocolViolation("ANNOUNCE_END by path on lite-07");
+ }
+ }
+}
+
+// Whether `reader` reports a claim at `path`.
+function visible(reader: Reader, path: Path.Valid): boolean {
+ const matches = reader.covering ? scopeOverlaps(reader.scope, path) : reader.scope.matches(path);
+ return matches && (reader.hidden || !hiddenBelow(reader.prefix, path));
+}
+
+// Where `reader` is shown a claim at `path`: there when it lies within the reader's prefix, at
+// the prefix itself when it covers more, and nowhere otherwise.
+function presentedAt(reader: Reader, path: Path.Valid): Path.Valid | undefined {
+ if (Path.hasPrefix(reader.prefix, path)) return path;
+ return Path.hasPrefix(path, reader.prefix) ? reader.prefix : undefined;
+}
diff --git a/js/net/src/lite/stream.ts b/js/net/src/lite/stream.ts
index 486487ede3b..2b10610f295 100644
--- a/js/net/src/lite/stream.ts
+++ b/js/net/src/lite/stream.ts
@@ -24,6 +24,8 @@ export const DataType = {
Setup: 1,
/** Arbitrary bytes discarded by lite-07+ receivers. */
Padding: 2,
+ /** Lite-07: the sender's one Announce Stream. */
+ Announce: 3,
} as const;
/** A unidirectional data-stream type. See {@link DataType}. */
diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts
index 6ca2d3ec1e5..3ad21d2f8c4 100644
--- a/js/net/src/lite/subscriber.test.ts
+++ b/js/net/src/lite/subscriber.test.ts
@@ -35,18 +35,21 @@ import { SEGMENT_BYTES } from "../util/limits.ts";
import { Budget } from "../util/yield.ts";
import * as Varint from "../varint.ts";
import {
+ type Add,
type AnnounceBroadcast,
AnnounceInit,
AnnounceOk,
- AnnounceRequest,
+ decodeAnnounceBroadcast,
+ decodeAnnounceBroadcastMaybe,
encodeAnnounceBroadcast,
type Offer,
ROUTE_MAX_HOPS,
} from "./announce.ts";
+import { closeViolation } from "./error.ts";
import { Group } from "./group.ts";
import { Probe } from "./probe.ts";
import { ProbeLevel, Setup } from "./setup.ts";
-import { StreamId } from "./stream.ts";
+import { DataType, StreamId } from "./stream.ts";
import {
encodeSubscribeResponse,
Subscribe,
@@ -90,8 +93,10 @@ test("closing the subscriber suppresses probe stream warnings", async () => {
});
// Drives a Subscriber's announce stream directly: the harness plays the peer, writing
-// forged announce messages into the stream the subscriber opens.
+// forged announce messages into the stream the subscriber opens. On lite-07 they are the
+// peer's Announce Stream instead, delivered in order as the session does.
function announceHarness(version: Version, origin = 1n) {
+ const routed = version === Version.DRAFT_07;
const reading = Promise.withResolvers();
let inbound!: ReadableStreamDefaultController;
// Resolves with the reason once the subscriber resets the stream, which is how a peer
@@ -114,6 +119,8 @@ function announceHarness(version: Version, origin = 1n) {
),
writable: new WritableStream({ abort: (reason) => void onAbort(reason) }),
}),
+ // Lite-07: the subscriber's own Announce Stream, whose first write is its request.
+ createUnidirectionalStream: async () => new WritableStream({ write: () => reading.resolve() }),
close: (info?: WebTransportCloseInfo) => {
// WebTransport throws rather than closing when the reason exceeds 1024 bytes of
// UTF-8. Enforced here so an unbounded reason fails as a session that stayed up,
@@ -144,8 +151,25 @@ function announceHarness(version: Version, origin = 1n) {
out.set(chunk, offset);
offset += chunk.byteLength;
}
- inbound.enqueue(out);
+ if (!routed) {
+ inbound.enqueue(out);
+ return;
+ }
+ // A violation ends the session, as the connection does for one on the peer's Announce Stream.
+ try {
+ const reader = new Reader(undefined, out, version);
+ while (!(await reader.done())) await subscriber.announce(await decodeAnnounceBroadcast(reader, version));
+ } catch (err) {
+ const violation = closeViolation(quic, err);
+ if (!violation) throw err;
+ subscriber.close(violation);
+ }
};
+ // The peer answers the request: ANNOUNCE_OK, on whichever stream the version carries it.
+ const ok = () =>
+ send((w) =>
+ routed ? encodeAnnounceBroadcast(w, { status: "ok" }, version) : new AnnounceOk(PEER, 0).encode(w, version),
+ );
// The reason the subscriber reset the stream, or a prompt rejection if it never did.
// A peer that is not told about its own violation should fail the test visibly rather
@@ -172,8 +196,9 @@ function announceHarness(version: Version, origin = 1n) {
return {
subscriber,
send,
+ ok,
// The peer FINs the announce stream.
- finish: () => inbound.close(),
+ finish: () => (routed ? subscriber.announceFinished() : inbound.close()),
abortReason,
sessionEnded,
settle: async () => {
@@ -228,7 +253,7 @@ function claimer(version: Version) {
const route = routes++;
await encodeAnnounceBroadcast(
w,
- { status: "routeStart", node, suffix: Path.empty(), offer: offer(node) },
+ { status: "routeStart", node, prefix: 0n, suffix: Path.empty(), exact: false, offer: offer(node) },
version,
);
await encodeAnnounceBroadcast(w, { status: "active", suffix, route, instance }, version);
@@ -412,13 +437,13 @@ test("an update moves the route in place, even from another publisher", async ()
});
test("a lite-07 restart replaces the instance, and the next consume subscribes fresh", async () => {
- const { subscriber, send, settle } = announceHarness(Version.DRAFT_07);
+ const { subscriber, send, ok, settle } = announceHarness(Version.DRAFT_07);
const announced = subscriber.routes();
await settle();
const room = Path.from("room");
const v = Version.DRAFT_07;
- await send((w) => new AnnounceOk(PEER, 0).encode(w, v));
+ await ok();
const first = Instance.random();
await send((w) => claimer(v)(w, room, { instance: first, hops: [], node: NODE_A }));
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { instance: first } });
@@ -457,13 +482,13 @@ test.each([
["a lite-07 restart", Version.DRAFT_07],
["an end then a start", Version.DRAFT_06],
] as const)("%s of a prefix stops sharing what was consumed beneath it", async (_, version) => {
- const { subscriber, send, settle } = announceHarness(version);
+ const { subscriber, send, ok, settle } = announceHarness(version);
const announced = subscriber.routes();
await settle();
const pool = Path.from("pool");
const job = Path.from("pool/job");
- await send((w) => new AnnounceOk(PEER, 0).encode(w, version));
+ await ok();
await send((w) => claimer(version)(w, pool, { instance: named(version, "a"), hops: [PUBLISHER_A], node: NODE_A }));
expect(await announced.next()).toMatchObject({ prefix: pool, kind: "start" });
const held = subscriber.consume(job);
@@ -2822,38 +2847,44 @@ test("retirement preserves an accepted fetch and its coalesced reads", async ()
// Drive discovery and inspect the TRACK request before answering it. Waiting for the
// reader means the whole request has reached the wire, without a clock-based settle.
test("consumes resolve the serving prefix and pin its instance", async () => {
- const version = Version.DRAFT_07;
- const { quic, streams, opened } = fakeSession();
- const subscriber = new Subscriber(quic, version, toHop(1n));
- const announced = subscriber.routes();
- await opened(0);
- const peer = new Writer(
- new WritableStream({
- write: (chunk) => streams[0].inbound.enqueue(new Uint8Array(chunk)),
- }),
- version,
- );
- await new AnnounceOk(PEER, 0).encode(peer, version);
+ const { subscriber, announced, send, opened, version } = await routedStream();
const pool = Path.from("pool");
const job = Path.from("pool/job");
const instance = Instance.random();
- const claim = claimer(version);
- await claim(peer, pool, { instance, hops: [], node: NODE_A });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: pool, route: 0n, instance });
+ await send({ status: "floor", id: 0n });
await announced.next();
const held = subscriber.consume(job);
const info = held
.track("video")
.info()
.catch(() => {});
- await opened(1);
- await streams[1].reading;
- const reader = new Reader(undefined, new Uint8Array(Buffer.concat(streams[1].written)), version);
+ const first = await opened(0);
+ await first.reading;
+ const reader = new Reader(undefined, new Uint8Array(Buffer.concat(first.written)), version);
expect(await reader.u53()).toBe(StreamId.Track);
expect(await Track.decode(reader, version)).toMatchObject({ instance, node: NODE_A });
// The more specific source's instance shadows the covering source's.
const other = Instance.random();
- await claim(peer, job, { instance: other, hops: [], node: NODE_B });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
+ await send({ status: "active", suffix: job, route: 1n, instance: other });
+ await send({ status: "floor", id: 1n });
await announced.next();
const fresh = subscriber.consume(job);
expect(fresh.closed).not.toBe(held.closed);
@@ -2862,14 +2893,14 @@ test("consumes resolve the serving prefix and pin its instance", async () => {
.track("video")
.info()
.catch(() => {});
- await opened(2);
- await streams[2].reading;
- const freshReader = new Reader(undefined, new Uint8Array(Buffer.concat(streams[2].written)), version);
+ const second = await opened(1);
+ await second.reading;
+ const freshReader = new Reader(undefined, new Uint8Array(Buffer.concat(second.written)), version);
await freshReader.u53();
expect(await Track.decode(freshReader, version)).toMatchObject({ instance: other, node: NODE_B });
- await encodeAnnounceBroadcast(peer, { status: "endedId", id: 1n }, version);
- await encodeAnnounceBroadcast(peer, { status: "floor", id: 1n }, version);
+ await send({ status: "endedId", id: 1n });
+ await send({ status: "floor", id: 1n });
await announced.next();
const fallback = subscriber.consume(job);
expect(fallback.closed).toBe(held.closed);
@@ -2885,10 +2916,10 @@ test("consumes resolve the serving prefix and pin its instance", async () => {
test.each([Version.DRAFT_06, Version.DRAFT_07])(
"a more specific start replaces an existing consume on %s",
async (version) => {
- const { subscriber, send, settle } = announceHarness(version);
+ const { subscriber, send, ok, settle } = announceHarness(version);
const announced = subscriber.routes();
await settle();
- await send((w) => new AnnounceOk(PEER, 0).encode(w, version));
+ await ok();
const pool = Path.from("pool");
const job = Path.from("pool/job");
const claim = claimer(version);
@@ -4300,30 +4331,73 @@ test("a cancelled legacy datagram lookup cannot refuse a stream answer arriving
}
});
-// A lite-07 peer's announce stream, after the request: what the peer writes, and the replies
-// the subscriber sends back. `another` opens a further one on the session as its `index`th stream.
-async function routedStream(options?: RouteOptions) {
+// A lite-07 session whose peer the test plays: what the peer sends on its Announce Stream reaches
+// the subscriber in order, as the session delivers it, and each edit the subscriber sends on its
+// own is answered at once. `another` adds a reader, which shares the stream.
+async function routedStream(options?: RouteOptions, scope?: Path.Pattern) {
const version = Version.DRAFT_07;
- const { quic, opened } = fakeSession();
+ const { quic, opened, closes } = fakeSession();
+ let sent!: ReadableStreamDefaultController;
+ const outbound = new ReadableStream({ start: (controller) => (sent = controller) });
+ Object.assign(quic, {
+ createUnidirectionalStream: async () =>
+ new WritableStream({
+ write: (chunk) => sent.enqueue(new Uint8Array(chunk)),
+ close: () => sent.close(),
+ }),
+ });
const subscriber = new Subscriber(quic, version, toHop(1n));
- const another = async (index: number, scope?: Path.Pattern) => {
- const announced = subscriber.routes(scope, options);
- const stream = await opened(index);
- const peer = new Writer(
- new WritableStream({ write: (chunk) => stream.inbound.enqueue(new Uint8Array(chunk)) }),
- version,
- );
- const replies = new Reader(stream.outbound, undefined, version);
- expect(await replies.u53()).toBe(StreamId.Announce);
- await AnnounceRequest.decode(replies, version);
- await new AnnounceOk(PEER, 0).encode(peer, version);
- const send = (msg: AnnounceBroadcast) => encodeAnnounceBroadcast(peer, msg, version);
- // Bytes as a newer or broken peer might write them.
- const raw = (bytes: number[]) => stream.inbound.enqueue(new Uint8Array(bytes));
- return { announced, send, raw, replies };
+
+ // One message at a time, in the order the peer wrote them. A violation ends the session, as
+ // the connection does for one on the peer's Announce Stream.
+ let delivering = Promise.resolve();
+ const deliver = (bytes: Uint8Array | Promise) => {
+ const done = delivering.then(async () => {
+ try {
+ const msg = await decodeAnnounceBroadcast(new Reader(undefined, await bytes, version), version);
+ await subscriber.announce(msg);
+ } catch (err) {
+ const violation = closeViolation(quic, err);
+ if (!violation) throw err;
+ subscriber.close(violation);
+ }
+ });
+ delivering = done.catch(() => {});
+ return done;
+ };
+ const send = (msg: AnnounceBroadcast) =>
+ deliver(Writer.collect(version, (w) => encodeAnnounceBroadcast(w, msg, version)));
+ // Bytes as a newer or broken peer might write them.
+ const raw = (bytes: number[]) => deliver(new Uint8Array(bytes));
+
+ // The edits the subscriber sent, each answered with an ANNOUNCE_OK as it arrives.
+ const edits: { remove: bigint[]; add: Add[] }[] = [];
+ const replies = new Reader(outbound, undefined, version);
+ void (async () => {
+ expect(await replies.u53()).toBe(DataType.Announce);
+ for (;;) {
+ const msg = await decodeAnnounceBroadcastMaybe(replies, version);
+ if (!msg) break;
+ if (msg.status !== "request") throw new Error(`the subscriber sent ${msg.status}`);
+ edits.push({ remove: msg.remove, add: msg.add });
+ void send({ status: "ok" });
+ }
+ })();
+ // Everything the subscriber had to send is out, and the peer's answers to it are in.
+ const settled = async () => {
+ for (let i = 0; i < 8; i++) await yieldNow();
+ await delivering;
+ };
+
+ const another = async (scope?: Path.Pattern, more?: RouteOptions) => {
+ const announced = subscriber.routes(scope, more ?? options);
+ await settled();
+ return { announced };
};
- const { announced, send, raw, replies } = await another(0);
- // The TRACK request a consume of `path` opens as its `index`th stream.
+ const announced = subscriber.routes(scope, options);
+ while (edits.length === 0) await yieldNow();
+ await settled();
+ // The TRACK request a consume of `path` opens as the session's `index`th bidirectional stream.
const trackRequest = async (path: Path.Valid, index: number, pin?: Instance.Valid) => {
const consumed = subscriber.consume(path, pin);
const info = consumed
@@ -4336,7 +4410,7 @@ async function routedStream(options?: RouteOptions) {
const request = await Track.decode(reader, version);
return { request, consumed, info };
};
- return { subscriber, announced, send, raw, replies, version, trackRequest, another, opened };
+ return { subscriber, announced, send, raw, edits, settled, closes, version, trackRequest, another, opened };
}
// A route or claim whose forwarded content this subscriber refuses is withdrawn alone: the session,
@@ -4346,8 +4420,22 @@ test("a lite-07 refused route or claim withdraws only itself", async () => {
const [a, b, c, d] = ["live/a", "live/b", "live/c", "live/d"].map((path) => Path.from(path));
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: a, route: 0n, instance });
await send({ status: "active", suffix: b, route: 1n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
@@ -4370,8 +4458,8 @@ test("a lite-07 refused route or claim withdraws only itself", async () => {
await send({ status: "routeUpdate", id: 0n, offer: offer(NODE_A) });
expect(await announced.next()).toMatchObject({ prefix: a, kind: "start" });
- // ROUTE_START to the reserved node takes Route ID 2, and its claims never show.
- raw([6, 8, 0, 0, 1, 5, 0, 0, 128, 0]);
+ // ROUTE_START to the reserved node, not exact, takes Route ID 2, and its claims never show.
+ raw([6, 10, 0, 0, 0, 0, 1, 5, 0, 0, 128, 0]);
await send({ status: "active", suffix: d, route: 2n, instance });
await send({ status: "floor", id: 2n, revision: 0n });
await send({ status: "endedId", id: 3n });
@@ -4390,7 +4478,14 @@ test("a lite-07 refused claim goes offline, its route certifying nothing meanwhi
const [a, b, e] = ["live/a", "live/b", "live/e"].map((path) => Path.from(path));
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: a, route: 0n, instance });
await send({ status: "active", suffix: b, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
@@ -4428,61 +4523,86 @@ test("a lite-07 refused claim goes offline, its route certifying nothing meanwhi
subscriber.close();
});
-// Another stream certifying the refused claim still present proves no withdrawal either: the
-// refusing stream's reader sees it go offline.
-test("a lite-07 refused claim another stream still certifies goes offline", async () => {
- const { subscriber, announced, send, raw, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+// A peer may carry one origin's claims on two routes, each at a prefix of its own. A claim this
+// node refuses on one of them is not withdrawn, and the other still certifies it: it stays shown.
+test("a lite-07 refused claim another route still certifies stays shown", async () => {
+ const { subscriber, announced, send, raw } = await routedStream();
+ const live = Path.from("live");
const x = Path.from("live/x");
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start" });
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance });
- await live.send({ status: "floor", id: 0n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start" });
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 1n, instance });
+ await send({ status: "floor", id: 1n, revision: 0n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 1 } });
// ANNOUNCE_RESTART of Announce ID 0 to an empty instance.
- raw([3, 5, 0, 0, 0, 0, 0]);
+ await raw([3, 5, 0, 0, 0, 0, 0]);
await send({ status: "floor", id: 0n, revision: 1n });
+ // Another node's claim marks the refusal as applied: live/x neither went offline nor ended.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
+ await send({ status: "active", suffix: Path.from("live/z"), route: 2n, instance });
+ await send({ status: "floor", id: 2n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
+
+ // Once the other route lets it go too, nothing certifies its withdrawal: it goes offline.
+ await send({ status: "routeEnd", id: 1n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "offline" });
- live.announced.close();
announced.close();
subscriber.close();
});
// What the refusing route's last floor certified still counts against an older floor of another
-// stream: a lagging stream that never had the claim proves no withdrawal of it.
-test("a lite-07 refused claim a lagging stream never had goes offline", async () => {
- const { subscriber, announced, send, raw, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+// route: a lagging route that never had the claim proves no withdrawal of it.
+test("a lite-07 refused claim a lagging route never had goes offline", async () => {
+ const { subscriber, announced, send, raw } = await routedStream();
+ const live = Path.from("live");
const x = Path.from("live/x");
+ const y = Path.from("live/y");
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 7n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start" });
- // The second stream lags at revision 5, before live/x existed.
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("y"), route: 0n, instance });
- await live.send({ status: "floor", id: 0n, revision: 5n });
- // A claim of another node on that stream shows once it read all of the above.
- await live.send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
- await live.send({ status: "active", suffix: Path.from("z"), route: 1n, instance });
- await live.send({ status: "floor", id: 1n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
+ // The second route lags at revision 5, before live/x existed.
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: y, route: 1n, instance });
+ await send({ status: "floor", id: 1n, revision: 5n });
+ // The first route's newer floor has no live/y, so the lagging copy of it stays held back.
// ANNOUNCE_RESTART of Announce ID 0 to an empty instance.
- raw([3, 5, 0, 0, 0, 0, 0]);
+ await raw([3, 5, 0, 0, 0, 0, 0]);
await send({ status: "floor", id: 0n, revision: 8n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "offline" });
- live.announced.close();
announced.close();
subscriber.close();
});
@@ -4493,7 +4613,14 @@ test("a lite-07 claim at the root updates and ends", async () => {
const { subscriber, announced, send } = await routedStream();
const root = Path.empty();
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: root, route: 0n, instance: Instance.parse("i") });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: root, kind: "start" });
@@ -4516,34 +4643,35 @@ test("a lite-07 claim at the root updates and ends", async () => {
});
// A route this subscriber refused is no view of its node: a newer floor on it hides nothing
-// another stream carries, as rs/moq-net's node never holds the route.
-test("a lite-07 refused route's floors hide no other stream's claims", async () => {
- const { subscriber, announced, send, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+// another route carries, as rs/moq-net's node never holds the route.
+test("a lite-07 refused route's floors hide no other route's claims", async () => {
+ const { subscriber, announced, send, raw } = await routedStream();
const instance = Instance.parse("i");
// One varint byte, so the raw message below spells it.
const node = 7n;
- await send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: node,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(node),
+ });
await send({ status: "active", suffix: Path.from("live/x"), route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: Path.from("live/x"), kind: "start" });
- // ROUTE_START to the node with a flag no version defines, then a newer floor without live/x.
- live.raw([6, 9, 7, 0, 1, 7, 0, 0x80, 128, 0, 42]);
- await live.send({ status: "floor", id: 0n, revision: 5n });
- // A claim of another node on that stream shows once it read all of the above.
- await live.send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
- await live.send({ status: "active", suffix: Path.from("z"), route: 1n, instance });
- await live.send({ status: "floor", id: 1n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
+ // ROUTE_START to the node at live under Prefix ID 0, not exact, with a flag no version defines,
+ // then a newer floor without live/x.
+ await raw([6, 15, 7, 0, 4, ...new TextEncoder().encode("live"), 0, 1, 7, 0, 0x80, 128, 0, 42]);
+ await send({ status: "floor", id: 1n, revision: 5n });
- // What the first stream shows next is its own new claim, not live/x withdrawn.
+ // What shows next is the first route's own new claim, not live/x withdrawn.
await send({ status: "active", suffix: Path.from("live/y"), route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 1n });
expect(await announced.next()).toMatchObject({ prefix: Path.from("live/y"), kind: "start" });
- live.announced.close();
announced.close();
subscriber.close();
});
@@ -4551,181 +4679,219 @@ test("a lite-07 refused route's floors hide no other stream's claims", async ()
// A route refused once live leaves what its last floor certified, as an ended one does: the claims
// it reads afterwards never rewrite that.
test("a lite-07 route refused once live leaves its last floor as it was", async () => {
- const { subscriber, announced, send, raw, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+ const { subscriber, announced, send, raw } = await routedStream();
+ const live = Path.from("live");
const instance = Instance.parse("i");
const node = 7n;
const x = Path.from("live/x");
- // Both streams carry live/x, the first at revision 5 and the second lagging at 3.
- await send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node) });
+ // Both routes carry live/x, the first at revision 5 and the second lagging at 3.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: node,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(node),
+ });
await send({ status: "active", suffix: x, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 5n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start" });
- await live.send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance });
- await live.send({ status: "floor", id: 0n, revision: 3n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start" });
-
- // The first stream's route is refused, then ends live/x at a floor no view takes.
- raw([7, 8, 0, 1, 5, 0, 0x80, 128, 0, 42]);
- expect(await announced.next()).toMatchObject({ prefix: x, kind: "offline" });
+ await send({ status: "routeStart", prefix: 0n, node: node, suffix: live, exact: false, offer: offer(node) });
+ await send({ status: "active", suffix: x, route: 1n, instance });
+ await send({ status: "floor", id: 1n, revision: 3n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 1 } });
+
+ // The first route is refused, then ends live/x at a floor no view takes.
+ await raw([7, 8, 0, 1, 5, 0, 0x80, 128, 0, 42]);
await send({ status: "endedId", id: 0n });
await send({ status: "floor", id: 0n, revision: 6n });
- // The second stream's next floor still has live/x, as revision 5 did: what it shows next is
+ // The second route's next floor still has live/x, as revision 5 did: what shows next is
// another node's claim, not live/x withdrawn.
- await live.send({ status: "floor", id: 0n, revision: 4n });
- await live.send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
- await live.send({ status: "active", suffix: Path.from("z"), route: 1n, instance });
- await live.send({ status: "floor", id: 1n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
+ await send({ status: "floor", id: 1n, revision: 4n });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
+ await send({ status: "active", suffix: Path.from("live/z"), route: 2n, instance });
+ await send({ status: "floor", id: 2n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
- live.announced.close();
announced.close();
subscriber.close();
});
-// Two streams carry one origin's claims, and a withdrawal reaches one first. Of the floors that
-// certify a claim, the newest decides whether it exists, so the lagging stream's copy is held back
-// rather than selected and refused: also once the newer stream's route ends.
-test("a lite-07 claim another stream's newer floor withdrew is not selected", async () => {
- const { subscriber, announced, send, trackRequest, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+// Two routes carry one origin's claims, and a withdrawal reaches one first. Of the floors that
+// certify a claim, the newest decides whether it exists, so the lagging route's copy is held back
+// rather than selected and refused: also once the newer route ends.
+test("a lite-07 claim another route's newer floor withdrew is not selected", async () => {
+ const { subscriber, announced, send, trackRequest } = await routedStream();
+ const live = Path.from("live");
const x = Path.from("live/x");
const a = Instance.parse("a");
const b = Instance.parse("b");
const y = Instance.parse("y");
- // NODE_A claims live/x and live/y, cheaply, on both streams at revision 0.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ // NODE_A claims live/x and live/y, cheaply, on both routes at revision 0.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance: a });
await send({ status: "active", suffix: Path.from("live/y"), route: 0n, instance: y });
await send({ status: "floor", id: 0n, revision: 0n });
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance: a });
- await live.send({ status: "active", suffix: Path.from("y"), route: 0n, instance: y });
- await live.send({ status: "floor", id: 0n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- expect(await live.announced.next()).toMatchObject({ prefix: Path.from("live/y"), kind: "start" });
- // A dearer NODE_B claims live/x on the first stream only.
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
- await send({ status: "active", suffix: x, route: 1n, instance: b });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/y"), kind: "start" });
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 1n, instance: a });
+ await send({ status: "active", suffix: Path.from("live/y"), route: 1n, instance: y });
await send({ status: "floor", id: 1n, revision: 0n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 1 } });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/y"), kind: "update", route: { depth: 1 } });
+ // A dearer NODE_B claims live/x.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
+ await send({ status: "active", suffix: x, route: 2n, instance: b });
+ await send({ status: "floor", id: 2n, revision: 0n });
- // NODE_A withdraws live/x at revision 1, which only the first stream has delivered.
+ // NODE_A withdraws live/x at revision 1, which only the first route has delivered: the
+ // lagging route's copy is held back, so the path moves to NODE_B.
await send({ status: "endedId", id: 0n });
await send({ status: "floor", id: 0n, revision: 1n });
- // The lagging stream ends its copy, and keeps the claim it still agrees on.
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "end", route: { node: NODE_A } });
- const first = await trackRequest(x, 2);
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "restart", route: { node: NODE_B } });
+ const first = await trackRequest(x, 0);
expect(first.request).toMatchObject({ instance: b, node: NODE_B });
- // The first stream lets its route to NODE_A go, and a third stream, as far behind, then lands
- // its copy: what the ended route's floor certified still holds, so a fresh pick of live/x
- // still passes over the cheaper copy.
- const exact = await another(3, Path.Pattern.subtree(x));
+ // The first route goes, and a third, as far behind, then lands its copy: what the ended
+ // route's floor certified still holds, so a fresh pick of live/x still passes over the cheaper
+ // copy.
await send({ status: "routeEnd", id: 0n });
- await exact.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await exact.send({ status: "active", suffix: Path.empty(), route: 0n, instance: a });
- await exact.send({ status: "floor", id: 0n, revision: 0n });
- await exact.send({
- status: "routeStart",
- node: NODE_B,
- suffix: Path.empty(),
- offer: offer(NODE_B, { metric: 5n }),
- });
- await exact.send({ status: "active", suffix: Path.empty(), route: 1n, instance: b });
- await exact.send({ status: "floor", id: 1n, revision: 0n });
- expect(await exact.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_B } });
- const second = await trackRequest(x, 4);
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: x, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 3n, instance: a });
+ await send({ status: "floor", id: 3n, revision: 0n });
+ const second = await trackRequest(x, 1);
expect(second.request).toMatchObject({ instance: b, node: NODE_B });
- // The lagging stream catches up and reports nothing more for live/x.
- await live.send({ status: "endedId", id: 0n });
- await live.send({ status: "floor", id: 0n, revision: 1n });
- await live.send({ status: "active", suffix: Path.from("z"), route: 0n, instance: y });
- await live.send({ status: "floor", id: 0n, revision: 2n });
- expect(await live.announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
+ // The lagging route catches up, and nothing more shows for live/x.
+ await send({ status: "endedId", id: 2n });
+ await send({ status: "floor", id: 1n, revision: 1n });
+ await send({ status: "active", suffix: Path.from("live/z"), route: 1n, instance: y });
+ await send({ status: "floor", id: 1n, revision: 2n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/z"), kind: "start" });
first.consumed.close();
second.consumed.close();
- exact.announced.close();
- live.announced.close();
announced.close();
subscriber.close();
await Promise.all([first.info, second.info]);
});
// A floor that certifies nothing withdraws no knowledge: the revision its route's last floor
-// certified still decides, so the lagging stream's withdrawn copy stays held back.
+// certified still decides, so the lagging route's withdrawn copy stays held back.
test("a lite-07 floor that certifies nothing keeps the withdrawal its route certified", async () => {
- const { subscriber, announced, send, trackRequest, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+ const { subscriber, announced, send, trackRequest } = await routedStream();
+ const live = Path.from("live");
const x = Path.from("live/x");
const a = Instance.parse("a");
const b = Instance.parse("b");
- // NODE_A claims live/x cheaply on both streams at revision 0; a dearer NODE_B on the first.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ // NODE_A claims live/x cheaply on both routes at revision 0, and a dearer NODE_B claims it too.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance: a });
- await live.send({ status: "floor", id: 0n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
- await send({ status: "active", suffix: x, route: 1n, instance: b });
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 1 } });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
+ await send({ status: "active", suffix: x, route: 2n, instance: b });
+ await send({ status: "floor", id: 2n, revision: 0n });
- // NODE_A withdraws live/x at revision 1 on the first stream, whose next floor certifies nothing.
+ // NODE_A withdraws live/x at revision 1 on the first route, whose next floor certifies nothing.
await send({ status: "endedId", id: 0n });
await send({ status: "floor", id: 0n, revision: 1n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "restart", route: { node: NODE_B } });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "end", route: { node: NODE_A } });
await send({ status: "floor", id: 0n });
// Another node's claim marks the uncertified floor as applied.
const w = Path.from("live/w");
- await send({ status: "active", suffix: w, route: 1n, instance: b });
- await send({ status: "floor", id: 1n, revision: 1n });
+ await send({ status: "active", suffix: w, route: 2n, instance: b });
+ await send({ status: "floor", id: 2n, revision: 1n });
expect(await announced.next()).toMatchObject({ prefix: w, kind: "start" });
- const pick = await trackRequest(x, 2);
+ const pick = await trackRequest(x, 0);
expect(pick.request).toMatchObject({ instance: b, node: NODE_B });
pick.consumed.close();
- live.announced.close();
announced.close();
subscriber.close();
await pick.info;
});
// A kept floor decides only while a live one certifies the path: once the last certifying route
-// ends, a claim another stream carries uncertified shows again.
+// ends, a claim another route carries uncertified shows again.
test("a lite-07 uncertified claim shows once its last certifier ends", async () => {
- const { subscriber, announced, send, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+ const { subscriber, announced, send } = await routedStream();
+ const live = Path.from("live");
const x = Path.from("live/x");
const y = Path.from("live/y");
const a = Instance.parse("a");
- // The first stream certifies NODE_A's claims at revision 1, without live/x.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ // The first route certifies NODE_A's claims at revision 1, without live/x.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: y, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 1n });
expect(await announced.next()).toMatchObject({ prefix: y, kind: "start" });
// The second carries live/x under a floor that certifies nothing: held back.
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance: a });
- await live.send({ status: "active", suffix: Path.from("y"), route: 0n, instance: a });
- await live.send({ status: "floor", id: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: y, kind: "start" });
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 1n, instance: a });
+ await send({ status: "active", suffix: y, route: 1n, instance: a });
+ await send({ status: "floor", id: 1n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: y, kind: "update", route: { depth: 1 } });
await send({ status: "routeEnd", id: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- live.announced.close();
announced.close();
subscriber.close();
});
@@ -4739,7 +4905,14 @@ test("a lite-07 claim a pruned floor held back shows without a batch", async ()
const w = Path.from("live/w");
const a = Instance.parse("a");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: y, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 1n });
expect(await announced.next()).toMatchObject({ prefix: y, kind: "start" });
@@ -4749,7 +4922,14 @@ test("a lite-07 claim a pruned floor held back shows without a batch", async ()
// A floor that certifies nothing leaves no floor to decide live/x, so it shows.
await send({ status: "floor", id: 0n });
// Another node's claim marks the floors as applied.
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: w, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
@@ -4761,62 +4941,198 @@ test("a lite-07 claim a pruned floor held back shows without a batch", async ()
// A claim above a newer floor's region is older knowledge there: a consume below that region does
// not select it, as Rust's `covering` presents no claim above the newest certifier.
test("a lite-07 consume skips a claim above a newer floor's region", async () => {
- const { subscriber, announced, send, trackRequest, another } = await routedStream();
- const x = await another(1, Path.Pattern.subtree(Path.from("live/x")));
+ const { subscriber, announced, send, trackRequest } = await routedStream();
const root = Path.from("live");
+ const below = Path.from("live/x");
const a = Instance.parse("a");
// NODE_A claims live at revision 0; a dearer NODE_B claims it too.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: root, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: root, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
await send({ status: "active", suffix: root, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
- // NODE_A's revision 1 presents only live/x/m under live/x: it withdrew live.
- await x.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await x.send({ status: "active", suffix: Path.from("m"), route: 0n, instance: a });
- await x.send({ status: "floor", id: 0n, revision: 1n });
- expect(await x.announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "start" });
+ // NODE_A's route at live/x presents only live/x/m there at revision 1: it withdrew live.
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: below, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: Path.from("live/x/m"), route: 2n, instance: a });
+ await send({ status: "floor", id: 2n, revision: 1n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "start" });
- const pick = await trackRequest(Path.from("live/x"), 2);
+ const pick = await trackRequest(below, 0);
expect(pick.request).toMatchObject({ instance: a, node: NODE_B });
pick.consumed.close();
- x.announced.close();
announced.close();
subscriber.close();
await pick.info;
});
+// A DEAD whose far end is gone takes a reader's only claim offline, not withdrawn: the publisher
+// may still hold it, whatever the coverage the route took with it.
+test("a lite-07 DEAD takes a reader's claim offline at its own prefix", async () => {
+ const below = Path.from("live/x");
+ const { subscriber, announced, send } = await routedStream(undefined, Path.Pattern.subtree(below));
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: below, route: 0n, instance: Instance.parse("a") });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: below, kind: "start" });
+
+ const hop = { node: PEER_NODE, session: NODE_A };
+ await send({ status: "dead", dead: { kind: "closed", hop, far: NODE_A, gone: true, moved: 0n } });
+ expect(await announced.next()).toMatchObject({ prefix: below, kind: "offline" });
+
+ announced.close();
+ subscriber.close();
+});
+
+// A DEAD ending several routes through its hop shows each reader its loss once, at the path it
+// presents, whichever route went first: a broader reader's recheck does not withdraw it first.
+test("a lite-07 DEAD takes a narrower reader's claim offline beside a broader reader", async () => {
+ const below = Path.from("live/x");
+ const { subscriber, announced, send, another } = await routedStream();
+ const hop = { node: PEER_NODE, session: NODE_A };
+ // An unrelated route through the same hop goes first.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { path: [{ node: NODE_B, session: 0n }, { node: NODE_A, session: 1n }, hop] }),
+ });
+ await send({ status: "active", suffix: Path.from("other"), route: 0n, instance: Instance.parse("b") });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: below, route: 1n, instance: Instance.parse("a") });
+ await send({ status: "floor", id: 1n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("other"), kind: "start" });
+ expect(await announced.next()).toMatchObject({ prefix: below, kind: "start" });
+ const { announced: under } = await another(Path.Pattern.subtree(below));
+ expect(await under.next()).toMatchObject({ prefix: below, kind: "start" });
+
+ await send({ status: "dead", dead: { kind: "closed", hop, far: NODE_A, gone: true, moved: 0n } });
+ expect(await under.next()).toMatchObject({ prefix: below, kind: "offline" });
+
+ under.close();
+ announced.close();
+ subscriber.close();
+});
+
+// A reader below a newer floor's region is not shown a claim above it, as a consume there skips
+// it: the covering claim it sees at its own prefix is the one that floor leaves current.
+test("a lite-07 reader skips a claim above a newer floor's region", async () => {
+ const { subscriber, announced, send, another } = await routedStream();
+ const root = Path.from("live");
+ const below = Path.from("live/x");
+ const a = Instance.parse("a");
+
+ // NODE_A claims live at revision 0; a dearer NODE_B claims it too.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: root, route: 0n, instance: a });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: root, kind: "start", route: { node: NODE_A } });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
+ await send({ status: "active", suffix: root, route: 1n, instance: a });
+ await send({ status: "floor", id: 1n, revision: 0n });
+ // NODE_A's route at live/x presents only live/x/m there at revision 1: it withdrew live.
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: below, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: Path.from("live/x/m"), route: 2n, instance: a });
+ await send({ status: "floor", id: 2n, revision: 1n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "start" });
+
+ const { announced: under } = await another(Path.Pattern.subtree(below));
+ const shown = [await under.next(), await under.next()];
+ expect(shown).toContainEqual(
+ expect.objectContaining({ prefix: below, route: expect.objectContaining({ node: NODE_B }) }),
+ );
+ expect(shown).not.toContainEqual(
+ expect.objectContaining({ prefix: below, route: expect.objectContaining({ node: NODE_A }) }),
+ );
+
+ under.close();
+ announced.close();
+ subscriber.close();
+});
+
// A route refused once live stops covering, as an ended one does: a held consume a narrower floor
// of it took off a broader claim is routable again once that floor is gone.
test("a lite-07 route refused once live gives a held consume back its broader claim", async () => {
- const { subscriber, announced, send, trackRequest, another, opened } = await routedStream();
+ const { subscriber, announced, send, raw, trackRequest, opened } = await routedStream();
const live = Path.from("live");
+ const below = Path.from("live/x");
const y = Path.from("live/x/y");
const minted = Mint.fresh();
// NODE_A claims live under a floor that certifies nothing, and a consume of live/x/y takes it.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: live, route: 0n, instance: minted });
await send({ status: "floor", id: 0n });
expect(await announced.next()).toMatchObject({ prefix: live, kind: "start" });
- const x = await another(1, Path.Pattern.subtree(Path.from("live/x")));
- const held = await trackRequest(y, 2);
+ const held = await trackRequest(y, 0);
expect(held.request).toMatchObject({ node: NODE_A });
- // NODE_A's revision 1 presents only live/x/m under live/x, so live no longer covers live/x/y.
- await x.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await x.send({ status: "active", suffix: Path.from("m"), route: 0n, instance: minted });
- await x.send({ status: "floor", id: 0n, revision: 1n });
- expect(await x.announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "start" });
+ // NODE_A's route at live/x presents only live/x/m there at revision 1, so live no longer
+ // covers live/x/y.
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: below, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: Path.from("live/x/m"), route: 1n, instance: minted });
+ await send({ status: "floor", id: 1n, revision: 1n });
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "start" });
await held.info;
// That route is refused: nothing certifies live/x/y any more, so live covers it again.
- x.raw([7, 8, 0, 1, 5, 0, 0x80, 128, 0, 42]);
- expect(await x.announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "offline" });
+ await raw([7, 8, 1, 1, 5, 0, 0x80, 128, 0, 42]);
+ expect(await announced.next()).toMatchObject({ prefix: Path.from("live/x/m"), kind: "offline" });
const info = held.consumed
.track("audio")
.info()
@@ -4824,10 +5140,9 @@ test("a lite-07 route refused once live gives a held consume back its broader cl
() => "answered",
() => "refused",
);
- expect(await Promise.race([opened(3).then(() => "opened"), info])).toBe("opened");
+ expect(await Promise.race([opened(1).then(() => "opened"), info])).toBe("opened");
held.consumed.close();
- x.announced.close();
announced.close();
subscriber.close();
await info;
@@ -4840,11 +5155,25 @@ test("a lite-07 tie keeps its winner across a route's unusable spell", async ()
const x = Path.from("live/x");
const a = Instance.parse("a");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await send({ status: "active", suffix: x, route: 0n, instance: a });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: x, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: x, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { node: NODE_B } });
@@ -4862,7 +5191,7 @@ test("a lite-07 tie keeps its winner across a route's unusable spell", async ()
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: w, kind: "start" });
- const pick = await trackRequest(x, 1);
+ const pick = await trackRequest(x, 0);
expect(pick.request).toMatchObject({ instance: a, node: NODE_B });
pick.consumed.close();
@@ -4879,12 +5208,26 @@ test("a lite-07 consume pinned to a minted instance asks for it", async () => {
const a = Instance.parse("a");
const minted = Mint.fresh();
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
// A dearer route carries the minted instance.
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
await send({ status: "active", suffix: x, route: 1n, instance: minted });
await send({ status: "floor", id: 1n, revision: 0n });
// Another claim marks the floor as applied.
@@ -4893,7 +5236,7 @@ test("a lite-07 consume pinned to a minted instance asks for it", async () => {
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: w, kind: "start" });
- const pick = await trackRequest(x, 1, minted);
+ const pick = await trackRequest(x, 0, minted);
expect(pick.request).toMatchObject({ instance: minted, node: NODE_B });
pick.consumed.close();
@@ -4902,38 +5245,55 @@ test("a lite-07 consume pinned to a minted instance asks for it", async () => {
await pick.info;
});
-// A lagging stream's copy of a claim takes the cost the newest certifying floor gives it, so a
+// A lagging route's copy of a claim takes the cost the newest certifying floor gives it, so a
// re-priced origin does not keep winning on the cost it had.
test("a lite-07 lagging copy carries the cost a newer floor certified", async () => {
- const { subscriber, announced, send, trackRequest, another } = await routedStream();
- const live = await another(1, Path.Pattern.subtree(Path.from("live")));
+ const { subscriber, announced, send, trackRequest } = await routedStream({ claims: true });
+ const live = Path.from("live");
const x = Path.from("live/x");
const a = Instance.parse("a");
const b = Instance.parse("b");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance: a });
await send({ status: "floor", id: 0n, revision: 0n });
- expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
- await live.send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
- await live.send({ status: "active", suffix: Path.from("x"), route: 0n, instance: a });
- await live.send({ status: "floor", id: 0n, revision: 0n });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { cost: 1n } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
- await send({ status: "active", suffix: x, route: 1n, instance: b });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A, cost: 1n } });
+ await send({ status: "routeStart", prefix: 0n, node: NODE_A, suffix: live, exact: false, offer: offer(NODE_A) });
+ await send({ status: "active", suffix: x, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
+ // The more specific route to the node now carries the claim, which a request names by its depth.
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 1, node: NODE_A } });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
+ await send({ status: "active", suffix: x, route: 2n, instance: b });
+ await send({ status: "floor", id: 2n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_B } });
- // NODE_A raises its cost to 100 at revision 1, which only the first stream has delivered.
+ // NODE_A raises its cost to 100 at revision 1, which only the first route has delivered: its
+ // claim is reported at that cost, whichever route carries it.
await send({ status: "update", id: 0n, cost: 100n });
await send({ status: "floor", id: 0n, revision: 1n });
- expect(await announced.next()).toMatchObject({ prefix: x, kind: "restart", route: { node: NODE_B } });
- expect(await live.announced.next()).toMatchObject({ prefix: x, kind: "update", route: { cost: 101n } });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "update", route: { node: NODE_A, cost: 101n } });
+ // The first route goes, and the lagging copy still carries the newer cost.
+ await send({ status: "routeEnd", id: 0n });
- const pick = await trackRequest(x, 2);
+ const pick = await trackRequest(x, 0);
expect(pick.request).toMatchObject({ instance: b, node: NODE_B });
pick.consumed.close();
- live.announced.close();
announced.close();
subscriber.close();
await pick.info;
@@ -4945,21 +5305,35 @@ test("a lite-07 consume names the node now serving its instance", async () => {
const a = Instance.parse("a");
// Two origin nodes reach one instance; the cheaper serves.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A, { metric: 5n }),
+ });
await send({ status: "active", suffix: room, route: 0n, instance: a });
await send({ status: "floor", id: 0n });
await announced.next();
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: room, route: 1n, instance: a });
await send({ status: "floor", id: 1n });
await announced.next();
- const first = await trackRequest(room, 1);
+ const first = await trackRequest(room, 0);
expect(first.request).toMatchObject({ instance: a, node: NODE_B });
// The cheaper one goes: the same instance's consume now names the other.
await send({ status: "routeEnd", id: 1n });
await announced.next();
- const second = await trackRequest(room, 2);
+ const second = await trackRequest(room, 1);
expect(second.consumed.closed).toBe(first.consumed.closed);
expect(second.request).toMatchObject({ instance: a, node: NODE_A });
@@ -4980,13 +5354,27 @@ test("a lite-07 certified route ending shows another route's copy of its claim",
};
// A narrow route's floor certifies nothing, so it shows x at b while it carries it.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.from("live"), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.from("live"),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance: Instance.parse("b") });
await send({ status: "floor", id: 0n });
expect(await next()).toBe("start live/x b");
// A wide route's floor certifies x at a, which then decides it.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 1n, instance: Instance.parse("a") });
await send({ status: "floor", id: 1n, revision: 1n });
let seen = await next();
@@ -5004,13 +5392,27 @@ test("a lite-07 path ranks only an instance's newest epoch", async () => {
const { subscriber, announced, send } = await routedStream();
const room = Path.from("room");
const a = Instance.parse("a");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: room, route: 0n, instance: a, epoch: 0n });
await send({ status: "floor", id: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, route: { epoch: 0n } });
// Dearer, but newer: it takes over.
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 9n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 9n }),
+ });
await send({ status: "active", suffix: room, route: 1n, instance: a, epoch: 1n });
await send({ status: "floor", id: 1n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "restart", route: { epoch: 1n, node: NODE_B } });
@@ -5029,7 +5431,14 @@ test("lite-07 children of a prefix two origins claim spread by node", async () =
const { subscriber, announced, send, trackRequest } = await routedStream();
const pool = Path.from("pool");
for (const [route, node] of order.entries()) {
- await send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(node),
+ });
await send({ status: "active", suffix: pool, route: BigInt(route), instance: Instance.parse(`i${node}`) });
await send({ status: "floor", id: BigInt(route) });
await announced.next();
@@ -5038,7 +5447,7 @@ test("lite-07 children of a prefix two origins claim spread by node", async () =
// Pinned to `lite07_ties_match_js` in rs/moq-net, which picks NODE_B too.
expect(rendezvousHash(job, NODE_A)).toBe(0xc376be7bf47aebbfn);
expect(rendezvousHash(job, NODE_B)).toBe(0x5813213b6508aaeen);
- const { request, consumed, info } = await trackRequest(job, 1);
+ const { request, consumed, info } = await trackRequest(job, 0);
expect(request.node).toBe(NODE_B);
consumed.close();
subscriber.close();
@@ -5057,11 +5466,25 @@ test("a lite-07 tie at a claim's path goes to the newest claim, a re-price keepi
const b = Instance.parse("b");
const c = Instance.parse("c");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: room, route: 0n, instance: a, cost: 4n });
await send({ status: "floor", id: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: room, route: 1n, instance: b });
await send({ status: "floor", id: 1n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "restart", route: { node: NODE_B } });
@@ -5072,7 +5495,7 @@ test("a lite-07 tie at a claim's path goes to the newest claim, a re-price keepi
await send({ status: "active", suffix: other, route: 1n, instance: b });
await send({ status: "floor", id: 1n });
expect(await announced.next()).toMatchObject({ prefix: other, kind: "start" });
- const kept = await trackRequest(room, 1);
+ const kept = await trackRequest(room, 0);
expect(kept.request).toMatchObject({ instance: b, node: NODE_B });
// A restart at the tie is the newest claim: both pick it.
@@ -5083,7 +5506,7 @@ test("a lite-07 tie at a claim's path goes to the newest claim, a re-price keepi
kind: "restart",
route: { instance: c, node: NODE_A },
});
- const restarted = await trackRequest(room, 2);
+ const restarted = await trackRequest(room, 1);
expect(restarted.request).toMatchObject({ instance: c, node: NODE_A });
kept.consumed.close();
@@ -5098,16 +5521,30 @@ test("a lite-07 consume pinned to an instance names its route's node", async ()
const { subscriber, announced, send, trackRequest } = await routedStream();
const pool = Path.from("pool");
const a = Instance.parse("a");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: pool, route: 0n, instance: a });
await send({ status: "floor", id: 0n });
await announced.next();
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: pool, route: 1n, instance: Instance.parse("b") });
await send({ status: "floor", id: 1n });
await announced.next();
// NODE_B wins `pool/job` unpinned; see the spread test above.
- const { request, consumed, info } = await trackRequest(Path.from("pool/job"), 1, a);
+ const { request, consumed, info } = await trackRequest(Path.from("pool/job"), 0, a);
expect(request).toMatchObject({ instance: a, node: NODE_A });
consumed.close();
announced.close();
@@ -5119,11 +5556,25 @@ test("a lite-07 consume pinned to an instance names its route's node", async ()
test("a lite-07 stream reports each node's claim with claims", async () => {
const { subscriber, announced, send } = await routedStream({ claims: true });
const pool = Path.from("pool");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: pool, route: 0n, instance: Instance.parse("a") });
await send({ status: "floor", id: 0n });
expect(await announced.next()).toMatchObject({ prefix: pool, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: pool, route: 1n, instance: Instance.parse("b") });
await send({ status: "floor", id: 1n });
expect(await announced.next()).toMatchObject({ prefix: pool, kind: "start", route: { node: NODE_B } });
@@ -5146,13 +5597,27 @@ test("a lite-07 batch applies at its floor, and a held route turns unavailable",
const a = Instance.parse("a");
// Nothing under a route shows before its floor, even while another route's batch lands.
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: room, route: 0n, instance: a, cost: 2n });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: other, route: 1n, instance: a });
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: other, kind: "start" });
- await send({ status: "synced" });
+ await send({ status: "synced", prefix: 0n });
await send({ status: "floor", id: 0n, revision: 0n });
// The claim's cost, the route's metric, and this session's link.
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { cost: 3n, node: NODE_A } });
@@ -5199,13 +5664,27 @@ test("a lite-07 path served by two routes fails over when the cheaper one ends",
const a = Instance.parse("a");
const b = Instance.parse("b");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A, { metric: 5n }),
+ });
await send({ status: "active", suffix: room, route: 0n, instance: a });
await send({ status: "floor", id: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { instance: a, node: NODE_A } });
// Another origin claims the path more cheaply: it takes over, as another instance.
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B),
+ });
await send({ status: "active", suffix: room, route: 1n, instance: b });
await send({ status: "floor", id: 1n });
expect(await announced.next()).toMatchObject({
@@ -5242,21 +5721,28 @@ test("a lite-07 DEAD moves, ends, or holds the routes through its hop", async ()
[1n, NODE_B, b],
[2n, NODE_C, c],
] as const) {
- await send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node) });
+ await send({ status: "routeStart", prefix: 0n, node, suffix: Path.empty(), exact: false, offer: offer(node) });
await send({ status: "active", suffix: path, route: id, instance });
await send({ status: "floor", id, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: path, kind: "start", route: { cost: 1n } });
}
- // The peer's session to a publisher, as each route's path names it.
- const closed = (node: bigint, gone: boolean, moved: { id: bigint; offer: Offer }[] = []): AnnounceBroadcast => ({
- status: "dead",
- dead: { kind: "closed", hop: { node: PEER_NODE, session: node }, far: node, gone, more: false, moved },
- });
+ // The peer's session to a publisher, as each route's path names it, closed: the DEAD, then
+ // the ROUTE_UPDATE of each route it moved.
+ const closed = async (
+ to: (msg: AnnounceBroadcast) => Promise,
+ node: bigint,
+ gone: boolean,
+ moved: { id: bigint; offer: Offer }[] = [],
+ ) => {
+ const hop = { node: PEER_NODE, session: node };
+ await to({ status: "dead", dead: { kind: "closed", hop, far: node, gone, moved: BigInt(moved.length) } });
+ for (const { id, offer } of moved) await to({ status: "routeUpdate", id, offer });
+ };
// A hop no route crosses changes nothing: the next thing shown is the move below.
await send({
status: "dead",
- dead: { kind: "closed", hop: { node: 0x55n, session: 4n }, far: 0x56n, gone: true, more: false, moved: [] },
+ dead: { kind: "closed", hop: { node: 0x55n, session: 4n }, far: 0x56n, gone: true, moved: 0n },
});
// Route 0 moved to a costlier path through another of the peer's sessions.
const detour = [
@@ -5264,12 +5750,12 @@ test("a lite-07 DEAD moves, ends, or holds the routes through its hop", async ()
{ node: 3n, session: 1n },
{ node: PEER_NODE, session: 3n },
];
- await send(closed(NODE_A, false, [{ id: 0n, offer: offer(NODE_A, { metric: 4n, path: detour }) }]));
+ await closed(send, NODE_A, false, [{ id: 0n, offer: offer(NODE_A, { metric: 4n, path: detour }) }]);
expect(await announced.next()).toMatchObject({ prefix: a, kind: "update", route: { cost: 5n, available: true } });
// Its old hop names it no more: a DEAD of that hop leaves it alone.
- await send(closed(NODE_A, true));
+ await closed(send, NODE_A, true);
// Route 1's far end is gone: it ends with its claims.
- await send(closed(NODE_B, true));
+ await closed(send, NODE_B, true);
expect(await announced.next()).toMatchObject({ prefix: b, kind: "offline" });
await send({ status: "endedId", id: 1n });
await expect(announced.next()).rejects.toThrow(ProtocolViolation);
@@ -5277,17 +5763,26 @@ test("a lite-07 DEAD moves, ends, or holds the routes through its hop", async ()
// Held, a route stays unavailable until its next offer.
const held = await routedStream();
- await held.send({ status: "routeStart", node: NODE_C, suffix: Path.empty(), offer: offer(NODE_C) });
+ await held.send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_C,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_C),
+ });
await held.send({ status: "active", suffix: c, route: 0n, instance });
await held.send({ status: "floor", id: 0n, revision: 0n });
expect(await held.announced.next()).toMatchObject({ prefix: c, kind: "start" });
- await held.send(closed(NODE_C, false));
+ await closed(held.send, NODE_C, false);
expect(await held.announced.next()).toMatchObject({ prefix: c, kind: "update", route: { available: false } });
await held.send({ status: "routeUpdate", id: 0n, offer: offer(NODE_C) });
expect(await held.announced.next()).toMatchObject({ prefix: c, kind: "update", route: { available: true } });
// Moved to an offer this node refuses, it goes offline, and a later usable offer brings it back.
- // DEAD: the session NODE_C (two varint bytes) closed, route 0 moved to an offer with an unknown flag.
- await held.raw([15, 17, 0, 2, 0x80, 0x9c, 0x80, 0x9c, 0, 0, 1, 0, 6, 1, 7, 0, 0x80, 128, 0]);
+ // DEAD: the session NODE_C (two varint bytes) closed, moving one route, then the ROUTE_UPDATE
+ // of route 0 to an offer with an unknown flag.
+ await held.raw([15, 8, 0, 2, 0x80, 0x9c, 0x80, 0x9c, 0, 1]);
+ await held.raw([7, 7, 0, 1, 7, 0, 0x80, 128, 0]);
expect(await held.announced.next()).toMatchObject({ prefix: c, kind: "offline" });
await held.send({ status: "routeUpdate", id: 0n, offer: offer(NODE_C) });
expect(await held.announced.next()).toMatchObject({ prefix: c, kind: "start" });
@@ -5295,42 +5790,327 @@ test("a lite-07 DEAD moves, ends, or holds the routes through its hop", async ()
held.subscriber.close();
});
-// A DEAD in parts applies with its last: a route an earlier part moved is not ended by the gone
-// hop, and nothing shows before the last part. Mirrors rs/moq-net.
-test("a lite-07 DEAD in parts applies with its last", async () => {
- const { subscriber, announced, send } = await routedStream();
- const [a, b] = ["a", "b"].map((path) => Path.from(path));
+// A DEAD and the ROUTE_UPDATEs it counts apply as one step: nothing shows before the last of
+// them, a route one of them moved is not ended by the gone hop, and the route none of them
+// names ends with it. Mirrors rs/moq-net.
+test("a lite-07 DEAD and its ROUTE_UPDATEs apply as one", async () => {
+ const { subscriber, announced, send, raw, settled } = await routedStream();
+ const [a, b, c] = ["a", "b", "c"].map((path) => Path.from(path));
const instance = Instance.parse("i");
- // Both routes reach the peer over its session to NODE_A.
+ // One varint byte, so the raw message below spells it.
+ const NODE_C = 0x9n;
+ // Every route reaches the peer over its session to NODE_A.
const hop = { node: PEER_NODE, session: NODE_A };
const through = (node: bigint) => [
{ node, session: 0n },
...(node === NODE_A ? [] : [{ node: NODE_A, session: 1n }]),
hop,
];
+ const NODE_D = 0x9dn;
+ const d = Path.from("d");
for (const [id, node, path] of [
[0n, NODE_A, a],
[1n, NODE_B, b],
+ [2n, NODE_C, c],
+ [3n, NODE_D, d],
] as const) {
- await send({ status: "routeStart", node, suffix: Path.empty(), offer: offer(node, { path: through(node) }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(node, { path: through(node) }),
+ });
await send({ status: "active", suffix: path, route: id, instance });
await send({ status: "floor", id, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: path, kind: "start" });
}
- const detour = (node: bigint) => ({
- id: node === NODE_A ? 0n : 1n,
- offer: offer(node, { metric: 4n, path: [...through(node).slice(0, -1), { node: PEER_NODE, session: 9n }] }),
+ const detour = (node: bigint) =>
+ offer(node, { metric: 4n, path: [...through(node).slice(0, -1), { node: PEER_NODE, session: 9n }] });
+
+ // Whatever shows before the last ROUTE_UPDATE is one applied alone.
+ const shown: unknown[] = [];
+ const reading = (async () => {
+ for (;;) {
+ const event = await announced.next();
+ if (!event) break;
+ shown.push(event);
+ }
+ })();
+ await send({ status: "dead", dead: { kind: "closed", hop, far: NODE_A, gone: true, moved: 3n } });
+ await settled();
+ expect(shown).toEqual([]);
+ await send({ status: "routeUpdate", id: 0n, offer: detour(NODE_A) });
+ await settled();
+ expect(shown).toEqual([]);
+ // ROUTE_UPDATE of Route ID 2 with a flag no version defines: refused, and counted all the same.
+ await raw([7, 9, 2, 2, 9, 0, 2, 9, 0x80, 128, 0]);
+ await settled();
+ expect(shown).toEqual([]);
+ await send({ status: "routeUpdate", id: 1n, offer: detour(NODE_B) });
+ await settled();
+ // The moved routes carry on along their new paths, the one moved to a refused offer goes
+ // offline, and the one no update named ends with the gone hop.
+ expect(shown).toMatchObject([
+ { prefix: a, kind: "update", route: { cost: 5n } },
+ { prefix: b, kind: "update", route: { cost: 5n } },
+ { prefix: c, kind: "offline" },
+ { prefix: d, kind: "offline" },
+ ]);
+ // The ended route's ids went with it, and the moved ones are live.
+ await send({ status: "routeHold", id: 0n });
+ await settled();
+ expect(shown.at(-1)).toMatchObject({ prefix: a, kind: "update", route: { available: false } });
+
+ announced.close();
+ await reading;
+ subscriber.close();
+});
+
+// A reader sees only where a DEAD leaves every route it moves, never a route only the middle of
+// it preferred: the publisher wins before and after, so its fallback never shows.
+test("a lite-07 DEAD shows readers only its end", async () => {
+ const { subscriber, announced, send, settled } = await routedStream();
+ const x = Path.from("live/x");
+ const [a, b] = [Instance.parse("a"), Instance.parse("b")];
+ // Two routes to NODE_A over the peer's session to it, the second held, and NODE_B dearer.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
});
- const part = (more: boolean, moved: { id: bigint; offer: Offer }[]): AnnounceBroadcast => ({
- status: "dead",
- dead: { kind: "closed", hop, far: NODE_A, gone: true, more, moved },
+ await send({ status: "active", suffix: x, route: 0n, instance: a });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { node: NODE_A } });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.from("live"),
+ exact: false,
+ offer: offer(NODE_A),
});
- await send(part(true, [detour(NODE_A)]));
- await send(part(false, [detour(NODE_B)]));
- for (const path of [a, b]) {
- expect(await announced.next()).toMatchObject({ prefix: path, kind: "update", route: { cost: 5n } });
+ await send({ status: "active", suffix: x, route: 1n, instance: a });
+ await send({ status: "floor", id: 1n, revision: 0n });
+ await send({ status: "routeHold", id: 1n });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 9n }),
+ });
+ await send({ status: "active", suffix: x, route: 2n, instance: b });
+ await send({ status: "floor", id: 2n, revision: 0n });
+ await settled();
+
+ const shown: { route: { node?: bigint } }[] = [];
+ const reading = (async () => {
+ for (;;) {
+ const event = await announced.next();
+ if (!event) break;
+ shown.push(event);
+ }
+ })();
+ // The session to NODE_A closed for good: the first route ends, and the held one moves to a
+ // usable path.
+ const detour = [
+ { node: NODE_A, session: 0n },
+ { node: 3n, session: 1n },
+ { node: PEER_NODE, session: 3n },
+ ];
+ const hop = { node: PEER_NODE, session: NODE_A };
+ await send({ status: "dead", dead: { kind: "closed", hop, far: NODE_A, gone: true, moved: 1n } });
+ await send({ status: "routeUpdate", id: 1n, offer: offer(NODE_A, { metric: 4n, path: detour }) });
+ await settled();
+ expect(shown.length).toBeGreaterThan(0);
+ for (const event of shown) expect(event).toMatchObject({ prefix: x, route: { node: NODE_A } });
+
+ announced.close();
+ await reading;
+ subscriber.close();
+});
+
+// A reader of a wider prefix widens the route in place: the claim it carried stays shown to
+// both readers, and only what the wider prefix adds arrives.
+test("a lite-07 edit to a wider prefix keeps the claims the route carried", async () => {
+ const half = Path.from("a");
+ const { subscriber, announced, send, edits, settled, another } = await routedStream(
+ undefined,
+ Path.Pattern.subtree(half),
+ );
+ const [x, y] = [Path.from("a/x"), Path.from("b/y")];
+ const instance = Instance.parse("i");
+ expect(edits).toMatchObject([{ remove: [], add: [{ prefix: half }] }]);
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: x, route: 0n, instance });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ await send({ status: "synced", prefix: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start" });
+
+ // The root takes the place of the narrower prefix in one edit, which the harness answers.
+ const wide = await another();
+ expect(edits).toMatchObject([{}, { remove: [0n], add: [{ prefix: Path.empty() }] }]);
+ expect(await wide.announced.next()).toMatchObject({ prefix: x, kind: "start" });
+ // The route now lies at the root, which a request through the claim names by its depth.
+ for (const reader of [announced, wide.announced]) {
+ expect(await reader.next()).toMatchObject({ prefix: x, kind: "update", route: { depth: 0, available: true } });
}
+ // The publisher sends what the widened route lacks, then its floor.
+ await send({ status: "active", suffix: y, route: 0n, instance });
+ await send({ status: "floor", id: 0n, revision: 1n });
+ await send({ status: "synced", prefix: 1n });
+ expect(await wide.announced.next()).toMatchObject({ prefix: y, kind: "start" });
+
+ // The claim both prefixes cover never left: its end is the next thing either reader sees.
+ await send({ status: "endedId", id: 0n });
+ await send({ status: "floor", id: 0n, revision: 2n });
+ await settled();
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "end" });
+ expect(await wide.announced.next()).toMatchObject({ prefix: x, kind: "end" });
+
+ wide.announced.close();
+ announced.close();
+ subscriber.close();
+});
+
+// A route its ROUTE_START marked exact lies at its own prefix: a wider request leaves it there, so
+// its claims keep their route's depth and the publisher keeps sending under it. Mirrors rs/moq-net.
+test("a lite-07 edit to a wider prefix leaves an exact route at its own prefix", async () => {
+ const half = Path.from("a");
+ const { subscriber, announced, send, edits, settled, another } = await routedStream(
+ undefined,
+ Path.Pattern.subtree(half),
+ );
+ const [x, z] = [Path.from("a/x"), Path.from("a/z")];
+ const instance = Instance.parse("i");
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: true,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: x, route: 0n, instance });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ await send({ status: "synced", prefix: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: x, kind: "start", route: { depth: 1 } });
+
+ const wide = await another();
+ expect(edits).toMatchObject([{}, { remove: [0n], add: [{ prefix: Path.empty() }] }]);
+ expect(await wide.announced.next()).toMatchObject({ prefix: x, kind: "start", route: { depth: 1 } });
+ await send({ status: "synced", prefix: 1n });
+
+ // Nothing moved, so the first reader's next event is the new claim under the same route.
+ await send({ status: "active", suffix: z, route: 0n, instance });
+ await send({ status: "floor", id: 0n, revision: 1n });
+ await settled();
+ expect(await announced.next()).toMatchObject({ prefix: z, kind: "start", route: { depth: 1 } });
+ expect(await wide.announced.next()).toMatchObject({ prefix: z, kind: "start", route: { depth: 1 } });
+
+ wide.announced.close();
+ announced.close();
+ subscriber.close();
+});
+
+// A reader below the shared request is shown a broader claim at its own prefix as that much above
+// it, so a claim exactly there outranks the broader one however cheaper that is, as a consume of
+// the path picks.
+test("a lite-07 reader below the request ranks a broader claim as above its prefix", async () => {
+ const { subscriber, announced, send, edits, settled, another } = await routedStream();
+ const [x, xa] = [Path.from("x"), Path.from("x/a")];
+ const narrow = await another(Path.Pattern.subtree(xa));
+ // The root covers the narrow reader, so its request is the root's alone.
+ expect(edits).toHaveLength(1);
+ const instance = Instance.parse("i");
+ for (const [id, node, path, cost] of [
+ [0n, NODE_A, x, 0n],
+ [1n, NODE_B, xa, 10n],
+ ] as const) {
+ await send({ status: "routeStart", prefix: 0n, node, suffix: Path.empty(), exact: false, offer: offer(node) });
+ await send({ status: "active", suffix: path, route: id, instance, cost });
+ await send({ status: "floor", id, revision: 0n });
+ }
+ await settled();
+ // The root's claim shows first, one segment above the reader's prefix, until the exact one
+ // arrives.
+ expect(await narrow.announced.next()).toMatchObject({
+ prefix: xa,
+ kind: "start",
+ route: { node: NODE_A, above: 1n },
+ });
+ expect(await narrow.announced.next()).toMatchObject({ prefix: xa, route: { node: NODE_B, above: 0n } });
+
+ narrow.announced.close();
+ announced.close();
+ subscriber.close();
+});
+
+// Two routes of one node, each showing its covering claim at its own prefix, merge when a reader
+// widens the request past both: each claim stays shown until the merged route's next floor.
+test("a lite-07 edit merging two routes keeps each one's covering claim", async () => {
+ const [a, b] = [Path.from("a"), Path.from("b")];
+ const { subscriber, announced, send, edits, settled, another } = await routedStream(
+ undefined,
+ Path.Pattern.subtree(a),
+ );
+ const instance = Instance.parse("i");
+ const other = await another(Path.Pattern.subtree(b));
+ expect(edits).toMatchObject([{ add: [{ prefix: a }] }, { add: [{ prefix: b }] }]);
+ // NODE_A claims the root, which each route presents at its requested prefix.
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: a, route: 0n, instance });
+ await send({ status: "floor", id: 0n, revision: 0n });
+ await send({
+ status: "routeStart",
+ prefix: 1n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({ status: "active", suffix: b, route: 1n, instance });
+ await send({ status: "floor", id: 1n, revision: 0n });
+ expect(await announced.next()).toMatchObject({ prefix: a, kind: "start" });
+ expect(await other.announced.next()).toMatchObject({ prefix: b, kind: "start" });
+
+ // A reader of everything widens the request to the root, which the harness answers.
+ const wide = await another();
+ await settled();
+ expect(edits.at(-1)).toMatchObject({ remove: [0n, 1n], add: [{ prefix: Path.empty() }] });
+ const seen: { kind: string }[] = [];
+ const reading = (async () => {
+ for (const reader of [announced, other.announced]) {
+ const event = await Promise.race([reader.next(), settled().then(() => undefined)]);
+ if (event) seen.push(event);
+ }
+ })();
+ await reading;
+ expect(seen.filter((event) => event.kind === "end")).toEqual([]);
+
+ wide.announced.close();
+ other.announced.close();
announced.close();
subscriber.close();
});
@@ -5341,11 +6121,25 @@ test("a lite-07 DRAIN ranks the routes through its node last", async () => {
const { subscriber, announced, send } = await routedStream();
const room = Path.from("room");
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: room, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
await send({ status: "active", suffix: room, route: 1n, instance });
await send({ status: "floor", id: 1n, revision: 0n });
@@ -5367,14 +6161,23 @@ test("a lite-07 claim ranks by its path's priority before its cost", async () =>
const instance = Instance.parse("i");
await send({
status: "routeStart",
+ prefix: 0n,
node: NODE_A,
suffix: Path.empty(),
+ exact: false,
offer: offer(NODE_A, { priority: -10 }),
});
await send({ status: "active", suffix: room, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "start", route: { node: NODE_A } });
- await send({ status: "routeStart", node: NODE_B, suffix: Path.empty(), offer: offer(NODE_B, { metric: 5n }) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_B,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_B, { metric: 5n }),
+ });
await send({ status: "active", suffix: room, route: 1n, instance });
await send({ status: "floor", id: 1n, revision: 0n });
expect(await announced.next()).toMatchObject({ prefix: room, kind: "update", route: { node: NODE_B } });
@@ -5390,8 +6193,22 @@ test("a lite-07 request names its route's prefix depth", async () => {
const x = Path.from("live/room/x");
const y = Path.from("y");
const instance = Instance.parse("i");
- await send({ status: "routeStart", node: NODE_A, suffix: Path.from("live/room"), offer: offer(NODE_A) });
- await send({ status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.from("live/room"),
+ exact: false,
+ offer: offer(NODE_A),
+ });
+ await send({
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ });
await send({ status: "active", suffix: x, route: 0n, instance });
await send({ status: "floor", id: 0n, revision: 0n });
await send({ status: "active", suffix: y, route: 1n, instance });
@@ -5399,9 +6216,9 @@ test("a lite-07 request names its route's prefix depth", async () => {
expect(await announced.next()).toMatchObject({ prefix: x, kind: "start" });
expect(await announced.next()).toMatchObject({ prefix: y, kind: "start" });
- const deep = await trackRequest(x, 1);
+ const deep = await trackRequest(x, 0);
expect(deep.request).toMatchObject({ node: NODE_A, depth: 2 });
- const shallow = await trackRequest(y, 2);
+ const shallow = await trackRequest(y, 1);
expect(shallow.request).toMatchObject({ node: NODE_A, depth: 0 });
for (const pick of [deep, shallow]) pick.consumed.close();
@@ -5410,86 +6227,107 @@ test("a lite-07 request names its route's prefix depth", async () => {
for (const pick of [deep, shallow]) await pick.info;
});
+// The routes a DEAD violation is checked against: NODE_A's and NODE_B's, each over the peer's
+// session to its node, and NODE_C's behind NODE_A's.
+const DEAD_HOP = { node: PEER_NODE, session: NODE_A };
+const DEAD_ROUTES: AnnounceBroadcast[] = [
+ { status: "routeStart", prefix: 0n, node: NODE_A, suffix: Path.empty(), exact: false, offer: offer(NODE_A) },
+ { status: "routeStart", prefix: 0n, node: NODE_B, suffix: Path.empty(), exact: false, offer: offer(NODE_B) },
+ {
+ status: "routeStart",
+ prefix: 0n,
+ node: 0x9cn,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(0x9cn, { path: [{ node: 0x9cn, session: 0n }, { node: NODE_A, session: 1n }, DEAD_HOP] }),
+ },
+];
+const deadMoving = (moved: bigint): AnnounceBroadcast => ({
+ status: "dead",
+ dead: { kind: "closed", hop: DEAD_HOP, far: NODE_A, gone: false, moved },
+});
+
+// Each closes the session: a violation on the peer's Announce Stream is the peer breaking it.
test.each([
[
"a second route to one origin node at one prefix",
[
- { status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) },
- { status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) },
+ {
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ },
+ {
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ },
],
],
[
- "a claim outside its route's prefix",
- [
- { status: "routeStart", node: NODE_A, suffix: Path.from("live"), offer: offer(NODE_A) },
- { status: "active", suffix: Path.from("other"), route: 0n, instance: Instance.parse("a") },
- ],
+ "a route under a Prefix ID never asked for",
+ [{ status: "routeStart", prefix: 1n, node: NODE_A, suffix: Path.empty(), exact: false, offer: offer(NODE_A) }],
],
[
- "a DEAD moving an unknown route",
+ "a claim outside its route's prefix",
[
{
- status: "dead",
- dead: {
- kind: "closed",
- hop: { node: PEER_NODE, session: 1n },
- far: 3n,
- gone: false,
- more: false,
- moved: [{ id: 0n, offer: offer(NODE_A) }],
- },
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.from("live"),
+ exact: false,
+ offer: offer(NODE_A),
},
+ { status: "active", suffix: Path.from("other"), route: 0n, instance: Instance.parse("a") },
],
],
+ // A DEAD and the ROUTE_UPDATEs it counts are one step, so nothing else lies between them.
[
- "a DEAD part followed by another message",
- [
- { status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) },
- {
- status: "dead",
- dead: {
- kind: "closed",
- hop: { node: PEER_NODE, session: NODE_A },
- far: NODE_A,
- gone: true,
- more: true,
- moved: [],
- },
- },
- { status: "floor", id: 0n },
- ],
+ "a DEAD followed by another message than its ROUTE_UPDATEs",
+ [...DEAD_ROUTES, deadMoving(1n), { status: "floor", id: 0n }],
],
[
- "a DEAD part continued by one naming another hop",
+ "a DEAD followed by an ANNOUNCE_REQUEST before its ROUTE_UPDATEs",
+ [...DEAD_ROUTES, deadMoving(1n), { status: "request", remove: [], add: [] }],
+ ],
+ ["a DEAD followed by another DEAD before its ROUTE_UPDATEs", [...DEAD_ROUTES, deadMoving(1n), deadMoving(0n)]],
+ [
+ "a DEAD moving a route that is not live",
+ [...DEAD_ROUTES, deadMoving(1n), { status: "routeUpdate", id: 7n, offer: offer(NODE_A) }],
+ ],
+ [
+ "a DEAD moving a route that does not cross its hop",
+ [...DEAD_ROUTES, deadMoving(1n), { status: "routeUpdate", id: 1n, offer: offer(NODE_B) }],
+ ],
+ [
+ "a DEAD moving one route twice",
[
- {
- status: "dead",
- dead: {
- kind: "closed",
- hop: { node: PEER_NODE, session: 1n },
- far: 3n,
- gone: true,
- more: true,
- moved: [],
- },
- },
- {
- status: "dead",
- dead: {
- kind: "closed",
- hop: { node: PEER_NODE, session: 2n },
- far: 3n,
- gone: true,
- more: false,
- moved: [],
- },
- },
+ ...DEAD_ROUTES,
+ deadMoving(2n),
+ { status: "routeUpdate", id: 0n, offer: offer(NODE_A) },
+ { status: "routeUpdate", id: 0n, offer: offer(NODE_A) },
],
],
+ ["a DEAD moving more routes than cross its hop", [...DEAD_ROUTES, deadMoving(3n)]],
+ ["a DEAD moving a route of a stream with none", [deadMoving(1n)]],
[
"a route claiming one path twice",
[
- { status: "routeStart", node: NODE_A, suffix: Path.empty(), offer: offer(NODE_A) },
+ {
+ status: "routeStart",
+ prefix: 0n,
+ node: NODE_A,
+ suffix: Path.empty(),
+ exact: false,
+ offer: offer(NODE_A),
+ },
{ status: "active", suffix: Path.from("room"), route: 0n, instance: Instance.parse("a") },
{ status: "active", suffix: Path.from("room"), route: 0n, instance: Instance.parse("b") },
],
@@ -5499,17 +6337,36 @@ test.each([
[{ status: "active", suffix: Path.from("room"), route: 0n, instance: Instance.parse("a") }],
],
["a floor for an unknown route", [{ status: "floor", id: 0n }]],
+ ["a SYNCED for a Prefix ID never asked for", [{ status: "synced", prefix: 1n }]],
+ [
+ "a second SYNCED for one Prefix ID",
+ [
+ { status: "synced", prefix: 0n },
+ { status: "synced", prefix: 0n },
+ ],
+ ],
+ ["an ANNOUNCE_OK answering no ANNOUNCE_REQUEST", [{ status: "ok" }]],
] as [string, AnnounceBroadcast[]][])("lite-07 %s is a violation", async (_, messages) => {
- const version = Version.DRAFT_07;
- const { subscriber, send, settle, sessionEnded } = announceHarness(version);
- const announced = subscriber.routes();
- await settle();
- await send(async (w) => {
- await new AnnounceOk(PEER, 0).encode(w, version);
- for (const msg of messages) await encodeAnnounceBroadcast(w, msg, version);
- });
+ const { subscriber, announced, send, closes } = await routedStream();
+ expect(closes).toEqual([]);
+ // Every message before the last is one the peer may send.
+ for (const msg of messages.slice(0, -1)) await send(msg);
+ expect(closes).toEqual([]);
+ await send(messages[messages.length - 1]);
await expect(announced.next()).rejects.toThrow(ProtocolViolation);
- expect((await sessionEnded())?.closeCode).toBe(SessionCode.ProtocolViolation);
+ expect(closes).toHaveLength(1);
+ expect(closes[0]?.closeCode).toBe(SessionCode.ProtocolViolation);
+ announced.close();
+ subscriber.close();
+});
+
+// The peer finishing its Announce Stream between a DEAD and the ROUTE_UPDATEs it counted leaves
+// the step half said.
+test("lite-07 an Announce Stream finished inside a DEAD is a violation", async () => {
+ const { subscriber, announced, send } = await routedStream();
+ for (const msg of DEAD_ROUTES) await send(msg);
+ await send(deadMoving(1n));
+ expect(() => subscriber.announceFinished()).toThrow(ProtocolViolation);
announced.close();
subscriber.close();
});
diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts
index 87c757209da..c6510aa72a0 100644
--- a/js/net/src/lite/subscriber.ts
+++ b/js/net/src/lite/subscriber.ts
@@ -25,11 +25,10 @@ import {
import { fillChanged, type Judged, rejectGroups } from "../fill.ts";
import type { Producer as FrameProducer } from "../frame.ts";
import * as netGroup from "../group.ts";
-import { Cost, type Hop, MAX_HOPS, routesEqual, UNKNOWN_HOP, WireRoute } from "../hop.ts";
+import { Cost, type Hop, MAX_HOPS, routesEqual, UNKNOWN_HOP, type WireRoute } from "../hop.ts";
import * as Instance from "../instance.ts";
import {
type Arriving,
- bestCandidate,
type DropRun,
hiddenBelow,
hooks,
@@ -63,27 +62,22 @@ import { Budget } from "../util/yield.ts";
import { hasTime, rangesStart, sameRanges, Window, wireRanges, wireSpan } from "../window.ts";
import { overrideBroadcastWire, wireOf } from "../wire.ts";
import {
+ type AnnounceBroadcast,
AnnounceHistory,
AnnounceInit,
AnnounceOk,
- type Refusal as AnnounceRefusal,
AnnounceRequest,
- type Base,
- type Dead,
decodeAnnounceBroadcastMaybe,
- type Offer,
- type Refusable,
- ROUTE_MAX_HOPS,
- type RouteHop,
- type Unusable,
} from "./announce.ts";
+import { AnnounceStream } from "./announce_stream.ts";
import { Datagram as DatagramMessage, Datagrams, DatagramsInfo, DatagramsUpdate } from "./datagram.ts";
import { closeViolation } from "./error.ts";
import { Fetch as FetchMessage } from "./fetch.ts";
import { dropped, passing, RangeFills, type RecoveryHandle, type Refusal, type Refused, refuseRun } from "./fills.ts";
-import { type Carrier, type Claim, Floors } from "./floors.ts";
+import { Floors } from "./floors.ts";
import { type Group as GroupMessage, type Ingest, readFrames } from "./group.ts";
import { Probe } from "./probe.ts";
+import { Routed } from "./routed.ts";
import { ProbeLevel, type Setup } from "./setup.ts";
import { StreamId } from "./stream.ts";
import {
@@ -482,8 +476,17 @@ export class Subscriber {
#pending = new Map>();
#subscribeNext = 0n;
- // Lite-07: what this session's announce streams certify of each origin node's claims.
+ // Lite-07: what this session's Announce Stream certifies of each origin node's claims.
#floors = new Floors();
+ /**
+ * Lite-07: changes whenever what {@link covers} answers may have changed with no route
+ * reporting it.
+ *
+ * @internal
+ */
+ readonly coverage = new Signal(0);
+ // Lite-07: the routes and claims the peer sends, shared by every `routes()` reader.
+ #routed?: Routed;
// Dedup consumed broadcasts per path: repeat consume() calls share one subscription.
#consumes = new BroadcastCache(this.#floors);
#consumeNext = 0;
@@ -550,6 +553,24 @@ export class Subscriber {
this.#goaway = goaway;
this.#grant = grant;
this.#auth = auth;
+ if (hasRoutes(version)) {
+ const stream = AnnounceStream.of(quic, version);
+ this.#routed = new Routed({
+ quic,
+ version,
+ hop,
+ floors: this.#floors,
+ consumes: this.#consumes,
+ slots: this.#announces,
+ drained: (route) => this.#drained(route),
+ revalidate: () => {
+ this.#consumes.revalidate();
+ this.coverage.update((n) => n + 1);
+ },
+ goaway,
+ send: (msg) => stream.send(msg),
+ });
+ }
}
// Whether the peer has sent GOAWAY. Requests keep opening here until a replacement
@@ -592,14 +613,15 @@ export class Subscriber {
const consumer = announced.consume();
// The last reader leaving ends the stream, which cancels the request.
closeUnwanted(announced);
- void this.#runAnnounced(
- announced,
- scopeHead(scope),
- scope,
- options?.hidden ?? false,
- options?.covering ?? false,
- options?.claims ?? false,
- );
+ const prefix = scopeHead(scope);
+ const hidden = options?.hidden ?? false;
+ const covering = options?.covering ?? false;
+ if (this.#routed) {
+ // Lite-07: every reader shares the session's one Announce Stream.
+ this.#routed.attach(announced, { prefix, scope, hidden, covering, perNode: options?.claims ?? false });
+ } else {
+ void this.#runAnnounced(announced, prefix, scope, hidden, covering);
+ }
return consumer;
}
@@ -609,14 +631,13 @@ export class Subscriber {
scope: Path.Pattern,
hidden: boolean,
covering: boolean,
- claims: boolean,
): Promise {
console.debug(`announced: prefix=${prefix}`);
// Lite04/05: send our own session-level Hop ID so the peer can skip announces
// whose hop chain already passed through us. Encoding drops it on every other
// version, where we drop the reflected announce on receipt instead. Matches the
// Rust subscriber's `exclude_hop: self.self_origin.id` in `run_announce_prefix`.
- const msg = new AnnounceRequest(prefix, this.hop, hidden);
+ const msg = new AnnounceRequest(prefix, this.hop);
const visible = (path: Path.Valid) =>
(covering ? scopeOverlaps(scope, path) : scope.matches(path)) && (hidden || !hiddenBelow(prefix, path));
@@ -701,13 +722,6 @@ export class Subscriber {
initial = ok.active;
}
- if (hasRoutes(this.version)) {
- await this.#receiveRouted(stream, announced, { prefix, scope, hidden, visible, perNode: claims });
- announced.close();
- stream.close();
- return;
- }
-
switch (this.version) {
case Version.DRAFT_01:
case Version.DRAFT_02: {
@@ -856,7 +870,7 @@ export class Subscriber {
case "skipped":
continue;
default:
- // The restart and the route layer are lite-07's, answered by #receiveRouted.
+ // The restart and the route layer are lite-07's, on its Announce Stream.
throw new ProtocolViolation(`unexpected ${announce.status} before lite-07`);
}
@@ -1020,597 +1034,33 @@ export class Subscriber {
}
}
- // Lite-07: the peer's routes and the claims under each, every route's claim changes applied
- // whole at its floor. Each path reports the best claim the stream holds for it, ranked as the
- // origin ranks routes, and a consume pins its requests to that claim's origin node. A claim
- // another stream's newer floor withdrew or replaced is held back until this stream catches up
- // (see `Floors`). Mirrors `subscriber/routed.rs` in rs/moq-net for a node that forwards
- // nothing: an endpoint applies the peer's withdrawals and holds, and casts no doubt.
- async #receiveRouted(
- stream: Stream,
- announced: announce.Producer,
- request: {
- prefix: Path.Valid;
- scope: Path.Pattern;
- hidden: boolean;
- visible: (path: Path.Valid) => boolean;
- perNode: boolean;
- },
- ): Promise {
- const { prefix, scope, hidden, visible, perNode } = request;
- type Route = {
- id: bigint;
- node: bigint;
- // The prefix it covers, the requested one joined with the peer's suffix, or undefined when
- // this node refused that suffix, which leaves the route refused for good.
- prefix: Path.Valid | undefined;
- // The path the peer last named, kept while this node refuses the rest, so a DEAD finds it.
- path: RouteHop[];
- // Undefined while this node refuses what the peer forwarded: no route at all, its claims
- // offline, until an offer it can use. Mirrors rs/moq-net.
- offer: Offer | undefined;
- // A DRAIN named a node on its path since its last offer: it ranks last.
- drain: boolean;
- // ROUTE_HOLD, or a DEAD that held it: no usable path to the publisher, until its next offer.
- held: boolean;
- // The Announce ID of each live claim, by path: one per path, checked by lookup.
- claims: Map;
- // The Announce IDs of its live claims whose path this node refused.
- unnamed: Set;
- // The Announce IDs of its live claims at a path whose content this node refuses.
- withheld: Set;
- // How the claims differ from the view's, applied whole at the next floor.
- pending: Map;
- // The claims as of the last floor, and the revision it certified them at, if any.
- view: { prefix: Path.Valid; hidden: boolean; revision: bigint | undefined; claims: Map };
- // The view's claims no other stream's newer floor withdrew or replaced, each as the
- // newest floor certifying it has it, cost included.
- shown: Map;
- carrier: Carrier;
- // Who the route's claims are filed under in the consume cache, which keeps every
- // route's claim so a consume ranks them for its own path.
- owner: object;
- };
- const same = (a: Claim | undefined, b: Claim | undefined) =>
- a === b ||
- (!!a &&
- !!b &&
- a.instance === b.instance &&
- a.epoch === b.epoch &&
- a.cost === b.cost &&
- a.above === b.above);
-
- // Each live route and claim holds one of the session's announce slots.
- const routes = new SlotMap(this.#announces);
- // A claim refused has no `claim` until a restart names all of it, and one refused before its
- // path has no `path` either, and never will.
- const claims = new SlotMap(this.#announces);
- // The live route to each publisher node and prefix: at most one each.
- const named = new Map();
- const key = (node: bigint, at: Path.Valid | undefined) =>
- at === undefined || node === 0n ? undefined : `${node} ${at}`;
- // The routes whose committed claims include each path.
- const byPath = new Map>();
- // What each visible path reports; see `refresh`.
- const reported = new Map>();
- const history = new AnnounceHistory();
- let nextRoute = 0n;
- let synced = false;
- // Set once the stream lets go of its routes, which then show nothing anew.
- let closing = false;
- const route = (id: bigint): Route => {
- const found = routes.get(id);
- if (!found) throw new ProtocolViolation(`unknown route ${id}`);
- return found;
- };
- const claimOf = (id: bigint) => {
- const found = claims.get(id);
- if (!found) throw new ProtocolViolation(`unknown announce id ${id}`);
- return found;
- };
- // The claim at `path` under `id` is now `claim`, pending the route's next floor.
- const change = (id: bigint, path: Path.Valid, claim: Claim | undefined) => {
- const wire = route(id);
- if (same(wire.view.claims.get(path), claim)) wire.pending.delete(path);
- else wire.pending.set(path, claim);
- };
-
- // Whether this node takes what the peer offers for the route at all: not a refused offer or
- // suffix, nor a route to the reserved node. Only such a route joins the node's floors.
- const joined = (wire: Route): wire is Route & { offer: Offer; prefix: Path.Valid } =>
- wire.offer !== undefined && wire.prefix !== undefined && wire.node !== 0n;
- // A route is no route at all once it crossed as many nodes as one may, or crossed us: a copy
- // of our own route, come back.
- // The same, without narrowing, to compare a route before and after it changes.
- const filed = (wire: Route): boolean => joined(wire);
- const usable = (wire: Route): wire is Route & { offer: Offer; prefix: Path.Valid } =>
- joined(wire) &&
- wire.offer.path.length < ROUTE_MAX_HOPS &&
- !wire.offer.path.some((hop) => hop.node === this.hop);
-
- // A claim through this session: its cost plus the route's metric, past this session's link
- // cost of 1 as Rust's default, ranked last while a node on its path drains, then by the
- // lowest priority on its path, past this link's 0 as Rust's default, and unavailable while
- // the route is held. Requests through it name its publisher and its prefix's depth.
- const candidate = (wire: Route & { offer: Offer; prefix: Path.Valid }, claim: Claim): WireRoute =>
- this.#drained(
- WireRoute.normalize({
- instance: claim.instance,
- epoch: claim.epoch,
- cost: claim.cost + wire.offer.metric + 1n,
- node: wire.node,
- depth: Path.parts(wire.prefix).length,
- priority: Math.min(0, wire.offer.priority),
- above: claim.above,
- draining: wire.offer.draining || wire.drain || undefined,
- available: !wire.held,
- }),
- );
-
- // File the route's claim at `path` in the consume cache, or take it out. Every route's claim
- // stays there, so each consume ranks them for its own path and its epochs. `restart` marks
- // another instance or epoch, whose consumes subscribe fresh. An unusable route's claim is
- // parked, keeping its place in a tie as `byPath` does, unless it just moved to the newest.
- const serve = (wire: Route, path: Path.Valid, restart: boolean) => {
- const claim = wire.shown.get(path);
- if (!claim || !visible(path) || (!usable(wire) && restart)) this.#consumes.withdraw(path, wire.owner);
- else if (!usable(wire)) this.#consumes.park(path, wire.owner);
- else this.#consumes.announce(path, wire.owner, candidate(wire, claim), restart);
- };
-
- // Report the best claim at `path`, or with `perNode` each origin node's, and each one's end.
- // `lost` marks the routes leaving as let go rather than withdrawn, so they go offline
- // rather than ending.
- const refresh = (path: Path.Valid, lost: boolean) => {
- // Keyed by origin node with `perNode`, and otherwise the one key `undefined`.
- const now = new Map();
- if (visible(path)) {
- const candidates: WireRoute[] = [];
- for (const id of byPath.get(path) ?? []) {
- const wire = route(id);
- const claim = wire.shown.get(path);
- if (claim && usable(wire)) candidates.push(candidate(wire, claim));
- }
- if (perNode) {
- for (const next of candidates) now.set(next.node, next);
- } else {
- // The same pick a consume of `path` makes from the cache.
- const best = bestCandidate(path, path, candidates, (next) => next);
- if (best) now.set(undefined, best);
- }
- }
-
- const captures = scopeCaptures(scope, path);
- const prev = reported.get(path) ?? new Map();
- // Arrivals first, so a reader sees a claim replaced rather than the path gone between.
- for (const [key, next] of now) {
- const was = prev.get(key);
- if (was && routesEqual(was, next)) continue;
- const restart = !!was && (was.instance !== next.instance || was.epoch !== next.epoch);
- const kind = !was ? "start" : restart ? "restart" : "update";
- console.debug(`announced: broadcast=${path} ${kind} instance=${next.instance} node=${next.node}`);
- announced.append({ prefix: path, captures, kind, route: next });
- }
- for (const [key, was] of prev) {
- if (now.has(key)) continue;
- console.debug(`announced: broadcast=${path} active=false lost=${lost} node=${was.node}`);
- announced.append({ prefix: path, captures, kind: lost ? "offline" : "end", route: was });
- }
- if (now.size > 0) reported.set(path, now);
- else reported.delete(path);
- };
- // Show or hide the route's claims at `touched`, each marked whether it restarted, or with
- // `all` every claim it holds, once another stream's floor may have moved what is current.
- const show = (wire: Route, touched: ReadonlyMap, all: boolean) => {
- const paths = all
- ? new Set([...touched.keys(), ...wire.shown.keys(), ...wire.view.claims.keys()])
- : touched.keys();
- for (const path of paths) {
- const now = this.#floors.shows(wire.node, wire.view, path);
- const was = wire.shown.get(path);
- const restart = touched.get(path);
- if (same(now, was) && restart === undefined) continue;
- if (now) wire.shown.set(path, now);
- else wire.shown.delete(path);
- // A claim shown anew or restarted iterates last, as the newest; a re-price keeps its
- // place, as the consume cache does. See bestCandidate.
- const ids = byPath.get(path);
- if (now && ids && (!was || restart)) {
- ids.delete(wire.id);
- ids.add(wire.id);
- }
- serve(wire, path, restart ?? false);
- // A claim gone was let go, not withdrawn by its origin, where the newest floor certifying it
- // still presents it, or where this node refuses one of the route's claims and no floor
- // certifies it gone. An uncertified withdrawal otherwise still ends: a relay certifies
- // nothing to a request wider than its scope's region, as this one may be.
- const said = now ? undefined : this.#floors.certified(wire.node, wire.view, path);
- const refusing = wire.unnamed.size > 0 || wire.withheld.size > 0;
- refresh(path, !now && (said === "present" || (refusing && said === undefined)));
- }
- };
- const refreshRoute = (wire: Route, lost: boolean) => {
- for (const path of wire.view.claims.keys()) {
- serve(wire, path, false);
- refresh(path, lost);
- }
- };
-
- // The peer started a route to `node` with the next Route ID, covering the requested prefix
- // joined with `suffix`, along `offer`, or along `path` when this node refuses the offer.
- const startRoute = (
- node: bigint,
- suffix: Path.Valid | undefined,
- offer: Offer | undefined,
- path: RouteHop[],
- ) => {
- // The interest prefix can push a joined path past the limits: refused too.
- const joinedPrefix = suffix === undefined ? undefined : Path.join(prefix, suffix);
- const at = joinedPrefix !== undefined && Path.check(joinedPrefix) === undefined ? joinedPrefix : undefined;
- const named_ = key(node, at);
- // The peer keeps one route per publisher and prefix.
- if (named_ !== undefined && named.has(named_))
- throw new ProtocolViolation(`second route to node ${node} at ${at}`);
- const id = nextRoute++;
- const view: Route["view"] = { prefix: at ?? prefix, hidden, revision: undefined, claims: new Map() };
- const wire: Route = {
- id,
- node,
- prefix: at,
- path,
- offer,
- drain: false,
- held: false,
- claims: new Map(),
- unnamed: new Set(),
- withheld: new Set(),
- pending: new Map(),
- view,
- shown: new Map(),
- carrier: {
- view,
- // Another route's floor or end runs this: a closed reader is about to let go.
- reshow: () => {
- if (!closing && announced.closed.peek() === undefined) show(wire, new Map(), true);
- },
- },
- owner: {},
- };
- routes.set(id, wire);
- if (named_ !== undefined) named.set(named_, id);
- // A refused route's floors stay its own, as Rust's node never holds the route.
- if (joined(wire)) this.#floors.join(node, wire.carrier);
- };
-
- // The peer let go of the route and every claim under it: offline, not withdrawn.
- const endRoute = (wire: Route) => {
- for (const id of [...wire.claims.values(), ...wire.unnamed]) {
- history.end(id);
- claims.delete(id);
- }
- routes.delete(wire.id);
- const named_ = key(wire.node, wire.prefix);
- if (named_ !== undefined && named.get(named_) === wire.id) named.delete(named_);
- // Unindexed first: leaving its floors reshows another carrier at once, which
- // refreshes these paths and so looks up every route indexed there.
- for (const path of wire.view.claims.keys()) byPath.get(path)?.delete(wire.id);
- // What its last floor certified stays, so no other stream's lagging claim revives.
- this.#floors.leave(wire.node, wire.carrier);
- wire.shown.clear();
- refreshRoute(wire, true);
- this.#consumes.revalidate();
- for (const path of wire.view.claims.keys()) if (byPath.get(path)?.size === 0) byPath.delete(path);
- };
-
- // The peer's `above` for the claim at `path`, if it can be: a claim is no more than its
- // path's segments broader than the path it is shown at. Mirrors rs/moq-net's `Routed::above`.
- const aboveOf = (path: Path.Valid | undefined, above: bigint | undefined): bigint => {
- const value = above ?? 0n;
- if (path === undefined || value === 0n) return value;
- if (value > BigInt(Path.parts(path).length))
- throw new ProtocolViolation(`a claim at ${path} cannot be ${value} segments above it`);
- return value;
- };
-
- // A claim the peer started under `routeId` with the next Announce ID, at `path` unless this
- // node refused it, and `claim` unless it refuses what the peer forwarded.
- const startClaim = (
- routeId: bigint,
- path: { suffix: Path.Valid; pathBase?: Base } | undefined,
- claim: Claim | undefined,
- ) => {
- const wire = route(routeId);
- const suffix = history.start(path);
- const id = history.next - 1n;
- // The interest prefix can push a joined path past the limits: refused too.
- const full = suffix === undefined ? undefined : Path.join(prefix, suffix);
- const named = full !== undefined && Path.check(full) === undefined ? full : undefined;
- if (claim) claim.above = aboveOf(named, claim.above);
- if (named === undefined) {
- if (claim) console.warn(`refusing a forwarded claim: route=${routeId} claim=${id} reason=path`);
- claims.set(id, { route: routeId });
- wire.unnamed.add(id);
- return;
- }
- if (wire.claims.has(named)) throw new ProtocolViolation(`route ${routeId} claims ${named} twice`);
- // A route covers only the claims at or below its prefix.
- if (wire.prefix !== undefined && !Path.hasPrefix(wire.prefix, named))
- throw new ProtocolViolation(`route ${routeId} at ${wire.prefix} claims ${named}`);
- claims.set(id, { route: routeId, path: named, claim });
- wire.claims.set(named, id);
- if (claim) change(routeId, named, claim);
- else wire.withheld.add(id);
- };
-
- // The route now reaches its node by `offer`, or along `path` while this node refuses what the
- // peer forwarded. Either way the peer no longer holds it, nor drains a node on it but as the
- // offer says. A refused route leaves the node's floors, as an ended one does, and joins again
- // once usable, with what its last floor certified, as rs/moq-net files it again.
- const reach = (wire: Route, offer: Offer | undefined, path: RouteHop[]) => {
- const was = filed(wire);
- wire.offer = offer;
- wire.path = path;
- wire.held = false;
- wire.drain = false;
- const now = filed(wire);
- if (was && !now) {
- this.#floors.leave(wire.node, wire.carrier);
- // What its floor certified stays as it was, while the route goes on reading claims.
- wire.view.claims = new Map(wire.view.claims);
- }
- if (was || !now) return;
- this.#floors.join(wire.node, wire.carrier);
- const others = this.#floors.floor(wire.node, wire.carrier);
- show(wire, new Map(), true);
- if (others) this.#consumes.revalidate();
- };
- // The route moved to `offer`, which this node refuses with its path when it is `Unusable`.
- const move = (wire: Route, offer: Offer | Unusable) => {
- const was = filed(wire);
- if ("refused" in offer) {
- console.warn(`refusing a forwarded route: node=${wire.node} route=${wire.id} reason=${offer.refused}`);
- reach(wire, undefined, offer.path);
- } else {
- reach(wire, offer, offer.path);
- }
- refreshRoute(wire, !usable(wire));
- // Its floors no longer cover anything, as on ROUTE_END.
- if (was && !filed(wire)) this.#consumes.revalidate();
- };
-
- // The parts of a DEAD read so far, which apply as one once its last arrives.
- let continued: { hop: RouteHop; far: bigint; gone: boolean; offers: Map } | undefined;
-
- // A DEAD: the stream's routes through the hop it names carry on as it says. A route this
- // node refuses still follows it, so both ends agree on which routes live. An endpoint casts
- // no doubt: it never forwards a route, so no detour waits on one.
- const dead = (news: Dead) => {
- if (news.kind === "drain") {
- for (const wire of routes.values()) {
- if (!wire.path.some((hop) => hop.node === news.node)) continue;
- wire.drain = true;
- refreshRoute(wire, false);
- }
- return;
- }
- // A DEAD in parts applies with its last, each part naming the same hop, far node, and gone.
- const parts = continued ?? { hop: news.hop, far: news.far, gone: news.gone, offers: new Map() };
- continued = undefined;
- const same =
- parts.hop.node === news.hop.node &&
- parts.hop.session === news.hop.session &&
- parts.far === news.far &&
- parts.gone === news.gone;
- if (!same) throw new ProtocolViolation("DEAD part names another hop");
- // Each part names live routes, so what the parts hold stays within the stream's.
- const offers = parts.offers;
- for (const { id, offer } of news.moved) offers.set(route(id).id, offer);
- if (news.more) {
- continued = parts;
- return;
- }
- const through = [...routes.values()].filter((wire) =>
- wire.path.some((hop) => hop.node === news.hop.node && hop.session === news.hop.session),
- );
- for (const wire of through) {
- const moved = offers.get(wire.id);
- const before = wire.path.at(-2);
- if (moved !== undefined) move(wire, moved);
- else if (news.gone) endRoute(wire);
- // The peer learned it from this node, which it holds under: unchanged.
- else if (before?.node === this.hop) continue;
- else {
- wire.held = true;
- refreshRoute(wire, false);
- }
- }
- };
-
- // Apply a message whose content, which the peer only forwarded, this node refuses: withdraw
- // the route or claim it names, and keep the stream. Mirrors rs/moq-net.
- const refused = (message: Refusable, reason: AnnounceRefusal) => {
- switch (message.kind) {
- case "routeStart":
- console.warn(`refusing a forwarded route: node=${message.node} reason=${reason}`);
- return startRoute(message.node, message.suffix, undefined, message.path);
- case "routeUpdate":
- return move(route(message.id), { refused: reason, path: message.path });
- case "start":
- console.warn(
- `refusing a forwarded claim: route=${message.route} claim=${history.next} reason=${reason}`,
- );
- return startClaim(message.route, message.path, undefined);
- case "change": {
- const live = claimOf(message.id);
- console.warn(
- `refusing a forwarded claim: route=${live.route} claim=${message.id} reason=${reason}`,
- );
- live.claim = undefined;
- if (live.path === undefined) return;
- route(live.route).withheld.add(message.id);
- change(live.route, live.path, undefined);
- return;
- }
- }
- };
+ /**
+ * Lite-07: one message the peer sent on its Announce Stream. Its routes and claims apply
+ * here; its own ANNOUNCE_REQUEST only has to arrive outside a DEAD's updates.
+ *
+ * @internal
+ */
+ announce(msg: AnnounceBroadcast): Promise {
+ if (!this.#routed) throw new ProtocolViolation("Announce Stream before lite-07");
+ return this.#routed.receive(msg);
+ }
- // A draining peer ranks its routes last from the GOAWAY itself.
- const stopDrain = this.#goaway?.changed(() => {
- if (announced.closed.peek() !== undefined) return;
- for (const wire of routes.values()) refreshRoute(wire, false);
- });
+ /**
+ * Lite-07: the peer finished its Announce Stream, so it is leaving and its routes end.
+ *
+ * @internal
+ */
+ announceFinished(): void {
+ this.#routed?.finish();
+ }
- try {
- for (;;) {
- const msg = await race([decodeAnnounceBroadcastMaybe(stream.reader, this.version), announced.closed]);
- if (msg === undefined && !synced) throw new ProtocolViolation("announce stream ended before SYNCED");
- // undefined: the stream ended. null: the consumer closed cleanly.
- if (!msg) break;
- if (msg instanceof Error) throw msg;
- // The parts of a DEAD arrive back to back.
- if (continued && !(msg.status === "dead" && msg.dead.kind === "closed"))
- throw new ProtocolViolation("DEAD part followed by another message");
-
- switch (msg.status) {
- case "routeStart":
- startRoute(msg.node, msg.suffix, msg.offer, msg.offer.path);
- break;
- case "routeUpdate":
- move(route(msg.id), msg.offer);
- break;
- case "routeHold": {
- const wire = route(msg.id);
- wire.held = true;
- refreshRoute(wire, false);
- break;
- }
- case "routeEnd":
- endRoute(route(msg.id));
- break;
- case "dead":
- dead(msg.dead);
- break;
- case "floor": {
- // The batch applies whole, at the revision its floor certifies: none while this
- // node refuses one of the route's claims, which its origin never withdrew.
- const wire = route(msg.id);
- const revision = wire.unnamed.size === 0 && wire.withheld.size === 0 ? msg.revision : undefined;
- // A floor that certifies less keeps what the route's last one certified.
- const certified = wire.view.revision;
- const filed = joined(wire);
- if (filed && certified !== undefined && (revision === undefined || revision < certified))
- this.#floors.regress(wire.node, wire.carrier);
- const touched = new Map();
- for (const [path, claim] of wire.pending) {
- // Another instance or epoch: what was consumed through the claim may be stale.
- const was = wire.view.claims.get(path);
- const restart =
- !!was && !!claim && (was.instance !== claim.instance || was.epoch !== claim.epoch);
- touched.set(path, restart);
- if (claim) {
- wire.view.claims.set(path, claim);
- let ids = byPath.get(path);
- if (!ids) {
- ids = new Set();
- byPath.set(path, ids);
- }
- ids.add(msg.id);
- } else {
- wire.view.claims.delete(path);
- const ids = byPath.get(path);
- ids?.delete(msg.id);
- if (ids?.size === 0) byPath.delete(path);
- }
- }
- wire.pending.clear();
- wire.view.revision = revision;
- // Every other route to the node is shown anew first; with one, this route's
- // unchanged claims may change state too.
- const others = filed && this.#floors.floor(wire.node, wire.carrier);
- show(wire, touched, others);
- // Another view of the node may now cover a consumed path differently.
- if (others) this.#consumes.revalidate();
- break;
- }
- case "active": {
- if (msg.route === undefined) throw new ProtocolViolation("lite-07 claim without a route");
- if (msg.instance === undefined)
- throw new ProtocolViolation("lite-07 claim without an instance");
- const claim = {
- instance: msg.instance,
- epoch: msg.epoch ?? 0n,
- cost: msg.cost ?? Cost.zero,
- above: msg.above ?? 0n,
- };
- startClaim(msg.route, msg, claim);
- break;
- }
- case "update": {
- const live = claimOf(msg.id);
- const above = aboveOf(live.path, msg.above);
- // An update names none of what a refused claim lacks.
- if (live.claim === undefined || live.path === undefined) break;
- live.claim = { ...live.claim, cost: msg.cost ?? Cost.zero, above };
- change(live.route, live.path, live.claim);
- break;
- }
- case "restart": {
- const live = claimOf(msg.id);
- if (msg.instance === undefined)
- throw new ProtocolViolation("lite-07 restart without an instance");
- // A claim without a path stays refused, whatever the peer sends for it.
- if (live.path === undefined) break;
- live.claim = {
- instance: msg.instance,
- epoch: msg.epoch ?? 0n,
- cost: msg.cost ?? Cost.zero,
- above: aboveOf(live.path, msg.above),
- };
- route(live.route).withheld.delete(msg.id);
- change(live.route, live.path, live.claim);
- break;
- }
- case "refused":
- await refused(msg.message, msg.reason);
- break;
- case "endedId": {
- const live = claimOf(msg.id);
- history.end(msg.id);
- claims.delete(msg.id);
- const wire = route(live.route);
- if (live.path === undefined) {
- wire.unnamed.delete(msg.id);
- break;
- }
- wire.claims.delete(live.path);
- wire.withheld.delete(msg.id);
- change(live.route, live.path, undefined);
- break;
- }
- case "synced":
- if (synced) throw new ProtocolViolation("duplicate SYNCED");
- synced = true;
- announced.synced();
- break;
- case "skipped":
- break;
- case "ended":
- // Lite-07 ends a claim by its Announce ID only.
- throw new ProtocolViolation("ANNOUNCE_END by path on lite-07");
- }
- }
- } finally {
- stopDrain?.();
- closing = true;
- for (const wire of routes.values()) {
- this.#floors.leave(wire.node, wire.carrier);
- for (const path of wire.view.claims.keys()) this.#consumes.withdraw(path, wire.owner);
- }
- if (routes.size > 0) this.#consumes.revalidate();
- reported.clear();
- claims.clear();
- routes.clear();
- }
+ /**
+ * Lite-07: this side finished its Announce Stream, so it asks the peer for nothing more.
+ *
+ * @internal
+ */
+ announceSilenced(): void {
+ this.#routed?.silence();
}
/**
@@ -4178,6 +3628,7 @@ export class Subscriber {
*/
close(err: Error = new SessionError(SessionCode.Cancel)) {
this.#closed.abort(err);
+ this.#routed?.close(err);
for (const { track, stream, end, datagrams } of this.#subscribes.values()) {
if (stream.kind === "finished") track.finish();
diff --git a/js/net/src/request-deadline.test.ts b/js/net/src/request-deadline.test.ts
index cb0eba142df..21d2653d6c5 100644
--- a/js/net/src/request-deadline.test.ts
+++ b/js/net/src/request-deadline.test.ts
@@ -176,9 +176,10 @@ for (const protocol of ["lite", "ietf"] as const) {
const clock = new Manual();
const installed = clock.install();
const transport = limited(mode);
+ // Lite-06 asks on a bidirectional stream of its own; lite-07 asks on its Announce Stream.
const subscriber =
protocol === "lite"
- ? new LiteSubscriber(transport.quic, Lite.DRAFT_07, toHop(1n))
+ ? new LiteSubscriber(transport.quic, Lite.DRAFT_06, toHop(1n))
: new IetfSubscriber({ session: new NativeSession(transport.quic, Ietf.DRAFT_18, true) });
const announced = subscriber.routes();
try {
diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts
index cc41b68d359..2565eae0082 100644
--- a/js/net/src/wire.ts
+++ b/js/net/src/wire.ts
@@ -172,6 +172,11 @@ export interface Established {
* it: lite-07's floors. Unset where every claim covers its subtree.
*/
covers?(node: bigint, at: Path.Valid, path: Path.Valid): boolean;
+ /**
+ * Changes whenever what {@link covers} answers may have changed with no route reporting it:
+ * a floor moving what another claim of its origin node covers.
+ */
+ readonly coverage?: Getter;
// private: plumbing
/**
* This session's side of its auth handle.
diff --git a/quest/m2/README.md b/quest/m2/README.md
index 87c4e57f072..da380c4cd31 100644
--- a/quest/m2/README.md
+++ b/quest/m2/README.md
@@ -322,7 +322,7 @@ what would need a breaking release had it landed right after the cut:
in m2.
- ✅ Stayed in m2 though not breaking itself:
[the interest set](https://github.com/moq-dev/moq/pull/8728), because
- [the route stream](/quest/m2/cluster-routes/route-stream.md) needs it.
+ [the route stream](https://github.com/moq-dev/moq/pull/8835) needs it.
The route stream stays because lite-07 would otherwise freeze with one
router key per announce stream, which the route model doesn't check, and
keep the reopen-and-hand-over machinery for its lifetime (rejected: moving
diff --git a/quest/m2/cluster-routes/README.md b/quest/m2/cluster-routes/README.md
index 120d0325e49..aea3f4fae93 100644
--- a/quest/m2/cluster-routes/README.md
+++ b/quest/m2/cluster-routes/README.md
@@ -36,8 +36,8 @@ The items below were decided 2026-11-10 (maintainer, /quest-plan on lite-07 rout
18. ✅ Asked who is an origin node: **every publisher stays an origin node.** Its identity is for signing, for one node across several links (redundant publishers), and because browsers may forward (peer-to-peer routing, outside this repository since 2026-11-14). Rejected: a relay originating, under its own Node ID, the claims of any peer that sends no Node ID or holds a scoped publish grant, as it already does for lite-06 and moq-transport publishers. That would make route state follow relays rather than publishers and remove qualified Node IDs, the Route Key, and the echo, but a publisher would have no identity of its own. Accepted cost: routes, holds, and seqno requests scale with publishers, so a node must still forget dead identities, and the echo that makes forgetting safe has to be sound.
19. ✅ Asked whether to keep them: **ROUTE_HOLD stays, and the newest floor wins per identity.** Rejected: dropping holds, where each stall takes every claim behind the node offline and re-sends them all; and claims that follow only the selected next hop, which re-pulls a node's whole claim table on each next-hop change and showed revivals and false ENDs in the sweeps below.
20. ~~✅ Asked when an origin advances its seqno unprompted: **only when a route link changes**, implemented as a bump when a route link closes. Rejected: only to answer a ROUTE_REQUEST; keeping the per-stream bump.~~ Superseded by item 27: there is no seqno; a closed route link is reported by the DEAD naming its hop ([moq-dev/moq#8731](https://github.com/moq-dev/moq/pull/8731)).
-21. ✅ Asked how one side's routing messages travel so none overtakes another: **one unidirectional Announce Stream per side**, carrying its requests, its answers, and its echoes in order ([the route stream](/quest/m2/cluster-routes/route-stream.md)). It replaces "There MAY be multiple Announce Streams". Rejected: one bidirectional Announce Stream per direction with the echo moved to the answering half; one bidirectional stream per session.
-22. ✅ **A side's interest is a set of prefixes edited in place.** The live set never nests, and one edit may swap prefixes for a wider or narrower one without re-sending what both cover ([the interest set](https://github.com/moq-dev/moq/pull/8728)). A token-scope change is such an edit, with one prefix per region of the scope. This supersedes items 16 and 17, and the handoff contract's "An interest change opens a new stream" below, once the route stream lands.
+21. ✅ Asked how one side's routing messages travel so none overtakes another: **one unidirectional Announce Stream per side**, carrying its requests, its answers, and its echoes in order ([the route stream](https://github.com/moq-dev/moq/pull/8835)). It replaces "There MAY be multiple Announce Streams". Rejected: one bidirectional Announce Stream per direction with the echo moved to the answering half; one bidirectional stream per session.
+22. ✅ **A side's interest is a set of prefixes edited in place.** The live set never nests, and one edit may swap prefixes for a wider or narrower one without re-sending what both cover ([the interest set](https://github.com/moq-dev/moq/pull/8728)). A token-scope change is such an edit, with one prefix per region of the scope. This supersedes items 16 and 17.
23. ✅ Asked what a floor covers with several prefixes on one stream: **one prefix, named by its id.** This amends item 10: the floor is still a marker and never a region list. Rejected: a floor over the whole set.
24. ✅ Asked how a side learns a newly added prefix's initial set is complete: **SYNCED per prefix, naming its id**, plus one marker per edit where it takes effect for both sides. Rejected: one SYNCED per edit, counted in order.
25. ✅ Asked how a relay safely forgets a publisher that is gone, after the model showed a per-neighbour echo is not enough once the origin sits behind a relay: **each route names the nodes it crossed, and a node never uses a route that names itself** ([moq-dev/moq#8487](https://github.com/moq-dev/moq/pull/8487), which holds the counterexamples and the model results). Feasibility still selects routes; the list is one per origin route, not the per-claim Hop ID list item 6 removed, and the echo and the drain rule stay as they are. As built: Babel's feasibility condition is unchanged, and a node never selects or certifies through such a route. Where the condition refuses it, it still counts as infeasible, so a relay whose neighbours all route through it holds its claims while one of them finds another way. Under a qualified key it counts only while its hops extend those of the route the node last sent under that key: a copy that a scoped peer hands back under a key derived from the node's own route never moves, so nothing waits on it. The origin names itself too, so a rotated identity's route is refused where it began, and a node names itself by a Hop ID that a rotation leaves alone. The hop limit is the wire's 32 and no lower. A change of hops or RTT alone lets go of no region, unless it changes whether the node may use the route: by crossing it, or by reaching the hop limit. This supersedes the claim, in the handoff contract and in decision 6 below, that the echo alone makes forgetting safe. First decided as a relay echoing a ROUTE_END only once its own downstream had; the model ran that the same day and it fails when a session closes before answering. Rejected: that deferred echo; never dropping the distance; a timer; relays as the only origin nodes. Its echo and distance drain are superseded by item 27, which keeps the self-naming rule and drops the distance, the seqno, the request, and the echo ([moq-dev/moq#8731](https://github.com/moq-dev/moq/pull/8731)).
@@ -76,7 +76,7 @@ The items below were decided 2026-11-10 (maintainer, /quest-plan on lite-07 rout
Neither rule looped, and churn's stale observations stayed near 3,300 under both.
-28. ✅ Decided 2026-11-12 (maintainer, /quest-plan on the quests still written for Babel), asked how item 27's removal is split and ordered: **the router first with no wire change, then a switch of the sessions, and the switch lands before the route stream.** [The router](https://github.com/moq-dev/moq/pull/8622) runs the path rule with a route per scope, sans-IO, beside today's router. [The switch](https://github.com/moq-dev/moq/pull/8731) puts it on the lite-07-wip wire, the draft, the Rust sessions, and `js/net`, and deletes Babel's router. [The interest set](https://github.com/moq-dev/moq/pull/8728) then edits the path router only, and [the route stream](/quest/m2/cluster-routes/route-stream.md) follows the switch with no echo or route request to carry. The switch's recommended wire puts the answer that ends a hold on the publisher half, behind the routes its sender sent first, so the path rule needs no order between streams that today's sessions lack for one prefix. [Acceptance](/quest/m2/cluster-routes/acceptance.md) is of the switched sessions, and the inductive invariant stays in m1, rewritten for the path, without `LoopFree` as an invariant. Rejected: the router quest in m1; one quest for router, wire, draft, sessions, and `js/net`; the route stream first, which builds the echo and ROUTE_REQUEST on the new stream and then deletes them; one wire change for the path rule and the one stream together; moving the inductive invariant to m4; deleting it. Reversed 2026-11-13 (maintainer, /quest-complete): ✅ the inductive invariant is deleted, since Apalache cannot step even one transition of the route model ([moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564)), no Rust TLA+ checker qualifies ([moq-dev/moq#8600](https://github.com/moq-dev/moq/pull/8600), [moq-dev/moq#8669](https://github.com/moq-dev/moq/pull/8669)), and no more JVM model checking is wanted here (rejected: keeping it paused).
+28. ✅ Decided 2026-11-12 (maintainer, /quest-plan on the quests still written for Babel), asked how item 27's removal is split and ordered: **the router first with no wire change, then a switch of the sessions, and the switch lands before the route stream.** [The router](https://github.com/moq-dev/moq/pull/8622) runs the path rule with a route per scope, sans-IO, beside today's router. [The switch](https://github.com/moq-dev/moq/pull/8731) puts it on the lite-07-wip wire, the draft, the Rust sessions, and `js/net`, and deletes Babel's router. [The interest set](https://github.com/moq-dev/moq/pull/8728) then edits the path router only, and [the route stream](https://github.com/moq-dev/moq/pull/8835) follows the switch with no echo or route request to carry. The switch's recommended wire puts the answer that ends a hold on the publisher half, behind the routes its sender sent first, so the path rule needs no order between streams that today's sessions lack for one prefix. [Acceptance](/quest/m2/cluster-routes/acceptance.md) is of the switched sessions, and the inductive invariant stays in m1, rewritten for the path, without `LoopFree` as an invariant. Rejected: the router quest in m1; one quest for router, wire, draft, sessions, and `js/net`; the route stream first, which builds the echo and ROUTE_REQUEST on the new stream and then deletes them; one wire change for the path rule and the one stream together; moving the inductive invariant to m4; deleting it. Reversed 2026-11-13 (maintainer, /quest-complete): ✅ the inductive invariant is deleted, since Apalache cannot step even one transition of the route model ([moq-dev/moq#8564](https://github.com/moq-dev/moq/pull/8564)), no Rust TLA+ checker qualifies ([moq-dev/moq#8600](https://github.com/moq-dev/moq/pull/8600), [moq-dev/moq#8669](https://github.com/moq-dev/moq/pull/8669)), and no more JVM model checking is wanted here (rejected: keeping it paused).
29. ✅ Decided 2026-11-12 (maintainer, /quest-audit before the cut), asked how the path rule keeps the `upstream` guarantee, which the prototype left out and [the switch](https://github.com/moq-dev/moq/pull/8731) had listed for removal with Babel's machinery: **a route keeps one sticky down-only flag beside its path.** A relay sets it on a route it learns over a link marked `upstream`, it is never cleared on a later hop, and a relay never sends a flagged route over an upstream link. The flag is export policy, not loop avoidance, so the path does not replace it: a relay that hears a CDN route from a LAN peer learned it on a link that is not upstream, and only the flag tells it the route came down from an upstream. That case is a local cluster with more than one uplink ([outbound federation](/quest/m3/cluster-routing/federation.md)), where a relay whose own uplink died reaches the CDN through a peer; an edge that dials only its cores needs just the local rule. This is the Only-To-Customer attribute of BGP roles (RFC 9234). The router (`rs/moq-net/src/route/path.rs`, built in [#8622](https://github.com/moq-dev/moq/pull/8622)) carries the flag and its export check, [the switch](https://github.com/moq-dev/moq/pull/8731) adds them to `RouteHandoff.tla` and keeps the flag on ROUTE_START and ROUTE_UPDATE, and [acceptance](/quest/m2/cluster-routes/acceptance.md)'s two-uplink mesh proves it. Sender side only, as today. Rejected: a mark on each hop of the path, which gives the same answer from more bytes and model state; the local rule with no flag, which either leaks through a LAN peer or gives up the dead-uplink fallback; also refusing a flagged route that arrives from a link whose grant marks the sender as a customer, which can be added later without a wire change.
@@ -340,11 +340,11 @@ loss never reports an END:
and the three-failure loop regression `Loop.cfg` pins, are in the
`verify-route` skill. A sender sends a route on a stream only while the
route meets the stream's interest (Plan item 15), and a node presents each
- stream only the claims that changed. An interest change opens a new
- stream, and the first batch answering it retires whatever lies outside it
- (`cancelling_interest_retires_its_region`), until
- [the route stream](/quest/m2/cluster-routes/route-stream.md) carries the
- router's edits in place (Plan item 22).
+ stream only the claims that changed. An interest change is one edit of the
+ prefixes on the session's Announce Stream, and at its ANNOUNCE_OK both ends
+ carry over the routes both sets cover and retire whatever lies outside the
+ new set ([the route stream](https://github.com/moq-dev/moq/pull/8835),
+ Plan item 22).
- Per identity and prefix, a node is usable (a live selected route:
announcements shown available), held, or let go. It holds as the path rule
says (above), showing the announcements as unavailable and sending
diff --git a/quest/m2/cluster-routes/mount-scope.md b/quest/m2/cluster-routes/mount-scope.md
new file mode 100644
index 00000000000..3c39fc12e38
--- /dev/null
+++ b/quest/m2/cluster-routes/mount-scope.md
@@ -0,0 +1,26 @@
+# [S] A mount files under its session's scope
+
+## Goal
+
+A scoped lite-07 session that closes casts no doubt on the Node ID it
+declared, whether or not a mount of it is open, and a test on the route node
+shows it.
+
+## Plan
+
+Decided 2026-11-15 (maintainer, on [the route
+stream](https://github.com/moq-dev/moq/pull/8835)): ✅ a follow-up quest in
+m2 (rejected: m3; ignoring it).
+
+Pullfrog found it on the route stream. `Stream::mount` in
+`rs/moq-net/src/route/node.rs` files a mount's key with `scope: None`, since
+a mount hears nothing. A scoped session's close then reaches the
+router as an unscoped one's, so the router doubts the Node ID the session
+declared, which a scoped peer must not be trusted to speak for.
+
+- File a mount's key with the scope of the session it belongs to, or have the
+ session's close carry that scope for every key it ends.
+- Test that a scoped session closing through a mount casts no doubt on its
+ declared Node ID, and that an unscoped one still does.
+
+Public API: none. Wire: none.
diff --git a/quest/m2/cluster-routes/route-stream.md b/quest/m2/cluster-routes/route-stream.md
deleted file mode 100644
index 58c367f6e31..00000000000
--- a/quest/m2/cluster-routes/route-stream.md
+++ /dev/null
@@ -1,223 +0,0 @@
-# [XL] Each side says everything about routing on one stream
-
-## Goal
-
-In a lite-07 session each endpoint opens at most one unidirectional Announce
-Stream and sends everything it says about routing on it, in order: the
-prefixes it wants, and the routes, withdrawals, hold answers, claims, floors,
-and SYNCED that answer the peer's prefixes. No routing message can overtake
-another, so production is the route model's one FIFO per direction by
-construction, whatever prefixes a side asked for. A change of token scope
-edits the prefix set
-instead of reopening a request. `announced()` in Rust and `routes()` in JS keep their APIs,
-and lite-06 and older and moq-transport are untouched.
-
-## Plan
-
-Decided 2026-11-14 (maintainer, /quest-complete on
-[moq-dev/moq#8768](https://github.com/moq-dev/moq/pull/8768), the m2 triage
-before the cut): ✅ stays in m2, gating lite-07's freeze (rejected: moving the route stream and the
-interest set to lite-08). Without it lite-07 would freeze with one router key per announce
-stream, which the route model doesn't check, and keep the reopen-and-hand-over
-machinery for its lifetime.
-
-Decided 2026-11-10 (maintainer, /quest-plan on lite-07 routing).
-
-Why: today a side's requests ride the streams it opened and its answers ride
-the streams its peer opened, and QUIC orders no stream against another. The
-first count to infinity the route model found is one message overtaking
-another across that split, and the router keys each stream as its own
-neighbour (`route/node.rs`), where the model and `Sim` have one neighbour per
-session. This removes the ordering hazard as a class and makes a
-token-scope change an edit.
-
-Decided 2026-11-12 (maintainer, /quest-plan on the quests still written for
-Babel): ✅ [the switch](https://github.com/moq-dev/moq/pull/8731) to the
-path rule lands first, on today's streams, and this follows (rejected: this
-first, which builds the echo and ROUTE_REQUEST on the new stream and then
-deletes them; one wire change for both, which joins two XL quests). The path
-rule ([epic](/quest/m2/cluster-routes/README.md) Plan item 27) has
-no echo and no route request, and the switch puts the hold answer on the
-publisher half, so a side's routing messages for one prefix already share a
-stream. What is left for this quest is the order between a side's prefixes,
-which still ride a stream each, one neighbour per session in the router,
-and the edit.
-
-Settled:
-
-1. ✅ **One one-way stream per side** (rejected: one bidirectional Announce
- Stream per direction with the echo moved to the answering half, which
- still splits a side's messages over two streams, needs a rule for a peer
- that opened none, and makes the model grow a second queue per direction;
- one bidirectional stream per session opened by the client, which the
- client must open even when it wants nothing).
-2. ✅ **Its interest is a set of prefixes edited in place**, with a floor per
- prefix, SYNCED per prefix, and swaps that keep what both sets cover:
- [the interest set](https://github.com/moq-dev/moq/pull/8728)
- settled and proved those rules, and this puts them on the wire.
-3. ✅ **A token-scope change is an edit.** The subscriber's set is its local
- interests met with its scope's regions, one prefix per region, so a
- multi-region scope is certifiable. Plan items 16 and 17 of the
- [epic](/quest/m2/cluster-routes/README.md) (reopen and replace,
- and the old request's own budget) are superseded, and the overlapping
- reopened requests that could hold three times the announce cap
- ([moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375)'s open
- review finding) cannot occur.
-4. ✅ **A DEAD has a fixed size, and the routes it moves follow it as its own
- ROUTE_UPDATEs** (decided 2026-11-14 by the maintainer on
- [moq-dev/moq#8731](https://github.com/moq-dev/moq/pull/8731)'s decision
- 22, [the comment](https://github.com/moq-dev/moq/pull/8731#issuecomment-6889336680)'s
- B; rejected: A, the parts with a More byte that the switch ships, where
- each part repeats the header and the sender splits by encoded size; C, a
- DEAD that holds every route through the hop with the moves after it as
- ordinary messages, which brings churn on every failure that moves routes
- and changes the model). A DEAD Closed is `Closed {Hop, Far, Gone, Moved
- Count}`, and the next Moved Count messages on the stream are ROUTE_UPDATEs,
- one per moved route. The receiver holds the DEAD until they arrive and
- then applies everything as one step, the route model's `Fail`, so
- `RouteHandoff.tla` and its configs are unchanged. A refused move is a
- refused ROUTE_UPDATE: the route ends, and its path still credits the DEAD.
- These are PROTOCOL_VIOLATIONs, so a DEAD always completes or ends the
- session:
- - any other message between the DEAD and its last update, whichever
- prefix it answers;
- - an update for a route that isn't live or doesn't cross the hop;
- - a count above the stream's routes through the hop.
-
- It replaces the parts in the same PR:
- - the DEAD codec (`rs/moq-net/src/lite/announce.rs`,
- `js/net/src/lite/announce.ts`): Moved Count in place of More and the
- moved list, and `Dead::split` goes;
- - `buffer_dead` (`rs/moq-net/src/lite/publisher/routed.rs`) writes the
- DEAD and then its updates, with nothing between them and no size split;
- - both receivers' reassembly (`Continued` in
- `rs/moq-net/src/lite/subscriber/routed.rs`, `continued` in
- `js/net/src/lite/subscriber.ts`) counts the updates instead of the
- parts;
- - the shared corpus (`rs/moq-net/fixtures/lite.json` and the fuzz seeds in
- `rs/moq-net/src/fuzz.rs`) drops the More cases and pins a DEAD with its
- updates and each violation;
- - the draft's DEAD section and Dead Hops describe the count.
-
- Test that a DEAD and its updates apply as one, in Rust and `js/net`, each
- failing when an update applies on its own, and that each violation
- closes the session.
-
-Recommended wire, for the implementer to confirm or improve in the PR; each
-departure is a choice to list there:
-
-- A unidirectional stream type, Announce. An endpoint opens it when it first
- has something to say, opens at most one, and keeps it for the session. A
- second one, or one that ends before the session, closes the session. No
- reopening means no generation of streams to order against each other.
-- The bidirectional Announce stream leaves lite-07.
-- What a side sends as a subscriber: ANNOUNCE_REQUEST, now an edit (prefix
- ids to remove, then prefixes to add, each with its `Hidden`). What it
- sends as a publisher: one ANNOUNCE_OK per edit it applied, the marker where
- the edit takes effect, then ROUTE_START, UPDATE, HOLD, and END, the
- withdrawal that names a dead hop and the hold answer as
- [the switch](https://github.com/moq-dev/moq/pull/8731) shaped them, the
- ANNOUNCE messages, ANNOUNCE_FLOOR with a Prefix ID, and SYNCED with a
- Prefix ID. ANNOUNCE_START names the Prefix ID its suffix is relative to,
- and a compression base shares it.
-- A side's Route IDs, Announce IDs, and Prefix IDs are one counter each per
- stream. A message names the peer's id where it answers the peer: a Prefix
- ID in ANNOUNCE_START, ANNOUNCE_FLOOR, and SYNCED.
-- A removed prefix may still be named until the ANNOUNCE_OK for the edit that
- removed it, and never after. The subscriber holds a removed prefix's claims
- against its cap until then.
-- The live set may be empty, never nests, and once encoded fits one
- ANNOUNCE_REQUEST (65,535 bytes), as SUBSCRIBE's Ranges do. Each claim is
- then presented once, plus one covering claim per route per prefix, so what
- a peer can make the other hold follows its own request, not a constant.
-- A prefix outside the subscriber's grant lists nothing and is SYNCED at
- once; there is no per-prefix refusal. The local library refuses an
- `announced()` outside its known scope, as it does today.
-
-Code this replaces, for lite-07 only:
-
-- `SubscriberDriver`'s `Tasks` and `Discovery`'s open and
- close of covering prefixes (`rs/moq-net/src/lite/subscriber.rs`,
- `rs/moq-net/src/discovery.rs`) become one stream machine whose edits follow
- `Discovery`. A narrower request no longer stays open under a broader one:
- one edit swaps them.
-- `AnnouncePrefix::reopen`, `replacing`, `Routed::hand_over`, and
- `Slots::split` go, with their tests in `routed.rs` and
- `tests/integration/auth.rs`. Delete each outright if no other version uses
- it.
-- `Routed` (`lite/subscriber/routed.rs`) and `Answer`
- (`lite/publisher/routed.rs`) become one per session, holding the prefixes.
- `Answer` already reads several node streams (one per mount).
-- `Router::admit` (`rs/moq-net/src/route/path.rs`) files a scoped
- neighbour's route that is wider than the scope's regions at each region
- inside it, because a stream for a scope with several regions asks for a
- prefix above them all ([the switch](https://github.com/moq-dev/moq/pull/8731)
- added this). Two routes of one node at nested prefixes then both file at
- a region inside both, so one overwrites the other and an end of either
- drops the route there until the other sends again. With one prefix per
- region in the set, the peer answers at the region and nothing narrows, so
- the narrowing goes. Test a two-region scope behind a peer that holds a
- broad and a narrow route of one node, ending each in turn: the region
- inside both stays online through whichever route still lives, and goes
- offline only once both have ended. Decided 2026-11-14 (maintainer, on
- [moq-dev/moq#8731](https://github.com/moq-dev/moq/pull/8731)'s decision
- 12): ✅ deferred here (rejected: tracking per filed slot which wire routes
- file into it in #8731, state this quest deletes).
-- `route::Node::closing` and the quiet retire of a stream dropped outside
- it (`route/node.rs`), for lite-07: with each side's stream living as long
- as its session, every drop of one is a session close. Today a session's
- receive stream lands under the guard only because `Publisher::poll`
- returns the transport error before `poll_protocol` reaches the
- subscriber, which [#8731](https://github.com/moq-dev/moq/pull/8731)'s
- review flagged as resting on poll order. A guard taken on every poll
- would lock the node on each session wake, so the stream's lifetime is
- the fix.
-- `route::Node` registers one neighbour per session for both directions,
- carrying [the interest set](https://github.com/moq-dev/moq/pull/8728)'s
- edits on it: `Out::Ask`, `Out::Edited` and `edited`, and SYNCED per
- prefix. Today `receive` and `answer` each take a fresh key
- (`route/node.rs`), so one session is two neighbours in the router, and
- each key's one edit is the prefix its stream opened with:
- `State::register` applies it at once and `flush` drops the router's asks
- and markers. A scoped peer's route wider than its regions is filed at each
- region inside it (`path::Router::admit`) rather than asked for per region.
-- `js/net`: each `routes()` call opens its own stream today
- (`js/net/src/lite/subscriber.ts`), so JS gains the covering-prefix logic
- `Discovery` has. Its route layer keeps per-stream state, so this PR changes `js/net`
- too. A floor that changes what a claim covers, with no route change, also
- re-runs admission for the open requests it affects: `forwardAnnounced`
- (`js/net/src/connection/forward.ts`) asks `session.covers` only when a
- request is admitted, so a request can stay on a node that no longer
- covers it until something else refreshes it (flagged in
- [#8731](https://github.com/moq-dev/moq/pull/8731)'s review).
-
-Also touching the same files, to merge around:
-[moq-dev/moq#8458](https://github.com/moq-dev/moq/pull/8458) (its
-per-announce-stream timer becomes per session) and
-[moq-dev/moq#8457](https://github.com/moq-dev/moq/pull/8457) (warm routes on
-the Announce Stream).
-
-Tests: every ordering hazard the old shape had, on real sessions with mocked
-time. A hold answer never overtakes a route, across prefixes too; a swap edit shows no claim offline and
-re-sends nothing both sets cover; a narrowing edit drops only what it
-uncovers; a scope renewal that changes the region keeps every still-covered
-claim shown; a second Announce stream, a nested prefix, and a set over the
-size limit each close the session; a peer that withholds ANNOUNCE_OK or a
-hold answer holds no more than the announce cap. Replace
-`tests/integration/lazy_discovery.rs`'s two-stream cases with edits.
-
-Update the draft's Streams tables, Announce, Floors, Routing, the message
-sections, and the lite-07 changelog in the same PR, and the `verify-route`
-skill's note on `node.rs`'s per-stream keys. Run `just check`,
-`just test interop --all`, and `just drafts check`.
-
-Rollout: lite-07 is `-wip`, so a relay on this wire refuses one on the old
-by misparse, not by version. Relays pinned to `main` roll together, as each
-earlier lite-07 wire change did.
-
-Public API: none expected. Wire: lite-07-wip only.
-
-## Related
-
-- [Announce interest streams share one route table](/quest/m4/announce-interest-memory.md) - lite-07 then holds one interest stream per side; the bound still matters for older versions
-- [A regrant re-checks only the routes it can change](/quest/m4/regrant-route-recheck.md) - its per-stream product loses a factor on lite-07
diff --git a/quest/m2/multicast/switch.md b/quest/m2/multicast/switch.md
index 8984b7e6f4b..48da7e29074 100644
--- a/quest/m2/multicast/switch.md
+++ b/quest/m2/multicast/switch.md
@@ -213,4 +213,3 @@ extension on moq-transport is no longer sent or read.
## Required
- [The router](/quest/m2/multicast/router.md) - the rule these sessions drive
-- [The route stream](/quest/m2/cluster-routes/route-stream.md) - one announce stream per side, which this builds on
diff --git a/quest/m4/announce-interest-memory.md b/quest/m4/announce-interest-memory.md
index 25fe256884d..f5b12e5e8f3 100644
--- a/quest/m4/announce-interest-memory.md
+++ b/quest/m4/announce-interest-memory.md
@@ -27,8 +27,8 @@ count (`moq_tokio::quic::DEFAULT_MAX_STREAMS = 10_000`), so the worst case
is about 10k times N times a few hundred bytes. Incoming announces are
bounded by `session::Limits`; outgoing interest streams are not.
-Once [one route stream](/quest/m2/cluster-routes/route-stream.md)
-lands, lite-07 holds one interest stream per side, so what remains exposed is
+Since [one route stream](https://github.com/moq-dev/moq/pull/8835),
+lite-07 holds one interest stream per side, so what remains exposed is
lite-05/06 and moq-transport.
Decided 2026-11-15 (maintainer, /quest-complete): ✅ a `session::Limits` cap
diff --git a/quest/m4/announce-run-lite07-branches.md b/quest/m4/announce-run-lite07-branches.md
new file mode 100644
index 00000000000..50b806bb640
--- /dev/null
+++ b/quest/m4/announce-run-lite07-branches.md
@@ -0,0 +1,24 @@
+# [S] AnnounceRun drops the lite-07 branches nothing reaches
+
+## Goal
+
+`AnnounceRun` in `rs/moq-net/src/lite/publisher.rs` answers a bidirectional
+Announce stream, which only lite-06 and older open. Its lite-07 branches
+(`has_synced` in the initial set and in `init`, and the encoder paths behind
+them) are gone, and so is whatever in the publisher only they used.
+
+## Plan
+
+From [the route stream](https://github.com/moq-dev/moq/pull/8835), which moved
+lite-07 to one Announce Stream answered by `lite::publisher::routed::Answer`
+and refuses the bidirectional Announce stream there (`ControlType::Announce`
+is unknown when `version.has_routes()`). It deleted the lite-07-only tests of
+`AnnounceRun` and left the branches, to keep its diff on the wire change.
+
+- Delete each branch of `AnnounceRun` and its helpers that only a lite-07
+ version takes, then whatever becomes unused.
+- `js/net`'s `Publisher.runAnnounce` has the same split: delete the lite-07
+ paths the Announce Stream's `Answer` replaced.
+- No behaviour changes: every existing test passes untouched.
+
+Public API: none. Wire: none.
diff --git a/quest/m4/js-announce-edit-deadline.md b/quest/m4/js-announce-edit-deadline.md
new file mode 100644
index 00000000000..79234f8fcab
--- /dev/null
+++ b/quest/m4/js-announce-edit-deadline.md
@@ -0,0 +1,27 @@
+# [S] js/net closes a session whose Announce Stream stalls
+
+## Goal
+
+A lite-07 `js/net` session whose peer stops reading its Announce Stream closes
+after 10 seconds without write progress while messages wait, as the Rust session
+does and the draft requires, and a test on mocked time shows it.
+
+## Plan
+
+From [the route stream](https://github.com/moq-dev/moq/pull/8835). The draft
+closes a session whose Announce Stream made no progress for 10 seconds with
+messages left to send. Rust arms `Stall` for `route::path::HOLD` only while a
+flush is pending (`rs/moq-net/src/lite/subscriber/route_stream.rs`), closing with
+`announce stream stalled`; `js/net/src/lite/announce_stream.ts` waits forever.
+
+- Mirror the Rust write-stall deadline and its close code in `js/net`.
+- An edit the peer reads but never answers with ANNOUNCE_OK has no deadline in
+ Rust, `js/net`, or the draft. Recommendation: leave it so. Withholding the
+ answer stalls only that session's discovery, which the peer can already do by
+ sending nothing, and a timer per edit buys no behavior a well-behaved peer
+ needs.
+- Decide with the same change whether the Announce Stream opens eagerly, as
+ Rust does, or with its first message, as `js/net` does today, and make the
+ two match if a reason to differ is not found.
+
+Public API: none. Wire: none.
diff --git a/quest/m4/regrant-route-recheck.md b/quest/m4/regrant-route-recheck.md
index e6bca73f85f..91aab8cfd61 100644
--- a/quest/m4/regrant-route-recheck.md
+++ b/quest/m4/regrant-route-recheck.md
@@ -15,7 +15,7 @@ So one `AUTH_OK` costs routes × members per stream, times K announce streams, a
Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): its own quest beside the grant-algebra bound.
-Re-scoped 2026-11-12 (/quest-audit): [Each side says everything about routing on one stream](/quest/m2/cluster-routes/route-stream.md) gives a lite-07 side one Announce Stream and turns a token-scope change into an edit of its interest set, so on lite-07 the factor K becomes one and the per-stream `regrant` this plan targets remains only for lite-06 and older. Start after it lands: re-measure the regrant (or interest-set edit) cost on lite-07 with the benches below, and apply the steps that follow to whichever path still pays routes × members. If only the older versions still pay it, say so in the PR and leave them.
+Re-scoped 2026-11-12 (/quest-audit): [Each side says everything about routing on one stream](https://github.com/moq-dev/moq/pull/8835) gives a lite-07 side one Announce Stream and turns a token-scope change into an edit of its interest set, so on lite-07 the factor K becomes one and the per-stream `regrant` this plan targets remains only for lite-06 and older. Re-measure the regrant (or interest-set edit) cost on lite-07 with the benches below, and apply the steps that follow to whichever path still pays routes × members. If only the older versions still pay it, say so in the PR and leave them.
- Derive the change once per `AUTH_OK`, not per stream: the members added and removed between the old and new grant. Each stream then re-checks only the routes under a changed member's prefix, using the origin's prefix structure to find them instead of walking every route. A member's prefix here is its literal segments before its first wildcard (`*` or `**`), since grant members are `Patterns` (`Permit` in `rs/moq-net/src/auth.rs`); a member that starts with a wildcard has no literal prefix and re-checks every route under the stream's prefix, as today. Bench that shape on its own row, and test wildcard members, removed members, and old and new grants that overlap.
- A grown grant needs only routes under the added members, not a whole-origin snapshot per stream.
@@ -24,10 +24,6 @@ Re-scoped 2026-11-12 (/quest-audit): [Each side says everything about routing on
Public API: none. Wire: none.
-## Required
-
-- [Each side says everything about routing on one stream](/quest/m2/cluster-routes/route-stream.md) - replaces lite-07's per-stream regrant with one interest-set edit per side, which this re-scopes to
-
## Related
- [Bound peer grant processing by design](/quest/m5/peer-grant-bound.md) - may change the received-grant type this compares against; whichever lands second adapts
diff --git a/rs/moq-mux/src/container/source.rs b/rs/moq-mux/src/container/source.rs
index b587746e209..d9061a9c8a5 100644
--- a/rs/moq-mux/src/container/source.rs
+++ b/rs/moq-mux/src/container/source.rs
@@ -1304,14 +1304,15 @@ mod tests {
for relay in [false, true] {
let (publisher, subscriber) = (origin(1), origin(3));
- // SETUP takes the first uni stream; the second is the backlog's oldest group.
+ // SETUP and the Announce Stream take the first two uni streams; the third is
+ // the backlog's oldest group.
let (held, _downstream) = match relay {
true => {
let middle = origin(2);
- let held = Link::held(&publisher, &middle, 2).await;
+ let held = Link::held(&publisher, &middle, 3).await;
(held, Some(Link::new(&middle, &subscriber).await))
}
- false => (Link::held(&publisher, &subscriber, 2).await, None),
+ false => (Link::held(&publisher, &subscriber, 3).await, None),
};
let broadcast = publisher
diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs
index 68a0d8e4502..e0439ac10ef 100644
--- a/rs/moq-mux/src/container/test_util.rs
+++ b/rs/moq-mux/src/container/test_util.rs
@@ -387,7 +387,7 @@ impl Link {
/// Link `from` into `to`, which accepts only its first `uni` uni streams until
/// [`open`](Self::open), as a network can deliver one group stream well ahead of the
- /// rest. SETUP takes the first.
+ /// rest. SETUP and the Announce Stream take the first two.
pub(crate) async fn held(from: &moq_net::origin::Producer, to: &moq_net::origin::Producer, uni: usize) -> Self {
use moq_net::transport::memory;
diff --git a/rs/moq-net/fixtures/lite.json b/rs/moq-net/fixtures/lite.json
index cfe9a585d8d..43e5fec6977 100644
--- a/rs/moq-net/fixtures/lite.json
+++ b/rs/moq-net/fixtures/lite.json
@@ -45,77 +45,98 @@
"name": "a Route Offer counting more hops than its body holds is malformed, though a count past the hop limit is otherwise refused as content",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0649070083e80100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019001a001b001c001d001e001f0020002100008001",
+ "hex": "064b0700000083e80100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019001a001b001c001d001e001f0020002100008001",
"expect": "refused"
},
{
- "name": "a ROUTE_START names its publisher, suffix, and offer: a path of node and session hops, then flags, priority, and metric",
+ "name": "a ROUTE_START names its publisher, Prefix ID, suffix, whether the route is exact, and offer: a path of node and session hops, then flags, priority, and metric",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "060b9234000207000903018005",
+ "hex": "060d92340000000207000903018005",
"expect": "decoded"
},
{
"name": "a ROUTE_START's suffix extends the requested prefix",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "060a9234016101090000ff00",
+ "hex": "060c92340001610001090000ff00",
"expect": "decoded"
},
+ {
+ "name": "a ROUTE_START's Exact marks a route its sender keeps at its own prefix",
+ "version": "moq-lite-07-wip",
+ "message": "announce",
+ "hex": "060d92340000010207000903018005",
+ "expect": "decoded"
+ },
+ {
+ "name": "a ROUTE_START's Exact marks a route below the requested prefix too",
+ "version": "moq-lite-07-wip",
+ "message": "announce",
+ "hex": "060c92340001610101090000ff00",
+ "expect": "decoded"
+ },
+ {
+ "name": "a ROUTE_START's Exact other than 0 or 1 is malformed",
+ "version": "moq-lite-07-wip",
+ "message": "announce",
+ "hex": "060d92340000020207000903018005",
+ "expect": "refused"
+ },
{
"name": "a Route Offer's 0x2 flag marks a path through a draining node",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0609923400010900027f03",
+ "hex": "060b9234000000010900027f03",
"expect": "decoded"
},
{
"name": "a Route Offer flag no version defines is withdrawn, with whatever field it adds",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "060a9234000109000880032a",
+ "hex": "060c92340000000109000880032a",
"expect": "withdrawn"
},
{
"name": "a Route Offer with more than 32 hops is withdrawn",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0649923400210100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019001a001b001c001d001e001f0020002100008001",
+ "hex": "064b9234000000210100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019001a001b001c001d001e001f0020002100008001",
"expect": "withdrawn"
},
{
"name": "a Route Offer with no hops is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "060792340000008001",
+ "hex": "0609923400000000008001",
"expect": "refused"
},
{
"name": "a Route Offer missing a hop it counts is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0609923400030700008003",
+ "hex": "060b9234000000030700008003",
"expect": "refused"
},
{
"name": "a ROUTE_START naming Node ID 0 is withdrawn",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "06080000010700008001",
+ "hex": "060a00000000010700008001",
"expect": "withdrawn"
},
{
"name": "a Route Offer hop naming Node ID 0 is withdrawn",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0609923400010000008001",
+ "hex": "060b9234000000010000008001",
"expect": "withdrawn"
},
{
"name": "a ROUTE_START suffix no path spells is withdrawn",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "060c9234032f2f78010700008001",
+ "hex": "060e923400032f2f7800010700008001",
"expect": "withdrawn"
},
{
@@ -147,67 +168,81 @@
"expect": "decoded"
},
{
- "name": "a DEAD CLOSED names the hop, the far node, Gone, More, and the routes that moved, each offer sized",
+ "name": "a DEAD CLOSED names the hop, the far node, Gone, and how many ROUTE_UPDATEs follow",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f110007020901000100080207000501008003",
+ "hex": "0f06000702090102",
"expect": "decoded"
},
{
- "name": "a DEAD CLOSED with More set continues in the next DEAD",
+ "name": "a DEAD DRAIN names the node that sent GOAWAY",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f0700070209000100",
+ "hex": "0f020107",
"expect": "decoded"
},
{
- "name": "a DEAD CLOSED moved offer this node refuses is withdrawn, and the entries after it still read",
+ "name": "a DEAD of a kind no version defines is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f1900070209010002000802070005010880030306010700008001",
- "expect": "withdrawn"
+ "hex": "0f020207",
+ "expect": "refused"
},
{
- "name": "a DEAD CLOSED moved offer short of its size is refused",
+ "name": "a DEAD whose Gone is neither 0 nor 1 is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f110007020901000100080107000080030000",
+ "hex": "0f06000702090200",
"expect": "refused"
},
{
- "name": "a DEAD DRAIN names the node that sent GOAWAY",
+ "name": "a DEAD naming Far Node ID 0 is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f020107",
+ "hex": "0f06000702000000",
+ "expect": "refused"
+ },
+ {
+ "name": "an ANNOUNCE_REQUEST continues its edit, removes Prefix IDs, and adds prefixes, each with its Hidden",
+ "version": "moq-lite-07-wip",
+ "message": "announce",
+ "hex": "100d010200070204726f6f6d010000",
"expect": "decoded"
},
{
- "name": "a DEAD of a kind no version defines is refused",
+ "name": "an ANNOUNCE_REQUEST ending its edit may be empty",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f020207",
- "expect": "refused"
+ "hex": "1003000000",
+ "expect": "decoded"
},
{
- "name": "a DEAD whose Gone is neither 0 nor 1 is refused",
+ "name": "an ANNOUNCE_REQUEST whose More is neither 0 nor 1 is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f0700070209020000",
+ "hex": "1003020000",
"expect": "refused"
},
{
- "name": "a DEAD whose More is neither 0 nor 1 is refused",
+ "name": "an ANNOUNCE_REQUEST whose Hidden is neither 0 nor 1 is refused",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f0700070209000200",
+ "hex": "10050000010002",
"expect": "refused"
},
{
- "name": "a DEAD naming Far Node ID 0 is refused",
+ "name": "an ANNOUNCE_OK is empty",
"version": "moq-lite-07-wip",
"message": "announce",
- "hex": "0f06000702000000",
- "expect": "refused"
+ "hex": "1100",
+ "expect": "decoded"
+ },
+ {
+ "name": "a SYNCED names its Prefix ID",
+ "version": "moq-lite-07-wip",
+ "message": "announce",
+ "hex": "040103",
+ "expect": "decoded"
},
{
"name": "an ANNOUNCE_FLOOR carries its Revision plus one",
diff --git a/rs/moq-net/src/auth.rs b/rs/moq-net/src/auth.rs
index 89980cdbdeb..b88b3830f8c 100644
--- a/rs/moq-net/src/auth.rs
+++ b/rs/moq-net/src/auth.rs
@@ -620,33 +620,17 @@ impl Permit {
Some(crate::model::interest_prefixes(limit))
}
- /// The one prefix the grant and the limit together confine this direction to, or
- /// `None` when they confine it to several or to none. A request for it, rather than
- /// one above it, is one the peer's floors can certify.
+ /// The prefixes the peer's grant confines this direction to, or `None` while it
+ /// grants everything: asking for one of them, rather than a prefix above it, is a
+ /// request the peer's floors can certify.
#[cfg(feature = "async")]
- pub(crate) fn region(&self) -> Option {
- let heads = |patterns: &Option| patterns.as_ref().map(crate::model::interest_prefixes);
- let regions = match (heads(&self.granted), heads(&self.limit)) {
- (None, None) => return None,
- (Some(regions), None) | (None, Some(regions)) => regions,
- (Some(granted), Some(limit)) => {
- let mut meet: Vec = (granted.iter())
- .flat_map(|a| limit.iter().map(move |b| (a, b)))
- .filter_map(|(a, b)| match () {
- _ if a.has_prefix(b) => Some(a.clone()),
- _ if b.has_prefix(a) => Some(b.clone()),
- _ => None,
- })
- .collect();
- meet.sort();
- meet.dedup();
- meet
- }
- };
- match regions.as_slice() {
- [region] => Some(region.clone()),
- _ => None,
+ pub(crate) fn granted_regions(&self) -> Option> {
+ let granted = self.granted.as_ref()?;
+ let everything = moq_pattern::Pattern::subtree("").ok()?;
+ if granted.contains(&everything) {
+ return None;
}
+ Some(crate::model::interest_prefixes(granted))
}
/// The limit this side set on the peer, or `None` while unrestricted.
@@ -3102,22 +3086,27 @@ mod tests {
assert_eq!(everything.regions(), None, "a root grant confines nothing");
}
- /// The region is where the grant and the limit meet, and only when that is one prefix.
+ /// What the peer granted confines the prefixes asked of it, whatever the limit is.
#[cfg(feature = "async")]
#[test]
- fn the_region_is_where_grant_and_limit_meet() {
+ fn the_granted_regions_follow_the_grant_alone() {
let permit = |granted: Option<&[&str]>, limit: Option<&[&str]>| Permit {
granted: granted.map(patterns),
limit: limit.map(patterns),
};
let path = crate::Path::from_static;
- assert_eq!(permit(None, None).region(), None);
- assert_eq!(permit(Some(&["a"]), None).region(), Some(path("a")));
- assert_eq!(permit(None, Some(&["a/b"])).region(), Some(path("a/b")));
- assert_eq!(permit(Some(&["a"]), Some(&["a/b", "c"])).region(), Some(path("a/b")));
- assert_eq!(permit(Some(&[""]), Some(&["a"])).region(), Some(path("a")));
- assert_eq!(permit(Some(&["a", "b"]), None).region(), None, "two regions");
- assert_eq!(permit(Some(&["a"]), Some(&["b"])).region(), None, "they never meet");
+ assert_eq!(permit(None, None).granted_regions(), None);
+ assert_eq!(permit(None, Some(&["a/b"])).granted_regions(), None);
+ assert_eq!(
+ permit(Some(&[""]), Some(&["a"])).granted_regions(),
+ None,
+ "a root grant"
+ );
+ assert_eq!(permit(Some(&["a"]), None).granted_regions(), Some(vec![path("a")]));
+ assert_eq!(
+ permit(Some(&["a", "b"]), Some(&["c"])).granted_regions(),
+ Some(vec![path("a"), path("b")])
+ );
}
/// A limit replaces the last one, narrower or wider, and applies to a version
diff --git a/rs/moq-net/src/coding/writer.rs b/rs/moq-net/src/coding/writer.rs
index 9236294e349..19753190b77 100644
--- a/rs/moq-net/src/coding/writer.rs
+++ b/rs/moq-net/src/coding/writer.rs
@@ -908,7 +908,7 @@ mod tests {
starts: vec![Some(3)],
})
.unwrap();
- writer.buffer(&lite::AnnounceBroadcast::Synced).unwrap();
+ writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 }).unwrap();
assert_eq!(
stats.snapshot().control().len(),
4,
diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs
index 39fb2b122e6..cf477a49358 100644
--- a/rs/moq-net/src/fuzz.rs
+++ b/rs/moq-net/src/fuzz.rs
@@ -600,7 +600,7 @@ pub struct RouteNode {
}
impl RouteNode {
- /// A node holding `claims` claims of its own under `live/`, answering one stream
+ /// A node holding `claims` claims of its own under `live/`, answering one session
/// per prefix in `streams`, each already sent its snapshot.
pub fn new(claims: usize, streams: &[&str]) -> Self {
let node = crate::route::Node::new(1, 1, route_clock());
@@ -612,7 +612,9 @@ impl RouteNode {
.map(|(i, prefix)| {
let link = crate::route::Link::default();
let peer = route_peer(i as u64 + 1, i as u64 + 2);
- node.answer(peer, link, Path::new(prefix).unwrap()).unwrap()
+ let stream = node.connect(peer, link).unwrap();
+ stream.edit(route_ask(Path::new(prefix).unwrap())).unwrap();
+ stream
})
.collect();
let node = Self {
@@ -678,7 +680,9 @@ impl RouteRelay {
let answer = (0..answers as u64)
.map(|i| {
let peer = route_peer(i + 1, i + 3);
- node.answer(peer, crate::route::Link::default(), Path::empty()).unwrap()
+ let stream = node.connect(peer, crate::route::Link::default()).unwrap();
+ stream.edit(route_ask(Path::empty())).unwrap();
+ stream
})
.collect();
let instance = crate::Instance::random();
@@ -686,7 +690,9 @@ impl RouteRelay {
let receive = (0..streams as u64)
.map(|i| {
let peer = route_peer(answers as u64 + i + 1, Self::ORIGIN);
- let receive = node.receive(peer, crate::route::Link::default()).unwrap();
+ let receive = node.connect(peer, crate::route::Link::default()).unwrap();
+ // The peer answered what the node asks it for.
+ receive.edited(Default::default());
let reach = crate::route::path::Reach::Finite(route_offer(Self::ORIGIN));
receive.route(Self::ORIGIN, Path::empty(), Some(reach));
let snapshot = (0..claims)
@@ -848,11 +854,17 @@ impl RouteFollower {
follower
}
- /// Open the stream for prefix `k`, which hears the origin's route and snapshot.
+ /// Open the session for prefix `k`, which hears the origin's route and snapshot.
fn follow(&self, k: usize) -> crate::route::node::Stream {
let prefix = Path::new(&format!("live/{k}")).unwrap();
- let peer = route_peer(Self::TAG, Self::ORIGIN);
- let stream = self.node.receive(peer, crate::route::Link::default()).unwrap();
+ let peer = route_peer(Self::TAG + k as u64, Self::ORIGIN);
+ let link = crate::route::Link {
+ interest: [prefix.clone()].into(),
+ ..Default::default()
+ };
+ let stream = self.node.connect(peer, link).unwrap();
+ // The peer answered what the node asks it for.
+ stream.edited(Default::default());
let reach = crate::route::path::Reach::Finite(route_offer(Self::ORIGIN));
stream.route(Self::ORIGIN, prefix.clone(), Some(reach));
let claim = crate::route::node::Claim {
@@ -871,7 +883,7 @@ impl RouteFollower {
stream
}
- /// Close one prefix's stream and open it again, returning how many outputs the
+ /// Close one prefix's session and open it again, returning how many outputs the
/// node queued.
pub fn refollow(&mut self) -> usize {
let k = self.next % self.streams.len();
@@ -882,9 +894,8 @@ impl RouteFollower {
self.drain()
}
- /// Close every stream at once, as a session ending does.
+ /// Close every session.
pub fn teardown(self) {
- let _closing = self.node.closing(Self::TAG);
drop(self.streams);
}
@@ -895,6 +906,14 @@ impl RouteFollower {
}
}
+/// A peer's request for `prefix` alone.
+fn route_ask(prefix: Path) -> crate::route::path::Edit {
+ crate::route::path::Edit {
+ remove: Vec::new(),
+ add: vec![prefix],
+ }
+}
+
/// The path router originating claims and answering streams, for the routes bench.
pub struct PathRouteNode(crate::route::path::Router);
@@ -2087,23 +2106,25 @@ pub fn pattern(data: &[u8]) -> bool {
}
/// Lite seeds whose envelope bytes the uniform sweep never forms, on the newest
-/// version: a typed empty TRACKS_SYNCED, an AUTH_OK reply, every route message and
-/// both kinds of DEAD (the sweep sets ROUTE_START's type byte but fills its size byte
-/// with the body pattern, so none decodes whole), and a DATAGRAMS request, whose eight
-/// fields outgrow the sweep's longest body.
+/// version: a typed empty TRACKS_SYNCED, an AUTH_OK reply, every route message,
+/// both kinds of DEAD, and an ANNOUNCE_REQUEST and its ANNOUNCE_OK (the sweep sets
+/// ROUTE_START's type byte but fills its size byte with the body pattern, so none
+/// decodes whole), and a DATAGRAMS request, whose eight fields outgrow the sweep's
+/// longest body.
const TYPED_LITE_SEEDS: &[(u8, &[u8])] = &[
(23, &[4, 0]),
(30, &[8, 0, 0, 0, 0, 0, 0, 0, 0]),
(27, &[0, 3, 0, 0, 0]),
- (5, &[6, 8, 1, 0, 1, 1, 0, 0, 0x80, 1]),
+ (5, &[6, 9, 1, 0, 0, 1, 1, 0, 0, 0x80, 1]),
(5, &[7, 7, 0, 1, 1, 0, 0, 0x80, 1]),
(5, &[8, 1, 0]),
(5, &[9, 2, 0, 1]),
(5, &[11, 1, 0]),
- (5, &[15, 7, 0, 1, 1, 2, 0, 0, 0]),
- (5, &[15, 7, 0, 1, 1, 2, 0, 1, 0]),
- (5, &[15, 15, 0, 1, 1, 2, 1, 0, 1, 0, 6, 1, 1, 1, 0, 0x80, 1]),
+ (5, &[15, 6, 0, 1, 1, 2, 0, 0]),
+ (5, &[15, 6, 0, 1, 1, 2, 1, 2]),
(5, &[15, 2, 1, 1]),
+ (5, &[16, 4, 0, 1, 0, 0]),
+ (5, &[17, 0]),
];
/// The inputs the fuzzer starts from: every (version, type) pair the dispatch knows,
diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs
index e62d9bcc764..15c87aafb41 100644
--- a/rs/moq-net/src/ietf/publisher.rs
+++ b/rs/moq-net/src/ietf/publisher.rs
@@ -12998,14 +12998,18 @@ mod serve_tests {
.await
.expect("no subscription")
.unwrap();
- for expected in [0, 1] {
+ // Read in arrival order, which the hop's streams do not fix.
+ let mut warmed = Vec::new();
+ for _ in 0..2 {
let group = kio::clock::timeout(TIMEOUT, reader.recv_group())
.await
.expect("no group")
.unwrap()
.expect("track finished");
- assert_eq!(group.start, expected, "{version} {filter:?}");
+ warmed.push(group.start);
}
+ warmed.sort_unstable();
+ assert_eq!(warmed, [0, 1], "{version} {filter:?}");
let session = ScriptedSession::per_stream(vec![Vec::new()]);
let peer_setup = peer::PeerSetup::default();
diff --git a/rs/moq-net/src/lite/announce.rs b/rs/moq-net/src/lite/announce.rs
index d2172d43474..ddae3c55039 100644
--- a/rs/moq-net/src/lite/announce.rs
+++ b/rs/moq-net/src/lite/announce.rs
@@ -33,6 +33,9 @@ const ROUTE_HOLD: u64 = 11;
// with an implementation that picks a source per broadcast.
// lite-07: news of a closed session or a draining node on the routes the sender offered.
const DEAD: u64 = 15;
+// lite-07: an edit of the prefixes the sender wants, and the marker answering one.
+const ANNOUNCE_REQUEST: u64 = 16;
+const ANNOUNCE_OK: u64 = 17;
/// DEAD kind: a session on the routes' paths closed.
const DEAD_CLOSED: u64 = 0;
@@ -52,7 +55,11 @@ pub fn update_supported(version: Version) -> bool {
)
}
-/// An announcement on the Announce Stream, advertising or retracting a broadcast.
+/// A message on the Announce Stream, advertising or retracting a broadcast.
+///
+/// Lite-07 carries everything one side says about routing on its one unidirectional
+/// stream: the edits of the prefixes it wants (`Request`), and the markers (`Ok`),
+/// routes, and claims answering the prefixes its peer wants.
///
/// On lite-06+ these are independently-typed messages (`ANNOUNCE_START`,
/// `ANNOUNCE_END`, `ANNOUNCE_UPDATE`, and on lite-07 `ANNOUNCE_RESTART`), each
@@ -114,13 +121,31 @@ pub enum AnnounceBroadcast {
/// An unknown lite-06+ announce type. The length-prefixed body was skipped so
/// the stream stays up; it does not assign an announce id.
Skipped,
- /// The initial set has been delivered (lite-07+).
- Synced,
+ /// SYNCED (lite-07+): the initial set of the prefix the receiver assigned this Prefix
+ /// ID has been delivered.
+ Synced { prefix: u64 },
+ /// ANNOUNCE_REQUEST (lite-07+): the sender edits the prefixes it wants claims for,
+ /// removing those with the Prefix IDs in `remove` and then adding each of `add`,
+ /// with whether it also wants hidden claims there, which assigns it the next Prefix
+ /// ID. With `more`, the edit continues in the next message, and takes effect whole
+ /// with its last (see [`request_parts`]).
+ Request {
+ more: bool,
+ remove: Vec,
+ add: Vec<(Path, bool)>,
+ },
+ /// ANNOUNCE_OK (lite-07+): the receiver's last edit takes effect here.
+ Ok,
/// ROUTE_START (lite-07+): the sender has a route to the publisher `node` covering
- /// the requested prefix joined with `suffix`, and assigns it the next Route ID.
+ /// the prefix the receiver assigned the Prefix ID `prefix` joined with `suffix`, and
+ /// assigns it the next Route ID. `exact` marks a route the sender keeps at its own
+ /// prefix whatever prefix the request lies at, so an edit widening the request past
+ /// it leaves it there.
RouteStart {
node: route::Id,
+ prefix: u64,
suffix: Path,
+ exact: bool,
offer: Offer,
},
/// ROUTE_UPDATE (lite-07+): the route with this id changed, and is usable.
@@ -149,11 +174,13 @@ pub enum AnnounceBroadcast {
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) enum Refusable {
/// ROUTE_START: assigns the next Route ID to the publisher `node`, covering the
- /// requested prefix joined with `suffix`, or no prefix when the suffix itself was
- /// refused, along `path`.
+ /// prefix with the Prefix ID `prefix` joined with `suffix`, or no prefix when the
+ /// suffix itself was refused, along `path`, at its own prefix when `exact`.
RouteStart {
node: route::Id,
+ prefix: u64,
suffix: Option,
+ exact: bool,
path: Arc<[route::path::Hop]>,
},
/// ROUTE_UPDATE of the route with this Route ID, now along `path`.
@@ -201,17 +228,16 @@ const DRAINING: u8 = 0x2;
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) enum Dead {
/// CLOSED: the session `hop` names closed, at the node `far` at its other end, which
- /// the receiver doubts. Each route on the stream through that hop that `moved` names
- /// by Route ID continues on its offer there; the rest end if `gone`, and are held if
- /// not, unless the receiver is the node just before the sender on the route's path.
- /// With `more`, the next message is a DEAD naming the same hop, far node, and `gone`
- /// whose `moved` continues this one's, and the receiver applies them as one.
+ /// the receiver doubts. The next `moved` messages on the stream are ROUTE_UPDATEs,
+ /// one per route on the stream through that hop that continues on another path, and
+ /// the receiver applies the DEAD and all of them as one. The rest end if `gone`, and
+ /// are held if not, unless the receiver is the node just before the sender on the
+ /// route's path.
Closed {
hop: route::path::Hop,
far: route::Id,
gone: bool,
- more: bool,
- moved: Vec<(u64, Heard)>,
+ moved: u64,
},
/// DRAIN: `node` sent GOAWAY, so every path through it ranks as draining.
Drain { node: route::Id },
@@ -310,13 +336,7 @@ fn decode_offer(r: &mut Decoder<'_>) -> Result {
fn encode_dead(w: &mut Encoder<'_>, dead: &Dead) -> Result<(), EncodeError> {
match dead {
- Dead::Closed {
- hop,
- far,
- gone,
- more,
- moved,
- } => {
+ Dead::Closed { hop, far, gone, moved } => {
if hop.node == 0 || *far == 0 {
return Err(EncodeError::InvalidState);
}
@@ -325,11 +345,7 @@ fn encode_dead(w: &mut Encoder<'_>, dead: &Dead) -> Result<(), EncodeError> {
w.varint(hop.session)?;
w.varint(*far)?;
w.u8(u8::from(*gone));
- w.u8(u8::from(*more));
- w.varint(moved.len() as u64)?;
- for (id, offer) in moved {
- encode_moved(w, *id, offer)?;
- }
+ w.varint(*moved)?;
}
Dead::Drain { node } => {
if *node == 0 {
@@ -342,66 +358,6 @@ fn encode_dead(w: &mut Encoder<'_>, dead: &Dead) -> Result<(), EncodeError> {
Ok(())
}
-fn encode_moved(w: &mut Encoder<'_>, id: u64, offer: &Heard) -> Result<(), EncodeError> {
- // A refused offer is decode-only: this node never forwards one.
- let Ok(offer) = offer else {
- return Err(EncodeError::Unsupported);
- };
- w.varint(id)?;
- // Sized, so a receiver refusing one offer still reads the next.
- let prefix = w.prefix_varint();
- encode_offer(w, offer)?;
- w.fill(prefix)?;
- Ok(())
-}
-
-impl Dead {
- /// Split into consecutive DEADs that each fit one message, in order, every one but
- /// the last with `more` set, so the receiver applies them as this one.
- pub(crate) fn split(self) -> Vec {
- let Self::Closed {
- hop,
- far,
- gone,
- more,
- moved,
- } = self
- else {
- return vec![self];
- };
- let form = Version::Lite07.into();
- let size = |encode: &dyn Fn(&mut Encoder<'_>) -> Result<(), EncodeError>| {
- let mut body = Vec::new();
- // What cannot encode fails the DEAD when it is sent instead.
- let _ = encode(&mut Encoder::new(&mut body, form));
- body.len()
- };
- let part = |more, moved| Self::Closed {
- hop,
- far,
- gone,
- more,
- moved,
- };
- // A DEAD without its moved routes, its Moved Count grown to the widest varint.
- let bare = size(&|w| encode_dead(w, &part(true, Vec::new()))) + 7;
- let mut parts = Vec::new();
- let mut listed = Vec::new();
- let mut used = bare;
- for (id, offer) in moved {
- let entry = size(&|w| encode_moved(w, id, &offer));
- if !listed.is_empty() && used + entry > MAX_MESSAGE_SIZE {
- parts.push(part(true, std::mem::take(&mut listed)));
- used = bare;
- }
- used += entry;
- listed.push((id, offer));
- }
- parts.push(part(more, listed));
- parts
- }
-}
-
fn decode_dead(r: &mut Decoder<'_>) -> Result {
let nonzero = |id: u64| match id {
0 => Err(DecodeError::InvalidValue),
@@ -414,38 +370,13 @@ fn decode_dead(r: &mut Decoder<'_>) -> Result {
session: r.varint()?,
};
let far = nonzero(r.varint()?)?;
- let flag = |byte| match byte {
- 0 => Ok(false),
- 1 => Ok(true),
- _ => Err(DecodeError::InvalidValue),
+ let gone = match r.u8()? {
+ 0 => false,
+ 1 => true,
+ _ => return Err(DecodeError::InvalidValue),
};
- let gone = flag(r.u8()?)?;
- let more = flag(r.u8()?)?;
- let count = r.varint()?;
- // Each entry takes at least a byte per field, so the body bounds the count.
- if count > r.remaining() as u64 {
- return Err(DecodeError::TooMany);
- }
- let moved = (0..count)
- .map(|_| {
- let id = r.varint()?;
- let size = usize::try_from(r.varint()?).map_err(|_| DecodeError::TooMany)?;
- let mut offer = r.sub(size)?;
- let heard = decode_offer(&mut offer)?;
- // A usable offer fills its size; a refused one was skipped to its end.
- if !offer.is_empty() {
- return Err(DecodeError::Long);
- }
- Ok((id, heard))
- })
- .collect::>()?;
- Dead::Closed {
- hop,
- far,
- gone,
- more,
- moved,
- }
+ let moved = r.varint()?;
+ Dead::Closed { hop, far, gone, moved }
}
DEAD_DRAIN => Dead::Drain {
node: nonzero(r.varint()?)?,
@@ -490,6 +421,124 @@ fn decode_floor(r: &mut Decoder<'_>) -> Result {
Ok(AnnounceBroadcast::Floor { id, revision })
}
+/// Encode an ANNOUNCE_REQUEST body: the Prefix IDs to remove, then the prefixes to add,
+/// each with its `Hidden`.
+fn encode_request(
+ w: &mut Encoder<'_>,
+ version: Version,
+ more: bool,
+ remove: &[u64],
+ add: &[(Path, bool)],
+) -> Result<(), EncodeError> {
+ w.bool(more);
+ w.varint(remove.len() as u64)?;
+ for id in remove {
+ w.varint(*id)?;
+ }
+ w.varint(add.len() as u64)?;
+ for (prefix, hidden) in add {
+ prefix.encode(w, version)?;
+ w.bool(*hidden);
+ }
+ Ok(())
+}
+
+/// The bytes `value` takes as a varint of `version`.
+fn varint_size(value: u64, version: Version) -> usize {
+ crate::coding::varint::size(value, version.into()).unwrap_or(usize::MAX)
+}
+
+/// The bytes one Add of `prefix` takes: its length, its bytes, and its flag.
+fn add_size(prefix: &Path, version: Version) -> usize {
+ let len = prefix.as_str().len();
+ varint_size(len as u64, version).saturating_add(len + 1)
+}
+
+/// The bytes an ANNOUNCE_REQUEST body takes with `removes` Prefix IDs totalling `ids`
+/// bytes and `adds` Adds totalling `added` bytes.
+fn request_size(removes: usize, ids: usize, adds: usize, added: usize, version: Version) -> usize {
+ (1 + varint_size(removes as u64, version) + varint_size(adds as u64, version))
+ .saturating_add(ids)
+ .saturating_add(added)
+}
+
+/// Whether the prefixes `set` fit the Adds of one ANNOUNCE_REQUEST, as a subscriber's
+/// whole set must.
+pub(crate) fn request_fits<'a>(set: impl Iterator, version: Version) -> bool {
+ let mut count = 0;
+ let mut bytes = 0usize;
+ for prefix in set {
+ count += 1;
+ bytes = bytes.saturating_add(add_size(prefix, version));
+ }
+ request_size(0, 0, count, bytes, version) <= super::message::MAX_MESSAGE_SIZE
+}
+
+/// The ANNOUNCE_REQUESTs of one edit removing the Prefix IDs `remove` and adding `add`:
+/// as many as it takes for each to fit one message, every one but the last with `more`.
+/// The receiver applies them together at the last, so how they split changes nothing.
+pub(crate) fn request_parts(remove: Vec, add: Vec<(Path, bool)>, version: Version) -> Vec {
+ let mut parts = Vec::new();
+ let mut part = (Vec::new(), Vec::new());
+ // The removes' count and bytes, then the adds'.
+ let mut size = (0, 0);
+ let fits = |part: &(Vec, Vec<(Path, bool)>), size: (usize, usize), more: (usize, usize)| {
+ let (removes, adds) = (
+ part.0.len() + usize::from(more.0 > 0),
+ part.1.len() + usize::from(more.1 > 0),
+ );
+ request_size(removes, size.0 + more.0, adds, size.1 + more.1, version) <= super::message::MAX_MESSAGE_SIZE
+ };
+ for id in remove {
+ let bytes = varint_size(id, version);
+ if !fits(&part, size, (bytes, 0)) {
+ parts.push(std::mem::take(&mut part));
+ size = (0, 0);
+ }
+ part.0.push(id);
+ size.0 += bytes;
+ }
+ for (prefix, hidden) in add {
+ let bytes = add_size(&prefix, version);
+ if !fits(&part, size, (0, bytes)) {
+ parts.push(std::mem::take(&mut part));
+ size = (0, 0);
+ }
+ part.1.push((prefix, hidden));
+ size.1 += bytes;
+ }
+ parts.push(part);
+ let last = parts.len() - 1;
+ (parts.into_iter().enumerate())
+ .map(|(i, (remove, add))| AnnounceBroadcast::Request {
+ more: i < last,
+ remove,
+ add,
+ })
+ .collect()
+}
+
+/// Decode an ANNOUNCE_REQUEST body. The prefixes are the sender's own, so one that is
+/// no path fails the message rather than being refused.
+fn decode_request(r: &mut Decoder<'_>, version: Version) -> Result {
+ let more = r.bool()?;
+ let count = r.varint()?;
+ // Each id takes at least a byte, so the body bounds the count.
+ if count > r.remaining() as u64 {
+ return Err(DecodeError::TooMany);
+ }
+ let remove = (0..count).map(|_| r.varint()).collect::>()?;
+ let count = r.varint()?;
+ // Each prefix takes at least its length and its flag.
+ if count > (r.remaining() / 2) as u64 {
+ return Err(DecodeError::TooMany);
+ }
+ let add = (0..count)
+ .map(|_| Ok((Path::decode(r, version)?, r.bool()?)))
+ .collect::>()?;
+ Ok(AnnounceBroadcast::Request { more, remove, add })
+}
+
/// A path suffix as it travels: the first `keep` segments of a base announcement's
/// suffix, followed by `rest`.
///
@@ -719,8 +768,10 @@ impl Encode for AnnounceBroadcast {
// subscribe stream). The body varies by type.
let typ = match self {
Self::Active { .. } => ANNOUNCE_START,
- Self::Synced if version.has_synced() => SYNCED,
- Self::Synced => return Err(EncodeError::Version),
+ Self::Synced { .. } if version.has_synced() => SYNCED,
+ Self::Request { .. } if version.has_routes() => ANNOUNCE_REQUEST,
+ Self::Ok if version.has_routes() => ANNOUNCE_OK,
+ Self::Synced { .. } | Self::Request { .. } | Self::Ok => return Err(EncodeError::Version),
Self::EndedId { .. } => ANNOUNCE_END,
Self::Update { .. } => ANNOUNCE_UPDATE,
Self::Restart { .. } if version.has_announce_restart() => ANNOUNCE_RESTART,
@@ -786,13 +837,23 @@ impl Encode for AnnounceBroadcast {
encode_above(w, version, *above)?;
encode_lite06_chain(w, version, hops)?;
}
- Self::Synced => {}
- Self::RouteStart { node, suffix, offer } => {
+ Self::Synced { prefix } => w.varint(*prefix)?,
+ Self::Request { more, remove, add } => encode_request(w, version, *more, remove, add)?,
+ Self::Ok => {}
+ Self::RouteStart {
+ node,
+ prefix,
+ suffix,
+ exact,
+ offer,
+ } => {
if *node == 0 {
return Err(EncodeError::InvalidState);
}
w.varint(*node)?;
+ w.varint(*prefix)?;
suffix.encode(w, version)?;
+ w.bool(*exact);
encode_offer(w, offer)?;
}
Self::RouteUpdate { id, offer } => {
@@ -841,7 +902,9 @@ impl Encode for AnnounceBroadcast {
| Self::Restart { .. }
| Self::Skipped
| Self::Refused { .. }
- | Self::Synced
+ | Self::Synced { .. }
+ | Self::Request { .. }
+ | Self::Ok
| Self::RouteStart { .. }
| Self::RouteUpdate { .. }
| Self::RouteHold { .. }
@@ -888,7 +951,9 @@ impl<'a> Decode<'a, Version> for AnnounceBroadcast {
above: 0,
}
}
- SYNCED if version.has_synced() => Self::Synced,
+ SYNCED if version.has_synced() => Self::Synced { prefix: body.varint()? },
+ ANNOUNCE_REQUEST if version.has_routes() => decode_request(&mut body, version)?,
+ ANNOUNCE_OK if version.has_routes() => Self::Ok,
ANNOUNCE_END => Self::EndedId { id: body.varint()? },
ANNOUNCE_UPDATE => {
let id = body.varint()?;
@@ -903,11 +968,19 @@ impl<'a> Decode<'a, Version> for AnnounceBroadcast {
}
ROUTE_START if version.has_routes() => {
let node = body.varint()?;
+ let prefix = body.varint()?;
// The suffix is sized, so the offer's path reads even when it is refused.
let suffix = decode_path(&mut body)?;
+ let exact = body.bool()?;
let heard = decode_offer(&mut body)?;
match (node, suffix, heard) {
- (1.., Ok(suffix), Ok(offer)) => Self::RouteStart { node, suffix, offer },
+ (1.., Ok(suffix), Ok(offer)) => Self::RouteStart {
+ node,
+ prefix,
+ suffix,
+ exact,
+ offer,
+ },
(node, suffix, heard) => {
let reason = match (node, &suffix, &heard) {
(0, _, _) => Refusal::Node,
@@ -918,7 +991,9 @@ impl<'a> Decode<'a, Version> for AnnounceBroadcast {
Self::Refused {
message: Refusable::RouteStart {
node,
+ prefix,
suffix: suffix.ok(),
+ exact,
path: heard.map_or_else(|unusable| unusable.path, |offer| offer.path),
},
reason,
@@ -1028,8 +1103,10 @@ fn encode_hops(w: &mut Encoder<'_>, version: Version, hops: &Hops) -> Result<(),
}
}
-/// ANNOUNCE_REQUEST: sent by the subscriber to request ANNOUNCE_BROADCAST messages
-/// for a path prefix. Renamed from ANNOUNCE_INTEREST in lite-05.
+/// ANNOUNCE_REQUEST before lite-07: sent by the subscriber to request
+/// ANNOUNCE_BROADCAST messages for a path prefix, opening a bidirectional Announce
+/// Stream. Renamed from ANNOUNCE_INTEREST in lite-05. Lite-07 edits a set of prefixes on
+/// its one stream instead ([`AnnounceBroadcast::Request`]).
#[derive(Clone, Debug)]
pub struct AnnounceRequest {
// Request tracks with this prefix.
@@ -1039,39 +1116,30 @@ pub struct AnnounceRequest {
// when encoding for another version and decodes as unknown; lite-06 carries the
// identity session-wide in the SETUP Hop parameter instead.
pub exclude_hop: Hop,
- // Lite07+: also announce routes with a `.`-prefixed segment below the prefix.
- // Not on the wire earlier, so the value set here is ignored when encoding for an
- // older version and decodes as false.
- pub hidden: bool,
}
impl<'a> Message<'a> for AnnounceRequest {
const CONTROL: Option = Some(crate::stats::Message::Announce);
fn decode_msg(r: &mut Decoder<'a>, version: Version) -> Result {
+ if version.has_routes() {
+ return Err(DecodeError::Version);
+ }
let prefix = Path::decode(r, version)?;
let exclude_hop = match version.has_exclude_hop() {
true => Hop::decode(r, version)?,
false => Hop::UNKNOWN,
};
- let hidden = match version.has_hidden() {
- true => r.bool()?,
- false => false,
- };
- Ok(Self {
- prefix,
- exclude_hop,
- hidden,
- })
+ Ok(Self { prefix, exclude_hop })
}
fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> {
+ if version.has_routes() {
+ return Err(EncodeError::Version);
+ }
self.prefix.encode(w, version)?;
if version.has_exclude_hop() {
self.exclude_hop.encode(w, version)?;
}
- if version.has_hidden() {
- w.bool(self.hidden);
- }
Ok(())
}
@@ -1182,9 +1250,7 @@ impl<'a> Message<'a> for AnnounceInit {
/// `origin` is the responder's session origin id. In Lite05 the publisher no
/// longer stamps it onto each Announce's hop chain; the subscriber appends it on
/// receipt instead. Lite05/06 carry it and `active`, the number of starts immediately
-/// following. Lite07 has an empty body: its claims name their route instead of a hop
-/// chain, and SYNCED marks the initial set, so neither field is sent and both decode
-/// as unknown.
+/// following. Lite07 answers on its one stream instead ([`AnnounceBroadcast::Ok`]).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct AnnounceOk {
pub origin: Hop,
@@ -1194,15 +1260,9 @@ pub struct AnnounceOk {
impl<'a> Message<'a> for AnnounceOk {
const CONTROL: Option = Some(crate::stats::Message::Announce);
fn decode_msg(r: &mut Decoder<'a>, version: Version) -> Result {
- if !version.has_announce_ok() {
+ if !version.has_announce_ok() || version.has_routes() {
return Err(DecodeError::Version);
}
- if version.has_routes() {
- return Ok(Self {
- origin: Hop::UNKNOWN,
- active: 0,
- });
- }
let origin = Hop::decode(r, version)?;
let active = r.varint()?;
@@ -1210,12 +1270,9 @@ impl<'a> Message<'a> for AnnounceOk {
}
fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> {
- if !version.has_announce_ok() {
+ if !version.has_announce_ok() || version.has_routes() {
return Err(EncodeError::Version);
}
- if version.has_routes() {
- return Ok(());
- }
self.origin.encode(w, version)?;
w.varint(self.active)
@@ -1252,49 +1309,36 @@ mod tests {
assert!(matches!(decode(&wire, usize::MAX), Err(DecodeError::InvalidValue)));
}
- /// Lite-07's ANNOUNCE_OK is empty: SYNCED replaces the count, and routes replace the
- /// Hop ID. Lite-06 keeps both.
+ /// Lite-06's ANNOUNCE_OK opens the stream with a Hop ID and a count. Lite-07's is a
+ /// message of the Announce Stream instead: SYNCED replaces the count, and routes
+ /// the Hop ID.
#[test]
- fn announce_ok_is_empty_on_lite07() {
- for version in [Version::Lite06, Version::Lite07] {
- let mut bytes = Vec::new();
- AnnounceOk {
- origin: Hop::new(1).unwrap(),
- active: 7,
- }
- .encode(&mut Encoder::new(&mut bytes, version.into()), version)
+ fn announce_ok_opens_only_a_lite06_stream() {
+ let ok = AnnounceOk {
+ origin: Hop::new(1).unwrap(),
+ active: 7,
+ };
+ let mut bytes = Vec::new();
+ ok.encode(&mut Encoder::new(&mut bytes, Version::Lite06.into()), Version::Lite06)
.unwrap();
- assert_eq!(
- bytes,
- if version == Version::Lite06 {
- vec![2, 1, 7]
- } else {
- vec![0]
- }
- );
- }
- assert_eq!(
- crate::coding::decode_buf(&mut &[0u8][..], Version::Lite07, |r, v| AnnounceOk::decode(r, v)).unwrap(),
- AnnounceOk {
- origin: Hop::UNKNOWN,
- active: 0,
- }
- );
- // A body where the draft has none is malformed.
- assert!(
- crate::coding::decode_buf(&mut &[1u8, 1][..], Version::Lite07, |r, v| AnnounceOk::decode(r, v)).is_err()
+ assert_eq!(bytes, [2, 1, 7]);
+ let res = ok.encode(
+ &mut Encoder::new(&mut Vec::new(), Version::Lite07.into()),
+ Version::Lite07,
);
+ assert!(matches!(res, Err(EncodeError::Version)));
+ assert!(crate::coding::decode_buf(&mut &[0u8][..], Version::Lite07, |r, v| AnnounceOk::decode(r, v)).is_err());
}
#[test]
fn synced_marks_lite07s_initial_set() {
let mut bytes = Vec::new();
- AnnounceBroadcast::Synced
+ AnnounceBroadcast::Synced { prefix: 3 }
.encode(&mut Encoder::new(&mut bytes, Version::Lite07.into()), Version::Lite07)
.unwrap();
- assert_eq!(bytes, vec![4, 0]);
+ assert_eq!(bytes, vec![4, 1, 3]);
assert!(matches!(
- AnnounceBroadcast::Synced.encode(
+ AnnounceBroadcast::Synced { prefix: 3 }.encode(
&mut Encoder::new(&mut Vec::new(), Version::Lite06.into()),
Version::Lite06
),
@@ -1860,6 +1904,52 @@ mod tests {
assert_eq!(buf.len(), 3);
}
+ /// An edit too large for one message goes in as many as it takes, each fitting one,
+ /// every one but the last continuing it, and together holding the whole edit in order.
+ #[test]
+ fn an_edit_too_large_for_one_message_goes_in_parts() {
+ const VERSION: Version = Version::Lite07;
+ // Prefix IDs from 2^28 take five bytes.
+ let remove: Vec = (0..20_000).map(|i| (1 << 28) + i).collect();
+ let add: Vec<(Path, bool)> = (0..3_000)
+ .map(|i| (Path::new(&format!("{i:0>20}")).unwrap(), i % 2 == 0))
+ .collect();
+ let parts = request_parts(remove.clone(), add.clone(), VERSION);
+ assert!(parts.len() > 2, "{}", parts.len());
+ let (mut removed, mut added) = (Vec::new(), Vec::new());
+ for (i, part) in parts.iter().enumerate() {
+ let mut bytes = Vec::new();
+ part.encode(&mut Encoder::new(&mut bytes, VERSION.into()), VERSION)
+ .unwrap();
+ assert_eq!(broadcast_round_trip(part, VERSION), *part);
+ let AnnounceBroadcast::Request { more, remove, add } = part else {
+ panic!("{part:?}");
+ };
+ assert_eq!(*more, i + 1 < parts.len());
+ // The type and the length take at most 4 bytes.
+ assert!(
+ bytes.len() <= super::super::message::MAX_MESSAGE_SIZE + 4,
+ "{}",
+ bytes.len()
+ );
+ removed.extend(remove.iter().copied());
+ added.extend(add.iter().cloned());
+ }
+ assert_eq!((removed, added), (remove, add));
+
+ // An edit that fits goes whole, and one with nothing in it still goes.
+ let small = request_parts(vec![1], vec![(Path::from_static("a"), false)], VERSION);
+ assert!(matches!(
+ small.as_slice(),
+ [AnnounceBroadcast::Request { more: false, .. }]
+ ));
+ let empty = request_parts(Vec::new(), Vec::new(), VERSION);
+ assert!(matches!(
+ empty.as_slice(),
+ [AnnounceBroadcast::Request { more: false, .. }]
+ ));
+ }
+
fn request_round_trip(msg: &AnnounceRequest, version: Version) -> AnnounceRequest {
let mut buf = Vec::new();
msg.encode(&mut Encoder::new(&mut buf, version.into()), version)
@@ -1870,40 +1960,72 @@ mod tests {
AnnounceRequest {
prefix: got.prefix.clone(),
exclude_hop: got.exclude_hop,
- hidden: got.hidden,
}
}
- // Lite07 carries the hidden opt-in; every earlier version decodes as not opted in.
+ // Lite-07 edits a set of prefixes on its one stream instead of opening a stream per
+ // prefix.
#[test]
- fn announce_request_carries_hidden_from_lite07() {
- for hidden in [false, true] {
- let msg = AnnounceRequest {
- prefix: Path::from_static("room"),
- exclude_hop: Hop::UNKNOWN,
- hidden,
- };
- assert_eq!(request_round_trip(&msg, Version::Lite07).hidden, hidden);
- assert!(!request_round_trip(&msg, Version::Lite06).hidden);
+ fn announce_request_edits_a_set_on_lite07() {
+ let msg = AnnounceRequest {
+ prefix: Path::from_static("room"),
+ exclude_hop: Hop::UNKNOWN,
+ };
+ let res = msg.encode(
+ &mut Encoder::new(&mut Vec::new(), Version::Lite07.into()),
+ Version::Lite07,
+ );
+ assert!(matches!(res, Err(EncodeError::Version)));
+
+ let edit = AnnounceBroadcast::Request {
+ more: true,
+ remove: vec![0, 7],
+ add: vec![(Path::from_static("room"), true), (Path::empty(), false)],
+ };
+ assert_eq!(broadcast_round_trip(&edit, Version::Lite07), edit);
+ let mut bytes = Vec::new();
+ edit.encode(&mut Encoder::new(&mut bytes, Version::Lite07.into()), Version::Lite07)
+ .unwrap();
+ // Type, length, More, the removed Prefix IDs, then each added prefix with its Hidden.
+ assert_eq!(bytes, [&[16, 13, 1, 2, 0, 7, 2, 4][..], b"room", &[1, 0, 0]].concat());
+ assert_eq!(
+ broadcast_round_trip(&AnnounceBroadcast::Ok, Version::Lite07),
+ AnnounceBroadcast::Ok
+ );
+ for msg in [edit, AnnounceBroadcast::Ok] {
+ let res = msg.encode(
+ &mut Encoder::new(&mut Vec::new(), Version::Lite06.into()),
+ Version::Lite06,
+ );
+ assert!(matches!(res, Err(EncodeError::Version)), "{msg:?}");
}
}
- // A flag byte other than 0 or 1 is malformed, not a future extension.
+ // A flag byte other than 0 or 1 is malformed, not a future extension, and a count
+ // the body cannot hold is refused before anything is allocated for it.
#[test]
- fn announce_request_rejects_a_bad_hidden_flag() {
- let mut buf = Vec::new();
- let mut body = Vec::new();
- Path::from_static("room")
- .encode(&mut Encoder::new(&mut body, Version::Lite07.into()), Version::Lite07)
- .unwrap();
- body.push(2);
- Encoder::new(&mut buf, Version::Lite07.into())
- .varint(body.len() as u64)
- .unwrap();
- buf.extend_from_slice(&body);
- assert!(
- crate::coding::decode_buf(&mut &buf[..], Version::Lite07, |r, v| AnnounceRequest::decode(r, v)).is_err()
- );
+ fn announce_request_rejects_a_bad_body() {
+ let bad_flag = forge(ANNOUNCE_REQUEST, |w| {
+ w.varint(0u64).unwrap();
+ w.varint(1u64).unwrap();
+ Path::from_static("room").encode(w, Version::Lite07).unwrap();
+ w.u8(2);
+ });
+ assert!(decode_broadcast(&bad_flag, Version::Lite07).is_err());
+ let overcount = forge(ANNOUNCE_REQUEST, |w| {
+ w.varint(u64::MAX >> 2).unwrap();
+ });
+ assert!(decode_broadcast(&overcount, Version::Lite07).is_err());
+ }
+
+ // The whole set has to fit the Adds of one message.
+ #[test]
+ fn a_request_set_fits_one_message() {
+ let long = Path::new(&"a".repeat(1000)).unwrap();
+ let fits: Vec = vec![long.clone(); 65];
+ assert!(request_fits(fits.iter(), Version::Lite07));
+ let over: Vec = vec![long; 66];
+ assert!(!request_fits(over.iter(), Version::Lite07));
}
// Lite04/05 carry the subscriber's origin id so the publisher can skip reflected
@@ -1913,7 +2035,6 @@ mod tests {
let msg = AnnounceRequest {
prefix: Path::from_static("room"),
exclude_hop: Hop::new(42).unwrap(),
- hidden: false,
};
assert_eq!(request_round_trip(&msg, Version::Lite05).exclude_hop.id(), 42);
}
@@ -1925,7 +2046,6 @@ mod tests {
let msg = AnnounceRequest {
prefix: Path::from_static("room"),
exclude_hop: Hop::new(42).unwrap(),
- hidden: false,
};
assert_eq!(request_round_trip(&msg, Version::Lite06).exclude_hop, Hop::UNKNOWN);
@@ -2042,12 +2162,16 @@ mod tests {
for msg in [
AnnounceBroadcast::RouteStart {
node: 0x7a3f,
+ prefix: 0,
suffix: Path::empty(),
+ exact: true,
offer: offer(&[(0x7a3f, 0)], 0, false),
},
AnnounceBroadcast::RouteStart {
node: route::QUALIFIED | 9,
+ prefix: 0,
suffix: Path::from_static("room/a"),
+ exact: false,
offer: offer(&[(9, 0), (4, u64::MAX >> 2)], route::MAX, true),
},
AnnounceBroadcast::RouteUpdate {
@@ -2060,7 +2184,9 @@ mod tests {
},
AnnounceBroadcast::RouteStart {
node: 0x7a40,
+ prefix: 0,
suffix: Path::empty(),
+ exact: false,
offer: Offer {
priority: i8::MAX,
..offer(&[(5, 0), (route::MAX, 7), (9, 2)], 0, false)
@@ -2076,18 +2202,13 @@ mod tests {
hop: route::path::Hop { node: 4, session: 2 },
far: 9,
gone: false,
- more: true,
- moved: Vec::new(),
+ moved: 0,
}),
AnnounceBroadcast::Dead(Dead::Closed {
hop: route::path::Hop { node: 4, session: 0 },
far: route::MAX,
gone: true,
- more: false,
- moved: vec![
- (0, Ok(offer(&[(1, 0), (5, 1)], 3, false))),
- (7, Ok(offer(&[(1, 0)], 0, true))),
- ],
+ moved: 2,
}),
AnnounceBroadcast::Dead(Dead::Drain { node: 4 }),
AnnounceBroadcast::Floor {
@@ -2143,7 +2264,9 @@ mod tests {
for msg in [
AnnounceBroadcast::RouteStart {
node: 1,
+ prefix: 0,
suffix: Path::empty(),
+ exact: false,
offer: offer(&[(1, 0)], 0, false),
},
AnnounceBroadcast::RouteUpdate {
@@ -2177,12 +2300,16 @@ mod tests {
let refused = [
AnnounceBroadcast::RouteStart {
node: 0,
+ prefix: 0,
suffix: Path::empty(),
+ exact: false,
offer: offer(&[(1, 0)], 1, false),
},
AnnounceBroadcast::RouteStart {
node: 1,
+ prefix: 0,
suffix: Path::empty(),
+ exact: false,
offer: offer(&[], 1, false),
},
AnnounceBroadcast::RouteUpdate {
@@ -2198,8 +2325,7 @@ mod tests {
hop: route::path::Hop { node: 1, session: 1 },
far: 0,
gone: false,
- more: false,
- moved: Vec::new(),
+ moved: 0,
}),
// The last revision has no room for the offset.
AnnounceBroadcast::Floor {
@@ -2231,7 +2357,10 @@ mod tests {
let start = |node: u64, offer: Box)>| {
forge(ROUTE_START, move |w| {
w.varint(node).unwrap();
+ // Prefix ID 0, an empty suffix, and not exact.
+ w.varint(0u64).unwrap();
w.varint(0u64).unwrap();
+ w.u8(0);
offer(w);
})
};
@@ -2242,7 +2371,9 @@ mod tests {
};
let named = |node, path| Refusable::RouteStart {
node,
+ prefix: 0,
suffix: Some(Path::empty()),
+ exact: false,
path,
};
let over = route::MAX_HOPS as u64 + 1;
@@ -2285,12 +2416,16 @@ mod tests {
(
forge(ROUTE_START, |w| {
w.varint(1u64).unwrap();
+ w.varint(5u64).unwrap();
w.bytes(b"x//y").unwrap();
+ w.u8(0);
path(0, 1, 1, 1)(w);
}),
Refusable::RouteStart {
node: 1,
+ prefix: 5,
suffix: None,
+ exact: false,
path: hops(1, 1),
},
Refusal::Path,
@@ -2315,34 +2450,32 @@ mod tests {
);
}
- // A DEAD's moved offer is sized, so one refused leaves the next readable.
+ // Exact is a boolean.
+ let wire = forge(ROUTE_START, |w| {
+ w.varint(1u64).unwrap();
+ w.varint(0u64).unwrap();
+ w.varint(0u64).unwrap();
+ w.u8(2);
+ path(0, 1, 1, 1)(w);
+ });
+ let res = decode_broadcast(&wire, Version::Lite07);
+ assert!(matches!(res, Err(DecodeError::InvalidValue)), "{res:?}");
+
+ // A DEAD has a fixed size: it counts the ROUTE_UPDATEs following it.
let wire = forge(DEAD, |w| {
for field in [DEAD_CLOSED, 1, 1, 2] {
w.varint(field).unwrap();
}
w.u8(1);
- w.u8(0);
w.varint(2u64).unwrap();
- for (id, flags) in [(4u64, 8u8), (5, 0)] {
- w.varint(id).unwrap();
- let prefix = w.prefix_varint();
- path(flags, 1, 1, 3)(w);
- w.fill(prefix).unwrap();
- }
});
- let AnnounceBroadcast::Dead(Dead::Closed { moved, .. }) = decode_broadcast(&wire, Version::Lite07).unwrap()
- else {
- panic!("a DEAD");
- };
- assert_eq!(
- moved.iter().map(|(id, heard)| (*id, heard.is_ok())).collect::>(),
- [(4, false), (5, true)]
- );
assert_eq!(
- moved[0].1,
- Err(Unusable {
- path: hops(1, 3),
- reason: Refusal::Flags(8)
+ decode_broadcast(&wire, Version::Lite07).unwrap(),
+ AnnounceBroadcast::Dead(Dead::Closed {
+ hop: route::path::Hop { node: 1, session: 1 },
+ far: 2,
+ gone: true,
+ moved: 2,
})
);
diff --git a/rs/moq-net/src/lite/compress.rs b/rs/moq-net/src/lite/compress.rs
index 1e4d87fc818..edf0359afff 100644
--- a/rs/moq-net/src/lite/compress.rs
+++ b/rs/moq-net/src/lite/compress.rs
@@ -104,6 +104,20 @@ impl AnnounceDecoder {
Ok(suffix)
}
+ /// The next unassigned id.
+ pub fn next(&self) -> u64 {
+ self.live.next
+ }
+
+ /// Assign the next id to a claim at `path` that no ANNOUNCE_START carries: an edit of
+ /// the stream's prefixes presents a live claim there.
+ pub fn insert(&mut self, path: Path) -> u64 {
+ let id = self.live.next;
+ self.live.next += 1;
+ self.live.suffixes.insert(id, path);
+ id
+ }
+
/// The id the last start assigned.
pub fn last(&self) -> u64 {
self.live.next.saturating_sub(1)
@@ -234,6 +248,21 @@ impl AnnounceEncoder {
(Some(id), path)
}
+ /// The next unassigned id.
+ pub fn next(&self) -> u64 {
+ self.live.next
+ }
+
+ /// Assign the next id to a claim at `path` without an ANNOUNCE_START: an edit of the
+ /// stream's prefixes presents a live claim there.
+ pub fn insert(&mut self, path: Path) -> u64 {
+ let id = self.live.next;
+ self.live.next += 1;
+ self.heads.insert(path.parts(), id);
+ self.live.suffixes.insert(id, path);
+ id
+ }
+
/// An ANNOUNCE_END for a live `id`.
pub fn end(&mut self, id: u64) {
if !self.version.has_announce_compression() {
diff --git a/rs/moq-net/src/lite/corpus.rs b/rs/moq-net/src/lite/corpus.rs
index 539f4cb67a9..f003cc790a6 100644
--- a/rs/moq-net/src/lite/corpus.rs
+++ b/rs/moq-net/src/lite/corpus.rs
@@ -74,11 +74,6 @@ fn run(case: &Case, data: &[u8]) -> String {
match case.message {
Message::Announce => match super::AnnounceBroadcast::decode_slice(data, version) {
Ok((super::AnnounceBroadcast::Refused { .. }, used)) if used == data.len() => "withdrawn".to_string(),
- Ok((super::AnnounceBroadcast::Dead(super::Dead::Closed { moved, .. }), used))
- if used == data.len() && moved.iter().any(|(_, heard)| heard.is_err()) =>
- {
- "withdrawn".to_string()
- }
decoded => outcome(data, version, decoded),
},
Message::AnnounceInit => {
diff --git a/rs/moq-net/src/lite/mod.rs b/rs/moq-net/src/lite/mod.rs
index ccc08dc1b63..e4661a6f936 100644
--- a/rs/moq-net/src/lite/mod.rs
+++ b/rs/moq-net/src/lite/mod.rs
@@ -35,6 +35,7 @@ mod publisher;
#[cfg(all(feature = "async", any(test, feature = "fuzz")))]
pub use publisher::{CoverageBench, PausedBench};
#[cfg(feature = "async")]
+mod rekey;
mod replay;
#[cfg(feature = "async")]
mod session;
diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs
index f60f510f90d..49b62411cb7 100644
--- a/rs/moq-net/src/lite/publisher.rs
+++ b/rs/moq-net/src/lite/publisher.rs
@@ -29,7 +29,7 @@ use crate::{
use super::Version;
-mod routed;
+pub(super) mod routed;
pub(super) struct PublisherConfig {
/// The runtime that arms the publisher's timers.
@@ -306,15 +306,16 @@ impl Publisher {
}
}
-impl Drop for Publisher {
- fn drop(&mut self) {
- // Every announce stream closes with the session: disconnect them together.
- let _closing = self.shared.origin.route_closing(self.shared.peer_setup.tag());
- self.children = kio::Tasks::new();
+impl Publisher {
+ /// What the session's route layer answers the peer from (lite-07).
+ pub fn route_publish(&self) -> super::subscriber::route_stream::Publish {
+ super::subscriber::route_stream::Publish {
+ origin: self.shared.origin.clone(),
+ auth: self.shared.auth.clone(),
+ withdrawal: self.shared.withdrawal.clone(),
+ }
}
-}
-impl Publisher {
/// The peer's latest PROBE target, or zero while it asks for none.
pub fn peer_target(&self) -> kio::Consumer {
self.shared.peer_target.consume()
@@ -447,6 +448,8 @@ impl Control {
// PROBE is lite-03+, GOAWAY lite-04+, the Track Stream and FETCH lite-05+, and the
// Auth and Tracks Streams lite-07+.
let kind = kind.filter(|kind| match kind {
+ // Lite-07 says everything about routing on a unidirectional stream.
+ lite::ControlType::Announce => !version.has_routes(),
lite::ControlType::Probe => !matches!(version, Version::Lite01 | Version::Lite02),
lite::ControlType::Goaway => version.has_goaway(),
lite::ControlType::Fetch | lite::ControlType::Track => version.has_track_stream(),
@@ -993,19 +996,6 @@ impl AnnounceServe {
stream.reader.poll_decode::(waiter)
})?;
let prefix: crate::Path = interest.prefix.clone();
- let hidden: bool = interest.hidden;
- // Lite-07 answers from the origin's route node instead of the route table.
- let answer: Option = self.shared.version.has_routes().then(|| {
- routed::Answer::new(
- self.shared.origin.clone(),
- prefix.clone(),
- hidden,
- self.shared.auth.clone(),
- )
- .with_peer_setup(self.shared.peer_setup.clone())
- .with_session(self.shared.session.clone(), self.shared.runtime.clone())
- });
-
// The identity whose routes we filter out. Lite-04/05 carry it per
// announce stream; lite-06+ reads the session-wide SETUP Hop
// parameter, the same identity the subscribe path excludes. A peer that
@@ -1025,33 +1015,16 @@ impl AnnounceServe {
Some(hop) => Poll::Ready(*hop),
None => self.shared.peer_setup.poll_hop(waiter).map(|hop| hop.unwrap_or(*assigned)),
});
- let (origin, announced, run): (origin::Consumer, origin::Routes, AnnounceRun) = match answer.is_some() {
- // The route table goes unread.
- true => {
- let origin = self.shared.origin.empty();
- let announced = origin.routes();
- (
- origin,
- announced,
- AnnounceRun::new(prefix, exclude_hop, self.shared.version),
- )
- }
- false => self.start(prefix, exclude_hop, hidden),
- };
+ let (origin, announced, run): (origin::Consumer, origin::Routes, AnnounceRun) = self.start(prefix, exclude_hop);
// Streaming announce updates.
wait!(Run => {
let stream = self.stream.as_mut().expect("stream present");
let withdrawing = self.shared.withdrawal.poll(waiter).is_ready();
- let res = match answer {
- Some(answer) => ready!(answer.poll(stream, withdrawing, waiter)),
- None => {
- if withdrawing {
- run.withdraw(stream, origin, announced)?;
- }
- ready!(run.poll(stream, origin, announced, waiter))
- }
- };
+ if withdrawing {
+ run.withdraw(stream, origin, announced)?;
+ }
+ let res = ready!(run.poll(stream, origin, announced, waiter));
if let Err(err) = res {
Self::abort(&mut self.stream, origin.root(), err);
}
@@ -1077,12 +1050,7 @@ impl AnnounceServe {
stream.take().expect("stream present").writer.abort(&err);
}
- fn start(
- &self,
- prefix: crate::Path,
- exclude_hop: Hop,
- hidden: bool,
- ) -> (origin::Consumer, origin::Routes, AnnounceRun) {
+ fn start(&self, prefix: crate::Path, exclude_hop: Hop) -> (origin::Consumer, origin::Routes, AnnounceRun) {
// If the requested prefix is outside our scope (an empty origin, or a token
// that doesn't grant it), we simply have nothing to announce. Respond with an
// empty set and keep the stream open (the subscriber treats a FIN here as a
@@ -1110,12 +1078,6 @@ impl AnnounceServe {
Version::Lite01 | Version::Lite02 => origin,
_ => origin.forwardable(),
};
- // Hidden routes are left out unless the peer opted in. A publish origin that
- // already opted in (the caller's choice for this peer) keeps them either way.
- let origin = match hidden {
- true => origin.with_hidden(true),
- false => origin,
- };
let announced = origin.routes();
let mut run = AnnounceRun::new(prefix, self.shared.self_origin, self.shared.version);
run.auth = Some(crate::auth::Gate::new(
@@ -1212,7 +1174,7 @@ impl AnnounceRun {
if let Some(remaining) = &mut self.initial {
*remaining = remaining.saturating_sub(1);
if *remaining == 0 {
- stream.writer.buffer(&lite::AnnounceBroadcast::Synced)?;
+ stream.writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 })?;
self.initial = None;
}
}
@@ -1561,7 +1523,7 @@ impl AnnounceRun {
self.retract(stream, suffix.clone(), &origin.root().rejoin(&suffix))?;
}
if self.initial.take().is_some() {
- stream.writer.buffer(&lite::AnnounceBroadcast::Synced)?;
+ stream.writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 })?;
}
self.phase = AnnouncePhase::Withdrawing;
Ok(())
@@ -1627,13 +1589,13 @@ impl AnnounceRun {
return Poll::Ready(res);
}
if self.initial == Some(0) {
- stream.writer.buffer(&lite::AnnounceBroadcast::Synced)?;
+ stream.writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 })?;
self.initial = None;
continue;
}
let Poll::Ready(next) = announced.poll_next(waiter) else {
if self.initial.take().is_some() {
- stream.writer.buffer(&lite::AnnounceBroadcast::Synced)?;
+ stream.writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 })?;
continue;
}
return Poll::Pending;
@@ -1654,7 +1616,7 @@ impl AnnounceRun {
}) => (update, false, false),
None => {
if self.initial.take().is_some() {
- stream.writer.buffer(&lite::AnnounceBroadcast::Synced)?;
+ stream.writer.buffer(&lite::AnnounceBroadcast::Synced { prefix: 0 })?;
continue;
}
// The buffer is empty (flushed at the loop top), so FIN now and
@@ -3671,459 +3633,432 @@ mod announce_test {
.scope(root.as_str(), &crate::Patterns::from(crate::Pattern::all()))
.unwrap();
let prefix = crate::Path::new(&"p".repeat(crate::Path::MAX_BYTES - 50)).unwrap();
- let mut answer = super::routed::Answer::new(consumer, prefix, true, crate::auth::Link::new(false));
+ let mut answer = super::routed::Answer::alone(consumer);
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::new(log)),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
+ Version::Lite07,
+ );
+ let waiter = kio::Waiter::noop();
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
};
- assert!(answer.poll(&mut stream, false, &kio::Waiter::noop()).is_pending());
- wire.take_ok();
- assert_eq!(wire.take_announces(), [lite::AnnounceBroadcast::Synced]);
+ answer.request(false, Vec::new(), vec![(prefix, true)]).unwrap();
+ assert_eq!(
+ sent(&mut answer),
+ [
+ lite::AnnounceBroadcast::Ok,
+ lite::AnnounceBroadcast::Synced { prefix: 0 }
+ ]
+ );
}
- /// A lite-07 peer sends nothing after its request: a byte more on the stream is a
- /// protocol violation.
+ /// A lite-07 peer edits one request at a time, names only prefixes in effect, and
+ /// asks for no prefix at or below another: each is a protocol violation.
#[test]
- fn a_lite07_peer_writing_after_its_request_violates_the_stream() {
- /// A peer that keeps writing.
- struct Chatty;
- impl crate::transport::poll::Chunks for Chatty {
- type Error = SinkError;
- fn poll_read_chunk(
- &mut self,
- _: &kio::Waiter,
- _: usize,
- ) -> std::task::Poll, SinkError>> {
- std::task::Poll::Ready(Ok(Some(bytes::Bytes::from_static(&[0x01]))))
- }
- fn stop(&mut self, _: u32) {}
- }
-
+ fn a_lite07_request_out_of_turn_or_nested_is_a_violation() {
+ let path = |path: &str| crate::Path::new(path).unwrap();
let origin = Hop::new(1).unwrap().produce();
- let mut answer = super::routed::Answer::new(
- origin.consume(),
- crate::Path::empty(),
- true,
- crate::auth::Link::new(false),
- );
+ let answer = || super::routed::Answer::alone(origin.consume());
let log = Log::default();
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::new(log)),
- Version::Lite07,
- ),
- reader: Reader::new(
- crate::transport::boxed::RecvStream::new(crate::transport::poll::Buffered::new(Chatty)),
- Version::Lite07,
- ),
+ let mut wire = Wire {
+ writes: log.writes.clone(),
+ cursor: 0,
+ version: Version::Lite07,
};
- let polled = answer.poll(&mut stream, false, &kio::Waiter::noop());
- assert!(
- matches!(polled, std::task::Poll::Ready(Err(Error::ProtocolViolation))),
- "{polled:?}"
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
+ Version::Lite07,
);
- }
-
- /// A lite-07 stream holds a route on the wire only while it shows one of the
- /// route's claims: an origin whose only claim is hidden from the peer starts no
- /// route, the route ends after the floor once its last shown claim goes, and it
- /// starts again under a new Route ID with the next one.
- #[kio::test]
- async fn a_lite07_route_goes_only_with_a_shown_claim() {
+ let waiter = kio::Waiter::noop();
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
+ };
+
+ // A second edit before the first is answered.
+ let mut first = answer();
+ first.request(false, Vec::new(), vec![(path("a"), false)]).unwrap();
+ let again = first.request(false, Vec::new(), vec![(path("b"), false)]);
+ assert!(matches!(again, Err(Error::ProtocolViolation)), "{again:?}");
+
+ // Answered, the next edit goes.
+ assert_eq!(sent(&mut first)[0], lite::AnnounceBroadcast::Ok);
+ first.request(false, vec![0], vec![(path("b"), false)]).unwrap();
+ assert_eq!(sent(&mut first)[0], lite::AnnounceBroadcast::Ok);
+
+ // A prefix that is not in effect, one asked for twice, and one below another.
+ for (remove, add) in [
+ (vec![0], Vec::new()),
+ (Vec::new(), vec![path("b")]),
+ (Vec::new(), vec![path("b/c")]),
+ (Vec::new(), vec![path("")]),
+ (Vec::new(), vec![path("x"), path("x/y")]),
+ ] {
+ let add: Vec<_> = add.into_iter().map(|path| (path, false)).collect();
+ let refused = first.request(false, remove.clone(), add.clone());
+ assert!(matches!(refused, Err(Error::ProtocolViolation)), "{remove:?} {add:?}");
+ }
+
+ // A set past one message.
+ let mut wide = answer();
+ let long = "p".repeat(crate::Path::MAX_BYTES - 8);
+ let add: Vec<_> = (0..80).map(|i| (path(&format!("{i:02}/{long}")), false)).collect();
+ let refused = wide.request(false, Vec::new(), add.clone());
+ assert!(matches!(refused, Err(Error::ProtocolViolation)), "{refused:?}");
+
+ // An edit in parts takes effect, and is answered, only at its last.
+ let mut parts = answer();
+ parts.request(true, Vec::new(), vec![(path("a"), false)]).unwrap();
+ assert!(parts.continuing());
+ assert!(sent(&mut parts).is_empty());
+ parts.request(false, Vec::new(), vec![(path("b"), false)]).unwrap();
+ assert!(!parts.continuing());
+ assert_eq!(sent(&mut parts)[0], lite::AnnounceBroadcast::Ok);
+ // Its parts remove only what the set held before it, so not a prefix one adds.
+ parts.request(true, vec![0], vec![(path("c"), false)]).unwrap();
+ let refused = parts.request(false, vec![2], Vec::new());
+ assert!(matches!(refused, Err(Error::ProtocolViolation)), "{refused:?}");
+
+ // Parts fail as soon as their Adds alone pass the limit, or they remove more
+ // prefixes than the set holds, before the last arrives.
+ let mut wide = answer();
+ wide.request(true, Vec::new(), add[..10].to_vec()).unwrap();
+ let refused = wide.request(true, Vec::new(), add[10..].to_vec());
+ assert!(matches!(refused, Err(Error::ProtocolViolation)), "{refused:?}");
+ let mut few = answer();
+ let refused = few.request(true, vec![0, 1], Vec::new());
+ assert!(matches!(refused, Err(Error::ProtocolViolation)), "{refused:?}");
+ }
+
+ /// A lite-07 edit removing a prefix and adding it again, as a peer changing its
+ /// Hidden opt-in does, keeps what a mount beneath it presents: the mount's route
+ /// and claim carry over to the new Prefix ID, and nothing ends.
+ #[kio::test]
+ async fn a_lite07_prefix_asked_again_keeps_its_mount() {
let origin = Hop::new(1).unwrap().produce();
- let _stats = origin.announce(".stats/node", origin::RouteInfo::new()).unwrap();
- let consumer = origin.consume();
- let mut answer =
- super::routed::Answer::new(consumer, crate::Path::empty(), false, crate::auth::Link::new(false));
+ let _cam = origin
+ .announce("tenant-a/public/cam", origin::RouteInfo::new())
+ .unwrap();
+ let consumer = (origin.mount("tenant-b/shared", "tenant-a/public").unwrap())
+ .scope("tenant-b", &crate::Patterns::from(crate::Pattern::all()))
+ .unwrap()
+ .consume();
+ let mut answer = super::routed::Answer::alone(consumer);
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::new(log)),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let waiter = kio::Waiter::noop();
- assert!(answer.poll(&mut stream, false, &waiter).is_pending());
- wire.take_ok();
- assert_eq!(
- wire.take_announces(),
- [lite::AnnounceBroadcast::Synced],
- "nothing shown"
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
+ Version::Lite07,
);
-
- let shown = origin.announce("x", origin::RouteInfo::new()).unwrap();
- // The origin feeds its node from a task.
+ let waiter = kio::Waiter::noop();
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
+ };
+ let shared = crate::Path::from_static("shared");
+ answer
+ .request(false, Vec::new(), vec![(shared.clone(), false)])
+ .unwrap();
kio::clock::sim::yield_now().await;
- assert!(answer.poll(&mut stream, false, &waiter).is_pending());
- let announces = wire.take_announces();
+ let first = sent(&mut answer);
assert!(
- matches!(
- announces.as_slice(),
- [
- lite::AnnounceBroadcast::RouteStart { .. },
- lite::AnnounceBroadcast::Active { route: 0, .. },
- lite::AnnounceBroadcast::Floor { id: 0, .. },
- ]
- ),
- "{announces:?}"
+ first
+ .iter()
+ .any(|msg| matches!(msg, lite::AnnounceBroadcast::Active { .. })),
+ "{first:?}"
);
- drop(shown);
+ let ends = |msgs: &[lite::AnnounceBroadcast]| {
+ msgs.iter().any(|msg| {
+ matches!(
+ msg,
+ lite::AnnounceBroadcast::RouteEnd { .. } | lite::AnnounceBroadcast::EndedId { .. }
+ )
+ })
+ };
+ answer.request(false, vec![0], vec![(shared, true)]).unwrap();
kio::clock::sim::yield_now().await;
- assert!(answer.poll(&mut stream, false, &waiter).is_pending());
- let announces = wire.take_announces();
- assert!(
- matches!(
- announces.as_slice(),
- [
- lite::AnnounceBroadcast::EndedId { .. },
- lite::AnnounceBroadcast::Floor { id: 0, .. },
- lite::AnnounceBroadcast::RouteEnd { id: 0 },
- ]
- ),
- "{announces:?}"
- );
-
- let _again = origin.announce("y", origin::RouteInfo::new()).unwrap();
+ let again = sent(&mut answer);
+ assert_eq!(again[0], lite::AnnounceBroadcast::Ok);
+ assert!(!ends(&again), "{again:?}");
+
+ // Widened past the mount point, the mount's stream stays, and so does its route,
+ // which its ROUTE_START marked as lying at its own prefix: nothing ends or starts.
+ answer
+ .request(false, vec![1], vec![(crate::Path::empty(), true)])
+ .unwrap();
kio::clock::sim::yield_now().await;
- assert!(answer.poll(&mut stream, false, &waiter).is_pending());
- let announces = wire.take_announces();
- assert!(
- matches!(
- announces.as_slice(),
- [
- lite::AnnounceBroadcast::RouteStart { .. },
- lite::AnnounceBroadcast::Active { route: 1, .. },
- lite::AnnounceBroadcast::Floor { id: 1, .. },
- ]
- ),
- "{announces:?}"
+ let wider = sent(&mut answer);
+ assert_eq!(
+ wider,
+ [
+ lite::AnnounceBroadcast::Ok,
+ lite::AnnounceBroadcast::Synced { prefix: 2 }
+ ]
);
}
- /// A DEAD whose moved routes outgrow one message goes in consecutive parts, each
- /// fitting one and all but the last with More set, which list its moved routes in
- /// order, so failing over many routes at once never fails the stream; one that fits
- /// goes whole.
- #[test]
- fn a_lite07_dead_past_one_message_goes_in_parts() {
- use crate::route::path::{Hop, Offer};
- let offer = Offer {
- // The longest path, of the widest Hop IDs.
- path: (0..crate::route::MAX_HOPS as u64)
- .map(|session| Hop {
- node: crate::route::GENERATED - session,
- session,
- })
- .collect(),
- draining: false,
- priority: 0,
- metric: 1,
- down_only: false,
- };
- let dead = |moved: std::ops::Range, more| lite::Dead::Closed {
- hop: Hop { node: 7, session: 1 },
- far: 8,
- gone: true,
- more,
- moved: moved.map(|id| (id, Ok(offer.clone()))).collect(),
- };
- let encodes = |msg: &lite::AnnounceBroadcast| {
- crate::coding::Encode::encode(
- msg,
- &mut crate::coding::Encoder::new(&mut Vec::new(), Version::Lite07.into()),
- Version::Lite07,
- )
- .is_ok()
- };
- // Past three times 65,535 bytes at about 300 a route.
- let many = 700;
- assert!(!encodes(&lite::AnnounceBroadcast::Dead(dead(0..many, false))));
-
+ /// A lite-07 mount's route at its mount point, asked for exactly there, starts
+ /// marked exact, and a route the session's own key presents there is not.
+ #[kio::test]
+ async fn a_lite07_mount_point_route_starts_exact() {
+ let origin = Hop::new(1).unwrap().produce();
+ let _cam = origin
+ .announce("tenant-a/public/cam", origin::RouteInfo::new())
+ .unwrap();
+ let consumer = (origin.mount("tenant-b/shared", "tenant-a/public").unwrap())
+ .scope("tenant-b", &crate::Patterns::from(crate::Pattern::all()))
+ .unwrap()
+ .consume();
+ let _own = origin.announce("tenant-b/own", origin::RouteInfo::new()).unwrap();
+ let mut answer = super::routed::Answer::alone(consumer);
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::new(log)),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
+ Version::Lite07,
+ );
let waiter = kio::Waiter::noop();
- super::routed::buffer_dead(&mut stream, dead(0..2, false)).unwrap();
- assert!(stream.writer.poll_flush(&waiter).is_ready());
- assert_eq!(
- wire.take_announces(),
- [lite::AnnounceBroadcast::Dead(dead(0..2, false))]
- );
-
- super::routed::buffer_dead(&mut stream, dead(0..many, false)).unwrap();
- assert!(stream.writer.poll_flush(&waiter).is_ready());
- let sent = wire.take_announces();
- assert!(sent.len() >= 3, "{} parts", sent.len());
- let mut next = 0;
- for (i, part) in sent.iter().enumerate() {
- let lite::AnnounceBroadcast::Dead(lite::Dead::Closed { moved, .. }) = part else {
- panic!("a DEAD part: {part:?}");
- };
- let end = next + moved.len() as u64;
- assert!(end > next, "part {i} lists no route");
- assert_eq!(
- *part,
- lite::AnnounceBroadcast::Dead(dead(next..end, i + 1 < sent.len()))
- );
- next = end;
- }
- assert_eq!(next, many);
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
+ };
+ let starts = |msgs: &[lite::AnnounceBroadcast]| -> Vec<(u64, bool)> {
+ (msgs.iter())
+ .filter_map(|msg| match msg {
+ lite::AnnounceBroadcast::RouteStart { prefix, exact, .. } => Some((*prefix, *exact)),
+ _ => None,
+ })
+ .collect()
+ };
+ answer
+ .request(
+ false,
+ Vec::new(),
+ vec![
+ (crate::Path::from_static("shared"), false),
+ (crate::Path::from_static("own"), false),
+ ],
+ )
+ .unwrap();
+ kio::clock::sim::yield_now().await;
+ let mut first = sent(&mut answer);
+ kio::clock::sim::yield_now().await;
+ first.extend(sent(&mut answer));
+ let mut starts = starts(&first);
+ starts.sort_unstable();
+ assert_eq!(starts, [(0, true), (1, false)], "{first:?}");
}
- #[test]
- fn synced_survives_a_removed_initial_route_and_live_churn() {
+ /// A lite-07 mount's route starts exact below the requested prefix too, so a
+ /// narrowing to its mount point and a widening back carry it over with nothing
+ /// ended or started again.
+ #[kio::test]
+ async fn a_lite07_mount_route_stays_through_a_narrowing_and_a_widening() {
let origin = Hop::new(1).unwrap().produce();
- let removed = origin.announce("z/removed", origin::RouteInfo::new()).unwrap();
- let _initial = origin.announce("z/retained", origin::RouteInfo::new()).unwrap();
- let consumer = origin.consume();
- let mut announced = consumer.routes();
- let budget = kio::Producer::new(0usize);
+ let _cam = origin
+ .announce("tenant-a/public/cam", origin::RouteInfo::new())
+ .unwrap();
+ let consumer = (origin.mount("tenant-b/shared", "tenant-a/public").unwrap())
+ .scope("tenant-b", &crate::Patterns::from(crate::Pattern::all()))
+ .unwrap()
+ .consume();
+ let mut answer = super::routed::Answer::alone(consumer);
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::stepped(log, budget.consume())),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let mut run = AnnounceRun::new(
- crate::Path::from_static(""),
- crate::Hop::from(consumer.hop()),
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
Version::Lite07,
);
let waiter = kio::Waiter::noop();
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- drop(removed);
- let mut churn = Vec::new();
- for limit in 1..=4 {
- churn.push(origin.announce(format!("a/{limit}"), origin::RouteInfo::new()).unwrap());
- *budget.write().ok().unwrap() = limit;
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- }
- assert_eq!(wire.take_ok().active, 0);
- let messages = wire.take_announces();
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
+ };
+ answer
+ .request(false, Vec::new(), vec![(crate::Path::empty(), true)])
+ .unwrap();
+ kio::clock::sim::yield_now().await;
+ let mut first = sent(&mut answer);
+ kio::clock::sim::yield_now().await;
+ first.extend(sent(&mut answer));
+ let shared = crate::Path::from_static("shared");
assert!(
- matches!(&messages[0], lite::AnnounceBroadcast::Active { suffix, .. } if suffix.rest.as_str() == "z/retained")
+ first.iter().any(|msg| matches!(
+ msg,
+ lite::AnnounceBroadcast::RouteStart { suffix, exact: true, .. } if *suffix == shared
+ )),
+ "{first:?}"
);
- assert_eq!(messages[2], lite::AnnounceBroadcast::Synced);
- assert!(!messages.iter().any(
- |msg| matches!(msg, lite::AnnounceBroadcast::Active { suffix, .. } if suffix.rest.as_str() == "z/removed")
- ));
+
+ // Narrowed to the mount point, then widened back to the root.
+ for (remove, add, prefix) in [(0, shared.clone(), 1), (1, crate::Path::empty(), 2)] {
+ answer.request(false, vec![remove], vec![(add, true)]).unwrap();
+ kio::clock::sim::yield_now().await;
+ assert_eq!(
+ sent(&mut answer),
+ [lite::AnnounceBroadcast::Ok, lite::AnnounceBroadcast::Synced { prefix }]
+ );
+ }
}
- #[test]
- fn a_partial_regrant_cannot_overtake_an_initial_route() {
+ /// A lite-07 claim covering the requested prefix from above is presented there with
+ /// how far above it is, and a narrowing carries it to the narrower prefix with nothing
+ /// sent for it: the peer adds the distance to its Above as this side does.
+ #[kio::test]
+ async fn a_lite07_narrowing_carries_a_covering_claim_further_above() {
let origin = Hop::new(1).unwrap().produce();
let _a = origin.announce("a", origin::RouteInfo::new()).unwrap();
- let _b = origin.announce("b", origin::RouteInfo::new()).unwrap();
- let consumer = origin.consume();
- let mut announced = consumer.routes();
- let budget = kio::Producer::new(1usize);
+ let mut answer = super::routed::Answer::alone(origin.consume());
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::stepped(log, budget.consume())),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let mut run = AnnounceRun::new(
- crate::Path::from_static(""),
- crate::Hop::from(consumer.hop()),
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
Version::Lite07,
);
let waiter = kio::Waiter::noop();
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- let auth = crate::auth::Link::new(false);
- auth.handle().authorize(&crate::auth::Grant {
- subscribe: crate::Pattern::literal("b").unwrap().into(),
- ..Default::default()
- });
- let Poll::Ready(restricted) = auth.poll_permit(crate::auth::Direction::Publish, &mut 0, &waiter) else {
- panic!("the changed grant is ready");
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
};
- run.regrant(&mut stream, &consumer, restricted).unwrap();
- *budget.write().ok().unwrap() = 8;
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- wire.take_ok();
- let messages = wire.take_announces();
- assert!(matches!(&messages[1], lite::AnnounceBroadcast::Active { suffix, .. } if suffix.rest.as_str() == "b"));
- assert_eq!(messages[2], lite::AnnounceBroadcast::Synced);
- assert_eq!(messages[3], lite::AnnounceBroadcast::EndedId { id: 0 });
+ answer
+ .request(false, Vec::new(), vec![(crate::Path::from_static("a/b"), false)])
+ .unwrap();
+ kio::clock::sim::yield_now().await;
+ let mut first = sent(&mut answer);
+ kio::clock::sim::yield_now().await;
+ first.extend(sent(&mut answer));
+ assert!(
+ first
+ .iter()
+ .any(|msg| matches!(msg, lite::AnnounceBroadcast::Active { above: 1, .. })),
+ "{first:?}"
+ );
+
+ answer
+ .request(false, vec![0], vec![(crate::Path::from_static("a/b/c"), false)])
+ .unwrap();
+ kio::clock::sim::yield_now().await;
+ assert_eq!(
+ sent(&mut answer),
+ [
+ lite::AnnounceBroadcast::Ok,
+ lite::AnnounceBroadcast::Synced { prefix: 1 }
+ ]
+ );
}
- #[test]
- fn synced_cuts_a_regrant_burst_at_the_initial_bound() {
+ /// A lite-07 stream holds a route on the wire only while it shows one of the
+ /// route's claims: an origin whose only claim is hidden from the peer starts no
+ /// route, the route ends after the floor once its last shown claim goes, and it
+ /// starts again under a new Route ID with the next one.
+ #[kio::test]
+ async fn a_lite07_route_goes_only_with_a_shown_claim() {
let origin = Hop::new(1).unwrap().produce();
- let _a = origin.announce("a", origin::RouteInfo::new()).unwrap();
- let _b = origin.announce("b", origin::RouteInfo::new()).unwrap();
- let consumer = origin.consume();
- let mut announced = consumer.routes();
- let budget = kio::Producer::new(0usize);
+ let _stats = origin.announce(".stats/node", origin::RouteInfo::new()).unwrap();
+ let mut answer = super::routed::Answer::alone(origin.consume());
let log = Log::default();
let mut wire = Wire {
writes: log.writes.clone(),
cursor: 0,
version: Version::Lite07,
};
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::stepped(log, budget.consume())),
- Version::Lite07,
- ),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let mut run = AnnounceRun::new(
- crate::Path::from_static(""),
- crate::Hop::from(consumer.hop()),
+ let mut writer = Writer::new(
+ crate::transport::boxed::SendStream::new(SinkSend::new(log)),
Version::Lite07,
);
let waiter = kio::Waiter::noop();
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- let auth = crate::auth::Link::new(false);
- auth.handle().authorize(&crate::auth::Grant::default());
- let Poll::Ready(restricted) = auth.poll_permit(crate::auth::Direction::Publish, &mut 0, &waiter) else {
- panic!("the changed grant is ready");
+ let mut sent = |answer: &mut super::routed::Answer| {
+ answer.drain(&mut writer).unwrap();
+ assert!(writer.poll_flush(&waiter).is_ready());
+ wire.take_announces()
};
- run.regrant(&mut stream, &consumer, restricted).unwrap();
- let _live = origin.announce("c", origin::RouteInfo::new()).unwrap();
- run.regrant(&mut stream, &consumer, crate::auth::Permit::default())
+ answer
+ .request(false, Vec::new(), vec![(crate::Path::empty(), false)])
.unwrap();
- *budget.write().ok().unwrap() = 4;
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- wire.take_ok();
- let messages = wire.take_announces();
- assert_eq!(messages.len(), 4);
- let initial: std::collections::BTreeSet<_> = messages[..2]
- .iter()
- .map(|message| match message {
- lite::AnnounceBroadcast::Active { suffix, .. } => suffix.rest.as_str(),
- other => panic!("expected an initial start, got {other:?}"),
- })
- .collect();
- assert_eq!(initial, std::collections::BTreeSet::from(["a", "b"]));
- assert_eq!(messages[2], lite::AnnounceBroadcast::Synced);
assert_eq!(
- messages
- .iter()
- .filter(|msg| matches!(msg, lite::AnnounceBroadcast::Synced))
- .count(),
- 1
+ sent(&mut answer),
+ [
+ lite::AnnounceBroadcast::Ok,
+ lite::AnnounceBroadcast::Synced { prefix: 0 }
+ ],
+ "nothing shown"
);
- }
- #[test]
- fn a_seen_retraction_can_precede_synced() {
- let origin = Hop::new(1).unwrap().produce();
- let first = origin.announce("a", origin::RouteInfo::new()).unwrap();
- let unseen = origin.announce("b", origin::RouteInfo::new()).unwrap();
- let consumer = origin.consume();
- let mut announced = consumer.routes();
- let budget = kio::Producer::new(1usize);
- let log = Log::default();
- let mut wire = Wire {
- writes: log.writes.clone(),
- cursor: 0,
- version: Version::Lite07,
- };
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::stepped(log, budget.consume())),
- Version::Lite07,
+ let shown = origin.announce("x", origin::RouteInfo::new()).unwrap();
+ // The origin feeds its node from a task.
+ kio::clock::sim::yield_now().await;
+ let announces = sent(&mut answer);
+ assert!(
+ matches!(
+ announces.as_slice(),
+ [
+ lite::AnnounceBroadcast::RouteStart { prefix: 0, .. },
+ lite::AnnounceBroadcast::Active { route: 0, .. },
+ lite::AnnounceBroadcast::Floor { id: 0, .. },
+ ]
),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let mut run = AnnounceRun::new(
- crate::Path::from_static(""),
- crate::Hop::from(consumer.hop()),
- Version::Lite07,
+ "{announces:?}"
);
- let waiter = kio::Waiter::noop();
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- drop(unseen);
- drop(first);
- *budget.write().ok().unwrap() = 4;
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- wire.take_ok();
- assert!(matches!(
- wire.take_announces().as_slice(),
- [
- lite::AnnounceBroadcast::Active { .. },
- lite::AnnounceBroadcast::EndedId { id: 0 },
- lite::AnnounceBroadcast::Synced
- ]
- ));
- assert!(run.poll(&mut stream, &consumer, &mut announced, &waiter).is_pending());
- wire.assert_quiet();
- }
- #[test]
- fn an_empty_cold_relay_syncs_without_its_upstream() {
- let origin = Hop::new(1).unwrap().produce();
- let _upstream = origin.snapshot("");
- let consumer = origin.consume();
- let mut announced = consumer.routes();
- assert!(!announced.is_synced());
- let log = Log::default();
- let mut wire = Wire {
- writes: log.writes.clone(),
- cursor: 0,
- version: Version::Lite07,
- };
- let mut stream = Stream:: {
- writer: Writer::new(
- crate::transport::boxed::SendStream::new(SinkSend::new(log)),
- Version::Lite07,
+ drop(shown);
+ kio::clock::sim::yield_now().await;
+ let announces = sent(&mut answer);
+ assert!(
+ matches!(
+ announces.as_slice(),
+ [
+ lite::AnnounceBroadcast::EndedId { .. },
+ lite::AnnounceBroadcast::Floor { id: 0, .. },
+ lite::AnnounceBroadcast::RouteEnd { id: 0 },
+ ]
),
- reader: Reader::new(crate::transport::boxed::RecvStream::new(PendingRecv), Version::Lite07),
- };
- let mut run = AnnounceRun::new(
- crate::Path::from_static(""),
- crate::Hop::from(consumer.hop()),
- Version::Lite07,
+ "{announces:?}"
);
+
+ let _again = origin.announce("y", origin::RouteInfo::new()).unwrap();
+ kio::clock::sim::yield_now().await;
+ let announces = sent(&mut answer);
assert!(
- run.poll(&mut stream, &consumer, &mut announced, &kio::Waiter::noop())
- .is_pending()
+ matches!(
+ announces.as_slice(),
+ [
+ lite::AnnounceBroadcast::RouteStart { prefix: 0, .. },
+ lite::AnnounceBroadcast::Active { route: 1, .. },
+ lite::AnnounceBroadcast::Floor { id: 1, .. },
+ ]
+ ),
+ "{announces:?}"
);
- wire.take_ok();
- assert_eq!(wire.take_announces(), vec![lite::AnnounceBroadcast::Synced]);
}
/// The hops stamped on every harness route.
@@ -4279,7 +4214,7 @@ mod announce_test {
);
let mut initial = wire.take_announces();
if version.has_synced() {
- assert_eq!(initial.pop(), Some(lite::AnnounceBroadcast::Synced));
+ assert_eq!(initial.pop(), Some(lite::AnnounceBroadcast::Synced { prefix: 0 }));
}
match initial.as_slice() {
[lite::AnnounceBroadcast::Active { suffix, hops, cost, .. }] => {
@@ -4363,126 +4298,32 @@ mod announce_test {
}
/// A new best route the wire cannot tell apart (another session, same chain and
- /// cost) under another instance is another source, in either direction: an
- /// ANNOUNCE_RESTART on lite-07, carrying the instance, and an end and a fresh
- /// start before it.
+ /// cost) under another instance is another source, in either direction: an end and
+ /// a fresh start.
#[kio::test]
async fn source_flip_restarts() {
- for version in [Version::Lite06, Version::Lite07] {
- let mut h = harness_on(version).await;
- let route = crate::origin::RouteInfo::new().with_hops(pub_hops()).with_cost(7);
- let newer = h.origin.announce("cam", route.clone()).unwrap();
- // A restart waits out the update hold like an update.
- kio::clock::sleep(crate::origin::Config::default().update_hold).await;
- settle().await;
- let restarted = |msgs: Vec| match (version, msgs.as_slice()) {
- (
- Version::Lite07,
- [
- lite::AnnounceBroadcast::Restart {
- id: 0,
- instance: Some(_),
- cost,
- ..
- },
- ],
- ) => {
- assert_eq!(*cost, crate::origin::Cost::new(7))
- }
- (
- Version::Lite06,
- [
- lite::AnnounceBroadcast::EndedId { .. },
- lite::AnnounceBroadcast::Active { suffix, .. },
- ],
- ) => {
- assert_eq!(suffix.rest.as_str(), "cam")
- }
- (_, other) => panic!("expected a restart on {version}, got {other:?}"),
- };
- restarted(h.wire.take_announces());
-
- drop(newer);
- kio::clock::sleep(crate::origin::Config::default().update_hold).await;
- settle().await;
- restarted(h.wire.take_announces());
- h.assert_idle();
- }
- }
-
- /// Another instance restarts the advertisement, and so does a newer epoch of the same
- /// one, which the peer must rank: lite-07 carries both.
- #[kio::test]
- async fn a_new_instance_or_epoch_restarts_with_it() {
- let mut h = harness_on(Version::Lite07).await;
- let instance = crate::Instance::random();
- let route = |epoch| {
- crate::origin::RouteInfo::new()
- .with_instance(instance.clone(), epoch)
- .with_hops(pub_hops())
- .with_cost(7)
- };
- let _first = h.origin.announce("cam", route(1)).unwrap();
- let _newer = h.origin.announce("cam", route(2)).unwrap();
+ let mut h = harness_on(Version::Lite06).await;
+ let route = crate::origin::RouteInfo::new().with_hops(pub_hops()).with_cost(7);
+ let newer = h.origin.announce("cam", route.clone()).unwrap();
+ // A restart waits out the update hold like an update.
kio::clock::sleep(crate::origin::Config::default().update_hold).await;
settle().await;
- match h.wire.take_announces().as_slice() {
+ let restarted = |msgs: Vec| match msgs.as_slice() {
[
- lite::AnnounceBroadcast::Restart {
- id: 0,
- instance: Some(sent),
- epoch: 2,
- ..
- },
- ] => assert_eq!(*sent, instance),
+ lite::AnnounceBroadcast::EndedId { .. },
+ lite::AnnounceBroadcast::Active { suffix, .. },
+ ] => {
+ assert_eq!(suffix.rest.as_str(), "cam")
+ }
other => panic!("expected a restart, got {other:?}"),
- }
- h.assert_idle();
- }
-
- /// A grant that grows while another instance replaces an advertised one still restarts it,
- /// whether the instance is named or minted: the regrant takes in what the old cursor had
- /// pending rather than dropping it.
- #[kio::test]
- async fn a_restart_pending_when_the_grant_grows_still_restarts() {
- // The peer's ceiling: what it may subscribe to is what we may announce to it.
- let grant = |paths: &[&str]| crate::auth::Grant {
- publish: Default::default(),
- subscribe: paths
- .iter()
- .map(|path| crate::Pattern::subtree(path).unwrap())
- .collect(),
- expires: None,
};
- for anonymous in [false, true] {
- let auth = crate::auth::Link::new(false);
- auth.handle().authorize(&grant(&["cam"]));
- let mut h = harness_with(Version::Lite07, Some(auth.clone())).await;
-
- // Same chain and cost, so only the instance changed. Both land before the loop polls.
- let route = crate::origin::RouteInfo::new().with_hops(pub_hops()).with_cost(7);
- let named = (!anonymous).then(crate::Instance::random);
- let route = match &named {
- Some(instance) => route.with_instance(instance.clone(), 0),
- None => route,
- };
- let newer = h.origin.announce("cam", route).unwrap();
- let instance = newer.route().and_then(|route| route.instance);
- auth.handle().authorize(&grant(&["cam", "other"]));
- kio::clock::sleep(crate::origin::Config::default().update_hold).await;
- settle().await;
- match h.wire.take_announces().as_slice() {
- [
- lite::AnnounceBroadcast::Restart {
- id: 0, instance: sent, ..
- },
- ] => {
- assert_eq!(*sent, instance, "anonymous={anonymous}")
- }
- other => panic!("expected a restart (anonymous={anonymous}), got {other:?}"),
- }
- h.assert_idle();
- }
+ restarted(h.wire.take_announces());
+
+ drop(newer);
+ kio::clock::sleep(crate::origin::Config::default().update_hold).await;
+ settle().await;
+ restarted(h.wire.take_announces());
+ h.assert_idle();
}
/// Costs past the wire ceiling clamp to the same value, so moving between them
diff --git a/rs/moq-net/src/lite/publisher/routed.rs b/rs/moq-net/src/lite/publisher/routed.rs
index 22a4433ebc4..bf9dce54542 100644
--- a/rs/moq-net/src/lite/publisher/routed.rs
+++ b/rs/moq-net/src/lite/publisher/routed.rs
@@ -1,42 +1,47 @@
-//! Lite-07: an announce stream the peer asked for, answered from the origin's route node.
+//! Lite-07: what this session sends on its Announce Stream as a publisher, answering
+//! the prefixes the peer asks for from the origin's route node.
//!
//! The node sends the peer its routes, each by its publisher and the prefix it covers,
//! the DEADs naming hops on them, and, closed by an ANNOUNCE_FLOOR, each route's claims
-//! under the requested prefix. The stream filters what it puts on the wire: hidden
+//! under the requested prefixes. The answer filters what it puts on the wire: hidden
//! claims unless the peer asked for them, and anything this handle's scope, the
//! session's grant, the upstream horizon, or the proxy preference leaves out, in which
//! case a floor certifies nothing, since the peer cannot tell what was left out from
-//! what is gone. A route goes on the wire only while the stream shows the peer one of
+//! what is gone. A route goes on the wire only while the answer shows the peer one of
//! its claims, so a route whose claims it filters out costs the peer nothing.
//!
-//! A handle's mounts beneath the prefix are read through a node stream each, at the
-//! target, and what they present is named under the mount point on the same routes.
+//! A handle's mounts beneath a requested prefix are read through a node stream each, at
+//! the target, and what they present is named under the mount point on the same routes.
//!
-//! The peer sends nothing back on the stream. One that stops reading it for
-//! [`route::path::HOLD`] closes its session: the node keeps no more than each route's
-//! net change for it meanwhile, and a hold waiting on it ends by then anyway.
+//! The peer edits its prefixes in place. Each edit takes effect at the ANNOUNCE_OK
+//! answering it, where both ends carry the stream's routes over alike
+//! ([`rekey::plan`]), and the answer then sends only what the carried routes lack.
use std::{
- collections::{BTreeMap, BTreeSet, HashMap},
- task::{Poll, ready},
+ collections::{BTreeMap, BTreeSet, HashMap, HashSet},
+ sync::Arc,
+ task::Poll,
};
use crate::{
- Error, Path, SessionError,
- coding::Stream,
- driver::Transport,
- lite::{self, AnnounceBroadcast as Msg, Version},
+ Error, Path,
+ lite::{self, AnnounceBroadcast as Msg, Version, rekey, subscriber::route_stream::Writer},
origin,
route::{
self, node,
path::{Dead, Hop, Offer, Reach},
},
- transport::poll::Session as _,
};
/// A route by its publisher and the prefix it covers, as the handle names it.
type RouteKey = (route::Id, Path);
+/// A node stream the answer reads: the session's own, or a mount's.
+type Source = u64;
+
+/// The session's own key, which answers the requested prefixes themselves.
+const MAIN: Source = 0;
+
/// A route as one side last knows it.
#[derive(Clone, Debug, PartialEq, Eq)]
struct Reached {
@@ -58,21 +63,26 @@ struct Wire {
id: u64,
/// What the peer holds for it.
told: Reached,
+ /// The requested prefix it lies in.
+ region: Path,
+ /// Its ROUTE_START said it lies at its own prefix (see [`Answer::start`]).
+ exact: bool,
}
-/// A route the node sends the stream.
+/// A route the node sends the session.
+#[derive(Default)]
struct Sent {
- /// While the peer holds it: from the first claim the stream shows it until the last
+ /// While the peer holds it: from the first claim the answer shows it until the last
/// one goes.
wire: Option,
/// How many of its claims the peer is shown.
shown: usize,
- /// Each source's reach for it, by index: the route stands while one has one, and
- /// the first one's is the peer's.
- reach: BTreeMap,
+ /// Each source's reach for it: the route stands while one has one, and the first
+ /// one's is the peer's.
+ reach: BTreeMap,
/// Every claim the node sent for it, named as the handle names it, before filtering.
claims: BTreeMap,
- /// How many of `claims` the stream's link never carries (see [`sends`]).
+ /// How many of `claims` the session's link never carries (see [`sends`]).
withheld: usize,
/// The node's floor for those claims.
floor: Option,
@@ -97,32 +107,31 @@ impl Sent {
}
}
-/// A node stream the answer reads.
-struct Source {
+/// A mount beneath a requested prefix, read at its target.
+struct Mount {
stream: node::Stream,
- /// The mount it reads through, which names what it presents. `None` for the
- /// requested prefix's own.
- mount: Option,
+ /// Names what the stream presents.
+ mount: origin::Mounted,
+ /// The Prefix ID of the requested prefix it lies beneath.
+ prefix: u64,
}
-enum Phase {
- /// Waiting for the session's grant before the first message.
- Init,
- Running,
- /// The session is going away: every route is ended, the stream finishes once
- /// that is flushed.
- Withdrawing,
- /// Finished: waiting for the peer to read it.
- Closing,
+/// A prefix the peer asked for.
+struct Prefix {
+ /// Relative to the session's root, as the peer named it.
+ relative: Path,
+ /// Absolute. `None` when it is too long to lie beneath the handle's root: nothing is
+ /// there, so it is answered with nothing.
+ path: Option,
+ hidden: bool,
+ /// SYNCED went out for it.
+ synced: bool,
}
-/// Buffer `dead`, in as many consecutive DEADs as its moved routes need (see
-/// [`lite::Dead::split`]).
-pub(super) fn buffer_dead(stream: &mut Stream, dead: lite::Dead) -> Result<(), Error> {
- for part in dead.split() {
- stream.writer.buffer(&Msg::Dead(part))?;
- }
- Ok(())
+/// The peer's ANNOUNCE_REQUEST the node has not put in effect yet.
+struct Request {
+ remove: Vec,
+ add: Vec<(u64, Prefix)>,
}
/// Whether a stream on `link` carries `claim` at all: never an upstream-learned claim
@@ -132,298 +141,525 @@ fn sends(link: &route::Link, claim: &node::Claim) -> bool {
!(link.upstream && claim.upstream) && (link.proxy || !claim.learned)
}
-/// Closes the session whose peer stopped reading the stream.
-struct Stall {
- session: Transport,
- runtime: crate::time::Deadlines,
- /// When the stream counts as stalled, and how much it had written when that was set.
- armed: Option<(crate::time::Deadline, u64)>,
+/// The ANNOUNCE_REQUESTs of an edit received so far, applied together at its last.
+struct Parts {
+ /// The Prefix IDs they remove.
+ remove: Vec,
+ /// The prefixes they add, each with its Hidden.
+ add: Vec<(Path, bool)>,
}
-impl Stall {
- /// Ready once the writer made no progress for [`route::path::HOLD`].
- fn poll(&mut self, written: u64, waiter: &kio::Waiter) -> Poll<()> {
- if self.armed.as_ref().is_none_or(|(_, at)| *at != written) {
- let deadline = crate::time::Deadline::after(&self.runtime, route::path::HOLD);
- self.armed = Some((deadline, written));
- }
- let (deadline, _) = self.armed.as_mut().expect("armed above");
- ready!(deadline.poll(waiter));
- self.close("announce stream stalled");
- Poll::Ready(())
- }
-
- fn close(&self, reason: &str) {
- self.session
- .close(lite::error::to_session_code(&SessionError::Timeout), reason);
- }
-}
-
-/// Answers one lite-07 announce stream.
-pub(super) struct Answer {
+/// Answers the prefixes a lite-07 peer asks for.
+pub(in crate::lite) struct Answer {
origin: origin::Consumer,
- /// The requested prefix, absolute. `None` when it is too long to lie beneath the
- /// handle's root: nothing is there, so the stream announces nothing.
- prefix: Option,
- hidden: bool,
+ /// What the session's link never carries (see [`sends`]).
link: route::Link,
- /// The prefix's own node stream, then one per mount beneath it. Empty until the
- /// grant is known, and once withdrawing.
- sources: Vec,
- /// The source read first next, so a busy one does not hold the others.
- turn: usize,
+ auth: crate::auth::Link,
+ epoch: u64,
+ permit: crate::auth::Permit,
+ /// The session's key on the node.
+ main: Arc,
+ mounts: BTreeMap,
+ next_source: Source,
+ /// The mount read first next, so a busy one does not hold the others.
+ turn: Source,
+ /// The prefixes in effect, by Prefix ID.
+ prefixes: BTreeMap,
+ next_prefix: u64,
+ request: Option,
+ /// The ANNOUNCE_REQUESTs of an edit that continues.
+ parts: Option,
encoder: lite::AnnounceEncoder,
routes: HashMap,
next_route: u64,
/// The claims the peer holds: their Announce ID and attributes.
live: HashMap<(RouteKey, Path), (u64, node::Claim)>,
- gate: crate::auth::Gate,
- epoch: u64,
- permit: crate::auth::Permit,
- phase: Phase,
- /// Passes before the stream yields, so a busy node cannot hold the task.
- budget: kio::coop::Budget,
- /// The peer's SETUP: its Node ID, its session, and whether it reads the Lite-06
- /// Chain. `None` sends no chain.
- peer_setup: Option,
- /// Whether the claims sent carry their Lite-06 Chain, known once the peer's SETUP is.
+ /// Whether the claims sent carry their Lite-06 Chain.
chains: bool,
- /// `None` never closes the session.
- stall: Option,
+ /// The session is leaving: every route was ended, and nothing more is sent.
+ withdrawn: bool,
}
impl Answer {
- /// Answer the peer's request for `prefix` (relative to the session's root).
- pub(super) fn new(origin: origin::Consumer, prefix: Path, hidden: bool, auth: crate::auth::Link) -> Self {
+ /// Answer the peer from `origin` through the session's key `main` on its node.
+ /// `chains` sends each claim's Lite-06 Chain, which the peer's SETUP offers to read.
+ pub(in crate::lite) fn new(
+ origin: origin::Consumer,
+ auth: crate::auth::Link,
+ main: Arc,
+ chains: bool,
+ ) -> Self {
let link = route::Link {
- // The peer prices the link on what it receives; nothing here charges it.
- cost: 1,
- priority: 0,
upstream: origin.upstream(),
proxy: origin.proxy(),
- scope: None,
- // The node files the stream under its session.
..Default::default()
};
- let absolute = origin.root().join(&prefix).ok();
+ // Start from the grant the setup token earns, so the initial set never
+ // advertises something it would end a moment later.
+ let mut epoch = 0;
+ let permit = match auth.poll_permit(crate::auth::Direction::Publish, &mut epoch, &kio::Waiter::noop()) {
+ Poll::Ready(permit) => permit,
+ Poll::Pending => Default::default(),
+ };
Self {
- gate: crate::auth::Gate::new(auth, prefix, crate::auth::Direction::Publish),
- prefix: absolute,
origin,
- hidden,
link,
- sources: Vec::new(),
- turn: 0,
+ auth,
+ epoch,
+ permit,
+ main,
+ mounts: BTreeMap::new(),
+ next_source: MAIN + 1,
+ turn: MAIN,
+ prefixes: BTreeMap::new(),
+ next_prefix: 0,
+ request: None,
+ parts: None,
encoder: lite::AnnounceEncoder::new(Version::Lite07),
routes: HashMap::new(),
next_route: 0,
live: HashMap::new(),
- epoch: 0,
- permit: Default::default(),
- phase: Phase::Init,
- budget: kio::coop::Budget::new(32),
- peer_setup: None,
- chains: false,
- stall: None,
+ chains,
+ withdrawn: false,
}
}
- /// Send each claim's Lite-06 Chain if the peer's SETUP offers to read it, and name
- /// the peer to the node as its SETUP does.
- pub(super) fn with_peer_setup(mut self, peer_setup: lite::PeerSetup) -> Self {
- self.peer_setup = Some(peer_setup);
- self
- }
-
- /// Close `session` if the peer stops reading the stream.
- pub(super) fn with_session(mut self, session: Transport, runtime: crate::time::Deadlines) -> Self {
- self.stall = Some(Stall {
- session,
- runtime,
- armed: None,
- });
- self
+ /// Whether the peer's edit continues in its next message, which must be its next
+ /// ANNOUNCE_REQUEST.
+ pub(in crate::lite) fn continuing(&self) -> bool {
+ self.parts.is_some()
}
- /// Run the stream until the peer closes it, or `withdrawing` closes it here.
- pub(super) fn poll(
+ /// ANNOUNCE_REQUEST: the peer edits the prefixes it asks for, over as many messages as
+ /// `more` says. The edit takes effect once the node says where, in order with what
+ /// the node already queued for the peer.
+ pub(in crate::lite) fn request(
&mut self,
- stream: &mut Stream,
- withdrawing: bool,
- waiter: &kio::Waiter,
- ) -> Poll> {
- if matches!(self.phase, Phase::Init) {
- // Start from the grant the setup token earns, so the initial set never
- // advertises something it would end a moment later.
- ready!(self.gate.poll_setup_answered(waiter));
- if let Some(peer_setup) = &self.peer_setup {
- self.chains = ready!(peer_setup.poll_lite06_chain(waiter));
- }
- if let Poll::Ready(permit) = self.gate.poll_permit(&mut self.epoch, waiter) {
- self.permit = permit;
+ more: bool,
+ remove: Vec,
+ add: Vec<(Path, bool)>,
+ ) -> Result<(), Error> {
+ if self.withdrawn {
+ return Ok(());
+ }
+ // The peer sends one edit at a time.
+ if self.request.is_some() {
+ return Err(Error::ProtocolViolation);
+ }
+ let (mut remove, mut add) = (remove, add);
+ if let Some(parts) = self.parts.take() {
+ remove.splice(0..0, parts.remove);
+ add.splice(0..0, parts.add);
+ }
+ // Parts that could never make a valid edit fail before more of them pile up: their
+ // Adds alone already pass the set's limit, or they remove more than the set holds.
+ if remove.len() > self.prefixes.len() || !lite::request_fits(add.iter().map(|(path, _)| path), Version::Lite07)
+ {
+ return Err(Error::ProtocolViolation);
+ }
+ if more {
+ self.parts = Some(Parts { remove, add });
+ return Ok(());
+ }
+ let mut left: BTreeMap = self.prefixes.iter().map(|(id, prefix)| (*id, prefix)).collect();
+ let mut edit = route::path::Edit {
+ remove: Vec::new(),
+ add: Vec::new(),
+ };
+ for id in &remove {
+ let prefix = left.remove(id).ok_or(Error::ProtocolViolation)?;
+ edit.remove.extend(prefix.path.clone());
+ }
+ let mut set: HashSet = left.values().map(|prefix| prefix.relative.clone()).collect();
+ let mut added = Vec::new();
+ for (relative, hidden) in add {
+ if !set.insert(relative.clone()) {
+ return Err(Error::ProtocolViolation);
}
- stream.writer.buffer(&lite::AnnounceOk {
- origin: crate::Hop::UNKNOWN,
- active: 0,
- })?;
- let peer = match &self.peer_setup {
- Some(peer_setup) => node::Peer {
- tag: peer_setup.tag(),
- node: peer_setup.node().map_or(0, |node| node.get()),
- },
- None => node::Peer {
- tag: lite::session_tag(),
- node: 0,
- },
+ let path = self.origin.root().join(&relative).ok();
+ edit.add.extend(path.clone());
+ let prefix = Prefix {
+ relative,
+ path,
+ hidden,
+ synced: false,
};
- let node = self.origin.route_node();
- let mounts = (self.prefix.as_ref()).map_or_else(Vec::new, |prefix| self.origin.mounted(prefix));
- let interests = (self.prefix.iter())
- .map(|prefix| (prefix.clone(), None))
- .chain(mounts.into_iter().map(|mount| (mount.interest.clone(), Some(mount))));
- for (interest, mount) in interests {
- let mut stream =
- (node.answer(peer, self.link.clone(), interest)).map_err(|_| Error::ProtocolViolation)?;
- if let Some(stall) = &self.stall {
- stream = stream.with_clock(stall.runtime.clone());
- }
- self.sources.push(Source { stream, mount });
- }
- for index in 0..self.sources.len() {
- while let Some(output) = self.sources[index].stream.try_output() {
- self.output(stream, index, output)?;
- }
+ added.push((self.next_prefix, prefix));
+ self.next_prefix += 1;
+ }
+ // No prefix in the set lies below another, and the whole set fits one message.
+ let nested = |prefix: &Path| {
+ route::ancestors(prefix)
+ .filter(|above| above != prefix)
+ .any(|above| set.contains(&above))
+ };
+ if set.iter().any(nested) || !lite::request_fits(set.iter(), Version::Lite07) {
+ return Err(Error::ProtocolViolation);
+ }
+ self.main.edit(edit).map_err(|_| Error::ProtocolViolation)?;
+ self.request = Some(Request { remove, add: added });
+ Ok(())
+ }
+
+ /// The grant changed: end what it no longer covers, start what it now does, and
+ /// floor every route again, which certifies only while the grant covers the prefix.
+ pub(in crate::lite) fn poll_permit(&mut self, writer: &mut Writer, waiter: &kio::Waiter) -> Result<(), Error> {
+ let direction = crate::auth::Direction::Publish;
+ while let Poll::Ready(permit) = self.auth.poll_permit(direction, &mut self.epoch, waiter) {
+ if permit == self.permit || self.withdrawn {
+ continue;
}
- // Every route the peer holds has a floor before SYNCED; one the node has
- // not certified yet certifies nothing.
- for sent in self.routes.values_mut() {
- if let Some(wire) = &sent.wire
- && !sent.floored
- {
- sent.floored = true;
- stream.writer.buffer(&Msg::Floor {
- id: wire.id,
- revision: None,
- })?;
+ self.permit = permit;
+ let mut keys: Vec = self.routes.keys().cloned().collect();
+ keys.sort_unstable();
+ for key in keys {
+ let paths: Vec = self.routes[&key].claims.keys().cloned().collect();
+ for path in paths {
+ self.sync(writer, &key, path)?;
}
+ self.floor(writer, &key)?;
+ self.settle(writer, &key)?;
}
- stream.writer.buffer(&Msg::Synced)?;
- self.phase = Phase::Running;
}
+ Ok(())
+ }
- loop {
- ready!(self.budget.poll_yield(waiter));
- // Deliver what is buffered before taking more work.
- if let Poll::Ready(flushed) = stream.writer.poll_flush(waiter) {
- flushed?;
- if let Some(stall) = &mut self.stall {
- stall.armed = None;
- }
- } else {
- if let Some(stall) = &mut self.stall {
- // A reader that keeps reading a trickle while its routes fail faster
- // is cut off too, before the node holds every failure for it.
- if (self.sources.iter()).any(|source| source.stream.poll_overrun(waiter).is_ready()) {
- stall.close("announce stream fell behind");
- return Poll::Ready(Err(Error::Timeout));
- }
- ready!(stall.poll(stream.writer.written(), waiter));
- return Poll::Ready(Err(Error::Timeout));
+ /// Ready once the peer reads its stream too slowly for what the node queued.
+ pub(in crate::lite) fn poll_overrun(&self, waiter: &kio::Waiter) -> Poll<()> {
+ let mut streams = std::iter::once(&*self.main).chain(self.mounts.values().map(|mount| &mount.stream));
+ match streams.any(|stream| stream.poll_overrun(waiter).is_ready()) {
+ true => Poll::Ready(()),
+ false => Poll::Pending,
+ }
+ }
+
+ /// Act on one thing a mount queued, if any has.
+ pub(in crate::lite) fn poll_mounts(
+ &mut self,
+ writer: &mut Writer,
+ waiter: &kio::Waiter,
+ ) -> Poll> {
+ let after = self.mounts.range(self.turn + 1..);
+ let ready = after
+ .chain(self.mounts.range(..=self.turn))
+ .find_map(|(source, mount)| match mount.stream.poll_output(waiter) {
+ Poll::Ready(output) => Some((*source, output)),
+ Poll::Pending => None,
+ });
+ let Some((source, output)) = ready else {
+ return Poll::Pending;
+ };
+ self.turn = source;
+ Poll::Ready(self.sourced(writer, source, output))
+ }
+
+ /// Act on what the node queued for the peer on the session's key.
+ pub(in crate::lite) fn output(&mut self, writer: &mut Writer, output: node::Output) -> Result<(), Error> {
+ self.sourced(writer, MAIN, output)
+ }
+
+ fn sourced(&mut self, writer: &mut Writer, source: Source, output: node::Output) -> Result<(), Error> {
+ if self.withdrawn {
+ return Ok(());
+ }
+ match output {
+ node::Output::Route { id, prefix, reach } => self.route(writer, source, id, prefix, reach),
+ node::Output::Batch {
+ id,
+ prefix,
+ changes,
+ floor,
+ } => self.batch(writer, source, id, prefix, changes, floor),
+ node::Output::Dead(dead) => self.dead(writer, source, dead),
+ node::Output::Edited { routes } => self.edited(writer, routes),
+ node::Output::Synced(path) => {
+ let id = (self.prefixes.iter())
+ .find(|(_, prefix)| !prefix.synced && prefix.path.as_ref() == Some(&path))
+ .map(|(id, _)| *id);
+ match id {
+ Some(id) => self.synced(writer, id),
+ None => Ok(()),
}
- return Poll::Pending;
}
+ // The other half of the session serves, and asks.
+ node::Output::Serve { .. } | node::Output::Unserve { .. } | node::Output::Ask(_) => Ok(()),
+ }
+ }
- match self.phase {
- Phase::Withdrawing => {
- stream.writer.finish()?;
- self.phase = Phase::Closing;
+ /// The session is leaving: let go of the node, so nothing more is queued, and end
+ /// every route.
+ pub(in crate::lite) fn withdraw(&mut self, writer: &mut Writer) -> Result<(), Error> {
+ if std::mem::replace(&mut self.withdrawn, true) {
+ return Ok(());
+ }
+ self.main.withdraw();
+ self.mounts.clear();
+ let mut ids: Vec = (self.routes.drain())
+ .filter_map(|(_, sent)| sent.wire.map(|wire| wire.id))
+ .collect();
+ ids.sort_unstable();
+ for id in ids {
+ writer.buffer(&Msg::RouteEnd { id })?;
+ }
+ self.live.clear();
+ Ok(())
+ }
+
+ /// The requested prefix `path` lies in, and its Prefix ID.
+ fn region(&self, path: &Path) -> Option<(u64, &Path)> {
+ self.prefixes.iter().find_map(|(id, prefix)| {
+ let prefix = prefix.path.as_ref()?;
+ route::within(path, prefix).then_some((*id, prefix))
+ })
+ }
+
+ /// The route `source` presents at the origin-side `prefix`, named as the handle names
+ /// it, within a requested prefix.
+ fn name(&self, source: Source, prefix: &Path) -> Option {
+ let named = match source {
+ MAIN => prefix.clone(),
+ _ => self.mounts.get(&source)?.mount.name(prefix)?,
+ };
+ self.region(&named).is_some().then_some(named)
+ }
+
+ /// The source presenting the claim at the handle-side `path`: the mount it lies
+ /// beneath, or the session's own.
+ fn source(&self, path: &Path) -> Source {
+ let mut mounts = self.mounts.iter();
+ mounts
+ .find(|(_, mount)| mount.mount.holds(path))
+ .map_or(MAIN, |(source, _)| *source)
+ }
+
+ /// The node put the peer's last request in effect, leaving the session's key the
+ /// routes `carried`: answer it, carry the wire's routes over, and send what they lack.
+ fn edited(&mut self, writer: &mut Writer, carried: Vec