Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions quest/m1/offline-cursor-fallback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# [S] A scoped cursor reports a lost route as offline when a broader route takes over

## Goal

A reader on a scoped announcement cursor whose exact route is lost with its
session, while a broader route still covers the same presented path, sees the
lost route go offline, never an END. An END still means the publisher
withdrew.

## Plan

Planned 2026-11-10 (maintainer, /quest-plan on lite-07 routing). Milestone:
✅ m1 (rejected: m2 in the routes epic; m0).

Found by the automated review of
[moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375), posted fifteen
minutes before it merged and never addressed. With a covering cursor rooted
at `room/alice`, an exact forwarded route presents with captures `Some([])`
and the broader `room` fallback with `None`. Losing the exact route's session
passes `Gone::Offline` into the update, but the capture-change branch
(`rs/moq-net/src/model/origin.rs`, about line 1753, the arm that replaces a
presented route whose captures differ) retracts the old identity with
`Gone::End`. Readers then see End, as if the publisher had withdrawn.

Reproduce it first by extending
`capture_change_retracts_before_reannouncing_a_presented_prefix` with a
forwarded route lost with its session. Then carry the actual removal reason
through that replacement instead of a fixed `Gone::End`.

Check the bindings that surface the reason (`moq-ffi`, and the Swift, Kotlin,
Python, and C wrappers of `announced`) need no change beyond the Rust fix.

Caused by: [moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375), which
introduced the offline reason and left this branch on End.

Public API: none. Wire: none.

## Related

- [A scoped JS handle that closes reports its routes offline](/quest/m1/offline-js-handle.md) - the same misreport in `js/net`
39 changes: 39 additions & 0 deletions quest/m1/offline-js-handle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# [S] A scoped JS handle that closes reports its routes offline

## Goal

In `@moq/net`, routes a session received through a scoped `Producer` go
offline when that scoped handle closes while the parent origin stays open.
The parent's announcement reader emits `offline`, not `end` or `absent`,
unless the announcer ended the route on the wire.

## Plan

Planned 2026-11-10 (maintainer, /quest-plan on lite-07 routing). Milestone:
✅ m1 (rejected: m2 in the routes epic; m0).

Found by the automated review of
[moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375), posted fifteen
minutes before it merged and never addressed. Closing the scoped handle runs
the raw callbacks `#hold` registered (`js/net/src/origin.ts`, about line
1695) and skips `close(lost)`, so received entries keep `lost = false` and
the reader reports a withdrawal. A later route close cannot repair it,
because `ended` is already true.

Reproduce it first in `js/net/src/origin.test.ts`: receive a route through a
scoped handle, close the handle, and read the parent's announcements. Then
put the reason in the shared release transition, so a received route's
teardown defaults to offline and only an explicit wire end selects
withdrawal.

Compare with Rust's scoped teardown and say in the PR whether it has the same
gap; if it does, fix it in the same PR, as `moq-net` and `js/net` stay in
step.

Caused by: [moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375).

Public API: none. Wire: none.

## Related

- [A scoped cursor reports a lost route as offline when a broader route takes over](/quest/m1/offline-cursor-fallback.md) - the same misreport in the Rust cursor
56 changes: 56 additions & 0 deletions quest/m1/route-seqno-bumps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# [S] A node advances its route seqno only when a route link changes

## Goal

A lite-07 node raises its own route Seqno unprompted only when a session it
exchanges routes over opens or closes, and otherwise only to answer a
ROUTE_REQUEST. A client connecting to or leaving a relay no longer makes the
relay send a ROUTE_UPDATE for its own identity on every link that carries it.
A test counts the ROUTE messages a mesh sends per client session and holds
them at zero.

## Plan

Decided 2026-11-10 (maintainer, /quest-plan on lite-07 routing), asked when
an origin should advance its seqno unprompted: ✅ only when a route link
changes (rejected: only to answer a ROUTE_REQUEST, which makes every path
change that lengthens a route wait a round trip to the origin; keeping
today's per-stream bump until a sweep measures it). Milestone: ✅ m1
(rejected: m2 in the routes epic; m0), since relays run lite-07 in production
now and this changes no wire.

Why it exists: relays that remember a short distance to a node refuse its
new, longer path until the node issues a higher seqno. Bumping when its links
change saves the ROUTE_REQUEST round trip.

What is wrong today: `register` and `Drop for Stream` in
`rs/moq-net/src/route/node.rs` call `router.bump()` for every announce stream
of every lite-07 session, and `Routed::limit` in
`rs/moq-net/src/lite/subscriber/routed.rs` registers again on a permit
change. A relay originates the claims of its lite-06 and moq-transport
publishers under its own identity, so each client session bumps that identity
at least four times. This was read from the code, not measured: reproduce it
first with a count of ROUTE messages per client connect and disconnect on a
three-relay mesh.

Open for the implementer, with a recommendation: what counts as a route
link. Recommended: a session whose peer forwards, known because it has sent a
route to a node other than itself, plus every session of a node that does not
forward (all of a publisher's links are route links). A session that only
ever carries claims, or only the peer's own route, never bumps. Confirm the
choice against `reconnect_without_restart_needs_a_seqno_advance` and
`restart_reusing_identity_is_refused` in `route/tests/`, which the bump
exists for, and say in the PR if a narrower rule (bump on close only, since a
new link lengthens no path) also keeps them passing.

Mirror any change to `bump`'s callers in `RouteHandoff.tla` as the
`verify-route` skill says, and state the ROUTE budget per session event so
[the mesh sweep](/quest/m1/route-mesh-sweep.md) can assert it.

Public API: none. Wire: none.

## Related

- [Routes and announces](/quest/m2/cluster-routing/routes/README.md) - the contract's seqno rule this narrows
- [Seeded lite-07 meshes hold the route rules and an announce budget](/quest/m1/route-mesh-sweep.md) - the budget that would have caught this
- [Each side says everything about routing on one stream](/quest/m2/cluster-routing/routes/route-stream.md) - registers one neighbour per session, which changes where the bump is called
18 changes: 16 additions & 2 deletions quest/m2/cluster-routing/routes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,17 @@ claim whose withdrawal is already known.
16. ✅ Decided 2026-11-09 (maintainer, /quest-unblock on [moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375)), asked how a token-scoped lite-07 session certifies withdrawals so a clean finish ENDs across a relay: ✅ **the subscriber narrows its ANNOUNCE_REQUEST to its scope's single region** (where its limit and its subscribe grant meet), and reopens the request when that region changes so a widening renewal isn't lost. Floors keep their exact whole-prefix meaning, with no wire change and one SHOULD in the lite-07 draft. A multi-region scope still certifies nothing. Rejected: floors carrying regions on the wire, which reverses item 10; landing with the finish lanes accepting offline.
17. ✅ Decided 2026-11-09 (maintainer, /quest-unblock on [moq-dev/moq#8375](https://github.com/moq-dev/moq/pull/8375)), asked how item 16's reopen counts the session's announce cap while the old request keeps serving until the new one is SYNCED: ✅ **the old request gets its own budget for the handover**, moving what it holds onto a separate counter with the same cap, so the session's count belongs wholly to the new request. A peer can make the session hold at most twice the cap, and only during one handover. Rejected: dropping the old request at once on every region change, which flickers claims offline; holding a client's first request until its credential is answered and dropping the old one on later renewals.

The items below were decided 2026-11-10 (maintainer, /quest-plan on lite-07 routing), which reopened the route layer's rules after the echo ordering bug.

18. ✅ Asked who is an origin node: **every publisher stays an origin node.** Its identity is for signing, for one node across several links (redundant publishers), and because browsers will forward ([browser mesh](/quest/m5/browser-mesh.md)). Rejected: a relay originating, under its own Node ID, the claims of any peer that sends no Node ID or holds a scoped publish grant, as it already does for lite-06 and moq-transport publishers. That would make route state follow relays rather than publishers and remove qualified Node IDs, the Route Key, and the echo, but a publisher would have no identity of its own. Accepted cost: routes, holds, and seqno requests scale with publishers, so a node must still forget dead identities, and the echo that makes forgetting safe has to be sound.
19. ✅ Asked whether to keep them: **ROUTE_HOLD stays, and the newest floor wins per identity.** Rejected: dropping holds, where each stall takes every claim behind the node offline and re-sends them all; and claims that follow only the selected next hop, which re-pulls a node's whole claim table on each next-hop change and showed revivals and false ENDs in the sweeps below.
20. ✅ Asked when an origin advances its seqno unprompted: **only when a route link changes**, which [the seqno bumps](/quest/m1/route-seqno-bumps.md) implements. Today every announce stream open or close bumps it. Rejected: only to answer a ROUTE_REQUEST; keeping the per-stream bump.
21. ✅ Asked how one side's routing messages travel so none overtakes another: **one unidirectional Announce Stream per side**, carrying its requests, its answers, and its echoes in order ([the route stream](/quest/m2/cluster-routing/routes/route-stream.md)). It replaces "There MAY be multiple Announce Streams". Rejected: one bidirectional Announce Stream per direction with the echo moved to the answering half; one bidirectional stream per session.
22. ✅ **A side's interest is a set of prefixes edited in place.** The live set never nests, and one edit may swap prefixes for a wider or narrower one without re-sending what both cover ([the interest set](/quest/m2/cluster-routing/routes/interest-set.md)). A token-scope change is such an edit, with one prefix per region of the scope. This supersedes items 16 and 17, and the handoff contract's "An interest change opens a new stream" below, once the route stream lands.
23. ✅ Asked what a floor covers with several prefixes on one stream: **one prefix, named by its id.** This amends item 10: the floor is still a marker and never a region list. Rejected: a floor over the whole set.
24. ✅ Asked how a side learns a newly added prefix's initial set is complete: **SYNCED per prefix, naming its id**, plus one marker per edit where it takes effect for both sides. Rejected: one SYNCED per edit, counted in order.
25. ✅ Asked how a relay safely forgets a publisher that is gone, after the model showed a per-neighbour echo is not enough once the origin sits behind a relay: **each route names the relays it crossed, and a node never uses a route that names itself** ([the fix](/quest/m2/cluster-routing/routes/echo-order.md), which holds the counterexamples and the model results). Feasibility still selects routes; the list is one per origin route, not the per-claim Hop ID list item 6 removed, and the echo and the drain rule stay as they are. This supersedes the claim, in the handoff contract and in decision 6 below, that the echo alone makes forgetting safe. First decided as a relay echoing a ROUTE_END only once its own downstream had; the model ran that the same day and it fails when a session closes before answering. Rejected: that deferred echo; never dropping the distance; a timer; relays as the only origin nodes.


The route layer's core landed in
[moq-dev/moq#7676](https://github.com/moq-dev/moq/pull/7676): a
Expand Down Expand Up @@ -180,8 +191,11 @@ the bullets below are the contract it carries.
accept a route if its seqno is newer, or equal with a metric below the
feasibility distance. Only the origin advances its seqno. Retraction is
ROUTE_HOLD or ROUTE_END. The route model found a count to infinity
(2026-11-10): an echo can overtake a route derived from the echoed one;
[the echo fix](/quest/m2/cluster-routing/routes/echo-order.md) orders it.
(2026-11-10): an echo can overtake a route derived from the echoed one,
and even in order a per-neighbour echo misses a derived copy held by a
third relay; [the fix](/quest/m2/cluster-routing/routes/echo-order.md)
adds the relays a route crossed, and a node refuses one naming itself
(Plan item 25).
- Starvation recovery is Babel's seqno request (RFC 8966 s3.8, decided
2026-10-12). A node whose routes to an origin are all infeasible sends
ROUTE_REQUEST for that origin, with its feasibility seqno plus one, to the
Expand Down
Loading
Loading