Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m2/cluster-routes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ The items below were decided 2026-11-10 (maintainer, /quest-plan on lite-07 rout

30. ✅ Decided 2026-11-12 (maintainer, /quest-audit before the cut), after a survey of BGP, EIGRP, and their successors against item 27: most of the path rule already matches that prior art (AS_PATH loop detection, root-cause withdrawal, graceful restart, End-of-RIB, outbound route filtering, and item 29's Only-To-Customer flag). Three gaps are taken, all in [the switch](https://github.com/moq-dev/moq/pull/8731) and [acceptance](/quest/m2/cluster-routes/acceptance.md): **a dead-hop message on a scoped session applies only to the routes heard on that session and casts no doubt**, since in BGP a neighbour withdraws only what it sent, and a scoped peer (a federated relay, later a browser) could otherwise name a core hop it only read in a path and drop healthy routes outside its scope; **holds are bounded in time**, a neighbour's missing answer counting as nothing after a deadline, and a session whose announce stream has stopped draining is closed, which avoids EIGRP's stuck-in-active and BGP's zombie routes (RFC 9687), and is safe because a hold protects presence, not loop freedom; and **three tests before any design change**: neighbours noticing a dead relay up to the 3 s keep-alive apart (to size the doubt timer, measured in hops of delay today), a three-relay priority dispute wheel (RFC 4264's wedgies, since priority ranks above cost), and a primary that fails and returns. No wire bytes change; the first defines what the dead-hop message means. Rejected: declaring the link relation in SETUP and refusing leaked routes on receipt (RFC 9234), since the accepting side already knows the relation from its grant and it adds wire for a misconfiguration; MRAI and route-flap damping, which exist because BGP delays announcements; link state; per-hop signatures.

31. Reopened 2026-11-15 (maintainer, /quest-plan on the whole route layer): ✅ **this rule is the baseline, and [cache-aware multicast](/quest/m2/multicast/README.md) replaces it on lite-07-wip before the cut.** An audit of what is built found requirements the path rule does not meet: its exhaustive checking stops at 3 relays and rests on a timer that outlasts every stale offer, route state follows publishers, nothing prefers a warm copy, and a stalled route is never left. Superseded there: item 18 (every publisher an origin node), item 19 (ROUTE_HOLD), item 26's route per scope, item 27's doubt and remembered dead hops, the Route Key and qualified Node IDs of the contract below, and item 14. Standing: the path with its self-naming rule and the withdrawal that names a dead hop (item 27), revisions and floors, one announce stream per side with its interest set (items 21 to 24), the sticky down-only flag (item 29), and drain, then priority, then cost. This epic's children still land: they are what the [bake-off](/quest/m2/multicast/bakeoff.md) measures against. Rejected: repairing this rule in place; freezing lite-07 with it and redesigning in lite-08.
31. Reopened 2026-11-15 (maintainer, /quest-plan on the whole route layer): ✅ **this rule is the baseline, and [cache-aware multicast](/quest/m2/multicast/README.md) replaces it on lite-07-wip before the cut.** An audit of what is built found requirements the path rule does not meet: its exhaustive checking stops at 3 relays and rests on a timer that outlasts every stale offer, route state follows publishers, nothing prefers a warm copy, and a stalled route is never left. Superseded there: item 18 (every publisher an origin node), item 19 (ROUTE_HOLD), item 26's route per scope, item 27's doubt and remembered dead hops, the Route Key and qualified Node IDs of the contract below, and item 14. Standing: the path with its self-naming rule and the withdrawal that names a dead hop (item 27), revisions and floors, one announce stream per side with its interest set (items 21 to 24), the sticky down-only flag (item 29), and drain, then priority, then cost. This epic's children still land: they are what the [bake-off](https://github.com/moq-dev/moq/pull/8880) measures against. Rejected: repairing this rule in place; freezing lite-07 with it and redesigning in lite-08.

The route layer's core landed in
[moq-dev/moq#7676](https://github.com/moq-dev/moq/pull/7676): a
Expand Down
5 changes: 3 additions & 2 deletions quest/m2/cluster-routes/doubt-holds.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ go while its neighbours answer that they have nothing.

Reordered 2026-11-15 (maintainer, /quest-plan on [cache-aware
multicast](/quest/m2/multicast/README.md)): this waits on the
[bake-off](/quest/m2/multicast/bakeoff.md). The new rule has neither holds nor
[bake-off](https://github.com/moq-dev/moq/pull/8880). The new rule has neither holds nor
doubt, so nothing would be left for this to fix. If the new rule goes ahead,
[the switch](/quest/m2/multicast/switch.md) deletes this quest with the
fixtures that pin it; if it does not, this starts as written.
Expand Down Expand Up @@ -43,7 +43,8 @@ before changing `rs/moq-net/src/route/path.rs`.

## Required

- [The bake-off](/quest/m2/multicast/bakeoff.md) - decides whether the hold and the doubt this fixes remain
- [The switch](/quest/m2/multicast/switch.md) - deletes this with the old rule, unless the maintainer keeps today's rule on [moq-dev/moq#8880](https://github.com/moq-dev/moq/pull/8880)


## Related

Expand Down
5 changes: 3 additions & 2 deletions quest/m2/cluster-routes/doubt-keepalive.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ apart its neighbours notice the death within the keep-alive.

Reordered 2026-11-15 (maintainer, /quest-plan on [cache-aware
multicast](/quest/m2/multicast/README.md)): this waits on the
[bake-off](/quest/m2/multicast/bakeoff.md). The maintainer deferred how the
[bake-off](https://github.com/moq-dev/moq/pull/8880). The maintainer deferred how the
doubt covers a silent death on 2026-11-15, "not yet sure about the doubt
design", and decided the same day that evidence replaces the doubt. If the new
rule goes ahead, [the switch](/quest/m2/multicast/switch.md) deletes this
Expand Down Expand Up @@ -37,7 +37,8 @@ that window goes to a dead relay. A 1 s doubt does not cover a 3 s spread.

## Required

- [The bake-off](/quest/m2/multicast/bakeoff.md) - decides whether the doubt this sizes remains
- [The switch](/quest/m2/multicast/switch.md) - deletes this with the old rule, unless the maintainer keeps today's rule on [moq-dev/moq#8880](https://github.com/moq-dev/moq/pull/8880)


## Related

Expand Down
5 changes: 3 additions & 2 deletions quest/m2/cluster-routes/held-after-close.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ route recovers.

Reordered 2026-11-15 (maintainer, /quest-plan on [cache-aware
multicast](/quest/m2/multicast/README.md)): this waits on the
[bake-off](/quest/m2/multicast/bakeoff.md). The new rule has no hold: a reader
[bake-off](https://github.com/moq-dev/moq/pull/8880). The new rule has no hold: a reader
debounces unavailable to offline locally, which answers the same symptom. If
the new rule goes ahead, [the switch](/quest/m2/multicast/switch.md) deletes
this quest with the fixtures that pin it; if it does not, this starts as
Expand Down Expand Up @@ -42,7 +42,8 @@ three pinned fixtures into the contract's held-then-available sequence.

## Required

- [The bake-off](/quest/m2/multicast/bakeoff.md) - decides whether the hold this shows remains
- [The switch](/quest/m2/multicast/switch.md) - deletes this with the old rule, unless the maintainer keeps today's rule on [moq-dev/moq#8880](https://github.com/moq-dev/moq/pull/8880)


## Related

Expand Down
5 changes: 3 additions & 2 deletions quest/m2/cluster-routes/route-fanout.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ remembered for sessions nothing was learned on.

Reordered 2026-11-15 (maintainer, /quest-plan on [cache-aware
multicast](/quest/m2/multicast/README.md)): this waits on the
[bake-off](/quest/m2/multicast/bakeoff.md). Its first and third findings are
[bake-off](https://github.com/moq-dev/moq/pull/8880). Its first and third findings are
in `path::Router`'s indexes and its doubts and dead hops, which the new router
replaces; the second, per-broadcast rewrites in `route/node.rs`, the
[router](/quest/m2/multicast/router.md) takes as a requirement. If the new
Expand Down Expand Up @@ -54,7 +54,8 @@ which also edits `path.rs`.

## Required

- [The bake-off](/quest/m2/multicast/bakeoff.md) - decides whether the path router these findings are in remains
- [The switch](/quest/m2/multicast/switch.md) - deletes this with the old rule, unless the maintainer keeps today's rule on [moq-dev/moq#8880](https://github.com/moq-dev/moq/pull/8880)


## Related

Expand Down
84 changes: 71 additions & 13 deletions quest/m2/multicast/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Settled:
interoperate for a release; lite-07 without a route layer, a large strip
that leaves lite-07 clusters with lite-06's hunting). The route stream and
the acceptance fixtures still land: they are the baseline the
[bake-off](/quest/m2/multicast/bakeoff.md) measures against, and one
[bake-off](https://github.com/moq-dev/moq/pull/8880) measures against, and one
announce stream per side is wanted by any design. The quests that only
polish holds, doubt, or the path router's indexes wait on the bake-off.
3. ✅ **A routing node is a node that forwards and is trusted to; claims,
Expand All @@ -95,8 +95,9 @@ Settled:
the prototype).
4. ✅ **Evidence, not timers.** The route layer is a plain path vector with
the dead-hop withdrawal as an accelerator: no ROUTE_HOLD, no doubt, and no
dead hop remembered past the routes that name it. Routing state is a
hint. A request is confirmed only once its upstream is, data is identified
dead hop remembered past the routes that name it (that last clause
reversed 2026-11-16 by item 13: a session floor is kept for good).
Routing state is a hint. A request is confirmed only once its upstream is, data is identified
by Instance, track, group, and frame, and a stale or stalled route is left
on evidence ([requests](https://github.com/moq-dev/moq/pull/8881),
[route health](/quest/m3/route-health/README.md)). What a hold protected,
Expand Down Expand Up @@ -168,10 +169,58 @@ flagged as costly, on
standby wherever a second route exists, which about doubles a track's
subscription state across the cluster).

Decided 2026-11-16 (maintainer, "go for it" on all three, on
[the bake-off](https://github.com/moq-dev/moq/pull/8880#issuecomment-6911055514),
whose tables hold every number here; 32 relays on CDN tiers with 2
publishers, in route messages, unless noted):

13. ✅ **The new rule replaces today's, with five additions, none a timer**
(rejected: the rule as written, which hunts through 6,625,456 for a
partition against today's 579 and 1,368,094 for a relay that drains and
closes; keeping today's rule, which leaves readers unfed about 1 s when
a link closes beside a detour, grows with publishers to 73,472 selected
routes at 256, and cannot cross two scoped grants). With all five a
partition costs 690, a link closing feeds every reader at once, and a
publisher leaving changes no route. No arm showed a false END or a
revival.
- `floor`: per pair of relays, a node keeps the highest session it knows
closed and refuses any path naming that pair at or below it, for good.
This reverses item 4's "no dead hop remembered past the routes that
name it": that clause is what the rule hunted through, a stale offer
arriving after the hop was forgotten (rejected: `quiet`, holding
adverts until neighbours agree, which can wait for good and cost 4,641
against 2,037 as written for a relay dying on a 12-relay chorded ring;
`agree`, refusing a route a neighbour's own offer contradicts, a small
gain `floor` covers).
- `epoch`: a relay's identity carries a start that only grows, and a
session orders by its relay's start and then its number, so a restart
replaces a pair's floor where it added one: 4 pairs after each of 8
restarts against 4 more per restart, with no row of any table changed.
- `batch`: a node advertises once it has read what was delivered, not
after every message. `floor` alone still spent 28.8 million on one
GOAWAY of a 32-relay chorded ring; with `batch`, 5,065.
- `drain`: a node that sent GOAWAY is named once per neighbour in a
message of its own, as a closed session is: 5,065 to 723.
- `claimed`: a node offers a route to itself only once it announces a
claim, and from then on until it restarts, so a route is per announcing
relay. Building the mesh for one publisher fell from 6,576 to 207,
under today's 221, and the partition from 7,549 to 690. A publisher's
first claim on a relay then costs a route at every node (201 for a
move). The route stands after the last claim leaves (rejected:
withdrawing it, which races the claim's own end and would show a
reader offline where the publisher ended cleanly).
14. ✅ **A silent death is left to keep-alives and the opt-in metadata
standby** ([route health](/quest/m3/route-health/README.md)). No arm
fixed it: readers behind a relay that dies unnoticed for 3 s stay unfed
until the last session notices, as today (rejected: `suspect`, ranking
the far end of a closed session as draining until its own word arrives,
which shortened some silent deaths, nearly doubled a flap from 656 to
1,133, and left 20 to 42 suspicions that never lifted).

Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit, which
judged the route layer at this plan's end state):

13. ✅ **The plan stands as item 2 has it** (rejected: freezing lite-07 with
15. ✅ **The plan stands as item 2 has it** (rejected: freezing lite-07 with
claims only and moving the route exchange behind a negotiated SETUP
parameter with a version of its own, so the cut stops waiting on the
router and the switch and a third-party client implements no route
Expand All @@ -184,20 +233,29 @@ judged the route layer at this plan's end state):
a non-forwarding endpoint must parse, and the moq-cluster draft, which
still specifies the warm discount item 5 forbids.

The shape those decisions give, for the children to confirm or improve; each
departure is a decision to list in its PR:
The shape those decisions give. The bake-off confirmed or changed each line;
a further departure is a decision to list in its PR:

- **Identity.** A path and an Instance name content, and a request names the
Instance. A relay holds one upstream subscription per Instance and track,
whichever origin node its downstream requests named, so a join stops at the
first node already carrying it.
- **Reachability.** One route per forwarding node: its path, drain, priority,
and cost. A node never uses a route whose path names it. Rank is not
draining, then the lowest priority on the path, then the summed cost, each
link adding at least 1, so extending a route never improves it.
- **Claims.** A prefix, Instance, Epoch, and cost, announced by a forwarding
node for itself and its local endpoints, ordered by the node's revision and
certified by floors as today, which measured no revival and no false END.
- **Reachability.** One route per announcing relay, offered once it has a
claim: its path, drain, priority, and cost. A node never uses a route
whose path names it, sends none to a neighbour already on the path, and
refuses one at or below a session floor. Rank is not draining, then the
lowest priority on the path, then the summed cost, each link adding at
least 1, so extending a route never improves it. A closed session and a
draining node are each named once per neighbour, and a closed session's
message ends every route its sender offered through it. A node advertises
once per burst of input.
- **Claims.** A prefix, Instance, Epoch, cost, the lowest link priority to
its publisher, and whether the way there is draining, announced by a
forwarding node for itself and its local endpoints, ordered by the node's
revision and certified by floors as today, which measured no revival and
no false END. A claims message carries its floor even with nothing
changed, and a withdrawal says whether the publisher ended the claim or
its announcer lost it.
- **Boundaries.** The chain of item 3.
- **Requests.** A request names the origin node its first hop chose, so
relays that rank differently never loop it, and fails closed.
Expand Down
Loading