Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
c1f1e7c
wip(net): pump a route's copy into a single-writer logical track
kixelated Oct 2, 2026
ee1dca4
wip(net): fronts serve tracks through a pump; IETF resumes mid-group
kixelated Oct 2, 2026
3a28cfa
wip(net): pump parks its cache, F2/F3 front fixes, lite drops unreque…
kixelated Oct 2, 2026
e17c0bd
wip(net): pump fetch semantics, track end, IETF widening tests
kixelated Oct 2, 2026
0f29216
wip(net): pump keeps in-flight groups past a forget, mirrors the sour…
kixelated Oct 2, 2026
74b5c2c
Merge origin/main into claude/subscription-route-change-audit-5ff44a
kixelated Oct 3, 2026
005db7b
fix(net): pump drops its input on forget, mirrors the first copy's st…
kixelated Oct 3, 2026
906f47f
test(net): a front never resumes a peer onto a route through that peer
kixelated Oct 3, 2026
80ef326
bench(net): delivery through a front, swept over tracks and readers
kixelated Oct 3, 2026
269946f
fix(net): an ended track's pump copies a group only while a reader ho…
kixelated Oct 3, 2026
5afa100
drafts: a relay resumes a moved subscription from the first frame the…
kixelated Oct 3, 2026
de7b51c
fix(net): drop a resumed group's delivered head on IETF ingest, skip …
kixelated Oct 3, 2026
c6fda23
chore(net): box the pump's feed command, format
kixelated Oct 3, 2026
58f0973
refactor(net): delete the route splicer and its spliced modes
kixelated Oct 3, 2026
3f6d910
doc: describe route failover through the pump
kixelated Oct 3, 2026
8b2528a
fix(net): a parked track lets its next source pick the start
kixelated Oct 3, 2026
0ae456c
fix(net): the pump copies every group and leaves the budget to readers
kixelated Oct 3, 2026
d643f58
test: let the origin relay writes before polling through a front
kixelated Oct 3, 2026
7c6cd8f
Merge remote-tracking branch 'origin/main' into claude/subscription-r…
kixelated Oct 3, 2026
b089f60
fix(net): a new source re-queries tracks still draining an older one
kixelated Oct 3, 2026
1c069f8
fix(net): pump keeps held groups past a park, refetches what a health…
kixelated Oct 3, 2026
9b15acb
chore(net): drop the pump's unread complete flag
kixelated Oct 3, 2026
8b109d5
feat(net)!: a path is one broadcast, whoever serves it
kixelated Oct 3, 2026
3523eee
test(mux): let the old broadcast end before republishing its path
kixelated Oct 3, 2026
2fb0aea
feat(js/net): keep sharing a path's broadcast when its first hop changes
kixelated Oct 3, 2026
c2df314
fix(net): close lifecycle gaps the adversarial review found in fronts…
kixelated Oct 3, 2026
cd6aa13
fix(net): a refused IETF joining fetch declares the live start
kixelated Oct 3, 2026
c435876
Merge remote-tracking branch 'origin/main' into claude/subscription-r…
kixelated Oct 3, 2026
4d821c6
chore(quest): move the identity quest edits to their own planning PR
kixelated Oct 3, 2026
d11b053
fix(net): pump copies are candidates, not owners
kixelated Oct 3, 2026
c07d1bd
fix(net): a quiet replaced route holds its group only until stale
kixelated Oct 3, 2026
66e949f
fix(net): an idle track hides its cache until a live route vouches fo…
kixelated Oct 3, 2026
a4acc38
fix(net): a route's subscription answer settles a hidden cache
kixelated Oct 3, 2026
947d195
style: cargo fmt
kixelated Oct 3, 2026
bf247c0
fix(net): bound a replaced route's subscription after its open groups
kixelated Oct 3, 2026
53fd063
quest: plan #4741 follow-ups
kixelated Oct 3, 2026
b78c5da
Merge remote-tracking branch 'origin/claude/subscription-route-change…
kixelated Oct 3, 2026
7f51407
test(net): pump edge cases a peer can trigger
kixelated Oct 3, 2026
0f55b7d
test(net): route changes flap over real sessions, lite-07 included
kixelated Oct 3, 2026
3244a58
fix(net): a hidden cache comes back only as a live route delivers it
kixelated Oct 3, 2026
792119c
quest: live media time builds on the delivery reveal
kixelated Oct 3, 2026
cd2898a
fix(net): a route's live edge settles a hidden cache
kixelated Oct 3, 2026
a9903b3
quest: live media time builds on the route's live edge
kixelated Oct 3, 2026
4b34d2f
feat(net)!: a track's live state gates its cache; lite-07 answers wit…
kixelated Oct 3, 2026
697a296
refactor(net)!: resume route changes by reading the routes' copies, n…
kixelated Oct 4, 2026
2135c9a
Merge remote-tracking branch 'origin/main' into claude/subscription-r…
kixelated Oct 4, 2026
f66e71f
fix(net): prune the resume reader registry on insert
kixelated Oct 4, 2026
b106202
feat(net): session copies linger with their cache; IETF subscriptions…
kixelated Oct 4, 2026
b243b08
fix(net): carry recovery state across route changes
kixelated Oct 4, 2026
90fd0f6
fix(net): keep stalled readers watching subscription preferences
kixelated Oct 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,11 +311,11 @@ moq --connect https://relay.example.com/anon \

## Redundant publishers

Two publishers that share a Hop ID (`--hop 42`) are treated as
interchangeable sources: relays hold both routes and fail over at a group
boundary. They must produce identical tracks with aligned groups. Everywhere
else leave `--hop` unset: a fresh id per run is what makes a restarted
encoder take over cleanly instead of splicing mid-stream.
Two publishers of the same broadcast name are interchangeable sources:
relays hold both routes and fail over between them mid-group. They must
produce identical tracks with aligned groups. A restarted encoder is the same
broadcast too, so one whose groups restart from 0 must publish under a new
name, or viewers wait for its sequence to catch up.

## Cluster

Expand Down
24 changes: 16 additions & 8 deletions doc/bin/relay/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,31 @@ has usually removed the other stale routes too, and the relay sends one
retraction instead of advertising each stale path in turn. Requests still
follow the current best route immediately; only the announcement waits.

A path names one broadcast, whoever publishes it. When the route serving a
broadcast dies, withdraws, or is beaten by a cheaper route, each subscription
continues on the new route from the first frame its readers lack, so they see
every frame once, mid-group included. A route that is still up finishes the
groups it has open, overlapping the new one. A group neither route delivers is
dropped once the readers' max age has passed it. A route through the subscribing peer
itself is never used. A publisher whose groups restart, such as an encoder
restarting from group 0, must publish under a new broadcast name; resumed under
the old one, readers wait for its sequence to catch up.

Failover routes must carry copies of the same broadcast. For each track, the
relay requires matching timescale, retention window, publisher priority, and
group ordering. A source with different properties is refused before its groups
are spliced in. If no compatible source remains, the track fails with
group ordering. A source with different properties is refused before it serves
the track. If no compatible source remains, the track fails with
`Unsupported`. New immutable properties require a new track name or broadcast
identity.
name.

A route whose original publisher (its first hop) changes is updated in place on
both wire protocols, so the broadcast never briefly vanishes downstream.
Subscriptions already in flight keep draining the old publisher until it ends
and are never spliced onto the new one. New requests resolve through the updated
route as a fresh broadcast, without the old publisher's track properties.
both wire protocols, so the broadcast never briefly vanishes downstream, and
subscriptions in flight carry on through it.

A publisher whose protocol names no hop (moq-transport without the cluster
extension, moq-lite 01 through 03, or a peer that sends 0) gets a random first
hop from the relay it connects to, fresh for each connection, followed by a 0.
Its reconnect is therefore a new publisher downstream, a reprice on the same
Its reconnect is therefore a new first hop downstream, a reprice on the same
connection stays in place, and the 0 keeps it ranked as anonymous.

## Topology
Expand Down
32 changes: 21 additions & 11 deletions doc/concept/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,11 @@ always a prefix, on every wire version and on moq-transport alike; a service
that serves only some of the paths beneath its prefix refuses the rest as they
are requested. Each route carries the chain of relay identities it passed
through, which is how forwarding loops are caught, and a cost, which is how a
subscriber picks among several routes to the same broadcast. A hop of 0 is the
anonymous mark and travels the chain unchanged; when it is the first hop, a relay
puts a random ID, fresh per connection, in front of it to name the publisher. A route that passed through an
subscriber picks among several routes to the same broadcast. A path names one
broadcast whoever publishes it, so a subscription moves between routes without
a seam; a publisher must not reuse a path for different content. A hop of 0 is
the anonymous mark and travels the chain unchanged; when it is the first hop, a
relay puts a random ID, fresh per connection, in front of it. A route that passed through an
anonymous hop at any depth ranks below every fully identified route, whatever
the costs say; among anonymous routes, cost keeps ordering.

Expand All @@ -110,8 +112,7 @@ without that declaration fails the subscription with `ProtocolViolation`; older
moq-lite versions use FIN alone. moq-transport requires `PUBLISH_DONE` before FIN.
moq-transport sessions behave the same when a namespace is withdrawn. A route
update that changes its first hop, the original publisher, is not a retraction:
subscriptions in flight drain the old publisher, and new requests resolve
through the new one.
subscriptions in flight carry on through it.

A graceful session close withdraws its announcements and waits up to one
second for transport acknowledgement before disconnecting. Rust uses
Expand Down Expand Up @@ -291,12 +292,21 @@ anything on its own. Both ends apply it: the publisher skips a group rather
than sending it, and the subscriber skips it again as it reads, since the
publisher only ever sees the most tolerant budget across its subscribers.

Across a native route failover, the reader still judges buffered groups against
the logical track's live edge, including groups it is draining from a retired
route. A successor group with no timestamp leaves the preceding group's reach
unbounded until its first frame arrives; if it is dropped first, the next group
takes its place. A cached open group's prefix remains
readable across repeated takeovers and idle resumes.
A route failover is invisible to max age: a group open across the change carries
on from the new route at the frame where the old one stopped, and a group only a
replaced route that went quiet still holds is given up once it falls a full
budget behind the new route's live edge. A successor group with no timestamp
leaves the preceding group's reach unbounded until its first frame arrives; if
it is dropped first, the next group takes its place.

A relay cancels its upstream subscription once nobody subscribes, but keeps its
copy of the track for 30 seconds after the last reader leaves, so a returning
reader or the next fetch finds its cache. That copy is not live meanwhile:
readers get nothing from its cache until the source answers again, since how
stale it is cannot be told. lite-07 and moq-transport answer with their largest
position, and older lite versions are asked from the head of the newest cached
group, so their first frame says where their feed is. A feed that has moved on
past everything cached leaves the cache below it to fetches.

The publisher may declare a retention window per track. Omission sets no limit;
zero keeps only the live edge. The origin cache ceiling and cache pool may still
Expand Down
4 changes: 2 additions & 2 deletions doc/concept/use-case/contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ captions on.
Because tracks are only pulled where they're needed, a publisher can open
several connections that might be used. Primary and secondary ingest is two
connections and no business logic: subscriptions ride the primary until it
fails, then move. Two encoders sharing a Hop ID become interchangeable
sources that relays fail over between at a group boundary; see
fails, then move. Two encoders publishing the same broadcast name are
interchangeable sources that relays fail over between mid-group; see
[redundant publishers](/bin/cli#redundant-publishers).

## One protocol both ways
Expand Down
31 changes: 12 additions & 19 deletions drafts/draft-lcurley-moq-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ informative:

This document defines a clustering extension for MoQ Transport {{moqt}}, used to build a mesh of relays.
Each namespace advertisement carries the list of Hop IDs it has passed through, starting with the original publisher, and the accumulated cost of that path.
A receiver uses the list to detect loops and to tell which advertisements come from the same publisher, and the cost to choose between paths.
A receiver uses the list to detect loops, and the cost to choose between paths.
Each endpoint declares its own Hop ID at setup, so a peer never advertises or serves it a path that already passed through it.

--- note_Note_to_Readers
Expand Down Expand Up @@ -99,8 +99,7 @@ A **Hop ID** is a variable-length integer naming one endpoint in a path.
Hop IDs SHOULD be unique among the endpoints an advertisement can traverse.
An endpoint MAY pick one at random, since collisions in a 64-bit space are unlikely, or use a configured identifier that survives restarts.

Loops and origins are detected by comparing Hop IDs for equality, so two endpoints sharing one are indistinguishable.
Redundant publishers of interchangeable content MAY share one deliberately, so the mesh treats their paths as failover options for the same content ({{selection}}).
Loops are detected by comparing Hop IDs for equality, so two endpoints sharing one are indistinguishable.

## The Reserved Hop ID 0 {#zero}
**0 means "no identity"** and is reserved.
Expand All @@ -109,11 +108,11 @@ It stands for an endpoint that did not negotiate this extension, and an endpoint
Since any number of endpoints can be 0, it identifies nothing:

- **Loop detection**: 0 in a HOP_PATH is never a loop. A receiver whose own Hop ID is 0 cannot detect loops through itself and MUST NOT discard an advertisement merely because the path contains 0.
- **Origin identity**: an advertisement whose first entry is 0 has an unknown publisher, which a relay names by putting a stamp of its own in front ({{stamping}}). A receiver MUST NOT treat two unstamped ones as interchangeable ({{selection}}).
- **Unknown publisher**: an advertisement whose first entry is 0 has an unknown publisher, which a relay names by putting a stamp of its own in front ({{stamping}}).
- **Filtering**: a peer that declared 0 gave the receiver nothing to filter that session on. The receiver MAY assign an ID of its own ({{assigned}}) as local selection state and MUST NOT write it into HOP_PATH.

Duplicate *non-zero* Hop IDs in one HOP_PATH are a loop; duplicate zeros are not.
Declaring 0 trades loop detection and failover for anonymity, except against a receiver that assigns an identity of its own.
Declaring 0 trades loop detection for anonymity, except against a receiver that assigns an identity of its own.

## Assigned Identities {#assigned}
A receiver MAY assign a Hop ID to a peer that declared none, whether by declaring 0 or by not negotiating the extension.
Expand All @@ -133,7 +132,7 @@ An anonymous accepted session cannot be correlated with anything, so it SHOULD g
A relay that records an advertisement whose first HOP_PATH entry is 0 MUST insert a random non-zero Hop ID in front of it, picked once for the session the advertisement arrived on.
One that arrived with no HOP_PATH ({{bridging}}) becomes that stamp followed by 0.

Nothing on the wire says whether an unnamed publisher that reconnects is the same one, so a fresh stamp per session makes its reconnect a change of publisher downstream ({{updating}}), while an update on the same session keeps the stamp and applies in place.
A fresh stamp per session keeps unnamed publishers on different sessions apart, while an update on the same session keeps the stamp and applies in place.
The 0 behind the stamp keeps the path ranked below fully identified ones ({{selection}}), since the unnamed hop may hide any depth.
A stamp names a session, not a peer: it MUST NOT be reused across sessions or taken as the peer's identity.

Expand Down Expand Up @@ -235,10 +234,7 @@ A receiver MUST NOT treat the repeat as a duplicate or a protocol violation.
An advertisement lives as long as its stream, so an update on a new stream would leave two streams claiming one namespace.
An endpoint MUST NOT open a second stream for an advertisement it already maintains on the session.

An update replaces the old parameters atomically, so a receiver MUST NOT tear down subscriptions or drop cached state because one arrived.
If the first HOP_PATH entry is unchanged the content is continuous and subscriptions MAY resume on the new route at a group boundary.
If the first entry changed, the publisher changed, and the endpoint still sends an ordinary update.
The receiver keeps each subscription it is already serving on the old source until that source ends, MUST NOT resume or splice it onto the updated route, and serves new requests from the updated route without state cached from the old publisher.
An update replaces the old parameters atomically, so a receiver MUST NOT tear down subscriptions or drop cached state because one arrived, even when the first HOP_PATH entry changed: the namespace still names the same content ({{publishers}}).

The expected update is a ROUTE_COST change, which is how a relay signals that it started or stopped carrying the namespace.

Expand All @@ -258,8 +254,7 @@ A receiver SHOULD NOT cache refusals.

A relay MUST NOT advertise a namespace merely because it resolved it: the covering advertisement stays the only one until the publisher advertises the concrete namespace, which it SHOULD do once producing, so a later request finds the running content by its exact namespace instead of resolving a second producer.

Two advertisements whose HOP_PATH begins with the same non-zero Hop ID come from the same publisher and carry interchangeable content: a receiver MAY hold them as redundant paths and fail an active subscription over to the survivor at a group boundary.
If the first entries differ, or either is 0, they are distinct publishers reusing a namespace ({{publishers}}).
Every advertisement of a namespace carries the same content ({{publishers}}): a receiver MAY hold several as redundant paths and fail an active subscription over to a survivor, continuing from the first object the subscriber lacks.

An endpoint MUST NOT advertise a path whose HOP_PATH contains the Hop ID the peer declared: the peer could only discard it, and acting on it would form a loop.
Of the paths that remain it SHOULD advertise the best, and advertises nothing when every path contains that Hop ID.
Expand All @@ -274,13 +269,10 @@ One rule for advertisement and dispatch keeps advertised paths truthful and prev
{{moqt}} lets several publishers advertise one namespace and leaves to the relay how it serves a SUBSCRIBE among them.
Under this extension an advertisement is a path, so a session advertises a namespace at most once, a relay forwards only the best path it knows ({{selection}}), and a subscription is served from one source at a time.

A receiver MAY still hold paths to several publishers of one namespace and choose between them as it sees fit: serve from the cheapest and move to the next when it fails.
A namespace names one broadcast whichever publisher serves it, so several publishers of it are redundant copies: their groups are one sequence, and a subscription moves between them without a gap.
A publisher MUST NOT reuse a namespace for different content, and publishes a new instance under a new namespace instead.
A refusal moves to another publisher only as {{selection}} allows: once, and only for NO_CAPACITY.
A relay that moves to another publisher MUST update its advertisement to the new path ({{updating}}), so the first Hop ID downstream names the publisher that new subscriptions reach.
Moving between distinct publishers is a discontinuity: their groups are not one sequence, so a subscriber sees an unrelated Location, and a FETCH that succeeds against one may fail against the other.

Redundant publishers of the same content avoid this by sharing a Hop ID ({{hop-ids}}), which makes their paths interchangeable and lets a subscription fail over at a group boundary.
Publishers that do not share one are treated as reusing a name.
A relay that moves to another publisher MUST update its advertisement to the new path ({{updating}}).


# Security Considerations {#security}
Expand Down Expand Up @@ -337,9 +329,10 @@ This document requests one registration in the "REQUEST_ERROR Codes" registry.
# Appendix A: Changelog

## moq-cluster-02
- A namespace names one broadcast whichever publisher serves it, and a publisher MUST NOT reuse one for different content. A receiver MAY fail a subscription over between any of its advertisements, continuing from the first object the subscriber lacks instead of at a group boundary. Replaces the first-Hop-ID publisher identity.
- Defined request resolution against the longest covering prefix and the NO_CAPACITY refusal with its single re-resolution; any other refusal is terminal, including between several publishers of one namespace.
- A relay does not advertise a namespace because it resolved it; the publisher advertises the concrete namespace once producing.
- A change of original publisher is an ordinary update instead of a withdrawal and a new advertisement. Subscriptions already served drain the old source; new requests take the updated route.
- A change of original publisher is an ordinary update instead of a withdrawal and a new advertisement.
- A relay puts a random Hop ID, picked per session, in front of an advertisement whose first HOP_PATH entry is 0, and writes that stamp followed by 0 for one with no HOP_PATH.

## moq-cluster-01
Expand Down
Loading
Loading