Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion doc/bin/relay/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@ versions translates: a moq-lite-06 peer's routes enter the route layer as the
relay's own claims, and lite-07 routes reach a moq-lite-06 peer as a hop list
that starts at their origin node. A relay drops a moq-lite-06 route whose hop
list names its own routing identity, since that is one of its claims coming
back.
back. It holds back one naming an identity it routes to over moq-lite-07, a
copy of that node's claims that would otherwise outlive their withdrawal, and
announces it afresh once that route is lost, so the moq-lite-06 path takes over.

A track from a moq-transport publisher keeps its Object IDs, object properties,
and END\_OF\_GROUP status objects across a moq-lite-07 link, since relays
Expand Down
1 change: 0 additions & 1 deletion quest/m2/cluster-routing/routes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,7 +542,6 @@ benchmarks.
- [Route layer acceptance](/quest/m2/cluster-routing/routes/acceptance.md) - forwarding fixtures ported and the model deleted, incident fixtures, end-to-end checks, and benchmarks
- [Firefox tabs prove their route key](/quest/m2/cluster-routing/routes/browser-proof.md) - a current Firefox tab opens a lite-07 session with a route-key proof the relay verifies, once neqo's exporter follows the draft
- [Playwright bundles Chromium's spec exporter](/quest/m2/cluster-routing/routes/playwright-pin.md) - the toolchain's Playwright bundles Chromium 158+, so the route key lane proves the Chromium tab
- [A declined lite-06 route is reconsidered when the lite-07 path fails](/quest/m2/cluster-routing/routes/lite06-failover.md) - a mixed-version relay fails over to the lite-06 route it declined
- [A covering predecessor's takeover shows under a mount](/quest/m2/cluster-routing/routes/mount-takeover.md) - lite-07 presents a takeover beneath a mount as lite-06 does

## Related
Expand Down
23 changes: 0 additions & 23 deletions quest/m2/cluster-routing/routes/lite06-failover.md

This file was deleted.

75 changes: 72 additions & 3 deletions rs/moq-net/src/lite/subscriber.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,9 @@ pub(super) struct Subscriber {
/// The instances minted for the paths the peer advertises without one, shared by the
/// announce streams.
minted: Minted,
/// The identities the origin routes to over lite-07, whose chains are held back where
/// those routes reach.
known: kio::Shared<crate::route::node::Known>,
}

/// The instance a session minted for each path a peer advertises without one (every
Expand Down Expand Up @@ -474,6 +477,7 @@ impl Subscriber {
// every session sharing that origin, required for cross-session
// loop detection.
let self_origin = crate::Hop::from(config.origin.hop());
let known = config.origin.known();
Self {
session: config.session,
runtime: config.runtime,
Expand All @@ -500,6 +504,7 @@ impl Subscriber {
announces: Default::default(),
viewers: Default::default(),
minted: Default::default(),
known,
}
}

Expand Down Expand Up @@ -743,6 +748,15 @@ impl Subscriber {
route
}

/// Whether the route the peer advertised at `path` names, in its chain, an identity
/// the origin routes to over lite-07 through a route that carries that path.
fn shadowed(&self, path: &Path, route: &crate::origin::RouteInfo) -> bool {
let Ok(path) = self.origin.root().join(path) else {
return false;
};
self.known.read().names(&route.hops, &path)
}

/// The announcing session's declared or assigned identity, for split-horizon.
///
/// A non-zero AnnounceOk origin is the declared id. Otherwise the caller-assigned
Expand Down Expand Up @@ -787,8 +801,11 @@ impl Subscriber {
let metadata = self.announced_route(&path, hops, cost, link_cost, responder_origin, announced);

// A re-price keeps the route's prefix (and its
// served paths) and re-prices in place. In-flight tracks keep flowing.
if let Some(entry) = announced.attached(&path) {
// served paths) and re-prices in place. In-flight tracks keep flowing. A chain
// that now names an identity routed over lite-07 is offered, to be held back.
if !self.shadowed(&path, &metadata)
&& let Some(entry) = announced.attached(&path)
{
entry.update(metadata);
return Ok(true);
}
Expand Down Expand Up @@ -2133,6 +2150,8 @@ struct PrefixRun {
decoder: lite::AnnounceDecoder,
/// The auth epoch last applied, so a new limit holds back or brings back routes.
auth_epoch: u64,
/// The [`crate::route::node::Known`] epoch last applied.
known_epoch: u64,
/// Lite05+: the ANNOUNCE_STARTs of the initial set still to read. Anything else
/// before them, or the stream ending, cuts the counted set short.
initial: u64,
Expand Down Expand Up @@ -2231,6 +2250,7 @@ impl AnnouncePrefix {
announced: Announced::new(self.subscriber.announces.clone(), self.subscriber.minted.clone()),
decoder: lite::AnnounceDecoder::default(),
auth_epoch: 0,
known_epoch: 0,
initial,
routed: match version.has_routes() {
true => {
Expand Down Expand Up @@ -2299,6 +2319,20 @@ impl AnnouncePrefix {
routed.limit(&self.subscriber, &permit)?;
}
}
// A hop chain naming an identity this node gains or loses a lite-07 route to
// is held back or taken up. Lite-07 streams carry no chains.
while run.routed.is_none() {
let seen = run.known_epoch;
let Poll::Ready(known) = self.subscriber.known.poll(waiter, |known| match known.epoch() != seen {
true => Poll::Ready(()),
false => Poll::Pending,
}) else {
break;
};
run.known_epoch = known.epoch();
drop(known);
run.announced.reconsider(&self.subscriber);
}
// Each announce routes into the origin, so a peer streaming them must not hold
// the session's other tasks. A yield has woken the task, and this arm always
// ends Pending, so the routes below are still served this turn.
Expand Down Expand Up @@ -9621,6 +9655,11 @@ enum Held {
/// Outside the session's limit: the route as advertised, attached once a
/// limit allows it.
Withheld(crate::origin::RouteInfo),
/// Its chain names an identity the origin routes to over lite-07 through a route
/// that carries this path: the route as advertised, attached once none is left. Filed now, it would be
/// another copy of that identity's claim under this node's own, outliving its
/// withdrawal until this chain retracts too.
Shadowed(crate::origin::RouteInfo),
/// In the origin, serving requests.
Attached(Box<AnnouncedRoute>),
}
Expand Down Expand Up @@ -9671,6 +9710,11 @@ impl Announced {
/// session on demand, or hold it back when the session's limit does not cover it.
/// Returns whether it attached.
fn offer(&mut self, subscriber: &Subscriber, path: Path, route: crate::origin::RouteInfo) -> bool {
if subscriber.shadowed(&path, &route) {
tracing::debug!(route = %subscriber.log_path(&path), "holding back announce of an identity routed over lite-07");
self.routes.insert(path, Held::Shadowed(route));
return false;
}
if !subscriber
.auth
.within_limit(crate::auth::Direction::Subscribe, path.as_str())
Expand Down Expand Up @@ -9786,7 +9830,7 @@ impl Announced {
.filter(|(path, held)| match held {
Held::Attached(_) => !permit.within_limit(path.as_str()),
Held::Withheld(_) => permit.within_limit(path.as_str()),
Held::Declined => false,
Held::Declined | Held::Shadowed(_) => false,
})
.map(|(path, _)| path.clone())
.collect();
Expand All @@ -9808,6 +9852,31 @@ impl Announced {
}
}

/// Apply a change in what the origin routes to over lite-07: hold back every attached
/// route whose chain now names such an identity, closing its sources, and offer every
/// held-back one whose chain no longer does, so the lite-06 path takes over.
fn reconsider(&mut self, subscriber: &Subscriber) {
let changed: Vec<Path> = (self.routes.iter())
.filter(|(path, held)| match held {
Held::Attached(entry) => subscriber.shadowed(path, &entry.route),
Held::Shadowed(route) => !subscriber.shadowed(path, route),
Held::Declined | Held::Withheld(_) => false,
})
.map(|(path, _)| path.clone())
.collect();
for path in changed {
let mut route = match self.routes.remove(&path) {
Some(Held::Attached(entry)) => entry.route.clone(),
Some(Held::Shadowed(route)) => route,
_ => unreachable!("only attached and held-back routes change"),
};
if subscriber.going_away.is_set() {
route.cost = crate::origin::Cost::DRAIN;
}
self.offer(subscriber, path, route);
}
}

/// Re-price every attached route to a draining cost (the peer sent a GOAWAY).
fn drain(&mut self) {
for held in self.routes.values_mut() {
Expand Down
15 changes: 14 additions & 1 deletion rs/moq-net/src/model/origin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2404,6 +2404,15 @@ impl Producer {
Some(crate::route::node::hop(node.declared()))
}

/// The identities this node routes to over moq-lite-07, as the lite-06 hops they
/// travel as. A lite-06 route naming one is held back while that route lasts, so
/// the broadcast is not filed again as this node's own claim, and taken up once it
/// goes.
#[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))]
pub(crate) fn known(&self) -> kio::Shared<crate::route::node::Known> {
self.shared.lock().known.clone()
}

/// This origin's construction config.
pub fn config(&self) -> Config {
let shared = self.shared.lock();
Expand Down Expand Up @@ -6004,6 +6013,9 @@ struct OriginState {
// The route layer node this origin's lite-07 sessions share; see
// [`Consumer::route_node`].
route_node: Option<crate::route::Node>,
// What that node routes to, published before it exists so older-version sessions
// can watch it from the start.
known: kio::Shared<crate::route::node::Known>,
}

impl OriginState {
Expand All @@ -6023,6 +6035,7 @@ impl OriginState {
next_listener: 0,
hidden_withheld: HiddenWithheld::default(),
route_node: None,
known: Default::default(),
}
}

Expand Down Expand Up @@ -7222,7 +7235,7 @@ impl Consumer {
if let Some(node) = &shared.route_node {
return node.clone();
}
let node = crate::route::Node::random();
let node = crate::route::Node::random(shared.known.clone());
shared.route_node = Some(node.clone());
drop(shared);
// The whole table, whatever this handle's scope.
Expand Down
32 changes: 32 additions & 0 deletions rs/moq-net/src/route/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,20 @@ impl Peer {
}
}

/// Which claims one unscoped route can carry: those within the prefix this node asked
/// its neighbor for, or every claim when unset. See [`Router::carried`].
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Carried {
asked: Option<Path>,
}

impl Carried {
/// Whether a claim at `claim` goes through this route.
pub fn carries(&self, claim: &Path) -> bool {
self.asked.as_ref().is_none_or(|asked| within(claim, asked))
}
}

/// An identity this node serves itself.
struct Origin<A> {
seqno: Seqno,
Expand Down Expand Up @@ -870,6 +884,24 @@ impl<N: Key, A: Clone + Eq> Router<N, A> {
.collect()
}

/// What each usable feasible route to `id` through an unscoped neighbor can carry,
/// as [`Self::routes`] filters it per claim. A scoped neighbor is left out: the
/// session serves only what its permit's limit admits, which its regions only
/// bound. Empty for an identity this node serves itself or has no such route to.
pub fn carried(&self, id: Id) -> Vec<Carried> {
if self.origins.contains_key(&id) {
return Vec::new();
}
(self.rx.get(&id).into_iter().flatten())
.filter(|(_, entry)| !entry.held && self.feasible(id, entry.advert))
.map(|(next, entry)| (&self.neighbors[next], entry))
.filter(|(peer, entry)| peer.link.scope.is_none() && add(entry.advert.metric, peer.link.cost).is_some())
.map(|(peer, _)| Carried {
asked: peer.asked.clone(),
})
.collect()
}

#[cfg(test)]
/// The route this node would offer an upstream link for `id`.
pub fn export(&self, id: Id) -> Option<Selected<N>> {
Expand Down
Loading