diff --git a/quest/m2/lite07-finalize.md b/quest/m2/lite07-finalize.md index dd811c8284..f9ce1cf54a 100644 --- a/quest/m2/lite07-finalize.md +++ b/quest/m2/lite07-finalize.md @@ -215,7 +215,6 @@ the version that ships them. - [Cache-aware multicast](/quest/m2/multicast/README.md) - lite-07 carries the redesigned route layer: routes to forwarding nodes, no holds or doubt, claims crossing trust boundaries by a chain, and no Route Key - [Routes and announces](/quest/m2/cluster-routes/README.md) - the route layer's fixtures pass on real lite-07 sessions, and the lite-06 churn mitigations they replace are retired -- [Route-free domain check](/quest/m2/lite07-routeless-domain.md) - the frozen wire lets a domain route below MoQ, keeping the node layer open ## Related diff --git a/quest/m2/lite07-routeless-domain.md b/quest/m2/lite07-routeless-domain.md deleted file mode 100644 index d4bd836f84..0000000000 --- a/quest/m2/lite07-routeless-domain.md +++ /dev/null @@ -1,46 +0,0 @@ -# [M] lite-07 lets a domain route below MoQ - -## Goal - -Before lite-07 freezes, show that two relays of one trusted domain can -exchange claims over lite-07 without either sending ROUTE, DEAD, or DRAIN, -so a domain whose reachability comes from a node layer underneath (see -[the node layer](/quest/m3/node-layer.md)) still interoperates on the frozen -wire. Either a test on real sessions shows it as shipped, or the PR names -the smallest wire change that would make it possible. - -Non-goals: building a node layer, or changing how today's routing nodes -route. - -## Plan - -Decided 2026-11-18 (maintainer, /quest-plan on decoupling routing from -moq-net): ✅ check before the cut and don't hold it for the exploration -(rejected: holding the cut for the whole node-layer exploration, which -stacks another L on the cut's serial path for a result that may keep -today's design; cutting with no check, which risks a frozen wire that forces -in-domain peers to exchange routes, so a layered relay cannot interoperate -across the release window). - -What to check. Today a neighbour is `Role::Routing` (routes exchanged), -`Role::Scoped` (claims with a chain, no routes), or `Role::Endpoint` -(`rs/moq-net/src/route/relay.rs`), and `Msg::Announce` already carries -claims with no route. The question is whether an in-domain peer can carry -claims without routes, with each claim still naming its announcing relay -and ordered by that relay's revision and floor: a scoped grant covering the -whole tree, a peer that announces itself as an endpoint, or neither. - -Recommendation: a route-free in-domain claim needs the announcing relay's -identity, revision, and floor, which `Msg::Claims` carries, without -`Msg::Route`. If the shipped wire only allows that across a scoped grant, -measure what the chain costs and whether a whole-tree scope breaks anything. -If no shipped form works, the PR proposes the smallest change, such as -`Claims` being valid from a peer that never sent its route, and lists the -decision under `## Waiting` for the maintainer, since whether to hold the -cut for it is theirs. - -## Related - -- [The node layer](/quest/m3/node-layer.md) - the exploration this keeps open -- [Finalize moq-lite-07](/quest/m2/lite07-finalize.md) - the cut this must precede -- [Cache-aware multicast](/quest/m2/multicast/README.md) - the route layer as shipped diff --git a/quest/m3/node-layer.md b/quest/m3/node-layer.md index f97ee4f246..68e1f0d4ea 100644 --- a/quest/m3/node-layer.md +++ b/quest/m3/node-layer.md @@ -61,10 +61,35 @@ moq-net): boundaries all 14 stand (rejected: all 14 as written, which rules link-state out before it is measured, against requirement 14; relaxing 6 too, which the controller write-up may argue for on evidence). -5. ✅ **The lite-07 cut does not wait**: the - [route-free domain check](/quest/m2/lite07-routeless-domain.md) runs +5. ✅ **The lite-07 cut does not wait**: a route-free domain check ran before it so the frozen wire commits to nothing, and this runs after. +What that check found (2026-11-18, `rs/moq-net/tests/integration/routeless_domain.rs`): +the frozen lite-07 wire already carries a domain that routes below MoQ, with +no change. A session declares a Node ID, and so exchanges ROUTE, DEAD, and +DRAIN, only when it wires both directions with whole grants and proxy on +(`origin::Producer::node_id`); otherwise each end files the other as scoped +and claims cross as route-free announces with a chain, drain, and the +`lost` flag that tells an end from a loss. Two results bind the design: + +- Claims inside such a domain need split horizon, as iBGP does: marking + every in-domain link `upstream`, so no relay passes one in-domain peer's + claim to another, ends a claim everywhere at once on a full mesh, with + relays still serving their own clients. That fits the flat dialable arm, + with claims sent from the announcer to every relay directly. +- Without the marks, a cycle echoes: when the publisher ends a claim, each + relay holds its neighbour's copy, which no route or revision orders, and + shows it unavailable until the debounce lets it go, as offline rather + than ended in the implementation as checked. Nothing loops or revives, + but readers lose the end's cause. + Route-free claims carry no announcer revision or floor (`Msg::Claims` from + a peer with no route is refused), so a design that wants cycles inside a + domain needs a wire change in a later version. +- Today a relay leaves routes off by wiring each direction as its own + session, or on a two-way session whose grants are not whole or whose + proxy is off. A knob that keeps a two-way session with whole grants and + proxy on route-free is API for the implementation quest, not this one. + Recommendations: - The layering precedent is PIM-SM (RFC 7761): it builds trees from @@ -92,7 +117,6 @@ Recommendations: ## Related -- [The route-free domain check](/quest/m2/lite07-routeless-domain.md) - keeps lite-07 open to a node layer - [Cache-aware multicast](/quest/m2/multicast/README.md) - the requirements and the shipped rule this measures against - [Cluster routing](/quest/m3/cluster-routing/README.md) - the node routes warm routes rank against - [Cache-aware routing](/quest/m3/cache-routing/README.md) - warm routes, which stay MoQ's either way diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index 7a2a39995e..48b3485359 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -77,6 +77,7 @@ mod route_failover; mod route_health; mod route_race; mod route_stats; +mod routeless_domain; mod segmented_frame; mod session_close; mod session_panic; diff --git a/rs/moq-net/tests/integration/routeless_domain.rs b/rs/moq-net/tests/integration/routeless_domain.rs new file mode 100644 index 0000000000..298c85abc1 --- /dev/null +++ b/rs/moq-net/tests/integration/routeless_domain.rs @@ -0,0 +1,170 @@ +#![cfg(feature = "async")] + +//! Relays of one domain that exchange claims on lite-07 without ROUTE, DEAD, or DRAIN, +//! leaving reachability to whatever sits below them, as a node layer would. +//! +//! A session wired one way (one relay publishes, the other subscribes) declares no Node +//! ID, so each end files the other as scoped: claims cross with a chain and no route is +//! ever offered. Relays pulling from each other over such sessions form the domain. + +use crate::path_routes::{DEBOUNCE, cut, land, lite07, publish, quiesce, record, withdrawn}; +use crate::route_change::{link, next, payload, produce_origin, read, timed}; +use crate::support::{self, AppendNext, AtStart}; + +use moq_net::{Version, origin, track}; +use support::harness::{MockConnectOptions, MockPair, connect_mock}; + +/// `to` pulls everything `from` publishes over a link both ends mark upstream, so +/// neither passes a claim it learned from one in-domain peer to another, as iBGP's +/// split horizon keeps a full mesh from echoing. +async fn marked(version: Version, from: &origin::Producer, to: &origin::Producer) -> MockPair { + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(from.clone().with_link(origin::Link::Upstream).consume()); + options.client_subscribe = Some(to.clone().with_link(origin::Link::Upstream)); + connect_mock(options).await +} + +/// `a` and `b` each pull everything the other publishes, over two one-way sessions. +async fn both(version: Version, a: &origin::Producer, b: &origin::Producer) -> [MockPair; 2] { + [link(version, a, b).await, link(version, b, a).await] +} + +/// A full mesh of `A`, `B`, and `C` with every in-domain link marked, and a viewer +/// pulling from `B` unmarked. A claim on `A` reaches each once, and its publisher +/// ending it shows as an end everywhere at once. +#[kio::test] +async fn a_marked_mesh_without_routes_ends_a_claim_at_once() { + let version = lite07(); + let [a, b, c, viewer] = [1, 2, 3, 4].map(produce_origin); + let mut links = Vec::new(); + for (from, to) in [(&a, &b), (&b, &a), (&b, &c), (&c, &b), (&c, &a), (&a, &c)] { + links.push(marked(version, from, to).await); + } + links.push(link(version, &b, &viewer).await); + let mut watches = Vec::new(); + for (name, node) in [("B", &b), ("C", &c), ("viewer", &viewer)] { + watches.push((name, record(node, "live").await)); + } + + let start = kio::clock::now(); + let live = publish(&a, "live"); + quiesce().await; + for (name, watch) in &mut watches { + let seen = watch.seen(start); + assert_eq!(seen.len(), 1, "{name} starts the claim once and keeps it: {seen:?}"); + assert_eq!(seen[0].kind, support::Kind::Start, "{name}: {seen:?}"); + assert!(seen[0].available, "{name} reaches it: {seen:?}"); + } + + let gone = kio::clock::now(); + drop(live); + quiesce().await; + for (name, watch) in &mut watches { + let seen = watch.seen(gone); + assert_eq!(seen.len(), 1, "{name} only ends the claim: {seen:?}"); + assert_eq!(seen[0].kind, support::Kind::End, "{name}: {seen:?}"); + assert!(seen[0].at.is_zero(), "{name} ends it at once: {seen:?}"); + } +} + +/// The ring `A`, `B`, `C` with nothing marked, so each relay passes on what it learns. A +/// claim on `A` reaches the others once each and never comes back to `A`. Its withdrawal +/// leaves each holding the other's echo, which no route or revision orders, so each lets +/// go only when the debounce runs out and never revives it. This pins only that much: +/// how the end shows (offline, as of this writing) is left to a later fix. +#[kio::test] +async fn a_ring_without_routes_lets_an_echo_go_within_the_debounce() { + let version = lite07(); + let [a, b, c] = [1, 2, 3].map(produce_origin); + let _ab = both(version, &a, &b).await; + let _bc = both(version, &b, &c).await; + let _ca = both(version, &c, &a).await; + let mut watches = Vec::new(); + for (name, node) in [("A", &a), ("B", &b), ("C", &c)] { + watches.push((name, record(node, "live").await)); + } + + let start = kio::clock::now(); + let live = publish(&a, "live"); + quiesce().await; + for (name, watch) in &mut watches { + let seen = watch.seen(start); + let starts = seen.iter().filter(|s| s.kind == support::Kind::Start).count(); + assert_eq!(starts, 1, "{name} starts the claim once: {seen:?}"); + assert!(!seen.iter().any(withdrawn), "{name} keeps the claim: {seen:?}"); + assert!(seen.last().unwrap().available, "{name} reaches it: {seen:?}"); + } + + let gone = kio::clock::now(); + drop(live); + quiesce().await; + for (name, watch) in &mut watches { + let seen = watch.seen(gone); + let end = seen.iter().position(withdrawn); + let end = end.unwrap_or_else(|| panic!("{name} lets the claim go: {seen:?}")); + assert_eq!(end + 1, seen.len(), "{name} shows nothing after it: {seen:?}"); + assert!(seen[end].at <= DEBOUNCE, "{name} lets go within the debounce: {seen:?}"); + } +} + +/// A reader on `B` served straight from `A` on the unmarked ring keeps reading through +/// `C` when both sessions between `A` and `B` close, with no restart. +#[kio::test] +async fn a_reader_moves_around_a_ring_without_routes() { + let version = lite07(); + let [a, b, c] = [1, 2, 3].map(produce_origin); + let broadcast = a.route("live", origin::RouteOptions::hidden()).unwrap(); + let track = broadcast.broadcast().create_track("video", timed()).unwrap(); + broadcast.announce().unwrap(); + let mut at_b = record(&b, "live").await; + let start = kio::clock::now(); + let ab = both(version, &a, &b).await; + let _bc = both(version, &b, &c).await; + let _ca = both(version, &c, &a).await; + land().await; + let seen = at_b.seen(start); + assert_eq!(seen.last().unwrap().cost, 1, "B is served straight from A: {seen:?}"); + + let remote = b + .consume() + .follow("live", None) + .unwrap() + .available() + .await + .unwrap() + .unwrap(); + let sub = remote + .track("video") + .unwrap() + .subscribe(track::Subscription::default()) + .await + .unwrap(); + let mut rx = read(sub); + + let mut group = track.append_next().unwrap(); + group.write_frame(group.at_start(), payload(0, 0)).unwrap(); + assert_eq!(next(&mut rx).await, (0, payload(0, 0))); + + let moved = kio::clock::now(); + for pair in &ab { + cut(pair); + } + quiesce().await; + let seen = at_b.seen(moved); + let last = seen.last().unwrap_or_else(|| panic!("B moves: {seen:?}")); + assert_eq!(last.cost, 2, "B moves onto the copy through C: {seen:?}"); + assert!( + last.available && !seen.iter().any(withdrawn), + "B never lets go: {seen:?}" + ); + assert!( + !seen.iter().any(|event| event.kind == support::Kind::Restart), + "B keeps the Instance: {seen:?}" + ); + group.write_frame(group.at_start(), payload(0, 1)).unwrap(); + group.finish().unwrap(); + assert_eq!(next(&mut rx).await, (0, payload(0, 1))); + let mut group = track.append_next().unwrap(); + group.write_frame(group.at_start(), payload(1, 0)).unwrap(); + assert_eq!(next(&mut rx).await, (1, payload(1, 0))); +}