From 2b8ba5c3fcb0295cad0ab335131c6f902a4e2c4d Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 18 Nov 2026 22:39:48 -0800 Subject: [PATCH 1/4] chore(quest): claim quest/m3/cache-routing/select.md Quest-Claim: 65a43ea4f5c2fa67 From 48e2f7cdf89d19d652f5ebd4d8d060be68c57b4d Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Thu, 19 Nov 2026 00:49:01 -0800 Subject: [PATCH 2/4] test(route): warmth measures a mesh read from a file Co-Authored-By: Claude Fable 5.2 --- quest/m2/multicast/README.md | 9 +- quest/m3/cache-routing/select.md | 60 +++++- rs/moq-net/src/route/tests/warm.rs | 7 +- rs/moq-net/src/route/tests/warmth.mesh | 26 +++ rs/moq-net/src/route/tests/warmth.rs | 281 +++++++++++++++++++++++++ 5 files changed, 370 insertions(+), 13 deletions(-) create mode 100644 rs/moq-net/src/route/tests/warmth.mesh diff --git a/quest/m2/multicast/README.md b/quest/m2/multicast/README.md index c3a17568a5d..572be9d6de2 100644 --- a/quest/m2/multicast/README.md +++ b/quest/m2/multicast/README.md @@ -41,9 +41,12 @@ whose fix changes the wire, [the mesh replay](/quest/m2/multicast/mesh-replay.md), and [draining dialed links](/quest/m2/multicast/drain-dialed-links.md), and on [route layer acceptance](/quest/m2/cluster-routes/acceptance.md) retiring -the lite-06 mitigations. After the cut come warmth's wire -([cache-aware routing](/quest/m3/cache-routing/README.md)), -[route health](/quest/m3/route-health/README.md), and +the lite-06 mitigations. It does not wait on +[warmth's port](/quest/m3/cache-routing/select.md), which has started: its +wire joins lite-07 only if the port has landed and passed the mesh replay +when the cut is made, and otherwise moves to the first wip version after it +(maintainer, 2026-11-19). After the cut come +[route health](/quest/m3/route-health/README.md) and [outbound federation](/quest/m3/cluster-routing/federation.md). Decided 2026-11-15 (maintainer, /quest-plan), after an audit of the lite-07 diff --git a/quest/m3/cache-routing/select.md b/quest/m3/cache-routing/select.md index 2a3bf2b82ce..e0a0c0646d9 100644 --- a/quest/m3/cache-routing/select.md +++ b/quest/m3/cache-routing/select.md @@ -63,13 +63,55 @@ reason for the change: a deployment expects warm pulls to cut what its backbone carries by a large part, and a release is the only way its production mesh gets them. -First, before building: measure that saving. The simulator's shapes are -generated, and against path joins the saving runs from under 1% on random -region meshes to 8% on tiers and 70% on rings (the table below), so it -depends on the graph. Let `warmth.rs` read a mesh from a file (relays, links -with costs and marks, and where readers and publishers sit), run ranked -readers and path joins on it, and report link-groups carried by each. A -deployment's graph is private: take the file by path and commit none. +Measured 2026-11-18, before building: the port stops here until the +maintainer has run a deployment's graph (Waiting, below). `warmth.rs` reads a +mesh from a file (relays, links with costs, marks, and a label, and where +publishers and readers sit) and `benchmark_warmth_mesh` reports the +link-groups ranked readers and path joins each put on it, in all, per label, +and per link. A deployment's graph is private: pass its file by path as +`MOQ_WARMTH_MESH` and commit none. `warmth.mesh` beside it is the example. + +A tree feeding the same readers spans as many links under either rule, give +or take a relay carrying for others, so the saving is in which links carry +it. Generated meshes, five seeds each summed, every one settled right under +both rules, path joins / ranked readers: + +| mesh | backbone link-groups | backbone cold cost | every link-group | control messages | +|---|---|---|---|---| +| CDN, full core mesh, equal uplinks | 95 / 95 | 1048 / 760 (-27%) | 339 / 339 | 718 / 2902 | +| CDN, full core mesh, unequal uplinks | 125 / 90 (-28%) | 1408 / 736 (-48%) | 368 / 368 | 776 / 3193 | +| CDN, ring of regions, equal uplinks | 97 / 95 (-2%) | 776 / 760 (-2%) | 341 / 339 | 722 / 2234 | +| CDN, ring of regions, unequal uplinks | 134 / 94 (-30%) | 1072 / 752 (-30%) | 377 / 372 | 794 / 2735 | +| CDN, one hub region, equal uplinks | 97 / 97 | 1088 / 1088 | 341 / 341 | 722 / 1914 | +| CDN, one hub region, unequal uplinks | 128 / 93 (-27%) | 1432 / 1052 (-27%) | 371 / 371 | 782 / 2482 | +| 8 relays with an uplink each, peered every pair | 71 / 20 (-72%) | 710 / 200 (-72%) | 71 / 71 | 182 / 1236 | +| 8 relays with an uplink each, peered in a ring | 71 / 42 (-41%) | 710 / 420 (-41%) | 71 / 71 | 182 / 508 | + +A CDN here is six regions on a circle, two cores each, and four edges a +region marking both cores upstream; the backbone is the links between +regions, priced 4 plus 4 per step around the circle, warm a third. Four +broadcasts are published at random edges and each read by a random half of +the others. Equal uplinks cost 3, unequal ones 3 or 4 at random. In the last +two rows the backbone is the uplinks to the publisher's relay (10 cold, 4 +warm, peer links 2), and half the relays read each of four broadcasts. + +What it shows: + +- With equal uplinks a region's edges already land on one core by the + rendezvous hash, the path joins already share the shortest-path tree, and + warmth saves no backbone link-group: 0% to 2%. +- On a full core mesh it carries as many link-groups over shorter links: a + far region pulls from a nearer warm one instead of straight from the + publisher's, 27% off the cost where a link is priced by its length and + nothing where every backbone link costs the same. Each such pull adds a + relay hop. +- With unequal uplinks a region's edges split over its cores and each core + pulls its own copy: 27% to 30% fewer backbone link-groups. Pricing each + edge's uplinks equally gets the same tree with no wire. +- Relays with an uplink each and links between them, the drone meshes, save + 41% to 72%, the whole of what item 9 of the + [epic](/quest/m2/multicast/README.md) asked for. +- It costs 2.6 to 6.8 times the control messages. **Why ranked readers settle.** A relay's choice reads only copies feeding lower ranks, and what those feed never depends on a higher rank's choice. @@ -432,6 +474,10 @@ later step, a query naming an Instance, answered once with an offer, path, mark, and rank or none. A rank is a route cost and a Node ID. Nothing lite-07 freezes changes. +## Waiting + +- Maintainer: whether the port goes on now, after running the deployment's graph through `benchmark_warmth_mesh` with its backbone links labelled. An agent cannot: the graph is private and no session holds it. Options: (1) run it and build the port for lite-07 only if the backbone's link-groups fall there, recommended, since on the CDN shapes above they fall only where edges' uplinks are priced unequally, which pricing them equally fixes with no wire; (2) build the port now whatever it shows, for the drone meshes it does serve, at the risk of rushing an XL wire into the cut; (3) leave the port for the first wip version after the cut, as decided 2026-11-17. Done when the answer is recorded in this Plan. + ## Related - [Real relays replay a mesh and its faults](/quest/m2/multicast/mesh-replay.md) - the port runs through it before its wire may join lite-07 diff --git a/rs/moq-net/src/route/tests/warm.rs b/rs/moq-net/src/route/tests/warm.rs index 09ef9efbc9f..f5216d2cc52 100644 --- a/rs/moq-net/src/route/tests/warm.rs +++ b/rs/moq-net/src/route/tests/warm.rs @@ -486,7 +486,7 @@ fn close(streams: &mut BTreeMap>, peer: Node, stream: u64) { } } -fn pair(a: Node, b: Node) -> (Node, Node) { +pub(super) fn pair(a: Node, b: Node) -> (Node, Node) { (a.min(b), a.max(b)) } @@ -621,8 +621,9 @@ impl WarmSim { self.net.link(node, home, 1, Tie::Peer); self.net.publish(node, id); (0..claims) - .map(|claim| { - let path = format!("live/{claim}"); + .map(|_| { + // Numbered across publishers, so no two announce one path. + let path = format!("live/{}", self.casts.len()); self.net.claim(node, &path, true); self.casts.push((id, p(&path))); self.casts.len() - 1 diff --git a/rs/moq-net/src/route/tests/warmth.mesh b/rs/moq-net/src/route/tests/warmth.mesh new file mode 100644 index 00000000000..56990bad7bb --- /dev/null +++ b/rs/moq-net/src/route/tests/warmth.mesh @@ -0,0 +1,26 @@ +# An example mesh for `benchmark_warmth_mesh` in `warmth.rs`, which documents the format. + +relay origin near middle far pop edge-a edge-b + +# Cores on a line, with long links from the origin that pass over the cores between, +# labelled so the report totals them. +link origin near 6 1 peer +link near middle 4 1 peer +link middle far 4 1 peer +link origin middle 6 6 peer backbone +link origin far 8 8 peer backbone + +# A pop beside the far core, with its own long link. +link far pop 2 1 peer +link origin pop 9 4 peer backbone + +# Edges mark their cores upstream. +link edge-a far 1 1 up +link edge-a middle 2 1 up +link edge-b pop 1 1 up + +publish live origin +publish back edge-b + +read live far near middle pop edge-a edge-b +read back origin edge-a near diff --git a/rs/moq-net/src/route/tests/warmth.rs b/rs/moq-net/src/route/tests/warmth.rs index b5cf594c1c5..5c0ad51d09c 100644 --- a/rs/moq-net/src/route/tests/warmth.rs +++ b/rs/moq-net/src/route/tests/warmth.rs @@ -6,6 +6,7 @@ //! costs against the cheapest, how many copies leave the publisher's relay, the //! messages and rounds it took, and whether pulls ever formed a cycle or left a reader //! unfed. The ignored benchmarks print the tables; the tests pin what they show. +//! [`benchmark_warmth_mesh`] measures a mesh read from a file instead. use super::multi::Tie; use super::warm::*; @@ -1479,3 +1480,283 @@ fn ranked_pulls_settle_random_case_503() { } assert_eq!(churn(NOT_DOWN_QUERY, 503..504, random).failed, 1); } + +// --- A mesh from a file --- + +/// A mesh read from a file, one statement per line, `#` starting a comment: +/// +/// - `relay ...` names relays. +/// - `link [