Repository navigation
quest(m1): a route change wakes only the fronts it affects #4615
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
7ff3fbd
quest(m1): a route change wakes only the fronts it affects
kixelated 26882db
quest(m1): key route-wake watches by pin
kixelated 5007745
quest(m1): spell route-wake sets for every change
kixelated 5dac5b4
quest(m1): wake parked waiters when a claim loses standing
kixelated aab9231
quest(m1): align route-wake facts with the leave rule
kixelated 55982f0
quest(route-wakes): drop front-upgrade, removed by the tiers decision
kixelated df36308
Merge remote-tracking branch 'origin/main' into quest/plan-route-wakes
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # [M] A route change wakes only the fronts it affects | ||
|
|
||
| ## Goal | ||
|
|
||
| A route change under a prefix (join, withdraw, re-price, restale) makes the | ||
| origin driver act only on the fronts that could change their selection: | ||
| fronts still waiting for a source, fronts served by the changed route, and | ||
| publisher-pinned fronts on its first hop. Today it wakes every front below | ||
| the prefix, so one equal-cost pool member joining or leaving a prefix that | ||
| serves 10k paths costs the driver 100 to 600 ms of single-threaded work | ||
| while no front switches (`origin/pool_churn`, #4607). A root claim, such as | ||
| an archive's catch-all, makes every front on the relay pay for every change | ||
| to it. Done when `origin/pool_churn` is flat in served paths. | ||
|
|
||
| ## Plan | ||
|
|
||
| Facts from the wildcard line (`rs/moq-net/src/model/origin.rs` there): | ||
|
|
||
| - Every route change calls `sync_route`, which calls | ||
| `routes.poke_below(prefix)`. That bumps a counter-only `Watch` on every path | ||
| below the prefix, so a woken front cannot tell what changed. Each re-runs | ||
| `select` under the table read lock (`retain_routes`, then `best_route`, | ||
| which hashes every qualifying pool member unless a Stay front's route | ||
| still serves), then the driver polls every track and rescans deadlines. | ||
| - `route_order` is rendezvous-style: FNV over the path and hop ids, lowest | ||
| wins. A join moves only paths the newcomer wins, a leave only paths the | ||
| leaver was winning. | ||
| - A front's `Pin` limits which routes it can take. `Stay` (an origin-named | ||
| front while serving) keeps its route while it serves; `Route` qualifies | ||
| only its own route; `Publisher` only routes whose first hop matches; | ||
| `Local` only local routes; `Any` (identity not yet known, or origin-named | ||
| and not serving) every route. Pool fronts in `origin/pool_churn` never | ||
| get an origin-naming reply, so they are `Publisher`, each on a different | ||
| member's hop. So on a join the fronts that can act are the `Any` ones, | ||
| `Publisher` ones on the new route's first hop, and `Local` ones for a | ||
| local route. On a leave, the fronts served by or requesting through that | ||
| route, plus those same pin sets: a deeper prefix appearing shadows | ||
| shallower routes for them, and one leaving can unshadow a broader route | ||
| for a waiter it never served. | ||
|
|
||
| Decisions: | ||
|
|
||
| - Covers every route change, not only equal-cost pools. The pool is where | ||
| the benchmark found it, but the wake is per prefix regardless of cost. | ||
| (2026-09-30) | ||
| - An index, not a payload filter on the watch: each route records the | ||
| fronts it serves, and each prefix node records only its non-Stay watches | ||
| (waiting fronts, publisher- and local-pinned fronts, `routed_broadcast` | ||
| waiters). A change wakes the ones it qualifies for (next decision) plus | ||
| the changed route's own fronts. A | ||
| payload filter would still walk every watch under the prefix, which stays | ||
| linear in paths. (2026-09-30) | ||
| - The prefix node keys its watches by what they qualify for: `Any` and | ||
| `routed_broadcast` waiters in one set, `Publisher` by hop, `Local` in its | ||
| own set. `Route` fronts hang off their route only. A front moves its | ||
| entries when it selects, and drops them when it closes. Any change to a | ||
| route wakes the `Any` set, its first hop's `Publisher` set, the `Local` | ||
| set if local, and the fronts it serves or is requesting for, so an | ||
| unrelated publisher joining or leaving wakes no serving pool front. The | ||
| pin sets wake on a loss too: withdrawing a deeper claim can unshadow a | ||
| broader route for a parked waiter it never served. A withdrawal takes | ||
| the served set before dropping the record. Fronts sit at paths below the changed prefix, so the lookup | ||
| walks descendants but skips subtrees whose matching set is empty, keeping | ||
| cost in woken fronts rather than served paths. (2026-09-30) | ||
| - A same-publisher join still wakes that hop's whole `Publisher` set, even | ||
| where the new route cannot win. Narrowing it to the paths the route wins | ||
| is left out: the `Publisher` pin goes with `--hop` removal in the cluster | ||
| routing line. (2026-09-30) | ||
| - Builds on shared-fronts' keying, so the index hangs off the final front | ||
| identity. `origin-front-parks.md` replaces the `routed_broadcast` retry | ||
| loop, one of the watch consumers here; whichever lands second adapts it. | ||
| (2026-09-30) | ||
| - Stay fronts never need a join or re-price: the edge/core tiers decision | ||
| (2026-09-30) dropped route upgrades, so a serving front moves only on | ||
| failover. | ||
|
|
||
| Verification: `origin/pool_churn` flat in served paths at every pool width | ||
| it already sweeps, and a unit test that counts `select` calls per route | ||
| change and asserts only affected fronts re-select, with zero for an | ||
| unrelated publisher's join and leave, and a parked waiter retrying when a | ||
| deeper advertise-only claim over a served root is withdrawn. Keep `pool_resolve` unchanged. | ||
|
|
||
| Public API: none. Wire: none. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Viewer sessions share a front](/quest/m0/shared-fronts.md) - re-keys fronts, which this index hangs off; it lands after the wildcard line, which is where the pool code lives | ||
|
|
||
| ## Related | ||
|
|
||
| - [Front deadlines](/quest/m1/front-deadline-index.md) - each spurious wake also pays that per-track poll and deadline scan | ||
| - [Front parking](/quest/m1/origin-front-parks.md) - replaces the `routed_broadcast` watch loop this wakes | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.