Repository navigation
fix(moq-net): a dead instance held unavailable never outranks a live one - #9156
Conversation
Quest-Claim: c7892a280d15f22f
The origin ranked the route layer's closest way (Lane::primary) ahead of availability, so while a route change moved a live instance's closest way to another session, a restarted publisher's dead instance, held unavailable, won the path back for a moment: a reader saw C -> B -> C. The closest way now ranks below availability, where the doc already put it, and the lite-07 draft ranks an Unavailable claim with a draining one, as JS already did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 1 minute. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
There was a problem hiding this comment.
✅ No new issues found.
Approach: sound. It fixes the cause in route_order: primary ranked ahead of availability, so a dead instance's held claim beat a live instance whose closest way is demoted for a moment. The fix doesn't hide the flip at the reader, and the new order matches the function's own doc, JS compareRoutes, and now the draft.
Reviewed changes
Reviewed the full diff at 45ad829: the route_order key change, the matching draft and doc updates, and the new regression test.
route_orderreorder:!entry.lane.primarynow ranks afterdraining || !available. This holds becauseroute/node.rsrelanemarks every wayavailable: trueor flips them all to false together.Router::closestalready ranks byclaim_orderwith drain first, soprimaryand non-draining rarely disagree. Where they do, a live, non-draining way should win.- Draft lite-07-wip Resolution: the receiver now prefers a claim that is neither draining nor
Unavailable, and the changelog says so. This matches the Rust and JS ranking. The encoding is unchanged. - Docs:
identity.mdstep 3,NodeFilter::Any, and theroute_orderdoc comment now describeprimaryas each broadcast's closest way. - Regression test: a cheaper dead instance is held unavailable while a live instance's way is demoted. The cursor stays on the live instance and requests go to its way. The test fails on
mainby construction.
claude-opus-5-5 | 𝕏

Finishes quest/m2/cluster-routes/dead-instance-flip.md and deletes it.
Problem
A reader of a path whose publisher restarted (old instance B, new instance C) could see the path go C → B → C. moq-dev/moq#9108 folds that flip into an Update when the reader doesn't poll in between. A reader that polls mid-flip still sees B, and the sweep's NoRegression rule would flag it.
The quest assumed B won through a stale path. Tracing the nightly case (
MOQ_ROUTE_SEED=255657690 MOQ_ROUTE_CASE=41) showed something else. At relay 0, when a route change moves C's closest way from session A to session B, the route node demotes the old way on A's queue (primary: false) and serves the new one on B's queue. The two queues apply separately. In between, C's only entry is non-primary.route_orderranked!lane.primaryahead ofdraining || !available, so B won: B is unavailable (its relay lost the publisher's session and holds the claim for 800 ms), but it still counts as primary. C won back once B's queue applied.Fix
route_orderranks the closest way below availability, as its own doc comment already said. The route layer marks every way to one broadcast available, or none, soprimarystill orders an instance's own ways. A live instance's other way now outranks a dead instance held unavailable.Unavailable, and the changelog says so. Before, the draft ranked by drain, priority, and cost only, so a cheaper Unavailable claim of a dead instance beat a live one. An Unavailable claim serves nothing (its sender fails requests with UNROUTABLE), so no case needs it to win. JS (compareRoutes) and Rust already ranked unavailable with draining; the draft was the outlier.doc/concept/identity.mdlists an unavailable route with a draining one. Stale docs onNodeFilter::Anyandroute_ordernow describe whatprimarymeans since feat(moq-net)!: lite-07 sessions run the new route rule #8987.Impact
Unavailableclaim below every available one, which matches both implementations. lite-03 to lite-06 and moq-transport are unchanged.Decisions
/verify-failoverfollows one track through one instance's routes, and/verify-warmmodels warm pulls. Neither has two instances at one path, or availability across instances, so neither can show a case where B must win. The argument stands without them: per the draft, an Unavailable claim's sender refuses every request for it, so ranking it above an available claim only routes requests to a refusal. The differential sweep below measures the effect.Validation
a_dead_instance_never_outranks_a_live_one_between_its_ways: a cheaper dead instance held unavailable, and a live one whose closest way is demoted. It fails onmain(the cursor restarts to the dead instance) and passes with the fix.MOQ_ROUTE_SEED=255657690 MOQ_ROUTE_CASE=41(release, asroute-sweepruns it): fails onmainwithRESTART to the same instanceand passes with this fix, without fix(moq-net): a restart away and back before the next poll is an update #9108.main+ fix(moq-net): a restart away and back before the next poll is an update #9108: the sweep passes, but 203 such events remain, 36 of them on cursors with no split horizon (readers). The flip still happens. fix(moq-net): a restart away and back before the next poll is an update #9108 only hides it from a reader that doesn't poll in between.Enforce), which reads paths, not instances, so nothing goes on the wire.just checkpasses (10,724 Rust tests). The first run hit four 60 s timeouts inutil::idleandtransport::tests::stop_discards_the_bufferunder load. Those passed alone in 0.03 s and in a full rerun, and this change doesn't touch them.Not covered
Caused by
selectedflag intoprimarybut keptprimarywhere the old "narrowest route" meaning ranked, ahead of availability. A dead instance held unavailable then outranked a live instance's demoted way (found by check: nightlyroute-sweepandbenchmark_route_mesh).Follow-ups
None.
Models
Size: M
Implementer: Claude, claude-opus-5-5, unknown effort (arm B)
(Written by Claude Opus 5.5)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.Head: 45ad829
Base: origin/main at 61e4a38
Risk: medium,
the base has no RISK.md to name what is always high or customer-facing, so every change is mediumClassified: medium by default
Reviewer: Codex, gpt-6.1-sol, high effort
Session:
01a1f307-9fbf-76b3-9863-6d584cbe4b4b, round 1Score: 9/10,
The shared ordering change prevents unavailable primary routes from displacing live alternatives while preserving specificity, epoch precedence, and existing JS behavior; no actionable regressions were found, though builds and tests were not run as instructed.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none