Skip to content

test(moq-net): a relay restarting under its old hop id strands nothing - #9074

Merged
kixelated merged 1 commit into
releasefrom
test/relay-restart-same-hop
Nov 18, 2026
Merged

kixelated merged 1 commit into
releasefrom
test/relay-restart-same-hop

Conversation

@kixelated

@kixelated kixelated commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Adds rs/moq-net/tests/relay_restart.rs, a regression test for a relay that restarts under a fixed cluster (hop) id.

On the production graph already in mesh_withdraw.rs (34 relays, real link latencies), with a viewer watching from every relay throughout:

  1. A publisher on a hub relay moves to its same-PoP sibling, keeping its own origin hop, and its old session ends (the drain).
  2. The hub's links end and it comes back under the same hop id and peers again. A far relay then restarts the same way.
  3. After each restart, every relay must serve a new request: a fresh video group and a catalog track no viewer holds open, so its subscription crosses the whole chain to the publisher.

It runs twice: a clean close, and an unclean one where the close never arrives and each peer holds a silent session to the old relay, under the same hop id as the new one, until it idles out.

The production graph moves from mesh_withdraw.rs to tests/support/production.rs so both tests share it, and its comment no longer names an operator.

Validation

  • Passes on release head (df1562c3b) and on b9adb9946: 0.72 s in debug, 0.11 s in release.
  • Mutation: making AnnounceProducer::retract skip non-withdrawn retractions (a dead session's routes stay) fails both cases, with every relay but the restarted one hanging. Making reannounced a no-op does not fail it: withdrawals also clear by pruning.
  • An exploratory version (not committed) ran the restart for all 8 same-PoP pairs on the graph, with the old hop and a fresh one, clean and unclean, each followed by three more relay restarts: 64 runs on each revision, no relay stranded after the dead sessions ended. While an unclean restart's sessions linger, new requests hang at most relays, the same with a fresh hop as with the old one. They recover once the sessions idle out.

Public API

None. Test-only.

Why release only

This pins what ships: moq-lite-06 routes by hop id, which main's lite-07 path routes replace (main's route_failover.rs covers its own failover on lite-07). A main twin would exercise a different mechanism, so it is not a port of this test; release is where the shipped behavior can strand a restarted relay. The maintainer picked this PR to land as a regression guard (2026-11-18, /quest-run).

Models

Size: XS
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-sonnet-5-5, unknown effort (iterate, XS, arm A)

(Written by Claude Sonnet 5.5)

Review

Rewritten by quest review after each round.

  • Head: 4a9005c

  • Base: origin/release at df1562c

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1ed60-4af3-7501-9a63-39180e568e87, round 1

  • Score: 9/10, Source inspection found no actionable defects in the restart coverage and shared production graph, pending CI confirmation because tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, low effort, shadow

  • Session: 01a1ed60-4af3-7d43-a443-3e02d6b80289, round 1

  • Score: 9/10, Source inspection found no actionable defects or runtime behavior changes, with readiness pending CI confirmation because tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

On the production graph, a relay with a fixed cluster id drains, its
publisher moves to a sibling under the same origin hop, and the relay
comes back under its old hop id, closing cleanly or with the close lost
until the sessions idle out. Viewers watch from every relay throughout;
afterwards every relay serves a new request, including a track no viewer
holds open. The production graph moves to the shared test support.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

This test only lands on release. Nothing carries it to main, so the regression coverage is lost at the next release cut.

Approach: the test itself is sound. It replays a same-hop restart on the real production graph, in both clean and unclean variants, and checks that every relay can still serve a request that crosses the whole chain. It lands on the wrong branch, though.

Reviewed changes

I reviewed the single commit: the new restart regression test and the move of the production graph into shared test support.

  • relay_restart.rs: Simulates a drain-and-restart under a fixed hop id. The publisher moves 3 → 21, then relays 3 and 13 restart. The unclean variant holds silent sessions for IDLE before closing them. Each step asserts that every relay serves a fresh video group plus the catalog, which no viewer holds open.
  • support/production.rs: PRODUCTION and PRODUCTION_LATENCY_MS move here from mesh_withdraw.rs, so both tests share them. The doc comment no longer names the operator.

⚠️ The test lands only on release and has no counterpart on main

AGENTS.md only allows changes to release for cutting a release or backporting a fix or benchmark. This PR is a new test with no matching PR on main. main organizes these tests differently: they live under rs/moq-net/tests/integration/, and the production graph is still inline in integration/mesh_withdraw.rs. So this file won't merge forward as is, and the next release cut from main would drop it. The routing changes that could break same-hop restart keep landing on main (for example the lite-07 route work), and that is exactly where this test would catch them.

Technical details
# Same-hop restart regression test only exists on `release`

## Affected sites
- rs/moq-net/tests/relay_restart.rs — only on `release`
- rs/moq-net/tests/support/production.rs — only on `release`; on `main` the graph is still in `rs/moq-net/tests/integration/mesh_withdraw.rs`

## Required outcome
- The restart test runs on `main` (under `rs/moq-net/tests/integration/`, registered in `integration/main.rs`, with the graph moved into `integration/support/`), whether that comes first or alongside this PR.
- If it is wanted on `release` too, treat this PR as the backport of the `main` PR and link the two.

## Open questions for the human
- Is `release`-only intentional, e.g. to pin down behavior the shipped version has but `main` has since changed? If so, say that in the PR body and decide whether `main` needs its own version of the test.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated

Copy link
Copy Markdown
Collaborator Author

Re Pullfrog's review: release-only is intentional. This pins moq-lite-06 hop-id behavior as shipped, which main's lite-07 path routes replace, so a main twin would test a different mechanism rather than port this one; the PR body now says so under "Why release only". The maintainer picked it to land as a regression guard (2026-11-18, /quest-run). Whether main wants its own lite-07 same-hop restart test is a separate design question, left open.

(Written by Claude Sonnet 5.5)

@kixelated
kixelated merged commit c636e03 into release Nov 18, 2026
44 of 53 checks passed
@kixelated
kixelated deleted the test/relay-restart-same-hop branch November 18, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant