Skip to content

fix(net): a refused replacement never ends a track a withdrawn copy still feeds - #8469

Merged
kixelated merged 2 commits into
releasefrom
fix/withdrawn-incumbent-release
Nov 11, 2026
Merged

kixelated merged 2 commits into
releasefrom
fix/withdrawn-incumbent-release

Conversation

@kixelated

@kixelated kixelated commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A publisher's withdrawal can reach a relay before the serving route delivers the track's end. The front then detaches the serving source and re-splices each read track from the next route for the same publisher. When that route is a stale standby whose peer already dropped the broadcast, the standby refuses the track (Unroutable), and Front::redispatch aborts the logical track with that refusal. The detached copy was still feeding the track and would have delivered its remaining groups and a clean end moments later, so the subscriber's track ends with Unroutable instead of finishing cleanly.

main already behaves correctly: moq-dev/moq#4741 keeps a draining copy spliced when the replacement refuses. That PR is a breaking rework of resume and is not on release, so this is a release-only fix rather than a port.

Approach

The front remembers that a copy detached while spliced still feeds the track (Track::draining), and the driver reports that copy's end as a new Event::Drained.

  • A refusal from the replacement no longer aborts a draining track. It waits for the detached copy.
  • The copy ending cleanly finishes the track; ending with an error falls back to the stored refusal (or re-dispatches, if a source can still serve it).
  • A replacement splicing in, a park, or the track ending clears the flag.
  • A replacement whose track metadata differs is still refused at once.

Tests:

  • front::tests::a_refusal_waits_for_the_detached_copy: the state machine, for both a clean and a failed end of the detached copy, and for both ways the copy detaches (its source closing, or a better route replacing it via Selected + Resolved). Each entry point's draining line, commented out, fails it.
  • origin::tests::a_refused_replacement_keeps_a_withdrawn_incumbent_open: end to end through the driver. It fails on release without the fix (the incumbent's clean end, not Err(Unroutable)) and passes with it.

Impact

  • No public API or wire change.
  • Behavior: a track whose serving route is withdrawn while a refusing replacement takes over now ends as the withdrawn copy ends, instead of aborting with the replacement's refusal.

Alternatives

  • Port main's shape (a draining: Option<u64> that puts the track back to Spliced on a refusal, reusing TrackEnded). release's driver buries a detached copy as dead rather than keeping it as the spliced copy, so that would rework the driver's copy handling; a separate Drained event is the smaller change for a backport.

Follow-ups

None.

Caused by

  • moq-dev/moq#3895 - the failover state machine aborts a track as soon as the serving source refuses it, without regard to a detached copy still feeding it.

Checks

just check origin/release (Nix dev shell) passes.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, S)

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, medium effort
  • Session: 01a1c9a9-3b7e-71b3-93e6-1508db59be99, round 1
  • Head: 861b3b2
  • Base: origin/release at adc9f46
  • Score: 9/10, The change preserves detached tracks until completion and immediately refuses incompatible metadata, with no actionable regressions found by source inspection; tests were not run as instructed.
  • Findings: none

…till feeds

When a route is withdrawn while a track it serves is still open, the front
detaches that copy and asks the next route for the same publisher. If that
route no longer has the broadcast (its own withdrawal has not arrived yet),
it refuses the track, and the front aborted the track with the refusal even
though the detached copy was about to deliver its remaining groups and a
clean end.

The front now remembers that a detached copy still feeds the track and
defers a refusal until that copy ends: a clean end finishes the track, an
error falls back to the refusal. Different track metadata is still refused
at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit 83506a3 against release adc9f46. No actionable introduced defects found.

Overall direction: the separate draining state fits the release driver's detached-copy handling. Refusals wait for that copy, successful splices clear the drain, clean completion finishes the logical track, and failed completion preserves pending queries or applies the stored refusal. Repeated failovers, demand/linger transitions, metadata mismatch, and terminal cleanup remain coherent. The state-machine and end-to-end tests target the reported race.

Verification limits: GitHub-only static inspection of both changed files and relevant track/resume semantics. No builds, tests, or independent reproduction executed.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Minor suggestions only — one uncovered branch in the new fix.

Reviewed changes

  • Detached copy as a first-class state: Track::draining plus Event::Drained let a refusal from a replacement wait for the copy that was detached while spliced instead of aborting the logical track, with Ok finishing the track and Err falling back to the stored refusal.
  • Driver reporting: TrackIo::draining is set when Action::Detach buries a spliced copy and cleared on splice, park, teardown, and the new Step::Drained, which polls dead.poll_complete; only io.copy feeds Step::Ended, so a detached copy's end can arrive no other way.
  • Tests: a front state-machine test for both a clean and a failed end of the detached copy, and an end-to-end origin test through a withdrawn incumbent and a refusing standby.

Both new tests pass. Removing the track.draining bail in redispatch makes the end-to-end test fail with the incumbent's clean end, not Err(Unroutable), so it pins the fix as the body claims. The full moq-net lib suite passes (1521 passed, 4 ignored). I also traced the two draining flags through every transition: they stay in agreement (a detach buries a copy whenever the front marks the track spliced, and splice/park/teardown clear both), so the front cannot stall a track waiting for a Drained that never arrives.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Comment thread rs/moq-net/src/model/front.rs Resolved
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Covered the attach-path refusal: a_refusal_waits_for_the_detached_copy now runs the same clean/failed end matrix over both detach paths, using Selected + Resolved Ok (no SourceClosed) to detach a live serving copy so Front::attach's track.draining line is exercised.

Verified the new branch actually pins that line: with track.draining |= ... in attach commented out, the test fails at front.rs:1124 with [Abort { track: "video", err: Unroutable }] where it expects []. Without the mutation the test and the full model::front::tests module pass (32 tests). The SourceClosed case still covers the other entry point, so both detach paths are now honest.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated enabled auto-merge November 11, 2026 08:29
@kixelated
kixelated merged commit cd78f6e into release Nov 11, 2026
21 checks passed
@kixelated
kixelated deleted the fix/withdrawn-incumbent-release branch November 11, 2026 08:54
@pullfrog pullfrog Bot mentioned this pull request Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant