Skip to content

fix(moq-quic): only a Data-space packet can be the MTU probe - #8967

Merged
kixelated merged 1 commit into
mainfrom
fix/quic-handshake-loss
Nov 17, 2026
Merged

kixelated merged 1 commit into
mainfrom
fix/quic-handshake-loss

Conversation

@kixelated

@kixelated kixelated commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Nightly's Tests (rs quic-sim) failed tests::sim::handshake_under_loss (job 133445816309) on a debug_assert in Slots::insert: "insertions must increase".

Root cause: detect_lost_packets compared every space's packet numbers against the in-flight MTU probe's number, but only the Data space sends probes and every space numbers from 0. In the failing case the client's Handshake pn 0 matched the Data-space probe's pn 0. Loss detection skipped Handshake pn 0 as a "lost MTU probe" and declared pn 1 lost. Then it removed the real probe from the Data space's sent packets, and the next pass declared Handshake pn 0 lost after pn 1, so lost_packets got an insert below its last entry. In release builds the assert is off, so the Data probe is marked lost by mistake and the Handshake packet's loss is delayed.

  • MtuDiscovery::in_flight_mtu_probe and on_non_probe_lost now take the SpaceId, as on_acked already does, and ignore other spaces. The black-hole detector orders losses against probes by packet number, so Handshake losses no longer go into it either.
  • The existing mtud search test now also checks that no Handshake packet number counts as the probe.

The bug came from quinn: quinn-proto 0.11.19 has the same in_flight_mtu_probe() comparison and space-blind on_non_probe_lost.

The quiche handshakeloss interop cell

The other Nightly failure in this area, MISMATCH client=moq-quic server=quiche handshakeloss (job 132842368343), is a different cause. The moq-quic client logged handshake timed out (no panic). That run was on 505a86f, before #8824 raised handshake_timeout from 10 s to 30 s for exactly this case. Any residual is owned by quest/m4/quic-handshake-speedup.md. Interop only began running in Nightly two days ago, and today's jobs (which include #8824) were still running when this opened.

Validation

  • Reproduced deterministically by running handshake(14917314294402731587, 26290µs, loss 0.2415 / 0.1728) (the failing case from the CI log). Before the fix, it panicked in Slots<LostPacket>::insert from detect_lost_packets via on_loss_detection_timeout. Instrumented, it showed client Handshake sent=[0,1,2,3], both well past loss_delay, with only pn 1 declared lost. After the fix it passes.
  • MOQ_QUIC_SEED=573977490 MOQ_QUIC_CASES=4096 (the Nightly configuration) passes, and so do seeds 1, 2, 3 at 4096 cases. Before the fix, seeds 1 to 3 pass too, so the collision is rare (one case in about 16k).
  • cargo test -p moq-quic --lib mtud passes (all mtud and MTU discovery connection tests).
  • just check passes.

Impact

  • Public API: none (crate-private MtuDiscovery).
  • Wire: none. Behavior: a lost Handshake or Initial packet that shares the probe's packet number is now declared lost on time, and the MTU probe is no longer marked lost by mistake.

Caused by

Inherited from quinn with the port (moq-dev/moq#6635 era), so it is as old as the code. Exposed by the new handshake_under_loss sweep from moq-dev/moq#8824.

Models

Size: XS
Implementer: Claude, claude-opus-5-5, unknown effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: 3fab898

  • Base: origin/main at 13e593c

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1e564-18d9-7b33-aebb-680c981acb3a, round 1

  • Score: 9/10, The fix correctly scopes MTU-probe identification and black-hole loss tracking to Data-space packet numbers, with no actionable regressions found by source inspection; builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

Loss detection matched every space's packet numbers against the in-flight
MTU probe, which only the Data space sends. A Handshake packet sharing the
probe's number was skipped as a lost probe (removing the real probe from the
Data space), then declared lost on a later pass after a higher packet, which
broke the lost-packet map's increasing-insert invariant. Black-hole detection
also took other spaces' packet numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 16, 2026 17:26
@coderabbitai

coderabbitai Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f570b169-338b-45c9-b4bc-8f0b2f15418f
Commits

Reviewing files that changed from the base of the PR and between 13e593c and 3fab898.

Files selected for processing: 2

  • rs/moq-quic/src/connection/mod.rs
  • rs/moq-quic/src/connection/mtud.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed the space-scoping fix for MTU probe identification in moq-quic loss detection.

  • in_flight_mtu_probe(space): returns None outside SpaceId::Data, so a Handshake or Initial packet that has the same number as the Data-space probe is no longer skipped as the probe. That also stops the later sent_packets.remove on SpaceId::Data from removing the real probe.
  • on_non_probe_lost(space, …): drops non-Data losses before the black-hole detector sees them. This also removes a latent pn - current.latest_non_probe underflow in BlackHoleDetector::on_non_probe_lost when a Handshake pn is lower than a Data pn already recorded. The signatures now match the existing on_acked(space, …).
  • Tests: call sites updated, and the search test now asserts that no Handshake pn counts as the probe.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed 3fab898a166e0708487691109211b438802ff4d5. No blocker. Probe lookup and black-hole loss accounting now use the Data packet-number space, consistent with ACK handling; Handshake loss can no longer remove the same-numbered Data probe. Since the failing case was rare in the sweep, I'd keep the exact deterministic handshake case as a regression fixture if it runs cheaply. Static review only; I did not rerun the simulator.

@kixelated
kixelated added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit a7e1f28 Nov 17, 2026
6 checks passed
@kixelated
kixelated deleted the fix/quic-handshake-loss branch November 17, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant