Skip to content

fix(quic): fail a dial at once when the peer has no route - #5668

Merged
kixelated merged 12 commits into
mainfrom
quest/m1/quic/send-errors
Oct 16, 2026
Merged

kixelated merged 12 commits into
mainfrom
quest/m1/quic/send-errors

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Completes quest/m1/quic/send-errors.md, which this PR deletes along with its entry in the quest/m1/quic epic.

Problem

A UDP send failing with ENETUNREACH, EHOSTUNREACH or ENETDOWN can never succeed. Today:

  • moq-tokio: the driver's UdpSenderHelper logs it as loss. A dial to an unroutable family (an IPv6 address on a host with no v6 route, say) waits out the 10s handshake timeout before the Happy Eyeballs race moves on.
  • moq-uring: the error fails the socket's whole send side. One unroutable destination kills every connection that shares the socket.

Change

  • moq_sock::udp::is_unreachable_err classifies the three errors by io::ErrorKind, so Windows WSAE* codes count too. ENOBUFS, EAGAIN, EMSGSIZE, the GSO EINVAL/EIO fallback, and ECONNREFUSED stay retryable.
  • moq-tokio driver: the sender returns these errors to the caller. A handshaking connection closes and fails with TransportError(NO_VIABLE_PATH), and failover::race already starts the next candidate as soon as an attempt fails. An established connection still treats the error as loss, so a route that drops for a moment (a client switching networks) doesn't end it.
  • moq-uring: a send completion with one of these errors records (destination, errno) and wakes the demux task. The demux task fails only the handshakes whose remote is that destination, with Error::Transport { code: 0x10 }. They close and drain normally. Established connections and the socket are unaffected. An ordered (peer, handle) set, filled at launch and emptied when the driver ends, keeps each report to the connections dialing that peer rather than a scan of the whole table.
  • rs/moq-quic/README.md lists this as a carried change.

This ports kixelated/quinn#3 in-tree. The version here differs in two ways. The driver already closed on transmit errors, so its implicit_close clobber fix isn't needed; the driver closes inner before terminating, so the cause survives. And there's a handshake gate, which quinn#3 didn't have.

Public API and wire impact

  • New: pub fn moq_sock::udp::is_unreachable_err(&io::Error) -> bool. moq-tokio and moq-uring both need it.
  • Behavior: on an unroutable destination, moq_tokio::quic::driver::Connecting resolves to ConnectionError::TransportError with code NO_VIABLE_PATH instead of TimedOut. moq-uring dials report quic::Error::Transport { code: 0x10, .. } instead of Io(..). UdpSender::poll_send now documents this contract.
  • No signature changes, and no new error variants.
  • Wire: none. NO_VIABLE_PATH stays local. No CONNECTION_CLOSE can reach a peer we have no route to.

Tests

Each regression test fails without its fix (checked by reverting the fix):

  • moq-tokio quic::driver::tests::unroutable_dial_fails_with_no_viable_path (Linux): a real IPv6-only socket dials a v4-mapped address, and the kernel answers ENETUNREACH without any route changes. Without the fix it reports TimedOut after 10s.
  • moq-tokio quic::driver::tests::unreachable_sends_are_loss_once_established: an established connection survives an outage of unreachable sends and delivers a stream once the route is back.
  • moq-tokio quic::driver::tests::unreachable_handshake_drains_while_held (paused time): a server's 0.5-RTT handle stays alive after its handshake fails on an unreachable peer, and the failed connection still drains, so wait_idle resolves. The driver keeps the closed connection running until it drains instead of returning early.
  • moq-uring::endpoint an_unroutable_dial_fails_alone: the same unroutable dial fails with NO_VIABLE_PATH, and the next dial on the same socket succeeds. Without the fix it fails with Io("Network is unreachable"), because the socket is poisoned.
  • Unit tests cover the errno classifier and the sender helper. a_raw_socket_sees_an_unreachable_send_fail checks that a socket with no consumer queues nothing and fails its later sends.
  • Benchmark moq-uring --bench unreachable_dials sweeps background handshakes {1, 256, 4096} against failed destinations {1, 8, 64}. The cost is flat at about 80 µs per failed dial. A full-table scan took 12 to 16 ms for 64 destinations, against 4.3 to 5 ms with the index.

just check against main passed: lint, plus 4748 of 4748 tests. I didn't run just test interop --all: this changes no wire format and touches no FFI or gateway code.

Decisions

The maintainer accepted these:

  1. Handshake gate: unreachable errors end only connections that are still handshaking; established ones treat them as loss.
  2. Error signal: this reuses the registered NO_VIABLE_PATH transport code instead of adding a ConnectionError variant.
  3. No consumer: fixed rather than left open. moq-uring queues unreachable reports only once the QUIC endpoint registers for them. A raw udp::Socket with no consumer gets the error through poll_acquire, as before.

Follow-ups

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 5 commits October 15, 2026 19:35
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…batim

Unmodified copies, not yet compiled, so a reviewer can diff the adaptation
that follows against upstream:

- quinn-rs/quinn 7616e6b2782722f3ce4a1b181ef08817d7f545e4 quinn/src ->
  rs/moq-tokio/src/quic/driver (lib.rs -> mod.rs), the commit moq-quic
  forked from.
- moq-dev/noq e27c0ce00741d81c8620b98e345d4f2c88e0768c web-transport-moq/src ->
  rs/moq-tokio/src/quic/web (lib.rs -> mod.rs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop every moq-noq dependency. moq-tokio's QUIC backend now drives moq-quic
through quinn's async layer (`quic::driver`) with web-transport-moq folded in
as `quic::web`; moq-uring's sans-IO driver moves from moq-noq-proto to
moq-quic. Upstream noq stays only behind moq-tokio's `iroh` feature.

- The `noq` cargo feature is `quic` everywhere it was forwarded.
- `moq_tokio::noq::Error` is `moq_tokio::quic::Error`; `Error::Noq` is
  `Error::Quic`.
- moq-quic takes `moq_clock::Instant` and raises a once-per-connection
  Draining endpoint event, which backs the driver's `wait_all_draining`.
- qlog writes one trace per endpoint (per dial on moq-uring's connect side),
  since moq-quic holds one sink per transport config.
- Session stats report the congestion controller's bandwidth estimate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UDP send that fails with ENETUNREACH, EHOSTUNREACH or ENETDOWN can
never succeed, but the tokio driver logged it as loss, so a dial to an
unroutable family waited out the 10s handshake timeout before the
Happy Eyeballs race moved on. moq-uring was worse: the error failed the
socket's whole send side, killing every connection sharing it.

moq_sock::udp::is_unreachable_err classifies these errors. The tokio
driver's sender surfaces them, and a handshaking connection fails at
once with NO_VIABLE_PATH. moq-uring hands each one to the demux task,
which fails only the handshakes dialing that destination. Established
connections keep treating them as loss, so a route that drops for a
moment (a client switching networks) does not end them.

Ported from kixelated/quinn#3, addressing quinn-rs/quinn#2766.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome: the quest is complete, so this PR deletes the quest file. The PR stays a draft, stacked on #5664.

  • just check against quest/m1/quic/fork/switch: lint passed, and 4706 of 4706 tests passed. A rerun after just fix hit a shared RLIMIT_MEMLOCK ENOMEM in the untouched moq-uring::ecn ect0_gso_train. A clean rerun of moq-uring --features quic --tests at low load then passed 90 of 90.
  • Each new regression test was checked to fail with its fix reverted.

Open decision for the maintainer: the handshake gate. Unreachable errors end only connections that are still handshaking; established ones still treat them as loss. I recommend keeping it, because a client switching networks briefly sees ENETUNREACH, and that shouldn't kill a live session. The other option is to fail every connection, as kixelated/quinn#3 did. That is simpler, but it regresses network handoff.

(Written by Claude Opus 5.5)

Base automatically changed from quest/m1/quic/fork/switch to main October 16, 2026 16:04
kixelated and others added 2 commits October 16, 2026 09:10
…rors

# Conflicts:
#	doc/setup/upgrade.md
#	rs/moq-tokio/src/log.rs
#	rs/moq-tokio/src/quic/backend.rs
#	rs/moq-tokio/src/quic/driver/connection.rs
#	rs/moq-tokio/src/quic/driver/endpoint.rs
#	rs/moq-tokio/src/quic/driver/recv_stream.rs
#	rs/moq-tokio/src/quic/driver/runtime/mod.rs
#	rs/moq-tokio/src/quic/driver/send_stream.rs
#	rs/moq-tokio/src/quic/driver/tests.rs
#	rs/moq-tokio/src/quic/web/connect.rs
#	rs/moq-tokio/src/quic/web/error.rs
#	rs/moq-tokio/src/quic/web/session.rs
#	rs/moq-tokio/src/quic/web/settings.rs
#	rs/moq-uring/README.md
#	rs/moq-uring/src/quic/driver/mod.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 16, 2026 16:26
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 6 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fec614d7-8c8a-4c6b-8e80-8bd0a0e9bf9d
📥 Commits

Reviewing files that changed from the base of the PR and between e600679 and fc3002d.

📒 Files selected for processing: 13
  • quest/m1/quic/README.md
  • quest/m1/quic/send-errors.md
  • rs/moq-quic/README.md
  • rs/moq-sock/src/udp/mod.rs
  • rs/moq-tokio/src/quic/driver/connection.rs
  • rs/moq-tokio/src/quic/driver/runtime/mod.rs
  • rs/moq-tokio/src/quic/driver/tests.rs
  • rs/moq-uring/Cargo.toml
  • rs/moq-uring/benches/unreachable_dials.rs
  • rs/moq-uring/src/quic/driver/connection.rs
  • rs/moq-uring/src/quic/driver/endpoint.rs
  • rs/moq-uring/src/udp.rs
  • rs/moq-uring/tests/endpoint.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T18:13:45.944044Z fc3002d Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99b6d83858

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Each unreachable send scanned the whole connection table, so failing M
dials among N connections cost O(M x N), quadratic when every dial has
its own destination, on the single-threaded demux task. An ordered set
of (peer, handle), filled at launch and emptied when the driver ends,
narrows each report to the connections dialing that peer.

The new unreachable_dials benchmark sweeps background handshakes
against failed destinations; the per-dial cost is now flat on both
axes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce1015327c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-uring/src/udp.rs Resolved
A raw udp::Socket, or a build without the quic feature, has nothing to
drain unreachable reports, so they piled up forever. The report queue
now exists only once the QUIC endpoint asks for it; without one, an
unreachable send fails later sends like any other error, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: ac9a4a5859

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…rors

# Conflicts:
#	quest/m1/quic/send-errors.md
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3706b4006e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

kixelated and others added 2 commits October 16, 2026 10:57
The driver returned as soon as it failed the handshake, before it could
publish the draining and drained events. An application still holding
the connection (0-RTT, or a server's 0.5-RTT handle) kept State alive,
so the endpoint never forgot it and wait_idle hung. The driver now
keeps running the closed connection until it drains; its
CONNECTION_CLOSE counts as loss like any send after the handshake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: fc3002d320

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated
kixelated added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 5f24a69 Oct 16, 2026
15 checks passed
@kixelated
kixelated deleted the quest/m1/quic/send-errors branch October 16, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant