diff --git a/doc/bin/relay/config.md b/doc/bin/relay/config.md index a9a19ba13af..11b228cdd29 100644 --- a/doc/bin/relay/config.md +++ b/doc/bin/relay/config.md @@ -174,8 +174,8 @@ first cores, so two relays on one host, or one limited to fewer CPUs than the host has, would crowd onto them. Turn it on for a host the relay has to itself. `io_uring` additionally needs Linux 6.12+ and the `io-uring` cargo feature; it -shares the same reloadable TLS identity, serves every configured version -like the shared runtime, and refuses `mtu_discovery`. A setting the build or host cannot deliver refuses to +shares the same reloadable TLS identity and serves every configured version +like the shared runtime. A setting the build or host cannot deliver refuses to start rather than being ignored. If io\_uring workers fail to start naming `RLIMIT_MEMLOCK`, raise it (`LimitMEMLOCK=` under systemd). diff --git a/quest/m1/README.md b/quest/m1/README.md index 2e0126409dd..3039e1dc96a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -87,7 +87,6 @@ tracks, so per-group stats stop costing cluster announces. - [No reused track names](/quest/m1/name-reuse.md) - no first-party publisher re-creates a finished track under its old name - [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation setup - [Controlled time](/quest/m1/time/README.md) - every crate and package reads time through moq-clock or @moq/clock, with regression tests for the changed contracts -- [io_uring MTU matches tokio](/quest/m1/uring-mtu.md) - a 1200-byte floor and `quic.mtu_discovery` honored, so the ring stops black-holing smaller paths - [io_uring listen extensions](/quest/m1/uring-listen-extensions.md) - ring listeners offer the configured `[listen] extensions` - [Rust request credit](/quest/m1/rs-request-credit.md) - a moq-net request fails at once without stream credit instead of waiting - [Generated C bindings](/quest/m1/c/README.md) - C generated from moq-ffi ships as `moq-c` 0.8.0 and replaces the hand-written libmoq diff --git a/quest/m1/uring-mtu.md b/quest/m1/uring-mtu.md deleted file mode 100644 index 4f7259bf8af..00000000000 --- a/quest/m1/uring-mtu.md +++ /dev/null @@ -1,46 +0,0 @@ -# [M] io_uring MTU matches tokio - -## Goal - -An io_uring relay connects over any path that carries 1200-byte QUIC -payloads, exactly as the tokio backend does, and `quic.mtu_discovery` is -honored instead of refused: off by default, discovering larger MTUs when on. - -## Plan - -`rs/moq-uring/src/quic/driver/mod.rs` pins the initial and minimum MTU to -`SEGMENT` (1350), disables discovery, and advertises a 1350 -`max_udp_payload_size`; `rs/moq-relay/src/uring.rs` refuses -`quic.mtu_discovery`. Tokio uses moq-quic's 1200 floor and 1472 -advertised, and maps the resolved `quic.mtu_discovery` (default false) -onto `mtu_discovery_config` (`rs/moq-tokio/src/quic/backend.rs`). A path below ~1378 bytes (IPv4) or ~1398 -(IPv6) black-holes on the ring with no fallback. - -The pin dates from the quiche-era driver and was never re-justified. Its GSO -rationale no longer holds: `flush_one` already sends with moq-quic's own -`transmit.segment_size`, and moq-quic sizes GSO batches itself (#6279). Only -`segments = capacity / SEGMENT` in `connection.rs` assumes the fixed size; -derive it from the current MTU instead. - -Drop the overrides so the ring uses the same `TransportConfig` defaults as -tokio, add the resolved setting to `moq_uring::quic::Transport` so -discovery stays off unless configured, and lift the relay's refusal. Also -pass `allow_mtud = true` to `moq_quic::Endpoint::new` -(`rs/moq-uring/src/quic/driver/endpoint.rs`), which hardcodes `false` -today and would veto discovery for every connection whatever the setting; -it is sound only once the sockets refuse to fragment (`PMTUDISC_PROBE` plus -`IPV6_DONTFRAG`, from the send-errors quest), so land that first or set -them here. Reproduce first: a test path that -passes 1200-byte payloads but drops 1350 fails today and connects after. -Then cover discovery staying off by default, a path MTU change -mid-connection, and a larger MTU being discovered when enabled, with GSO -trains still batching. - -Decided 2026-10-22 from the io_uring audit: m1, since it black-holes real -paths. - -Public API: additive, a `mtu_discovery: bool` field on the -`#[non_exhaustive]` `moq_uring::quic::Transport`, named after -`moq_tokio::quic::Resolved::mtu_discovery`. Wire: none. Config: -`quic.mtu_discovery` works under `--runtime-io-uring`; update -`doc/bin/relay/` if it mentions the refusal. diff --git a/quest/m2/perf/uring-profile.md b/quest/m2/perf/uring-profile.md index 221ca382e1f..63af9989598 100644 --- a/quest/m2/perf/uring-profile.md +++ b/quest/m2/perf/uring-profile.md @@ -28,8 +28,8 @@ profile, and plan a quest only for one that shows up: - atomic RMWs in worker metrics (remote wake counters genuinely have several writers); - receive rearm churn that `IORING_RECVSEND_POLL_FIRST` would cut; -- GSO batching on ordinary and jumbo paths after - [MTU parity](/quest/m1/uring-mtu.md). +- GSO batching on ordinary and jumbo paths, now that discovery can raise + the MTU. Decided 2026-10-22 from the io_uring audit. diff --git a/quest/m2/relay-io-uring-package.md b/quest/m2/relay-io-uring-package.md index b3be3f85f02..54705a0c632 100644 --- a/quest/m2/relay-io-uring-package.md +++ b/quest/m2/relay-io-uring-package.md @@ -42,7 +42,6 @@ Public API: none. Wire: none. ## Required -- [io_uring MTU matches tokio](/quest/m1/uring-mtu.md) - `quic.mtu_discovery` honored and smaller paths reachable - [io_uring handle lifecycle](/quest/m2/uring-lifecycle.md) - handles fail with their worker and close on last drop - [Ring drain GOAWAY](/quest/m2/uring-ring-drain.md) - closes are delivered on the ring for moq-transport too - [One port on the io_uring workers](/quest/m2/uring-demux.md) - a one-port config is served on the ring too diff --git a/rs/moq-relay/src/uring.rs b/rs/moq-relay/src/uring.rs index 8a348d88763..c31971b8944 100644 --- a/rs/moq-relay/src/uring.rs +++ b/rs/moq-relay/src/uring.rs @@ -414,12 +414,6 @@ impl std::fmt::Debug for Workers { /// One qlog sink is built for the whole group and cloned into every worker, so /// the traces share a single writer thread however many workers there are. fn transport(quic: &moq_tokio::quic::Resolved) -> anyhow::Result { - // The datagram path fixes both payload ceilings at SEGMENT and hands - // `conn.send` slices of exactly that, so there is no larger size to find. - anyhow::ensure!( - !quic.mtu_discovery, - "io_uring workers send a fixed-size UDP payload, so quic.mtu_discovery has nothing to discover" - ); // The tokio path reports this through `moq_tokio::Error::QlogUnsupported`, // which nothing builds here: this listener is the io_uring stack. #[cfg(not(feature = "qlog"))] @@ -449,6 +443,7 @@ fn transport(quic: &moq_tokio::quic::Resolved) -> anyhow::Result return Poll::Pending, }; - let segments = (tx.capacity() / SEGMENT).min(TRAIN_SEGMENTS); - if segments == 0 { - return Poll::Ready(Err(Error::Io(format!( - "transmit buffer of {} bytes holds no {SEGMENT} byte segment", - tx.capacity() - )))); - } - let (transmit, closing) = { let mut conn = self.shared.conn.borrow_mut(); + // moq-quic packs every full datagram at the current MTU, which + // discovery can raise, so a train sized for any smaller one would + // outgrow the buffer. + let mtu = usize::from(conn.current_mtu()); + let segments = (tx.capacity() / mtu).min(TRAIN_SEGMENTS); + if segments == 0 { + return Poll::Ready(Err(Error::Io(format!( + "transmit buffer of {} bytes holds no {mtu} byte datagram", + tx.capacity() + )))); + } // moq-quic encrypts the train straight into the socket's buffer. // Nothing to send, or paced; the buffer returns to the pool on drop. let Some(transmit) = conn.poll_transmit(self.shared.owner.now(), segments, &mut tx) else { diff --git a/rs/moq-uring/src/quic/driver/endpoint.rs b/rs/moq-uring/src/quic/driver/endpoint.rs index b34e88b2c94..2159e85778b 100644 --- a/rs/moq-uring/src/quic/driver/endpoint.rs +++ b/rs/moq-uring/src/quic/driver/endpoint.rs @@ -102,12 +102,13 @@ impl Endpoint { None => None, }; let accepting = server.is_some().then(|| Accepting { queue: VecDeque::new() }); - // MTU discovery is off (the GSO pool sends fixed SEGMENT datagrams), - // so the endpoint has no reason to allow it either. + // Every `udp::Socket` refuses to fragment, so a probe too large for the + // path is lost rather than delivered in pieces, and discovery can trust + // what arrives. Each connection's transport still decides whether to probe. let endpoint = moq_quic::Endpoint::new( - super::endpoint_config(socket.shard(), config.reset_secret.as_ref())?, + super::endpoint_config(socket.shard(), config.reset_secret.as_ref()), server, - false, + true, ); let inner = Rc::new(Inner { diff --git a/rs/moq-uring/src/quic/driver/mod.rs b/rs/moq-uring/src/quic/driver/mod.rs index b4d516fca0a..603edf4020d 100644 --- a/rs/moq-uring/src/quic/driver/mod.rs +++ b/rs/moq-uring/src/quic/driver/mod.rs @@ -29,7 +29,7 @@ use rustls::pki_types::{CertificateDer, PrivateKeyDer}; #[cfg(feature = "qlog")] use super::qlog; -use super::{Congestion, Error, Identity, SEGMENT, Transport, client, server}; +use super::{Congestion, Error, Identity, Transport, client, server}; use crate::udp; /// Per-stream flow control credit, which returns as bytes are read, so it only paces a stream. @@ -41,8 +41,8 @@ const CONNECTION_WINDOW: u32 = 128 * 1024 * 1024; const _: () = assert!(CONNECTION_WINDOW as u64 >= moq_net::transport::MIN_RECEIVE_WINDOW); /// Unacknowledged outgoing data to buffer, whatever the peer allows. const SEND_WINDOW: u32 = 16 * 1024 * 1024; -/// How many datagrams to buffer in each direction. -const DATAGRAM_WINDOW: usize = 64 * SEGMENT; +/// Datagram bytes to buffer in each direction, about 64 full-size datagrams. +const DATAGRAM_WINDOW: usize = 96 * 1024; fn ecn_to_quic(ecn: udp::Ecn) -> moq_quic::EcnCodepoint { match ecn { @@ -96,12 +96,11 @@ fn provider() -> Arc { .clone() } -/// The endpoint-wide configuration: how connection ids are minted, the reset -/// key, and the largest datagram we tell peers we can receive. +/// The endpoint-wide configuration: how connection ids are minted and the reset key. pub(crate) fn endpoint_config( shard: Option, reset_secret: Option<&[u8; 32]>, -) -> Result, Error> { +) -> Arc { let mut config = moq_quic::EndpointConfig::default(); // The random ids below validate a predecessor's too, so the reset reaches them. if let Some(secret) = reset_secret { @@ -119,10 +118,7 @@ pub(crate) fn endpoint_config( } Box::new(cids) })); - config - .max_udp_payload_size(SEGMENT as u16) - .map_err(|err| Error::Quic(err.to_string()))?; - Ok(Arc::new(config)) + Arc::new(config) } /// Dial as `config` says. @@ -282,11 +278,10 @@ fn transport_config(config: &Transport) -> Result Arc::new(moq_quic::congestion::CubicConfig::default()) as Arc, diff --git a/rs/moq-uring/src/quic/mod.rs b/rs/moq-uring/src/quic/mod.rs index 675335908b1..7d5a2d31e18 100644 --- a/rs/moq-uring/src/quic/mod.rs +++ b/rs/moq-uring/src/quic/mod.rs @@ -35,10 +35,6 @@ pub use endpoint::Endpoint; /// The flow-control limits a live connection can change; see [`Connection::set_limits`]. pub use moq_quic::{Limits, LimitsError}; -/// The QUIC payload size every full datagram in a GSO train uses, and the -/// stride GRO coalesces with. -pub(crate) const SEGMENT: usize = 1350; - /// A TLS certificate chain and the private key that signs for it, as PEM. /// One value, so neither half can be configured alone. /// @@ -138,6 +134,10 @@ pub struct Transport { /// connection, or `None` (the default) to send none and let the idle /// timeout decide. pub keep_alive: Option, + /// Probe for a path MTU above the 1200-byte floor every QUIC path carries, + /// off by default. A probe the path drops costs only itself, and a path + /// that later shrinks falls back to the floor. + pub mtu_discovery: bool, /// Where to write qlog traces, or `None` (the default) to write none. /// /// moq-quic takes one sink per transport config, so this writes one file @@ -160,6 +160,7 @@ impl Default for Transport { send_window: None, congestion: Congestion::default(), keep_alive: None, + mtu_discovery: false, #[cfg(feature = "qlog")] qlog: None, } diff --git a/rs/moq-uring/tests/integration/main.rs b/rs/moq-uring/tests/integration/main.rs index b4a292848d6..737bd2649ec 100644 --- a/rs/moq-uring/tests/integration/main.rs +++ b/rs/moq-uring/tests/integration/main.rs @@ -15,6 +15,7 @@ mod ecn; mod endpoint; mod identity; mod local_addr; +mod mtu; mod qlog; mod receive_credit; mod session; diff --git a/rs/moq-uring/tests/integration/mtu.rs b/rs/moq-uring/tests/integration/mtu.rs new file mode 100644 index 00000000000..b03de41da57 --- /dev/null +++ b/rs/moq-uring/tests/integration/mtu.rs @@ -0,0 +1,374 @@ +//! Path MTU through the io_uring driver: a path that carries only the QUIC +//! floor still connects, discovery stays off unless asked, and when on it +//! finds a larger MTU, keeps batching GSO trains at it, and falls back when +//! the path shrinks. +//! +//! The path is a userspace hop that drops whatever exceeds its MTU without a +//! word, the way a tunnel behind a firewall that eats ICMP does. It crosses +//! real sockets, so these run on the wall clock. +//! +//! Unsupported local kernels skip loudly; MOQ_STRICT makes CI refuse them. + +#![cfg(all(target_os = "linux", feature = "quic"))] + +use crate::support; + +use std::net::{SocketAddr, UdpSocket}; +use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; +use std::task::Poll; +use std::time::Duration; + +use moq_uring::{Config, Error, Worker, quic, udp}; +use web_transport_trait::poll::{RecvStream as _, SendStream as _, Session as _}; + +const ALPN: &str = "moq-uring-mtu"; +/// The UDP payload every QUIC path must carry, and moq-quic's floor. +const FLOOR: usize = 1200; +/// The UDP payload an IPv6 path must carry: the 1280-byte minimum link MTU +/// less the IPv6 and UDP headers. Above the floor, below a 1350-byte pin. +const IPV6_MIN: usize = 1232; +/// The UDP payload of an Ethernet path over IPv4. +const ETHERNET: usize = 1472; +/// Enough data for full GSO trains. +const PAYLOAD: usize = 1 << 20; + +fn worker() -> Option { + match Worker::new(Config::default()) { + Ok(worker) => Some(worker), + Err(Error::Unsupported(reason)) if std::env::var_os("MOQ_STRICT").is_none() => { + eprintln!("skipping io_uring MTU test: {reason}"); + None + } + Err(err) => panic!("worker setup failed: {err}"), + } +} + +/// What one direction of a [`Path`] forwarded. +#[derive(Default)] +struct Seen { + /// The largest datagram, in bytes. + largest: AtomicUsize, + /// Datagrams larger than the floor. + large: AtomicU64, +} + +impl Seen { + fn forwarded(&self, len: usize) { + self.largest.fetch_max(len, Ordering::Relaxed); + if len > FLOOR { + self.large.fetch_add(1, Ordering::Relaxed); + } + } +} + +/// A hop between one client and `server` that silently drops every datagram +/// larger than its MTU, which can change mid-connection. +struct Path { + addr: SocketAddr, + mtu: Arc, + /// Toward the server, then back toward the client. + seen: Arc<[Seen; 2]>, + stop: Arc, + threads: Vec>, +} + +impl Path { + fn new(server: SocketAddr, mtu: usize) -> Self { + let front = UdpSocket::bind("127.0.0.1:0").expect("bind the client side"); + let back = UdpSocket::bind("127.0.0.1:0").expect("bind the server side"); + back.connect(server).expect("connect to the server"); + // Short, so a thread notices `stop` promptly. + for sock in [&front, &back] { + sock.set_read_timeout(Some(Duration::from_millis(10))) + .expect("read timeout"); + } + let addr = front.local_addr().expect("path address"); + let front = Arc::new(front); + let back = Arc::new(back); + let client: Arc>> = Arc::default(); + let mtu = Arc::new(AtomicUsize::new(mtu)); + let seen: Arc<[Seen; 2]> = Arc::default(); + let stop = Arc::new(AtomicBool::new(false)); + + let up = { + let (front, back, client, mtu, seen, stop) = ( + front.clone(), + back.clone(), + client.clone(), + mtu.clone(), + seen.clone(), + stop.clone(), + ); + std::thread::spawn(move || { + let mut buf = vec![0u8; 64 * 1024]; + while !stop.load(Ordering::Relaxed) { + let Ok((len, from)) = front.recv_from(&mut buf) else { + continue; + }; + *client.lock().unwrap() = Some(from); + if len <= mtu.load(Ordering::Relaxed) && back.send(&buf[..len]).is_ok() { + seen[0].forwarded(len); + } + } + }) + }; + let down = { + let (mtu, seen, stop) = (mtu.clone(), seen.clone(), stop.clone()); + std::thread::spawn(move || { + let mut buf = vec![0u8; 64 * 1024]; + while !stop.load(Ordering::Relaxed) { + let Ok(len) = back.recv(&mut buf) else { + continue; + }; + let Some(to) = *client.lock().unwrap() else { + continue; + }; + if len <= mtu.load(Ordering::Relaxed) && front.send_to(&buf[..len], to).is_ok() { + seen[1].forwarded(len); + } + } + }) + }; + + Self { + addr, + mtu, + seen, + stop, + threads: vec![up, down], + } + } + + fn set_mtu(&self, mtu: usize) { + self.mtu.store(mtu, Ordering::Relaxed); + } + + /// The largest datagram forwarded either way. + fn largest(&self) -> usize { + self.seen + .iter() + .map(|seen| seen.largest.load(Ordering::Relaxed)) + .max() + .unwrap_or_default() + } + + /// Datagrams larger than the floor forwarded toward the server. + fn large_up(&self) -> u64 { + self.seen[0].large.load(Ordering::Relaxed) + } +} + +impl Drop for Path { + fn drop(&mut self) { + self.stop.store(true, Ordering::Relaxed); + for thread in self.threads.drain(..) { + let _ = thread.join(); + } + } +} + +/// Await `future`, failing loudly rather than hanging if it never resolves. +async fn within(handle: &moq_uring::Handle, what: &str, future: impl Future) -> T { + let mut deadline = moq_uring::Timer::after(handle, Duration::from_secs(10)); + let mut future = std::pin::pin!(future); + kio::wait(|waiter| { + let mut cx = waiter.context(); + if let Poll::Ready(value) = future.as_mut().poll(&mut cx) { + return Poll::Ready(Some(value)); + } + deadline.poll(waiter).map(|()| None) + }) + .await + .unwrap_or_else(|| panic!("timed out waiting for {what}")) +} + +/// A server endpoint and a dial-only client, both running `transport`, with +/// the client's dial going through a path of `mtu`. +fn endpoints( + handle: &moq_uring::Handle, + transport: &quic::Transport, + mtu: usize, +) -> (quic::Endpoint, quic::Endpoint, quic::client::Config, Path) { + let certs = support::certs().expect("certificates"); + let socket = |handle: &moq_uring::Handle| { + handle + .udp(UdpSocket::bind("127.0.0.1:0").expect("bind"), udp::Config::default()) + .expect("socket") + }; + let mut server_config = quic::server::Config::new( + quic::Identity::open(&certs.cert, &certs.key) + .expect("identity") + .server(&[ALPN], quic::server::ClientAuth::None) + .expect("tls"), + ); + server_config.transport = transport.clone(); + let server = quic::Endpoint::new( + socket(handle), + quic::endpoint::Config::default().with_server(server_config), + ) + .expect("server endpoint"); + let client = quic::Endpoint::new(socket(handle), quic::endpoint::Config::default()).expect("client endpoint"); + let path = Path::new(server.local_addr(), mtu); + let mut dial = quic::client::Config::new(path.addr, "localhost"); + dial.alpn = vec![ALPN.to_string()]; + dial.verify = false; + dial.transport = transport.clone(); + (server, client, dial, path) +} + +/// Dial through the path and accept the far end. +async fn connect( + handle: &moq_uring::Handle, + server: &quic::Endpoint, + client: &quic::Endpoint, + dial: &quic::client::Config, +) -> (quic::Connection, quic::Connection) { + let dialed = within(handle, "the handshake", client.connect(dial)) + .await + .expect("dial"); + let accepted = within(handle, "the accept", server.accept()).await.expect("accept"); + (dialed, accepted) +} + +/// Send `len` bytes on a fresh stream from `from` and read them all at `to`. +async fn transfer(handle: &moq_uring::Handle, from: &mut quic::Connection, to: &mut quic::Connection, len: usize) { + let mut send = std::future::poll_fn(|cx| from.poll_open_uni(cx)) + .await + .expect("open_uni"); + handle.spawn(async move { + let payload = vec![7u8; len]; + let mut buf = &payload[..]; + while !buf.is_empty() { + match std::future::poll_fn(|cx| send.poll_write(cx, buf)).await { + Ok(n) => buf = &buf[n..], + Err(_) => return, + } + } + let _ = send.finish(); + let _ = std::future::poll_fn(|cx| send.poll_closed(cx)).await; + }); + + let read = within(handle, "the transfer", async { + let mut recv = std::future::poll_fn(|cx| to.poll_accept_uni(cx)) + .await + .expect("accept_uni"); + let mut buf = vec![0u8; 64 * 1024]; + let mut read = 0; + while let Some(n) = std::future::poll_fn(|cx| recv.poll_read(cx, &mut buf)) + .await + .expect("read") + { + read += n; + } + read + }) + .await; + assert_eq!(read, len, "the whole stream arrived"); +} + +/// A path that carries the floor but not a byte more than IPv6 promises +/// connects and carries data. +#[test] +fn a_path_at_the_ipv6_minimum_connects() { + let Some(mut worker) = worker() else { return }; + let handle = worker.handle(); + let (server, client, dial, path) = endpoints(&handle, &quic::Transport::default(), IPV6_MIN); + + worker + .block_on(async move { + let (mut dialed, mut accepted) = connect(&handle, &server, &client, &dial).await; + transfer(&handle, &mut dialed, &mut accepted, PAYLOAD).await; + transfer(&handle, &mut accepted, &mut dialed, PAYLOAD).await; + }) + .expect("worker"); + assert!(path.largest() <= IPV6_MIN); +} + +/// Without discovery, nothing either way exceeds the floor, however much the +/// path would carry. +#[test] +fn discovery_stays_off_by_default() { + let Some(mut worker) = worker() else { return }; + let handle = worker.handle(); + let (server, client, dial, path) = endpoints(&handle, &quic::Transport::default(), ETHERNET); + + worker + .block_on(async move { + let (mut dialed, mut accepted) = connect(&handle, &server, &client, &dial).await; + transfer(&handle, &mut dialed, &mut accepted, PAYLOAD).await; + transfer(&handle, &mut accepted, &mut dialed, PAYLOAD).await; + }) + .expect("worker"); + assert_eq!(path.largest(), FLOOR, "every full datagram sits at the floor"); +} + +/// Transfer until discovery has raised the MTU past the floor, so a train +/// staged after it packs the larger datagrams. +async fn discover(handle: &moq_uring::Handle, from: &mut quic::Connection, to: &mut quic::Connection, path: &Path) { + for _ in 0..50 { + transfer(handle, from, to, PAYLOAD / 16).await; + if path.large_up() > 0 { + return; + } + } + panic!("discovery never sent past the floor"); +} + +/// With discovery on, a path that carries more than the floor gets it, and +/// trains keep batching at the larger size. +#[test] +fn discovery_finds_a_larger_mtu() { + let Some(mut worker) = worker() else { return }; + let handle = worker.handle(); + let mut transport = quic::Transport::default(); + transport.mtu_discovery = true; + let (server, client, dial, path) = endpoints(&handle, &transport, ETHERNET); + + let metrics = handle.metrics(); + let (path, (datagrams, sends)) = worker + .block_on(async move { + let (mut dialed, mut accepted) = connect(&handle, &server, &client, &dial).await; + discover(&handle, &mut dialed, &mut accepted, &path).await; + + let before = (metrics.snapshot(), path.large_up()); + transfer(&handle, &mut dialed, &mut accepted, PAYLOAD).await; + let after = metrics.snapshot(); + // Most of the transfer rode above the floor. + let large = path.large_up() - before.1; + assert!(large as usize > PAYLOAD / ETHERNET / 2, "{large} large datagrams"); + ( + path, + ( + after.tx_datagrams - before.0.tx_datagrams, + after.tx_sends - before.0.tx_sends, + ), + ) + }) + .expect("worker"); + assert!(path.largest() > FLOOR, "discovered {}", path.largest()); + assert!(path.largest() <= ETHERNET, "discovered {}", path.largest()); + assert!(datagrams > 2 * sends, "{datagrams} datagrams in {sends} sends"); +} + +/// A path that shrinks after discovery black-holes the larger datagrams, and +/// the connection falls back to the floor instead of stalling. +#[test] +fn a_shrinking_path_falls_back_to_the_floor() { + let Some(mut worker) = worker() else { return }; + let handle = worker.handle(); + let mut transport = quic::Transport::default(); + transport.mtu_discovery = true; + let (server, client, dial, path) = endpoints(&handle, &transport, ETHERNET); + + worker + .block_on(async move { + let (mut dialed, mut accepted) = connect(&handle, &server, &client, &dial).await; + discover(&handle, &mut dialed, &mut accepted, &path).await; + + path.set_mtu(IPV6_MIN); + transfer(&handle, &mut dialed, &mut accepted, PAYLOAD).await; + transfer(&handle, &mut accepted, &mut dialed, PAYLOAD).await; + }) + .expect("worker"); +}