From 6d4704c4e005f54a462cf18eb90370d5661a3e30 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 02:00:54 -0800 Subject: [PATCH 1/5] chore(quest): claim quest/a1/dual-stack-bind.md Quest-Claim: 864491d953bb3bfa From 5910da3b26618ce3843e5ea639f1afc1f20e4128 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 02:57:03 -0800 Subject: [PATCH 2/5] fix(native): bind default clients by address family Lazily share IPv4 and IPv6-only endpoints across Quinn and Noq client clones, and reuse Quinn lifecycle handling from FFI. Resolve Quiche remotes before choosing its default socket family. Preserve custom endpoints and native Session ownership while rejecting connects after close. Co-Authored-By: GPT-6.1 Sol --- quest/a1/README.md | 1 - quest/a1/dual-stack-bind.md | 32 -------------------------------- quest/a1/native-client-config.md | 1 - quest/a1/rind/quinn.md | 1 - 4 files changed, 35 deletions(-) delete mode 100644 quest/a1/dual-stack-bind.md diff --git a/quest/a1/README.md b/quest/a1/README.md index 951db889..22c841fc 100644 --- a/quest/a1/README.md +++ b/quest/a1/README.md @@ -12,7 +12,6 @@ The Required list is ordered by priority. Independent children may proceed concu - [QMux send-rate filter](/quest/a1/qmux-rate-filter.md) - app-limited samples no longer lower the estimate, on main and 0.5 - [QMux send-rate filter on 0.5](/quest/a1/qmux-rate-filter-0.5.md) - backport for published moq-relay -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - native clients bind by the server's address family - [Router crypto features](/quest/a1/router-crypto-features.md) - interim aws-lc-rs/ring features on `web-transport` - [Rust crates on rind](/quest/a1/rind/README.md) - no C crypto outside Quiche - [Binding runtime coverage](/quest/a1/bindings/README.md) - independent language jobs diff --git a/quest/a1/dual-stack-bind.md b/quest/a1/dual-stack-bind.md deleted file mode 100644 index 02cc8104..00000000 --- a/quest/a1/dual-stack-bind.md +++ /dev/null @@ -1,32 +0,0 @@ -# [M] Bind native clients by the server's address family - -## Goal - -Native clients' default sockets match the server's address family, so a macOS dual-stack socket that shares an ephemeral port with another process's IPv4 socket can no longer swallow replies and hang `connect()` until the idle timeout. This covers the Quinn and Noq ClientBuilders, the FFI client and the Quiche ez client. Caller-supplied endpoints and sockets are unchanged. - -## Plan - -Decided with the maintainer on 2026-11-06 (planning interview): - -- Every native default, not only the router's Quinn path. A bind option alone would leave the default broken. -- Quinn and Noq `ClientBuilder` clients (and FFI, which builds its own Quinn endpoint) hold an IPv4 `0.0.0.0:0` endpoint and an IPv6-only `[::]:0` endpoint (`IPV6_V6ONLY`). Each is bound lazily on the first connect to that family and shared by later sessions and clones. Bind failures become a `ClientError` from `connect()`, replacing today's `unwrap()` in `build()`. Hosts with IPv6 disabled fail only IPv6 connects. An IPv4-mapped remote (`::ffff:a.b.c.d`) uses the IPv4 endpoint and its unmapped address. -- `close`, `wait_closed` and `Drop` (in FFI and elsewhere) cover every endpoint bound so far. A connect after `close` fails rather than binding a new endpoint, and so does one still in progress: the bind and the closed check happen under the same lock, so no endpoint is ever bound after `close`. -- `Client::new(endpoint, config)` keeps using the single endpoint it is given. -- The Quiche ez client already binds once per connect. Without `with_bind`, it binds after DNS resolution, using the chosen remote's family. -- No new public bind API. Keep it private until a consumer needs it. -- Regression tests assert the bound family per remote (IPv4 server: IPv4 socket; IPv6 server: v6-only socket) and an IPv4-mapped remote. They fail on today's dual-stack bind. The macOS collision itself is the reporter's repro and isn't run in CI. - -Decision prompts (✅ chosen): - -- Scope: ✅ all native defaults / Quinn router path only / bind option only / upstream, won't fix. -- Strategy: ✅ lazy per-family / eager both / endpoint per connect. -- Placement: ✅ a1 as one quest / a0 / split per backend. - -## Closes - -- [#455](https://github.com/moq-dev/web-transport/issues/455) - the implementation completes this report - -## Related - -- [Native client composition](/quest/a1/native-client-config.md) - edits the same Quinn client construction -- [Quinn on rind](/quest/a1/rind/quinn.md) - rebuilds the same builders' crypto diff --git a/quest/a1/native-client-config.md b/quest/a1/native-client-config.md index d68b8681..1e6e4f42 100644 --- a/quest/a1/native-client-config.md +++ b/quest/a1/native-client-config.md @@ -16,5 +16,4 @@ Do not add a pretend browser certificate-verification bypass, a browser panic or ## Related -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - edits the same Quinn client construction - [Quinn on rind](/quest/a1/rind/quinn.md) - replaces the crypto these builders configure diff --git a/quest/a1/rind/quinn.md b/quest/a1/rind/quinn.md index effa0be7..b3de5bf5 100644 --- a/quest/a1/rind/quinn.md +++ b/quest/a1/rind/quinn.md @@ -19,4 +19,3 @@ Decided with the maintainer on 2026-11-06 (planning interview): ## Related - [Native client composition](/quest/a1/native-client-config.md) - converts a configured Quinn client into the router -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - rebuilds the same builders' endpoints From 64a1ed03cb6cfbc9104b45243ebbf0cf48cc455b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 02:57:21 -0800 Subject: [PATCH 3/5] fix(native): share lazy family-matched client endpoints Use IPv4 and IPv6-only sockets for default Quinn, Noq and FFI clients, and bind Quiche defaults after resolution. Cover mapped IPv4, concurrent clone reuse, endpoint errors, custom bindings, close races, FFI teardown and native Session ownership. Co-Authored-By: GPT-6.1 Sol --- rs/web-transport-ffi/README.md | 6 + rs/web-transport-ffi/src/client.rs | 21 +- rs/web-transport-ffi/src/test.rs | 80 ++++++ rs/web-transport-noq/Cargo.toml | 1 + rs/web-transport-noq/README.md | 30 +-- rs/web-transport-noq/src/client.rs | 172 ++++++++++++- rs/web-transport-noq/src/client/tests.rs | 267 +++++++++++++++++++++ rs/web-transport-noq/src/error.rs | 3 + rs/web-transport-quiche/Cargo.toml | 1 + rs/web-transport-quiche/README.md | 5 +- rs/web-transport-quiche/src/ez/client.rs | 84 ++++++- rs/web-transport-quiche/tests/connect.rs | 48 +++- rs/web-transport-quinn/Cargo.toml | 1 + rs/web-transport-quinn/README.md | 31 +-- rs/web-transport-quinn/src/client.rs | 172 ++++++++++++- rs/web-transport-quinn/src/client/tests.rs | 238 ++++++++++++++++++ rs/web-transport-quinn/src/error.rs | 3 + 17 files changed, 1074 insertions(+), 89 deletions(-) create mode 100644 rs/web-transport-noq/src/client/tests.rs create mode 100644 rs/web-transport-quinn/src/client/tests.rs diff --git a/rs/web-transport-ffi/README.md b/rs/web-transport-ffi/README.md index 8c20901e..0059360c 100644 --- a/rs/web-transport-ffi/README.md +++ b/rs/web-transport-ffi/README.md @@ -4,6 +4,12 @@ Low-level UniFFI bindings for WebTransport over HTTP/3 (QUIC). Generates Python, For an ergonomic Python API (async context managers, legacy exception hierarchy), see [`web-transport-rs`](https://pypi.org/project/web-transport-rs/) which wraps this package. +Clients lazily share one socket per destination address family: IPv4 uses +`0.0.0.0:0`, IPv6 uses IPv6-only `[::]:0`, and IPv4-mapped IPv6 uses IPv4. +Socket setup errors are returned by `connect`. `close` and final client drop +close both sockets, including pending connections. Call `close` before awaiting +`wait_closed` to prevent new connections while both sockets drain. + ## Building ```bash diff --git a/rs/web-transport-ffi/src/client.rs b/rs/web-transport-ffi/src/client.rs index 6904bb9c..a6ff3b52 100644 --- a/rs/web-transport-ffi/src/client.rs +++ b/rs/web-transport-ffi/src/client.rs @@ -55,7 +55,6 @@ impl Default for ClientConfig { #[derive(uniffi::Object)] pub struct Client { inner: web_transport_quinn::Client, - endpoint: quinn::Endpoint, } #[uniffi::export] @@ -142,16 +141,8 @@ impl Client { let mut client_config = quinn::ClientConfig::new(Arc::new(quic_config)); client_config.transport_config(transport.into()); - let _guard = RUNTIME.enter(); - let endpoint = quinn::Endpoint::client("[::]:0".parse().unwrap()) - .or_else(|_| quinn::Endpoint::client("0.0.0.0:0".parse().unwrap())) - .map_err(|e| WebTransportError::Io(format!("failed to create endpoint: {e}")))?; - - let client = web_transport_quinn::Client::new(endpoint.clone(), client_config); - Ok(Arc::new(Self { - inner: client, - endpoint, + inner: web_transport_quinn::Client::from(client_config), })) } @@ -178,15 +169,15 @@ impl Client { let var_code = quinn::VarInt::from_u64(code) .map_err(|_| WebTransportError::invalid("code must be < 2^62"))?; let _guard = RUNTIME.enter(); - self.endpoint.close(var_code, reason.as_bytes()); + self.inner.close(var_code, reason.as_bytes()); Ok(()) } - /// Wait until all connections have shut down cleanly. + /// Wait for bound endpoints to drain; close first to prevent new connections. pub async fn wait_closed(&self) { - let endpoint = self.endpoint.clone(); + let client = self.inner.clone(); let handle = RUNTIME.spawn(async move { - endpoint.wait_idle().await; + client.wait_idle().await; }); let _ = handle.await; } @@ -195,7 +186,7 @@ impl Client { impl Drop for Client { fn drop(&mut self) { let _guard = RUNTIME.enter(); - self.endpoint.close(quinn::VarInt::from_u32(0), b""); + self.inner.close(quinn::VarInt::from_u32(0), b""); } } diff --git a/rs/web-transport-ffi/src/test.rs b/rs/web-transport-ffi/src/test.rs index 983fd5ec..4bae179f 100644 --- a/rs/web-transport-ffi/src/test.rs +++ b/rs/web-transport-ffi/src/test.rs @@ -101,3 +101,83 @@ fn sha256(cert_der: &[u8]) -> Vec { hasher.update(cert_der); hasher.finalize().to_vec() } + +async fn dual_family_sessions( + client: &Client, +) -> (Vec, Vec>) { + let (chain, key) = self_signed_cert(); + let mut sessions = Vec::new(); + let mut servers = Vec::new(); + for bind in ["127.0.0.1:0", "[::1]:0"] { + let server = Server::new(ServerConfig { + certificate_chain: chain.clone(), + private_key: key.clone(), + bind: bind.into(), + congestion_control: CongestionControl::Default, + max_idle_timeout_secs: Some(10.0), + keep_alive_interval_secs: None, + }) + .unwrap(); + let addr = server.local_addr(); + let remote = std::net::SocketAddr::new(addr.host.parse().unwrap(), addr.port); + let (session, peer) = tokio::join!(client.connect(format!("https://{remote}/")), async { + server.accept().await.unwrap().accept().await.unwrap() + },); + sessions.push((session.unwrap(), peer)); + servers.push(server); + } + (sessions, servers) +} + +type ServerSession = ( + std::sync::Arc, + std::sync::Arc, +); + +#[tokio::test] +async fn client_close_and_drop_cover_both_families() { + tokio::time::timeout(TIMEOUT, async { + for explicit_close in [true, false] { + let client = Client::new(ClientConfig { + no_cert_verification: true, + ..Default::default() + }) + .unwrap(); + let (sessions, servers) = dual_family_sessions(&client).await; + if explicit_close { + client.close(42, "all sockets".into()).unwrap(); + assert!(client + .connect("https://127.0.0.1:443/".into()) + .await + .is_err()); + assert!(client.connect("https://[::1]:443/".into()).await.is_err()); + client.wait_closed().await; + } + drop(client); + for (session, peer) in &sessions { + assert!(session.receive_datagram().await.is_err()); + assert!(peer.receive_datagram().await.is_err()); + } + for server in servers { + server.close(0, String::new()).unwrap(); + server.wait_closed().await; + } + } + }) + .await + .expect("client shutdown timed out"); +} + +#[tokio::test] +async fn closing_unused_client_prevents_first_bind() { + let client = Client::new(ClientConfig::default()).unwrap(); + client.close(0, String::new()).unwrap(); + client.wait_closed().await; + for url in [ + "https://127.0.0.1:443/", + "https://[::1]:443/", + "https://[::ffff:127.0.0.1]:443/", + ] { + assert!(client.connect(url.into()).await.is_err()); + } +} diff --git a/rs/web-transport-noq/Cargo.toml b/rs/web-transport-noq/Cargo.toml index 5bf7234a..8b1267d3 100644 --- a/rs/web-transport-noq/Cargo.toml +++ b/rs/web-transport-noq/Cargo.toml @@ -39,6 +39,7 @@ rustls = { version = "0.23", default-features = false, features = [ "std", ] } rustls-native-certs = "0.8" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ "io-util", diff --git a/rs/web-transport-noq/README.md b/rs/web-transport-noq/README.md index a6d7dc15..b7872991 100644 --- a/rs/web-transport-noq/README.md +++ b/rs/web-transport-noq/README.md @@ -28,28 +28,22 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-noq, first you need to create a [noq::Endpoint](https://docs.rs/noq/latest/noq/struct.Endpoint.html); see the documentation and examples for more information. -The only requirement is that the ALPN is set to `web_transport_noq::ALPN` (aka `h3`). -Afterwards, you use [web_transport_noq::accept](https://docs.rs/web-transport-noq/latest/web_transport_noq/fn.accept.html) (as a server) or [web_transport_noq::connect](https://docs.rs/web-transport-noq/latest/web_transport_noq/fn.connect.html) (as a client) to establish a WebTransport session. -This will take over the QUIC connection and perform the boring HTTP/3 handshake for you. +Use `ClientBuilder::new().with_system_roots()?` to build a reusable client, +then call `client.connect(url).await?` to establish a session. Clones share the +client's endpoints. The first connection to each address family binds an +IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations +use IPv4. Socket setup failures are returned by `connect`, so building a client +does not require IPv6 support or a running async runtime. -See the [examples](examples) or [moq-native](https://github.com/moq-dev/moq-rs/blob/main/moq-native/src/quic.rs) for a full setup. +For an existing `noq::ClientConfig`, `Client::new(endpoint, config)` keeps +the caller's endpoint and address handling. Set its ALPN to `h3`. -```rust - // Create a QUIC client. - let mut endpoint = noq::Endpoint::client("[::]:0".parse()?)?; - endpoint.set_default_client_config(/* ... */); +`client.close(code, reason)` closes all bound endpoints and prevents new +connections through any clone. Await `client.wait_idle()` after closing to +wait for every bound endpoint to drain. - // Connect to the given URL. - let session = web_transport_noq::connect(&client, Url::parse("https://localhost")?).await?; - - // Create a bidirectional stream. - let (mut send, mut recv) = session.open_bi().await?; - - // Send a message. - send.write(b"hello").await?; -``` +See the [examples](examples) for certificate setup and server construction. ## API The `web-transport-noq` API is almost identical to the Noq API, except that [Connection](https://docs.rs/noq/latest/noq/struct.Connection.html) is called [Session](https://docs.rs/web-transport-noq/latest/web_transport_noq/struct.Session.html). diff --git a/rs/web-transport-noq/src/client.rs b/rs/web-transport-noq/src/client.rs index c3dcee47..22a93707 100644 --- a/rs/web-transport-noq/src/client.rs +++ b/rs/web-transport-noq/src/client.rs @@ -1,5 +1,5 @@ use std::net::{IpAddr, SocketAddr}; -use std::sync::Arc; +use std::sync::{Arc, Mutex}; use crate::proto::ConnectRequest; #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] @@ -148,11 +148,7 @@ impl ClientBuilder { client_config.transport_config(transport.into()); - let client = noq::Endpoint::client("[::]:0".parse().unwrap()).unwrap(); - Ok(Client { - endpoint: client, - config: client_config, - }) + Ok(Client::from_config(client_config)) } } @@ -199,7 +195,7 @@ impl DangerousClientBuilder { /// A client for connecting to a WebTransport server. #[derive(Clone, Debug)] pub struct Client { - endpoint: noq::Endpoint, + endpoints: Arc>, config: noq::ClientConfig, } @@ -208,7 +204,13 @@ impl Client { /// /// The ALPN MUST be set to [ALPN]. pub fn new(endpoint: noq::Endpoint, config: noq::ClientConfig) -> Self { - Self { endpoint, config } + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Custom(endpoint), + closed: false, + })), + config, + } } /// Connect to the server. @@ -217,6 +219,9 @@ impl Client { request: impl Into, ) -> Result { let request = request.into(); + if self.endpoints.lock().unwrap().closed { + return Err(noq::ConnectError::EndpointStopping.into()); + } let port = request.url.port().unwrap_or(443); @@ -248,15 +253,157 @@ impl Client { // Connect to the server using the addr we just resolved. let conn = self - .endpoint - .connect_with(self.config.clone(), remote, &host)?; + .endpoints + .lock() + .unwrap() + .connect(self.config.clone(), remote, &host)?; let conn = conn.await?; // Connect with the connection we established. - Session::connect(conn, request).await + let session = Session::connect(conn, request).await?; + if self.endpoints.lock().unwrap().closed { + return Err(noq::ConnectionError::LocallyClosed.into()); + } + Ok(session) + } + + /// Close all bound endpoints and reject future connections from every clone. + pub fn close(&self, code: noq::VarInt, reason: &[u8]) { + let mut endpoints = self.endpoints.lock().unwrap(); + if endpoints.closed { + return; + } + endpoints.closed = true; + for endpoint in endpoints.binding.endpoints() { + endpoint.close(code, reason); + } + } + + /// Wait for currently bound endpoints to become idle; close first to prevent new connections. + pub async fn wait_idle(&self) { + let endpoints: Vec<_> = self + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .cloned() + .collect(); + futures::future::join_all(endpoints.iter().map(noq::Endpoint::wait_idle)).await; } } +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl Client { + /// Create a client that lazily binds one socket for each server address family. + /// + /// The config must negotiate [`ALPN`]; socket setup errors are returned by [`Self::connect`]. + fn from_config(config: noq::ClientConfig) -> Self { + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Default { + ipv4: None, + ipv6: None, + }, + closed: false, + })), + config, + } + } +} + +#[derive(Debug)] +struct ClientEndpoints { + binding: Binding, + closed: bool, +} + +#[derive(Debug)] +enum Binding { + Custom(noq::Endpoint), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Default { + ipv4: Option, + ipv6: Option, + }, +} + +impl Binding { + fn endpoints(&self) -> impl Iterator { + match self { + Self::Custom(endpoint) => [Some(endpoint), None], + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Self::Default { ipv4, ipv6 } => [ipv4.as_ref(), ipv6.as_ref()], + } + .into_iter() + .flatten() + } +} + +impl ClientEndpoints { + fn connect( + &mut self, + config: noq::ClientConfig, + remote: SocketAddr, + host: &str, + ) -> Result { + if self.closed { + return Err(noq::ConnectError::EndpointStopping.into()); + } + // Binding and starting the connection share close's lock, so close + // cannot miss a newly created endpoint or an in-progress connection. + let (endpoint, remote) = match &mut self.binding { + Binding::Custom(endpoint) => (endpoint, remote), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Binding::Default { ipv4, ipv6 } => { + let remote = match remote { + SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; + if slot.is_none() { + *slot = Some( + bind_endpoint(remote).map_err(|err| ClientError::IoError(Arc::new(err)))?, + ); + } + (slot.as_mut().unwrap(), remote) + } + }; + Ok(endpoint.connect_with(config, remote, host)?) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint(remote: SocketAddr) -> std::io::Result { + let bind: SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = bind_socket(bind)?; + let runtime = + noq::default_runtime().ok_or_else(|| std::io::Error::other("no async runtime found"))?; + noq::Endpoint::new(noq::EndpointConfig::default(), None, socket, runtime) +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_socket(bind: SocketAddr) -> std::io::Result { + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.bind(&bind.into())?; + Ok(socket.into()) +} + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] impl Default for Client { fn default() -> Self { @@ -376,3 +523,6 @@ impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification { self.0.signature_verification_algorithms.supported_schemes() } } + +#[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] +mod tests; diff --git a/rs/web-transport-noq/src/client/tests.rs b/rs/web-transport-noq/src/client/tests.rs new file mode 100644 index 00000000..edb03d29 --- /dev/null +++ b/rs/web-transport-noq/src/client/tests.rs @@ -0,0 +1,267 @@ +use super::*; +use noq as backend; + +fn client() -> Client { + #[cfg(feature = "aws-lc-rs")] + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); + #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] + let _ = rustls::crypto::ring::default_provider().install_default(); + ClientBuilder::new() + .dangerous() + .with_no_certificate_verification() + .unwrap() +} + +fn bound(client: &Client) -> Vec { + client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .map(|endpoint| endpoint.local_addr().unwrap()) + .collect() +} + +fn url(remote: SocketAddr) -> url::Url { + url::Url::parse(&format!("https://{remote}/")).unwrap() +} + +#[test] +fn construction_needs_no_runtime_or_socket() { + assert!(bound(&client()).is_empty()); +} + +#[test] +fn ipv6_socket_is_v6_only() { + let socket = bind_socket("[::]:0".parse().unwrap()).unwrap(); + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); +} + +#[tokio::test] +async fn default_ipv4_socket() { + let client = client(); + let connect = client.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + assert!(bound(&client)[0].is_ipv4()); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_prevents_binding_either_family() { + let client = client(); + let clone = client.clone(); + let pending = clone.connect(url("127.0.0.1:443".parse().unwrap())); + client.close(0u32.into(), b""); + assert!(pending.await.is_err()); + for host in ["127.0.0.1", "[::1]", "localhost", "[::ffff:127.0.0.1]"] { + assert!(clone + .connect(url::Url::parse(&format!("https://{host}:443/")).unwrap()) + .await + .is_err()); + } + assert!(bound(&client).is_empty()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_interrupts_connect_and_prevents_second_family() { + let client = client(); + let clone = client.clone(); + let connect = clone.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + let before = bound(&client); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + assert!(client + .connect(url("[::1]:443".parse().unwrap())) + .await + .is_err()); + assert_eq!(bound(&client), before); + clone.wait_idle().await; +} + +#[test] +fn endpoint_failure_is_a_connect_error() { + // Poll an IP-literal connect without a runtime. Construction succeeds, + // while the lazy endpoint reports its setup error instead of panicking. + let client = client(); + let error = futures::executor::block_on(client.connect(url("127.0.0.1:443".parse().unwrap()))) + .unwrap_err(); + assert!(matches!(error, ClientError::IoError(_))); + assert!(bound(&client).is_empty()); +} + +#[tokio::test] +async fn caller_endpoint_keeps_its_address_and_remote_handling() { + let endpoint = backend::Endpoint::client("127.0.0.1:0".parse().unwrap()).unwrap(); + let addr = endpoint.local_addr().unwrap(); + let client = Client::new(endpoint, client().config); + let error = client + .connect(url("[::ffff:127.0.0.1]:443".parse().unwrap())) + .await + .unwrap_err(); + assert!(matches!( + error, + ClientError::NoqError(backend::ConnectError::InvalidRemoteAddress(_)) + )); + assert_eq!(bound(&client), vec![addr]); + client.close(0u32.into(), b""); + client.wait_idle().await; +} + +fn server(addr: SocketAddr) -> crate::Server { + let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); + crate::ServerBuilder::new() + .with_addr(addr) + .with_certificate( + vec![cert.cert.der().clone()], + rustls::pki_types::PrivatePkcs8KeyDer::from(cert.signing_key.serialize_der()).into(), + ) + .unwrap() +} + +async fn pair( + client: &Client, + server: &mut crate::Server, + remote: SocketAddr, +) -> (Session, Session) { + tokio::time::timeout(std::time::Duration::from_secs(10), async { + let (client, server) = tokio::join!(client.connect(url(remote)), async { + server.accept().await.unwrap().ok().await.unwrap() + },); + (client.unwrap(), server) + }) + .await + .expect("loopback connection timed out") +} + +#[tokio::test] +async fn families_mapped_addresses_and_clones_share_endpoints() { + let client = client(); + let clone = client.clone(); + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + let addr4 = ipv4.local_addr().unwrap(); + let addr6 = ipv6.local_addr().unwrap(); + let (session4, peer4) = pair(&client, &mut ipv4, addr4).await; + let first = bound(&client); + assert_eq!(first.len(), 1); + assert!(first[0].is_ipv4()); + let (session6, peer6) = pair(&clone, &mut ipv6, addr6).await; + let both = bound(&client); + assert_eq!(both.len(), 2); + assert!(both[1].is_ipv6()); + assert_eq!(both[0], first[0]); + let mapped = SocketAddr::new( + std::net::Ipv4Addr::LOCALHOST.to_ipv6_mapped().into(), + addr4.port(), + ); + let (session_mapped, peer_mapped) = pair(&clone, &mut ipv4, mapped).await; + assert!(session_mapped + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + .is_ipv4()); + assert_eq!( + peer4 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap(), + peer_mapped + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + ); + assert_eq!(bound(&clone), both); + + client.close(42u32.into(), b"all endpoints"); + clone.close(43u32.into(), b"second close"); + for session in [&session4, &session6, &session_mapped] { + let error = backend::Connection::closed(session).await; + assert!( + matches!(error, backend::ConnectionError::LocallyClosed), + "{error:?}" + ); + } + for peer in [&peer4, &peer6, &peer_mapped] { + backend::Connection::closed(peer).await; + } + tokio::join!(client.wait_idle(), clone.wait_idle()); + assert!(client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .all(|endpoint| endpoint.open_connections() == 0)); + drop((session4, session6, session_mapped)); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn simultaneous_first_connections_share_one_socket() { + let client = client(); + let clone = client.clone(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (first, second, peers) = tokio::time::timeout(std::time::Duration::from_secs(10), async { + tokio::join!( + client.connect(url(remote)), + clone.connect(url(remote)), + async { + let first = server.accept().await.unwrap().ok().await.unwrap(); + let second = server.accept().await.unwrap().ok().await.unwrap(); + (first, second) + } + ) + }) + .await + .expect("concurrent loopback connections timed out"); + let first = first.unwrap(); + let second = second.unwrap(); + assert_eq!(bound(&client).len(), 1); + assert_eq!( + peers + .0 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap(), + peers + .1 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + ); + client.close(0u32.into(), b""); + client.wait_idle().await; + drop((first, second, peers)); +} + +#[tokio::test] +async fn dropping_native_client_keeps_returned_session_usable() { + let client = client(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, &mut server, remote).await; + drop(client); + session + .send_datagram(bytes::Bytes::from_static(b"still open")) + .unwrap(); + let received = tokio::time::timeout(std::time::Duration::from_secs(10), peer.read_datagram()) + .await + .expect("datagram after client drop timed out") + .unwrap(); + assert_eq!(received.as_ref(), b"still open"); + server.close(0u32.into(), b""); + backend::Connection::closed(&session).await; + server.wait_idle().await; +} diff --git a/rs/web-transport-noq/src/error.rs b/rs/web-transport-noq/src/error.rs index 804b88cc..0f60fe5b 100644 --- a/rs/web-transport-noq/src/error.rs +++ b/rs/web-transport-noq/src/error.rs @@ -7,6 +7,9 @@ use crate::{ConnectError, SettingsError}; /// An error returned when connecting to a WebTransport endpoint. #[derive(Error, Debug, Clone)] pub enum ClientError { + #[error("failed to bind client endpoint: {0}")] + IoError(Arc), + #[error("unexpected end of stream")] UnexpectedEnd, diff --git a/rs/web-transport-quiche/Cargo.toml b/rs/web-transport-quiche/Cargo.toml index 386922e5..3674b499 100644 --- a/rs/web-transport-quiche/Cargo.toml +++ b/rs/web-transport-quiche/Cargo.toml @@ -29,6 +29,7 @@ http = "1" kio = "0.6.2" rustls-pki-types = "1" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ diff --git a/rs/web-transport-quiche/README.md b/rs/web-transport-quiche/README.md index 871922ce..3964d0b7 100644 --- a/rs/web-transport-quiche/README.md +++ b/rs/web-transport-quiche/README.md @@ -39,7 +39,10 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-quiche, figure it out yourself lul. +The default client resolves the remote before binding an ephemeral socket of +the same address family. IPv6 sockets are IPv6-only; IPv4-mapped IPv6 remotes +use IPv4. Explicit `with_bind` and `with_socket` inputs retain their address +handling. ## Session lifetime diff --git a/rs/web-transport-quiche/src/ez/client.rs b/rs/web-transport-quiche/src/ez/client.rs index 5dceec52..cf193879 100644 --- a/rs/web-transport-quiche/src/ez/client.rs +++ b/rs/web-transport-quiche/src/ez/client.rs @@ -160,12 +160,6 @@ impl ClientBuilder { /// /// This takes ownership because the underlying quiche implementation doesn't support reusing the same socket. pub async fn connect(mut self, host: &str, port: u16) -> io::Result { - if self.socket.is_none() { - self = self.with_bind("[::]:0")?; - } - - let socket = self.socket.take().unwrap(); - let mut remotes = match tokio::net::lookup_host((host, port)).await { Ok(remotes) => remotes, Err(err) => { @@ -187,6 +181,7 @@ impl ClientBuilder { } }; + let (socket, remote) = self.socket(remote)?; socket.connect(remote).await?; // Enable the offloads the kernel supports before the socket is wrapped; @@ -255,6 +250,40 @@ impl ClientBuilder { driver, }) } + + fn socket( + &mut self, + remote: std::net::SocketAddr, + ) -> io::Result<(tokio::net::UdpSocket, std::net::SocketAddr)> { + if let Some(socket) = self.socket.take() { + return Ok((socket, remote)); + } + + let remote = match remote { + std::net::SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| std::net::SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let bind: std::net::SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.set_nonblocking(true)?; + socket.bind(&bind.into())?; + Ok((tokio::net::UdpSocket::from_std(socket.into())?, remote)) + } } /// A QUIC connection that is still completing the TLS handshake. @@ -277,3 +306,46 @@ impl Connecting { Ok(self.connection) } } + +#[cfg(test)] +mod tests { + use super::*; + use std::net::SocketAddr; + + #[tokio::test] + async fn default_socket_matches_each_remote_family() { + for remote in ["127.0.0.1:443", "[::1]:443", "[::ffff:127.0.0.1]:443"] { + let remote: SocketAddr = remote.parse().unwrap(); + let (socket, target) = ClientBuilder::new().socket(remote).unwrap(); + let local = socket.local_addr().unwrap(); + assert_eq!(local.is_ipv4(), target.is_ipv4()); + assert!(local.ip().is_unspecified()); + assert_ne!(local.port(), 0); + if target.is_ipv6() { + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); + } else { + assert_eq!(target, "127.0.0.1:443".parse().unwrap()); + } + } + } + + #[tokio::test] + async fn explicit_socket_and_bind_are_preserved() { + let remote = "[::ffff:127.0.0.1]:443".parse().unwrap(); + let socket = std::net::UdpSocket::bind("127.0.0.1:0").unwrap(); + let addr = socket.local_addr().unwrap(); + let (socket, target) = ClientBuilder::new() + .with_socket(socket) + .unwrap() + .socket(remote) + .unwrap(); + assert_eq!(socket.local_addr().unwrap(), addr); + assert_eq!(target, remote); + + let mut builder = ClientBuilder::new().with_bind("[::1]:0").unwrap(); + let addr = builder.socket.as_ref().unwrap().local_addr().unwrap(); + let (socket, target) = builder.socket(remote).unwrap(); + assert_eq!(socket.local_addr().unwrap(), addr); + assert_eq!(target, remote); + } +} diff --git a/rs/web-transport-quiche/tests/connect.rs b/rs/web-transport-quiche/tests/connect.rs index 6406980d..1033ae3c 100644 --- a/rs/web-transport-quiche/tests/connect.rs +++ b/rs/web-transport-quiche/tests/connect.rs @@ -60,6 +60,15 @@ async fn spawn_server(bind: SocketAddr) -> Result<(SocketAddr, tokio::task::Join Ok((addr, handle)) } +async fn stop_server(server: tokio::task::JoinHandle<()>) -> Result<()> { + server.abort(); + match server.await { + Ok(()) => Ok(()), + Err(err) if err.is_cancelled() => Ok(()), + Err(err) => Err(err.into()), + } +} + fn client() -> ClientBuilder { // The certificate is self-signed and the subject here is address handling, // which verify.rs already covers from the other side. @@ -79,7 +88,6 @@ async fn connect_ipv6_literal_url() -> Result<()> { let url = Url::parse(&format!("https://[::1]:{}/", addr.port()))?; let session = client() - .with_bind((Ipv6Addr::LOCALHOST, 0))? .connect(url) .await .context("an IPv6 literal URL should resolve to the address it names")? @@ -88,7 +96,7 @@ async fn connect_ipv6_literal_url() -> Result<()> { session.close(0, "bye"); session.closed().await; - server.abort(); + stop_server(server).await?; Ok(()) } @@ -101,15 +109,37 @@ async fn connect_ipv4_literal_url() -> Result<()> { let (addr, server) = spawn_server((Ipv4Addr::LOCALHOST, 0).into()).await?; let url = Url::parse(&format!("https://127.0.0.1:{}/", addr.port()))?; - let session = client() - .with_bind((Ipv4Addr::LOCALHOST, 0))? - .connect(url) - .await? - .established() - .await?; + let session = client().connect(url).await?.established().await?; session.close(0, "bye"); session.closed().await; - server.abort(); + stop_server(server).await?; + Ok(()) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn connect_ipv4_mapped_literal_url() -> Result<()> { + let (addr, server) = spawn_server((Ipv4Addr::LOCALHOST, 0).into()).await?; + let url = Url::parse(&format!("https://[::ffff:127.0.0.1]:{}/", addr.port()))?; + let session = client().connect(url).await?.established().await?; + session.close(0, "bye"); + session.closed().await; + stop_server(server).await?; + Ok(()) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn connect_hostname_uses_resolved_family() -> Result<()> { + use std::net::ToSocketAddrs; + let bind = ("localhost", 0) + .to_socket_addrs()? + .next() + .context("localhost did not resolve")?; + let (addr, server) = spawn_server(bind).await?; + let url = Url::parse(&format!("https://localhost:{}/", addr.port()))?; + let session = client().connect(url).await?.established().await?; + session.close(0, "bye"); + session.closed().await; + stop_server(server).await?; Ok(()) } diff --git a/rs/web-transport-quinn/Cargo.toml b/rs/web-transport-quinn/Cargo.toml index 7dc01064..4bcbb07b 100644 --- a/rs/web-transport-quinn/Cargo.toml +++ b/rs/web-transport-quinn/Cargo.toml @@ -39,6 +39,7 @@ rustls = { version = "0.23", default-features = false, features = [ "std", ] } rustls-native-certs = "0.8" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ diff --git a/rs/web-transport-quinn/README.md b/rs/web-transport-quinn/README.md index e17180b3..a04566b2 100644 --- a/rs/web-transport-quinn/README.md +++ b/rs/web-transport-quinn/README.md @@ -28,28 +28,23 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-quinn, first you need to create a [quinn::Endpoint](https://docs.rs/quinn/latest/quinn/struct.Endpoint.html); see the documentation and examples for more information. -The only requirement is that the ALPN is set to `web_transport_quinn::ALPN` (aka `h3`). -Afterwards, you use [web_transport_quinn::accept](https://docs.rs/web-transport-quinn/latest/web_transport_quinn/fn.accept.html) (as a server) or [web_transport_quinn::connect](https://docs.rs/web-transport-quinn/latest/web_transport_quinn/fn.connect.html) (as a client) to establish a WebTransport session. -This will take over the QUIC connection and perform the boring HTTP/3 handshake for you. +Use `ClientBuilder::new().with_system_roots()?` to build a reusable client, +then call `client.connect(url).await?` to establish a session. Clones share the +client's endpoints. The first connection to each address family binds an +IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations +use IPv4. Socket setup failures are returned by `connect`, so building a client +does not require IPv6 support or a running async runtime. -See the [examples](examples) or [moq-native](https://github.com/moq-dev/moq-rs/blob/main/moq-native/src/quic.rs) for a full setup. +For an existing `quinn::ClientConfig`, `Client::from(config)` uses the same +lazy sockets. Set its ALPN to `h3`. `Client::new(endpoint, config)` instead keeps +the caller's endpoint and address handling. -```rust - // Create a QUIC client. - let mut endpoint = quinn::Endpoint::client("[::]:0".parse()?)?; - endpoint.set_default_client_config(/* ... */); +`client.close(code, reason)` closes all bound endpoints and prevents new +connections through any clone. Await `client.wait_idle()` after closing to +wait for every bound endpoint to drain. - // Connect to the given URL. - let session = web_transport_quinn::connect(&client, Url::parse("https://localhost")?).await?; - - // Create a bidirectional stream. - let (mut send, mut recv) = session.open_bi().await?; - - // Send a message. - send.write(b"hello").await?; -``` +See the [examples](examples) for certificate setup and server construction. ## Async runtimes Each WebTransport session runs a background task that reads capsules from the CONNECT stream, and `Session::close` runs the graceful close in another. diff --git a/rs/web-transport-quinn/src/client.rs b/rs/web-transport-quinn/src/client.rs index 9edeef04..83453aa2 100644 --- a/rs/web-transport-quinn/src/client.rs +++ b/rs/web-transport-quinn/src/client.rs @@ -1,5 +1,5 @@ use std::net::{IpAddr, SocketAddr}; -use std::sync::Arc; +use std::sync::{Arc, Mutex}; use crate::proto::ConnectRequest; #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] @@ -162,11 +162,7 @@ impl ClientBuilder { let mut client_config = quinn::ClientConfig::new(Arc::new(client_config)); client_config.transport_config(transport_config(self.congestion_controller.as_ref())); - let client = quinn::Endpoint::client("[::]:0".parse().unwrap()).unwrap(); - Ok(Client { - endpoint: client, - config: client_config, - }) + Ok(Client::from(client_config)) } } @@ -213,7 +209,7 @@ impl DangerousClientBuilder { /// A client for connecting to a WebTransport server. #[derive(Clone, Debug)] pub struct Client { - endpoint: quinn::Endpoint, + endpoints: Arc>, config: quinn::ClientConfig, } @@ -222,7 +218,13 @@ impl Client { /// /// The ALPN MUST be set to [ALPN]. pub fn new(endpoint: quinn::Endpoint, config: quinn::ClientConfig) -> Self { - Self { endpoint, config } + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Custom(endpoint), + closed: false, + })), + config, + } } /// Connect to the server. @@ -231,6 +233,9 @@ impl Client { request: impl Into, ) -> Result { let request = request.into(); + if self.endpoints.lock().unwrap().closed { + return Err(quinn::ConnectError::EndpointStopping.into()); + } let port = request.url.port().unwrap_or(443); @@ -262,15 +267,157 @@ impl Client { // Connect to the server using the addr we just resolved. let conn = self - .endpoint - .connect_with(self.config.clone(), remote, &host)?; + .endpoints + .lock() + .unwrap() + .connect(self.config.clone(), remote, &host)?; let conn = conn.await?; // Connect with the connection we established. - Session::connect(conn, request).await + let session = Session::connect(conn, request).await?; + if self.endpoints.lock().unwrap().closed { + return Err(quinn::ConnectionError::LocallyClosed.into()); + } + Ok(session) + } + + /// Close all bound endpoints and reject future connections from every clone. + pub fn close(&self, code: quinn::VarInt, reason: &[u8]) { + let mut endpoints = self.endpoints.lock().unwrap(); + if endpoints.closed { + return; + } + endpoints.closed = true; + for endpoint in endpoints.binding.endpoints() { + endpoint.close(code, reason); + } + } + + /// Wait for currently bound endpoints to become idle; close first to prevent new connections. + pub async fn wait_idle(&self) { + let endpoints: Vec<_> = self + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .cloned() + .collect(); + futures::future::join_all(endpoints.iter().map(quinn::Endpoint::wait_idle)).await; + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl From for Client { + /// Create a client that lazily binds one socket for each server address family. + /// + /// The config must negotiate [`ALPN`]; socket setup errors are returned by [`Self::connect`]. + fn from(config: quinn::ClientConfig) -> Self { + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Default { + ipv4: None, + ipv6: None, + }, + closed: false, + })), + config, + } + } +} + +#[derive(Debug)] +struct ClientEndpoints { + binding: Binding, + closed: bool, +} + +#[derive(Debug)] +enum Binding { + Custom(quinn::Endpoint), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Default { + ipv4: Option, + ipv6: Option, + }, +} + +impl Binding { + fn endpoints(&self) -> impl Iterator { + match self { + Self::Custom(endpoint) => [Some(endpoint), None], + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Self::Default { ipv4, ipv6 } => [ipv4.as_ref(), ipv6.as_ref()], + } + .into_iter() + .flatten() + } +} + +impl ClientEndpoints { + fn connect( + &mut self, + config: quinn::ClientConfig, + remote: SocketAddr, + host: &str, + ) -> Result { + if self.closed { + return Err(quinn::ConnectError::EndpointStopping.into()); + } + // Binding and starting the connection share close's lock, so close + // cannot miss a newly created endpoint or an in-progress connection. + let (endpoint, remote) = match &mut self.binding { + Binding::Custom(endpoint) => (endpoint, remote), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Binding::Default { ipv4, ipv6 } => { + let remote = match remote { + SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; + if slot.is_none() { + *slot = Some( + bind_endpoint(remote).map_err(|err| ClientError::IoError(Arc::new(err)))?, + ); + } + (slot.as_mut().unwrap(), remote) + } + }; + Ok(endpoint.connect_with(config, remote, host)?) } } +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint(remote: SocketAddr) -> std::io::Result { + let bind: SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = bind_socket(bind)?; + let runtime = + quinn::default_runtime().ok_or_else(|| std::io::Error::other("no async runtime found"))?; + quinn::Endpoint::new(quinn::EndpointConfig::default(), None, socket, runtime) +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_socket(bind: SocketAddr) -> std::io::Result { + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.bind(&bind.into())?; + Ok(socket.into()) +} + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] impl Default for Client { fn default() -> Self { @@ -390,3 +537,6 @@ impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification { self.0.signature_verification_algorithms.supported_schemes() } } + +#[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] +mod tests; diff --git a/rs/web-transport-quinn/src/client/tests.rs b/rs/web-transport-quinn/src/client/tests.rs new file mode 100644 index 00000000..ee767976 --- /dev/null +++ b/rs/web-transport-quinn/src/client/tests.rs @@ -0,0 +1,238 @@ +use super::*; +use quinn as backend; + +fn client() -> Client { + #[cfg(feature = "aws-lc-rs")] + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); + #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] + let _ = rustls::crypto::ring::default_provider().install_default(); + ClientBuilder::new() + .dangerous() + .with_no_certificate_verification() + .unwrap() +} + +fn bound(client: &Client) -> Vec { + client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .map(|endpoint| endpoint.local_addr().unwrap()) + .collect() +} + +fn url(remote: SocketAddr) -> url::Url { + url::Url::parse(&format!("https://{remote}/")).unwrap() +} + +#[test] +fn construction_needs_no_runtime_or_socket() { + assert!(bound(&client()).is_empty()); +} + +#[test] +fn ipv6_socket_is_v6_only() { + let socket = bind_socket("[::]:0".parse().unwrap()).unwrap(); + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); +} + +#[tokio::test] +async fn default_ipv4_socket() { + let client = client(); + let connect = client.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + assert!(bound(&client)[0].is_ipv4()); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_prevents_binding_either_family() { + let client = client(); + let clone = client.clone(); + let pending = clone.connect(url("127.0.0.1:443".parse().unwrap())); + client.close(0u32.into(), b""); + assert!(pending.await.is_err()); + for host in ["127.0.0.1", "[::1]", "localhost", "[::ffff:127.0.0.1]"] { + assert!(clone + .connect(url::Url::parse(&format!("https://{host}:443/")).unwrap()) + .await + .is_err()); + } + assert!(bound(&client).is_empty()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_interrupts_connect_and_prevents_second_family() { + let client = client(); + let clone = client.clone(); + let connect = clone.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + let before = bound(&client); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + assert!(client + .connect(url("[::1]:443".parse().unwrap())) + .await + .is_err()); + assert_eq!(bound(&client), before); + clone.wait_idle().await; +} + +#[test] +fn endpoint_failure_is_a_connect_error() { + // Poll an IP-literal connect without a runtime. Construction succeeds, + // while the lazy endpoint reports its setup error instead of panicking. + let client = client(); + let error = futures::executor::block_on(client.connect(url("127.0.0.1:443".parse().unwrap()))) + .unwrap_err(); + assert!(matches!(error, ClientError::IoError(_))); + assert!(bound(&client).is_empty()); +} + +#[tokio::test] +async fn caller_endpoint_keeps_its_address_and_remote_handling() { + let endpoint = backend::Endpoint::client("127.0.0.1:0".parse().unwrap()).unwrap(); + let addr = endpoint.local_addr().unwrap(); + let client = Client::new(endpoint, client().config); + let error = client + .connect(url("[::ffff:127.0.0.1]:443".parse().unwrap())) + .await + .unwrap_err(); + assert!(matches!( + error, + ClientError::QuinnError(backend::ConnectError::InvalidRemoteAddress(_)) + )); + assert_eq!(bound(&client), vec![addr]); + client.close(0u32.into(), b""); + client.wait_idle().await; +} + +fn server(addr: SocketAddr) -> crate::Server { + let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); + crate::ServerBuilder::new() + .with_addr(addr) + .with_certificate( + vec![cert.cert.der().clone()], + rustls::pki_types::PrivatePkcs8KeyDer::from(cert.signing_key.serialize_der()).into(), + ) + .unwrap() +} + +async fn pair( + client: &Client, + server: &mut crate::Server, + remote: SocketAddr, +) -> (Session, Session) { + tokio::time::timeout(std::time::Duration::from_secs(10), async { + let (client, server) = tokio::join!(client.connect(url(remote)), async { + server.accept().await.unwrap().ok().await.unwrap() + },); + (client.unwrap(), server) + }) + .await + .expect("loopback connection timed out") +} + +#[tokio::test] +async fn families_mapped_addresses_and_clones_share_endpoints() { + let client = client(); + let clone = client.clone(); + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + let addr4 = ipv4.local_addr().unwrap(); + let addr6 = ipv6.local_addr().unwrap(); + let (session4, peer4) = pair(&client, &mut ipv4, addr4).await; + let first = bound(&client); + assert_eq!(first.len(), 1); + assert!(first[0].is_ipv4()); + let (session6, peer6) = pair(&clone, &mut ipv6, addr6).await; + let both = bound(&client); + assert_eq!(both.len(), 2); + assert!(both[1].is_ipv6()); + assert_eq!(both[0], first[0]); + let mapped = SocketAddr::new( + std::net::Ipv4Addr::LOCALHOST.to_ipv6_mapped().into(), + addr4.port(), + ); + let (session_mapped, peer_mapped) = pair(&clone, &mut ipv4, mapped).await; + assert!(session_mapped.remote_address().is_ipv4()); + assert_eq!(peer4.remote_address(), peer_mapped.remote_address()); + assert_eq!(bound(&clone), both); + + client.close(42u32.into(), b"all endpoints"); + clone.close(43u32.into(), b"second close"); + for session in [&session4, &session6, &session_mapped] { + let error = backend::Connection::closed(session).await; + assert!( + matches!(error, backend::ConnectionError::LocallyClosed), + "{error:?}" + ); + } + for peer in [&peer4, &peer6, &peer_mapped] { + backend::Connection::closed(peer).await; + } + tokio::join!(client.wait_idle(), clone.wait_idle()); + assert!(client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .all(|endpoint| endpoint.open_connections() == 0)); + drop((session4, session6, session_mapped)); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn simultaneous_first_connections_share_one_socket() { + let client = client(); + let clone = client.clone(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (first, second, peers) = tokio::time::timeout(std::time::Duration::from_secs(10), async { + tokio::join!( + client.connect(url(remote)), + clone.connect(url(remote)), + async { + let first = server.accept().await.unwrap().ok().await.unwrap(); + let second = server.accept().await.unwrap().ok().await.unwrap(); + (first, second) + } + ) + }) + .await + .expect("concurrent loopback connections timed out"); + let first = first.unwrap(); + let second = second.unwrap(); + assert_eq!(bound(&client).len(), 1); + assert_eq!(peers.0.remote_address(), peers.1.remote_address()); + client.close(0u32.into(), b""); + client.wait_idle().await; + drop((first, second, peers)); +} + +#[tokio::test] +async fn dropping_native_client_keeps_returned_session_usable() { + let client = client(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, &mut server, remote).await; + drop(client); + session + .send_datagram(bytes::Bytes::from_static(b"still open")) + .unwrap(); + let received = tokio::time::timeout(std::time::Duration::from_secs(10), peer.read_datagram()) + .await + .expect("datagram after client drop timed out") + .unwrap(); + assert_eq!(received.as_ref(), b"still open"); + server.close(0u32.into(), b""); + backend::Connection::closed(&session).await; + server.wait_idle().await; +} diff --git a/rs/web-transport-quinn/src/error.rs b/rs/web-transport-quinn/src/error.rs index 11880256..a1175948 100644 --- a/rs/web-transport-quinn/src/error.rs +++ b/rs/web-transport-quinn/src/error.rs @@ -7,6 +7,9 @@ use crate::{ConnectError, SettingsError}; /// An error returned when connecting to a WebTransport endpoint. #[derive(Error, Debug, Clone)] pub enum ClientError { + #[error("failed to bind client endpoint: {0}")] + IoError(Arc), + #[error("unexpected end of stream")] UnexpectedEnd, From f75e76770e0905304562e1828edc3a3eb53e1a84 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 03:46:57 -0800 Subject: [PATCH 4/5] fix(native): preserve lazy endpoint runtime ownership Capture one runtime for both family sockets and retain the Tokio context for lazy endpoint registration. Keep Noq lifecycle helpers private to tests until a consumer needs them. Co-Authored-By: GPT-6.1 Sol --- rs/web-transport-ffi/src/client.rs | 1 + rs/web-transport-noq/README.md | 8 +- rs/web-transport-noq/src/client.rs | 119 +++++++++----------- rs/web-transport-noq/src/client/tests.rs | 125 +++++++++++++-------- rs/web-transport-quinn/README.md | 4 +- rs/web-transport-quinn/src/client.rs | 52 ++++++++- rs/web-transport-quinn/src/client/tests.rs | 54 ++++++++- 7 files changed, 239 insertions(+), 124 deletions(-) diff --git a/rs/web-transport-ffi/src/client.rs b/rs/web-transport-ffi/src/client.rs index a6ff3b52..408e8859 100644 --- a/rs/web-transport-ffi/src/client.rs +++ b/rs/web-transport-ffi/src/client.rs @@ -141,6 +141,7 @@ impl Client { let mut client_config = quinn::ClientConfig::new(Arc::new(quic_config)); client_config.transport_config(transport.into()); + let _guard = RUNTIME.enter(); Ok(Arc::new(Self { inner: web_transport_quinn::Client::from(client_config), })) diff --git a/rs/web-transport-noq/README.md b/rs/web-transport-noq/README.md index b7872991..a4a6c149 100644 --- a/rs/web-transport-noq/README.md +++ b/rs/web-transport-noq/README.md @@ -34,15 +34,13 @@ then call `client.connect(url).await?` to establish a session. Clones share the client's endpoints. The first connection to each address family binds an IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations use IPv4. Socket setup failures are returned by `connect`, so building a client -does not require IPv6 support or a running async runtime. +does not require IPv6 support or a running async runtime. Both sockets use the +runtime present at construction, or at the first connection if constructed +outside a runtime. Keep that runtime alive while using the client. For an existing `noq::ClientConfig`, `Client::new(endpoint, config)` keeps the caller's endpoint and address handling. Set its ALPN to `h3`. -`client.close(code, reason)` closes all bound endpoints and prevents new -connections through any clone. Await `client.wait_idle()` after closing to -wait for every bound endpoint to drain. - See the [examples](examples) for certificate setup and server construction. ## API diff --git a/rs/web-transport-noq/src/client.rs b/rs/web-transport-noq/src/client.rs index 22a93707..c48d71fa 100644 --- a/rs/web-transport-noq/src/client.rs +++ b/rs/web-transport-noq/src/client.rs @@ -195,7 +195,7 @@ impl DangerousClientBuilder { /// A client for connecting to a WebTransport server. #[derive(Clone, Debug)] pub struct Client { - endpoints: Arc>, + endpoints: Arc>, config: noq::ClientConfig, } @@ -205,10 +205,7 @@ impl Client { /// The ALPN MUST be set to [ALPN]. pub fn new(endpoint: noq::Endpoint, config: noq::ClientConfig) -> Self { Self { - endpoints: Arc::new(Mutex::new(ClientEndpoints { - binding: Binding::Custom(endpoint), - closed: false, - })), + endpoints: Arc::new(Mutex::new(Binding::Custom(endpoint))), config, } } @@ -219,10 +216,6 @@ impl Client { request: impl Into, ) -> Result { let request = request.into(); - if self.endpoints.lock().unwrap().closed { - return Err(noq::ConnectError::EndpointStopping.into()); - } - let port = request.url.port().unwrap_or(443); // TODO error on username:password in host @@ -260,36 +253,7 @@ impl Client { let conn = conn.await?; // Connect with the connection we established. - let session = Session::connect(conn, request).await?; - if self.endpoints.lock().unwrap().closed { - return Err(noq::ConnectionError::LocallyClosed.into()); - } - Ok(session) - } - - /// Close all bound endpoints and reject future connections from every clone. - pub fn close(&self, code: noq::VarInt, reason: &[u8]) { - let mut endpoints = self.endpoints.lock().unwrap(); - if endpoints.closed { - return; - } - endpoints.closed = true; - for endpoint in endpoints.binding.endpoints() { - endpoint.close(code, reason); - } - } - - /// Wait for currently bound endpoints to become idle; close first to prevent new connections. - pub async fn wait_idle(&self) { - let endpoints: Vec<_> = self - .endpoints - .lock() - .unwrap() - .binding - .endpoints() - .cloned() - .collect(); - futures::future::join_all(endpoints.iter().map(noq::Endpoint::wait_idle)).await; + Session::connect(conn, request).await } } @@ -300,24 +264,16 @@ impl Client { /// The config must negotiate [`ALPN`]; socket setup errors are returned by [`Self::connect`]. fn from_config(config: noq::ClientConfig) -> Self { Self { - endpoints: Arc::new(Mutex::new(ClientEndpoints { - binding: Binding::Default { - ipv4: None, - ipv6: None, - }, - closed: false, + endpoints: Arc::new(Mutex::new(Binding::Default { + ipv4: None, + ipv6: None, + runtime: ClientRuntime::current(), })), config, } } } -#[derive(Debug)] -struct ClientEndpoints { - binding: Binding, - closed: bool, -} - #[derive(Debug)] enum Binding { Custom(noq::Endpoint), @@ -325,37 +281,38 @@ enum Binding { Default { ipv4: Option, ipv6: Option, + runtime: Option, }, } +#[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] impl Binding { fn endpoints(&self) -> impl Iterator { match self { Self::Custom(endpoint) => [Some(endpoint), None], #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] - Self::Default { ipv4, ipv6 } => [ipv4.as_ref(), ipv6.as_ref()], + Self::Default { ipv4, ipv6, .. } => [ipv4.as_ref(), ipv6.as_ref()], } .into_iter() .flatten() } } -impl ClientEndpoints { +impl Binding { fn connect( &mut self, config: noq::ClientConfig, remote: SocketAddr, host: &str, ) -> Result { - if self.closed { - return Err(noq::ConnectError::EndpointStopping.into()); - } - // Binding and starting the connection share close's lock, so close - // cannot miss a newly created endpoint or an in-progress connection. - let (endpoint, remote) = match &mut self.binding { + let (endpoint, remote) = match self { Binding::Custom(endpoint) => (endpoint, remote), #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] - Binding::Default { ipv4, ipv6 } => { + Binding::Default { + ipv4, + ipv6, + runtime, + } => { let remote = match remote { SocketAddr::V6(addr) => addr .ip() @@ -366,8 +323,18 @@ impl ClientEndpoints { }; let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; if slot.is_none() { + if runtime.is_none() { + *runtime = ClientRuntime::current(); + } + let runtime = runtime.as_ref().ok_or_else(|| { + ClientError::IoError(Arc::new(std::io::Error::other( + "no async runtime found", + ))) + })?; *slot = Some( - bind_endpoint(remote).map_err(|err| ClientError::IoError(Arc::new(err)))?, + runtime + .bind(remote) + .map_err(|err| ClientError::IoError(Arc::new(err)))?, ); } (slot.as_mut().unwrap(), remote) @@ -378,15 +345,39 @@ impl ClientEndpoints { } #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] -fn bind_endpoint(remote: SocketAddr) -> std::io::Result { +#[derive(Debug)] +struct ClientRuntime { + inner: Arc, + tokio: Option, +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl ClientRuntime { + fn current() -> Option { + Some(Self { + inner: noq::default_runtime()?, + tokio: tokio::runtime::Handle::try_current().ok(), + }) + } + + fn bind(&self, remote: SocketAddr) -> std::io::Result { + // TokioRuntime registers sockets and spawns drivers in the current context. + let _guard = self.tokio.as_ref().map(tokio::runtime::Handle::enter); + bind_endpoint(remote, self.inner.clone()) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint( + remote: SocketAddr, + runtime: Arc, +) -> std::io::Result { let bind: SocketAddr = if remote.is_ipv4() { (std::net::Ipv4Addr::UNSPECIFIED, 0).into() } else { (std::net::Ipv6Addr::UNSPECIFIED, 0).into() }; let socket = bind_socket(bind)?; - let runtime = - noq::default_runtime().ok_or_else(|| std::io::Error::other("no async runtime found"))?; noq::Endpoint::new(noq::EndpointConfig::default(), None, socket, runtime) } diff --git a/rs/web-transport-noq/src/client/tests.rs b/rs/web-transport-noq/src/client/tests.rs index edb03d29..e8397dc0 100644 --- a/rs/web-transport-noq/src/client/tests.rs +++ b/rs/web-transport-noq/src/client/tests.rs @@ -17,12 +17,28 @@ fn bound(client: &Client) -> Vec { .endpoints .lock() .unwrap() - .binding .endpoints() .map(|endpoint| endpoint.local_addr().unwrap()) .collect() } +fn close(client: &Client, code: backend::VarInt, reason: &[u8]) { + for endpoint in client.endpoints.lock().unwrap().endpoints() { + endpoint.close(code, reason); + } +} + +async fn wait_idle(client: &Client) { + let endpoints: Vec<_> = client + .endpoints + .lock() + .unwrap() + .endpoints() + .cloned() + .collect(); + futures::future::join_all(endpoints.iter().map(backend::Endpoint::wait_idle)).await; +} + fn url(remote: SocketAddr) -> url::Url { url::Url::parse(&format!("https://{remote}/")).unwrap() } @@ -45,48 +61,13 @@ async fn default_ipv4_socket() { tokio::pin!(connect); assert!(futures::poll!(&mut connect).is_pending()); assert!(bound(&client)[0].is_ipv4()); - client.close(0u32.into(), b""); - assert!(connect.await.is_err()); - client.wait_idle().await; -} - -#[tokio::test] -async fn close_prevents_binding_either_family() { - let client = client(); - let clone = client.clone(); - let pending = clone.connect(url("127.0.0.1:443".parse().unwrap())); - client.close(0u32.into(), b""); - assert!(pending.await.is_err()); - for host in ["127.0.0.1", "[::1]", "localhost", "[::ffff:127.0.0.1]"] { - assert!(clone - .connect(url::Url::parse(&format!("https://{host}:443/")).unwrap()) - .await - .is_err()); - } - assert!(bound(&client).is_empty()); - client.wait_idle().await; -} - -#[tokio::test] -async fn close_interrupts_connect_and_prevents_second_family() { - let client = client(); - let clone = client.clone(); - let connect = clone.connect(url("127.0.0.1:443".parse().unwrap())); - tokio::pin!(connect); - assert!(futures::poll!(&mut connect).is_pending()); - let before = bound(&client); - client.close(0u32.into(), b""); + close(&client, 0u32.into(), b""); assert!(connect.await.is_err()); - assert!(client - .connect(url("[::1]:443".parse().unwrap())) - .await - .is_err()); - assert_eq!(bound(&client), before); - clone.wait_idle().await; + wait_idle(&client).await; } #[test] -fn endpoint_failure_is_a_connect_error() { +fn missing_runtime_is_a_connect_error_and_allows_later_connections() { // Poll an IP-literal connect without a runtime. Construction succeeds, // while the lazy endpoint reports its setup error instead of panicking. let client = client(); @@ -94,6 +75,17 @@ fn endpoint_failure_is_a_connect_error() { .unwrap_err(); assert!(matches!(error, ClientError::IoError(_))); assert!(bound(&client).is_empty()); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + runtime.block_on(async { + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (_session, _peer) = pair(&client, &mut server, remote).await; + close(&client, 0u32.into(), b""); + wait_idle(&client).await; + }); } #[tokio::test] @@ -110,8 +102,8 @@ async fn caller_endpoint_keeps_its_address_and_remote_handling() { ClientError::NoqError(backend::ConnectError::InvalidRemoteAddress(_)) )); assert_eq!(bound(&client), vec![addr]); - client.close(0u32.into(), b""); - client.wait_idle().await; + close(&client, 0u32.into(), b""); + wait_idle(&client).await; } fn server(addr: SocketAddr) -> crate::Server { @@ -182,8 +174,7 @@ async fn families_mapped_addresses_and_clones_share_endpoints() { ); assert_eq!(bound(&clone), both); - client.close(42u32.into(), b"all endpoints"); - clone.close(43u32.into(), b"second close"); + close(&client, 42u32.into(), b"all endpoints"); for session in [&session4, &session6, &session_mapped] { let error = backend::Connection::closed(session).await; assert!( @@ -194,12 +185,11 @@ async fn families_mapped_addresses_and_clones_share_endpoints() { for peer in [&peer4, &peer6, &peer_mapped] { backend::Connection::closed(peer).await; } - tokio::join!(client.wait_idle(), clone.wait_idle()); + wait_idle(&client).await; assert!(client .endpoints .lock() .unwrap() - .binding .endpoints() .all(|endpoint| endpoint.open_connections() == 0)); drop((session4, session6, session_mapped)); @@ -241,8 +231,8 @@ async fn simultaneous_first_connections_share_one_socket() { .remote_address() .unwrap() ); - client.close(0u32.into(), b""); - client.wait_idle().await; + close(&client, 0u32.into(), b""); + wait_idle(&client).await; drop((first, second, peers)); } @@ -265,3 +255,44 @@ async fn dropping_native_client_keeps_returned_session_usable() { backend::Connection::closed(&session).await; server.wait_idle().await; } + +#[test] +fn construction_runtime_survives_first_connect_on_transient_runtime() { + let owner = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let client = { + let _guard = owner.enter(); + client() + }; + let transient = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + transient.block_on(async { + for addr in ["127.0.0.1:443", "[::1]:443"] { + let connect = client.connect(url(addr.parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + } + }); + drop(transient); + owner.block_on(async { + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + for server in [&mut ipv4, &mut ipv6] { + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, server, remote).await; + session + .send_datagram(bytes::Bytes::from_static(b"owner runtime")) + .unwrap(); + assert_eq!( + peer.read_datagram().await.unwrap().as_ref(), + b"owner runtime" + ); + } + close(&client, 0u32.into(), b""); + wait_idle(&client).await; + }); +} diff --git a/rs/web-transport-quinn/README.md b/rs/web-transport-quinn/README.md index a04566b2..e9044724 100644 --- a/rs/web-transport-quinn/README.md +++ b/rs/web-transport-quinn/README.md @@ -34,7 +34,9 @@ then call `client.connect(url).await?` to establish a session. Clones share the client's endpoints. The first connection to each address family binds an IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations use IPv4. Socket setup failures are returned by `connect`, so building a client -does not require IPv6 support or a running async runtime. +does not require IPv6 support or a running async runtime. Both sockets use the +runtime present at construction, or at the first connection if constructed +outside a runtime. Keep that runtime alive while using the client. For an existing `quinn::ClientConfig`, `Client::from(config)` uses the same lazy sockets. Set its ALPN to `h3`. `Client::new(endpoint, config)` instead keeps diff --git a/rs/web-transport-quinn/src/client.rs b/rs/web-transport-quinn/src/client.rs index 83453aa2..e659ee24 100644 --- a/rs/web-transport-quinn/src/client.rs +++ b/rs/web-transport-quinn/src/client.rs @@ -318,6 +318,7 @@ impl From for Client { binding: Binding::Default { ipv4: None, ipv6: None, + runtime: ClientRuntime::current(), }, closed: false, })), @@ -339,6 +340,7 @@ enum Binding { Default { ipv4: Option, ipv6: Option, + runtime: Option, }, } @@ -347,7 +349,7 @@ impl Binding { match self { Self::Custom(endpoint) => [Some(endpoint), None], #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] - Self::Default { ipv4, ipv6 } => [ipv4.as_ref(), ipv6.as_ref()], + Self::Default { ipv4, ipv6, .. } => [ipv4.as_ref(), ipv6.as_ref()], } .into_iter() .flatten() @@ -369,7 +371,11 @@ impl ClientEndpoints { let (endpoint, remote) = match &mut self.binding { Binding::Custom(endpoint) => (endpoint, remote), #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] - Binding::Default { ipv4, ipv6 } => { + Binding::Default { + ipv4, + ipv6, + runtime, + } => { let remote = match remote { SocketAddr::V6(addr) => addr .ip() @@ -380,8 +386,18 @@ impl ClientEndpoints { }; let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; if slot.is_none() { + if runtime.is_none() { + *runtime = ClientRuntime::current(); + } + let runtime = runtime.as_ref().ok_or_else(|| { + ClientError::IoError(Arc::new(std::io::Error::other( + "no async runtime found", + ))) + })?; *slot = Some( - bind_endpoint(remote).map_err(|err| ClientError::IoError(Arc::new(err)))?, + runtime + .bind(remote) + .map_err(|err| ClientError::IoError(Arc::new(err)))?, ); } (slot.as_mut().unwrap(), remote) @@ -392,15 +408,39 @@ impl ClientEndpoints { } #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] -fn bind_endpoint(remote: SocketAddr) -> std::io::Result { +#[derive(Debug)] +struct ClientRuntime { + inner: Arc, + tokio: Option, +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl ClientRuntime { + fn current() -> Option { + Some(Self { + inner: quinn::default_runtime()?, + tokio: tokio::runtime::Handle::try_current().ok(), + }) + } + + fn bind(&self, remote: SocketAddr) -> std::io::Result { + // TokioRuntime registers sockets and spawns drivers in the current context. + let _guard = self.tokio.as_ref().map(tokio::runtime::Handle::enter); + bind_endpoint(remote, self.inner.clone()) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint( + remote: SocketAddr, + runtime: Arc, +) -> std::io::Result { let bind: SocketAddr = if remote.is_ipv4() { (std::net::Ipv4Addr::UNSPECIFIED, 0).into() } else { (std::net::Ipv6Addr::UNSPECIFIED, 0).into() }; let socket = bind_socket(bind)?; - let runtime = - quinn::default_runtime().ok_or_else(|| std::io::Error::other("no async runtime found"))?; quinn::Endpoint::new(quinn::EndpointConfig::default(), None, socket, runtime) } diff --git a/rs/web-transport-quinn/src/client/tests.rs b/rs/web-transport-quinn/src/client/tests.rs index ee767976..a7081caa 100644 --- a/rs/web-transport-quinn/src/client/tests.rs +++ b/rs/web-transport-quinn/src/client/tests.rs @@ -86,7 +86,7 @@ async fn close_interrupts_connect_and_prevents_second_family() { } #[test] -fn endpoint_failure_is_a_connect_error() { +fn missing_runtime_is_a_connect_error_and_allows_later_connections() { // Poll an IP-literal connect without a runtime. Construction succeeds, // while the lazy endpoint reports its setup error instead of panicking. let client = client(); @@ -94,6 +94,17 @@ fn endpoint_failure_is_a_connect_error() { .unwrap_err(); assert!(matches!(error, ClientError::IoError(_))); assert!(bound(&client).is_empty()); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + runtime.block_on(async { + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (_session, _peer) = pair(&client, &mut server, remote).await; + client.close(0u32.into(), b""); + client.wait_idle().await; + }); } #[tokio::test] @@ -236,3 +247,44 @@ async fn dropping_native_client_keeps_returned_session_usable() { backend::Connection::closed(&session).await; server.wait_idle().await; } + +#[test] +fn construction_runtime_survives_first_connect_on_transient_runtime() { + let owner = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let client = { + let _guard = owner.enter(); + client() + }; + let transient = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + transient.block_on(async { + for addr in ["127.0.0.1:443", "[::1]:443"] { + let connect = client.connect(url(addr.parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + } + }); + drop(transient); + owner.block_on(async { + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + for server in [&mut ipv4, &mut ipv6] { + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, server, remote).await; + session + .send_datagram(bytes::Bytes::from_static(b"owner runtime")) + .unwrap(); + assert_eq!( + peer.read_datagram().await.unwrap().as_ref(), + b"owner runtime" + ); + } + client.close(0u32.into(), b""); + client.wait_idle().await; + }); +} From 391c93ccd859009f14c48e79bd6122c21a9cd2a9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 8 Nov 2026 10:18:26 -0800 Subject: [PATCH 5/5] test(quinn): build client-test servers with an explicit crypto provider The client tests' server() helper used ServerBuilder::new(), which panics under --all-features unless another test already installed a process-wide provider. Reuse the server tests' builder(), which picks one explicitly. Co-Authored-By: Claude Opus 5.5 --- rs/web-transport-quinn/src/client/tests.rs | 2 +- rs/web-transport-quinn/src/server.rs | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/rs/web-transport-quinn/src/client/tests.rs b/rs/web-transport-quinn/src/client/tests.rs index 24d4490b..169c9e6c 100644 --- a/rs/web-transport-quinn/src/client/tests.rs +++ b/rs/web-transport-quinn/src/client/tests.rs @@ -254,7 +254,7 @@ async fn caller_endpoint_keeps_its_address_and_remote_handling() { fn server(addr: SocketAddr) -> crate::Server { let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); - crate::ServerBuilder::new() + crate::server::tests::builder() .with_addr(addr) .with_certificate( vec![cert.cert.der().clone()], diff --git a/rs/web-transport-quinn/src/server.rs b/rs/web-transport-quinn/src/server.rs index 2decf9ed..d766aa4c 100644 --- a/rs/web-transport-quinn/src/server.rs +++ b/rs/web-transport-quinn/src/server.rs @@ -253,7 +253,7 @@ impl core::ops::Deref for Request { } #[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] -mod tests { +pub(crate) mod tests { use super::*; use crate::client::tests::{assert_transport, custom_transport}; use rustls::pki_types::PrivatePkcs8KeyDer; @@ -269,7 +269,7 @@ mod tests { /// `ServerBuilder::new` goes through [crypto::default_provider], which panics when /// both backends are compiled in and no process-wide default is installed. Pick one /// here rather than mutating global state from a test. - fn builder() -> ServerBuilder { + pub(crate) fn builder() -> ServerBuilder { #[cfg(feature = "aws-lc-rs")] let provider = Arc::new(rustls::crypto::aws_lc_rs::default_provider()); #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))]