diff --git a/quest/a1/README.md b/quest/a1/README.md index 21e8bc71..fc54c06c 100644 --- a/quest/a1/README.md +++ b/quest/a1/README.md @@ -10,7 +10,6 @@ The Required list is ordered by priority. Independent children may proceed concu ## Required -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - native clients bind by the server's address family - [Rust crates on rind](/quest/a1/rind/README.md) - no C crypto outside Quiche - [Binding runtime coverage](/quest/a1/bindings/README.md) - independent language jobs - [Node idle configuration](/quest/a1/node-idle/README.md) - typed native input and Node options diff --git a/quest/a1/dual-stack-bind.md b/quest/a1/dual-stack-bind.md deleted file mode 100644 index 02cc8104..00000000 --- a/quest/a1/dual-stack-bind.md +++ /dev/null @@ -1,32 +0,0 @@ -# [M] Bind native clients by the server's address family - -## Goal - -Native clients' default sockets match the server's address family, so a macOS dual-stack socket that shares an ephemeral port with another process's IPv4 socket can no longer swallow replies and hang `connect()` until the idle timeout. This covers the Quinn and Noq ClientBuilders, the FFI client and the Quiche ez client. Caller-supplied endpoints and sockets are unchanged. - -## Plan - -Decided with the maintainer on 2026-11-06 (planning interview): - -- Every native default, not only the router's Quinn path. A bind option alone would leave the default broken. -- Quinn and Noq `ClientBuilder` clients (and FFI, which builds its own Quinn endpoint) hold an IPv4 `0.0.0.0:0` endpoint and an IPv6-only `[::]:0` endpoint (`IPV6_V6ONLY`). Each is bound lazily on the first connect to that family and shared by later sessions and clones. Bind failures become a `ClientError` from `connect()`, replacing today's `unwrap()` in `build()`. Hosts with IPv6 disabled fail only IPv6 connects. An IPv4-mapped remote (`::ffff:a.b.c.d`) uses the IPv4 endpoint and its unmapped address. -- `close`, `wait_closed` and `Drop` (in FFI and elsewhere) cover every endpoint bound so far. A connect after `close` fails rather than binding a new endpoint, and so does one still in progress: the bind and the closed check happen under the same lock, so no endpoint is ever bound after `close`. -- `Client::new(endpoint, config)` keeps using the single endpoint it is given. -- The Quiche ez client already binds once per connect. Without `with_bind`, it binds after DNS resolution, using the chosen remote's family. -- No new public bind API. Keep it private until a consumer needs it. -- Regression tests assert the bound family per remote (IPv4 server: IPv4 socket; IPv6 server: v6-only socket) and an IPv4-mapped remote. They fail on today's dual-stack bind. The macOS collision itself is the reporter's repro and isn't run in CI. - -Decision prompts (✅ chosen): - -- Scope: ✅ all native defaults / Quinn router path only / bind option only / upstream, won't fix. -- Strategy: ✅ lazy per-family / eager both / endpoint per connect. -- Placement: ✅ a1 as one quest / a0 / split per backend. - -## Closes - -- [#455](https://github.com/moq-dev/web-transport/issues/455) - the implementation completes this report - -## Related - -- [Native client composition](/quest/a1/native-client-config.md) - edits the same Quinn client construction -- [Quinn on rind](/quest/a1/rind/quinn.md) - rebuilds the same builders' crypto diff --git a/quest/a1/native-client-config.md b/quest/a1/native-client-config.md index d68b8681..1e6e4f42 100644 --- a/quest/a1/native-client-config.md +++ b/quest/a1/native-client-config.md @@ -16,5 +16,4 @@ Do not add a pretend browser certificate-verification bypass, a browser panic or ## Related -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - edits the same Quinn client construction - [Quinn on rind](/quest/a1/rind/quinn.md) - replaces the crypto these builders configure diff --git a/quest/a1/rind/quinn.md b/quest/a1/rind/quinn.md index fc683dde..94baca10 100644 --- a/quest/a1/rind/quinn.md +++ b/quest/a1/rind/quinn.md @@ -19,4 +19,3 @@ Decided with the maintainer on 2026-11-06 (planning interview): ## Related - [Native client composition](/quest/a1/native-client-config.md) - converts a configured Quinn client into the router -- [Dual-stack bind](/quest/a1/dual-stack-bind.md) - rebuilds the same builders' endpoints diff --git a/rs/web-transport-ffi/README.md b/rs/web-transport-ffi/README.md index 8c20901e..0059360c 100644 --- a/rs/web-transport-ffi/README.md +++ b/rs/web-transport-ffi/README.md @@ -4,6 +4,12 @@ Low-level UniFFI bindings for WebTransport over HTTP/3 (QUIC). Generates Python, For an ergonomic Python API (async context managers, legacy exception hierarchy), see [`web-transport-rs`](https://pypi.org/project/web-transport-rs/) which wraps this package. +Clients lazily share one socket per destination address family: IPv4 uses +`0.0.0.0:0`, IPv6 uses IPv6-only `[::]:0`, and IPv4-mapped IPv6 uses IPv4. +Socket setup errors are returned by `connect`. `close` and final client drop +close both sockets, including pending connections. Call `close` before awaiting +`wait_closed` to prevent new connections while both sockets drain. + ## Building ```bash diff --git a/rs/web-transport-ffi/src/client.rs b/rs/web-transport-ffi/src/client.rs index 6904bb9c..408e8859 100644 --- a/rs/web-transport-ffi/src/client.rs +++ b/rs/web-transport-ffi/src/client.rs @@ -55,7 +55,6 @@ impl Default for ClientConfig { #[derive(uniffi::Object)] pub struct Client { inner: web_transport_quinn::Client, - endpoint: quinn::Endpoint, } #[uniffi::export] @@ -143,15 +142,8 @@ impl Client { client_config.transport_config(transport.into()); let _guard = RUNTIME.enter(); - let endpoint = quinn::Endpoint::client("[::]:0".parse().unwrap()) - .or_else(|_| quinn::Endpoint::client("0.0.0.0:0".parse().unwrap())) - .map_err(|e| WebTransportError::Io(format!("failed to create endpoint: {e}")))?; - - let client = web_transport_quinn::Client::new(endpoint.clone(), client_config); - Ok(Arc::new(Self { - inner: client, - endpoint, + inner: web_transport_quinn::Client::from(client_config), })) } @@ -178,15 +170,15 @@ impl Client { let var_code = quinn::VarInt::from_u64(code) .map_err(|_| WebTransportError::invalid("code must be < 2^62"))?; let _guard = RUNTIME.enter(); - self.endpoint.close(var_code, reason.as_bytes()); + self.inner.close(var_code, reason.as_bytes()); Ok(()) } - /// Wait until all connections have shut down cleanly. + /// Wait for bound endpoints to drain; close first to prevent new connections. pub async fn wait_closed(&self) { - let endpoint = self.endpoint.clone(); + let client = self.inner.clone(); let handle = RUNTIME.spawn(async move { - endpoint.wait_idle().await; + client.wait_idle().await; }); let _ = handle.await; } @@ -195,7 +187,7 @@ impl Client { impl Drop for Client { fn drop(&mut self) { let _guard = RUNTIME.enter(); - self.endpoint.close(quinn::VarInt::from_u32(0), b""); + self.inner.close(quinn::VarInt::from_u32(0), b""); } } diff --git a/rs/web-transport-ffi/src/test.rs b/rs/web-transport-ffi/src/test.rs index 983fd5ec..4bae179f 100644 --- a/rs/web-transport-ffi/src/test.rs +++ b/rs/web-transport-ffi/src/test.rs @@ -101,3 +101,83 @@ fn sha256(cert_der: &[u8]) -> Vec { hasher.update(cert_der); hasher.finalize().to_vec() } + +async fn dual_family_sessions( + client: &Client, +) -> (Vec, Vec>) { + let (chain, key) = self_signed_cert(); + let mut sessions = Vec::new(); + let mut servers = Vec::new(); + for bind in ["127.0.0.1:0", "[::1]:0"] { + let server = Server::new(ServerConfig { + certificate_chain: chain.clone(), + private_key: key.clone(), + bind: bind.into(), + congestion_control: CongestionControl::Default, + max_idle_timeout_secs: Some(10.0), + keep_alive_interval_secs: None, + }) + .unwrap(); + let addr = server.local_addr(); + let remote = std::net::SocketAddr::new(addr.host.parse().unwrap(), addr.port); + let (session, peer) = tokio::join!(client.connect(format!("https://{remote}/")), async { + server.accept().await.unwrap().accept().await.unwrap() + },); + sessions.push((session.unwrap(), peer)); + servers.push(server); + } + (sessions, servers) +} + +type ServerSession = ( + std::sync::Arc, + std::sync::Arc, +); + +#[tokio::test] +async fn client_close_and_drop_cover_both_families() { + tokio::time::timeout(TIMEOUT, async { + for explicit_close in [true, false] { + let client = Client::new(ClientConfig { + no_cert_verification: true, + ..Default::default() + }) + .unwrap(); + let (sessions, servers) = dual_family_sessions(&client).await; + if explicit_close { + client.close(42, "all sockets".into()).unwrap(); + assert!(client + .connect("https://127.0.0.1:443/".into()) + .await + .is_err()); + assert!(client.connect("https://[::1]:443/".into()).await.is_err()); + client.wait_closed().await; + } + drop(client); + for (session, peer) in &sessions { + assert!(session.receive_datagram().await.is_err()); + assert!(peer.receive_datagram().await.is_err()); + } + for server in servers { + server.close(0, String::new()).unwrap(); + server.wait_closed().await; + } + } + }) + .await + .expect("client shutdown timed out"); +} + +#[tokio::test] +async fn closing_unused_client_prevents_first_bind() { + let client = Client::new(ClientConfig::default()).unwrap(); + client.close(0, String::new()).unwrap(); + client.wait_closed().await; + for url in [ + "https://127.0.0.1:443/", + "https://[::1]:443/", + "https://[::ffff:127.0.0.1]:443/", + ] { + assert!(client.connect(url.into()).await.is_err()); + } +} diff --git a/rs/web-transport-noq/Cargo.toml b/rs/web-transport-noq/Cargo.toml index 5bf7234a..8b1267d3 100644 --- a/rs/web-transport-noq/Cargo.toml +++ b/rs/web-transport-noq/Cargo.toml @@ -39,6 +39,7 @@ rustls = { version = "0.23", default-features = false, features = [ "std", ] } rustls-native-certs = "0.8" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ "io-util", diff --git a/rs/web-transport-noq/README.md b/rs/web-transport-noq/README.md index a6d7dc15..a4a6c149 100644 --- a/rs/web-transport-noq/README.md +++ b/rs/web-transport-noq/README.md @@ -28,28 +28,20 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-noq, first you need to create a [noq::Endpoint](https://docs.rs/noq/latest/noq/struct.Endpoint.html); see the documentation and examples for more information. -The only requirement is that the ALPN is set to `web_transport_noq::ALPN` (aka `h3`). -Afterwards, you use [web_transport_noq::accept](https://docs.rs/web-transport-noq/latest/web_transport_noq/fn.accept.html) (as a server) or [web_transport_noq::connect](https://docs.rs/web-transport-noq/latest/web_transport_noq/fn.connect.html) (as a client) to establish a WebTransport session. -This will take over the QUIC connection and perform the boring HTTP/3 handshake for you. +Use `ClientBuilder::new().with_system_roots()?` to build a reusable client, +then call `client.connect(url).await?` to establish a session. Clones share the +client's endpoints. The first connection to each address family binds an +IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations +use IPv4. Socket setup failures are returned by `connect`, so building a client +does not require IPv6 support or a running async runtime. Both sockets use the +runtime present at construction, or at the first connection if constructed +outside a runtime. Keep that runtime alive while using the client. -See the [examples](examples) or [moq-native](https://github.com/moq-dev/moq-rs/blob/main/moq-native/src/quic.rs) for a full setup. +For an existing `noq::ClientConfig`, `Client::new(endpoint, config)` keeps +the caller's endpoint and address handling. Set its ALPN to `h3`. -```rust - // Create a QUIC client. - let mut endpoint = noq::Endpoint::client("[::]:0".parse()?)?; - endpoint.set_default_client_config(/* ... */); - - // Connect to the given URL. - let session = web_transport_noq::connect(&client, Url::parse("https://localhost")?).await?; - - // Create a bidirectional stream. - let (mut send, mut recv) = session.open_bi().await?; - - // Send a message. - send.write(b"hello").await?; -``` +See the [examples](examples) for certificate setup and server construction. ## API The `web-transport-noq` API is almost identical to the Noq API, except that [Connection](https://docs.rs/noq/latest/noq/struct.Connection.html) is called [Session](https://docs.rs/web-transport-noq/latest/web_transport_noq/struct.Session.html). diff --git a/rs/web-transport-noq/src/client.rs b/rs/web-transport-noq/src/client.rs index c3dcee47..c48d71fa 100644 --- a/rs/web-transport-noq/src/client.rs +++ b/rs/web-transport-noq/src/client.rs @@ -1,5 +1,5 @@ use std::net::{IpAddr, SocketAddr}; -use std::sync::Arc; +use std::sync::{Arc, Mutex}; use crate::proto::ConnectRequest; #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] @@ -148,11 +148,7 @@ impl ClientBuilder { client_config.transport_config(transport.into()); - let client = noq::Endpoint::client("[::]:0".parse().unwrap()).unwrap(); - Ok(Client { - endpoint: client, - config: client_config, - }) + Ok(Client::from_config(client_config)) } } @@ -199,7 +195,7 @@ impl DangerousClientBuilder { /// A client for connecting to a WebTransport server. #[derive(Clone, Debug)] pub struct Client { - endpoint: noq::Endpoint, + endpoints: Arc>, config: noq::ClientConfig, } @@ -208,7 +204,10 @@ impl Client { /// /// The ALPN MUST be set to [ALPN]. pub fn new(endpoint: noq::Endpoint, config: noq::ClientConfig) -> Self { - Self { endpoint, config } + Self { + endpoints: Arc::new(Mutex::new(Binding::Custom(endpoint))), + config, + } } /// Connect to the server. @@ -217,7 +216,6 @@ impl Client { request: impl Into, ) -> Result { let request = request.into(); - let port = request.url.port().unwrap_or(443); // TODO error on username:password in host @@ -248,8 +246,10 @@ impl Client { // Connect to the server using the addr we just resolved. let conn = self - .endpoint - .connect_with(self.config.clone(), remote, &host)?; + .endpoints + .lock() + .unwrap() + .connect(self.config.clone(), remote, &host)?; let conn = conn.await?; // Connect with the connection we established. @@ -257,6 +257,144 @@ impl Client { } } +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl Client { + /// Create a client that lazily binds one socket for each server address family. + /// + /// The config must negotiate [`ALPN`]; socket setup errors are returned by [`Self::connect`]. + fn from_config(config: noq::ClientConfig) -> Self { + Self { + endpoints: Arc::new(Mutex::new(Binding::Default { + ipv4: None, + ipv6: None, + runtime: ClientRuntime::current(), + })), + config, + } + } +} + +#[derive(Debug)] +enum Binding { + Custom(noq::Endpoint), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Default { + ipv4: Option, + ipv6: Option, + runtime: Option, + }, +} + +#[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] +impl Binding { + fn endpoints(&self) -> impl Iterator { + match self { + Self::Custom(endpoint) => [Some(endpoint), None], + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Self::Default { ipv4, ipv6, .. } => [ipv4.as_ref(), ipv6.as_ref()], + } + .into_iter() + .flatten() + } +} + +impl Binding { + fn connect( + &mut self, + config: noq::ClientConfig, + remote: SocketAddr, + host: &str, + ) -> Result { + let (endpoint, remote) = match self { + Binding::Custom(endpoint) => (endpoint, remote), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Binding::Default { + ipv4, + ipv6, + runtime, + } => { + let remote = match remote { + SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; + if slot.is_none() { + if runtime.is_none() { + *runtime = ClientRuntime::current(); + } + let runtime = runtime.as_ref().ok_or_else(|| { + ClientError::IoError(Arc::new(std::io::Error::other( + "no async runtime found", + ))) + })?; + *slot = Some( + runtime + .bind(remote) + .map_err(|err| ClientError::IoError(Arc::new(err)))?, + ); + } + (slot.as_mut().unwrap(), remote) + } + }; + Ok(endpoint.connect_with(config, remote, host)?) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +#[derive(Debug)] +struct ClientRuntime { + inner: Arc, + tokio: Option, +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl ClientRuntime { + fn current() -> Option { + Some(Self { + inner: noq::default_runtime()?, + tokio: tokio::runtime::Handle::try_current().ok(), + }) + } + + fn bind(&self, remote: SocketAddr) -> std::io::Result { + // TokioRuntime registers sockets and spawns drivers in the current context. + let _guard = self.tokio.as_ref().map(tokio::runtime::Handle::enter); + bind_endpoint(remote, self.inner.clone()) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint( + remote: SocketAddr, + runtime: Arc, +) -> std::io::Result { + let bind: SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = bind_socket(bind)?; + noq::Endpoint::new(noq::EndpointConfig::default(), None, socket, runtime) +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_socket(bind: SocketAddr) -> std::io::Result { + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.bind(&bind.into())?; + Ok(socket.into()) +} + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] impl Default for Client { fn default() -> Self { @@ -376,3 +514,6 @@ impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification { self.0.signature_verification_algorithms.supported_schemes() } } + +#[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] +mod tests; diff --git a/rs/web-transport-noq/src/client/tests.rs b/rs/web-transport-noq/src/client/tests.rs new file mode 100644 index 00000000..e8397dc0 --- /dev/null +++ b/rs/web-transport-noq/src/client/tests.rs @@ -0,0 +1,298 @@ +use super::*; +use noq as backend; + +fn client() -> Client { + #[cfg(feature = "aws-lc-rs")] + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); + #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] + let _ = rustls::crypto::ring::default_provider().install_default(); + ClientBuilder::new() + .dangerous() + .with_no_certificate_verification() + .unwrap() +} + +fn bound(client: &Client) -> Vec { + client + .endpoints + .lock() + .unwrap() + .endpoints() + .map(|endpoint| endpoint.local_addr().unwrap()) + .collect() +} + +fn close(client: &Client, code: backend::VarInt, reason: &[u8]) { + for endpoint in client.endpoints.lock().unwrap().endpoints() { + endpoint.close(code, reason); + } +} + +async fn wait_idle(client: &Client) { + let endpoints: Vec<_> = client + .endpoints + .lock() + .unwrap() + .endpoints() + .cloned() + .collect(); + futures::future::join_all(endpoints.iter().map(backend::Endpoint::wait_idle)).await; +} + +fn url(remote: SocketAddr) -> url::Url { + url::Url::parse(&format!("https://{remote}/")).unwrap() +} + +#[test] +fn construction_needs_no_runtime_or_socket() { + assert!(bound(&client()).is_empty()); +} + +#[test] +fn ipv6_socket_is_v6_only() { + let socket = bind_socket("[::]:0".parse().unwrap()).unwrap(); + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); +} + +#[tokio::test] +async fn default_ipv4_socket() { + let client = client(); + let connect = client.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + assert!(bound(&client)[0].is_ipv4()); + close(&client, 0u32.into(), b""); + assert!(connect.await.is_err()); + wait_idle(&client).await; +} + +#[test] +fn missing_runtime_is_a_connect_error_and_allows_later_connections() { + // Poll an IP-literal connect without a runtime. Construction succeeds, + // while the lazy endpoint reports its setup error instead of panicking. + let client = client(); + let error = futures::executor::block_on(client.connect(url("127.0.0.1:443".parse().unwrap()))) + .unwrap_err(); + assert!(matches!(error, ClientError::IoError(_))); + assert!(bound(&client).is_empty()); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + runtime.block_on(async { + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (_session, _peer) = pair(&client, &mut server, remote).await; + close(&client, 0u32.into(), b""); + wait_idle(&client).await; + }); +} + +#[tokio::test] +async fn caller_endpoint_keeps_its_address_and_remote_handling() { + let endpoint = backend::Endpoint::client("127.0.0.1:0".parse().unwrap()).unwrap(); + let addr = endpoint.local_addr().unwrap(); + let client = Client::new(endpoint, client().config); + let error = client + .connect(url("[::ffff:127.0.0.1]:443".parse().unwrap())) + .await + .unwrap_err(); + assert!(matches!( + error, + ClientError::NoqError(backend::ConnectError::InvalidRemoteAddress(_)) + )); + assert_eq!(bound(&client), vec![addr]); + close(&client, 0u32.into(), b""); + wait_idle(&client).await; +} + +fn server(addr: SocketAddr) -> crate::Server { + let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); + crate::ServerBuilder::new() + .with_addr(addr) + .with_certificate( + vec![cert.cert.der().clone()], + rustls::pki_types::PrivatePkcs8KeyDer::from(cert.signing_key.serialize_der()).into(), + ) + .unwrap() +} + +async fn pair( + client: &Client, + server: &mut crate::Server, + remote: SocketAddr, +) -> (Session, Session) { + tokio::time::timeout(std::time::Duration::from_secs(10), async { + let (client, server) = tokio::join!(client.connect(url(remote)), async { + server.accept().await.unwrap().ok().await.unwrap() + },); + (client.unwrap(), server) + }) + .await + .expect("loopback connection timed out") +} + +#[tokio::test] +async fn families_mapped_addresses_and_clones_share_endpoints() { + let client = client(); + let clone = client.clone(); + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + let addr4 = ipv4.local_addr().unwrap(); + let addr6 = ipv6.local_addr().unwrap(); + let (session4, peer4) = pair(&client, &mut ipv4, addr4).await; + let first = bound(&client); + assert_eq!(first.len(), 1); + assert!(first[0].is_ipv4()); + let (session6, peer6) = pair(&clone, &mut ipv6, addr6).await; + let both = bound(&client); + assert_eq!(both.len(), 2); + assert!(both[1].is_ipv6()); + assert_eq!(both[0], first[0]); + let mapped = SocketAddr::new( + std::net::Ipv4Addr::LOCALHOST.to_ipv6_mapped().into(), + addr4.port(), + ); + let (session_mapped, peer_mapped) = pair(&clone, &mut ipv4, mapped).await; + assert!(session_mapped + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + .is_ipv4()); + assert_eq!( + peer4 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap(), + peer_mapped + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + ); + assert_eq!(bound(&clone), both); + + close(&client, 42u32.into(), b"all endpoints"); + for session in [&session4, &session6, &session_mapped] { + let error = backend::Connection::closed(session).await; + assert!( + matches!(error, backend::ConnectionError::LocallyClosed), + "{error:?}" + ); + } + for peer in [&peer4, &peer6, &peer_mapped] { + backend::Connection::closed(peer).await; + } + wait_idle(&client).await; + assert!(client + .endpoints + .lock() + .unwrap() + .endpoints() + .all(|endpoint| endpoint.open_connections() == 0)); + drop((session4, session6, session_mapped)); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn simultaneous_first_connections_share_one_socket() { + let client = client(); + let clone = client.clone(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (first, second, peers) = tokio::time::timeout(std::time::Duration::from_secs(10), async { + tokio::join!( + client.connect(url(remote)), + clone.connect(url(remote)), + async { + let first = server.accept().await.unwrap().ok().await.unwrap(); + let second = server.accept().await.unwrap().ok().await.unwrap(); + (first, second) + } + ) + }) + .await + .expect("concurrent loopback connections timed out"); + let first = first.unwrap(); + let second = second.unwrap(); + assert_eq!(bound(&client).len(), 1); + assert_eq!( + peers + .0 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap(), + peers + .1 + .path(noq::PathId::ZERO) + .unwrap() + .remote_address() + .unwrap() + ); + close(&client, 0u32.into(), b""); + wait_idle(&client).await; + drop((first, second, peers)); +} + +#[tokio::test] +async fn dropping_native_client_keeps_returned_session_usable() { + let client = client(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, &mut server, remote).await; + drop(client); + session + .send_datagram(bytes::Bytes::from_static(b"still open")) + .unwrap(); + let received = tokio::time::timeout(std::time::Duration::from_secs(10), peer.read_datagram()) + .await + .expect("datagram after client drop timed out") + .unwrap(); + assert_eq!(received.as_ref(), b"still open"); + server.close(0u32.into(), b""); + backend::Connection::closed(&session).await; + server.wait_idle().await; +} + +#[test] +fn construction_runtime_survives_first_connect_on_transient_runtime() { + let owner = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let client = { + let _guard = owner.enter(); + client() + }; + let transient = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + transient.block_on(async { + for addr in ["127.0.0.1:443", "[::1]:443"] { + let connect = client.connect(url(addr.parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + } + }); + drop(transient); + owner.block_on(async { + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + for server in [&mut ipv4, &mut ipv6] { + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, server, remote).await; + session + .send_datagram(bytes::Bytes::from_static(b"owner runtime")) + .unwrap(); + assert_eq!( + peer.read_datagram().await.unwrap().as_ref(), + b"owner runtime" + ); + } + close(&client, 0u32.into(), b""); + wait_idle(&client).await; + }); +} diff --git a/rs/web-transport-noq/src/error.rs b/rs/web-transport-noq/src/error.rs index 804b88cc..0f60fe5b 100644 --- a/rs/web-transport-noq/src/error.rs +++ b/rs/web-transport-noq/src/error.rs @@ -7,6 +7,9 @@ use crate::{ConnectError, SettingsError}; /// An error returned when connecting to a WebTransport endpoint. #[derive(Error, Debug, Clone)] pub enum ClientError { + #[error("failed to bind client endpoint: {0}")] + IoError(Arc), + #[error("unexpected end of stream")] UnexpectedEnd, diff --git a/rs/web-transport-quiche/Cargo.toml b/rs/web-transport-quiche/Cargo.toml index 386922e5..3674b499 100644 --- a/rs/web-transport-quiche/Cargo.toml +++ b/rs/web-transport-quiche/Cargo.toml @@ -29,6 +29,7 @@ http = "1" kio = "0.6.2" rustls-pki-types = "1" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ diff --git a/rs/web-transport-quiche/README.md b/rs/web-transport-quiche/README.md index 871922ce..3964d0b7 100644 --- a/rs/web-transport-quiche/README.md +++ b/rs/web-transport-quiche/README.md @@ -39,7 +39,10 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-quiche, figure it out yourself lul. +The default client resolves the remote before binding an ephemeral socket of +the same address family. IPv6 sockets are IPv6-only; IPv4-mapped IPv6 remotes +use IPv4. Explicit `with_bind` and `with_socket` inputs retain their address +handling. ## Session lifetime diff --git a/rs/web-transport-quiche/src/ez/client.rs b/rs/web-transport-quiche/src/ez/client.rs index 5dceec52..cf193879 100644 --- a/rs/web-transport-quiche/src/ez/client.rs +++ b/rs/web-transport-quiche/src/ez/client.rs @@ -160,12 +160,6 @@ impl ClientBuilder { /// /// This takes ownership because the underlying quiche implementation doesn't support reusing the same socket. pub async fn connect(mut self, host: &str, port: u16) -> io::Result { - if self.socket.is_none() { - self = self.with_bind("[::]:0")?; - } - - let socket = self.socket.take().unwrap(); - let mut remotes = match tokio::net::lookup_host((host, port)).await { Ok(remotes) => remotes, Err(err) => { @@ -187,6 +181,7 @@ impl ClientBuilder { } }; + let (socket, remote) = self.socket(remote)?; socket.connect(remote).await?; // Enable the offloads the kernel supports before the socket is wrapped; @@ -255,6 +250,40 @@ impl ClientBuilder { driver, }) } + + fn socket( + &mut self, + remote: std::net::SocketAddr, + ) -> io::Result<(tokio::net::UdpSocket, std::net::SocketAddr)> { + if let Some(socket) = self.socket.take() { + return Ok((socket, remote)); + } + + let remote = match remote { + std::net::SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| std::net::SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let bind: std::net::SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.set_nonblocking(true)?; + socket.bind(&bind.into())?; + Ok((tokio::net::UdpSocket::from_std(socket.into())?, remote)) + } } /// A QUIC connection that is still completing the TLS handshake. @@ -277,3 +306,46 @@ impl Connecting { Ok(self.connection) } } + +#[cfg(test)] +mod tests { + use super::*; + use std::net::SocketAddr; + + #[tokio::test] + async fn default_socket_matches_each_remote_family() { + for remote in ["127.0.0.1:443", "[::1]:443", "[::ffff:127.0.0.1]:443"] { + let remote: SocketAddr = remote.parse().unwrap(); + let (socket, target) = ClientBuilder::new().socket(remote).unwrap(); + let local = socket.local_addr().unwrap(); + assert_eq!(local.is_ipv4(), target.is_ipv4()); + assert!(local.ip().is_unspecified()); + assert_ne!(local.port(), 0); + if target.is_ipv6() { + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); + } else { + assert_eq!(target, "127.0.0.1:443".parse().unwrap()); + } + } + } + + #[tokio::test] + async fn explicit_socket_and_bind_are_preserved() { + let remote = "[::ffff:127.0.0.1]:443".parse().unwrap(); + let socket = std::net::UdpSocket::bind("127.0.0.1:0").unwrap(); + let addr = socket.local_addr().unwrap(); + let (socket, target) = ClientBuilder::new() + .with_socket(socket) + .unwrap() + .socket(remote) + .unwrap(); + assert_eq!(socket.local_addr().unwrap(), addr); + assert_eq!(target, remote); + + let mut builder = ClientBuilder::new().with_bind("[::1]:0").unwrap(); + let addr = builder.socket.as_ref().unwrap().local_addr().unwrap(); + let (socket, target) = builder.socket(remote).unwrap(); + assert_eq!(socket.local_addr().unwrap(), addr); + assert_eq!(target, remote); + } +} diff --git a/rs/web-transport-quiche/tests/connect.rs b/rs/web-transport-quiche/tests/connect.rs index 6406980d..1033ae3c 100644 --- a/rs/web-transport-quiche/tests/connect.rs +++ b/rs/web-transport-quiche/tests/connect.rs @@ -60,6 +60,15 @@ async fn spawn_server(bind: SocketAddr) -> Result<(SocketAddr, tokio::task::Join Ok((addr, handle)) } +async fn stop_server(server: tokio::task::JoinHandle<()>) -> Result<()> { + server.abort(); + match server.await { + Ok(()) => Ok(()), + Err(err) if err.is_cancelled() => Ok(()), + Err(err) => Err(err.into()), + } +} + fn client() -> ClientBuilder { // The certificate is self-signed and the subject here is address handling, // which verify.rs already covers from the other side. @@ -79,7 +88,6 @@ async fn connect_ipv6_literal_url() -> Result<()> { let url = Url::parse(&format!("https://[::1]:{}/", addr.port()))?; let session = client() - .with_bind((Ipv6Addr::LOCALHOST, 0))? .connect(url) .await .context("an IPv6 literal URL should resolve to the address it names")? @@ -88,7 +96,7 @@ async fn connect_ipv6_literal_url() -> Result<()> { session.close(0, "bye"); session.closed().await; - server.abort(); + stop_server(server).await?; Ok(()) } @@ -101,15 +109,37 @@ async fn connect_ipv4_literal_url() -> Result<()> { let (addr, server) = spawn_server((Ipv4Addr::LOCALHOST, 0).into()).await?; let url = Url::parse(&format!("https://127.0.0.1:{}/", addr.port()))?; - let session = client() - .with_bind((Ipv4Addr::LOCALHOST, 0))? - .connect(url) - .await? - .established() - .await?; + let session = client().connect(url).await?.established().await?; session.close(0, "bye"); session.closed().await; - server.abort(); + stop_server(server).await?; + Ok(()) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn connect_ipv4_mapped_literal_url() -> Result<()> { + let (addr, server) = spawn_server((Ipv4Addr::LOCALHOST, 0).into()).await?; + let url = Url::parse(&format!("https://[::ffff:127.0.0.1]:{}/", addr.port()))?; + let session = client().connect(url).await?.established().await?; + session.close(0, "bye"); + session.closed().await; + stop_server(server).await?; + Ok(()) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn connect_hostname_uses_resolved_family() -> Result<()> { + use std::net::ToSocketAddrs; + let bind = ("localhost", 0) + .to_socket_addrs()? + .next() + .context("localhost did not resolve")?; + let (addr, server) = spawn_server(bind).await?; + let url = Url::parse(&format!("https://localhost:{}/", addr.port()))?; + let session = client().connect(url).await?.established().await?; + session.close(0, "bye"); + session.closed().await; + stop_server(server).await?; Ok(()) } diff --git a/rs/web-transport-quinn/Cargo.toml b/rs/web-transport-quinn/Cargo.toml index 7dc01064..4bcbb07b 100644 --- a/rs/web-transport-quinn/Cargo.toml +++ b/rs/web-transport-quinn/Cargo.toml @@ -39,6 +39,7 @@ rustls = { version = "0.23", default-features = false, features = [ "std", ] } rustls-native-certs = "0.8" +socket2 = "0.6" thiserror = "2" tokio = { version = "1", default-features = false, features = [ diff --git a/rs/web-transport-quinn/README.md b/rs/web-transport-quinn/README.md index 2ab5b96c..5c43071f 100644 --- a/rs/web-transport-quinn/README.md +++ b/rs/web-transport-quinn/README.md @@ -28,28 +28,25 @@ Both endpoints can send datagrams below the MTU size (~1.2kb minimum) and they m They are basically UDP packets, except they are encrypted and congestion controlled. # Usage -To use web-transport-quinn, first you need to create a [quinn::Endpoint](https://docs.rs/quinn/latest/quinn/struct.Endpoint.html); see the documentation and examples for more information. -The only requirement is that the ALPN is set to `web_transport_quinn::ALPN` (aka `h3`). -Afterwards, you use [web_transport_quinn::accept](https://docs.rs/web-transport-quinn/latest/web_transport_quinn/fn.accept.html) (as a server) or [web_transport_quinn::connect](https://docs.rs/web-transport-quinn/latest/web_transport_quinn/fn.connect.html) (as a client) to establish a WebTransport session. -This will take over the QUIC connection and perform the boring HTTP/3 handshake for you. +Use `ClientBuilder::new().with_system_roots()?` to build a reusable client, +then call `client.connect(url).await?` to establish a session. Clones share the +client's endpoints. The first connection to each address family binds an +IPv4 `0.0.0.0:0` or IPv6-only `[::]:0` socket. IPv4-mapped IPv6 destinations +use IPv4. Socket setup failures are returned by `connect`, so building a client +does not require IPv6 support or a running async runtime. Both sockets use the +runtime present at construction, or at the first connection if constructed +outside a runtime. Keep that runtime alive while using the client. -See the [examples](examples) or [moq-native](https://github.com/moq-dev/moq-rs/blob/main/moq-native/src/quic.rs) for a full setup. +For an existing `quinn::ClientConfig`, `Client::from(config)` uses the same +lazy sockets. Set its ALPN to `h3`. `Client::new(endpoint, config)` instead keeps +the caller's endpoint and address handling. -```rust - // Create a QUIC client. - let mut endpoint = quinn::Endpoint::client("[::]:0".parse()?)?; - endpoint.set_default_client_config(/* ... */); +`client.close(code, reason)` closes all bound endpoints and prevents new +connections through any clone. Await `client.wait_idle()` after closing to +wait for every bound endpoint to drain. - // Connect to the given URL. - let session = web_transport_quinn::connect(&client, Url::parse("https://localhost")?).await?; - - // Create a bidirectional stream. - let (mut send, mut recv) = session.open_bi().await?; - - // Send a message. - send.write(b"hello").await?; -``` +See the [examples](examples) for certificate setup and server construction. ## Async runtimes Each WebTransport session runs a background task that reads capsules from the CONNECT stream, and `Session::close` runs the graceful close in another. @@ -80,7 +77,7 @@ let client = ClientBuilder::new() An explicit `LowLatency` or `Throughput` congestion choice overrides only the configuration's congestion controller, regardless of builder call order. `Default` removes that override and preserves the supplied controller. Calling `with_transport_config` again replaces the supplied configuration; other settings survive the congestion override. -Quinn's `TransportConfig` is not cloneable, so `ClientBuilder` owns it and does not implement `Clone`. Construct a builder for each configuration, then clone the finished `Client` to share its endpoint and settings across connections. +Quinn's `TransportConfig` is not cloneable, so `ClientBuilder` owns it and does not implement `Clone`. Construct a builder for each configuration, then clone the finished `Client` to share its endpoints and settings across connections. ## API The `web-transport-quinn` API is almost identical to the Quinn API, except that [Connection](https://docs.rs/quinn/latest/quinn/struct.Connection.html) is called [Session](https://docs.rs/web-transport-quinn/latest/web_transport_quinn/struct.Session.html). diff --git a/rs/web-transport-quinn/src/client.rs b/rs/web-transport-quinn/src/client.rs index eab00450..fb793690 100644 --- a/rs/web-transport-quinn/src/client.rs +++ b/rs/web-transport-quinn/src/client.rs @@ -1,5 +1,5 @@ use std::net::{IpAddr, SocketAddr}; -use std::sync::Arc; +use std::sync::{Arc, Mutex}; use crate::proto::ConnectRequest; #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] @@ -58,7 +58,7 @@ pub(crate) fn transport_config( /// Construct a WebTransport [Client] using sane defaults. /// /// This is optional; advanced users may use [Client::new] directly. -/// Build a client before cloning it to share its endpoint and configuration. +/// Build a client before cloning it to share its endpoints and configuration. pub struct ClientBuilder { provider: crypto::Provider, transport: quinn::TransportConfig, @@ -179,11 +179,7 @@ impl ClientBuilder { self.congestion_controller.as_ref(), )); - let client = quinn::Endpoint::client("[::]:0".parse().unwrap()).unwrap(); - Ok(Client { - endpoint: client, - config: client_config, - }) + Ok(Client::from(client_config)) } } @@ -230,7 +226,7 @@ impl DangerousClientBuilder { /// A client for connecting to a WebTransport server. #[derive(Clone, Debug)] pub struct Client { - endpoint: quinn::Endpoint, + endpoints: Arc>, config: quinn::ClientConfig, } @@ -239,7 +235,13 @@ impl Client { /// /// The ALPN MUST be set to [ALPN]. pub fn new(endpoint: quinn::Endpoint, config: quinn::ClientConfig) -> Self { - Self { endpoint, config } + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Custom(endpoint), + closed: false, + })), + config, + } } /// Connect to the server. @@ -248,6 +250,9 @@ impl Client { request: impl Into, ) -> Result { let request = request.into(); + if self.endpoints.lock().unwrap().closed { + return Err(quinn::ConnectError::EndpointStopping.into()); + } let port = request.url.port().unwrap_or(443); @@ -279,15 +284,197 @@ impl Client { // Connect to the server using the addr we just resolved. let conn = self - .endpoint - .connect_with(self.config.clone(), remote, &host)?; + .endpoints + .lock() + .unwrap() + .connect(self.config.clone(), remote, &host)?; let conn = conn.await?; // Connect with the connection we established. - Session::connect(conn, request).await + let session = Session::connect(conn, request).await?; + if self.endpoints.lock().unwrap().closed { + return Err(quinn::ConnectionError::LocallyClosed.into()); + } + Ok(session) + } + + /// Close all bound endpoints and reject future connections from every clone. + pub fn close(&self, code: quinn::VarInt, reason: &[u8]) { + let mut endpoints = self.endpoints.lock().unwrap(); + if endpoints.closed { + return; + } + endpoints.closed = true; + for endpoint in endpoints.binding.endpoints() { + endpoint.close(code, reason); + } + } + + /// Wait for currently bound endpoints to become idle; close first to prevent new connections. + pub async fn wait_idle(&self) { + let endpoints: Vec<_> = self + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .cloned() + .collect(); + futures::future::join_all(endpoints.iter().map(quinn::Endpoint::wait_idle)).await; } } +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl From for Client { + /// Create a client that lazily binds one socket for each server address family. + /// + /// The config must negotiate [`ALPN`]; socket setup errors are returned by [`Self::connect`]. + fn from(config: quinn::ClientConfig) -> Self { + Self { + endpoints: Arc::new(Mutex::new(ClientEndpoints { + binding: Binding::Default { + ipv4: None, + ipv6: None, + runtime: ClientRuntime::current(), + }, + closed: false, + })), + config, + } + } +} + +#[derive(Debug)] +struct ClientEndpoints { + binding: Binding, + closed: bool, +} + +#[derive(Debug)] +enum Binding { + Custom(quinn::Endpoint), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Default { + ipv4: Option, + ipv6: Option, + runtime: Option, + }, +} + +impl Binding { + fn endpoints(&self) -> impl Iterator { + match self { + Self::Custom(endpoint) => [Some(endpoint), None], + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Self::Default { ipv4, ipv6, .. } => [ipv4.as_ref(), ipv6.as_ref()], + } + .into_iter() + .flatten() + } +} + +impl ClientEndpoints { + fn connect( + &mut self, + config: quinn::ClientConfig, + remote: SocketAddr, + host: &str, + ) -> Result { + if self.closed { + return Err(quinn::ConnectError::EndpointStopping.into()); + } + // Binding and starting the connection share close's lock, so close + // cannot miss a newly created endpoint or an in-progress connection. + let (endpoint, remote) = match &mut self.binding { + Binding::Custom(endpoint) => (endpoint, remote), + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] + Binding::Default { + ipv4, + ipv6, + runtime, + } => { + let remote = match remote { + SocketAddr::V6(addr) => addr + .ip() + .to_ipv4_mapped() + .map(|ip| SocketAddr::from((ip, addr.port()))) + .unwrap_or(remote), + _ => remote, + }; + let slot = if remote.is_ipv4() { ipv4 } else { ipv6 }; + if slot.is_none() { + if runtime.is_none() { + *runtime = ClientRuntime::current(); + } + let runtime = runtime.as_ref().ok_or_else(|| { + ClientError::IoError(Arc::new(std::io::Error::other( + "no async runtime found", + ))) + })?; + *slot = Some( + runtime + .bind(remote) + .map_err(|err| ClientError::IoError(Arc::new(err)))?, + ); + } + (slot.as_mut().unwrap(), remote) + } + }; + Ok(endpoint.connect_with(config, remote, host)?) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +#[derive(Debug)] +struct ClientRuntime { + inner: Arc, + tokio: Option, +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +impl ClientRuntime { + fn current() -> Option { + Some(Self { + inner: quinn::default_runtime()?, + tokio: tokio::runtime::Handle::try_current().ok(), + }) + } + + fn bind(&self, remote: SocketAddr) -> std::io::Result { + // TokioRuntime registers sockets and spawns drivers in the current context. + let _guard = self.tokio.as_ref().map(tokio::runtime::Handle::enter); + bind_endpoint(remote, self.inner.clone()) + } +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_endpoint( + remote: SocketAddr, + runtime: Arc, +) -> std::io::Result { + let bind: SocketAddr = if remote.is_ipv4() { + (std::net::Ipv4Addr::UNSPECIFIED, 0).into() + } else { + (std::net::Ipv6Addr::UNSPECIFIED, 0).into() + }; + let socket = bind_socket(bind)?; + quinn::Endpoint::new(quinn::EndpointConfig::default(), None, socket, runtime) +} + +#[cfg(any(feature = "aws-lc-rs", feature = "ring"))] +fn bind_socket(bind: SocketAddr) -> std::io::Result { + let socket = socket2::Socket::new( + socket2::Domain::for_address(bind), + socket2::Type::DGRAM, + Some(socket2::Protocol::UDP), + )?; + if bind.is_ipv6() { + socket.set_only_v6(true)?; + } + socket.bind(&bind.into())?; + Ok(socket.into()) +} + #[cfg(any(feature = "aws-lc-rs", feature = "ring"))] impl Default for Client { fn default() -> Self { @@ -409,137 +596,4 @@ impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification { } #[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] -pub(crate) mod tests { - use super::*; - use std::sync::atomic::{AtomicUsize, Ordering}; - use std::time::Duration; - - pub(crate) fn builder() -> ClientBuilder { - // Avoid a process-wide provider when both features are enabled. - #[cfg(feature = "aws-lc-rs")] - let provider = Arc::new(rustls::crypto::aws_lc_rs::default_provider()); - #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] - let provider = Arc::new(rustls::crypto::ring::default_provider()); - - ClientBuilder { - provider, - transport: quinn::TransportConfig::default(), - congestion_controller: None, - } - } - - struct CountingController(Arc); - - impl quinn::congestion::ControllerFactory for CountingController { - fn build( - self: Arc, - now: std::time::Instant, - mtu: u16, - ) -> Box { - self.0.fetch_add(1, Ordering::SeqCst); - Arc::new(quinn::congestion::NewRenoConfig::default()).build(now, mtu) - } - } - - pub(crate) fn custom_transport() -> (quinn::TransportConfig, Arc) { - let built = Arc::new(AtomicUsize::new(0)); - let mut transport = quinn::TransportConfig::default(); - transport - .max_idle_timeout(Some(Duration::from_secs(12).try_into().unwrap())) - .keep_alive_interval(Some(Duration::from_secs(3))) - .congestion_controller_factory(Arc::new(CountingController(built.clone()))); - (transport, built) - } - - pub(crate) fn assert_transport(config: &impl std::fmt::Debug, custom: bool) { - // Quinn exposes these settings only through Debug. - let expected = if custom { - custom_transport().0 - } else { - quinn::TransportConfig::default() - }; - let expected = format!("{expected:?}"); - let config = format!("{config:?}"); - assert!(config.contains(&expected), "{config}"); - } - - fn finish(builder: ClientBuilder, path: u8) -> Client { - match path { - 0 => builder.with_system_roots(), - 1 => builder.with_server_certificate_hashes(vec![]), - 2 => builder.with_server_certificates(vec![]), - 3 => builder.dangerous().with_no_certificate_verification(), - _ => unreachable!(), - } - .unwrap() - } - - #[tokio::test] - async fn builder_transport_reaches_every_client_config() { - for path in 0..4 { - let client = finish(builder(), path); - assert_transport(&client.config, false); - - for transport_first in [false, true] { - for algorithm in [ - CongestionControl::Default, - CongestionControl::LowLatency, - CongestionControl::Throughput, - ] { - let overridden = !matches!(algorithm, CongestionControl::Default); - let (transport, built) = custom_transport(); - let builder = builder(); - let builder = if transport_first { - builder - .with_transport_config(transport) - .with_congestion_control(algorithm) - } else { - builder - .with_congestion_control(algorithm) - .with_transport_config(transport) - }; - let client = finish(builder, path); - assert_transport(&client.config, true); - - // Controller construction happens before any network/TLS result, - // so the system-root path needs no publicly trusted test server. - let connecting = client - .endpoint - .connect_with( - client.config.clone(), - "[::1]:9".parse().unwrap(), - "localhost", - ) - .unwrap(); - assert_eq!(built.load(Ordering::SeqCst), usize::from(!overridden)); - drop(connecting); - client.endpoint.close(0u32.into(), b"done"); - } - } - } - } - - #[tokio::test] - async fn default_congestion_control_removes_only_the_overlay() { - let (transport, built) = custom_transport(); - let client = builder() - .with_transport_config(transport) - .with_congestion_control(CongestionControl::LowLatency) - .with_congestion_control(CongestionControl::Default) - .dangerous() - .with_no_certificate_verification() - .unwrap(); - assert_transport(&client.config, true); - let connecting = client - .endpoint - .connect_with( - client.config.clone(), - "[::1]:9".parse().unwrap(), - "localhost", - ) - .unwrap(); - assert_eq!(built.load(Ordering::SeqCst), 1); - drop(connecting); - client.endpoint.close(0u32.into(), b"done"); - } -} +pub(crate) mod tests; diff --git a/rs/web-transport-quinn/src/client/tests.rs b/rs/web-transport-quinn/src/client/tests.rs new file mode 100644 index 00000000..169c9e6c --- /dev/null +++ b/rs/web-transport-quinn/src/client/tests.rs @@ -0,0 +1,417 @@ +use super::*; +use quinn as backend; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Duration; + +pub(crate) fn builder() -> ClientBuilder { + // Avoid a process-wide provider when both features are enabled. + #[cfg(feature = "aws-lc-rs")] + let provider = Arc::new(rustls::crypto::aws_lc_rs::default_provider()); + #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] + let provider = Arc::new(rustls::crypto::ring::default_provider()); + + ClientBuilder { + provider, + transport: quinn::TransportConfig::default(), + congestion_controller: None, + } +} + +fn client() -> Client { + builder() + .dangerous() + .with_no_certificate_verification() + .unwrap() +} + +struct CountingController(Arc); + +impl quinn::congestion::ControllerFactory for CountingController { + fn build( + self: Arc, + now: std::time::Instant, + mtu: u16, + ) -> Box { + self.0.fetch_add(1, Ordering::SeqCst); + Arc::new(quinn::congestion::NewRenoConfig::default()).build(now, mtu) + } +} + +pub(crate) fn custom_transport() -> (quinn::TransportConfig, Arc) { + let built = Arc::new(AtomicUsize::new(0)); + let mut transport = quinn::TransportConfig::default(); + transport + .max_idle_timeout(Some(Duration::from_secs(12).try_into().unwrap())) + .keep_alive_interval(Some(Duration::from_secs(3))) + .congestion_controller_factory(Arc::new(CountingController(built.clone()))); + (transport, built) +} + +pub(crate) fn assert_transport(config: &impl std::fmt::Debug, custom: bool) { + // Quinn exposes these settings only through Debug. + let expected = if custom { + custom_transport().0 + } else { + quinn::TransportConfig::default() + }; + let expected = format!("{expected:?}"); + let config = format!("{config:?}"); + assert!(config.contains(&expected), "{config}"); +} + +fn finish(builder: ClientBuilder, path: u8) -> Client { + match path { + 0 => builder.with_system_roots(), + 1 => builder.with_server_certificate_hashes(vec![]), + 2 => builder.with_server_certificates(vec![]), + 3 => builder.dangerous().with_no_certificate_verification(), + _ => unreachable!(), + } + .unwrap() +} + +/// Start a connection synchronously so the congestion controller is built +/// before any network or TLS result. +fn start_connecting(client: &Client) -> quinn::Connecting { + client + .endpoints + .lock() + .unwrap() + .connect( + client.config.clone(), + "127.0.0.1:9".parse().unwrap(), + "localhost", + ) + .unwrap() +} + +#[tokio::test] +async fn builder_transport_reaches_every_client_config() { + for path in 0..4 { + let client = finish(builder(), path); + assert_transport(&client.config, false); + + for transport_first in [false, true] { + for algorithm in [ + CongestionControl::Default, + CongestionControl::LowLatency, + CongestionControl::Throughput, + ] { + let overridden = !matches!(algorithm, CongestionControl::Default); + let (transport, built) = custom_transport(); + let builder = builder(); + let builder = if transport_first { + builder + .with_transport_config(transport) + .with_congestion_control(algorithm) + } else { + builder + .with_congestion_control(algorithm) + .with_transport_config(transport) + }; + let client = finish(builder, path); + assert_transport(&client.config, true); + + // The system-root path needs no publicly trusted test server. + let connecting = start_connecting(&client); + assert_eq!(built.load(Ordering::SeqCst), usize::from(!overridden)); + drop(connecting); + client.close(0u32.into(), b"done"); + } + } + } +} + +#[tokio::test] +async fn default_congestion_control_removes_only_the_overlay() { + let (transport, built) = custom_transport(); + let client = builder() + .with_transport_config(transport) + .with_congestion_control(CongestionControl::LowLatency) + .with_congestion_control(CongestionControl::Default) + .dangerous() + .with_no_certificate_verification() + .unwrap(); + assert_transport(&client.config, true); + let connecting = start_connecting(&client); + assert_eq!(built.load(Ordering::SeqCst), 1); + drop(connecting); + client.close(0u32.into(), b"done"); +} + +fn bound(client: &Client) -> Vec { + client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .map(|endpoint| endpoint.local_addr().unwrap()) + .collect() +} + +fn url(remote: SocketAddr) -> url::Url { + url::Url::parse(&format!("https://{remote}/")).unwrap() +} + +#[test] +fn construction_needs_no_runtime_or_socket() { + assert!(bound(&client()).is_empty()); +} + +#[test] +fn ipv6_socket_is_v6_only() { + let socket = bind_socket("[::]:0".parse().unwrap()).unwrap(); + assert!(socket2::SockRef::from(&socket).only_v6().unwrap()); +} + +#[tokio::test] +async fn default_ipv4_socket() { + let client = client(); + let connect = client.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + assert!(bound(&client)[0].is_ipv4()); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_prevents_binding_either_family() { + let client = client(); + let clone = client.clone(); + let pending = clone.connect(url("127.0.0.1:443".parse().unwrap())); + client.close(0u32.into(), b""); + assert!(pending.await.is_err()); + for host in ["127.0.0.1", "[::1]", "localhost", "[::ffff:127.0.0.1]"] { + assert!(clone + .connect(url::Url::parse(&format!("https://{host}:443/")).unwrap()) + .await + .is_err()); + } + assert!(bound(&client).is_empty()); + client.wait_idle().await; +} + +#[tokio::test] +async fn close_interrupts_connect_and_prevents_second_family() { + let client = client(); + let clone = client.clone(); + let connect = clone.connect(url("127.0.0.1:443".parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + let before = bound(&client); + client.close(0u32.into(), b""); + assert!(connect.await.is_err()); + assert!(client + .connect(url("[::1]:443".parse().unwrap())) + .await + .is_err()); + assert_eq!(bound(&client), before); + clone.wait_idle().await; +} + +#[test] +fn missing_runtime_is_a_connect_error_and_allows_later_connections() { + // Poll an IP-literal connect without a runtime. Construction succeeds, + // while the lazy endpoint reports its setup error instead of panicking. + let client = client(); + let error = futures::executor::block_on(client.connect(url("127.0.0.1:443".parse().unwrap()))) + .unwrap_err(); + assert!(matches!(error, ClientError::IoError(_))); + assert!(bound(&client).is_empty()); + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + runtime.block_on(async { + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (_session, _peer) = pair(&client, &mut server, remote).await; + client.close(0u32.into(), b""); + client.wait_idle().await; + }); +} + +#[tokio::test] +async fn caller_endpoint_keeps_its_address_and_remote_handling() { + let endpoint = backend::Endpoint::client("127.0.0.1:0".parse().unwrap()).unwrap(); + let addr = endpoint.local_addr().unwrap(); + let client = Client::new(endpoint, client().config); + let error = client + .connect(url("[::ffff:127.0.0.1]:443".parse().unwrap())) + .await + .unwrap_err(); + assert!(matches!( + error, + ClientError::QuinnError(backend::ConnectError::InvalidRemoteAddress(_)) + )); + assert_eq!(bound(&client), vec![addr]); + client.close(0u32.into(), b""); + client.wait_idle().await; +} + +fn server(addr: SocketAddr) -> crate::Server { + let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); + crate::server::tests::builder() + .with_addr(addr) + .with_certificate( + vec![cert.cert.der().clone()], + rustls::pki_types::PrivatePkcs8KeyDer::from(cert.signing_key.serialize_der()).into(), + ) + .unwrap() +} + +async fn pair( + client: &Client, + server: &mut crate::Server, + remote: SocketAddr, +) -> (Session, Session) { + tokio::time::timeout(std::time::Duration::from_secs(10), async { + let (client, server) = tokio::join!(client.connect(url(remote)), async { + server.accept().await.unwrap().ok().await.unwrap() + },); + (client.unwrap(), server) + }) + .await + .expect("loopback connection timed out") +} + +#[tokio::test] +async fn families_mapped_addresses_and_clones_share_endpoints() { + let client = client(); + let clone = client.clone(); + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + let addr4 = ipv4.local_addr().unwrap(); + let addr6 = ipv6.local_addr().unwrap(); + let (session4, peer4) = pair(&client, &mut ipv4, addr4).await; + let first = bound(&client); + assert_eq!(first.len(), 1); + assert!(first[0].is_ipv4()); + let (session6, peer6) = pair(&clone, &mut ipv6, addr6).await; + let both = bound(&client); + assert_eq!(both.len(), 2); + assert!(both[1].is_ipv6()); + assert_eq!(both[0], first[0]); + let mapped = SocketAddr::new( + std::net::Ipv4Addr::LOCALHOST.to_ipv6_mapped().into(), + addr4.port(), + ); + let (session_mapped, peer_mapped) = pair(&clone, &mut ipv4, mapped).await; + assert!(session_mapped.remote_address().is_ipv4()); + assert_eq!(peer4.remote_address(), peer_mapped.remote_address()); + assert_eq!(bound(&clone), both); + + client.close(42u32.into(), b"all endpoints"); + clone.close(43u32.into(), b"second close"); + for session in [&session4, &session6, &session_mapped] { + let error = backend::Connection::closed(session).await; + assert!( + matches!(error, backend::ConnectionError::LocallyClosed), + "{error:?}" + ); + } + for peer in [&peer4, &peer6, &peer_mapped] { + backend::Connection::closed(peer).await; + } + tokio::join!(client.wait_idle(), clone.wait_idle()); + assert!(client + .endpoints + .lock() + .unwrap() + .binding + .endpoints() + .all(|endpoint| endpoint.open_connections() == 0)); + drop((session4, session6, session_mapped)); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn simultaneous_first_connections_share_one_socket() { + let client = client(); + let clone = client.clone(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (first, second, peers) = tokio::time::timeout(std::time::Duration::from_secs(10), async { + tokio::join!( + client.connect(url(remote)), + clone.connect(url(remote)), + async { + let first = server.accept().await.unwrap().ok().await.unwrap(); + let second = server.accept().await.unwrap().ok().await.unwrap(); + (first, second) + } + ) + }) + .await + .expect("concurrent loopback connections timed out"); + let first = first.unwrap(); + let second = second.unwrap(); + assert_eq!(bound(&client).len(), 1); + assert_eq!(peers.0.remote_address(), peers.1.remote_address()); + client.close(0u32.into(), b""); + client.wait_idle().await; + drop((first, second, peers)); +} + +#[tokio::test] +async fn dropping_native_client_keeps_returned_session_usable() { + let client = client(); + let mut server = server("127.0.0.1:0".parse().unwrap()); + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, &mut server, remote).await; + drop(client); + session + .send_datagram(bytes::Bytes::from_static(b"still open")) + .unwrap(); + let received = tokio::time::timeout(std::time::Duration::from_secs(10), peer.read_datagram()) + .await + .expect("datagram after client drop timed out") + .unwrap(); + assert_eq!(received.as_ref(), b"still open"); + server.close(0u32.into(), b""); + backend::Connection::closed(&session).await; + server.wait_idle().await; +} + +#[test] +fn construction_runtime_survives_first_connect_on_transient_runtime() { + let owner = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let client = { + let _guard = owner.enter(); + client() + }; + let transient = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + transient.block_on(async { + for addr in ["127.0.0.1:443", "[::1]:443"] { + let connect = client.connect(url(addr.parse().unwrap())); + tokio::pin!(connect); + assert!(futures::poll!(&mut connect).is_pending()); + } + }); + drop(transient); + owner.block_on(async { + let mut ipv4 = server("127.0.0.1:0".parse().unwrap()); + let mut ipv6 = server("[::1]:0".parse().unwrap()); + for server in [&mut ipv4, &mut ipv6] { + let remote = server.local_addr().unwrap(); + let (session, peer) = pair(&client, server, remote).await; + session + .send_datagram(bytes::Bytes::from_static(b"owner runtime")) + .unwrap(); + assert_eq!( + peer.read_datagram().await.unwrap().as_ref(), + b"owner runtime" + ); + } + client.close(0u32.into(), b""); + client.wait_idle().await; + }); +} diff --git a/rs/web-transport-quinn/src/error.rs b/rs/web-transport-quinn/src/error.rs index 11880256..a1175948 100644 --- a/rs/web-transport-quinn/src/error.rs +++ b/rs/web-transport-quinn/src/error.rs @@ -7,6 +7,9 @@ use crate::{ConnectError, SettingsError}; /// An error returned when connecting to a WebTransport endpoint. #[derive(Error, Debug, Clone)] pub enum ClientError { + #[error("failed to bind client endpoint: {0}")] + IoError(Arc), + #[error("unexpected end of stream")] UnexpectedEnd, diff --git a/rs/web-transport-quinn/src/server.rs b/rs/web-transport-quinn/src/server.rs index 2decf9ed..d766aa4c 100644 --- a/rs/web-transport-quinn/src/server.rs +++ b/rs/web-transport-quinn/src/server.rs @@ -253,7 +253,7 @@ impl core::ops::Deref for Request { } #[cfg(all(test, any(feature = "aws-lc-rs", feature = "ring")))] -mod tests { +pub(crate) mod tests { use super::*; use crate::client::tests::{assert_transport, custom_transport}; use rustls::pki_types::PrivatePkcs8KeyDer; @@ -269,7 +269,7 @@ mod tests { /// `ServerBuilder::new` goes through [crypto::default_provider], which panics when /// both backends are compiled in and no process-wide default is installed. Pick one /// here rather than mutating global state from a test. - fn builder() -> ServerBuilder { + pub(crate) fn builder() -> ServerBuilder { #[cfg(feature = "aws-lc-rs")] let provider = Arc::new(rustls::crypto::aws_lc_rs::default_provider()); #[cfg(all(feature = "ring", not(feature = "aws-lc-rs")))] diff --git a/rs/web-transport/src/quinn.rs b/rs/web-transport/src/quinn.rs index 3ff5b046..d68afc47 100644 --- a/rs/web-transport/src/quinn.rs +++ b/rs/web-transport/src/quinn.rs @@ -7,7 +7,7 @@ pub use web_transport_quinn as quinn; pub use web_transport_quinn::CongestionControl; /// Create a [Client] that can be used to dial multiple [Session]s. -/// Build a client before cloning it to share its endpoint and configuration. +/// Build a client before cloning it to share its endpoints and configuration. #[derive(Default)] pub struct ClientBuilder { inner: quinn::ClientBuilder,