diff --git a/doc/bin/relay/auth.md b/doc/bin/relay/auth.md index 93f1c4e8605..6a4da1b9994 100644 --- a/doc/bin/relay/auth.md +++ b/doc/bin/relay/auth.md @@ -36,7 +36,7 @@ The fields are documented on `moq_auth::Request` and `moq_auth::Grant` ([Rust](/lib/rs/moq-auth), [TypeScript](/lib/js/auth)). `server_name` is the SNI that selected the relay's certificate. It is absent -over plaintext, on the `tls://` TCP listener, and for an IP-literal dial, and +over plaintext and for an IP-literal dial, and never filled from a `Host` header or a WebTransport authority. The client still picks it among the names the relay serves, so a hostname-scoped server binds the tenant to the credential, not to `server_name` alone. @@ -304,8 +304,8 @@ the relay reports it in the request and enforces the grant it gets back. `moq auth serve` grants a certificate only what `--mtls-publish` and `--mtls-subscribe` name, nothing by default. Public rules ignore certificates, so `--auth-public` with a client CA refuses to start. Only the QUIC listener -verifies client certificates, so a relay without `listen.bind` refuses -`listen.tls.root` too. +and a `tls://` TCP listener (`listen.tcp.tls`) verify client certificates, so a +relay with neither refuses `listen.tls.root` too. Cluster peers are admitted the same way, so a mesh runs `moq auth serve --mtls-publish '**' --mtls-subscribe '**' --mtls-link peer` (or a @@ -438,5 +438,6 @@ identity. The Unix socket is created mode `0666`, so gate it with a restrictive parent directory or an allowlist. `listen.tcp.tls = true` serves the TCP listener over TLS instead, for `tls://` -dials such as [cluster links](/bin/relay/cluster#tls-links). It asks for no -client certificate, so a peer on it presents a token, not mTLS. +dials such as [cluster links](/bin/relay/cluster#tls-links). It asks for a +client certificate as the QUIC listener does, under the same `listen.tls.root`, +so a peer on it presents a certificate or a token. diff --git a/doc/bin/relay/cluster.md b/doc/bin/relay/cluster.md index 189032036ad..1f191c3644b 100644 --- a/doc/bin/relay/cluster.md +++ b/doc/bin/relay/cluster.md @@ -134,9 +134,10 @@ topology check; whatever generates the peer list applies the layout's rules. A link inside a region is rarely congested, so it can skip QUIC: a `tls://` peer URL dials qmux over TLS on TCP, verified with the same `connect.tls` settings as any other dial. The accepting relay serves it from its TCP -listener with TLS on. That listener asks for no client certificate, so a peer -on it authenticates with a token (`cluster.token`, `token`, or `?jwt=`), not -mTLS. +listener with TLS on. That listener applies the same client-certificate policy +as the QUIC listener, so a peer on it authenticates with its `connect.tls` +certificate against `listen.tls.root`, or with a token (`cluster.token`, +`token`, or `?jwt=`). ```toml # core.toml diff --git a/doc/bin/relay/config.md b/doc/bin/relay/config.md index 621ea141b1f..de017e00f63 100644 --- a/doc/bin/relay/config.md +++ b/doc/bin/relay/config.md @@ -28,11 +28,11 @@ secret = "/etc/moq/reset.key" # File holding 64 hex chars. Omit for a ran cert = "cert.pem" # Certificate chain and key. Reloaded on change. key = "key.pem" generate = ["localhost"] # Or: a self-signed cert for development. -root = ["peer-ca.pem"] # Optional: CAs for client certs (mTLS), reported to the auth server. Needs QUIC. +root = ["peer-ca.pem"] # Optional: CAs for client certs (mTLS), reported to the auth server. Needs QUIC or tcp.tls. [listen.tcp] # Plaintext qmux over TCP for trusted local workers. bind = "127.0.0.1:4444" -# tls = true # Or: qmux over TLS (tls://) with the listen certificate, no client certs. +# tls = true # Or: qmux over TLS (tls://) with the listen certificate and client-cert policy. [listen.unix] # Plaintext qmux over a Unix socket, gated by peer credentials. bind = "/run/moq/internal.sock" diff --git a/quest/m2/README.md b/quest/m2/README.md index 8ae0ce6a480..8fe46b3e7f8 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -160,7 +160,6 @@ lite-07; bitrate enforcement is not part of this release. - [#3199](/quest/m2/3199-moq-uring-remove-sq-indirection-and-per-enter-ring-fd.md) - moq-uring: remove SQ indirection and per-enter ring fd lookup, if a profile shows the win - [#3200](/quest/m2/3200-moq-uring-batch-completion-wakeups-with-min-timeout.md) - moq-uring: batch completion wakeups with MIN_TIMEOUT - [Browser media report](/quest/m2/rs2ts-browser-report.md) - compare Rust-backed and handwritten @moq/net through full media workloads, supplementing the pre-adoption comparisons -- [mTLS on tls://](/quest/m2/tls-listener-mtls.md) - a `tls://` listener can identify a cluster peer by its client certificate - [Link quality](/quest/m2/link-quality.md) - a radio link's cost follows its measured quality without flapping routes - [io_uring handle lifecycle](/quest/m2/uring-lifecycle.md) - handles fail once their worker is gone, close on last drop like tokio, and surface a close behind unread data - [io_uring socket teardown](/quest/m2/uring-socket-teardown.md) - socket churn never exhausts buffer-group ids, and a failed receive cancel surfaces diff --git a/quest/m2/one-port/tcp-demux.md b/quest/m2/one-port/tcp-demux.md index c3f8da62751..a86ee9225da 100644 --- a/quest/m2/one-port/tcp-demux.md +++ b/quest/m2/one-port/tcp-demux.md @@ -37,11 +37,14 @@ ALPN to the router. A connection with no ALPN keeps the decrypted-byte sniff, since RTMPS clients negotiate none (`moq-rtmp` documents an empty ALPN list): `0x03` is RTMP, anything else is HTTP. -Decided 2026-10-08: under one acceptor, the client-certificate request from -[mTLS on tls://](/quest/m2/tls-listener-mtls.md) is optional (never required) -and sits on the moq-ALPN TLS config only, so browsers and RTMPS clients on -the same port are never asked for one. Whichever lands second keeps `tls://` -client-certificate verification and the `MtlsUnsupported` lift working. +Decided 2026-10-08: under one acceptor, the client-certificate request the +`tls://` listener makes (since +[moq-dev/moq#7358](https://github.com/moq-dev/moq/pull/7358)) sits on the moq-ALPN TLS +config only, so browsers and RTMPS clients on the same port are never asked +for one. ✅ Decided 2026-10-30 (maintainer): it is optional under CA roots and +required under pinned `tls.peers`, as on QUIC, so pinned mode still means +the listener knows who connected. Keep `tls://` client-certificate +verification and the `MtlsUnsupported` lift working. Keep the accepted `TcpStream`'s handle reachable after it is boxed: the relay already captures socket stats at accept for qmux (`SocketStats` in @@ -55,5 +58,4 @@ right arm. ## Related - [UDP demux](/quest/m2/p2p/udp-demux.md) - the UDP half -- [mTLS on tls://](/quest/m2/tls-listener-mtls.md) - client-certificate verification on the `tls://` arm this acceptor carries - [WebSocket bitrate caps](/quest/m2/rate-websocket.md) - sizes the kernel buffers of the socket this acceptor boxes diff --git a/quest/m2/perf/stream/driver.md b/quest/m2/perf/stream/driver.md index dda73276d49..6ff72686b6a 100644 --- a/quest/m2/perf/stream/driver.md +++ b/quest/m2/perf/stream/driver.md @@ -51,5 +51,4 @@ binding that starts a qmux session. The Impact bullet says how to migrate. ## Related - [qmux byte counters](/quest/m2/wire-stats/qmux.md) - refreshes `Wire` counters on qmux's read/write path -- [tls:// peers authenticate by certificate](/quest/m2/tls-listener-mtls.md) - changes `qmux::tls::Server::accept` - [qmux leases receive memory](/quest/m2/qmux-receive-lease.md) - changes qmux's `RecvStream` and window defaults diff --git a/quest/m2/tls-listener-mtls.md b/quest/m2/tls-listener-mtls.md deleted file mode 100644 index bf89741cb99..00000000000 --- a/quest/m2/tls-listener-mtls.md +++ /dev/null @@ -1,46 +0,0 @@ -# [M] tls:// peers can authenticate by certificate - -## Goal - -A relay's `tls://` (qmux over TLS on TCP) listener can identify a cluster -peer by its client certificate, as the QUIC listener does, so an upstream -link doesn't need a token. - -## Plan - -#4816 added `tls://` with no client certificate request, so peers present a -token. `moq_tokio::tcp::Listener::with_tls` (`rs/moq-tokio/src/tcp.rs`) -already takes an `Arc`, so requesting a certificate is -config. The real gap is getting the peer certificate out afterwards: -`qmux::tls::Server::accept` finishes the handshake and returns a `Session` -without it. The same accessor also returns the SNI the client sent, so a -`tls://` session reports it as QUIC sessions do (from -[moq-dev/moq#6509](https://github.com/moq-dev/moq/pull/6509), decided -2026-10-23). - -Decided by the maintainer (2026-10-06): the peer certificate comes from a -new accessor on qmux's TLS session. Rejected: owning the TLS accept outside -qmux, or leaving it open. Decided 2026-10-20 (maintainer): no new work in -moq-dev/web-transport, so the accessor goes on the in-tree qmux crate (`rs/qmux`). - -Keep the token path working on the same listener, and keep pinned peers and -CA roots meaning the same as on QUIC. - -A stream-only server refuses a listener client CA or pinned `tls.peers` (the -`MtlsUnsupported` case in `rs/moq-tokio/src/server.rs`), since nothing checks -them there. Once a `tls://` listener verifies the certificate, this quest lifts -that refusal for a server with a `tls://` listener and updates the refusal's -test. - -Decided 2026-10-08: request the certificate optionally, never require it, -and only on the moq-ALPN TLS config, so once -[the TCP acceptor](/quest/m2/one-port/tcp-demux.md) shares the port with -HTTP and RTMPS, those clients are never asked for one. Whichever lands second -keeps both working. - -Public API: relay config may gain a listener option. Wire: none. - -## Related - -- [TCP acceptor](/quest/m2/one-port/tcp-demux.md) - carries `tls://` on the shared TCP port, where the certificate request stays on the moq ALPN -- [qmux drives its own socket](/quest/m2/perf/stream/driver.md) - replaces qmux's reader and writer tasks and its constructors' return type diff --git a/rs/moq-relay/MIGRATE.md b/rs/moq-relay/MIGRATE.md index bd005003de3..0f943eedb71 100644 --- a/rs/moq-relay/MIGRATE.md +++ b/rs/moq-relay/MIGRATE.md @@ -5,6 +5,13 @@ Rolling a cluster across a release also follows the [upgrade guide](https://moq. ## Unreleased +### `tls://` links verify client certificates ([#7358](https://github.com/moq-dev/moq/pull/7358)) + +The `listen.tcp.tls` listener applies `listen.tls.root` and pinned `tls.peers` as the QUIC listener does, so a stream-only relay with that listener no longer refuses them. +A peer dialing `tls://` with a `connect.tls` certificate now presents it, and a certificate presented beside a JWT is refused as it is on QUIC: give such a peer `cluster.token` or a certificate, not both. +With pinned `tls.peers`, a `tls://` peer must present a pinned certificate; a token alone is refused. +The auth request's `server_name` reports the SNI on `tls://` too. + ### Paths with an empty segment are refused ([#6648](https://github.com/moq-dev/moq/pull/6648)) A session dialed at a URL path with a trailing or doubled `/`, such as `/room/` or `/room//cam`, is refused as a bad request instead of collapsed; dial `/room`. An auth grant whose `root`, `mounts`, `tier`, or `label` has one is refused as forbidden, and `stats.prefix`, `stats.node`, and `--cluster-tier` with one refuse to start. @@ -145,8 +152,8 @@ Set `load_balancer`, or `lb_id` alone (nonce 8), or `lb_id` with `lb_nonce`. ### Settings no listener reads stop startup (#5006) -A stream-only relay (TCP or Unix, no `--listen`) refuses the QUIC-only `--listen-preferred-v4` / `-v6`, `--listen-quic-lb-id`, and pinned `tls.peers`. -It also refuses `--listen-tls-cert`, `-key`, or `-generate` unless `--listen-tcp-tls` serves it, and `listen.tls.root`, since only QUIC verifies a client CA. +A stream-only relay (TCP or Unix, no `--listen`) refuses the QUIC-only `--listen-preferred-v4` / `-v6` and `--listen-quic-lb-id`. +It also refuses `--listen-tls-cert`, `-key`, or `-generate` unless `--listen-tcp-tls` serves it, and `listen.tls.root` or pinned `tls.peers` unless `--listen-tcp-tls` verifies them. `--listen-unix-allow-*` needs `--listen-unix-bind`, and `web.https.cert`, `key`, and `root` need `web.https.listen`. Each used to be ignored: drop it, or add the listener it configures. diff --git a/rs/moq-relay/tests/integration/public_rules.rs b/rs/moq-relay/tests/integration/public_rules.rs index 1136482ac5a..ad423872858 100644 --- a/rs/moq-relay/tests/integration/public_rules.rs +++ b/rs/moq-relay/tests/integration/public_rules.rs @@ -283,11 +283,11 @@ async fn a_client_ca_needs_an_auth_server() { } } -/// A stream-only relay has no QUIC listener to verify a client CA, and its stream -/// listeners never ask for a client certificate, so the CA is refused rather than -/// silently ignored. +/// A relay with only a plaintext TCP listener has no QUIC or `tls://` listener to +/// verify a client CA, and plaintext TCP never asks for a client certificate, so +/// the CA is refused rather than silently ignored. #[tokio::test] -async fn a_client_ca_needs_a_quic_listener() { +async fn a_client_ca_needs_a_verifying_listener() { let mut config = moq_relay::Config::default(); config.auth.url = Some("http://127.0.0.1:4440/".parse().expect("parse url")); config.listen.tcp.bind = Some("127.0.0.1:0".parse().expect("parse addr")); diff --git a/rs/moq-tokio/MIGRATE.md b/rs/moq-tokio/MIGRATE.md index da5b89a8310..ffbe2a04c7f 100644 --- a/rs/moq-tokio/MIGRATE.md +++ b/rs/moq-tokio/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `tls://` listeners verify client certificates ([#7358](https://github.com/moq-dev/moq/pull/7358)) + +A `listen.tcp.tls` listener serves the listen policy's client-certificate verifier, so `--listen-tls-root` and pinned `tls.peers` apply to it as they do to QUIC, and a stream-only server with that listener accepts them instead of returning `Error::MtlsUnsupported`. +Its `Request::peer_identity`, `server_name`, and `authority` report the client chain and SNI, which were always `None`. +Under pinned peers, a `tls://` client must present a pinned certificate, as on QUIC. + ### `Connection::allocator` ([#6751](https://github.com/moq-dev/moq/pull/6751)) `Connection::bandwidth()` is `Connection::allocator()`, mirroring `moq_net::Session::allocator()`. @@ -138,13 +144,13 @@ Set `load_balancer`, or `lb_id` alone (nonce 8), or `lb_id` with `lb_nonce`. ### Settings no listener reads stop startup (#5006) -A stream-only listener (TCP or Unix, no QUIC bind) refuses the QUIC-only `preferred_v4` / `preferred_v6`, `lb_id`, pinned `tls.peers`, and a TLS cert, key, or `generate` unless `tcp.tls` serves it. +A stream-only listener (TCP or Unix, no QUIC bind) refuses the QUIC-only `preferred_v4` / `preferred_v6` and `lb_id`, and a TLS cert, key, or `generate` unless `tcp.tls` serves it. `unix.allow_*` needs `unix.bind`. Each used to be ignored: drop it, or add the listener it configures. -### A client CA needs a QUIC listener +### A client CA needs a QUIC or `tls://` listener -A stream-only listener with `listen.tls.root` returns `Error::MtlsUnsupported` instead of silently verifying nothing. +A stream-only listener with `listen.tls.root` or pinned `tls.peers` returns `Error::MtlsUnsupported` unless `tcp.tls` verifies them, instead of silently verifying nothing. ### `Transport` names WebTransport diff --git a/rs/moq-tokio/src/error.rs b/rs/moq-tokio/src/error.rs index 64d8fd61c2d..55d4b2f1c85 100644 --- a/rs/moq-tokio/src/error.rs +++ b/rs/moq-tokio/src/error.rs @@ -166,9 +166,9 @@ pub enum Error { #[error("Iroh support is not enabled")] IrohDisabled, - /// A client CA or pinned peers were configured on a stream-only server, which has no QUIC listener to verify them. + /// A client CA or pinned peers were configured on a server with no QUIC or `tls://` listener to verify them. #[error( - "mTLS (--listen-tls-root or tls.peers) needs a QUIC listener (--listen); the TCP and Unix listeners never ask for a client certificate" + "mTLS (--listen-tls-root or tls.peers) needs a QUIC listener (--listen) or a TLS TCP listener (--listen-tcp-tls); the plaintext TCP and Unix listeners never ask for a client certificate" )] MtlsUnsupported, diff --git a/rs/moq-tokio/src/server.rs b/rs/moq-tokio/src/server.rs index 41597e4f74b..ae5f188d35c 100644 --- a/rs/moq-tokio/src/server.rs +++ b/rs/moq-tokio/src/server.rs @@ -322,14 +322,19 @@ impl Server { return Err(Error::NoBackend("--listen-reset-secret requires the quic feature")); } - // Only the QUIC backend verifies client certificates; the qmux listeners - // (tcp/unix/websocket) never ask for one, even over `tls://`, so a - // stream-only server would ignore the CA or pinned peers. A caller opening + // The QUIC backend and a `tls://` listener verify client certificates; the + // other qmux listeners (plaintext tcp/unix/websocket) never ask for one, so a + // server with neither would ignore the CA or pinned peers. A caller opening // only the streams owns QUIC elsewhere. - if parts.quic() && !build_quic && (!config.tls.root.is_empty() || config.tls.peers.is_some()) { + #[cfg(feature = "tcp")] + let tcp_tls = build_streams && config.tcp.bind.is_some() && config.tcp.tls == Some(true); + #[cfg(not(feature = "tcp"))] + let tcp_tls = false; + let client_auth = !config.tls.root.is_empty() || config.tls.peers.is_some(); + if parts.quic() && !build_quic && !tcp_tls && client_auth { return Err(Error::MtlsUnsupported); } - if parts.quic() && !config.tls.root.is_empty() { + if parts.quic() && !tcp_tls && !config.tls.root.is_empty() { #[cfg(not(feature = "quic"))] return Err(Error::NoBackend("--listen-tls-root requires the quic feature")); } @@ -341,10 +346,7 @@ impl Server { } #[allow(unused_mut, reason = "only the stream listener features set it")] let mut needed = cfg!(feature = "quic") && build_quic; - #[cfg(feature = "tcp")] - { - needed |= build_streams && config.tcp.bind.is_some() && config.tcp.tls == Some(true); - } + needed |= tcp_tls; match (tls, needed) { // Mirrors `validate_stream_only`: an identity no listener serves is refused. // A streams-only caller owns QUIC elsewhere, which serves it. @@ -359,7 +361,7 @@ impl Server { } }; #[cfg(feature = "_certs")] - if parts.quic() && !build_quic && tls.as_ref().is_some_and(crate::tls::Serve::client_auth) { + if parts.quic() && !build_quic && !tcp_tls && tls.as_ref().is_some_and(crate::tls::Serve::client_auth) { return Err(Error::MtlsUnsupported); } @@ -1211,8 +1213,8 @@ struct TcpTls { #[cfg(all(feature = "tcp", feature = "_certs"))] impl TcpTls { - // qmux does not report peer certificates; keep token-only authentication while - // using the same certificate resolver as the QUIC listeners. + // The same certificate resolver and client-certificate policy as the QUIC + // listeners, so CA roots and pinned peers mean the same on `tls://`. fn new( _config: &crate::listen::Config, versions: &moq_net::Versions, @@ -1224,7 +1226,7 @@ impl TcpTls { .map(|alpn| format!("{}{alpn}", qmux::Version::QMux01.prefix()).into_bytes()) .collect(); Ok(Self { - config: serve.anonymous(alpn)?, + config: serve.config(alpn)?, }) } } @@ -1471,9 +1473,16 @@ fn spawn_tcp_loop( handshakes.spawn(crate::panic::isolate(async move { match within(deadline, pending).await { Ok((session, remote)) => { + // Set only on `tls://`: the SNI, and the client certificate its + // verifier already accepted. + let server_name = session.server_name().map(str::to_owned); + let identity = session + .peer_certificates() + .map(|chain| crate::tls::PeerIdentity::from_chain(chain.to_vec())); let link = Link { remote: Some(remote), local, + server_name: server_name.clone(), ..Default::default() }; stream_request( @@ -1483,6 +1492,9 @@ fn spawn_tcp_loop( Incoming { transport: Transport::Tcp, link, + // Like raw QUIC, the SNI is the only authority a `tls://` client offers. + authority: server_name, + identity, deadline, active, }, @@ -1530,6 +1542,8 @@ fn spawn_unix_loop( Incoming { transport: Transport::Unix, link: Link::default(), + authority: None, + identity: None, deadline, active, }, @@ -1548,6 +1562,8 @@ fn spawn_unix_loop( struct Incoming { transport: Transport, link: Link, + authority: Option, + identity: Option, deadline: Option, active: moq_net::server::Active, } @@ -1563,6 +1579,8 @@ async fn stream_request( let Incoming { transport, link, + authority, + identity, deadline, active, } = incoming; @@ -1571,8 +1589,8 @@ async fn stream_request( let request = Request { transport, url: None, - authority: None, - identity: None, + authority, + identity, link, handshake: Box::new(request), #[cfg(feature = "quic")] @@ -1628,8 +1646,9 @@ pub struct Request { transport: Transport, /// The request URL for WebTransport/WebSocket. `None` for bindings whose path rides SETUP. url: Option, - /// The authority the client dialed, when it offered one: the TLS SNI on raw QUIC, the CONNECT - /// authority on WebTransport. `None` on the URL-less stream bindings and on iroh. + /// The authority the client dialed, when it offered one: the TLS SNI on raw QUIC and + /// `tls://`, the CONNECT authority on WebTransport. `None` on the plaintext stream + /// bindings and on iroh. authority: Option, /// The peer's validated mTLS identity, captured at the transport handshake (before /// the MoQ SETUP), when the backend supports it. @@ -1752,10 +1771,11 @@ impl Request { } /// The host authority the client dialed, or `None` when the client offered none or the - /// transport carries no host (iroh, stream bindings). + /// transport carries no host (iroh, plaintext stream bindings). On raw QUIC and + /// `tls://` it is the TLS SNI. /// /// Reported as offered: rustls sends no SNI for an IP-literal dial (RFC 6066), - /// so the QUIC backend reports `None`. + /// so the QUIC and `tls://` listeners report `None`. /// /// Not the moq-net IETF SETUP `Authority` parameter. Client-asserted and not authenticated, /// so authorize on the token or [`Self::peer_identity`] rather than on this value. @@ -1858,8 +1878,8 @@ impl Request { /// The client certificate chain the peer presented, if any, validated /// against a configured [`crate::tls::Listen::root`] during the handshake. /// - /// Captured at the transport handshake (before the SETUP). The QUIC backend supports mTLS; - /// other transports always return `None`. Use it to grant + /// Captured at the transport handshake (before the SETUP). The QUIC backend and the + /// `tls://` TCP listener support mTLS; other transports always return `None`. Use it to grant /// elevated access or to close the session once the certificate expires (see /// [`crate::tls::PeerIdentity::expiry`]). pub fn peer_identity(&self) -> Option { @@ -2916,15 +2936,21 @@ mod tests { drop((session, connection)); } - /// A client CA or pinned peers on a stream-only server are refused: no QUIC - /// listener would verify them, and the stream listeners never ask for a client certificate. + /// A client CA or pinned peers on a stream-only server are refused unless a + /// `tls://` listener verifies them: the plaintext stream listeners never ask + /// for a client certificate. #[cfg(feature = "tcp")] #[tokio::test] - async fn client_auth_without_a_quic_listener_is_rejected() { - type Set = fn(&mut crate::listen::Config); + async fn client_auth_without_a_tls_listener_is_rejected() { + let dir = tempfile::tempdir().unwrap(); + let ca = dir.path().join("ca.pem"); + let cert = rcgen::generate_simple_self_signed(["ca".to_string()]).unwrap().cert; + std::fs::write(&ca, cert.pem()).unwrap(); + + type Set = Box; let cases: [Set; 2] = [ - |c| c.tls.root = vec!["ca.pem".into()], - |c| c.tls.peers = Some(crate::tls::Peers::new()), + Box::new(move |c| c.tls.root = vec![ca.clone()]), + Box::new(|c| c.tls.peers = Some(crate::tls::Peers::new())), ]; for set in cases { let mut config = crate::listen::Config::default(); @@ -2936,7 +2962,17 @@ mod tests { Err(Error::MtlsUnsupported) )); // A worker group owning QUIC verifies it, so the streams alone accept it. - config.init_streams().expect("streams beside worker-owned QUIC"); + config.clone().init_streams().expect("streams beside worker-owned QUIC"); + + // A `tls://` listener verifies it. + config.tcp.tls = Some(true); + config.tls.generate = vec!["localhost".into()]; + #[cfg(feature = "_certs")] + config + .init(Default::default()) + .expect("tls:// verifies client certificates"); + #[cfg(not(feature = "_certs"))] + assert!(matches!(config.init(Default::default()), Err(Error::NoBackend(_)))); } } diff --git a/rs/moq-tokio/src/tcp.rs b/rs/moq-tokio/src/tcp.rs index 821421ff95e..2b396e2e051 100644 --- a/rs/moq-tokio/src/tcp.rs +++ b/rs/moq-tokio/src/tcp.rs @@ -4,7 +4,8 @@ //! Plaintext `tcp://` has no transport encryption and no authentication, so only //! use it on a trusted network (loopback, a private VPC interface, etc.). //! `tls://` encrypts and verifies the server with the same TLS settings as the -//! other transports, for links that need neither QUIC nor a WebSocket. +//! other transports, for links that need neither QUIC nor a WebSocket, and its +//! listener verifies client certificates as the QUIC listener does. //! //! Plaintext TCP has no TLS handshake, so the application protocol (the moq //! ALPN) is negotiated in-band; TLS negotiates it in its handshake. Either way @@ -41,8 +42,9 @@ pub struct Config { pub bind: Option, /// Serve TLS on the TCP listener with the listen certificate, for `tls://` - /// dials. It asks for no client certificate, so a peer on it authenticates - /// with a token rather than mTLS. + /// dials. It asks for a client certificate as the QUIC listener does: optionally + /// under CA roots, required under pinned peers, so a peer on it authenticates + /// with mTLS or a token. #[usage( long = "listen-tcp-tls", name = "listen-tcp-tls", @@ -366,37 +368,135 @@ mod tests { } /// A `tls://` dial carries qmux and the request target over TLS, and a client - /// that does not trust the certificate is refused. Client-certificate policy - /// meant for QUIC (CA roots or pinned peers) never makes the TCP listener ask - /// for one, so a token-only client still gets through. + /// that does not trust the certificate is refused. #[cfg(feature = "aws-lc-rs")] #[tokio::test] async fn tls_carries_qmux_and_refuses_an_untrusted_certificate() { + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); + let mut listen = crate::listen::Config::default(); + listen.tcp.bind = Some("127.0.0.1:0".parse().unwrap()); + listen.tcp.tls = Some(true); + listen.tls.generate = vec!["localhost".into()]; + tls_round_trip(listen).await; + } + + /// A `tls://` listener with no QUIC beside it verifies client certificates + /// as QUIC does: CA roots accept a chained certificate or a token-only client, + /// pinned peers require a pinned certificate. The request reports the chain + /// and the SNI. + #[cfg(feature = "aws-lc-rs")] + #[tokio::test] + async fn tls_verifies_a_client_certificate() { + use rcgen::{BasicConstraints, CertificateParams, ExtendedKeyUsagePurpose, IsCa, Issuer, KeyPair}; + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); let dir = tempfile::tempdir().unwrap(); - for pinned in [false, true] { + let write = |name: &str, contents: String| { + let path = dir.path().join(name); + std::fs::write(&path, contents).unwrap(); + path + }; + + let ca_key = KeyPair::generate().unwrap(); + let mut ca_params = CertificateParams::new(Vec::new()).unwrap(); + ca_params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained); + let ca_cert = ca_params.self_signed(&ca_key).unwrap(); + let issuer = Issuer::from_params(&ca_params, &ca_key); + let client_key = KeyPair::generate().unwrap(); + let mut client_params = CertificateParams::new(vec!["client.example".to_string()]).unwrap(); + client_params.extended_key_usages = vec![ExtendedKeyUsagePurpose::ClientAuth]; + let client_cert = client_params.signed_by(&client_key, &issuer).unwrap(); + let fingerprint = crate::tls::PeerIdentity::from_chain(vec![client_cert.der().clone()]) + .fingerprint() + .unwrap(); + let ca = write("ca.pem", ca_cert.pem()); + let cert = ( + write("client.pem", client_cert.pem()), + write("client.key", client_key.serialize_pem()), + ); + + let listen = |set: &dyn Fn(&mut crate::listen::Config)| { let mut listen = crate::listen::Config::default(); listen.tcp.bind = Some("127.0.0.1:0".parse().unwrap()); listen.tcp.tls = Some(true); listen.tls.generate = vec!["localhost".into()]; - // Client-certificate policy is refused without a QUIC listener to read it. - if !cfg!(feature = "quic") { - continue; - } - listen.bind = Some("127.0.0.1:0".parse().unwrap()); - match pinned { - true => listen.tls.peers = Some(crate::tls::Peers::new()), - false => { - let ca = dir.path().join("client-ca.pem"); - let cert = rcgen::generate_simple_self_signed(["client-ca".to_string()]) - .unwrap() - .cert; - std::fs::write(&ca, cert.pem()).unwrap(); - listen.tls.root = vec![ca]; - } - } - tls_round_trip(listen).await; + set(&mut listen); + listen + }; + + // CA roots: a certificate is optional, so the token path still works. + let server = listen(&|l| l.tls.root = vec![ca.clone()]); + let server = server.init(Default::default()).unwrap().listen().await.unwrap(); + let (server, identity, sni) = mtls_dial(server, Some(&cert)) + .await + .expect("CA-rooted certificate refused"); + let identity = identity.expect("the chain is reported"); + assert_eq!(identity.name().as_deref(), Some("client.example")); + assert_eq!(sni.as_deref(), Some("localhost")); + let (_server, identity, _) = mtls_dial(server, None).await.expect("token-only client refused"); + assert!(identity.is_none()); + + // Pinned peers: a certificate is required, and must be pinned. + let peers = crate::tls::Peers::new(); + peers.insert(&fingerprint).unwrap(); + let server = listen(&|l| l.tls.peers = Some(peers.clone())); + let server = server.init(Default::default()).unwrap().listen().await.unwrap(); + let (server, identity, _) = mtls_dial(server, Some(&cert)) + .await + .expect("pinned certificate refused"); + assert_eq!(identity.unwrap().fingerprint(), Some(fingerprint.clone())); + // A refused dial consumes its listener, so the token-only check gets its own. + let token_only = listen(&|l| l.tls.peers = Some(peers.clone())); + let token_only = token_only.init(Default::default()).unwrap().listen().await.unwrap(); + assert!( + mtls_dial(token_only, None).await.is_none(), + "a token-only client must be refused under pinned peers" + ); + peers.remove(&fingerprint).unwrap(); + assert!( + mtls_dial(server, Some(&cert)).await.is_none(), + "an unpinned certificate must be refused" + ); + } + + /// Dial `server` over `tls://localhost`, presenting `cert` (a PEM cert and key) + /// if given. Returns the server with the accepted request's identity and SNI, + /// or `None` when the dial is refused. + #[cfg(feature = "aws-lc-rs")] + async fn mtls_dial( + mut server: crate::Listener, + cert: Option<&(std::path::PathBuf, std::path::PathBuf)>, + ) -> Option<(crate::Listener, Option, Option)> { + let port = server.tcp_local_addr().unwrap().port(); + let url: Url = format!("tls://localhost:{port}/room?jwt=credential").parse().unwrap(); + let mut config = crate::connect::Config::default(); + config.tls.insecure = Some(true); + if let Some((cert, key)) = cert { + config.tls.cert = Some(cert.clone()); + config.tls.key = Some(key.clone()); } + let client = config.init(Default::default()).unwrap().with_reconnect(false); + + let accept = tokio::spawn(async move { + let request = server.accept().await.unwrap(); + let identity = request.peer_identity(); + let sni = request.server_name().map(str::to_owned); + assert_eq!(request.authority(), sni.as_deref()); + let session = request.ok().await.unwrap(); + (server, identity, sni, session) + }); + let dialed = tokio::time::timeout(Duration::from_secs(5), client.connect(url).established()) + .await + .expect("dial timed out"); + if dialed.is_err() { + accept.abort(); + return None; + } + let (server, identity, sni, _session) = tokio::time::timeout(Duration::from_secs(5), accept) + .await + .expect("accept timed out") + .expect("accept task panicked"); + Some((server, identity, sni)) } #[cfg(feature = "aws-lc-rs")] diff --git a/rs/moq-tokio/src/tls.rs b/rs/moq-tokio/src/tls.rs index 8e5b10e8965..f0e34daf576 100644 --- a/rs/moq-tokio/src/tls.rs +++ b/rs/moq-tokio/src/tls.rs @@ -1627,23 +1627,9 @@ impl Serve { /// Build a TLS policy with these ALPNs and the shared identity and client verifier. pub fn config(&self, alpn: Vec>) -> Result> { - self.build(alpn, true) - } - - #[cfg(feature = "_transport")] - pub(crate) fn client_auth(&self) -> bool { - self.0.verifier.is_some() - } - - #[cfg(feature = "tcp")] - pub(crate) fn anonymous(&self, alpn: Vec>) -> Result> { - self.build(alpn, false) - } - - fn build(&self, alpn: Vec>, client_auth: bool) -> Result> { let builder = rustls::ServerConfig::builder_with_provider(crypto::provider()).with_safe_default_protocol_versions()?; - let verifier = client_auth.then_some(self.0.verifier.as_ref()).flatten(); + let verifier = self.0.verifier.as_ref(); let mut config = match verifier { Some(verifier) => builder .with_client_cert_verifier(verifier.clone()) @@ -1657,6 +1643,11 @@ impl Serve { } Ok(Arc::new(config)) } + + #[cfg(feature = "_transport")] + pub(crate) fn client_auth(&self) -> bool { + self.0.verifier.is_some() + } } /// A peer's validated client-certificate chain from the mTLS handshake. diff --git a/rs/qmux/src/session.rs b/rs/qmux/src/session.rs index 536816a1f4d..f2e401d2cad 100644 --- a/rs/qmux/src/session.rs +++ b/rs/qmux/src/session.rs @@ -432,6 +432,9 @@ pub struct Session { _guard: Arc, /// The application protocol, cached at establishment so `protocol()` can borrow it. protocol: Option, + /// What the TLS handshake learned about the peer, on a session over TLS. + #[cfg(feature = "tls")] + pub(crate) tls: Option>, } impl Session { @@ -490,6 +493,8 @@ impl Session { _guard: Arc::new(Guard(inner.clone())), inner, protocol: config.negotiated_protocol(), + #[cfg(feature = "tls")] + tls: None, }) } diff --git a/rs/qmux/src/tls.rs b/rs/qmux/src/tls.rs index 473b9b19424..5416ae3bebb 100644 --- a/rs/qmux/src/tls.rs +++ b/rs/qmux/src/tls.rs @@ -1,4 +1,6 @@ use std::sync::Arc; + +use rustls::pki_types::CertificateDer; use tokio::net::{TcpStream, ToSocketAddrs}; use tokio_rustls::TlsAcceptor; use tokio_rustls::TlsConnector; @@ -101,12 +103,13 @@ impl Client { } let session_config = self.session.clone().with_negotiated(version, protocol); + let handshake = Handshake::new(tls_stream.get_ref().1.peer_certificates(), None); let mut transport = Stream::new(tls_stream); if let Some(stats) = socket_stats { transport = transport.with_socket_stats(stats); } // `connect` awaits the peer's transport parameters so `protocol()` is resolved. - Session::connect(transport, session_config).await + Ok(Session::connect(transport, session_config).await?.with_tls(handshake)) } } @@ -146,6 +149,10 @@ impl Server { /// [`handshake_timeout`](crate::Config::handshake_timeout)). Drive each /// connection with `tokio::spawn` so a slow or non-cooperative peer can't /// stall your `listener.accept()` loop. + /// + /// The returned session reports the client's certificate chain, when the + /// `rustls::ServerConfig` asked for one and the client presented it, and the + /// SNI it sent (see [`Session::peer_certificates`] and [`Session::server_name`]). pub async fn accept(&self, stream: TcpStream) -> Result { let socket_stats = crate::tcp::adopt(&stream)?; let acceptor = TlsAcceptor::from(self.config.clone()); @@ -159,11 +166,53 @@ impl Server { tracing::debug!(?version, ?protocol, "parsed ALPN"); let session_config = self.session.clone().with_negotiated(version, protocol); + let conn = tls_stream.get_ref().1; + let handshake = Handshake::new(conn.peer_certificates(), conn.server_name()); let mut transport = Stream::new(tls_stream); if let Some(stats) = socket_stats { transport = transport.with_socket_stats(stats); } // `accept` awaits the peer's transport parameters so `protocol()` is resolved. - Session::accept(transport, session_config).await + Ok(Session::accept(transport, session_config).await?.with_tls(handshake)) + } +} + +/// What a session's TLS handshake learned about the peer, captured before the +/// stream is handed to the session. +#[derive(Debug)] +pub(crate) struct Handshake { + peer_certificates: Option>>, + server_name: Option, +} + +impl Handshake { + fn new(peer_certificates: Option<&[CertificateDer<'static>]>, server_name: Option<&str>) -> Arc { + Arc::new(Self { + peer_certificates: peer_certificates.map(<[_]>::to_vec), + server_name: server_name.map(str::to_owned), + }) + } +} + +impl Session { + fn with_tls(mut self, handshake: Arc) -> Self { + self.tls = Some(handshake); + self + } + + /// The certificate chain the peer presented in the TLS handshake, leaf first. + /// + /// On a [`Client`] session this is the server's chain. On a [`Server`] session + /// it is the client's, and only when the `rustls::ServerConfig` asked for one + /// and the client sent it; its client-certificate verifier has already + /// accepted the chain. `None` on a session not carried by TLS. + pub fn peer_certificates(&self) -> Option<&[CertificateDer<'static>]> { + self.tls.as_ref()?.peer_certificates.as_deref() + } + + /// The SNI the client sent, on a session a [`Server`] accepted. `None` when the + /// client sent none, on a [`Client`] session, or off TLS. + pub fn server_name(&self) -> Option<&str> { + self.tls.as_ref()?.server_name.as_deref() } }