Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/bin/relay/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -480,3 +480,7 @@ parent directory or an allowlist.
dials such as [cluster links](/bin/relay/cluster#tls-links). It asks for a
client certificate as the QUIC listener does, under the same `listen.tls.root`,
so a peer on it presents a certificate or a token.
The HTTPS listener (`web.https`) accepts `tls://` too, with no setting: a
ClientHello that offers a moq ALPN is served the listen certificate and the
same client-certificate request, while every other connection keeps
`web.https.cert`.
5 changes: 3 additions & 2 deletions doc/bin/relay/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,9 @@ topology check; whatever generates the peer list applies the layout's rules.
A link inside a region is rarely congested, so it can skip QUIC: a `tls://`
peer URL dials qmux over TLS on TCP, verified with the same `connect.tls`
settings as any other dial. The accepting relay serves it from its TCP
listener with TLS on. That listener applies the same client-certificate policy
as the QUIC listener, so a peer on it authenticates with its `connect.tls`
listener with TLS on, or from its HTTPS listener, which serves `tls://` with
the listen certificate. Both apply the same client-certificate policy as the
QUIC listener, so a peer on them authenticates with its `connect.tls`
certificate against `listen.tls.root`, or with a token (`cluster.token`,
`token`, or `?jwt=`).

Expand Down
2 changes: 1 addition & 1 deletion doc/bin/relay/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ start rather than being ignored. If io\_uring workers fail to start naming
listen = "[::]:4443" # HTTP: fingerprint, announced, fetch, health.

[web.https]
listen = "[::]:443" # HTTPS plus the WebSocket fallback.
listen = "[::]:443" # HTTPS, the WebSocket fallback, and tls:// with the listen certificate.
cert = "cert.pem" # cert, key, and root need listen. Omit cert and key with [acme].
key = "key.pem"

Expand Down
4 changes: 4 additions & 0 deletions doc/bin/relay/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,10 @@ relay.with_web(web).run().await?;
send its path in the SETUP.
- To decide admissions yourself, leave `[auth]` empty and answer
`relay.admissions()`; see [Authentication](/bin/relay/auth#in-process).
- To serve RTMP on the web ports, take `relay.rtmp()` and pass each accepted
stream to `moq_rtmp::accept_stream`. The HTTP and HTTPS listeners hand it
plaintext RTMP, and RTMPS (TLS with no ALPN) already decrypted; untaken, the
relay closes them.

See [docs.rs/moq-relay](https://docs.rs/moq-relay) and
[`rs/moq-relay/examples/embed.rs`](https://github.com/moq-dev/moq/blob/main/rs/moq-relay/examples/embed.rs).
Expand Down
16 changes: 2 additions & 14 deletions quest/m2/one-port/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ A relay, or a binary embedding the gateway crates beside one, speaks
everything it serves on one UDP port and one TCP port. UDP carries QUIC
(WebTransport and raw moq), the WebRTC media path (STUN, DTLS, SRTP) for
WHIP and WHEP, and SRT. TCP carries TLS-terminated
HTTP (WebSocket qmux, WHIP and WHEP signaling, HLS, ops), RTMP, and RTMPS.
HTTP (WebSocket qmux, WHIP and WHEP signaling, HLS, ops), `tls://` qmux,
RTMP, and RTMPS.
Upstream delivers the demux and stacks that accept a fed socket or stream;
`moq-relay` itself serves QUIC and HTTP on them,
and an embedder such as moq.pro's edge wires WebRTC, RTMP, and SRT, which the
Expand Down Expand Up @@ -81,22 +82,9 @@ and goes to WebRTC; any other is dropped and counted. Decided 2026-10-13:
the responder binds its own port ([relay STUN](/doc/bin/relay/config.md#stun)),
so the shared port never answers public Binding requests.

### TCP

The 1935 listener downstream already peeks one byte to split RTMPS from RTMP.
The unified acceptor generalizes it: 0x16 is TLS, terminated here, then the
first decrypted byte is peeked again; 0x03 is an RTMP handshake and anything
else is HTTP, which goes to the axum router. A plaintext 0x03 is RTMP and a
plaintext ASCII method is HTTP. RTMPS clients send no ALPN, so ALPN cannot do
this. The acceptor yields classified connections; `moq-rtmp`'s
`accept_stream` already takes any `AsyncRead + AsyncWrite`, and
`axum_server::Server::from_listener` takes a listener that a channel of
pre-accepted streams can stand behind.

## Required

- [WebRTC on the shared socket](/quest/m2/one-port/rtc-feed.md) - `moq-rtc` serves WHIP and WHEP media from the WebRTC hook and pins ICE tuples
- [TCP acceptor](/quest/m2/one-port/tcp-demux.md) - one listener carries TLS-terminated HTTP, RTMP, and RTMPS
- [SRT on the shared socket](/quest/m2/one-port/srt-demux.md) - moq-srt drives `srt-protocol` on demuxed packets and the flow table pins its 4-tuples

## Related
Expand Down
61 changes: 0 additions & 61 deletions quest/m2/one-port/tcp-demux.md

This file was deleted.

7 changes: 2 additions & 5 deletions quest/m2/rate-websocket.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ excess:
`SO_SNDBUF` to about the burst, since paced reads cannot stop bytes the
kernel already accepted and paced writes can queue a send buffer that drains
faster than the cap. Fixing the size disables autotuning, which is the
point. The buffers bound the burst only; the sustained cap comes from the
point. The web acceptor keeps the accepted socket reachable through
`listener::Stream::tcp` (`rs/moq-relay/src/listener/demux.rs`) for this. The buffers bound the burst only; the sustained cap comes from the
paced IO. Behind a TCP-terminating proxy, the buffers bound only the
relay's side.
- Reads stop at the bucket, so the client's TCP window closes; writes stop
Expand All @@ -48,7 +49,3 @@ live session applies.
## Required

- [QUIC caps](/quest/m2/rate-quic.md) - introduces the moq-auth cap type this reuses

## Related

- [TCP acceptor](/quest/m2/one-port/tcp-demux.md) - boxes the accepted stream, so it must keep the socket handle these buffer sizes need
18 changes: 18 additions & 0 deletions rs/moq-relay/src/listener.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@
//! back. Because the wrapper never yields an error upward, `axum_server`'s flat
//! sleep is unreachable.

mod demux;
mod idle;
pub(crate) use demux::{Classified, Stream, Tls, classify};
pub(crate) use idle::Admission;

use std::{
Expand All @@ -41,6 +43,17 @@ impl Accepted {
}
}

#[cfg(all(test, unix, feature = "websocket"))]
impl Accepted {
/// A socket accepted outside [`Listener`], holding no admission.
pub(crate) fn new(stream: TcpStream) -> Self {
Self {
stream,
admission: None,
}
}
}

impl AsyncRead for Accepted {
fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.stream).poll_read(cx, buf)
Expand All @@ -57,6 +70,11 @@ impl AsyncWrite for Accepted {
Pin::new(&mut self.stream).poll_shutdown(cx)
}
}
impl AsRef<TcpStream> for Accepted {
fn as_ref(&self) -> &TcpStream {
&self.stream
}
}
#[cfg(unix)]
impl std::os::fd::AsFd for Accepted {
fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
Expand Down
181 changes: 181 additions & 0 deletions rs/moq-relay/src/listener/demux.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
//! Splits a public web connection into HTTP, RTMP, and `tls://` qmux by its first bytes.
//!
//! `axum_server` runs the acceptor on each connection's own task, so a peer that
//! connects and sends nothing stalls only itself, under the same idle deadline
//! that bounds its TLS handshake and first request.

use std::{
io,
pin::Pin,
task::{Context, Poll},
time::Duration,
};

use axum_server::tls_rustls::RustlsConfig;
use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, ReadBuf};
use tokio::net::TcpStream;
use tokio_rustls::{LazyConfigAcceptor, server::TlsStream};

use super::Accepted;

/// The RTMP version byte (C0) an RTMP handshake opens with.
const RTMP_VERSION: u8 = 0x03;

/// The bound on a TLS handshake, as `axum_server`'s own rustls acceptor sets.
const TLS_HANDSHAKE: Duration = Duration::from_secs(10);

/// A classified connection: the protocol its first bytes named.
pub(crate) enum Classified {
Http(Stream),
Rtmp(Stream),
/// `tls://`: its ClientHello offered a moq ALPN, and the handshake negotiated one.
Qmux(Box<TlsStream<Accepted>>),
}

/// The TLS a listener terminates: HTTP's config, and `tls://`'s when it is served.
#[derive(Clone)]
pub(crate) struct Tls {
pub(crate) http: RustlsConfig,
pub(crate) qmux: Option<moq_tokio::tcp::TlsAcceptor>,
}

/// Classify `accepted`, terminating TLS with `tls` when the listener serves it.
///
/// A plaintext C0 is RTMP on either listener. On a TLS listener anything else is
/// handed to rustls, which refuses whatever is not a ClientHello (0x16), so
/// plaintext HTTP is still refused there. A ClientHello that offers a moq ALPN is
/// `tls://`, finished with the listen identity. Otherwise a negotiated ALPN names
/// HTTP; RTMPS clients offer none, so a connection without one is sniffed again
/// on its first decrypted byte.
pub(crate) async fn classify(accepted: Accepted, tls: Option<&Tls>) -> io::Result<Classified> {
let mut first = [0u8; 1];
if accepted.stream.peek(&mut first).await? == 0 {
return Err(io::ErrorKind::UnexpectedEof.into());
}
let Some(tls) = tls.filter(|_| first[0] != RTMP_VERSION) else {
let stream = Stream {
transport: Transport::Plain(accepted),
peeked: None,
};
return Ok(match first[0] {
RTMP_VERSION => Classified::Rtmp(stream),
_ => Classified::Http(stream),
});
};

let handshake = async {
let start = LazyConfigAcceptor::new(rustls::server::Acceptor::default(), accepted).await?;
let alpns = start.client_hello().alpn();
let qmux = tls
.qmux
.as_ref()
.filter(|qmux| alpns.is_some_and(|alpns| qmux.offered(alpns)));
let config = match qmux {
Some(qmux) => qmux.config(),
None => tls.http.get_inner(),
};
Ok::<_, io::Error>((start.into_stream(config).await?, qmux.is_some()))
};
let (mut tls, qmux) = kio::clock::timeout(TLS_HANDSHAKE, handshake)
.await
.map_err(|_| io::Error::from(io::ErrorKind::TimedOut))??;
if qmux {
return Ok(Classified::Qmux(Box::new(tls)));
}
if tls.get_ref().1.alpn_protocol().is_some() {
return Ok(Classified::Http(Stream {
transport: Transport::Tls(Box::new(tls)),
peeked: None,
}));
}
let first = tls.read_u8().await?;
let stream = Stream {
transport: Transport::Tls(Box::new(tls)),
peeked: Some(first),
};
Ok(match first {
RTMP_VERSION => Classified::Rtmp(stream),
_ => Classified::Http(stream),
})
}

/// An accepted connection after classification, plaintext or with TLS terminated.
pub(crate) struct Stream {
transport: Transport,
/// The decrypted byte read to classify the connection, replayed to the first read.
peeked: Option<u8>,
}

enum Transport {
Plain(Accepted),
Tls(Box<TlsStream<Accepted>>),
}

impl Stream {
fn accepted(&self) -> &Accepted {
match &self.transport {
Transport::Plain(accepted) => accepted,
Transport::Tls(tls) => tls.get_ref().0,
}
}

/// The accepted socket, for socket options and kernel statistics.
pub(crate) fn tcp(&self) -> &TcpStream {
&self.accepted().stream
}

/// The TLS session, when the connection carried one.
pub(crate) fn tls(&self) -> Option<&rustls::ServerConnection> {
match &self.transport {
Transport::Plain(_) => None,
Transport::Tls(tls) => Some(tls.get_ref().1),
}
}

pub(crate) fn admission(&self) -> Option<super::Admission> {
self.accepted().admission.clone()
}
}

impl AsyncRead for Stream {
fn poll_read(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll<io::Result<()>> {
if buf.remaining() > 0
&& let Some(byte) = self.peeked.take()
{
buf.put_slice(&[byte]);
return Poll::Ready(Ok(()));
}
match &mut self.transport {
Transport::Plain(accepted) => Pin::new(accepted).poll_read(cx, buf),
Transport::Tls(tls) => Pin::new(tls.as_mut()).poll_read(cx, buf),
}
}
}

impl AsyncWrite for Stream {
fn poll_write(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll<io::Result<usize>> {
match &mut self.transport {
Transport::Plain(accepted) => Pin::new(accepted).poll_write(cx, buf),
Transport::Tls(tls) => Pin::new(tls.as_mut()).poll_write(cx, buf),
}
}
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
match &mut self.transport {
Transport::Plain(accepted) => Pin::new(accepted).poll_flush(cx),
Transport::Tls(tls) => Pin::new(tls.as_mut()).poll_flush(cx),
}
}
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
match &mut self.transport {
Transport::Plain(accepted) => Pin::new(accepted).poll_shutdown(cx),
Transport::Tls(tls) => Pin::new(tls.as_mut()).poll_shutdown(cx),
}
}
}

#[cfg(unix)]
impl std::os::fd::AsFd for Stream {
fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
std::os::fd::AsFd::as_fd(self.tcp())
}
}
9 changes: 8 additions & 1 deletion rs/moq-relay/src/relay.rs
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,8 @@ impl Relay {
.with_extensions(server_extensions)
.with_implementation(implementation)
.with_timeout(handshake_timeout)
.with_sessions(sessions.clone());
.with_sessions(sessions.clone())
.with_tls_acceptor(server.tls_acceptor());
#[cfg(feature = "acme")]
let web = web.with_acme(acme.as_ref().map(crate::acme::Acme::resolver));
let web = web.bind()?;
Expand Down Expand Up @@ -446,6 +447,12 @@ impl Relay {
self.admissions.take()
}

/// The RTMP and RTMPS connections the public web listeners classify, for the
/// embedder to serve; see [`web::Web::rtmp`]. Untaken, they are logged and closed.
pub fn rtmp(&mut self) -> web::rtmp::Incoming {
self.web.rtmp()
}

/// The shared cluster: the origin every session and peer publishes into.
pub fn cluster(&self) -> &cluster::Cluster {
&self.cluster
Expand Down
Loading