Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion doc/bin/relay/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ solicit = false # Decline MoQ Solicit, so peers may announc

`timeout` bounds the whole handshake, transport through MoQ SETUP, so a peer
that connects and never speaks is closed instead of held open by keep-alives.
The `io_uring` workers do not apply it.

A setting no configured listener reads stops startup rather than being ignored.
A stream-only relay refuses `preferred_v4`, `preferred_v6`, and `lb_id` (or `load_balancer`), which
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,6 @@ disabled rung strands WHEP, HLS, and moq-mux export viewers.
- [Plan: watch worker](/quest/m1/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides
- [Watch worker](/quest/m1/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio
- [Front deadlines](/quest/m1/front-deadline-index.md) - a front's per-event cost stops growing with its track count: an expiry index and per-track wakes, proven by a churn benchmark
- [io_uring handshake deadline](/quest/m1/listener-deadlines.md) - the io_uring workers apply `listen.timeout` to the handshake
- [Rust papercuts](/quest/m1/papercuts-rs.md) - HTTP refusals are counted, and a `u64::MAX` resume is unbounded
- [Duration parser defaults](/quest/m1/cli-duration-defaults.md) - a moq-tokio duration set in code beats the parser default on a parsed or merged config, zero included, after #5350
- [Front parking](/quest/m1/origin-front-parks.md) - an unroutable request waits on a front instead of re-asking on every route-table move
Expand Down
34 changes: 0 additions & 34 deletions quest/m1/listener-deadlines.md

This file was deleted.

4 changes: 4 additions & 0 deletions quest/m1/time/uring.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ flavors, and relay CPU via `just bench BASE` on Linux. The existing keep-alive
and idle-timeout tests stay unchanged. Ban direct clock reads in moq-uring
outside its backend (the ratchet).

Move `uring_workers_bound_stalled_handshakes` (`rs/moq-relay/tests/runtime_uring.rs`)
onto controlled time too; it landed on the wall clock in #5244 rather than
waiting for this quest.

Public API: breaking. `moq_uring::Timer` is re-exported, and `Timer::set`
takes `Option<moq_time::Instant>` instead of std's; no conversion shim.
Wire: none.
Expand Down
1 change: 0 additions & 1 deletion quest/m2/relay-io-uring-package.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,5 +37,4 @@ Public API: none. Wire: none.
- [moq-transport on io_uring](/quest/m2/uring-ietf.md) - a packaged relay
must not drop protocols when the ring is on
- [Dropped session close](/quest/m2/uring-drop-close.md) - closes are delivered when a session is dropped on the ring
- [Listener deadlines](/quest/m1/listener-deadlines.md) - the io_uring workers apply `listen.timeout` like the default runtime
- [One port on the io_uring workers](/quest/m2/uring-demux.md) - a one-port config is served on the ring too
1 change: 1 addition & 0 deletions rs/moq-relay/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ rand = { workspace = true }
rcgen = "0.14"
tempfile = { workspace = true }
tokio = { workspace = true, features = ["test-util"] }
web-transport-moq = { workspace = true }

# Which sockets the process still holds, so the embed tests can prove a stopped
# relay closed its own without rebinding a port another test could take.
Expand Down
66 changes: 59 additions & 7 deletions rs/moq-relay/src/uring.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ struct Serve {
versions: moq_net::Versions,
/// The address every worker socket shares, reported as a session's `local`.
local: SocketAddr,
/// `listen.timeout`, which each worker's endpoint starts at a connection's
/// first packet.
timeout: Option<std::time::Duration>,
}

/// A bound group of io_uring QUIC workers sharing one port.
Expand All @@ -94,6 +97,7 @@ pub struct Workers {
pin: bool,
alpns: Arc<Vec<String>>,
versions: moq_net::Versions,
timeout: Option<std::time::Duration>,

threads: Vec<(u16, std::thread::JoinHandle<()>)>,
stops: Vec<Arc<Stop>>,
Expand Down Expand Up @@ -249,6 +253,7 @@ impl Workers {
pin: config.pin,
alpns: Arc::new(alpns),
versions,
timeout: listen.resolved_timeout(),
threads: Vec::new(),
stops: Vec::new(),
failures,
Expand Down Expand Up @@ -300,6 +305,7 @@ impl Workers {
alpns: self.alpns.clone(),
versions: self.versions.clone(),
local: self.addr,
timeout: self.timeout,
};

let cores = match self.pin {
Expand Down Expand Up @@ -537,6 +543,7 @@ fn serve_worker(spawn: Spawn) -> bool {
.context("failed to adopt the worker socket")?;
let mut endpoint_config = moq_uring::quic::endpoint::Config::default().with_server(server);
endpoint_config.activity = serve.shutdown.activity.clone();
endpoint_config.handshake.timeout = serve.timeout;
let endpoint =
moq_uring::quic::Endpoint::new(socket, endpoint_config).context("failed to build the QUIC endpoint")?;
Ok((worker, endpoint))
Expand Down Expand Up @@ -583,6 +590,45 @@ fn serve_worker(spawn: Spawn) -> bool {
true
}

/// What remains of a connection's handshake deadline, and how to tell the peer
/// it passed.
struct Deadline {
timer: moq_uring::Timer,
/// Closes with the timeout code once a session exists to carry it. Until
/// then a timed-out handshake is just dropped, and its guard closes in the
/// handshake's own error space, as HTTP/3 has no MoQ codes.
close: Option<moq_uring::quic::web::Session>,
}

impl Deadline {
fn new(handle: &moq_uring::Handle, conn: &moq_uring::quic::Connection) -> Self {
let mut timer = handle.timer();
timer.set(conn.handshake().deadline);
Self { timer, close: None }
}

async fn bound<T, E: Into<anyhow::Error>>(
&mut self,
handshake: impl Future<Output = Result<T, E>>,
) -> anyhow::Result<T> {
use web_transport_trait::poll::Session as _;
let timer = std::pin::pin!(self.timer.wait());
let handshake = std::pin::pin!(handshake);
match futures::future::select(timer, handshake).await {
futures::future::Either::Left(_) => {
// Close before dropping the pending handshake, whose cancellation guard
// would otherwise replace the timeout with an abandoned-handshake error.
let err = moq_net::Error::Timeout;
if let Some(close) = &mut self.close {
close.close(moq_net::SessionError::Timeout.to_code(), &err.to_string());
}
Err(err.into())
}
futures::future::Either::Right((result, _)) => result.map_err(Into::into),
}
}
}

/// Serve one accepted connection on its worker: handshake and session start
/// here; authentication and supervision on the shared runtime.
async fn serve_connection(
Expand All @@ -593,6 +639,10 @@ async fn serve_connection(
) -> anyhow::Result<()> {
use web_transport_trait::poll::Session as _;

// The endpoint started this deadline at the connection's first packet and
// enforced it through the QUIC handshake; it covers SETUP OK too.
let mut deadline = Deadline::new(&handle, &conn);

// Read the link facts now: the handshake below consumes the connection.
let remote = conn.remote_addr();
let sni = conn.server_name().map(str::to_owned);
Expand All @@ -617,7 +667,8 @@ async fn serve_connection(
let mut alpn = conn.protocol().map(str::to_owned);
let (transport, url) = match conn.protocol() {
Some("h3") => {
let request = moq_uring::quic::web::Request::accept(conn)
let request = deadline
.bound(moq_uring::quic::web::Request::accept(conn))

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Own the H3 close guard before racing an already-expired deadline

If TLS completes near the deadline, or the worker delays serving the accepted connection, Deadline::bound can first poll an already-expired timer. select(timer, handshake) then drops Request::accept(conn) without polling it, so the guard constructed inside that async function (quic/web.rs:139-143) never exists. deadline.close is still None, and dropping Connection does not close QUIC (quic/noq/connection.rs:265-270). The relay returns Timeout locally while the peer remains connected until the idle timeout, which keep-alives can extend. Establish H3-aware close ownership before entering this race, and test an accepted H3 connection whose deadline has elapsed before the first accept poll.

@kixelated kixelated Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4315feb. I fixed it at the source rather than by reordering the select. Request::accept is now a plain fn returning a future, and it creates the close guard before it returns. A handshake that is dropped before its first poll therefore still closes with H3_GENERAL_PROTOCOL_ERROR. The new an_unpolled_handshake_closes_the_connection test in rs/moq-uring/tests/web.rs drops an unpolled accept and waits for the close. It fails when the guard is created at the first poll.

(Written by Claude Opus 5.5)

.await
.context("WebTransport handshake failed")?;
let url = request.url().clone();
Expand All @@ -631,18 +682,19 @@ async fn serve_connection(
response = response.with_protocol(protocol);
}
alpn = protocol.clone();
let session = request
.respond(response)
let session = deadline
.bound(request.respond(response))
.await
.context("WebTransport response failed")?;
(session, Some(url))
}
_ => (moq_uring::quic::web::Session::raw(conn), None),
};

let request = moq_net::Server::new()
.with_versions(serve.versions.clone())
.accept_request_lite(std::time::Instant::now(), moq_uring::transport::Session::new(transport))
deadline.close = Some(transport.clone());
let server = moq_net::Server::new().with_versions(serve.versions.clone());
let request = deadline
.bound(server.accept_request_lite(std::time::Instant::now(), moq_uring::transport::Session::new(transport)))
.await
.context("moq handshake failed")?;
Comment on lines +697 to 699

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep enforcing the deadline while authorization runs

When a client finishes SETUP before the deadline but the configured auth server or embedded admission remains pending past it, this bound call returns and nothing polls Deadline::timer again until request.ok() after authorization. Although the timer fires, it has no waiter that closes the transport, so a 200 ms listen.timeout can leave the handshake open until the separate 10-second admission timeout. Race the authorization wait against the same deadline, or make expiration proactively close the pending session, so listen.timeout actually bounds the whole handshake.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this as is for this PR. The tokio listener has the same shape: its deadline bounds the SETUP read and Request::ok (moq-tokio/src/server.rs), while admission in Connection::run runs unbounded by listen.timeout and is capped by ADMIT_TIMEOUT (10s). The io_uring path now matches that: once auth returns, an expired deadline closes with the timeout code on the request.ok() race. This PR's goal is that parity, not a change to what the deadline covers.

Whether listen.timeout should also cover admission, on both listeners, is a separate behavior change (it affects slow auth servers, which have their own cap today). I'm recording it as a follow-up for the maintainer.

(Written by Claude Opus 5.5)


Expand Down Expand Up @@ -766,7 +818,7 @@ async fn serve_connection(
request = request.with_subscriber(publisher);
}
let lease = lease.preparing(request.auth(), request.role())?;
let (session, driver) = request.ok().await?;
let (session, driver) = deadline.bound(request.ok()).await?;
let driver_handle = handle.clone();
handle.spawn(async move {
match driver_handle.run(driver).await {
Expand Down
114 changes: 114 additions & 0 deletions rs/moq-relay/tests/runtime_uring.rs
Original file line number Diff line number Diff line change
Expand Up @@ -759,3 +759,117 @@ async fn uring_auth_reports_distinct_quic_and_webtransport() {
running.abort();
let _ = running.await;
}

/// `listen.timeout` bounds every handshake phase from the first packet: TLS,
/// HTTP/3 before CONNECT, and MoQ SETUP over raw QUIC and WebTransport. Each
/// phase closes in its own error space.
#[cfg(feature = "noq")]
#[tokio::test]
async fn uring_workers_bound_stalled_handshakes() {
use web_transport_moq::noq;
use web_transport_trait::Error as _;

let _ = rustls::crypto::aws_lc_rs::default_provider().install_default();
if !supported() {
return;
}
let dir = tempfile::tempdir().expect("tempdir");
let (cert, key) = certificate(dir.path());
let mut config = uring_config(&cert, &key);
config.listen.timeout = Duration::from_millis(200);
let relay = Relay::load(config).await.expect("load relay");
let port = relay.addr().expect("workers bound an address").port();
let addr = SocketAddr::from(([127, 0, 0, 1], port));
let running = tokio::spawn(relay.run());

let mut config = moq_tokio::connect::Config::default();
config.tls.insecure = Some(true);
let tls = config.tls.build().expect("tls config");
let crypto = |alpn: &[u8]| {
let mut tls = tls.clone();
tls.alpn_protocols = vec![alpn.to_vec()];
let crypto = noq::crypto::rustls::QuicClientConfig::try_from(tls).expect("QUIC crypto");
noq::ClientConfig::new(std::sync::Arc::new(crypto))
};
let client = noq::Endpoint::client("127.0.0.1:0".parse().unwrap()).expect("client");

// TLS: the relay never hears more than the ClientHello, so only its own
// deadline can end the handshake before the idle timeout does.
let hop = stall_after_hello(addr).await;
let stalled = client.connect_with(crypto(b"moq-lite-06"), hop, "localhost").unwrap();
let error = tokio::time::timeout(TIMEOUT, async {
match stalled.await {
Ok(conn) => conn.closed().await,
Err(err) => err,
}
})
.await
.expect("listener did not bound the QUIC handshake");
assert!(
!matches!(error, noq::ConnectionError::TimedOut),
"the relay must close a stalled QUIC handshake: {error}"
);

// `h3` stalls before CONNECT, where only HTTP/3 codes apply.
const H3_GENERAL_PROTOCOL_ERROR: u32 = 0x0101;
let timeout = moq_net::SessionError::Timeout.to_code();
for (scheme, code) in [("moql", timeout), ("https", timeout), ("h3", H3_GENERAL_PROTOCOL_ERROR)] {
let session = tokio::time::timeout(TIMEOUT, async {
let alpn: &[u8] = if scheme == "moql" { b"moq-lite-06" } else { b"h3" };
let conn = client
.connect_with(crypto(alpn), addr, "localhost")
.unwrap()
.await
.unwrap();
if scheme == "https" {
let url: url::Url = format!("https://{addr}/stalled").parse().unwrap();
let request =
web_transport_moq::proto::ConnectRequest::new(url).with_protocol("moq-lite-06".to_string());
web_transport_moq::Session::connect(conn, request).await.unwrap()
} else {
web_transport_moq::Session::raw(conn)
}
})
.await
.expect("transport handshake timeout");
let error = tokio::time::timeout(TIMEOUT, session.closed())
.await
.expect("listener did not enforce its handshake deadline");
// The WebTransport wrapper translates its close code back to the MoQ value.
assert_eq!(
error.session_error().map(|(code, _)| code),
Some(code),
"{scheme}: {error}"
);
}
client.close(noq::VarInt::from_u32(0), b"test complete");
client.wait_idle().await;
running.abort();
let _ = running.await;
}

/// A UDP hop to `server` that forwards one client's datagrams until the server
/// first answers, and every answer back, so the server never sees the client
/// finish its handshake. Returns the address to dial.
#[cfg(feature = "noq")]
async fn stall_after_hello(server: SocketAddr) -> SocketAddr {
let socket = tokio::net::UdpSocket::bind("127.0.0.1:0").await.expect("bind hop");
let addr = socket.local_addr().expect("hop address");
tokio::spawn(async move {
let mut client = None;
let mut answered = false;
let mut buf = vec![0; usize::from(u16::MAX)];
while let Ok((len, from)) = socket.recv_from(&mut buf).await {
if from == server {
answered = true;
if let Some(client) = client {
let _ = socket.send_to(&buf[..len], client).await;
}
} else if !answered {
client = Some(from);
let _ = socket.send_to(&buf[..len], server).await;
}
}
});
addr
}
4 changes: 4 additions & 0 deletions rs/moq-uring/src/quic/endpoint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ pub struct Config {
/// peer retransmits and lands once the application drains the backlog or
/// a handshake fails.
pub backlog: usize,

/// How incoming handshakes are bounded.
pub handshake: super::handshake::Config,
}

impl Default for Config {
Expand All @@ -57,6 +60,7 @@ impl Default for Config {
server: None,
activity: Default::default(),
backlog: 1024,
handshake: Default::default(),
}
}
}
Expand Down
29 changes: 29 additions & 0 deletions rs/moq-uring/src/quic/handshake.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
//! The handshake an accepted connection goes through before it is served.
//!
//! An [`Endpoint`](super::Endpoint) bounds it by [`Config::timeout`] from the
//! connection's first packet, closing a QUIC handshake still running at the
//! deadline. The rest of the deadline is handed to whatever the application
//! layers on top (HTTP/3, MoQ SETUP) as [`Info::deadline`].

use std::time::{Duration, Instant};

/// How an endpoint bounds the handshakes it accepts.
#[derive(Clone, Debug, Default)]
#[non_exhaustive]
pub struct Config {
/// How long an incoming connection has from its first packet to finish its
/// whole handshake, or `None` (the default) for no bound but the idle
/// timeout.
pub timeout: Option<Duration>,
}

/// An established connection's handshake, from
/// [`Connection::handshake`](super::Connection::handshake).
#[derive(Clone, Copy, Debug)]
#[non_exhaustive]
pub struct Info {
/// When the handshake, including whatever the application layers on top of
/// QUIC, must be done. `None` on a dialed connection or without a
/// [`Config::timeout`].
pub deadline: Option<Instant>,
}
Loading
Loading