Skip to content
22 changes: 5 additions & 17 deletions quest/m1/session-policy/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Session policy lives in moq-net
# [M] Session policy lives in moq-net

## Goal

Expand All @@ -7,9 +7,7 @@ SETUP, admission and the reply, closing with Timeout), the admission bound,
widening the receive window on admit, and reconnect with backoff come from
moq-net for tokio, io_uring and WebRTC sessions alike. moq-tokio keeps only
sockets, DNS, TLS, accept loops, the fallback runtime and spawning. The epic is
done when moq-relay's hand-copied policy in `uring.rs` is deleted in favour of
the moq-net calls, and the WebRTC transport's session runs through the same
policy.
done when the WebRTC transport's session runs through the same policy.
Comment thread
pullfrog[bot] marked this conversation as resolved.
Resolved

## Plan

Expand All @@ -28,8 +26,8 @@ in moq-net, and decided:
- ✅ **Reconnect is in scope**: the reconnect, backoff and GOAWAY candidate
walk move too, as a poll machine. Not chosen: leaving them in moq-tokio
until io_uring dials.
- ✅ **Fold** the m2 relay-serve-path quest into
[the io_uring path](/quest/m1/session-policy/uring.md) and the
- ✅ **Fold** the m2 relay-serve-path quest into the io_uring path, which landed
in [moq-dev/moq#8082](https://github.com/moq-dev/moq/pull/8082), and the
poll-first server-machines quest into accept policy, which landed in
[moq-dev/moq#8003](https://github.com/moq-dev/moq/pull/8003).
- ✅ **Break moq-net's public API in m1**, before the cut, with MIGRATE notes.
Expand All @@ -47,13 +45,7 @@ Research (2026-11-04, `origin/main` at e6be13d5f8):
`Goaway`, `run`). All of it runs on `kio::clock`, so none of it needs tokio.
Only sockets, DNS, Happy Eyeballs, TLS, accept loops, `runtime::scope`,
spawning and WebRTC's own UDP socket are tokio-specific.
- moq-relay's io_uring path re-implements the server side by hand
(`rs/moq-relay/src/uring.rs`: its own `Deadline`, the admission bound,
the drain-bounded reply, and the `with_deadline` and `set_limits` calls
that wire them up). Eight commits
patched drift between the copies (58eaa6afd5, 8a13d58910, 6bd1306d6e,
1f56330d21, d17b84ee7f, 4576bb1231, c11192c7e6, 67df70012d). moq-uring
itself has no client side.
- moq-uring has no client side.
- Widening on admit needs a transport hook: moq-net's `transport::poll`
session exposes only `receive_window()`, while tokio QUIC, qmux and uring's
`web::Session` all have `set_limits`. Add a trait method whose default
Expand All @@ -62,7 +54,3 @@ Research (2026-11-04, `origin/main` at e6be13d5f8):
- moq-net stays poll-first and spawns nothing
([libraries spawn nothing](https://github.com/moq-dev/moq/pull/6849)):
policy returns drivers and machines, and the caller spawns and does the I/O.

## Required

- [The io_uring relay path uses moq-net's policy](/quest/m1/session-policy/uring.md) - deletes `uring.rs`'s copy
37 changes: 0 additions & 37 deletions quest/m1/session-policy/uring.md

This file was deleted.

8 changes: 4 additions & 4 deletions quest/m3/relay-io-uring-package.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,13 @@ ships, not the default.
(`listen.lb_id` QUIC-LB server ids). That refusal is a recorded exception
to parity, not a prerequisite, until a consumer asks.

Decided 2026-11-05 (maintainer, /quest-audit): ✅ this waits for
[the io_uring relay path using moq-net's policy](/quest/m1/session-policy/uring.md),
so packaging never publishes the hand-copied lifecycle path.
Decided 2026-11-05 (maintainer, /quest-audit): ✅ this waited for the io_uring
relay path to serve through moq-net's policy, so packaging never publishes the
hand-copied lifecycle path; that landed in
[moq-dev/moq#8082](https://github.com/moq-dev/moq/pull/8082).

Public API: none. Wire: none.

## Required

- [One port on the io_uring workers](/quest/m3/uring-demux.md) - a one-port config is served on the ring too
- [The io_uring relay path uses moq-net's policy](/quest/m1/session-policy/uring.md) - the ring serves through the shared accept policy before it ships
4 changes: 3 additions & 1 deletion rs/moq-relay/benches/http_auth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,10 @@ impl Fixture {
let driver = tokio::spawn(async move { driver.run().await.unwrap() });
// No admission timeout, so held requests stay waiting however long setup and the
// measurement take.
let mut listen = moq_tokio::listen::Config::default();
listen.timeout = Some(std::time::Duration::ZERO.into());
let web = web::Web::new(auth, cluster, certificates, config)
.with_timeout(None)
.with_listen(&listen)
.bind()
.unwrap();
let addr = web.addrs().http.unwrap();
Expand Down
113 changes: 71 additions & 42 deletions rs/moq-relay/src/connection.rs
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
use crate::{auth, cluster, incoming::Incoming, refusals::RefusalKind};

use futures::FutureExt as _;
use tracing::Instrument;
#[cfg(all(target_os = "linux", feature = "_uring"))]
use tracing::instrument::WithSubscriber;

/// An admission refusal carrying the session code to close the request with.
///
/// Used only on the pre-accept auth path so the caller can close once with
/// the right code instead of sprinkling close/return at each failure site.
pub(crate) struct Rejection {
pub(crate) code: moq_net::SessionError,
struct Rejection {
code: moq_net::SessionError,
refusal: Option<RefusalKind>,
pub(crate) source: anyhow::Error,
source: anyhow::Error,
}

impl From<auth::Error> for Rejection {
Expand All @@ -30,16 +30,16 @@ impl From<auth::Error> for Rejection {
/// [`run`](Self::run) to admit the request through its lease, wire up
/// publish/subscribe origins, and serve the session until it closes.
///
/// The request is a moq-tokio [`Request`](moq_tokio::server::Request), or an
/// in-memory accepted session when the `test-support` feature is enabled.
/// The request is a moq-tokio [`Request`](moq_tokio::server::Request), an io_uring
/// worker's, or an in-memory accepted session when the `test-support` feature is enabled.
pub struct Connection<R: Incoming = moq_tokio::server::Request> {
request: R,
lifecycle: Lifecycle,
}

/// Shared owner of admission, supervision and the connection's log span.
#[derive(Clone)]
pub(crate) struct Lifecycle {
struct Lifecycle {
id: u64,
span: tracing::Span,
cluster: cluster::Cluster,
Expand Down Expand Up @@ -83,21 +83,34 @@ impl<R: Incoming> Connection<R> {
/// ends only this connection, as an error.
pub async fn run(self) {
let Self { request, lifecycle } = self;
lifecycle.run(Self::serve(request, &lifecycle)).await;
match request.shared().cloned() {
Some(shared) => lifecycle.start(&shared, Self::start(request, &lifecycle)).await,
None => {
lifecycle
.run(async { Self::start(request, &lifecycle).await?.await })
.await
}
}
}

async fn serve(mut request: R, lifecycle: &Lifecycle) -> anyhow::Result<()> {
/// Admit the request and answer its SETUP, returning the session's supervision.
async fn start(mut request: R, lifecycle: &Lifecycle) -> anyhow::Result<Supervision> {
let shared = request.shared().cloned();
let admit = {
let admission = request.admission(&lifecycle.auth);
let pacer = request.pacer();
let lifecycle = lifecycle.clone();
async move { lifecycle.admit(admission, pacer).await }
};
// Abandoned at the drain deadline, so the deadline bounds this task and its
// socket like it bounds a session, and at the handshake deadline, so a peer
// cannot hold the connection open behind a decision that never comes.
// Boxed: the admission future is large, and nesting it inline overflows debug stacks.
let admitted = request
.within(Box::pin(
lifecycle.admit(request.admission(&lifecycle.auth), request.pacer()),
))
.within(Box::pin(lifecycle.on(shared.as_ref(), admit)))
.await
.inspect_err(|_| lifecycle.cluster.refusals.record(RefusalKind::Unavailable))
.map_err(|err| err.context("admission timed out"))?;
.map_err(|err| err.context("admission timed out"))??;
let (admitted, registration) = match admitted {
Ok(admitted) => admitted,
Err(err) => {
Expand Down Expand Up @@ -142,23 +155,34 @@ impl<R: Incoming> Connection<R> {
let serving = lifecycle.shutdown.serve();
// Bounded too: a peer that stalls the SETUP reply must not outlive the relay.
// Abandoning the reply refuses the session with the timeout code, which its stats count.
let (session, driver) = lifecycle
.shutdown
.until_deadline(request.ok())
.await
.ok_or_else(|| anyhow::anyhow!("the SETUP reply was still pending at the drain deadline"))??;
let expired = {
let shutdown = lifecycle.shutdown.clone();
lifecycle.on(shared.as_ref(), async move { shutdown.expired().await })
};
let (session, driver) = tokio::select! {
biased;
_ = expired => anyhow::bail!("the SETUP reply was still pending at the drain deadline"),
reply = request.ok() => reply?,
};

// Joined rather than awaited in turn, for a driver that runs only when polled. A
// spawned driver's panic still surfaces here, so it cannot look like cancellation.
let (result, driven) = futures::join!(
lifecycle.supervise(serving, session, lease, registration, transport),
R::drive(driver),
);
driven?;
result
let lifecycle = lifecycle.clone();
Ok(async move {
let (result, driven) = futures::join!(
lifecycle.supervise(serving, session, lease, registration, transport),
R::drive(driver),
);
driven?;
result
}
.boxed())
}
}

/// A started session's supervision, which runs until the session closes.
type Supervision = futures::future::BoxFuture<'static, anyhow::Result<()>>;

/// The span every line about one connection logs under, on whichever runtime emits it.
fn span(id: u64, remote: Option<std::net::SocketAddr>) -> tracing::Span {
tracing::info_span!(
Expand All @@ -171,12 +195,7 @@ fn span(id: u64, remote: Option<std::net::SocketAddr>) -> tracing::Span {
}

impl Lifecycle {
pub(crate) fn new(
cluster: cluster::Cluster,
auth: auth::Auth,
id: u64,
remote: Option<std::net::SocketAddr>,
) -> Self {
fn new(cluster: cluster::Cluster, auth: auth::Auth, id: u64, remote: Option<std::net::SocketAddr>) -> Self {
Self {
id,
span: span(id, remote),
Expand All @@ -187,26 +206,25 @@ impl Lifecycle {
}
}

pub(crate) fn with_shutdown(mut self, shutdown: crate::shutdown::Observer) -> Self {
fn with_shutdown(mut self, shutdown: crate::shutdown::Observer) -> Self {
self.shutdown = shutdown;
self
}

pub(crate) fn with_sessions(mut self, sessions: crate::session::Registry) -> Self {
fn with_sessions(mut self, sessions: crate::session::Registry) -> Self {
self.sessions = Some(sessions);
self
}

/// Run `task` in this connection's span, then log its close; a panic closes it as an error.
pub(crate) async fn run(&self, task: impl std::future::Future<Output = anyhow::Result<()>>) {
async fn run(&self, task: impl std::future::Future<Output = anyhow::Result<()>>) {
let result = self.isolate(task).await;
self.closed(result);
}

/// Run the ring worker's half, then supervise the started session on `runtime`, which logs
/// Run the connection task's half, then supervise the started session on `runtime`, which logs
/// the close there: a worker stopped at shutdown drops its tasks, and the line with them.
#[cfg(all(target_os = "linux", feature = "_uring"))]
pub(crate) async fn start<S>(
async fn start<S>(
&self,
runtime: &tokio::runtime::Handle,
start: impl std::future::Future<Output = anyhow::Result<S>>,
Expand Down Expand Up @@ -240,10 +258,21 @@ impl Lifecycle {
}
}

/// Run `task` on `runtime`, or in place without one; see [`shared`](Self::shared).
async fn on<T: Send + 'static>(
&self,
runtime: Option<&tokio::runtime::Handle>,
task: impl std::future::Future<Output = T> + Send + 'static,
) -> anyhow::Result<T> {
match runtime {
Some(runtime) => self.shared(runtime, task).await,
None => Ok(task.await),
}
}

/// Keep the connection span across a worker-to-shared-runtime handoff and return panics to its owner.
/// Dropping the returned future aborts the task, so it never outlives its connection.
#[cfg(all(target_os = "linux", feature = "_uring"))]
pub(crate) async fn shared<T: Send + 'static>(
async fn shared<T: Send + 'static>(
&self,
runtime: &tokio::runtime::Handle,
task: impl std::future::Future<Output = T> + Send + 'static,
Expand All @@ -270,7 +299,7 @@ impl Lifecycle {

/// Admit `request`, holding its connection to a capped grant through `pacer`, or
/// refusing a capped grant without one.
pub(crate) async fn admit(
async fn admit(
&self,
request: moq_auth::Request,
pacer: Option<auth::Pacer>,
Expand Down Expand Up @@ -343,7 +372,7 @@ impl Lifecycle {
Ok((admitted, registration))
}

pub(crate) async fn supervise(
async fn supervise(
&self,
serving: crate::shutdown::Serving,
session: moq_net::Session,
Expand Down Expand Up @@ -438,11 +467,11 @@ pub(crate) fn session_bytes(session: &moq_net::Session) -> moq_auth::Bytes {
}
}

#[cfg(all(test, target_os = "linux", feature = "_uring"))]
#[cfg(test)]
#[path = "../tests/integration/logs.rs"]
mod logs;

#[cfg(all(test, target_os = "linux", feature = "_uring"))]
#[cfg(test)]
mod tests {
use super::*;

Expand Down
Loading
Loading