diff --git a/doc/bin/relay/index.md b/doc/bin/relay/index.md index ddadbe80bd1..ed6344349e5 100644 --- a/doc/bin/relay/index.md +++ b/doc/bin/relay/index.md @@ -77,7 +77,10 @@ relay.with_web(web).run().await?; - `Relay::load` binds every socket, so a taken port fails there, and the accessors report the bound addresses, including ports assigned for `:0`. -- `run` consumes the relay, so clone the handles your tasks need first. +- `run` consumes the relay, so clone the handles your tasks need first. Stats + publishing starts when `run` drives the cluster; an application building a + `Cluster` itself sets stats in `cluster::Options` and must keep polling the + returned driver while serving. - Build on `web().routes()`: `with_web` replaces the router, so `Router::new()` drops the built-in routes. - A route reads the connection's peer address with axum's diff --git a/quest/m1/poll-first/no-spawn/README.md b/quest/m1/poll-first/no-spawn/README.md index 75f629befad..2a142fed161 100644 --- a/quest/m1/poll-first/no-spawn/README.md +++ b/quest/m1/poll-first/no-spawn/README.md @@ -52,7 +52,6 @@ never runs inside the root. It owns no state and polls no background work. ## Required -- [Admissions and stats are polled](/quest/m1/poll-first/no-spawn/admissions.md) - HTTP admissions run inside their waiter, and the stats publisher runs in the cluster's driver - [The relay's in-memory tests run on kio's simulator](/quest/m1/poll-first/no-spawn/sim-tests.md) - `rs/moq-relay/tests/integration/connection.rs` runs on the simulator instead of paused tokio ## Related diff --git a/quest/m1/poll-first/no-spawn/admissions.md b/quest/m1/poll-first/no-spawn/admissions.md deleted file mode 100644 index 4ada3dda796..00000000000 --- a/quest/m1/poll-first/no-spawn/admissions.md +++ /dev/null @@ -1,39 +0,0 @@ -# [S] Admissions and stats are polled - -## Goal - -moq-relay spawns no task per HTTP admission, and `Cluster::with_stats` spawns -no stats publisher. The `Server` decider's request runs inside the caller that -waits on its answer, and the publisher runs as a child of the cluster's -`Driver`. - -## Plan - -Spawns in scope on `main` (2026-11-01): - -- `rs/moq-relay/src/auth.rs`, `Decider::Server`: one - `tokio::spawn(moq_tokio::panic::isolate(..))` per admission HTTP request. - [moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) (merged - 2026-11-01) already removed the decider loop around it. -- `rs/moq-relay/src/cluster.rs`, `Cluster::with_stats`: the stats publisher, - through `Critical`. Since #7360, `moq_stats::Producer::new` returns - `(Producer, Publisher)`, so moq-stats itself spawns nothing. - -✅ **The waiter polls the admission** (maintainer, 2026-11-01, D3 on -[moq-dev/moq#7519](https://github.com/moq-dev/moq/pull/7519)): the caller -waiting on `Answer` polls the `moq_auth::Client` request inline, and dropping -the wait cancels the request, which replaces the `reply.closed()` select. A -panic then ends the session being admitted, the same scope the per-admission -`isolate` gives today. Rejected: one long-lived `kio::Tasks` of admissions -polled by `Relay::run`, which brings back the long-lived decider task that -#7360 removed. - -The publisher joins the cluster's `Driver` (`rs/moq-relay/src/cluster.rs`, -from [moq-dev/moq#7626](https://github.com/moq-dev/moq/pull/7626)), so a -publisher panic still ends the relay. `Cluster::new` has already handed the -driver out by the time `Cluster::with_stats` runs, so the publisher either -reaches it through a queue the driver polls or moves into `Options` for -`Cluster::new` to add. `Publisher::run` is an `async fn`; giving `Publisher` a -`poll` is optional and only worth it if `Driver` needs it. - -Public API: `Cluster::with_stats` no longer needs a tokio runtime. Wire: none. diff --git a/quest/m1/poll-first/no-spawn/sim-tests.md b/quest/m1/poll-first/no-spawn/sim-tests.md index e317ad695f5..065e5927f99 100644 --- a/quest/m1/poll-first/no-spawn/sim-tests.md +++ b/quest/m1/poll-first/no-spawn/sim-tests.md @@ -16,7 +16,3 @@ m2, or require the part of the spawn audit that converts those two helpers. Decide that when it starts. Public API: none. Wire: none. - -## Required - -- [Admissions and stats are polled](/quest/m1/poll-first/no-spawn/admissions.md) - admissions stop spawning on tokio diff --git a/rs/moq-relay/MIGRATE.md b/rs/moq-relay/MIGRATE.md index 5710fa7ffd9..045e9acf92e 100644 --- a/rs/moq-relay/MIGRATE.md +++ b/rs/moq-relay/MIGRATE.md @@ -5,6 +5,22 @@ Rolling a cluster across a release also follows the [upgrade guide](https://moq. ## Unreleased +### Stats are configured in `cluster::Options` ([#7673](https://github.com/moq-dev/moq/pull/7673)) + +`Cluster::with_stats` is removed, and `stats::Config::build` is no longer public. Pass the stats config to `Cluster::new` instead, which builds the producer on the cluster's origin and runs its publisher in the returned `Driver`: + +```rust +// Before +let (cluster, driver) = Cluster::new(options)?; +let (stats, publisher) = stats_config.build(cluster.origin.clone())?; +let cluster = cluster.with_stats(stats, publisher); + +// After +let (cluster, driver) = Cluster::new(options.with_stats(stats_config))?; +``` + +Read the registry from `cluster.stats`; `Relay::stats` still returns the producer. Publishing runs only while the driver is polled, which `Relay::run` does, and stops when the driver drops. Building stats needs no Tokio runtime, and an invalid stats config now fails `Cluster::new`. + ### The iroh transport is removed ([#7345](https://github.com/moq-dev/moq/pull/7345)) The `iroh` feature, the `[iroh]` table, the `--iroh-*` flags, and `Cluster::with_iroh` are gone. diff --git a/rs/moq-relay/benches/http_auth.rs b/rs/moq-relay/benches/http_auth.rs index 21d0f8bbe8c..910b7b1182f 100644 --- a/rs/moq-relay/benches/http_auth.rs +++ b/rs/moq-relay/benches/http_auth.rs @@ -2,15 +2,12 @@ //! holds waiting on the decider. A slope means admitting one request costs more as others //! wait, rather than what the request itself touches. //! The real web listener runs over loopback TCP; timings include one request round trip. -#![expect(clippy::disallowed_methods, reason = "host time measures benchmark throughput")] use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main}; +use futures::{StreamExt, stream::FuturesUnordered}; use moq_auth::{Grant, lease}; use moq_relay::{auth, cluster, web}; -use std::sync::{ - Arc, - atomic::{AtomicUsize, Ordering}, -}; +use std::task::Poll; use std::time::Duration; type Send = h2::client::SendRequest; @@ -30,25 +27,35 @@ impl Drop for Fixture { } impl Fixture { - async fn new(connections: usize, streams: usize) -> Self { + async fn new(connections: usize, streams: usize, server: bool) -> Self { let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); - let (auth, mut admissions) = auth::Auth::embedded("bench"); - let waiting = Arc::new(AtomicUsize::new(0)); - let decider = tokio::spawn({ + let waiting = kio::Shared::new(0usize); + let (auth, decider) = if server { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let mut config = auth::Config::default(); + config.url = Some(format!("http://{}/", listener.local_addr().unwrap()).parse().unwrap()); + let auth = config.init("bench", &Default::default(), false).unwrap(); + let app = axum::Router::new() + .route("/", axum::routing::post(decide)) + .with_state(waiting.clone()); + let decider = tokio::spawn(async move { axum::serve(listener, app).await.unwrap() }); + (auth, decider) + } else { + let (auth, mut admissions) = auth::Auth::embedded("bench"); let waiting = waiting.clone(); - async move { + let decider = tokio::spawn(async move { let mut held = Vec::new(); while let Some(admission) = admissions.next().await { if admission.request.path.starts_with("/held") { held.push(admission); - waiting.store(held.len(), Ordering::Release); + *waiting.lock() = held.len(); continue; } - let grant = Grant::new(moq_auth::Patterns::new(), moq_auth::Pattern::all()); - admission.grant(lease::Consumer::fixed(grant)); + admission.grant(lease::Consumer::fixed(grant())); } - } - }); + }); + (auth, decider) + }; let pem = rcgen::generate_simple_self_signed(vec!["localhost".into()]) .unwrap() @@ -58,7 +65,7 @@ impl Fixture { let mut config = web::Config::default(); config.http.listen = Some("127.0.0.1:0".parse().unwrap()); let (cluster, driver) = cluster::Cluster::new(Default::default()).unwrap(); - tokio::spawn(driver.run()); + let driver = tokio::spawn(async move { driver.run().await.unwrap() }); // No admission timeout, so held requests stay waiting however long setup and the // measurement take. let web = web::Web::new(auth, cluster, certificates, config) @@ -71,7 +78,7 @@ impl Fixture { let _ = web.serve(routes).await; }); - let mut tasks = vec![decider, server]; + let mut tasks = vec![decider, server, driver]; let mut held = Vec::new(); for connection in 0..connections { let mut send = connect(addr, &mut tasks).await; @@ -84,9 +91,25 @@ impl Fixture { held.push((send, responses)); } let measured = connect(addr, &mut tasks).await; - while waiting.load(Ordering::Acquire) < connections * streams { - tokio::time::sleep(Duration::from_millis(1)).await; + let mut responses: FuturesUnordered<_> = held + .iter_mut() + .flat_map(|(_, responses)| responses.iter_mut()) + .collect(); + tokio::select! { + biased; + result = responses.next(), if !responses.is_empty() => { + panic!("a held admission completed during setup: {result:?}"); + } + _ = waiting + .wait(|count| { + if **count == connections * streams { + Poll::Ready(()) + } else { + Poll::Pending + } + }) => {} } + drop(responses); Self { measured, @@ -106,6 +129,22 @@ impl Fixture { } } +fn grant() -> Grant { + Grant::new(moq_auth::Patterns::new(), moq_auth::Pattern::all()) +} + +async fn decide( + axum::extract::State(waiting): axum::extract::State>, + axum::Json(request): axum::Json, +) -> axum::Json { + if request.path.starts_with("/held") { + *waiting.lock() += 1; + std::future::pending().await + } else { + axum::Json(grant()) + } +} + async fn connect(addr: std::net::SocketAddr, tasks: &mut Vec>) -> Send { let tcp = tokio::net::TcpStream::connect(addr).await.unwrap(); let (send, connection) = h2::client::handshake(tcp).await.unwrap(); @@ -126,24 +165,38 @@ fn admission(c: &mut Criterion) { .enable_all() .build() .unwrap(); - let mut group = c.benchmark_group("relay_http_admission"); - group.sample_size(10); - group.warm_up_time(Duration::from_millis(100)); - group.measurement_time(Duration::from_millis(300)); - // Each connection may hold up to the verification budget (128 by default) at once. - for connections in [1, 16, 64] { - for streams in [0, 16, 64] { - let mut fixture = runtime.block_on(Fixture::new(connections, streams)); - group.bench_with_input( - BenchmarkId::new(format!("connections={connections}"), streams), - &(connections, streams), - |b, _| { - b.iter(|| runtime.block_on(fixture.round())); - }, - ); + for server in [false, true] { + let name = if server { + "relay_http_auth_server" + } else { + "relay_http_admission" + }; + let mut group = c.benchmark_group(name); + group.sample_size(10); + group.warm_up_time(Duration::from_millis(100)); + group.measurement_time(Duration::from_millis(300)); + // A held HTTP/1.1 auth POST takes two sockets, unlike a multiplexed web + // stream. Keep its full sweep below a process's standard 1024-fd budget. + let (connections, streams) = if server { + ([1, 4, 16], [0, 4, 16]) + } else { + ([1, 16, 64], [0, 16, 64]) + }; + // Each connection may hold up to the verification budget (128 by default) at once. + for connections in connections { + for streams in streams { + let mut fixture = runtime.block_on(Fixture::new(connections, streams, server)); + group.bench_with_input( + BenchmarkId::new(format!("connections={connections}"), streams), + &(connections, streams), + |b, _| { + b.iter(|| runtime.block_on(fixture.round())); + }, + ); + } } + group.finish(); } - group.finish(); } criterion_group!(benches, admission); diff --git a/rs/moq-relay/benches/session.rs b/rs/moq-relay/benches/session.rs index 7e3a9ed4220..730adc32264 100644 --- a/rs/moq-relay/benches/session.rs +++ b/rs/moq-relay/benches/session.rs @@ -43,13 +43,11 @@ impl Backend for Relay { options.config.id = Some(id); // The shared sweeps include more announcements than a default ordinary client. options.session.announces = options.peer.announces; - let (cluster, driver) = cluster::Cluster::new(options.with_cache(cache)).unwrap(); - tokio::spawn(driver.run()); let mut stats = stats::Config::default(); stats.enabled = true; stats.node = Some(id.to_string()); - let (producer, publisher) = stats.build(cluster.origin.clone()).unwrap(); - let cluster = cluster.with_stats(producer, publisher); + let (cluster, driver) = cluster::Cluster::new(options.with_cache(cache).with_stats(stats)).unwrap(); + tokio::spawn(driver.run()); let (auth, mut admissions) = auth::Auth::embedded(format!("bench-{id}")); let decider = tokio::spawn(async move { diff --git a/rs/moq-relay/src/auth.rs b/rs/moq-relay/src/auth.rs index b9a70aadfbd..34f40c64cc9 100644 --- a/rs/moq-relay/src/auth.rs +++ b/rs/moq-relay/src/auth.rs @@ -8,7 +8,9 @@ mod refresh; pub(crate) use caps::{Caps, Pacer}; -use std::sync::Arc; +use std::collections::VecDeque; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex, MutexGuard, PoisonError}; use axum::http; use moq_auth::{Bytes, Grant, Request, lease}; @@ -16,7 +18,7 @@ use moq_auth::{Pattern, Patterns}; use moq_net::{Path, stats::Tier}; use serde::{Deserialize, Serialize}; use serde_with::{OneOrMany, serde_as}; -use tokio::sync::{mpsc, oneshot}; +use tokio::sync::{Notify, oneshot}; use url::Url; /// The longest a decider may take to verify a token presented in band. A @@ -812,7 +814,7 @@ pub(crate) async fn hold(mut lease: Lease, work: impl std::future::Future), + Embedded(Arc), Server(moq_auth::Client), /// The public rules, as a token with an empty root. Public(Arc), @@ -820,55 +822,59 @@ enum Decider { } impl Decider { - fn decide(&self, mut admission: Admission) -> Result<(), Error> { + fn decide(&self, request: Request, connection: Option) -> Result { match self { - Self::Embedded(admissions) => admissions - .send(admission) - .map_err(|_| Error::Unavailable("nobody is answering admissions".into()))?, + Self::Embedded(admissions) => { + let (reply, answer) = oneshot::channel(); + let waiting = Arc::new(AtomicBool::new(true)); + admissions.0.push(Admission { + request, + connection, + reply, + waiting: waiting.clone(), + })?; + let abandon = Abandon { + queue: admissions.0.clone(), + waiting, + }; + Ok(Box::pin(async move { + let _abandon = abandon; + answer + .await + .map_err(|_| Error::Unavailable("the admission went unanswered".into()))? + })) + } Self::Server(client) => { let client = client.clone(); - // A panic drops the admission, which answers it as unavailable. - tokio::spawn(moq_tokio::panic::isolate(async move { - let ask = async { - match &admission.connection { - Some(connection) => client.attach(connection, admission.request.clone()).await, - None => client.connect(admission.request.clone()).await, - } - }; - // The relay stopped waiting (its handshake deadline passed): drop the - // request to the server, so an expired admission leaves nothing in flight. - let lease = tokio::select! { - biased; - () = admission.reply.closed() => return, - lease = ask => lease, - }; - match lease { - Ok(lease) => admission.grant(lease), - Err(err) => admission.refuse(err.into()), - } - })); + // The waiter owns the request, so dropping its answer cancels the HTTP + // exchange and a panic unwinds only the session being admitted. + Ok(Box::pin(async move { + Ok(match connection { + Some(connection) => client.attach(&connection, request).await?, + None => client.connect(request).await?, + }) + })) } // Nothing here can verify a token, so one is refused rather than ignored: // its holder expects it to count, and the public grant is not what it says. - Self::Public(_) if presents_token(&admission.request) => { - tracing::debug!(path = ?admission.request.path, "a token was presented to public rules"); - admission.refuse(Error::Refused(Default::default())); + Self::Public(_) if presents_token(&request) => { + tracing::debug!(path = ?request.path, "a token was presented to public rules"); + Ok(Box::pin(std::future::ready(Err(Error::Refused(Default::default()))))) } - Self::Public(rules) => match rules.authorize(&admission.request.path) { - Ok(access) => { - let grant = Grant::new(access.publish, access.subscribe); - admission.grant(lease::Consumer::fixed(grant)); - } - // A path the rules don't reach is a refusal, never an outage a - // client would retry. - Err(err) => { - tracing::debug!(path = ?admission.request.path, ?err, "public rules refused"); - admission.refuse(Error::Refused(Default::default())); - } - }, - Self::Refuse => admission.refuse(Error::Refused(Default::default())), + Self::Public(rules) => { + let result = match rules.authorize(&request.path) { + Ok(access) => Ok(lease::Consumer::fixed(Grant::new(access.publish, access.subscribe))), + // A path the rules don't reach is a refusal, never an outage a + // client would retry. + Err(err) => { + tracing::debug!(path = ?request.path, ?err, "public rules refused"); + Err(Error::Refused(Default::default())) + } + }; + Ok(Box::pin(std::future::ready(result))) + } + Self::Refuse => Ok(Box::pin(std::future::ready(Err(Error::Refused(Default::default()))))), } - Ok(()) } } @@ -900,8 +906,17 @@ impl Auth { /// returned [`Admissions`] and waits for its [`Admission`] to be granted or /// refused. Dropping the `Admissions` fails every later session as unavailable. pub fn embedded(node: impl Into) -> (Self, Admissions) { - let (sender, receiver) = mpsc::unbounded_channel(); - (Self::new(node, Decider::Embedded(sender)), Admissions(receiver)) + let queue = Arc::new(Queue { + queued: Mutex::new(Queued { + pending: VecDeque::new(), + abandoned: 0, + closed: false, + dropped: false, + }), + ready: Notify::new(), + }); + let submitter = Arc::new(Submitter(queue.clone())); + (Self::new(node, Decider::Embedded(submitter)), Admissions(queue)) } fn new(node: impl Into, decider: Decider) -> Self { @@ -984,13 +999,7 @@ impl Auth { /// Hand one admission to the decider, returning its pending answer. fn submit(&self, request: Request, connection: Option) -> Result { - let (reply, answer) = oneshot::channel(); - self.decider.decide(Admission { - request, - connection, - reply, - })?; - Ok(answer) + self.decider.decide(request, connection) } /// Admit a session on a grant decided locally, bypassing the server: the LAN @@ -1007,7 +1016,7 @@ fn navigates(path: &str) -> bool { } /// A decider's pending answer to one [`Admission`]. -type Answer = oneshot::Receiver>; +type Answer = futures::future::BoxFuture<'static, Result>; fn timed_out() -> Error { Error::Unavailable("the admission timed out".into()) @@ -1016,10 +1025,8 @@ fn timed_out() -> Error { /// Check a decider's answer against the contract every decider is held to: a /// grant that admits nothing or asks for a re-check without a bound is a bug, not /// a refusal. Returns the grant snapshot it checked. -fn answered_with( - answered: Result, oneshot::error::RecvError>, -) -> Result<(lease::Consumer, Grant), Error> { - let consumer = answered.map_err(|_| Error::Unavailable("the admission went unanswered".into()))??; +fn answered_with(answered: Result) -> Result<(lease::Consumer, Grant), Error> { + let consumer = answered?; let grant = consumer.grant(); grant.validate()?; Ok((consumer, grant)) @@ -1047,12 +1054,139 @@ fn still_valid(grant: &Grant) -> Result<(), lease::Reason> { /// slot before they reach the decider. /// Answer in place or hand each [`Admission`] to its own task; nothing here /// serializes them. -pub struct Admissions(mpsc::UnboundedReceiver); +/// +/// An admission whose session stopped waiting is skipped, never handed out, and +/// the queue sweeps them out as waiters give up, so a slow or stalled decider holds +/// memory for the sessions still waiting rather than every one that gave up. +pub struct Admissions(Arc); impl Admissions { /// The next session to decide, or `None` once every clone of the [`Auth`] is gone. pub async fn next(&mut self) -> Option { - self.0.recv().await + loop { + // A push before this waits stores a permit, so it is never missed. + let ready = self.0.ready.notified(); + { + let mut queued = self.0.lock(); + while let Some(admission) = queued.pending.pop_front() { + // A waiter that gave up already cleared the flag. Claiming the flag + // here stops its drop from counting it as abandoned. + if admission.waiting.swap(false, Ordering::AcqRel) { + return Some(admission); + } + } + if queued.closed { + return None; + } + } + ready.await; + } + } +} + +impl Drop for Admissions { + fn drop(&mut self) { + // Fail every waiting session as unanswered, and every later one as unavailable. + let pending = { + let mut queued = self.0.lock(); + queued.dropped = true; + std::mem::take(&mut queued.pending) + }; + drop(pending); + } +} + +/// The admissions an embedded [`Auth`] has submitted and its [`Admissions`] has +/// yet to take. +struct Queue { + queued: Mutex, + /// Wakes [`Admissions::next`] for a new admission or the last [`Auth`] dropping. + ready: Notify, +} + +struct Queued { + pending: VecDeque, + /// Waiters that gave up since the last sweep: at least as many as the abandoned + /// admissions left in `pending`. + abandoned: usize, + /// Every clone of the [`Auth`] is gone, so no admission arrives again. + closed: bool, + /// The [`Admissions`] is gone, so no admission is answered again. + dropped: bool, +} + +impl Queue { + fn lock(&self) -> MutexGuard<'_, Queued> { + self.queued.lock().unwrap_or_else(PoisonError::into_inner) + } + + /// Queue `admission`, failing when nothing will answer it. + fn push(&self, admission: Admission) -> Result<(), Error> { + { + let mut queued = self.lock(); + if queued.dropped { + return Err(Error::Unavailable("nobody is answering admissions".into())); + } + queued.pending.push_back(admission); + } + self.ready.notify_one(); + Ok(()) + } + + /// Count one waiter giving up, sweeping out the abandoned admissions once they + /// may be half the queue. + /// + /// Abandoned admissions so stay at most half the queue after each give-up, + /// however large a past burst was and whether or not anything is pushed later. + /// A sweep costs the queue's length, paid for by the give-ups counted before it. + fn abandon(&self) { + let abandoned = { + let mut queued = self.lock(); + if queued.pending.is_empty() { + return; + } + queued.abandoned += 1; + if queued.abandoned < queued.pending.len().div_ceil(2) { + return; + } + queued.abandoned = 0; + let (waiting, abandoned): (VecDeque<_>, _) = std::mem::take(&mut queued.pending) + .into_iter() + .partition(|admission| admission.waiting.load(Ordering::Acquire)); + queued.pending = waiting; + abandoned + }; + // Dropped outside the lock, like every other admission the queue releases. + drop(abandoned); + } +} + +/// Marks a queued admission abandoned once its waiter stops waiting. +/// +/// The guard rides in the answer future, so dropping that future, even before its +/// first poll, reports the give-up without waiting on a later push. +struct Abandon { + queue: Arc, + waiting: Arc, +} + +impl Drop for Abandon { + fn drop(&mut self) { + // An admission already handed out is no longer the queue's to sweep. + if self.waiting.swap(false, Ordering::AcqRel) { + self.queue.abandon(); + } + } +} + +/// The handle every clone of an embedded [`Auth`] shares: the last one dropping +/// ends [`Admissions::next`]. +struct Submitter(Arc); + +impl Drop for Submitter { + fn drop(&mut self) { + self.0.lock().closed = true; + self.0.ready.notify_one(); } } @@ -1064,6 +1198,8 @@ pub struct Admission { pub request: Request, connection: Option, reply: oneshot::Sender>, + /// Whether its waiter still wants it: cleared when handed out or abandoned. + waiting: Arc, } #[cfg(test)] @@ -1183,6 +1319,149 @@ pub(crate) fn peer(identity: &moq_tokio::tls::PeerIdentity) -> Option accepted.unwrap(), + result = &mut admission => panic!("admitted before the server accepted: {:?}", result.err()), + }; + let mut buffer = [0; 1024]; + let received = tokio::select! { + received = socket.read(&mut buffer) => received.unwrap(), + result = &mut admission => panic!("admitted before the server replied: {:?}", result.err()), + }; + assert_ne!(received, 0, "the caller drove an HTTP request"); + drop(admission); + // A request awaiting its response cannot return its connection to the pool. + // Cancelling it closes that connection even while the auth server is stalled. + loop { + match socket.read(&mut buffer).await { + Ok(0) => break, + Ok(_) => {} + Err(err) if err.kind() == std::io::ErrorKind::ConnectionReset => break, + Err(err) => panic!("reading the cancelled request: {err}"), + } + } + } + + fn submit_embedded(auth: &Auth, path: &str) -> Answer { + auth.submit(auth.request(moq_auth::Transport::Http, path), None) + .unwrap() + } + + #[tokio::test] + async fn an_abandoned_embedded_admission_is_skipped() { + let (auth, mut admissions) = Auth::embedded("test"); + drop(submit_embedded(&auth, "/gone")); + let waiting = submit_embedded(&auth, "/waiting"); + let admission = admissions.next().await.unwrap(); + assert_eq!( + admission.request.path, "/waiting", + "the decider never sees a session that gave up" + ); + admission.refuse(Error::Refused(Default::default())); + assert!(matches!(waiting.await, Err(Error::Refused(_)))); + } + + #[tokio::test] + async fn abandoned_embedded_admissions_are_pruned_without_a_decider() { + let (auth, mut admissions) = Auth::embedded("test"); + let waiting: Vec<_> = (0..32) + .map(|i| submit_embedded(&auth, &format!("/waiting/{i}"))) + .collect(); + for i in 0..1024 { + drop(submit_embedded(&auth, &format!("/gone/{i}"))); + let queued = admissions.0.lock().pending.len(); + assert!(queued <= 2 * waiting.len(), "{queued} admissions queued"); + } + for i in 0..waiting.len() { + let admission = admissions.next().await.unwrap(); + assert_eq!( + admission.request.path, + format!("/waiting/{i}"), + "waiting sessions keep their order" + ); + } + drop(waiting); + assert!( + admissions + .0 + .lock() + .pending + .iter() + .all(|admission| admission.reply.is_closed()) + ); + } + + #[tokio::test] + async fn a_cancelled_burst_is_reclaimed_without_a_later_push() { + let (auth, admissions) = Auth::embedded("test"); + // A stalled decider takes none of the burst, then every session times out + // and nothing is pushed afterwards. + let burst: Vec<_> = (0..1024) + .map(|i| submit_embedded(&auth, &format!("/burst/{i}"))) + .collect(); + assert_eq!(admissions.0.lock().pending.len(), 1024); + drop(burst); + assert!( + admissions.0.lock().pending.is_empty(), + "a burst whose sessions all gave up is reclaimed" + ); + } + + #[tokio::test] + async fn dropping_embedded_admissions_fails_waiting_and_later_sessions() { + let (auth, admissions) = Auth::embedded("test"); + let waiting = submit_embedded(&auth, "/waiting"); + drop(admissions); + assert!(matches!(waiting.await, Err(Error::Unavailable(_)))); + let later = auth.submit(auth.request(moq_auth::Transport::Http, "/later"), None); + assert!(matches!(later.err(), Some(Error::Unavailable(_)))); + } + + #[tokio::test] + async fn embedded_admissions_end_once_every_auth_is_gone() { + let (auth, mut admissions) = Auth::embedded("test"); + let clone = auth.clone(); + let next = tokio::spawn(async move { admissions.next().await.is_none() }); + tokio::task::yield_now().await; + drop(auth); + tokio::task::yield_now().await; + assert!(!next.is_finished(), "a clone can still submit"); + drop(clone); + assert!(next.await.unwrap(), "no admission arrives once every clone is gone"); + } + + #[tokio::test] + async fn queued_embedded_admissions_survive_the_last_auth() { + let (auth, mut admissions) = Auth::embedded("test"); + let waiting = submit_embedded(&auth, "/waiting"); + drop(auth); + let admission = admissions + .next() + .await + .expect("a queued admission outlives the last Auth"); + assert_eq!(admission.request.path, "/waiting"); + admission.refuse(Error::Refused(Default::default())); + assert!(matches!(waiting.await, Err(Error::Refused(_)))); + assert!(admissions.next().await.is_none()); + } + fn patterns(texts: &[&str]) -> moq_auth::Patterns { texts.iter().map(|text| text.parse().unwrap()).collect() } diff --git a/rs/moq-relay/src/auth/admission_tests.rs b/rs/moq-relay/src/auth/admission_tests.rs index 4d1b08a96f1..c010b7281b7 100644 --- a/rs/moq-relay/src/auth/admission_tests.rs +++ b/rs/moq-relay/src/auth/admission_tests.rs @@ -150,11 +150,12 @@ async fn a_moved_gateway_tier_retags_its_stats() { #[tokio::test(start_paused = true)] async fn gateway_admission_refuses_once_in_the_shared_stats_registry() { - let cluster = cluster::Cluster::spawn(cluster::Options::default()).unwrap(); - let (producer, publisher) = - moq_stats::Producer::new(moq_stats::produce::Config::new().with_origin(cluster.origin.clone())); - let registry = producer.registry().clone(); - let cluster = cluster.with_stats(producer, publisher); + let cluster = cluster::Cluster::spawn(cluster::Options::default().with_stats(crate::stats::Config { + enabled: true, + ..Default::default() + })) + .unwrap(); + let registry = cluster.stats.clone(); let (auth, mut admissions) = auth::Auth::embedded("edge"); let (result, ()) = tokio::join!( cluster.admit(&auth, auth.request(Transport::Srt, "/unattributed")), @@ -184,14 +185,12 @@ async fn gateway_admission_refuses_once_in_the_shared_stats_registry() { #[tokio::test] async fn embedded_expired_refusals_keep_verified_root_and_tier() { tokio::time::pause(); - let cluster = cluster::Cluster::spawn(cluster::Options::default()).unwrap(); - let config = crate::stats::Config { + let cluster = cluster::Cluster::spawn(cluster::Options::default().with_stats(crate::stats::Config { enabled: true, ..Default::default() - }; - let (producer, publisher) = config.build(cluster.origin.clone()).unwrap(); - let registry = producer.registry().clone(); - let cluster = cluster.with_stats(producer, publisher); + })) + .unwrap(); + let registry = cluster.stats.clone(); let (auth, mut admissions) = auth::Auth::embedded("edge"); for tier in [None, Some("edge")] { let mut refusal = moq_auth::Refusal::default(); diff --git a/rs/moq-relay/src/cluster.rs b/rs/moq-relay/src/cluster.rs index db4f1fd25ed..390ac649e4b 100644 --- a/rs/moq-relay/src/cluster.rs +++ b/rs/moq-relay/src/cluster.rs @@ -981,11 +981,13 @@ struct LanAuth { credential: String, } -/// Construction settings for a [`Cluster`]: identity, discovery, and the origin cache. +/// Construction settings for a [`Cluster`]: identity, discovery, the origin cache, +/// and stats publishing. /// /// The origin is built once from these, so cache settings cannot detach a handle -/// taken after construction. Independent services ([`Cluster::with_connect`], -/// [`Cluster::with_stats`]) attach afterwards without rebuilding it. +/// taken after construction, and the stats publisher joins the [`Driver`] from the +/// start. Independent services ([`Cluster::with_connect`]) attach afterwards +/// without rebuilding it. #[derive(Default)] #[non_exhaustive] pub struct Options { @@ -1001,6 +1003,9 @@ pub struct Options { /// `None` uses an unbounded pool with the standard LRU window and no /// media-timestamp ceiling. pub cache: Option, + + /// Stats publishing on the cluster's origin. Disabled by default. + pub stats: crate::stats::Config, } impl Options { @@ -1017,10 +1022,16 @@ impl Options { self.cache = Some(cache); self } + + /// Publish stats on the cluster's origin with this config. + pub fn with_stats(mut self, stats: crate::stats::Config) -> Self { + self.stats = stats; + self + } } -/// The work a [`Cluster`] needs to serve at all: its origin's lifecycle driver and, -/// when the cache has one, the headroom governor. Returned by [`Cluster::new`]. +/// The work a [`Cluster`] needs to serve: its origin's lifecycle driver, cache +/// governor, and stats publisher. Returned by [`Cluster::new`]. /// /// Nothing runs on its own: poll it, or await [`run`](Self::run), for as long as the /// cluster serves. Dropping it tears the origin down, failing every later mutation. @@ -1085,9 +1096,9 @@ impl Started { } /// Run the cluster's dials and peer sources on this task until they stop, or - /// until one of them or a task the relay cannot serve without (its stats - /// publisher, ACME renewal, or certificate reload) panics. The origin's own work - /// is the [`Driver`]'s. + /// until one of them or a task the relay cannot serve without (its + /// ACME renewal or certificate reload) panics. The origin, cache governor, and + /// stats publisher are the [`Driver`]'s work. /// /// Standalone, this only waits for such a panic. Returning, a failure /// included, closes every dial as [`Cluster::close`] does. A cancelled `run` @@ -1223,10 +1234,12 @@ fn dialer<'a>(cluster: &'a Cluster, token: &'a str) -> impl FnMut(DialTarget) -> /// origin. Loop prevention and route preference come from the hop list carried /// on each announced route (see [`moq_net::origin::Route`]). /// -/// Construct with [`Cluster::new`], then attach a dial template and (optionally) -/// a [`stats::Registry`](moq_net::stats::Registry) with the `with_*` builder -/// methods. Those builders do not rebuild the origin. A cluster without a -/// dial template can serve local sessions but cannot dial remote peers. +/// Construct with [`Cluster::new`]: its [`Options`] carry the limits, cache, and +/// stats ([`Options::with_stats`]), and [`Cluster::stats`] is the resulting +/// registry. Then attach a dial template with [`Cluster::with_connect`] and a LAN +/// advertisement with [`Cluster::with_advertise`]; neither rebuilds the origin. A +/// cluster without a dial template can serve local sessions but cannot dial +/// remote peers. #[derive(Clone)] pub struct Cluster { session_limits: moq_net::session::Limits, @@ -1260,8 +1273,8 @@ pub struct Cluster { /// view always points at the same session in the logs. connection_ids: Arc, - /// The stats publisher, ACME renewal, and certificate reload: a panic in any - /// of them stops [`Relay::run`](crate::Relay::run). + /// ACME renewal and certificate reload: a panic in either stops + /// [`Relay::run`](crate::Relay::run). pub(crate) critical: crate::critical::Critical, /// All broadcasts, local and remote. Downstream sessions read from here @@ -1271,16 +1284,15 @@ pub struct Cluster { /// Stats registry. One instance per relay; sessions pick a billing tier via /// [`stats::Registry::tier`](moq_net::stats::Registry::tier) at acceptance time /// (the default tier unless configured otherwise, or any label the auth API - /// returns) so traffic classes land in separate counter sets. Defaults - /// to a disabled (no-op) registry until [`with_stats`](Self::with_stats) is called. + /// returns) so traffic classes land in separate counter sets. A disabled + /// (no-op) registry unless [`Options::stats`] enabled publishing. pub stats: moq_net::stats::Registry, - /// Keeps the stats publish task running for as long as any handle on this - /// cluster lives. The task holds only a `Weak` to its producer, so it stops + /// Keeps the stats publisher alive while any cluster handle lives and its + /// driver runs. The publisher holds only a `Weak` to its producer, so it stops /// when the last [`moq_stats::Producer`] clone drops; parking one here means - /// serving and publishing end together, instead of publishing ending early - /// because a caller dropped a producer it never asked for. - _stats_publisher: Option, + /// serving and publishing end together. + pub(crate) stats_producer: moq_stats::Producer, } /// A gateway or network session admitted with its lease, scoped origins, and stats. @@ -1325,29 +1337,31 @@ impl Cluster { self.origin.hop().id() } - /// Creates a cluster with one origin, using [`Options`] for identity - /// and cache. + /// Creates a cluster with one origin, using [`Options`] for identity, cache, + /// and stats publishing. /// /// Use [`with_connect`](Self::with_connect) to enable dialing remote peers - /// (required when `config.connect` is non-empty), and - /// [`with_stats`](Self::with_stats) to enable metrics publishing. Those - /// builders do not rebuild the origin. + /// (required when `config.connect` is non-empty). That builder does not + /// rebuild the origin. /// - /// Returns the cluster with its [`Driver`]: the origin's lifecycle work and the - /// cache governor, which run only while the owner polls it. Poll it for as long - /// as the cluster serves, whether or not [`start`](Self::start) (the mesh half) - /// is ever called; [`Relay::run`](crate::Relay::run) does. Dropping it tears the - /// origin down. + /// Returns the cluster with its [`Driver`]: the origin's lifecycle work, the + /// cache governor, and the stats publisher. They run only while the owner polls + /// it. Poll it for as long as the cluster serves, whether or not + /// [`start`](Self::start) (the mesh half) is ever called; + /// [`Relay::run`](crate::Relay::run) does. Dropping it tears the origin down + /// and stops publishing stats. A stats publisher panic ends the driver like + /// its other work. Needs no Tokio runtime. /// /// Errors if `config.id` is set to 0. An unset id picks a fresh random origin. - /// Errors too if the `peer.quic` receive window does not fit a QUIC varint, or - /// `config.tier` is not a valid stats tier. + /// Errors too if the `peer.quic` receive window does not fit a QUIC varint, + /// `config.tier` is not a valid stats tier, or the stats config is invalid. pub fn new(options: Options) -> anyhow::Result<(Self, Driver)> { let Options { config, cache, session, peer, + stats, } = options; let id = match config.id { Some(id) => Hop::new(id).context("--cluster-id must be non-zero")?, @@ -1380,6 +1394,10 @@ impl Cluster { work.push(Child::new("origin driver", async move { moq_net::time::run(driver).await; })); + // Publish through the same origin cluster peers read from. + let (stats_producer, publisher) = stats.build(origin.clone())?; + work.push(Child::new("stats publisher", publisher.run())); + let stats = stats_producer.registry().clone(); let nodes = crate::nodes::Nodes::default(); tracing::info!(hop_id = %origin.hop(), configured = config.id.is_some(), "cluster initialized"); let cluster = Cluster { @@ -1395,12 +1413,12 @@ impl Cluster { #[cfg(feature = "cluster-lan")] lan_auth: Arc::new(std::sync::OnceLock::new()), nodes, - refusals: Default::default(), + refusals: crate::refusals::Refusals::new(&stats), connection_ids: Arc::default(), critical: Default::default(), origin, - stats: moq_net::stats::Registry::disabled(), - _stats_publisher: None, + stats, + stats_producer, }; Ok((cluster, Driver { work })) } @@ -1545,23 +1563,6 @@ impl Cluster { self.connection_ids.fetch_add(1, Ordering::Relaxed) } - /// Attach a stats producer, replacing the default disabled registry with its - /// registry, and run its publisher. - /// - /// Build both with [`stats::Config::build`](crate::stats::Config::build), passing - /// [`Self::origin`] so it publishes through the same origin cluster peers read - /// from. The cluster holds the producer from here on, so the caller may drop - /// its own handle: publishing lasts exactly as long as the cluster does. A - /// panic in the publisher ends [`Relay::run`](crate::Relay::run), like the - /// cluster's other critical tasks. Must be called within a Tokio runtime. - pub fn with_stats(mut self, stats: moq_stats::Producer, publisher: moq_stats::Publisher) -> Self { - self.critical.spawn("stats publisher", publisher.run()); - self.stats = stats.registry().clone(); - self.refusals = crate::refusals::Refusals::new(&self.stats); - self._stats_publisher = Some(stats); - self - } - /// Billing tier cluster-peer traffic records under (`--cluster-tier`). /// An absent or empty label selects the default unprefixed tier. fn cluster_tier(&self) -> Tier { @@ -2598,28 +2599,66 @@ mod tests { } } + fn with_stats() -> Options { + Options::default().with_stats(crate::stats::Config { + enabled: true, + ..Default::default() + }) + } + + #[test] + fn stats_build_without_a_runtime() { + let (cluster, driver) = Cluster::new(with_stats()).unwrap(); + assert!(cluster.stats.is_enabled()); + drop((cluster, driver)); + } + + #[test] + fn invalid_stats_config_errors() { + let options = Options::default().with_stats(crate::stats::Config { + enabled: true, + interval: Some(Duration::ZERO.into()), + ..Default::default() + }); + let err = Cluster::new(options).err().expect("a zero interval is refused"); + assert!(format!("{err}").contains("stats.interval"), "got: {err}"); + } + + #[kio::test] + async fn stats_end_with_the_driver() { + let (cluster, driver) = Cluster::new(with_stats()).unwrap(); + let consumer = cluster.origin.consume(); + let path = Path::from_static(".stats/node"); + let announced = async { consumer.follow(&path, None).unwrap().available().await.unwrap() }; + let futures::future::Either::Right((broadcast, driver)) = + futures::future::select(Box::pin(driver.run()), Box::pin(announced)).await + else { + panic!("the driver stopped before publishing stats"); + }; + assert!(!broadcast.is_closed()); + drop(driver); + assert!(broadcast.is_closed(), "dropping the driver stopped the publisher"); + drop(cluster); + } + /// The publish task holds only a `Weak` to its producer, so it stops when the - /// last `moq_stats::Producer` clone drops. Attaching one must therefore hand - /// its lifetime to the cluster: an embedder clones handles off a `Relay` and - /// does not have to keep the producer, and a relay that keeps serving while - /// silently publishing nothing is exactly the class of failure this API - /// exists to rule out. Moving the ONLY handle in must keep it alive. + /// last `moq_stats::Producer` clone drops. The cluster must therefore hold the + /// producer itself: a relay that keeps serving while silently publishing + /// nothing is exactly the class of failure this rules out. /// /// Asserts the broadcast is still live AFTER a few publish intervals, not /// merely that one appeared: the task publishes once before it can notice its /// producer is gone, so a first announcement races through either way and /// proves nothing. #[tokio::test(start_paused = true)] - async fn stats_publishing_outlives_the_producer_handle() { + async fn stats_publishing_lasts_with_the_cluster() { let config = crate::stats::Config { enabled: true, node: Some("test".to_string()), ..Default::default() }; - let cluster = new_cluster(Config::default()).expect("cluster"); - let (stats, publisher) = config.build(cluster.origin.clone()).expect("stats"); - let cluster = cluster.with_stats(stats, publisher); + let cluster = Cluster::spawn(Options::default().with_stats(config)).expect("cluster"); let path = moq_net::Path::from_static(".stats/node/test"); let consumer = cluster.origin.consume(); @@ -2632,9 +2671,9 @@ mod tests { .expect("stats broadcast resolves"); assert!(broadcast.info().instance.is_some(), "stats announce under an instance"); - // Several publish intervals (1s each by default) after the handle went out - // of scope, the task is still running and its broadcast still announced: a - // request refuses a front whose route retracted. + // Several publish intervals (1s each by default) later, the task is still + // running and its broadcast still announced: a request refuses a front whose + // route retracted. kio::clock::sleep(std::time::Duration::from_millis(2500)).await; let still_live = consumer.request(&path, None).await; assert!( @@ -3243,7 +3282,12 @@ mod tests { id: Some(42), ..Default::default() }) - .with_cache(cache), + .with_cache(cache) + .with_stats(crate::stats::Config { + enabled: true, + node: Some("test".to_string()), + ..Default::default() + }), ) .expect("cluster"); @@ -3252,19 +3296,6 @@ mod tests { assert_eq!(origin.config().cache_duration, duration); assert_eq!(origin.config().pool.expiry(), Some(duration)); - let (stats, publisher) = crate::stats::Config { - enabled: true, - node: Some("test".to_string()), - ..Default::default() - } - .build(origin.clone()) - .expect("stats"); - let cluster = cluster.with_stats(stats, publisher); - - assert_eq!(cluster.origin.hop().id(), origin.hop().id()); - assert_eq!(cluster.origin.config().cache_duration, duration); - assert_eq!(cluster.origin.config().pool.expiry(), Some(duration)); - let broadcast = origin.create_broadcast("cam").expect("create"); broadcast.announce(Default::default()).expect("announce"); let mut track = broadcast.create_track("data", timed()).expect("track"); diff --git a/rs/moq-relay/src/config.rs b/rs/moq-relay/src/config.rs index d2468acb045..1ee853cfac8 100644 --- a/rs/moq-relay/src/config.rs +++ b/rs/moq-relay/src/config.rs @@ -902,8 +902,7 @@ root = ["ca.pem"] ]; let config = Config::parse_and_merge(args).expect("config load"); assert!(config.stats.enabled); - let cluster = cluster::Cluster::spawn(cluster::Options::default()).expect("cluster"); - let _ = config.stats.build(cluster.origin.clone()).expect("stats build"); + let _ = cluster::Cluster::new(cluster::Options::default().with_stats(config.stats)).expect("stats build"); } /// Bare defaults loaded from TOML survive when the CLI does not mention them. diff --git a/rs/moq-relay/src/relay.rs b/rs/moq-relay/src/relay.rs index d3ceb4fc696..e63f59b41e5 100644 --- a/rs/moq-relay/src/relay.rs +++ b/rs/moq-relay/src/relay.rs @@ -89,10 +89,9 @@ pub struct Relay { /// config named a source; `run` refuses to start while this is still held. admissions: Option, cluster: cluster::Cluster, - /// The cluster's origin and cache work, polled for as long as [`Self::run`] + /// The cluster's origin, cache, and stats work, polled for as long as [`Self::run`] /// runs. `None` only once `run` took it. driver: Option, - stats: moq_stats::Producer, internal: internal::Internal, web: web::Web, /// The ACME client, issuing before [`Self::run`] serves and renewing after. @@ -137,7 +136,7 @@ impl Relay { /// reading key material), so a returned `Relay` reports its ephemeral ports and /// a taken port fails here. Nothing runs yet: the cluster's /// [`Driver`](crate::cluster::Driver), which holds the origin's lifecycle work and - /// the cache governor, is only stored, and no session is admitted until + /// the cache governor and stats publisher, is only stored, and no session is admitted until /// [`Self::run`] drives both. pub async fn load(config: Config) -> anyhow::Result { // Keep the large debug state machine off the embedder's main-thread stack. @@ -336,17 +335,15 @@ impl Relay { if generated && let Some(fingerprint) = certificates.fingerprints().into_iter().next() { advertise = advertise.with_fingerprint(fingerprint); } - let mut cluster_options = cluster::Options::new(config.cluster).with_cache(cache); + let mut cluster_options = cluster::Options::new(config.cluster) + .with_cache(cache) + .with_stats(config.stats); cluster_options.session = config.limits.client; cluster_options.peer = config.limits.peer; let (cluster, driver) = cluster::Cluster::new(cluster_options)?; let cluster = cluster .with_connect(config.connect.clone(), config.quic.clone()) .with_advertise(advertise); - let (stats, publisher) = config.stats.build(cluster.origin.clone())?; - // The cluster takes over keeping the publish task alive, so an embedder that - // drops the producer keeps publishing for as long as it serves. - let cluster = cluster.with_stats(stats.clone(), publisher); let sessions = crate::session::Registry::new(); let (ready, _) = tokio::sync::watch::channel(Startup::Starting); @@ -411,7 +408,6 @@ impl Relay { admissions, cluster, driver: Some(driver), - stats, internal, web, #[cfg(feature = "acme")] @@ -506,11 +502,11 @@ impl Relay { &self.cluster } - /// The stats producer, for publishing extra counters through the same - /// registry. [`Self::cluster`] holds a clone, so the publish task lives as - /// long as the cluster whether or not this handle is kept. + /// The stats producer, for publishing extra counters through the same registry. + /// The cluster holds it, so publishing lasts as long as the cluster and its + /// driver do, whether or not this handle is cloned. pub fn stats(&self) -> &moq_stats::Producer { - &self.stats + &self.cluster.stats_producer } /// Graceful-shutdown signal shared by every accepted session and web handler. diff --git a/rs/moq-relay/src/stats.rs b/rs/moq-relay/src/stats.rs index 2d5fc208a80..a51ec2bf361 100644 --- a/rs/moq-relay/src/stats.rs +++ b/rs/moq-relay/src/stats.rs @@ -114,12 +114,14 @@ impl Config { /// Build a [`moq_stats::Producer`] and its [`moq_stats::Publisher`] from this /// config, publishing on `origin`. /// - /// Returns a no-op producer when [`Self::enabled`] is false, so the relay can - /// attach the result unconditionally. Hand both to - /// [`Cluster::with_stats`](crate::cluster::Cluster::with_stats), which takes over - /// the registry, running the publisher, and keeping it alive (it stops when the - /// last clone of the producer drops). Fails on a zero interval. - pub fn build(&self, origin: origin::Producer) -> anyhow::Result<(moq_stats::Producer, moq_stats::Publisher)> { + /// Returns a no-op producer when [`Self::enabled`] is false, so + /// [`Cluster::new`](crate::cluster::Cluster::new) can build one unconditionally, + /// keep the producer, and run the publisher on its driver. Fails on a zero + /// interval. + pub(crate) fn build( + &self, + origin: origin::Producer, + ) -> anyhow::Result<(moq_stats::Producer, moq_stats::Publisher)> { if !self.enabled { return Ok(moq_stats::Producer::new(moq_stats::produce::Config::new())); } diff --git a/rs/moq-relay/tests/integration/connection.rs b/rs/moq-relay/tests/integration/connection.rs index 77973741ec9..1131818bf43 100644 --- a/rs/moq-relay/tests/integration/connection.rs +++ b/rs/moq-relay/tests/integration/connection.rs @@ -269,11 +269,10 @@ async fn the_setup_path_reaches_the_decider_and_roots_the_grant() { /// by its reason, under the default tier. #[tokio::test(start_paused = true)] async fn stats_count_admitted_and_refused_sessions() { - let cluster = memory::cluster(); - let (producer, publisher) = - moq_stats::Producer::new(moq_stats::produce::Config::new().with_origin(cluster.origin.clone())); - let registry = producer.registry().clone(); - let cluster = cluster.with_stats(producer, publisher); + let mut config = moq_relay::stats::Config::default(); + config.enabled = true; + let cluster = crate::spawn_cluster(moq_relay::cluster::Options::default().with_stats(config)); + let registry = cluster.stats.clone(); let presence = || { registry .snapshot()