diff --git a/quest/m1/README.md b/quest/m1/README.md index 13b46b1670d..8b9593abed6 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -174,7 +174,6 @@ per-frame CPU growth since #5926. - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature - [Authored tracks default to a 30 s max-age](/quest/m1/max-age-defaults.md) - raw and media tracks advertise 30 s unless their author chooses otherwise, with explicit unlimited in every binding - [HLS export lists a same-codec reconfigure's first group under the new init](/quest/m1/hls-reinit-same-codec.md) - the hand-over pick, whichever of the catalog and the records arrives first, after the catalog `hls` removal is planned -- [A cancelled MoqTrackDemand wait never loses the next change](/quest/m1/demand-cancel-race.md) - cancelling a `MoqTrackDemand::wanted_changed` call, from any binding, never consumes a demand change - [One Ed25519 public key type with one z-base-32 codec](/quest/m1/mesh-key-codec.md) - `moq_auth::mesh::EndpointId` is merged into `moq_crypto::sign::PublicKey` - [Ordered reads can skip owed holes, and snapshot latest() does](/quest/m1/ordered-owed-holes.md) - moq-net's `Ordered` gains a read that skips past an owed hole to the newest buffered group, while keeping the reader's bounds - [A Rust relay never replays from group 0 to a lite-03/04 subscriber](/quest/m1/relay-inflight-replay.md) - A Rust relay or publisher serving a lite-03 or lite-04 SUBSCRIBE from a track copy that is still resolving starts at the newest group once the copy resolves diff --git a/quest/m1/demand-cancel-race.md b/quest/m1/demand-cancel-race.md deleted file mode 100644 index 663a4fe958d..00000000000 --- a/quest/m1/demand-cancel-race.md +++ /dev/null @@ -1,31 +0,0 @@ -# [S] A cancelled MoqTrackDemand wait never loses the next change - -## Goal - -Cancelling a `MoqTrackDemand::wanted_changed` call, from any binding, never -consumes a demand change: the next call returns that change instead of -waiting for the one after it. `MoqFill::wanted_changed` already behaves this -way. - -## Plan - -Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 -/quest-complete run), from -[moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036). - -Milestone: ✅ m1 (moved from m2). Not chosen: m2. - -- `MoqTrackDemand::wanted_changed` (`rs/moq-ffi/src/demand.rs:45`) runs the - watch through `crate::ffi::detached`, a spawned task. A call cancelled - before the binding frees it leaves that task running; it advances the - shared watcher and its result is dropped, so the change is lost. -- `MoqFill::wanted_changed` (`rs/moq-ffi/src/producer.rs:511`, at - be39188000) fixed the same race by driving the watch in place through - `crate::ffi::Task::run` ("a cancelled call left unfreed must not advance - the watcher"). Use the same - shape for `MoqTrackDemand`, and check the other `detached` watchers in - moq-ffi and moq-c's demand callbacks for the same pattern. -- Test: start a wait, cancel it, change demand, and assert the next call - returns the change at once. - -Public API: none. Wire: none. diff --git a/quest/m1/poll-first/no-spawn/README.md b/quest/m1/poll-first/no-spawn/README.md index 07487d518da..75f629befad 100644 --- a/quest/m1/poll-first/no-spawn/README.md +++ b/quest/m1/poll-first/no-spawn/README.md @@ -45,12 +45,14 @@ These spawns stay and belong to [the spawn audit](/quest/m2/spawn-audit.md): reload (`rs/moq-relay/src/web.rs`), the io_uring handoff in `rs/moq-relay/src/connection.rs`, and moq-relay's web, shutdown, and uring spawns. Each UniFFI call's task on the bindings' runtime stays too: that is the -binding boundary. +binding boundary. So does moq-ffi's wake dispatch (decided 2026-11-03, +[moq-dev/moq#7720](https://github.com/moq-dev/moq/pull/7720)): a wake its root +raises for a parked host call runs on the blocking pool, so a host continuation +never runs inside the root. It owns no state and polls no background work. ## Required - [Admissions and stats are polled](/quest/m1/poll-first/no-spawn/admissions.md) - HTTP admissions run inside their waiter, and the stats publisher runs in the cluster's driver -- [moq-ffi spawns only its root](/quest/m1/poll-first/no-spawn/ffi.md) - one runtime-owned driver task holds every `Background` as `kio::Tasks` children - [The relay's in-memory tests run on kio's simulator](/quest/m1/poll-first/no-spawn/sim-tests.md) - `rs/moq-relay/tests/integration/connection.rs` runs on the simulator instead of paused tokio ## Related diff --git a/quest/m1/poll-first/no-spawn/ffi.md b/quest/m1/poll-first/no-spawn/ffi.md deleted file mode 100644 index e9e71f0d2e1..00000000000 --- a/quest/m1/poll-first/no-spawn/ffi.md +++ /dev/null @@ -1,30 +0,0 @@ -# [M] moq-ffi spawns only its root - -## Goal - -moq-ffi spawns one task outside its UniFFI calls: a runtime-owned driver that -polls every `Background` as a `kio::Tasks` child. Origin drivers, the server -accept loop, cleanups, and demand waits no longer spawn tasks of their own. - -## Plan - -Spawns in scope on `main` (2026-11-01), all in `rs/moq-ffi/src/`: - -- `ffi.rs`, `ffi::spawn` and `Background::driver`: origin drivers - (`origin.rs`), the server accept loop (`server.rs`), and cleanups. -- `ffi.rs`, `detached`: demand waits (`demand.rs`, `producer.rs`). -- `ffi.rs`, each UniFFI call's task on the bindings' runtime. This stays: each - call's own future is the binding boundary. - -✅ **One root driver** (maintainer, 2026-11-01, D4 on -[moq-dev/moq#7519](https://github.com/moq-dev/moq/pull/7519)): moq-ffi's root -is one runtime-owned driver task that holds every `Background` as `kio::Tasks` -children. Rejected: having each handle's calls poll its `Background`, which -fails for an origin with no reader, since nobody would poll its driver. - -[moq-dev/moq#7497](https://github.com/moq-dev/moq/pull/7497) replaced -`ffi::Task` with `ffi::Polled` for kio-backed reads and changed `Background` -and shutdown in `ffi.rs`, keeping demand waits `detached`. Recount the sites -above against it before converting. - -Public API: no binding signature changes. Wire: none. diff --git a/rs/moq-ffi/src/bench.rs b/rs/moq-ffi/src/bench.rs index 3c05c5af5bb..549f9e213db 100644 --- a/rs/moq-ffi/src/bench.rs +++ b/rs/moq-ffi/src/bench.rs @@ -319,3 +319,83 @@ async fn benchmark_ffi() { rust_ffi().await; rust_net().await; } + +/// Wake only `active` children while independently growing the background population. +/// The acknowledged turn measures actual root dispatch, including its per-child wakes. +#[tokio::test] +#[ignore = "benchmark: nightly ignored tests"] +async fn benchmark_background() { + let mut report = Report::default(); + for held in [1, 64, 1024, 8192] { + for active in [1, 8, 64] { + if active > held { + continue; + } + let ack = kio::Queue::new(); + let mut pulses = Vec::new(); + let mut children = Vec::new(); + for _ in 0..held { + let pulse = kio::Producer::new(0u64); + let reader = pulse.consume(); + let ack = ack.clone(); + children.push(crate::ffi::Background::new(async move { + let mut previous = 0; + while let Ok(next) = kio::wait(|waiter| { + reader.poll(waiter, |value| { + if **value > previous { + std::task::Poll::Ready(**value) + } else { + std::task::Poll::Pending + } + }) + }) + .await + { + previous = next; + ack.try_push(()).unwrap(); + } + Ok(()) + })); + pulses.push(pulse); + } + let round = |n| { + let (pulses, ack) = (&pulses, &ack); + async move { + for pulse in &pulses[..active] { + *pulse + .write() + .unwrap_or_else(|_| unreachable!("benchmark owns the pulse")) = n; + } + for _ in 0..active { + ack.pop().await.unwrap(); + } + } + }; + for n in 1..=WARMUP { + round(n).await; + } + let (start, cpu_start, budget) = (Instant::now(), cpu(), budget()); + let mut rounds = 0; + while start.elapsed() < budget { + kio::clock::timeout(ROUND_TIMEOUT, round(WARMUP + rounds + 1)) + .await + .expect("root wedged"); + rounds += 1; + } + let elapsed = start.elapsed(); + let cpu_used = cpu() - cpu_start; + let name = format!("held={held}/active={active}"); + report.push(format!("turn/{name}"), "ns", elapsed.as_nanos() as f64 / rounds as f64); + report.push( + format!("cpu/{name}"), + "ns", + cpu_used.as_nanos() as f64 / (rounds * active as u64) as f64, + ); + drop(pulses); + for child in children { + child.finished().await.unwrap(); + } + } + } + report.write("rust-background"); +} diff --git a/rs/moq-ffi/src/demand.rs b/rs/moq-ffi/src/demand.rs index c5808271a23..9975266ae8b 100644 --- a/rs/moq-ffi/src/demand.rs +++ b/rs/moq-ffi/src/demand.rs @@ -12,13 +12,13 @@ use crate::error::MoqError; #[derive(uniffi::Object)] pub struct MoqTrackDemand { inner: moq_net::track::Demand, - // The watcher `wanted_changed` advances, kept apart so concurrent waits queue in turn. - changes: Arc>, + // Each poll advances one shared cursor; a parked call never holds it locked. + changes: crate::ffi::Polled, } impl MoqTrackDemand { pub(crate) fn new(inner: moq_net::track::Demand) -> Arc { - let changes = Arc::new(tokio::sync::Mutex::new(inner.clone())); + let changes = crate::ffi::Polled::new(inner.clone()); Arc::new(Self { inner, changes }) } @@ -43,8 +43,9 @@ impl MoqTrackDemand { /// Wait for `wanted` to change and return the new value. The first call returns at once. pub async fn wanted_changed(&self) -> Result, MoqError> { - let changes = self.changes.clone(); - crate::ffi::detached(async move { gone(changes.lock().await.wanted_changed().await) }).await + self.changes + .wait(|changes, waiter| changes.poll_wanted_changed(waiter).map(gone)) + .await } } diff --git a/rs/moq-ffi/src/ffi.rs b/rs/moq-ffi/src/ffi.rs index 2f1f46538ac..373a72fdb39 100644 --- a/rs/moq-ffi/src/ffi.rs +++ b/rs/moq-ffi/src/ffi.rs @@ -1,7 +1,8 @@ use std::future::Future; +use std::pin::Pin; use std::sync::Arc; -use std::sync::atomic::{AtomicUsize, Ordering}; -use std::task::{Poll, ready}; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::task::{Context, Poll, Waker, ready}; use crate::error::MoqError; @@ -19,6 +20,8 @@ static RUNTIME: std::sync::OnceLock = std::sync::OnceLock::new(); /// The worker runtime and the owner thread that joins it on shutdown. pub(crate) struct Runtime { handle: tokio::runtime::Handle, + /// Work submitted by local handles, closed and drained with the root. + pending: kio::Shared>>, /// The clock a test drives, when the runtime started with one. #[cfg(feature = "testing")] clock: Option, @@ -97,13 +100,33 @@ impl Runtime { #[cfg(feature = "testing")] let clock = test_clock.then(|| kio::clock::Manual::new().install(&handle)); let (stop, stopped) = tokio::sync::oneshot::channel(); + let pending = kio::Shared::new(Some(Vec::new())); + let mut root = Root { + pending: pending.clone(), + children: kio::Tasks::new(), + }; + // The only FFI background task: each operation is a child polled by this owner. + let driver = runtime.spawn(async move { + if moq_tokio::panic::isolate(kio::wait(move |waiter| root.poll(waiter))) + .await + .is_none() + { + // Individual children contain their failures. Losing the owner itself would + // leave every handle looking healthy while no background work progresses. + std::process::abort(); + } + }); let join = std::thread::Builder::new() .name("moq-ffi".into()) .spawn(move || { // `Err` when the sender is dropped, which the shutdown never does; either way the // runtime is torn down here, joining the threads that ran its tasks. - let _ = runtime.block_on(stopped); + runtime.block_on(async move { + let _ = stopped.await; + driver.abort(); + let _ = driver.await; + }); // Join every worker and blocking task before the host can unload callbacks. drop(runtime); }) @@ -111,6 +134,7 @@ impl Runtime { Runtime { handle, + pending, #[cfg(feature = "testing")] clock, thread: std::sync::Mutex::new(Some(Thread { stop, join })), @@ -190,6 +214,230 @@ pub(crate) fn enter() -> tokio::runtime::EnterGuard<'static> { runtime().enter() } +type ChildFuture = Pin> + Send>>; + +/// One background operation, with panic isolation shared by every kind of child. +struct Child { + future: Option, + result: kio::Producer>>, +} + +impl Child { + /// Teardown may call user code too. One child's destructor must not stop its siblings. + fn release(&mut self) -> moq_tokio::Result<()> { + let future = self.future.take(); + std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| drop(future))).map_err(|payload| { + tracing::error!( + panic = moq_tokio::panic::message(payload.as_ref()), + "owned task teardown panicked" + ); + moq_tokio::Error::Panic + }) + } + + fn complete(&self, completed: moq_tokio::Result<()>) { + *self + .result + .write() + .unwrap_or_else(|_| unreachable!("only the child closes its completion")) = Some(completed); + } +} + +impl kio::Task for Child { + type Output = (); + + fn poll(&mut self, waiter: &kio::Waiter) -> Poll<()> { + let completed = { + // Isolate polling without consuming the stored future; release it separately + // so a destructor panic cannot escape the child. + let mut isolated = std::pin::pin!(moq_tokio::panic::isolate( + self.future.as_mut().expect("polled before completion").as_mut() + )); + ready!(waiter.poll_future(isolated.as_mut())).unwrap_or(Err(moq_tokio::Error::Panic)) + }; + // The remaining captures may wake readers when dropped. Publish the failure before + // releasing them; drivers also protect captures from the async state's unwind. + if matches!(&completed, Err(moq_tokio::Error::Panic)) { + self.complete(completed.clone()); + } + let released = self.release(); + self.complete(released.and(completed)); + Poll::Ready(()) + } +} + +impl Drop for Child { + fn drop(&mut self) { + if let Err(err) = self.release() { + self.complete(Err(err)); + } + } +} + +/// Owns every background child, including operations whose host discarded its completion. +struct Root { + pending: kio::Shared>>, + children: kio::Tasks, +} + +impl Root { + fn poll(&mut self, waiter: &kio::Waiter) -> Poll<()> { + let _root = InRoot::enter(); + waiter.batch(|| { + if let Poll::Ready(mut pending) = self.pending.poll(waiter, |pending| { + if pending.as_ref().is_some_and(|pending| !pending.is_empty()) { + Poll::Ready(()) + } else { + Poll::Pending + } + }) { + let incoming = std::mem::take(pending.as_mut().expect("open until root drops")); + drop(pending); + for child in incoming { + self.children.push(child); + } + // Re-arm the inbox next turn, including work submitted by a child's poll. + waiter.waker().wake_by_ref(); + } + let _ = self.children.poll(waiter); + // An empty set is idle, not finished: later local handles can submit more work. + Poll::Pending + }) + } +} + +impl Drop for Root { + fn drop(&mut self) { + // Refuse submissions atomically with taking the last batch, then drop outside the + // lock: cleanup can itself submit work. Each Child contains its teardown panic. + let _root = InRoot::enter(); + let pending = self.pending.lock().take(); + drop(pending); + // Here rather than after this returns, so their teardown wakes still see the root. + drop(std::mem::take(&mut self.children)); + } +} + +thread_local! { + /// Set while this thread polls or releases the root's children. + static ROOT: std::cell::Cell = const { std::cell::Cell::new(false) }; +} + +/// Marks this thread as inside the root until dropped. +struct InRoot(bool); + +impl InRoot { + fn enter() -> Self { + Self(ROOT.replace(true)) + } +} + +impl Drop for InRoot { + fn drop(&mut self) { + ROOT.set(self.0); + } +} + +/// A host call's waker, which keeps the host's continuation out of the root. +/// +/// UniFFI resumes the foreign continuation on the waking thread, and a binding may run it there +/// to its next suspension, host code included (Kotlin's `Dispatchers.Unconfined` does). Woken +/// inside the root's poll, that would hold every background child until the host returns, so +/// the wake moves to the runtime's blocking pool instead: a thread per continuation in flight, +/// owning no state and polling nothing but the host's waker. Woken anywhere else, it wakes the +/// host directly, as before. +struct Dispatch { + host: Waker, + /// A dispatched wake not delivered yet, so repeated wakes before it lands send one. + queued: AtomicBool, +} + +impl std::task::Wake for Dispatch { + fn wake(self: Arc) { + self.wake_by_ref(); + } + + fn wake_by_ref(self: &Arc) { + if !ROOT.get() { + self.host.wake_by_ref(); + return; + } + if self.queued.swap(true, Ordering::AcqRel) { + return; + } + let deferred = Deferred(self.clone()); + match tokio::runtime::Handle::try_current() { + // A runtime already shutting down drops the closure unrun, which still delivers it. + Ok(handle) => drop(handle.spawn_blocking(move || drop(deferred))), + Err(_) => drop(deferred), + } + } +} + +/// Delivers a dispatched wake when dropped, whether the blocking pool ran it or refused it. +struct Deferred(Arc); + +impl Drop for Deferred { + fn drop(&mut self) { + // Before waking, so a wake raised while the host polls dispatches again. + self.0.queued.store(false, Ordering::Release); + self.0.host.wake_by_ref(); + } +} + +/// Poll a host call with a [`Dispatch`] waker, so the root cannot run its continuation. +/// +/// Every primitive a binding call parks in goes through this: [`run`], [`Polled::wait`], and +/// [`Background::finished`] and [`Background::watch`]. Nesting is harmless: every wake chain +/// ends at the outermost host waker, and any one dispatch moves it off the root. +pub(crate) async fn host(future: F) -> F::Output { + let mut future = std::pin::pin!(future); + // Reused while the host keeps its waker, which UniFFI's call future does. + let mut dispatch: Option<(Arc, Waker)> = None; + std::future::poll_fn(|cx| { + // Polled by the root itself, the waker is the root's own, not a host's. + if ROOT.get() { + return future.as_mut().poll(cx); + } + if !dispatch + .as_ref() + .is_some_and(|(current, _)| current.host.will_wake(cx.waker())) + { + let current = Arc::new(Dispatch { + host: cx.waker().clone(), + queued: AtomicBool::new(false), + }); + dispatch = Some((current.clone(), Waker::from(current))); + } + let (_, waker) = dispatch.as_ref().expect("set above"); + future.as_mut().poll(&mut Context::from_waker(waker)) + }) + .await +} + +impl Runtime { + fn background(&self, future: impl Future> + Send + 'static) -> Background { + let result = kio::Producer::new(None); + self.submit(Child { + future: Some(Box::pin(future)), + result, + }) + } + + fn submit(&self, child: Child) -> Background { + let receiver = child.result.consume(); + let mut pending = self.pending.lock(); + if let Some(pending) = pending.as_mut() { + pending.push(child); + } else { + // Shutdown joined the root. A late handle must release its work immediately. + drop(pending); + drop(child); + } + Background { result: Some(receiver) } + } +} + /// Completion of an owned operation, retained by the object's host calls. #[derive(Clone, Default)] pub(crate) struct Background { @@ -198,53 +446,38 @@ pub(crate) struct Background { impl Background { pub fn new(future: impl Future> + Send + 'static) -> Self { - let result = kio::Producer::new(None); - let receiver = result.consume(); - runtime().spawn(async move { - let completed = moq_tokio::panic::isolate(future) - .await - .unwrap_or(Err(moq_tokio::Error::Panic)); - *result - .write() - .unwrap_or_else(|_| unreachable!("only the task closes its completion")) = Some(completed); - }); - Self { result: Some(receiver) } + runtime().background(future) } pub fn driver(mut driver: impl moq_net::time::Driver + Send + 'static) -> Self { let result = kio::Producer::new(None); - let receiver = result.consume(); - runtime().spawn(async move { - // Keep channel teardown outside the unwind and after panic publication. - let completed = moq_tokio::panic::isolate(async { - match moq_net::time::run(Borrowed(&mut driver)).await { - moq_net::Error::Closed => Ok(()), - error => Err(error.into()), + let failed = result.clone(); + runtime().submit(Child { + result, + future: Some(Box::pin(async move { + // Keep the driver outside the unwinding future: its teardown wakes the host. + let completed = moq_tokio::panic::isolate(async { + match moq_net::time::run(Borrowed(&mut driver)).await { + moq_net::Error::Closed => Ok(()), + error => Err(error.into()), + } + }) + .await + .unwrap_or(Err(moq_tokio::Error::Panic)); + if matches!(&completed, Err(moq_tokio::Error::Panic)) { + *failed + .write() + .unwrap_or_else(|_| unreachable!("only the child closes its completion")) = Some(completed.clone()); } - }) - .await - .unwrap_or(Err(moq_tokio::Error::Panic)); - if matches!(&completed, Err(moq_tokio::Error::Panic)) { - *result - .write() - .unwrap_or_else(|_| unreachable!("only the task closes its completion")) = Some(completed.clone()); - } - let completed = moq_tokio::panic::isolate(async move { drop(driver); completed - }) - .await - .unwrap_or(Err(moq_tokio::Error::Panic)); - *result - .write() - .unwrap_or_else(|_| unreachable!("only the task closes its completion")) = Some(completed); - }); - Self { result: Some(receiver) } + })), + }) } pub async fn finished(&self) -> Result<(), MoqError> { let Some(result) = &self.result else { return Ok(()) }; - let completed = kio::wait(|waiter| { + let completed = host(kio::wait(|waiter| { match std::task::ready!(result.poll(waiter, |state| { match &**state { Some(completed) => std::task::Poll::Ready(completed.clone()), @@ -254,7 +487,7 @@ impl Background { Ok(completed) => std::task::Poll::Ready(completed), Err(_) => std::task::Poll::Ready(Err(moq_net::Error::Cancel.into())), } - }) + })) .await; completed.map_err(crate::session::map_closed_error) } @@ -291,16 +524,19 @@ impl Background { pub async fn watch(&self, future: impl Future>) -> Result { self.check()?; - let result = if self.result.is_some() { - tokio::pin!(future); - tokio::select! { - biased; - result = self.finished() => { result?; future.await } - result = &mut future => result, + let result = host(async { + if self.result.is_some() { + tokio::pin!(future); + tokio::select! { + biased; + result = self.finished() => { result?; future.await } + result = &mut future => result, + } + } else { + future.await } - } else { - future.await - }; + }) + .await; self.check()?; result } @@ -346,7 +582,7 @@ impl moq_net::time::Driver for Borrowed<'_, D> { } } -/// Spawn an owned background future on the runtime. +/// Submit an owned background future to the runtime's root driver. pub(crate) fn spawn + Send + 'static>(future: F) -> Background { Background::new(async move { future.await; @@ -354,35 +590,31 @@ pub(crate) fn spawn + Send + 'static>(future: F) -> Backg }) } -/// Aborts a spawned task when the future awaiting it is dropped. +/// Poll a binding call in place, inside the runtime context and shutdown accounting. /// -/// Aborting one that already finished is a no-op, so this is inert on the happy path. -struct AbortOnDrop(tokio::task::AbortHandle); - -impl Drop for AbortOnDrop { - fn drop(&mut self) { - self.0.abort(); - } -} - -/// Run a future to completion off the caller's thread, mapping its error into [`MoqError`]. -pub(crate) async fn detached(future: F) -> Result +/// A foreign cancel stops polling immediately. Freeing the call drops its future; nothing +/// keeps running independently to consume an event in the meantime. +pub(crate) fn run(future: F) -> impl Future> where - F: Future> + Send + 'static, - T: Send + 'static, - E: Into + Send + 'static, + F: Future>, + E: Into, { - let task = runtime().spawn(async move { - moq_tokio::panic::isolate(async move { future.await.map_err(Into::into) }) - .await - .unwrap_or_else(|| Err(crate::session::map_closed_error(moq_tokio::Error::Panic))) - }); - let _abort = AbortOnDrop(task.abort_handle()); - match task.await { - Ok(result) => result, - Err(e) if e.is_cancelled() => Err(MoqError::Cancelled), - Err(e) => Err(e.into()), - } + let mut call = Box::pin(moq_tokio::panic::isolate(async move { + tokio::select! { + biased; + _ = stopped() => Err(MoqError::Cancelled), + result = future => result.map_err(Into::into), + } + })); + host(std::future::poll_fn(move |cx| { + let Some(_polling) = Polling::start() else { + return Poll::Ready(Err(MoqError::Cancelled)); + }; + let _runtime = enter(); + call.as_mut().poll(cx).map(|completed| { + completed.unwrap_or_else(|| Err(crate::session::map_closed_error(moq_tokio::Error::Panic))) + }) + })) } /// A handle's state for async calls that only poll it, which UniFFI's `RustFuture` polls directly. @@ -453,7 +685,7 @@ impl Polled { let _call = Call(&self.calls); // The waiter this call polls the state with, on the bridge's waker. let mut park = kio::Park::default(); - kio::wait(|waiter| self.poll(waiter, &mut park, &mut poll)).await + host(kio::wait(|waiter| self.poll(waiter, &mut park, &mut poll))).await } fn poll( @@ -642,26 +874,10 @@ impl Task { if let Some(cleanup) = self.cleanup.lock().unwrap().as_ref() { cleanup.check()?; } - let result = { - let drive = self - .driver - .watch(Self::drive(self.cancel.subscribe(), self.state.clone(), f)); - let mut run = std::pin::pin!(async { - tokio::select! { - biased; - _ = stopped() => Err(MoqError::Cancelled), - result = drive => result, - } - }); - std::future::poll_fn(|cx| { - let Some(_polling) = Polling::start() else { - return std::task::Poll::Ready(Err(MoqError::Cancelled)); - }; - let _runtime = enter(); - run.as_mut().poll(cx) - }) - .await - }; + let result = run(self + .driver + .watch(Self::drive(self.cancel.subscribe(), self.state.clone(), f))) + .await; // A foreign future may stop polling while holding the state, so cancellation cannot // wait for cleanup. Its next call still reports a completed cleanup panic. self.driver.check()?; @@ -731,6 +947,79 @@ impl Drop for Task { #[cfg(test)] mod tests { + /// Teardown includes pending submissions and children that never reached their first + /// poll. One panicking destructor must not prevent the rest from being released. + #[tokio::test] + async fn root_shutdown_releases_every_child_and_refuses_late_work() { + struct Tracked { + dropped: Arc, + panics: bool, + } + impl Drop for Tracked { + fn drop(&mut self) { + self.dropped.fetch_add(1, Ordering::SeqCst); + assert!(!self.panics, "injected child teardown panic"); + } + } + let dropped = Arc::new(AtomicUsize::new(0)); + let mut root = Root { + pending: kio::Shared::new(Some(Vec::new())), + children: kio::Tasks::new(), + }; + let mut completions = Vec::new(); + for panics in [true, false, false] { + let state = Tracked { + dropped: dropped.clone(), + panics, + }; + let result = kio::Producer::new(None); + completions.push(Background { + result: Some(result.consume()), + }); + root.pending.lock().as_mut().unwrap().push(Child { + future: Some(Box::pin(async move { + let _state = state; + std::future::pending().await + })), + result, + }); + } + assert!(root.poll(&kio::Waiter::noop()).is_pending()); + // A submission not yet enrolled in the child set must be released too. + let state = Tracked { + dropped: dropped.clone(), + panics: false, + }; + root.pending.lock().as_mut().unwrap().push(Child { + future: Some(Box::pin(async move { + drop(state); + Ok(()) + })), + result: kio::Producer::new(None), + }); + let pending = root.pending.clone(); + drop(root); + assert_eq!(dropped.load(Ordering::SeqCst), 4); + assert!(pending.read().is_none()); + assert!(matches!(completions[0].finished().await, Err(MoqError::Task { .. }))); + for completion in &completions[1..] { + assert!(matches!(completion.finished().await, Err(MoqError::Cancelled))); + } + + let runtime = Runtime::new(false); + runtime.shutdown(); + let state = Tracked { + dropped: dropped.clone(), + panics: false, + }; + let late = runtime.background(async move { + drop(state); + Ok(()) + }); + assert_eq!(dropped.load(Ordering::SeqCst), 5); + assert!(matches!(late.finished().await, Err(MoqError::Cancelled))); + } + #[tokio::test] async fn a_driver_teardown_panic_reaches_its_owner() { struct Driver; @@ -782,10 +1071,11 @@ mod tests { let (release, released) = std::sync::mpsc::channel(); let finished = Arc::new(std::sync::atomic::AtomicBool::new(false)); let callback = finished.clone(); - runtime.spawn(async move { + runtime.background(async move { entered.send(()).unwrap(); released.recv().unwrap(); callback.store(true, std::sync::atomic::Ordering::SeqCst); + Ok(()) }); entering.recv().unwrap(); @@ -824,6 +1114,114 @@ mod tests { runtime.shutdown(); } + /// A host continuation that blocks once woken from a root child, as one resumed inline by + /// Kotlin's `Dispatchers.Unconfined` can, holds neither the root nor its other children. + #[test] + fn a_blocking_host_continuation_does_not_hold_the_root() { + /// The host side of a call: woken, it reports entering and blocks until released. + struct Continuation { + entered: std::sync::Mutex>>, + release: std::sync::Mutex>, + } + impl std::task::Wake for Continuation { + fn wake(self: Arc) { + self.wake_by_ref(); + } + fn wake_by_ref(self: &Arc) { + if let Some(entered) = self.entered.lock().unwrap().take() { + entered.send(std::thread::current().id()).unwrap(); + let _ = self.release.lock().unwrap().recv(); + } + } + } + + let runtime = Runtime::new(false); + let ready = kio::Producer::new(false); + let consumer = ready.consume(); + let mut call = Box::pin(host(kio::wait(|waiter| { + consumer.poll(waiter, |ready| if **ready { Poll::Ready(()) } else { Poll::Pending }) + }))); + let (entered, entering) = std::sync::mpsc::channel(); + let (release, released) = std::sync::mpsc::channel(); + let waker = Waker::from(Arc::new(Continuation { + entered: std::sync::Mutex::new(Some(entered)), + release: std::sync::Mutex::new(released), + })); + assert!(call.as_mut().poll(&mut Context::from_waker(&waker)).is_pending()); + + // A root child wakes the parked call; its continuation enters and stays. + let (root, rooted) = std::sync::mpsc::channel(); + runtime.background(async move { + root.send(std::thread::current().id()).unwrap(); + *ready.write().ok().unwrap() = true; + std::future::pending().await + }); + let root = rooted + .recv_timeout(Duration::from_secs(5)) + .expect("the root polls its child"); + let continuation = entering + .recv_timeout(Duration::from_secs(5)) + .expect("the continuation is woken"); + assert_ne!(continuation, root, "the continuation ran inside the root"); + + // Another child still runs to completion while the continuation blocks. + let sibling = runtime.background(async { Ok(()) }); + tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap() + .block_on(soon(sibling.finished())) + .unwrap(); + + release.send(()).unwrap(); + runtime.shutdown(); + } + + /// Children released by the root's teardown wake host calls off the root too. + #[tokio::test(flavor = "multi_thread")] + async fn root_teardown_wakes_host_calls_off_the_root() { + struct Record(std::sync::Mutex>); + impl std::task::Wake for Record { + fn wake(self: Arc) { + self.wake_by_ref(); + } + fn wake_by_ref(self: &Arc) { + let _ = self.0.lock().unwrap().send(std::thread::current().id()); + } + } + + let mut root = Root { + pending: kio::Shared::new(Some(Vec::new())), + children: kio::Tasks::new(), + }; + let held = kio::Producer::new(()); + let consumer = held.consume(); + root.pending.lock().as_mut().unwrap().push(Child { + future: Some(Box::pin(async move { + let _held = held; + std::future::pending().await + })), + result: kio::Producer::new(None), + }); + // Enroll the child, so the teardown below releases it from the child set. + assert!(root.poll(&kio::Waiter::noop()).is_pending()); + + let mut call = Box::pin(host(kio::wait(|waiter| consumer.poll(waiter, |_| Poll::<()>::Pending)))); + let (woke, woken) = std::sync::mpsc::channel(); + let waker = Waker::from(Arc::new(Record(std::sync::Mutex::new(woke)))); + assert!(call.as_mut().poll(&mut Context::from_waker(&waker)).is_pending()); + + drop(root); + let woken = woken + .recv_timeout(Duration::from_secs(5)) + .expect("teardown wakes the call"); + assert_ne!( + woken, + std::thread::current().id(), + "the continuation ran inside the root" + ); + } + /// A state that reports its own drop, which is what a terminal cancel has to trigger. struct Tracked(Option>); diff --git a/rs/moq-ffi/src/producer.rs b/rs/moq-ffi/src/producer.rs index 8787053c394..96904e3699f 100644 --- a/rs/moq-ffi/src/producer.rs +++ b/rs/moq-ffi/src/producer.rs @@ -489,7 +489,7 @@ impl MoqFill { /// Wait for `wanted` to change and return the new value. The first call returns at once. pub async fn wanted_changed(&self) -> Result, MoqError> { - // Polled in place, not detached: a cancelled call left unfreed must not advance the + // Polled in place: a cancelled call left unfreed must not advance the // watcher and swallow the change the next call waits for. self.changes .wait(|fill, waiter| Poll::Ready(Ok(ready!(fill.poll_wanted_changed(waiter))))) @@ -521,9 +521,8 @@ impl MoqFill { /// Wait until no reader waits for group `sequence` from this fill, so stop serving it. pub async fn unwanted(&self, sequence: u64) -> Result<(), MoqError> { - let fill = self.inner.clone(); - crate::ffi::detached(async move { - fill.unwanted(sequence).await; + crate::ffi::run(async { + self.inner.unwanted(sequence).await; Ok::<_, MoqError>(()) }) .await diff --git a/rs/moq-ffi/src/session.rs b/rs/moq-ffi/src/session.rs index a4be3154650..8d76459e449 100644 --- a/rs/moq-ffi/src/session.rs +++ b/rs/moq-ffi/src/session.rs @@ -646,10 +646,7 @@ impl MoqTokens { /// [`MoqError::Unsupported`] when the session cannot carry an in-band token, and the /// connection's end, such as a refused dial, is its close reason. pub async fn answered(&self) -> Result<(), MoqError> { - { - let tokens = self.inner.clone(); - crate::ffi::detached(async move { tokens.answered().await.map_err(MoqError::from) }).await - } + crate::ffi::run(self.inner.answered()).await } } @@ -810,13 +807,13 @@ impl Shutdown { async fn finished(&self) -> Result<(), MoqError> { // Publish a background panic before reading the channel it dropped. self.task.finished().await?; - self.result - .wait(|result| match &**result { - Some(result) => std::task::Poll::Ready(result.clone()), - None => std::task::Poll::Pending, - }) - .await - .map_err(|_| MoqError::Cancelled)? + // The drain is a root child, so its wake must not run the host's continuation there. + crate::ffi::host(self.result.wait(|result| match &**result { + Some(result) => std::task::Poll::Ready(result.clone()), + None => std::task::Poll::Pending, + })) + .await + .map_err(|_| MoqError::Cancelled)? } } diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index a0d8bfa4fba..aed8bb82fa3 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -183,7 +183,7 @@ fn assert_protocol(err: &MoqError, scope: crate::error::MoqErrorScope, kind: cra } #[tokio::test] -async fn detached_cancels_inner_on_drop() { +async fn call_cancels_inner_on_drop() { struct DropSignal(Option>); impl Drop for DropSignal { @@ -194,7 +194,7 @@ async fn detached_cancels_inner_on_drop() { let (started_tx, started_rx) = tokio::sync::oneshot::channel(); let (dropped_tx, dropped_rx) = tokio::sync::oneshot::channel(); - let outer = tokio::spawn(crate::ffi::detached(async move { + let outer = tokio::spawn(crate::ffi::run(async move { let _drop = DropSignal(Some(dropped_tx)); started_tx.send(()).unwrap(); std::future::pending::>().await @@ -202,13 +202,13 @@ async fn detached_cancels_inner_on_drop() { kio::clock::timeout(TIMEOUT, started_rx) .await - .expect("timed out waiting for detached task to start") + .expect("timed out waiting for call to start") .unwrap(); outer.abort(); assert!(outer.await.unwrap_err().is_cancelled()); kio::clock::timeout(TIMEOUT, dropped_rx) .await - .expect("timed out waiting for detached task to be dropped") + .expect("timed out waiting for call to be dropped") .unwrap(); } @@ -8422,7 +8422,7 @@ async fn shutdown_delivers_the_finished_track() { let closing = publisher.clone(); kio::clock::timeout( TIMEOUT, - crate::ffi::detached(async move { + crate::ffi::run(async move { let group = track.append_group()?; group.write_frame(MoqFrame { timestamp: Some(us(0)), @@ -9192,6 +9192,32 @@ fn shutdown_waits_for_a_polled_call() { assert!(matches!(parked, Err(MoqError::Cancelled)), "{parked:?}"); } +/// A cancelled foreign wait can remain allocated until the host frees it. It must neither +/// consume another demand change nor prevent a second call from observing that change. +#[tokio::test] +async fn demand_cancelled_before_free_leaves_the_change() { + let broadcast = MoqBroadcastProducer::new().unwrap(); + let track = broadcast.publish_track("events".into(), None).unwrap(); + let demand = track.demand().unwrap(); + assert_eq!(demand.wanted_changed().await.unwrap(), None); + let mut abandoned = Box::pin(demand.wanted_changed()); + assert!( + abandoned + .as_mut() + .poll(&mut std::task::Context::from_waker(std::task::Waker::noop())) + .is_pending() + ); + ffi_caught_up().await; + let consumer = track.consume(None).unwrap(); + let wanted = kio::clock::timeout(TIMEOUT, demand.wanted_changed()) + .await + .expect("the abandoned wait stole the change") + .unwrap(); + assert!(wanted.is_some()); + drop(abandoned); + drop(consumer); +} + /// Setup, rather than the first packet, decides whether a rendition is usable. #[cfg(feature = "audio")] #[tokio::test] diff --git a/sh/gh/nightly.sh b/sh/gh/nightly.sh index 60c4ef42c57..88c4b5042f0 100755 --- a/sh/gh/nightly.sh +++ b/sh/gh/nightly.sh @@ -189,12 +189,12 @@ case "$1" in MOQ_BENCH_DIR=${BENCH_RESULTS:-$(git rev-parse --show-toplevel)/target/bench/nightly/bindings} export MOQ_BENCH_DIR rm -rf "$MOQ_BENCH_DIR" - run just rs test -p moq-ffi --cargo-profile release -E 'test(bench::benchmark_ffi)' --run-ignored only + run just rs test -p moq-ffi --cargo-profile release -E 'test(bench::benchmark_ffi) | test(bench::benchmark_background)' --run-ignored only run just go bench -benchtime="${MOQ_BENCH_SECONDS}s" for runner in py kt cpp dart; do run just "$runner" bench done - for result in rust-ffi rust-net go python kotlin cpp dart; do + for result in rust-ffi rust-net rust-background go python kotlin cpp dart; do if [[ ! -s $MOQ_BENCH_DIR/$result.json ]]; then failed+=("no $MOQ_BENCH_DIR/$result.json"); fi done ;; diff --git a/sh/gh/nightly.test.py b/sh/gh/nightly.test.py index bd99cb19d2f..09d596baf3e 100644 --- a/sh/gh/nightly.test.py +++ b/sh/gh/nightly.test.py @@ -123,7 +123,7 @@ def test_binding_runner_fails_without_its_report(self): self.assertEqual(result.returncode, 1) self.assertEqual(calls[-1], "just dart bench") self.assertIn("failed: just py bench", result.stderr) - for runner in ("rust-ffi", "rust-net", "go", "python", "kotlin", "cpp", "dart"): + for runner in ("rust-ffi", "rust-net", "rust-background", "go", "python", "kotlin", "cpp", "dart"): self.assertIn(f"/{runner}.json", result.stderr) def test_bench_targets_share_one_invocation(self):