diff --git a/quest/m0/README.md b/quest/m0/README.md index 2a423878fab..5aebcb5601f 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -48,8 +48,8 @@ publishers mint epochs. Backport patches cut from `release` don't carry Liveness: a serve loop with work always ready never yields, which starved an FFI publisher's QUIC driver and fails hosted Interop's go lanes (found -2026-10-08 landing #4225). The [serve budget](/quest/m0/serve-budget.md) -bounds every kio task's loop. The Go and Python interop cells it fails +2026-10-08 landing #4225). The publish serve loops now yield through a +`kio::coop::Budget`. The Go and Python interop cells it fails moved here from m1 the same day, because the stall masks interop on every wire PR and hides as slow passes; the harness now fails a cell whose connection idles out. @@ -61,9 +61,8 @@ remain and no release waits on them. ## Required -- [Serve budget](/quest/m0/serve-budget.md) - a kio task that always has work ready yields after a budget, so a fast publisher can't starve its own QUIC driver - [Draft-22 media on 0.17](/quest/m0/release-22/README.md) - a 0.17.x with the LOCATION_FILTER and FIRST_OBJECT fixes and moq-noq 1.3.4, before Seattle -- [FFI publisher stall](/quest/m0/ffi-publisher-stall.md) - every Go, Python, and C++ publisher cell passes reliably once the serve budget lands, and a cell fails when a connection idles out +- [FFI publisher stall](/quest/m0/ffi-publisher-stall.md) - every Go, Python, and C++ publisher cell passes reliably with the serve budgets, and a cell fails when a connection idles out - [FFI cancelled reads](/quest/m0/ffi-cancel-read.md) - cancelling a moq-ffi read in any binding leaves its data for the next read, so the C++ probe stops flaking - [web-transport releases the qmux fixes](/quest/m0/qmux-credit-upstream.md) - waiting on moq-dev/web-transport#412 and #413 to merge and ship, which qmux credit bumps to - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines diff --git a/quest/m0/ffi-publisher-stall.md b/quest/m0/ffi-publisher-stall.md index 068baf5b7a0..13ae5d63937 100644 --- a/quest/m0/ffi-publisher-stall.md +++ b/quest/m0/ffi-publisher-stall.md @@ -16,14 +16,16 @@ cell. The cause is known: a serve loop with work always ready never yields and starves moq-ffi's single runtime thread until the relay times the publisher -out (found 2026-10-08 landing #4225). [Serve budget](/quest/m0/serve-budget.md) -fixes it, so this quest starts once it lands and no bisect is needed. +out (found 2026-10-08 landing #4225). The publish serve loops now yield +through a `kio::coop::Budget` (#5088), which is meant to fix it: interop run +37889112106 passed every Go and Python publisher cell on an earlier design of +that PR. Confirm on `main` and nightly before closing. Decided 2026-10-08: - In m0, moved from m1 and widened from the browser cells: the stall masks interop on every wire PR and may disconnect real FFI publishers. -- Re-check every Go, Python, and C++ publisher cell once the serve budget lands. +- Re-check every Go, Python, and C++ publisher cell now that the budget landed. The #4225 root-cause comment saw a 64-iteration yield still time the publisher out, so if a cell still stalls, find the rest of the cause. Never raise a timeout or add a retry. @@ -52,10 +54,6 @@ Harness facts (2026-10-08): Public API: none expected. Wire: none. -## Required - -- [Serve budget](/quest/m0/serve-budget.md) - fixes the stall's cause; this quest verifies the cells and hardens the harness - ## Related - [FFI runtime](/quest/m1/ffi-runtime.md) - FFI apps run on more than one thread, which also hides the stall diff --git a/quest/m0/serve-budget.md b/quest/m0/serve-budget.md deleted file mode 100644 index 3359c393cb2..00000000000 --- a/quest/m0/serve-budget.md +++ /dev/null @@ -1,98 +0,0 @@ -# [M] A busy task yields within a budget - -## Goal - -No kio task can hold its thread with an unbounded loop of async kio -readiness polls while work keeps arriving. Synchronous `try_*` and peek -loops, and code outside a `kio::coop::budget`, stay unbounded, as today. A serve loop that always has another group ready yields after a -fixed budget of progress and resumes on its next poll, so the QUIC driver, -timers, and the session's other tasks run in between, on every runtime -(tokio multi-thread and current-thread, wasm, and the io_uring worker). - -Found while landing #4225 (2026-10-08): the go interop publisher writes -2.5 ms Opus frames, one group each (about 400 groups/s). When a relay -subscribes, `RequestServe::poll_serve` (`rs/moq-net/src/lite/publisher.rs`) -keeps returning `Continue` and one poll ran over 4,096 iterations. moq-ffi's -current-thread runtime then starves noq's connection driver until the relay -times the publisher out after 10 s. Hosted Interop's `go -> *` lanes fail on -#4225 and stall for about 10 s on `main`. The gate is the mocked serve test -below, not those lanes: the m1 quests each hide the stall on their own, and -the #4225 root-cause comment saw a 64-iteration yield still time the -publisher out, so a yield alone may not be the whole fix. Re-check the lanes -and #4225 when this lands. - -Not here: moq-net spawning tasks (it stays poll-only by design, #2302, -#2736, #3825), the FFI runtime flavor, or the audio grouping default; those -are separate quests below. - -## Plan - -Decided 2026-10-08 in a `/quest-plan` interview (paper trail in the PR that -added this quest): - -- The bound is generic, modeled on tokio's cooperative budget, and lives in - kio, which takes no runtime dependency. Tokio's budget cannot see this loop, - because the work goes through kio primitives, never a tokio resource. - Rejected: per-loop budgets in the two serve loops (not generic), and - bridging kio to `tokio::task::coop` (leaves wasm and io_uring unbounded). -- Mechanics: - - a per-thread budget in kio. An async kio readiness poll that makes - progress spends one unit, and refunds it if the operation ends `Pending` - anyway; - - an exhausted budget wakes the current task - (`waiter.waker().wake_by_ref()`) and returns `Pending`; - - synchronous `try_*` and peek APIs, and any poll with - `kio::Waiter::noop()`, spend nothing, since their callers read "nothing - now" as empty, not as "come back". `Waiter` carries an explicit noop - flag for this, since comparing wakers is not reliable. -- A budget `Pending` must only postpone, never change a decision or lose - state. Spend only at state-safe boundaries, and audit every caller that - reads `Pending` as a value or commits state before a later poll. Known - sites: `TrackRun::start` reads Largest for SUBSCRIBE_START with - `Pending => None`; `poll_recv_next` probes `poll_finished` after - `poll_recv_group` ends, and a `Pending` there skips SUBSCRIBE_END; - `GroupServe` reads `poll_expired` as a bool; and `Cursor::poll_recv_group` - advances `index` or takes a parked group before `poll_stale`, so a - `Pending` there drops the group. Tests drive each site to exhaustion and - check every eligible group is delivered exactly once, fresh and parked. -- The budget is per kio task: `Tasks::poll` refills it for each child it - polls, and `moq_net::time::run` refills it once per `kio::wait` poll, - outside its loop, so an always-ready timer can't refill per iteration. A child that runs out lands in the next pass by - `Tasks`' one-snapshot rule (`a_self_waking_task_yields_to_the_owner`), and - the owner returns to its runtime after the pass. Work per turn scales with - the children that were ready, so a busy fan-out session is not throttled. - If [Run to quiescence](/quest/m1/perf/uring-quiescence.md) lets a turn run - several passes, each pass refills every child it polls, so a turn is - bounded by passes times ready children times the budget, and the owner - returns only after the last pass. Owners that poll their `Tasks` more than - once per poll (`Publisher::poll`, `SubscribeServe::poll_step` inside - `RequestServe::poll_serve`'s loop) compound the same way: a nested serve - can spend about the budget squared in one runtime poll. That is bounded, - and the bench decides whether it matters. - Rejected: one budget shared by the whole runtime task (tokio's model), - which caps a session with many subscriptions at one budget per turn. -- Refill mirrors tokio: `kio::coop::budget(f)` sets a fresh budget and - restores the previous value on exit, so nesting refills. Public, documented - as called only where a driver hands a task its turn. Code outside any - `budget` call is unconstrained, as today. The moq-uring worker's pass calls - it too if it polls anything outside `Tasks`. -- The size starts at 128 (tokio's) and is a constant, not configurable. A - bench picks it: sweep unbounded, 32, 128, and 512 against `main` and keep - the smallest whose throughput matches unbounded. -- Tests, with mocked time: a producer appends 10,000 groups during a serve - with many groups in flight, the serve poll returns `Pending` while a - sibling task and the transport make progress (fails on `main`), and the - units spent per runtime poll are counted against the bound; kio unit tests - for spend, refund, exhaustion, and nested refill. -- Bench: extend `rs/moq-net/benches/session.rs` swept over groups/s and - subscriptions per session, and kio's `channel` and `tasks` benches for the per-operation cost - of the thread-local. - -Public API: `kio::coop::budget` (new). Wire: none. - -## Related - -- [FFI runtime](/quest/m1/ffi-runtime.md) - FFI apps run on more than one thread -- [Audio group duration](/quest/m1/audio-group-duration.md) - small audio frames stop minting a group each -- [FFI publisher stall](/quest/m0/ffi-publisher-stall.md) - verifies the Go, Python, and C++ interop cells once this lands, and fails a cell whose connection idles out -- [Run to quiescence](/quest/m1/perf/uring-quiescence.md) - its pass count bounds passes per turn; this budget bounds one task's loop, so they compose diff --git a/quest/m1/README.md b/quest/m1/README.md index 7e4ab09148f..27823dc90b8 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -66,6 +66,8 @@ were done or not worth their cost. - [FFI runtime](/quest/m1/ffi-runtime.md) - moq-ffi drives moq on a multi-thread runtime instead of one thread - [Audio group duration](/quest/m1/audio-group-duration.md) - audio groups span at least 20 ms by default, so small frames don't mint a group each - [Pipelined requests](/quest/m1/pipeline-requests/README.md) - SUBSCRIBE and the first FETCH go out with the track-info request at every hop, so first data arrives a round trip sooner per hop +- [A spinning loop fails a sim test](/quest/m1/spinning-loops.md) - a sim test names any loop that holds a task poll too long, and each one yields through a budget +- [A busy js/net serve yields](/quest/m1/js-serve-yield.md) - js/net's serve loop leaves the browser its event loop under a fast publisher, if it does not already - [Kotlin wrapper POMs](/quest/m1/kt-ffi-pom.md) - Maven builds of `dev.moq:moq` resolve a published moq-ffi instead of the missing `0.0.0-dev` - [Installable moq-gst](/quest/m1/gst-brew-path.md) - `brew install moq-gst` installs the plugin from the tarball's `lib/gstreamer-1.0/`, and the Nix package carries `x264enc`, `avenc_aac`, and `avdec_h264` - [Same-epoch importers](/quest/m1/hop-aligned-import.md) - importers sharing one `--epoch` and fed one stream publish identical groups and timestamps, so failover between a redundant pair survives diff --git a/quest/m1/audio-group-duration.md b/quest/m1/audio-group-duration.md index 2cb9203a659..925efdf99f3 100644 --- a/quest/m1/audio-group-duration.md +++ b/quest/m1/audio-group-duration.md @@ -32,5 +32,4 @@ Public API: a changed default in moq-audio and `@moq/publish`. Wire: none. ## Related -- [Serve budget](/quest/m0/serve-budget.md) - bounds the serve loop whatever the group rate - [FFI codec namespaces](/quest/m1/ffi-shape/codec.md) - owns the FFI encoder's frame duration default, a separate knob diff --git a/quest/m1/ffi-runtime.md b/quest/m1/ffi-runtime.md index ba15c0bd0c0..b7fe67151a2 100644 --- a/quest/m1/ffi-runtime.md +++ b/quest/m1/ffi-runtime.md @@ -20,8 +20,8 @@ added this quest): #1071) with no stated reason. - Use `new_multi_thread().enable_all()` with tokio's default worker count. No configuration knob until an embedder needs one. Rejected: a small fixed - pool, and keeping `current_thread` behind the - [serve budget](/quest/m0/serve-budget.md) alone. + pool, and keeping `current_thread` behind the serve budgets + (`kio::coop::Budget`) alone. - Keep the "foreign caller's thread never drives our futures" property: work still runs via `Task::run` and `detached`. Check the exported async fns that await in place (`subscribe_catalog`, `subscribe_track`, `fetch_group`, @@ -51,6 +51,5 @@ Public API: callbacks may fire concurrently on any runtime thread ## Related -- [Serve budget](/quest/m0/serve-budget.md) - the root fix for a task hogging its thread - [Kotlin/JVM exit](/quest/m2/kt-jvm-exit.md) - runtime-thread shutdown - [Generated C bindings](/quest/m1/c/README.md) - inherits the moq-ffi runtime, so hand-written moq-c needs no change diff --git a/quest/m1/js-serve-yield.md b/quest/m1/js-serve-yield.md new file mode 100644 index 00000000000..127c56082c9 --- /dev/null +++ b/quest/m1/js-serve-yield.md @@ -0,0 +1,20 @@ +# [S] A busy js/net serve leaves the browser its event loop + +## Goal + +Find whether a js/net serve loop whose awaits keep resolving immediately (a +fast publisher with a backlog) runs only microtasks and starves the browser's +event loop: rendering, input, and WebTransport reads. Fix it at the cause if it +does, and keep a browser bench that shows it. + +## Plan + +Found 2026-10-09 while reworking #5088, which bounds the Rust serve loops with +`kio::coop::Budget`. Microtask starvation is the JS analog of a Rust task that +never returns `Pending`. Measure first: a browser bench publishing 2.5 ms +groups with a sibling timer, checking that the timer keeps firing. Node and Bun +only stand in for the browser. + +## Related + +- [A spinning loop fails a sim test](/quest/m1/spinning-loops.md) - the Rust side's guard diff --git a/quest/m1/perf/uring-quiescence.md b/quest/m1/perf/uring-quiescence.md index ee9c09ce2c1..42254f0849b 100644 --- a/quest/m1/perf/uring-quiescence.md +++ b/quest/m1/perf/uring-quiescence.md @@ -52,7 +52,3 @@ meaning with a budget of 1 and gains a sibling proving the budget bound. - [Hard fork](/quest/m1/quic/fork/README.md) - the noq driver this edits moves onto `moq-quic` (decided in the 2026-10-05 audit) - [One enter per turn](/quest/m1/perf/uring-one-enter.md) - the metrics and the submit placement this sweep is measured with - -## Related - -- [Serve budget](/quest/m0/serve-budget.md) - a per-task budget bounds one task's loop; this quest's pass count bounds passes per turn, and the sweep runs with both in place diff --git a/quest/m1/session-burst-hang.md b/quest/m1/session-burst-hang.md index 6835a8149e4..bfda5fbc1ef 100644 --- a/quest/m1/session-burst-hang.md +++ b/quest/m1/session-burst-hang.md @@ -3,8 +3,8 @@ ## Goal `rs/moq-net/benches/session.rs`'s `burst` sweep runs to completion at 16 -subscriptions and 16 or more groups per round, with or without the kio -budget, and the sweep covers those shapes again. +subscriptions and 16 or more groups per round, with or without the serve +budgets, and the sweep covers those shapes again. ## Plan @@ -15,7 +15,3 @@ budget too, so the sweep was limited to shapes that complete. Decided 2026-10-08 treat it as a possible session stall, not a bench setup limit. Find whether it is cache size, stream credit, or a real stall, fix it at the cause, and add a regression test if the cause is in the session rather than the bench. - -## Required - -- [A busy task yields within a budget](/quest/m0/serve-budget.md) - adds the `burst` axis diff --git a/quest/m1/spinning-loops.md b/quest/m1/spinning-loops.md new file mode 100644 index 00000000000..a6ce372f850 --- /dev/null +++ b/quest/m1/spinning-loops.md @@ -0,0 +1,38 @@ +# [M] A spinning loop fails a sim test + +## Goal + +A moq-net sim test fails, naming the loop, when one task poll makes more than a +fixed number of kio `Ready` polls, and every loop it flags yields through a +`kio::coop::Budget`. Today a loop with no budget that keeps finding work ready +holds its thread with nothing to flag it, the way `RequestServe::poll_serve` +starved a go publisher's runtime (#5088). + +## Plan + +Decided 2026-10-09 while reworking #5088: + +- #5088 gives only the publish serve loops a `Budget`: lite `RequestServe` and + `GroupServe`, and ietf `TrackServe`, `GroupServe`, `read_fetch`, and + `write_fetch_group`. A search found about 31 more loops that can keep + finding work in one poll (receive, accept, announce, and request loops in + `lite/subscriber.rs`, `ietf/subscriber.rs`, `ietf/session.rs`, and + `model/origin.rs`). This guard decides which of them actually spin. +- Also unbudgeted: ietf `run_fetch_stream`'s write loop over the frames + `read_fetch` collected, which sends a whole cached group in one poll over + always-ready writes. Add an end-to-end FETCH test that checks a sibling + still gets its turns (found by the OpenAI review of #5088). +- The count is test-only, so kio's hot path stays untouched: a feature or + `cfg` that only the sim runtime enables, reset on each task poll. +- A loop takes a `Budget` only where its `Pending` goes straight up to the + task. Three known sites need care: lite `FetchServeRun` reads a + `FrameIngest` yield as "still short" and may `abort_unused`; + `Announced::poll_serve` must re-queue the route through `route_waiter` and + yield in the outer loop too; and ietf `poll_datagrams` still spins on a + datagram backlog after `TrackServe` yields, and its drain arm would skip + datagrams after a yield unless it returns `Poll`. + +## Related + +- [Session burst hang](/quest/m1/session-burst-hang.md) - the same bench shapes that hang +- [FFI runtime](/quest/m1/ffi-runtime.md) - FFI apps run on more than one thread diff --git a/quest/m1/track-tail-interop.md b/quest/m1/track-tail-interop.md index 8531b9b21a4..a3e8755b983 100644 --- a/quest/m1/track-tail-interop.md +++ b/quest/m1/track-tail-interop.md @@ -29,10 +29,10 @@ run 37139033242 (2026-10-03) failed `tail rust -> rust`, `rust -> js-native-node`, and `rust -> js-native-bun`. Merge `main` (after #4741 and #4813) into #4225, rerun, and record here whatever defect still fails. -Found 2026-10-08: #4225's `go -> *` lanes fail on the -[serve budget](/quest/m0/serve-budget.md) stall, where the go publisher's -serve loop starves its own QUIC driver until the relay times it out. Rerun -the lanes once that lands. +Found 2026-10-08: #4225's `go -> *` lanes fail on a stall where the go +publisher's serve loop starves its own QUIC driver until the relay times it +out. The serve budgets (`kio::coop::Budget`) are meant to fix it; rerun the +lanes. Decided in the 2026-09-30 audit: the lite-07 drop case moved into [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md)'s tests, so the basic Rust and @@ -41,10 +41,6 @@ JS tail interop lands now instead of waiting on that [L] quest. QUIC on localhost rarely reorders, so this is a smoke check that the end is delivered and clean. The ordering race itself stays in the unit tests. -## Required - -- [Serve budget](/quest/m0/serve-budget.md) - is meant to fix the stall that fails #4225's go lanes; rerun them once it lands - ## Related - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - owns the lite-07 drop case on top of this harness diff --git a/rs/kio/README.md b/rs/kio/README.md index bebde65b805..be6159a765c 100644 --- a/rs/kio/README.md +++ b/rs/kio/README.md @@ -18,6 +18,8 @@ The channel auto-closes when all producers are dropped. poll-native FIFO queue (bounded or unbounded) built in the same style. `Park` holds a waiter for as long as a poll stays pending, bridging a `std::task::Context` to kio's waiter-based polls when implementing `Future` or `poll_*` on top of kio channels. +`coop::Budget` bounds the passes a loop makes per poll, so one that always finds +work ready still yields to the rest of its runtime. It's used internally by [moq-net](https://github.com/moq-dev/moq/tree/main/rs/moq-net) and friends, but is generic enough to be useful on its own. diff --git a/rs/kio/src/coop.rs b/rs/kio/src/coop.rs new file mode 100644 index 00000000000..d7c2fa2293e --- /dev/null +++ b/rs/kio/src/coop.rs @@ -0,0 +1,80 @@ +//! Cooperative scheduling: bound how many passes a loop makes per poll. +//! +//! A task whose loop keeps finding work ready never yields on its own: a serve loop +//! that always has another item ready holds its thread for as long as items keep +//! arriving, starving everything else on it. Tokio's own budget cannot see that loop, +//! since the work goes through kio rather than a tokio resource. + +use std::task::Poll; + +use crate::Waiter; + +/// Yields a loop after a fixed number of passes, sized by the loop's cost per pass. +/// +/// Keep one per loop and call [`Self::poll_yield`] at the head of each pass. +#[derive(Debug, Clone)] +pub struct Budget { + passes: u32, + left: u32, +} + +impl Budget { + /// A budget allowing `passes` passes between yields. + pub const fn new(passes: u32) -> Self { + Self { passes, left: passes } + } + + /// Spend a pass, or wake the task and return `Pending` once none are left. + /// + /// The yield refills the budget for the next poll. Call it only where `Pending` goes + /// straight up to the task: a caller that reads `Pending` as "nothing now" would + /// mistake the yield for an answer. + pub fn poll_yield(&mut self, waiter: &Waiter) -> Poll<()> { + match self.left.checked_sub(1) { + Some(left) => { + self.left = left; + Poll::Ready(()) + } + None => { + self.left = self.passes; + waiter.waker().wake_by_ref(); + Poll::Pending + } + } + } +} + +#[cfg(all(test, not(loom)))] +mod tests { + use std::{ + sync::{ + Arc, + atomic::{AtomicUsize, Ordering}, + }, + task::{Wake, Waker}, + }; + + use super::*; + + struct Count(AtomicUsize); + + impl Wake for Count { + fn wake(self: Arc) { + self.0.fetch_add(1, Ordering::SeqCst); + } + } + + #[test] + fn yields_after_its_passes_and_refills() { + let wakes = Arc::new(Count(AtomicUsize::new(0))); + let waiter = Waiter::new(Waker::from(wakes.clone())); + let mut budget = Budget::new(4); + for round in 1..=3 { + for _ in 0..4 { + assert!(budget.poll_yield(&waiter).is_ready()); + } + assert!(budget.poll_yield(&waiter).is_pending()); + assert_eq!(wakes.0.load(Ordering::SeqCst), round, "a yield must wake the task"); + } + } +} diff --git a/rs/kio/src/lib.rs b/rs/kio/src/lib.rs index 40f597acbf7..50bb0def9a8 100644 --- a/rs/kio/src/lib.rs +++ b/rs/kio/src/lib.rs @@ -12,6 +12,9 @@ //! [`Queue`] is a poll-native FIFO queue built in the same style: role-less //! clone-able handles, bounded or unbounded, with separate wake lists for the //! push and pop sides. +//! +//! [`coop`] bounds the passes a loop makes per poll, so one that always finds +//! work ready still yields. use std::{ fmt, @@ -24,6 +27,8 @@ mod lock; mod sync; mod waiter; +pub mod coop; + mod consumer; mod pending; mod producer; diff --git a/rs/moq-net/benches/session.rs b/rs/moq-net/benches/session.rs index c949251303b..6078d923458 100644 --- a/rs/moq-net/benches/session.rs +++ b/rs/moq-net/benches/session.rs @@ -81,6 +81,9 @@ struct Shape { frame: usize, /// Write one frame per round, as a live source does, instead of a whole group. paced: bool, + /// Whole groups each watched broadcast writes per round, so a serve loop finds a + /// backlog ready: the group rate a round stands in for. + burst: usize, } impl Shape { @@ -94,8 +97,15 @@ impl Shape { watch: 1, frame: 64, paced: false, + burst: 1, }; + /// What each viewer subscribes with: a burst is a backlog, which the default + /// subscription writes off as late. + fn subscription(&self) -> Option { + (self.burst > 1).then(|| track::Subscription::default().with_max_delay(Duration::from_secs(30))) + } + fn total(&self) -> usize { self.publishers * self.broadcasts } @@ -107,19 +117,28 @@ impl Shape { /// Payload bytes every viewer reads in one round. fn expected(&self) -> usize { - self.viewers * self.watch * self.frames() * self.frame + self.viewers * self.watch * self.burst * self.frames() * self.frame } fn id(&self, version: &str) -> String { format!( - "{version}/relays={}/publishers={}/broadcasts={}/viewers={}/watch={}/frame={}{}", + "{version}/relays={}/publishers={}/broadcasts={}/viewers={}/watch={}/frame={}{}{}", self.relays, self.publishers, self.broadcasts, self.viewers, self.watch, self.frame, - if self.paced { "/paced" } else { "" }, + if self.paced { + "/paced".to_string() + } else { + String::new() + }, + if self.burst > 1 { + format!("/burst={}", self.burst) + } else { + String::new() + }, ) } } @@ -289,11 +308,17 @@ struct Viewer { impl Viewer { /// Resolve `path` and subscribe to each of its `tracks`. - async fn watch(&mut self, path: &str, tracks: &[impl AsRef]) { + async fn watch(&mut self, path: &str, tracks: &[impl AsRef], subscription: Option) { let broadcast = self.origin.consume().routed_broadcast(path).await.unwrap(); for track in tracks { - self.subscribers - .push(broadcast.track(track.as_ref()).unwrap().subscribe(None).await.unwrap()); + self.subscribers.push( + broadcast + .track(track.as_ref()) + .unwrap() + .subscribe(subscription.clone()) + .await + .unwrap(), + ); } } } @@ -320,6 +345,7 @@ impl Room { shape.watch <= shape.total(), "a viewer can't watch more broadcasts than exist" ); + assert!(!shape.paced || shape.burst == 1, "a paced round writes one frame"); let mut cluster = Cluster::new(version, shape).await; // Viewer v watches a contiguous window starting at v * watch, so viewers @@ -337,7 +363,7 @@ impl Room { // publisher it watches whenever there is more than one. let mut viewer = cluster.join((viewer + 1) % shape.relays).await; for broadcast in broadcasts { - viewer.watch(&path(broadcast), &[TRACK]).await; + viewer.watch(&path(broadcast), &[TRACK], shape.subscription()).await; } viewers.push(viewer); } @@ -371,12 +397,17 @@ impl Room { return self.paced_round().await; } for &broadcast in &self.watched { - write_group(&self.cluster.tracks[broadcast], &self.payload, FRAMES); + for _ in 0..self.shape.burst { + write_group(&self.cluster.tracks[broadcast], &self.payload, FRAMES); + } } let mut bytes = 0; - for viewer in &mut self.viewers { - for subscriber in &mut viewer.subscribers { - bytes += read_group(subscriber).await; + // Round-robin, so no subscription's backlog waits on another's being read. + for _ in 0..self.shape.burst { + for viewer in &mut self.viewers { + for subscriber in &mut viewer.subscribers { + bytes += read_group(subscriber).await; + } } } assert_eq!(bytes, self.shape.expected()); @@ -472,7 +503,7 @@ fn join(c: &mut Criterion, name: &str, shapes: impl IntoIterator) let broadcast = hosted[iter % hosted.len()]; let start = Instant::now(); let mut viewer = cluster.join(relay).await; - viewer.watch(&path(broadcast), &[TRACK]).await; + viewer.watch(&path(broadcast), &[TRACK], None).await; let bytes = read_group(&mut viewer.subscribers[0]).await; elapsed += start.elapsed(); assert_eq!(bytes, FRAMES * shape.frame); @@ -560,7 +591,7 @@ impl DashRoom { let mut dash = cluster.join(0).await; for path in &paths { - dash.watch(path, &names).await; + dash.watch(path, &names, None).await; } let mut room = Self { @@ -696,7 +727,7 @@ impl ChurnRoom { let mut broadcasts = Vec::new(); for _ in 0..shape.sessions { let mut viewer = cluster.join(0).await; - viewer.watch(&path(0), &held).await; + viewer.watch(&path(0), &held, None).await; broadcasts.push(viewer.origin.consume().routed_broadcast(&path(0)).await.unwrap()); viewers.push(viewer); } @@ -852,6 +883,20 @@ fn session(c: &mut Criterion) { }), ); delivery(c, "frame", [64, 1024, 16 * 1024].map(|frame| Shape { frame, ..base })); + // One session serving a backlog on each of its subscriptions: the serve loop that a + // `kio::coop::Budget` bounds per poll. + delivery( + c, + "burst", + [(1, 1), (1, 8), (1, 64), (8, 1), (8, 8)].map(|(watch, burst)| Shape { + publishers: 1, + broadcasts: 16, + viewers: 1, + watch, + burst, + ..base + }), + ); delivery( c, "relays", diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index 9bb9061bd23..0c8d4a4b282 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -123,7 +123,10 @@ async fn read_fetch( let first = group.index(); let mut frames = Vec::new(); let mut complete = false; + // A long cached group is read a slice per poll, so the session's other tasks still run. + let mut budget = kio::coop::Budget::new(32); while until.is_none_or(|until| first + (frames.len() as u64) < until) { + kio::wait(|waiter| budget.poll_yield(waiter)).await; match group.read_frame().await? { Some(frame) => frames.push(frame), None => { @@ -1125,7 +1128,10 @@ where let mut first = true; let mut buf: frame::Buffer = frame::Buffer::new(); + // A long cached group is written a slice per poll, so the session's other tasks still run. + let mut budget = kio::coop::Budget::new(32); 'serve: loop { + kio::wait(|waiter| budget.poll_yield(waiter)).await; // The cap is the Largest Object snapshot: the group may keep growing, but // everything past the snapshot belongs to the subscription, not the fill. if until.is_some_and(|until| index >= until) { @@ -2662,6 +2668,8 @@ struct TrackServe { /// Serve the track's datagrams too, as OBJECT_DATAGRAMs. Off when the transport has no /// datagrams; there is no stream fallback. datagrams: bool, + /// A track that always has another group ready still lets the session's other tasks run. + budget: kio::coop::Budget, } impl TrackServe { @@ -2700,6 +2708,7 @@ impl TrackServe { draining: false, opened: Default::default(), end: None, + budget: kio::coop::Budget::new(32), } } @@ -2758,6 +2767,10 @@ impl TrackServe { let _ = self.children.poll(waiter); loop { + // Still serves the datagrams and the groups already started below. + if self.budget.poll_yield(waiter).is_pending() { + break; + } match self.track.poll_recv_group(waiter) { Poll::Ready(Ok(Some(group))) => { let sequence = group.sequence; @@ -2897,6 +2910,8 @@ struct GroupServe { version: Version, object_delta: u64, state: GroupState, + /// A long cached group drains a slice per poll, so its siblings still run. + budget: kio::coop::Budget, } // A state machine's enum is its storage: one transient instance per stream, so the @@ -2956,6 +2971,7 @@ impl GroupServe { version, object_delta, state: GroupState::Open, + budget: kio::coop::Budget::new(32), } } @@ -3012,6 +3028,7 @@ impl GroupServe { } let res = 'serve: { loop { + ready!(self.budget.poll_yield(waiter)); match writer.poll_flush(&mut cx) { Poll::Ready(Ok(())) => {} Poll::Ready(Err(err)) => break 'serve Err(err), @@ -3333,6 +3350,132 @@ mod group_priority_test { ); } + /// Drive `serve` beside a sibling that only counts its turns, until `done`, checking every + /// poll stays under `most` by `progress` and leaves the sibling its turn. + fn serve_beside_a_sibling( + mut serve: TrackServe, + mut produce: impl FnMut() -> bool + 'static, + progress: impl Fn() -> usize, + most: usize, + done: usize, + ) { + type Task = Box Poll<()>>; + let mut tasks: kio::Tasks = kio::Tasks::new(); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + if produce() { + return Poll::Ready(()); + } + waiter.waker().wake_by_ref(); + Poll::Pending + })); + let turns = std::sync::Arc::new(AtomicU64::new(0)); + let counted = turns.clone(); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + counted.fetch_add(1, Ordering::Relaxed); + waiter.waker().wake_by_ref(); + Poll::Pending + })); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + serve.poll(waiter).map(|res| res.expect("serve")) + })); + + let owner = kio::Waiter::noop(); + let mut polls = 0; + while progress() < done { + let before = progress(); + assert!(tasks.poll(&owner).is_pending(), "the serve never ends"); + polls += 1; + let made = progress() - before; + assert!(made <= most, "one poll made {made} progress"); + assert_eq!(turns.load(Ordering::Relaxed), polls, "the sibling missed a turn"); + assert!(polls <= 2 * done as u64, "the serve stalled"); + } + } + + /// A track that always has another group ready yields within its budget. + #[test] + fn a_busy_track_yields_to_its_siblings() { + /// `TrackServe`'s passes per poll; each starts at most one group. + const BUDGET: usize = 32; + const GROUPS: u64 = 10_000; + const BATCH: u64 = 1_000; + + let session = SinkSession::new(crate::lite::test_transport::Log::default()).with_unacked_fin(); + let track = track::Producer::new(std::sync::Arc::new(crate::broadcast::Info::default()), "test", None); + let subscriber = + track.subscribe(track::Subscription::default().with_max_delay(std::time::Duration::from_secs(3600))); + let serve = TrackServe::new( + session, + subscriber, + RequestId(0), + Version::Draft14, + ServeRange::default(), + None, + ); + let opened = serve.opened.clone(); + + let producer = track.clone(); + let mut appended = 0; + let produce = move || { + for _ in 0..BATCH.min(GROUPS - appended) { + let mut group = producer.create_group(group::Info { sequence: appended }).unwrap(); + group + .write_frame(crate::Timestamp::from_millis(appended).unwrap(), b"x".as_slice()) + .unwrap(); + group.finish().unwrap(); + appended += 1; + } + appended == GROUPS + }; + serve_beside_a_sibling( + serve, + produce, + || opened.load(Ordering::Relaxed) as usize, + BUDGET, + GROUPS as usize, + ); + } + + /// A group with more objects ready than one poll's budget drains over several polls. + #[test] + fn a_long_group_drains_within_the_budget() { + /// `GroupServe`'s passes per poll; each writes at most one object. + const BUDGET: usize = 32; + const FRAMES: usize = 4_000; + const PAYLOAD: usize = 100; + + let log = crate::lite::test_transport::Log::default(); + let session = SinkSession::new(log.clone()).with_unacked_fin(); + let track = track::Producer::new(std::sync::Arc::new(crate::broadcast::Info::default()), "test", None); + let subscriber = + track.subscribe(track::Subscription::default().with_max_delay(std::time::Duration::from_secs(3600))); + let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); + for millis in 0..FRAMES as u64 { + group + .write_frame(crate::Timestamp::from_millis(millis).unwrap(), vec![0u8; PAYLOAD]) + .unwrap(); + } + group.finish().unwrap(); + let serve = TrackServe::new( + session, + subscriber, + RequestId(0), + Version::Draft14, + ServeRange::default(), + None, + ); + + // An object's header takes well under a payload more, and a poll may also flush what + // the last one buffered. + serve_beside_a_sibling( + serve, + || true, + || log.writes.lock().unwrap().len(), + 2 * (BUDGET + 1) * 2 * PAYLOAD, + FRAMES * PAYLOAD, + ); + } + /// Serve one group at `version` with `timescale` and return the header plus what follows it. async fn serve_group(version: Version, timescale: Option) -> (ietf::GroupHeader, bytes::Bytes) { let log = crate::lite::test_transport::Log::default(); diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 370ec4ca544..a7679fa9263 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -1067,6 +1067,8 @@ struct RequestServe> { // Log context, filled in after the decode. absolute: crate::PathOwned, track: String, + // A reply that always has another group ready still lets the session's other tasks run. + budget: kio::coop::Budget, } enum RequestState { @@ -1103,6 +1105,7 @@ impl> RequestServe { update: None, absolute: Default::default(), track: Default::default(), + budget: kio::coop::Budget::new(32), } } @@ -1138,6 +1141,7 @@ impl> RequestServe { fn poll_serve(&mut self, waiter: &kio::Waiter) -> Poll> { let mut cx = waiter.context(); loop { + ready!(self.budget.poll_yield(waiter)); // Once answered, the requester's FIN is the normal end, not a cancel. if matches!(self.state, RequestState::Resolve { .. } | RequestState::Serve(_)) && let Poll::Ready(res) = self.poll_requester(&mut cx) @@ -2969,6 +2973,8 @@ struct GroupServe { // subsequent delta is signed against the previous frame. prev_ts: u64, state: GroupState, + // A long cached group drains a slice per poll, so its siblings still run. + budget: kio::coop::Budget, } // A state machine's enum is its storage: one transient instance per stream, so the @@ -3011,6 +3017,7 @@ impl GroupServe { frame_start, prev_ts: 0, state: GroupState::Open, + budget: kio::coop::Budget::new(32), } } @@ -3108,6 +3115,7 @@ impl GroupServe { break 'serve Err(Error::Cancel); } loop { + ready!(self.budget.poll_yield(waiter)); match writer.poll_flush(&mut cx) { Poll::Ready(Ok(())) => {} Poll::Ready(Err(err)) => break 'serve Err(err), @@ -3763,6 +3771,58 @@ mod serve_group_test { group.finish().unwrap(); } + /// A group with more frames ready than one poll's budget drains over several polls, + /// so a sibling (the transport's driver, say) runs in between. + #[test] + fn a_long_group_drains_within_the_budget() { + /// `GroupServe`'s passes per poll; each writes at most one frame. + const BUDGET: usize = 32; + const FRAMES: usize = 4_000; + const PAYLOAD: usize = 100; + + type Task = Box Poll<()>>; + + let track = track::Producer::new(Arc::new(broadcast::Info::default()), "test", None); + let subscriber = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let log = Log::default(); + let (mut run, mut writer, _) = lite07_run(SinkSession::new(log.clone()).with_unacked_fin(), subscriber); + + let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); + for millis in 0..FRAMES as u64 { + group + .write_frame(Timestamp::from_millis(millis).unwrap(), vec![0u8; PAYLOAD]) + .unwrap(); + } + group.finish().unwrap(); + + let mut tasks: kio::Tasks = kio::Tasks::new(); + let turns = Arc::new(AtomicU64::new(0)); + let counted = turns.clone(); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + counted.fetch_add(1, Ordering::Relaxed); + waiter.waker().wake_by_ref(); + Poll::Pending + })); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + run.poll(&mut writer, waiter).map(|res| res.expect("serve")) + })); + + // A frame's header takes well under a payload more, and a poll may also flush what + // the last one buffered. + let most = 2 * (BUDGET + 1) * 2 * PAYLOAD; + let owner = kio::Waiter::noop(); + let mut polls = 0; + while log.writes.lock().unwrap().len() < FRAMES * PAYLOAD { + let before = log.writes.lock().unwrap().len(); + assert!(tasks.poll(&owner).is_pending(), "the serve never ends"); + polls += 1; + let wrote = log.writes.lock().unwrap().len() - before; + assert!(wrote <= most, "one poll wrote {wrote} bytes"); + assert_eq!(turns.load(Ordering::Relaxed), polls, "the sibling missed a turn"); + assert!(polls <= FRAMES as u64, "the serve stalled"); + } + } + /// SUBSCRIBE_END counts the group streams opened, not the groups below the end: a /// group the track never produced has no stream and is not counted. #[moq_net_sim::test] @@ -4669,6 +4729,129 @@ mod tests { assert_eq!(*run.track_priority_tx.read(), 7, "the update was lost"); } + /// A subscription that always has another group ready yields within its budget, so a + /// sibling (the transport's driver, say) runs between its polls. Without it, one poll + /// served every group the producer kept appending and starved the rest of the runtime + /// (a go publisher's 2.5 ms audio groups timed its session out). + #[moq_net_sim::test] + async fn a_busy_subscription_yields_to_its_siblings() { + /// `RequestServe`'s passes per poll; each serves at most one group. + const BUDGET: u64 = 32; + const GROUPS: u64 = 10_000; + const BATCH: u64 = 1_000; + + type Task = Box Poll<()>>; + + let origin = crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(); + let broadcast = origin.create_broadcast("room").unwrap(); + let mut dynamic = broadcast.dynamic(); + broadcast.announce(Default::default()).unwrap(); + + let peer_setup = crate::lite::PeerSetup::default(); + peer_setup.set(crate::lite::Setup::default()); + let (_, goaway) = crate::goaway::Handle::new(true); + let version = Version::Lite07; + let publisher = Publisher::new(PublisherConfig { + runtime: crate::time::Clock::sim(), + session: ScriptedSession::new(Vec::new()), + origin: origin.consume(), + version, + peer_setup, + goaway, + peer_hop: None, + subscriptions: Default::default(), + }); + + let mut script = Vec::new(); + lite::Subscribe { + epoch: None, + id: 0, + broadcast: crate::Path::new("room"), + track: "video".into(), + priority: 0, + max_delay: Duration::from_secs(3600), + start_group: None, + end_group: None, + start_frame: 0, + end_frame: None, + } + .encode(&mut crate::coding::Encoder::new(&mut script, version.into()), version) + .unwrap(); + let mut session = ScriptedSession::new(script); + let (send, recv) = futures::future::poll_fn(|cx| { + ::poll_open_bi(&mut session, cx) + }) + .await + .unwrap(); + let stream = Stream:: { + writer: Writer::new(send, version), + reader: crate::coding::Reader::new(recv, version), + }; + let mut serve = RequestServe::<_, SubscribeServe>::new(publisher.shared.clone(), stream); + assert!(!drive(&mut serve).await); + let track = dynamic + .requested_track() + .now_or_never() + .expect("track requested") + .unwrap() + .accept(None); + assert!(!drive(&mut serve).await, "the subscription ended"); + let RequestState::Serve(SubscribeServe::Run(run)) = &serve.state else { + panic!("the subscription is not running"); + }; + let opens = run.ctx.opens.clone(); + + let mut tasks: kio::Tasks = kio::Tasks::new(); + + // The producer appends a batch of ready groups on each of its turns. + let mut appended = 0; + tasks.push(Box::new(move |waiter: &kio::Waiter| { + for _ in 0..BATCH.min(GROUPS - appended) { + let mut group = track.create_group(group::Info { sequence: appended }).unwrap(); + group + .write_frame(crate::Timestamp::from_millis(appended).unwrap(), b"x".as_slice()) + .unwrap(); + group.finish().unwrap(); + appended += 1; + } + if appended == GROUPS { + return Poll::Ready(()); + } + waiter.waker().wake_by_ref(); + Poll::Pending + })); + + // Stands in for the transport's driver: it only needs a turn. + let turns = Arc::new(AtomicU64::new(0)); + let counted = turns.clone(); + tasks.push(Box::new(move |waiter: &kio::Waiter| { + counted.fetch_add(1, Ordering::Relaxed); + waiter.waker().wake_by_ref(); + Poll::Pending + })); + + tasks.push(Box::new(move |waiter: &kio::Waiter| { + serve.poll(waiter).map(|res| res.expect("serve")) + })); + + let owner = kio::Waiter::noop(); + let mut polls = 0; + while opens.opened.load(Ordering::Relaxed) < GROUPS { + let before = opens.opened.load(Ordering::Relaxed); + assert!(tasks.poll(&owner).is_pending(), "the serve never ends"); + polls += 1; + let served = opens.opened.load(Ordering::Relaxed) - before; + assert!(served <= BUDGET, "one poll served {served} groups"); + assert_eq!(turns.load(Ordering::Relaxed), polls, "the sibling missed a turn"); + assert!(polls <= 2 * GROUPS, "the serve stalled"); + } + assert!( + session.log.resets().is_empty(), + "a group was reset: {:?}", + session.log.resets() + ); + } + /// A SUBSCRIBE or a TRACK past the session's cap closes the session with /// TOO_MANY_REQUESTS. #[moq_net_sim::test]