diff --git a/quest/m1/release-perf/README.md b/quest/m1/release-perf/README.md index b3b29a96056..b42debd821f 100644 --- a/quest/m1/release-perf/README.md +++ b/quest/m1/release-perf/README.md @@ -35,8 +35,14 @@ noise. release's bare track handle opened a live SUBSCRIBE, while main sends a real FETCH per viewer ([#5820](https://github.com/moq-dev/moq/pull/5820) made handles lazy, and [#4974](https://github.com/moq-dev/moq/pull/4974) added - fetch-only demand). The lite `/fetch` cost is different: it is real, and - [FETCH fan-out](/quest/m1/release-perf/fetch-fanout.md) owns it. + fetch-only demand). The lite `/fetch` cost was real: a relay sent one + upstream FETCH per group, but revived the parked logical track for every + viewer's FETCH. [#6286](https://github.com/moq-dev/moq/pull/6286) serves a + whole cached group from the parked copy instead, cutting 12 to 19% of the + instructions per viewer where viewers share a group. The rest of the gap to + a viewer that holds a query is accepted: each viewer's own fetch of a group + it lacks revives its logical track, so the open group keeps its route and + fails over mid-read, and its copy releases the route's demand once idle. - `stats/session-report` is accepted at O(live sessions), after a cheap fix. Per-label payload bytes ([#5731](https://github.com/moq-dev/moq/pull/5731)) match payload billing per label, such as per signing key. After removing @@ -96,7 +102,6 @@ outside the worktree to keep the next one. ## Required -- [FETCH fan-out](/quest/m1/release-perf/fetch-fanout.md) - a relay serves later viewers' lite FETCHes of a group from its cache, so per-viewer cost falls with viewers again - [Sparse gaps](/quest/m1/release-perf/sparse-gaps.md) - an arrival and a gap flush cost O(log gaps), not O(gaps) - [Idle GC](/quest/m1/release-perf/gc-idle.md) - an idle track GC skips the wall-clock pass when nothing is due - [Re-check](/quest/m1/release-perf/recheck.md) - the screen-only moves are re-measured with instruction counts, and the ones that hold are fixed or accepted diff --git a/quest/m1/release-perf/fetch-fanout.md b/quest/m1/release-perf/fetch-fanout.md deleted file mode 100644 index 536d3c4a2b7..00000000000 --- a/quest/m1/release-perf/fetch-fanout.md +++ /dev/null @@ -1,44 +0,0 @@ -# [M] FETCH fan-out - -## Goal - -When many viewers FETCH the same group through a relay over moq-lite, the -relay fetches it upstream once and serves later viewers from its cache. Cost -per viewer falls as viewers grow, as it does on `release`. - -## Plan - -Measured 2026-10-21 (`session_delivery_group_cost/.../fetch` on lite-06, -cost per viewer): - -| Viewers | release | main nightly | -|---|---|---| -| 1 | 57 us | 67 us | -| 16 | n/a | 61 us | -| 64 | 16.5 us | 59 us | - -Main's non-fetch path still amortizes (55 us down to 34 us), so on main every -viewer's FETCH appears to cost as much as the first. The relay seems to go -upstream or rebuild state for each fetch. Lite's `IDLE_LINGER` should keep -the relay's copy and cache, so a simple "the track was dropped" explanation is -not enough. - -1. Reproduce first. Count the upstream FETCHes the publisher session sees in - each round. One per group means the cache works and the cost is elsewhere. - One per viewer means the cache is bypassed. -2. If it is bypassed, compare callgrind instruction counts at the parents of - the suspects. The likely fix is for the relay's FETCH serve to resolve its - track once and reuse the lingering copy. Suspects: - [#5820](https://github.com/moq-dev/moq/pull/5820) (the relay's FETCH serve - resolves `broadcast.track(..).fetch_group` per request), - [#5809](https://github.com/moq-dev/moq/pull/5809) (a fetch keeps its track - read), [#5680](https://github.com/moq-dev/moq/pull/5680) (fetch age - budgets; the bench advances the cache clock 1 s per round), and - [#6020](https://github.com/moq-dev/moq/pull/6020) (one writer per group). -3. Land a regression test that asserts one upstream FETCH per group across - many viewers. Also check that the `session_delivery_group_cost` lite - `/fetch` sweep over viewers slopes down again. - -A relay that fetches upstream once per viewer turns N viewers into N upstream -requests. If step 1 confirms that, raise the priority with the maintainer, -since m0 bounds what a peer can make the relay do. diff --git a/rs/moq-net/benches/session.rs b/rs/moq-net/benches/session.rs index 5110aa3422a..c8baa670ceb 100644 --- a/rs/moq-net/benches/session.rs +++ b/rs/moq-net/benches/session.rs @@ -16,6 +16,7 @@ //! groups have a `_cpu` suffix. Cache-clock advancement remains outside both clocks. //! `session_delivery_group_cost` sweeps publishers and viewers independently for SUBSCRIBE //! and FETCH; its constrained cases release held streams only after a real blocked write. +//! A one-relay lite FETCH fixture asserts one upstream FETCH per group during setup. //! `session_delivery_credit` grants a viewer one uni stream at a time, swept over its //! subscriptions and the backlog each holds, so every group waits for a stream grant. A //! publisher that parks state per backlogged group pays for it on each grant, which shows @@ -576,6 +577,22 @@ impl Room { "the constrained fixture must exercise flow control" ); } + if shape.fetch && shape.relays == 1 && room.cluster.version.is_lite() { + let upstream_fetches = room + .cluster + ._pairs + .iter() + .map(|pair| { + // Lite FETCH's stream type is 3, encoded in one byte. + pair.server_transport + .bidi_types() + .into_iter() + .filter(|kind| *kind == 3) + .count() + }) + .sum::(); + assert_eq!(upstream_fetches, room.groups, "one upstream FETCH per written group"); + } room } diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index d59a83a9456..710ff840845 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -3272,6 +3272,9 @@ async fn serve_front(task: FrontTask) -> Vec { if io.staged.as_ref().is_some_and(|asked| asked.source == source) { io.staged = None; } + // Any source leaving is rare: drop every parked shortcut rather + // than track which source each came from. + io.routes.unpark(); } } Action::Resolve => { @@ -12302,6 +12305,69 @@ mod tests { drop(upstream); } + /// The reader leaves, parking the track. A fetch of a whole group the parked copy + /// caches answers at once, without the front serving the copy again; an open group + /// still waits on the front, and is served once the copy is back. + #[moq_clock::test] + #[cfg(feature = "async")] + async fn a_parked_track_serves_whole_groups_from_its_copy() { + let (producer, _server, _upstream, old, subscription) = remote_front().await; + let resolved = producer.consume().request_broadcast("room/alice", None).await.unwrap(); + let mut open = old.append_group().unwrap(); + open.write_frame(crate::Timestamp::ZERO, b"open".as_ref()).unwrap(); + drop(subscription); + moq_clock::timeout(Duration::from_secs(1), old.demand().until(|wanted| wanted.is_none())) + .await + .expect("parked") + .expect("open"); + + let track = resolved.track("video").unwrap(); + let mut whole = track + .fetch_group(0, None) + .now_or_never() + .expect("a whole group needs no front") + .unwrap(); + assert_eq!(&whole.read_frame().await.unwrap().unwrap().payload[..], b"before"); + + let mut fetching = track.fetch_group(1, None); + assert!( + (&mut fetching).now_or_never().is_none(), + "an open group needs the route" + ); + open.finish().unwrap(); + let mut group = fetching.await.unwrap(); + assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], b"open"); + } + + /// A new source takes over a parked track: a whole group its old copy caches is no + /// longer served from that copy, but asked of the new route, which lacks it. + #[moq_clock::test] + #[cfg(feature = "async")] + async fn a_new_source_unparks_the_old_copy() { + let (producer, _server, _upstream, old, subscription) = remote_front().await; + let resolved = producer.consume().request_broadcast("room/alice", None).await.unwrap(); + drop(subscription); + moq_clock::timeout(Duration::from_secs(1), old.demand().until(|wanted| wanted.is_none())) + .await + .expect("parked") + .expect("open"); + + let newcomer = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); + let _video = newcomer.create_track("video", crate::track::Info::timed()).unwrap(); + for _ in 0..10 { + moq_clock::sim::yield_now().await; + } + + let mut fetching = resolved.track("video").unwrap().fetch_group(0, None); + assert!( + (&mut fetching).now_or_never().is_none(), + "served from the detached copy" + ); + assert!(matches!(fetching.await, Err(Error::NotFound))); + } + /// A newer local source's copy of "video" is refused for its metadata while the /// incumbent keeps serving: the refused copy is let go, not kept subscribed. #[moq_clock::test] diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index a62d857a1aa..f962a704c2b 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -47,6 +47,10 @@ struct Route { /// The serving route's copy; `None` while nobody reads the track, between routes, or /// once the front let go. copy: Option, + /// The copy the front let go of when nobody read the track, until the front serves + /// another or its source detaches: a whole group it still caches is served without + /// waking the front. A lazy handle, so it keeps neither the copy nor its route alive. + parked: Option, /// How the track ended, once the front decided. end: Option>, /// No front is left to replace the copy, though one may still hold it for readers @@ -122,6 +126,7 @@ impl Producer { if let Ok(mut route) = self.state.write() { route.generation += 1; route.copy = Some(copy); + route.parked = None; } let readers = { let mut readers = self.readers.lock().unwrap_or_else(PoisonError::into_inner); @@ -142,7 +147,15 @@ impl Producer { && route.copy.is_some() { route.generation += 1; - route.copy = None; + route.parked = route.copy.take(); + } + } + + /// Stop serving whole groups from the parked copy: its source detached, so a fetch + /// resolves the current route again. + pub(super) fn unpark(&self) { + if let Ok(mut route) = self.state.write() { + route.parked = None; } } @@ -168,6 +181,7 @@ impl Producer { pub(super) fn release(self) { if let Ok(mut route) = self.state.write() { route.copy = None; + route.parked = None; } } @@ -192,6 +206,15 @@ impl Consumer { self.state.read().copy.clone() } + /// The copy the front parked, while it serves none and may still serve one again. + pub(super) fn parked(&self) -> Option { + let route = self.state.read(); + match route.copy.is_none() && route.end.is_none() && !route.concluded && !route.is_closed() { + true => route.parked.clone(), + false => None, + } + } + /// Poll for a largest location within `bounds` newer than `seen` on the serving copy, /// recording it there. A copy ending is its route failing, so the logical track ends /// only with the route: at once when the front ends it, or with the last copy once the diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 3b75eb5b41f..dcab85d3f56 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4392,6 +4392,32 @@ impl Consumer { }) } + /// The group a front's parked copy would serve for `sequence` from `frame_start`, when it + /// is whole: finished from frame 0 on a copy still open. Anything less (open, headless, + /// aborted, or on a failed copy) needs the route. The fetch joins the copy's demand, so + /// a copy kept busy only by fetches stays cached at its session. + fn parked_whole(&self, sequence: u64, frame_start: u64) -> Option<(kio::Consumer, group::Consumer)> { + let copy = self.with_state(|state| state?.read().routes.as_ref()?.parked())?; + copy.with_state(|state| { + let state = state?; + let group = { + let track = state.read(); + let group = track.covering_group(sequence, frame_start)?; + let whole = !track.is_closed() + && track.abort.is_none() + && group.live_first_frame() == Some(0) + && group.is_finished(); + if !whole { + return None; + } + let consumer = group.try_consume()?; + group.cache_refresh(); + consumer + }; + Some((state.consume(), group)) + }) + } + /// The serving route's copy, for a front's logical track; see [`super::resume`]. fn serving(&self) -> Option { let routes = self.with_state(|state| state?.read().routes.clone())?; @@ -4443,14 +4469,32 @@ impl Consumer { /// handler request. pub fn fetch_group(&self, sequence: u64, options: impl Into>) -> kio::Pending { let options = options.into().unwrap_or_default(); - let state = self.resolve(None).0; - let resume = state.read().routes.clone(); // One fetch per calling context, counted here (coalesced upstream work is // still one request served). Independent of `subscriptions` and the viewer // refcount. self.stats.fetch(); + // Nothing more feeds a whole group, so no route needs to stay up for it: the copy a + // front parked serves it without reviving the logical track for one read. + if let Some((state, group)) = self.parked_whole(sequence, options.frame_start) { + let fetch = state.read().fetch.clone(); + return kio::Pending::new(Fetching { + state, + fetch, + sequence, + frame_start: options.frame_start, + priority: options.priority, + max_delay: options.max_delay, + result: None, + hit: Some(Box::new(group)), + stats: self.stats.clone(), + resume: None, + }); + } + + let state = self.resolve(None).0; + let resume = state.read().routes.clone(); let mut result = None; // A front's logical track caches nothing: the serving route answers. diff --git a/rs/moq-net/tests/integration/fetch_fanout.rs b/rs/moq-net/tests/integration/fetch_fanout.rs new file mode 100644 index 00000000000..3b09be36bf8 --- /dev/null +++ b/rs/moq-net/tests/integration/fetch_fanout.rs @@ -0,0 +1,83 @@ +#![cfg(feature = "async")] + +//! A relay fetches each group's payload upstream once, independently of how many +//! viewers ask for it, and keeps serving the cached copy after a reader leaves. + +use crate::support; +use moq_net::{Hop, Version, origin, track}; +use std::time::Duration; +use support::harness::{MockConnectOptions, connect_mock}; + +fn origin(hop: u64) -> origin::Producer { + let (origin, driver) = origin::Producer::new(origin::Config::new(Hop::new(hop).unwrap())); + support::harness::spawn(driver); + origin +} + +#[moq_clock::test] +async fn sequential_viewers_share_the_relays_fetched_groups() { + for version in ["moq-lite-05", "moq-lite-06", "moq-lite-07-wip"] { + let version: Version = version.parse().unwrap(); + let publisher = origin(1); + let broadcast = publisher.publish("room", Default::default()).unwrap(); + let track = broadcast + .create_track( + "video", + track::Info::default().with_timescale(moq_net::Timescale::MILLI), + ) + .unwrap(); + let relay = origin(2); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(relay.clone()); + let upstream = connect_mock(options).await; + let mut viewers = Vec::new(); + for i in 0..64 { + let viewer = origin(3 + i); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(relay.consume()); + options.client_subscribe = Some(viewer.clone()); + let pair = connect_mock(options).await; + let remote = viewer.consume().routed_broadcast("room").await.unwrap(); + viewers.push((viewer, pair, remote)); + } + for round in 0..3 { + let mut group = track.append_group().unwrap(); + let sequence = group.sequence; + group + .write_frame( + moq_net::Timestamp::from_millis(round * 1000).unwrap(), + b"payload".as_ref(), + ) + .unwrap(); + group.finish().unwrap(); + for (i, (_, _, remote)) in viewers.iter().enumerate() { + let mut group = remote + .track("video") + .unwrap() + .fetch_group(sequence, None) + .await + .unwrap(); + assert_eq!(group.read_frame().await.unwrap().unwrap().payload.as_ref(), b"payload"); + assert!(group.read_frame().await.unwrap().is_none()); + // FETCH's lite stream type is 3, and its varint occupies one byte. + let fetches = upstream + .client_transport + .bidi_types() + .into_iter() + .filter(|kind| *kind == 3) + .count(); + assert_eq!(fetches, round as usize + 1, "{version}: viewer {i}, group {sequence}"); + } + moq_clock::sleep(Duration::from_secs(1)).await; + } + let types = upstream.client_transport.bidi_types(); + assert_eq!( + types.iter().filter(|kind| **kind == 6).count(), + 1, + "metadata is learned once" + ); + assert!(!types.contains(&2), "a fetch-only relay must never subscribe"); + assert!(track.subscription().is_none()); + } +} diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index e9b9d79d8e5..bde4b316f50 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -21,6 +21,7 @@ mod empty_group; mod extensions; mod fetch_cancel; mod fetch_caps; +mod fetch_fanout; mod fetch_from_feed; mod fetch_only; mod finished_broadcast_mock;