From ffa5629db7a60c36c44714f9253036fa321c4ded Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 11:52:38 -0700 Subject: [PATCH 01/21] quest: claim m0/broadcast-epoch/resume-duplicates Co-Authored-By: Claude Opus 5.5 From 5ff7dd5d0360a5ac8ad5eb9d4dce4f807c7d3e69 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 12:38:28 -0700 Subject: [PATCH 02/21] fix(net): hand a late duplicate group out once after a route change A logical track's reader remembered the last 1024 groups it handed out and forgot older ones, so a replacement route delivering an early group after 1024 newer ones handed it out twice. Each copy now remembers only the groups other copies handed out within its own range: from its resume floor through its newest group at replacement. A copy never repeats a group itself, so that is everything it needs, the bookkeeping is bounded by how far the copies overlap, and it goes away with the copy. Genuinely unseen late groups stay eligible. Adds regressions for the late duplicate, ordered readers, repeated route flaps, and held groups with moved bounds, plus an origin/resume_overlap bench over readers and retained, reordered groups. Co-Authored-By: Claude Opus 5.5 --- quest/m0/broadcast-epoch/README.md | 1 - quest/m0/broadcast-epoch/resume-duplicates.md | 38 --- quest/m1/js-group-handover.md | 6 +- quest/m1/resume-reorder.md | 1 - rs/moq-net/benches/origin.rs | 101 +++++++- rs/moq-net/src/model/resume.rs | 236 ++++++++++++++++-- 6 files changed, 318 insertions(+), 65 deletions(-) delete mode 100644 quest/m0/broadcast-epoch/resume-duplicates.md diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index d4885ac745b..8467f76058d 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -123,7 +123,6 @@ End+Start. The child quests own their focused regressions. ## Required -- [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - a delayed group is delivered once even beyond the old 1024-entry history - [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - players and exporters pin siblings consistently, with optional reference epochs and documented legacy best effort ## Related diff --git a/quest/m0/broadcast-epoch/resume-duplicates.md b/quest/m0/broadcast-epoch/resume-duplicates.md deleted file mode 100644 index 3fa1765f039..00000000000 --- a/quest/m0/broadcast-epoch/resume-duplicates.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] Resume never delivers a late group twice - -## Goal - -An unordered reader receives each group once across same-epoch route -changes, even when a duplicate arrives very late. A genuinely unseen late -group remains eligible within the reader's bounds and budget. - -## Plan - -Reproduced in the 2026-10-10 audit: read group 0 from A, switch to B, read -groups 1 through 1024, then deliver B's delayed copy of group 0. The reader -receives group 0 twice. `model/resume.rs` forgets delivered IDs at -`MAX_DELIVERED = 1024` without proving those groups can no longer arrive. - -Decided: this data-correctness fix gates m0. Preserve both duplicate -suppression and unordered delivery; advancing a floor past every late -group would hide the duplication by losing valid data. Bound bookkeeping -using the actual reader/copy lifetime, retained ranges, and terminal -dispositions rather than an arbitrary number of recently delivered IDs. -Choose the representation during implementation; do not retain one entry -per delivered group for the lifetime of an endless broadcast. - -Land the failing regression, a genuinely unseen late group beyond the old -window, repeated route flaps, and ordered-reader controls. Cover readers -that keep old groups open and change their group bounds, and prove finished -copies release their bookkeeping. Tests use mocked time and existing CI -lanes. Extend the resume benchmark over readers and retained/reordered -groups so memory and delivery costs are measured. - -Public API: duplicate suppression is corrected; no signature changes. -Wire: none. Update stale resume comments inline. JS handover must preserve -the corrected contract instead of copying the fixed-size history. - -## Related - -- [Resume reorder](/quest/m1/resume-reorder.md) - an unseen reordered group is not mistaken for a missing one -- [JS track handover](/quest/m1/js-group-handover.md) - the same delivery contract in JS diff --git a/quest/m1/js-group-handover.md b/quest/m1/js-group-handover.md index 7ff59c9dcbe..04aa714982d 100644 --- a/quest/m1/js-group-handover.md +++ b/quest/m1/js-group-handover.md @@ -28,7 +28,10 @@ them at replacement, and recovers a group from the new copy at the first missing frame and byte. It does not copy through the earlier single-writer pump or immediately cancel every old copy. Decided in the 2026-10-10 audit: mirror the current identity, reader, recovery and lifetime contract, including -the late-duplicate correction, rather than porting the obsolete pump. +the late-duplicate correction, rather than porting the obsolete pump. Each copy +remembers the groups other copies handed out within its own range (its resume +floor through its newest group at replacement) and drops that with the copy; +no fixed-size history of delivered groups. Things to settle along the way: - Whether the request's `active` broadcast stays the same object across a swap, @@ -66,7 +69,6 @@ subscriptions across a swap. Report it in the PR. ## Required - [One max_age meaning](/quest/m1/cache-max-age.md) - the rule that gives up a resumed group no route continues -- [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - the corrected once-only delivery contract for reordered copies ## Related diff --git a/quest/m1/resume-reorder.md b/quest/m1/resume-reorder.md index db77e0e8134..0f61117f1e5 100644 --- a/quest/m1/resume-reorder.md +++ b/quest/m1/resume-reorder.md @@ -34,5 +34,4 @@ Public API: behavior only. Wire: none. ## Related -- [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - retain once-only delivery without excluding genuinely unseen late groups - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - explicit terminal dispositions end recovery without waiting for a newer group diff --git a/rs/moq-net/benches/origin.rs b/rs/moq-net/benches/origin.rs index b9d5b9d0492..5956a624a7b 100644 --- a/rs/moq-net/benches/origin.rs +++ b/rs/moq-net/benches/origin.rs @@ -19,6 +19,8 @@ //! //! A route swap on one front is swept over its tracks and the copies each track //! still holds from earlier routes, and a front retiring over the fronts around it. +//! Readers draining a swapped track are swept over their count and the groups +//! both routes delivered. //! //! Run with `cargo bench -p moq-net --bench origin`. @@ -1125,6 +1127,102 @@ fn bench_copy_walk(c: &mut Criterion) { group.finish(); } +/// `(readers, retained)` for [`bench_resume_overlap`]: readers of one track, and +/// the groups both routes hold when it swaps. +const RESUME_OVERLAP: [(usize, usize); 6] = [(1, 1), (1, 64), (1, 1024), (32, 64), (256, 64), (32, 1024)]; + +/// A track that swapped routes before its readers read anything: the replaced copy +/// and the new one hold the same groups, the new one in reverse arrival order. +struct ResumeOverlap { + _producer: origin::Producer, + _generations: [broadcast::Producer; 2], + _tracks: [track::Producer; 2], + readers: Vec, +} + +async fn resume_overlap(readers: usize, retained: usize) -> ResumeOverlap { + let (producer, driver) = origin::Producer::new(origin::Config::default()); + tokio::spawn(moq_net::time::run(driver)); + let epoch = Epoch::mint(); + let generation = |order: &mut dyn Iterator| { + let broadcast = producer.create_broadcast(COPY_PATH).unwrap(); + let track = broadcast.create_track("0", timed()).unwrap(); + for sequence in order { + let mut group = track + .create_group(moq_net::group::Info { + sequence: sequence as u64, + }) + .unwrap(); + group.write_frame(Timestamp::ZERO, b"f".as_ref()).unwrap(); + group.finish().unwrap(); + } + (broadcast, track) + }; + let (first, first_track) = generation(&mut (0..retained)); + first + .announce(origin::Route::default().with_epoch(epoch.clone())) + .unwrap(); + let resolved = tokio::time::timeout(COPY_WAIT, producer.consume().request_broadcast(COPY_PATH, None)) + .await + .expect("front did not resolve") + .expect("front refused"); + let track = resolved.track("0").unwrap(); + // A budget covering every group, so none is written off as late. + let subscription = track::Subscription::default().with_max_delay(Duration::from_secs(3600)); + let mut subscribers = Vec::with_capacity(readers); + for _ in 0..readers { + let subscriber = tokio::time::timeout(COPY_WAIT, track.subscribe(subscription.clone())) + .await + .expect("track did not splice") + .expect("subscribe failed"); + subscribers.push(subscriber); + } + let (second, second_track) = generation(&mut (0..retained).rev()); + second.announce(origin::Route::default().with_epoch(epoch)).unwrap(); + tokio::time::timeout(COPY_WAIT, second_track.demand().used()) + .await + .expect("route swap did not splice") + .expect("replacement track closed"); + ResumeOverlap { + _producer: producer, + _generations: [first, second], + _tracks: [first_track, second_track], + readers: subscribers, + } +} + +/// Readers draining a track whose two routes both delivered every group: each group +/// is handed out once, and the copy still owed it remembers that until it is let go. +/// The slope over retained groups is that bookkeeping's delivery cost. +fn bench_resume_overlap(c: &mut Criterion) { + let mut group = c.benchmark_group("origin/resume_overlap"); + group.sample_size(10); + for (readers, retained) in RESUME_OVERLAP { + group.throughput(Throughput::Elements((readers * retained) as u64)); + group.bench_function(BenchmarkId::from_parameter(format!("{readers}r_{retained}g")), |b| { + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let waiter = kio::Waiter::noop(); + b.iter_batched_ref( + || runtime.block_on(resume_overlap(readers, retained)), + |rig| { + for reader in &mut rig.readers { + let mut delivered = 0; + while let Poll::Ready(Ok(Some(_))) = reader.poll_recv_group(&waiter) { + delivered += 1; + } + assert_eq!(delivered, retained, "every group once"); + } + }, + BatchSize::PerIteration, + ); + }); + } + group.finish(); +} + criterion_group!( benches, bench_announce, @@ -1145,6 +1243,7 @@ criterion_group!( bench_narrow, bench_relay, bench_parked, - bench_copy_walk + bench_copy_walk, + bench_resume_overlap ); criterion_main!(benches); diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 5b760582c87..18922800306 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -13,6 +13,12 @@ //! its [`Recover`]: once its copy fails with its route, or stalls while a newer route //! serves, it continues from the serving route's copy of the same group at the same frame //! (and byte), found in that copy's cache or fetched from it. +//! +//! A group is handed out once however many copies deliver it. A copy never repeats a +//! group itself, so each copy remembers only the groups handed out from the others +//! within its own range: from where it was asked to start through its newest group at +//! replacement. That bounds the bookkeeping by how far the copies overlap, and it goes +//! away with the copy. use std::collections::BTreeSet; use std::sync::{Arc, Mutex, MutexGuard, Weak}; @@ -22,10 +28,6 @@ use crate::{Datagram, Error, Result, StreamError, group, stats, track}; use super::subscription::{Cap, Position, Subscription, max_some}; -/// How many delivered group sequences a reader remembers, so a group two routes both -/// deliver is handed out once. -const MAX_DELIVERED: usize = 1024; - /// What the front says about a logical track: the copy to read now, and how it ended. #[derive(Default)] struct Route { @@ -147,7 +149,6 @@ impl Consumer { mirrored, generation: None, copies: Vec::new(), - delivered: BTreeSet::new(), newest: None, ordered: None, datagram: None, @@ -248,6 +249,9 @@ struct Copy { /// Held by every group handed out from this copy, so a replaced copy stays /// subscribed while one is still read. lease: Arc<()>, + /// Groups within this copy's range that another copy handed out, so this one does + /// not hand them out again. + delivered: BTreeSet, } enum Sub { @@ -255,7 +259,31 @@ enum Sub { Ready(track::Subscriber), } +impl Sub { + /// The subscription, once the copy answered. `Err` once it refused. + fn poll_ready(&mut self, groups: (u64, Cap), waiter: &kio::Waiter) -> Poll> { + if let Self::Pending(pending) = self { + let mut sub = ready!(pending.poll_ok(waiter))?; + sub.raise_start_to(groups.0); + sub.end_at(groups.1); + *self = Self::Ready(sub); + } + let Self::Ready(sub) = self else { unreachable!() }; + Poll::Ready(Ok(sub)) + } +} + impl Copy { + /// Whether this copy may still deliver `sequence`: it has not run out, and the group + /// lies between where it was asked to start and where it was replaced. + fn covers(&self, sequence: u64) -> bool { + self.done.is_none() + && self.floor.is_none_or(|floor| sequence >= floor.group) + && self + .until + .is_none_or(|until| until.is_some_and(|until| sequence <= until)) + } + /// What this copy is asked for: the reader's own preferences, from no earlier than /// where it was asked to start, and through no later than where it was replaced. fn wire(&self, reader: &Subscription) -> Subscription { @@ -280,14 +308,7 @@ impl Copy { /// The subscription, once the copy answered. `Err` once it refused. fn poll_ready(&mut self, groups: (u64, Cap), waiter: &kio::Waiter) -> Poll> { - if let Sub::Pending(pending) = &self.sub { - let mut sub = ready!(pending.poll_ok(waiter))?; - sub.raise_start_to(groups.0); - sub.end_at(groups.1); - self.sub = Sub::Ready(sub); - } - let Sub::Ready(sub) = &mut self.sub else { unreachable!() }; - Poll::Ready(Ok(sub)) + self.sub.poll_ready(groups, waiter) } /// Whether this copy is still needed: it serves, or a group read from it is still out. @@ -404,8 +425,6 @@ struct Reader { generation: Option, /// Oldest first: the last one is the serving route's, when it is subscribed. copies: Vec, - /// The groups handed out, so a group two copies deliver is handed out once. - delivered: BTreeSet, /// The newest group handed out: where a new copy is asked to start. newest: Option, /// Reading in sequence order: the last group handed out. @@ -472,6 +491,8 @@ impl Reader { datagrams_done: None, datagrams_seen: self.datagram, lease: Arc::new(()), + // Every group handed out so far is at or below its floor, the newest of them. + delivered: self.newest.into_iter().collect(), }); } @@ -480,7 +501,6 @@ impl Reader { let serving = self.generation; let groups = self.groups; let ordered = self.ordered; - let delivered = &self.delivered; self.copies.retain_mut(|copy| { if copy.needed(Some(copy.generation) == serving) { return true; @@ -489,7 +509,7 @@ impl Reader { return false; } let until = copy.until; - let Poll::Ready(Ok(sub)) = copy.poll_ready(groups, &kio::Waiter::noop()) else { + let Poll::Ready(Ok(sub)) = copy.sub.poll_ready(groups, &kio::Waiter::noop()) else { return false; }; // Inspect without moving either cursor: a later group poll still chooses its @@ -498,7 +518,7 @@ impl Reader { sub.has_unread_group(&track::Unread { start: groups.0.max(ordered.map_or(0, |last| last.saturating_add(1))), end: until.and_then(|until| until.map_or(Some(0), |last| last.checked_add(1))), - delivered, + delivered: ©.delivered, }) }); } @@ -512,7 +532,7 @@ impl Reader { } match self.ordered { Some(last) => sequence > last, - None => !self.delivered.contains(&sequence), + None => !copy.delivered.contains(&sequence), } } @@ -521,9 +541,10 @@ impl Reader { if ordered { self.ordered = Some(sequence); } - self.delivered.insert(sequence); - if self.delivered.len() > MAX_DELIVERED { - self.delivered.pop_first(); + for (other, copy) in self.copies.iter_mut().enumerate() { + if other != index && copy.covers(sequence) { + copy.delivered.insert(sequence); + } } self.newest = self.newest.max(Some(sequence)); let copy = &self.copies[index]; @@ -1253,6 +1274,177 @@ mod test { assert!(sub.recv_group().now_or_never().is_none()); } + /// A budget that never writes off a group in these tests. + const HOUR: Duration = Duration::from_secs(3600); + + /// A reader on the routes themselves, so a test can inspect its copies. + fn reader(routes: &Producer, max_delay: Duration) -> Subscriber { + routes + .consume() + .subscribe(kio::Producer::new(Subscription::default().with_max_delay(max_delay))) + } + + /// Write a finished one-frame group, stamped with its sequence. + fn write(track: &track::Producer, sequence: u64) { + let mut group = track.create_group(group::Info { sequence }).unwrap(); + group.write_frame(ts(sequence), b"x".as_ref()).unwrap(); + group.finish().unwrap(); + } + + /// Every group the reader hands out right now. + fn drain(sub: &mut Subscriber, ordered: bool) -> Vec { + let mut sequences = Vec::new(); + while let Poll::Ready(Ok(Some(group))) = sub.poll_group(ordered, &kio::Waiter::noop()) { + sequences.push(group.sequence); + } + sequences + } + + /// How many groups handed out elsewhere each of the reader's copies remembers. + fn remembered(sub: &Subscriber) -> Vec { + sub.reader().copies.iter().map(|copy| copy.delivered.len()).collect() + } + + /// A copy delivering a group long after the reader moved past it does not hand it out + /// again, however many groups came between; one never handed out is still delivered. + #[test] + fn a_late_group_is_handed_out_once_however_late() { + let routes = Producer::new(); + let a = copy(); + routes.serve(a.consume()); + let mut sub = reader(&routes, HOUR); + write(&a, 0); + assert_eq!(drain(&mut sub, false), [0]); + + let b = copy(); + routes.serve(b.consume()); + for sequence in 2..=2000 { + write(&b, sequence); + } + assert_eq!(drain(&mut sub, false), (2..=2000).collect::>()); + + // The replacement's copies of groups 1 and 0 land last: 1 is new, 0 is not. + write(&b, 1); + write(&b, 0); + assert_eq!(drain(&mut sub, false), [1]); + assert_eq!(remembered(&sub), [1], "only the group the replaced copy handed out"); + } + + /// An ordered reader never goes back, late duplicate or not. + #[test] + fn an_ordered_reader_skips_a_late_group() { + let routes = Producer::new(); + let a = copy(); + routes.serve(a.consume()); + let mut sub = reader(&routes, HOUR); + write(&a, 0); + assert_eq!(drain(&mut sub, true), [0]); + + let b = copy(); + routes.serve(b.consume()); + for sequence in 1..=2000 { + write(&b, sequence); + } + assert_eq!(drain(&mut sub, true), (1..=2000).collect::>()); + write(&b, 0); + assert_eq!(drain(&mut sub, true), Vec::::new()); + } + + /// Two routes taking turns, each also delivering the other's groups late and out of + /// order, hand out every group once. What a reader remembers stays bounded by the + /// overlap, not by how many groups or flaps went by. + #[test] + fn route_flaps_hand_out_every_group_once() { + let routes = Producer::new(); + let tracks = [copy(), copy()]; + routes.serve(tracks[0].consume()); + let mut sub = reader(&routes, HOUR); + let mut seen = Vec::new(); + let mut memory = Vec::new(); + for round in 0..16u64 { + let (serving, other) = (&tracks[round as usize % 2], &tracks[1 - round as usize % 2]); + let groups = round * 10..round * 10 + 10; + // The serving route brings half the round, the reader takes it, then the rest. + for sequence in groups.clone().take(5) { + write(serving, sequence); + } + seen.extend(drain(&mut sub, false)); + for sequence in groups.clone().skip(5) { + write(serving, sequence); + } + // The other route takes over and the replaced copy hands out the rest; the + // other's own copy of the round lands only afterwards, newest first. + routes.serve(other.consume()); + seen.extend(drain(&mut sub, false)); + for sequence in groups.clone().rev() { + write(other, sequence); + } + seen.extend(drain(&mut sub, false)); + memory.push(remembered(&sub)); + } + let mut sorted = seen.clone(); + sorted.sort_unstable(); + assert_eq!(sorted, (0..160).collect::>(), "handed out: {seen:?}"); + // Each round, only the serving copy is left, remembering its floor and the half + // round the replaced copy handed out. + assert!( + memory.iter().all(|copies| copies == &[6]), + "bookkeeping grew with the flaps: {memory:?}" + ); + } + + /// A reader holding an old group open keeps its replaced copy, and moving its group + /// bounds neither repeats a group nor loses one. Once that copy finishes, what it + /// remembered goes with it. + #[test] + fn held_groups_and_moved_bounds_hand_out_each_group_once() { + let routes = Producer::new(); + let a = copy(); + routes.serve(a.consume()); + let mut sub = reader(&routes, HOUR); + sub.end_at(Cap::from(..=0)); + let mut held = a.create_group(group::Info { sequence: 0 }).unwrap(); + held.write_frame(ts(0), b"a".as_ref()).unwrap(); + let Poll::Ready(Ok(Some(reading))) = sub.poll_group(false, &kio::Waiter::noop()) else { + panic!("group 0 is handed out") + }; + // Past the cap: parked on the copy, not handed out. + write(&a, 1); + write(&a, 2); + assert_eq!(drain(&mut sub, false), Vec::::new()); + + let b = copy(); + routes.serve(b.consume()); + for sequence in [2, 0, 3, 1] { + write(&b, sequence); + } + assert_eq!(drain(&mut sub, false), Vec::::new()); + assert_eq!(sub.reader().copies.len(), 2, "the held group keeps its copy"); + + sub.end_at(Cap::from(..)); + let mut seen = drain(&mut sub, false); + seen.sort_unstable(); + assert_eq!(seen, [1, 2, 3]); + sub.start_at(0); + assert_eq!(drain(&mut sub, false), Vec::::new()); + // The replaced copy remembers the groups the serving one handed out through its + // newest at replacement; the serving one, the group handed out before it. + assert_eq!(remembered(&sub), [2, 1]); + + // The replaced copy finishes; the held group is all it has left. + held.finish().unwrap(); + a.finish().unwrap(); + drop(reading); + assert_eq!(drain(&mut sub, false), Vec::::new()); + assert_eq!( + remembered(&sub), + [1], + "the finished copy is let go with what it remembered" + ); + write(&b, 4); + assert_eq!(drain(&mut sub, false), [4]); + } + /// A replacement copy asked to start at the newest group handed out keeps that /// floor when the reader's floor widens: an older group its route cached was /// never owed, and handing it out now would be out of order. From b0639740a14e9dd928fb9127ce123483c21286a2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:27:01 -0700 Subject: [PATCH 03/21] bench(net): sweep resume_overlap over the routes a reader still holds Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/origin.rs | 121 +++++++++++++++++++++-------------- 1 file changed, 74 insertions(+), 47 deletions(-) diff --git a/rs/moq-net/benches/origin.rs b/rs/moq-net/benches/origin.rs index 5956a624a7b..9d84e018f4e 100644 --- a/rs/moq-net/benches/origin.rs +++ b/rs/moq-net/benches/origin.rs @@ -19,8 +19,8 @@ //! //! A route swap on one front is swept over its tracks and the copies each track //! still holds from earlier routes, and a front retiring over the fronts around it. -//! Readers draining a swapped track are swept over their count and the groups -//! both routes delivered. +//! Readers draining a swapped track are swept over their count, the groups every +//! route delivered, and the routes each reader still holds a copy of. //! //! Run with `cargo bench -p moq-net --bench origin`. @@ -1127,20 +1127,30 @@ fn bench_copy_walk(c: &mut Criterion) { group.finish(); } -/// `(readers, retained)` for [`bench_resume_overlap`]: readers of one track, and -/// the groups both routes hold when it swaps. -const RESUME_OVERLAP: [(usize, usize); 6] = [(1, 1), (1, 64), (1, 1024), (32, 64), (256, 64), (32, 1024)]; - -/// A track that swapped routes before its readers read anything: the replaced copy -/// and the new one hold the same groups, the new one in reverse arrival order. +/// `(readers, retained, routes)` for [`bench_resume_overlap`]: readers of one track, +/// the groups every route holds, and the routes it swapped through. +const RESUME_OVERLAP: [(usize, usize, usize); 9] = [ + (1, 1, 2), + (1, 64, 2), + (1, 1024, 2), + (32, 64, 2), + (256, 64, 2), + (32, 1024, 2), + (1, 64, 8), + (1, 64, 32), + (32, 64, 8), +]; + +/// A track that swapped through its routes while its readers' local cap held every +/// group back, so each reader keeps a copy per route and every copy holds the same +/// groups, each replacement in reverse arrival order. struct ResumeOverlap { _producer: origin::Producer, - _generations: [broadcast::Producer; 2], - _tracks: [track::Producer; 2], + _generations: Vec<(broadcast::Producer, track::Producer)>, readers: Vec, } -async fn resume_overlap(readers: usize, retained: usize) -> ResumeOverlap { +async fn resume_overlap(readers: usize, retained: usize, routes: usize) -> ResumeOverlap { let (producer, driver) = origin::Producer::new(origin::Config::default()); tokio::spawn(moq_net::time::run(driver)); let epoch = Epoch::mint(); @@ -1156,12 +1166,12 @@ async fn resume_overlap(readers: usize, retained: usize) -> ResumeOverlap { group.write_frame(Timestamp::ZERO, b"f".as_ref()).unwrap(); group.finish().unwrap(); } + broadcast + .announce(origin::Route::default().with_epoch(epoch.clone())) + .unwrap(); (broadcast, track) }; - let (first, first_track) = generation(&mut (0..retained)); - first - .announce(origin::Route::default().with_epoch(epoch.clone())) - .unwrap(); + let mut generations = vec![generation(&mut (0..retained))]; let resolved = tokio::time::timeout(COPY_WAIT, producer.consume().request_broadcast(COPY_PATH, None)) .await .expect("front did not resolve") @@ -1171,54 +1181,71 @@ async fn resume_overlap(readers: usize, retained: usize) -> ResumeOverlap { let subscription = track::Subscription::default().with_max_delay(Duration::from_secs(3600)); let mut subscribers = Vec::with_capacity(readers); for _ in 0..readers { - let subscriber = tokio::time::timeout(COPY_WAIT, track.subscribe(subscription.clone())) + let mut subscriber = tokio::time::timeout(COPY_WAIT, track.subscribe(subscription.clone())) .await .expect("track did not splice") .expect("subscribe failed"); + // Held back until the drain, so every route's copy is kept for its unread groups. + subscriber.set_groups(..0); subscribers.push(subscriber); } - let (second, second_track) = generation(&mut (0..retained).rev()); - second.announce(origin::Route::default().with_epoch(epoch)).unwrap(); - tokio::time::timeout(COPY_WAIT, second_track.demand().used()) - .await - .expect("route swap did not splice") - .expect("replacement track closed"); + let waiter = kio::Waiter::noop(); + for _ in 1..routes { + generations.push(generation(&mut (0..retained).rev())); + let (_, latest) = generations.last().unwrap(); + tokio::time::timeout(COPY_WAIT, latest.demand().used()) + .await + .expect("route swap did not splice") + .expect("replacement track closed"); + // Each reader follows the swap, subscribing to the new copy. + for subscriber in &mut subscribers { + assert!( + subscriber.poll_recv_group(&waiter).is_pending(), + "the cap holds every group" + ); + } + } + for subscriber in &mut subscribers { + subscriber.set_groups(..); + } ResumeOverlap { _producer: producer, - _generations: [first, second], - _tracks: [first_track, second_track], + _generations: generations, readers: subscribers, } } -/// Readers draining a track whose two routes both delivered every group: each group -/// is handed out once, and the copy still owed it remembers that until it is let go. -/// The slope over retained groups is that bookkeeping's delivery cost. +/// Readers draining a track whose routes all delivered every group: each group is +/// handed out once, and every other copy still owed it remembers that until it is let +/// go. The slopes over retained groups and routes are that bookkeeping's delivery cost. fn bench_resume_overlap(c: &mut Criterion) { let mut group = c.benchmark_group("origin/resume_overlap"); group.sample_size(10); - for (readers, retained) in RESUME_OVERLAP { + for (readers, retained, routes) in RESUME_OVERLAP { group.throughput(Throughput::Elements((readers * retained) as u64)); - group.bench_function(BenchmarkId::from_parameter(format!("{readers}r_{retained}g")), |b| { - let runtime = tokio::runtime::Builder::new_current_thread() - .enable_all() - .build() - .unwrap(); - let waiter = kio::Waiter::noop(); - b.iter_batched_ref( - || runtime.block_on(resume_overlap(readers, retained)), - |rig| { - for reader in &mut rig.readers { - let mut delivered = 0; - while let Poll::Ready(Ok(Some(_))) = reader.poll_recv_group(&waiter) { - delivered += 1; + group.bench_function( + BenchmarkId::from_parameter(format!("{readers}r_{retained}g_{routes}c")), + |b| { + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let waiter = kio::Waiter::noop(); + b.iter_batched_ref( + || runtime.block_on(resume_overlap(readers, retained, routes)), + |rig| { + for reader in &mut rig.readers { + let mut delivered = 0; + while let Poll::Ready(Ok(Some(_))) = reader.poll_recv_group(&waiter) { + delivered += 1; + } + assert_eq!(delivered, retained, "every group once"); } - assert_eq!(delivered, retained, "every group once"); - } - }, - BatchSize::PerIteration, - ); - }); + }, + BatchSize::PerIteration, + ); + }, + ); } group.finish(); } From 9a29f7fe397c703c73ab01aba80e75039816e7d7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 13:46:34 -0700 Subject: [PATCH 04/21] fix(net): hand each group sequence out once per reader Move duplicate suppression from each route copy onto track::Subscriber, shared by direct and resumed readers: a run-length set of delivered sequences. A group aborted and re-created, or delivered by a second route, is handed out once. Once a group is past the reader's budget (max delay clamped by max age), every group below it counts as delivered, so only gaps within the budget cost memory. Co-Authored-By: Claude Opus 5.5 --- quest/m1/js-group-handover.md | 14 +- rs/moq-net/benches/origin.rs | 4 +- rs/moq-net/src/model/delivered.rs | 133 +++++++++++++++ rs/moq-net/src/model/mod.rs | 1 + rs/moq-net/src/model/resume.rs | 273 ++++++++++++++++-------------- rs/moq-net/src/model/track.rs | 180 ++++++++++++++++++-- 6 files changed, 463 insertions(+), 142 deletions(-) create mode 100644 rs/moq-net/src/model/delivered.rs diff --git a/quest/m1/js-group-handover.md b/quest/m1/js-group-handover.md index 04aa714982d..592b749471d 100644 --- a/quest/m1/js-group-handover.md +++ b/quest/m1/js-group-handover.md @@ -28,10 +28,16 @@ them at replacement, and recovers a group from the new copy at the first missing frame and byte. It does not copy through the earlier single-writer pump or immediately cancel every old copy. Decided in the 2026-10-10 audit: mirror the current identity, reader, recovery and lifetime contract, including -the late-duplicate correction, rather than porting the obsolete pump. Each copy -remembers the groups other copies handed out within its own range (its resume -floor through its newest group at replacement) and drops that with the copy; -no fixed-size history of delivered groups. +the late-duplicate correction, rather than porting the obsolete pump. +Decided 2026-10-13: every track reader, direct or resumed, hands out each group +sequence at most once, including a group aborted and re-created and one a +second route delivers, so no consumer ever dedupes itself. The record is a +per-reader interval set of delivered sequences on the track subscriber, shared +by direct and resumed reads, with no fixed-size history. Once a group is past +the reader's budget (max delay, clamped by max age), the reader stops expecting +every group below it and counts them as delivered, so only gaps within the +budget cost memory. A genuinely unseen late group within the budget is still +delivered. Things to settle along the way: - Whether the request's `active` broadcast stays the same object across a swap, diff --git a/rs/moq-net/benches/origin.rs b/rs/moq-net/benches/origin.rs index 9d84e018f4e..e29c7873942 100644 --- a/rs/moq-net/benches/origin.rs +++ b/rs/moq-net/benches/origin.rs @@ -1216,8 +1216,8 @@ async fn resume_overlap(readers: usize, retained: usize, routes: usize) -> Resum } /// Readers draining a track whose routes all delivered every group: each group is -/// handed out once, and every other copy still owed it remembers that until it is let -/// go. The slopes over retained groups and routes are that bookkeeping's delivery cost. +/// handed out once, and every other copy's duplicate is skipped against what the reader +/// handed out. The slopes over retained groups and routes are that skipping's cost. fn bench_resume_overlap(c: &mut Criterion) { let mut group = c.benchmark_group("origin/resume_overlap"); group.sample_size(10); diff --git a/rs/moq-net/src/model/delivered.rs b/rs/moq-net/src/model/delivered.rs new file mode 100644 index 00000000000..f11a38242e2 --- /dev/null +++ b/rs/moq-net/src/model/delivered.rs @@ -0,0 +1,133 @@ +//! The group sequences a reader handed out, so it never hands one out twice. + +use std::collections::BTreeMap; + +/// The group sequences a reader handed out, as runs of consecutive sequences. +/// +/// A reader stops expecting a group once it is past its budget, and counts everything +/// below it as handed out (see [`Self::forget_below`]). The run starting at 0 is that +/// floor, so only the gaps above it cost memory. +#[derive(Default)] +pub(crate) struct Delivered { + /// First to last sequence, inclusive, never adjacent to each other. + runs: BTreeMap, +} + +impl Delivered { + /// Whether `sequence` was handed out, or is below the floor. + pub fn contains(&self, sequence: u64) -> bool { + self.runs + .range(..=sequence) + .next_back() + .is_some_and(|(_, last)| sequence <= *last) + } + + /// Record `sequence` as handed out; `false` when it already was. + pub fn insert(&mut self, sequence: u64) -> bool { + if self.contains(sequence) { + return false; + } + let first = sequence + .checked_sub(1) + .and_then(|below| self.runs.range(..=below).next_back()) + .filter(|(_, last)| **last + 1 == sequence) + .map_or(sequence, |(first, _)| *first); + let last = sequence + .checked_add(1) + .and_then(|above| self.runs.remove(&above)) + .unwrap_or(sequence); + self.runs.insert(first, last); + true + } + + /// Count every sequence below `floor` as handed out. + pub fn forget_below(&mut self, floor: u64) { + let Some(below) = floor.checked_sub(1) else { + return; + }; + // Every run starting at or below the floor joins it, including one starting right at it. + let above = match floor.checked_add(1) { + Some(above) => self.runs.split_off(&above), + None => BTreeMap::new(), + }; + let last = self.runs.last_key_value().map_or(below, |(_, last)| below.max(*last)); + self.runs = above; + self.runs.insert(0, last); + } + + /// The lowest sequence handed out above a gap: once it is past the reader's budget, + /// so is the gap. + pub fn above_gap(&self) -> Option { + let mut runs = self.runs.keys(); + match runs.next()? { + 0 => runs.next().copied(), + first => Some(*first), + } + } + + /// How many runs it holds: one per gap, plus the floor. + #[cfg(test)] + pub fn runs(&self) -> usize { + self.runs.len() + } +} + +#[cfg(test)] +mod test { + use super::*; + + fn runs(delivered: &Delivered) -> Vec<(u64, u64)> { + delivered.runs.iter().map(|(first, last)| (*first, *last)).collect() + } + + #[test] + fn insert_joins_neighbouring_runs() { + let mut delivered = Delivered::default(); + assert!(delivered.insert(5)); + assert!(delivered.insert(7)); + assert_eq!(runs(&delivered), [(5, 5), (7, 7)]); + assert!(delivered.insert(6)); + assert_eq!(runs(&delivered), [(5, 7)]); + assert!(!delivered.insert(6)); + assert!(delivered.insert(4)); + assert!(delivered.insert(8)); + assert_eq!(runs(&delivered), [(4, 8)]); + assert!(delivered.contains(4) && delivered.contains(8)); + assert!(!delivered.contains(3) && !delivered.contains(9)); + } + + #[test] + fn the_extremes_are_sequences_too() { + let mut delivered = Delivered::default(); + assert!(delivered.insert(0)); + assert!(delivered.insert(u64::MAX)); + assert!(!delivered.insert(u64::MAX)); + assert_eq!(runs(&delivered), [(0, 0), (u64::MAX, u64::MAX)]); + delivered.forget_below(u64::MAX); + assert_eq!(runs(&delivered), [(0, u64::MAX)]); + } + + #[test] + fn forget_below_closes_the_gaps_under_the_floor() { + let mut delivered = Delivered::default(); + for sequence in [3, 5, 6, 9, 12] { + delivered.insert(sequence); + } + assert_eq!(delivered.above_gap(), Some(3)); + delivered.forget_below(0); + assert_eq!(delivered.runs(), 4); + + // A floor at a run's start takes the run along. + delivered.forget_below(5); + assert_eq!(runs(&delivered), [(0, 6), (9, 9), (12, 12)]); + assert_eq!(delivered.above_gap(), Some(9)); + assert!(delivered.contains(4), "below the floor counts as handed out"); + + // A floor inside a gap keeps the run above it. + delivered.forget_below(11); + assert_eq!(runs(&delivered), [(0, 10), (12, 12)]); + delivered.forget_below(13); + assert_eq!(runs(&delivered), [(0, 12)]); + assert_eq!(delivered.above_gap(), None); + } +} diff --git a/rs/moq-net/src/model/mod.rs b/rs/moq-net/src/model/mod.rs index 1c28f154c4e..a562b751c5a 100644 --- a/rs/moq-net/src/model/mod.rs +++ b/rs/moq-net/src/model/mod.rs @@ -19,6 +19,7 @@ mod front; mod bytes; pub(crate) mod clock; mod datagram; +mod delivered; mod expiry; mod requests; mod resume; diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 18922800306..8c9924d55bc 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -14,18 +14,16 @@ //! serves, it continues from the serving route's copy of the same group at the same frame //! (and byte), found in that copy's cache or fetched from it. //! -//! A group is handed out once however many copies deliver it. A copy never repeats a -//! group itself, so each copy remembers only the groups handed out from the others -//! within its own range: from where it was asked to start through its newest group at -//! replacement. That bounds the bookkeeping by how far the copies overlap, and it goes -//! away with the copy. +//! A group is handed out once however many copies deliver it: the reader skips any +//! sequence its [`track::Subscriber`] already handed out, the same record that keeps a +//! direct reader from taking a re-created group twice. -use std::collections::BTreeSet; use std::sync::{Arc, Mutex, MutexGuard, Weak}; use std::task::{Poll, ready}; use crate::{Datagram, Error, Result, StreamError, group, stats, track}; +use super::delivered::Delivered; use super::subscription::{Cap, Position, Subscription, max_some}; /// What the front says about a logical track: the copy to read now, and how it ended. @@ -150,7 +148,6 @@ impl Consumer { generation: None, copies: Vec::new(), newest: None, - ordered: None, datagram: None, groups: (0, Cap::from(..)), }) @@ -249,9 +246,6 @@ struct Copy { /// Held by every group handed out from this copy, so a replaced copy stays /// subscribed while one is still read. lease: Arc<()>, - /// Groups within this copy's range that another copy handed out, so this one does - /// not hand them out again. - delivered: BTreeSet, } enum Sub { @@ -259,31 +253,7 @@ enum Sub { Ready(track::Subscriber), } -impl Sub { - /// The subscription, once the copy answered. `Err` once it refused. - fn poll_ready(&mut self, groups: (u64, Cap), waiter: &kio::Waiter) -> Poll> { - if let Self::Pending(pending) = self { - let mut sub = ready!(pending.poll_ok(waiter))?; - sub.raise_start_to(groups.0); - sub.end_at(groups.1); - *self = Self::Ready(sub); - } - let Self::Ready(sub) = self else { unreachable!() }; - Poll::Ready(Ok(sub)) - } -} - impl Copy { - /// Whether this copy may still deliver `sequence`: it has not run out, and the group - /// lies between where it was asked to start and where it was replaced. - fn covers(&self, sequence: u64) -> bool { - self.done.is_none() - && self.floor.is_none_or(|floor| sequence >= floor.group) - && self - .until - .is_none_or(|until| until.is_some_and(|until| sequence <= until)) - } - /// What this copy is asked for: the reader's own preferences, from no earlier than /// where it was asked to start, and through no later than where it was replaced. fn wire(&self, reader: &Subscription) -> Subscription { @@ -308,7 +278,14 @@ impl Copy { /// The subscription, once the copy answered. `Err` once it refused. fn poll_ready(&mut self, groups: (u64, Cap), waiter: &kio::Waiter) -> Poll> { - self.sub.poll_ready(groups, waiter) + if let Sub::Pending(pending) = &self.sub { + let mut sub = ready!(pending.poll_ok(waiter))?; + sub.raise_start_to(groups.0); + sub.end_at(groups.1); + self.sub = Sub::Ready(sub); + } + let Sub::Ready(sub) = &mut self.sub else { unreachable!() }; + Poll::Ready(Ok(sub)) } /// Whether this copy is still needed: it serves, or a group read from it is still out. @@ -335,14 +312,36 @@ impl Subscriber { &self.subscription } - /// Poll for the next group: in arrival order, or in sequence order for `ordered`. - pub(crate) fn poll_group(&mut self, ordered: bool, waiter: &kio::Waiter) -> Poll>> { - self.reader().poll_group(ordered, waiter) + /// Poll for the next group not in `delivered`: in arrival order, or in sequence order + /// for `ordered`. + pub(crate) fn poll_group( + &mut self, + ordered: bool, + delivered: &Delivered, + waiter: &kio::Waiter, + ) -> Poll>> { + self.reader().poll_group(ordered, delivered, waiter) } - /// Poll for the next datagram in arrival order. - pub(crate) fn poll_recv_datagram(&mut self, waiter: &kio::Waiter) -> Poll>> { - self.reader().poll_recv_datagram(waiter) + /// Poll for the next datagram in arrival order. `delivered` decides which replaced + /// copies still have a group to hand out. + pub(crate) fn poll_recv_datagram( + &mut self, + delivered: &Delivered, + waiter: &kio::Waiter, + ) -> Poll>> { + self.reader().poll_recv_datagram(delivered, waiter) + } + + /// Whether a group at `sequence` is past the reader's budget, judged on the newest + /// copy that answered; see [`track::Subscriber::is_past_budget`]. + pub(crate) fn is_past_budget(&self, sequence: u64) -> bool { + let reader = self.reader(); + let mut ready = reader.copies.iter().rev().filter_map(|copy| match ©.sub { + Sub::Ready(sub) => Some(sub), + Sub::Pending(_) => None, + }); + ready.next().is_some_and(|sub| sub.is_past_budget(sequence)) } /// Poll for the track's declared exclusive end. @@ -427,8 +426,6 @@ struct Reader { copies: Vec, /// The newest group handed out: where a new copy is asked to start. newest: Option, - /// Reading in sequence order: the last group handed out. - ordered: Option, /// The newest datagram handed out. datagram: Option, /// The local group filter; see [`track::Subscriber::set_groups`]. @@ -491,16 +488,13 @@ impl Reader { datagrams_done: None, datagrams_seen: self.datagram, lease: Arc::new(()), - // Every group handed out so far is at or below its floor, the newest of them. - delivered: self.newest.into_iter().collect(), }); } /// Drop replaced copies only once neither a handed-out nor an unread group needs them. - fn let_go(&mut self) { + fn let_go(&mut self, delivered: &Delivered) { let serving = self.generation; let groups = self.groups; - let ordered = self.ordered; self.copies.retain_mut(|copy| { if copy.needed(Some(copy.generation) == serving) { return true; @@ -509,43 +503,29 @@ impl Reader { return false; } let until = copy.until; - let Poll::Ready(Ok(sub)) = copy.sub.poll_ready(groups, &kio::Waiter::noop()) else { + let Poll::Ready(Ok(sub)) = copy.poll_ready(groups, &kio::Waiter::noop()) else { return false; }; // Inspect without moving either cursor: a later group poll still chooses its // order and applies the current bounds and drift budget. Ignore the local // cap here, since unread groups become eligible again when it rises. sub.has_unread_group(&track::Unread { - start: groups.0.max(ordered.map_or(0, |last| last.saturating_add(1))), + start: groups.0, end: until.and_then(|until| until.map_or(Some(0), |last| last.checked_add(1))), - delivered: ©.delivered, + delivered, }) }); } - fn deliverable(&self, copy: &Copy, sequence: u64) -> bool { - if copy + fn deliverable(copy: &Copy, sequence: u64, delivered: &Delivered) -> bool { + let replaced = copy .until - .is_some_and(|until| until.is_none_or(|until| sequence > until)) - { - return false; - } - match self.ordered { - Some(last) => sequence > last, - None => !copy.delivered.contains(&sequence), - } + .is_some_and(|until| until.is_none_or(|until| sequence > until)); + !replaced && !delivered.contains(sequence) } - fn hand_out(&mut self, index: usize, group: group::Consumer, ordered: bool) -> group::Consumer { + fn hand_out(&mut self, index: usize, group: group::Consumer) -> group::Consumer { let sequence = group.sequence; - if ordered { - self.ordered = Some(sequence); - } - for (other, copy) in self.copies.iter_mut().enumerate() { - if other != index && copy.covers(sequence) { - copy.delivered.insert(sequence); - } - } self.newest = self.newest.max(Some(sequence)); let copy = &self.copies[index]; group.with_recover(Recover { @@ -559,9 +539,14 @@ impl Reader { }) } - fn poll_group(&mut self, ordered: bool, waiter: &kio::Waiter) -> Poll>> { + fn poll_group( + &mut self, + ordered: bool, + delivered: &Delivered, + waiter: &kio::Waiter, + ) -> Poll>> { loop { - if let Some(res) = self.poll_group_once(ordered, waiter) { + if let Some(res) = self.poll_group_once(ordered, delivered, waiter) { return res; } } @@ -572,6 +557,7 @@ impl Reader { fn poll_group_once( &mut self, ordered: bool, + delivered: &Delivered, waiter: &kio::Waiter, ) -> Option>>> { self.sync(waiter); @@ -599,11 +585,10 @@ impl Reader { match res { Poll::Pending => break, Poll::Ready(Ok(Some(group))) => { - let copy = &self.copies[index]; - if !self.deliverable(copy, group.sequence) { + if !Self::deliverable(&self.copies[index], group.sequence, delivered) { continue; } - let group = self.hand_out(index, group, ordered); + let group = self.hand_out(index, group); return Some(Poll::Ready(Ok(Some(group)))); } // Drained: the track ended as the copy did, which is not cleanly when @@ -622,7 +607,7 @@ impl Reader { } } } - self.let_go(); + self.let_go(delivered); self.poll_end(waiter, |copy| copy.done.clone()) } @@ -665,17 +650,21 @@ impl Reader { } } - fn poll_recv_datagram(&mut self, waiter: &kio::Waiter) -> Poll>> { + fn poll_recv_datagram(&mut self, delivered: &Delivered, waiter: &kio::Waiter) -> Poll>> { loop { - if let Some(res) = self.poll_datagram_once(waiter) { + if let Some(res) = self.poll_datagram_once(delivered, waiter) { return res; } } } - fn poll_datagram_once(&mut self, waiter: &kio::Waiter) -> Option>>> { + fn poll_datagram_once( + &mut self, + delivered: &Delivered, + waiter: &kio::Waiter, + ) -> Option>>> { self.sync(waiter); - self.let_go(); + self.let_go(delivered); let groups = self.groups; for index in (0..self.copies.len()).rev() { loop { @@ -1277,14 +1266,7 @@ mod test { /// A budget that never writes off a group in these tests. const HOUR: Duration = Duration::from_secs(3600); - /// A reader on the routes themselves, so a test can inspect its copies. - fn reader(routes: &Producer, max_delay: Duration) -> Subscriber { - routes - .consume() - .subscribe(kio::Producer::new(Subscription::default().with_max_delay(max_delay))) - } - - /// Write a finished one-frame group, stamped with its sequence. + /// Write a finished one-frame group, presented at its sequence in milliseconds. fn write(track: &track::Producer, sequence: u64) { let mut group = track.create_group(group::Info { sequence }).unwrap(); group.write_frame(ts(sequence), b"x".as_ref()).unwrap(); @@ -1292,17 +1274,18 @@ mod test { } /// Every group the reader hands out right now. - fn drain(sub: &mut Subscriber, ordered: bool) -> Vec { + fn drain(sub: &mut track::Subscriber, ordered: bool) -> Vec { let mut sequences = Vec::new(); - while let Poll::Ready(Ok(Some(group))) = sub.poll_group(ordered, &kio::Waiter::noop()) { + loop { + let res = match ordered { + true => sub.poll_next_group(&kio::Waiter::noop()), + false => sub.poll_recv_group(&kio::Waiter::noop()), + }; + let Poll::Ready(Ok(Some(group))) = res else { + return sequences; + }; sequences.push(group.sequence); } - sequences - } - - /// How many groups handed out elsewhere each of the reader's copies remembers. - fn remembered(sub: &Subscriber) -> Vec { - sub.reader().copies.iter().map(|copy| copy.delivered.len()).collect() } /// A copy delivering a group long after the reader moved past it does not hand it out @@ -1310,9 +1293,10 @@ mod test { #[test] fn a_late_group_is_handed_out_once_however_late() { let routes = Producer::new(); + let logical = logical(&routes); let a = copy(); routes.serve(a.consume()); - let mut sub = reader(&routes, HOUR); + let mut sub = subscribe(&logical, HOUR); write(&a, 0); assert_eq!(drain(&mut sub, false), [0]); @@ -1322,21 +1306,23 @@ mod test { write(&b, sequence); } assert_eq!(drain(&mut sub, false), (2..=2000).collect::>()); + assert_eq!(sub.delivered_runs(), 2, "the gap at 1 is within the budget"); // The replacement's copies of groups 1 and 0 land last: 1 is new, 0 is not. write(&b, 1); write(&b, 0); assert_eq!(drain(&mut sub, false), [1]); - assert_eq!(remembered(&sub), [1], "only the group the replaced copy handed out"); + assert_eq!(sub.delivered_runs(), 1); } /// An ordered reader never goes back, late duplicate or not. #[test] fn an_ordered_reader_skips_a_late_group() { let routes = Producer::new(); + let logical = logical(&routes); let a = copy(); routes.serve(a.consume()); - let mut sub = reader(&routes, HOUR); + let mut sub = subscribe(&logical, HOUR); write(&a, 0); assert_eq!(drain(&mut sub, true), [0]); @@ -1350,17 +1336,36 @@ mod test { assert_eq!(drain(&mut sub, true), Vec::::new()); } + /// A group the serving copy aborts and re-creates is handed out once, as it is to a + /// direct reader of that copy. + #[test] + fn a_recreated_group_is_handed_out_once() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let mut sub = subscribe(&logical, HOUR); + write(&a, 0); + let open = a.create_group(group::Info { sequence: 1 }).unwrap(); + assert_eq!(drain(&mut sub, false), [0, 1]); + + open.abort(Error::Cancel).unwrap(); + write(&a, 1); + write(&a, 2); + assert_eq!(drain(&mut sub, false), [2]); + } + /// Two routes taking turns, each also delivering the other's groups late and out of - /// order, hand out every group once. What a reader remembers stays bounded by the - /// overlap, not by how many groups or flaps went by. + /// order, hand out every group once. What the reader remembers does not grow with the + /// groups or the flaps. #[test] fn route_flaps_hand_out_every_group_once() { let routes = Producer::new(); + let logical = logical(&routes); let tracks = [copy(), copy()]; routes.serve(tracks[0].consume()); - let mut sub = reader(&routes, HOUR); + let mut sub = subscribe(&logical, HOUR); let mut seen = Vec::new(); - let mut memory = Vec::new(); for round in 0..16u64 { let (serving, other) = (&tracks[round as usize % 2], &tracks[1 - round as usize % 2]); let groups = round * 10..round * 10 + 10; @@ -1380,32 +1385,26 @@ mod test { write(other, sequence); } seen.extend(drain(&mut sub, false)); - memory.push(remembered(&sub)); + assert_eq!(sub.delivered_runs(), 1, "round {round}"); } let mut sorted = seen.clone(); sorted.sort_unstable(); assert_eq!(sorted, (0..160).collect::>(), "handed out: {seen:?}"); - // Each round, only the serving copy is left, remembering its floor and the half - // round the replaced copy handed out. - assert!( - memory.iter().all(|copies| copies == &[6]), - "bookkeeping grew with the flaps: {memory:?}" - ); } /// A reader holding an old group open keeps its replaced copy, and moving its group - /// bounds neither repeats a group nor loses one. Once that copy finishes, what it - /// remembered goes with it. + /// bounds neither repeats a group nor loses one. #[test] fn held_groups_and_moved_bounds_hand_out_each_group_once() { let routes = Producer::new(); + let logical = logical(&routes); let a = copy(); routes.serve(a.consume()); - let mut sub = reader(&routes, HOUR); + let mut sub = subscribe(&logical, HOUR); sub.end_at(Cap::from(..=0)); let mut held = a.create_group(group::Info { sequence: 0 }).unwrap(); held.write_frame(ts(0), b"a".as_ref()).unwrap(); - let Poll::Ready(Ok(Some(reading))) = sub.poll_group(false, &kio::Waiter::noop()) else { + let Poll::Ready(Ok(Some(reading))) = sub.poll_recv_group(&kio::Waiter::noop()) else { panic!("group 0 is handed out") }; // Past the cap: parked on the copy, not handed out. @@ -1419,7 +1418,6 @@ mod test { write(&b, sequence); } assert_eq!(drain(&mut sub, false), Vec::::new()); - assert_eq!(sub.reader().copies.len(), 2, "the held group keeps its copy"); sub.end_at(Cap::from(..)); let mut seen = drain(&mut sub, false); @@ -1427,22 +1425,51 @@ mod test { assert_eq!(seen, [1, 2, 3]); sub.start_at(0); assert_eq!(drain(&mut sub, false), Vec::::new()); - // The replaced copy remembers the groups the serving one handed out through its - // newest at replacement; the serving one, the group handed out before it. - assert_eq!(remembered(&sub), [2, 1]); - // The replaced copy finishes; the held group is all it has left. + // The replaced copy finishes with the held group. held.finish().unwrap(); a.finish().unwrap(); drop(reading); assert_eq!(drain(&mut sub, false), Vec::::new()); - assert_eq!( - remembered(&sub), - [1], - "the finished copy is let go with what it remembered" - ); write(&b, 4); assert_eq!(drain(&mut sub, false), [4]); + assert_eq!(sub.delivered_runs(), 1); + } + + /// Across route changes, an endless track with groups missing remembers only the gaps + /// within the reader's budget: one past it is forgotten, one within it still delivered. + #[test] + fn a_resumed_reader_forgets_gaps_past_its_budget() { + let routes = Producer::new(); + let logical = logical(&routes); + let tracks = [copy(), copy()]; + routes.serve(tracks[0].consume()); + // Groups present 1ms apart, so the budget spans ten of them. + let mut sub = subscribe(&logical, Duration::from_millis(10)); + let mut most = 0; + for sequence in 0..3000u64 { + let track = &tracks[(sequence / 500) as usize % 2]; + // Every 500 groups the other route takes over. + if sequence % 500 == 0 { + routes.serve(track.consume()); + } + // Every third group never arrives. + if sequence % 3 == 1 { + continue; + } + write(track, sequence); + assert_eq!(drain(&mut sub, false), [sequence]); + most = most.max(sub.delivered_runs()); + } + // The floor and the four gaps a budget spans, twice over across a route change: the + // new copy starts at its first group, so the gaps just before it wait for that one + // to fall past the budget. + assert_eq!(most, 9); + + // 2998 is within the budget, 2500 far past it. + write(&tracks[1], 2998); + write(&tracks[1], 2500); + assert_eq!(drain(&mut sub, false), [2998]); } /// A replacement copy asked to start at the newest group handed out keeps that diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 2e8e834c28b..0a49ab50f52 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -16,6 +16,7 @@ use crate::{Error, Result, Timescale, Timestamp, coding}; use crate::{broadcast, cache, group, stats}; +use super::delivered::Delivered; use super::{Datagram, Requests}; use super::Cap; @@ -2287,6 +2288,7 @@ impl Producer { broadcast, info, inner: Inner::Plain(Cursor::new(self.state.consume(), subscription)), + delivered: Delivered::default(), // A producer-side (in-process) subscribe is not egress: stay untagged. stats: stats::Scope::default(), _stats_sub: stats::Subscription::default(), @@ -3339,6 +3341,7 @@ impl Subscribing { broadcast: self.broadcast.clone(), info, inner, + delivered: Delivered::default(), stats: self.stats.clone(), _stats_sub: self.stats.subscribe(), })) @@ -3678,6 +3681,9 @@ pub struct Subscriber { broadcast: Arc, info: Info, inner: Inner, + /// The groups handed out, so none is handed out twice: not when its sequence is + /// re-created, nor when another route delivers it too. + delivered: Delivered, // Egress stats scope, used to meter the groups this subscriber reads. Empty // (no-op) for an untagged track. stats: stats::Scope, @@ -3690,7 +3696,7 @@ pub struct Subscriber { pub(crate) struct Unread<'a> { pub start: u64, pub end: Option, - pub delivered: &'a std::collections::BTreeSet, + pub delivered: &'a Delivered, } /// How a [`Subscriber`] reads: its track's own cache, or a front's routes. @@ -3935,6 +3941,15 @@ impl Cursor { }) } + /// Whether a group at `sequence` is past this cursor's budget now, whether this track + /// holds it or not: one landing there would be written off. + fn is_past_budget(&self, sequence: u64) -> bool { + let max_delay = self.subscription.read().max_delay; + let state = self.state.read(); + let budget = clamp_max_delay(max_delay, state.max_age_bound()); + state.drifted(sequence, &state.drift_edge(self.end_sequence), budget) + } + fn with_expiry(&self, group: group::Consumer) -> group::Consumer { let sequence = group.sequence; group.with_expiry(Arc::new(GroupExpiry { @@ -4193,6 +4208,11 @@ impl Subscriber { /// [`Self::ordered`] if you only want groups whose sequence number is higher than any /// previously returned. /// + /// Once means once per sequence: a group the publisher aborts and re-creates, or that + /// a second route delivers too, is not handed out again. Once a group is past the + /// budget below, the subscriber stops expecting every group before it, so a gap that + /// old is never filled. + /// /// Groups are semi-reliable, and the [`Subscription::max_delay`] budget is the other /// thing (alongside eviction and a moving start) that decides which of them arrive: /// one that has drifted further behind the live edge than the budget tolerates is @@ -4215,15 +4235,62 @@ impl Subscriber { /// `Poll::Ready(Err(e))` when the track has been aborted, or /// `Poll::Pending` when no group is available yet. pub fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { + self.poll_group(false, waiter) + } + + /// The next group in arrival order, or in sequence order for `ordered`, skipping any + /// already handed out. + fn poll_group(&mut self, ordered: bool, waiter: &kio::Waiter) -> Poll>> { let meter = self.stats.meter(); let res = match &mut self.inner { - Inner::Plain(cursor) => cursor.poll_recv_group(waiter), - Inner::Resume(resume, _) => resume.poll_group(false, waiter), + Inner::Plain(cursor) => loop { + let res = match ordered { + true => cursor.poll_next_group(waiter), + false => cursor.poll_recv_group(waiter), + }; + match res { + Poll::Ready(Ok(Some(group))) if self.delivered.contains(group.sequence) => continue, + res => break res, + } + }, + Inner::Resume(resume, _) => resume.poll_group(ordered, &self.delivered, waiter), }; + if let Poll::Ready(Ok(Some(group))) = &res { + self.hand_out(group.sequence, ordered); + } self.count_stale(&meter); res.map(|res| res.map(|group| group.map(|group| group.with_meter(meter)))) } + /// Record `sequence` as handed out. An ordered read never goes back, and any reader + /// stops expecting what is past its budget, so only the gaps within it cost memory. + fn hand_out(&mut self, sequence: u64, ordered: bool) { + if ordered { + self.delivered.forget_below(sequence); + } + self.delivered.insert(sequence); + while let Some(oldest) = self.delivered.above_gap() + && self.is_past_budget(oldest) + { + self.delivered.forget_below(oldest); + } + } + + /// Whether a group at `sequence` is past this subscriber's budget now: one landing + /// there would be written off. + pub(crate) fn is_past_budget(&self, sequence: u64) -> bool { + match &self.inner { + Inner::Plain(cursor) => cursor.is_past_budget(sequence), + Inner::Resume(resume, _) => resume.is_past_budget(sequence), + } + } + + /// How many runs of handed-out groups this subscriber remembers. + #[cfg(test)] + pub(crate) fn delivered_runs(&self) -> usize { + self.delivered.runs() + } + /// Whether an unread cached group needs this subscription, without moving its cursor. pub(crate) fn has_unread_group(&self, unread: &Unread<'_>) -> bool { let cursor = match &self.inner { @@ -4235,7 +4302,7 @@ impl Subscriber { let eligible = |sequence: u64| { sequence >= floor && super::subscription::before_end(sequence, unread.end) - && !unread.delivered.contains(&sequence) + && !unread.delivered.contains(sequence) }; cursor .parked @@ -4257,7 +4324,7 @@ impl Subscriber { /// Receive the next group in arrival order. /// - /// Every group is returned exactly once, in the order it landed on the wire, which may + /// Every group sequence is returned at most once, in the order it landed on the wire, which may /// be out of sequence due to network reordering or loss. Use [`Self::ordered`] if you /// only want groups whose sequence number is higher than any previously returned. /// See [`Self::poll_recv_group`] for how [`Self::set_groups`] applies. @@ -4281,7 +4348,7 @@ impl Subscriber { let meter = self.stats.meter(); let res = match &mut self.inner { Inner::Plain(cursor) => cursor.poll_recv_datagram(waiter), - Inner::Resume(resume, _) => resume.poll_recv_datagram(waiter), + Inner::Resume(resume, _) => resume.poll_recv_datagram(&self.delivered, waiter), }; // Unlike a group (metered lazily as its frames are read), a datagram is // delivered whole here, so count it as the single-frame group it stands in for. @@ -4303,13 +4370,7 @@ impl Subscriber { /// The sequence cursor behind [`Ordered`], which owns the only public door to it. pub(crate) fn poll_next_group(&mut self, waiter: &kio::Waiter) -> Poll>> { - let meter = self.stats.meter(); - let res = match &mut self.inner { - Inner::Plain(cursor) => cursor.poll_next_group(waiter), - Inner::Resume(resume, _) => resume.poll_group(true, waiter), - }; - self.count_stale(&meter); - res.map(|res| res.map(|group| group.map(|group| group.with_meter(meter)))) + self.poll_group(true, waiter) } /// Read this track's groups in sequence order instead of arrival order. @@ -10632,4 +10693,97 @@ mod test { assert!(pending.await.is_err()); drop(dynamic); } + + /// Write a finished one-frame group, presented at its sequence in milliseconds. + fn write_group(producer: &Producer, sequence: u64) { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(sequence).unwrap(), &b"x"[..]) + .unwrap(); + group.finish().unwrap(); + } + + /// Every group the subscriber hands out right now, in arrival order. + fn drain_groups(subscriber: &mut Subscriber) -> Vec { + let mut sequences = Vec::new(); + while let Poll::Ready(Ok(Some(group))) = subscriber.poll_recv_group(&kio::Waiter::noop()) { + sequences.push(group.sequence); + } + sequences + } + + /// A sequence the publisher aborts and re-creates is handed out once. + #[test] + fn a_recreated_group_is_handed_out_once() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(replay()); + write_group(&producer, 0); + let open = producer.create_group(group::Info { sequence: 1 }).unwrap(); + assert_eq!(drain_groups(&mut subscriber), [0, 1]); + + open.abort(Error::Cancel).unwrap(); + write_group(&producer, 1); + write_group(&producer, 2); + assert_eq!(drain_groups(&mut subscriber), [2]); + } + + /// A gap is remembered until a group above it is past the budget, max delay or max + /// age alike. Then every group below that one counts as handed out. + #[test] + fn a_gap_is_forgotten_once_a_group_above_it_is_past_the_budget() { + let budget = Duration::from_millis(10); + let readers = [ + (Info::timed(), Subscription::default().with_max_delay(budget)), + ( + Info::timed().with_max_age(budget), + Subscription::default().with_max_delay(Duration::from_secs(3600)), + ), + ]; + for (info, subscription) in readers { + let producer = track_producer("test", info); + let mut subscriber = producer.subscribe(subscription); + // Group 3 never arrives. + for sequence in [0, 1, 2, 4, 5, 6] { + write_group(&producer, sequence); + } + assert_eq!(drain_groups(&mut subscriber), [0, 1, 2, 4, 5, 6]); + // Group 4 reaches until group 5 presents at 5ms, so it is past a 10ms budget once + // the edge presents at 15ms. + for sequence in 7..=14 { + write_group(&producer, sequence); + assert_eq!(drain_groups(&mut subscriber), [sequence]); + assert_eq!(subscriber.delivered_runs(), 2, "the gap at 3 is kept at {sequence}"); + } + write_group(&producer, 15); + assert_eq!(drain_groups(&mut subscriber), [15]); + assert_eq!(subscriber.delivered_runs(), 1, "the gap at 3 is forgotten"); + write_group(&producer, 3); + assert_eq!(drain_groups(&mut subscriber), Vec::::new()); + } + } + + /// An endless track with groups missing remembers only the gaps within the budget, and + /// still delivers an unseen group that arrives late within it. + #[test] + fn an_endless_track_with_gaps_stays_bounded() { + let producer = track_producer("test", Info::timed()); + // Groups present 1ms apart, so the budget spans ten of them. + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + let mut most = 0; + for sequence in 0..3000u64 { + // Every third group never arrives. + if sequence % 3 == 1 { + continue; + } + write_group(&producer, sequence); + assert_eq!(drain_groups(&mut subscriber), [sequence]); + most = most.max(subscriber.delivered_runs()); + } + assert_eq!(most, 5, "the floor and the gaps within the budget"); + + // 2998 is within the budget, 2500 far past it. + write_group(&producer, 2998); + write_group(&producer, 2500); + assert_eq!(drain_groups(&mut subscriber), [2998]); + } } From 839061f9ec906af06ad306df3aaba0aa9adf83ee Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 14:22:50 -0700 Subject: [PATCH 05/21] quest: plan bounding the delivered record on untimed tracks Unblock parked-drain now that resume-duplicates lands here. Co-Authored-By: Claude Opus 5.5 --- quest/m0/parked-drain.md | 4 ---- quest/m1/cache-max-age.md | 9 +++++++++ 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/quest/m0/parked-drain.md b/quest/m0/parked-drain.md index 8ce8682a52b..ad34f3f5601 100644 --- a/quest/m0/parked-drain.md +++ b/quest/m0/parked-drain.md @@ -39,7 +39,3 @@ against capped subscribers per track: the parked axis flattens, and a per-abort cost that grows with subscribers shows up as a slope. Public API: none. Wire: none. - -## Required - -- [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - reshapes `track::Subscriber`'s delivery record in the same code diff --git a/quest/m1/cache-max-age.md b/quest/m1/cache-max-age.md index 97643b8c2b9..7c3a2d0032c 100644 --- a/quest/m1/cache-max-age.md +++ b/quest/m1/cache-max-age.md @@ -65,6 +65,11 @@ Decided (2026-10-06, maintainer): reclamation from the rule), and the untimed text in `drafts/draft-lcurley-moq-timestamp.md` that the untimed model adds ("never media-stale, starts at the latest group"). No new guide. +- Decided 2026-10-13 (maintainer): the wall term also advances the floor of + `track::Subscriber`'s `Delivered` record (`rs/moq-net/src/model/delivered.rs`). + An untimed track has no media time, so that floor never advances on it, and + an endless untimed track that skips groups grows the record by one range per + gap. Facts, and work carried over: @@ -120,6 +125,10 @@ Tests, with mocked time: hide the stall. Nothing writes after the successor arrives, so only the deadline wake can release the reader. It fails without the fix. - An untimed subscriber starts at the oldest group that isn't stale. +- An endless untimed track with gaps keeps a subscriber's `Delivered` record + bounded: the wall term forgets a gap once a delivered group above it is + stale, as `track::an_endless_track_with_gaps_stays_bounded` checks for + timed tracks. Public API: no signature change; `max_age` behaviour changes. Wire: no encoding change; Max Age semantics in the lite draft change. From 557fce7654143f2fdaf4b72275bdb1521ac63427 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 14:37:03 -0700 Subject: [PATCH 06/21] fix(net): end an ordered read after the maximum sequence The ordered cursor saturated at u64::MAX and returned that group again, which the new duplicate skip turned into a busy loop. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 0a49ab50f52..177341b6aa7 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -3803,9 +3803,10 @@ struct Cursor { datagram_index: usize, /// Minimum sequence to return from any `recv` method. Set by `start_at`. min_sequence: u64, - /// One past the highest sequence returned by `next_group`. + /// One past the highest sequence returned by `next_group`, or `None` once it returned + /// `u64::MAX` and no group can follow. /// Used only by that method to skip late arrivals; does not affect `recv_group`. - next_sequence: u64, + next_sequence: Option, /// Exclusive upper sequence bound for `next_group` and `recv_group`, in the form /// [`Cap::exclusive`] produces. `None` means no cap. Set by `end_at`; /// can be raised, lowered, or unset at any time. Groups at or past the cap stay in @@ -3873,7 +3874,7 @@ impl Cursor { min_sequence, index: 0, datagram_index, - next_sequence: 0, + next_sequence: Some(0), end_sequence: None, parked: BTreeMap::new(), drift_cap: kio::Producer::new(None), @@ -4069,7 +4070,10 @@ impl Cursor { /// The lowest servable group past the last one returned, walking off everything the /// drift budget has convicted on the way. fn poll_next_group(&mut self, waiter: &kio::Waiter) -> Poll>> { - let mut floor = self.next_sequence.max(self.min_sequence); + let Some(next) = self.next_sequence else { + return Poll::Ready(Ok(None)); + }; + let mut floor = next.max(self.min_sequence); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. let drift = ready!(self.poll_drift(end, waiter))?; @@ -4102,9 +4106,9 @@ impl Cursor { break self.with_expiry(group); }; - self.next_sequence = group.sequence.saturating_add(1); + self.next_sequence = group.sequence.checked_add(1); // The delivery commits everything the seek stepped over to reach this group. - self.commit_seek_stale(self.next_sequence); + self.commit_seek_stale(self.next_sequence.unwrap_or(u64::MAX)); // Delivery is a cache access, same as the arrival-order path. group.cache_refresh(); Poll::Ready(Ok(Some(group))) @@ -9076,6 +9080,8 @@ mod test { consumer.next_group().now_or_never().unwrap().unwrap().unwrap().sequence, u64::MAX ); + // Nothing can follow the maximum sequence. + assert!(consumer.next_group().now_or_never().unwrap().unwrap().is_none()); } #[test] From 211c42e2b51cde154099fb01f5872bcbbb2c3270 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 15:49:48 -0700 Subject: [PATCH 07/21] docs(net): state that group timestamps never rewind for the delivered floor Co-Authored-By: Claude Opus 5.5 --- quest/m1/js-group-handover.md | 4 +++- rs/moq-net/src/model/delivered.rs | 2 +- rs/moq-net/src/model/track.rs | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/quest/m1/js-group-handover.md b/quest/m1/js-group-handover.md index 27bdcd4a683..2a9f0d67aae 100644 --- a/quest/m1/js-group-handover.md +++ b/quest/m1/js-group-handover.md @@ -37,7 +37,9 @@ by direct and resumed reads, with no fixed-size history. Once a group is past the reader's budget (max delay, clamped by max age), the reader stops expecting every group below it and counts them as delivered, so only gaps within the budget cost memory. A genuinely unseen late group within the budget is still -delivered. +delivered. Group timestamps on a track never rewind; which late groups are +delivered after a rewind is unspecified, so the floor is judged by the +delivered group above a gap rather than the gap itself. Things to settle along the way: - Whether the request's `active` broadcast stays the same object across a swap, diff --git a/rs/moq-net/src/model/delivered.rs b/rs/moq-net/src/model/delivered.rs index f11a38242e2..07cbe2c7dab 100644 --- a/rs/moq-net/src/model/delivered.rs +++ b/rs/moq-net/src/model/delivered.rs @@ -56,7 +56,7 @@ impl Delivered { } /// The lowest sequence handed out above a gap: once it is past the reader's budget, - /// so is the gap. + /// so is the gap, since group timestamps on a track never rewind. pub fn above_gap(&self) -> Option { let mut runs = self.runs.keys(); match runs.next()? { diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 177341b6aa7..8350196cb05 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4215,7 +4215,8 @@ impl Subscriber { /// Once means once per sequence: a group the publisher aborts and re-creates, or that /// a second route delivers too, is not handed out again. Once a group is past the /// budget below, the subscriber stops expecting every group before it, so a gap that - /// old is never filled. + /// old is never filled. That assumes group timestamps on a track never rewind; which + /// late groups are handed out after a rewind is unspecified. /// /// Groups are semi-reliable, and the [`Subscription::max_delay`] budget is the other /// thing (alongside eviction and a moving start) that decides which of them arrive: From f6629aac85372e1bb3ebb2498749e7b1ee386649 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 20:00:02 -0700 Subject: [PATCH 08/21] fix(net): forget gaps below the newest group past the budget A group followed by an empty one never falls past the budget, so judging only the oldest group above a gap let it pin every later gap on a timed track. Walk the groups above gaps newest first instead. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/delivered.rs | 19 ++++++-------- rs/moq-net/src/model/track.rs | 41 ++++++++++++++++++++++++++++--- 2 files changed, 45 insertions(+), 15 deletions(-) diff --git a/rs/moq-net/src/model/delivered.rs b/rs/moq-net/src/model/delivered.rs index 07cbe2c7dab..42c2568d5da 100644 --- a/rs/moq-net/src/model/delivered.rs +++ b/rs/moq-net/src/model/delivered.rs @@ -55,14 +55,11 @@ impl Delivered { self.runs.insert(0, last); } - /// The lowest sequence handed out above a gap: once it is past the reader's budget, - /// so is the gap, since group timestamps on a track never rewind. - pub fn above_gap(&self) -> Option { - let mut runs = self.runs.keys(); - match runs.next()? { - 0 => runs.next().copied(), - first => Some(*first), - } + /// Each sequence handed out right above a gap, lowest first: once one is past the + /// reader's budget, so is every gap below it, since group timestamps on a track never + /// rewind. + pub fn above_gaps(&self) -> impl DoubleEndedIterator + '_ { + self.runs.keys().copied().filter(|first| *first != 0) } /// How many runs it holds: one per gap, plus the floor. @@ -113,14 +110,14 @@ mod test { for sequence in [3, 5, 6, 9, 12] { delivered.insert(sequence); } - assert_eq!(delivered.above_gap(), Some(3)); + assert_eq!(delivered.above_gaps().collect::>(), [3, 5, 9, 12]); delivered.forget_below(0); assert_eq!(delivered.runs(), 4); // A floor at a run's start takes the run along. delivered.forget_below(5); assert_eq!(runs(&delivered), [(0, 6), (9, 9), (12, 12)]); - assert_eq!(delivered.above_gap(), Some(9)); + assert_eq!(delivered.above_gaps().collect::>(), [9, 12]); assert!(delivered.contains(4), "below the floor counts as handed out"); // A floor inside a gap keeps the run above it. @@ -128,6 +125,6 @@ mod test { assert_eq!(runs(&delivered), [(0, 10), (12, 12)]); delivered.forget_below(13); assert_eq!(runs(&delivered), [(0, 12)]); - assert_eq!(delivered.above_gap(), None); + assert_eq!(delivered.above_gaps().next(), None); } } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 8350196cb05..1b5b649a7ba 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4274,10 +4274,15 @@ impl Subscriber { self.delivered.forget_below(sequence); } self.delivered.insert(sequence); - while let Some(oldest) = self.delivered.above_gap() - && self.is_past_budget(oldest) - { - self.delivered.forget_below(oldest); + // Newest first: a group followed by an empty one never falls past the budget, so + // the oldest alone could pin every gap above it. + let floor = self + .delivered + .above_gaps() + .rev() + .find(|above| self.is_past_budget(*above)); + if let Some(floor) = floor { + self.delivered.forget_below(floor); } } @@ -10793,4 +10798,32 @@ mod test { write_group(&producer, 2500); assert_eq!(drain_groups(&mut subscriber), [2998]); } + + /// An empty group leaves the group below it unbounded, so it never falls past the + /// budget. Gaps above it are still forgotten once a group above them is past it. + #[test] + fn an_empty_group_does_not_pin_later_gaps() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + write_group(&producer, 0); + write_group(&producer, 2); + producer + .create_group(group::Info { sequence: 3 }) + .unwrap() + .finish() + .unwrap(); + assert_eq!(drain_groups(&mut subscriber), [0, 2, 3]); + let mut most = 0; + for sequence in 5..3000u64 { + // Every third group never arrives. + if sequence % 3 == 1 { + continue; + } + write_group(&producer, sequence); + assert_eq!(drain_groups(&mut subscriber), [sequence]); + most = most.max(subscriber.delivered_runs()); + } + // One more than the endless track: the gap at 1 waits for a group above it to fall past the budget. + assert_eq!(most, 6, "the floor and the gaps within the budget"); + } } From 5144a50d1e04737c6df83de6fbf15ffafd4bebab Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 21:48:02 -0700 Subject: [PATCH 09/21] fix(net): bound a group by the next one past a finished empty successor A group that finished without a frame never presents, yet reach treated it as an unstamped successor and left the group before it unbounded. That kept the group from ever drifting, and let one such group pin every later gap in the delivered record. Reach now skips a finished empty successor. That restores the O(1) check of the lowest group above a gap in hand_out, reverting the newest-first walk, which rescanned every retained gap per hand-out. The drift edge is None on an untimed track, so a poll no longer scans the whole cache for a timestamp that cannot exist. track_sparse_read sweeps hand-out cost over retained gaps. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 49 ++++++++++- rs/moq-net/src/model/delivered.rs | 19 ++-- rs/moq-net/src/model/group.rs | 6 ++ rs/moq-net/src/model/track.rs | 140 ++++++++++++++++++++++++------ 4 files changed, 179 insertions(+), 35 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 32a1542489b..317b7d58f57 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -23,6 +23,10 @@ //! each append must wake only the reads it can expire, so waking the backlog shows up //! as a slope over it. //! +//! `track_sparse_read` hands out groups from a feed with every other sequence missing, +//! after a backlog of such gaps the reader still remembers: each hand-out must cost the +//! same however many gaps it holds, so a walk over them shows up as a slope. +//! //! Run with `cargo bench -p moq-net --bench track`. use std::hint::black_box; @@ -521,6 +525,48 @@ fn bench_parked_read(c: &mut Criterion) { group.finish(); } +/// One hand-out from a sparse feed past `gaps` remembered gaps, on an untimed track and +/// on a timed one whose budget never writes a gap off. Swept over the gaps so a per-hand-out +/// cost that grows with them shows up as a slope. +fn bench_sparse_read(c: &mut Criterion) { + let mut group = c.benchmark_group("track_sparse_read"); + group.throughput(Throughput::Elements(1)); + for (name, info) in [("untimed", track::Info::default()), ("timed", timed())] { + for gaps in [16u64, 256, 4096] { + group.bench_function(BenchmarkId::new(name, gaps), |b| { + b.iter_custom(|iterations| { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", info.clone()).unwrap(); + let mut sub = + track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let timestamp = |sequence: u64| info.timescale.map(|_| Timestamp::from_millis(sequence).unwrap()); + let mut hand_out = |sequence: u64| { + let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); + group + .write_frame(timestamp(sequence), Bytes::from_static(b"x")) + .unwrap(); + group.finish().unwrap(); + let Poll::Ready(Ok(Some(read))) = sub.poll_recv_group(&kio::Waiter::noop()) else { + panic!("the group is cached"); + }; + black_box(read); + }; + // Every odd sequence never arrives. + for gap in 0..gaps { + hand_out(gap * 2); + } + let start = Instant::now(); + for iteration in 0..iterations { + hand_out((gaps + iteration) * 2); + } + start.elapsed() + }); + }); + } + } + group.finish(); +} + criterion_group!( benches, bench_fanout, @@ -531,6 +577,7 @@ criterion_group!( bench_subscriber_churn_after_peak, bench_subscriber_join, bench_datagram_subscriber_join, - bench_parked_read + bench_parked_read, + bench_sparse_read ); criterion_main!(benches); diff --git a/rs/moq-net/src/model/delivered.rs b/rs/moq-net/src/model/delivered.rs index 42c2568d5da..07cbe2c7dab 100644 --- a/rs/moq-net/src/model/delivered.rs +++ b/rs/moq-net/src/model/delivered.rs @@ -55,11 +55,14 @@ impl Delivered { self.runs.insert(0, last); } - /// Each sequence handed out right above a gap, lowest first: once one is past the - /// reader's budget, so is every gap below it, since group timestamps on a track never - /// rewind. - pub fn above_gaps(&self) -> impl DoubleEndedIterator + '_ { - self.runs.keys().copied().filter(|first| *first != 0) + /// The lowest sequence handed out above a gap: once it is past the reader's budget, + /// so is the gap, since group timestamps on a track never rewind. + pub fn above_gap(&self) -> Option { + let mut runs = self.runs.keys(); + match runs.next()? { + 0 => runs.next().copied(), + first => Some(*first), + } } /// How many runs it holds: one per gap, plus the floor. @@ -110,14 +113,14 @@ mod test { for sequence in [3, 5, 6, 9, 12] { delivered.insert(sequence); } - assert_eq!(delivered.above_gaps().collect::>(), [3, 5, 9, 12]); + assert_eq!(delivered.above_gap(), Some(3)); delivered.forget_below(0); assert_eq!(delivered.runs(), 4); // A floor at a run's start takes the run along. delivered.forget_below(5); assert_eq!(runs(&delivered), [(0, 6), (9, 9), (12, 12)]); - assert_eq!(delivered.above_gaps().collect::>(), [9, 12]); + assert_eq!(delivered.above_gap(), Some(9)); assert!(delivered.contains(4), "below the floor counts as handed out"); // A floor inside a gap keeps the run above it. @@ -125,6 +128,6 @@ mod test { assert_eq!(runs(&delivered), [(0, 10), (12, 12)]); delivered.forget_below(13); assert_eq!(runs(&delivered), [(0, 12)]); - assert_eq!(delivered.above_gaps().next(), None); + assert_eq!(delivered.above_gap(), None); } } diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index e43cba217a7..6a802d4ead6 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -959,6 +959,12 @@ impl Producer { self.state.read().started() } + /// Whether the group finished without a frame, so it never presents anything. + pub(crate) fn ended_empty(&self) -> bool { + let state = self.state.read(); + state.fin.is_some() && matches!(state.timeline, Timeline::Empty) + } + /// Where the group starts in presentation time: its first frame's timestamp, or /// `None` while no frame has been opened or when that frame is untimed. /// diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 1b5b649a7ba..bf81104c597 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -740,9 +740,13 @@ impl TrackState { } /// This track's own edge under the exclusive `cap`, for measuring drift. + /// + /// None on an untimed track, where nothing is ever stale: its groups carry no + /// timestamp, so looking for one would scan the whole cache on every poll. fn drift_edge(&self, cap: Option) -> Edge { + let untimed = self.info.as_ref().is_some_and(|info| info.timescale.is_none()); Edge { - presentation: self.live_edge(cap), + presentation: if untimed { None } else { self.live_edge(cap) }, cap, } } @@ -764,18 +768,20 @@ impl TrackState { /// so a later stamped group proves nothing about where an unstamped successor will /// begin, and shrinking the bound is the unsafe direction. An unstamped successor /// therefore leaves the reach unbounded until it presents its first frame, and so - /// does having no servable successor below `cap` at all. + /// does having no servable successor below `cap` at all. A successor that finished + /// without a frame never presents, so it is skipped and the next one bounds both. fn reach(&self, sequence: u64, cap: Option) -> Option { - self.first_servable(sequence.saturating_add(1), cap)?.timestamp() + self.successor(sequence, cap)?.timestamp() } - /// The first servable group in `from..cap`, stamped or not. - fn first_servable(&self, from: u64, cap: Option) -> Option<&group::Producer> { + /// The first servable group above `sequence` and below `cap` that could still + /// present, stamped or not. + fn successor(&self, sequence: u64, cap: Option) -> Option<&group::Producer> { self.lookup - .range(from..) + .range(sequence.checked_add(1)?..) .map(|(_, slot)| slot) .take_while(|slot| super::subscription::before_end(slot.group.sequence, cap)) - .find(|slot| slot.is_shown()) + .find(|slot| slot.is_shown() && !slot.group.ended_empty()) .map(|slot| &slot.group) } @@ -836,7 +842,7 @@ impl TrackState { // An abort only closes the group, never touching the track, so one that // landed before registering must re-select or the replacement goes unwatched. let successor = loop { - let successor = self.first_servable(sequence.saturating_add(1), cap); + let successor = self.successor(sequence, cap); match successor { Some(group) if group.poll_closed(waiter).is_ready() && group.is_aborted() => continue, successor => break successor, @@ -849,7 +855,12 @@ impl TrackState { if let Some(reach) = successor.timestamp() { break reach; } - if successor.poll_started(waiter).is_pending() || successor.timestamp().is_none() { + if successor.poll_started(waiter).is_pending() { + return false; + } + // An untimed successor stays unbounded. One that finished without a frame since + // it was selected is skipped on the next pass. + if successor.timestamp().is_none() && !successor.ended_empty() { return false; } }; @@ -4274,15 +4285,10 @@ impl Subscriber { self.delivered.forget_below(sequence); } self.delivered.insert(sequence); - // Newest first: a group followed by an empty one never falls past the budget, so - // the oldest alone could pin every gap above it. - let floor = self - .delivered - .above_gaps() - .rev() - .find(|above| self.is_past_budget(*above)); - if let Some(floor) = floor { - self.delivered.forget_below(floor); + while let Some(oldest) = self.delivered.above_gap() + && self.is_past_budget(oldest) + { + self.delivered.forget_below(oldest); } } @@ -6497,6 +6503,35 @@ mod test { assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); } + /// An open empty successor leaves a held group's reach unbounded. Once it finishes + /// without a frame, the held read wakes and is bounded by the group after it. + #[moq_net_sim::test] + async fn an_empty_successor_finishing_wakes_a_parked_read() { + let mut producer = track_producer("test", Info::timed()); + let mut head = producer.append_group().unwrap(); + head.write_frame(Timestamp::ZERO, b"head".as_slice()).unwrap(); + let successor = producer.append_group().unwrap(); + append_at(&mut producer, 1000); + append_at(&mut producer, 20_000); + let mut sub = producer.subscribe(None); + let mut reading = sub.recv_group().await.unwrap().unwrap(); + assert_eq!(reading.sequence, 0, "the open successor leaves the reach unbounded"); + assert!(reading.read_frame().await.unwrap().is_some()); + + let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); + let mut cx = std::task::Context::from_waker(&waker); + let mut next = std::pin::pin!(reading.read_frame()); + assert!(next.as_mut().poll(&mut cx).is_pending()); + successor.finish().unwrap(); + assert!( + woken.load(Ordering::SeqCst), + "the empty successor's finish lost its wakeup" + ); + let result = next.as_mut().poll(&mut cx); + assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); + } + /// A first timestamp on a *newer* group can convict a held one, so the held reader has /// to be woken by it. The conviction needs a group beyond the held one's successor: /// a group is bounded by where its successor begins, so the successor itself never @@ -10799,19 +10834,37 @@ mod test { assert_eq!(drain_groups(&mut subscriber), [2998]); } - /// An empty group leaves the group below it unbounded, so it never falls past the - /// budget. Gaps above it are still forgotten once a group above them is past it. + /// Write a finished group with no frames. + fn write_empty_group(producer: &Producer, sequence: u64) { + producer + .create_group(group::Info { sequence }) + .unwrap() + .finish() + .unwrap(); + } + + /// A group followed by one that finished without a frame is bounded by the next + /// group instead, so it falls past the budget like any other. + #[test] + fn a_group_before_an_empty_one_can_drift() { + let producer = track_producer("test", Info::timed()); + write_group(&producer, 0); + write_empty_group(&producer, 1); + write_group(&producer, 50); + write_group(&producer, 100); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + assert_eq!(drain_groups(&mut subscriber), [50, 100]); + } + + /// A group that finished without a frame never presents, so the group below it is + /// bounded by the next one instead, and gaps above it are still forgotten. #[test] fn an_empty_group_does_not_pin_later_gaps() { let producer = track_producer("test", Info::timed()); let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); write_group(&producer, 0); write_group(&producer, 2); - producer - .create_group(group::Info { sequence: 3 }) - .unwrap() - .finish() - .unwrap(); + write_empty_group(&producer, 3); assert_eq!(drain_groups(&mut subscriber), [0, 2, 3]); let mut most = 0; for sequence in 5..3000u64 { @@ -10823,7 +10876,42 @@ mod test { assert_eq!(drain_groups(&mut subscriber), [sequence]); most = most.max(subscriber.delivered_runs()); } - // One more than the endless track: the gap at 1 waits for a group above it to fall past the budget. + // One more than the endless track: group 2 reaches until group 5, past the empty 3. assert_eq!(most, 6, "the floor and the gaps within the budget"); } + + /// Every run starting with a stamped group followed by an empty one still falls past + /// the budget, so the record stays bounded. + #[test] + fn repeated_empty_groups_do_not_pin_gaps() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + write_group(&producer, 0); + assert_eq!(drain_groups(&mut subscriber), [0]); + let mut most = 0; + for k in 0..1000u64 { + // 3k + 1 never arrives. + write_group(&producer, 3 * k + 2); + write_empty_group(&producer, 3 * k + 3); + assert_eq!(drain_groups(&mut subscriber), [3 * k + 2, 3 * k + 3]); + most = most.max(subscriber.delivered_runs()); + } + assert_eq!(most, 6, "the floor and the gaps within the budget"); + } + + /// An untimed track has nothing past the budget, so a sparse feed keeps every gap + /// and still hands out every group. The hand-out cost stays flat over the retained + /// gaps; `track_sparse_read` measures it. + #[test] + fn an_untimed_sparse_feed_keeps_its_gaps() { + let producer = untimed_producer(); + let mut subscriber = producer.subscribe(replay()); + for sequence in (0..2000u64).step_by(2) { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group.write_frame(None, &b"x"[..]).unwrap(); + group.finish().unwrap(); + assert_eq!(drain_groups(&mut subscriber), [sequence]); + } + assert_eq!(subscriber.delivered_runs(), 1000); + } } From c0a07c0aa6d4d8bb1bdb6a1e1ea9f088484f92cc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 22:40:38 -0700 Subject: [PATCH 10/21] bench(net): hold the retained gap count fixed in track_sparse_read Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 317b7d58f57..881814865b9 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -24,7 +24,7 @@ //! as a slope over it. //! //! `track_sparse_read` hands out groups from a feed with every other sequence missing, -//! after a backlog of such gaps the reader still remembers: each hand-out must cost the +//! holding a fixed number of gaps the reader still remembers: each hand-out must cost the //! same however many gaps it holds, so a walk over them shows up as a slope. //! //! Run with `cargo bench -p moq-net --bench track`. @@ -525,12 +525,14 @@ fn bench_parked_read(c: &mut Criterion) { group.finish(); } -/// One hand-out from a sparse feed past `gaps` remembered gaps, on an untimed track and -/// on a timed one whose budget never writes a gap off. Swept over the gaps so a per-hand-out -/// cost that grows with them shows up as a slope. +/// Hand-outs from a sparse feed holding `gaps` remembered gaps, on an untimed track and on +/// a timed one whose budget never writes a gap off. Each iteration hands out the next group, +/// opening a gap at the top, and the lowest missing one, closing the gap at the bottom, so +/// the count stays fixed. Swept over the gaps so a per-hand-out cost that grows with them +/// shows up as a slope. fn bench_sparse_read(c: &mut Criterion) { let mut group = c.benchmark_group("track_sparse_read"); - group.throughput(Throughput::Elements(1)); + group.throughput(Throughput::Elements(2)); for (name, info) in [("untimed", track::Info::default()), ("timed", timed())] { for gaps in [16u64, 256, 4096] { group.bench_function(BenchmarkId::new(name, gaps), |b| { @@ -539,7 +541,8 @@ fn bench_sparse_read(c: &mut Criterion) { let track = broadcast.create_track("bench", info.clone()).unwrap(); let mut sub = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); - let timestamp = |sequence: u64| info.timescale.map(|_| Timestamp::from_millis(sequence).unwrap()); + // Microseconds, so no run lasts long enough to cross the budget. + let timestamp = |sequence: u64| info.timescale.map(|_| Timestamp::from_micros(sequence).unwrap()); let mut hand_out = |sequence: u64| { let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); group @@ -551,13 +554,14 @@ fn bench_sparse_read(c: &mut Criterion) { }; black_box(read); }; - // Every odd sequence never arrives. + // Every odd sequence is missing until the bottom gap closes. for gap in 0..gaps { hand_out(gap * 2); } let start = Instant::now(); for iteration in 0..iterations { hand_out((gaps + iteration) * 2); + hand_out(iteration * 2 + 1); } start.elapsed() }); From 4063abb9d0df8c578d65315d6a716b00aecbb4f5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 10:02:01 -0700 Subject: [PATCH 11/21] fix(net): walk a run of groups that cannot present once per track Skipping finished empty successors made every group judged against a run of them walk the rest of the run: O(N^2) per reader, whether one poll wrote the run off or each poll handed one group out. The track now records where each walk stopped and later walks jump there; a group landing inside a recorded run clears it. track_empty_run sweeps run length and readers. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 86 +++++++++++++++++++- rs/moq-net/src/model/track.rs | 144 ++++++++++++++++++++++++++++++++-- 2 files changed, 222 insertions(+), 8 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 881814865b9..6d480b4f48b 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -27,6 +27,10 @@ //! holding a fixed number of gaps the reader still remembers: each hand-out must cost the //! same however many gaps it holds, so a walk over them shows up as a slope. //! +//! `track_empty_run` drains a run of groups that finished without a frame, written off in +//! one poll or handed out a group per poll, across readers: judging each group must not +//! walk the rest of the run, so a per-group walk shows up as a slope over it. +//! //! Run with `cargo bench -p moq-net --bench track`. use std::hint::black_box; @@ -571,6 +575,85 @@ fn bench_sparse_read(c: &mut Criterion) { group.finish(); } +/// Readers on one track holding group 0, a run of groups that finished without a frame, +/// and the groups after it. +struct EmptyRun { + _broadcast: broadcast::Producer, + _track: track::Producer, + readers: Vec, + /// Groups each reader should be handed. + handed: usize, +} + +impl EmptyRun { + /// `stale` stamps the groups after the run 50 ms and 100 ms in, so a 10 ms budget writes + /// off group 0 and the run in one poll. Otherwise one group 5 ms in follows the run and + /// keeps it within the budget, so each poll hands out one group of it. + fn new(run: u64, readers: usize, stale: bool) -> Self { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", timed()).unwrap(); + let append = |millis: Option| { + let mut group = track.append_group().unwrap(); + if let Some(millis) = millis { + group + .write_frame(Timestamp::from_millis(millis).unwrap(), Bytes::from_static(b"x")) + .unwrap(); + } + group.finish().unwrap(); + }; + append(Some(0)); + for _ in 0..run { + append(None); + } + let after: &[u64] = if stale { &[50, 100] } else { &[5] }; + for millis in after { + append(Some(*millis)); + } + let budget = track::Subscription::default().with_max_delay(Duration::from_millis(10)); + Self { + readers: (0..readers).map(|_| track.subscribe(budget.clone())).collect(), + _broadcast: broadcast, + _track: track, + handed: if stale { 2 } else { run as usize + 2 }, + } + } + + fn drain(&mut self) { + for reader in &mut self.readers { + let mut handed = 0; + while let Poll::Ready(Ok(Some(group))) = reader.poll_recv_group(&kio::Waiter::noop()) { + black_box(group); + handed += 1; + } + assert_eq!(handed, self.handed); + } + } +} + +/// Readers draining a run of groups that finished without a frame, swept over the run and +/// the readers. `stale` writes the run off in one poll; `fresh` hands it out a group per +/// poll. Either way a reader's cost must stay linear in the run, so a walk over the rest of +/// the run per group shows up as a slope over it. +fn bench_empty_run(c: &mut Criterion) { + let mut group = c.benchmark_group("track_empty_run"); + group.sample_size(10); + for (name, stale) in [("stale", true), ("fresh", false)] { + for readers in [1usize, 8, 64] { + for run in [64u64, 512, 4096] { + group.throughput(Throughput::Elements(run * readers as u64)); + group.bench_function(BenchmarkId::new(format!("{name}/{readers}"), run), |b| { + b.iter_batched_ref( + || EmptyRun::new(run, readers, stale), + EmptyRun::drain, + BatchSize::PerIteration, + ); + }); + } + } + } + group.finish(); +} + criterion_group!( benches, bench_fanout, @@ -582,6 +665,7 @@ criterion_group!( bench_subscriber_join, bench_datagram_subscriber_join, bench_parked_read, - bench_sparse_read + bench_sparse_read, + bench_empty_run ); criterion_main!(benches); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 3bbc0f645ce..bfafa119a75 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -23,7 +23,7 @@ use super::Cap; pub use super::subscription::{Order, Position, Subscription}; use std::{ - collections::{BTreeMap, VecDeque}, + collections::{BTreeMap, HashMap, VecDeque}, ops::{Bound, RangeBounds}, sync::Arc, sync::OnceLock, @@ -234,6 +234,10 @@ pub(crate) struct TrackState { // every cached group for each delivery. lookup: BTreeMap, + // Runs of `lookup` that cannot present, shared by every reader; see [`Quiet`]. + // Behind its own lock because walks record them under a read guard. + quiet: kio::Lock, + // Publisher-produced groups in arrival order as (sequence, stamp), walked by // subscriptions; an entry only resolves while its stamp matches the slot's. // Fetched backfill (`insert_group_request`) is deliberately absent: it is @@ -458,6 +462,48 @@ impl Slot { } } +/// Where walks for a group's successor stopped, so a run of groups that cannot present +/// (hidden, aborted, or finished without a frame) is walked once, not once for every +/// group judged against it. A publisher can make that run as long as the cache. +/// +/// Each entry maps a group a walk passed to where that walk stopped: no group strictly +/// between the two could present. A group stops presenting on its own, but only starts +/// by landing, so an entry holds until a group lands inside it. A stop itself is looked +/// at again, so a walk past one that stopped presenting records the further stop. +#[derive(Default)] +struct Quiet { + next: HashMap, + /// The highest recorded stop. A live group lands at or past it, keeping every entry. + end: u64, + /// Groups walked past in total, so tests can bound the work. + #[cfg(test)] + passed: usize, +} + +impl Quiet { + /// Point every group in `walked` at `stop`. Entries left by evicted groups are dropped + /// once they could outnumber the `cached` groups, so this stays bounded by the cache. + fn record(&mut self, walked: Vec, stop: u64, cached: usize) { + #[cfg(test)] + { + self.passed += walked.len(); + } + if self.next.len() >= cached.saturating_mul(2) { + self.next.clear(); + } + self.end = self.end.max(stop); + self.next.extend(walked.into_iter().map(|at| (at, stop))); + } + + /// A group landed at `sequence`: a run recorded across it no longer holds. + fn landed(&mut self, sequence: u64) { + if sequence < self.end { + self.next.clear(); + self.end = 0; + } + } +} + /// Heap the track keeps per cached group, excluding the group itself /// ([`group::CACHE_OVERHEAD`]). /// @@ -776,13 +822,38 @@ impl TrackState { /// The first servable group above `sequence` and below `cap` that could still /// present, stamped or not. + /// + /// Jumps over runs an earlier walk recorded in [`Self::quiet`], and records the groups + /// it walks past, so judging every group of a run costs one walk over it, not one each. fn successor(&self, sequence: u64, cap: Option) -> Option<&group::Producer> { - self.lookup - .range(sequence.checked_add(1)?..) - .map(|(_, slot)| slot) - .take_while(|slot| super::subscription::before_end(slot.group.sequence, cap)) - .find(|slot| slot.is_shown() && !slot.group.ended_empty()) - .map(|slot| &slot.group) + let mut walked = Vec::new(); + // The next sequence to look at, and where this walk stops once it breaks. + let mut from = sequence.checked_add(1)?; + let found = loop { + let Some((&at, slot)) = self.lookup.range(from..).next() else { + break None; + }; + if !super::subscription::before_end(at, cap) { + from = at; + break None; + } + if slot.is_shown() && !slot.group.ended_empty() { + from = at; + break Some(&slot.group); + } + // Not held across `ended_empty`, which takes the group's lock. + let jump = self.quiet.lock().next.get(&at).copied(); + let Some(next) = jump.or_else(|| at.checked_add(1)) else { + from = at; + break None; + }; + walked.push(at); + from = next; + }; + if !walked.is_empty() { + self.quiet.lock().record(walked, from, self.lookup.len()); + } + found } /// Whether the group at `sequence` has drifted further behind `edge` than `budget` @@ -1251,6 +1322,7 @@ impl TrackState { /// Content at `sequence` became servable: wake the reads it succeeds, and show an answer /// waiting on it. fn landed(&mut self, sequence: u64) { + self.quiet.lock().landed(sequence); let below = self .lookup .range(..sequence) @@ -10900,6 +10972,64 @@ mod test { assert_eq!(most, 6, "the floor and the gaps within the budget"); } + /// Write a finished one-frame group at `sequence`, presented at `millis`. + fn write_group_at(producer: &Producer, sequence: u64, millis: u64) { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(millis).unwrap(), &b"x"[..]) + .unwrap(); + group.finish().unwrap(); + } + + /// Judging every group of a run that finished without a frame walks the run once, + /// whether one poll writes it all off or each poll hands one out, and however many + /// readers judge it. Without the recorded runs, each group walks the rest of the run: + /// two million groups here. + #[test] + fn a_run_of_empty_groups_is_walked_once() { + const RUN: u64 = 1000; + const READERS: usize = 4; + // Stamped groups after the run, and how many groups each reader is handed. + for (after, handed) in [(&[50, 100][..], 2), (&[5][..], RUN as usize + 2)] { + let producer = track_producer("test", Info::timed()); + write_group_at(&producer, 0, 0); + for sequence in 1..=RUN { + write_empty_group(&producer, sequence); + } + for (sequence, millis) in (RUN + 1..).zip(after) { + write_group_at(&producer, sequence, *millis); + } + for _ in 0..READERS { + let mut subscriber = + producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + assert_eq!(drain_groups(&mut subscriber).len(), handed); + } + let passed = producer.state.read().quiet.lock().passed; + assert!(passed <= 2 * RUN as usize * READERS, "walked past {passed} groups"); + } + } + + /// A group landing inside a run an earlier walk passed bounds the groups below it, + /// rather than the walk's recorded stop. + #[test] + fn a_group_landing_inside_a_walked_run_bounds_the_groups_below() { + let producer = track_producer("test", Info::timed()); + write_group_at(&producer, 0, 0); + for sequence in 1..=3 { + write_empty_group(&producer, sequence); + } + write_group_at(&producer, 5, 50); + write_group_at(&producer, 6, 100); + let budget = Subscription::default().with_max_delay(Duration::from_millis(10)); + let mut first = producer.subscribe(budget.clone()); + assert_eq!(drain_groups(&mut first), [5, 6]); + + // Open without a frame, so everything below it could present until it does. + let _open = producer.create_group(group::Info { sequence: 4 }).unwrap(); + let mut second = producer.subscribe(budget); + assert_eq!(drain_groups(&mut second), [0, 1, 2, 3, 5, 6]); + } + /// An untimed track has nothing past the budget, so a sparse feed keeps every gap /// and still hands out every group. The hand-out cost stays flat over the retained /// gaps; `track_sparse_read` measures it. From db04cd13057d421b6b7124013cebc897a61a91bc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 11:01:31 -0700 Subject: [PATCH 12/21] fix(net): keep walked runs above a landing and drop them with the cache A landing inside a recorded run cleared the whole record, so revealing a pending run one group at a time and reading in between walked the rest of it again each time. A landing now drops only the entries below it, which are the only ones that can stop past it. Removing a group from the cache removes its entry, so the record never outlives the cache and is counted in the per-group overhead. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 106 +++++++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 27 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index bfafa119a75..02440cb8a42 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -23,7 +23,7 @@ use super::Cap; pub use super::subscription::{Order, Position, Subscription}; use std::{ - collections::{BTreeMap, HashMap, VecDeque}, + collections::{BTreeMap, VecDeque}, ops::{Bound, RangeBounds}, sync::Arc, sync::OnceLock, @@ -466,13 +466,14 @@ impl Slot { /// (hidden, aborted, or finished without a frame) is walked once, not once for every /// group judged against it. A publisher can make that run as long as the cache. /// -/// Each entry maps a group a walk passed to where that walk stopped: no group strictly -/// between the two could present. A group stops presenting on its own, but only starts -/// by landing, so an entry holds until a group lands inside it. A stop itself is looked -/// at again, so a walk past one that stopped presenting records the further stop. +/// Each entry maps a cached group a walk passed to where that walk stopped: no group +/// strictly between the two could present. A group stops presenting on its own, but only +/// starts by landing, so an entry holds until a group lands inside it. +/// A stop itself is looked at again, so a walk past one that stopped presenting records +/// the further stop. Removing a group from the cache removes its entry. #[derive(Default)] struct Quiet { - next: HashMap, + next: BTreeMap, /// The highest recorded stop. A live group lands at or past it, keeping every entry. end: u64, /// Groups walked past in total, so tests can bound the work. @@ -481,25 +482,21 @@ struct Quiet { } impl Quiet { - /// Point every group in `walked` at `stop`. Entries left by evicted groups are dropped - /// once they could outnumber the `cached` groups, so this stays bounded by the cache. - fn record(&mut self, walked: Vec, stop: u64, cached: usize) { + /// Point every group in `walked` at `stop`. + fn record(&mut self, walked: Vec, stop: u64) { #[cfg(test)] { self.passed += walked.len(); } - if self.next.len() >= cached.saturating_mul(2) { - self.next.clear(); - } self.end = self.end.max(stop); self.next.extend(walked.into_iter().map(|at| (at, stop))); } - /// A group landed at `sequence`: a run recorded across it no longer holds. + /// A group landed at `sequence`. Only an entry below it can stop past it, so those go + /// and the runs above it stay. fn landed(&mut self, sequence: u64) { if sequence < self.end { - self.next.clear(); - self.end = 0; + self.next = self.next.split_off(&sequence); } } } @@ -507,12 +504,13 @@ impl Quiet { /// Heap the track keeps per cached group, excluding the group itself /// ([`group::CACHE_OVERHEAD`]). /// -/// One [`Slot`] under its sequence in `lookup`, plus a hint in each of `arrival` and -/// `evict`. Doubled because both containers run half empty in the worst case: a -/// `BTreeMap` node sits between half and fully packed, and a `VecDeque` holds up to -/// twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; see it for why this is -/// derived rather than measured. -pub(crate) const CACHE_OVERHEAD: u64 = 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>()) as u64; +/// One [`Slot`] under its sequence in `lookup`, a hint in each of `arrival` and `evict`, +/// and at most one [`Quiet`] entry. Doubled because these containers run half empty in +/// the worst case: a `BTreeMap` node sits between half and fully packed, and a +/// `VecDeque` holds up to twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; +/// see it for why this is derived rather than measured. +pub(crate) const CACHE_OVERHEAD: u64 = + 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>() + size_of::<(u64, u64)>()) as u64; /// The registered subscriptions, aggregated by the producer. type Subscriptions = Vec>; @@ -851,7 +849,7 @@ impl TrackState { from = next; }; if !walked.is_empty() { - self.quiet.lock().record(walked, from, self.lookup.len()); + self.quiet.lock().record(walked, from); } found } @@ -1103,7 +1101,7 @@ impl TrackState { // Already aborted: the frames are gone, reclaim the slot so a // later fetch can serve the sequence again. if slot.is_aborted() { - self.lookup.remove(&sequence); + self.remove_slot(sequence); continue; } if self.protects(sequence) || !slot.is_expired(&scan) { @@ -1118,7 +1116,7 @@ impl TrackState { // Take the group out of the cache and abort it, so any consumer // still reading surfaces `Error::Old` instead of blocking forever // on a frame that will never arrive. - self.lookup.remove(&sequence).unwrap().abort(Error::Old); + self.remove_slot(sequence).unwrap().abort(Error::Old); } } @@ -1237,12 +1235,18 @@ impl TrackState { /// A group that starts above `frame_start` is also replaceable: it cannot answer a /// request from there, so the wider producer takes the slot. Readers already /// draining the old one keep their own handle. + /// Take `sequence` out of the cache, with any run recorded from it. + fn remove_slot(&mut self, sequence: u64) -> Option { + self.quiet.lock().next.remove(&sequence); + self.lookup.remove(&sequence) + } + fn claim_sequence(&mut self, sequence: u64, frame_start: u64) -> Result<()> { // Either copy that can still answer from `frame_start` takes the sequence. if self.covering_group(sequence, frame_start).is_some() { return Err(Error::Duplicate); } - self.lookup.remove(&sequence); + self.remove_slot(sequence); Ok(()) } @@ -1434,7 +1438,7 @@ impl TrackState { } if slot.is_aborted() { // Aborted upstream: the frames are already gone, reclaim the slot. - self.lookup.remove(&sequence); + self.remove_slot(sequence); continue; } if self.protects(sequence) { @@ -1461,7 +1465,7 @@ impl TrackState { self.debt -= size; paid = paid.saturating_add(size); - self.lookup.remove(&sequence).unwrap().abort(Error::Evicted); + self.remove_slot(sequence).unwrap().abort(Error::Evicted); } } @@ -11030,6 +11034,54 @@ mod test { assert_eq!(drain_groups(&mut second), [0, 1, 2, 3, 5, 6]); } + /// A group landing inside a walked run keeps the part of it above: a pending run + /// revealed one group at a time, read in between, is still walked once. + #[test] + fn revealing_a_run_one_group_at_a_time_walks_it_once() { + const RUN: u64 = 1000; + let producer = track_producer("test", Info::timed()); + write_group_at(&producer, 0, 0); + let pending: Vec<_> = (1..=RUN) + .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) + .collect(); + write_group_at(&producer, RUN + 1, 50); + write_group_at(&producer, RUN + 2, 100); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + assert_eq!(drain_groups(&mut subscriber), [RUN + 1, RUN + 2]); + for group in pending { + group.finish().unwrap(); + producer.reveal_group(&group); + assert!( + drain_groups(&mut subscriber).is_empty(), + "group {} is past the budget", + group.sequence + ); + } + let passed = producer.state.read().quiet.lock().passed; + assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); + } + + /// What was recorded for a walked run goes with the run when the cache drops it, even + /// with no read after. + #[test] + fn expiring_a_walked_run_forgets_it() { + let pool = cache::Pool::new(cache::Config::default().with_expiry(Duration::from_secs(1))); + let producer = track_producer_pooled("test", pool.clone()); + write_group_at(&producer, 0, 0); + for sequence in 1..=100 { + write_empty_group(&producer, sequence); + } + write_group_at(&producer, 101, 50); + write_group_at(&producer, 102, 100); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + assert_eq!(drain_groups(&mut subscriber), [101, 102]); + assert_eq!(producer.state.read().quiet.lock().next.len(), 100); + + elapse(&producer, Duration::from_secs(2)); + pool.sweep(); + assert!(producer.state.read().quiet.lock().next.is_empty()); + } + /// An untimed track has nothing past the budget, so a sparse feed keeps every gap /// and still hands out every group. The hand-out cost stays flat over the retained /// gaps; `track_sparse_read` measures it. From d05525cf15d64bf6dc27cfbc273a30464aae72ea Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 11:13:51 -0700 Subject: [PATCH 13/21] fix(net): keep a walked run across an empty reveal and a track teardown A group revealed after finishing without a frame cannot present, so it no longer drops the entries below it. Revealing a pending run newest first made every later judge of the group below walk the run again. Dropping the open groups on an abrupt end now drops their entries too. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 65 +++++++++++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 7 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 02440cb8a42..afb8c2371ef 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -468,7 +468,7 @@ impl Slot { /// /// Each entry maps a cached group a walk passed to where that walk stopped: no group /// strictly between the two could present. A group stops presenting on its own, but only -/// starts by landing, so an entry holds until a group lands inside it. +/// starts by landing, so an entry holds until a group that could present lands inside it. /// A stop itself is looked at again, so a walk past one that stopped presenting records /// the further stop. Removing a group from the cache removes its entry. #[derive(Default)] @@ -492,12 +492,10 @@ impl Quiet { self.next.extend(walked.into_iter().map(|at| (at, stop))); } - /// A group landed at `sequence`. Only an entry below it can stop past it, so those go - /// and the runs above it stay. + /// A group that could present landed at `sequence`. Only an entry below it can stop + /// past it, so those go and the runs above it stay. fn landed(&mut self, sequence: u64) { - if sequence < self.end { - self.next = self.next.split_off(&sequence); - } + self.next = self.next.split_off(&sequence); } } @@ -1179,6 +1177,11 @@ impl TrackState { fn drop_open_groups(&mut self) { self.lookup .retain(|_, slot| slot.copies().any(group::Producer::is_finished)); + let lookup = &self.lookup; + self.quiet + .lock() + .next + .retain(|sequence, _| lookup.contains_key(sequence)); } /// Attach the publisher's immutable metadata without replacing it with local cache policy. @@ -1326,7 +1329,12 @@ impl TrackState { /// Content at `sequence` became servable: wake the reads it succeeds, and show an answer /// waiting on it. fn landed(&mut self, sequence: u64) { - self.quiet.lock().landed(sequence); + // A group revealed after finishing without a frame still cannot present. Read + // apart from the group's lock, which never nests under this one. + let end = self.quiet.lock().end; + if sequence < end && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { + self.quiet.lock().landed(sequence); + } let below = self .lookup .range(..sequence) @@ -11061,6 +11069,49 @@ mod test { assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); } + /// A group revealed after finishing without a frame still cannot present, so it keeps + /// the run recorded across it: judging the group below the run after each of its groups + /// is revealed, newest first, walks the run once. + #[test] + fn revealing_empty_groups_newest_first_keeps_the_walked_run() { + const RUN: u64 = 1000; + let producer = track_producer("test", Info::timed()); + write_group_at(&producer, 0, 0); + let pending: Vec<_> = (1..=RUN) + .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) + .collect(); + write_group_at(&producer, RUN + 1, 50); + let successor = || producer.state.read().successor(0, None).map(|group| group.sequence); + assert_eq!(successor(), Some(RUN + 1)); + for group in pending.into_iter().rev() { + group.finish().unwrap(); + producer.reveal_group(&group); + assert_eq!(successor(), Some(RUN + 1)); + } + let passed = producer.state.read().quiet.lock().passed; + assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); + } + + /// Open groups dropped together when a track ends abruptly take their recorded runs. + #[test] + fn dropping_open_groups_forgets_their_runs() { + let producer = track_producer("test", Info::timed()); + let consumer = producer.consume(); + write_group_at(&producer, 0, 0); + let _pending: Vec<_> = (1..=10) + .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) + .collect(); + write_group_at(&producer, 11, 50); + assert_eq!( + producer.state.read().successor(0, None).map(|group| group.sequence), + Some(11) + ); + assert_eq!(producer.state.read().quiet.lock().next.len(), 10); + + producer.abort(Error::Cancel).unwrap(); + assert!(consumer.state.read().quiet.lock().next.is_empty()); + } + /// What was recorded for a walked run goes with the run when the cache drops it, even /// with no read after. #[test] From 9a4067eb2ad5de5d654e2b34dba7dfc07e3da440 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 11:40:11 -0700 Subject: [PATCH 14/21] fix(net): record quiet runs as disjoint intervals split on a landing Dropping every entry below a landing also dropped runs that stopped short of it, so landings below an old stop made each judge re-walk the run. The record is now a map of disjoint runs: a walk merges the run it passed, and a group that could present landing inside a run splits it there in O(log n). Runs holding no cached group are dropped once they could outnumber the cache, which keeps the record within two runs per group. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 198 +++++++++++++++++++++++----------- 1 file changed, 135 insertions(+), 63 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index afb8c2371ef..b27b1f6574c 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -24,7 +24,7 @@ pub use super::subscription::{Order, Position, Subscription}; use std::{ collections::{BTreeMap, VecDeque}, - ops::{Bound, RangeBounds}, + ops::{Bound, Range, RangeBounds}, sync::Arc, sync::OnceLock, sync::atomic::{AtomicBool, AtomicU64, Ordering}, @@ -466,36 +466,80 @@ impl Slot { /// (hidden, aborted, or finished without a frame) is walked once, not once for every /// group judged against it. A publisher can make that run as long as the cache. /// -/// Each entry maps a cached group a walk passed to where that walk stopped: no group -/// strictly between the two could present. A group stops presenting on its own, but only -/// starts by landing, so an entry holds until a group that could present lands inside it. -/// A stop itself is looked at again, so a walk past one that stopped presenting records -/// the further stop. Removing a group from the cache removes its entry. +/// Disjoint runs of sequences, each ending where a walk stopped: no group in a run could +/// present. A group stops presenting on its own, but only starts by landing, so a run +/// holds until a group that could present lands inside it, which splits it there. The +/// stop is outside its run and looked at again, so a walk past one that stopped +/// presenting records the longer run. #[derive(Default)] struct Quiet { - next: BTreeMap, - /// The highest recorded stop. A live group lands at or past it, keeping every entry. - end: u64, + /// Each run's first sequence, to its stop. + runs: BTreeMap, /// Groups walked past in total, so tests can bound the work. #[cfg(test)] passed: usize, } impl Quiet { - /// Point every group in `walked` at `stop`. - fn record(&mut self, walked: Vec, stop: u64) { + /// A walk passes `sequence`, which cannot present: where its run stops, if one holds it. + fn pass(&mut self, sequence: u64) -> Option { #[cfg(test)] { - self.passed += walked.len(); + self.passed += 1; } - self.end = self.end.max(stop); - self.next.extend(walked.into_iter().map(|at| (at, stop))); + self.stop(sequence) } - /// A group that could present landed at `sequence`. Only an entry below it can stop - /// past it, so those go and the runs above it stay. + /// Where the run holding `sequence` stops, if one does. + fn stop(&self, sequence: u64) -> Option { + let (_, &stop) = self.runs.range(..=sequence).next_back()?; + (stop > sequence).then_some(stop) + } + + /// No group in `run` could present: record it, merged with the runs it overlaps. + fn record(&mut self, run: Range) { + let mut start = run.start; + let mut stop = run.end; + if let Some((&first, &end)) = self.runs.range(..run.start).next_back() + && end > run.start + { + start = first; + stop = stop.max(end); + } + while let Some((&first, &end)) = self.runs.range(run.start..stop).next() { + self.runs.remove(&first); + stop = stop.max(end); + } + self.runs.insert(start, stop); + } + + /// A group that could present landed at `sequence`: split the run holding it. fn landed(&mut self, sequence: u64) { - self.next = self.next.split_off(&sequence); + let Some((&first, &stop)) = self.runs.range(..=sequence).next_back() else { + return; + }; + if stop <= sequence { + return; + } + if first < sequence { + self.runs.insert(first, sequence); + } else { + self.runs.remove(&first); + } + if let Some(after) = sequence.checked_add(1) + && after < stop + { + self.runs.insert(after, stop); + } + } + + /// Drop the runs holding no `cached` group once they could outnumber the cache. Runs are + /// disjoint, so those left number at most the cache, and this stays within twice it. + fn prune(&mut self, cached: &BTreeMap) { + if self.runs.len() > 2 * cached.len() { + self.runs + .retain(|&first, &mut stop| cached.range(first..stop).next().is_some()); + } } } @@ -503,12 +547,12 @@ impl Quiet { /// ([`group::CACHE_OVERHEAD`]). /// /// One [`Slot`] under its sequence in `lookup`, a hint in each of `arrival` and `evict`, -/// and at most one [`Quiet`] entry. Doubled because these containers run half empty in +/// and at most two [`Quiet`] runs. Doubled because these containers run half empty in /// the worst case: a `BTreeMap` node sits between half and fully packed, and a /// `VecDeque` holds up to twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; /// see it for why this is derived rather than measured. pub(crate) const CACHE_OVERHEAD: u64 = - 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>() + size_of::<(u64, u64)>()) as u64; + 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>() + 2 * size_of::<(u64, u64)>()) as u64; /// The registered subscriptions, aggregated by the producer. type Subscriptions = Vec>; @@ -819,10 +863,11 @@ impl TrackState { /// The first servable group above `sequence` and below `cap` that could still /// present, stamped or not. /// - /// Jumps over runs an earlier walk recorded in [`Self::quiet`], and records the groups - /// it walks past, so judging every group of a run costs one walk over it, not one each. + /// Jumps over runs an earlier walk recorded in [`Self::quiet`], and records the run it + /// walks past, so judging every group of a run costs one walk over it, not one each. fn successor(&self, sequence: u64, cap: Option) -> Option<&group::Producer> { - let mut walked = Vec::new(); + // The first group this walk passed. + let mut passed = None; // The next sequence to look at, and where this walk stops once it breaks. let mut from = sequence.checked_add(1)?; let found = loop { @@ -837,17 +882,17 @@ impl TrackState { from = at; break Some(&slot.group); } + passed.get_or_insert(at); // Not held across `ended_empty`, which takes the group's lock. - let jump = self.quiet.lock().next.get(&at).copied(); + let jump = self.quiet.lock().pass(at); let Some(next) = jump.or_else(|| at.checked_add(1)) else { from = at; break None; }; - walked.push(at); from = next; }; - if !walked.is_empty() { - self.quiet.lock().record(walked, from); + if let Some(passed) = passed { + self.quiet.lock().record(passed..from); } found } @@ -1177,11 +1222,7 @@ impl TrackState { fn drop_open_groups(&mut self) { self.lookup .retain(|_, slot| slot.copies().any(group::Producer::is_finished)); - let lookup = &self.lookup; - self.quiet - .lock() - .next - .retain(|sequence, _| lookup.contains_key(sequence)); + self.quiet.lock().prune(&self.lookup); } /// Attach the publisher's immutable metadata without replacing it with local cache policy. @@ -1238,10 +1279,11 @@ impl TrackState { /// A group that starts above `frame_start` is also replaceable: it cannot answer a /// request from there, so the wider producer takes the slot. Readers already /// draining the old one keep their own handle. - /// Take `sequence` out of the cache, with any run recorded from it. + /// Take `sequence` out of the cache, keeping the runs recorded over it within bounds. fn remove_slot(&mut self, sequence: u64) -> Option { - self.quiet.lock().next.remove(&sequence); - self.lookup.remove(&sequence) + let slot = self.lookup.remove(&sequence); + self.quiet.lock().prune(&self.lookup); + slot } fn claim_sequence(&mut self, sequence: u64, frame_start: u64) -> Result<()> { @@ -1331,8 +1373,8 @@ impl TrackState { fn landed(&mut self, sequence: u64) { // A group revealed after finishing without a frame still cannot present. Read // apart from the group's lock, which never nests under this one. - let end = self.quiet.lock().end; - if sequence < end && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { + let inside = self.quiet.lock().stop(sequence).is_some(); + if inside && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { self.quiet.lock().landed(sequence); } let below = self @@ -11092,45 +11134,75 @@ mod test { assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); } - /// Open groups dropped together when a track ends abruptly take their recorded runs. + /// Groups landing one after another inside a walked run split it there, keeping the + /// parts on either side, so judging the group below after each landing stays cheap. #[test] - fn dropping_open_groups_forgets_their_runs() { + fn landings_inside_a_walked_run_keep_the_rest_of_it() { + const RUN: u64 = 1000; let producer = track_producer("test", Info::timed()); - let consumer = producer.consume(); write_group_at(&producer, 0, 0); - let _pending: Vec<_> = (1..=10) - .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) - .collect(); - write_group_at(&producer, 11, 50); - assert_eq!( - producer.state.read().successor(0, None).map(|group| group.sequence), - Some(11) - ); - assert_eq!(producer.state.read().quiet.lock().next.len(), 10); + for sequence in 1..=RUN { + write_empty_group(&producer, sequence); + } + write_group_at(&producer, 3 * RUN, 3 * RUN); + let successor = || producer.state.read().successor(0, None).map(|group| group.sequence); + assert_eq!(successor(), Some(3 * RUN)); + for sequence in RUN + 1..=2 * RUN { + write_group_at(&producer, sequence, sequence); + assert_eq!(successor(), Some(RUN + 1)); + } + let passed = producer.state.read().quiet.lock().passed; + assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); + } - producer.abort(Error::Cancel).unwrap(); - assert!(consumer.state.read().quiet.lock().next.is_empty()); + /// Walk a run of one group below each of `stamped`, so the track records one run each. + fn record_runs(producer: &Producer, stamped: impl Iterator) { + let state = producer.state.read(); + for sequence in stamped { + assert_eq!( + state.successor(sequence, None).map(|group| group.sequence), + Some(sequence + 2) + ); + } } - /// What was recorded for a walked run goes with the run when the cache drops it, even - /// with no read after. + /// Runs recorded over groups that left the cache are dropped once they could outnumber + /// what is left, so the record stays within the overhead each cached group pays. #[test] - fn expiring_a_walked_run_forgets_it() { + fn the_record_shrinks_with_the_cache() { + // Expiry drops groups one at a time. let pool = cache::Pool::new(cache::Config::default().with_expiry(Duration::from_secs(1))); let producer = track_producer_pooled("test", pool.clone()); - write_group_at(&producer, 0, 0); - for sequence in 1..=100 { - write_empty_group(&producer, sequence); + for k in 0..100 { + write_group_at(&producer, 2 * k, k); + write_empty_group(&producer, 2 * k + 1); } - write_group_at(&producer, 101, 50); - write_group_at(&producer, 102, 100); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - assert_eq!(drain_groups(&mut subscriber), [101, 102]); - assert_eq!(producer.state.read().quiet.lock().next.len(), 100); - + write_group_at(&producer, 200, 100); + record_runs(&producer, (0..100).map(|k| 2 * k)); + assert_eq!(producer.state.read().quiet.lock().runs.len(), 100); elapse(&producer, Duration::from_secs(2)); pool.sweep(); - assert!(producer.state.read().quiet.lock().next.is_empty()); + let state = producer.state.read(); + assert!(state.quiet.lock().runs.len() <= 2 * state.lookup.len()); + + // Dropping the open groups of a track that ends abruptly drops many at once. + let producer = track_producer("test", Info::timed()); + let consumer = producer.consume(); + let mut open = Vec::new(); + for k in 0..100 { + open.push(producer.create_group(group::Info { sequence: 2 * k }).unwrap()); + open.push(producer.receive_group(group::Info { sequence: 2 * k + 1 }).unwrap()); + } + write_group_at(&producer, 200, 100); + for (k, group) in open.iter_mut().step_by(2).enumerate() { + group + .write_frame(Timestamp::from_millis(k as u64).unwrap(), &b"x"[..]) + .unwrap(); + } + record_runs(&producer, (0..100).map(|k| 2 * k)); + producer.abort(Error::Cancel).unwrap(); + let state = consumer.state.read(); + assert!(state.quiet.lock().runs.len() <= 2 * state.lookup.len()); } /// An untimed track has nothing past the budget, so a sparse feed keeps every gap From 39bc37d54fe3a2f196c8cf890b324ee7484eb87e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 11:49:21 -0700 Subject: [PATCH 15/21] fix(net): judge every forgotten gap against one edge, and prune on record Advancing the delivered floor resolved the drift edge once per gap it forgot, and the edge scans every trailing unstamped group, so forgetting N gaps behind a run of empty groups cost N scans. One hand-out now resolves the edge once for the whole loop. The quiet record was pruned only when the cache shrank, so a record grown after a shrink could exceed its two runs per cached group. Recording and splitting a run now prune too. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 10 +++--- rs/moq-net/src/model/track.rs | 64 +++++++++++++++++++++++----------- 2 files changed, 49 insertions(+), 25 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index e8a60e26dd8..c8f195f3a0d 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -339,15 +339,17 @@ impl Subscriber { self.reader().poll_recv_datagram(delivered, waiter) } - /// Whether a group at `sequence` is past the reader's budget, judged on the newest - /// copy that answered; see [`track::Subscriber::is_past_budget`]. - pub(crate) fn is_past_budget(&self, sequence: u64) -> bool { + /// Forget the gaps in `delivered` below each group past the reader's budget, judged on + /// the newest copy that answered; see [`track::Subscriber::forget_past_budget`]. + pub(crate) fn forget_past_budget(&self, delivered: &mut Delivered) { let reader = self.reader(); let mut ready = reader.copies.iter().rev().filter_map(|copy| match ©.sub { Sub::Ready(sub) => Some(sub), Sub::Pending(_) => None, }); - ready.next().is_some_and(|sub| sub.is_past_budget(sequence)) + if let Some(sub) = ready.next() { + sub.forget_past_budget(delivered); + } } /// Poll for the track's declared exclusive end. diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index b27b1f6574c..2dc142c0b0b 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -497,7 +497,7 @@ impl Quiet { } /// No group in `run` could present: record it, merged with the runs it overlaps. - fn record(&mut self, run: Range) { + fn record(&mut self, run: Range, cached: &BTreeMap) { let mut start = run.start; let mut stop = run.end; if let Some((&first, &end)) = self.runs.range(..run.start).next_back() @@ -511,10 +511,11 @@ impl Quiet { stop = stop.max(end); } self.runs.insert(start, stop); + self.prune(cached); } /// A group that could present landed at `sequence`: split the run holding it. - fn landed(&mut self, sequence: u64) { + fn landed(&mut self, sequence: u64, cached: &BTreeMap) { let Some((&first, &stop)) = self.runs.range(..=sequence).next_back() else { return; }; @@ -531,9 +532,10 @@ impl Quiet { { self.runs.insert(after, stop); } + self.prune(cached); } - /// Drop the runs holding no `cached` group once they could outnumber the cache. Runs are + /// Drop the runs holding no `cached` group once they outnumber twice the cache. Runs are /// disjoint, so those left number at most the cache, and this stays within twice it. fn prune(&mut self, cached: &BTreeMap) { if self.runs.len() > 2 * cached.len() { @@ -892,7 +894,7 @@ impl TrackState { from = next; }; if let Some(passed) = passed { - self.quiet.lock().record(passed..from); + self.quiet.lock().record(passed..from, &self.lookup); } found } @@ -1375,7 +1377,7 @@ impl TrackState { // apart from the group's lock, which never nests under this one. let inside = self.quiet.lock().stop(sequence).is_some(); if inside && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { - self.quiet.lock().landed(sequence); + self.quiet.lock().landed(sequence, &self.lookup); } let below = self .lookup @@ -4080,13 +4082,23 @@ impl Cursor { }) } - /// Whether a group at `sequence` is past this cursor's budget now, whether this track - /// holds it or not: one landing there would be written off. - fn is_past_budget(&self, sequence: u64) -> bool { + /// Forget the gaps in `delivered` below each group past this cursor's budget now, + /// whether this track holds it or not: one landing there would be written off. + /// + /// One edge for every group judged, so forgetting N gaps never resolves it N times. + fn forget_past_budget(&self, delivered: &mut Delivered) { + if delivered.above_gap().is_none() { + return; + } let max_delay = self.subscription.read().max_delay; let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); - state.drifted(sequence, &state.drift_edge(self.end_sequence), budget) + let edge = state.drift_edge(self.end_sequence); + while let Some(oldest) = delivered.above_gap() + && state.drifted(oldest, &edge, budget) + { + delivered.forget_below(oldest); + } } fn with_expiry(&self, group: group::Consumer) -> group::Consumer { @@ -4412,19 +4424,17 @@ impl Subscriber { self.delivered.forget_below(sequence); } self.delivered.insert(sequence); - while let Some(oldest) = self.delivered.above_gap() - && self.is_past_budget(oldest) - { - self.delivered.forget_below(oldest); - } + let mut delivered = std::mem::take(&mut self.delivered); + self.forget_past_budget(&mut delivered); + self.delivered = delivered; } - /// Whether a group at `sequence` is past this subscriber's budget now: one landing - /// there would be written off. - pub(crate) fn is_past_budget(&self, sequence: u64) -> bool { + /// Forget the gaps in `delivered` below each group past this subscriber's budget now: + /// one landing there would be written off. + pub(crate) fn forget_past_budget(&self, delivered: &mut Delivered) { match &self.inner { - Inner::Plain(cursor) => cursor.is_past_budget(sequence), - Inner::Resume(resume, _) => resume.is_past_budget(sequence), + Inner::Plain(cursor) => cursor.forget_past_budget(delivered), + Inner::Resume(resume, _) => resume.forget_past_budget(delivered), } } @@ -11181,9 +11191,21 @@ mod test { record_runs(&producer, (0..100).map(|k| 2 * k)); assert_eq!(producer.state.read().quiet.lock().runs.len(), 100); elapse(&producer, Duration::from_secs(2)); + // Fresh groups outlive the sweep, and are walked after it. + for k in 101..126 { + write_group_at(&producer, 2 * k, k); + write_empty_group(&producer, 2 * k + 1); + } + write_group_at(&producer, 252, 126); + let within = || { + let state = producer.state.read(); + let runs = state.quiet.lock().runs.len(); + runs <= 2 * state.lookup.len() + }; pool.sweep(); - let state = producer.state.read(); - assert!(state.quiet.lock().runs.len() <= 2 * state.lookup.len()); + assert!(within()); + record_runs(&producer, (101..126).map(|k| 2 * k)); + assert!(within()); // Dropping the open groups of a track that ends abruptly drops many at once. let producer = track_producer("test", Info::timed()); From f719f423c3e3154bf23689b6d202afd00c21f69b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 12:00:20 -0700 Subject: [PATCH 16/21] fix(net): wake the read below a run of empty groups when a group lands past it A landing woke reads from the nearest shown group below it, but a group that finished without a frame is shown and cannot present, so the group below it kept a parked read that the landing had just made stale. The landing now wakes from the nearest group that could present, found by a reverse walk that jumps over the quiet runs and records the one it passes. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 79 +++++++++++++++++++++++++++++++---- 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 2dc142c0b0b..5a6ccc0bf70 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -492,8 +492,13 @@ impl Quiet { /// Where the run holding `sequence` stops, if one does. fn stop(&self, sequence: u64) -> Option { - let (_, &stop) = self.runs.range(..=sequence).next_back()?; - (stop > sequence).then_some(stop) + Some(self.holding(sequence)?.end) + } + + /// The run holding `sequence`, if one does. + fn holding(&self, sequence: u64) -> Option> { + let (&first, &stop) = self.runs.range(..=sequence).next_back()?; + (stop > sequence).then_some(first..stop) } /// No group in `run` could present: record it, merged with the runs it overlaps. @@ -1379,17 +1384,39 @@ impl TrackState { if inside && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { self.quiet.lock().landed(sequence, &self.lookup); } - let below = self - .lookup - .range(..sequence) - .rev() - .find(|(_, slot)| slot.is_shown()) - .map(|(below, _)| *below); + let below = self.predecessor(sequence); let oldest = self.lookup.first_key_value().map_or(sequence, |(oldest, _)| *oldest); self.cache.wakes().landed(below, sequence, oldest); self.shown(sequence); } + /// The nearest group below `sequence` that could present: the one whose successor + /// a landing at `sequence` becomes, with every group between. + /// + /// [`Self::successor`] in reverse, so it jumps over the runs in [`Self::quiet`] and + /// records the one it walks, which `sequence` stops. + fn predecessor(&self, sequence: u64) -> Option { + // The lowest group this walk passed, and the bound of the next look. + let mut passed = None; + let mut to = sequence; + let found = loop { + let Some((&at, slot)) = self.lookup.range(..to).next_back() else { + break None; + }; + if slot.is_shown() && !slot.group.ended_empty() { + break Some(at); + } + // Not held across `ended_empty`, which takes the group's lock. + let run = self.quiet.lock().holding(at); + to = run.map_or(at, |run| run.start); + passed = Some(to); + }; + if let Some(passed) = passed { + self.quiet.lock().record(passed..sequence, &self.lookup); + } + found + } + /// Content at `sequence` became readable: an answer waiting on it now shows. fn shown(&mut self, sequence: u64) { if let Feed::Answered { from } = self.feed @@ -7020,6 +7047,42 @@ mod test { assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); } + /// A group landing past a finished empty one becomes the successor of the group below + /// that, so it wakes a read parked there, which then finds the group stale. + #[test] + fn a_landing_past_an_empty_group_wakes_the_read_below_it() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + let mut head = producer.create_group(group::Info { sequence: 0 }).unwrap(); + head.write_frame(Timestamp::ZERO, bytes::Bytes::from_static(b"head")) + .unwrap(); + write_empty_group(&producer, 1); + write_group_at(&producer, 3, 100); + write_group_at(&producer, 4, 105); + + let mut held = subscriber + .recv_group() + .now_or_never() + .unwrap() + .unwrap() + .expect("head group"); + assert_eq!(held.sequence, 0); + assert!(held.read_frame().now_or_never().unwrap().unwrap().is_some()); + let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); + let mut cx = std::task::Context::from_waker(&waker); + let mut next = std::pin::pin!(held.read_frame()); + assert!(next.as_mut().poll(&mut cx).is_pending(), "group 3 bounds the head"); + + let mut late = producer.create_group(group::Info { sequence: 2 }).unwrap(); + assert!(woken.load(Ordering::SeqCst), "group 2 is the head's successor now"); + assert!(next.as_mut().poll(&mut cx).is_pending(), "group 2 has no frame yet"); + late.write_frame(Timestamp::from_millis(50).unwrap(), bytes::Bytes::from_static(b"x")) + .unwrap(); + let result = next.as_mut().poll(&mut cx); + assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); + } + /// A route can declare the end, then land every group's header before its first frame. /// Those groups are withheld until their frames land, so the end is not reached before /// they show: a reader ending at the boundary would never see them. From 490408facee09933e3d7cda5cab3ad041e58fa5a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 12:17:53 -0700 Subject: [PATCH 17/21] fix(net): leave reads below an empty reveal parked A group revealed after finishing without a frame cannot present, so it succeeds no read. Waking every read parked below it on each such reveal cost N held reads times N empty reveals; it now wakes only a held copy of its own sequence. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 50 ++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 1794414803a..0314e5d2821 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -1379,13 +1379,18 @@ impl TrackState { /// Content at `sequence` became servable: wake the reads it succeeds, and show an answer /// waiting on it. fn landed(&mut self, sequence: u64) { - // A group revealed after finishing without a frame still cannot present. Read - // apart from the group's lock, which never nests under this one. - let inside = self.quiet.lock().stop(sequence).is_some(); - if inside && self.lookup.get(&sequence).is_some_and(|slot| !slot.group.ended_empty()) { - self.quiet.lock().landed(sequence, &self.lookup); - } - let below = self.predecessor(sequence); + // A group revealed after finishing without a frame still cannot present, so it + // succeeds no read and only continues a held copy of itself. + let empty = self.lookup.get(&sequence).is_some_and(|slot| slot.group.ended_empty()); + let below = if empty { + Some(sequence) + } else { + // Read apart from the group's lock, which never nests under this one. + if self.quiet.lock().stop(sequence).is_some() { + self.quiet.lock().landed(sequence, &self.lookup); + } + self.predecessor(sequence) + }; let oldest = self.lookup.first_key_value().map_or(sequence, |(oldest, _)| *oldest); self.cache.wakes().landed(below, sequence, oldest); self.shown(sequence); @@ -7087,6 +7092,37 @@ mod test { assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); } + /// A group revealed after finishing without a frame succeeds no read, so it leaves a + /// read parked below it asleep: waking every such read per empty reveal is quadratic. + #[test] + fn an_empty_reveal_leaves_the_read_below_it_parked() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); + let mut head = producer.create_group(group::Info { sequence: 0 }).unwrap(); + head.write_frame(Timestamp::ZERO, bytes::Bytes::from_static(b"head")) + .unwrap(); + let empty = producer.receive_group(group::Info { sequence: 1 }).unwrap(); + write_group_at(&producer, 3, 100); + + let mut held = subscriber + .recv_group() + .now_or_never() + .unwrap() + .unwrap() + .expect("head group"); + assert_eq!(held.sequence, 0); + assert!(held.read_frame().now_or_never().unwrap().unwrap().is_some()); + let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); + let mut cx = std::task::Context::from_waker(&waker); + let mut next = std::pin::pin!(held.read_frame()); + assert!(next.as_mut().poll(&mut cx).is_pending(), "group 3 bounds the head"); + + empty.finish().unwrap(); + producer.reveal_group(&empty); + assert!(!woken.load(Ordering::SeqCst), "the empty group moves no edge"); + } + /// A route can declare the end, then land every group's header before its first frame. /// Those groups are withheld until their frames land, so the end is not reached before /// they show: a reader ending at the boundary would never see them. From 040256ba50fc8b346ebd592055bd84a36f2fd3e6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 13:35:43 -0700 Subject: [PATCH 18/21] fix(net): hand empty groups out like any other, without a run record A finished empty successor no longer gets skipped in reach. Skipping it made judging a run of N empty groups O(N^2), and the per-track record that kept it linear was heavy machinery for a case the wall-clock term in quest/m1/cache-max-age.md covers, along with groups that have no frame yet. Each empty group is now handed out in O(1) and the reader drops it. Until that quest lands, a group followed by an empty one can't prove drift, as on main. The drift rule, the landing wakes and the cache overhead match main again; the per-reader dedup, forget_past_budget and the untimed drift edge stay. track_empty_run now shows both cases flat over run length and readers. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 19 +- rs/moq-net/src/model/group.rs | 6 - rs/moq-net/src/model/track.rs | 579 ++-------------------------------- 3 files changed, 40 insertions(+), 564 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 6d480b4f48b..ffdda1d710e 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -27,9 +27,9 @@ //! holding a fixed number of gaps the reader still remembers: each hand-out must cost the //! same however many gaps it holds, so a walk over them shows up as a slope. //! -//! `track_empty_run` drains a run of groups that finished without a frame, written off in -//! one poll or handed out a group per poll, across readers: judging each group must not -//! walk the rest of the run, so a per-group walk shows up as a slope over it. +//! `track_empty_run` drains a run of groups that finished without a frame across readers, +//! with the groups after it past the budget or within it: each empty group is handed out +//! like any other, so a walk over the rest of the run per group shows up as a slope over it. //! //! Run with `cargo bench -p moq-net --bench track`. @@ -587,8 +587,8 @@ struct EmptyRun { impl EmptyRun { /// `stale` stamps the groups after the run 50 ms and 100 ms in, so a 10 ms budget writes - /// off group 0 and the run in one poll. Otherwise one group 5 ms in follows the run and - /// keeps it within the budget, so each poll hands out one group of it. + /// off the run's last group, the only one they bound. Otherwise one group 5 ms in follows + /// the run and keeps all of it within the budget. fn new(run: u64, readers: usize, stale: bool) -> Self { let broadcast = broadcast::Info::default().produce(); let track = broadcast.create_track("bench", timed()).unwrap(); @@ -614,7 +614,8 @@ impl EmptyRun { readers: (0..readers).map(|_| track.subscribe(budget.clone())).collect(), _broadcast: broadcast, _track: track, - handed: if stale { 2 } else { run as usize + 2 }, + // The run's last group is written off when stale, but two groups follow it. + handed: run as usize + 2, } } @@ -631,9 +632,9 @@ impl EmptyRun { } /// Readers draining a run of groups that finished without a frame, swept over the run and -/// the readers. `stale` writes the run off in one poll; `fresh` hands it out a group per -/// poll. Either way a reader's cost must stay linear in the run, so a walk over the rest of -/// the run per group shows up as a slope over it. +/// the readers, with the groups after it past the budget (`stale`) or within it (`fresh`). +/// A reader's cost must stay linear in the run, so a walk over the rest of the run per +/// group shows up as a slope over it. fn bench_empty_run(c: &mut Criterion) { let mut group = c.benchmark_group("track_empty_run"); group.sample_size(10); diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index b9c356c5851..11783daa92d 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -959,12 +959,6 @@ impl Producer { self.state.read().started() } - /// Whether the group finished without a frame, so it never presents anything. - pub(crate) fn ended_empty(&self) -> bool { - let state = self.state.read(); - state.fin.is_some() && matches!(state.timeline, Timeline::Empty) - } - /// Where the group starts in presentation time: its first frame's timestamp, or /// `None` while no frame has been opened or when that frame is untimed. /// diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 0314e5d2821..a76984f87e6 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -24,7 +24,7 @@ pub use super::subscription::{Order, Position, Subscription}; use std::{ collections::{BTreeMap, VecDeque}, - ops::{Bound, Range, RangeBounds}, + ops::{Bound, RangeBounds}, sync::Arc, sync::OnceLock, sync::atomic::{AtomicBool, AtomicU64, Ordering}, @@ -234,10 +234,6 @@ pub(crate) struct TrackState { // every cached group for each delivery. lookup: BTreeMap, - // Runs of `lookup` that cannot present, shared by every reader; see [`Quiet`]. - // Behind its own lock because walks record them under a read guard. - quiet: kio::Lock, - // Publisher-produced groups in arrival order as (sequence, stamp), walked by // subscriptions; an entry only resolves while its stamp matches the slot's. // Fetched backfill (`insert_group_request`) is deliberately absent: it is @@ -462,104 +458,15 @@ impl Slot { } } -/// Where walks for a group's successor stopped, so a run of groups that cannot present -/// (hidden, aborted, or finished without a frame) is walked once, not once for every -/// group judged against it. A publisher can make that run as long as the cache. -/// -/// Disjoint runs of sequences, each ending where a walk stopped: no group in a run could -/// present. A group stops presenting on its own, but only starts by landing, so a run -/// holds until a group that could present lands inside it, which splits it there. The -/// stop is outside its run and looked at again, so a walk past one that stopped -/// presenting records the longer run. -#[derive(Default)] -struct Quiet { - /// Each run's first sequence, to its stop. - runs: BTreeMap, - /// Groups walked past in total, so tests can bound the work. - #[cfg(test)] - passed: usize, -} - -impl Quiet { - /// A walk passes `sequence`, which cannot present: where its run stops, if one holds it. - fn pass(&mut self, sequence: u64) -> Option { - #[cfg(test)] - { - self.passed += 1; - } - self.stop(sequence) - } - - /// Where the run holding `sequence` stops, if one does. - fn stop(&self, sequence: u64) -> Option { - Some(self.holding(sequence)?.end) - } - - /// The run holding `sequence`, if one does. - fn holding(&self, sequence: u64) -> Option> { - let (&first, &stop) = self.runs.range(..=sequence).next_back()?; - (stop > sequence).then_some(first..stop) - } - - /// No group in `run` could present: record it, merged with the runs it overlaps. - fn record(&mut self, run: Range, cached: &BTreeMap) { - let mut start = run.start; - let mut stop = run.end; - if let Some((&first, &end)) = self.runs.range(..run.start).next_back() - && end > run.start - { - start = first; - stop = stop.max(end); - } - while let Some((&first, &end)) = self.runs.range(run.start..stop).next() { - self.runs.remove(&first); - stop = stop.max(end); - } - self.runs.insert(start, stop); - self.prune(cached); - } - - /// A group that could present landed at `sequence`: split the run holding it. - fn landed(&mut self, sequence: u64, cached: &BTreeMap) { - let Some((&first, &stop)) = self.runs.range(..=sequence).next_back() else { - return; - }; - if stop <= sequence { - return; - } - if first < sequence { - self.runs.insert(first, sequence); - } else { - self.runs.remove(&first); - } - if let Some(after) = sequence.checked_add(1) - && after < stop - { - self.runs.insert(after, stop); - } - self.prune(cached); - } - - /// Drop the runs holding no `cached` group once they outnumber twice the cache. Runs are - /// disjoint, so those left number at most the cache, and this stays within twice it. - fn prune(&mut self, cached: &BTreeMap) { - if self.runs.len() > 2 * cached.len() { - self.runs - .retain(|&first, &mut stop| cached.range(first..stop).next().is_some()); - } - } -} - /// Heap the track keeps per cached group, excluding the group itself /// ([`group::CACHE_OVERHEAD`]). /// -/// One [`Slot`] under its sequence in `lookup`, a hint in each of `arrival` and `evict`, -/// and at most two [`Quiet`] runs. Doubled because these containers run half empty in -/// the worst case: a `BTreeMap` node sits between half and fully packed, and a -/// `VecDeque` holds up to twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; -/// see it for why this is derived rather than measured. -pub(crate) const CACHE_OVERHEAD: u64 = - 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>() + 2 * size_of::<(u64, u64)>()) as u64; +/// One [`Slot`] under its sequence in `lookup`, plus a hint in each of `arrival` and +/// `evict`. Doubled because both containers run half empty in the worst case: a +/// `BTreeMap` node sits between half and fully packed, and a `VecDeque` holds up to +/// twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; see it for why this is +/// derived rather than measured. +pub(crate) const CACHE_OVERHEAD: u64 = 2 * (size_of::() + size_of::() + 2 * size_of::<(u64, u32)>()) as u64; /// The registered subscriptions, aggregated by the producer. type Subscriptions = Vec>; @@ -862,47 +769,19 @@ impl TrackState { /// so a later stamped group proves nothing about where an unstamped successor will /// begin, and shrinking the bound is the unsafe direction. An unstamped successor /// therefore leaves the reach unbounded until it presents its first frame, and so - /// does having no servable successor below `cap` at all. A successor that finished - /// without a frame never presents, so it is skipped and the next one bounds both. + /// does having no servable successor below `cap` at all. fn reach(&self, sequence: u64, cap: Option) -> Option { - self.successor(sequence, cap)?.timestamp() + self.first_servable(sequence.saturating_add(1), cap)?.timestamp() } - /// The first servable group above `sequence` and below `cap` that could still - /// present, stamped or not. - /// - /// Jumps over runs an earlier walk recorded in [`Self::quiet`], and records the run it - /// walks past, so judging every group of a run costs one walk over it, not one each. - fn successor(&self, sequence: u64, cap: Option) -> Option<&group::Producer> { - // The first group this walk passed. - let mut passed = None; - // The next sequence to look at, and where this walk stops once it breaks. - let mut from = sequence.checked_add(1)?; - let found = loop { - let Some((&at, slot)) = self.lookup.range(from..).next() else { - break None; - }; - if !super::subscription::before_end(at, cap) { - from = at; - break None; - } - if slot.is_shown() && !slot.group.ended_empty() { - from = at; - break Some(&slot.group); - } - passed.get_or_insert(at); - // Not held across `ended_empty`, which takes the group's lock. - let jump = self.quiet.lock().pass(at); - let Some(next) = jump.or_else(|| at.checked_add(1)) else { - from = at; - break None; - }; - from = next; - }; - if let Some(passed) = passed { - self.quiet.lock().record(passed..from, &self.lookup); - } - found + /// The first servable group in `from..cap`, stamped or not. + fn first_servable(&self, from: u64, cap: Option) -> Option<&group::Producer> { + self.lookup + .range(from..) + .map(|(_, slot)| slot) + .take_while(|slot| super::subscription::before_end(slot.group.sequence, cap)) + .find(|slot| slot.is_shown()) + .map(|slot| &slot.group) } /// Whether the group at `sequence` has drifted further behind `edge` than `budget` @@ -962,7 +841,7 @@ impl TrackState { // An abort only closes the group, never touching the track, so one that // landed before registering must re-select or the replacement goes unwatched. let successor = loop { - let successor = self.successor(sequence, cap); + let successor = self.first_servable(sequence.saturating_add(1), cap); match successor { Some(group) if group.poll_closed(waiter).is_ready() && group.is_aborted() => continue, successor => break successor, @@ -975,12 +854,7 @@ impl TrackState { if let Some(reach) = successor.timestamp() { break reach; } - if successor.poll_started(waiter).is_pending() { - return false; - } - // An untimed successor stays unbounded. One that finished without a frame since - // it was selected is skipped on the next pass. - if successor.timestamp().is_none() && !successor.ended_empty() { + if successor.poll_started(waiter).is_pending() || successor.timestamp().is_none() { return false; } }; @@ -1152,7 +1026,7 @@ impl TrackState { // Already aborted: the frames are gone, reclaim the slot so a // later fetch can serve the sequence again. if slot.is_aborted() { - self.remove_slot(sequence); + self.lookup.remove(&sequence); continue; } if self.protects(sequence) || !slot.is_expired(&scan) { @@ -1167,7 +1041,7 @@ impl TrackState { // Take the group out of the cache and abort it, so any consumer // still reading surfaces `Error::Old` instead of blocking forever // on a frame that will never arrive. - self.remove_slot(sequence).unwrap().abort(Error::Old); + self.lookup.remove(&sequence).unwrap().abort(Error::Old); } } @@ -1230,7 +1104,6 @@ impl TrackState { fn drop_open_groups(&mut self) { self.lookup .retain(|_, slot| slot.copies().any(group::Producer::is_finished)); - self.quiet.lock().prune(&self.lookup); } /// Attach the publisher's immutable metadata without replacing it with local cache policy. @@ -1287,19 +1160,12 @@ impl TrackState { /// A group that starts above `frame_start` is also replaceable: it cannot answer a /// request from there, so the wider producer takes the slot. Readers already /// draining the old one keep their own handle. - /// Take `sequence` out of the cache, keeping the runs recorded over it within bounds. - fn remove_slot(&mut self, sequence: u64) -> Option { - let slot = self.lookup.remove(&sequence); - self.quiet.lock().prune(&self.lookup); - slot - } - fn claim_sequence(&mut self, sequence: u64, frame_start: u64) -> Result<()> { // Either copy that can still answer from `frame_start` takes the sequence. if self.covering_group(sequence, frame_start).is_some() { return Err(Error::Duplicate); } - self.remove_slot(sequence); + self.lookup.remove(&sequence); Ok(()) } @@ -1379,50 +1245,17 @@ impl TrackState { /// Content at `sequence` became servable: wake the reads it succeeds, and show an answer /// waiting on it. fn landed(&mut self, sequence: u64) { - // A group revealed after finishing without a frame still cannot present, so it - // succeeds no read and only continues a held copy of itself. - let empty = self.lookup.get(&sequence).is_some_and(|slot| slot.group.ended_empty()); - let below = if empty { - Some(sequence) - } else { - // Read apart from the group's lock, which never nests under this one. - if self.quiet.lock().stop(sequence).is_some() { - self.quiet.lock().landed(sequence, &self.lookup); - } - self.predecessor(sequence) - }; + let below = self + .lookup + .range(..sequence) + .rev() + .find(|(_, slot)| slot.is_shown()) + .map(|(below, _)| *below); let oldest = self.lookup.first_key_value().map_or(sequence, |(oldest, _)| *oldest); self.cache.wakes().landed(below, sequence, oldest); self.shown(sequence); } - /// The nearest group below `sequence` that could present: the one whose successor - /// a landing at `sequence` becomes, with every group between. - /// - /// [`Self::successor`] in reverse, so it jumps over the runs in [`Self::quiet`] and - /// records the one it walks, which `sequence` stops. - fn predecessor(&self, sequence: u64) -> Option { - // The lowest group this walk passed, and the bound of the next look. - let mut passed = None; - let mut to = sequence; - let found = loop { - let Some((&at, slot)) = self.lookup.range(..to).next_back() else { - break None; - }; - if slot.is_shown() && !slot.group.ended_empty() { - break Some(at); - } - // Not held across `ended_empty`, which takes the group's lock. - let run = self.quiet.lock().holding(at); - to = run.map_or(at, |run| run.start); - passed = Some(to); - }; - if let Some(passed) = passed { - self.quiet.lock().record(passed..sequence, &self.lookup); - } - found - } - /// Content at `sequence` became readable: an answer waiting on it now shows. fn shown(&mut self, sequence: u64) { if let Feed::Answered { from } = self.feed @@ -1523,7 +1356,7 @@ impl TrackState { } if slot.is_aborted() { // Aborted upstream: the frames are already gone, reclaim the slot. - self.remove_slot(sequence); + self.lookup.remove(&sequence); continue; } if self.protects(sequence) { @@ -1550,7 +1383,7 @@ impl TrackState { self.debt -= size; paid = paid.saturating_add(size); - self.remove_slot(sequence).unwrap().abort(Error::Evicted); + self.lookup.remove(&sequence).unwrap().abort(Error::Evicted); } } @@ -6676,35 +6509,6 @@ mod test { assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); } - /// An open empty successor leaves a held group's reach unbounded. Once it finishes - /// without a frame, the held read wakes and is bounded by the group after it. - #[moq_net_sim::test] - async fn an_empty_successor_finishing_wakes_a_parked_read() { - let mut producer = track_producer("test", Info::timed()); - let mut head = producer.append_group().unwrap(); - head.write_frame(Timestamp::ZERO, b"head".as_slice()).unwrap(); - let successor = producer.append_group().unwrap(); - append_at(&mut producer, 1000); - append_at(&mut producer, 20_000); - let mut sub = producer.subscribe(None); - let mut reading = sub.recv_group().await.unwrap().unwrap(); - assert_eq!(reading.sequence, 0, "the open successor leaves the reach unbounded"); - assert!(reading.read_frame().await.unwrap().is_some()); - - let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); - let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); - let mut cx = std::task::Context::from_waker(&waker); - let mut next = std::pin::pin!(reading.read_frame()); - assert!(next.as_mut().poll(&mut cx).is_pending()); - successor.finish().unwrap(); - assert!( - woken.load(Ordering::SeqCst), - "the empty successor's finish lost its wakeup" - ); - let result = next.as_mut().poll(&mut cx); - assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); - } - /// A first timestamp on a *newer* group can convict a held one, so the held reader has /// to be woken by it. The conviction needs a group beyond the held one's successor: /// a group is bounded by where its successor begins, so the successor itself never @@ -7056,73 +6860,6 @@ mod test { assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); } - /// A group landing past a finished empty one becomes the successor of the group below - /// that, so it wakes a read parked there, which then finds the group stale. - #[test] - fn a_landing_past_an_empty_group_wakes_the_read_below_it() { - let producer = track_producer("test", Info::timed()); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - let mut head = producer.create_group(group::Info { sequence: 0 }).unwrap(); - head.write_frame(Timestamp::ZERO, bytes::Bytes::from_static(b"head")) - .unwrap(); - write_empty_group(&producer, 1); - write_group_at(&producer, 3, 100); - write_group_at(&producer, 4, 105); - - let mut held = subscriber - .recv_group() - .now_or_never() - .unwrap() - .unwrap() - .expect("head group"); - assert_eq!(held.sequence, 0); - assert!(held.read_frame().now_or_never().unwrap().unwrap().is_some()); - let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); - let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); - let mut cx = std::task::Context::from_waker(&waker); - let mut next = std::pin::pin!(held.read_frame()); - assert!(next.as_mut().poll(&mut cx).is_pending(), "group 3 bounds the head"); - - let mut late = producer.create_group(group::Info { sequence: 2 }).unwrap(); - assert!(woken.load(Ordering::SeqCst), "group 2 is the head's successor now"); - assert!(next.as_mut().poll(&mut cx).is_pending(), "group 2 has no frame yet"); - late.write_frame(Timestamp::from_millis(50).unwrap(), bytes::Bytes::from_static(b"x")) - .unwrap(); - let result = next.as_mut().poll(&mut cx); - assert!(matches!(result, Poll::Ready(Ok(None))), "the head is stale: {result:?}"); - } - - /// A group revealed after finishing without a frame succeeds no read, so it leaves a - /// read parked below it asleep: waking every such read per empty reveal is quadratic. - #[test] - fn an_empty_reveal_leaves_the_read_below_it_parked() { - let producer = track_producer("test", Info::timed()); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - let mut head = producer.create_group(group::Info { sequence: 0 }).unwrap(); - head.write_frame(Timestamp::ZERO, bytes::Bytes::from_static(b"head")) - .unwrap(); - let empty = producer.receive_group(group::Info { sequence: 1 }).unwrap(); - write_group_at(&producer, 3, 100); - - let mut held = subscriber - .recv_group() - .now_or_never() - .unwrap() - .unwrap() - .expect("head group"); - assert_eq!(held.sequence, 0); - assert!(held.read_frame().now_or_never().unwrap().unwrap().is_some()); - let woken = Arc::new(std::sync::atomic::AtomicBool::new(false)); - let waker = futures::task::waker(Arc::new(FlagWake(woken.clone()))); - let mut cx = std::task::Context::from_waker(&waker); - let mut next = std::pin::pin!(held.read_frame()); - assert!(next.as_mut().poll(&mut cx).is_pending(), "group 3 bounds the head"); - - empty.finish().unwrap(); - producer.reveal_group(&empty); - assert!(!woken.load(Ordering::SeqCst), "the empty group moves no edge"); - } - /// A route can declare the end, then land every group's header before its first frame. /// Those groups are withheld until their frames land, so the end is not reached before /// they show: a reader ending at the boundary would never see them. @@ -11180,262 +10917,6 @@ mod test { assert_eq!(drain_groups(&mut subscriber), [2998]); } - /// Write a finished group with no frames. - fn write_empty_group(producer: &Producer, sequence: u64) { - producer - .create_group(group::Info { sequence }) - .unwrap() - .finish() - .unwrap(); - } - - /// A group followed by one that finished without a frame is bounded by the next - /// group instead, so it falls past the budget like any other. - #[test] - fn a_group_before_an_empty_one_can_drift() { - let producer = track_producer("test", Info::timed()); - write_group(&producer, 0); - write_empty_group(&producer, 1); - write_group(&producer, 50); - write_group(&producer, 100); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - assert_eq!(drain_groups(&mut subscriber), [50, 100]); - } - - /// A group that finished without a frame never presents, so the group below it is - /// bounded by the next one instead, and gaps above it are still forgotten. - #[test] - fn an_empty_group_does_not_pin_later_gaps() { - let producer = track_producer("test", Info::timed()); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - write_group(&producer, 0); - write_group(&producer, 2); - write_empty_group(&producer, 3); - assert_eq!(drain_groups(&mut subscriber), [0, 2, 3]); - let mut most = 0; - for sequence in 5..3000u64 { - // Every third group never arrives. - if sequence % 3 == 1 { - continue; - } - write_group(&producer, sequence); - assert_eq!(drain_groups(&mut subscriber), [sequence]); - most = most.max(subscriber.delivered_runs()); - } - // One more than the endless track: group 2 reaches until group 5, past the empty 3. - assert_eq!(most, 6, "the floor and the gaps within the budget"); - } - - /// Every run starting with a stamped group followed by an empty one still falls past - /// the budget, so the record stays bounded. - #[test] - fn repeated_empty_groups_do_not_pin_gaps() { - let producer = track_producer("test", Info::timed()); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - write_group(&producer, 0); - assert_eq!(drain_groups(&mut subscriber), [0]); - let mut most = 0; - for k in 0..1000u64 { - // 3k + 1 never arrives. - write_group(&producer, 3 * k + 2); - write_empty_group(&producer, 3 * k + 3); - assert_eq!(drain_groups(&mut subscriber), [3 * k + 2, 3 * k + 3]); - most = most.max(subscriber.delivered_runs()); - } - assert_eq!(most, 6, "the floor and the gaps within the budget"); - } - - /// Write a finished one-frame group at `sequence`, presented at `millis`. - fn write_group_at(producer: &Producer, sequence: u64, millis: u64) { - let mut group = producer.create_group(group::Info { sequence }).unwrap(); - group - .write_frame(Timestamp::from_millis(millis).unwrap(), &b"x"[..]) - .unwrap(); - group.finish().unwrap(); - } - - /// Judging every group of a run that finished without a frame walks the run once, - /// whether one poll writes it all off or each poll hands one out, and however many - /// readers judge it. Without the recorded runs, each group walks the rest of the run: - /// two million groups here. - #[test] - fn a_run_of_empty_groups_is_walked_once() { - const RUN: u64 = 1000; - const READERS: usize = 4; - // Stamped groups after the run, and how many groups each reader is handed. - for (after, handed) in [(&[50, 100][..], 2), (&[5][..], RUN as usize + 2)] { - let producer = track_producer("test", Info::timed()); - write_group_at(&producer, 0, 0); - for sequence in 1..=RUN { - write_empty_group(&producer, sequence); - } - for (sequence, millis) in (RUN + 1..).zip(after) { - write_group_at(&producer, sequence, *millis); - } - for _ in 0..READERS { - let mut subscriber = - producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - assert_eq!(drain_groups(&mut subscriber).len(), handed); - } - let passed = producer.state.read().quiet.lock().passed; - assert!(passed <= 2 * RUN as usize * READERS, "walked past {passed} groups"); - } - } - - /// A group landing inside a run an earlier walk passed bounds the groups below it, - /// rather than the walk's recorded stop. - #[test] - fn a_group_landing_inside_a_walked_run_bounds_the_groups_below() { - let producer = track_producer("test", Info::timed()); - write_group_at(&producer, 0, 0); - for sequence in 1..=3 { - write_empty_group(&producer, sequence); - } - write_group_at(&producer, 5, 50); - write_group_at(&producer, 6, 100); - let budget = Subscription::default().with_max_delay(Duration::from_millis(10)); - let mut first = producer.subscribe(budget.clone()); - assert_eq!(drain_groups(&mut first), [5, 6]); - - // Open without a frame, so everything below it could present until it does. - let _open = producer.create_group(group::Info { sequence: 4 }).unwrap(); - let mut second = producer.subscribe(budget); - assert_eq!(drain_groups(&mut second), [0, 1, 2, 3, 5, 6]); - } - - /// A group landing inside a walked run keeps the part of it above: a pending run - /// revealed one group at a time, read in between, is still walked once. - #[test] - fn revealing_a_run_one_group_at_a_time_walks_it_once() { - const RUN: u64 = 1000; - let producer = track_producer("test", Info::timed()); - write_group_at(&producer, 0, 0); - let pending: Vec<_> = (1..=RUN) - .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) - .collect(); - write_group_at(&producer, RUN + 1, 50); - write_group_at(&producer, RUN + 2, 100); - let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_millis(10))); - assert_eq!(drain_groups(&mut subscriber), [RUN + 1, RUN + 2]); - for group in pending { - group.finish().unwrap(); - producer.reveal_group(&group); - assert!( - drain_groups(&mut subscriber).is_empty(), - "group {} is past the budget", - group.sequence - ); - } - let passed = producer.state.read().quiet.lock().passed; - assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); - } - - /// A group revealed after finishing without a frame still cannot present, so it keeps - /// the run recorded across it: judging the group below the run after each of its groups - /// is revealed, newest first, walks the run once. - #[test] - fn revealing_empty_groups_newest_first_keeps_the_walked_run() { - const RUN: u64 = 1000; - let producer = track_producer("test", Info::timed()); - write_group_at(&producer, 0, 0); - let pending: Vec<_> = (1..=RUN) - .map(|sequence| producer.receive_group(group::Info { sequence }).unwrap()) - .collect(); - write_group_at(&producer, RUN + 1, 50); - let successor = || producer.state.read().successor(0, None).map(|group| group.sequence); - assert_eq!(successor(), Some(RUN + 1)); - for group in pending.into_iter().rev() { - group.finish().unwrap(); - producer.reveal_group(&group); - assert_eq!(successor(), Some(RUN + 1)); - } - let passed = producer.state.read().quiet.lock().passed; - assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); - } - - /// Groups landing one after another inside a walked run split it there, keeping the - /// parts on either side, so judging the group below after each landing stays cheap. - #[test] - fn landings_inside_a_walked_run_keep_the_rest_of_it() { - const RUN: u64 = 1000; - let producer = track_producer("test", Info::timed()); - write_group_at(&producer, 0, 0); - for sequence in 1..=RUN { - write_empty_group(&producer, sequence); - } - write_group_at(&producer, 3 * RUN, 3 * RUN); - let successor = || producer.state.read().successor(0, None).map(|group| group.sequence); - assert_eq!(successor(), Some(3 * RUN)); - for sequence in RUN + 1..=2 * RUN { - write_group_at(&producer, sequence, sequence); - assert_eq!(successor(), Some(RUN + 1)); - } - let passed = producer.state.read().quiet.lock().passed; - assert!(passed <= 2 * RUN as usize, "walked past {passed} groups"); - } - - /// Walk a run of one group below each of `stamped`, so the track records one run each. - fn record_runs(producer: &Producer, stamped: impl Iterator) { - let state = producer.state.read(); - for sequence in stamped { - assert_eq!( - state.successor(sequence, None).map(|group| group.sequence), - Some(sequence + 2) - ); - } - } - - /// Runs recorded over groups that left the cache are dropped once they could outnumber - /// what is left, so the record stays within the overhead each cached group pays. - #[test] - fn the_record_shrinks_with_the_cache() { - // Expiry drops groups one at a time. - let pool = cache::Pool::new(cache::Config::default().with_expiry(Duration::from_secs(1))); - let producer = track_producer_pooled("test", pool.clone()); - for k in 0..100 { - write_group_at(&producer, 2 * k, k); - write_empty_group(&producer, 2 * k + 1); - } - write_group_at(&producer, 200, 100); - record_runs(&producer, (0..100).map(|k| 2 * k)); - assert_eq!(producer.state.read().quiet.lock().runs.len(), 100); - elapse(&producer, Duration::from_secs(2)); - // Fresh groups outlive the sweep, and are walked after it. - for k in 101..126 { - write_group_at(&producer, 2 * k, k); - write_empty_group(&producer, 2 * k + 1); - } - write_group_at(&producer, 252, 126); - let within = || { - let state = producer.state.read(); - let runs = state.quiet.lock().runs.len(); - runs <= 2 * state.lookup.len() - }; - pool.sweep(); - assert!(within()); - record_runs(&producer, (101..126).map(|k| 2 * k)); - assert!(within()); - - // Dropping the open groups of a track that ends abruptly drops many at once. - let producer = track_producer("test", Info::timed()); - let consumer = producer.consume(); - let mut open = Vec::new(); - for k in 0..100 { - open.push(producer.create_group(group::Info { sequence: 2 * k }).unwrap()); - open.push(producer.receive_group(group::Info { sequence: 2 * k + 1 }).unwrap()); - } - write_group_at(&producer, 200, 100); - for (k, group) in open.iter_mut().step_by(2).enumerate() { - group - .write_frame(Timestamp::from_millis(k as u64).unwrap(), &b"x"[..]) - .unwrap(); - } - record_runs(&producer, (0..100).map(|k| 2 * k)); - producer.abort(Error::Cancel).unwrap(); - let state = consumer.state.read(); - assert!(state.quiet.lock().runs.len() <= 2 * state.lookup.len()); - } - /// An untimed track has nothing past the budget, so a sparse feed keeps every gap /// and still hands out every group. The hand-out cost stays flat over the retained /// gaps; `track_sparse_read` measures it. From de14ce2d90c0a0f72614bc47f8f206ae1ef484c8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 13:46:59 -0700 Subject: [PATCH 19/21] fix(net): find the successor once for every gap one hand-out forgets The floor judged each forgotten gap with its own successor scan. Gaps below groups aborted since, but still cached, all share the successor past that run, so one hand-out putting N of them past the budget walked it N times: O(N^2) under the track's read guard. Reuse the last successor for every group below it. track_gap_flush sweeps the case. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 76 +++++++++++++++++++++++++++++++++++ rs/moq-net/src/model/track.rs | 67 +++++++++++++++++++++++++----- 2 files changed, 132 insertions(+), 11 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index ffdda1d710e..376993fe895 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -27,6 +27,9 @@ //! holding a fixed number of gaps the reader still remembers: each hand-out must cost the //! same however many gaps it holds, so a walk over them shows up as a slope. //! +//! `track_gap_flush` forgets every remembered gap in one hand-out, each below an aborted +//! group still cached: the successor they share must be found once, not once per gap. +//! //! `track_empty_run` drains a run of groups that finished without a frame across readers, //! with the groups after it past the budget or within it: each empty group is handed out //! like any other, so a walk over the rest of the run per group shows up as a slope over it. @@ -575,6 +578,78 @@ fn bench_sparse_read(c: &mut Criterion) { group.finish(); } +/// A reader remembering `gaps` gaps, each below a handed-out group since aborted but still +/// cached, then a stamped survivor after them all. +struct GapFlush { + _broadcast: broadcast::Producer, + track: track::Producer, + sub: track::Subscriber, + gaps: u64, +} + +impl GapFlush { + fn new(gaps: u64) -> Self { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", timed()).unwrap(); + let mut sub = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let mut hand_out = |sequence: u64, millis: u64| { + let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(millis).unwrap(), Bytes::from_static(b"x")) + .unwrap(); + let Poll::Ready(Ok(Some(read))) = sub.poll_recv_group(&kio::Waiter::noop()) else { + panic!("the group is cached"); + }; + black_box(read); + group + }; + // Every odd sequence is missing. + let handed: Vec<_> = (0..gaps).map(|gap| hand_out(gap * 2, gap)).collect(); + hand_out(gaps * 2, gaps).finish().unwrap(); + for group in handed { + group.abort(Error::Cancel).unwrap(); + } + Self { + _broadcast: broadcast, + track, + sub, + gaps, + } + } + + /// Hand out a group two hours on, which puts every remembered gap past the budget. + fn flush(&mut self) { + let mut group = self + .track + .create_group(moq_net::group::Info { + sequence: self.gaps * 2 + 1, + }) + .unwrap(); + group + .write_frame(Timestamp::from_millis(7_200_000).unwrap(), Bytes::from_static(b"x")) + .unwrap(); + let Poll::Ready(Ok(Some(read))) = self.sub.poll_recv_group(&kio::Waiter::noop()) else { + panic!("the group is cached"); + }; + black_box(read); + } +} + +/// One hand-out forgetting every remembered gap at once, each below a group that was +/// aborted but is still cached, swept over the gaps. The groups judged share one +/// successor past the aborted run, so finding it once per gap shows up as a slope. +fn bench_gap_flush(c: &mut Criterion) { + let mut group = c.benchmark_group("track_gap_flush"); + group.sample_size(10); + for gaps in [64u64, 512, 4096] { + group.throughput(Throughput::Elements(gaps)); + group.bench_function(BenchmarkId::from_parameter(gaps), |b| { + b.iter_batched_ref(|| GapFlush::new(gaps), GapFlush::flush, BatchSize::PerIteration); + }); + } + group.finish(); +} + /// Readers on one track holding group 0, a run of groups that finished without a frame, /// and the groups after it. struct EmptyRun { @@ -667,6 +742,7 @@ criterion_group!( bench_datagram_subscriber_join, bench_parked_read, bench_sparse_read, + bench_gap_flush, bench_empty_run ); criterion_main!(benches); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index a76984f87e6..1954464d813 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -815,17 +815,16 @@ impl TrackState { /// [`Self::is_stale`] for group `sequence`, whether this track holds it or not: its /// successor here starts a full `budget` behind the edge. fn drifted(&self, sequence: u64, edge: &Edge, budget: Duration) -> bool { - // The edge was resolved under an earlier lock, so confirm it still names the - // same servable incarnation before it convicts a candidate. Failing safe - // (delivering) is right, since the next poll resolves a fresh edge. - let Some(live) = edge - .presentation + self.live_above(sequence, edge) + .is_some_and(|live| live.behind(self.reach(sequence, edge.cap), budget)) + } + + /// The edge that could convict group `sequence`: above it, and still holding the + /// servable incarnation it was resolved from. It was resolved under an earlier lock, + /// and failing safe (delivering) is right, since the next poll resolves a fresh edge. + fn live_above(&self, sequence: u64, edge: &Edge) -> Option { + edge.presentation .filter(|live| live.sequence > sequence && self.holds(live.sequence, live.stamp)) - else { - return false; - }; - self.reach(sequence, edge.cap) - .is_some_and(|reach| matches!(live.timestamp.checked_sub(reach), Ok(age) if Duration::from(age) >= budget)) } /// [`Self::drifted`] under `cap`, parking `waiter` on exactly what else can move the @@ -3801,6 +3800,13 @@ struct PresentationEdge { timestamp: Timestamp, } +impl PresentationEdge { + /// Whether a group that can present until `reach` is a full `budget` behind this edge. + fn behind(&self, reach: Option, budget: Duration) -> bool { + reach.is_some_and(|reach| matches!(self.timestamp.checked_sub(reach), Ok(age) if Duration::from(age) >= budget)) + } +} + /// The read cursor behind a [`Subscriber`]. struct Cursor { state: kio::Consumer, @@ -3963,9 +3969,18 @@ impl Cursor { let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); let edge = state.drift_edge(self.end_sequence); + // Every group below the last successor found shares it, so a run of cached groups + // that cannot serve is walked once per pass, not once per gap. + let mut successor: Option<&group::Producer> = None; while let Some(oldest) = delivered.above_gap() - && state.drifted(oldest, &edge, budget) + && let Some(live) = state.live_above(oldest, &edge) { + if successor.is_none_or(|group| group.sequence <= oldest) { + successor = state.first_servable(oldest.saturating_add(1), edge.cap); + } + if !live.behind(successor.and_then(|group| group.timestamp()), budget) { + break; + } delivered.forget_below(oldest); } } @@ -10917,6 +10932,36 @@ mod test { assert_eq!(drain_groups(&mut subscriber), [2998]); } + /// Gaps below groups aborted since, but still cached, share the successor past them, so + /// one hand-out that puts them all past the budget forgets every one. The gap below a + /// group still within it is kept. `track_gap_flush` measures the cost. + #[test] + fn gaps_below_aborted_groups_are_forgotten_together() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(Duration::from_secs(1))); + let mut handed = Vec::new(); + for sequence in (0..20).step_by(2) { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(sequence).unwrap(), &b"x"[..]) + .unwrap(); + handed.push(group); + } + assert_eq!(drain_groups(&mut subscriber).len(), 10); + for group in handed.drain(..5) { + group.abort(Error::Cancel).unwrap(); + } + // Groups 0 to 8 reach until group 10 at 10ms, which a group at 1.01s puts past the + // budget. Group 10 reaches until 12ms, so the gap at 9 is kept. + write_group(&producer, 1010); + assert_eq!(drain_groups(&mut subscriber), [1010]); + assert_eq!( + subscriber.delivered_runs(), + 7, + "the floor at 8, then 10 to 18 and 1010 apart" + ); + } + /// An untimed track has nothing past the budget, so a sparse feed keeps every gap /// and still hands out every group. The hand-out cost stays flat over the retained /// gaps; `track_sparse_read` measures it. From f1ef408bcba7d0eed7d3fc1094ca83e4c5410024 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 14:13:32 -0700 Subject: [PATCH 20/21] test(net): build the delivered-record tests without the async feature The once-only tests are synchronous, so the timestamp and untimed track helpers they share with async tests are no longer gated on async. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 2 -- rs/moq-net/src/model/track.rs | 1 - 2 files changed, 3 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index b706badca71..85e43c55d0e 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -1105,7 +1105,6 @@ mod test { use futures::FutureExt; use super::*; - #[cfg(feature = "async")] use crate::Timestamp; use crate::broadcast; @@ -1150,7 +1149,6 @@ mod test { frame.map(|frame| frame.payload.to_vec()) } - #[cfg(feature = "async")] fn ts(ms: u64) -> Timestamp { Timestamp::from_millis(ms).unwrap() } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index bcf573f6ebf..65f7c311090 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -7392,7 +7392,6 @@ mod test { ); } - #[cfg(feature = "async")] fn untimed_producer() -> Producer { track_producer("test", None) } From 5960144bff713ff2e6c837a18f7df75b0eb6ed57 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 14 Oct 2026 14:34:42 -0700 Subject: [PATCH 21/21] quest: leave the cache-max-age quest to its own PR Co-Authored-By: Claude Opus 5.5 --- quest/m1/cache-max-age.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/quest/m1/cache-max-age.md b/quest/m1/cache-max-age.md index 7c3a2d0032c..97643b8c2b9 100644 --- a/quest/m1/cache-max-age.md +++ b/quest/m1/cache-max-age.md @@ -65,11 +65,6 @@ Decided (2026-10-06, maintainer): reclamation from the rule), and the untimed text in `drafts/draft-lcurley-moq-timestamp.md` that the untimed model adds ("never media-stale, starts at the latest group"). No new guide. -- Decided 2026-10-13 (maintainer): the wall term also advances the floor of - `track::Subscriber`'s `Delivered` record (`rs/moq-net/src/model/delivered.rs`). - An untimed track has no media time, so that floor never advances on it, and - an endless untimed track that skips groups grows the record by one range per - gap. Facts, and work carried over: @@ -125,10 +120,6 @@ Tests, with mocked time: hide the stall. Nothing writes after the successor arrives, so only the deadline wake can release the reader. It fails without the fix. - An untimed subscriber starts at the oldest group that isn't stale. -- An endless untimed track with gaps keeps a subscriber's `Delivered` record - bounded: the wall term forgets a gap once a delivered group above it is - stale, as `track::an_endless_track_with_gaps_stays_bounded` checks for - timed tracks. Public API: no signature change; `max_age` behaviour changes. Wire: no encoding change; Max Age semantics in the lite draft change.