From b7ba17dc9d2adce3aeba5cef544b0b82195e3439 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 00:57:06 -0700 Subject: [PATCH 1/4] chore(quest): claim quest/m1/release-perf/sparse-gaps.md Quest-Claim: 1c5188ad748fbbdf From cf6c316ee80801bae2787661220d8204944eab36 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 01:32:40 -0700 Subject: [PATCH 2/4] perf(net): index sparse arrivals and batch gap retirement Co-Authored-By: GPT-6 --- quest/m1/release-perf/sparse-gaps.md | 103 +++++++++---- rs/moq-net/benches/track.rs | 158 ++++++++++++++++--- rs/moq-net/src/model/delivered.rs | 10 ++ rs/moq-net/src/model/ledger.rs | 218 +++++++++++++++++++-------- rs/moq-net/src/model/track.rs | 70 ++++++--- 5 files changed, 428 insertions(+), 131 deletions(-) diff --git a/quest/m1/release-perf/sparse-gaps.md b/quest/m1/release-perf/sparse-gaps.md index 8b7a74e03d1..9e50716c3ed 100644 --- a/quest/m1/release-perf/sparse-gaps.md +++ b/quest/m1/release-perf/sparse-gaps.md @@ -8,30 +8,79 @@ faster than their size parameter. ## Plan -Found by reading code on 2026-10-21. Confirm each step with callgrind -instruction counts at two sizes before and after the change. - -- `Ledger` (`rs/moq-net/src/model/ledger.rs`) stores its runs in a `Vec`. - [#5802](https://github.com/moq-dev/moq/pull/5802) moved it from the session - tail ([#4533](https://github.com/moq-dev/moq/pull/4533), which only ever - held a few gaps) onto every track's live arrivals. Each arrival now pays - O(gaps) twice. `account` uses `Vec::splice`: closing the lowest gap moves - every run, about 100 KB at 4096 gaps. And while gaps exist, `expire` runs a - `dedup_by` over all runs plus `snapshot_subscription`. Under the 3600 s - delay in `track_sparse_read/timed`, gaps never fold, so a sparse feed is - quadratic overall. Fix: key the runs in a `BTreeMap`, - as `Delivered` already does. Cache the earliest fold instant (the smallest - `since` plus grace), so `expire` returns in O(1) when nothing is due. -- `track_gap_flush` (4.1x at 64 gaps, 96x at 4096): `Cursor::forget_past_budget` - in `track.rs` loops once per gap. Each pass calls `delivered.forget_below` - (a `BTreeMap::split_off` plus an insert) and reads the wall clock again in - `state.aged()`. Fix: read `now` once. Every gap below the shared successor - gets the same verdict, so look up the largest run start below it with - `runs.range(..s).next_back()` and call `forget_below` once. - `Delivered` comes from [#5446](https://github.com/moq-dev/moq/pull/5446) and - the wall-clock check from [#5548](https://github.com/moq-dev/moq/pull/5548). - Both are intended features, so only the cost per gap is wrong. - -Release forgot nothing here because it had no per-reader delivery record. -Don't aim for its absolute numbers. Aim for flat cost per gap across the -`track_gap_flush` and `track_sparse_read/timed` sweeps (16, 256, 4096). +The source changes in this PR replace the ledger's vector with a sequence-keyed +`BTreeMap` and an age-keyed `BTreeSet`. Splits inherit gap ages; merges and expiry +remove both index entries. In-order arrivals extend the existing run in place. +An age index, rather than only a cached minimum plus a full scan, also keeps an +arrival cheap when one older gap expires every time. + +The cursor samples the clock once and finds the last delivered run below the +successor supplying the stale verdict. It calls `forget_below` once per shared +successor, preserving later gaps whose successor remains within budget. + +### Remaining decision + +The original goal says a gap flush costs O(log gaps). The implementation follows +the planned indexed lookup and single split, but dropping the retired map entries +necessarily costs O(retired entries). No teardown was deferred or moved outside +the operation. The bound is O(log gaps + retired entries) per shared successor; +the sparse arrival bound is O(log gaps) amortized, plus any gaps it expires. + +1. Accept this bound for a batch, with instruction sweeps confirming no repeated + work per gap (recommended). +2. Require the whole batch, including reclaiming retired storage, to be O(log + gaps). That requires a different ownership/representation design; retain the + source checkpoint while that scope is settled. + +Do not finish this quest until the maintainer settles that interpretation and the +validation limits are resolved. The original Goal above is unchanged. + +### Verification + +`track_sparse_read` and `track_gap_flush` retain their existing nightly target. +The new `track_sparse_expire` workload opens one gap while one older gap expires, +swept over 16/256/4096 retained gaps. The ledger tests check both indices after +split/merge/expiry operations, demand reordering, a replaced leading run, grace +changes, and repeated individual expiry. Wall-clock and media-time batch tests +preserve the first gap with a fresh successor. + +Profiling boundaries exclude fixture setup. Both builds warm the same initial +cache-queue capacity expansion before collection: a single iteration immediately +after a 4096-item setup otherwise measures an unrelated `VecDeque` growth copy. +The final counts measure steady-state arrival work; cold allocation/growth remains +an amortized production cost and is not claimed to be logarithmic per operation. + +Callgrind 3.27.1 with Rust 1.98.1, optimized with symbols and line tables, +compares [c78547efff](https://github.com/moq-dev/moq/commit/c78547efff) plus the +identical benchmark fixture on both builds. One +`--bench --test --exact` iteration collects only `*read_pair`, `*Expiring*arrive`, +or `*GapFlush*flush`, excluding setup. The expiring workload warms two arrivals +because the eviction queue trails the arrival queue by one entry. + +| Workload | Gaps | Before instructions | After instructions | +|---|---|---|---| +| read | 16 | 29,591 | 30,838 | +| read | 256 | 43,625 | 34,389 | +| read | 4096 | 315,641 | 40,856 | +| expire | 16 | 15,484 | 16,242 | +| expire | 256 | 23,780 | 17,791 | +| expire | 4096 | 138,439 | 22,050 | +| flush | 16 | 38,996 | 25,405 | +| flush | 256 | 443,152 | 49,921 | +| flush | 4096 | 9,106,805 | 463,636 | + +At 4096 gaps, read-pair instructions fall 87.1%, steady expiry 84.1%, and flush +94.9%. At 16 gaps, the indices add 4.2% to a read pair and 4.9% to expiry; the +larger retained-gap costs no longer come from full ledger walks. Flush retains +linear storage reclamation rather than charging that work elsewhere. + +Validation: all 2918 moq-net tests pass (eight ignored nightly tests). `just ci` +passes Rust lint/docs/wasm, quest validation, and drill applicability, then fails +on the unchanged Nix runner image because this host cannot write +`/proc/self/uid_map`. Broader test results remain recorded in the PR. + +Found in the release comparison. The arrival ledger came from +[moq-dev/moq#5802](https://github.com/moq-dev/moq/pull/5802), delivery records from +[moq-dev/moq#5446](https://github.com/moq-dev/moq/pull/5446), and wall-clock aging +from [moq-dev/moq#5548](https://github.com/moq-dev/moq/pull/5548). Those features +remain; only their accidental repeated work changes. diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index f471d193563..1a9f75186cb 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -27,6 +27,9 @@ //! holding a fixed number of gaps the reader still remembers: each hand-out must cost the //! same however many gaps it holds, so a walk over them shows up as a slope. //! +//! `track_sparse_expire` opens one new gap while one old gap expires, so a full expiry +//! rescan on each arrival shows up as a slope over the gaps still within budget. +//! //! `track_gap_flush` forgets every remembered gap in one hand-out, each below an aborted //! group still cached: the successor they share must be found once, not once per gap. //! @@ -1006,6 +1009,64 @@ fn bench_parked_read(c: &mut Criterion) { /// opening a gap at the top, and the lowest missing one, closing the gap at the bottom, so /// the count stays fixed. Swept over the gaps so a per-hand-out cost that grows with them /// shows up as a slope. +struct Sparse { + _broadcast: broadcast::Producer, + track: track::Producer, + sub: track::Subscriber, + next: u64, + missing: u64, + timed: bool, +} + +impl Sparse { + fn new(info: track::Info, gaps: u64) -> Self { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", info.clone()).unwrap(); + let sub = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let mut sparse = Self { + _broadcast: broadcast, + track, + sub, + next: gaps * 2, + missing: 1, + timed: info.timescale.is_some(), + }; + for gap in 0..gaps { + sparse.hand_out(gap * 2); + } + // Fill beyond the cache queue's initial power-of-two capacity outside timing. + // Its amortized growth is independent of how many ledger gaps remain open. + if gaps != 0 { + sparse.advance(); + } + sparse + } + + fn hand_out(&mut self, sequence: u64) { + let timestamp = self.timed.then(|| Timestamp::from_micros(sequence).unwrap()); + let mut group = self.track.create_group(moq_net::group::Info { sequence }).unwrap(); + group.write_frame(timestamp, Bytes::from_static(b"x")).unwrap(); + group.finish().unwrap(); + let Poll::Ready(Ok(Some(read))) = self.sub.poll_recv_group(&kio::Waiter::noop()) else { + panic!("the group is cached"); + }; + black_box(read); + } + + // A profiling boundary excludes filling the initial sparse backlog. + #[inline(never)] + fn read_pair(&mut self) { + self.advance(); + } + + fn advance(&mut self) { + self.hand_out(self.next); + self.hand_out(self.missing); + self.next += 2; + self.missing += 2; + } +} + fn bench_sparse_read(c: &mut Criterion) { let mut group = c.benchmark_group("track_sparse_read"); group.throughput(Throughput::Elements(2)); @@ -1013,31 +1074,10 @@ fn bench_sparse_read(c: &mut Criterion) { for gaps in [16u64, 256, 4096] { group.bench_function(BenchmarkId::new(name, gaps), |b| { b.iter_custom(|iterations| { - let broadcast = broadcast::Info::default().produce(); - let track = broadcast.create_track("bench", info.clone()).unwrap(); - let mut sub = - track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); - // Microseconds, so no run lasts long enough to cross the budget. - let timestamp = |sequence: u64| info.timescale.map(|_| Timestamp::from_micros(sequence).unwrap()); - let mut hand_out = |sequence: u64| { - let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); - group - .write_frame(timestamp(sequence), Bytes::from_static(b"x")) - .unwrap(); - group.finish().unwrap(); - let Poll::Ready(Ok(Some(read))) = sub.poll_recv_group(&kio::Waiter::noop()) else { - panic!("the group is cached"); - }; - black_box(read); - }; - // Every odd sequence is missing until the bottom gap closes. - for gap in 0..gaps { - hand_out(gap * 2); - } + let mut sparse = Sparse::new(info.clone(), gaps); let start = WallInstant::now(); - for iteration in 0..iterations { - hand_out((gaps + iteration) * 2); - hand_out(iteration * 2 + 1); + for _ in 0..iterations { + sparse.read_pair(); } start.elapsed() }); @@ -1047,6 +1087,71 @@ fn bench_sparse_read(c: &mut Criterion) { group.finish(); } +/// A sparse feed that opens one gap per millisecond, keeping `gaps` within its budget. +struct Expiring { + sparse: Sparse, + pool: cache::Pool, + now: Instant, +} + +impl Expiring { + fn new(gaps: u64) -> Self { + let mut sparse = Sparse::new(track::Info::default(), 0); + sparse + .sub + .update(track::Subscription::default().with_max_delay(Duration::from_millis(gaps))) + .unwrap(); + let pool = sparse._broadcast.info().pool.clone(); + // Supply time ahead of the host clock, so only these steps age the gaps even + // under an instruction profiler. No cache capacity or idle expiry is enabled. + let mut fixture = Self { + sparse, + pool, + now: Instant::now() + Duration::from_secs(3600), + }; + for gap in 0..gaps { + fixture.pool.gc(fixture.now, &kio::Waiter::noop()); + fixture.sparse.hand_out(gap * 2); + fixture.now += Duration::from_millis(1); + } + fixture.sparse.next = gaps * 2; + // The eviction queue trails arrivals by one entry: warm both capacities. + fixture.step(); + fixture.step(); + fixture + } + + // One new arrival expires one old gap; excludes building the initial ledger. + #[inline(never)] + fn arrive(&mut self) { + self.step(); + } + + fn step(&mut self) { + self.now += Duration::from_millis(1); + self.pool.gc(self.now, &kio::Waiter::noop()); + self.sparse.hand_out(self.sparse.next); + self.sparse.next += 2; + } +} + +fn bench_sparse_expire(c: &mut Criterion) { + let mut group = c.benchmark_group("track_sparse_expire"); + for gaps in [16u64, 256, 4096] { + group.bench_function(BenchmarkId::from_parameter(gaps), |b| { + b.iter_custom(|iterations| { + let mut expiring = Expiring::new(gaps); + let start = WallInstant::now(); + for _ in 0..iterations { + expiring.arrive(); + } + start.elapsed() + }); + }); + } + group.finish(); +} + /// A reader remembering `gaps` gaps, each below a handed-out group since aborted but still /// cached, then a stamped survivor after them all. struct GapFlush { @@ -1087,6 +1192,8 @@ impl GapFlush { } /// Hand out a group two hours on, which puts every remembered gap past the budget. + // A profiling boundary excludes publishing the groups and their aborts. + #[inline(never)] fn flush(&mut self) { let mut group = self .track @@ -1110,7 +1217,7 @@ impl GapFlush { fn bench_gap_flush(c: &mut Criterion) { let mut group = c.benchmark_group("track_gap_flush"); group.sample_size(10); - for gaps in [64u64, 512, 4096] { + for gaps in [16u64, 64, 256, 512, 4096] { group.throughput(Throughput::Elements(gaps)); group.bench_function(BenchmarkId::from_parameter(gaps), |b| { b.iter_batched_ref(|| GapFlush::new(gaps), GapFlush::flush, BatchSize::PerIteration); @@ -1362,6 +1469,7 @@ criterion_group!( bench_datagram_subscriber_join, bench_parked_read, bench_sparse_read, + bench_sparse_expire, bench_gap_flush, bench_wall, bench_empty_run, diff --git a/rs/moq-net/src/model/delivered.rs b/rs/moq-net/src/model/delivered.rs index 6ee61a91267..8ad4ae7d1c5 100644 --- a/rs/moq-net/src/model/delivered.rs +++ b/rs/moq-net/src/model/delivered.rs @@ -75,6 +75,11 @@ impl Delivered { } } + /// The last run starting below `sequence`. + pub fn last_before(&self, sequence: u64) -> Option { + self.runs.range(..sequence).next_back().map(|(start, _)| *start) + } + /// How many runs it holds: one per gap, plus the floor. #[cfg(test)] pub fn runs(&self) -> usize { @@ -126,6 +131,11 @@ mod test { delivered.insert(sequence); } assert_eq!(delivered.above_gap(), Some(3)); + assert_eq!(delivered.last_before(0), None); + assert_eq!(delivered.last_before(3), None); + assert_eq!(delivered.last_before(5), Some(3)); + assert_eq!(delivered.last_before(9), Some(5)); + assert_eq!(delivered.last_before(13), Some(12)); delivered.forget_below(0); assert_eq!(delivered.runs(), 4); diff --git a/rs/moq-net/src/model/ledger.rs b/rs/moq-net/src/model/ledger.rs index 85de2455a08..ac9806757b4 100644 --- a/rs/moq-net/src/model/ledger.rs +++ b/rs/moq-net/src/model/ledger.rs @@ -3,7 +3,11 @@ //! A subscription's tail (the session's `tail::Tail`) and a track's record of live arrivals //! both keep one, so they give up on a missing group by the same rule. -use std::{ops::Range, time::Duration}; +use std::{ + collections::{BTreeMap, BTreeSet}, + ops::Range, + time::Duration, +}; use crate::time::Instant; @@ -25,10 +29,10 @@ pub(crate) fn grace(max_delay: Duration) -> Duration { } } -/// A run of accounted sequences. +/// A run of accounted sequences, keyed by its inclusive start. #[derive(Debug, Clone, PartialEq, Eq)] struct Run { - groups: Range, + end: u64, /// When the gap below this run opened, which is when its missing groups became late. since: Instant, } @@ -41,7 +45,10 @@ pub(crate) struct Ledger { // can no longer be waited for, so it folds into the runs around it, which bounds this by // the gaps opened within the grace rather than every gap ever seen. A fold is final: a // later, longer grace or a lowered floor cannot reopen it. - runs: Vec, + runs: BTreeMap, + // Demand can restart a gap below an older one, so sequence order is not age order. + // Index both: one due gap must not make every arrival scan all the younger ones. + ages: BTreeSet<(Instant, u64)>, grace: Duration, } @@ -55,7 +62,8 @@ impl Ledger { /// A ledger that gives up on a missing group after `grace`. pub fn new(grace: Duration) -> Self { Self { - runs: Vec::new(), + runs: BTreeMap::new(), + ages: BTreeSet::new(), grace, } } @@ -65,43 +73,69 @@ impl Ledger { self.grace = grace; } + fn remove(&mut self, start: u64) -> Run { + let run = self.runs.remove(&start).expect("an indexed run"); + self.ages.remove(&(run.since, start)); + run + } + /// Record `groups` as accounted for. Gaps fold only on [`Self::expire`]. pub fn account(&mut self, groups: Range, now: Instant) { if groups.is_empty() { return; } - // Every run the insert overlaps or touches merges with it into one. - let first = self.runs.partition_point(|run| run.groups.end < groups.start); - let last = self.runs.partition_point(|run| run.groups.start <= groups.end); - let merged = match &self.runs[first..last] { - [] => Run { - groups, - // Splitting a gap leaves both halves as late as it was. A run past every - // other one opens a new gap. - since: self.runs.get(first).map_or(now, |above| above.since), - }, - [head, .., tail] => Run { - groups: head.groups.start.min(groups.start)..tail.groups.end.max(groups.end), - since: head.since, - }, - [only] => Run { - groups: only.groups.start.min(groups.start)..only.groups.end.max(groups.end), - since: only.since, - }, + // Keep the lower run in place when extending it: the in-order path changes + // only its end, without reallocating either index or changing its gap's age. + let below = self + .runs + .range(..=groups.start) + .next_back() + .filter(|(_, run)| run.end >= groups.start) + .map(|(&start, run)| (start, run.end)); + if below.is_some_and(|(_, end)| groups.end <= end) { + return; + } + let first = below.map_or(groups.start, |(start, _)| start); + // Splitting a gap inherits the upper run's age; only an arrival above every + // run opens a fresh gap. + let since = self.runs.range(first..).next().map_or(now, |(_, run)| run.since); + let mut end = groups.end.max(below.map_or(groups.end, |(_, end)| end)); + let from = match below { + Some(_) => std::ops::Bound::Excluded(first), + None => std::ops::Bound::Included(first), }; - self.runs.splice(first..last, [merged]); + while let Some((&start, run)) = self.runs.range((from, std::ops::Bound::Included(end))).next() { + end = end.max(run.end); + self.remove(start); + } + match below { + Some(_) => self.runs.get_mut(&first).expect("the lower run stayed indexed").end = end, + None => { + self.ages.insert((since, first)); + self.runs.insert(first, Run { end, since }); + } + } } /// Fold every gap older than the grace into the runs around it. pub fn expire(&mut self, now: Instant) { - let grace = self.grace; - self.runs.dedup_by(|run, below| { - let expired = now.duration_since(run.since) >= grace; - if expired { - below.groups.end = run.groups.end; + let Some((&first, _)) = self.runs.first_key_value() else { + return; + }; + // There is no lower run to fold the leading gap into. Keep its age for a + // later lowered floor, but do not let it hide due gaps elsewhere in the index. + while let Some(&(since, start)) = self.ages.iter().find(|(_, start)| *start != first) { + if now.duration_since(since) < self.grace { + break; } - expired - }); + let run = self.remove(start); + self.runs + .range_mut(..start) + .next_back() + .expect("not the first run") + .1 + .end = run.end; + } } /// Whether a gap separates two runs, so [`Self::expire`] has anything to fold. @@ -111,25 +145,21 @@ impl Ledger { /// The lowest sequence accounted for. pub fn first(&self) -> Option { - self.runs.first().map(|run| run.groups.start) + self.runs.first_key_value().map(|(start, _)| *start) } /// The exclusive end of the accounted run holding `sequence`, `None` while it is not. pub fn run_end(&self, sequence: u64) -> Option { - let run = self - .runs - .get(self.runs.partition_point(|run| run.groups.end <= sequence))?; - (run.groups.start <= sequence).then_some(run.groups.end) + self.runs + .range(..=sequence) + .next_back() + .filter(|(_, run)| sequence < run.end) + .map(|(_, run)| run.end) } /// Whether every group in `groups` is accounted for. pub fn covers(&self, groups: Range) -> bool { - // Runs merge on insert, so one run covers the span or none does. - groups.is_empty() - || self - .runs - .get(self.runs.partition_point(|run| run.groups.end <= groups.start)) - .is_some_and(|run| run.groups.start <= groups.start && groups.end <= run.groups.end) + groups.is_empty() || self.run_end(groups.start).is_some_and(|end| groups.end <= end) } } @@ -147,11 +177,13 @@ impl Ledger { return; } let mut below = 0; - for run in &mut self.runs { - if below < groups.end && groups.start < run.groups.start { + for (&start, run) in &mut self.runs { + if below < groups.end && groups.start < start { + self.ages.remove(&(run.since, start)); run.since = now; + self.ages.insert((now, start)); } - below = run.groups.end; + below = run.end; } } @@ -159,23 +191,25 @@ impl Ledger { /// group opened the gap below it. `None` while it is accounted for, or while no later /// group has arrived, since nothing says it is late yet. pub fn folds_at(&self, group: u64, grace: Duration) -> Option { - let above = self - .runs - .get(self.runs.partition_point(|run| run.groups.end <= group))?; - match above.groups.start <= group { - true => None, - false => above.since.checked_add(grace), + if self.run_end(group).is_some() { + return None; } + self.runs.range(group..).next()?.1.since.checked_add(grace) } /// The gaps within `groups`, each with when a later arrival opened it: `None` for the /// one past the last run, which nothing has opened yet. pub fn gaps(&self, groups: Range) -> impl Iterator, Option)> + '_ { - let first = self.runs.partition_point(|run| run.groups.end <= groups.start); + let first = self + .runs + .range(..=groups.start) + .next_back() + .filter(|(_, run)| run.end > groups.start) + .map_or(groups.start, |(start, _)| *start); let mut below = groups.start; - self.runs[first..] - .iter() - .map(|run| (run.groups.clone(), Some(run.since))) + self.runs + .range(first..) + .map(|(&start, run)| (start..run.end, Some(run.since))) .chain(std::iter::once((u64::MAX..u64::MAX, None))) .map_while(move |(run, since)| { let gap = below..run.start.min(groups.end); @@ -188,9 +222,9 @@ impl Ledger { /// Gaps whose age bounds a held fetch, including the gap before the first arrival. pub fn folds(&self, grace: Duration) -> impl Iterator, Instant)> + '_ { let mut below = 0; - self.runs.iter().filter_map(move |run| { - let gap = below..run.groups.start; - below = run.groups.end; + self.runs.iter().filter_map(move |(&start, run)| { + let gap = below..start; + below = run.end; if gap.is_empty() { return None; } @@ -210,7 +244,11 @@ mod tests { } fn groups(ledger: &Ledger) -> Vec> { - ledger.runs.iter().map(|run| run.groups.clone()).collect() + assert_eq!( + ledger.ages, + ledger.runs.iter().map(|(&start, run)| (run.since, start)).collect() + ); + ledger.runs.iter().map(|(&start, run)| start..run.end).collect() } #[test] @@ -327,4 +365,66 @@ mod tests { ledger.expire(lowered + GRACE); assert!(ledger.covers(1..10)); } + #[test] + fn demand_can_move_an_older_gap_after_a_newer_one() { + let start = Instant::now(); + let mut ledger = Ledger::new(GRACE); + arrive(&mut ledger, 0..1, start); + arrive(&mut ledger, 4..5, start); + arrive(&mut ledger, 8..9, start + GRACE / 4); + ledger.demand(1..4, start + GRACE / 2); + + ledger.expire(start + GRACE + GRACE / 4); + assert_eq!(groups(&ledger), vec![0..1, 4..9]); + assert_eq!(ledger.folds_at(1, GRACE), Some(start + GRACE + GRACE / 2)); + ledger.expire(start + GRACE + GRACE / 2); + assert_eq!(groups(&ledger), vec![0..9]); + } + + #[test] + fn changing_grace_uses_current_gap_ages_without_reopening_folds() { + let start = Instant::now(); + let mut ledger = Ledger::new(GRACE); + arrive(&mut ledger, 0..1, start); + arrive(&mut ledger, 2..3, start); + ledger.set_grace(GRACE * 2); + ledger.expire(start + GRACE); + assert_eq!(groups(&ledger), vec![0..1, 2..3]); + + ledger.set_grace(GRACE / 2); + ledger.expire(start + GRACE); + assert_eq!(groups(&ledger), vec![0..3]); + ledger.set_grace(GRACE * 2); + arrive(&mut ledger, 4..5, start + GRACE); + ledger.expire(start + GRACE * 2); + assert_eq!(groups(&ledger), vec![0..3, 4..5]); + } + + #[test] + fn a_former_first_run_keeps_its_age_when_a_lower_run_arrives() { + let start = Instant::now(); + let mut ledger = Ledger::new(GRACE); + arrive(&mut ledger, 10..11, start); + ledger.expire(start + GRACE * 2); + assert_eq!(groups(&ledger), vec![10..11]); + arrive(&mut ledger, 0..1, start + GRACE * 2); + assert_eq!(groups(&ledger), vec![0..11]); + } + #[test] + fn one_gap_expires_per_arrival_without_retaining_older_gaps() { + let start = Instant::now(); + for retained in [0u64, 1, 4, 16] { + let mut ledger = Ledger::new(Duration::from_millis(retained)); + for arrival in 0..retained * 4 + 8 { + arrive( + &mut ledger, + arrival * 2..arrival * 2 + 1, + start + Duration::from_millis(arrival), + ); + let runs = groups(&ledger); + assert_eq!(runs.len() as u64, (arrival + 1).min(retained + 1)); + assert_eq!(runs[0], 0..arrival.saturating_sub(retained) * 2 + 1); + } + } + } } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 1d758e23ed7..2fa926d2e23 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -5852,31 +5852,28 @@ impl Cursor { let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); let edge = state.drift_edge(&self.servable); - // Every group below the last successor found shares it, so a run of cached groups - // that cannot serve is walked once per pass, not once per gap. - let mut arrival: Option<&Slot> = None; - let mut successor: Option<&Slot> = None; + let now = state.cache.pool().now_wall(); while let Some(oldest) = delivered.above_gap() { - if arrival.is_none_or(|slot| slot.group.sequence <= oldest) { - arrival = state.arrival(oldest, &edge.servable); - } // Either clock, as `drifted` judges a group: without a successor, neither. - let Some(first) = arrival else { + let Some(arrival) = state.arrival(oldest, &edge.servable) else { break; }; - if successor.is_none_or(|slot| slot.group.sequence <= oldest) { - successor = state.successor(oldest, &edge.servable); - } - let past = state.aged(first, budget) - || successor.is_some_and(|slot| { - state - .live_above(oldest, &edge) - .is_some_and(|live| live.behind(slot.group.timestamp(), budget)) - }); - if !past { + let shared = if now.saturating_duration_since(arrival.arrived) >= budget { + arrival.group.sequence + } else if let Some(successor) = state.successor(oldest, &edge.servable) + && let Some(live) = state.live_above(oldest, &edge) + && live.behind(successor.group.timestamp(), budget) + { + successor.group.sequence.min(live.sequence) + } else { break; - } - delivered.forget_below(oldest); + }; + // Every delivered run below this successor shares its verdict. Retire + // them in one split, including the highest run, rather than once per gap. + let last = delivered + .last_before(shared) + .expect("the oldest run is below its successor"); + delivered.forget_below(last); } } @@ -15544,6 +15541,39 @@ mod test { ); } + /// A common wall-clock successor retires the gaps below an aborted prefix, but a + /// later arrival keeps the next gap within its budget. + #[test] + fn wall_time_batches_aborted_gaps_without_forgetting_a_fresh_successor() { + let budget = Duration::from_secs(1); + let producer = track_producer("test", None); + let mut subscriber = producer.subscribe(Subscription::default().with_max_delay(budget)); + let mut open = Vec::new(); + for sequence in (0..=10).step_by(2) { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group.write_frame(None, &b"x"[..]).unwrap(); + open.push(group); + } + assert_eq!(drain_groups(&mut subscriber), [0, 2, 4, 6, 8, 10]); + for group in open.drain(..5) { + group.abort(Error::Cancel).unwrap(); + } + elapse(&producer, budget / 2); + write_untimed_group(&producer, 12); + assert_eq!(drain_groups(&mut subscriber), [12]); + elapse(&producer, budget / 2); + write_untimed_group(&producer, 14); + assert_eq!(drain_groups(&mut subscriber), [14]); + assert_eq!(subscriber.delivered_runs(), 4, "floor through 8, then 10, 12 and 14"); + assert!(!subscriber.delivered.contains(9), "the next gap is still remembered"); + write_untimed_group(&producer, 11); + assert_eq!( + drain_groups(&mut subscriber), + [11], + "the gap below fresh group 12 remains" + ); + } + /// An untimed sparse feed whose groups all arrive within the budget keeps every gap /// and still hands out every group. The hand-out cost stays flat over the retained /// gaps; `track_sparse_read` measures it. From 53efed87c7ec585cd7713e8f7c3d6a1f82969144 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 14:32:04 -0700 Subject: [PATCH 3/4] chore(quest): accept the sparse gap flush bound and retire its quest A shared verdict retires its gaps in O(log gaps + retired entries). Each retired entry is freed exactly once, so the batch is amortized optimal. Co-Authored-By: Claude Opus 5.5 --- quest/m1/release-perf/README.md | 14 +++-- quest/m1/release-perf/sparse-gaps.md | 86 ---------------------------- 2 files changed, 10 insertions(+), 90 deletions(-) delete mode 100644 quest/m1/release-perf/sparse-gaps.md diff --git a/quest/m1/release-perf/README.md b/quest/m1/release-perf/README.md index a35788fa4bb..e00fe90957e 100644 --- a/quest/m1/release-perf/README.md +++ b/quest/m1/release-perf/README.md @@ -42,6 +42,13 @@ noise. match payload billing per label, such as per signing key. The drain runs once a second, using under 0.01% of a core at 10,000 sessions. Deleting those bytes was rejected. +- `track_gap_flush` is accepted at O(log gaps + retired entries) per shared + successor, after [#6287](https://github.com/moq-dev/moq/pull/6287) indexed + the gap ledger and retired a shared verdict's gaps with one split. Each + retired entry must be freed exactly once, so the batch is already amortized + optimal; a representation that hides that reclamation would widen scope for + no measured benefit. Sparse arrivals cost O(log gaps) amortized, and at 4096 + gaps flush, read, and expiry instructions fell 84 to 95%. ### Release as a baseline @@ -56,9 +63,9 @@ usually pays for a feature. | Bench | release | main | Ratio | Owner | |---|---|---|---|---| -| `track_gap_flush/4096` | 5.7 us | 551 us | 96x | [Sparse gaps](/quest/m1/release-perf/sparse-gaps.md) | -| `track_gap_flush/512` | 1.8 us | 48 us | 27x | same | -| `track_gap_flush/64` | 1.5 us | 6.2 us | 4.1x | same | +| `track_gap_flush/4096` | 5.7 us | 551 us | 96x | accepted (above) | +| `track_gap_flush/512` | 1.8 us | 48 us | 27x | accepted (above) | +| `track_gap_flush/64` | 1.5 us | 6.2 us | 4.1x | accepted (above) | | `stats/session-report/1000x10` | 11.0 us | 86.8 us | 7.9x | [Stats report](/quest/m1/release-perf/stats-report.md) | | `track_gc/idle/16384` | 11.6 ns | 73.7 ns | 6.3x | [Idle GC](/quest/m1/release-perf/gc-idle.md) | | `origin/resume_overlap/sequence/1r_64g_32c` | 93 us | 426 us | 4.6x | [Resume duplicates](/quest/m1/release-perf/resume-duplicates.md) | @@ -93,7 +100,6 @@ one. ## Required - [FETCH fan-out](/quest/m1/release-perf/fetch-fanout.md) - a relay serves later viewers' lite FETCHes of a group from its cache, so per-viewer cost falls with viewers again -- [Sparse gaps](/quest/m1/release-perf/sparse-gaps.md) - an arrival and a gap flush cost O(log gaps), not O(gaps) - [Resume duplicates](/quest/m1/release-perf/resume-duplicates.md) - an ordered resume skips copies of an already-delivered group without consuming them - [Idle GC](/quest/m1/release-perf/gc-idle.md) - an idle track GC skips the wall-clock pass when nothing is due - [Stats report](/quest/m1/release-perf/stats-report.md) - the stats drain locks each row once and its bench counts the wire walk; per-label bytes stay, at an accepted O(live sessions) diff --git a/quest/m1/release-perf/sparse-gaps.md b/quest/m1/release-perf/sparse-gaps.md deleted file mode 100644 index 9e50716c3ed..00000000000 --- a/quest/m1/release-perf/sparse-gaps.md +++ /dev/null @@ -1,86 +0,0 @@ -# [M] Sparse gaps - -## Goal - -On a track with many open gaps, recording an arrival and flushing gaps cost -O(log gaps). `track_sparse_read/timed` and `track_gap_flush` stop growing -faster than their size parameter. - -## Plan - -The source changes in this PR replace the ledger's vector with a sequence-keyed -`BTreeMap` and an age-keyed `BTreeSet`. Splits inherit gap ages; merges and expiry -remove both index entries. In-order arrivals extend the existing run in place. -An age index, rather than only a cached minimum plus a full scan, also keeps an -arrival cheap when one older gap expires every time. - -The cursor samples the clock once and finds the last delivered run below the -successor supplying the stale verdict. It calls `forget_below` once per shared -successor, preserving later gaps whose successor remains within budget. - -### Remaining decision - -The original goal says a gap flush costs O(log gaps). The implementation follows -the planned indexed lookup and single split, but dropping the retired map entries -necessarily costs O(retired entries). No teardown was deferred or moved outside -the operation. The bound is O(log gaps + retired entries) per shared successor; -the sparse arrival bound is O(log gaps) amortized, plus any gaps it expires. - -1. Accept this bound for a batch, with instruction sweeps confirming no repeated - work per gap (recommended). -2. Require the whole batch, including reclaiming retired storage, to be O(log - gaps). That requires a different ownership/representation design; retain the - source checkpoint while that scope is settled. - -Do not finish this quest until the maintainer settles that interpretation and the -validation limits are resolved. The original Goal above is unchanged. - -### Verification - -`track_sparse_read` and `track_gap_flush` retain their existing nightly target. -The new `track_sparse_expire` workload opens one gap while one older gap expires, -swept over 16/256/4096 retained gaps. The ledger tests check both indices after -split/merge/expiry operations, demand reordering, a replaced leading run, grace -changes, and repeated individual expiry. Wall-clock and media-time batch tests -preserve the first gap with a fresh successor. - -Profiling boundaries exclude fixture setup. Both builds warm the same initial -cache-queue capacity expansion before collection: a single iteration immediately -after a 4096-item setup otherwise measures an unrelated `VecDeque` growth copy. -The final counts measure steady-state arrival work; cold allocation/growth remains -an amortized production cost and is not claimed to be logarithmic per operation. - -Callgrind 3.27.1 with Rust 1.98.1, optimized with symbols and line tables, -compares [c78547efff](https://github.com/moq-dev/moq/commit/c78547efff) plus the -identical benchmark fixture on both builds. One -`--bench --test --exact` iteration collects only `*read_pair`, `*Expiring*arrive`, -or `*GapFlush*flush`, excluding setup. The expiring workload warms two arrivals -because the eviction queue trails the arrival queue by one entry. - -| Workload | Gaps | Before instructions | After instructions | -|---|---|---|---| -| read | 16 | 29,591 | 30,838 | -| read | 256 | 43,625 | 34,389 | -| read | 4096 | 315,641 | 40,856 | -| expire | 16 | 15,484 | 16,242 | -| expire | 256 | 23,780 | 17,791 | -| expire | 4096 | 138,439 | 22,050 | -| flush | 16 | 38,996 | 25,405 | -| flush | 256 | 443,152 | 49,921 | -| flush | 4096 | 9,106,805 | 463,636 | - -At 4096 gaps, read-pair instructions fall 87.1%, steady expiry 84.1%, and flush -94.9%. At 16 gaps, the indices add 4.2% to a read pair and 4.9% to expiry; the -larger retained-gap costs no longer come from full ledger walks. Flush retains -linear storage reclamation rather than charging that work elsewhere. - -Validation: all 2918 moq-net tests pass (eight ignored nightly tests). `just ci` -passes Rust lint/docs/wasm, quest validation, and drill applicability, then fails -on the unchanged Nix runner image because this host cannot write -`/proc/self/uid_map`. Broader test results remain recorded in the PR. - -Found in the release comparison. The arrival ledger came from -[moq-dev/moq#5802](https://github.com/moq-dev/moq/pull/5802), delivery records from -[moq-dev/moq#5446](https://github.com/moq-dev/moq/pull/5446), and wall-clock aging -from [moq-dev/moq#5548](https://github.com/moq-dev/moq/pull/5548). Those features -remain; only their accidental repeated work changes. From 812d4ba2531b4128313cf38834ad9c072925b803 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 15:24:22 -0700 Subject: [PATCH 4/4] fix(quest): record the re-measured sparse-gap expiry drop Co-Authored-By: Claude Opus 5.5 --- quest/m1/release-perf/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/release-perf/README.md b/quest/m1/release-perf/README.md index e00fe90957e..935acc8747f 100644 --- a/quest/m1/release-perf/README.md +++ b/quest/m1/release-perf/README.md @@ -48,7 +48,7 @@ noise. retired entry must be freed exactly once, so the batch is already amortized optimal; a representation that hides that reclamation would widen scope for no measured benefit. Sparse arrivals cost O(log gaps) amortized, and at 4096 - gaps flush, read, and expiry instructions fell 84 to 95%. + gaps flush, read, and expiry instructions fell 80 to 95%. ### Release as a baseline