From fab8899ff5530a96cf2024725cee0884316b0190 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 17 Nov 2026 17:21:41 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m1/precut-fixes/relay-high-key.md Quest-Claim: 73ab3acf97698c91 From 9b11a255b42780e653f0163a7f9e1b72b948aa62 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 17 Nov 2026 17:50:25 -0800 Subject: [PATCH 2/3] fix(net): a far-ahead group key pins no arrival entries behind it Arrival entries carry their own position, so dead ones are compacted from anywhere once they outnumber the cached groups, and the arrival walk moves a reader's cursor past leading entries it is never handed. A front forgets a track's metadata with the track. Co-Authored-By: Claude Opus 5.5 --- quest/m0/backport-relay-high-key.md | 29 ++++ quest/m1/precut-fixes/relay-high-key.md | 37 ---- rs/moq-net/benches/track.rs | 97 +++++++++++ rs/moq-net/src/model/front.rs | 15 +- rs/moq-net/src/model/track.rs | 217 +++++++++++++++++------- 5 files changed, 291 insertions(+), 104 deletions(-) create mode 100644 quest/m0/backport-relay-high-key.md delete mode 100644 quest/m1/precut-fixes/relay-high-key.md diff --git a/quest/m0/backport-relay-high-key.md b/quest/m0/backport-relay-high-key.md new file mode 100644 index 00000000000..49f77abe625 --- /dev/null +++ b/quest/m0/backport-relay-high-key.md @@ -0,0 +1,29 @@ +# [M] Backport the far-ahead group key fix to release + +## Goal + +A relay on `release` holds no arrival entries for groups it no longer caches, +and a reader's wake no longer walks groups below its floor, so one publisher +that jumps to a far-ahead group key and keeps publishing below it cannot grow +a relay's memory or burn its shared runtime's CPU. Ships in a `moq-net` patch +in the next release. + +## Plan + +Decided 2026-11-17 (maintainer, [moq-dev/moq#9010](https://github.com/moq-dev/moq/pull/9010#issuecomment-6959731587)): ✅ +backport the track fix to `release` as a separate PR after it lands on +`main`, per RELEASE.md (rejected: shipping it only with the cut). + +Port the track half of the fix from +[moq-dev/moq#9044](https://github.com/moq-dev/moq/pull/9044) in +`rs/moq-net/src/model/track.rs`: arrival entries carry their own position, so +dead ones are compacted from anywhere once they outnumber the cached groups, +and the arrival walk moves a reader's cursor past leading entries it is never +handed (dead, or below its floor). `release` still indexes the queue by an +`offset`, so this is a hand port, not a cherry-pick. Reproduce first with the +PR's `a_far_ahead_key_pins_nothing_behind_it` test; if `release` cannot hit it, +delete this quest and say why in the PR. + +The front's track-info cleanup stays on `main`: it is not part of the decision. + +Public API: none. Wire: none. diff --git a/quest/m1/precut-fixes/relay-high-key.md b/quest/m1/precut-fixes/relay-high-key.md deleted file mode 100644 index f17db6f3d2c..00000000000 --- a/quest/m1/precut-fixes/relay-high-key.md +++ /dev/null @@ -1,37 +0,0 @@ -# [M] A publisher's far-ahead group key costs a relay nothing extra - -## Goal - -A publisher that sends one group with a huge key and then keeps publishing -below it makes a relay hold memory bounded by what it caches, and each -subscriber wake costs work bounded by what that subscriber can still be -handed, not by every group the track ever received. A subscriber asking a -broadcast for endless track names grows nothing once each name is forgotten. - -## Plan - -Found by the 2026-11-17 pre-cut correctness audit, from source; reproduce -first. Both are on `release` too. - -- `rs/moq-net/src/model/track.rs`: `insert_group` makes the highest key the - live edge, `protects` keeps it from eviction, and the arrival trim only pops - from the front, so behind a protected `1<<40` group nothing is ever trimmed - (16 B per group forever). `poll_recv_group` skips entries below a reader's - `min_key` without advancing its index, so every insert wakes every live - subscriber into a walk over all N stale entries under the track mutex: - quadratic CPU on the relay's shared runtime. Recommended: compact - `arrival` the way `evict` is rebuilt, and let a reader's cursor skip - entries it will never be handed. -- `rs/moq-net/src/model/front.rs`: the front's track-info map keeps entries - past `Forgotten`; drop them there. - -Decided 2026-11-17 (maintainer, [moq-dev/moq#9010](https://github.com/moq-dev/moq/pull/9010#issuecomment-6959731587)): ✅ -backport the track fix to `release` as a separate PR after it -lands on `main`, per RELEASE.md, since one publisher can burn a shared -relay's CPU (rejected: shipping it only with the cut). - -Test: a publisher writes key `1<<40`, then keys 0..N; assert the arrival -queue stays bounded and a live subscriber's per-wake scan does not grow with -N. Add a bench sweep over N and subscribers, per AGENTS.md's fan-out rule. - -Public API: none. Wire: none. diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 2da36f3f781..5bc20ded3dc 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -120,6 +120,13 @@ //! refusals and readers. Each reader consumes each refusal before the next, so the cost per //! refusal must not grow with the history of answers. //! +//! `track_high_key` writes one group below a far-ahead key every reader already took, behind +//! a backlog of groups written there before, swept over the readers and the backlog: the +//! protected live edge must pin none of it, and a reader never hands out a group under its +//! floor, so a walk over the backlog on each write shows up as a slope over it. Every +//! iteration adds to the backlog too, so such a walk also lifts every point far above one +//! reader's constant. +//! //! Run with `cargo bench -p moq-net --bench track`. #![expect( @@ -339,6 +346,95 @@ impl Fanout { } } +/// A track whose readers took a far-ahead key, as the publisher writes on below it. +struct HighKey { + _broadcast: broadcast::Producer, + track: track::Producer, + subscribers: Vec, + waiter: kio::Waiter, + payload: Bytes, + next: u64, +} + +impl HighKey { + fn new(subscribers: usize, backlog: usize) -> Self { + let mut info = broadcast::Info::default(); + let config = cache::Config::default() + .with_capacity(CACHE_CAPACITY) + .with_expiry(cache::DEFAULT_EXPIRY); + info.pool = cache::Pool::new(config); + let broadcast = broadcast::Producer::new(info); + let track = broadcast + .create_track("bench", moq_net::track::Info::default().with_max_age(None)) + .unwrap(); + // Each reads from the far-ahead key on, so every later group lands below its floor. + let mut subscribers: Vec<_> = (0..subscribers) + .map(|_| track.subscribe(None).with_range(HIGH..)) + .collect(); + let waiter = kio::Waiter::noop(); + let mut high = Self { + _broadcast: broadcast, + track, + subscribers: Vec::new(), + waiter, + payload: Bytes::from(vec![0; PAYLOAD]), + next: 0, + }; + high.write(HIGH); + for subscriber in &mut subscribers { + let Poll::Ready(Ok(Some(group))) = subscriber.poll_recv_group(&high.waiter) else { + unreachable!("the far-ahead group must be ready"); + }; + black_box(group); + } + high.subscribers = subscribers; + for _ in 0..backlog { + high.step(); + } + high + } + + fn write(&self, key: u64) { + let mut group = self.track.create_group(moq_net::group::Info { start: key }).unwrap(); + group + .write_frame(Timestamp::unscaled(key), self.payload.clone()) + .unwrap(); + group.finish().unwrap(); + } + + /// Write the next group below the far-ahead key and wake every reader, none of which + /// hands it out. + fn step(&mut self) { + self.write(self.next); + self.next += 1; + for subscriber in &mut self.subscribers { + assert!(subscriber.poll_recv_group(&self.waiter).is_pending()); + } + } +} + +/// The far-ahead key `track_high_key` jumps to first. +const HIGH: u64 = 1 << 40; + +/// Groups written below the far-ahead key before `track_high_key` measures. +const BACKLOG: [usize; 3] = [0, 1024, 16_384]; + +fn bench_high_key(c: &mut Criterion) { + let mut group = c.benchmark_group("track_high_key"); + for subscribers in FANOUT { + for backlog in BACKLOG { + group.throughput(Throughput::Elements(subscribers as u64)); + // Criterion calls the closure once per sample, so the fixture is built on the first call and kept. + let mut fixture = None; + group.bench_function(BenchmarkId::new(format!("{subscribers}r"), backlog), |b| { + let high = fixture.get_or_insert_with(|| HighKey::new(subscribers, backlog)); + b.iter(|| high.step()); + }); + } + } + group.finish(); +} + /// A cached aborted prefix followed by one live group. struct AbortedScan { _broadcast: broadcast::Producer, @@ -2232,6 +2328,7 @@ criterion_group!( bench_fanout, bench_fanout_frame, bench_evict_fanout, + bench_high_key, bench_fetch_expiry, bench_fetch_budgets, bench_fetch_park, diff --git a/rs/moq-net/src/model/front.rs b/rs/moq-net/src/model/front.rs index ac11d384dac..7b5541760e1 100644 --- a/rs/moq-net/src/model/front.rs +++ b/rs/moq-net/src/model/front.rs @@ -239,8 +239,8 @@ pub(super) struct Front { resolved: bool, tracks: BTreeMap, Track>, parked: BTreeSet<(Instant, Arc)>, - /// Immutable track metadata, fixed by the first copy of each track. Every - /// source of the broadcast must serve the same content. + /// Immutable track metadata, fixed by the first copy of each track until it is + /// forgotten. Every source of the broadcast must serve the same content. info: BTreeMap, track::Info>, /// How long an unread track stays before it is forgotten. linger: Duration, @@ -330,9 +330,9 @@ impl Front { Event::Resolved { route, result } => self.resolved(route, result, &mut actions), Event::SourceClosed { source } => self.source_closed(source, &mut actions), // A fresh logical track, even under a name served before: an earlier - // verdict belonged to that request, and a later one asks afresh. The - // broadcast's track metadata is what persists. Unread until a reader - // shows up, so a lookup whose reader left early is forgotten too. + // verdict and metadata belonged to that request, and a later one asks + // afresh. Unread until a reader shows up, so a lookup whose reader left + // early is forgotten too. Event::TrackAssigned { track, now } => { self.tracks.insert( track, @@ -365,8 +365,10 @@ impl Front { Event::Wanted { track } => self.wanted(track, &mut actions), Event::Unwanted { track, now } => self.unwanted(track, now, &mut actions), Event::Deadline { now } => self.deadline(now, &mut actions), + // Its metadata goes too, so a peer naming endless tracks grows nothing. Event::Forgotten { track } => { self.tracks.remove(&track); + self.info.remove(&track); } Event::Held => { self.held = true; @@ -2099,8 +2101,11 @@ mod tests { }), &[Action::Forget { track: name("video") }], ); + assert!(front.info.contains_key(&name("video"))); assert_actions(front.step(Event::Forgotten { track: name("video") }), &[]); assert!(!front.tracks.contains_key(&name("video"))); + // Its metadata goes with it, so endless track names grow nothing. + assert!(!front.info.contains_key(&name("video"))); } /// A reader that looked the track up before the driver could forget it keeps diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 4a8f327b4cb..060cad2d24c 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -43,7 +43,8 @@ pub(crate) use hole::Hole; pub(crate) use fill::{GroupFill, Readers as FillReaders}; use std::{ - collections::{BTreeMap, BTreeSet, VecDeque, btree_map}, + cell::Cell, + collections::{BTreeMap, BTreeSet, VecDeque, btree_map, vec_deque}, ops::{self, Bound, RangeBounds}, sync::Arc, sync::OnceLock, @@ -598,12 +599,13 @@ pub(crate) struct TrackState { // [`Hold`], which a reader of that copy waits out when it fails; see [`successor`]. holds: BTreeMap>, - // Publisher-produced groups in arrival order as (key, stamp), walked by - // subscriptions; an entry only resolves while its stamp matches the slot's. - // Fetched backfill (`insert_group_request`) is deliberately absent: it is - // served by key, never replayed to arrival-order subscribers. A fetched - // head of a live group is the exception, served through that group's entry. - arrival: VecDeque<(u64, u32)>, + // Publisher-produced groups in arrival order, walked by subscriptions; an entry + // only resolves while its stamp matches the slot's. Fetched backfill + // (`insert_group_request`) is deliberately absent: it is served by key, never + // replayed to arrival-order subscribers. A fetched head of a live group is the + // exception, served through that group's entry. Dead entries are dropped anywhere, + // not just at the front, so the protected live edge pins none behind it. + arrival: VecDeque, // Eviction order under memory pressure as (key, stamp): every cached // group except the protected latest. `pay_debt` scans victims from the front; @@ -620,9 +622,9 @@ pub(crate) struct TrackState { // what each track writes and never touches another track's cache. debt: u64, - // We've popped the front of `arrival` this many times, mapping a subscriber's - // absolute cursor to an index. - offset: usize, + // The position the next entry of `arrival` takes: positions only grow, so a reader's + // cursor stays valid as dead entries leave from anywhere. + next_arrival: usize, // The highest key successfully appended to the track, which can run ahead // of any cached group. @@ -770,6 +772,15 @@ pub(crate) struct TrackState { routes: Option, } +/// A publisher-produced group in [`TrackState::arrival`]. +#[derive(Clone, Copy, Debug)] +struct Arrived { + /// Its position in arrival order, which a reader's cursor counts in. + at: usize, + key: u64, + stamp: u32, +} + /// The starts a group's first frame must fall between: a timed track never holds a group /// whose start is at or before the one before it. Read under the track lock and checked /// under it, so no two groups stamped concurrently can each accept a rewind. @@ -1733,22 +1744,31 @@ impl TrackState { /// Returns the group and its absolute index so the consumer can advance past it. /// The cached producer rather than a consumer: `consume` reads the clock and can /// take the group's own lock, which the caller does once the track guard is gone. + /// + /// Moves `index` past the leading entries this reader is never handed: dead, or below + /// `min_key`, which a lower floor re-offers from the cache instead. So a wake walks + /// only what the reader may still be handed, however many stale entries arrived. fn poll_recv_group( &self, - index: usize, + index: &Cell, min_key: u64, window: &Window, ) -> Poll>> { - let start = index.saturating_sub(self.offset); - let readable = self.serves_bounded(window); - for (i, (key, stamp)) in self.arrival.iter().enumerate().skip(start).filter(|_| readable) { - // A cut group is handed out too, so its reader sees why it ended. - if *key >= min_key - && let Some(slot) = self.lookup.get(key) - && slot.stamp == *stamp - && slot.is_read() - { - return Poll::Ready(Ok(Some((slot.serving().clone(), self.offset + i)))); + if self.serves_bounded(window) { + let mut passed = true; + for entry in self.arrivals_from(index.get()) { + let slot = self.lookup.get(&entry.key).filter(|slot| slot.stamp == entry.stamp); + let Some(slot) = slot.filter(|_| entry.key >= min_key) else { + if passed { + index.set(entry.at + 1); + } + continue; + }; + // A cut group is handed out too, so its reader sees why it ended. + if slot.is_read() { + return Poll::Ready(Ok(Some((slot.serving().clone(), entry.at)))); + } + passed = false; } } @@ -2416,12 +2436,13 @@ impl TrackState { || self .arrival .front() - .is_some_and(|(key, stamp)| !self.is_current(*key, *stamp)) + .is_some_and(|entry| !self.is_current(entry.key, entry.stamp)) || self .evict .front() .is_some_and(|(key, stamp)| !self.is_current(*key, *stamp)) || self.evict.len() > 2 * self.lookup.len() + EVICT_SLACK + || self.arrival.len() > 2 * self.lookup.len() + EVICT_SLACK } /// Expire an ended track's idle groups, whose closed channel refuses the write @@ -2488,15 +2509,15 @@ impl TrackState { self.expire_drops(scan.now); - // Trim dead leading arrival entries to advance the subscriber offset. An - // entry is dead once its slot is gone or re-stamped by a newer incarnation. - while let Some((key, stamp)) = self.arrival.front() { - if self.lookup.get(key).is_some_and(|slot| slot.stamp == *stamp) { + // Trim dead leading arrival entries. An entry is dead once its slot is gone or + // re-stamped by a newer incarnation. + while let Some(entry) = self.arrival.front() { + if self.is_current(entry.key, entry.stamp) { break; } self.arrival.pop_front(); - self.offset += 1; } + self.compact_arrival(); // Drop dead leading eviction entries so scans stay over live candidates. while let Some((key, stamp)) = self.evict.front() { @@ -2515,6 +2536,23 @@ impl TrackState { } } + /// Drop the dead arrival entries once they clearly outnumber the cached groups, wherever + /// they sit: behind a protected live edge, none ever reaches the front. A rebuild drops + /// more entries than it keeps, each pushed once, so it costs a constant per push. + fn compact_arrival(&mut self) { + if self.arrival.len() > 2 * self.lookup.len() + EVICT_SLACK { + let lookup = &self.lookup; + self.arrival + .retain(|entry| lookup.get(&entry.key).is_some_and(|slot| slot.stamp == entry.stamp)); + } + } + + /// The arrival entries at or after position `at`. + fn arrivals_from(&self, at: usize) -> vec_deque::Iter<'_, Arrived> { + let start = self.arrival.partition_point(|entry| entry.at < at); + self.arrival.range(start..) + } + /// Whether `(key, stamp)` names the currently cached incarnation. fn is_current(&self, key: u64, stamp: u32) -> bool { self.lookup.get(&key).is_some_and(|slot| slot.stamp == stamp) @@ -2848,7 +2886,13 @@ impl TrackState { if visible { // Readers and group demand reach it at once. Its arrival starts the wall clock of // the reads it succeeds, but it counts as content only once it starts. - self.arrival.push_back((key, stamp)); + self.arrival.push_back(Arrived { + at: self.next_arrival, + key, + stamp, + }); + self.next_arrival += 1; + self.compact_arrival(); // Publishes the landing to reads judging expiry without the track lock, then // checks for one parked. Reveal only holds a read lock and cannot use the shortcut. if self.cache.wakes().land(key) { @@ -7847,13 +7891,12 @@ impl Indexed { /// Take in what arrived since the last call. fn catch_up(&mut self, state: &TrackState, fixed: &Runs) { - let skip = self.taken.saturating_sub(state.offset).min(state.arrival.len()); - for (key, stamp) in state.arrival.range(skip..) { - if fixed.contains(*key) { - self.groups.insert(*key, *stamp); + for entry in state.arrivals_from(self.taken) { + if fixed.contains(entry.key) { + self.groups.insert(entry.key, entry.stamp); } } - self.taken = state.offset + state.arrival.len(); + self.taken = state.next_arrival; // A live entry is a visible slot, so past twice their count most entries are stale, // and a prune costs each entry it drops one lookup. if self.groups.len() > 2 * state.visible.len().max(LIVE_BATCH) { @@ -8486,13 +8529,11 @@ impl Cursor { /// cursor already read past them, so it never hands them out. fn late(&mut self, below: u64) -> Vec { let state = self.state.read(); - let from = self.late.max(state.offset); - self.late = state.offset + state.arrival.len(); + let from = self.late; + self.late = state.next_arrival; state - .arrival - .iter() - .skip(from - state.offset) - .map(|(key, _)| *key) + .arrivals_from(from) + .map(|entry| entry.key) .filter(|key| *key < below && self.filter.window.contains(*key)) .collect() } @@ -9264,9 +9305,11 @@ impl Cursor { Some((consumer, taken)) => (consumer, taken), None => { let floor = self.floor(); + let index = Cell::new(self.index); let polled = self.poll(waiter, |state| { - state.poll_recv_group(self.index, floor, &self.filter.window) + state.poll_recv_group(&index, floor, &self.filter.window) }); + self.index = index.get(); // A moved window moved the servable edge `drift` was judged against. if polled.is_pending() && self.watch_time() { self.admit(); @@ -10212,17 +10255,13 @@ impl Subscriber { }) }; state.readable() - && (state - .arrival - .iter() - .skip(cursor.index.saturating_sub(state.offset)) - .any(|(key, stamp)| { - eligible(*key) - && state - .lookup - .get(key) - .is_some_and(|slot| slot.stamp == *stamp && slot.is_read()) - }) || owed()) + && (state.arrivals_from(cursor.index).any(|entry| { + eligible(entry.key) + && state + .lookup + .get(&entry.key) + .is_some_and(|slot| slot.stamp == entry.stamp && slot.is_read()) + }) || owed()) } /// The keys that landed on this copy since the last call, below `below`. @@ -11875,13 +11914,18 @@ mod test { state.lookup.len() } + /// Helper: the position of the first arrival entry still held, or the next one's. + fn first_arrival(state: &TrackState) -> usize { + state.arrival.front().map_or(state.next_arrival, |entry| entry.at) + } + /// Helper: get the key of the first live group in arrival order. fn first_live_key(state: &TrackState) -> u64 { state .arrival .iter() - .find(|(key, stamp)| state.lookup.get(key).is_some_and(|slot| slot.stamp == *stamp)) - .map(|(key, _)| *key) + .find(|entry| state.is_current(entry.key, entry.stamp)) + .map(|entry| entry.key) .unwrap() } @@ -13118,7 +13162,7 @@ mod test { { let state = producer.state.read(); assert_eq!(live_groups(&state), 3); - assert_eq!(state.offset, 0); + assert_eq!(first_arrival(&state), 0); } // Advance time past the pool's LRU window. @@ -13134,7 +13178,7 @@ mod test { let state = producer.state.read(); assert_eq!(live_groups(&state), 1); assert_eq!(first_live_key(&state), 3); - assert_eq!(state.offset, 3); + assert_eq!(first_arrival(&state), 3); assert!(!state.lookup.contains_key(&0)); assert!(!state.lookup.contains_key(&1)); assert!(!state.lookup.contains_key(&2)); @@ -13378,10 +13422,59 @@ mod test { let state = producer.state.read(); assert_eq!(live_groups(&state), 1); assert_eq!(first_live_key(&state), 1); - assert_eq!(state.offset, 1); + assert_eq!(first_arrival(&state), 1); } } + /// A publisher that jumps far ahead and then keeps publishing below it holds the + /// relay to what it caches: the protected high key never pins the dead entries + /// behind it, and a live reader past them never walks them again. + #[test] + fn a_far_ahead_key_pins_nothing_behind_it() { + let producer = track_producer("test", Info::timed()); + let high = 1 << 40; + // Read from the high key on: everything after lands below this reader's floor. + let mut subscriber = producer.subscribe(Subscription::default()).with_range(high..); + let waiter = kio::Waiter::noop(); + producer + .create_group(group::Info { start: high }) + .unwrap() + .finish() + .unwrap(); + let Poll::Ready(Ok(Some(group))) = subscriber.poll_recv_group(&waiter) else { + panic!("the high group is handed out"); + }; + assert_eq!(group.start, high); + + let n = 4 * EVICT_SLACK as u64; + for key in 0..n { + elapse(&producer, cache::DEFAULT_EXPIRY + Duration::from_secs(1)); + producer + .create_group(group::Info { start: key }) + .unwrap() + .finish() + .unwrap(); + // Below the reader's floor: never handed out. + assert!(subscriber.poll_recv_group(&waiter).is_pending()); + } + + let state = producer.state.read(); + assert!(state.lookup.contains_key(&high), "the live edge stays"); + assert!( + state.arrival.len() <= 2 * state.lookup.len() + EVICT_SLACK, + "{} arrival entries for {} cached groups", + state.arrival.len(), + state.lookup.len() + ); + let Inner::Plain(cursor) = &subscriber.inner else { + unreachable!() + }; + assert_eq!( + cursor.index, state.next_arrival, + "the reader passed what it is never handed" + ); + } + #[test] fn no_eviction_when_fresh() { let producer = track_producer("test", Info::timed()); @@ -13392,7 +13485,7 @@ mod test { { let state = producer.state.read(); assert_eq!(live_groups(&state), 3); - assert_eq!(state.offset, 0); + assert_eq!(first_arrival(&state), 0); } } @@ -16974,7 +17067,7 @@ mod test { { let state = producer.state.read(); assert_eq!(live_groups(&state), 2); - assert_eq!(state.offset, 0); + assert_eq!(first_arrival(&state), 0); } // Expire seq 3 as well. @@ -16994,7 +17087,7 @@ mod test { { let state = producer.state.read(); assert_eq!(live_groups(&state), 2); - assert_eq!(state.offset, 0); + assert_eq!(first_arrival(&state), 0); assert!(state.lookup.contains_key(&5)); assert!(!state.lookup.contains_key(&3)); assert!(state.lookup.contains_key(&2)); @@ -23045,14 +23138,14 @@ mod test { // The backfill serves by key, but never in arrival order. assert!(consumer.peek_group(1).is_some()); let mut walked = Vec::new(); - let mut index = 0; + let index = Cell::new(0); let window = { let state = producer.state.read(); Window::new(&replay().ranges, None, None, &*state).unwrap() }; - while let Poll::Ready(Ok(Some((group, next)))) = producer.state.read().poll_recv_group(index, 0, &window) { + while let Poll::Ready(Ok(Some((group, next)))) = producer.state.read().poll_recv_group(&index, 0, &window) { walked.push(group.start); - index = next + 1; + index.set(next + 1); } assert_eq!(walked, [0, 2]); // A reader takes it as the fill answer for the group the feed gave up, as it would From 1a967c3010460f4b0f436e2cb5a5bfe386777f24 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 17 Nov 2026 20:16:42 -0800 Subject: [PATCH 3/3] fix(net): budget the Arrived entry and drop stale offset comments Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 060cad2d24c..0ac874494b1 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -1009,8 +1009,12 @@ impl Slot { /// `BTreeMap` node sits between half and fully packed, and a `VecDeque` holds up to /// twice the entries in it. Half of [`cache::ENTRY_OVERHEAD`]; see it for why this is /// derived rather than measured. -pub(super) const CACHE_OVERHEAD: u64 = - 2 * (2 * size_of::() + size_of::() + size_of::<(u64, Timestamp)>() + 2 * size_of::<(u64, u32)>()) as u64; +pub(super) const CACHE_OVERHEAD: u64 = 2 + * (2 * size_of::() + + size_of::() + + size_of::<(u64, Timestamp)>() + + size_of::() + + size_of::<(u64, u32)>()) as u64; /// Heap the track keeps per run of its drop set: the run and its reason in `drops`, at most /// one coalesced run in `dropped`, and its two entries in `drops_changes`. Doubled for half-full `BTreeMap` nodes, as @@ -13172,8 +13176,8 @@ mod test { producer.append_next().unwrap().finish().unwrap(); // seq 3 // Groups 0, 1, 2 are expired but seq 3 (the live edge) is kept. Their arrival - // entries no longer resolve, so the leading ones are trimmed and the offset - // advances past them. + // entries no longer resolve, so the leading ones are trimmed and the queue now + // starts at seq 3's position. { let state = producer.state.read(); assert_eq!(live_groups(&state), 1); @@ -17081,9 +17085,9 @@ mod test { .unwrap(); // Seq 5 is the live edge (protected) and still resolves at the front of - // `arrival`, so nothing is trimmed and the offset stays. Seq 3 expired out of - // `lookup`, leaving a hole its arrival entry no longer resolves; seq 2 is - // fresh and kept. + // `arrival`, so nothing is trimmed and the queue still starts at position 0. + // Seq 3 expired out of `lookup`, leaving a hole its arrival entry no longer + // resolves; seq 2 is fresh and kept. { let state = producer.state.read(); assert_eq!(live_groups(&state), 2);