From 5645bd5be529ff34bea43772ad13ec656d00e378 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 17:11:46 -0700 Subject: [PATCH 1/5] chore(quest): claim 3161-retention-should-reclaim-idle-open-groups-now-that-expiry Claim UUID: 84956197-6a07-4815-be7e-ae7bbca8fb4e Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01DLtoPv9B3779kGxLFTUGHR From c57b0987d441d2f4abe476d24556c1afac63a2c1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 18:28:58 -0700 Subject: [PATCH 2/5] fix(net): reclaim an idle open group without waiting for a write The cache pool's wall-clock LRU window was only ever enforced from the write path: `cache::Track::settle` runs the expiry scan as a track writes frames or creates groups. A publisher that stalls with a group still open stops writing, so nothing ran the scan for that track, the group pinned its buffer indefinitely, and a reader parked inside it was never told. Subscription expiry cannot cover it: since #3099 that is timestamp-only, so a successor starting less than the max age budget ahead in media time correctly keeps the group. The pool now owns both halves of its own window. A pool with an expiry keeps a registry of its live track accounts, `Pool::sweep` runs each track's usual bounded scan, and `origin::Driver` drives it on a wall-clock cadence (half the window) using the timers it already holds. A pool with no expiry registers nothing and pays nothing. js/net had the same gap twice over: the track prune deliberately retained every open group, and it only ran on a publish or a subscribe. It now ages a group from its last write, evicts an idle open group (ending it with `Lagged`, since the publisher never closed it), and arms a timer for the next entry due. The newest group is protected while it is still open, which is the live edge a publisher legitimately keeps open across a quiet stretch (a catalog snapshot, a JSON stream). Completes /quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md Closes #3161 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01DLtoPv9B3779kGxLFTUGHR --- doc/bin/relay/config.md | 14 +- js/net/src/group.ts | 14 ++ js/net/src/track.test.ts | 46 ++++++ js/net/src/track.ts | 56 +++++++- ...et-shedding-drops-discontinuity-markers.md | 1 - ...eclaim-idle-open-groups-now-that-expiry.md | 30 ---- quest/m1/README.md | 1 - quest/m1/group-overflow-abort.md | 7 +- rs/moq-net/src/model/cache.rs | 131 ++++++++++++++++-- rs/moq-net/src/model/origin.rs | 121 +++++++++++++++- rs/moq-net/src/model/track.rs | 26 +++- 11 files changed, 389 insertions(+), 58 deletions(-) delete mode 100644 quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md diff --git a/doc/bin/relay/config.md b/doc/bin/relay/config.md index acc30b35940..92362df517f 100644 --- a/doc/bin/relay/config.md +++ b/doc/bin/relay/config.md @@ -659,12 +659,14 @@ slack below physical memory (or just use `headroom`, which measures actual available memory). `duration` is the age counterpart: it stops idle history from pinning memory when the byte budget alone leaves room for it. -All eviction happens as tracks write (there is no background reaper), so both -`duration` and the byte budget cap how much history *active* publishers build -up. A publisher that stops writing but stays connected keeps what it had cached -until it resumes or the broadcast closes; under memory pressure the byte budget -is repaid by the tracks that are still writing. A publisher that disconnects has -its groups released as soon as the broadcast closes with it. +The byte budget is repaid as tracks write, so it caps how much history *active* +publishers build up: a publisher that stops writing pays none of it down, and +under memory pressure the budget is repaid by the tracks that are still writing. +`duration` does not depend on writes. The relay sweeps every cached track on a +wall-clock cadence, so a publisher that stalls but stays connected still has its +idle groups reclaimed, including one it left a group open in, and a subscriber +parked inside that group is told rather than waiting forever. A publisher that +disconnects has its groups released as soon as the broadcast closes with it. All three flags also accept CLI arguments (`--cache-capacity`, `--cache-headroom`, `--cache-duration`) and environment variables diff --git a/js/net/src/group.ts b/js/net/src/group.ts index dba0251a163..ec1eba04832 100644 --- a/js/net/src/group.ts +++ b/js/net/src/group.ts @@ -137,11 +137,24 @@ export class Producer { // so this is what stops a reader-less fetch instead of the stream ending on its own. #used = new Signal(false); + // When the group was created or last written. Retention ages a group from this, + // not from its creation, so a group still being filled is never reclaimed. + #activity = performance.now(); + constructor(sequence: number) { this.#state = new GroupState(sequence); this.sequence = sequence; } + /** + * When the group was created or last written, in `performance.now()` milliseconds. + * + * @internal Track retention only. + */ + get activity(): number { + return this.#activity; + } + /** * Settles once the group closes: `null` on a clean close, or the abort {@link Error}. * Peek it synchronously (`undefined` while open), observe it reactively, or `await` it. @@ -228,6 +241,7 @@ export class Producer { // `Error::FrameTooLarge`; do the same rather than silently dropping it. if (frame.payload.byteLength > MAX_GROUP_CACHE_BYTES) throw new FrameTooLarge(); + this.#activity = performance.now(); appendFrame(this.#state, frame); if (this.#mirrors) { diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index a064e2746db..ae3815e01cb 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -804,6 +804,52 @@ test("retention pruning aborts a held mirror without a new live edge", async () } }); +test("retention reclaims a group the publisher abandoned open", async () => { + const clock = mockMonotonicTime(10_000); + try { + const producer = new TrackProducer("test").accept({ maxAge: 100 }); + const track = producer.subscribe({ maxAge: 100 }); + const stalled = producer.appendGroup(); + stalled.writeString("first"); + // A successor, so the stalled group is not the live edge the publisher is + // still filling. It is never closed: the publisher simply walked away. + producer.appendGroup(); + + const group = await track.recvGroup(); + if (!group) throw new Error("missing group"); + expect(await group.readString()).toBe("first"); + + clock.set(10_200); + producer.subscribe({ maxAge: 100 }); + + // A gap, not a clean finish: the publisher never ended this group. + await expect(group.readFrame()).rejects.toBeInstanceOf(Lagged); + } finally { + clock.restore(); + } +}); + +test("an abandoned open group ages out with no further write", async () => { + // Real time, since this is about the wakeup: nothing writes to the track again, so + // without a timer the read below parks forever. + const producer = new TrackProducer("test").accept({ maxAge: 30 }); + const track = producer.subscribe({ maxAge: 30 }); + const stalled = producer.appendGroup(); + stalled.writeString("first"); + producer.appendGroup(); + + const group = await track.recvGroup(); + if (!group) throw new Error("missing group"); + expect(await group.readString()).toBe("first"); + + const read = group.readFrame().then( + () => "clean end", + (err: unknown) => err, + ); + const timeout = new Promise((resolve) => setTimeout(() => resolve("still parked"), 1000)); + expect(await Promise.race([read, timeout])).toBeInstanceOf(Lagged); +}); + test("retention pruning preserves clean EOF for a drained mirror", async () => { const clock = mockMonotonicTime(10_000); try { diff --git a/js/net/src/track.ts b/js/net/src/track.ts index f23b4938213..20f0e840b22 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -365,11 +365,16 @@ export class Producer { #sequence: TrackSequence = { next: 0 }; // Recently written source groups, retained for replay to late subscribers and - // pruned once closed and older than the cache window. Each entry tracks the mirror + // pruned once idle for longer than the cache window. Each entry tracks the mirror // it handed to every sink so eviction can drop them too: otherwise a slow consumer // that never reads would pin old groups (and their frame bytes) forever. #cache: CachedGroup[] = []; + // Wakeup for the next entry due to age out. Writes settle retention inline, but a + // publisher that stalls stops writing, so without this an abandoned group (and any + // reader parked in it) would wait for a write that never comes. + #pruneTimer?: ReturnType; + // One independent downstream state per live subscriber. #sinks = new Set(); @@ -533,6 +538,11 @@ export class Producer { // Drop a cached group's mirror from every sink so no consumer can pin it. #evict(entry: CachedGroup): void { + // Reclaiming a group the publisher never closed ends it as the gap it is, before + // the mirrors below would otherwise report a clean finish to a reader that had + // drained it. The usual case, an already-closed group aging out, keeps its own + // terminal state. + if (!entry.group.isClosed) entry.group.close(new Lagged()); for (const [sink, mirror] of entry.mirrors) { hooks.evictGroup(mirror); sink.groups.mutate((groups) => { @@ -545,20 +555,58 @@ export class Producer { entry.mirrors.clear(); } - // Evict cached groups that are closed and older than the cache window. + // The one group retention never takes: the newest, while it is still open. That is + // the live edge a publisher is appending to, and a track may legitimately keep it + // open across a long quiet stretch (a catalog snapshot, a JSON stream). Every other + // open group is an abandoned one, and ages out like a closed one. + #liveEdge(): GroupProducer | undefined { + const latest = this.#cache.at(-1)?.group; + return latest?.closed.peek() === undefined ? latest : undefined; + } + + // Evict cached groups idle for longer than the cache window. Idle means nothing + // written, so an abandoned open group ages out instead of pinning its buffer (and + // any reader parked in it) forever. #prune(): void { const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS; const cutoff = performance.now() - maxAgeMs; + const live = this.#liveEdge(); const retained: CachedGroup[] = []; for (const entry of this.#cache) { - if (entry.time >= cutoff || entry.group.closed.peek() === undefined) { + if (entry.group === live || entry.group.activity >= cutoff) { retained.push(entry); continue; } this.#evict(entry); } this.#cache = retained; + this.#schedulePrune(); + } + + // Arm the wakeup for the next entry due to age out, replacing any pending one. + // Writes settle retention inline, so this only has to cover the case no write + // follows. Cheap to over-arm: an entry written since is retained and re-armed. + #schedulePrune(): void { + clearTimeout(this.#pruneTimer); + this.#pruneTimer = undefined; + if (this.#state.closed.peek() !== undefined) return; + + const live = this.#liveEdge(); + const ages = this.#cache.filter((entry) => entry.group !== live).map((entry) => entry.group.activity); + if (ages.length === 0) return; + + const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS; + const timer = setTimeout( + () => { + this.#pruneTimer = undefined; + this.#prune(); + }, + Math.max(0, Math.min(...ages) + maxAgeMs - performance.now()), + ); + // A cache prune is never a reason to hold a Node/Bun process open. + (timer as unknown as { unref?: () => void }).unref?.(); + this.#pruneTimer = timer; } // Retain a source group and fan it out to every live sink. @@ -675,6 +723,8 @@ export class Producer { for (const sink of this.#sinks) sink.final = this.#state.final; } closeTrackState(this.#state, abort); + clearTimeout(this.#pruneTimer); + this.#pruneTimer = undefined; for (const { group } of this.#cache) group.close(abort); for (const sink of this.#sinks) { for (const group of sink.groups.peek()) group.close(abort); diff --git a/quest/m0/3291-moq-net-zero-budget-shedding-drops-discontinuity-markers.md b/quest/m0/3291-moq-net-zero-budget-shedding-drops-discontinuity-markers.md index 7918bf59240..f448e44d1f0 100644 --- a/quest/m0/3291-moq-net-zero-budget-shedding-drops-discontinuity-markers.md +++ b/quest/m0/3291-moq-net-zero-budget-shedding-drops-discontinuity-markers.md @@ -34,4 +34,3 @@ consumer's `discontinuity()` bumps once. Fix on `dev`. ## Related - [Monotonic timeline](/quest/m1/monotonic-timeline.md) - makes the declared marker the only reset signal -- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - the other open-group lifecycle rule in the same area diff --git a/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md b/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md deleted file mode 100644 index 59e8e7d7582..00000000000 --- a/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md +++ /dev/null @@ -1,30 +0,0 @@ -# [S] Retention should reclaim idle open groups now that expiry is timestamp-only - -## Goal - -Implement and verify the behavior tracked in [#3161](https://github.com/moq-dev/moq/issues/3161) -within the issue's stated scope and boundaries. - -## Plan - -Rescoped during the 2026-08 grooming: #3169 moved the wall-clock LRU into the -cache pool on dev, but reclamation fires only on the track's next write, the -latest group is protected, and the JS prune still retains open groups. -Remaining: a write-independent timer for idle open groups, plus JS coverage. - -### Issue context - -\#3099 split group lifetime into two rules: subscription expiry is timestamp-only (a group is stale once its reach falls a full max-age budget behind the newest frame of the latest group), and wall-clock reclamation of idle content belongs to the retention/cache layer. - -The expiry half landed in #3099. This issue tracks the reclamation half, which currently has gaps at the model layer: - -- The Rust `cache::Pool` ages by last access, but an in-process track without a pool has no wall-clock reclamation at all, and a group held open by a stalled publisher pins its buffer until something else closes it. -- The JS track retention prune deliberately retains open groups, so an abandoned open group survives `Publisher Max Age` indefinitely. - -Concrete symptom (from Codex's adversarial review of #3099): a reader drains an open group whose successor starts less than the budget ahead in media time, production goes quiet, and the pending read stays parked forever. Timestamp expiry is correct to keep the group (nothing proves it is useless), so the bound has to come from retention: an idle open group past the retention window should be reclaimed (aborted), which surfaces to a parked reader as the gap it is. - -Needs a decision on what "idle" means for an open group (no producer writes for the retention window is the obvious candidate), a timer/wakeup to arm it, and coverage in both Rust and JS. The draft's Expiration section already frames retention as the publisher's own policy, so this is implementation work, not a wire change. - -## Closes - -- [#3161](https://github.com/moq-dev/moq/issues/3161) - close this issue when the quest finishes diff --git a/quest/m1/README.md b/quest/m1/README.md index df1c8b87707..1ef71edde6f 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -25,7 +25,6 @@ with the current dev tree before starting. - [#2979](/quest/m1/2979-moq-tokio-does-not-compile-with-no-default-features-and.md) - moq-tokio compiles with any subset of transport features, and nightly checks it per crate - [#2853](/quest/m1/2853-quiche-with-a-pinned-source-port-can-dial-only-a-broken.md) - quiche with a pinned source port can dial only a broken IPv4 address - [#2624](/quest/m1/2624-moq-native-goaway-redirect-guard-classifies-hosts-by-name.md) - moq-native: GOAWAY redirect guard classifies hosts by name, not by resolved address -- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - Retention should reclaim idle open groups now that expiry is timestamp-only - [Monotonic timeline](/quest/m1/monotonic-timeline.md) - a track's timestamps never fall below its live edge; publishers declare a discontinuity and continue forward, consumers stop detecting rewinds - [Group overflow](/quest/m1/group-overflow-abort.md) - an oversized open group aborts for every reader instead of shedding its head - [#2895](/quest/m1/2895-add-an-atomic-readiness-gate-for-origin-broadcasts.md) - Add an atomic readiness gate for Origin broadcasts diff --git a/quest/m1/group-overflow-abort.md b/quest/m1/group-overflow-abort.md index 368a4234909..9158ba60be2 100644 --- a/quest/m1/group-overflow-abort.md +++ b/quest/m1/group-overflow-abort.md @@ -34,9 +34,9 @@ evicted prefix. - **A new error variant**, `Error::GroupTooLarge`, mirroring the existing `FrameTooLarge` naming. `Lagged` is named from the consumer's side and would keep blaming the reader for the writer's overrun; `Evicted` already means the - pool dropped a whole group under external memory pressure, and - [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - will abort idle open groups, so these three need to stay distinguishable. + pool dropped a whole group under external memory pressure, and the cache + pool's idle sweep aborts an idle open group with `Old`, so these three need to + stay distinguishable. - **The writer learns synchronously.** The write that pushes the group past its budget returns `Err(Error::GroupTooLarge)` and aborts the group, which is the shape `FrameTooLarge` already has in `write_frame`. Today `evict()` returns @@ -126,6 +126,5 @@ already claims its top end "intentionally reaches the raised ## Related - [Group charge](/quest/m0/group-charge.md) - pool-level budget accounting, unaffected by this change -- [#3161](/quest/m1/3161-retention-should-reclaim-idle-open-groups-now-that-expiry.md) - also turns "open group hit a limit" into an abort, so the two must not collide on the error variant - [JS stream codes](/quest/m1/js-net-stream-error-codes.md) - without it a JS publisher sends this new code to the wire as Internal - [#3001](/quest/m1/3001-ietf-stream-resets-send-moq-lite-error-codes-so-routine.md) - the moq-transport half of the code mapping diff --git a/rs/moq-net/src/model/cache.rs b/rs/moq-net/src/model/cache.rs index c4648c21ec8..21d38e0c8b3 100644 --- a/rs/moq-net/src/model/cache.rs +++ b/rs/moq-net/src/model/cache.rs @@ -19,20 +19,29 @@ //! frontier moves with cache turnover on its own. //! //! The pool also owns the wall-clock LRU window ([`Pool::expiry`]): a non-latest -//! group that nobody has read or written for that long is reclaimed by its track's -//! next write, no matter what retention its track advertises. Track retention +//! group that nobody has read or written for that long is reclaimed, no matter what +//! retention its track advertises. Track retention //! ([`max_age`](crate::track::Info::max_age)) is measured in media timestamps, so a //! congestion stall can't age content out; the pool's expiry is the orthogonal //! wall-clock bound that keeps unwatched content from pinning RAM. //! +//! Expiry runs from two places. A track's own writes settle it inline, which keeps a +//! busy track's backlog draining without any wakeup. That alone is not a bound: a +//! publisher that stalls with a group still open stops writing, so nothing reclaims +//! its buffer and a reader parked in that group waits forever. So a pool with an +//! expiry window also keeps a registry of its live track accounts and sweeps them on +//! a wall-clock cadence, driven by [`origin::Driver`](crate::origin::Driver). The byte +//! budget deliberately has no such machinery: it is repaid by writes, and a track that +//! never writes never grows the pool. +//! //! A bare pool is inert by default ([`Pool::unbounded`]): publishers and subscribers -//! that never set a capacity or expiry pay only a couple of atomic counters. A -//! standalone [`origin`](crate::origin::Info) enables [`DEFAULT_EXPIRY`], while a -//! relay creates one configured pool and shares it across every origin so the whole -//! process caches into a single policy. +//! that never set a capacity or expiry pay only a couple of atomic counters, and +//! register nothing. A standalone [`origin`](crate::origin::Info) enables +//! [`DEFAULT_EXPIRY`], while a relay creates one configured pool and shares it across +//! every origin so the whole process caches into a single policy. -use std::sync::Arc; use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; +use std::sync::{Arc, OnceLock, Weak}; use std::time::Duration; use super::track::TrackState; @@ -86,7 +95,7 @@ impl Config { /// Set the wall-clock LRU window. `None` disables idle reclamation. /// /// A non-latest cached group that nobody reads or writes for this long is - /// reclaimed by its track's next write, surfacing to any remaining reader as + /// reclaimed, surfacing to any remaining reader as /// [`Error::Old`](crate::Error::Old). This is independent of track retention: /// [`max_age`](crate::track::Info::max_age) uses media timestamps, while this /// window keeps idle content from pinning memory. The value is fixed when the @@ -131,6 +140,11 @@ struct Inner { // from the live edge, or inserted behind it) and remove it when it leaves. access_sum: AtomicU64, access_count: AtomicU64, + // Live track accounts, so [`Pool::sweep`] can expire idle groups in a track that + // has stopped writing. Empty and never touched when expiry is disabled: the byte + // budget needs no registry, since a track that never writes never grows the pool. + // Weak, because a track owns its account and the account must not outlive it. + tracks: kio::Lock>>, } impl Pool { @@ -156,6 +170,7 @@ impl Pool { epoch: crate::model::clock::now(), access_sum: AtomicU64::new(0), access_count: AtomicU64::new(0), + tracks: kio::Lock::new(slab::Slab::new()), }), } } @@ -211,6 +226,53 @@ impl Pool { self.expiry().unwrap_or(DEFAULT_EXPIRY) / 2 } + /// How often [`Self::sweep`] should run, or `None` when idle reclamation is off. + /// + /// Half the window, so a group is reclaimed within 1.5 windows of its last access + /// rather than 2. The sweep is otherwise rate-limited by the same per-track gate + /// the write path uses ([`EXPIRY_SCAN_TICKS`]), so a shorter cadence buys nothing. + pub(crate) fn sweep_interval(&self) -> Option { + self.expiry().map(|expiry| expiry / 2) + } + + /// Expire idle groups in every track holding an account against this pool. + /// + /// This is the write-independent half of the LRU window: a track whose publisher + /// has stalled runs no write path, so nothing else would ever reclaim the group it + /// left open, and a reader parked inside that group would never be told. Each + /// track's scan is the same bounded, rate-limited one a write runs, so a sweep over + /// an idle pool costs a relaxed atomic per track. + /// + /// A no-op when idle reclamation is disabled: nothing registers. + pub(crate) fn sweep(&self) { + // Upgraded under the lock but settled outside it: settling takes the track's + // own state lock, and dropping the last handle to a dead account would + // re-enter this lock through `Track::drop`. + let tracks: Vec> = self + .inner + .tracks + .lock() + .iter() + .filter_map(|(_, track)| track.upgrade()) + .collect(); + + for track in tracks { + track.settle(None); + } + } + + /// Enter a track account into the sweep registry, returning its key. `None` when + /// idle reclamation is off, which is what keeps a bare pool free of bookkeeping. + fn register(&self, track: &Arc) -> Option { + self.expiry()?; + Some(self.inner.tracks.lock().insert(Arc::downgrade(track))) + } + + /// Drop a track account from the sweep registry. + fn unregister(&self, key: usize) { + self.inner.tracks.lock().remove(key); + } + /// Returns true if both handles share the same underlying pool. pub fn same_pool(&self, other: &Self) -> bool { Arc::ptr_eq(&self.inner, &other.inner) @@ -335,18 +397,27 @@ pub(crate) struct Track { // The track that pays this account off, holding the groups being charged. state: kio::Weak, + + // This account's slot in the pool's sweep registry, absent when the pool has no + // expiry window (nothing is registered) or for the detached default account. + sweep: OnceLock, } impl Track { /// Open an account against `pool` for the track behind `state`. pub(crate) fn new(pool: Pool, state: kio::Weak) -> Arc { - Arc::new(Self { + let track = Arc::new(Self { pool, written: AtomicU64::new(0), next_expiry: AtomicU64::new(0), expiry_cursor: AtomicUsize::new(0), state, - }) + sweep: OnceLock::new(), + }); + if let Some(key) = track.pool.register(&track) { + let _ = track.sweep.set(key); + } + track } /// The pool this track caches into. @@ -439,6 +510,14 @@ impl Track { } } +impl Drop for Track { + fn drop(&mut self) { + if let Some(key) = self.sweep.get() { + self.pool.unregister(*key); + } + } +} + /// The RAII byte accounting for one cached group, owned by the group's state. /// /// `add`/`sub` mirror the group's cached payload bytes into the pool with plain @@ -811,6 +890,38 @@ mod test { assert!(!track.expiry_due(None)); } + #[test] + fn sweep_interval_is_half_the_window() { + assert_eq!(Pool::unbounded().sweep_interval(), None); + let pool = Pool::new(Config::default().with_expiry(Duration::from_secs(4))); + assert_eq!(pool.sweep_interval(), Some(Duration::from_secs(2))); + } + + #[test] + fn the_sweep_registry_follows_account_lifetime() { + let pool = bounded(1000); + assert!(pool.inner.tracks.lock().is_empty()); + + let track = Track::new(pool.clone(), kio::Weak::new()); + assert_eq!(pool.inner.tracks.lock().len(), 1); + + // A registered account whose track is already gone is swept harmlessly. + pool.sweep(); + + drop(track); + assert!(pool.inner.tracks.lock().is_empty(), "a dropped account leaves no entry"); + } + + #[test] + fn an_inert_pool_registers_nothing() { + // No window means no sweep, so a bare pool pays no registry cost. + let pool = Pool::unbounded(); + let track = Track::new(pool.clone(), kio::Weak::new()); + assert!(pool.inner.tracks.lock().is_empty()); + pool.sweep(); + drop(track); + } + #[test] fn expiry_gate_stays_closed_without_a_window() { // No window means no time gate, and no clock read to reach it. diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index fce36959767..e4a786e889f 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -1090,6 +1090,7 @@ impl Producer { let nodes = OriginNodes::default(); let shared = kio::Shared::::default(); let timers = TimersSlot::default(); + let pool = info.pool.clone(); let producer = Self { info: info.id, nodes: nodes.clone(), @@ -1108,8 +1109,10 @@ impl Producer { nodes, shared, done: false, + sweep: None, }, timers, + pool, }; (producer, driver) } @@ -1601,6 +1604,9 @@ pub struct Driver { state: DriverState, // The producer's slot, filled by `run` so lifecycle work can mint deadlines. timers: TimersSlot, + // The cache pool this origin's groups charge into, swept on a wall-clock + // cadence so its idle window binds a track whose publisher stopped writing. + pool: cache::Pool, } /// Everything the driver polls and tears down, split from the park so the two @@ -1615,6 +1621,9 @@ struct DriverState { shared: kio::Shared, /// Cached completion so a poll after `Ready` doesn't re-poll the drained set. done: bool, + /// The cache pool's idle sweep, installed by [`Driver::run`] (which is where the + /// timers arrive) and absent when the pool never expires content. + sweep: Option, } impl Driver { @@ -1627,14 +1636,54 @@ impl Driver { T: crate::runtime::Timers + MaybeSend + MaybeSync + 'static, T::Timer: MaybeSend + 'static, { - self.timers.install(AnyTimers::new(timers)); + let timers = AnyTimers::new(timers); + self.timers.install(timers.clone()); + let mut state = self.state; + state.sweep = Sweep::new(&timers, self.pool); Run { - state: self.state, + state, park: kio::Park::default(), } } } +/// The wall-clock half of the cache pool's idle window. +/// +/// A track settles its own expiry as it writes, which covers every track that is +/// still producing. A publisher that stalls with a group open stops writing, so this +/// is what still reclaims that group (and unblocks whoever is parked inside it): a +/// periodic [`cache::Pool::sweep`] on the origin's own timers. +/// +/// Disarmed, and never allocated, when the pool has no expiry window. +struct Sweep { + timers: AnyTimers, + pool: cache::Pool, + interval: Duration, + deadline: crate::runtime::Deadline, +} + +impl Sweep { + fn new(timers: &AnyTimers, pool: cache::Pool) -> Option { + let interval = pool.sweep_interval()?; + Some(Self { + timers: timers.clone(), + pool, + interval, + deadline: crate::runtime::Deadline::after(timers, interval), + }) + } + + fn poll(&mut self, waiter: &kio::Waiter) { + // A `Deadline` stays ready until it is re-armed, so re-arm before sweeping + // again; a clock that has not moved lands the next one in the future and + // this returns after one pass. + while self.deadline.poll(waiter).is_ready() { + self.deadline.set(self.timers.now().checked_add(self.interval)); + self.pool.sweep(); + } + } +} + /// The future running an origin's lifecycle work, from [`Driver::run`]. /// /// Poll it for the life of the origin, either by `.await`ing it (typically @@ -1667,6 +1716,12 @@ impl Run { impl DriverState { fn poll(&mut self, waiter: &kio::Waiter) -> Poll<()> { + // Never gates completion: the pool outlives this origin (a relay shares one + // across every origin), so a sweep that is still due must not keep the driver + // alive after its lifecycle work has drained. + if let Some(sweep) = &mut self.sweep { + sweep.poll(waiter); + } if !self.done { ready!(self.set.poll(waiter)); self.done = true; @@ -4765,6 +4820,68 @@ mod tests { assert_eq!(Cost::UNKNOWN.charged(3).cold, MAX_COST); } + /// Mint an origin whose pool reclaims idle content after `expiry`. + fn expiring_origin(expiry: Duration) -> Producer { + let pool = cache::Pool::new(cache::Config::default().with_expiry(expiry)); + Info::default().with_pool(pool).produce() + } + + /// A publisher that stalls with a group still open runs no write path, so the + /// track's own write-driven expiry never fires and a reader parked in that group + /// is never told. The driver's wall-clock sweep is the bound. + #[tokio::test(start_paused = true)] + async fn stalled_publisher_open_group_is_reclaimed() { + let expiry = Duration::from_secs(1); + let origin = expiring_origin(expiry); + let mut broadcast = origin.create_broadcast("test").unwrap(); + let mut track = broadcast.create_track("video", None).unwrap(); + + let mut stalled = track.append_group().unwrap(); + stalled.write_frame(crate::Timestamp::ZERO, b"x".as_slice()).unwrap(); + // A successor, so the stalled group is not the protected live edge. Its + // timestamp is inside the retention budget, so subscription expiry keeps the + // stalled group: only reclamation can bound it. + let _successor = track.append_group().unwrap(); + + let mut reading = stalled.consume(); + assert!(reading.read_frame().await.unwrap().is_some()); + + // Production goes quiet: nothing writes to this track again. + crate::model::clock::advance(expiry * 2); + + // Bounded so a regression fails rather than parking forever, which is the + // bug itself. Time is virtual, so the wait costs nothing. + let reclaimed = tokio::time::timeout(Duration::from_secs(60), reading.read_frame()).await; + assert!( + matches!(reclaimed, Ok(Err(Error::Old))), + "the sweep must reclaim an idle open group and surface the gap, got {reclaimed:?}" + ); + } + + /// Reclamation is the pool's policy, not the origin's: a pool with no expiry + /// window keeps idle content until byte pressure takes it, sweep or no sweep. + #[tokio::test(start_paused = true)] + async fn sweep_respects_a_disabled_expiry() { + let origin = Info::default().with_pool(cache::Pool::unbounded()).produce(); + let mut broadcast = origin.create_broadcast("test").unwrap(); + let mut track = broadcast.create_track("video", None).unwrap(); + + let mut stalled = track.append_group().unwrap(); + stalled.write_frame(crate::Timestamp::ZERO, b"x".as_slice()).unwrap(); + let _successor = track.append_group().unwrap(); + + let mut reading = stalled.consume(); + assert!(reading.read_frame().await.unwrap().is_some()); + + crate::model::clock::advance(Duration::from_secs(3600)); + tokio::time::advance(Duration::from_secs(3600)).await; + + assert!( + reading.read_frame().now_or_never().is_none(), + "a pool without an expiry window never reclaims" + ); + } + /// A draining cost still has to fit the wire, since the route keeps being /// announced downstream while it drains. #[test] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index fec37990a71..4b7c2fa2cda 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4504,7 +4504,11 @@ mod test { /// Mint a track under an origin whose pool has the given wall-clock LRU window. fn track_producer_expiring(name: impl Into>, expiry: impl Into>) -> Producer { - let pool = cache::Pool::new(cache::Config::default().with_expiry(expiry)); + track_producer_pooled(name, cache::Pool::new(cache::Config::default().with_expiry(expiry))) + } + + /// Mint a track under an origin caching into `pool`. + fn track_producer_pooled(name: impl Into>, pool: cache::Pool) -> Producer { let origin = crate::origin::Info::default().with_pool(pool); Producer::new( Arc::new(broadcast::Info { @@ -4516,6 +4520,26 @@ mod test { ) } + /// The write path is not the only thing that runs expiry: a pool sweep reclaims a + /// track's idle groups even when the track never writes again, which is the only + /// bound on a publisher that stalls with a group still open. + #[tokio::test] + async fn pool_sweep_expires_without_a_write() { + let pool = cache::Pool::new(cache::Config::default().with_expiry(Duration::from_secs(1))); + let mut producer = track_producer_pooled("test", pool.clone()); + let mut stalled = producer.append_group().unwrap(); // seq 0, left open + stalled.write_frame(Timestamp::ZERO, b"x".as_slice()).unwrap(); + producer.append_group().unwrap(); // seq 1, the live edge + + crate::model::clock::advance(Duration::from_secs(2)); + pool.sweep(); + + assert!( + !producer.state.read().lookup.contains_key(&0), + "the sweep reclaimed an idle open group with no write behind it" + ); + } + #[tokio::test] async fn pool_expiry_controls_eviction() { // A shorter LRU window on the pool evicts sooner than the default. From 23ee416c2f0ac3e6e88036fd5402d6f24b42ca7d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 21:01:20 -0700 Subject: [PATCH 3/5] fix(net): drain a whole idle backlog per sweep, and cap the JS prune timer The sweep reused the write path's rotating EVICT_SCAN window, which is the right size on the hot path only because more writes follow to revisit the rest. Nothing follows on a track that stopped writing, so a deep backlog needed its own length in sweeps before the oldest entry was examined: about an hour for a 900-group track against a 30-second window, not the 1.5 windows the window promises. ExpiryScan now carries a width, and the sweep sets it to the whole eviction order. The cost is bounded by the track's cached groups, which the byte budget already caps, and it stays behind the same time gate a write is. In js/net, setTimeout truncates its delay to a signed 32-bit int, so a maxAge past that fired immediately and re-armed the same oversized delay: a millisecond timer loop instead of a wakeup. Cap the delay and let #prune re-arm from what it retains. Also record the limitation the sweep does not cover: it rides origin::Driver, so a standalone broadcast built without an origin keeps only the write-driven half. Every non-test call site in this tree spawns the driver, but it is a public path, so the cache module doc and Config::with_expiry say so rather than over-claiming. Co-Authored-By: Claude Opus 5 --- js/net/src/track.test.ts | 26 +++++++++++++++++ js/net/src/track.ts | 19 +++++++----- rs/moq-net/src/model/cache.rs | 51 +++++++++++++++++++++++++------- rs/moq-net/src/model/track.rs | 55 ++++++++++++++++++++++++++++++++--- 4 files changed, 129 insertions(+), 22 deletions(-) diff --git a/js/net/src/track.test.ts b/js/net/src/track.test.ts index ae3815e01cb..fc600609599 100644 --- a/js/net/src/track.test.ts +++ b/js/net/src/track.test.ts @@ -850,6 +850,32 @@ test("an abandoned open group ages out with no further write", async () => { expect(await Promise.race([read, timeout])).toBeInstanceOf(Lagged); }); +test("the prune wakeup never exceeds the setTimeout limit", () => { + // A window past 2^31 ms truncates the delay to a signed 32-bit int, so the wakeup + // fires immediately and re-arms the same oversized delay: a millisecond timer loop + // instead of a wakeup. Capped, it just wakes early and finds nothing due. + const real = globalThis.setTimeout; + const delays: number[] = []; + // @ts-expect-error a stub, not a full setTimeout + globalThis.setTimeout = (_fn: () => void, delay: number) => { + delays.push(delay); + return { unref: () => {} }; + }; + + try { + const producer = new TrackProducer("test").accept({ maxAge: 2 ** 31 + 10_000 }); + const stalled = producer.appendGroup(); + stalled.writeString("first"); + producer.appendGroup(); // the live edge, so the stalled group is prunable + producer.close(); + } finally { + globalThis.setTimeout = real; + } + + expect(delays.length).toBeGreaterThan(0); + for (const delay of delays) expect(delay).toBeLessThanOrEqual(2 ** 31 - 1); +}); + test("retention pruning preserves clean EOF for a drained mirror", async () => { const clock = mockMonotonicTime(10_000); try { diff --git a/js/net/src/track.ts b/js/net/src/track.ts index 20f0e840b22..e5f60cf621d 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -11,6 +11,10 @@ import { Timescale, type Timestamp } from "./time.ts"; export type { Datagram } from "./datagram.ts"; +// The largest delay setTimeout keeps: anything more truncates to a signed 32-bit int +// and fires right away. +const MAX_TIMEOUT_MS = 2 ** 31 - 1; + /** Default {@link Info.maxAge} window (milliseconds) when the publisher does not set one. */ export const DEFAULT_MAX_AGE_MS = 5000; @@ -597,13 +601,14 @@ export class Producer { if (ages.length === 0) return; const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS; - const timer = setTimeout( - () => { - this.#pruneTimer = undefined; - this.#prune(); - }, - Math.max(0, Math.min(...ages) + maxAgeMs - performance.now()), - ); + // setTimeout truncates its delay to a signed 32-bit int, so a longer window + // would fire immediately and spin. Wake at the cap instead and re-arm: #prune + // retains anything still fresh, so the extra wakeups are the only cost. + const delay = Math.min(MAX_TIMEOUT_MS, Math.max(0, Math.min(...ages) + maxAgeMs - performance.now())); + const timer = setTimeout(() => { + this.#pruneTimer = undefined; + this.#prune(); + }, delay); // A cache prune is never a reason to hold a Node/Bun process open. (timer as unknown as { unref?: () => void }).unref?.(); this.#pruneTimer = timer; diff --git a/rs/moq-net/src/model/cache.rs b/rs/moq-net/src/model/cache.rs index 21d38e0c8b3..3e0735b0e93 100644 --- a/rs/moq-net/src/model/cache.rs +++ b/rs/moq-net/src/model/cache.rs @@ -30,9 +30,11 @@ //! publisher that stalls with a group still open stops writing, so nothing reclaims //! its buffer and a reader parked in that group waits forever. So a pool with an //! expiry window also keeps a registry of its live track accounts and sweeps them on -//! a wall-clock cadence, driven by [`origin::Driver`](crate::origin::Driver). The byte -//! budget deliberately has no such machinery: it is repaid by writes, and a track that -//! never writes never grows the pool. +//! a wall-clock cadence, driven by [`origin::Driver`](crate::origin::Driver). A pool +//! whose origin driver is never run therefore keeps the write-driven half only, which +//! is what a standalone [`broadcast`](crate::broadcast::Info) built without an origin +//! gets. The byte budget deliberately has no such machinery: it is repaid by writes, +//! and a track that never writes never grows the pool. //! //! A bare pool is inert by default ([`Pool::unbounded`]): publishers and subscribers //! that never set a capacity or expiry pay only a couple of atomic counters, and @@ -98,7 +100,10 @@ impl Config { /// reclaimed, surfacing to any remaining reader as /// [`Error::Old`](crate::Error::Old). This is independent of track retention: /// [`max_age`](crate::track::Info::max_age) uses media timestamps, while this - /// window keeps idle content from pinning memory. The value is fixed when the + /// window keeps idle content from pinning memory. Reclaiming without a write behind + /// it needs the [`origin::Driver`](crate::origin::Driver) running, which every + /// session-backed origin has; a standalone broadcast keeps only the write-driven + /// half. The value is fixed when the /// pool is created. Values are rounded up to the pool's 100 ms clock tick, with /// 100 ms as the minimum effective window. pub fn with_expiry(mut self, expiry: impl Into>) -> Self { @@ -229,8 +234,11 @@ impl Pool { /// How often [`Self::sweep`] should run, or `None` when idle reclamation is off. /// /// Half the window, so a group is reclaimed within 1.5 windows of its last access - /// rather than 2. The sweep is otherwise rate-limited by the same per-track gate - /// the write path uses ([`EXPIRY_SCAN_TICKS`]), so a shorter cadence buys nothing. + /// rather than 2, and each track's pass covers its whole eviction order so that + /// bound holds however deep the backlog is. A shorter cadence buys nothing: the + /// same per-track gate the write path uses ([`EXPIRY_SCAN_TICKS`]) floors an + /// actual scan at one per second, which is also why a window under two seconds + /// reclaims within two windows instead of 1.5. pub(crate) fn sweep_interval(&self) -> Option { self.expiry().map(|expiry| expiry / 2) } @@ -239,9 +247,10 @@ impl Pool { /// /// This is the write-independent half of the LRU window: a track whose publisher /// has stalled runs no write path, so nothing else would ever reclaim the group it - /// left open, and a reader parked inside that group would never be told. Each - /// track's scan is the same bounded, rate-limited one a write runs, so a sweep over - /// an idle pool costs a relaxed atomic per track. + /// left open, and a reader parked inside that group would never be told. Each track + /// covers its whole eviction order, since nothing else will, but behind the same + /// time gate a write uses: a sweep over a pool with nothing due costs a relaxed + /// atomic per track. /// /// A no-op when idle reclamation is disabled: nothing registers. pub(crate) fn sweep(&self) { @@ -257,7 +266,7 @@ impl Pool { .collect(); for track in tracks { - track.settle(None); + track.sweep(); } } @@ -454,6 +463,22 @@ impl Track { /// [`Charge::add`]), reused so a frame write reads the clock once instead of /// twice. `None` leaves the gate to sample it, and only if it needs it. pub(crate) fn settle(&self, now: Option) { + self.settle_inner(now, false); + } + + /// Settle from [`Pool::sweep`], covering the whole eviction order. + /// + /// The write path's rotating window is bounded so it stays off the hot path, which + /// holds the deadline only because writes keep coming. Nothing else runs on a track + /// that stopped writing, so the sweep scans the queue in one pass instead: a + /// backlog otherwise needs its own length in windows before the sweep reaches its + /// oldest entry. Bounded by the track's cached groups, which the byte budget caps, + /// and gated to the same interval a write is. + pub(crate) fn sweep(&self) { + self.settle_inner(None, true); + } + + fn settle_inner(&self, now: Option, full: bool) { let settle_debt = self.written.load(Ordering::Relaxed) >= WRITE_CHARGE_THRESHOLD; let scan_expiry = self.expiry_due(now); if !settle_debt && !scan_expiry { @@ -464,7 +489,11 @@ impl Track { let Some(state) = self.state.upgrade() else { return }; let expiry = if scan_expiry { let state = state.read(); - let scan = state.expiry_scan(); + let scan = if full { + state.expiry_scan_full() + } else { + state.expiry_scan() + }; state.expiry_mutation_due(scan).then_some(scan) } else { None diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 4b7c2fa2cda..9567c3e9940 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -48,10 +48,13 @@ const EVICT_SLACK: usize = 64; /// groups, small enough that a write never scans a long queue. const EVICT_SCAN: usize = 4; -/// One bounded pass over the eviction order at a fixed cache time. +/// One pass over the eviction order at a fixed cache time. #[derive(Clone, Copy)] pub(super) struct ExpiryScan { start: usize, + // How many entries this pass examines. `EVICT_SCAN` from the write path, the + // whole queue from the pool's sweep. + width: usize, now: u64, max_ticks: u64, } @@ -577,7 +580,8 @@ impl TrackState { /// entries beyond one scan window can't be starved by fresh (recently read, /// fetched, or written) entries in front of them: every position is revisited /// within a few writes. Expiry throughput is therefore EVICT_SCAN groups per write; the - /// byte budget reclaims the remainder under memory pressure. + /// byte budget reclaims the remainder under memory pressure, and the pool's sweep + /// ([`Self::expiry_scan_full`]) covers a track that stopped writing entirely. pub(super) fn evict_expired(&mut self) { let scan = self.expiry_scan(); self.evict_expired_scan(scan); @@ -587,6 +591,25 @@ impl TrackState { pub(super) fn expiry_scan(&self) -> ExpiryScan { ExpiryScan { start: self.cache.next_expiry_scan(EVICT_SCAN), + width: EVICT_SCAN, + now: self.cache.pool().now(), + max_ticks: self.cache.pool().expiry_ticks(), + } + } + + /// Describe a scan covering the whole eviction order, for the pool's sweep. + /// + /// A write's window is deliberately tiny so it stays off the hot path, which is + /// fine while writes keep coming. The sweep is the only thing running on a track + /// nobody writes, so a rotating window there would take a backlog's length in + /// sweeps to reach the oldest entry, missing the window's deadline by orders of + /// magnitude. It runs at most once per gate interval, off any write, so it covers + /// the queue in one pass instead. The cost is one pass over the track's cached + /// groups, which the pool's byte budget already bounds. + pub(super) fn expiry_scan_full(&self) -> ExpiryScan { + ExpiryScan { + start: 0, + width: self.evict.len(), now: self.cache.pool().now(), max_ticks: self.cache.pool().expiry_ticks(), } @@ -597,7 +620,7 @@ impl TrackState { let len = self.evict.len(); if len > 0 { let start = scan.start % len; - for step in 0..len.min(EVICT_SCAN) { + for step in 0..len.min(scan.width) { let (sequence, stamp) = self.evict[(start + step) % len]; let Some(slot) = self.lookup.get(&sequence) else { continue; @@ -629,7 +652,7 @@ impl TrackState { let len = self.evict.len(); if len > 0 { let start = scan.start % len; - for step in 0..len.min(EVICT_SCAN) { + for step in 0..len.min(scan.width) { let (sequence, stamp) = self.evict[(start + step) % len]; let Some(slot) = self.lookup.get(&sequence) else { continue; @@ -4540,6 +4563,30 @@ mod test { ); } + /// One sweep drains a whole idle backlog, not a rotating window of it: a quiet + /// track has no writes left to revisit the rest of the queue with, so a bounded + /// pass would leave the oldest groups parked for a backlog's length in windows. + #[tokio::test] + async fn pool_sweep_drains_a_deep_backlog() { + let pool = cache::Pool::new(cache::Config::default().with_expiry(Duration::from_secs(1))); + let mut producer = track_producer_pooled("test", pool.clone()); + + // Comfortably more than one write-driven scan window (EVICT_SCAN). + let backlog = 4 * EVICT_SCAN; + for _ in 0..backlog { + let mut group = producer.append_group().unwrap(); + group.write_frame(Timestamp::ZERO, b"x".as_slice()).unwrap(); + } + producer.append_group().unwrap(); // the live edge, always protected + + crate::model::clock::advance(Duration::from_secs(2)); + pool.sweep(); + + let state = producer.state.read(); + let stale = (0..backlog as u64).filter(|seq| state.lookup.contains_key(seq)).count(); + assert_eq!(stale, 0, "one sweep reclaimed the whole idle backlog"); + } + #[tokio::test] async fn pool_expiry_controls_eviction() { // A shorter LRU window on the pool evicts sooner than the default. From 193c3e0325abc59527fac2bc3a46a87fd447b02b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 22:24:48 -0700 Subject: [PATCH 4/5] perf(net): stop an idle sweep once the front stops yielding victims Covering the whole eviction order made the steady-state sweep cost scale with everything cached, not with what it reclaims. Measured on 100k cached groups in release, a sweep with nothing due took 26ms and touched every group header: 260ns each, memory-latency bound, walking the entire cache working set every half-window. The eviction order is oldest-first, and reads are what reorder it, so a track that went quiet has its whole stale backlog contiguous at the front. Start there instead of at the rotating cursor, and stop once the pass has retained EVICT_SCAN entries. A quiet track still drains its backlog in one sweep, which is what the deep-backlog test covers, while a track with nothing due costs a handful of entries rather than its depth. The write path is unchanged: its window is EVICT_SCAN wide, so the stop rule can never fire before the width cap it already had. Same measurement after: 491 ns/track at 40k cached groups and 376 ns/track at 160k, against a 366 ns/track floor with the queues empty. The scan is now depth-independent, so a sweep is bounded by track count. expiry_mutation_due mirrors the stop rule, since a memo that scanned further would report work the scan itself will not do. Also drop the intermediate arrays and the spread from js/net's #schedulePrune, which runs on every publish: one pass finds the oldest entry, and Math.min(...ages) also capped how many groups a track could hold. Co-Authored-By: Claude Opus 5 --- js/net/src/track.ts | 12 +++++++--- rs/moq-net/src/model/cache.rs | 27 ++++++++++----------- rs/moq-net/src/model/track.rs | 44 +++++++++++++++++++++++++---------- 3 files changed, 55 insertions(+), 28 deletions(-) diff --git a/js/net/src/track.ts b/js/net/src/track.ts index e5f60cf621d..1a17aeb8416 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -596,15 +596,21 @@ export class Producer { this.#pruneTimer = undefined; if (this.#state.closed.peek() !== undefined) return; + // One pass, no intermediate arrays: this runs on every publish, and a spread + // over the cache would also cap how many groups a track can hold. const live = this.#liveEdge(); - const ages = this.#cache.filter((entry) => entry.group !== live).map((entry) => entry.group.activity); - if (ages.length === 0) return; + let oldest: number | undefined; + for (const entry of this.#cache) { + if (entry.group === live) continue; + if (oldest === undefined || entry.group.activity < oldest) oldest = entry.group.activity; + } + if (oldest === undefined) return; const maxAgeMs = this.#state.info.peek()?.maxAge ?? DEFAULT_MAX_AGE_MS; // setTimeout truncates its delay to a signed 32-bit int, so a longer window // would fire immediately and spin. Wake at the cap instead and re-arm: #prune // retains anything still fresh, so the extra wakeups are the only cost. - const delay = Math.min(MAX_TIMEOUT_MS, Math.max(0, Math.min(...ages) + maxAgeMs - performance.now())); + const delay = Math.min(MAX_TIMEOUT_MS, Math.max(0, oldest + maxAgeMs - performance.now())); const timer = setTimeout(() => { this.#pruneTimer = undefined; this.#prune(); diff --git a/rs/moq-net/src/model/cache.rs b/rs/moq-net/src/model/cache.rs index 3e0735b0e93..f261ba884b2 100644 --- a/rs/moq-net/src/model/cache.rs +++ b/rs/moq-net/src/model/cache.rs @@ -234,8 +234,8 @@ impl Pool { /// How often [`Self::sweep`] should run, or `None` when idle reclamation is off. /// /// Half the window, so a group is reclaimed within 1.5 windows of its last access - /// rather than 2, and each track's pass covers its whole eviction order so that - /// bound holds however deep the backlog is. A shorter cadence buys nothing: the + /// rather than 2, and each track's pass drains from its oldest entry so that bound + /// holds however deep the backlog is. A shorter cadence buys nothing: the /// same per-track gate the write path uses ([`EXPIRY_SCAN_TICKS`]) floors an /// actual scan at one per second, which is also why a window under two seconds /// reclaims within two windows instead of 1.5. @@ -248,9 +248,10 @@ impl Pool { /// This is the write-independent half of the LRU window: a track whose publisher /// has stalled runs no write path, so nothing else would ever reclaim the group it /// left open, and a reader parked inside that group would never be told. Each track - /// covers its whole eviction order, since nothing else will, but behind the same - /// time gate a write uses: a sweep over a pool with nothing due costs a relaxed - /// atomic per track. + /// drains from its oldest entry rather than a rotating window, since nothing else + /// will, but stops as soon as the front stops yielding victims: a sweep over a pool + /// with nothing due costs a clock read and a few entries per track, not a walk of + /// everything cached. /// /// A no-op when idle reclamation is disabled: nothing registers. pub(crate) fn sweep(&self) { @@ -466,14 +467,14 @@ impl Track { self.settle_inner(now, false); } - /// Settle from [`Pool::sweep`], covering the whole eviction order. + /// Settle from [`Pool::sweep`], draining the stale front of the eviction order. /// - /// The write path's rotating window is bounded so it stays off the hot path, which - /// holds the deadline only because writes keep coming. Nothing else runs on a track - /// that stopped writing, so the sweep scans the queue in one pass instead: a - /// backlog otherwise needs its own length in windows before the sweep reaches its - /// oldest entry. Bounded by the track's cached groups, which the byte budget caps, - /// and gated to the same interval a write is. + /// The write path's rotating window is fine while writes keep coming, because the + /// next one revisits the rest. Nothing follows on a track that stopped writing, so + /// the sweep starts at the oldest entry and keeps going while it finds victims: a + /// backlog otherwise needs its own length in windows before the oldest entry is + /// even examined. A track with nothing due costs a handful of entries, not its + /// depth, and this is gated to the same interval a write is. pub(crate) fn sweep(&self) { self.settle_inner(None, true); } @@ -490,7 +491,7 @@ impl Track { let expiry = if scan_expiry { let state = state.read(); let scan = if full { - state.expiry_scan_full() + state.expiry_scan_drain() } else { state.expiry_scan() }; diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 9567c3e9940..724b612ac0c 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -52,8 +52,10 @@ const EVICT_SCAN: usize = 4; #[derive(Clone, Copy)] pub(super) struct ExpiryScan { start: usize, - // How many entries this pass examines. `EVICT_SCAN` from the write path, the - // whole queue from the pool's sweep. + // Ceiling on how many entries this pass examines. `EVICT_SCAN` from the write + // path, the whole queue from the pool's sweep. Either way the pass stops once it + // has retained `EVICT_SCAN` entries, so its cost tracks what it reclaims rather + // than how much the track has cached. width: usize, now: u64, max_ticks: u64, @@ -597,16 +599,17 @@ impl TrackState { } } - /// Describe a scan covering the whole eviction order, for the pool's sweep. + /// Describe a scan that drains the stale front of the eviction order, for the + /// pool's sweep. /// - /// A write's window is deliberately tiny so it stays off the hot path, which is - /// fine while writes keep coming. The sweep is the only thing running on a track - /// nobody writes, so a rotating window there would take a backlog's length in - /// sweeps to reach the oldest entry, missing the window's deadline by orders of - /// magnitude. It runs at most once per gate interval, off any write, so it covers - /// the queue in one pass instead. The cost is one pass over the track's cached - /// groups, which the pool's byte budget already bounds. - pub(super) fn expiry_scan_full(&self) -> ExpiryScan { + /// A write's rotating window is fine while writes keep coming, because the next + /// one revisits the rest. The sweep is the only thing running on a track nobody + /// writes, so that window would take a backlog's length in sweeps to reach the + /// oldest entry. This starts at the front, where the oldest entries are, and the + /// shared stop rule ends it once the front stops yielding victims. A track that + /// went quiet has its whole backlog stale and contiguous there, so one sweep takes + /// all of it; a track with nothing due costs `EVICT_SCAN` entries, not its depth. + pub(super) fn expiry_scan_drain(&self) -> ExpiryScan { ExpiryScan { start: 0, width: self.evict.len(), @@ -616,10 +619,14 @@ impl TrackState { } /// Whether an expiry scan would change observable track state. + /// + /// Mirrors [`Self::evict_expired_scan`]'s walk exactly, stop rule included: a memo + /// that scanned further would report work the scan itself will not do. pub(super) fn expiry_mutation_due(&self, scan: ExpiryScan) -> bool { let len = self.evict.len(); if len > 0 { let start = scan.start % len; + let mut retained = 0; for step in 0..len.min(scan.width) { let (sequence, stamp) = self.evict[(start + step) % len]; let Some(slot) = self.lookup.get(&sequence) else { @@ -634,6 +641,10 @@ impl TrackState { { return true; } + retained += 1; + if retained >= EVICT_SCAN { + break; + } } } @@ -647,11 +658,13 @@ impl TrackState { || self.evict.len() > 2 * self.lookup.len() + EVICT_SLACK } - /// Apply a scan previously selected by [`Self::expiry_scan`]. + /// Apply a scan previously selected by [`Self::expiry_scan`] or + /// [`Self::expiry_scan_drain`]. pub(super) fn evict_expired_scan(&mut self, scan: ExpiryScan) { let len = self.evict.len(); if len > 0 { let start = scan.start % len; + let mut retained = 0; for step in 0..len.min(scan.width) { let (sequence, stamp) = self.evict[(start + step) % len]; let Some(slot) = self.lookup.get(&sequence) else { @@ -670,6 +683,13 @@ impl TrackState { if Some(sequence) == self.latest_group || scan.now.saturating_sub(slot.group.cache_accessed_tick()) <= scan.max_ticks { + // Nothing to reclaim here. The front of the queue is the oldest + // content, so a run of these means the rest is fresher still: stop + // rather than walk a whole cache to find nothing. + retained += 1; + if retained >= EVICT_SCAN { + break; + } continue; } // Take the group out of the cache and abort it, so any consumer From 53dd2508f6c06a4e69062a099e47eadbd8fefbc0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 23:35:06 -0700 Subject: [PATCH 5/5] docs(relay): correct the --cache-duration help for the idle sweep The help text still said expiry is evaluated when a track writes, so a publisher that stops writing keeps its cache until it resumes. This change makes that false, and doc/bin/relay/config.md was updated for it while the `--help` it mirrors was not, so an operator reading the two got opposite answers. Say the same thing in both: the window holds whether or not a publisher is writing, and `capacity` is the budget that writes repay. Co-Authored-By: Claude Opus 5 --- rs/moq-relay/src/cache.rs | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/rs/moq-relay/src/cache.rs b/rs/moq-relay/src/cache.rs index ef16c8fc129..b9af0e93f53 100644 --- a/rs/moq-relay/src/cache.rs +++ b/rs/moq-relay/src/cache.rs @@ -54,11 +54,12 @@ pub struct CacheConfig { /// LRU window keeps its 30 second default and each track keeps its own /// retention window. /// - /// Two caveats. The latest group of every track is always retained, since it - /// is the live edge. And expiry is evaluated when a track writes, so a - /// publisher that stops writing without disconnecting keeps whatever it had - /// cached until it resumes or the broadcast closes; under memory pressure the - /// `capacity` budget is repaid by the tracks that are still writing. + /// One caveat: the latest group of every track is always retained, since it is + /// the live edge. This window otherwise holds whether or not a publisher is + /// still writing, since the relay sweeps every cached track on a wall-clock + /// cadence. The `capacity` budget is the one that depends on writes: a + /// publisher that stops writing pays none of it down, so under memory pressure + /// it is repaid by the tracks that are still writing. #[usage(long = "cache-duration", env = "MOQ_CACHE_DURATION")] pub duration: Option, }