Skip to content
42 changes: 22 additions & 20 deletions doc/bin/relay/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -599,15 +599,18 @@ environment variable (`MOQ_STATS_ENABLED`, `MOQ_STATS_PREFIX`,

### \[cache]

Memory budget for cached groups. Old (non-latest) groups stay cached until their
track's retention window expires, the `duration` ceiling is reached, or the pool
runs out of room, whichever comes first. Under memory pressure each track evicts
its own stalest groups as it writes, ordered by when each was last written or
served from cache, and proportional to how much it writes, so usage converges on
the budget without any global scan; groups that FETCH requests keep hitting are
retained over ones nobody reads. The latest group of every track is
always retained. With none of the knobs set the cache is unbounded and only each
track's own window limits memory.
Memory budget for cached groups. Old (non-latest) groups stay cached until they
sit unaccessed past the wall-clock LRU window (`duration`, 30 seconds by
default) or the pool runs out of room, whichever comes first. Under memory
pressure each track evicts its own stalest groups as it writes, ordered by when
each was last read or written, and proportional to how much it
writes, so usage converges on the budget without any global scan; groups that
FETCH requests keep hitting are retained over ones nobody reads. The latest
group of every track is always retained. With none of the knobs set the cache
is unbounded in bytes and only the default LRU window limits memory. Each
track's own retention window (its publisher's max age) is separate: that is
measured in media timestamps and bounds what subscribers may wait for, not
when idle memory is reclaimed.

```toml
[cache]
Expand All @@ -624,21 +627,20 @@ capacity = "8GiB"
# with `capacity` to also bound the target from above.
headroom = "2GiB"

# Maximum time a non-latest cached group is retained since it was last written
# or served from cache by a FETCH ("30s", "500ms"). Caps each track's own
# retention window: a publisher advertising a longer window is clamped down to
# this, bounding how much history a track accumulates no matter what upstream
# asks for. A FETCH cache hit restarts the clock, so actively-read history
# stays cached. The latest group of every track is always retained, as it is
# the live edge. Unbounded (each track keeps its own window) when unset.
duration = "30s"
# Maximum time a non-latest cached group is retained since it was last read or
# written ("30s", "500ms"). Sets the pool's wall-clock
# LRU window (30s when unset), and also clamps each track's media-timestamp
# retention window, so a publisher advertising a longer window can't promise
# more history than the relay keeps. A FETCH cache hit restarts the clock, so
# actively-read history stays cached. The latest group of every track is
# always retained, as it is the live edge.
duration = "60s"
```

The `capacity` budget counts group payload bytes, not process RSS, so leave
slack below physical memory (or just use `headroom`, which measures actual
available memory). `duration` is the age counterpart: it stops a long-running
relay from accumulating hours of history per track when the byte budget alone
leaves room for it.
available memory). `duration` is the age counterpart: it stops idle history
from pinning memory when the byte budget alone leaves room for it.

All eviction happens as tracks write (there is no background reaper), so both
`duration` and the byte budget cap how much history *active* publishers build
Expand Down
6 changes: 4 additions & 2 deletions doc/concept/layer/moq-lite.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,15 +124,17 @@ Each Subscription consists of a few properties:
- **Group Order**: The order in which groups are delivered. Defaults to descending; higher IDs are delivered first.
- **Subscriber Max Age**: How old a non-latest group may get before it is skipped. Defaults to zero, so stale groups are skipped immediately.

That age is measured both on the media timeline (a group's first timestamp against the newest stamped one) and by wall-clock arrival time, and either limit can expire the group.
The media timeline keeps a backlog delivered as a burst old, while wall-clock time backstops stalled or empty groups that have no timestamp.
That age is measured on the media timeline: a group is stale once everything it could still hold falls a full budget behind the newest frame, bounded by where its successor begins.
Timestamps rather than the wall clock, so a backlog delivered as a burst is still old, while a congestion stall (nothing arrives, timestamps stand still) never expires content on its own.
Reclaiming idle cached content by wall clock is a separate cache policy, not part of this budget.
Both ends apply it: the publisher skips a group rather than sending it, and the subscriber skips it again as it reads.
The publisher only ever sees the most tolerant budget across its subscribers, which is why the subscriber applies it too.
Asking for old groups with `Group Start` does not exempt them: a floor only bounds what you are sent, and a subscriber that wants history has to raise its budget to match.
That is also why the budget is what decides where delivery starts: the groups worth sending are exactly the ones it would not immediately skip.

The publisher also keeps old groups around for a best-effort **Publisher Max Age** cache window so relays and late subscribers can still fetch them. This defaults to 5 seconds.
The subscriber's max age is bounded by this window: a group can't be waited for longer than it's actually kept around.
Best-effort cuts the other way too: a cache reclaims groups nobody has read or written for a while (30 seconds by default) regardless of the advertised window, so idle history doesn't pin memory.

It is declared per track, so a publisher raises it on the tracks that are read as history rather than followed at the live edge.
hang media tracks ask for 30 seconds on that basis: a segmented egress (HLS/DASH) may only advertise segments a fetch can still reach, and a standard player starts several segments behind the live edge.
Expand Down
5 changes: 4 additions & 1 deletion rs/moq-ffi/src/origin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,10 @@ impl MoqOriginProducer {
fn from_options(options: MoqOriginOptions) -> Self {
let mut info = moq_net::origin::Info::new(moq_net::Origin::random());
if let Some(capacity) = options.cache_capacity_bytes {
info = info.with_pool(moq_net::cache::Pool::new(capacity));
let config = moq_net::cache::Config::default()
.with_capacity(capacity)
.with_expiry(info.pool.expiry());
info = info.with_pool(moq_net::cache::Pool::new(config));
}

Self { inner: spawn(info) }
Expand Down
4 changes: 4 additions & 0 deletions rs/moq-ffi/src/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ fn origin_options_set_cache_capacity() {
cache_capacity_bytes: Some(4096),
});
assert_eq!(origin.inner().info().pool.capacity(), Some(4096));
assert_eq!(
origin.inner().info().pool.expiry(),
Some(moq_net::cache::DEFAULT_EXPIRY)
);
}

#[test]
Expand Down
5 changes: 4 additions & 1 deletion rs/moq-net/benches/track.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,10 @@ struct Fanout {
impl Fanout {
fn new(subscribers: usize) -> Self {
let mut info = broadcast::Info::default();
info.origin.pool = cache::Pool::new(CACHE_CAPACITY);
let config = cache::Config::default()
.with_capacity(CACHE_CAPACITY)
.with_expiry(cache::DEFAULT_EXPIRY);
info.origin.pool = cache::Pool::new(config);
let mut broadcast = broadcast::Producer::new(info);
let track = broadcast.create_track("bench", None).unwrap();
let mut subscribers: Vec<_> = (0..subscribers).map(|_| track.subscribe(None)).collect();
Expand Down
Loading
Loading