Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m2/unbounded-groups/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,4 +183,4 @@ the model layer (maintainer, 2026-10-30).
- [Byte holes](/quest/m2/unbounded-groups/byte-holes.md) - segments inside a frame shed like front frames and flights start at `(frame, byte)` on the current lite wip; the frame cap goes
- [JS byte holes](/quest/m2/unbounded-groups/js-byte-holes.md) - `@moq/net` mirrors the byte-offset wire and continues a frame cut off mid-way
- [Compressed frames have no size cap and inflate as a bounded stream](/quest/m2/unbounded-groups/compressed-stream.md) - A compressed frame of any size publishes and is received in Rust and `@moq/net`, like a plain one
- [Run-based relay fills are paced](/quest/m2/unbounded-groups/run-fill-pacing.md) - FETCH runs and lite-07 range fills backpressure upstream once a reader loses a batch to eviction
- [Lite-07 history runs are paced](/quest/m2/unbounded-groups/run-fill-pacing.md) - range fills, recoveries, and a reader's own history range backpressure upstream once a reader loses a group to eviction
68 changes: 56 additions & 12 deletions quest/m2/unbounded-groups/run-fill-pacing.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,67 @@
# [M] Run-based relay fills are paced
# [M] Lite-07 history runs are paced

## Goal

A relay pacing a run-based fill (a moq-transport FETCH run, or a lite-07
range fill) backpressures the upstream stream once it sees a downstream reader
lose a batch to eviction, so a slow reader of a long run stays within the
tail instead of falling off it.
A lite-07 copy bringing a long run of history to a slow reader (a range fill, a
recovery, or the reader's own history range on the copy's subscription)
backpressures the upstream once it sees a downstream reader lose a group to
eviction, so a slow reader of a long run stays within the tail instead of
falling off it.

## Plan

Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`), from
[moq-dev/moq#8017](https://github.com/moq-dev/moq/pull/8017), which paces a
batch fill's handler by its fastest reader. Run-based fills still read
upstream at full speed.
batch fill's handler by its fastest reader. Pace on evidence, not up front: an
unpaced run is fine while every reader keeps up.

Pace on evidence, not up front: an unpaced run is fine while every reader
keeps up, so start backpressure only after the relay sees a downstream
reader lose a batch to eviction, then hold the upstream stream as #8017
holds a batch handler. Test a slow reader of a long FETCH run and of a
lite-07 range fill through a relay.
Landed in [moq-dev/moq#8117](https://github.com/moq-dev/moq/pull/8117): the
evidence is the track's eviction count (`TrackState::evictions`, bumped when a
reader loses a group it judged cached), and a moq-transport FETCH run holds its
fetch stream unread at each group boundary until a reader picked up the group
before (`Fill::poll_run_paced`), through the session's pause budget.

What remains is lite-07, which that PR's prototypes found harder than the plan
assumed:

- History reaches a lite-07 copy three ways, all unpaced: the copy's upstream
subscription carrying a reader's own history range (the common case, and not a
fill), `RangeFills` asking a held fill's run through that subscription, and a
`Recovery` subscription asking again for what the subscription already
accounted for. Each arrives as one group stream per group, concurrently.
- Holding each arriving group stream unread until the fill's readers picked up
the one before admits streams in arrival order, not the fill's, so groups a
reader takes last evict each other first. A slow reader of 40 groups on a
4-group cache still got 10.
- Fetching a paced fill a group at a time instead (lite's `FillServeRun`, paced
since #8017) got that reader 26 of 40, but costs a round trip per group, which
#8017 and [moq-dev/moq#7339](https://github.com/moq-dev/moq/pull/7339)
rejected, and paces neither the subscription's own history range nor several
fills at once.
- #8017's per-fill pacing does not hold when one reader has several fills out,
split by groups already cached or by refills: each fill moves on as the reader
picks up its answer, so together they still outrun it. This affects batch
handlers and FETCH runs too.

The mock transport moves a whole run in an instant, before any reader polls, so
a test needs the evidence first: one slow reader loses part of a run, then a
second slow reader of another long run gets all of it.

Public API: none expected. Wire: none.

## Waiting

- Maintainer: pick lite-07's pacing mechanism. (a) Hold arriving group streams
unread and admit them in the fill's order, within the session's pause budget,
extended to the history range a reader's subscription carries: no round trips,
but out-of-order streams spend the pause budget and read on unpaced once it
runs out. (b) Narrow the subscription's ranges to the next owed group once
paced: in-order, but a SUBSCRIBE_UPDATE and a round trip per group. (c) Fetch
paced runs a group at a time: reuses `FillServeRun`, but a round trip per group
and the subscription's own range stays unpaced. (d) Drop lite-07 pacing and
rely on refill-once. Recommended: (a), since it is the plan's "hold the
upstream stream" without round trips; pair it with pacing per reader across
its fills (a read picks up one answer per group it takes, all of them while it
waits), so several fills of one reader cannot outrun it. An agent cannot take
this: (b) and (c) reopen #7339's rejection, and the per-reader change rewrites
#8017's settled pick-up rule.
7 changes: 4 additions & 3 deletions rs/moq-net/src/coding/reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -474,12 +474,13 @@ impl<S: RecvStream, V: StreamCodes> Reader<S, V> {
self.poll_pace(waiter, group.poll_admit(waiter, header, self.stream.window()))
}

/// Wait out `paced`, the group's pace gate, if the session has room for the credit this
/// stream holds while paused (see [`Pause`]); otherwise read on, and the group sheds.
/// Wait out `paced`, a pace gate on what this stream writes next, if the session has room
/// for the credit this stream holds while paused (see [`Pause`]); otherwise read on,
/// unpaced.
///
/// A paused stream is not read, so it is watched for a reset instead, which fails the
/// wait rather than leaving it to the group's reader to notice.
fn poll_pace(&mut self, waiter: &kio::Waiter, paced: Poll<()>) -> Poll<Result<(), Error>>
pub(crate) fn poll_pace(&mut self, waiter: &kio::Waiter, paced: Poll<()>) -> Poll<Result<(), Error>>
where
S: Pause,
{
Expand Down
170 changes: 154 additions & 16 deletions rs/moq-net/src/ietf/subscriber.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7528,6 +7528,12 @@ impl RecvGroupFetch {
/// properties, never past FETCH_OK's End Location. A group's objects end where the next
/// group's begin, since a FETCH carries every object of its range in order. Before
/// draft-22 the stream must also reach an object bound inside the last group.
///
/// Once a reader of the track lost a group to eviction, each group after the first waits,
/// its stream unread, until a reader picked up the one before, so a run wider than the
/// cache never evicts its own groups ahead of its readers (see [`track::Fill::poll_run_paced`]).
/// The pause holds receive credit, so it is taken only while the session's paused streams
/// leave room for it, as within a group.
struct GroupFetchObjects {
version: Version,
/// The next Object ID expected in the current group.
Expand All @@ -7536,6 +7542,9 @@ struct GroupFetchObjects {
first: bool,
/// The object whose payload is being read: its group, its Object ID, and where it goes.
payload: Option<(u64, u64, FetchPayloadRead)>,
/// The first object of the next group and its sequence, read while the run waits for its
/// readers to pick up the group before.
paced: Option<(FetchedObject, u64)>,
}

/// Where a fetch object's payload goes.
Expand All @@ -7557,6 +7566,7 @@ impl GroupFetchObjects {
prior_group: None,
first: true,
payload: None,
paced: None,
}
}

Expand Down Expand Up @@ -7585,22 +7595,38 @@ impl GroupFetchObjects {
}
}

let Some(object) = ready!(poll_fetch_object(stream, self.first, waiter))? else {
break;
};
self.first = false;
if !object.subgroup_ok && !object.datagram {
tracing::warn!("subgroup ID is not supported, dropping group fetch");
return Poll::Ready(Err(Error::Unsupported));
}
let (object, sequence) = match self.paced.take() {
Some(paced) => paced,
None => {
let Some(object) = ready!(poll_fetch_object(stream, self.first, waiter))? else {
break;
};
self.first = false;
if !object.subgroup_ok && !object.datagram {
tracing::warn!("subgroup ID is not supported, dropping group fetch");
return Poll::Ready(Err(Error::Unsupported));
}

let group = resolve_fetch_group(self.prior_group, object.group)?;
if let Some(group) = group {
self.prior_group = Some(group);
}
// The first object always names its group, so a later one leaving it out stays in it.
let sequence = group.or(writer.sequence()).ok_or(Error::ProtocolViolation)?;
let group = resolve_fetch_group(self.prior_group, object.group)?;
if let Some(group) = group {
self.prior_group = Some(group);
}
// The first object always names its group, so a later one leaving it out stays in it.
let sequence = group.or(writer.sequence()).ok_or(Error::ProtocolViolation)?;
(object, sequence)
}
};
if writer.sequence() != Some(sequence) {
// The group before is whole, so its readers see it end while the stream waits,
// unread, for them to pick it up: flow control then holds the publisher back.
writer.finish()?;
match stream.poll_pace(waiter, writer.fetch.fill.poll_run_paced(waiter)) {
Poll::Ready(res) => res?,
Poll::Pending => {
self.paced = Some((object, sequence));
return Poll::Pending;
}
}
self.next = writer.open(sequence, object.datagram)?;
} else if object.datagram {
// A datagram group has one object, so it never shares one with a stream.
Expand Down Expand Up @@ -17115,6 +17141,116 @@ mod stitch_tests {
assert!(matches!(receive.await, Err(Error::Evicted)));
}

/// What a slow reader of a FETCH run ten times wider than the cache gets, taking a group
/// each time the run's stream is polled, once a reader of the track lost a group to
/// eviction or not. An ascending reader reads in sequence, a descending one as groups
/// arrive.
async fn read_a_fetch_run_slowly(outran: bool, order: track::Order) -> Vec<u64> {
const GROUPS: u64 = 40;
const SIZE: usize = 10_000;
let payload = vec![0u8; SIZE];
let mut objects = Vec::new();
// The first Group ID is 0, and each later one, a delta, 0 again: the next group.
for _ in 0..GROUPS {
objects.extend(group_fetch_objects(0, 0, &[&payload[..]]));
}
// The session grants a stream credit of one group, so a paused stream holds one.
let window = crate::transport::ReceiveWindow::new(GROUPS * SIZE as u64, SIZE as u64);
let session = ScriptedSession::per_stream_eof(vec![objects]).with_receive_window(window);
let mut run = GroupFetchRun::with_session(VERSION, session).await;
let pool = crate::cache::Pool::new(crate::cache::Config::default().with_capacity(4 * SIZE as u64));
run.track = track::Producer::new(
std::sync::Arc::new(crate::broadcast::Info {
pool,
..Default::default()
}),
"video",
track::Info::default().with_timescale(None).with_max_age(None),
);
let dynamic = run.track.requests();
run.track.start_at(GROUPS).unwrap();
let mut live = run.track.create_group(group::Info { sequence: GROUPS }).unwrap();
live.write_frame(None, &payload[..]).unwrap();
let waiter = kio::Waiter::noop();
if outran {
// A reader judges the live edge cached and loses it before taking it.
let mut lost = run
.track
.subscribe(
track::Subscription::default()
.with_ranges([track::Range::groups(0..GROUPS + 1)])
.with_delay(track::Delay::default().with_max(None)),
)
.ordered();
assert!(lost.poll_next_group(&waiter).is_pending());
run.track.evict(GROUPS);
}

let subscriber = run.track.subscribe(
track::Subscription::default()
.with_ranges([track::Range::groups(0..GROUPS)])
.with_order(order)
.with_delay(track::Delay::default().with_max(None)),
);
type Read = Box<dyn FnMut(&kio::Waiter) -> Poll<crate::Result<Option<group::Consumer>>>>;
let mut reader: Read = match order {
track::Order::Ascending => {
let mut ordered = subscriber.ordered();
Box::new(move |waiter| ordered.poll_next_group(waiter))
}
track::Order::Descending => {
let mut subscriber = subscriber;
Box::new(move |waiter| subscriber.poll_recv_group(waiter))
}
};
assert!(reader(&waiter).is_pending(), "group 0 is a miss");
let fill = dynamic.requested_fill().await.expect("the reader asked for the run");
assert_eq!((fill.start().group, fill.end()), (0, GROUPS));
let slot = kio::Producer::new(GroupFetch::Ready(RangeFetch {
fill: std::sync::Arc::new(fill),
timescale: None,
first: 0,
last: GROUPS - 1,
start: 0,
end: None,
}));
let mut receiving = run.subscriber.clone();
let mut receive = std::pin::pin!(receiving.recv_group_fetch(&mut run.stream, slot));
let (mut read, mut received) = (Vec::new(), false);
for _ in 0..4 * GROUPS {
if !received && let Poll::Ready(res) = futures::poll!(&mut receive) {
res.expect("the run is received");
received = true;
}
match reader(&waiter) {
Poll::Ready(Ok(Some(group))) => read.push(group.sequence),
// What it lost is given up, which ends its range.
Poll::Ready(Ok(None)) => break,
Poll::Ready(Err(err)) => panic!("the reader failed: {err}"),
Poll::Pending if received => break,
Poll::Pending => {}
}
}
read
}

/// Once a reader of the track lost a group to eviction, a FETCH run writes its next group
/// only once a reader picked up the one before, leaving the stream unread meanwhile, so a
/// slow reader of a run far wider than the cache gets all of it. Until then the run moves
/// at the stream's speed and evicts what the reader has yet to take.
#[kio::test]
async fn a_fetch_run_waits_for_its_reader_once_one_lost_a_group() {
for order in [track::Order::Ascending, track::Order::Descending] {
let paced = read_a_fetch_run_slowly(true, order).await;
assert_eq!(paced, (0..40).collect::<Vec<_>>(), "{order:?}");
let unpaced = read_a_fetch_run_slowly(false, order).await;
assert!(
unpaced.len() < 40,
"the unpaced run outran its {order:?} reader: {unpaced:?}"
);
}
}

#[kio::test]
async fn internal_group_fetch_setup_cancellation_is_cancel() {
let mut run = GroupFetchRun::new(VERSION, Vec::new()).await;
Expand Down Expand Up @@ -17160,7 +17296,9 @@ mod stitch_tests {
tasks.0.clone(),
Default::default(),
);
let (_, recv) = session.open_bi(Default::default()).await.unwrap();
// Through the subscriber's transport, so it pauses as a session's stream does where
// the session grants a receive window.
let stream = Stream::open(&subscriber.session, version).await.unwrap();
let track = track::Producer::new(
std::sync::Arc::new(crate::broadcast::Info::default()),
"video",
Expand All @@ -17169,7 +17307,7 @@ mod stitch_tests {

Self {
subscriber,
stream: Reader::new(crate::transport::boxed::RecvStream::new(recv), version),
stream: stream.reader,
track,
_tasks: tasks,
}
Expand Down
14 changes: 14 additions & 0 deletions rs/moq-net/src/lite/test_transport.rs
Original file line number Diff line number Diff line change
Expand Up @@ -974,6 +974,8 @@ pub struct ScriptedSession {
groups: bool,
/// Set by [`Self::with_send_window`].
send_window: Option<usize>,
/// Set by [`Self::with_receive_window`].
receive_window: Option<crate::transport::ReceiveWindow>,
}

impl ScriptedSession {
Expand Down Expand Up @@ -1014,9 +1016,17 @@ impl ScriptedSession {
unacked_unis: Default::default(),
groups: false,
send_window: None,
receive_window: None,
}
}

/// Report `window` as the receive windows in force, so a stream the session stops reading
/// holds credit.
pub fn with_receive_window(mut self, window: crate::transport::ReceiveWindow) -> Self {
self.receive_window = Some(window);
self
}

/// Let each opened bidi stream send only its first `bytes`, then park its writes for good,
/// like a peer that never grants more send credit.
pub fn with_send_window(mut self, bytes: usize) -> Self {
Expand Down Expand Up @@ -1147,6 +1157,10 @@ impl poll::Session for ScriptedSession {
type RecvStream = ScriptedRecv;
type Error = SinkError;

fn receive_window(&self) -> Option<crate::transport::ReceiveWindow> {
self.receive_window
}

fn poll_accept_uni(&self, waiter: &kio::Waiter) -> Poll<Result<Self::RecvStream, Self::Error>> {
let Some(script) = self.incoming_unis.lock().unwrap().pop_front() else {
return self.log.poll_closed(waiter).map(Err);
Expand Down
11 changes: 11 additions & 0 deletions rs/moq-net/src/model/track/fill.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1060,6 +1060,17 @@ impl Fill {
.map(drop)
}

/// Poll until a run's next group may be written: at once until a reader of the track lost
/// a group to eviction, then as [`Self::poll_paced`]. An unpaced run moves at its source's
/// speed, which is fine while every reader keeps up.
#[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))]
pub(crate) fn poll_run_paced(&self, waiter: &kio::Waiter) -> Poll<()> {
Comment thread
pullfrog[bot] marked this conversation as resolved.
Resolved
if self.inner.state.read().evictions == 0 {
return Poll::Ready(());
}
self.poll_paced(waiter)
}

/// Refuse every group still owed with `err`, which says nothing about whether the source
/// has them, so nothing is recorded dropped.
#[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))]
Expand Down
Loading