Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion quest/m1/subscribe-ranges/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,8 @@ every sequence delivered or dropped.

## Required

- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, and a subscriber reads only its own ranges
- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, replacing `requested_group`
- [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md) - `fetch_group` becomes a one-range subscription that ends once its range is accounted for
- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft
- [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream
- [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity
Expand Down
101 changes: 78 additions & 23 deletions quest/m1/subscribe-ranges/dynamic.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@
## Goal

A publisher's `track::Dynamic` receives range requests for misses instead of
one request per group, and a track serves a subscription's ranges from one
cursor, in its order, without duplicating a group, naming every sequence in
range it won't deliver. `requested_group` and `fetch_group` are replaced, not
kept beside it.
one request per group, and every sequence in a subscription's range either
arrives or is named as dropped, so a reader never waits on a miss.
`requested_group` is replaced by `requested_fill`, not kept beside it, in every
caller and binding (the frozen moq-c ABI excepted, per the moq-c decision
below). Removing `fetch_group` is the follow-up
[fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md).

## Plan

Expand All @@ -15,26 +17,79 @@ replaced. What remains is the miss: one below the cache becomes a range request
on `Dynamic` rather than a group request. Time anchors land separately; see
[time anchors](/quest/m1/subscribe-ranges/time-anchors.md).

- Replace `requested_group` and `fetch_group` rather than keep both. Update
every caller (relay, moq-archive, hls, the ladder) in the same PR,
including the archive consumer's time resolution from its key listing if it
has landed, plus the
published surfaces built on them, per the cross-package table: moq-ffi
(`fetch_group`, `requested_group`), moq-c (`poll_requested_group`), every
binding wrapper and its `doc/lib` page, and `moq fetch` through
`moq_relay::fetch_group` (added in the 2026-10-05 audit).
- Every sequence in range either arrives or is named as dropped, so a reader
never waits on a miss. Share the accounting with
SUBSCRIBE_DROP rather than adding a second one:
the cursor's drop scan already names only sequences inside the reader's own
ranges.
- Replace `requested_group` rather than keep both. Update every caller
(sessions, relay, moq-archive, hls, the ladder) in the same PR, including
the archive consumer's time resolution from its key listing if it has
landed, plus the published surfaces built on it, per the cross-package
table: moq-ffi (`requested_group`), every binding wrapper and its `doc/lib`
page. Until the follow-up lands, a `fetch_group` miss queues a one-group
fill.
- Every sequence in range either arrives or is named as dropped. Share the
accounting with SUBSCRIBE_DROP rather than adding a second one: the cursor's
drop scan already names only sequences inside the reader's own ranges.
- Decided in the 2026-10-10 audit: `max_delay` caps every range, including
historical FETCH replacements, and publisher `max_age` always clamps it.
A FETCH with no reader delay uses publisher max-age alone, not the live
SUBSCRIBE zero-delay default. Preserve that distinction when translating
FETCH into a range request. Sharing a range never relaxes a caller's
SUBSCRIBE zero-delay default. Sharing a range never relaxes a caller's
budget or lets one expired caller cancel another still-valid reader.
Reuse FETCH's optional per-reader budget and publisher clamp rather than
restoring a historical exemption.
- Benchmark filling over range count and span as separate axes, beside
`track_range_serving`, which covers serving.

Decided by the maintainer on 2026-10-20, when the quest was first started
([moq-dev/moq#6123](https://github.com/moq-dev/moq/pull/6123)):

- **Split.** This quest replaces `requested_group`; removing `fetch_group`
is [fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md), so each
PR breaks one API once. That half needs head fills, which
[frame bounds](/quest/m1/subscribe-ranges/frame-bounds.md) provides.
- **Handler shape.** `Dynamic::requested_fill` yields a `track::Fill` naming
one bounded run: `start: Position`, exclusive end group, priority, the most

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Carry the terminal frame bound in each fill

When a requested range ends partway through a missing group, this shape conveys only the start frame and an end group, so the handler cannot distinguish ..Position { group: 5, frame: 1 } from a request for all of group 5. The planned per-group FETCH therefore over-fetches the terminal group, allowing a peer asking for one frame to induce delivery and caching of an entire group. Make the fill end a Position so downstream demand bounds upstream work.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping the end as an exclusive group, per the maintainer's handler-shape decision. A fill populates the shared cache, not one reader: a group fetched only up to some frame would sit in the cache missing its tail, which a later reader wanting the whole group cannot tell from a complete one, and there is no tail fill to repair it (frame bounds covers head fills only). The work is also bounded the same either way: a peer asking for frame 0 of group 5 could ask for all of group 5, so the terminal group costs at most one group it was entitled to request. Each reader still sees only its own frames through the cursor's frame bounds.

(Written by Claude Opus 5.5)

tolerant budget among its readers, and order. Each reader's cursor still
skips what is stale under its own `max_delay`, so sharing never relaxes it.
The handler calls `accept(sequence, info)` for each group it has and
`drop_groups(range)` for the rest. Dropping the `Fill`
names whatever is left as dropped, in the track's existing `drops`, but only
while a reader still wants it: once demand leaves, the fill is withdrawn like
a `group::Request` today and records nothing, so `drops` (durable) never
hides still-fetchable history from a later reader.
Rejected: writing through `Producer::create_group`, since a fetch-only copy
or an archive track has no live producer, and backfill must stay out of the
arrival walk.
- **What a miss is.** A sequence in a reader's window, below its cap, that
the cache doesn't cover and the feed will never deliver: below the feed's
start, or at or below its newest group and not awaited by the arrival
ledger (evicted or skipped). Nothing above that frontier is a miss, since it
arrives live, so a fill is bounded by construction. A relay records the
span a SUBSCRIBE_START skipped past its requested floor as dropped (today it
only settles the tail), or it would FETCH what upstream already gave up.
- **Fetch budget.** `Subscription::max_delay` becomes `Option<Duration>`:
`None` means publisher max-age alone, the FETCH default above. This breaks
`Subscription` in every binding, with a `MIGRATE.md` entry in each affected
Comment on lines +63 to +65

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Define how frozen C represents an absent delay

If the generated C package has not landed, which lines 85-90 explicitly allow, the current frozen moq_subscription.max_delay_us is a required u64 and cannot represent None. Requiring this change in every binding therefore forces the implementation either to break the frozen C ABI, introduce an unspecified sentinel despite rejecting that representation, or silently omit publisher-only max-age semantics for C callers. Settle the C representation or make the generated package a prerequisite.

AGENTS.md reference: AGENTS.md:L59-L60

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ea5166a, within the maintainer's moq-c decision (frozen ABI unchanged unless moq-dev/moq#6052 lands first): the frozen crate maps max_delay_us to Some, so a frozen C subscriber keeps today's semantics and cannot ask for max-age alone. The generated package exposes the Option natively once it lands.

(Written by Claude Opus 5.5)

package. Rejected (recommended, but overruled): a `Duration::MAX` sentinel
that the wire saturates, which avoids the break but encodes the meaning in a
magic value.
- **Sessions.** A session serves a fill as it serves a group request today:
one FETCH per group, one in flight per fill, newest first for a descending
reader, withdrawing (not dropping) the fill when demand leaves. Once a
fetched group is stale under the fill's budget, the groups older than it
are too (a timed track's starts never rewind), so the fill stops fetching
them; in either order, newer groups still arrive. They are withdrawn, not
dropped, and each reader's cursor names them stale under its own budget,
since a later reader with a wider one may still fetch them. Only staleness
under publisher max-age, which binds every reader, goes in `drops`.
[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) turns lite-07
fills into range subscriptions, and
[moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) into one FETCH
per run. Until then, one SUBSCRIBE range can cost up to frontier minus floor
serialized upstream FETCHes, each bounded by the fill's budget and stopped
when demand leaves, the same cost per-group requests carry today; accepted
rather than blocking on those quests.
- **moq-c.** Whichever lands first decides it. If the generated C package
([moq-dev/moq#6052](https://github.com/moq-dev/moq/pull/6052)) has landed,
`poll_requested_group` leaves the hand-written ABI with a `MIGRATE.md` entry
and the fill reaches C through the generated API. Otherwise the frozen crate
splits each fill into its per-group requests internally, keeping its ABI
unchanged, and maps `moq_subscription.max_delay_us` to `Some`: a frozen C
subscriber cannot ask for max-age alone, as today. Not a blocker.

## Related

- [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md) - removes `fetch_group` once fills land
38 changes: 38 additions & 0 deletions quest/m1/subscribe-ranges/fetch-range.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# [M] Fetch as a range

## Goal

`fetch_group` is gone: every caller reads a past group through a one-range
subscription, which ends once its range is accounted for. That covers the
relay, moq-archive, hls, moq-ffi and every binding wrapper with its `doc/lib`
page, and `moq fetch` through `moq_relay::fetch_group`. A benchmark covers
filling over range count and span as separate axes.

## Plan

Split out of [dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md)
by the maintainer on 2026-10-20
([moq-dev/moq#6123](https://github.com/moq-dev/moq/pull/6123)), so each PR
breaks one API once.

- **Ending a bounded read** (decided 2026-10-20). `recv_group` returns `None`
once every sequence in a bounded window is delivered, dropped, stale, or
unfillable. Today it parks past the window's end on purpose, so an update
can still widen it; a reader that wants more keeps an open range instead.
`Ok(None)` before any group arrives is a miss, which the moq-ffi fetch
wrapper reports as `NotFound`; after one, it is the range's end. A one-range
read surfaces no `NotFetchable` or `Old` result, and a peer's reset with
either names the group dropped. Every fetch boundary already treats
`NotFetchable` as not found (relay HTTP 404, hls `is_cache_miss`, moq-ffi
`map_fetch_error`, moq-c `NOT_FOUND`); a fetched `Old` becomes a miss too,
where relay HTTP answers 500 and moq-c reports `OLD` today. Both stay as wire
stream errors, and `Old` still aborts a cut group mid-read.
Each affected package gets a `MIGRATE.md` entry.
- A one-range subscription for a cached live group missing the frames its
range starts at needs a head fill, which is why this requires frame bounds.
- Benchmark filling beside `track_range_serving`, which covers serving.

## Required

- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `requested_fill` and subscriptions filling their misses
- [Frame bounds](/quest/m1/subscribe-ranges/frame-bounds.md) - head fills for a range starting mid-group
Loading