Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions quest/m1/missing-head-asks-upstream.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# [M] A relay fetches a group's missing head instead of dropping it

## Goal

When a relay's copy holds only the later frames of a group a subscription
covers, as when its upstream subscription started at a Position, the relay
fetches the earlier frames from upstream and delivers the group whole. It
names the group in SUBSCRIBE_DROP only when upstream can't supply them. A
subscriber gets the same group through the relay, warm or cold, as it would
from the publisher. Rust and `@moq/net` match, and the moq-lite draft says so.

## Plan

Found by the 2026-11-20 audit of the subscription rules before the lite-07
cut. A cache only makes things faster (root `AGENTS.md`), so a partial hit
asks upstream for the miss. The draft's Positions section says the opposite:
"A publisher that holds only the later frames of a Group a subscription
covers names it in a SUBSCRIBE_DROP rather than deliver the part it holds".
The Rust publisher does that. `position_group` (`rs/moq-net/src/lite/publisher.rs`)
finds no head, and its caller logs "skipping group with a missing head" and
drops the group with `NotFound` (test `a_missing_head_is_dropped_early`).
Check what `@moq/net`'s publisher does.

- Recommended: the copy FETCHes the missing frames (`Frame Start` 0 through
the first frame it holds) through the group-fill path a miss already uses,
and the publisher serves the group once the head lands, within the
subscriber's budget. A failed or refused fetch falls back to the DROP.
- The draft's sentence becomes: a relay fetches the missing earlier frames
from upstream, and a publisher that can't obtain them names the group in a
SUBSCRIBE_DROP. A subscriber already handles both.
- A copy holding the earlier frames of a group its upstream closed short keeps
delivering them and closing without GROUP_END. Upstream has no more, so
that matches a direct read.

Reproduce first: a relay whose first reader resumes at a Position partway
through a group, then a second reader whose range covers that group whole.

Public API: none expected. Wire: none; the draft's Positions rule changes for
relays.
52 changes: 52 additions & 0 deletions quest/m1/switch-caps-old.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# [L] A route switch caps the old subscription at one boundary

## Goal

When a front moves a track to another route of the same instance, the reader
sees one subscription split at one boundary, chosen at the switch. The
boundary is the largest group any copy holds, past the frames held
contiguously in it. The new subscription starts there. The old one is capped
at that group on the wire at once and drains what lies below it within the
budget. Each frame is delivered once, from whichever copy brings it first.
Rust and `@moq/net` match.

## Plan

Found by the 2026-11-20 audit of the subscription rules before the lite-07
cut. Today:

- Rust `Reader::sync` (`rs/moq-net/src/model/resume.rs`) resumes from the
newest group handed to the reader (`held_at`, `resume_at`), not the largest
any copy holds. JS `sync` (`js/net/src/resume.ts`) asks that group from
frame 0, which [js-route-resume](/quest/m4/js-route-resume.md) owns.
- A switch only marks the old copy replaced. Its bound is chosen later, by
`observe` when the replacement hands out its first group, or by
`bound_at_end` (JS `#observe`, `#boundAtEnd`). It reaches the wire as
SUBSCRIBE_UPDATE only if the copy's aggregate demand narrows, so until then
both routes send every new group.

Decided 2026-11-20 (maintainer, /quest-plan): ✅ cap the old at the switch, by
group (rejected: keep today's late bound; cap by frame now, which
[switch-frame-cap](/quest/m1/switch-frame-cap.md) adds on top of this).

- At the switch, pick the boundary group G and frame N. The replacement starts
at Position (G, N). The old copy's ranges end at G, so it finishes G and
sends nothing newer. That goes out as SUBSCRIBE_UPDATE right away.
- The old copy keeps draining every group below G within the budget, as
`let_go` already allows. A group it can't deliver is asked of the new route
(`backfill_gaps`).
- Once the replacement delivers past the old copy's position in G, stop only
G's stream on the old copy, which also ends a group that never finishes.
- Run the resume and failover models (`/verify-resume`, `/verify-failover`)
after updating them, since they encode today's bound. Teach the resume
oracles (`resume_model.rs`, `resume.model.test.ts`) the fixed boundary.

Test: a switch while the old route still delivers sends no group above G from
the old route, loses no group below G, and delivers G's frames once, in both
languages.

Public API: none expected. Wire: none.

## Related

- [A replaced copy's short group continues from the replacement](/quest/m1/switch-partial-close.md) - the other gap the audit found at a switch
41 changes: 41 additions & 0 deletions quest/m1/switch-frame-cap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# [M] A route switch caps the old subscription at a frame

## Goal

A lite-07 Range can end at a Position (a group's start and a frame), so a
route switch caps the old subscription at the exact frame where the
replacement starts. The old publisher sends nothing of the boundary group
past that frame, beyond what was already in flight. Rust, `@moq/net`, and the
moq-lite draft change together.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan auditing the subscription rules
before the lite-07 cut): a switch caps the old subscription by group first
([switch-caps-old](/quest/m1/switch-caps-old.md)), and this quest adds the
frame cap on top. It isn't Required by
[the lite-07 cut](/quest/m2/lite07-finalize.md). It rides lite-07 if it has
landed when the cut is made, and otherwise moves to the next wip version.

- Today a Position is a `Start` only (draft, SUBSCRIBE's Anchor Type), and the
Positions section says a subscription "never stops partway through" a
Group. Allow Position as an `End` too, ending before that frame. Check how
it reads beside "both ends include the Group they name".
- A publisher reaching a Position end stops the group's stream without a FIN
(maintainer: a later SUBSCRIBE_UPDATE may widen the range to include the
rest). The stream stays open until the range widens, the group expires, or
the subscription ends. Account for the credit it holds, per moq-net's rule
that no stream is finished and forgotten.
- The switch then ends the old copy's ranges at Position (G, N) instead of
group G, and drops the step that stops G's stream once the replacement
passes it.
- moq-transport has no frame end; its publisher serves the whole group, as
today.
- Run `just test interop --all`.

Public API: the range end type gains a Position in Rust, JS, and the
bindings. Wire: lite-07-wip if before the cut, else the next wip version.

## Required

- [A route switch caps the old subscription at one boundary](/quest/m1/switch-caps-old.md) - the switch-time cap this narrows to a frame
36 changes: 36 additions & 0 deletions quest/m1/switch-partial-close.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# [M] A replaced copy's short group continues from the replacement

## Goal

A group that a replaced copy closes short (FIN without GROUP_END) continues
from the replacement at the next frame, so the reader gets every frame the
group holds, as if read from one route. The group ends early only when no
route can supply more. Rust and `@moq/net` match.

## Plan

Found by the 2026-11-20 audit of the subscription rules before the lite-07
cut, and reproduced in JS. In the resume test "a stalled group continues from
the replacement at the next frame", have the old copy close its one-frame
group with `finish()` before `swap()`, under a five-minute budget. The other
copy holds three matching frames, yet the reader ends the group after frame 0.

- Rust: a bare FIN finishes the received group (`recv::Group::end(None)`,
`rs/moq-net/src/recv.rs`), and `poll_resumed` (`model/group.rs`) treats a
finished read as needing no route ("Read to its end").
- JS: `#readFrame` (`js/net/src/group.ts`) returns the end: "The copy finished
the group: a replacement asked for meanwhile is moot".
- Only when the reader parks on the next frame before the FIN lands does
`Recover::wants` move it to the replacement. Which happens depends on
arrival order.

Recommended: a read that reaches a closed group's end on a copy the front has
replaced, or while another route may hold more, recovers the next frame from
the replacement as a stall does. It ends the group only once a copy holds it
whole, or the replacement also has nothing past that frame. A group read to a
GROUP_END is whole and ends as today.

Add the reproduction in both languages, and the case to the resume models
(`/verify-resume`), whose oracles should require every frame some route held.

Public API: none expected. Wire: none.
92 changes: 92 additions & 0 deletions quest/m2/groups-never-overlap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# [L] Every frame of a group presents before every frame of a higher group

## Goal

Groups never go back in time: every frame of a group presents strictly before
every frame of any higher group, so a reader bounds a group that isn't whole
by the lowest timestamp it knows in a higher group. Frames within a group may
still reorder (B-frames, open-GOP leading pictures). Sources that overlap
today, such as a TS content join, publish without ending the broadcast and
without rewriting a timestamp. The hang draft states the rule, and the
producers and every importer keep it, in Rust and JS.
[Group times](/quest/m2/time-addressing/group-times.md) puts it in the moq-lite
draft, as spans from a group's lowest frame to its End that never overlap.

## Plan

[moq-dev/moq#4543](https://github.com/moq-dev/moq/pull/4543) replaced the
importers' live-edge floor and TS `Reanchor` shift with monotonic group starts.
With nothing shifted, a TS content join
([#3798](https://github.com/moq-dev/moq/issues/3798)) publishes an IDR below
the old content's last frame, so the draft now lets frames present below the
previous group's content (`drafts/draft-lcurley-moq-hang.md`, the malformed-group
rules), and even lets a group start at the previous group's start.
`container::Producer::prepare` opens a group for a keyframe above the open
group's start, even one below its newest frame
(`a_keyframe_overlapping_the_previous_group_publishes`). The moq-lite draft
only orders frame 0s ("a publisher MUST NOT start a Group's frame 0 at or
before that of any Group with a lower start", in its Frame section), and
`@moq/net` accepted a frame at 250 in group 100 after group 200 opened.

That overlap is why
[moq-dev/moq#8874](https://github.com/moq-dev/moq/pull/8874) cannot place an
HLS group early with certainty: when the overlap straddles a grid line, an
edge that saw the earlier group and one that joined at the overlapping group
list one segment differently. #8874 documents that case in
`rs/moq-hls/src/export/grid.rs` and `doc/bin/hls.md`; delete both notes here.

Start by finding every source that overlaps and why: the TS content join,
other importers (fMP4, FLV, MKV, RTMP, SRT, HLS), encoders, and the JS
publisher.

Decided 2026-11-15 (maintainer, /quest-plan):

- ✅ Enforce the rule end to end: draft, producer, importers (rejected:
investigate only and plan the fix later; end the import at an overlapping
keyframe, which would stop a TS feed at every content join).
- ✅ A keyframe that doesn't start after the open group's newest frame continues
that group, as one at or below its start already does (rejected: shift the
broadcast forward at each join, which brings back the rewriting #4543
removed and drifts the catalog clock; leave it to the implementer). The
group at a join then runs about two GOPs: a viewer can't join exactly there,
and HLS lists one over-long segment through its overrun path.
- ✅ Readers assume the rule and don't detect it, per AGENTS.md (rejected:
`container::Consumer` refusing an overlapping group). A publisher that
breaks it hurts only its own broadcast. Delete any reader-side overlap
handling the rule makes dead. moq-net's frame 0 check (`check_start`) stays
as it is.

Decided 2026-11-20 (maintainer, /quest-plan auditing the subscription rules
before the lite-07 cut):

- ✅ The strong form: every frame of a lower group presents before every frame
of a higher one (rejected: the weak form, every frame before the next
group's frame 0, which bounds an incomplete group only by a higher group's
frame 0). Conforming HEVC and H.264 already keep it, since a leading picture
follows every picture of the previous GOP in output order. Only a splice can
break it.
- ✅ The cut waits on this quest (moved from m3): lite-07 publishes the rule
as a publisher MUST through group times, and first-party producers break
it today.

Recommended for a splice: a frame after the keyframe that would present at or
before the previous group's newest frame, such as a RASL picture, is dropped.
It references pictures from before the splice, so it can't decode there
anyway. Check RADL pictures, which do decode, before settling on dropping
them too.

The hang draft's "a group MAY start at the same timestamp as the previous
group" goes too.

Tests: the producer test above becomes one where the overlapping keyframe
continues the group, and a TS import across a content join, including leading
pictures, publishes groups whose frames all precede every frame of the next
group.

Public API: none expected. Wire: none; the hang draft's group rule tightens.

## Related

- [Group Streams carry each time once](/quest/m2/time-addressing/group-times.md) - states the rule in the moq-lite draft, with GROUP_END's End closing each span
- [An incomplete group expires by the lowest timestamp of a higher group](/quest/m3/expire-by-higher-min.md) - the expiry bound this rule makes safe
- [An fMP4 group start never rewinds the decode timeline](/quest/m3/fmp4-group-start-tfdt.md) - a keyframe repeating its predecessor's timestamp would now continue the group; check which of its cases remain
18 changes: 18 additions & 0 deletions quest/m2/lite07-finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,29 @@ The padding paragraph above is superseded: lite-07 freezes with no Padding
type, and [the padding probes](/quest/m5/probe-padding.md) bring one back in
the version that ships them.

Decided 2026-11-20 (maintainer, /quest-plan auditing the cache, timestamp,
and switching rules in `rs/moq-net/AGENTS.md` before the cut):

- ✅ The cut waits for [groups never overlap](/quest/m2/groups-never-overlap.md)
(rejected: after the cut). [Group times](/quest/m2/time-addressing/group-times.md)
puts the strong rule in the draft as a publisher MUST: a group's span, from
its lowest frame to its End, never overlaps another's. Expiry relies on it,
and first-party producers break it today.
- ✅ [The frame cap](/quest/m1/switch-frame-cap.md) is not Required, like the
warmth port: if it has landed when the cut is made, its Position end is in
lite-07, and otherwise it moves to the next wip version.
- The audit's other gaps change no wire and don't gate the cut: a relay
asking upstream for a missing group head, the switch's group cap, a
replaced copy's short group, and expiry by a higher group's lowest
timestamp. A Min Age join that asks upstream is
[no relative anchors](/quest/m2/time-addressing/no-relative-anchors.md)'s.

## Required

- [No relative anchors](/quest/m2/time-addressing/no-relative-anchors.md) - SUBSCRIBE's anchors lose Relative, and Min Age places a live join
- [Group Streams carry each time once](/quest/m2/time-addressing/group-times.md) - GROUP loses Previous Start, FRAME's first delta counts from the start, and GROUP_END carries an end
- [Cache-aware multicast](/quest/m2/multicast/README.md) - lite-07 carries the redesigned route layer: routes to forwarding nodes, no holds or doubt, claims crossing trust boundaries by a chain, and no Route Key
- [Groups never overlap](/quest/m2/groups-never-overlap.md) - first-party producers and importers keep the no-overlap MUST the cut publishes
- [A lite-07 claim is shown no broader than the grants it crossed](/quest/m2/lite07-claim-reach-routes.md) - a peer is never shown a claim at a path no route of it can serve, through any number of relays (maintainer, 2026-11-20)

## Related
Expand Down
4 changes: 2 additions & 2 deletions quest/m2/time-addressing/group-times.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Recommended, for review:
bounded by its successor's start, as today.
- With open-GOP leading pictures, a group's lowest frame sits below its
start. Check the Time Anchor rule against spans there, alongside
[groups never overlap](/quest/m3/groups-never-overlap.md).
[groups never overlap](/quest/m2/groups-never-overlap.md).
- `group::Producer::finish_at(end)` and `group::Consumer::end()` stay, since
the end is what GROUP_END encodes.

Expand Down Expand Up @@ -113,4 +113,4 @@ which is unpublished.

- [lite-07 has no relative anchors](/quest/m2/time-addressing/no-relative-anchors.md) - its live join resolves as a time start, which this changes inside a skipped span
- [A scale-less track ages by arrival](/quest/m2/time-addressing/scale-less-arrival.md) - lite-05/06 still stamp a scale-less frame, with its group's start
- [Groups never overlap](/quest/m3/groups-never-overlap.md) - enforces the non-overlap rule in hang's producers and importers
- [Groups never overlap](/quest/m2/groups-never-overlap.md) - enforces the non-overlap rule in hang's producers and importers
50 changes: 50 additions & 0 deletions quest/m3/expire-by-higher-min.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# [M] A group that isn't whole expires by a higher group's lowest timestamp

## Goal

Timestamp expiry bounds a whole group by its End (GROUP_END), and any other
group, or a hole, by the lowest frame timestamp this hop knows in any higher
group. That bound holds whether or not the higher group is the immediate
successor. The latest group never ages. Arrival age, the wall time since a
group's successor arrived, still expires a superseded group on its own. Rust,
`@moq/net`, and the moq-lite draft's Expiration section change together.

## Plan

Decided 2026-11-20 (maintainer, /quest-plan auditing the subscription rules
before the lite-07 cut): skipping is by timestamp, from a whole group's end,
or from a higher group's minimum while the group isn't whole. Arrival age
stays beside it as the backstop for a publisher whose timestamps stall, or an
abusive one that never advances them, whose stale streams would otherwise
hold connection credit other broadcasts need.

- ✅ Both clocks (rejected: timestamps only).
- ✅ The latest group never ages by the wall clock (rejected: main's
`ended_at`, which ages a whole group with no successor from its arrival and
span; and closed [moq-dev/moq#9221](https://github.com/moq-dev/moq/pull/9221)'s
estimate of a mean frame interval).

[Group times](/quest/m2/time-addressing/group-times.md) gives a whole group its
End and recommends bounding any other group by its successor's start, as
`reach` (`rs/moq-net/src/model/track.rs`) and `#reach` (`js/net/src/track.ts`)
do today. This quest tightens the second half once producers keep the strong
rule.

- A higher group's lowest timestamp bounds a lower group only under the strong
no-overlap rule, so this waits on it. Until then moq-mux's overlapping groups
would expire early. With open-GOP leading pictures, that lowest frame sits
below the higher group's start.
- Use any known frame of any higher group, a tail copy's included. Under the
rule, every one of them is above the lower group's frames.
- Drop `ended_at`'s wall-clock aging of the latest group (JS `#endedAt`), if
group times has not already.
- Find every other place that bounds a group or a hole by a successor's start,
such as a reader giving up a hole by the max delay, and apply the same bound.

Public API: none expected. Wire: none. The draft's Expiration text changes,
which only lets a publisher expire sooner.

## Required

- [Every frame of a group presents before every frame of a higher group](/quest/m2/groups-never-overlap.md) - first-party producers keep the rule that makes a higher group's lowest timestamp an upper bound
- [Group Streams carry each time once](/quest/m2/time-addressing/group-times.md) - GROUP_END's End, the whole group's bound, and the draft's no-overlap MUST
Loading