Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions quest/m0/broadcast-epoch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@

A path and the epoch on its route are the only content identity, and no
first-party publisher reuses a pair for different content. Only routes with
the same epoch resume a subscription from the first frame it lacks; a route
without one keeps its subscriptions until it goes. So epochs are what make
the same epoch resume a subscription from the first frame it lacks, within
the subscriber's max age; a route without one keeps its subscriptions until
it goes. So epochs are what make
failover seamless. Routes without an epoch never splice (maintainer,
2026-10-08): a `Restart` (or an end and start on older versions) tells every
downstream subscriber to stop using its copy of the old source and
Expand Down
1 change: 1 addition & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ lands before the cut.
- [SUBSCRIBE_OK names where delivery starts](/quest/m1/subscribe-ok-start.md) - a warm relay names the same start group as a cold one instead of groups it skipped as stale
- [Resumed close](/quest/m1/resume-closed.md) - a resumed JS track keeps its declared end across copies, and its `closed` and `largest()` settle on a clean end
- [Resumed in-flight groups](/quest/m1/resume-in-flight.md) - a replaced route copy stays while a group below its bound is in flight or it backs a handed-out group late, and no longer than the reader's budget
- [Resume through an outage](/quest/m1/resume-outage.md) - a same-epoch resume after an unclosed route dies delivers the backlog its max age still holds, pinned by a regression test
- [Resumed datagrams](/quest/m1/resume-datagrams.md) - a resumed track skips only datagrams it already delivered, not every lower sequence
- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - a received timed track's timestamps are media time, so the rewind check can run across hops
- [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS
Expand Down
68 changes: 68 additions & 0 deletions quest/m1/resume-outage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# [S] Resume through an undetected outage

## Goal

A same-epoch resume after a route dies without a close (a killed relay,
detected only at the QUIC idle timeout) delivers every group the outage
backlog still holds within the subscriber's max age, from the first frame the
reader lacks. A regression test pins this, and the docs stop promising more
than max age allows.

## Plan

Reported in [#6062](https://github.com/moq-dev/moq/issues/6062): a diamond
O -> M1/M2 -> E on lite-07 with `--quic-idle-timeout 6s`, M1 SIGKILLed, and
`moq export ts` on E. Each exporter logged `media skipped upstream` for
2-7 s on every track, at both the default `--delay` and `--delay 10s`.

Triage against `5284d969bf` (2026-10-20, code reading only):

- Resume already asks for the right floor. `Reader::sync` subscribes the new
copy at `floor = self.newest` (`rs/moq-net/src/model/resume.rs`), and
`Subscription::clamped` raises the wire start to it. A cold M2 forwards that
start to O.
- Every hop then applies the subscriber's max age, which is `--delay / 2` for
`export ts`. Under it, groups that went stale during detection are expired,
as the draft requires ("A floor is not a request", `Group Start` in
`drafts/draft-lcurley-moq-lite.md`). At the default 1 s delay, losing most
of the 6.6 s window is intended.
- At `--delay 10s` (a 5 s budget) the code predicts about 2 s lost, not the
same loss as the default. Nothing read explains the report there, so it may
be a real bug. Candidates: the wall-clock `superseded` rule in
`model/track.rs`; `sync` finding no copies and dropping the floor
(`resume.rs`, `subscription.rs` `clamped`); or the container consumer's
hole and latency skips (`rs/moq-mux/src/container/consumer.rs`).
- [#6019](https://github.com/moq-dev/moq/pull/6019) and
[#6024](https://github.com/moq-dev/moq/pull/6024) do not touch this path.

Decided with the maintainer, 2026-10-20:

- Keep max-age semantics. Recovering the whole gap regardless of max age
(widening max age on resume, or fetching the gap) was declined.
- Test at the model level only, in
`rs/moq-net/tests/integration/route_change.rs` under mock time, next to
`lite_resumes_after_a_lagging_route_dies`. The alternate route starts cold.
Comment thread
kixelated marked this conversation as resolved.
Resolved
The test writes timestamped groups while A's route is silent but not yet
closed, then aborts it. With a budget above the outage, every group
arrives. With a budget below it, only the in-budget tail arrives. No
end-to-end relay kill test.
- The `--delay 10s` loss must be explained before this closes #6062. If the
moq-net test passes, add a mock-time unit test in
`rs/moq-mux/src/container/consumer.rs` that feeds the consumer a resumed
backlog burst within its budget, and assert it reads every group without a
`Read::Gap`.
- Fix any bug the test exposes at its source in moq-net, not in the exporter.
- Correct `doc/concept/moq-lite.md` (*Same epoch resumes*): the resume point
is a floor, and the subscriber's max age still bounds the backlog.
- Faster detection of the dead route belongs to
[route health](/quest/m2/route-health/README.md), not here.

## Closes

- [#6062](https://github.com/moq-dev/moq/issues/6062) - same-epoch resume after a relay loss continues at a later group

## Related

- [Resumed in-flight groups](/quest/m1/resume-in-flight.md) - which replaced copies stay while groups below them are in flight
- [A widened max age re-offers cached groups](/quest/m1/resolved-demand.md) - the same max-age bound on a warm relay
- [Redundant failover drift](/quest/m1/test-flakes-2/redundant-failover-drift.md) - the same idle-timeout outage hitting the TS jitter clock
5 changes: 5 additions & 0 deletions quest/m2/route-health/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ Decided with the maintainer, 2026-10-13:
- Ranked in m2, after clustering stability, federation, and P2P. Health
design must first reproduce a stall that transport failure and ordinary
reconnect do not recover, or the line is abandoned.
- A route that dies without a close counts as such a stall until the QUIC
idle timeout fires (decided 2026-10-20, from
[#6062](https://github.com/moq-dev/moq/issues/6062)): a killed relay
starves a same-epoch subscription for the whole timeout, and a subscriber
max age below it sheds that backlog even though resume asks for it.

- Application-assigned connection priority remains ahead of additive cost.
Deterministic hashing still chooses distinct equal routes; a new connection
Expand Down
5 changes: 4 additions & 1 deletion quest/m2/route-health/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@ do not add speculative production code or choose arbitrary timing thresholds.

The [line's decisions](/quest/m2/route-health/README.md) are settled. Start by
reproducing a stall that transport failure and ordinary reconnect miss; if
none exists, bring abandoning the line back to the maintainer. Preserve
none exists, bring abandoning the line back to the maintainer. A killed relay
in a same-epoch diamond qualifies until the idle timeout fires
([#6062](https://github.com/moq-dev/moq/issues/6062)); measure the media it
loses and how far a standby shortens that. Preserve
application priorities, automatic health-driven switching and failback, a
payload-free steady standby, and bounded payload overlap to verify delivery.
The mechanism is entirely excluded for epochless broadcasts.
Expand Down
2 changes: 2 additions & 0 deletions quest/m2/route-health/recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
Endpoints and relays automatically move a stalled track to a verified working
same-epoch alternate, then return to the preferred route after demonstrated
recovery, while preserving logical readers and bounded control/media overhead.
This includes a route that dies without a close: the track moves before the
QUIC idle timeout declares it dead.
Epochless broadcasts never enter this active/passive mechanism.

## Plan
Expand Down