Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion quest/m0/nightly/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,14 @@ This README's own work, after the children:
threshold to [bench CI](/quest/m1/bench-ci.md), and the moq-net bench
panics in "Rust bench targets" to
[moq-net's benches run to completion](/quest/m1/bench-test-mode.md).
The receive-windows benchmark's overcommitted cells, skipped since
[moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838), moved to
[m1](/quest/m1/receive-windows-evicted.md) on 2026-11-04 (maintainer).

Public API: none. Wire: none.

## Required

- [The receive-windows benchmark runs to the end](/quest/m0/nightly/receive-windows-evicted.md) - `benchmark_receive_windows` stops failing `frame: Evicted`, so the Rust benchmark-tests job publishes its report
- [moq-net's ingest fuzz target finds no stall](/quest/m0/nightly/ingest-fuzz.md) - the 13-byte stream that stalls ingest is fixed at its cause and kept as a regression test
- [Nightly's shipping-size check passes](/quest/m0/nightly/shipping-size.md) - js/net, js/watch, and moq-relay growth over 5% is traced to its commits, then shrunk or accepted
- [Nightly's Miri job stays inside its time budget](/quest/m0/nightly/miri.md) - Miri gets `rust-src`, then finishes inside its 60-minute job and 15-minute-per-test budgets
4 changes: 4 additions & 0 deletions quest/m0/test-flakes-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,14 @@ has failed it, so it needs no child of its own (noted 2026-11-03,
Added 2026-11-03 (maintainer, `/quest-plan`): five more merge-queue failures
from 2026-11-03 and early 2026-11-04 UTC, one child each.

Added 2026-11-04 (maintainer, `/quest-plan`): moq-gst's interleaved restart
test, which failed two merge-queue batches and two loaded local runs.

Public API: none. Wire: none.

## Required

- [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio
- [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - the two `harness.browser.ts` buffering tests and the Firefox player's audio-tone checks pass reliably in Interop and Nightly, each fixed at its cause under the epic's rules
- [moq-gst's interleaved restart test passes under load](/quest/m0/test-flakes-2/gst-reconnect-interleave.md) - `restarts_under_reconnects_interleave_on_one_pad` passes in loaded runs and the merge queue, its missed delivery and its stray `Eos` each fixed at the cause
- [reload.test.ts's close-drain tests pass under load](/quest/m0/test-flakes-2/reload-close-drain.md) - the drain cases in `js/net`'s reload tests pass in loaded multi-file runs
72 changes: 72 additions & 0 deletions quest/m0/test-flakes-2/gst-reconnect-interleave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# [S] moq-gst's interleaved restart test passes under load

## Goal

`restarts_under_reconnects_interleave_on_one_pad` in
`rs/moq-gst/src/source/imp.rs` passes reliably in a loaded `just check` and
in the merge queue, fixed at its cause under the
[epic](/quest/m0/test-flakes-2/README.md)'s rules, and runs again with its
`#[ignore]` removed. Done when the PR explains
each failure below and shows the loaded run that reproduced it now passing.

## Plan

Planned 2026-11-04 with the maintainer, from
[moq-dev/moq#7888](https://github.com/moq-dev/moq/pull/7888)'s suggested
follow-ups. No quest or open PR tracked it.

Ignored, decided 2026-11-04 (maintainer, on
[moq-dev/moq#7917](https://github.com/moq-dev/moq/pull/7917)): the test is
`#[ignore]`d with a reason naming this quest, so it stops failing
merge-queue batches. The PR that fixes it removes the
`#[ignore]`, re-enabling the test, and shows it passing in the loaded runs
below.

Failures seen so far, each passing when run alone:

- [moq-dev/moq#7888](https://github.com/moq-dev/moq/pull/7888) (2026-11-04,
full `just check`): `5 never reached a pad`, the panic in
`Recorder::wait` after its 10 s wall-clock `wait_timeout_while`. Tag 5 is
round 1's restart: the new publisher's first write never reached a pad.
- [moq-dev/moq#7761](https://github.com/moq-dev/moq/pull/7761) (full
`just check` with seven repositories' checks on one host): failed, then
passed 5 of 5 isolated runs. Its output wasn't kept.
- [moq-dev/moq#7606](https://github.com/moq-dev/moq/pull/7606) (merge queue,
2026-11-02, the land fixer's requeue comment): failed in 0.1 s, so not on
a timeout. `assert_one_pad_held` found an `Eos`. The recorded events show
every restart starting the pad again and replaying the previous round's
buffers (2 and 3, then 5, 6 and 7 twice), and a last segment at 2809000
before the `Eos`.
- [moq-dev/moq#7296](https://github.com/moq-dev/moq/pull/7296) (merge queue,
2026-11-01, the land fixer's handback comment): panicked unwrapping
`Err(unroutable)` at `imp.rs:3822:48` of that head's numbering.

So at least two causes may be at work: a delivery that misses its wait
under load, and an ordering race (an `Eos` or `Unroutable`) that shows up
in a fast run. Find each from the code under test before changing the test.

- The `Recorder` waits on a wall clock while the session runs on
`super::RUNTIME`. Prefer driving the session's timers with a mocked clock
or waiting on an event, as the epic's rules say. If the session itself is
slow under load, fix that.
- The `Eos` breaks the test's own claim that no round ends the pad. The
pad actor retires a pad whose rendition is still delisted at its turn
(`retired()` checks the pad table's `listed`), and the test doesn't wait
for the catalog relist to apply before it unannounces. First lead: the
retirement is intended and the test is missing that wait (a relist of an
existing rendition adds no pad, so the sibling test's `await_pad` doesn't
apply). If the retirement happens even after the relist applied, it's a
moqsrc bug, fixed in moqsrc
([moq-dev/moq#7368](https://github.com/moq-dev/moq/pull/7368) made the
actor own the pad).
- Find which call returned `Unroutable` in #7296's run, and whether
`publication.announce` or a follow can meet an unroutable path in the
window after `unannounce`.

Reproduce first: run moq-gst's tests together with the rest of the
workspace under CPU contention, repeating this test in a loop beside a
loaded `just check`. If it doesn't reproduce in a reasonable number
of runs, report that and the runs tried rather than changing the test
blind.

Public API: none. Wire: none.
3 changes: 3 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,10 @@ their end report, so an embedder's drain alert pages on real cuts.
- [A job's exact route continues its claim's readers](/quest/m1/announce-takeover.md) - a job's route inherits its claim's instance, replacing the ANNOUNCE_TAKEOVER predecessor design
- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport
- [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07
- [Group errors are cached by kind](/quest/m1/group-error-cache.md) - permanent group errors bind every reader, retryable ones are kept only for a bounded time, and a live SUBSCRIBE_DROP or FETCH answered `Old` no longer caches it for good
- [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - the wire names every abandoned group and the drop signal is public, so readers count losses instead of stalling
- [The overcommitted receive-window cells run again](/quest/m1/receive-windows-evicted.md) - `benchmark_receive_windows` measures its 256-stream and 4 MiB-frame cells again, reporting delivered frames and evictions instead of stalling
- [JS group demand counts only readers that will read](/quest/m1/js-group-demand.md) - a buffered copy whose reader already passed it no longer keeps a group wanted, so JS fills stop where Rust's do
- [Datagram tracks](/quest/m1/datagram-tracks/README.md) - a track is datagrams or streams; datagram tracks have their own request and sequence, no ranges, no cache, and map to object ID 0 on moq-transport
- [Stream credit by priority](/quest/m1/stream-credit/README.md) - every stream opens through one priority entry point, and a higher-priority group takes over or resets a lower one when credit runs out
- [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it
Expand Down
14 changes: 14 additions & 0 deletions quest/m1/datagram-tracks/ietf.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,20 @@ NOT_FOUND, and a stream track never sends one. Restore what it removed:
moq-net's `session_delivery_datagram`;
- the OBJECT_DATAGRAM sentence in `doc/concept/standard.md`.

An OBJECT_DATAGRAM whose Object Status is undefined is a protocol violation
in JS too, as it is in Rust, on every subscription kind. Planned 2026-11-04
with the maintainer, from
[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846). Rust's
`recv_datagram` (`rs/moq-net/src/ietf/subscriber.rs`, about line 4839)
closes the session with `ProtocolViolation` when `status_defined`
(`rs/moq-net/src/ietf/object.rs`, about line 21: Normal, END_OF_GROUP and
END_OF_TRACK) rejects the status, even while it drops the datagram. JS's
`#recvDatagram` (`js/net/src/ietf/subscriber.ts`) decodes and drops it
silently, on main and after #7846. Check the shared status set once for
both datagram paths, so a datagram-track subscription keeps the check
when it starts accepting datagrams, and add a JS test that an undefined
status closes the session.

Public API: none beyond the model. Wire: no format change.

## Required
Expand Down
2 changes: 1 addition & 1 deletion quest/m1/group-drops.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,4 @@ Wire: possibly lite-07 (SUBSCRIBE_DROP for a stopped group), decided here.
## Related

- [Drop set](/quest/m1/drop-set.md) - owns the per-cursor drop runs this reports
- [The receive-windows benchmark](/quest/m0/nightly/receive-windows-evicted.md) - its overcommitted cells wait on this (skipped by [moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) until it lands)
- [The receive-windows benchmark](/quest/m1/receive-windows-evicted.md) - its overcommitted cells wait on this (skipped by [moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) until it lands)
87 changes: 87 additions & 0 deletions quest/m1/group-error-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# [M] Group errors are cached by kind, retryable ones for a bounded time

## Goal

Every place moq-net records that a group won't be delivered, in Rust and
JS, follows one rule. A permanent error, where the group can never be
served, is recorded and binds every later reader. A retryable error is kept
only for a bounded time, so a later reader or a widened one asks again once
it lapses, while a burst of readers doesn't re-ask the source for each one.
Both paths that cache `Old` permanently on main follow the rule.

## Plan

Planned 2026-11-04 with the maintainer, while settling
[moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872). In the
maintainer's words: "we should cache errors for some amount of time. I
really hate classifying them though, but I think we kind of have to know
the difference between permanent and retryable errors." Milestone m1
(maintainer).

Where this starts: #7872 (decision 2, 2026-11-04) made a range's own start
skip, a second stale drop, and history recovered once non-permanent.
`Fill::refuse_groups` records nothing, while `Fill::drop_groups` records
the groups as lost for good (`rs/moq-net/src/model/track/fill.rs`). The
only permanent losses it kept are a SUBSCRIBE_DROP NotFound and groups past
the track's end. Two paths on main still cache `Old` permanently, listed as
#7872's follow-ups:

- A live SUBSCRIBE_DROP records every error on the track, `Old` included:
`self.serving.drop_groups(...)` in `rs/moq-net/src/lite/subscriber.rs`
(the `SubscribeResponse::Drop` arm), into `track::Producer::drop_groups`
(`rs/moq-net/src/model/track.rs`).
- `Fill::reject` for lite and IETF FETCH (`reject` in
`rs/moq-net/src/model/track/fill.rs`) records `Old` as dropped when
`old_binds_everyone`, that is when the publisher's max age applies to
every reader.

The work:

- **Inventory** every place a group error is recorded or refused, in Rust
(`TrackState::drop_groups`, `drop_unfed`, `Fill::drop_groups`,
`refuse_groups`, `reject_groups`, `reject`, the IETF subscriber's fetch
and range drops in `rs/moq-net/src/ietf/subscriber.rs`) and JS (`#drop`
in `js/net/src/track.ts`, `Fill.dropGroups` and `refuseGroups` in
`js/net/src/fill.ts`, and `js/net/src/ietf/range.ts`). For each, note
which errors reach it and whether it caches them. Put the inventory in the
PR.
- **The smallest classification that works.** Recommended starting point:
permanent for not found, past the track's end, and unauthorized;
retryable for old, lagged, evicted, cancelled, and unroutable. Keep it a
property of the error kind, decided in one place, rather than a per-call
choice. Prefer fewer classes if the inventory shows one is never needed.
Don't match on error variants at the call sites (AGENTS.md); classify
once where the error is defined.
- **Bounded caching for retryable errors.** Decide whether a retryable
error is cached at all, and for how long. Prefer a bound the track
already has over a new constant: for `Old`, the budget it was judged
under (a widening already re-asks), or the publisher's max age. The
bound must also hold up against a hostile peer: what the cache holds is
sized by the pool, as the [drop set](/quest/m1/drop-set.md) bills its
runs.
- **Fix both main paths** under the rule: a live SUBSCRIBE_DROP records
only a permanent error, and `Fill::reject` stops recording `Old` as lost
for good. Mirror both in JS.
- **To settle in this quest, with the maintainer:** the drop set keeps a
received stale run with the budget it was judged under, reversed only by a
widening ("Only a widening reverses a stale verdict"). That is a
retryable error held without a time bound. Either the bounded-time rule
covers those runs too, or the drop set's budget scoping already counts as
the bound for `Old`. Recommended: the latter, so `Old` has one bound (its
budget) and the time bound covers the other retryable kinds.
- Tests: a retryable refusal is asked again after its bound lapses and not
before; a permanent one never is; a live SUBSCRIBE_DROP with `Old` no
longer binds a wider reader; a FETCH answered `Old` under the publisher's
max age is asked again once the bound lapses.

Public API: possibly a classification query on `Error` (Rust) and its JS
mirror, if a caller outside moq-net needs it; otherwise none. Wire: none.

## Required

- [Drop set](/quest/m1/drop-set.md) - its per-cursor runs and their reasons are what this bounds; its PR, [moq-dev/moq#7670](https://github.com/moq-dev/moq/pull/7670), rewrites the same recording paths
- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - its PR, [moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872), makes the start skip and recovery non-permanent, the rule this generalizes

## Related

- [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - reports what a subscriber gives up; this decides how long the track remembers it
51 changes: 51 additions & 0 deletions quest/m1/js-group-demand.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# [M] JS group demand counts only readers that will read

## Goal

In `@moq/net`, a group is wanted only while some reader will still read it,
as in Rust. A copy another reader's sink holds but that reader has passed,
or will never reach, no longer keeps the group wanted. So a received copy's
fill FETCH, and the IETF range fetch, stop downloading a group as soon as no
reader will read it, as Rust's do.

## Plan

Planned 2026-11-04 with the maintainer, from the follow-ups of
[moq-dev/moq#7845](https://github.com/moq-dev/moq/pull/7845), whose fill
ingest stops once nobody holds the group (`untilAbandoned`). Milestone m1
(maintainer).

The gap: a cached group is mirrored into every subscribed sink
(`#mirror` in `js/net/src/track.ts`), and a mirror counts as holding the
group. So a reader reading in arrival order that already passed the group,
or one whose range or budget will skip it, still keeps it wanted, and the
fill keeps downloading. IETF's `RangeFetch.unwanted`
(`js/net/src/ietf/range.ts`) reads the same group demand. Rust judges a fill
by the runs its joins still read (`Fill::wanted` in
`rs/moq-net/src/model/track/fill.rs`: "a reader still reads it, and it was
neither answered nor delivered live since it was asked").

- Match Rust's rule, not a new one: first confirm what Rust counts (a
join's unread runs, and whether a queued, not-yet-received group counts),
and write it down in the PR.
- A buffered mirror the reader will still reach keeps counting. Only one
behind its cursor, or outside what it will read, stops counting.
- Fix it where demand is computed, so the lite fill ingest, IETF range
fetches and any other consumer of group demand follow the same rule.
Don't special-case the fill.
- Keep the order with the [drop set](/quest/m1/drop-set.md), which changes
how a cursor judges a missing group and asks `js/net` to mirror each case.
Whether a reader "will read" a group depends on that judgment, so build on
whichever has landed and leave the other a note.
- Tests: a fill FETCH stops once the only remaining mirror belongs to a
reader that passed the group, and keeps going while a mirror's reader
still has to reach it. An IETF range fetch stops once its last reader
passes the requested group.

Public API: none. Wire: none.

## Related

- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - the JS received-copy fills this stops sooner
- [Drop set](/quest/m1/drop-set.md) - each cursor's judgment of what it will still read
- [JS fastest-reader pacing](/quest/m2/js-fastest-reader.md) - also paces by which readers still read a group
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,15 @@ alongside throughput and latency, instead of failing.

## Plan

Decided 2026-11-03 (maintainer): m0, as a child of
[Nightly passes on main](/quest/m0/nightly/README.md).
Moved to m1 2026-11-04 with the maintainer, after
[moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) landed the
fitting cells: what remains waits on
[group-drops](/quest/m1/group-drops.md), an m1 quest, and with the
overcommitted cells skipped the benchmark no longer fails Nightly, so this
no longer holds the m0
[Nightly passes on main](/quest/m0/nightly/README.md) epic open. It replaces
the 2026-11-03 decision (maintainer) that placed it in m0 as that epic's
child.

Overload policy, decided 2026-11-04 (maintainer):

Expand Down Expand Up @@ -90,10 +97,6 @@ connection-wide admission semaphore are recorded in

Public API: none. Wire: none.

## Waiting
## Required

- Outside condition: `quest/m1/group-drops.md`, planned in
[moq-dev/moq#7890](https://github.com/moq-dev/moq/pull/7890), finishes,
making `track::Subscriber`'s drop signal public on `main`. Check that
[moq-dev/moq#7890](https://github.com/moq-dev/moq/pull/7890) has
merged and that the quest file has since been deleted from `main`.
- [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - the public drop signal the reader needs to return a dropped group's permit
1 change: 1 addition & 0 deletions quest/m1/rind/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,5 @@ The epic's own work, once every child has merged:
## Required

- [Transport on rind](/quest/m1/rind/transport.md) - moq-quic, moq-tokio, moq-uring, moq-rtmp, and qmux take rind, and the `aws-lc-rs`/`ring` features go
- [One typed unsupported-CPU error](/quest/m1/rind/cpu-error.md) - moq-auth, moq-hls and moq-crypto e2ee carry `moq_crypto::provider::UnsupportedCpu` and drop their own CPU-check helpers
- [moq-rtc on rind](/quest/m1/rind/rtc.md) - str0m and dimpl run DTLS, SRTP, and STUN on rind
Loading
Loading