diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index bddd0482811..05f149e50d4 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -33,12 +33,14 @@ This README's own work, after the children: threshold to [bench CI](/quest/m1/bench-ci.md), and the moq-net bench panics in "Rust bench targets" to [moq-net's benches run to completion](/quest/m1/bench-test-mode.md). + The receive-windows benchmark's overcommitted cells, skipped since + [moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838), moved to + [m1](/quest/m1/receive-windows-evicted.md) on 2026-11-04 (maintainer). Public API: none. Wire: none. ## Required -- [The receive-windows benchmark runs to the end](/quest/m0/nightly/receive-windows-evicted.md) - `benchmark_receive_windows` stops failing `frame: Evicted`, so the Rust benchmark-tests job publishes its report - [moq-net's ingest fuzz target finds no stall](/quest/m0/nightly/ingest-fuzz.md) - the 13-byte stream that stalls ingest is fixed at its cause and kept as a regression test - [Nightly's shipping-size check passes](/quest/m0/nightly/shipping-size.md) - js/net, js/watch, and moq-relay growth over 5% is traced to its commits, then shrunk or accepted - [Nightly's Miri job stays inside its time budget](/quest/m0/nightly/miri.md) - Miri gets `rust-src`, then finishes inside its 60-minute job and 15-minute-per-test budgets diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index bc2c05c9f72..31833ee99da 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -88,10 +88,14 @@ has failed it, so it needs no child of its own (noted 2026-11-03, Added 2026-11-03 (maintainer, `/quest-plan`): five more merge-queue failures from 2026-11-03 and early 2026-11-04 UTC, one child each. +Added 2026-11-04 (maintainer, `/quest-plan`): moq-gst's interleaved restart +test, which failed two merge-queue batches and two loaded local runs. + Public API: none. Wire: none. ## Required - [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio - [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - the two `harness.browser.ts` buffering tests and the Firefox player's audio-tone checks pass reliably in Interop and Nightly, each fixed at its cause under the epic's rules +- [moq-gst's interleaved restart test passes under load](/quest/m0/test-flakes-2/gst-reconnect-interleave.md) - `restarts_under_reconnects_interleave_on_one_pad` passes in loaded runs and the merge queue, its missed delivery and its stray `Eos` each fixed at the cause - [reload.test.ts's close-drain tests pass under load](/quest/m0/test-flakes-2/reload-close-drain.md) - the drain cases in `js/net`'s reload tests pass in loaded multi-file runs diff --git a/quest/m0/test-flakes-2/gst-reconnect-interleave.md b/quest/m0/test-flakes-2/gst-reconnect-interleave.md new file mode 100644 index 00000000000..ee5b8b42e36 --- /dev/null +++ b/quest/m0/test-flakes-2/gst-reconnect-interleave.md @@ -0,0 +1,72 @@ +# [S] moq-gst's interleaved restart test passes under load + +## Goal + +`restarts_under_reconnects_interleave_on_one_pad` in +`rs/moq-gst/src/source/imp.rs` passes reliably in a loaded `just check` and +in the merge queue, fixed at its cause under the +[epic](/quest/m0/test-flakes-2/README.md)'s rules, and runs again with its +`#[ignore]` removed. Done when the PR explains +each failure below and shows the loaded run that reproduced it now passing. + +## Plan + +Planned 2026-11-04 with the maintainer, from +[moq-dev/moq#7888](https://github.com/moq-dev/moq/pull/7888)'s suggested +follow-ups. No quest or open PR tracked it. + +Ignored, decided 2026-11-04 (maintainer, on +[moq-dev/moq#7917](https://github.com/moq-dev/moq/pull/7917)): the test is +`#[ignore]`d with a reason naming this quest, so it stops failing +merge-queue batches. The PR that fixes it removes the +`#[ignore]`, re-enabling the test, and shows it passing in the loaded runs +below. + +Failures seen so far, each passing when run alone: + +- [moq-dev/moq#7888](https://github.com/moq-dev/moq/pull/7888) (2026-11-04, + full `just check`): `5 never reached a pad`, the panic in + `Recorder::wait` after its 10 s wall-clock `wait_timeout_while`. Tag 5 is + round 1's restart: the new publisher's first write never reached a pad. +- [moq-dev/moq#7761](https://github.com/moq-dev/moq/pull/7761) (full + `just check` with seven repositories' checks on one host): failed, then + passed 5 of 5 isolated runs. Its output wasn't kept. +- [moq-dev/moq#7606](https://github.com/moq-dev/moq/pull/7606) (merge queue, + 2026-11-02, the land fixer's requeue comment): failed in 0.1 s, so not on + a timeout. `assert_one_pad_held` found an `Eos`. The recorded events show + every restart starting the pad again and replaying the previous round's + buffers (2 and 3, then 5, 6 and 7 twice), and a last segment at 2809000 + before the `Eos`. +- [moq-dev/moq#7296](https://github.com/moq-dev/moq/pull/7296) (merge queue, + 2026-11-01, the land fixer's handback comment): panicked unwrapping + `Err(unroutable)` at `imp.rs:3822:48` of that head's numbering. + +So at least two causes may be at work: a delivery that misses its wait +under load, and an ordering race (an `Eos` or `Unroutable`) that shows up +in a fast run. Find each from the code under test before changing the test. + +- The `Recorder` waits on a wall clock while the session runs on + `super::RUNTIME`. Prefer driving the session's timers with a mocked clock + or waiting on an event, as the epic's rules say. If the session itself is + slow under load, fix that. +- The `Eos` breaks the test's own claim that no round ends the pad. The + pad actor retires a pad whose rendition is still delisted at its turn + (`retired()` checks the pad table's `listed`), and the test doesn't wait + for the catalog relist to apply before it unannounces. First lead: the + retirement is intended and the test is missing that wait (a relist of an + existing rendition adds no pad, so the sibling test's `await_pad` doesn't + apply). If the retirement happens even after the relist applied, it's a + moqsrc bug, fixed in moqsrc + ([moq-dev/moq#7368](https://github.com/moq-dev/moq/pull/7368) made the + actor own the pad). +- Find which call returned `Unroutable` in #7296's run, and whether + `publication.announce` or a follow can meet an unroutable path in the + window after `unannounce`. + +Reproduce first: run moq-gst's tests together with the rest of the +workspace under CPU contention, repeating this test in a loop beside a +loaded `just check`. If it doesn't reproduce in a reasonable number +of runs, report that and the runs tried rather than changing the test +blind. + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index c5ceed1362e..e3bbdf1e474 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -149,7 +149,10 @@ their end report, so an embedder's drain alert pages on real cuts. - [A job's exact route continues its claim's readers](/quest/m1/announce-takeover.md) - a job's route inherits its claim's instance, replacing the ANNOUNCE_TAKEOVER predecessor design - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 +- [Group errors are cached by kind](/quest/m1/group-error-cache.md) - permanent group errors bind every reader, retryable ones are kept only for a bounded time, and a live SUBSCRIBE_DROP or FETCH answered `Old` no longer caches it for good - [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - the wire names every abandoned group and the drop signal is public, so readers count losses instead of stalling +- [The overcommitted receive-window cells run again](/quest/m1/receive-windows-evicted.md) - `benchmark_receive_windows` measures its 256-stream and 4 MiB-frame cells again, reporting delivered frames and evictions instead of stalling +- [JS group demand counts only readers that will read](/quest/m1/js-group-demand.md) - a buffered copy whose reader already passed it no longer keeps a group wanted, so JS fills stop where Rust's do - [Datagram tracks](/quest/m1/datagram-tracks/README.md) - a track is datagrams or streams; datagram tracks have their own request and sequence, no ranges, no cache, and map to object ID 0 on moq-transport - [Stream credit by priority](/quest/m1/stream-credit/README.md) - every stream opens through one priority entry point, and a higher-priority group takes over or resets a lower one when credit runs out - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it diff --git a/quest/m1/datagram-tracks/ietf.md b/quest/m1/datagram-tracks/ietf.md index c0a7c874677..37b3a70b460 100644 --- a/quest/m1/datagram-tracks/ietf.md +++ b/quest/m1/datagram-tracks/ietf.md @@ -36,6 +36,20 @@ NOT_FOUND, and a stream track never sends one. Restore what it removed: moq-net's `session_delivery_datagram`; - the OBJECT_DATAGRAM sentence in `doc/concept/standard.md`. +An OBJECT_DATAGRAM whose Object Status is undefined is a protocol violation +in JS too, as it is in Rust, on every subscription kind. Planned 2026-11-04 +with the maintainer, from +[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846). Rust's +`recv_datagram` (`rs/moq-net/src/ietf/subscriber.rs`, about line 4839) +closes the session with `ProtocolViolation` when `status_defined` +(`rs/moq-net/src/ietf/object.rs`, about line 21: Normal, END_OF_GROUP and +END_OF_TRACK) rejects the status, even while it drops the datagram. JS's +`#recvDatagram` (`js/net/src/ietf/subscriber.ts`) decodes and drops it +silently, on main and after #7846. Check the shared status set once for +both datagram paths, so a datagram-track subscription keeps the check +when it starts accepting datagrams, and add a JS test that an undefined +status closes the session. + Public API: none beyond the model. Wire: no format change. ## Required diff --git a/quest/m1/group-drops.md b/quest/m1/group-drops.md index 786670d8a4b..3ae504de173 100644 --- a/quest/m1/group-drops.md +++ b/quest/m1/group-drops.md @@ -72,4 +72,4 @@ Wire: possibly lite-07 (SUBSCRIBE_DROP for a stopped group), decided here. ## Related - [Drop set](/quest/m1/drop-set.md) - owns the per-cursor drop runs this reports -- [The receive-windows benchmark](/quest/m0/nightly/receive-windows-evicted.md) - its overcommitted cells wait on this (skipped by [moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) until it lands) +- [The receive-windows benchmark](/quest/m1/receive-windows-evicted.md) - its overcommitted cells wait on this (skipped by [moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) until it lands) diff --git a/quest/m1/group-error-cache.md b/quest/m1/group-error-cache.md new file mode 100644 index 00000000000..b51a73d9077 --- /dev/null +++ b/quest/m1/group-error-cache.md @@ -0,0 +1,87 @@ +# [M] Group errors are cached by kind, retryable ones for a bounded time + +## Goal + +Every place moq-net records that a group won't be delivered, in Rust and +JS, follows one rule. A permanent error, where the group can never be +served, is recorded and binds every later reader. A retryable error is kept +only for a bounded time, so a later reader or a widened one asks again once +it lapses, while a burst of readers doesn't re-ask the source for each one. +Both paths that cache `Old` permanently on main follow the rule. + +## Plan + +Planned 2026-11-04 with the maintainer, while settling +[moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872). In the +maintainer's words: "we should cache errors for some amount of time. I +really hate classifying them though, but I think we kind of have to know +the difference between permanent and retryable errors." Milestone m1 +(maintainer). + +Where this starts: #7872 (decision 2, 2026-11-04) made a range's own start +skip, a second stale drop, and history recovered once non-permanent. +`Fill::refuse_groups` records nothing, while `Fill::drop_groups` records +the groups as lost for good (`rs/moq-net/src/model/track/fill.rs`). The +only permanent losses it kept are a SUBSCRIBE_DROP NotFound and groups past +the track's end. Two paths on main still cache `Old` permanently, listed as +#7872's follow-ups: + +- A live SUBSCRIBE_DROP records every error on the track, `Old` included: + `self.serving.drop_groups(...)` in `rs/moq-net/src/lite/subscriber.rs` + (the `SubscribeResponse::Drop` arm), into `track::Producer::drop_groups` + (`rs/moq-net/src/model/track.rs`). +- `Fill::reject` for lite and IETF FETCH (`reject` in + `rs/moq-net/src/model/track/fill.rs`) records `Old` as dropped when + `old_binds_everyone`, that is when the publisher's max age applies to + every reader. + +The work: + +- **Inventory** every place a group error is recorded or refused, in Rust + (`TrackState::drop_groups`, `drop_unfed`, `Fill::drop_groups`, + `refuse_groups`, `reject_groups`, `reject`, the IETF subscriber's fetch + and range drops in `rs/moq-net/src/ietf/subscriber.rs`) and JS (`#drop` + in `js/net/src/track.ts`, `Fill.dropGroups` and `refuseGroups` in + `js/net/src/fill.ts`, and `js/net/src/ietf/range.ts`). For each, note + which errors reach it and whether it caches them. Put the inventory in the + PR. +- **The smallest classification that works.** Recommended starting point: + permanent for not found, past the track's end, and unauthorized; + retryable for old, lagged, evicted, cancelled, and unroutable. Keep it a + property of the error kind, decided in one place, rather than a per-call + choice. Prefer fewer classes if the inventory shows one is never needed. + Don't match on error variants at the call sites (AGENTS.md); classify + once where the error is defined. +- **Bounded caching for retryable errors.** Decide whether a retryable + error is cached at all, and for how long. Prefer a bound the track + already has over a new constant: for `Old`, the budget it was judged + under (a widening already re-asks), or the publisher's max age. The + bound must also hold up against a hostile peer: what the cache holds is + sized by the pool, as the [drop set](/quest/m1/drop-set.md) bills its + runs. +- **Fix both main paths** under the rule: a live SUBSCRIBE_DROP records + only a permanent error, and `Fill::reject` stops recording `Old` as lost + for good. Mirror both in JS. +- **To settle in this quest, with the maintainer:** the drop set keeps a + received stale run with the budget it was judged under, reversed only by a + widening ("Only a widening reverses a stale verdict"). That is a + retryable error held without a time bound. Either the bounded-time rule + covers those runs too, or the drop set's budget scoping already counts as + the bound for `Old`. Recommended: the latter, so `Old` has one bound (its + budget) and the time bound covers the other retryable kinds. +- Tests: a retryable refusal is asked again after its bound lapses and not + before; a permanent one never is; a live SUBSCRIBE_DROP with `Old` no + longer binds a wider reader; a FETCH answered `Old` under the publisher's + max age is asked again once the bound lapses. + +Public API: possibly a classification query on `Error` (Rust) and its JS +mirror, if a caller outside moq-net needs it; otherwise none. Wire: none. + +## Required + +- [Drop set](/quest/m1/drop-set.md) - its per-cursor runs and their reasons are what this bounds; its PR, [moq-dev/moq#7670](https://github.com/moq-dev/moq/pull/7670), rewrites the same recording paths +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - its PR, [moq-dev/moq#7872](https://github.com/moq-dev/moq/pull/7872), makes the start skip and recovery non-permanent, the rule this generalizes + +## Related + +- [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - reports what a subscriber gives up; this decides how long the track remembers it diff --git a/quest/m1/js-group-demand.md b/quest/m1/js-group-demand.md new file mode 100644 index 00000000000..36393bd4a64 --- /dev/null +++ b/quest/m1/js-group-demand.md @@ -0,0 +1,51 @@ +# [M] JS group demand counts only readers that will read + +## Goal + +In `@moq/net`, a group is wanted only while some reader will still read it, +as in Rust. A copy another reader's sink holds but that reader has passed, +or will never reach, no longer keeps the group wanted. So a received copy's +fill FETCH, and the IETF range fetch, stop downloading a group as soon as no +reader will read it, as Rust's do. + +## Plan + +Planned 2026-11-04 with the maintainer, from the follow-ups of +[moq-dev/moq#7845](https://github.com/moq-dev/moq/pull/7845), whose fill +ingest stops once nobody holds the group (`untilAbandoned`). Milestone m1 +(maintainer). + +The gap: a cached group is mirrored into every subscribed sink +(`#mirror` in `js/net/src/track.ts`), and a mirror counts as holding the +group. So a reader reading in arrival order that already passed the group, +or one whose range or budget will skip it, still keeps it wanted, and the +fill keeps downloading. IETF's `RangeFetch.unwanted` +(`js/net/src/ietf/range.ts`) reads the same group demand. Rust judges a fill +by the runs its joins still read (`Fill::wanted` in +`rs/moq-net/src/model/track/fill.rs`: "a reader still reads it, and it was +neither answered nor delivered live since it was asked"). + +- Match Rust's rule, not a new one: first confirm what Rust counts (a + join's unread runs, and whether a queued, not-yet-received group counts), + and write it down in the PR. +- A buffered mirror the reader will still reach keeps counting. Only one + behind its cursor, or outside what it will read, stops counting. +- Fix it where demand is computed, so the lite fill ingest, IETF range + fetches and any other consumer of group demand follow the same rule. + Don't special-case the fill. +- Keep the order with the [drop set](/quest/m1/drop-set.md), which changes + how a cursor judges a missing group and asks `js/net` to mirror each case. + Whether a reader "will read" a group depends on that judgment, so build on + whichever has landed and leave the other a note. +- Tests: a fill FETCH stops once the only remaining mirror belongs to a + reader that passed the group, and keeps going while a mirror's reader + still has to reach it. An IETF range fetch stops once its last reader + passes the requested group. + +Public API: none. Wire: none. + +## Related + +- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - the JS received-copy fills this stops sooner +- [Drop set](/quest/m1/drop-set.md) - each cursor's judgment of what it will still read +- [JS fastest-reader pacing](/quest/m2/js-fastest-reader.md) - also paces by which readers still read a group diff --git a/quest/m0/nightly/receive-windows-evicted.md b/quest/m1/receive-windows-evicted.md similarity index 86% rename from quest/m0/nightly/receive-windows-evicted.md rename to quest/m1/receive-windows-evicted.md index 69cc9ef5463..95702b3a105 100644 --- a/quest/m0/nightly/receive-windows-evicted.md +++ b/quest/m1/receive-windows-evicted.md @@ -17,8 +17,15 @@ alongside throughput and latency, instead of failing. ## Plan -Decided 2026-11-03 (maintainer): m0, as a child of -[Nightly passes on main](/quest/m0/nightly/README.md). +Moved to m1 2026-11-04 with the maintainer, after +[moq-dev/moq#7838](https://github.com/moq-dev/moq/pull/7838) landed the +fitting cells: what remains waits on +[group-drops](/quest/m1/group-drops.md), an m1 quest, and with the +overcommitted cells skipped the benchmark no longer fails Nightly, so this +no longer holds the m0 +[Nightly passes on main](/quest/m0/nightly/README.md) epic open. It replaces +the 2026-11-03 decision (maintainer) that placed it in m0 as that epic's +child. Overload policy, decided 2026-11-04 (maintainer): @@ -90,10 +97,6 @@ connection-wide admission semaphore are recorded in Public API: none. Wire: none. -## Waiting +## Required -- Outside condition: `quest/m1/group-drops.md`, planned in - [moq-dev/moq#7890](https://github.com/moq-dev/moq/pull/7890), finishes, - making `track::Subscriber`'s drop signal public on `main`. Check that - [moq-dev/moq#7890](https://github.com/moq-dev/moq/pull/7890) has - merged and that the quest file has since been deleted from `main`. +- [A group a subscriber gives up on is always reported](/quest/m1/group-drops.md) - the public drop signal the reader needs to return a dropped group's permit diff --git a/quest/m1/rind/README.md b/quest/m1/rind/README.md index cd55ca0c004..7d2167751d2 100644 --- a/quest/m1/rind/README.md +++ b/quest/m1/rind/README.md @@ -66,4 +66,5 @@ The epic's own work, once every child has merged: ## Required - [Transport on rind](/quest/m1/rind/transport.md) - moq-quic, moq-tokio, moq-uring, moq-rtmp, and qmux take rind, and the `aws-lc-rs`/`ring` features go +- [One typed unsupported-CPU error](/quest/m1/rind/cpu-error.md) - moq-auth, moq-hls and moq-crypto e2ee carry `moq_crypto::provider::UnsupportedCpu` and drop their own CPU-check helpers - [moq-rtc on rind](/quest/m1/rind/rtc.md) - str0m and dimpl run DTLS, SRTP, and STUN on rind diff --git a/quest/m1/rind/cpu-error.md b/quest/m1/rind/cpu-error.md new file mode 100644 index 00000000000..eb346d78695 --- /dev/null +++ b/quest/m1/rind/cpu-error.md @@ -0,0 +1,53 @@ +# [S] One typed unsupported-CPU error + +## Goal + +moq-auth's and moq-hls's `Error::UnsupportedCpu` and moq-crypto e2ee's +`Error::Unsupported` carry the shared +`moq_crypto::provider::UnsupportedCpu` instead of a `String`, converted with +`?`, and each crate's own `rind::cpu::check()` wrapper is gone +(`unsupported_cpu` in `rs/moq-auth/src/error.rs` and +`rs/moq-hls/src/error.rs`, and the `map_err` in +`rs/moq-crypto/src/e2ee/credential.rs`). A caller tells a CPU refusal from +any other failure by its type, the same way in every crate. + +## Plan + +Planned 2026-11-04 with the maintainer, from the suggested follow-up of +[moq-dev/moq#7873](https://github.com/moq-dev/moq/pull/7873) +([Transport on rind](/quest/m1/rind/transport.md)). That PR adds the +`provider` feature to moq-crypto, holding `provider::UnsupportedCpu` +(`Clone + Copy + Eq`, wrapping rind's `MissingFeature` privately), and moves +moq-quic, moq-tokio, qmux and moq-uring onto it as +`UnsupportedCpu(#[from] ...)` with `#[error(transparent)]`. This quest does +the same for the three crates it left, so it starts once that PR lands. + +- ✅ #7873 lands as is, with the type in `provider`; this quest moves it + later (maintainer, 2026-11-04, on + [moq-dev/moq#7917](https://github.com/moq-dev/moq/pull/7917)). +- **The type must not need rustls.** In #7873, `provider` pulls in rustls, + since it also holds `provider()` and `install_default()`. moq-auth's core + build has no rustls (only its `client` feature does), and the e2ee feature + doesn't either. Recommended: put `UnsupportedCpu` and `check()` where they + need only rind, such as a moq-crypto feature that `provider` and `e2ee` + both enable, and keep `provider` for the rustls half. The type's path + may then move out of `provider`, which nothing released uses yet. Check with + `cargo tree` that moq-auth's and moq-crypto's default and + `--no-default-features` graphs gain no rustls. +- moq-hls doesn't depend on moq-crypto today and gains it for this type. +- moq-rtmp kept `rind::cpu::check()` behind `io::Error` in #7873 only + because `provider` needs rustls. If the split above lands, move it onto + the typed error too, or record why not. +- Keep each variant's name: `UnsupportedCpu` in moq-auth and moq-hls. Rename + e2ee's `Unsupported` to `UnsupportedCpu` too, since it only ever means a + CPU refusal, so all three read alike. +- Docs naming these variants (`rs/moq-hls/src/lib.rs`, the moq-hls export + and server rustdoc) follow the change. + +Public API: breaking for moq-auth, moq-hls and moq-crypto (the variants' +payload type, and e2ee's variant name). Each gets a `## Unreleased` note in +its `MIGRATE.md`. Wire: none. + +## Required + +- [Transport on rind](/quest/m1/rind/transport.md) - adds `moq_crypto::provider::UnsupportedCpu`, which this wraps diff --git a/quest/m2/README.md b/quest/m2/README.md index fe50c4f2054..a8aff7c8165 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -86,8 +86,10 @@ lite-07; bitrate enforcement is not part of this release. - [Paired bench BASE](/quest/m2/bench-base-paired.md) - `just bench ` uses the nightly's paired comparison and the old one is deleted - [Receive pool](/quest/m2/receive-pool.md) - one byte budget per worker covers every connection's receive credit and the cache, billing each frame once - [Front parking](/quest/m2/origin-front-parks.md) - an unroutable request waits on a front instead of re-asking on every route-table move +- [A refused follower asks a replacement handler](/quest/m2/follow-refused-retry.md) - a same-instance handler swap with identical route details re-asks a refused `Following`, while listeners still see no change - [Route wakes](/quest/m2/route-wakes.md) - a route change wakes only the fronts it can move, so pool churn stops scaling with served paths - [A route stream's interest is a set of prefixes](/quest/m2/route-interest-prefixes.md) - one Announce Stream covers several subtrees, with floors and certification proved over the set +- [A socket dial learns its peer is behind an older hop](/quest/m2/socket-old-hop-signal.md) - a relay finds the dialed endpoint behind its older moq-lite session and marks that path, so the dial fails `Unsupported` while one to an absent peer still waits - [Fewer allocations in announce fan-out](/quest/m2/announce-fanout-allocs.md) - a deep path costs no more allocations per subscriber than a shallow one - [moq-room scales to large rooms](/quest/m2/room-bench.md) - a benchmark measures `Room` with large participant counts, and polling narrows only if it shows a slope - [Route health](/quest/m2/route-health/README.md) - epoch-only active/passive track recovery over metadata-only subscriptions diff --git a/quest/m2/follow-refused-retry.md b/quest/m2/follow-refused-retry.md new file mode 100644 index 00000000000..c867c37b705 --- /dev/null +++ b/quest/m2/follow-refused-retry.md @@ -0,0 +1,65 @@ +# [S] A refused follower asks a replacement handler + +## Goal + +A `Following` whose last resolution was refused asks again once the +handler that refused it is replaced, even when the replacement registers +the same instance with identical route details while the old one is still +up, so the path stays covered. Announcement listeners still see no change +for such an overlapping swap. A swap that drops the old route first already +ends and restarts the path, which a refused follower re-asks today. + +## Plan + +Planned 2026-11-04 with the maintainer, from a review thread declined on +[moq-dev/moq#7836](https://github.com/moq-dev/moq/pull/7836) +([`follow.rs`](https://github.com/moq-dev/moq/pull/7836#discussion_r4471135777)). +Milestone m2 (maintainer). + +The gap: since #7836, a refused `Following` (its `refused` flag in +`rs/moq-net/src/model/follow.rs`) waits for an announcement change before +asking again: another instance, the path going and coming back, or the +same instance over another route. `TableCursor::update` +(`rs/moq-net/src/model/origin.rs`, the `previous.meta == meta && +previous.served == served` arm) deliberately delivers nothing when the +winning route keeps its instance, prefix, servability and metadata. So a +handler dropped and re-registered with an identical route stays refused +until something else changes. Making that swap an announcement event was +declined: it would change announcement semantics for every listener, not +just refused followers. + +Recommended: after a refusal, the follower also waits on the route table's +path-scoped `Watch` (crate-private, in `origin.rs`), the generation +`origin::Consumer::request` (its `Requesting` future) already waits on +after an `Unroutable`. It moves on any +change to the routes covering the path, including a handler swap, and +wakes only what watches that path, so listeners see nothing new. Check that +re-registering an identical route does bump it; if it doesn't, bump it +there, not in the announcement. The watch wakes on any change to a +covering route, including a lower-ranked one that leaves the refusing +handler selected, so after each wake ask again only if the selected route +entry is no longer the one that refused. Not chosen: a timed retry, which #7836's +decision 2 (maintainer, 2026-11-04) ruled out. + +Fold this into whatever `follow` becomes under +[Listing and follow](/quest/m1/api-audit/route/events.md), which moves +instance changes inside `follow`. If the path watch changes under +[front parking](/quest/m2/origin-front-parks.md), use its replacement. + +Tests: a refused follower asks the replacement after an overlapping +identical-route handler swap; an announcement listener sees no event for +that swap; a change to a lower-ranked covering route doesn't re-ask the +refusing handler; and a follower that nothing replaced doesn't ask it +again. + +Public API: none. Wire: none. + +## Required + +- [HLS lists a broadcast's contributions without starting their media](/quest/m0/hls-contributions.md) - its PR, [moq-dev/moq#7836](https://github.com/moq-dev/moq/pull/7836), adds the refused follower this extends + +## Related + +- [Listing and follow](/quest/m1/api-audit/route/events.md) - `follow` takes over instance changes from announce events +- [Front parking](/quest/m2/origin-front-parks.md) - the same refusal wait for `request` +- [The HLS server follows each path once](/quest/m2/hls-follow-once.md) - another `Following` addition diff --git a/quest/m2/socket-old-hop-signal.md b/quest/m2/socket-old-hop-signal.md new file mode 100644 index 00000000000..21504ceb311 --- /dev/null +++ b/quest/m2/socket-old-hop-signal.md @@ -0,0 +1,98 @@ +# [L] A socket dial learns its peer is behind an older hop + +## Goal + +A socket dial to a peer whose endpoint an older moq-lite hop withholds fails +with `Unsupported` instead of waiting, while a dial to a peer that isn't +online yet keeps waiting. The relay holding the older session tells the +dialer on the wire. Dialers and relays on versions without that signal keep +today's wait. + +## Plan + +Planned 2026-11-04 with the maintainer, from the limit +[moq-dev/moq#7888](https://github.com/moq-dev/moq/pull/7888) documented. +Milestone m2 (maintainer). + +Where it gets lost: an endpoint's `.socket/` broadcast is hidden, and +before lite-07 ANNOUNCE_REQUEST has no hidden opt-in. When a peer `P` +reaches its relay `R` over a moq-lite-06 (or older) client session whose +publish half isn't opted into hidden routes, `R` never learns `P`'s +endpoint. Since #7888, `P` sees this from its own session version +(crate-private `HiddenWithheld` in `rs/moq-net/src/model/origin.rs`) and +fails its own dials and accepts. A lite-07 dialer `D` gets nothing: it waits +for `P`'s announcement as for a peer not online yet. `R` can't tell the two +apart either, since it never sees `P`'s hidden path. What `R` does know is +which of its sessions are older moq-lite with a publish half not opted into +hidden, and each one's publish scope from its grants. + +The signal must be about the peer, not the grant. A session's publish +grant says only that it *may* publish under a prefix, not that the dialed +endpoint exists behind it. A marker covering an older session's grant would +also fail a dial to a peer that isn't online yet under that grant, which is +exactly the case that must keep waiting. So the grant only narrows where to +look, and the evidence has to name the endpoint. + +Recommended design, to settle in this quest: + +- **Find the endpoint.** When a dial's interest in `/.socket/` + reaches a relay, the relay asks each of its older sessions whose publish + grant covers that path, with an ANNOUNCE_REQUEST naming the path, held + open while the dial waits so a peer that comes online later is seen. Such a + request names the hidden segment, so a lite-06 peer does announce its + endpoint (#7888's round-1 review response, + `rs/moq-net/src/lite/publisher.rs`). A session that may publish nothing + (a viewer) is never asked. +- **Tell the dialer.** If an older session announces the endpoint, the relay + sends a marker for that exact path on the wire: the endpoint exists behind + an older hop. `D`'s endpoint fails the dial with `Unsupported`, and only + for that path. With no answer, nothing is marked, and the dial waits as + for an absent peer. The marker tracks that announcement: it goes when the + older session unannounces the endpoint or ends, whichever comes first. +- **An announced route wins.** While any route announces the endpoint (for + example, a current session overlapping an older one during a reconnect), + the dial uses it and ignores the marker. +- **Across a cluster.** Relay-to-relay links ask for everything, not for a + dialer's narrow interest, so the relay holding the older session may never + see the dial, and a dial sends no SUBSCRIBE until the endpoint is + announced. Settle how the dialer's narrow interest reaches the holding + relay (for example, cluster links forwarding narrow interest under + `.socket`), and how markers cross cluster links. +- **Where the marker rides.** Its own announce-stream message, or a flag on + the [route layer](/quest/m1/cluster-routing/routes/README.md)'s messages if + that layer has landed. Choose whichever leaves the draft simpler. +- **Not chosen:** routing the dial once the probe finds the endpoint. The + peer can't tell that its relay probes, so it keeps failing its own dials + and accepts (#7888), and the session would connect from one side only. + +Versions and compatibility: + +- The signal goes in whichever moq-lite wip is current when this lands + (lite-07-wip before its cut, per the + [freeze policy](/quest/m1/lite07-finalize.md), otherwise the next wip), + with its draft text in the same PR. +- A dialer whose relay doesn't send it, or whose own session predates it, + waits as it does today. Published versions are unchanged. +- moq-transport hops carry no marker, so a dial across one waits as today. + No IETF-only surface is added. +- Rust, `@moq/net`'s socket and moq-ffi behave alike. `doc/concept/socket.md` + and the `Endpoint` rustdoc replace the "waits as for an absent peer" + limit. + +Tests, in the relay harness (`rs/moq-net/tests/integration/socket.rs`): a +lite-07 dial to a peer behind a lite-06 hop fails `Unsupported`, including +across a cluster hop. A dial to an absent peer still waits, including one +whose path lies under another lite-06 session's publish grant. A viewer-only +lite-06 session is never asked. A dial that starts before the older +session announces the endpoint fails `Unsupported` once it does. A marker goes when the older session +unannounces the endpoint, and a current route announcing the same endpoint +wins over it. A peer opted into hidden routes over lite-06 +(`an_older_hop_opted_into_hidden_connects`) still connects. + +Public API: none expected; dials return the existing `Error::Unsupported`. +Wire: a new per-path marker on the current moq-lite wip. + +## Related + +- [Sockets](/quest/m1/socket/README.md) - the dial and accept this reports on, and #7888's first-hop check +- [Routes and announces](/quest/m1/cluster-routing/routes/README.md) - the route layer a marker may ride, and how interest crosses cluster links diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index a05d8a9676f..838a92acb20 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -4936,6 +4936,7 @@ mod session_tests { /// route drops and returns as across a reconnect. The pad's actor takes each play, idle, and /// retirement in turn, so every round lands on the first pad and none ends it. #[test] + #[ignore = "flaky under load; see quest/m0/test-flakes-2/gst-reconnect-interleave.md"] fn restarts_under_reconnects_interleave_on_one_pad() { let _pad_ids = pad_ids(); let element = element(); diff --git a/sh/gh/test-inventory.allow b/sh/gh/test-inventory.allow index 29342f1414c..6f44c275b3f 100644 --- a/sh/gh/test-inventory.allow +++ b/sh/gh/test-inventory.allow @@ -2,6 +2,9 @@ # one: ` `. sh/gh/test-inventory.py prints the name of any # orphan missing here, and fails on an entry that has a caller. +# Flaky under load; the quest makes it reliable, then removes its `#[ignore]`. +ignore moq-gst::source::imp::session_tests::restarts_under_reconnects_interleave_on_one_pad /quest/m0/test-flakes-2/gst-reconnect-interleave.md + # Need a GPU, a desktop, or a device that no CI runner has. ignore moq-carla-bridge::gpu_video_reaches_the_worker /quest/m2/carla-bridge.md recipe carla::bridge-gpu /quest/m2/carla-bridge.md