diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index d7a6897dee1..6f7f7f1d6cc 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -167,5 +167,6 @@ The historical samples locate candidates; they are not controlled benchmarks. - [The queue host keeps the queue's disk](/quest/m0/ci/queue-host-disk.md) - agent sessions can't starve the queue runner of disk, and a crashed runner restarts - [The next test-CPU cuts](/quest/m0/ci/test-cost-followups.md) - the slow-test leftovers from #6987, cut at their source - [Biome checks every file it can parse](/quest/m0/ci/biome-whole-repo.md) - Biome runs over the whole repository and main is clean under it +- [Dispatcher tool changes run the dispatcher tests](/quest/m0/ci/dispatch-selection-gaps.md) - a tool-list change runs `sh/dispatch-query.test.py` in CI, and `sh/rs/uring.test.py` stops printing jq errors - [dispatch.sh's tests pass without KVM](/quest/m0/ci/dispatch-test-kvm.md) - `sh/dispatch-query.test.py` fakes the KVM probe instead of failing 20 tests on hosts without `/dev/kvm` - [mbx launches Cargo with any argument length](/quest/m0/ci/mbx-arg-max.md) - no "Argument list too long" in `sh/rs/select.test.py`'s recipe test diff --git a/quest/m0/ci/dispatch-selection-gaps.md b/quest/m0/ci/dispatch-selection-gaps.md new file mode 100644 index 00000000000..430879cc10c --- /dev/null +++ b/quest/m0/ci/dispatch-selection-gaps.md @@ -0,0 +1,36 @@ +# [S] Dispatcher tool changes run the dispatcher tests + +## Goal + +A change to the dispatcher's tool lists (`sh/dispatch.sh` and the files it +reads them from) runs `sh/dispatch-query.test.py` in CI, so a tool added to a +module can't break that suite on `main` unnoticed. `sh/rs/uring.test.py` +runs without the repeated `jq: error ... null (null) cannot be matched` +lines. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), from +[moq-dev/moq#7432](https://github.com/moq-dev/moq/pull/7432). + +- **Selection.** [6633d2f2e2](https://github.com/moq-dev/moq/commit/6633d2f2e2) + added `zizmor` to the `gh` module's tools in `sh/dispatch.sh`, and every + `CI_STRICT` case of `sh/dispatch-query.test.py` then failed with "these + tools are missing: zizmor" on `main`; #7432 fixes the stub. `run_lane` in + `sh/dispatch.sh` says gh's tests run on every `check` diff, and a change to + `sh/dispatch.sh` selects everything, so find why the commit's CI didn't run + or didn't fail on the suite (`.github/justfile` `test`, the CI lane that + runs it, the land gate) and fix that gap rather than adding a path rule. +- **jq noise.** `python3 sh/rs/uring.test.py` passes but prints + `jq: error (at :36): null (null) cannot be matched, as it is not a + string` four times. Its cargo-metadata fixture has no `workspace_root` or + `targets`, which the seed filter in `sh/rs/select.sh` (around line 66) + reads. Make the fixture complete, or make the filter tolerate their + absence, whichever matches real `cargo metadata` output. + +Public API: none. Wire: none. + +## Related + +- [workspace.test.py ignores the caller's gate filter](https://github.com/moq-dev/moq/pull/7432) - #7432's quest, the same dispatcher test environment +- [moq-dev/moq#7440](https://github.com/moq-dev/moq/pull/7440) - plans `quest/m0/ci/check-selection.md`, which makes a jq error in Rust selection fail the run; that turns this jq noise into a failure, so land the fixture fix first or together. It does not cover the tool-list selection gap diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index f2b20490eed..effc428413c 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -59,3 +59,4 @@ Public API: none. Wire: none. - [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio - [moq-relay's auth-trust test survives load](/quest/m0/test-flakes-2/relay-auth-trust.md) - it stops timing out on a busy host - [Draft-22 interop with a native publisher](/quest/m0/test-flakes-2/draft22-interop.md) - no "invalid fetch range" or audio gap with a Rust or C++ publisher +- [A Kotlin cancel test and a JS late-publisher test hold up](/quest/m0/test-flakes-2/kotlin-js-flakes.md) - the Kotlin flow-cancel test stops failing `unroutable` and the IETF blind-handle test passes under load diff --git a/quest/m0/test-flakes-2/kotlin-js-flakes.md b/quest/m0/test-flakes-2/kotlin-js-flakes.md new file mode 100644 index 00000000000..67a43838498 --- /dev/null +++ b/quest/m0/test-flakes-2/kotlin-js-flakes.md @@ -0,0 +1,29 @@ +# [S] A Kotlin cancel test and a JS late-publisher test hold up + +## Goal + +Two intermittent failures are fixed at their cause: the Kotlin test +"cancelling a flow collection cancels its native handle" no longer fails +with `unroutable`, and the JS integration test "ietf blind handle picks up a +publisher that arrives late" passes under a heavily loaded `just check`. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session): +root-cause both in one quest, since each is small and they surfaced +together in that session's merge runs. + +- **Kotlin.** `kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt:579`. + An `unroutable` answer means no route served the path when the request + resolved. A request made before the publisher's route reached the origin + returns it, and so does one made after that route was withdrawn or the + broadcast closed. Check both a wait on announcement that the test skips + and a route withdrawal or close during the collection it cancels. +- **JS.** `js/net/src/integration.test.ts:1994`, the moq-transport variant + of the lite test at `:1929`. Under load the late publisher's arrival and + the blind handle's wait race some deadline; find which, and replace the + wall-time dependency with an event or a controlled clock. + +Follow the epic's rules: no raised timeout and no retry. + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index 492020cee7c..39e5be624b6 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -135,6 +135,8 @@ per-frame CPU growth since #5926. - [Address groups by timestamp](/quest/m1/time-addressing/README.md) - group start timestamps replace sequence numbers in the moq-net API and lite-07 wire, while old subscribers, old publishers, and published JS clients keep working - [Pre-cut audit findings](/quest/m1/cut-audit/README.md) - fix the crashes, hostile-input costs, and correctness bugs the 2026-10-24 audit of main found - [Vendored code has every upstream fix it needs](/quest/m1/vendor-sync/README.md) - quinn, libv4l-rs, nvidia-video-codec-sdk, rust-media-libs, cros-codecs, and UniFFI generator fixes are ported or planned, once per release cycle +- [fMP4 refusals keep framing](/quest/m1/fmp4-refusal-framing.md) - a box refused at its header stops dropping the rest of its call's input, and the interleaved import benchmark is re-measured +- [MKV leaf element limits](/quest/m1/mkv-leaf-limits.md) - `CodecID`, `DocType`, and `CodecPrivate` are refused past per-element size limits instead of read whole - [Delete the tests that don't earn their cost](/quest/m1/prune-tests.md) - an off-cycle test cost sweep deletes tests whose cost outweighs their signal and makes the rest cheaper - [Remove the dependencies that don't earn their weight](/quest/m1/prune-deps.md) - Rust, JS, and vendored dependencies that are trivial, duplicated, or serve features not worth keeping are removed - [Remove what dominates build time](/quest/m1/prune-build-time.md) - the crates, features, and code that cost the most build time are removed or made cheaper @@ -177,6 +179,7 @@ per-frame CPU growth since #5926. - [Interop coverage](/quest/m1/interop-coverage/README.md) - the riskiest moq-transport features moq-net exposes get cross-implementation tests over drafts 18 and 22 - [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - A/A and deliberately slowed Nightly runs set the paired comparison's threshold, so its regression alerts can be trusted - [The shared pacer holds its rate under 2 Mbit/s](/quest/m1/pacer-low-rate.md) - a rate-paced controller stops sending up to twice its pacing rate at low rates, landed with a BBR3 retune that keeps the probe budgets +- [A reconfigure keeps the published jitter and delay floor](/quest/m1/catalog-jitter-floor.md) - `Rendition::set` floors unsupplied jitter and delay at the catalog's values, so a same-codec reconfigure after a jitter rise no longer aborts TS import - [HLS and DASH segment conformance](/quest/m1/hls-conformance.md) - nightly ffprobe checks that advertised timing, sync flags, and bytes match the media served - [Unbounded groups](/quest/m1/unbounded-groups/README.md) - groups and frames lose their size caps; a group keeps a 32 MB tail, and readers behind it replay through paced flights or refresh - [Legacy lite close drain](/quest/m1/lite-legacy-close-drain.md) - a lite-01..06 graceful close never cuts a final group the subscriber has not read diff --git a/quest/m1/announce-takeover.md b/quest/m1/announce-takeover.md index b65fb866c63..c6e8fcc3c62 100644 --- a/quest/m1/announce-takeover.md +++ b/quest/m1/announce-takeover.md @@ -116,6 +116,28 @@ Remove the cold-start warm-up reads (`claimed` in assert that the first subscription succeeds. Keep them for moq-transport. Run `just check`, `just test interop --all` and `just drafts check`. +### Once identity replaces ANNOUNCE_TAKEOVER + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session). Kept +separate from the identity rewrite of this Plan, which landed with +[moq-dev/moq#7420](https://github.com/moq-dev/moq/pull/7420). +"We're replacing EPOCH with IDENTITY soon" (maintainer, 2026-10-31). + +- Reword "takeover" as an event or operation, in terms of identity + continuity, in [moq-bench pins the instances it measures](/quest/m2/bench-epoch-pin.md) + (Goal and Plan), [the HLS server follows each path once](/quest/m2/hls-follow-once.md) + (Goal and Tests), and [routes and announces](/quest/m1/cluster-routing/routes.md) + (around line 37, "the separately proven Takeover operation"). +- Rewrite the EPOCH references in `drafts/draft-lcurley-moq-cluster.md` and + `drafts/draft-lcurley-moq-epoch.md` in terms of IDENTITY. +- Add a regression test where a publisher restart's new announcement + arrives before the old one's reset, on `solicit = false` moq-transport + sessions. [moq-dev/moq#7239](https://github.com/moq-dev/moq/pull/7239) + found that the subscriber (`rs/moq-net/src/ietf/subscriber.rs`, `attach`) + folds such a PUBLISH_NAMESPACE into the same epochless route as an update + instead of End then Start; per-announcement identities resolve it, and + the test proves they do. + Public API: breaking; removes `Takeover` and the predecessor fields in every language. Wire: removes lite-07's ANNOUNCE_TAKEOVER (0x5), adding nothing beyond [Identity and epoch](/quest/m1/api-audit/route/identity.md)'s fields. diff --git a/quest/m1/catalog-jitter-floor.md b/quest/m1/catalog-jitter-floor.md new file mode 100644 index 00000000000..1979c48bab2 --- /dev/null +++ b/quest/m1/catalog-jitter-floor.md @@ -0,0 +1,44 @@ +# [S] A reconfigure keeps the published jitter and delay floor + +## Goal + +Re-setting an already-published rendition, such as an H.264 or H.265 +same-codec resolution change, never lowers the jitter or delay its catalog +already advertises for a field the new config leaves unset. The reconfigure +publishes with the higher of the two, as a measurement does, instead of +failing with `JitterDecreased` and aborting the TS import. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), found +by the sessions of [moq-dev/moq#7436](https://github.com/moq-dev/moq/pull/7436) +and [moq-dev/moq#7387](https://github.com/moq-dev/moq/pull/7387). Planned +here in m1 as its own [S] rather than fixed inside #7436 (no test there needs +it) or dropped: it blocks the conformance test in #7387 once that fixture is +repaired. + +`Rendition::set` (`rs/moq-mux/src/catalog/tracks.rs:655`) resolves the new +config from what it supplies plus the last measurement (`Self::resolved`), +then `check_decrease` refuses anything below the catalog's current jitter or +delay. `Rendition::estimate` (`:711`) already floors its result at the +published values (`resolved.jitter.max(published.jitter)`, `:725-726`). +After a measurement raises jitter mid-run, a reconfigure that supplies no +jitter resolves to the lower detected value and fails. Floor the unsupplied +jitter and delay in `set` at the catalog's published values the same way, so +`check_decrease` still refuses only a value the caller explicitly lowers. + +`published_text_jitter_never_decreases` (`:1078`) expects a `None` re-set +after a supplied 100 ms to fail; with the floor it publishes 100 ms. Keep +the explicit-lower case failing and update the `None` case to the floor. + +Regression test: a unit test in `tracks.rs` that publishes a video config, +raises its jitter with `estimate`, then `set`s a config with new dimensions +and no jitter, and asserts it succeeds with the raised jitter still +published. + +Public API: none. Wire: none. + +## Related + +- [HLS and DASH segment conformance](/quest/m1/hls-conformance.md) - the conformance test this unblocks +- [HLS export follows a resolution change](/quest/m1/hls-reinit-same-codec.md) - the same-codec reconfigure that hits it diff --git a/quest/m1/fmp4-refusal-framing.md b/quest/m1/fmp4-refusal-framing.md new file mode 100644 index 00000000000..ca06ec8e596 --- /dev/null +++ b/quest/m1/fmp4-refusal-framing.md @@ -0,0 +1,43 @@ +# [S] fMP4 refusals keep framing + +## Goal + +A box the fMP4 importer refuses at its header no longer leaves a caller +feeding arbitrary chunks out of frame. Either the importer reads the rest of +that decode call's input past the refused box, so later calls stay +correctly framed whatever chunk boundaries the caller picks, or the refusal +is terminal and every later call fails, so no input is silently dropped +mid-stream. The 4-track +interleaved import benchmark is no slower than before the streaming change, +or the cost is measured and accepted. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), from +the follow-ups of [moq-dev/moq#7232](https://github.com/moq-dev/moq/pull/7232) +(fMP4 streams mdat) and [moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234) +(MKV streams blocks). Both have landed. #7234 landed first, so #7232 already +streams the importer's samples through `Producer::open`/`commit`; the +original open/commit bullet is done, and this quest keeps the other two +([noted on moq-dev/moq#7468](https://github.com/moq-dev/moq/pull/7468#issuecomment-6554670247)). + +- **A refused box.** `Import::decode` (`rs/moq-mux/src/container/fmp4/import.rs`) + documents it: a refused box or fragment discards the rest of its body, and + the rest of the refusing call's input is not read. That suits box-aligned + callers (HLS ingest feeds one segment per call) but loses framing for + arbitrary chunking. Either continue past the refused box, or make a + refusal terminal for the importer as the MKV importer's is. Choose during + implementation the option that keeps the importer's state simpler while + meeting HLS ingest's needs, and record the choice in the PR. +- **Benchmark.** #7232 measured a possible ~25% slowdown in the 4-track + 16-run whole-input `cmaf-interleaved-import` case, in a noisy run where the + 4-track cases swung ±40% between runs of one source. Re-measure it with the + paired comparison on a quiet host, against the commit before #7232, and + fix it or record why it is accepted. + +Public API: none expected. Wire: none. + +## Related + +- [MKV leaf element limits](/quest/m1/mkv-leaf-limits.md) - the other leftover of the importer streaming work +- [Media fuzz](/quest/m2/media-fuzz.md) - would catch a dropped-input bug like the refused-box one diff --git a/quest/m1/hls-conformance.md b/quest/m1/hls-conformance.md index 72a06a67c69..ca4c6426ff2 100644 --- a/quest/m1/hls-conformance.md +++ b/quest/m1/hls-conformance.md @@ -33,9 +33,15 @@ Decided 2026-10-30 (maintainer): fix first. The two bugs the test found are their own quests, required below, and the test lands once it passes with both checks intact. +Added 2026-10-31 (maintainer, the /quest-complete requeue session): once +the fixture is repaired, the test's TS import hits a third bug, a +reconfigure failing `JitterDecreased` after a mid-run jitter rise, also its +own quest below. + ## Required - [HLS export follows a resolution change](/quest/m1/hls-reinit-same-codec.md) - the conformance test's same-codec switch fails until it lands +- [A reconfigure keeps the published jitter and delay floor](/quest/m1/catalog-jitter-floor.md) - the conformance test's TS import aborts with `JitterDecreased` at its reconfigure until it lands ## Related diff --git a/quest/m1/mkv-leaf-limits.md b/quest/m1/mkv-leaf-limits.md new file mode 100644 index 00000000000..0c9ddfc782b --- /dev/null +++ b/quest/m1/mkv-leaf-limits.md @@ -0,0 +1,35 @@ +# [XS] MKV leaf element limits + +## Goal + +The MKV importer refuses a `CodecID`, `DocType`, or `CodecPrivate` element +whose declared size is past a limit specific to that element, so a header +declaring a huge one cannot pin importer memory. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), a +follow-up of [moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234), +which streams blocks and discards ignored elements but still reads these +three leaf elements whole (`rs/moq-mux/src/container/mkv/import.rs`, where +the track entry collects `CodecID` and `CodecPrivate`). #7234 has landed. + +The maintainer earlier ruled out a blanket tag-size cap (it would refuse +large Segments and Clusters). These limits are per element +and bound only what the importer must hold whole: a `CodecID` and `DocType` +are short ASCII strings, and `CodecPrivate` is the decoder config of a +codec the importer supports: avcC (`V_MPEG4/ISO/AVC`), hvcC +(`V_MPEGH/ISO/HEVC`), av1C (`V_AV1`), an OpusHead (`A_OPUS`), an AAC +AudioSpecificConfig (`A_AAC`), or a FLAC header (`A_FLAC`). The FLAC header +carries every metadata block, so it can hold embedded cover art; size its +limit for that or refuse such art explicitly. Pick each limit from its spec +and the largest real decoder config, check them against the MKV fixtures +for each codec, and return the +importer's existing error for a refused element. Test each element declared +past its limit, fed in small chunks. + +Public API: none. Wire: none. + +## Related + +- [fMP4 refusals keep framing](/quest/m1/fmp4-refusal-framing.md) - the fMP4 leftover of the same streaming work diff --git a/quest/m2/README.md b/quest/m2/README.md index deb0d525d3a..3687d0395c0 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -30,14 +30,15 @@ lite-07; bitrate enforcement is not part of this release. - [One policy for oversized container metadata](/quest/m2/import-metadata-policy.md) - fMP4 and MKV refuse or stream huge metadata by one documented rule - [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the cmsg walks in moq-sock and moq-uring have invariant fuzz targets, and the unsafe peer input reaches runs under Miri - [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the VA-API renderer, decode, and transcode tests run in `just rs gpu` on AMD and Intel and fail instead of skipping, and hosted CI reports them ignored +- [One Ruff pin and Swift formatting on aarch64-linux](/quest/m2/format-gate-followups.md) - only the dev shell pins Ruff, and aarch64-linux checks Swift formatting once nixpkgs builds `swiftc` there - [Weekly mutation testing on critical modules](/quest/m2/mutants.md) - cargo-mutants runs weekly on moq-auth and moq-net's permission, resume, and expiry code - [Video quality](/quest/m2/video-quality/README.md) - a nightly VMAF harness grades every encoder backend and on-demand GoP pops against floors, and moq-boy's encode is fixed under it -- [noq adopts moq-congestion](/quest/m2/noq-congestion.md) - noq's BBR3 and Cubic delegate to the published crate behind noq's unchanged API - [moq-relay forwards QUIC packets by QUIC-LB server ID](/quest/m2/relay-quic-lb-forward.md) - an established connection that lands on the wrong instance behind an anycast IP or VIP is forwarded to its owner, which replies from the shared address - [P2P](/quest/m2/p2p/README.md) - application-selected peer sessions use shared priorities, routing, and same-epoch handover - [rust2 publishes its UniFFI fork](/quest/m2/rust2-uniffi.md) - waiting on moq-dev/rust2 to import UniFFI and moq's generators and publish them - [Switch to rust2's UniFFI](/quest/m2/uniffi-fork.md) - moq-ffi and every wrapper build from moq-dev/rust2's fork under one pin, and moq's generator code is deleted - [Generated C bindings](/quest/m2/c/README.md) - C generated from moq-ffi ships as `moq-c` 0.8.0 and replaces the hand-written crate +- [A C origin can discover eagerly, like a relay](/quest/m2/c-origin-relay.md) - the C API puts an origin in relay mode, so forwarding a client session's routes needs no local announce watcher - [A runtime-free QUIC driver](/quest/m2/quic-sansio-driver.md) - a caller's own loop moves UDP datagrams and polls moq sessions over moq-quic, for rust2's bindings - [MoQ on rind](/quest/m2/rind/README.md) - every MoQ crate takes its crypto from rind, a pure-Rust graviola fork, so no build compiles C crypto - [Benchmark sources](/quest/m2/bench-sources.md) - every measuring nightly benchmark feeds the regression table, and none is a smoke run @@ -45,6 +46,7 @@ lite-07; bitrate enforcement is not part of this release. - [QoS](/quest/m2/qos/README.md) - the relay's egress `lag` histogram and `dropped` counters on the stats totals and prefix tracks - [Lazy retained HLS instances](/quest/m2/hls-lazy-instances.md) - a replaced HLS instance starts its watchers only when a request reaches it - [HLS and DASH browser playback](/quest/m2/hls-player-test.md) - hls.js and dash.js play, reload, switch, and survive reconfiguration nightly +- [The DASH manifest keeps segments across a reconfigure](/quest/m2/dash-reconfigure.md) - a same-codec reconfigure keeps the old instance's segments listed in DASH, as HLS does - [Re-init retires timelines](/quest/m2/reinit-retire.md) - fMP4 and MKV re-inits retire the old timeline, so import state stays bounded - [Replaced producers retire their records](/quest/m2/producer-retire.md) - encoders and bindings retire a replaced producer's HLS records - [Broadcast fairness](/quest/m2/broadcast-fairness.md) - broadcasts share a cluster link fairly, so one tenant's priority never starves another's @@ -62,6 +64,7 @@ lite-07; bitrate enforcement is not part of this release. - [Native synchronous codecs](/quest/m2/native-compression-sync.md) - report the stateful small-frame synchronous API requirement - [Native browser codecs](/quest/m2/native-compression-adopt.md) - adopt native operations when timing, frame and resource contracts hold +- [moq-net's library tests pass under cargo test](/quest/m2/net-test-subscriber.md) - two moq-net tests stop each installing a global tracing subscriber, so plain `cargo test -p moq-net --lib` passes - [OBS native codecs](/quest/m2/obs-moq-video/README.md) - replace FFmpeg video and audio decoding with moq-video and moq-audio, deliver GPU frames, and use native audio/video encoders - [MoQ in obs-studio](/quest/m2/obs-studio/README.md) - a native MoQ output and service merged into obs-studio, once its maintainers agree; the plugin carries it until then @@ -195,6 +198,8 @@ lite-07; bitrate enforcement is not part of this release. - [Publisher timeliness at relay ingest](/quest/m2/qos-publisher-timeliness.md) - follow-on capability after the release contract is stable - [Auth session caps cost O(1)](/quest/m2/auth-session-index.md) - per-token and per-remote counts and a time-ordered expiry queue replace the scans, after #7211 +- [The relay refuses a cluster peer it cannot dial](/quest/m2/relay-undialable-peers.md) - moq-tokio reports the schemes it can dial, and the relay refuses an undialable peer at startup instead of retrying forever +- [Drop moq-auth's iroh transport](/quest/m2/auth-drop-iroh.md) - the next breaking moq-auth release drops `Transport::Iroh` and the JS `"iroh"` value - [H.264 parsing with maintained crates](/quest/m2/h264-reader.md) - moq-mux parses SPS VUI with h264-reader and shares one bit reader across codec config parsers - [moq-transport finish interop](/quest/m2/ietf-finish.md) - the interop `finish` case runs on every even moq-transport draft instead of being skipped - [JS joining FETCH](/quest/m2/ietf-js-joining-fetch.md) - the JS publisher answers a joining FETCH that arrives after a quick finish, mirroring the Rust bound @@ -220,7 +225,7 @@ lite-07; bitrate enforcement is not part of this release. - [Compile time audit](/quest/m2/audit-compile-time.md) - build-time hotspots are measured against the m1 baselines, starting with the transport share now that moq-net compiles once, and the worst are removed, made cheaper, or planned - [Lint override and deprecation audit](/quest/m2/audit-lints.md) - repeat the suppression and deprecation audit after the release ships, starting with clippy on the macOS and Windows targets - [Archive reads against a real store](/quest/m2/archive-store-bench.md) - LIST/GET counts and latency over track count -- [Branch checks lint examples](/quest/m2/lint-examples.md) - `just check` runs clippy on the selected crates' examples +- [Branch checks lint examples](/quest/m2/lint-examples.md) - `just check` runs clippy on the selected crates' examples, and PR CI builds the C++, Go, and Python interop clients - [Drop kio's epoch fence](/quest/m2/crossbeam-pin-fence.md) - waiting on crossbeam#1352, then the `Read::drop` fence goes - [A cancelled miss stops waiting](/quest/m2/fill-wait-cancel-test.md) - a test proves a relay miss waiting on its own fill ends when no reader wants it - [No MOQ_* knobs outside the binaries](/quest/m2/env-knobs.md) - test, bench, and library knobs leave the `MOQ_*` prefix diff --git a/quest/m2/audit-lints.md b/quest/m2/audit-lints.md index 280543d688c..881e58c306a 100644 --- a/quest/m2/audit-lints.md +++ b/quest/m2/audit-lints.md @@ -74,6 +74,20 @@ Python wrapper. This run only reviews what those gates can't see. enable `reportUnnecessaryTypeIgnoreComment` without falsely flagging its three needed ignores. Audit it with the bindings installed. +- `set_var`/`remove_var` (decided 2026-10-31, maintainer, the + /quest-complete requeue session): replace `rs lint`'s `git grep` text + check (`rs/justfile`, the `lint` recipe) with a `disallowed-methods` entry + for `std::env::set_var` and `std::env::remove_var` in the root + `clippy.toml`, which arrives with + [moq-dev/moq#7329](https://github.com/moq-dev/moq/pull/7329) (queued). The + recipe's comment says why it is text today: the clock lint's test + allowances (`expect(clippy::disallowed_methods)`) would also silence an + environment write, and a specific `reason` does not narrow what an + expectation silences. Move crate- and module-wide test expectations (such + as the one on `rs/moq-ffi/src/lib.rs`'s test-only `mod bench`) onto the + items that read the clock, and drop the text check only once none remain, so the + clippy entry still catches a write in a test. + ### Results (m1, 2026-10-24) - Rust: removed the Apple-only `msghdr_x` `non_camel_case_types` (Rust diff --git a/quest/m2/auth-drop-iroh.md b/quest/m2/auth-drop-iroh.md new file mode 100644 index 00000000000..9de43fbdf5c --- /dev/null +++ b/quest/m2/auth-drop-iroh.md @@ -0,0 +1,30 @@ +# [XS] Drop moq-auth's iroh transport + +## Goal + +The next breaking moq-auth release drops `Transport::Iroh` +(`rs/moq-auth/src/request.rs`) and the `"iroh"` transport value in +`js/auth/src/contract.ts`, which nothing sends once the iroh backend is gone. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), a +follow-up of [moq-dev/moq#7345](https://github.com/moq-dev/moq/pull/7345). +That PR kept both because the enum is the relay-to-auth-server JSON contract +and its quest said "Wire: none"; the relay just never sends `iroh` now. + +Starts after #7345 lands; check with `gh pr view 7345`. Land it with the +first moq-auth release that already breaks for another reason, rather than +cutting a breaking release for this alone. The relay sends the transport +name and the auth server reads it. An old auth server is unaffected, since +a new relay never sends `iroh`. A new auth server still accepts `iroh` from +an older relay: an unrecognized name deserializes to `Unknown` +(`#[serde(other)]` in Rust, `z.catch` in JS). Update the auth contract docs +and any test or fixture naming `iroh`. + +Public API: breaking in moq-auth and `@moq/auth` (one enum variant). Wire: +the auth JSON contract loses the `iroh` transport value. + +## Related + +- [The relay refuses a cluster peer it cannot dial](/quest/m2/relay-undialable-peers.md) - the other iroh leftover diff --git a/quest/m2/c-origin-relay.md b/quest/m2/c-origin-relay.md new file mode 100644 index 00000000000..1fbf9e18586 --- /dev/null +++ b/quest/m2/c-origin-relay.md @@ -0,0 +1,34 @@ +# [S] A C origin can discover eagerly, like a relay + +## Goal + +The C API (`rs/moq-c`, `moq.h`) can put an origin in relay mode, with the +eager discovery moq-net gives an origin handle on a relay link, so a C +application that forwards a client session's routes needs no local +announce watcher of its own to make discovery run. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), a +follow-up of [moq-dev/moq#7246](https://github.com/moq-dev/moq/pull/7246) +(moq-c API gaps), whose follow-ups note that C has no way to make an origin +relay-mode for a client-ingest relay. + +In moq-net, a handle's link decides it: `origin::Producer::with_link` +(`rs/moq-net/src/model/origin.rs`) takes `Link::Local`, `Link::Peer`, or +`Link::Upstream`, and a non-local link's sessions discover everything they +may (`relays()`). `moq_origin_create` (`rs/moq-c/src/api.rs`) always makes a +local one. Expose the choice in C, by a create option or a setter mirroring +`with_link`, following whatever moq-ffi exposes if it has gained the same +by then, and document it in `moq.h` and `doc/lib/c`. If the +[generated C line](/quest/m2/c/README.md) has replaced the hand-written crate +before this starts, add it to moq-ffi instead so every binding gets it. +Test a C origin forwarding a client session's routes with no announce +watcher. + +Public API: additive in moq-c, or in moq-ffi if the generated C line has +replaced it. Wire: none. + +## Related + +- [moq-c at parity](/quest/m1/c/README.md) - the hand-written crate's m1 parity work, which this does not block diff --git a/quest/m2/dash-reconfigure.md b/quest/m2/dash-reconfigure.md new file mode 100644 index 00000000000..757651ccdec --- /dev/null +++ b/quest/m2/dash-reconfigure.md @@ -0,0 +1,44 @@ +# [M] The DASH manifest keeps segments across a reconfigure + +## Goal + +After a same-codec reconfigure (such as a resolution change) replaces a +rendition's instance, the DASH manifest keeps listing the old instance's +segments, each under the init it was muxed for, until the window slides past +them, as the HLS media playlist does. A DASH player loses no listed segment +at the switch. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), a +follow-up of [moq-dev/moq#7436](https://github.com/moq-dev/moq/pull/7436). +That PR keeps a replaced instance (`Rendition::previous`) and lists its +segments in the HLS playlist behind `EXT-X-DISCONTINUITY`, each with its own +`EXT-X-MAP`. `Rendition::segment` and `Rendition::init` already resolve a +predecessor's URLs. The DASH manifest (`rs/moq-hls/src/export/mpd.rs`, +`render_manifest`) still renders one `Period id="0"` with only the current +instance's segments, and `doc/bin/hls.md` says so. + +Represent the predecessor in DASH by either: + +- **multiple Periods**, one per instance, each with its own + `SegmentTemplate` and initialization URL; or +- **a per-segment init** in one Period. + +Choose during implementation by which dash.js and the conformance test +accept without a reload, and record the choice in the PR. Reuse the +predecessor window #7436 adds rather than tracking a second one, and update +`doc/bin/hls.md`. Test that a reconfigure keeps the old segments listed in +the manifest, and extend the DASH checks of +[HLS and DASH segment conformance](/quest/m1/hls-conformance.md) if it has +landed. + +Public API: none. Wire: none. + +## Required + +- [HLS export follows a resolution change](/quest/m1/hls-reinit-same-codec.md) - the instance hand-over (#7436) this extends to DASH + +## Related + +- [HLS and DASH browser playback](/quest/m2/hls-player-test.md) - dash.js survives reconfiguration nightly diff --git a/quest/m2/format-gate-followups.md b/quest/m2/format-gate-followups.md new file mode 100644 index 00000000000..41cddd6a133 --- /dev/null +++ b/quest/m2/format-gate-followups.md @@ -0,0 +1,27 @@ +# [XS] One Ruff pin and Swift formatting on aarch64-linux + +## Goal + +Only the dev shell pins Ruff: `ruff` leaves the uv dev group in +`pyproject.toml` (and `uv.lock`). The Linux gate checks Swift formatting on +aarch64-linux too, once nixpkgs builds `swiftc` there. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), the +follow-ups of [moq-dev/moq#7319](https://github.com/moq-dev/moq/pull/7319) +(format gates), which has landed. + +- **Ruff.** #7319 makes the repo-wide `python` check module run the dev + shell's Ruff and keeps `ruff` in the uv dev group (`pyproject.toml:61`) + only for editors, so two Ruff versions are pinned. Drop it from the dev + group and `uv.lock`, and point editor setup at the dev shell's Ruff. +- **Swift on aarch64-linux.** #7319 adds `swift-format` to the dev shell + only off aarch64-linux (`flake.nix`, around line 306), because nixpkgs + marks its `swiftc` broken there, so `sh/swift/format.sh` skips and + release-PR Check (`ubuntu-24.04-arm`) checks no Swift formatting. When + nixpkgs builds it, add `swift-format` there and drop the skip. Until then + this half waits; check nixpkgs' `swift` package `meta.broken` for + aarch64-linux. + +Public API: none. Wire: none. diff --git a/quest/m2/import-metadata-policy.md b/quest/m2/import-metadata-policy.md index b0aa9d63bed..7b8c83c8ed6 100644 --- a/quest/m2/import-metadata-policy.md +++ b/quest/m2/import-metadata-policy.md @@ -23,4 +23,4 @@ Public API: possibly new `pub const` bounds and a shared refusal error in moq-mu ## Related - [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - would measure parsed-state amplification under either policy -- [MKV leaf limits](https://github.com/moq-dev/moq/pull/7468) - planned in that open PR as `quest/m1/security-audit/mkv-leaf-limits.md`: per-element limits on the three MKV fields read whole, in m1. This quest starts from whatever it lands and aligns the two importers' rules, not the limits it picked +- [MKV leaf limits](/quest/m1/mkv-leaf-limits.md) - per-element limits on the three MKV fields read whole, in m1. This quest starts from whatever it lands and aligns the two importers' rules, not the limits it picked diff --git a/quest/m2/lint-examples.md b/quest/m2/lint-examples.md index e580d988493..7f8b02bb825 100644 --- a/quest/m2/lint-examples.md +++ b/quest/m2/lint-examples.md @@ -4,7 +4,7 @@ On Linux, `just check` runs clippy on the example targets of each crate it selects, so a broken or lint-failing example fails the branch rather than the -next Nightly. +next Nightly. PR CI also builds the C++, Go, and Python interop clients. ## Plan @@ -18,4 +18,12 @@ Decided 2026-10-28 (maintainer): examples only. The 68 benches stay with Nightly, since they cost more to build. Measure the added build time on a moq-cli change and record it in the PR. +Decided 2026-10-31 (maintainer, the /quest-complete requeue session): PR CI +also builds the C++, Go, and Python interop clients +(`test/interop/clients/{cpp,go,python}`) when their bindings or the client +change. They went stale unnoticed while +[moq-dev/moq#7311](https://github.com/moq-dev/moq/pull/7311) changed the +decoder API, since only the interop runs build them. A build (and Python +import or type check) is enough; running them stays with interop. + Public API: none. Wire: none. diff --git a/quest/m2/media-fuzz.md b/quest/m2/media-fuzz.md index aa8ea8fbcc9..64248d4af3c 100644 --- a/quest/m2/media-fuzz.md +++ b/quest/m2/media-fuzz.md @@ -24,4 +24,15 @@ the existing fixtures. Assert a work and allocation bound proportional to input size. Wire the targets into the nightly gate next to moq-net's, and replay committed regressions in the merge gate. +Measure parse amplification too (decided 2026-10-31, maintainer, from +[moq-dev/moq#7232](https://github.com/moq-dev/moq/pull/7232)). That PR bounds +a held `moov` or `moof` at 32 MiB (`fmp4::MAX_HELD_BYTES`) and a `moof` at +2^21 samples (`fmp4::MAX_FRAGMENT_SAMPLES`), but a body at the bound +decodes into larger structures: an explicit `trun` entry costs 4 to 16 +input bytes and about 40 parsed, so 2.5x to 10x. A `trun` without +per-sample fields parses to no entries, but planning still resolves every +sample it declares, up to the 2^21 bound. Record the peak allocation for a +maximal `moov`, a maximal `moof` of 4-byte entries, and a field-less `moof` +at the sample bound, and fix each or accept it with the number. + Public API: none. Wire: none. diff --git a/quest/m2/net-test-subscriber.md b/quest/m2/net-test-subscriber.md new file mode 100644 index 00000000000..cb99ac19a5f --- /dev/null +++ b/quest/m2/net-test-subscriber.md @@ -0,0 +1,31 @@ +# [XS] moq-net's library tests pass under cargo test + +## Goal + +`cargo test -p moq-net --lib` passes, not only under nextest: +`setup::tests::credentials_stay_out_of_traces` and the model tests using +`model/test_tracing.rs` no longer each install a process-global tracing +subscriber, so whichever runs second in the shared process still captures +its events. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session). +Nextest runs each test in its own process, which hides the clash. + +`credentials_stay_out_of_traces` (`rs/moq-net/src/setup.rs:306`) uses +`#[tracing_test::traced_test]`, and `count_drop_warnings` +(`rs/moq-net/src/model/test_tracing.rs:39`) calls +`tracing::subscriber::set_global_default` and expects no other global +subscriber. Its doc explains why it is global: tracing caches callsite +interest process-wide, so a scoped dispatcher misses events another thread +cached as disabled. Keep that property. Likely one shared global test +subscriber that both use (`count_drop_warnings`' thread-local capture slot +extended to the credential check), or the credential test moved onto +`count_drop_warnings`' capture. Confirm with `cargo test -p moq-net --lib`. + +Public API: none. Wire: none. + +## Related + +- [moq-dev/moq#7243](https://github.com/moq-dev/moq/pull/7243) - fixed the same nextest-only pass in moq-clock (now `kio::clock`) by isolating its process state diff --git a/quest/m2/net-wrap/e2ee.md b/quest/m2/net-wrap/e2ee.md index cb62ab2bf20..cbc3814c939 100644 --- a/quest/m2/net-wrap/e2ee.md +++ b/quest/m2/net-wrap/e2ee.md @@ -58,6 +58,38 @@ the reverse (decided 2026-10-20, maintainer): without changing moq-net's `track(name)`; propose the encoding in the PR. Record it in `draft-lcurley-moq-e2ee.md`, replacing the Catalogs section's `.z` sibling rule. +- Datagram header allowance (decided 2026-10-31, maintainer, a follow-up + of [moq-dev/moq#7435](https://github.com/moq-dev/moq/pull/7435)): + `MAX_DATAGRAM_HEADER` (`rs/moq-e2ee/src/limits.rs`, + `js/e2ee/src/limits.ts`; 28 after #7435, lite-07's Type plus three 9-byte + varints) also counts the optional lite datagram fields: the relayed fields + the lite `ietf` extension adds, and moq-net's Inflated Size header field, + which a compressed physical track's datagram carries outside the AEAD + (`rs/moq-net/src/lite/datagram.rs`). Without them a maximum-size protected + datagram can encode past the 1200-byte body and be dropped. That header + field is not the wrapper's own inflated size above, which sits inside the + authenticated payload and counts against the plaintext budget instead. + Today a caller's `track::Info` reaches the physical track unchanged, so it + can be compressed. Once the wrapper creates the physical track + uncompressed, the header field never appears and the allowance needs only + the `ietf` fields. Of those, the relay state, object ID, priority, and + status have a fixed worst case, but `encode_datagram` + (`rs/moq-net/src/lite/ietf.rs`) also writes the forwarded properties + block, which may run to `MAX_OBJECT_EXTENSIONS` (64 KiB, + `rs/moq-net/src/ietf/group.rs`), so no fixed allowance covers it, and the + lite publisher drops a datagram whose body exceeds 1200 bytes + (`rs/moq-net/src/lite/publisher.rs`). Choose during implementation between + capping the properties a hop forwards on a protected datagram (and + counting that cap) or documenting that relay properties are outside the + allowance, so such a datagram may be dropped, and record the choice in the + PR. Sealing and opening share + `MAX_DATAGRAM_PAYLOAD` today (`rs/moq-e2ee/src/track.rs`, + `js/e2ee/src/track.ts`), so a wider allowance would also make `open` + refuse a datagram an older sender sealed under the narrower one. Split + the seal and open limits, or pick the limit by wire version, so a + receiver still opens anything that fits the body. Update the e2ee + draft's plaintext bound and the `moq-e2ee-00` vectors with it, covering + both limits and a datagram with non-empty relay properties. - Ranges: every moq-net range starts at a group's first frame ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)), so the wrapper reads each physical group whole and inflates from frame 0. diff --git a/quest/m2/noq-congestion.md b/quest/m2/noq-congestion.md deleted file mode 100644 index 5936dd8ae36..00000000000 --- a/quest/m2/noq-congestion.md +++ /dev/null @@ -1,31 +0,0 @@ -# [M] noq adopts moq-congestion - -## Goal - -noq's BBR3 and Cubic run `moq-congestion`, its own copies are deleted, and -noq's public congestion API is unchanged. The PR merges in -[n0-computer/noq](https://github.com/n0-computer/noq) with its maintainers' -approval. - -## Plan - -Decided with the maintainer on 2026-10-23, answering the noq maintainers' -offer: moq owns BBR3 as an external crate rather than maintaining it inside -noq. - -- Starts once the m1 cut publishes `moq-congestion` to crates.io (decided: - no backport or manual publish). Check with `cargo search moq-congestion`. -- noq keeps its `Controller` trait public and stable; its `Bbr3`/`Cubic` - configs become adapters that delegate to `moq-congestion`. noq does not - re-export `moq-congestion` types, so each crate bump is an explicit noq - change that wires any new signal. -- noq has no `SpaceId` and no `on_app_limited`; the adapter supplies what - noq can and states what BBR3 loses without it. noq keeps its pacer and - reads the controller's rate. -- Check against noq's open BBR3 reports before claiming them fixed: - [n0-computer/noq#800](https://github.com/n0-computer/noq/issues/800), - [n0-computer/noq#852](https://github.com/n0-computer/noq/issues/852), and - [n0-computer/noq#819](https://github.com/n0-computer/noq/issues/819). -- Once iroh ships it, `moq-tokio`'s iroh path (its noq `ControllerFactory`) - runs the same controllers; bumping that pin is a separate quest if it - needs code. diff --git a/quest/m2/relay-undialable-peers.md b/quest/m2/relay-undialable-peers.md new file mode 100644 index 00000000000..2b89bd63295 --- /dev/null +++ b/quest/m2/relay-undialable-peers.md @@ -0,0 +1,40 @@ +# [S] The relay refuses a cluster peer it cannot dial + +## Goal + +moq-tokio exposes which URL schemes its build can dial, and moq-relay +refuses to start when a configured cluster peer uses a scheme it can't +dial (for example `iroh://` once the iroh backend is gone, or `ftp://`), +naming the peer, instead of retrying it forever. + +## Plan + +Decided 2026-10-31 (maintainer, the /quest-complete requeue session), a +follow-up of [moq-dev/moq#7345](https://github.com/moq-dev/moq/pull/7345), +which drops moq-tokio's iroh backend. A CodeRabbit thread there +([discussion](https://github.com/moq-dev/moq/pull/7345#discussion_r4432489908)) +found the gap; the behavior predates that PR and covers every unsupported +scheme. + +`rs/moq-tokio/src/client.rs` (`connect_inner`) picks a transport by scheme +under feature gates (`tcp`, `tls`, `unix`, `ws`/`wss`), and every other +scheme falls through to QUIC when it is compiled (raced with WebSocket when +that is too) or to WebSocket alone. Those reject what they can't carry +only at dial time: QUIC takes `http`, `https`, `moqt` and `moql` +(`rs/moq-tokio/src/quic/backend.rs`), WebSocket `http`, `https`, `ws` and +`wss` (`rs/moq-tokio/src/websocket.rs`), so a peer like `ftp://` fails each +attempt and is retried. So which schemes dial depends on the build's +features and on that fallthrough. Expose a dialability check from moq-tokio +(a function or method beside `Client`, derived from the same dispatch and +`cfg`s so it cannot drift and never rejects a scheme that dials today), and +check every peer in `Config::connect` +(`rs/moq-relay/src/cluster.rs`) against it at startup. A peer returned later +by an `api.url` endpoint can't fail startup; decide in the PR what it does +(recommended: log once and skip it rather than retry). Test a config naming +an unsupported scheme. + +Public API: additive in moq-tokio. Wire: none. + +## Related + +- [Drop moq-auth's iroh transport](/quest/m2/auth-drop-iroh.md) - the other iroh leftover