diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 54451fb5dda..92b967a3dd3 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -67,4 +67,5 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Require platform gates](/quest/m0/ci/platform-queue.md) - activate the consolidated platform checks in the main and release rulesets after the workflow lands - [mbx on R2](/quest/m0/ci/mbx-r2.md) - Rust builds on every OS read an R2 cache that only `main` writes, escaping the 10 GB Actions cache - [Nix cache](/quest/m0/ci/nix-cache.md) - `nix.moq.dev` on R2 replaces Cachix for CI and releases +- [TS path filter](/quest/m0/ci/ts-path-filter.md) - TS validation runs whenever a crate it builds changes, checked against `cargo metadata` - [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the first `moq-ffi-v*` tag that runs `Release FFI`, and a `Shipping sizes` run; verify the wrappers and size keys diff --git a/quest/m0/ci/ts-path-filter.md b/quest/m0/ci/ts-path-filter.md new file mode 100644 index 00000000000..7535bfd6441 --- /dev/null +++ b/quest/m0/ci/ts-path-filter.md @@ -0,0 +1,25 @@ +# [XS] TS validation runs whenever a crate it builds changes + +## Goal + +`.github/workflows/ts.yml` triggers on every workspace crate the TS harness +builds, and a lint fails Check when its path filter drifts from that set. + +## Plan + +`test/ts/run.sh` builds `moq-relay`, `moq-cli` and `moq-shaper` with their +default features. That build compiles workspace crates the filter misses, +found 2026-10-17 with `cargo tree` over those roots: `moq-clock`, +`moq-auth`, `moq-stats`, `moq-archive`, `moq-hls`, `moq-rtc`, `moq-rtmp` and +`moq-srt`. Optional crates such as `moq-transcode`, `moq-video` and +`moq-nvenc` are not compiled and must not trigger the workflow. +crates.io dependencies such as `web-transport-*` are already covered by the +`Cargo.lock` filter. + +Decided 2026-10-17 (maintainer): derive the set from Cargo in a lint that +runs in Check, rather than hand-listing paths that drift with the next +dependency. Resolve it for the harness's exact packages and features (the +activated closure), not the whole workspace graph, or unrelated changes will +trigger this expensive workflow. + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index 5c42f0cecc8..7fbba762284 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -79,6 +79,9 @@ lands before the cut. - [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native FILL_TIMEOUT bounds upstream waiting on drafts 18-22, alongside local content-age enforcement - [Browser follows a renamed track](/quest/m1/browser-rename-check.md) - a real browser keeps playing when `@moq/publish` renames a track after a codec change, while size-only changes keep their identity - [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` refuses a redefined rendition, and every Rust publisher mints a new rendition name for a decoder identity change and updates everything else in place +- [JSON duplicate keys](/quest/m1/json-duplicate-keys.md) - Rust and JS JSON decoders refuse duplicate keys in every frame kind +- [Catalog too new](/quest/m1/catalog-unsupported.md) - `@moq/watch` reports a distinct status when its player cannot read the catalog version +- [One hang catalog consumer](/quest/m1/catalog-consumer-fold.md) - `moq-mux` reads hang catalogs through `hang`'s consumer - [Broadcast archives](/quest/m1/archive/README.md) - SUBSCRIBE_ALL records every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for an opt-in `hls` field - [In-band auth](/quest/m1/auth/README.md) - a session tells its peer what it may publish and subscribe to, unions tokens presented in band, and fails loud on an out-of-scope publish - [IETF epochs](/quest/m1/ietf-epochs.md) - negotiate explicit broadcast identity on drafts 17-22, retaining unchanged paths and epochless clients on updated relays @@ -98,9 +101,14 @@ lands before the cut. - [Publish channel count](/quest/m1/publish-audio-channel-count.md) - forcing a channel count on an Audio.Capture stops costing the subscriber gaps of silence, and the media fixture's cold-start audio-tone flake is fixed at its cause - [Handshake deadline covers admission](/quest/m1/handshake-admission-deadline.md) - `listen.timeout` bounds a connection through auth admission on both listeners, replacing the hardcoded admission timeout - [Drain cleanup](/quest/m1/drain-cleanup.md) - sessions force-closed at the drain deadline record their lease end and bytes before the relay or CLI exits, while unfinished handshakes stay bounded +- [GOAWAY coverage](/quest/m1/goaway-coverage.md) - every IETF request stream handles GOAWAY, a closed control stream is a violation, and the draft-18 cutoff is honored - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust +- [Decode errors end the reader](/quest/m1/decode-errors.md) - a container decode error fails the reader in Rust and JS instead of reading as loss +- [Transcode finishes its group](/quest/m1/transcode-open-group.md) - `moq-transcode` never aborts a group a fetch's live loop opened - [Rust request credit](/quest/m1/rs-request-credit.md) - a moq-net request fails at once without stream credit instead of waiting - [qmux no-wait opens](/quest/m1/qmux-no-wait.md) - @moq/qmux rejects an over-limit create when waitUntilAvailable is false, like Chrome +- [Bufferbloat harness](/quest/m1/bufferbloat.md) - acceptance and performance checks impair paths with a rate limit and a deep queue, not random loss +- [Random-loss audit](/quest/m1/random-loss-audit.md) - every decision random-loss gates drove is kept, reworked, or reverted on shaped paths - [Timed-out requests clean up](/quest/m1/legacy-request-timeout-cleanup.md) - on drafts 14-16, Rust and JS send PUBLISH_NAMESPACE_DONE or UNSUBSCRIBE_NAMESPACE for a timed-out request before its local reset - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide - [Resume reorder](/quest/m1/resume-reorder.md) - a group whose stream header arrives after its successor's still resumes on failover; only a dropped or long-gone group fails diff --git a/quest/m1/bufferbloat.md b/quest/m1/bufferbloat.md new file mode 100644 index 00000000000..c8c64d673dc --- /dev/null +++ b/quest/m1/bufferbloat.md @@ -0,0 +1,53 @@ +# [M] Acceptance and performance run on bufferbloat, not random loss + +## Goal + +Every acceptance or performance (QoS) check that impairs a network does it +with a rate limit below the media bitrate behind a deep but finite queue, so +it congests and tail-drops like a real path. Random loss remains only where +a test proves correctness, such as retransmits happening. + +## Plan + +Decided 2026-10-17 (maintainer), and recorded in the root `AGENTS.md`: +bufferbloat does double duty, since its overflow drops packets too, so it is +the impairment for anything judged on quality, latency, or throughput. +Random loss judged that way pushed decisions toward surviving a failure mode +real paths do not show (see the [audit](/quest/m1/random-loss-audit.md)). + +Classify each random-loss user, move the acceptance and performance ones to +shaping, and leave correctness checks alone: + +- TS harness: `just test ts` gains `--rate` and `--queue` beside `--loss` + (decided 2026-10-17), passed to `moq-shaper`'s existing token bucket; its + 100 ms default queue is shallow for bufferbloat. The nightly + `--hrd --loss 1` and `--passthrough --loss 1` steps in `ts.yml`, + `passthrough.py --loss-ok`, and the `test/ts/README.md` lossy-path section + grade output quality, so they move to `--rate`/`--queue`; `--loss` stays + for correctness runs. +- Relay drills: the 5% profile every `::impaired` lane and the cluster + drills use (`drills.rs`, `test/drill/README.md`). Decide per drill + whether it proves correctness or judges behavior under impairment. +- `moq-quic`'s sim (`Link.loss`, the lossy proptests) and BBR3's + deterministic periodic-drop unit tests prove recovery is correct; they + stay unless they assert throughput or latency. + +`moq-shaper --loss` stays for correctness use. Earlier in the same day the +plan was to delete it; the maintainer then scoped the rule to acceptance and +performance. + +The TS run's oracle under a path narrower than the muxrate, decided +2026-10-17: the export stays T-STD conformant and continuous (PCR, +continuity counters, no partial PES), sheds whole groups rather than +stalling, and end-to-end latency stays bounded instead of growing with the +queue. No loss-percentage or hole-count thresholds. Pick the shaped rate and +queue depth from what the run must exercise (the queue must overflow), and +say why in the harness. + +Public API: none. Wire: none. + +## Related + +- [Random-loss audit](/quest/m1/random-loss-audit.md) - re-judges what random-loss tests drove, on this harness +- [Relay burst drill](/quest/m1/test-flakes-2/relay-burst-impaired.md) - owns the impaired burst drill's flake +- [TS recovery](/quest/m1/ts-reliability/recovery.md) - MoQ content-loss schedules, separate from network impairment diff --git a/quest/m1/catalog-consumer-fold.md b/quest/m1/catalog-consumer-fold.md new file mode 100644 index 00000000000..da7dbccc3f2 --- /dev/null +++ b/quest/m1/catalog-consumer-fold.md @@ -0,0 +1,25 @@ +# [S] One hang catalog consumer + +## Goal + +`moq-mux` reads hang catalogs through `hang`'s catalog consumer instead of +its own near-copy, so rendition and resolvability checks live in one place. + +## Plan + +`rs/moq-mux/src/catalog/hang/consumer.rs` and +`rs/hang/src/catalog/consumer.rs` both wrap +`moq_json::snapshot::Consumer>` with the same `new`, +`compressed`, `poll_next` (rendition and resolvability checks) and `next`. +The `moq-mux` copy adds only a `CatalogFormat` (hang or hangz) and the +`CatalogExt` bound. Fold that into `hang`'s consumer, or keep a thin +`moq-mux` adapter over it; pick whichever keeps `hang`'s public surface +smaller. Found by #5806. + +Public API: `moq-mux`'s hang catalog consumer may become a re-export or +adapter. Wire: none. + +## Related + +- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` catalog checks +- [Compressed tracks](/quest/m1/compress/README.md) - drops `catalog.json.z`, so the folded consumer need not keep the hangz format diff --git a/quest/m1/catalog-unsupported.md b/quest/m1/catalog-unsupported.md new file mode 100644 index 00000000000..7b487a389ea --- /dev/null +++ b/quest/m1/catalog-unsupported.md @@ -0,0 +1,17 @@ +# [S] A viewer sees that its player is too old for the catalog + +## Goal + +When `@moq/watch` reads a catalog whose version it does not support, the +broadcast reports a distinct status that says the player needs an update, +instead of going "offline" (fetched catalog) or staying "loading" (inline). + +## Plan + +After #5806, an unsupported catalog version is only logged. Decided +2026-10-17 (maintainer): add a distinct broadcast status rather than reusing +the error path, so the UI can tell a viewer to update instead of implying +the broadcast is down. Mirror the shape in Rust consumers' error so native +players can say the same. + +Public API: a new broadcast status value in `@moq/watch`. Wire: none. diff --git a/quest/m1/decode-errors.md b/quest/m1/decode-errors.md new file mode 100644 index 00000000000..9a54c249f94 --- /dev/null +++ b/quest/m1/decode-errors.md @@ -0,0 +1,35 @@ +# [S] A container decode error ends the reader in Rust and JS + +## Goal + +A container payload that fails to decode ends the media reader with that +error, the same way in Rust and JS. Only a transport abort or eviction of a +group is skipped as a gap. Nothing logs a decode error and keeps reading. + +## Plan + +Rust already does this in the main read path (`moq-mux` container consumer +ends the reader on a payload decode error), with two holes: + +- JS `@moq/hang`'s container consumer marks the group truncated on any + error, keeps the frames it decoded, and lets the rest reach `spawn`, which + only logs. The reader never fails, so malformed media reads as loss. +- Rust's `poll_min_timestamp` lookahead returns the error, but its callers + (startup, the max-end scan, the rewind check) discard it. The bad frame is + already consumed, so a malformed object at the head of a group then reads + as a clean or short end. + +Decided 2026-10-17 (maintainer): a quest of its own in m1, since both are +silent data loss. Found while reviewing #5815; the JS behavior was declined +there as out of scope. + +Each fix lands with a regression test that feeds a malformed object (head of +group and mid-group) and asserts the reader's error, in both languages. + +Public API: JS readers can now reject with a decode error where they used to +truncate. Wire: none. + +## Related + +- [Container reads drop backfill](/quest/m1/read-backfill.md) - the other Rust/JS container-consumer parity gap +- [Dropped sources](/quest/m1/dropped-sources.md) - consumers see the producer's real error on every end diff --git a/quest/m1/goaway-coverage.md b/quest/m1/goaway-coverage.md new file mode 100644 index 00000000000..caeee3bdf13 --- /dev/null +++ b/quest/m1/goaway-coverage.md @@ -0,0 +1,36 @@ +# [M] GOAWAY reaches every IETF request stream + +## Goal + +On moq-transport, every request stream in Rust and JS (publisher and +subscriber side, including FETCH, TRACK_STATUS and SUBSCRIBE_TRACKS) handles +a GOAWAY the way SUBSCRIBE does, a control stream that ends or resets is a +PROTOCOL_VIOLATION, and draft-18's Request ID cutoff answers requests past it +with GOING_AWAY. + +## Plan + +Found by #5800 (2026-10-17 state of main): + +- Only SUBSCRIBE and SUBSCRIBE_NAMESPACE on the publisher side read GOAWAY + on their request streams (Rust `request_stream`, JS `nextMessage` and + `namespaceUpdate`). FETCH, TRACK_STATUS, SUBSCRIBE_TRACKS, and every + subscriber-side request stream end the request on a GOAWAY instead. +- A control stream FIN is treated as a clean end: Rust's draft-17+ GOAWAY + reader returns Ok, the draft 14-16 adapter tears the session down with no + code, and JS closes with no code. Draft 21 section 6.3 makes it a + PROTOCOL_VIOLATION; check the SETUP STOP_SENDING ordering too. +- Draft 18's GOAWAY Request ID is only parity-checked. Requests at or past + the cutoff never get GOING_AWAY, and our own GOAWAY sends a fixed 0 or 1. +- Check JS for a duplicate-GOAWAY race. + +Decided 2026-10-17 (maintainer): m1, beside drain cleanup. Draft 17 stays +low priority; fix it where the shared code makes it free. + +Public API: none expected. Wire: closer conformance on drafts 14-22; no new +fields. + +## Related + +- [Drain cleanup](/quest/m1/drain-cleanup.md) - what a drain does to sessions at its deadline +- #5798 - the legacy drafts' request handling diff --git a/quest/m1/json-duplicate-keys.md b/quest/m1/json-duplicate-keys.md new file mode 100644 index 00000000000..1ec59a374e7 --- /dev/null +++ b/quest/m1/json-duplicate-keys.md @@ -0,0 +1,28 @@ +# [S] JSON decoders refuse duplicate keys in Rust and JS + +## Goal + +Every `moq-json` and `@moq/json` decoder (snapshot, delta, stream, and +window) refuses an object with a duplicate key, so a catalog or data frame +can never mean two things to two readers. + +## Plan + +Today Rust refuses duplicates in deltas only; snapshots go through +`serde_json` into `Value`, which keeps the last key. JS uses `JSON.parse` +everywhere, which also keeps the last key. #5806 adds a typed refusal for a +duplicate catalog `version` only. + +Decided 2026-10-17 (maintainer): refuse everywhere, in both languages. For +JS, prefer a maintained parser over hand-rolling: `@humanwhocodes/momoa` +(maintained, widely used) keeps duplicate members in its AST, so walk it; +`json-dup-key-validator` is unmaintained since 2021. Measure the parse cost +against `JSON.parse` on catalog-sized and stats-sized frames, since every +reader pays it. + +Public API: decoders return a new error on duplicate keys. Wire: frames +with duplicate keys, previously accepted, are refused. + +## Related + +- #5806 - adds the duplicate `version` refusal diff --git a/quest/m1/random-loss-audit.md b/quest/m1/random-loss-audit.md new file mode 100644 index 00000000000..c480ce3aa1b --- /dev/null +++ b/quest/m1/random-loss-audit.md @@ -0,0 +1,52 @@ +# [M] Audit decisions made to pass random-loss tests + +## Goal + +Every acceptance or performance decision a random-loss test drove (code, +defaults, harness choices, CI steps, and quests) is re-judged on the +bufferbloat harness and kept, reworked, or reverted, each with a recorded +verdict and its evidence. + +## Plan + +Decided 2026-10-17 (maintainer): random loss may prove correctness, but it +is not how real paths fail, so anything tuned to survive it as a quality or +performance target may be wrong. Scope is every random-loss-driven +decision, not only the 10% TS work. Small reverts land in the audit's PR; +anything bigger becomes its own quest. Verdicts are measured on the +[bufferbloat harness](/quest/m1/bufferbloat.md), not argued. + +Leads found 2026-10-17 (no production default or congestion threshold was +changed for random loss; `LOSS_THRESH` is still 2%): + +- `quest/m2/quic-bbr-natural-drain.md` targets "BBR's response to random + loss" on a 10% profile and asks whether BBR should tell random loss from + congestion. The main live risk: it would weaken BBR's loss response. Its + premise came from #5805 moving 10% loss out of the T-STD epic. +- #4645's fixed-delay jitter buffer and skip design was partly justified by + the 10% netem rig (#4613); re-check the skip budget and acquire wait. +- The BBR fork fixes (#4193, noq 1.3.1) were accepted on a 2% random-loss + netem check; the fixes are correctness fixes, but re-check the evidence. +- Harness choices from #5805 and #5856: nightly `--hrd` because + `--headroom` loses audio at 1% loss, WebSocket fallback disabled under + `--loss`, a run that fails if the shaper dropped nothing, and + `passthrough.py --loss-ok`. +- Decided 2026-10-17 (maintainer): the audit decides whether + `quic-bbr-natural-drain.md` and `m3/quic-gcc.md` survive, since their + premise rests on random loss. +- Other quests only list random loss among their measurement inputs: + `quic-probe.md` (with noq#811 and noq#476), `quic-sim.md`, + `quic-deadline.md` and `quic-careful-resume.md` ("the impaired path + profile"), `m3/cut-through/bench.md`, and + `m1/ts-reliability/recovery.md`'s network impairment arm. Their goals + stand; the audit only moves any acceptance or performance input among + them to bufferbloat. + +Record each verdict (decision, origin PR, keep/rework/revert, evidence) in +the PR description. + +Public API: depends on the verdicts. Wire: none expected. + +## Required + +- [Bufferbloat harness](/quest/m1/bufferbloat.md) - verdicts are measured on shaped paths diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index 80209fa9c89..d6cfa7a5f63 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -40,6 +40,10 @@ Public API: none. Wire: none. - [uring teardown close](/quest/m1/test-flakes-2/uring-teardown-close.md) - moq-uring's published-close teardown test holds under load, fixed at its cause - [Relay burst drill](/quest/m1/test-flakes-2/relay-burst-impaired.md) - the impaired transport-peer burst drill never aborts its subscription while the link is up - [BBR marking seed](/quest/m1/test-flakes-2/bbr-marking-seed.md) - the BBR marking test seeds its probe schedule, replays a failing seed, and its rare drop is fixed at the cause +- [gst redial port](/quest/m1/test-flakes-2/gst-redial-port.md) - the moq-gst redial tests never lose their relay's port to `AddrInUse` +- [js/net local close](/quest/m1/test-flakes-2/js-local-close.md) - the draft-17 locally closed track test holds under load +- [WASM refused track](/quest/m1/test-flakes-2/wasm-refused-track.md) - the refused-track case passes and fails on its refusal, not a timeout +- [Drill shaper failure](/quest/m1/test-flakes-2/drill-shaper-failure.md) - a drill whose shaper stopped says so directly ## Related diff --git a/quest/m1/test-flakes-2/drill-shaper-failure.md b/quest/m1/test-flakes-2/drill-shaper-failure.md new file mode 100644 index 00000000000..aa8f8ebf233 --- /dev/null +++ b/quest/m1/test-flakes-2/drill-shaper-failure.md @@ -0,0 +1,18 @@ +# [XS] A drill names a shaper that stopped + +## Goal + +When a relay drill's shaper stops forwarding, the failure says so directly +instead of surfacing later as a status timeout. + +## Plan + +`Drop for Path` in `rs/moq-relay/tests/drills.rs` prints only the shaper's +counters; only `verify()` reports the shaper's error. Surface that error +when a drill fails. Found while chasing #5796. + +Public API: none. Wire: none. + +## Related + +- [Relay burst drill](/quest/m1/test-flakes-2/relay-burst-impaired.md) - another impaired drill failure diff --git a/quest/m1/test-flakes-2/gst-redial-port.md b/quest/m1/test-flakes-2/gst-redial-port.md new file mode 100644 index 00000000000..c538883cc30 --- /dev/null +++ b/quest/m1/test-flakes-2/gst-redial-port.md @@ -0,0 +1,18 @@ +# [XS] moq-gst redial tests keep their relay's port + +## Goal + +The moq-gst redial tests (`a_refused_redial_posts_a_bus_error_after_playback`, +`relay_loss_reconnects_after_the_default_budget`) never fail with +`AddrInUse` under load, fixed at its cause. + +## Plan + +Seen at least three times on #5798, #5803, #5815 and #5820 runs. The tests +stop their test relay (`Relay::bind`/`stop` in +`rs/moq-gst/src/source/imp.rs`) and rebind the freed ephemeral port to bring +it back; on a loaded host a parallel test can take that port in between. +The cause is unconfirmed: reproduce it first. Never add a retry around the +bind. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/js-local-close.md b/quest/m1/test-flakes-2/js-local-close.md new file mode 100644 index 00000000000..4fd5fb2d34e --- /dev/null +++ b/quest/m1/test-flakes-2/js-local-close.md @@ -0,0 +1,15 @@ +# [XS] js/net's locally closed track test holds under load + +## Goal + +`js/net`'s "ends a locally closed track cleanly" integration test +(`runSessionDeath(..., true)`, draft 17) passes reliably, fixed at its +cause. + +## Plan + +Failed once in a full `bun test` run under load on #5803 and passed 6/6 +alone. No root cause yet: reproduce it under load first, then decide whether +the product or the test races. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/wasm-refused-track.md b/quest/m1/test-flakes-2/wasm-refused-track.md new file mode 100644 index 00000000000..9e7605e1198 --- /dev/null +++ b/quest/m1/test-flakes-2/wasm-refused-track.md @@ -0,0 +1,16 @@ +# [XS] The WASM refused-track case fails fast + +## Goal + +`test/wasm`'s "refuses a track the publisher does not serve" case passes +reliably and, when it does fail, fails on the refusal it expects rather than +the 30 s case timeout. + +## Plan + +Timed out once in 40 runs, including on docs-only #5801 (run 39040909046). +On IETF and lite-02 the refusal only shows as no group arriving, so a hang +looks the same as a pass that never resolves. Reproduce it, find which side +stalls, and give the case an observable refusal where the protocol has one. + +Public API: none. Wire: none. diff --git a/quest/m1/transcode-open-group.md b/quest/m1/transcode-open-group.md new file mode 100644 index 00000000000..44bd7e008a5 --- /dev/null +++ b/quest/m1/transcode-open-group.md @@ -0,0 +1,28 @@ +# [S] moq-transcode finishes the group it is transcoding + +## Goal + +When demand for a transcoded rendition ends mid-group, `moq-transcode` +finishes the open output group before leaving its source session. A later +fetch of that sequence gets the complete group, never an aborted one. + +## Plan + +Since #5820 a fetch counts as demand for its whole lifetime, so a fetch +starts the rung's live loop. When the fetch ends, the loop sees the track +unused and aborts its open output group with `Error::Cancel`; a later fetch +of that sequence hits the aborted group. #5820's tests worked around it by +holding queries open. + +Decided 2026-10-17 (maintainer): finish the group, then leave. A group name +always means the same content, so the published group must be the whole +group for every later reader, not whatever was transcoded before demand +went away. Rejected: aborting and re-transcoding on the next fetch (two +contents under one name), and serving fetches on a separate path (more +machinery for the same contract). + +Drop the held-query workaround in the tests once the loop finishes groups; +a regression test fetches a sequence after an earlier fetch that started +the live loop has ended. + +Public API: none. Wire: none. diff --git a/quest/m1/ts-reliability/README.md b/quest/m1/ts-reliability/README.md index 8f67b26d85d..2300c371410 100644 --- a/quest/m1/ts-reliability/README.md +++ b/quest/m1/ts-reliability/README.md @@ -99,6 +99,7 @@ remain with their current quests. ## Required +- [Deterministic grading](/quest/m1/ts-reliability/grader-determinism.md) - the same input gets the same start and the same verdict - [Plan: semantic fidelity](/quest/m1/ts-reliability/fidelity.md) - explicit profiles and PMT lifecycle contracts produce implementation quests - [Whole-PES import](/quest/m1/ts-reliability/pes-import.md) - private PES travels whole at its source placement, a missing PTS stays missing, and the `pes` framing is deleted - [Section placement](/quest/m1/ts-reliability/section-placement.md) - sections share the PCR placement clock instead of the video PTS diff --git a/quest/m1/ts-reliability/grader-determinism.md b/quest/m1/ts-reliability/grader-determinism.md new file mode 100644 index 00000000000..2a1b19dccbd --- /dev/null +++ b/quest/m1/ts-reliability/grader-determinism.md @@ -0,0 +1,28 @@ +# [S] TS validation gives the same verdict for the same input + +## Goal + +The TS export picks the same start and anchor for the same input, and the +harness's probes read the whole capture, so a TS validation verdict depends +only on what was sent. + +## Plan + +Two sources of run-to-run variance, found by #5810 and #5854: + +- `Export::fill` walks its tracks in `HashMap` order, and a track's fill + stops once the jitter buffer is full. Jitter acquisition and the video + start can complete partway through a pass, before other tracks' available + frames are read, so the start keyframe varies (0 ms vs 600 ms seen). The + export should see every track's available frames before it picks the + start; a deterministic iteration order alone only hides the dependency. +- `test/ts/decode.py` runs `ffprobe` with default probing (5 MB). #5854 fixed + `holes.py`, where the first MP2 PES 5.9 MB in made ffprobe guess MP3 and a + repeated timestamp read as a hole; `decode.py` has the same exposure. + +Decided 2026-10-17 (maintainer): one quest for the grader, with the CI path +filter as a separate [m0 quest](/quest/m0/ci/ts-path-filter.md). The harness +never gates on a random-loss percentage (see +[bufferbloat](/quest/m1/bufferbloat.md)). + +Public API: none. Wire: none. diff --git a/quest/m2/README.md b/quest/m2/README.md index c4b934a361d..cff538150a2 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -125,7 +125,9 @@ ranked just below deterministic TS legs, which it requires. - [Browser media report](/quest/m2/rs2ts-browser-report.md) - compare Rust-backed and handwritten @moq/net through full media workloads, supplementing the pre-adoption comparisons - [mTLS on tls://](/quest/m2/tls-listener-mtls.md) - a `tls://` listener can identify a cluster peer by its client certificate - [Link quality](/quest/m2/link-quality.md) - a radio link's cost follows its measured quality without flapping routes +- [Ring drain GOAWAY](/quest/m2/uring-ring-drain.md) - the ring sends moq-transport 17+ sessions their drain GOAWAY - [Relay io_uring packages](/quest/m2/relay-io-uring-package.md) - Linux relay packages ship io_uring once the ring is on par with tokio +- [moq-transport machines](/quest/m2/uring-transport-machines.md) - moq-transport polls named state machines, retiring `Boxable` and `moq_uring::transport::Local` - [Opus implementation](/quest/m2/audio-opus-backend.md) - compare Opus codec quality, CPU, build cost, and the loss recovery each backend offers - [Latency ledger](/quest/m2/latency-ledger.md) - a viewer reports its share of playback delay stage by stage: jitter buffer, decode, render, and device - [Media Foundation decode](/quest/m2/audio-decode-mediafoundation.md) - Windows decodes HE-AAC, multichannel AAC, and what else the MFTs offer diff --git a/quest/m2/e2ee/README.md b/quest/m2/e2ee/README.md index ded8d48b7ea..db42791ecbc 100644 --- a/quest/m2/e2ee/README.md +++ b/quest/m2/e2ee/README.md @@ -52,6 +52,11 @@ Decided with the maintainer on 2026-10-17: physical names are a one-way hash, so - [Rust protected publisher seams](/quest/m2/e2ee/rust-publish.md) - Rust media and catalog publishers accept opaque physical names and emit no plaintext semantic catalog in E2EE mode +- [Dynamic names](/quest/m2/e2ee/dynamic-names.md) - a reviewed design for + answering requests for undeclared track names, split into implementation + quests; #5876 waits on it +- [Refuse a re-created track](/quest/m2/e2ee/reopen.md) - a name the producer already carried fails loudly +- [Encrypted consumer shape](/quest/m2/e2ee/consumer.md) - one consumer shape, the producer's mirror, shared by browser, CLI, and FFI - [Encrypted browser components](/quest/m2/e2ee/browser.md) - browser publish and watch components handle encrypted audio and video without leaking credentials into browser-owned surfaces @@ -61,9 +66,6 @@ Decided with the maintainer on 2026-10-17: physical names are a one-way hash, so - [Cross-language encrypted proof](/quest/m2/e2ee/interop.md) - browser TypeScript and native Rust exchange encrypted media both ways, with relays forwarding and metering blind -- [Dynamic names](/quest/m2/e2ee/dynamic-names.md) - a reviewed design for - answering requests for undeclared track names, split into implementation - quests; #5876 waits on it ## Closes diff --git a/quest/m2/e2ee/browser.md b/quest/m2/e2ee/browser.md index 0c716be897f..d8b299abfd2 100644 --- a/quest/m2/e2ee/browser.md +++ b/quest/m2/e2ee/browser.md @@ -13,3 +13,7 @@ Existing plaintext operation remains the default; an explicit application creden - Derive the opaque broadcast name from the semantic one, discover the newest epoch under that prefix, derive the encrypted catalog name from that generation, decrypt it before existing Hang selection, then subscribe to its opaque media and timeline tracks. Publish every catalog representation and semantic track name through the same protected naming contract. - Refuse the stats track and the `echo` attribute while a credential is set: they would publish rendition IDs and per-track counters in plaintext beside the protected broadcast. This quest owns the refusal (decided 2026-10-08), since the stats quests have no encrypted mode to refuse until this credential exists. - Cover camera/microphone publication, the stats and `echo` refusal, audio/video playback, late subscription, WebCrypto backpressure through awaited writes, a restarted publisher under a new epoch, bad-group termination, and bad-datagram events in browser tests. + +## Required + +- [Encrypted consumer shape](/quest/m2/e2ee/consumer.md) - watch uses the shared consumer instead of its own decryption diff --git a/quest/m2/e2ee/cli.md b/quest/m2/e2ee/cli.md index 6e8a3fa13fb..44b2de150b3 100644 --- a/quest/m2/e2ee/cli.md +++ b/quest/m2/e2ee/cli.md @@ -18,3 +18,4 @@ Existing plaintext commands remain the default; an explicit application credenti ## Required - [Rust protected publisher seams](/quest/m2/e2ee/rust-publish.md) - prevents native convenience publishers from leaking semantic names or catalogs +- [Encrypted consumer shape](/quest/m2/e2ee/consumer.md) - playback uses the shared consumer instead of its own decryption diff --git a/quest/m2/e2ee/consumer.md b/quest/m2/e2ee/consumer.md new file mode 100644 index 00000000000..55ab44d3c8d --- /dev/null +++ b/quest/m2/e2ee/consumer.md @@ -0,0 +1,26 @@ +# [M] One encrypted consumer shape for every player + +## Goal + +Encrypted playback has one consumer shape, the mirror of the protected +producer, that the browser, the CLI, and the FFI bindings share, so each +player does not reinvent decryption and physical-name mapping. + +## Plan + +#5822's CLI plan names the consumer as "the consuming mirror of that +producer in `moq-mux`", but nothing gives it a shape the browser and FFI +can mirror. Mock up the Rust shape first, then mirror the names in JS and +the bindings. Kept in m2 (decided 2026-10-17). + +The moq-ffi consumer follows the `rs/moq-ffi` row of the root `AGENTS.md` +cross-package table: wrappers, C and C++, OBS where it plays, and the +language docs land with it, or the FFI part waits for a quest that does. + +Public API: a new e2ee consumer in `moq-mux`, `@moq/e2ee`, and moq-ffi with +its wrappers. Wire: none. + +## Required + +- [Rust protected publisher seams](/quest/m2/e2ee/rust-publish.md) - the producer this mirrors +- [Dynamic names](/quest/m2/e2ee/dynamic-names.md) - the approved naming design can change the physical-name mapping this fixes diff --git a/quest/m2/e2ee/reopen.md b/quest/m2/e2ee/reopen.md new file mode 100644 index 00000000000..3f916a905ef --- /dev/null +++ b/quest/m2/e2ee/reopen.md @@ -0,0 +1,21 @@ +# [XS] A re-created encrypted track is refused, not dropped + +## Goal + +Publishing a track under a name the e2ee producer already carried fails +loudly instead of being silently left out. + +## Plan + +`moq-mux`'s e2ee producer carries each name at most once, and `moq-e2ee` +never releases a claim because reopening would restart its nonce counters, +so a re-created track today is just not carried. Decided 2026-10-17 +(maintainer): refuse it. A track name always means the same content, so a +restarted encoder publishes a new name or a new broadcast. + +Public API: a new error from the e2ee producer. Wire: none. + +## Required + +- [Rust protected publisher seams](/quest/m2/e2ee/rust-publish.md) - the producer that claims names +- [Dynamic names](/quest/m2/e2ee/dynamic-names.md) - the approved naming design can change the physical-name mapping this fixes diff --git a/quest/m2/perf/README.md b/quest/m2/perf/README.md index 117f545465e..22e0f4e7d60 100644 --- a/quest/m2/perf/README.md +++ b/quest/m2/perf/README.md @@ -71,6 +71,7 @@ optimization. ## Required - [kio channel contention](/quest/m2/perf/kio-channel-contention/README.md) - workers spend less time blocked on kio channel-state mutexes (up to 215% of worker CPU on chat at 16 tokio workers), via a version counter, per-thread spikes, and an append-only log +- [Relay allocations](/quest/m2/perf/relay-allocs.md) - fewer allocations per delivered frame, starting with per-stream session clones - [Run to quiescence](/quest/m2/perf/uring-quiescence.md) - a received packet's reply is staged in the same turn, under a pass and train budget that keeps the fairness rule - [Ingest batch](/quest/m2/perf/ingest-batch.md) - relay ingest pays one lock, wake, and clock read per burst of whole frames instead of per frame - [Shard by default](/quest/m2/perf/shard-default.md) - a saturating shard sweep decides whether an unset `runtime.workers` runs one worker per core on Linux diff --git a/quest/m2/perf/relay-allocs.md b/quest/m2/perf/relay-allocs.md new file mode 100644 index 00000000000..125894f5c50 --- /dev/null +++ b/quest/m2/perf/relay-allocs.md @@ -0,0 +1,27 @@ +# [M] The relay allocates less per delivered frame + +## Goal + +The fanout relay's heap churn per delivered frame drops measurably, starting +with the per-stream `Session::clone` and `SendStream::poll_write` sites, and +the heap profile tracks it so a regression shows. + +## Plan + +Measured by #5830's heap profile (`just bench-profile heap fanout-heavy 4`): +about 1.08M allocations/s, roughly 70 per delivered frame. Top churn: +`SendStream::poll_write` 41.5%, the web transport `Session::clone` 15.9%, +moq-net `SendStream` poll 10.3%, `Session` poll 8.2%, +`group::Consumer::with_recover` 5.9%, kio `Tasks` 5.5%. + +`moq-tokio`'s `poll_open_uni` clones the session and boxes a future per +opened stream, and the web `Session` holds header `Vec`s and optional +connect request/response, so each clone allocates. Start there. + +Validate #5830's CPU profile mode with a real capture before relying on its +stacks. Sweep the fanout benchmark over publishers and subscribers so a cost +that grows with the table shows as a slope. + +Decided 2026-10-17 (maintainer): m2, under perf. + +Public API: none expected. Wire: none. diff --git a/quest/m2/relay-io-uring-package.md b/quest/m2/relay-io-uring-package.md index ed006de0604..84344fe5c9e 100644 --- a/quest/m2/relay-io-uring-package.md +++ b/quest/m2/relay-io-uring-package.md @@ -34,4 +34,5 @@ Public API: none. Wire: none. ## Required +- [Ring drain GOAWAY](/quest/m2/uring-ring-drain.md) - closes are delivered on the ring for moq-transport too - [One port on the io_uring workers](/quest/m2/uring-demux.md) - a one-port config is served on the ring too diff --git a/quest/m2/uring-ring-drain.md b/quest/m2/uring-ring-drain.md new file mode 100644 index 00000000000..89e81ba7169 --- /dev/null +++ b/quest/m2/uring-ring-drain.md @@ -0,0 +1,16 @@ +# [XS] The ring drains moq-transport sessions with GOAWAY + +## Goal + +A test proves the io_uring workers send moq-transport 17+ sessions their +drain-window GOAWAY, the same as the tokio runtime. + +## Plan + +Today only `shutdown_signal.rs` covers the drain GOAWAY, through the shared +runtime. Extend the ring's drain test (`uring_workers_drain_on_the_trigger` +in `rs/moq-relay/tests/integration/runtime_uring.rs`, from #5821) to a +moq-transport 17+ client. Closes delivered on the ring is part of the +packaged-relay parity bar. + +Public API: none. Wire: none. diff --git a/quest/m2/uring-transport-machines.md b/quest/m2/uring-transport-machines.md new file mode 100644 index 00000000000..e4664aec709 --- /dev/null +++ b/quest/m2/uring-transport-machines.md @@ -0,0 +1,24 @@ +# [L] moq-transport runs as named state machines + +## Goal + +moq-transport sessions poll named state machines instead of boxed futures, +so the io_uring workers drive them without `transport::poll::Boxable` or +`moq_uring::transport::Local`'s per-call thread check. + +## Plan + +#5821 serves moq-transport on the ring by wrapping its transport in +`moq_uring::transport::Local` (renamed from `Pinned`), which checks the +thread on every call because the ietf path requires `Boxable`. Decided with +#5821 (maintainer): ship `Local` public now and convert to named machines +later, retiring both. Kept in m2 (decided 2026-10-17). + +Measure the per-call cost `Local` adds on the ring before and after, on the +relay fanout benchmark swept over sessions and subscribers. + +Public API: `moq_uring::transport::Local` and `Boxable` go away. Wire: none. + +## Related + +- [Relay io_uring packages](/quest/m2/relay-io-uring-package.md) - not blocked on this; `Local` already gives parity