diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index ef1a68b41bc..b845bc20f0a 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -154,6 +154,7 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md) - write secrets and publisher identities reach only protected branches and tags, never a pushed branch - [Land main through quest land](/quest/m0/ci/land.md) - a host daemon tests small batches of queued PRs once and lands each as its tested squash commit - [A cache error is a miss](/quest/m0/ci/nix-cache-5xx.md) - a transient `nix.moq.dev` 5xx no longer fails `just ci` +- [Every io_uring test in the io-uring group](/quest/m0/ci/io-uring-test-group.md) - the parallel tests of one `just ci` stop exhausting the shared `RLIMIT_MEMLOCK`, and a too-low limit fails clearly - [Cheap checks skip the queue](/quest/m0/ci/gate-quick.md) - condition: quest ships `quest gate --quick`; then cheap modules wait at most a minute - [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - `just check` and `just test` each finish within a worker's 5-minute prompt cache, admission wait included - [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out diff --git a/quest/m0/ci/io-uring-test-group.md b/quest/m0/ci/io-uring-test-group.md new file mode 100644 index 00000000000..2b3a88b5a08 --- /dev/null +++ b/quest/m0/ci/io-uring-test-group.md @@ -0,0 +1,21 @@ +# [XS] Every io_uring test runs in the nextest io-uring group + +## Goal + +Every test that opens an io_uring, including the relay's `wire_stats::uring_*`, `peer_limits::uring_*`, and `refusals::uring_*` integration tests and the `bitrate_caps::uring_*` tests [moq-dev/moq#7535](https://github.com/moq-dev/moq/pull/7535) adds, runs inside the `io-uring` nextest test group, so the tests of one `just ci` running in parallel no longer exhaust the user's shared `RLIMIT_MEMLOCK`. The group caps rings only within one nextest invocation; separate invocations still share the budget, as `rs/moq-uring/README.md` documents, and this quest does not change that. An io_uring test that still hits too low a limit fails with a message naming `RLIMIT_MEMLOCK` and how to raise it, not a generic setup error or a timeout. + +## Plan + +Found by [moq-dev/moq#7535](https://github.com/moq-dev/moq/pull/7535) and [moq-dev/moq#7211](https://github.com/moq-dev/moq/pull/7211): #7535's io_uring tests passed serially but could hit the shared limit in parallel, and #7211's `just ci` failed once in `uring_peer_session_widens_quic_limits` on `RLIMIT_MEMLOCK`. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m0, since each such failure fails a whole merge-gate batch. + +- The group's filter in `.config/nextest.toml` names only `runtime_uring::` and `embed::.*uring` in `moq-relay::integration`. Match every io_uring test instead, for example `test(/uring/)` in that binary plus moq-relay's own unit tests that start ring workers, so a new test joins by its name. Check that the inventory of uring tests (`rg 'fn uring_' rs/moq-relay`) all land in the group with `cargo nextest show-config test-groups`, which lists each group's members; plain `cargo nextest list` shows no group. +- moq-uring already names the limit on a ring's `ENOMEM` (`rs/moq-uring/src/error.rs`). Check that this error reaches the failing test's output on the relay paths (worker startup, `start_relay`) instead of being wrapped or swallowed; if a test sees only a timeout, surface the startup error there. +- Keep the group's `max-threads` unless measured otherwise; the limit is shared with other checkouts, which `rs/moq-uring/README.md` documents. + +Public API: none. Wire: none. + +## Related + +- [More tests under load](/quest/m0/test-flakes-2/README.md) - the flake epic this would otherwise land in diff --git a/quest/m1/README.md b/quest/m1/README.md index 5d548e00383..f89845fb6ed 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -120,6 +120,12 @@ contradict the code. Its other findings already had owners or contradicted settled decisions (archives break in place; OBS video-only output stays removed). +Added 2026-11-01 (maintainer, follow-ups from the 2026-11-01 /quest-afk run): +the shared pacer's overshoot under 2 Mbit/s with the BBR3 retune it needs, +since BBR3 is the default controller the cut ships; a truncated MKV that +left its tracks open; and, under release performance, a bisect of the relay's +per-frame CPU growth since #5926. + ## Required - [Address groups by timestamp](/quest/m1/time-addressing/README.md) - group start timestamps replace sequence numbers in the moq-net API and lite-07 wire, while old subscribers, old publishers, and published JS clients keep working @@ -169,6 +175,7 @@ removed). - [Interop coverage](/quest/m1/interop-coverage/README.md) - the riskiest moq-transport features moq-net exposes get cross-implementation tests over drafts 18 and 22 - [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - A/A and deliberately slowed Nightly runs set the paired comparison's threshold, so its regression alerts can be trusted - [Probe scenario seeds randomize the media's start](/quest/m1/sim-media-offset.md) - each seed draws its own media start offset against the handshake and competitor, so probe budgets are not judged against one shared timing alignment +- [The shared pacer holds its rate under 2 Mbit/s](/quest/m1/pacer-low-rate.md) - a rate-paced controller stops sending up to twice its pacing rate at low rates, landed with a BBR3 retune that keeps the probe budgets - [HLS and DASH segment conformance](/quest/m1/hls-conformance.md) - nightly ffprobe checks that advertised timing, sync flags, and bytes match the media served - [Unbounded groups](/quest/m1/unbounded-groups/README.md) - groups and frames lose their size caps; a group keeps a 32 MB tail, and readers behind it replay through paced flights or refresh - [Legacy lite close drain](/quest/m1/lite-legacy-close-drain.md) - a lite-01..06 graceful close never cuts a final group the subscriber has not read @@ -180,6 +187,7 @@ removed). - [HLS export follows a resolution change](/quest/m1/hls-reinit-same-codec.md) - a decoder-identity change under the same codec string starts a new init and segment tag - [A JS reader finishes after a clean close](/quest/m1/js-close-range-finish.md) - a range ending below the cache delivers what's held and then ends - [fMP4 decode times keep increasing](/quest/m1/fmp4-bframe-decode-time.md) - fragments shorter than a GOP of B-frame video never step `tfdt` backwards +- [A truncated MKV ends its tracks at finish](/quest/m1/mkv-truncated-finish.md) - `finish()` refusing truncated input still ends the tracks, so subscribers stop waiting - [moq-quic deletes its remaining 0-RTT code](/quest/m1/quic-zero-rtt-leftovers.md) - only the code that drops a 0-RTT packet remains, with a test - [Gate the no-em-dash rule](/quest/m1/em-dash-gate.md) - `just check` fails on an em dash in any tracked text file diff --git a/quest/m1/mkv-truncated-finish.md b/quest/m1/mkv-truncated-finish.md new file mode 100644 index 00000000000..a11ea1e5c73 --- /dev/null +++ b/quest/m1/mkv-truncated-finish.md @@ -0,0 +1,19 @@ +# [XS] A truncated MKV ends its tracks at finish + +## Goal + +When the MKV importer's `finish()` refuses truncated input (`Mkv(Truncated)`), it still ends its tracks with what was accepted, as it does after a refused `decode`, so a subscriber's `read()` returns instead of waiting until the importer is dropped. `finish()` still returns the error, and the frame cut short still fails its group with the reason. + +## Plan + +Found by the verification in [moq-dev/moq#7515](https://github.com/moq-dev/moq/pull/7515), which fed the live `opus.webm` truncated at every length and with every byte flipped: 1462 truncation lengths and 29 byte flips ended with tracks neither finished nor aborted after `finish()` returned `Truncated`, at every chunk size. Callers that drop the importer after an error were unaffected. + +In `Import::finish` (`rs/moq-mux/src/container/mkv/import.rs`, about lines 838-852), a failed `drain_eof` aborts only the streaming block's group and returns early, skipping the per-track finish. A `decode` that already failed (`self.failed`) skips `drain_eof` and still reaches the track loop, so it ends its tracks. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m1, a small fix in an importer the cut ships. End the tracks on the truncation path the way a refused `decode` already does (`decode`'s docs: "`finish` ends the tracks with what was accepted"), so the two refusals behave alike. #7515 also floated aborting every track with the truncation error instead; matching the refused `decode` keeps one rule and keeps the groups that did arrive. Test with #7515's harness shape: a truncated `opus.webm` whose subscriber's read ends after `finish()` while the importer is still held. + +Public API: none. Wire: none. + +## Related + +- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - could assert that every track ends after `finish()`, whatever the input diff --git a/quest/m1/pacer-low-rate.md b/quest/m1/pacer-low-rate.md new file mode 100644 index 00000000000..7ead477b80f --- /dev/null +++ b/quest/m1/pacer-low-rate.md @@ -0,0 +1,23 @@ +# [S] The shared pacer holds its rate under 2 Mbit/s + +## Goal + +Under 2 Mbit/s, a rate-paced controller on moq-quic (BBR3 today, and any candidate that sets a pacing rate) sends at its pacing rate, not up to twice it. BBR3 is retuned in the same PR so the nightly probe scenarios, `competing_cubic` included, stay within their late-frame and overhead budgets. + +## Plan + +Found by [moq-dev/moq#7546](https://github.com/moq-dev/moq/pull/7546) (the GCC candidate). Below 2 Mbit/s the pacer's burst budget is one packet, and moq-quic checks a datagram's pace while the packet before it is still being built, so a batch of two leaves on one packet's credit. The shared pacer (`rs/moq-congestion/src/pacer.rs`) then forgets the overshoot instead of charging it to the next send. #7546 fixed it by carrying the overshoot as debt (its commit `b62e538b94`), which moved BBR3's nightly `competing_cubic` (`rs/moq-quic/src/tests/probe.rs`) to 441.5 late frames against an allowed 439.4, so it reverted the fix rather than retune BBR3 there. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m1, because the next release ships BBR3 as moq-quic's default and a 2x overshoot at low rates is a correctness bug in what it sends. Land the pacer fix and BBR3's retune together, so the nightly never runs one without the other. + +- Start from `b62e538b94`'s debt approach, with a unit test in `pacer.rs` that a batch outrunning a one-packet budget is charged to the next send, and the existing `aggregate_throughput_matches_controller_pacing_rate` swept below 2 Mbit/s. +- Retune where the change shifts BBR3's probing, not the budgets: the probe scenarios' budgets judge the probe, and loosening them to pass would hide the fix's effect. Report every probe scenario's paired late frames, p99, and overhead before and after, on the scenario's full seed set. +- `competing_cubic`'s 8-seed late budget is fragile on its own ([moq-dev/moq#7538](https://github.com/moq-dev/moq/pull/7538) found 28% of 8-seed windows fail it); its seed count is planned separately in [moq-dev/moq#7440](https://github.com/moq-dev/moq/pull/7440). Judge this change on a seed set large enough to separate it from that noise, not on whether seeds 1-8 pass. +- Re-run the realtime candidates' rows that pace below 2 Mbit/s ([GCC](https://github.com/moq-dev/moq/pull/7546), [BBR3 delay bound](https://github.com/moq-dev/moq/pull/7544), [SCReAMv2](https://github.com/moq-dev/moq/pull/7547)) for whichever have landed, and record the shift in the [epic](/quest/m2/quic-realtime/README.md). + +Public API: none. Wire: none. + +## Related + +- [Realtime uplink congestion control](/quest/m2/quic-realtime/README.md) - every candidate paces through this pacer +- [Probe up to a target bitrate](/quest/m2/probe/README.md) - its scenario budgets are what the retune must keep diff --git a/quest/m1/quic-api/poll.md b/quest/m1/quic-api/poll.md index 011ad7f037b..79e8b7383db 100644 --- a/quest/m1/quic-api/poll.md +++ b/quest/m1/quic-api/poll.md @@ -1,4 +1,4 @@ -# [S] The QUIC drivers' waits poll +# [M] The QUIC drivers' waits poll ## Goal @@ -57,6 +57,25 @@ until a caller needs one. Report the public API: new poll methods on `Connection`, `send::Stream` and `Endpoint`, with the async forms unchanged. +Added 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 +/quest-afk run), from [moq-dev/moq#7490](https://github.com/moq-dev/moq/pull/7490), +which finishes [waits](/quest/m1/quic-api/waits.md): + +- **Remove `ShouldTransmit`.** Once moq-quic wakes the transmit waiter itself, + the `ShouldTransmit` that `Chunks::finalize` and the send lease methods + (`lease`, `poll_lease`, `restore`, `release` in + `rs/moq-quic/src/connection/streams/lease.rs`) return is ignored by every + consumer. Drop it from moq-quic's public API and its callers, as part of + this reshape, since moq-quic is unreleased. +- **Test moq-tokio's close while the socket is blocked.** moq-tokio's driver + skips `poll_transmit` while the socket holds output back, so a local close + could not re-arm the closing period's timer; #7490's round 1 fixed it by + also parking the driver on the non-consuming `poll_closed`. No driver test + has a socket that holds sends back, so nothing covers it. + Add one (a test socket whose sends return `WouldBlock` until released) and + a test that a close behind it is still bounded, mirroring qmux's + `a_close_behind_a_stalled_peer_is_bounded`. + ## Required - [Waits](/quest/m1/quic-api/waits.md) - moq-quic's `poll_*` forms these forward to diff --git a/quest/m1/release-perf/README.md b/quest/m1/release-perf/README.md index 0bdaaf5a476..c265a7c08c1 100644 --- a/quest/m1/release-perf/README.md +++ b/quest/m1/release-perf/README.md @@ -7,9 +7,9 @@ Before the next release cut, every benchmark where `main` is slower than is fixed. Cost that pays for a feature `main` deliberately added is accepted, and the reason is written here. -This epic's own work: once its child has landed, re-pair the relay `fanout` -workload against `release` and confirm main's relay CPU is no higher in every -round. The sparse rows are accepted (2026-10-30). The browser frame path +This epic's own work: once its children have landed, re-pair the relay +`fanout` workload against `release` and confirm main's relay CPU is no higher +in every round. The sparse rows are accepted (2026-10-30). The browser frame path needs no repeat; the [performance audit](/quest/m2/audit-perf.md) now pairs every later cut with `release`. @@ -271,6 +271,7 @@ Decided 2026-10-30 (maintainer), in ## Required - [Fanout lock contention](/quest/m1/release-perf/fanout-lock.md) - relay fanout subscribers stop contending one track lock, so main's relay CPU matches release's in every round +- [Relay CPU per frame since #5926](/quest/m1/release-perf/relay-cpu-bisect.md) - the 18-56% rise in relay instructions per delivered frame is bisected, attributed, and its avoidable cost fixed ## Related diff --git a/quest/m1/release-perf/relay-cpu-bisect.md b/quest/m1/release-perf/relay-cpu-bisect.md new file mode 100644 index 00000000000..b206f31d9c9 --- /dev/null +++ b/quest/m1/release-perf/relay-cpu-bisect.md @@ -0,0 +1,33 @@ +# [M] Find and fix the relay's per-frame CPU growth since #5926 + +## Goal + +The commits that raised the relay's instructions per delivered frame between [`cdff3c8f95`](https://github.com/moq-dev/moq/commit/cdff3c8f95) ([moq-dev/moq#5926](https://github.com/moq-dev/moq/pull/5926) as merged) and `main` are found, each rise is attributed to a cause, and the avoidable costs are fixed. A rise that pays for a deliberate feature is accepted with its reason in the [release-perf epic](/quest/m1/release-perf/README.md). + +## Plan + +Found by [moq-dev/moq#7505](https://github.com/moq-dev/moq/pull/7505), which profiled the io_uring clock read. At `main` [`30f23c3f8a`](https://github.com/moq-dev/moq/commit/30f23c3f8a), relay instructions per delivered frame had risen 25-56% on io_uring since `cdff3c8f95` and 18-38% on the tokio workers since `1dd9b83d1`, with the same delivery: + +| workload | runtime | before, kinstr/frame | `30f23c3f8a`, kinstr/frame | +|---|---|---|---| +| chat | io_uring | 86.3 [84.5-87.4] at `cdff3c8f95` | 134.8 [128.5-151.3] | +| video | io_uring | 108.7 [106.6-116.2] at `cdff3c8f95` | 136.1 [132.6-152.8] | +| chat | tokio workers | 104.0 [101.4-105.0] at `1dd9b83d1` | 143.1 [127.5-205.3] | +| video | tokio workers | 141.6 [139.3-148.3] at `1dd9b83d1` | 167.1 [164.9-173.6] | + +Medians of 5 interleaved reps, min-max in brackets. The tokio rows start at [`1dd9b83d1`](https://github.com/moq-dev/moq/commit/1dd9b83d19), #5926's base, since #7505 measured the tokio workers only there and at `main`. Both runtimes moved, so at least one cause is shared code (moq-net's model, kio, or the session layer). That does not rule out a further io_uring-only cost, such as the clock read #7505 profiled. The tokio range also spans #5926 itself, so the two rises can't be subtracted to find one. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m1, in this epic, since `main` must not reach the cut carrying accidental per-frame cost. + +- Measure as #7505 did: `--profile profiling` builds with frame pointers and `moq-relay/io-uring,moq-bench/uring`, one target directory per revision, `perf stat` `cpu_core` instructions (user and kernel) over the steady window, divided by the frames the generator's 500 ms reports count over the same window. 4 io_uring workers, not 1, so chat does not saturate a worker. Pin the relay and generator cores so a loaded host still reads instruction deltas. +- Bisect `cdff3c8f95..main` on chat over io_uring first (the largest rise and the tightest ranges), with a step size that keeps each build's cost bounded, then confirm the culprits on video and on the tokio workers. A rise that shows on io_uring but not on the tokio workers is attributed on its own, not folded into a shared culprit. +- Attribute each step with `perf record` at a fixed period, as #7505 did, before deciding fix or accept. Record accepted costs in the epic's Plan with the feature they pay for. +- Commit the driver this needs as a deterministic mode of `just bench-profile` or `just bench-runtime` (pinned cores, instructions per delivered frame, fixed-period samples) instead of another throwaway script; #7505's driver on `bench/relay.sh` was not committed. +- [Fanout lock contention](/quest/m1/release-perf/fanout-lock.md) covers the bimodal futex cost on the fanout workload; don't fix it twice. A bisect step that lands on the same lock goes there. + +Public API: none expected. Wire: none. + +## Related + +- [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - the nightly series that should have caught this growth +- [Performance audit](/quest/m2/audit-perf.md) - counts instructions when bisecting, for the same reason diff --git a/quest/m1/vendor-sync/uniffi.md b/quest/m1/vendor-sync/uniffi.md index 44fa8ba4651..30dfebb7ff4 100644 --- a/quest/m1/vendor-sync/uniffi.md +++ b/quest/m1/vendor-sync/uniffi.md @@ -49,3 +49,14 @@ First run, check these first: [`d1f766f4`](https://github.com/mozilla/uniffi-rs/commit/d1f766f4) (Kotlin Throwable handling), and [`f903c36c`](https://github.com/mozilla/uniffi-rs/commit/f903c36c) (Python prefers the running event loop). + +Fork CI, added 2026-11-01 (maintainer, follow-up planning after the +2026-11-01 /quest-afk run): kixelated/uniffi-dart's `Testing` +(`.github/workflows/tests.yml`) and `Test Downstream` +(`.github/workflows/test-downstream.yml`) workflows failed in all 14 of their +runs among the fork's 30 most recent PR runs that day, while `Nix` passed, so +a red check on a fork PR, including a vendor sync's own, says nothing. In `Testing`, `Test Suite (stable)` +failed at `Run cargo nextest` while nightly passed and 1.91 was cancelled. +Fix them, or drop them where the fork's `Nix` job and moq's own binding +checks already cover what they test, and record which in the fork's README. +Keep the fork's CI read-only, as `quest guide`'s Reviews section requires. diff --git a/quest/m1/vendor-sync/vaapi.md b/quest/m1/vendor-sync/vaapi.md index c9914261d86..dfd697bb5b9 100644 --- a/quest/m1/vendor-sync/vaapi.md +++ b/quest/m1/vendor-sync/vaapi.md @@ -27,3 +27,12 @@ at `59384456`, five commits ahead, of which `59384456` (a VP9 struct field for libva 2.23+) is the only fix and moq-vaapi is H.264-only, and chromeos/cros-codecs was at `a050c6cd`, one commit ahead. Record the commits checked in moq-dev/vaapi's README, and copy them here for the next run. + +CI, added 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 +/quest-afk run): moq-dev/vaapi's only workflow, `.github/workflows/check.yml`, +sets `id-token: write` at the workflow level and runs on `pull_request`, so +every PR's check job can mint an OIDC token without a protected environment. +CI on pull requests must stay read-only. Remove it, leaving +`contents: read`; if a step needs OIDC (`magic-nix-cache-action` is the likely one), confirm +it still works without it, and otherwise move that step to a job limited to +`main` behind a protected environment. diff --git a/quest/m2/README.md b/quest/m2/README.md index aae6ce7d51b..df1ee6dd000 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -27,6 +27,7 @@ lite-07; bitrate enforcement is not part of this release. - [Byte streams](/quest/m2/stream/README.md) - TCP connections ride MoQ losslessly through relays, proxied per hop rather than as IP packets - [Release execution](/quest/m2/release/README.md) - publication, installed-artifact proof, cleanup, and downstream adoption after m1 - [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - fMP4, MKV, FLV, TS, Annex-B, catalog, and AMF0 parsing fuzzed with a bounded-work invariant +- [One policy for oversized container metadata](/quest/m2/import-metadata-policy.md) - fMP4 and MKV refuse or stream huge metadata by one documented rule - [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the cmsg walks in moq-sock and moq-uring have invariant fuzz targets, and the unsafe peer input reaches runs under Miri - [In-memory unsafe runs under Miri](/quest/m2/miri-memory.md) - kio's and moq-net's memory-only unsafe runs under a nightly Miri job - [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the VA-API renderer, decode, and transcode tests run in `just rs gpu` on AMD and Intel and fail instead of skipping, and hosted CI reports them ignored @@ -50,6 +51,7 @@ lite-07; bitrate enforcement is not part of this release. - [Broadcast fairness](/quest/m2/broadcast-fairness.md) - broadcasts share a cluster link fairly, so one tenant's priority never starves another's - [Cache fair share](/quest/m2/cache-fairness.md) - one tenant cannot crowd another out of the relay's shared group cache - [Peer grant bound](/quest/m2/peer-grant-bound.md) - a peer's AUTH_OK costs near-linear time in its bytes on every input shape, in Rust and in a browser +- [A regrant re-checks only the routes it can change](/quest/m2/regrant-route-recheck.md) - an `AUTH_OK` no longer costs routes times grant members on every lite announce stream - [Plan: watch worker](/quest/m2/plan-watch-worker.md) - prototype an invisible page worker against app-spawned workers, and land the jank harness that decides - [Watch worker](/quest/m2/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio - [Vulkan Video encode on AMD](/quest/m2/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070 @@ -106,6 +108,7 @@ lite-07; bitrate enforcement is not part of this release. - [Bindings in the interop matrix](/quest/m2/interop-bindings.md) - Kotlin, Swift, and Dart publish and subscribe against every other interop client - [WebGPU HDR](/quest/m2/webgpu-hdr.md) - HDR renditions play as HDR where the browser and display can show it, and tone-map to SDR elsewhere - [Resume as producer replacement](/quest/m2/resume-redesign.md) - route changes replace the writer of one track state, deleting the per-reader resume path in Rust and JS +- [Migrated sessions close once their replacement synced](/quest/m2/goaway-migration-followups.md) - moq-tokio holds the early close for the replacement's announce sync, and the JS `Connection` closes early too - [moq-transport requests end through one lease](/quest/m2/ietf-request-lease.md) - every IETF request ends through one polled lease, and the two 18k-line files split by mechanism - [A moq-transport group head fills its hole in place](/quest/m2/ietf-fill-refill.md) - fills and joining FETCHes write into the tail's group, deleting the `Fill` handoff - [moq-transport hole markers](/quest/m2/ietf-hole-marker.md) - a negotiated extension names a SUBSCRIBE's holes, so our publishers answer multi-group FETCH from caches that know every group; built when an IETF client wants it @@ -133,8 +136,9 @@ lite-07; bitrate enforcement is not part of this release. - [Windows GPU input](/quest/m2/obs-windows.md) - import or blit OBS D3D11 textures with explicit synchronization - [WASM-to-JS evaluation](/quest/m2/rs2ts-wasm2js.md) - try the Rust implementation as JS-only browser output and compare source-translator alternatives if needed - [JS-only @moq/net](/quest/m2/rs2ts-js.md) - adopt the selected Rust-to-JS route after browser parity and measured tradeoffs are accepted -- [Per-stream deadlines](/quest/m2/quic-deadline.md) - hopeless retransmits - become resets, and a tail loss probe fires early while there is still time +- [Per-stream and per-datagram deadlines](/quest/m2/quic-deadline.md) - hopeless retransmits + become resets, a tail loss probe fires early while there is still time, and + datagrams past their subscription's max delay are dropped instead of sent late - [Measure ECN on the backbone](/quest/m2/quic-ecn-measure.md) - a written verdict on marking versus dropping, and whether Linode and OVH keep marks - [poll_acked on moq-net's send stream](/quest/m2/quic-ack-hook.md) - the @@ -149,6 +153,7 @@ lite-07; bitrate enforcement is not part of this release. - [Ring drain GOAWAY](/quest/m2/uring-ring-drain.md) - the ring sends moq-transport 17+ sessions their drain GOAWAY - [Relay io_uring packages](/quest/m2/relay-io-uring-package.md) - Linux relay packages ship io_uring once the ring is on par with tokio - [io_uring thread check](/quest/m2/uring-transport-machines.md) - the relay fanout bench measures `moq_uring::transport::Local`'s per-call thread check after the transport is boxed, removing it only if it shows +- [io_uring quiescence on a quiet host](/quest/m2/uring-bench-followups.md) - quiet-host acceptance numbers and a nightly `worker_turns` QUIC request-reply series - [Opus implementation](/quest/m2/audio-opus-backend.md) - compare Opus codec quality, CPU, build cost, and the loss recovery each backend offers - [Latency ledger](/quest/m2/latency-ledger.md) - a viewer reports its share of playback delay stage by stage: jitter buffer and device - [Media Foundation decode](/quest/m2/audio-decode-mediafoundation.md) - Windows decodes HE-AAC, multichannel AAC, and what else the MFTs offer @@ -179,9 +184,11 @@ lite-07; bitrate enforcement is not part of this release. - [A release carries multi-vendor GPU input](/quest/m2/gpu-release.md) - `release` carries the GPU surface validated on AMD and Intel; health follows separately - [The remaining library spawns](/quest/m2/spawn-audit.md) - every async spawn left in a library crate becomes a polled `kio::Tasks` child or is listed with why it stays - [The relay over QUIC on the sim](/quest/m2/quic-sim.md) - the relay and moq-tokio's `Client` and `Server` run over `moq-quic` on the seeded sim +- [moq-shaper's own socket never passes for path loss](/quest/m2/shaper-socket.md) - relay-sized buffers and its own socket drops reported beside path drops - [Archive time ranges](/quest/m2/archive-time-ranges.md) - a `Time(pts)` range on an archive consumer broadcast resolves from the archive's keys, however old - [DVR rewind](/quest/m2/archive-dvr.md) - follow-on capability after the release contract is stable +- [Archive lookups cancel when abandoned](/quest/m2/archive-lookup-withdrawal.md) - a public withdrawal signal on `Fill` stops a time lookup's store read once its readers leave - [JS HLS field](/quest/m2/js-hls.md) - follow-on capability after the release contract is stable @@ -189,6 +196,7 @@ lite-07; bitrate enforcement is not part of this release. - [Publisher timeliness at relay ingest](/quest/m2/qos-publisher-timeliness.md) - follow-on capability after the release contract is stable +- [Auth session caps cost O(1)](/quest/m2/auth-session-index.md) - per-token and per-remote counts and a time-ordered expiry queue replace the scans, after #7211 - [H.264 parsing with maintained crates](/quest/m2/h264-reader.md) - moq-mux parses SPS VUI with h264-reader and shares one bit reader across codec config parsers - [Relay metrics through prometheus-client](/quest/m2/relay-prometheus-client.md) - `/metrics` renders through the maintained client with the same series - [moq-transport finish interop](/quest/m2/ietf-finish.md) - the interop `finish` case runs on every even moq-transport draft instead of being skipped @@ -226,19 +234,24 @@ lite-07; bitrate enforcement is not part of this release. - [The HLS server follows each path once](/quest/m2/hls-follow-once.md) - `Following` reports synced absence, so moq-hls resolves with one follow - [moq-gst playback shares one session context](/quest/m2/gst-session-context.md) - the playback helpers take one context instead of six arguments - [moq-bench pins the instances it measures](/quest/m2/bench-epoch-pin.md) - pooled requests pin their announced epoch, so a takeover is not counted as the same broadcast +- [The memory bench finishes at 1 MiB frames](/quest/m2/memory-bench-hang.md) - the allocation mode's park at `size=1048576` is root-caused and fixed - [Audio's Latest start reaches back](/quest/m2/audio-latest-reach-back.md) - moq_audio's start covers min plus warmup like other consumers - [moq-lite tails count drops](/quest/m2/lite-tail-drop-count.md) - SUBSCRIBE_DROP accounting replaces the session-tail grace from lite-05 - [Re-run the SPS fuzzer on h264-reader](/quest/m2/sps-fuzz-h264-reader.md) - the H.264 SPS overflow is rechecked on the new parser - [Bindings see a broadcast end](/quest/m2/binding-broadcast-end.md) - `MoqBroadcastConsumer` exposes the close, and a moq-transport run end is a protocol error - [Audio track changes are gapless](/quest/m2/gapless-audio-switch.md) - no silence beyond a frame on audio format or rendition changes, enforced by media QA +- [PCM encodes in one pass](/quest/m2/pcm-bulk-encode.md) - the PCM encoder converts a frame in bulk instead of one sample at a time - [We own our WebRTC stack](/quest/m2/webrtc/README.md) - moq-rtc becomes moq-webrtc with moq-ice, the data-channel transport, and a vendored, trimmed str0m/dimpl/is/sctp-proto - [Certificate reload and qmux's tasks are supervised](/quest/m2/tls-reload-supervision.md) - a panic in either fails loudly instead of silently stopping - [PUBLISH_DONE says why](/quest/m2/publish-done-codes.md) - GOING_AWAY, TOO_FAR_BEHIND, and EXPIRED round-trip as stream codes - [An interrupted mp4 recording keeps its last fragment](/quest/m2/mp4-sigint-flush.md) - SIGINT flushes the buffering fragment before finishing - [Repair a crashed mp4 recording](/quest/m2/mp4-repair.md) - a crashed recording's fragmented file becomes a regular MP4 +- [moq repair refuses a live recording](/quest/m2/mp4-repair-lock.md) - the recorder's advisory lock keeps `moq repair` off a file still being written - [Windows receives notice truncated control data](/quest/m2/cmsg-windows-truncation.md) - Windows handles `MSG_CTRUNC` like the Unix paths +- [The error-queue reader decodes truncated control data](/quest/m2/errqueue-truncation.md) - `MSG_ERRQUEUE` keeps what fit on `MSG_CTRUNC`, like the receive paths - [moq-quic's internal dev-dependencies are path-only](/quest/m2/quic-dev-dep-path.md) - moq-congestion is a path dev-dependency - [moq-mux's AAC config reports SBR](/quest/m2/aac-sbr-config.md) - moq-audio reads backward-compatible SBR from the config +- [fMP4 decode times from the declared reorder depth](/quest/m2/fmp4-declared-clock.md) - `DeclaredClock` replaces the sub-GOP cadence cap, as in TS, FLV, and MKV - [An ended HLS playlist has no trailing gap](/quest/m2/hls-trailing-gap.md) - no zero-duration `EXT-X-GAP` before `EXT-X-ENDLIST` - [moq-video's Apple Sink encodes on its own thread](/quest/m2/apple-sink-thread.md) - macOS and iOS ladders never encode on a tokio worker - [The test inventory sees JS bench scripts](/quest/m2/js-bench-inventory.md) - JS bench entry points are resolved by the inventory gate diff --git a/quest/m2/archive-lookup-withdrawal.md b/quest/m2/archive-lookup-withdrawal.md new file mode 100644 index 00000000000..60acbebd62e --- /dev/null +++ b/quest/m2/archive-lookup-withdrawal.md @@ -0,0 +1,22 @@ +# [XS] moq-archive cancels an abandoned lookup's store read + +## Goal + +A `track::Fill` exposes a public signal for "no reader waits on this fill any more", and moq-archive's handler uses it to stop the store read of a time lookup whose readers left, so time-range readers that churn don't leave slow stores serving reads nobody takes. + +## Plan + +Follow-up of [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310), which resolves archive time ranges through a fill lookup: a fill over `0..reach` carrying the unresolved time in `Fill::time()` and owing no group. moq-archive used `fill.unused()` to stop a slow store read once nobody waited, but [moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036) removed it, and a lookup owes no group, so `unwanted(sequence)` can't stand in. #7310 dropped lookup cancellation by maintainer decision (2026-11-01, "no new moq-net API" in that PR), so an abandoned lookup now finishes its read and answers nobody. Group fills still stop on `unwanted(sequence)`. This waits on #7310 landing. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, adding the public withdrawal signal #7310 deferred, since wasted store reads under churn are a real cost for slow stores. + +- `Fill::poll_withdrawn` exists but is crate-private (`rs/moq-net/src/model/track/fill.rs`, about line 759, and async-only). Make it, or a `withdrawn()` future over it, public, rather than #7310's other alternative of `time()` turning `None` once withdrawn, which would overload a value with a lifecycle. +- moq-archive races its lookup's store read against it and drops the read on withdrawal. +- Test: a lookup whose only reader leaves mid-read stops its store GET (a slow mock store counts completed reads). +- Mirror it in `@moq/net` only if a JS handler needs it; none does today. + +Public API: a public withdrawal signal on `moq_net::track::Fill`. Wire: none. + +## Related + +- [A failed archive time lookup fails the subscription](https://github.com/moq-dev/moq/pull/7440) - planned in that open PR as `quest/m2/archive-time-followups.md`: a lookup refusal on `Fill`, the other half of a lookup's lifecycle; whichever lands second shares its `Fill` surface diff --git a/quest/m2/auth-session-index.md b/quest/m2/auth-session-index.md new file mode 100644 index 00000000000..62b739a70fe --- /dev/null +++ b/quest/m2/auth-session-index.md @@ -0,0 +1,18 @@ +# [S] The auth server's session caps cost O(1) per connect + +## Goal + +With `--limit-token` or `--limit-remote` set, the auth server's `connect`, `end`, and `revalidate` cost the same at 16,384 live sessions as at 0, instead of scanning every live session under the server's one mutex. The session table keeps per-token and per-remote counts, and ages out sessions whose relay died from a time-ordered queue instead of a full sweep. + +## Plan + +Found by [moq-dev/moq#7512](https://github.com/moq-dev/moq/pull/7512), which attributed JWT verification cost and runs it nightly: one `connect` plus `end` costs 66.5k instructions (9.1 µs) at 0 live sessions, 147k (18.6 µs) at 1024, and 1.27M (158 µs) at 16,384, about 73 instructions per live session. The cost is `Sessions::sweep` plus the per-token and per-remote count scans in `Sessions::connect` (`rs/moq-auth/src/serve.rs`, about line 375); `Revalidate` pays the sweep too. Its `verify/sessions/{0,1024,16384}` bench is the baseline. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. This waits on [moq-dev/moq#7211](https://github.com/moq-dev/moq/pull/7211) landing, which rewrites `Sessions`' token bookkeeping (shared request-token grants and attachment leases); indexing the table first would conflict with it, as #7512 noted. + +- Keep `HashMap` and `HashMap` beside the slots, updated on connect, end, a new token attaching to a known session, and expiry. `Sessions::revalidate` updates them too, since it re-inserts an aged-out or restart-lost slot and appends a token the slot hadn't seen, and it keeps today's rule of enforcing no cap. +- Age slots from a queue ordered by `seen`, refreshed on each sighting (a re-push with a generation, or an ordered map keyed by `(seen, id)`), so a sweep pops only expired entries. Keep today's rule: a session that missed two cadences is dropped. +- Keep the semantics the doc comment on the limits describes, including a known id attaching another credential counting only against the new JWT's cap. +- Measure with #7512's `verify/sessions` bench before and after; the slope should be flat. + +Public API: none. Wire: none. diff --git a/quest/m2/errqueue-truncation.md b/quest/m2/errqueue-truncation.md new file mode 100644 index 00000000000..be37cb65e4d --- /dev/null +++ b/quest/m2/errqueue-truncation.md @@ -0,0 +1,17 @@ +# [XS] The Linux error-queue reader decodes truncated control data + +## Goal + +moq-sock's `MSG_ERRQUEUE` reader (`rs/moq-sock/src/udp/linux.rs`) handles `MSG_CTRUNC` the way the datagram receive paths do: it decodes whatever control messages fit and warns once, instead of returning an `InvalidData` error for the whole entry. + +## Plan + +Found by [moq-dev/moq#7543](https://github.com/moq-dev/moq/pull/7543), which made the Windows receive path keep a datagram with truncated control data, matching what [moq-dev/moq#7394](https://github.com/moq-dev/moq/pull/7394) did for `recvmmsg`, `recvmsg`, and Apple's `recvmsg_x`. The error-queue reader is a different path and still turns `MSG_CTRUNC` into `io::ErrorKind::InvalidData` (about line 55). It only matters if error-queue metadata ever grows past `ERR_CMSG_LEN`, so this is defensive. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, matching the receive paths. Walk the cmsgs that fit (the iterator is already bounded by the returned control length), return the decoded entry if one is complete, and warn once through a `Once` as `decode_recv` in `unix.rs` does. Extend the existing mocked `MSG_ERRQUEUE` test (`LinuxError::decode`, about line 309) with a truncated case. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - fuzzes the cmsg walks this path shares diff --git a/quest/m2/fmp4-declared-clock.md b/quest/m2/fmp4-declared-clock.md new file mode 100644 index 00000000000..a08653a5477 --- /dev/null +++ b/quest/m2/fmp4-declared-clock.md @@ -0,0 +1,21 @@ +# [S] fMP4 decode times come from the declared reorder depth + +## Goal + +The fMP4 exporter authors decode times from the rendition's declared reorder depth with `container::declared::DeclaredClock`, as the TS, FLV, and MKV exporters do, so sub-GOP fragments of variable-rate B-frame video decode at their group's presentation slots instead of at durations capped at the catalog cadence: a stall longer than the cadence shows as its real gap, and only each group's first depth frames are spaced one tick apart, as in the other exporters. `moq export fmp4`, HLS packaging, and `moq export mp4` share the change. + +## Plan + +Follows [moq-dev/moq#7491](https://github.com/moq-dev/moq/pull/7491), which keeps fMP4 decode times increasing in fragments shorter than a GOP by capping inferred durations at the catalog cadence for a rendition that may reorder. That cap is exact for constant-rate B-frame video, but an H.264 or H.265 rendition that doesn't declare `reorder: 0` and stalls longer than the cadence gets the cadence rather than the real gap. #7491 named this as the alternative it didn't take. It needs #7491 to land first; this replaces its cap. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. One decode-time rule across every exporter is simpler than a cadence cap in one of them. + +- `DeclaredClock` (`rs/moq-mux/src/container/declared.rs`) holds depth frames per group before releasing them and decodes the first depth frames one tick apart up to the earliest slot, so a DTS can fall before the timeline's zero. Settle how fMP4 writes that: a version 1 `trun` with signed composition offsets, or an edit list, and what each costs HLS and `export mp4`'s rebuilt sample tables. +- A rendition that declares no depth keeps today's inference, including #7491's rule. +- Keep #7491's tests (`tfdt` never steps back across sub-GOP fragments, ffprobe reads every fragment) and add a variable-rate B-frame case with a stall longer than the cadence, whose decode times follow the presentation slots across the stall. `DeclaredClock` derives DTS from PTS and depth alone, so a gap that only the encoder's own decode times would show is out of reach and not a goal. + +Public API: none. Wire: none. + +## Related + +- [HLS and DASH segment conformance](/quest/m1/hls-conformance.md) - its nightly ffprobe checks would catch a regression here diff --git a/quest/m2/goaway-migration-followups.md b/quest/m2/goaway-migration-followups.md new file mode 100644 index 00000000000..384f4a40aa7 --- /dev/null +++ b/quest/m2/goaway-migration-followups.md @@ -0,0 +1,18 @@ +# [S] A migrated session closes early only once its replacement has synced, in Rust and JS + +## Goal + +After a GOAWAY migration, moq-tokio's `Connection` closes the idle predecessor only once the replacement's announcements have synced, so a broadcast nobody subscribes to never reads as ended and started again during the switch. The JS `Connection` closes an idle predecessor early the same way, instead of holding it for `goaway.handover`. + +## Plan + +Follow-ups of [moq-dev/moq#7462](https://github.com/moq-dev/moq/pull/7462) (merged), which added moq-net's `Session::poll_idle` / `Session::idle` (resolve once the session carries no subscription or fetch in either direction) and has moq-tokio close a migrated predecessor once it is idle (`rs/moq-tokio/src/connection.rs`, `poll_idle`), instead of waiting out `connect.goaway.handover` (10 s by default). + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, both parts in one quest, since the JS half should copy the Rust half's final rule rather than #7462's. + +- **Hold for the replacement's announce sync.** Today the idle predecessor can close as soon as the replacement connects, before the replacement's announcements reach either side, so an unsubscribed broadcast briefly reads as ended then started. Gate the close on both idle and the replacement's announce sync. That needs a signal moq-net doesn't expose yet: a per-session "announcements synced" readiness (lite-07's announce sync, and the IETF equivalent or a bound where the version can't say). Keep the handover cap as the upper bound. +- **JS early close.** `js/net/src/connection/reload.ts` holds the predecessor to `goaway.handover` (`handover(...)`, about line 670). Add an idle signal on `Established` mirroring `Session::idle` (subscriptions and fetches in either direction; announcements, track info, PROBE, and AUTH don't count), the same announce-sync gate, and a retire in `Reload` that closes the predecessor when both hold. +- Test both on simulated time over the mock transport, as #7462's `rs/moq-net/tests/integration/session_idle.rs` does: a broadcast announced on both sessions and subscribed by nobody never shows an end across the migration, and a busy predecessor still waits for its readers. + +Public API: a moq-net readiness for the announce sync, `Established`'s idle signal in `@moq/net`, and their mirrors. Wire: none. + diff --git a/quest/m2/import-metadata-policy.md b/quest/m2/import-metadata-policy.md new file mode 100644 index 00000000000..b0aa9d63bed --- /dev/null +++ b/quest/m2/import-metadata-policy.md @@ -0,0 +1,26 @@ +# [S] One policy for oversized container metadata + +## Goal + +The fMP4 and MKV importers follow one documented policy for container metadata that declares a huge size: the same rule for when it is refused, the same bound or the same reason for having none, and the same error shape, so a caller (HLS ingest, `moq import`, the bindings) sees one behavior whichever container it feeds. + +## Plan + +The two importers settled this separately during the 2026-10-27 security audit: + +- fMP4 ([moq-dev/moq#7232](https://github.com/moq-dev/moq/pull/7232), merged): what must be held whole, a `moov` or `moof` body or one text sample, is refused at its header when it declares more than `fmp4::MAX_HELD_BYTES` (32 MiB, the transport's per-group `MAX_CACHE_BYTES`), and a `moof` declaring more than `fmp4::MAX_FRAGMENT_SAMPLES` (2^21) is refused when it parses. That reversed the 2026-10-28 incremental plan, because incremental parsing could not bound what it kept. +- MKV ([moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234), merged): metadata parses incrementally with no blanket size cap (maintainer, 2026-10-28). Each collected master keeps one value per field, so repeats replace rather than accumulate, but fields read whole (`CodecID`, `DocType`, `CodecPrivate`) have no bound. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. Pick one policy for both importers, rather than leaving each at the decision its own audit PR made. Starting point: a byte bound at the header for whatever must be held whole, since #7232 found that is what actually bounds memory, while MKV keeps streaming what it can consume incrementally. Bring the choice to the maintainer in the PR if the evidence points elsewhere. + +- #7232 has landed the fMP4 side (`MAX_HELD_BYTES` and `MAX_FRAGMENT_SAMPLES` in `rs/moq-mux/src/container/fmp4/mod.rs`). Wait for MKV leaf limits (below), since it sets MKV's bounds on the fields read whole. +- Inventory what each importer still holds whole, and how much parsed state it builds from it (#7232 noted a 32 MiB `moof` decodes into larger structures). +- Keep audio and video payloads unbounded in both: streaming each sample into its frame was decided 2026-10-27, and this quest is about metadata only. An fMP4 text sample is the one payload held whole, because its rewrite needs every byte, so `MAX_HELD_BYTES` covers it; keep that exception. MKV has no counterpart, since it refuses subtitle tracks. +- Document the policy once, in moq-mux's import docs, and make both importers' bounds `pub const`s as #7232's are. + +Public API: possibly new `pub const` bounds and a shared refusal error in moq-mux. Wire: none. + +## Related + +- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - would measure parsed-state amplification under either policy +- [MKV leaf limits](https://github.com/moq-dev/moq/pull/7468) - planned in that open PR as `quest/m1/security-audit/mkv-leaf-limits.md`: per-element limits on the three MKV fields read whole, in m1. This quest starts from whatever it lands and aligns the two importers' rules, not the limits it picked diff --git a/quest/m2/memory-bench-hang.md b/quest/m2/memory-bench-hang.md new file mode 100644 index 00000000000..7eee2124f3e --- /dev/null +++ b/quest/m2/memory-bench-hang.md @@ -0,0 +1,13 @@ +# [XS] The moq-net memory bench's allocation mode finishes at 1 MiB frames + +## Goal + +`MEMORY_ALLOCS=1 cargo bench -p moq-net --features fuzz --bench memory` runs every case to completion, including `size=1048576`, and the cause of today's hang is fixed where it lives (the bench, its counting allocator, or moq-net). If the cause is in moq-net, a regression test covers it. + +## Plan + +Found by [moq-dev/moq#7522](https://github.com/moq-dev/moq/pull/7522): on its base [`c1a1412820`](https://github.com/moq-dev/moq/commit/c1a1412820) (`main` at the time), the allocation mode parks with no CPU after `streams=1/size=16384`, at `size=1048576`. A park with no CPU points at a lost wake or a flow-control stall, not slow work. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. Root-cause before fixing: take a stack of every thread while parked (`gdb -batch -p -ex 'thread apply all bt'` on the parked bench process), check whether the counting allocator's own state is involved (compare a run without `MEMORY_ALLOCS`), and whether a 1 MiB frame exceeds a window or cache bound the bench's in-memory transport waits on. A lost wake in moq-net gets its own regression test; a bench-only cause is fixed in `rs/moq-net/benches/memory.rs`. + +Public API: none. Wire: none. diff --git a/quest/m2/mp4-repair-lock.md b/quest/m2/mp4-repair-lock.md new file mode 100644 index 00000000000..97736e180e9 --- /dev/null +++ b/quest/m2/mp4-repair-lock.md @@ -0,0 +1,22 @@ +# [XS] moq repair refuses a recording still being written + +## Goal + +`moq export mp4 --output ` holds an advisory lock on its output while it records, and `moq repair ` refuses, with a message saying a recorder still has it open, when it can't take that lock. Neither side runs unlocked: where the filesystem can't lock, both refuse. Repairing a file mid-recording no longer corrupts it. + +## Plan + +Found by [moq-dev/moq#7560](https://github.com/moq-dev/moq/pull/7560), which adds `moq repair`: it can't tell a running `export mp4` still writes the file, and repairing one corrupts it. #7560 is still a draft, so this waits on it landing; `moq repair` and `moq_mux::container::mp4::repair` don't exist on `main` until then. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. An advisory lock taken by the recorder and checked by repair, as #7560 suggested. + +- The CLI owns the lock, not moq-mux: `repair` in moq-mux takes `impl Read + Seek` and stays file-agnostic. Use an exclusive `flock`-style lock on the open file (`std::fs::File::try_lock` is stable in Rust 1.89), held for the recording's life and released by the OS if the recorder dies, so a crashed recording stays repairable. +- Supported or refused (`AGENTS.md`), on both sides. The recorder fails at startup if it can't take the lock, a filesystem without lock support included, since its recording could never be repaired safely. `repair` takes the same lock non-blocking before scanning and refuses if it can't: held means a recorder still has the file open, and any other lock error (no lock support) refuses with that error rather than repairing unlocked. +- Test: a held lock makes `moq repair` refuse and leave the file untouched; #7560's nightly `export::a_killed_mp4_recording_is_repaired` still repairs after SIGKILL, since the lock dies with the process. +- Update `doc/bin/cli.md`'s `repair` entry. + +Public API: none (CLI behavior only). Wire: none. + +## Related + +- [An interrupted mp4 recording keeps its last fragment](/quest/m2/mp4-sigint-flush.md) - the other half of recording durability diff --git a/quest/m2/pcm-bulk-encode.md b/quest/m2/pcm-bulk-encode.md new file mode 100644 index 00000000000..d1d0bdd65b6 --- /dev/null +++ b/quest/m2/pcm-bulk-encode.md @@ -0,0 +1,13 @@ +# [XS] The PCM encoder converts a frame in one pass + +## Goal + +moq-audio's PCM encoder (`rs/moq-audio/src/encode/backend/pcm.rs`) writes a frame's samples to its little-endian payload in one bulk conversion instead of growing the payload with `extend_from_slice` one sample at a time. Its output bytes are unchanged, its instructions per sample fall, and they stop shifting with LTO code generation. + +## Plan + +Found by [moq-dev/moq#7520](https://github.com/moq-dev/moq/pull/7520), whose `batches` bench uses PCM as the codec so buffering cost isn't hidden by encoder work. The per-sample `extend_from_slice` loop's instruction count moves with LTO code generation, which makes the PCM control noisier than it needs to be. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. On little-endian targets the `f32` slice's bytes are already the payload, so copy them at once; keep a per-sample path only for big-endian, behind `cfg(target_endian = "big")`. Check the PCM decoder's mirror loop (`rs/moq-audio/src/decode/backend/pcm.rs`) and give it the same treatment if it has one. Measure with #7520's `rs/moq-audio/benches/batches.rs` count table (`--bench __none__`) once #7520 lands, and keep its `batch_size_does_not_change_the_published_track` test passing. + +Public API: none. Wire: none. diff --git a/quest/m2/perf/README.md b/quest/m2/perf/README.md index f750d75f295..8e3bf71344f 100644 --- a/quest/m2/perf/README.md +++ b/quest/m2/perf/README.md @@ -91,13 +91,16 @@ revisions on the same host and worker count, and profiles with - [Run to quiescence](/quest/m2/perf/uring-quiescence.md) - a received packet's reply is staged in the same turn, under a pass and train budget that keeps the fairness rule - [Ingest batch](/quest/m2/perf/ingest-batch.md) - relay ingest pays one lock, wake, and clock read per burst of whole frames instead of per frame - [Streaming frame read owners](/quest/m2/perf/frame-read-owners.md) - readers of a frame still arriving share one byte owner per publication instead of allocating one each per read +- [kio waiter costs](/quest/m2/perf/kio-waiter-costs.md) - long-lived waiters stop being rebuilt every poll, IETF fetch keeps one close watch per stream, and a short `kio::log` costs less - [Shard by default](/quest/m2/perf/shard-default.md) - a saturating shard sweep decides whether an unset `runtime.workers` runs one worker per core on Linux - [io_uring buffer pool memory](/quest/m2/perf/uring-pool-memory.md) - measure reserved versus resident pool memory before any shrink policy - [An idle export poll is cheap](/quest/m2/perf/export-idle-poll.md) - an export poll nothing woke skips the track fill and catalog work it repeats today - [Mesh session cost](/quest/m2/perf/mesh-session-cost.md) - moq-net's per-session announce, route sync, and scope work drops, swept over sessions and announces +- [Cluster route memory](/quest/m2/perf/cluster-route-memory.md) - a full-mesh cluster's ~3.7 KB per route per broadcast shrinks, the in-memory hop is profiled, and `bench-profile lock` tolerates lost records - [Adapter reads reuse their waker](/quest/m2/perf/adapter-waker-alloc.md) - a measured verdict on the per-read waker allocation moq-tokio and moq-uring reads pay since #7042, fixed if real - [Lite request allocations](/quest/m2/perf/lite-request-allocs.md) - `Path::concat` allocates once and the lite request handler keeps one track-name copy - [Coverage before START](/quest/m2/perf/covered-before-start.md) - the lite-07 publisher's coverage check costs O(1) per wake before SUBSCRIBE_START - [io_uring demux churn](/quest/m2/perf/uring-demux-churn.md) - the demux snapshots stats and builds its fed list once per completion, not per datagram - [Lock-free single-delivery writes](/quest/m2/perf/single-delivery-write.md) - a frame with one delivery writes without a lock, recovering #6922's remaining FETCH cost - [JS publisher per-frame cost](/quest/m2/perf/js-publisher-frame-cost.md) - fewer promise races per frame, and a per-frame cost that stays flat as streams grow +- [Small hot-path waste](/quest/m2/perf/hot-path-waste.md) - one track-name copy per lite subscription, no full-history `#drift` walk, and one header buffer per JS object diff --git a/quest/m2/perf/cluster-route-memory.md b/quest/m2/perf/cluster-route-memory.md new file mode 100644 index 00000000000..f45ee0c3f26 --- /dev/null +++ b/quest/m2/perf/cluster-route-memory.md @@ -0,0 +1,26 @@ +# [M] Full-mesh cluster routes cost less memory, and the hop's group cost is profiled + +## Goal + +Three follow-ups of the kio channel-contention spikes, each measured: + +- The announce state a relay holds per route per broadcast in a full-mesh cluster shrinks from about 3.7 KB, so a mesh of W peers stops paying W squared times that for every broadcast, or the quest records why that state is needed. +- The in-memory moq-lite hop's cost of about 140 thousand user instructions for a one-frame group is profiled (`cpu`), with the avoidable parts fixed or handed to the quest that owns them. +- `just bench-profile lock` tolerates a few lost perf records and nonalternating switch records, reporting how many it lost, instead of failing the run. + +## Plan + +From [moq-dev/moq#7534](https://github.com/moq-dev/moq/pull/7534) (the relay-per-thread cluster spike) and [moq-dev/moq#7533](https://github.com/moq-dev/moq/pull/7533) (the replica spike). + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, in the perf epic, one quest for the spikes' three measurement follow-ups. + +- **Route memory.** #7534 meshed W origins over moq-lite-06 `memory` sessions, as dialed cluster peers do. Announce heap per broadcast went from 4.9 KB with one origin to 45 KB at 4 workers and 886 KB at 16 (240 routes, about 3.7 KB each), and allocations from 64 to 14,600; 10,000 broadcasts at 16 would hold about 9 GB. Real full-mesh relay clusters pay the same per route. Measure it on a relay cluster with the counting allocator (#7534's `rs/moq-relay/examples/spike_announce.rs` is in its prototype commit [`ddd8b04`](https://github.com/moq-dev/moq/commit/ddd8b042ab82c7a4d53957e31fe35942957b7dfb), not on `main`), break the 3.7 KB down by structure (origin route entry, per-session announce state, hop lists, cursors), and shrink what's duplicated per route. Check against [cluster routing](/quest/m1/cluster-routing/README.md)'s per-origin routes first, since they change what a route holds. +- **Hop cost.** About 140 thousand user instructions per one-frame group across one in-memory hop; fanout's 64 subscribers spread over 15 hops raised user instructions 19%, while chat amortized it. Take a `cpu` profile of one hop and attribute it before fixing; a per-reader part belongs to [mesh session cost](/quest/m2/perf/mesh-session-cost.md), which owns the per-reader group cost. +- **`bench-profile lock`** (`bench/profile.py`): both spikes lost runs to it. #7533 could not record base chat at 16 workers on that host (every attempt lost a perf record or was refused for nonalternating switch records), and #7534 raised the perf buffer by hand (`SPIKE_PERF_BUFFER`). Raise the buffer for high-switch shapes, skip or pair up nonalternating switch records, and report the lost-record count beside the result so a reader can judge it. Cover it in `bench/profile.test.py`. + +Public API: none expected. Wire: none. + +## Related + +- [kio channel contention](/quest/m2/perf/kio-channel-contention/README.md) - the epic both spikes reported to +- [Announce interest streams share one route table](/quest/m2/announce-interest-memory.md) - the same per-stream announce state, from a subscriber's side diff --git a/quest/m2/perf/hot-path-waste.md b/quest/m2/perf/hot-path-waste.md new file mode 100644 index 00000000000..35f911a195a --- /dev/null +++ b/quest/m2/perf/hot-path-waste.md @@ -0,0 +1,25 @@ +# [S] Small per-request and per-frame waste in the lite subscriber and js/net + +## Goal + +Four small hot-path costs are removed or measured, each with a before/after count: + +- The Rust lite subscriber keeps one copy of a track name per subscription instead of copying it twice. +- `@moq/net`'s `Subscriber.#drift` stops walking the whole retained timeline on every read when a track has no info, or the quest shows no real subscriber reaches that state. +- The JS IETF and lite publishers encode an object or frame header into one scratch buffer instead of allocating per varint. +- The JS fetch paths' per-frame write cost is measured, and fixed like the live path if it's the same waste. + +## Plan + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, in the perf epic, one quest because each part is small and found by reading the same profiles. + +- **Lite subscriber track name** ([moq-dev/moq#7500](https://github.com/moq-dev/moq/pull/7500)): #7500 brought the lite publisher's request serve to one copy per request; its allocation test's backtraces showed the subscriber also copying the name twice per subscription, in `SubscriberDriver::poll` and `TrackServe::prepare_establish` (`rs/moq-net/src/lite/subscriber.rs`). Count it the way #7500's `a_lite_request_keeps_one_copy_of_its_track_name` does, on the subscriber's side. +- **`Subscriber.#drift`** ([moq-dev/moq#7523](https://github.com/moq-dev/moq/pull/7523)): `js/net/src/track.ts` walks back from the cap for the newest stamped group, and when `info` is undefined and no group carries a timestamp, the loop never breaks. In #7523's first fixture (an un-accepted, unstamped `Track.Producer`) it took 74% of steady-state CPU at 1024 groups. First check whether a real subscriber can read with history while its track's info is still undefined (for example before the info arrives, or on a version or path that never sets it), and how many groups it holds then. If it can, remember where the last walk stopped or skip the walk until a stamped group arrives; if not, record why and drop this part. +- **Header encoding** ([moq-dev/moq#7526](https://github.com/moq-dev/moq/pull/7526)): `encodeVarint` (`js/net/src/stream.ts`) allocates an `ArrayBuffer(9)` and a view per varint, and an IETF `Frame.encode` makes five varints plus two `concat`s per object, about 12% self time in #7526's ietf 128x8 case and part of the GC share that grows with streams. Lite's `frameHeader` (`js/net/src/lite/publisher.ts`) does the same. Encode into one reused scratch buffer and copy out once. Measure with `just js serve-bench` in Chromium, as #7526 did. +- **Fetch paths**: lite's `#runFetchGroup` and the IETF fetch writer still write each frame separately with no give-up watch; #7526 didn't measure them. Measure first, then apply #7526's live-path treatment if the cost is the same. + +Public API: none. Wire: none. + +## Related + +- [Performance audit](/quest/m2/audit-perf.md) - finds the next round of these diff --git a/quest/m2/perf/kio-waiter-costs.md b/quest/m2/perf/kio-waiter-costs.md new file mode 100644 index 00000000000..afce903e351 --- /dev/null +++ b/quest/m2/perf/kio-waiter-costs.md @@ -0,0 +1,27 @@ +# [S] kio waiters stop churning on long-lived drivers and per-chunk watches + +## Goal + +Three kio waiter costs are measured and fixed where they are real, each with before/after instruction counts: + +- A long-lived waiter parked on more than `PARKED` (8) lists is no longer discarded and rebuilt on every poll, measured on a relay session driver. +- IETF fetch serving keeps one close watch per fetch stream instead of building one per chunk. +- A short-lived `kio::log` costs measurably less than about 1.3k instructions to set up and clean up, or the measurement shows why it can't. + +## Plan + +From [moq-dev/moq#7556](https://github.com/moq-dev/moq/pull/7556) (a measured no-win on adapter waker allocation) and [moq-dev/moq#7531](https://github.com/moq-dev/moq/pull/7531) (borrow-free QUIC state prototypes). + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, in the perf epic, one quest for the three, since they share kio's waiter and log internals and one measurement setup. + +- **`Park::hold`'s retire rule** (`rs/kio/src/waiter.rs`): `hold` replaces a waiter that is not `reusable()`, and a waiter parked on more than `PARKED = 8` lists is not. #7556 traced about 4 `Arc` per group on a session driver (`Handle::run` / `time::run`) to this: the new waiter's first registration lands on the session-close `Bridge` (moq-uring `adapter.rs` and moq-tokio `owned.rs` `poll_closed`). A relay session driver parks on many more lists, so measure there first (allocations and instructions per delivered group over `just bench-runtime` or a session bench). Then choose between raising `PARKED`, changing the retire rule, or leaving it, on the numbers. +- **IETF fetch close watch** (`rs/moq-net/src/ietf/publisher.rs`): the fetch chunk loop pins a fresh `stream.closed()` per chunk (about lines 2387, 2450, and 2488), so each chunk builds a `kio::wait` whose waiter registers on the send stream's close `Bridge`. Hoist it to one per fetch stream. #7556 did not measure this path; a FETCH delivery bench with many chunks per stream shows the before/after. +- **`kio::log` lifetime cost**: #7531 attributed about 1.3k instructions per short-lived log to its epoch pin, flush, and collect, plus segment retirement on drop. That matters to any design holding one log per stream or per group. Profile a create/append/drop microbench, and cut what doesn't need to run for a log that never grew past its first segment. + +A measured no-win for any of the three is a valid outcome; record it here and drop that part. + +Public API: none expected. Wire: none. + +## Related + +- [kio channel contention](/quest/m2/perf/kio-channel-contention/README.md) - the append-only log design that would pay `kio::log`'s cost per stream diff --git a/quest/m2/probe/README.md b/quest/m2/probe/README.md index 6b50b9ffd41..8ebcdd6ed79 100644 --- a/quest/m2/probe/README.md +++ b/quest/m2/probe/README.md @@ -84,6 +84,7 @@ This README's own work, once the children land: ## Required - [Native rung disable](/quest/m1/simulcast-rung-disable.md) - OBS stops sending the top renditions its grant cannot fund, as the Rust/FFI publishers do, and a shaped uplink through moq-ffi proves the resume +- [Browser send estimate](/quest/m2/probe/browser-send-estimate.md) - shipping Chrome and Safari publishers adapt to the relay's PROBE estimate, since they have no `getStats()` - [Padding probes](/quest/m2/probe/padding.md) - a browser probes with a lowest-priority padding stream on an ALR schedule - [JS rung disable](/quest/m2/probe/rung-disable-js.md) - `@moq/publish` applies the native rung rule, recovering through padding - [Viewer targets](/quest/m2/probe/viewer-target.md) - a session sends its viewer handles as its PROBE target, probes to the peer's, splits the receive estimate into viewer grants, and advertises `Increase` where a sender exists diff --git a/quest/m2/probe/browser-send-estimate.md b/quest/m2/probe/browser-send-estimate.md new file mode 100644 index 00000000000..0a71de3a50e --- /dev/null +++ b/quest/m2/probe/browser-send-estimate.md @@ -0,0 +1,22 @@ +# [M] Browser publishers adapt to the relay's PROBE estimate + +## Goal + +A browser publisher on shipping Chrome or Safari gets a send-rate estimate, so its bandwidth reservations (`Reservation.rate`) follow the path and `@moq/publish` encoders adapt, instead of holding their ceiling because the estimate is always `undefined`. Where `WebTransport.getStats()` reports an estimate, the connection keeps using it. + +## Plan + +Found by [moq-dev/moq#7524](https://github.com/moq-dev/moq/pull/7524), which gated `getStats()` sampling on a reservation: shipping Chromium has no `WebTransport.getStats()` (only behind `--enable-blink-features=WebTransportStats`), and neither does Safari. So `js/net`'s `Allocator` never receives an estimate there, and publishers never follow one. qmux reports no counters, and WebRTC's `getStats` throws. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, in the probe epic, feeding the peer's PROBE estimate to the send allocator. The relay already reports what it receives from us: `Probe.estimatedRecvRate` in `js/net/src/connection/stats.ts`, exposed as `Reload.probe` (`js/net/src/connection/reload.ts`). + +- Source order: a transport `getStats()` estimate where the browser has one, else the peer's PROBE `estimatedRecvRate`, else `undefined` (hold the rate, as today). Settle in the PR whether a present-but-empty `getStats` counts as absent. +- A receive-rate report lags the send side by about an RTT plus the peer's report cadence, and reads what arrived rather than what the path could carry. Check how the allocator should treat it while app-limited (it can only fall or hold, not rise past what was sent), and leave raising it to [padding](/quest/m2/probe/padding.md). +- Observing `Reload.probe` requests reports from the peer; request them only while a reservation reads the estimate, matching #7524's gate. +- Test on `@moq/clock`'s `Manual` with a mock session whose transport has no `getStats`: a PROBE report reaches `Reservation.rate`, a falling report lowers it, and a reconnect clears it. Verify in shipping Chromium (no flag) through a shaped link that a publisher's bitrate follows a capacity drop. + +Public API: none expected (the estimate's source is internal). Wire: none; PROBE `Report` is already sent. + +## Related + +- [Media stats](/quest/m2/stats/README.md) - its JS stats also read PROBE because `getStats()` is missing diff --git a/quest/m2/probe/padding.md b/quest/m2/probe/padding.md index a41a5c36937..9e0093541ad 100644 --- a/quest/m2/probe/padding.md +++ b/quest/m2/probe/padding.md @@ -71,3 +71,7 @@ Tests, on `@moq/clock`'s `Manual`: Assert that repeated clusters on a path with no room stay within the overhead budget. Wire it into CI at least nightly, sharing the epic's shaped-uplink setup. + +## Required + +- [Browser send estimate](/quest/m2/probe/browser-send-estimate.md) - shipping Chrome and Safari have no estimate for a probe cluster to pace against or raise until the PROBE estimate feeds the allocator diff --git a/quest/m2/quic-deadline.md b/quest/m2/quic-deadline.md index a066ab3249f..ed127ed9d58 100644 --- a/quest/m2/quic-deadline.md +++ b/quest/m2/quic-deadline.md @@ -1,4 +1,4 @@ -# [L] Per-stream deadlines +# [L] Per-stream and per-datagram deadlines ## Goal @@ -10,7 +10,9 @@ still unacknowledged and there is time for an acknowledgment and one retransmission to land, the sender asks for an immediate ACK and probes early instead of waiting a full PTO. moq-net sets the deadline per group stream from the subscription's `max_delay` and the group's expiry, so no MoQ wire -change is needed. +change is needed. Datagrams get the same: a queued datagram past its +deadline is dropped from the send buffer instead of sent late, so a +congested viewer receives fresh datagrams rather than seconds-old ones. ## Plan @@ -44,6 +46,37 @@ Implement in `moq-quic`. existing group-expired code on the MoQ wire, so a viewer sees the same signal it sees for a relay-side expiry today. +Datagrams. Decided 2026-11-01 (maintainer, follow-up planning after the +2026-11-01 /quest-afk run), from +[moq-dev/moq#7517](https://github.com/moq-dev/moq/pull/7517)'s datagram +delivery bench: behind a 2 Mbit/s bottleneck fed 4 Mbit/s, about 25% of +datagrams were lost and the survivors' p99 was about 8.2 s (p50 about 50 ms) +on lite-06, lite-07-wip, and moq-transport-22 alike. Congestion control kept +the shaper's queue from overflowing; the sender's 1 MiB QUIC datagram buffer +absorbed the overload and dropped only on overflow, and nothing expired a +datagram by age. Of #7517's options (a send deadline, a smaller send buffer, +or oldest-first within a subscription), the deadline was picked, because it +uses the delay budget the subscriber already stated instead of a buffer size +or ordering that suits only some rates. + +- moq-quic: a datagram carries an optional deadline, and the send path + drops a queued datagram whose deadline has passed (counted, and reported + in the connection's stats) before building a packet with it. A datagram + without one behaves as today. The same estimate of forward delay applies: + drop one that can't arrive before its deadline. +- moq-net: the datagram writer sets the deadline from the subscription's + max delay (`Delay::max` once [subscription delay](/quest/m1/subscription-delay.md) + renames it), through the same `transport::poll` seam as the stream + deadline; a backend without it sends as today. +- Keep newest-first within a subscription; with stale datagrams expiring, + it no longer starves the fresh ones. +- Re-run #7517's `datagram_delivery` bench: p99 behind the bottleneck stays + within the subscription's max delay, and the loopback viewer is unchanged. + Percentiles are over delivered datagrams, so also report offered, + delivered, and expired counts separately, and check that datagrams + delivered within the max delay don't fall below today's: the deadline may + drop only what would have arrived too late anyway. + Measure on bufferbloat: a rate below the media bitrate behind a deep but finite queue whose overflow demonstrably drops packets, with competing loss-based traffic if needed to fill it. Random loss may prove recovery diff --git a/quest/m2/quic-realtime/README.md b/quest/m2/quic-realtime/README.md index 2192afeb0b0..5770fc0ecf3 100644 --- a/quest/m2/quic-realtime/README.md +++ b/quest/m2/quic-realtime/README.md @@ -62,6 +62,27 @@ Remaining work once the candidates finish (this README's own job): - Run the shared comparison on the uplink presets: queuing delay p50/p95/p99, throughput, reaction time to a capacity step down, and share against one competing CUBIC flow. Pick the winner on queuing delay. +- Run every candidate under both sender models in one harness: a bulk + upload, and an encoder that follows `bandwidth_estimate` (the model moq-cli + and moq-ffi use, and the one GoogCC is built for). Decided 2026-11-01 + (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): the + candidates' own runs each used one model, so their rows don't compare. + [moq-dev/moq#7544](https://github.com/moq-dev/moq/pull/7544)'s + `tests::realtime` uploaded in bulk, which + [moq-dev/moq#7546](https://github.com/moq-dev/moq/pull/7546) found + understates GoogCC (low utilization from its 350 ms window allowance and + additive increase), while + [moq-dev/moq#7547](https://github.com/moq-dev/moq/pull/7547)'s + `tests::scream` drove an adaptive encoder (30 fps, 2 s GOP, 5x keyframes). +- Before trusting a candidate's estimate to re-enable a stopped rung, weigh + how far it overshoots the path under app-limited bursty media. SCReAMv2's + estimate is its reference window over the RTT, not a delivery rate, so each + keyframe burst grew it past the path: 241 kB/s on a 125 kB/s path in #7547, + which let a rung with keyframes recover without a probe. The + [native rung disable](/quest/m1/simulcast-rung-disable.md) could re-enable + a rung the path can't carry on such an estimate, so the comparison reports + each candidate's estimate against the path's capacity under the encoder + model. - Ship the winner as `Realtime` in moq-tokio and moq-uring, mapped through `MoqCongestionControl` in moq-ffi and every binding in the cross-package sync table, and a `congestion-control` property on `moqsink` forwarded @@ -87,11 +108,13 @@ owns that feedback. - [GCC candidate](/quest/m2/quic-realtime/gcc.md) - libwebrtc's GoogCC vendored into `moq-congestion` and refreshed to current WebRTC - [BBR3 delay bound](/quest/m2/quic-realtime/bbr-delay-bound.md) - BBR3 with inflight capped when the forward delay gradient shows queue growth - [SCReAMv2 candidate](/quest/m2/quic-realtime/scream.md) - the RFC 8298bis delay path ported to `moq-congestion` +- [io_uring runs an application's controller](/quest/m2/quic-realtime/uring-controller.md) - moq-uring takes a controller factory, so the winner installs the same way on both runtimes - [moqsink encoder feedback](/quest/m2/quic-realtime/moqsink-feedback.md) - GStreamer publishers retune their encoder to the target rate ## Related -- [Per-stream deadlines](/quest/m2/quic-deadline.md) - also consumes the forward-delay estimate +- [The shared pacer holds its rate under 2 Mbit/s](/quest/m1/pacer-low-rate.md) - every candidate paces through it, so its fix shifts their low-rate rows +- [Per-stream and per-datagram deadlines](/quest/m2/quic-deadline.md) - also consumes the forward-delay estimate - [BBR media study](/quest/m2/quic-bbr-study.md) - BBR3 behavior under media, the baseline here - [Measure ECN on the backbone](/quest/m2/quic-ecn-measure.md) - decides whether SCReAMv2's L4S path is worth porting - [QUIC on the sim](/quest/m2/quic-sim.md) - deterministic successor to the moq-shaper harness once it lands diff --git a/quest/m2/quic-realtime/uring-controller.md b/quest/m2/quic-realtime/uring-controller.md new file mode 100644 index 00000000000..bf0c248695e --- /dev/null +++ b/quest/m2/quic-realtime/uring-controller.md @@ -0,0 +1,18 @@ +# [S] io_uring workers run an application's own congestion controller + +## Goal + +moq-uring's QUIC transport config (`rs/moq-uring/src/quic/transport.rs`) takes a `moq_congestion::ControllerFactory` as well as its built-in `Congestion` families, and the relay's io_uring runtime installs the controller a `moq_tokio::quic::Config::controller` carries, instead of refusing that config. An application embedding the relay gets the same controller on either runtime. + +## Plan + +Follows [moq-dev/moq#7467](https://github.com/moq-dev/moq/pull/7467) (open), which adds `quic::Config::controller` to moq-tokio, a factory the QUIC backend builds one controller from per connection, and has the iroh backend and the relay's io_uring runtime refuse a config that carries one. This waits on #7467 landing. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, in this epic, because shipping `Realtime` "in moq-tokio and moq-uring" (the README's own work) is simpler once moq-uring takes a factory like moq-quic's `transport::Config::congestion` does: the winner becomes one factory both runtimes install, not a new enum variant per crate. + +- moq-uring's `Congestion` enum keeps its built-in families; add a factory variant or field, mirroring how #7467 shaped moq-tokio's, so the two crates' configs stay mirrored (the [moq-quic API](/quest/m1/quic-api/README.md) epic's rule). +- `rs/moq-relay/src/uring.rs` (the `transport.congestion` mapping, about line 500) passes the factory through and drops #7467's refusal. +- The io_uring workers are thread-per-core: the factory is shared across workers, and each connection's controller lives on its worker. State the factory's `Send + Sync` requirement in its docs. +- Test as #7467's `a_custom_controller_reaches_the_live_connection` does, on a ring worker: a counting factory builds the controller a live connection runs. Linux-only; run it in the `io-uring` nextest group. + +Public API: a factory option on moq-uring's QUIC transport config; the relay's io_uring runtime accepts `quic::Config::controller`. Wire: none. diff --git a/quest/m2/quic-sim.md b/quest/m2/quic-sim.md index daac805d9bc..090f89d1de3 100644 --- a/quest/m2/quic-sim.md +++ b/quest/m2/quic-sim.md @@ -39,5 +39,5 @@ queue is not evidence about congestion loss. ## Related -- [Per-stream deadlines](/quest/m2/quic-deadline.md) - a consumer +- [Per-stream and per-datagram deadlines](/quest/m2/quic-deadline.md) - a consumer - [QUIC bitrate caps](/quest/m2/rate-quic.md) - wants a simulated clock diff --git a/quest/m2/readers-range-live.md b/quest/m2/readers-range-live.md index ef239bac22d..93c0e0151f9 100644 --- a/quest/m2/readers-range-live.md +++ b/quest/m2/readers-range-live.md @@ -3,18 +3,35 @@ ## Goal The readers that wait for the largest group and raise their floor to -start near it (moq-audio, moq-video, `moq play`, and both moq-room and -`@moq/room` chat) subscribe with a relative range instead, with no change in -what they play: `Range::live()` for a start at the newest group, and a -relative start for a reader that looks back, such as moq-video's warmup lane, -which keeps its one-group lookback so the preceding intra-refresh group still -decodes. +start near it (moq-audio, moq-video, and both moq-room and `@moq/room` chat) +subscribe with a relative range instead, with no change in what they play: +`Range::live()` for a start at the newest group, and a relative start for a +reader that looks back, such as moq-video's warmup lane, which keeps its +one-group lookback so the preceding intra-refresh group still decodes. +`moq play` drops its floor entirely and starts where its consumers' `Delay` +puts them. ## Plan Since [moq-dev/moq#6278](https://github.com/moq-dev/moq/pull/6278), `Range::live()` starts at the live edge on a warm relay copy too. +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 +/quest-afk run): `moq play` drops its one-group `lookback` range floor and +relies on the `Delay` start instead. Since +[moq-dev/moq#7311](https://github.com/moq-dev/moq/pull/7311), its `open` +(`rs/moq-cli/src/play/media.rs`) passes each decoder's consumer a `Delay`: +the live decoder `{ min: video_delay, max: max_delay }` and the refresh +decoder `{ min + warmup, max + warmup }`, as `moq_video::decode::Consumer` +does. That start already decides where each decoder reads from, so the +`largest().await` plus `with_range(largest.group.saturating_sub(lookback)..)` +floor (and its copy in `play/video.rs`'s test helper `join`) is a second +start rule; #7311 kept it only to bound how long the refresh decoder's +start reach waits. Delete it rather than convert it to a relative range. +Check that the reason in its comment still holds without it (a warm relay +copy waiting on its source must not replay its backlog), and that the +refresh decoder still starts on the group below the live edge. + Inventory with `rg 'largest\(|poll_largest' rs/ js/` first, since Rust chat polls rather than awaits; keep any caller that needs the position itself rather than a start. Each reader keeps its existing tests, diff --git a/quest/m2/regrant-route-recheck.md b/quest/m2/regrant-route-recheck.md new file mode 100644 index 00000000000..65f0aa3b131 --- /dev/null +++ b/quest/m2/regrant-route-recheck.md @@ -0,0 +1,28 @@ +# [S] A regrant re-checks only the routes it can change + +## Goal + +When a peer's grant changes (a new `AUTH_OK`), a moq-lite announce stream's cost to apply it no longer grows as live routes times grant members per stream. A benchmark swept over announce streams, live routes, and grant size shows the per-stream regrant cost bounded by what the grant change can affect. Check whether `@moq/net`'s publisher re-matches its announcements the same way on a regrant, and bound it there too if so. + +## Plan + +Found by [moq-dev/moq#7536](https://github.com/moq-dev/moq/pull/7536), which swept the worst known peer grant shapes over `AUTH_OK` size and left this scope question to a follow-up (its decision 2, recommended there because it is route fan-out rather than grant algebra). On a regrant, each announce stream's `regrant` (`AnnounceRun` in `rs/moq-net/src/lite/publisher.rs`, about line 1045): + +- filters every live route through `permitted` (a match against every member of the new permit) to find what the new grant revokes, and +- when the grant grew, takes a fresh `origin.announced()` snapshot of every route under its prefix and matches each one it doesn't hold. + +So one `AUTH_OK` costs routes × members per stream, times K announce streams, a product none of [peer grant bound](/quest/m2/peer-grant-bound.md)'s options remove. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, its own quest beside the grant-algebra bound. + +- Derive the change once per `AUTH_OK`, not per stream: the members added and removed between the old and new grant. Each stream then re-checks only the routes under a changed member's prefix, using the origin's prefix structure to find them instead of walking every route. A member's prefix here is its literal segments before its first wildcard (`*` or `**`), since grant members are `Patterns` (`Permit` in `rs/moq-net/src/auth.rs`); a member that starts with a wildcard has no literal prefix and re-checks every route under the stream's prefix, as today. Bench that shape on its own row, and test wildcard members, removed members, and old and new grants that overlap. +- A grown grant needs only routes under the added members, not a whole-origin snapshot per stream. +- Keep today's semantics: a revoked route retracts, a newly covered route advertises, and a route both grants cover is untouched. +- Bench with #7536's grant shapes over announce streams and live routes, both swept; report before/after. + +Public API: none. Wire: none. + +## Related + +- [Bound peer grant processing by design](/quest/m2/peer-grant-bound.md) - may change the received-grant type this compares against; whichever lands second adapts +- [Announce interest streams share one route table](/quest/m2/announce-interest-memory.md) - the per-stream route state that this walk reads diff --git a/quest/m2/shaper-socket.md b/quest/m2/shaper-socket.md new file mode 100644 index 00000000000..03da4387618 --- /dev/null +++ b/quest/m2/shaper-socket.md @@ -0,0 +1,15 @@ +# [XS] moq-shaper's own socket never passes for path loss + +## Goal + +moq-shaper's sockets are sized like the relay's, so they don't drop at the rates drills run (480 Mbit/s in the socket-queues survey), and the shaper reports any drops at its own socket beside its path drops, so a drill that reads loss off the shaper attributes every drop correctly. + +## Plan + +Found by [moq-dev/moq#7529](https://github.com/moq-dev/moq/pull/7529)'s UDP queue-loss survey: moq-shaper binds its listen socket (`UdpSocket::bind` in `rs/moq-shaper/src/lib.rs`, `Shaper::bind`) at the kernel's 208 KiB default. Once in a shaped run it dropped 10 datagrams at its own socket, which showed as unattributed, since the shaper reports only its queue overflows; a drill would read them as path loss. #7529 also measured the relay's 8 MiB buffers absorbing a 100 ms stall with no drops, where 208 KiB dropped about 13% under 20 ms stalls. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. Do both halves, since they're each small: give the shaper's sockets the relay's buffer sizing (moq-sock's 8 MiB `UDP_BUFFER` sizing in `rs/moq-sock/src/bind.rs`; reuse it rather than a second constant), and count its own socket's `sk_drops` (`SO_MEMINFO`, as #7529's survey read it) into the shaper's stats so drills report them separately. + +Optional, only if it stays small: export per-socket `sk_drops` on the relay's `/metrics`. #7529 found nothing in production that needed it, so skip it if it needs new plumbing through both runtimes. + +Public API: a socket-drop counter in moq-shaper's stats. Wire: none. diff --git a/quest/m2/uring-bench-followups.md b/quest/m2/uring-bench-followups.md new file mode 100644 index 00000000000..9b1b3ea2ca2 --- /dev/null +++ b/quest/m2/uring-bench-followups.md @@ -0,0 +1,20 @@ +# [XS] Confirm io_uring quiescence on a quiet host and track its turn counts nightly + +## Goal + +Run-to-quiescence on the io_uring workers has acceptance numbers from a quiet host (fanout CPU per Gbps and p99 from `just bench-runtime`, against the tokio workers), and a QUIC request-reply variant of the `worker_turns` bench gives its turn and enter counts a nightly series, so a regression in batching shows up. + +## Plan + +Follow-ups of [moq-dev/moq#7530](https://github.com/moq-dev/moq/pull/7530), which runs a turn's tasks to quiescence before submitting. Its deterministic counts were the evidence (fanout 40.2 to 20.9 turns per round, -3.6% instructions); its `just bench-runtime` rows were indicative only, because host load put every runtime's p99 in seconds and moved the unchanged tokio rows by up to 8%. This waits on #7530 landing. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, one quest for both. + +- **Quiet re-run.** Record `/proc/loadavg` and run `just bench-runtime` for fanout and fanout-heavy, base against #7530, alternating. [moq-dev/moq#7533](https://github.com/moq-dev/moq/pull/7533) and [moq-dev/moq#7534](https://github.com/moq-dev/moq/pull/7534) saw the load generator's p99 sit near 1.4 s in every run, a startup artifact inside the throughput window, so check whether a quiet host alone fixes p99 or the generator's latency window needs to exclude startup first. Record the result in the [perf epic](/quest/m2/perf/README.md). +- **Nightly series.** #7530's harness (`rs/moq-uring/examples/quiescence/`, in its PR body, uncommitted) drove chat, bulk, and fanout shapes over QUIC and counted instructions, turns, and enters per round. Add a QUIC request-reply variant to the existing `worker_turns` bench (`rs/moq-uring/benches/worker_turns.rs`, already in `sh/gh/nightly.sh`'s bench list), reporting turns and enters per round as that harness did, so the nightly series covers it. + +Public API: none. Wire: none. + +## Related + +- [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - sets the threshold the new series is judged by diff --git a/quest/m3/README.md b/quest/m3/README.md index 6bba4f84f76..65639ecef44 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -25,6 +25,7 @@ m4's upstream waits in here and moved m2 work with no consumer. - [An Ada NVIDIA GPU is available](/quest/m3/ada-gpu.md) - the hardware AV1 NVENC is verified on - [NVENC AV1](/quest/m3/nvenc-av1.md) - AV1 encode through NVENC, once an Ada-generation GPU is available - [#2147](/quest/m3/2147-moq-video-10-bit-hevc-and-av1-support-in-the-nvidia-codec.md) - NVIDIA Main10 encode, once a 10-bit source exists in the pipeline +- [NVDEC and NVENC share CUDA arrays](/quest/m3/nvenc-cuda-arrays.md) - SDK 13.1 application-owned CUDA arrays drop the per-frame device copy, on driver 610+ - [Embedded video](/quest/m3/video-embedded.md) - verify a Pi 4/5 presents through the existing Vulkan and CPU paths, adding EGL import only if that fails - [Multi-plane PipeWire cameras](/quest/m3/pipewire-camera-planes.md) - I420 and NV12 cameras that deliver one memory block per plane capture - [Video codec coverage](/quest/m3/video-codec-coverage.md) - prioritize remaining native AV1 and portable decoder gaps once a consumer asks diff --git a/quest/m3/nvenc-cuda-arrays.md b/quest/m3/nvenc-cuda-arrays.md new file mode 100644 index 00000000000..c3e5a4a1e55 --- /dev/null +++ b/quest/m3/nvenc-cuda-arrays.md @@ -0,0 +1,24 @@ +# [M] NVDEC and NVENC share CUDA arrays without a per-frame copy + +## Goal + +On a driver that supports NVIDIA Video Codec SDK 13.1's application-allocated CUDA arrays, moq-video's NVDEC decoder outputs into, and its NVENC encoder reads from, CUDA arrays the application owns, so a decoded frame reaches the encoder (or a renderer) without today's device-to-device copy per frame. A benchmark shows the saving at 1080p and 4K, and older drivers keep the copy path. + +## Plan + +Found by the NVENC vendor audit in [moq-dev/moq#7464](https://github.com/moq-dev/moq/pull/7464), which kept NVENC API 12.1 and recommended this as separate, measured work. Today `rs/moq-video/src/decode/backend/nvdec.rs` maps each decoded NV12 surface and copies it with `memcpy_dtod_sync` into an owned `cuda::Frame` (about line 409) before the surface is unmapped. SDK 13.1 lets the application allocate the CUarray input and output, and needs driver 610 or newer. + +Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m3. It needs SDK 13.1 bindings and a driver-610 host to validate on, and no consumer has measured the copy as its bottleneck yet. + +- Bindings: add the SDK 13.1 entry points to `rs/moq-nvenc` behind a runtime capability check, keeping 12.1 as the floor the crate loads against. +- Decoder path: per the [SDK 13.1 NVDEC guide](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.1/nvdec-video-decoder-api-prog-guide/index.html)'s direct output to block-linear CUDA arrays, the decoder is created for an opaque block-linear output, the application's arrays (allocated with `cuArray3DCreate` and `CUDA_ARRAY3D_VIDEO_ENCODE_DECODE`) are registered as its output surfaces, and the `cuvidMapVideoFrame`, copy, and unmap steps go away. Event ordering alone removes no copy. List the decoder-creation and registration changes in the PR, following the SDK's `AppTransZeroCopy` sample. A renderer can't read the opaque layout as a pitch-linear frame, so state what a non-encoder consumer gets. +- Ownership: a bounded pool of CUDA arrays shared by decoder and encoder, with a frame holding its array until every consumer releases it, so the decoder can't overwrite a surface still being encoded. +- Synchronization: order decode completion before encode or render with CUDA events or stream ordering, not a blocking sync per frame. +- Validate on hardware: decoded pixels match the copy path, surface lifetime under drain and reset, and refusal (falling back to the copy) on an older driver. Measure frame-to-packet time and GPU memory bandwidth against the copy path; if it doesn't win, abandon the quest and report the numbers. + +Public API: possibly a CUDA-array frame variant in moq-video's GPU frame types. Wire: none. + +## Related + +- [NVENC reuses its input and output buffers](/quest/m2/nvenc-pool.md) - per-frame allocation on the encode side, distinct from this copy +- [GPU CI](/quest/m2/gpu-ci.md) - the NVIDIA host this would validate on once its driver is new enough