diff --git a/quest/m1/quic/README.md b/quest/m1/quic/README.md index 2221a214602..0f867694617 100644 --- a/quest/m1/quic/README.md +++ b/quest/m1/quic/README.md @@ -62,6 +62,7 @@ cache limits remain separate; bitrate pacing composes with this memory cap. ## Required +- [Regression net](/quest/m1/quic/coverage/README.md) - benchmarks, fuzzing, characterization tests, and a seeded simulation, so later QUIC changes see every behavior or cost change - [Bound message assembly](/quest/m1/quic/receive-memory.md) - complete headers and frames return credit, and receiver priority cancellation restores progress - [A reset group keeps its header](/quest/m1/quic/group-reset.md) - a lite publisher resets a group with `RESET_STREAM_AT`, so its subscriber drops the @@ -73,6 +74,7 @@ cache limits remain separate; bitrate pacing composes with this memory cap. datagrams join their subscription's send group instead of preempting every stream - [Relay peers get wider limits](/quest/m1/quic/peer-limits.md) - MAX_STREAMS and MAX_DATA are raised after SETUP identifies a cluster peer +- [Delete NewReno](/quest/m1/quic/new-reno.md) - the unused public controller goes ## Related diff --git a/quest/m1/quic/coverage/README.md b/quest/m1/quic/coverage/README.md new file mode 100644 index 00000000000..a585f2f0565 --- /dev/null +++ b/quest/m1/quic/coverage/README.md @@ -0,0 +1,54 @@ +# Regression net for moq-quic + +## Goal + +`moq-quic` has benchmarks, fuzzing, characterization tests, and a seeded +simulation dense enough that later QUIC work can refactor aggressively and +see every behavior or cost change as a failing test or a benchmark delta. + +The net pins today's behavior; it does not change it. A bug the net finds +lands as an `#[ignore]`d failing test naming a new follow-up quest, never as +an inline fix. + +## Plan + +Decided in the 2026-10-16 plan with the maintainer: + +- **Baseline is `origin/main` at b30289127**, before the rewrites + (#5711 scheduler, #5690 reliable reset, #5735 receive memory, #5688 + shard, #5674 peer limits) merge. #5690 and #5688 merged while this was + planned, so they take the merged-first comparison in + [the benchmarks](/quest/m1/quic/coverage/bench.md) and the + characterization tests are written against b30289127, then rebased. The net lands first so each rewrite can + compare against it: benchmarks via `just bench` against the commit that + lands them (Criterion only compares cases present at the base), tests by + rebasing, where every behavior change shows up as a failing test to + accept or fix. +- **Ranked first in the QUIC epic, but blocks nothing.** Every m1 QUIC + rewrite already has an open PR, and `Required` gates starting, not + merging. The net mainly protects the next wave of changes. +- **Benchmarks and characterization tests drive the public + `Connection`/`Endpoint` API** through the test `Pair`, so they compile on + both sides of a rewrite. Internal unit tests may break with a refactor and + get rewritten there. + +Survey on the baseline (462 tests, 1.4 s): 88.1% of production lines run, +but behavior is thin where the rewrites cut. RESET_STREAM is received 4 +times in the whole suite, STREAMS_BLOCKED_BIDI never, +`set_max_concurrent_streams` and `RecvStream::received_reset` are 0%, +`endpoint.rs` is 86.5%, frame error paths are unexercised, upstream +quinn's fuzz targets were never imported, there are no crate benchmarks, +and the harness defaults to Cubic while production runs BBR3. + +## Required + +- [Benchmarks](/quest/m1/quic/coverage/bench.md) - Criterion sweeps of stream scheduling, receive and reset, endpoint routing, and the wire codec, on the baseline +- [Coverage report](/quest/m1/quic/coverage/report.md) - `just rs coverage ` and a nightly lcov artifact, with no threshold +- [Characterization tests](/quest/m1/quic/coverage/characterize.md) - golden scheduling order, and every uncovered frame, reset, limit, and endpoint path, on today's harness +- [Fuzzing](/quest/m1/quic/coverage/fuzz.md) - a moq-net-style fuzz crate with the upstream parser targets, frame decode, and a connection op-sequence target, run nightly +- [Seeded simulation](/quest/m1/quic/coverage/sim.md) - a deterministic, impaired network in `Pair`, BBR3 as the harness default, and property tests over random workloads + +## Related + +- [Receive memory coverage](https://github.com/moq-dev/moq/pull/5752) - the end-to-end moq-tokio receive-window benchmark; the crate benchmarks here sit below it +- [Delete NewReno](/quest/m1/quic/new-reno.md) - the dead controller the survey found at 0% diff --git a/quest/m1/quic/coverage/bench.md b/quest/m1/quic/coverage/bench.md new file mode 100644 index 00000000000..e1b885065f0 --- /dev/null +++ b/quest/m1/quic/coverage/bench.md @@ -0,0 +1,38 @@ +# [M] Benchmark moq-quic + +## Goal + +`rs/moq-quic/benches/` holds Criterion benchmarks that each QUIC rewrite +compares against the code it replaced. They run in the nightly Criterion +smoke with the other crates. + +## Plan + +Decided 2026-10-16. Land this first in the epic, so the open rewrites can +compare against it. + +`just bench ` only compares cases that exist at ``, so the base +is the commit that lands this suite, not b30289127. Landed before a +rewrite, that commit is still pre-rewrite: the rewrite rebases and runs +`just bench `. For a rewrite that merges first, compare its +parent with this suite cherry-picked onto it against its merge commit with +the same suite, and record those numbers in this PR. + +- Drive a sans-IO `Pair` (or a bench-local equivalent built from the public + API) with no-op packet protection, ported from quinn `perf`'s + `noprotection.rs`, so only protocol CPU is measured. Use only API present + on the baseline and kept by the open rewrites, or the base comparison + cannot build. +- Cases, each swept over both of its axes so a cost that grows with the + table shows up as a slope: + - **Stream send and scheduling**: stream count x priority levels, and + bytes per write. This is what #5711 rewrites. + - **Receive, reassembly, and reset**: out-of-order ingest across many + partial streams, and reset and STOP_SENDING churn. Covers #5690 and + #5735. + - **Endpoint routing**: datagram dispatch over connections x CIDs per + connection. Covers #5688. + - **Wire codec and ACKs**: frame and packet encode and decode, and ACK + processing over large range sets. +- The existing `#[ignore]` print-only benchmarks (`bbr3/cleanup_bench.rs`, + `benchmark_receive_credit`) stay as they are. diff --git a/quest/m1/quic/coverage/characterize.md b/quest/m1/quic/coverage/characterize.md new file mode 100644 index 00000000000..45addc8f65f --- /dev/null +++ b/quest/m1/quic/coverage/characterize.md @@ -0,0 +1,44 @@ +# [M] Characterize moq-quic behavior + +## Goal + +Tests on today's harness pin the behavior the QUIC rewrites touch, so a +rebase onto any of them fails loudly wherever behavior changed. + +- **Scheduling**: golden tests of the exact order today's scheduler sends + stream bytes across priorities, equal-priority streams, retransmits, and + datagrams interleaved with streams. #5711 changes this order on purpose + and must update each golden in its own diff. +- **Frame error paths**: CRYPTO in 0-RTT, STREAM_DATA_BLOCKED or + MAX_STREAM_DATA on the wrong direction or an unopened stream, STOP_SENDING + on a receive-only or unopened stream, STREAMS_BLOCKED and MAX_STREAMS + above the limit (bidi included), NEW_CONNECTION_ID limit and retirement + errors, NEW_TOKEN and HANDSHAKE_DONE from a client, offsets at 2^62, and + final-size errors on data and reset. +- **Resets**: `RecvStream::received_reset`, reset codes, `Chunks` ending in + Reset or Finished, and connection credit returned on reset, stop, and + 0-RTT rejection. +- **Runtime limits**: `set_max_concurrent_streams`, `set_receive_window`, + and `set_send_window` raised and lowered mid-connection, including the + remote-bidi window raise in `set_params`. +- **Endpoint**: unknown and invalid CIDs, long headers for unknown + connections, short, undecodable, and wrong-version initials, the + `cids_exhausted` refusal, and a stateless reset from a client-only + endpoint. +- **Packets and paths**: the AEAD integrity limit, duplicate discard, + invalid Retry handling, 0-RTT retransmit after Retry, path-validation + failure reverting the path, and IPv4 rebinding through + `PathData::from_previous`. + +## Plan + +Decided 2026-10-16: the scheduling tests pin the exact old order rather +than only invariants, so every intentional change in #5711 is visible and +reviewed. If #5711 merges first, write the goldens against the baseline +(b30289127) anyway and update them on rebase, so the PR still shows the +diff. + +The survey's line numbers for each uncovered path come from +`cargo llvm-cov` on the baseline; [the coverage report](/quest/m1/quic/coverage/report.md) +reproduces them. This quest uses today's `Pair` as is; seeded impairments +belong to [the simulation](/quest/m1/quic/coverage/sim.md). diff --git a/quest/m1/quic/coverage/fuzz.md b/quest/m1/quic/coverage/fuzz.md new file mode 100644 index 00000000000..c9dc903556a --- /dev/null +++ b/quest/m1/quic/coverage/fuzz.md @@ -0,0 +1,27 @@ +# [M] Fuzz moq-quic + +## Goal + +`rs/moq-quic/fuzz` is a cargo-fuzz crate laid out like `rs/moq-net/fuzz`, +run nightly for 300 s per target, with every crash minimized into a +regression replayed by `just check` on stable. + +Targets: + +- Upstream quinn's `packet` (`PartialDecode`), `params` + (`TransportParameters::read`), `streamid`, and `streams` (an op sequence + against `StreamsState`), ported from its `fuzz/fuzz_targets/`. +- Frame decode through `frame::Iter`. +- A connection op sequence over a connected `Pair`: opens, writes, finishes, + resets, stops, runtime limit changes, datagrams, and arbitrary inbound + packets, checking that nothing panics and that delivered bytes match what + was written. + +## Plan + +Decided 2026-10-16 to mirror moq-net: one-line target shims call a hidden +`moq_quic::fuzz` module behind a `fuzz` feature (and `cfg(test)`), seeds +come from an example, and `regressions//` replays in the normal test +run. This replaces today's `cfg(fuzzing)` `pub mod fuzzing` and the implicit +`arbitrary` feature. The fuzz crate joins `no_fuzz` exclusions, the +`just rs fuzz` recipe, and the nightly `fuzz` matrix. diff --git a/quest/m1/quic/coverage/report.md b/quest/m1/quic/coverage/report.md new file mode 100644 index 00000000000..2d89bcdf71e --- /dev/null +++ b/quest/m1/quic/coverage/report.md @@ -0,0 +1,17 @@ +# [S] Report moq-quic coverage + +## Goal + +`just rs coverage ` prints per-file line and region coverage and +writes lcov and HTML, and a nightly job uploads the report for `moq-quic` +as an artifact. There is no threshold and no gate; the report is how later +quests find untested paths. + +## Plan + +Decided 2026-10-16. `cargo-llvm-cov` and LLVM's `llvm-cov` and +`llvm-profdata` join the Nix dev shell. The LLVM major version must match +rustc's (LLVM 22 for rustc 1.98); the survey ran +`nix shell nixpkgs#cargo-llvm-cov nixpkgs#llvmPackages_22.llvm` with +`LLVM_COV` and `LLVM_PROFDATA` pointed at those tools. Derive the version +from the toolchain rather than hardcoding it, if the flake allows. diff --git a/quest/m1/quic/coverage/sim.md b/quest/m1/quic/coverage/sim.md new file mode 100644 index 00000000000..2b5124000b1 --- /dev/null +++ b/quest/m1/quic/coverage/sim.md @@ -0,0 +1,34 @@ +# [L] Seeded moq-quic simulation + +## Goal + +The `Pair` harness in `rs/moq-quic/src/tests/util.rs` runs a deterministic, +impaired network: one seed reproduces a whole run, including loss, +duplication, reordering, and jitter in both directions. Property tests run +random workloads over it and assert that every byte arrives once and in +order, flow-control and stream limits hold, priority is honored, and the +connection closes cleanly. A failing case prints its seed. + +The harness defaults to BBR3, the production controller, instead of Cubic. + +## Plan + +Decided 2026-10-16: + +- **Determinism**: one seed flows into the CID generator, the reset key, + token randomness, and BBR3's `probe_rng_seed` (which picks the ProbeBW + wait and Reno-round bound), which today use `rand::rng()` and OS entropy. TLS key + shares may stay nondeterministic if no observable behavior depends on + them; otherwise use a deterministic test crypto provider. Ports stop + depending on test order. +- **Impairments**: a seeded model in `drive_client` and `drive_server`, + plus the bottleneck in the server-to-client direction, which today exists + only client-to-server. +- **BBR3 default**: flip `cubic_transport()` in the harness to BBR3. A test + that asserts Cubic-specific timing pins Cubic explicitly. A test that + fails under BBR3 for a real bug stays on BBR3, `#[ignore]`d with a + follow-up quest, rather than being hidden on Cubic. + +## Related + +- [Characterization tests](/quest/m1/quic/coverage/characterize.md) - targeted tests on today's harness, which can land before this diff --git a/quest/m1/quic/new-reno.md b/quest/m1/quic/new-reno.md new file mode 100644 index 00000000000..cf882d92df3 --- /dev/null +++ b/quest/m1/quic/new-reno.md @@ -0,0 +1,14 @@ +# [XS] Delete NewReno + +## Goal + +`moq_quic::congestion::{NewReno, NewRenoConfig}` and `new_reno.rs` are +gone. MoQ names congestion families, never algorithms, and no family maps +to NewReno, so the controller is never instantiated and is 0% covered. + +## Plan + +Found by the 2026-10-16 coverage survey; kept out of the coverage epic +because the net does not change behavior. A breaking public API removal, +so it belongs to the m1 cut and needs a MIGRATE.md entry. Update the +`congestion_controller_factory` docs in `config/transport.rs` that name it.