From 673299071a79d0090b80b7e098dfd295a9d028c0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 10 Nov 2026 15:55:23 -0800 Subject: [PATCH] quest: plan proofs and nightly checks for moq-net Adds ten quests from a /quest-plan on what could be proved about moq-net: an inductive routing invariant, a Tamarin model of the route key proof, whole-session fuzzing, Kani codec proofs, a JS replay of the fuzz corpus, mutation testing, Miri seeds with ThreadSanitizer, and trace validation for the resume, reconnect, and subscribe models. Moves route trace validation to m1 and adds a Coverage-table symbol check to the nightly models quest. Co-Authored-By: Claude --- quest/m1/README.md | 4 ++ quest/m1/model-checks-nightly.md | 12 ++++- quest/m1/route-inductive.md | 54 +++++++++++++++++++++ quest/m1/route-key-model.md | 55 ++++++++++++++++++++++ quest/{m4 => m1}/route-trace-validation.md | 15 +++++- quest/m1/session-fuzz.md | 55 ++++++++++++++++++++++ quest/m2/lite07-finalize.md | 7 +++ quest/m4/README.md | 8 +++- quest/m4/codec-kani.md | 44 +++++++++++++++++ quest/m4/fuzz-corpus-js.md | 39 +++++++++++++++ quest/m4/miri-seeds-tsan.md | 39 +++++++++++++++ quest/m4/mutants.md | 40 ++++++++++++++++ quest/m4/reconnect-trace-validation.md | 35 ++++++++++++++ quest/m4/resume-trace-validation.md | 39 +++++++++++++++ quest/m4/subscribe-trace-validation.md | 41 ++++++++++++++++ 15 files changed, 483 insertions(+), 4 deletions(-) create mode 100644 quest/m1/route-inductive.md create mode 100644 quest/m1/route-key-model.md rename quest/{m4 => m1}/route-trace-validation.md (60%) create mode 100644 quest/m1/session-fuzz.md create mode 100644 quest/m4/codec-kani.md create mode 100644 quest/m4/fuzz-corpus-js.md create mode 100644 quest/m4/miri-seeds-tsan.md create mode 100644 quest/m4/mutants.md create mode 100644 quest/m4/reconnect-trace-validation.md create mode 100644 quest/m4/resume-trace-validation.md create mode 100644 quest/m4/subscribe-trace-validation.md diff --git a/quest/m1/README.md b/quest/m1/README.md index bfcdf92eb96..26c944ed22c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -49,6 +49,10 @@ moves what remains of them once their claims merge. - [The route model covers a mesh with clients and an announce budget](/quest/m1/route-mesh-model.md) - every ordering on 4-5 relay meshes with two origins and clients, with an announce budget and convergence - [Seeded lite-07 meshes hold the route rules and an announce budget](/quest/m1/route-mesh-sweep.md) - the route rules and message budget checked on real sessions over random meshes - [A model of a relay chain's failover delivers every group once](/quest/m1/failover-chain-model.md) - route, resume, and subscribe composed across a relay chain whose hops fail +- [Route sweeps are checked against the TLA+ model](/quest/m1/route-trace-validation.md) - each `sweep_handoff` run is a behaviour of `RouteHandoff.tla`, with a trace format the other models reuse +- [The route rules hold from any state, not only the runs TLC reaches](/quest/m1/route-inductive.md) - Apalache checks an inductive invariant behind the route rules on meshes of up to five relays +- [A symbolic model proves a route key proof binds to its session and side](/quest/m1/route-key-model.md) - Tamarin proves no replay, reflection, or relaying of lite-07's Route Key Parameter, and gates the cut +- [A hostile peer's session cannot panic, leak, or disturb another broadcast](/quest/m1/session-fuzz.md) - whole-session fuzz targets for moq-lite and moq-transport with a bystander broadcast as the oracle - [A reused moq-transport alias refuses its track at once](/quest/m1/ietf-alias-refuse.md) - a SUBSCRIBE_OK on an alias already bound fails its readers and removes its entry without waiting on the peer's FIN acknowledgement - [Unused crates, scripts, and files are deleted](/quest/m1/unused-code.md) - moq-room, the moq-native tombstone, demo/throttle, sh/dart/publish.sh, demo/web's Solid deps, and unpublished packages' release notes go - [The recurring audits are fewer and carry plans, not results](/quest/m1/audit-merge.md) - m3's 13 audits become about five, each a Goal plus a short check-first list diff --git a/quest/m1/model-checks-nightly.md b/quest/m1/model-checks-nightly.md index 88603f8c8e2..b6e282bb7c2 100644 --- a/quest/m1/model-checks-nightly.md +++ b/quest/m1/model-checks-nightly.md @@ -5,8 +5,9 @@ Every TLA+ model in the tree runs in Nightly: each passing config must pass, and each open-finding config must still fail, so a fix that should promote its rule is noticed. A PR that changes code a model covers gets that model -rerun and resynced in review, so no model keeps passing on code that no -longer exists. +rerun and resynced in review, and renaming or deleting a function a model +covers fails `just check` until its skill names the new one, so no model +keeps passing on code that no longer exists. ## Plan @@ -30,5 +31,12 @@ skills. Coverage table, and the PR updates the model and its sync commit. - The periodic resync for changes that skipped review is a check-first item in the [test coverage audit](/quest/m3/audit-test-coverage.md). +- ✅ `just check` also fails when a function a `verify-*` skill's Coverage + table names no longer exists in the file the row names (decided 2026-11-10, + maintainer, a later /quest-plan the same day; rejected: the review rule + alone). Unlike the touched-file check rejected above, it fires only when + modelled code is renamed or deleted, which is when the table's mapping is + already wrong. Give the tables one parseable form (a file, then its + symbols) and test the check on a renamed symbol. Public API: none. Wire: none. diff --git a/quest/m1/route-inductive.md b/quest/m1/route-inductive.md new file mode 100644 index 00000000000..91dcece786b --- /dev/null +++ b/quest/m1/route-inductive.md @@ -0,0 +1,54 @@ +# [L] The route rules hold from any state, not only the runs TLC reaches + +## Goal + +An inductive invariant of the route model implies `LoopFree`, `NoRevival`, +`NoFalseEnd`, `NoRegression`, and `Truthful`. Apalache checks in Nightly that +it holds initially and that every step preserves it from any state satisfying +it, on each mesh shape of up to five relays. The rules then hold for runs of +any length and any number of changes, where TLC's pass covers one claim, two +revisions, and two session opens or closes. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): + +- ✅ An inductive invariant checked by Apalache (rejected: a TLAPS proof for + every mesh size, whose hand-written steps break on each model change, until + this invariant has settled; verifying the Rust itself in Verus or Lean, + which needs the code rewritten around the prover). +- ✅ m1 (rejected: m4), because a flaw in the routing design is cheapest to + fix before lite-07 is cut. It does not gate the cut: a flaw it finds becomes + a fix quest that joins [Finalize moq-lite-07](/quest/m2/lite07-finalize.md) + then. + +What it removes is the bound on the run, not on the mesh: sequence numbers, +revisions, and churn are unbounded, while the relay count and queue lengths +stay constants the config states. Say both bounds in the `verify-route` skill. + +- Start from the argument Babel and DUAL rest on: along selected next hops the + (seqno, distance) pair strictly improves, and every message in flight carries + a value no better than its sender's feasibility distance when it was sent. + Strengthen until Apalache finds no counterexample to induction. +- A counterexample to induction starts from a state that may be unreachable. + If TLC cannot reach it, the invariant is too weak: strengthen it. If it can, + replay it on `route::tests::Sim` as the skill says before calling it a bug. +- Apalache needs type annotations. Add them to the spec TLC already checks, + so both tools read one file and neither drifts. +- `NoCountToInfinity` stays out until + [the echo fix](/quest/m2/cluster-routing/routes/echo-order.md) lands; expect + the search for the invariant to find that path again. +- Run it through the `tla` recipes with Apalache from the flake. If five + relays does not fit Nightly, record the largest mesh that does and its time. + +Public API: none. Wire: none. + +## Required + +- [The route model covers a mesh with clients and an announce budget](/quest/m1/route-mesh-model.md) - the spec parametrised over mesh shapes that this invariant is stated on + +## Related + +- [The TLA+ models run nightly and reviews resync them](/quest/m1/model-checks-nightly.md) - the Nightly job this check joins +- [Route sweeps are checked against the TLA+ model](/quest/m1/route-trace-validation.md) - ties the same spec to the code diff --git a/quest/m1/route-key-model.md b/quest/m1/route-key-model.md new file mode 100644 index 00000000000..5c692d265e7 --- /dev/null +++ b/quest/m1/route-key-model.md @@ -0,0 +1,55 @@ +# [M] A symbolic model proves a route key proof binds to its session and side + +## Goal + +A Tamarin model of lite-07's Route Key Parameter proves that when an endpoint +accepts a key on a session, the holder of that key signed for that session +and that side of it. The attacker controls the network, runs its own sessions +with both endpoints, and holds other route keys. The draft states what each +transport binding's exporter must guarantee for the proof to hold, each +binding in Rust and JS is checked against that, and the model runs in Nightly. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): + +- ✅ Tamarin (rejected: no formal analysis; the proof is a hand-rolled + construction, the kind symbolic provers find reflection and relaying + attacks in). +- ✅ m1 and it gates the cut (rejected: ungated like the TLA+ models; m4). + The signed transcript freezes when lite-07 is cut, and a flaw in it can + only be fixed on the wire, so + [Finalize moq-lite-07](/quest/m2/lite07-finalize.md) lists this quest. + +The proof (`rs/moq-net/src/route/proof.rs`, `js/net/src/lite/`, the draft's +Route Key Parameter section) is an Ed25519 signature over the domain string, +one byte for the sender's side, and 32 bytes exported from the session. + +- Model the handshake as a session whose exporter secret only its two + endpoints know and no other session shares. That assumption is the result + to pin down: state it in the draft, then check each binding against it. + Cover at least TLS 1.3 only (an older TLS without extended master secret + lets one party give two sessions the same secret), resumption and 0-RTT, the + WebTransport session exporter, and a session reached through an intermediary + that terminates TLS. +- Lemmas: the protocol can run; an accepted proof implies its key's holder + signed on that session as that side (injective agreement); no reflection to + the sender; no replay on another session; a proof the attacker relays + between two sessions it terminates is rejected. +- Out of scope: an endpoint whose own key is compromised, and relays that + share one key on purpose, which the draft allows. +- A lemma that fails is a finding: fix the transcript in Rust, JS, and the + draft in this PR when the fix is clear, and bring the maintainer the wire + change when it is not. +- Keep the model beside the others (`rs/moq-net/verify/`), with a `verify-*` + skill in their shape and Tamarin run from the flake as the `tla` recipe + runs TLC. + +Public API: none unless a finding changes the proof. Wire: none unless a +finding changes the signed transcript. + +## Related + +- [The TLA+ models run nightly and reviews resync them](/quest/m1/model-checks-nightly.md) - the Nightly job and Coverage-table check this model joins +- [Routes and announces](/quest/m2/cluster-routing/routes/README.md) - files a peer's routes under the key this proves diff --git a/quest/m4/route-trace-validation.md b/quest/m1/route-trace-validation.md similarity index 60% rename from quest/m4/route-trace-validation.md rename to quest/m1/route-trace-validation.md index 2171bcf87bd..cf16769e271 100644 --- a/quest/m4/route-trace-validation.md +++ b/quest/m1/route-trace-validation.md @@ -11,7 +11,20 @@ resync the model. Decided 2026-11-10 (maintainer, /quest-plan), as the strongest guard against the route model drifting from the code ([moq-dev/moq#8447](https://github.com/moq-dev/moq/pull/8447)). -Route only for now; judge the other models after. + +Decided 2026-11-10 (maintainer, a later /quest-plan the same day): + +- ✅ m1 (rejected: staying in m4). The [mesh model](/quest/m1/route-mesh-model.md), + the [inductive invariant](/quest/m1/route-inductive.md), and the + [cache-aware routing proof](/quest/m2/cache-routing-proof.md) all build on + `RouteHandoff.tla`, so drift there costs the most. +- ✅ The other three models get trace validation too, each its own m4 quest + behind this one (rejected: "route only for now; judge the other models + after", decided earlier the same day). So the recorder's trace format and + the recipe that hands a trace to TLC are built here for + [resume](/quest/m4/resume-trace-validation.md), + [reconnect](/quest/m4/reconnect-trace-validation.md), and + [subscribe](/quest/m4/subscribe-trace-validation.md) to reuse. This is TLA+ trace validation (Cirstea, Kuppe, Loillier, Merz, 2024, used on CCF and etcd's raft): `route::tests::Sim` records each step as the model diff --git a/quest/m1/session-fuzz.md b/quest/m1/session-fuzz.md new file mode 100644 index 00000000000..47644142f28 --- /dev/null +++ b/quest/m1/session-fuzz.md @@ -0,0 +1,55 @@ +# [L] A hostile peer's session cannot panic, leak, or disturb another broadcast + +## Goal + +A fuzz target per protocol drives a whole moq-net session, as a relay runs +it, against a peer that does anything: any message in any order, on any +stream, with resets, stalls, and closes. It asserts that the session never +panics, that what it holds returns to its baseline once the peer is gone, and +that a bystander's broadcast on another session of the same origin delivers +every group unchanged. Nightly runs both targets, and `just check` replays +their regressions. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net). Today's fuzz targets stop at the codecs and stream framing; nothing +tests the rule in `AGENTS.md` that a misbehaving peer must never panic, cost +unbounded work, or affect other broadcasts. + +- ✅ moq-lite and moq-transport (rejected: moq-lite only). A relay accepts + both from untrusted peers, and a hostile peer picks the weaker one. The + input picks the version from those a server accepts. +- ✅ m1 (rejected: m4), since what it finds is in code that ships with the + cut. + +Shape: + +- The input decodes to a script of peer actions: open a stream of a type, + write a well-formed message with fuzzed fields or raw bytes, FIN, reset or + stop with a code, send a datagram, advance the clock, close. Run it over + the in-memory transport doubles on the mocked clock, single-threaded, so a + failing input replays exactly. +- The victim is one origin serving two sessions: the fuzzed peer, and a + bystander publishing and subscribing to its own broadcast. +- Oracles: no panic; the bystander's subscriber reads every group, in order, + byte for byte; after the peer closes and the clock runs past every timeout, + the origin's pool charges, live tasks, and open streams are back to what the + bystander alone accounts for. Unbounded work shows as libFuzzer's per-input + timeout, set low. +- Seed with transcripts recorded from the integration tests (announce, + subscribe, a group, a fetch), so the fuzzer starts deep in the state + machine instead of at SETUP. +- Follow `rs/moq-net/fuzz/README.md`: bodies in the `fuzz` module, a one-line + shim each, committed regressions, a row in `nightly.yml`'s matrix. Measure + whether five minutes reaches past setup and say what budget it needs. + +Out of scope: `js/net`, where a hostile server can only hurt the tab that +dialed it. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m4/wire-fuzz-coverage.md) - the socket-level unsafe a peer reaches +- [Peer grant bound](/quest/m4/peer-grant-bound.md) - one known cost a peer's message can impose diff --git a/quest/m2/lite07-finalize.md b/quest/m2/lite07-finalize.md index f284423a969..d1dc62c499d 100644 --- a/quest/m2/lite07-finalize.md +++ b/quest/m2/lite07-finalize.md @@ -117,6 +117,12 @@ Decided 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and r The explicit `Oldest` anchor rides [time addressing's model](/quest/m2/time-addressing/model.md), whose lite-07 encoding [lite-07 by start](/quest/m2/time-addressing/lite07.md) carries. +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): ✅ the cut waits for the symbolic model of the Route Key Parameter +(rejected: neither proof gating, as the TLA+ models do not; the inductive +routing invariant gating too, which sits behind the mesh model). The signed +transcript freezes at the cut and a flaw in it can only be fixed on the wire. + ## Required - [Routes and announces](/quest/m2/cluster-routing/routes/README.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the native hop list gone (only the temporary lite-06 compatibility chain remains) @@ -128,6 +134,7 @@ whose lite-07 encoding [lite-07 by start](/quest/m2/time-addressing/lite07.md) c - [Delete the old start bookkeeping](/quest/m2/subscribe-ranges/start-bookkeeping.md) - may change START per update on lite-07 - [Prove cache-aware routing](/quest/m2/cache-routing-proof.md) - may add the warm-route message to lite-07; leaves this list if the proof places it in a SETUP extension - [Route latency](/quest/m2/route-latency.md) - may add a latency sum to ROUTE; leaves this list if the maintainer takes the no-wire path +- [A symbolic model proves a route key proof binds to its session and side](/quest/m1/route-key-model.md) - may change what the Route Key Parameter signs, if the model finds a replay, reflection, or relaying attack ## Related diff --git a/quest/m4/README.md b/quest/m4/README.md index 811c81e723c..6da9dc34ea8 100644 --- a/quest/m4/README.md +++ b/quest/m4/README.md @@ -106,4 +106,10 @@ deleted. - [The dev shell builds Kotlin with Gradle 9](/quest/m4/kt-gradle-9.md) - `gradle_9` in the dev shell, so the Android target builds there - [A hung Python test fails instead of hanging the suite](/quest/m4/py-test-timeouts.md) - one pytest-timeout bound covers every `py/` test - [Chromium's Opus encoder stamps chunks from their input](/quest/m4/chromium-opus-timestamps.md) - waiting on Chromium; unlocks native `usedtx` for push-to-talk -- [Route sweeps are checked against the TLA+ model](/quest/m4/route-trace-validation.md) - each `sweep_handoff` run is a behaviour of `RouteHandoff.tla` +- [Resume sweeps are checked against the TLA+ model](/quest/m4/resume-trace-validation.md) - each resume sweep case is a behaviour of `ResumeReader.tla` +- [Reconnect runs are checked against the TLA+ model](/quest/m4/reconnect-trace-validation.md) - a seeded sweep of the connect policy whose runs are behaviours of `Handover.tla` +- [Subscription runs are checked against the TLA+ model](/quest/m4/subscribe-trace-validation.md) - a seeded sweep of one lite-07 subscription whose runs are behaviours of `SubscribeAccounting.tla` +- [Kani proves the varint codecs and length arithmetic for every input](/quest/m4/codec-kani.md) - bounded proofs of no panic, no overflow, and round-trip, nightly +- [js/net decodes the Rust fuzzers' corpus the same way](/quest/m4/fuzz-corpus-js.md) - nightly, both codecs agree on acceptance and re-encoding for every corpus input +- [No mutation of the route and resume code survives its tests](/quest/m4/mutants.md) - cargo-mutants nightly, failing on any survivor +- [Concurrent tests run under many Miri schedules and ThreadSanitizer](/quest/m4/miri-seeds-tsan.md) - shim-free race checks on kio and moq-net, nightly diff --git a/quest/m4/codec-kani.md b/quest/m4/codec-kani.md new file mode 100644 index 00000000000..47314f081f7 --- /dev/null +++ b/quest/m4/codec-kani.md @@ -0,0 +1,44 @@ +# [M] Kani proves the varint codecs and length arithmetic for every input + +## Goal + +Kani proves, for every input within a stated size and not a sampled few, that +moq-net's varint codecs never panic, never read past their input, and +round-trip every value, and that the length and bound arithmetic a peer's +numbers reach cannot overflow. The proofs run in Nightly on the same bodies +the fuzz targets run. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net). Milestone m4: it hardens code whose design is not in question. + +Kani is a bounded model checker for Rust (s2n-quic, Firecracker, and the +standard library use it). It runs the real code, so it cannot drift from it +the way a model can. + +- Scope: both varint codecs in `rs/moq-net/src/coding/varint.rs` (every `u64` + encodes and decodes back to itself at the length the draft gives; every + byte string up to the longest encoding either needs more input or decodes to + a value whose encoding is a prefix of it), the decoder's length-prefix and + cap checks, and the 2^61 - 1 timestamp bound. Add other integer arithmetic + a peer's numbers reach, such as stream credit and byte accounting, only + where a proof finishes in minutes. +- Whole message decoders stay with the fuzzers: Kani scales badly through + `Bytes`, `Vec`, and `HashMap`. +- ✅ Plain `#[kani::proof]` harnesses that call the existing `fuzz` module + bodies with `kani::any()` bytes (rejected: the Bolero framework, a + dependency whose one job here a ten-line harness does). +- Cost to the tree: one `cfg(kani)` check-cfg entry, and no shims around + anything, unlike loom. Only Nightly compiles the harnesses, so its job is + what keeps them building. +- A `just rs kani` recipe and a Nightly row, installing Kani as the fuzz job + installs cargo-fuzz. Each harness states its unwind bound. A proof that + stops finishing is narrowed in the open, never dropped or given a longer + timeout to hide it. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m4/wire-fuzz-coverage.md) - the fuzz targets and Miri run beside these proofs diff --git a/quest/m4/fuzz-corpus-js.md b/quest/m4/fuzz-corpus-js.md new file mode 100644 index 00000000000..5ee2a68ac13 --- /dev/null +++ b/quest/m4/fuzz-corpus-js.md @@ -0,0 +1,39 @@ +# [M] js/net decodes the Rust fuzzers' corpus the same way + +## Goal + +Every Nightly, `js/net` decodes each input in the Rust wire fuzzers' minimized +corpus and agrees with `moq-net` on whether it is accepted and on the bytes it +re-encodes to. A codec change made in one language and not the other fails +Nightly with the input that tells them apart. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net). Milestone m4. + +`AGENTS.md` requires a Rust wire change to update JS in the same PR, and +today only a hand-written corpus (`rs/moq-net/fixtures/lite.json`, replayed +by `js/net/src/lite/corpus.test.ts`) checks that they agree. The fuzzers +already produce thousands of inputs at the edges of the decoders; this reuses +them. + +- After a fuzz target minimizes its corpus, a small Rust example (beside + `rs/moq-net/examples/fuzz-seeds.rs`) writes one record per input: the bytes, + whether the target's decoder accepted them, and the re-encoding. A Bun test + reads the records and asserts `js/net` gives the same answer. +- Cover each target whose decoder `js/net` also has: the lite messages, + announces, both varints, paths, and the moq-transport messages JS speaks. + List the targets left out and why. +- A disagreement is a bug in one codec or the other. Fix it, and commit the + input to `fixtures/` so `just check` replays it in both languages from then + on. +- Where the two differ on purpose (a version only Rust speaks, a map whose + order is not stable), say so in the record's schema, not with a skip list + in the test. + +Public API: none. Wire: none. + +## Related + +- [Shared corpora live in each crate's fixtures/](/quest/m4/corpus-fixtures.md) - where the committed inputs live diff --git a/quest/m4/miri-seeds-tsan.md b/quest/m4/miri-seeds-tsan.md new file mode 100644 index 00000000000..95295f03401 --- /dev/null +++ b/quest/m4/miri-seeds-tsan.md @@ -0,0 +1,39 @@ +# [M] Concurrent tests run under many Miri schedules and ThreadSanitizer + +## Goal + +Nightly runs kio's and moq-net's multi-threaded tests under Miri across many +scheduler seeds, and their whole library test suites under ThreadSanitizer. +A data race or an ordering bug in the locks, atomics, wakers, and kio's +lock-free log fails Nightly, with no change to the code under test. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net). Milestone m4. + +- ✅ Shim-free tools only (rejected: Shuttle, and loom back for kio alone). + Loom was removed in [moq-dev/moq#7474](https://github.com/moq-dev/moq/pull/7474) + because it never found a race and needed a `cfg(loom)` shim around every + primitive; Shuttle needs the same shims. These two run the tests as they + are, so there is nothing to keep in sync. +- Miri: `sh/rs/miri.sh` runs each test once. Add a pass over the tests that + spawn threads with `-Zmiri-many-seeds`, each seed a different schedule and + a different weak-memory outcome. Choose the seed count from the measured + time, and widen the moq-net filter past `model::frame`, `model::group`, and + `path` where a module shares state across threads. +- ThreadSanitizer: a `just rs tsan` recipe builds std with the sanitizer + (`-Zsanitizer=thread -Zbuild-std`) and runs the kio and moq-net library + tests at near-native speed, reaching the tests Miri ignores for foreign + calls or cost. A suppression needs a comment naming the false positive. +- Both are random, not exhaustive: a pass is evidence, not a proof. Say so in + the recipe's doc line. +- Miri's GenMC mode would explore every schedule without shims, but does not + support `cargo miri test` today. It is not part of this quest. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m4/wire-fuzz-coverage.md) - adds moq-c, moq-uring, and moq-sock to the same Miri recipe +- [Drop kio's epoch fence](/quest/m4/crossbeam-pin-fence.md) - the ordering in kio's log these runs watch diff --git a/quest/m4/mutants.md b/quest/m4/mutants.md new file mode 100644 index 00000000000..868ed9bf3c5 --- /dev/null +++ b/quest/m4/mutants.md @@ -0,0 +1,40 @@ +# [L] No mutation of the route and resume code survives its tests + +## Goal + +cargo-mutants changes `rs/moq-net/src/route/` and +`rs/moq-net/src/model/resume.rs` one small edit at a time, and the tests fail +on every edit that changes behaviour. Nightly runs it and fails on any +survivor, so logic that lands without a test that would notice it breaking is +caught the next morning. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net). Milestone m4. Coverage says which lines ran; this says whether a +bug on those lines would fail anything, for the two modules behind most +production issues. + +- ✅ Nightly fails on any survivor (rejected: a report the test coverage + audit reads, which nothing forces anyone to open; loom faded the same way). + This PR first kills or excludes every survivor of the first run. +- A survivor is killed with a test, or excluded by a pattern in + `.cargo/mutants.toml` with a comment saying why the edit cannot change + behaviour. Prefer the config file over `#[mutants::skip]`, which needs a + dependency. An edit that makes the tests hang counts as caught. +- Run only the tests that exercise the two modules, with the sweeps on fixed + seeds: a mutant caught by one random seed and missed by the next would make + the job flaky. +- Shard it across a Nightly matrix (`--shard k/n`) and report the mutant + count, survivors found, and wall time in the PR. If the full run does not + fit Nightly, say what does and rotate the rest. +- The other modelled code (`reconnect.rs`, `lite/publisher.rs`, + `lite/subscriber.rs`, `tail.rs`) joins once this run's cost is known; say + in the PR whether it fits. + +Public API: none. Wire: none. + +## Related + +- [Test coverage audit](/quest/m3/audit-test-coverage.md) - the line-coverage reports this complements +- [Resume as producer replacement](/quest/m4/resume-redesign.md) - rewrites `resume.rs`; whichever lands second carries the other's survivors diff --git a/quest/m4/reconnect-trace-validation.md b/quest/m4/reconnect-trace-validation.md new file mode 100644 index 00000000000..c7e13d27115 --- /dev/null +++ b/quest/m4/reconnect-trace-validation.md @@ -0,0 +1,35 @@ +# [M] Reconnect runs are checked against the TLA+ model + +## Goal + +A seeded sweep of the client connect policy (`rs/moq-net/src/reconnect.rs`) +and the `moq_tokio::Connection` loop that drives it records every run, and +each is also a behaviour of `Handover.tla`: a change to handovers, probes, or +reported status that the model does not allow fails the sweep, without anyone +remembering to resync the model. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): every TLA+ model gets trace validation, not route alone. Milestone +m4, behind route's. + +- Reuse the trace format and the TLC recipe + [route trace validation](/quest/m1/route-trace-validation.md) builds, with + the `verify-reconnect` skill's Coverage table as the mapping from code to + model actions. +- `reconnect/tests.rs` holds fixtures, not a sweep. Add a seeded one on the + mocked clock that draws dial results, GOAWAYs, redirects, session ends, and + probe outcomes, following the repository's seed convention. It is worth + having without the model: check the skill's rules on each run directly too. +- The model also covers `moq-tokio/src/connection.rs`. Record that loop's + steps as well, or say why the policy's trace alone is enough. +- The open-finding configs describe behaviour the code still has. A trace + must be accepted by the spec those configs check, so a fix that removes a + finding shows up here as well as in Nightly. + +Public API: none. Wire: none. + +## Required + +- [Route sweeps are checked against the TLA+ model](/quest/m1/route-trace-validation.md) - the trace format and TLC recipe this reuses diff --git a/quest/m4/resume-trace-validation.md b/quest/m4/resume-trace-validation.md new file mode 100644 index 00000000000..dabe593fc8c --- /dev/null +++ b/quest/m4/resume-trace-validation.md @@ -0,0 +1,39 @@ +# [L] Resume sweeps are checked against the TLA+ model + +## Goal + +Every case of the resume sweep (`rs/moq-net/src/model/resume_model.rs`) is +also a behaviour of `ResumeReader.tla`: a change to how a reader resumes +across route switches that the model does not allow fails the sweep, without +anyone remembering to resync the model. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): every TLA+ model gets trace validation, not route alone, so none +can keep passing on code that has moved on. Milestone m4, behind route's. + +- Reuse the trace format and the TLC recipe + [route trace validation](/quest/m1/route-trace-validation.md) builds. The + sweep records each step as the model action it corresponds to (the + `verify-resume` skill's Coverage table is the mapping) with the variables + the model keeps, and TLC checks the trace is a behaviour of the spec. +- The sweep covers more than the model does (ranges, cache expiry, a second + instance). Restrict the validated cases to what the model covers or grow + the model, whichever costs less, and say which. +- Before starting, check [Resume as producer replacement](/quest/m4/resume-redesign.md): + it deletes the per-reader path this model describes. If it has landed or is + in flight, validate against the redesign's model instead, or delete this + quest with the old one. +- Nightly on the sweep's fresh seed (`test resume-model`); the merge gate + only if it stays fast on the fixed cases. + +Public API: none. Wire: none. + +## Required + +- [Route sweeps are checked against the TLA+ model](/quest/m1/route-trace-validation.md) - the trace format and TLC recipe this reuses + +## Related + +- [Resume as producer replacement](/quest/m4/resume-redesign.md) - rewrites the code this validates diff --git a/quest/m4/subscribe-trace-validation.md b/quest/m4/subscribe-trace-validation.md new file mode 100644 index 00000000000..4ca65440ba7 --- /dev/null +++ b/quest/m4/subscribe-trace-validation.md @@ -0,0 +1,41 @@ +# [L] Subscription runs are checked against the TLA+ model + +## Goal + +A seeded sweep of one lite-07 subscription, publisher and subscriber over the +in-memory transport, records every run, and each is also a behaviour of +`SubscribeAccounting.tla`: a change to drops, skips, updates, or group stream +handling that the model does not allow fails the sweep, without anyone +remembering to resync the model. + +## Plan + +Decided 2026-11-10 (maintainer, /quest-plan on what could be proved about +moq-net): every TLA+ model gets trace validation, not route alone. Milestone +m4, behind route's. + +- Reuse the trace format and the TLC recipe + [route trace validation](/quest/m1/route-trace-validation.md) builds, with + the `verify-subscribe` skill's Coverage table as the mapping from code to + model actions. +- The model's steps are spread over `lite/publisher.rs` (`TrackRun`, + `GroupServe`), `lite/subscriber.rs` (`ServeLoop`, `TrackEntry`), and + `tail.rs`, on two sides of a session. Record both sides into one trace + ordered by the mocked clock and the transport's delivery order. +- No seeded sweep of this path exists. Add one that draws group arrivals, + give-ups, updates, resets, expiry, and the order the Subscribe Stream and + Group Streams are delivered in, following the repository's seed convention, + and check the skill's rules on each run directly too. +- [Shared group serving and ingest](/quest/m4/shared-serve.md) moves the + serving loop both protocols share. Whichever lands second re-maps the + Coverage table. + +Public API: none. Wire: none. + +## Required + +- [Route sweeps are checked against the TLA+ model](/quest/m1/route-trace-validation.md) - the trace format and TLC recipe this reuses + +## Related + +- [Shared group serving and ingest](/quest/m4/shared-serve.md) - moves the code this validates