diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ae7d00caf4..00de85d7648 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -94,7 +94,7 @@ For an advisory against Quinn, follow the advisory triage in [`rs/moq-quic/READM - Smoke clients call only the latest published APIs. A release cut updates `test/smoke/clients` to the APIs being published, then verifies the affected smoke lanes. - A release is cut by hand: a PR merging `main` into `release`, with a merge commit. After the pre-cut merge of `release` into `main`, run `just version 0.22.0` on `main` for the first shared cut, or the next shared minor for later cuts. Commit the resulting versions and lockfiles before opening the cut PR. Run `just version` to check them. - A breaking cut first writes, on `main`, each changed package's version-transition section in its `MIGRATE.md` (`## 0.3 -> 0.4`) from the migration audit: the `!` subjects since the `release` merge base, each of those PRs' Impact section, and the API diff. -- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. +- Suggest a `release` backport for any serious bug fix (crash, security, data loss, broken interop), whether found on `main` or reported against a release. Land it on `main` first, then port it onto `release` as a separate PR that keeps the published API, so it ships as a patch. A backport quest or PR names "the next release", never a version number, since the `release` bump can land first. The Release gate's API diff (`just api-diff --deny`) fails a backport that changes one; on the cut PR, whose head is `main`, it only reports. - A bench-only backport ports `main`'s benchmarks onto `release`, so a pairing against `release` (`just bench origin/release`) compares them. It carries only benchmarks and their harness: bench targets, bench-only dev-dependencies, and `bench/`. It changes no shipped code, and it shims locally any API that `release` lacks, keeping each case's ID the same as on `main`. A case that measures code `release` doesn't have is left out, and the PR lists it. Merging one makes release-plz propose patch bumps for the crates it touches, though no shipped code changed. - After any release publishes, patch or breaking, run `just test wire-compat --remeasure` on `main` and land the rewritten `test/interop/compat/planned-breaks.json` with refreshed reasons. Otherwise the nightly wire-compat check fails on the stale pins. - Nothing merges `release` into `main` automatically. Before cutting a minor or breaking release, the maintainer merges `release` into `main` once, so trunk carries the published versions, CHANGELOGs, and any `release`-only commits. It lands as a merge commit with `--admin`, never squashed, so the cut starts from an advanced merge base. diff --git a/quest/m0/README.md b/quest/m0/README.md index cd89717b08b..91d3434c33c 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -29,6 +29,8 @@ edge, which embeds moq-relay; it moved there on 2026-09-28. - [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break - [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the scheduled Interop and Video workflows pass, so red means a new break - [Publish the release line's drain() wrappers](/quest/m0/ffi-shutdown-backport.md) - Dart `moq` 0.1.1 reaches pub.dev from `release` +- [Release's audio encoder rolls groups before the cap](/quest/m0/release-audio-group-cap.md) - a long `groupDuration` with short frames never throws `GroupTooLarge` on `release` +- [Local checks pass on unchanged main](/quest/m0/local-checks-main.md) - `just ci` and `just check` pass on a clean `main`, so a local gate fails only for its own change - [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in GitHub's merge queue, one tested entry at a time, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last - [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one drops a queued PR - [Tests can't wait on the real clock](/quest/m0/clock-guard/README.md) - a test whose real-clock timer fires fails every run, in JS and, if the spike holds, Rust diff --git a/quest/m0/local-checks-main.md b/quest/m0/local-checks-main.md new file mode 100644 index 00000000000..da73e5914d1 --- /dev/null +++ b/quest/m0/local-checks-main.md @@ -0,0 +1,32 @@ +# [S] Local checks pass on unchanged main + +## Goal + +`just ci` and `just check` pass in a clean checkout of `main` on the +maintainer's desktop, so an agent's local gate fails only for its own change. + +## Plan + +Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`). Several +`/quest-iterate` agents this session saw the same local failures on unchanged +`main`: + +- `test_interop_consumers_survive_collection` in + `test/interop/staging.test.py`. Pullfrog's + [8f2668b5ab](https://github.com/moq-dev/moq/commit/8f2668b5ab64a5c5a046f3f685e6581d4dbcaec9), + pushed to [moq-dev/moq#8001](https://github.com/moq-dev/moq/pull/8001) and + left out of it, fixes two causes: `bare-fin` in `test/justfile` builds + moq-net without `async`, which its async-gated `read_frame` needs, and the + staging check seeds only the examples named in `$EXAMPLES`, missing + `interop-socket-js`. Port it as is. +- The direnv fixture failures and the supervised JavaScript executable lookup + ([moq-dev/moq#7981](https://github.com/moq-dev/moq/pull/7981)'s agent + reproduced both on unchanged `main`). +- The CARLA launcher environment assertion + ([moq-dev/moq#7986](https://github.com/moq-dev/moq/pull/7986)'s agent). + +Reproduce each on a clean `main` first and keep the output; fix each at its +cause. A failure that only reproduces under load joins +[More tests under load](/quest/m0/test-flakes-2/README.md) instead. + +Public API: none. Wire: none. diff --git a/quest/m0/release-audio-group-cap.md b/quest/m0/release-audio-group-cap.md new file mode 100644 index 00000000000..ddf4148f525 --- /dev/null +++ b/quest/m0/release-audio-group-cap.md @@ -0,0 +1,24 @@ +# [S] Release's audio encoder rolls groups before the cap + +## Goal + +On `release`, `@moq/publish`'s audio encoder starts a new group before a group +reaches `MAX_GROUP_FRAMES` or `MAX_GROUP_CACHE_BYTES`, so a long +`groupDuration` with short frames never throws `GroupTooLarge`. It ships in +the next release as a patch. + +## Plan + +Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`), from +[moq-dev/moq#7279](https://github.com/moq-dev/moq/pull/7279), which shipped +`groupDuration` in `@moq/publish` 0.5.5: a `groupDuration` of tens of +seconds with 2.5 ms Opus frames passes `release`'s per-group frame cap and +throws. + +`main` has no group cap ([unbounded groups](/quest/m2/unbounded-groups/README.md)), +so this is a fix on `release` only, a PR into `release` with no `main` +change. Roll the group at the cap the way `js/json`'s window encoder does on +`release` (`MAX_GROUP_FRAMES`, kept below the net cap), and test a long +`groupDuration` with 2.5 ms frames. + +Public API: none. Wire: none. diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index d86985652eb..50340c0762a 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -114,3 +114,4 @@ Public API: none. Wire: none. - [io_uring tests fit the shared locked-memory budget](/quest/m0/test-flakes-2/uring-memlock.md) - concurrent moq-relay and moq-uring test runs stop exhausting the user's `RLIMIT_MEMLOCK`, fixed at the cause rather than by raising the limit - [The FFI concurrent-ready test wakes both calls under load](/quest/m0/test-flakes-2/ffi-concurrent-ready.md) - `concurrent_ready_calls_all_wake` stops stalling its reader drain under load and is re-enabled - [lite-05 copy fill checks idleness without wall time](/quest/m0/test-flakes-2/copy-fill-idle.md) - the copy-fill test proves no FETCH went out on the `Manual` clock, not a 200 ms real-time wait +- [OBS TSan runs report no leaked FFI worker thread](/quest/m0/test-flakes-2/obs-tsan-thread-leak.md) - the OBS TSan tests stop flagging a finished `moq-ffi-worker` thread at exit under load diff --git a/quest/m0/test-flakes-2/obs-tsan-thread-leak.md b/quest/m0/test-flakes-2/obs-tsan-thread-leak.md new file mode 100644 index 00000000000..1d34ac62682 --- /dev/null +++ b/quest/m0/test-flakes-2/obs-tsan-thread-leak.md @@ -0,0 +1,26 @@ +# [S] OBS TSan runs report no leaked FFI worker thread + +## Goal + +The OBS TSan unit tests (`Nightly (OBS TSan)` in +`.github/workflows/nightly.yml`) stop reporting a `thread leak` of a finished +`moq-ffi-worker` thread at exit on a loaded host, fixed at the cause under the +[epic](/quest/m0/test-flakes-2/README.md)'s rules. + +## Plan + +Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`), from +[moq-dev/moq#7996](https://github.com/moq-dev/moq/pull/7996), whose agent +saw the report on a loaded host while fixing the stop-drain test. + +`moq-ffi-worker` is the tokio runtime's worker thread in +`rs/moq-ffi/src/ffi.rs`. Lead, unverified: TSan flags a thread that finished +but was never joined, so the runtime's workers may outlive the last FFI call +without a join at shutdown. Check whether the test exits before the runtime +shuts down, or whether moq-ffi's shutdown detaches its workers, before choosing +between joining them on shutdown and changing the test's teardown. + +Reproduce first: run the OBS TSan tests repeatedly beside `just check --all` +load and keep the failing output. + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index ebc75594efe..02f0add849b 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -48,6 +48,8 @@ the same reason, gates the cut through the - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor - [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - an unmute is heard from its first frame with no catalog round trip: mute moves from holding a disabled rendition to suppressed silence with comfort noise, after a route spike; capture delivers frames on a steady cadence - [Push-to-talk's press-to-audible latency fits a budget on the defaults](/quest/m1/push-to-talk-latency.md) - press to audible on the browser defaults, excluding device output, stays under a kixtop-measured budget that `just test push-to-talk` enforces +- [Socket endpoints split into servers and clients](/quest/m1/socket-roles.md) - servers announce `.server/` and clients `.client/`, so plain grants keep a client from posing as a server or reading another client +- [CARLA's driver ids stay private](/quest/m1/carla-driver-ids.md) - 128-bit driver ids, and `status` shows a handle that isn't the endpoint id - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [Main is no slower than release](/quest/m1/release-perf/README.md) - every benchmark where main is slower than release is fixed or accepted with a reason, checked by a final paired run before the cut diff --git a/quest/m1/carla-driver-ids.md b/quest/m1/carla-driver-ids.md new file mode 100644 index 00000000000..37f8b2a2ef0 --- /dev/null +++ b/quest/m1/carla-driver-ids.md @@ -0,0 +1,31 @@ +# [S] CARLA's driver ids stay private + +## Goal + +A CARLA driver's endpoint id is 128 random bits, like a boy viewer's, and the +truck's public `status` track shows a handle that isn't the endpoint id. So no +visitor learns a lease holder's id from `status`, and none can announce it to +disrupt that driver's next session. + +## Plan + +Decided 2026-11-06 (maintainer, `/quest-complete` on +[moq-dev/moq#8011](https://github.com/moq-dev/moq/pull/8011)): let #8011 +land and fix this after it. Not chosen: fix it in #8011 first; accept it as a +cooperative demo. + +After #8011, a driver id is `driver-<12 hex>` (48 bits), the lease binds to +`(driver_id, session)`, and `status` echoes `"driver": "driver-ab12cd"`. +Every viewer reads `status`, and two announcements of one path coexist with +the relay picking a route, so an impostor that announces a leaked id can win +the route to it. It can't drive (the session id stays secret), but it can break +the holder's next dial. + +- Generate 128-bit driver ids in the web driver, as `js/moq-boy` does. +- Publish a display handle in `status`, such as a short hash of the session or + a per-lease counter, that the truck maps to its lease. `driver` and + `estop_by` keep showing who holds the truck. +- If [socket roles](/quest/m1/socket-roles.md) lands first, the ids live + under `.client/` and need no `driver-` prefix. + +Public API: none. Wire: none. diff --git a/quest/m1/socket-roles.md b/quest/m1/socket-roles.md new file mode 100644 index 00000000000..1a70610028b --- /dev/null +++ b/quest/m1/socket-roles.md @@ -0,0 +1,72 @@ +# [L] Socket endpoints split into servers and clients + +## Goal + +A socket endpoint is a server, a client, or both, and the relay's ordinary +grants enforce which. A server announces `/.server/` and accepts +dials; a client announces `/.client/` and dials. A client's grant +can't announce as a server or read another client's endpoint, so roles come +from the grant rather than an id prefix such as boy's `viewer-*`. The +symmetric `.socket/` layout is gone from moq-net (Rust, JS, Python through +moq-ffi), the [socket concept page](/doc/concept/socket.md), the interop +clients, moq-boy, and CARLA. + +| Role | Publishes | Subscribes | +| --- | --- | --- | +| Server | `.server/` | `.client/**` | +| Client | `.client/` | `.server/**` | + +Non-goals: stopping a client from announcing another client's id once it learns +it. Ids stay unguessable secrets until the m3 [mesh](/quest/m3/mesh/README.md) +makes them public keys; [CARLA's driver ids](/quest/m1/carla-driver-ids.md) +stops CARLA leaking them. + +## Plan + +Decided 2026-11-06 (maintainer, follow-up planning in `/quest-complete`, raised +while landing [moq-dev/moq#8011](https://github.com/moq-dev/moq/pull/8011) and +asking how anonymous viewers are kept from posing as cars or emulators): + +- **What the split enforces?** The dialer still publishes: its half of a + session, `P/A/B/S`, lives in its own broadcast, and the acceptor's + subscription to it is the return-routability check. + - ✅ Two hidden segments, one per role: a server publishes its own and reads + clients', a client the reverse. Plain grants enforce it, with no relay + change. + - Relay-enforced roles on session track names (dial = `P//*/*`, + listen = `P/*//*`). + - Dial ids bound to the connection, assigned by the relay or held by the + first session to claim them: hard across a cluster. + - Rely on the mesh's public-key ids alone. +- **Names?** ✅ `.server` / `.client`, read as roles. Not chosen: `.accept` / + `.dial` (the API verbs); `.listen` / `.dial`. +- **The old layout?** ✅ Replace `.socket/`: one convention, and a peer + that both dials and accepts announces both broadcasts under one id. Not + chosen: keep `.socket` for symmetric peers beside the split. +- **Milestone?** ✅ m1, before sockets first publish from the cut. Not chosen: + m3 beside the mesh's public-key ids; m4. + +Lead, for the PR to confirm: an endpoint announces `.client/` at its first +dial and `.server/` at its first accept, so a pure client never announces a +server endpoint, and today's dial-reserves-its-half and +accept-from-the-first-accept rules carry over. The session track name, the +handshake, the return-routability check, and both protocols are unchanged; only +which broadcast holds each half moves (`P/A/B/S` in `.client/A` and in +`.server/B`). + +Update with it: + +- moq-boy (`rs/moq-boy`, `js/moq-boy`): the emulator is a server, viewers are + clients; drop `VIEWER_ID_PREFIX` and the emulator's refusal of `viewer-` + names. +- CARLA: trucks are servers, drivers clients; drop the `driver-` id prefix and + the worker's refusal of `driver-*` trucks. +- `demo/relay/prod.toml` and its documented `moq auth serve` line: anonymous + publish on `demo/boy/.client/*` and `demo/carla/.client/*`, subscribe on + their `.server/**`. +- The [hosted demo deploy](/quest/m3/release/demo-sockets-deploy.md) uses + these grants. + +Public API: breaks `moq-net`'s `socket` module, `@moq/net`'s `socket` export, +and moq-ffi's socket bindings wherever they name the layout. Wire: none (a +path convention), but endpoints on the old layout can't reach new ones. diff --git a/quest/m2/unbounded-groups/README.md b/quest/m2/unbounded-groups/README.md index 1157a11f5b9..5e8ba9843fe 100644 --- a/quest/m2/unbounded-groups/README.md +++ b/quest/m2/unbounded-groups/README.md @@ -183,3 +183,4 @@ the model layer (maintainer, 2026-10-30). - [Byte holes](/quest/m2/unbounded-groups/byte-holes.md) - segments inside a frame shed like front frames and flights start at `(frame, byte)` on the current lite wip; the frame cap goes - [JS byte holes](/quest/m2/unbounded-groups/js-byte-holes.md) - `@moq/net` mirrors the byte-offset wire and continues a frame cut off mid-way - [Compressed frames have no size cap and inflate as a bounded stream](/quest/m2/unbounded-groups/compressed-stream.md) - A compressed frame of any size publishes and is received in Rust and `@moq/net`, like a plain one +- [Run-based relay fills are paced](/quest/m2/unbounded-groups/run-fill-pacing.md) - FETCH runs and lite-07 range fills backpressure upstream once a reader loses a batch to eviction diff --git a/quest/m2/unbounded-groups/run-fill-pacing.md b/quest/m2/unbounded-groups/run-fill-pacing.md new file mode 100644 index 00000000000..a248499dcf1 --- /dev/null +++ b/quest/m2/unbounded-groups/run-fill-pacing.md @@ -0,0 +1,23 @@ +# [M] Run-based relay fills are paced + +## Goal + +A relay pacing a run-based fill (a moq-transport FETCH run, or a lite-07 +range fill) backpressures the upstream stream once it sees a downstream reader +lose a batch to eviction, so a slow reader of a long run stays within the +tail instead of falling off it. + +## Plan + +Added 2026-11-06 (maintainer, follow-up planning in `/quest-complete`), from +[moq-dev/moq#8017](https://github.com/moq-dev/moq/pull/8017), which paces a +batch fill's handler by its fastest reader. Run-based fills still read +upstream at full speed. + +Pace on evidence, not up front: an unpaced run is fine while every reader +keeps up, so start backpressure only after the relay sees a downstream +reader lose a batch to eviction, then hold the upstream stream as #8017 +holds a batch handler. Test a slow reader of a long FETCH run and of a +lite-07 range fill through a relay. + +Public API: none expected. Wire: none. diff --git a/quest/m3/release/README.md b/quest/m3/release/README.md index b817c0bbca5..460ca0d495f 100644 --- a/quest/m3/release/README.md +++ b/quest/m3/release/README.md @@ -45,3 +45,4 @@ production stability proven merely because upstream artifacts published. - [Runner client bump](/quest/m3/release/runner-client-bump.md) - the upstream interop runner's Rust client moves to the published moq-tokio, so it stops stamping every object - [Smoke clients use request and follow](/quest/m3/release/smoke-bindings-rename.md) - condition: a binding release ships the renames; then the native smoke clients move to them - [Relay loader adoption](/quest/m3/release/relay-loader-adoption.md) - once their pins carry it, embedding consumers load relay config through `Config::merge` and refuse obsolete spellings +- [moq.dev's hosted demos run on sockets](/quest/m3/release/demo-sockets-deploy.md) - after the cut, a maintainer deploys lite-07 and the socket grants so the hosted boy and CARLA demos connect diff --git a/quest/m3/release/demo-sockets-deploy.md b/quest/m3/release/demo-sockets-deploy.md new file mode 100644 index 00000000000..077d859ef13 --- /dev/null +++ b/quest/m3/release/demo-sockets-deploy.md @@ -0,0 +1,40 @@ +# [S] moq.dev's hosted demos run on sockets + +## Goal + +moq.dev's hosted MoQ Boy and CARLA demos work with the socket versions the +cut publishes: the relays accept lite-07, the auth server and the emulator's +and worker's tokens carry the socket grants, and the site's `` drops +its viewer prefix. + +## Plan + +Decided 2026-11-06 (maintainer, follow-up planning in `/quest-complete`): the +hosted demos ship from `release`, so they switch when the cut lands. Not +chosen: an m2 cut gate; deploying now in m0. + +Main's moq-boy ([moq-dev/moq#8008](https://github.com/moq-dev/moq/pull/8008)) +and CARLA ([moq-dev/moq#8011](https://github.com/moq-dev/moq/pull/8011)) talk +over sockets, which need lite-07 (or moq-transport) on every hop. Until +moq.dev's relays accept it, the released `` can't connect at all. +The grants below follow [socket roles](/quest/m1/socket-roles.md); if it +hasn't landed by the cut, use the `.socket/viewer-*` and `.socket/driver-*` +grants documented in `demo/relay/prod.toml` instead. + +## Waiting + +- Maintainer: after the cut publishes, deploy moq.dev: + - Relays accept lite-07 in `listen.version` (the opt-in + `moq-lite-07-wip` until the cut finalizes it). + - The auth server's anonymous rules match `demo/relay/prod.toml`: publish + `demo/boy/.client/*` and `demo/carla/.client/*`, subscribe their + `.server/**`, and no `demo/viewer/**`. + - The emulator's token publishes `demo/boy/.server/` and subscribes + `demo/boy/.client/**`; the CARLA worker's token publishes its trucks' + `.server/` endpoints and subscribes `demo/carla/.client/**`. + - The site's `src/components/boy.tsx` drops `prefix-viewer`. + + Then open both demos in a browser and drive a truck and a game; an agent can + drive Chrome for that check. + +Public API: none. Wire: none.