From 6b2ad6360dd66b7c1a60730f235c5bd9e36632b6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 27 Oct 2026 14:23:38 -0700 Subject: [PATCH 1/3] quest: unblock m0/m1 chains from the 2026-10-27 audit Co-Authored-By: Claude Opus 5.5 --- bench/README.md | 5 +++ quest/m0/ci/README.md | 4 +- quest/m0/ci/hakari.md | 16 ++++--- quest/m0/ci/land-bump.md | 5 ++- quest/m0/ci/land.md | 9 ++-- quest/m0/ci/mbx-r2-verify.md | 20 +++++---- quest/m0/ci/nightly-levers.md | 28 ++++++------ quest/m1/README.md | 10 +++-- quest/m1/announce-takeover.md | 26 ++++++++--- quest/m1/api-audit/wanted.md | 3 +- quest/m1/archive/hls-target.md | 5 +-- .../m1/audio-jitter-target/capped-budgets.md | 24 +++++++--- quest/m1/audio-jitter-target/watch.md | 7 ++- quest/m1/av-clock.md | 7 ++- quest/m1/binding-timestamps.md | 6 +++ quest/m1/buffered-ascending.md | 14 +++--- quest/m1/cluster-routing/README.md | 12 +++-- quest/m1/cluster-routing/federation.md | 5 +++ quest/m1/cluster-routing/grant-subject.md | 38 ++++++++++++++++ quest/m1/cluster-routing/route-epochs.md | 35 +++++++++++++++ quest/m1/cluster-routing/routes.md | 31 +++++++------ quest/m1/drop-set.md | 8 +++- quest/m1/drop-web-transport-trait.md | 17 +++++-- quest/m1/identity-guide.md | 17 ++++--- quest/m1/ietf-fill-timeout-js.md | 32 ++++++++++++++ quest/m1/ietf-fill-timeout.md | 27 +++++++----- quest/m1/interop-coverage/late-join.md | 5 ++- quest/m1/js-fetch-on-demand.md | 28 ++++++++++++ quest/m1/js-fetch.md | 32 +++++++------- quest/m1/lite07-finalize.md | 22 ++++++---- quest/m1/perf-host.md | 16 ------- quest/m1/quic-api/native-web.md | 1 - quest/m1/quic-api/poll.md | 1 - quest/m1/quic-api/streams.md | 14 +++--- quest/m1/read-backfill.md | 7 ++- quest/m1/subscribe-ranges/README.md | 36 +++------------ quest/m1/subscribe-ranges/fetch-range.md | 20 +++++---- quest/m1/subscribe-ranges/fill-linear.md | 27 +++++++++++- quest/m1/subscribe-ranges/js.md | 33 +++++++------- quest/m1/subscribe-ranges/lite.md | 28 +++++++----- quest/m1/subscribe-ranges/lite07-fetch.md | 35 +++++++++++++++ quest/m1/subscribe-ranges/range-cost.md | 24 ---------- .../m1/subscribe-ranges/start-bookkeeping.md | 44 +++++++++++++++++++ quest/m1/time-addressing/bindings.md | 3 +- quest/m1/time-addressing/e2ee.md | 8 ++-- quest/m1/time-addressing/js.md | 4 +- quest/m1/time-addressing/model.md | 14 +++++- quest/m1/unbounded-groups/README.md | 20 +++++---- quest/m1/unbounded-groups/byte-holes.md | 11 +++-- quest/m1/unbounded-groups/front-eviction.md | 23 +++++----- quest/m1/unbounded-groups/js-byte-holes.md | 31 +++++++++++++ quest/m1/unbounded-groups/js.md | 18 ++++---- quest/m1/unbounded-groups/log-lock-profile.md | 16 ++++--- quest/m1/watch-buffer-switch.md | 24 ++++++++++ 54 files changed, 658 insertions(+), 298 deletions(-) create mode 100644 quest/m1/cluster-routing/grant-subject.md create mode 100644 quest/m1/cluster-routing/route-epochs.md create mode 100644 quest/m1/ietf-fill-timeout-js.md create mode 100644 quest/m1/js-fetch-on-demand.md delete mode 100644 quest/m1/perf-host.md create mode 100644 quest/m1/subscribe-ranges/lite07-fetch.md delete mode 100644 quest/m1/subscribe-ranges/range-cost.md create mode 100644 quest/m1/subscribe-ranges/start-bookkeeping.md create mode 100644 quest/m1/unbounded-groups/js-byte-holes.md create mode 100644 quest/m1/watch-buffer-switch.md diff --git a/bench/README.md b/bench/README.md index 81ac8fb193d..e4314f2f031 100644 --- a/bench/README.md +++ b/bench/README.md @@ -184,6 +184,11 @@ refused rather than reported. sudo sysctl -w kernel.perf_event_paranoid=-1 ``` +The maintainer's 24-core `desktop` workstation is the profiling host: it +keeps `kernel.perf_event_paranoid=-1`. Profile there only while it is quiet, +with no other agents running and load near zero, since other work on the +host shows up as lock waits and CPU the relay did not cause. + ### Heap The heap mode builds the relay with `moq-relay/jemalloc` (the other modes keep diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index f71b5737ddb..2c991353a34 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -127,8 +127,8 @@ The historical samples locate candidates; they are not controlled benchmarks. ## Required - [Land main through quest land](/quest/m0/ci/land.md) - one self-hosted runner tests every queued PR as a squash chain and fast-forwards `main` to the longest passing prefix -- [Bump quest to quest land](/quest/m0/ci/land-bump.md) - condition: kixelated/quest ships `quest land`; then bump the flake input -- [Measure nightly compiler levers](/quest/m0/ci/nightly-levers.md) - the parallel frontend and Cranelift, adopted only if the numbers beat their risks +- [Bump quest to quest land](/quest/m0/ci/land-bump.md) - kixelated/quest shipped `quest land` in kixelated/quest#229; bump the flake input +- [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out - [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands - [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix - [Unify dependency features](/quest/m0/ci/hakari.md) - a cargo-hakari workspace-hack so every job compiles each dependency alike and mbx hits across jobs diff --git a/quest/m0/ci/hakari.md b/quest/m0/ci/hakari.md index 14f000c680a..16ae437e734 100644 --- a/quest/m0/ci/hakari.md +++ b/quest/m0/ci/hakari.md @@ -31,12 +31,14 @@ feature set in another job is a miss, however warm the cache. - Unifying features means some packages build features they don't use. Measure whether that costs more CPU than the cache hits save; target, profile, and `RUSTFLAGS` differences still split keys. -- Start from the hit report in [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md), - which lists the compilations that miss only because their features or flags - differ. +- Measure the baseline here, before generating the hack: across jobs, which + compilations one job misses although another job built the same crate at + the same version, grouped by what differs (features, profile, `RUSTFLAGS`, + target). Repeat it after, from the same commit. -Public API: none; the workspace-hack crate is not published. - -## Required +Decided 2026-10-27 (maintainer): this quest no longer waits on +[Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md). It measures its own +before and after baseline, so the cross-job miss report moved here from that +quest, which stays a waiting quest for the R2 numbers alone. -- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - its cross-job hit report is this quest's baseline +Public API: none; the workspace-hack crate is not published. diff --git a/quest/m0/ci/land-bump.md b/quest/m0/ci/land-bump.md index 28b36d986ea..c9b8bb0d241 100644 --- a/quest/m0/ci/land-bump.md +++ b/quest/m0/ci/land-bump.md @@ -3,11 +3,14 @@ ## Goal Condition: a kixelated/quest commit on `main` ships `quest land` -([kixelated/quest's plan](https://github.com/kixelated/quest/blob/main/quest/m0/helpers/land.md)). +([kixelated/quest#229](https://github.com/kixelated/quest/pull/229)). Check with `git ls-remote https://github.com/kixelated/quest main` and `quest land --help` from that revision. Then bump the `quest` flake input in `flake.nix` to that revision, and delete this quest. +The condition holds: kixelated/quest#229 merged as `195bcee560` on +2026-10-27, so only the bump remains. + ## Plan Decided 2026-10-26 (maintainer): the batched merge gate's engine lives in diff --git a/quest/m0/ci/land.md b/quest/m0/ci/land.md index 28bbaa26651..b8ce5f73da3 100644 --- a/quest/m0/ci/land.md +++ b/quest/m0/ci/land.md @@ -58,9 +58,10 @@ Decided 2026-10-26 (maintainer): on the head it tested, to that PR's commit, then fast-forwards `main` to the tip in one push. GitHub marks a PR merged once its head is on the base, so the PRs show as merged, `main` stays linear, and the landed tree is - exactly the tested one. A conflict outside `quest/` ejects that PR. The - engine quest decides whether a fork branch that blocks maintainer edits is - refused at enqueue or lands another way. + exactly the tested one. A conflict outside `quest/` ejects that PR, and so + does a fork branch that blocks maintainer edits, whose head the engine + cannot move: it is ejected as `no_maintainer_edits` + ([kixelated/quest#229](https://github.com/kixelated/quest/pull/229)). - Batches take every queued PR, with no size constant. On failure the gate tests the first half. A passing prefix lands at once and the rest continues. A PR that fails alone is ejected with a comment linking the @@ -103,4 +104,4 @@ Public API: none. Wire: none. ## Required -- [Bump quest to `quest land`](/quest/m0/ci/land-bump.md) - condition: kixelated/quest ships `quest land`; then bump the flake input +- [Bump quest to `quest land`](/quest/m0/ci/land-bump.md) - kixelated/quest shipped `quest land` in kixelated/quest#229; bump the flake input diff --git a/quest/m0/ci/mbx-r2-verify.md b/quest/m0/ci/mbx-r2-verify.md index ce0b0201b70..01515718628 100644 --- a/quest/m0/ci/mbx-r2-verify.md +++ b/quest/m0/ci/mbx-r2-verify.md @@ -20,10 +20,6 @@ and record the numbers in the PR that closes this: job pays that rebuild. - Bucket size and request counts after a month, against the 30-day expiry in `infra/mbx/lifecycle.json`. Shorten or lengthen it from that. -- Across jobs: which compilations one job misses although another job built - the same crate at the same version, grouped by what differs (features, - profile, `RUSTFLAGS`, target). This is the baseline for - [unifying dependency features](/quest/m0/ci/hakari.md). Use scheduled Nightly runs on `main` only (`gh run list --workflow nightly.yml --branch main --event schedule`), never a dispatch on a branch. @@ -47,16 +43,20 @@ Evidence so far, from the platform jobs of scheduled Nightlies effective read-write`. That proves credentials and policy, not reuse. - macOS refuses task manifests written past the reader's 16 MiB bound (`a90076a4...` at 17204435 bytes, then `5cb8fb79...` and `b0310ce4...`), so - those tasks get no remote predictions. No macOS summary downloads more than + those tasks get no remote predictions. Two more were refused before the + bump, both under mbx 1.25.0: `2ed9df47...` (17218002 bytes) on 2026-10-25 + ([40192013303](https://github.com/moq-dev/moq/actions/runs/40192013303)) + and 2026-10-26, and `0d4f4e3f...` (17290876 bytes) on 2026-10-26 + ([40332938121](https://github.com/moq-dev/moq/actions/runs/40332938121)). No macOS summary downloads more than 15 KiB. Reported as [jdx/mr-boxington#792](https://github.com/jdx/mr-boxington/issues/792) and fixed by [jdx/mr-boxington#798](https://github.com/jdx/mr-boxington/pull/798), which keeps the read bound and caps the writer instead. The Rust cache now pins 1.28.0, the first release carrying the fix. It does not repair manifests already oversized on R2: if task identities carry over, the - three known digests may still refuse until about 2026-11-22, when the - 30-day lifecycle deletes them. A refusal for any other digest means the - fix did not hold. + five known digests may still refuse until the 30-day lifecycle deletes + them, about 2026-11-22 for the first three and 2026-11-26 for the last + two. A refusal for any other digest means the fix did not hold. - Windows logs no refusal but still no reuse. Its first build reports 0 hits and 1084 of 1091 compilations "not looked up" for want of a matching prediction on 2026-10-22, after a 1.25.0 Nightly wrote the day before, and @@ -64,6 +64,10 @@ Evidence so far, from the platform jobs of scheduled Nightlies persists after the bump, find why before timing builds. - No release pull request has run with the R2 cache yet. +Decided 2026-10-27 (maintainer): the cross-job miss report moved to +[unifying dependency features](/quest/m0/ci/hakari.md), which measures its own +before and after baseline and no longer waits on this quest. + ## Related - [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - the same R2 and Environment bootstrap for Nix diff --git a/quest/m0/ci/nightly-levers.md b/quest/m0/ci/nightly-levers.md index 14261285a45..adc05fdf608 100644 --- a/quest/m0/ci/nightly-levers.md +++ b/quest/m0/ci/nightly-levers.md @@ -1,33 +1,31 @@ -# [M] Measure nightly compiler levers in the queue +# [S] Measure the parallel frontend in the queue ## Goal A report says whether building the queue's tests on a pinned nightly with the -parallel frontend and the Cranelift backend saves enough wall time to justify -its risks, measured after the structural queue changes land. If it does, the -queue adopts it, and Nightly keeps a stable LLVM build of the same tests as the -backstop. +parallel frontend saves enough wall time to justify its risks, measured after +the structural queue changes land. If it does, the queue adopts it, and +Nightly keeps a stable build of the same tests as the backstop. ## Plan Decided with the maintainer on 2026-10-23: a measured experiment, last. Estimated 2026-10-23 at 25 to 35% of the test build's compile, about 10% of -total queue time. Risks to measure, not assume: +total queue time, with Cranelift included. Risks to measure, not assume: -- Cranelift's unwinding against moq's 30 `#[should_panic]` tests and 16 - `catch_unwind` files. -- sonora's SIMD echo canceller, forced to opt-level 3 because its realtime - tests time out unoptimized; it may need a per-package LLVM backend. -- Intrinsics in `memchr`, `simd-adler32`, `crc32fast`, and `blake3`. - The parallel frontend's remaining incremental races and differing cycle errors. - The gate no longer compiles with the toolchain that ships, and nightly mbx keys split from Nightly's stable R2 writers. -kixelated/rust2's [build-speed benchmark](https://github.com/kixelated/rust2/blob/main/quest/m1/bench.md) -measures these levers on moq; use its numbers, and its explanation of nightly -incremental rebuilds costing near-full builds, before running the queue -experiment. +Decided 2026-10-27 (maintainer): moq keeps LLVM, accepting kixelated/rust2's +Cranelift verdict ([build benchmark](https://github.com/kixelated/rust2/blob/main/bench/build/README.md), +"Cranelift for moq's CI", 2026-10-25). 42 of moq's 5418 tests fail under +Cranelift (panics it cannot unwind, and an x86 SIMD intrinsic it lacks), and +the CPU saving is only 4% with a warm cache and 12% cold. This quest is +narrowed to the parallel frontend. Use the same benchmark's numbers, and its +explanation of nightly incremental rebuilds costing near-full builds, before +running the queue experiment. Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index 042df11f5c6..d6efb7f4888 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -136,15 +136,18 @@ surface and zero-copy texture import, land before the cut. - [moq-transport 18 and 22 only](/quest/m1/ietf-drop/README.md) - drafts 14-17 and 19-21 are deleted from Rust, JS, tests, interop lanes, and docs, removing the legacy control-stream adapter and its request limit - [moq-quic's API improves on quinn's](/quest/m1/quic-api/README.md) - moq-quic and both QUIC drivers ship one mirrored API: `send::`/`recv::` namespaces, one error per direction, consuming terminal operations, plain configs, no quinn baggage - [Drop web-transport-trait](/quest/m1/drop-web-transport-trait.md) - iroh, wasm, and ffi implement moq-net's `transport::poll` traits, and the remaining `web_transport_trait` adapters are deleted -- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface -- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native FILL_TIMEOUT bounds upstream waiting on drafts 18 and 22, alongside local content-age enforcement +- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer cached and joining IETF FETCH, with the draft-18 Largest fix +- [On-demand JavaScript FETCH](/quest/m1/js-fetch-on-demand.md) - browser publishers answer IETF FETCH for evicted or uncached groups through the JS ranges request surface +- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - Rust honors native FILL_TIMEOUT to bound upstream waiting on drafts 18 and 22, alongside local content-age enforcement +- [JS FETCH fill timeout](/quest/m1/ietf-fill-timeout-js.md) - the JS FETCH responder honors FILL_TIMEOUT as Rust does - [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` refuses a redefined rendition, and every Rust publisher mints a new rendition name for a decoder identity change and updates everything else in place - [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension -- [Identity and recovery guide](/quest/m1/identity-guide.md) - explain pinned catalogs, lazy jobs, takeover, restart and version compatibility +- [Identity and recovery guide](/quest/m1/identity-guide.md) - explain epochs, pinned catalogs, lazy jobs, restart and version compatibility as they exist; takeover adds its own section - [Native rung disable](/quest/m1/simulcast-rung-disable.md) - OBS stops sending the top renditions its grant cannot fund, as the Rust/FFI publishers do, and a shaped uplink through moq-ffi proves the resume - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; a manual browser proof and a native trace replay remain - [A/V clock](/quest/m1/av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust +- [Switching buffer at runtime](/quest/m1/watch-buffer-switch.md) - changing a player's `buffer` mid-playback switches the audio ring between live and buffered - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor - [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS @@ -158,7 +161,6 @@ surface and zero-copy texture import, land before the cut. - [App-limited BBR3 estimate ages](/quest/m1/bbr-app-limited.md) - an app-limited BBR3 estimate falls within seconds after the path drops - [Aborted epoch reads keep their cause](/quest/m1/epoch-abort-cause.md) - an aborted read through an epoch route ends with its own error, not NotFound - [Bindings name every session code](/quest/m1/ffi-malformed-auth-kind.md) - MALFORMED_AUTH_TOKEN and its siblings reach moq-ffi, moq-c, and the wrappers -- [A profiling host](/quest/m1/perf-host.md) - condition: a quiet host with `perf_event_paranoid=-1`, recorded in `bench/README.md` - [Unbounded groups](/quest/m1/unbounded-groups/README.md) - groups and frames lose their size caps; the cache sheds parts of a group under pressure and refills the holes in place - [Messages borrow instead of owning](/quest/m1/borrowed-messages.md) - message structs borrow from the wire on decode and from callers on encode, after lifetime-free `Path` - [Bindings decode media as media](/quest/m1/obs-data-decode.md) - moq-ffi decodes a media rendition with its kind, so OBS stops losing every other group diff --git a/quest/m1/announce-takeover.md b/quest/m1/announce-takeover.md index 92c54595791..ab35bd09fc5 100644 --- a/quest/m1/announce-takeover.md +++ b/quest/m1/announce-takeover.md @@ -65,12 +65,23 @@ fallback, including the accepted cold-start reset before this lands. On older wires, translate this optimization into ordinary announcements and required Restart rather than changing published framing. -Implement against lite-07-wip and reconcile the planned ROUTE/ANNOUNCE split: -reuse its routing fields and identity model rather than adding a second one. -Account for TAKEOVER in announcement IDs, compression bases, snapshot counts, -route filtering, duplicate-advertisement rules and retirement. Update the -lite draft and existing identity/compatibility docs in the implementation PR. -The dedicated guide follows separately; it does not replace inline docs. +Decided 2026-10-27 (maintainer): this no longer waits on +[Routes and announces](/quest/m1/cluster-routing/routes.md), only on +[Route epochs](/quest/m1/cluster-routing/route-epochs.md). Takeover builds on +what lite-07-wip and `main` already have: stream-local Announce IDs, +ANNOUNCE_RESTART, Base compression, `Request::prepare`/`publish`, and the +claim helpers. Its ANNOUNCE_START field layout overlaps with routes (hops +become a Route ID), so whichever of the two lands second rebases onto the +other. + +Implement against lite-07-wip, and reuse the routing fields and identity +model of whichever ROUTE/ANNOUNCE shape is on `main` rather than adding a +second one. Account for TAKEOVER in announcement IDs, compression bases, +snapshot counts, route filtering, duplicate-advertisement rules and +retirement. Update the lite draft and existing identity/compatibility docs in +the implementation PR, and add a takeover section to the +[identity and recovery guide](/quest/m1/identity-guide.md) if that has +landed; the guide does not replace inline docs. Tests with controlled time: first view before/after takeover; absent reader; invalid/retired predecessor; same ID after Restart; two workers at one prefix; @@ -91,11 +102,12 @@ ANNOUNCE_TAKEOVER; published versions retain their framing. ## Required -- [Routes and announces](/quest/m1/cluster-routing/routes.md) - shared routing layer and per-stream identity +- [Route epochs](/quest/m1/cluster-routing/route-epochs.md) - every route has an epoch, so takeover continues an epoch rather than a route generation ## Related - [Claim answer epochs](https://github.com/moq-dev/moq/pull/5678) - `Request::prepare` then `publish` answers a claim at the exact path under a fresh epoch, the operation TAKEOVER continues +- [Routes and announces](/quest/m1/cluster-routing/routes.md) - the route layer whose ANNOUNCE_START layout overlaps this one's - [IETF takeover](/quest/m1/ietf-takeover.md) - negotiated IETF counterpart - [Retired requests](https://github.com/moq-dev/moq/pull/5294) - replacements never substitute content into held handles - [Identity and recovery guide](/quest/m1/identity-guide.md) - explains takeover alongside restart and legacy behavior diff --git a/quest/m1/api-audit/wanted.md b/quest/m1/api-audit/wanted.md index 4de07d0bf78..728092c4d1a 100644 --- a/quest/m1/api-audit/wanted.md +++ b/quest/m1/api-audit/wanted.md @@ -32,7 +32,8 @@ Decided 2026-10-25 (maintainer), extending the `wanted` shape that - moq-c's `MOQ_DEMAND_USED/UNUSED` become `WANTED/UNWANTED`; moq-c takes the break with every binding. -Fix the stale `is_used()` advice in the Go wrapper README. The Impact note +Fix the stale `Used`/`Unused` advice in the Go wrapper README +(`go/wrapper/README.md`, about line 143). The Impact note corrects the `rs/moq-json` and `rs/moq-flate` Unreleased advice for the cut. Public API: breaking renames and deletions in every language. Wire: none. diff --git a/quest/m1/archive/hls-target.md b/quest/m1/archive/hls-target.md index f5e08504feb..1278eb8c1ac 100644 --- a/quest/m1/archive/hls-target.md +++ b/quest/m1/archive/hls-target.md @@ -11,7 +11,7 @@ with `hls` records: the edge never skips, splits, or renumbers one. ## Plan [#4280](https://github.com/moq-dev/moq/pull/4280) derives the target from the -current window's longest segment (`snapshot_as` in +current window's longest segment (`snapshot_from_window` in `rs/moq-hls/src/export/rendition.rs`), so it rises when a long record arrives and falls when that record is evicted. Codex flagged it ([r4113921580](https://github.com/moq-dev/moq/pull/4280#discussion_r4113921580)). @@ -36,8 +36,7 @@ non-reference `EXTINF` rule and the split-record handling are already gone. Replace `target_duration_follows_the_observed_segments` with tests that pin the declared target across a window whose segment durations vary, and that -cover an overrun at the rounding boundary (listed, warned). Fix its stale -comment ("default 1s minimum"; the minimum is 2s). Hold the target per +cover an overrun at the rounding boundary (listed, warned). Hold the target per playlist URI, not per `Rendition`: `renditions::Producer::sync` replaces the `Rendition` on a catalog reconfigure, so test that a reconfigure mid-run keeps the original target. diff --git a/quest/m1/audio-jitter-target/capped-budgets.md b/quest/m1/audio-jitter-target/capped-budgets.md index 46e0dda4743..bd84f1f28b0 100644 --- a/quest/m1/audio-jitter-target/capped-budgets.md +++ b/quest/m1/audio-jitter-target/capped-budgets.md @@ -2,8 +2,9 @@ ## Goal -Seven scheduled nightlies on `main` after the estimator began capping how much -weight one observation takes show the AAC `bursty` browser rows discarding +Seven scheduled nightlies on `main` from 2026-10-25 on, after the estimator +began capping how much weight one observation takes and the player changes +that followed, show the AAC `bursty` browser rows discarding about as much audio a minute as the Opus `bursty` rows, and the Opus `near-zero` row holding a 60 ms target with no underrun episodes. `test/audio-quality/budgets.json` carries every browser row's ceilings re-measured from them. @@ -12,8 +13,19 @@ carries every browser row's ceilings re-measured from them. Waits on the nightlies: check the `audio-quality-run` artifacts of the scheduled Nightly runs on `main` (`gh run download -n audio-quality-run`) -once seven have run since the change merged. Never dispatch the nightly by -hand. Re-measure by the rule in the budgets note, and drop the note's sentence +once seven have run. Never dispatch the nightly by hand. The artifacts expire +after 7 days, so download each run's as it lands rather than waiting for the +seventh. + +Decided 2026-10-27 (maintainer): the count restarts at the 2026-10-25 +nightly, because estimator and player changes landed after the cap +([moq-dev/moq#6359](https://github.com/moq-dev/moq/pull/6359), +[moq-dev/moq#6525](https://github.com/moq-dev/moq/pull/6525), +[moq-dev/moq#6484](https://github.com/moq-dev/moq/pull/6484), +[moq-dev/moq#6609](https://github.com/moq-dev/moq/pull/6609)), the last on +2026-10-24. That makes 3 of 7 so far (2026-10-25, 10-26 and 10-27), with the +seventh due about 2026-10-31. The earlier post-cap runs below no longer +count toward the seven. Re-measure by the rule in the budgets note, and drop the note's sentence that budgets the AAC bursty discards against Opus's as measured. Nightlies from before the one-ring removal also carry `-isolated` rows; grade only the `-plain` rows, which are the browser rows now. @@ -30,8 +42,8 @@ failed only the browser Opus `wide` target ceilings: `isolated` p95 was series hold their startup target until about 21 s, then settle at 200 to 220 ms; both matching `.longtasks.json` arrays are empty. The isolated row has zero underruns, stalls, and silence; plain has 2.2 underruns a -minute, 0 stalls, and 0 silence. This supplies one of the seven required -scheduled runs, not a reason to change the estimator or recalibrate early. +minute, 0 stalls, and 0 silence. It predates the restarted count, and is +not a reason to change the estimator or recalibrate early. The near-zero stall was never the isolated ring. That row is simply the matrix's first, and a cold `cargo build` left about 800 MB dirty when it diff --git a/quest/m1/audio-jitter-target/watch.md b/quest/m1/audio-jitter-target/watch.md index b9f90258e52..8b79046df75 100644 --- a/quest/m1/audio-jitter-target/watch.md +++ b/quest/m1/audio-jitter-target/watch.md @@ -28,7 +28,12 @@ fixtures retain their original arrivals with exact 1024-sample durations; `relay-mic` was re-recorded with Chromium's fake capture device because the old recording retained no payloads from which to recover Opus durations. -Re-measured on 2026-10-22: all four replay rows have zero underruns. The +Re-measured on 2026-10-22: all four replay rows have zero underruns. These +numbers predate +[moq-dev/moq#6484](https://github.com/moq-dev/moq/pull/6484), +[moq-dev/moq#6609](https://github.com/moq-dev/moq/pull/6609), and +[moq-dev/moq#6595](https://github.com/moq-dev/moq/pull/6595); the manual run +re-measures them (decided 2026-10-27, maintainer). The fresh `relay-mic` has zero skips, stalls, or silence, a 60 ms p95 target, and 3.5 s convergence, over a 2 ms minimum RTT. It is a new fake-device recording, so that result proves neither an improvement on diff --git a/quest/m1/av-clock.md b/quest/m1/av-clock.md index abf39ed1188..91df6575b47 100644 --- a/quest/m1/av-clock.md +++ b/quest/m1/av-clock.md @@ -58,9 +58,14 @@ Recommendations for the implementation: catalog schema (`js/msf/src/catalog.ts`) accepts a negative `delay` and folds it into absent; refuse it on decode instead. +Decided 2026-10-27 (maintainer): this waits only on +[capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md), not the +whole [audio jitter target](/quest/m1/audio-jitter-target/README.md) epic, +whose remaining manual browser proof does not gate reshaping the clock. + ## Required -- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the estimator this sits on, and the per-track targets this shape carries +- [Capped budgets](/quest/m1/audio-jitter-target/capped-budgets.md) - the estimator's browser budgets, re-measured, before this reshapes the clock around them ## Related diff --git a/quest/m1/binding-timestamps.md b/quest/m1/binding-timestamps.md index 2eb693c580e..09bc4a61cfc 100644 --- a/quest/m1/binding-timestamps.md +++ b/quest/m1/binding-timestamps.md @@ -62,5 +62,11 @@ on the generated C package: the hand-written `moq-c` ships at this cut with parity to moq-ffi for every m1 binding feature, so the moq-c half lands there, and the generated C line follows after the cut. +Decided 2026-10-27 (maintainer): "untimed as absent" is needed only on the +read side, and only until the cut, when +[time addressing](/quest/m1/time-addressing/README.md) makes every track +timed; [bindings by start](/quest/m1/time-addressing/bindings.md) then drops +that `Option` for reads. + Wall and monotonic times stay `moq_clock::Wall` and `moq_clock::Instant`, not media timestamps; keep the types distinct. diff --git a/quest/m1/buffered-ascending.md b/quest/m1/buffered-ascending.md index 8a1d709abc0..98051ae87bf 100644 --- a/quest/m1/buffered-ascending.md +++ b/quest/m1/buffered-ascending.md @@ -72,15 +72,10 @@ Implementation: every reply decodes from its first frame, plain and barge-in, with no overflow. -### Runtime mode switch (2026-10-24 pre-cut audit) - -Changing `buffer` at runtime does not switch the audio ring's mode: -`js/watch/src/audio/decoder.ts:293` captures `sync.out.buffered` once when it -builds the worklet, and `#order()` (`:679`) peeks it once per attempt, so -going from live to buffered after playback started leaves `Backpressure` -and the worklet `Init` live (future-dated TTS audio is shed), and the -reverse leaves a live player in play-through. Key the ring and `Order` on -`sync.out.buffered`: rebuild the ring, or post a mode message plus a reset. +Decided 2026-10-27 (maintainer): the runtime mode-switch bug found by the +2026-10-24 pre-cut audit is its own ready quest, +[Switching buffer at runtime](/quest/m1/watch-buffer-switch.md). This quest +keeps waiting on lite-07 for the head-hold simplification. ## Required @@ -88,5 +83,6 @@ reverse leaves a live player in play-through. Key the ring and `Order` on ## Related +- [Switching buffer at runtime](/quest/m1/watch-buffer-switch.md) - the audio ring follows `buffer` changes; whichever lands second re-runs the other's decoder tests - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS consumer stops delivering groups below its cursor - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - owns the wire's `Group Order` and the ordered cursor's floor diff --git a/quest/m1/cluster-routing/README.md b/quest/m1/cluster-routing/README.md index 13215421fc2..22372436efe 100644 --- a/quest/m1/cluster-routing/README.md +++ b/quest/m1/cluster-routing/README.md @@ -98,9 +98,13 @@ Existing mesh deployments remain supported; the full edge/core topology migration is independent. Basic outbound federation proves the common routing contract without requiring that deployment migration. -Recent baseline: #5375's two-second retraction quarantine reduced stale -revivals but left residual cases; #5391 bounded its queue under repeated -same-clock churn. #5382 and #5393 separately reduced idle-session polling. +Recent baseline: [moq-dev/moq#5375](https://github.com/moq-dev/moq/pull/5375)'s +two-second retraction quarantine reduced stale revivals but left residual +cases; [moq-dev/moq#5391](https://github.com/moq-dev/moq/pull/5391) bounded +its queue under repeated same-clock churn. +[moq-dev/moq#5382](https://github.com/moq-dev/moq/pull/5382) and +[moq-dev/moq#5393](https://github.com/moq-dev/moq/pull/5393) separately +reduced idle-session polling. The 300 ms update hold alone is no longer the baseline. Keep regression coverage for announcement churn, idle work, active continuity, and retained state separately; fewer control bytes does not prove cheaper runtime work. @@ -122,6 +126,8 @@ setup after the capabilities land. ## Required +- [Grant subject](/quest/m1/cluster-routing/grant-subject.md) - grants carry an issuer-qualified subject, scoped tokens without one are refused, and the routing model drops its `proved` exemption +- [Route epochs](/quest/m1/cluster-routing/route-epochs.md) - every route has an epoch, deleting the epochless instance machinery - [Routes and announces](/quest/m1/cluster-routing/routes.md) - implement the proven route layer with Rust/JS parity and tiered-cluster acceptance - [Outbound federation](/quest/m1/cluster-routing/federation.md) - an outbound-only local relay exchanges authorized broadcasts with the CDN and survives partition/healing diff --git a/quest/m1/cluster-routing/federation.md b/quest/m1/cluster-routing/federation.md index 55a9f4b8635..f31f0fcca14 100644 --- a/quest/m1/cluster-routing/federation.md +++ b/quest/m1/cluster-routing/federation.md @@ -50,12 +50,17 @@ Update `doc/bin/relay/cluster.md` (`proxy`, `upstream`, and the one-copy-per-broadcast priority recipe) and affected auth/config examples in the same PR; the combined concept/setup guide follows separately. +Decided 2026-10-27 (maintainer): this requires +[Grant subject](/quest/m1/cluster-routing/grant-subject.md) directly, since +scoped admission keys on it. + Public API: report any peer configuration or admission change. Wire: use the proven lite-07 routing contract; any additional change requires draft and Rust/JS parity. Run `just check`, `just test interop --all`, and `just drafts check`. ## Required +- [Grant subject](/quest/m1/cluster-routing/grant-subject.md) - the issuer-qualified subject the local relay's identities are filed under - [Routes and announces](/quest/m1/cluster-routing/routes.md) - identity, scoped claims, reachability, and export policy ## Related diff --git a/quest/m1/cluster-routing/grant-subject.md b/quest/m1/cluster-routing/grant-subject.md new file mode 100644 index 00000000000..4b46fbd7bfb --- /dev/null +++ b/quest/m1/cluster-routing/grant-subject.md @@ -0,0 +1,38 @@ +# [M] Grants carry an issuer-qualified subject + +## Goal + +`moq_net::auth::Grant` and moq-auth carry the verified token subject, +qualified by its issuer, in Rust and JS. A scoped token without a subject is +refused, and so is a token added to a session (refresh or `auth().add`) +whose issuer-qualified subject differs from that of the session's current +grants. The routing reference model no longer exempts a node's own proven +identity, and a fixture shows a forger that declares the victim's id as its +own captures nothing outside its scope. No wire change. + +## Plan + +Decided 2026-10-27 (maintainer): split out of +[Routes and announces](/quest/m1/cluster-routing/routes.md) as a ready +quest, since the subject is the key routes and +[federation](/quest/m1/cluster-routing/federation.md) file scoped identities +under (`hash(subject, id)`, decided 2026-10-15), and it lands without the +route layer. + +- Today `moq_auth::Claims` reads and drops `sub` and `iss` + (`rs/moq-auth/src/wire.rs`), and `Grant` (`rs/moq-net/src/auth.rs`) holds + only patterns and expiry. Mirror the change in `js/auth` and `js/net`. +- Fail loud, per the 2026-10-15 decision in routes. Report the public API + change to `Grant` and to the auth error surface. +- The routing contracts' reference model still exempts a proven identity + (the `proved` map in `rs/moq-net/tests/integration/routing_contracts/model.rs`). + Drop that exemption, add the forger that declares the victim's id as its + own, and fix the comment on + `forged_identity_hijacks_claims_outside_the_forgers_scope` + (`routing_contracts/mod.rs`), which still says the forger's own proven + identity stays bare: since 2026-10-26 every identity a scoped connection + sends is subject-qualified, its own included. +- Update `doc/bin/relay/auth.md` for the refusals. + +Public API: `Grant` gains the issuer-qualified subject in Rust and JS, and +scoped tokens without a subject are refused. Wire: none. diff --git a/quest/m1/cluster-routing/route-epochs.md b/quest/m1/cluster-routing/route-epochs.md new file mode 100644 index 00000000000..b5b43bad76f --- /dev/null +++ b/quest/m1/cluster-routing/route-epochs.md @@ -0,0 +1,35 @@ +# [L] Mint an epoch for every route + +## Goal + +Every route has an epoch: the receiving side mints one at ingest or local +announce when a peer sends none, so `Instance` collapses to `Epoch`. +`Instance::Route(generation)` and its renewals, `restarted_since`, the +held-restart Release for epochless routes, and Selection retirement are +gone, leaving Follow's same-epoch Update/Restart split as the only takeover +rule. No wire change. + +## Plan + +Decided 2026-10-27 (maintainer): this was the 2026-10-24 pre-cut audit's +proposal in [Routes and announces](/quest/m1/cluster-routing/routes.md), and +is now decided rather than a proposal for routes to weigh. It is a ready +quest of its own, and routes and +[announcement takeover](/quest/m1/announce-takeover.md) both build on it. + +The epochless instance machinery keeps drawing fixes (35d935f428, +3655358c2f, c39780f952, 29e74a26b7, a7acf21e2a, ef2511e101, aa0d915867, +0f4f81f9ed, f0fcbefd66, 4c8ab3ca41, 001482144f, 07127f8f5b), as does front +splice, park, and linger in `serve_front` (166f9e87bc, 3b4ad5bf02, +88848954db, 24b9177b6e, 0e1995017d, 732101ed1e, 00d27780dd, 37b49d8042). +Most of it lives in `rs/moq-net/src/model/origin.rs`. + +- Watch that a minted epoch never makes an epochless change of serving route + look like continuity: routes keeps that a Restart, and the routing + contract's epochless fixtures + (`rs/moq-net/tests/integration/routing_contracts/`) keep passing. +- Check whether `js/net` keeps a matching route generation, and mirror the + change there if it does. + +Public API: `Route::epoch` stops being optional (breaking), and the Impact +bullet says how to migrate. Wire: none. diff --git a/quest/m1/cluster-routing/routes.md b/quest/m1/cluster-routing/routes.md index 71077e7a80e..712dc7f4464 100644 --- a/quest/m1/cluster-routing/routes.md +++ b/quest/m1/cluster-routing/routes.md @@ -536,12 +536,13 @@ region's knowledge independently: the proof keeps them all or, if any lost floor is newer than a kept one, rebuilds them all, a collection whose replay is bounded like any other. -Why not a hold-down: #4644 retracted at once and re-announced a replacement +Why not a hold-down: [moq-dev/moq#4644](https://github.com/moq-dev/moq/pull/4644) retracted at once and re-announced a replacement after 1 s, which ended a downstream front whose only path ran through the relay, dropping subscribers for about a second while the relay kept serving; holding without retracting brought hunting back in measurements, and #4642's 300 ms cursor hold was the earlier mitigation. Main also contains -#5375's two-second retraction quarantine and #5391's same-clock queue dedup. +[moq-dev/moq#5375](https://github.com/moq-dev/moq/pull/5375)'s two-second retraction quarantine and +[moq-dev/moq#5391](https://github.com/moq-dev/moq/pull/5391)'s same-clock queue dedup. #5375 measured residual stale revivals, so neither longer holds nor one passing mesh establish correctness. Separating reachability from claim freshness is the direction, not yet a proof. Retire mitigations only after the incident @@ -596,7 +597,7 @@ broadcast stays live. Pin today's sequence beside the `residual_*` tests; the epochless Restart rule above decides what it becomes. Tests, time mocked: the handoff fixtures and sweep in `handoff.rs`; -#4644's live-graph withdrawal and failover tests (the PR closed and +[moq-dev/moq#4644](https://github.com/moq-dev/moq/pull/4644)'s live-graph withdrawal and failover tests (the PR closed and its branch is gone; fetch `refs/pull/4644/head`) as acceptance, run on the wip version; a link flap with many broadcasts behind one origin costs one ROUTE change per link; a ring of three where a stale neighbour cannot revive an ended @@ -644,20 +645,18 @@ continues to rank feasible transit routes; report it. This may split at start (Rust and draft, then JS), as long as both land in one release. -### Mint an epoch for every route (2026-10-24 pre-cut audit) +Decided 2026-10-27 (maintainer): two ready quests split out of this one, +and this quest builds on both. [Grant subject](/quest/m1/cluster-routing/grant-subject.md) +carries the issuer-qualified subject the scoped keys above are named by, and +fixes the reference model's `proved` exemption. +[Route epochs](/quest/m1/cluster-routing/route-epochs.md) mints an epoch for +every route, the 2026-10-24 pre-cut audit's proposal, now decided, so +`Instance` collapses to `Epoch` before the route layer lands. -The epochless instance machinery keeps drawing fixes (35d935f428, -3655358c2f, c39780f952, 29e74a26b7, a7acf21e2a, ef2511e101, aa0d915867, -0f4f81f9ed, f0fcbefd66, 4c8ab3ca41, 001482144f, 07127f8f5b), as does front -splice, park, and linger in `serve_front` (166f9e87bc, 3b4ad5bf02, -88848954db, 24b9177b6e, 0e1995017d, 732101ed1e, 00d27780dd, 37b49d8042). -Proposal for this quest to weigh: mint an epoch on the receiving side at -ingest or local announce when a peer sends none, so `Instance` collapses to -`Epoch`. That deletes the `Instance::Route(generation)` renewals, -`restarted_since`, the held-restart Release for epochless routes, and -Selection retirement, leaving Follow's same-epoch Update/Restart split as -the only takeover rule. `Route::epoch` stops being optional (a breaking -API change); no wire change. +## Required + +- [Grant subject](/quest/m1/cluster-routing/grant-subject.md) - the issuer-qualified subject scoped identities are filed under +- [Route epochs](/quest/m1/cluster-routing/route-epochs.md) - every route has an epoch, so the epochless instance machinery is gone ## Related diff --git a/quest/m1/drop-set.md b/quest/m1/drop-set.md index 7f185afe8d1..7f3a20b270e 100644 --- a/quest/m1/drop-set.md +++ b/quest/m1/drop-set.md @@ -14,6 +14,12 @@ narrower budget is served after a widening, as a cold relay would. ## Plan +Decided 2026-10-27 (maintainer): this runs after +[Linear fill path](/quest/m1/subscribe-ranges/fill-linear.md), since both edit +`given_up` in `rs/moq-net/src/model/track/miss.rs`, and before +[Rust groups by start](/quest/m1/time-addressing/model.md), which requires it +because runs keyed by group sequence convert mechanically to starts. + Decided by the maintainer on 2026-10-19, splitting this from the widened max age re-offer ([moq-dev/moq#6717](https://github.com/moq-dev/moq/pull/6717)), which keeps @@ -258,7 +264,7 @@ unchanged. Besides memory, run fragmentation costs CPU today, new on main: every SUBSCRIBE_DROP bumps `drops_revision`, and each reader's `DropCursor::next` rescans the overlapping runs (`rs/moq-net/src/model/track.rs` about lines -5574-5609), so N fragmented runs and R downstream readers cost O(R*N) per +5802-5870), so N fragmented runs and R downstream readers cost O(R*N) per 6-byte DROP. JS `insertDrop` (`js/net/src/internal.ts:136`) copies the whole run array once for the track and once per sink (`track.ts:914`). The replacement keeps the per-cursor walk incremental in both languages, with a diff --git a/quest/m1/drop-web-transport-trait.md b/quest/m1/drop-web-transport-trait.md index 8fb5bc8a3ca..33a9c39944b 100644 --- a/quest/m1/drop-web-transport-trait.md +++ b/quest/m1/drop-web-transport-trait.md @@ -8,8 +8,9 @@ moq-tokio, moq-uring, moq-relay, moq-ffi and moq-wasm no longer depend on `transport::poll` traits, either directly or through a thin adapter per backend. The generic adapters over `web_transport_trait` are deleted: moq-tokio's `transport::Session` with its async-to-poll emulation, and -moq-uring's `transport::Session`. The sealed `Grouped`/`GroupedSend` pair -goes with them. A custom transport implements moq-net's traits, which are +moq-uring's `transport::Session`, by whichever of this quest and +[native WebTransport](/quest/m1/quic-api/native-web.md) lands second. The +sealed `Grouped`/`GroupedSend` pair goes with them. A custom transport implements moq-net's traits, which are already generic and send-group-aware (`Session::send_group`, `SendStream::Group`, `Ungrouped`). @@ -53,9 +54,19 @@ migration quest. web-transport-iroh decodes CONNECT with it, so an in-tree copy would duplicate the crate whenever iroh is on. Its QPACK bound ships as the upstream web-transport-proto 0.6.3 patch. +Decided 2026-10-27 (maintainer): this waits on +[moq-net polls on kio waiters](/quest/m1/quic-api/net-poll.md), not on +[native WebTransport](/quest/m1/quic-api/native-web.md). The iroh, wasm, and +ffi impls need only net-poll's trait shape, so they need not wait for native +QUIC to move off the adapter. Whichever of native WebTransport and this quest +lands second deletes the generic adapters; native WebTransport already +expects to leave the adapter serving iroh. qmux uses `web-transport-trait` +only as a dev-dependency, for its 0.6 interop tests, so it does not block the +removal. + ## Required -- [Native WebTransport](/quest/m1/quic-api/native-web.md) - native QUIC moves off the adapter first +- [moq-net polls on kio waiters](/quest/m1/quic-api/net-poll.md) - the `&self, &kio::Waiter` trait shape the iroh, wasm, and ffi impls implement ## Related diff --git a/quest/m1/identity-guide.md b/quest/m1/identity-guide.md index 4e9c3978aed..00beee4004a 100644 --- a/quest/m1/identity-guide.md +++ b/quest/m1/identity-guide.md @@ -21,10 +21,15 @@ as a bug. Show lifetime pins for readers without announcement following and Start/End/Restart for those that follow siblings. Explain optional expected epochs, known mismatch refusal, and the weaker legacy best-effort guarantee. -Include lazy epochless claims, exact per-job announcements, the accepted -cold-start Restart fallback, and proven Takeover preserving the same instance -with an independent route. Distinguish lineage from arbitrary cross-worker -stitching, and show what happens when either announcement ends. Cover +Include lazy epochless claims, exact per-job announcements under fresh +epochs ([claim answer epochs](https://github.com/moq-dev/moq/pull/5678)), and +the cold-start Restart fallback. + +Decided 2026-10-27 (maintainer): the guide documents what exists (epochs, +Restart, claim answers, and the cold-start fallback) and no longer waits on +[announcement takeover](/quest/m1/announce-takeover.md); the takeover PR adds +its own section, distinguishing lineage from cross-worker stitching and +showing what happens when either announcement ends. Cover SUBSCRIBE and FETCH, publisher max-age versus reader max-delay, unordered arrivals and duplicates, and terminal DROP behavior by supported version. Do not claim unimplemented behavior: identify current capabilities and link @@ -38,6 +43,6 @@ of duplicating them. Use the existing documentation build checks. Public API and wire: none; this is a user guide for existing feature owners. -## Required +## Related -- [Announcement takeover](/quest/m1/announce-takeover.md) - establishes the continuity operation the guide demonstrates +- [Announcement takeover](/quest/m1/announce-takeover.md) - adds its own section once it lands diff --git a/quest/m1/ietf-fill-timeout-js.md b/quest/m1/ietf-fill-timeout-js.md new file mode 100644 index 00000000000..20205170683 --- /dev/null +++ b/quest/m1/ietf-fill-timeout-js.md @@ -0,0 +1,32 @@ +# [M] JS IETF FETCH honors its fill timeout + +## Goal + +The JS IETF FETCH responder and subscriber use FILL_TIMEOUT on drafts 18 +and 22 as [Rust does](/quest/m1/ietf-fill-timeout.md): it bounds the total +wait for missing upstream objects, zero is cache-only, and a timed-out range +gets the draft's timeout-gap disposition without hanging or discarding +objects that are available. + +## Plan + +Decided 2026-10-27 (maintainer): split out of +[IETF fill timeout](/quest/m1/ietf-fill-timeout.md), whose Rust half starts +now. This half waits on the cached +[JavaScript FETCH](/quest/m1/js-fetch.md) responder, not on the on-demand +one. + +Follow the Rust half's decisions: reuse the native parameter rather than a +content-age extension, keep exact max-delay and max-age enforcement in the +model, never reset the total wait per missing object, and never let a timeout +claim the content never existed. Cover the Rust half's test cases on the JS +side on mocked time, plus Rust/JS interop in both directions if the Rust half +has landed. Run +`just check` and `just test interop --all`. + +Public API: reuse FETCH's max-delay option. Wire: support the existing IETF +parameter and gap statuses on their applicable versions. + +## Required + +- [JavaScript FETCH](/quest/m1/js-fetch.md) - the cached JS responder this extends diff --git a/quest/m1/ietf-fill-timeout.md b/quest/m1/ietf-fill-timeout.md index 530dd160d2b..d3349424142 100644 --- a/quest/m1/ietf-fill-timeout.md +++ b/quest/m1/ietf-fill-timeout.md @@ -1,10 +1,10 @@ -# [L] IETF FETCH honors its fill timeout +# [M] Rust IETF FETCH honors its fill timeout ## Goal -Rust and JS use IETF FILL_TIMEOUT to bound waiting for missing FETCH -objects on drafts 18 and 22, and return the draft's timeout-gap disposition -without hanging or discarding objects that are available. Local FETCH age +Rust uses IETF FILL_TIMEOUT to bound waiting for missing FETCH objects on +drafts 18 and 22, and returns the draft's timeout-gap disposition without +hanging or discarding objects that are available. Local FETCH age enforcement works on all versions. ## Plan @@ -17,10 +17,15 @@ bounds the total wait for missing upstream objects; zero is cache-only. It does not judge how old an immediately available object is. Keep exact max-delay/max-age enforcement in the model alongside this approximation. -Today Rust decodes only FILL_TIMEOUT's presence and refuses such requests -because it cannot emit the required gap statuses. Preserve the value and -honor its semantics before advertising or sending support. JS's IETF FETCH -responder comes from its existing quest; build on that surface. +Today Rust decodes only FILL_TIMEOUT's presence (`fill_timeout: bool` in +`rs/moq-net/src/ietf/fetch.rs`) and refuses such requests because it cannot +emit the required gap statuses. Preserve the value and honor its semantics +before advertising or sending support. + +Decided 2026-10-27 (maintainer): split by language. This Rust half starts +now; the JS half is [JS FETCH fill timeout](/quest/m1/ietf-fill-timeout-js.md), +which waits on the cached [JavaScript FETCH](/quest/m1/js-fetch.md) +responder. Keep the two wire behaviors identical. Propagate an explicit FETCH max-delay as the upstream wait budget where the draft supports it. For omitted max-delay, retain the wire's omitted @@ -37,7 +42,7 @@ range, reset the entire session, or silently reinterpret FILL_TIMEOUT as publisher retention. Test standalone and applicable joining/fill forms, reordered arrivals, cache-only requests, partial availability, no subsequent writes, cancellation, shared callers, and an older peer without this field. -Use mocked time and the existing Rust/JS and interop CI lanes; run +Use mocked time and the existing Rust and interop CI lanes; run `just check` and `just test interop --all`. Public API: reuse FETCH's max-delay option; no new content-age extension @@ -45,6 +50,6 @@ configuration. Wire: support existing IETF parameters and gap statuses on their applicable versions. Update existing interop/deviation and FETCH docs inline, clearly distinguishing upstream wait time from content age. -## Required +## Related -- [JavaScript FETCH](/quest/m1/js-fetch.md) - the JS responder this extends +- [JS FETCH fill timeout](/quest/m1/ietf-fill-timeout-js.md) - the same semantics in the JS responder diff --git a/quest/m1/interop-coverage/late-join.md b/quest/m1/interop-coverage/late-join.md index be8ed57f80a..5702170932e 100644 --- a/quest/m1/interop-coverage/late-join.md +++ b/quest/m1/interop-coverage/late-join.md @@ -19,8 +19,11 @@ the JS publisher refuses every FETCH. Publish a track whose first group outlives the subscriber's arrival (a catalog written once, and a slow video GOP) so the case cannot pass on timing. +Decided 2026-10-27 (maintainer): this needs only the cached and joining half +of JavaScript FETCH, not the on-demand half that waits on JS ranges. + Public API: none. Wire: none. ## Required -- [JavaScript FETCH](/quest/m1/js-fetch.md) - the JS publisher answers FETCH +- [JavaScript FETCH](/quest/m1/js-fetch.md) - the JS publisher answers cached and joining FETCH diff --git a/quest/m1/js-fetch-on-demand.md b/quest/m1/js-fetch-on-demand.md new file mode 100644 index 00000000000..1ce3911d390 --- /dev/null +++ b/quest/m1/js-fetch-on-demand.md @@ -0,0 +1,28 @@ +# [S] Serve on-demand IETF FETCH in JavaScript + +## Goal + +A JavaScript publisher answers IETF FETCH from a native subscriber for groups +that are no longer in its live cache, or never were, by dispatching onto the +on-demand request surface `@moq/net` gains in +[JS ranges](/quest/m1/subscribe-ranges/js.md). + +## Plan + +Decided 2026-10-27 (maintainer): split out of +[cached JavaScript FETCH](/quest/m1/js-fetch.md), which starts now, so only +this half waits on JS ranges. Decided in the 2026-10-05 audit: the +producer-side request surface folded into JS ranges, so it takes range +requests from the start, and this half only dispatches IETF FETCH onto it. + +Verify with an in-memory application responder: a browser publisher serves a +native IETF subscriber after a group is evicted or was never cached. Run the +supported-draft matrix and `just test interop --all` through CI. + +Public API: none beyond JS ranges' surface. Wire: none beyond the cached +half's FETCH support. + +## Required + +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - the on-demand request surface this dispatches onto +- [Cached JavaScript FETCH](/quest/m1/js-fetch.md) - the IETF FETCH dispatch and codecs this extends diff --git a/quest/m1/js-fetch.md b/quest/m1/js-fetch.md index 86781d9cf5a..007eb0db527 100644 --- a/quest/m1/js-fetch.md +++ b/quest/m1/js-fetch.md @@ -1,16 +1,19 @@ -# [L] Serve IETF FETCH in JavaScript +# [L] Serve cached IETF FETCH in JavaScript ## Goal -A JavaScript publisher answers IETF FETCH from a native subscriber, including -groups that are no longer in its live cache, through the on-demand request -surface `@moq/net` gains in [JS ranges](/quest/m1/subscribe-ranges/js.md). +A JavaScript publisher answers IETF FETCH from a native subscriber for groups +in its cache, standalone and joining, and honors the draft-18 Largest Object. ## Plan -Decided in the 2026-10-05 audit: the producer-side request surface folded -into [JS ranges](/quest/m1/subscribe-ranges/js.md), so it takes range -requests from the start. This quest keeps only IETF FETCH dispatch onto it. +Decided 2026-10-27 (maintainer): split in two. This half, cached and joining +FETCH with the draft-18 Largest fix, starts now: the JS IETF publisher refuses +every FETCH today (`#runFetch` in `js/net/src/ietf/publisher.ts`), while the +JS lite publisher already serves cached FETCH. Serving evicted or uncached +groups through the on-demand request surface `@moq/net` gains in +[JS ranges](/quest/m1/subscribe-ranges/js.md) moved to +[on-demand JavaScript FETCH](/quest/m1/js-fetch-on-demand.md). Implement IETF FETCH dispatch and codecs across the supported draft versions. Cover standalone and relative joining requests, subscription lifetime @@ -22,7 +25,7 @@ forms must receive the protocol's explicit refusal rather than hang. Datagrams are never fetchable (#4982): Rust answers an IETF FETCH that reaches a datagram with DOES_NOT_EXIST; match it. Decided 2026-10-08: JS adds no `NotFetchable` code, since nothing would send it. Only lite-07 has the -code, and lite-07 drops FETCH for [ranges](/quest/m1/subscribe-ranges/js.md); +code, and lite-07 drops FETCH for [ranges](/quest/m1/subscribe-ranges/lite07-fetch.md); lite-05 and lite-06 answer `NotFound`. The JS publisher already answers the TRACK_STATUS a native fetch-only request @@ -66,20 +69,15 @@ Joining tests: unit tests per draft for each refusal and for a served head plus the browser publisher on draft 18, except a track that finishes right after SUBSCRIBE_OK, which the m2 quest covers. -Verify with an in-memory application responder: a browser publisher serves a -native IETF subscriber after a group is evicted or was never cached. Run the -supported-draft matrix and `just test interop --all` through CI. +Run the supported-draft matrix and `just test interop --all` through CI. -Public API: none beyond JS ranges' surface. Wire: implement the existing +Public API: none. Wire: implement the existing supported IETF FETCH formats, including the joining FETCH, and serve a Largest Object subscription from past Largest; update relevant documentation and any MoQ draft claims that change. -## Required - -- [JS ranges](/quest/m1/subscribe-ranges/js.md) - the on-demand request surface this dispatches onto - ## Related -- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native timeout budgets and gap dispositions build on this responder +- [On-demand JavaScript FETCH](/quest/m1/js-fetch-on-demand.md) - serves evicted and uncached groups once JS ranges lands +- [JS FETCH fill timeout](/quest/m1/ietf-fill-timeout-js.md) - JS timeout budgets and gap dispositions build on this responder - [JS join after a quick finish](/quest/m2/ietf-js-joining-fetch.md) - answers a joining FETCH that arrives after its subscription ended diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 65707f8467c..97ca3bf8361 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -52,12 +52,9 @@ The cut: - Every quest under Required has landed. - The identifier becomes `moq-lite-07` in `rs/moq-net` and `js/net`, in the draft (whose text already names the rename), and at every site spelling - the wip ALPN: `rs/moq-tokio/src/connect.rs` and `listen.rs`, - `rs/moq-tokio/tests/`, `rs/moq-relay/src/cluster.rs`, - `rs/moq-relay/tests/integration/smoke.rs`, - `test/interop/bare-fin.ts`, `test/interop/lite-varint.ts`, and - `doc/concept/moq-lite.md`; grep for `moq-lite-07-wip` to catch new ones. A wip peer and a final peer refuse each other by ALPN - rather than misparse; no compatibility shim. + the wip ALPN: grep the repository for `moq-lite-07-wip`. A wip peer and a + final peer refuse each other by ALPN rather than misparse; no + compatibility shim. - The draft's lite-07 changelog matches the wire and `just drafts check` passes. - SETUP keeps its stronger rule: #4820 put it under the general Message @@ -75,7 +72,14 @@ replaced SUBSCRIBE_ALL) landed in [#6158](https://github.com/moq-dev/moq/pull/61 and [#6167](https://github.com/moq-dev/moq/pull/6167), and the padding stream type for the m2 browser [padding probes](/quest/m2/probe/padding.md) landed in [#6233](https://github.com/moq-dev/moq/pull/6233). -The `Time(pts)` range end rides in Subscribe ranges, already Required. +The `Time(pts)` range end rides in the subscribe-ranges children already Required. + +Decided 2026-10-27 (maintainer): of [Subscribe ranges](/quest/m1/subscribe-ranges/README.md), +only the children that change the lite-07 wire gate the cut: +[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md), +[JS ranges](/quest/m1/subscribe-ranges/js.md), and +[Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md). The +fill path, fetch as a range, and the start bookkeeping change no wire. Decided 2026-10-25 (maintainer): addressing groups by timestamp gates the cut, since it removes group sequence numbers from the lite-07 wire. @@ -99,7 +103,9 @@ timestamp bound in [#6168](https://github.com/moq-dev/moq/pull/6168). - [Routes and announces](/quest/m1/cluster-routing/routes.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the hop list gone - [Drop set](/quest/m1/drop-set.md) - lite-07's SUBSCRIBE_DROP carries `Max Age`, so a widening re-offers groups reported dropped -- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - SUBSCRIBE carries ranges and an order and lite FETCH is gone, in Rust and JS +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - lite-07 SUBSCRIBE carries ranges and an order, and SUBSCRIBE_OK the resolved start +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - JS speaks lite-07 ranges +- [Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md) - lite FETCH is gone from lite-07, in Rust and JS - [Lite-07 by start](/quest/m1/time-addressing/lite07.md) - lite-07 carries group starts, previous-start links, GROUP_END, and time bounds instead of sequence numbers - [@moq/net by start](/quest/m1/time-addressing/js.md) - JS speaks the same lite-07 wire - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route diff --git a/quest/m1/perf-host.md b/quest/m1/perf-host.md deleted file mode 100644 index f68a4f192c6..00000000000 --- a/quest/m1/perf-host.md +++ /dev/null @@ -1,16 +0,0 @@ -# [XS] A profiling host - -## Goal - -Condition: a quiet Linux host has `kernel.perf_event_paranoid=-1` and -`bench/README.md` names it and how to reach it. Check with -`sysctl kernel.perf_event_paranoid` on that host. This is a maintainer action; -delete this quest once it holds. - -## Plan - -- `bench/profile.py` refuses `lock` and `cpu` captures without it (heap - captures need only `jeprof`). The - [log lock profile](/quest/m1/unbounded-groups/log-lock-profile.md) and - [moq-dev/moq#6680](https://github.com/moq-dev/moq/pull/6680) need CPU and lock - profiles from such a host. diff --git a/quest/m1/quic-api/native-web.md b/quest/m1/quic-api/native-web.md index ba438a9cf4d..f4f0a65a450 100644 --- a/quest/m1/quic-api/native-web.md +++ b/quest/m1/quic-api/native-web.md @@ -49,7 +49,6 @@ the same workload. - [Poll forms](/quest/m1/quic-api/poll.md) - the leak-free driver polls these impls forward to - [moq-net polls on kio waiters](/quest/m1/quic-api/net-poll.md) - the `&self, &kio::Waiter` trait shape these impls implement -- [Streams](/quest/m1/quic-api/streams.md) - `send::Finished` and `Buf` datagram sends ## Related diff --git a/quest/m1/quic-api/poll.md b/quest/m1/quic-api/poll.md index f8d85cb11f9..397a1955042 100644 --- a/quest/m1/quic-api/poll.md +++ b/quest/m1/quic-api/poll.md @@ -60,7 +60,6 @@ async forms unchanged. ## Required - [Waits](/quest/m1/quic-api/waits.md) - moq-quic's `poll_*` forms these forward to -- [Streams](/quest/m1/quic-api/streams.md) - the `send::`/`recv::` names and `send::Finished` these signatures use ## Related diff --git a/quest/m1/quic-api/streams.md b/quest/m1/quic-api/streams.md index f13cb017093..42cb3a7f673 100644 --- a/quest/m1/quic-api/streams.md +++ b/quest/m1/quic-api/streams.md @@ -58,8 +58,9 @@ built; if it is tagged to its stream, a foreign one is the only error left. zero-copy split, so one generic write covers owned and borrowed data. Send: `write(&mut impl Buf) -> usize`, `write_all(impl Buf)`, `AsyncWrite`. Recv: `read(&mut impl BufMut)`, `read_exact`, `read_chunk(max) -> Bytes`, -`read_to_end(limit)`, `lease`, `AsyncRead`. `write_chunk`, `write_chunks`, -`write_all_chunks`, and `read_chunks` go; moq-quic's `write` and +`read_to_end(limit)`, `lease`, `AsyncRead`. `write_chunk`, `write_chunks`, and +`write_all_chunks` go (`read_chunks` already went in +[moq-dev/moq#6809](https://github.com/moq-dev/moq/pull/6809)); moq-quic's `write` and `write_chunks` become one `write(&mut impl Buf)`. Confirm on `benches/streams.rs` that owned writes stay zero-copy. A buffer pool is out of scope: [QUIC I/O boundary](/quest/m2/quic-io-boundary.md) owns it after a @@ -95,10 +96,11 @@ in the drivers"). for unordered reads, and a four-level match in `session.rs`; collapse it onto `send::Error` and `recv::Error`. -**Acked.** Both drivers match `StreamEvent::Acked` explicitly instead of -moq-tokio's `_ => {}` catch-all, so a new event is a compile error until -[waits](/quest/m1/quic-api/waits.md) deletes the events. The driver's waiter -method arrives with [poll_acked](/quest/m2/quic-ack-hook.md). +**Acked.** The driver's waiter method arrives with +[poll_acked](/quest/m2/quic-ack-hook.md). Decided 2026-10-27 (maintainer): +the drivers do not match `StreamEvent::Acked` explicitly here, since +[waits](/quest/m1/quic-api/waits.md) deletes `StreamEvent` right after, so it +would be throwaway work. **moq-uring.** Its driver gets the same names and signatures and passes `send::Options` to the open, as moq-tokio's does, instead of applying them diff --git a/quest/m1/read-backfill.md b/quest/m1/read-backfill.md index b6293eac558..62a90a542df 100644 --- a/quest/m1/read-backfill.md +++ b/quest/m1/read-backfill.md @@ -53,10 +53,15 @@ delivers a group that arrives below the cursor as a late backfill; Rust `Sync`) and adjust them; one that wants every group reads an arrival-order API instead. +Decided 2026-10-27 (maintainer): this waits only on Lite-07 ranges (the +resolved start in SUBSCRIBE_OK) and JS ranges, not the whole +[Subscribe ranges](/quest/m1/subscribe-ranges/README.md) epic. + Public API: no type change; the ordered read changes in Rust and JS (the floor comes from reported bounds, and JS drops backfill). Wire: the resolved start on lite-07 SUBSCRIBE_OK, through Lite-07 ranges. ## Required -- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - the reader's requested ranges set the floor, and every sequence in them arrives or is dropped +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - lite-07 SUBSCRIBE_OK carries the resolved start the Rust floor comes from +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - the JS reader's requested ranges set its floor diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 4c7fe240906..c0a2c04e4c6 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -60,7 +60,9 @@ relay bounds a sparse FETCH only if the model can request ranges: which is the starvation the flag existed to prevent. Subscriber Max Age still caps every range, so `Absolute(0)..` means everything not stale. A relay that cannot resolve a relative end locally forwards it upstream as - relative. + relative over lite-07; a moq-transport bridge resolves it to an absolute + Group ID first ([Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md), + decided 2026-10-27). - **Time ends** (maintainer, 2026-10-17, for DVR seek in the [archive line](/quest/m1/archive/README.md)). A range end may also be `Time(pts)`, in the track's timescale. Whoever holds the track resolves it: @@ -94,38 +96,14 @@ This line owns the end-to-end test: a relay with a sparse cache answers a wide past range plus live in both orders, over lite-07 and moq-transport, with every sequence delivered or dropped. -### Delete the old start bookkeeping (2026-10-24 pre-cut audit) - -Lite `TrackRun`'s fixes cluster on where a subscription starts and what it -is owed or dropped (e713c30991, 36f82ac450, 35fa44f6bf, e1b5d2741f, -f235fcfcbd, df0bd5c36f, 342445f251, 06765f4c1a, 4927167904, f52848d634, -0513c07369, 5ad3652871). That state spans about eight fields (`floor`, -`cap`, `start`, `first`, `skipped`, `served`, `start_frame`, `end_frame`), -plus `cursor_floor` and a pre-06 re-pin branch, encoding three wire -generations of "start". When ranges land: - -- Move what each cursor is owed and what was dropped into the model, so - `TrackRun` only translates it into SUBSCRIBE_START/DROP/END per version; - delete `skipped`, `served`, `first`, and the pre-06 branches rather than - layering lite-07 ranges on top. -- On lite-07 a stream or a DROP accounts for every sequence except one sent - only as a datagram, which can be lost unseen - (`a_lost_datagram_delays_the_end` in `tests/integration/track_tail.rs`). - So the subscriber's end is "owed range covered and no stream active", - except that a hole a datagram may have left still settles within a bounded - grace; only reliable accounting of datagram sequences could remove that. - Keep `Settle`'s full grace for lite-05/06 and moq-transport behind one - version predicate. -- Lite-07 is unpublished, so sending a fresh START per update can remove the - `requested == start` gate (`lite/subscriber.rs` about line 7366). - ## Required -- [Linear fill path](/quest/m1/subscribe-ranges/fill-linear.md) - filling stays flat as range count and span grow +- [Linear fill path](/quest/m1/subscribe-ranges/fill-linear.md) - filling stays flat as range count and span grow, so a subscription's range count cannot multiply per-group cost - [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md) - `fetch_group` becomes a one-range subscription that ends once its range is accounted for - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft -- [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity, and FETCH removed from lite-07 in both implementations -- [Range count cost](/quest/m1/subscribe-ranges/range-cost.md) - a subscription's range count cannot multiply per-group cost +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity, and the on-demand request surface +- [Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md) - FETCH removed from lite-07 in Rust, JS, and the draft +- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - what each lite cursor is owed and dropped lives in the model, and `TrackRun` only translates it per version ## Related diff --git a/quest/m1/subscribe-ranges/fetch-range.md b/quest/m1/subscribe-ranges/fetch-range.md index 96b2d08e443..15bac7c425e 100644 --- a/quest/m1/subscribe-ranges/fetch-range.md +++ b/quest/m1/subscribe-ranges/fetch-range.md @@ -38,14 +38,14 @@ the questions decided below and landed only the `track_range_filling` benchmark. read has to request it instead of letting `Lagged` reach the caller. `fill::Ask` already carries a start frame, so the miss judge can count a headless slot as a hole for a bounded window. -- **Wire publishers keep parking, except on lite-07.** The lite subscriber - reads an upstream FIN as the track's end on a published version. Once - `recv_group` ends a bounded read, the IETF publisher and lite publishers - before lite-07 keep parking on a bounded downstream SUBSCRIBE rather than FIN - it, or a peer reads the range's end as the track's. A lite-07 publisher FINs - a covered bounded subscription without SUBSCRIBE_END, which ends only the - subscription (maintainer, 2026-10-25, on - [moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807)). +- **Wire publishers keep parking.** The lite subscriber reads an upstream + FIN as the track's end on a published version. Once `recv_group` ends a + bounded read, the IETF publisher and lite publishers before lite-07 keep + parking on a bounded downstream SUBSCRIBE rather than FIN it, or a peer + reads the range's end as the track's. The lite-07 half already landed in + [moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807): a lite-07 + publisher FINs a covered bounded subscription without SUBSCRIBE_END, which + ends only the subscription. - **Route recovery keeps a frame start.** `resume::Recover` fetches the rest of a group from frame N on the new route (`group::Fetch::frame_start`), which a whole-group range cannot say, so a crate-private fetch stays for it. @@ -75,7 +75,9 @@ the questions decided below and landed only the `track_range_filling` benchmark. moq-transport fetch never subscribes, and a lite SUBSCRIBE before lite-05 cannot name a missing group. A range wholly below the live edge stays out of the upstream SUBSCRIBE on any wire without range SUBSCRIBE and is filled by - per-group FETCH, keeping today's wire behavior until + FETCH, keeping today's wire behavior: since + [moq-dev/moq#6724](https://github.com/moq-dev/moq/pull/6724) a run of misses + is one range FETCH on moq-transport, and per-group FETCH is lite-only, until [lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) land. A bounded upstream SUBSCRIBE accounted by SUBSCRIBE_START and SUBSCRIBE_DROP was rejected: it works on lite-05 and later only. diff --git a/quest/m1/subscribe-ranges/fill-linear.md b/quest/m1/subscribe-ranges/fill-linear.md index 44bb220be11..0a27852fd7b 100644 --- a/quest/m1/subscribe-ranges/fill-linear.md +++ b/quest/m1/subscribe-ranges/fill-linear.md @@ -3,7 +3,10 @@ ## Goal `track_range_filling` (`rs/moq-net/benches/track.rs`) holds roughly flat -groups/s as range count and span grow. +groups/s as range count and span grow. The model's cost per group no longer +grows with the number of ranges in a subscription, or that number is bounded +by the design (flow control or an owned budget), not by a refusal constant. A +benchmark sweeps range count and subscriber count, so a slope shows up. ## Plan @@ -14,8 +17,28 @@ drift edge (`live_edge`) and each candidate's successor search filled group is never marked shown, so every scan passes it again. `Misses::retain` also rescans every filled group per poll. +The range count is hostile input too +([moq-dev/moq#6715](https://github.com/moq-dev/moq/pull/6715)): lite-07 +SUBSCRIBE carries a list of ranges, a peer can send as many as fit in one +message, and the model's per-group cost grows with that count, so a hostile +peer can multiply the CPU a relay spends on every group of a track. + +- Measure first: extend `rs/moq-net/benches/track.rs` with ranges × + subscribers beside `track_range_filling`'s ranges × span. +- Prefer merging overlapping or adjacent ranges at decode, and a lookup that + touches only the ranges around a group. +- Include `track::Subscriber::covered`, which the lite-07 publisher runs on + each idle wake of a bounded subscription and walks every range. + +Decided 2026-10-27 (maintainer): + +- The range count cost quest folds in here, since both fix the same walk. +- Re-baseline `track_range_filling` on current `main` before optimizing: + [moq-dev/moq#6921](https://github.com/moq-dev/moq/pull/6921) changed how + sparse misses are judged after the numbers above. + [FETCH as a range](/quest/m1/subscribe-ranges/fetch-range.md) waits on this (maintainer, 2026-10-25, on #6722), since one-range reads would put every past read on this path. -Public API: none. Wire: none. +Public API: none expected. Wire: none expected. diff --git a/quest/m1/subscribe-ranges/js.md b/quest/m1/subscribe-ranges/js.md index daeae7325db..c8ea0c9db4f 100644 --- a/quest/m1/subscribe-ranges/js.md +++ b/quest/m1/subscribe-ranges/js.md @@ -4,20 +4,20 @@ `@moq/net` matches Rust: `Subscription` carries ranges beside its existing `order`, the lite-07 SUBSCRIBE carries them on the wire, and `fetchGroup` -becomes a one-range subscription or is removed. FETCH is gone from lite-07 in -both Rust and JS. A JavaScript publisher serves requested ranges that are no -longer in its live cache. +becomes a one-range subscription or is removed. A JavaScript publisher serves +requested ranges that are no longer in its live cache. ## Plan -Decided 2026-10-25 (maintainer, on -[moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807)): the coordinated -removal of FETCH from lite-07 lands here, in Rust and JS together, once -[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) fills through the -subscription. Rust stops sending and serving lite-07 FETCH, and -`drafts/draft-lcurley-moq-lite.md` drops FETCH from lite-07 with its changelog -bullets (FETCH max age and epoch). Published versions (lite-03..06) keep FETCH. -Mirror lite-07 bounded completion in the publisher too: a covered bounded +Decided 2026-10-27 (maintainer): removing FETCH from lite-07 moved to +[Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md), and +this quest no longer waits on +[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md). The rest starts now: +the ranges model, anchors, `maxDelay: null`, and the on-demand request +surface. Rust's lite-07 SUBSCRIBE already carries ranges +([moq-dev/moq#6715](https://github.com/moq-dev/moq/pull/6715)). + +Mirror lite-07 bounded completion in the publisher: a covered bounded subscription FINs without SUBSCRIBE_END. The JS subscriber already accepts that FIN, closing the request's track once its tail settles, and failing it when an update moved the range and the groups that arrived leave it uncovered. Reopen @@ -33,8 +33,10 @@ omitted one means the live edge only. Decided in the 2026-10-05 audit: the producer half of [JavaScript FETCH](/quest/m1/js-fetch.md) folds in here, so `@moq/net`'s on-demand request surface takes range requests from the start instead of the -per-group `requested_group` shape Rust replaced with `requested_fill`. That quest keeps only -IETF FETCH dispatch. Resized from [M] to [L] for it. +per-group `requested_group` shape Rust replaced with `requested_fill`. IETF +FETCH dispatch onto it is +[on-demand JavaScript FETCH](/quest/m1/js-fetch-on-demand.md). Resized from +[M] to [L] for it. - Expose an owned request handle rather than a storage callback; dropping a request refuses it rather than leaving the subscriber waiting. Preserve @@ -53,6 +55,7 @@ violation, as `lite/subscriber.ts` `runGroup` does. Preserve the per-reader group-bound rule for reliable reads, live-edge selection, subscription updates, and publisher datagrams. -## Required +## Related -- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the wire this speaks +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the Rust lite-07 range fill this mirrors +- [Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md) - follows this and Lite-07 ranges diff --git a/quest/m1/subscribe-ranges/lite.md b/quest/m1/subscribe-ranges/lite.md index 13956fd7d33..ccfa8186641 100644 --- a/quest/m1/subscribe-ranges/lite.md +++ b/quest/m1/subscribe-ranges/lite.md @@ -2,23 +2,26 @@ ## Goal -A relay fills a lite-07 upstream by subscribing to the missing ranges. Older -wires and moq-transport carry the anchors they can, and refuse only the rest. -FETCH stays on lite-07 until [JS ranges](/quest/m1/subscribe-ranges/js.md) +A relay fills a lite-07 upstream by subscribing to the missing ranges, and +SUBSCRIBE_OK carries the resolved start. Older wires and moq-transport carry +the anchors they can, and refuse only the rest. FETCH stays on lite-07 until +[Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md) removes it from both implementations. ## Plan Decided 2026-10-25 (maintainer, on [moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807)): FETCH removal -moves to JS ranges, since JS still requests and serves wire FETCH on lite-07 +moves out of this quest (to its own quest since 2026-10-27), since JS still requests and serves wire FETCH on lite-07 (`track.Consumer.fetchGroup`), so a Rust-only removal would drop a current cross-language capability. This quest keeps FETCH on lite-07 and makes the range fill work without it. Decided 2026-10-22 (maintainer): SUBSCRIBE_OK carries the resolved start for a relative range start, so a reader's ordered cursor takes its floor from it -([Container reads drop backfill](/quest/m1/read-backfill.md)). +([Container reads drop backfill](/quest/m1/read-backfill.md)). Decided +2026-10-27 (maintainer): it is part of the Goal, since read-backfill now +requires this quest for it. ### Range fill through the subscription @@ -57,12 +60,13 @@ first. ### moq-transport anchors -moq-transport's relative offsets differ by one. Draft-22's relative filter -counts back from the next group: `Filter::Relative(0)` is the next group and -`Relative(1)` the current one, so an open-ended `Relative(n)..` maps to -`Filter::Relative(n + 1)`, saturating. Draft 18 only encodes a relative -0 (NEXT_GROUP), so there it is SUBSCRIBE plus a relative joining FETCH n -groups back, as the live join already does. A range with a bounded end +Decided 2026-10-27 (maintainer): a bridge never sends a relative anchor to +moq-transport. It resolves `Relative(n)` to an absolute Group ID, from its +cache or the live edge, before sending, replacing the planned mapping of +`Relative(n)..` onto draft-22's `Filter::Relative(n + 1)` and draft 18's +relative joining FETCH. Reason: both count back by Group ID arithmetic, which +breaks once [time addressing](/quest/m1/time-addressing/model.md) makes legacy +Group IDs sparse start ticks. A range with a bounded end resolves both bounds against the Largest Object and goes upstream as an absolute range. Live keeps today's join (`ietf::subscriber::subscribe_join`). Each mapping that makes a wire carry an anchor lifts that wire's @@ -75,4 +79,4 @@ Update `drafts/draft-lcurley-moq-lite.md` with each wire change here. Run ## Related - [Container reads drop backfill](/quest/m1/read-backfill.md) - floors its ordered cursor on the resolved start -- [JS ranges](/quest/m1/subscribe-ranges/js.md) - removes FETCH from lite-07 in both implementations once this lands +- [Remove FETCH from lite-07](/quest/m1/subscribe-ranges/lite07-fetch.md) - removes FETCH from lite-07 in both implementations once this and JS ranges land diff --git a/quest/m1/subscribe-ranges/lite07-fetch.md b/quest/m1/subscribe-ranges/lite07-fetch.md new file mode 100644 index 00000000000..49f653de502 --- /dev/null +++ b/quest/m1/subscribe-ranges/lite07-fetch.md @@ -0,0 +1,35 @@ +# [M] Remove FETCH from lite-07 + +## Goal + +Lite-07 has no FETCH, in Rust, JS, and the draft. Rust and JS stop sending +and serving lite-07 FETCH, and read past groups through range subscriptions +instead. Published versions (lite-03..06) keep FETCH. + +## Plan + +Decided 2026-10-25 (maintainer, on +[moq-dev/moq#6807](https://github.com/moq-dev/moq/pull/6807)): the removal +lands in Rust and JS together, once both fill through the subscription, since +JS still requests and serves wire FETCH on lite-07 +(`track.Consumer.fetchGroup`) and a Rust-only removal would drop a current +cross-language capability. + +Decided 2026-10-27 (maintainer): split out of +[JS ranges](/quest/m1/subscribe-ranges/js.md), so the rest of JS ranges +starts now instead of waiting on +[Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md). This quest waits on +both. + +- `drafts/draft-lcurley-moq-lite.md` drops FETCH from lite-07, with its + changelog bullets (FETCH max age and epoch). Run `just drafts check`. +- Lite-07 has the only `NotFetchable` code; check whether it still has a + sender once FETCH is gone. +- Run `just test interop --all`. + +Public API: none expected. Wire: lite-07 (unpublished) loses FETCH. + +## Required + +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - Rust fills lite-07 through the subscription +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - JS reads and serves ranges, so it no longer needs lite-07 FETCH diff --git a/quest/m1/subscribe-ranges/range-cost.md b/quest/m1/subscribe-ranges/range-cost.md deleted file mode 100644 index 03572a554df..00000000000 --- a/quest/m1/subscribe-ranges/range-cost.md +++ /dev/null @@ -1,24 +0,0 @@ -# [M] A subscription's range count cannot multiply per-group cost - -## Goal - -The model's cost per group no longer grows with the number of ranges in a -subscription, or that number is bounded by the design (flow control or an -owned budget), not by a refusal constant. A benchmark sweeps range count and -subscriber count, so a slope shows up. - -## Plan - -From [moq-dev/moq#6715](https://github.com/moq-dev/moq/pull/6715): lite-07 -SUBSCRIBE carries a list of ranges, a peer can send as many as fit in one -message, and the model's per-group cost grows with that count. A hostile peer -can multiply the CPU a relay spends on every group of a track. - -- Measure first: extend `rs/moq-net/benches/track.rs` with ranges × subscribers. -- Prefer merging overlapping or adjacent ranges at decode, and a lookup that - touches only the ranges around a group. -- Include `track::Subscriber::covered`, which the lite-07 publisher runs on - each idle wake of a bounded subscription and walks every range. -- Shares code with [Linear fill path](/quest/m1/subscribe-ranges/fill-linear.md). - -Public API: none expected. Wire: none expected. diff --git a/quest/m1/subscribe-ranges/start-bookkeeping.md b/quest/m1/subscribe-ranges/start-bookkeeping.md new file mode 100644 index 00000000000..8567464585c --- /dev/null +++ b/quest/m1/subscribe-ranges/start-bookkeeping.md @@ -0,0 +1,44 @@ +# [M] Delete the old start bookkeeping + +## Goal + +What each lite cursor is owed and what was dropped lives in the model, and +lite `TrackRun` only translates it into SUBSCRIBE_START, SUBSCRIBE_DROP, and +SUBSCRIBE_END per version. `skipped`, `served`, `first`, and the pre-06 +branches are deleted rather than layering lite-07 ranges on top, and on +lite-07 a subscriber's end is "owed range covered and no stream active". + +## Plan + +Moved out of the [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) +README's own work, where the 2026-10-24 pre-cut audit put it. Decided +2026-10-27 (maintainer): a child quest of its own, since +[Rust groups by start](/quest/m1/time-addressing/model.md) requires it before +the cut moves this state onto times, while the rest of the README waits on +every child. + +Lite `TrackRun`'s fixes cluster on where a subscription starts and what it +is owed or dropped (e713c30991, 36f82ac450, 35fa44f6bf, e1b5d2741f, +f235fcfcbd, df0bd5c36f, 342445f251, 06765f4c1a, 4927167904, f52848d634, +0513c07369, 5ad3652871). That state spans about eight fields (`floor`, +`cap`, `start`, `first`, `skipped`, `served`, `start_frame`, `end_frame`), +plus `cursor_floor` and a pre-06 re-pin branch, encoding three wire +generations of "start". + +- On lite-07 a stream or a DROP accounts for every sequence except one sent + only as a datagram, which can be lost unseen + (`a_lost_datagram_delays_the_end` in `tests/integration/track_tail.rs`). + So the subscriber's end is "owed range covered and no stream active", + except that a hole a datagram may have left still settles within a bounded + grace; only reliable accounting of datagram sequences could remove that. + Keep `Settle`'s full grace for lite-05/06 and moq-transport behind one + version predicate. +- Lite-07 is unpublished, so sending a fresh START per update can remove the + `requested == start` gate (`lite/subscriber.rs`, about line 4898). + +Public API: none expected. Wire: lite-07 (unpublished) may send a START per +update; published versions are unchanged. + +## Required + +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 range fill this bookkeeping is rebuilt around diff --git a/quest/m1/time-addressing/bindings.md b/quest/m1/time-addressing/bindings.md index 9b6885b261d..34b051f96a8 100644 --- a/quest/m1/time-addressing/bindings.md +++ b/quest/m1/time-addressing/bindings.md @@ -11,7 +11,8 @@ failover, with each `doc/lib` page and the PR's Impact section saying how to mig Follow the cross-package sync table from `rs/moq-ffi` outward. Timestamps already cross the bindings as `{value, scale}` ([exact timestamps](/quest/m1/binding-timestamps.md)), so a start reuses that -type. +type. Every track is timed after the cut, so reads drop the `Option` that +exact timestamps keeps for untimed (decided 2026-10-27, maintainer). Public API: breaking in every binding. Wire: none. diff --git a/quest/m1/time-addressing/e2ee.md b/quest/m1/time-addressing/e2ee.md index 389209c6f04..d8a6040c30c 100644 --- a/quest/m1/time-addressing/e2ee.md +++ b/quest/m1/time-addressing/e2ee.md @@ -8,12 +8,14 @@ the cut. ## Plan -Update `drafts/draft-lcurley-moq-e2ee.md` and `rs/moq-e2ee`, plus the JS -side if it has landed. The start is unique per track because starts strictly -increase. Run `just drafts check`. +Update `drafts/draft-lcurley-moq-e2ee.md`, `rs/moq-e2ee`, and the JS side, +which has landed: `js/e2ee/src/derive.ts` builds `nonce(group, frame)` (about +line 156). The start is unique per track because starts strictly increase. +Run `just drafts check`. Public API: none. Wire: the E2EE nonce input changes. ## Required - [Rust groups by start](/quest/m1/time-addressing/model.md) - the start the nonce binds +- [@moq/net by start](/quest/m1/time-addressing/js.md) - the JS start the JS half binds diff --git a/quest/m1/time-addressing/js.md b/quest/m1/time-addressing/js.md index 05f0e96f7a8..c3fcb646510 100644 --- a/quest/m1/time-addressing/js.md +++ b/quest/m1/time-addressing/js.md @@ -17,10 +17,12 @@ start past 2^53 is refused as today's group IDs are. The spike found no sparse-ID breakage in direct `@moq/net` sessions over lite-06 and lite-07; its relay fills follow Rust's once -[JS ranges](/quest/m1/subscribe-ranges/js.md) land. +[JS ranges](/quest/m1/subscribe-ranges/js.md) land. Decided 2026-10-27 +(maintainer): this requires JS ranges, so the ranges it rekeys exist first. Public API: breaking, mirroring Rust's. Wire: as the lite-07 quest. ## Required - [Lite-07 by start](/quest/m1/time-addressing/lite07.md) - the wire this speaks +- [JS ranges](/quest/m1/subscribe-ranges/js.md) - the JS range model and relay fills this moves onto starts diff --git a/quest/m1/time-addressing/model.md b/quest/m1/time-addressing/model.md index 3b873b58222..e8d0bd401d2 100644 --- a/quest/m1/time-addressing/model.md +++ b/quest/m1/time-addressing/model.md @@ -19,7 +19,7 @@ a producer would need per-track memory of its last number. Frames inside a group stay in decode order and their PTS may go backwards (B-frames). - **Starts strictly increase.** An equal start is refused loudly like a - rewind; `Neighbors::admit` (`model/track.rs:491`) allows ties today. + rewind; `Neighbors::admit` (`model/track.rs`, about line 502) allows ties today. - **Three ways a group ends**: `finish(end)` with an exclusive end timestamp (the group is whole), `close()` (incomplete, but every frame written arrives), and `abort(err)` (reset). A relay closes an expired or partial @@ -51,6 +51,13 @@ a producer would need per-track memory of its last number. [moq-dev/moq#6780](https://github.com/moq-dev/moq/pull/6780); the cut removes its sequence half, leaving a re-created track to keep its PTS moving forward. +- **What it waits on** (decided 2026-10-27, maintainer): only the + subscribe-ranges children that put sequence ranges into the model and the + wire, [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md), + [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md), and + [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md), + not the whole epic. [Drop set](/quest/m1/drop-set.md) lands first too: its + runs keyed by group sequence convert mechanically to starts. - **Old relays** (decided 2026-10-26, maintainer): a released relay fills a hole over lite-05/06 with one FETCH per integer ID, so sparse IDs cost it one FETCH per tick in the hole. Accepted: "This was a lingering problem. @@ -109,8 +116,11 @@ Wire: no format change; legacy and lite-07-wip group IDs become start ticks. ## Required -- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - the range ends and drops this rework moves onto times +- [Fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md) - the one-range reads this rework moves onto times +- [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the range ends and drops this rework moves onto times +- [Delete the old start bookkeeping](/quest/m1/subscribe-ranges/start-bookkeeping.md) - the lite cursor state lives in the model before the cut rekeys it - [Lite-05/06 timestamps pass through](/quest/m1/lite-timestamps-through.md) - lite-05/06 tracks stay timed, so their ingest keeps real starts +- [Drop set](/quest/m1/drop-set.md) - its dropped and expired runs convert mechanically from sequences to starts ## Related diff --git a/quest/m1/unbounded-groups/README.md b/quest/m1/unbounded-groups/README.md index 946e2155e85..ed61d91fefe 100644 --- a/quest/m1/unbounded-groups/README.md +++ b/quest/m1/unbounded-groups/README.md @@ -72,11 +72,10 @@ Decided 2026-10-16 (maintainer interview): benchmark, [moq-dev/moq#6627](https://github.com/moq-dev/moq/pull/6627)): the lock-free append-only log (`kio::log`) is the tail run, from a base that only advances, and refilled runs sit below it. - It moved here from m2, ahead of Front eviction, so the end-state storage - lands once instead of a mutex layout swapped later. Prototyped, the log - cost 10 to 40% less per reader across cores at 1,024 readers and about - 15 ns more per append than today's `VecDeque`. A ring buffer was rejected: - an uncapped group has no size to give it. Numbers are in the + It landed in [moq-dev/moq#6782](https://github.com/moq-dev/moq/pull/6782) + (d854bf98b1), ahead of Front eviction, including `log::Writer::evict` + advancing the base (`rs/kio/src/log.rs`). A ring buffer was rejected: an + uncapped group has no size to give it. Numbers are in #6782 and the [kio contention epic](/quest/m2/perf/kio-channel-contention/README.md). - Readers pin nothing: reads refresh recency as today. A reader that falls into an evicted run refills like a joiner. With nothing to refill @@ -89,8 +88,10 @@ Decided 2026-10-16 (maintainer interview): rather than refetching. Only whole-object holes refill. - `GroupTooLarge` and `FrameTooLarge` stay, raised only when an allocation can't be satisfied. -- The byte offset goes on the next unpublished lite version after lite-07, - not lite-07. Frame-boundary holes refill over lite-07's frame offsets once +- The byte offset goes on lite-07-wip (decided 2026-10-27, maintainer, + replacing "the next unpublished lite version after lite-07"), so + [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) waits only on Front + eviction. Frame-boundary holes refill over lite-07's frame offsets once [transport frame offsets](/quest/m1/transport-frame-offsets.md) lets a refill start at a frame the relay no longer holds below (decided 2026-10-25, maintainer: the application API stays whole-group). @@ -112,8 +113,9 @@ moq-transport, with memory held near the pool size. - [Front eviction](/quest/m1/unbounded-groups/front-eviction.md) - groups shed whole frames from their front under pressure and refill them in place; the group caps go - [Fastest-reader pacing](/quest/m1/unbounded-groups/fastest-reader.md) - every hop buffers at most a window past its fastest reader and pushes back through QUIC flow control -- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - segments inside a frame evict and refill by `(frame, byte)` on the next lite version; the frame cap goes -- [JS unbounded groups](/quest/m1/unbounded-groups/js.md) - `@moq/net` drops its caps, streams huge frames, and matches front eviction +- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - segments inside a frame evict and refill by `(frame, byte)` on lite-07; the frame cap goes +- [JS unbounded groups](/quest/m1/unbounded-groups/js.md) - `@moq/net` drops its caps and matches front eviction +- [JS byte holes](/quest/m1/unbounded-groups/js-byte-holes.md) - `@moq/net` reads a huge frame as chunks and mirrors the byte-offset wire - [Log lock profile](/quest/m1/unbounded-groups/log-lock-profile.md) - a lock profile on a perf host confirms the epoch-based `kio::log` cut `GroupState` contention ## Related diff --git a/quest/m1/unbounded-groups/byte-holes.md b/quest/m1/unbounded-groups/byte-holes.md index 7042a6563d1..19aa5e2fb3e 100644 --- a/quest/m1/unbounded-groups/byte-holes.md +++ b/quest/m1/unbounded-groups/byte-holes.md @@ -4,13 +4,16 @@ A frame of any size, such as a file delivered as one frame, passes through a relay whose pool is smaller than the frame. Sealed segments inside the frame -are evicted like front frames and refilled by `(frame, byte)` position on the -next unpublished lite version after lite-07. The frame cap +are evicted like front frames and refilled by `(frame, byte)` position on +lite-07. The frame cap (`MAX_CACHE_BYTES`) is removed. ## Plan See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. +Decided 2026-10-27 (maintainer): the byte offset targets lite-07-wip, not the +next unpublished lite version after lite-07, so this waits only on Front +eviction and stays in m1. - Segments are `frame::SEGMENT` (1 MiB) long, so segment `i` always holds bytes from `i * SEGMENT`, and a completed chain is `frame::Payload::Chain`. @@ -42,13 +45,13 @@ See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. - Rust readers get a chunked payload read, so a frame larger than memory never has to be collected whole. [moq-dev/moq#6628](https://github.com/moq-dev/moq/pull/6628) kept its segments internal; the JS half is in - [JS](/quest/m1/unbounded-groups/js.md). + [JS byte holes](/quest/m1/unbounded-groups/js-byte-holes.md). - Test a frame several times the pool size through a relay chain: a late joiner reads it whole, and memory stays near the pool size. Public API: `Position` gains a byte field, frames gain a chunked read, and `MAX_CACHE_BYTES` is removed. -Wire: a byte offset on the next lite version's range ends and group header. +Wire: a byte offset on lite-07-wip's range ends and group header. ## Required diff --git a/quest/m1/unbounded-groups/front-eviction.md b/quest/m1/unbounded-groups/front-eviction.md index 7a7122d79cb..56e8711fe1c 100644 --- a/quest/m1/unbounded-groups/front-eviction.md +++ b/quest/m1/unbounded-groups/front-eviction.md @@ -18,19 +18,16 @@ records". See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. -- Storage is the lock-free - append-only log (`kio::log`) (decided 2026-10-24 - by the maintainer over the group storage benchmark, - [moq-dev/moq#6627](https://github.com/moq-dev/moq/pull/6627): land the - end-state storage now rather than a mutex layout swapped later; numbers in - the [kio contention epic](/quest/m2/perf/kio-channel-contention/README.md)). - Frames keep absolute indices. The log is the tail run, from a base that - only advances: front eviction raises its base. Refilled frames go into - sorted runs below the base, behind their own mutex, and never rejoin the - log. Readers keep one absolute cursor, read the log lock-free, and search - the refill runs only when the cursor falls below the base. Prototyped this - way, the log cost 10 to 40% less per reader across cores and about 15 ns - more per append. +- Storage is the lock-free append-only log (`kio::log`), which landed in + [moq-dev/moq#6782](https://github.com/moq-dev/moq/pull/6782) (decided + 2026-10-24 by the maintainer over the group storage benchmark, + [moq-dev/moq#6627](https://github.com/moq-dev/moq/pull/6627)). Frames keep + absolute indices. The log is the tail run, from a base that only advances: + front eviction raises its base through `log::Writer::evict`, which already + exists (`rs/kio/src/log.rs`). Refilled frames go into sorted runs below + the base, behind their own mutex, and never rejoin the log. Readers keep + one absolute cursor, read the log lock-free, and search the refill runs + only when the cursor falls below the base. - Pool debt evicts whole cold groups first (`pay_debt` in `model/track.rs`, as today). Front frames are shed only from a group that is protected or being read. diff --git a/quest/m1/unbounded-groups/js-byte-holes.md b/quest/m1/unbounded-groups/js-byte-holes.md new file mode 100644 index 00000000000..142daa5c0a7 --- /dev/null +++ b/quest/m1/unbounded-groups/js-byte-holes.md @@ -0,0 +1,31 @@ +# [S] JS byte holes + +## Goal + +A huge frame can be read in `@moq/net` as a stream of chunks instead of one +`Uint8Array`, so a browser can receive a file-sized frame without holding it +whole, and `@moq/net` speaks the lite-07 byte offset on range ends and group +headers as Rust does. + +## Plan + +Decided 2026-10-27 (maintainer): split out of +[JS unbounded groups](/quest/m1/unbounded-groups/js.md), which now waits only +on front eviction. This half mirrors +[Byte holes](/quest/m1/unbounded-groups/byte-holes.md). + +- Mirror the Rust names for the chunked frame read. +- Mirror the byte offset on the lite-07 wire, and refill a hole inside a + frame the way Rust does. +- Test a frame several times the cache window through a relay, read as + chunks, plus Rust/JS interop of a mid-frame refill. Run + `just test interop --all`. +- If [Rust-backed @moq/net](/quest/m2/rs2ts/README.md) is adopted first, + this shrinks to exposing the streaming read. + +Public API: adds a chunked frame read. Wire: the lite-07 byte offset, as +Byte holes defines it. + +## Required + +- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - the Rust contract and the lite wire this mirrors diff --git a/quest/m1/unbounded-groups/js.md b/quest/m1/unbounded-groups/js.md index 254b9643721..b1fe1fb5bfa 100644 --- a/quest/m1/unbounded-groups/js.md +++ b/quest/m1/unbounded-groups/js.md @@ -2,9 +2,7 @@ ## Goal -`@moq/net` has no group frame-count, group byte, or frame size cap. A huge -frame can be read as a stream of chunks instead of one `Uint8Array`, so a -browser can receive a file-sized frame without holding it whole. A reader +`@moq/net` has no group frame-count, group byte, or frame size cap. A reader that falls behind an evicted front gets the same refill-or-`Evicted` behavior as Rust. @@ -12,6 +10,12 @@ as Rust. See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. +Decided 2026-10-27 (maintainer): this waits only on +[Front eviction](/quest/m1/unbounded-groups/front-eviction.md), since +removing the caps and adding the JS front trigger need nothing from byte +holes. The chunked frame read and the byte-offset wire mirror moved to +[JS byte holes](/quest/m1/unbounded-groups/js-byte-holes.md). + - Today `js/net/src/group.ts` caps frames and bytes and throws `GroupTooLarge`, and JS has no byte pool. Its eviction is whole groups by idle time, which never fires on a group that is still being written. @@ -22,16 +26,14 @@ See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. - Test a single group written continuously for several windows: memory stays near one window, and a reader that falls behind gets refill or `Evicted`. -- Mirror the Rust names for the streaming frame read. - Update `js/json` and its tests, which lean on the removed constants, and the compression paths now in `js/net`. The [m1 compression migration](/quest/m1/compress/migrate.md) deletes `js/flate`. - If [Rust-backed @moq/net](/quest/m2/rs2ts/README.md) is adopted first, - this shrinks to exposing the streaming read. + Rust's caps and eviction apply and this quest is done. -Public API: removes the JS cap constants, and adds a chunked frame read. -Wire: none beyond Byte holes. +Public API: removes the JS cap constants. Wire: none. ## Required -- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - the Rust contract and the lite wire this mirrors +- [Front eviction](/quest/m1/unbounded-groups/front-eviction.md) - the Rust contract this mirrors diff --git a/quest/m1/unbounded-groups/log-lock-profile.md b/quest/m1/unbounded-groups/log-lock-profile.md index eb88d17630b..e174a7bca70 100644 --- a/quest/m1/unbounded-groups/log-lock-profile.md +++ b/quest/m1/unbounded-groups/log-lock-profile.md @@ -4,19 +4,23 @@ `just bench-profile lock video-heavy` runs before and after the epoch-based `kio::log` ([moq-dev/moq#6782](https://github.com/moq-dev/moq/pull/6782)) on a -host with `kernel.perf_event_paranoid=-1`, and shows no lock-contention +quiet profiling host (see `bench/README.md`), and shows no lock-contention regression: `GroupState`'s share of mutex wait with multi-frame groups drops or holds. ## Plan - Compare the commit before #6782 with the one that merged it, at each worker - count, and post the wait-by-stack diff on the PR that finishes this. + count, and post the wait-by-stack diff on the PR that finishes this. The + recipe's base revision + ([moq-dev/moq#6680](https://github.com/moq-dev/moq/pull/6680)) covers the + "before" side in one run: + `just bench-profile lock video-heavy "1 4 16" tokio-workers d854bf98b1~1`. +- Decided 2026-10-27 (maintainer): the profiling host is the 24-core + `desktop` workstation (`kernel.perf_event_paranoid=-1`), run while quiet: + no other agents running and load near zero. That retired the profiling-host + condition this waited on. - #6782's body holds the criterion numbers, taken on an oversubscribed host: `group_contended` costs 13 to 61% less per reader across cores, and about 4 ns more per frame on one thread. A profile that contradicts them goes back to the maintainer. - -## Required - -- [A profiling host](/quest/m1/perf-host.md) - the `perf_event_paranoid=-1` host the lock profile runs on diff --git a/quest/m1/watch-buffer-switch.md b/quest/m1/watch-buffer-switch.md new file mode 100644 index 00000000000..82702510910 --- /dev/null +++ b/quest/m1/watch-buffer-switch.md @@ -0,0 +1,24 @@ +# [S] Switching buffer at runtime switches the audio ring + +## Goal + +Changing a ``'s `buffer` after playback started switches the +audio ring's mode: going from live to buffered stops shedding future-dated +audio, and going back leaves no live player in play-through. + +## Plan + +Found by the 2026-10-24 pre-cut audit. Decided 2026-10-27 (maintainer): its +own ready quest, split out of +[buffered ascending](/quest/m1/buffered-ascending.md), which waits on +lite-07 while this bug does not. + +`js/watch/src/audio/decoder.ts` reads `sync.out.buffered` once when it builds +the worklet's ring (about line 294), and `#order()` peeks it once per +attempt, so going from live to buffered after playback started leaves +`Backpressure` and the worklet `Init` live (future-dated TTS audio is shed), +and the reverse leaves a live player in play-through. Key the ring and +`Order` on `sync.out.buffered`: rebuild the ring, or post a mode message plus +a reset. Test both directions on a running player. + +Public API: none. Wire: none. From 6162e764174d987f83b4472c914f7de8b6ff1d0e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 27 Oct 2026 14:35:42 -0700 Subject: [PATCH 2/3] quest: producers keep optional timestamps until the cut Co-Authored-By: Claude Opus 5.5 --- quest/m1/binding-timestamps.md | 9 +++++---- quest/m1/time-addressing/bindings.md | 5 +++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/quest/m1/binding-timestamps.md b/quest/m1/binding-timestamps.md index 09bc4a61cfc..9b75e4b5baa 100644 --- a/quest/m1/binding-timestamps.md +++ b/quest/m1/binding-timestamps.md @@ -62,11 +62,12 @@ on the generated C package: the hand-written `moq-c` ships at this cut with parity to moq-ffi for every m1 binding feature, so the moq-c half lands there, and the generated C line follows after the cut. -Decided 2026-10-27 (maintainer): "untimed as absent" is needed only on the -read side, and only until the cut, when +Decided 2026-10-27 (maintainer): producer arguments keep an optional +timestamp until the cut, so producers may still write untimed frames. Reads +keep "untimed as absent" until then too. At the cut [time addressing](/quest/m1/time-addressing/README.md) makes every track -timed; [bindings by start](/quest/m1/time-addressing/bindings.md) then drops -that `Option` for reads. +timed, and [bindings by start](/quest/m1/time-addressing/bindings.md) drops +that `Option` from reads only. Wall and monotonic times stay `moq_clock::Wall` and `moq_clock::Instant`, not media timestamps; keep the types distinct. diff --git a/quest/m1/time-addressing/bindings.md b/quest/m1/time-addressing/bindings.md index 34b051f96a8..67efa3eef4c 100644 --- a/quest/m1/time-addressing/bindings.md +++ b/quest/m1/time-addressing/bindings.md @@ -11,8 +11,9 @@ failover, with each `doc/lib` page and the PR's Impact section saying how to mig Follow the cross-package sync table from `rs/moq-ffi` outward. Timestamps already cross the bindings as `{value, scale}` ([exact timestamps](/quest/m1/binding-timestamps.md)), so a start reuses that -type. Every track is timed after the cut, so reads drop the `Option` that -exact timestamps keeps for untimed (decided 2026-10-27, maintainer). +type. Decided 2026-10-27 (maintainer): every track is timed after the cut, +so reads drop the `Option` that exact timestamps keeps for untimed. Only +reads drop it here; producer arguments keep their optional timestamp. Public API: breaking in every binding. Wire: none. From 0f8af69e3d3ac5bce5c2e0232882956e6367b524 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 27 Oct 2026 14:46:40 -0700 Subject: [PATCH 3/3] quest: byte holes targets the current lite wip, not the lite-07 cut The lite-07 cut does not wait on byte holes. Per lite07-finalize's freeze policy, it lands on whichever lite wip is current (lite-07-wip today); if lite-07 is cut first, lite-07 peers refill only at frame boundaries. Co-Authored-By: Claude Opus 5.5 --- quest/m1/unbounded-groups/README.md | 10 ++++++---- quest/m1/unbounded-groups/byte-holes.md | 15 +++++++++------ quest/m1/unbounded-groups/js-byte-holes.md | 8 ++++---- 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/quest/m1/unbounded-groups/README.md b/quest/m1/unbounded-groups/README.md index ed61d91fefe..2c6ea82eabc 100644 --- a/quest/m1/unbounded-groups/README.md +++ b/quest/m1/unbounded-groups/README.md @@ -88,10 +88,12 @@ Decided 2026-10-16 (maintainer interview): rather than refetching. Only whole-object holes refill. - `GroupTooLarge` and `FrameTooLarge` stay, raised only when an allocation can't be satisfied. -- The byte offset goes on lite-07-wip (decided 2026-10-27, maintainer, - replacing "the next unpublished lite version after lite-07"), so +- The byte offset goes on the current lite wip version (lite-07-wip today); + the lite-07 cut does not wait on it. Decided 2026-10-27 (maintainer): wire + quests target whichever wip is current when they land, so [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) waits only on Front - eviction. Frame-boundary holes refill over lite-07's frame offsets once + eviction. If lite-07 is cut first, lite-07 peers refill only at frame + boundaries. Frame-boundary holes refill over lite-07's frame offsets once [transport frame offsets](/quest/m1/transport-frame-offsets.md) lets a refill start at a frame the relay no longer holds below (decided 2026-10-25, maintainer: the application API stays whole-group). @@ -113,7 +115,7 @@ moq-transport, with memory held near the pool size. - [Front eviction](/quest/m1/unbounded-groups/front-eviction.md) - groups shed whole frames from their front under pressure and refill them in place; the group caps go - [Fastest-reader pacing](/quest/m1/unbounded-groups/fastest-reader.md) - every hop buffers at most a window past its fastest reader and pushes back through QUIC flow control -- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - segments inside a frame evict and refill by `(frame, byte)` on lite-07; the frame cap goes +- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - segments inside a frame evict and refill by `(frame, byte)` on the current lite wip; the frame cap goes - [JS unbounded groups](/quest/m1/unbounded-groups/js.md) - `@moq/net` drops its caps and matches front eviction - [JS byte holes](/quest/m1/unbounded-groups/js-byte-holes.md) - `@moq/net` reads a huge frame as chunks and mirrors the byte-offset wire - [Log lock profile](/quest/m1/unbounded-groups/log-lock-profile.md) - a lock profile on a perf host confirms the epoch-based `kio::log` cut `GroupState` contention diff --git a/quest/m1/unbounded-groups/byte-holes.md b/quest/m1/unbounded-groups/byte-holes.md index 19aa5e2fb3e..ae03b21e694 100644 --- a/quest/m1/unbounded-groups/byte-holes.md +++ b/quest/m1/unbounded-groups/byte-holes.md @@ -5,15 +5,18 @@ A frame of any size, such as a file delivered as one frame, passes through a relay whose pool is smaller than the frame. Sealed segments inside the frame are evicted like front frames and refilled by `(frame, byte)` position on -lite-07. The frame cap -(`MAX_CACHE_BYTES`) is removed. +the current lite wip version. The frame cap (`MAX_CACHE_BYTES`) is removed. ## Plan See the [epic](/quest/m1/unbounded-groups/README.md) for the decisions. -Decided 2026-10-27 (maintainer): the byte offset targets lite-07-wip, not the -next unpublished lite version after lite-07, so this waits only on Front -eviction and stays in m1. +Decided 2026-10-27 (maintainer): the byte offset targets the current lite wip +version (lite-07-wip today); the lite-07 cut does not wait on it. This follows +[lite-07's freeze policy](/quest/m1/lite07-finalize.md): wire quests target +whichever wip is current when they land, and its Required list is not a +closed freeze set. If lite-07 is cut first, the offset goes on the next wip +and lite-07 peers refill only at frame boundaries, like older lite versions. +This waits only on Front eviction and stays in m1. - Segments are `frame::SEGMENT` (1 MiB) long, so segment `i` always holds bytes from `i * SEGMENT`, and a completed chain is `frame::Payload::Chain`. @@ -51,7 +54,7 @@ eviction and stays in m1. Public API: `Position` gains a byte field, frames gain a chunked read, and `MAX_CACHE_BYTES` is removed. -Wire: a byte offset on lite-07-wip's range ends and group header. +Wire: a byte offset on the current lite wip's range ends and group header. ## Required diff --git a/quest/m1/unbounded-groups/js-byte-holes.md b/quest/m1/unbounded-groups/js-byte-holes.md index 142daa5c0a7..459fd6e96d5 100644 --- a/quest/m1/unbounded-groups/js-byte-holes.md +++ b/quest/m1/unbounded-groups/js-byte-holes.md @@ -4,7 +4,7 @@ A huge frame can be read in `@moq/net` as a stream of chunks instead of one `Uint8Array`, so a browser can receive a file-sized frame without holding it -whole, and `@moq/net` speaks the lite-07 byte offset on range ends and group +whole, and `@moq/net` speaks the lite wip byte offset on range ends and group headers as Rust does. ## Plan @@ -15,15 +15,15 @@ on front eviction. This half mirrors [Byte holes](/quest/m1/unbounded-groups/byte-holes.md). - Mirror the Rust names for the chunked frame read. -- Mirror the byte offset on the lite-07 wire, and refill a hole inside a - frame the way Rust does. +- Mirror the byte offset on the same lite wip version as Rust, and refill a + hole inside a frame the way Rust does. - Test a frame several times the cache window through a relay, read as chunks, plus Rust/JS interop of a mid-frame refill. Run `just test interop --all`. - If [Rust-backed @moq/net](/quest/m2/rs2ts/README.md) is adopted first, this shrinks to exposing the streaming read. -Public API: adds a chunked frame read. Wire: the lite-07 byte offset, as +Public API: adds a chunked frame read. Wire: the lite wip byte offset, as Byte holes defines it. ## Required