Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,11 @@ refused rather than reported.
sudo sysctl -w kernel.perf_event_paranoid=-1
```

The maintainer's 24-core `desktop` workstation is the profiling host: it
keeps `kernel.perf_event_paranoid=-1`. Profile there only while it is quiet,
with no other agents running and load near zero, since other work on the
host shows up as lock waits and CPU the relay did not cause.

### Heap

The heap mode builds the relay with `moq-relay/jemalloc` (the other modes keep
Expand Down
4 changes: 2 additions & 2 deletions quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@ The historical samples locate candidates; they are not controlled benchmarks.
## Required

- [Land main through quest land](/quest/m0/ci/land.md) - one self-hosted runner tests every queued PR as a squash chain and fast-forwards `main` to the longest passing prefix
- [Bump quest to quest land](/quest/m0/ci/land-bump.md) - condition: kixelated/quest ships `quest land`; then bump the flake input
- [Measure nightly compiler levers](/quest/m0/ci/nightly-levers.md) - the parallel frontend and Cranelift, adopted only if the numbers beat their risks
- [Bump quest to quest land](/quest/m0/ci/land-bump.md) - kixelated/quest shipped `quest land` in kixelated/quest#229; bump the flake input
- [Measure the parallel frontend](/quest/m0/ci/nightly-levers.md) - a pinned nightly's parallel frontend, adopted only if the numbers beat its risks; Cranelift is ruled out
- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands
- [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix
- [Unify dependency features](/quest/m0/ci/hakari.md) - a cargo-hakari workspace-hack so every job compiles each dependency alike and mbx hits across jobs
Expand Down
16 changes: 9 additions & 7 deletions quest/m0/ci/hakari.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ feature set in another job is a miss, however warm the cache.
- Unifying features means some packages build features they don't use. Measure
whether that costs more CPU than the cache hits save; target, profile, and
`RUSTFLAGS` differences still split keys.
- Start from the hit report in [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md),
which lists the compilations that miss only because their features or flags
differ.
- Measure the baseline here, before generating the hack: across jobs, which
compilations one job misses although another job built the same crate at
the same version, grouped by what differs (features, profile, `RUSTFLAGS`,
target). Repeat it after, from the same commit.

Public API: none; the workspace-hack crate is not published.

## Required
Decided 2026-10-27 (maintainer): this quest no longer waits on
[Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md). It measures its own
before and after baseline, so the cross-job miss report moved here from that
quest, which stays a waiting quest for the R2 numbers alone.

- [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - its cross-job hit report is this quest's baseline
Public API: none; the workspace-hack crate is not published.
5 changes: 4 additions & 1 deletion quest/m0/ci/land-bump.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
## Goal

Condition: a kixelated/quest commit on `main` ships `quest land`
([kixelated/quest's plan](https://github.com/kixelated/quest/blob/main/quest/m0/helpers/land.md)).
([kixelated/quest#229](https://github.com/kixelated/quest/pull/229)).
Check with `git ls-remote https://github.com/kixelated/quest main` and
`quest land --help` from that revision. Then bump the `quest` flake input in
`flake.nix` to that revision, and delete this quest.

The condition holds: kixelated/quest#229 merged as `195bcee560` on
2026-10-27, so only the bump remains.

## Plan

Decided 2026-10-26 (maintainer): the batched merge gate's engine lives in
Expand Down
9 changes: 5 additions & 4 deletions quest/m0/ci/land.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,10 @@ Decided 2026-10-26 (maintainer):
on the head it tested, to that PR's commit, then fast-forwards `main` to
the tip in one push. GitHub marks a PR merged once its head is on the base,
so the PRs show as merged, `main` stays linear, and the landed tree is
exactly the tested one. A conflict outside `quest/` ejects that PR. The
engine quest decides whether a fork branch that blocks maintainer edits is
refused at enqueue or lands another way.
exactly the tested one. A conflict outside `quest/` ejects that PR, and so
does a fork branch that blocks maintainer edits, whose head the engine
cannot move: it is ejected as `no_maintainer_edits`
([kixelated/quest#229](https://github.com/kixelated/quest/pull/229)).
- Batches take every queued PR, with no size constant. On failure the gate
tests the first half. A passing prefix lands at once and the rest
continues. A PR that fails alone is ejected with a comment linking the
Expand Down Expand Up @@ -103,4 +104,4 @@ Public API: none. Wire: none.

## Required

- [Bump quest to `quest land`](/quest/m0/ci/land-bump.md) - condition: kixelated/quest ships `quest land`; then bump the flake input
- [Bump quest to `quest land`](/quest/m0/ci/land-bump.md) - kixelated/quest shipped `quest land` in kixelated/quest#229; bump the flake input
20 changes: 12 additions & 8 deletions quest/m0/ci/mbx-r2-verify.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ and record the numbers in the PR that closes this:
job pays that rebuild.
- Bucket size and request counts after a month, against the 30-day expiry in
`infra/mbx/lifecycle.json`. Shorten or lengthen it from that.
- Across jobs: which compilations one job misses although another job built
the same crate at the same version, grouped by what differs (features,
profile, `RUSTFLAGS`, target). This is the baseline for
[unifying dependency features](/quest/m0/ci/hakari.md).

Use scheduled Nightly runs on `main` only (`gh run list --workflow
nightly.yml --branch main --event schedule`), never a dispatch on a branch.
Expand All @@ -47,23 +43,31 @@ Evidence so far, from the platform jobs of scheduled Nightlies
effective read-write`. That proves credentials and policy, not reuse.
- macOS refuses task manifests written past the reader's 16 MiB bound
(`a90076a4...` at 17204435 bytes, then `5cb8fb79...` and `b0310ce4...`), so
those tasks get no remote predictions. No macOS summary downloads more than
those tasks get no remote predictions. Two more were refused before the
bump, both under mbx 1.25.0: `2ed9df47...` (17218002 bytes) on 2026-10-25
([40192013303](https://github.com/moq-dev/moq/actions/runs/40192013303))
and 2026-10-26, and `0d4f4e3f...` (17290876 bytes) on 2026-10-26
([40332938121](https://github.com/moq-dev/moq/actions/runs/40332938121)). No macOS summary downloads more than
15 KiB. Reported as
[jdx/mr-boxington#792](https://github.com/jdx/mr-boxington/issues/792) and
fixed by [jdx/mr-boxington#798](https://github.com/jdx/mr-boxington/pull/798),
which keeps the read bound and caps the writer instead. The Rust cache
now pins 1.28.0, the first release carrying the fix. It does not repair
manifests already oversized on R2: if task identities carry over, the
three known digests may still refuse until about 2026-11-22, when the
30-day lifecycle deletes them. A refusal for any other digest means the
fix did not hold.
five known digests may still refuse until the 30-day lifecycle deletes
them, about 2026-11-22 for the first three and 2026-11-26 for the last
two. A refusal for any other digest means the fix did not hold.
- Windows logs no refusal but still no reuse. Its first build reports 0 hits
and 1084 of 1091 compilations "not looked up" for want of a matching
prediction on 2026-10-22, after a 1.25.0 Nightly wrote the day before, and
1091 of 1098 on 1.26.0. The manifest fix does not explain this. If it
persists after the bump, find why before timing builds.
- No release pull request has run with the R2 cache yet.

Decided 2026-10-27 (maintainer): the cross-job miss report moved to
[unifying dependency features](/quest/m0/ci/hakari.md), which measures its own
before and after baseline and no longer waits on this quest.

## Related

- [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - the same R2 and Environment bootstrap for Nix
28 changes: 13 additions & 15 deletions quest/m0/ci/nightly-levers.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,31 @@
# [M] Measure nightly compiler levers in the queue
# [S] Measure the parallel frontend in the queue

## Goal

A report says whether building the queue's tests on a pinned nightly with the
parallel frontend and the Cranelift backend saves enough wall time to justify
its risks, measured after the structural queue changes land. If it does, the
queue adopts it, and Nightly keeps a stable LLVM build of the same tests as the
backstop.
parallel frontend saves enough wall time to justify its risks, measured after
the structural queue changes land. If it does, the queue adopts it, and
Nightly keeps a stable build of the same tests as the backstop.

## Plan

Decided with the maintainer on 2026-10-23: a measured experiment, last.
Estimated 2026-10-23 at 25 to 35% of the test build's compile, about 10% of
total queue time. Risks to measure, not assume:
total queue time, with Cranelift included. Risks to measure, not assume:

- Cranelift's unwinding against moq's 30 `#[should_panic]` tests and 16
`catch_unwind` files.
- sonora's SIMD echo canceller, forced to opt-level 3 because its realtime
tests time out unoptimized; it may need a per-package LLVM backend.
- Intrinsics in `memchr`, `simd-adler32`, `crc32fast`, and `blake3`.
- The parallel frontend's remaining incremental races and differing cycle
errors.
- The gate no longer compiles with the toolchain that ships, and nightly mbx
keys split from Nightly's stable R2 writers.

kixelated/rust2's [build-speed benchmark](https://github.com/kixelated/rust2/blob/main/quest/m1/bench.md)
measures these levers on moq; use its numbers, and its explanation of nightly
incremental rebuilds costing near-full builds, before running the queue
experiment.
Decided 2026-10-27 (maintainer): moq keeps LLVM, accepting kixelated/rust2's
Cranelift verdict ([build benchmark](https://github.com/kixelated/rust2/blob/main/bench/build/README.md),
"Cranelift for moq's CI", 2026-10-25). 42 of moq's 5418 tests fail under
Cranelift (panics it cannot unwind, and an x86 SIMD intrinsic it lacks), and
the CPU saving is only 4% with a warm cache and 12% cold. This quest is
narrowed to the parallel frontend. Use the same benchmark's numbers, and its
explanation of nightly incremental rebuilds costing near-full builds, before
running the queue experiment.

Public API: none. Wire: none.

Expand Down
10 changes: 6 additions & 4 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,15 +136,18 @@ surface and zero-copy texture import, land before the cut.
- [moq-transport 18 and 22 only](/quest/m1/ietf-drop/README.md) - drafts 14-17 and 19-21 are deleted from Rust, JS, tests, interop lanes, and docs, removing the legacy control-stream adapter and its request limit
- [moq-quic's API improves on quinn's](/quest/m1/quic-api/README.md) - moq-quic and both QUIC drivers ship one mirrored API: `send::`/`recv::` namespaces, one error per direction, consuming terminal operations, plain configs, no quinn baggage
- [Drop web-transport-trait](/quest/m1/drop-web-transport-trait.md) - iroh, wasm, and ffi implement moq-net's `transport::poll` traits, and the remaining `web_transport_trait` adapters are deleted
- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface
- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native FILL_TIMEOUT bounds upstream waiting on drafts 18 and 22, alongside local content-age enforcement
- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer cached and joining IETF FETCH, with the draft-18 Largest fix
- [On-demand JavaScript FETCH](/quest/m1/js-fetch-on-demand.md) - browser publishers answer IETF FETCH for evicted or uncached groups through the JS ranges request surface
- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - Rust honors native FILL_TIMEOUT to bound upstream waiting on drafts 18 and 22, alongside local content-age enforcement
- [JS FETCH fill timeout](/quest/m1/ietf-fill-timeout-js.md) - the JS FETCH responder honors FILL_TIMEOUT as Rust does
- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` refuses a redefined rendition, and every Rust publisher mints a new rendition name for a decoder identity change and updates everything else in place
- [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension
- [Identity and recovery guide](/quest/m1/identity-guide.md) - explain pinned catalogs, lazy jobs, takeover, restart and version compatibility
- [Identity and recovery guide](/quest/m1/identity-guide.md) - explain epochs, pinned catalogs, lazy jobs, restart and version compatibility as they exist; takeover adds its own section
- [Native rung disable](/quest/m1/simulcast-rung-disable.md) - OBS stops sending the top renditions its grant cannot fund, as the Rust/FFI publishers do, and a shaped uplink through moq-ffi proves the resume
- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; a manual browser proof and a native trace replay remain
- [A/V clock](/quest/m1/av-clock.md) - the audio playhead drives Sync.reference while audio plays, through per-track sync handles
- [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust
- [Switching buffer at runtime](/quest/m1/watch-buffer-switch.md) - changing a player's `buffer` mid-playback switches the audio ring between live and buffered
- [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `<moq-watch>` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor
- [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide
- [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS
Expand All @@ -158,7 +161,6 @@ surface and zero-copy texture import, land before the cut.
- [App-limited BBR3 estimate ages](/quest/m1/bbr-app-limited.md) - an app-limited BBR3 estimate falls within seconds after the path drops
- [Aborted epoch reads keep their cause](/quest/m1/epoch-abort-cause.md) - an aborted read through an epoch route ends with its own error, not NotFound
- [Bindings name every session code](/quest/m1/ffi-malformed-auth-kind.md) - MALFORMED_AUTH_TOKEN and its siblings reach moq-ffi, moq-c, and the wrappers
- [A profiling host](/quest/m1/perf-host.md) - condition: a quiet host with `perf_event_paranoid=-1`, recorded in `bench/README.md`
- [Unbounded groups](/quest/m1/unbounded-groups/README.md) - groups and frames lose their size caps; the cache sheds parts of a group under pressure and refills the holes in place
- [Messages borrow instead of owning](/quest/m1/borrowed-messages.md) - message structs borrow from the wire on decode and from callers on encode, after lifetime-free `Path`
- [Bindings decode media as media](/quest/m1/obs-data-decode.md) - moq-ffi decodes a media rendition with its kind, so OBS stops losing every other group
Expand Down
26 changes: 19 additions & 7 deletions quest/m1/announce-takeover.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,12 +65,23 @@ fallback, including the accepted cold-start reset before this lands.
On older wires, translate this optimization into ordinary announcements
and required Restart rather than changing published framing.

Implement against lite-07-wip and reconcile the planned ROUTE/ANNOUNCE split:
reuse its routing fields and identity model rather than adding a second one.
Account for TAKEOVER in announcement IDs, compression bases, snapshot counts,
route filtering, duplicate-advertisement rules and retirement. Update the
lite draft and existing identity/compatibility docs in the implementation PR.
The dedicated guide follows separately; it does not replace inline docs.
Decided 2026-10-27 (maintainer): this no longer waits on
[Routes and announces](/quest/m1/cluster-routing/routes.md), only on
[Route epochs](/quest/m1/cluster-routing/route-epochs.md). Takeover builds on
what lite-07-wip and `main` already have: stream-local Announce IDs,
ANNOUNCE_RESTART, Base compression, `Request::prepare`/`publish`, and the
claim helpers. Its ANNOUNCE_START field layout overlaps with routes (hops
become a Route ID), so whichever of the two lands second rebases onto the
other.

Implement against lite-07-wip, and reuse the routing fields and identity
model of whichever ROUTE/ANNOUNCE shape is on `main` rather than adding a
second one. Account for TAKEOVER in announcement IDs, compression bases,
snapshot counts, route filtering, duplicate-advertisement rules and
retirement. Update the lite draft and existing identity/compatibility docs in
the implementation PR, and add a takeover section to the
[identity and recovery guide](/quest/m1/identity-guide.md) if that has
landed; the guide does not replace inline docs.

Tests with controlled time: first view before/after takeover; absent reader;
invalid/retired predecessor; same ID after Restart; two workers at one prefix;
Expand All @@ -91,11 +102,12 @@ ANNOUNCE_TAKEOVER; published versions retain their framing.

## Required

- [Routes and announces](/quest/m1/cluster-routing/routes.md) - shared routing layer and per-stream identity
- [Route epochs](/quest/m1/cluster-routing/route-epochs.md) - every route has an epoch, so takeover continues an epoch rather than a route generation

## Related

- [Claim answer epochs](https://github.com/moq-dev/moq/pull/5678) - `Request::prepare` then `publish` answers a claim at the exact path under a fresh epoch, the operation TAKEOVER continues
- [Routes and announces](/quest/m1/cluster-routing/routes.md) - the route layer whose ANNOUNCE_START layout overlaps this one's
- [IETF takeover](/quest/m1/ietf-takeover.md) - negotiated IETF counterpart
- [Retired requests](https://github.com/moq-dev/moq/pull/5294) - replacements never substitute content into held handles
- [Identity and recovery guide](/quest/m1/identity-guide.md) - explains takeover alongside restart and legacy behavior
3 changes: 2 additions & 1 deletion quest/m1/api-audit/wanted.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ Decided 2026-10-25 (maintainer), extending the `wanted` shape that
- moq-c's `MOQ_DEMAND_USED/UNUSED` become `WANTED/UNWANTED`; moq-c takes the
break with every binding.

Fix the stale `is_used()` advice in the Go wrapper README. The Impact note
Fix the stale `Used`/`Unused` advice in the Go wrapper README
(`go/wrapper/README.md`, about line 143). The Impact note
corrects the `rs/moq-json` and `rs/moq-flate` Unreleased advice for the cut.

Public API: breaking renames and deletions in every language. Wire: none.
5 changes: 2 additions & 3 deletions quest/m1/archive/hls-target.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ with `hls` records: the edge never skips, splits, or renumbers one.
## Plan

[#4280](https://github.com/moq-dev/moq/pull/4280) derives the target from the
current window's longest segment (`snapshot_as` in
current window's longest segment (`snapshot_from_window` in
`rs/moq-hls/src/export/rendition.rs`), so it rises when a long record arrives
and falls when that record is evicted. Codex flagged it
([r4113921580](https://github.com/moq-dev/moq/pull/4280#discussion_r4113921580)).
Expand All @@ -36,8 +36,7 @@ non-reference `EXTINF` rule and the split-record handling are already gone.

Replace `target_duration_follows_the_observed_segments` with tests that pin
the declared target across a window whose segment durations vary, and that
cover an overrun at the rounding boundary (listed, warned). Fix its stale
comment ("default 1s minimum"; the minimum is 2s). Hold the target per
cover an overrun at the rounding boundary (listed, warned). Hold the target per
playlist URI, not per `Rendition`: `renditions::Producer::sync` replaces the
`Rendition` on a catalog reconfigure, so test that a reconfigure mid-run
keeps the original target.
Expand Down
Loading
Loading