Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
f6a3c02
quest: plan the 2026-11-03 follow-ups
kixelated Nov 3, 2026
a1888e4
quest: add SUBSCRIBE_STARTED, the widening bench, and revoked announc…
kixelated Nov 3, 2026
b5c5bb8
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
2bcd5fe
quest: drop the announce reset's Required link to the finished expiry…
kixelated Nov 3, 2026
3896e22
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
4e8cc3e
quest: plan a host lock so benchmark sessions can't overlap
kixelated Nov 3, 2026
7ef3984
quest: drop the bench lock's link to the log lock profile
kixelated Nov 3, 2026
0740289
quest: plan JSON snapshot resync and prototype-safe rendition names
kixelated Nov 3, 2026
af9df73
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
e81565d
quest: plan native time-stretch in moq-audio, copying NetEq
kixelated Nov 3, 2026
ecbb202
quest: name the browser stretch's current owner in Native stretch
kixelated Nov 3, 2026
1c2fd13
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
961da68
quest: fix two review threads and plan a self-contained Moxygen smoke…
kixelated Nov 3, 2026
27c4553
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
c8d8471
quest: bound every received box and count, and narrow the MSF half
kixelated Nov 3, 2026
f42dba2
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 3, 2026
d5070b5
quest: state the CMAF Goal over every box and count
kixelated Nov 3, 2026
46ea32c
quest: native stretch vectors grade the accelerate/expand/hold choice
kixelated Nov 3, 2026
daffb4a
Merge origin/main into quest-plan/followups-2026-11-03
kixelated Nov 3, 2026
755b044
Merge base and reconcile quest links after completed plans
Nov 3, 2026
4e0a6e5
Merge origin/main into quest-plan/followups-2026-11-03
kixelated Nov 4, 2026
e3e320e
Merge remote-tracking branch 'origin/quest-plan/followups-2026-11-03'…
kixelated Nov 4, 2026
f75370a
quest land: merge tested base
Nov 4, 2026
352e671
Merge base and retain completed JS ranges quest deletion
Nov 4, 2026
92c61a2
quest land: merge tested base
Nov 4, 2026
3ec3dbd
Merge base and follow rind quest directory rename
Nov 4, 2026
6ddf97a
Merge origin/main into quest-plan/followups-2026-11-03
kixelated Nov 4, 2026
2c56451
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 4, 2026
961dd6c
quest: fold the e2ee stub and LibreSSL smoke duplicates into #7747's …
kixelated Nov 4, 2026
c77c416
Merge remote-tracking branch 'origin/main' into quest-plan/followups-…
kixelated Nov 4, 2026
608f138
Merge origin/main into quest-plan/followups-2026-11-03
kixelated Nov 4, 2026
d574a1a
quest land: merge tested base
Nov 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,15 @@ edge, which embeds moq-relay; it moved there on 2026-09-28.

- [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break
- [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the scheduled Interop and Video workflows pass, so red means a new break
- [moq-video shares moq-mux's bounded H.265 SPS walk](/quest/m0/video-sps-walk.md) - a hostile publisher's H.265 SPS can't panic a subscriber's decoder
- [Subscriber-side CMAF parsing is bounded](/quest/m0/cmaf-receive-bounds.md) - a received `moof` or init over `MAX_METADATA_BYTES` is refused before mp4-atom parses it, so one frame can't cost a subscriber hundreds of megabytes
- [Publish the release line's drain() wrappers](/quest/m0/ffi-shutdown-backport.md) - Python `moq-rs` 0.5.2 and Dart `moq` 0.1.1 reach PyPI and pub.dev from `release`
- [Backport audio groupDuration](/quest/m0/backport-audio-group-duration.md) - `@moq/publish` 0.5.4 on `release` can pack several Opus frames per group, making Cloudflare's relay drop audio less often
- [Backport the SETUP-path and iroh-key fixes](/quest/m0/backport-security-fixes.md) - `release` refuses a conflicting SETUP Path and keeps persisted iroh keys private, in one patch release
- [Backport the joined-head FETCH race fix](/quest/m0/backport-fill-head-race.md) - a `release` relay stops FETCHing a group head its own SUBSCRIBE is bringing, which left live readers hanging
- [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in a batched merge gate as one `just ci` pass per batch, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last
- [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one fails a whole batch
- [Benchmark sessions can't overlap on a host](/quest/m0/bench-host-lock.md) - a host-level lock keeps a second benchmark or profile session from measuring under the first's load
- [A waiter's record limit changes no behavior](/quest/m0/kio-waiter-records.md) - two moq-net tests stop failing when kio's waiters remember 16 lists, and the limit becomes a tuning knob worth 3.9% per delivered group
- [Backport the relay and auth hardening to release](/quest/m0/backport-release-hardening.md) - `release`'s relay refuses dot-segment paths, serves its HTTP routes with main's response hardening, and withdraws advertisements a peer starves of send credit; its auth client refuses redirects
- [quest init keeps moq's nix-wrapped worktree hooks](/quest/m0/quest-worktree-hooks.md) - quest pin bumps keep the hooks that run moq's pinned `quest`
Expand Down
54 changes: 54 additions & 0 deletions quest/m0/bench-host-lock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# [S] Benchmark sessions can't overlap on a host

## Goal

Two benchmark or profile sessions never run at once on one host. A
host-level lock, taken by `quest gate` or by moq's bench recipes (for example
`just bench-profile`, the relay CPU bisect driver, and `bench/run.sh`), makes
a second session wait, or refuse with a message naming the holder, instead of
measuring under the first one's load.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03
/quest-afk and /quest-complete runs), milestone:

- m2
- ✅ m0
- drop

Evidence: on 2026-11-03 `nixos2` had three rind agents and
[moq-dev/moq#7605](https://github.com/moq-dev/moq/pull/7605)'s relay-CPU
bisect running while [moq-dev/moq#7742](https://github.com/moq-dev/moq/pull/7742)'s
lock profile ran, which voided two of its three runs. Instruction counts are
load-sensitive too, so overlapping sessions skew even the metrics that avoid
wall-clock time.

Decide in this quest's PR whether the lock lives in moq or needs a
moq-dev/quest change, and record the choice and why here:

- **In moq.** The bench recipes take one host-wide lock, for example
`flock` on a fixed path outside the checkout, so every worktree and agent
shares it. That serializes benchmark sessions with each other, but builds
and tests that other agents run through `quest gate` still start, since
the gate only waits on load average and free memory.
- **In quest.** `quest gate` gains an exclusive admission that a bench
takes, holding back other gated commands while it runs. Today the gate
has no exclusive mode and no configuration (`quest gate --help`). That
change goes into moq-dev/quest first, then moq adopts it with a pin bump,
and this quest waits on the release.

Either way:

- Cover every entry point that measures: `bench/run.sh` (and the `just bench*`
recipes that call it), `just bench-profile`, and the relay CPU bisect
driver once #7605 lands it.
- The holder is visible (command, PID, start time), and a stale lock from a
dead process is released.
- Nightly's hosted runners run one job per machine, so they need no change.

Public API: none. Wire: none.

## Related

- [Find and fix the relay's per-frame CPU growth since #5926](/quest/m1/release-perf/relay-cpu-bisect.md) - the bisect that overlapped
1 change: 1 addition & 0 deletions quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ The historical samples locate candidates; they are not controlled benchmarks.
- [Binary release publishers run on their tag](/quest/m0/ci/release-tag-publishers.md) - brew, Go and Swift releases stop running through `workflow_run` on main
- [The bench comparison builds moq-gst's catalog bench on its base](/quest/m0/ci/bench-base-build.md) - A paired bench comparison that selects `rust/gst/moq-gst/catalog` against `origin/main` builds the base side and reports a comparison, instead of marking it unavailable with "base build failed"
- [The workspace-hack refuses trybuild edges and spares non-media builds](/quest/m0/ci/hakari-followups.md) - `hakari.py` refuses trybuild dev-dependency edges, and non-media Linux test builds stop needing libpipewire and libclang
- [Local test runs honor NEXTEST_NO_FAIL_FAST](/quest/m0/ci/no-fail-fast.md) - `just check` and `sh/rs/nextest.sh` pass `--no-fail-fast` through, so one host failure doesn't hide the rest
- [A release merge opens no stale release PR](/quest/m0/ci/release-pr-after-release.md) - `just rs release` publishes before it proposes, so a merged release isn't re-proposed
- [Dependabot leaves the workspace-hack to hakari](/quest/m0/ci/dependabot-workspace-hack.md) - cargo group PRs stop hand-editing `rs/workspace-hack/Cargo.toml`
- [Bun runs only the test files we name](/quest/m0/ci/bun-test-paths.md) - scripted `bun test` calls pass exact paths, so generated copies never run
24 changes: 24 additions & 0 deletions quest/m0/ci/no-fail-fast.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# [XS] Local test runs honor NEXTEST_NO_FAIL_FAST

## Goal

`just check` and `sh/rs/nextest.sh` pass `--no-fail-fast` to nextest when
`NEXTEST_NO_FAIL_FAST` is set, so one host-specific failure doesn't hide the
rest of the suite's results.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0, as a child of [CI cost and latency](/quest/m0/ci/README.md)
- m1
- m2

- Trace how `just check` reaches nextest (`sh/rs/nextest.sh` execs
`sh/local.sh`) and pass the flag through there, so every entry point gets
it.
- Check first whether the pinned cargo-nextest already reads the variable
itself; if so, make sure nothing in between overrides it.
- Default behavior stays fail-fast.

Public API: none. Wire: none.
54 changes: 54 additions & 0 deletions quest/m0/cmaf-receive-bounds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# [M] Subscriber-side CMAF parsing is bounded

## Goal

A hostile publisher can't make a subscriber allocate hundreds of megabytes
from one frame. Every place a subscriber parses received CMAF with mp4-atom
refuses any top-level box whose body exceeds `MAX_METADATA_BYTES`, and any
count over the importer's cap, before parsing it. A peak-memory test like
[moq-dev/moq#7687](https://github.com/moq-dev/moq/pull/7687)'s shows the
bounds holding on each path.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0 (security: one hostile frame costs about 320 MB per subscriber)
- m1
- m2

#7687 (merged) bounds the importer (publisher side). Received frames, up to 32 MiB
each, still reach mp4-atom unchecked:

- `fmp4::Wire::decode` and its `decode` helper
(`rs/moq-mux/src/container/fmp4/mod.rs`, around line 556).
- `Wire::from_init` (same file, around line 440).
- MSF's init parse, `cmaf_moov`
(`rs/moq-mux/src/catalog/msf/consumer.rs`, around line 416).
- `rs/moq-mux/src/container/mp4.rs` (the `Moof::decode` around line 162 and
the init's `Moov::decode`).

- **Every box, not only `moof` and `moov`.** These paths decode each
top-level box through `mp4_atom::Any::decode_maybe` before discarding the
ones they don't use, and mp4-atom expands any box it knows (a stray
top-level `trun`, say) wherever it appears. So read each top-level box's
header and refuse or skip any body over #7687's `MAX_METADATA_BYTES` before
`Any` sees it, keeping the limit one constant shared with the importer.
- **Counts, not only bytes.** A small box can still declare a large count.
Bound received sample counts as the importer does (`MAX_FRAGMENT_SAMPLES`),
including runs without per-sample fields. `sgpd`'s zero-byte entries are
fixed upstream and picked up by
[moq-mux on a fixed mp4-atom](/quest/m1/fuzz-findings/mp4-atom-bump.md),
which waits on a release. If that hasn't landed, refuse an `sgpd` entry
count the input can't hold here too.
- Measure each path's peak with #7687's counting allocator
(`allocation::Usage::peak`), as its `fragment_plans_within_the_held_bound`
does, with both a large-box and a small-box/large-count input per path, each
failing without the fix.

Public API: none expected (a new error variant at most). Wire: none.

## Related

- [One policy for oversized container metadata](/quest/m2/import-metadata-policy.md) - the importer-side rule this mirrors
- [moq-mux on a fixed mp4-atom](/quest/m1/fuzz-findings/mp4-atom-bump.md) - the upstream `sgpd` count fix
4 changes: 4 additions & 0 deletions quest/m0/nightly/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ Public API: none. Wire: none.

## Required

- [The receive-windows benchmark runs to the end](/quest/m0/nightly/receive-windows-evicted.md) - `benchmark_receive_windows` stops failing `frame: Evicted`, so the Rust benchmark-tests job publishes its report
- [The relay's startup fits its stack under all features](/quest/m0/nightly/relay-stack.md) - `debug_relay_load_and_run_fit_windows_stack` passes under `--all-features`, fixed at the feature that grew the future
- [The binding benchmark's 64 KiB case measures](/quest/m0/nightly/bindings-bench-evicted.md) - the 64 KiB case completes instead of failing `evicted`
- [The range-widening bench expects a narrowed read to end](/quest/m0/nightly/track-range-widening.md) - `track_range_widening/1r/64` stops asserting a pending read that #7545 made end
- [moq-net's ingest fuzz target finds no stall](/quest/m0/nightly/ingest-fuzz.md) - the 13-byte stream that stalls ingest is fixed at its cause and kept as a regression test
- [Nightly's shipping-size check passes](/quest/m0/nightly/shipping-size.md) - js/net, js/watch, and moq-relay growth over 5% is traced to its commits, then shrunk or accepted
- [Nightly's Miri job stays inside its time budget](/quest/m0/nightly/miri.md) - Miri gets `rust-src`, then finishes inside its 60-minute job and 15-minute-per-test budgets
27 changes: 27 additions & 0 deletions quest/m0/nightly/bindings-bench-evicted.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# [S] The binding benchmark's 64 KiB case measures

## Goal

Nightly's `Binding benchmark runners` job passes: the 64 KiB case of the
moq-ffi throughput benchmark (`rs/moq-ffi/src/bench.rs`, run by each
binding's runner such as `py/moq-rs/benches/ffi.py`) completes and reports
a number instead of failing `evicted` on `main`.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0, as a child of [Nightly passes on main](/quest/m0/nightly/README.md)
- m1
- m2

- Confirm the cause first. The likely one: each case's origin cache is
`CACHE_BYTES` (64 MiB), and a 64 KiB case writes 64-frame groups
(`frames()` caps at 64), so a round can outgrow the cache before a slow
binding reads it.
- Then size the case or its cache so it measures throughput rather than
eviction, and keep the other sizes' numbers comparable with past nights.
- Run every binding's runner locally (`just gh nightly bindings`) before
pushing.

Public API: none. Wire: none.
6 changes: 6 additions & 0 deletions quest/m0/nightly/miri.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ quest until a scheduled run containing that fix records the job duration,
strict and epoch pass durations, slowest test, and headroom against both
budgets. Do not dispatch Nightly by hand to obtain it.

In the same log, confirm moq-net's `model::frame::` tests ran (the strict
pass selects them in `sh/rs/miri.sh`), including the lock-free single-delivery
path [moq-dev/moq#7508](https://github.com/moq-dev/moq/pull/7508) added; extend
the selection if any are missed. (Folded in from moq-dev/moq#7748's
`miri-frame.md`, 2026-11-04, whose `rust-src` half #7812 landed.)

Act on those timings before optimizing. Earlier local measurements on 24
cores took 11.5 min wall: strict pass 118 tests in 109 s, epoch pass 94 in
307 s. These are desktop measurements, not proof of the 4-core ARM budget.
Expand Down
34 changes: 34 additions & 0 deletions quest/m0/nightly/receive-windows-evicted.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# [S] The receive-windows benchmark runs to the end

## Goal

`moq-tokio`'s `benchmark_receive_windows`
(`rs/moq-tokio/tests/integration/receive_credit.rs:1056`) passes on `main`
instead of failing with `frame: Evicted`, so Nightly's `Rust benchmark
tests` job (`gh nightly rust-test`) passes and that night's report is
published. Fixed at the cause, never by loosening what the benchmark
measures.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0, as a child of [Nightly passes on main](/quest/m0/nightly/README.md)
- m1
- m2

It failed in 3 of 3 local runs on `main`, and the job's failure skips the
rest of the night's report. Likely since
[moq-dev/moq#7597](https://github.com/moq-dev/moq/pull/7597) changed its
subscription to a track `Delay { min, max }`: confirm by running the
benchmark at #7597's parent and at #7597.

- Repro: `cargo nextest run -p moq-tokio --release -E
'test(benchmark_receive_windows)' --run-ignored only`.
- Decide whether the benchmark's subscription now asks for less than it
reads (a zero `delay.max` evicting groups the reader still wants), or
whether eviction changed under it, and fix that side.
- [Receive pool](/quest/m2/receive-pool.md) sweeps this harness later; keep
its cases runnable.

Public API: none. Wire: none.
26 changes: 26 additions & 0 deletions quest/m0/nightly/relay-stack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# [S] The relay's startup fits its stack under all features

## Goal

`moq-relay`'s `stack::debug_relay_load_and_run_fit_windows_stack`
(`rs/moq-relay/tests/integration/stack.rs`) passes under `--all-features`
on `main` instead of overflowing its 1 MiB thread. Fixed at the cause, by
shrinking the future that grew, never by raising the stack size the test
models.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0, as a child of [Nightly passes on main](/quest/m0/nightly/README.md)
- m1
- m2

- Find which feature grows `Relay::load`/`Relay::run`'s future: run the
test with each feature added to the defaults, then measure the futures'
sizes (`-Zprint-type-sizes`, or `size_of_val` on the futures in a test).
- Box or split the large state that feature adds, the way the default build
already fits.
- Keep the test's 1 MiB stack, which stands for a Windows main thread.

Public API: none expected. Wire: none.
33 changes: 33 additions & 0 deletions quest/m0/nightly/track-range-widening.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# [S] The range-widening bench expects a narrowed read to end

## Goal

`rs/moq-net/benches/track.rs`'s `track_range_widening/1r/64` passes on
`main`, under `cargo test -p moq-net --all-targets` and Nightly's `cargo
bench -p moq-net --bench '*'`, with an assertion that matches bounded reads
as [moq-dev/moq#7545](https://github.com/moq-dev/moq/pull/7545) defined them.

## Plan

Decided 2026-11-03 (maintainer, follow-up planning after the 2026-11-03 /quest-afk and /quest-complete runs), milestone:

- ✅ m0, as a child of [Nightly passes on main](/quest/m0/nightly/README.md)
- m1
- m2

Found by [moq-dev/moq#7536](https://github.com/moq-dev/moq/pull/7536), which
bisected it to #7545 ("end bounded reads",
[64cba13](https://github.com/moq-dev/moq/commit/64cba1337136c67ce77aaf6ae337f355c97a895e)).
After widening, the bench narrows its reader back to the already-delivered
`0..1` and asserts the next `poll_recv_group` is pending
(`bench_range_widening`, around line 1730). Since #7545 the bounded read
ends instead, returning `Ready(Ok(None))`. CI's `cargo test` skips bench
targets, so it went unnoticed.

- Either assert the end, or narrow to a range that isn't finished yet so the
read stays pending. Pick the one that keeps the bench measuring the widening
it names, and say why in the PR.
- Confirm with `cargo test -p moq-net --all-targets` and a short `cargo
bench -p moq-net --bench track -- track_range_widening`.

Public API: none. Wire: none.
1 change: 0 additions & 1 deletion quest/m0/test-flakes-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ Public API: none. Wire: none.

- [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio
- [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - the two `harness.browser.ts` buffering tests and the Firefox player's audio-tone checks pass reliably in Interop and Nightly, each fixed at its cause under the epic's rules
- [moq-tokio's close-drain test survives load](/quest/m0/test-flakes-2/tokio-close-drain.md) - `quic_client_close_drains_finished_track` stops failing land batches with "Unroutable"
- [moq-transcode's live multi-rung test survives load](/quest/m0/test-flakes-2/transcode-multi-rung.md) - `live_multi_rung` stops failing queue batches with a cancelled rung group
- [moq-relay's embedded io_uring stop test survives load](/quest/m0/test-flakes-2/relay-embed-stop.md) - `uring_custom_route_and_quic` stops seeing HTTP accepted after its owner stopped
- [The JS worklet library-build test survives load](/quest/m0/test-flakes-2/js-worklet-build.md) - the lazy-blob worklet build test stops timing out at bun's 5 seconds
Expand Down
25 changes: 0 additions & 25 deletions quest/m0/test-flakes-2/tokio-close-drain.md

This file was deleted.

Loading
Loading