Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,6 @@ The historical samples locate candidates; they are not controlled benchmarks.
- [Binary release publishers run on their tag](/quest/m0/ci/release-tag-publishers.md) - brew, Go and Swift releases stop running through `workflow_run` on main
- [The bench comparison builds moq-gst's catalog bench on its base](/quest/m0/ci/bench-base-build.md) - A paired bench comparison that selects `rust/gst/moq-gst/catalog` against `origin/main` builds the base side and reports a comparison, instead of marking it unavailable with "base build failed"
- [The workspace-hack refuses trybuild edges and spares non-media builds](/quest/m0/ci/hakari-followups.md) - `hakari.py` refuses trybuild dev-dependency edges, and non-media Linux test builds stop needing libpipewire and libclang
- [moq-audio's default-feature test build is clippy-clean and gated](/quest/m0/ci/default-feature-test-lint.md) - the dead `retains`/`fail` under default-feature `--tests` clippy is fixed, and the gate catches it next time
- [A release merge opens no stale release PR](/quest/m0/ci/release-pr-after-release.md) - `just rs release` publishes before it proposes, so a merged release isn't re-proposed
- [Dependabot leaves the workspace-hack to hakari](/quest/m0/ci/dependabot-workspace-hack.md) - cargo group PRs stop hand-editing `rs/workspace-hack/Cargo.toml`
54 changes: 54 additions & 0 deletions quest/m0/ci/default-feature-test-lint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# [S] moq-audio's default-feature test build is clippy-clean and gated

## Goal

`cargo clippy --locked -p moq-audio --tests -- -D warnings` (default
features, test cfg) passes on main, and the PR gate fails a change that
breaks that configuration again. Done when that configuration is clippy-clean
and the gate covers it.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`), from the
[moq-dev/moq#7652](https://github.com/moq-dev/moq/pull/7652) iterate, which
saw the same finding on its branch and on main.

Verified on `origin/main` at `3c9facb322` (2026-11-03):

```text
$ cargo clippy --locked -p moq-audio --tests -- -D warnings
error: methods `retains` and `fail` are never used
--> rs/moq-audio/src/encode/producer.rs:517:16
error: could not compile `moq-audio` (lib test) due to 1 previous error
```

Both methods are `#[cfg(any(feature = "capture", test))]`, but their only
callers are in `rs/moq-audio/src/encode/capture.rs`, which compiles only
with `capture`. A test build without `capture` compiles them with no caller.

Why the gate misses it: the land gate lints Rust only through the one
`just rs test` build that `sh/rs/select.sh test` runs, and that build turns on
`moq-audio/capture` and `moq-audio/playback` whenever moq-audio is selected.
So moq-audio's default-feature test build is never compiled in the gate.
[moq-dev/moq#7786](https://github.com/moq-dev/moq/pull/7786), queued on 2026-11-03,
adds `--benches --examples` and each target's `required-features` to the same
build. It keeps the capture picks, so it doesn't catch this either.

- Fix the cfgs: gate `retains` and `fail` on `feature = "capture"` alone, or
give them a caller outside `capture`. Audit the other
`cfg(any(feature = "...", test))` items the gate's feature picks hide, in
moq-audio and moq-video.
- Cover it in the gate without a second full compile, since the gate already
has only one Rust build (see `rs/justfile`'s `test` recipe comments). One
way is a lint, for example in `just rs lint`, that refuses
`cfg(any(feature = "X", test))` on an item whose callers are all behind
`X`, if that can be checked cheaply. Otherwise, measure the cost of
linting moq-audio's and moq-video's default-feature `--tests` build when
they are selected, and bring the numbers to the maintainer if it is not
small.

Public API: none. Wire: none.

## Related

- [Gate each diff in under five minutes](/quest/m0/ci/five-minutes.md) - the gate's time budget any extra build must fit
45 changes: 45 additions & 0 deletions quest/m0/ci/dependabot-workspace-hack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# [S] Dependabot leaves the workspace-hack to hakari

## Goal

A Dependabot cargo group PR arrives with `rs/workspace-hack/Cargo.toml`
matching `cargo hakari generate`, so nobody regenerates it by hand. Dependabot
no longer bumps the hack's transitive pins and pulls in duplicate majors.
Dependabot still updates every other cargo manifest and `Cargo.lock`. Done
when a Dependabot cargo group PR passes `cargo hakari generate --diff` (in
`just rs lint`) without a manual commit.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`), from
[moq-dev/moq#7782](https://github.com/moq-dev/moq/pull/7782). Dependabot
edited the hakari-generated `rs/workspace-hack/Cargo.toml` by hand, bumping
transitive pins (rustc-hash 1→2, num-bigint, rand_chacha, spin, const-oid,
itertools, quick-xml, tower-http, windows-core). Those bumps only added
duplicate majors, and a maintainer commit had to regenerate the file back to
main's contents. Every cargo group PR does the same, since the cargo entry in
`.github/dependabot.yml` groups `"*"` from `/`.

Options to evaluate, in this order:

1. **Exclude the file.** Dependabot's `exclude-paths` for the cargo
ecosystem, pointing at `rs/workspace-hack/`. Risk: excluding a workspace
member may break Dependabot's workspace resolution, so it updates nothing
or fails. Test it on a fork or a scratch branch before relying on it.
2. **Ignore rules.** `ignore` entries for the hack's pins don't scale: the
pins are every unified transitive dependency, and an ignore also stops the
same crate's legitimate direct bumps.
3. **Regenerate in CI.** A workflow that runs `cargo hakari generate` on
`dependabot/cargo/*` branches and pushes the result. It needs a write
token on a pushed branch, which conflicts with
[Read-only pull-request CI](/quest/m0/ci/read-only-secrets.md); bring it
to the maintainer before building it.

Pick the first one that works, and record the test that proved it. If none
works without a write credential, report the options to the maintainer.

Public API: none. Wire: none.

## Related

- [Unify dependency features with cargo-hakari](/quest/m0/ci/hakari.md) - the workspace-hack this keeps generated
39 changes: 39 additions & 0 deletions quest/m0/ci/release-pr-after-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# [XS] A release merge opens no stale release PR

## Goal

Merging a release-plz release PR into `release` publishes that release and
opens no new release PR for the versions it just shipped. A new release PR
appears only when `release` holds unreleased changes. Done when a release
merge doesn't produce a stale release PR.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`).

[moq-dev/moq#7651](https://github.com/moq-dev/moq/pull/7651) merged at
19:33Z, and release-plz opened
[moq-dev/moq#7750](https://github.com/moq-dev/moq/pull/7750) at 19:57Z,
before 0.19.33 was published. The only change in #7750 was a second `## [0.19.33]` heading in
`rs/moq-tokio/CHANGELOG.md`, so it was closed. Release PRs routinely open
within half an hour of the previous merge
([moq-dev/moq#7009](https://github.com/moq-dev/moq/pull/7009), opened after
[moq-dev/moq#6508](https://github.com/moq-dev/moq/pull/6508) merged, was
also closed unmerged).

The likely cause: `release-rs.yml` runs `just rs release` on every push to
`release`, and that recipe (`rs/justfile`, `release:`) runs
`release-plz release-pr` before `release-plz release`. The PR step then sees
the merged versions as unpublished and untagged, and re-proposes them.

- Confirm the cause from the workflow run that #7651's merge started, or
from a dry run (`release-plz release-pr --dry-run` or similar) against the
merged state.
- If confirmed, run `release-plz release` first, then `release-pr`, in the
same recipe. Check how release-plz's own action orders the two commands, and
copy that order.
- Keep the change compatible with
[Rust releases publish through OIDC](/quest/m1/release-rs-oidc.md), which
edits the same workflow.

Public API: none. Wire: none.
17 changes: 16 additions & 1 deletion quest/m0/test-flakes-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,17 @@ yields covering lite-03 and lite-04, fixed by [moq-dev/moq#7663](https://github.
The JS close-drain fixture advanced after the setup group's first payload, so a
zero-delay subscription could expire its sender while the write budget yielded.
It now reads that group through FIN before starting the next one, fixed by
[moq-dev/moq#7658](https://github.com/moq-dev/moq/pull/7658).
[moq-dev/moq#7658](https://github.com/moq-dev/moq/pull/7658). The lite-04 case
of the same test failed once more in the batch for
[moq-dev/moq#7431](https://github.com/moq-dev/moq/pull/7431)
([run 41573678257](https://github.com/moq-dev/moq/actions/runs/41573678257),
2026-11-03 03:45Z) with `Session: remote error: 0`, the cancellation that PR
explains. That batch predates #7658's merge at 15:56Z, and no queue run since
has failed it, so it needs no child of its own (noted 2026-11-03,
`/quest-plan`).

Added 2026-11-03 (maintainer, `/quest-plan`): five more merge-queue failures
from 2026-11-03 and early 2026-11-04 UTC, one child each.

Public API: none. Wire: none.

Expand All @@ -80,3 +90,8 @@ Public API: none. Wire: none.
- [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio
- [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - the two `harness.browser.ts` buffering tests and the Firefox player's audio-tone checks pass reliably in Interop and Nightly, each fixed at its cause under the epic's rules
- [moq-tokio's close-drain test survives load](/quest/m0/test-flakes-2/tokio-close-drain.md) - `quic_client_close_drains_finished_track` stops failing land batches with "Unroutable"
- [moq-transcode's live multi-rung test survives load](/quest/m0/test-flakes-2/transcode-multi-rung.md) - `live_multi_rung` stops failing queue batches with a cancelled rung group
- [moq-relay's embedded io_uring stop test survives load](/quest/m0/test-flakes-2/relay-embed-stop.md) - `uring_custom_route_and_quic` stops seeing HTTP accepted after its owner stopped
- [The JS worklet library-build test survives load](/quest/m0/test-flakes-2/js-worklet-build.md) - the lazy-blob worklet build test stops timing out at bun's 5 seconds
- [moq-archive's cross-backend recording test survives load](/quest/m0/test-flakes-2/archive-backends-identical.md) - `recordings_are_byte_identical_on_every_backend` stops splitting S3's video objects differently
- [moq-ffi's panicked-origin supervision test survives load](/quest/m0/test-flakes-2/ffi-origin-panic.md) - a read the origin driver's panic ends always reports `MoqError::Task`
43 changes: 43 additions & 0 deletions quest/m0/test-flakes-2/archive-backends-identical.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# [S] moq-archive's cross-backend recording test survives load

## Goal

`moq-archive producer::backends::recordings_are_byte_identical_on_every_backend`
(`rs/moq-archive/src/producer/backends.rs`) passes reliably in a loaded
merge-queue `just ci`, fixed at its cause, never by raising a timeout or
adding a retry. Done when the test no longer fails the queue and a loaded
local repeat of it passes.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`). It failed the merge-queue
batch for [moq-dev/moq#7754](https://github.com/moq-dev/moq/pull/7754)
([run 41760215336](https://github.com/moq-dev/moq/actions/runs/41760215336),
2026-11-04 02:21Z) after 2.05 s, on the S3 backend:

```text
FAIL [ 2.053s] ( 819/10037) moq-archive producer::backends::recordings_are_byte_identical_on_every_backend
thread 'producer::backends::recordings_are_byte_identical_on_every_backend' (1814852) panicked at rs/moq-archive/src/producer/backends.rs:198:5:
assertion `left == right` failed: S3
```

Comparing the two maps in the assertion shows only the video objects
differed. The in-memory recording stored groups 2–3 in object `…02-…-…2000`
and groups 4–5 in `…04-…-…4000`. The S3 recording stored group 2 alone in
`…02`, then groups 3–4 in `…03`, then group 5 alone in `…05`.

A likely cause is in the test's wait rather than the producer. `record`
aborts the first run once the `…02` object exists (`stored` only `head`s the
path). With a 200 ms flush, a slow S3 put can leave that object holding only
group 2 when the abort lands. The second run then correctly resumes after the
last stored group (2), so the split differs from the backends that stored
groups 2 and 3 together.

- Confirm it, for example by delaying the S3 mock's puts.
- If confirmed, make `record` wait for the stored object to hold both groups
(decode it) before aborting, not just for it to exist. If the producer is
wrong to flush a partial object there, fix the producer instead.
- [moq-dev/moq#7752](https://github.com/moq-dev/moq/pull/7752) edits this
file. Start from whichever version is on main.

Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md).
40 changes: 40 additions & 0 deletions quest/m0/test-flakes-2/ffi-origin-panic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# [S] moq-ffi's panicked-origin supervision test survives load

## Goal

`moq-ffi origin::supervision_tests::a_panicked_origin_task_reaches_the_host_while_another_origin_keeps_working`
(`rs/moq-ffi/src/origin.rs`) passes reliably in a loaded merge-queue
`just ci`, fixed at its cause, never by raising a timeout or adding a retry.
Done when the test no longer fails the queue and a loaded local repeat of it
passes.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`). It failed the merge-queue
batch for [moq-dev/moq#7771](https://github.com/moq-dev/moq/pull/7771)
([run 41752982583](https://github.com/moq-dev/moq/actions/runs/41752982583),
2026-11-04 01:17Z) after 0.070 s:

```text
FAIL [ 0.070s] ( 2086/11649) moq-ffi origin::supervision_tests::a_panicked_origin_task_reaches_the_host_while_another_origin_keeps_working
thread 'moq-ffi-worker' (1035695) panicked at rs/moq-ffi/src/ffi.rs:558:17:
injected origin driver panic
thread 'origin::supervision_tests::a_panicked_origin_task_reaches_the_host_while_another_origin_keeps_working' (1035639) panicked at rs/moq-ffi/src/origin.rs:688:9:
assertion failed: matches!(failed.next().await, Err(MoqError::Task { .. }))
```

The injected panic fired on the worker (`panicking_origin` in
`rs/moq-ffi/src/ffi.rs`), but the pending `announced().next()` returned
something other than `MoqError::Task`. That matters beyond the test: a host
waiting on announcements can miss the origin task's failure, which is the
behavior the test exists to prove.

- Find what `next()` returned instead: most likely the origin closing (an end
or `Closed`) reached the read before the driver's panic was recorded as the
task's failure.
- If so, fix the order in the driver supervision, so a read the panic ends
always reports `Task`. Don't loosen the assertion.
- [moq-dev/moq#7752](https://github.com/moq-dev/moq/pull/7752) edits
`rs/moq-ffi/src/ffi.rs`. Start from whichever version is on main.

Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md).
37 changes: 37 additions & 0 deletions quest/m0/test-flakes-2/js-worklet-build.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# [S] The JS worklet library-build test survives load

## Goal

"library builds load worklets from a lazy blob or a hosted base, leaving no
child process" (`js/common/worklet.test.ts`) passes reliably in a loaded
merge-queue `just ci`, fixed at its cause, never by raising a timeout or
adding a retry. Done when the test no longer fails the queue and a loaded
local repeat of it passes.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`). It timed out in the
merge-queue batch for [moq-dev/moq#7536](https://github.com/moq-dev/moq/pull/7536)
([run 41700523666](https://github.com/moq-dev/moq/actions/runs/41700523666),
2026-11-03 19:13Z):

```text
#[group]js/common/worklet.test.ts:
(fail) library builds load worklets from a lazy blob or a hosted base, leaving no child process [5004.51ms]
^ this test timed out after 5000ms.
(pass) the dev server serves hosted worklets where builds emit them [803.45ms]
```

This test already failed the queue once and was counted fixed in the epic's
2026-10-26 entry. It runs a real `vite build` of a one-line worklet, so a
loaded host can push one build past bun's default 5-second test timeout.

- Measure where the time goes under load: Vite's startup, the plugin's
in-process worklet bundle, or a wait on a child process.
- If the build is slow because the plugin does needless work, make it cheaper.
If it is a whole Vite build that no smaller unit can replace, move the check
to a cheaper level, for example by testing the plugin's transform without
a full build, and keep one build where it is cheap enough to stay reliable.
- A longer per-test timeout is not a fix under the epic's rules.

Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md).
36 changes: 36 additions & 0 deletions quest/m0/test-flakes-2/relay-embed-stop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# [S] moq-relay's embedded io_uring stop test survives load

## Goal

`moq-relay::integration embed::uring_custom_route_and_quic`
(`rs/moq-relay/tests/integration/embed.rs`) passes reliably in a loaded
merge-queue `just ci`, fixed at its cause, never by raising a timeout or
adding a retry. Done when the test no longer fails the queue and a loaded
local repeat of it passes.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`). It failed the merge-queue
batch for [moq-dev/moq#7753](https://github.com/moq-dev/moq/pull/7753)
([run 41736459964](https://github.com/moq-dev/moq/actions/runs/41736459964),
2026-11-03 23:06Z) after 0.371 s, while its `shared_tokio` and `worker_tokio`
siblings passed:

```text
FAIL [ 0.371s] (7970/9724) moq-relay::integration embed::uring_custom_route_and_quic
thread 'embed::uring_custom_route_and_quic' (3744147) panicked at rs/moq-relay/tests/integration/embed.rs:51:5:
HTTP still accepted after the owner stopped
```

`assert_owner_stopped` connects to the HTTP port after the owner stops and
expects a refusal.

- Find out whether the io_uring listener was still open (the stop returned
before its accept socket closed) or another test bound the freed ephemeral
port in between. The relay's auth-trust flake was a reused port
([moq-dev/moq#7550](https://github.com/moq-dev/moq/pull/7550)).
- If the listener outlives the stop, fix the stop so it returns only after
the socket closes. If the port was reused, prove the close another way, for
example with the socket-inode check the QUIC half already uses on Linux.

Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md).
42 changes: 42 additions & 0 deletions quest/m0/test-flakes-2/transcode-multi-rung.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# [S] moq-transcode's live multi-rung test survives load

## Goal

`moq-transcode tests::live_multi_rung` (`rs/moq-transcode/src/lib.rs`) passes
reliably in a loaded merge-queue `just ci`, fixed at its cause, never by
raising a timeout or adding a retry. Done when the test no longer fails the
queue and a loaded local repeat of it passes.

## Plan

Planned 2026-11-03 (maintainer, `/quest-plan`). It failed two merge-queue
batches on 2026-11-03, both with the same panic:

- [Run 41599906715](https://github.com/moq-dev/moq/actions/runs/41599906715),
the batch for [moq-dev/moq#7544](https://github.com/moq-dev/moq/pull/7544),
at 08:03Z after 0.909 s.
- [Run 41666527065](https://github.com/moq-dev/moq/actions/runs/41666527065),
the batch for [moq-dev/moq#7692](https://github.com/moq-dev/moq/pull/7692),
at 15:44Z after 1.566 s.

```text
FAIL [ 1.566s] (9500/9597) moq-transcode tests::live_multi_rung
thread 'tests::live_multi_rung' (26884) panicked at rs/moq-transcode/src/lib.rs:550:52:
called `Result::unwrap()` on an `Err` value: Cancel
```

Line 550 is the `group.read_frame().await.unwrap()` that drains each rung's
first group, so a rung's output group was cancelled before it held all five
source frames.

- The test runs in real time on purpose: its comment says a paused clock let
the source outrun the feed's bounded broadcast while the rungs encoded on
their own threads. Under load, real time can do the same thing. Find what
cancels the rung's group: the feed's lag, a superseding group, or the rung
restarting its encoder.
- Follow the epic's rules: drive the source from an event the rungs report,
or from a `kio::clock::Manual` clock, rather than a wall-clock pace. If the
code under test drops a group it should finish, fix the code.
- Reproduce it under load first, for example beside a full `cargo nextest`.

Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md).
Loading
Loading