Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions nix/Cargo.nix

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 10 additions & 8 deletions quest/m0/test-flakes-3/srt-server-load.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# [S] moq-srt's server tests hold under load
# [XS] Tests queued for an mbx permit time out under load

## Goal

`moq-srt`'s `server::tests` pass however loaded the machine is, and its test target compiles on its own.
No test hits nextest's timeout while it waits for an mbx scheduler permit, so `moq-srt`'s `server::tests` (and every other test) pass however loaded the host is.

## Plan

Four of them (`a_new_clock_flushes_the_partial_chunk_first`, `a_new_clock_re_anchors_the_pacing`, `a_publisher_marker_keeps_an_srt_receiver_playing`, `a_reorder_is_not_a_rewind`) hit nextest's 60s timeout under a full `just check` on 2026-11-18 at load average 40, on a branch that touched no moq-srt code ([moq-dev/moq#9123](https://github.com/moq-dev/moq/pull/9123)).
Run alone, all fifteen pass in 3.4s.
Reproduce under load first, then fix what waits on wall time.
Four `moq-srt` `server::tests` hit nextest's 60s timeout in a loaded `just check` ([moq-dev/moq#9123](https://github.com/moq-dev/moq/pull/9123)); three are purely synchronous, so the wait was never in the test code.
The cause is mbx, the Cargo wrapper: the host's global mbx config sets `scheduler.tests = true`, so each test waits for a host-wide permit inside `mbx-test-runner`, and nextest counts that wait toward the test's `slow-timeout`.
Any test in the workspace can hit it; moq-srt's were just in flight.

Run alone, the test target does not compile: `src/server.rs` calls `hang::Catalog::subscribe`, which needs hang's `async` feature, and only workspace feature unification enables it.
Enable it in moq-srt's dev-dependencies, as moq-json does for its own wrappers.
Decided 2026-11-19 (maintainer, [moq-dev/moq#9163](https://github.com/moq-dev/moq/pull/9163#issuecomment-7020242924)): fix it in mbx or the dotfiles mbx config, not with a checked-in `.mbx.toml` opt-out or a longer `terminate-after` here, so host-wide test scheduling stays on.
Once that lands, reproduce below to confirm, then delete this quest.

Decided 2026-11-19 (coordinator, maintainer away, merge granted): keep this follow-up.
## Waiting

- mbx or the dotfiles mbx config: stop a test's queue wait for a scheduler permit from counting toward nextest's timeout, for example by taking the permit before nextest starts the test's clock. Check with the one-core reproduction from [moq-dev/moq#9163](https://github.com/moq-dev/moq/pull/9163): run `cargo nextest run -p moq-srt --lib --test-threads 4 --stress-count 20` with `KIO_DENY_HOST_TIMERS=1` inside `systemd-run --user --scope -p CPUQuota=100%`, with no `MBX_SCHEDULER_TESTS` override. Before the fix about 20 tests time out at 60s, each printing `mbx: waiting for machine-wide scheduler capacity`; after it, all pass.
2 changes: 2 additions & 0 deletions rs/moq-srt/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ tokio = { workspace = true, features = ["full"] }
tracing = { workspace = true }

[dev-dependencies]
# `Catalog::subscribe` in the tests.
hang = { path = "../hang", features = ["async"] }
kio = { workspace = true, features = ["runtime"] }
srt-protocol = "0.4"
tracing-subscriber = { workspace = true }
Expand Down