From 23887065d9c855cf90cf110d17a5072fc7794746 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 19:55:25 -0800 Subject: [PATCH 1/2] quest: plan the receive-credit flake and Python test timeouts m0: javascript_partial_frame_gives_way_qmux completed an oversized frame on moq-transport-18 in 1 of 4 local runs during #7729's iterate; no CI run has failed it. m2: bound every py/ test with one pytest-timeout setting, so a hung read fails instead of hanging the suite. Co-Authored-By: Claude Opus 5.5 --- quest/m0/test-flakes-2/README.md | 1 + quest/m0/test-flakes-2/receive-credit-qmux.md | 49 +++++++++++++++++++ quest/m2/README.md | 1 + quest/m2/py-test-timeouts.md | 44 +++++++++++++++++ 4 files changed, 95 insertions(+) create mode 100644 quest/m0/test-flakes-2/receive-credit-qmux.md create mode 100644 quest/m2/py-test-timeouts.md diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index a3062473da5..875eb5e3ece 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -99,3 +99,4 @@ Public API: none. Wire: none. - [@moq/net's lite subscribe tests wait for the whole request](/quest/m0/test-flakes-2/js-subscribe-fixture.md) - the three lite subscribe-with-track cases pass under forced yields - [Concurrent PulseAudio harness runs don't share a D-Bus name](/quest/m0/test-flakes-2/pulse-dbus.md) - parallel media runs no longer collide on `org.PulseAudio1` - [@moq/net's uni-stream flood test survives reruns](/quest/m0/test-flakes-2/js-stream-flood.md) - "a flood of uni streams yields" passes under `--rerun-each` and load +- [The JS receive-credit qmux test never completes an oversized frame](/quest/m0/test-flakes-2/receive-credit-qmux.md) - `javascript_partial_frame_gives_way_qmux` stops completing a frame larger than the window on moq-transport-18 diff --git a/quest/m0/test-flakes-2/receive-credit-qmux.md b/quest/m0/test-flakes-2/receive-credit-qmux.md new file mode 100644 index 00000000000..9a453f2a63b --- /dev/null +++ b/quest/m0/test-flakes-2/receive-credit-qmux.md @@ -0,0 +1,49 @@ +# [S] The JS receive-credit qmux test never completes an oversized frame + +## Goal + +`moq-tokio::integration receive_credit::javascript_partial_frame_gives_way_qmux` +(`rs/moq-tokio/tests/integration/receive_credit.rs`) passes reliably on +moq-transport-18, fixed at its cause, never by raising a timeout or adding a +retry. Done when a loaded local repeat of `just test receive-credit` passes +every run. + +## Plan + +Planned 2026-11-04 (maintainer, `/quest-plan`), from the follow-ups of +[moq-dev/moq#7729](https://github.com/moq-dev/moq/pull/7729). During that +PR's iterate, `just test receive-credit` failed 1 of 4 local runs, on +moq-transport-18 over qmux, with: + +```text +moq-transport-18: a frame larger than the window completed +``` + +#7729 found the failure unrelated to its change: the Bun sender advertises +the same absent max age as it does on main. + +**The evidence is local only.** The test is `#[ignore]`d, so merge-queue +runs never run it. Only the Interop workflow's "Receive credit recovery" step +does (`just test receive-credit`, in `.github/workflows/interop.yml`). None +of the 40 most recent Interop runs failed it: the 12 failed scheduled runs +from 2026-10-23 to 2026-11-03 failed other steps, and each log shows +`javascript_partial_frame_gives_way_qmux` passing (for example +[run 41668668824](https://github.com/moq-dev/moq/actions/runs/41668668824)). +No release-gate run failed either. + +The assertion is in `javascript_exhausts`. A low-priority frame twice the +window must give way to a high-priority frame. So the partial low frame must +end in an error, never complete. Completing it means the receiver granted +the low stream credit past the window, or the sender finished it before the +high frame preempted it. + +- Reproduce it by repeating the qmux case on moq-transport-18 under load, + for example beside a full `cargo nextest`. +- Find which side let the frame complete: qmux's receive-credit lease on the + WebSocket session, the IETF draft-18 stream priority, or the Bun sender's + command timing (`test/receive-credit/sender.ts`). Fix that side. +- If the test starts the high frame before the low frame reaches + three-quarters of the window on the wire, make it wait on the event it + needs rather than adding a delay. + +Follow the rules in [More tests under load](/quest/m0/test-flakes-2/README.md). diff --git a/quest/m2/README.md b/quest/m2/README.md index 51c007af2ba..ae0bdf9eeb6 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -285,3 +285,4 @@ lite-07; bitrate enforcement is not part of this release. - [Kotlin dependencies live in a Gradle version catalog](/quest/m2/kt-version-catalog.md) - one catalog entry per library, so a Dependabot bump can't drag another along - [The dev shell builds Kotlin with Gradle 9](/quest/m2/kt-gradle-9.md) - `gradle_9` in the dev shell, so the Android target builds there - [The publish support panel lists only codecs publishing accepts](/quest/m2/publish-support-probe.md) - the settings panel runs the publisher's Annex B encoder probe +- [A hung Python test fails instead of hanging the suite](/quest/m2/py-test-timeouts.md) - one pytest-timeout bound covers every `py/` test diff --git a/quest/m2/py-test-timeouts.md b/quest/m2/py-test-timeouts.md new file mode 100644 index 00000000000..a2af244ad00 --- /dev/null +++ b/quest/m2/py-test-timeouts.md @@ -0,0 +1,44 @@ +# [XS] A hung Python test fails instead of hanging the suite + +## Goal + +Every test that `just py test` runs (`py/moq-rs/tests/` and +`py/moq-ffi/tests/`) has a time bound, so a regression that leaves a read or +loop waiting forever fails that test with a traceback rather than hanging +`just py test` and the gate. Done when a test that awaits forever fails +within the bound, and the suite passes as before. + +## Plan + +Planned 2026-11-04 (maintainer, `/quest-plan`), from the follow-ups of +[moq-dev/moq#7729](https://github.com/moq-dev/moq/pull/7729): Python tests +that loop over groups without a timeout hang instead of failing. + +On `origin/main` (`dc0bd09b11`), 33 test functions and helpers loop or +await a read with no `wait_for` or `timeout` anywhere in their bodies. A few +of the helpers may be bounded by their callers. Most are in +`py/moq-rs/tests/test_local.py`, for example `test_local_publish_consume_audio`, +`test_raw_publish_consume`, `test_raw_group_sequence`, and the `routes` and +`until_wanted` helpers. `py/moq-rs/tests/test_server.py` has more, for +example `test_server_client_roundtrip` and `test_broadcast_route_over_wire`, +plus its own `routes`. Other tests already wrap their reads in +`asyncio.wait_for` or `asyncio.timeout` one at a time (89 calls), so the suite has no single +rule. + +Recommended, as the simplest approach that fits: add `pytest-timeout` to the +root `pyproject.toml`'s `dev` dependency group and set one `timeout` in the +existing `[tool.pytest.ini_options]`. Size it well above the slowest test's +loaded time, measured with `pytest --durations`. One setting covers every +current and future test, including sync ones, while per-loop deadlines would +mean editing about 30 sites and remembering the rule in every new test. + +- Keep the existing `wait_for` calls. They give a precise failure, and the + global bound only catches what they miss. +- Check which `timeout_method` fires on a hang inside a native moq-ffi call + that holds the GIL. The default `signal` method can't interrupt that, and + `thread` can, at the cost of ending the run. Pick the method that fires + for both an awaited hang and a native one, and note why in the config. +- Update `uv.lock`. Dependabot's `uv` entry for `/` then covers the new + dependency. + +Public API: none. Wire: none. From 1a2eb55acebfe80f6dc35d3364d02028825c23b4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 20:27:46 -0800 Subject: [PATCH 2/2] quest(m2): pick the pytest-timeout method from how moq-ffi blocks pytest-timeout's thread method also needs the GIL, but moq-ffi's ctypes calls release it: signal stops awaited hangs, thread also stops a blocking native call, and neither stops code that holds the GIL. Co-Authored-By: Claude Opus 5.5 --- quest/m2/py-test-timeouts.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/quest/m2/py-test-timeouts.md b/quest/m2/py-test-timeouts.md index a2af244ad00..a64fe247fd2 100644 --- a/quest/m2/py-test-timeouts.md +++ b/quest/m2/py-test-timeouts.md @@ -34,10 +34,18 @@ mean editing about 30 sites and remembering the rule in every new test. - Keep the existing `wait_for` calls. They give a precise failure, and the global bound only catches what they miss. -- Check which `timeout_method` fires on a hang inside a native moq-ffi call - that holds the GIL. The default `signal` method can't interrupt that, and - `thread` can, at the cost of ending the run. Pick the method that fires - for both an awaited hang and a native one, and note why in the config. +- Pick the `timeout_method` from how the suite can hang, and note why in the + config. The default `signal` method stops an awaited hang, because the + event loop keeps running Python and CPython runs the handler there. It + can't stop a synchronous moq-ffi call that blocks: the generated bindings + load the library with `ctypes.cdll`, which releases the GIL for the call, + and CPython runs a signal handler only once the call returns. `thread` stops + both, since its timer thread can take the released GIL and call + `os._exit`, but it ends the whole run. Neither method can stop code that + holds the GIL forever, and pytest-timeout doesn't try to cover that. A + moq-ffi call doesn't hold the GIL, so that case is out of scope. Use + `thread` only if a test makes a blocking synchronous moq-ffi call. Prove + the chosen method with a deliberately hung test, then remove the test. - Update `uv.lock`. Dependabot's `uv` entry for `/` then covers the new dependency.