Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m0/test-flakes-2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,6 @@ Public API: none. Wire: none.
- [reload.test.ts's close-drain tests pass under load](/quest/m0/test-flakes-2/reload-close-drain.md) - the drain cases in `js/net`'s reload tests pass in loaded multi-file runs
- [dispatch-query's terminate test survives load](/quest/m0/test-flakes-2/dispatch-query-terminate.md) - `test_terminating_the_run_stops_the_running_phase` passes reliably under host load, re-enabled once fixed
- [release.test.ts never reaches the network](/quest/m0/test-flakes-2/release-test-offline.md) - the JSR release test fakes its registry lookup and passes offline
- [moq-ffi's datagram tests don't hinge on one datagram](/quest/m0/test-flakes-2/ffi-datagram-roundtrip.md) - the default-connection datagram tests pass under CI load
- [Diagnose and re-enable the FFI datagram connection test](/quest/m0/test-flakes-2/ffi-datagram-roundtrip.md) - diagnose the default-connection timeout and re-enable the quarantined test
- [Test PulseAudio servers stay off the desktop's D-Bus](/quest/m0/test-flakes-2/pulse-dbus.md) - concurrent media runs on one desktop never collide on `org.PulseAudio1`
- [The open-GOP browser fixture fails loudly](/quest/m0/test-flakes-2/open-gop-fixture.md) - its broadcast stub matches current stats dependencies and an effect exception fails the test
58 changes: 48 additions & 10 deletions quest/m0/test-flakes-2/ffi-datagram-roundtrip.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,59 @@
# [S] moq-ffi's datagram tests don't hinge on one datagram
# [S] Diagnose and re-enable the FFI datagram connection test

## Goal

`datagrams_cross_a_default_connection` and its `requested_` sibling
(`rs/moq-ffi/src/test.rs`) pass reliably under CI load, without depending on
a single unreliable datagram arriving.
(`rs/moq-ffi/src/test.rs`) pass reliably under CI load after the timeout
cause is confirmed and fixed.

## Plan

Decided 2026-11-05 (maintainer, follow-up from
[moq-dev/moq#7914](https://github.com/moq-dev/moq/pull/7914)): ✅ m0.

Decided 2026-11-05 (maintainer, /quest-plan): ✅ Keep only
`datagrams_cross_a_default_connection` quarantined and land the quarantine in
[moq-dev/moq#7954](https://github.com/moq-dev/moq/pull/7954). Re-diagnose under
CI load after [moq-dev/moq#7951](https://github.com/moq-dev/moq/pull/7951)
lands. Re-enabling now is rejected. Do not add repeated sending without
evidence of ordinary loss.

The test sends one datagram over a real connection and waits 10 s for it;
it timed out in Check on #7914 while passing 30 of 30 runs on an idle host.
Confirm the cause first. If it's loss, keep writing datagrams until the first
arrives and assert on what arrives, rather than waiting longer. If it's the
datagram answer race planned in
[moq-dev/moq#7929](https://github.com/moq-dev/moq/pull/7929)
(`quest/m1/datagram-tracks/answer-race.md`), say so here and wait on that
quest instead.
it timed out in Check on
[moq-dev/moq#7914](https://github.com/moq-dev/moq/pull/7914) while passing
30 of 30 runs on an idle host. Confirm the cause with send/route diagnostics
before choosing the fix and re-enabling the test.

Investigation 2026-11-05 on `main` at `419b799c75`:

- The unchanged three connection cases pass locally. With temporary diagnostic
prints at `DatagramSend::send` and `Subscriber::route_datagram`,
`quest gate just rs test -p moq-ffi --lib -E 'test(datagrams_cross)'
--stress-count 100 --no-capture` passes all 300 cases. Every send returns
`Ok(())`, and all 300 received datagrams find a route. The diagnostic
prints were removed. This does not establish the cause of the CI timeout.
- All three tests await `consumer.info()` before appending. In
`DatagramServe::poll_serve`, the answer decodes before `request.accept`
exposes that info; route registration follows synchronously on that driver
before its next datagram receive pass. The answer race in
[moq-dev/moq#7951](https://github.com/moq-dev/moq/pull/7951) is real,
but was not reproduced by these tests. Do not claim it caused this timeout.
- `datagrams_cross_a_default_connection` is disabled pending diagnosis, per
the repository's flaky-test rule. Its requested siblings have not failed
in this investigation and remain enabled. No retry or longer wait was added.

## Waiting

- moq-dev/moq: wait for
[moq-dev/moq#7951](https://github.com/moq-dev/moq/pull/7951) to merge, then
capture a failing run under CI load with send/route diagnostics. Check the
PR's merged state with `gh pr view 7951 --json state,mergedAt`; once merged,
run the ignored case locally under CI-like concurrent build/test load,
with diagnostics at `DatagramSend::send` and `Subscriber::route_datagram`,
retaining a failing trace. Explicitly include the quarantine with
`quest gate just rs test -p moq-ffi --lib
-E 'test(datagrams_cross_a_default_connection)' --run-ignored all --no-capture`;
normal Check and queue runs intentionally skip it. Use it to distinguish ordinary transport loss, a route miss, or
another lifecycle bug and choose the corresponding fix before re-enabling.
Passing stress runs alone do not clear this wait or establish transport loss.
No workflow dispatch or host change is authorized for this investigation.
1 change: 1 addition & 0 deletions rs/moq-ffi/src/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6041,6 +6041,7 @@ async fn datagram_connection(

/// A native datagram-only reader reaches a publisher on the default lite-06 path.
#[tokio::test]
#[ignore = "flaky under CI load; tracked in quest/m0/test-flakes-2/ffi-datagram-roundtrip.md"]
async fn datagrams_cross_a_default_connection() {
let (broadcast, remote, session, _server) = datagram_connection("sensor", vec![]).await;
let producer = broadcast
Expand Down
1 change: 1 addition & 0 deletions sh/gh/test-inventory.allow
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# orphan missing here, and fails on an entry that has a caller.

# Flaky under load; the quest makes it reliable, then removes its `#[ignore]`.
ignore moq-ffi::test::datagrams_cross_a_default_connection /quest/m0/test-flakes-2/ffi-datagram-roundtrip.md
ignore moq-gst::source::imp::session_tests::restarts_under_reconnects_interleave_on_one_pad /quest/m0/test-flakes-2/gst-reconnect-interleave.md

# Need a GPU, a desktop, or a device that no CI runner has.
Expand Down
Loading