From 08b9cc8893b8e08f7b39c79f76b6b3c9edfad118 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 12 Nov 2026 23:41:16 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m0/test-flakes-2/interop-go-python.md Quest-Claim: e91b98828074d63c From d4c2eb2bec6441f9178051690d311f9290ea0e20 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 00:33:09 -0800 Subject: [PATCH 2/3] docs(quest): checkpoint go to python interop investigation --- quest/m0/test-flakes-2/interop-go-python.md | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/quest/m0/test-flakes-2/interop-go-python.md b/quest/m0/test-flakes-2/interop-go-python.md index b6a0b53f1c3..6e22f8492b4 100644 --- a/quest/m0/test-flakes-2/interop-go-python.md +++ b/quest/m0/test-flakes-2/interop-go-python.md @@ -27,3 +27,25 @@ Reproduce with `just test interop --publishers go --subscribers python` and the interop logs kept. Confirm or rule out #8531 first, then fix the cause. Public API: none expected. Wire: none expected. + +Investigation 2026-11-13: #8531 does reach the Python audio decoder: +`moq_audio::decode::Consumer::next` calls `track.set_wait(self.delay())`. +This confirms the code path, not causation. The Go publisher stamps each +20 ms PCM batch on its contiguous sample clock, independently of its ticker. +The failing run's retained `go-python-audio.opus.txt` contains `48000 2`, +then timestamp/sample pairs `0/48`, `1000/120`, `3500/120`, `6000/120`, +`8500/120`, `11000/120`. These total 648 samples before the gap, one +20 ms batch minus the 312-sample Opus pre-skip. Its warning-level subscriber +log does not identify a skipped group or the gap's timestamp/duration. + +On main at 65ca6b6ce, the focused command passed including finished-media +and sockets. A matrix attempt with `RUST_LOG=warn,moq_mux=debug` also passed +`go -> python` and `python -> cpp`, but failed overall: `tsc` was missing, +so all three JS clients were unavailable. This is not a complete-matrix +reproduction. No playback or client changes were made. #8611 was read first; +it currently changes only its own quest notes. + +Next: restore the JS clients' TypeScript toolchain and repeat the complete +matrix with the same debug filter and retained logs until the gap reproduces. +Capture both the container skip reason and the audio gap timestamp/duration +before attributing the failure to #8531. Do not raise a budget on this evidence. From 9d131e85f5bdfdc5cec69621d074a6c065b7a528 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 13 Nov 2026 03:17:44 -0800 Subject: [PATCH 3/3] test(interop): record ten complete go to python matrix runs --- quest/m0/interop-green/README.md | 17 +++---- quest/m0/test-flakes-2/README.md | 16 ++++--- quest/m0/test-flakes-2/interop-go-python.md | 51 --------------------- 3 files changed, 16 insertions(+), 68 deletions(-) delete mode 100644 quest/m0/test-flakes-2/interop-go-python.md diff --git a/quest/m0/interop-green/README.md b/quest/m0/interop-green/README.md index e7ffe2091f0..546b778bd1e 100644 --- a/quest/m0/interop-green/README.md +++ b/quest/m0/interop-green/README.md @@ -177,16 +177,13 @@ Scheduled run carries it, as the Rust side of the test expects. [moq-dev/moq#8572](https://github.com/moq-dev/moq/pull/8572) fixes the expectation, and is open. -- Interop (full matrix): `go -> python` fails with `audio gap after 648 samples` - at 2 s of 30 s. It passed once locally, and 8 more local runs of - `--publishers go --subscribers python,rust,go` passed every cell. A lead, not - checked: [moq-dev/moq#8531](https://github.com/moq-dev/moq/pull/8531) now waits on - a missing group in moq-audio only for the playout delay, which is small at a - cold start, so a reordered group early on reads as a gap. No open PR names this cell; - [moq-dev/moq#8587](https://github.com/moq-dev/moq/pull/8587) covers `python -> go`. - Decided 2026-11-12 (maintainer): the cell moves to - [test flakes](/quest/m0/test-flakes-2/interop-go-python.md), which checks - #8531 first. +- Interop (full matrix): `go -> python` failed with `audio gap after 648 samples` + in the scheduled run. [moq-dev/moq#8633](https://github.com/moq-dev/moq/pull/8633) + could not reproduce it in ten completed full matrices on 2026-11-13: the cell + passed all ten, nine matrices passed entirely, and one failed other cells. + No runtime fix is claimed. The related + [moq-dev/moq#8611](https://github.com/moq-dev/moq/pull/8611) reproduces a + missing-group skip under CPU starvation in the same consumer and owns that fix. - Media output and lifecycle: after the pause case resumes, the audio trails the picture by about 1.7 s. [moq-dev/moq#8507](https://github.com/moq-dev/moq/pull/8507) keeps audio that trails the clock up to its 2 s stretch bound and plays it diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index ba124f00daf..56235316edb 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -280,12 +280,14 @@ its third failure since 2026-10-16 and its first after [moq-dev/moq#8526](https://github.com/moq-dev/moq/pull/8526). It is quarantined with `#[ignore]` until its child fixes it. -Added 2026-11-12 (maintainer, from the Interop triage of scheduled run -[43260674067](https://github.com/moq-dev/moq/actions/runs/43260674067)): the -`go -> python` media cell, which failed `audio gap after 648 samples` once and -passed 8 local runs after it. Decided: ✅ add it here as the child -[interop's go -> python media cell](/quest/m0/test-flakes-2/interop-go-python.md), -not quarantined (rejected: a new quest after a second red run; quarantining the -cell). Check [moq-dev/moq#8531](https://github.com/moq-dev/moq/pull/8531) first. +The `go -> python` media cell failed with `audio gap after 648 samples` in +scheduled run [43260674067](https://github.com/moq-dev/moq/actions/runs/43260674067). +[moq-dev/moq#8633](https://github.com/moq-dev/moq/pull/8633) investigated +[moq-dev/moq#8531](https://github.com/moq-dev/moq/pull/8531) and ran ten complete +full matrices in the dev shell with retained debug logs. The cell passed every +one; nine matrices passed entirely. The repeated-run outcome, authorized by the +maintainer on 2026-11-13, retires its separate quest without claiming a runtime +fix or ruling out a rarer failure. The shared container's CPU-starvation skip +is being fixed in [moq-dev/moq#8611](https://github.com/moq-dev/moq/pull/8611). Public API: none. Wire: none. diff --git a/quest/m0/test-flakes-2/interop-go-python.md b/quest/m0/test-flakes-2/interop-go-python.md deleted file mode 100644 index 6e22f8492b4..00000000000 --- a/quest/m0/test-flakes-2/interop-go-python.md +++ /dev/null @@ -1,51 +0,0 @@ -# [S] Interop's go -> python media cell holds up - -## Goal - -The `go -> python` media cell of `just test interop --all` passes reliably, -fixed at its cause, never by a quarantine, a longer timeout, or a retry. - -## Plan - -Found 2026-11-12 in the Interop triage of scheduled run -[43260674067](https://github.com/moq-dev/moq/actions/runs/43260674067) (head -e9c6df852, the first run after #8455 and #8512). Its full-matrix step failed -`go -> python` with `audio gap after 648 samples` at 2 s of 30 s. It passed -once locally, and 8 more local runs of `--publishers go --subscribers -python,rust,go` passed every cell. No open PR names this cell. - -Decided 2026-11-12 (maintainer): ✅ track it as this child of test flakes -(rejected: a new quest after a second red run; quarantining the cell). - -Check first: [moq-dev/moq#8531](https://github.com/moq-dev/moq/pull/8531) -(`fix(hang,moq-mux): wait on a missing group for the delay, not the max`) merged -on 2026-11-12 and is in the failing run's head. A lead, not checked: it waits on -a missing group only for the playout delay, which is small at a cold start, so a -group that arrives out of order early on reads as an audio gap. - -Reproduce with `just test interop --publishers go --subscribers python` and the -interop logs kept. Confirm or rule out #8531 first, then fix the cause. - -Public API: none expected. Wire: none expected. - -Investigation 2026-11-13: #8531 does reach the Python audio decoder: -`moq_audio::decode::Consumer::next` calls `track.set_wait(self.delay())`. -This confirms the code path, not causation. The Go publisher stamps each -20 ms PCM batch on its contiguous sample clock, independently of its ticker. -The failing run's retained `go-python-audio.opus.txt` contains `48000 2`, -then timestamp/sample pairs `0/48`, `1000/120`, `3500/120`, `6000/120`, -`8500/120`, `11000/120`. These total 648 samples before the gap, one -20 ms batch minus the 312-sample Opus pre-skip. Its warning-level subscriber -log does not identify a skipped group or the gap's timestamp/duration. - -On main at 65ca6b6ce, the focused command passed including finished-media -and sockets. A matrix attempt with `RUST_LOG=warn,moq_mux=debug` also passed -`go -> python` and `python -> cpp`, but failed overall: `tsc` was missing, -so all three JS clients were unavailable. This is not a complete-matrix -reproduction. No playback or client changes were made. #8611 was read first; -it currently changes only its own quest notes. - -Next: restore the JS clients' TypeScript toolchain and repeat the complete -matrix with the same debug filter and retained logs until the gap reproduces. -Capture both the container skip reason and the audio gap timestamp/duration -before attributing the failure to #8531. Do not raise a budget on this evidence.