From bc27bed055eab62378a87235c2ced71eb7f380da Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 21 Oct 2026 22:35:29 -0700 Subject: [PATCH 1/5] chore(quest): claim quest/m1/test-flakes-2/three-flakes.md Co-Authored-By: Claude Opus 5.5 From ce4f9abb081cb8b33b4b93f42ab5d9bc8afd56ad Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 21 Oct 2026 23:37:19 -0700 Subject: [PATCH 2/5] test(obs): suppress Rust races reported under demangled names just obs test failed on its first relay: a tokio task-cell handoff reported with only debug-info frames, which are demangled, so race:^_R missed it. Match frames by their .rs source file as well. Co-Authored-By: Claude Opus 5.5 --- cpp/obs/test/tsan.supp | 9 +++++--- quest/m1/test-flakes-2/three-flakes.md | 31 -------------------------- 2 files changed, 6 insertions(+), 34 deletions(-) delete mode 100644 quest/m1/test-flakes-2/three-flakes.md diff --git a/cpp/obs/test/tsan.supp b/cpp/obs/test/tsan.supp index 47676c995b9..2662f7dd4d0 100644 --- a/cpp/obs/test/tsan.supp +++ b/cpp/obs/test/tsan.supp @@ -4,10 +4,13 @@ # (tokio's atomics and parking, the handoffs between its threads) is invisible and # reads as a race wherever memory crosses it. That includes the generated bindings' # future plumbing, which hands its state to Rust on one thread and gets it back in a -# Rust callback on another. This matches reports with a Rust frame on either stack: -# the symbolizer leaves Rust's v0-mangled names as they are, and every one starts -# with _R, which no demangled C++ name does. +# Rust callback on another. These match reports with a Rust frame on either stack. +# A frame's name depends on how it was symbolized: from the symbol table it is a +# v0-mangled name starting with _R, which no demangled C++ name does, but from an +# inlined call's debug info it is a demangled one. Its source file ends in .rs +# either way. race:^_R +race:.rs$ # aws-lc is C built inside moq-ffi without the sanitizer too. A TLS connection's # state moves between moq-ffi's worker threads through tokio's handoffs, so its diff --git a/quest/m1/test-flakes-2/three-flakes.md b/quest/m1/test-flakes-2/three-flakes.md deleted file mode 100644 index e9349a6cea6..00000000000 --- a/quest/m1/test-flakes-2/three-flakes.md +++ /dev/null @@ -1,31 +0,0 @@ -# [XS] OBS drain and JS allocation flakes - -## Goal - -Each passes reliably under a loaded `just check`, fixed at its cause: - -- OBS "stop drains the session" (`cpp/obs/test/moq-output-test.cpp`, a - 16 MiB frame under TSan), seen in - [moq-dev/moq#6131](https://github.com/moq-dev/moq/pull/6131) and - [moq-dev/moq#6137](https://github.com/moq-dev/moq/pull/6137). -- The js/net RSS memory test, which flakes under load - ([moq-dev/moq#6128](https://github.com/moq-dev/moq/pull/6128)). - -## Plan - -Reproduce before fixing, with no retries, sleeps, or raised timeouts. -The unchanged OBS suite passed both a full TSan run and an output-only TSan -run pinned to one CPU on 2026-10-21. Its artificial 16 MiB frame competes -with the session's intentional one-second drain deadline, but a local -failure has not yet established the right regression. Keep the lifecycle -assertions when separating that contract from throughput. - -[moq-dev/moq#6188](https://github.com/moq-dev/moq/pull/6188) owns the RSS -replacement: bounded algorithm assertions rather than process RSS. Do not -duplicate it; verify its merged result under load. - -The original audio-quality RTT and fresh-worktree interop conditions are -implemented: the sampler observes PROBE while sampling, and the interop -entry point installs Bun dependencies before Rust invokes the JS codecs. - -Public API: none. Wire: none. From cfb6432543f5f096f7c9ef71fa73b94c0b10b3bd Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 21 Oct 2026 23:37:19 -0700 Subject: [PATCH 3/5] fix(obs): record the connect time before the session reads as live A reader that saw IsLiveSession() could still read connect_time_ms as 0, OBS's never-connected sentinel. Seen as a loaded TSan failure of the real-session publish test. Co-Authored-By: Claude Opus 5.5 --- cpp/obs/src/moq-output.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/cpp/obs/src/moq-output.cpp b/cpp/obs/src/moq-output.cpp index 8a7763028e3..b6e12385428 100644 --- a/cpp/obs/src/moq-output.cpp +++ b/cpp/obs/src/moq-output.cpp @@ -272,6 +272,11 @@ void MoQOutput::OnConnect(const std::shared_ptr ¤t, moq::expected auto ms = static_cast(std::chrono::duration_cast(elapsed).count()); const int connects = static_cast((*result)->connects()); + // OBS and older dock paths treat connect_time_ms == 0 as "never connected". + // Clamp sub-millisecond connects to 1 so that sentinel stays honest. Set before + // the session reads as live, so nothing sees a live session that never connected. + connect_time_ms = ms > 0 ? ms : 1; + current->session = *result; current->connected = true; current->live = true; @@ -290,9 +295,6 @@ void MoQOutput::OnConnect(const std::shared_ptr ¤t, moq::expected last_failure_code = MoQError::None; last_failure_reason.clear(); } - // OBS and older dock paths treat connect_time_ms == 0 as "never connected". - // Clamp sub-millisecond connects to 1 so that sentinel stays honest. - connect_time_ms = ms > 0 ? ms : 1; LOG_INFO("MoQ session connected (%d ms, connect %d): %s", ms, connects, MoQRedactUrl(current->url).c_str()); From c67580276bc822f45e4dd1839e0ee022a46b293b Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 21 Oct 2026 23:37:19 -0700 Subject: [PATCH 4/5] test(obs): size the stop-drain frame inside the drain deadline The 16 MiB frame took 750-800 ms of moq-net's one-second drain deadline under TSan and overran it on a loaded host, so the deadline cut the track off by design. 2 MiB still catches a Stop() that cancels instead of draining, and drains in under 300 ms under the same load. Co-Authored-By: Claude Opus 5.5 --- cpp/obs/test/moq-output-test.cpp | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/cpp/obs/test/moq-output-test.cpp b/cpp/obs/test/moq-output-test.cpp index f05f5848318..99f7f66ca82 100644 --- a/cpp/obs/test/moq-output-test.cpp +++ b/cpp/obs/test/moq-output-test.cpp @@ -632,9 +632,12 @@ int main() auto first = TestOk(next.get(), "first frame"); CHECK(first.has_value()); - // Large enough that it is still in flight when Stop() runs. + // Large enough to be in flight when Stop() runs, so a Stop() that cancels + // rather than drains cuts it off (256 KiB can land first under TSan). Small + // enough to drain well inside moq-net's one-second drain deadline, which cuts + // the track off by design, even under TSan on a loaded host. auto large = keyframe; - large.resize(large.size() + 16 * 1024 * 1024, 0xff); + large.resize(large.size() + 2 * 1024 * 1024, 0xff); auto last = videoPacket(large, 1); o->Data(&last); // OBS may destroy the output right after stopping it. From 0cb28a6af0321b558e26ecae100a6e6f0a7e8f0a Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Wed, 21 Oct 2026 23:37:20 -0700 Subject: [PATCH 5/5] quest(m1): finish the OBS drain and JS allocation flakes Co-Authored-By: Claude Opus 5.5 --- quest/m1/test-flakes-2/README.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index 342aff67ec7..770f01d1a92 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -1,4 +1,4 @@ -# More tests hold up under load +# [S] More tests hold up under load ## Goal @@ -40,16 +40,11 @@ which #6032 had already finished, so it reuses #6032's reserved-port fix as its own child. moq-uring's `host_tracks_moq_clock` was seen the same day but #6056 had already fixed it by measuring the calibration gap. -Added 2026-10-21 (maintainer): three small flakes and a fresh-worktree -interop failure from the m0/m1 merge run, kept in one child since each is -small. +Added 2026-10-21 (maintainer): two flakes (OBS drain, js/net RSS) from the +m0/m1 merge run, kept in one child since each is small. Public API: none. Wire: none. -## Required - -- [OBS drain and JS allocation flakes](/quest/m1/test-flakes-2/three-flakes.md) - reproduce the OBS stop-drain failure and verify the existing RSS replacement under load - ## Related - [Controlled time](/quest/m1/time/README.md) - the systemic fix these rules ask for: every crate on a controlled clock