From b22ee690aa5626c2c5a6a3d0cbd9c2540f1a1716 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Thu, 15 Oct 2026 19:21:52 -0700 Subject: [PATCH 1/2] claim: export failure exit quest From be80d6d0d681ee8fc34d530bf0d0d9106a662e8a Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Thu, 15 Oct 2026 19:26:57 -0700 Subject: [PATCH 2/2] quest(test): close resolved export exit flake --- quest/m1/cli-stdin-release.md | 8 --- quest/m1/test-flakes-2/README.md | 1 - quest/m1/test-flakes-2/auth-serve-signals.md | 5 +- quest/m1/test-flakes-2/export-failure-exit.md | 65 ------------------- 4 files changed, 2 insertions(+), 77 deletions(-) delete mode 100644 quest/m1/test-flakes-2/export-failure-exit.md diff --git a/quest/m1/cli-stdin-release.md b/quest/m1/cli-stdin-release.md index f74d69d94fc..469e53ee81a 100644 --- a/quest/m1/cli-stdin-release.md +++ b/quest/m1/cli-stdin-release.md @@ -30,13 +30,5 @@ Recommended unattended follow-up for the approved overnight batch: applicable CLI shutdown integrations; record tested SHA, release base, and results in a separate PR targeting `release`. -The existing failed-export load quest is related but not proven to share -this cause. Its cleanup closes stdin first, so this fix alone does not -complete that quest. - Public API: no signatures or CLI flags change; a stopped process no longer waits for an unrelated stdin writer to close. Wire: none. - -## Related - -- [Failed-export exit under load](/quest/m1/test-flakes-2/export-failure-exit.md) - independently identify the process and bound that fail under load diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index cf799b235e0..fb4dd92e8c2 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -42,7 +42,6 @@ Public API: none. Wire: none. - [Auth refresh mocks time](/quest/m1/test-flakes-2/auth-refresh-mock-time.md) - moq-relay's auth refresh and fixed-lease tests measure a grant's life on one mocked clock - [uring teardown close](/quest/m1/test-flakes-2/uring-teardown-close.md) - moq-uring's published-close teardown test holds under load, fixed at its cause - [Auth serve signals](/quest/m1/test-flakes-2/auth-serve-signals.md) - moq-cli's auth serve signal tests hold under load -- [Export exit tests](/quest/m1/test-flakes-2/export-failure-exit.md) - moq-cli's failed-export and interrupted-publisher exit tests hold under load - [js -> go rewind](/quest/m1/test-flakes-2/js-go-rewind.md) - the `js -> go` interop cell never reads a frame below the previous group's start, fixed on the side that produced it - [WASM IETF read](/quest/m1/test-flakes-2/wasm-ietf-read.md) - the WASM suite's IETF read case reads whole groups on a loaded runner, fixed at the burst or the assertion as the evidence says - [announced fixture connect](/quest/m1/test-flakes-2/cli-announced-connect.md) - moq-cli's `announced` fixture publisher connects to its ready relay under load diff --git a/quest/m1/test-flakes-2/auth-serve-signals.md b/quest/m1/test-flakes-2/auth-serve-signals.md index d26be67c6ec..685a169fbc1 100644 --- a/quest/m1/test-flakes-2/auth-serve-signals.md +++ b/quest/m1/test-flakes-2/auth-serve-signals.md @@ -8,9 +8,8 @@ their cause. They still prove that a signal exits `auth serve` cleanly. ## Plan -Seen 2026-10-13 on #5458 and #5481 at load averages near 47 on 24 cores, -beside [export failure exit](/quest/m1/test-flakes-2/export-failure-exit.md): -each timed out and passed alone. Each test spawns a real `moq auth serve` and +Seen 2026-10-13 on #5458 and #5481 at load averages near 47 on 24 cores. +Each test timed out and passed alone. Each test spawns a real `moq auth serve` and bounds two waits with a 10 s `TIMEOUT`: the "auth server listening" log line, which only gates setup, and the exit after the signal, which is the property. diff --git a/quest/m1/test-flakes-2/export-failure-exit.md b/quest/m1/test-flakes-2/export-failure-exit.md deleted file mode 100644 index 714ac452e62..00000000000 --- a/quest/m1/test-flakes-2/export-failure-exit.md +++ /dev/null @@ -1,65 +0,0 @@ -# [S] moq-cli export exit tests hold under load - -## Goal - -`moq-cli`'s `an_export_failure_with_the_broadcast_up_exits_without_lingering` -(`rs/moq-cli/tests/export.rs`) passes under a loaded `just check`, fixed at -its cause. It still proves that a failed export exits 1 without waiting for -the broadcast to end. `a_drop_exits_one_once_the_linger_expires` in the same -file also passes under load, still proving an interrupted publisher's drop -makes the export exit 1 once the linger expires. - -## Plan - -Seen 2026-10-13 on #5458 and #5481 at load averages near 47 on 24 cores, -beside [auth serve signals](/quest/m1/test-flakes-2/auth-serve-signals.md): -it timed out and passed alone. - -The test runs an in-process relay with real `moq import` and `moq export` -processes. Two bounds apply. The exit wait has a 30 s `TIMEOUT`, so a child -that lingers fails there. The "exited within 10 s" assertion runs only after -the child exits, so a slow but completed exit fails there. That 10 s is the -property (no lingering) and must stay bounded. It starts after both `spawn()` -calls return, just before the AV1 write, so it excludes the parent's spawn -but still covers each child's own startup and relay connection, the import, -the export's catalog read, and its exit. Cleanup then closes the import's -stdin and waits on it with the same 30 s guard, which fails with the same -"moq never exited" message. - -Reproduce under load first and record which bound fired, and for "moq never -exited" which process. Find what in that -interval is slow under load and fix it, or move the check to a deterministic -in-process test if the process boundary isn't what it proves. Never widen a -timeout or add a retry. - -Folded in 2026-10-14 (maintainer): `a_drop_exits_one_once_the_linger_expires` -bounced #5528 from the merge queue, exiting 0 instead of 1. Its `interrupt` -helper (`rs/moq-cli/tests/export.rs:142`) sends SIGINT, sleeps 500 ms, then -closes the publisher's stdin, because the interrupted process only exits once -its stdin read returns. On a loaded runner stdin closed first, so the -publisher logged "input ended" and finished the broadcast cleanly instead of -dropping it. Fix the cause, not the sleep: preferably the publisher exits on -SIGINT without waiting for its stdin read, so the test never closes stdin. -#5510 landed that runtime-teardown fix on main, so drop the stdin close and -the sleep here. -`interrupted` (`export.rs:264`) also sleeps 1 s after the interrupt; replace -it with the exporter's paced tail draining, not the publisher's exit. Fixing -this race alone does not resolve the failed-export timing above. - -Fixture cleanup on 2026-10-15: hold the publisher's stdin writer until its -signal-driven exit, and mark interrupted output only after the follower enters -linger and the stdout reader drains the queued tail. No scheduling sleep or -stdin EOF decides whether the publication ended cleanly. The test pins the -follower's log level for its lifecycle event. - -The separate historical failed-export timing failure remains open. On current -main `0fc797747`, both named tests passed before these fixture changes under a -loaded host (about 32): failed-export exited in 2.285 s and the interrupted -case in 4.787 s. Those passes do not identify or resolve the historical cause. -Reproduce and record the failed bound and process before changing that arm. - -Public API: none. Wire: none. - -## Related - -- [CLI stdin release](/quest/m1/cli-stdin-release.md) - the release backport of that fix