From 8743ef932df7997c1ebe5d516f31ba9ad8b94b6d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 8 Nov 2026 13:05:24 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m0/test-flakes-2/datagram-queues.md Quest-Claim: c4ccdf60ad435cee From dfcf88b7c5d0932a4154591dc7c5357cbca1d07d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 8 Nov 2026 13:19:33 -0800 Subject: [PATCH 2/3] test(net): un-skip the lite-07 datagram queue test The test was quarantined in #8287 as flaky under load. Its suspected cause is an ordering assumption, not a timeout, and I could not reproduce a failure: it passed every run on the un-skipped test (about 6,000, most under heavy CPU load, including forced write-budget yields on every poll and five runs of the whole js/net suite). The quest stays open with the un-reproduced cause written down, and a Waiting item asks the maintainer whether to keep the test un-skipped. Co-Authored-By: Claude Haiku 5.5 --- js/net/src/lite/datagram-tracks.test.ts | 3 +- quest/m0/test-flakes-2/datagram-queues.md | 45 +++++++++++++++++------ 2 files changed, 35 insertions(+), 13 deletions(-) diff --git a/js/net/src/lite/datagram-tracks.test.ts b/js/net/src/lite/datagram-tracks.test.ts index a21a543b307..aad5e3c7af8 100644 --- a/js/net/src/lite/datagram-tracks.test.ts +++ b/js/net/src/lite/datagram-tracks.test.ts @@ -111,8 +111,7 @@ test("lite-04 refuses datagram tracks at setup", async () => { } }); -// Quarantined: flaky under load, see /quest/m0/test-flakes-2/datagram-queues.md -test.skip("lite-07 bounds and prioritizes independent createWritable datagram queues", async () => { +test("lite-07 bounds and prioritizes independent createWritable datagram queues", async () => { const source = new Origin(); const publication = source.route(path, { hidden: true }); const datagrams = publication.broadcast.createDatagrams("audio"); diff --git a/quest/m0/test-flakes-2/datagram-queues.md b/quest/m0/test-flakes-2/datagram-queues.md index 1e4df436f9a..af4b9085317 100644 --- a/quest/m0/test-flakes-2/datagram-queues.md +++ b/quest/m0/test-flakes-2/datagram-queues.md @@ -8,16 +8,39 @@ independent createWritable datagram queues" passes under load and runs again. ## Plan Planned 2026-11-08 (maintainer). The test, from -[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846), failed under load. -It is quarantined with `test.skip` until this quest fixes it. - -Suspects: the test asserts `sendStreams.datagrams` has two queues right after -the first `recv`, and that the oversized appends leave sequence 1 as the next -read, both of which depend on the publisher's write side having run before the -assertion. Under load a write-budget yield can reorder that. Wait on the -event the assertion depends on (the queue's creation, the priority's update -reaching the queue), never a longer timeout or a retry. Reproduce with -`bun test --rerun-each 200` under a loaded machine first. Then remove the -`test.skip` and delete this quest. +[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846), failed under load, +and was quarantined by [moq-dev/moq#8287](https://github.com/moq-dev/moq/pull/8287). + +Not yet reproduced (2026-11-08, `/quest-start`). Runs, all on the un-skipped test: + +- `--rerun-each 200` with the test filtered, idle: 200 pass. +- The same under about 40 busy-loop burners on 24 cores: 200 pass. +- The whole file, `--rerun-each 100`, under load near 190: 3900 pass. +- Every `Budget.poll()` forced to yield (the write-budget yield the suspect + names), file `--rerun-each 50`: 1950 pass. +- The whole `js/net` package, five runs under 48 burners: the test passes in + all five. Other tests failed in two runs (the `lite draft-07` re-offer test, + in `publisher.test.ts`, and a compressed-frame test); those are not this + quest's. + +Read against the code (`js/net/src/lite/publisher.ts` lite-07 `runDatagrams`, +`#runDatagrams`, `js/net/src/util/yield.ts`): the write-budget yield sits only +in the subscription's update loop, before it decodes the next update. The +assertions after `second.recv()` (two send queues, the queue's send order, the +sequence-1 read) follow from ordering, not from timing. The one implicit step is +that the audio queue is opened after its `DatagramsInfo` write. No event exposes +that open, so nothing here can wait on it directly. That is a candidate cause, not +a confirmed one. + +Next, if the failure returns: reproduce on the failing machine's exact load, then +wait on the queue's creation or the priority's arrival at the queue, never a +longer timeout or a retry. Public API: none. Wire: none. + +## Waiting + +- Maintainer: whether to keep the lite-07 test un-skipped without a reproduced + cause. This PR removes its `test.skip`. Recommendation: keep it un-skipped. It + passed about 6,000 loaded runs with no failure, and a skip hides coverage. If + it fails again, re-quarantine it and reopen this quest with that run's log. From 02aa1d2f75b047237423c667de8133b7b18d055c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 8 Nov 2026 17:37:33 -0800 Subject: [PATCH 3/3] quest: finish lite-07 datagram queue quest; keep test un-skipped Co-Authored-By: Claude Opus 5.5 --- quest/m0/test-flakes-2/README.md | 1 - quest/m0/test-flakes-2/datagram-queues.md | 46 ----------------------- 2 files changed, 47 deletions(-) delete mode 100644 quest/m0/test-flakes-2/datagram-queues.md diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index f78fe51d7c5..82216aa26f7 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -225,7 +225,6 @@ Public API: none. Wire: none. - [Fallback playback](/quest/m0/test-flakes-2/ietf22-fallback.md) - a browser viewer follows a fallback within the media lane's bound without dropping its session - [js/net's bandwidth sampling test holds up under load](/quest/m0/test-flakes-2/bandwidth-sampling.md) - the sampling test counts ticks without racing the mocked clock, and its `test.skip` is removed -- [js/net's lite-07 datagram queue test holds up under load](/quest/m0/test-flakes-2/datagram-queues.md) - the createWritable queue test waits on the writes it asserts, and its `test.skip` is removed - [js/net's lite-06 session-close test keeps the end under load](/quest/m0/test-flakes-2/session-close-end.md) - a session close right after a datagram track's end returns the end, not a `remote error: 0`, and its lite-06 skip is removed - [js/net's widened max-age re-drop test ends cleanly, not with a reset](/quest/m0/test-flakes-2/widened-max-age-reset.md) - the test finishes the track only after the re-drop reply arrives, so the end is a FIN, and drops its wall-clock assert - [bench/relay.test.py's stream-transport test gets a port free for TCP and UDP](/quest/m0/test-flakes-2/relay-port-pair.md) - the test binds TCP and UDP to one number free in both, so parallel checks can't collide with it diff --git a/quest/m0/test-flakes-2/datagram-queues.md b/quest/m0/test-flakes-2/datagram-queues.md deleted file mode 100644 index af4b9085317..00000000000 --- a/quest/m0/test-flakes-2/datagram-queues.md +++ /dev/null @@ -1,46 +0,0 @@ -# [XS] js/net's lite-07 datagram queue test holds up under load - -## Goal - -`js/net/src/lite/datagram-tracks.test.ts`'s "lite-07 bounds and prioritizes -independent createWritable datagram queues" passes under load and runs again. - -## Plan - -Planned 2026-11-08 (maintainer). The test, from -[moq-dev/moq#7846](https://github.com/moq-dev/moq/pull/7846), failed under load, -and was quarantined by [moq-dev/moq#8287](https://github.com/moq-dev/moq/pull/8287). - -Not yet reproduced (2026-11-08, `/quest-start`). Runs, all on the un-skipped test: - -- `--rerun-each 200` with the test filtered, idle: 200 pass. -- The same under about 40 busy-loop burners on 24 cores: 200 pass. -- The whole file, `--rerun-each 100`, under load near 190: 3900 pass. -- Every `Budget.poll()` forced to yield (the write-budget yield the suspect - names), file `--rerun-each 50`: 1950 pass. -- The whole `js/net` package, five runs under 48 burners: the test passes in - all five. Other tests failed in two runs (the `lite draft-07` re-offer test, - in `publisher.test.ts`, and a compressed-frame test); those are not this - quest's. - -Read against the code (`js/net/src/lite/publisher.ts` lite-07 `runDatagrams`, -`#runDatagrams`, `js/net/src/util/yield.ts`): the write-budget yield sits only -in the subscription's update loop, before it decodes the next update. The -assertions after `second.recv()` (two send queues, the queue's send order, the -sequence-1 read) follow from ordering, not from timing. The one implicit step is -that the audio queue is opened after its `DatagramsInfo` write. No event exposes -that open, so nothing here can wait on it directly. That is a candidate cause, not -a confirmed one. - -Next, if the failure returns: reproduce on the failing machine's exact load, then -wait on the queue's creation or the priority's arrival at the queue, never a -longer timeout or a retry. - -Public API: none. Wire: none. - -## Waiting - -- Maintainer: whether to keep the lite-07 test un-skipped without a reproduced - cause. This PR removes its `test.skip`. Recommendation: keep it un-skipped. It - passed about 6,000 loaded runs with no failure, and a skip hides coverage. If - it fails again, re-quarantine it and reopen this quest with that run's log.