diff --git a/quest/m1/README.md b/quest/m1/README.md index b355b87a40a..232a76619d2 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -118,7 +118,6 @@ Binding durations wait on Duration support in the generated C. - [Moxygen minimal interop](/quest/m1/smoke-moxygen.md) - waiting on Moxygen to send a registered filter type, then the smoke lane stops hiding its failure - [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause - [Broadcast lifecycle](/quest/m1/broadcast-lifecycle/README.md) - broadcasts are online or offline with no finished state in moq-net, finished-broadcast retention is a fixed moq-mux policy, and each linger timer has one meaning -- [JS interop tone](/quest/m1/interop-js-tone.md) - the `js ->` cells pass the audio check on every draft again, after the 2026-10-23 regression - [fMP4 edit lists](/quest/m1/fmp4-edit-lists.md) - the fMP4 importer applies edit lists to frame timestamps, so the interop AAC tone passes the phase check - [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break says how to migrate - [moq-quic's API improves on quinn's](/quest/m1/quic-api/README.md) - moq-quic and both QUIC drivers ship one mirrored API: `send::`/`recv::` namespaces, one error per direction, consuming terminal operations, plain configs, no quinn baggage diff --git a/quest/m1/interop-js-tone.md b/quest/m1/interop-js-tone.md deleted file mode 100644 index c8f0944d0b0..00000000000 --- a/quest/m1/interop-js-tone.md +++ /dev/null @@ -1,27 +0,0 @@ -# [M] The JS interop publisher's tone survives every draft - -## Goal - -Interop's `js ->` cells pass the audio check on every moq-transport draft -again, with the cause of the 2026-10-23 regression fixed at its source. - -## Plan - -The scheduled Interop run on 2026-10-23 -([run 39958710886](https://github.com/moq-dev/moq/actions/runs/39958710886), -`--all --versions ietf`) failed every `js ->` cell on drafts 18 and 22, and -`js -> rust` and `js -> c` on draft 20, with "wrong tone: channel 0 carries -~440 Hz, not 440 Hz" at 0.10 to 0.77 of the energy at 440 Hz. `rust -> js` -also failed on drafts 20 and 22. The same job passed all of these on -2026-10-22, and drafts 14 and 16 pass on both days, so it is not a load -flake. It started after -[moq-dev/moq#6299](https://github.com/moq-dev/moq/pull/6299) merged (pause -and resume as a new subscriber), which is a guess from timing only. - -Reproduce one cell locally, bisect between the two runs, and fix the cause. -Separate from [tone grace](/quest/m1/test-flakes-2/tone-grace.md), which -covers a false "wrong tone" under CPU starvation. - -Decided 2026-10-23 (maintainer): its own quest, ranked above the load flakes. - -Public API: depends on the cause. Wire: depends on the cause. diff --git a/test/interop/README.md b/test/interop/README.md index 74dbb10a68d..3e0ad28f65e 100644 --- a/test/interop/README.md +++ b/test/interop/README.md @@ -20,8 +20,11 @@ decodes it. Python and C++ send a mono 440 Hz tone; the rest send 440 Hz left an 660 Hz right, so a channel swap fails too. The Rust CLI imports it from ffmpeg in fragmented MP4, as both an Opus and an AAC rendition, so AAC import and decode cross every subscriber (the wire-compat lanes send Opus alone; see -[Released wire compatibility](#released-wire-compatibility)). The browser encodes its fake microphone playing the tone from a WAV -file, and the Python, Go, and C++ clients encode it through `moq-ffi` at a 2.5 ms +[Released wire compatibility](#released-wire-compatibility)). The browser decodes the picture +and the tone from a looping file through ``'s file source and encodes both, rather +than capturing Chromium's fake camera and microphone: those run on timers that skip a late tick, +so a runner short of CPU splices the captured tone with no timestamp gap. The Python, Go, and +C++ clients encode it through `moq-ffi` at a 2.5 ms frame duration, so the matrix covers the FFI audio path with a non-default codec config. Each subscriber decodes every audio rendition with what it already has: moq-audio for Rust (the `interop-audio` example, since the CLI has no PCM output), `moq-ffi` for Python, @@ -34,8 +37,8 @@ most of its energy by a Goertzel filter. A garbled decode (wrong channel order o sample rate), silence, a timestamp gap, or a rendition left undecoded fails the cell. Each publisher's tone starts at timestamp zero, so `--phase` also holds every Opus rendition's tone to its timestamps within a sample, which fails a decoder that keeps -or drops the wrong pre-skip. It skips the browser publisher, whose capture clock is -not tied to its WAV, and the native-JS subscriber, which leaves the pre-skip to the +or drops the wrong pre-skip. It skips the browser publisher, which stamps its file +from its own page clock, and the native-JS subscriber, which leaves the pre-skip to the player as WebCodecs does and stamps milliseconds. AAC is skipped until the fMP4 importer applies the edit list holding ffmpeg's priming ([quest](/quest/m1/fmp4-edit-lists.md)). The browser plays each rendition in a page diff --git a/test/interop/clients/js/driver.ts b/test/interop/clients/js/driver.ts index 309b0d085ac..3b8c4c6a485 100644 --- a/test/interop/clients/js/driver.ts +++ b/test/interop/clients/js/driver.ts @@ -1,6 +1,6 @@ /** * Drives a headless Chromium against the vite-built page (see `serve`) for the interop matrix. publish - * streams a fake camera and a microphone playing `--audio-file` until killed; subscribe verifies, + * streams the picture and tone decoded from `--media`, looping, until killed; subscribe verifies, * once per `--codecs` audio rendition with the player pinned to it, rendered playback, that each * audio channel the player outputs carries its `--tone`, and pause/resume. * @@ -8,7 +8,7 @@ * so this driver tolerates a slow announcement with one reload. `media.ts` is the strict path: it * waits on an explicit publisher-ready state and never reloads. * - * bun driver.ts publish --url http://127.0.0.1:4443 --broadcast b.hang --audio-file tone.wav [--version V] + * bun driver.ts publish --url http://127.0.0.1:4443 --broadcast b.hang --media tone.mov [--version V] * bun driver.ts subscribe --url http://127.0.0.1:4443 --broadcast b.hang --timeout 20 --tone 440,660 \ * --codecs opus,mp4a [--version V] * @@ -66,7 +66,7 @@ const { positionals, values } = parseArgs({ "finish-ack": { type: "string" }, frames: { type: "string", default: "10" }, timeout: { type: "string", default: "20" }, - "audio-file": { type: "string" }, + media: { type: "string" }, tone: { type: "string" }, codecs: { type: "string" }, }, @@ -77,7 +77,7 @@ const url = values.url; const broadcast = values.broadcast; const timeoutMs = Number.parseFloat(values.timeout ?? "20") * 1000; const finishing = values.finish || values["finish-ack"] !== undefined; -const audioFile = values["audio-file"]; +const media = values.media; const tones = (values.tone ?? "").split(",").filter(Boolean).map(Number); // Each audio rendition by codec: the catalog's up to its first `.`. const codecs = (values.codecs ?? "").split(",").filter(Boolean); @@ -87,11 +87,11 @@ if ( !broadcast || !Number.isFinite(timeoutMs) || timeoutMs <= 0 || - (!finishing && role === "publish" && !audioFile) || + (!finishing && role === "publish" && !media) || (!finishing && role === "subscribe" && (!tones.length || tones.some((hz) => !(hz > 0)) || !codecs.length)) ) { console.error( - "usage: driver.ts publish --url U --broadcast B (--audio-file WAV | --finish-ack FIFO) | subscribe --url U --broadcast B (--tone HZ,... --codecs CODEC,... | --finish [--frames N]) [--version V] [--timeout S>0]", + "usage: driver.ts publish --url U --broadcast B (--media FILE | --finish-ack FIFO) | subscribe --url U --broadcast B (--tone HZ,... --codecs CODEC,... | --finish [--frames N]) [--version V] [--timeout S>0]", ); process.exit(2); } @@ -134,14 +134,10 @@ async function waitForStablePause(page: Page, errors: BrowserErrors, deadline: n throw new Error(`playback did not stop after pause: ${JSON.stringify(previous)}`); } -const server = serve(); -const browser = await launch([ - "--use-fake-device-for-media-stream", - "--use-fake-ui-for-media-stream", - "--autoplay-policy=no-user-gesture-required", - // The fake microphone loops this file instead of its default beeps. - ...(audioFile ? [`--use-file-for-fake-audio-capture=${audioFile}`] : []), -]); +// The page fetches the publisher's file from here. +const MEDIA_PATH = "/interop-media"; +const server = serve(media ? { [MEDIA_PATH]: media } : {}); +const browser = await launch(["--autoplay-policy=no-user-gesture-required"]); /** * Play the broadcast in a page of its own, pinned to its `codec` audio rendition: rendered video, the @@ -321,10 +317,15 @@ try { } else { await open( browser, - pageUrl(server.origin, role, { url, broadcast, ...(values.version ? { versions: values.version } : {}) }), + pageUrl(server.origin, role, { + url, + broadcast, + ...(media ? { media: MEDIA_PATH } : {}), + ...(values.version ? { versions: values.version } : {}), + }), role, ); - console.error(`publishing ${broadcast} (fake camera + ${audioFile} as the microphone) to ${url}`); + console.error(`publishing ${broadcast} (${media}) to ${url}`); // Stream until the orchestrator stops us. await new Promise((resolve) => process.once("SIGTERM", resolve)); code = 0; diff --git a/test/interop/clients/js/harness.ts b/test/interop/clients/js/harness.ts index f37f49a725a..6787e9453a0 100644 --- a/test/interop/clients/js/harness.ts +++ b/test/interop/clients/js/harness.ts @@ -79,9 +79,10 @@ export const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve * Serve the prebuilt page on localhost, a secure context so WebTransport and WebCodecs are enabled. * * `interop.sh` builds the page into its run directory, so a concurrent run's rebuild cannot empty - * it mid-load; outside a harness run, it is vite's default `dist/`. + * it mid-load; outside a harness run, it is vite's default `dist/`. `files` serves each path from a + * file of its own. */ -export function serve(): { origin: string; stop: () => void } { +export function serve(files: Record = {}): { origin: string; stop: () => void } { const run = process.env.MOQ_TEST_RUN; const root = run ? join(run, "js-dist") : join(new URL(".", import.meta.url).pathname, "dist"); const server = Bun.serve({ @@ -89,7 +90,7 @@ export function serve(): { origin: string; stop: () => void } { async fetch(req) { let path = new URL(req.url).pathname; if (path === "/") path = "/index.html"; - const file = Bun.file(join(root, path)); + const file = Bun.file(files[path] ?? join(root, path)); if (await file.exists()) return new Response(file); return new Response(Bun.file(join(root, "index.html"))); // SPA fallback }, diff --git a/test/interop/clients/js/src/setup.ts b/test/interop/clients/js/src/setup.ts index 876b2056ddd..ddcd2b59cff 100644 --- a/test/interop/clients/js/src/setup.ts +++ b/test/interop/clients/js/src/setup.ts @@ -31,22 +31,32 @@ if (role === "finish-publish" || role === "finish-subscribe") { el.setAttribute("url", url); if (versions) el.setAttribute("versions", versions); el.setAttribute("name", broadcast); - // Chromium's --use-fake-device-for-media-stream feeds getUserMedia fake - // camera and microphone input. Audio is encoded lazily when a player asks. - el.setAttribute("source", "camera"); - // The microphone plays the driver's test tone. Keep Chromium's voice processing off it: that - // mixes stereo to mono and suppresses a steady tone as noise. - el.signals.run((effect) => { - const mic = effect.get(el.sources.audio); - if (mic && "constraints" in mic) { - mic.constraints.set({ - channelCount: 2, - echoCancellation: false, - noiseSuppression: false, - autoGainControl: false, - }); - } - }); + // Media is encoded lazily when a player asks. + const media = params.get("media"); + if (media) { + // The interop driver's file of the picture and test tone, decoded and looped rather than + // captured: a fake device splices the tone whenever the machine is short of CPU. + const response = await fetch(media); + if (!response.ok) throw new Error(`no publisher media at ${media}: ${response.status}`); + el.controls.source.set(new File([await response.blob()], "interop-media")); + } else { + // Chromium's --use-fake-device-for-media-stream feeds getUserMedia fake camera and microphone + // input, for the capture checks in media.ts. + el.setAttribute("source", "camera"); + // Keep Chromium's voice processing off the microphone: that mixes stereo to mono and + // suppresses a steady tone as noise. + el.signals.run((effect) => { + const mic = effect.get(el.sources.audio); + if (mic && "constraints" in mic) { + mic.constraints.set({ + channelCount: 2, + echoCancellation: false, + noiseSuppression: false, + autoGainControl: false, + }); + } + }); + } document.body.appendChild(el); // So the driver can wait for the session to close before closing the browser. watchResources(); diff --git a/test/interop/interop.sh b/test/interop/interop.sh index 901b197656d..88e1d2826bc 100755 --- a/test/interop/interop.sh +++ b/test/interop/interop.sh @@ -362,7 +362,7 @@ publisher_codecs() { } # phase_checked : whether tone.py also holds the tone's phase to the # decode's timestamps, which fails a misapplied pre-skip. Every publisher's tone starts at -# timestamp zero except the browser's, whose capture clock is not tied to its WAV. The native-JS +# timestamp zero except the browser's, which stamps its file from its own page clock. The native-JS # subscriber hands WebCodecs raw Opus without the OpusHead, as the player does, which keeps the # pre-skip for the player to trim, and its libav fallback stamps milliseconds. AAC waits on the # fMP4 importer applying the edit list that carries ffmpeg's 1024 priming samples @@ -981,14 +981,19 @@ ffmpeg_fmp4() { -f mp4 -movflags cmaf+separate_moof+delay_moov+skip_trailer -frag_duration 1000 - } -# The browser's microphone: as a WAV Chromium loops. Ten seconds is a whole number of -# cycles of any whole-Hz tone, so the loop is seamless. +# The browser's input: the picture and as a file it decodes and loops, H.264 beside 16-bit +# PCM so decoding it loses nothing. Not Chromium's fake devices: their timers skip a late 10 ms +# tick, and Web Audio then splices the tone without a timestamp gap whenever the runner is short +# of CPU. Ten seconds is a whole number of cycles of any whole-Hz tone and of video frames, so the +# loop is seamless. # shellcheck disable=SC2329 # reached from a function 'harness_spawn' invokes -tone_wav() { - local wav - wav="$HARNESS_RUN/tone-${1//,/-}.wav" - [[ -f "$wav" ]] || ffmpeg -hide_banner -loglevel error -f lavfi -i "$(tone_source "$1"):d=10" -c:a pcm_s16le "$wav" - printf '%s' "$wav" +tone_media() { + local media + media="$HARNESS_RUN/tone-${1//,/-}.mov" + [[ -f "$media" ]] || ffmpeg -hide_banner -loglevel error -f lavfi -i "testsrc=size=${SIZE}:rate=${FPS}:duration=10" \ + -f lavfi -i "$(tone_source "$1"):d=10" -c:v libx264 -profile:v baseline -preset ultrafast -pix_fmt yuv420p \ + -x264-params "keyint=${FPS}:min-keyint=${FPS}:scenecut=0" -c:a pcm_s16le "$media" + printf '%s' "$media" } # The publisher pipeline, run in a process group of its own by `harness_spawn` @@ -1031,14 +1036,13 @@ run_publisher() { exec bun "$INTEROP_COMPAT_CLIENT/client.ts" publish "$url" "$broadcast" ;; js) - # Headless Chromium encodes its own H.264 from a fake camera, and Opus from a - # fake microphone playing the tone, via WebCodecs (lazily, once a subscriber - # creates demand). exec, so the driver leads the group and `stop_publisher` - # waits for its own close. - local wav - wav=$(tone_wav "$tone") + # Headless Chromium decodes the tone file and encodes its own H.264 and Opus + # via WebCodecs (lazily, once a subscriber creates demand). exec, so the driver + # leads the group and `stop_publisher` waits for its own close. + local media + media=$(tone_media "$tone") cd "$CLIENTS/js" && exec bun driver.ts publish \ - --url "$url" --broadcast "$broadcast" --audio-file "$wav" ${FLAGS[@]+"${FLAGS[@]}"} + --url "$url" --broadcast "$broadcast" --media "$media" ${FLAGS[@]+"${FLAGS[@]}"} ;; *) echo "unknown publisher: $lang" >&2