Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,6 @@ Binding durations wait on Duration support in the generated C.
- [Moxygen minimal interop](/quest/m1/smoke-moxygen.md) - waiting on Moxygen to send a registered filter type, then the smoke lane stops hiding its failure
- [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause
- [Broadcast lifecycle](/quest/m1/broadcast-lifecycle/README.md) - broadcasts are online or offline with no finished state in moq-net, finished-broadcast retention is a fixed moq-mux policy, and each linger timer has one meaning
- [JS interop tone](/quest/m1/interop-js-tone.md) - the `js ->` cells pass the audio check on every draft again, after the 2026-10-23 regression
- [fMP4 edit lists](/quest/m1/fmp4-edit-lists.md) - the fMP4 importer applies edit lists to frame timestamps, so the interop AAC tone passes the phase check
- [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break says how to migrate
- [moq-quic's API improves on quinn's](/quest/m1/quic-api/README.md) - moq-quic and both QUIC drivers ship one mirrored API: `send::`/`recv::` namespaces, one error per direction, consuming terminal operations, plain configs, no quinn baggage
Expand Down
27 changes: 0 additions & 27 deletions quest/m1/interop-js-tone.md

This file was deleted.

11 changes: 7 additions & 4 deletions test/interop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,11 @@ decodes it. Python and C++ send a mono 440 Hz tone; the rest send 440 Hz left an
660 Hz right, so a channel swap fails too. The Rust CLI imports it from ffmpeg in
fragmented MP4, as both an Opus and an AAC rendition, so AAC import and decode cross
every subscriber (the wire-compat lanes send Opus alone; see
[Released wire compatibility](#released-wire-compatibility)). The browser encodes its fake microphone playing the tone from a WAV
file, and the Python, Go, and C++ clients encode it through `moq-ffi` at a 2.5 ms
[Released wire compatibility](#released-wire-compatibility)). The browser decodes the picture
and the tone from a looping file through `<moq-publish>`'s file source and encodes both, rather
than capturing Chromium's fake camera and microphone: those run on timers that skip a late tick,
so a runner short of CPU splices the captured tone with no timestamp gap. The Python, Go, and
C++ clients encode it through `moq-ffi` at a 2.5 ms
frame duration, so the matrix covers the FFI audio path with a non-default codec config.
Each subscriber decodes every audio rendition with what it already has: moq-audio for Rust (the
`interop-audio` example, since the CLI has no PCM output), `moq-ffi` for Python,
Expand All @@ -34,8 +37,8 @@ most of its energy by a Goertzel filter. A garbled decode (wrong channel order o
sample rate), silence, a timestamp gap, or a rendition left undecoded fails the cell.
Each publisher's tone starts at timestamp zero, so `--phase` also holds every Opus
rendition's tone to its timestamps within a sample, which fails a decoder that keeps
or drops the wrong pre-skip. It skips the browser publisher, whose capture clock is
not tied to its WAV, and the native-JS subscriber, which leaves the pre-skip to the
or drops the wrong pre-skip. It skips the browser publisher, which stamps its file
from its own page clock, and the native-JS subscriber, which leaves the pre-skip to the
player as WebCodecs does and stamps milliseconds. AAC is skipped until the fMP4
importer applies the edit list holding ffmpeg's priming
([quest](/quest/m1/fmp4-edit-lists.md)). The browser plays each rendition in a page
Expand Down
33 changes: 17 additions & 16 deletions test/interop/clients/js/driver.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
/**
* Drives a headless Chromium against the vite-built page (see `serve`) for the interop matrix. publish
* streams a fake camera and a microphone playing `--audio-file` until killed; subscribe verifies,
* streams the picture and tone decoded from `--media`, looping, until killed; subscribe verifies,
* once per `--codecs` audio rendition with the player pinned to it, rendered playback, that each
* audio channel the player outputs carries its `--tone`, and pause/resume.
*
* The publishers on the other side of the matrix are separate processes with no readiness signal,
* so this driver tolerates a slow announcement with one reload. `media.ts` is the strict path: it
* waits on an explicit publisher-ready state and never reloads.
*
* bun driver.ts publish --url http://127.0.0.1:4443 --broadcast b.hang --audio-file tone.wav [--version V]
* bun driver.ts publish --url http://127.0.0.1:4443 --broadcast b.hang --media tone.mov [--version V]
* bun driver.ts subscribe --url http://127.0.0.1:4443 --broadcast b.hang --timeout 20 --tone 440,660 \
* --codecs opus,mp4a [--version V]
*
Expand Down Expand Up @@ -66,7 +66,7 @@ const { positionals, values } = parseArgs({
"finish-ack": { type: "string" },
frames: { type: "string", default: "10" },
timeout: { type: "string", default: "20" },
"audio-file": { type: "string" },
media: { type: "string" },
tone: { type: "string" },
codecs: { type: "string" },
},
Expand All @@ -77,7 +77,7 @@ const url = values.url;
const broadcast = values.broadcast;
const timeoutMs = Number.parseFloat(values.timeout ?? "20") * 1000;
const finishing = values.finish || values["finish-ack"] !== undefined;
const audioFile = values["audio-file"];
const media = values.media;
const tones = (values.tone ?? "").split(",").filter(Boolean).map(Number);
// Each audio rendition by codec: the catalog's up to its first `.`.
const codecs = (values.codecs ?? "").split(",").filter(Boolean);
Expand All @@ -87,11 +87,11 @@ if (
!broadcast ||
!Number.isFinite(timeoutMs) ||
timeoutMs <= 0 ||
(!finishing && role === "publish" && !audioFile) ||
(!finishing && role === "publish" && !media) ||
(!finishing && role === "subscribe" && (!tones.length || tones.some((hz) => !(hz > 0)) || !codecs.length))
) {
console.error(
"usage: driver.ts publish --url U --broadcast B (--audio-file WAV | --finish-ack FIFO) | subscribe --url U --broadcast B (--tone HZ,... --codecs CODEC,... | --finish [--frames N]) [--version V] [--timeout S>0]",
"usage: driver.ts publish --url U --broadcast B (--media FILE | --finish-ack FIFO) | subscribe --url U --broadcast B (--tone HZ,... --codecs CODEC,... | --finish [--frames N]) [--version V] [--timeout S>0]",
);
process.exit(2);
}
Expand Down Expand Up @@ -134,14 +134,10 @@ async function waitForStablePause(page: Page, errors: BrowserErrors, deadline: n
throw new Error(`playback did not stop after pause: ${JSON.stringify(previous)}`);
}

const server = serve();
const browser = await launch([
"--use-fake-device-for-media-stream",
"--use-fake-ui-for-media-stream",
"--autoplay-policy=no-user-gesture-required",
// The fake microphone loops this file instead of its default beeps.
...(audioFile ? [`--use-file-for-fake-audio-capture=${audioFile}`] : []),
]);
// The page fetches the publisher's file from here.
const MEDIA_PATH = "/interop-media";
const server = serve(media ? { [MEDIA_PATH]: media } : {});
const browser = await launch(["--autoplay-policy=no-user-gesture-required"]);

/**
* Play the broadcast in a page of its own, pinned to its `codec` audio rendition: rendered video, the
Expand Down Expand Up @@ -321,10 +317,15 @@ try {
} else {
await open(
browser,
pageUrl(server.origin, role, { url, broadcast, ...(values.version ? { versions: values.version } : {}) }),
pageUrl(server.origin, role, {
url,
broadcast,
...(media ? { media: MEDIA_PATH } : {}),
...(values.version ? { versions: values.version } : {}),
}),
role,
);
console.error(`publishing ${broadcast} (fake camera + ${audioFile} as the microphone) to ${url}`);
console.error(`publishing ${broadcast} (${media}) to ${url}`);
// Stream until the orchestrator stops us.
await new Promise((resolve) => process.once("SIGTERM", resolve));
code = 0;
Expand Down
7 changes: 4 additions & 3 deletions test/interop/clients/js/harness.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,17 +79,18 @@ export const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve
* Serve the prebuilt page on localhost, a secure context so WebTransport and WebCodecs are enabled.
*
* `interop.sh` builds the page into its run directory, so a concurrent run's rebuild cannot empty
* it mid-load; outside a harness run, it is vite's default `dist/`.
* it mid-load; outside a harness run, it is vite's default `dist/`. `files` serves each path from a
* file of its own.
*/
export function serve(): { origin: string; stop: () => void } {
export function serve(files: Record<string, string> = {}): { origin: string; stop: () => void } {
const run = process.env.MOQ_TEST_RUN;
const root = run ? join(run, "js-dist") : join(new URL(".", import.meta.url).pathname, "dist");
const server = Bun.serve({
port: 0,
async fetch(req) {
let path = new URL(req.url).pathname;
if (path === "/") path = "/index.html";
const file = Bun.file(join(root, path));
const file = Bun.file(files[path] ?? join(root, path));
if (await file.exists()) return new Response(file);
return new Response(Bun.file(join(root, "index.html"))); // SPA fallback
},
Expand Down
42 changes: 26 additions & 16 deletions test/interop/clients/js/src/setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,22 +31,32 @@ if (role === "finish-publish" || role === "finish-subscribe") {
el.setAttribute("url", url);
if (versions) el.setAttribute("versions", versions);
el.setAttribute("name", broadcast);
// Chromium's --use-fake-device-for-media-stream feeds getUserMedia fake
// camera and microphone input. Audio is encoded lazily when a player asks.
el.setAttribute("source", "camera");
// The microphone plays the driver's test tone. Keep Chromium's voice processing off it: that
// mixes stereo to mono and suppresses a steady tone as noise.
el.signals.run((effect) => {
const mic = effect.get(el.sources.audio);
if (mic && "constraints" in mic) {
mic.constraints.set({
channelCount: 2,
echoCancellation: false,
noiseSuppression: false,
autoGainControl: false,
});
}
});
// Media is encoded lazily when a player asks.
const media = params.get("media");
if (media) {
// The interop driver's file of the picture and test tone, decoded and looped rather than
// captured: a fake device splices the tone whenever the machine is short of CPU.
const response = await fetch(media);
if (!response.ok) throw new Error(`no publisher media at ${media}: ${response.status}`);
el.controls.source.set(new File([await response.blob()], "interop-media"));
} else {
// Chromium's --use-fake-device-for-media-stream feeds getUserMedia fake camera and microphone
// input, for the capture checks in media.ts.
el.setAttribute("source", "camera");
// Keep Chromium's voice processing off the microphone: that mixes stereo to mono and
// suppresses a steady tone as noise.
el.signals.run((effect) => {
const mic = effect.get(el.sources.audio);
if (mic && "constraints" in mic) {
mic.constraints.set({
channelCount: 2,
echoCancellation: false,
noiseSuppression: false,
autoGainControl: false,
});
}
});
}
document.body.appendChild(el);
// So the driver can wait for the session to close before closing the browser.
watchResources();
Expand Down
34 changes: 19 additions & 15 deletions test/interop/interop.sh
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ publisher_codecs() {
}
# phase_checked <round> <subscriber> <codec>: whether tone.py also holds the tone's phase to the
# decode's timestamps, which fails a misapplied pre-skip. Every publisher's tone starts at
# timestamp zero except the browser's, whose capture clock is not tied to its WAV. The native-JS
# timestamp zero except the browser's, which stamps its file from its own page clock. The native-JS
# subscriber hands WebCodecs raw Opus without the OpusHead, as the player does, which keeps the
# pre-skip for the player to trim, and its libav fallback stamps milliseconds. AAC waits on the
# fMP4 importer applying the edit list that carries ffmpeg's 1024 priming samples
Expand Down Expand Up @@ -981,14 +981,19 @@ ffmpeg_fmp4() {
-f mp4 -movflags cmaf+separate_moof+delay_moov+skip_trailer -frag_duration 1000 -
}

# The browser's microphone: <tone> as a WAV Chromium loops. Ten seconds is a whole number of
# cycles of any whole-Hz tone, so the loop is seamless.
# The browser's input: the picture and <tone> as a file it decodes and loops, H.264 beside 16-bit
# PCM so decoding it loses nothing. Not Chromium's fake devices: their timers skip a late 10 ms
# tick, and Web Audio then splices the tone without a timestamp gap whenever the runner is short
# of CPU. Ten seconds is a whole number of cycles of any whole-Hz tone and of video frames, so the
# loop is seamless.
# shellcheck disable=SC2329 # reached from a function 'harness_spawn' invokes
tone_wav() {
local wav
wav="$HARNESS_RUN/tone-${1//,/-}.wav"
[[ -f "$wav" ]] || ffmpeg -hide_banner -loglevel error -f lavfi -i "$(tone_source "$1"):d=10" -c:a pcm_s16le "$wav"
printf '%s' "$wav"
tone_media() {
local media
media="$HARNESS_RUN/tone-${1//,/-}.mov"
[[ -f "$media" ]] || ffmpeg -hide_banner -loglevel error -f lavfi -i "testsrc=size=${SIZE}:rate=${FPS}:duration=10" \
-f lavfi -i "$(tone_source "$1"):d=10" -c:v libx264 -profile:v baseline -preset ultrafast -pix_fmt yuv420p \
-x264-params "keyint=${FPS}:min-keyint=${FPS}:scenecut=0" -c:a pcm_s16le "$media"
printf '%s' "$media"
}

# The publisher pipeline, run in a process group of its own by `harness_spawn`
Expand Down Expand Up @@ -1031,14 +1036,13 @@ run_publisher() {
exec bun "$INTEROP_COMPAT_CLIENT/client.ts" publish "$url" "$broadcast"
;;
js)
# Headless Chromium encodes its own H.264 from a fake camera, and Opus from a
# fake microphone playing the tone, via WebCodecs (lazily, once a subscriber
# creates demand). exec, so the driver leads the group and `stop_publisher`
# waits for its own close.
local wav
wav=$(tone_wav "$tone")
# Headless Chromium decodes the tone file and encodes its own H.264 and Opus
# via WebCodecs (lazily, once a subscriber creates demand). exec, so the driver
# leads the group and `stop_publisher` waits for its own close.
local media
media=$(tone_media "$tone")
cd "$CLIENTS/js" && exec bun driver.ts publish \
--url "$url" --broadcast "$broadcast" --audio-file "$wav" ${FLAGS[@]+"${FLAGS[@]}"}
--url "$url" --broadcast "$broadcast" --media "$media" ${FLAGS[@]+"${FLAGS[@]}"}
;;
*)
echo "unknown publisher: $lang" >&2
Expand Down
Loading