From dcd86c976ffd48c44ee30b5b9c16eaf3fb1a9b13 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 17:17:23 -0800 Subject: [PATCH 1/2] quest: plan the VR demo that reads the catalog's spatial fields Co-Authored-By: Claude Opus 5.5 --- quest/m1/vr-demo.md | 50 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 quest/m1/vr-demo.md diff --git a/quest/m1/vr-demo.md b/quest/m1/vr-demo.md new file mode 100644 index 00000000000..953c3a83b36 --- /dev/null +++ b/quest/m1/vr-demo.md @@ -0,0 +1,50 @@ +# [M] A VR demo reads the catalog's spatial fields, tested end to end + +## Goal + +`demo/vr` plays a broadcast in a WebXR headset (or a desktop tab) and picks +its layout from the hang catalog: video `projection` and its `fov` choose a +flat screen or an equirectangular sphere, `stereo` crops each eye from a +left-right or top-bottom frame, and an audio `layout` of `ambisonic` decodes +the sound field around the head. A harness imports a small VR180 left-right +fixture with first-order ambisonic Opus, publishes it through a relay, and +asserts the demo sees and applies the right projection, fov, stereo, and +layout. + +This gives the fields that +[moq-dev/moq#5543](https://github.com/moq-dev/moq/pull/5543) added and +[moq-dev/moq#7308](https://github.com/moq-dev/moq/pull/7308) fills on import +their first reader, so they ship with the cut instead of being deleted. + +## Plan + +Decided 2026-11-14 (maintainer): keep the fields and finish +[moq-dev/moq#6373](https://github.com/moq-dev/moq/pull/6373) so they are +exercised (rejected: deleting them before the cut, as the feature review's +`catalog-vr-fields` quest in +[moq-dev/moq#8764](https://github.com/moq-dev/moq/pull/8764) recommended). +✅ m1, since a demo and a harness break nothing (rejected: m2, which holds +breaking work). + +- Build on @nuts-rice's commits from `nuts-rice/moq:webxr_vr_demo`, keeping + their authorship; the PR credits them and links #6373. Maintainer edits + are off on their fork, so the work lands from a moq-dev branch. +- Drop #6373's catalog plumbing that #7308 made redundant: + `moq_mux::catalog::Spatial`, its producer and track hunks, the `import + --projection/--stereo/--audio-layout` flags, and the `pub` on the hang + `check` helpers. The importers fill the fields from MP4 `st3d`/`sv3d`, + Matroska, and Opus family 2. Adding CLI or catalog API needs the + maintainer first. +- `@moq/watch` drops every Opus `description`, so Chromium refuses Opus past + stereo ("description is required for multi-channel Opus"). Passing the + catalog's OpusHead, as the Rust importers intend, is part of this quest, + since the ambisonic decode needs it. +- The harness publishes the fixture through `moq import fmp4`, so it proves + the importer's side too, and runs nightly in `interop.yml` beside the + portrait harness, outside the merge gate. + +Public API: none (a behavior fix in `@moq/watch`). Wire: none. + +## Related + +- [fMP4 import fills the ambisonic layout](/quest/m5/mux-vr-ambisonic-fmp4.md) - the harness's fixture exercises that importer path From 3803157a7a3d1e0dd2e0c1b686bddfb28b4da810 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 14 Nov 2026 17:26:58 -0800 Subject: [PATCH 2/2] quest: the VR demo's Opus fix passes the OpusHead to the probe and keeps one pre-skip owner Co-Authored-By: Claude Opus 5.5 --- quest/m1/vr-demo.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/quest/m1/vr-demo.md b/quest/m1/vr-demo.md index 953c3a83b36..8b69a945cce 100644 --- a/quest/m1/vr-demo.md +++ b/quest/m1/vr-demo.md @@ -37,8 +37,12 @@ breaking work). maintainer first. - `@moq/watch` drops every Opus `description`, so Chromium refuses Opus past stereo ("description is required for multi-channel Opus"). Passing the - catalog's OpusHead, as the Rust importers intend, is part of this quest, - since the ambisonic decode needs it. + catalog's OpusHead to both the support probe and the decoder, as the Rust + importers intend, is part of this quest, since the ambisonic decode needs + it. Chromium then trims the header's pre-skip itself, again after every + flush (measured: 648 of 960 frames), so WebCodecs gets a copy with the + pre-skip zeroed and the decoder's `Terminal` stays its one owner, gap + resumes included. - The harness publishes the fixture through `moq import fmp4`, so it proves the importer's side too, and runs nightly in `interop.yml` beside the portrait harness, outside the merge gate.