diff --git a/quest/m3/README.md b/quest/m3/README.md index 8b16f273dbc..3f662ee3dbb 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -188,3 +188,4 @@ federation) land. - [An audio encoder flushes without ending its track](/quest/m3/audio-encode-flush.md) - a clip's last partial frame is heard while the source stays live - [Padding probes from the browser](/quest/m3/probe-padding.md) - parked: the maintainer doubts padding is the right way for a browser to probe - [JS rung disable](/quest/m3/probe-rung-disable-js.md) - `@moq/publish` applies the native rung rule, recovering through padding; parked with it +- [Layered broadcasts](/quest/m3/layers/README.md) - hang specifies a layer layout track, and `moq play` and `@moq/watch` composite layers over the main picture diff --git a/quest/m3/layers/README.md b/quest/m3/layers/README.md new file mode 100644 index 00000000000..c6356f15a90 --- /dev/null +++ b/quest/m3/layers/README.md @@ -0,0 +1,36 @@ +# Layered broadcasts + +## Goal + +hang specifies a layer layout track: a broadcast's main picture and sound plus +other tracks layered over it, each placed, timed, and mixed on the media +clock. `moq play` and `@moq/watch` composite those layers over the main +picture and mix their audio, so a moq-dev/studio broadcast plays as its +creator composed it, while sources published on their own tracks stay +separately fetchable and toggleable. + +This README's own work is the end-to-end proof: one fixture broadcast with a +picture layer and an audio layer plays in `moq play` and `@moq/watch` with the +same placement and timing, within a stated tolerance, and with the layer's +sound mixed in. + +## Plan + +Decided 2026-11-08 (maintainer, moq-dev/studio /quest-plan): + +- The studio publishes the program plus each independently published source + as its own track, and players composite those sources over the program. + Rejected: viewers rebuilding the whole scene from every source's track, + which would take more encoder sessions than consumer GPUs have. +- hang specifies the layout and each player implements it with its own code. + The studio's compositor (moq-scene) stays in moq-dev/studio, since moq + never depends on that repository, so parity comes from the shared fixture, + not shared code. Rejected: moving moq-scene into moq. +- The studio prototypes the format first (`moq-studio --watch`, reading a + provisional `dev.moq.studio.layers` track), and the spec starts from what + that prototype settles (rejected: drafting the spec in parallel). +- Players mix layers' audio in, at the gain and mute the layout carries. +- Each layer carries a priority and whether a viewer may toggle it. A player + maps priority onto moq-lite's subscription priorities and unsubscribes a + toggled-off layer. +- m3: moq-dev/studio is the named consumer, and its side sits in its m2. diff --git a/quest/m3/layers/play.md b/quest/m3/layers/play.md new file mode 100644 index 00000000000..5761e0e0cbd --- /dev/null +++ b/quest/m3/layers/play.md @@ -0,0 +1,19 @@ +# [M] moq play composites layers + +## Goal + +`moq play` plays a broadcast that carries a layer layout track: it composites +each visible layer over the main picture where and when the layout says, and +mixes each audio layer in at its gain and mute. A broadcast without one plays +as today. + +## Plan + +Fetch only visible layers, at the layout's priorities through moq-lite's +subscription priorities. Prove placement and timing against the epic's +fixture. moq-dev/studio's `hang-layers` quest deletes its watch prototype +once this lands. + +## Required + +- [hang specifies a layer layout track](/quest/m3/layers/spec.md) - what `moq play` composites diff --git a/quest/m3/layers/spec.md b/quest/m3/layers/spec.md new file mode 100644 index 00000000000..da1074e7ae8 --- /dev/null +++ b/quest/m3/layers/spec.md @@ -0,0 +1,23 @@ +# [M] hang specifies a layer layout track + +## Goal + +`drafts/draft-lcurley-moq-hang.md`, `rs/hang`, and `js/hang` define a layer +layout track: a versioned JSON track in snapshot mode on the media clock that +names each layer's track and gives its transform, crop, z-order, opacity, +visibility, keyframes, gain and mute, priority, and whether a viewer may +toggle it. + +## Plan + +Start from the format moq-dev/studio's watch prototype settled, and keep only +what every player can reproduce. A consumer refuses an unknown version loudly +and plays the main picture alone. Layers name tracks and catalog references, +never files. + +## Waiting + +- moq-dev/studio: its `moq-studio --watch` prototype plays program plus + layers from the provisional `dev.moq.studio.layers` track, planned as + `quest/m2/compose/watch.md` there. Check: that quest's file is gone from + moq-dev/studio's `main` because its PR merged. diff --git a/quest/m3/layers/watch.md b/quest/m3/layers/watch.md new file mode 100644 index 00000000000..f9fec74a15c --- /dev/null +++ b/quest/m3/layers/watch.md @@ -0,0 +1,20 @@ +# [M] The watch player composites layers + +## Goal + +`@moq/watch` plays a broadcast that carries a layer layout track: it +composites each visible layer over the main picture where and when the layout +says, mixes each audio layer in at its gain and mute, and lets a viewer +toggle each layer the layout marks toggleable. A broadcast without one plays +as today. + +## Plan + +Fetch only visible layers, at the layout's priorities through moq-lite's +subscription priorities, and unsubscribe a toggled-off layer. Prove placement +and timing against the epic's fixture, matching `moq play` within its +tolerance. + +## Required + +- [hang specifies a layer layout track](/quest/m3/layers/spec.md) - what `@moq/watch` composites