Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
85248ee
chore(quest): claim quest/m2/compose/layout.md
kixelated Nov 12, 2026
5f85208
wip: the studio publishes a composition
kixelated Nov 12, 2026
6aef6a2
wip: docs, quests, watch
kixelated Nov 12, 2026
4f3999f
test(studio): a viewer plays a guest where the studio did
kixelated Nov 12, 2026
a84f075
fix(scene): a layer moved on its clock keeps its picture
kixelated Nov 12, 2026
4fb2e63
fix(scene): an offline guest keeps its composition entry, and a layer…
kixelated Nov 12, 2026
b2c6598
fix(scene): an offline broadcast source clears its picture, and the c…
kixelated Nov 12, 2026
6c47d61
fix(scene): a new offset wakes a layer, and a replaced video track ke…
kixelated Nov 12, 2026
1a0107c
feat(scene): the studio hears what is published independently
kixelated Nov 12, 2026
da69368
fix(studio): an audio scene's meter reads what Listen plays
kixelated Nov 12, 2026
548b232
Merge origin/main; the composition rides its broadcast's hang catalog
kixelated Nov 19, 2026
59c2af0
test(studio): composition tests hold under main's arrival-anchored clock
kixelated Nov 19, 2026
099b44e
Merge origin/main: a layer that keeps out of the way (#194)
kixelated Nov 19, 2026
0ec26f6
fix: review round 1 on the composition
kixelated Nov 19, 2026
6c7bc1e
fix(scene): a layer holds its sound as long as its publisher keeps it
kixelated Nov 19, 2026
7c48839
Merge origin/main: landscape and portrait at once (#200)
kixelated Nov 19, 2026
0ef0bfc
Merge remote-tracking branch 'origin/main' into quest/m2/compose/layout
kixelated Nov 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 158 additions & 76 deletions README.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion justfile
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ render-sw:
icd=("$mesa"/share/vulkan/icd.d/lvp_icd.*.json)
export VK_DRIVER_FILES="${icd[0]}"
cargo nextest run --locked -p moq-scene --lib --run-ignored only -E 'test(/^compositor::tests::/) | test(/^dark::tests::/) | test(/^browser::layers::tests::/)'
cargo nextest run --locked -p moq-studio --features openh264 --run-ignored only -E '(test(/^engine::tests::/) - test(=engine::tests::the_virtual_camera_reaches_a_pipewire_app)) | test(=setup::tests::times_every_canvas)'
cargo nextest run --locked -p moq-studio --features openh264 --run-ignored only -E '(test(/^engine::tests::/) - test(=engine::tests::the_virtual_camera_reaches_a_pipewire_app)) | test(=setup::tests::times_every_canvas) | test(=watch::tests::plays_a_guest_where_the_studio_did)'
cargo nextest run --locked -p moq-studio --features cef --run-ignored only -E 'binary(browser)'
sh/pipewire-session.sh cargo nextest run --locked -p moq-studio --features pipewire --run-ignored only -E 'test(=engine::tests::the_virtual_camera_reaches_a_pipewire_app)'

Expand Down
2 changes: 1 addition & 1 deletion quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ each encoding costs an encoder session and consumer GPUs have few, so the
[viewer-side composition](/quest/m2/compose/README.md) epic left m0 (now m2,
replanned 2026-11-08 around independently published sources layered over the
program). Overlays and animation render into the program here; that epic's
layout track carries only the layers of independently published sources.
composition lists only the independently published sources over it.

Changed 2026-11-09 with the maintainer (/quest-plan): the studio gets one
designed layout, a live picture with pages around it, in place of sixteen
Expand Down
9 changes: 5 additions & 4 deletions quest/m2/compose/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,10 @@ offset onto the composition's clock, and every independently published
source, local or guest, its own broadcast (rejected: a layer layout track in
the program's broadcast naming tracks; local sources as tracks of the
program's broadcast, the 2026-10-26 rule this reverses). The studio
prototypes the format first. See [the composition](/quest/m2/compose/layout.md).
prototypes the format first: `rs/moq-scene/src/composition.rs`, in a section
of its broadcast's hang catalog applied at each update's timestamp,
documented in the README's "Composition".

moq's side, in its own quest tree (m3): hang specifies the composition, and
`moq play` and `@moq/watch` composite it. moq's three layers quests were
planned for a layer layout track and need replanning there.
moq's side, in its own quest tree (m3, `quest/m3/layers/`): hang specifies
the composition, and `moq play` and `@moq/watch` composite it.

13 changes: 7 additions & 6 deletions quest/m2/compose/controls.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

## Goal

A viewer watching a studio broadcast can switch off any layer the layout
marks toggleable (a facecam, a screen share), and stops fetching its track.
The layout's per-layer priorities steer delivery, so under congestion the
least important sources degrade first. The creator sets both per source in
A viewer watching a studio broadcast can switch off any source the
composition marks toggleable (a facecam, a screen share), and stops fetching
its broadcast. The composition's per-source priorities steer delivery, so
under congestion the least important sources degrade first. The creator sets both per source in
the studio.

## Plan
Expand All @@ -26,5 +26,6 @@ Map layer priority onto the subscription priorities moq-lite already has
(`moq/doc/concept/moq-lite.md`), rather than a second mechanism. A toggled-off
layer is unsubscribed, so it costs the viewer and the relay nothing. The
studio keeps encoding it, so toggling it back on can reach its recent past.
Subscriptions are per track, not per layer: a source shown by several layers
stays subscribed while any of them is on, at the highest priority among them.
The composition carries `priority` and `toggle` on each source's entry
(`rs/moq-scene/src/composition.rs`), not on each picture, so a source shown by
several pictures is one subscription, on or off as a whole.
5 changes: 2 additions & 3 deletions quest/m2/compose/hang-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ quests (rejected: "both quests", unnamed, which left out `@moq/watch`).

Decided 2026-11-12 (maintainer, /quest-unblock): hang specifies a
composition, a broadcast listing a catalog per source, rather than a layer
layout track; see [the composition](/quest/m2/compose/layout.md), whose
prototype moq's spec starts from. moq's three quests below were planned for a
layout track. The studio's prototype, which moq's spec starts from, is
`rs/moq-scene/src/composition.rs`, documented in the README's "Composition". moq's three quests below were planned for a
layer layout track and are replanned in moq's tree.

## Waiting
Expand All @@ -39,5 +39,4 @@ layer layout track and are replanned in moq's tree.

## Required

- [The studio publishes a composition](/quest/m2/compose/layout.md) - the prototype hang's spec starts from
- [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries
53 changes: 0 additions & 53 deletions quest/m2/compose/layout.md

This file was deleted.

10 changes: 3 additions & 7 deletions quest/m2/compose/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,10 @@ too and moq never depends on this repository. Settle its name in this quest
and document it in `README.md`.

It flattens the composition the studio publishes
([the composition](/quest/m2/compose/layout.md)); until that lands, it
flattens the provisional `live.kixel.studio.layers` track (`rs/moq-scene/src/layout.rs`,
and the README's "Layer layout").
(`rs/moq-scene/src/composition.rs`, and the README's "Composition"), placing
each listed broadcast on the program's clock at its offset as the
`moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`) does.

## Required

- [Destinations](/quest/m0/destinations.md) - the egress library the server pushes RTMP and WHIP through

## Related

- [The studio publishes a composition](/quest/m2/compose/layout.md) - the format it flattens
6 changes: 4 additions & 2 deletions rs/moq-scene/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ humantime-serde = { workspace = true }
image = { version = "0.25", default-features = false, features = ["png", "jpeg"] }
kio = { workspace = true, features = ["runtime"] }
moq-audio = { workspace = true }
# Writes the composition broadcast's catalog, each update at its own timestamp.
moq-json = { workspace = true }
moq-mux = { workspace = true }
moq-net = { workspace = true }
# `render` is the compositor's whole input path: it turns every decoded or
Expand Down Expand Up @@ -102,8 +104,8 @@ x11rb = { version = "0.13", features = ["dri3"] }
wayland-client = "0.31"

[dev-dependencies]
# Reads the layer layout track back as a player would.
moq-json = { workspace = true, features = ["async"] }
# Reads the composition broadcast's catalog back as a player would.
hang = { workspace = true, features = ["async"] }
# An encoder on every machine, for the source track and replay tests.
moq-video = { workspace = true, features = ["openh264"] }
tokio = { workspace = true, features = ["macros", "rt", "test-util"] }
Expand Down
66 changes: 40 additions & 26 deletions rs/moq-scene/src/collection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,15 @@ pub struct Collection {
/// wherever the source is placed. A source without an entry is unfiltered.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub video: BTreeMap<String, Video>,
/// The sources published independently, by name: each one's own track
/// goes to the relay beside the program, and the
/// [layer layout](crate::layout) says where viewers show it over the
/// The sources published independently, by name: each reaches viewers
/// as a broadcast of its own, a local source's track in one the studio
/// publishes and a broadcast source as its publisher's, and the
/// [composition](crate::composition) says where viewers show it over the
/// program and how they play its sound. The program and every audio
/// scene's mix leave it out. Only a source with a track of its own can
/// be, and not one with its background removed, since its track would
/// show the background. Wherever it is placed, it takes only what every
/// scene's encoded mix leave it out, but the mix people in the studio
/// hear holds it ([`Sources::listened`](crate::source::Sources::listened)). Only a broadcast or a source with a track of
/// its own can be, and not one with its background removed, since its
/// track would show the background. Wherever it is placed, it takes only what every
/// player can reproduce, and no scene that another scene draws holds it.
#[serde(default, skip_serializing_if = "BTreeSet::is_empty")]
pub published: BTreeSet<String>,
Expand Down Expand Up @@ -580,11 +582,11 @@ impl Collection {
"`{name}` draws scene `{scene}`, which is not a scene"
)));
}
if let Source::Layer { broadcast, .. } = source
&& !matches!(self.sources.get(broadcast), Some(Source::Broadcast { .. }))
if let Source::Layer { clock, .. } = source
&& !matches!(self.sources.get(clock), Some(Source::Broadcast { .. }))
{
return Err(Error::Invalid(format!(
"`{name}` plays on `{broadcast}`, which is not a broadcast source"
"`{name}` plays on `{clock}`, which is not a broadcast source"
)));
}
if let Source::Role { source: Some(source) } = source {
Expand Down Expand Up @@ -649,10 +651,17 @@ impl Collection {
"`{name}` is published, but there is no such source"
)));
};
if !source.publishes() {
if !source.independent() {
return Err(Error::Invalid(format!(
"`{name}` cannot be published independently: only a camera, display, window, microphone, \
system audio, color, image, media, or replay source has a track of its own"
"`{name}` cannot be published independently: only a broadcast, or a camera, display, window, \
microphone, system audio, color, image, media, or replay source with a track of its own, can be"
)));
}
let segment = !name.contains('/') && moq_net::Path::new(name).is_ok() && name != "." && name != "..";
if source.publishes() && !segment {
return Err(Error::Invalid(format!(
"`{name}` is published independently, so its name is one segment of its broadcast's path: no \
`/`, not `.` or `..`, and a `\\` only before another"
)));
}
if self
Expand Down Expand Up @@ -778,10 +787,10 @@ impl Collection {
}

/// Refuse an item showing a source published independently that the
/// layer layout cannot carry: one blended or filtered past opacity, which
/// composition cannot carry: one blended or filtered past opacity, which
/// not every player can reproduce, one keeping clear, which moves where
/// no player sees, or one in a scene another scene draws, which the
/// layout cannot place.
/// composition cannot place.
fn validate_published(&self) -> Result<(), Error> {
let nested: BTreeSet<&str> = self
.sources
Expand Down Expand Up @@ -1637,7 +1646,7 @@ fn is_true(value: &bool) -> bool {
/// What a source or item is drawn through, in order.
///
/// Every filter is a pure function of the picture, so every host that composes
/// a scene with moq-scene draws it the same way, and it travels in a layout as
/// a scene with moq-scene draws it the same way, and it travels in a composition as
/// its name and settings. The CSS ones mean what the CSS Filter Effects
/// function of the same name does, in the gamma-encoded sRGB the canvas holds,
/// with lengths in canvas pixels. A filter naming another source uses that
Expand Down Expand Up @@ -2263,18 +2272,23 @@ pub enum Source {
/// same frame. A scene drawing it cannot go on program, and the top layer
/// cannot draw it, since either would draw the program into itself.
Program,
/// A source a studio broadcast publishes independently, played from the
/// broadcast the [`Source::Broadcast`] called `broadcast` plays, on that
/// source's playout clock, so it lands with the program as the
/// [layer layout](crate::layout) places it. Never in a scene file: only
/// a player of the layout makes one.
/// A broadcast a [composition](crate::composition) lists, played on the
/// playout clock of the [`Source::Broadcast`] called `clock`, its
/// timestamps moved `offset` onto that clock's, so it lands with the
/// program as the composition places it. Never in a scene file: only a
/// player of the composition makes one.
#[serde(skip)]
Layer {
/// The broadcast source whose broadcast and clock it plays on.
broadcast: String,
/// Its entry under the broadcast catalog's
/// [`SECTION`](crate::publish::SECTION) `sources`.
source: String,
/// The broadcast source whose clock it plays on: the program.
clock: String,
/// Its broadcast's path.
path: String,
/// The stream `offset` is for, or `None` for whichever serves the
/// path: another stream shows nothing, unless a relay minted it.
instance: Option<moq_net::Instance>,
/// Microseconds added to its timestamps to put them on `clock`'s
/// timeline. A change plays on without opening the broadcast again.
offset: i64,
},
}

Expand Down Expand Up @@ -3474,7 +3488,7 @@ items = [
collection.scenes[0].items[1].rect = rect;
collection.validate().unwrap();

// A player of the layer layout places a published source where the
// A player of the composition places a published source where the
// file does, so it cannot move.
let mut published = Collection::new();
published.sources.insert("cam".into(), Source::Camera { id: None });
Expand Down
Loading