From d9a92bf4990409c3311b652429ce70e98d209859 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 20 Nov 2026 11:08:23 -0800 Subject: [PATCH 1/3] chore(quest): plan the studio's moq follow-ups MP4 chapters, a catalog producer stamped by its caller, decode consumers that take a priority, and players that pick a rendition by orientation. AGENTS.md: code outside moq-net never branches on the protocol version. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 1 + quest/m1/catalog-timestamp.md | 20 ++++++++++++++++++++ quest/m1/decode-priority.md | 20 ++++++++++++++++++++ quest/m1/mp4-chapters.md | 26 ++++++++++++++++++++++++++ quest/m1/orientation.md | 29 +++++++++++++++++++++++++++++ 5 files changed, 96 insertions(+) create mode 100644 quest/m1/catalog-timestamp.md create mode 100644 quest/m1/decode-priority.md create mode 100644 quest/m1/mp4-chapters.md create mode 100644 quest/m1/orientation.md diff --git a/AGENTS.md b/AGENTS.md index 29bdc9b9c0..1ef037ccb9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,6 +28,7 @@ The API is the most important thing to get right: a bad shape costs a breaking c - Replace a callback parameter with a handle, an event, or a Producer/Consumer split. - Nest related API, configuration, and output fields under their shared role. - Mirror API names across Rust, JS, and the bindings. +- Code outside `moq-net` may report the negotiated protocol version but never branches on it: a behavior that differs by version is moq-net's to hide, so change its API instead. - Keep package boundaries modular and reusable; prefer a module in an existing package over a one-off package. - Never mark a special case inside a value: not a huge number like `u64::MAX` (it is the longest varint), and not a reserved prefix or character in a string. Use 0, a separate message or type, a typed field, or a boolean. The one exception is a path segment's leading `.`, which hides the path from announce discovery. - Enforce a policy where the thing is created, not where it is checked: the signer bounds a token's lifetime, and the verifier accepts what a valid token carries. Don't add `allow-*` opt-out flags; move or generalize the rule. diff --git a/quest/m1/catalog-timestamp.md b/quest/m1/catalog-timestamp.md new file mode 100644 index 0000000000..e5a40e0104 --- /dev/null +++ b/quest/m1/catalog-timestamp.md @@ -0,0 +1,20 @@ +# [S] The catalog producer publishes at the caller's timestamp + +## Goal + +moq-mux's catalog producer (`catalog::Producer`) can publish an update +stamped with a media timestamp its caller gives, rather than the time it is +written, so a publisher whose catalog changes at a frame (a studio's +composition switching layout) gets hang and MSF output from one producer. +Updates without a timestamp behave as today. + +## Plan + +[moq-dev/studio#146](https://github.com/moq-dev/studio/pull/146) publishes its +composition through a hand-rolled hang-only `moq_json::snapshot::Producer`, +calling `Timed::new` itself, because the moq-mux producer stamps each emit on +its own clock; it cannot carry MSF beside hang until this lands. + +Recommended: the timestamp is on the producer's clock, the same one its +`sync.clock` advertises, and a timestamp older than the last published one is +an error rather than reordered. diff --git a/quest/m1/decode-priority.md b/quest/m1/decode-priority.md new file mode 100644 index 0000000000..3c04c3ca3f --- /dev/null +++ b/quest/m1/decode-priority.md @@ -0,0 +1,20 @@ +# [S] Decode consumers take a subscription priority + +## Goal + +`moq_video::decode::Options` and `moq_audio::decode::Options` carry the +subscription priority their consumer fetches at, defaulting to hang's +`PRIORITY.video` and `PRIORITY.audio` as fixed today, so a player can fetch a +less important source behind a more important one under congestion. + +## Plan + +Each consumer hardcodes hang's priority for its kind today +(`rs/moq-video/src/decode/consumer.rs`, and its audio twin). Recommended: a +consumer that changes its priority while running updates its subscription in +place rather than resubscribing. Prove it with a test that reads the +priority off the publisher's aggregate subscription. + +Needed by [the studio's viewer priorities](https://github.com/moq-dev/studio/blob/main/quest/m2/compose/controls.md), +found by [moq-dev/studio#218](https://github.com/moq-dev/studio/pull/218), +and by [moq play composites layers](/quest/m3/layers/play.md). diff --git a/quest/m1/mp4-chapters.md b/quest/m1/mp4-chapters.md new file mode 100644 index 0000000000..39724f6aa0 --- /dev/null +++ b/quest/m1/mp4-chapters.md @@ -0,0 +1,26 @@ +# [S] A finished MP4 carries chapters + +## Goal + +moq-mux's regular MP4 export (`container::mp4`, both `Export::finish` and +crash `repair`) takes a list of chapters, each a moment on the recorded +broadcast's timeline and a title, and writes them into the finished `moov`. +ffmpeg, mpv, VLC, and Apple's players (QuickTime Player, Safari, Photos) list +them. Reading chapters back on import is out of scope. + +## Plan + +Decided 2026-11-20 (maintainer, /quest-plan): ✅ write both forms, Nero's +`chpl` box in the `moov`'s `udta` and a QuickTime chapter text track linked +from the video track by `tref chap` (rejected: `chpl` only, which Apple's +players ignore; the QuickTime track only, which some editors ignore). + +moq-mux alone knows where the movie starts on the broadcast's timeline, so it +maps each chapter's moment onto the movie's time; a chapter before the first +sample or after the last is dropped, not clamped. mp4-atom already encodes +`chpl`. Prove it with ffprobe listing the chapters of a finished and of a +repaired file. + +Found by [moq-dev/studio#209](https://github.com/moq-dev/studio/pull/209), +whose markers become a recording's chapters through +[the studio's pin](https://github.com/moq-dev/studio/blob/main/quest/m2/pin-mp4-chapters.md). diff --git a/quest/m1/orientation.md b/quest/m1/orientation.md new file mode 100644 index 0000000000..a6445beaad --- /dev/null +++ b/quest/m1/orientation.md @@ -0,0 +1,29 @@ +# [M] Players pick a rendition by the viewport's orientation + +## Goal + +When a broadcast's video renditions mix portrait and landscape, `@moq/watch` +and `moq play` prefer the renditions shaped like their viewport, and switch +when it rotates or is resized past square. A broadcast whose renditions share +one orientation selects as today. A consumer with no viewport, such as +moq-mux's export, a recorder, or a restream, takes the catalog's ranking +(`VideoConfig::preference`, then hang's `ranked` order). + +## Plan + +Decided 2026-11-20 (maintainer, /quest-plan): + +- ✅ portrait and landscape are renditions of one broadcast, told apart by + their coded size (rejected: a catalog field naming a sibling portrait + broadcast; a `/portrait` path convention in every player). +- ✅ with no viewport, the catalog's ranking decides, so a publisher whose + original is portrait ranks portrait first (rejected: always landscape; an + error when orientations mix and none is asked for). + +Orientation filters before the existing pixel, dimension, and bitrate +selection (`js/watch/src/video/source.ts`), so a phone in portrait takes the +best portrait rendition that fits its rate. A square rendition matches either. + +Found by [moq-dev/studio#200](https://github.com/moq-dev/studio/pull/200), +which publishes a portrait program beside the landscape one; the studio moves +it into the program's renditions once this lands. From 60e2d70a07c52274b4bc96b4bf167a615fab72dc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 20 Nov 2026 11:14:24 -0800 Subject: [PATCH 2/3] chore(quest): orientation filters before the preference tier @moq/watch keeps only a catalog's highest preference before it selects, so a secondary orientation ranked lower would never reach a portrait phone. Co-Authored-By: Claude Opus 5.5 --- quest/m1/orientation.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/quest/m1/orientation.md b/quest/m1/orientation.md index a6445beaad..1c16b5c12d 100644 --- a/quest/m1/orientation.md +++ b/quest/m1/orientation.md @@ -7,7 +7,7 @@ and `moq play` prefer the renditions shaped like their viewport, and switch when it rotates or is resized past square. A broadcast whose renditions share one orientation selects as today. A consumer with no viewport, such as moq-mux's export, a recorder, or a restream, takes the catalog's ranking -(`VideoConfig::preference`, then hang's `ranked` order). +(hang's `ranked` order, highest `VideoConfig::preference` first). ## Plan @@ -20,9 +20,13 @@ Decided 2026-11-20 (maintainer, /quest-plan): original is portrait ranks portrait first (rejected: always landscape; an error when orientations mix and none is asked for). -Orientation filters before the existing pixel, dimension, and bitrate -selection (`js/watch/src/video/source.ts`), so a phone in portrait takes the -best portrait rendition that fits its rate. A square rendition matches either. +Orientation filters first (`js/watch/src/video/source.ts`), so a phone in +portrait takes the best portrait rendition that fits its rate. It runs before +the preference tier (`preferredRenditions`), not only before the pixel, +dimension, and bitrate selection: a publisher ranks its secondary orientation +with a lower `preference`, which that tier drops today as a decode fallback, +so update `preference`'s docs in hang (Rust and JS) to say so. A square +rendition matches either. Found by [moq-dev/studio#200](https://github.com/moq-dev/studio/pull/200), which publishes a portrait program beside the landscape one; the studio moves From fa806d7e406d5b11959c8be741e4fddfbc2680e5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 20 Nov 2026 11:22:41 -0800 Subject: [PATCH 3/3] chore(quest): orientation reuses preference Co-Authored-By: Claude Opus 5.5 --- quest/m1/orientation.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/quest/m1/orientation.md b/quest/m1/orientation.md index 1c16b5c12d..9bc5bd3c98 100644 --- a/quest/m1/orientation.md +++ b/quest/m1/orientation.md @@ -19,6 +19,9 @@ Decided 2026-11-20 (maintainer, /quest-plan): - ✅ with no viewport, the catalog's ranking decides, so a publisher whose original is portrait ranks portrait first (rejected: always landscape; an error when orientations mix and none is asked for). +- ✅ that ranking is `VideoConfig::preference`, which orientation filters + before, so preference ranks within an orientation (rejected: a separate + catalog field marking the main orientation). Orientation filters first (`js/watch/src/video/source.ts`), so a phone in portrait takes the best portrait rendition that fits its rate. It runs before