Repository navigation
chore(quest): plan the studio's moq follow-ups #9247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # [S] The catalog producer publishes at the caller's timestamp | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-mux's catalog producer (`catalog::Producer`) can publish an update | ||
| stamped with a media timestamp its caller gives, rather than the time it is | ||
| written, so a publisher whose catalog changes at a frame (a studio's | ||
| composition switching layout) gets hang and MSF output from one producer. | ||
| Updates without a timestamp behave as today. | ||
|
|
||
| ## Plan | ||
|
|
||
| [moq-dev/studio#146](https://github.com/moq-dev/studio/pull/146) publishes its | ||
| composition through a hand-rolled hang-only `moq_json::snapshot::Producer`, | ||
| calling `Timed::new` itself, because the moq-mux producer stamps each emit on | ||
| its own clock; it cannot carry MSF beside hang until this lands. | ||
|
|
||
| Recommended: the timestamp is on the producer's clock, the same one its | ||
| `sync.clock` advertises, and a timestamp older than the last published one is | ||
| an error rather than reordered. | ||
|
Comment on lines
+19
to
+20
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Specify the behavior for timestamp collisions. The plan rejects a timestamp older than the last published one. It does not say what happens when a caller timestamp equals the last published timestamp, or when it rescales to the same group start. Add the required outcome for both collision cases to the plan. The plan must state whether such updates are rejected or whether a shifted publish timestamp is accepted.
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # [S] Decode consumers take a subscription priority | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq_video::decode::Options` and `moq_audio::decode::Options` carry the | ||
| subscription priority their consumer fetches at, defaulting to hang's | ||
| `PRIORITY.video` and `PRIORITY.audio` as fixed today, so a player can fetch a | ||
| less important source behind a more important one under congestion. | ||
|
|
||
| ## Plan | ||
|
|
||
| Each consumer hardcodes hang's priority for its kind today | ||
| (`rs/moq-video/src/decode/consumer.rs`, and its audio twin). Recommended: a | ||
| consumer that changes its priority while running updates its subscription in | ||
| place rather than resubscribing. Prove it with a test that reads the | ||
| priority off the publisher's aggregate subscription. | ||
|
|
||
| Needed by [the studio's viewer priorities](https://github.com/moq-dev/studio/blob/main/quest/m2/compose/controls.md), | ||
| found by [moq-dev/studio#218](https://github.com/moq-dev/studio/pull/218), | ||
| and by [moq play composites layers](/quest/m3/layers/play.md). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # [S] A finished MP4 carries chapters | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-mux's regular MP4 export (`container::mp4`, both `Export::finish` and | ||
| crash `repair`) takes a list of chapters, each a moment on the recorded | ||
| broadcast's timeline and a title, and writes them into the finished `moov`. | ||
| ffmpeg, mpv, VLC, and Apple's players (QuickTime Player, Safari, Photos) list | ||
| them. Reading chapters back on import is out of scope. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-20 (maintainer, /quest-plan): ✅ write both forms, Nero's | ||
| `chpl` box in the `moov`'s `udta` and a QuickTime chapter text track linked | ||
|
Comment on lines
+13
to
+14
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Define the behavior for chapter lists that The plan takes a chapter list with no maximum, and
|
||
| from the video track by `tref chap` (rejected: `chpl` only, which Apple's | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Define the chapter link for audio-only exports. The exporter accepts recordings with audio and no video. Audio renditions are built as independent tracks, not attached to video. The plan links the chapter track only from the video track, so an audio-only export has no track to carry that link. The plan does not say how such an export links its chapter track. Apple's players then show no chapters for that recording, which the Goal section requires. Specify which track carries the
|
||
| players ignore; the QuickTime track only, which some editors ignore). | ||
|
|
||
| moq-mux alone knows where the movie starts on the broadcast's timeline, so it | ||
| maps each chapter's moment onto the movie's time; a chapter before the first | ||
| sample or after the last is dropped, not clamped. mp4-atom already encodes | ||
| `chpl`. Prove it with ffprobe listing the chapters of a finished and of a | ||
| repaired file. | ||
|
Comment on lines
+21
to
+22
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Check the QuickTime chapter track separately.
|
||
|
|
||
| Found by [moq-dev/studio#209](https://github.com/moq-dev/studio/pull/209), | ||
| whose markers become a recording's chapters through | ||
| [the studio's pin](https://github.com/moq-dev/studio/blob/main/quest/m2/pin-mp4-chapters.md). | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # [M] Players pick a rendition by the viewport's orientation | ||
|
|
||
| ## Goal | ||
|
|
||
| When a broadcast's video renditions mix portrait and landscape, `@moq/watch` | ||
| and `moq play` prefer the renditions shaped like their viewport, and switch | ||
| when it rotates or is resized past square. A broadcast whose renditions share | ||
| one orientation selects as today. A consumer with no viewport, such as | ||
| moq-mux's export, a recorder, or a restream, takes the catalog's ranking | ||
| (hang's `ranked` order, highest `VideoConfig::preference` first). | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-20 (maintainer, /quest-plan): | ||
|
|
||
| - ✅ portrait and landscape are renditions of one broadcast, told apart by | ||
| their coded size (rejected: a catalog field naming a sibling portrait | ||
| broadcast; a `<broadcast>/portrait` path convention in every player). | ||
|
Comment on lines
+16
to
+18
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Account for video rotation when classifying renditions. If the catalog specifies a 90-degree rotation, a coded landscape rendition displays in portrait and a coded portrait rendition displays in landscape. Classifying by coded size alone selects the opposite displayed orientation. Include the catalog rotation in the orientation calculation.
|
||
| - ✅ with no viewport, the catalog's ranking decides, so a publisher whose | ||
| original is portrait ranks portrait first (rejected: always landscape; an | ||
| error when orientations mix and none is asked for). | ||
| - ✅ that ranking is `VideoConfig::preference`, which orientation filters | ||
| before, so preference ranks within an orientation (rejected: a separate | ||
| catalog field marking the main orientation). | ||
|
|
||
| Orientation filters first (`js/watch/src/video/source.ts`), so a phone in | ||
| portrait takes the best portrait rendition that fits its rate. It runs before | ||
| the preference tier (`preferredRenditions`), not only before the pixel, | ||
| dimension, and bitrate selection: a publisher ranks its secondary orientation | ||
| with a lower `preference`, which that tier drops today as a decode fallback, | ||
| so update `preference`'s docs in hang (Rust and JS) to say so. A square | ||
| rendition matches either. | ||
|
|
||
| Found by [moq-dev/studio#200](https://github.com/moq-dev/studio/pull/200), | ||
| which publishes a portrait program beside the landscape one; the studio moves | ||
| it into the program's renditions once this lands. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Limit the rule to the negotiated MoQ version.
@moq/qmuxderives the QMux version from the negotiated WebSocket subprotocol and branches on it to select transport parameters. The current wording also prohibits that work because@moq/qmuxis outsidemoq-net. Specify that the rule covers the MoQ version, not the separate QMux wire-format version. (raw.githubusercontent.com)