Repository navigation
quest: plan thirteen m3 quests moq-dev/studio waits on #8028
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [M] Audio capture opens a format the mix accepts | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-audio capture opens any input device at a channel count and sample rate | ||
| the bus accepts: a multichannel interface with no stereo or mono format, and | ||
| a device offering stereo or mono only at a non-default rate, both capture, | ||
| and a caller can pick which discrete inputs to use. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| Without an override, capture opens the device's default config as is | ||
| (`rs/moq-audio/src/capture.rs`); with one, `negotiate()` needs an exact match | ||
| and otherwise fails `Unsupported`. A device with more than two channels opens | ||
| as `Layout::Discrete(n)`, which `Remix` refuses to turn into stereo | ||
| (`rs/moq-audio/src/resample.rs`), and `capture::Config` has no input mapping. | ||
|
|
||
| Consumer: studio microphones ([moq-dev/studio#27](https://github.com/moq-dev/studio/pull/27)); a 4-input interface | ||
| worked before a moq bump via surround guessing and fails now. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # [S] An audio encoder flushes without ending its track | ||
|
|
||
| ## Goal | ||
|
|
||
| An audio encoder's caller can flush the samples it holds into the track and | ||
| keep the track open, so the last partial frame of a clip is heard when the | ||
| source pauses rather than dropped. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `encode::Producer::finish` (`rs/moq-audio/src/encode/producer.rs`) drains and | ||
| then ends the track, and `discontinuity` drops buffered samples. The | ||
| non-consuming `Encoder::drain` is `pub(super)` | ||
| (`rs/moq-audio/src/encode/encoder.rs`). A studio media file's sound | ||
| ([moq-dev/studio#44](https://github.com/moq-dev/studio/pull/44)) loses its last ~26 ms when a clip ends but the | ||
| source stays live. Related: [An audio track's final flush joins its open | ||
| group](/quest/m3/audio-flush-group.md). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # [XS] Denied macOS system audio names its settings pane | ||
|
|
||
| ## Goal | ||
|
|
||
| A macOS system-audio source denied the Screen & System Audio Recording | ||
| permission fails with a permission error naming that System Settings pane, as | ||
| the display source and microphone already do, instead of the OS's "The user | ||
| declined TCCs". | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| moq-audio's system audio surfaces `NSError.localizedDescription` as a capture | ||
| error (`rs/moq-audio/src/capture/screencapture.rs`). moq-video's display | ||
| source checks first and names the pane (`ensure_screen_access` in | ||
| `rs/moq-video/src/capture/screencapture.rs`), and the microphone path does too | ||
| (`rs/moq-audio/src/capture/permission.rs`). Reuse that check rather than | ||
| matching on the OS error. | ||
|
|
||
| Consumer: the studio's system-audio source | ||
| ([moq-dev/studio#14](https://github.com/moq-dev/studio/pull/14)). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # [L] One encoder follows several connections' estimates | ||
|
|
||
| ## Goal | ||
|
|
||
| A track published over several connections at once, such as a studio | ||
| publishing to a primary and a backup relay, has its encoder follow every | ||
| connection's bandwidth estimate, so losing the primary does not leave it | ||
| holding the bitrate the primary last granted. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `bandwidth::Allocator::new` divides one estimate (`rs/moq-net/src/model/bandwidth.rs`, | ||
| "one estimate covers a whole connection"), and each allocator comes from one | ||
| connection (`moq_tokio::Connection::allocator`, `Session::allocator`). Nothing | ||
| combines them. | ||
|
|
||
| Consumer: studio failover ([moq-dev/studio#19](https://github.com/moq-dev/studio/pull/19)), which publishes to two | ||
| relays while its encoders follow only the first relay's estimate. | ||
| Recommendation, not settled: one reservation per connection, with the encoder | ||
| following the lowest grant among connections that still have readers, rather | ||
| than merging estimates into one allocator. Benchmark the fan-out over both | ||
| connection and track count. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # [S] A mix-minus removes several sources | ||
|
|
||
| ## Goal | ||
|
|
||
| A `bus` mix-minus takes out any number of sources, so a guest's return feed | ||
| can drop every copy of that guest's own sound. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `Mix::minus` (`rs/moq-audio/src/bus.rs`) takes one `Strip`. A studio guest | ||
| with a camera and a screen share has two sources of their own sound, so their | ||
| return feed echoes one of them ([moq-dev/studio#37](https://github.com/moq-dev/studio/pull/37)). Change `minus` | ||
| itself rather than adding a second method. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # [M] Bus writes land at a bus position | ||
|
|
||
| ## Goal | ||
|
|
||
| A `bus::Channel` write can say which bus position its samples belong at, and | ||
| the bus places them there, so a caller lines sound up with video without | ||
| copying the bus's private latency and padding rules. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| - ✅ **Positioned writes**, a breaking change to `Channel::write`. | ||
| Rejected: exposing a channel's buffered depth and latency, which leaves the | ||
| caller doing the placement and keeps the dry-channel error below. | ||
|
|
||
| Today `Channel::write` (`rs/moq-audio/src/bus.rs`) takes no position, and | ||
| `LATENCY` (50 ms) and `HEADROOM` are private; positions exist only on the | ||
| output side (`Block::position`, `Driver::position`). Studio broadcast sync | ||
| ([moq-dev/studio#38](https://github.com/moq-dev/studio/pull/38)) copies those constants, and sound can trail | ||
| video by up to 37.5 ms after a channel runs dry. Decide what a write for a | ||
| position already mixed or beyond the headroom does, and fail loudly rather | ||
| than silently shifting it. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [S] macOS capture listing futures are Send | ||
|
|
||
| ## Goal | ||
|
|
||
| `capture::{displays, windows, apps}` and the capture start on macOS return | ||
| `Send` futures, so a caller can spawn them on a multi-threaded runtime | ||
| instead of wrapping them in `spawn_blocking`. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `screencapture::shareable_content` (`rs/moq-video/src/capture/screencapture.rs`) | ||
| keeps a ScreenCaptureKit completion `RcBlock`, which is not `Send`, alive | ||
| across its `await`; `start_capture` does the same, and so does moq-audio's copy | ||
| (`rs/moq-audio/src/capture/screencapture.rs`), whose compile-time `Send` check | ||
| is disabled on macOS (`rs/moq-audio/src/capture.rs`). Drop the block before | ||
| the await in each and turn the check back on. | ||
|
Comment on lines
+13
to
+18
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [P2] Separate listing Send from live-capture ownership Dropping these
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Narrowed in the new head: the goal is now listing-only, live capture stays (Written by Claude Opus 5.5) |
||
|
|
||
| Consumer: the studio's macOS sources ([moq-dev/studio#14](https://github.com/moq-dev/studio/pull/14)) work | ||
| around it with `spawn_blocking`; the workaround goes once this lands. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # [S] Container export starts apart from its skip budget | ||
|
|
||
| ## Goal | ||
|
|
||
| An fMP4 or MP4 export starts at the live edge even when it is allowed to fall | ||
| behind by a long skip budget, and reports how many samples it wrote per | ||
| track. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `with_max_delay` (`rs/moq-mux/src/container/fmp4/export.rs`, and | ||
| `container/mp4.rs`) sets both how far an export may fall behind and where it | ||
| starts, since it reads from the oldest group within that window. The studio | ||
|
pullfrog[bot] marked this conversation as resolved.
Resolved
|
||
| recorder ([moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18)) sets 2 s so a disk stall loses nothing, | ||
| and so every recording also starts up to 2 s before it was started | ||
| (`rs/moq-studio/src/record.rs`). Only TS export reports stats today; fMP4 and | ||
|
pullfrog[bot] marked this conversation as resolved.
Resolved
|
||
| MP4 report no sample counts, which the recorder needs to say what it wrote. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # [M] MP4 export trims exactly at both ends | ||
|
|
||
| ## Goal | ||
|
|
||
| An MP4 export takes a presentation window and presents exactly that span: no | ||
| keyframe lead-in before its start, and frames past its end kept only as | ||
| references for B-frame decoding, never presented. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `mp4::Export` (`rs/moq-mux/src/container/mp4.rs`) already writes an edit list | ||
| from its `Timeline`, which starts at the earliest presentation time and covers | ||
| every written sample. There is no caller-supplied window: an export starts | ||
| where its subscription starts, keyframe included, and `stop()` is the only | ||
| end. | ||
|
|
||
| Consumer: studio replay clips ([moq-dev/studio#20](https://github.com/moq-dev/studio/pull/20)) cut up to about | ||
| 200 ms early today to avoid presenting the lead-in. Test with B-frame content | ||
| that the trimmed file plays the window and nothing else in at least one | ||
| standard player. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [M] moq-mux imports a regular MP4 | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-mux imports a regular, non-fragmented MP4, with `moov` before or after | ||
| `mdat`, as it imports fMP4. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `ContainerFormat` offers fMP4, MKV, TS, and FLV | ||
| (`rs/moq-mux/src/import/format.rs`), and fMP4 import refuses an `mdat` that | ||
| follows no `moof` (`Error::NoMoof` in | ||
| `rs/moq-mux/src/container/fmp4/import.rs`). `container/mp4.rs` handles regular | ||
| MP4 only for export and repair; reuse its parsing. A `moov`-last file needs | ||
| the whole file or a seekable source, so bound what it holds by design. | ||
|
|
||
| Consumer: the studio's media-file source ([moq-dev/studio#44](https://github.com/moq-dev/studio/pull/44)), which | ||
| refuses typical intro clips with a remux hint today. Fuzz the new path with | ||
| the existing media fuzz targets. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # [XS] The test relay can enable stats | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq_relay::test_relay` can start a relay with stats enabled, so a | ||
| downstream test reads relay stats without building its own relay and skipping | ||
| certificate checks on loopback. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| `test_relay()` (`rs/moq-relay/src/test_support.rs`) builds `Config::default()`, | ||
| where `stats::Config.enabled` is false, and takes no arguments. The studio's | ||
| health test builds its own relay to get stats. Prefer a config the helper | ||
| takes over a second helper, since `foo_with_x` variants are banned. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # [S] Bandwidth reservations ignore in-process readers | ||
|
|
||
| ## Goal | ||
|
|
||
| An in-process reader of a track, such as a local recorder, does not make the | ||
| track's bandwidth reservation demanded or the link probe treat the uplink as | ||
| watched; only readers across a connection do. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| A reservation is gated on `track::Demand`, which counts every reader, | ||
| including an in-process `subscribe` (`rs/moq-net/src/model/bandwidth.rs`, | ||
| tests around the demand gate). So a studio recording of its own broadcast | ||
| ([moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18)) reserves uplink and makes Health grade an | ||
| unwatched uplink as watched. Distinguish remote demand without making local | ||
| readers invisible to the encoder's own on-demand start. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # [S] A published capture exposes its bandwidth grant | ||
|
|
||
| ## Goal | ||
|
|
||
| A caller that publishes a capture through moq-video can read the bitrate the | ||
| bandwidth allocator grants that track, not only the encoder's target, so a UI | ||
| shows each track's share of the uplink. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. | ||
|
|
||
| moq-net already exposes the grant (`bandwidth::Reservation::rate`, also in | ||
| moq-ffi), but `encode::publish_capture` (`rs/moq-video/src/encode/producer.rs`) | ||
| holds the reservation privately in its capture loop and returns nothing. The | ||
| only figure a caller sees is the target reported through the catalog stats. | ||
|
|
||
| Consumer: the studio's encode-control rungs and Health panel | ||
| ([moq-dev/studio#25](https://github.com/moq-dev/studio/pull/25)). Recommendation: return a handle from the publish | ||
| call that carries the grant, rather than a callback. |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Script executed:
Repository: moq-dev/moq
Length of output: 4665
Script executed:
Repository: moq-dev/moq
Length of output: 38993
Web query:
docs.rs objc2-screen-capture-kit 0.3.2 SCStream auto trait implementations Send SyncResult:
Web query:
docs.rs objc2-screen-capture-kit 0.3.2 struct SCStream "!Sync" "!Send" auto trait implementationsResult:
Web query:
Rust reference async fn captures all function parameters future even if not used async block argumentResult:
Script executed:
Repository: moq-dev/moq
Length of output: 9393
Account for
SCStreamin the Send plan.Dropping the completion block alone will not make capture start
Send. Bothstart_capturefutures capture&SCStream, butSCStreamis notSync. Expand the plan to address the stream handles and verify both public capture-start futures with compile-timeSendchecks.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed by narrowing to listing only; capture start and its SCStream borrow are now explicitly out of scope.
(Written by Claude Opus 5.5)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Analysis chain
Script executed:
Length of output: 3124
Script executed:
Length of output: 2751
@kixelatedNarrowing the quest to listing futures addresses the concern. However, I could not verify that change at the current PR head,f4b32d0:quest/m3/capture-shareable-send.md, Line 5, still includes capture start.start_capturein the proposed fix.quest/m3/README.md, Line 176, still says “listing and start.”Please push the scope update and request another check. This finding remains unresolved until the updated plan is available.
Would you like a follow-up issue for capture-start
Sendsupport?Learnings used
You are interacting with an AI system.