diff --git a/quest/m3/README.md b/quest/m3/README.md index 14548b8c020..c24f4b321cc 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -171,3 +171,16 @@ federation) land. - [MKV imports ffmpeg's multichannel FLAC](/quest/m3/mkv-flac-channel-mask.md) - a channel-mask-only Vorbis comment is accepted, so default ffmpeg multichannel FLAC needs no remux - [The publish support panel lists only codecs publishing accepts](/quest/m3/publish-support-probe.md) - the settings panel runs the publisher's Annex B encoder probe - [TLS reload follows symlinked certificate files](/quest/m3/watch-symlink-targets.md) - in-place rewrites of symlink targets in other directories reload roots, served chains, and client certificates +- [A published capture exposes its bandwidth grant](/quest/m3/video-publish-grant.md) - a moq-video publish shows each track's granted bitrate, not only its target +- [The test relay can enable stats](/quest/m3/relay-test-stats.md) - `moq_relay::test_relay` starts with stats on when asked +- [macOS capture listing futures are Send](/quest/m3/capture-shareable-send.md) - ScreenCaptureKit listing and start no longer hold a non-Send block across an await +- [Denied macOS system audio names its settings pane](/quest/m3/audio-system-denied-hint.md) - a permission error pointing at Screen & System Audio Recording +- [One encoder follows several connections' estimates](/quest/m3/bandwidth-multi-connection.md) - a track published to two relays tracks both estimates +- [Bandwidth reservations ignore in-process readers](/quest/m3/reservation-local-readers.md) - a local recorder no longer counts as uplink demand +- [Container export starts apart from its skip budget](/quest/m3/export-live-start.md) - fMP4 and MP4 export start at the live edge and report sample counts +- [MP4 export trims exactly at both ends](/quest/m3/mp4-export-window.md) - a presentation window with no lead-in and no presented tail +- [Audio capture opens a format the mix accepts](/quest/m3/audio-capture-negotiate.md) - multichannel and odd-rate devices capture, with discrete input selection +- [A mix-minus removes several sources](/quest/m3/bus-minus-several.md) - a guest's return feed drops every copy of their own sound +- [Bus writes land at a bus position](/quest/m3/bus-positioned-write.md) - callers sync audio to video without copying the bus's private latency +- [moq-mux imports a regular MP4](/quest/m3/mp4-import.md) - non-fragmented MP4, `moov` first or last +- [An audio encoder flushes without ending its track](/quest/m3/audio-encode-flush.md) - a clip's last partial frame is heard while the source stays live diff --git a/quest/m3/audio-capture-negotiate.md b/quest/m3/audio-capture-negotiate.md new file mode 100644 index 00000000000..afd0ff2fb10 --- /dev/null +++ b/quest/m3/audio-capture-negotiate.md @@ -0,0 +1,21 @@ +# [M] Audio capture opens a format the mix accepts + +## Goal + +moq-audio capture opens any input device at a channel count and sample rate +the bus accepts: a multichannel interface with no stereo or mono format, and +a device offering stereo or mono only at a non-default rate, both capture, +and a caller can pick which discrete inputs to use. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +Without an override, capture opens the device's default config as is +(`rs/moq-audio/src/capture.rs`); with one, `negotiate()` needs an exact match +and otherwise fails `Unsupported`. A device with more than two channels opens +as `Layout::Discrete(n)`, which `Remix` refuses to turn into stereo +(`rs/moq-audio/src/resample.rs`), and `capture::Config` has no input mapping. + +Consumer: studio microphones ([moq-dev/studio#27](https://github.com/moq-dev/studio/pull/27)); a 4-input interface +worked before a moq bump via surround guessing and fails now. diff --git a/quest/m3/audio-encode-flush.md b/quest/m3/audio-encode-flush.md new file mode 100644 index 00000000000..391c6d8c3be --- /dev/null +++ b/quest/m3/audio-encode-flush.md @@ -0,0 +1,19 @@ +# [S] An audio encoder flushes without ending its track + +## Goal + +An audio encoder's caller can flush the samples it holds into the track and +keep the track open, so the last partial frame of a clip is heard when the +source pauses rather than dropped. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`encode::Producer::finish` (`rs/moq-audio/src/encode/producer.rs`) drains and +then ends the track, and `discontinuity` drops buffered samples. The +non-consuming `Encoder::drain` is `pub(super)` +(`rs/moq-audio/src/encode/encoder.rs`). A studio media file's sound +([moq-dev/studio#44](https://github.com/moq-dev/studio/pull/44)) loses its last ~26 ms when a clip ends but the +source stays live. Related: [An audio track's final flush joins its open +group](/quest/m3/audio-flush-group.md). diff --git a/quest/m3/audio-system-denied-hint.md b/quest/m3/audio-system-denied-hint.md new file mode 100644 index 00000000000..b1b0e4e8d29 --- /dev/null +++ b/quest/m3/audio-system-denied-hint.md @@ -0,0 +1,22 @@ +# [XS] Denied macOS system audio names its settings pane + +## Goal + +A macOS system-audio source denied the Screen & System Audio Recording +permission fails with a permission error naming that System Settings pane, as +the display source and microphone already do, instead of the OS's "The user +declined TCCs". + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +moq-audio's system audio surfaces `NSError.localizedDescription` as a capture +error (`rs/moq-audio/src/capture/screencapture.rs`). moq-video's display +source checks first and names the pane (`ensure_screen_access` in +`rs/moq-video/src/capture/screencapture.rs`), and the microphone path does too +(`rs/moq-audio/src/capture/permission.rs`). Reuse that check rather than +matching on the OS error. + +Consumer: the studio's system-audio source +([moq-dev/studio#14](https://github.com/moq-dev/studio/pull/14)). diff --git a/quest/m3/bandwidth-multi-connection.md b/quest/m3/bandwidth-multi-connection.md new file mode 100644 index 00000000000..9c40b97c7a9 --- /dev/null +++ b/quest/m3/bandwidth-multi-connection.md @@ -0,0 +1,24 @@ +# [L] One encoder follows several connections' estimates + +## Goal + +A track published over several connections at once, such as a studio +publishing to a primary and a backup relay, has its encoder follow every +connection's bandwidth estimate, so losing the primary does not leave it +holding the bitrate the primary last granted. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`bandwidth::Allocator::new` divides one estimate (`rs/moq-net/src/model/bandwidth.rs`, +"one estimate covers a whole connection"), and each allocator comes from one +connection (`moq_tokio::Connection::allocator`, `Session::allocator`). Nothing +combines them. + +Consumer: studio failover ([moq-dev/studio#19](https://github.com/moq-dev/studio/pull/19)), which publishes to two +relays while its encoders follow only the first relay's estimate. +Recommendation, not settled: one reservation per connection, with the encoder +following the lowest grant among connections that still have readers, rather +than merging estimates into one allocator. Benchmark the fan-out over both +connection and track count. diff --git a/quest/m3/bus-minus-several.md b/quest/m3/bus-minus-several.md new file mode 100644 index 00000000000..2a3125bd10c --- /dev/null +++ b/quest/m3/bus-minus-several.md @@ -0,0 +1,15 @@ +# [S] A mix-minus removes several sources + +## Goal + +A `bus` mix-minus takes out any number of sources, so a guest's return feed +can drop every copy of that guest's own sound. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`Mix::minus` (`rs/moq-audio/src/bus.rs`) takes one `Strip`. A studio guest +with a camera and a screen share has two sources of their own sound, so their +return feed echoes one of them ([moq-dev/studio#37](https://github.com/moq-dev/studio/pull/37)). Change `minus` +itself rather than adding a second method. diff --git a/quest/m3/bus-positioned-write.md b/quest/m3/bus-positioned-write.md new file mode 100644 index 00000000000..aacfe606719 --- /dev/null +++ b/quest/m3/bus-positioned-write.md @@ -0,0 +1,23 @@ +# [M] Bus writes land at a bus position + +## Goal + +A `bus::Channel` write can say which bus position its samples belong at, and +the bus places them there, so a caller lines sound up with video without +copying the bus's private latency and padding rules. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +- ✅ **Positioned writes**, a breaking change to `Channel::write`. + Rejected: exposing a channel's buffered depth and latency, which leaves the + caller doing the placement and keeps the dry-channel error below. + +Today `Channel::write` (`rs/moq-audio/src/bus.rs`) takes no position, and +`LATENCY` (50 ms) and `HEADROOM` are private; positions exist only on the +output side (`Block::position`, `Driver::position`). Studio broadcast sync +([moq-dev/studio#38](https://github.com/moq-dev/studio/pull/38)) copies those constants, and sound can trail +video by up to 37.5 ms after a channel runs dry. Decide what a write for a +position already mixed or beyond the headroom does, and fail loudly rather +than silently shifting it. diff --git a/quest/m3/capture-shareable-send.md b/quest/m3/capture-shareable-send.md new file mode 100644 index 00000000000..89833a3fdb1 --- /dev/null +++ b/quest/m3/capture-shareable-send.md @@ -0,0 +1,21 @@ +# [S] macOS capture listing futures are Send + +## Goal + +`capture::{displays, windows, apps}` and the capture start on macOS return +`Send` futures, so a caller can spawn them on a multi-threaded runtime +instead of wrapping them in `spawn_blocking`. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`screencapture::shareable_content` (`rs/moq-video/src/capture/screencapture.rs`) +keeps a ScreenCaptureKit completion `RcBlock`, which is not `Send`, alive +across its `await`; `start_capture` does the same, and so does moq-audio's copy +(`rs/moq-audio/src/capture/screencapture.rs`), whose compile-time `Send` check +is disabled on macOS (`rs/moq-audio/src/capture.rs`). Drop the block before +the await in each and turn the check back on. + +Consumer: the studio's macOS sources ([moq-dev/studio#14](https://github.com/moq-dev/studio/pull/14)) work +around it with `spawn_blocking`; the workaround goes once this lands. diff --git a/quest/m3/export-live-start.md b/quest/m3/export-live-start.md new file mode 100644 index 00000000000..a947821ec2c --- /dev/null +++ b/quest/m3/export-live-start.md @@ -0,0 +1,19 @@ +# [S] Container export starts apart from its skip budget + +## Goal + +An fMP4 or MP4 export starts at the live edge even when it is allowed to fall +behind by a long skip budget, and reports how many samples it wrote per +track. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`with_max_delay` (`rs/moq-mux/src/container/fmp4/export.rs`, and +`container/mp4.rs`) sets both how far an export may fall behind and where it +starts, since it reads from the oldest group within that window. The studio +recorder ([moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18)) sets 2 s so a disk stall loses nothing, +and so every recording also starts up to 2 s before it was started +(`rs/moq-studio/src/record.rs`). Only TS export reports stats today; fMP4 and +MP4 report no sample counts, which the recorder needs to say what it wrote. diff --git a/quest/m3/mp4-export-window.md b/quest/m3/mp4-export-window.md new file mode 100644 index 00000000000..4748ba8d90b --- /dev/null +++ b/quest/m3/mp4-export-window.md @@ -0,0 +1,22 @@ +# [M] MP4 export trims exactly at both ends + +## Goal + +An MP4 export takes a presentation window and presents exactly that span: no +keyframe lead-in before its start, and frames past its end kept only as +references for B-frame decoding, never presented. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`mp4::Export` (`rs/moq-mux/src/container/mp4.rs`) already writes an edit list +from its `Timeline`, which starts at the earliest presentation time and covers +every written sample. There is no caller-supplied window: an export starts +where its subscription starts, keyframe included, and `stop()` is the only +end. + +Consumer: studio replay clips ([moq-dev/studio#20](https://github.com/moq-dev/studio/pull/20)) cut up to about +200 ms early today to avoid presenting the lead-in. Test with B-frame content +that the trimmed file plays the window and nothing else in at least one +standard player. diff --git a/quest/m3/mp4-import.md b/quest/m3/mp4-import.md new file mode 100644 index 00000000000..609f5a7c285 --- /dev/null +++ b/quest/m3/mp4-import.md @@ -0,0 +1,21 @@ +# [M] moq-mux imports a regular MP4 + +## Goal + +moq-mux imports a regular, non-fragmented MP4, with `moov` before or after +`mdat`, as it imports fMP4. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`ContainerFormat` offers fMP4, MKV, TS, and FLV +(`rs/moq-mux/src/import/format.rs`), and fMP4 import refuses an `mdat` that +follows no `moof` (`Error::NoMoof` in +`rs/moq-mux/src/container/fmp4/import.rs`). `container/mp4.rs` handles regular +MP4 only for export and repair; reuse its parsing. A `moov`-last file needs +the whole file or a seekable source, so bound what it holds by design. + +Consumer: the studio's media-file source ([moq-dev/studio#44](https://github.com/moq-dev/studio/pull/44)), which +refuses typical intro clips with a remux hint today. Fuzz the new path with +the existing media fuzz targets. diff --git a/quest/m3/relay-test-stats.md b/quest/m3/relay-test-stats.md new file mode 100644 index 00000000000..cb465d2f417 --- /dev/null +++ b/quest/m3/relay-test-stats.md @@ -0,0 +1,16 @@ +# [XS] The test relay can enable stats + +## Goal + +`moq_relay::test_relay` can start a relay with stats enabled, so a +downstream test reads relay stats without building its own relay and skipping +certificate checks on loopback. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +`test_relay()` (`rs/moq-relay/src/test_support.rs`) builds `Config::default()`, +where `stats::Config.enabled` is false, and takes no arguments. The studio's +health test builds its own relay to get stats. Prefer a config the helper +takes over a second helper, since `foo_with_x` variants are banned. diff --git a/quest/m3/reservation-local-readers.md b/quest/m3/reservation-local-readers.md new file mode 100644 index 00000000000..3932891a7e8 --- /dev/null +++ b/quest/m3/reservation-local-readers.md @@ -0,0 +1,18 @@ +# [S] Bandwidth reservations ignore in-process readers + +## Goal + +An in-process reader of a track, such as a local recorder, does not make the +track's bandwidth reservation demanded or the link probe treat the uplink as +watched; only readers across a connection do. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +A reservation is gated on `track::Demand`, which counts every reader, +including an in-process `subscribe` (`rs/moq-net/src/model/bandwidth.rs`, +tests around the demand gate). So a studio recording of its own broadcast +([moq-dev/studio#18](https://github.com/moq-dev/studio/pull/18)) reserves uplink and makes Health grade an +unwatched uplink as watched. Distinguish remote demand without making local +readers invisible to the encoder's own on-demand start. diff --git a/quest/m3/video-publish-grant.md b/quest/m3/video-publish-grant.md new file mode 100644 index 00000000000..8302b8d4ea3 --- /dev/null +++ b/quest/m3/video-publish-grant.md @@ -0,0 +1,20 @@ +# [S] A published capture exposes its bandwidth grant + +## Goal + +A caller that publishes a capture through moq-video can read the bitrate the +bandwidth allocator grants that track, not only the encoder's target, so a UI +shows each track's share of the uplink. + +## Plan + +Decided 2026-11-05 (maintainer, on moq-dev/studio's /quest-plan): m3, since the studio is a named consumer that is not urgent. + +moq-net already exposes the grant (`bandwidth::Reservation::rate`, also in +moq-ffi), but `encode::publish_capture` (`rs/moq-video/src/encode/producer.rs`) +holds the reservation privately in its capture loop and returns nothing. The +only figure a caller sees is the target reported through the catalog stats. + +Consumer: the studio's encode-control rungs and Health panel +([moq-dev/studio#25](https://github.com/moq-dev/studio/pull/25)). Recommendation: return a handle from the publish +call that carries the grant, rather than a callback.