From d3c3f27790b4a27964b4f89f99799ecf0794eb4c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 15:33:03 -0700 Subject: [PATCH 1/6] chore(quest): claim quest/m1/push-to-talk/hold-rs.md Quest-Claim: f92a75f38dc2d388 From 96b9e8381bc22d36420c454a7786655f97928fda Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 16:02:56 -0700 Subject: [PATCH 2/6] feat!: Rust consumers hold a disabled rendition Co-Authored-By: Claude Opus 5.5 --- doc/bin/cli.md | 2 +- doc/lib/c/index.md | 2 +- doc/lib/cpp/index.md | 2 +- doc/lib/dart/index.md | 2 +- doc/lib/go/index.md | 2 +- doc/lib/kt/index.md | 2 +- doc/lib/py/index.md | 2 +- doc/lib/swift/index.md | 2 +- rs/moq-c/src/api.rs | 7 +- rs/moq-c/src/consume.rs | 14 --- rs/moq-cli/src/play/media.rs | 77 +++++++++++-- rs/moq-ffi/src/audio.rs | 13 ++- rs/moq-ffi/src/media.rs | 8 +- rs/moq-ffi/src/video.rs | 12 +- rs/moq-mux/src/catalog/select.rs | 76 ++++++++++--- rs/moq-mux/src/select.rs | 189 +++++++++++++++++++++++++------ rs/moq-rtmp/src/server.rs | 6 +- 17 files changed, 311 insertions(+), 107 deletions(-) diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 00aefd0d5f8..dd7ab045650 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -236,7 +236,7 @@ audio-device dependencies: cargo install moq-cli --no-default-features --features "quic,websocket,play,openh264" ``` -`moq play` skips disabled renditions. If none are enabled, it waits for a later catalog. Disabling the current rendition stops it and selects an enabled replacement. +`moq play` skips disabled renditions when it picks one. If none are enabled, it waits for a later catalog. Disabling the current rendition switches to an enabled one of its kind; with none, it stays subscribed, so re-enabling it (an unmute) plays at once. ## Capture diff --git a/doc/lib/c/index.md b/doc/lib/c/index.md index cf9f889a4bd..16be1f588bf 100644 --- a/doc/lib/c/index.md +++ b/doc/lib/c/index.md @@ -84,7 +84,7 @@ inside. The header is the reference: each function carries a doc comment. - **Decoded reads.** `moq_decode_audio` and `moq_decode_video` deliver read ids in order: a frame, a gap where media was lost, or an end where the publisher stopped its encoder. Check `kind` on what `moq_decode_audio_read` or `moq_decode_video_read` writes. Free each id with `moq_decode_audio_read_free` or `moq_decode_video_read_free` promptly: a held video frame holds a decoder buffer, and the decoder stalls. - **Stats.** `moq_session_stats()` fills `moq_connection_stats` with `rtt_us`, `estimated_send_rate_bps`, and the byte and packet counters (`bytes_sent`, `bytes_received`, `bytes_lost`, `packets_sent`, `packets_received`, `packets_lost`). Each has a `_valid` flag, false when the transport does not report it, which is not the same as zero. `moq_session_snapshot()` adds the negotiated `protocol`. A failed dial or dropped connection reaches the session's `on_state` with its `failure` code, and `moq_error()` gives the reason during that callback. Hold the handle returned by `moq_session_recv_bandwidth()` to request receive estimates. `moq_bandwidth_consumer_peek()` reads the estimate and `moq_bandwidth_consumer_changed()` waits for a change. `moq_bandwidth_consumer_close()` releases demand; the last consumer stops PROBE. Stats snapshots do not request it. - **Demand.** A watcher from `moq_publish_track_demand()` (or the media, video, and audio variants) calls back `MOQ_DEMAND_USED` or `MOQ_DEMAND_UNUSED`, again whenever the wanted group moves. `moq_publish_demand_wanted()` reads that group: start a new one once it passes your largest. Set `refresh` in `moq_track_info` to have a joining subscriber ask for a new group once the newest has grown past `refresh` times its first frame. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. ## Reference diff --git a/doc/lib/cpp/index.md b/doc/lib/cpp/index.md index 20995a40a57..2ce60865289 100644 --- a/doc/lib/cpp/index.md +++ b/doc/lib/cpp/index.md @@ -197,7 +197,7 @@ future completed, which is the executor thread. ## Disabled renditions -Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. ## Bandwidth estimates diff --git a/doc/lib/dart/index.md b/doc/lib/dart/index.md index 51dd58c06ba..bf7e73c6f6e 100644 --- a/doc/lib/dart/index.md +++ b/doc/lib/dart/index.md @@ -152,7 +152,7 @@ shows which module holds each. - **Authorization.** On `moq-lite-07-wip` (opt in with `ConnectOptions(versions: ...)`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), listen to `moq.grants().updates()` for the `Grant` the relay sent back and each change, `null` until it answers and while a client session reconnects. - **Tokens.** Pass `ConnectOptions(tokens: [...])` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `moq.tokens().set_(tokens: [...])` replaces them before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. `await tokens().answered()` returns once the relay accepts them and throws its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release. - **Errors.** Read diagnostic payloads from `reason` and structured protocol failures from `details`. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. - **Track demand.** Obtain a dynamic track request’s `demand()` before accepting it to watch subscribers through the handoff to its producer. The watch does not keep the track alive and ends when the track closes or the request is rejected. ## Runtime diff --git a/doc/lib/go/index.md b/doc/lib/go/index.md index b5d42aa5407..7a57287cd90 100644 --- a/doc/lib/go/index.md +++ b/doc/lib/go/index.md @@ -138,7 +138,7 @@ shows which module holds each. - **Authorization.** On `moq-lite-07-wip` (opt in with `ClientConfig.Versions`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), `Session().Grants()` returns a `GrantConsumer`: `Grant()` is the `Grant` the relay sent back, nil until it answers and while a client session reconnects, and `Changed(ctx)` waits for the next one. `Close()` it when done. - **Tokens.** Set `ClientConfig.Tokens` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `Session().Tokens().Set(...)` replaces a client session's tokens before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. `Answered(ctx)` returns nil once the relay accepts them, or its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release. - **Errors.** Read diagnostic payloads from `Reason` and structured protocol failures from `Details`. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `Preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `Preference`; `0` is the default and `-1` marks a compatibility transcode. - **Track demand.** Obtain a dynamic track request’s `Demand()` before accepting it to watch subscribers through the handoff to its producer. The watch does not keep the track alive and ends when the track closes or the request is rejected. ## Runtime diff --git a/doc/lib/kt/index.md b/doc/lib/kt/index.md index 52e2a492ac1..8a00725be47 100644 --- a/doc/lib/kt/index.md +++ b/doc/lib/kt/index.md @@ -109,7 +109,7 @@ shows which module holds each. - **Authorization.** On `moq-lite-07-wip` (opt in with `ClientConfig(versions = ...)`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), collect `moq.grants().updates()` for the `Grant` the relay sent back and each change, `null` until it answers and while a client session reconnects. Ending the collection cancels the consumer. - **Tokens.** Pass `ClientConfig(tokens = listOf(...))` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `moq.tokens().set(listOf(...))` replaces them before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. Suspending `tokens().answered()` returns once the relay accepts them and throws its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release. - **Errors.** Read diagnostic payloads from `reason` and structured protocol failures from `details`. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. - **Track demand.** Obtain a dynamic track request’s `demand()` before accepting it to watch subscribers through the handoff to its producer. The watch does not keep the track alive and ends when the track closes or the request is rejected. ## Runtime diff --git a/doc/lib/py/index.md b/doc/lib/py/index.md index 8f56f83d11d..450236217db 100644 --- a/doc/lib/py/index.md +++ b/doc/lib/py/index.md @@ -126,7 +126,7 @@ shows which module holds each. - **Authorization.** On `moq-lite-07-wip` (opt in with `versions=`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), `async for grant in session.grants()` yields the `Grant` the relay sent back and each change, `None` until it answers and while a client session reconnects. `grants().grant()` reads it once; cancelling or leaving an `async with` releases the wait. - **Tokens.** Pass `tokens=[...]` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `session.tokens().set([...])` replaces a client session's tokens before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. `await tokens.answered()` returns once the relay accepts them and raises its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release. - **Errors.** Read diagnostic payloads from `reason` and structured protocol failures from `details`. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. - **Track demand.** Obtain a dynamic track request’s `demand()` before accepting it to watch subscribers through the handoff to its producer. The watch does not keep the track alive and ends when the track closes or the request is rejected. ## Runtime diff --git a/doc/lib/swift/index.md b/doc/lib/swift/index.md index 6245d8f7f8d..b711882f558 100644 --- a/doc/lib/swift/index.md +++ b/doc/lib/swift/index.md @@ -109,7 +109,7 @@ shows which module holds each. - **Authorization.** On `moq-lite-07-wip` (opt in with `Client(versions:)`) and on moq-transport draft-18+ with the [MoQ Auth extension](/draft/moq-auth), `for try await grant in session.grants()` yields the `Grant` the relay sent back and each change, `nil` until it answers and while a client session reconnects. Cancelling the iterating task cancels the native wait. - **Tokens.** Pass `Client(tokens: [...])` instead of putting the credential in the URL. The first is the connection credential, sent in SETUP and, while an offered version can't carry it there (lite before 07), as the URL's `?jwt=`; each other token widens the grant over AUTH. `try session.tokens()?.set([...])` replaces them before they expire: the next dial presents them, a live AUTH session at once, and after an authorization refusal the session redials only if they changed. `try await tokens.answered()` returns once the relay accepts them and throws its refusal with the relay's code. A `?jwt=` already in the URL stays the connection credential, which is how to reach a relay older than this release. - **Errors.** Diagnostic cases use the `reason:` argument label; match their associated value to inspect it. Protocol errors carry `details`. -- **Disabled renditions.** Audio and video convenience decoders reject a catalog configuration with `enabled: false`. Select an enabled rendition from the latest catalog before opening a decoder. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. +- **Disabled renditions.** A rendition with `enabled: false` has no frames coming until its publisher re-enables it. Select an enabled rendition from the latest catalog before opening a decoder. Keep one already playing while every rendition of its kind is disabled: the convenience decoders accept a disabled configuration, and a re-enable, such as an unmute, then plays without a resubscribe. Among the video renditions you can decode, prefer the highest `preference`; `0` is the default and `-1` marks a compatibility transcode. - **Track demand.** Obtain a dynamic track request’s `demand()` before accepting it to watch subscribers through the handoff to its producer. The watch does not keep the track alive and ends when the track closes or the request is rejected. ## Runtime diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 81a8dd8190b..a23f0d6fe53 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -3452,10 +3452,11 @@ pub unsafe extern "C" fn moq_consume_video_config(catalog: u32, index: u32, dst: }) } -/// Query whether a video rendition may be selected. +/// Query whether the publisher is serving a video rendition. /// -/// False means no frames are coming and the rendition must not be selected. A catalog that -/// omits the optional field reads as true. +/// False means no frames are coming until it re-enables. Pick an enabled rendition, but keep one +/// already playing while every video rendition is disabled, so a re-enable plays without a +/// resubscribe. A catalog that omits the optional field reads as true. /// /// Returns zero on success, or a negative code on failure. /// diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index dfc32c1f552..8ac5ce54f7b 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -58,9 +58,6 @@ pub(crate) trait Media: hang::catalog::Reference + Clone { /// Its renditions, keyed by ID. fn renditions(catalog: &moq_mux::catalog::hang::Catalog) -> &BTreeMap; - - /// Whether the publisher is serving this rendition. - fn enabled(&self) -> bool; } impl Media for hang::catalog::VideoConfig { @@ -69,10 +66,6 @@ impl Media for hang::catalog::VideoConfig { fn renditions(catalog: &moq_mux::catalog::hang::Catalog) -> &BTreeMap { &catalog.video.renditions } - - fn enabled(&self) -> bool { - self.enabled - } } impl Media for hang::catalog::AudioConfig { @@ -81,10 +74,6 @@ impl Media for hang::catalog::AudioConfig { fn renditions(catalog: &moq_mux::catalog::hang::Catalog) -> &BTreeMap { &catalog.audio.renditions } - - fn enabled(&self) -> bool { - self.enabled - } } /// A catalog rendition to decode: the broadcast and track serving it, and the owner whose panic @@ -478,9 +467,6 @@ impl Consume { for<'a> moq_mux::catalog::hang::Container: TryFrom<&'a C, Error = moq_mux::Error>, { let (consume, name, config) = self.select::(catalog, index)?; - if !config.enabled() { - return Err(moq_net::Error::NotFound.into()); - } let name = hang::catalog::Reference::track(config, name).to_string(); // Consume with the container the catalog actually advertises (Legacy / Cmaf / Loc) // instead of assuming Legacy, otherwise CMAF/fMP4 sources (e.g. ffmpeg moqenc, diff --git a/rs/moq-cli/src/play/media.rs b/rs/moq-cli/src/play/media.rs index faaa17e7f82..658d192e43a 100644 --- a/rs/moq-cli/src/play/media.rs +++ b/rs/moq-cli/src/play/media.rs @@ -1,7 +1,7 @@ //! The media task: subscribe, pick renditions, and decode into the window and //! the speaker. -use std::collections::VecDeque; +use std::collections::{BTreeMap, VecDeque}; use std::sync::{Arc, Mutex}; use std::time::Duration; @@ -130,8 +130,9 @@ impl Media { let mut catalogs = catalog::Consumer::<()>::new(&broadcast, self.args.catalog_format(&self.broadcast)) .await .context("failed to subscribe to the catalog")?; - // Applied by hand rather than with `select`, which drops disabled renditions: a disabled - // rendition stops now, while one that leaves the catalog plays out its tail. + // Applied by hand rather than with `select`, so each snapshot holds the renditions that are + // actually playing, and so the player can tell a disabled rendition it lets go, which stops + // now, from one that leaves the catalog, which plays out its tail. let selection = self.args.select.selection(None); let mut tasks = tokio::task::JoinSet::new(); let mut playback = Playback::default(); @@ -189,9 +190,17 @@ impl Media { snapshot = catalogs.next(), if playback.following() => { match snapshot.context("failed to read the catalog")? { Some(mut snapshot) => { - stop_disabled(&mut video_demand, snapshot.video.renditions.iter().filter(|(_, config)| !config.enabled).map(|(name, _)| name)); - stop_disabled(&mut audio_demand, snapshot.audio.renditions.iter().filter(|(_, config)| !config.enabled).map(|(name, _)| name)); - selection.retain(&mut snapshot); + // A playing rendition its publisher disabled is held while nothing else + // of its kind is enabled, so an unmute plays without a resubscribe. + let video_disabled = disabled(&video_demand, &snapshot.video.renditions, |config| config.enabled); + let audio_disabled = disabled(&audio_demand, &snapshot.audio.renditions, |config| config.enabled); + let playing = moq_mux::select::Selected { + video: video_demand.iter().map(|demand| demand.name.clone()).collect(), + audio: audio_demand.iter().map(|demand| demand.name.clone()).collect(), + }; + selection.retain(&mut snapshot, &playing); + stop_released(&mut video_demand, video_disabled, &snapshot.video.renditions); + stop_released(&mut audio_demand, audio_disabled, &snapshot.audio.renditions); if let Some(demand) = &mut video_demand && demand.retire_moved(&source, snapshot.video.renditions.get(&demand.name)) { @@ -441,13 +450,20 @@ impl Demand { } } -// Cancel through the task's own result so it still reports which half ended. -fn stop_disabled<'a>(playing: &mut Option, mut disabled: impl Iterator) { - if playing +/// Whether the catalog still lists the rendition `playing` names, disabled. +fn disabled(playing: &Option, renditions: &BTreeMap, enabled: impl Fn(&C) -> bool) -> bool { + playing .as_ref() - .is_some_and(|playing| disabled.any(|disabled| *disabled == playing.name)) - { - let _ = playing.take().unwrap().stop.send(()); + .and_then(|playing| renditions.get(&playing.name)) + .is_some_and(|config| !enabled(config)) +} + +/// Stop the rendition `playing` if it was `disabled` and the selection let it go, because +/// another of its kind is enabled. Cancelled through the task's own result so it still reports +/// which half ended. +fn stop_released(playing: &mut Option, disabled: bool, selected: &BTreeMap) { + if disabled && let Some(demand) = playing.take_if(|playing| !selected.contains_key(&playing.name)) { + let _ = demand.stop.send(()); } } @@ -1453,6 +1469,43 @@ mod tests { ); } + /// Muting the only audio rendition holds it, so the unmute plays on the subscription it + /// already had rather than a new one. + #[tokio::test] + async fn a_muted_rendition_is_held_through_the_unmute() { + tokio::time::pause(); + let origin = moq_tokio::origin::spawn(); + let broadcast = origin.create_broadcast("room").unwrap(); + broadcast.announce(Default::default()).unwrap(); + let mut catalog = catalog::Producer::new(&mut broadcast.broadcast().clone(), Default::default()).unwrap(); + let mut audio = rendition(&broadcast, &catalog, "audio"); + let recorder = Recorder::default(); + let player = tokio::spawn(media(&origin, Duration::from_millis(50), recorder.clone()).run()); + let mut enable = |enabled: bool| { + catalog + .mutate(|catalog| catalog.audio.renditions.get_mut("audio").unwrap().enabled = enabled) + .unwrap() + }; + + write(&mut audio, 10, 0.25).await; + enable(false); + // Half a second muted, with the timeline running on through it. + let demand = audio.demand(); + for _ in 0..25 { + tokio::time::sleep(PACKET_DURATION).await; + assert!(demand.wanted().is_some(), "the mute unsubscribed"); + } + enable(true); + for index in 35..60 { + audio.write(packet(index, 0.5)).unwrap(); + tokio::time::sleep(PACKET_DURATION).await; + } + + settle(player, &recorder).await; + assert!(played(&recorder, 0.25).0 > Duration::ZERO); + assert!(played(&recorder, 0.5).0 > Duration::ZERO, "the unmute did not play"); + } + /// A publisher retires an audio rendition by naming its replacement and then /// finishing the old track. The retired sink still holds a delay of audio, and /// the replacement's sink holds its own before its first sample sounds, so diff --git a/rs/moq-ffi/src/audio.rs b/rs/moq-ffi/src/audio.rs index dfa5dce5332..4d008863b9a 100644 --- a/rs/moq-ffi/src/audio.rs +++ b/rs/moq-ffi/src/audio.rs @@ -611,9 +611,6 @@ impl MoqAudioConsumer { } fn audio_config(catalog_audio: crate::media::MoqAudio) -> Result { - if !catalog_audio.enabled { - return Err(MoqError::NotFound); - } let codec = catalog_audio.codec.parse().map_err(|_| MoqError::Unsupported)?; // What moq-audio's decoder opens. It rejects the rest itself, so this is only // here to report an unusable rendition as a plain Unsupported rather than a @@ -642,6 +639,11 @@ impl MoqAudioConsumer { /// [`MoqMediaCatalogConsumer::next`](crate::media::MoqMediaCatalogConsumer::next)); /// the codec is inferred from it. Only Opus, AAC-LC, and PCM are supported. /// + /// A [disabled](crate::media::MoqAudio::enabled) rendition subscribes too, and decodes once its + /// publisher re-enables it: a muted speaker's listener stays subscribed, so the unmute plays + /// from its first frame. Don't pick one, but keep one already playing while no enabled + /// rendition is on offer. + /// /// `id` is the rendition's catalog key. Its [`track`](crate::media::MoqAudio::track), or `id` /// when unset, is subscribed in the broadcast its /// [`broadcast`](crate::media::MoqAudio::broadcast) reference names. @@ -705,12 +707,11 @@ mod tests { } } + /// A consumer holds a disabled rendition it already plays, so one converts like any other. #[test] - fn audio_config_refuses_disabled_renditions() { + fn audio_config_accepts_disabled_renditions() { let mut config = catalog_audio("opus"); config.enabled = false; - assert!(matches!(audio_config(config.clone()), Err(MoqError::NotFound))); - config.enabled = true; assert!(audio_config(config).is_ok()); } diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index c3b5397aba8..c108a281486 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -132,7 +132,9 @@ pub struct MoqVideo { pub display_aspect: Option, /// The maximum encoded bitrate in bits per second. pub bitrate: Option, - /// Whether this rendition may be selected; when false, no frames are coming. + /// Whether the publisher is serving this rendition; when false, no frames are coming until it + /// re-enables. Pick an enabled rendition, but keep one already playing while every rendition + /// of its kind is disabled, so a re-enable plays without a resubscribe. #[uniffi(default = true)] pub enabled: bool, /// Higher values are preferred; compatibility transcodes use -1 and the default is 0. @@ -173,7 +175,9 @@ pub struct MoqAudio { pub channel_count: u32, /// The maximum encoded bitrate in bits per second. pub bitrate: Option, - /// Whether this rendition may be selected; when false, no frames are coming. + /// Whether the publisher is serving this rendition; when false, no frames are coming until it + /// re-enables. Pick an enabled rendition, but keep one already playing while every rendition + /// of its kind is disabled, so a re-enable plays without a resubscribe. #[uniffi(default = true)] pub enabled: bool, /// The framing used to carry this rendition. diff --git a/rs/moq-ffi/src/video.rs b/rs/moq-ffi/src/video.rs index 459d29c2619..2b75a78de4c 100644 --- a/rs/moq-ffi/src/video.rs +++ b/rs/moq-ffi/src/video.rs @@ -1230,9 +1230,6 @@ impl MoqVideoConsumer { /// is rejected here; a recognized one the native backends can't open is rejected when the decoder /// opens, which is still before the first frame. fn video_config(catalog_video: crate::media::MoqVideo) -> Result { - if !catalog_video.enabled { - return Err(MoqError::NotFound); - } let codec: hang::catalog::VideoCodec = catalog_video.codec.parse().map_err(|_| MoqError::Unsupported)?; // Parsing is total: an unrecognized name becomes `Unknown` rather than failing. Reject it here // so a typo in the catalog is an error at subscribe, not an opaque backend failure later. @@ -1284,6 +1281,10 @@ impl MoqVideoConsumer { /// frame. Also fails with [`MoqError::Unsupported`] when /// [`surface`](MoqVideoDecoderOutput::surface) is set on a platform with no surface to expose. /// + /// A [disabled](crate::media::MoqVideo::enabled) rendition subscribes too, and decodes once its + /// publisher re-enables it. Don't pick one, but keep one already playing while no enabled + /// rendition is on offer. + /// /// `id` is the rendition's catalog key. Its [`track`](crate::media::MoqVideo::track), or `id` /// when unset, is subscribed in the broadcast its /// [`broadcast`](crate::media::MoqVideo::broadcast) reference names. @@ -1343,13 +1344,12 @@ mod decode_tests { } } + /// A consumer holds a disabled rendition it already plays, so one converts like any other. #[test] - fn video_config_refuses_disabled_renditions() { + fn video_config_accepts_disabled_renditions() { let mut config = catalog_video("avc1.64001f"); config.enabled = false; config.preference = -1; - assert!(matches!(video_config(config.clone()), Err(MoqError::NotFound))); - config.enabled = true; assert_eq!(video_config(config).unwrap().preference, -1); } diff --git a/rs/moq-mux/src/catalog/select.rs b/rs/moq-mux/src/catalog/select.rs index 4105e9f2446..5193d0b9b1a 100644 --- a/rs/moq-mux/src/catalog/select.rs +++ b/rs/moq-mux/src/catalog/select.rs @@ -2,7 +2,8 @@ //! //! [`Select`] wraps any [`Stream`] and applies a [`select::Broadcast`](crate::select::Broadcast) //! to each snapshot, dropping renditions that aren't selected before handing the -//! catalog to an exporter. +//! catalog on. Each snapshot holds what the previous one selected, so a rendition its +//! publisher disables stays while nothing else of its role is enabled. use std::task::{Poll, ready}; @@ -13,17 +14,24 @@ use crate::select; /// A [`Stream`] that keeps only the renditions a [`select::Broadcast`] selects. /// /// The selection is fixed at construction; every snapshot from the inner stream is -/// narrowed by it. Build one with [`Stream::select`](super::Stream::select) or +/// narrowed by it, holding the renditions the previous snapshot selected (see +/// [`select::Broadcast::retain`]). Build one with [`Stream::select`](super::Stream::select) or /// [`Select::new`]. pub struct Select { inner: S, selection: select::Broadcast, + /// What the last snapshot selected. + selected: select::Selected, } impl Select { /// Wrap `inner`, narrowing every snapshot by `selection`. pub fn new(inner: S, selection: select::Broadcast) -> Self { - Self { inner, selection } + Self { + inner, + selection, + selected: select::Selected::default(), + } } } @@ -33,7 +41,8 @@ impl Stream for Select { fn poll_next(&mut self, waiter: &kio::Waiter) -> Poll>>> { let next = ready!(self.inner.poll_next(waiter))?; Poll::Ready(Ok(next.map(|mut catalog| { - self.selection.retain(&mut catalog); + self.selection.retain(&mut catalog, &self.selected); + self.selected = select::Selected::of(&catalog); catalog }))) } @@ -48,14 +57,27 @@ mod tests { use super::super::hang::Catalog; use super::*; - /// A one-shot stream: yields its snapshot once, then ends. - struct Once(Option); + /// A finite stream: yields its snapshots in order, then ends. + struct Snapshots(std::collections::VecDeque); - impl Stream for Once { + impl Snapshots { + fn new(snapshots: impl IntoIterator) -> Self { + Self(snapshots.into_iter().collect()) + } + } + + impl Stream for Snapshots { type Ext = (); fn poll_next(&mut self, _: &kio::Waiter) -> Poll>> { - Poll::Ready(Ok(self.0.take())) + Poll::Ready(Ok(self.0.pop_front())) + } + } + + fn next(stream: &mut impl Stream) -> Catalog { + match stream.poll_next(&kio::Waiter::noop()) { + Poll::Ready(Ok(Some(c))) => c, + other => panic!("expected snapshot, got {other:?}"), } } @@ -76,12 +98,9 @@ mod tests { catalog.video.renditions = BTreeMap::from_iter(vec![h264("lo"), h264("hi")]); let selection = select::Broadcast::default().video(select::Video::default().name("hi")); - let mut stream = Once(Some(catalog)).select(selection); + let mut stream = Snapshots::new([catalog]).select(selection); - let out = match stream.poll_next(&kio::Waiter::noop()) { - Poll::Ready(Ok(Some(c))) => c, - other => panic!("expected snapshot, got {other:?}"), - }; + let out = next(&mut stream); assert_eq!(out.video.renditions.keys().collect::>(), vec!["hi"]); assert!(matches!(stream.poll_next(&kio::Waiter::noop()), Poll::Ready(Ok(None)))); } @@ -91,15 +110,36 @@ mod tests { let mut catalog = Catalog::default(); catalog.video.renditions = BTreeMap::from_iter(vec![h264("lo"), h264("hi")]); - let boxed: Box> = Box::new(Once(Some(catalog))); + let boxed: Box> = Box::new(Snapshots::new([catalog])); let selection = select::Broadcast::default().video(select::Video::default().name("lo")); let mut stream = boxed.select(selection); - let out = match stream.poll_next(&kio::Waiter::noop()) { - Poll::Ready(Ok(Some(c))) => c, - other => panic!("expected snapshot, got {other:?}"), - }; + let out = next(&mut stream); assert_eq!(out.video.renditions.keys().collect::>(), vec!["lo"]); assert!(matches!(stream.poll_next(&kio::Waiter::noop()), Poll::Ready(Ok(None)))); } + + /// A selected rendition its publisher disables stays selected until it re-enables, while one + /// disabled before it was ever selected is skipped. + #[test] + fn holds_what_the_previous_snapshot_selected() { + let snapshot = |enabled: bool| { + let mut catalog = Catalog::default(); + let (name, mut config) = h264("v"); + config.enabled = enabled; + catalog.video.renditions.insert(name, config); + catalog + }; + let selection = select::Broadcast::default().video(select::Video::default()); + + let mut stream = + Snapshots::new([snapshot(false), snapshot(true), snapshot(false), snapshot(true)]).select(selection); + assert!( + next(&mut stream).video.renditions.is_empty(), + "disabled before it was selected" + ); + assert!(next(&mut stream).video.renditions["v"].enabled); + assert!(!next(&mut stream).video.renditions["v"].enabled, "held while disabled"); + assert!(next(&mut stream).video.renditions["v"].enabled); + } } diff --git a/rs/moq-mux/src/select.rs b/rs/moq-mux/src/select.rs index c8421052eef..e4bcadfbbe8 100644 --- a/rs/moq-mux/src/select.rs +++ b/rs/moq-mux/src/select.rs @@ -4,15 +4,22 @@ //! additive: a default [`Broadcast`] selects *nothing*, and you opt a role in with //! [`video`](Broadcast::video) / [`audio`](Broadcast::audio). Within an opted-in //! role, an empty field matches everything; listing values keeps renditions matching -//! any one of them (a union within a field, intersected across fields). Per the hang spec, a -//! disabled rendition (catalog `enabled: false`) and one whose container `kind` this build does -//! not recognize are never selected. Selection runs on every snapshot, so a rendition that -//! re-enables is selected again. +//! any one of them (a union within a field, intersected across fields). A rendition whose +//! container `kind` this build does not recognize is never selected. +//! +//! A disabled rendition (catalog `enabled: false`) is not selected, with one exception: one +//! already [`Selected`] is held, still selected, while it stays in the catalog and no other +//! rendition of its role that this selection matches is enabled. So a muted speaker keeps its +//! listeners subscribed and its unmute plays at once, while a simulcast ladder, whose lowest rung +//! never disables, moves to an enabled rung. Selection runs on every snapshot, so a rendition +//! that re-enables is selected again. //! //! The same [`Broadcast`] drives selection at either end of the pipeline: narrowing //! a published catalog on the consume side (see [`catalog::Select`](crate::catalog::Select)), //! or choosing which tracks to publish on the import side. +use std::collections::{BTreeMap, BTreeSet}; + use hang::catalog::{AudioCodecKind, AudioConfig, VideoCodecKind, VideoConfig}; use crate::catalog::hang::{Catalog, CatalogExt, supported}; @@ -50,25 +57,66 @@ impl Broadcast { self.audio.is_some() } - /// Drop every rendition from `catalog` that isn't selected. - pub fn retain(&self, catalog: &mut Catalog) { + /// Drop every rendition from `catalog` that isn't selected, holding a disabled one in + /// `selected` (see the [module docs](self)). + /// + /// Pass [`Selected::default`] to select from scratch, or the renditions already in use, + /// such as [`Selected::of`] the previous snapshot this returned. + pub fn retain(&self, catalog: &mut Catalog, selected: &Selected) { match &self.video { - Some(video) => catalog - .video - .renditions - .retain(|name, config| video.matches(name, config)), + Some(video) => hold( + &mut catalog.video.renditions, + |name, config| video.matches(name, config), + |config| config.enabled, + &selected.video, + ), None => catalog.video.renditions.clear(), } match &self.audio { - Some(audio) => catalog - .audio - .renditions - .retain(|name, config| audio.matches(name, config)), + Some(audio) => hold( + &mut catalog.audio.renditions, + |name, config| audio.matches(name, config), + |config| config.enabled, + &selected.audio, + ), None => catalog.audio.renditions.clear(), } } } +/// The renditions a consumer already selected, by role and catalog name, which +/// [`Broadcast::retain`] holds while their publisher has them disabled. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Selected { + /// Selected video renditions. + pub video: BTreeSet, + /// Selected audio renditions. + pub audio: BTreeSet, +} + +impl Selected { + /// Every rendition `catalog` lists, such as the snapshot [`Broadcast::retain`] last returned. + pub fn of(catalog: &Catalog) -> Self { + Self { + video: catalog.video.renditions.keys().cloned().collect(), + audio: catalog.audio.renditions.keys().cloned().collect(), + } + } +} + +/// Keep the renditions of one role that match, dropping the disabled ones unless none that +/// match is enabled, in which case those in `selected` stay. +fn hold( + renditions: &mut BTreeMap, + matches: impl Fn(&str, &C) -> bool, + enabled: impl Fn(&C) -> bool, + selected: &BTreeSet, +) { + renditions.retain(|name, config| matches(name, config)); + let alternative = renditions.values().any(&enabled); + renditions.retain(|name, config| enabled(config) || (!alternative && selected.contains(name))); +} + /// Video rendition criteria. An empty field matches every rendition. #[derive(Clone, Debug, Default)] pub struct Video { @@ -90,8 +138,7 @@ impl Video { } fn matches(&self, name: &str, config: &VideoConfig) -> bool { - config.enabled - && supported(name, &config.container) + supported(name, &config.container) && (self.name.is_empty() || self.name.iter().any(|n| n == name)) && (self.codec.is_empty() || self.codec.contains(&config.codec.kind())) } @@ -118,8 +165,7 @@ impl Audio { } fn matches(&self, name: &str, config: &AudioConfig) -> bool { - config.enabled - && supported(name, &config.container) + supported(name, &config.container) && (self.name.is_empty() || self.name.iter().any(|n| n == name)) && (self.codec.is_empty() || self.codec.contains(&config.codec.kind())) } @@ -183,7 +229,7 @@ mod tests { #[test] fn default_selects_nothing() { let mut catalog = catalog(vec![h264("v")], vec![opus("a")]); - Broadcast::default().retain(&mut catalog); + Broadcast::default().retain(&mut catalog, &Selected::default()); assert!(catalog.video.renditions.is_empty()); assert!(catalog.audio.renditions.is_empty()); } @@ -191,7 +237,9 @@ mod tests { #[test] fn empty_axis_keeps_everything() { let mut catalog = catalog(vec![h264("lo"), h264("hi")], vec![opus("a")]); - Broadcast::default().video(Video::default()).retain(&mut catalog); + Broadcast::default() + .video(Video::default()) + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["hi", "lo"]); // Audio was never selected, so it's dropped. assert!(catalog.audio.renditions.is_empty()); @@ -202,7 +250,7 @@ mod tests { let mut catalog = catalog(vec![h264("lo"), h264("hi")], vec![]); Broadcast::default() .video(Video::default().name("hi")) - .retain(&mut catalog); + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["hi"]); } @@ -211,7 +259,7 @@ mod tests { let mut catalog = catalog(vec![], vec![opus("en"), opus("es"), opus("fr")]); Broadcast::default() .audio(Audio::default().name("en").name("es")) - .retain(&mut catalog); + .retain(&mut catalog, &Selected::default()); assert_eq!(audio_names(&catalog), vec!["en", "es"]); } @@ -220,7 +268,7 @@ mod tests { let mut catalog = catalog(vec![h264("a"), vp9("b"), h264("c")], vec![]); Broadcast::default() .video(Video::default().codec(VideoCodecKind::H264)) - .retain(&mut catalog); + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["a", "c"]); } @@ -229,7 +277,7 @@ mod tests { let mut catalog = catalog(vec![h264("a"), vp9("b")], vec![]); Broadcast::default() .video(Video::default().codec(VideoCodecKind::H264).codec(VideoCodecKind::VP9)) - .retain(&mut catalog); + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["a", "b"]); } @@ -239,31 +287,102 @@ mod tests { future.container = serde_json::from_str(r#"{"kind":"future"}"#).unwrap(); let mut catalog = catalog(vec![h264("known"), (name, future)], vec![]); - Broadcast::default().video(Video::default()).retain(&mut catalog); + Broadcast::default() + .video(Video::default()) + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["known"]); } + fn off_video((name, mut config): (String, VideoConfig)) -> (String, VideoConfig) { + config.enabled = false; + (name, config) + } + + fn off_audio((name, mut config): (String, AudioConfig)) -> (String, AudioConfig) { + config.enabled = false; + (name, config) + } + + fn selected(video: &[&str], audio: &[&str]) -> Selected { + Selected { + video: video.iter().map(|name| name.to_string()).collect(), + audio: audio.iter().map(|name| name.to_string()).collect(), + } + } + + fn everything() -> Broadcast { + Broadcast::default().video(Video::default()).audio(Audio::default()) + } + + /// Selecting from scratch skips a disabled rendition, so a listener who joins during a + /// pause subscribes on the unmute. + #[test] + fn disabled_not_selected_initially() { + let mut catalog = catalog(vec![h264("lo"), off_video(h264("hi"))], vec![off_audio(opus("a"))]); + everything().retain(&mut catalog, &Selected::default()); + assert_eq!(video_names(&catalog), vec!["lo"]); + assert!(catalog.audio.renditions.is_empty()); + } + + /// A selected rendition stays selected while disabled, if nothing else of its role is enabled. #[test] - fn disabled_never_selected() { - let (name, mut disabled) = h264("hi"); - disabled.enabled = false; - let (audio, mut muted) = opus("a"); - muted.enabled = false; - - let mut catalog = catalog(vec![h264("lo"), (name, disabled)], vec![(audio, muted)]); - let selection = Broadcast::default().video(Video::default()).audio(Audio::default()); - selection.retain(&mut catalog); + fn disabled_selection_is_held() { + let mut catalog = catalog(vec![off_video(h264("v"))], vec![off_audio(opus("a"))]); + everything().retain(&mut catalog, &selected(&["v"], &["a"])); + assert_eq!(video_names(&catalog), vec!["v"]); + assert_eq!(audio_names(&catalog), vec!["a"]); + } + + /// Only what was selected is held: a rendition that was disabled all along is not picked up. + #[test] + fn only_the_selection_is_held() { + let mut catalog = catalog(vec![], vec![off_audio(opus("a")), off_audio(opus("b"))]); + everything().retain(&mut catalog, &selected(&[], &["a"])); + assert_eq!(audio_names(&catalog), vec!["a"]); + } + + /// An enabled alternative wins, as a simulcast ladder's lowest rung does. + #[test] + fn enabled_alternative_wins() { + let mut catalog = catalog(vec![h264("lo"), off_video(h264("hi"))], vec![]); + everything().retain(&mut catalog, &selected(&["hi"], &[])); assert_eq!(video_names(&catalog), vec!["lo"]); + } + + /// A rendition the selection doesn't match is no alternative. + #[test] + fn unmatched_rendition_is_no_alternative() { + let mut catalog = catalog(vec![h264("lo"), off_video(h264("hi"))], vec![]); + Broadcast::default() + .video(Video::default().name("hi")) + .retain(&mut catalog, &selected(&["hi"], &[])); + assert_eq!(video_names(&catalog), vec!["hi"]); + } + + /// A held rendition removed from the catalog is dropped. + #[test] + fn removed_selection_is_dropped() { + let mut catalog = catalog(vec![], vec![off_audio(opus("b"))]); + everything().retain(&mut catalog, &selected(&[], &["a"])); assert!(catalog.audio.renditions.is_empty()); } + /// Each role holds on its own: enabled video does not release held audio. + #[test] + fn roles_hold_independently() { + let mut catalog = catalog(vec![h264("v")], vec![off_audio(opus("a"))]); + everything().retain(&mut catalog, &selected(&["v"], &["a"])); + assert_eq!(video_names(&catalog), vec!["v"]); + assert_eq!(audio_names(&catalog), vec!["a"]); + } + #[test] fn name_and_codec_intersect() { let mut catalog = catalog(vec![h264("hi"), vp9("hi2"), h264("lo")], vec![]); // name in {hi, hi2} AND codec H264 -> only "hi". Broadcast::default() .video(Video::default().name("hi").name("hi2").codec(VideoCodecKind::H264)) - .retain(&mut catalog); + .retain(&mut catalog, &Selected::default()); assert_eq!(video_names(&catalog), vec!["hi"]); } } diff --git a/rs/moq-rtmp/src/server.rs b/rs/moq-rtmp/src/server.rs index 65f256302da..da12b26ed3a 100644 --- a/rs/moq-rtmp/src/server.rs +++ b/rs/moq-rtmp/src/server.rs @@ -2448,7 +2448,7 @@ mod tests { fn best(catalog: &moq_mux::catalog::hang::Catalog, capabilities: &ClientCapabilities) -> Option { let select = play_selection(catalog, capabilities).unwrap(); let mut catalog = catalog.clone(); - select.retain(&mut catalog); + select.retain(&mut catalog, &Default::default()); catalog.video.ranked().next().map(|(name, _)| name.clone()) } @@ -2508,7 +2508,7 @@ mod tests { let legacy = ClientCapabilities::default(); let select = play_selection(&catalog, &legacy).expect("aac is playable"); let mut kept = catalog.clone(); - select.retain(&mut kept); + select.retain(&mut kept, &Default::default()); assert_eq!( kept.audio.renditions.keys().map(String::as_str).collect::>(), ["b"] @@ -2542,7 +2542,7 @@ mod tests { }; let select = play_selection(&ac3, &ClientCapabilities::new(0, FourCcSupport::default(), support)).unwrap(); let mut kept = ac3.clone(); - select.retain(&mut kept); + select.retain(&mut kept, &Default::default()); assert_eq!( kept.audio.renditions.keys().map(String::as_str).collect::>(), ["a"] From 9afb3528f50bee18dc9651e10baf300193650ac6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 16:18:42 -0700 Subject: [PATCH 3/6] chore(dart): regenerate bindings Co-Authored-By: Claude Opus 5.5 --- dart/moq_ffi/lib/src/moq.dart | 6 ++---- dart/moq_ffi/lib/src/uniffi_runtime.dart | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index 92a0e4d32f3..71ac59575f0 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -8,9 +8,7 @@ import "dart:ffi"; import "dart:io" show Platform, File, Directory; import "dart:isolate"; import "dart:typed_data"; - import "package:ffi/ffi.dart"; - import "uniffi_runtime.dart"; export "uniffi_runtime.dart"; @@ -21169,7 +21167,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqaudioconsumer_subscribe() != - 43304) { + 10630) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqaudioencoder_new() != 52616) { @@ -21253,7 +21251,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqvideoconsumer_subscribe() != - 44639) { + 15507) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_constructor_moqvideoencoder_new() != 22531) { diff --git a/dart/moq_ffi/lib/src/uniffi_runtime.dart b/dart/moq_ffi/lib/src/uniffi_runtime.dart index 9018eb4e97e..c61178325f7 100644 --- a/dart/moq_ffi/lib/src/uniffi_runtime.dart +++ b/dart/moq_ffi/lib/src/uniffi_runtime.dart @@ -8,7 +8,6 @@ import "dart:ffi"; import "dart:io" show Platform, File, Directory; import "dart:isolate"; import "dart:typed_data"; - import "package:ffi/ffi.dart"; class UniffiInternalError implements Exception { @@ -251,7 +250,6 @@ mixin FfiConverterPrimitive on FfiConverter { @override T lower(T value) => value; } - Uint8List createUint8ListFromInt(int value) { int length = value.bitLength ~/ 8 + 1; if (length != 4 && length != 8) { From 20aa6b376ee484a5f1e0108df2fba8efa63d43a1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 16:24:55 -0700 Subject: [PATCH 4/6] test(ffi): a held disabled rendition subscribes; quest: rewrite hold-rs to what remains Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 2 +- quest/m1/push-to-talk/hold-rs.md | 35 ++++++++--------- rs/moq-ffi/src/test.rs | 64 ++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+), 18 deletions(-) diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 42008789dad..63759cde833 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -46,7 +46,7 @@ Record both in the PR. ## Required - [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - watch keeps a disabled audio or video rendition subscribed, and the hang spec says "should not select" -- [Rust consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moq-cli play, moq-ffi and moq-mux select follow the same rule +- [Rust consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moqsrc follows the same rule as moq play, moq-ffi and moq-mux select, and the exports settle whether they do - [Re-enable without a probe](/quest/m1/push-to-talk/reenable.md) - the JS audio encoder reuses its probe, so an unmute flips the catalog at once and keeps capture timestamps - [`muted` and `invisible` keep capture open](/quest/m1/push-to-talk/mute.md) - the element disables only the encoder, so unmute skips getUserMedia and AudioContext setup diff --git a/quest/m1/push-to-talk/hold-rs.md b/quest/m1/push-to-talk/hold-rs.md index c48c8cb6504..cc7a20f0026 100644 --- a/quest/m1/push-to-talk/hold-rs.md +++ b/quest/m1/push-to-talk/hold-rs.md @@ -12,24 +12,25 @@ Decided 2026-10-28 (maintainer): the hold applies in every first-party consumer, with the same rule as the JS quest. Initial selection skips disabled renditions, an enabled alternative wins, and a removed rendition is dropped. -- `rs/moq-mux/src/select.rs:93,121`: the selection filters require - `config.enabled`. -- `rs/moq-cli/src/play/media.rs:192-193,445-452`: `stop_disabled` drops demand - on the playing rendition. -- `rs/moq-ffi/src/audio.rs:613-616`: a disabled rendition returns `NotFound`. - It should keep a held one, and the wrappers' docs follow. - -[moq-audio follows the grant](/quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md) -already expects a held listener to keep demand on a single audio rendition -disabled below the codec floor. - -The spec wording lands with whichever hold quest lands first. - -Tests: the JS quest's cases, in moq-mux select and moq-ffi. - -Public API: moq-ffi returns a held disabled rendition where it returned -`NotFound`. Wire: none. +Already holding: `moq_mux::select::Broadcast::retain` and `catalog::Select`, +`moq play`, moq-ffi's audio and video consumers, and moq-c's `moq_consume_*`. + +Left: + +- moq-gst `moqsrc`: `reconcile` (`rs/moq-gst/src/source/imp.rs`) stops a + disabled rendition's reader and holds its pad idle. Keep the reader running + while no rendition of its kind is enabled. The disable tests + (`a_reenabled_rendition_resumes_on_the_same_pad`, + `a_reenabled_sibling_rendition_plays_the_next_instance_after_a_finish`) + change with it. +- Open: whether the exports hold too. `Source::retain_selectable` + (`rs/moq-mux/src/source.rs`) drops a disabled rendition for every + fMP4/TS/MKV/FLV/Annex B export, and the HLS export, WebRTC egress and RTMP + play follow the flag on their own. A held track sends nothing, which an + interleaving muxer or an HLS player may wait on, so the recommendation is + that exports keep dropping it. ## Related - [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - the same rule in JS +- [OBS source picks renditions by the catalog contract](/quest/m2/obs-source-renditions.md) - holds through moq-c diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index 66a463c41a1..871e6b11a34 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -671,6 +671,70 @@ fn tone_power(samples: &[f32], frequency: f64) -> f64 { previous * previous + older * older - coefficient * previous * older } +/// A listener holding a muted rendition passes its disabled config, which subscribes and plays +/// whatever the publisher sends once it unmutes. +#[cfg(feature = "audio")] +#[tokio::test] +async fn audio_subscribes_a_held_disabled_rendition() { + use crate::audio::*; + let broadcast = MoqBroadcastProducer::new().unwrap(); + let consumer = broadcast.consume().unwrap(); + let catalogs = MoqMediaCatalogConsumer::subscribe(&consumer).await.unwrap(); + let catalog = MoqMediaCatalogProducer::new(broadcast.clone()).unwrap(); + let audio = MoqAudioProducer::new( + &catalog, + MoqAudioProducerOptions { + input: MoqAudioEncoderInput { + format: MoqAudioSampleFormat::F32, + sample_rate: 48_000, + channels: 2, + }, + output: MoqAudioEncoderOutput { + codec: MoqAudioCodec::opus(), + sample_rate: None, + channels: None, + bitrate: None, + frame_duration: Some(Duration::from_millis(20)), + }, + track: Some("voice".into()), + bandwidth: None, + }, + ) + .unwrap(); + let snapshot = catalogs.next().await.unwrap().unwrap(); + let mut muted = snapshot.audio["voice"].clone(); + muted.enabled = false; + let decoder = MoqAudioConsumer::subscribe( + &consumer, + "voice".into(), + muted, + MoqAudioDecoderOutput { + max_delay: Some(Duration::from_secs(30)), + ..audio_output() + }, + ) + .await + .unwrap(); + + for index in 0..4 { + audio + .write(MoqAudioFrame { + timestamp_us: index * 20_000, + data: audio_tone(index as usize * 960, 960, 2), + }) + .unwrap(); + } + audio.finish().unwrap(); + let mut frames = 0; + while let Some(read) = tokio::time::timeout(TIMEOUT, decoder.next()).await.unwrap().unwrap() { + if matches!(read, MoqAudioRead::Frame { .. }) { + frames += 1; + } + } + assert!(frames > 0, "the held rendition played nothing"); + broadcast.close().unwrap(); +} + /// PCM written and finished from separate host threads survives the binding's locking and drain. #[cfg(feature = "audio")] #[tokio::test] From f2653852198cbb4859a4297acd5579638762b723 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 19:19:09 -0700 Subject: [PATCH 5/6] chore(quest): narrow hold-rs to moqsrc; exports keep dropping a disabled rendition Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk/README.md | 10 ++++---- quest/m1/push-to-talk/hold-rs.md | 40 +++++++++++++++----------------- 2 files changed, 25 insertions(+), 25 deletions(-) diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 913e9e8a7ae..7737a885fa7 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -22,9 +22,11 @@ Decided 2026-10-28 (maintainer): - **Hold, not DTX.** DTX is for silence the app can't predict. Push-to-talk knows when it is silent, and a held subscription costs no media bytes and no encoder CPU. DTX is off on main anyway ([#4919](https://github.com/moq-dev/moq/pull/4919)). -- **Consumers hold a disabled rendition**, audio and video, by default, in - every first-party consumer. The hang spec weakens from "must not select" to - "should not select" a disabled rendition. +- **Players hold a disabled rendition**, audio and video, by default, in + every first-party player. Exports keep dropping it, since a held track + sends nothing and an interleaving muxer or HLS player may wait on it. The + hang spec weakens from "must not select" to "should not select" a disabled + rendition. - **The publisher keeps its probe result across a disable**, and the element's `muted` and `invisible` leave capture open. - **Encoding no longer waits for demand** @@ -46,7 +48,7 @@ Record both in the PR. ## Required - [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - watch keeps a disabled audio or video rendition subscribed, and the hang spec says "should not select" -- [Rust consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moqsrc follows the same rule as moq play, moq-ffi and moq-mux select, and the exports settle whether they do +- [moqsrc holds a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moqsrc follows the same rule as moq play, moq-ffi and moq-mux select - [`muted` and `invisible` keep capture open](/quest/m1/push-to-talk/mute.md) - the element disables only the encoder, so unmute skips getUserMedia and AudioContext setup ## Closes diff --git a/quest/m1/push-to-talk/hold-rs.md b/quest/m1/push-to-talk/hold-rs.md index cc7a20f0026..32e3db5bac8 100644 --- a/quest/m1/push-to-talk/hold-rs.md +++ b/quest/m1/push-to-talk/hold-rs.md @@ -1,36 +1,34 @@ -# [M] Rust consumers hold a disabled rendition +# [M] moqsrc holds a disabled rendition ## Goal -Every first-party Rust consumer keeps its selected rendition, and its +Every first-party Rust player keeps its selected rendition, and its subscription, while the publisher has it disabled and no enabled alternative -exists, as [@moq/watch does](/quest/m1/push-to-talk/hold-js.md). +exists, as `@moq/watch` does. Exports keep dropping a disabled rendition. ## Plan -Decided 2026-10-28 (maintainer): the hold applies in every first-party -consumer, with the same rule as the JS quest. Initial selection skips disabled -renditions, an enabled alternative wins, and a removed rendition is dropped. +Decided 2026-10-28 (maintainer): the hold applies in every first-party player. +The rule matches `@moq/watch`: initial selection skips disabled renditions, an +enabled alternative wins, and a removed rendition is dropped. Already holding: `moq_mux::select::Broadcast::retain` and `catalog::Select`, `moq play`, moq-ffi's audio and video consumers, and moq-c's `moq_consume_*`. -Left: - -- moq-gst `moqsrc`: `reconcile` (`rs/moq-gst/src/source/imp.rs`) stops a - disabled rendition's reader and holds its pad idle. Keep the reader running - while no rendition of its kind is enabled. The disable tests - (`a_reenabled_rendition_resumes_on_the_same_pad`, - `a_reenabled_sibling_rendition_plays_the_next_instance_after_a_finish`) - change with it. -- Open: whether the exports hold too. `Source::retain_selectable` - (`rs/moq-mux/src/source.rs`) drops a disabled rendition for every - fMP4/TS/MKV/FLV/Annex B export, and the HLS export, WebRTC egress and RTMP - play follow the flag on their own. A held track sends nothing, which an - interleaving muxer or an HLS player may wait on, so the recommendation is - that exports keep dropping it. +Left: moq-gst `moqsrc`. `reconcile` (`rs/moq-gst/src/source/imp.rs`) stops a +disabled rendition's reader and holds its pad idle. Keep the reader running +while no rendition of its kind is enabled. The disable tests +(`a_reenabled_rendition_resumes_on_the_same_pad`, +`a_reenabled_sibling_rendition_plays_the_next_instance_after_a_finish`) +change with it. + +Decided 2026-10-28 (maintainer): exports do not hold. That covers every +fMP4/TS/MKV/FLV/Annex B export through `Source::retain_selectable` +(`rs/moq-mux/src/source.rs`), plus the HLS export, WebRTC egress and RTMP +play. A held track sends nothing, so an interleaving muxer or an HLS player +may wait on it, and an export has no unmute latency at stake the way a +listener does. ## Related -- [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - the same rule in JS - [OBS source picks renditions by the catalog contract](/quest/m2/obs-source-renditions.md) - holds through moq-c From d879345960bd195ecaae19beda7cca311e6f74d3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 21:40:03 -0700 Subject: [PATCH 6/6] docs: state moq play's filter condition and moqsrc's rule precisely Co-Authored-By: Claude Opus 5.5 --- doc/bin/cli.md | 6 +++++- quest/m1/push-to-talk/README.md | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/bin/cli.md b/doc/bin/cli.md index ce308011bb5..845a153df79 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -236,7 +236,11 @@ audio-device dependencies: cargo install moq-cli --no-default-features --features "quic,websocket,play,openh264" ``` -`moq play` skips disabled renditions when it picks one. If none are enabled, it waits for a later catalog. Disabling the current rendition switches to an enabled one of its kind; with none, it stays subscribed, so re-enabling it (an unmute) plays at once. +`moq play` skips disabled renditions when it picks one. If no enabled rendition +matches its name and codec filters, it waits for a later catalog. Disabling the +current rendition switches to an enabled one of its kind that matches the +filters; with none, it stays subscribed, so re-enabling it (an unmute) plays at +once. ## Capture diff --git a/quest/m1/push-to-talk/README.md b/quest/m1/push-to-talk/README.md index 7737a885fa7..ba19730676c 100644 --- a/quest/m1/push-to-talk/README.md +++ b/quest/m1/push-to-talk/README.md @@ -48,7 +48,7 @@ Record both in the PR. ## Required - [JS consumers hold a disabled rendition](/quest/m1/push-to-talk/hold-js.md) - watch keeps a disabled audio or video rendition subscribed, and the hang spec says "should not select" -- [moqsrc holds a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moqsrc follows the same rule as moq play, moq-ffi and moq-mux select +- [moqsrc holds a disabled rendition](/quest/m1/push-to-talk/hold-rs.md) - moqsrc keeps its reader running under the selection rule moq play and moq-mux already apply - [`muted` and `invisible` keep capture open](/quest/m1/push-to-talk/mute.md) - the element disables only the encoder, so unmute skips getUserMedia and AudioContext setup ## Closes