From 3a274710366160200af5ced6548222587f384ebf Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 05:09:39 -0700 Subject: [PATCH 1/7] chore(quest): claim quest/m2/stats/rust.md Quest-Claim: f903613a58238ff5 From cf8e8dc27ad59b035d9564aa58d57d7aa078ade3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 05:21:02 -0700 Subject: [PATCH 2/7] feat(cli): report capture stats and solicit feedback from capture `import capture` takes `--report` and `--echo-path`. The encoders report what the catalog cannot count itself: the video capture loop counts the frames the camera replaced before the encoder reached them as `skipped`, and both encoders report their target bitrate, through new `container::Producer::report_skipped` and `report_bitrate` hooks into the catalog's `Sent` registry. A reservation held until the encoders start keeps both sections on the first catalog. The quest is rewritten to what remains: `moq play --echo`, the gateways and `archive`, and audio capture skips. Co-Authored-By: Claude Opus 5.5 --- doc/bin/cli.md | 5 +- doc/concept/stats.md | 5 +- quest/m2/stats/README.md | 4 +- quest/m2/stats/rust.md | 50 ++++++++++--------- rs/moq-audio/src/encode/producer.rs | 26 +++++++++- rs/moq-cli/src/args.rs | 24 ++++------ rs/moq-cli/src/publish.rs | 45 +++++++++++++---- rs/moq-mux/src/codec/h264/import.rs | 12 +++++ rs/moq-mux/src/codec/h265/import.rs | 12 +++++ rs/moq-mux/src/container/producer.rs | 16 +++++++ rs/moq-mux/src/stats.rs | 31 ++++++++++++ rs/moq-video/src/capture/channel.rs | 22 ++++++++- rs/moq-video/src/capture/mod.rs | 6 +++ rs/moq-video/src/encode/producer.rs | 72 +++++++++++++++++++++++++--- 14 files changed, 268 insertions(+), 62 deletions(-) diff --git a/doc/bin/cli.md b/doc/bin/cli.md index ac6b3650772..8d826c5d63a 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -500,8 +500,9 @@ of `--follow`. `import --report` names a track in the catalog's `stats` section and reports what each rendition has written, about once a second while someone subscribes, -with the connection's RTT, estimated send rate, and loss. It covers the stdin imports; -capture, the gateways, `archive`, and `ts --program all` refuse it. +with the connection's RTT, estimated send rate, and loss. It covers the stdin imports +and `capture`, whose renditions also report the encoder's target bitrate and the +video frames it fell behind on; the gateways, `archive`, and `ts --program all` refuse it. `import --echo-path ` solicits feedback from the same imports: it sets the catalog's `echo` path, resolved against the broadcast like a relative URL, diff --git a/doc/concept/stats.md b/doc/concept/stats.md index a7954b0bfd8..d2dde7a5af0 100644 --- a/doc/concept/stats.md +++ b/doc/concept/stats.md @@ -147,8 +147,9 @@ A viewer's loss counters describe its own uplink, so they are reported but not graded. Loss and headroom divide the connection's transport counters by the report's own media, so they hold when the broadcast is the only one its connection publishes. The transport limits were measured on one paced TS feed -through a shaped link over five-second intervals; no CLI reporter emits -`skipped`, `late`, `errors`, or `stalled`, so their limits are not measured. +through a shaped link over five-second intervals. Only `import capture` +emits `skipped`, and no CLI reporter emits `late`, `errors`, or `stalled`, so +those limits are not measured. A publisher whose link falls below its media rate sends no report at all, since its report track sits below every media track. Its health goes stale diff --git a/quest/m2/stats/README.md b/quest/m2/stats/README.md index 81d4fc0110b..15c5bda6157 100644 --- a/quest/m2/stats/README.md +++ b/quest/m2/stats/README.md @@ -111,8 +111,8 @@ connection view and the dashboard flow. ## Required -- [Rust reporters](/quest/m2/stats/rust.md) - the player, encoders, and - gateways publish stats and feedback, as the CLI remuxes do +- [Rust reporters](/quest/m2/stats/rust.md) - the player and gateways + publish stats and feedback, as the CLI remuxes and capture do - [Browser reporters](/quest/m2/stats/js.md) - `` publishes stats and `` publishes feedback - [Preflight](/quest/m2/stats/preflight.md) - a bounded test run over a diff --git a/quest/m2/stats/rust.md b/quest/m2/stats/rust.md index a6105d56dc7..809ba2102e1 100644 --- a/quest/m2/stats/rust.md +++ b/quest/m2/stats/rust.md @@ -1,41 +1,46 @@ -# [M] The Rust player, encoders, and gateways report stats and feedback +# [M] The Rust player and gateways report stats and feedback ## Goal -- `moq import capture --report` announces a stats track in the catalog and - fills it with what its encoders send, `skipped` and `bitrate` included. - `moq play`, given `--echo `, publishes an `.echo` broadcast under each watched catalog's echo path and fills its feedback track with what it received, decoded, and played. - The gateway imports (`hls`, `rtmp`, `srt`, `rtsp`, `rtc`) and `archive` take - `--report`. -- Capture, the gateways, and `archive` take `--echo-path`, which sets the - catalog's `echo` section as it does for the stdin imports. + `--report` and `--echo-path`, which set the catalog's `stats` and `echo` + sections as they do for the stdin imports and capture. +- `import capture --report` counts the audio frames a microphone capture + dropped in `skipped`, as it already does for video. ## Plan -- `moq import --report` for the stdin imports, `moq export --echo`, and the - `export report` and `export echo` sinks already report what the `moq-mux` - importers write and exporters read (`rs/moq-mux/src/stats.rs`, - `rs/moq-cli/src/report.rs`), and `import --echo-path` solicits feedback for - the stdin imports. This quest adds the counters they lack and reuses the - rest; `import --report` and `--echo-path` refuse capture and the gateways - until it lands. +- `import --report` and `--echo-path` already cover the stdin imports and + capture, and `export --echo` and the `export report` and `export echo` sinks + already read and print the reports (`rs/moq-mux/src/stats.rs`, + `rs/moq-cli/src/report.rs`). The catalog's `Sent` registry counts what every + track minted under it writes, and an encoder reports its skips and target + through `container::Producer::report_skipped` and `report_bitrate`. This quest + adds the counters still missing and reuses the rest; `--report` and + `--echo-path` refuse the gateways until it lands. - Counters come from the source, per track, as cumulative `stats()` snapshots: no callback, no per-frame lock. - - Publisher: the `moq-video` and `moq-audio` encode producers create their - tracks directly on the broadcast, so they count frames, bytes, keyframes, - and drops themselves, and expose the target bitrate. - Viewer: the decode and render paths count received, decoded, late, stalled, underruns, and errors. `underruns` in - `rs/moq-audio/src/playback` is already counted privately. -- The gateways build their catalogs in their own crates, so each needs a way - to name the stats track and the echo path before its first catalog. - Capture publishes its catalog as its encoders start, so both sections reach - it before they do. + `rs/moq-audio/src/playback` is already counted privately. The player reads + video through `moq_mux::container::Consumer` directly, so it bypasses the + `Source`'s `Received` registry today. + - Audio capture: `capture::Samples::gap` only flags that buffers were + dropped. Counting `skipped` needs the drop size, converted to codec frames. +- The gateways build their catalogs in their own crates (`moq-rtmp`, + `moq-srt`, `moq-rtc`, `moq-archive`; `hls` and `rtsp` build theirs in + `rs/moq-cli`), so each needs a way to name the stats track and the echo path + before its first catalog, and a reporter to run beside each session's + catalog. The report writer lives in `rs/moq-cli/src/report.rs` and reads the + transport from a `moq_tokio` connection monitor, which the gateway crates do + not see. - `moq play --echo` reconciles each watched catalog's `echo` section as `export --echo` does, and refuses `--catalog-format msf` the same way: the - MSF catalog does not carry an `echo` solicitation. + MSF catalog does not carry an `echo` solicitation. `play` attaches only the + consume direction today, so it also needs the publish direction. - The media test captures with `--report` and plays with `--echo` against a publisher that solicits feedback. It asserts the encoder's frame count and that the player's newest arrival advances. @@ -43,7 +48,6 @@ broadcast yet, so [Encrypted native CLI](/quest/m2/e2ee/cli.md) refuses `--report` and `--echo` when it adds one: they would publish rendition IDs and per-track counters in plaintext beside it. - - Decided 2026-10-22 (maintainer): the publisher's solicit option moved here from [encoder feedback](/quest/m3/stats-encoder-feedback.md), which keeps only the adaptation loop and reuses `--echo-path`. diff --git a/rs/moq-audio/src/encode/producer.rs b/rs/moq-audio/src/encode/producer.rs index 6b067bc9a74..e73c9741550 100644 --- a/rs/moq-audio/src/encode/producer.rs +++ b/rs/moq-audio/src/encode/producer.rs @@ -188,8 +188,10 @@ impl Reserved { /// Spend the reservation on a registered encoder, publishing through the track. pub(crate) fn encode(self, registered: Registered) -> Producer { + let bitrate = registered.encoder.bitrate(); + self.track.report_bitrate(bitrate); Producer { - bitrate: registered.encoder.bitrate(), + bitrate, encoder: Some(registered.encoder), input: registered.input, remix: registered.remix, @@ -297,6 +299,7 @@ impl Producer { let encoder = self.encoder.as_mut().ok_or(moq_net::Error::Closed)?; encoder.set_bitrate(bitrate)?; self.bitrate = encoder.bitrate(); + self.track.report_bitrate(self.bitrate); Ok(()) } @@ -592,6 +595,27 @@ mod tests { } } + /// The catalog's stats follow the encoder's target from the first rendition on. + #[tokio::test] + async fn stats_follow_the_target_bitrate() { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = moq_mux::catalog::Producer::new(&mut broadcast, moq_mux::catalog::Config::default()).unwrap(); + let options = Options { + track: Some("audio".into()), + ..Options::default() + }; + let mut producer = Producer::new(&mut broadcast, catalog.clone(), Input::default(), &options).unwrap(); + let reported = || catalog.stats().remove("audio").and_then(|stats| stats.bitrate); + assert_eq!(reported(), Some(producer.bitrate().as_bps())); + + let initial = producer.bitrate(); + producer + .set_bitrate(moq_net::bandwidth::Rate::from_bps(initial.as_bps() / 2)) + .unwrap(); + assert_ne!(producer.bitrate(), initial); + assert_eq!(reported(), Some(producer.bitrate().as_bps())); + } + #[tokio::test] async fn demand_follows_subscribers_and_closes_with_the_producer() { let mut broadcast = moq_net::broadcast::Info::new().produce(); diff --git a/rs/moq-cli/src/args.rs b/rs/moq-cli/src/args.rs index 049e40ec9e8..bb956c1f0d8 100644 --- a/rs/moq-cli/src/args.rs +++ b/rs/moq-cli/src/args.rs @@ -326,7 +326,7 @@ impl Invocation { for command in &self.stages { if let Command::Import(import) = command && import.report - && let Some(reason) = import.source.report_refusal() + && let Some(reason) = import.source.catalog_refusal() { anyhow::bail!("--report names a track in the catalog the import builds, but {reason}"); } @@ -809,13 +809,14 @@ pub struct Import { latency_max: Option, /// Name a media report track in the catalog's `stats` section and fill it with what each - /// track has written, about once a second while subscribed. The stdin imports only. + /// track has written, about once a second while subscribed. The stdin imports and capture + /// only. #[usage(long)] pub report: bool, /// Solicit viewer feedback in the catalog: `export --echo ` publishes `.echo` /// under this prefix, resolved against the broadcast like a relative URL (`viewers` from - /// `room/event` is `room/viewers`). The stdin imports only. + /// `room/event` is `room/viewers`). The stdin imports and capture only. #[usage(long, value_name = "PATH")] pub echo_path: Option, @@ -887,7 +888,8 @@ impl ImportSource { } /// Why the CLI can't add a section to this source's first catalog, or `None` when it can: - /// a stdin import's one catalog waits for its importer's tracks. + /// a stdin import's one catalog waits for its importer's tracks, and a capture's for its + /// encoders. fn catalog_refusal(&self) -> Option<&'static str> { match self { Self::Ts(args) if args.program == Some(TsProgram::All) => { @@ -897,21 +899,11 @@ impl ImportSource { Some("the gateway and archive imports build their own catalogs") } #[cfg(feature = "capture")] - Self::Capture(_) => Some("`import capture` publishes its catalog as its encoders start"), + Self::Capture(_) => None, Self::Avc3 | Self::Fmp4(_) | Self::Ts(_) | Self::Flv => None, } } - /// Why `--report` can't report this source, or `None` when it can: the CLI builds the one - /// catalog of a stdin import, whose tracks `moq-mux` counts. - fn report_refusal(&self) -> Option<&'static str> { - #[cfg(feature = "capture")] - if matches!(self, Self::Capture(_)) { - return Some("`import capture` encodes, and its encoders do not count yet"); - } - self.catalog_refusal() - } - /// Why `--linger` can't apply to this source, or `None` when it can: only a source with a /// clean end has a finished broadcast to keep up. fn linger_refusal(&self) -> Option<&'static str> { @@ -1714,6 +1706,8 @@ mod tests { ] { import(flags, source).validate().unwrap(); } + #[cfg(feature = "capture")] + import(flags, &["capture"]).validate().unwrap(); for source in [ ["ts", "--program", "all"].as_slice(), &["hls", "http://origin/master.m3u8"], diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index 068c293f6d1..f6df80ad41e 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -296,6 +296,9 @@ enum Source { catalog: moq_mux::catalog::Producer, video: Option, audio: Option<(moq_audio::capture::Config, moq_audio::encode::Options)>, + /// Withholds the catalog until [`Publish::run`] starts the encoders, so the `stats` and + /// `echo` sections set before then ride the first catalog rather than an empty one. + gate: moq_mux::catalog::Reserved, }, } @@ -494,6 +497,7 @@ impl Publish { Ok(Self { source: Source::Capture { broadcast: broadcast.clone(), + gate: catalog.reserve(), catalog, video, audio, @@ -512,13 +516,17 @@ impl Publish { /// /// Call it before the importer declares its tracks, so the section rides the first catalog. pub fn with_report(mut self, connection: Option) -> anyhow::Result { - let (Some(broadcast), Source::Stream { catalog, .. }) = (self.broadcast.as_mut(), &self.source) else { - anyhow::bail!("--report reports a stdin import's one catalog"); + let Some(broadcast) = self.broadcast.as_mut() else { + anyhow::bail!("`import ts --program all` builds a catalog per program"); }; - let report = match catalog { - PublishCatalog::Media(catalog) => crate::report::Stats::new(broadcast, catalog, connection)?, - PublishCatalog::Ts(catalog) => crate::report::Stats::new(broadcast, catalog, connection)?, - PublishCatalog::TsPrograms => anyhow::bail!("`import ts --program all` builds a catalog per program"), + let report = match &self.source { + Source::Stream { catalog, .. } => match catalog { + PublishCatalog::Media(catalog) => crate::report::Stats::new(broadcast, catalog, connection)?, + PublishCatalog::Ts(catalog) => crate::report::Stats::new(broadcast, catalog, connection)?, + PublishCatalog::TsPrograms => anyhow::bail!("`import ts --program all` builds a catalog per program"), + }, + #[cfg(feature = "capture")] + Source::Capture { catalog, .. } => crate::report::Stats::new(broadcast, catalog, connection)?, }; self.report = Some(report); Ok(self) @@ -531,7 +539,10 @@ impl Publish { let catalog = match &mut self.source { Source::Stream { catalog, .. } => catalog, #[cfg(feature = "capture")] - Source::Capture { .. } => anyhow::bail!("--echo-path solicits feedback in a stdin import's one catalog"), + Source::Capture { catalog, .. } => { + catalog.mutate(|catalog| catalog.echo = Some(echo))?; + return Ok(self); + } }; match catalog { PublishCatalog::Media(catalog) => catalog.mutate(|catalog| catalog.echo = Some(echo))?, @@ -585,7 +596,10 @@ impl Publish { catalog, video, audio, + gate, } => { + drop(gate); + let mut report = self.report; // Each enabled medium publishes its own track onto the shared // broadcast + catalog. Both stamp frames on the catalog's // advertised clock, so HLS/DASH wall times match the mapping on @@ -619,8 +633,21 @@ impl Publish { } }; - tokio::try_join!(video_fut, audio_fut)?; - Ok(()) + let capture = async { tokio::try_join!(video_fut, audio_fut).map(|_| ()) }; + // Written beside the encoders, so a capture waiting on its first viewer still reports. + let result = match report.as_mut() { + Some(report) => tokio::select! { + result = capture => result, + never = report.run() => match never {}, + }, + None => capture.await, + }; + match (&result, report) { + (Ok(()), Some(mut report)) => report.finish(), + (Err(err), Some(report)) => report.abort(moq_net::Error::Transport(err.to_string())), + (_, None) => {} + } + result } } } diff --git a/rs/moq-mux/src/codec/h264/import.rs b/rs/moq-mux/src/codec/h264/import.rs index 842f26a252f..8dc96c04af5 100644 --- a/rs/moq-mux/src/codec/h264/import.rs +++ b/rs/moq-mux/src/codec/h264/import.rs @@ -166,6 +166,18 @@ impl Import { Ok(()) } + /// Report frames the encoder feeding this track skipped. See + /// [`Producer::report_skipped`](crate::container::Producer::report_skipped). + pub fn report_skipped(&self, frames: u64) { + self.track.report_skipped(frames); + } + + /// Report the target bitrate of the encoder feeding this track. See + /// [`Producer::report_bitrate`](crate::container::Producer::report_bitrate). + pub fn report_bitrate(&self, bitrate: moq_net::bandwidth::Rate) { + self.track.report_bitrate(bitrate); + } + /// Close the current group and open the next one at `sequence`. pub fn seek(&mut self, sequence: u64) -> Result<()> { self.track.seek(sequence)?; diff --git a/rs/moq-mux/src/codec/h265/import.rs b/rs/moq-mux/src/codec/h265/import.rs index 97c9a13066f..bbda5aa4757 100644 --- a/rs/moq-mux/src/codec/h265/import.rs +++ b/rs/moq-mux/src/codec/h265/import.rs @@ -169,6 +169,18 @@ impl Import { Ok(()) } + /// Report frames the encoder feeding this track skipped. See + /// [`Producer::report_skipped`](crate::container::Producer::report_skipped). + pub fn report_skipped(&self, frames: u64) { + self.track.report_skipped(frames); + } + + /// Report the target bitrate of the encoder feeding this track. See + /// [`Producer::report_bitrate`](crate::container::Producer::report_bitrate). + pub fn report_bitrate(&self, bitrate: moq_net::bandwidth::Rate) { + self.track.report_bitrate(bitrate); + } + /// Close the current group and open the next one at `sequence`. pub fn seek(&mut self, sequence: u64) -> Result<()> { self.track.seek(sequence)?; diff --git a/rs/moq-mux/src/container/producer.rs b/rs/moq-mux/src/container/producer.rs index 08d482f472f..e9a252786d1 100644 --- a/rs/moq-mux/src/container/producer.rs +++ b/rs/moq-mux/src/container/producer.rs @@ -384,6 +384,22 @@ where self } + /// Report `frames` frames the source produced but never handed to this track, such as ones an + /// encoder fell behind on, in its catalog's [`stats`](crate::catalog::Producer::stats). + pub fn report_skipped(&self, frames: u64) { + if let Some(sent) = &self.sent { + sent.skip(frames); + } + } + + /// Report the encoder's latest target bitrate in its catalog's + /// [`stats`](crate::catalog::Producer::stats). The catalog's `bitrate` stays the maximum. + pub fn report_bitrate(&self, bitrate: moq_net::bandwidth::Rate) { + if let Some(sent) = &self.sent { + sent.target(bitrate); + } + } + /// Count a written frame. Only video keyframes count as keyframes: elsewhere the bit only /// marks where a group starts. fn count(&self, bytes: usize, keyframe: bool) { diff --git a/rs/moq-mux/src/stats.rs b/rs/moq-mux/src/stats.rs index a46724ace88..1a82833844c 100644 --- a/rs/moq-mux/src/stats.rs +++ b/rs/moq-mux/src/stats.rs @@ -17,6 +17,9 @@ pub(crate) struct Sent { frames: AtomicU64, bytes: AtomicU64, keyframes: AtomicU64, + skipped: AtomicU64, + /// The encoder's latest target, in bits per second, or zero before one is reported. + bitrate: AtomicU64, } impl Sent { @@ -27,11 +30,23 @@ impl Sent { self.keyframes.fetch_add(keyframes, Relaxed); } + /// Count `frames` frames the source produced but never sent. + pub(crate) fn skip(&self, frames: u64) { + self.skipped.fetch_add(frames, Relaxed); + } + + /// Record the encoder's latest target bitrate. + pub(crate) fn target(&self, bitrate: moq_net::bandwidth::Rate) { + self.bitrate.store(bitrate.as_bps(), Relaxed); + } + fn report(&self) -> hang::stats::Rendition { let mut report = hang::stats::Rendition::default(); report.frames = self.frames.load(Relaxed); report.bytes = self.bytes.load(Relaxed); report.keyframes = self.keyframes.load(Relaxed); + report.skipped = self.skipped.load(Relaxed); + report.bitrate = Some(self.bitrate.load(Relaxed)).filter(|&bitrate| bitrate != 0); report } } @@ -177,6 +192,22 @@ mod tests { } } + /// An encoder's skips accumulate, and its target is the latest one set: none until then. + #[test] + fn an_encoder_reports_skips_and_its_latest_target() { + let registry = Registry::::default(); + let track = registry.track("video"); + assert_eq!(registry.sent()["video"].bitrate, None); + + track.skip(2); + track.skip(1); + track.target(moq_net::bandwidth::Rate::from_bps(2_000_000)); + track.target(moq_net::bandwidth::Rate::from_bps(1_500_000)); + let report = ®istry.sent()["video"]; + assert_eq!(report.skipped, 3); + assert_eq!(report.bitrate, Some(1_500_000)); + } + #[test] fn a_returning_track_continues_its_counts() { let registry = Registry::::default(); diff --git a/rs/moq-video/src/capture/channel.rs b/rs/moq-video/src/capture/channel.rs index c0463b423ae..9c40004f258 100644 --- a/rs/moq-video/src/capture/channel.rs +++ b/rs/moq-video/src/capture/channel.rs @@ -30,6 +30,17 @@ struct State { native: Option, closed: bool, error: Option, + /// Frames replaced before the consumer reached them. + skipped: u64, +} + +impl State { + /// Hold `frame` as the latest, counting the one it replaces as skipped. + fn replace(&mut self, frame: Frame) { + if self.frame.replace(frame).is_some() { + self.skipped += 1; + } + } } /// How a device's own timeline maps onto this stream's. @@ -54,6 +65,7 @@ impl FrameChannel { native: None, closed: false, error: None, + skipped: 0, }), notify: Notify::new(), epoch: Instant::now(), @@ -122,7 +134,7 @@ impl FrameChannel { mapped: timestamp, ..anchor }); - state.frame = Some(Frame::new(surface, timestamp)); + state.replace(Frame::new(surface, timestamp)); drop(state); self.notify.notify_one(); } @@ -132,11 +144,16 @@ impl FrameChannel { if state.closed { return; } - state.frame = Some(frame); + state.replace(frame); drop(state); self.notify.notify_one(); } + /// How many frames a newer one replaced before [`recv`](Self::recv) reached them. + pub(super) fn skipped(&self) -> u64 { + self.state.lock().unwrap().skipped + } + /// Mark the source ended, so a parked [`recv`](Self::recv) returns `None`. pub(super) fn close(&self) { let mut state = self.state.lock().unwrap(); @@ -235,6 +252,7 @@ mod tests { chan.push(frame(id)); } assert_eq!(chan.recv().await.unwrap().unwrap().surface.width(), 6); + assert_eq!(chan.skipped(), 5); } #[tokio::test] diff --git a/rs/moq-video/src/capture/mod.rs b/rs/moq-video/src/capture/mod.rs index 5f017b2ad1b..2b4bcd40933 100644 --- a/rs/moq-video/src/capture/mod.rs +++ b/rs/moq-video/src/capture/mod.rs @@ -356,6 +356,12 @@ impl Stream { Ok(Some(Frame::new(surface, frame.timestamp))) } + /// How many captured frames no [`read`](Self::read) returned, because a newer one replaced + /// them while the reader was busy. + pub(crate) fn skipped(&self) -> u64 { + self.chan.skipped() + } + /// Width of the captured frames in pixels. pub fn width(&self) -> u32 { self.width diff --git a/rs/moq-video/src/encode/producer.rs b/rs/moq-video/src/encode/producer.rs index 3acba2f2f75..077052694f4 100644 --- a/rs/moq-video/src/encode/producer.rs +++ b/rs/moq-video/src/encode/producer.rs @@ -237,6 +237,24 @@ impl Producer { Ok(()) } + /// Report capture frames the encoder never saw, in the catalog's stats. + #[cfg(feature = "capture")] + fn report_skipped(&self, frames: u64) { + match &self.codecs { + Codecs::H264 { import, .. } => import.report_skipped(frames), + Codecs::H265 { import, .. } => import.report_skipped(frames), + } + } + + /// Report the encoder's target bitrate, in the catalog's stats. + #[cfg(feature = "capture")] + fn report_bitrate(&self, bitrate: moq_net::bandwidth::Rate) { + match &self.codecs { + Codecs::H264 { import, .. } => import.report_bitrate(bitrate), + Codecs::H265 { import, .. } => import.report_bitrate(bitrate), + } + } + /// Finalize the track. /// /// Borrows rather than consumes, so a later [`abort`](Self::abort) can still @@ -595,37 +613,46 @@ async fn next_target(rate: &mut RateControl) -> Result) { - let Some(follower) = rate.as_mut() else { return }; +async fn apply_target( + encoder: &mut Sink, + rate: &mut RateControl, + target: Result, +) -> Option { + let follower = rate.as_mut()?; let Ok(target) = target else { tracing::debug!("bandwidth estimate ended; holding the current encoder bitrate"); *rate = None; - return; + return None; }; let rate::Target::Enabled(bitrate) = target else { unreachable!("a capture follows its grant with no lower rendition, so it never disables") }; match encoder.set_bitrate(bitrate).await { - Ok(()) => tracing::debug!(bitrate = bitrate.as_bps(), "adjusted encoder bitrate"), + Ok(()) => { + tracing::debug!(bitrate = bitrate.as_bps(), "adjusted encoder bitrate"); + Some(bitrate) + } // The encoder can't retune, so keep encoding at the rate it opened with // and stop asking. Dropping the source also stops the estimate arm, which // would otherwise wake this loop for nothing on every change. Err(Error::BitrateUnsupported(name)) => { tracing::warn!(encoder = name, "encoder cannot follow the bandwidth estimate"); *rate = None; + None } // A transient failure: the follower takes the target back and retries it, // so a quiet grant still recovers. Err(err) => { tracing::warn!(error = %err, bitrate = bitrate.as_bps(), "failed to adjust encoder bitrate"); follower.failed(); + None } } } @@ -777,8 +804,11 @@ async fn capture_loop( // reopened camera starts optimistic again rather than inheriting the // backed-off rate from whatever the link was doing last time. let mut rate = Some(rate::Follower::new(reservation.consume(), rate::Policy::new(ceiling))); + producer.report_bitrate(ceiling); // Per encoder, since a fresh one opens with a keyframe that serves every earlier ask. let mut forced = Cuts::new(served); + // The camera's skips already reported, since each open counts from zero. + let mut reported = 0; loop { // Race the next frame against the last viewer leaving so we release the @@ -796,7 +826,9 @@ async fn capture_loop( // Retune between frames rather than mid-encode, and only when // the policy says the target actually moved. target = next_target(&mut rate) => { - apply_target(&mut encoder, &mut rate, target).await; + if let Some(bitrate) = apply_target(&mut encoder, &mut rate, target).await { + producer.report_bitrate(bitrate); + } continue; } // A read error is terminal for this selection (the source is gone @@ -805,6 +837,10 @@ async fn capture_loop( }; let Some(mut frame) = frame else { break }; + // Frames the camera replaced while this loop was encoding or retuning. + let skipped = camera.skipped(); + producer.report_skipped(skipped - reported); + reported = skipped; frame.timestamp = map_capture_timestamp(capture_epoch, frame.timestamp)?; let count = counted.read().count; // A backend that cannot cut leaves every group to its cadence. @@ -1355,6 +1391,30 @@ mod tests { .await } + /// The catalog's stats carry the encoder's target and every frame the camera replaced + /// before the loop read it. + #[tokio::test] + async fn stats_report_the_target_and_skipped_frames() { + tokio::task::LocalSet::new() + .run_until(async { + let fixture = Fixture::start(Duration::from_secs(1), moq_clock::wall()).await; + let mut track = fixture.subscribe().await; + let camera = fixture.camera(); + + // Back to back, so the loop only ever sees the second. + camera.push_at(surface(), Instant::now()); + camera.push_at(surface(), Instant::now()); + read(&mut track).await; + + let stats = fixture.catalog.stats().remove("video").expect("the video rendition"); + assert_eq!(stats.skipped, 1); + let ceiling = Config::new(320, 240, crate::FrameRate::new(30, 1).unwrap()).resolved_bitrate(); + assert_eq!(stats.bitrate, Some(ceiling.as_bps())); + fixture.finish().await; + }) + .await + } + /// A device clock that restarts at zero, mid-stream or across a reopen, continues the /// broadcast forward with the device's spacing instead of rewinding it. #[tokio::test] From a2b025c2bdf42d0ba4a01977dfcfbab1b3c911ad Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 05:30:56 -0700 Subject: [PATCH 3/7] fix(video): count capture skips only from an encoder's first read Frames the camera replaces while an encoder is still opening are startup, not frames it fell behind on; counting them would grade every reopen as degraded. Co-Authored-By: Claude Opus 5.5 --- rs/moq-video/src/encode/producer.rs | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/rs/moq-video/src/encode/producer.rs b/rs/moq-video/src/encode/producer.rs index 077052694f4..dde68bd9bae 100644 --- a/rs/moq-video/src/encode/producer.rs +++ b/rs/moq-video/src/encode/producer.rs @@ -807,8 +807,9 @@ async fn capture_loop( producer.report_bitrate(ceiling); // Per encoder, since a fresh one opens with a keyframe that serves every earlier ask. let mut forced = Cuts::new(served); - // The camera's skips already reported, since each open counts from zero. - let mut reported = 0; + // The camera's skips already accounted for, from its first read on: frames it replaced + // while this encoder was still opening are startup, not frames the encoder fell behind on. + let mut reported = None; loop { // Race the next frame against the last viewer leaving so we release the @@ -839,8 +840,10 @@ async fn capture_loop( let Some(mut frame) = frame else { break }; // Frames the camera replaced while this loop was encoding or retuning. let skipped = camera.skipped(); - producer.report_skipped(skipped - reported); - reported = skipped; + if let Some(reported) = reported { + producer.report_skipped(skipped - reported); + } + reported = Some(skipped); frame.timestamp = map_capture_timestamp(capture_epoch, frame.timestamp)?; let count = counted.read().count; // A backend that cannot cut leaves every group to its cadence. @@ -1392,7 +1395,7 @@ mod tests { } /// The catalog's stats carry the encoder's target and every frame the camera replaced - /// before the loop read it. + /// before the loop read it, once the loop has read its first. #[tokio::test] async fn stats_report_the_target_and_skipped_frames() { tokio::task::LocalSet::new() @@ -1401,7 +1404,11 @@ mod tests { let mut track = fixture.subscribe().await; let camera = fixture.camera(); - // Back to back, so the loop only ever sees the second. + // Replaced before the first read, while the encoder was still opening: not a skip. + camera.push_at(surface(), Instant::now()); + camera.push_at(surface(), Instant::now()); + read(&mut track).await; + // Back to back once encoding, so the loop only ever sees the second. camera.push_at(surface(), Instant::now()); camera.push_at(surface(), Instant::now()); read(&mut track).await; From 95908bcd6e97e16158690fee2081f674d3f96135 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 10:18:22 -0700 Subject: [PATCH 4/7] quest(stats): record the gateway reporting shape Co-Authored-By: Claude Opus 5.5 --- quest/m2/stats/rust.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/quest/m2/stats/rust.md b/quest/m2/stats/rust.md index 809ba2102e1..b24203cd5dd 100644 --- a/quest/m2/stats/rust.md +++ b/quest/m2/stats/rust.md @@ -37,6 +37,13 @@ catalog. The report writer lives in `rs/moq-cli/src/report.rs` and reads the transport from a `moq_tokio` connection monitor, which the gateway crates do not see. +- Decided 2026-10-31 (maintainer): move the report writer into `moq-mux`, with + the transport passed in as data rather than read from a `moq_tokio` monitor, + and give each gateway builder `with_report` and `with_echo`. A library user + embedding a gateway gets reports too; the cost, accepted, is that `Stats` + becomes a library API. Rejected: handing each session's catalog back to the + CLI through a returned handle, which keeps the writer in `moq-cli` but leaves + library users without reports. - `moq play --echo` reconciles each watched catalog's `echo` section as `export --echo` does, and refuses `--catalog-format msf` the same way: the MSF catalog does not carry an `echo` solicitation. `play` attaches only the From ef7bb8ecc256f3be72ab5bd5d472cc5db219c88d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 31 Oct 2026 10:43:15 -0700 Subject: [PATCH 5/7] fix(video): report the encoder's target on a track capture moves to Co-Authored-By: Claude Opus 5.5 --- rs/moq-video/src/encode/producer.rs | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/rs/moq-video/src/encode/producer.rs b/rs/moq-video/src/encode/producer.rs index 127eb3ff0b4..6523b3acd0c 100644 --- a/rs/moq-video/src/encode/producer.rs +++ b/rs/moq-video/src/encode/producer.rs @@ -869,7 +869,9 @@ async fn capture_loop( // reopened camera starts optimistic again rather than inheriting the // backed-off rate from whatever the link was doing last time. let mut rate = Some(rate::Follower::new(claim.consume(), rate::Policy::new(ceiling))); - producer.report_bitrate(ceiling); + // The encoder's current target, reported again on any track the capture moves to. + let mut bitrate = ceiling; + producer.report_bitrate(bitrate); // Per encoder, since a fresh one opens with a keyframe that serves every earlier ask. let mut forced = Cuts::new(served); // Rate control follows the grant only while someone watches. The first poll of `watch` @@ -898,8 +900,9 @@ async fn capture_loop( break; // no viewers: release the camera, then wait for one } if watched - && !wanted && let Some(bitrate) = rest(&mut encoder, &mut rate, claim, ceiling).await + && !wanted && let Some(rested) = rest(&mut encoder, &mut rate, claim, ceiling).await { + bitrate = rested; producer.report_bitrate(bitrate); } watched = wanted; @@ -908,7 +911,8 @@ async fn capture_loop( // Retune between frames rather than mid-encode, and only when // the policy says the target actually moved. target = next_target(&mut rate), if watched => { - if let Some(bitrate) = apply_target(&mut encoder, &mut rate, target).await { + if let Some(applied) = apply_target(&mut encoder, &mut rate, target).await { + bitrate = applied; producer.report_bitrate(bitrate); } continue; @@ -950,9 +954,11 @@ async fn capture_loop( } watch = demand.clone(); watched = false; - if let Some(bitrate) = rest(&mut encoder, &mut rate, claim, ceiling).await { - producer.report_bitrate(bitrate); + // The new track's stats start empty, so they learn the target whether or not it rests. + if let Some(rested) = rest(&mut encoder, &mut rate, claim, ceiling).await { + bitrate = rested; } + producer.report_bitrate(bitrate); } } From 28310aad9cb106dc272ee0a665bd5f510071c4aa Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 21:14:35 -0800 Subject: [PATCH 6/7] fix(video): report capture skips left when the loop stops A capture loop reported the camera's replacements only after a read, so frames it replaced after the last one went unreported when the last viewer left, the stream ended, or an encode was abandoned. Flush the remaining delta before dropping the camera. Co-Authored-By: Claude Opus 5.5 --- rs/moq-video/src/encode/producer.rs | 32 +++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/rs/moq-video/src/encode/producer.rs b/rs/moq-video/src/encode/producer.rs index db973adf0b0..5d7a0ea958d 100644 --- a/rs/moq-video/src/encode/producer.rs +++ b/rs/moq-video/src/encode/producer.rs @@ -834,6 +834,11 @@ async fn capture_loop( } } + // Frames the camera replaced after the last read, before demand or the stream ended + // the loop: no read follows to report them. + if let Some(reported) = reported { + producer.report_skipped(camera.skipped() - reported); + } // Drop the camera (LED off) and encoder before opening another. drop(camera); drop(encoder); @@ -1393,6 +1398,33 @@ mod tests { .await } + /// Frames the camera replaced after the last read still count when the last viewer + /// leaves before another read. + #[tokio::test] + async fn stats_report_skips_left_when_the_viewer_leaves() { + tokio::task::LocalSet::new() + .run_until(async { + let fixture = Fixture::start(Duration::from_secs(1), kio::clock::wall()).await; + let mut track = fixture.subscribe().await; + let camera = fixture.camera(); + camera.push_at(surface(), Instant::now()); + read(&mut track).await; + + // The viewer leaves, then the camera replaces a frame before the loop runs again. + drop(track); + camera.push_at(surface(), Instant::now()); + camera.push_at(surface(), Instant::now()); + while !camera.released() { + tokio::task::yield_now().await; + } + + let stats = fixture.catalog.stats().remove("video").expect("the video rendition"); + assert_eq!(stats.skipped, 1); + fixture.finish().await; + }) + .await + } + /// A device clock that restarts at zero, mid-stream or across a reopen, continues the /// broadcast forward with the device's spacing instead of rewinding it. #[tokio::test] From 2a8a28bc8d44656d6f461bde9e73e47d05594dfa Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 21:53:17 -0800 Subject: [PATCH 7/7] fix(relay): gate the listener's Io re-export on websocket Only the WebSocket upgrade names `listener::Io`, so a relay built without `websocket` (moq-cli's dev-dependency) warned on the unused import and failed under `build.warnings = "deny"`. Co-Authored-By: Claude Opus 5.5 --- rs/moq-relay/src/listener.rs | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rs/moq-relay/src/listener.rs b/rs/moq-relay/src/listener.rs index 09d5a12fc2b..65b7a8dc294 100644 --- a/rs/moq-relay/src/listener.rs +++ b/rs/moq-relay/src/listener.rs @@ -17,7 +17,10 @@ mod demux; mod idle; pub(crate) use demux::{Classified, Stream, Tls, classify}; -pub(crate) use idle::{Admission, Io}; +pub(crate) use idle::Admission; +// Only the WebSocket upgrade names the wrapped stream. +#[cfg(feature = "websocket")] +pub(crate) use idle::Io; use std::{ io,