Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/lib/rs/moq-mux.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ set at the init segment, so a new rendition or a changed configuration ends the
export. The [CLI page](/bin/cli#import) covers these and the MPEG-TS specifics:
one program per broadcast, damaged packets, and feed checks.

Exporters take a catalog stream alongside the source. Choose its encoding and
rendition selection before exporting; the same stream shape drives every container.
MPEG-TS metadata is retained when the stream uses `ts::Ext`.

A container read yields frames plus two markers in delivery order: a gap where
media after what was read will never arrive (a skip, a hole), carrying where it
starts and how long it lasts as an upper bound, and an end where the publisher
Expand Down
5 changes: 5 additions & 0 deletions doc/setup/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ These land with the next breaking release, not the 2026-09-23 train.
the same two flags. In Rust, moq-mux's `ts::Export::resume` is
`ts::Export::follow`, `Source::returned` is gone, and `ts::Follower` follows a
path's announcements the way both exports do.
`ts::Export::new(source, catalog)` now takes a catalog stream, like the other
container exporters. Subscribe with `source.catalog::<ts::Ext>(format).await?`
to retain MPEG-TS metadata, or use `()` for media only; select renditions on the
stream before passing it in. `follow(broadcast, catalog)` also takes the replacement
stream. Both methods are synchronous; `with_ts` and `with_catalog_format` are removed.
- **moq-mux's `Source` is built from the resolved catalog broadcast.**
`Source::new(origin, path)` is `Source::new(origin, broadcast)`: resolve the
broadcast first (`origin.request_broadcast(path, None)`), and its path is
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,6 @@ above them.
- [Vulkan Video encode on AMD](/quest/m1/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070
- [VA-API encodes an external Vulkan image](/quest/m1/vaapi-vulkan-import.md) - an explicit-sync DMA-BUF reaches Intel's encoder, proven on Arrow Lake
- [Malformed moq-transport input](/quest/m1/ietf-malformed-close.md) - malformed draft-18 and draft-21 control input closes the session with the draft's code, or PROTOCOL_VIOLATION where a code is a real burden and the fallback is recorded in `doc/concept/standard.md`, in moq-net and js/net
- [ts::Export catalog stream](/quest/m1/ts-export-catalog.md) - TS export takes (source, catalog) like the other exporters
- [Binding track resolver](/quest/m1/binding-track-helper.md) - binding apps subscribe to a rendition without inlining `track` or the key
- [Exporters follow a rendition to a new track](/quest/m1/export-rendition-rebind.md) - the fMP4, TS, MKV and FLV exporters rebuild a rendition whose resolved `track` or `broadcast` changes under the same ID

Expand Down
7 changes: 0 additions & 7 deletions quest/m1/ts-export-catalog.md

This file was deleted.

18 changes: 13 additions & 5 deletions rs/moq-cli/src/publish.rs
Original file line number Diff line number Diff line change
Expand Up @@ -675,10 +675,18 @@ pub(crate) mod tests {
const VERBATIM_PES_PID: u16 = 0x104;
const VERBATIM_PES_STREAM_ID: u8 = 0xC0;

async fn open_export(
source: moq_mux::Source,
format: CatalogFormat,
) -> moq_mux::Result<Export<moq_mux::catalog::Consumer<tscat::Ext>>> {
let catalog = source.catalog::<tscat::Ext>(format).await?;
Ok(Export::new(source, catalog))
}

/// Drain an exporter, concatenating every frame's payload until output stops. The
/// producers stay alive (retained tracks), so the stream never hard-ends; pull until a
/// `next()` blocks, surfaced here as a timeout once the buffered frames are gone.
async fn drain(mut exporter: Export<tscat::Ext>) -> Vec<u8> {
async fn drain(mut exporter: Export<moq_mux::catalog::Consumer<tscat::Ext>>) -> Vec<u8> {
let mut out = Vec::new();
// Past the export's release delay and the send-ahead window behind it, so the first
// frame goes out before it ends.
Expand Down Expand Up @@ -780,7 +788,7 @@ pub(crate) mod tests {
// `catalog`, the producers, and `import` stay alive: the exporter subscribes to
// the retained tracks.
drain(
Export::with_ts(resolved(&origin, "cli").await, CatalogFormat::Hang)
open_export(resolved(&origin, "cli").await, CatalogFormat::Hang)
.await
.unwrap()
.with_delay(RECORDING_MAX_AGE),
Expand Down Expand Up @@ -812,7 +820,7 @@ pub(crate) mod tests {
}

/// Full CLI round-trip: a TS feed with undecoded streams goes through `Publish`
/// (which selects the `mpegts` catalog) and the subscribe-side `Export::with_ts`,
/// (which selects the `mpegts` catalog) and the subscribe-side catalog stream,
/// and the SCTE-35 section and the verbatim PES survive with their PIDs, framing,
/// PES stream_id, and byte-exact payloads.
async fn ts_verbatim_round_trip(format: CatalogFormat) {
Expand All @@ -835,10 +843,10 @@ pub(crate) mod tests {
decoder.decode_chunk(&input).unwrap();
decoder.finish().unwrap();

// Subscribe side: the same `with_ts` call `run_ts` makes, re-emitting the
// Subscribe side: the same catalog stream `run_ts` selects, re-emitting the
// ancillary streams verbatim.
let output = drain(
Export::with_ts(resolved(&origin, "cli").await, format)
open_export(resolved(&origin, "cli").await, format)
.await
.unwrap()
.with_delay(RECORDING_MAX_AGE),
Expand Down
7 changes: 3 additions & 4 deletions rs/moq-cli/src/subscribe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -378,12 +378,11 @@ impl Subscribe {

// TS emits PAT/PMT then a continuous PES stream (re-emitting PAT/PMT at
// keyframes for tune-in). Avc3/Hev1 sources pass through as Annex-B; AAC
// is re-framed as ADTS. `fragment_duration` does not apply to TS. `with_ts`
// is re-framed as ADTS. `fragment_duration` does not apply to TS. The catalog stream
// selects the `mpegts` catalog extension so undecoded elementary streams
// (SCTE-35, teletext, DVB AC-3, ...) are re-emitted verbatim on their PIDs.
let mut ts = moq_mux::container::ts::Export::with_ts(self.source, self.catalog)
.await?
.with_delay(self.args.max_delay);
let catalog = self.source.catalog::<moq_mux::container::ts::Ext>(self.catalog).await?;
let mut ts = moq_mux::container::ts::Export::new(self.source, catalog).with_delay(self.args.max_delay);
if let Some(mux_rate) = self.args.mux_rate {
ts = ts.with_mux_rate(mux_rate);
}
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-mux/benches/export.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ async fn export(format: &str, source: moq_mux::Source, stream: moq_mux::catalog:
chunks += 1;
}
} else if format == "ts" {
let mut export = moq_mux::container::ts::Export::new(source).await.unwrap();
let mut export = moq_mux::container::ts::Export::new(source, stream);
while let Some(chunk) = export.next().await.unwrap() {
std::hint::black_box(chunk);
chunks += 1;
Expand Down
54 changes: 51 additions & 3 deletions rs/moq-mux/src/catalog/consumer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ use super::{CatalogFormat, Stream};
/// [`Select`](super::Select) to narrow the rendition set before handing the
/// stream to an exporter.
///
/// Reaching EOF releases the subscription and decoder while retaining its format.
///
/// The variants are an implementation detail: drive it through the [`Stream`]
/// trait rather than matching on them. New catalog encodings may be added.
#[non_exhaustive]
Expand All @@ -24,9 +26,19 @@ pub enum Consumer<E: CatalogExt = ()> {
Hang(super::hang::Consumer<E>),
#[doc(hidden)]
Msf(super::msf::Consumer<E>),
#[doc(hidden)]
Closed(CatalogFormat),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the EOF sentinel private

When downstream crates use Consumer, this variant remains constructible and matchable despite #[doc(hidden)], so internal EOF bookkeeping becomes a supported public API and lets callers fabricate a consumer that was never sourced from a broadcast. Removing or reshaping that state later would require another breaking change; retain the format and release the decoder within private state instead of adding a public variant.

AGENTS.md reference: AGENTS.md:L57-L61

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disagree, keeping it. catalog::Consumer is already a #[non_exhaustive] enum whose Hang/Msf variants are #[doc(hidden)] and constructed directly by the benches, so Closed adds no new kind of exposure: hidden variants are not supported API, and the type is documented as driven through Stream only. Moving EOF into private state would need either making msf::Consumer::track non-public or turning the unified consumer into an opaque struct (plus a constructor for the benches), both wider API changes than this PR's approved scope. An opaque catalog::Consumer is a reasonable follow-up on its own.

(Written by Claude Opus 5.5)

}

impl<E: CatalogExt> Consumer<E> {
pub(crate) fn format(&self) -> CatalogFormat {
match self {
Self::Hang(consumer) => consumer.format(),
Self::Msf(_) => CatalogFormat::Msf,
Self::Closed(format) => *format,
}
}

/// Subscribe to the catalog track advertised by `format`.
pub async fn new(broadcast: &moq_net::broadcast::Consumer, format: CatalogFormat) -> Result<Self, crate::Error> {
Ok(match format {
Expand Down Expand Up @@ -56,9 +68,45 @@ impl<E: CatalogExt> Stream for Consumer<E> {
type Ext = E;

fn poll_next(&mut self, waiter: &kio::Waiter) -> Poll<crate::Result<Option<Catalog<E>>>> {
match self {
Self::Hang(c) => c.poll_next(waiter),
Self::Msf(c) => c.poll_next(waiter),
let next = match self {
Self::Hang(consumer) => consumer.poll_next(waiter),
Self::Msf(consumer) => consumer.poll_next(waiter),
Self::Closed(_) => return Poll::Ready(Ok(None)),
};
if matches!(&next, Poll::Ready(Ok(None))) {
// A follower still needs the format, but no ended subscription or decoder.
*self = Self::Closed(self.format());
}
next
}
}

#[cfg(test)]
mod tests {
use super::*;

#[tokio::test(start_paused = true)]
async fn eof_releases_each_catalog_subscription() {
for format in [CatalogFormat::Hang, CatalogFormat::HangZ, CatalogFormat::Msf] {
let name = match format {
CatalogFormat::Hang => hang::Catalog::DEFAULT_NAME,
CatalogFormat::HangZ => hang::Catalog::COMPRESSED_NAME,
CatalogFormat::Msf => moq_msf::DEFAULT_NAME,
};
let broadcast = moq_net::broadcast::Info::new().produce();
let track = broadcast
.create_track(name, hang::Catalog::default_track_info())
.unwrap();
let mut consumer = Consumer::<()>::new(&broadcast.consume(), format).await.unwrap();
assert!(track.subscription().is_some(), "{format:?}: subscribed");
track.finish().unwrap();
assert!(consumer.next().await.unwrap().is_none());
assert!(
track.subscription().is_none(),
"{format:?}: EOF must release its subscriber"
);
assert_eq!(consumer.format(), format);
assert!(consumer.next().await.unwrap().is_none());
}
}
}
8 changes: 8 additions & 0 deletions rs/moq-mux/src/catalog/hang/consumer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ use moq_net::PathOwned;

use super::{Catalog, CatalogExt};
use crate::Result;
use crate::catalog::CatalogFormat;

/// A catalog consumer, used to receive catalog updates and discover tracks.
///
Expand All @@ -18,6 +19,7 @@ use crate::Result;
/// A catalog that misuses that reference is rejected outright (see [`Self::poll_next`]).
pub struct Consumer<E: CatalogExt = ()> {
inner: moq_json::snapshot::Consumer<Catalog<E>>,
format: CatalogFormat,
/// Path of the broadcast serving this catalog, which every rendition's `broadcast`
/// reference resolves against.
base: PathOwned,
Expand All @@ -27,6 +29,7 @@ impl<E: CatalogExt> Consumer<E> {
/// Create a new catalog consumer from a MoQ track subscriber (uncompressed `catalog.json`).
pub fn new(track: moq_net::track::Subscriber) -> Self {
Self {
format: CatalogFormat::Hang,
base: track.broadcast().path.clone(),
inner: moq_json::snapshot::Consumer::new(track, moq_json::snapshot::consumer::Config::default()),
}
Expand All @@ -39,11 +42,16 @@ impl<E: CatalogExt> Consumer<E> {
let mut config = moq_json::snapshot::consumer::Config::default();
config.compression = moq_json::Compression::Deflate;
Self {
format: CatalogFormat::HangZ,
base: track.broadcast().path.clone(),
inner: moq_json::snapshot::Consumer::new(track, config),
}
}

pub(crate) fn format(&self) -> CatalogFormat {
self.format
}

/// Poll for the next catalog update.
///
/// Fails with [`Error::EscapingBroadcast`](crate::Error::EscapingBroadcast) if any rendition's
Expand Down
Loading
Loading