diff --git a/doc/lib/rs/moq-mux.md b/doc/lib/rs/moq-mux.md index b4595f52ea7..fa838cbc3c6 100644 --- a/doc/lib/rs/moq-mux.md +++ b/doc/lib/rs/moq-mux.md @@ -16,7 +16,7 @@ back. This is what `moq import` / `export` and the gateways are built on. | fMP4 / CMAF | yes | yes | Audio/video unwrapped to `legacy` by default, or passed through as `cmaf` with `with_passthrough`; `wvtt` subtitles import as timed WebVTT text and `stpp` TTML (IMSC timing profile) as `ttml` documents on the broadcast clock, preserving source language and label when present; other subtitle codecs are skipped. | | MPEG-TS | yes | yes | H.264/H.265; AAC, MP2, AC-3, E-AC-3, Opus up to 7.1; SCTE-35 sections and every other undecoded PES stream (KLV, subtitles, DVB AC-3) as tracks; service tables round-trip. | | FLV / RTMP | yes | yes | Legacy H.264 + AAC + MP3, plus enhanced-RTMP HEVC, AV1, VP9, Opus, AC-3, E-AC-3, and multitrack. | -| Matroska / WebM | yes | yes | Timed SRT and WebVTT subtitles import as WebVTT text renditions, preserving source language and label when present; other subtitle codecs are skipped. | +| Matroska / WebM | yes | yes | Import streams each block into its frame in one pass, retaining no declared size. It refuses subtitle tracks, CMAF output, and video `BlockGroup`s with fields after their `Block`: remux video as `SimpleBlock`s and extract subtitles separately. | | Annex-B (H.264, H.265) | yes | yes | Parameter sets extracted to the catalog or re-injected per keyframe. | Importers fill the catalog from the bitstream, split groups at keyframes, and diff --git a/quest/m1/security-audit/README.md b/quest/m1/security-audit/README.md index 818f6a2088c..d6324a9ecd5 100644 --- a/quest/m1/security-audit/README.md +++ b/quest/m1/security-audit/README.md @@ -55,7 +55,6 @@ Dropped, with reasons: ## Required - [fMP4 import streams mdat](/quest/m1/security-audit/fmp4-stream-mdat.md) - a box header declaring a huge size cannot pin importer memory across segments -- [MKV import streams blocks](/quest/m1/security-audit/mkv-stream-blocks.md) - a declared block, metadata master, or ignored element cannot pin importer memory - [Relay HTTP surface](/quest/m1/security-audit/relay-http.md) - `/fetch` cannot be sniffed as active content, and no web page can reach the ops listener - [Browser input checks](/quest/m1/security-audit/js-input.md) - room metadata is schema-validated and the http:// fingerprint fetch is aborted and length-checked - [CI tool pins](/quest/m1/security-audit/ci-pins.md) - binding generators are pinned by commit in one place, and the remaining unpinned installs and broad token grants are fixed diff --git a/quest/m1/security-audit/fmp4-stream-mdat.md b/quest/m1/security-audit/fmp4-stream-mdat.md index 8314f5eaf33..01513353ec9 100644 --- a/quest/m1/security-audit/fmp4-stream-mdat.md +++ b/quest/m1/security-audit/fmp4-stream-mdat.md @@ -25,12 +25,11 @@ frame cap so a sample can be large. Since [8d2fa6653a](https://github.com/moq-dev/moq/commit/8d2fa6653a) a fragment can hold several GOPs, so a fragment-sized cap would also refuse legal input. -Open, for the maintainer before building: `moov` and `moof` are parsed whole -today, so a header declaring a huge one still pins input. Bring options with -a recommendation, such as parsing them incrementally (a `moof` is a run of -small child boxes) or bounding them by the work they describe; a plain byte -cap is ruled out above. Settle this together with the same question in -[MKV import streams blocks](/quest/m1/security-audit/mkv-stream-blocks.md). +`moov` and `moof` are parsed whole today, so a header declaring a huge one +still pins input. Decided 2026-10-28 (maintainer, for both importers): parse +metadata children incrementally, keeping only the fields consumed, with no +blanket byte cap. MKV import does so since +[moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234). Keep `sample::Resolver`'s range checks and `Claimed`'s no-reuse rule. Tests feed a huge declared `moof`, `moov`, `mdat`, and skipped box in small chunks @@ -40,5 +39,4 @@ Public API: none expected. Wire: none. ## Related -- [MKV import streams blocks](/quest/m1/security-audit/mkv-stream-blocks.md) - the same principle for MKV - [Media fuzz](/quest/m2/media-fuzz.md) - would catch this class of buffer lifetime bug diff --git a/quest/m1/security-audit/mkv-stream-blocks.md b/quest/m1/security-audit/mkv-stream-blocks.md deleted file mode 100644 index 881626c2b54..00000000000 --- a/quest/m1/security-audit/mkv-stream-blocks.md +++ /dev/null @@ -1,37 +0,0 @@ -# [M] MKV import streams blocks - -## Goal - -The MKV importer streams a SimpleBlock or BlockGroup payload into its frame -as bytes arrive instead of buffering the whole tag, so a tag declaring -gigabytes holds memory only as fast as the output accepts it. - -## Plan - -From the 2026-10-27 audit (MUX-3). `rs/moq-mux/src/container/mkv/import.rs` -never sets ebml-iterable's tag limit, whose default is about 4 GB, and -SimpleBlock and BlockGroup are buffered whole. - -The audit's fix (lower `max_allowable_tag_size`) is wrong: the library applies -it to every sized tag, including the Segment and Cluster masters, so it would -refuse ordinary files larger than the limit. - -Decided 2026-10-27 (maintainer): stream the block payload, the same principle -as [fMP4 import streams mdat](/quest/m1/security-audit/fmp4-stream-mdat.md), -rather than a size pre-check. A pre-check would compare against the frame cap -that [unbounded groups](/quest/m1/unbounded-groups/README.md) removes, after -which a large block is a legal large frame. This needs a small block reader -beside ebml-iterable for the block header and payload; prefer an upstream -ebml-iterable option if one fits. Ignored elements are discarded as their -bytes arrive instead of buffered. - -Open, for the maintainer before building: `Import::new` decodes `Info`, -`Tracks`, and `TrackEntry` as whole masters, so a header declaring a huge one -still pins input. Bring options with a recommendation, such as handling -their children incrementally as the importer already does for Clusters; a -blanket tag-size cap is ruled out above. Tests feed a huge declared block, -metadata master, and ignored element in chunks and assert held bytes stay -bounded. Settle this together with the same question in -[fMP4 import streams mdat](/quest/m1/security-audit/fmp4-stream-mdat.md). - -Public API: none expected. Wire: none. diff --git a/rs/moq-mux/benches/containers.rs b/rs/moq-mux/benches/containers.rs index 08fec84acfd..21616ef220c 100644 --- a/rs/moq-mux/benches/containers.rs +++ b/rs/moq-mux/benches/containers.rs @@ -679,10 +679,85 @@ fn mkv_chunks(c: &mut Criterion) { group.finish(); } +/// A one-track VP9 WebM of `total` bytes of keyframe SimpleBlocks, each carrying `payload` bytes. +fn mkv_payload_file(payload: usize, total: usize) -> Vec { + use webm_iterable::WebmWriter; + use webm_iterable::matroska_spec::{Master, MatroskaSpec, SimpleBlock}; + + let frame = vec![0x5a; payload]; + let blocks: Vec = (0..total / payload) + .map(|index| SimpleBlock::new_uncheked(&frame, 1, (index % 1024) as i16, false, None, false, true).into()) + .collect(); + let mut data = Vec::new(); + let mut writer = WebmWriter::new(&mut data); + writer + .write(&MatroskaSpec::Ebml(Master::Full(vec![MatroskaSpec::DocType( + "webm".into(), + )]))) + .unwrap(); + writer.write(&MatroskaSpec::Segment(Master::Start)).unwrap(); + writer + .write(&MatroskaSpec::Tracks(Master::Full(vec![MatroskaSpec::TrackEntry( + Master::Full(vec![ + MatroskaSpec::TrackNumber(1), + MatroskaSpec::TrackType(1), + MatroskaSpec::CodecID("V_VP9".into()), + ]), + )]))) + .unwrap(); + // Relative block timestamps are 16-bit, so each cluster holds a bounded run of them. + for (cluster, blocks) in blocks.chunks(1024).enumerate() { + let mut children = vec![MatroskaSpec::Timestamp(cluster as u64 * 1024)]; + children.extend_from_slice(blocks); + writer.write(&MatroskaSpec::Cluster(Master::Full(children))).unwrap(); + } + writer.write(&MatroskaSpec::Segment(Master::End)).unwrap(); + writer.flush().unwrap(); + drop(writer); + data +} + +/// MKV block payloads across payload and transport chunk sizes. +fn mkv_payloads(c: &mut Criterion) { + let runtime = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .unwrap(); + let _enter = runtime.enter(); + let mut group = c.benchmark_group("mkv-payloads"); + for payload in [1024, 64 * 1024, 1024 * 1024] { + let data = mkv_payload_file(payload, 4 * 1024 * 1024); + group.throughput(Throughput::Bytes(data.len() as u64)); + for chunk in [1500, 64 * 1024, data.len()] { + let id = BenchmarkId::new(format!("payload-{payload}"), format!("chunk-{chunk}")); + group.bench_function(id, |b| { + b.iter_batched( + || { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = moq_mux::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let import = moq_mux::container::mkv::Import::new(broadcast, catalog.reserve()); + (catalog, import) + }, + |(catalog, mut import)| { + for bytes in std::hint::black_box(&data).chunks(chunk) { + import.decode(bytes).unwrap(); + } + import.finish().unwrap(); + (catalog, import) + }, + BatchSize::PerIteration, + ); + }); + } + } + group.finish(); +} + criterion_group!( benches, containers, mkv_chunks, + mkv_payloads, interleaved_runs, queued_groups, ts_si_renewal, diff --git a/rs/moq-mux/src/catalog/hang/container.rs b/rs/moq-mux/src/catalog/hang/container.rs index 85acc91d783..0186bd8e140 100644 --- a/rs/moq-mux/src/catalog/hang/container.rs +++ b/rs/moq-mux/src/catalog/hang/container.rs @@ -47,6 +47,18 @@ impl TryFrom<&crate::catalog::VideoHint> for Container { } } +impl Container { + /// The bytes preceding a frame's payload, when they depend on nothing but its timestamp, so the + /// payload can stream in after them. `None` for CMAF, whose `moof` describes the payload. + pub(crate) fn prefix(&self, timestamp: moq_net::Timestamp) -> crate::Result> { + match self { + Self::Legacy(kind) => legacy::Wire(*kind).prefix(timestamp).map(Some), + Self::Loc(kind) => loc::Wire(*kind).prefix(timestamp).map(Some), + Self::Cmaf(_) => Ok(None), + } + } +} + /// Whether a rendition's frames can be parsed by this build, logging the ones dropped. /// /// The hang spec requires a consumer to ignore a rendition whose container `kind` it does not diff --git a/rs/moq-mux/src/container/legacy/mod.rs b/rs/moq-mux/src/container/legacy/mod.rs index 6a30aa60dfa..d7fcd521a2b 100644 --- a/rs/moq-mux/src/container/legacy/mod.rs +++ b/rs/moq-mux/src/container/legacy/mod.rs @@ -12,6 +12,19 @@ pub struct Wire( pub Kind, ); +impl Wire { + /// The bytes preceding a frame's payload: its varint timestamp. + pub(crate) fn prefix(&self, timestamp: moq_net::Timestamp) -> crate::Result { + let mut prefix = bytes::BytesMut::new(); + hang::container::Frame { + timestamp, + payload: bytes::Bytes::new(), + } + .encode(&mut prefix)?; + Ok(prefix.freeze()) + } +} + impl Container for Wire { type Error = crate::Error; diff --git a/rs/moq-mux/src/container/loc/mod.rs b/rs/moq-mux/src/container/loc/mod.rs index 16b53cc5dd1..80ca19e6cf2 100644 --- a/rs/moq-mux/src/container/loc/mod.rs +++ b/rs/moq-mux/src/container/loc/mod.rs @@ -19,6 +19,14 @@ pub struct Wire( pub Kind, ); +impl Wire { + /// The bytes preceding a frame's payload: its property block. + pub(crate) fn prefix(&self, timestamp: Timestamp) -> crate::Result { + let timestamp = timestamp.convert(DEFAULT_TIMESCALE).map_err(hang::Error::from)?; + Ok(moq_loc::encode(timestamp.value(), &[])?) + } +} + impl Container for Wire { type Error = crate::Error; diff --git a/rs/moq-mux/src/container/mkv/ebml.rs b/rs/moq-mux/src/container/mkv/ebml.rs new file mode 100644 index 00000000000..980029b7c62 --- /dev/null +++ b/rs/moq-mux/src/container/mkv/ebml.rs @@ -0,0 +1,687 @@ +//! Incremental EBML framing for the MKV importer. +//! +//! [`Reader`] walks the element tree as bytes arrive, retaining only a split element header and +//! the body of an element its caller chose to read whole. The caller answers every +//! [`Event::Start`] with how to consume that element, so a block streams through and an ignored +//! element is discarded as its bytes arrive, whatever size either declares. + +use super::Error; + +/// Matroska element IDs, with their length markers, as they appear on the wire. +pub(super) mod id { + pub const EBML: u32 = 0x1A45_DFA3; + pub const DOC_TYPE: u32 = 0x4282; + + pub const SEGMENT: u32 = 0x1853_8067; + pub const SEEK_HEAD: u32 = 0x114D_9B74; + pub const INFO: u32 = 0x1549_A966; + pub const TRACKS: u32 = 0x1654_AE6B; + pub const CLUSTER: u32 = 0x1F43_B675; + pub const CUES: u32 = 0x1C53_BB6B; + pub const CHAPTERS: u32 = 0x1043_A770; + pub const TAGS: u32 = 0x1254_C367; + pub const ATTACHMENTS: u32 = 0x1941_A469; + + pub const TIMESTAMP_SCALE: u32 = 0x2A_D7B1; + + pub const TRACK_ENTRY: u32 = 0xAE; + pub const TRACK_NUMBER: u32 = 0xD7; + pub const TRACK_TYPE: u32 = 0x83; + pub const CODEC_ID: u32 = 0x86; + pub const CODEC_PRIVATE: u32 = 0x63A2; + pub const DEFAULT_DURATION: u32 = 0x23_E383; + + pub const VIDEO: u32 = 0xE0; + pub const PIXEL_WIDTH: u32 = 0xB0; + pub const PIXEL_HEIGHT: u32 = 0xBA; + pub const DISPLAY_WIDTH: u32 = 0x54B0; + pub const DISPLAY_HEIGHT: u32 = 0x54BA; + pub const DISPLAY_UNIT: u32 = 0x54B2; + pub const PIXEL_CROP_BOTTOM: u32 = 0x54AA; + pub const PIXEL_CROP_TOP: u32 = 0x54BB; + pub const PIXEL_CROP_LEFT: u32 = 0x54CC; + pub const PIXEL_CROP_RIGHT: u32 = 0x54DD; + + pub const STEREO_MODE: u32 = 0x53B8; + pub const PROJECTION: u32 = 0x7670; + pub const PROJECTION_TYPE: u32 = 0x7671; + pub const PROJECTION_PRIVATE: u32 = 0x7672; + pub const PROJECTION_POSE_YAW: u32 = 0x7673; + pub const PROJECTION_POSE_PITCH: u32 = 0x7674; + pub const PROJECTION_POSE_ROLL: u32 = 0x7675; + + pub const COLOUR: u32 = 0x55B0; + pub const MATRIX_COEFFICIENTS: u32 = 0x55B1; + pub const RANGE: u32 = 0x55B9; + pub const TRANSFER_CHARACTERISTICS: u32 = 0x55BA; + pub const PRIMARIES: u32 = 0x55BB; + pub const MAX_CLL: u32 = 0x55BC; + pub const MAX_FALL: u32 = 0x55BD; + pub const MASTERING_METADATA: u32 = 0x55D0; + /// The first of MasteringMetadata's chromaticity and luminance fields, which are contiguous. + pub const PRIMARY_R_CHROMATICITY_X: u32 = 0x55D1; + /// The last of MasteringMetadata's fields. + pub const LUMINANCE_MIN: u32 = 0x55DA; + + pub const AUDIO: u32 = 0xE1; + pub const SAMPLING_FREQUENCY: u32 = 0xB5; + pub const CHANNELS: u32 = 0x9F; + + pub const TIMESTAMP: u32 = 0xE7; + pub const SIMPLE_BLOCK: u32 = 0xA3; + pub const BLOCK_GROUP: u32 = 0xA0; + pub const BLOCK: u32 = 0xA1; + pub const REFERENCE_BLOCK: u32 = 0xFB; +} + +/// The longest element header: a 4-byte ID (EBMLMaxIDLength) and an 8-byte size. +const MAX_HEADER: usize = 12; + +/// An element's ID and where its body ends. +#[derive(Clone, Copy, Debug)] +pub(super) struct Header { + pub id: u32, + /// The input position one past the body, or `None` for an unknown size. + pub end: Option, + /// Body bytes the header declares, or `None` for an unknown size. + pub size: Option, +} + +/// One step through the element tree. +#[derive(Debug)] +pub(super) enum Event<'a> { + /// An element starts. Answer with [`Reader::descend`], [`Reader::read`], [`Reader::stream`], + /// or [`Reader::skip`] before the next call. + Start(Header), + /// A [`read`](Reader::read) element's whole body. + Value(u32, Vec), + /// The next bytes of a [`stream`](Reader::stream)ed element's body, borrowed from the input. + Chunk(&'a [u8]), + /// A descended or streamed element ended. + End(u32), +} + +struct Open { + id: u32, + end: Option, +} + +enum Body { + /// Between elements: the next bytes are a header. + Between, + /// A parsed header, until no unknown-size master it closes remains open. + Parsed(Header), + /// A started element awaiting the caller's choice. + Pending(Header), + Read { + id: u32, + end: u64, + data: Vec, + }, + Stream { + id: u32, + end: u64, + }, + Skip { + end: u64, + }, +} + +/// Walks EBML elements across input chunks split anywhere. +pub(super) struct Reader { + /// Descended masters, outermost first. + open: Vec, + /// Input position of the next unconsumed byte. + position: u64, + /// A header split across chunks, at most [`MAX_HEADER`] bytes. + header: Vec, + body: Body, +} + +impl Reader { + pub fn new() -> Self { + Self { + open: Vec::new(), + position: 0, + header: Vec::with_capacity(MAX_HEADER), + body: Body::Between, + } + } + + /// Input bytes consumed so far. + #[cfg(test)] + pub fn position(&self) -> u64 { + self.position + } + + /// Bytes held for a header split across chunks or a read element's body. + #[cfg(test)] + pub fn retained(&self) -> usize { + let body = match &self.body { + Body::Read { data, .. } => data.capacity(), + _ => 0, + }; + self.header.capacity() + body + } + + /// The innermost descended master. + pub fn parent(&self) -> Option { + self.open.last().map(|open| open.id) + } + + /// Where the innermost descended master ends, if its size is known. + pub fn parent_end(&self) -> Option { + self.open.last().and_then(|open| open.end) + } + + /// Where the innermost open master ends: its declared end, or an unknown-size master's nearest + /// sized ancestor's. + fn bound(&self) -> Option { + self.open.iter().rev().find_map(|open| open.end) + } + + /// Enter the started master; its children follow. + pub fn descend(&mut self) -> Result<(), Error> { + let header = self.take_pending(); + // Only Segment and Cluster may have an unknown size, and only they can find their end + // without a schema of every child. + if header.end.is_none() && !matches!(header.id, id::SEGMENT | id::CLUSTER) { + return Err(Error::MatroskaParse); + } + self.open.push(Open { + id: header.id, + end: header.end, + }); + Ok(()) + } + + /// Collect the started element's body into one [`Event::Value`]. + pub fn read(&mut self) -> Result<(), Error> { + let header = self.take_pending(); + let end = header.end.ok_or(Error::MatroskaParse)?; + self.body = Body::Read { + id: header.id, + end, + data: Vec::new(), + }; + Ok(()) + } + + /// Pass the started element's body through as [`Event::Chunk`]s, then [`Event::End`]. + pub fn stream(&mut self) -> Result<(), Error> { + let header = self.take_pending(); + let end = header.end.ok_or(Error::MatroskaParse)?; + self.body = Body::Stream { id: header.id, end }; + Ok(()) + } + + /// Discard the started element's body as it arrives. + pub fn skip(&mut self) -> Result<(), Error> { + let header = self.take_pending(); + let end = header.end.ok_or(Error::MatroskaParse)?; + self.body = Body::Skip { end }; + Ok(()) + } + + fn take_pending(&mut self) -> Header { + match std::mem::replace(&mut self.body, Body::Between) { + Body::Pending(header) => header, + _ => panic!("no started element awaits a choice"), + } + } + + /// The next event `input` completes, consuming the bytes it spans. `None` once `input` is + /// exhausted mid-element or between elements. + pub fn next<'a>(&mut self, input: &mut &'a [u8]) -> Result>, Error> { + loop { + match &mut self.body { + Body::Between => { + // A master ends at its declared end, and an unknown-size one at its nearest + // sized ancestor's. + if let Some(open) = self.open.last() + && self.bound() == Some(self.position) + { + let id = open.id; + self.open.pop(); + return Ok(Some(Event::End(id))); + } + let Some(header) = self.header(input)? else { + return Ok(None); + }; + self.body = Body::Parsed(header); + } + Body::Parsed(header) => { + let header = *header; + // An unknown-size master ends where an element it cannot contain starts. + if let Some(open) = self.open.last() + && open.end.is_none() + && closes(open.id, header.id) + { + let id = open.id; + self.open.pop(); + return Ok(Some(Event::End(id))); + } + if let Some(bound) = self.bound() + && header.end.unwrap_or(self.position) > bound + { + return Err(Error::MatroskaParse); + } + self.body = Body::Pending(header); + return Ok(Some(Event::Start(header))); + } + // A refused choice leaves the element unread, so nothing after it can be framed. + Body::Pending(_) => return Err(Error::MatroskaParse), + Body::Read { id, end, data } => { + let taken = take(input, *end - self.position); + data.extend_from_slice(taken); + self.position += taken.len() as u64; + if self.position < *end { + return Ok(None); + } + let event = Event::Value(*id, std::mem::take(data)); + self.body = Body::Between; + return Ok(Some(event)); + } + Body::Stream { id, end } => { + if self.position == *end { + let id = *id; + self.body = Body::Between; + return Ok(Some(Event::End(id))); + } + let taken = take(input, *end - self.position); + if taken.is_empty() { + return Ok(None); + } + self.position += taken.len() as u64; + return Ok(Some(Event::Chunk(taken))); + } + Body::Skip { end } => { + let taken = take(input, *end - self.position); + self.position += taken.len() as u64; + if self.position < *end { + return Ok(None); + } + self.body = Body::Between; + } + } + } + } + + /// The next event at the end of input: unknown-size masters close, and anything else still + /// open was truncated. + pub fn finish(&mut self) -> Result>, Error> { + if let Some(event) = self.next(&mut &[][..])? { + return Ok(Some(event)); + } + if !self.header.is_empty() || !matches!(self.body, Body::Between) { + return Err(Error::Truncated); + } + match self.open.pop() { + Some(Open { id, end: None }) => Ok(Some(Event::End(id))), + Some(Open { end: Some(_), .. }) => Err(Error::Truncated), + None => Ok(None), + } + } + + /// Parse the next element header, joining one split across chunks. + fn header(&mut self, input: &mut &[u8]) -> Result, Error> { + let start = self.position - self.header.len() as u64; + let parsed = if self.header.is_empty() { + let parsed = parse(input)?; + if parsed.is_none() { + self.header.extend_from_slice(input); + } + parsed + } else { + // Join byte by byte, so nothing past the header is consumed. + let mut parsed = None; + while parsed.is_none() + && let Some((&byte, rest)) = input.split_first() + { + self.header.push(byte); + *input = rest; + parsed = parse(&self.header)?; + } + parsed + }; + let Some((id, size, len)) = parsed else { + // The bytes held are a header prefix that needs more bytes, so it cannot reach the + // parent's end. + self.position = start + self.header.len() as u64; + if let Some(bound) = self.bound() + && self.position >= bound + { + return Err(Error::MatroskaParse); + } + *input = &[]; + return Ok(None); + }; + if self.header.is_empty() { + *input = &input[len..]; + } + self.header.clear(); + self.position = start + len as u64; + let end = size + .map(|size| self.position.checked_add(size).ok_or(Error::MatroskaParse)) + .transpose()?; + Ok(Some(Header { id, end, size })) + } +} + +/// Split up to `max` bytes off the front of `input`. +fn take<'a>(input: &mut &'a [u8], max: u64) -> &'a [u8] { + let len = usize::try_from(max).unwrap_or(usize::MAX).min(input.len()); + let (taken, rest) = input.split_at(len); + *input = rest; + taken +} + +/// Parse an element header from the front of `buf`: its ID, its size (`None` when unknown), and +/// the header's length. `None` when `buf` holds only part of it. +fn parse(buf: &[u8]) -> Result, usize)>, Error> { + let Some(&first) = buf.first() else { + return Ok(None); + }; + // IDs keep their length marker, and are at most 4 bytes (EBMLMaxIDLength). + let id_len = first.leading_zeros() as usize + 1; + if id_len > 4 { + return Err(Error::MatroskaParse); + } + let Some(id) = buf.get(..id_len) else { + return Ok(None); + }; + let id = id.iter().fold(0u32, |id, byte| (id << 8) | u32::from(*byte)); + + let Some(&first) = buf.get(id_len) else { + return Ok(None); + }; + let size_len = first.leading_zeros() as usize + 1; + if size_len > 8 { + return Err(Error::MatroskaParse); + } + let Some(size) = buf.get(id_len..id_len + size_len) else { + return Ok(None); + }; + let mut value = u64::from(first) & (0xff >> size_len); + for byte in &size[1..] { + value = (value << 8) | u64::from(*byte); + } + // Every value bit set marks an unknown size, at any length. + let unknown = value == (1u64 << (7 * size_len)) - 1; + Ok(Some((id, (!unknown).then_some(value), id_len + size_len))) +} + +/// Whether an element with `child`'s ID ends the unknown-size master `parent`. +fn closes(parent: u32, child: u32) -> bool { + match parent { + id::CLUSTER => matches!( + child, + id::EBML + | id::SEGMENT + | id::SEEK_HEAD + | id::INFO | id::TRACKS + | id::CLUSTER + | id::CUES | id::CHAPTERS + | id::TAGS | id::ATTACHMENTS + ), + id::SEGMENT => matches!(child, id::EBML | id::SEGMENT), + _ => false, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use ebml_iterable::specs::EbmlTag; + use webm_iterable::matroska_spec::{Master, MatroskaSpec}; + + #[test] + fn ids_match_the_matroska_spec() { + let expected = [ + (id::EBML, MatroskaSpec::Ebml(Master::Start)), + (id::DOC_TYPE, MatroskaSpec::DocType(String::new())), + (id::SEGMENT, MatroskaSpec::Segment(Master::Start)), + (id::SEEK_HEAD, MatroskaSpec::SeekHead(Master::Start)), + (id::INFO, MatroskaSpec::Info(Master::Start)), + (id::TRACKS, MatroskaSpec::Tracks(Master::Start)), + (id::CLUSTER, MatroskaSpec::Cluster(Master::Start)), + (id::CUES, MatroskaSpec::Cues(Master::Start)), + (id::CHAPTERS, MatroskaSpec::Chapters(Master::Start)), + (id::TAGS, MatroskaSpec::Tags(Master::Start)), + (id::ATTACHMENTS, MatroskaSpec::Attachments(Master::Start)), + (id::TIMESTAMP_SCALE, MatroskaSpec::TimestampScale(0)), + (id::TRACK_ENTRY, MatroskaSpec::TrackEntry(Master::Start)), + (id::TRACK_NUMBER, MatroskaSpec::TrackNumber(0)), + (id::TRACK_TYPE, MatroskaSpec::TrackType(0)), + (id::CODEC_ID, MatroskaSpec::CodecID(String::new())), + (id::CODEC_PRIVATE, MatroskaSpec::CodecPrivate(Vec::new())), + (id::DEFAULT_DURATION, MatroskaSpec::DefaultDuration(0)), + (id::VIDEO, MatroskaSpec::Video(Master::Start)), + (id::PIXEL_WIDTH, MatroskaSpec::PixelWidth(0)), + (id::PIXEL_HEIGHT, MatroskaSpec::PixelHeight(0)), + (id::DISPLAY_WIDTH, MatroskaSpec::DisplayWidth(0)), + (id::DISPLAY_HEIGHT, MatroskaSpec::DisplayHeight(0)), + (id::DISPLAY_UNIT, MatroskaSpec::DisplayUnit(0)), + (id::PIXEL_CROP_BOTTOM, MatroskaSpec::PixelCropBottom(0)), + (id::PIXEL_CROP_TOP, MatroskaSpec::PixelCropTop(0)), + (id::PIXEL_CROP_LEFT, MatroskaSpec::PixelCropLeft(0)), + (id::PIXEL_CROP_RIGHT, MatroskaSpec::PixelCropRight(0)), + (id::STEREO_MODE, MatroskaSpec::StereoMode(0)), + (id::PROJECTION, MatroskaSpec::Projection(Master::Start)), + (id::PROJECTION_TYPE, MatroskaSpec::ProjectionType(0)), + (id::PROJECTION_PRIVATE, MatroskaSpec::ProjectionPrivate(Vec::new())), + (id::PROJECTION_POSE_YAW, MatroskaSpec::ProjectionPoseYaw(0.0)), + (id::PROJECTION_POSE_PITCH, MatroskaSpec::ProjectionPosePitch(0.0)), + (id::PROJECTION_POSE_ROLL, MatroskaSpec::ProjectionPoseRoll(0.0)), + (id::COLOUR, MatroskaSpec::Colour(Master::Start)), + (id::MATRIX_COEFFICIENTS, MatroskaSpec::MatrixCoefficients(0)), + (id::RANGE, MatroskaSpec::Range(0)), + (id::TRANSFER_CHARACTERISTICS, MatroskaSpec::TransferCharacteristics(0)), + (id::PRIMARIES, MatroskaSpec::Primaries(0)), + (id::MAX_CLL, MatroskaSpec::MaxCLL(0)), + (id::MAX_FALL, MatroskaSpec::MaxFALL(0)), + (id::MASTERING_METADATA, MatroskaSpec::MasteringMetadata(Master::Start)), + (id::PRIMARY_R_CHROMATICITY_X, MatroskaSpec::PrimaryRChromaticityX(0.0)), + (id::LUMINANCE_MIN, MatroskaSpec::LuminanceMin(0.0)), + (id::AUDIO, MatroskaSpec::Audio(Master::Start)), + (id::SAMPLING_FREQUENCY, MatroskaSpec::SamplingFrequency(0.0)), + (id::CHANNELS, MatroskaSpec::Channels(0)), + (id::TIMESTAMP, MatroskaSpec::Timestamp(0)), + (id::SIMPLE_BLOCK, MatroskaSpec::SimpleBlock(Vec::new())), + (id::BLOCK_GROUP, MatroskaSpec::BlockGroup(Master::Start)), + (id::BLOCK, MatroskaSpec::Block(Vec::new())), + (id::REFERENCE_BLOCK, MatroskaSpec::ReferenceBlock(0)), + ]; + for (id, tag) in expected { + assert_eq!(u64::from(id), tag.get_id(), "{tag:?}"); + } + } + + /// Feed `data` in `chunk`-byte pieces, choosing `descend` for masters and `read` otherwise. + fn walk(data: &[u8], chunk: usize) -> Vec { + let mut reader = Reader::new(); + let mut events = Vec::new(); + // Chunk boundaries differ by split, so a streamed body is reported once, whole. + let mut streamed = Vec::new(); + let mut handle = |reader: &mut Reader, event: Event| match event { + Event::Start(header) => { + events.push(format!("start {:x}", header.id)); + match header.id { + id::SEGMENT | id::CLUSTER | id::TRACKS => reader.descend().unwrap(), + id::SIMPLE_BLOCK => reader.stream().unwrap(), + _ => reader.read().unwrap(), + } + } + Event::Value(id, data) => events.push(format!("value {id:x} {data:?}")), + Event::Chunk(data) => streamed.extend_from_slice(data), + Event::End(id) => { + if !streamed.is_empty() { + events.push(format!("chunk {:?}", std::mem::take(&mut streamed))); + } + events.push(format!("end {id:x}")); + } + }; + for mut input in data.chunks(chunk) { + while let Some(event) = reader.next(&mut input).unwrap() { + handle(&mut reader, event); + } + assert!(input.is_empty()); + } + while let Some(event) = reader.finish().unwrap() { + handle(&mut reader, event); + } + events + } + + #[test] + fn every_split_yields_the_same_events() { + // Unknown-size Segment > unknown-size Cluster > Timestamp, SimpleBlock; then a sized + // Cluster that closes the first. + let data = [ + 0x18, 0x53, 0x80, 0x67, 0x01, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, // Segment + 0x1f, 0x43, 0xb6, 0x75, 0xff, // Cluster, unknown size in one byte + 0xe7, 0x81, 0x05, // Timestamp 5 + 0xa3, 0x83, 1, 2, 3, // SimpleBlock + 0x1f, 0x43, 0xb6, 0x75, 0x83, // Cluster of 3 bytes + 0xe7, 0x81, 0x06, // Timestamp 6 + ]; + let expected = walk(&data, data.len()); + assert_eq!( + expected, + [ + "start 18538067", + "start 1f43b675", + "start e7", + "value e7 [5]", + "start a3", + "chunk [1, 2, 3]", + "end a3", + "end 1f43b675", + "start 1f43b675", + "start e7", + "value e7 [6]", + "end 1f43b675", + "end 18538067", + ] + ); + for chunk in 1..data.len() { + assert_eq!(walk(&data, chunk), expected, "chunk {chunk}"); + } + } + + #[test] + fn an_unknown_size_cluster_ends_with_its_sized_segment() { + let data = [ + 0x18, 0x53, 0x80, 0x67, 0x88, // Segment of 8 bytes + 0x1f, 0x43, 0xb6, 0x75, 0xff, // Cluster, unknown size + 0xe7, 0x81, 0x05, // Timestamp 5 + 0xe7, 0x81, 0x06, // a top-level Timestamp after the Segment + ]; + let expected = [ + "start 18538067", + "start 1f43b675", + "start e7", + "value e7 [5]", + "end 1f43b675", + "end 18538067", + "start e7", + "value e7 [6]", + ]; + for chunk in 1..=data.len() { + assert_eq!(walk(&data, chunk), expected, "chunk {chunk}"); + } + } + + #[test] + fn a_child_cannot_overrun_its_parent() { + // Tracks of 3 bytes holding a 2-byte element whose body needs 2 more. + let data = [0x16, 0x54, 0xae, 0x6b, 0x83, 0xd7, 0x82, 0x01, 0x02]; + let mut reader = Reader::new(); + let mut input = &data[..]; + let Some(Event::Start(_)) = reader.next(&mut input).unwrap() else { + panic!("expected Tracks"); + }; + reader.descend().unwrap(); + assert!(matches!(reader.next(&mut input), Err(Error::MatroskaParse))); + } + + #[test] + fn a_header_cannot_straddle_its_parents_end() { + // Tracks of 1 byte, followed by a 2-byte ID and an empty size: an empty CodecPrivate if the + // header escaped its parent. + let data = [0x16, 0x54, 0xae, 0x6b, 0x81, 0x63, 0xa2, 0x80]; + // Every split is refused, including one exactly at the parent's end. + for split in 0..=data.len() { + let mut reader = Reader::new(); + let mut refused = false; + 'chunks: for mut input in [&data[..split], &data[split..]] { + loop { + match reader.next(&mut input) { + Ok(Some(Event::Start(header))) if header.id == id::TRACKS => reader.descend().unwrap(), + Ok(Some(event)) => panic!("split {split}: unexpected {event:?}"), + Ok(None) => break, + Err(err) => { + assert!(matches!(err, Error::MatroskaParse), "split {split}: {err:?}"); + refused = true; + break 'chunks; + } + } + } + } + assert!(refused, "split {split}"); + } + } + + #[test] + fn only_segment_and_cluster_may_have_an_unknown_size() { + let data = [0x16, 0x54, 0xae, 0x6b, 0xff]; + let mut reader = Reader::new(); + let mut input = &data[..]; + let Some(Event::Start(header)) = reader.next(&mut input).unwrap() else { + panic!("expected Tracks"); + }; + assert!(header.end.is_none()); + assert!(matches!(reader.descend(), Err(Error::MatroskaParse))); + } + + #[test] + fn truncation_is_refused_at_eof() { + for data in [ + &[0x1a, 0x45][..], // a split ID + &[0x1a, 0x45, 0xdf, 0xa3, 0x84, 0x42][..], // a sized master's partial child header + ] { + let mut reader = Reader::new(); + let mut input = data; + while let Some(event) = reader.next(&mut input).unwrap() { + if let Event::Start(_) = event { + reader.descend().unwrap(); + } + } + assert!(matches!(reader.finish(), Err(Error::Truncated)), "{data:?}"); + } + } + + #[test] + fn skipped_bytes_are_never_retained() { + // An EBML header declaring 1 GiB, holding a Void declaring nearly as much. + let mut reader = Reader::new(); + let mut input = &[ + 0x1a, 0x45, 0xdf, 0xa3, 0x08, 0x40, 0, 0, 0, 0xec, 0x08, 0x3f, 0xff, 0xff, 0xfa, + ][..]; + let Some(Event::Start(_)) = reader.next(&mut input).unwrap() else { + panic!("expected EBML"); + }; + reader.descend().unwrap(); + let Some(Event::Start(_)) = reader.next(&mut input).unwrap() else { + panic!("expected Void"); + }; + reader.skip().unwrap(); + for _ in 0..64 { + let mut input = &[0u8; 1024][..]; + assert!(reader.next(&mut input).unwrap().is_none()); + assert!(input.is_empty()); + assert_eq!(reader.retained(), MAX_HEADER); + } + assert!(matches!(reader.finish(), Err(Error::Truncated))); + } +} diff --git a/rs/moq-mux/src/container/mkv/import.rs b/rs/moq-mux/src/container/mkv/import.rs index 5bb44b04a26..16c35cca0eb 100644 --- a/rs/moq-mux/src/container/mkv/import.rs +++ b/rs/moq-mux/src/container/mkv/import.rs @@ -1,15 +1,17 @@ use std::collections::{HashMap, HashSet}; -use std::convert::TryFrom; use std::io::Cursor; use crate::Result; use bytes::{Bytes, BytesMut}; +use ebml_iterable::specs::{EbmlSpecification, EbmlTag, TagDataType}; use hang::catalog::{AAC, AudioCodec, AudioConfig, H265, VP9, VideoCodec, VideoConfig}; use moq_net::Timestamp; use mp4_atom::Atom; -use webm_iterable::matroska_spec::{BlockLacing, Master, MatroskaSpec, SimpleBlock}; +use webm_iterable::matroska_spec::{Master, MatroskaSpec}; use super::Error; +use super::ebml::{self, Event, Header, id}; +use crate::container::producer::Streamed; /// Default Matroska TimestampScale: 1 ms (in nanoseconds). const DEFAULT_TIMESTAMP_SCALE_NS: u64 = 1_000_000; @@ -34,11 +36,24 @@ const DEFAULT_TIMESTAMP_SCALE_NS: u64 = 1_000_000; /// - FLAC (`A_FLAC`) /// - MP3 (`A_MPEG/L3`) /// -/// **Text:** timed SRT (`S_TEXT/UTF8`) and WebVTT (`S_TEXT/WEBVTT`), emitted as WebVTT. -/// /// Laced blocks require a positive TrackEntry DefaultDuration to timestamp each frame. /// /// Unsupported track roles are skipped; malformed tracks and unsupported codecs fail the import. +/// +/// ## One pass +/// +/// Each block's payload streams into its frame as its bytes arrive, readable before the block +/// completes, and elements the importer does not use are discarded as they arrive. The importer +/// retains no more than an element header and the track fields it reads, whatever sizes the input +/// declares; the published frames are held by their tracks, which refuse one too large for a group. +/// +/// Input whose output depends on bytes after a payload is refused before that payload is published: +/// - a video `BlockGroup` whose `Block` neither follows a `ReferenceBlock` nor ends the group, +/// since a later `ReferenceBlock` would decide whether it starts a group; +/// - subtitle tracks, whose WebVTT cues change length with their content; +/// - [`Cmaf`](hang::catalog::Container::Cmaf) output, whose `moof` describes a payload before it. +/// +/// Remux refused video so its blocks are `SimpleBlock`s, and extract subtitles separately. pub struct Import { broadcast: moq_net::broadcast::Producer, catalog: crate::catalog::Producer, @@ -53,12 +68,24 @@ pub struct Import { /// offset onto the catalog clock. timebase: crate::catalog::Timebase, - /// Accumulated unparsed input. - buffer: BytesMut, - /// Incremental element state; an incomplete master retains its own children across chunks. - decoder: ebml_iterable::TagDecoder, - /// Whether `finish()` ended the input, even if it failed before the decoder drained. + /// Incremental element framing across input chunks. + reader: ebml::Reader, + /// The fields read from each metadata master still open, innermost last. A repeated field + /// replaces its earlier value, so each holds at most one value per field it reads. + fields: Vec>, + /// TrackNumbers the Tracks element declared so far. + numbers: HashSet, + /// Whether a ReferenceBlock preceded the current BlockGroup's Block. + referenced: bool, + /// The SimpleBlock or Block streaming in. + block: Option, + /// A seek waiting for the streaming block to end, since its later frames continue its group. + seek: Option, + /// Whether `finish()` ended the input, even if it failed before the input drained. finished: bool, + /// Why a chunk was refused. The rest of that chunk was never read, so later input cannot + /// continue it. + failed: Option, /// Whether the Tracks element has been processed. tracks_seen: bool, @@ -75,7 +102,6 @@ pub struct Import { enum TrackKind { Video, Audio, - Text, } impl TrackKind { @@ -85,7 +111,6 @@ impl TrackKind { match self { Self::Video => hang::catalog::PRIORITY.video, Self::Audio => hang::catalog::PRIORITY.audio, - Self::Text => hang::catalog::PRIORITY.text, } } } @@ -93,34 +118,32 @@ impl TrackKind { struct MkvTrack { default_duration: Option, kind: TrackKind, - text_header: Option, track: Media, - group: Option, -} - -/// BlockGroup fields that only subtitle tracks consume. -#[derive(Default)] -struct Cue<'a> { - /// BlockDuration, in Matroska ticks. - duration: Option, - /// WebVTT settings, identifier and comments (BlockAddID 1). - addition: Option<&'a [u8]>, - /// Whether the BlockGroup replaces the codec initialization (CodecState). - codec_state: bool, } enum Media { Video(crate::container::Producer), Audio(crate::container::Producer), - Text(crate::container::Producer), } impl Media { - fn write(&mut self, frame: crate::container::Frame) -> crate::Result<()> { + fn open( + &mut self, + timestamp: Timestamp, + keyframe: bool, + duration: Option, + size: usize, + ) -> crate::Result { match self { - Self::Video(track) => track.write(frame), - Self::Audio(track) => track.write(frame), - Self::Text(track) => track.write(frame), + Self::Video(track) => track.open(timestamp, keyframe, duration, size), + Self::Audio(track) => track.open(timestamp, keyframe, duration, size), + } + } + + fn commit(&mut self, frame: Streamed) -> crate::Result<()> { + match self { + Self::Video(track) => track.commit(frame), + Self::Audio(track) => track.commit(frame), } } @@ -128,7 +151,6 @@ impl Media { match self { Self::Video(track) => track.cut(timestamp), Self::Audio(track) => track.cut(timestamp), - Self::Text(track) => track.cut(timestamp), } } @@ -136,7 +158,6 @@ impl Media { match self { Self::Video(track) => track.seek(sequence), Self::Audio(track) => track.seek(sequence), - Self::Text(track) => track.seek(sequence), } } @@ -144,7 +165,6 @@ impl Media { match self { Self::Video(track) => track.finish(), Self::Audio(track) => track.finish(), - Self::Text(track) => track.finish(), } } @@ -152,7 +172,6 @@ impl Media { match self { Self::Video(track) => track.end(), Self::Audio(track) => track.end(), - Self::Text(track) => track.end(), } } @@ -160,7 +179,6 @@ impl Media { match self { Self::Video(track) => track.abort(err), Self::Audio(track) => track.abort(err), - Self::Text(track) => track.abort(err), } } @@ -168,11 +186,98 @@ impl Media { match self { Self::Video(track) => track.name(), Self::Audio(track) => track.name(), - Self::Text(track) => track.name(), } } } +/// How the importer consumes an element. +enum Action { + /// Walk its children. + Descend, + /// Walk its children, keeping the fields it reads for when it ends. + Collect, + /// Read its value whole: a field the importer uses. + Read, + /// Pass its body through: a block. + Stream, + /// Discard its body as it arrives. + Skip, +} + +/// A SimpleBlock or Block whose body is streaming in. +struct Block { + /// Body bytes still to arrive. + remaining: u64, + /// For a Block, how its BlockGroup decides its keyframe flag; a SimpleBlock's flags carry it. + grouped: Option, + state: BlockState, +} + +#[derive(Clone, Copy)] +struct Grouped { + /// A ReferenceBlock preceded the Block, so it continues a group. + referenced: bool, + /// The Block ends its BlockGroup, so no ReferenceBlock can follow it. + last: bool, +} + +enum BlockState { + /// The track number, relative timestamp, and flags, which may split across chunks. + Header(Vec), + /// Lace sizes, parsed a byte at a time. + Lacing(Lacing), + /// Frames streaming into their track. + Frames(Box), + /// A skipped track's payload, or the rest of a failed block, discarded. + Skip, +} + +/// What a block header says. +#[derive(Clone, Copy)] +struct Fixed { + track: u64, + timestamp: i16, + keyframe: bool, +} + +/// Matroska lacing, from the block flags. +#[derive(Clone, Copy, PartialEq)] +enum Lace { + Xiph, + FixedSize, + Ebml, +} + +struct Lacing { + block: Fixed, + lace: Lace, + /// The frame count, once its byte arrives. + count: Option, + /// Sizes of all but the last frame, as each completes. + sizes: Vec, + /// The Xiph size being summed. + sum: u64, + /// The EBML size being read. + vint: Vec, +} + +struct Frames { + track: u64, + kind: TrackKind, + /// The first frame's presentation time, in nanoseconds. + pts_ns: u64, + /// Nanoseconds between laced frames. + step: u64, + keyframe: bool, + duration: Option, + /// Every frame's size, in order. + sizes: Vec, + /// The next frame to open. + index: usize, + /// The frame being written. + open: Option, +} + impl Import { /// Import a byte stream into the broadcast using its reserved catalog producer. pub fn new(broadcast: moq_net::broadcast::Producer, reserved: crate::catalog::Reserved) -> Self { @@ -183,17 +288,14 @@ impl Import { container, timebase: reserved.timebase(), initial_reservation: Some(reserved), - buffer: BytesMut::new(), - decoder: ebml_iterable::TagDecoder::new(&[ - MatroskaSpec::Ebml(Master::Start), - MatroskaSpec::Info(Master::Start), - MatroskaSpec::Tracks(Master::Start), - MatroskaSpec::TrackEntry(Master::Start), - MatroskaSpec::Audio(Master::Start), - MatroskaSpec::Video(Master::Start), - MatroskaSpec::BlockGroup(Master::Start), - ]), + reader: ebml::Reader::new(), + fields: Vec::new(), + numbers: HashSet::new(), + referenced: false, + block: None, + seek: None, finished: false, + failed: None, tracks_seen: false, timestamp_scale_ns: DEFAULT_TIMESTAMP_SCALE_NS, cluster_timestamp: 0, @@ -205,84 +307,200 @@ impl Import { /// /// [`Legacy`](hang::catalog::Container::Legacy) unless selected. It applies to every rendition /// this input demuxes, since the tracks are discovered rather than named by the caller. + /// [`Cmaf`](hang::catalog::Container::Cmaf) is refused when the first track is declared, since a + /// block's payload streams into its frame. pub fn with_container(mut self, container: hang::catalog::Container) -> Self { self.container = container; self } - /// Append the buffer to the internal scratch and parse as many tags as possible. + /// Parse `data`, a chunk of the input split anywhere. /// - /// The buffer is fully consumed on every call (data is moved into the internal - /// scratch). Bytes that cannot yet form a complete top-level tag are retained - /// for the next call. + /// Every byte is consumed: a block's payload is written to its frame, an ignored element's is + /// discarded, and only an element header split across chunks or a field being read is retained. + /// A refused chunk fails the import: later input is refused with the same error, and + /// [`finish`](Self::finish) ends the tracks with what was accepted. pub fn decode(&mut self, data: &[u8]) -> Result<()> { if self.finished && !data.is_empty() { return Err(Error::TracksAfterFinish.into()); } - self.buffer.extend_from_slice(data); + if let Some(err) = &self.failed { + return Err(err.clone()); + } + let result = self.drain(data); + if let Err(err) = &result { + self.failed = Some(err.clone()); + } + result + } - self.drain() + fn drain_eof(&mut self) -> Result<()> { + while let Some(event) = self.reader.finish()? { + self.handle(event)?; + } + Ok(()) } - /// Decode complete elements, leaving incomplete bytes and master state for the next chunk. - fn drain(&mut self) -> Result<()> { - while let Some(tag) = self - .decoder - .decode(&mut self.buffer) - .map_err(|_| Error::MatroskaParse)? - { - self.handle_tag(tag.tag)?; + fn drain(&mut self, mut input: &[u8]) -> Result<()> { + while let Some(event) = self.reader.next(&mut input)? { + self.handle(event)?; } Ok(()) } - fn handle_tag(&mut self, tag: MatroskaSpec) -> Result<()> { - match tag { - MatroskaSpec::Ebml(Master::Full(children)) => { - self.handle_ebml(&children)?; + fn handle(&mut self, event: Event<'_>) -> Result<()> { + match event { + Event::Start(header) => self.start(header), + Event::Value(id, data) => self.value(id, data), + Event::Chunk(chunk) => self.chunk(chunk), + Event::End(id) => self.end(id), + } + } + + fn action(&self, parent: Option, header: &Header) -> Result { + use id::*; + let action = match (parent, header.id) { + (None, EBML) => Action::Collect, + (None, SEGMENT) => Action::Descend, + (Some(EBML), DOC_TYPE) => Action::Read, + (Some(SEGMENT), INFO | CLUSTER) => Action::Descend, + // A second Tracks element would redeclare a published track set. + (Some(SEGMENT), TRACKS) if !self.tracks_seen => Action::Descend, + (Some(INFO), TIMESTAMP_SCALE) => Action::Read, + (Some(TRACKS), TRACK_ENTRY) => Action::Collect, + (Some(TRACK_ENTRY), VIDEO | AUDIO) => Action::Collect, + (Some(TRACK_ENTRY), TRACK_NUMBER | TRACK_TYPE | CODEC_ID | CODEC_PRIVATE | DEFAULT_DURATION) => { + Action::Read } - MatroskaSpec::Segment(Master::Start) => { - // Just descend. + (Some(VIDEO), COLOUR | PROJECTION) => Action::Collect, + ( + Some(VIDEO), + PIXEL_WIDTH | PIXEL_HEIGHT | DISPLAY_WIDTH | DISPLAY_HEIGHT | DISPLAY_UNIT | PIXEL_CROP_BOTTOM + | PIXEL_CROP_TOP | PIXEL_CROP_LEFT | PIXEL_CROP_RIGHT | STEREO_MODE, + ) => Action::Read, + ( + Some(PROJECTION), + PROJECTION_TYPE | PROJECTION_POSE_YAW | PROJECTION_POSE_PITCH | PROJECTION_POSE_ROLL, + ) => Action::Read, + // Only the equirectangular `equi` body is understood, so other sizes are skipped and + // recorded as unsupported in `start`. + (Some(PROJECTION), PROJECTION_PRIVATE) if header.size == Some(EQUI_PRIVATE_LEN) => Action::Read, + (Some(COLOUR), MASTERING_METADATA) => Action::Collect, + (Some(COLOUR), MATRIX_COEFFICIENTS | RANGE | TRANSFER_CHARACTERISTICS | PRIMARIES | MAX_CLL | MAX_FALL) => { + Action::Read } - MatroskaSpec::Segment(Master::End) => {} - MatroskaSpec::Info(Master::Full(children)) => { - for c in &children { - if let MatroskaSpec::TimestampScale(v) = c { - if *v == 0 { - return Err(Error::ValueOutOfRange("TimestampScale").into()); - } - self.timestamp_scale_ns = *v; - } + (Some(MASTERING_METADATA), PRIMARY_R_CHROMATICITY_X..=LUMINANCE_MIN) => Action::Read, + (Some(AUDIO), SAMPLING_FREQUENCY | CHANNELS) => Action::Read, + (Some(CLUSTER), TIMESTAMP) => Action::Read, + (Some(CLUSTER), SIMPLE_BLOCK) => Action::Stream, + (Some(CLUSTER), BLOCK_GROUP) => Action::Descend, + (Some(BLOCK_GROUP), BLOCK) => Action::Stream, + _ => Action::Skip, + }; + // A number is at most 8 bytes, so a larger one is refused before its bytes are held. + if let Action::Read = action + && matches!( + MatroskaSpec::get_tag_data_type(header.id.into()), + Some(TagDataType::UnsignedInt | TagDataType::Integer | TagDataType::Float) + ) && header.size.is_none_or(|size| size > 8) + { + return Err(Error::MatroskaParse.into()); + } + Ok(action) + } + + fn start(&mut self, header: Header) -> Result<()> { + let parent = self.reader.parent(); + match self.action(parent, &header)? { + Action::Descend => self.reader.descend()?, + Action::Collect => { + self.reader.descend()?; + self.fields.push(Vec::new()); + } + Action::Read => self.reader.read()?, + Action::Stream => { + self.reader.stream()?; + let grouped = (header.id == id::BLOCK).then(|| Grouped { + referenced: self.referenced, + last: header.end == self.reader.parent_end(), + }); + self.block = Some(Block { + remaining: header.size.unwrap_or_default(), + grouped, + state: BlockState::Header(Vec::new()), + }); + } + Action::Skip => { + self.reader.skip()?; + if parent == Some(id::BLOCK_GROUP) && header.id == id::REFERENCE_BLOCK { + self.referenced = true; + } + // An empty body is never a supported one, so a projection that needs it is refused + // when its master ends, without holding the bytes. + if parent == Some(id::PROJECTION) && header.id == id::PROJECTION_PRIVATE { + set( + self.fields.last_mut().expect("Projection is collected"), + MatroskaSpec::ProjectionPrivate(Vec::new()), + )?; } } - // A second Tracks element would redeclare a published track set. - MatroskaSpec::Tracks(Master::Full(children)) if !self.tracks_seen => { + } + match header.id { + id::CLUSTER => self.cluster_timestamp = 0, + id::BLOCK_GROUP => self.referenced = false, + _ => {} + } + Ok(()) + } + + fn value(&mut self, id: u32, data: Vec) -> Result<()> { + let value = field(id, data)?; + match value { + MatroskaSpec::TimestampScale(0) => return Err(Error::ValueOutOfRange("TimestampScale").into()), + MatroskaSpec::TimestampScale(scale) => self.timestamp_scale_ns = scale, + // Within a Cluster, this is the cluster timestamp (in Matroska ticks). + MatroskaSpec::Timestamp(timestamp) => self.cluster_timestamp = timestamp, + value => set( + self.fields.last_mut().expect("a read field has a collecting parent"), + value, + )?, + } + Ok(()) + } + + fn end(&mut self, id: u32) -> Result<()> { + match id { + id::EBML => { + let children = self.fields.pop().unwrap_or_default(); + self.handle_ebml(&children)?; + } + id::TRACK_ENTRY => { + let children = self.fields.pop().unwrap_or_default(); // Only `finish()` releases the reservation before Tracks, since a block needs a track. let reserved = self.initial_reservation.clone().ok_or(Error::TracksAfterFinish)?; - self.handle_tracks(&reserved, children)?; + self.add_track(&reserved, children)?; + } + id::VIDEO | id::AUDIO | id::COLOUR | id::MASTERING_METADATA | id::PROJECTION => { + let children = self.fields.pop().unwrap_or_default(); + let master = + MatroskaSpec::get_master_tag(id.into(), Master::Full(children)).ok_or(Error::MatroskaParse)?; + set( + self.fields + .last_mut() + .expect("a collected master has a collecting parent"), + master, + )?; + } + id::TRACKS => { self.tracks_seen = true; + self.numbers.clear(); // The reservation stays held until the first block anchors the clock, unless no track // was declared: then no block ever will. if self.tracks.is_empty() { self.initial_reservation = None; } } - MatroskaSpec::Cluster(Master::Start) => { - self.cluster_timestamp = 0; - } - MatroskaSpec::Cluster(Master::End) => {} - MatroskaSpec::Timestamp(v) => { - // Within a Cluster, this is the cluster timestamp (in Matroska ticks). - self.cluster_timestamp = v; - } - MatroskaSpec::SimpleBlock(ref data) => { - let sb = SimpleBlock::try_from(data.as_slice()).map_err(|_| Error::InvalidSimpleBlock)?; - self.handle_block(&sb, sb.keyframe, Cue::default())?; - } - MatroskaSpec::BlockGroup(Master::Full(children)) => { - self.handle_block_group(&children)?; - } - // Tags we deliberately ignore. + id::SIMPLE_BLOCK | id::BLOCK => self.block_end()?, _ => {} } Ok(()) @@ -300,31 +518,13 @@ impl Import { Err(Error::MissingDocType.into()) } - fn handle_tracks(&mut self, reserved: &crate::catalog::Reserved, entries: Vec) -> Result<()> { - let mut numbers = HashSet::new(); - for entry in entries { - if let MatroskaSpec::TrackEntry(Master::Full(children)) = entry { - self.add_track(reserved, children, &mut numbers)?; - } - } - Ok(()) - } - - fn add_track( - &mut self, - reserved: &crate::catalog::Reserved, - children: Vec, - numbers: &mut HashSet, - ) -> Result<()> { + fn add_track(&mut self, reserved: &crate::catalog::Reserved, children: Vec) -> Result<()> { let mut track_number: Option = None; let mut track_type: Option = None; let mut codec_id: Option = None; let mut codec_private: Option = None; let mut audio_children: Option> = None; let mut video_children: Option> = None; - let mut language = None; - let mut language_ietf = None; - let mut name = None; let mut default_duration = None; for c in children { @@ -332,16 +532,9 @@ impl Import { MatroskaSpec::DefaultDuration(v) if v > 0 => { default_duration = Some(std::time::Duration::from_nanos(v)) } - MatroskaSpec::TrackNumber(v) => { - if track_number.replace(v).is_some() { - return Err(moq_net::Error::Duplicate.into()); - } - } + MatroskaSpec::TrackNumber(v) => track_number = Some(v), MatroskaSpec::TrackType(v) => track_type = Some(v), MatroskaSpec::CodecID(v) => codec_id = Some(v), - MatroskaSpec::Language(v) => language = Some(v), - MatroskaSpec::LanguageIETF(v) => language_ietf = Some(v), - MatroskaSpec::Name(v) => name = Some(v), MatroskaSpec::CodecPrivate(v) => codec_private = Some(Bytes::from(v)), MatroskaSpec::Audio(Master::Full(v)) => audio_children = Some(v), MatroskaSpec::Video(Master::Full(v)) => video_children = Some(v), @@ -353,7 +546,7 @@ impl Import { if track_number == 0 { return Err(Error::ValueOutOfRange("TrackNumber").into()); } - if !numbers.insert(track_number) { + if !self.numbers.insert(track_number) { return Err(moq_net::Error::Duplicate.into()); } let track_type = track_type.ok_or(Error::MissingTrackType)?; @@ -363,13 +556,15 @@ impl Import { let (kind, suffix) = match track_type { 1 => (TrackKind::Video, ".mkv-v"), 2 => (TrackKind::Audio, ".mkv-a"), - 17 if matches!(codec_id.as_str(), "S_TEXT/UTF8" | "S_TEXT/WEBVTT") => (TrackKind::Text, ".mkv-t"), - 17 => return Err(crate::Error::UnknownFormat(format!("MKV subtitle codec {codec_id}"))), + 17 => return Err(Error::SubtitleTrack(codec_id).into()), other => { tracing::warn!(track_type = other, codec_id, "unsupported MKV track type, skipping"); return Ok(()); } }; + if matches!(self.container, hang::catalog::Container::Cmaf { .. }) { + return Err(Error::StreamedCmaf.into()); + } let track = self.broadcast.create_track( self.broadcast.unique_name(suffix), @@ -383,30 +578,9 @@ impl Import { match kind { TrackKind::Video => crate::container::Kind::Video, TrackKind::Audio => crate::container::Kind::Audio, - TrackKind::Text => crate::container::Kind::Data, }, )?; - let text_header = if codec_id == "S_TEXT/WEBVTT" { - let private = codec_private - .as_deref() - .ok_or(Error::InvalidSubtitle("missing WebVTT codec private"))?; - let header = - std::str::from_utf8(private).map_err(|_| Error::InvalidSubtitle("invalid WebVTT header UTF-8"))?; - Some(crate::container::text::header(header).map_err(Error::InvalidSubtitle)?) - } else { - None - }; let media = match kind { - TrackKind::Text => { - let mut config = hang::catalog::TextConfig::new(hang::catalog::TextFormat::Vtt); - if let Some(header) = &text_header { - config.role = crate::container::text::role(header); - } - config.lang = language_ietf.or(language); - config.label = name; - config.container = self.container.clone(); - Media::Text(reserved.text(track, wire, config)?) - } TrackKind::Video => { let mut config = build_video_config(&codec_id, codec_private.as_ref(), video_children.as_deref())?; config.container = self.container.clone(); @@ -424,79 +598,101 @@ impl Import { MkvTrack { default_duration, kind, - text_header, track: media, - group: None, }, ); Ok(()) } - fn handle_block_group(&mut self, children: &[MatroskaSpec]) -> Result<()> { - let mut block_data: Option<&[u8]> = None; - let mut has_reference = false; - let mut duration = None; - let mut addition = None; - let mut codec_state = false; + /// Feed a chunk of the streaming block's body. + fn chunk(&mut self, chunk: &[u8]) -> Result<()> { + let Some(mut block) = self.block.take() else { + return Ok(()); + }; + let result = self.feed(&mut block, chunk); + if let Err(err) = &result { + // The rest of a failed block is discarded, and the frame it cut short fails its group. + if let BlockState::Frames(frames) = std::mem::replace(&mut block.state, BlockState::Skip) + && let Some(frame) = frames.open + { + frame.abort(err.reason()); + } + } + self.block = Some(block); + result + } - for c in children { - match c { - MatroskaSpec::Block(data) => block_data = Some(data.as_slice()), - MatroskaSpec::ReferenceBlock(_) => has_reference = true, - MatroskaSpec::BlockAdditions(Master::Full(entries)) => { - for entry in entries { - if let MatroskaSpec::BlockMore(Master::Full(fields)) = entry { - let mut id = 1; - let mut data = None; - for field in fields { - match field { - MatroskaSpec::BlockAddID(value) => id = *value, - MatroskaSpec::BlockAdditional(value) => data = Some(value.as_slice()), - _ => {} - } - } - if id == 1 { - if addition.is_some() { - return Err(Error::InvalidSubtitle("duplicate cue additions").into()); - } - addition = data; - } - } + fn feed(&mut self, block: &mut Block, mut chunk: &[u8]) -> Result<()> { + loop { + let next = match &mut block.state { + BlockState::Header(header) => { + let Some((&byte, rest)) = chunk.split_first() else { + return Ok(()); + }; + chunk = rest; + block.remaining -= 1; + header.push(byte); + let Some((fixed, lace)) = block_header(header, block.grouped.is_some())? else { + continue; + }; + if !self.tracks.contains_key(&fixed.track) { + // Unknown or skipped track. + BlockState::Skip + } else if let Some(lace) = lace { + BlockState::Lacing(Lacing { + block: fixed, + lace, + count: None, + sizes: Vec::new(), + sum: 0, + vint: Vec::new(), + }) + } else { + BlockState::Frames(Box::new(self.frames( + fixed, + block.grouped, + vec![block.remaining], + false, + )?)) } } - MatroskaSpec::BlockDuration(value) => duration = Some(*value), - MatroskaSpec::CodecState(_) => codec_state = true, - _ => {} - } + BlockState::Lacing(lacing) => { + let Some((&byte, rest)) = chunk.split_first() else { + return Ok(()); + }; + chunk = rest; + block.remaining -= 1; + let Some(sizes) = lacing.push(byte, block.remaining).map_err(anyhow::Error::new)? else { + continue; + }; + let fixed = lacing.block; + BlockState::Frames(Box::new(self.frames(fixed, block.grouped, sizes, true)?)) + } + BlockState::Frames(frames) => { + let written = self.write(frames, chunk)?; + block.remaining -= written as u64; + return Ok(()); + } + BlockState::Skip => { + block.remaining -= chunk.len() as u64; + return Ok(()); + } + }; + block.state = next; } - - let Some(data) = block_data else { - return Ok(()); - }; - - // `Block` has the same on-wire header as `SimpleBlock` minus the keyframe flag. - // We parse it via `SimpleBlock::try_from` (which works on the raw slice) but - // derive keyframe from the absence of `ReferenceBlock`. - let parsed = SimpleBlock::try_from(data).map_err(|_| Error::InvalidBlock)?; - let keyframe = !has_reference; - - self.handle_block( - &parsed, - keyframe, - Cue { - duration, - addition, - codec_state, - }, - ) } - fn handle_block(&mut self, block: &SimpleBlock<'_>, keyframe: bool, cue: Cue<'_>) -> Result<()> { - let Some(track) = self.tracks.get_mut(&block.track) else { - // Unknown or skipped track. - return Ok(()); - }; + /// Check a block's frames before any is published, and plan them. + fn frames(&mut self, block: Fixed, grouped: Option, sizes: Vec, laced: bool) -> Result { + let track = self.tracks.get(&block.track).expect("a planned block's track exists"); + if track.kind == TrackKind::Video + && let Some(grouped) = grouped + && !grouped.referenced + && !grouped.last + { + return Err(Error::TrailingReference.into()); + } // Compute PTS in MKV's native nanosecond units and stamp it on the // timestamp at NANO scale so a passthrough re-emit preserves precision. @@ -508,123 +704,149 @@ impl Import { } else { Error::TimestampOverflow })?; - let pts_ns = block_ticks .checked_mul(self.timestamp_scale_ns) .ok_or(Error::TimestampOverflow)?; - if track.kind == TrackKind::Text { - if block.lacing.is_some() { - return Err(Error::InvalidSubtitle("laced subtitles are unsupported").into()); - } - // A replaced WebVTT header would otherwise leave stale global blocks on every later cue. - if cue.codec_state { - return Err(Error::InvalidSubtitle("subtitle CodecState updates are unsupported").into()); - } - let duration = match cue.duration { - Some(ticks) => ticks - .checked_mul(self.timestamp_scale_ns) - .ok_or(Error::TimestampOverflow)?, - None => { - let duration = track - .default_duration - .ok_or(Error::InvalidSubtitle("missing cue duration"))?; - u64::try_from(duration.as_nanos()).map_err(|_| Error::TimestampOverflow)? - } - }; - let duration = Timestamp::from_nanos(duration)?; - let timestamp = self.timebase.shift(Timestamp::from_nanos(pts_ns)?)?; - self.initial_reservation = None; - let payload = cue_payload( - track.text_header.as_deref(), - cue.addition, - block.raw_frame_data(), - timestamp, - duration, - )?; - track.track.write(crate::container::Frame { - timestamp, - payload, - keyframe: true, - duration: Some(duration), - })?; - // Each cue is self-contained, so close its group immediately. - return track.track.cut(None); - } - let frames = split_lacing(block).map_err(anyhow::Error::new)?; - let duration = if block.lacing.is_some() { - Some( + let duration = match laced { + true => Some( track .default_duration .ok_or_else(|| anyhow::Error::new(LacingError::MissingDuration))?, - ) - } else { - None + ), + false => None, }; // Validate the last timestamp before publishing any part of this block. let step = duration.map(|duration| duration.as_nanos()).unwrap_or_default(); let duration = duration .map(|duration| Timestamp::from_nanos(duration.as_nanos() as u64)) .transpose()?; - let last = u128::from(pts_ns) + step * (frames.len() - 1) as u128; + let last = u128::from(pts_ns) + step * (sizes.len() - 1) as u128; Timestamp::from_nanos(u64::try_from(last).map_err(|_| Error::TimestampOverflow)?)?; - for (index, payload) in frames.into_iter().enumerate() { - let pts_ns = (u128::from(pts_ns) + step * index as u128) as u64; - // Anchor before releasing the reservation, so the first catalog carries its clock. - let timestamp = self.timebase.shift(Timestamp::from_nanos(pts_ns)?)?; - self.initial_reservation = None; - let frame = crate::container::Frame { - timestamp, - payload: Bytes::copy_from_slice(payload), - keyframe: match track.kind { - TrackKind::Audio => index == 0, - TrackKind::Video | TrackKind::Text => keyframe, - }, - duration, + + let mut frames = Frames { + track: block.track, + kind: track.kind, + pts_ns, + step: step as u64, + keyframe: grouped.map_or(block.keyframe, |grouped| !grouped.referenced), + duration, + sizes, + index: 0, + open: None, + }; + // An empty frame is due before any byte arrives. + self.write(&mut frames, &[])?; + Ok(frames) + } + + /// Write `chunk` into the block's frames, opening each as the last completes. Returns the bytes + /// written, which is all of `chunk` unless it overruns the block. + fn write(&mut self, frames: &mut Frames, mut chunk: &[u8]) -> Result { + let total = chunk.len(); + let track = self + .tracks + .get_mut(&frames.track) + .expect("a planned block's track exists"); + loop { + let frame = match frames.open.as_mut() { + Some(frame) => frame, + None => { + let Some(&size) = frames.sizes.get(frames.index) else { + break; + }; + let pts_ns = frames.pts_ns + frames.step * frames.index as u64; + // Anchor before releasing the reservation, so the first catalog carries its clock. + let timestamp = self.timebase.shift(Timestamp::from_nanos(pts_ns)?)?; + self.initial_reservation = None; + let keyframe = match frames.kind { + TrackKind::Audio => frames.index == 0, + TrackKind::Video => frames.keyframe, + }; + let size = usize::try_from(size).map_err(|_| moq_net::Error::FrameTooLarge)?; + frames.index += 1; + frames + .open + .insert(track.track.open(timestamp, keyframe, frames.duration, size)?) + } }; - track.track.write(frame)?; - } - if matches!(track.kind, TrackKind::Audio) { - track.track.cut(None)?; + let (bytes, rest) = chunk.split_at(frame.remaining().min(chunk.len())); + if !bytes.is_empty() { + frame.write(bytes)?; + } + chunk = rest; + if frame.remaining() > 0 { + break; + } + let frame = frames.open.take().expect("a frame is open"); + track.track.commit(frame)?; } + Ok(total - chunk.len()) + } + /// The streaming block ended: its frames are complete, or it was malformed. + fn block_end(&mut self) -> Result<()> { + let Some(block) = self.block.take() else { + return Ok(()); + }; + match block.state { + BlockState::Header(_) => { + return Err(match block.grouped { + Some(_) => Error::InvalidBlock, + None => Error::InvalidSimpleBlock, + } + .into()); + } + BlockState::Lacing(_) => return Err(anyhow::Error::new(LacingError::Invalid).into()), + BlockState::Frames(frames) => { + // The frame sizes add up to the body, so every frame was written as its bytes arrived. + debug_assert!(frames.open.is_none() && frames.index == frames.sizes.len()); + if frames.kind == TrackKind::Audio + && let Some(track) = self.tracks.get_mut(&frames.track) + { + track.track.cut(None)?; + } + } + BlockState::Skip => {} + } + if let Some(sequence) = self.seek.take() { + self.seek(sequence)?; + } Ok(()) } /// Close the current group on every track and open the next one at `sequence`. /// /// Broadcast-wide: every track inside this MKV import advances together; per-track - /// control is intentionally not exposed. Text spends a group per cue, so a text track that is - /// already past `sequence` continues from its own next group instead. + /// control is intentionally not exposed. While a block streams in, the seek waits for it to end, + /// since its later frames continue its group. pub fn seek(&mut self, sequence: u64) -> Result<()> { + if self.block.is_some() { + self.seek = Some(sequence); + return Ok(()); + } for track in self.tracks.values_mut() { - let sequence = match &track.track { - Media::Text(text) => sequence.max(text.next_sequence()), - _ => sequence, - }; track.track.seek(sequence)?; } Ok(()) } /// Finish all tracks, flushing current groups. + /// + /// Input that ends inside an element other than an unknown-size Segment or Cluster is refused as + /// truncated. pub fn finish(&mut self) -> Result<()> { self.finished = true; - while let Some(tag) = self - .decoder - .decode_eof(&mut self.buffer) - .map_err(|_| Error::MatroskaParse)? + if self.failed.is_none() + && let Err(err) = self.drain_eof() { - self.handle_tag(tag.tag)?; + // A frame cut short fails its group with the reason, rather than waiting for the rest. + self.abort_block(err.reason()); + return Err(err); } - // Finished sources can remain owned after EOF; release the last element's allocation. - self.buffer = BytesMut::new(); // No frame follows to anchor the clock, so publish the declared track set now. self.initial_reservation = None; for track in self.tracks.values_mut() { - if let Some(mut g) = track.group.take() { - g.finish()?; - } track.track.finish()?; } Ok(()) @@ -636,19 +858,31 @@ impl Import { /// Unlike a drop, this leaves the tracks' HLS records to the catalog, whose abort fails them /// rather than ending them as a complete recording. pub fn abort(mut self, err: moq_net::Error) { - for mut track in std::mem::take(&mut self.tracks).into_values() { - if let Some(g) = track.group.take() { - let _ = g.abort(err.clone()); - } + self.abort_block(err.clone()); + for track in std::mem::take(&mut self.tracks).into_values() { track.track.abort(err.clone()); } } + + /// Fail the frame a block was streaming into, which holds its track's group. + fn abort_block(&mut self, err: moq_net::Error) { + if let Some(Block { + state: BlockState::Frames(frames), + .. + }) = self.block.take() + && let Some(frame) = frames.open + { + frame.abort(err); + } + } } impl Drop for Import { /// End each track, then retire its HLS records: every track takes a fresh name, so nothing /// publishes it again. fn drop(&mut self) { + // A frame cut short fails its group before the track ends. + self.abort_block(moq_net::Error::Dropped); for mut track in std::mem::take(&mut self.tracks).into_values() { let name = track.track.name().to_string(); let ended = track.track.end(); @@ -661,6 +895,76 @@ impl Drop for Import { } } +/// Decode a field the importer reads into its Matroska element. +fn field(id: u32, data: Vec) -> Result { + let id = u64::from(id); + let invalid = |_| Error::MatroskaParse; + let value = match MatroskaSpec::get_tag_data_type(id) { + Some(TagDataType::UnsignedInt) => { + MatroskaSpec::get_unsigned_int_tag(id, ebml_iterable::tools::arr_to_u64(&data).map_err(invalid)?) + } + Some(TagDataType::Float) => MatroskaSpec::get_float_tag( + id, + match data.len() { + 0 => 0.0, + _ => ebml_iterable::tools::arr_to_f64(&data).map_err(invalid)?, + }, + ), + Some(TagDataType::Utf8) => { + MatroskaSpec::get_utf8_tag(id, String::from_utf8(data).map_err(|_| Error::MatroskaParse)?) + } + Some(TagDataType::Binary) => MatroskaSpec::get_binary_tag(id, &data), + _ => None, + }; + Ok(value.ok_or(Error::MatroskaParse)?) +} + +/// Record `value` in its master's fields, replacing an earlier value of the same field. +fn set(fields: &mut Vec, value: MatroskaSpec) -> Result<()> { + let id = value.get_id(); + match fields.iter_mut().find(|field| field.get_id() == id) { + // A track declares one number; a second would make its blocks ambiguous. + Some(_) if matches!(value, MatroskaSpec::TrackNumber(_)) => return Err(moq_net::Error::Duplicate.into()), + Some(field) => *field = value, + None => fields.push(value), + } + Ok(()) +} + +/// Parse a block header: the track number, the relative timestamp, and the flags. `None` until +/// `buf` holds all of it. +fn block_header(buf: &[u8], grouped: bool) -> Result)>> { + let invalid = || match grouped { + true => Error::InvalidBlock, + false => Error::InvalidSimpleBlock, + }; + let first = buf[0]; + if first == 0 { + return Err(invalid().into()); + } + let len = first.leading_zeros() as usize + 1; + let Some(&[high, low, flags]) = buf.get(len..len + 3) else { + return Ok(None); + }; + let mut track = u64::from(first) & (0xff >> len); + for byte in &buf[1..len] { + track = (track << 8) | u64::from(*byte); + } + let lace = match (flags >> 1) & 0b11 { + 0 => None, + 1 => Some(Lace::Xiph), + 2 => Some(Lace::FixedSize), + _ => Some(Lace::Ebml), + }; + let fixed = Fixed { + track, + timestamp: i16::from_be_bytes([high, low]), + // A Block's keyframe bit is reserved; its BlockGroup decides instead. + keyframe: flags & 0x80 != 0, + }; + Ok(Some((fixed, lace))) +} + fn build_video_config( codec_id: &str, codec_private: Option<&Bytes>, @@ -826,6 +1130,9 @@ fn matroska_spatial( Ok((projection, stereo)) } +/// The size of an `equi` box's body: version and flags, then four 32-bit bounds. +const EQUI_PRIVATE_LEN: u64 = 20; + /// The projection a Matroska `Projection` element signals, `None` for a rectangular one. fn matroska_projection(fields: &[MatroskaSpec]) -> Result> { let unsupported = Error::UnsupportedSpatial; @@ -1062,50 +1369,6 @@ fn build_av1_config(codec_private: Option<&Bytes>) -> Result { Ok(config) } -/// Render one Matroska subtitle block as a self-contained WebVTT document. -/// -/// `header` is the WebVTT CodecPrivate; SRT (`S_TEXT/UTF8`) tracks have none and get a bare header. -fn cue_payload( - header: Option<&str>, - addition: Option<&[u8]>, - body: &[u8], - start: Timestamp, - duration: Timestamp, -) -> Result { - let end = start.checked_add(duration)?; - let body = std::str::from_utf8(body).map_err(|_| Error::InvalidSubtitle("invalid UTF-8"))?; - let mut output = header.unwrap_or("WEBVTT\n\n").to_string(); - let mut id = ""; - let mut settings = ""; - if let Some(addition) = addition.filter(|_| header.is_some()) { - let addition = - std::str::from_utf8(addition).map_err(|_| Error::InvalidSubtitle("invalid cue additions UTF-8"))?; - let mut lines = addition.splitn(3, '\n'); - settings = lines.next().unwrap_or_default(); - id = lines.next().unwrap_or_default(); - if let Some(comments) = lines.next().filter(|value| !value.is_empty()) { - crate::container::text::comments(&mut output, comments).map_err(Error::InvalidSubtitle)?; - } - } - let body = match header { - Some(_) => crate::container::text::rebase(body, start..end, Timestamp::ZERO).map_err(Error::InvalidSubtitle)?, - // SRT is plain text, where a literal `-->` is valid but would end a WebVTT cue. - None => body.replace("-->", "-->"), - }; - crate::container::text::append( - &mut output, - crate::container::text::Cue { - start, - end, - body: &body, - id, - settings, - }, - ) - .map_err(Error::InvalidSubtitle)?; - Ok(Bytes::from(output)) -} - #[derive(Debug, thiserror::Error)] enum LacingError { #[error("invalid MKV lacing")] @@ -1114,77 +1377,75 @@ enum LacingError { MissingDuration, } -// webm-iterable's splitter uses unchecked indexing and size arithmetic on malformed laces. -// Keep bounds and signed EBML size deltas checked before slicing untrusted input. -fn split_lacing<'a>(block: &'a SimpleBlock<'_>) -> std::result::Result, LacingError> { - let Some(lacing) = block.lacing else { - return Ok(vec![block.raw_frame_data()]); - }; - let invalid = || LacingError::Invalid; - let mut data = block.raw_frame_data(); - let count = usize::from(*data.first().ok_or_else(invalid)?) + 1; - if count < 2 { - return Err(invalid()); - } - data = &data[1..]; - let mut sizes = Vec::with_capacity(count); - match lacing { - BlockLacing::FixedSize => { - if !data.len().is_multiple_of(count) { - return Err(invalid()); +impl Lacing { + /// Take the next lace header byte, returning every frame's size once the header ends. + /// `remaining` counts the body bytes after `byte`; the last frame takes what the others leave. + // webm-iterable's splitter uses unchecked indexing and size arithmetic on malformed laces, so + // sizes and signed EBML deltas stay checked here. + fn push(&mut self, byte: u8, remaining: u64) -> std::result::Result>, LacingError> { + let invalid = LacingError::Invalid; + let Some(count) = self.count else { + let count = usize::from(byte) + 1; + if count < 2 { + return Err(invalid); } - sizes.resize(count - 1, data.len() / count); - } - BlockLacing::Xiph => { - for _ in 0..count - 1 { - let mut size = 0usize; - loop { - let byte = *data.first().ok_or_else(invalid)?; - data = &data[1..]; - size = size.checked_add(usize::from(byte)).ok_or_else(invalid)?; - if byte != 255 { - break; - } + self.count = Some(count); + if self.lace != Lace::FixedSize { + return Ok(None); + } + let frames = count as u64; + if !remaining.is_multiple_of(frames) { + return Err(invalid); + } + return Ok(Some(vec![remaining / frames; count])); + }; + match self.lace { + Lace::Xiph => { + self.sum = self.sum.checked_add(byte.into()).ok_or(invalid)?; + if byte == 255 { + return Ok(None); } - sizes.push(size); + self.sizes.push(std::mem::take(&mut self.sum)); } - } - BlockLacing::Ebml => { - for index in 0..count - 1 { - let first = *data.first().ok_or_else(invalid)?; + Lace::Ebml => { + self.vint.push(byte); + let first = self.vint[0]; let len = first.leading_zeros() as usize + 1; - if len > 8 || data.len() < len { - return Err(invalid()); + if len > 8 { + return Err(invalid); + } + if self.vint.len() < len { + return Ok(None); } - let mut value = u64::from(first) & (0xffu64 >> len); - for byte in &data[1..len] { + let mut value = u64::from(first) & (0xff >> len); + for byte in &self.vint[1..] { value = (value << 8) | u64::from(*byte); } - data = &data[len..]; - let size = if index == 0 { - value as i64 - } else { - let delta = value as i64 - ((1i64 << (7 * len - 1)) - 1); - i64::try_from(sizes[index - 1]) - .map_err(|_| invalid())? - .checked_add(delta) - .ok_or_else(invalid)? + self.vint.clear(); + let size = match self.sizes.last() { + None => value, + Some(&previous) => { + let delta = value as i64 - ((1i64 << (7 * len - 1)) - 1); + previous.checked_add_signed(delta).ok_or(invalid)? + } }; - sizes.push(usize::try_from(size).map_err(|_| invalid())?); + self.sizes.push(size); } + Lace::FixedSize => unreachable!("fixed-size lacing has no sizes"), } - } - let mut frames = Vec::with_capacity(count); - for size in sizes { - if size > data.len() { - return Err(invalid()); + if self.sizes.len() < count - 1 { + return Ok(None); } - let (frame, remaining) = data.split_at(size); - frames.push(frame); - data = remaining; + let sum = self + .sizes + .iter() + .try_fold(0u64, |sum, size| sum.checked_add(*size)) + .ok_or(LacingError::Invalid)?; + let last = remaining.checked_sub(sum).ok_or(LacingError::Invalid)?; + let mut sizes = std::mem::take(&mut self.sizes); + sizes.push(last); + Ok(Some(sizes)) } - frames.push(data); - Ok(frames) } #[cfg(test)] @@ -1192,6 +1453,29 @@ mod tests { use super::*; use webm_iterable::{WebmWriter, WriteOptions}; + #[test] + fn ignored_payload_does_not_accumulate_under_a_large_metadata_master() { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut import = Import::new(broadcast, catalog.reserve()); + // A 1 GiB EBML header containing a nearly 1 GiB Void. Neither declaration is + // a reason to retain ignored bytes while waiting for the master to finish. + import.decode(&[0x1a, 0x45, 0xdf, 0xa3, 0x08, 0x40, 0, 0, 0]).unwrap(); + import.decode(&[0xec, 0x08, 0x3f, 0xff, 0xff, 0xfa]).unwrap(); + for _ in 0..64 { + import.decode(&[0; 1024]).unwrap(); + assert!( + import.reader.retained() <= 12, + "ignored bytes retained: {}", + import.reader.retained() + ); + } + assert!( + matches!(import.finish(), Err(crate::Error::Mkv(Error::Truncated))), + "truncated ignored payload must fail" + ); + } + #[test] fn completed_live_elements_do_not_accumulate_a_metadata_prefix() { let mut header = Vec::new(); @@ -1214,7 +1498,7 @@ mod tests { let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); let mut import = Import::new(broadcast, catalog.reserve()); import.decode(&header).unwrap(); - let mut consumed = header.len(); + let mut consumed = header.len() as u64; for timestamp in [0, u8::MAX as u64, u16::MAX as u64, u32::MAX as u64] { let mut cluster = Vec::new(); { @@ -1232,10 +1516,10 @@ mod tests { for byte in &cluster { import.decode(&[*byte]).unwrap(); consumed += 1; - assert_eq!(import.decoder.position() + import.buffer.len(), consumed); + assert_eq!(import.reader.position(), consumed); + // At most a split element header and the 8-byte timestamp being read. + assert!(import.reader.retained() <= 12 + 8, "{}", import.reader.retained()); } - assert!(import.buffer.is_empty(), "a complete cluster left its prefix buffered"); - assert_eq!(import.decoder.position(), consumed); assert_eq!(import.cluster_timestamp, timestamp); } import.finish().unwrap(); @@ -1246,9 +1530,8 @@ mod tests { )); } - #[test] - fn input_after_a_failed_finish_is_refused() { - // Unknown-size Tracks closes only at EOF, queueing Segment's end behind the refused track. + /// A WebM prefix declaring `entries`, leaving an unknown-size Segment and Cluster open. + fn live(entries: Vec) -> Vec { let mut data = Vec::new(); { let mut writer = WebmWriter::new(&mut data); @@ -1257,31 +1540,306 @@ mod tests { "webm".into(), )]))) .unwrap(); - for master in [ - MatroskaSpec::Segment(Master::Start), - MatroskaSpec::Tracks(Master::Start), - ] { - writer - .write_advanced(&master, WriteOptions::is_unknown_sized_element()) - .unwrap(); - } writer - .write(&MatroskaSpec::TrackEntry(Master::Full(vec![ - MatroskaSpec::TrackNumber(0), - MatroskaSpec::TrackType(2), - MatroskaSpec::CodecID("A_OPUS".into()), - ]))) + .write_advanced( + &MatroskaSpec::Segment(Master::Start), + WriteOptions::is_unknown_sized_element(), + ) .unwrap(); + writer.write(&MatroskaSpec::Tracks(Master::Full(entries))).unwrap(); writer.flush().unwrap(); } + // An unknown-size Cluster at timestamp 0. + data.extend_from_slice(&[0x1f, 0x43, 0xb6, 0x75, 0xff, 0xe7, 0x81, 0x00]); + data + } + + fn entry(number: u64, track_type: u64, codec: &str) -> MatroskaSpec { + MatroskaSpec::TrackEntry(Master::Full(vec![ + MatroskaSpec::TrackNumber(number), + MatroskaSpec::TrackType(track_type), + MatroskaSpec::CodecID(codec.into()), + ])) + } + + /// An element header declaring `size` body bytes. + fn header(id: &[u8], size: u64) -> Vec { + let mut header = id.to_vec(); + header.push(0x01); + header.extend_from_slice(&size.to_be_bytes()[1..]); + header + } + + /// A SimpleBlock header for a keyframe on `track` declaring `size` payload bytes. + fn simple_block(track: u8, size: u64) -> Vec { + let mut block = header(&[0xa3], size + 4); + block.extend_from_slice(&[0x80 | track, 0, 0, 0x80]); + block + } + + fn import() -> (Import, crate::catalog::Producer, moq_net::broadcast::Consumer) { let mut broadcast = moq_net::broadcast::Info::new().produce(); + let consumer = broadcast.consume(); let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut import = Import::new(broadcast, catalog.reserve()); - import.decode(&data).unwrap(); + let import = Import::new(broadcast, catalog.reserve()); + (import, catalog, consumer) + } + + async fn first_frame( + catalog: &crate::catalog::Producer, + consumer: &moq_net::broadcast::Consumer, + ) -> moq_net::frame::Consumer { + let name = catalog.snapshot().video.renditions.keys().next().unwrap().clone(); + let mut track = consumer.track(&name).unwrap().subscribe(None).await.unwrap(); + let mut group = track.recv_group().await.unwrap().unwrap(); + group.next_frame().await.unwrap().unwrap() + } + + const WAIT: std::time::Duration = std::time::Duration::from_secs(5); + + #[tokio::test] + async fn a_block_streams_into_its_frame_before_it_completes() { + const SIZE: usize = 1024 * 1024; + let (mut import, catalog, consumer) = import(); + import.decode(&live(vec![entry(1, 1, "V_VP9")])).unwrap(); + import.decode(&simple_block(1, SIZE as u64)).unwrap(); + import.decode(&[1; 1024]).unwrap(); + + let mut frame = first_frame(&catalog, &consumer).await; + // The Legacy timestamp prefix and the first chunk are readable while the rest is in flight. + let mut read = Vec::new(); + while read.len() < 1 + 1024 { + let chunk = tokio::time::timeout(WAIT, frame.read_chunk()).await.unwrap().unwrap(); + read.extend_from_slice(&chunk.expect("the frame is still open")); + } + assert_eq!(read, [&[0][..], &[1; 1024]].concat()); + + for _ in 1..SIZE / 1024 { + import.decode(&[1; 1024]).unwrap(); + assert!(import.reader.retained() <= 12, "{}", import.reader.retained()); + } + while let Some(chunk) = tokio::time::timeout(WAIT, frame.read_chunk()).await.unwrap().unwrap() { + read.extend_from_slice(&chunk); + } + assert_eq!(read.len(), 1 + SIZE); + import.finish().unwrap(); + } + + #[test] + fn a_block_past_the_frame_limit_is_refused_before_its_payload() { + let (mut import, _catalog, _consumer) = import(); + import.decode(&live(vec![entry(1, 1, "V_VP9")])).unwrap(); + let err = import.decode(&simple_block(1, 1 << 40)).unwrap_err(); + assert!( + matches!(err, crate::Error::Moq(moq_net::Error::FrameTooLarge)), + "{err:?}" + ); + assert!(import.reader.retained() <= 12); + // The rest of the refused chunk was never read, so later input cannot continue it. assert!(matches!( - import.finish(), - Err(crate::Error::Mkv(Error::ValueOutOfRange("TrackNumber"))) + import.decode(&[0]), + Err(crate::Error::Moq(moq_net::Error::FrameTooLarge)) )); + import.finish().unwrap(); + } + + #[test] + fn a_track_entry_publishes_before_its_tracks_master_ends() { + let (mut import, _catalog, _consumer) = import(); + let mut data = Vec::new(); + { + let mut writer = WebmWriter::new(&mut data); + writer + .write(&MatroskaSpec::Ebml(Master::Full(vec![MatroskaSpec::DocType( + "webm".into(), + )]))) + .unwrap(); + writer + .write_advanced( + &MatroskaSpec::Segment(Master::Start), + WriteOptions::is_unknown_sized_element(), + ) + .unwrap(); + writer.flush().unwrap(); + } + // Tracks declares 1 GiB, of which the entry is a few bytes and a Void claims the rest. + data.extend(header(&[0x16, 0x54, 0xae, 0x6b], 1 << 30)); + // TrackEntry: TrackNumber 1, TrackType 1 (video), CodecID V_VP9. + data.extend_from_slice(&[0xae, 0x8d, 0xd7, 0x81, 0x01, 0x83, 0x81, 0x01, 0x86, 0x85]); + data.extend_from_slice(b"V_VP9"); + import.decode(&data).unwrap(); + assert_eq!(import.tracks.len(), 1); + assert!(import.fields.is_empty(), "the entry's fields were released"); + + import.decode(&header(&[0xec], (1 << 30) - 64)).unwrap(); + for _ in 0..64 { + import.decode(&[0; 1024]).unwrap(); + assert!(import.reader.retained() <= 12, "{}", import.reader.retained()); + } + assert!(matches!(import.finish(), Err(crate::Error::Mkv(Error::Truncated)))); + } + + /// A WebM prefix with an unknown-size Segment, then a Tracks holding one VP9 TrackEntry whose + /// last child is `field`'s header, so `size` body bytes of it remain. + fn entry_ending_in(field: &[u8], size: u64) -> Vec { + let mut data = Vec::new(); + { + let mut writer = WebmWriter::new(&mut data); + writer + .write(&MatroskaSpec::Ebml(Master::Full(vec![MatroskaSpec::DocType( + "webm".into(), + )]))) + .unwrap(); + writer + .write_advanced( + &MatroskaSpec::Segment(Master::Start), + WriteOptions::is_unknown_sized_element(), + ) + .unwrap(); + writer.flush().unwrap(); + } + // TrackNumber 1, TrackType 1 (video), CodecID V_VP9. + let mut entry = vec![0xd7, 0x81, 0x01, 0x83, 0x81, 0x01, 0x86, 0x85]; + entry.extend_from_slice(b"V_VP9"); + entry.extend_from_slice(field); + let entry_size = entry.len() as u64 + size; + let mut tracks = header(&[0xae], entry_size); + tracks.extend(entry); + data.extend(header(&[0x16, 0x54, 0xae, 0x6b], tracks.len() as u64 + size)); + data.extend(tracks); + data + } + + /// Feed `size` body bytes in 1 KiB chunks, checking none are held, and return the last + /// chunk's result. + fn feed_unretained(import: &mut Import, size: u64) -> crate::Result<()> { + for _ in 1..size / 1024 { + import.decode(&[b'a'; 1024]).unwrap(); + assert!(import.reader.retained() <= 12, "{}", import.reader.retained()); + } + import.decode(&[b'a'; 1024]) + } + + #[test] + fn unused_track_fields_are_discarded_as_they_arrive() { + const SIZE: u64 = 1024 * 1024; + // Name, Language, and LanguageIETF have no effect on output. + for id in [&[0x53, 0x6e][..], &[0x22, 0xb5, 0x9c], &[0x22, 0xb5, 0x9d]] { + let (mut import, _catalog, _consumer) = import(); + import.decode(&entry_ending_in(&header(id, SIZE), SIZE)).unwrap(); + feed_unretained(&mut import, SIZE).unwrap(); + assert_eq!(import.tracks.len(), 1, "{id:x?}"); + import.finish().unwrap(); + } + } + + #[test] + fn unsupported_projection_private_data_is_not_held() { + const SIZE: u64 = 1024 * 1024; + for projection_type in [0, 1] { + // Video: PixelWidth 320, PixelHeight 240, then Projection: ProjectionType, then the + // ProjectionPrivate header. + let mut projection = vec![0x76, 0x71, 0x81, projection_type]; + projection.extend(header(&[0x76, 0x72], SIZE)); + let mut video = vec![0xb0, 0x82, 0x01, 0x40, 0xba, 0x81, 0xf0]; + video.extend(header(&[0x76, 0x70], projection.len() as u64 + SIZE)); + video.extend(projection); + let mut field = header(&[0xe0], video.len() as u64 + SIZE); + field.extend(video); + + let (mut import, _catalog, _consumer) = import(); + import.decode(&entry_ending_in(&field, SIZE)).unwrap(); + let result = feed_unretained(&mut import, SIZE); + match projection_type { + // A rectangular projection ignores its private data. + 0 => { + result.unwrap(); + assert_eq!(import.tracks.len(), 1); + } + // An equirectangular one is refused, as any size but an `equi` body is. + _ => assert!( + matches!(result, Err(crate::Error::Mkv(Error::UnsupportedSpatial(_)))), + "{result:?}" + ), + } + } + } + + #[tokio::test] + async fn skipped_and_unknown_track_payloads_are_discarded_as_they_arrive() { + let (mut import, catalog, consumer) = import(); + // Track 2 is a role the importer skips; track 9 is never declared. + import + .decode(&live(vec![entry(1, 1, "V_VP9"), entry(2, 3, "B_VOBBTN")])) + .unwrap(); + assert_eq!(import.tracks.len(), 1); + for track in [2, 9] { + import.decode(&simple_block(track, 1024 * 1024)).unwrap(); + for _ in 0..1024 { + import.decode(&[2; 1024]).unwrap(); + assert!(import.reader.retained() <= 12); + } + } + import.decode(&simple_block(1, 3)).unwrap(); + import.decode(b"key").unwrap(); + let mut frame = first_frame(&catalog, &consumer).await; + assert_eq!(frame.read_all().await.unwrap(), b"\0key"[..]); + import.finish().unwrap(); + } + + /// Output nobody reads is evicted by its track, so the importer keeps accepting input. + #[test] + fn unread_output_does_not_stall_the_importer() { + let (mut import, _catalog, _consumer) = import(); + import.decode(&live(vec![entry(1, 1, "V_VP9")])).unwrap(); + let payload = vec![3; 1024 * 1024]; + // Twice the most a track holds. + for _ in 0..64 { + import.decode(&simple_block(1, payload.len() as u64)).unwrap(); + import.decode(&payload).unwrap(); + } + import.finish().unwrap(); + } + + #[tokio::test] + async fn truncation_fails_the_frame_cut_short() { + let (mut import, catalog, consumer) = import(); + import.decode(&live(vec![entry(1, 1, "V_VP9")])).unwrap(); + import.decode(&simple_block(1, 1024)).unwrap(); + import.decode(&[4; 512]).unwrap(); + let mut frame = first_frame(&catalog, &consumer).await; + assert!(matches!(import.finish(), Err(crate::Error::Mkv(Error::Truncated)))); + // The importer is still owned, and the frame fails rather than waiting on it. + let read = tokio::time::timeout(WAIT, frame.read_all()).await.unwrap(); + assert!(read.is_err(), "a truncated frame must not read as complete: {read:?}"); + drop(import); + } + + #[tokio::test] + async fn an_abort_fails_the_frame_cut_short() { + for drop in [false, true] { + let (mut import, catalog, consumer) = import(); + import.decode(&live(vec![entry(1, 1, "V_VP9")])).unwrap(); + import.decode(&simple_block(1, 1024)).unwrap(); + import.decode(&[4; 512]).unwrap(); + let mut frame = first_frame(&catalog, &consumer).await; + match drop { + true => std::mem::drop(import), + false => import.abort(moq_net::Error::Cancel), + } + let read = tokio::time::timeout(WAIT, frame.read_all()).await.unwrap(); + assert!(read.is_err(), "a frame cut short must not read as complete: {read:?}"); + } + } + + #[test] + fn input_after_a_failed_finish_is_refused() { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut import = Import::new(broadcast, catalog.reserve()); + // An EBML header declaring a byte it never sends. + import.decode(&[0x1a, 0x45, 0xdf, 0xa3, 0x81]).unwrap(); + assert!(matches!(import.finish(), Err(crate::Error::Mkv(Error::Truncated)))); assert!(matches!( import.decode(&[0]), Err(crate::Error::Mkv(Error::TracksAfterFinish)) diff --git a/rs/moq-mux/src/container/mkv/import_test.rs b/rs/moq-mux/src/container/mkv/import_test.rs index b7a98080573..a4e6b754e6b 100644 --- a/rs/moq-mux/src/container/mkv/import_test.rs +++ b/rs/moq-mux/src/container/mkv/import_test.rs @@ -1200,226 +1200,6 @@ fn text_entry(number: u64, codec: &str, language: &str, name: &str) -> MatroskaS ])) } -#[tokio::test] -async fn subtitle_languages_and_empty_tracks_are_published() { - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![ - text_entry(1, "S_TEXT/UTF8", "en", "English"), - text_entry(2, "S_TEXT/UTF8", "es", "Spanish"), - text_entry(3, "S_TEXT/UTF8", "fr", "French"), - ]) - .cluster(0, || { - let MatroskaSpec::SimpleBlock(block) = simple_block(1, 250, true, b"Hello") else { - unreachable!() - }; - vec![MatroskaSpec::BlockGroup(Master::Full(vec![ - MatroskaSpec::Block(block), - MatroskaSpec::BlockDuration(1250), - ]))] - }) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - importer.decode(&data).unwrap(); - let snapshot = catalog.snapshot(); - assert_eq!(snapshot.text.renditions.len(), 3); - let (name, english) = snapshot - .text - .renditions - .iter() - .find(|(_, config)| config.lang.as_deref() == Some("en")) - .unwrap(); - assert_eq!(english.label.as_deref(), Some("English")); - assert_eq!(english.format, hang::catalog::TextFormat::Vtt); - let track = consumer - .track(name) - .unwrap() - .subscribe(moq_net::track::Subscription::default()) - .await - .unwrap(); - let mut reader = crate::container::Consumer::new( - track, - crate::catalog::hang::Container::Legacy(crate::container::Kind::Data), - ); - let frame = reader.read_frame().await.unwrap().unwrap(); - assert_eq!( - &frame.payload[..], - b"WEBVTT\n\n00:00:00.250 --> 00:00:01.500\nHello\n\n" - ); - assert_eq!(frame.timestamp.as_micros(), 250_000); - importer.finish().unwrap(); -} - -/// An SRT cue keeps its explicit end, and a plain-text `-->` is escaped for WebVTT. -#[tokio::test(start_paused = true)] -async fn subtitle_duration_survives_as_a_timed_cue() { - let MatroskaSpec::SimpleBlock(block) = simple_block(1, 250, true, b"Go --> there") else { - unreachable!() - }; - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![text_entry(1, "S_TEXT/UTF8", "en", "English")]) - .cluster(0, || { - vec![MatroskaSpec::BlockGroup(Master::Full(vec![ - MatroskaSpec::Block(block), - MatroskaSpec::BlockDuration(1250), - ]))] - }) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - importer.decode(&data).unwrap(); - importer.finish().unwrap(); - let snapshot = catalog.snapshot(); - let (name, config) = snapshot.text.renditions.iter().next().unwrap(); - assert_eq!(config.format, hang::catalog::TextFormat::Vtt); - let track = consumer - .track(name) - .unwrap() - .subscribe(moq_net::track::Subscription::default()) - .await - .unwrap(); - let mut reader = crate::container::Consumer::new( - track, - crate::catalog::hang::Container::Legacy(crate::container::Kind::Data), - ); - let frame = reader.read_frame().await.unwrap().unwrap(); - assert_eq!(frame.timestamp.as_micros(), 250_000); - assert_eq!( - &frame.payload[..], - b"WEBVTT\n\n00:00:00.250 --> 00:00:01.500\nGo --> there\n\n" - ); -} - -#[tokio::test(start_paused = true)] -async fn webvtt_preserves_cue_metadata_and_relative_timestamps() { - let MatroskaSpec::TrackEntry(Master::Full(mut fields)) = text_entry(1, "S_TEXT/WEBVTT", "en", "English") else { - unreachable!() - }; - fields.push(MatroskaSpec::CodecPrivate( - b"WEBVTT\n\nSTYLE\n::cue { color: lime; }\n".to_vec(), - )); - fields.push(MatroskaSpec::LanguageIETF("en-GB".into())); - let MatroskaSpec::SimpleBlock(block) = simple_block(1, 250, true, b"Hello <00:00.500>world") else { - unreachable!() - }; - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![MatroskaSpec::TrackEntry(Master::Full(fields))]) - .cluster(0, || { - vec![MatroskaSpec::BlockGroup(Master::Full(vec![ - MatroskaSpec::Block(block), - MatroskaSpec::BlockDuration(1250), - MatroskaSpec::BlockAdditions(Master::Full(vec![MatroskaSpec::BlockMore(Master::Full(vec![ - MatroskaSpec::BlockAddID(1), - MatroskaSpec::BlockAdditional(b"align:start\ncue-1\nNOTE source".to_vec()), - ]))])), - ]))] - }) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - importer.decode(&data).unwrap(); - importer.finish().unwrap(); - let snapshot = catalog.snapshot(); - let (name, config) = snapshot.text.renditions.iter().next().unwrap(); - assert_eq!(config.lang.as_deref(), Some("en-GB")); - let track = consumer - .track(name) - .unwrap() - .subscribe(moq_net::track::Subscription::default()) - .await - .unwrap(); - let mut reader = crate::container::Consumer::new( - track, - crate::catalog::hang::Container::Legacy(crate::container::Kind::Data), - ); - let frame = reader.read_frame().await.unwrap().unwrap(); - assert_eq!(&frame.payload[..], b"WEBVTT\n\nSTYLE\n::cue { color: lime; }\n\nNOTE source\n\ncue-1\n00:00:00.250 --> 00:00:01.500 align:start\nHello <00:00:00.750>world\n\n"); -} - -#[test] -fn text_requires_valid_utf8_and_explicit_or_default_duration() { - for (default_duration, payload, valid) in [ - (None, b"text".as_slice(), false), - (Some(0), b"text".as_slice(), false), - (Some(1_000_000_000), b"text".as_slice(), true), - (Some(1_000_000_000), b"\xff".as_slice(), false), - ] { - let MatroskaSpec::TrackEntry(Master::Full(mut fields)) = text_entry(1, "S_TEXT/UTF8", "en", "English") else { - unreachable!() - }; - if let Some(duration) = default_duration { - fields.push(MatroskaSpec::DefaultDuration(duration)); - } - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![MatroskaSpec::TrackEntry(Master::Full(fields))]) - .cluster(0, || vec![simple_block(1, 0, true, payload)]) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - let result = importer.decode(&data); - if valid { - result.unwrap(); - } else { - assert!( - matches!(result, Err(crate::Error::Mkv(super::Error::InvalidSubtitle(_)))), - "{result:?}" - ); - } - } -} - -#[test] -fn laced_subtitle_blocks_are_refused() { - let MatroskaSpec::TrackEntry(Master::Full(mut fields)) = text_entry(1, "S_TEXT/UTF8", "en", "English") else { - unreachable!() - }; - fields.push(MatroskaSpec::DefaultDuration(1_000_000_000)); - let MatroskaSpec::SimpleBlock(mut block) = simple_block(1, 0, true, b"\x01ab") else { - unreachable!() - }; - block[3] |= 0x04; // Fixed-size lacing, two one-byte frames. - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![MatroskaSpec::TrackEntry(Master::Full(fields))]) - .cluster(0, || vec![MatroskaSpec::SimpleBlock(block)]) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - assert!(matches!( - importer.decode(&data), - Err(crate::Error::Mkv(super::Error::InvalidSubtitle( - "laced subtitles are unsupported" - ))) - )); -} - /// Opus packets packed like mkvmerge's laced audio blocks retain their boundaries and times. #[tokio::test] async fn laced_opus_frames() { @@ -1558,10 +1338,12 @@ async fn unlaced_block_duration_does_not_use_track_default() { let MatroskaSpec::SimpleBlock(raw) = simple_block(1, timestamp, true, b"frame") else { unreachable!() }; - let mut children = vec![MatroskaSpec::Block(raw), MatroskaSpec::BlockDuration(40)]; + // A video Block streams once a ReferenceBlock precedes it or it ends its group. + let mut children = vec![MatroskaSpec::BlockDuration(40)]; if timestamp != 0 { children.push(MatroskaSpec::ReferenceBlock(-40)); } + children.push(MatroskaSpec::Block(raw)); MatroskaSpec::BlockGroup(Master::Full(children)) }) .collect() @@ -1587,80 +1369,6 @@ async fn unlaced_block_duration_does_not_use_track_default() { assert_eq!(last, Some((80_000, 0))); } -/// Text closes a group per cue, so a segment seek must not rewind onto a cue group already written. -#[test] -fn subtitle_seek_after_several_cues_does_not_reuse_a_group() { - let MatroskaSpec::TrackEntry(Master::Full(mut fields)) = text_entry(1, "S_TEXT/UTF8", "en", "English") else { - unreachable!() - }; - fields.push(MatroskaSpec::DefaultDuration(500_000_000)); - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![MatroskaSpec::TrackEntry(Master::Full(fields))]) - .cluster(0, || { - (0..3).map(|index| simple_block(1, index * 1000, true, b"hi")).collect() - }) - .cluster(4000, || vec![simple_block(1, 0, true, b"hi")]) - .segment_end() - .build(); - // Split just before each Cluster's ID. - const CLUSTER: [u8; 4] = [0x1f, 0x43, 0xb6, 0x75]; - let clusters: Vec = data - .windows(4) - .enumerate() - .filter(|(_, window)| *window == CLUSTER) - .map(|(index, _)| index) - .collect(); - let (head, rest) = data.split_at(clusters[0]); - let (first, second) = rest.split_at(clusters[1] - clusters[0]); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - importer.decode(head).unwrap(); - importer.seek(100).unwrap(); - importer.decode(first).unwrap(); - // The next segment is 101, but the cues already spent groups past it. - importer.seek(101).unwrap(); - importer.decode(second).unwrap(); - importer.finish().unwrap(); -} - -/// A subtitle BlockGroup that replaces the codec initialization is refused, not silently ignored. -#[test] -fn subtitle_codec_state_is_refused() { - let MatroskaSpec::TrackEntry(Master::Full(mut fields)) = text_entry(1, "S_TEXT/WEBVTT", "en", "English") else { - unreachable!() - }; - fields.push(MatroskaSpec::CodecPrivate(b"WEBVTT\n".to_vec())); - let MatroskaSpec::SimpleBlock(block) = simple_block(1, 0, true, b"Hello") else { - unreachable!() - }; - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(vec![MatroskaSpec::TrackEntry(Master::Full(fields))]) - .cluster(0, || { - vec![MatroskaSpec::BlockGroup(Master::Full(vec![ - MatroskaSpec::Block(block), - MatroskaSpec::BlockDuration(1000), - MatroskaSpec::CodecState(b"WEBVTT\n\nSTYLE\n::cue { color: red; }\n".to_vec()), - ]))] - }) - .segment_end() - .build(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); - let mut importer = super::Import::new(broadcast, catalog.reserve()); - let result = importer.decode(&data); - assert!( - matches!(result, Err(crate::Error::Mkv(super::Error::InvalidSubtitle(_)))), - "{result:?}" - ); -} - /// The SPS outranks the Colour element, which still fills what the SPS leaves unsaid: here the /// static HDR metadata. #[test] @@ -1898,27 +1606,35 @@ fn block_tick_overflow_is_refused_before_timestamp_conversion() { } } +/// A subtitle cue's WebVTT length depends on its content, so no subtitle track is advertised. #[test] -fn unsupported_subtitle_codec_is_refused() { - for reverse in [false, true] { - let mut tracks = vec![ - track_entry_audio_opus(1, 48_000.0, 2), - text_entry(2, "S_TEXT/ASS", "de", "Unsupported"), - ]; - if reverse { - tracks.reverse(); - } - let data = MkvBuilder::new() - .header("matroska") - .segment_start() - .info(1_000_000) - .tracks(tracks) - .segment_end() - .build(); - for initial in [false, true] { - assert!( - matches!(import_result(&data, initial), Err(crate::Error::UnknownFormat(codec)) if codec.contains("S_TEXT/ASS")) - ); +fn subtitle_tracks_are_refused_before_their_rendition() { + for codec in ["S_TEXT/UTF8", "S_TEXT/WEBVTT", "S_TEXT/ASS"] { + for reverse in [false, true] { + let mut tracks = vec![ + track_entry_audio_opus(1, 48_000.0, 2), + text_entry(2, codec, "de", "German"), + ]; + if reverse { + tracks.reverse(); + } + let data = MkvBuilder::new() + .header("matroska") + .segment_start() + .info(1_000_000) + .tracks(tracks) + .segment_end() + .build(); + for initial in [false, true] { + assert!( + matches!(import_result(&data, initial), Err(crate::Error::Mkv(super::Error::SubtitleTrack(id))) if id == codec) + ); + } + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut importer = super::Import::new(broadcast, catalog.reserve()); + assert!(importer.decode(&data).is_err()); + assert!(catalog.snapshot().text.renditions.is_empty()); } } } @@ -1973,7 +1689,7 @@ fn partial_track_metadata_is_refused_at_eof() { assert!( matches!( import_result(&data[..data.len() - missing], initial), - Err(crate::Error::Mkv(super::Error::MatroskaParse)) + Err(crate::Error::Mkv(super::Error::Truncated)) ), "missing {missing}, initial {initial}" ); @@ -2053,6 +1769,193 @@ async fn real_live_webm_imports_across_chunk_boundaries() { } } +/// A BlockGroup holding `children` around a Block for `track` with `payload`. +fn block_group( + track: u64, + timestamp: i16, + payload: &[u8], + before: Vec, + after: Vec, +) -> MatroskaSpec { + let MatroskaSpec::SimpleBlock(raw) = simple_block(track, timestamp, false, payload) else { + unreachable!() + }; + let mut children = before; + children.push(MatroskaSpec::Block(raw)); + children.extend(after); + MatroskaSpec::BlockGroup(Master::Full(children)) +} + +/// A video Block streams when what decides its group precedes it: a ReferenceBlock, or its group's +/// end. Any child after it is refused before the Block publishes, since a ReferenceBlock could follow. +#[tokio::test(start_paused = true)] +async fn video_block_groups_stream_only_when_their_group_is_known() { + use MatroskaSpec::{BlockDuration, ReferenceBlock}; + let accepted = || { + vec![ + block_group(1, 0, b"key", vec![], vec![]), + block_group(1, 33, b"delta", vec![ReferenceBlock(-33)], vec![BlockDuration(33)]), + block_group(1, 66, b"key", vec![BlockDuration(33)], vec![]), + ] + }; + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 16, 16) + .cluster(0, accepted) + .segment_end() + .build(); + for chunk in [1, 5, data.len()] { + let imported = import_chunked(&data, chunk).await; + assert!(imported.refused.is_none(), "{:?}", imported.refused); + assert_eq!(imported.published.values().next().unwrap(), &[0, 33_000, 66_000]); + } + + for after in [vec![BlockDuration(33)], vec![ReferenceBlock(-33)]] { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 16, 16) + .cluster(0, || { + vec![ + simple_block(1, 0, true, b"key"), + block_group(1, 33, b"refused", vec![], after.clone()), + ] + }) + .segment_end() + .build(); + for chunk in [1, 5, data.len()] { + let imported = import_chunked(&data, chunk).await; + assert!( + matches!( + imported.refused, + Some(crate::Error::Mkv(super::Error::TrailingReference)) + ), + "{:?}", + imported.refused + ); + assert_eq!(imported.published.values().next().unwrap(), &[0]); + } + } +} + +/// Audio ignores a BlockGroup's trailing fields, so its Block streams whatever follows it. +#[tokio::test(start_paused = true)] +async fn audio_block_groups_stream_with_trailing_fields() { + use MatroskaSpec::{BlockDuration, ReferenceBlock}; + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .tracks(vec![track_entry_audio_opus(1, 48_000.0, 2)]) + .cluster(0, || { + vec![ + block_group(1, 0, b"\xf8\xff\xfe", vec![], vec![BlockDuration(20)]), + block_group(1, 20, b"\xf8\xff\xfe", vec![], vec![ReferenceBlock(-20)]), + ] + }) + .segment_end() + .build(); + let imported = import_chunked(&data, 3).await; + assert!(imported.refused.is_none(), "{:?}", imported.refused); + assert_eq!(imported.published.values().next().unwrap(), &[0, 20_000]); +} + +/// A block header's variable-length track number may split anywhere. +#[tokio::test(start_paused = true)] +async fn multibyte_track_numbers_split_anywhere() { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(300, "V_VP9", 16, 16) + .cluster(0, || { + vec![simple_block(300, 0, true, b"k0"), simple_block(300, 33, false, b"p1")] + }) + .segment_end() + .build(); + for chunk in 1..=9 { + let imported = import_chunked(&data, chunk).await; + assert!(imported.refused.is_none(), "{:?}", imported.refused); + assert_eq!( + imported.published.values().next().unwrap(), + &[0, 33_000], + "chunk {chunk}" + ); + } +} + +/// A CMAF `moof` describes a payload before it, so MKV import refuses CMAF before any rendition. +#[test] +fn cmaf_output_is_refused_before_its_rendition() { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 16, 16) + .segment_end() + .build(); + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut importer = super::Import::new(broadcast, catalog.reserve()).with_container(Container::Cmaf { + init: bytes::Bytes::from_static(&[0; 32]), + }); + assert!(matches!( + importer.decode(&data), + Err(crate::Error::Mkv(super::Error::StreamedCmaf)) + )); + assert!(catalog.snapshot().video.renditions.is_empty()); +} + +/// A seek while a block streams waits for it to end, so its frames stay in one group. +#[tokio::test(start_paused = true)] +async fn a_seek_mid_block_waits_for_the_block() { + use webm_iterable::matroska_spec::{BlockLacing, Frame}; + let mut entry = track_entry_audio_opus(1, 48_000.0, 2); + let MatroskaSpec::TrackEntry(Master::Full(children)) = &mut entry else { + unreachable!() + }; + children.push(MatroskaSpec::DefaultDuration(20_000_000)); + let packets: Vec<&[u8]> = vec![b"\xf8\xff\xfe", b"\xf8\xff\xfd"]; + let mut laced = SimpleBlock::new_uncheked(&[], 1, 0, false, Some(BlockLacing::Xiph), false, true); + laced.set_frame_data(&packets.iter().map(|data| Frame { data }).collect()); + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .tracks(vec![entry]) + .cluster(0, || vec![laced.into(), simple_block(1, 40, true, b"\xf8\xff\xfc")]) + .segment_end() + .build(); + // Split inside the laced block's second frame. + let split = data.windows(3).position(|window| window == b"\xf8\xff\xfd").unwrap() + 1; + + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let consumer = broadcast.consume(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut importer = super::Import::new(broadcast, catalog.reserve()); + importer.decode(&data[..split]).unwrap(); + importer.seek(100).unwrap(); + importer.decode(&data[split..]).unwrap(); + importer.finish().unwrap(); + + let snapshot = catalog.snapshot(); + let name = snapshot.audio.renditions.keys().next().unwrap(); + let replay = moq_net::track::Subscription::default().with_max_delay(std::time::Duration::from_secs(3600)); + let mut track = consumer.track(name).unwrap().subscribe(replay).await.unwrap(); + let mut groups = Vec::new(); + while let Some(mut group) = track.recv_group().await.unwrap() { + let mut frames = 0; + while group.read_frame().await.unwrap().is_some() { + frames += 1; + } + groups.push((group.sequence, frames)); + } + assert_eq!(groups, [(0, 2), (100, 1)]); +} + /// A WebM with one VP9 track whose `Video` element holds `children` beside its dimensions. fn spatial_webm(children: Vec) -> Vec { use MatroskaSpec::*; diff --git a/rs/moq-mux/src/container/mkv/mod.rs b/rs/moq-mux/src/container/mkv/mod.rs index 1657222c78f..c52bf03e9a3 100644 --- a/rs/moq-mux/src/container/mkv/mod.rs +++ b/rs/moq-mux/src/container/mkv/mod.rs @@ -4,6 +4,7 @@ //! format only, not as a wire format: [`Import`] parses MKV byte streams //! into a broadcast and [`Export`] does the reverse. +mod ebml; mod export; mod import; @@ -27,9 +28,23 @@ pub enum Error { #[error("invalid display aspect: {0}")] InvalidAspect(&'static str), - /// A subtitle cue cannot be represented with valid text and timing. - #[error("invalid subtitle: {0}")] - InvalidSubtitle(&'static str), + /// A subtitle track cannot be imported in one pass: a WebVTT cue's length depends on its + /// content, so it cannot stream into a frame of a declared size. + #[error("MKV subtitle track {0} is unsupported: its cues cannot stream")] + SubtitleTrack(String), + + /// A video Block neither follows a ReferenceBlock nor ends its BlockGroup, so whether it starts + /// a group would wait on fields after its payload. + #[error("video BlockGroup with fields after its Block is unsupported: a later ReferenceBlock decides its group")] + TrailingReference, + + /// CMAF output needs a `moof` describing each payload before it, but MKV import streams payloads. + #[error("MKV import cannot write CMAF; select Legacy or LOC")] + StreamedCmaf, + + /// The input ended inside an element. + #[error("truncated MKV element")] + Truncated, /// The EBML header declares an unsupported DocType. #[error("unsupported EBML DocType: {0}")] diff --git a/rs/moq-mux/src/container/producer.rs b/rs/moq-mux/src/container/producer.rs index 2550db388fa..153dc38ace2 100644 --- a/rs/moq-mux/src/container/producer.rs +++ b/rs/moq-mux/src/container/producer.rs @@ -101,6 +101,83 @@ pub struct Producer { sent: Option>, } +/// A frame whose payload streams into its track as it arrives, opened by [`Producer::open`]. +/// +/// It holds the track's open group until [`Producer::commit`] returns it, so nothing else writes to +/// the track meanwhile. Dropping it unfinished aborts that group. +pub(crate) struct Streamed { + writer: moq_net::frame::Producer, + timestamp: moq_net::Timestamp, + duration: Option, + keyframe: bool, + size: usize, +} + +impl Streamed { + /// Append payload bytes, readable by subscribers at once. + pub fn write(&mut self, chunk: &[u8]) -> crate::Result<()> { + self.writer.write(chunk)?; + Ok(()) + } + + /// Payload bytes still to write. + pub fn remaining(&self) -> usize { + self.writer.remaining() + } + + /// Abort the frame and its group with `err`. + pub fn abort(self, err: moq_net::Error) { + let _ = self.writer.abort(err); + } +} + +impl Producer { + /// Open a frame of `size` payload bytes, written through the returned [`Streamed`] as they + /// arrive, for a source that learns a frame's size before its bytes. + /// + /// Groups and refuses frames the way [`write`](Self::write) does; the frame is measured once + /// [`commit`](Self::commit)ted. The group's storage takes the bytes as they are written, so a + /// declared size commits no memory, and one past the group's limit is refused up front. A + /// buffering producer, or CMAF, whose `moof` describes the payload, cannot stream. + pub(crate) fn open( + &mut self, + timestamp: moq_net::Timestamp, + keyframe: bool, + duration: Option, + size: usize, + ) -> crate::Result { + let prefix = match self.container.prefix(timestamp)? { + Some(prefix) if self.buffer_duration.is_zero() => prefix, + _ => return Err(crate::Error::UnsupportedContainer("a streamed frame".into())), + }; + self.prepare(timestamp, keyframe)?; + let group = self.group.take().expect("prepare opens a group"); + let total = prefix.len().checked_add(size).ok_or(moq_net::Error::FrameTooLarge)?; + let mut writer = group.create_frame(moq_net::frame::Info { + timestamp: Some(timestamp), + size: total as u64, + })?; + writer.write(prefix)?; + Ok(Streamed { + writer, + timestamp, + duration, + keyframe, + size, + }) + } + + /// Finish a frame [`open`](Self::open) returned once its payload is written, handing its group + /// back to the track. + pub(crate) fn commit(&mut self, frame: Streamed) -> crate::Result<()> { + let group = frame.writer.finish()?; + let sequence = group.sequence; + self.group = Some(group); + self.written(sequence, frame.timestamp, frame.duration, frame.size, frame.keyframe); + Ok(()) + } +} + trait Rendition: Send { fn name(&self) -> &str; fn timestamp(&self, hint: Option) -> crate::Result; @@ -498,48 +575,7 @@ where /// broadcast. Frames may still dip below the previous group's content: B-frames, open-GOP /// leading pictures, and a keyframe that overlaps the previous group's last frame. pub fn write(&mut self, frame: Frame) -> crate::Result<()> { - let floor = match frame.keyframe { - true => self.start, - false => self.floor, - }; - if let Some(floor) = floor - && timestamp_lt(frame.timestamp, floor) - { - return Err(super::TimestampRewind { - timestamp: frame.timestamp, - floor, - } - .into()); - } - - // A keyframe cuts the previous group, using its timestamp as the boundary where the - // previous group's content ends, unless it overlaps that content. - if frame.keyframe { - let overlaps = self - .previous_timestamp - .is_some_and(|previous| timestamp_lt(frame.timestamp, previous)); - self.cut((!overlaps).then_some(frame.timestamp))?; - if overlaps { - self.cadence = None; - } - } - - // Start a new group if needed; the first frame of a group must be a keyframe. - if self.group.is_none() { - if !frame.keyframe { - // No group yet and this delta can't anchor one. The caller (e.g. a - // mid-stream join) decides whether to skip until the first keyframe. - return Err(super::MissingKeyframe.into()); - } - let group = match self.pending_sequence.take() { - Some(sequence) => self.inner.create_group(moq_net::group::Info { sequence })?, - None => self.inner.append_group()?, - }; - - self.floor = self.start; - self.start = Some(frame.timestamp); - self.group = Some(group); - } + self.prepare(frame.timestamp, frame.keyframe)?; // Buffer or write the frame. if self.buffer_duration.is_zero() { @@ -547,18 +583,10 @@ where let (timestamp, duration, bytes) = (frame.timestamp, frame.duration, frame.payload.len()); let (sequence, keyframe) = (group.sequence, frame.keyframe); self.container.write(group, &[frame])?; - // The HLS records absorb publish failures themselves (they are an optional sidecar), so - // reporting can't abort the media write. - if let Some(recorder) = self.recorder.as_mut() { - recorder.frame(sequence, timestamp); - } - // Only what the container accepted is measured. A rejected frame (too large for the // group, a timestamp that won't convert) leaves the producer usable, and the estimate's // extrema never fall, so counting one would inflate the catalog for good. - self.estimator.write(timestamp, bytes); - self.count(bytes, keyframe); - self.observe_end(timestamp, duration); + self.written(sequence, timestamp, duration, bytes, keyframe); } else { // Buffered frames count toward the rates on the way in instead. The flush that eventually // writes them takes the track down with it when it fails, so there is nothing to unwind. @@ -585,6 +613,69 @@ where Ok(()) } + /// Check `timestamp` against the group starts, and open a group for a keyframe. + fn prepare(&mut self, timestamp: moq_net::Timestamp, keyframe: bool) -> crate::Result<()> { + let floor = match keyframe { + true => self.start, + false => self.floor, + }; + if let Some(floor) = floor + && timestamp_lt(timestamp, floor) + { + return Err(super::TimestampRewind { timestamp, floor }.into()); + } + + // A keyframe cuts the previous group, using its timestamp as the boundary where the + // previous group's content ends, unless it overlaps that content. + if keyframe { + let overlaps = self + .previous_timestamp + .is_some_and(|previous| timestamp_lt(timestamp, previous)); + self.cut((!overlaps).then_some(timestamp))?; + if overlaps { + self.cadence = None; + } + } + + // Start a new group if needed; the first frame of a group must be a keyframe. + if self.group.is_none() { + if !keyframe { + // No group yet and this delta can't anchor one. The caller (e.g. a + // mid-stream join) decides whether to skip until the first keyframe. + return Err(super::MissingKeyframe.into()); + } + let group = match self.pending_sequence.take() { + Some(sequence) => self.inner.create_group(moq_net::group::Info { sequence })?, + None => self.inner.append_group()?, + }; + + self.floor = self.start; + self.start = Some(timestamp); + self.group = Some(group); + } + + Ok(()) + } + + /// Measure and report a frame the group accepted unbuffered. + fn written( + &mut self, + sequence: u64, + timestamp: moq_net::Timestamp, + duration: Option, + bytes: usize, + keyframe: bool, + ) { + // The HLS records absorb publish failures themselves (they are an optional sidecar), so + // reporting can't abort the media write. + if let Some(recorder) = self.recorder.as_mut() { + recorder.frame(sequence, timestamp); + } + self.estimator.write(timestamp, bytes); + self.count(bytes, keyframe); + self.observe_end(timestamp, duration); + } + /// Cut the current group, flushing buffered frames and closing it. /// /// `end` bounds the final buffered frame when the publisher knows where the