Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/lib/rs/moq-mux.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ back. This is what `moq import` / `export` and the gateways are built on.

| Format | Import | Export | Notes |
| --- | --- | --- | --- |
| fMP4 / CMAF | yes | yes | Audio/video unwrapped to `legacy` by default, or passed through as `cmaf` with `with_passthrough`; `wvtt` subtitles import as timed WebVTT text and `stpp` TTML (IMSC timing profile) as `ttml` documents on the broadcast clock, preserving source language and label when present; other subtitle codecs are skipped. Import streams media samples as they arrive; each track's samples must be in forward byte order, and a `moov`, `moof`, or text sample over 32 MiB is refused. |
| fMP4 / CMAF | yes | yes | Audio/video unwrapped to `legacy` by default, or passed through as `cmaf` with `with_passthrough`; `wvtt` subtitles import as timed WebVTT text and `stpp` TTML (IMSC timing profile) as `ttml` documents on the broadcast clock, preserving source language and label when present; other subtitle codecs are skipped. Import streams media samples as they arrive; each track's samples must be in forward byte order, and a `moov` or `moof` over 2 MiB, a text sample over 32 MiB, a `moof` of more than 131,072 samples, or a `moov` of more than 256 tracks is refused. |
| MPEG-TS | yes | yes | H.264/H.265; AAC, MP2, AC-3, E-AC-3, Opus up to 7.1; SCTE-35 sections and every other undecoded PES stream (KLV, subtitles, DVB AC-3) as tracks; service tables round-trip. |
| FLV / RTMP | yes | yes | Legacy H.264 + AAC + MP3, plus enhanced-RTMP HEVC, AV1, VP9, Opus, AC-3, E-AC-3, and multitrack. |
| Matroska / WebM | yes | yes | Import streams each block into its frame in one pass, retaining no declared size. It refuses subtitle tracks, CMAF output, video `BlockGroup`s with fields after their `Block`, a `CodecPrivate` before its `CodecID`, and decoder configs carrying extra metadata (any FLAC metadata block after STREAMINFO, OpusHead extension fields, av1C OBUs beyond the sequence header and HDR metadata, AAC AudioSpecificConfig bytes past its last field or with a non-AAC object type's config): remux video as `SimpleBlock`s, extract subtitles separately, and remux FLAC so its `CodecPrivate` holds only STREAMINFO. Stripping tags is not enough, since seek tables and padding are refused too. ffmpeg's `-c:a copy` keeps only STREAMINFO, except for channel layouts FLAC does not define (such as 5.1 with back surrounds, ffmpeg's `-ac 6` default, or 4.0): there ffmpeg adds a Vorbis comment holding the channel mask, so those are refused. An hvcC keeps only its parameter sets, dropping SEI such as x265's encoder settings. |
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,6 @@ per-frame CPU growth since #5926.
- [Pre-cut audit findings](/quest/m1/cut-audit/README.md) - fix the crashes, hostile-input costs, and correctness bugs the 2026-10-24 audit of main found
- [Vendored code has every upstream fix it needs](/quest/m1/vendor-sync/README.md) - quinn, libv4l-rs, nvidia-video-codec-sdk, rust-media-libs, cros-codecs, and UniFFI generator fixes are ported or planned, once per release cycle
- [Media fuzz findings](/quest/m1/fuzz-findings/README.md) - the nightly media fuzz targets run clean: mp4-atom, MKV block, TS step, SPS overflow, and catalog encoder bugs they found are fixed
- [fMP4 sample plans, trun counts, and trak counts are bounded](/quest/m1/fmp4-metadata-limits.md) - a `moof` or `moov` within the importer's bounds cannot plan hundreds of megabytes or take quadratic time; 116 bytes cost 584 MB today
- [Mesh dials recover from a stuck flow](/quest/m1/mesh-fresh-port.md) - a relay redials a failing cluster peer from a fresh UDP port, so one dropped 5-tuple can't strand a link
- [Cluster routing](/quest/m1/cluster-routing/README.md) - prove shared routing contracts, implement per-origin routes, and connect outbound local clusters
- [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation setup
Expand Down
66 changes: 0 additions & 66 deletions quest/m1/fmp4-metadata-limits.md

This file was deleted.

10 changes: 3 additions & 7 deletions quest/m2/import-metadata-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,14 @@ The fMP4 and MKV importers follow one documented policy for container metadata t

The two importers settled this separately during the 2026-10-27 security audit:

- fMP4 ([moq-dev/moq#7232](https://github.com/moq-dev/moq/pull/7232), merged): what must be held whole, a `moov` or `moof` body or one text sample, is refused at its header when it declares more than `fmp4::MAX_HELD_BYTES` (32 MiB, the transport's per-group `MAX_CACHE_BYTES`), and a `moof` declaring more than `fmp4::MAX_FRAGMENT_SAMPLES` (2^21) is refused when it parses. That reversed the 2026-10-28 incremental plan, because incremental parsing could not bound what it kept.
- fMP4 ([moq-dev/moq#7232](https://github.com/moq-dev/moq/pull/7232), merged): what must be held whole is refused at its header, which reversed the 2026-10-28 incremental plan, because incremental parsing could not bound what it kept. [moq-dev/moq#7687](https://github.com/moq-dev/moq/pull/7687) then measured that mp4-atom expands a parsed body up to tenfold, so a `moov` or `moof` body is bounded by `fmp4::MAX_METADATA_BYTES` (2 MiB) and a text sample by `fmp4::MAX_HELD_BYTES` (32 MiB, the transport's per-group `MAX_CACHE_BYTES`). A `moof` declaring more than `fmp4::MAX_FRAGMENT_SAMPLES` (2^17) is refused when it parses, and a `moov` declaring more than `fmp4::MAX_TRACKS` (256) before any track is created.
- MKV ([moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234), merged): metadata parses incrementally with no blanket size cap (maintainer, 2026-10-28). Each collected master keeps one value per field, so repeats replace rather than accumulate. [moq-dev/moq#7629](https://github.com/moq-dev/moq/pull/7629) bounded the fields read whole: `DocType` and `CodecID` by the longest supported value, and `CodecPrivate` by each codec's supported configuration, refused at its header (FLAC STREAMINFO alone, the RFC 7845 OpusHead, av1C with a sequence header and HDR metadata, avcC by its count fields at about 34 MiB, and an AudioSpecificConfig by its syntax maximum). An hvcC is canonicalized as it arrives instead: its parameter sets are kept, at most one per id, and SEI is discarded unread. These are private constants in `rs/moq-mux/src/container/mkv/import.rs`, `codec::h264::AVCC_MAX_LEN`, and `codec::aac::CONFIG_MAX_LEN`.

Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2. Pick one policy for both importers, rather than leaving each at the decision its own audit PR made. Starting point: a byte bound at the header for whatever must be held whole, since #7232 found that is what actually bounds memory, while MKV keeps streaming what it can consume incrementally. Bring the choice to the maintainer in the PR if the evidence points elsewhere.

- #7232 has landed the fMP4 side (`MAX_HELD_BYTES` and `MAX_FRAGMENT_SAMPLES` in `rs/moq-mux/src/container/fmp4/mod.rs`). Note the MKV avcC bound (about 34 MiB) sits above fMP4's 32 MiB `MAX_HELD_BYTES`.
- Inventory what each importer still holds whole, and how much parsed state it builds from it (#7232 noted a 32 MiB `moof` decodes into larger structures).
- The fMP4 side has landed (`MAX_HELD_BYTES`, `MAX_METADATA_BYTES`, `MAX_FRAGMENT_SAMPLES`, and `MAX_TRACKS` in `rs/moq-mux/src/container/fmp4/mod.rs`). Note the MKV avcC bound (about 34 MiB) sits far above fMP4's 2 MiB `MAX_METADATA_BYTES`.
- Inventory what each importer still holds whole, and how much parsed state it builds from it (fMP4's parse expands up to tenfold, which is what sets `MAX_METADATA_BYTES`).
- Keep audio and video payloads unbounded in both: streaming each sample into its frame was decided 2026-10-27, and this quest is about metadata only. An fMP4 text sample is the one payload held whole, because its rewrite needs every byte, so `MAX_HELD_BYTES` covers it; keep that exception. MKV has no counterpart, since it refuses subtitle tracks.
- Document the policy once, in moq-mux's import docs, and make both importers' bounds `pub const`s as #7232's are.

Public API: possibly new `pub const` bounds and a shared refusal error in moq-mux. Wire: none.

## Related

- [fMP4 sample plans, trun counts, and trak counts are bounded](/quest/m1/fmp4-metadata-limits.md) - measured what fMP4's bounds let a `moof` or `moov` build, and sets the bounds this policy documents
2 changes: 2 additions & 0 deletions rs/moq-mux/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- fMP4 import bounds what a `moov` or `moof` costs to parse and plan: a body over `fmp4::MAX_METADATA_BYTES` (2 MiB) is refused at its header, since parsing expands it up to tenfold, a `moof` of more than `fmp4::MAX_FRAGMENT_SAMPLES` (now 131,072) samples is refused before planning, and a `moov` of more than `fmp4::MAX_TRACKS` (256) tracks with `Error::TooManyTracks` before any is created. `fmp4::MAX_HELD_BYTES` (32 MiB) now bounds only a text sample. A 116-byte `moof` no longer plans 584 MB.
- Declaring a catalog rendition costs the same however many the catalog already holds: it is checked on its own rather than by cloning and serializing the whole catalog, which made N tracks cost N².
- `container::Consumer` no longer plays a backlog that landed before its first read from the oldest group: until the first item, the start moves past each group whose first frame trails the newest content by more than the max delay plus the warmup, as `@moq/hang`'s consumer does, keeping the group covering the live edge less `Delay::min` and the warmup. A join starts at the same group in Rust and the browser. Once reading begins, only a blocking group is skipped, as before.
- fMP4 import applies each track's edit list to its frame timestamps and drops the samples before the edit, such as AAC priming, along with the video samples decoding from a dropped sync sample, and a passthrough init no longer carries the `edts`. A consumer that offsets AAC priming itself must stop, or it shifts the track twice. An edit list it can't reduce to one shift (several media edits, or a rate other than 1) is refused with `Error::UnsupportedEditList`.

Expand Down
12 changes: 12 additions & 0 deletions rs/moq-mux/src/allocation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ pub struct Usage {
pub freed: usize,
/// Calls to allocate or reallocate.
pub allocations: usize,
/// The most bytes held at once beyond what the thread held when the call began.
pub peak: usize,
}

impl Usage {
/// Bytes allocated and not yet freed; freeing what the call did not allocate lowers it.
fn live(&self) -> usize {
self.allocated.saturating_sub(self.freed)
}
}

thread_local! {
Expand All @@ -41,6 +50,7 @@ unsafe impl GlobalAlloc for Counting {
record(|usage| {
usage.allocated += layout.size();
usage.allocations += 1;
usage.peak = usage.peak.max(usage.live());
});
unsafe { System.alloc(layout) }
}
Expand All @@ -55,6 +65,8 @@ unsafe impl GlobalAlloc for Counting {
usage.allocated += new_size;
usage.freed += layout.size();
usage.allocations += 1;
// Both blocks are live while a realloc copies between them.
usage.peak = usage.peak.max(usage.live() + layout.size());
});
unsafe { System.realloc(ptr, layout, new_size) }
}
Expand Down
77 changes: 66 additions & 11 deletions rs/moq-mux/src/catalog/tracks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,23 @@ fn quantize(published: Option<Duration>, measured: Option<Duration>, period: Dur
}
}

/// Write `config` under `name`, refusing it before any change if it redefines the rendition's
/// identity or won't serialize, so the reserved snapshot never retains a bad edit.
///
/// The entry is checked in a catalog holding only it and the extension: the media and data sections
/// serialize each entry on its own, and the rest was checked when it was written, so cloning or
/// serializing them would make declaring N renditions cost N². The extension is kept whole because
/// its serializer may depend on its combined entries.
fn write<E: CatalogExt, C: RenditionConfig<E>>(catalog: &mut Catalog<E>, name: &str, config: C) -> crate::Result<()> {
let mut alone = Catalog::default();
alone.ext = catalog.ext.clone();
config.clone().insert(&mut alone, name);
check_identity(catalog, &alone, name)?;
serde_json::to_writer(std::io::sink(), &alone).map_err(moq_json::Error::from)?;
config.insert(catalog, name);
Ok(())
}

/// Refuse an edit that redefines rendition `name`'s track identity in place: a publisher whose
/// decoder identity changed publishes under a new name instead.
fn check_identity<E: CatalogExt>(previous: &Catalog<E>, next: &Catalog<E>, name: &str) -> crate::Result<()> {
Expand Down Expand Up @@ -669,13 +686,8 @@ impl<E: CatalogExt, C: RenditionConfig<E>> Rendition<E, C> {
config.set_estimate(resolved.clone());
{
let mut guard = self.catalog.modify()?;
let mut next = (*guard).clone();
config.insert(&mut next, &self.name);
check_identity(&guard, &next, &self.name)?;
// Serialization must succeed before the reserved snapshot retains the edit.
serde_json::to_writer(std::io::sink(), &next).map_err(moq_json::Error::from)?;
write(&mut guard, &self.name, config)?;
self.present = true;
*guard = next;
guard.list::<C>(&self.name);
if C::detects() {
guard.hint::<C>(&self.name, supplied.clone());
Expand Down Expand Up @@ -816,11 +828,7 @@ impl<E: CatalogExt, C: RenditionConfig<E>> Rendition<E, C> {
}
Self::check_decrease(&self.config()?.estimate(), &config.estimate())?;
let mut guard = self.catalog.modify()?;
let mut next = (*guard).clone();
config.insert(&mut next, &self.name);
check_identity(&guard, &next, &self.name)?;
serde_json::to_writer(std::io::sink(), &next).map_err(moq_json::Error::from)?;
*guard = next;
write(&mut guard, &self.name, config)?;
Ok(guard)
}
}
Expand Down Expand Up @@ -1465,6 +1473,17 @@ mod tests {
exploding: BTreeMap<String, Exploding>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
stubborn: BTreeMap<String, Stubborn>,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty", serialize_with = "at_most_one")]
solo: BTreeMap<String, Solo>,
}

/// Serialize a section that refuses to hold more than one entry, so whether it serializes
/// depends on every entry, not just the one being written.
fn at_most_one<S: serde::Serializer>(solo: &BTreeMap<String, Solo>, serializer: S) -> Result<S::Ok, S::Error> {
if solo.len() > 1 {
return Err(serde::ser::Error::custom("only one solo entry"));
}
solo.serialize(serializer)
}

impl CatalogExt for TelemetryExt {}
Expand Down Expand Up @@ -1621,6 +1640,42 @@ mod tests {
.expect("a stranded entry would refuse this forever");
}

/// A config whose section serializes only while it holds a single entry.
#[derive(Serialize, Deserialize, Clone, Default, Debug, PartialEq)]
struct Solo;

impl RenditionConfig<TelemetryExt> for Solo {
fn insert(self, catalog: &mut Catalog<TelemetryExt>, name: &str) {
catalog.ext.solo.insert(name.to_string(), self);
}
fn get_mut<'a>(catalog: &'a mut Catalog<TelemetryExt>, name: &str) -> Option<&'a mut Self> {
catalog.ext.solo.get_mut(name)
}
fn remove(catalog: &mut Catalog<TelemetryExt>, name: &str) {
catalog.ext.solo.remove(name);
}
}

/// An entry that serializes alone but not beside the extension's other entries is refused
/// before the catalog retains it.
#[test]
fn an_entry_that_breaks_the_extension_is_refused() {
let (_broadcast, catalog) = produce();
let reserved = catalog.reserve();
let mut first = reserved.init::<Solo>("a").unwrap();
let mut second = reserved.init::<Solo>("b").unwrap();

first.set(Solo).unwrap();
assert!(
matches!(second.set(Solo), Err(crate::Error::Json(_))),
"the second entry makes the extension unserializable"
);

let snapshot = catalog.snapshot();
assert_eq!(snapshot.ext.solo.keys().collect::<Vec<_>>(), ["a"]);
serde_json::to_string(&snapshot).expect("the refused edit left the catalog serializable");
}

/// A custom kind gets the same detection and drop-removal as video/audio.
#[test]
fn detects_and_advertises() {
Expand Down
Loading
Loading