diff --git a/doc/lib/rs/moq-mux.md b/doc/lib/rs/moq-mux.md index a3fd8d42e88..4d9f6b115f5 100644 --- a/doc/lib/rs/moq-mux.md +++ b/doc/lib/rs/moq-mux.md @@ -14,7 +14,7 @@ Turns existing container formats into hang broadcasts and back. This is what | Format | Import | Export | Notes | | --- | --- | --- | --- | | fMP4 / CMAF | yes | yes | Passthrough as `cmaf` or repackaged as `legacy`. | -| MPEG-TS | yes | yes | H.264/H.265; AAC, MP2, AC-3, E-AC-3; SCTE-35 and subtitle PIDs carried as tracks; service tables round-trip; signalled timebase discontinuities preserved; paced export. | +| MPEG-TS | yes | yes | H.264/H.265; AAC, MP2, AC-3, E-AC-3, Opus up to 7.1; SCTE-35 and subtitle PIDs carried as tracks; service tables round-trip; signalled timebase discontinuities preserved; paced export. | | FLV / RTMP | yes | yes | Legacy H.264 + AAC + MP3, plus enhanced-RTMP HEVC, AV1, VP9, Opus, AC-3, E-AC-3, and multitrack. | | Matroska / WebM | yes | yes | | | Annex-B (H.264, H.265) | yes | yes | Parameter sets extracted to the catalog or re-injected per keyframe. | diff --git a/quest/m1/audio-codecs/README.md b/quest/m1/audio-codecs/README.md index 762769dc5ae..6dc3b3e0d16 100644 --- a/quest/m1/audio-codecs/README.md +++ b/quest/m1/audio-codecs/README.md @@ -45,10 +45,12 @@ its own decode and encode quest so verification stays per host. - [AudioToolbox decode](/quest/m1/audio-codecs/decode-audiotoolbox.md) - macOS and iOS decode HE-AAC, multichannel AAC, and what else the framework offers - [HE-AAC catalog output](/quest/m1/audio-codecs/he-aac-catalog-output.md) - HE-AAC catalog entries name the output rate and layout, not the LC core +- [TS Opus export refusals](/quest/m1/audio-codecs/ts-opus-export-refusals.md) - the TS exporter refuses Opus heads its channel code cannot describe instead of mislabeling them +- [TS Opus channel codes](/quest/m1/audio-codecs/ts-opus-channel-codes.md) - TS Opus with a channel code of 0x81 or above imports with its real head or is refused, never guessed as stereo - [AudioToolbox encode](/quest/m1/audio-codecs/encode-audiotoolbox.md) - macOS and iOS encode AAC-LC -- [TS surround Opus](/quest/m1/audio-codecs/ts-opus-surround.md) - a 3 to 8 channel Opus stream from MPEG-TS imports with a family 1 OpusHead that decodes - [AAC encode refusals](/quest/m1/audio-codecs/aac-encode-refusals.md) - `Config::encode` refuses channel counts it cannot name instead of writing stereo - [AAC parse truncated SBR](/quest/m1/audio-codecs/aac-parse-truncated-sbr.md) - `Config::parse` refuses SBR and PS configs cut off before their core +- [GStreamer surround Opus](/quest/m1/audio-codecs/gst-opus-surround.md) - the moq-gst sink publishes 3 to 8 channel Opus with the OpusHead its caps describe ## Related diff --git a/quest/m1/audio-codecs/gst-opus-surround.md b/quest/m1/audio-codecs/gst-opus-surround.md new file mode 100644 index 00000000000..8b7a195ebcf --- /dev/null +++ b/quest/m1/audio-codecs/gst-opus-surround.md @@ -0,0 +1,21 @@ +# [S] Surround Opus from GStreamer + +## Goal + +`moq-gst`'s sink publishes a 3 to 8 channel `audio/x-opus` stream with the +OpusHead its caps describe, so surround Opus from `opusenc` plays natively like +the same stream from Matroska or MPEG-TS. + +## Plan + +- The sink refuses more than two channels because it builds the head from + `channels` and `rate` alone. Multichannel Opus caps also carry + `channel-mapping-family`, `stream-count`, `coupled-count`, and + `channel-mapping`; build the mapping with `opus::Mapping::new`, which + validates them, and refuse caps that omit or contradict them. +- Prefer a `streamheader` OpusHead when the caps carry one, if that path is + simpler and covers the same streams. +- Regression: an `opusenc` 5.1 pipeline publishes a family 1 description that + `moq-audio` decodes to six channels. + +Public API: none. Wire: none. diff --git a/quest/m1/audio-codecs/ts-opus-channel-codes.md b/quest/m1/audio-codecs/ts-opus-channel-codes.md new file mode 100644 index 00000000000..2fbce343d24 --- /dev/null +++ b/quest/m1/audio-codecs/ts-opus-channel-codes.md @@ -0,0 +1,27 @@ +# [S] TS Opus import refuses or parses extended channel codes + +## Goal + +An MPEG-TS Opus stream whose extension descriptor carries a +`channel_config_code` of 0x81 or above imports with its real OpusHead or is +refused. It never decodes as a guessed stereo stream, which misreads every +multistream packet. + +## Plan + +- The importer falls back to stereo for codes it does not know. 0x81 is + specified as an explicitly coded layout (channel count, mapping family, stream + counts, and table in the descriptor), and ffmpeg's muxer also writes + `0x80 | channels` for an alternate table its own demuxer does not read. Check + the Opus-in-TS spec and ffmpeg before deciding which codes to parse. +- Build parsed layouts with the public `opus::Mapping::new`, which validates + the table; refuse any code that is reserved or does not parse, per stream, + without failing the rest of the program. +- Fixtures from a real muxer where one exists; a hand-built descriptor + otherwise, noted as such. + +Public API: none. Wire: none. + +## Related + +- [TS Opus export refusals](/quest/m1/audio-codecs/ts-opus-export-refusals.md) - the exporter side of the same descriptor diff --git a/quest/m1/audio-codecs/ts-opus-export-refusals.md b/quest/m1/audio-codecs/ts-opus-export-refusals.md new file mode 100644 index 00000000000..c501cb032f5 --- /dev/null +++ b/quest/m1/audio-codecs/ts-opus-export-refusals.md @@ -0,0 +1,23 @@ +# [S] TS Opus export refuses what it cannot label + +## Goal + +`moq-mux`'s MPEG-TS exporter never labels an Opus track with a channel +configuration its packets do not have. A track whose OpusHead the Opus +extension descriptor can describe exports as today; anything else is refused, +not written with a guessed `channel_config_code`. + +## Plan + +- The exporter derives `channel_config_code` from the catalog channel count + alone, clamped to 1..=8, so a family 255 or ambisonics head, a family 1 head + with a non-Vorbis table, or more than eight channels exports mislabeled. + Decide from the parsed description: family 0, and family 1 with the Vorbis + default mapping, keep the plain code; refuse the rest, or write the explicit + layout if the import side of + [TS Opus channel codes](/quest/m1/audio-codecs/ts-opus-channel-codes.md) settles + one both sides agree on. +- A track with no description stays mono or stereo only. +- Tests with real heads, checked against ffprobe where ffmpeg reads the result. + +Public API: none. Wire: none; TS output changes only for inputs it mislabeled. diff --git a/quest/m1/audio-codecs/ts-opus-surround.md b/quest/m1/audio-codecs/ts-opus-surround.md deleted file mode 100644 index 5e6e6be4fc3..00000000000 --- a/quest/m1/audio-codecs/ts-opus-surround.md +++ /dev/null @@ -1,20 +0,0 @@ -# [S] Surround Opus from MPEG-TS - -## Goal - -An MPEG-TS Opus stream with 3 to 8 channels imports with an OpusHead that -decodes, so it plays natively like the same stream from Matroska or FLV. - -## Plan - -- The TS importer knows only the `channel_config_code` from the Opus - extension descriptor and builds `opus::Config::new(48_000, channels)`, whose - `encode` refuses more than two channels. The catalog then carries no - description and `moq-audio` refuses the track. -- Codes 3 to 8 mean family 1 with the Vorbis default stream and coupled counts - and mapping (the Vorbis orders of RFC 7845 §5.1.1.2), so the - importer can synthesize the head. That needs a way to build a family 1 - `Config` and have `encode` emit its table; decide whether that is a - constructor on `opus::Mapping` or a TS-local head writer. -- Regression: a 5.1 TS fixture imports with a family 1 description and decodes - to six channels. diff --git a/rs/moq-audio/tests/import.rs b/rs/moq-audio/tests/import.rs new file mode 100644 index 00000000000..010c1976c4e --- /dev/null +++ b/rs/moq-audio/tests/import.rs @@ -0,0 +1,64 @@ +//! Container imports that must decode end to end: a stream `moq-mux` imports +//! plays through [`decode::Consumer`] like the same stream from any container. + +use std::time::Duration; + +use moq_audio::{Format, Layout, decode}; + +/// MPEG-TS names surround Opus only by channel count, so the importer has to +/// synthesize the OpusHead mapping table or the decoder refuses the track. +#[tokio::test] +async fn ts_surround_opus_decodes() { + // A 440 Hz center channel in 5.1; see `import_opus_surround_catalog` in moq-mux. + let data = include_bytes!("../../moq-mux/src/container/ts/test_data/opus_5_1.ts"); + + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let broadcast_consumer = broadcast.consume(); + let catalog = moq_mux::catalog::Producer::new(&mut broadcast, moq_mux::catalog::Config::default()).unwrap(); + let mut import = moq_mux::container::ts::Import::new(broadcast, catalog.reserve()); + import.decode(&data[..]).unwrap(); + import.finish().unwrap(); + + let snapshot = catalog.snapshot(); + let (name, config) = snapshot.audio.renditions.iter().next().expect("an Opus track"); + + let mut options = decode::Options::default(); + options.output.format = Format::F32; + // The whole file is imported before anything decodes it. + options.max_age = Duration::from_secs(30); + let mut consumer = decode::Consumer::new(&broadcast_consumer, config, name, options) + .await + .unwrap(); + assert_eq!(consumer.layout(), Layout::FivePointOne); + + // Canonical 5.1 order: FL, FR, FC, LFE, SL, SR. + let mut energy = [0.0f64; 6]; + let mut frames = 0; + while let Some(frame) = tokio::time::timeout(Duration::from_secs(5), consumer.read()) + .await + .expect("decoded frame timed out") + .unwrap() + { + let (samples, rest) = frame.data.as_chunks::<4>(); + assert!(rest.is_empty()); + for chunk in samples.as_chunks::<6>().0 { + for (channel, sample) in energy.iter_mut().zip(chunk) { + *channel += f32::from_le_bytes(*sample).powi(2) as f64; + } + frames += 1; + } + } + + assert!(frames > 12_000, "expected most of 0.5 s at 48 kHz, got {frames} frames"); + let center = energy[2]; + assert!( + center / frames as f64 > 0.001, + "center should carry the tone: {energy:?}" + ); + for (channel, &other) in energy.iter().enumerate().filter(|&(channel, _)| channel != 2) { + assert!( + other < center / 100.0, + "channel {channel} should be near silent: {energy:?}" + ); + } +} diff --git a/rs/moq-mux/src/codec/opus/import.rs b/rs/moq-mux/src/codec/opus/import.rs index 69b4631c86f..829bef3736c 100644 --- a/rs/moq-mux/src/codec/opus/import.rs +++ b/rs/moq-mux/src/codec/opus/import.rs @@ -115,7 +115,7 @@ impl Import { pub fn config(init: &[u8]) -> crate::Result { let mut buf = init; let mut config: hang::catalog::AudioConfig = Config::parse(&mut buf)?.into(); - // Publish the head as given: re-encoding it would drop a channel mapping table. + // Publish the head as given: re-encoding it would drop the output gain. config.description = Some(bytes::Bytes::copy_from_slice(init)); Ok(config) } diff --git a/rs/moq-mux/src/codec/opus/mapping.rs b/rs/moq-mux/src/codec/opus/mapping.rs new file mode 100644 index 00000000000..4157dd5303f --- /dev/null +++ b/rs/moq-mux/src/codec/opus/mapping.rs @@ -0,0 +1,179 @@ +//! The OpusHead channel mapping table (RFC 7845 §5.1.1). + +use bytes::Buf; + +use super::{Error, Result}; + +/// An OpusHead channel mapping table (RFC 7845 §5.1.1), present for every family but 0. +#[derive(Clone, Copy, PartialEq, Eq)] +pub struct Mapping { + family: u8, + streams: u8, + coupled: u8, + channels: u8, + // Sized for the most channels any family allows, so `opus::Config` stays `Copy`. + table: [u8; 255], +} + +/// The fields of a channel mapping table, checked into a [`Mapping`] by [`Mapping::new`]. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Config<'a> { + /// The channel mapping family; 0 has no table and is refused. + pub family: u8, + /// Number of Opus streams in each packet. + pub streams: u8, + /// How many of those streams are coupled (stereo); they come first. + pub coupled: u8, + /// The decoded channel feeding each output channel, 255 for silence. + pub table: &'a [u8], +} + +/// The family 1 stream counts and table for 1 to 8 channels, as libopus and +/// RFC 7845 §5.1.1.2 lay out the Vorbis channel orders. +const VORBIS: [(u8, u8, &[u8]); 8] = [ + (1, 0, &[0]), + (1, 1, &[0, 1]), + (2, 1, &[0, 2, 1]), + (2, 2, &[0, 1, 2, 3]), + (3, 2, &[0, 4, 1, 2, 3]), + (4, 2, &[0, 4, 1, 2, 3, 5]), + (4, 3, &[0, 4, 1, 2, 3, 5, 6]), + (5, 3, &[0, 6, 1, 2, 3, 4, 5, 7]), +]; + +impl Mapping { + /// The family 1 mapping a surround encoder uses for `channels` in Vorbis + /// order, for sources that name only a channel count. + pub(crate) fn vorbis(channels: u8) -> Result { + let (streams, coupled, table) = *channels + .checked_sub(1) + .and_then(|index| VORBIS.get(index as usize)) + .ok_or(Error::UnsupportedChannelCount(channels as u32))?; + Self::new(Config { + family: 1, + streams, + coupled, + table, + }) + } + + /// Check `config` into a mapping with one output channel per table entry. + /// + /// Refuses family 0, a channel count the family does not allow, and a table + /// that names no streams, more coupled streams than streams, or a channel + /// past the decoded ones. + pub fn new(config: Config<'_>) -> Result { + let Config { + family, + streams, + coupled, + table, + } = config; + let max_channels = match family { + 0 => return Err(Error::UnsupportedMappingFamily(0)), + 1 => 8, + _ => 255, + }; + if table.is_empty() || table.len() > max_channels { + return Err(Error::UnsupportedChannelCount(table.len() as u32)); + } + + let decoded = streams as u32 + coupled as u32; + if streams == 0 || coupled > streams || decoded > 255 { + return Err(Error::InvalidMappingTable); + } + if table.iter().any(|&entry| entry != 255 && entry as u32 >= decoded) { + return Err(Error::InvalidMappingTable); + } + + let mut padded = [0u8; 255]; + padded[..table.len()].copy_from_slice(table); + Ok(Self { + family, + streams, + coupled, + channels: table.len() as u8, + table: padded, + }) + } + + pub(super) fn parse(buf: &mut T, family: u8, channels: u8) -> Result { + if buf.remaining() < 2 + channels as usize { + return Err(Error::MappingTableTooShort); + } + let streams = buf.get_u8(); + let coupled = buf.get_u8(); + let mut table = [0u8; 255]; + buf.copy_to_slice(&mut table[..channels as usize]); + Self::new(Config { + family, + streams, + coupled, + table: &table[..channels as usize], + }) + } + + /// The channel mapping family: 1 is the Vorbis speaker order, 2 and 3 are + /// ambisonics, and 255 is channels with no declared position. + pub fn family(&self) -> u8 { + self.family + } + + /// Number of Opus streams in each packet. + pub fn streams(&self) -> u8 { + self.streams + } + + /// How many of those streams are coupled (stereo); they come first. + pub fn coupled(&self) -> u8 { + self.coupled + } + + /// The decoded channel feeding each output channel, 255 for silence. + pub fn table(&self) -> &[u8] { + &self.table[..self.channels as usize] + } +} + +impl std::fmt::Debug for Mapping { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Mapping") + .field("family", &self.family) + .field("streams", &self.streams) + .field("coupled", &self.coupled) + .field("table", &self.table()) + .finish() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn new_validates_the_table() { + fn new(family: u8, streams: u8, coupled: u8, table: &[u8]) -> Result { + Mapping::new(Config { + family, + streams, + coupled, + table, + }) + } + + let mapping = new(255, 2, 0, &[0, 1, 255]).unwrap(); + assert_eq!(mapping.family(), 255); + assert_eq!((mapping.streams(), mapping.coupled()), (2, 0)); + assert_eq!(mapping.table(), &[0, 1, 255]); + + assert!(matches!(new(0, 1, 1, &[0, 1]), Err(Error::UnsupportedMappingFamily(0)))); + assert!(matches!(new(1, 1, 0, &[]), Err(Error::UnsupportedChannelCount(0)))); + assert!(matches!(new(1, 9, 0, &[0; 9]), Err(Error::UnsupportedChannelCount(9)))); + for (streams, coupled, table) in [(0, 0, &[0][..]), (1, 2, &[0]), (1, 1, &[2]), (200, 100, &[0])] { + assert!( + matches!(new(1, streams, coupled, table), Err(Error::InvalidMappingTable)), + "{streams} streams, {coupled} coupled, {table:?}" + ); + } + } +} diff --git a/rs/moq-mux/src/codec/opus/mod.rs b/rs/moq-mux/src/codec/opus/mod.rs index a73e9227a6c..749cce7217c 100644 --- a/rs/moq-mux/src/codec/opus/mod.rs +++ b/rs/moq-mux/src/codec/opus/mod.rs @@ -4,8 +4,10 @@ //! publishes raw Opus frames (no Ogg framing) to a moq broadcast. mod import; +pub mod mapping; pub use import::*; +pub use mapping::Mapping; use bytes::{Buf, Bytes}; @@ -37,8 +39,8 @@ pub enum Error { #[error("invalid OpusHead channel mapping table")] InvalidMappingTable, - /// [`Config::encode`] only emits channel mapping family 0. - #[error("cannot encode channel mapping family {0}")] + /// [`Mapping::new`] was given family 0, which has no mapping table. + #[error("channel mapping family {0} has no mapping table")] UnsupportedMappingFamily(u8), } @@ -58,79 +60,6 @@ pub struct Config { pub mapping: Option, } -/// An OpusHead channel mapping table (RFC 7845 §5.1.1), present for every family but 0. -#[derive(Clone, Copy, PartialEq, Eq)] -pub struct Mapping { - family: u8, - streams: u8, - coupled: u8, - channels: u8, - // Sized for the most channels any family allows, so `Config` stays `Copy`. - table: [u8; 255], -} - -impl Mapping { - fn parse(buf: &mut T, family: u8, channels: u8) -> Result { - if buf.remaining() < 2 + channels as usize { - return Err(Error::MappingTableTooShort); - } - let streams = buf.get_u8(); - let coupled = buf.get_u8(); - if streams == 0 || coupled > streams || streams as u32 + coupled as u32 > 255 { - return Err(Error::InvalidMappingTable); - } - - let mut table = [0u8; 255]; - for entry in &mut table[..channels as usize] { - *entry = buf.get_u8(); - // 255 marks a silent channel. - if *entry != 255 && *entry >= streams + coupled { - return Err(Error::InvalidMappingTable); - } - } - - Ok(Self { - family, - streams, - coupled, - channels, - table, - }) - } - - /// The channel mapping family: 1 is the Vorbis speaker order, 2 and 3 are - /// ambisonics, and 255 is channels with no declared position. - pub fn family(&self) -> u8 { - self.family - } - - /// Number of Opus streams in each packet. - pub fn streams(&self) -> u8 { - self.streams - } - - /// How many of those streams are coupled (stereo); they come first. - pub fn coupled(&self) -> u8 { - self.coupled - } - - /// The decoded channel feeding each output channel, 255 for silence. - pub fn table(&self) -> &[u8] { - &self.table[..self.channels as usize] - } -} - -impl std::fmt::Debug for Mapping { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.debug_struct("Mapping") - .field("family", &self.family) - .field("streams", &self.streams) - .field("coupled", &self.coupled) - .field("table", &self.table()) - .finish() - } -} - impl Config { /// Build a mono/stereo Opus config with no pre-skip. pub fn new(sample_rate: u32, channel_count: u32) -> Self { @@ -169,17 +98,9 @@ impl Config { buf.advance(2); // Skip gain until if/when we support it. let family = buf.get_u8(); - let max_channels = match family { - 0 => 2, - 1 => 8, - _ => 255, - }; - if channel_count == 0 || channel_count > max_channels { - return Err(Error::UnsupportedChannelCount(channel_count)); - } - let mapping = match family { - 0 => None, + 0 if (1..=2).contains(&channel_count) => None, + 0 => return Err(Error::UnsupportedChannelCount(channel_count)), family => Some(Mapping::parse(buf, family, channel_count as u8)?), }; @@ -195,28 +116,35 @@ impl Config { }) } - /// Encode the minimal OpusHead packet (19 bytes; channel mapping family - /// 0 and zero gain). + /// Encode an OpusHead packet (RFC 7845 §5.1) with zero gain, followed by + /// the channel mapping table when there is one. /// - /// Errors with [`Error::UnsupportedChannelCount`] unless `channel_count` is 1 - /// or 2, since mapping family 0 is only defined for mono/stereo per RFC 7845 §5.1. - /// Multi-channel streams need family 1 with a channel mapping table, which - /// this helper does not emit, so any `mapping` is [`Error::UnsupportedMappingFamily`]. + /// Errors with [`Error::UnsupportedChannelCount`] when `channel_count` is not + /// 1 or 2 without a `mapping` (family 0 is only defined for mono/stereo), or + /// is not the mapping's own channel count. pub fn encode(&self) -> Result { - if let Some(mapping) = &self.mapping { - return Err(Error::UnsupportedMappingFamily(mapping.family)); - } - if !(1..=2).contains(&self.channel_count) { + let valid = match &self.mapping { + None => (1..=2).contains(&self.channel_count), + Some(mapping) => self.channel_count == mapping.table().len() as u32, + }; + if !valid { return Err(Error::UnsupportedChannelCount(self.channel_count)); } - let mut head = Vec::with_capacity(19); + + let mut head = Vec::with_capacity(21 + self.channel_count as usize); head.extend_from_slice(b"OpusHead"); head.push(1); // version head.push(self.channel_count as u8); head.extend_from_slice(&self.pre_skip.to_le_bytes()); head.extend_from_slice(&self.sample_rate.to_le_bytes()); head.extend_from_slice(&0i16.to_le_bytes()); // output gain - head.push(0); // channel mapping family (0 = mono/stereo) + match &self.mapping { + None => head.push(0), + Some(mapping) => { + head.extend_from_slice(&[mapping.family(), mapping.streams(), mapping.coupled()]); + head.extend_from_slice(mapping.table()); + } + } Ok(Bytes::from(head)) } } @@ -322,8 +250,36 @@ mod tests { assert_eq!(mapping.coupled(), 2); assert_eq!(mapping.table(), &[0, 4, 1, 2, 3, 5]); - // Encode only emits family 0. - assert!(matches!(parsed.encode(), Err(Error::UnsupportedMappingFamily(1)))); + // The table survives a re-encode. + assert_eq!(parsed.encode().unwrap(), bytes); + } + + #[test] + fn encodes_the_vorbis_mappings() { + for channels in 1..=8u8 { + let mut config = Config::new(48_000, channels as u32); + config.mapping = Some(Mapping::vorbis(channels).unwrap()); + let head = config.encode().unwrap(); + assert_eq!(head.len(), 21 + channels as usize, "{channels} channels"); + assert_eq!( + Config::parse(&mut head.as_ref()).unwrap(), + config, + "{channels} channels" + ); + } + + // 5.1 is the table libopus and ffmpeg write. + let five_one = Mapping::vorbis(6).unwrap(); + assert_eq!((five_one.streams(), five_one.coupled()), (4, 2)); + assert_eq!(five_one.table(), &[0, 4, 1, 2, 3, 5]); + + assert!(matches!(Mapping::vorbis(0), Err(Error::UnsupportedChannelCount(0)))); + assert!(matches!(Mapping::vorbis(9), Err(Error::UnsupportedChannelCount(9)))); + + // The channel count must agree with the table. + let mut config = Config::new(48_000, 5); + config.mapping = Some(five_one); + assert!(matches!(config.encode(), Err(Error::UnsupportedChannelCount(5)))); } #[test] diff --git a/rs/moq-mux/src/container/ts/import.rs b/rs/moq-mux/src/container/ts/import.rs index 057d998aa1e..c8ac4d5090a 100644 --- a/rs/moq-mux/src/container/ts/import.rs +++ b/rs/moq-mux/src/container/ts/import.rs @@ -497,11 +497,10 @@ impl Import { // by an 'Opus' registration descriptor. Channels and the (always 48 kHz) rate // come from the descriptors, so the importer is built up front. StreamType::Mpeg2PacketizedData if registration_format(descriptors) == Some(*b"Opus") => { - let channel_count = opus_channel_count(descriptors).unwrap_or(2); + let config = opus_config(descriptors)?; let track = self .broadcast .unique_track(".opus", self.catalog.track_info(hang::catalog::PRIORITY.audio))?; - let config = opus::Config::new(48_000, channel_count); let mut config: hang::catalog::AudioConfig = config.into(); config.container = self.container.clone(); Stream::Opus(Box::new(OpusStream { @@ -2342,14 +2341,15 @@ fn registration_format(descriptors: &[mpeg2ts::ts::Descriptor]) -> Option<[u8; 4 .and_then(|s| s.try_into().ok()) } -/// The Opus channel count from the DVB extension descriptor (tag 0x7f, ext tag 0x80). +/// The OpusHead implied by the DVB extension descriptor (tag 0x7f, ext tag 0x80). /// /// `channel_config_code` follows the Opus-in-TS mapping (and ffmpeg's demuxer): 0 is dual -/// mono (decoded as stereo), 1..=8 is the channel count directly. Higher codes (0x81 -/// explicitly-coded layouts, reserved values) aren't supported, so they fall back to the -/// caller's default rather than being read as a raw 129..=255 count. -fn opus_channel_count(descriptors: &[mpeg2ts::ts::Descriptor]) -> Option { - descriptors +/// mono (decoded as stereo), 1..=8 is the channel count directly, with family 0 for mono and +/// stereo and the Vorbis family 1 mapping above that. Higher codes (0x81 explicitly-coded +/// layouts, reserved values) aren't supported, so they fall back to stereo rather than being +/// read as a raw 129..=255 count. +fn opus_config(descriptors: &[mpeg2ts::ts::Descriptor]) -> crate::Result { + let channels = descriptors .iter() .find(|d| d.tag == 0x7f && d.data.first() == Some(&0x80)) .and_then(|d| d.data.get(1)) @@ -2358,6 +2358,13 @@ fn opus_channel_count(descriptors: &[mpeg2ts::ts::Descriptor]) -> Option { 1..=8 => Some(cc as u32), _ => None, }) + .unwrap_or(2); + + let mut config = opus::Config::new(48_000, channels); + if channels > 2 { + config.mapping = Some(opus::Mapping::vorbis(channels as u8)?); + } + Ok(config) } /// Parse one Opus-in-TS access-unit control header, returning `(header_len, payload_size)`. diff --git a/rs/moq-mux/src/container/ts/import_test.rs b/rs/moq-mux/src/container/ts/import_test.rs index d0c48534ac6..46f8b2c792a 100644 --- a/rs/moq-mux/src/container/ts/import_test.rs +++ b/rs/moq-mux/src/container/ts/import_test.rs @@ -200,6 +200,29 @@ fn import_opus_catalog() { assert_eq!(audio.channel_count, 2); } +/// `opus_5_1.ts` is a 440 Hz center channel in 5.1, which ffmpeg's libopus +/// encodes as family 1 and its muxer labels `channel_config_code` 6: +/// `ffmpeg -f lavfi -i sine=frequency=440:sample_rate=48000:duration=0.5 +/// -ac 6 -c:a libopus -b:a 128k -f mpegts opus_5_1.ts`. The descriptor names only +/// the channel count, so the importer must synthesize the Vorbis mapping table +/// or the track has no OpusHead a decoder accepts. +#[test] +fn import_opus_surround_catalog() { + let data = include_bytes!("test_data/opus_5_1.ts"); + let catalog = import_ts(data); + + assert_eq!(catalog.audio.renditions.len(), 1, "expected one Opus track"); + let audio = catalog.audio.renditions.values().next().unwrap(); + assert_eq!(audio.channel_count, 6); + + let head = crate::codec::opus::Config::parse(&mut audio.description.as_deref().expect("an OpusHead")).unwrap(); + assert_eq!(head.channel_count, 6); + let mapping = head.mapping.expect("a family 1 mapping"); + assert_eq!(mapping.family(), 1); + assert_eq!((mapping.streams(), mapping.coupled()), (4, 2)); + assert_eq!(mapping.table(), &[0, 4, 1, 2, 3, 5]); +} + /// Opus frames from real ffmpeg output must decode: a non-empty run of Opus packets, /// each a plausible size (the control header was stripped, not left in the payload). #[tokio::test(start_paused = true)] diff --git a/rs/moq-mux/src/container/ts/test_data/opus_5_1.ts b/rs/moq-mux/src/container/ts/test_data/opus_5_1.ts new file mode 100644 index 00000000000..14ad7cb9c04 Binary files /dev/null and b/rs/moq-mux/src/container/ts/test_data/opus_5_1.ts differ