Repository navigation
feat(ts): import surround Opus with a family 1 OpusHead #4282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
kixelated
merged 6 commits into
quest/m1/audio-codecs/README
from
quest/m1/audio-codecs/ts-opus-surround
Sep 27, 2026
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
b66546e
quest(audio-codecs): claim ts-opus-surround
kixelated d7928d7
feat(ts): import surround Opus with a family 1 OpusHead
kixelated 02fcd3b
feat(opus): a validated public Mapping::new, and follow-up quests
kixelated a3e53e1
refactor(opus): Mapping::new takes a MappingParts struct
kixelated 9083694
refactor(opus): move Mapping into opus::mapping with mapping::Config
kixelated 2a66165
Merge quest/m1/audio-codecs/README into ts-opus-surround
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [S] Surround Opus from GStreamer | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-gst`'s sink publishes a 3 to 8 channel `audio/x-opus` stream with the | ||
| OpusHead its caps describe, so surround Opus from `opusenc` plays natively like | ||
| the same stream from Matroska or MPEG-TS. | ||
|
|
||
| ## Plan | ||
|
|
||
| - The sink refuses more than two channels because it builds the head from | ||
| `channels` and `rate` alone. Multichannel Opus caps also carry | ||
| `channel-mapping-family`, `stream-count`, `coupled-count`, and | ||
| `channel-mapping`; build the mapping with `opus::Mapping::new`, which | ||
| validates them, and refuse caps that omit or contradict them. | ||
| - Prefer a `streamheader` OpusHead when the caps carry one, if that path is | ||
| simpler and covers the same streams. | ||
| - Regression: an `opusenc` 5.1 pipeline publishes a family 1 description that | ||
| `moq-audio` decodes to six channels. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # [S] TS Opus import refuses or parses extended channel codes | ||
|
|
||
| ## Goal | ||
|
|
||
| An MPEG-TS Opus stream whose extension descriptor carries a | ||
| `channel_config_code` of 0x81 or above imports with its real OpusHead or is | ||
| refused. It never decodes as a guessed stereo stream, which misreads every | ||
| multistream packet. | ||
|
|
||
| ## Plan | ||
|
|
||
| - The importer falls back to stereo for codes it does not know. 0x81 is | ||
| specified as an explicitly coded layout (channel count, mapping family, stream | ||
| counts, and table in the descriptor), and ffmpeg's muxer also writes | ||
| `0x80 | channels` for an alternate table its own demuxer does not read. Check | ||
| the Opus-in-TS spec and ffmpeg before deciding which codes to parse. | ||
| - Build parsed layouts with the public `opus::Mapping::new`, which validates | ||
| the table; refuse any code that is reserved or does not parse, per stream, | ||
| without failing the rest of the program. | ||
| - Fixtures from a real muxer where one exists; a hand-built descriptor | ||
| otherwise, noted as such. | ||
|
|
||
| Public API: none. Wire: none. | ||
|
|
||
| ## Related | ||
|
|
||
| - [TS Opus export refusals](/quest/m1/audio-codecs/ts-opus-export-refusals.md) - the exporter side of the same descriptor |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # [S] TS Opus export refuses what it cannot label | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-mux`'s MPEG-TS exporter never labels an Opus track with a channel | ||
| configuration its packets do not have. A track whose OpusHead the Opus | ||
| extension descriptor can describe exports as today; anything else is refused, | ||
| not written with a guessed `channel_config_code`. | ||
|
|
||
| ## Plan | ||
|
|
||
| - The exporter derives `channel_config_code` from the catalog channel count | ||
| alone, clamped to 1..=8, so a family 255 or ambisonics head, a family 1 head | ||
| with a non-Vorbis table, or more than eight channels exports mislabeled. | ||
| Decide from the parsed description: family 0, and family 1 with the Vorbis | ||
| default mapping, keep the plain code; refuse the rest, or write the explicit | ||
| layout if the import side of | ||
| [TS Opus channel codes](/quest/m1/audio-codecs/ts-opus-channel-codes.md) settles | ||
| one both sides agree on. | ||
| - A track with no description stays mono or stereo only. | ||
| - Tests with real heads, checked against ffprobe where ffmpeg reads the result. | ||
|
|
||
| Public API: none. Wire: none; TS output changes only for inputs it mislabeled. |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| //! Container imports that must decode end to end: a stream `moq-mux` imports | ||
| //! plays through [`decode::Consumer`] like the same stream from any container. | ||
|
|
||
| use std::time::Duration; | ||
|
|
||
| use moq_audio::{Format, Layout, decode}; | ||
|
|
||
| /// MPEG-TS names surround Opus only by channel count, so the importer has to | ||
| /// synthesize the OpusHead mapping table or the decoder refuses the track. | ||
| #[tokio::test] | ||
| async fn ts_surround_opus_decodes() { | ||
| // A 440 Hz center channel in 5.1; see `import_opus_surround_catalog` in moq-mux. | ||
| let data = include_bytes!("../../moq-mux/src/container/ts/test_data/opus_5_1.ts"); | ||
|
|
||
| let mut broadcast = moq_net::broadcast::Info::new().produce(); | ||
| let broadcast_consumer = broadcast.consume(); | ||
| let catalog = moq_mux::catalog::Producer::new(&mut broadcast, moq_mux::catalog::Config::default()).unwrap(); | ||
| let mut import = moq_mux::container::ts::Import::new(broadcast, catalog.reserve()); | ||
| import.decode(&data[..]).unwrap(); | ||
| import.finish().unwrap(); | ||
|
|
||
| let snapshot = catalog.snapshot(); | ||
| let (name, config) = snapshot.audio.renditions.iter().next().expect("an Opus track"); | ||
|
|
||
| let mut options = decode::Options::default(); | ||
| options.output.format = Format::F32; | ||
| // The whole file is imported before anything decodes it. | ||
| options.max_age = Duration::from_secs(30); | ||
| let mut consumer = decode::Consumer::new(&broadcast_consumer, config, name, options) | ||
| .await | ||
| .unwrap(); | ||
| assert_eq!(consumer.layout(), Layout::FivePointOne); | ||
|
|
||
| // Canonical 5.1 order: FL, FR, FC, LFE, SL, SR. | ||
| let mut energy = [0.0f64; 6]; | ||
| let mut frames = 0; | ||
| while let Some(frame) = tokio::time::timeout(Duration::from_secs(5), consumer.read()) | ||
| .await | ||
| .expect("decoded frame timed out") | ||
| .unwrap() | ||
| { | ||
| let (samples, rest) = frame.data.as_chunks::<4>(); | ||
| assert!(rest.is_empty()); | ||
| for chunk in samples.as_chunks::<6>().0 { | ||
| for (channel, sample) in energy.iter_mut().zip(chunk) { | ||
| *channel += f32::from_le_bytes(*sample).powi(2) as f64; | ||
| } | ||
| frames += 1; | ||
| } | ||
| } | ||
|
|
||
| assert!(frames > 12_000, "expected most of 0.5 s at 48 kHz, got {frames} frames"); | ||
| let center = energy[2]; | ||
| assert!( | ||
| center / frames as f64 > 0.001, | ||
| "center should carry the tone: {energy:?}" | ||
| ); | ||
| for (channel, &other) in energy.iter().enumerate().filter(|&(channel, _)| channel != 2) { | ||
| assert!( | ||
| other < center / 100.0, | ||
| "channel {channel} should be near silent: {energy:?}" | ||
| ); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| //! The OpusHead channel mapping table (RFC 7845 §5.1.1). | ||
|
|
||
| use bytes::Buf; | ||
|
|
||
| use super::{Error, Result}; | ||
|
|
||
| /// An OpusHead channel mapping table (RFC 7845 §5.1.1), present for every family but 0. | ||
| #[derive(Clone, Copy, PartialEq, Eq)] | ||
| pub struct Mapping { | ||
| family: u8, | ||
| streams: u8, | ||
| coupled: u8, | ||
| channels: u8, | ||
| // Sized for the most channels any family allows, so `opus::Config` stays `Copy`. | ||
| table: [u8; 255], | ||
| } | ||
|
|
||
| /// The fields of a channel mapping table, checked into a [`Mapping`] by [`Mapping::new`]. | ||
| #[derive(Clone, Copy, Debug, PartialEq, Eq)] | ||
| pub struct Config<'a> { | ||
| /// The channel mapping family; 0 has no table and is refused. | ||
| pub family: u8, | ||
| /// Number of Opus streams in each packet. | ||
| pub streams: u8, | ||
| /// How many of those streams are coupled (stereo); they come first. | ||
| pub coupled: u8, | ||
| /// The decoded channel feeding each output channel, 255 for silence. | ||
| pub table: &'a [u8], | ||
| } | ||
|
|
||
| /// The family 1 stream counts and table for 1 to 8 channels, as libopus and | ||
| /// RFC 7845 §5.1.1.2 lay out the Vorbis channel orders. | ||
| const VORBIS: [(u8, u8, &[u8]); 8] = [ | ||
| (1, 0, &[0]), | ||
| (1, 1, &[0, 1]), | ||
| (2, 1, &[0, 2, 1]), | ||
| (2, 2, &[0, 1, 2, 3]), | ||
| (3, 2, &[0, 4, 1, 2, 3]), | ||
| (4, 2, &[0, 4, 1, 2, 3, 5]), | ||
| (4, 3, &[0, 4, 1, 2, 3, 5, 6]), | ||
| (5, 3, &[0, 6, 1, 2, 3, 4, 5, 7]), | ||
| ]; | ||
|
|
||
| impl Mapping { | ||
| /// The family 1 mapping a surround encoder uses for `channels` in Vorbis | ||
| /// order, for sources that name only a channel count. | ||
| pub(crate) fn vorbis(channels: u8) -> Result<Self> { | ||
| let (streams, coupled, table) = *channels | ||
| .checked_sub(1) | ||
| .and_then(|index| VORBIS.get(index as usize)) | ||
| .ok_or(Error::UnsupportedChannelCount(channels as u32))?; | ||
| Self::new(Config { | ||
| family: 1, | ||
| streams, | ||
| coupled, | ||
| table, | ||
| }) | ||
| } | ||
|
|
||
| /// Check `config` into a mapping with one output channel per table entry. | ||
| /// | ||
| /// Refuses family 0, a channel count the family does not allow, and a table | ||
| /// that names no streams, more coupled streams than streams, or a channel | ||
| /// past the decoded ones. | ||
| pub fn new(config: Config<'_>) -> Result<Self> { | ||
| let Config { | ||
| family, | ||
| streams, | ||
| coupled, | ||
| table, | ||
| } = config; | ||
| let max_channels = match family { | ||
| 0 => return Err(Error::UnsupportedMappingFamily(0)), | ||
| 1 => 8, | ||
| _ => 255, | ||
| }; | ||
| if table.is_empty() || table.len() > max_channels { | ||
| return Err(Error::UnsupportedChannelCount(table.len() as u32)); | ||
| } | ||
|
|
||
| let decoded = streams as u32 + coupled as u32; | ||
| if streams == 0 || coupled > streams || decoded > 255 { | ||
| return Err(Error::InvalidMappingTable); | ||
| } | ||
| if table.iter().any(|&entry| entry != 255 && entry as u32 >= decoded) { | ||
| return Err(Error::InvalidMappingTable); | ||
| } | ||
|
|
||
| let mut padded = [0u8; 255]; | ||
| padded[..table.len()].copy_from_slice(table); | ||
| Ok(Self { | ||
| family, | ||
| streams, | ||
| coupled, | ||
| channels: table.len() as u8, | ||
| table: padded, | ||
| }) | ||
| } | ||
|
|
||
| pub(super) fn parse<T: Buf>(buf: &mut T, family: u8, channels: u8) -> Result<Self> { | ||
| if buf.remaining() < 2 + channels as usize { | ||
| return Err(Error::MappingTableTooShort); | ||
| } | ||
| let streams = buf.get_u8(); | ||
| let coupled = buf.get_u8(); | ||
| let mut table = [0u8; 255]; | ||
| buf.copy_to_slice(&mut table[..channels as usize]); | ||
| Self::new(Config { | ||
| family, | ||
| streams, | ||
| coupled, | ||
| table: &table[..channels as usize], | ||
| }) | ||
| } | ||
|
|
||
| /// The channel mapping family: 1 is the Vorbis speaker order, 2 and 3 are | ||
| /// ambisonics, and 255 is channels with no declared position. | ||
| pub fn family(&self) -> u8 { | ||
| self.family | ||
| } | ||
|
|
||
| /// Number of Opus streams in each packet. | ||
| pub fn streams(&self) -> u8 { | ||
| self.streams | ||
| } | ||
|
|
||
| /// How many of those streams are coupled (stereo); they come first. | ||
| pub fn coupled(&self) -> u8 { | ||
| self.coupled | ||
| } | ||
|
|
||
| /// The decoded channel feeding each output channel, 255 for silence. | ||
| pub fn table(&self) -> &[u8] { | ||
| &self.table[..self.channels as usize] | ||
| } | ||
| } | ||
|
|
||
| impl std::fmt::Debug for Mapping { | ||
| fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
| f.debug_struct("Mapping") | ||
| .field("family", &self.family) | ||
| .field("streams", &self.streams) | ||
| .field("coupled", &self.coupled) | ||
| .field("table", &self.table()) | ||
| .finish() | ||
| } | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| #[test] | ||
| fn new_validates_the_table() { | ||
| fn new(family: u8, streams: u8, coupled: u8, table: &[u8]) -> Result<Mapping> { | ||
| Mapping::new(Config { | ||
| family, | ||
| streams, | ||
| coupled, | ||
| table, | ||
| }) | ||
| } | ||
|
|
||
| let mapping = new(255, 2, 0, &[0, 1, 255]).unwrap(); | ||
| assert_eq!(mapping.family(), 255); | ||
| assert_eq!((mapping.streams(), mapping.coupled()), (2, 0)); | ||
| assert_eq!(mapping.table(), &[0, 1, 255]); | ||
|
|
||
| assert!(matches!(new(0, 1, 1, &[0, 1]), Err(Error::UnsupportedMappingFamily(0)))); | ||
| assert!(matches!(new(1, 1, 0, &[]), Err(Error::UnsupportedChannelCount(0)))); | ||
| assert!(matches!(new(1, 9, 0, &[0; 9]), Err(Error::UnsupportedChannelCount(9)))); | ||
| for (streams, coupled, table) in [(0, 0, &[0][..]), (1, 2, &[0]), (1, 1, &[2]), (200, 100, &[0])] { | ||
| assert!( | ||
| matches!(new(1, streams, coupled, table), Err(Error::InvalidMappingTable)), | ||
| "{streams} streams, {coupled} coupled, {table:?}" | ||
| ); | ||
| } | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this commit, repo-wide callers of this constructor are limited to internal parsing,
Mapping::vorbis, and this module's tests; the only external use described is a future GStreamer quest. Publishingmapping::ConfigandMapping::newnow unnecessarily freezes this shape intomoq-mux's API even though the TS change can remain crate-private, so keep them private until that consumer lands.AGENTS.md reference: AGENTS.md:L55-L60
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping it public: this was a deliberate call recorded under Decisions in the PR description.
opus::Config::mappingis already a public field andencodenow emits any mapping, so without a public constructor an outside crate could only obtain aMappingby parsing a head it already has.Mapping::newvalidates every field (the struct fields stay private), and the GStreamer surround quest in this PR (quest/m1/audio-codecs/gst-opus-surround.md) is the planned consumer.Mapping::vorbisstays crate-private since nothing outside moq-mux needs it yet.(Written by Claude Opus 5.5)