Skip to content

fix(ts): split and buffer-model DVB E-AC-3 access units - #5758

Merged
kixelated merged 14 commits into
mainfrom
quest/m2/ts-eac3
Oct 17, 2026
Merged

kixelated merged 14 commits into
mainfrom
quest/m2/ts-eac3

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

DVB E-AC-3 private-data PES were forwarded whole without decoder-buffer admission. Multi-frame PES could overflow B, and the T-STD grader refused the profile.

Approach

  • DVB E-AC-3 takes the treatment DVB AC-3 has, through one shared path (dolby_units, generalized from ac3_frames): split each PES into access units, carry a sync frame the PES leaves unfinished into the next, search for sync at a join, and drop a frame cut by a jump in the playhead or a lost PES.
  • An E-AC-3 access unit is independent substream 0 plus every substream after it (ETSI TS 102 366 E.1.3.1.2), all on its presentation interval. A substream the next PES carries, whole or completing, goes out on its own at its unit's time, since that unit already left with the last PES. One whose unit began before the join is dropped with a debug log.
  • Units count on from the PES's PTS by summed samples, so 44.1 kHz frames do not drift by a rounded period each.
  • A substream whose blocks or rate differ from its unit's is refused with an error. Transport damage (lost sync, a gap) keeps the AC-3 path's drop-and-warn, and a PES in which no sync frame can begin is dropped rather than forwarded whole.
  • DVB's 5,696-byte main buffer from ETSI TS 101 154 V2.9.1, 4.1.6.11 applies in admission and compliance grading.
  • An AC-3 core (bsid 10 or less) that an E-AC-3 stream carries as independent substream 0 begins its access unit.
  • Placed whole-PES tracks (feat(mux): export whole-PES tracks at their placement #5832) time DVB E-AC-3 removals by access unit too (dolby_removals, generalized from ac3_removals).
  • The compliance model gives every substream after independent substream 0 no duration of its own, additional independent ones included, and refuses what the export's parsers refuse: bsid 9, 10 and above 16, E-AC-3 strmtyp 3, and the reserved fscod2.

Impact

  • Public API and MoQ wire: no shape or format change.
  • MPEG-TS output: DVB E-AC-3 is scheduled one access unit per PES. A substream split from its unit by a source PES boundary goes out in its own PES with its unit's PTS.
  • DVB AC-3: same behavior, except that a PES with no sync frame is now dropped rather than forwarded whole, and a timestamp overflow while splitting is an error.

Decisions

  1. Unaligned E-AC-3 PES (no alignment rule in ETSI TS 102 366 Annex A; fix(ts): reassemble AC-3 sync frames split across passed-through PES #5363 met a hardware encoder that splits AC-3 frames):
    • ✅ Reassemble access units across PES in this PR, by generalizing ac3_frames (picked by the maintainer). Scope grows from S to M.
    • Land aligned-only with a clearer refusal, and plan reassembly as a follow-up quest.
    • Land as is (export stops with missing E-AC-3 sync word).
  2. A dependent substream whose access unit already left with the last PES (picked unattended):
    • ✅ Send it in its own PES at its unit's PTS (recommended): no added latency; the T-STD removes it with its unit.
    • Hold each PES's last unit until the next PES shows whether more substreams follow: one whole PES per unit, but a PES of latency on the boundary unit and a flush at end of stream.

Validation

  • New export regressions: 6 frames cut across PES at bytes 800 and 2000 (the export failed with missing E-AC-3 sync word before), a dependent substream cut from its unit at bytes 512 and 800, dependent substreams before the join, a substream PES timed by its unit, and a PES with no sync frame. Unit tests cover 44.1 kHz timing, a substream off its unit's interval, an AC-3 core, and placed-PES removals; test/ts/compliance.test.py (new, in just test ts) covers the model's substream durations and refused headers.
  • just check, just test ts, and just test ts-tstd (including the DVB E-AC-3 reference and the unsplit-PES B-overflow control) pass on this head.

Follow-ups

  • An AC-3 core with bsid 9 or 10 still loses sync, as ac3::parse_header takes bsid 8 or less.

(written by GPT-6)

(Written by Claude Opus 5.5)

kixelated and others added 7 commits October 16, 2026 11:48
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>

# Conflicts:
#	test/ts/tstd-controls.py
Generalize the AC-3 passthrough reassembly to E-AC-3: carry a sync frame
a PES leaves unfinished, search for sync at a join, and drop a frame cut
by a lost PES. Dependent and additional substreams join their access
unit; one the next PES carries goes out at its unit's time. Units count
on from the PES's PTS by summed samples, and a substream whose blocks or
rate differ from its unit's is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Placed whole-PES tracks model DVB AC-3 sync frames leaving the decoder
buffer one by one. Extend that to E-AC-3 access units, so a substream
leaves with its unit rather than the whole PES at its PTS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 14:58
@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 99aee4fa-0bdd-4d0a-a920-eab004feb60d
📥 Commits

Reviewing files that changed from the base of the PR and between 67df700 and 98f4ac0.

📒 Files selected for processing: 9
  • quest/m2/README.md
  • quest/m2/ts-eac3.md
  • rs/moq-mux/src/codec/eac3.rs
  • rs/moq-mux/src/container/ts/export.rs
  • rs/moq-mux/src/container/ts/export_test.rs
  • test/justfile
  • test/ts/compliance.py
  • test/ts/compliance.test.py
  • test/ts/tstd-controls.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-17T17:50:09.911377Z 98f4ac0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b8154985c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/export.rs Outdated
fn parse(self, data: &[u8]) -> crate::codec::legacy::Result<crate::codec::legacy::Header> {
match self {
Self::Ac3 => crate::codec::ac3::parse_header(data),
Self::Eac3 => crate::codec::eac3::sync_header(data),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Handle AC-3 cores before splitting E-AC-3

When a DVB Enhanced AC-3 stream begins an access unit with a valid AC-3 core (bsid <= 10), sync_header rejects that core; the sync search can then find the following enhanced dependent substream, which is dropped because no unit is open, silently removing valid audio that was previously passed through intact. Parse the permitted core or explicitly refuse this track instead of partially dropping its payload.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3ce7e16: an AC-3 core (bsid 10 or less) now parses as independent substream 0 and begins its access unit, with dolby_units_begin_at_an_ac3_core covering it. bsid 9 and 10 still fail to parse and take the lost-sync warning, as the AC-3 parser takes only bsid 8 or less.

(Written by Claude Opus 5.5)

Comment thread test/ts/compliance.py
"ac3-atsc": ("AC-3 (ATSC)", AC3_ATSC_BS, ac3_frame),
"eac3-atsc": ("E-AC-3 (ATSC)", EAC3_ATSC_BS, ac3_frame),
"ac3-dvb": ("AC-3 (DVB)", AC3_DVB_BS, ac3_frame),
"eac3-dvb": ("E-AC-3 (DVB)", EAC3_DVB_BS, ac3_frame),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Group additional independent E-AC-3 substreams

For DVB E-AC-3 containing a nonzero independent substream, this new mapping uses ac3_frame, which assigns that substream a nonzero duration and therefore models it as a separate sequential access unit. The exporter instead treats only independent substream 0 as an access-unit boundary and groups every following substream on the same presentation interval, so the compliance model advances removals too far and can report incorrect B underflow or overflow results. Use E-AC-3-specific grouping that starts a new unit only at independent substream 0.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3ce7e16: ac3_frame gives any substream after independent substream 0 no duration, dependent or independent, and the new test/ts/compliance.test.py (wired into just test ts) covers it.

(Written by Claude Opus 5.5)

… its unit

An E-AC-3 stream may carry AC-3 as independent substream 0; the export
now parses it and begins an access unit there instead of losing sync. The
compliance model gives any substream after independent substream 0 no
duration of its own, so additional independent substreams share their
unit's interval as the export schedules them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft October 17, 2026 17:01
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 17:01

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ce7e16f83

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/export.rs Outdated
Comment on lines 442 to 445
let Some(frames) = frames else {
let pending = Pending {
frame,
restart,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject Dolby PES payloads that have no sync

When a descriptor-marked E-AC-3 PES contains no credible sync frame, such as after a corrupted split-frame continuation, dolby_units returns None and this branch forwards the original malformed payload as a standalone PES. That silently emits undecodable audio instead of applying the documented drop/refusal behavior; return an error or explicitly drop the damaged payload rather than falling back to push.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e2afd11: a PES in which no sync frame can begin is now dropped (warning, or debug at a join) for both DVB AC-3 and E-AC-3, rather than forwarded whole. export_drops_a_dvb_eac3_pes_with_no_sync_frame covers it.

(Written by Claude Opus 5.5)

Comment thread test/ts/compliance.py
Comment on lines 775 to +779
blocks = (1, 2, 3, 6)[(h[4] >> 4) & 0x03] if h[4] >> 6 != 3 else 6
fscod = h[4] >> 6
rate = (48000, 44100, 32000)[fscod] if fscod != 3 else (24000, 22050, 16000)[(h[4] >> 4) & 0x03]
dependent = h[2] >> 6 == 1
return ((((h[2] & 0x07) << 8) | h[3]) + 1) * 2, 0.0 if dependent else 256 * blocks / rate
extends = h[2] >> 6 == 1 or (h[2] >> 3) & 0x07 != 0
return ((((h[2] & 0x07) << 8) | h[3]) + 1) * 2, 0.0 if extends else 256 * blocks / rate

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate reserved E-AC-3 rate codes before indexing

For a newly modeled DVB E-AC-3 stream with fscod == 3 and reserved fscod2 == 3, indexing the three-element reduced-rate tuple raises IndexError; access_units catches that exception as an end-of-capture truncation and stops parsing. If valid units precede the malformed frame, the analyzer can therefore report PASS while silently ignoring the malformed tail, so range-check this code and raise Refused explicitly.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e2afd11: ac3_frame raises Refused for fscod 3 with the reserved fscod2, covered in test/ts/compliance.test.py.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 17, 2026 10:07
…-AC-3 rate

A passed-through Dolby PES in which no sync frame can begin went out
whole; no decoder can use it, so it is dropped like the bytes ahead of a
first sync frame. The compliance model refuses an E-AC-3 frame with the
reserved fscod2 instead of mistaking it for the end of the capture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	rs/moq-mux/src/container/ts/export_test.rs
@kixelated
kixelated marked this pull request as draft October 17, 2026 17:15
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 17:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb65548b6a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/export.rs Outdated
Comment on lines +3437 to +3438
if end.as_micros().abs_diff(start.as_micros()) <= length.as_micros() / 2 {
open = Some(unit);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the open unit for a same-PTS dependent PES

When a dependent substream is placed in its own PES, that PES can carry the open access unit's PTS; this exporter itself produces that layout when dolby_place emits a carried dependent at unit.at. This check accepts only the following unit's timestamp, so re-exporting that output closes the open unit and then drops the dependent substream, silently removing enhancement channels. Preserve the unit when start matches either unit.at or its end, based on the first frame in the PES.

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 80d014d: a PES timed by the open unit (as one holding only its substreams is) keeps that unit open, and the next unit in it counts on from the unit's end. export_keeps_a_dvb_eac3_substream_pes_timed_by_its_unit covers it.

(Written by Claude Opus 5.5)

A PES of only a dependent substream may carry its access unit's PTS, as
the export's own output does. Keep that unit open for it, and time the
next unit from the unit's end, rather than closing it as a gap and
dropping the substream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft October 17, 2026 17:31
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 17:31

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80d014d8e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/ts/compliance.py
Comment on lines 829 to +830
if kind == 0x06 and tags & {"DVB_enhanced_AC3_descriptor", "enhanced_AC3_descriptor"}:
raise Refused("E-AC-3 as DVB private data takes its buffer from ETSI TS 101 154, not modelled")
return "eac3-dvb"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject reserved bsid values before grading DVB E-AC-3

For descriptor-marked DVB E-AC-3 containing a reserved bsid such as 17, this new branch now sends the stream through ac3_frame, whose > 10 test accepts every value through 31 even though its own comment limits E-AC-3 to 11–16. A syntactically regular sequence of those frames can therefore receive a PASS instead of the prior Refused; validate the upper bound before enabling the buffer model.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 80e8d1e: ac3_frame refuses bsid above 16, covered in test/ts/compliance.test.py.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft October 17, 2026 17:41
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 17:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80e8d1e317

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/ts/compliance.py Outdated
h = es[pos : pos + 6]
if len(h) < 6 or h[0] != 0x0B or h[1] != 0x77:
raise Refused(f"no AC-3 sync at ES offset {pos}")
if h[5] >> 3 > 16:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refuse unsupported low-rate AC-3 bsids

When a descriptor-marked DVB E-AC-3 stream contains an AC-3 core with bsid 9 or 10, neither this check nor the E-AC-3 branch rejects it, so the frame is modeled as ordinary AC-3 at the full sample rate. The Rust ac3::parse_header explicitly rejects values above 8 as unsupported low-sample-rate variants, so a regular sequence of these frames can incorrectly receive PASS with wrong durations and buffer timing; refuse 9 and 10 here as well.

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 98f4ac0: ac3_frame refuses bsid 9 and 10, matching ac3::parse_header.

(Written by Claude Opus 5.5)

Comment thread test/ts/compliance.py
Comment on lines +782 to +783
extends = h[2] >> 6 == 1 or (h[2] >> 3) & 0x07 != 0
return ((((h[2] & 0x07) << 8) | h[3]) + 1) * 2, 0.0 if extends else 256 * blocks / rate

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refuse the reserved E-AC-3 stream type

For a DVB E-AC-3 frame with reserved strmtyp == 3 (for example, byte 2 equal to 0xC0), this expression treats it as an independent access unit and returns a normal duration. A syntactically regular sequence can therefore pass the newly enabled eac3-dvb model even though the Rust sync_header rejects the same input as Eac3ReservedStreamType; reject this stream type before grading it.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 98f4ac0: ac3_frame refuses strmtyp 3, matching eac3::sync_header. Both are covered in test/ts/compliance.test.py.

(Written by Claude Opus 5.5)

… compliance model

The model now refuses what the export's parsers refuse: bsid 9 and 10,
and strmtyp 3, rather than grading them as ordinary frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft October 17, 2026 17:50
@kixelated
kixelated marked this pull request as ready for review October 17, 2026 17:50
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@kixelated
kixelated added this pull request to the merge queue Oct 17, 2026
Merged via the queue into main with commit 0df49c1 Oct 17, 2026
16 of 17 checks passed
@kixelated
kixelated deleted the quest/m2/ts-eac3 branch October 17, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant