Skip to content

fix(mux): a TS import ends on a packet header it cannot parse - #9243

Merged
kixelated merged 3 commits into
mainfrom
quest/m1/ts-packet-abort
Nov 21, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m1/ts-packet-abort

Conversation

@kixelated

@kixelated kixelated commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Closes the quest quest/m1/ts-packet-abort.md and deletes it.

Problem

moq-mux's TS import counted an intact, unflagged adaptation field or PES header that does not parse as damaged and resumed at the next PES. Over UDP and SRT a packet arrives whole or not at all, and RF corruption sets the transport error indicator, so such a header is a muxer bug or an unexplained flip. The maintainer's rule (2026-11-18): "We should not try to support random bit flips that we don't understand and limp along with broken functionality." moq-dev/moq#9170 applied it to undecodable units and left these packet-level paths.

Approach

In rs/moq-mux/src/container/ts/import.rs:

  • A malformed adaptation field on a carried stream or the PCR PID returns Error::Malformed (naming the PID) before its clock bits are read. A PID the import does not carry (Stream::Ignored) is still not read, so its header cannot end anything.
  • A PES start whose prefix, marker bits, or lengths do not parse returns Error::Malformed, after the preceding PES flushes, so the units ahead of it still publish. pes_packet now hands every start to pes_header, removing the duplicated parse.
  • A header still partial when the next start arrives also ends the import: loss, a TEI-flagged packet, and scrambling all drop the partial first, so one still there is a header the muxer cut short.
  • Unchanged: loss (continuity break), TEI-flagged packets, and headers cut short by loss, a PMT revision, or end of input stay damage.
  • PES-level scrambling (PES_scrambling_control) goes through the same path as TS-level scrambling on every carriage: counted in refused, warned once per PID, its unit lost, and the stream resumes once clear (video at its next keyframe). Previously it counted as damage on a decoded stream and ended a whole-PES (pesPacket) track for good.

Docs: doc/bin/ts.md and doc/bin/srt.md say a header that arrives whole but does not parse ends the import/session; Error's doc says which faults are damage and which end the import.

Decisions

Made within the quest's remit, for review:

  • Scrambled PES: refused, route kept (not refuse(), not damage). Conditional access is a known source, so it is not damage; refuse() would end the track and drop the PID for good, while TS-level scrambling, the common CA case, already refuses per packet and resumes when clear. Matching that keeps one rule for scrambling, so it applies to whole-PES carriage (KLV, DVB AC-3, private PES) too, which refusal() used to end for good (Pullfrog's finding). Tests: scrambled_pes_is_refused_until_clear, a_scrambled_pes_is_refused_until_clear.
  • "PES header truncated at the next start" ends the import. The quest did not list it; it is an intact short header by the same argument.
  • Programs multiplex: any program's import error already ends the whole Programs decode (as for fix(mux): a TS import ends on a unit it cannot decode #9170's undecodable unit). a_corrupt_pmt_counts_once_whoever_reads_it's last case maps one program's audio PID onto another program's PMT PID, an illegal PID collision, so it now expects the end (the scan still counts the CRC error first).

Impact

  • Public API: none.
  • Behavior: a TS or SRT ingest with an intact malformed adaptation field or PES header ends with the cause instead of continuing; a PES-scrambled PES counts as refused rather than damaged, and no longer ends a whole-PES track.
  • Wire: none.

Validation

  • just ci: all 4937 Rust tests and every check pass.
  • Real binary: built moq-relay and moq, ran a local relay with test/ts/relay.toml, and piped a 6 s ffmpeg H.264+AAC TS through moq import ts with one video packet altered:
    • adaptation field length 255: exits 1 with Error: malformed TS adaptation field on PID 256.
    • PES marker bits zeroed: exits 1 with Error: PES header on PID 256: unexpected PES header marker bits.
    • same packet TEI-flagged: keeps publishing (killed by the timeout), logs dropped a damaged TS unit ... error=transport error indicator, damaged=1.
    • PES_scrambling_control set: keeps publishing, logs the scrambling refusal, refused=1 on the video PID.
    • unaltered clip: keeps publishing, no damage.
  • Unit tests rewritten as "ends with the cause": malformed_headers_end_the_import (PES, adaptation, adaptation carrying a PCR flag; checks the clock never moved, no damage counted, and the four pictures ahead published), invalid_adaptation_fields_end_the_import_on_media_and_clock_pids (media, dedicated PCR, and an ignored PID that must not end it), malformed_pes_start_ends_the_import_after_the_unit_before_it, malformed_pes_prefix_ends_the_import_without_waiting_for_more_bytes, pes_length_overruns_fail_before_waiting_for_the_optional_header, pes_fragmented_header_resets_on_loss_and_eof_and_ends_at_a_new_start. damaged_tei_recovers and clean_video_has_no_damage keep the damage path covered.
  • Whole-PES scrambling: a_scrambled_pes_is_refused_until_clear (KLV PES, scrambled PES, KLV PES) publishes the first and third, keeps the track and its catalog entry, ends it cleanly, and counts refused=1. With the import.rs fix reverted it fails (the track leaves the catalog). cargo test -p moq-mux --lib container::ts: 774 pass; just check passes. Not run on the binary: the path is the same importer call the CLI makes, and the earlier binary run covered the decoded case.

Caused by

No single PR: counting malformed headers as damage is as old as the import's damage handling, and became wrong when the maintainer set the rule on 2026-11-18.

Alternatives

  • Refusing the stream for good on a scrambled PES (refuse()): rejected above, since TS-level scrambling resumes when clear.
  • Ending the import on scrambled PES: CA is understood source behavior, not a muxer bug.

Follow-ups

None. Open PRs touching the same file, for merge awareness: moq-dev/moq#9105, moq-dev/moq#9069.

Models

Size: S
Implementer: Claude, claude-opus-5-5, unknown effort (arm A)
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, S, arm A)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: dd98e9f

  • Base: origin/main at ba44b82

  • Risk: medium, TS and SRT import users now lose the session on malformed intact headers, while PES scrambling changes refusal and recovery behavior; no API or wire format changes, and a revert restores the previous behavior.

  • Classified: medium by Codex, gpt-6.1-sol, low effort, session 01a1fb9e-df25-7692-bd2a-fd99ea2072ee

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1fb9e-f994-7472-8942-ea802749d422, round 1

  • Score: 9/10, Source review found no actionable defects and supports landing, with builds and tests intentionally not run.

  • Rounds: 2; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 2 commits November 20, 2026 10:26
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title A TS import ends on a packet header it cannot parse fix(mux): a TS import ends on a packet header it cannot parse Nov 20, 2026
@kixelated
kixelated marked this pull request as ready for review November 20, 2026 19:45
@coderabbitai

coderabbitai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b85e0049-034e-4d84-a4c6-519550ce766f
Commits

Reviewing files that changed from the base of the PR and between 9828a20 and dd98e9f.

Files selected for processing: 9

  • doc/bin/srt.md
  • doc/bin/ts.md
  • quest/m1/ts-packet-abort.md
  • rs/moq-mux/src/container/ts/error.rs
  • rs/moq-mux/src/container/ts/import.rs
  • rs/moq-mux/src/container/ts/pes_packet_test.rs
  • rs/moq-mux/src/container/ts/programs.rs
  • rs/moq-mux/src/container/ts/stats.rs
  • rs/moq-mux/src/container/ts/step.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one doc claim overreaches.

Approach: sound. It applies #9170's rule to the packet-header paths at the single points where they're parsed, and it keeps loss, TEI, PMT revision, and EOF truncation as damage.

Reviewed changes

Reviewed the full diff at 2eb74ba against main.

  • Malformed adaptation field ends the import: Import::packet returns Error::Malformed naming the PID for any carried non-Ignored stream or the PCR PID, before the clock reads it. Ignored PIDs never reach pes_packet (import.rs route match), so they can't end anything.
  • PES header parse unified: pes_packet now flushes the prior PES and hands every start to pes_header, which turns header_len and parse failures into Error::Malformed.
  • Header truncated at the next start ends the import: the change relies on every damage path (damage, salvage, scrambled, end_route) already clearing headers, and I checked that each one does.
  • PES-level scrambling on decoded streams is refused, not damaged: it goes through scrambled(), so it counts in refused, the warning is shared, and the route is kept.
  • Tests and docs: the old damage tests are rewritten as end-with-cause tests, scrambled and ignored-PID cases are added, and the Programs collision test now expects the end.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread doc/bin/ts.md Resolved
…clear, like any carriage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed dd98e9f, which resolves the earlier finding by giving PES scrambling one rule on every carriage.

  • Whole-PES scrambling is refused until clear: handle_pes_start now checks pes.scrambled() before refusal() and routes every carriage through scrambled(). A KLV, DVB AC-3, or private PES stream keeps its route and catalog entry, and it resumes at the next clear PES instead of ending.
  • refusal() drops scrambling: it now lists only header-export and clock-field refusals. The stats::Stream::refused doc and doc/bin/ts.md match the new behavior.
  • Tests: a_scrambled_pes_is_refused_until_clear checks that the PES on each side publishes, the track ends cleanly, and refused == 1. The scrambling case is gone from the refuse-the-stream table.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 21, 2026
Merged via the queue into main with commit e93789e Nov 21, 2026
5 checks passed
@kixelated
kixelated deleted the quest/m1/ts-packet-abort branch November 21, 2026 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant