Skip to content
Merged
7 changes: 5 additions & 2 deletions doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,11 @@ supplies a PES boundary or clock fails rather than retaining bytes indefinitely.
It takes one program: a multi-program
stream is refused until `--program` picks one, or `--program all` publishes
each as its own broadcast (`event.hang` becomes `event/1.hang`,
`event/2.hang`). A damaged packet is dropped on its own PID, so video freezes
until its next keyframe while the other tracks carry on. The importer also
`event/2.hang`), following the PAT: a program it adds publishes, one it drops
ends, and one that returns publishes as a new instance. A listed program whose
PMT does not arrive within 0.5 s of program clock ends the import. A damaged packet is
dropped on its own PID, so video freezes until its next keyframe while the
other tracks carry on. The importer also
logs feed health (stalled streams, TR 101 290 errors) without changing what it
publishes. `--passthrough` publishes the multiplex whole instead, every packet
in order on one [`m2ts`](/concept/hang#transport-streams) track with every
Expand Down
3 changes: 2 additions & 1 deletion doc/bin/srt.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ return or a replacement on the same connection, as they do for
A multi-program feed is refused unless `--program`
picks one: `--program 2` imports program 2 alone, and `--program all`
publishes each program as its own broadcast (`event.hang` becomes
`event/1.hang`, `event/2.hang`, and so on).
`event/1.hang`, `event/2.hang`, and so on), following the PAT as programs
join and leave.

```bash
moq --connect https://relay.example.com/anon --broadcast event.hang import srt --listen '[::]:9000' --program all
Expand Down
9 changes: 4 additions & 5 deletions doc/concept/stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,10 @@ program clock since its last access unit. Its losses are `resyncs`, bytes

Every PID but the null PID either takes a route or is counted. A listed stream
the import cannot carry, such as `0x86` without a SCTE-35 registration, gets a
row with an empty `track` and one `refused`. Once every program's PMT is in, or
has been missing for 0.5 s of program clock, a PID that no PAT, PMT, or
captured SI table claims, such as the CAT, conditional access, or ATSC PSIP,
gets a `data` row with an empty `track` and `refused` counting its packets,
with one warning when it starts. An import of one `--program` of a
row with an empty `track` and one `refused`. Once every program's PMT is in, a
PID that no PAT, PMT, or captured SI table claims, such as the CAT, conditional
access, or ATSC PSIP, gets a `data` row with an empty `track` and `refused`
counting its packets, with one warning when it starts. An import of one `--program` of a
multi-program stream cannot tell another program's PIDs from these, so it
counts none of them.

Expand Down
4 changes: 2 additions & 2 deletions quest/m1/ts-reliability/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ refused means an error or a counted drop.
| DVB SI on 0x0010, 0x0011, 0x0012, 0x0014 | Preserved: snapshot track per PID and `table_id` | landed |
| Program identity, program and ES descriptors | Preserved: descriptors follow each PMT revision; an identity change refuses | landed |
| PMT add, remove, retype; PID reuse | Normalized without a new epoch: a changed or reused PID ends its track and continues under a fresh name, and the export rebuilds the PMT at the next `version_number` | landed |
| PAT changes: moved PMT PID, programs added or removed | Normalized: followed; a removed program retires, and returns as a new epoch. Today ignored, or the whole multiplex stops | [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) |
| PAT changes: moved PMT PID, programs added or removed | Normalized: followed; a removed program retires, and returns as a new epoch. Today a running export keeps the PMT on its first PID | [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) |
| Programs without video | Preserved when the PMT names a PCR PID; refused without one | landed |
| PCR | Normalized: rebased, carried on the first video PID, else audio, else a dedicated PCR-only PID | landed |
| Mux rate, null stuffing | Normalized: re-padded to `muxRate` while the source is CBR | landed |
Expand All @@ -98,7 +98,7 @@ catalog linger (#5305).

## Required

- [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a revised PAT never leaves a broadcast meaning another program
- [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a running export follows a moved PMT PID
- [TS workflow passes](/quest/m1/ts-reliability/validation-green.md) - the scheduled TS lanes pass on main again, with no lane loosened
- [Export SI order](/quest/m1/ts-reliability/export-si-order.md) - a TS export sends SI and media after its PMT and a PCR interval, so a hop keeps first emissions
- [A stitched switch returns to --delay](/quest/m1/ts-reliability/stitch-catch-up.md) - `--stitch` drops the old program's tail once the replacement acquires, so latency settles back to `--delay`
Expand Down
75 changes: 30 additions & 45 deletions quest/m1/ts-reliability/pat-lifecycle.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,36 @@
# [M] Live PAT changes keep programs honest
# [S] A live export follows a moved PMT PID

## Goal

A live TS source whose PAT changes mid-stream keeps every broadcast meaning
one program. A moved PMT PID, a program added, and a program removed or
returning each have one observable outcome: followed, published, retired,
or published as a new epoch. None is silently ignored.
A running `export ts` whose catalog moves `program.pmtPid` follows it: a PAT
at the next `version_number` names the new PID before any PMT goes out
there, and nothing goes out on the old PID after it. The output stays one
program throughout, and no two PATs at one version name different PIDs.

## Plan

What the code does today:

- A PAT naming another PMT PID for the program adds a reassembler but never
drops the old one, and `program.pmtPid` keeps the first.
- `Programs` ignores a program a later PAT adds, without a warning. Its
PIDs hold back the unclaimed-PID count for 0.5 s of program clock, then
count as unclaimed in `ts::stats`, since no program reads them.
- Under `--program all`, the shared parser hands a later PAT that removes one
program to every program's `Import`; the removed program's `handle_pat`
errors, `Programs::decode` propagates it, and the whole multiplex ingest
stops, including unrelated programs. `Programs` sees each whole PAT first
(`Packet::pat`), so it can retire or start programs there instead. A
program still waiting for its PMT is already dropped there, or moved to
the PMT PID the later PAT names.
- A PAT naming another `transport_stream_id`, or a single-program import's
PAT naming another program, already refuses the import.

Decided 2026-10-20 by the maintainer:

- A new PMT PID for the same program moves the reassembler and updates
`program.pmtPid`, which the export follows with a new PAT. `Programs`
publishes a program a later PAT adds as its own broadcast, as it does for
the first PAT, rather than ignoring it.
- A later PAT that removes a program retires that program's broadcast: its
tracks finish and it is unannounced, while the other programs carry on.
If the program returns, it publishes a new epoch of its broadcast, never
the retired instance's names for new content. A single-program import
whose selected program leaves the PAT ends, as today.
- Open: a PAT that names a program whose PMT never arrives. `Programs`
creates a program's broadcast only once its PMT lists a stream, so such a
program waits holding only its PAT entry. Decide whether the import refuses
it instead, and which clock bounds the wait (found in
[moq-dev/moq#6122](https://github.com/moq-dev/moq/pull/6122)).

Rust: `rs/moq-mux` TS `Import::handle_pat` and `Programs`.

Tests use the generated `fixture::Mux` builder: a PMT PID move, and a
`--program all` MPTS whose PAT adds a program, drops one and later restores
it: the other program runs uninterrupted, and the returning one publishes a
new epoch.
The import follows the PAT: a moved PMT PID moves its reassembler and
`program.pmtPid`, and `Programs` publishes, retires and republishes programs
as the PAT adds, drops and restores them. A running export still keeps the
PMT on the PID its tables first went out on, whatever `program.pmtPid` later
records: its output stays consistent, but a move is followed only by an
export started after it.

The maintainer's plan, from closing
[moq-dev/moq#6301](https://github.com/moq-dev/moq/pull/6301): the export
follows a new `program.pmtPid` with a new PAT, built on the table rework
that lays the PAT and PMT from the current table state when their slot is
laid instead of riding queued units, so a move is one more state change
rather than a rewrite of queued copies. That rework has not landed.

The draft then says that `pmtPid` can change in place, and when a consumer
rebuilding the stream moves the PMT.

Rust: `rs/moq-mux` TS export (`pmt_pid`, `build_psi`, the PSI schedule).

Tests: a live export whose catalog moves `pmtPid` emits PATs at versions 0
then 1, with nothing on the new PID before the moving PAT and nothing on the
old one after it, and a move while every copy is still queued goes out as
one PAT naming the new PID. `revision_test`'s
`a_track_recorded_on_the_pmt_pid_after_the_catalog_moves_it_takes_another`
then pins the move instead of the PMT staying put.
41 changes: 20 additions & 21 deletions rs/moq-mux/benches/ts_programs.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
//! `--program all` TS ingest across program count and per-program bitrate, over one second of
//! a generated multiplex. Throughput is the whole input, so a parse that repeats per program
//! shows as a falling rate with program count. A PAT that also names programs no PMT
//! describes shows what a program costs before it starts. Stray PIDs no table lists sweep the
//! shows as a falling rate with program count. A PAT that also names programs whose PMT lists
//! no stream shows what a program costs before it starts. Stray PIDs no table lists sweep the
//! unclaimed-PID accounting, against the same packets sent as null stuffing, which skips it.

use std::time::Duration;
Expand Down Expand Up @@ -54,8 +54,8 @@ fn video_pid(program: u16) -> u16 {
0x0200 + program
}

/// The PMT PID a PAT names for an undescribed program, clear of every PID that carries data.
fn undescribed_pid(program: u16) -> u16 {
/// The PMT PID a PAT names for an idle program, clear of every PID that carries data.
fn idle_pid(program: u16) -> u16 {
0x1000 + program
}

Expand Down Expand Up @@ -142,8 +142,8 @@ impl Mux {
/// What the multiplex carries besides its described programs.
#[derive(Clone, Copy)]
struct Extra {
/// Programs the PAT names whose PMT never arrives.
undescribed: u16,
/// Programs the PAT names whose PMT lists no stream, so they wait without starting.
idle: u16,
/// PIDs no table lists, one packet each per frame.
strays: u16,
/// Send the strays' packets as null stuffing instead.
Expand All @@ -153,16 +153,12 @@ struct Extra {
/// One second of `programs` H.264 programs at `bitrate` bits per second each, interleaved
/// frame by frame, each program carrying its PCR on its video PID, beside `extra`.
fn multiplex(programs: u16, extra: Extra, bitrate: u64) -> Vec<u8> {
let Extra {
undescribed,
strays,
null,
} = extra;
let Extra { idle, strays, null } = extra;
let packets = (bitrate / FPS / 8).div_ceil(188) as usize;
let mut mux = Mux::default();
let entries: Vec<u8> = (1..=programs)
.map(|program| (program, pmt_pid(program)))
.chain((programs + 1..=programs + undescribed).map(|program| (program, undescribed_pid(program))))
.chain((programs + 1..=programs + idle).map(|program| (program, idle_pid(program))))
.flat_map(|(program, pid)| {
let [a, b] = program.to_be_bytes();
let [c, d] = (0xe000 | pid).to_be_bytes();
Expand All @@ -186,6 +182,10 @@ fn multiplex(programs: u16, extra: Extra, bitrate: u64) -> Vec<u8> {
section(0x02, program, 0, 0, &body)
})
.collect();
// No PCR PID, no program info, no stream.
let idle_pmts: Vec<_> = (programs + 1..=programs + idle)
.map(|program| section(0x02, program, 0, 0, &[0xff, 0xff, 0xf0, 0x00]))
.collect();
for frame in 0..FPS {
if frame % PSI_EVERY == 0 {
for pat in &pat {
Expand All @@ -194,6 +194,9 @@ fn multiplex(programs: u16, extra: Extra, bitrate: u64) -> Vec<u8> {
for (program, pmt) in (1..=programs).zip(&pmts) {
mux.psi(pmt_pid(program), pmt);
}
for (program, pmt) in (programs + 1..).zip(&idle_pmts) {
mux.psi(idle_pid(program), pmt);
}
}
let pts = 90_000 + frame * 90_000 / FPS;
for program in 1..=programs {
Expand Down Expand Up @@ -236,27 +239,23 @@ fn programs(c: &mut Criterion) {
group.sample_size(10);
let mut cases = Vec::new();
for programs in [1u16, 4, 16] {
for undescribed in [0u16, 1000] {
for idle in [0u16, 1000] {
for mbps in [1u64, 4] {
let extra = Extra {
undescribed,
idle,
strays: 0,
null: false,
};
let name = match undescribed {
let name = match idle {
0 => format!("{programs}"),
_ => format!("{programs}+{undescribed}"),
_ => format!("{programs}+{idle}"),
};
cases.push((programs, extra, mbps, name));
}
}
for strays in [16u16, 256] {
for (lane, null) in [("null", true), ("stray", false)] {
let extra = Extra {
undescribed: 0,
strays,
null,
};
let extra = Extra { idle: 0, strays, null };
cases.push((programs, extra, 1, format!("{programs}-{lane}{strays}")));
}
}
Expand Down
7 changes: 6 additions & 1 deletion rs/moq-mux/src/container/ts/fixture.rs
Original file line number Diff line number Diff line change
Expand Up @@ -182,11 +182,16 @@ impl Mux {
}

pub(super) fn pat(&mut self, programs: &[(u16, u16)]) -> &mut Self {
self.pat_in(1, programs)
}

/// A [`Self::pat`] naming `transport_stream_id`.
pub(super) fn pat_in(&mut self, transport_stream_id: u16, programs: &[(u16, u16)]) -> &mut Self {
let body: Vec<u8> = programs
.iter()
.flat_map(|&(number, pmt)| [(number >> 8) as u8, number as u8, 0xe0 | (pmt >> 8) as u8, pmt as u8])
.collect();
self.section(0, &psi::section(psi::PAT_TABLE_ID, 1, 0, 0, 0, &body))
self.section(0, &psi::section(psi::PAT_TABLE_ID, transport_stream_id, 0, 0, 0, &body))
}

/// A PMT whose `program_info` holds `info` bytes of private descriptors, so a large one
Expand Down
28 changes: 17 additions & 11 deletions rs/moq-mux/src/container/ts/health.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,19 +61,25 @@ impl Errors {
/// Fold in another checker's count of the same input. Programs sharing a PAT, PMT or PCR PID
/// time the same packets, so each check takes the larger count rather than the sum.
pub(super) fn max(&mut self, other: &Self) {
self.ts_sync_loss = self.ts_sync_loss.max(other.ts_sync_loss);
self.sync_byte_error = self.sync_byte_error.max(other.sync_byte_error);
self.pat_error = self.pat_error.max(other.pat_error);
self.combine(other, u64::max);
}

/// Set each count to `f` of it and `other`'s, check by check and PID by PID.
pub(super) fn combine(&mut self, other: &Self, f: impl Fn(u64, u64) -> u64) {
self.ts_sync_loss = f(self.ts_sync_loss, other.ts_sync_loss);
self.sync_byte_error = f(self.sync_byte_error, other.sync_byte_error);
self.pat_error = f(self.pat_error, other.pat_error);
for (pid, theirs) in &other.pids {
let ours = self.pids.entry(*pid).or_default();
ours.continuity_count_error = ours.continuity_count_error.max(theirs.continuity_count_error);
ours.transport_error = ours.transport_error.max(theirs.transport_error);
ours.pmt_error = ours.pmt_error.max(theirs.pmt_error);
ours.pcr_repetition_error = ours.pcr_repetition_error.max(theirs.pcr_repetition_error);
ours.pcr_discontinuity_indicator_error = ours
.pcr_discontinuity_indicator_error
.max(theirs.pcr_discontinuity_indicator_error);
ours.pts_error = ours.pts_error.max(theirs.pts_error);
ours.continuity_count_error = f(ours.continuity_count_error, theirs.continuity_count_error);
ours.transport_error = f(ours.transport_error, theirs.transport_error);
ours.pmt_error = f(ours.pmt_error, theirs.pmt_error);
ours.pcr_repetition_error = f(ours.pcr_repetition_error, theirs.pcr_repetition_error);
ours.pcr_discontinuity_indicator_error = f(
ours.pcr_discontinuity_indicator_error,
theirs.pcr_discontinuity_indicator_error,
);
ours.pts_error = f(ours.pts_error, theirs.pts_error);
}
}

Expand Down
Loading