diff --git a/doc/bin/cli.md b/doc/bin/cli.md index b42a7222f3b..6bf34381e2f 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -465,7 +465,10 @@ narrow to absorb that one, is logged and counted in `overruns`: the legs can differ there. So can reordered video whose SPS declares no reorder depth (`max_num_reorder_frames`), which is logged, and SI while one leg holds an older revision than the other or around an SI table the catalog adds or drops, which -each leg takes up when its catalog update arrives. +each leg takes up when its catalog update arrives. A track the catalog adds, or +one that ends, revises the PMT and the lanes at the same slot on every leg; a +change to the catalog's program descriptors or `muxRate` alone is logged and +ignored. A stdout export ends with the broadcast. `export ts --linger 10s` waits that long for the same publisher instance to come back, and carries on with the same diff --git a/quest/m1/ts-reliability/README.md b/quest/m1/ts-reliability/README.md index ccbb46d0361..932c670cbfb 100644 --- a/quest/m1/ts-reliability/README.md +++ b/quest/m1/ts-reliability/README.md @@ -72,7 +72,7 @@ refused means an error or a counted drop. | CAT, EMM/ECM, ATSC PSIP, other SI PIDs, unlisted PIDs, TS scrambling | Outside the profile, raw lane; counted when the import reads every program. Today dropped uncounted | [routes](/quest/m1/ts-reliability/stream-routes.md) | | DVB SI on 0x0010, 0x0011, 0x0012, 0x0014 | Preserved: snapshot track per PID and `table_id` | landed | | Program identity, program and ES descriptors | Preserved: descriptors follow each PMT revision; an identity change refuses | landed | -| PMT add, remove, retype; PID reuse | Normalized without a new epoch: a changed or reused PID ends its track and continues under a fresh name, and the export rebuilds the PMT at the next `version_number`. Today `export ts --sync` ends at an added track and announces no other revision | [PMT export](/quest/m1/ts-reliability/pmt-export.md) | +| PMT add, remove, retype; PID reuse | Normalized without a new epoch: a changed or reused PID ends its track and continues under a fresh name, and the export rebuilds the PMT at the next `version_number`; `export ts --sync` at a slot and version every leg derives alike. Today `--sync` ignores a program descriptor or `muxRate` change alone | [PMT export](/quest/m1/ts-reliability/pmt-export.md) | | PAT changes: moved PMT PID, programs added or removed | Normalized: followed; a removed program retires, and returns as a new epoch. Today ignored, or the whole multiplex stops | [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) | | Programs without video | Preserved when the PMT names a PCR PID; refused without one | landed | | PCR | Normalized: rebased, carried on the first video PID, else audio, else a dedicated PCR-only PID | landed | @@ -102,7 +102,7 @@ remain with their current quests. - [State bound](/quest/m1/ts-reliability/state-bound.md) - SI and revision state retire with what the source stops repeating - [Decode order](/quest/m1/ts-reliability/decode-order.md) - TS, MKV and FLV author DTS from decode order and a declared `reorder` depth - [Shared parser](/quest/m1/ts-reliability/shared-parser.md) - `--program all` scans the multiplex once -- [PMT export](/quest/m1/ts-reliability/pmt-export.md) - synced TS legs revise their PMT and grid identically +- [PMT export](/quest/m1/ts-reliability/pmt-export.md) - synced TS legs agree on a program descriptor or `muxRate` change - [Access units](/quest/m1/ts-reliability/pes-access-units.md) - a PES carrying several access units publishes each at its own time - [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a revised PAT never leaves a broadcast meaning another program - [Stream routes](/quest/m1/ts-reliability/stream-routes.md) - every PID gets a declared route, and what is not carried is counted diff --git a/quest/m1/ts-reliability/pmt-export.md b/quest/m1/ts-reliability/pmt-export.md index 026d97b6ccc..dc3e610dfdb 100644 --- a/quest/m1/ts-reliability/pmt-export.md +++ b/quest/m1/ts-reliability/pmt-export.md @@ -1,52 +1,38 @@ -# [M] Synced TS legs revise their PMT identically +# [S] Synced TS legs agree on catalog-only revisions ## Goal -Two `moq export ts --sync` legs of a broadcast whose layout changes -mid-stream revise their PMT identically: the same `version_number`, in the -same slot, listing the same streams, wherever both legs hold the content -around it. A track added no longer ends a synced export. +Under `moq export ts --sync`, a catalog change no stream's media anchors +lands at the same slot on every leg, including one that joins after it, or +is refused loudly. That covers a `programDescriptors` change and a +`muxRate` change. Today each leg logs and ignores them, so a leg that opens +after the change lays different tables or a different grid than the legs +that saw it. ## Plan -Outside `--sync`, the export follows a PMT revision at the next -`version_number`: a stream joins the PMT with its first frame, and leaves -once the catalog has dropped it and nothing of it is left to go out. A track -recorded on the PID of a dropped stream waits for that stream to drain, and -cuts it if it never ends. Each revision is one boundary: versions go out in -order, no packet of a stream a revision adds goes ahead of it, and its PCR -PID and `muxRate` take over after its last packet. The PCR keeps its PID -unless its track retires. - -What `--sync` does today: - -- The tables are laid once, from the catalog the leg opened on. A track - added afterward fails the export ("… under --sync"). -- A track that leaves the catalog keeps its PID and its lane, and an entry - or `programDescriptors` change is not announced. -- A `muxRate` change is warned and ignored: the leg keeps the rate its grid - was laid at. -- So legs that open on either side of a catalog change lay different PMTs - and grids, and a pair does not survive an added stream. - -Decided 2026-10-22 by the maintainer: under `moq export ts --sync`, the -revision is a function of the catalog alone, so identical legs revise -their PMT identically. A revision is one transaction over everything the -catalog configures: a `muxRate` change applies with the tables at the same -boundary and, under `--sync`, rebuilds the grid there. - -Open: where a revision lands. Each leg reads the catalog update at its own -moment, so the slot that carries the revised PMT, and its `version_number`, -must follow from something both legs read alike, such as an added track's -first unit on its lane or the end of a retired track's last group. A -revision that adds a track also needs a lane the rate can carry. - -Rust: `rs/moq-mux` TS export (`with_sync`). - -Tests: two synced legs, joined apart, across a PMT add, remove, retype, -descriptor change and `muxRate` change, write the same packets from the -revision on, PMTs included; a leg that joins after a revision lays the -revised PMT at the same `version_number`, and the revised grid. +A track added, a track that ends, and a PID another stream takes over +already revise the PMT and the lanes at the first table repetition two +slots past the jitter-buffer key of that stream's first or last unit, which +every leg reads alike. The revised `version_number` follows from what the +PMT lists, so a joiner announces it alike. The catalog carries no media +time, so nothing in it says where a catalog-only change would land. + +Options, to settle with the maintainer: + +- The TS importer records where the change took effect, as a media time + (and perhaps the source's PMT `version_number`) in the `mpegts` section. + Export lands it there. A wire change, so the MPEG-TS draft changes too. +- The change rides the next media-anchored revision, as one transaction. A + leg that opens in between differs until then. +- Keep ignoring it, and document that a leg must restart to take it up. + +A `muxRate` change also needs lane positions that survive a rate change: +today every layout counts packets at one rate. + +Also open: a track without a recorded PID takes the next free one in catalog +name order, so a track added under a name that sorts before the others gets +a different PID on a leg that joins later. TS sources record their PIDs. ## Related diff --git a/rs/moq-mux/benches/ts_schedule.rs b/rs/moq-mux/benches/ts_schedule.rs index 565ebd33eed..9a1218fb7b1 100644 --- a/rs/moq-mux/benches/ts_schedule.rs +++ b/rs/moq-mux/benches/ts_schedule.rs @@ -10,6 +10,17 @@ mod export { pub const PSI_INTERVAL: Duration = Duration::from_millis(500); use std::time::Duration; } +#[allow(dead_code)] +mod psi { + /// What a synced slot's tables say, which the schedule only carries. + #[derive(Clone, Debug)] + pub struct Pmt { + pub program_number: u16, + pub pcr_pid: Option, + pub program_info: Vec<()>, + pub streams: Vec<()>, + } +} #[path = "../src/container/ts/lane.rs"] #[allow(dead_code)] mod lane; diff --git a/rs/moq-mux/src/container/ts/export.rs b/rs/moq-mux/src/container/ts/export.rs index b61bc65f93c..55e7a74c587 100644 --- a/rs/moq-mux/src/container/ts/export.rs +++ b/rs/moq-mux/src/container/ts/export.rs @@ -22,7 +22,7 @@ //! length-prefixed -> Annex-B conversion, re-injecting the parameter sets as //! inline NALs on every keyframe. CMAF tracks are rejected with a clear error. -use std::collections::{BTreeMap, HashMap, HashSet, VecDeque}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet, VecDeque}; use std::sync::Arc; use std::task::Poll; use std::time::Duration; @@ -54,7 +54,7 @@ use crate::jitter::{self, Arrival, Push}; use super::adts; use super::catalog; -use super::lane::{self, Grid, Lane, Layout, Need, Unplaced}; +use super::lane::{self, Epoch, Grid, Lane, Lanes, Layout, Need, Run, Unplaced}; use super::psi; use super::schedule::{self, Buffer, Schedule}; @@ -260,8 +260,8 @@ struct Queued { description: Option, /// The PES it may share ([`Track::pack_start`]). pes: Option, - /// Its packets on the track's lane, under [`Export::with_sync`]. - planned: Option, + /// Its entry in the track's plan log, under [`Export::with_sync`]. + unit: Option, } /// A frame's packets, numbered, and their positions on the timeline ([`Track::plan`]). @@ -272,10 +272,275 @@ struct Planned { /// A track's lane under [`Export::with_sync`]. struct Synced { - layout: Arc, + lanes: Arc, lane: Lane, /// The release slot of the last frame planned ([`Track::plan`]). released: Option, + /// The slots its lane runs in, `None` until its first unit is read. + run: Option, + /// The jitter buffer key of its first unit, read before the revision listing it is laid + /// out ([`Export::revise_lanes`]). + anchor: Option, + /// A unit fell past the end of its run, so nothing after it goes out. + ended: bool, + /// Each unit planned and not yet out, in the order planned, so a revision of the lanes + /// re-plans the ones it reaches as a leg that read them after it would. + log: VecDeque, + /// The id of the next entry logged. + next: u64, +} + +/// An entry of [`Synced::log`]. +struct Logged { + id: u64, + /// What the unit was planned from, or `None` for a restart of the timeline, which forgot + /// every group before it. + unit: Option, + /// The lane, and the release slot of the last frame planned, as they were before it. + before: (Lane, Option), + planned: Option, + /// It went out to the grid. + out: bool, +} + +/// What [`Synced::place`] plans a unit from. +struct Unit { + frame: Frame, + dts: Option, + description: Option, + group: (u64, bool), + /// When it may first go out, for a placed frame, which goes out from its placement. + release: Option, + /// When it must have arrived by, in nanoseconds of media time. + by: u128, + /// How many slots ahead of `by` it may go out before its lane bounds it ([`Track::ahead_slots`]). + ahead: u128, + /// Its decoder buffer's size, which bounds that. + size: Option, + /// The room a group's first picture says its HRD has left ([`Timing::room`]). + room: Option, +} + +impl Synced { + fn new(lanes: Arc, run: Option) -> Self { + Self { + lanes, + lane: Lane::default(), + released: None, + run, + anchor: None, + ended: false, + log: VecDeque::new(), + next: 0, + } + } + + /// Log a unit, or a restart with `None`, planned now: its id, and what it placed. + fn log(&mut self, kind: &Kind, pid: u16, name: &str, unit: Option) -> anyhow::Result<(u64, bool)> { + let id = self.next; + self.next += 1; + let before = (self.lane.clone(), self.released); + let planned = self.step(kind, pid, name, unit.as_ref())?; + let placed = planned.is_some(); + self.log.push_back(Logged { + id, + unit, + before, + planned, + out: false, + }); + Ok((id, placed)) + } + + /// Plan `unit` on, or restart the lane for `None`. + fn step(&mut self, kind: &Kind, pid: u16, name: &str, unit: Option<&Unit>) -> anyhow::Result> { + match unit { + Some(unit) => self.place(kind, pid, name, unit), + None => { + self.lane.reset(); + self.released = None; + Ok(None) + } + } + } + + /// Plan every logged unit not yet out again on lanes a revision changed, from the lane as it + /// was before the first. One that went out after them comes out the same, as the revision + /// lies past it. + fn replan(&mut self, kind: &Kind, pid: u16, name: &str) -> anyhow::Result<()> { + let Some(start) = self.log.iter().position(|logged| !logged.out) else { + return Ok(()); + }; + (self.lane, self.released) = self.log[start].before.clone(); + for at in start..self.log.len() { + self.log[at].before = (self.lane.clone(), self.released); + let unit = self.log[at].unit.take(); + let planned = self.step(kind, pid, name, unit.as_ref()); + let logged = &mut self.log[at]; + logged.unit = unit; + if !logged.out { + logged.planned = planned?; + } + } + Ok(()) + } + + /// Forget the entries no revision can reach any more: those out, and those due before + /// slot `next`, the first the grid has still to lay. + fn forget(&mut self, next: u128) { + while let Some(front) = self.log.front() + && (front.out + || front + .unit + .as_ref() + .is_none_or(|unit| schedule::slot(unit.by) + 2 <= next)) + { + self.log.pop_front(); + } + } + + /// Plan a frame on the track's lane: packetized, numbered on from its group's earlier units, + /// at the positions [`Lane::plan`] gives. `None` places nothing: a group this leg did not + /// hold from its first unit, or that opened before its lane, a unit past its lane's end, a + /// malformed section, or a section its lane cannot carry by its deadline. A placed frame's + /// deadline is the delay after its placement, which it goes out from; any other's is its + /// decode time, less the drain. + fn place(&mut self, kind: &Kind, pid: u16, name: &str, unit: &Unit) -> anyhow::Result> { + let Unit { + frame, + dts, + description, + group, + by, + .. + } = unit; + let (group, by) = (*group, *by); + // Listed only once the revision its first unit brings about is laid out. + let Some(run) = self.run else { + return Ok(None); + }; + let rate = self.lanes.payload_rate(pid, schedule::slot(by)); + let release = unit + .release + .unwrap_or_else(|| lane::release(by, filled(unit.ahead, unit.size, rate))); + let video = matches!(kind, Kind::Video(_)); + let opening = self.lane.opening(group); + // A receiver tunes in at the tables, so an AAC program config element leads the first + // frame its group places and the first released on or after each repetition of them. + let slot = schedule::slot(release); + let pce = opening + || self + .released + .is_some_and(|last| last / lane::PSI_SLOTS < slot / lane::PSI_SLOTS); + self.released = Some(slot); + let mut packets = Vec::new(); + match es_payload(kind, description.as_ref(), frame, pce)? { + Some(es) => { + let pes = PesUnit { + pid, + is_video: video, + keyframe: frame.keyframe, + timestamp: frame.timestamp, + dts: *dts, + stream_id: stream_id(kind), + }; + pes_packets(&mut packets, &pes, &es, opening)?; + } + None if let Kind::PesPacket { stream_type } = kind => { + PesPacket::parse(&frame.payload, *stream_type) + .with_context(|| format!("MPEG-TS track '{name}' carries a malformed PES packet"))?; + whole_pes_packets(&mut packets, pid, &frame.payload, opening)?; + } + None if is_complete_section(&frame.payload) => { + section_packets(&mut packets, pid, &frame.payload, opening)?; + } + None => { + tracing::warn!(track = %name, len = frame.payload.len(), "dropping malformed private section on export"); + return Ok(None); + } + } + let count = packets.len() / TsPacket::SIZE; + // A unit is released as early as a full CPB would need it. A group's first picture whose + // buffering period says how much room its encoder's HRD has left when it decodes it goes + // out as much later as its lane takes to carry that room. The HRD received every earlier + // picture before that picture's first bit (ITU-T H.264 C.3), so a lane faster than the + // HRD's rate has by then carried the group before on any leg that holds it. It never goes + // out so late that it would miss its own deadline. + let release = match unit.room { + Some(bits) => self.lanes.later((pid, run), release, bits, by, count), + None => release, + }; + let overruns = self.lane.overruns(); + let plan = self.lane.plan(&self.lanes, (pid, run), group, release, by, count); + if self.lane.overruns() > overruns { + tracing::warn!(track = %name, pid, "--sync group began behind the one before it: its lane is too narrow for legs that lose different data to agree here"); + } + match plan { + Ok(placed) => { + lane::number(&mut packets, placed.counter); + Ok(Some(Planned { + packets, + positions: placed.positions, + })) + } + Err(Unplaced::Partial) => Ok(None), + Err(Unplaced::Ended) => { + self.ended = true; + Ok(None) + } + Err(Unplaced::Late) if matches!(kind, Kind::Verbatim { .. }) => { + tracing::warn!(track = %name, count, "verbatim unit its lane cannot carry by its deadline; dropped"); + Ok(None) + } + Err(Unplaced::Late) => anyhow::bail!( + "MPEG-TS --sync cannot carry a {count}-packet unit on PID {pid} by its deadline on its lane at {} b/s; raise --mux-rate or the delay", + self.lanes.rate() + ), + } + } +} + +/// Under [`Export::with_sync`], the `version_number` of the PMT `pmt`, from `base`: a function of +/// what it says, so a leg that joins after a revision announces it as the legs that saw it do. +/// Each stream adds a term of 1 to 31, so a revision adding or dropping one stream always moves +/// it; the program's descriptors add one too. Where the clock rides is left out: it follows the +/// streams. +fn synced_version(base: VersionNumber, pmt: &psi::Pmt) -> VersionNumber { + // FNV-1a: stable across builds, as every leg has to agree on it. + let hash = |bytes: &[u8]| { + bytes.iter().fold(0xcbf2_9ce4_8422_2325_u64, |hash, &byte| { + (hash ^ u64::from(byte)).wrapping_mul(0x0100_0000_01b3) + }) + }; + let descriptors = |out: &mut Vec, descriptors: &[catalog::Descriptor]| { + for descriptor in descriptors { + out.extend_from_slice(&[descriptor.tag, descriptor.data.len() as u8]); + out.extend_from_slice(&descriptor.data); + } + }; + let mut program = pmt.program_number.to_be_bytes().to_vec(); + descriptors(&mut program, &pmt.program_info); + let mut sum = u64::from(base.as_u8()) + hash(&program) % 32; + for es in &pmt.streams { + let mut entry = vec![es.stream_type]; + entry.extend_from_slice(&es.pid.as_u16().to_be_bytes()); + descriptors(&mut entry, &es.descriptors); + sum += 1 + hash(&entry) % 31; + } + VersionNumber::from_u8((sum % 32) as u8).expect("a version below 32") +} + +/// `slots` ahead, bounded under [`Export::with_sync`] to how far ahead a lane carrying `rate` +/// bits a second fills a decoder buffer of `size` bytes in, less a slot, so the buffer holds +/// whatever the frames' sizes. +fn filled(slots: u128, size: Option, rate: Option) -> u128 { + match (size, rate) { + (Some(size), Some(rate)) => { + let fills = size as u128 * 8 * 1_000_000_000 / u128::from(rate.max(1)); + slots.min(schedule::slot(fills).saturating_sub(1)) + } + _ => slots, + } } /// The PES an audio frame may share ([`Track::pack_start`]): the place of its first frame on @@ -405,8 +670,7 @@ impl Track { self.restart = pending.restart; self.clock = DecodeClock::default(); if let Some(sync) = &mut self.sync { - sync.lane.reset(); - sync.released = None; + sync.log(&self.kind, self.pid, name, None)?; } } let Kind::Video(stream_type) = self.kind else { @@ -450,6 +714,13 @@ impl Track { if self.pending.is_some() || self.finished { return Ok(()); } + // A run another stream's revision cut short is read to the cut, as on a leg that read + // that revision later ([`Export::revise_lanes`]). + if self.sync.as_ref().is_some_and(|sync| sync.ended) { + self.finished = true; + self.release(name, input.jitter, true)?; + return Ok(()); + } let is_video = matches!(self.kind, Kind::Video(_)); // A placed frame is judged at release, against the keyframe's departure ([`Export::leads_video`]). let aligned = !is_video && !self.kind.placed(); @@ -495,6 +766,10 @@ impl Track { break; } self.queue(name, pending, input.jitter)?; + if self.sync.as_ref().is_some_and(|sync| sync.ended) { + self.finished = true; + break; + } } // A cut takes a start group it left short now, not with the next group, which may // arrive after that start was due, or with the stream's end. Under sync the frames @@ -604,31 +879,31 @@ impl Track { self.buffer().unwrap_or(Buffer::SYSTEM).drain() } - /// How far ahead of its decode time the track's units may go out under `delay`: as far - /// as [`schedule::ahead`] lets them, and nothing without a delay or for a placed frame, - /// whose timestamp is when it goes out. - fn ahead(&self, delay: Duration) -> Duration { - PCR_INTERVAL * u32::try_from(self.ahead_slots(delay)).unwrap_or(u32::MAX) + /// How far ahead of its decode time the track's unit decoding at `decode` may go out under + /// `delay`: as far as [`schedule::ahead`] lets them, and nothing without a delay or for a + /// placed frame, whose timestamp is when it goes out. Under [`Export::with_sync`] a unit + /// goes out no further ahead than its lane where it is due fills its decoder buffer in + /// ([`filled`]), as [`Synced::place`] lays it. + fn ahead(&self, delay: Duration, decode: Timestamp) -> Duration { + let slots = self.ahead_slots(delay); + let slots = match &self.sync { + Some(sync) => { + let by = decode.as_nanos().saturating_sub(self.drain().as_nanos()); + let rate = sync.lanes.payload_rate(self.pid, schedule::slot(by)); + filled(slots, self.buffer().and_then(|buffer| buffer.size), rate) + } + None => slots, + }; + PCR_INTERVAL * u32::try_from(slots).unwrap_or(u32::MAX) } - /// [`Self::ahead`] in slots. Under [`Export::with_sync`] a unit goes out no further ahead - /// than its lane fills its decoder buffer in, less a slot, so the buffer holds whatever - /// the frames' sizes. + /// [`Self::ahead`] in slots, before a lane bounds it. fn ahead_slots(&self, delay: Duration) -> u128 { // A placed frame goes out at its placement, never ahead of it. if delay.is_zero() || self.kind.placed() { return 0; } - let buffer = self.buffer(); - let slots = schedule::ahead(buffer.as_ref(), schedule::slot(delay.as_nanos())); - match (&self.sync, buffer.and_then(|buffer| buffer.size)) { - (Some(sync), Some(size)) => { - let rate = u128::from(sync.layout.payload_rate(sync.lane.index()).max(1)); - let fills = size as u128 * 8 * 1_000_000_000 / rate; - slots.min(schedule::slot(fills).saturating_sub(1)) - } - _ => slots, - } + schedule::ahead(self.buffer().as_ref(), schedule::slot(delay.as_nanos())) } /// The lane the track takes under [`Export::with_sync`], in packets a second. Lanes are @@ -676,111 +951,53 @@ impl Track { } } - /// Plan a frame on the track's lane under [`Export::with_sync`]: packetized, numbered on - /// from its group's earlier units, at the positions [`Lane::plan`] gives. `None` places - /// nothing: a group this leg did not hold from its first unit, a malformed section, or - /// a section its lane cannot carry by its deadline. A placed frame's deadline is the delay - /// after its placement, which it goes out from; any other's is its decode time, less the - /// drain. + /// Plan a frame on the track's lane under [`Export::with_sync`] ([`Synced::place`]), logged + /// so a revision of the lanes can plan it again. Returns its log entry if it placed + /// anything, or may once the revision listing the track is laid out. A placed frame's + /// deadline is the delay after its placement, which it goes out from; any other's is its + /// decode time, less the drain. fn plan( &mut self, name: &str, frame: &Frame, (decode, dts): (Timestamp, Option), description: Option<&Bytes>, - group: (u64, bool), + (group, key): ((u64, bool), Timestamp), delay: Duration, - ) -> anyhow::Result> { + ) -> anyhow::Result> { let (release, by) = match self.kind.placed() { // The PCR runs a slot behind the positions ([`Layout::time`]), so a placed unit is // released a slot past its placement to go out no earlier than it. true => ( - decode.as_nanos() + PCR_INTERVAL.as_nanos(), + Some(decode.as_nanos() + PCR_INTERVAL.as_nanos()), decode.as_nanos() + delay.as_nanos(), ), - false => { - let by = decode.as_nanos().saturating_sub(self.drain().as_nanos()); - (lane::release(by, self.ahead_slots(delay)), by) - } + false => (None, decode.as_nanos().saturating_sub(self.drain().as_nanos())), }; - let video = matches!(self.kind, Kind::Video(_)); - let room = match video && group.1 { + let room = match matches!(self.kind, Kind::Video(_)) && group.1 { true => self.timing.room(frame), false => None, }; + let unit = Unit { + frame: frame.clone(), + dts, + description: description.cloned(), + group, + release, + by, + ahead: self.ahead_slots(delay), + size: self.buffer().and_then(|buffer| buffer.size), + room, + }; let Some(sync) = &mut self.sync else { return Ok(None); }; - let opening = sync.lane.opening(group); - // A receiver tunes in at the tables, so an AAC program config element leads the first - // frame its group places and the first released on or after each repetition of them. - let slot = schedule::slot(release); - let pce = opening - || sync - .released - .is_some_and(|last| last / lane::PSI_SLOTS < slot / lane::PSI_SLOTS); - sync.released = Some(slot); - let mut packets = Vec::new(); - match es_payload(&self.kind, description, frame, pce)? { - Some(es) => { - let unit = PesUnit { - pid: self.pid, - is_video: video, - keyframe: frame.keyframe, - timestamp: frame.timestamp, - dts, - stream_id: stream_id(&self.kind), - }; - pes_packets(&mut packets, &unit, &es, opening)?; - } - None if let Kind::PesPacket { stream_type } = self.kind => { - PesPacket::parse(&frame.payload, stream_type) - .with_context(|| format!("MPEG-TS track '{name}' carries a malformed PES packet"))?; - whole_pes_packets(&mut packets, self.pid, &frame.payload, opening)?; - } - None if is_complete_section(&frame.payload) => { - section_packets(&mut packets, self.pid, &frame.payload, opening)?; - } - None => { - tracing::warn!(track = %name, len = frame.payload.len(), "dropping malformed private section on export"); - return Ok(None); - } - } - let count = packets.len() / TsPacket::SIZE; - // A unit is released as early as a full CPB would need it. A group's first picture whose - // buffering period says how much room its encoder's HRD has left when it decodes it goes - // out as much later as its lane takes to carry that room. The HRD received every earlier - // picture before that picture's first bit (ITU-T H.264 C.3), so a lane faster than the - // HRD's rate has by then carried the group before on any leg that holds it. It never goes - // out so late that it would miss its own deadline. - let release = match room { - Some(bits) => sync.layout.later(sync.lane.index(), release, bits, by, count), - None => release, - }; - let overruns = sync.lane.overruns(); - let plan = sync.lane.plan(&sync.layout, group, release, by, count); - if sync.lane.overruns() > overruns { - tracing::warn!(track = %name, pid = self.pid, "--sync group began behind the one before it: its lane is too narrow for legs that lose different data to agree here"); - } - match plan { - Ok(placed) => { - lane::number(&mut packets, placed.counter); - Ok(Some(Planned { - packets, - positions: placed.positions, - })) - } - Err(Unplaced::Partial) => Ok(None), - Err(Unplaced::Late) if matches!(self.kind, Kind::Verbatim { .. }) => { - tracing::warn!(track = %name, count, "verbatim unit its lane cannot carry by its deadline; dropped"); - Ok(None) - } - Err(Unplaced::Late) => anyhow::bail!( - "MPEG-TS --sync cannot carry a {count}-packet unit on PID {} by its deadline on its lane at {} b/s; raise --mux-rate or the delay", - self.pid, - sync.layout.rate() - ), + // Its first unit lists it, at a revision every leg lays out alike. + if sync.run.is_none() && sync.anchor.is_none() { + sync.anchor = Some(key); } + let (id, placed) = sync.log(&self.kind, self.pid, name, Some(unit))?; + Ok((placed || sync.run.is_none()).then_some(id)) } /// Where the jitter buffer orders a frame decoding at `decode` under `delay`: at its release @@ -804,7 +1021,7 @@ impl Track { delay: Duration, frame: &Frame, ) -> anyhow::Result<(Timestamp, Option)> { - let lead = delay.saturating_sub(self.ahead(delay)); + let lead = delay.saturating_sub(self.ahead(delay, decode)); let key = Duration::from(decode) .checked_add(lead) .context("MPEG-TS decode time past the end of the timeline")?; @@ -873,9 +1090,16 @@ impl Track { .unwrap_or(frame.timestamp); let (key, pes) = self.release_key(decode, jitter.delay(), &frame)?; // Planned as it is read, so a frame the jitter buffer drops still moves its group on. - let planned = match self.sync.is_some() { - true => match self.plan(name, &frame, (decode, dts), description.as_ref(), group, jitter.delay())? { - Some(planned) => Some(planned), + let unit = match self.sync.is_some() { + true => match self.plan( + name, + &frame, + (decode, dts), + description.as_ref(), + (group, key), + jitter.delay(), + )? { + Some(unit) => Some(unit), None => return Ok(()), }, false => None, @@ -915,7 +1139,7 @@ impl Track { dts, description, pes, - planned, + unit, }, }; if jitter.push(name.to_string(), arrival)? == Push::Late { @@ -1455,6 +1679,15 @@ where /// of its own, and each PID's continuity counter restarts at every group, its first packet /// flagging the break. Audio is not packed: one frame a PES. /// + /// A track the catalog adds, or one that ends, revises the program at the first slot that + /// repeats the tables after its first or last unit's deadline, the same slot on every leg: the + /// PMT there lists what runs from then on, at a `version_number` that follows from what it + /// lists, so a leg that joins later announces it alike, and the lanes are laid out again + /// for it. An added track goes out from its first group after that slot, and one the catalog + /// lists on a PID another still holds cuts that one there. The program descriptors and the + /// rate stay as the leg opened on, since nothing in the media says where a change to them + /// alone would land; a change is logged. + /// /// Needs a constant rate ([`Self::with_mux_rate`], or the catalog's), and fails at the /// program tables without one, or if the rate cannot carry each track's lane. The lanes /// take a worst case per codec, and the video lane needs headroom over the video's rate. @@ -1677,7 +1910,8 @@ where self.last_timestamp = Some(ready.item.frame.timestamp); let decode = ready.item.decode.as_nanos(); self.lay_before(ready.item.key.as_nanos())?; - match self.list(&name)? { + // Under sync the lanes list a track ([`Self::revise_lanes`]). + match self.sync || self.list(&name)? { true => self.mux(&name, ready.item)?, false => self .tracks @@ -1832,6 +2066,7 @@ where for (name, track) in &mut self.tracks { track.fill(name, waiter, &mut input)?; } + self.revise_lanes()?; Ok(()) } @@ -1843,10 +2078,16 @@ where reject_ambiguous_tracks(&catalog, &mpegts)?; self.source.retain_selectable(&mut catalog); - // Under sync the tables and the grid are laid once, from the catalog the leg opened on. + // Under sync a revision lands where the media that brings it about says, which a change + // to the program descriptors or the rate alone has none of, so they stay as the leg + // opened on. let locked = self.sync && self.psi.is_some(); - if !locked { - self.program_descriptors = mpegts.program_descriptors.clone(); + match locked { + true if mpegts.program_descriptors != self.program_descriptors => tracing::warn!( + "ignoring a program descriptor change under --sync, which keeps the descriptors it opened with" + ), + true => {} + false => self.program_descriptors = mpegts.program_descriptors.clone(), } self.program = mpegts.program.clone(); // An explicit override wins even when it is refused (leaving the output @@ -1940,14 +2181,6 @@ where } if self.psi.is_some() { - if locked { - for name in active.keys() { - anyhow::ensure!( - self.tracks.contains_key(name), - "TS track layout changed after PAT/PMT was emitted under --sync: '{name}' added" - ); - } - } if !self.stale.is_empty() { self.resubscribe(&catalog, &mpegts)?; } @@ -1968,7 +2201,7 @@ where // withdraws leaves nothing behind but what it already sent, which still goes out. A // name that returns before that is gone is the same stream, so it is read on. for (name, track) in self.tracks.iter_mut() { - track.cataloged = locked || active.contains_key(name); + track.cataloged = active.contains_key(name); if track.cataloged && std::mem::take(&mut track.withdrawn) { track.finished = false; } @@ -1986,15 +2219,20 @@ where // fills PIDs, descriptors, and stream_ids across several catalog publishes, so this runs // every snapshot until the PMT lists the track, which is *refreshed*, not latched from // the first (partial) snapshot. A recorded PID may take over one a stream the catalog - // dropped still drains on ([`Self::list`]); any other PID the PMT lists is taken. + // dropped still drains on ([`Self::list`]); any other PID the PMT lists is taken. Under + // sync a leg assigns what one joining now would, which knows neither the PID the clock + // rides nor a stream the catalog dropped: the clock moves off a PID a track takes, and + // a track that takes a PID still draining cuts that stream ([`Self::revise_lanes`]). let pmt_pid = self.pmt_pid(); let mut taken: Vec = vec![0x0000, pmt_pid, 0x1FFF]; - taken.extend(self.psi.as_ref().and_then(|psi| psi.own_clock())); + if !self.sync { + taken.extend(self.psi.as_ref().and_then(|psi| psi.own_clock())); + } taken.extend(self.tracks.values().filter(|t| t.listed && t.cataloged).map(|t| t.pid)); let draining: Vec = self .tracks .values() - .filter(|t| t.listed && !t.cataloged) + .filter(|t| t.listed && !t.cataloged && !self.sync) .map(|t| t.pid) .collect(); let joining: Vec<&String> = active @@ -2020,9 +2258,12 @@ where pids.insert(name.clone(), pid); } } + // Under sync the clock rides its own PID, which the PMT names. + let clock = self.psi.as_ref().and_then(|psi| psi.own_clock()).filter(|_| self.sync); let es_pids: Vec = taken .iter() .chain(&draining) + .chain(&clock) .copied() .filter(|&pid| pid >= 0x10 && pid != 0x1FFF) .collect(); @@ -2110,6 +2351,13 @@ where .filter(|(_, track)| !track.finished) .filter(|(_, track)| track.kind.class() != super::stats::Class::Data || track.buffer.is_some()); self.jitter.expect(continuous.map(|(name, _)| name.clone())); + // A track added under sync plans its first unit on the lanes laid out, which list it at + // the revision that unit brings about. + if let Some(lanes) = self.schedule.grid().map(|grid| grid.lanes().clone()) { + for track in self.tracks.values_mut().filter(|track| track.sync.is_none()) { + track.sync = Some(Synced::new(lanes.clone(), None)); + } + } Ok(()) } @@ -2283,7 +2531,7 @@ where /// The same publisher instance (an equal epoch) continues the stream: the program and /// its PSI carry on, and its catalog resubscribes each track as it lists it, so a gap is a /// skip like any other. A track it adds or drops revises the PMT, as any catalog update - /// does, except under [`Self::with_sync`], where an added one fails as a layout change. + /// does. /// /// Another instance (another epoch, or an epochless route) is a full program switch: a /// fresh export of `broadcast` with a new PMT from its catalog, and nothing carried from @@ -2525,10 +2773,10 @@ where let pcr_pid = match sync { // A media PID flags the break at every group, which on the PCR's PID would declare a // break in the time base, so under sync the PCR rides a PID of its own. - Some(_) => self.clock_pid()?, + Some(_) => self.clock_pid(&self.tracks.values().map(|track| track.pid).collect::>())?, None => self.pick_clock()?, }; - let pmt = self.program_map(pcr_pid)?; + let pmt = self.program_map(&self.listed(), pcr_pid)?; // Preserve the source's program identity so the rebuilt PAT stays consistent with the // carried SI; synthesize a minimal identity otherwise. @@ -2556,11 +2804,16 @@ where queued: VecDeque::new(), out: None, }; - if let Some(rate) = sync { - self.lay_lanes(rate, &psi)?; - } self.psi = Some(psi); self.psi_due = !self.sync; + if sync.is_some() { + // Laid out from each track's run, which every track has from the start. + let lanes = Arc::new(Lanes::default()); + for track in self.tracks.values_mut() { + track.sync = Some(Synced::new(lanes.clone(), Some(Run::WHOLE))); + } + self.lay_lanes()?; + } Ok(()) } @@ -2591,9 +2844,14 @@ where .context("TS export has no unused PID for the PCR") } - /// The PMT listing every listed track in PID order, with the clock on `pcr_pid`. - fn program_map(&self, pcr_pid: u16) -> anyhow::Result { - let mut tracks: Vec<&Track> = self.tracks.values().filter(|t| t.listed).collect(); + /// The tracks the PMT lists. + fn listed(&self) -> Vec<&Track> { + self.tracks.values().filter(|track| track.listed).collect() + } + + /// The PMT listing `tracks` in PID order, with the clock on `pcr_pid`. + fn program_map(&self, tracks: &[&Track], pcr_pid: u16) -> anyhow::Result { + let mut tracks = tracks.to_vec(); tracks.sort_by_key(|t| t.pid); let streams = tracks @@ -2681,7 +2939,7 @@ where true => current, false => self.pick_clock()?, }; - let pmt = self.program_map(pcr_pid)?; + let pmt = self.program_map(&self.listed(), pcr_pid)?; let psi = self.psi.as_mut().expect("checked above"); if pmt != psi.pmt { psi.pmt = pmt; @@ -2726,7 +2984,12 @@ where /// Retire each track the catalog dropped once nothing of it is left to go out, and list /// the tracks that waited for its PID ([`Self::list`]). fn settle(&mut self) -> anyhow::Result<()> { - if self.psi.is_none() || self.sync { + if self.psi.is_none() { + return Ok(()); + } + if self.sync { + self.revise_lanes()?; + self.settle_sync(); return Ok(()); } let gone: Vec = @@ -2788,6 +3051,40 @@ where Ok(()) } + /// Under [`Self::with_sync`], forget each track that ended once the slot its revision retires + /// it in is laid, or that ended before one listed it, and nothing of it waits to go out, and + /// each planned unit no revision can reach any more. + fn settle_sync(&mut self) { + let Some(next) = self.schedule.grid().map(Grid::floor) else { + return; + }; + for sync in self.tracks.values_mut().filter_map(|track| track.sync.as_mut()) { + sync.forget(next); + } + let gone: Vec = self + .tracks + .iter() + .filter(|(name, track)| { + track.finished + && !self.stale.contains(name.as_str()) + && !self.jitter.holds(name.as_str()) + && self.held.as_ref().is_none_or(|held| held.track != **name) + && track.sync.as_ref().is_some_and(|sync| match sync.run { + Some(run) => run.until.is_some_and(|until| until <= next), + None => sync.anchor.is_none(), + }) + }) + .map(|(name, _)| name.clone()) + .collect(); + for name in gone { + let track = self.tracks.remove(&name).expect("collected above"); + self.jitter.forget(name.as_str()); + if track.listed { + self.liveness.retire(track.pid); + } + } + } + /// Hand the program tables to the schedule ahead of the next unit, if they are due. A PMT /// that differs from the last handed over goes out under the next version. fn lead_tables(&mut self) -> anyhow::Result<()> { @@ -2828,66 +3125,158 @@ where Ok(()) } - /// The PID the PCR rides alone under [`Self::with_sync`]: the first above the program's - /// elementary streams that nothing else in it uses, else the first below them, so every + /// The PID the PCR rides alone under [`Self::with_sync`], beside streams on `pids`: the first + /// above them that nothing else in the program uses, else the first below them, so every /// leg picks the same one. - fn clock_pid(&self) -> anyhow::Result { - let used: HashSet = self - .tracks - .values() - .map(|track| track.pid) + fn clock_pid(&self, pids: &[u16]) -> anyhow::Result { + let used: HashSet = pids + .iter() + .copied() .chain([self.pmt_pid()]) .chain(self.si.keys().map(|(pid, _)| *pid)) .collect(); - let top = self - .tracks - .values() - .map(|track| track.pid) - .max() - .unwrap_or(FIRST_ES_PID); + let top = pids.iter().copied().max().unwrap_or(FIRST_ES_PID); (top + 1..0x1FFF) .chain(FIRST_ES_PID..top) .find(|pid| !used.contains(pid)) .context("MPEG-TS --sync found no PID left for the PCR") } - /// Give each track its lane of `rate` under [`Self::with_sync`], beside the tables in - /// `psi` and room for the SI, and lay every slot on them from now on. - fn lay_lanes(&mut self, rate: u64, psi: &Psi) -> anyhow::Result<()> { + /// Under [`Self::with_sync`], put in force the revisions the media brought about since the + /// lanes were laid out: a track's first unit lists it, and its end retires it, at the first + /// slot repeating the tables two slots after that unit's jitter buffer key ([`Lanes::revision`]). + /// Every leg reads that key alike, and reads it before its clock reaches it, while that slot + /// and every unit placed past it is still to go out. A leg that reads it later than that + /// revises at the first such slot it still can. + /// + /// A track listed on a PID another still holds cuts that one at its revision, as a recorded + /// PID taking over a stream the catalog dropped does. That one is read on to the cut. + fn revise_lanes(&mut self) -> anyhow::Result<()> { + if !self.sync { + return Ok(()); + } + let Some(grid) = self.schedule.grid() else { + return Ok(()); + }; + // Nothing laid or placed is revised: the slot after it is the earliest a revision may land + // two slots after. + let floor = grid.floor(); + let revision = |key: Option| { + let slot = key.map_or(0, |key| schedule::slot(key.as_nanos())); + Lanes::revision(slot.max(floor)) + }; + let mut listed = Vec::new(); + let mut changed = false; + for (name, track) in self.tracks.iter_mut() { + let last = track.released.map(|(_, key, _)| key); + let ends = track.finished && !self.stale.contains(name); + let Some(sync) = &mut track.sync else { + continue; + }; + if let Some(key) = sync.anchor.take() { + let from = revision(Some(key)); + sync.run = Some(Run { from, until: None }); + track.listed = true; + listed.push((name.clone(), track.pid, from)); + changed = true; + } + if ends && let Some(run @ Run { until: None, .. }) = &mut sync.run { + run.until = Some(revision(last)); + changed = true; + } + } + for (name, pid, from) in listed { + for (holder, track) in self.tracks.iter_mut() { + if *holder == name || track.pid != pid { + continue; + } + let Some(Some(run)) = track.sync.as_mut().map(|sync| sync.run.as_mut()) else { + continue; + }; + if run.until.is_none_or(|until| until > from) { + run.until = Some(from); + track.withdrawn = false; + } + } + } + if changed { + self.lay_lanes()?; + } + Ok(()) + } + + /// Lay the lanes out under [`Self::with_sync`] from each track's run: from slot zero and + /// every revision on, each listed track a lane of the constant rate, beside the tables and + /// room for the SI. The units planned on lanes a revision changed are planned again. + fn lay_lanes(&mut self) -> anyhow::Result<()> { + let rate = self.mux_rate.context("MPEG-TS --sync lanes without a constant rate")?; + let mut bounds = BTreeSet::from([0]); + for run in self.tracks.values().filter_map(|track| track.sync.as_ref()?.run) { + bounds.insert(run.from); + bounds.extend(run.until); + } + let mut epochs = Vec::new(); + for at in bounds { + let members: Vec<&Track> = self + .tracks + .values() + .filter(|track| { + track + .sync + .as_ref() + .and_then(|sync| sync.run) + .is_some_and(|run| run.holds(at)) + }) + .collect(); + epochs.push((at, self.epoch(rate, &members)?)); + } + let lanes = Arc::new(Lanes::new(epochs)); + let pmt = lanes.last().pmt.1.clone(); + self.psi.as_mut().context("PSI not built")?.pmt = pmt; + for (name, track) in self.tracks.iter_mut() { + let Some(sync) = &mut track.sync else { + continue; + }; + sync.lanes = lanes.clone(); + sync.replan(&track.kind, track.pid, name)?; + if sync.run.is_some() { + self.liveness.register(track.pid); + } + } + match self.schedule.sync() { + Some(grid) => grid.set_lanes(lanes), + None => self.schedule.set_sync(Grid::new(lanes)), + } + Ok(()) + } + + /// The layout of `members` at `rate` under [`Self::with_sync`], with the tables listing them. + fn epoch(&self, rate: u64, members: &[&Track]) -> anyhow::Result { + let psi = self.psi.as_ref().context("PSI not built")?; + let pids: Vec = members.iter().map(|track| track.pid).collect(); + let clock = self.clock_pid(&pids)?; + let pmt = self.program_map(members, clock)?; + let version = synced_version(self.version, &pmt); // Flagged, so each emission has the adaptation field a jump sets the flag in ([`Grid::lay`]). let mut tables = Vec::new(); section_packets(&mut tables, Pid::PAT, &psi.pat, true)?; - section_packets( - &mut tables, - psi.pmt_pid, - &super::psi::encode_pmt(&psi.pmt, self.version)?, - true, - )?; + section_packets(&mut tables, psi.pmt_pid, &super::psi::encode_pmt(&pmt, version)?, true)?; let tables = tables.as_chunks::<{ TsPacket::SIZE }>().0.to_vec(); - let mut needs: Vec<(u16, Need)> = self - .tracks - .values() - .map(|track| (track.pid, track.lane_need())) - .collect(); + let mut needs: Vec<(u16, Need)> = members.iter().map(|track| (track.pid, track.lane_need())).collect(); needs.sort_unstable_by_key(|(pid, _)| *pid); - let layout = Arc::new(Layout::new(rate, tables.len(), SI_LANE, &needs)?); - for track in self.tracks.values_mut() { - let lane = layout.lane(track.pid).context("a track without a lane")?; - track.sync = Some(Synced { - layout: layout.clone(), - lane: Lane::new(lane), - released: None, - }); - } - self.schedule.set_sync(Grid::new(layout, tables)); - Ok(()) + Ok(Epoch { + layout: Layout::new(rate, tables.len(), SI_LANE, &needs)?, + tables, + pmt: (version, pmt), + clock, + }) } /// Packetize one media frame and queue it on the [`Schedule`], re-emitting PAT/PMT /// before video keyframes (and periodically) so receivers can tune in mid-stream. fn mux(&mut self, name: &str, queued: Queued) -> anyhow::Result<()> { if self.sync { - return self.mux_sync(queued); + return self.mux_sync(name, queued); } let Queued { frame, @@ -3067,13 +3456,26 @@ where /// Put a frame's planned packets on their lane under [`Self::with_sync`]. The tables and SI /// have positions of their own ([`Self::queue_si`]), so nothing rides with it. - fn mux_sync(&mut self, queued: Queued) -> anyhow::Result<()> { - let Some(planned) = queued.planned else { + fn mux_sync(&mut self, name: &str, queued: Queued) -> anyhow::Result<()> { + let Some(id) = queued.unit else { return Ok(()); }; - self.schedule.decodes_at(queued.decode.as_nanos()); + let sync = self + .tracks + .get_mut(name) + .and_then(|track| track.sync.as_mut()) + .context("MPEG-TS --sync track without a lane")?; let grid = self.schedule.sync().context("MPEG-TS --sync lanes not laid out")?; - grid.place(&planned.packets, &planned.positions, queued.frame.keyframe); + // Forgotten once due before every slot still to lay ([`Self::settle_sync`]): it could + // only go out late. + let Some(logged) = sync.log.iter_mut().find(|logged| logged.id == id) else { + return Ok(()); + }; + logged.out = true; + if let Some(planned) = &logged.planned { + grid.place(&planned.packets, &planned.positions, queued.frame.keyframe); + self.schedule.decodes_at(queued.decode.as_nanos()); + } Ok(()) } @@ -3393,9 +3795,9 @@ where keyframe: slot.keyframe, }; let psi = self.psi.as_mut().expect("checked above"); - // Under sync the grid lays the one version the tables were built at. + // Under sync the grid lays the tables in force in the slot. let tables = match self.sync { - true => psi.out.is_none().then(|| (self.version, psi.pmt.clone())), + true => slot.pmt, false => (0..slot.tables).filter_map(|_| psi.queued.pop_front()).last(), }; let tally = Tally { diff --git a/rs/moq-mux/src/container/ts/export_timing_test.rs b/rs/moq-mux/src/container/ts/export_timing_test.rs index 1a81554710e..bb7c976a6be 100644 --- a/rs/moq-mux/src/container/ts/export_timing_test.rs +++ b/rs/moq-mux/src/container/ts/export_timing_test.rs @@ -73,6 +73,9 @@ struct Live { stalling: bool, stalled: Vec, audio: Producer, + /// More AAC tracks, each sent beside the first until it ends, with whether it has begun its + /// first group: see [`Self::add_audio`]. + extra: Vec<(Producer, bool)>, source: crate::Source, /// Media time per unit of the receivers' clock: above 1 the source runs fast. scale: f64, @@ -152,6 +155,7 @@ impl Live { stalling: false, stalled: Vec::new(), audio, + extra: Vec::new(), scale, rate: MUX_RATE, sync: false, @@ -329,20 +333,64 @@ impl Live { } } - /// Write the next audio frame. + /// Write the next audio frame, and one to each track added, filled with its mark: 0x55 for + /// the first, counting up. fn send_audio(&mut self) { let index = self.audio_index; + let timestamp = Timestamp::from_micros(self.origin + index * AUDIO_US).unwrap(); + let keyframe = index.is_multiple_of(AUDIO_GROUP); + let frame = |payload: Bytes| Frame { + timestamp, + duration: None, + payload, + keyframe, + }; self.audio - .write(Frame { - timestamp: Timestamp::from_micros(self.origin + index * AUDIO_US).unwrap(), - duration: None, - payload: Bytes::from_iter((0..180u16).map(|i| (i ^ index as u16) as u8)), - keyframe: index.is_multiple_of(AUDIO_GROUP), - }) + .write(frame(Bytes::from_iter((0..180u16).map(|i| (i ^ index as u16) as u8)))) .unwrap(); + for (mark, (track, begun)) in (0x55..).zip(self.extra.iter_mut()) { + if *begun || keyframe { + track.write(frame(Bytes::from(vec![mark; 180]))).unwrap(); + *begun = true; + } + } self.audio_index += 1; } + /// List a second AAC track in the catalog, sent from the next audio group on. Its name sorts + /// after the others, so it takes the next PID on every leg. + fn add_audio(&mut self) { + let track = self + ._broadcast + .create_track( + self._broadcast.unique_name(".aac"), + hang::container::track_info(hang::catalog::PRIORITY.audio), + ) + .unwrap(); + let mut cfg = AudioConfig::new(AAC { profile: 2 }, 48_000, 2); + cfg.container = Container::Legacy; + self.catalog + .modify() + .unwrap() + .audio + .renditions + .insert(track.name().to_string(), cfg); + self.extra.push(( + Producer::new(track, HangContainer::Legacy(crate::container::Kind::Audio)), + false, + )); + } + + /// Drop the last AAC track added from the catalog, and end it unless it runs on. + fn remove_audio(&mut self, ends: bool) { + let (track, _) = self.extra.last_mut().expect("a track to remove"); + self.catalog.modify().unwrap().audio.renditions.remove(track.name()); + if ends { + let (mut track, _) = self.extra.pop().expect("checked above"); + track.finish().unwrap(); + } + } + /// Send audio alone, each frame at its instant, until media time `until` (µs), as a /// broadcast whose video has stopped. async fn run_audio(&mut self, until: u64, legs: &mut [&mut Leg]) { @@ -364,6 +412,9 @@ impl Live { self.stalled.clear(); self.video.finish().unwrap(); self.audio.finish().unwrap(); + for (mut track, _) in self.extra.drain(..) { + track.finish().unwrap(); + } for leg in legs.iter_mut() { leg.drain().await; } @@ -1297,6 +1348,213 @@ async fn sync_refuses_a_rate_its_lanes_do_not_fit() { live.run(2 * GOP, &mut [&mut leg], |_| false).await; } +/// The PID the export gives the PMT of a program that recorded none. +const PMT_PID: u16 = 0x1000; +/// The PID the second AAC track ([`Live::add_audio`]) takes. +const EXTRA_PID: u16 = 0x1003; + +/// A PMT a leg sent: its `version_number`, PCR PID and stream PIDs. +type Pmt = (u8, u16, Vec); + +/// Every PMT copy a leg sent, at the media time (µs) of the slot it ends in. +fn pmts(leg: &Leg) -> Vec<(u64, Pmt)> { + let mut out = Vec::new(); + let mut section = Vec::new(); + for (_, frame) in &leg.out { + for packet in frame.payload.chunks(188) { + if pid(packet) != PMT_PID { + continue; + } + let mut at = 4; + if packet[3] & 0x20 != 0 { + at += 1 + usize::from(packet[4]); + } + if packet[1] & 0x40 != 0 { + section = packet[at + 1 + usize::from(packet[at])..].to_vec(); + } else { + section.extend_from_slice(&packet[at..]); + } + let len = 3 + (usize::from(section[1] & 0x0f) << 8 | usize::from(section[2])); + if section.len() < len { + continue; + } + let whole = std::mem::take(&mut section); + let pmt = super::psi::Pmt::parse(&whole[..len]).expect("a whole PMT"); + let streams = pmt.streams.iter().map(|es| es.pid.as_u16()).collect(); + let pcr = pmt.pcr_pid.map_or(0x1fff, |pid| pid.as_u16()); + out.push(( + frame.timestamp.as_micros() as u64, + ((whole[5] >> 1) & 0x1f, pcr, streams), + )); + } + } + out +} + +/// The media time (µs) of the slot a leg first sent a PMT listing `pid` in, or no longer listing +/// it with `listed` false, after first listing it. +fn revised(leg: &Leg, pid: u16, listed: bool) -> u64 { + let pmts = pmts(leg); + let lists = |streams: &Vec| streams.contains(&pid); + let from = match listed { + true => 0, + false => pmts + .iter() + .position(|(_, (_, _, s))| lists(s)) + .expect("a PMT listing it"), + }; + pmts[from..] + .iter() + .find(|(_, (_, _, s))| lists(s) == listed) + .expect("a revised PMT") + .0 +} + +/// The PMT versions and streams a leg sent from media time `from` (µs), each run of repeats once. +fn pmt_runs(leg: &Leg, from: u64) -> Vec { + let mut runs: Vec = Vec::new(); + for (at, pmt) in pmts(leg) { + if at >= from && runs.last() != Some(&pmt) { + runs.push(pmt); + } + } + runs +} + +/// Under sync a track added mid-stream revises the PMT where every leg revises it: two legs +/// there before it render the same packets across the revision, PMTs included, and a leg that +/// joins after it lays the revised PMT, at the same `version_number`, and the revised grid. +#[tokio::test(start_paused = true)] +async fn synced_legs_revise_their_pmt_alike_for_an_added_track() { + for lag in LAGS { + let mut live = Live::new(1.0, lag).await; + live.sync = true; + let mut a = live.join(Duration::ZERO).await; + live.run(JOIN, &mut [&mut a], |_| false).await; + let mut b = live.join(Duration::from_millis(100)).await; + live.run(6 * GOP, &mut [&mut a, &mut b], |_| false).await; + live.add_audio(); + live.run(10 * GOP, &mut [&mut a, &mut b], |_| false).await; + let mut c = live.join(Duration::from_millis(100)).await; + live.run(16 * GOP, &mut [&mut a, &mut b, &mut c], |_| false).await; + live.finish(&mut [&mut a, &mut b, &mut c]).await; + + let joined = |leg: &Leg| leg.out.first().expect("the leg rendered").1.timestamp.as_micros() as u64; + let from = joined(&b) + 2_000_000; + let at = revised(&a, EXTRA_PID, true); + assert!( + at > from, + "lag {lag}: the revision at {at} µs comes before the legs agree" + ); + assert_eq!(revised(&b, EXTRA_PID, true), at, "lag {lag}"); + assert_eq!(assert_same_synced(&a, &b, from), 0, "lag {lag}"); + assert_eq!(assert_same_synced(&a, &c, joined(&c) + 2_000_000), 0, "lag {lag}"); + let runs = pmt_runs(&a, 0); + assert_eq!(runs.len(), 2, "lag {lag}: {runs:?}"); + assert_ne!(runs[0].0, runs[1].0, "lag {lag}: the revision keeps its version"); + assert_eq!(runs[0].2, [VIDEO_PID, AUDIO_PID], "lag {lag}"); + assert_eq!(runs[1].2, [VIDEO_PID, AUDIO_PID, EXTRA_PID], "lag {lag}"); + assert_eq!(pmt_runs(&c, 0), runs[1..], "lag {lag}: the joiner's tables"); + assert!( + c.units().iter().any(|unit| unit.pid == EXTRA_PID), + "lag {lag}: the added track went out" + ); + } +} + +/// Under sync a track that ends and leaves the catalog leaves the PMT where every leg drops it, +/// and a leg that joins after lays the revised PMT and grid. +#[tokio::test(start_paused = true)] +async fn synced_legs_revise_their_pmt_alike_for_a_removed_track() { + for lag in LAGS { + let mut live = Live::new(1.0, lag).await; + live.sync = true; + // Lanes narrowed by two audio tracks cannot carry the first keyframe by a decode time of zero. + live.origin = 10_000_000; + live.add_audio(); + let mut a = live.join(Duration::ZERO).await; + live.run(JOIN, &mut [&mut a], |_| false).await; + let mut b = live.join(Duration::from_millis(100)).await; + live.run(6 * GOP, &mut [&mut a, &mut b], |_| false).await; + live.remove_audio(true); + live.run(10 * GOP, &mut [&mut a, &mut b], |_| false).await; + let mut c = live.join(Duration::from_millis(100)).await; + live.run(16 * GOP, &mut [&mut a, &mut b, &mut c], |_| false).await; + live.finish(&mut [&mut a, &mut b, &mut c]).await; + + let joined = |leg: &Leg| leg.out.first().expect("the leg rendered").1.timestamp.as_micros() as u64; + let from = joined(&b) + 2_000_000; + let at = revised(&a, EXTRA_PID, false); + assert!( + at > from, + "lag {lag}: the revision at {at} µs comes before the legs agree" + ); + assert_eq!(revised(&b, EXTRA_PID, false), at, "lag {lag}"); + assert_eq!(assert_same_synced(&a, &b, from), 0, "lag {lag}"); + assert_eq!(assert_same_synced(&a, &c, joined(&c) + 2_000_000), 0, "lag {lag}"); + let runs = pmt_runs(&a, 0); + assert_eq!(runs.len(), 2, "lag {lag}: {runs:?}"); + assert_ne!(runs[0].0, runs[1].0, "lag {lag}: the revision keeps its version"); + assert_eq!(runs[1].2, [VIDEO_PID, AUDIO_PID], "lag {lag}"); + assert_eq!(pmt_runs(&c, 0), runs[1..], "lag {lag}: the joiner's tables"); + } +} + +/// Under sync a track the catalog lists on the PID of one it dropped, which runs on, cuts that +/// one where every leg cuts it, and takes the PID over there: the legs render the same packets +/// across it, and the PMT lists the PID once. +#[tokio::test(start_paused = true)] +async fn synced_legs_hand_a_pid_over_alike() { + for lag in LAGS { + let mut live = Live::new(1.0, lag).await; + live.sync = true; + live.origin = 10_000_000; + live.add_audio(); + let mut a = live.join(Duration::ZERO).await; + live.run(JOIN, &mut [&mut a], |_| false).await; + let mut b = live.join(Duration::from_millis(100)).await; + live.run(6 * GOP, &mut [&mut a, &mut b], |_| false).await; + // The next PID free of the streams the catalog lists is the dropped track's. + live.remove_audio(false); + live.add_audio(); + live.run(10 * GOP, &mut [&mut a, &mut b], |_| false).await; + let mut c = live.join(Duration::from_millis(100)).await; + live.run(16 * GOP, &mut [&mut a, &mut b, &mut c], |_| false).await; + live.finish(&mut [&mut a, &mut b, &mut c]).await; + + let joined = |leg: &Leg| leg.out.first().expect("the leg rendered").1.timestamp.as_micros() as u64; + assert_eq!(assert_same_synced(&a, &b, joined(&b) + 2_000_000), 0, "lag {lag}"); + assert_eq!(assert_same_synced(&a, &c, joined(&c) + 2_000_000), 0, "lag {lag}"); + for leg in [&a, &b, &c] { + let runs = pmt_runs(leg, 0); + assert_eq!(runs.len(), 1, "lag {lag}: {runs:?}"); + assert_eq!(runs[0].2, [VIDEO_PID, AUDIO_PID, EXTRA_PID], "lag {lag}"); + } + // The PID carries the dropped track's frames up to the cut, and the new one's after. + let marks = |leg: &Leg| { + let mut marks = Vec::new(); + for (_, frame) in &leg.out { + for packet in frame.payload.chunks(188).filter(|packet| pid(packet) == EXTRA_PID) { + // A PES start: its header, then ADTS, then the frame, filled with its track's mark. + if packet[1] & 0x40 != 0 && packet[3] & 0x10 != 0 { + let at = match packet[3] & 0x20 != 0 { + true => 5 + usize::from(packet[4]), + false => 4, + }; + let pes = &packet[at..]; + let mark = pes[9 + usize::from(pes[8]) + 7]; + if marks.last() != Some(&mark) { + marks.push(mark); + } + } + } + } + marks + }; + assert_eq!(marks(&a), [0x55, 0x56], "lag {lag}"); + } +} + /// Whatever follows the source's clock has to follow it identically on both legs, though they /// joined at different points on it. #[tokio::test(start_paused = true)] diff --git a/rs/moq-mux/src/container/ts/lane.rs b/rs/moq-mux/src/container/ts/lane.rs index a70ca46f61d..7c4debcaea8 100644 --- a/rs/moq-mux/src/container/ts/lane.rs +++ b/rs/moq-mux/src/container/ts/lane.rs @@ -8,6 +8,9 @@ //! packets a second, spread evenly over the rest. The positions no lane owns carry the SI, //! else stuffing. //! +//! A revision of the program puts another layout in force from a slot every leg derives alike +//! ([`Lanes`]): the slot that repeats the tables after the media that brought it about. +//! //! A unit takes the next positions of its PID's lane from where its MoQ group was anchored: //! the first slot the group's first unit may go out in. Where it lands depends on its group's //! earlier units and nothing else, so a leg that lacks a group, or joined after it, lays the @@ -20,9 +23,10 @@ use std::collections::{BTreeMap, HashMap, HashSet, VecDeque}; use std::sync::Arc; -use mpeg2ts::ts::TsPacket; +use mpeg2ts::ts::{TsPacket, VersionNumber}; use super::export::{PCR_INTERVAL, PSI_INTERVAL}; +use super::psi; use super::schedule::{PACKET, packets_before}; /// Grid slots per second. @@ -128,10 +132,6 @@ impl Layout { }) } - pub fn rate(&self) -> u64 { - self.rate - } - /// The lane `pid` owns, if any. pub fn lane(&self, pid: u16) -> Option { self.lanes.iter().position(|&(lane, _)| lane == pid) @@ -220,28 +220,145 @@ impl Layout { } owners } +} - /// `release` moved later by as long as lane `lane` takes to carry `bits` of payload, unless a - /// unit of `count` packets laid from there would arrive after `by`. - pub fn later(&self, lane: usize, release: u128, bits: u64, by: u128, count: usize) -> u128 { - let later = release + u128::from(bits) * 1_000_000_000 / u128::from(self.payload_rate(lane).max(1)); - let positions = self.take(lane, self.at(later), count.max(1)); - let last = *positions.last().expect("at least one position"); - match self.time(last + 1) > by { - true => release, - false => later, +/// One layout of a [`Lanes`], and the program tables it lays. +#[derive(Debug)] +pub(super) struct Epoch { + pub layout: Layout, + /// The PAT then the PMT, flagged, so each emission has the adaptation field a jump sets the + /// flag in ([`Grid::lay`]). + pub tables: Vec<[u8; TsPacket::SIZE]>, + /// The PMT's version and what it says. + pub pmt: (VersionNumber, psi::Pmt), + /// The PID the PCR rides alone. + pub clock: u16, +} + +/// The layouts a constant-rate multiplex runs through, each in force from the first slot of +/// a revision of the program ([`Export::with_sync`](super::Export::with_sync)), on one rate. +/// +/// A revision's slot repeats the tables, which go out revised there, ahead of every packet of +/// the streams it adds: a group opens on a lane only in a slot after the one the lane began in. +/// Its PCR still rides the PID the tables before it named, which the new clock takes over from +/// the next slot on. +#[derive(Debug, Default)] +pub(super) struct Lanes { + /// Each layout by the first slot it lays, the first from slot zero. Empty until laid out. + epochs: Vec<(u128, Epoch)>, +} + +impl Lanes { + /// `epochs` in slot order, the first from slot zero, all at one rate. + pub fn new(epochs: Vec<(u128, Epoch)>) -> Self { + debug_assert!(epochs.first().is_some_and(|(from, _)| *from == 0)); + debug_assert!(epochs.windows(2).all(|pair| pair[0].0 < pair[1].0)); + debug_assert!( + epochs + .iter() + .all(|(_, epoch)| epoch.layout.rate == epochs[0].1.layout.rate) + ); + Self { epochs } + } + + /// The first slot that repeats the tables after `slot`, and at least two after it: a unit due + /// in `slot` may still take positions in the slot after it, whose PCR runs a slot behind. + pub fn revision(slot: u128) -> u128 { + (slot + 2).next_multiple_of(PSI_SLOTS) + } + + fn at_slot(&self, index: u128) -> usize { + self.epochs.partition_point(|(from, _)| *from <= index) - 1 + } + + /// The layout in force in slot `index`, and the slot it came into force. + pub fn epoch(&self, index: u128) -> (u128, &Epoch) { + let (from, epoch) = &self.epochs[self.at_slot(index)]; + (*from, epoch) + } + + /// The layout in force before the one in slot `index` came into force, if any. + fn previous(&self, index: u128) -> Option<&Epoch> { + let at = self.at_slot(index); + (at > 0).then(|| &self.epochs[at - 1].1) + } + + /// The layout of the last revision. + pub fn last(&self) -> &Epoch { + &self.epochs.last().expect("at least one layout").1 + } + + fn layout(&self) -> &Layout { + &self.epochs[0].1.layout + } + + /// The multiplex rate, in bits per second. + pub fn rate(&self) -> u64 { + self.layout().rate + } + + /// See [`Layout::first`]. + pub fn first(&self, index: u128) -> u128 { + self.layout().first(index) + } + + /// See [`Layout::at`]. + pub fn at(&self, time: u128) -> u128 { + self.layout().at(time) + } + + /// See [`Layout::slot_of`]. + pub fn slot_of(&self, position: u128) -> u128 { + self.layout().slot_of(position) + } + + /// See [`Layout::time`]. + pub fn time(&self, position: u128) -> u128 { + self.layout().time(position) + } + + /// The PID slot `index`'s PCR rides: the clock the tables in force before it named. + pub fn clock(&self, index: u128) -> u16 { + let (from, epoch) = self.epoch(index); + match from == index { + true => self.previous(index).unwrap_or(epoch).clock, + false => epoch.clock, + } + } + + /// How fast `pid`'s lane in slot `index` carries elementary bytes, in bits per second. + pub fn payload_rate(&self, pid: u16, index: u128) -> Option { + let layout = &self.epoch(index).1.layout; + Some(layout.payload_rate(layout.lane(pid)?)) + } + + /// `release` moved later by as long as `pid`'s lane in `run` takes to carry `bits` of payload, + /// unless a unit of `count` packets laid from there would arrive after `by`. + pub fn later(&self, (pid, run): (u16, Run), release: u128, bits: u64, by: u128, count: usize) -> u128 { + let Some(rate) = self.payload_rate(pid, self.slot_of(self.at(release))) else { + return release; + }; + let later = release + u128::from(bits) * 1_000_000_000 / u128::from(rate.max(1)); + match self.take((pid, run), self.at(later), count.max(1)) { + Some(positions) if self.time(positions.last().expect("at least one position") + 1) <= by => later, + _ => release, } } - /// The next `count` positions of lane `lane` at or after the timeline's packet `from`. - pub fn take(&self, lane: usize, from: u128, count: usize) -> Vec { + /// The next `count` positions of `pid`'s lane in `run` at or after the timeline's packet + /// `from`, or `None` if the run ends first. + pub fn take(&self, (pid, run): (u16, Run), from: u128, count: usize) -> Option> { let mut out = Vec::with_capacity(count); let mut index = self.slot_of(from); while out.len() < count { + if !run.holds(index) { + return None; + } + let lane = Owner::Lane(self.epoch(index).1.layout.lane(pid)?); let first = self.first(index); for (position, owner) in self.owners(index).into_iter().enumerate() { let at = first + position as u128; - if at >= from && owner == Owner::Lane(lane) { + if at >= from && owner == lane { out.push(at); if out.len() == count { break; @@ -250,7 +367,36 @@ impl Layout { } index += 1; } - out + Some(out) + } + + /// What each position of slot `index` carries ([`Layout::owners`]). + pub fn owners(&self, index: u128) -> Vec { + self.epoch(index).1.layout.owners(index) + } +} + +/// The slots a PID's lane runs in for one stream: from the revision that lists it, up to the one +/// that retires it. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(super) struct Run { + pub from: u128, + pub until: Option, +} + +impl Run { + /// From the start of the timeline on. + pub const WHOLE: Self = Self { from: 0, until: None }; + + /// Whether it runs in slot `index`. + pub fn holds(&self, index: u128) -> bool { + self.from <= index && self.until.is_none_or(|until| index < until) + } + + /// Whether a group may open in slot `index`: one after the slot the run began in, whose + /// tables list the stream, unless it runs from the start. + fn opens(&self, index: u128) -> bool { + self.holds(index) && (self.from == 0 || self.from < index) } } @@ -265,16 +411,18 @@ pub(super) struct Placed { /// Why [`Lane::plan`] placed nothing. #[derive(Debug, PartialEq, Eq)] pub(super) enum Unplaced { - /// This leg did not hold the unit's group from its first unit, so it cannot know where - /// the unit goes. + /// This leg did not hold the unit's group from its first unit, or the group opened before + /// its lane did, so it cannot know where the unit goes. Partial, /// The lane cannot carry the unit by its deadline. Late, + /// The lane ends before the unit would. + Ended, } /// A PID's place in its lane. +#[derive(Clone, Default)] pub(super) struct Lane { - lane: usize, /// The group being laid. group: Option, /// One past the last position any group took. @@ -283,6 +431,7 @@ pub(super) struct Lane { overruns: u64, } +#[derive(Clone)] struct Group { sequence: u64, /// The next position it may take. @@ -294,20 +443,6 @@ struct Group { } impl Lane { - pub fn new(lane: usize) -> Self { - Self { - lane, - group: None, - end: 0, - overruns: 0, - } - } - - /// Which lane of its [`Layout`] it is. - pub fn index(&self) -> usize { - self.lane - } - /// Groups that began behind the one before them, on a lane short of headroom. pub fn overruns(&self) -> u64 { self.overruns @@ -328,28 +463,29 @@ impl Lane { } } - /// Place a unit of `count` packets from group `sequence`, which it `opens` when it is the - /// group's first unit, that may go out from `release` and must have arrived by `by`, - /// both in nanoseconds of media time. + /// Place a unit of `count` packets on `pid`'s lane in `run` from group `sequence`, which it + /// `opens` when it is the group's first unit, that may go out from `release` and must have + /// arrived by `by`, both in nanoseconds of media time. /// /// A group opens at its lane's first position from its first unit's release, and each unit /// after it takes the next positions from there, never before its own release. A unit that /// misses its deadline takes nothing, so the units after it keep theirs. pub fn plan( &mut self, - layout: &Layout, + lanes: &Lanes, + (pid, run): (u16, Run), (sequence, opens): (u64, bool), release: u128, by: u128, count: usize, ) -> Result { - let earliest = layout.at(release); + let earliest = lanes.at(release); let from = match &mut self.group { Some(group) if group.sequence == sequence => match group.whole { true => group.next.max(earliest), false => return Err(Unplaced::Partial), }, - _ if !opens => { + _ if !opens || !run.opens(lanes.slot_of(self.end.max(earliest))) => { self.group = Some(Group { sequence, next: 0, @@ -371,9 +507,9 @@ impl Lane { self.end.max(earliest) } }; - let positions = layout.take(self.lane, from, count.max(1)); + let positions = lanes.take((pid, run), from, count.max(1)).ok_or(Unplaced::Ended)?; let last = *positions.last().expect("at least one position"); - if layout.time(last + 1) > by { + if lanes.time(last + 1) > by { return Err(Unplaced::Late); } let group = self.group.as_mut().expect("a group was just set"); @@ -407,18 +543,19 @@ pub(super) struct Laid { pub units: Vec, /// It does not follow the slot laid before it. pub jump: bool, + /// The PID its PCR rides. + pub clock: u16, + /// The PMT's version and what it says, if the slot sent the tables. + pub tables: Option<(VersionNumber, psi::Pmt)>, } -/// The packets placed on a [`Layout`], laid out slot by slot. +/// The packets placed on [`Lanes`], laid out slot by slot. pub(super) struct Grid { - layout: Arc, + lanes: Arc, /// Each placed packet, by its timeline position. placed: BTreeMap, /// Where each unit's first packet sits: its PID and whether it opens a keyframe. starts: BTreeMap, - /// The PAT then the PMT, numbered as they go out. Each section's first packet carries an - /// adaptation field, whose `discontinuity_indicator` is set where the grid jumps. - tables: Vec<[u8; TsPacket::SIZE]>, /// SI packets still to go out, in order, each no earlier than its position. si: VecDeque<(u128, [u8; TsPacket::SIZE])>, /// Where each SI PID's last packet went. @@ -432,12 +569,11 @@ pub(super) struct Grid { } impl Grid { - pub fn new(layout: Arc, tables: Vec<[u8; TsPacket::SIZE]>) -> Self { + pub fn new(lanes: Arc) -> Self { Self { - layout, + lanes, placed: BTreeMap::new(), starts: BTreeMap::new(), - tables, si: VecDeque::new(), si_last: HashMap::new(), next: None, @@ -446,8 +582,23 @@ impl Grid { } } - pub fn layout(&self) -> &Arc { - &self.layout + pub fn lanes(&self) -> &Arc { + &self.lanes + } + + /// Lay every slot not yet laid on `lanes`, which a revision of the program changed from a + /// slot not yet laid on. + pub fn set_lanes(&mut self, lanes: Arc) { + self.lanes = lanes; + } + + /// The latest slot nothing laid or placed lies two slots past: the next slot to lay, and + /// the one before the last a placed packet takes, whose unit was due in it at the latest. + pub fn floor(&self) -> u128 { + let placed = self.placed.keys().next_back().map(|&last| self.lanes.slot_of(last)); + self.next + .unwrap_or(0) + .max(placed.map_or(0, |slot| slot.saturating_sub(1))) } /// Packets that reached the grid after their position went out. @@ -457,7 +608,7 @@ impl Grid { /// Put a unit's `packets` at their `positions`, dropping any whose position went out. pub fn place(&mut self, packets: &[u8], positions: &[u128], keyframe: bool) { - let gone = self.next.map_or(0, |next| self.layout.first(next)); + let gone = self.next.map_or(0, |next| self.lanes.first(next)); for (n, (packet, &at)) in packets .as_chunks::<{ TsPacket::SIZE }>() .0 @@ -481,7 +632,7 @@ impl Grid { /// carry skips emissions rather than piling up; each emission numbers from zero and flags /// its break, so a skipped one leaves no gap. pub fn push_si(&mut self, index: u128, packets: &[u8]) { - let from = self.layout.first(index); + let from = self.lanes.first(index); let queued: HashSet = self.si.iter().map(|(_, packet)| pid(packet)).collect(); for packet in packets.as_chunks::<{ TsPacket::SIZE }>().0 { if !queued.contains(&pid(packet)) { @@ -505,7 +656,7 @@ impl Grid { /// restarted grid's timeline runs on from the slot laid last. pub fn index(&self) -> Option { self.next.or_else(|| { - let first = self.layout.slot_of(*self.placed.keys().next()?); + let first = self.lanes.slot_of(*self.placed.keys().next()?); Some(match self.last { Some(last) if first + BEHIND > last && first <= last + AHEAD => last + 1, _ => first.saturating_sub(1), @@ -534,17 +685,24 @@ impl Grid { /// /// Tables that start a timeline number as the last repetition before them did, so a /// receiver tuning in on them counts on into the next; after a jump they flag the break - /// from the repetition the stream carried last. + /// from the repetition the stream carried last, as does a revised PMT longer or shorter + /// than the one before it, which numbers on from where its own length would have. pub fn lay(&mut self, index: u128) -> Laid { self.next = Some(index + 1); let jump = self.last.is_some_and(|last| index != last + 1); - let tables = Layout::repeats_tables(index) || jump || self.last.is_none(); + let sends = Layout::repeats_tables(index) || jump || self.last.is_none(); self.last = Some(index); - let first = self.layout.first(index); + let lanes = self.lanes.clone(); + let (from, epoch) = lanes.epoch(index); + let resized = from == index + && lanes + .previous(index) + .is_some_and(|before| before.tables.len() != epoch.tables.len()); + let first = lanes.first(index); // What a restarted grid holds behind the slot it runs on from went out as stuffing. let ahead = self.placed.split_off(&first); self.late += std::mem::replace(&mut self.placed, ahead).len() as u64; - let owners = self.layout.owners(index); + let owners = epoch.layout.owners(index); let count = owners.len(); let mut out = Laid { packets: Vec::new(), @@ -553,24 +711,29 @@ impl Grid { keyframe: false, units: Vec::new(), jump, + clock: lanes.clock(index), + tables: sends.then(|| epoch.pmt.clone()), }; let emission = index / PSI_SLOTS; - let spacing = self.layout.spacing as u128; + let spacing = epoch.layout.spacing as u128; for (position, owner) in owners.into_iter().enumerate() { let at = first + position as u128; let packet = match owner { Owner::Clock => None, - Owner::Table(n) => self.tables.get(n).filter(|_| tables).map(|table| { + Owner::Table(n) => epoch.tables.get(n).filter(|_| sends).map(|table| { let mut packet = *table; // The PAT is one packet; the PMT's packets follow it, numbered on from the // last emission's on the absolute grid. - let counter = match n { - 0 => emission, - n => emission * (self.tables.len() as u128 - 1) + n as u128 - 1, + let (counter, flag) = match n { + 0 => (emission, jump), + n => ( + emission * (epoch.tables.len() as u128 - 1) + n as u128 - 1, + jump || resized, + ), }; packet[3] = (packet[3] & 0xf0) | (counter & 0x0f) as u8; if packet[1] & 0x40 != 0 && packet[3] & 0x20 != 0 && packet[4] > 0 { - packet[5] = (packet[5] & !0x80) | u8::from(jump) << 7; + packet[5] = (packet[5] & !0x80) | u8::from(flag) << 7; } packet }), @@ -629,6 +792,27 @@ mod test { .unwrap() } + /// `layout` with [`tables`], its PCR on `clock`. + fn epoch(layout: Layout, clock: u16) -> Epoch { + let pmt = psi::Pmt { + program_number: 1, + pcr_pid: None, + program_info: Vec::new(), + streams: Vec::new(), + }; + Epoch { + layout, + tables: tables(), + pmt: (VersionNumber::default(), pmt), + clock, + } + } + + /// `layout` in force from the start. + fn lanes(layout: Layout) -> Lanes { + Lanes::new(vec![(0, epoch(layout, 0x1ff))]) + } + #[test] fn every_slot_gives_each_lane_its_share() { let layout = layout(); @@ -657,7 +841,7 @@ mod test { /// the break in their counters as the PCR does; elsewhere they flag nothing. #[test] fn the_tables_flag_a_jump() { - let mut grid = Grid::new(Arc::new(layout()), tables()); + let mut grid = Grid::new(Arc::new(lanes(layout()))); let flagged = |laid: Laid| -> Vec { laid.packets .chunks(TsPacket::SIZE) @@ -681,7 +865,7 @@ mod test { assert_eq!(owners[layout.tables[0]], Owner::Table(0)); assert_eq!(owners[layout.tables[1]], Owner::Table(1)); } - let mut grid = Grid::new(Arc::new(layout), tables()); + let mut grid = Grid::new(Arc::new(lanes(layout))); let mut sent = Vec::new(); for index in 21..=41 { let laid = grid.lay(index); @@ -702,12 +886,12 @@ mod test { /// timeline further off starts afresh. #[test] fn a_grid_restarted_behind_its_last_slot_runs_on() { - let layout = Arc::new(layout()); + let layout = Arc::new(lanes(layout())); let mut packet = [0xaau8; TsPacket::SIZE]; packet[..4].copy_from_slice(&[0x47, 0x41, 0x00, 0x10]); let last = 10_000u128; for (behind, next) in [(3 * u128::from(SLOTS), last + 1), (BEHIND + 1, last - BEHIND - 2)] { - let mut grid = Grid::new(layout.clone(), tables()); + let mut grid = Grid::new(layout.clone()); grid.lay(last); grid.restart(); grid.place(&packet, &[layout.first(last - behind)], false); @@ -758,19 +942,31 @@ mod test { #[test] fn a_group_lays_from_its_own_anchor_whatever_came_before() { - let layout = layout(); + let layout = lanes(layout()); let by = 10_000_000_000; - let mut whole = Lane::new(0); - let mut joiner = Lane::new(0); + let mut whole = Lane::default(); + let mut joiner = Lane::default(); // Group 1, which the joiner lacks. - whole.plan(&layout, (1, true), start(100), by, 50).unwrap(); - whole.plan(&layout, (1, false), start(101), by, 5).unwrap(); - let a = whole.plan(&layout, (2, true), start(110), by, 20).unwrap(); - let b = joiner.plan(&layout, (2, true), start(110), by, 20).unwrap(); + whole + .plan(&layout, (0x100, Run::WHOLE), (1, true), start(100), by, 50) + .unwrap(); + whole + .plan(&layout, (0x100, Run::WHOLE), (1, false), start(101), by, 5) + .unwrap(); + let a = whole + .plan(&layout, (0x100, Run::WHOLE), (2, true), start(110), by, 20) + .unwrap(); + let b = joiner + .plan(&layout, (0x100, Run::WHOLE), (2, true), start(110), by, 20) + .unwrap(); assert_eq!(a.positions, b.positions); assert_eq!((a.counter, b.counter), (0, 0)); - let a = whole.plan(&layout, (2, false), start(110), by, 3).unwrap(); - let b = joiner.plan(&layout, (2, false), start(110), by, 3).unwrap(); + let a = whole + .plan(&layout, (0x100, Run::WHOLE), (2, false), start(110), by, 3) + .unwrap(); + let b = joiner + .plan(&layout, (0x100, Run::WHOLE), (2, false), start(110), by, 3) + .unwrap(); assert_eq!(a.positions, b.positions); assert_eq!((a.counter, b.counter), (4, 4)); assert_eq!((whole.overruns(), joiner.overruns()), (0, 0)); @@ -778,65 +974,180 @@ mod test { #[test] fn groups_released_in_one_slot_keep_their_own_anchors() { - let layout = layout(); + let layout = lanes(layout()); // 48 kHz AAC: frames 21.3 ms apart, two to a slot now and then. let frame = 1_000_000_000 * 1024 / 48_000; - let mut whole = Lane::new(1); - let mut joiner = Lane::new(1); + let mut whole = Lane::default(); + let mut joiner = Lane::default(); let last = start(100) + 2_000_000; - whole.plan(&layout, (1, true), last - frame, u128::MAX, 2).unwrap(); - whole.plan(&layout, (1, false), last, u128::MAX, 2).unwrap(); + whole + .plan(&layout, (0x101, Run::WHOLE), (1, true), last - frame, u128::MAX, 2) + .unwrap(); + whole + .plan(&layout, (0x101, Run::WHOLE), (1, false), last, u128::MAX, 2) + .unwrap(); assert_eq!(layout.slot_of(layout.at(last)), layout.slot_of(layout.at(last + frame))); - let a = whole.plan(&layout, (2, true), last + frame, u128::MAX, 2).unwrap(); - let b = joiner.plan(&layout, (2, true), last + frame, u128::MAX, 2).unwrap(); + let a = whole + .plan(&layout, (0x101, Run::WHOLE), (2, true), last + frame, u128::MAX, 2) + .unwrap(); + let b = joiner + .plan(&layout, (0x101, Run::WHOLE), (2, true), last + frame, u128::MAX, 2) + .unwrap(); assert_eq!(a.positions, b.positions); assert_eq!(whole.overruns(), 0); } #[test] fn a_group_held_from_its_middle_is_not_placed() { - let layout = layout(); - let mut lane = Lane::new(0); + let layout = lanes(layout()); + let mut lane = Lane::default(); assert_eq!( - lane.plan(&layout, (1, false), start(10), u128::MAX, 2).err(), + lane.plan(&layout, (0x100, Run::WHOLE), (1, false), start(10), u128::MAX, 2) + .err(), Some(Unplaced::Partial) ); assert_eq!( - lane.plan(&layout, (1, false), start(10), u128::MAX, 2).err(), + lane.plan(&layout, (0x100, Run::WHOLE), (1, false), start(10), u128::MAX, 2) + .err(), Some(Unplaced::Partial) ); - assert!(lane.plan(&layout, (2, true), start(30), u128::MAX, 2).is_ok()); + assert!( + lane.plan(&layout, (0x100, Run::WHOLE), (2, true), start(30), u128::MAX, 2) + .is_ok() + ); } #[test] fn a_late_unit_leaves_its_positions_to_the_next() { - let layout = layout(); - let mut lane = Lane::new(1); + let layout = lanes(layout()); + let mut lane = Lane::default(); let at = layout.time(layout.first(11)); assert_eq!( - lane.plan(&layout, (1, true), start(10), at, 30).err(), + lane.plan(&layout, (0x101, Run::WHOLE), (1, true), start(10), at, 30) + .err(), Some(Unplaced::Late) ); - let placed = lane.plan(&layout, (1, false), start(10), u128::MAX, 2).unwrap(); + let placed = lane + .plan(&layout, (0x101, Run::WHOLE), (1, false), start(10), u128::MAX, 2) + .unwrap(); assert_eq!(placed.counter, 0); assert!(placed.positions[0] >= layout.first(10)); } #[test] fn a_release_moves_later_only_while_its_unit_meets_its_deadline() { - let layout = layout(); + let layout = lanes(layout()); // Half a second of lane 0's payload. - let bits = layout.payload_rate(0) / 2; - let later = layout.later(0, start(100), bits, start(200), 2); + let bits = layout.payload_rate(0x100, 100).unwrap() / 2; + let later = layout.later((0x100, Run::WHOLE), start(100), bits, start(200), 2); assert!((start(119)..=start(120)).contains(&later), "{later}"); - assert_eq!(layout.later(0, start(100), bits, start(110), 2), start(100)); - assert_eq!(layout.later(0, start(100), 0, start(110), 2), start(100)); + assert_eq!( + layout.later((0x100, Run::WHOLE), start(100), bits, start(110), 2), + start(100) + ); + assert_eq!( + layout.later((0x100, Run::WHOLE), start(100), 0, start(110), 2), + start(100) + ); + } + + /// Lanes for 0x100 and 0x101 from the start, and from slot [`REVISED`] on 0x102 too, its PMT a + /// packet longer, the clock moving from 0x1ff to 0x200. + fn revised() -> Lanes { + let before = Layout::new(RATE, 2, 0, &[(0x100, Need::Rest(u64::MAX)), (0x101, Need::Fixed(422))]).unwrap(); + let needs = [ + (0x100, Need::Rest(u64::MAX)), + (0x101, Need::Fixed(422)), + (0x102, Need::Fixed(40)), + ]; + let mut after = epoch(Layout::new(RATE, 3, 0, &needs).unwrap(), 0x200); + let mut more = after.tables[1]; + more[1] &= !0x40; + after.tables.push(more); + Lanes::new(vec![(0, epoch(before, 0x1ff)), (REVISED, after)]) + } + + /// The slot [`revised`] revises the program in. + const REVISED: u128 = 2 * PSI_SLOTS; + + /// A unit straddling a revision takes its lane's positions on each side of it, and a stream + /// the revision adds opens its first group in a slot after it, behind the tables listing it. + #[test] + fn a_revision_lays_its_lanes_from_its_slot() { + let lanes = revised(); + assert_eq!(Lanes::revision(REVISED - 2), REVISED); + assert_eq!(Lanes::revision(REVISED - 1), REVISED + PSI_SLOTS); + let positions = lanes.take((0x100, Run::WHOLE), lanes.first(REVISED - 1), 40).unwrap(); + for at in positions { + let index = lanes.slot_of(at); + let owner = lanes.epoch(index).1.layout.owners(index)[(at - lanes.first(index)) as usize]; + assert_eq!(owner, Owner::Lane(0), "position {at} in slot {index}"); + } + assert!(lanes.take((0x102, Run::WHOLE), lanes.first(REVISED - 1), 1).is_none()); + + let run = Run { + from: REVISED, + until: None, + }; + let mut lane = Lane::default(); + assert_eq!( + lane.plan(&lanes, (0x102, run), (1, true), start(REVISED), u128::MAX, 1) + .err(), + Some(Unplaced::Partial), + "a group opening in the revision's slot" + ); + let placed = lane + .plan(&lanes, (0x102, run), (2, true), start(REVISED + 1), u128::MAX, 1) + .unwrap(); + assert_eq!(lanes.slot_of(placed.positions[0]), REVISED + 1); + } + + /// A unit that would run past the revision retiring its stream takes nothing. + #[test] + fn a_unit_past_its_run_is_not_placed() { + let lanes = revised(); + let run = Run { + from: 0, + until: Some(REVISED), + }; + let mut lane = Lane::default(); + lane.plan(&lanes, (0x101, run), (1, true), start(REVISED - 2), u128::MAX, 2) + .unwrap(); + assert_eq!( + lane.plan(&lanes, (0x101, run), (1, false), start(REVISED - 1), u128::MAX, 40) + .err(), + Some(Unplaced::Ended) + ); + } + + /// The revision's slot sends the revised tables, its PMT flagging the break in its counter + /// when it runs longer or shorter, under the PCR the tables before it named; the new clock + /// takes over from the slot after. + #[test] + fn a_revision_sends_its_tables_under_the_clock_before_it() { + let mut grid = Grid::new(Arc::new(revised())); + let pmt = |laid: &Laid| -> Vec<(u8, bool)> { + laid.packets + .chunks(TsPacket::SIZE) + .filter(|packet| pid(packet) == 0x1000) + .map(|packet| (packet[3] & 0x0f, packet[1] & 0x40 != 0 && packet[5] & 0x80 != 0)) + .collect() + }; + let before = grid.lay(REVISED - PSI_SLOTS); + assert_eq!((before.clock, pmt(&before)), (0x1ff, vec![(1, false)])); + for index in REVISED - PSI_SLOTS + 1..REVISED { + grid.lay(index); + } + let revised = grid.lay(REVISED); + assert_eq!(revised.clock, 0x1ff); + assert_eq!(pmt(&revised), [(4, true), (5, false)]); + assert_eq!(grid.lay(REVISED + 1).clock, 0x200); } #[test] fn the_grid_lays_tables_and_si_beside_the_lanes() { let layout = Layout::new(RATE, 2, 80, &[(0x100, Need::Rest(u64::MAX)), (0x101, Need::Fixed(422))]).unwrap(); - let mut grid = Grid::new(Arc::new(layout), tables()); + let mut grid = Grid::new(Arc::new(lanes(layout))); let mut si = [0xffu8; TsPacket::SIZE]; si[..4].copy_from_slice(&[0x47, 0x40, 0x11, 0x10]); grid.push_si(40, &[si, si].concat()); @@ -844,7 +1155,7 @@ mod test { for index in 40..44 { let laid = grid.lay(index); assert_eq!(laid.count, 34); - let first = grid.layout().first(index); + let first = grid.lanes().first(index); for (packet, position) in laid .packets .as_chunks::<{ TsPacket::SIZE }>() @@ -855,7 +1166,7 @@ mod test { seen.push((pid(packet), first + *position as u128, packet[3] & 0x0f)); } } - let first = grid.layout().first(40); + let first = grid.lanes().first(40); let on = |pid: u16| -> Vec<(u128, u8)> { seen.iter() .filter(|seen| seen.0 == pid) @@ -864,13 +1175,13 @@ mod test { }; // The PAT and PMT sit at their positions, numbered on from the emissions before them on // the grid. - let tables = &grid.layout().tables; + let tables = &grid.lanes().epoch(40).1.layout.tables; assert_eq!(on(0x0000), [(tables[0] as u128, 2)]); assert_eq!(on(0x1000), [(tables[1] as u128, 2)]); let si = on(0x0011); assert_eq!(si.len(), 2); assert!( - si[1].0 >= si[0].0 + grid.layout().spacing as u128, + si[1].0 >= si[0].0 + grid.lanes().epoch(40).1.layout.spacing as u128, "SI paced like the tables" ); } @@ -878,7 +1189,7 @@ mod test { #[test] fn si_the_free_positions_cannot_carry_does_not_pile_up() { let layout = Layout::new(RATE, 2, 80, &[(0x100, Need::Rest(u64::MAX)), (0x101, Need::Fixed(422))]).unwrap(); - let mut grid = Grid::new(Arc::new(layout), tables()); + let mut grid = Grid::new(Arc::new(lanes(layout))); let mut si = [0xffu8; TsPacket::SIZE]; si[..4].copy_from_slice(&[0x47, 0x40, 0x11, 0x10]); let emission = si.repeat(40); diff --git a/rs/moq-mux/src/container/ts/revision_test.rs b/rs/moq-mux/src/container/ts/revision_test.rs index 39fc48f0e71..5b8b69f101c 100644 --- a/rs/moq-mux/src/container/ts/revision_test.rs +++ b/rs/moq-mux/src/container/ts/revision_test.rs @@ -687,41 +687,6 @@ fn remove(catalog: &mut crate::catalog::Producer, name: &str) { guard.ext.mpegts.tracks.remove(name); } -/// Under sync the tables stay as laid, so a track added after them fails the export. -#[tokio::test(start_paused = true)] -async fn a_track_added_under_sync_fails_the_export() { - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); - let mut catalog = crate::catalog::Producer::new( - &mut broadcast, - crate::catalog::Config::default().with_catalog(crate::catalog::hang::Catalog::::default()), - ) - .unwrap(); - let (_, mut first) = aac_track(&mut broadcast, &mut catalog, Some(AUDIO_PID)); - for millis in (0..3_000).step_by(20) { - write_aac(&mut [&mut first], millis); - } - let source = crate::source::announced(&consumer).await; - let opened = super::export_test::open_ts(source, crate::catalog::CatalogFormat::Hang) - .await - .unwrap(); - let mut export = opened.with_delay(DELAY).with_mux_rate(2_000_000).with_sync(); - let out = tokio::time::timeout(DELAY * 2, export.next()).await; - out.expect("the tables went out").unwrap().unwrap(); - - let _second = aac_track(&mut broadcast, &mut catalog, Some(OTHER_AUDIO_PID)); - let err = tokio::time::timeout(DELAY * 2, async { - loop { - if let Err(err) = export.next().await { - break err; - } - } - }) - .await - .expect("the export reads the catalog update"); - assert!(err.to_string().contains("under --sync"), "{err}"); -} - /// A track the catalog lists after the tables went out, but before its PID is recorded, /// takes the frames it holds along to the recorded PID when that arrives. #[tokio::test(start_paused = true)] diff --git a/rs/moq-mux/src/container/ts/schedule.rs b/rs/moq-mux/src/container/ts/schedule.rs index e0226eed4f4..2b9141b98b6 100644 --- a/rs/moq-mux/src/container/ts/schedule.rs +++ b/rs/moq-mux/src/container/ts/schedule.rs @@ -427,6 +427,8 @@ pub(super) struct Slot { /// How many units of program tables finish in the slot ([`Schedule::lead`]), which go /// out in the order they were pushed. pub tables: usize, + /// On a [`Grid`], the PMT's version and what it says, if the slot sent the tables. + pub pmt: Option<(mpeg2ts::ts::VersionNumber, super::psi::Pmt)>, } impl Slot { @@ -1027,7 +1029,7 @@ impl Schedule { let laid = grid.lay(index); return Ok(Some(Slot { index, - pcr: rate_pcr(index, grid.layout().rate()) as u64, + pcr: rate_pcr(index, grid.lanes().rate()) as u64, nulls: laid.count - 1 - laid.positions.len(), packets: laid.packets, keyframe: laid.keyframe, @@ -1035,8 +1037,9 @@ impl Schedule { positions: Some(laid.positions), clocked: false, jump: laid.jump, - clock: self.clock, + clock: Some(laid.clock), tables: 0, + pmt: laid.tables, })); } let Some(index) = self.index(known) else { @@ -1116,6 +1119,7 @@ impl Schedule { jump: false, clock, tables, + pmt: None, })) } @@ -1140,6 +1144,7 @@ impl Schedule { jump: false, clock: self.clock, tables: 0, + pmt: None, }; let clock = self.clock.filter(|pid| self.buffers.contains_key(pid)); let mut holding = self.holding(|i| self.units[i].remaining()); @@ -2823,6 +2828,7 @@ mod tests { jump: false, clock: None, tables: 0, + pmt: None, }; let mut null = unit(0, 1); (null[1], null[2]) = (0x1f, 0xff); @@ -2864,6 +2870,7 @@ mod tests { jump: false, clock: None, tables: 0, + pmt: None, }; let mut null = unit(0, 1); (null[1], null[2]) = (0x1f, 0xff);