diff --git a/quest/m1/ts-reliability/README.md b/quest/m1/ts-reliability/README.md index c973202be12..511eb691fcb 100644 --- a/quest/m1/ts-reliability/README.md +++ b/quest/m1/ts-reliability/README.md @@ -93,13 +93,19 @@ measurement (decided 2026-10-28). Re-planned the same day: decode order covers M absorbs the release audit's export-reorder quest, and the shared parser lands before the open stream-routes and PAT drafts, which rebase onto it. +Decided 2026-11-03 (maintainer, #7221): the TS export counts an unmarked step +back as a restart only where a reorder bound proves the frame falls outside +valid reordering: audio, which never reorders, and video that steps back past +its catalog `reorder` depth, in both cases by more than the export's delay. +Video that declares no `reorder` is left untouched: no detection, no rewrite, +no error. + PMT fragmentation (#5220), initial mux-rate measurement (#5287), and decoder-time admission (#5306) have landed, as have PSI runs (#5606) and catalog linger (#5305). ## Required -- [A small step back keeps the TS export's video](/quest/m1/ts-reliability/step-back.md) - the export re-acquires on a sub-group step back from any source instead of dropping its video - [Clock-table revisions land at their media position](/quest/m1/ts-reliability/tdt-placement.md) - two exports of one broadcast stay byte-identical across TDT/TOT revisions - [A skipped group flags the next PCR](/quest/m1/ts-reliability/skip-flag.md) - a within-instance skip reaches the wire as a flagged PCR discontinuity - [A republish announces as soon as its tracks are known](/quest/m1/ts-reliability/republish-announce.md) - a stitched export keeps the new instance's first group on a short-GOP feed diff --git a/quest/m1/ts-reliability/step-back.md b/quest/m1/ts-reliability/step-back.md deleted file mode 100644 index d8c65eb6236..00000000000 --- a/quest/m1/ts-reliability/step-back.md +++ /dev/null @@ -1,34 +0,0 @@ -# [S] A small step back keeps the TS export's video - -## Goal - -When a source's decode time steps back by more than the export's delay but -within one group, the TS export re-acquires and keeps rendering video, -instead of dropping nearly every video frame until the next group. - -## Plan - -Decided 2026-10-29 (maintainer), a follow-up of -[moq-dev/moq#7165](https://github.com/moq-dev/moq/pull/7165), which made the -jitter buffer re-acquire on an unmarked step in either direction. - -Found while landing #7165: with 600 ms groups and a 500 ms delay, a −560 ms -step drops 105 of 120 video frames over 8 groups, on main's engine as well. -The export's video decode-time reconstruction clamps DTS to stay -non-decreasing, so only the first frame after the step regresses, and it is -still in time. The rest arrive late without a regression, so the jitter -buffer's backward rule never sees the step. Audio is unaffected. - -The fix belongs where video decode time is reconstructed, not in the jitter -buffer: the step has to reach the jitter buffer as a regression it can act -on. - -Since [moq-dev/moq#7144](https://github.com/moq-dev/moq/pull/7144), a TS or SRT -import turns any rewind into a new epoch, so this remains -for every other source (fMP4, MKV, FLV, RTMP, hang publishers). Kept a -separate quest for that reason. - -Tests: an end-to-end TS export of an H.264 + AAC source with the −560 ms -step, which fails on main and keeps video rendering with the fix. - -Public API: none. Wire: none. diff --git a/quest/m1/ts-small-step-back.md b/quest/m1/ts-small-step-back.md index 839dcdd961a..b03670bfc40 100644 --- a/quest/m1/ts-small-step-back.md +++ b/quest/m1/ts-small-step-back.md @@ -17,14 +17,11 @@ regresses within one clock period, and declared-reorder video fails with declared depth for video. - #7221's `Track::steps_back` (`rs/moq-mux/src/container/ts/export.rs`) is where it goes. -- Check the narrow case #7221 also raised: a slow, deep B-frame stream that - declares no reorder depth must not be mistaken for a step. +- Video that declares no `reorder` stays untouched, so a slow, deep B-frame + stream is never mistaken for a step (decided 2026-11-03, maintainer, in + #7221). - Extend `an_unmarked_step_keeps_the_export_rendering`. -## Required - -- [A small step back keeps the TS export's video](/quest/m1/ts-reliability/step-back.md) - adds `Track::steps_back`, which this extends, in [moq-dev/moq#7221](https://github.com/moq-dev/moq/pull/7221) - ## Related - [TS reliability](/quest/m1/ts-reliability/README.md) - the epic whose step-back fix this extends diff --git a/rs/moq-mux/src/container/declared.rs b/rs/moq-mux/src/container/declared.rs index c5aea9039d5..f60fd4e7956 100644 --- a/rs/moq-mux/src/container/declared.rs +++ b/rs/moq-mux/src/container/declared.rs @@ -70,15 +70,21 @@ impl DeclaredClock { self.held.len() } + /// Whether a frame presented at `pts` would follow more than the depth frames read before it + /// in presentation order, which [`Self::push`] refuses. + pub(crate) fn reorders_past(&self, pts: u64) -> bool { + self.highest.len() - self.highest.partition_point(|&prior| prior <= pts) > self.depth + } + /// Hold the group's next frame in decode order, presented at `pts`. pub(crate) fn push(&mut self, item: T, pts: u64) -> crate::Result<()> { if self.held.len() > self.depth { return Err(ClockError::Undrained.into()); } - let at = self.highest.partition_point(|&prior| prior <= pts); - if self.highest.len() - at > self.depth { + if self.reorders_past(pts) { return Err(ClockError::Reordered(self.depth).into()); } + let at = self.highest.partition_point(|&prior| prior <= pts); self.highest.insert(at, pts); if self.highest.len() > self.depth + 1 { self.highest.remove(0); diff --git a/rs/moq-mux/src/container/ts/export.rs b/rs/moq-mux/src/container/ts/export.rs index bfa1f838447..fafe9ba9400 100644 --- a/rs/moq-mux/src/container/ts/export.rs +++ b/rs/moq-mux/src/container/ts/export.rs @@ -316,6 +316,11 @@ struct Track { /// The marker, skip and cut counts of the sources the track read before this one, so the /// counts keep rising across a resubscription. base: (u64, u64, u64), + /// How many unmarked steps back the track's timeline took ([`Self::steps_back`]), each + /// counted as a marker of its own. + steps: u64, + /// The highest PTS read on the track's timeline, and the marker count it was read under. + high: Option<(u64, Timestamp)>, finished: bool, pid: u16, /// The PMT lists it: from its first frame out until its last has gone and the catalog has @@ -393,6 +398,8 @@ impl Track { latest: kio::clock::Instant::ZERO, restart: 0, base: (0, 0, 0), + steps: 0, + high: None, finished: false, pid: 0, listed: false, @@ -429,11 +436,15 @@ impl Track { jitter: &mut jitter::Buffer, ) -> anyhow::Result<()> { if pending.restart != self.restart { - // A marker declared a break in the timeline, so the decode clock restarts too. + // A marker declared a break in the timeline, or it stepped back unmarked + // ([`Self::steps_back`]), so the decode clock restarts too, and the last keyframe, + // on the timeline left behind, no longer marks what leads it. self.release(name, jitter, true)?; self.restart = pending.restart; self.clock = DecodeClock::default(); self.declared = None; + self.keyframe = None; + self.leading_floor = None; } // MPEG-1/2 video is placed, so its pictures rather than its timestamps say what leads. let stream_type = match self.kind { @@ -512,6 +523,48 @@ impl Track { self.release(name, jitter, false) } + /// Count a frame that steps the track's timeline back with no marker as a marker of its own: + /// presented further below the highest PTS read on the timeline than the export's `delay`, + /// where a reorder bound proves no valid reordering explains it. + /// + /// Audio presents in decode order, so any frame below one read before it is out of order. + /// Video reorders within only the catalog `reorder` its rendition declares, so the frame must + /// also follow more frames than that in presentation order ([`DeclaredClock::reorders_past`]). + /// Video that declares none is left as it is: its decode clock infers reordering from the + /// timestamps, so nothing tells a step back from a reorder deeper than any it has seen, and a + /// guess would restart a healthy stream. + /// + /// The jitter buffer would not see the step: the group's declared clock would refuse the frame + /// and fail the export, and audio with slack left would decode before the frames ahead of it + /// on the same clock, which the output's buffers cannot carry. As a marker, the step restarts + /// the decode clock and opens a new generation, as one the source declared would. A placed + /// frame's timestamp is its arrival rather than its decode time, so it never steps. Without a + /// delay nothing is late, so a step costs nothing there. + fn steps_back(&mut self, pending: &mut Pending, delay: Duration) { + if self.kind.placed() { + return; + } + let pts = pending.frame.timestamp; + let high = (self.high).and_then(|(restart, high)| (restart == pending.restart).then_some(high)); + let delay = u64::try_from(delay.as_micros() * 90_000 / 1_000_000).unwrap_or(u64::MAX); + let below = delay > 0 && high.is_some_and(|high| to_ticks(pts).saturating_add(delay) < to_ticks(high)); + let stepped = below + && match self.kind { + Kind::Video(_) => (self.declared) + .as_ref() + .is_some_and(|clock| clock.reorders_past(to_ticks(pts))), + _ => true, + }; + if stepped { + tracing::debug!(track = %self.key.name, ?pts, ?high, "unmarked step back; restarting the timeline"); + self.steps += 1; + pending.restart += 1; + pending.skip += 1; + } + let high = high.filter(|_| !stepped).map_or(pts, |high| high.max(pts)); + self.high = Some((pending.restart, high)); + } + /// Read within the jitter window, or hold the first frame until the program tables are built. fn fill(&mut self, name: &str, waiter: &kio::Waiter, input: &mut Input<'_>) -> crate::Result<()> { if self.key.name.is_empty() { @@ -544,14 +597,15 @@ impl Track { *input.trimmed += 1; continue; } - let pending = Pending { + let mut pending = Pending { frame, - restart: self.base.0 + self.source.markers(), - skip: self.base.1 + self.source.skips(), + restart: self.base.0 + self.source.markers() + self.steps, + skip: self.base.1 + self.source.skips() + self.steps, cut: self.base.2 + self.source.cuts(), arrived, read: kio::clock::now(), }; + self.steps_back(&mut pending, input.jitter.delay()); // A new timeline must reach the reset before tune-in alignment can drop it. if let Some(start) = input.video_start && aligned && pending.restart == self.restart @@ -4995,6 +5049,111 @@ mod tests { assert_eq!(pat.programs[0].pmt_pid.as_u16(), 0x30); } + /// How many unmarked steps back [`super::Track::steps_back`] counts on video read in decode + /// order, each frame presented at `(ms, keyframe)`, clocked as [`super::Track::queue`] clocks + /// it: on a clock per group at the catalog `reorder` depth, or inferred when it declares none. + async fn video_steps(reorder: Option, frames: &[(u64, bool)]) -> u64 { + use crate::catalog::hang::Container; + use crate::container::{Kind as MediaKind, Producer}; + + let broadcast = moq_net::broadcast::Info::new().produce(); + let publisher = broadcast.create_track("video", hang::container::track_info(0)).unwrap(); + let _publisher = Producer::new(publisher, Container::Legacy(MediaKind::Video)); + let source = crate::source::announced(&broadcast.consume()).await; + let source = super::ExportSource::for_stream(&source, "video", Duration::from_secs(2)).unwrap(); + let mut track = super::Track::new(source, Kind::Video(mpeg2ts::es::StreamType::H264)); + let arrived = kio::clock::now(); + for &(at, keyframe) in frames { + let mut pending = super::Pending { + frame: super::Frame { + timestamp: ms(at), + duration: None, + payload: bytes::Bytes::new(), + keyframe, + }, + restart: track.steps, + skip: track.steps, + cut: 0, + arrived, + read: arrived, + }; + track.steps_back(&mut pending, Duration::from_millis(500)); + if pending.restart != track.restart { + track.restart = pending.restart; + track.clock = DecodeClock::default(); + track.declared = None; + } + if keyframe { + track.declared = reorder.map(super::DeclaredClock::new); + } + let pts = super::to_ticks(ms(at)); + match &mut track.declared { + Some(clock) => { + clock.push(Some((pending, None)), pts).unwrap(); + while clock.pop(false).unwrap().is_some() {} + } + None => track.clock.push((pending, None), pts, arrived, 0), + } + } + track.steps + } + + /// Two groups of a B-pyramid two frames deep, 40 ms a frame, in decode order from `at` ms, its + /// second group stepped back `step` ms after its first `before` frames. + fn pyramid(at: u64, step: u64, before: usize) -> Vec<(u64, bool)> { + const ORDER: [u64; 9] = [0, 4, 2, 1, 3, 8, 6, 5, 7]; + (0..2) + .flat_map(|group| ORDER.map(move |place| (group, place))) + .enumerate() + .map(|(index, (group, place))| { + let pts = at + (group * 9 + place) * 40; + let pts = if index >= 9 + before { pts - step } else { pts }; + (pts, place == 0) + }) + .collect() + } + + /// A rendition that declares its `reorder` proves a frame following more frames than that in + /// presentation order stepped back, while its own reordering is left as it is. + #[tokio::test(start_paused = true)] + async fn a_declared_reorder_bounds_a_step_back() { + assert_eq!( + video_steps(Some(2), &pyramid(10_000, 0, 0)).await, + 0, + "in-bound reordering" + ); + assert_eq!( + video_steps(Some(2), &pyramid(10_000, 560, 4)).await, + 1, + "a step within a group" + ); + assert_eq!( + video_steps(Some(2), &pyramid(10_000, 560, 0)).await, + 1, + "a group stepped whole" + ); + // Two frames back is within the declared depth, however far back they are presented. + let deep = [(10_000, true), (11_000, false), (11_040, false), (10_040, false)]; + assert_eq!( + video_steps(Some(2), &deep).await, + 0, + "in-bound reordering past the delay" + ); + } + + /// Video that declares no `reorder` is left as it is: neither a deep reorder nor a step back + /// can be told apart from the timestamps, so neither restarts the timeline. + #[tokio::test(start_paused = true)] + async fn undeclared_reordering_never_steps_back() { + let slow = [(0, true), (1_600, false), (800, false)]; + assert_eq!(video_steps(None, &slow).await, 0, "reordering past the delay"); + assert_eq!( + video_steps(None, &pyramid(10_000, 560, 4)).await, + 0, + "a step past the delay" + ); + } + /// The same available video/audio backlog must choose the same start in either fill order. #[tokio::test(start_paused = true)] async fn track_fill_observes_every_available_track_before_acquisition() { diff --git a/rs/moq-mux/src/container/ts/export_timing_test.rs b/rs/moq-mux/src/container/ts/export_timing_test.rs index 9d10061505e..c21574ca408 100644 --- a/rs/moq-mux/src/container/ts/export_timing_test.rs +++ b/rs/moq-mux/src/container/ts/export_timing_test.rs @@ -740,20 +740,29 @@ async fn a_track_sent_later_than_the_delay_loses_nothing() { /// A changeover between two encoders whose clocks started hours apart steps the whole timeline /// forward with no marker, mid-group, as in the field. The export re-acquires its clock as at -/// startup, rather than read the step as drift and end, and renders on without a drop. (A step -/// back further than the group before its own never reaches it: the reader refuses it.) +/// startup, rather than read the step as drift and end, and renders on without a drop. So it +/// does for a step back of more than the delay that lands within the group before its own, on +/// video that declares its `reorder`, which proves the step is no reordering, rather than clamp +/// the video onto the old timeline and drop it as late until the timeline catches up, or fail on +/// a frame reordered deeper than declared. Video that declares none is left as it is. (A step +/// back further than that never reaches it: the reader refuses it.) #[tokio::test(start_paused = true)] async fn an_unmarked_step_keeps_the_export_rendering() { - for step in [16_844_500_000, 2_000_000] { + let forward = [16_844_500_000, 2_000_000].map(|step| [(step, false), (step, true)]); + let cases = forward.into_iter().flatten().chain([(-560_000, true)]); + for (step, declared) in cases { for lag in [0, LAGS[0], LAGS[1]] { let mut live = Live::new(1.0, lag).await; + if declared { + live.reorder(); + } let mut leg = live.join(Duration::ZERO).await; live.run(8 * GOP + GOP / 2, &mut [&mut leg], |_| false); live.offset += step; let stepped = kio::clock::now(); live.run(live.tick + 8 * GOP, &mut [&mut leg], |_| false); - let case = format!("{} s step, lag {lag}", step as f64 / 1e6); + let case = format!("{} s step, declared {declared}, lag {lag}", step as f64 / 1e6); assert_eq!(leg.export.dropped(), 0, "{case}"); let video = leg .units()