Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions doc/concept/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,9 +170,10 @@ service. The root `m2ts` section names the one track holding it:

Each frame is a run of whole 188-byte packets in input order, timestamped on
the multiplex's own PCR clock, which never steps back: a source whose clock
steps back is a new broadcast. `randomAccess` says whether every group begins
where a receiver can start decoding, and `muxRate` is the constant multiplex
rate in bits per second, absent for a variable-rate source. A player has
steps back is a new broadcast. `randomAccess` says whether every group of the
latest two-second window began where a receiver can start decoding, and
`muxRate` is that window's constant multiplex rate in bits per second, absent
for a variable-rate source. A player has
nothing to decode here. `moq import ts --passthrough` writes this section (see
[moq-cli](/bin/cli)); the encoding is in
[draft-lcurley-moq-mpegts](/draft/moq-mpegts).
Expand Down
21 changes: 13 additions & 8 deletions drafts/draft-lcurley-moq-mpegts.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,10 +278,10 @@ type M2ts = {
`track` names the one track carrying the multiplex, in the same broadcast as the catalog.
A second multiplex is a second broadcast.

`randomAccess` says that every group starts at a random access point ({{m2ts-groups}}).
`randomAccess` says that every group of the latest window the section describes started at a random access point ({{m2ts-groups}}).
A publisher MUST write it, and a consumer MUST read an absent `randomAccess` as false.

`muxRate` follows {{field-mux-rate}}, measured over every packet of the track.
`muxRate` follows {{field-mux-rate}}, measured over every packet of that window.

The section records no program, PID, descriptor, or table: they all reach the consumer inside the packets, as the publisher received them.
A track named here MUST NOT also be described as a rendition or in an `mpegts` section.
Expand All @@ -302,11 +302,15 @@ A group starts at:

- a packet with `random_access_indicator` set on the video PID of the clock's program (the first PID its PMT declares with a video `stream_type`), or on the clock PID when that program has no video;
- a PCR on the clock PID with `discontinuity_indicator` set that steps forward ({{m2ts-timestamps}}), since a group MUST NOT contain a PCR discontinuity;
- otherwise, the first PCR at least one second of PCR time after the group's start;
- the first PCR at or past a whole second of the PCR count ({{m2ts-timestamps}}), unless a random access point or a forward jump fell in the second before it, each counted at the PCR before it (its own, if its packet carries one);
- the frame that would take a group past a size bound the publisher chooses below what a relay caches of one group; frames are never split to meet it.

`randomAccess` is true only while the PAT lists one program, its PMT declares at most one video stream, and the clock PID is that PMT's `PCR_PID`, and only if every group so far has started at a `random_access_indicator`.
The first group that starts any other way, or that carries packets while that layout does not hold, republishes the section with `randomAccess` false for the rest of the track.
Before its first group, a publisher starts one at a second only once it has watched from a PCR before the second before it, with the PMT of the clock's program read, so it never cuts where a publisher running longer saw a random access point or a jump (the first PCR it saw may carry a flag it missed).

The section describes two-second windows of the PCR count, each holding the PCR intervals that start in it.
As the count passes each window, from the first that starts after the PCR the publisher's first group follows, the section takes that window's values; a window whose values match the last one's leaves the catalog unchanged.
An input that ends before its first such window takes `randomAccess` from every group it carried and `muxRate` from the window in progress.
For a window, `randomAccess` is true only if the PAT listed one program, its PMT declared at most one video stream, and the clock PID was that PMT's `PCR_PID` throughout it, and every group it started began at a `random_access_indicator`.
With more programs or video streams the random access points can stagger, so a group start is not one for all of them.

Both indicators sit in the adaptation field, which TS-level scrambling leaves in the clear, so a scrambled multiplex groups like its clear twin.
Expand All @@ -318,11 +322,11 @@ A PCR gives the time of the byte holding the last bit of its `program_clock_refe
A byte between two PCRs is placed at the rate between them; a byte past the last PCR of a timebase takes the rate of the interval before it, and one before the first PCR of a timebase takes the rate of the interval after it.

PCRs are unwrapped into a 64-bit count of 27 MHz ticks, and timestamps never decrease.
A step forward of at most half the 2^33 × 300 tick period adds to the count, across the wrap included, so a wrap is not a discontinuity.
A PCR with `discontinuity_indicator` set whose step is forward by that measure adds its step too, after the bytes before it are timed at the rate before it, and the group it starts MUST follow hang's discontinuity marker ({{hang}}).
Each PCR takes the number of 2^33 × 300 tick wraps that lands it nearest the current time on the catalog clock ({{hang}}), so the count runs on across the wrap, which is not a discontinuity, and a PCR more than half a wrap ahead of that time lands a wrap back.
A PCR with `discontinuity_indicator` set that lands at or past the one before it adds its step too, after the bytes before it are timed at the rate before it, and the group it starts MUST follow hang's discontinuity marker ({{hang}}).
A PCR that steps backwards MUST end the track with an error, flagged or not: a flagged one is a restart, which is new content, so a publisher that carries on publishes it as a new broadcast.

The timeline depends only on the bytes since the publisher's first PCR, so two publishers that start at the same packet of one input publish identical frames.
The count therefore depends only on the bytes and the catalog clock, so publishers of one input whose catalog clocks agree publish identical frames, group boundaries, and sections wherever each started, from the later one's first group on.

## Mapping onto MSFTS {#m2ts-msfts}
Stripping each frame's timestamp leaves an {{msfts}} object of whole source packets, and the groups carry over unchanged.
Expand Down Expand Up @@ -443,6 +447,7 @@ A broadcast demultiplexed from a DVB transport stream: video and audio described
- Added the `m2ts` section, a transport stream carried whole, with its mapping onto MSFTS; the comparison uses MSFTS's `mpeg2ts` names.
- Added the `pesPacket` record: each frame one whole PES packet, timestamped with its placement. A consumer that predates it drops the track rather than refusing the catalog.
- A flagged PCR jump on an `m2ts` track follows hang's discontinuity marker rather than a skipped group sequence.
- An `m2ts` track's fallback cut falls on whole seconds of a PCR count whose wraps follow the catalog clock, and `randomAccess` and `muxRate` describe the latest two-second window rather than the whole track, so publishers sharing a clock cut and describe the multiplex alike.

## draft-lcurley-moq-mpegts-00
{:numbered="false"}
Expand Down
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ ranked just below deterministic TS legs, which it requires.
- [Release execution](/quest/m2/release/README.md) - publication, installed-artifact proof, cleanup, and downstream adoption after m1
- [P2P](/quest/m2/p2p/README.md) - application-selected peer sessions use shared priorities, routing, and same-epoch handover
- [Epoch anchor](/quest/m2/epoch-anchor.md) - same-epoch importers each derive the catalog clock and timestamp era from the epoch and stay live, so they agree across a 33-bit wrap, a forward clock break, and a restart
- [Raw TS cuts on the epoch timeline](/quest/m2/ts-passthrough-epoch-cuts.md) - passthrough fallback cuts fall on the epoch's grid and the M2ts section derives from the epoch and content, so instances started apart publish identical groups and catalogs
- [Same-epoch importers](/quest/m2/hop-aligned-import.md) - importers sharing one `--epoch` and fed one stream publish identical groups and timestamps, so failover between a redundant pair survives
- [Hard-fork the binding generator](/quest/m2/uniffi-fork.md) - one moq-dev repo holds the chosen generator's core and every language backend, released under one tag and one pin
- [Benchmark trend site](/quest/m2/bench-site.md) - a GitHub Pages site charts the binding, Rust, and JS nightly benchmarks over time
Expand Down
6 changes: 2 additions & 4 deletions quest/m2/epoch-anchor.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,8 @@ precondition.
`Clock::wall_clock` never enters it.
- Every catalog field an importer derives from its history rather than the
content at hand derives from the epoch and the content instead (maintainer,
superseding #5412's adoption from the live catalog). In TS passthrough that
is the M2ts section's `random_access` and recorded `mux_rate`, owned by
[raw TS cuts](/quest/m2/ts-passthrough-epoch-cuts.md); audit the other
importers for more.
superseding #5412's adoption from the live catalog). TS passthrough's M2ts
section already does; audit the other importers for more.
- A restart under the same epoch re-derives the same coordinate, including
after a total outage, so the epoch keeps naming one timeline and resuming
across the outage is a gap, not a stitch (maintainer, reversing #5412's
Expand Down
1 change: 0 additions & 1 deletion quest/m2/hop-aligned-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ stopping and restarting; repeat it with `--passthrough`.
## Required

- [Epoch anchor](/quest/m2/epoch-anchor.md) - every importer derives one catalog clock and timestamp era from the epoch
- [Raw TS cuts on the epoch timeline](/quest/m2/ts-passthrough-epoch-cuts.md) - passthrough groups start where every instance cuts

## Related

Expand Down
67 changes: 0 additions & 67 deletions quest/m2/ts-passthrough-epoch-cuts.md

This file was deleted.

9 changes: 5 additions & 4 deletions rs/hang/src/catalog/m2ts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,18 @@ pub struct M2ts {
/// The track carrying the packets, published in the same broadcast as the catalog.
pub track: String,

/// Whether every group starts at a random access point of the multiplex's one video stream,
/// so a subscriber can decode from the first object of any group.
/// Whether every group the publisher's latest two-second window started began at a random
/// access point of the multiplex's video stream, or of its clock PID when it has none, so a
/// subscriber can decode from the first object of any of them.
Comment thread
kixelated marked this conversation as resolved.
Resolved
///
/// Only ever true for a single program with at most one video stream: programs or video
/// streams sharing a clock can stagger their GOPs, leaving no packet that starts all of them.
/// Always written; a reader treats it as false when absent.
#[serde(default)]
pub random_access: bool,

/// The rate the source's PCR paced the whole multiplex at, in bits per second, null packets
/// included. Present only while the source holds a constant rate.
/// The rate the source's PCR paced the whole multiplex at over that window, in bits per
/// second, null packets included. Present only while the source holds a constant rate.
#[serde(default)]
pub mux_rate: Option<u64>,
}
Expand Down
50 changes: 29 additions & 21 deletions rs/moq-cli/src/publish.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1187,24 +1187,32 @@ pub(crate) mod tests {
#[tokio::test]
async fn ts_passthrough_crosses_a_relay_through_a_flagged_jump() {
const KYRION: &[u8] = include_bytes!("../../moq-mux/src/container/ts/test_data/kyrion_mpeg2av_ac3.ts");
// The capture twice, the second copy's clock twenty seconds ahead of the first, its first
// PCR flagged as the discontinuity. Inside the track's max age, so the recording keeps
// every group.
const STEP: u64 = 20 * 90_000;
let mut second = KYRION.to_vec();
let mut pcrs = second
.as_chunks_mut::<188>()
.0
.iter_mut()
.filter(|pkt| pkt[3] & 0x20 != 0 && pkt[4] > 0 && pkt[5] & 0x10 != 0)
.peekable();
pcrs.peek_mut().expect("a PCR")[5] |= 0x80;
for pkt in pcrs {
let base = pkt[6..11].iter().fold(0u64, |base, byte| base << 8 | u64::from(*byte)) >> 7;
let base = ((base + STEP) & ((1 << 33) - 1)) << 7 | u64::from(pkt[10] & 0x7f);
pkt[6..11].copy_from_slice(&base.to_be_bytes()[3..]);
}
let input = [KYRION, &second].concat();
// The capture with its clock `seconds` ahead, its first PCR flagged as a discontinuity
// when `flagged`.
let shifted = |seconds: u64, flagged: bool| {
let mut copy = KYRION.to_vec();
let mut pcrs = copy
.as_chunks_mut::<188>()
.0
.iter_mut()
.filter(|pkt| pkt[3] & 0x20 != 0 && pkt[4] > 0 && pkt[5] & 0x10 != 0)
.peekable();
if flagged {
pcrs.peek_mut().expect("a PCR")[5] |= 0x80;
}
for pkt in pcrs {
let base = pkt[6..11].iter().fold(0u64, |base, byte| base << 8 | u64::from(*byte)) >> 7;
let base = ((base + seconds * 90_000) & ((1 << 33) - 1)) << 7 | u64::from(pkt[10] & 0x7f);
pkt[6..11].copy_from_slice(&base.to_be_bytes()[3..]);
}
copy
};
// The capture, carried straight on by a copy four seconds ahead so a whole catalog window
// follows the first group, then a copy twenty seconds past that one behind a flagged
// discontinuity. Inside the track's max age, so the recording keeps every group.
let lead = [KYRION, &shifted(4, false)].concat();
let second = shifted(24, true);
let input = [&lead[..], &second].concat();

let _env = crate::test_env::EnvGuard::clear(&["MOQ_CONNECT", "MOQ_HOP"]);
let _ = moq_tokio::crypto::install_default();
Expand Down Expand Up @@ -1263,10 +1271,10 @@ pub(crate) mod tests {
.await
.unwrap();

// The catalog names the track once the first group is out, which the first copy holds;
// the jump is in the second.
// The catalog names the track once a whole window has followed the first group, which
// the lead holds; the jump is in the copy after it.
const WAIT: Duration = Duration::from_secs(10);
decoder.decode_chunk(KYRION).unwrap();
decoder.decode_chunk(&lead).unwrap();
let consumer = tokio::time::timeout(WAIT, remote.consume().routed_broadcast("pt.hang"))
.await
.unwrap()
Expand Down
Loading
Loading