Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ folded into Dropped sources.
- [Lazy discovery](/quest/m1/lazy-discovery/README.md) - a `consume` session requests announcements only for prefixes the origin is watching, in JS then Rust; `follow`, `broadcasts`, and the `discovery` option go away
- [A spinning loop fails a sim test](/quest/m1/spinning-loops.md) - a sim test names any loop that holds a task poll too long, and each one yields through a budget
- [Kotlin wrapper POMs](/quest/m1/kt-ffi-pom.md) - Maven builds of `dev.moq:moq` resolve a published moq-ffi instead of the missing `0.0.0-dev`
- [Plan: redundant TS alignment](/quest/m1/plan-ts-alignment.md) - settle shared source coordinates and distinguish TS identity from coordinated RTP recovery
- [Same-epoch importers](/quest/m1/hop-aligned-import.md) - importers sharing one `--epoch` and fed one stream publish identical groups and timestamps, so failover between a redundant pair survives
- [moq.sh deploys from CI](/quest/m1/moq-sh-deploy.md) - the first `release` run of the moq.sh workflow deploys with the Workers Editor token
- [Plan: untimed verbatim PES](/quest/m1/plan-ts-pes-untimed.md) - decide how a verbatim TS track carries a PES that has no PTS, then write the implementation quest
Expand Down Expand Up @@ -162,7 +163,8 @@ folded into Dropped sources.
- [TTML caption import](/quest/m1/captions-ttml.md) - fMP4 `stpp` tracks within a supported TTML/IMSC profile import as `ttml` renditions on the shared clock instead of being skipped
- [fMP4 WebVTT ctim and vtta](/quest/m1/wvtt-ctim-vtta.md) - `wvtt` samples with `ctim` cue times or `vtta` comments import instead of failing
- [Encoder colour](/quest/m1/color-model.md) - every moq-video encode path signals the colour its output actually has, or refuses instead of mislabelling
- [T-STD TS export](/quest/m1/tstd/README.md) - `moq export ts` is a proper remux that passes the T-STD buffer model, starting with a fixed `--delay`
- [TS fidelity and recovery](/quest/m1/ts-reliability/README.md) - both modes have explicit profiles, automated validation, loss tests, and a dedicated guide
- [T-STD TS export](/quest/m1/tstd/README.md) - `moq export ts` is a proper remux that passes the supported T-STD buffer models, starting with a fixed `--delay`
- [ATSC AC-3](/quest/m1/ts-atsc-ac3.md) - `export ts` carries 48 kHz ATSC AC-3 at every A/52 rate up to 640 kb/s instead of aborting from 384 kb/s up
- [Rust non-continuous signal](/quest/m1/rust-continuous.md) - the Rust container consumer reports a frame after a subscribe or discontinuity as non-continuous, like JS, for the tune-in and warmup trims
- [Open-GOP leading pictures](/quest/m1/open-gop-leading-pictures.md) - a viewer joining at a recovery point drops the leading pictures it cannot decode; continuous viewers keep them
Expand Down
11 changes: 11 additions & 0 deletions quest/m1/hop-aligned-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ failover `export ts` sees a timestamp rewind and exits (#4354).

## Plan

The 2026-10-11 audit adds a planning prerequisite for the shared source
coordinate. The group-number and wall-anchor proposals below are provisional
until it settles how the input supplies a common anchor and wrap era. A
shared epoch or raw PTS/PCR value alone does not supply that mapping. Also
resolve raw passthrough explicitly: today's `--epoch` accepts whole MPTS
there, whereas the semantic importer scope below says one program.

Decided: every publisher of one path and epoch MUST publish the same
broadcasts and tracks, since routes with the same epoch resume each other's
subscriptions. A redundant pair is keyed by a shared explicit epoch (decided
Expand Down Expand Up @@ -61,6 +68,10 @@ issues' 1+1 setup (one relay, two `import ts` sharing one `--epoch`, two
- [#4352](https://github.com/moq-dev/moq/issues/4352) - close this issue when the quest finishes
- [#4354](https://github.com/moq-dev/moq/issues/4354) - close this issue when the quest finishes

## Required

- [Plan: redundant TS alignment](/quest/m1/plan-ts-alignment.md) - settle the source-to-wall anchor, wrap era and raw group coordinate that a shared epoch alone does not provide

## Related

- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - a redundant pair shares one epoch
Expand Down
18 changes: 11 additions & 7 deletions quest/m1/open-gop-leading-pictures.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# [M] Open-GOP leading pictures are dropped at tune-in only
# [M] Open-GOP recovery drops orphaned leading pictures after each delivery break

## Goal

A viewer joining an open-GOP broadcast at a recovery point never hands the
decoder a leading picture whose references it does not have, while a viewer
already playing keeps every frame. The rule is the same for H.264
A viewer joining or resuming an open-GOP broadcast after a non-continuous
transition never hands the decoder a leading picture whose references it
does not have. Continuous groups keep every frame. The rule is the same for H.264
recovery-point keyframes with `recovery_frame_cnt = 0` (the broadcast
contribution case, and what the reference capture carries) and H.265 CRA
pictures. Gradual recovery (`recovery_frame_cnt > 0`) is out of scope: unsafe pictures
Expand Down Expand Up @@ -56,13 +56,17 @@ decode, and the JS test should drive the software decoder.
export still sends a join's orphaned leading pictures, so 3 of 500
frames on the open-GOP fixture decode after they present
(`dts-before-pts`). Trim them at tune-in from the same signal, on the
export's first group only, and make `just test ts --open-gop` pass under
first group after each non-continuous transition, and make
`just test ts --open-gop` pass under
`--strict`. Decided in the 2026-10-05 audit: the grader changes with it.
`test/ts/open-gop.py`'s decode-order check (a contiguous run of source
access units) and leading-pictures check, and the `test/ts/README.md` text
that says the round-trip must hand every leading picture on, both expect
every picture today; rewrite them to expect the first group's orphaned
leading pictures trimmed and every later group's kept. Rejected: dropping
every picture today; rewrite them to expect orphaned leading pictures
trimmed after each delivery break and every continuous group's kept.
The 2026-10-11 audit applies the same recovery rule to TS as to the
playback paths: the previous first-group-only wording missed later skips.
Comment thread
kixelated marked this conversation as resolved.
Resolved
Rejected: dropping
this bullet and keeping the grader.
- Tests: a synthetic group with a keyframe followed by two earlier-stamped
deltas is trimmed on the first group and kept on the second; and a viewer
Expand Down
46 changes: 46 additions & 0 deletions quest/m1/plan-ts-alignment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# [S] Plan: shared coordinates for redundant TS publishers and exporters

## Goal

Settle what shared information makes redundant TS publishers and exporters
identify the same content and schedule. Rewrite the implementation quests
around that contract before promising failover or ST 2022-7 compatibility.

## Plan

The 2026-10-11 audit found two missing prerequisites:

- Same epoch identity does not establish a source PCR/PTS-to-wall mapping,
a 33-bit timestamp wrap era, or a common raw-packet position. Importers
starting at different offsets cannot infer these from the UUID alone.
`catalog::Timebase::place` applies a supplied mapping; it does not discover
one. Raw fallback's timeout cuts also depend on acquisition history, even
though the CLI currently accepts a shared `--epoch` there.
- Identical TS packets are insufficient for ST 2022-7, which requires the
matching RTP packets, including headers. Complementary losses on redundant
paths are a reason to merge them; requiring the two legs to drop the same
content defeats that benefit.

Present the maintainer with the actual shared-coordinate choices: an
explicit common source anchor and era, a common upstream packetizer, or an
honestly narrowed importer contract. Include what each can prove when the
standby starts after a timestamp wrap, a clock break, or a fallback group cut.
Do not derive a media clock from wall arrival and call it deterministic.

For egress, separate deterministic TS slot/counter identity for common
content from coordinated RTP packetization, sequence/timestamp/SSRC identity
and any receiver merge. Decide the intended scope and API before implementing
`--sync`. NTP-synchronized hosts alone do not supply the missing source map.

Produce focused implementation quests and explicit refusals for unsupported
shared-epoch modes. Their automated tests start peers at different source
offsets, cross wraps, vary delivery skew, and introduce complementary loss;
compare common content in its shared coordinate rather than whole files
whose delivered content differs. Actual ST 2022-7 claims require RTP-level
tests against the [standard](https://pub.smpte.org/latest/st2022-7/st2022-7-2019.pdf).

## Related

- [Same-epoch importers](/quest/m1/hop-aligned-import.md) - currently assumes an input-derived wall anchor is available
- [Deterministic TS legs](/quest/m2/ts-hitless.md) - TS identity and RTP recovery need separate acceptance evidence
- [TS passthrough export](/quest/m1/ts-passthrough-export.md) - unchanged packets do not establish identical groups from independent starts
34 changes: 28 additions & 6 deletions quest/m1/plan-ts-pes-untimed.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
# [S] Plan: untimed verbatim PES
# [M] Plan: private PES timing and header fidelity

## Goal

Decide how a verbatim MPEG-TS track carries a PES that has no PTS without
inventing a time, from import through export. The usual case is asynchronous
private data such as KLV. The output is an implementation quest, or a
recorded reason to keep today's behaviour.
inventing a presentation time, from import through export, and which PES
header semantics it retains. The usual case is asynchronous private data
such as KLV. Produce focused implementation quests with explicit supported
profiles and refusal rules.

## Plan

Expand Down Expand Up @@ -35,5 +36,26 @@ the decision covers:
The maintainer decides the carriage, because it touches the catalog and
container contract.

Test plan for the implementation quest: importer to encoded track to
exporter, for a PES without a PTS and for one with a real PTS of 0.
The 2026-10-11 audit expands the decision to avoid settling timestamps in
isolation. A source placement time used for mux scheduling is distinct from
optional PTS and DTS. Today's private-PES track retains payload, `stream_id`,
and present PTS as the frame timestamp, but not the original DTS or other
PES-header fields. The parser skips ESCR, ES rate, trick mode, copy information, CRC and extension
fields; export synthesizes a generic PTS header and cannot reproduce all
special stream-ID header forms. A track called verbatim must say which bytes
or semantics that means. Unknown stream types also need to survive export or
be refused at import, not fail later through an enum conversion.

Settle per-PES timing presence, scheduling of asynchronous data alongside
advancing media, header preservation versus normalization, and the legacy
container migration together. The exporter currently drops non-video frames
before the initial video time; demonstrate that permitted untimed/private
content is not lost through that rule. Keep the catalog/container decision
separate from raw whole-TS fallback, whose packets already contain the headers.

Automated CI tests for the implementation quests: importer to encoded track
to exporter for missing PTS versus real PTS 0, distinct PTS/DTS, asynchronous
private data before and between media, optional headers, and special stream
IDs. Fixtures assert either retained semantics or the precise supported-profile
refusal, using TSDuck/FFmpeg where applicable. Update Rust, JS, the MPEG-TS
draft and the guide for any catalog/container wire change.
43 changes: 37 additions & 6 deletions quest/m1/ts-passthrough-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,31 @@
`m2ts` catalog section names and writes the same bytes back out, paced on the
source's own PCR and released at a fixed delay like an SRT receiver. From the
first group it releases, the output is byte-identical to the input, less any
object dropped late, so it is exactly as T-STD-conformant as the source.
object dropped late. On a complete path, validate preservation of the source
schedule as well as the bytes. Missing bytes can break a decoder's acquisition
or buffers; packet identity alone does not preserve T-STD conformance under loss.
There is no JS player for it.

## Plan

Decided (2026-10-01), from a discussion with t0ms, and split from the TS
passthrough quest on #5003, whose import half it completes:

Confirmed in the 2026-10-11 audit: carry either one supplied SPTS or the whole
MPTS unchanged. Do not extract programs or split PIDs in this mode. Separate
SPTS inputs have separate raw broadcasts/tracks under the existing one-multiplex
catalog contract; extracting a program from an MPTS changes
the packet sequence and is not the byte-identical fallback. This matches the
distinction between `unmodified-program`, `unmodified-multiplex` and
`per-program` in [MSFTS](https://datatracker.ietf.org/doc/html/draft-gregoire-moq-msfts).

The exactness boundary is the admitted source interval, starting at the first
released group. Account for the importer's current acquisition prefix and
188-byte framing restrictions explicitly; do not call this whole-file identity.
Test group acquisition without inserting synthetic tables or configuration
into the output. A periodic group boundary or one program's RAI does not
promise immediate decode for every program in an MPTS.

- The export reuses the TS export's jitter buffer (`rs/moq-mux/src/jitter.rs`),
keyed on each object's PCR time instead of a DTS, which also spreads each
object's bytes at the PCR-implied rate. It must pace on the source's PCR,
Expand All @@ -36,12 +53,23 @@ passthrough quest on #5003, whose import half it completes:
a coordinated RTP egress.
- Rust only.

Test: an export of a broadcast capture is byte-identical to the input from the
first released group, and the strict T-STD check gives the same verdict on
output and input. Two exporters fed the same objects with different arrival
This quest owns the reusable raw packet/position and release-schedule oracle
and its basic omission fixtures; the recovery quest extends it with the wider
fault matrix rather than building a second harness.

Automated tests in CI: an export of a supported capture is byte-identical to
the admitted input interval on a complete path, and the applicable T-STD check
gives the same verdict on output and input. Assert packet order and release
schedule as well as bytes. Two exporters fed the same objects with different arrival
skew emit identical bytes; when only one misses a deadline, its output is the
other's less that object's packets. A dropped object is counted, and the rest
still go out on time. Rerun the #4613 netem rig (10% loss, 120 s) against it.
still go out on their original schedule, leaving the missing time intact.
Use deterministic object/group omissions as well as a nightly real-network
impairment test: IP packet loss may be retransmitted and is not proof of
missing MoQ content. Fixtures include null/unknown PIDs, tables, adaptation
bytes, split PES, clock wrap, and MPTS with unaligned program recovery points.
Run TSDuck and FFmpeg automatically for profiles where their assertions apply;
record expected decode outages under loss separately from byte preservation.

Update `doc/bin/cli.md` for the flag, which today says `export ts` does not
read the passthrough track.
Expand All @@ -50,5 +78,8 @@ Public API: a new export mode; nothing existing breaks.

## Related

- [TS hitless](/quest/m2/ts-hitless.md) - the demultiplexed lane's 2022-7 legs and the `--sync` anchor
- [Deterministic TS legs](/quest/m2/ts-hitless.md) - common packet identity and the proposed `--sync` anchor
- [T-STD TS export](/quest/m1/tstd/README.md) - the jitter buffer and clock recovery this release shares
- [TS validation](/quest/m1/ts-reliability/validation.md) - shared automatic graders and controls
- [Loss and recovery](/quest/m1/ts-reliability/recovery.md) - actual missing content and reacquisition proof
- [Redundant alignment](/quest/m1/plan-ts-alignment.md) - identical common objects do not prove independent importers publish identical groups
Loading
Loading