Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions quest/m1/release-audit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,13 +88,13 @@ work, because obs-moq ships on its own schedule.

## Required

The children comprise reproduced or source-confirmed findings,
The entries comprise reproduced or source-confirmed findings,
contract/lifetime investigations, and one security-review pointer. Each
child records whether its behavior is new on main or inherited from release.

- [Review HLS admission](/quest/m1/release-audit/hls-admission.md) - resolve the security-sensitive resource-boundary review before the cut
- [Bound decoded datagram retention](/quest/m1/release-audit/datagram-memory.md) - shared ownership accounts for retained plain and compressed payloads across tracks
- [Bound export reorder storage](/quest/m1/release-audit/export-reorder.md) - flat timestamps cannot bypass MKV/TS backpressure or cause quadratic draining
- [Decode order](/quest/m1/ts-reliability/decode-order.md) - the exporters' decode clock holds at most the declared reorder depth, so flat timestamps cannot bypass MKV/TS backpressure or cause quadratic draining
- [Validate CMAF header fields](/quest/m1/release-audit/cmaf-counts.md) - tiny fragments cannot force allocations proportional to an untrusted sample count, and a zero timescale is refused
- [Preserve fMP4 sample runs](/quest/m1/release-audit/fmp4-passthrough.md) - passthrough does not substitute gap or interleaved bytes for media samples
- [JS honors subscription bounds](/quest/m1/release-audit/js-subscription-bounds.md) - each reader and each datagram stays inside its own subscription's group bounds, not the aggregate's
Expand Down
36 changes: 0 additions & 36 deletions quest/m1/release-audit/export-reorder.md

This file was deleted.

6 changes: 4 additions & 2 deletions quest/m1/ts-reliability/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ and PAT plans build on, rank ahead of the remaining rows. Rejected: carrying
decode time in every frame container, one shared ingest byte budget,
and replacing the demux with GStreamer or another parser (it fixes neither
cause). Out of scope for now: decoupling catalog publication from mux-rate
measurement.
measurement. Re-planned the same day: decode order covers MKV and FLV too and
absorbs the release audit's export-reorder quest, and the shared parser lands
before the open stream-routes and PAT drafts, which rebase onto it.

PMT fragmentation (#5220), initial mux-rate measurement (#5287), and
decoder-time admission (#5306) have landed; PSI runs and catalog linger
Expand All @@ -98,7 +100,7 @@ remain with their current quests.
## Required

- [State bound](/quest/m1/ts-reliability/state-bound.md) - SI and revision state retire with what the source stops repeating
- [Decode order](/quest/m1/ts-reliability/decode-order.md) - exporters author DTS from decode order and fixed per-rendition constants
- [Decode order](/quest/m1/ts-reliability/decode-order.md) - TS, MKV and FLV author DTS from decode order and a declared `reorder` depth
- [Shared parser](/quest/m1/ts-reliability/shared-parser.md) - `--program all` scans the multiplex once
- [PMT export](/quest/m1/ts-reliability/pmt-export.md) - a TS export announces each PMT revision, mux rate included, at the next `version_number`
- [Access units](/quest/m1/ts-reliability/pes-access-units.md) - a PES carrying several access units publishes each at its own time
Expand Down
141 changes: 102 additions & 39 deletions quest/m1/ts-reliability/decode-order.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,116 @@
# [L] Decode time comes from decode order and fixed constants
# [XL] Decode time comes from decode order and a declared reorder depth

## Goal

An exporter that needs a decode time (TS, FLV) derives it only from the
frames of the current group, in the decode order they arrive, and from
constants fixed for the rendition's life. Two exporters that read the same
group author the same DTS for it, whatever they joined at or lost before.
Media frames stay presentation-time only: no decode time on the wire.
Every exporter that authors a decode time (TS, MKV, FLV) derives it only from
the frames of the current group, in the decode order they arrive, and from the
rendition's declared reorder depth. Two exporters that read the same group
author the same DTS for it, whatever they joined at or lost before, and an
exporter never holds more video frames than that depth. A rendition that
declares no reordering adds no decode delay. Media frames stay
presentation-time only: no decode time on the wire.

## Plan

Why: a DTS that depends on what the exporter read before has been patched
six times ([moq-dev/moq#1843](https://github.com/moq-dev/moq/pull/1843),
[#2967](https://github.com/moq-dev/moq/pull/2967),
[#4500](https://github.com/moq-dev/moq/pull/4500),
[#4574](https://github.com/moq-dev/moq/pull/4574),
[#4645](https://github.com/moq-dev/moq/pull/4645),
[#5949](https://github.com/moq-dev/moq/pull/5949)). Today there are three
authors: `DecodeClock::pop` (`rs/moq-mux/src/container/decode.rs`, TS without
`--sync` and MKV) settles a slot by the highest PTS read and a delay that only
grows; `pop_counted` (TS `--sync`) depends on the held frames alone; FLV
export authors `PTS - jitter` bumped past the last DTS, which passes the PTS
on B-frames. fMP4 export already anchors its decode timeline at each group's
first frame and needs no change.

Decided 2026-10-22 by the maintainer, after the TS root-cause audit:

- Frames carry PTS only, in decode order. A decoder needs order; a TS
output needs *a* DTS that is monotonic, never after its PTS, and fits the
T-STD buffer, not the source's own. Source DTS spacing (field coding,
pulldown) is normalized, not preserved.
- Frames carry PTS only, in decode order. A decoder needs order; an export
needs *a* DTS that is monotonic, never after its PTS, and fits the T-STD
buffer, not the source's own. Source DTS spacing (field coding, pulldown)
is normalized, not preserved.
- Rejected: carrying decode time in the frame (CMAF composition offsets, a
LOC property, Legacy inference). It adds a wire field to every container
to preserve spacing no consumer needs.
- `DecodeClock::pop_counted` (`rs/moq-mux/src/container/decode.rs`), today
the `--sync` path, becomes the only path. Delete the history-dependent
`pop` settling and the delay that only grows; the catalog `jitter` stops
feeding DTS.
- The constants: an optional per-rendition reorder delay in the Hang
catalog, set by an importer that knows it (TS from source `PTS - DTS`,
fMP4 from `ctts`/`trun` offsets, an encoder from its config) and fixed for
the rendition's life. A source that exceeds it publishes a new rendition
under a fresh name, as a PMT revision does. Absent, export falls back to
the SPS: its declared reorder depth, else the level's `MaxDpbFrames`
bound. Propose the field's name with alternatives in the PR.

Why: [moq-dev/moq#1843](https://github.com/moq-dev/moq/pull/1843),
[#2967](https://github.com/moq-dev/moq/pull/2967),
[#4500](https://github.com/moq-dev/moq/pull/4500),
[#4574](https://github.com/moq-dev/moq/pull/4574),
[#4645](https://github.com/moq-dev/moq/pull/4645) and
[#5949](https://github.com/moq-dev/moq/pull/5949) each patched a DTS that
depended on what an exporter had read before.

Rust: `rs/hang` catalog, `rs/moq-mux` TS, fMP4 import and TS, FLV export.
JS: `js/hang` catalog. Draft: `drafts/draft-lcurley-moq-hang.md` (the
catalog field), `drafts/draft-lcurley-moq-mpegts.md` (how export authors
DTS). Public API: a new optional catalog field; the wire change is additive.

Tests: two exporters joining one B-frame source at different groups, and
one losing a group, author identical DTS for every group both read; a source
exceeding its declared delay publishes a new rendition; a source without the
field falls back to the SPS bound. Keep the T-STD controls passing.
- One clock serves TS, MKV and FLV. The counted path becomes the only one,
and it is not called `pop_counted`: the depth is fixed for the
rendition's life, so it moves into the constructor
(`DecodeClock::new(reorder)`) and the release is a plain `pop(flush)`.
A new rendition gets a new clock. Delete the timestamp-settled `pop`, its
growing delay, `reach`, FLV's `author_dts`/`dts_reserve`, and the catalog
`jitter`'s role in DTS.
- The depth is a new optional `reorder` field on the Hang video rendition:
the most frames that can precede a frame in decode order and follow it in
presentation, the unit of H.264 `max_num_reorder_frames` and HEVC
`sps_max_num_reorder_pics`.
- DTS shifts by slots, not by a time: the nth frame of a group decodes at the
group's (n - `reorder`)th presentation slot, which no frame still to come
can precede. The group's first `reorder` frames step one 90 kHz tick apart
up to the keyframe's slot: the clock cannot know the group's spacing until
frames it may not hold have arrived (`references_far_apart_keep_their_slots`),
and one tick needs no spacing, keeps DTS increasing within a group and
across groups without history, holds no frame past `reorder`, and only
relaxes those frames' delivery deadlines (found in review). Rejected: a delay of
`reorder` times a frame period, since the catalog `framerate` may change in
place and two exporters would build different clocks (found in review), and
freezing the rate or requiring an SPS fixed rate, which both add a rule
the slot shift does not need.
- Those first frames can decode before the timeline's zero, so the clock
works in signed ticks and each exporter maps them to its wire: TS wraps
modulo 2^33 as its timestamps already do, MKV writes no DTS and only
schedules by it, and FLV, whose tag time is unsigned, rebases its whole
audio and video timeline at its first DTS. Never saturate or push a DTS
past its PTS.
- Absent means 0: no reordering and no decode delay, so a stream without
B-frames pays no latency. Rejected: absent meaning the level's
`MaxDpbFrames` (hundreds of ms on High profile streams that never reorder),
and inferring 0 from Baseline profile only (still charges High profile).
- Every producer that can reorder sets it before publishing the rendition:
encoders from their config; importers (TS, FLV, fMP4, MKV) from the SPS's
declared depth, else from the source's own decode offsets on its first
keyframe (TS PTS - DTS, FLV composition time, fMP4 `ctts`/`trun`): none
means 0, any means the level's `MaxDpbFrames`. MKV carries no decode
offsets, so it takes `MaxDpbFrames` when the SPS declares nothing. An
importer knows a frame reorders when its PTS is below an earlier frame's in
decode order; one that reorders deeper than declared publishes a new
rendition under a fresh name, as a PMT revision does. Rejected (from
review): sampling the whole first group before publishing, which delays
every start by a group to catch a source the new rendition already covers.
`js/publish` refuses encoder output that reorders, since WebCodecs exposes
no B-frame setting to declare.
- An exporter that reads a frame reordered deeper than its rendition's
`reorder` refuses the export, never authors an invalid DTS. So does a
`reorder` above the codec level's `MaxDpbFrames` (at most 16), which no
conformant stream can reach, and a nonzero `reorder` on a codec whose
presentation order is its decode order. This bounds the
clock's held frames by the declared depth whatever the timestamps do, which
replaces the release audit's export-reorder quest: flat or backward PTS can
no longer hold frames without bound or drain quadratically.
Comment on lines +84 to +91

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Cap the declared reorder depth before buffering

When a hostile publisher supplies an extremely large reorder, retaining up to the declared depth still allows it to make each exporter hold an arbitrary number of frame payloads. The plan replaces the release-audit resource-bound quest without defining any schema/codec maximum or capacity-backed admission check for this network-provided value, so the claimed memory bound is not operationally finite; validate the depth before constructing the clock or tie retention to owned capacity.

AGENTS.md reference: AGENTS.md:L18-L19

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed: a reorder above the codec level's MaxDpbFrames (at most 16) refuses, as does a nonzero reorder on a codec that never reorders, so the held frames are bounded by the codec, not by the peer's value.

(Written by Claude Opus 5.5)


Rust: `rs/hang` catalog; `rs/moq-mux` decode clock, TS/MKV/FLV export, and
the TS, FLV, fMP4 and MKV importers; `rs/moq-video` encoders. JS: `js/hang`
catalog, `js/publish`. Draft: `drafts/draft-lcurley-moq-hang.md` (the field
Comment on lines +93 to +95

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include the MKV importer in reorder publication

Add rs/moq-mux's MKV importer to the producer work: it supports H.264/H.265 and forwards blocks in file order with their presentation timestamps, so an MKV containing B-frames can produce the reordered Hang stream that this new field governs. As scoped here, only TS, FLV, and fMP4 importers populate reorder; an MKV-imported rendition therefore leaves it absent and its next TS/MKV/FLV export is required to refuse the reordered frames.

AGENTS.md reference: AGENTS.md:L23-L23

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The MKV importer is now a producer: with no decode offsets in MKV, it takes the SPS's declared depth, else the level's MaxDpbFrames.

(Written by Claude Opus 5.5)

and its absent meaning), `drafts/draft-lcurley-moq-mpegts.md` (how export
authors DTS). Public API and wire: an optional catalog field whose absence
now means "never reorders", a breaking change for any producer that reorders
without it; add a `MIGRATE.md` entry for `hang`, `@moq/hang` and `moq-mux`, whose
direct users must now set it.

Tests: two exporters joining one B-frame source at different groups, and one
losing a group, author identical DTS for every group both read, in TS, MKV
and FLV; a stream without `reorder` exports with DTS equal to PTS; a frame
reordered past the declared depth refuses the export, and so does a `reorder`
above the level's `MaxDpbFrames`; a framerate change mid-rendition leaves every
DTS unchanged; a group starting at timestamp zero with a nonzero `reorder`
exports valid TS, MKV and FLV; a keyframe without a decode offset followed by
a reordered frame publishes a new rendition; a source deepening its
reordering publishes a new rendition; flat and backward PTS through MKV and
TS export retain at most `reorder` frames, swept over item count and payload
size; each importer sets `reorder` from the SPS and from source offsets.
Keep the T-STD controls passing.

## Related

Expand Down
5 changes: 4 additions & 1 deletion quest/m1/ts-reliability/shared-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ Decided 2026-10-22 by the maintainer: this lands before
[PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md), whose plans work
around per-program reparsing (multiplex-wide PID rows moving between owners,
a removed program's `handle_pat` error stopping the whole ingest). Revisit
both plans once it lands.
both plans once it lands. Re-confirmed 2026-10-22 after drafts of both opened
first: they rebase onto this and drop the accounting it replaces, such as the
`max()` merges in `Programs::stats` and PID rows moving between owners,
rather than landing ahead of it.

Add a benchmark in `rs/moq-mux/benches` that feeds `Programs` a generated
MPTS and sweeps program count and input rate independently; land it first
Expand Down
Loading