Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3ee7680
chore(quest): claim quest/m1/ts-reliability/decode-order.md
kixelated Oct 23, 2026
9a2f2a6
fix(mux): bound declared AVC decode slots in MKV
kixelated Oct 23, 2026
a0e3fee
Merge remote-tracking branch 'origin/main' into iterate-decode-order
kixelated Oct 23, 2026
2f539bb
fix(mux): clock declared catalog reorder per group in TS, MKV and FLV
kixelated Oct 23, 2026
8f14a6d
feat(mux): declare reorder from the SPS and @moq/publish's encoder
kixelated Oct 23, 2026
fe4c64e
Merge remote-tracking branch 'origin/main' into iterate-decode-order
kixelated Oct 23, 2026
0c282df
fix(mux): carry reorder through VideoHint and format
kixelated Oct 23, 2026
9fe6ca6
Merge remote-tracking branch 'origin/main' into iterate-decode-order
kixelated Oct 24, 2026
33923b1
fix(mux): flush FLV's declared clock when its group ends
kixelated Oct 24, 2026
95e9d39
fix(mux): keep trimmed leading pictures' slots on TS's declared clock
kixelated Oct 24, 2026
16e29ec
docs(drafts): log the video reorder field in the moq-hang-04 changelog
kixelated Oct 24, 2026
52780bf
fix(mux): raise FLV's shift for a reorder declared after the header
kixelated Oct 24, 2026
b68685e
fix(publish): drop a repeated capture timestamp before encoding
kixelated Oct 24, 2026
dfe8b6e
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 24, 2026
eec258c
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 24, 2026
522c8cb
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 24, 2026
f3828d5
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 24, 2026
09a92ad
Merge remote-tracking branch 'origin/main' into rq-6453
kixelated Oct 24, 2026
7cfcce4
quest: drop the release audit's emptied Required list
kixelated Oct 24, 2026
2a456de
Merge remote-tracking branch 'origin/main' into rq-6453
kixelated Oct 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/concept/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ A few things the catalog can express beyond decoder config:
- **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition.
- **Jitter.** A rendition can say how much media it writes behind a frame's timestamp, in whole milliseconds rounded up: the reorder delay of B-frames and the span of a fragment or batch written together. It is measured from media timestamps alone, never the time a frame was written, so neither a constant encoder delay nor a capture clock counts. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent.
- **Delay.** A rendition can also say how far its frames' timestamps trail those another rendition already wrote, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. A rendition alone measures none. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's.
- **Reorder depth.** A video rendition may declare `reorder`, the most frames that can precede one in decode order and follow it in presentation, as H.264 `max_num_reorder_frames` counts them; `0` declares none, which is not the same as absent. The Rust H.264 and H.265 importers copy it from the SPS when the SPS declares one (progressive H.264 only), and `@moq/publish` declares `0`. The MPEG-TS, Matroska, and FLV exporters then derive each group's decode times from that group's frames alone, so exports that joined at different groups agree, and they end the export on a frame reordered deeper than declared or a depth the codec level cannot reach. Without it they infer decode times from the timestamps they have read, as before. A publisher never changes it once declared.
- **Colour.** A video rendition may carry `color`: the H.273 primaries, transfer, matrix, and range, plus the HDR10 mastering display and content light levels, so a renderer can set up HDR without parsing the bitstream. The Rust importers fill it from the SPS VUI, AV1 sequence header, and VP9 header, then fill gaps from a container's `colr` box or Matroska `Colour` element; the stream wins a disagreement. The HDR10 values come from H.264 and H.265 SEI or AV1 metadata OBUs in MPEG-TS and raw elementary streams, an AV1 track's av1C, or Matroska, but not yet from H.264 or H.265 in fMP4 or FLV. Nothing in `@moq/watch` reads it yet.
- **Spatial media.** A video rendition may carry `projection` (flat `rectilinear`, or `equirectangular` with the degrees of longitude it spans, 180 for VR180) and `stereo` (`mono`, `left-right`, or `top-bottom` frame packing), and an audio rendition may carry `layout: "ambisonic"` for AmbiX channels. Readers treat a value they don't recognize as absent. Nothing in `@moq/watch` renders them yet.
- **Disabled renditions.** An audio or video rendition with `enabled: false` has no frames coming, and a player must not select it. Pausing a rendition in `@moq/publish` keeps it in the catalog this way rather than removing it, so a mute is a one-field update; a disabled video rendition encodes one black keyframe first, so a player that predates the field shows black instead of a frozen picture. The field is written only when false. The older video `stalled` field is no longer written, and readers ignore it.
Expand Down
14 changes: 13 additions & 1 deletion drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ This draft needs `version` 4 for:
- a `track` ({{field-track}}) other than the entry's ID, which an older consumer subscribes to instead;
- a video `warmup` ({{field-warmup}}) above zero, since an older consumer starts decoding a group at a picture that is not a keyframe.

Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `archive` ({{archive-catalog}}) is ignored when undecodable; `stats` and `echo` ({{media-stats}}) only offer optional tracks; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.
Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `archive` ({{archive-catalog}}) is ignored when undecodable; `stats` and `echo` ({{media-stats}}) only offer optional tracks; video `reorder` leaves an older consumer inferring decode times from timestamps; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.

## Clock {#clock}
The catalog's root `clock` field is the broadcast's one continuous clock:
Expand Down Expand Up @@ -258,6 +258,7 @@ type VideoDecoderConfigExtensions = {
"displayAspectWidth": number | undefined,
"displayAspectHeight": number | undefined,
"preference": number | undefined,
"reorder": number | undefined,
"color": Color | undefined,
"projection": Projection | undefined,
"stereo": string | undefined,
Expand All @@ -276,6 +277,16 @@ A consumer selecting a rendition automatically MUST choose among the enabled ({{
A consumer MAY select another rendition at the viewer's explicit request.
A consumer that carries every rendition, rather than selecting one, ignores the field.

`reorder` is the most frames that can precede a frame in decode order and follow it in presentation order, the unit of H.264 `max_num_reorder_frames` and HEVC `sps_max_num_reorder_pics`, as an integer from 0 to 255.
`0` declares that presentation order is decode order.
A publisher that does not know the depth MUST omit the field; absent declares nothing.
A publisher that declares it MUST NOT send a frame that more than `reorder` earlier frames of its group follow in presentation order, and MUST NOT change or remove it for the track once declared.

A consumer that authors decode times, such as a transmuxer to a container that carries them, derives them from a declared `reorder` and the frames of the current group alone, so consumers that joined at different groups agree.
The n-th frame of a group in decode order decodes at the group's (n - `reorder`)-th earliest presentation time, counting from zero, and the group's first `reorder` frames decode at successive instants just before its earliest presentation time.
A consumer MUST refuse a frame reordered deeper than its rendition declares rather than author an invalid decode time, and SHOULD refuse a `reorder` the codec cannot reach: above the level's decoded picture buffer, or nonzero for a codec whose presentation order is its decode order.
Comment thread
kixelated marked this conversation as resolved.
Resolved
Without `reorder`, a consumer infers decode times from the timestamps it has read.

`color` describes the decoded pixels so a renderer can configure its pipeline without parsing the bitstream:

~~~
Expand Down Expand Up @@ -1204,6 +1215,7 @@ This document has no IANA actions.
- Each archive timeline declares its own fixed nominal record duration, omitted for an untimed track; removed the broadcast-wide duration bound.
- Removed the Recording section: the store layout is a moq-archive format, documented with that crate, not part of this draft.
- Added the optional video `preference` field: a consumer selects automatically among the highest preference it can decode.
- Added the optional video `reorder` field: the most frames that can precede one in decode order and follow it in presentation, fixed for the track once declared. A consumer that authors decode times derives them from it and the current group alone, refusing deeper reordering; without it, a consumer infers them from timestamps.
- Added an optional `epoch` beside every rendition and data track `broadcast` reference, and `replayEpoch` beside the archive's `replay` ({{field-epoch}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `epoch` fails a rendition proven to resolve another instance, and is best effort over a route without an Epoch.
- Derived HLS and DASH exports hand disabled references to enabled renditions at the first record numbered past the advertised ones, keeping the record `sequence` as the HLS media sequence and starting a DASH period.
- Added optional `stats` and `echo` catalog sections, publisher/viewer snapshots keyed by rendition ID, and the `.echo` feedback broadcast convention.
Expand Down
8 changes: 8 additions & 0 deletions js/hang/src/catalog/video.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,14 @@ test("video config accepts a signed 32-bit preference", () => {
expect(parse(2 ** 31).success).toBe(false);
});

test("video config keeps a zero reorder distinct from an absent one", () => {
const parse = (reorder?: number) =>
VideoConfigSchema.safeParse({ codec: "vp8", container: { kind: "legacy" }, reorder });
expect(parse().data?.reorder).toBeUndefined();
expect(parse(0).data?.reorder).toBe(Catalog.u8(0));
expect(parse(256).success).toBe(false);
});

test("ranked puts a known zero bitrate above an unknown one", () => {
const renditions = {
unknown: rendition({ codedWidth: 1280, codedHeight: 720 }),
Expand Down
6 changes: 6 additions & 0 deletions js/hang/src/catalog/video.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,12 @@ export const VideoConfigSchema = z.object({
// The maximum frame rate of the video in frames per second.
framerate: z.optional(z.number()),

// The most frames that can precede a frame in decode order and follow it in presentation order,
// as H.264 max_num_reorder_frames and HEVC sps_max_num_reorder_pics count them. 0 declares that
// presentation order is decode order. Absent declares nothing: a consumer that authors decode
// times infers them from the timestamps instead. Once declared, it holds for the track's life.
reorder: z.optional(u8Schema),

// The maximum bitrate of the video in bits per second.
// TODO: Support up to Number.MAX_SAFE_INTEGER
bitrate: z.optional(u53Schema),
Expand Down
13 changes: 13 additions & 0 deletions js/publish/src/video/encoder.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1001,6 +1001,19 @@ test("cut forces a keyframe on the next frame, coalescing requests without spaci
expect(keys).toEqual([0, 200, 300]);
});

test("a repeated capture timestamp is dropped rather than failing the track", async () => {
let closed: unknown;
const keys = await recordKeyframes({}, async ({ encoder, track, send }) => {
await send(0);
await send(0);
encoder.cut();
await send(100);
closed = track.closed.peek();
});
expect(keys).toEqual([0, 100]);
expect(closed).toBeUndefined();
});

test("a join over the refresh ratio forces a keyframe on the next frame", async () => {
const keys = await recordKeyframes({ refresh: 2 }, async ({ track, send }) => {
const watching = track.subscribe();
Expand Down
15 changes: 15 additions & 0 deletions js/publish/src/video/encoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,8 @@ export class Encoder {

let lastKeyframe: Micro | undefined;
let lastEncoded: Micro | undefined;
// The catalog declares no reordering, so each chunk must present after the one before it.
let lastOutput: number | undefined;
// The largest group demand asked for that a keyframe already answers. A join over the refresh
// ratio asks for one past the largest, which the next frame opens.
let answered: number | undefined;
Expand All @@ -387,6 +389,14 @@ export class Encoder {
effect.spawn(async () => {
const encoder = new VideoEncoder({
output: (frame: EncodedVideoChunk) => {
if (lastOutput !== undefined && frame.timestamp <= lastOutput) {
this.#abort(
new Error(`encoder reordered a frame at ${frame.timestamp}us behind ${lastOutput}us`),
track,
);
return;
}
lastOutput = frame.timestamp;
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
const key = frame.type === "key";
if (key) {
lastKeyframe = frame.timestamp as Micro;
Expand Down Expand Up @@ -467,6 +477,9 @@ export class Encoder {
// Allow half an interval of slack so jittery capture timestamps don't drop
// a frame we meant to keep.
const targetFrameRate = config?.frameRate;
// A repeated timestamp would present with the frame before it, which no reordering
// declared allows; a backward one still fails the track at output.
if (frame.timestamp === lastEncoded) continue;
if (targetFrameRate && lastEncoded !== undefined) {
const minGap = Micro.fromSecond((1 / targetFrameRate) as Second);
if (frame.timestamp - lastEncoded < minGap - minGap / 2) continue;
Expand Down Expand Up @@ -581,6 +594,8 @@ export class Encoder {
codedWidth: Catalog.u53(config.width),
codedHeight: Catalog.u53(config.height),
optimizeForLatency: true,
// Realtime WebCodecs encoders emit no B-frames; the output callback refuses one that does.
reorder: Catalog.u8(0),
container: { kind: "legacy" } as const,
...estimate,
};
Expand Down
14 changes: 6 additions & 8 deletions quest/m1/encoder-reorder.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,11 @@ guessing.

## Plan

[Decode order](/quest/m1/ts-reliability/decode-order.md) adds
`VideoHint.reorder` and lists the moq-video encoders as a producer that
declares it, but leaves them out. Set it in `rendition_hint`
(`rs/moq-video/src/encode/producer.rs`) and assert it in an encoder test.
[moq-dev/moq#6453](https://github.com/moq-dev/moq/pull/6453) adds
`VideoHint.reorder`, which `rendition_hint`
(`rs/moq-video/src/encode/producer.rs`) copies from the probed rendition, but
the encoders leave it unset, so their renditions stay on the inferred clock.
Declare `0` there, even when the SPS omits its bitstream restriction, and
assert it in an encoder test.

Public API: none. Wire: none.

## Required

- [Decode order](/quest/m1/ts-reliability/decode-order.md) - adds the `reorder` field
8 changes: 0 additions & 8 deletions quest/m1/release-audit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,14 +85,6 @@ The 2026-10-22 binding and OBS audit at main
rate recovery child. Its two OBS findings went to m2 next to the other OBS
work, because obs-moq ships on its own schedule.

## Required

The entries comprise reproduced or source-confirmed findings
and contract/lifetime investigations. Each
child records whether its behavior is new on main or inherited from release.

- [Decode order](/quest/m1/ts-reliability/decode-order.md) - the exporters' decode clock holds at most the declared reorder depth, so flat timestamps cannot bypass MKV/TS backpressure or cause quadratic draining

## Related

- [JS subscription ranges](/quest/m1/subscribe-ranges/js.md) - range lists and frame offsets build on the bounds fixed here
Expand Down
3 changes: 1 addition & 2 deletions quest/m1/ts-reliability/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ refused means an error or a counted drop.

| Mechanism | Contract | Owner |
|:--|:--|:--|
| H.264 (0x1B), H.265 (0x24) | Preserved: video rendition, PID and ES descriptors in `mpegts.tracks`; DTS normalized from decode order. Today one access unit per PES | [access units](/quest/m1/ts-reliability/pes-access-units.md), [decode order](/quest/m1/ts-reliability/decode-order.md) |
| H.264 (0x1B), H.265 (0x24) | Preserved: video rendition, PID and ES descriptors in `mpegts.tracks`; DTS normalized from decode order. Today one access unit per PES | [access units](/quest/m1/ts-reliability/pes-access-units.md) |
| AAC ADTS (0x0F), MP2, AC-3, E-AC-3, Opus (0x06 + `Opus`) | Preserved: audio rendition; Opus export labels mono and stereo, and 3 to 8 channels with an OpusHead in the Vorbis layout. Other mapping families (such as 255) refuse | landed |
| Private and unknown PES types, LATM (0x11), metadata (0x15) | Preserved: `pesPacket`, PTS and DTS rebased | landed |
| PES clock fields (ESCR, pack header, TREF), PES scrambling | Refused, counted in `refused` | landed |
Expand Down Expand Up @@ -99,7 +99,6 @@ remain with their current quests.
## Required

- [State bound](/quest/m1/ts-reliability/state-bound.md) - SI and revision state retire with what the source stops repeating
- [Decode order](/quest/m1/ts-reliability/decode-order.md) - TS, MKV and FLV author DTS from decode order and a declared `reorder` depth
- [Access units](/quest/m1/ts-reliability/pes-access-units.md) - a PES carrying several access units publishes each at its own time
- [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a revised PAT never leaves a broadcast meaning another program
- [Stream routes](/quest/m1/ts-reliability/stream-routes.md) - every PID gets a declared route, and what is not carried is counted
Expand Down
Loading
Loading