diff --git a/quest/m1/README.md b/quest/m1/README.md index 35d86047a53..9a70d02d357 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -95,11 +95,13 @@ above them. - [Epoch anchor](/quest/m1/epoch-anchor.md) - same-epoch importers each derive the catalog clock and timestamp era from the epoch and stay live, so they agree across a 33-bit wrap, a forward clock break, and a restart - [Raw TS cuts on the epoch timeline](/quest/m1/ts-passthrough-epoch-cuts.md) - passthrough fallback cuts fall on the epoch's grid and the M2ts section derives from the epoch and content, so instances started apart publish identical groups and catalogs - [Same-epoch importers](/quest/m1/hop-aligned-import.md) - importers sharing one `--epoch` and fed one stream publish identical groups and timestamps, so failover between a redundant pair survives +- [Importers announce with a track](/quest/m1/import-declared-track.md) - every importer announces once its catalog names a servable track (a rendition or an extension track like `m2ts`), and input that ends with none exits non-zero unannounced - [Plan: untimed verbatim PES](/quest/m1/plan-ts-pes-untimed.md) - decide how a verbatim TS track carries a PES that has no PTS, then write the implementation quest - [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding; snapshots add `latest()` beside an in-order `next()` - [JSON window timestamps](/quest/m1/json-window-timestamps.md) - Rust and JS window consumers return each event's frame timestamp like the snapshot and stream consumers - [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface - [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native FILL_TIMEOUT bounds upstream waiting on drafts 18-22, alongside local content-age enforcement +- [Browser follows a renamed track](/quest/m1/browser-rename-check.md) - a real browser keeps playing when `@moq/publish` renames a track after a codec change or ceiling growth - [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` refuses a redefined rendition, and every Rust publisher mints a new rendition name for a decoder identity change and updates everything else in place - [Archive](/quest/m1/archive/README.md) - record selected tracks to any object_store and replay them over FETCH or derived HLS; the catalog entry and format may break in place, since no archives exist - [In-band auth](/quest/m1/auth/README.md) - a session tells its peer what it may publish and subscribe to, unions tokens presented in band, and fails loud on an out-of-scope publish @@ -161,6 +163,8 @@ above them. - [Shared timebase in the bindings](/quest/m1/ffi-timebase.md) - binding apps publish several containers from one source on one shared offset - [Vulkan Video encode on AMD](/quest/m1/vulkan-encode.md) - H.264 and H.265 from an external Vulkan image on RADV, hand-rolled on ash, proven on an RX 9070 - [VA-API encodes an external Vulkan image](/quest/m1/vaapi-vulkan-import.md) - an explicit-sync DMA-BUF reaches Intel's encoder, proven on Arrow Lake +- [Current LOC Timestamp only](/quest/m1/ietf-loc-timestamp.md) - moq-transport Timestamps are read only from LOC property 0x10, and 0x06 is opaque on every draft +- [Refuse draft-14 END_OF_TRACK properties](/quest/m1/ietf-eot-properties.md) - a draft-14 END_OF_TRACK carrying properties is refused instead of losing them - [Malformed moq-transport input](/quest/m1/ietf-malformed-close.md) - malformed draft-18 and draft-21 control input closes the session with the draft's code, or PROTOCOL_VIOLATION where a code is a real burden and the fallback is recorded in `doc/concept/standard.md`, in moq-net and js/net - [Binding track resolver](/quest/m1/binding-track-helper.md) - binding apps subscribe to a rendition without inlining `track` or the key - [Exporters follow a rendition to a new track](/quest/m1/export-rendition-rebind.md) - the fMP4, TS, MKV and FLV exporters rebuild a rendition whose resolved `track` or `broadcast` changes under the same ID diff --git a/quest/m1/audio-warmup.md b/quest/m1/audio-warmup.md index 4f334674b83..c68e2f33978 100644 --- a/quest/m1/audio-warmup.md +++ b/quest/m1/audio-warmup.md @@ -24,6 +24,11 @@ frames decode independently and set nothing; HE-AAC is out of scope. which drops the first three decoded frames of every legacy or LOC subscription, CMAF excepted. That is the start loss left on a rendition's return after an absence: 60 ms of 20 ms Opus. +- Name the 80 ms pre-roll once in Rust and once in JS. Every site that + advertises or derives it uses that constant: the encoder catalog, the + shared Opus conversion, and the MSF consumer. Add a test that the audio + catalog `Producer` publishes exactly `Encoder::catalog()`, because the two + have drifted before. - Tests in both languages: a mid-stream join discards exactly the warmup span and a continuous listener loses nothing. diff --git a/quest/m1/browser-rename-check.md b/quest/m1/browser-rename-check.md new file mode 100644 index 00000000000..5f3c314ed37 --- /dev/null +++ b/quest/m1/browser-rename-check.md @@ -0,0 +1,24 @@ +# [S] A browser follows a renamed track + +## Goal + +A real browser keeps playing across both renames +[#5552](https://github.com/moq-dev/moq/pull/5552) introduced: a codec change, +and a `Video.Encoder` growing past its coded-size ceiling. In both cases +`@moq/publish` mints a new `track` for the same rendition, and `js/watch` +follows it without a stall. The check runs at least nightly in CI. + +## Plan + +#5552 landed with unit tests only and listed this check as a follow-up. Unit +tests do not show what WebCodecs and the watch path do across the switch. + +Extend the browser media QA (`just test media`) rather than building a new +harness. Fail on a stall, a decode error, or a frame from the old track after +the switch. Check that it fails with the rename disabled. + +Public API: none. Wire: none. + +## Related + +- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - the same contract for the Rust publishers diff --git a/quest/m1/dropped-internal.md b/quest/m1/dropped-internal.md index b2395995047..0849a41ac37 100644 --- a/quest/m1/dropped-internal.md +++ b/quest/m1/dropped-internal.md @@ -28,6 +28,11 @@ Decided with the maintainer on 2026-10-13: (an existing `Dropped` -> `Closed` remap shows the intent), and a copy closed while its completion is still pending. - `Action::Forget` in `model/origin.rs` ends with `Cancel`; we chose to stop. + - `moq import` failures: the stdin decode error path in + `rs/moq-cli/src/publish.rs` and `Programs::abort` in + `rs/moq-mux/src/container/ts/programs.rs` drop their catalog producers, + so catalog subscribers see `Dropped` instead of the import's error. + `catalog::Producer` has no public abort yet, and they need one. - The `moq-transcode` rung whose source closes while idle passes on the source's own end, or `Cancel` if it ended cleanly. - Use existing variants; no new public variant. @@ -48,7 +53,8 @@ Decided with the maintainer on 2026-10-13: - Guard with tests only: one regression test per fixed site asserting the specific error. No runtime check on the refcount fallback. -Public API: none; error values consumers observe change. Wire: no new wire +Public API: adds `abort` to `moq_mux::catalog::Producer` for the import +failure paths; error values consumers observe change. Wire: no new wire format or codes, though replacing `Dropped` with `Cancel` or `Closed` can change which existing codes are emitted. diff --git a/quest/m1/ietf-eot-properties.md b/quest/m1/ietf-eot-properties.md new file mode 100644 index 00000000000..5407fa45216 --- /dev/null +++ b/quest/m1/ietf-eot-properties.md @@ -0,0 +1,34 @@ +# [XS] Refuse properties on a draft-14 END_OF_TRACK + +## Goal + +In Rust and JS, a draft-14 END_OF_TRACK object that carries Object +Properties is refused instead of having its properties silently dropped. An END_OF_TRACK without +properties ends the track cleanly, as today. + +## Plan + +Decided with the maintainer on 2026-10-15: refuse it, don't carry it. +Carrying it would mean storing terminal fields on the track rather than the +group. The project's rule is supported or refused. + +Context from [#5494](https://github.com/moq-dev/moq/pull/5494): + +- END_OF_TRACK stays a track-level Location. +- Only draft 14 allows properties on it. +- From draft 15, a status object carrying properties is already refused at + ingest. + +Draft 14 permits these properties, so a session `PROTOCOL_VIOLATION` would be +wrong. End the subscription with an error that names it unsupported. + +`js/net/src/ietf/object.ts` drops them the same way: it reads past the +extension block and returns the END_OF_TRACK frame without them. Fix both, and +add a regression test in each language that fails without the change. + +Public API: none. Wire: a draft-14 publisher that puts properties on +END_OF_TRACK now gets an error instead of a clean end. + +## Related + +- [moq-transport passthrough](/quest/m1/ietf-extension/README.md) - carries END_OF_GROUP status objects with their properties diff --git a/quest/m1/ietf-loc-timestamp.md b/quest/m1/ietf-loc-timestamp.md new file mode 100644 index 00000000000..7ed20246c65 --- /dev/null +++ b/quest/m1/ietf-loc-timestamp.md @@ -0,0 +1,37 @@ +# [XS] Read only the current LOC Timestamp ID + +## Goal + +Rust and JS read a moq-transport object's Timestamp only from the current LOC +property ID (0x10). Property 0x06 is opaque on every draft, never read as the +draft-ietf-moq-loc-03 Timestamp. + +## Plan + +Decided with the maintainer on 2026-10-15: we support only the latest LOC +Timestamp ID. LOC and moq-transport are versioned independently, so a LOC ID +must not depend on the transport draft. Today 0x06 is read as the loc-03 +Timestamp only before draft 19, keyed on the transport version, because +draft 19 reassigns 0x06 to SUBGROUP_DELIVERY_TIMEOUT. + +Delete the loc-03 decode and everything that exists only for it: + +- `PROP_TIMESTAMP_DRAFT03` and its decode in `rs/moq-net/src/ietf/`. +- The unit conversion of 0x06 when relaying into an egress with different + units, and its drop for a unitless egress (from + [#5494](https://github.com/moq-dev/moq/pull/5494)). +- `hasDraft03Timestamp` in `js/net/src/ietf/object.ts`. +- The tests for these paths. + +Keep the draft-19 boundary guard: a mutable 0x06 is dropped and a sealed one +refused when relaying between drafts 18 and earlier and drafts 19 and later. +That guard is about moq-transport reassigning the ID, not about LOC. Without +it, a draft-18 publisher's 0x06 would arrive downstream as +SUBGROUP_DELIVERY_TIMEOUT. Below and above the boundary, 0x06 is forwarded +like any other unknown property. + +Public API: none. Wire: a peer still writing 0x06 Timestamps reads as untimed. + +## Related + +- [moq-transport passthrough](/quest/m1/ietf-extension/README.md) - carries object properties through relays diff --git a/quest/m1/import-declared-track.md b/quest/m1/import-declared-track.md new file mode 100644 index 00000000000..007e9f8e5b3 --- /dev/null +++ b/quest/m1/import-declared-track.md @@ -0,0 +1,51 @@ +# [S] Every importer announces only once it has a track + +## Goal + +Every `moq import` input follows one rule for when it has declared a track: +the broadcast is announced once its catalog names at least one servable track, +and an input that ends before then exits non-zero without announcing. A +servable track is a rendition, or an extension track such as `--passthrough`'s +`m2ts` multiplex, which publishes no rendition. Today a TS +input carrying only program tables can still announce an empty catalog at end +of input and exit 0. + +The rule covers the stdin path, TS (including `--program all`), fMP4, FLV, +Annex-B/avc3, and HLS. + +Non-goals, both declined in [#5517](https://github.com/moq-dev/moq/pull/5517): + +- HLS holding the announce until every discovered rendition is ready. A dead + variant is left out of the first catalog, as moq-hls intends. +- FLV waiting for every track type its header flags declare. + +## Plan + +[#5517](https://github.com/moq-dev/moq/pull/5517) made the stdin path wait for +the catalog's first snapshot and bail with "input ended before it declared a +track". Each importer still decides on its own when to release that snapshot: + +- TS releases at its first PES with a PTS. The `Pending` doc in + `rs/moq-cli/src/publish.rs` says the first PMT, so the doc and the code + disagree. +- End of input releases every reservation. A program whose PMT never + arrived is announced anyway. +- fMP4 releases at the moov, and FLV at its first media frame. + +Recommended definition: a PMT or header entry alone does not count. A track +counts once the catalog names it: a rendition with a known codec config, or an +extension track. Settle it in one place that every importer shares, rather than per +importer. Fix the `Pending` doc to match. + +Tests: a TS input with a PAT and PMT but no PES, an FLV with no frames, and +an avc3 input with no SPS each fail unannounced. Each importer's normal +fixture still announces, and so does `--passthrough`. + +Update `doc/bin/cli.md` inline if it describes the exit behavior. + +Public API: none expected. Wire: none. + +## Related + +- [No made-up Dropped](/quest/m1/dropped-internal.md) - covers the same imports' failure path, which drops its catalog instead of aborting it with the cause +- [Same-epoch importers](/quest/m1/hop-aligned-import.md) - same-epoch importers publish identical tracks diff --git a/quest/m1/test-flakes-2/handshake-ack.md b/quest/m1/test-flakes-2/handshake-ack.md index 2a1d630a125..9716e3afd2c 100644 --- a/quest/m1/test-flakes-2/handshake-ack.md +++ b/quest/m1/test-flakes-2/handshake-ack.md @@ -6,6 +6,9 @@ The impaired cluster drill connects its publisher and subscriber however the shaper drops their handshake ACKs. `MOQ_SHAPER_SEED=13876846873975217772` `bursts_cross_a_cluster::impaired` (`rs/moq-relay/tests/drills.rs`), which fails on `main` every run with `publisher connect timed out`, passes. +`bursts_cross_a_transport_peer::impaired` failed the same way in the merge +queue on 2026-10-15, under the same shaper. Treat it as the same bug. Open a +separate investigation only if it still fails after this fix lands. ## Plan diff --git a/quest/m2/README.md b/quest/m2/README.md index 01356504a32..d74752286f7 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -74,6 +74,7 @@ ranked just below deterministic TS legs, which it requires. - [Mobile completion](/quest/m2/mobile-completion.md) - verify the Swift and Kotlin bindings over Rust-owned capture before closing #700 - [#709](/quest/m2/709-automatic-letsencrypt-support.md) - the relay provisions and renews its own ACME certificate through rustls-acme over TLS-ALPN-01, persisted on disk - [Catalog colour](/quest/m2/color-catalog.md) - the catalog describes a rendition's colour and HDR properties, which the WebGPU HDR renderer reads +- [Display aspect](/quest/m2/display-aspect.md) - H.264 and H.265 (Annex-B and hvcC) importers report a non-square sample aspect as the catalog display aspect, and AV1 too if it carries one - [WebGPU HDR](/quest/m2/webgpu-hdr.md) - HDR renditions play as HDR where the browser and display can show it, and tone-map to SDR elsewhere - [Grant stats label](/quest/m2/grant-label.md) - an auth grant's optional `label` names what its session's presence and live bytes count under in stats, apart from the path scope it may access - [moq-bench readiness](/quest/m2/bench-ready.md) - every JSONL line carries the rolled targets and a `ready` flag, so a harness opens its window on readiness @@ -90,6 +91,7 @@ ranked just below deterministic TS legs, which it requires. - [TS over RTP legs](/quest/m2/ts-rtp-egress.md) - two legs send identical RTP packets, headers included, so an ST 2022-7 receiver can merge them - [DVB E-AC-3](/quest/m2/ts-eac3.md) - E-AC-3 private data is split per access unit (independent frame plus its dependent substreams) and buffer-modelled in TS export - [T-STD controls](/quest/m2/tstd-controls.md) - the harness gains an MB-overflow control and an AAC broadcast reference +- [moqsrc A/V sync](/quest/m2/moqsrc-av-sync.md) - a nightly test measures A/V skew from `moqsrc` through clock-synced sinks - [MP4 export](/quest/m2/mp4-export.md) - `moq export mp4 --output` records crash-safe fragments, then finishes a regular MP4 with moov at the end - [fMP4 edit lists](/quest/m2/fmp4-edit-lists.md) - the fMP4 importer applies edit lists to frame timestamps - [Linux decoded frames](/quest/m2/obs-decode-linux.md) - present supported native decoded surfaces with visible CPU fallback diff --git a/quest/m2/display-aspect.md b/quest/m2/display-aspect.md new file mode 100644 index 00000000000..c7bbbfd8d2b --- /dev/null +++ b/quest/m2/display-aspect.md @@ -0,0 +1,44 @@ +# [S] Every video importer reports the display aspect + +## Goal + +A video rendition with non-square samples gets the catalog display aspect +from its bitstream, whichever importer reads it: + +- H.264 (the VUI sample aspect) +- H.265 from both Annex-B and an `hvcC` record +- AV1, if it carries a stream-level signal + +A square-sample stream still leaves the fields unset, as the encode hint does +today. + +## Plan + +[#5574](https://github.com/moq-dev/moq/pull/5574) fixed the H.265 Annex-B path +(`config_from_sps`, `rs/moq-mux/src/codec/h265/import.rs`). The remaining +gaps: + +- `config_from_hvcc` sets only the coded size and description. Share the + Annex-B path's VUI read rather than duplicating it. +- The H.264 SPS parser skips `sar_width`/`sar_height`. +- AV1 has no sample aspect in its sequence header. Check whether + `render_size` should count as a display aspect before adding it. If not, + record why and leave AV1 out. +- `MoqVideo.display_aspect` (and `coded`) in `rs/moq-ffi/src/media.rs` have + no doc line. + +Keep #5574's choice that an encode hint fills in only square samples. + +Tests: a non-square fixture per codec path, and a square one that stays +unset. + +Not planned: an upstream scuffle_h265 issue for the unchecked arithmetic in +`cropped_width()`/`cropped_height()`. We never call them, since our SPS +reader crops with checked math. + +Public API: doc only. Wire: the catalog's existing display aspect fields are +populated in more cases. + +## Related + +- [Catalog colour](/quest/m2/color-catalog.md) - parses the same H.264/H.265 VUI for colour diff --git a/quest/m2/moqsrc-av-sync.md b/quest/m2/moqsrc-av-sync.md new file mode 100644 index 00000000000..2dc15be409d --- /dev/null +++ b/quest/m2/moqsrc-av-sync.md @@ -0,0 +1,29 @@ +# [M] moqsrc keeps audio and video in sync through real sinks + +## Goal + +A test plays a broadcast through `moqsrc` into clock-synchronized audio and +video sinks and measures the A/V skew of what they render. It fails when the +skew exceeds the bound that `just test media` holds the browser to. It runs +at least nightly in CI. + +## Plan + +[#5556](https://github.com/moq-dev/moq/pull/5556) fixed `moqsrc` alignment and +tested it at the source pads: buffer timestamps and segment events, not +decoded playback. + +- Reuse the browser media QA's tone/frame pattern fixture + (`test/interop/clients/js/media.ts`, `src/pattern.ts`) rather than + inventing a new marker scheme. +- Read the markers back after decode on a synced pipeline. Today the interop + gst cell runs `moqsrc ! filesink` with no sync. +- Cover a cold start and a mid-stream join. +- Check that the test fails with #5556 reverted. + +Public API: none. Wire: none. + +## Related + +- [moqsrc reconnect](/quest/m1/moqsrc-reconnect.md) - redials and resumes on the same pads, a natural second scenario for this check +- [A/V clock](/quest/m1/av-clock.md) - the browser side of A/V sync diff --git a/rs/moq-net/AGENTS.md b/rs/moq-net/AGENTS.md index cdba32d624a..f2b1f80a8c3 100644 --- a/rs/moq-net/AGENTS.md +++ b/rs/moq-net/AGENTS.md @@ -6,7 +6,7 @@ The wire layer. Generic over the transport and media-agnostic: the relay never i - Closes never cascade. Aborting a group must not tear down its track, broadcast, or siblings. - Sessions are caller-driven: `connect`/`accept` return `(Session, Driver)`. The `Driver` holds no `Session` clone, so spawning it never keeps the session alive. - Version matching: outside the crate branch on `is_lite()` / `is_ietf()` and `alpn()`; inside it match the draft enums. List old drafts explicitly and let the newest behavior be the fallthrough so future drafts fall forward. -- Interop rule of thumb: respond to every feature a peer may use, but don't request new ones. Never emit or accept partial groups; a group starts at frame 0. +- Interop rule of thumb: respond to every feature a peer may use, but don't request new ones. Never emit or accept partial groups; a group starts at frame 0. Exception: moq-transport passthrough forwards a mid-group start as the publisher sent it. - `Timestamp` is an instant with a scale, not a scalar. `checked_add`/`checked_sub` error on mismatched scales; `.convert()` first. `ZERO` is second-scale, so seed a max with `Option`, not `ZERO`. # Testing