From 118dc94f3a7d4fb88bd039d1bb461797d6ba984a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 04:28:35 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m2/hang04/shape.md Quest-Claim: a8c82ea98462ae13 From 9da2f49411e7f5517fd9a50ac67b46801b71308d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 05:20:26 -0800 Subject: [PATCH 2/3] feat(hang): one rule refuses a mis-shaped catalog section A root member the hang draft defines has its shape, or the catalog is refused: `media`, `text`, `json`, `binary`, `stats`, and `echo` no longer read a wrong-shaped value or `null` as absent, a malformed `media.join` no longer decodes as absent, and Rust refuses an array where an entry or section object belongs, as JS already did. The shared parity fixture covers every root key in both languages. The draft states the rule once in its Root section, adopts the code's `delay` rule with a nonzero example, drops the stale Discovery section, describes hang as the media format every gateway shares, and lists `instance` on data tracks. The `jitter` and `delay` doc comments describe what the publisher measures. Co-Authored-By: Claude Opus 5.5 --- dart/moq_ffi/lib/src/moq.dart | 2 +- doc/concept/hang.md | 4 +- drafts/draft-lcurley-moq-hang.md | 71 +- js/hang/api/catalog.api.md | 33 +- js/hang/src/catalog/audio.ts | 24 +- js/hang/src/catalog/binary.ts | 10 +- js/hang/src/catalog/data.test.ts | 25 - js/hang/src/catalog/json.ts | 10 +- js/hang/src/catalog/media.ts | 5 +- js/hang/src/catalog/root.test.ts | 59 -- js/hang/src/catalog/root.ts | 31 +- js/hang/src/catalog/section.ts | 78 +- js/hang/src/catalog/text.ts | 13 +- js/hang/src/catalog/video.ts | 26 +- quest/m2/hang04/README.md | 23 +- quest/m2/hang04/m2ts.md | 4 - quest/m2/hang04/null.md | 30 + quest/m2/hang04/shape.md | 90 -- quest/m2/hang04/sync.md | 17 +- rs/hang/fixtures/catalog-parity.json | 1210 ++++++++++++++++++++++++++ rs/hang/src/catalog/audio/mod.rs | 19 +- rs/hang/src/catalog/binary.rs | 14 +- rs/hang/src/catalog/include.rs | 2 + rs/hang/src/catalog/json.rs | 14 +- rs/hang/src/catalog/media.rs | 19 +- rs/hang/src/catalog/mod.rs | 108 +-- rs/hang/src/catalog/root.rs | 142 +-- rs/hang/src/catalog/text/mod.rs | 13 +- rs/hang/src/catalog/video/mod.rs | 22 +- rs/moq-c/src/publish.rs | 3 +- rs/moq-ffi/src/media.rs | 5 +- rs/moq-mux/src/catalog/hang/ext.rs | 35 +- 32 files changed, 1467 insertions(+), 694 deletions(-) create mode 100644 quest/m2/hang04/null.md delete mode 100644 quest/m2/hang04/shape.md diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index 74bbb704978..eb0fb11460a 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -25344,7 +25344,7 @@ void _checkApiChecksums() { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqmediacatalogproducer_set_section() != - 51040) { + 17264) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqmediacatalogproducer_set_video_properties() != diff --git a/doc/concept/hang.md b/doc/concept/hang.md index fe1b0afc9ba..aa1a730b28a 100644 --- a/doc/concept/hang.md +++ b/doc/concept/hang.md @@ -66,7 +66,7 @@ A few things the catalog can express beyond decoder config: - **Whole includes.** A root `include` map names whole broadcasts, such as transcoded renditions of the source, by the same relative path and optional `instance` as a rendition's reference. They share the listing catalog's PTS timeline and root settings, including its clock and join interval. Listing one opens nothing, so an include produced on demand starts only once a viewer selects it. `@moq/watch` selects every include it lists, as an HLS master does: it reads each include's catalog and offers its video and audio renditions beside the broadcast's own, as `/`, to the same rendition selection, subscribing media only for the rendition it picks. `broadcast.include(effect, id)` reads one include as a broadcast of its own. Its rendition IDs stay its own. - **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition. - **Jitter.** A rendition can say how much media it writes behind a frame's timestamp, in whole milliseconds rounded up: the reorder delay of B-frames and the span of a fragment or batch written together. It is measured from media timestamps alone, never the time a frame was written, so neither a constant encoder delay nor a capture clock counts. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent. -- **Delay.** A rendition can also say how far its frames' timestamps trail those another rendition already wrote, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. A rendition alone measures none. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's. +- **Delay.** A rendition can also say how far its frames' timestamps persistently trail those another rendition already wrote, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. The publisher takes the least any frame trails within each window of a second or more and keeps the largest window, so lateness only some frames have, such as B-frame reorder, is left to jitter. A rendition alone measures none. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's. - **Reorder depth.** A video rendition may declare `reorder`, the most frames that can precede one in decode order and follow it in presentation, as H.264 `max_num_reorder_frames` counts them; `0` declares none, which is not the same as absent. The Rust H.264 and H.265 importers copy it from the SPS when the SPS declares one (progressive H.264 only), and `@moq/publish` declares `0`. The MPEG-TS, Matroska, and FLV exporters then derive each group's decode times from that group's frames alone, so exports that joined at different groups agree, and they end the export on a frame reordered deeper than declared or a depth the codec level cannot reach. Without it they infer decode times from the timestamps they have read, as before. A publisher never changes it once declared. - **Colour.** A video rendition may carry `color`: the H.273 primaries, transfer, matrix, and range, plus the HDR10 mastering display and content light levels, so a renderer can set up HDR without parsing the bitstream. The Rust importers fill it from the SPS VUI, AV1 sequence header, and VP9 header, then fill gaps from a container's `colr` box or Matroska `Colour` element; the stream wins a disagreement. The HDR10 values come from H.264 and H.265 SEI or AV1 metadata OBUs in MPEG-TS and raw elementary streams, an AV1 track's av1C, or Matroska, but not yet from H.264 or H.265 in fMP4 or FLV. Nothing in `@moq/watch` reads it yet. - **Spatial media.** A video rendition may carry `projection` (flat `rectilinear`, or `equirectangular` with the degrees of longitude it spans, 180 for VR180) and `stereo` (`mono`, `left-right`, or `top-bottom` frame packing), and an audio rendition may carry `layout: "ambisonic"` for AmbiX channels. Readers treat a value they don't recognize as absent. The Rust importers fill them from MP4 `sv3d` and `st3d`, Matroska, and Opus channel mapping family 2. `@moq/watch` plays the picture and channels as they are; the [VR demo](/bin/demo#vr) lays them out in a headset. @@ -75,7 +75,7 @@ A few things the catalog can express beyond decoder config: - **Join interval.** The root `media.join.interval` promises that every audio and video track starts a group at least that often: the longest GOP any of them publishes. It is fixed for the broadcast, and the [HLS gateway](/bin/hls) numbers every rendition's segments on the grid it fixes, segments the smallest whole multiple of it at least a second long, from each group's first timestamp, so renditions share segment numbers. A publisher that knows its GOP declares it; Rust publishers (`moq-mux` and everything built on it) without one measure each track over its first second or group, whichever is longer, and `@moq/publish` declares its encoders' `join`. A broadcast without it gets no HLS playlists. - **Clock.** The optional `media.clock` maps PTS zero to wall time so every media track shares one fixed epoch after timescale conversion. Publish independent sources as sibling broadcasts under one prefix, such as `robot/front.hang` and `robot/rear.hang`; each broadcast's audio and video entries are renditions of that source. In one Rust publisher, pass the same `moq_mux::Clock` to each `catalog::Config::with_clock`; JS publishers on one page already share the page clock. Consumers discover the siblings by prefix and align their timestamps through the advertised wall mapping; `@moq/watch` plays one broadcast at a time. - **Version.** The root `version` is the oldest moq-hang draft that reads the catalog correctly, written only when a field needs one newer than 03: a `track` that differs from its ID, or a video `warmup`. Readers ignore fields they don't know, but refuse a `version` above their own with a typed "unsupported catalog version" error, so a player can say to upgrade instead of misreading the catalog. An additive field never raises it. -- **Extensions.** The root is a loose object. Applications add their own sections (`scte35`, for example) next to the ones hang defines, optionally naming a track that carries the data. Every library exposes a way to write your section without clobbering the built-in ones, and readers ignore what they don't know. +- **Extensions.** The root is a loose object. Applications add their own sections next to the ones hang defines, under a namespaced key such as `com.example.scte35`, optionally naming a track that carries the data. Every library exposes a way to write your section without clobbering the built-in ones, and readers ignore what they don't know. A section hang defines must have its shape: a `text`, `stats`, or any other built-in key holding something else refuses the whole catalog rather than reading as absent. ## Media reports diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index c61e9acf4aa..69acb62b33b 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -60,9 +60,9 @@ informative: --- abstract -Hang is a real-time conferencing protocol built on top of moq-lite. -A room consists of multiple participants who publish media tracks. -All updates are live, such as a change in participants or media tracks. +Hang is the media format built on top of moq-lite: a catalog describing a broadcast's audio, video, text, and data tracks, and a container framing each media payload with its timestamp. +It is the format every gateway shares, so media ingested over one protocol can be served over another. +All updates are live, such as a change in media tracks. --- note_Note_to_Readers @@ -95,39 +95,6 @@ Hang introduces additional terminology: - **Container**: A tiny header in front of each media payload containing the timestamp. -# Discovery -The first requirement for a real-time conferencing application is to discover other participants in the same room. -Hang does this using moq-lite's ANNOUNCE capabilities. - -A room consists of a path. -Any participants within the room MUST publish a broadcast with the room path as a prefix which SHOULD end with the `.hang` suffix. - -For example: - -~~~ -/room123/alice.hang -/room123/bob.hang -/room456/zoe.hang -~~~ - -A participant issues an ANNOUNCE_PLEASE message to discover any other participants in the same room. -The server (relay) will then respond with an ANNOUNCE message for any matching broadcasts, including their own. - -For example: - -~~~ -ANNOUNCE_PLEASE prefix=/room/ -ANNOUNCE suffix=alice.hang active=true -ANNOUNCE suffix=bob.hang active=true -~~~ - -If a participant leaves or is disconnected, their broadcast is unannounced. -Publishers and subscribers SHOULD terminate any subscriptions once a participant is unannounced. - -~~~ -ANNOUNCE suffix=alice.hang active=false -~~~ - # Catalog {#catalog} The catalog describes the available media tracks for a single participant. It's a JSON document that extends the W3C WebCodecs specification {{webcodecs}}. @@ -164,8 +131,12 @@ type Catalog = { } ~~~ +Each root member this document defines MUST have the shape this document gives it. +A consumer MUST refuse a catalog in which such a member, or a field within it that this document defines, has another type, is `null`, or is out of range, rather than read the member as absent; only a field this document says to treat as absent is spared. +A section that silently read as empty would hide a publisher bug. + Additional fields MAY be added based on the application. -An application SHOULD name its own root sections with a namespaced key, such as a reverse-DNS name (`com.example.telemetry`), so they cannot collide with a section a later version of this specification defines. +An application MUST NOT carry its own data under a root key this document defines, and SHOULD name its own root sections with a namespaced key, such as a reverse-DNS name (`com.example.telemetry`), so they cannot collide with a section a later version of this specification defines, whose consumers would refuse the catalog. The catalog SHOULD be mostly static, delegating any dynamic content to other tracks. For example, a chat entry should name a chat track, not carry individual chat messages. @@ -200,8 +171,6 @@ type MediaSchema = { } ~~~ -A `media` value that is not an object belongs to another application and is ignored. - ### Clock {#clock} The `clock` field is the broadcast's one continuous clock: @@ -235,7 +204,7 @@ The `interval` field promises that every audio and video track starts a group at It is the grid an edge numbers HLS and DASH segments on ({{hls}}), so it MUST NOT change for the broadcast. A publisher that knows its group duration declares it; one that does not MAY measure each track over its first second or group, whichever is longer, and declare the longest. A group longer than the interval is an overrun, which an edge tolerates. -A consumer that cannot decode `join`, including a zero `interval`, MUST treat it as absent. +A zero `interval` is out of range. Nesting the interval under `join` leaves room for a later join mechanism that is not a fixed grid. ## Includes {#include} @@ -624,6 +593,7 @@ type JsonSchema = { "mode": Mode, "schema": string | undefined, "broadcast": string | undefined, + "instance": string | undefined, "track": string | undefined, "bitrate": number | undefined, "jitter": number | undefined, @@ -640,6 +610,7 @@ type BinarySchema = { "mode": Mode, "mime": string | undefined, "broadcast": string | undefined, + "instance": string | undefined, "track": string | undefined, "bitrate": number | undefined, "jitter": number | undefined, @@ -814,18 +785,22 @@ For video, `warmup` can describe a refresh cycle during which decoding converges For audio, it describes decoder pre-roll, for example 80 milliseconds for Opus, and does not change the group-start rule. ### delay {#field-delay} -The maximum amount, in milliseconds, by which a frame's presentation timestamp has trailed one that another rendition of the broadcast had already written. +How far, in milliseconds, a rendition's frames persistently trail the media other renditions of the broadcast had already written. If absent, a consumer SHOULD assume the rendition does not trail the others. -A publisher compares each frame with the highest timestamp written so far by every other rendition on the same clock, and advertises the largest shortfall observed. +A frame trails by how far the highest timestamp any other rendition on the same clock has written so far is ahead of its presentation timestamp, or by nothing when none is ahead. +A publisher measures over windows spanning at least a second of its rendition's media, or a group if that is longer, and takes the least any frame of a window trails, so one frame that trails nothing clears the window. +It advertises the largest of those values among the windows it has closed. +A constant offset between renditions is in every window, so it shows; lateness that only some frames have, such as B-frame reordering, has a frame trailing nothing in every window, so it never does. A rendition alone on the broadcast measures no `delay`, and a frame trailing only its own rendition's earlier frames counts toward `jitter` instead. A rendition that ends, or breaks its timeline, stops holding the others back. The rounding, `0`, and never-lower rules of `jitter` apply unchanged. A consumer SHOULD hold at least the largest `delay` plus `jitter` among the renditions it plays together. -A consumer MUST NOT subtract one rendition's `delay` from another's: each is a maximum over the life of the stream, so two values need not share an origin. +A consumer MUST NOT subtract one rendition's `delay` from another's: each is measured against whichever rendition was ahead at the time and only ever rises, so two values need not share an origin. -For example, with video at 20 millisecond intervals and audio at 15 written as video 0, video 20, audio 0, audio 15, video 40, audio 30, video 60, audio 45, audio 60, the audio frame at 0 trails video already written at 20, so audio advertises a `delay` of 20 and video none. +For example, with video and audio both at 20 millisecond intervals, written as video 0, video 20, video 40, audio 0, video 60, audio 20, video 80, audio 40, and so on, every audio frame trails video already written 40 milliseconds ahead of it, so audio advertises a `delay` of 40, and video, never behind audio, none. +Had audio been written as video 0, audio 0, video 20, audio 20, and so on, some audio frame in every window would trail nothing, and neither would advertise a `delay`. ### enabled {#field-enabled} The `enabled` field says whether a consumer may select an audio or video rendition; a text rendition does not carry it. @@ -850,8 +825,6 @@ A reader recognizes feedback by the `.echo` suffix on the broadcast's final path The application controls names and publish/subscribe authorization; a report is not authenticated beyond those rights. Without `echo`, no feedback is solicited. -An application may already carry its own `stats` or `echo` root key, so a reader MUST treat a `stats` value without a string `track`, or an `echo` value without a string `path`, as absent rather than refuse the catalog. - Both tracks carry JSON snapshots, and a publisher SHOULD publish them compressed ({{compression}}). Each group starts with a full snapshot as a UTF-8 JSON frame and MAY continue with RFC 7396 merge patches. A producer SHOULD update about once per second while observed; a report need not leave the process when unsubscribed. @@ -886,7 +859,7 @@ Every number is a nonnegative integer no larger than 2^53 - 1; malformed or out- Unknown fields MUST be ignored; application and container sections MAY be flattened beside `transport` and `renditions`. Absent rendition counters mean zero, and zero counters MAY be omitted; JavaScript preserves omission instead of synthesizing fields. Unavailable gauges and transport measurements MUST remain absent, and a known zero MUST remain explicit. -A reader treats JSON `null` in an optional `stats` or `echo` catalog section, gauge, transport measurement, or `arrival` as absent; publishers omit unavailable fields. +A reader treats JSON `null` in an optional gauge, transport measurement, or `arrival` as absent; publishers omit unavailable fields. An `arrival` MUST carry both `timestamp` and `wall`; a reader MUST refuse one missing either. Rendition IDs are the unique keys from the soliciting catalog, independent of media kind and wire track name. A viewer reports referenced renditions under the IDs in the catalog it read, even when their media comes from another broadcast. @@ -1145,8 +1118,8 @@ This document has no IANA actions. - Timed CMAF samples from the frame timestamp, which is the fragment's earliest presentation time; `tfdt` is only relative within the fragment, and places the samples of an untimed frame. A timed CMAF track counts in its `mdhd` ticks. - Replaced the live-edge floor with monotonic group starts: a group may overlap the previous group's content but not start before it. - Added optional `bitrate` and `jitter` fields to `json` and `binary` track entries. -- Added the optional `delay` rendition field: how far a rendition's timestamps trail those other renditions already wrote, never lowered once advertised and never subtracted across renditions. -- Recommended namespaced keys for application root sections. +- Added the optional `delay` rendition field: how far a rendition's timestamps persistently trail those other renditions already wrote, the least trail within each window of a second or a group and the largest of those windows, never lowered once advertised and never subtracted across renditions. +- A root member this draft defines has its shape, or the catalog is refused: a `media`, `text`, `json`, `binary`, `stats`, or `echo` value of the wrong type, or `null`, and a malformed `join`, no longer read as absent. An application keeps its own root data under a namespaced key. - Added the optional `delay` field to `json` and `binary` track entries, measured only from payloads stamped with their capture time on the broadcast clock. - Replaced the video `stalled` field with an optional `enabled` field on audio and video renditions. A consumer MUST NOT select a disabled rendition and ignores `stalled`. A consumer that predates `enabled` keeps selecting a disabled rendition. - Audio `sampleRate` and `numberOfChannels` describe the decoded output, so HE-AAC names its SBR rate and parametric stereo names two channels. diff --git a/js/hang/api/catalog.api.md b/js/hang/api/catalog.api.md index d0c8e83767a..3ca2b3abb1f 100644 --- a/js/hang/api/catalog.api.md +++ b/js/hang/api/catalog.api.md @@ -9,7 +9,6 @@ import { $strip } from "zod/v4/core"; import { Net } from "@moq/hang"; import { ZodMiniBase64 } from "zod/mini"; import { ZodMiniBoolean } from "zod/mini"; -import { ZodMiniCatch } from "zod/mini"; import { ZodMiniCustom } from "zod/mini"; import { ZodMiniDefault } from "zod/mini"; import { ZodMiniDiscriminatedUnion } from "zod/mini"; @@ -280,9 +279,9 @@ export const MediaSchema: ZodMiniObject<{ timescale: ZodMiniDefault>; wall: $ZodBranded, "u53", "out">; }, $strip>>; - join: ZodMiniCatch, "u53", "out">; - }, $strip>>>; + }, $strip>>; }, $strip>; export type Mode = infer; @@ -361,7 +360,7 @@ export const RootSchema: ZodMiniObject<{ warmup?: number & $brand<"u53"> | undefined; }>, Record>>; }, $strip>>; - binary: ZodMiniOptional, Record>, ZodMiniTransform | undefined; @@ -373,15 +372,15 @@ export const RootSchema: ZodMiniObject<{ mode: string; track?: string | undefined; }>, Record>>; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; - echo: ZodMiniOptional>; + echo: ZodMiniOptional>; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; + }, $strip>>; include: ZodMiniOptional, Record>, ZodMiniTransform, Record>>>; - json: ZodMiniOptional, Record>, ZodMiniTransform | undefined; @@ -393,20 +392,20 @@ export const RootSchema: ZodMiniObject<{ schema?: string | undefined; track?: string | undefined; }>, Record>>; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; - media: ZodMiniOptional>; + media: ZodMiniOptional>; wall: $ZodBranded, "u53", "out">; }, $strip>>; - join: ZodMiniCatch, "u53", "out">; - }, $strip>>>; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; - stats: ZodMiniOptional>; + }, $strip>>; + stats: ZodMiniOptional; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; - text: ZodMiniOptional>; + text: ZodMiniOptional, Record>, ZodMiniTransform, Record>>; - }, $strip>, ZodMiniPipe, ZodMiniTransform>]>>; + }, $strip>>; version: ZodMiniOptional>; video: ZodMiniOptional { expect(parsed.json?.tracks.future).toMatchObject({ mode: "windowed", window: 10 }); expect(parsed.binary?.tracks.future).toMatchObject({ mode: "windowed", chunk: "4k" }); }); - -test("a foreign data section does not fail the catalog", () => { - // A section name is only reserved from the version that defines it, and `json` and `binary` are - // generic enough that an application could already be using one. Dropping the section we can't - // read keeps video and audio playable, instead of failing the whole catalog over that key. - for (const section of ["json", "binary"] as const) { - const parsed = RootSchema.parse({ video: { renditions: {} }, [section]: { messages: "chat" } }); - expect(parsed[section]).toBeUndefined(); - expect(parsed.video).toBeDefined(); - } -}); - -test("a foreign section with a non-object tracks value does not fail the catalog", () => { - // The gate is the map's *shape*, not just its presence: an application section that happens to - // carry its own `tracks` scalar is still someone else's key. Rust's `deserialize_section` gates - // on `is_object`, so recognizing this as ours would make the two parsers disagree and take the - // media sections down over a key we never owned. - for (const section of ["json", "binary"] as const) { - for (const tracks of [null, 3, "chat", [], true]) { - const parsed = RootSchema.parse({ video: { renditions: {} }, [section]: { tracks } }); - expect(parsed[section]).toBeUndefined(); - expect(parsed.video).toBeDefined(); - } - } -}); diff --git a/js/hang/src/catalog/json.ts b/js/hang/src/catalog/json.ts index d3fa83b565c..b136b0ec0af 100644 --- a/js/hang/src/catalog/json.ts +++ b/js/hang/src/catalog/json.ts @@ -37,8 +37,8 @@ export const JsonConfigSchema = z.looseObject({ // The maximum bitrate of the track in bits per second, if known. bitrate: z.optional(u53Schema), - // The maximum delay between a payload being ready and the publisher flushing it, in whole - // milliseconds rounded up, with the same meaning as a video rendition's `jitter`. + // The largest media span the publisher hands to the transport behind a payload's timestamp, in + // whole milliseconds rounded up, with the same meaning as a video rendition's `jitter`. jitter: z.optional( z.pipe( u53Schema, @@ -46,9 +46,9 @@ export const JsonConfigSchema = z.looseObject({ ), ), - // How far this track's payloads reach the transport behind the broadcast's earliest rendition, - // with the same meaning and encoding as a video rendition's `delay`. Only measured for payloads - // that carry a capture time. + // How far this track's payloads persistently trail what the broadcast's other tracks already + // wrote, with the same meaning and encoding as a video rendition's `delay`. Only measured for + // payloads that carry a capture time. delay: z.optional( z.pipe( u53Schema, diff --git a/js/hang/src/catalog/media.ts b/js/hang/src/catalog/media.ts index 21ec0eb8ecd..9bf5cab636c 100644 --- a/js/hang/src/catalog/media.ts +++ b/js/hang/src/catalog/media.ts @@ -20,14 +20,13 @@ export type Join = z.infer; /** * What applies to every audio and video track, at the catalog root's `media`. * - * Omitted when it holds nothing. A malformed `clock` fails the catalog. A malformed `join` decodes - * as absent, so it can never reject the media: the broadcast is merely not HLS-renderable. + * Omitted when it holds nothing. A malformed `clock` or `join` fails the catalog. */ export const MediaSchema = z.object({ // The broadcast's one continuous clock, if the publisher exposes one. clock: z.optional(ClockSchema), // Nested so a later join mechanism that is not a fixed grid can sit beside `interval`. - join: z.catch(z.optional(JoinSchema), undefined), + join: z.optional(JoinSchema), }); /** What applies to every audio and video track: the broadcast's clock and join interval. */ diff --git a/js/hang/src/catalog/root.test.ts b/js/hang/src/catalog/root.test.ts index 1abfc205ad0..1dad07e741d 100644 --- a/js/hang/src/catalog/root.test.ts +++ b/js/hang/src/catalog/root.test.ts @@ -217,34 +217,6 @@ test("a malformed clock fails the catalog", () => { } }); -test("a media value that isn't an object is someone else's", () => { - const video = { renditions: { video: { codec: "avc1.64001f", container: { kind: "legacy" } } } }; - for (const media of ["clock", 7, null, [], [{ join: { interval: 2000 } }]]) { - const parsed = RootSchema.parse({ video, media }); - expect(parsed.media).toBeUndefined(); - expect(parsed.video?.renditions.video).toBeDefined(); - } -}); - -test("a malformed join leaves the media playable", () => { - const video = { renditions: { video: { codec: "avc1.64001f", container: { kind: "legacy" } } } }; - const clock = { wall: 1000, timescale: 1_000_000 }; - for (const join of [ - {}, - { interval: 0 }, - { interval: 1.5 }, - { interval: -2000 }, - { interval: "2000" }, - 2000, - null, - ]) { - const parsed = RootSchema.parse({ video, media: { clock, join } }); - expect(parsed.media?.join).toBeUndefined(); - expect(parsed.media?.clock).toMatchObject(clock); - expect(parsed.video?.renditions.video?.codec).toBe("avc1.64001f"); - } -}); - // The root `clock` moved to `media.clock` and the per-rendition `hls` field is gone: a catalog from // an older publisher still parses, with its media intact and both left unread. test("an old root clock and rendition hls are ignored", () => { @@ -284,24 +256,6 @@ test("an old root archive is ignored", () => { expect(parsed.video).toEqual(media.video); }); -test("an absent or foreign text section parses to empty", () => { - // `text` was an ordinary application key before captions reserved it, so a value that isn't - // a section costs its captions and nothing else: video and audio keep playing. - expect(RootSchema.parse({ video: { renditions: {} } }).text).toBeUndefined(); - for (const legacy of ["a caption overlay", ["a", "b"], { overlay: { x: 1 } }, { renditions: 42 }]) { - const parsed = RootSchema.parse({ video: { renditions: {} }, text: legacy }); - expect(parsed.text).toBeUndefined(); - expect(parsed.video).toBeDefined(); - } -}); - -test("a malformed text rendition rejects the catalog", () => { - // The counterpart: the fallback covers someone else's key, not our own bugs. A value that IS - // a text section still has to decode, or a rendition with no format would silently cost a - // publisher its captions in the browser while Rust refuses the same catalog. - expect(() => RootSchema.parse({ text: { renditions: { captions: { container: { kind: "legacy" } } } } })).toThrow(); -}); - const avc = { codec: "avc1.64001f", container: { kind: "legacy" } }; const opus = { codec: "opus", sampleRate: 48000, numberOfChannels: 2, container: { kind: "legacy" } }; @@ -378,16 +332,3 @@ test("shared stats and echo discovery round trips without changing old catalogs" expect(RootSchema.parse(fixture)).toEqual(fixture); expect(RootSchema.parse({})).toEqual({}); }); - -test("a foreign stats or echo value decodes as absent instead of failing the catalog", () => { - for (const foreign of [{}, 5, "on", { track: 5 }, { path: ["a"] }, null]) { - for (const key of ["stats", "echo"]) { - const parsed = RootSchema.parse({ video: { renditions: {} }, [key]: foreign }); - expect(parsed.stats).toBeUndefined(); - expect(parsed.echo).toBeUndefined(); - } - } - const parsed = RootSchema.parse({ stats: { track: "stats.json" }, echo: { path: "viewers" } }); - expect(parsed.stats).toEqual({ track: "stats.json" }); - expect(parsed.echo?.path).toBe("viewers" as RelativeBroadcast); -}); diff --git a/js/hang/src/catalog/root.ts b/js/hang/src/catalog/root.ts index 80f80eb11d4..181e17c944c 100644 --- a/js/hang/src/catalog/root.ts +++ b/js/hang/src/catalog/root.ts @@ -11,7 +11,7 @@ import { JsonSchema } from "./json"; import { MediaSchema } from "./media"; import type { RelativeBroadcast } from "./path"; import { PRIORITY } from "./priority"; -import { named, object, renditions, section } from "./section"; +import { renditions } from "./section"; import { StatsSchema } from "./stats"; import { TextSchema } from "./text"; import { track } from "./track"; @@ -56,7 +56,9 @@ const VersionSchema = z.number().check( * This is a *loose* object: unknown root sections pass through validation untouched, so an * application can add its own sections (e.g. `scte35`) without modifying hang. A base consumer * ignores the extra sections; an extended consumer validates them with its own schema, typically - * built via `z.extend(RootSchema, { ... })`. + * built via `z.extend(RootSchema, { ... })`. An application names its own sections with a + * namespaced key (`com.example.telemetry`): every section defined here must have its shape when + * present, or parsing refuses the whole catalog, the same as Rust. * * The root `version` is the oldest moq-hang draft that reads the catalog correctly, absent while * that is 3. Parsing throws {@link UnsupportedVersion} when it is above {@link VERSION}; a @@ -67,24 +69,13 @@ export const RootSchema = z.looseObject({ version: z.optional(VersionSchema), video: z.optional(VideoSchema), audio: z.optional(AudioSchema), - // What applies to every audio and video track, omitted when it holds nothing. `media` is a - // generic key an application may already carry, so a value that isn't an object decodes as - // absent; an object with a malformed clock fails the catalog, the same as Rust. See `MediaSchema` - // for its lenient `join`. - media: object(MediaSchema), - // Decoded leniently like `text` below: a value without a string `track` or `path` is someone - // else's and decodes as absent. See `named`. - stats: named(StatsSchema, "track"), - echo: named(EchoSchema, "path"), - // `text`, `json`, and `binary` are generic enough keys that an application could have been - // carrying its own before these sections were reserved, so a value that isn't a section decodes - // as absent rather than failing the whole catalog. A value that IS a section but carries a - // malformed entry (a rendition with no `format`, a track with no `mode`) still fails, the same - // as Rust, rather than silently dropping every caption or data track. See `section`. - text: section(TextSchema, "renditions"), - json: section(JsonSchema, "tracks"), - binary: section(BinarySchema, "tracks"), - // A new key, so a malformed entry fails the catalog, the same as Rust. + // What applies to every audio and video track, omitted when it holds nothing. + media: z.optional(MediaSchema), + stats: z.optional(StatsSchema), + echo: z.optional(EchoSchema), + text: z.optional(TextSchema), + json: z.optional(JsonSchema), + binary: z.optional(BinarySchema), include: z.optional(renditions(IncludeSchema, "include")), }); diff --git a/js/hang/src/catalog/section.ts b/js/hang/src/catalog/section.ts index b739c2944ad..8d4442be3d8 100644 --- a/js/hang/src/catalog/section.ts +++ b/js/hang/src/catalog/section.ts @@ -1,46 +1,13 @@ import * as z from "zod/mini"; import { u53Schema } from "./integers"; -/** - * Wrap a catalog section so a value that isn't one decodes as absent rather than failing the - * whole catalog. - * - * A section name is only reserved from the version that defines it, so an application may already - * be carrying its own key under that name. Dropping the one key we can't read keeps the rest of - * the catalog readable, instead of taking video and audio down with it. - * - * The fallback is deliberately narrow: only a value whose `map` key is missing or not an object is - * treated as someone else's. A value that *is* a section but carries a malformed entry still fails, - * since swallowing that would hide a publisher bug (a data track with no `mode`) behind the silence - * that exists to protect an unrelated key. Mirrors the Rust `deserialize_section`, which recognizes - * the section only when the map is an object. - * - * @param schema - the section's own schema, applied when the value looks like a section. - * @param map - the key holding the section's map of tracks (`tracks`, `renditions`). - */ -export function section(schema: T, map: string) { - return z.optional( - z.union([ - schema, - z.pipe( - z.custom((value) => { - if (typeof value !== "object" || value === null || !(map in value)) return true; - const inner = (value as Record)[map]; - return typeof inner !== "object" || inner === null || Array.isArray(inner); - }), - z.transform(() => undefined), - ), - ]), - ); -} - /** * A catalog map keyed by ID (renditions, data tracks, includes), refusing the catalog if any * entry is malformed. * * Each entry must parse with `schema`; the issue names the entry. An unknown container * or extension field is not malformed, so it never refuses the catalog. Mirrors the Rust - * `deserialize_renditions`. + * `deserialize_entries`. * * @param schema - one entry's schema. * @param kind - what an entry is, naming it in the issue (`rendition`, `include`). @@ -84,46 +51,3 @@ export function codecNumber(field: string | undefined, radix: 10 | 16, max: numb const digits = radix === 10 ? /^[0-9]+$/ : /^[0-9a-fA-F]+$/; return digits.test(field) && Number.parseInt(field, radix) <= max; } - -/** - * Like {@link section}, for a section identified by one string field (`stats.track`, `echo.path`) - * rather than a map. Mirrors the Rust `deserialize_named`. - * - * @param schema - the section's own schema, applied when the field is a string. - * @param key - the section's required string field. - */ -export function named(schema: T, key: string) { - return z.optional( - z.union([ - schema, - z.pipe( - z.custom( - (value) => - typeof value !== "object" || - value === null || - typeof (value as Record)[key] !== "string", - ), - z.transform(() => undefined), - ), - ]), - ); -} - -/** - * Like {@link section}, for a section recognized by being an object at all (`media`): any other - * value, `null` and arrays included, is someone else's and decodes as absent, while an object is - * checked by `schema` and fails the catalog when malformed. Mirrors the Rust `deserialize_media`. - * - * @param schema - the section's own schema, applied when the value is an object. - */ -export function object(schema: T) { - return z.optional( - z.union([ - schema, - z.pipe( - z.custom((value) => typeof value !== "object" || value === null || Array.isArray(value)), - z.transform(() => undefined), - ), - ]), - ); -} diff --git a/js/hang/src/catalog/text.ts b/js/hang/src/catalog/text.ts index d7dff58fff1..d2112003b92 100644 --- a/js/hang/src/catalog/text.ts +++ b/js/hang/src/catalog/text.ts @@ -60,8 +60,8 @@ export const TextConfigSchema = z.object({ // The container format, used to decode the timestamp and frame the payload. container: ContainerSchema, - // The maximum jitter before the next cue is flushed, in milliseconds. The player's jitter buffer - // should be at least this large; absent means each cue is flushed immediately. + // The largest media span the publisher hands to the transport behind a cue's timestamp, with + // the same meaning as a video rendition's `jitter`. A cue written in timestamp order adds nothing. jitter: z.optional( z.pipe( u53Schema, @@ -69,10 +69,11 @@ export const TextConfigSchema = z.object({ ), ), - // How far this rendition's frames reach the transport behind the broadcast's earliest - // rendition, in whole milliseconds rounded up. A player holds `delay + jitter` for it and never - // subtracts one rendition's `delay` from another's. Absent on the earliest rendition. It only - // ever grows over the life of a stream. + // How far this rendition's frames persistently trail what the broadcast's other renditions + // already wrote: the least any frame trails within a window of a second or more, and the + // largest of those windows, in whole milliseconds rounded up. A player holds `delay + jitter` + // for it and never subtracts one rendition's `delay` from another's. Absent on a rendition that + // trails nothing. It only ever grows over the life of a stream. delay: z.optional( z.pipe( u53Schema, diff --git a/js/hang/src/catalog/video.ts b/js/hang/src/catalog/video.ts index a04b81d3aad..0b928fb59d6 100644 --- a/js/hang/src/catalog/video.ts +++ b/js/hang/src/catalog/video.ts @@ -190,17 +190,14 @@ export const VideoConfigSchema = z.object({ // Default: true optimizeForLatency: z.optional(z.boolean()), - // The maximum delay between a frame being ready and the publisher flushing it, in whole - // milliseconds rounded up. The player's jitter buffer should be larger than this value. - // If not provided, the player should assume each frame is flushed immediately. + // The largest media span the publisher hands to the transport behind a frame's presentation + // timestamp, in whole milliseconds rounded up: the reorder delay of a frame presented after + // frames decoded behind it, or the span of a batch written together. The player's jitter buffer + // should be at least this large. Absent when the publisher measured none. // - // This is measured at the publisher (encoder latency, segment size, B-frame reordering), - // never on the network a consumer sees. It only ever grows over the life of a stream. - // - // ex: - // - If each frame is flushed immediately, this would be 1000/fps. - // - If there can be up to 3 b-frames in a row, this would be 3 * 1000/fps. - // - If frames are buffered into 2s segments, this would be 2s. + // Measured from media timestamps alone, so a frame written in presentation order adds nothing, + // whatever the frame rate; encoder latency and the network don't count. It only ever grows over + // the life of a stream. jitter: z.optional( z.pipe( u53Schema, @@ -212,10 +209,11 @@ export const VideoConfigSchema = z.object({ // and join that many milliseconds earlier. warmup: z.optional(u53Schema), - // How far this rendition's frames reach the transport behind the broadcast's earliest - // rendition, in whole milliseconds rounded up. A player holds `delay + jitter` for it and never - // subtracts one rendition's `delay` from another's. Absent on the earliest rendition. It only - // ever grows over the life of a stream. + // How far this rendition's frames persistently trail what the broadcast's other renditions + // already wrote: the least any frame trails within a window of a second or more, and the + // largest of those windows, in whole milliseconds rounded up. A player holds `delay + jitter` + // for it and never subtracts one rendition's `delay` from another's. Absent on a rendition that + // trails nothing. It only ever grows over the life of a stream. delay: z.optional( z.pipe( u53Schema, diff --git a/quest/m2/hang04/README.md b/quest/m2/hang04/README.md index e4d7b2aab49..dbbd0a4ffbf 100644 --- a/quest/m2/hang04/README.md +++ b/quest/m2/hang04/README.md @@ -24,18 +24,17 @@ read the catalog as a frozen format: hang-03 is what the datatracker and readers are the first that check `version` at all. Each child's Plan records its own decisions. -The children land in any order. They overlap in three places, so whichever -lands second merges main first: - -- [Entry versions](/quest/m2/hang04/entry-version.md) and - [video warmup](/quest/m2/hang04/video-warmup.md) both edit the version - computation (`min_version` in `rs/hang/src/catalog/root.rs`, `minVersion` - in `js/hang/src/catalog/root.ts`) and the draft's Version section. -- [The sync section](/quest/m2/hang04/sync.md) is born strict, under the - rule [one strict rule](/quest/m2/hang04/shape.md) applies to the sections - that exist today. -- [The MPEG-TS section](/quest/m2/hang04/m2ts.md) removes a root member - that the strict rule would otherwise cover. +The children land in any order. +[Entry versions](/quest/m2/hang04/entry-version.md) and +[video warmup](/quest/m2/hang04/video-warmup.md) both edit the version +computation (`min_version` in `rs/hang/src/catalog/root.rs`, `minVersion` in +`js/hang/src/catalog/root.ts`) and the draft's Version section, so whichever +lands second merges main first. + +Every root section already reads under the hang draft's one rule (a root +member the draft defines has its shape, or the catalog is refused), so +[the sync section](/quest/m2/hang04/sync.md) and +[the MPEG-TS section](/quest/m2/hang04/m2ts.md) are born strict. What the audit judged and kept, so nobody reopens it without new evidence: `include`, `track` apart from the ID, `instance` on references, `enabled`, diff --git a/quest/m2/hang04/m2ts.md b/quest/m2/hang04/m2ts.md index 4288f258531..aa61dd1f59c 100644 --- a/quest/m2/hang04/m2ts.md +++ b/quest/m2/hang04/m2ts.md @@ -64,7 +64,3 @@ description and a passthrough track is refused if they name the same track. Public API: breaking. Rust `hang::catalog::M2ts` and `Catalog::m2ts` move to moq-mux. Wire: the unpublished `m2ts` root section of the catalog. - -## Related - -- [One rule refuses a mis-shaped catalog section](/quest/m2/hang04/shape.md) - `m2ts` is one of the root keys its rule would otherwise cover diff --git a/quest/m2/hang04/null.md b/quest/m2/hang04/null.md new file mode 100644 index 00000000000..60e3f4e8699 --- /dev/null +++ b/quest/m2/hang04/null.md @@ -0,0 +1,30 @@ +# [S] Rust refuses a null catalog field as JS does + +## Goal + +`rs/hang` refuses a catalog in which a field the hang draft defines is +`null`, as `js/hang` already does and the draft's Root and Malformed +Renditions sections require. Today Rust reads `"bitrate": null`, or any +other optional field set to `null`, as absent, so the two languages accept +different catalogs. + +## Plan + +Found while landing the draft's one strict rule: that PR made every root +section and `media`'s `clock` and `join` refuse `null` in Rust +(`deserialize_some` in `rs/hang/src/catalog/mod.rs`), but the +`Option<_>` fields inside renditions, data tracks, includes, and `color` +still take serde's default, where `null` decodes as `None`. JS uses +`z.optional`, which refuses `null`. + +- Pick one mechanism over annotating each field: a field-level + `deserialize_some` breaks the `serde_as` adapters (`MillisCeil`, + `DisplayFromStr`) those fields carry, so prefer a `serde_with` adapter + that wraps them, or a catalog-wide check that walks only the fields the + draft defines. An unknown extension field inside an entry may still be + `null`. +- Add a `null` case per field kind to `rs/hang/fixtures/catalog-parity.json` + so both languages run it. + +Public API: none. Wire: a catalog with a `null` field, which JS already +refuses, is refused by Rust too. diff --git a/quest/m2/hang04/shape.md b/quest/m2/hang04/shape.md deleted file mode 100644 index 5b325a7272d..00000000000 --- a/quest/m2/hang04/shape.md +++ /dev/null @@ -1,90 +0,0 @@ -# [S] One rule refuses a mis-shaped catalog section - -## Goal - -One rule decides what a hang catalog section with the wrong shape means: a -root key the hang draft defines has its shape, or the catalog is refused. -An application keeps its own root data under a namespaced key. `rs/hang` -and `js/hang` implement the one rule, and the draft states it once. The -draft's `delay` definition, its worked example, and the Rust and JS doc -comments for `delay` and `jitter` describe what the publisher measures, and -the stale passages in the hang draft are gone. - -## Plan - -Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit): - -- ✅ One strict rule (rejected: one lenient rule for every key; keep the - per-key rules). Fail loud: a section that silently reads as empty hides a - publisher bug. -- ✅ The draft adopts the code's `delay` rule, with a corrected example and - doc comments (rejected: the code returns to the maximum shortfall). The - code's rule is the one that was measured. - -Facts found by the audit: - -- A mis-shaped section is, by key: refused (`video`, `audio`, `include`, - `m2ts`, `version`); empty unless it is an object (`media`, - `deserialize_media`); empty unless it has its map key (`text`, `json`, - `binary`, `deserialize_section`); absent unless it has a string field - (`stats`, `echo`, `deserialize_named`); absent on any error - (`media.join`, `DefaultOnError`). The decoders are in - `rs/hang/src/catalog/mod.rs`, mirrored by `section`, `named`, and - `object` in `js/hang/src/catalog/section.ts`. -- The lenient rules exist because an application "may already carry" the - key. `media`, `stats`, and `echo` are new since hang-03, added in the - same window as "an application SHOULD name its own root sections with a - namespaced key", and `include`, also new, got the strict rule. `text`, - `json`, and `binary` are in hang-03's root schema, so they were already - reserved. -- `delay` is defined three ways. The draft: "the maximum amount" a frame - trailed another rendition, with an example whose answer is 20. The code - since [moq-dev/moq#7689](https://github.com/moq-dev/moq/pull/7689): the - least a frame trailed within a window, and the largest of those windows - (`rs/moq-mux/src/catalog/estimate.rs`, "a window's delay is the least of - those trails"), which gives 0 on the draft's example. The JS comment - (`js/hang/src/catalog/video.ts`): "behind the broadcast's earliest - rendition". -- `jitter`: the draft says a frame written in presentation order - contributes none "regardless of frame rate", while - `rs/hang/src/catalog/video/mod.rs` and `video.ts` still say "if each - frame is flushed immediately, this would be 1000/fps". - -For the PR: - -- Rule: `stats`, `echo`, `text`, `json`, and `binary` decode strictly. - Delete `deserialize_section`'s and `deserialize_named`'s fallbacks and - their JS twins, keep `deserialize_object`, and replace the draft's - per-key "is ignored" and "treat as absent" sentences with one statement - in the Root section. [The sync section](/quest/m2/hang04/sync.md) and - [the MPEG-TS section](/quest/m2/hang04/m2ts.md) own their keys; whichever - lands second meets the rule already in place. -- Before deleting a fallback, search the tree and the docs for an - application that writes its own value under one of those keys - (`js/room` writes a root `hang` section, which no rule here touches). - One found in a published package is the evidence that would keep that - key lenient; record it. -- `delay`: rewrite the field section to the per-window least and the - largest window, with an example that yields a nonzero value under that - rule, and align the "never subtract one rendition's `delay` from - another's" sentence. Fix the `delay` and `jitter` doc comments on audio - and video in both languages. -- Stale draft text, none of which changes the format: delete Discovery - (it uses `ANNOUNCE_PLEASE` and `active=true`, which lite-07 does not - have, and names rooms `/room/alice.hang` where the Rooms section and - `js/room` use `{identity}/camera.hang`); the abstract's "real-time - conferencing protocol" becomes what the document is, the media format - every gateway shares; `JsonSchema` and `BinarySchema` gain the `instance` - the code carries. -- Not here: the duplicated MPEG-TS SI section goes with - [the MPEG-TS section](/quest/m2/hang04/m2ts.md), the dual-decoder - sentences with [video warmup](/quest/m2/hang04/video-warmup.md), and the - marker rules with - [media consumers](/quest/m2/time-addressing/consumers.md). - -Tests: for each key, a wrong-shaped value refuses the catalog in both -languages (the existing `a_foreign_media_value_is_ignored_but_a_bad_clock_refuses` -style tests flip); `parity.test.ts` covers the same documents. - -Public API: none. Wire: the reading rule for a catalog that is already -wrong; a document that read as "section absent" is now refused. diff --git a/quest/m2/hang04/sync.md b/quest/m2/hang04/sync.md index 90458a616b8..8c16c05ed74 100644 --- a/quest/m2/hang04/sync.md +++ b/quest/m2/hang04/sync.md @@ -58,14 +58,14 @@ For the PR: (`Option`, whole milliseconds rounded up, as `Join::interval` is today), `Catalog::media` becomes `Catalog::sync`, and `Join`, `JoinWire`, and `Media::interval()` go. JS mirrors it: `root.sync`, `SyncSchema`. -- The section is strict from its first day, under the rule - [one strict rule](/quest/m2/hang04/shape.md) applies to the rest: a `sync` - that is not an object, a malformed `clock`, or an `interval` that is not a - positive integer refuses the catalog. Delete `deserialize_media` and its - JS twin, the `DefaultOnError` on the interval, and the draft's "belongs to - another application" and "MUST treat it as absent" sentences. A root - `media` key is then an application's own again, and is ignored like the - retired root `clock` (`a_retired_root_clock_and_hls_field_are_ignored`). +- The section reads under the draft Root section's one rule, as every root + section does (`deserialize_object` for the section and `deserialize_some` + for its members in Rust, plain `z.optional` in JS): a `sync` that is not + an object or is `null`, a malformed `clock`, or an `interval` that is not + a positive integer refuses the catalog. A root `media` key is then an + application's own again, and is ignored like the retired root `clock` + (`a_retired_root_clock_and_hls_field_are_ignored`). Move the `media` cases + in `rs/hang/fixtures/catalog-parity.json` to `sync`. - Draft: rename the Media section and its anchors, fold Join into it, and rewrite the hang-04 changelog lines that describe `media`. The mpegts draft and the HLS section cite the interval by name. @@ -90,4 +90,3 @@ finds no clock in a new catalog. ## Related - [hang applies catalog updates at their timestamp](/quest/m2/catalog-at-timestamp.md) - puts the catalog track itself on this clock -- [One rule refuses a mis-shaped catalog section](/quest/m2/hang04/shape.md) - the rule this section follows from the start diff --git a/rs/hang/fixtures/catalog-parity.json b/rs/hang/fixtures/catalog-parity.json index c64baa31b66..2a062fbe03c 100644 --- a/rs/hang/fixtures/catalog-parity.json +++ b/rs/hang/fixtures/catalog-parity.json @@ -3759,5 +3759,1215 @@ } }, "refused": true + }, + { + "name": "mis-shaped version null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "version": null + }, + "refused": true + }, + { + "name": "mis-shaped version \"4\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "version": "4" + }, + "refused": true + }, + { + "name": "mis-shaped version -1", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "version": -1 + }, + "refused": true + }, + { + "name": "mis-shaped version 1.5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "version": 1.5 + }, + "refused": true + }, + { + "name": "mis-shaped video null", + "catalog": { + "video": null + }, + "refused": true + }, + { + "name": "mis-shaped video 5", + "catalog": { + "video": 5 + }, + "refused": true + }, + { + "name": "mis-shaped video \"x\"", + "catalog": { + "video": "x" + }, + "refused": true + }, + { + "name": "mis-shaped video true", + "catalog": { + "video": true + }, + "refused": true + }, + { + "name": "mis-shaped video []", + "catalog": { + "video": [] + }, + "refused": true + }, + { + "name": "mis-shaped video {}", + "catalog": { + "video": {} + }, + "refused": true + }, + { + "name": "mis-shaped video {\"renditions\": 5}", + "catalog": { + "video": { + "renditions": 5 + } + }, + "refused": true + }, + { + "name": "mis-shaped video {\"renditions\": {\"a\": [\"avc1.64001f\"]}}", + "catalog": { + "video": { + "renditions": { + "a": ["avc1.64001f"] + } + } + }, + "refused": true + }, + { + "name": "mis-shaped audio null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": null + }, + "refused": true + }, + { + "name": "mis-shaped audio 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": 5 + }, + "refused": true + }, + { + "name": "mis-shaped audio \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": "x" + }, + "refused": true + }, + { + "name": "mis-shaped audio true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": true + }, + "refused": true + }, + { + "name": "mis-shaped audio []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": [] + }, + "refused": true + }, + { + "name": "mis-shaped audio {\"renditions\": {\"a\": [\"opus\", 48000, 2]}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "audio": { + "renditions": { + "a": ["opus", 48000, 2] + } + } + }, + "refused": true + }, + { + "name": "mis-shaped media null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": null + }, + "refused": true + }, + { + "name": "mis-shaped media 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": 5 + }, + "refused": true + }, + { + "name": "mis-shaped media \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": "x" + }, + "refused": true + }, + { + "name": "mis-shaped media true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": true + }, + "refused": true + }, + { + "name": "mis-shaped media []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": [] + }, + "refused": true + }, + { + "name": "mis-shaped media {\"clock\": null}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "clock": null + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"clock\": [1000]}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "clock": [1000] + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"clock\": {\"wall\": \"1000\"}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "clock": { + "wall": "1000" + } + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"join\": null}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "join": null + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"join\": [2000]}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "join": [2000] + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"join\": {}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "join": {} + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"join\": {\"interval\": 0}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "join": { + "interval": 0 + } + } + }, + "refused": true + }, + { + "name": "mis-shaped media {\"join\": {\"interval\": 1.5}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "join": { + "interval": 1.5 + } + } + }, + "refused": true + }, + { + "name": "mis-shaped text null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": null + }, + "refused": true + }, + { + "name": "mis-shaped text 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": 5 + }, + "refused": true + }, + { + "name": "mis-shaped text \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": "x" + }, + "refused": true + }, + { + "name": "mis-shaped text true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": true + }, + "refused": true + }, + { + "name": "mis-shaped text []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": [] + }, + "refused": true + }, + { + "name": "mis-shaped text {}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": {} + }, + "refused": true + }, + { + "name": "mis-shaped text {\"overlay\": {\"x\": 1}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": { + "overlay": { + "x": 1 + } + } + }, + "refused": true + }, + { + "name": "mis-shaped text {\"renditions\": 42}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": { + "renditions": 42 + } + }, + "refused": true + }, + { + "name": "mis-shaped text {\"renditions\": []}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": { + "renditions": [] + } + }, + "refused": true + }, + { + "name": "mis-shaped text {\"renditions\": {\"a\": [\"vtt\"]}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "text": { + "renditions": { + "a": ["vtt"] + } + } + }, + "refused": true + }, + { + "name": "mis-shaped json null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": null + }, + "refused": true + }, + { + "name": "mis-shaped json 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": 5 + }, + "refused": true + }, + { + "name": "mis-shaped json \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": "x" + }, + "refused": true + }, + { + "name": "mis-shaped json true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": true + }, + "refused": true + }, + { + "name": "mis-shaped json []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": [] + }, + "refused": true + }, + { + "name": "mis-shaped json {}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": {} + }, + "refused": true + }, + { + "name": "mis-shaped json {\"messages\": \"chat\"}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": { + "messages": "chat" + } + }, + "refused": true + }, + { + "name": "mis-shaped json {\"tracks\": \"chat\"}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": { + "tracks": "chat" + } + }, + "refused": true + }, + { + "name": "mis-shaped json {\"tracks\": {\"a\": [\"snapshot\"]}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "json": { + "tracks": { + "a": ["snapshot"] + } + } + }, + "refused": true + }, + { + "name": "mis-shaped binary null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": null + }, + "refused": true + }, + { + "name": "mis-shaped binary 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": 5 + }, + "refused": true + }, + { + "name": "mis-shaped binary \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": "x" + }, + "refused": true + }, + { + "name": "mis-shaped binary true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": true + }, + "refused": true + }, + { + "name": "mis-shaped binary []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": [] + }, + "refused": true + }, + { + "name": "mis-shaped binary {}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": {} + }, + "refused": true + }, + { + "name": "mis-shaped binary {\"messages\": \"chat\"}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": { + "messages": "chat" + } + }, + "refused": true + }, + { + "name": "mis-shaped binary {\"tracks\": \"chat\"}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": { + "tracks": "chat" + } + }, + "refused": true + }, + { + "name": "mis-shaped binary {\"tracks\": {\"a\": [\"stream\"]}}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "binary": { + "tracks": { + "a": ["stream"] + } + } + }, + "refused": true + }, + { + "name": "mis-shaped stats null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": null + }, + "refused": true + }, + { + "name": "mis-shaped stats 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": 5 + }, + "refused": true + }, + { + "name": "mis-shaped stats \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": "x" + }, + "refused": true + }, + { + "name": "mis-shaped stats true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": true + }, + "refused": true + }, + { + "name": "mis-shaped stats []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": [] + }, + "refused": true + }, + { + "name": "mis-shaped stats {}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": {} + }, + "refused": true + }, + { + "name": "mis-shaped stats {\"track\": 5}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": { + "track": 5 + } + }, + "refused": true + }, + { + "name": "mis-shaped stats [\"stats.json\"]", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "stats": ["stats.json"] + }, + "refused": true + }, + { + "name": "mis-shaped echo null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": null + }, + "refused": true + }, + { + "name": "mis-shaped echo 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": 5 + }, + "refused": true + }, + { + "name": "mis-shaped echo \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": "x" + }, + "refused": true + }, + { + "name": "mis-shaped echo true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": true + }, + "refused": true + }, + { + "name": "mis-shaped echo []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": [] + }, + "refused": true + }, + { + "name": "mis-shaped echo {}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": {} + }, + "refused": true + }, + { + "name": "mis-shaped echo {\"path\": [\"a\"]}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": { + "path": ["a"] + } + }, + "refused": true + }, + { + "name": "mis-shaped echo [\"viewers\"]", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "echo": ["viewers"] + }, + "refused": true + }, + { + "name": "mis-shaped include null", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": null + }, + "refused": true + }, + { + "name": "mis-shaped include 5", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": 5 + }, + "refused": true + }, + { + "name": "mis-shaped include \"x\"", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": "x" + }, + "refused": true + }, + { + "name": "mis-shaped include true", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": true + }, + "refused": true + }, + { + "name": "mis-shaped include []", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": [] + }, + "refused": true + }, + { + "name": "mis-shaped include {\"a\": 5}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": { + "a": 5 + } + }, + "refused": true + }, + { + "name": "mis-shaped include {\"a\": [\"./a.hang\"]}", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "include": { + "a": ["./a.hang"] + } + }, + "refused": true + }, + { + "name": "media without a clock or join", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": {} + }, + "video": ["ok"] + }, + { + "name": "media with a clock and join", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "media": { + "clock": { + "wall": 1000 + }, + "join": { + "interval": 2000 + } + } + }, + "video": ["ok"] + }, + { + "name": "an application's own root keys are ignored", + "catalog": { + "video": { + "renditions": { + "ok": { + "codec": "avc1.64001f" + } + } + }, + "com.example.telemetry": [1, 2], + "com.example.text": null + }, + "video": ["ok"] } ] diff --git a/rs/hang/src/catalog/audio/mod.rs b/rs/hang/src/catalog/audio/mod.rs index 0df4866edfd..0ffb428b7d2 100644 --- a/rs/hang/src/catalog/audio/mod.rs +++ b/rs/hang/src/catalog/audio/mod.rs @@ -36,7 +36,7 @@ pub struct Audio { /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. /// /// A malformed rendition refuses the whole catalog when decoding. - #[serde(deserialize_with = "crate::catalog::deserialize_renditions")] + #[serde(deserialize_with = "crate::catalog::deserialize_entries")] pub renditions: BTreeMap, } @@ -174,18 +174,17 @@ pub struct AudioConfig { #[serde(default)] pub container: Container, - /// The maximum delay between a frame being ready and the publisher flushing it. - /// The player's jitter buffer should be larger than this value. - /// If not provided, the player should assume each frame is flushed immediately. + /// The largest media span the publisher hands to the transport behind a frame's presentation + /// timestamp: the reorder delay of a frame presented after frames decoded behind it, or the span + /// of a batch written together, such as a run of audio packets between video packets in TS. A consumer's jitter buffer should be at + /// least this large. Absent when the publisher measured none. /// - /// This is measured at the publisher (encoder latency, packet packing, reordering), - /// never on the network a consumer sees. It only ever grows over the life of a stream. + /// Measured from media timestamps alone, so a frame written in presentation order adds nothing, + /// whatever the frame rate; encoder latency and the network don't count. It only ever grows + /// over the life of a stream. /// /// Serialized as a whole number of milliseconds, rounded up, so an upper bound never /// rounds down into a promise the publisher can't keep. - /// - /// NOTE: The audio "frame" duration depends on the codec, sample rate, etc. - /// ex: AAC often uses 1024 samples per frame, so at 44100Hz, this would be 1024/44100 = 24ms #[serde_as(as = "MillisCeil")] #[serde(default)] pub jitter: Option, @@ -261,7 +260,7 @@ impl AudioConfig { } } -impl crate::catalog::Rendition for AudioConfig { +impl crate::catalog::Entry for AudioConfig { fn check(&self) -> crate::Result<()> { if self.sample_rate == 0 { return Err(crate::Error::Zero("sampleRate")); diff --git a/rs/hang/src/catalog/binary.rs b/rs/hang/src/catalog/binary.rs index 464e353c19d..914d4a34bcf 100644 --- a/rs/hang/src/catalog/binary.rs +++ b/rs/hang/src/catalog/binary.rs @@ -25,12 +25,11 @@ pub struct Binary { /// /// A map rather than an array so it works with JSON Merge Patch, and a `BTreeMap` so keys are /// sorted for *some* deterministic behavior. + #[serde(deserialize_with = "crate::catalog::deserialize_entries")] pub tracks: BTreeMap, } -impl crate::catalog::Section for Binary { - const MAP: &'static str = "tracks"; -} +impl crate::catalog::Entry for BinaryConfig {} impl Binary { /// Insert a track config, returning an error if the name already exists. @@ -107,14 +106,15 @@ pub struct BinaryConfig { #[serde(default)] pub bitrate: Option, - /// The maximum delay between a payload being ready and the publisher flushing it, with the same - /// meaning and whole-millisecond encoding as [`VideoConfig::jitter`](crate::catalog::VideoConfig::jitter). + /// The largest media span the publisher hands to the transport behind a payload's timestamp, + /// with the same meaning and whole-millisecond encoding as + /// [`VideoConfig::jitter`](crate::catalog::VideoConfig::jitter). #[serde_as(as = "MillisCeil")] #[serde(default)] pub jitter: Option, - /// How far this track's payloads reach the transport behind the broadcast's earliest - /// rendition, with the same meaning and encoding as + /// How far this track's payloads persistently trail what the broadcast's other tracks already + /// wrote, with the same meaning and encoding as /// [`VideoConfig::delay`](crate::catalog::VideoConfig::delay). Only measured for payloads that /// carry a capture time. #[serde_as(as = "MillisCeil")] diff --git a/rs/hang/src/catalog/include.rs b/rs/hang/src/catalog/include.rs index 5d7c893f421..f8ce0df7487 100644 --- a/rs/hang/src/catalog/include.rs +++ b/rs/hang/src/catalog/include.rs @@ -30,6 +30,8 @@ pub struct Include { pub instance: Option, } +impl crate::catalog::Entry for Include {} + impl Include { /// An include at `broadcast`, accepting whichever stream resolves. pub fn new(broadcast: impl Into) -> Self { diff --git a/rs/hang/src/catalog/json.rs b/rs/hang/src/catalog/json.rs index 389b74c78c5..6456adf1664 100644 --- a/rs/hang/src/catalog/json.rs +++ b/rs/hang/src/catalog/json.rs @@ -23,12 +23,11 @@ pub struct Json { /// /// A map rather than an array so it works with JSON Merge Patch, and a `BTreeMap` so keys are /// sorted for *some* deterministic behavior. + #[serde(deserialize_with = "crate::catalog::deserialize_entries")] pub tracks: BTreeMap, } -impl crate::catalog::Section for Json { - const MAP: &'static str = "tracks"; -} +impl crate::catalog::Entry for JsonConfig {} impl Json { /// Insert a track config, returning an error if the name already exists. @@ -105,14 +104,15 @@ pub struct JsonConfig { #[serde(default)] pub bitrate: Option, - /// The maximum delay between a payload being ready and the publisher flushing it, with the same - /// meaning and whole-millisecond encoding as [`VideoConfig::jitter`](crate::catalog::VideoConfig::jitter). + /// The largest media span the publisher hands to the transport behind a payload's timestamp, + /// with the same meaning and whole-millisecond encoding as + /// [`VideoConfig::jitter`](crate::catalog::VideoConfig::jitter). #[serde_as(as = "MillisCeil")] #[serde(default)] pub jitter: Option, - /// How far this track's payloads reach the transport behind the broadcast's earliest - /// rendition, with the same meaning and encoding as + /// How far this track's payloads persistently trail what the broadcast's other tracks already + /// wrote, with the same meaning and encoding as /// [`VideoConfig::delay`](crate::catalog::VideoConfig::delay). Only measured for payloads that /// carry a capture time. #[serde_as(as = "MillisCeil")] diff --git a/rs/hang/src/catalog/media.rs b/rs/hang/src/catalog/media.rs index 5c15718f2b3..f38d08fb60c 100644 --- a/rs/hang/src/catalog/media.rs +++ b/rs/hang/src/catalog/media.rs @@ -7,7 +7,6 @@ use super::millis::MillisCeil; /// What applies to every audio and video track of a broadcast: its clock and how a viewer joins. /// /// Serialized under the catalog root's `media` key, and omitted when it holds nothing. -#[serde_with::serde_as] #[serde_with::skip_serializing_none] #[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, Default)] #[serde(rename_all = "camelCase")] @@ -18,14 +17,11 @@ pub struct Media { /// `wall` is the wall-clock time of PTS zero in `timescale` units since the moq epoch /// (2020-01-01); every media track refers to this mapping after timescale conversion. See /// [`Clock`](crate::catalog::Clock). - #[serde(default)] + #[serde(default, deserialize_with = "crate::catalog::deserialize_some")] pub clock: Option, /// How often a viewer can join every media track, if the publisher promises it. - /// - /// An undecodable value is absent, so it can never reject the media. - #[serde(default)] - #[serde_as(as = "serde_with::DefaultOnError<_>")] + #[serde(default, deserialize_with = "crate::catalog::deserialize_some")] pub join: Option, } @@ -55,8 +51,7 @@ pub struct Join { /// It is also the grid an HLS or DASH export numbers its segments on, so every rendition shares /// segment numbers, and it is fixed for the broadcast instance since that numbering depends on /// it. A broadcast without one is not HLS-renderable. Serialized as whole milliseconds, rounded - /// up. Zero is no promise: [`Join::new`] refuses it, and a reader decodes it as absent, so set - /// the field only to a positive duration. + /// up. Zero is no promise: [`Join::new`] refuses it, and so does a reader. #[serde_as(as = "MillisCeil")] pub interval: Duration, } @@ -79,8 +74,7 @@ impl TryFrom for Join { } impl Join { - /// Promise a group at least every `interval`, or `None` for zero, which readers decode as no - /// promise at all. + /// Promise a group at least every `interval`, or `None` for zero, which readers refuse. pub fn new(interval: Duration) -> Option { (!interval.is_zero()).then_some(Self { interval }) } @@ -121,7 +115,7 @@ mod test { } #[test] - fn an_undecodable_join_is_absent() { + fn an_undecodable_join_refuses() { for join in [ r#"{"interval":0}"#, r#"{"interval":-1}"#, @@ -130,8 +124,7 @@ mod test { r#""2000""#, "null", ] { - let media: Media = serde_json::from_str(&format!(r#"{{"join":{join}}}"#)).unwrap(); - assert_eq!(media.join, None, "{join}"); + serde_json::from_str::(&format!(r#"{{"join":{join}}}"#)).expect_err(join); } } } diff --git a/rs/hang/src/catalog/mod.rs b/rs/hang/src/catalog/mod.rs index 634bf60a76e..fb4d94b2120 100644 --- a/rs/hang/src/catalog/mod.rs +++ b/rs/hang/src/catalog/mod.rs @@ -89,39 +89,37 @@ where serde_json::from_value(serde_json::Value::Object(object)).map_err(serde::de::Error::custom) } -/// Decode the root `media` section, falling back to an empty one when the value is not an object. +/// Decode an optional section or member, which must be an object when present. /// -/// `media` is a generic enough key that an application could have carried its own before it was -/// reserved, so a non-object is someone else's, for the same reason as [`deserialize_section`]. An -/// object is ours and must decode. -pub(crate) fn deserialize_media<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result { - use serde::Deserialize; - let value = serde_json::Value::deserialize(deserializer)?; - if !value.is_object() { - return Ok(Media::default()); - } - serde_json::from_value(value).map_err(serde::de::Error::custom) +/// `null` is not absent: a publisher leaves the section off the wire instead. +pub(crate) fn deserialize_some<'de, D, T>(deserializer: D) -> Result, D::Error> +where + D: serde::Deserializer<'de>, + T: serde::de::DeserializeOwned, +{ + deserialize_object(deserializer).map(Some) } -/// A media rendition config, checked as it decodes. -pub(crate) trait Rendition: serde::de::DeserializeOwned { +/// An entry of a catalog map keyed by ID (a rendition, data track, or include), checked as it +/// decodes. +pub(crate) trait Entry: serde::de::DeserializeOwned { /// Refuse a field out of the range its type allows. - fn check(&self) -> crate::Result<()>; + fn check(&self) -> crate::Result<()> { + Ok(()) + } } -/// Decode a media section's renditions, refusing the catalog if any one is malformed. +/// Decode a catalog map keyed by ID, refusing the catalog if any entry is malformed. /// -/// Each rendition must be an object whose fields are in range; the error names the rendition. +/// Each entry must be an object whose fields are in range; the error names the entry. /// An unknown container or extension field is not malformed, so it never refuses the catalog. -pub(crate) fn deserialize_renditions<'de, D, T>( - deserializer: D, -) -> Result, D::Error> +pub(crate) fn deserialize_entries<'de, D, T>(deserializer: D) -> Result, D::Error> where D: serde::Deserializer<'de>, - T: Rendition, + T: Entry, { use serde::Deserialize; - let renditions = std::collections::BTreeMap::::deserialize(deserializer)?; + let entries = std::collections::BTreeMap::::deserialize(deserializer)?; let decode = |value: serde_json::Value| -> crate::Result { let config: T = deserialize_object(value)?; @@ -129,75 +127,11 @@ where Ok(config) }; - renditions + entries .into_iter() .map(|(id, value)| match decode(value) { Ok(config) => Ok((id, config)), - Err(err) => Err(serde::de::Error::custom(format!("malformed rendition {id}: {err}"))), + Err(err) => Err(serde::de::Error::custom(format!("malformed entry {id}: {err}"))), }) .collect() } - -/// A catalog section: a map of track name to config, published under one well-known root key. -/// -/// The associated [`MAP`](Self::MAP) is what tells a real section apart from an application's own -/// value under the same name. See [`deserialize_section`]. -pub(crate) trait Section: Default + serde::de::DeserializeOwned { - /// The key holding the section's map of tracks. - const MAP: &'static str; -} - -/// Decode a catalog section, falling back to an empty one when the value isn't a section at all. -/// -/// A section name is only reserved from the version that defines it, so an application may already -/// be carrying its own key under that name (through [`Extra`](crate::catalog::Catalog), or in a -/// catalog this build has never seen). Dropping the one key we can't read keeps the rest of the -/// catalog readable, rather than failing the whole document and taking video and audio down with it. -/// -/// The fallback is deliberately narrow: only a value with no [`Section::MAP`] is treated as someone -/// else's. A value that *is* a section but carries a malformed entry still fails, since swallowing -/// that would hide a publisher bug (a data track with no `mode`) behind the silence that exists to -/// protect an unrelated key. -pub(crate) fn deserialize_section<'de, D, T>(deserializer: D) -> Result -where - D: serde::Deserializer<'de>, - T: Section, -{ - use serde::Deserialize; - let value = serde_json::Value::deserialize(deserializer)?; - - if !value.get(T::MAP).is_some_and(serde_json::Value::is_object) { - return Ok(T::default()); - } - - serde_json::from_value(value).map_err(serde::de::Error::custom) -} - -/// Decode an optional section identified by one string field (`stats.track`, `echo.path`). -/// -/// A value without that string field is someone else's key and decodes as absent, for the same -/// reason as [`deserialize_section`]; a value with it is ours and must decode. -fn deserialize_named<'de, D, T>(deserializer: D, key: &str) -> Result, D::Error> -where - D: serde::Deserializer<'de>, - T: serde::de::DeserializeOwned, -{ - use serde::Deserialize; - let value = serde_json::Value::deserialize(deserializer)?; - - if !value.get(key).is_some_and(serde_json::Value::is_string) { - return Ok(None); - } - - serde_json::from_value(value) - .map(Some) - .map_err(serde::de::Error::custom) -} - -pub(crate) fn deserialize_stats<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result, D::Error> { - deserialize_named(deserializer, "track") -} - -pub(crate) fn deserialize_echo<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result, D::Error> { - deserialize_named(deserializer, "path") -} diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 155e2753cd0..e111c075d70 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -18,6 +18,10 @@ use serde::{Deserialize, Serialize}; /// does not deny unknown fields, so a base consumer ignores the extra sections and an extended /// catalog stays wire-compatible. See the `extension_roundtrip` test. /// +/// Every section this crate defines must have its shape when present, or decoding refuses the +/// whole catalog: a section that silently read as empty would hide a publisher bug. An +/// application's own root data goes under a namespaced key (`com.example.telemetry`) instead. +/// /// Marked `#[non_exhaustive]` so a future base section can be added without bumping the major /// version. External callers start from [`Catalog::default`] and fill in the sections they /// publish; struct-literal construction (with or without `..base`) is not available outside @@ -49,25 +53,16 @@ pub struct Catalog { /// Contains a map of text track renditions that the viewer can choose from /// based on their preferences (language, role). Omitted from the wire when empty, so a /// broadcast without captions stays byte-identical to before this section existed. - /// - /// A `text` value that isn't a caption section decodes as empty rather than failing the - /// catalog, since applications could carry their own `text` key before this was reserved. pub text: Text, /// JSON tracks: application data published as live JSON documents or logs. /// /// Omitted from the wire when empty, so a media-only catalog is unchanged. - /// - /// A `json` value that isn't a data-track section decodes as empty rather than failing the - /// catalog, the same as [`text`](Self::text): `json` is a generic enough key that an - /// application could have been carrying its own before this one was reserved, and losing the - /// tracks we can't read beats losing the whole catalog. pub json: Json, /// Binary tracks: application data published as opaque payloads. /// - /// Omitted from the wire when empty, so a media-only catalog is unchanged. Decoded leniently - /// for the same reason as [`json`](Self::json). + /// Omitted from the wire when empty, so a media-only catalog is unchanged. pub binary: Binary, /// A transport stream carried whole rather than demultiplexed: the one track holding its @@ -75,22 +70,16 @@ pub struct Catalog { pub m2ts: Option, /// The publisher's optional media snapshot track. - /// - /// A `stats` value without a string `track` decodes as absent rather than failing the - /// catalog, for the same reason as [`json`](Self::json). pub stats: Option, /// The optional prefix soliciting viewer feedback broadcasts. - /// - /// An `echo` value without a string `path` decodes as absent, the same as [`stats`](Self::stats). pub echo: Option, /// Whole broadcasts listed for explicit, lazy selection, keyed by include ID. /// /// Each shares this catalog's PTS timeline and root settings, including `media.clock` and /// `media.join.interval`; its own settings are ignored. Listing one subscribes to nothing; - /// see [`Include`](crate::catalog::Include). - /// Omitted from the wire when empty. A malformed entry refuses the catalog. + /// see [`Include`](crate::catalog::Include). Omitted from the wire when empty. pub include: std::collections::BTreeMap, /// Application-defined root sections, flattened alongside the base catalog sections. @@ -133,41 +122,45 @@ struct Wire { #[serde( default, skip_serializing_if = "crate::catalog::Media::is_empty", - deserialize_with = "crate::catalog::deserialize_media" + deserialize_with = "crate::catalog::deserialize_object" )] media: crate::catalog::Media, #[serde( default, skip_serializing_if = "Text::is_empty", - deserialize_with = "crate::catalog::deserialize_section" + deserialize_with = "crate::catalog::deserialize_object" )] text: Text, #[serde( default, skip_serializing_if = "Json::is_empty", - deserialize_with = "crate::catalog::deserialize_section" + deserialize_with = "crate::catalog::deserialize_object" )] json: Json, #[serde( default, skip_serializing_if = "Binary::is_empty", - deserialize_with = "crate::catalog::deserialize_section" + deserialize_with = "crate::catalog::deserialize_object" )] binary: Binary, - #[serde(default)] + #[serde(default, deserialize_with = "crate::catalog::deserialize_some")] m2ts: Option, - #[serde(default, deserialize_with = "crate::catalog::deserialize_stats")] + #[serde(default, deserialize_with = "crate::catalog::deserialize_some")] stats: Option, - #[serde(default, deserialize_with = "crate::catalog::deserialize_echo")] + #[serde(default, deserialize_with = "crate::catalog::deserialize_some")] echo: Option, - #[serde(default, skip_serializing_if = "std::collections::BTreeMap::is_empty")] + #[serde( + default, + skip_serializing_if = "std::collections::BTreeMap::is_empty", + deserialize_with = "crate::catalog::deserialize_entries" + )] include: std::collections::BTreeMap, #[serde(flatten, serialize_with = "serialize_ext")] @@ -277,7 +270,7 @@ impl Catalog { /// /// Decoding already refuses an out-of-range rendition, so this catches a catalog built by hand. pub fn check_renditions(&self) -> Result<()> { - use crate::catalog::Rendition; + use crate::catalog::Entry; if let Some(id) = self .video @@ -1043,35 +1036,6 @@ mod test { assert_eq!(catalog, decoded, "text section did not round-trip"); } - #[test] - fn legacy_text_section_keeps_the_catalog() { - // `text` was an ordinary application section before captions reserved it, so a value with - // the wrong shape must cost its captions and nothing else. Audio and video keep playing. - for legacy in [ - r#""a caption overlay""#, - r#"["a","b"]"#, - r#"{"overlay":{"x":1}}"#, - r#"{"renditions":42}"#, - ] { - let json = format!(r#"{{"video":{{"renditions":{{}}}},"audio":{{"renditions":{{}}}},"text":{legacy}}}"#); - let catalog = - Catalog::<()>::from_str(&json).unwrap_or_else(|e| panic!("legacy text {legacy} broke it: {e}")); - assert!(catalog.text.is_empty(), "legacy text {legacy} decoded as captions"); - } - } - - /// The counterpart to the above: the fallback covers someone else's key, not our own bugs. A - /// value that *is* a text section still has to decode, or a rendition with no format would - /// silently cost a publisher every caption it advertised. - #[test] - fn a_malformed_text_section_still_fails() { - let json = r#"{"text":{"renditions":{"captions":{"container":{"kind":"legacy"}}}}}"#; - assert!( - Catalog::<()>::from_str(json).is_err(), - "a format-less text rendition decoded instead of failing" - ); - } - #[test] fn unknown_text_role_keeps_the_catalog() { // A future `role` value must not take down the whole catalog: audio and video have to keep @@ -1089,59 +1053,6 @@ mod test { assert!(!encoded.contains("\"audio\""), "empty audio section leaked: {encoded}"); } - /// A section name is only reserved from the version that defines it, and `json` and `binary` are - /// generic enough that an application could already be using one. Dropping the section we can't - /// read keeps video and audio playable, instead of failing the whole catalog over that key. - #[test] - fn a_foreign_data_section_does_not_fail_the_catalog() { - for section in ["json", "binary"] { - let wire = format!(r#"{{"video":{{"renditions":{{}}}},"{section}":{{"messages":"chat"}}}}"#); - let catalog = - Catalog::<()>::from_str(&wire).unwrap_or_else(|err| panic!("{section} took the catalog down: {err}")); - assert!(catalog.json.is_empty(), "{section}"); - assert!(catalog.binary.is_empty(), "{section}"); - } - } - - /// The counterpart to the above: the fallback covers someone else's key, not our own bugs. A - /// value that *is* a data section still has to decode, or a mode-less track would silently cost - /// a publisher every data track it advertised. - #[test] - fn a_malformed_data_section_still_fails() { - for section in ["json", "binary"] { - let wire = format!(r#"{{"{section}":{{"tracks":{{"chat":{{"schema":"chat"}}}}}}}}"#); - assert!( - Catalog::<()>::from_str(&wire).is_err(), - "a mode-less {section} track decoded instead of failing" - ); - } - } - - /// `stats` and `echo` are as generic as `json`: a value without their string field is someone - /// else's and decodes as absent, while one with it is ours. - #[test] - fn a_foreign_stats_or_echo_value_does_not_fail_the_catalog() { - for foreign in ["{}", "5", r#""on""#, r#"{"track":5}"#, r#"{"path":["a"]}"#, "null"] { - for section in ["stats", "echo"] { - let wire = format!(r#"{{"video":{{"renditions":{{}}}},"{section}":{foreign}}}"#); - let catalog = Catalog::<()>::from_str(&wire) - .unwrap_or_else(|err| panic!("{section}: {foreign} took the catalog down: {err}")); - assert!( - catalog.stats.is_none() && catalog.echo.is_none(), - "{section}: {foreign}" - ); - } - } - let catalog = Catalog::<()>::from_str(r#"{"stats":{"track":"stats.json"},"echo":{"path":"viewers"}}"#).unwrap(); - assert_eq!(catalog.stats, Some(crate::catalog::Stats::new("stats.json"))); - assert_eq!( - catalog.echo, - Some(crate::catalog::Echo::new( - moq_net::path::Relative::new("viewers").unwrap() - )) - ); - } - #[test] fn data_sections_stay_off_the_wire_when_empty() { // Empty sections stay off the wire, video and audio included. @@ -1184,16 +1095,13 @@ mod test { assert_eq!(catalog.to_json().unwrap(), media.to_json().unwrap()); } - /// `media` is a generic key an application may have carried before it was reserved, so a value - /// that is not an object is someone else's; an object holding a malformed clock is ours. + /// A section with the wrong shape refuses the catalog rather than reading as absent. The shared + /// parity fixture holds the cases for every section both languages read; `m2ts` is Rust's alone. #[test] - fn a_foreign_media_value_is_ignored_but_a_bad_clock_refuses() { - let catalog = Catalog::<()>::from_str(r#"{"media":"podcast"}"#).expect("a foreign media value"); - assert!(catalog.media.is_empty()); - Catalog::<()>::from_str(r#"{"media":{"clock":{"wall":0,"timescale":0}}}"#) - .expect_err("a malformed clock refuses the catalog"); - let catalog = Catalog::<()>::from_str(r#"{"media":{"join":{"interval":0}}}"#).expect("a malformed join"); - assert_eq!(catalog.media.join, None); + fn a_mis_shaped_m2ts_refuses_the_catalog() { + for value in ["null", "5", r#""0.m2ts""#, r#"["0.m2ts"]"#, "{}", r#"{"track":5}"#] { + Catalog::<()>::from_str(&format!(r#"{{"m2ts":{value}}}"#)).expect_err(value); + } } #[test] diff --git a/rs/hang/src/catalog/text/mod.rs b/rs/hang/src/catalog/text/mod.rs index 45292ac6349..d4fbc8cce58 100644 --- a/rs/hang/src/catalog/text/mod.rs +++ b/rs/hang/src/catalog/text/mod.rs @@ -28,12 +28,11 @@ pub struct Text { /// A map of track name to rendition configuration. /// This is not an array so it will work with JSON Merge Patch. /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. + #[serde(deserialize_with = "crate::catalog::deserialize_entries")] pub renditions: BTreeMap, } -impl crate::catalog::Section for Text { - const MAP: &'static str = "renditions"; -} +impl crate::catalog::Entry for TextConfig {} impl Text { /// Insert a track config, returning an error if the name already exists. @@ -121,10 +120,10 @@ pub struct TextConfig { #[serde(default)] pub container: Container, - /// The maximum delay, in milliseconds, before the publisher flushes the next cue. A consumer's - /// jitter buffer should be at least this large. Absent means each cue is flushed immediately. - /// - /// Serialized as an integer number of milliseconds, rounded up. + /// The largest media span the publisher hands to the transport behind a cue's timestamp, with + /// the same meaning and whole-millisecond encoding as + /// [`VideoConfig::jitter`](crate::catalog::VideoConfig::jitter). A cue written in timestamp + /// order adds nothing. #[serde_as(as = "MillisCeil")] #[serde(default)] pub jitter: Option, diff --git a/rs/hang/src/catalog/video/mod.rs b/rs/hang/src/catalog/video/mod.rs index e2faf3e660a..9db31d501e0 100644 --- a/rs/hang/src/catalog/video/mod.rs +++ b/rs/hang/src/catalog/video/mod.rs @@ -46,7 +46,7 @@ pub struct Video { /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. /// /// A malformed rendition refuses the whole catalog when decoding. - #[serde(deserialize_with = "crate::catalog::deserialize_renditions")] + #[serde(deserialize_with = "crate::catalog::deserialize_entries")] pub renditions: BTreeMap, /// Render the video at this size in pixels. @@ -316,21 +316,17 @@ pub struct VideoConfig { #[serde(default)] pub container: Container, - /// The maximum delay between a frame being ready and the publisher flushing it. - /// The player's jitter buffer should be larger than this value. - /// If not provided, the player should assume each frame is flushed immediately. + /// The largest media span the publisher hands to the transport behind a frame's presentation + /// timestamp: the reorder delay of a frame presented after frames decoded behind it, or the span + /// of a batch written together, such as an fMP4 fragment. A consumer's jitter buffer should be at + /// least this large. Absent when the publisher measured none. /// - /// This is measured at the publisher (encoder latency, segment size, B-frame - /// reordering), never on the network a consumer sees. It only ever grows over the life of a - /// stream. + /// Measured from media timestamps alone, so a frame written in presentation order adds nothing, + /// whatever the frame rate; encoder latency and the network don't count. It only ever grows + /// over the life of a stream. /// /// Serialized as a whole number of milliseconds, rounded up, so an upper bound never /// rounds down into a promise the publisher can't keep. - /// - /// ex: - /// - If each frame is flushed immediately, this would be 1000/fps. - /// - If there can be up to 3 b-frames in a row, this would be 3 * 1000/fps. - /// - If frames are buffered into 2s segments, this would be 2s. #[serde_as(as = "MillisCeil")] #[serde(default)] pub jitter: Option, @@ -416,7 +412,7 @@ impl VideoConfig { } } -impl crate::catalog::Rendition for VideoConfig { +impl crate::catalog::Entry for VideoConfig { fn check(&self) -> crate::Result<()> { for (field, value) in [ ("codedWidth", self.coded_width), diff --git a/rs/moq-c/src/publish.rs b/rs/moq-c/src/publish.rs index 2436e473864..8c5403ef136 100644 --- a/rs/moq-c/src/publish.rs +++ b/rs/moq-c/src/publish.rs @@ -825,7 +825,8 @@ impl Publish { /// Insert or replace a top-level application catalog section by name. /// /// `value` is any JSON document. Errors if `name` is a HANG root (`video`, `audio`, `media`, `text`, - /// `archive`, `clock`, `json`, `binary`, or retired `timeline`) or an MSF root (`version`, + /// `json`, `binary`, `stats`, `echo`, `include`, `m2ts`, or the retired `archive`, `clock`, and + /// `timeline`) or an MSF root (`version`, /// `generatedAt`, `isComplete`, `tracks`, or `initDataList`). The catalog is republished /// automatically. pub fn catalog_section_set(&mut self, broadcast: Id, name: &str, value: serde_json::Value) -> Result<(), Error> { diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index 7eaacedcf2c..5ac052bdca3 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -868,8 +868,9 @@ impl MoqMediaCatalogProducer { /// Set an application catalog section from JSON, refusing reserved section names. /// /// `json` is any JSON document. Errors with [`MoqError::Json`] if it doesn't parse, or with - /// the reserved-section error if `name` is a HANG root (`video`, `audio`, `media`, `text`, `archive`, `clock`, `json`, `binary`, or - /// retired `timeline`) or an MSF root (`version`, `generatedAt`, `isComplete`, `tracks`, or + /// the reserved-section error if `name` is a HANG root (`video`, `audio`, `media`, `text`, `json`, + /// `binary`, `stats`, `echo`, `include`, `m2ts`, or the retired `archive`, `clock`, and + /// `timeline`) or an MSF root (`version`, `generatedAt`, `isComplete`, `tracks`, or /// `initDataList`). pub fn set_section(&self, name: String, json: String) -> Result<(), MoqError> { let _guard = crate::ffi::enter(); diff --git a/rs/moq-mux/src/catalog/hang/ext.rs b/rs/moq-mux/src/catalog/hang/ext.rs index 1b6e4d8cddb..cb0968a2869 100644 --- a/rs/moq-mux/src/catalog/hang/ext.rs +++ b/rs/moq-mux/src/catalog/hang/ext.rs @@ -40,7 +40,7 @@ impl CatalogExt for () {} /// The default extension stays `()` (unknown sections dropped); opt into `Extra` explicitly. /// /// `version`, `video`, `audio`, `media`, `text`, `archive`, `clock`, `json`, `binary`, `stats`, -/// `echo`, and the retired `timeline` key are reserved, so decoding and [`set`](Self::set) reject +/// `echo`, `include`, `m2ts`, and the retired `timeline` key are reserved, so decoding and [`set`](Self::set) reject /// them to keep the wire JSON free of duplicate or retired keys. The exception is a root `clock`, /// which moved under `media`: an older publisher's is dropped on decode, so its media still plays. #[derive(Serialize, Clone, Default, Debug, PartialEq)] @@ -70,7 +70,8 @@ fn reserved(name: &str) -> bool { | "clock" | "json" | "binary" | "stats" | "echo" - | "timeline" + | "include" + | "m2ts" | "timeline" ) || moq_msf::reserved_root(name) } @@ -97,7 +98,7 @@ impl Extra { /// Set (or replace) a section. Errors if `name` collides with a reserved member of either /// catalog: hang's root `version` and base sections - /// (`video`/`audio`/`media`/`text`/`archive`/`clock`/`json`/`binary`/`stats`/`echo`), + /// (`video`/`audio`/`media`/`text`/`archive`/`clock`/`json`/`binary`/`stats`/`echo`/`include`/`m2ts`), /// the retired `timeline` key that the wire format forbids beside `archive`, or a root member /// MSF defines ([`moq_msf::reserved_root`]). /// @@ -141,23 +142,17 @@ mod test { impl CatalogExt for Scte35Ext {} + /// A section hang defines is hang's, whatever an application puts there: one with the wrong + /// shape refuses the catalog rather than passing to `Extra` or reading as empty. #[test] - fn legacy_text_section_keeps_the_catalog() { - // `Extra` is exactly the mechanism an application would have used to carry its own `text` - // section before captions reserved the name, so an unreadable one must cost its captions - // and nothing else rather than failing the whole decode. - let json = - r#"{"video":{"renditions":{}},"audio":{"renditions":{}},"text":{"overlay":"hi"},"scte35":{"spliceId":7}}"#; - - let catalog: Catalog = serde_json::from_str(json).expect("legacy text section broke the catalog"); - assert!(catalog.text.is_empty()); - assert!(catalog.video.is_empty()); - assert!(catalog.audio.is_empty()); - assert!(catalog.ext.get("scte35").is_some(), "unrelated sections still decode"); - - // Republishing drops the empty media sections the old payload carried. - let output = serde_json::to_string(&catalog).expect("failed to encode"); - assert_eq!(output, r#"{"scte35":{"spliceId":7}}"#); + fn a_mis_shaped_text_section_refuses_the_catalog() { + let json = r#"{"text":{"overlay":"hi"},"scte35":{"spliceId":7}}"#; + serde_json::from_str::>(json).expect_err("a mis-shaped text section"); + + // Republishing drops the empty media sections an older payload carried. + let json = r#"{"video":{"renditions":{}},"audio":{"renditions":{}},"scte35":{"spliceId":7}}"#; + let catalog: Catalog = serde_json::from_str(json).unwrap(); + assert_eq!(serde_json::to_string(&catalog).unwrap(), r#"{"scte35":{"spliceId":7}}"#); } /// A publisher from before `media` writes its clock at the root: that key is dropped rather than @@ -235,7 +230,7 @@ mod test { .set_section("timeline", serde_json::json!({})), Err(crate::Error::ReservedSection(_)) )); - for name in ["media", "clock", "stats", "echo"] { + for name in ["media", "clock", "stats", "echo", "include", "m2ts"] { assert!( matches!( producer.modify().unwrap().set_section(name, serde_json::json!({})), From aec444113685cbd1ec658e142be15928c9f7524b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 12:46:34 -0800 Subject: [PATCH 3/3] docs(hang): namespaced extension examples, rewrapped audio jitter, full reserved list Co-Authored-By: Claude Opus 5.5 --- js/hang/src/catalog/root.ts | 2 +- rs/hang/src/catalog/audio/mod.rs | 5 +++-- rs/hang/src/catalog/root.rs | 2 +- rs/moq-mux/src/catalog/producer.rs | 6 +++--- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/js/hang/src/catalog/root.ts b/js/hang/src/catalog/root.ts index 181e17c944c..3c92f6283a9 100644 --- a/js/hang/src/catalog/root.ts +++ b/js/hang/src/catalog/root.ts @@ -54,7 +54,7 @@ const VersionSchema = z.number().check( * broadcasts without opening them; each shares this catalog's PTS timeline and root settings. * * This is a *loose* object: unknown root sections pass through validation untouched, so an - * application can add its own sections (e.g. `scte35`) without modifying hang. A base consumer + * application can add its own sections (e.g. `com.example.scte35`) without modifying hang. A base consumer * ignores the extra sections; an extended consumer validates them with its own schema, typically * built via `z.extend(RootSchema, { ... })`. An application names its own sections with a * namespaced key (`com.example.telemetry`): every section defined here must have its shape when diff --git a/rs/hang/src/catalog/audio/mod.rs b/rs/hang/src/catalog/audio/mod.rs index 0ffb428b7d2..fc618432b10 100644 --- a/rs/hang/src/catalog/audio/mod.rs +++ b/rs/hang/src/catalog/audio/mod.rs @@ -176,8 +176,9 @@ pub struct AudioConfig { /// The largest media span the publisher hands to the transport behind a frame's presentation /// timestamp: the reorder delay of a frame presented after frames decoded behind it, or the span - /// of a batch written together, such as a run of audio packets between video packets in TS. A consumer's jitter buffer should be at - /// least this large. Absent when the publisher measured none. + /// of a batch written together, such as a run of audio packets between video packets in TS. A + /// consumer's jitter buffer should be at least this large. Absent when the publisher measured + /// none. /// /// Measured from media timestamps alone, so a frame written in presentation order adds nothing, /// whatever the frame rate; encoder latency and the network don't count. It only ever grows diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index e111c075d70..b129ece39dd 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -14,7 +14,7 @@ use serde::{Deserialize, Serialize}; /// transport stream carried whole), plus `stats` naming the publisher's media snapshot track and /// `echo` soliciting viewer feedback under a relative broadcast prefix, and `include` naming /// whole broadcasts for explicit selection. -/// Applications extend it with their own root sections (e.g. `scte35`) through `E`. The catalog +/// Applications extend it with their own root sections (e.g. `com.example.scte35`) through `E`. The catalog /// does not deny unknown fields, so a base consumer ignores the extra sections and an extended /// catalog stays wire-compatible. See the `extension_roundtrip` test. /// diff --git a/rs/moq-mux/src/catalog/producer.rs b/rs/moq-mux/src/catalog/producer.rs index 75795805588..a794a06d887 100644 --- a/rs/moq-mux/src/catalog/producer.rs +++ b/rs/moq-mux/src/catalog/producer.rs @@ -1281,9 +1281,9 @@ impl DerefMut for Guard<'_, E> { impl Guard<'_, Extra> { /// Set (or replace) a top-level application catalog section, republished on drop. /// - /// Errors if `name` is a HANG root (`video`, `audio`, `media`, `text`, `archive`, `clock`, - /// `json`, `binary`, or retired `timeline`) or an MSF root (`version`, `generatedAt`, `isComplete`, - /// `tracks`, or `initDataList`). + /// Errors if `name` is a HANG root (`video`, `audio`, `media`, `text`, `json`, `binary`, + /// `stats`, `echo`, `include`, `m2ts`, or the retired `archive`, `clock`, and `timeline`) or an + /// MSF root (`version`, `generatedAt`, `isComplete`, `tracks`, or `initDataList`). pub fn set_section(&mut self, name: impl Into, value: serde_json::Value) -> crate::Result<()> { self.state.catalog.ext.set(name, value)?; self.updated = true;