From 8e23556bb7f5012d4249c44b4a11b7318ae45096 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 07:10:18 -0800 Subject: [PATCH 1/6] chore(quest): claim quest/m2/hang04/entry-version.md Quest-Claim: 021f288f1dcf08e7 From c3b3b28e76cea959b4434086fe95b73cd2a7d28f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 07:48:23 -0800 Subject: [PATCH 2/6] feat(hang): a player skips a catalog entry newer than it reads Each rendition, data track, and include carries its own computed version. A reader skips an entry whose version is above its own, reports it in Catalog::skipped (Rust) and Catalog.skipped(root) (JS), and plays the rest. The renamed-track and video-warmup triggers move from the root to the entry; the root version still refuses the whole catalog. Co-Authored-By: Claude Opus 5.5 --- doc/concept/hang.md | 2 +- doc/lib/js/hang.md | 2 +- doc/lib/rs/hang.md | 2 +- drafts/draft-lcurley-moq-hang.md | 22 +- js/hang/api/catalog.api.md | 12 + js/hang/src/catalog/parity.test.ts | 9 +- js/hang/src/catalog/root.test.ts | 101 +++++++- js/hang/src/catalog/root.ts | 120 ++++++--- js/hang/src/catalog/section.ts | 42 ++- js/hang/src/catalog/version.ts | 22 ++ js/publish/src/catalog.test.ts | 10 +- js/watch/src/broadcast.ts | 12 +- quest/m2/hang04/README.md | 6 +- quest/m2/hang04/entry-version.md | 76 ------ quest/m2/hang04/video-warmup.md | 6 +- quest/m4/intra-refresh/catalog-warmup.md | 5 +- rs/hang/fixtures/catalog-parity.json | 192 ++++++++++++++ rs/hang/src/catalog/audio/mod.rs | 11 +- rs/hang/src/catalog/binary.rs | 17 +- rs/hang/src/catalog/include.rs | 2 + rs/hang/src/catalog/json.rs | 17 +- rs/hang/src/catalog/mod.rs | 106 ++++++-- rs/hang/src/catalog/reference.rs | 6 + rs/hang/src/catalog/root.rs | 316 ++++++++++++++++------- rs/hang/src/catalog/skipped.rs | 225 ++++++++++++++++ rs/hang/src/catalog/text/mod.rs | 10 + rs/hang/src/catalog/video/mod.rs | 14 +- 27 files changed, 1085 insertions(+), 280 deletions(-) create mode 100644 js/hang/src/catalog/version.ts delete mode 100644 quest/m2/hang04/entry-version.md create mode 100644 rs/hang/src/catalog/skipped.rs diff --git a/doc/concept/hang.md b/doc/concept/hang.md index fe1b0afc9ba..155177866d0 100644 --- a/doc/concept/hang.md +++ b/doc/concept/hang.md @@ -74,7 +74,7 @@ A few things the catalog can express beyond decoder config: - **Preference.** A video rendition may carry a signed `preference`, default `0`. A player keeps only the highest preference it can decode, then picks by size and bitrate within it, as do the single-rendition egresses (RTMP play, WHEP, FLV, the transcoder's source). A publisher that adds its own H.264 copy of an H.265 source marks the copy `-1`, so only a viewer that can't decode H.265 pulls it. Picking a rendition by name still overrides it. The field is written only when nonzero. - **Join interval.** The root `media.join.interval` promises that every audio and video track starts a group at least that often: the longest GOP any of them publishes. It is fixed for the broadcast, and the [HLS gateway](/bin/hls) numbers every rendition's segments on the grid it fixes, segments the smallest whole multiple of it at least a second long, from each group's first timestamp, so renditions share segment numbers. A publisher that knows its GOP declares it; Rust publishers (`moq-mux` and everything built on it) without one measure each track over its first second or group, whichever is longer, and `@moq/publish` declares its encoders' `join`. A broadcast without it gets no HLS playlists. - **Clock.** The optional `media.clock` maps PTS zero to wall time so every media track shares one fixed epoch after timescale conversion. Publish independent sources as sibling broadcasts under one prefix, such as `robot/front.hang` and `robot/rear.hang`; each broadcast's audio and video entries are renditions of that source. In one Rust publisher, pass the same `moq_mux::Clock` to each `catalog::Config::with_clock`; JS publishers on one page already share the page clock. Consumers discover the siblings by prefix and align their timestamps through the advertised wall mapping; `@moq/watch` plays one broadcast at a time. -- **Version.** The root `version` is the oldest moq-hang draft that reads the catalog correctly, written only when a field needs one newer than 03: a `track` that differs from its ID, or a video `warmup`. Readers ignore fields they don't know, but refuse a `version` above their own with a typed "unsupported catalog version" error, so a player can say to upgrade instead of misreading the catalog. An additive field never raises it. +- **Version.** A `version` is the oldest moq-hang draft that reads what carries it correctly. Each rendition, data track, and include carries its own, written only when it uses a field newer than 03 that an older reader would misread: a `track` that differs from its ID, or a video `warmup`. A reader skips an entry whose `version` is above its own and plays the rest, so a new-style rendition beside a compatibility one costs an older player only the new one, and the parsed catalog lists what was skipped so a player can say to upgrade. The root `version` is for a change to the root itself, which nothing in 04 makes: a reader refuses a catalog whose root `version` is above its own with a typed "unsupported catalog version" error. An additive field never raises either. - **Extensions.** The root is a loose object. Applications add their own sections (`scte35`, for example) next to the ones hang defines, optionally naming a track that carries the data. Every library exposes a way to write your section without clobbering the built-in ones, and readers ignore what they don't know. ## Media reports diff --git a/doc/lib/js/hang.md b/doc/lib/js/hang.md index ab9f35a116b..4e713188ac0 100644 --- a/doc/lib/js/hang.md +++ b/doc/lib/js/hang.md @@ -11,7 +11,7 @@ description: The media layer in TypeScript The [hang media format](/concept/hang) as TypeScript types and codecs, shared by [`@moq/watch`](/lib/js/watch) and [`@moq/publish`](/lib/js/publish). -- **Catalog**: zod schemas for the root, renditions, data tracks, and `archive`. The root is a loose object, so `z.extend(Catalog.RootSchema, { yourSection })` adds your own. `Catalog.watch(broadcast)` iterates validated catalog updates, and parsing throws `Catalog.UnsupportedVersion` on a catalog that needs a newer moq-hang draft. `Catalog.Audio.ranked(renditions)` and `Catalog.Video.ranked(renditions)` order renditions best first, with enabled renditions ahead; `` uses the video order. +- **Catalog**: zod schemas for the root, renditions, data tracks, `media`, and includes. The root is a loose object, so `z.extend(Catalog.RootSchema, { yourSection })` adds your own. `Catalog.watch(broadcast)` iterates validated catalog updates, and parsing throws `Catalog.UnsupportedVersion` on a catalog that needs a newer moq-hang draft; an entry that needs one is skipped instead, and `Catalog.skipped(root)` lists it. `Catalog.Audio.ranked(renditions)` and `Catalog.Video.ranked(renditions)` order renditions best first, with enabled renditions ahead; `` uses the video order. - **Containers**: `Container.Legacy` producer/consumer and separate `@moq/hang/cmaf` decoding. Import the CMAF module only when reading fMP4. - **Utilities**: priority and latency math, a libav polyfill (Opus, plus AAC decoding) for runtimes without native WebCodecs audio, and the browser quirks the media packages work around. diff --git a/doc/lib/rs/hang.md b/doc/lib/rs/hang.md index 1662ba86bf0..76d6952be01 100644 --- a/doc/lib/rs/hang.md +++ b/doc/lib/rs/hang.md @@ -12,7 +12,7 @@ The [hang media format](/concept/hang) on top of `moq-net`: the catalog that describes renditions with WebCodecs-style decoder configs, and the frame that carries a timestamp with each payload. -- **Catalog.** `Catalog` types video, audio, and text renditions, the `json` and `binary` data tracks, and the `archive` entry. Add your own sections through its type parameter, or read unknown ones as raw JSON. `Catalog::subscribe(&broadcast)` follows it live, failing with `Error::UnsupportedVersion` on a catalog that needs a newer moq-hang draft. +- **Catalog.** `Catalog` types video, audio, and text renditions, the `json` and `binary` data tracks, the `media` section, and includes. Add your own sections through its type parameter, or read unknown ones as raw JSON. `Catalog::subscribe(&broadcast)` follows it live, failing with `Error::UnsupportedVersion` on a catalog that needs a newer moq-hang draft; an entry that needs one is skipped instead and listed in `Catalog::skipped`. - **Frames.** `container::Frame` encodes and decodes the `legacy` container, and `container::track_info` declares a media track. A rendition may instead declare `cmaf` or `loc`; an unknown container kind is kept rather than failing the catalog. - **Codecs described**: H.264, H.265, VP8, VP9, AV1, AAC, Opus, PCM, FLAC, MP3, MP2, AC-3, E-AC-3. diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index c61e9acf4aa..a6b9bd9b5d6 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -157,6 +157,7 @@ type Catalog = { "json": JsonTracks | undefined, "binary": BinaryTracks | undefined, "include": Record | undefined, @@ -172,18 +173,22 @@ For example, a chat entry should name a chat track, not carry individual chat me This way catalog updates are rare and a client MAY choose to not subscribe. The `json` and `binary` sections ({{data}}) are how a track like that is listed. -This specification defines audio, video, and text media tracks, an optional `media` section ({{media}}) holding the clock that maps content time to wall time and the interval a viewer can join on, the root `version` ({{catalog-version}}), the application data tracks in {{data}}, optional publisher statistics and solicited viewer feedback ({{media-stats}}), and whole broadcasts listed in `include` ({{include}}). +This specification defines audio, video, and text media tracks, an optional `media` section ({{media}}) holding the clock that maps content time to wall time and the interval a viewer can join on, the `version` of the catalog and of each entry ({{catalog-version}}), the application data tracks in {{data}}, optional publisher statistics and solicited viewer feedback ({{media-stats}}), and whole broadcasts listed in `include` ({{include}}). ## Version {#catalog-version} -The root `version` field is the oldest draft of this specification that reads the catalog correctly, by draft number: `4` needs a consumer of moq-hang-04 or later. +The `version` field is the oldest draft of this specification that reads what carries it correctly, by draft number: `4` needs a consumer of moq-hang-04 or later. If absent, it is 3. +The root carries one for the whole catalog, and each entry of a map keyed by ID (an audio, video, or text rendition, a data track, or an include) carries one for itself. A consumer MUST ignore fields it does not recognize. -A consumer MUST refuse a catalog whose `version` is above the draft it implements, or is not a non-negative integer, rather than read it; one that refuses SHOULD tell its user to upgrade. +A consumer MUST refuse a catalog whose root `version` is above the draft it implements, or any `version` that is not a non-negative integer, rather than read it; one that refuses SHOULD tell its user to upgrade. +A consumer MUST read an entry's `version` before the rest of the entry, and MUST skip an entry whose `version` is above the draft it implements: it reads the catalog as if that entry were absent, so the entry is never malformed. +One that skips an entry SHOULD tell its user that an upgrade would show more. -A publisher writes the highest draft whose fields the catalog uses that an older consumer would misread, and omits `version` while that is 3. -A change raises the version only when a consumer that ignored it would misread the catalog, never for a field it can safely ignore. -This draft needs `version` 4 for: +A publisher writes on each entry the highest draft whose fields the entry uses that an older consumer would misread, and on the root the highest such draft for the root's own fields, omitting each `version` while it is 3. +A field one entry carries raises only that entry's `version`, so an older consumer still plays the rest of the catalog, including a compatibility rendition beside the new one. +A change raises a version only when a consumer that ignored it would misread the catalog, never for a field it can safely ignore. +This draft writes no root `version`, and needs an entry `version` of 4 for: - a `track` ({{field-track}}) other than the entry's ID, which an older consumer subscribes to instead; - a video `warmup` ({{field-warmup}}) above zero, since an older consumer starts decoding a group at a picture that is not a keyframe. @@ -621,6 +626,7 @@ A consumer discovers these tracks the same way it discovers media, by reading th ### JSON {#json-tracks} ~~~ type JsonSchema = { + "version": number | undefined, "mode": Mode, "schema": string | undefined, "broadcast": string | undefined, @@ -637,6 +643,7 @@ It is descriptive: a consumer that does not recognize it MUST still be able to r ### Binary {#binary-tracks} ~~~ type BinarySchema = { + "version": number | undefined, "mode": Mode, "mime": string | undefined, "broadcast": string | undefined, @@ -708,6 +715,7 @@ Audio, video, and text renditions share the following fields, extending the WebC ~~~ type CommonExtensions = { + "version": number | undefined, "broadcast": string | undefined, "instance": string | undefined, "track": string | undefined, @@ -1123,7 +1131,7 @@ This document has no IANA actions. - Replaced the root `archive` section and its per-track timelines with the root `media` section ({{media}}). Its `join.interval` is the longest group of any audio or video track, fixed for the broadcast, and the grid an edge numbers HLS and DASH segments on, each segment the smallest whole multiple of it at least a second long, from each group's first timestamp ({{hls}}); the root `clock` moved to `media.clock`. Segment indexes, frame positions, `end`, `keyframe`, `timescale`, `replay`, `replayEpoch`, `store`, and `version` are gone. - A reader refuses a JSON frame in which an object repeats a member name. - A malformed audio or video rendition ({{malformed}}) refuses the catalog: a known codec string without valid parameters, a zero or out-of-range dimension, sample rate, or channel count, a bitrate or duration above 2^53 - 1, or a container without a string `kind`. -- Added the root `version` ({{catalog-version}}): the oldest draft that reads the catalog, which a consumer refuses when above its own. A `track` other than the entry's ID or a video `warmup` needs 4. Consumers MUST ignore unknown fields. +- Added `version` on the root and on each rendition, data track, and include ({{catalog-version}}): the oldest draft that reads it. A consumer refuses a catalog whose root `version` is above its own, and skips an entry whose `version` is. A `track` other than the entry's ID or a video `warmup` needs an entry `version` of 4. Consumers MUST ignore unknown fields. - A data track marks a discontinuity with a group of one object with no bytes, since its empty payload is data. An object with no bytes on an audio or video track, or sharing its group, is malformed. - Removed the Recording section: the store layout is a moq-archive format, documented with that crate, not part of this draft. - Removed the viewer feedback `latency` field. diff --git a/js/hang/api/catalog.api.md b/js/hang/api/catalog.api.md index d0c8e83767a..e806c132ce0 100644 --- a/js/hang/api/catalog.api.md +++ b/js/hang/api/catalog.api.md @@ -523,6 +523,16 @@ export const RootSchema: ZodMiniObject<{ }, $strip>>; }, $loose>; +export interface Skipped { + audio: Record; + binary: Record; + include: Record; + json: Record; + text: Record; + version: number; + video: Record; +} + export type Stats = infer; export const StatsSchema: ZodMiniObject<{ @@ -835,6 +845,8 @@ export function modeSupported(mode: Mode): mode is KnownMode; export const nonzeroU53Schema: $ZodBranded, "u53", "out">; +export function skipped(root: Root): Skipped | undefined; + export function track(id: string, config: { track?: string; }): string; diff --git a/js/hang/src/catalog/parity.test.ts b/js/hang/src/catalog/parity.test.ts index a33d90219a7..a75149895ab 100644 --- a/js/hang/src/catalog/parity.test.ts +++ b/js/hang/src/catalog/parity.test.ts @@ -1,5 +1,5 @@ import { expect, test } from "bun:test"; -import { checkRenditions, RootSchema } from "./root.ts"; +import { checkRenditions, RootSchema, skipped } from "./root.ts"; interface Case { name: string; @@ -11,10 +11,12 @@ interface Case { json?: string[]; binary?: string[]; include?: string[]; + skipped?: Record>; } // The corpus rs/hang also loads: both languages accept the same catalogs and keep the same -// rendition, data track, and include IDs, so a divergence fails CI on whichever side drifts. +// rendition, data track, and include IDs, and skip the same newer entries, so a divergence fails CI +// on whichever side drifts. const cases = (await Bun.file( new URL("../../../../rs/hang/fixtures/catalog-parity.json", import.meta.url), ).json()) as Case[]; @@ -36,5 +38,8 @@ for (const c of cases) { expect(ids(root.json?.tracks)).toEqual(c.json ?? []); expect(ids(root.binary?.tracks)).toEqual(c.binary ?? []); expect(ids(root.include)).toEqual(c.include ?? []); + const { version: _, ...sections } = skipped(root) ?? { version: 0 }; + const nonempty = Object.entries(sections).filter(([, entries]) => Object.keys(entries).length > 0); + expect(Object.fromEntries(nonempty)).toEqual(c.skipped ?? {}); }); } diff --git a/js/hang/src/catalog/root.test.ts b/js/hang/src/catalog/root.test.ts index 1abfc205ad0..c83ef56bda8 100644 --- a/js/hang/src/catalog/root.test.ts +++ b/js/hang/src/catalog/root.test.ts @@ -2,7 +2,16 @@ import { expect, test } from "bun:test"; import * as z from "zod/mini"; import { u53 } from "./integers.ts"; import type { ReferenceInstance, RelativeBroadcast } from "./path.ts"; -import { checkRenditions, DuplicateRendition, encode, RootSchema, UnsupportedVersion, VERSION } from "./root.ts"; +import { + checkRenditions, + DuplicateRendition, + encode, + type Root, + RootSchema, + skipped, + UnsupportedVersion, + VERSION, +} from "./root.ts"; import { track } from "./track.ts"; // The base catalog carries the media sections (`video`/`audio`) and the data track sections @@ -355,20 +364,92 @@ test("a supported catalog version is read", () => { } }); -// Only a field an older reader would misread raises the version. -test("the catalog version is needed only for misread fields", () => { +// Only a field an older reader would misread raises an entry's version, and no moq-hang-04 field +// raises the root's. +test("an entry version is needed only for misread fields", () => { const audio = { ...opus, warmup: 80 }; const root = (hd: object) => - RootSchema.parse({ video: { renditions: { hd } }, audio: { renditions: { en: audio } } }); + RootSchema.parse({ + video: { renditions: { hd } }, + audio: { renditions: { en: audio } }, + include: { extra: { broadcast: "../extra" } }, + }); + const versions = (value: Root) => { + expect(value.version).toBeUndefined(); + const entry = (map: Record | undefined, id: string) => + (map?.[id] as { version?: number } | undefined)?.version; + return { + hd: entry(value.video?.renditions, "hd"), + en: entry(value.audio?.renditions, "en"), + extra: entry(value.include, "extra"), + }; + }; + const none = { hd: undefined, en: undefined, extra: undefined }; + + expect(versions(encode(root(avc)))).toEqual(none); + expect(versions(encode(root({ ...avc, track: "hd" })))).toEqual(none); + expect(versions(encode(root({ ...avc, track: "video" })))).toEqual({ ...none, hd: 4 }); + expect(versions(encode(root({ ...avc, warmup: 500 })))).toEqual({ ...none, hd: 4 }); + expect(JSON.stringify(encode(root({ ...avc, track: "video" })))).toContain('"hd":{"version":4,'); - expect(encode(root(avc)).version).toBeUndefined(); - expect(encode(root({ ...avc, track: "hd" })).version).toBeUndefined(); - expect(encode(root({ ...avc, track: "video" })).version).toBe(4); - expect(encode(root({ ...avc, warmup: 500 })).version).toBe(4); const data = RootSchema.parse({ json: { tracks: { chat: { mode: "stream", track: "chat.json" } } } }); - expect(encode(data).version).toBe(4); + expect((encode(data).json?.tracks.chat as { version?: number } | undefined)?.version).toBe(4); + // A carried version is recomputed, never trusted. - expect(encode({ ...root(avc), version: 4 }).version).toBeUndefined(); + const carried = { + ...root(avc), + version: 4, + json: { tracks: { chat: { mode: "stream", version: 4 } } }, + } as Root; + expect(JSON.stringify(encode(carried))).not.toContain("version"); +}); + +// An entry a newer draft defines is skipped and reported, even one this reader would call +// malformed, while the rest of the catalog plays. +test("a newer entry is skipped", () => { + const catalog = { + video: { renditions: { compat: avc, next: { version: 5, codec: 42 } } }, + audio: { renditions: { next: { version: 6 } } }, + text: { renditions: { next: { version: 5 } } }, + json: { tracks: { next: { version: 5 } } }, + binary: { tracks: { next: { version: 5, mode: [] } } }, + include: { next: { version: 5, broadcast: 7 } }, + }; + const parsed = RootSchema.parse(catalog); + expect(Object.keys(parsed.video?.renditions ?? {})).toEqual(["compat"]); + expect(parsed.audio?.renditions).toEqual({}); + expect(parsed.include).toEqual({}); + expect(skipped(parsed)).toEqual({ + version: 6, + video: { next: 5 }, + audio: { next: 6 }, + text: { next: 5 }, + json: { next: 5 }, + binary: { next: 5 }, + include: { next: 5 }, + }); + + // Never written, so a republish lists only what this reader read. + expect(skipped(RootSchema.parse(JSON.parse(JSON.stringify(encode(parsed)))))).toBeUndefined(); + + // The root version still refuses the whole catalog. + expect(() => RootSchema.parse({ ...catalog, version: 5 })).toThrow(UnsupportedVersion); +}); + +test("a supported entry version is read, and a malformed one refuses the catalog", () => { + for (const version of [0, 3, VERSION]) { + const parsed = RootSchema.parse({ video: { renditions: { hd: { ...avc, version } } } }); + expect(Object.keys(parsed.video?.renditions ?? {})).toEqual(["hd"]); + expect(skipped(parsed)).toBeUndefined(); + } + for (const version of ["5", -1, 4.5, null, []]) { + expect(() => RootSchema.parse({ json: { tracks: { chat: { mode: "stream", version } } } })).toThrow( + z.core.$ZodError, + ); + } + // The entry version is not kept as an extension field. + const data = RootSchema.parse({ json: { tracks: { chat: { mode: "stream", version: 4 } } } }); + expect(data.json?.tracks.chat).toEqual({ mode: "stream" }); }); test("shared stats and echo discovery round trips without changing old catalogs", async () => { diff --git a/js/hang/src/catalog/root.ts b/js/hang/src/catalog/root.ts index 80f80eb11d4..98b751ee367 100644 --- a/js/hang/src/catalog/root.ts +++ b/js/hang/src/catalog/root.ts @@ -11,32 +11,18 @@ import { JsonSchema } from "./json"; import { MediaSchema } from "./media"; import type { RelativeBroadcast } from "./path"; import { PRIORITY } from "./priority"; -import { named, object, renditions, section } from "./section"; +import { named, object, renditions, SKIPPED, section } from "./section"; import { StatsSchema } from "./stats"; import { TextSchema } from "./text"; import { track } from "./track"; +import { UnsupportedVersion, VERSION, VersionSchema } from "./version"; import { VideoSchema } from "./video"; -/** The newest moq-hang draft this package reads; a catalog whose `version` is higher is refused. */ -export const VERSION = 4; - -/** A catalog needs a newer moq-hang draft than this reader implements. */ -export class UnsupportedVersion extends Error { - /** The catalog version that was read. */ - readonly version: number; - - constructor(version: number) { - super(`unsupported catalog version ${version}: upgrade to read it`); - this.name = "UnsupportedVersion"; - this.version = version; - } -} +export { UnsupportedVersion, VERSION } from "./version"; // Thrown rather than reported as an issue, so it surfaces as itself and before any section, which // the newer draft may have reshaped. -const VersionSchema = z.number().check( - z.int(), - z.nonnegative(), +const RootVersionSchema = VersionSchema.check( z.refine((version) => { if (Number.isInteger(version) && version > VERSION) throw new UnsupportedVersion(version); return true; @@ -59,12 +45,14 @@ const VersionSchema = z.number().check( * built via `z.extend(RootSchema, { ... })`. * * The root `version` is the oldest moq-hang draft that reads the catalog correctly, absent while - * that is 3. Parsing throws {@link UnsupportedVersion} when it is above {@link VERSION}; a - * publisher sets it with {@link encode}. + * that is 3, and parsing throws {@link UnsupportedVersion} when it is above {@link VERSION}. Each + * entry (rendition, data track, include) carries its own `version` the same way, and parsing skips + * an entry whose `version` is above {@link VERSION}; {@link skipped} reports it. A publisher writes + * the entry versions with {@link encode}. */ export const RootSchema = z.looseObject({ // First, so it is checked before any section. - version: z.optional(VersionSchema), + version: z.optional(RootVersionSchema), video: z.optional(VideoSchema), audio: z.optional(AudioSchema), // What applies to every audio and video track, omitted when it holds nothing. `media` is a @@ -91,26 +79,77 @@ export const RootSchema = z.looseObject({ /** The root catalog object: the media sections, `media`, the data track sections, plus any app extensions. */ export type Root = z.infer; +/** The catalog entries a reader skipped because a newer moq-hang draft defines them. */ +export interface Skipped { + /** The draft a reader needs to read every skipped entry. */ + version: number; + /** Skipped video renditions, mapping each ID to its `version`. */ + video: Record; + /** Skipped audio renditions. */ + audio: Record; + /** Skipped text renditions. */ + text: Record; + /** Skipped JSON tracks. */ + json: Record; + /** Skipped binary tracks. */ + binary: Record; + /** Skipped includes. */ + include: Record; +} + /** - * The oldest moq-hang draft that reads `root` correctly, or undefined while that is 3. + * The entries parsing `root` skipped because a newer moq-hang draft defines them, or undefined if + * it read them all. The rest of the catalog reads as if they were absent, so a player plays what + * it can and may tell its user to upgrade for the rest. * - * Only a field an older reader would misread raises it; one it would merely ignore does not. + * Reads what {@link RootSchema} recorded on each parsed map, so pass the parsed value itself, not + * a copy. Mirrors Rust's `Catalog::skipped`. */ -function minVersion(root: Root): number | undefined { +export function skipped(root: Root): Skipped | undefined { + const of = (map: object | undefined) => (map && SKIPPED.get(map)) ?? {}; + const sections = { + video: of(root.video?.renditions), + audio: of(root.audio?.renditions), + text: of(root.text?.renditions), + json: of(root.json?.tracks), + binary: of(root.binary?.tracks), + include: of(root.include), + }; + const versions = Object.values(sections).flatMap((section) => Object.values(section)); + if (versions.length === 0) return undefined; + return { version: Math.max(...versions), ...sections }; +} + +/** + * The oldest moq-hang draft that reads an entry correctly given its ID, or undefined while that is + * 3. Only a field an older reader would misread raises it; one it would merely ignore does not. + */ +function entryVersion(id: string, entry: { track?: string; warmup?: number }): number | undefined { // moq-hang-04: a reader that ignores `track` subscribes to the ID, a different track. - const renamed = [ - root.video?.renditions, - root.audio?.renditions, - root.text?.renditions, - root.json?.tracks, - root.binary?.tracks, - ].some((entries) => Object.entries(entries ?? {}).some(([id, config]) => track(id, config) !== id)); + if (track(id, entry) !== id) return 4; + return undefined; +} +/** Video adds `warmup` to {@link entryVersion}. */ +function videoVersion(id: string, entry: { track?: string; warmup?: number }): number | undefined { // moq-hang-04: a reader that ignores video `warmup` assumes every group starts at a keyframe, // and hands its decoder one that starts at a converging picture instead. - const warmup = Object.values(root.video?.renditions ?? {}).some((config) => (config.warmup ?? 0) > 0); + if ((entry.warmup ?? 0) > 0) return 4; + return entryVersion(id, entry); +} - return renamed || warmup ? 4 : undefined; +/** Rewrite a map's entries with their computed `version` first, dropping any they carried. */ +function versioned( + entries: Record, + version: (id: string, entry: T) => number | undefined, +): Record { + return Object.fromEntries( + Object.entries(entries).map(([id, entry]) => { + const { version: _, ...rest } = entry as T & { version?: unknown }; + const computed = version(id, entry); + return [id, computed === undefined ? rest : { version: computed, ...rest }]; + }), + ) as Record; } /** A catalog update used one rendition ID for both a video and an audio rendition. */ @@ -134,9 +173,10 @@ export function checkRenditions(root: Root): Root { } /** - * The catalog as a publisher writes it: refused as every reader refuses it, with the root - * `version` computed rather than carried, so an edit can't claim less than the catalog uses, and an - * empty `media` left off the wire. + * The catalog as a publisher writes it: refused as every reader refuses it, with each entry's + * `version` computed rather than carried, so an edit can't claim less than an entry uses, and an + * empty `media` left off the wire. No moq-hang-04 field reshapes the root, so the root `version` is + * never written. * * Throws {@link DuplicateRendition}. Mirrors Rust's catalog serializer. */ @@ -145,8 +185,12 @@ export function encode(root: Root): Root { const value = { ...root }; delete value.version; if (value.media && value.media.clock === undefined && value.media.join === undefined) delete value.media; - const version = minVersion(root); - if (version !== undefined) value.version = version; + if (value.video) value.video = { ...value.video, renditions: versioned(value.video.renditions, videoVersion) }; + if (value.audio) value.audio = { ...value.audio, renditions: versioned(value.audio.renditions, entryVersion) }; + if (value.text) value.text = { ...value.text, renditions: versioned(value.text.renditions, entryVersion) }; + if (value.json) value.json = { ...value.json, tracks: versioned(value.json.tracks, entryVersion) }; + if (value.binary) value.binary = { ...value.binary, tracks: versioned(value.binary.tracks, entryVersion) }; + if (value.include) value.include = versioned(value.include, () => undefined); return value; } diff --git a/js/hang/src/catalog/section.ts b/js/hang/src/catalog/section.ts index b739c2944ad..0696d93a506 100644 --- a/js/hang/src/catalog/section.ts +++ b/js/hang/src/catalog/section.ts @@ -1,5 +1,6 @@ import * as z from "zod/mini"; import { u53Schema } from "./integers"; +import { VERSION, VersionSchema } from "./version"; /** * Wrap a catalog section so a value that isn't one decodes as absent rather than failing the @@ -34,13 +35,17 @@ export function section(schema: T, map: string) { ); } +/** The entries each parsed map skipped, keyed by the map itself. Read through `skipped` in `root.ts`. */ +export const SKIPPED = new WeakMap>(); + /** - * A catalog map keyed by ID (renditions, data tracks, includes), refusing the catalog if any - * entry is malformed. + * A catalog map keyed by ID (renditions, data tracks, includes), skipping an entry newer than this + * reader and refusing the catalog if any other entry is malformed. * - * Each entry must parse with `schema`; the issue names the entry. An unknown container - * or extension field is not malformed, so it never refuses the catalog. Mirrors the Rust - * `deserialize_renditions`. + * An entry's `version` is read before the rest of it, since a newer draft may have reshaped the + * entry; one above {@link VERSION} is skipped and recorded in {@link SKIPPED}. Every other entry + * must parse with `schema`; the issue names the entry. An unknown container or extension field is + * not malformed, so it never refuses the catalog. Mirrors the Rust `deserialize_entries`. * * @param schema - one entry's schema. * @param kind - what an entry is, naming it in the issue (`rendition`, `include`). @@ -53,8 +58,29 @@ export function renditions(schema: T, kind = "rendition ), z.transform((raw, ctx) => { const kept: [string, z.infer][] = []; + const skipped: [string, number][] = []; for (const [id, value] of Object.entries(raw)) { - const result = schema.safeParse(value); + let entry = value; + if (typeof value === "object" && value !== null && Object.hasOwn(value, "version")) { + // Taken out rather than left to the entry: written versions are computed, never carried. + const { version, ...rest } = value as Record; + const checked = VersionSchema.safeParse(version); + if (!checked.success) { + ctx.issues.push({ + code: "custom", + message: `malformed ${kind} ${id}: ${checked.error.message}`, + input: value, + }); + continue; + } + if (checked.data > VERSION) { + skipped.push([id, checked.data]); + continue; + } + entry = rest; + } + + const result = schema.safeParse(entry); if (result.success) kept.push([id, result.data]); else ctx.issues.push({ @@ -64,7 +90,9 @@ export function renditions(schema: T, kind = "rendition }); } // fromEntries defines each key as its own property, so `__proto__` stays an ID. - return Object.fromEntries(kept) as Record>; + const map = Object.fromEntries(kept) as Record>; + if (skipped.length > 0) SKIPPED.set(map, Object.fromEntries(skipped)); + return map; }), ); } diff --git a/js/hang/src/catalog/version.ts b/js/hang/src/catalog/version.ts new file mode 100644 index 00000000000..d06b50af2c8 --- /dev/null +++ b/js/hang/src/catalog/version.ts @@ -0,0 +1,22 @@ +import * as z from "zod/mini"; + +/** + * The newest moq-hang draft this package reads. A catalog whose root `version` is higher is + * refused, and an entry whose `version` is higher is skipped. + */ +export const VERSION = 4; + +/** A catalog needs a newer moq-hang draft than this reader implements. */ +export class UnsupportedVersion extends Error { + /** The catalog version that was read. */ + readonly version: number; + + constructor(version: number) { + super(`unsupported catalog version ${version}: upgrade to read it`); + this.name = "UnsupportedVersion"; + this.version = version; + } +} + +/** A `version` member: the oldest moq-hang draft that reads what carries it, absent while that is 3. */ +export const VersionSchema = z.number().check(z.int(), z.nonnegative()); diff --git a/js/publish/src/catalog.test.ts b/js/publish/src/catalog.test.ts index e50c26e196a..d423897acfb 100644 --- a/js/publish/src/catalog.test.ts +++ b/js/publish/src/catalog.test.ts @@ -53,7 +53,7 @@ test("catalog producer refuses an ID naming both a video and an audio rendition" ).toThrow(Catalog.DuplicateRendition); }); -test("catalog producer computes the version from the fields it uses", async () => { +test("catalog producer computes each entry version from the fields it uses", async () => { const catalog = new CatalogProducer(); const effect = new Effect(); const track = new Track.Producer("catalog.json"); @@ -61,17 +61,19 @@ test("catalog producer computes the version from the fields it uses", async () = const consumer = new Json.Snapshot.Consumer({ track }); const video = { codec: "avc1.64001f", container: { kind: "legacy" as const } }; - // A caller's own version is replaced by the computed one. + // A caller's own version is replaced by the computed ones: the renamed rendition's, not the root's. catalog.mutate((c) => { c.video = { renditions: { hd: { ...video, track: "video" } } }; c.version = 3; }); - expect((await consumer.latest())?.value.version).toBe(4); + const renamed = (await consumer.latest())?.value; + expect(renamed).not.toHaveProperty("version"); + expect(renamed?.video?.renditions.hd).toHaveProperty("version", 4); catalog.mutate((c) => { c.video = { renditions: { hd: video } }; }); - expect((await consumer.latest())?.value).not.toHaveProperty("version"); + expect((await consumer.latest())?.value.video?.renditions.hd).not.toHaveProperty("version"); effect.close(); }); diff --git a/js/watch/src/broadcast.ts b/js/watch/src/broadcast.ts index f86aa9e62ac..245d914b3b7 100644 --- a/js/watch/src/broadcast.ts +++ b/js/watch/src/broadcast.ts @@ -581,7 +581,10 @@ export class Broadcast { accepted = catalog && Catalog.checkResolvable(Catalog.checkRenditions(Catalog.RootSchema.parse(catalog)), name); - if (accepted) status = "live"; + if (accepted) { + status = "live"; + warnSkipped(name, accepted); + } } catch (err) { if (err instanceof Catalog.UnsupportedVersion) status = "unsupported"; console.error("rejecting catalog", name, err); @@ -650,6 +653,7 @@ export class Broadcast { this.#raw.set(Catalog.checkResolvable(Catalog.checkRenditions(update), name), true); this.#progress.set("live"); + warnSkipped(name, update); } } catch (err) { if (err instanceof Catalog.UnsupportedVersion) status = "unsupported"; @@ -809,3 +813,9 @@ export class Broadcast { this.#children.clear(); } } + +// The rest of the catalog plays; an entry a newer draft defines only costs the viewer that entry. +function warnSkipped(name: Moq.Path.Valid, root: Catalog.Root) { + const skipped = Catalog.skipped(root); + if (skipped) console.warn("catalog entries need moq-hang draft", skipped.version, "to read", name, skipped); +} diff --git a/quest/m2/hang04/README.md b/quest/m2/hang04/README.md index e4d7b2aab49..6a50f847c41 100644 --- a/quest/m2/hang04/README.md +++ b/quest/m2/hang04/README.md @@ -24,13 +24,9 @@ read the catalog as a frozen format: hang-03 is what the datatracker and readers are the first that check `version` at all. Each child's Plan records its own decisions. -The children land in any order. They overlap in three places, so whichever +The children land in any order. They overlap in two places, so whichever lands second merges main first: -- [Entry versions](/quest/m2/hang04/entry-version.md) and - [video warmup](/quest/m2/hang04/video-warmup.md) both edit the version - computation (`min_version` in `rs/hang/src/catalog/root.rs`, `minVersion` - in `js/hang/src/catalog/root.ts`) and the draft's Version section. - [The sync section](/quest/m2/hang04/sync.md) is born strict, under the rule [one strict rule](/quest/m2/hang04/shape.md) applies to the sections that exist today. diff --git a/quest/m2/hang04/entry-version.md b/quest/m2/hang04/entry-version.md deleted file mode 100644 index c2c143549e6..00000000000 --- a/quest/m2/hang04/entry-version.md +++ /dev/null @@ -1,76 +0,0 @@ -# [M] A player skips a catalog entry newer than it reads - -## Goal - -A hang catalog entry (a rendition, a data track, an include) can carry its -own `version`: the oldest moq-hang draft that reads that entry correctly. A -reader skips an entry whose `version` is above its own and plays the rest, -and can tell its user that something was skipped. The root `version` stays, -and still refuses the whole catalog, for a change that reshapes the root. -hang-04's one remaining trigger, a `track` that differs from the entry's ID, -moves from the root to the entry. `rs/hang`, `js/hang`, and the hang draft -agree. - -## Plan - -Decided 2026-11-16 (maintainer, /quest-plan of the pre-cut audit): - -- ✅ Skip per entry (rejected: keep the root gate and tell publishers never - to mix new-style and compatibility renditions in one catalog; keep as - is). Both hang-04 triggers are per-rendition features, so one such - rendition takes the whole catalog from an older reader, including the - compatibility rendition the draft tells publishers to add. - -Facts found by the audit: - -- A reader "MUST refuse a catalog whose `version` is above the draft it - implements" (the draft's Version section). `min_version` in - `rs/hang/src/catalog/root.rs` and `minVersion` in - `js/hang/src/catalog/root.ts` raise it to 4 for a renamed `track` or a - video `warmup`, both properties of one entry. -- [moq-dev/moq#5806](https://github.com/moq-dev/moq/pull/5806) saw this - cost and avoided it for `enabled` ("a live rendition toggle would flip - the whole catalog unreadable"). -- The draft already has the gentler tool: an unknown `container.kind`, - text `format`, or data `mode` ignores that entry alone. -- Published 0.21 readers check no `version`, so 4 protects nobody deployed - today. 0.22 is the first reader that checks, and the rule it ships is the - only forward-compatibility lever every later catalog change gets. - -Recommendations for the PR: - -- The entry member is named `version`, an integer, absent while it is 3. - Both serializers compute it from the entry, as the root's is computed - today, so a caller cannot claim less than the entry uses. -- A reader checks an entry's `version` before it validates the entry, as - `Probe` reads the root's before any section: a newer entry may have a - shape this reader would call malformed, and the malformed-rendition rule - refuses the whole catalog. -- A skipped entry is reported, not silent: the parsed catalog says which - IDs were skipped and the highest version among them, the per-entry twin - of `Error::UnsupportedVersion` and JS `UnsupportedVersion`, so a player - can say "upgrade to see everything". -- With [video warmup](/quest/m2/hang04/video-warmup.md) out of hang-04, the - renamed `track` is the only trigger, and nothing in hang-04 raises the - root `version`. Keep the root check and its tests: it is what a reshaped - root will use. -- An include is an entry too. Decide whether a skipped include is reported - the same way, and say so in the draft. -- A reference that an entry makes to a skipped entry (if any section can) - is the case to look for before settling the rule. - -Tests, in both languages and in the parity fixtures: a catalog holding a -compatibility rendition and a version-5 rendition reads as the -compatibility rendition alone and reports the skip; a version-5 entry that -would be malformed to this reader is skipped, not refused; a root -`version` of 5 still refuses; a duplicate `version` member in an entry is -refused as the root's is. - -Public API: additive on the parsed catalog (the skipped-entries report); -the computed root `version` changes for a catalog with a renamed `track`. -Wire: the hang-04 catalog JSON, unpublished. - -## Related - -- [Video warmup leaves hang-04](/quest/m2/hang04/video-warmup.md) - removes the other trigger from the same lines -- [Audio languages](/quest/m3/audio-lang.md) - a later rendition a 0.22 player should skip can be marked with this diff --git a/quest/m2/hang04/video-warmup.md b/quest/m2/hang04/video-warmup.md index 8ae3d5de379..e8640a2112b 100644 --- a/quest/m2/hang04/video-warmup.md +++ b/quest/m2/hang04/video-warmup.md @@ -48,8 +48,9 @@ Facts found by the audit: Firefox and Safari are unverified, and Chromium refuses a non-IRAP H.265 frame sent as `key`, so an HEVC refresh rendition fails ([second decoder](/quest/m4/intra-refresh/dual-decoder.md)). -- The draft makes video `warmup` above zero one of two triggers for - `version` 4 (`min_version`, `rs/hang/src/catalog/root.rs`; `minVersion`, +- The draft makes video `warmup` above zero one of two triggers for an + entry `version` of 4 (`VideoConfig`'s `Entry::version`, + `rs/hang/src/catalog/video/mod.rs`; `videoVersion`, `js/hang/src/catalog/root.ts`), and lets a video group start at "a picture from which decoding converges within that duration" (the Container section's group rules). @@ -99,4 +100,3 @@ catalog JSON and group rule, unpublished. ## Related - [Video warmup returns with its producer](/quest/m4/intra-refresh/catalog-warmup.md) - restores this from history as the next catalog version -- [A player skips a catalog entry newer than it reads](/quest/m2/hang04/entry-version.md) - how a 0.22 player will refuse that later rendition; edits the same version lines diff --git a/quest/m4/intra-refresh/catalog-warmup.md b/quest/m4/intra-refresh/catalog-warmup.md index c7e6c3948bd..f4ff14f8453 100644 --- a/quest/m4/intra-refresh/catalog-warmup.md +++ b/quest/m4/intra-refresh/catalog-warmup.md @@ -27,8 +27,9 @@ rendition. It returns here, with the producers this epic adds. - The field comes back in the hang draft's next version, with the group rule that lets a video group start at a picture from which decoding converges. It raises the entry's `version`, not the root's (the - per-entry rule hang-04 ships, planned as `quest/m2/hang04/entry-version.md`), - so a player that cannot play it keeps the other renditions. + per-entry rule hang-04 ships, `Entry::version` in `rs/hang` and + `videoVersion` in `js/hang`), so a player that cannot play it keeps the + other renditions. - Audio `warmup` never left; reuse `moq_mux::container::Playout` and the JS consumer's `warmup` reach as they are. - The draft says which engines play it. The Firefox and Safari check in diff --git a/rs/hang/fixtures/catalog-parity.json b/rs/hang/fixtures/catalog-parity.json index c64baa31b66..2e6d98cf15e 100644 --- a/rs/hang/fixtures/catalog-parity.json +++ b/rs/hang/fixtures/catalog-parity.json @@ -3759,5 +3759,197 @@ } }, "refused": true + }, + { + "name": "a newer entry is skipped", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "next": { + "version": 5, + "codec": 42 + } + } + }, + "audio": { + "renditions": { + "next": { + "version": 6 + } + } + }, + "text": { + "renditions": { + "next": { + "version": 5 + } + } + }, + "json": { + "tracks": { + "next": { + "version": 5 + } + } + }, + "binary": { + "tracks": { + "next": { + "version": 5, + "mode": [] + } + } + }, + "include": { + "next": { + "version": 5, + "broadcast": 7 + } + } + }, + "video": ["compat"], + "skipped": { + "video": { + "next": 5 + }, + "audio": { + "next": 6 + }, + "text": { + "next": 5 + }, + "json": { + "next": 5 + }, + "binary": { + "next": 5 + }, + "include": { + "next": 5 + } + } + }, + { + "name": "a supported entry version is read", + "catalog": { + "video": { + "renditions": { + "zero": { + "version": 0, + "codec": "avc1.64001f" + }, + "four": { + "version": 4, + "codec": "avc1.64001f" + } + } + } + }, + "video": ["four", "zero"] + }, + { + "name": "a newer root version refuses despite entry versions", + "catalog": { + "version": 5, + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "next": { + "version": 5 + } + } + } + }, + "refused": true + }, + { + "name": "entry version \"5\" refuses the catalog", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "bad": { + "version": "5", + "codec": "avc1.64001f" + } + } + } + }, + "refused": true + }, + { + "name": "entry version -1 refuses the catalog", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "bad": { + "version": -1, + "codec": "avc1.64001f" + } + } + } + }, + "refused": true + }, + { + "name": "entry version 4.5 refuses the catalog", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "bad": { + "version": 4.5, + "codec": "avc1.64001f" + } + } + } + }, + "refused": true + }, + { + "name": "entry version null refuses the catalog", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "bad": { + "version": null, + "codec": "avc1.64001f" + } + } + } + }, + "refused": true + }, + { + "name": "entry version [] refuses the catalog", + "catalog": { + "video": { + "renditions": { + "compat": { + "codec": "avc1.64001f" + }, + "bad": { + "version": [], + "codec": "avc1.64001f" + } + } + } + }, + "refused": true } ] diff --git a/rs/hang/src/catalog/audio/mod.rs b/rs/hang/src/catalog/audio/mod.rs index 0df4866edfd..f80432d4ad9 100644 --- a/rs/hang/src/catalog/audio/mod.rs +++ b/rs/hang/src/catalog/audio/mod.rs @@ -36,7 +36,10 @@ pub struct Audio { /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. /// /// A malformed rendition refuses the whole catalog when decoding. - #[serde(deserialize_with = "crate::catalog::deserialize_renditions")] + #[serde( + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] pub renditions: BTreeMap, } @@ -261,7 +264,7 @@ impl AudioConfig { } } -impl crate::catalog::Rendition for AudioConfig { +impl crate::catalog::Entry for AudioConfig { fn check(&self) -> crate::Result<()> { if self.sample_rate == 0 { return Err(crate::Error::Zero("sampleRate")); @@ -274,6 +277,10 @@ impl crate::catalog::Rendition for AudioConfig { } Ok(()) } + + fn version(&self, id: &str) -> Option { + crate::catalog::renamed_version(self, id) + } } #[cfg(test)] diff --git a/rs/hang/src/catalog/binary.rs b/rs/hang/src/catalog/binary.rs index 464e353c19d..7beb466e703 100644 --- a/rs/hang/src/catalog/binary.rs +++ b/rs/hang/src/catalog/binary.rs @@ -25,6 +25,10 @@ pub struct Binary { /// /// A map rather than an array so it works with JSON Merge Patch, and a `BTreeMap` so keys are /// sorted for *some* deterministic behavior. + #[serde( + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] pub tracks: BTreeMap, } @@ -32,6 +36,16 @@ impl crate::catalog::Section for Binary { const MAP: &'static str = "tracks"; } +impl crate::catalog::Entry for BinaryConfig { + fn version(&self, id: &str) -> Option { + crate::catalog::renamed_version(self, id) + } + + fn claims_version(&self) -> bool { + self.extra.contains_key("version") + } +} + impl Binary { /// Insert a track config, returning an error if the name already exists. pub fn insert(&mut self, name: &str, config: BinaryConfig) -> crate::Result<()> { @@ -121,7 +135,8 @@ pub struct BinaryConfig { #[serde(default, skip_serializing_if = "crate::catalog::millis::no_delay")] pub delay: Option, - /// Fields this build doesn't recognize, kept so the entry round-trips. + /// Fields this build doesn't recognize, kept so the entry round-trips. Never `version`, which + /// is computed when the entry is written. /// /// A future [`Mode`] almost certainly comes with fields describing it, and /// preserving the discriminant without them would leave a relay republishing an entry nothing diff --git a/rs/hang/src/catalog/include.rs b/rs/hang/src/catalog/include.rs index 5d7c893f421..9afd1b29f16 100644 --- a/rs/hang/src/catalog/include.rs +++ b/rs/hang/src/catalog/include.rs @@ -39,3 +39,5 @@ impl Include { } } } + +impl crate::catalog::Entry for Include {} diff --git a/rs/hang/src/catalog/json.rs b/rs/hang/src/catalog/json.rs index 389b74c78c5..ebbca69c2b7 100644 --- a/rs/hang/src/catalog/json.rs +++ b/rs/hang/src/catalog/json.rs @@ -23,6 +23,10 @@ pub struct Json { /// /// A map rather than an array so it works with JSON Merge Patch, and a `BTreeMap` so keys are /// sorted for *some* deterministic behavior. + #[serde( + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] pub tracks: BTreeMap, } @@ -30,6 +34,16 @@ impl crate::catalog::Section for Json { const MAP: &'static str = "tracks"; } +impl crate::catalog::Entry for JsonConfig { + fn version(&self, id: &str) -> Option { + crate::catalog::renamed_version(self, id) + } + + fn claims_version(&self) -> bool { + self.extra.contains_key("version") + } +} + impl Json { /// Insert a track config, returning an error if the name already exists. pub fn insert(&mut self, name: &str, config: JsonConfig) -> crate::Result<()> { @@ -119,7 +133,8 @@ pub struct JsonConfig { #[serde(default, skip_serializing_if = "crate::catalog::millis::no_delay")] pub delay: Option, - /// Fields this build doesn't recognize, kept so the entry round-trips. + /// Fields this build doesn't recognize, kept so the entry round-trips. Never `version`, which + /// is computed when the entry is written. /// /// A future [`Mode`] almost certainly comes with fields describing it, and /// preserving the discriminant without them would leave a relay republishing an entry nothing diff --git a/rs/hang/src/catalog/mod.rs b/rs/hang/src/catalog/mod.rs index 634bf60a76e..2b9a3277bcd 100644 --- a/rs/hang/src/catalog/mod.rs +++ b/rs/hang/src/catalog/mod.rs @@ -21,6 +21,7 @@ mod mode; mod priority; mod reference; mod root; +mod skipped; mod stats; mod text; mod video; @@ -39,10 +40,15 @@ pub use mode::*; pub use priority::*; pub use reference::*; pub use root::*; +pub use skipped::Skipped; pub use stats::*; pub use text::*; pub use video::*; +/// The newest moq-hang draft this crate reads. A catalog whose root `version` is higher is refused, +/// and an entry whose `version` is higher is skipped. +pub(crate) const VERSION: u64 = 4; + /// Serde default for a rendition's `enabled` field. pub(crate) fn enabled_default() -> bool { true @@ -103,39 +109,99 @@ pub(crate) fn deserialize_media<'de, D: serde::Deserializer<'de>>(deserializer: serde_json::from_value(value).map_err(serde::de::Error::custom) } -/// A media rendition config, checked as it decodes. -pub(crate) trait Rendition: serde::de::DeserializeOwned { +/// An entry of a catalog map keyed by ID (a rendition, data track, or include), checked as it +/// decodes and versioned as it encodes. +pub(crate) trait Entry: serde::de::DeserializeOwned + serde::Serialize { /// Refuse a field out of the range its type allows. - fn check(&self) -> crate::Result<()>; + fn check(&self) -> crate::Result<()> { + Ok(()) + } + + /// The oldest moq-hang draft that reads this entry correctly given its ID, or `None` while that + /// is 3. Written as the entry's `version`. + /// + /// Only a field an older reader would misread raises it; one it would merely ignore does not. + fn version(&self, _id: &str) -> Option { + None + } + + /// Whether the entry keeps an unrecognized member named `version`, which would sit beside the + /// computed one as a duplicate key. + fn claims_version(&self) -> bool { + false + } } -/// Decode a media section's renditions, refusing the catalog if any one is malformed. +/// Decode a catalog map keyed by ID, skipping an entry newer than this reader and refusing the +/// catalog if any other entry is malformed. /// -/// Each rendition must be an object whose fields are in range; the error names the rendition. -/// An unknown container or extension field is not malformed, so it never refuses the catalog. -pub(crate) fn deserialize_renditions<'de, D, T>( - deserializer: D, -) -> Result, D::Error> +/// An entry's `version` is read before the rest of it, since a newer draft may have reshaped the +/// entry; one above [`VERSION`] is skipped, and [`Catalog::skipped`](crate::catalog::Catalog::skipped) +/// reports it. Every other entry must be an object whose fields are in range; the error names the +/// entry. An unknown container or extension field is not malformed, so it never refuses the catalog. +pub(crate) fn deserialize_entries<'de, D, T>(deserializer: D) -> Result, D::Error> where D: serde::Deserializer<'de>, - T: Rendition, + T: Entry, { use serde::Deserialize; - let renditions = std::collections::BTreeMap::::deserialize(deserializer)?; + let entries = std::collections::BTreeMap::::deserialize(deserializer)?; - let decode = |value: serde_json::Value| -> crate::Result { + let decode = |mut value: serde_json::Value| -> crate::Result> { + // Taken out rather than left to the entry: written versions are computed, never carried. + if let Some(version) = value.as_object_mut().and_then(|entry| entry.remove("version")) + && serde_json::from_value::(version)? > VERSION + { + return Ok(None); + } let config: T = deserialize_object(value)?; config.check()?; - Ok(config) + Ok(Some(config)) }; - renditions - .into_iter() - .map(|(id, value)| match decode(value) { - Ok(config) => Ok((id, config)), - Err(err) => Err(serde::de::Error::custom(format!("malformed rendition {id}: {err}"))), - }) - .collect() + let mut decoded = std::collections::BTreeMap::new(); + for (id, value) in entries { + match decode(value) { + Ok(Some(config)) => { + decoded.insert(id, config); + } + Ok(None) => {} + Err(err) => return Err(serde::de::Error::custom(format!("malformed entry {id}: {err}"))), + } + } + Ok(decoded) +} + +/// Encode a catalog map keyed by ID, writing each entry's computed `version` ahead of its fields. +pub(crate) fn serialize_entries( + entries: &std::collections::BTreeMap, + serializer: S, +) -> Result +where + S: serde::Serializer, + T: Entry, +{ + #[derive(serde::Serialize)] + struct Versioned<'a, T> { + #[serde(skip_serializing_if = "Option::is_none")] + version: Option, + #[serde(flatten)] + config: &'a T, + } + + if let Some(id) = entries + .iter() + .find_map(|(id, config)| config.claims_version().then_some(id)) + { + return Err(serde::ser::Error::custom(format!( + "catalog entry {id} must not carry a member `version`" + ))); + } + + serializer.collect_map(entries.iter().map(|(id, config)| { + let version = config.version(id); + (id, Versioned { version, config }) + })) } /// A catalog section: a map of track name to config, published under one well-known root key. diff --git a/rs/hang/src/catalog/reference.rs b/rs/hang/src/catalog/reference.rs index 20a6bf40976..171f38352ed 100644 --- a/rs/hang/src/catalog/reference.rs +++ b/rs/hang/src/catalog/reference.rs @@ -36,3 +36,9 @@ macro_rules! reference { } reference!(VideoConfig, AudioConfig, TextConfig, JsonConfig, BinaryConfig); + +/// moq-hang-04 for a `track` other than the ID: a reader that ignores it subscribes to the ID, a +/// different track. +pub(crate) fn renamed_version(config: &impl Reference, id: &str) -> Option { + (config.track(id) != id).then_some(4) +} diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 155e2753cd0..ca3924f3848 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -1,6 +1,7 @@ //! This module contains the structs and functions for the MoQ catalog format use crate::Result; -use crate::catalog::{Audio, Binary, Json, PRIORITY, Text, Video}; +use crate::catalog::skipped::Probe; +use crate::catalog::{Audio, Binary, Json, PRIORITY, Text, VERSION, Video}; use serde::de::DeserializeOwned; use std::str::FromStr; @@ -95,16 +96,20 @@ pub struct Catalog { /// Application-defined root sections, flattened alongside the base catalog sections. /// - /// The root `version` is reserved: serializing computes it from the base sections, and fails if - /// the extension names a member `version`. + /// The root `version` is reserved: serializing fails if the extension names a member `version`. pub ext: E, -} -/// The newest moq-hang draft this crate reads; a catalog whose `version` is higher is refused. -const VERSION: u64 = 4; + /// The entries decoding skipped because a newer moq-hang draft defines them. + /// + /// Filled when a catalog is decoded and never written. + pub skipped: crate::catalog::Skipped, +} -/// The wire shape of [`Catalog`]: its sections plus the root `version`, the oldest moq-hang draft -/// that reads the catalog correctly. An absent `version` means 3, the draft before the field. +/// The wire shape of [`Catalog`]: its sections, without the root `version`. +/// +/// The root `version` is the oldest moq-hang draft that reads the catalog correctly, absent while +/// that is 3. [`Probe`] checks it before any section is decoded. No moq-hang-04 field reshapes the +/// root, so this crate never writes it; a field one entry uses raises that entry's `version`. #[serde_with::serde_as] #[serde_with::skip_serializing_none] #[derive(Serialize, Deserialize)] @@ -112,10 +117,6 @@ const VERSION: u64 = 4; #[serde(remote = "Catalog", rename_all = "camelCase")] #[serde(bound(serialize = "E: Serialize", deserialize = "E: DeserializeOwned + Default"))] struct Wire { - // Computed rather than stored, so a republished catalog never claims less than it uses. - #[serde(default, getter = "Catalog::min_version", deserialize_with = "deserialize_version")] - version: Option, - #[serde( default, skip_serializing_if = "Video::is_empty", @@ -167,15 +168,24 @@ struct Wire { #[serde(default, deserialize_with = "crate::catalog::deserialize_echo")] echo: Option, - #[serde(default, skip_serializing_if = "std::collections::BTreeMap::is_empty")] + #[serde( + default, + skip_serializing_if = "std::collections::BTreeMap::is_empty", + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] include: std::collections::BTreeMap, #[serde(flatten, serialize_with = "serialize_ext")] ext: E, + + // Filled from `Probe` by `Catalog::from_value`. + #[serde(skip)] + skipped: crate::catalog::Skipped, } -/// Refuse an extension member named `version`: flattened beside the computed one, it would emit a -/// duplicate key that a reader resolves to whichever comes last. +/// Refuse an extension member named `version`: the root `version` is reserved for this +/// specification, and a reader would refuse a catalog whose extension claimed a newer draft. fn serialize_ext(ext: &E, serializer: S) -> std::result::Result { use serde::ser::Error; @@ -194,7 +204,6 @@ fn serialize_ext(ext: &E, serializer: S) -> impl From> for Catalog { fn from(wire: Wire) -> Self { let Wire { - version: _, video, audio, media, @@ -206,6 +215,7 @@ impl From> for Catalog { echo, include, ext, + skipped, } = wire; Self { video, @@ -219,6 +229,7 @@ impl From> for Catalog { echo, include, ext, + skipped, } } } @@ -230,21 +241,13 @@ impl Serialize for Catalog { } impl<'de, E: DeserializeOwned + Default> Deserialize<'de> for Catalog { + /// Buffered, so every `version` is read before any section, which a newer draft may have + /// reshaped. An entry needing a newer draft is skipped and listed in [`skipped`](Self::skipped). fn deserialize>(deserializer: D) -> std::result::Result { - Wire::deserialize(deserializer) + Self::from_value(serde_json::Value::deserialize(deserializer)?).map_err(serde::de::Error::custom) } } -/// Refuse a `version` above [`VERSION`]: the newer draft changed something this reader would -/// misread, not merely added something it would miss. -fn deserialize_version<'de, D: serde::Deserializer<'de>>( - deserializer: D, -) -> std::result::Result, D::Error> { - let version = Some(u64::deserialize(deserializer)?); - check_version(version).map_err(serde::de::Error::custom)?; - Ok(version) -} - impl Catalog<()> { /// The default name for the catalog track. pub const DEFAULT_NAME: &str = "catalog.json"; @@ -277,7 +280,7 @@ impl Catalog { /// /// Decoding already refuses an out-of-range rendition, so this catches a catalog built by hand. pub fn check_renditions(&self) -> Result<()> { - use crate::catalog::Rendition; + use crate::catalog::Entry; if let Some(id) = self .video @@ -296,30 +299,6 @@ impl Catalog { Ok(()) } - /// The oldest moq-hang draft that reads this catalog correctly, or `None` while that is 3. - /// - /// Only a field an older reader would misread raises it; one it would merely ignore does not. - fn min_version(&self) -> Option { - use crate::catalog::Reference; - - fn renamed<'a, C: Reference + 'a>(entries: impl IntoIterator) -> bool { - entries.into_iter().any(|(id, config)| config.track(id) != id) - } - - // moq-hang-04: a reader that ignores `track` subscribes to the ID, a different track. - let renamed = renamed(&self.video.renditions) - || renamed(&self.audio.renditions) - || renamed(&self.text.renditions) - || renamed(&self.json.tracks) - || renamed(&self.binary.tracks); - - // moq-hang-04: a reader that ignores video `warmup` assumes every group starts at a - // keyframe, and hands its decoder one that starts at a converging picture instead. - let warmup = (self.video.renditions.values()).any(|config| config.warmup.is_some_and(|w| !w.is_zero())); - - (renamed || warmup).then_some(4) - } - /// Copy the media sections into a catalog with no application extension. pub fn without_ext(&self) -> Catalog<()> { Catalog { @@ -356,10 +335,10 @@ impl Catalog { /// Fails with [`Error::UnsupportedVersion`](crate::Error::UnsupportedVersion) if the catalog /// needs a newer moq-hang draft than this crate reads. pub fn from_slice(v: &[u8]) -> Result { - // Decoded straight from the bytes rather than via a `Value`, which would keep only the - // last of two `version` members instead of refusing them. + // Probed straight from the bytes as well, since a `Value` keeps only the last of two + // `version` members instead of refusing them. check_version(serde_json::from_slice::(v)?.version)?; - Ok(serde_json::from_slice(v)?) + Self::from_value(serde_json::from_slice(v)?) } /// Parse a catalog from a reader. @@ -375,20 +354,24 @@ impl Catalog { /// Parse a catalog from a JSON value. /// /// Fails with [`Error::UnsupportedVersion`](crate::Error::UnsupportedVersion) if the catalog - /// needs a newer moq-hang draft than this crate reads. - pub(crate) fn from_value(value: serde_json::Value) -> Result { - check_version(value.get("version").and_then(serde_json::Value::as_u64))?; - Ok(serde_json::from_value(value)?) + /// needs a newer moq-hang draft than this crate reads. Every `version` is read before any + /// section, which the newer draft may have reshaped; an entry needing a newer draft is + /// skipped and reported in [`skipped`](Self::skipped). + pub(crate) fn from_value(mut value: serde_json::Value) -> Result { + let probe = Probe::deserialize(&value)?; + check_version(probe.version)?; + // Checked, so taken out of what an extension would otherwise receive. + if let Some(root) = value.as_object_mut() { + root.remove("version"); + } + let mut catalog: Self = Wire::deserialize(value)?; + catalog.skipped = probe.skipped; + Ok(catalog) } } -/// Just the root `version`, read before any section since the newer draft may have reshaped one. -#[derive(Deserialize)] -struct Probe { - #[serde(default)] - version: Option, -} - +/// Refuse a root `version` above [`VERSION`]: the newer draft changed something this reader would +/// misread, not merely added something it would miss. fn check_version(version: Option) -> Result<()> { match version { Some(version) if version > VERSION => Err(crate::Error::UnsupportedVersion(version)), @@ -439,10 +422,13 @@ mod test { binary: Vec, #[serde(default)] include: Vec, + #[serde(default)] + skipped: BTreeMap>, } /// The corpus `js/hang` also loads: both languages accept the same catalogs and keep the same - /// rendition, data track, and include IDs, so a divergence fails CI on whichever side drifts. + /// rendition, data track, and include IDs, and skip the same newer entries, so a divergence + /// fails CI on whichever side drifts. #[test] fn shared_parity_fixtures() { let cases: Vec = serde_json::from_str(include_str!("../../fixtures/catalog-parity.json")).unwrap(); @@ -477,6 +463,22 @@ mod test { case.name )); } + let skipped = &catalog.skipped; + let skipped: BTreeMap> = [ + ("video", &skipped.video), + ("audio", &skipped.audio), + ("text", &skipped.text), + ("json", &skipped.json), + ("binary", &skipped.binary), + ("include", &skipped.include), + ] + .into_iter() + .filter(|(_, ids)| !ids.is_empty()) + .map(|(section, ids)| (section.to_string(), ids.clone())) + .collect(); + if skipped != case.skipped { + failures.push(format!("{}: skipped {skipped:?}", case.name)); + } } } } @@ -1545,57 +1547,179 @@ mod test { assert_eq!(ok.to_json().unwrap(), r#"{"scte35":{}}"#); } - /// Only a field an older reader would misread raises the version; one it would merely ignore - /// leaves the catalog readable by moq-hang-03. + fn h264() -> VideoConfig { + let mut config = VideoConfig::new(H264 { + profile: 0x64, + constraints: 0x00, + level: 0x1f, + inline: false, + }); + config.container = Container::Legacy; + config + } + + /// Only a field an older reader would misread raises an entry's version; one it would merely + /// ignore leaves the entry readable by moq-hang-03. No moq-hang-04 field raises the root's. #[test] fn the_version_is_written_only_when_needed() { - let rendition = || { - let mut config = VideoConfig::new(H264 { - profile: 0x64, - constraints: 0x00, - level: 0x1f, - inline: false, - }); - config.container = Container::Legacy; - config + let versions = |catalog: &Catalog| { + let value = serde_json::to_value(catalog).unwrap(); + assert_eq!(value.get("version"), None, "the root version is never written"); + let mut versions = BTreeMap::new(); + for (section, map) in [ + ("video", "/video/renditions"), + ("audio", "/audio/renditions"), + ("json", "/json/tracks"), + ("include", "/include"), + ] { + for (id, entry) in value.pointer(map).and_then(|v| v.as_object()).into_iter().flatten() { + if let Some(version) = entry.get("version") { + versions.insert(format!("{section}.{id}"), version.as_u64().unwrap()); + } + } + } + versions }; - let version = |catalog: &Catalog| serde_json::to_value(catalog).unwrap().get("version").cloned(); let mut catalog = Catalog::<()>::default(); - catalog.video.renditions.insert("hd".into(), rendition()); + catalog.video.renditions.insert("hd".into(), h264()); let mut audio = AudioConfig::new(Opus, 48_000, 2); audio.warmup = Some(std::time::Duration::from_millis(80)); catalog.audio.renditions.insert("en".into(), audio); - assert_eq!(version(&catalog), None, "an older reader still plays audio warmup"); + catalog.include.insert( + "extra".into(), + crate::catalog::Include::new(moq_net::path::Relative::new("../extra").unwrap()), + ); + assert_eq!( + versions(&catalog), + BTreeMap::new(), + "an older reader still plays audio warmup" + ); // A `track` naming the ID is what an older reader subscribes to anyway. catalog.video.renditions.get_mut("hd").unwrap().track = Some("hd".into()); - assert_eq!(version(&catalog), None); + assert_eq!(versions(&catalog), BTreeMap::new()); catalog.video.renditions.get_mut("hd").unwrap().track = Some("video".into()); - assert_eq!(version(&catalog), Some(4.into())); + assert_eq!(versions(&catalog), BTreeMap::from([("video.hd".into(), 4)])); let json = catalog.to_json().unwrap(); - assert!(json.starts_with(r#"{"version":4,"#), "{json}"); + assert!(json.contains(r#""hd":{"version":4,"#), "{json}"); assert_eq!(Catalog::<()>::from_str(&json).unwrap(), catalog); - let mut data = Catalog::<()>::default(); let mut chat = JsonConfig::new(Mode::Stream); chat.track = Some("chat.json".into()); - data.json.insert("chat", chat).unwrap(); - assert_eq!(version(&data), Some(4.into())); - - let mut warmup = Catalog::<()>::default(); - let mut config = rendition(); - config.warmup = Some(std::time::Duration::from_millis(500)); - warmup.video.renditions.insert("hd".into(), config); - assert_eq!(version(&warmup), Some(4.into())); + catalog.json.insert("chat", chat).unwrap(); + let mut warmup = h264(); + warmup.warmup = Some(std::time::Duration::from_millis(500)); + catalog.video.renditions.insert("refresh".into(), warmup); + assert_eq!( + versions(&catalog), + BTreeMap::from([ + ("video.hd".into(), 4), + ("video.refresh".into(), 4), + ("json.chat".into(), 4) + ]) + ); } - /// The version is computed, never carried: republishing a catalog that claimed more than it + /// Versions are computed, never carried: republishing a catalog that claimed more than it /// uses writes what it actually needs. #[test] fn a_republish_recomputes_the_version() { - let catalog = Catalog::<()>::from_str(r#"{"version":4}"#).unwrap(); - assert_eq!(catalog.to_json().unwrap(), "{}"); + let json = r#"{"version":4,"video":{"renditions":{"hd":{"version":4,"codec":"avc1.64001f"}}}, + "json":{"tracks":{"chat":{"version":4,"mode":"stream"}}}}"#; + let catalog = Catalog::<()>::from_str(json).unwrap(); + assert!( + catalog.json.tracks["chat"].extra.is_empty(), + "the entry version is not an extension" + ); + assert!(!catalog.to_json().unwrap().contains("version")); + } + + /// An entry a newer draft defines is skipped and reported, even one this reader would call + /// malformed, while the rest of the catalog plays. + #[test] + fn a_newer_entry_is_skipped() { + let json = r#"{ + "video": {"renditions": { + "compat": {"codec": "avc1.64001f"}, + "next": {"version": 5, "codec": 42} + }}, + "audio": {"renditions": {"next": {"version": 6}}}, + "text": {"renditions": {"next": {"version": 5}}}, + "json": {"tracks": {"next": {"version": 5}}}, + "binary": {"tracks": {"next": {"version": 5, "mode": []}}}, + "include": {"next": {"version": 5, "broadcast": 7}} + }"#; + + let check = |catalog: Catalog| { + assert_eq!(catalog.video.renditions.keys().collect::>(), ["compat"]); + assert!(catalog.audio.is_empty() && catalog.text.is_empty() && catalog.include.is_empty()); + assert!(catalog.json.is_empty() && catalog.binary.is_empty()); + + let next = |version| BTreeMap::from([("next".to_string(), version)]); + assert_eq!(catalog.skipped.video, next(5)); + assert_eq!(catalog.skipped.audio, next(6)); + assert_eq!(catalog.skipped.text, next(5)); + assert_eq!(catalog.skipped.json, next(5)); + assert_eq!(catalog.skipped.binary, next(5)); + assert_eq!(catalog.skipped.include, next(5)); + assert_eq!(catalog.skipped.version(), Some(6)); + + // Never written, so a republish lists only what this reader read. + let republished = Catalog::<()>::from_str(&catalog.to_json().unwrap()).unwrap(); + assert!(republished.skipped.is_empty()); + assert_eq!(republished.skipped.version(), None); + }; + check(Catalog::from_str(json).unwrap()); + check(serde_json::from_str(json).unwrap()); + + // The root version still refuses the whole catalog. + let root = json.replacen('{', r#"{"version":5,"#, 1); + assert!(matches!( + Catalog::<()>::from_str(&root), + Err(crate::Error::UnsupportedVersion(5)) + )); + } + + /// An entry readable by this draft is read whatever version it claims. + #[test] + fn a_supported_entry_version_is_read() { + for version in [0, 3, 4] { + let json = + format!(r#"{{"video":{{"renditions":{{"hd":{{"version":{version},"codec":"avc1.64001f"}}}}}}}}"#); + let catalog = Catalog::<()>::from_str(&json).unwrap(); + assert!(catalog.video.renditions.contains_key("hd"), "{json}"); + assert!(catalog.skipped.is_empty(), "{json}"); + } + } + + /// An entry's version is refused, with the catalog, when malformed or repeated, as the root's is: + /// otherwise a newer entry could slip past the check by also claiming an old version. + #[test] + fn a_malformed_entry_version_is_refused() { + for version in [r#""5""#, "-1", "4.5", "null", "[]"] { + let json = format!(r#"{{"json":{{"tracks":{{"chat":{{"version":{version},"mode":"stream"}}}}}}}}"#); + assert!(Catalog::<()>::from_str(&json).is_err(), "{json}"); + assert!(serde_json::from_str::(&json).is_err(), "{json}"); + } + + for versions in [r#""version":5,"version":3"#, r#""version":3,"version":3"#] { + let json = format!(r#"{{"video":{{"renditions":{{"hd":{{{versions},"codec":"avc1.64001f"}}}}}}}}"#); + let err = Catalog::<()>::from_str(&json).unwrap_err(); + assert!(err.to_string().contains("duplicate field `version`"), "{json}: {err}"); + assert!(Catalog::<()>::from_reader(json.as_bytes()).is_err(), "{json}"); + } + } + + /// A data track extension member named `version` would sit beside the computed one. + #[test] + fn an_entry_extension_version_is_refused() { + let mut catalog = Catalog::<()>::default(); + let mut chat = JsonConfig::new(Mode::Stream); + chat.extra.insert("version".into(), 3.into()); + catalog.json.insert("chat", chat).unwrap(); + let err = catalog.to_json().unwrap_err(); + assert!(err.to_string().contains("must not carry a member `version`"), "{err}"); } } diff --git a/rs/hang/src/catalog/skipped.rs b/rs/hang/src/catalog/skipped.rs new file mode 100644 index 00000000000..ce234dd89a7 --- /dev/null +++ b/rs/hang/src/catalog/skipped.rs @@ -0,0 +1,225 @@ +use std::collections::BTreeMap; + +use serde::de::{DeserializeSeed, Deserializer, Error as _, IgnoredAny, MapAccess, SeqAccess, Visitor}; + +use crate::catalog::VERSION; + +/// The catalog entries a reader skipped because a newer moq-hang draft defines them. +/// +/// Each section maps a skipped entry's ID to its `version`. The rest of the catalog reads as if +/// those entries were absent, so a player plays what it can and may tell its user to upgrade for +/// the rest. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +#[non_exhaustive] +pub struct Skipped { + /// Skipped video renditions. + pub video: BTreeMap, + /// Skipped audio renditions. + pub audio: BTreeMap, + /// Skipped text renditions. + pub text: BTreeMap, + /// Skipped JSON tracks. + pub json: BTreeMap, + /// Skipped binary tracks. + pub binary: BTreeMap, + /// Skipped includes. + pub include: BTreeMap, +} + +impl Skipped { + fn sections(&self) -> [&BTreeMap; 6] { + [ + &self.video, + &self.audio, + &self.text, + &self.json, + &self.binary, + &self.include, + ] + } + + /// Whether every entry was read. + pub fn is_empty(&self) -> bool { + self.sections().iter().all(|section| section.is_empty()) + } + + /// The draft a reader needs to read every skipped entry, or `None` if none was skipped. + pub fn version(&self) -> Option { + self.sections().into_iter().flat_map(BTreeMap::values).copied().max() + } +} + +/// What a reader checks before decoding any section: the root `version`, and which entries are too +/// new to read. +/// +/// Read straight from the bytes where it can, so a repeated `version` member is refused rather +/// than resolved to either copy. Every other shape is left to the catalog's own decoding. +#[derive(Default)] +pub(crate) struct Probe { + pub version: Option, + pub skipped: Skipped, +} + +impl<'de> serde::Deserialize<'de> for Probe { + fn deserialize>(deserializer: D) -> Result { + Lenient(RootSeed).deserialize(deserializer) + } +} + +/// Reads one JSON object's members. +trait MapSeed { + type Value: Default; + fn visit<'de, A: MapAccess<'de>>(self, map: A) -> Result; +} + +/// Reads an object with its [`MapSeed`] and anything else as the seed's default. +struct Lenient(S); + +impl<'de, S: MapSeed> DeserializeSeed<'de> for Lenient { + type Value = S::Value; + + fn deserialize>(self, deserializer: D) -> Result { + deserializer.deserialize_any(self) + } +} + +impl<'de, S: MapSeed> Visitor<'de> for Lenient { + type Value = S::Value; + + fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { + f.write_str("any JSON value") + } + + fn visit_map>(self, map: A) -> Result { + self.0.visit(map) + } + + fn visit_seq>(self, mut seq: A) -> Result { + while seq.next_element::()?.is_some() {} + Ok(Default::default()) + } + + fn visit_bool(self, _: bool) -> Result { + Ok(Default::default()) + } + + fn visit_i64(self, _: i64) -> Result { + Ok(Default::default()) + } + + fn visit_u64(self, _: u64) -> Result { + Ok(Default::default()) + } + + fn visit_f64(self, _: f64) -> Result { + Ok(Default::default()) + } + + fn visit_str(self, _: &str) -> Result { + Ok(Default::default()) + } + + fn visit_unit(self) -> Result { + Ok(Default::default()) + } +} + +/// Reads a `version` member, refusing a second one. +fn version<'de, A: MapAccess<'de>>(map: &mut A, version: &mut Option) -> Result<(), A::Error> { + if version.is_some() { + return Err(A::Error::duplicate_field("version")); + } + *version = Some(map.next_value()?); + Ok(()) +} + +struct RootSeed; + +impl MapSeed for RootSeed { + type Value = Probe; + + fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result { + let mut probe = Probe::default(); + while let Some(key) = map.next_key::()? { + let skipped = &mut probe.skipped; + let (section, entries) = match key.as_str() { + "version" => { + version(&mut map, &mut probe.version)?; + continue; + } + "video" => (&mut skipped.video, Some("renditions")), + "audio" => (&mut skipped.audio, Some("renditions")), + "text" => (&mut skipped.text, Some("renditions")), + "json" => (&mut skipped.json, Some("tracks")), + "binary" => (&mut skipped.binary, Some("tracks")), + "include" => (&mut skipped.include, None), + _ => { + map.next_value::()?; + continue; + } + }; + *section = match entries { + Some(key) => map.next_value_seed(Lenient(SectionSeed(key)))?, + None => map.next_value_seed(Lenient(EntriesSeed))?, + }; + } + Ok(probe) + } +} + +/// A section holding its entries under one key (`renditions`, `tracks`). +struct SectionSeed(&'static str); + +impl MapSeed for SectionSeed { + type Value = BTreeMap; + + fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result { + let mut skipped = BTreeMap::new(); + while let Some(key) = map.next_key::()? { + if key == self.0 { + skipped = map.next_value_seed(Lenient(EntriesSeed))?; + } else { + map.next_value::()?; + } + } + Ok(skipped) + } +} + +/// A map of entries keyed by ID, yielding the ones newer than this reader. +struct EntriesSeed; + +impl MapSeed for EntriesSeed { + type Value = BTreeMap; + + fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result { + let mut skipped = BTreeMap::new(); + while let Some(id) = map.next_key::()? { + if let Some(version) = map.next_value_seed(Lenient(EntrySeed))? + && version > VERSION + { + skipped.insert(id, version); + } + } + Ok(skipped) + } +} + +/// One entry, yielding its `version`. +struct EntrySeed; + +impl MapSeed for EntrySeed { + type Value = Option; + + fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result, A::Error> { + let mut entry = None; + while let Some(key) = map.next_key::()? { + if key == "version" { + version(&mut map, &mut entry)?; + } else { + map.next_value::()?; + } + } + Ok(entry) + } +} diff --git a/rs/hang/src/catalog/text/mod.rs b/rs/hang/src/catalog/text/mod.rs index 45292ac6349..e08fdf8b8cb 100644 --- a/rs/hang/src/catalog/text/mod.rs +++ b/rs/hang/src/catalog/text/mod.rs @@ -28,6 +28,10 @@ pub struct Text { /// A map of track name to rendition configuration. /// This is not an array so it will work with JSON Merge Patch. /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. + #[serde( + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] pub renditions: BTreeMap, } @@ -35,6 +39,12 @@ impl crate::catalog::Section for Text { const MAP: &'static str = "renditions"; } +impl crate::catalog::Entry for TextConfig { + fn version(&self, id: &str) -> Option { + crate::catalog::renamed_version(self, id) + } +} + impl Text { /// Insert a track config, returning an error if the name already exists. pub fn insert(&mut self, name: &str, config: TextConfig) -> crate::Result<()> { diff --git a/rs/hang/src/catalog/video/mod.rs b/rs/hang/src/catalog/video/mod.rs index e2faf3e660a..6ec776a9bcb 100644 --- a/rs/hang/src/catalog/video/mod.rs +++ b/rs/hang/src/catalog/video/mod.rs @@ -46,7 +46,10 @@ pub struct Video { /// We use a BTreeMap so keys are sorted alphabetically for *some* deterministic behavior. /// /// A malformed rendition refuses the whole catalog when decoding. - #[serde(deserialize_with = "crate::catalog::deserialize_renditions")] + #[serde( + deserialize_with = "crate::catalog::deserialize_entries", + serialize_with = "crate::catalog::serialize_entries" + )] pub renditions: BTreeMap, /// Render the video at this size in pixels. @@ -416,7 +419,7 @@ impl VideoConfig { } } -impl crate::catalog::Rendition for VideoConfig { +impl crate::catalog::Entry for VideoConfig { fn check(&self) -> crate::Result<()> { for (field, value) in [ ("codedWidth", self.coded_width), @@ -433,6 +436,13 @@ impl crate::catalog::Rendition for VideoConfig { } Ok(()) } + + fn version(&self, id: &str) -> Option { + // moq-hang-04: a reader that ignores video `warmup` assumes every group starts at a + // keyframe, and hands its decoder one that starts at a converging picture instead. + let warmup = self.warmup.is_some_and(|warmup| !warmup.is_zero()).then_some(4); + crate::catalog::renamed_version(self, id).max(warmup) + } } fn is_zero(value: &i32) -> bool { From 5268e2924edcd4f7e6fdb9d59533d7ac6788e02c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 07:53:55 -0800 Subject: [PATCH 3/6] quest: bindings report the catalog entries they skipped Co-Authored-By: Claude Opus 5.5 --- quest/m3/ffi-catalog-skipped.md | 35 +++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 quest/m3/ffi-catalog-skipped.md diff --git a/quest/m3/ffi-catalog-skipped.md b/quest/m3/ffi-catalog-skipped.md new file mode 100644 index 00000000000..0d2e62f814c --- /dev/null +++ b/quest/m3/ffi-catalog-skipped.md @@ -0,0 +1,35 @@ +# [M] Bindings report the catalog entries they skipped + +## Goal + +A native player built on a binding can tell its user that the catalog +listed renditions it skipped because a newer moq-hang draft defines them, +the way `@moq/watch` warns and `hang::Catalog::skipped` reports. `MoqCatalog` +in `rs/moq-ffi/src/media.rs` carries the skipped video and audio rendition +IDs with their versions, and every wrapper (Python, Go, Kotlin, Swift, Dart, +C, C++) exposes them under the same name. + +## Plan + +Today a binding gets a `MoqCatalog` built from `hang::Catalog`, so a +rendition a newer draft defines just vanishes: the root +`MoqError::UnsupportedCatalog` covers a reshaped root only. + +Recommendations: + +- One field, `skipped`, holding the video and audio maps of ID to + `version` (the only sections `MoqCatalog` exposes), plus the highest + version, mirroring `hang::catalog::Skipped`. Data tracks and includes + join it when `MoqCatalog` exposes those sections. +- `moq play` and the OBS source log a warning naming the version, as + `@moq/watch` does. + +Tests: the ffi test that a catalog with a compatibility rendition and a +version-5 rendition yields the compatibility one plus the report, and each +wrapper's smoke test reads the field. + +Public API: additive in moq-ffi and every wrapper. Wire: none. + +## Related + +- [hang-04 freezes a catalog worth keeping](/quest/m2/hang04/README.md) - the epic whose entry versions this exposes From 1eb1aade3b37bad91fd87de63afcf55db564bf94 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 16 Nov 2026 08:16:01 -0800 Subject: [PATCH 4/6] fix(hang): check the root version before refusing a malformed entry version Co-Authored-By: Claude Opus 5.5 --- rs/hang/src/catalog/root.rs | 28 +++++++------ rs/hang/src/catalog/skipped.rs | 75 +++++++++++++++++++++------------- 2 files changed, 62 insertions(+), 41 deletions(-) diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index ca3924f3848..68c4ccbf594 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -1,7 +1,7 @@ //! This module contains the structs and functions for the MoQ catalog format use crate::Result; use crate::catalog::skipped::Probe; -use crate::catalog::{Audio, Binary, Json, PRIORITY, Text, VERSION, Video}; +use crate::catalog::{Audio, Binary, Json, PRIORITY, Text, Video}; use serde::de::DeserializeOwned; use std::str::FromStr; @@ -337,7 +337,7 @@ impl Catalog { pub fn from_slice(v: &[u8]) -> Result { // Probed straight from the bytes as well, since a `Value` keeps only the last of two // `version` members instead of refusing them. - check_version(serde_json::from_slice::(v)?.version)?; + serde_json::from_slice::(v)?.check()?; Self::from_value(serde_json::from_slice(v)?) } @@ -359,7 +359,7 @@ impl Catalog { /// skipped and reported in [`skipped`](Self::skipped). pub(crate) fn from_value(mut value: serde_json::Value) -> Result { let probe = Probe::deserialize(&value)?; - check_version(probe.version)?; + probe.check()?; // Checked, so taken out of what an extension would otherwise receive. if let Some(root) = value.as_object_mut() { root.remove("version"); @@ -370,15 +370,6 @@ impl Catalog { } } -/// Refuse a root `version` above [`VERSION`]: the newer draft changed something this reader would -/// misread, not merely added something it would miss. -fn check_version(version: Option) -> Result<()> { - match version { - Some(version) if version > VERSION => Err(crate::Error::UnsupportedVersion(version)), - _ => Ok(()), - } -} - impl Catalog { /// Serialize the catalog to a JSON string. pub fn to_json(&self) -> Result { @@ -1680,6 +1671,19 @@ mod test { Catalog::<()>::from_str(&root), Err(crate::Error::UnsupportedVersion(5)) )); + + // Even when the newer draft reshaped an entry's own `version`, wherever the root's sits. + for entry in [r#"{"version":"new-shape"}"#, r#"{"version":5,"version":3}"#] { + for json in [ + format!(r#"{{"version":5,"video":{{"renditions":{{"hd":{entry}}}}}}}"#), + format!(r#"{{"video":{{"renditions":{{"hd":{entry}}}}},"version":5}}"#), + ] { + assert!( + matches!(Catalog::<()>::from_str(&json), Err(crate::Error::UnsupportedVersion(5))), + "{json}" + ); + } + } } /// An entry readable by this draft is read whatever version it claims. diff --git a/rs/hang/src/catalog/skipped.rs b/rs/hang/src/catalog/skipped.rs index ce234dd89a7..2e9ea454bcc 100644 --- a/rs/hang/src/catalog/skipped.rs +++ b/rs/hang/src/catalog/skipped.rs @@ -52,12 +52,32 @@ impl Skipped { /// What a reader checks before decoding any section: the root `version`, and which entries are too /// new to read. /// -/// Read straight from the bytes where it can, so a repeated `version` member is refused rather -/// than resolved to either copy. Every other shape is left to the catalog's own decoding. +/// Read straight from the bytes where it can, so a repeated `version` member is caught rather than +/// resolved to either copy. Nothing here refuses an entry before the root `version` is checked, +/// since a newer draft may have reshaped any section; every other shape is left to the catalog's +/// own decoding. #[derive(Default)] pub(crate) struct Probe { pub version: Option, pub skipped: Skipped, + /// An entry repeats its `version` member, which refuses the catalog once the root is read. + pub repeated: bool, +} + +impl Probe { + /// Refuse a root `version` above [`VERSION`], then an entry that repeats its `version`. + /// + /// A root `version` above this reader means the newer draft changed something it would + /// misread, not merely added something it would miss. + pub fn check(&self) -> crate::Result<()> { + match self.version { + Some(version) if version > VERSION => Err(crate::Error::UnsupportedVersion(version)), + _ if self.repeated => Err(crate::Error::Json( + "duplicate field `version` in a catalog entry".to_string(), + )), + _ => Ok(()), + } + } } impl<'de> serde::Deserialize<'de> for Probe { @@ -124,15 +144,6 @@ impl<'de, S: MapSeed> Visitor<'de> for Lenient { } } -/// Reads a `version` member, refusing a second one. -fn version<'de, A: MapAccess<'de>>(map: &mut A, version: &mut Option) -> Result<(), A::Error> { - if version.is_some() { - return Err(A::Error::duplicate_field("version")); - } - *version = Some(map.next_value()?); - Ok(()) -} - struct RootSeed; impl MapSeed for RootSeed { @@ -142,9 +153,13 @@ impl MapSeed for RootSeed { let mut probe = Probe::default(); while let Some(key) = map.next_key::()? { let skipped = &mut probe.skipped; + let repeated = &mut probe.repeated; let (section, entries) = match key.as_str() { "version" => { - version(&mut map, &mut probe.version)?; + if probe.version.is_some() { + return Err(A::Error::duplicate_field("version")); + } + probe.version = Some(map.next_value()?); continue; } "video" => (&mut skipped.video, Some("renditions")), @@ -159,8 +174,8 @@ impl MapSeed for RootSeed { } }; *section = match entries { - Some(key) => map.next_value_seed(Lenient(SectionSeed(key)))?, - None => map.next_value_seed(Lenient(EntriesSeed))?, + Some(key) => map.next_value_seed(Lenient(SectionSeed(key, repeated)))?, + None => map.next_value_seed(Lenient(EntriesSeed(repeated)))?, }; } Ok(probe) @@ -168,16 +183,17 @@ impl MapSeed for RootSeed { } /// A section holding its entries under one key (`renditions`, `tracks`). -struct SectionSeed(&'static str); +struct SectionSeed<'a>(&'static str, &'a mut bool); -impl MapSeed for SectionSeed { +impl MapSeed for SectionSeed<'_> { type Value = BTreeMap; fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result { + let Self(entries, repeated) = self; let mut skipped = BTreeMap::new(); while let Some(key) = map.next_key::()? { - if key == self.0 { - skipped = map.next_value_seed(Lenient(EntriesSeed))?; + if key == entries { + skipped = map.next_value_seed(Lenient(EntriesSeed(&mut *repeated)))?; } else { map.next_value::()?; } @@ -187,15 +203,15 @@ impl MapSeed for SectionSeed { } /// A map of entries keyed by ID, yielding the ones newer than this reader. -struct EntriesSeed; +struct EntriesSeed<'a>(&'a mut bool); -impl MapSeed for EntriesSeed { +impl MapSeed for EntriesSeed<'_> { type Value = BTreeMap; fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result { let mut skipped = BTreeMap::new(); while let Some(id) = map.next_key::()? { - if let Some(version) = map.next_value_seed(Lenient(EntrySeed))? + if let Some(version) = map.next_value_seed(Lenient(EntrySeed(&mut *self.0)))? && version > VERSION { skipped.insert(id, version); @@ -205,21 +221,22 @@ impl MapSeed for EntriesSeed { } } -/// One entry, yielding its `version`. -struct EntrySeed; +/// One entry, yielding its `version` when it is one. A malformed `version` is left to the entry's +/// own decoding, which refuses it once the root `version` has been checked. +struct EntrySeed<'a>(&'a mut bool); -impl MapSeed for EntrySeed { +impl MapSeed for EntrySeed<'_> { type Value = Option; fn visit<'de, A: MapAccess<'de>>(self, mut map: A) -> Result, A::Error> { - let mut entry = None; + let mut version = None; while let Some(key) = map.next_key::()? { - if key == "version" { - version(&mut map, &mut entry)?; - } else { + if key != "version" { map.next_value::()?; + } else if version.replace(map.next_value::()?).is_some() { + *self.0 = true; } } - Ok(entry) + Ok(version.as_ref().and_then(serde_json::Value::as_u64)) } } From 262f3985502916236630065441bd7e1b570b339b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 17 Nov 2026 02:46:38 -0800 Subject: [PATCH 5/6] fix(hang): plain serde refuses a repeated member, as the byte probe does --- rs/hang/src/catalog/root.rs | 77 ++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 06f28bff80a..084ac5b030a 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -233,7 +233,77 @@ impl<'de, E: DeserializeOwned + Default> Deserialize<'de> for Catalog { /// Buffered, so every `version` is read before any section, which a newer draft may have /// reshaped. An entry needing a newer draft is skipped and listed in [`skipped`](Self::skipped). fn deserialize>(deserializer: D) -> std::result::Result { - Self::from_value(serde_json::Value::deserialize(deserializer)?).map_err(serde::de::Error::custom) + Self::from_value(Strict::deserialize(deserializer)?.0).map_err(serde::de::Error::custom) + } +} + +/// A JSON value that refuses an object repeating a member, which a [`Value`](serde_json::Value) +/// would silently collapse to its last. Without it, `{"version":5,"version":3}` would read as 3. +struct Strict(serde_json::Value); + +impl<'de> Deserialize<'de> for Strict { + fn deserialize>(deserializer: D) -> std::result::Result { + struct Visitor; + + impl<'de> serde::de::Visitor<'de> for Visitor { + type Value = serde_json::Value; + + fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { + f.write_str("any JSON value") + } + + fn visit_bool(self, v: bool) -> std::result::Result { + Ok(v.into()) + } + + fn visit_i64(self, v: i64) -> std::result::Result { + Ok(v.into()) + } + + fn visit_u64(self, v: u64) -> std::result::Result { + Ok(v.into()) + } + + fn visit_f64(self, v: f64) -> std::result::Result { + Ok(serde_json::Number::from_f64(v).map_or(serde_json::Value::Null, serde_json::Value::Number)) + } + + fn visit_str(self, v: &str) -> std::result::Result { + Ok(v.into()) + } + + fn visit_string(self, v: String) -> std::result::Result { + Ok(v.into()) + } + + fn visit_none(self) -> std::result::Result { + Ok(serde_json::Value::Null) + } + + fn visit_unit(self) -> std::result::Result { + Ok(serde_json::Value::Null) + } + + fn visit_seq>(self, mut seq: A) -> std::result::Result { + let mut items = Vec::new(); + while let Some(Strict(item)) = seq.next_element()? { + items.push(item); + } + Ok(items.into()) + } + + fn visit_map>(self, mut map: A) -> std::result::Result { + let mut object = serde_json::Map::new(); + while let Some((key, Strict(value))) = map.next_entry::()? { + if object.insert(key.clone(), value).is_some() { + return Err(serde::de::Error::custom(format!("duplicate member `{key}`"))); + } + } + Ok(object.into()) + } + } + + deserializer.deserialize_any(Visitor).map(Self) } } @@ -1396,7 +1466,12 @@ mod test { let err = Catalog::<()>::from_str(json).unwrap_err(); assert!(err.to_string().contains("duplicate field `version`"), "{json}: {err}"); assert!(Catalog::<()>::from_reader(json.as_bytes()).is_err(), "{json}"); + // So does plain serde, at the root and in an entry. + assert!(serde_json::from_str::(json).is_err(), "{json}"); } + let entry = r#"{"video":{"renditions":{"hd":{"version":5,"version":3,"codec":"avc1.64001f"}}}}"#; + assert!(Catalog::<()>::from_str(entry).is_err(), "{entry}"); + assert!(serde_json::from_str::(entry).is_err(), "{entry}"); } #[test] From 3b9068d3be25a92af470a3ce733d8c565c5045f2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 17 Nov 2026 03:06:43 -0800 Subject: [PATCH 6/6] fix(hang): from_slice refuses a repeated member anywhere, as plain serde does --- rs/hang/src/catalog/root.rs | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/rs/hang/src/catalog/root.rs b/rs/hang/src/catalog/root.rs index 084ac5b030a..908d563cd46 100644 --- a/rs/hang/src/catalog/root.rs +++ b/rs/hang/src/catalog/root.rs @@ -401,10 +401,10 @@ impl Catalog { /// Fails with [`Error::UnsupportedVersion`](crate::Error::UnsupportedVersion) if the catalog /// needs a newer moq-hang draft than this crate reads. pub fn from_slice(v: &[u8]) -> Result { - // Probed straight from the bytes as well, since a `Value` keeps only the last of two - // `version` members instead of refusing them. + // Probed straight from the bytes first, so an unsupported root `version` is reported before + // any other fault; `Strict` then refuses a repeated member, which a `Value` would collapse. serde_json::from_slice::(v)?.check()?; - Self::from_value(serde_json::from_slice(v)?) + Self::from_value(serde_json::from_slice::(v)?.0) } /// Parse a catalog from a reader. @@ -1472,6 +1472,11 @@ mod test { let entry = r#"{"video":{"renditions":{"hd":{"version":5,"version":3,"codec":"avc1.64001f"}}}}"#; assert!(Catalog::<()>::from_str(entry).is_err(), "{entry}"); assert!(serde_json::from_str::(entry).is_err(), "{entry}"); + + // Any repeated member, not only `version`. + let repeated = r#"{"video":{"renditions":{"hd":{"codec":"avc1.64001f","codec":"vp8"}}}}"#; + assert!(Catalog::<()>::from_str(repeated).is_err(), "{repeated}"); + assert!(serde_json::from_str::(repeated).is_err(), "{repeated}"); } #[test]