Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/concept/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ A few things the catalog can express beyond decoder config:

- **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the rendition's ID, so labels need not be unique and renaming one doesn't rename the track.
- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. `@moq/watch` and `moqsrc` follow a referenced path's announcements like their own, restarting the rendition on each instance that serves it; every other reader, such as `moq play`, the exporters, and the bindings, waits for the first one announced and keeps it, and an exporter ends the rendition with it. An optional `epoch` beside the path names the [publisher instance](/concept/moq-lite#publisher-epochs) it expects: a route proving another instance ends the rendition instead of falling back to it, while a route without an epoch (an older version) cannot prove one, so there it is best effort. Omit `epoch` for a broadcast produced on demand, which has no instance until a worker answers; see [identity and recovery](/concept/identity).
- **Whole contributions.** A root `contributions` map names whole broadcasts, such as each guest feed in a composed room, by the same relative path and optional `epoch` as a rendition's reference. Listing one opens nothing, so a contribution produced on demand starts only once a viewer selects it, which `@moq/watch` does with `broadcast.contribution(effect, id)`. Its rendition IDs stay its own.
- **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition.
- **Jitter.** A rendition can say how much media it writes behind a frame's timestamp, in whole milliseconds rounded up: the reorder delay of B-frames and the span of a fragment or batch written together. It is measured from media timestamps alone, never the time a frame was written, so neither a constant encoder delay nor a capture clock counts. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent.
- **Delay.** A rendition can also say how far its frames' timestamps trail those another rendition already wrote, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. A rendition alone measures none. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's.
Expand Down
25 changes: 22 additions & 3 deletions drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,10 @@ type Catalog = {
"text": TextSchema | undefined,
"json": JsonTracks | undefined,
"binary": BinaryTracks | undefined,
"contributions": Record<string, {
"broadcast": string,
"epoch": string | undefined,
}> | undefined,
// ... any custom fields ...
}
~~~
Expand All @@ -176,7 +180,7 @@ For example, a chat entry should name a chat track, not carry individual chat me
This way catalog updates are rare and a client MAY choose to not subscribe.
The `json` and `binary` sections ({{data}}) are how a track like that is listed.

This specification defines audio, video, and text media tracks, with optional HLS records ({{hls}}) on video and audio, an optional root `clock` entry ({{clock}}) mapping content time to wall time, the root `version` ({{catalog-version}}), the application data tracks in {{data}}, and optional publisher statistics and solicited viewer feedback ({{media-stats}}).
This specification defines audio, video, and text media tracks, with optional HLS records ({{hls}}) on video and audio, an optional root `clock` entry ({{clock}}) mapping content time to wall time, the root `version` ({{catalog-version}}), the application data tracks in {{data}}, optional publisher statistics and solicited viewer feedback ({{media-stats}}), and whole broadcasts listed as `contributions` ({{contributions}}).

## Version {#catalog-version}
The root `version` field is the oldest draft of this specification that reads the catalog correctly, by draft number: `4` needs a consumer of moq-hang-04 or later.
Expand All @@ -192,7 +196,7 @@ This draft needs `version` 4 for:
- a `track` ({{field-track}}) other than the entry's ID, which an older consumer subscribes to instead;
- a video `warmup` ({{field-warmup}}) above zero, since an older consumer starts decoding a group at a picture that is not a keyframe.

Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `hls` ({{hls-catalog}}) is ignored when undecodable; `stats` and `echo` ({{media-stats}}) only offer optional tracks; video `reorder` leaves an older consumer inferring decode times from timestamps; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.
Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `hls` ({{hls-catalog}}) is ignored when undecodable; `stats` and `echo` ({{media-stats}}) only offer optional tracks; `contributions` ({{contributions}}) only offer broadcasts to open on selection; video `reorder` leaves an older consumer inferring decode times from timestamps; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.

## Clock {#clock}
The catalog's root `clock` field is the broadcast's one continuous clock:
Expand All @@ -214,6 +218,20 @@ A zero `timescale` is invalid, as is a `wall` outside the JSON-safe integer rang
The mapping is fixed for the broadcast.
A discontinuity marker is a delivery event, not a new epoch, and a system-clock adjustment never retimes the mapping.

## Contributions {#contributions}
The optional root `contributions` map names whole broadcasts, such as each guest feed in a composed room, whose own catalogs describe their renditions.
Each key is a contribution ID.
Each entry MUST contain a string `broadcast`, resolved against the catalog's broadcast as in {{field-broadcast}}, and MAY contain an `epoch` as in {{field-epoch}}.
A consumer MUST reject a catalog with a malformed entry or a reference that escapes above the root.

Listing an entry MUST NOT subscribe to the contribution: a consumer opens its catalog only when it selects the contribution or one of its renditions, so a contribution produced on demand starts only for a viewer that wants it.
A selected contribution resolves like a rendition's reference, under the same rules for `epoch` and restarts ({{field-epoch}}).
A contribution produced on demand has no instance until it resolves, so a publisher omits its `epoch` until then.

A contribution's rendition IDs belong to its own catalog and MUST NOT replace IDs in the listing catalog, so the same ID MAY name a contribution, a rendition in the listing catalog, and a rendition in the contribution.
An unavailable contribution does not invalidate the listing catalog.
A consumer does not expand a contribution's own `contributions`: each needs a selection of its own.

## Track Identity {#identity}
A track name always means the same content, so a catalog update never redefines an audio or video track.
A track's identity is the fields a consumer cannot change without reinitializing its decoder or demuxer: `codec`, `description`, and `container`, plus `displayAspectWidth`, `displayAspectHeight`, and `interlaced` for video and `sampleRate` and `numberOfChannels` for audio.
Expand Down Expand Up @@ -1125,7 +1143,7 @@ the broadcast path, not the payload. This track is distinct from an application'
`hang/chat.json` snapshot extension.

# Security Considerations
A rendition's `broadcast` reference ({{field-broadcast}}) resolves against the consumer's root, which is the subtree it is authorized for.
A rendition's `broadcast` reference ({{field-broadcast}}), like a contribution's ({{contributions}}), resolves against the consumer's root, which is the subtree it is authorized for.
Clamping a reference that escapes above that root would silently redirect the subscription to an unrelated broadcast, so a consumer rejects the catalog instead.

TODO Security
Expand Down Expand Up @@ -1155,6 +1173,7 @@ This document has no IANA actions.
- Added the optional video `preference` field: a consumer selects automatically among the highest preference it can decode.
- Added the optional video `reorder` field: the most frames that can precede one in decode order and follow it in presentation, fixed for the track once declared. A consumer that authors decode times derives them from it and the current group alone, refusing deeper reordering; without it, a consumer infers them from timestamps.
- Added an optional `epoch` beside every rendition and data track `broadcast` reference ({{field-epoch}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `epoch` fails a rendition proven to resolve another instance, and is best effort over a route without an Epoch.
- Added the optional root `contributions` map ({{contributions}}): whole broadcasts named by a relative `broadcast` and an optional `epoch`, opened only on selection, whose rendition IDs stay their own.
- Added optional `stats` and `echo` catalog sections, publisher/viewer snapshots keyed by rendition ID, and the `.echo` feedback broadcast convention.
- An Opus frame timestamp counts the OpusHead pre-skip: a consumer presents each decoded sample the pre-skip earlier than its frame timestamp places it.

Expand Down
11 changes: 11 additions & 0 deletions js/hang/api/catalog.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,13 @@ export const ContainerSchema: ZodMiniDefault<ZodMiniUnion<readonly [ZodMiniDiscr
kind: string;
}>>]>>;

export type Contribution = infer<typeof ContributionSchema>;

export const ContributionSchema: ZodMiniObject<{
broadcast: ZodMiniType<RelativeBroadcast, string, $ZodTypeInternals<RelativeBroadcast, string>>;
epoch: ZodMiniOptional<ZodMiniType<ReferenceEpoch, string, $ZodTypeInternals<ReferenceEpoch, string>>>;
}, $strip>;

export const DEFAULT_FORMAT: Format;

export class DuplicateRendition extends Error {
Expand Down Expand Up @@ -331,6 +338,10 @@ export const RootSchema: ZodMiniObject<{
timescale: ZodMiniDefault<ZodMiniNumber<number>>;
wall: $ZodBranded<ZodMiniNumber<number>, "u53", "out">;
}, $strip>>;
contributions: ZodMiniOptional<ZodMiniRecord<ZodMiniString<string>, ZodMiniObject<{
broadcast: ZodMiniType<RelativeBroadcast, string, $ZodTypeInternals<RelativeBroadcast, string>>;
epoch: ZodMiniOptional<ZodMiniType<ReferenceEpoch, string, $ZodTypeInternals<ReferenceEpoch, string>>>;
}, $strip>>>;
echo: ZodMiniOptional<ZodMiniUnion<readonly [ZodMiniObject<{
path: ZodMiniType<RelativeBroadcast, string, $ZodTypeInternals<RelativeBroadcast, string>>;
}, $strip>, ZodMiniPipe<ZodMiniCustom<unknown, unknown>, ZodMiniTransform<undefined, unknown>>]>>;
Expand Down
6 changes: 6 additions & 0 deletions js/hang/src/catalog/consumer.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ test("the shared containment check covers every section carrying a broadcast ref
expect(() => checkResolvable(reference("../../x"), base)).toThrow(EscapingBroadcast);
expect(checkResolvable(reference("../x"), base)).toBeDefined();
}

// A contribution is a whole broadcast, keyed directly under its section.
const contribution = (broadcast: string) =>
({ contributions: { entry: { broadcast: Moq.Path.normalizeRelative(broadcast) } } }) as Root;
expect(() => checkResolvable(contribution("../../x"), base)).toThrow(EscapingBroadcast);
expect(checkResolvable(contribution("../x"), base)).toBeDefined();
});

test("watch yields a catalog with many renditions", async () => {
Expand Down
20 changes: 20 additions & 0 deletions js/hang/src/catalog/contribution.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import * as z from "@zod/mini";
import { EpochSchema, RelativeBroadcastSchema } from "./path";

/**
* A whole broadcast whose own catalog describes its renditions, listed without opening it.
*
* Listing one subscribes to nothing: a consumer opens the contribution's catalog only when it
* selects one of its renditions. Its rendition IDs belong to that broadcast, never the listing
* catalog.
*/
export const ContributionSchema = z.object({
// The contribution's broadcast, relative to the broadcast serving this catalog.
broadcast: RelativeBroadcastSchema,
// The publisher instance the reference names, with a rendition `epoch`'s meaning. A composer
// leaves it out until the contribution resolves.
epoch: z.optional(EpochSchema),
});

/** A whole broadcast listed for explicit, lazy selection. */
export type Contribution = z.infer<typeof ContributionSchema>;
4 changes: 3 additions & 1 deletion js/hang/src/catalog/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
/**
* JSON catalog schema and types describing a broadcast's tracks: audio/video decoder configs for
* WebCodecs, container format, rendition layout and HLS records, the root `clock`,
* plus the `json` and `binary` data sections and optional `stats` and `echo` discovery.
* plus the `json` and `binary` data sections, optional `stats` and `echo` discovery, and whole
* `contributions` listed without opening them.
*
* @module
*/
Expand All @@ -23,6 +24,7 @@ export { AudioConfigSchema, AudioSchema } from "./audio";
export * from "./binary";
export * from "./clock";
export * from "./container";
export * from "./contribution";
export * from "./echo";
export * from "./format";
export * from "./hex";
Expand Down
25 changes: 25 additions & 0 deletions js/hang/src/catalog/root.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ test("rendition broadcast reference is parsed and normalized", () => {
test("every reference-bearing section carries an optional epoch", () => {
const EPOCH = "01890a5d-ac96-774b-bcce-b302099a8057" as ReferenceEpoch;
const catalog = (epoch: string) => ({
contributions: { c: { broadcast: "./a", epoch } },
video: {
renditions: {
v: { broadcast: "./a", epoch, codec: "avc1.64001f", container: { kind: "legacy" } },
Expand Down Expand Up @@ -84,13 +85,37 @@ test("every reference-bearing section carries an optional epoch", () => {
expect(parsed.text?.renditions.t?.epoch).toBe(EPOCH);
expect(parsed.json?.tracks.j?.epoch).toBe(EPOCH);
expect(parsed.binary?.tracks.b?.epoch).toBe(EPOCH);
expect(parsed.contributions?.c?.epoch).toBe(EPOCH);

// A malformed or noncanonical epoch refuses the catalog, as in Rust.
for (const bad of ["not-a-uuid", EPOCH.toUpperCase(), "01890a5d-ac96-474b-bcce-b302099a8057"]) {
expect(RootSchema.safeParse(catalog(bad)).success).toBe(false);
}
});

// Mirrors Rust's `contributions_roundtrip_and_refuse_malformed_entries`.
test("a contribution parses beside a same-ID rendition and a malformed one is refused", () => {
const video = { alice: { codec: "avc1.64001f", container: { kind: "legacy" } } };
const parsed = RootSchema.parse({
contributions: { alice: { broadcast: "./alice.hang" } },
video: { renditions: video },
});
expect(parsed.contributions?.alice).toEqual({ broadcast: "alice.hang" as RelativeBroadcast });
// Contribution IDs are their own namespace, so one may share a rendition's ID.
expect(() => checkRenditions(parsed)).not.toThrow();
expect(encode(parsed).version).toBeUndefined();

for (const contributions of [
[],
{ alice: null },
{ alice: {} },
{ alice: { broadcast: 42 } },
{ alice: { broadcast: "./alice.hang", epoch: "not-an-epoch" } },
]) {
expect(RootSchema.safeParse({ contributions }).success).toBe(false);
}
});

test("rendition parent broadcast reference stays distinct from empty", () => {
const catalog = {
video: {
Expand Down
9 changes: 7 additions & 2 deletions js/hang/src/catalog/root.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import * as z from "@zod/mini";
import { AudioSchema } from "./audio";
import { BinarySchema } from "./binary";
import { ClockSchema } from "./clock";
import { ContributionSchema } from "./contribution";
import { EchoSchema } from "./echo";
import { TRACK } from "./format";
import { JsonSchema } from "./json";
Expand Down Expand Up @@ -48,8 +49,9 @@ const VersionSchema = z.number().check(
* The media sections are `video`, `audio`, and `text`, alongside the broadcast's `clock` (the
* one wall-clock mapping);
* `json` and `binary` list application data tracks that aren't media. A section is omitted
* when it holds no tracks. `stats` names the publisher's media snapshot track, and `echo`
* solicits viewer feedback under a relative broadcast prefix.
* when it holds no tracks. `stats` names the publisher's media snapshot track, `echo`
* solicits viewer feedback under a relative broadcast prefix, and `contributions` lists whole
* broadcasts without opening them.
*
* This is a *loose* object: unknown root sections pass through validation untouched, so an
* application can add its own sections (e.g. `scte35`) without modifying hang. A base consumer
Expand Down Expand Up @@ -79,6 +81,8 @@ export const RootSchema = z.looseObject({
text: section(TextSchema, "renditions"),
json: section(JsonSchema, "tracks"),
binary: section(BinarySchema, "tracks"),
// A new key, so a malformed entry fails the catalog, the same as Rust.
contributions: z.optional(z.record(z.string(), ContributionSchema)),
});

/** The root catalog object: the media sections, the clock, the data track sections, plus any app extensions. */
Expand Down Expand Up @@ -163,6 +167,7 @@ export function checkResolvable(root: Root, base: Moq.Path.Valid): Root {
root.text?.renditions,
root.json?.tracks,
root.binary?.tracks,
root.contributions,
];
for (const section of sections) {
for (const { broadcast } of Object.values(section ?? {})) {
Expand Down
1 change: 1 addition & 0 deletions js/watch/api/index.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ export namespace Audio {
export class Broadcast {
constructor(props?: Signals.Inputs<BroadcastInput>);
close(): void;
contribution(effect: Signals.Effect, id: string): Broadcast;
readonly in: Signals.Readonlys<BroadcastInput>;
readonly out: Signals.Readonlys<BroadcastOutput>;
}
Expand Down
Loading