Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions doc/concept/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ A few things the catalog can express beyond decoder config:
- **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the rendition's ID, so labels need not be unique and renaming one doesn't rename the track.
- **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected. `@moq/watch` and `moqsrc` follow a referenced path's announcements like their own, restarting the rendition on each instance that serves it; every other reader, such as `moq play`, the exporters, and the bindings, waits for the first one announced and keeps it, and an exporter ends the rendition with it. An optional `instance` beside the path names the [stream](/concept/moq-lite#publisher-instances) it expects: a route of another instance ends the rendition instead of falling back to it, while a route whose instance a session minted (one that crossed an older version) cannot prove one, so there it is best effort. Omit `instance` for a broadcast produced on demand, which has none until a worker answers; see [identity and recovery](/concept/identity).
- **Whole includes.** A root `include` map names whole broadcasts, such as transcoded renditions of the source, by the same relative path and optional `instance` as a rendition's reference. They share the listing catalog's PTS timeline and root settings, including its clock and join interval. Listing one opens nothing, so an include produced on demand starts only once a viewer selects it, which `@moq/watch` does with `broadcast.include(effect, id)`. Its rendition IDs stay its own.
- **DRM.** A root `drm` map describes the DRM systems protecting Common Encryption CMAF renditions, one entry per system, scheme, and key: its `system` and `kid` UUIDs, the `pssh` box, and the `license` and `certificate` servers. A protected rendition lists its entries' IDs in its own `drm` field, keeps the encrypted sample entry in its `init`, and names the codec that entry wraps. A rendition listing any ID is protected, even when its entry is one a reader cannot parse, so an unknown scheme or a malformed entry hides only the renditions it protects. An entry that is not a JSON object, or an ID the map lacks, still refuses the whole catalog. The MSF catalog carries the same entries as CMSF `contentProtections`. See [Content protection](#content-protection).
- **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition.
- **Jitter.** A rendition can say how much media it writes behind a frame's timestamp, in whole milliseconds rounded up: the reorder delay of B-frames and the span of a fragment or batch written together. It is measured from media timestamps alone, never the time a frame was written, so neither a constant encoder delay nor a capture clock counts. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent.
- **Delay.** A rendition can also say how far its frames' timestamps trail those another rendition already wrote, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. A rendition alone measures none. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's.
Expand Down Expand Up @@ -240,6 +241,42 @@ derived from the credential and a semantic name such as `foo.hang`; it carries
no format marker. `moq-crypto` is the Rust implementation. Relays need no
configuration to carry it.

## Content protection

DRM is separate from moq-e2ee: the media itself is Common Encryption CMAF,
encrypted before it reaches MoQ by a packager such as Shaka Packager, and a
relay carries it like any other CMAF. The catalog's `drm` section names each
protected rendition's DRM systems: the scheme, key, and `pssh` from its init's
`schm`, `tenc`, and `pssh` boxes, and the license and certificate servers the
publisher supplies, which the media does not carry.

```json
{
"drm": {
"1": {
"system": "1077efec-c0b2-4d02-ace3-3c1e52e2fb4b",
"scheme": "cbcs",
"kid": "a1b2c3d4-e5f6-0718-293a-4b5c6d7e8f90",
"pssh": "AAAANHBzc2gB...",
"license": { "url": "https://license.example.com/clearkey" }
}
},
"video": { "renditions": { "video": { "codec": "avc1.64000c", "drm": ["1"], "container": { "kind": "cmaf", "init": "..." } } } }
}
```

WebCodecs cannot decrypt, so `@moq/watch` treats a protected rendition as
unplayable and never selects it. Playback through MSE and EME is not built yet.
Nothing in the Rust tools decrypts either: `moq play`, `moq export`, the HLS
gateway, transcoding, and the C and UniFFI bindings leave a protected rendition
out, as they would a codec they cannot decode. A recording stores every track as
published, protected ones included, and playing it back leaves them out the
same way. Every reader, `@moq/watch` included, also counts a CMAF rendition
whose `init` holds an encrypted sample entry (`encv`/`enca`) as protected when
it lists no `drm`. An entry a reader cannot parse, such as a scheme a later
draft adds, hides only the renditions that list it; an entry that is not a JSON
object refuses the catalog.

## Your own format

hang is a convention, not a requirement. If you control both ends, publish
Expand Down
45 changes: 43 additions & 2 deletions drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ author:

normative:
moqmsf: I-D.ietf-moq-msf
moqcmsf: I-D.ietf-moq-cmsf
cenc:
title: "Information technology - MPEG systems technologies - Part 7: Common encryption in ISO base media file format files"
author:
- org: ISO/IEC
seriesinfo:
ISO/IEC: 23001-7
date: false
moql: I-D.lcurley-moq-lite
moqflate:
title: "DEFLATE Compressed Tracks for MoQ"
Expand Down Expand Up @@ -160,6 +168,7 @@ type Catalog = {
"broadcast": string,
"instance": string | undefined,
}> | undefined,
"drm": Record<string, DrmSchema> | undefined,
// ... any custom fields ...
}
~~~
Expand All @@ -172,7 +181,7 @@ For example, a chat entry should name a chat track, not carry individual chat me
This way catalog updates are rare and a client MAY choose to not subscribe.
The `json` and `binary` sections ({{data}}) are how a track like that is listed.

This specification defines audio, video, and text media tracks, an optional `media` section ({{media}}) holding the clock that maps content time to wall time and the interval a viewer can join on, the root `version` ({{catalog-version}}), the application data tracks in {{data}}, optional publisher statistics and solicited viewer feedback ({{media-stats}}), and whole broadcasts listed in `include` ({{include}}).
This specification defines audio, video, and text media tracks, an optional `media` section ({{media}}) holding the clock that maps content time to wall time and the interval a viewer can join on, the root `version` ({{catalog-version}}), the application data tracks in {{data}}, optional publisher statistics and solicited viewer feedback ({{media-stats}}), whole broadcasts listed in `include` ({{include}}), and the DRM systems protecting encrypted renditions ({{drm}}).

## Version {#catalog-version}
The root `version` field is the oldest draft of this specification that reads the catalog correctly, by draft number: `4` needs a consumer of moq-hang-04 or later.
Expand All @@ -188,7 +197,7 @@ This draft needs `version` 4 for:
- a `track` ({{field-track}}) other than the entry's ID, which an older consumer subscribes to instead;
- a video `warmup` ({{field-warmup}}) above zero, since an older consumer starts decoding a group at a picture that is not a keyframe.

Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `media` ({{media}}) only offers the clock and join interval, which an older consumer goes without; `stats` and `echo` ({{media-stats}}) only offer optional tracks; `include` ({{include}}) only offers broadcasts to open on selection; video `reorder` leaves an older consumer inferring decode times from timestamps; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.
Every other catalog change in this draft is safe to ignore: `enabled` ({{field-enabled}}) leaves an older consumer selecting a disabled rendition, which plays nothing; `media` ({{media}}) only offers the clock and join interval, which an older consumer goes without; `stats` and `echo` ({{media-stats}}) only offer optional tracks; `include` ({{include}}) only offers broadcasts to open on selection; `drm` ({{drm}}) leaves an older consumer selecting a protected rendition it then fails to decode, which raising `version` would not improve, since that would also refuse it the clear renditions; video `reorder` leaves an older consumer inferring decode times from timestamps; `jitter`, `delay`, audio `warmup`, and the data track estimates are hints.

## Media {#media}
The catalog's root `media` field holds what applies to every audio and video track:
Expand Down Expand Up @@ -258,6 +267,37 @@ An include's rendition IDs belong to its own catalog and MUST NOT replace IDs in
An unavailable include does not invalidate the listing catalog.
A consumer does not expand an include's own `include`: each needs a selection of its own.

## DRM {#drm}
The optional root `drm` map describes the DRM systems protecting encrypted renditions.
Each key is an entry ID, and each entry is one DRM system for one Common Encryption scheme and one key:

~~~
type DrmSchema = {
"system": string,
"scheme": string,
"kid": string,
"pssh": string | undefined,
"license": { "url": string, "type": string | undefined } | undefined,
"certificate": { "url": string, "type": string | undefined } | undefined,
}
~~~

The `system` field is the DRM system ID and `kid` the default key ID, each a hyphenated UUID string, and `scheme` is the Common Encryption scheme {{cenc}}, `cenc` or `cbcs` as CMSF allows {{moqcmsf}}.
The `pssh` field is the system's whole `pssh` box, base64-encoded ({{!RFC4648, Section 4}}).
The `license` field names the server a consumer requests a license from, and `certificate` the server certificate a system such as FairPlay needs first; each `type` is the license protocol or the certificate's MIME type.
Renditions encrypted under the same system, scheme, and key MAY share an entry.

A protected audio or video rendition lists the IDs of the entries that can decrypt it in its `drm` field, an array of strings; absent or empty lists none, and the rendition is clear unless its `init` holds an encrypted sample entry, as below.
Common Encryption is defined for CMAF, so a protected rendition is in practice a `cmaf` one ({{cmaf}}): its `init` keeps the encrypted sample entry with its `sinf`, and the `pssh` boxes, while its `codec` names the codec the encrypted entry wraps.
A rendition listing any ID is protected, whether or not the consumer can parse the entries it names.
A consumer MUST NOT reject a catalog for an entry it cannot parse, such as one with another `scheme` or a malformed UUID or URL: the renditions listing that entry stay protected, and the rest of the catalog stays usable.
A consumer MUST reject a catalog with a rendition listing an ID the root `drm` lacks, and an entry that is not a JSON object.
A consumer MUST treat a `cmaf` rendition whose `init` holds an encrypted sample entry (`encv` or `enca`) as protected, even when it lists no entry.
A consumer that cannot decrypt a protected rendition MUST treat it as one it cannot decode.

Each entry maps onto a CMSF `contentProtections` entry {{moqcmsf}}: the ID is its `refID`, `kid` its single `defaultKID`, `system`, `license`, `certificate`, and `pssh` its `drmSystem`'s `systemID`, `laURL`, `certURL`, and `pssh`, and a rendition's `drm` its `contentProtectionRefIDs`.
A CMSF entry naming several `defaultKID`s has no single hang entry; a consumer converting it keeps it as an entry it cannot parse, so the renditions listing it stay protected.

## Track Identity {#identity}
A track name always means the same content, so a catalog update never redefines an audio or video track.
A track's identity is the fields a consumer cannot change without reinitializing its decoder or demuxer: `codec`, `description`, and `container`, plus `displayAspectWidth`, `displayAspectHeight`, and `interlaced` for video and `sampleRate` and `numberOfChannels` for audio.
Expand Down Expand Up @@ -1129,6 +1169,7 @@ This document has no IANA actions.
- Added the optional video `preference` field: a consumer selects automatically among the highest preference it can decode.
- Added the optional video `reorder` field: the most frames that can precede one in decode order and follow it in presentation, fixed for the track once declared. A consumer that authors decode times derives them from it and the current group alone, refusing deeper reordering; without it, a consumer infers them from timestamps.
- Added an optional `instance` beside every rendition and data track `broadcast` reference ({{field-instance}}). A consumer reads each referenced path from the first broadcast it resolves for a catalog instance; a present `instance` fails a rendition that resolves another stream, and is best effort against an Instance a receiver minted.
- Added the optional root `drm` map ({{drm}}) and the audio and video `drm` field: the DRM systems, scheme, and key protecting a Common Encryption `cmaf` rendition, mapping onto CMSF `contentProtections`. A rendition listing any entry is protected, and an entry a consumer cannot parse hides only the renditions that list it.
- Added the optional root `include` map ({{include}}): whole broadcasts named by a relative `broadcast` and an optional `instance`, opened only on selection, whose rendition IDs stay their own.
- Includes share the listing catalog's PTS timeline and root settings, including its clock and join interval.
- An HLS master selects the includes it lists by reading their catalogs, without demanding media, and segments them on the listing catalog's `media`; an on-demand publisher answers its catalog without starting media production.
Expand Down
Loading
Loading