Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"rs/libmoq",
"rs/moq-audio",
"rs/moq-bench",
"rs/moq-binary",
"rs/moq-boy",
"rs/moq-cli",
"rs/moq-ffi",
Expand Down Expand Up @@ -39,6 +40,7 @@ default-members = [
"rs/moq-boy",
"rs/moq-audio",
"rs/moq-bench",
"rs/moq-binary",
"rs/moq-cli",
# "rs/moq-ffi", # requires Python/maturin
# "rs/moq-gst", # requires GStreamer
Expand Down Expand Up @@ -113,6 +115,7 @@ loom = { version = "0.7.2", features = ["futures"] }
# `async` awaits the event channel instead of blocking a thread on it.
mdns-sd = { version = "0.20", features = ["async"] }
moq-audio = { version = "0.0.20", path = "rs/moq-audio" }
moq-binary = { version = "0.1.0", path = "rs/moq-binary" }
moq-flate = { version = "0.1.1", path = "rs/moq-flate" }
moq-hls = { version = "0.4.11", path = "rs/moq-hls", default-features = false }
moq-json = { version = "0.3.6", path = "rs/moq-json" }
Expand Down
15 changes: 15 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

96 changes: 94 additions & 2 deletions doc/concept/layer/hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ The compression is the group-scoped `deflate-raw` ([RFC 1951](https://www.rfc-ed
To read the compressed track, opt in explicitly: pass `--catalog-format hangz` to `moq export`, `CatalogFormat::HangZ` in Rust, or `catalogFormat: "hangz"` to `@moq/watch`.
The `.hang` broadcast suffix is unchanged: the compressed track is an extra track on the same broadcast, not a different broadcast name.

`catalog.json.z` and the `.timeline.z` tracks are always compressed and known by their role.
Everywhere else compression is declared, not guessed: a data track carries a `compression` field (see below), and the `.z` suffix on a track name is a naming convention a consumer must never read as a signal.

### Audio

[See the latest schema](https://github.com/moq-dev/moq/blob/main/js/hang/src/catalog/audio.ts).
Expand Down Expand Up @@ -119,9 +122,96 @@ Both refuse a target no reference can name, since a path segment may itself be c

`@moq/watch` resolves the reference automatically. In Rust, the `moq-mux` exporters do the same: they take a `Source::new(origin, path)`, and both the catalog broadcast and any referenced broadcast resolve through the origin over the same connection.

### Data tracks

Not everything in a broadcast is media. A chat log, a telemetry feed, a thumbnail, a serialized game state: the catalog lists these in two sections alongside `video` and `audio`, split by what a generic consumer can do with the payload.

```json
{
"json": {
"tracks": {
"chat": { "mode": "stream", "compression": "deflate" },
"status": { "mode": "snapshot" }
}
},
"binary": {
"tracks": {
"thumbnail": { "mode": "snapshot", "mime": "image/jpeg" }
}
}
}
```

A `json` track's frames are UTF-8 JSON values, so a relay, a debugger, or an archiver can parse and re-serialize them without knowing the application. A `binary` track's frames are opaque bytes, which only the application can interpret. Each map is keyed by track name, and unlike the media sections these are not rendition sets: the entries are distinct tracks, not alternatives to choose between. A section is omitted entirely when it holds no tracks.

`mode` is required and says how a track's groups compose its frames. There is no default, because reading an append log as a latest-value document silently discards every payload but the last:

- `snapshot` is lossy. Each group is self-contained and supersedes the previous one, so a consumer reads only the newest and a publisher may drop older ones. A JSON track may follow a group's first frame with merge-patch deltas; a binary track writes one frame per group.
- `stream` is lossless in the sense that nothing supersedes anything else. One payload per frame, in order, all in a single group that is never rolled. A publisher that cannot write a payload closes the track, since a second group would present a gap as if it were a complete log. A consumer reads the one group the track carries and fails the read if a second appears, rather than yielding the remainder as though the log were continuous. Retention is still bounded by the group cache: a log longer than the cache holds evicts its earliest frames, and a consumer that falls behind or joins late then cannot read the log at all, since the read fails when it reaches the evicted prefix rather than silently resuming partway through. That is deliberate, because a partial log presented as a whole one is what this mode exists to prevent, and under compression the retained frames are undecodable anyway without the evicted prefix as context. Keep a stream track's log inside what its groups retain, and split anything unbounded across successive tracks.

`compression` names the compression applied to the frames, or is absent when they are uncompressed. `deflate` is the same group-scoped `deflate-raw` the catalog uses. The remaining fields are optional and descriptive: `schema` (a JSON Schema URL) on a JSON track, `mime` on a binary one. Both kinds also accept the `broadcast` and `timeline` fields a media rendition takes.

A consumer that does not recognize a track's `mode` or `compression` must ignore that track. It still round-trips verbatim, so a relay that reparses and republishes the catalog never corrupts a track it cannot read.
Comment thread
kixelated marked this conversation as resolved.
Resolved

#### Publishing and reading

In Rust you create the track on the broadcast, as you would for a media track, and hand it to the catalog. The catalog writes the entry and drops it when the handle drops, so a track is never advertised without a publisher behind it. The catalog key is the track's own name, with no `.z` suffix even when compressed, since the entry's compression flag is what a consumer reads:

```rust
let track = broadcast.create_track("chat", None)?;
let mut chat = catalog.json_stream::<Message>(track, json::Config::default().with_compression(true))?;
chat.append(&message)?;

let track = broadcast.create_track("thumbnail", None)?;
let mut thumbnail = catalog.binary_snapshot(track, binary::Config::default().with_mime("image/jpeg"))?;
thumbnail.update(jpeg)?;
```

There is a producer type per mode (`json_snapshot` / `json_stream`, `binary_snapshot` / `binary_stream`) so `append` on a latest-value track does not compile.

The read side names the track once. `catalog.json_track(name)` returns an entry pairing the name with its config, and the entry subscribes itself:

```rust
let entry = catalog.json_track("chat").expect("no chat track");
let mut chat = entry.subscribe::<Message>(&source).await?;
while let Some(message) = chat.next().await? {
// ...
}
```

The entry supplies the track's mode and compression, so a reader cannot pair the wrong ones with the track, and it resolves through a `Source`, so an entry pointing at a sibling broadcast is followed rather than read from the wrong place.

There is one consumer type rather than one per mode. Both modes hand the caller the same thing, a sequence of values ending when the track does, so a reader writes one loop either way; what differs is loss semantics, and `consumer.mode()` answers that for the rare reader that can only work with one. Discovery is the same entries: `catalog.json_tracks()` and `catalog.binary_tracks()` enumerate them, since the catalog is the only thing that announces a data track.

In the browser the pieces are the same, assembled by hand: read the entry from `catalog.json.tracks` (or `catalog.binary.tracks`), subscribe to the track by name, and hand it to `@moq/json` or `@moq/binary`. Two mappings to do yourself, since the packages take the mode and compression as code rather than as catalog values:

```ts
const entry = catalog.json?.tracks.chat;
if (!entry || !Catalog.modeSupported(entry.mode)) return;
if (entry.compression !== undefined && !Catalog.compressionSupported(entry.compression)) return;

const compression = entry.compression === "deflate";

// Honor a cross-broadcast reference the same way the Rust `Entry::subscribe` does; without this
// you would read an unrelated same-named track in the catalog's own broadcast. A catalog is
// untrusted input, so use `tryResolve`: it returns undefined when the reference escapes above the
// root, which the spec says to ignore rather than clamp onto some other valid broadcast.
let source = broadcast;
if (entry.broadcast !== undefined) {
const path = Path.tryResolve(catalogPath, entry.broadcast);
if (!path) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Accept valid references to the root broadcast

When catalogPath is "a/b" and the entry references "..", Path.tryResolve returns Path.empty(), represented by the empty string, which is a valid root broadcast path. This falsy check treats it like an escaping reference and skips the track, so consumers copied from the canonical browser recipe cannot follow legitimate root references. Check path === undefined instead. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

source = await connection.consume(path);
}
const track = source.subscribe("chat");
const consumer =
entry.mode === "stream" ? new Json.Stream.Consumer(track, { compression }) : new Json.Snapshot.Consumer(track, { compression });
```

`mode` picks the namespace (`Stream` or `Snapshot`), and `compression` is a boolean there while the catalog carries `"deflate"` or nothing. Check both against `modeSupported` / `compressionSupported` first and skip the track if either is unrecognized, rather than guessing.

### Extensions

The base catalog carries only the media sections (`video` and `audio`).
The base catalog carries the media sections (`video` and `audio`) and the data track sections (`json` and `binary`).
Applications add their own root sections (for example `scte35`) without modifying hang.

The catalog is a JSON document published through the merge-patch snapshot helper (the `Snapshot` mode of `@moq/json` / `moq-json`), and an extension is just an extra top-level key:
Expand All @@ -140,7 +230,9 @@ This keeps application-specific sections in the application layer while the base

### Custom tracks

A custom catalog section can carry its payload inline (for low-rate metadata), or it can reference a separate track in the same broadcast (for a stream of data, e.g. a `meta.json` track or an SCTE-35 event track). The relay treats such a track like any other; only the publisher and consumer give it meaning.
Reach for the `json` and `binary` sections above first: they name a track and say how to read it, so a generic consumer can find and decode one without application support.

An application still needs its own catalog section when the metadata is not a track at all (a low-rate value carried inline in the catalog) or when it has to say something the data sections do not model. Such a section can reference a separate track in the same broadcast, which the relay treats like any other; only the publisher and consumer give it meaning.

The `@moq/publish` and `@moq/watch` components publish and subscribe to these tracks generically, with no per-application support. Each exposes a low-level track hook, and the application uses `@moq/json` to encode the payload itself:

Expand Down
110 changes: 107 additions & 3 deletions drafts/draft-lcurley-moq-hang.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,17 +113,20 @@ type Catalog = {
"video": VideoSchema | undefined,
"timeline": TimelineSchema | undefined,
"text": TextSchema | undefined,
"json": JsonTracks | undefined,
"binary": BinaryTracks | undefined,
// ... any custom fields ...
}
~~~

Additional fields MAY be added based on the application.
The catalog SHOULD be mostly static, delegating any dynamic content to other tracks.

For example, a `"chat"` section should include the name of a chat track, not individual chat messages.
For example, a chat entry should name a chat track, not carry individual chat messages.
This way catalog updates are rare and a client MAY choose to not subscribe.
The `json` and `binary` sections ({{data}}) are how a track like that is listed.

This specification defines audio, video, and text media tracks, plus an optional timeline track ({{timeline}}) indexing their segments.
This specification defines audio, video, and text media tracks, plus an optional timeline track ({{timeline}}) indexing their segments and the application data tracks in {{data}}.

## Video
A video track contains the necessary information to decode a video stream.
Expand Down Expand Up @@ -321,6 +324,102 @@ For example:
}
~~~

## Data Tracks {#data}
Beyond media, a broadcast MAY publish application data tracks: a chat log, a telemetry feed, a thumbnail, a serialized state blob.
The catalog lists them in two sections, split by payload encoding:

~~~
type JsonTracks = {
"tracks": Map<TrackName, JsonSchema>,
}

type BinaryTracks = {
"tracks": Map<TrackName, BinarySchema>,
}
~~~

A `json` track's frames are UTF-8 JSON values; a `binary` track's frames are opaque to everything but the application.
The split is by what a generic consumer can do without knowing the application: parse, inspect, and re-serialize a JSON track, but only copy a binary one.

Each map is keyed by track name, so an entry's key is the name to subscribe to.
Unlike the media sections these are not rendition sets: entries are distinct tracks, not alternatives to choose between.
A publisher MUST omit a section that holds no tracks.

A consumer discovers these tracks the same way it discovers media, by reading the catalog; nothing else in the broadcast announces them.

### JSON {#json-tracks}
~~~
type JsonSchema = {
"mode": Mode,
"compression": Compression | undefined,
"schema": string | undefined,
"broadcast": string | undefined,
"timeline": Timeline | undefined,
}
~~~

The `schema` field is an optional identifier for the shape of each value, typically the URL of a JSON Schema.
It is descriptive: a consumer that does not recognize it MUST still be able to read the track.

### Binary {#binary-tracks}
~~~
type BinarySchema = {
"mode": Mode,
"compression": Compression | undefined,
"mime": string | undefined,
"broadcast": string | undefined,
"timeline": Timeline | undefined,
}
~~~

The `mime` field is an optional media type ({{!RFC6838}}) for each payload, for example `image/jpeg`.
It is descriptive, the same as `schema` above.

### mode {#field-mode}
~~~
type Mode = "snapshot" | "stream"
~~~

The `mode` field says how a track's groups compose its frames into what a consumer sees.
It is REQUIRED and has no default: reading an append log as a latest-value document silently discards every payload but the last.
A consumer MUST ignore a track whose `mode` it does not recognize.

A `snapshot` track is lossy.
Each group is self-contained and supersedes the previous one, so a consumer reads only the newest group and a publisher MAY drop older ones.
A group's first frame is a complete value.
A `json` track MAY follow it with JSON Merge Patch ({{!RFC7396}}) deltas applied in order; a `binary` track MUST write exactly one frame per group, since an opaque payload has no delta form.

A `stream` track is lossless in the sense that nothing supersedes anything else.
It is a single group, never rolled, carrying one self-contained payload per frame, delivered in order.

A publisher that cannot write a payload MUST close the track rather than continue the log in a second group.
Comment thread
kixelated marked this conversation as resolved.
Resolved
A second group would present a gap as if it were a complete log; ending the track surfaces the failure instead, and a publisher with more to say opens a new track.

A consumer MUST NOT skip to the newest group, since a later group does not supersede an earlier one the way a `snapshot` group does.
It reads the one group the track carries, and MUST fail the read if the track carries a second, since whatever would have completed the first group is gone and yielding the remainder would present that gap as a continuous log.
This is why a consumer takes groups in arrival order rather than by ascending sequence: groups are separate streams, so a second group MAY arrive with a lower sequence than the first, and skipping it would report a truncated log as a whole one.

Retention is bounded, and this is the limit of "lossless".
A group's cache is finite, so a publisher that writes more than it holds evicts the log's earliest frames.
A consumer that has not kept up, or that subscribes later, then cannot read the log at all: the read fails once it reaches the evicted prefix, rather than silently resuming at whatever the cache still holds.
That is the intended behaviour, since a partial log presented as a whole one is exactly what this mode exists to prevent, and under compression ({{compression}}) the retained frames are undecodable anyway without the evicted prefix as context.
A publisher SHOULD therefore keep a `stream` track's log within what its groups retain, and split anything unbounded across successive tracks; a consumer that needs the whole log SHOULD subscribe before the publisher exceeds that.

### compression {#field-compression}
~~~
type Compression = "deflate"
~~~

The `compression` field names the compression applied to the track's frames.
If absent, the frames are uncompressed.
A consumer MUST ignore a track whose `compression` it does not recognize, since it cannot decode the frames.

The `deflate` value is the group-scoped DEFLATE of {{compression}}.
A `snapshot` group covers a single value (plus any deltas), so its window spans that group alone; a `stream` group's frames compress against the earlier ones in the log.

### broadcast and timeline {#data-shared}
The `broadcast` ({{field-broadcast}}) and `timeline` ({{field-timeline}}) fields carry the same meaning here as they do for a media rendition.

## Binary Fields {#binary}
A decoder config field carrying raw bytes, notably `description` (an `AllowSharedBufferSource` in WebCodecs), is carried in the catalog as a hex string ({{!RFC4648, Section 8}}).
A publisher SHOULD emit lowercase hexadecimal characters and MUST NOT emit a `0x` prefix or any separators.
Expand Down Expand Up @@ -427,7 +526,12 @@ Each frame is a Low Overhead Container frame {{!I-D.ietf-moq-loc}}: a property b


# Compression {#compression}
Some metadata tracks are compressed, conventionally marked with a `.z` suffix on the track name.
Some metadata tracks are compressed.

Compression is signalled per track, never inferred from the track's name.
A data track ({{data}}) declares it with the `compression` field ({{field-compression}}).
The two tracks this specification defines as always compressed, `catalog.json.z` ({{catalog}}) and the timeline track ({{timeline}}), are identified by their role instead.
The `.z` suffix on those names is a naming convention, and a consumer MUST NOT treat it as a signal.

Each group is one raw DEFLATE stream ({{!RFC1951}}), sync-flushed at each frame boundary.
Each frame is therefore a self-delimited, byte-aligned slice, while later frames compress against the earlier ones in the same group.
Expand Down
Loading
Loading