Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/concept/standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ subset drops. What a peer actually observes against this implementation:
- **Pull.** Subscribers ask. Single-track `PUBLISH` offers are declined; announce a namespace and serve the resulting subscriptions. Announcements go out unsolicited, and we also ask for every prefix we may discover. The solicit `SETUP` option makes us wait to be asked. Without it, a draft 16+ peer also gets each match as a `NAMESPACE` on its `SUBSCRIBE_NAMESPACE` stream, so it hears the namespace twice. We send `PUBLISH` only to answer a `SUBSCRIBE_TRACKS` (below): on drafts 16 and 17 a `SUBSCRIBE_NAMESPACE` asking only for `PUBLISH` is refused, one asking for both gets only `NAMESPACE`, and any other Subscribe Options value closes the session.
- **`SUBSCRIBE_TRACKS` lists, it never pushes.** In Rust, on draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Each paused `PUBLISH` holds a subscription slot, and past the cap a track gets `PUBLISH_SKIPPED` (`PUBLISH_BLOCKED` on draft 18) instead. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. A Rust relay does not yet send `SUBSCRIBE_TRACKS` upstream, and JavaScript refuses it with `NOT_SUPPORTED`.
- **History is one group.** In Rust, Object ID gaps must be marked by Prior Object ID Gap properties for a group to be fetchable. A `FETCH` returns one group from the cache, or on drafts 14 through 19 the saved prefix of the group a new subscription just joined. A range of groups is refused, as is a draft 20+ `LOCATION_FILTER` that ends at Largest Object rather than an absolute End Location. On draft 20 and later, `FETCH_OK`'s End Location is the requested end, capped at Largest Object: a range holding no objects gets an empty fetch stream, and only a start past Largest Object is refused, with `INVALID_RANGE`. A relay's copy with no upstream subscription can't know Largest Object short of the track's end, so it echoes the requested end instead. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed, and so does a reader whose publisher refuses `TRACK_STATUS` as `NOT_SUPPORTED`.
- **Compression.** Rust peers on draft 17 and later exchange compressed tracks when both offer the [Flate extension](/draft/moq-flate). Older peers receive one shared inflated copy. A publisher that seals the compression declaration makes that fallback unavailable: an incompatible peer is refused, preserving the sealed metadata. JavaScript does not offer the IETF extension.
- **Timing.** A track whose `SUBSCRIBE_OK` declares no `TIMESCALE` is untimed, as is every track on drafts 14 through 16. A Rust reader that never subscribed takes the units from `TRACK_STATUS` on draft 18 and later.
- **Strict SETUP and parameters.** Repeated unknown `SETUP` options, GREASE included, are accepted if well-formed; a repeated known option is rejected. On draft 17 and later, a `GROUP_ORDER` other than Ascending (1) or Descending (2) closes the session. Message parameters follow the negotiated draft's lists: drafts 14 and 15 ignore an unknown or misplaced parameter, draft 16 ignores one defined only for another message but closes the session on an unknown one, and draft 17 and later close it on both. Other malformed control input closes the session with the draft's code too, such as a server `SETUP` carrying `PATH` or `AUTHORITY`, a request cut short by FIN, or a second `GOAWAY` on one stream; a first `GOAWAY` on a request stream is legal.
- **One `SETUP` credential**, forwarded to the [auth server](/bin/relay/auth#the-contract) unverified. A second `SETUP` token closes the session. We advertise no token cache, so an alias reference in `SETUP` closes it with `PROTOCOL_VIOLATION`, one on a request with `UNKNOWN_AUTH_TOKEN_ALIAS`, and a registration on a request with `AUTH_TOKEN_CACHE_OVERFLOW`. Where the Rust side authorizes the peer or verifies tokens, a token on a `SUBSCRIBE` or `PUBLISH_NAMESPACE` is verified like an in-band session token and authorizes that one request alone, even where that authorization already covers it; it never widens the session. It is refused with `NOT_SUPPORTED` when nothing verifies it, when it is empty, or when its Token Type is anything but 0 (out of band). A side that does neither, JavaScript included, ignores the token. Rust refuses a request carrying more than one distinct token with `NOT_SUPPORTED`, since it verifies one; JavaScript, which verifies none, ignores them all. A token-authorized `SUBSCRIBE` still stays within the grant the peer gave us. A token-authorized `PUBLISH_NAMESPACE` needs a grant covering everything beneath the namespace, and the tracks we subscribe under it rely on that announce instead of our limit, ending with it. An update renews the token, except on a drafts 14 and 15 `PUBLISH_NAMESPACE`, which has no update, and a refused renewal ends only that request. Rust and JavaScript send the token a route names (`Route::auth`) on its `PUBLISH_NAMESPACE`, including the one drafts 14 and 15 answer a solicited `SUBSCRIBE_NAMESPACE` with, and swapping it renews the request with `REQUEST_UPDATE` from draft 16, or ends and re-sends it before; the inline `NAMESPACE` that answers a solicited `SUBSCRIBE_NAMESPACE` from draft 16 carries no token. The token belongs to the session that added it, so a reconnect means adding it again and re-announcing. A token on any other request is decoded and grants nothing, and a request repeating one is refused with `MALFORMED_AUTH_TOKEN`. JavaScript's `SUBSCRIBE_TRACKS` reads its namespace before refusal; its token is not decoded. On draft 17 and later, the negotiated [MoQ Auth extension](/concept/moq-lite#authorization) accepts additional session tokens in band.
Expand Down
45 changes: 43 additions & 2 deletions drafts/draft-lcurley-moq-flate.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,39 @@ A consumer MUST treat an object that inflates to more or less than its declared

A relay forwards the compressed objects as received to a peer whose transport declares compression.
For any other peer it inflates them, once per subgroup however many such peers it serves, and sends the original objects without the declaration.
If a declaration is sealed inside Immutable Properties, the relay MUST preserve those bytes and MUST refuse delivery when it cannot remove the declaration for an inflated copy.
A peer is never sent compressed bytes it was not told about.


# MoQ Transport Signaling {#signaling}
This extension applies to MoQ Transport draft-17 and later.
A peer offers it with the FLATE Setup Option, key `0x40B6A`, an integer with value 1.
Other values are invalid.
Both endpoints MUST offer FLATE before either sends a compressed track.
Draft-14 through draft-16, and a peer that did not offer FLATE receive inflated payloads without these declarations, subject to the sealed-property refusal in {{declare}}.

## Compression Track Property {#property}
COMPRESSION, property type `0x40B6C`, has Track scope and integer value 1 for the format in {{stream}}.
Absence means plain payloads; other values are invalid.
A receiver MUST refuse a compressed track unless FLATE was negotiated.
The property can appear in the mutable list or Immutable Properties, but MUST NOT appear more than once across them.
A relay recognizes it rather than retaining a second mutable copy among unknown properties.
The declaration applies even when a subscriber omits Track Properties from an answer using INCLUDE_PROPERTIES=0: the subscriber already knows the track's properties.

## Inflated Size Object Property {#size}
INFLATED_SIZE, property type `0x40B6E`, has Object scope and an integer value equal to the payload's size before compression.
Every Normal Object on a compressed track MUST carry it, including an empty payload, whose size is zero.
A plain track or a status object MUST NOT carry it.
It can appear in the mutable list or Immutable Properties, but MUST NOT appear more than once across them.
The base Object Payload Length remains the size of the compressed bytes.
The same rules apply to subgroup streams, FETCH streams, and datagrams.

A receiver MUST validate the declared size against the same resource limits as an uncompressed payload before admitting it, and MUST NOT allocate a buffer solely from that declaration.
Incremental inflate output is charged to the receiver's cache budget and shared across its plain readers.
A relay restates the size beside forwarded compressed bytes and removes its mutable declaration beside inflated bytes.
A sealed declaration MUST match the stored object's inflated size; a relay MUST refuse forwarding when that cannot be preserved.


# Security Considerations
A shared window is a side channel.
The compressed size of one payload reveals how much it has in common with the payloads before it, enough to recover a secret an attacker can partially guess, as CRIME and BREACH demonstrated against TLS and HTTP compression.
Expand All @@ -99,8 +129,18 @@ Each open subgroup also holds a window of up to 32 KiB, so a consumer SHOULD bou

# IANA Considerations

This document requests no registrations.
The format lives inside object payloads, and the declaration in the transport that carries them.
This document requests the following entry in the MOQT Setup Options registry:

| Value | Name | Specification |
|:------|:-----|:--------------|
| 0x40B6A | FLATE | This Document |

It also requests these entries in the MOQ Properties registry, outside the application-specific and Mandatory Track Property ranges:

| Value | Name | Scope | Specification |
|:------|:-----|:------|:--------------|
| 0x40B6C | COMPRESSION | Track | This Document |
| 0x40B6E | INFLATED_SIZE | Object | This Document |


--- back
Expand All @@ -109,6 +149,7 @@ The format lives inside object payloads, and the declaration in the transport th

## Since draft-lcurley-moq-flate-00 (in progress)

- Negotiated FLATE signaling adds COMPRESSION and INFLATED_SIZE properties to MoQ Transport draft-17 and later; sealed declarations are forwarded unchanged or refused when incompatible.
- Compression is declared by the transport instead of the application: {{moql}} carries a Compression codec in TRACK_INFO and each object's Inflated Size on FRAME and datagrams.
- A relay inflates for a peer whose transport does not declare compression.
- A datagram is compressed alone, as a stream of its own.
Expand Down
12 changes: 6 additions & 6 deletions quest/m1/compress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,12 @@ Every interview prompt and the selected answer (✅) are in the planning PR.
built on first demand, shared across readers, billed to the cache pool, and
dropped with its group. A thousand old-lite viewers cost one inflate.
- **Signaling:** lite-07 TrackInfo carries the codec id and the frame header the
size; IETF draft-16+ carries them only
after both peers advertise a setup parameter
([IETF](/quest/m1/compress/ietf.md)), because unknown track properties are
discarded. Lite-01..06, IETF draft-14/15, and IETF peers without the
parameter get the inflated copy, with no flag.
size; IETF draft-17+ carries them only
after both peers advertise a setup parameter, because unknown track
properties are discarded. Lite-01..06, IETF draft-14/15/16, and IETF peers without the
parameter get the inflated copy, with no flag. A sealed compression
declaration is preserved, so delivery is refused when plain fallback would
remove it.
- **Ranges.** A range starting at frame N of a compressed group is served
from N. The requester only asks from N because it holds 0..N-1, so it
inflates from its own cached prefix. Reading frame N without 0..N-1 is a
Expand Down Expand Up @@ -81,7 +82,6 @@ This README owns the end-to-end proof and the guide once every child lands:
## Required

- [Mid-group subscriptions widen](/quest/m1/compress/app-range-widen.md) - an application range starting mid-group on a compressed track widens to frame 0 at the subscriber instead of failing with `Lagged`
- [IETF extension](/quest/m1/compress/ietf.md) - moq-transport draft-16+ peers that negotiate the extension exchange compressed tracks
- [JavaScript](/quest/m1/compress/js.md) - js/net publishes and consumes compressed tracks on lite-07
- [Migrate](/quest/m1/compress/migrate.md) - catalogs, stats, and timelines publish one compressed track; moq-flate, @moq/flate, and every `.z` name are gone

Expand Down
33 changes: 0 additions & 33 deletions quest/m1/compress/ietf.md

This file was deleted.

23 changes: 22 additions & 1 deletion rs/moq-net/benches/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,8 @@ struct Shape {
window: usize,
/// Group streams a viewer's relay may have open toward it at once.
credit: Option<usize>,
/// Publish compressed groups and exercise negotiated versus plain fallback egress.
compress: bool,
/// Read each group via a fresh FETCH rather than a live subscription.
fetch: bool,
}
Expand All @@ -138,6 +140,7 @@ impl Shape {
window: 1024 * 1024,
credit: None,
fetch: false,
compress: false,
};

/// What each viewer subscribes with: a burst is a backlog, which the default
Expand Down Expand Up @@ -373,7 +376,11 @@ impl Cluster {
let broadcast = origin
.publish(path(publisher * shape.broadcasts + index), Default::default())
.unwrap();
this.tracks.push(broadcast.create_track(TRACK, timed()).unwrap());
this.tracks.push(
broadcast
.create_track(TRACK, timed().with_compress(shape.compress))
.unwrap(),
);
this._broadcasts.push(broadcast);
}

Expand Down Expand Up @@ -1205,6 +1212,20 @@ fn session(c: &mut Criterion) {
}
let base = Shape::BASE;

delivery(
c,
"compression",
[1, 16].into_iter().flat_map(|publishers| {
[1, 16].map(move |viewers| Shape {
publishers,
viewers,
compress: true,
frame: 1024,
..base
})
}),
);

delivery(
c,
"publishers",
Expand Down
2 changes: 2 additions & 0 deletions rs/moq-net/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ impl Client {
ietf::hidden::into_setup(&mut parameters, ietf_encoding);
ietf::max_age::into_setup(&mut parameters, ietf_encoding);
ietf::active_count::into_setup(&mut parameters, ietf_encoding);
ietf::flate::into_setup(&mut parameters, ietf_encoding);
let parameters = parameters.encode_bytes(ietf_encoding)?;

let client = setup::Client {
Expand Down Expand Up @@ -462,6 +463,7 @@ impl Client {
hidden: ietf::hidden::from_setup(&parameters, v),
max_age: ietf::max_age::from_setup(&parameters, v),
active_count: ietf::active_count::from_setup(&parameters, v),
flate: ietf::flate::from_setup(&parameters, v)?,
..Default::default()
};

Expand Down
Loading
Loading