Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 23 additions & 38 deletions drafts/draft-lcurley-moq-ietf.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Submit an [issue](https://github.com/moq-dev/moq/issues) or [PR](https://github.

A **relayed** group or datagram arrived from a MoQ Transport publisher, so it has object state that moq-lite cannot express.
A **plain** one has none: its Object IDs are its frame indices, it has no properties, and its group ends where its last frame does.
A track's **Version** is the {{moqt}} version its original publisher spoke, which serialized every property it carries, or none for a track published over moq-lite ([Track Stream](#track)).


# Introduction
Expand All @@ -63,6 +64,7 @@ The value MUST be empty; a receiver MUST close the session with a PROTOCOL_VIOLA
The extension is negotiated only when both endpoints offer it, and only on moq-lite-07 and later.
An endpoint MUST NOT send any field this document defines unless the extension is negotiated, and MUST NOT parse a Group Stream, FETCH response, Track Stream, or datagram from the peer until it has the peer's SETUP and so knows whether it was.
An endpoint MAY drop a datagram that arrives before then.
A Group Stream, FETCH response, or datagram is read with the Version of its track, which the Track Stream carries before any subscription or fetch opens.


# Group Streams {#group}
Expand Down Expand Up @@ -91,6 +93,7 @@ A bit field describing the group's relay state, or 0 for a plain group:
- `0x8` **END_OF_GROUP**: The stream's end ends the group, as a {{moqt}} subgroup that contains the end of its group.

A receiver MUST reset the stream with a PROTOCOL_VIOLATION on any other bit, a bit without RELAYED, or a record with Properties when EXTENSIONS is clear.
A track without a Version has no {{moqt}} state, so a receiver MUST reset the stream with a PROTOCOL_VIOLATION on any Group State but 0 for one.
A publisher MUST clear FIRST_OBJECT when the stream starts after the first frame it holds, and END_OF_GROUP when the stream may stop before the end of the group, such as at a subscription's end frame.
A stream that stops at an end frame carries no END_OF_GROUP record, even when that frame is the group's last.

Expand All @@ -114,7 +117,7 @@ Object Record {
**Properties**:
The object's [properties](#properties), in `Properties Length` bytes.
An empty one means the object has none.
A receiver MUST reset the stream if the length exceeds 65,537, the largest Property Block beside an Encoding byte.
A receiver MUST reset the stream if the length exceeds 65,536, the largest Property Block.

In a plain group, every record has a `Skip` of 0 after the first, whose Object ID is the index of its frame, no properties, and `End` of 0.
A receiver MUST reset the stream with a PROTOCOL_VIOLATION on any other record in a plain group.
Expand All @@ -124,39 +127,19 @@ The Object ID is relay state beside it, so a plain subscriber still reads consec


# Properties {#properties}
An object's properties, as one block in the encoding of the {{moqt}} draft it arrived on.
An object's properties, as one Property Block in the encoding of its track's Version.
A relay forwards the block to a peer of that encoding unchanged, so the block never grows on a hop, and holds it as received rather than as a decoded list many times its size.

~~~
Properties {
Encoding (i)
Property Block (..)
}
~~~

**Encoding**:
The {{moqt}} drafts the Property Block came from, which serialize it alike and give each property the same meaning:

| Code | Drafts | IDs | Varints |
|-----:|:-------|:----|:--------|
| 1 | 14 and 15 | absolute | QUIC |
| 2 | 16 | delta | QUIC |
| 3 | 17 | delta | draft 17 |
| 4 | 18 | delta | draft 18 |
| 5 | 19 and later | delta | draft 18 |

A receiver MUST treat any other code as malformed, including 0: an empty Properties says the object has none.
Codes 4 and 5 serialize alike, but only drafts 19 and later assign property 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so a relay knows not to forward 0x06 to a draft that would read it differently.

**Property Block**:
Every property of the object, serialized as the {{moqt}} drafts of its encoding serialize an object's properties, in ascending ID order.
Every property of the object, serialized as the track's Version serializes an object's properties, in ascending ID order.
Immutable Properties (0x0b) are in the same encoding, as the publisher sent them.
It MUST NOT be empty or longer than 65,536 bytes.
A track without a Version carries no properties: a Property Block on one is malformed.
On a timed track an object with a FRAME, or a datagram without a Status, MUST NOT carry one longer than 65,526 bytes, leaving room for the largest Timestamp a relay restates ([Timestamps](#timestamps)): a one-byte key and a value of up to nine bytes.
A relay MUST refuse a {{moqt}} object whose properties, less its mutable Timestamp, exceed that, so every object it forwards fits.
A receiver MUST treat a block its encoding's drafts would refuse as malformed.
The Encoding names the drafts the object came from, so a receiver MUST also treat a block as malformed on an object none of them could send with properties.
Only draft 14 lets a status object carry them, so a block on the END_OF_GROUP record MUST have Encoding 1, and a datagram with a block MUST have fields an OBJECT_DATAGRAM Type of those drafts carries ([Datagrams](#datagrams)).
A receiver MUST treat a block the track's Version would refuse as malformed, as it MUST a block on an object that Version could not send with properties.
Only draft 14 lets a status object carry them, so a block on the END_OF_GROUP record needs a Version of draft 14, and a datagram with a block MUST have fields an OBJECT_DATAGRAM Type of its Version carries ([Datagrams](#datagrams)).
The Version also gives each property its meaning: drafts 18 and 19 serialize a block alike, but only drafts 19 and later assign property 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so a relay knows not to forward a 0x06 to a draft that would read it differently.
Other properties are opaque to this extension.

## Timestamps {#timestamps}
Expand All @@ -175,27 +158,29 @@ When the extension is negotiated, the publisher sends a TRACK_PROPERTIES message
~~~
TRACK_PROPERTIES Message {
Message Length (i)
Encoding (i)
Version (i)
Property List (..)
Comment on lines 159 to 162

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Record the wire change in the draft changelog

This replaces the TRACK_PROPERTIES Encoding field with Version and removes each object's Encoding field, but the draft changelog still contains only “Initial version.” Add a bullet under the in-progress version so revision readers can identify this wire-incompatible semantic change.

AGENTS.md reference: drafts/AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining: draft-lcurley-moq-ietf has never been published (no -00 on the datatracker), so the in-progress -00 is the initial version and "Initial version." already covers it. A changelog bullet would describe a change from a revision no reader has seen.

(Written by Claude Opus 5.5)

}
~~~

**Encoding**:
The drafts the Property List came from, with the codes of [Properties](#properties).
A message without properties MUST have an Encoding of 0 and an empty Property List.
Track Properties start at draft 16, so any other message with an Encoding of 0 or 1 is malformed.
**Version**:
The track's Version: the {{moqt}} version its original publisher spoke, as {{moqt}} numbers it, such as 0xff000012 for draft 18.
It is 0 for a track published over moq-lite, which carries no {{moqt}} state, and then the Property List MUST be empty.
A receiver MUST treat a Version that names no {{moqt}} version it implements as malformed.
A track has one Version, since each publisher is its own track: a relay forwards it unchanged, and a track from another publisher is another track.

**Property List**:
The {{moqt}} Track Properties a relay forwards without interpreting, serialized as the publisher's draft serializes them: in ascending ID order, with delta-encoded IDs and the varints of that encoding.
A relay keeps the publisher's encoding, so a repeated ID still costs a one-byte delta and the list never grows on a hop.
The {{moqt}} Track Properties a relay forwards without interpreting, serialized as the Version serializes them: in ascending ID order, with delta-encoded IDs and the varints of that version.
Track Properties start at draft 16, so the list MUST be empty for drafts 14 and 15.
A relay keeps the Version's encoding, so a repeated ID still costs a one-byte delta and the list never grows on a hop.
Immutable Properties (0x0b), at most one, are forwarded as the publisher's bytes, in the same encoding.

A message MUST NOT carry more than 64 properties, nor any a relay interprets rather than forwards: MAX_CACHE_DURATION (0x04), TIMESCALE (0x08), DEFAULT_PUBLISHER_PRIORITY (0x0e), DEFAULT_PUBLISHER_GROUP_ORDER (0x22), and DYNAMIC_GROUPS (0x30).
TRACK_INFO carries what a relay keeps of them, so a second copy could only contradict it.
A message that breaks either rule is malformed.

A relay restates up to five of those from TRACK_INFO, so it forwards at most 59 properties, and it refuses a track with more, from a {{moqt}} publisher or over this extension, as unsupported.
Dropping those five can widen the delta that spans them, so a relay also refuses a {{moqt}} track whose remaining list would not fit a TRACK_PROPERTIES message, as unsupported.
Dropping those five can widen the delta that spans them, so a relay also refuses a {{moqt}} track whose remaining list would not fit a TRACK_PROPERTIES message beside its Version, as unsupported.

The Immutable Properties may still hold one of those five, which a relay forwards in place of the one it would restate.
A relay acts on TRACK_INFO, so a sealed value MUST equal the one it restates: MAX_CACHE_DURATION the Max Age in milliseconds, TIMESCALE the Timescale, DEFAULT_PUBLISHER_PRIORITY the priority, and DYNAMIC_GROUPS whether its Refresh is nonzero.
Expand Down Expand Up @@ -238,14 +223,14 @@ A bit field, or 0 for a plain datagram, which has none of the optional fields be
- `0x8` **END_OF_GROUP**: The object ends its group.
- `0x10` **STATUS**: The {{moqt}} Object Status follows.

A receiver MUST drop a datagram with any other bit, or a bit without RELAYED.
A receiver MUST drop a datagram with any other bit, a bit without RELAYED, or any bit on a track without a Version.

A datagram with a Status MUST have an empty Payload, as {{moqt}} sends no payload beside one; a Normal (0) Status is an empty object.
It has no Inflated Size ({{moql}} DATAGRAM), even on a compressed track, as there is no payload to inflate.
A non-zero Status carries no object: it has no Timestamp, even on a timed track.
A receiver MUST drop a datagram whose Status no {{moqt}} draft defines: only 0 (Normal), 1 (DOES_NOT_EXIST, drafts 14 and 15), 3 (END_OF_GROUP), and 4 (END_OF_TRACK) are relayed.
It MUST also drop one whose fields no {{moqt}} OBJECT_DATAGRAM Type can carry, such as STATUS with END_OF_GROUP.
With Properties, only the drafts of their Encoding count: from draft 17 on a non-zero Status carries no properties, so one needs Encoding 1 or 2, and DOES_NOT_EXIST needs Encoding 1.
It MUST also drop one its track's Version could not send: whose fields no OBJECT_DATAGRAM Type of that Version carries, such as STATUS with END_OF_GROUP, or whose Status it does not define.
From draft 17 on a non-zero Status carries no properties, and only drafts 14 and 15 define DOES_NOT_EXIST.


# Security Considerations
Expand Down
5 changes: 2 additions & 3 deletions quest/m1/ietf-extension/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ cannot answer unambiguously is refused.
property. A private request path carries NEW_GROUP_REQUEST upstream and
coalesces redundant requests until the largest group advances. Plain
moq-lite does not advertise this capability.
- **Immutable properties** keep their serialized inner bytes and ingress
encoding family, including across relay hops. Refuse forwarding between
- **Immutable properties** keep their serialized inner bytes, and the track
its ingress moq-transport version, including across relay hops. Refuse forwarding between
the absolute-key/QUIC-varint drafts 14/15, delta-key/QUIC-varint draft 16,
and delta-key/leading-ones-varint drafts 17+ when immutable properties
are present. Draft 17 is a separate family because it forbids 7-byte varints.
Expand Down Expand Up @@ -83,7 +83,6 @@ Work owned by this README, after the children land:

## Required

- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - a track crossing a lite-07 hop carries its moq-transport ingress version once, replacing per-block Encoding codes
- [Timed tracks cross lite-07](/quest/m1/ietf-extension/serve-timed.md) - moq-transport timestamps reach `ietf` extension peers over lite-07, with `unchecked` derived from `Info::ietf()`

## Related
Expand Down
56 changes: 0 additions & 56 deletions quest/m1/ietf-extension/lite.md

This file was deleted.

3 changes: 1 addition & 2 deletions quest/m1/ietf-extension/serve-timed.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,4 @@ encoding on that hop needs a wire change, update

## Required

- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - adds `unchecked` and the untimed serving this changes
- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - adds `Info::ietf()`, which `unchecked` derives from
- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - adds `unchecked` and the untimed serving this changes
1 change: 0 additions & 1 deletion quest/m1/lite07-finalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@ is published; older versions are unchanged.
- [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route
- [Compressed tracks in js/net](/quest/m1/compress/js.md) - JS reads and writes compressed lite-07 tracks instead of refusing them
- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - JS speaks lite-07's Tracks Stream, which Rust already does
- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - the `ietf` extension carries a track's ingress version once instead of a code per block

## Related

Expand Down
63 changes: 41 additions & 22 deletions rs/moq-net/src/fuzz.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,17 @@ pub type Target = fn(&[u8]) -> bool;
/// Build a cached group with optional private object fields for model benchmarks.
pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, track::Producer, group::Producer) {
let broadcast = broadcast::Producer::new(broadcast::Info::default());
let track = broadcast.create_track("objects", None).unwrap();
// Object properties come from a moq-transport publisher, whose draft serialized them.
let info = track::Info {
properties: properties.then(|| {
std::sync::Arc::new(crate::model::object::TrackProperties {
version: ietf::Version::Draft22,
values: Vec::new(),
})
}),
..Default::default()
};
let track = broadcast.create_track("objects", info).unwrap();
let mut group = track.append_group().unwrap();
let payload = Bytes::from_static(&[0; 64]);
for id in 0..frames {
Expand All @@ -40,7 +50,7 @@ pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, tr
id: 0x80,
value: crate::model::object::Value::Int(96_000),
}],
crate::model::object::Encoding::Draft19,
ietf::Version::Draft22,
)
.unwrap(),
..Default::default()
Expand Down Expand Up @@ -151,30 +161,38 @@ where

/// [`roundtrip`] for a datagram body, which runs to the end of its buffer.
fn datagram(data: &[u8], version: lite::Version) -> bool {
// The body doesn't say whether it carries a timestamp or an inflated size, or whether
// the session negotiated the `ietf` extension, so try every reading.
let ietf: &[bool] = match version.has_ietf() {
// The body doesn't say whether it carries a timestamp or an inflated size, whether the
// session negotiated the `ietf` extension, or which draft its track came from, so try
// every reading.
let negotiated: &[bool] = match version.has_ietf() {
true => &[false, true],
false => &[false],
};
let mut any = false;
for &ietf in ietf {
for &negotiated in negotiated {
for timescale in [Some(crate::Timescale::MILLI), None] {
for compressed in [false, true] {
let format = lite::Format { timescale, compressed };
let Ok(decoded) = lite::Datagram::decode(Bytes::copy_from_slice(data), version, ietf, |_| format)
else {
continue;
};
any = true;

let Ok(first) = decoded.encode_bytes(version) else {
continue;
};

let echo = lite::Datagram::decode(first, version, ietf, |_| format)
.expect("could not decode our own encoding");
assert_eq!(echo, decoded, "datagram did not survive a round trip");
for ietf in [None, Some(ietf::Version::Draft14), Some(ietf::Version::Draft22)] {
for compressed in [false, true] {
let framing = lite::Framing {
timescale,
ietf,
compressed,
};
let Ok(decoded) =
lite::Datagram::decode(Bytes::copy_from_slice(data), version, negotiated, |_| framing)
else {
continue;
};
any = true;

let Ok(first) = decoded.encode_bytes(version) else {
continue;
};

let echo = lite::Datagram::decode(first, version, negotiated, |_| framing)
.expect("could not decode our own encoding");
assert_eq!(echo, decoded, "datagram did not survive a round trip");
}
}
}
}
Expand Down Expand Up @@ -484,8 +502,9 @@ impl LiteSample {
Self::Group => lite::Group::decode(r, version).map(|_| 1).unwrap(),
Self::Subscribe => lite::Subscribe::decode(r, version).map(|_| 1).unwrap(),
Self::Datagram => {
lite::Datagram::decode(bytes::Bytes::copy_from_slice(data), version, false, |_| lite::Format {
lite::Datagram::decode(bytes::Bytes::copy_from_slice(data), version, false, |_| lite::Framing {
timescale: Some(crate::Timescale::MILLI),
ietf: None,
compressed: false,
})
.map(|_| 1)
Expand Down
Loading
Loading