diff --git a/drafts/draft-lcurley-moq-ietf.md b/drafts/draft-lcurley-moq-ietf.md index a1a9a7401f5..ac7b0680e59 100644 --- a/drafts/draft-lcurley-moq-ietf.md +++ b/drafts/draft-lcurley-moq-ietf.md @@ -39,6 +39,7 @@ Submit an [issue](https://github.com/moq-dev/moq/issues) or [PR](https://github. A **relayed** group or datagram arrived from a MoQ Transport publisher, so it has object state that moq-lite cannot express. A **plain** one has none: its Object IDs are its frame indices, it has no properties, and its group ends where its last frame does. +A track's **Version** is the {{moqt}} version its original publisher spoke, which serialized every property it carries, or none for a track published over moq-lite ([Track Stream](#track)). # Introduction @@ -63,6 +64,7 @@ The value MUST be empty; a receiver MUST close the session with a PROTOCOL_VIOLA The extension is negotiated only when both endpoints offer it, and only on moq-lite-07 and later. An endpoint MUST NOT send any field this document defines unless the extension is negotiated, and MUST NOT parse a Group Stream, FETCH response, Track Stream, or datagram from the peer until it has the peer's SETUP and so knows whether it was. An endpoint MAY drop a datagram that arrives before then. +A Group Stream, FETCH response, or datagram is read with the Version of its track, which the Track Stream carries before any subscription or fetch opens. # Group Streams {#group} @@ -91,6 +93,7 @@ A bit field describing the group's relay state, or 0 for a plain group: - `0x8` **END_OF_GROUP**: The stream's end ends the group, as a {{moqt}} subgroup that contains the end of its group. A receiver MUST reset the stream with a PROTOCOL_VIOLATION on any other bit, a bit without RELAYED, or a record with Properties when EXTENSIONS is clear. +A track without a Version has no {{moqt}} state, so a receiver MUST reset the stream with a PROTOCOL_VIOLATION on any Group State but 0 for one. A publisher MUST clear FIRST_OBJECT when the stream starts after the first frame it holds, and END_OF_GROUP when the stream may stop before the end of the group, such as at a subscription's end frame. A stream that stops at an end frame carries no END_OF_GROUP record, even when that frame is the group's last. @@ -114,7 +117,7 @@ Object Record { **Properties**: The object's [properties](#properties), in `Properties Length` bytes. An empty one means the object has none. -A receiver MUST reset the stream if the length exceeds 65,537, the largest Property Block beside an Encoding byte. +A receiver MUST reset the stream if the length exceeds 65,536, the largest Property Block. In a plain group, every record has a `Skip` of 0 after the first, whose Object ID is the index of its frame, no properties, and `End` of 0. A receiver MUST reset the stream with a PROTOCOL_VIOLATION on any other record in a plain group. @@ -124,39 +127,19 @@ The Object ID is relay state beside it, so a plain subscriber still reads consec # Properties {#properties} -An object's properties, as one block in the encoding of the {{moqt}} draft it arrived on. +An object's properties, as one Property Block in the encoding of its track's Version. A relay forwards the block to a peer of that encoding unchanged, so the block never grows on a hop, and holds it as received rather than as a decoded list many times its size. -~~~ -Properties { - Encoding (i) - Property Block (..) -} -~~~ - -**Encoding**: -The {{moqt}} drafts the Property Block came from, which serialize it alike and give each property the same meaning: - -| Code | Drafts | IDs | Varints | -|-----:|:-------|:----|:--------| -| 1 | 14 and 15 | absolute | QUIC | -| 2 | 16 | delta | QUIC | -| 3 | 17 | delta | draft 17 | -| 4 | 18 | delta | draft 18 | -| 5 | 19 and later | delta | draft 18 | - -A receiver MUST treat any other code as malformed, including 0: an empty Properties says the object has none. -Codes 4 and 5 serialize alike, but only drafts 19 and later assign property 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so a relay knows not to forward 0x06 to a draft that would read it differently. - **Property Block**: -Every property of the object, serialized as the {{moqt}} drafts of its encoding serialize an object's properties, in ascending ID order. +Every property of the object, serialized as the track's Version serializes an object's properties, in ascending ID order. Immutable Properties (0x0b) are in the same encoding, as the publisher sent them. It MUST NOT be empty or longer than 65,536 bytes. +A track without a Version carries no properties: a Property Block on one is malformed. On a timed track an object with a FRAME, or a datagram without a Status, MUST NOT carry one longer than 65,526 bytes, leaving room for the largest Timestamp a relay restates ([Timestamps](#timestamps)): a one-byte key and a value of up to nine bytes. A relay MUST refuse a {{moqt}} object whose properties, less its mutable Timestamp, exceed that, so every object it forwards fits. -A receiver MUST treat a block its encoding's drafts would refuse as malformed. -The Encoding names the drafts the object came from, so a receiver MUST also treat a block as malformed on an object none of them could send with properties. -Only draft 14 lets a status object carry them, so a block on the END_OF_GROUP record MUST have Encoding 1, and a datagram with a block MUST have fields an OBJECT_DATAGRAM Type of those drafts carries ([Datagrams](#datagrams)). +A receiver MUST treat a block the track's Version would refuse as malformed, as it MUST a block on an object that Version could not send with properties. +Only draft 14 lets a status object carry them, so a block on the END_OF_GROUP record needs a Version of draft 14, and a datagram with a block MUST have fields an OBJECT_DATAGRAM Type of its Version carries ([Datagrams](#datagrams)). +The Version also gives each property its meaning: drafts 18 and 19 serialize a block alike, but only drafts 19 and later assign property 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so a relay knows not to forward a 0x06 to a draft that would read it differently. Other properties are opaque to this extension. ## Timestamps {#timestamps} @@ -175,19 +158,21 @@ When the extension is negotiated, the publisher sends a TRACK_PROPERTIES message ~~~ TRACK_PROPERTIES Message { Message Length (i) - Encoding (i) + Version (i) Property List (..) } ~~~ -**Encoding**: -The drafts the Property List came from, with the codes of [Properties](#properties). -A message without properties MUST have an Encoding of 0 and an empty Property List. -Track Properties start at draft 16, so any other message with an Encoding of 0 or 1 is malformed. +**Version**: +The track's Version: the {{moqt}} version its original publisher spoke, as {{moqt}} numbers it, such as 0xff000012 for draft 18. +It is 0 for a track published over moq-lite, which carries no {{moqt}} state, and then the Property List MUST be empty. +A receiver MUST treat a Version that names no {{moqt}} version it implements as malformed. +A track has one Version, since each publisher is its own track: a relay forwards it unchanged, and a track from another publisher is another track. **Property List**: -The {{moqt}} Track Properties a relay forwards without interpreting, serialized as the publisher's draft serializes them: in ascending ID order, with delta-encoded IDs and the varints of that encoding. -A relay keeps the publisher's encoding, so a repeated ID still costs a one-byte delta and the list never grows on a hop. +The {{moqt}} Track Properties a relay forwards without interpreting, serialized as the Version serializes them: in ascending ID order, with delta-encoded IDs and the varints of that version. +Track Properties start at draft 16, so the list MUST be empty for drafts 14 and 15. +A relay keeps the Version's encoding, so a repeated ID still costs a one-byte delta and the list never grows on a hop. Immutable Properties (0x0b), at most one, are forwarded as the publisher's bytes, in the same encoding. A message MUST NOT carry more than 64 properties, nor any a relay interprets rather than forwards: MAX_CACHE_DURATION (0x04), TIMESCALE (0x08), DEFAULT_PUBLISHER_PRIORITY (0x0e), DEFAULT_PUBLISHER_GROUP_ORDER (0x22), and DYNAMIC_GROUPS (0x30). @@ -195,7 +180,7 @@ TRACK_INFO carries what a relay keeps of them, so a second copy could only contr A message that breaks either rule is malformed. A relay restates up to five of those from TRACK_INFO, so it forwards at most 59 properties, and it refuses a track with more, from a {{moqt}} publisher or over this extension, as unsupported. -Dropping those five can widen the delta that spans them, so a relay also refuses a {{moqt}} track whose remaining list would not fit a TRACK_PROPERTIES message, as unsupported. +Dropping those five can widen the delta that spans them, so a relay also refuses a {{moqt}} track whose remaining list would not fit a TRACK_PROPERTIES message beside its Version, as unsupported. The Immutable Properties may still hold one of those five, which a relay forwards in place of the one it would restate. A relay acts on TRACK_INFO, so a sealed value MUST equal the one it restates: MAX_CACHE_DURATION the Max Age in milliseconds, TIMESCALE the Timescale, DEFAULT_PUBLISHER_PRIORITY the priority, and DYNAMIC_GROUPS whether its Refresh is nonzero. @@ -238,14 +223,14 @@ A bit field, or 0 for a plain datagram, which has none of the optional fields be - `0x8` **END_OF_GROUP**: The object ends its group. - `0x10` **STATUS**: The {{moqt}} Object Status follows. -A receiver MUST drop a datagram with any other bit, or a bit without RELAYED. +A receiver MUST drop a datagram with any other bit, a bit without RELAYED, or any bit on a track without a Version. A datagram with a Status MUST have an empty Payload, as {{moqt}} sends no payload beside one; a Normal (0) Status is an empty object. It has no Inflated Size ({{moql}} DATAGRAM), even on a compressed track, as there is no payload to inflate. A non-zero Status carries no object: it has no Timestamp, even on a timed track. A receiver MUST drop a datagram whose Status no {{moqt}} draft defines: only 0 (Normal), 1 (DOES_NOT_EXIST, drafts 14 and 15), 3 (END_OF_GROUP), and 4 (END_OF_TRACK) are relayed. -It MUST also drop one whose fields no {{moqt}} OBJECT_DATAGRAM Type can carry, such as STATUS with END_OF_GROUP. -With Properties, only the drafts of their Encoding count: from draft 17 on a non-zero Status carries no properties, so one needs Encoding 1 or 2, and DOES_NOT_EXIST needs Encoding 1. +It MUST also drop one its track's Version could not send: whose fields no OBJECT_DATAGRAM Type of that Version carries, such as STATUS with END_OF_GROUP, or whose Status it does not define. +From draft 17 on a non-zero Status carries no properties, and only drafts 14 and 15 define DOES_NOT_EXIST. # Security Considerations diff --git a/quest/m1/ietf-extension/README.md b/quest/m1/ietf-extension/README.md index cf8b1f2971f..d438815430e 100644 --- a/quest/m1/ietf-extension/README.md +++ b/quest/m1/ietf-extension/README.md @@ -51,8 +51,8 @@ cannot answer unambiguously is refused. property. A private request path carries NEW_GROUP_REQUEST upstream and coalesces redundant requests until the largest group advances. Plain moq-lite does not advertise this capability. -- **Immutable properties** keep their serialized inner bytes and ingress - encoding family, including across relay hops. Refuse forwarding between +- **Immutable properties** keep their serialized inner bytes, and the track + its ingress moq-transport version, including across relay hops. Refuse forwarding between the absolute-key/QUIC-varint drafts 14/15, delta-key/QUIC-varint draft 16, and delta-key/leading-ones-varint drafts 17+ when immutable properties are present. Draft 17 is a separate family because it forbids 7-byte varints. @@ -83,7 +83,6 @@ Work owned by this README, after the children land: ## Required -- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - a track crossing a lite-07 hop carries its moq-transport ingress version once, replacing per-block Encoding codes - [Timed tracks cross lite-07](/quest/m1/ietf-extension/serve-timed.md) - moq-transport timestamps reach `ietf` extension peers over lite-07, with `unchecked` derived from `Info::ietf()` ## Related diff --git a/quest/m1/ietf-extension/lite.md b/quest/m1/ietf-extension/lite.md deleted file mode 100644 index b70fe90fcb9..00000000000 --- a/quest/m1/ietf-extension/lite.md +++ /dev/null @@ -1,56 +0,0 @@ -# [M] The ietf extension records provenance as the ingress version - -## Goal - -A track crossing a lite-07 hop with the `ietf` SETUP extension carries its -ingress moq-transport version once, instead of an Encoding code on every -Property Block and on TRACK_PROPERTIES. A track published over moq-lite -carries none, and a peer cannot attach moq-transport state to it. Behavior -across the hop is otherwise unchanged from -[moq-dev/moq#6058](https://github.com/moq-dev/moq/pull/6058), which landed -the extension with per-block codes. - -## Plan - -Decided 2026-10-20 in -[moq-dev/moq#6072](https://github.com/moq-dev/moq/pull/6072). - -- In the model, the track's optional ingress `ietf::Version` replaces - `object::Encoding` on `object::Block` and `TrackProperties`, which already - derives 0x06's meaning from the draft instead of a flag. One - version per track suffices because each publisher is its own stream; a - different ingress is a new instance. -- On the wire, track info carries the ingress version once, or none for a - track published over moq-lite, which has no IETF provenance and no - properties to derive rules for. Records and datagrams drop their Encoding - code. Update `drafts/draft-lcurley-moq-ietf.md` and pass - `just drafts check`. -- Lite ingress refuses a track with no version that carries any IETF field - (track, object, or datagram properties, or object IDs), including on a - later object. -- Immutable properties keep their unchanged inner bytes. Downstream IETF - egress applies today's single-relay rules from that version: refuse an - incompatible immutable encoding (drafts 14/15 carry no immutable track - block, so egress there skips that check), and across draft 19, where 0x06 - becomes SUBGROUP_DELIVERY_TIMEOUT, drop a mutable 0x06 and refuse a sealed - one. -- Rejected 2026-10-20: a generic crate-private `unknown` KVP block on route, - track, group, frame, and datagram, tagged by source version, that would - also tunnel future lite fields through a lite-07 hop. A relay cannot act on - fields it forwards blindly, and most lite additions (routes, refresh, - ranges, auth) change relay behavior. With no must-understand marker, only - relay-ignorable end-to-end hints could tunnel safely. - -Tests: 0x06 provenance across the hop: a draft-18 publisher's object and -track properties exit draft 19 with a mutable 0x06 dropped and a sealed one -refused. Both drafts share an encoding family, so only the carried version -distinguishes them. Cover a lite-published track crossing the same hop with -no ingress version, and a malformed peer that sends properties on a -versionless track, at track info and on a later frame and datagram. Run -`just test interop --all`. - -Wire: lite-07-wip only, when the extension is negotiated. - -## Related - -- [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - this changes the lite-07 wire, so it lands before the cut diff --git a/quest/m1/ietf-extension/serve-timed.md b/quest/m1/ietf-extension/serve-timed.md index a131316a5ff..c4669fc90b9 100644 --- a/quest/m1/ietf-extension/serve-timed.md +++ b/quest/m1/ietf-extension/serve-timed.md @@ -27,5 +27,4 @@ encoding on that hop needs a wire change, update ## Required -- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - adds `unchecked` and the untimed serving this changes -- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - adds `Info::ietf()`, which `unchecked` derives from +- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - adds `unchecked` and the untimed serving this changes \ No newline at end of file diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 764dc7066fa..e7664d4ae6e 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -89,7 +89,6 @@ is published; older versions are unchanged. - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route - [Compressed tracks in js/net](/quest/m1/compress/js.md) - JS reads and writes compressed lite-07 tracks instead of refusing them - [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - JS speaks lite-07's Tracks Stream, which Rust already does -- [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - the `ietf` extension carries a track's ingress version once instead of a code per block ## Related diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index a1305df7a61..1f96483fc23 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -27,7 +27,17 @@ pub type Target = fn(&[u8]) -> bool; /// Build a cached group with optional private object fields for model benchmarks. pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, track::Producer, group::Producer) { let broadcast = broadcast::Producer::new(broadcast::Info::default()); - let track = broadcast.create_track("objects", None).unwrap(); + // Object properties come from a moq-transport publisher, whose draft serialized them. + let info = track::Info { + properties: properties.then(|| { + std::sync::Arc::new(crate::model::object::TrackProperties { + version: ietf::Version::Draft22, + values: Vec::new(), + }) + }), + ..Default::default() + }; + let track = broadcast.create_track("objects", info).unwrap(); let mut group = track.append_group().unwrap(); let payload = Bytes::from_static(&[0; 64]); for id in 0..frames { @@ -40,7 +50,7 @@ pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, tr id: 0x80, value: crate::model::object::Value::Int(96_000), }], - crate::model::object::Encoding::Draft19, + ietf::Version::Draft22, ) .unwrap(), ..Default::default() @@ -151,30 +161,38 @@ where /// [`roundtrip`] for a datagram body, which runs to the end of its buffer. fn datagram(data: &[u8], version: lite::Version) -> bool { - // The body doesn't say whether it carries a timestamp or an inflated size, or whether - // the session negotiated the `ietf` extension, so try every reading. - let ietf: &[bool] = match version.has_ietf() { + // The body doesn't say whether it carries a timestamp or an inflated size, whether the + // session negotiated the `ietf` extension, or which draft its track came from, so try + // every reading. + let negotiated: &[bool] = match version.has_ietf() { true => &[false, true], false => &[false], }; let mut any = false; - for &ietf in ietf { + for &negotiated in negotiated { for timescale in [Some(crate::Timescale::MILLI), None] { - for compressed in [false, true] { - let format = lite::Format { timescale, compressed }; - let Ok(decoded) = lite::Datagram::decode(Bytes::copy_from_slice(data), version, ietf, |_| format) - else { - continue; - }; - any = true; - - let Ok(first) = decoded.encode_bytes(version) else { - continue; - }; - - let echo = lite::Datagram::decode(first, version, ietf, |_| format) - .expect("could not decode our own encoding"); - assert_eq!(echo, decoded, "datagram did not survive a round trip"); + for ietf in [None, Some(ietf::Version::Draft14), Some(ietf::Version::Draft22)] { + for compressed in [false, true] { + let framing = lite::Framing { + timescale, + ietf, + compressed, + }; + let Ok(decoded) = + lite::Datagram::decode(Bytes::copy_from_slice(data), version, negotiated, |_| framing) + else { + continue; + }; + any = true; + + let Ok(first) = decoded.encode_bytes(version) else { + continue; + }; + + let echo = lite::Datagram::decode(first, version, negotiated, |_| framing) + .expect("could not decode our own encoding"); + assert_eq!(echo, decoded, "datagram did not survive a round trip"); + } } } } @@ -484,8 +502,9 @@ impl LiteSample { Self::Group => lite::Group::decode(r, version).map(|_| 1).unwrap(), Self::Subscribe => lite::Subscribe::decode(r, version).map(|_| 1).unwrap(), Self::Datagram => { - lite::Datagram::decode(bytes::Bytes::copy_from_slice(data), version, false, |_| lite::Format { + lite::Datagram::decode(bytes::Bytes::copy_from_slice(data), version, false, |_| lite::Framing { timescale: Some(crate::Timescale::MILLI), + ietf: None, compressed: false, }) .map(|_| 1) diff --git a/rs/moq-net/src/ietf/datagram.rs b/rs/moq-net/src/ietf/datagram.rs index 1383a620044..6d7f18bbae9 100644 --- a/rs/moq-net/src/ietf/datagram.rs +++ b/rs/moq-net/src/ietf/datagram.rs @@ -80,10 +80,10 @@ impl ObjectDatagram { } impl crate::model::datagram::Wire { - /// Whether a moq-transport draft defines a datagram of this shape, as its ingress checks - /// one, and a draft of its block's encoding when it has one: that draft sent it. Egress - /// resolves the Object ID and priority each draft needs. - pub(crate) fn defined(&self) -> bool { + /// Whether `version`, the draft its track's publisher spoke, defines a datagram of this + /// shape, as its ingress checks one. Egress resolves the Object ID and priority each + /// draft needs. + pub(crate) fn defined(&self, version: Version) -> bool { let shape = ObjectDatagram { track_alias: 0, group_id: 0, @@ -96,17 +96,10 @@ impl crate::model::datagram::Wire { None => DatagramBody::Payload(Bytes::new()), }, }; - let kind = shape.kind(); - let family = self.properties.block.as_ref().map(|block| block.encoding); - Version::ALL - .into_iter() - .filter(|version| family.is_none_or(|family| super::object::encoding(*version) == family)) - .any(|version| { - ObjectDatagram::valid(kind, version) - && self.status.is_none_or(|status| { - super::object::status_defined(status, version) - && (family.is_none() || carries_properties(status, version)) - }) + ObjectDatagram::valid(shape.kind(), version) + && self.status.is_none_or(|status| { + super::object::status_defined(status, version) + && (self.properties.block.is_none() || carries_properties(status, version)) }) } } diff --git a/rs/moq-net/src/ietf/object.rs b/rs/moq-net/src/ietf/object.rs index e95fab140fa..ed38d40734f 100644 --- a/rs/moq-net/src/ietf/object.rs +++ b/rs/moq-net/src/ietf/object.rs @@ -4,38 +4,20 @@ use super::Version; use crate::{ Error, Timescale, Timestamp, coding::{Decoder, Encoder, Form}, - model::object::{Encoding, Properties, Property, Value}, + model::object::{Properties, Property, Value}, }; -pub(crate) fn encoding(version: Version) -> Encoding { - match version { - Version::Draft14 | Version::Draft15 => Encoding::Draft14, - Version::Draft16 => Encoding::Draft16, - Version::Draft17 => Encoding::Draft17, - Version::Draft18 => Encoding::Draft18, - _ => Encoding::Draft19, - } +/// Whether `version` serializes property IDs as they are, rather than as deltas. +fn absolute(version: Version) -> bool { + matches!(version, Version::Draft14 | Version::Draft15) } -/// Whether lists in encodings `a` and `b` are the same bytes: the same keys and varints. -pub(crate) fn same_bytes(a: Encoding, b: Encoding) -> bool { - let form = |encoding| (encoding == Encoding::Draft14, Form::from(family(encoding))); +/// Whether lists from drafts `a` and `b` are the same bytes: the same keys and varints. +pub(crate) fn same_bytes(a: Version, b: Version) -> bool { + let form = |version| (absolute(version), Form::from(version)); form(a) == form(b) } -/// Whether property 0x06 means SUBGROUP_DELIVERY_TIMEOUT in a list from `encoding`. -pub(crate) fn delivery_timeout(encoding: Encoding) -> bool { - has_delivery_timeout(family(encoding)) -} - -/// The drafts that serialize properties in the `family` encoding, one of which sent any -/// block claiming it. -pub(crate) fn drafts(family: Encoding) -> impl Iterator { - Version::ALL - .into_iter() - .filter(move |version| encoding(*version) == family) -} - /// Draft-19 assigns property 0x06 to SUBGROUP_DELIVERY_TIMEOUT. pub(crate) fn has_delivery_timeout(version: Version) -> bool { !matches!( @@ -129,7 +111,6 @@ struct List { /// Parse a property list. `lift` takes the mutable Timestamp out of `values`, as a timed /// track's frame holds it. fn list(data: &Bytes, version: Version, immutable: bool, lift: bool) -> Result { - let keys = encoding(version); let mut decoder = Decoder::new(data, version.into()); let mut result = List { values: Vec::new(), @@ -142,9 +123,9 @@ fn list(data: &Bytes, version: Version, immutable: bool, lift: bool) -> Result key, - _ => previous.checked_add(key).ok_or(Error::MalformedTrack)?, + let id = match absolute(version) { + true => key, + false => previous.checked_add(key).ok_or(Error::MalformedTrack)?, }; previous = id; let value = if id % 2 == 0 { @@ -188,17 +169,17 @@ fn list(data: &Bytes, version: Version, immutable: bool, lift: bool) -> Result(values: impl IntoIterator, encoding: Encoding) -> Result, Error> { +/// Serialize `values` as `version` does, in ascending ID order. Equal IDs keep their order. +fn serialize<'a>(values: impl IntoIterator, version: Version) -> Result, Error> { let mut sorted: Vec<_> = values.into_iter().collect(); sorted.sort_by_key(|property| property.id); let mut out = Vec::new(); - let mut encoder = Encoder::new(&mut out, family(encoding).into()); + let mut encoder = Encoder::new(&mut out, version.into()); let mut previous = 0; for property in sorted { - encoder.varint(match encoding { - Encoding::Draft14 => property.id, - _ => property.id - previous, + encoder.varint(match absolute(version) { + true => property.id, + false => property.id - previous, })?; previous = property.id; match &property.value { @@ -209,18 +190,17 @@ fn serialize<'a>(values: impl IntoIterator, encoding: Encod Ok(out) } -/// Encode `values` as the block of an object whose properties arrived as `encoding`, or +/// Encode `values` as the block of an object whose properties arrived on `version`, or /// `None` without any. /// /// Sorting a delta-encoded list, which arrives ascending, and dropping its Timestamp /// never grows it, so the block fits wherever the publisher's list did. -pub(crate) fn block(values: &[Property], encoding: Encoding) -> Result, Error> { +pub(crate) fn block(values: &[Property], version: Version) -> Result, Error> { if values.is_empty() { return Ok(None); } Ok(Some(crate::model::object::Block { - bytes: serialize(values, encoding)?.into(), - encoding, + bytes: serialize(values, version)?.into(), immutable: values.iter().any(|property| property.id == 0x0b), })) } @@ -232,8 +212,9 @@ pub(crate) struct Checked { pub gap: Option, } -/// Validate a block received as `encoding` on a track in `timescale`, as moq-transport -/// ingress validates a list, for an object that is `framed` or a frameless status. +/// Validate a block received on a track in `timescale` that a `version` publisher +/// published, as moq-transport ingress validates a list, for an object that is `framed` or +/// a frameless status. /// /// A framed object's mutable Timestamp on a timed track is its frame's, so one inside is /// malformed, as is an empty block, which is sent as none. Absolute IDs must ascend, as @@ -241,7 +222,7 @@ pub(crate) struct Checked { /// as the publisher sent them. pub(crate) fn check( bytes: &Bytes, - encoding: Encoding, + version: Version, timescale: Option, framed: bool, ) -> Result { @@ -249,7 +230,7 @@ pub(crate) fn check( if bytes.is_empty() || bytes.len() > max_block(lifted) { return Err(Error::MalformedTrack); } - let parsed = list(bytes, family(encoding), false, lifted.is_some())?; + let parsed = list(bytes, version, false, lifted.is_some())?; if lifted.is_some() && parsed.timestamp.is_some() { return Err(Error::MalformedTrack); } @@ -271,18 +252,18 @@ fn timed(value: Option, timescale: Option) -> Result Vec { +pub(crate) fn listed(properties: &Properties, version: Version) -> Vec { properties .block .as_ref() - .map_or_else(Vec::new, |block| values(block).unwrap()) + .map_or_else(Vec::new, |block| values(block, version).unwrap()) } -/// Decode a block's properties, whose values borrow its bytes. -pub(crate) fn values(block: &crate::model::object::Block) -> Result, Error> { - Ok(list(&block.bytes, family(block.encoding), false, false)?.values) +/// Decode the properties of a block from a `version` publisher, whose values borrow its bytes. +fn values(block: &crate::model::object::Block, version: Version) -> Result, Error> { + Ok(list(&block.bytes, version, false, false)?.values) } /// Read an object's property list on a track in `timescale`. A timed track's frame holds @@ -306,7 +287,7 @@ fn decode(data: &Bytes, version: Version, timescale: Option, framed: let parsed = list(data, version, false, lifted.is_some())?; let properties = Properties { sealed_time: timed(parsed.sealed_time, timescale)?, - block: block(&parsed.values, encoding(version))?, + block: block(&parsed.values, version)?, }; if properties.heap() > MAX_BLOCK as u64 { return Err(Error::Unsupported); @@ -350,47 +331,39 @@ pub(crate) fn egress_time( .transpose() } -/// A draft that serializes properties in the `encoding` family. Any draft in the family -/// parses its keys and integers the same way. -pub(crate) fn family(encoding: Encoding) -> Version { - match encoding { - Encoding::Draft14 => Version::Draft14, - Encoding::Draft16 => Version::Draft16, - Encoding::Draft17 => Version::Draft17, - Encoding::Draft18 => Version::Draft18, - Encoding::Draft19 => Version::Draft19, - } -} - /// Serialize relayed properties for `version`, with the `frame`'s time as the mutable /// Timestamp in the `timescale` the egress declares, if any. +/// +/// `source` is the draft the track's publisher spoke, which serialized the block. A track +/// without one carries no properties. pub(crate) fn write( properties: &Properties, + source: Option, frame: Option, version: Version, timescale: Option, ) -> Result, Error> { - let target = encoding(version); let timestamp = egress_time(properties, frame, timescale)?; let timestamp = timestamp.map(|value| Property { id: 0x10, value: Value::Int(value), }); let Some(block) = &properties.block else { - return serialize(×tamp, target); + return serialize(×tamp, version); }; - if block.immutable && !same_bytes(block.encoding, target) { + let source = source.ok_or(Error::MalformedTrack)?; + if block.immutable && !same_bytes(source, version) { return Err(Error::Unsupported); } // Across draft-19, forwarding 0x06 would give it a different meaning. // A mutable one is dropped; a sealed one is refused. - let flipped = delivery_timeout(block.encoding) != has_delivery_timeout(version); + let flipped = has_delivery_timeout(source) != has_delivery_timeout(version); // The common case: the block is already what this draft would send. - if timestamp.is_none() && !flipped && same_bytes(block.encoding, target) { + if timestamp.is_none() && !flipped && same_bytes(source, version) { return Ok(block.bytes.to_vec()); } - let values = values(block)?; - if flipped && sealed_key(&values, version, 0x06)? { + let values = values(block, source)?; + if flipped && sealed_key(&values, source, 0x06)? { return Err(Error::Unsupported); } let out = serialize( @@ -398,7 +371,7 @@ pub(crate) fn write( .iter() .filter(|property| property.id != 0x06 || !flipped) .chain(×tamp), - target, + version, )?; // Ingress leaves room for the Timestamp, but another draft's varints can be wider, so a // block re-encoded for it may not fit: refused, as the receiver would refuse it. @@ -446,7 +419,14 @@ mod tests { assert_eq!(decoded.properties.sealed_time, None, "{version}"); for timescale in [Some(Timescale::MICRO), None] { assert_eq!( - write(&decoded.properties, decoded.timestamp, version, timescale).unwrap(), + write( + &decoded.properties, + Some(version), + decoded.timestamp, + version, + timescale + ) + .unwrap(), bytes, "{version}" ); @@ -487,7 +467,7 @@ mod tests { drop(received); assert!(dropped.load(Ordering::Relaxed)); assert_eq!( - listed(&decoded.properties)[0].value, + listed(&decoded.properties, Version::Draft22)[0].value, Value::Bytes(Bytes::from_static(&[42])) ); } @@ -500,24 +480,50 @@ mod tests { assert_eq!(decoded.properties.sealed_time, decoded.timestamp); let milli = Some(Timescale::MILLI); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft18, milli).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft18, + milli + ) + .unwrap(), bytes ); assert!(matches!( - write(&decoded.properties, decoded.timestamp, Version::Draft17, milli), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft17, + milli + ), Err(Error::Unsupported) )); // Its units cannot follow the egress's, or vanish with them, without rewriting the block. for timescale in [Some(Timescale::MICRO), None] { assert!(matches!( - write(&decoded.properties, decoded.timestamp, Version::Draft18, timescale), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft18, + timescale + ), Err(Error::Unsupported) )); } // An untimed ingress has no units to lose. let untimed = read(&bytes, Version::Draft22, None).unwrap(); assert_eq!( - write(&untimed.properties, untimed.timestamp, Version::Draft18, None).unwrap(), + write( + &untimed.properties, + Some(Version::Draft22), + untimed.timestamp, + Version::Draft18, + None + ) + .unwrap(), bytes ); } @@ -529,22 +535,29 @@ mod tests { for version in [Version::Draft18, Version::Draft22] { let bytes = Bytes::from_static(&[0x10, 42, 0x20, 7]); let decoded = read(&bytes, version, Some(Timescale::MILLI)).unwrap(); - let written = write(&decoded.properties, decoded.timestamp, version, Some(Timescale::MICRO)).unwrap(); + let written = write( + &decoded.properties, + Some(version), + decoded.timestamp, + version, + Some(Timescale::MICRO), + ) + .unwrap(); let again = read(&Bytes::from(written), version, Some(Timescale::MICRO)).unwrap(); let micros = Timestamp::new(42_000, Timescale::MICRO).unwrap(); assert_eq!(again.timestamp, Some(micros), "{version}"); - assert_eq!(listed(&again.properties).last().unwrap().value, Value::Int(7)); + assert_eq!(listed(&again.properties, version).last().unwrap().value, Value::Int(7)); // An egress that declares no units drops it, as a plain frame there has none. - let written = write(&decoded.properties, decoded.timestamp, version, None).unwrap(); + let written = write(&decoded.properties, Some(version), decoded.timestamp, version, None).unwrap(); let again = read(&Bytes::from(written), version, None).unwrap(); assert_eq!(again.timestamp, None, "{version}"); - assert_eq!(listed(&again.properties).len(), 1, "{version}"); - assert_eq!(listed(&again.properties)[0].value, Value::Int(7)); + assert_eq!(listed(&again.properties, version).len(), 1, "{version}"); + assert_eq!(listed(&again.properties, version)[0].value, Value::Int(7)); // An untimed ingress forwards it untouched. let untimed = read(&bytes, version, None).unwrap(); assert_eq!( - write(&untimed.properties, untimed.timestamp, version, None).unwrap(), + write(&untimed.properties, Some(version), untimed.timestamp, version, None).unwrap(), bytes ); } @@ -559,7 +572,7 @@ mod tests { let at = |millis| Some(Timestamp::from_millis(millis).unwrap()); // Kept properties that say nothing of time still leave with the frame's. - let written = write(&Properties::default(), at(500), Version::Draft22, milli).unwrap(); + let written = write(&Properties::default(), None, at(500), Version::Draft22, milli).unwrap(); let again = read(&Bytes::from(written), Version::Draft22, milli).unwrap(); assert_eq!(again.timestamp, at(500)); @@ -568,7 +581,14 @@ mod tests { let decoded = read(&both, Version::Draft22, milli).unwrap(); assert_eq!((decoded.timestamp, decoded.properties.sealed_time), (at(50), at(42))); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft22, milli).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft22, + milli + ) + .unwrap(), both ); @@ -576,7 +596,14 @@ mod tests { let equal = Bytes::from_static(&[0x0b, 2, 0x10, 42, 0x10 - 0x0b, 42]); let decoded = read(&equal, Version::Draft22, milli).unwrap(); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft22, milli).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft22, + milli + ) + .unwrap(), [0x0b, 2, 0x10, 42] ); } @@ -610,7 +637,14 @@ mod tests { assert_eq!(decoded.properties.heap(), limit as u64); assert!(check_frame(&decoded.properties, milli).is_ok()); let latest = Timestamp::new((1 << 62) - 1, Timescale::MILLI).unwrap(); - let written = write(&decoded.properties, Some(latest), Version::Draft22, milli).unwrap(); + let written = write( + &decoded.properties, + Some(Version::Draft22), + Some(latest), + Version::Draft22, + milli, + ) + .unwrap(); assert_eq!(written.len(), MAX_BLOCK); let over = read(&list(limit + 1), Version::Draft22, milli).unwrap(); @@ -620,10 +654,10 @@ mod tests { assert!(check_frame(&untimed.properties, None).is_ok()); // The same bound on a block arriving over the lite hop. - let encoding = Encoding::Draft19; - assert!(check(&block(limit), encoding, milli, true).is_ok()); - assert!(check(&block(limit + 1), encoding, milli, true).is_err()); - assert!(check(&block(limit + 1), encoding, None, true).is_ok()); + let version = Version::Draft22; + assert!(check(&block(limit), version, milli, true).is_ok()); + assert!(check(&block(limit + 1), version, milli, true).is_err()); + assert!(check(&block(limit + 1), version, None, true).is_ok()); } /// A status object has no frame to hold its time, so its mutable Timestamp stays in the @@ -635,12 +669,15 @@ mod tests { let framed = read(&list, Version::Draft14, milli).unwrap(); assert_eq!(framed.timestamp, Some(Timestamp::new(7, Timescale::MILLI).unwrap())); let status = unframed(&list, Version::Draft14, milli).unwrap(); - assert_eq!(write(&status, None, Version::Draft14, None).unwrap(), list.to_vec()); + assert_eq!( + write(&status, Some(Version::Draft14), None, Version::Draft14, None).unwrap(), + list.to_vec() + ); // Over the lite hop, only a status block may hold it. let block = status.block.unwrap().bytes; - assert!(check(&block, Encoding::Draft14, milli, false).is_ok()); - assert!(check(&block, Encoding::Draft14, milli, true).is_err()); + assert!(check(&block, Version::Draft14, milli, false).is_ok()); + assert!(check(&block, Version::Draft14, milli, true).is_err()); } /// A draft-14 block crosses the lite hop in ascending order, as ingress sorts it, so @@ -648,11 +685,11 @@ mod tests { #[test] fn a_draft14_block_ascends() { let milli = Some(Timescale::MILLI); - assert!(check(&Bytes::from_static(&[0x02, 1, 0x20, 1]), Encoding::Draft14, milli, true).is_ok()); - assert!(check(&Bytes::from_static(&[0x02, 1, 0x02, 2]), Encoding::Draft14, milli, true).is_ok()); - assert!(check(&Bytes::from_static(&[0x20, 1, 0x02, 1]), Encoding::Draft14, milli, true).is_err()); + assert!(check(&Bytes::from_static(&[0x02, 1, 0x20, 1]), Version::Draft14, milli, true).is_ok()); + assert!(check(&Bytes::from_static(&[0x02, 1, 0x02, 2]), Version::Draft14, milli, true).is_ok()); + assert!(check(&Bytes::from_static(&[0x20, 1, 0x02, 1]), Version::Draft14, milli, true).is_err()); let sealed = Bytes::from_static(&[0x02, 1, 0x0b, 4, 0x20, 1, 0x02, 1]); - assert!(check(&sealed, Encoding::Draft14, milli, true).is_ok()); + assert!(check(&sealed, Version::Draft14, milli, true).is_ok()); } /// Draft-19 assigns 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so it cannot cross that boundary. @@ -664,7 +701,14 @@ mod tests { assert_eq!(decoded.timestamp, None); for timescale in [Some(Timescale::MICRO), None] { assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft19, timescale).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft19, + timescale + ) + .unwrap(), bytes ); } @@ -672,12 +716,26 @@ mod tests { // Across draft-19 neither reading survives: a mutable one is dropped. let modern = Bytes::from_static(&[0x20, 7]); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft18, milli).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft18, + milli + ) + .unwrap(), modern ); let legacy = read(&bytes, Version::Draft18, milli).unwrap(); assert_eq!( - write(&legacy.properties, legacy.timestamp, Version::Draft19, milli).unwrap(), + write( + &legacy.properties, + Some(Version::Draft18), + legacy.timestamp, + Version::Draft19, + milli + ) + .unwrap(), modern ); @@ -686,16 +744,35 @@ mod tests { let decoded = read(&sealed, Version::Draft22, milli).unwrap(); assert_eq!(decoded.properties.sealed_time, None); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft22, None).unwrap(), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft22, + None + ) + .unwrap(), sealed ); assert!(matches!( - write(&decoded.properties, decoded.timestamp, Version::Draft18, milli), + write( + &decoded.properties, + Some(Version::Draft22), + decoded.timestamp, + Version::Draft18, + milli + ), Err(Error::Unsupported) )); let legacy = read(&sealed, Version::Draft18, milli).unwrap(); assert!(matches!( - write(&legacy.properties, legacy.timestamp, Version::Draft19, milli), + write( + &legacy.properties, + Some(Version::Draft18), + legacy.timestamp, + Version::Draft19, + milli + ), Err(Error::Unsupported) )); } @@ -709,12 +786,25 @@ mod tests { let bytes = Bytes::from_static(&[0x0b, 8, 2, 0xfc, 0, 0, 0, 0, 0, 1]); let decoded = read(&bytes, Version::Draft18, None).unwrap(); assert_eq!( - write(&decoded.properties, decoded.timestamp, Version::Draft22, None).unwrap(), + write( + &decoded.properties, + Some(Version::Draft18), + decoded.timestamp, + Version::Draft22, + None + ) + .unwrap(), bytes ); assert!(read(&bytes, Version::Draft17, None).is_err()); assert!(matches!( - write(&decoded.properties, decoded.timestamp, Version::Draft17, None), + write( + &decoded.properties, + Some(Version::Draft18), + decoded.timestamp, + Version::Draft17, + None + ), Err(Error::Unsupported) )); } diff --git a/rs/moq-net/src/ietf/properties.rs b/rs/moq-net/src/ietf/properties.rs index bbdf0442a9d..d724f0775ea 100644 --- a/rs/moq-net/src/ietf/properties.rs +++ b/rs/moq-net/src/ietf/properties.rs @@ -73,8 +73,9 @@ pub struct Properties { /// Whether the track takes NEW_GROUP_REQUEST (DYNAMIC_GROUPS = 1), mutable or sealed. pub dynamic_groups: bool, - /// Everything else, Immutable Properties included, forwarded as the publisher sent it. - pub(crate) unknown: TrackProperties, + /// Everything else, Immutable Properties included, forwarded as the publisher sent it, + /// serialized as the message's draft serializes them. + pub(crate) unknown: Vec, } impl Properties { @@ -99,7 +100,7 @@ impl Properties { group_order: Some(GroupOrder::Descending), // A track opted in to refresh answers a NEW_GROUP_REQUEST as a join. dynamic_groups: info.refresh.is_some(), - unknown: TrackProperties::default(), + unknown: Vec::new(), } } @@ -147,8 +148,8 @@ impl Properties { let mut properties: Vec<_> = known .iter() - .filter(|property| !self.unknown.sealed().any(|id| id == property.id)) - .chain(&self.unknown.values) + .filter(|property| !sealed_keys(&self.unknown, version).any(|id| id == property.id)) + .chain(&self.unknown) .collect(); // Delta types require ascending order. Equal types keep the publisher's order. properties.sort_by_key(|property| property.id); @@ -208,7 +209,7 @@ impl Properties { // Even type: single varint value let value = r.varint()?; if !properties.known(abs, value, &mut seen)? { - properties.unknown.values.push(Property { + properties.unknown.push(Property { id: abs, value: Value::Int(value), }); @@ -222,22 +223,19 @@ impl Properties { let value = r.slice(len)?; if abs == IMMUTABLE { // A second block would leave the sealed list ambiguous. - if properties.unknown.immutable().is_some() { + if immutable(&properties.unknown).is_some() { return Err(DecodeError::InvalidValue); } sealed(&mut Decoder::new(value, version.into()), &mut properties, &mut seen)?; } // A small kept property must not pin the message's larger buffer. - properties.unknown.values.push(Property { + properties.unknown.push(Property { id: abs, value: Value::Bytes(Bytes::copy_from_slice(value)), }); } } - // Recorded only beside a value they describe, so a set with nothing to forward stays empty. - let forwarded = !properties.unknown.values.is_empty(); - properties.unknown.encoding = forwarded.then(|| super::object::encoding(version)); Ok(properties) } @@ -294,14 +292,13 @@ pub(crate) fn understood(id: u64) -> bool { /// within the [`MAX_PROPERTIES`] a message may carry. pub(crate) const MAX_FORWARDED: usize = MAX_PROPERTIES as usize - UNDERSTOOD.len(); -/// Check a Track Properties' Immutable Properties `block`, serialized in the `encoding` -/// family, as [`Properties::decode`] does. -pub(crate) fn check_immutable(block: &[u8], encoding: crate::model::object::Encoding) -> Result<(), DecodeError> { - // Track Properties start at draft-16, whose families all delta-encode their keys. - if encoding == crate::model::object::Encoding::Draft14 { +/// Check a Track Properties' Immutable Properties `block` from a `version` publisher, as +/// [`Properties::decode`] does. +pub(crate) fn check_immutable(block: &[u8], version: Version) -> Result<(), DecodeError> { + // Track Properties start at draft-16. + if matches!(version, Version::Draft14 | Version::Draft15) { return Err(DecodeError::InvalidValue); } - let version = super::object::family(encoding); sealed( &mut Decoder::new(block, version.into()), &mut Properties::default(), @@ -309,36 +306,41 @@ pub(crate) fn check_immutable(block: &[u8], encoding: crate::model::object::Enco ) } +/// The Immutable Properties block among `values`. +fn immutable(values: &[Property]) -> Option<&Bytes> { + values.iter().find_map(|property| match (property.id, &property.value) { + (IMMUTABLE, Value::Bytes(block)) => Some(block), + _ => None, + }) +} + +/// The keys inside the Immutable Properties among `values`, serialized as `version` does. +/// +/// Read from the block each time rather than held as a list, which could be many times its +/// size. Ingress checked the block, so a malformed key cannot occur and ends it. +fn sealed_keys(values: &[Property], version: Version) -> impl Iterator + '_ { + let mut r = immutable(values).map(|block| Decoder::new(block, version.into())); + let mut id: u64 = 0; + std::iter::from_fn(move || { + let r = r.as_mut().filter(|r| !r.is_empty())?; + id = id.checked_add(r.varint().ok()?)?; + match id.is_multiple_of(2) { + true => r.varint().ok().map(drop)?, + false => r.bytes().ok().map(drop)?, + } + Some(id) + }) +} + impl TrackProperties { - /// The Immutable Properties block, serialized as [`Self::encoding`] says. + /// The Immutable Properties block, serialized as [`Self::version`] does. pub(crate) fn immutable(&self) -> Option<&Bytes> { - self.values - .iter() - .find_map(|property| match (property.id, &property.value) { - (IMMUTABLE, Value::Bytes(block)) => Some(block), - _ => None, - }) + immutable(&self.values) } /// The keys inside Immutable Properties, which egress must not restate as mutable. - /// - /// Read from the block each time rather than held as a list, which could be many times - /// its size. Ingress checked the block, so a malformed key cannot occur and ends it. pub(crate) fn sealed(&self) -> impl Iterator + '_ { - let mut r = self - .immutable() - .zip(self.encoding) - .map(|(block, encoding)| Decoder::new(block, super::object::family(encoding).into())); - let mut id: u64 = 0; - std::iter::from_fn(move || { - let r = r.as_mut().filter(|r| !r.is_empty())?; - id = id.checked_add(r.varint().ok()?)?; - match id.is_multiple_of(2) { - true => r.varint().ok().map(drop)?, - false => r.bytes().ok().map(drop)?, - } - Some(id) - }) + sealed_keys(&self.values, self.version) } /// Why a track carrying these beside `info` is neither processed nor forwarded, if it @@ -365,12 +367,12 @@ impl TrackProperties { /// Whether Immutable Properties seal a property we understand at another value than /// the `restated` one the relay acts on. fn contradicts(&self, restated: &Properties) -> bool { - let Some((block, encoding)) = self.immutable().zip(self.encoding) else { + let Some(block) = self.immutable() else { return false; }; let mut sealed = Properties::default(); let mut seen = Vec::new(); - let r = &mut Decoder::new(block, super::object::family(encoding).into()); + let r = &mut Decoder::new(block, self.version.into()); // Ingress checked the block, so a malformed one cannot occur and is refused. if self::sealed(r, &mut sealed, &mut seen).is_err() { return true; @@ -693,14 +695,10 @@ mod tests { let (properties, written) = round_trip(version, &block); assert_eq!(written, block, "{version}"); - assert_eq!(properties.unknown.values.len(), 3, "{version}"); - assert!(properties.unknown.sealed().eq([0x02]), "{version}"); - assert_eq!( - properties.unknown.encoding, - Some(super::super::object::encoding(version)), - "{version}" - ); - assert!(!properties.unknown.mandatory(), "{version}"); + assert_eq!(properties.unknown.len(), 3, "{version}"); + let track = forwarded(&properties, version); + assert!(track.sealed().eq([0x02]), "{version}"); + assert!(!track.mandatory(), "{version}"); // Ours interleave by type. let mut ours = properties.clone(); @@ -757,18 +755,26 @@ mod tests { )); } + /// What a relay keeps of `properties`, which arrived on `version`. + fn forwarded(properties: &Properties, version: Version) -> TrackProperties { + TrackProperties { + version, + values: properties.unknown.clone(), + } + } + /// A relay forwards only as many properties as leave room for every one it restates, so /// what it sends always decodes. One more is refused at ingress. #[test] fn forwarded_properties_leave_room_for_the_restated() { let unknown = |count: usize| TrackProperties { + version: Version::Draft22, values: (0..count as u64) .map(|i| Property { id: 0x40 + 2 * i, value: Value::Int(1), }) .collect(), - encoding: Some(crate::model::object::Encoding::Draft19), }; let properties = Properties { max_cache_duration: Some(Duration::from_secs(1)), @@ -776,9 +782,13 @@ mod tests { priority: Some(1), group_order: Some(GroupOrder::Descending), dynamic_groups: true, - unknown: unknown(MAX_FORWARDED), + unknown: unknown(MAX_FORWARDED).values, }; - assert!(properties.unknown.refused(&crate::track::Info::default()).is_none()); + assert!( + forwarded(&properties, Version::Draft22) + .refused(&crate::track::Info::default()) + .is_none() + ); let mut buf = Vec::new(); properties .encode(&mut Encoder::new(&mut buf, Version::Draft22.into()), Version::Draft22) @@ -801,11 +811,11 @@ mod tests { w.varint(id).unwrap(); w.varint(value).unwrap(); let unknown = TrackProperties { + version, values: vec![Property { id: IMMUTABLE, value: Value::Bytes(block.into()), }], - encoding: Some(super::super::object::encoding(version)), }; let info = crate::track::Info::default() .with_max_age(Duration::from_secs(7)) @@ -857,13 +867,10 @@ mod tests { let w = &mut Encoder::new(&mut sealed, version.into()); w.varint(DEFAULT_PUBLISHER_GROUP_ORDER).unwrap(); w.varint(1).unwrap(); - let unknown = TrackProperties { - values: vec![Property { - id: IMMUTABLE, - value: Value::Bytes(sealed.clone().into()), - }], - encoding: Some(super::super::object::encoding(version)), - }; + let unknown = vec![Property { + id: IMMUTABLE, + value: Value::Bytes(sealed.clone().into()), + }]; let egress = Properties { unknown: unknown.clone(), ..Properties::restated(&crate::track::Info::default()) @@ -905,16 +912,17 @@ mod tests { let mut w = Encoder::new(&mut mandatory, Version::Draft22.into()); w.varint(0x4000).unwrap(); w.varint(1).unwrap(); - assert!(decode_draft22(&mandatory).unwrap().unknown.mandatory()); + let mandatory_in = |list: &[u8]| forwarded(&decode_draft22(list).unwrap(), Version::Draft22).mandatory(); + assert!(mandatory_in(&mandatory)); let mut sealed = vec![0x0b, mandatory.len() as u8]; sealed.extend(&mandatory); - assert!(decode_draft22(&sealed).unwrap().unknown.mandatory()); + assert!(mandatory_in(&sealed)); let mut below = Vec::new(); let mut w = Encoder::new(&mut below, Version::Draft22.into()); w.varint(0x3ffe).unwrap(); w.varint(1).unwrap(); - assert!(!decode_draft22(&below).unwrap().unknown.mandatory()); + assert!(!mandatory_in(&below)); } } diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index e7107564ede..7c75b1a667e 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -53,25 +53,21 @@ fn track_properties(info: &track::Info, wanted: bool, version: Version) -> Resul if !wanted { return Ok(ietf::Properties::default()); } - let mut unknown = info.properties.as_deref().cloned().unwrap_or_default(); + let Some(track) = info.properties.as_deref() else { + return Ok(ietf::Properties::restated(info)); + }; + let mut unknown = track.values.clone(); let block = !matches!(version, Version::Draft14 | Version::Draft15); - if block - && unknown.immutable().is_some() - && !unknown - .encoding - .is_some_and(|encoding| super::object::same_bytes(encoding, super::object::encoding(version))) - { + if block && track.immutable().is_some() && !super::object::same_bytes(track.version, version) { return Err(Error::Unsupported); } // Across draft-19, forwarding 0x06 would give it a different meaning, as for object // properties. A mutable one is dropped; a sealed one is refused. - if unknown.encoding.is_some_and(|encoding| { - super::object::delivery_timeout(encoding) != super::object::has_delivery_timeout(version) - }) { - if block && unknown.sealed().any(|id| id == 0x06) { + if super::object::has_delivery_timeout(track.version) != super::object::has_delivery_timeout(version) { + if block && track.sealed().any(|id| id == 0x06) { return Err(Error::Unsupported); } - unknown.values.retain(|property| property.id != 0x06); + unknown.retain(|property| property.id != 0x06); } Ok(ietf::Properties { unknown, @@ -130,6 +126,8 @@ struct FetchObjectInfo<'a> { timescale: Option, version: Version, fields: Option<&'a crate::model::object::Fields>, + /// The draft the track's publisher spoke, which serialized `fields`. + source: Option, } fn fetch_prior(version: Version, prior: Option, object: u64) -> Result { @@ -1961,6 +1959,7 @@ where timescale, version, fields: frame.fields.as_deref(), + source: group.ietf(), }, ) .await?; @@ -2007,6 +2006,7 @@ where timescale, version, fields: frame.fields.as_deref(), + source: group.ietf(), }, ) .await?; @@ -2059,12 +2059,13 @@ where timescale, version, fields, + source, } = header; // Relayed properties survive without a timescale. Plain frames generate a // timestamp only on drafts that can declare its units. let timescale = timescale.filter(|_| ietf::Properties::sends_timescale(version)); let properties = if let Some(fields) = fields { - let properties = super::object::write(&fields.properties, timestamp, version, timescale)?; + let properties = super::object::write(&fields.properties, source, timestamp, version, timescale)?; (!properties.is_empty()).then_some(properties) } else { match timescale { @@ -2459,7 +2460,9 @@ where .iter() .filter_map(|frame| Some((frame.fields.as_deref()?, frame.timestamp))); for (fields, frame) in frames.chain(group.end_object.as_deref().map(|end| (end, None))) { - if let Err(err) = super::object::write(&fields.properties, frame, self.version, timescale) { + if let Err(err) = + super::object::write(&fields.properties, group.info.ietf(), frame, self.version, timescale) + { return self.reject_fetch(stream, msg.request_id, &err, &err.to_string()).await; } } @@ -2599,6 +2602,7 @@ where timescale, version: self.version, fields: frame.fields.as_deref(), + source: group.info.ietf(), }, ) .await?; @@ -2615,6 +2619,7 @@ where timescale, version: self.version, fields: Some(end), + source: group.info.ietf(), }, ) .await?; @@ -4492,6 +4497,7 @@ impl TrackServe { }; let properties = match super::object::write( forwarded, + self.track.info().ietf(), datagram.datagram.timestamp, self.version, self.timescale, @@ -5023,6 +5029,7 @@ impl GroupServe { extensions: self.msg.flags.has_extensions, timescale: self.timescale, version: self.version, + source: self.group.ietf(), }, )?; self.prior_object = Some(id); @@ -5069,6 +5076,7 @@ impl GroupServe { extensions: self.msg.flags.has_extensions, timescale: self.timescale, version: self.version, + source: self.group.ietf(), }, )?; self.prior_object = Some(id); @@ -5099,6 +5107,7 @@ impl GroupServe { extensions: self.msg.flags.has_extensions, timescale: self.timescale, version: self.version, + source: self.group.ietf(), }, )?; self.ended = true; @@ -5196,6 +5205,8 @@ struct ObjectEncoding { extensions: bool, timescale: Option, version: Version, + /// The draft the track's publisher spoke, which serialized the objects' properties. + source: Option, } fn object_delta(prior: Option, id: u64) -> Result { @@ -5232,6 +5243,7 @@ fn buffer_object( let ext = match object.fields { Some(fields) => super::object::write( &fields.properties, + encoding.source, object.timestamp, encoding.version, encoding.timescale, @@ -9907,7 +9919,7 @@ mod serve_tests { w.bytes(b"opaque").unwrap(); let properties = crate::coding::decode_buf(&mut bytes::Bytes::from(block), version, ietf::Properties::decode).unwrap(); - assert_eq!(properties.unknown.values.len(), 2, "{version}"); + assert_eq!(properties.unknown.len(), 2, "{version}"); properties } @@ -9918,7 +9930,10 @@ mod serve_tests { for version in [Version::Draft16, Version::Draft17, Version::Draft22] { let upstream = unknown_properties(version); let info = track::Info { - properties: Some(Arc::new(upstream.unknown.clone())), + properties: Some(Arc::new(crate::model::object::TrackProperties { + version, + values: upstream.unknown.clone(), + })), ..track::Info::timed() }; @@ -9982,7 +9997,10 @@ mod serve_tests { assert!(upstream.dynamic_groups); let info = track::Info { refresh: Some(0), - properties: Some(Arc::new(upstream.unknown.clone())), + properties: Some(Arc::new(crate::model::object::TrackProperties { + version: Version::Draft22, + values: upstream.unknown.clone(), + })), ..track::Info::timed() }; @@ -10016,10 +10034,23 @@ mod serve_tests { } } + /// The info a relay behind a moq-lite hop with the `ietf` extension learns of a track + /// published with `info`. + async fn across_a_hop(info: track::Info) -> track::Info { + let upstream = crate::origin::Config::new(crate::Hop::new(2).unwrap()).produce(); + let broadcast = upstream.publish("room", crate::origin::Route::default()).unwrap(); + let _track = broadcast.create_track("video", info).unwrap(); + let (downstream, _sessions) = crate::lite::ietf::test_hop(&upstream, "room", [true, true]).await; + let broadcast = downstream.consume().request_broadcast("room", None).await.unwrap(); + broadcast.track("video").unwrap().query().await.unwrap() + } + /// Draft 19 assigns 0x06 to SUBGROUP_DELIVERY_TIMEOUT, so a track property 0x06 crosses /// that boundary as an object property does: dropped when mutable, refused when sealed. - #[test] - fn track_property_0x06_does_not_cross_draft_19() { + /// Drafts 18 and 19 serialize alike, so across a relay hop only the track's carried + /// ingress draft says which meaning it has. + #[moq_clock::test] + async fn track_property_0x06_does_not_cross_draft_19() { let upstream = |block: &[u8]| { let properties = crate::coding::decode_buf( &mut bytes::Bytes::copy_from_slice(block), @@ -10028,11 +10059,14 @@ mod serve_tests { ) .unwrap(); track::Info { - properties: Some(Arc::new(properties.unknown)), + properties: Some(Arc::new(crate::model::object::TrackProperties { + version: Version::Draft18, + values: properties.unknown, + })), ..Default::default() } }; - let ids = |properties: ietf::Properties| properties.unknown.values.iter().map(|p| p.id).collect::>(); + let ids = |properties: ietf::Properties| properties.unknown.iter().map(|p| p.id).collect::>(); // 0x06 = 42, 0x3801 = "hi" let mut block = Vec::new(); @@ -10041,22 +10075,33 @@ mod serve_tests { w.varint(42).unwrap(); w.varint(0x3801 - 0x06).unwrap(); w.bytes(b"hi").unwrap(); - let mutable = upstream(&block); - assert_eq!( - ids(track_properties(&mutable, true, Version::Draft18).unwrap()), - [0x06, 0x3801] - ); - assert_eq!( - ids(track_properties(&mutable, true, Version::Draft22).unwrap()), - [0x3801] - ); - let sealed = upstream(&[0x0b, 2, 0x06, 42]); - assert!(track_properties(&sealed, true, Version::Draft18).is_ok()); - assert!(matches!( - track_properties(&sealed, true, Version::Draft22), - Err(Error::Unsupported) - )); + for hop in [false, true] { + let (mutable, sealed) = match hop { + false => (upstream(&block), sealed.clone()), + true => (across_a_hop(upstream(&block)).await, across_a_hop(sealed.clone()).await), + }; + assert_eq!(mutable.ietf(), Some(Version::Draft18), "hop={hop}"); + assert_eq!( + ids(track_properties(&mutable, true, Version::Draft18).unwrap()), + [0x06, 0x3801], + "hop={hop}" + ); + assert_eq!( + ids(track_properties(&mutable, true, Version::Draft19).unwrap()), + [0x3801], + "hop={hop}" + ); + + assert!(track_properties(&sealed, true, Version::Draft18).is_ok(), "hop={hop}"); + assert!( + matches!( + track_properties(&sealed, true, Version::Draft19), + Err(Error::Unsupported) + ), + "hop={hop}" + ); + } } /// A range ending inside its group stops at its End Object, which draft 20 counts diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 085c9ee70d0..9814616d695 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -462,12 +462,13 @@ impl Target { } } -/// The model's info for a track the publisher described (SUBSCRIBE_OK or TRACK_STATUS_OK). +/// The model's info for a track a `version` publisher described (SUBSCRIBE_OK or +/// TRACK_STATUS_OK). /// /// The copy keeps the units the publisher declared, so it matches a copy of the same /// instance that arrived over another route: resuming across routes refuses one whose /// metadata differs. A sealed TIMESCALE travels downstream unchanged, so it must too. -fn accepted_info(properties: &ietf::Properties) -> track::Info { +fn accepted_info(properties: &ietf::Properties, version: Version) -> track::Info { let mut info = track::Info::default() .with_timescale(properties.timescale) .with_max_age(properties.max_cache_duration) @@ -475,14 +476,18 @@ fn accepted_info(properties: &ietf::Properties) -> track::Info { // A publisher answering NEW_GROUP_REQUEST refreshes on any join with a delta to skip; // this session carries the request back to it. .with_refresh(properties.dynamic_groups.then_some(0)); - info.properties = - Some(std::sync::Arc::new(properties.unknown.clone())).filter(|unknown| !unknown.values.is_empty()); + info.properties = Some(std::sync::Arc::new(crate::model::object::TrackProperties { + version, + values: properties.unknown.clone(), + })); info } -/// Why a track answered with `properties` is neither processed nor forwarded, if it is not. -fn refused(properties: &ietf::Properties) -> Option { - properties.unknown.refused(&accepted_info(properties)) +/// Why a track a `version` publisher answered with `properties` is neither processed nor +/// forwarded, if it is not. +fn refused(properties: &ietf::Properties, version: Version) -> Option { + let info = accepted_info(properties, version); + info.properties.as_deref()?.refused(&info) } /// The Largest Group a subscription knows: its SUBSCRIBE_OK's, or a newer one since. @@ -2933,8 +2938,8 @@ where // A track that needs a property we do not understand, or carries more than we // could forward, is neither processed nor forwarded: the subscription is // cancelled and the subscribers refused. - Ok(Some(accepted)) if refused(&accepted.properties).is_some() => { - let err = refused(&accepted.properties).expect("checked by the guard"); + Ok(Some(accepted)) if refused(&accepted.properties, self.version).is_some() => { + let err = refused(&accepted.properties, self.version).expect("checked by the guard"); tracing::debug!(broadcast = %self.origin.absolute(broadcast_path), track = %track_name, %err, "subscribe refused"); // Refuse before waiting on the peer to acknowledge the cancellation. if let Some(pending) = self.take_pending(request_id) { @@ -2984,10 +2989,13 @@ where let timescale = properties .timescale .or(resumed.as_ref().and_then(|idle| idle.timescale)); - let info = accepted_info(&ietf::Properties { - timescale, - ..properties - }); + let info = accepted_info( + &ietf::Properties { + timescale, + ..properties + }, + self.version, + ); // The copy already holds the track: it is current again up to the answer's Largest // Location once the join's head lands or a newer group does, since leaving cut the // group it was in. @@ -3865,7 +3873,7 @@ where let _ = stream.writer.finish(); let ok = match response { // A track that needs a property we do not understand is not forwarded. - Ok(ok) => match refused(&ok.properties) { + Ok(ok) => match refused(&ok.properties, self.version) { Some(err) => Err(err), None => Ok(ok), }, @@ -3882,7 +3890,7 @@ where tracing::debug!(broadcast = %self.origin.absolute(broadcast_path), track = %track_name, "track status known"); let dynamic = request.dynamic(); - let mut track = request.accept(accepted_info(&ok.properties)); + let mut track = request.accept(accepted_info(&ok.properties, self.version)); track.received(); // Nothing feeds this copy live: what it caches is fetched, never the live edge. track.set_idle(); @@ -12415,10 +12423,13 @@ mod stitch_tests { let track = broadcast .create_track( "video", - accepted_info(&ietf::Properties { - timescale, - ..Default::default() - }), + accepted_info( + &ietf::Properties { + timescale, + ..Default::default() + }, + ingress, + ), ) .unwrap(); let mut h = Harness::with_timescale(Fill::Done, vec![script], timescale); @@ -12599,7 +12610,7 @@ mod stitch_tests { let relayed = bytes::Bytes::copy_from_slice(decoder.bytes().unwrap()); let decoded = super::super::object::read(&relayed, version, Some(Timescale::MILLI)).unwrap(); assert_eq!(decoded.timestamp, Some(Timestamp::new(42, Timescale::MILLI).unwrap())); - assert_eq!(super::super::object::listed(&decoded.properties).len(), 1); + assert_eq!(super::super::object::listed(&decoded.properties, version).len(), 1); } fn object_stream(version: Version, first: bool, objects: &[(u64, &[u8], Option)]) -> Vec { @@ -12755,7 +12766,7 @@ mod stitch_tests { ); if let Some(end) = end_object { assert_eq!( - super::super::object::listed(&end.properties) + super::super::object::listed(&end.properties, version) .first() .map(|property| property.id), legacy.then_some(0x3e) @@ -12835,7 +12846,7 @@ mod stitch_tests { let version = Version::Draft14; let status = [0x10, 7]; let leaves = |properties: &crate::model::object::Properties| { - super::super::object::write(properties, None, version, None).unwrap() + super::super::object::write(properties, Some(version), None, version, None).unwrap() }; let script = object_stream( @@ -13012,7 +13023,7 @@ mod stitch_tests { .unwrap(); assert_eq!(frame.fields.as_ref().unwrap().id, id); assert_eq!( - super::super::object::listed(&frame.fields.as_ref().unwrap().properties)[0].id, + super::super::object::listed(&frame.fields.as_ref().unwrap().properties, version)[0].id, 0x3e ); } @@ -13108,8 +13119,7 @@ mod stitch_tests { ] { let script = object_stream(ingress, true, &[(0, &properties, None)]); let (bytes, log) = relay_passthrough_stream(script, ingress, egress).await; - let encoding = super::super::object::encoding; - if !super::super::object::same_bytes(encoding(ingress), encoding(egress)) { + if !super::super::object::same_bytes(ingress, egress) { assert!(!log.resets().is_empty(), "incompatible immutable bytes must be refused"); continue; } @@ -13122,6 +13132,41 @@ mod stitch_tests { } } + /// Draft 19 assigns 0x06 to SUBGROUP_DELIVERY_TIMEOUT. A draft-18 publisher's 0x06 is + /// dropped on its way to draft 19 when mutable, and refused when sealed, through a + /// moq-lite hop too: drafts 18 and 19 serialize a block alike, so only the track's + /// carried ingress draft tells them apart. + #[moq_clock::test] + async fn passthrough_keeps_0x06_from_crossing_draft_19() { + let (ingress, egress) = (Version::Draft18, Version::Draft19); + // 0x06 = 42 and 0x20 = 7, mutable. + let mutable = [0x06, 42, 0x20 - 0x06, 7]; + let script = object_stream(ingress, true, &[(0, &mutable, None)]); + let (bytes, log) = relay_passthrough_stream(script, ingress, egress).await; + assert!(log.resets().is_empty()); + let mut decoder = Decoder::new(&bytes, egress.into()); + ietf::GroupHeader::decode(&mut decoder, egress).unwrap(); + assert_eq!(decoder.varint().unwrap(), 0); + assert_eq!(decoder.bytes().unwrap(), [0x20, 7], "the mutable 0x06 crossed"); + + // 0x0b = { 0x06 = 42 }, sealed. + let sealed = [0x0b, 2, 0x06, 42]; + let script = object_stream(ingress, true, &[(0, &sealed, None)]); + let (_, log) = relay_passthrough_stream(script, ingress, egress).await; + assert!(!log.resets().is_empty(), "the sealed 0x06 crossed"); + + // Within a side of draft 19 both cross unchanged. + for properties in [&mutable[..], &sealed] { + let script = object_stream(ingress, true, &[(0, properties, None)]); + let (bytes, log) = relay_passthrough_stream(script, ingress, ingress).await; + assert!(log.resets().is_empty()); + let mut decoder = Decoder::new(&bytes, ingress.into()); + ietf::GroupHeader::decode(&mut decoder, ingress).unwrap(); + assert_eq!(decoder.varint().unwrap(), 0); + assert_eq!(decoder.bytes().unwrap(), properties); + } + } + #[moq_clock::test] async fn passthrough_preserves_datagram_objects_through_dispatch() { // Also through a second relay behind a moq-lite hop with the `ietf` extension. @@ -13134,7 +13179,9 @@ mod stitch_tests { { let origin = crate::origin::Config::new(crate::Hop::new(2).unwrap()).produce(); let broadcast = origin.publish("room", crate::origin::Route::default()).unwrap(); - let track = broadcast.create_track("video", None).unwrap(); + let track = broadcast + .create_track("video", accepted_info(&Default::default(), version)) + .unwrap(); let mut h = Harness::with_timescale(Fill::Done, vec![], None); h.subscriber.version = version; h.subscriber.state.lock().subscribes.get_mut(&REQUEST).unwrap().producer = Some(track.clone()); @@ -13260,7 +13307,9 @@ mod stitch_tests { for (egress, status) in pairs { let origin = crate::origin::Config::new(crate::Hop::new(2).unwrap()).produce(); let broadcast = origin.publish("room", crate::origin::Route::default()).unwrap(); - let track = broadcast.create_track("video", None).unwrap(); + let track = broadcast + .create_track("video", accepted_info(&Default::default(), ingress)) + .unwrap(); let mut h = Harness::with_timescale(Fill::Done, vec![], None); h.subscriber.version = ingress; h.subscriber.state.lock().subscribes.get_mut(&REQUEST).unwrap().producer = Some(track.clone()); diff --git a/rs/moq-net/src/ietf/version.rs b/rs/moq-net/src/ietf/version.rs index 3eddd2be5a8..0e29f184548 100644 --- a/rs/moq-net/src/ietf/version.rs +++ b/rs/moq-net/src/ietf/version.rs @@ -15,21 +15,6 @@ pub enum Version { Draft22, } -impl Version { - /// Every implemented draft, oldest first. - pub(crate) const ALL: [Self; 9] = [ - Self::Draft14, - Self::Draft15, - Self::Draft16, - Self::Draft17, - Self::Draft18, - Self::Draft19, - Self::Draft20, - Self::Draft21, - Self::Draft22, - ]; -} - impl fmt::Display for Version { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { diff --git a/rs/moq-net/src/lite/datagram.rs b/rs/moq-net/src/lite/datagram.rs index d418cef2a04..b0ed3a7e0db 100644 --- a/rs/moq-net/src/lite/datagram.rs +++ b/rs/moq-net/src/lite/datagram.rs @@ -7,7 +7,6 @@ //! payload runs to the datagram boundary, so unlike a [`super::Message`] there is no inner length //! prefix. The model counterpart is [`crate::Datagram`]. -use crate::Timescale; use bytes::Bytes; use crate::coding::{DecodeError, Decoder, Encode, EncodeError, Encoder}; @@ -34,16 +33,6 @@ pub struct Datagram { pub ietf: Option>, } -/// What a datagram body carries beyond its payload, which the subscription's track decides -/// and the body itself does not say. -#[derive(Clone, Copy, Debug, Default)] -pub struct Format { - /// The units of the track's timestamps, or `None` when it carries none. - pub timescale: Option, - /// The track is compressed, so the payload's inflated size precedes it. - pub compressed: bool, -} - impl Encode for Datagram { fn encode(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { if !version.has_datagrams() { @@ -73,7 +62,7 @@ impl Encode for Datagram { { return Err(EncodeError::InvalidState); } - super::ietf::encode_datagram(w, wire.as_ref(), version)?; + super::ietf::encode_datagram(w, wire.as_ref())?; } if let Some(timestamp) = self.timestamp { w.varint(timestamp)?; @@ -91,13 +80,13 @@ impl Encode for Datagram { impl Datagram { /// Decode a whole datagram body. The payload shares `buf` rather than copying it. /// - /// `ietf` says whether the session negotiated the extension, and `format` what the track - /// of the given subscribe ID carries, which the body itself does not say. - pub fn decode( + /// `ietf` says whether the session negotiated the extension, and `framing` describes + /// the track of the given subscribe ID, which the body itself does not. + pub(crate) fn decode( buf: Bytes, version: Version, ietf: bool, - format: impl FnOnce(u64) -> Format, + framing: impl FnOnce(u64) -> super::Framing, ) -> Result { if !version.has_datagrams() || (ietf && !version.has_ietf()) { return Err(DecodeError::Version); @@ -106,9 +95,10 @@ impl Datagram { let mut r = Decoder::new(&buf, version.into()); let subscribe = r.varint()?; let sequence = r.varint()?; - let Format { timescale, compressed } = format(subscribe); + let framing = framing(subscribe); + let timescale = framing.timescale; let ietf = ietf - .then(|| super::ietf::decode_datagram(&mut r, timescale, version)) + .then(|| super::ietf::decode_datagram(&mut r, framing)) .transpose()?; // Any status object carries no payload, which moq-transport could not send beside // it, and a non-zero one carries no object and so no time either. @@ -122,7 +112,7 @@ impl Datagram { None }; // A status has no payload to inflate. - let inflated = match compressed && version.has_compression() && status.is_none() { + let inflated = match framing.compressed && version.has_compression() && status.is_none() { true => Some(r.varint()?), false => None, }; @@ -147,16 +137,32 @@ impl Datagram { #[cfg(test)] mod test { use super::*; + use crate::Timescale; - const TIMED: Format = Format { - timescale: Some(Timescale::MILLI), - compressed: false, - }; + /// A millisecond track a moq-lite publisher published. + fn timed() -> super::super::Framing { + super::super::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + } + } - const COMPRESSED: Format = Format { - timescale: Some(Timescale::MILLI), - compressed: true, - }; + /// A millisecond track a draft-22 moq-transport publisher published. + fn relayed() -> super::super::Framing { + super::super::Framing { + ietf: Some(crate::ietf::Version::Draft22), + ..timed() + } + } + + /// A millisecond compressed track a moq-lite publisher published. + fn compressed() -> super::super::Framing { + super::super::Framing { + compressed: true, + ..timed() + } + } /// A compressed track's inflated size sits between the timestamp and the payload. #[test] @@ -172,7 +178,7 @@ mod test { let buf = dg.encode_bytes(Version::Lite07).unwrap(); assert_eq!(&buf[..4], [1, 2, 3, 40]); assert_eq!( - Datagram::decode(buf, Version::Lite07, false, |_| COMPRESSED).unwrap(), + Datagram::decode(buf, Version::Lite07, false, |_| compressed()).unwrap(), dg ); @@ -195,7 +201,7 @@ mod test { ietf: None, }; let buf = original.encode_bytes(Version::Lite05).unwrap(); - let decoded = Datagram::decode(buf, Version::Lite05, false, |_| TIMED).unwrap(); + let decoded = Datagram::decode(buf, Version::Lite05, false, |_| timed()).unwrap(); assert_eq!(decoded, original, "payload has no trailing length prefix"); } @@ -210,7 +216,7 @@ mod test { ietf: None, }; let buf = original.encode_bytes(Version::Lite05).unwrap(); - let decoded = Datagram::decode(buf, Version::Lite05, false, |_| TIMED).unwrap(); + let decoded = Datagram::decode(buf, Version::Lite05, false, |_| timed()).unwrap(); assert_eq!(decoded, original); } @@ -247,7 +253,7 @@ mod test { assert_eq!(&buf[2..], b"world"); let decoded = Datagram::decode(buf, Version::Lite07, false, |id| { assert_eq!(id, 1); - Format::default() + Default::default() }) .unwrap(); assert_eq!(decoded, dg); @@ -292,7 +298,7 @@ mod test { id, value: crate::model::object::Value::Int(7), }], - crate::model::object::Encoding::Draft19, + crate::ietf::Version::Draft22, ) .unwrap(), ..Default::default() @@ -301,7 +307,7 @@ mod test { }; let decode = |dg: Datagram| { let buf = dg.encode_bytes(Version::Lite07).unwrap(); - Datagram::decode(buf, Version::Lite07, true, |_| TIMED) + Datagram::decode(buf, Version::Lite07, true, |_| relayed()) }; assert!(decode(dg(0x20)).is_ok()); assert!(matches!(decode(dg(0x10)), Err(DecodeError::InvalidValue))); @@ -341,14 +347,20 @@ mod test { "{status}" ); let mut buf = empty.encode_bytes(Version::Lite07).unwrap().to_vec(); - for format in [TIMED, COMPRESSED] { - let decoded = Datagram::decode(Bytes::from(buf.clone()), Version::Lite07, true, |_| format).unwrap(); + for framing in [ + relayed(), + super::super::Framing { + compressed: true, + ..relayed() + }, + ] { + let decoded = Datagram::decode(Bytes::from(buf.clone()), Version::Lite07, true, |_| framing).unwrap(); assert_eq!(decoded, empty, "{status}"); } buf.push(b'x'); assert!( matches!( - Datagram::decode(Bytes::from(buf), Version::Lite07, true, |_| TIMED), + Datagram::decode(Bytes::from(buf), Version::Lite07, true, |_| relayed()), Err(DecodeError::InvalidValue) ), "{status}" @@ -369,7 +381,8 @@ mod test { assert!(matches!(dg.encode_bytes(Version::Lite04), Err(EncodeError::Version))); assert!(matches!( - Datagram::decode(Bytes::from_static(b"\x01\x02\x03x"), Version::Lite04, false, |_| TIMED), + Datagram::decode(Bytes::from_static(b"\x01\x02\x03x"), Version::Lite04, false, |_| timed( + )), Err(DecodeError::Version) )); } diff --git a/rs/moq-net/src/lite/ietf.rs b/rs/moq-net/src/lite/ietf.rs index c5e32a0def9..e1541398bb0 100644 --- a/rs/moq-net/src/lite/ietf.rs +++ b/rs/moq-net/src/lite/ietf.rs @@ -12,14 +12,14 @@ use crate::{ Error, Timescale, coding::{DecodeError, Decoder, EncodeError, Encoder}, group, - model::object::{Encoding, Fields, Properties, Property, Value}, + model::object::{Fields, Properties, Property, Value}, }; use super::{Message, Version}; /// The largest record body a peer may make us buffer: the largest block moq-transport -/// admits, beside a Flags byte. -pub(crate) const MAX_PROPERTIES: usize = crate::ietf::object::MAX_BLOCK + 1; +/// admits. +pub(crate) const MAX_PROPERTIES: usize = crate::ietf::object::MAX_BLOCK; // Group State bits. const RELAYED: u64 = 0x1; @@ -104,42 +104,14 @@ pub(crate) fn decode_object(prior: Option, code: u64) -> Result<(u64, bool) Ok((id.ok_or(Error::ProtocolViolation)?, code & 1 != 0)) } -fn encode_encoding(encoding: Option) -> u64 { - match encoding { - None => 0, - Some(Encoding::Draft14) => 1, - Some(Encoding::Draft16) => 2, - Some(Encoding::Draft17) => 3, - Some(Encoding::Draft18) => 4, - Some(Encoding::Draft19) => 5, - } -} - -fn decode_encoding(code: u64) -> Result, DecodeError> { - Ok(match code { - 0 => None, - 1 => Some(Encoding::Draft14), - 2 => Some(Encoding::Draft16), - 3 => Some(Encoding::Draft17), - 4 => Some(Encoding::Draft18), - 5 => Some(Encoding::Draft19), - _ => return Err(DecodeError::InvalidValue), - }) -} - -/// Track Properties delta-encoded in the family they arrived in, as moq-transport sent -/// them less the properties TRACK_INFO carries, so the list is never wider than it arrived. +/// Track Properties delta-encoded as the publisher's draft encodes them, as moq-transport +/// sent them less the properties TRACK_INFO carries, so the list is never wider than it +/// arrived. /// /// An even ID has an integer value, an odd one bytes. fn encode_list(properties: &crate::model::object::TrackProperties) -> Result, EncodeError> { let mut out = Vec::new(); - let Some(encoding) = properties.encoding else { - return match properties.values.is_empty() { - true => Ok(out), - false => Err(EncodeError::InvalidState), - }; - }; - let w = &mut Encoder::new(&mut out, crate::ietf::object::family(encoding).into()); + let w = &mut Encoder::new(&mut out, properties.version.into()); let mut prior = 0; for property in &properties.values { w.varint(property.id.checked_sub(prior).ok_or(EncodeError::InvalidState)?)?; @@ -155,18 +127,22 @@ fn encode_list(properties: &crate::model::object::TrackProperties) -> Result bool { - // The Flags beside it are one byte. - encode_list(properties).is_ok_and(|list| list.len() < super::message::MAX_MESSAGE_SIZE) + let code = crate::Version::Ietf(properties.version).code(); + let Ok(code) = crate::coding::varint::size(code, Version::Lite07.into()) else { + return false; + }; + encode_list(properties).is_ok_and(|list| code + list.len() <= super::message::MAX_MESSAGE_SIZE) } -/// Decode Track Properties to the end of `r`, delta-encoded in the `encoding` family. +/// Decode Track Properties to the end of `r`, delta-encoded as `version` encodes them. /// /// No more than moq-transport ingress keeps, which is all a relay forwards. -fn decode_list(r: &mut Decoder<'_>, encoding: Encoding) -> Result, DecodeError> { - let r = &mut Decoder::new(r.rest(), crate::ietf::object::family(encoding).into()); +fn decode_list(r: &mut Decoder<'_>, version: crate::ietf::Version) -> Result, DecodeError> { + let r = &mut Decoder::new(r.rest(), version.into()); let mut values: Vec = Vec::new(); let mut prior = 0u64; while !r.is_empty() { @@ -185,19 +161,12 @@ fn decode_list(r: &mut Decoder<'_>, encoding: Encoding) -> Result, Ok(values) } -/// Encode an object's properties as a record body, empty when it has none. +/// An object's properties as a record body, empty when it has none. /// /// The block is sent as it is held. A timed track's mutable Timestamp is the frame's, so /// it is never sent here. -pub(crate) fn encode_properties(properties: &Properties, version: Version) -> Result, EncodeError> { - let mut out = Vec::new(); - let Some(block) = &properties.block else { - return Ok(out); - }; - let w = &mut Encoder::new(&mut out, version.into()); - w.varint(encode_encoding(Some(block.encoding)))?; - w.slice(&block.bytes); - Ok(out) +fn encode_properties(properties: &Properties) -> &[u8] { + properties.block.as_ref().map_or(&[], |block| &block.bytes) } /// An object's decoded properties and the Prior Object ID Gap they mark, if any. @@ -206,36 +175,29 @@ pub(crate) struct Decoded { pub gap: Option, } -/// Decode an object's record body on a track in `timescale`, the units the track -/// declares, which its sealed Timestamp is in. A `framed` object's frame holds its time; -/// a status object's block holds it. +/// Decode an object's record body on a track `framing` describes: the units its sealed +/// Timestamp is in, and the draft that serialized its block. A `framed` object's frame +/// holds its time; a status object's block holds it. /// /// Only the canonical form is accepted, so what decodes re-encodes to the same bytes. -pub(crate) fn decode_properties( - body: &[u8], - timescale: Option, - framed: bool, - version: Version, -) -> Result { +pub(crate) fn decode_properties(body: &[u8], framing: super::Framing, framed: bool) -> Result { if body.is_empty() { return Ok(Decoded { properties: Properties::default(), gap: None, }); } - let r = &mut Decoder::new(body, version.into()); - // An empty body is how nothing at all is sent. - let encoding = decode_encoding(r.varint()?)?.ok_or(DecodeError::InvalidValue)?; + // A track published over moq-lite has no moq-transport properties. + let version = framing.ietf.ok_or(DecodeError::InvalidValue)?; // A small cached block must not pin the reader's larger receive buffer. - let bytes = Bytes::copy_from_slice(&body[body.len() - r.remaining()..]); - let checked = - crate::ietf::object::check(&bytes, encoding, timescale, framed).map_err(|_| DecodeError::InvalidValue)?; + let bytes = Bytes::copy_from_slice(body); + let checked = crate::ietf::object::check(&bytes, version, framing.timescale, framed) + .map_err(|_| DecodeError::InvalidValue)?; Ok(Decoded { properties: Properties { sealed_time: checked.sealed_time, block: Some(crate::model::object::Block { bytes, - encoding, immutable: checked.immutable, }), }, @@ -243,14 +205,12 @@ pub(crate) fn decode_properties( }) } -/// Decode the END_OF_GROUP record's body on a track in `timescale`. It is a status -/// object, so a block must claim a draft that lets one carry properties. +/// Decode the END_OF_GROUP record's body on a track `framing` describes. It is a status +/// object, so a block needs a draft that lets one carry properties. #[cfg(feature = "async")] -pub(crate) fn decode_end(body: &[u8], timescale: Option, version: Version) -> Result { - let decoded = decode_properties(body, timescale, false, version)?; - if let Some(block) = &decoded.properties.block - && !crate::ietf::object::drafts(block.encoding).any(crate::ietf::object::status_has_properties) - { +pub(crate) fn decode_end(body: &[u8], framing: super::Framing) -> Result { + let decoded = decode_properties(body, framing, false)?; + if decoded.properties.block.is_some() && !framing.ietf.is_some_and(crate::ietf::object::status_has_properties) { return Err(DecodeError::InvalidValue); } Ok(decoded) @@ -266,7 +226,6 @@ pub(crate) struct Records { capped: bool, /// The units the track declares, which a sealed Timestamp must already be in. timescale: Option, - version: Version, } #[cfg(feature = "async")] @@ -282,7 +241,6 @@ impl Records { group: &group::Consumer, capped: bool, timescale: Option, - version: Version, ) -> Result, Error> { if !ietf { return Ok(None); @@ -294,7 +252,6 @@ impl Records { prior: None, capped, timescale, - version, })) } @@ -344,12 +301,12 @@ impl Records { { return Err(Error::Unsupported); } - encode_properties(&fields.properties, self.version)? + encode_properties(&fields.properties) } - None => Vec::new(), + None => &[], }; writer.buffer_varint(block.len() as u64)?; - writer.buffer_raw(&block); + writer.buffer_raw(block); Ok(()) } } @@ -361,31 +318,34 @@ pub struct TrackProperties(pub Option impl Message for TrackProperties { fn encode_msg(&self, w: &mut Encoder<'_>, _: Version) -> Result<(), EncodeError> { - // An encoding only ever describes values, so none at all is a lone 0. - let Some(properties) = self.0.as_ref().filter(|properties| !properties.values.is_empty()) else { + // A track published over moq-lite is a lone 0. + let Some(properties) = &self.0 else { return w.varint(0); }; - w.varint(encode_encoding(properties.encoding))?; + w.varint(crate::Version::Ietf(properties.version).code())?; w.slice(&encode_list(properties)?); Ok(()) } fn decode_msg(r: &mut Decoder<'_>, _: Version) -> Result { let code = r.varint()?; - if r.is_empty() { - // Only the canonical form, so what decodes re-encodes to the same bytes. - return match code { - 0 => Ok(Self(None)), - _ => Err(DecodeError::InvalidValue), + if code == 0 { + // A track published over moq-lite has no moq-transport properties. + return match r.is_empty() { + true => Ok(Self(None)), + false => Err(DecodeError::InvalidValue), }; } - // Track Properties start at draft-16, whose families all delta-encode their keys. - let encoding = decode_encoding(code)? - .filter(|encoding| *encoding != Encoding::Draft14) - .ok_or(DecodeError::InvalidValue)?; + let Some(crate::Version::Ietf(version)) = crate::Version::from_code(code) else { + return Err(DecodeError::InvalidValue); + }; + // Track Properties start at draft-16. + if matches!(version, crate::ietf::Version::Draft14 | crate::ietf::Version::Draft15) && !r.is_empty() { + return Err(DecodeError::InvalidValue); + } let properties = crate::model::object::TrackProperties { - values: decode_list(r, encoding)?, - encoding: Some(encoding), + version, + values: decode_list(r, version)?, }; // TRACK_INFO carries these, so a second copy could only contradict it. if properties @@ -405,7 +365,7 @@ impl Message for TrackProperties { return Err(DecodeError::InvalidValue); } if let Some(block) = properties.immutable() { - crate::ietf::properties::check_immutable(block, encoding)?; + crate::ietf::properties::check_immutable(block, version)?; } Ok(Self(Some(Arc::new(properties)))) } @@ -415,7 +375,6 @@ impl Message for TrackProperties { pub(crate) fn encode_datagram( w: &mut Encoder<'_>, wire: Option<&crate::model::datagram::Wire>, - version: Version, ) -> Result<(), EncodeError> { let Some(wire) = wire else { return w.varint(0); @@ -441,14 +400,13 @@ pub(crate) fn encode_datagram( if let Some(status) = wire.status { w.varint(status)?; } - w.bytes(&encode_properties(&wire.properties, version)?) + w.bytes(encode_properties(&wire.properties)) } -/// Decode a datagram's relay state on a track in `timescale`. +/// Decode a datagram's relay state on a track `framing` describes. pub(crate) fn decode_datagram( r: &mut Decoder<'_>, - timescale: Option, - version: Version, + framing: super::Framing, ) -> Result, DecodeError> { let state = r.varint()?; if state == 0 { @@ -457,13 +415,15 @@ pub(crate) fn decode_datagram( if state & RELAYED == 0 || state & !(RELAYED | OBJECT_ID | PRIORITY | DATAGRAM_END | STATUS) != 0 { return Err(DecodeError::InvalidValue); } + // A track published over moq-lite has no moq-transport state. + let version = framing.ietf.ok_or(DecodeError::InvalidValue)?; let object_id = (state & OBJECT_ID != 0).then(|| r.varint()).transpose()?; let priority = (state & PRIORITY != 0).then(|| r.u8()).transpose()?; let status = (state & STATUS != 0).then(|| r.varint()).transpose()?; // A status object has no frame to hold its time. let framed = status.is_none_or(|status| status == 0); let block = r.bytes()?; - let properties = decode_properties(block, timescale, framed, version)?.properties; + let properties = decode_properties(block, framing, framed)?.properties; let wire = crate::model::datagram::Wire { object_id, priority, @@ -471,8 +431,8 @@ pub(crate) fn decode_datagram( status, properties, }; - // A relay holds and forwards it, so it must be one moq-transport ingress would accept. - if !wire.defined() { + // A relay holds and forwards it, so it must be one its publisher's draft could send. + if !wire.defined(version) { return Err(DecodeError::InvalidValue); } Ok(Some(wire)) @@ -481,9 +441,24 @@ pub(crate) fn decode_datagram( #[cfg(test)] mod tests { use super::*; + use crate::ietf::Version as Draft; const V: Version = Version::Lite07; + /// A track in `timescale` that a `version` moq-transport publisher published. + fn from(version: Draft, timescale: Option) -> super::super::Framing { + super::super::Framing { + timescale, + ietf: Some(version), + compressed: false, + } + } + + /// The moq-transport version code TRACK_PROPERTIES carries for `version`. + fn code(version: Draft) -> u64 { + crate::Version::Ietf(version).code() + } + #[test] fn object_codes_round_trip() { for (prior, id, end) in [ @@ -530,10 +505,10 @@ mod tests { } #[test] - fn properties_round_trip_with_their_provenance() { - // An immutable block in the draft-16 family holding a gap of 2 (0x3e) and a timestamp. + fn properties_round_trip() { + // An immutable block in draft-16 holding a gap of 2 (0x3e) and a timestamp. let mut sealed = Vec::new(); - let e = &mut Encoder::new(&mut sealed, crate::ietf::Version::Draft16.into()); + let e = &mut Encoder::new(&mut sealed, Draft::Draft16.into()); for value in [0x10u64, 77, 0x3e - 0x10, 2] { e.varint(value).unwrap(); } @@ -549,51 +524,47 @@ mod tests { ]; let properties = Properties { sealed_time: Some(crate::Timestamp::new(77, Timescale::MILLI).unwrap()), - block: crate::ietf::object::block(&values, Encoding::Draft16).unwrap(), + block: crate::ietf::object::block(&values, Draft::Draft16).unwrap(), }; - let body = encode_properties(&properties, V).unwrap(); - let decoded = decode_properties(&body, Some(Timescale::MILLI), true, V).unwrap(); + let body = encode_properties(&properties); + let decoded = decode_properties(body, from(Draft::Draft16, Some(Timescale::MILLI)), true).unwrap(); assert_eq!(decoded.properties, properties); assert_eq!(decoded.gap, Some(2)); // None at all is an empty body. - assert!(encode_properties(&Properties::default(), V).unwrap().is_empty()); - let empty = decode_properties(&[], Some(Timescale::MILLI), true, V).unwrap(); + assert!(encode_properties(&Properties::default()).is_empty()); + let empty = decode_properties(&[], from(Draft::Draft16, Some(Timescale::MILLI)), true).unwrap(); assert_eq!(empty.properties, Properties::default()); // An untimed track's Timestamp has no frame to hold it, so it stays in the block. - let untimed = [4, 0x10, 7]; - let decoded = decode_properties(&untimed, None, true, V).unwrap(); - assert_eq!(encode_properties(&decoded.properties, V).unwrap(), untimed); + let untimed = [0x10, 7]; + let decoded = decode_properties(&untimed, from(Draft::Draft18, None), true).unwrap(); + assert_eq!(encode_properties(&decoded.properties), untimed); } /// Only the canonical form decodes, so whatever decodes re-encodes to the same bytes. #[test] fn properties_refuse_what_is_not_canonical() { - let delta = 4; + let timed = from(Draft::Draft18, Some(Timescale::MILLI)); for (body, reason) in [ - (&[0][..], "an encoding saying nothing"), - (&[delta][..], "an encoding with an empty block"), - (&[6, 0x20, 1][..], "an unknown encoding"), ( - &[delta, 0x10, 5][..], + &[0x10, 5][..], "a timed track's mutable Timestamp, which its frame holds", ), - (&[0, 0x20, 1][..], "a block without an encoding"), - (&[delta, 0x0b, 2, 0x0b, 0][..], "a nested Immutable Properties"), + (&[0x0b, 2, 0x0b, 0][..], "a nested Immutable Properties"), ] { - assert!( - decode_properties(body, Some(Timescale::MILLI), true, V).is_err(), - "accepted {reason}" - ); + assert!(decode_properties(body, timed, true).is_err(), "accepted {reason}"); } + // A track published over moq-lite has no moq-transport properties. + let lite = super::super::Framing { ietf: None, ..timed }; + assert!(decode_properties(&[0x20, 1], lite, true).is_err()); } /// The worst case for re-encoding, a full moq-transport list of zero-delta properties /// after a large first ID, crosses the hop byte for byte and is held at its wire size. #[test] fn a_full_block_crosses_at_its_wire_size() { - for version in [crate::ietf::Version::Draft16, crate::ietf::Version::Draft22] { + for version in [Draft::Draft16, Draft::Draft22] { let mut list = Vec::new(); let e = &mut Encoder::new(&mut list, version.into()); // A Timestamp, then IDs from 2^40 on, each repeated as a two-byte zero delta. @@ -610,21 +581,35 @@ mod tests { let properties = decoded.properties; let block = properties.block.as_ref().unwrap(); assert!(block.bytes.len() < list.len(), "{version}"); - assert_eq!(properties.heap(), block.bytes.len() as u64, "{version}: decoded state"); + assert_eq!(properties.heap(), block.bytes.len() as u64, "{version}"); - let body = encode_properties(&properties, V).unwrap(); + let body = encode_properties(&properties); assert!(body.len() <= MAX_PROPERTIES, "{version}: {} bytes", body.len()); - let received = decode_properties(&body, Some(Timescale::MILLI), true, V).unwrap(); + let received = decode_properties(body, from(version, Some(Timescale::MILLI)), true).unwrap(); assert_eq!(received.properties, properties, "{version}"); - let written = - crate::ietf::object::write(&received.properties, frame, version, Some(Timescale::MILLI)).unwrap(); + let written = crate::ietf::object::write( + &received.properties, + Some(version), + frame, + version, + Some(Timescale::MILLI), + ) + .unwrap(); assert_eq!(written, list, "{version}: the hop changed the list"); } } + fn round_trip(msg: &TrackProperties) -> Result { + let buf = crate::coding::Encode::encode_bytes(msg, V).unwrap(); + >::decode(&mut Decoder::new(&buf, V.into()), V) + } + + /// A moq-transport track carries its publisher's draft once, with or without + /// properties, and a moq-lite track carries none. #[test] fn track_properties_round_trip() { let properties = crate::model::object::TrackProperties { + version: Draft::Draft19, values: vec![ Property { id: IMMUTABLE, @@ -635,22 +620,28 @@ mod tests { value: Value::Bytes(Bytes::from_static(b"opaque")), }, ], - encoding: Some(Encoding::Draft19), }; - let msg = TrackProperties(Some(Arc::new(properties.clone()))); - let buf = crate::coding::Encode::encode_bytes(&msg, V).unwrap(); - let decoded = - >::decode(&mut Decoder::new(&buf, V.into()), V).unwrap(); + let decoded = round_trip(&TrackProperties(Some(Arc::new(properties.clone())))).unwrap(); assert_eq!(decoded.0.as_deref(), Some(&properties)); + // Drafts 14 and 15 have no Track Properties, but their objects may have properties. + for version in [Draft::Draft14, Draft::Draft22] { + let bare = crate::model::object::TrackProperties { + version, + values: Vec::new(), + }; + let decoded = round_trip(&TrackProperties(Some(Arc::new(bare.clone())))).unwrap(); + assert_eq!(decoded.0.as_deref(), Some(&bare), "{version}"); + } + let none = crate::coding::Encode::encode_bytes(&TrackProperties(None), V).unwrap(); - let decoded = - >::decode(&mut Decoder::new(&none, V.into()), V).unwrap(); - assert!(decoded.0.is_none()); + assert_eq!(&none[1..], [0], "a moq-lite track is a lone 0"); + assert!(round_trip(&TrackProperties(None)).unwrap().0.is_none()); } /// A peer cannot make a relay hold or forward more than moq-transport ingress would, - /// restate what TRACK_INFO carries, or claim sealed keys its block does not hold. + /// restate what TRACK_INFO carries, claim a draft it could not have come from, or + /// claim sealed keys its block does not hold. #[test] fn track_properties_refuse_what_ingress_would_not_keep() { let wire = |values: &[Property], code: u64| { @@ -658,8 +649,8 @@ mod tests { let w = &mut Encoder::new(&mut buf, V.into()); w.varint(code).unwrap(); let list = encode_list(&crate::model::object::TrackProperties { + version: Draft::Draft18, values: values.to_vec(), - encoding: Some(Encoding::Draft18), }) .unwrap(); w.slice(&list); @@ -670,29 +661,44 @@ mod tests { id, value: Value::Int(1), }; + let draft18 = code(Draft::Draft18); let max = crate::ietf::properties::MAX_PROPERTIES; let list: Vec<_> = (0..=max).map(|i| opaque(0x40 + 2 * i)).collect(); - assert!(decode(&wire(&list[..max as usize], 4)).is_ok()); - assert!(decode(&wire(&list, 4)).is_err(), "more than ingress keeps"); + assert!(decode(&wire(&list[..max as usize], draft18)).is_ok()); + assert!(decode(&wire(&list, draft18)).is_err(), "more than ingress keeps"); // TIMESCALE belongs to TRACK_INFO. - assert!(decode(&wire(&[opaque(0x08)], 4)).is_err(), "an understood property"); - assert!(decode(&wire(&[opaque(0x40)], 0)).is_err(), "a list needs its encoding"); - assert!(decode(&[2]).is_err(), "an encoding without a list"); assert!( - decode(&wire(&[opaque(0x40)], 1)).is_err(), - "track properties never use absolute keys" + decode(&wire(&[opaque(0x08)], draft18)).is_err(), + "an understood property" ); + assert!( + decode(&wire(&[opaque(0x40)], 0)).is_err(), + "properties on a moq-lite track" + ); + for version in [Draft::Draft14, Draft::Draft15] { + assert!(decode(&wire(&[], code(version))).is_ok(), "{version}"); + assert!( + decode(&wire(&[opaque(0x40)], code(version))).is_err(), + "{version} has no Track Properties" + ); + } + for code in [1, 0xff000099, crate::Version::Lite(V).code()] { + assert!(decode(&wire(&[], code)).is_err(), "{code:#x} is no moq-transport draft"); + } // The sealed keys come from the block, never the sender. let sealed = Property { id: IMMUTABLE, value: Value::Bytes(Bytes::from_static(&[0x20, 0x01, 0x04, 0x05])), }; - let decoded = decode(&wire(std::slice::from_ref(&sealed), 4)).unwrap().0.unwrap(); + let decoded = decode(&wire(std::slice::from_ref(&sealed), draft18)) + .unwrap() + .0 + .unwrap(); assert!(decoded.sealed().eq([0x20, 0x24])); assert!( - decode(&wire(&[sealed.clone(), sealed], 4)).is_err(), + decode(&wire(&[sealed.clone(), sealed], draft18)).is_err(), "a second block leaves the sealed keys ambiguous" ); } @@ -706,17 +712,13 @@ mod tests { let mut sealed = vec![0x20, 1]; sealed.extend([0, 1].repeat(count - 1)); let msg = TrackProperties(Some(Arc::new(crate::model::object::TrackProperties { + version: Draft::Draft18, values: vec![Property { id: IMMUTABLE, value: Value::Bytes(sealed.into()), }], - encoding: Some(Encoding::Draft18), }))); - let buf = crate::coding::Encode::encode_bytes(&msg, V).unwrap(); - let decoded = >::decode(&mut Decoder::new(&buf, V.into()), V) - .unwrap() - .0 - .unwrap(); + let decoded = round_trip(&msg).unwrap().0.unwrap(); assert_eq!(decoded.values.len(), 1); assert_eq!(decoded.sealed().count(), count); assert!(decoded.sealed().all(|id| id == 0x20)); @@ -727,13 +729,9 @@ mod tests { /// message would no longer fit a lite one. #[test] fn track_properties_cross_at_their_wire_size() { - for version in [ - crate::ietf::Version::Draft16, - crate::ietf::Version::Draft17, - crate::ietf::Version::Draft22, - ] { + for version in [Draft::Draft16, Draft::Draft17, Draft::Draft22] { // As many repeats of an ID from 2^60 as moq-transport forwards, then one value - // that fills the rest of a lite message beside its Flags. + // that fills the rest of a lite message beside its Version. let id = (1 << 60) + 1; let repeats = crate::ietf::properties::MAX_FORWARDED - 1; let mut list = Vec::new(); @@ -744,7 +742,9 @@ mod tests { e.varint(0).unwrap(); e.bytes(&[]).unwrap(); } - let rest = super::super::message::MAX_MESSAGE_SIZE - 1 - list.len() - 1; + let prefix = crate::coding::varint::size(code(version), V.into()).unwrap(); + let room = super::super::message::MAX_MESSAGE_SIZE - prefix; + let rest = room - list.len() - 1; let fill = (1..=8) .map(|prefix| rest - prefix) .find(|fill| { @@ -754,31 +754,36 @@ mod tests { let e = &mut Encoder::new(&mut list, version.into()); e.varint(0).unwrap(); e.bytes(&vec![0; fill]).unwrap(); - assert_eq!(list.len(), super::super::message::MAX_MESSAGE_SIZE - 1, "{version}"); + assert_eq!(list.len(), room, "{version}"); let ingress = crate::ietf::Properties::decode(&mut Decoder::new(&list, version.into()), version).unwrap(); - assert_eq!(ingress.unknown.values.len(), repeats + 1, "{version}"); - assert!( - ingress.unknown.refused(&crate::track::Info::default()).is_none(), - "{version}" - ); + assert_eq!(ingress.unknown.len(), repeats + 1, "{version}"); + let forwarded = crate::model::object::TrackProperties { + version, + values: ingress.unknown.clone(), + }; + assert!(forwarded.refused(&crate::track::Info::default()).is_none(), "{version}"); - let msg = TrackProperties(Some(Arc::new(ingress.unknown.clone()))); - let buf = crate::coding::Encode::encode_bytes(&msg, V).unwrap(); - let decoded = - >::decode(&mut Decoder::new(&buf, V.into()), V) - .unwrap(); - assert_eq!(decoded.0.as_deref(), Some(&ingress.unknown), "{version}"); + let decoded = round_trip(&TrackProperties(Some(Arc::new(forwarded.clone())))).unwrap(); + assert_eq!(decoded.0.as_deref(), Some(&forwarded), "{version}"); let mut written = Vec::new(); let egress = crate::ietf::Properties { - unknown: decoded.0.unwrap().as_ref().clone(), + unknown: decoded.0.unwrap().values.clone(), ..Default::default() }; egress .encode(&mut Encoder::new(&mut written, version.into()), version) .unwrap(); assert_eq!(written, list, "{version}: the hop changed the list"); + + // A byte more would not fit beside the Version. + let mut wider = forwarded; + let Value::Bytes(last) = &mut wider.values.last_mut().unwrap().value else { + unreachable!() + }; + *last = vec![0; fill + 1].into(); + assert!(!fits(&wider), "{version}"); } } @@ -786,7 +791,7 @@ mod tests { /// could not cross is refused where it arrives rather than where it would be sent. #[test] fn track_properties_that_cannot_cross_are_refused_at_ingress() { - let version = crate::ietf::Version::Draft16; + let version = Draft::Draft16; // The most a draft-16 answer holds beside a Request ID and an empty parameter count. let size = super::super::message::MAX_MESSAGE_SIZE - 2; // DYNAMIC_GROUPS, then the last ID whose delta from it fits four bytes. @@ -804,28 +809,40 @@ mod tests { // The delta from zero takes eight bytes once DYNAMIC_GROUPS leaves the list. let ingress = crate::ietf::Properties::decode(&mut Decoder::new(&list, version.into()), version).unwrap(); assert!(ingress.dynamic_groups); - assert!(!fits(&ingress.unknown)); + let forwarded = crate::model::object::TrackProperties { + version, + values: ingress.unknown, + }; + assert!(!fits(&forwarded)); assert!(matches!( - ingress.unknown.refused(&crate::track::Info::default()), + forwarded.refused(&crate::track::Info::default()), Some(crate::Error::Unsupported) )); } + fn datagram(wire: &crate::model::datagram::Wire) -> Vec { + let mut buf = Vec::new(); + encode_datagram(&mut Encoder::new(&mut buf, V.into()), Some(wire)).unwrap(); + buf + } + /// A relay holds and forwards a datagram's Status, so one moq-transport does not define /// would close the session it reached: it is refused where it arrives. #[test] fn datagram_statuses_are_ones_moq_transport_defines() { for (status, ok) in [(0, true), (1, true), (2, false), (3, true), (4, true), (5, false)] { + // Draft 14 sends a status datagram's Object ID and priority. let wire = crate::model::datagram::Wire { - object_id: None, - priority: None, + object_id: Some(3), + priority: Some(9), end_of_group: false, status: Some(status), properties: Properties::default(), }; - let mut buf = Vec::new(); - encode_datagram(&mut Encoder::new(&mut buf, V.into()), Some(&wire), V).unwrap(); - let decoded = decode_datagram(&mut Decoder::new(&buf, V.into()), None, V); + let decoded = decode_datagram( + &mut Decoder::new(&datagram(&wire), V.into()), + from(Draft::Draft14, None), + ); assert_eq!(decoded.is_ok(), ok, "{status}"); } } @@ -841,9 +858,13 @@ mod tests { status: Some(3), properties: Properties::default(), }; - let mut buf = Vec::new(); - encode_datagram(&mut Encoder::new(&mut buf, V.into()), Some(&wire), V).unwrap(); - assert!(decode_datagram(&mut Decoder::new(&buf, V.into()), None, V).is_err()); + assert!( + decode_datagram( + &mut Decoder::new(&datagram(&wire), V.into()), + from(Draft::Draft22, None) + ) + .is_err() + ); } #[test] @@ -856,19 +877,60 @@ mod tests { status, properties: Properties::default(), }; - let mut buf = Vec::new(); - encode_datagram(&mut Encoder::new(&mut buf, V.into()), Some(&wire), V).unwrap(); - let decoded = decode_datagram(&mut Decoder::new(&buf, V.into()), None, V).unwrap(); + let decoded = decode_datagram( + &mut Decoder::new(&datagram(&wire), V.into()), + from(Draft::Draft22, None), + ) + .unwrap(); assert_eq!(decoded, Some(wire)); } let mut plain = Vec::new(); - encode_datagram(&mut Encoder::new(&mut plain, V.into()), None, V).unwrap(); + encode_datagram(&mut Encoder::new(&mut plain, V.into()), None).unwrap(); assert_eq!(plain, [0]); - assert_eq!( - decode_datagram(&mut Decoder::new(&plain, V.into()), None, V).unwrap(), - None - ); + // A plain datagram crosses whether or not its track came from moq-transport. + for framing in [from(Draft::Draft22, None), Default::default()] { + assert_eq!( + decode_datagram(&mut Decoder::new(&plain, V.into()), framing).unwrap(), + None + ); + } + } + + /// A track published over moq-lite has no moq-transport state, so a datagram on one + /// with any is refused. + #[test] + fn datagram_state_needs_a_moq_transport_track() { + let id = crate::model::datagram::Wire { + object_id: Some(3), + priority: None, + end_of_group: false, + status: None, + properties: Properties::default(), + }; + let properties = crate::model::datagram::Wire { + object_id: None, + properties: Properties { + block: crate::ietf::object::block( + &[Property { + id: 0x20, + value: Value::Int(1), + }], + Draft::Draft22, + ) + .unwrap(), + ..Default::default() + }, + ..id.clone() + }; + for wire in [id, properties] { + let buf = datagram(&wire); + assert!(decode_datagram(&mut Decoder::new(&buf, V.into()), from(Draft::Draft22, None)).is_ok()); + assert!( + decode_datagram(&mut Decoder::new(&buf, V.into()), Default::default()).is_err(), + "{wire:?}" + ); + } } /// A retained datagram holds its properties as the bytes they arrived as, not a decoded @@ -880,74 +942,73 @@ mod tests { for _ in 1..COUNT { block.extend([0, 1]); } - let mut body = vec![4]; - body.extend(&block); let mut buf = Vec::new(); let w = &mut Encoder::new(&mut buf, V.into()); w.varint(RELAYED).unwrap(); - w.bytes(&body).unwrap(); - let wire = decode_datagram(&mut Decoder::new(&buf, V.into()), None, V) + w.bytes(&block).unwrap(); + let wire = decode_datagram(&mut Decoder::new(&buf, V.into()), from(Draft::Draft18, None)) .unwrap() .unwrap(); assert_eq!(wire.properties.heap(), block.len() as u64); assert!(wire.properties.heap() < (COUNT * size_of::()) as u64 / 10); } - /// A body holding property 0x20 = 1, which every family serializes alike, claiming the - /// encoding `code`. - fn claiming(code: u8) -> [u8; 3] { - [code, 0x20, 1] - } - - /// Each code names the drafts a block came from, so whether 0x06 means - /// SUBGROUP_DELIVERY_TIMEOUT follows from it: only for code 5, drafts 19 and later. + /// Drafts 18 and 19 serialize a block alike, so only the track's draft says whether 0x06 + /// is SUBGROUP_DELIVERY_TIMEOUT: a draft-18 one is dropped on its way to draft-19. #[test] - fn encoding_codes_name_their_drafts() { - for code in 1..=5 { - let decoded = decode_properties(&claiming(code), None, true, V).unwrap(); - let encoding = decoded.properties.block.unwrap().encoding; - assert_eq!(encode_encoding(Some(encoding)), u64::from(code)); - assert_eq!(crate::ietf::object::delivery_timeout(encoding), code == 5, "{code}"); - let list = TrackProperties::decode_msg(&mut Decoder::new(&claiming(code), V.into()), V); - // Track Properties start at draft-16. - assert_eq!(list.is_ok(), code != 1, "{code}"); - } - assert!(decode_properties(&claiming(6), None, true, V).is_err()); - - // A draft-18 0x06 is dropped on its way to draft-19, where it would mean a timeout. - for (code, kept) in [(4, false), (5, true)] { - let decoded = decode_properties(&[code, 0x06, 42], None, true, V).unwrap(); + fn a_block_means_what_its_tracks_draft_says() { + for (version, kept) in [(Draft::Draft18, false), (Draft::Draft19, true)] { + let decoded = decode_properties(&[0x06, 42], from(version, None), true).unwrap(); let written = - crate::ietf::object::write(&decoded.properties, None, crate::ietf::Version::Draft19, None).unwrap(); - assert_eq!(!written.is_empty(), kept, "{code}"); + crate::ietf::object::write(&decoded.properties, Some(version), None, Draft::Draft19, None).unwrap(); + assert_eq!(!written.is_empty(), kept, "{version}"); } } - /// Only draft-14 lets a status object carry properties, so the END_OF_GROUP record's - /// block must claim the drafts 14 and 15 encoding. + /// Only draft-14 lets a status object carry properties, so the END_OF_GROUP record has a + /// block only on a draft-14 track. #[cfg(feature = "async")] #[test] - fn an_end_record_block_claims_a_draft_that_carries_one() { - for code in 1..=5 { - assert_eq!(decode_end(&claiming(code), None, V).is_ok(), code == 1, "{code}"); + fn an_end_record_block_needs_a_draft_that_carries_one() { + for version in [Draft::Draft14, Draft::Draft15, Draft::Draft16, Draft::Draft22] { + let framing = from(version, None); + assert_eq!( + decode_end(&[0x20, 1], framing).is_ok(), + version == Draft::Draft14, + "{version}" + ); + assert!(decode_end(&[], framing).is_ok(), "{version}"); } - assert!(decode_end(&[], None, V).is_ok()); } - /// A datagram's block names its family, so one of those drafts must define the whole - /// datagram: from draft-17 on a non-zero Status carries no properties, and only drafts - /// 14 and 15 define DOES_NOT_EXIST. + /// A datagram must be one its track's draft defines: from draft-17 on a non-zero Status + /// carries no properties, and only drafts 14 and 15 define DOES_NOT_EXIST. #[test] - fn a_datagram_block_claims_a_draft_that_sends_its_shape() { - for (status, codes) in [(0, &[1, 2, 3, 4, 5][..]), (1, &[1]), (3, &[1, 2]), (4, &[1, 2])] { - for code in 1..=5 { + fn a_datagram_block_needs_a_draft_that_sends_its_shape() { + let drafts = [ + Draft::Draft14, + Draft::Draft15, + Draft::Draft16, + Draft::Draft17, + Draft::Draft22, + ]; + for (status, carried) in [ + (0, &drafts[..]), + (1, &drafts[..2]), + (3, &drafts[..3]), + (4, &drafts[..3]), + ] { + for version in drafts { + // Object 3 at priority 9, which every draft can send beside a status. let mut buf = Vec::new(); let w = &mut Encoder::new(&mut buf, V.into()); - w.varint(RELAYED | STATUS).unwrap(); + w.varint(RELAYED | OBJECT_ID | PRIORITY | STATUS).unwrap(); + w.varint(3).unwrap(); + w.u8(9); w.varint(status).unwrap(); - w.bytes(&claiming(code)).unwrap(); - let decoded = decode_datagram(&mut Decoder::new(&buf, V.into()), None, V); - assert_eq!(decoded.is_ok(), codes.contains(&code), "{status}: {code}"); + w.bytes(&[0x20, 1]).unwrap(); + let decoded = decode_datagram(&mut Decoder::new(&buf, V.into()), from(version, None)); + assert_eq!(decoded.is_ok(), carried.contains(&version), "{status}: {version}"); } } } @@ -1024,7 +1085,15 @@ mod hop_tests { /// The block a draft-22 publisher's `values` arrive as. fn block(values: &[Property]) -> Option { - crate::ietf::object::block(values, Encoding::Draft19).unwrap() + crate::ietf::object::block(values, crate::ietf::Version::Draft22).unwrap() + } + + /// A draft-22 publisher's track with `values`. + fn draft22(values: Vec) -> Option> { + Some(Arc::new(crate::model::object::TrackProperties { + version: crate::ietf::Version::Draft22, + values, + })) } /// Write a frame carrying relay `fields`, as moq-transport ingress does, returning the @@ -1102,11 +1171,8 @@ mod hop_tests { info.max_age = Some(Duration::from_secs(7)); info.priority = 9; info.refresh = Some(0); - let track_properties = crate::model::object::TrackProperties { - values: vec![property(0x40, 3)], - encoding: Some(Encoding::Draft19), - }; - info.properties = Some(Arc::new(track_properties.clone())); + let track_properties = draft22(vec![property(0x40, 3)]); + info.properties = track_properties.clone(); let track = broadcast.create_track("video", info).unwrap(); let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); @@ -1185,7 +1251,7 @@ mod hop_tests { assert_eq!(fields(&frame), Some((second.id, second.properties.clone()))); assert_eq!(received.end_object().map(|end| end.id), Some(10)); assert!(received.wire().unwrap().fetchable, "every jump was marked"); - assert_eq!(info.properties.as_deref(), Some(&track_properties)); + assert_eq!(info.properties, track_properties); } false => { assert!(frame.fields.is_none()); @@ -1202,9 +1268,9 @@ mod hop_tests { async fn a_capped_stream_carries_no_end_record() { let upstream = crate::origin::Config::new(crate::Hop::new(2).unwrap()).produce(); let broadcast = upstream.publish("room", crate::origin::Route::default()).unwrap(); - let track = broadcast - .create_track("video", track::Info::timed().with_timescale(Timescale::MILLI)) - .unwrap(); + let mut info = track::Info::timed().with_timescale(Timescale::MILLI); + info.properties = draft22(Vec::new()); + let track = broadcast.create_track("video", info).unwrap(); let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); group .set_wire(group::Wire { @@ -1251,10 +1317,11 @@ mod hop_tests { assert!(received.end_object().is_none(), "an END_OF_GROUP past the cap"); } - /// Subscribe across a negotiated hop to a timed track carrying `properties`. - async fn subscribe_across(properties: crate::model::object::TrackProperties) -> Result<(), crate::Error> { + /// Subscribe across a negotiated hop to a timed track a draft-22 publisher published + /// with `values`. + async fn subscribe_across(values: Vec) -> Result<(), crate::Error> { let mut info = track::Info::timed().with_timescale(Timescale::MILLI); - info.properties = Some(Arc::new(properties)); + info.properties = draft22(values); subscribe_across_info(info).await } @@ -1291,11 +1358,7 @@ mod hop_tests { value: Value::Bytes(sealed.into()), }], ] { - let res = subscribe_across(crate::model::object::TrackProperties { - values: values.clone(), - encoding: Some(Encoding::Draft19), - }) - .await; + let res = subscribe_across(values.clone()).await; assert!( matches!(res, Err(crate::Error::UnsupportedExtension)), "{values:?}: {res:?}" @@ -1312,13 +1375,10 @@ mod hop_tests { let e = &mut Encoder::new(&mut sealed, crate::ietf::Version::Draft22.into()); e.varint(0x08).unwrap(); e.varint(timescale).unwrap(); - let res = subscribe_across(crate::model::object::TrackProperties { - values: vec![Property { - id: IMMUTABLE, - value: Value::Bytes(sealed.into()), - }], - encoding: Some(Encoding::Draft19), - }) + let res = subscribe_across(vec![Property { + id: IMMUTABLE, + value: Value::Bytes(sealed.into()), + }]) .await; match ok { true => assert!(res.is_ok(), "{timescale}: {res:?}"), @@ -1343,13 +1403,10 @@ mod hop_tests { e.varint(value).unwrap(); let mut info = track::Info::timed().with_timescale(Timescale::MILLI); info.refresh = refresh; - info.properties = Some(Arc::new(crate::model::object::TrackProperties { - values: vec![Property { - id: IMMUTABLE, - value: Value::Bytes(sealed.into()), - }], - encoding: Some(Encoding::Draft19), - })); + info.properties = draft22(vec![Property { + id: IMMUTABLE, + value: Value::Bytes(sealed.into()), + }]); let res = subscribe_across_info(info).await; match ok { true => assert!(res.is_ok(), "{refresh:?} {value}: {res:?}"), @@ -1367,11 +1424,7 @@ mod hop_tests { async fn forwarded_track_properties_leave_room_for_track_info() { let max = crate::ietf::properties::MAX_FORWARDED; for (count, ok) in [(max, true), (max + 1, false)] { - let res = subscribe_across(crate::model::object::TrackProperties { - values: (0..count as u64).map(|i| property(0x40 + 2 * i, 1)).collect(), - encoding: Some(Encoding::Draft19), - }) - .await; + let res = subscribe_across((0..count as u64).map(|i| property(0x40 + 2 * i, 1)).collect()).await; match ok { true => assert!(res.is_ok(), "{count}: {res:?}"), false => assert!(matches!(res, Err(crate::Error::Unsupported)), "{count}: {res:?}"), @@ -1380,7 +1433,7 @@ mod hop_tests { } /// Groups and datagrams from a moq-lite publisher cross a negotiated hop as plain as - /// they left: no relay state appears on the way. + /// they left: no relay state appears on the way, nor a moq-transport draft. #[moq_clock::test] async fn plain_media_crosses_unchanged() { let upstream = crate::origin::Config::new(crate::Hop::new(2).unwrap()).produce(); @@ -1389,7 +1442,9 @@ mod hop_tests { .create_track("video", track::Info::timed().with_timescale(Timescale::MILLI)) .unwrap(); let (downstream, sessions) = test_hop(&upstream, "room", [true, true]).await; - let (_copy, mut subscriber) = subscribe(&downstream).await; + let (copy, mut subscriber) = subscribe(&downstream).await; + let info = moq_clock::timeout(TIMEOUT, copy.query()).await.unwrap().unwrap(); + assert!(info.properties.is_none()); let mut group = track.append_group().unwrap(); group.write_frame(ts(5), b"key".to_vec()).unwrap(); diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index ca4117a855f..64b5bbedf40 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -2321,7 +2321,7 @@ impl Request for FetchServe { group = group.compressed(); } let capped = msg.end_frame.is_some(); - let records = lite::ietf::Records::start(writer, ietf, &group, capped, timescale, shared.version)?; + let records = lite::ietf::Records::start(writer, ietf, &group, capped, timescale)?; *self = Self::Serve { group, @@ -4371,14 +4371,7 @@ impl GroupServe { }; let header = writer.buffer(&lite::DataType::Group).and_then(|()| writer.buffer(&msg)); let records = header.and_then(|()| { - lite::ietf::Records::start( - &mut writer, - self.ctx.ietf, - &self.group, - self.capped, - self.ctx.timescale, - self.ctx.version, - ) + lite::ietf::Records::start(&mut writer, self.ctx.ietf, &self.group, self.capped, self.ctx.timescale) }); let records = match records { Ok(records) => records, diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 93fed76d213..80d99714a69 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -7,7 +7,7 @@ use std::{ }; use crate::{ - AsPath, Error, Path, PathOwned, Timescale, Timestamp, bandwidth, + AsPath, Error, Path, PathOwned, Timestamp, bandwidth, coding::{Reader, Stream}, lite, track::{Position, Subscription}, @@ -93,9 +93,9 @@ pub(super) struct Subscriber { #[derive(Clone)] struct TrackEntry { producer: track::Producer, - /// Timestamp scale from this track's TRACK_INFO, known before the SUBSCRIBE is - /// even opened, so group streams decode frames without blocking. - timescale: Option, + /// How to read the track, from its TRACK_INFO, known before the SUBSCRIBE is even + /// opened, so group streams decode frames without blocking. + framing: lite::Framing, /// The groups received so far, so the subscription's end can wait for the ones owed. tail: kio::Producer, } @@ -511,12 +511,8 @@ impl Subscriber { }; let mut subscribes = self.subscribes.lock(); // An unknown subscription's datagram is dropped below, however it parses. - let dg = lite::Datagram::decode(payload, self.version, ietf, |id| match subscribes.get(&id) { - Some(entry) => lite::Format { - timescale: entry.timescale, - compressed: entry.producer.info().compress, - }, - None => lite::Format::default(), + let dg = lite::Datagram::decode(payload, self.version, ietf, |id| { + subscribes.get(&id).map(|entry| entry.framing).unwrap_or_default() })?; let Some(entry) = subscribes.get_mut(&dg.subscribe) else { // Unknown or already-closed subscription: drop the datagram. @@ -525,7 +521,7 @@ impl Subscriber { let timestamp = dg .timestamp - .zip(entry.timescale) + .zip(entry.framing.timescale) .map(|(value, scale)| Timestamp::new(value, scale)) .transpose() .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; @@ -890,7 +886,7 @@ impl GroupRecv { let mut cx = waiter.context(); let hdr = ready!(self.reader.poll_decode::(&mut cx))?; - let (group, track, timescale, reading) = { + let (group, track, framing, reading) = { let mut subs = self.subscriber.subscribes.lock(); let entry = subs.get_mut(&hdr.subscribe).ok_or(Error::Cancel)?; @@ -938,7 +934,7 @@ impl GroupRecv { // from where the publisher said they start so a front continuing the // group across routes lines them up. group.start_at(hdr.frame_start)?; - (group, entry.producer.clone(), entry.timescale, reading) + (group, entry.producer.clone(), entry.framing, reading) }; // The timescale came from TRACK_INFO (read before this subscription was @@ -947,7 +943,7 @@ impl GroupRecv { self.state = GroupRecvState::Serve { group: crate::recv::Group::new(group), track, - ingest: FrameIngest::new(&self.subscriber, timescale, ietf, hdr.frame_start), + ingest: FrameIngest::new(&self.subscriber, framing, ietf, hdr.frame_start), shown: false, _reading: reading, }; @@ -1023,15 +1019,14 @@ impl GroupRecv { /// Pumps bare FRAME messages from a reader into a group producer: the wire /// format shared by GROUP streams and FETCH responses. struct FrameIngest { - /// `Some` decodes the lite-05 zigzag-delta timestamp prefix; `None` leaves frames + /// A timescale decodes the lite-05 zigzag-delta timestamp prefix; none leaves frames /// untimed (pre-lite-05). - timescale: Option, + framing: lite::Framing, /// Previous frame's raw timestamp value (in `timescale` units), for the /// zigzag-delta decode. The first frame's delta is absolute (prev = 0). prev_ts: u64, phase: IngestPhase, budget: frame::Budget, - version: Version, /// Where the stream's frames start, the first Object ID until a record names it. frame_start: u64, /// `Some` once the Group State of the `ietf` extension arrived: every frame follows an @@ -1084,19 +1079,18 @@ impl FrameIngest { /// `ietf` when the session negotiated the extension, so the stream starts with a Group State. fn new( subscriber: &Subscriber, - timescale: Option, + framing: lite::Framing, ietf: bool, frame_start: u64, ) -> Self { Self { - timescale, + framing, prev_ts: 0, phase: match ietf { true => IngestPhase::State, false => IngestPhase::Timing, }, budget: subscriber.frames.clone(), - version: subscriber.version, frame_start, records: None, } @@ -1138,6 +1132,10 @@ impl FrameIngest { let state = ready!(reader.poll_varint(&mut cx))?; let wire = lite::ietf::decode_state(state, self.frame_start)?; let relayed = wire.is_some(); + // A track published over moq-lite has no moq-transport state. + if relayed && self.framing.ietf.is_none() { + return Poll::Ready(Err(Error::ProtocolViolation)); + } let extensions = wire.as_ref().is_some_and(|wire| wire.extensions); if let Some(wire) = wire { group.set_wire(wire)?; @@ -1183,7 +1181,7 @@ impl FrameIngest { return Poll::Ready(Err(Error::ProtocolViolation)); } if end { - let decoded = lite::ietf::decode_end(&block, self.timescale, self.version)?; + let decoded = lite::ietf::decode_end(&block, self.framing)?; group.mark_gap(id, decoded.gap)?; group.set_end_object(crate::model::object::Fields { id, @@ -1202,7 +1200,7 @@ impl FrameIngest { return Poll::Ready(Ok(())); } IngestPhase::Timing => { - let Some(scale) = self.timescale else { + let Some(scale) = self.framing.timescale else { self.phase = IngestPhase::Inflated { timestamp: None }; continue; }; @@ -1248,7 +1246,7 @@ impl FrameIngest { Some(records) => { let (id, block) = records.pending.take().expect("a record precedes each frame"); let lite::ietf::Decoded { properties, gap } = - lite::ietf::decode_properties(&block, self.timescale, true, self.version)?; + lite::ietf::decode_properties(&block, self.framing, true)?; group.mark_gap(id, gap)?; let plain = id == group.frame_count() as u64 && properties.is_empty(); if !plain && !records.relayed { @@ -1981,6 +1979,7 @@ impl kio::Task for SourceServe { #[cfg(test)] mod tests { use super::*; + use crate::Timescale; use crate::coding::{Decode, Encode}; use crate::lite::test_transport::SinkSession; use crate::model::ProduceTest; @@ -2127,13 +2126,11 @@ mod tests { }); if update { h.serve - .handle_subscription(&mut track, &mut sub, lowered, true, None) + .handle_subscription(&mut track, &mut sub, lowered, true) .await .unwrap(); } else { - let begin = h - .serve - .begin_subscription(&mut track, &mut Sub::None, lowered, true, None); + let begin = h.serve.begin_subscription(&mut track, &mut Sub::None, lowered, true); assert!(matches!(begin, Ok(Begin::Establish(_)))); } @@ -2614,7 +2611,11 @@ mod tests { 7, TrackEntry { producer: track, - timescale: Some(Timescale::MILLI), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, tail: Default::default(), }, ); @@ -2670,7 +2671,11 @@ mod tests { 7, TrackEntry { producer: track.clone(), - timescale: Some(Timescale::MILLI), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, tail: Default::default(), }, ); @@ -2694,11 +2699,20 @@ mod tests { } } - /// Receive group 0 of a track in `timescale` over a lite-07 session that negotiated the - /// `ietf` extension, its stream holding `body` after the GROUP header. Also returns the - /// cache pool the track charges. + /// A track in `timescale` that a draft-18 moq-transport publisher published. + fn draft18(timescale: Option) -> lite::Framing { + lite::Framing { + timescale, + ietf: Some(crate::ietf::Version::Draft18), + compressed: false, + } + } + + /// Receive group 0 of a track `framing` describes over a lite-07 session that negotiated + /// the `ietf` extension, its stream holding `body` after the GROUP header. Also returns + /// the cache pool the track charges. async fn receive_ietf( - timescale: Option, + framing: lite::Framing, body: &[u64], ) -> (Result<(), Error>, track::Producer, crate::cache::Pool) { use crate::transport::poll::Session as _; @@ -2735,19 +2749,26 @@ mod tests { auth: crate::auth::Handle::new(false), }); let pool = crate::cache::Pool::unbounded(); + let mut info = crate::track::Info::default().with_timescale(framing.timescale); + info.properties = framing.ietf.map(|version| { + std::sync::Arc::new(crate::model::object::TrackProperties { + version, + values: Vec::new(), + }) + }); let track = track::Producer::new( std::sync::Arc::new(crate::broadcast::Info { pool: pool.clone(), ..Default::default() }), "video", - crate::track::Info::default().with_timescale(timescale), + info, ); subscriber.subscribes.lock().insert( 7, TrackEntry { producer: track.clone(), - timescale, + framing, tail: Default::default(), }, ); @@ -2762,7 +2783,7 @@ mod tests { #[moq_clock::test] async fn ietf_records_frame_a_group() { // A plain group: Group State 0, then frame 0 with an empty record. - let (res, track, _) = receive_ietf(None, &[0, 0, 0, 1, u64::from(b'x')]).await; + let (res, track, _) = receive_ietf(draft18(None), &[0, 0, 0, 1, u64::from(b'x')]).await; res.unwrap(); let mut group = track.consume().peek_group(0).unwrap(); assert_eq!( @@ -2771,7 +2792,7 @@ mod tests { ); // A relayed group starting at Object 4, then END_OF_GROUP at Object 6. - let (res, track, _) = receive_ietf(None, &[0x1, 4 << 1, 0, 1, u64::from(b'x'), (1 << 1) | 1, 0]).await; + let (res, track, _) = receive_ietf(draft18(None), &[0x1, 4 << 1, 0, 1, u64::from(b'x'), (1 << 1) | 1, 0]).await; res.unwrap(); let group = track.consume().peek_group(0).unwrap(); assert_eq!(group.end_object().map(|end| end.id), Some(6)); @@ -2790,22 +2811,36 @@ mod tests { (&[0x1, 1, 0, 0], "a byte after END_OF_GROUP"), (&[0x1, 0, 0], "a stream ending inside a frame"), ] { - let (res, ..) = receive_ietf(None, body).await; + let (res, ..) = receive_ietf(draft18(None), body).await; assert!(res.is_err(), "accepted {reason}"); } } + /// A track published over moq-lite has no moq-transport state, so a relayed group on + /// one is refused, though its plain groups cross. + #[moq_clock::test] + async fn ietf_records_need_a_moq_transport_track() { + let plain = [0, 0, 0, 1, u64::from(b'x')]; + let relayed = [0x1, 0, 0, 1, u64::from(b'x')]; + for (framing, ok) in [(draft18(None), true), (lite::Framing::default(), false)] { + let (res, ..) = receive_ietf(framing, &plain).await; + res.unwrap(); + let (res, ..) = receive_ietf(framing, &relayed).await; + assert_eq!(res.is_ok(), ok, "{framing:?}: {res:?}"); + } + } + /// A relayed group's records carry properties only when its Group State sets EXTENSIONS, /// as the moq-transport subgroup it mirrors could hold none otherwise. #[moq_clock::test] async fn ietf_records_need_extensions_for_properties() { // Object 0 with a draft-18 block holding 0x20 = 1, or an untimed Timestamp. - for record in [&[3, 4, 0x20, 1][..], &[3, 4, 0x10, 7]] { + for record in [&[2, 0x20, 1][..], &[2, 0x10, 7]] { for (state, ok) in [(0x1 | 0x4, true), (0x1, false)] { let mut body = vec![state, 0]; body.extend(record); body.push(0); - let (res, ..) = receive_ietf(None, &body).await; + let (res, ..) = receive_ietf(draft18(None), &body).await; assert_eq!(res.is_ok(), ok, "Group State {state:#x}, record {record:?}"); } } @@ -2820,15 +2855,15 @@ mod tests { // Each record after its Object code: Object 0 unless it says otherwise. for (record, ok, reason) in [ (&[0, 0][..], true, "nothing at all"), - (&[0, 4, 4, 0x10, 500], false, "a mutable Timestamp"), - (&[0, 3, 4, 0x20, 1], true, "properties without a Timestamp"), - (&[0, 6, 4, 0x0b, 3, 0x10, 499], true, "a sealed Timestamp"), + (&[0, 3, 0x10, 500], false, "a mutable Timestamp"), + (&[0, 2, 0x20, 1], true, "properties without a Timestamp"), + (&[0, 5, 0x0b, 3, 0x10, 499], true, "a sealed Timestamp"), (&[1 << 1, 0], true, "Object 1 without properties"), ] { let mut body = vec![0x1 | 0x4]; body.extend(record); body.extend([frame, 0]); - let (res, track, _) = receive_ietf(Some(Timescale::MILLI), &body).await; + let (res, track, _) = receive_ietf(draft18(Some(Timescale::MILLI)), &body).await; assert_eq!(res.is_ok(), ok, "{reason}: {res:?}"); if ok { let mut group = track.consume().peek_group(0).unwrap(); @@ -2845,12 +2880,12 @@ mod tests { const COUNT: u64 = 1000; // A relayed group's Object 0, whose draft-18 block holds COUNT two-byte properties // (0x20 = 1, then zero deltas), then an empty frame. - let mut body = vec![0x1 | 0x4, 0, 1 + 2 * COUNT, 4, 0x20, 1]; + let mut body = vec![0x1 | 0x4, 0, 2 * COUNT, 0x20, 1]; for _ in 1..COUNT { body.extend([0, 1]); } body.push(0); - let (res, track, pool) = receive_ietf(None, &body).await; + let (res, track, pool) = receive_ietf(draft18(None), &body).await; res.unwrap(); let group = track.consume().peek_group(0).unwrap(); assert_eq!(group.frame_count(), 1); @@ -2909,7 +2944,11 @@ mod tests { 7, TrackEntry { producer: track.clone(), - timescale: Some(Timescale::MILLI), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, tail: Default::default(), }, ); @@ -2976,7 +3015,11 @@ mod tests { 7, TrackEntry { producer: track.clone(), - timescale: Some(Timescale::MILLI), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, tail: tail.clone(), }, ); @@ -3037,7 +3080,7 @@ mod tests { }; let broadcast = crate::broadcast::Info::new().produce(); let request = broadcast.reserve_track("video").unwrap(); - let serving = ServeLoop::new(&serve, request, crate::track::Info::timed(), Some(Timescale::MILLI)); + let serving = ServeLoop::new(&serve, request, crate::track::Info::timed()); let mut group = serving.serving.create_group(group::Info { sequence: 2 }).unwrap(); group.write_frame(crate::Timestamp::ZERO, b"2".as_slice()).unwrap(); group.finish().unwrap(); @@ -3109,7 +3152,7 @@ mod tests { }; let broadcast = crate::broadcast::Info::new().produce(); let request = broadcast.reserve_track("video").unwrap(); - let serving = ServeLoop::new(&serve, request, crate::track::Info::timed(), Some(Timescale::MILLI)); + let serving = ServeLoop::new(&serve, request, crate::track::Info::timed()); let mut reader = broadcast .consume() .track("video") @@ -3156,7 +3199,7 @@ mod tests { }; let broadcast = crate::broadcast::Info::new().produce(); let request = broadcast.reserve_track("video").unwrap(); - let serving = ServeLoop::new(&serve, request, crate::track::Info::timed(), Some(Timescale::MILLI)); + let serving = ServeLoop::new(&serve, request, crate::track::Info::timed()); let mut reader = broadcast .consume() .track("video") @@ -3257,7 +3300,11 @@ mod tests { 7, TrackEntry { producer, - timescale: Some(Timescale::MILLI), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, tail: Default::default(), }, ); @@ -3501,8 +3548,7 @@ mod tests { .create_track("catalog.json", crate::track::Info::timed()) .unwrap(); let mut sub = Sub::None; - let mut establish = - std::pin::pin!(serve.establish(&mut producer, &mut sub, Subscription::default(), Some(Timescale::MILLI),)); + let mut establish = std::pin::pin!(serve.establish(&mut producer, &mut sub, Subscription::default())); // Parked on the first write: the stream is open and nothing has been sent yet. assert!(futures::poll!(establish.as_mut()).is_pending()); @@ -3566,8 +3612,8 @@ mod tests { let broadcast = crate::broadcast::Info::new().produce(); let request = broadcast.reserve_track("audio").unwrap(); - let mut serving = ServeLoop::new(&serve, request, Default::default(), Some(Timescale::MILLI)); - let establish = serve.prepare_establish(&mut serving.serving, Subscription::default(), Some(Timescale::MILLI)); + let mut serving = ServeLoop::new(&serve, request, Default::default()); + let establish = serve.prepare_establish(&mut serving.serving, Subscription::default()); serving.mode = ServeMode::Establish(establish); let gate_run = serve.gate(); let mut running = TrackServeRun { @@ -3655,7 +3701,7 @@ mod tests { for dynamic_first in [true, false] { let h = Harness::new(VERSION); let request = h._broadcast.reserve_track("pending").unwrap(); - let mut serving = ServeLoop::new(&h.serve, request, Default::default(), None); + let mut serving = ServeLoop::new(&h.serve, request, Default::default()); let other = h._broadcast.create_track("other", None).unwrap(); if !dynamic_first { // Isolate the subscription-changed arm, which can observe a concurrent @@ -3695,12 +3741,7 @@ mod tests { name: "video".into(), }; let broadcast = crate::broadcast::Info::new().produce(); - let mut serving = ServeLoop::new( - &serve, - broadcast.reserve_track("video").unwrap(), - Default::default(), - None, - ); + let mut serving = ServeLoop::new(&serve, broadcast.reserve_track("video").unwrap(), Default::default()); let _reader = serving.serving.subscribe(None); let tail = kio::Producer::new(Tail::default()); serving.mode = ServeMode::Tail { @@ -3745,12 +3786,7 @@ mod tests { name: "video".into(), }; let broadcast = crate::broadcast::Info::new().produce(); - let mut serving = ServeLoop::new( - &serve, - broadcast.reserve_track("video").unwrap(), - Default::default(), - None, - ); + let mut serving = ServeLoop::new(&serve, broadcast.reserve_track("video").unwrap(), Default::default()); let mut reader = serving.serving.subscribe(None); assert!(serving.poll(&serve, &kio::Waiter::noop()).is_pending()); assert!(matches!(serving.sub, Sub::Active(_)), "the SUBSCRIBE is on the wire"); @@ -3810,13 +3846,7 @@ mod tests { let mut sub = Sub::None; h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(mid_group_demand()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) .await .expect("an older peer must not fail the subscribe"); @@ -3843,13 +3873,7 @@ mod tests { let mut sub = Sub::None; h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(mid_group_demand()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) .await .unwrap(); @@ -3918,7 +3942,7 @@ mod tests { let empty = Subscription::default().with_end(Position::group(0)); h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await .unwrap(); @@ -3934,13 +3958,7 @@ mod tests { let mut sub = Sub::None; h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(Subscription::default()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(Subscription::default()), true) .await .unwrap(); assert!(matches!(sub, Sub::Active(_)), "the first subscriber opens one"); @@ -3948,7 +3966,7 @@ mod tests { let empty = Subscription::default().with_end(Position::group(0)); h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await .unwrap(); @@ -3971,7 +3989,7 @@ mod tests { .with_start(Position::group(5)) .with_end(Position::group(5)); h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await .unwrap(); @@ -3987,13 +4005,7 @@ mod tests { let mut sub = Sub::None; h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(Subscription::default()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(Subscription::default()), true) .await .unwrap(); assert!(matches!(sub, Sub::Active(_)), "the first subscriber opens one"); @@ -4003,7 +4015,7 @@ mod tests { .with_start(Position::group(5)) .with_end(Position::group(5)); h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(empty), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await .unwrap(); @@ -4043,26 +4055,14 @@ mod tests { let mut sub = Sub::None; h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(Subscription::default()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(Subscription::default()), true) .await .unwrap(); let established = h.wire().len(); // A lite-06 subscriber downstream now wants to resume mid-group. h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(mid_group_demand()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) .await .expect("a downstream frame offset must not tear down an older upstream"); @@ -4080,7 +4080,7 @@ mod tests { async fn refresh_joins_coalesce_upstream() { let h = Harness::new(Version::Lite07); let request = h._broadcast.reserve_track("refresh").unwrap(); - let mut serving = ServeLoop::new(&h.serve, request, track::Info::default().with_refresh(1), None); + let mut serving = ServeLoop::new(&h.serve, request, track::Info::default().with_refresh(1)); let track = h._broadcast.consume().track("refresh").unwrap(); let _baseline = track.subscribe(None).await.unwrap(); assert!(serving.poll(&h.serve, &kio::Waiter::noop()).is_pending()); @@ -4141,7 +4141,7 @@ mod tests { // Establish from group 3; the peer's START is considered in flight. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(3), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(3), true) .await .unwrap(); assert!(applies(&sub), "a fresh subscription accepts its START"); @@ -4157,7 +4157,6 @@ mod tests { .with_end(Position::group(9)), ), true, - Some(Timescale::MILLI), ) .await .unwrap(); @@ -4165,7 +4164,7 @@ mod tests { // The start moves: a buffered START is stale while it sits elsewhere. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(8), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(8), true) .await .unwrap(); assert!(!applies(&sub), "a moved start must invalidate a buffered START"); @@ -4173,7 +4172,7 @@ mod tests { // The start returns: the declaration matches the demand again, so the // publisher's skip (it sends no replacement START) must land. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(3), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(3), true) .await .unwrap(); assert!(applies(&sub), "demand returning restores the START"); @@ -4192,21 +4191,21 @@ mod tests { // Establish from group 5: the floor tracks the request until START lands. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(5), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(5), true) .await .unwrap(); assert_eq!(h.producer.start_sequence(), Some(5)); // Forward: a reader waiting in [5, 8) must fail over, not stall. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(8), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(8), true) .await .unwrap(); assert_eq!(h.producer.start_sequence(), Some(8)); // Backward: the reopened range must stop being a permanent miss. h.serve - .handle_subscription(&mut h.producer, &mut sub, demand(3), true, Some(Timescale::MILLI)) + .handle_subscription(&mut h.producer, &mut sub, demand(3), true) .await .unwrap(); assert_eq!(h.producer.start_sequence(), Some(3)); @@ -4214,13 +4213,7 @@ mod tests { // Live edge: the floor is unknown until the next declaration, so a // group below the stale one must not be a permanent miss. h.serve - .handle_subscription( - &mut h.producer, - &mut sub, - Some(Subscription::default()), - true, - Some(Timescale::MILLI), - ) + .handle_subscription(&mut h.producer, &mut sub, Some(Subscription::default()), true) .await .unwrap(); assert_eq!(h.producer.start_sequence(), None); @@ -5795,7 +5788,6 @@ impl TrackServe { sub: &mut Sub, pref: Option, supports_update: bool, - timescale: Option, ) -> Result, Error> { // An empty half-open range asks for nothing, and the wire cannot say that: its // bounds are inclusive, so the nearest encoding either hands back the position @@ -5825,11 +5817,7 @@ impl TrackServe { producer.demand_arrivals(start.group..u64::MAX); } // Open an upstream SUBSCRIBE for the first subscriber. - Ok(Begin::Establish(self.prepare_establish( - producer, - subscription, - timescale, - ))) + Ok(Begin::Establish(self.prepare_establish(producer, subscription))) } Sub::Active(active) => { // Downstream preferences changed: forward them upstream as a @@ -5900,12 +5888,7 @@ impl TrackServe { /// deregisters `id` if the establish fails. /// /// The subscription's bounds come straight from the demand aggregate. - fn prepare_establish( - &self, - producer: &mut track::Producer, - subscription: Subscription, - timescale: Option, - ) -> Establish { + fn prepare_establish(&self, producer: &mut track::Producer, subscription: Subscription) -> Establish { let id = self.subscriber.next_id.fetch_add(1, atomic::Ordering::Relaxed); // Both halves of each bound come from the same position, so a frame can never @@ -5928,7 +5911,7 @@ impl TrackServe { id, TrackEntry { producer: producer.clone(), - timescale, + framing: producer.info().into(), tail: tail.clone(), }, ); @@ -5953,9 +5936,8 @@ impl TrackServe { producer: &mut track::Producer, sub: &mut Sub, subscription: Subscription, - timescale: Option, ) -> Result<(), Error> { - let mut est = Box::new(self.prepare_establish(producer, subscription, timescale)); + let mut est = Box::new(self.prepare_establish(producer, subscription)); let id = est.id; match kio::wait(move |waiter| est.poll(waiter)).await { Ok(active) => { @@ -5978,9 +5960,8 @@ impl TrackServe { sub: &mut Sub, pref: Option, supports_update: bool, - timescale: Option, ) -> Result<(), Error> { - match self.begin_subscription(producer, sub, pref, supports_update, timescale)? { + match self.begin_subscription(producer, sub, pref, supports_update)? { Begin::Establish(est) => { let mut est = Box::new(est); let id = est.id; @@ -6183,7 +6164,7 @@ impl TrackServeRun { // Older wires declare no publisher retention limit, and no timeline: their // frames arrive untimed. let info = track::Info::default().with_timescale(None); - TrackRunState::Serve(ServeLoop::new(&serve, request, info, None)) + TrackRunState::Serve(ServeLoop::new(&serve, request, info)) }; let gate = serve.gate(); Self { serve, state, gate } @@ -6261,10 +6242,7 @@ impl kio::Task for TrackServeRun { let request = request.take().expect("request pending"); match res { Ok((info, held)) => { - // Lite05 carries per-frame timestamps on the wire at this scale; - // `Some` tells the ingest to decode them. - let timescale = info.timescale; - let mut serve_loop = ServeLoop::new(&self.serve, request, info, timescale); + let mut serve_loop = ServeLoop::new(&self.serve, request, info); serve_loop.held = Some(held); self.state = TrackRunState::Serve(serve_loop); } @@ -6502,7 +6480,6 @@ struct ServeLoop { // preference change to an established subscription. supports_update: bool, supports_fetch: bool, - timescale: Option, mode: ServeMode, /// Armed while nobody holds the copy: it stays, cache and all, for a returning /// reader until this fires. @@ -6692,7 +6669,7 @@ impl FeedFetches { } impl ServeLoop { - fn new(serve: &TrackServe, request: track::Request, info: track::Info, timescale: Option) -> Self { + fn new(serve: &TrackServe, request: track::Request, info: track::Info) -> Self { // Register the fetch handler before accepting: `accept` releases the // request's own fetch gate, and a cache-miss fetch queued while TRACK_INFO // was in flight would be drained as NotFound in the gap. @@ -6716,7 +6693,6 @@ impl ServeLoop { closed: serve.subscriber.session.clone(), supports_update: !matches!(serve.subscriber.version, Version::Lite01 | Version::Lite02), supports_fetch: serve.subscriber.version.has_track_stream(), - timescale, mode: ServeMode::Select, linger: crate::time::Deadline::new(&serve.subscriber.runtime), held: None, @@ -6799,7 +6775,7 @@ impl ServeLoop { Poll::Ready(Ok(req)) => { if self.supports_fetch { self.fetches - .push(FetchServeRun::new(serve.clone(), req, self.timescale)); + .push(FetchServeRun::new(serve.clone(), req, self.serving.info().clone())); } else { // No FETCH on the wire: wait for the live feed, which a // subscription arriving alongside may still be opening. @@ -6813,13 +6789,8 @@ impl ServeLoop { } match self.serving.poll_subscription_changed(waiter) { Poll::Ready(Ok(pref)) => { - match serve.begin_subscription( - &mut self.serving, - &mut self.sub, - pref, - self.supports_update, - self.timescale, - ) { + match serve.begin_subscription(&mut self.serving, &mut self.sub, pref, self.supports_update) + { Ok(Begin::Establish(est)) => self.mode = ServeMode::Establish(est), Ok(Begin::None) => {} // Updating the upstream failed: hand the track back for @@ -7028,15 +6999,15 @@ impl ServeLoop { } /// Serves one downstream fetch end-to-end on its own bidi stream: send FETCH, -/// then fill the group from the bare FRAME messages that follow. The timescale -/// comes from this track's TRACK_INFO (already known), and the group sequence +/// then fill the group from the bare FRAME messages that follow. The track's +/// info comes from its TRACK_INFO (already known), and the group sequence /// is implicit from the request. struct FetchServeRun { validity: crate::model::broadcast::RequestValidity, gate: crate::auth::Gate, serve: TrackServe, session: S, - timescale: Option, + info: track::Info, group: u64, state: FetchRunState, } @@ -7079,7 +7050,7 @@ impl FetchRunState { } impl FetchServeRun { - fn new(serve: TrackServe, request: group::Request, timescale: Option) -> Self { + fn new(serve: TrackServe, request: group::Request, info: track::Info) -> Self { let session = serve.subscriber.session.clone(); let group = request.sequence(); Self { @@ -7087,7 +7058,7 @@ impl FetchServeRun { gate: serve.gate(), serve, session, - timescale, + info, group, state: FetchRunState::Open { request: Some(request) }, } @@ -7286,8 +7257,8 @@ impl kio::Task for FetchServeRun { // Make the group available (resolving the downstream fetch) and fill // it. The track::Info only takes effect if the track isn't accepted yet // (a fetch with no live subscription); otherwise the group inherits the - // accepted timescale. - let group_info = track::Info::default().with_timescale(self.timescale); + // accepted info. + let group_info = self.info.clone(); // The joined fetches pick the group up from the cache only when next // polled, so their demand outlives the request until then. let joined = request.result.clone(); @@ -7315,7 +7286,7 @@ impl kio::Task for FetchServeRun { joined, ingest: Box::new(FrameIngest::new( &self.serve.subscriber, - self.timescale, + (&self.info).into(), ietf, frame_start, )), diff --git a/rs/moq-net/src/lite/track.rs b/rs/moq-net/src/lite/track.rs index 6c7f27e8908..ab302455770 100644 --- a/rs/moq-net/src/lite/track.rs +++ b/rs/moq-net/src/lite/track.rs @@ -81,6 +81,29 @@ const COMPRESSION_NONE: u64 = 0; /// TRACK_INFO `Compression`: per-group DEFLATE (draft-lcurley-moq-flate). const COMPRESSION_DEFLATE: u64 = 1; +/// What reading a track's groups and datagrams needs from its TRACK_INFO and, with the +/// `ietf` extension, TRACK_PROPERTIES. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub(crate) struct Framing { + /// The units of every frame Timestamp, or `None` on an untimed track. + pub timescale: Option, + /// The draft the track's moq-transport publisher spoke, which serialized its objects' + /// properties, or `None` for a moq-lite one, which carries no moq-transport state. + pub ietf: Option, + /// The track is compressed, so a datagram's payload follows its inflated size. + pub compressed: bool, +} + +impl From<&crate::track::Info> for Framing { + fn from(info: &crate::track::Info) -> Self { + Self { + timescale: info.timescale, + ietf: info.ietf(), + compressed: info.compress, + } + } +} + impl Message for TrackInfo { fn decode_msg(r: &mut Decoder<'_>, version: Version) -> Result { if !version.has_track_stream() { diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 252fb384e85..82644f2d4a6 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -2724,6 +2724,14 @@ impl Consumer { self.track.timescale } + /// The draft the parent track's moq-transport publisher spoke, which serialized every + /// object's properties, or `None` for a moq-lite one. + #[cfg(feature = "async")] + // private: passthrough + pub(crate) fn ietf(&self) -> Option { + self.track.ietf() + } + /// The index of the next frame this consumer will return. /// /// Starts at 0, or at the group's first available frame once [`Self::set_frames`] has diff --git a/rs/moq-net/src/model/object.rs b/rs/moq-net/src/model/object.rs index 1d5dcd8f3a6..0350544bf54 100644 --- a/rs/moq-net/src/model/object.rs +++ b/rs/moq-net/src/model/object.rs @@ -3,23 +3,6 @@ use bytes::Bytes; use crate::Timestamp; -/// The moq-transport drafts a list of properties came from, which share its key and -/// integer encoding and what each property means. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -// private: passthrough -pub(crate) enum Encoding { - /// Drafts 14 and 15: absolute keys and QUIC varints. - Draft14, - /// Draft 16: delta keys and QUIC varints. - Draft16, - /// Draft 17: delta keys and leading-ones varints without the seven-byte form. - Draft17, - /// Draft 18: delta keys and leading-ones varints. - Draft18, - /// Draft 19 and later: as draft 18, with property 0x06 as SUBGROUP_DELIVERY_TIMEOUT. - Draft19, -} - #[derive(Clone, Debug, PartialEq, Eq)] // private: passthrough pub(crate) enum Value { @@ -55,11 +38,10 @@ pub(crate) struct Properties { #[derive(Clone, Debug, PartialEq, Eq)] // private: passthrough pub(crate) struct Block { - /// The properties in ascending ID order, serialized as `encoding` does. Never empty. + /// The properties in ascending ID order, serialized as the track's + /// [`TrackProperties::version`] does. Never empty. pub bytes: Bytes, - /// How `bytes` is serialized: its key and integer encoding. - pub encoding: Encoding, - /// `bytes` holds Immutable Properties (0x0b), serialized as `encoding` does. + /// `bytes` holds Immutable Properties (0x0b). pub immutable: bool, } @@ -74,16 +56,17 @@ impl Properties { } } -/// The Track Properties moq-net does not interpret, which a relay forwards unchanged. -#[derive(Clone, Debug, Default, PartialEq, Eq)] +/// The moq-transport state of a track a moq-transport publisher published, which a relay +/// forwards unchanged. +#[derive(Clone, Debug, PartialEq, Eq)] // private: passthrough pub(crate) struct TrackProperties { - /// In arrival order, which is ascending, with Immutable Properties (0x0b) as its - /// serialized bytes. + /// The draft the publisher spoke, which serialized these and every object's [`Block`] + /// and gives each property its meaning. A publisher on another draft is a new instance. + pub version: crate::ietf::Version, + /// The Track Properties moq-net does not interpret, in arrival order, which is + /// ascending, with Immutable Properties (0x0b) as its serialized bytes. pub values: Vec, - /// How the publisher serialized the list and the keys inside Immutable Properties. - /// Set beside any values, and never [`Encoding::Draft14`], which predates the list. - pub encoding: Option, } /// Object numbering is separate from the public frame's ordinal in its group. diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 490001c6cf0..69599bf73eb 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -158,11 +158,12 @@ pub struct Info { /// /// On moq-transport, `Some` is advertised as DYNAMIC_GROUPS, which arrives as `Some(0)`. pub refresh: Option, - /// The moq-transport Track Properties moq-net does not interpret, learned from an - /// upstream moq-transport session and forwarded unchanged. `None` when there are none. + /// The draft a moq-transport publisher spoke and the Track Properties moq-net does not + /// interpret, forwarded unchanged. `None` for a track published over moq-lite, which + /// therefore carries no moq-transport state. /// - /// Relay-only: moq-lite does not carry them. Shared, since every group holds a copy - /// of the info. + /// Relay-only: moq-lite carries them only between relays, with the `ietf` extension. + /// Shared, since every group holds a copy of the info. #[cfg_attr(not(feature = "async"), allow(dead_code))] // private: passthrough pub(crate) properties: Option>, @@ -215,6 +216,12 @@ impl Info { self.refresh = refresh.into(); self } + + /// The draft the track's moq-transport publisher spoke, `None` for a moq-lite one. + // private: passthrough + pub(crate) fn ietf(&self) -> Option { + self.properties.as_ref().map(|properties| properties.version) + } } #[cfg(test)] @@ -8413,8 +8420,9 @@ mod test { .unwrap(); // Subscriber decodes the body and writes it downstream, preserving the sequence. - let wire = lite::Datagram::decode(body, version, false, |_| lite::Format { + let wire = lite::Datagram::decode(body, version, false, |_| lite::Framing { timescale: Some(crate::Timescale::MICRO), + ietf: None, compressed: false, }) .unwrap();