diff --git a/doc/concept/standard.md b/doc/concept/standard.md index 260533efe9d..97d777e3631 100644 --- a/doc/concept/standard.md +++ b/doc/concept/standard.md @@ -26,11 +26,11 @@ maps everything else to "not supported" or a harmless equivalent. The subset drops. What a peer actually observes against this implementation: - **Pull.** Subscribers ask. Single-track `PUBLISH` offers are declined; announce a namespace and serve the resulting subscriptions. Announcements go out unsolicited, and we also ask for every prefix we may discover. The solicit `SETUP` option makes us wait to be asked. Without it, a draft 16+ peer also gets each match as a `NAMESPACE` on its `SUBSCRIBE_NAMESPACE` stream, so it hears the namespace twice. We never send `PUBLISH`: on drafts 16 and 17 a `SUBSCRIBE_NAMESPACE` asking only for `PUBLISH` is refused, one asking for both gets only `NAMESPACE`, and any other Subscribe Options value closes the session. -- **History is one group.** On drafts 14 through 19 a `FETCH` returns one whole group from the cache, or the saved prefix of the group a new subscription just joined. A range of groups is refused, as is any `FETCH` on draft 20 and later. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. -- **Timing.** A track whose `SUBSCRIBE_OK` declares no `TIMESCALE` is untimed, as is every track on drafts 14 through 16. A reader that never subscribed reads a standalone fetch untimed. +- **History is one group.** On drafts 14 through 19 a `FETCH` returns one whole group from the cache, or the saved prefix of the group a new subscription just joined. A range of groups is refused, as is any `FETCH` on draft 20 and later. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed. +- **Timing.** A track whose `SUBSCRIBE_OK` declares no `TIMESCALE` is untimed, as is every track on drafts 14 through 16. A Rust reader that never subscribed takes the units from `TRACK_STATUS` on draft 18 and later. - **Strict SETUP and parameters.** Repeated unknown `SETUP` options, GREASE included, are accepted if well-formed; a repeated known option is rejected. On draft 17 and later, a `GROUP_ORDER` other than Ascending (1) or Descending (2) closes the session. Message parameters follow the negotiated draft's lists: drafts 14 and 15 ignore an unknown or misplaced parameter, draft 16 ignores one defined only for another message but closes the session on an unknown one, and draft 17 and later close it on both. - **One credential per session**, carried in `SETUP` and forwarded to the [auth server](/bin/relay/auth#the-contract) unverified. A token attached to an individual request is ignored. An alias reference or a second token closes the session. -- **Refused, not fatal.** A legal request this stack does not serve is rejected on its own and the session stays up: `FORWARD=0`, range filters, `TRACK_STATUS`, `SUBSCRIBE_TRACKS`, and the fetch forms above. On draft 19 and later, and in Rust on drafts 14 through 16, a subscription update may change only priority; any other update ends that subscription. +- **Refused, not fatal.** A legal request this stack does not serve is rejected on its own and the session stays up: `FORWARD=0`, range filters, `TRACK_STATUS` to a JavaScript publisher, `SUBSCRIBE_TRACKS`, and the fetch forms above. On draft 19 and later, and in Rust on drafts 14 through 16, a subscription update may change only priority; any other update ends that subscription. - **Datagrams** are a single normal object at object 0, forwarded without renumbering. Anything else is dropped, and a malformed one closes the session. Rust and JavaScript both carry them on every draft. - **Priority.** Higher is served first. The IETF default of 128 is this stack's 127, and a track that never sets one is 127. - **Size.** An object extension block larger than 64 KiB ends that subgroup stream. The session stays up. This cap is ours, not the draft's. diff --git a/quest/m1/README.md b/quest/m1/README.md index 6c7da96751a..b4aafa0da1a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -75,7 +75,6 @@ were done or not worth their cost. - [Data consumer timestamps](/quest/m1/data-consumer-timestamps.md) - json and binary consumers return each value's timestamp, in Rust and every binding; snapshots add `latest()` beside an in-order `next()` - [JS data consumer timestamps](/quest/m1/js-data-consumer-timestamps.md) - @moq/json and @moq/flate consumers return each value's timestamp, with snapshot `next()` and `latest()` - [Draft-20 FETCH](/quest/m1/ietf-fetch-location.md) - a draft-20+ FETCH within one group is served from its LOCATION_FILTER, as older drafts are -- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - a relay fetches from an IETF upstream without subscribing, finished tracks included, with End of Track always reported - [FETCH_OK properties](/quest/m1/fetch-ok-properties.md) - our FETCH_OK carries the track properties SUBSCRIBE_OK does, as draft 16+ requires - [IETF object gaps](/quest/m1/ietf-object-gaps.md) - a gapped object ID is refused loudly like a subgroup in Rust and JS, and an overflowing one closes the session in Rust - [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface diff --git a/quest/m1/fetch-ok-properties.md b/quest/m1/fetch-ok-properties.md index 5257db1e281..35364c36aff 100644 --- a/quest/m1/fetch-ok-properties.md +++ b/quest/m1/fetch-ok-properties.md @@ -9,16 +9,14 @@ meets draft 16+'s rule to include all of a track's properties. ## Plan -Found (2026-10-01) in #4647: a group FETCH with no -prior SUBSCRIBE_OK now reads its max age from FETCH_OK, but our publisher -sends an empty block. +Found (2026-10-01) in #4647: our publisher's FETCH_OK sends an empty +properties block. Our own subscriber no longer reads it (#4974): it learns a +track from SUBSCRIBE_OK or TRACK_STATUS_OK before any FETCH, and from draft 20 +opts out with INCLUDE_PROPERTIES = 0. Other fetch-only readers still need it. - `run_fetch_stream` (`rs/moq-net/src/ietf/publisher.rs`) sends FETCH_OK with `properties: Default::default()`. `run_subscribe_stream` fills them in its `ietf::SubscribeOk` encode. Share that code so the two can't diverge. -- Our group-fetch accept path reads only the max age from FETCH_OK and - hardcodes a microsecond timescale. Apply the same properties SUBSCRIBE_OK - does (timescale, priority, group order) there too. - Drafts 14-15 allow the omission: MAX_CACHE_DURATION is a MAY there, and FETCH_OK has no properties block. From draft 16, a relay MUST include all Extension Headers / Properties associated with a track in FETCH_OK (d16 @@ -37,21 +35,19 @@ sends an empty block. - Honour INCLUDE_PROPERTIES (0x35) on FETCH from draft 20, once [Draft-20 FETCH](/quest/m1/ietf-fetch-location.md) serves draft-20 FETCH at all (today our publisher refuses every one). It defaults to sending the - properties; at 0 the block is present but empty. `fetch.rs` decodes it on - draft 20+ and drops it, so it is not honoured today. + properties; at 0 the block is present but empty. `fetch.rs` decodes it into + `Fetch::properties_wanted`, which the publisher does not honour yet. - Test per draft range: FETCH_OK round-trips the properties SUBSCRIBE_OK would carry, a standalone FETCH's objects arrive stamped, and INCLUDE_PROPERTIES = 0 empties the block. -Decided 2026-10-08: [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) -lands first, since both change how a fetch-only reader learns the track's -properties. +Decided 2026-10-08: #4974 (fetch without SUBSCRIBE) lands first, since both +change how a fetch-only reader learns the track's properties. Public API: none. Wire: FETCH_OK gains its properties on drafts that define the block. Interop: run `just test interop --all`. ## Required -- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - lands first ([#4974](https://github.com/moq-dev/moq/pull/4974)); it asks TRACK_STATUS for the same properties when a publisher omits them - [Draft-20 FETCH](/quest/m1/ietf-fetch-location.md) - draft-20 FETCH is served at all, and edits the same `run_fetch_stream` diff --git a/quest/m1/ietf-fetch-only.md b/quest/m1/ietf-fetch-only.md deleted file mode 100644 index 07d4a865c34..00000000000 --- a/quest/m1/ietf-fetch-only.md +++ /dev/null @@ -1,48 +0,0 @@ -# [M] Fetch without SUBSCRIBE - -## Goal - -A relay serves a FETCH-only demand for an IETF upstream track without -SUBSCRIBE upstream. A finished upstream track can still be fetched, and a -FETCH_OK that reaches the end of the track says so every time. - -## Plan - -The origin splices a route only once the track's info is known. moq-lite gets -it from TRACK_INFO, so its fetch-only demand never subscribes. IETF has no -TRACK_INFO, so today fetch-only demand still makes the relay SUBSCRIBE upstream. -Two symptoms follow: - -- A finished upstream track refuses the SUBSCRIBE, so its groups cannot be - fetched either. -- The live subscription races the group FETCHes. End of Track is known only - once an upstream FETCH_OK reports it, so a downstream FETCH that runs to the - end can answer before that and leave End of Track unset. moxygen's "FETCH - with large objects" case flakes on this. - -TRACK_STATUS is the likely source of the info. The publisher refuses it today, -so both sides are in scope. Keep the SUBSCRIBE path for real subscription -demand. - -Decided (2026-10-01): - -- Our subscriber learns a track's properties from TRACK_STATUS, sent in - parallel with every FETCH or SUBSCRIBE. From draft 20 it sets - INCLUDE_PROPERTIES to 0 on those requests, so the properties aren't sent - twice. Before draft 20 there's no opt-out, so it ignores the duplicate. -- First decouple object timestamps from INCLUDE_PROPERTIES: the publisher - derives its serving timescale from `properties_wanted` - (`run_subscribe_stream` in `rs/moq-net/src/ietf/publisher.rs`), so opting out today strips the Timestamp property - from every object and the subscriber falls back to arrival time. Test - TRACK_STATUS plus the opt-out with source timestamps that differ from - arrival. -- The publisher serves TRACK_STATUS. Its reply carries what it would set in a - SUBSCRIBE_OK (d22 ยง9.13), from the same code that fills SUBSCRIBE_OK and - FETCH_OK. -- The publisher still follows the spec for subscribers that don't opt out: - [FETCH_OK properties](/quest/m1/fetch-ok-properties.md). - -## Related - -- [FETCH_OK properties](/quest/m1/fetch-ok-properties.md) - the publisher side of the same properties, which lands after this -- [JavaScript FETCH](/quest/m1/js-fetch.md) - the browser publisher answers these fetches diff --git a/quest/m2/pipeline-fetch-info.md b/quest/m2/pipeline-fetch-info.md index 0b81990e530..e22829f224b 100644 --- a/quest/m2/pipeline-fetch-info.md +++ b/quest/m2/pipeline-fetch-info.md @@ -10,9 +10,7 @@ and passes the origin's consistency check. ## Plan Decided 2026-10-07, while landing #4974 (fetch-only IETF demand), which keeps -a sequential TRACK_STATUS before the first FETCH. Decided 2026-10-08: it -Requires [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) (#4974), -since that adds the TRACK_STATUS this pipelines. +a sequential TRACK_STATUS before the first FETCH. Why the round trip exists (facts, 2026-10-07): nothing in a FETCH request needs the info. Two serial gates do: @@ -49,7 +47,3 @@ released before its route's info passes. Measure the first-fetch latency before and after. Public API: none. Wire: none (ordering only). - -## Required - -- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - adds the sequential TRACK_STATUS before the first FETCH that this pipelines diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 8af5bfe0fd4..0ec56ed3683 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -891,6 +891,7 @@ pub fn seeds() -> Vec { fetch_type, range_filters: false, fill_timeout: false, + properties_wanted: true, }; fetch.encode_bytes(*version).ok() }) else { diff --git a/rs/moq-net/src/ietf/adapter.rs b/rs/moq-net/src/ietf/adapter.rs index 8c39957c753..7734c6b83d0 100644 --- a/rs/moq-net/src/ietf/adapter.rs +++ b/rs/moq-net/src/ietf/adapter.rs @@ -961,6 +961,14 @@ fn classify(type_id: u64, body: &Bytes, version: Version, namespaces: &Namespace } _ => Err(Error::UnexpectedMessage), }, + // TRACK_STATUS_OK and TRACK_STATUS_ERROR (v14 only): the one answer the request gets. + ietf::TrackStatusOk::ID_14 | ietf::TRACK_STATUS_ERROR_14 => match version { + Version::Draft14 => { + let id = decode_request_id(body, version)?; + Ok(Route::CloseStream(id)) + } + _ => Err(Error::UnexpectedMessage), + }, // PublishOk (0x1E) ietf::PublishOk::ID => { let id = decode_response_request_id(body, version)?; diff --git a/rs/moq-net/src/ietf/fetch.rs b/rs/moq-net/src/ietf/fetch.rs index 6793193ad6b..3794efc84ca 100644 --- a/rs/moq-net/src/ietf/fetch.rs +++ b/rs/moq-net/src/ietf/fetch.rs @@ -133,6 +133,9 @@ pub struct Fetch<'a> { /// that ends in Timed-Out gaps we cannot write, so the request is refused rather than /// left to wait without it. Never encoded; we send no fill timeout. pub fill_timeout: bool, + /// Whether the requester wants Track Properties in FETCH_OK (INCLUDE_PROPERTIES, + /// draft-20+). Always `true` before draft-20, which has no way to opt out. + pub properties_wanted: bool, } impl Message for Fetch<'_> { @@ -174,10 +177,13 @@ impl Message for Fetch<'_> { }; encode_namespace(w, namespace)?; w.string(track)?; + // INCLUDE_PROPERTIES defaults to 1, so only the opt-out is worth bytes. + let include_properties = (!self.properties_wanted).then_some(false); encode_params!(w, version, 0x20 => self.subscriber_priority, 0x21 => *filter, 0x22 => group_order, + 0x35 => include_properties, ); } } @@ -190,85 +196,95 @@ impl Message for Fetch<'_> { let _required_request_id_delta = buf.varint()?; } - // The token is ignored: the session's grant is what authorizes the request, and - // INCLUDE_PROPERTIES only shapes a FETCH_OK we don't send on draft-20. - let (fetch_type, subscriber_priority, group_order, range_filters, fill_timeout) = match version { - Version::Draft14 => { - let subscriber_priority = buf.u8()?; - let group_order = GroupOrder::decode(buf, version)?; - let fetch_type = FetchType::decode(buf, version)?; - Parameters::skip(buf)?; - (fetch_type, Some(subscriber_priority), Some(group_order), false, false) - } - Version::Draft15 | Version::Draft16 | Version::Draft17 | Version::Draft18 | Version::Draft19 => { - let fetch_type = FetchType::decode(buf, version)?; - decode_params!(buf, version, - 0x03 => _authorization_token: Vec, - 0x0A => fill_timeout: Option where !matches!(version, Version::Draft15 | Version::Draft16 | Version::Draft17), - 0x20 => subscriber_priority: Option, - 0x22 => group_order: Option, - 0x25 => subgroup_filter: Vec where has_range_filters(version), - 0x26 => object_id_filter: Vec where has_range_filters(version), - 0x27 => priority_filter: Vec where has_range_filters(version), - 0x28 => object_property_filter: Vec where has_range_filters(version), - ); - let range_filters = [ - subgroup_filter, - object_id_filter, - priority_filter, - object_property_filter, - ] - .iter() - .any(|filter| !filter.is_empty()); - - ( - fetch_type, - subscriber_priority, - group_order, - range_filters, - fill_timeout.is_some(), - ) - } - // Draft-20 names the track up front and moves the range into LOCATION_FILTER. - _ => { - let namespace = decode_namespace(buf)?; - let track = Cow::Owned(buf.string()?); - decode_params!(buf, version, - 0x03 => _authorization_token: Vec, - 0x0A => fill_timeout: Option, - 0x20 => subscriber_priority: Option, - 0x21 => filter: Option, - 0x22 => group_order: Option, - 0x25 => subgroup_filter: Vec, - 0x26 => object_id_filter: Vec, - 0x27 => priority_filter: Vec, - 0x28 => object_property_filter: Vec, - 0x35 => _include_properties: Option, - ); - let range_filters = [ - subgroup_filter, - object_id_filter, - priority_filter, - object_property_filter, - ] - .iter() - .any(|filter| !filter.is_empty()); - - let fetch_type = FetchType::Filtered { - namespace, - track, - // An absent LOCATION_FILTER fetches the whole track. - filter: filter.unwrap_or(Filter::Unfiltered), - }; - ( - fetch_type, - subscriber_priority, - group_order, - range_filters, - fill_timeout.is_some(), - ) - } - }; + // The token is ignored: the session's grant is what authorizes the request. + let (fetch_type, subscriber_priority, group_order, range_filters, fill_timeout, properties_wanted) = + match version { + Version::Draft14 => { + let subscriber_priority = buf.u8()?; + let group_order = GroupOrder::decode(buf, version)?; + let fetch_type = FetchType::decode(buf, version)?; + Parameters::skip(buf)?; + ( + fetch_type, + Some(subscriber_priority), + Some(group_order), + false, + false, + true, + ) + } + Version::Draft15 | Version::Draft16 | Version::Draft17 | Version::Draft18 | Version::Draft19 => { + let fetch_type = FetchType::decode(buf, version)?; + decode_params!(buf, version, + 0x03 => _authorization_token: Vec, + 0x0A => fill_timeout: Option where !matches!(version, Version::Draft15 | Version::Draft16 | Version::Draft17), + 0x20 => subscriber_priority: Option, + 0x22 => group_order: Option, + 0x25 => subgroup_filter: Vec where has_range_filters(version), + 0x26 => object_id_filter: Vec where has_range_filters(version), + 0x27 => priority_filter: Vec where has_range_filters(version), + 0x28 => object_property_filter: Vec where has_range_filters(version), + ); + let range_filters = [ + subgroup_filter, + object_id_filter, + priority_filter, + object_property_filter, + ] + .iter() + .any(|filter| !filter.is_empty()); + + ( + fetch_type, + subscriber_priority, + group_order, + range_filters, + fill_timeout.is_some(), + true, + ) + } + // Draft-20 names the track up front and moves the range into LOCATION_FILTER. + _ => { + let namespace = decode_namespace(buf)?; + let track = Cow::Owned(buf.string()?); + decode_params!(buf, version, + 0x03 => _authorization_token: Vec, + 0x0A => fill_timeout: Option, + 0x20 => subscriber_priority: Option, + 0x21 => filter: Option, + 0x22 => group_order: Option, + 0x25 => subgroup_filter: Vec, + 0x26 => object_id_filter: Vec, + 0x27 => priority_filter: Vec, + 0x28 => object_property_filter: Vec, + 0x35 => include_properties: Option, + ); + let range_filters = [ + subgroup_filter, + object_id_filter, + priority_filter, + object_property_filter, + ] + .iter() + .any(|filter| !filter.is_empty()); + + let fetch_type = FetchType::Filtered { + namespace, + track, + // An absent LOCATION_FILTER fetches the whole track. + filter: filter.unwrap_or(Filter::Unfiltered), + }; + ( + fetch_type, + subscriber_priority, + group_order, + range_filters, + fill_timeout.is_some(), + // Defaults to 1, so an absent parameter means the requester wants them. + include_properties.unwrap_or(true), + ) + } + }; Ok(Self { request_id, @@ -278,6 +294,7 @@ impl Message for Fetch<'_> { fetch_type, range_filters, fill_timeout, + properties_wanted, }) } } @@ -679,6 +696,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft14); @@ -702,6 +720,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft15); @@ -743,6 +762,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft16); @@ -766,6 +786,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft17); @@ -843,6 +864,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft18); @@ -922,6 +944,7 @@ mod tests { assert_eq!(fetch.group_order, GroupOrder::Ascending); assert!(fetch.range_filters, "{version}"); assert!(fetch.fill_timeout, "{version}"); + assert!(!fetch.properties_wanted, "{version}"); assert_eq!( fetch.fetch_type, FetchType::Filtered { @@ -952,6 +975,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }; #[rustfmt::skip] @@ -963,6 +987,21 @@ mod tests { assert_eq!(encode_message(&fetch, Version::Draft20), expected); assert_eq!(decode_message::(&expected, Version::Draft20).unwrap(), fetch); + // Only the opt-out is written, as a uint8 with no Length. + let opted_out = Fetch { + properties_wanted: false, + ..fetch.clone() + }; + #[rustfmt::skip] + let expected = [FETCH_HEAD, &[ + 0x03, // Number of Parameters + 0x20, 0x80, // SUBSCRIBER_PRIORITY = 128 + 0x02, 0x02, // GROUP_ORDER (0x22) = Descending + 0x13, 0x00, // INCLUDE_PROPERTIES (0x35) = 0 + ]].concat(); + assert_eq!(encode_message(&opted_out, Version::Draft20), expected); + assert_eq!(decode_message::(&expected, Version::Draft20).unwrap(), opted_out); + // The tagged forms have no spelling from draft-20 on, and this one none before it. let mut buf = Vec::new(); assert!( diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index b8bf06b0591..b2ea00c2f8b 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -37,6 +37,25 @@ fn serving_subscription(subscriber_priority: u8) -> Subscription { } } +/// The Track Properties an answer describing the track carries: SUBSCRIBE_OK and +/// TRACK_STATUS_OK. Empty when the requester opted out with INCLUDE_PROPERTIES=0, which +/// keeps the block present but empty. +/// +/// Declaring the timescale is what opts the track into timestamps; every object Timestamp +/// is in these units. We serve the newest group first, matching moq-lite. A draft without +/// room for a property drops it on encode. +fn track_properties(info: &track::Info, wanted: bool) -> ietf::Properties { + if !wanted { + return ietf::Properties::default(); + } + ietf::Properties { + max_cache_duration: info.max_age, + timescale: info.timescale, + priority: Some(super::priority::to_wire(info.priority)), + group_order: Some(GroupOrder::Descending), + } +} + enum FillStep { Batch, Partial(frame::Consumer), @@ -579,9 +598,10 @@ where if !data.is_empty() { return Err(Error::WrongSize); } + tracing::debug!(message = ?msg, "received track_status"); async move { - if let Err(err) = this.reject_track_status(stream, msg.request_id).await { - tracing::debug!(%err, "track status refusal failed"); + if let Err(err) = this.run_track_status_stream(stream, msg).await { + tracing::debug!(%err, "track_status stream error"); } } .maybe_boxed() @@ -698,13 +718,13 @@ where None => track::Position::after_group(end.group), }); let _ = track.update(subscription); - // A Timestamp goes out only when this SUBSCRIBE_OK actually carries TIMESCALE. - // Drafts 14-16 never write that property, so their objects stay unstamped. - // An untimed track declares none, and its objects carry no Timestamp. - let timescale = msg - .properties_wanted - .then(|| track.info().timescale) - .flatten() + // A Timestamp goes out wherever the draft can declare its units and the track has + // them. A subscriber that opted out of this SUBSCRIBE_OK's properties learns them from + // TRACK_STATUS, so the opt-out strips nothing from the objects. Drafts 14-16 never + // write TIMESCALE, and an untimed track declares none, so their objects stay unstamped. + let timescale = track + .info() + .timescale .filter(|_| ietf::Properties::sends_timescale(self.version)); // Draft-20 replaced joining FETCH with subscription fills. Older drafts save @@ -744,21 +764,7 @@ where track_alias: request_id.0, // The subscription floor and FETCH/FILL cap use this same snapshot. largest: edge.largest, - properties: match msg.properties_wanted { - // Declaring the timescale is what opts the track into timestamps; every - // object Timestamp below is in these units. `timescale` is None when this - // version cannot send TIMESCALE. We serve the newest group first, - // matching moq-lite. - true => ietf::Properties { - max_cache_duration: track.info().max_age, - timescale, - priority: Some(super::priority::to_wire(track.info().priority)), - group_order: Some(GroupOrder::Descending), - }, - // INCLUDE_PROPERTIES=0. The field stays present but empty, which also - // means the track opts out of timestamps for this subscriber. - false => ietf::Properties::default(), - }, + properties: track_properties(track.info(), msg.properties_wanted), }) .await?; @@ -1650,16 +1656,87 @@ where Ok(Err(refusal)) } - async fn reject_track_status(&self, mut stream: Stream, request_id: RequestId) -> Result<(), Error> { - let error_code = request::to_code(&Error::Unsupported, request::Kind::TrackStatus, self.version); + /// Answer a TRACK_STATUS with what a SUBSCRIBE_OK for the track would carry, without + /// subscribing. + /// + /// The track resolves as it would for a SUBSCRIBE, so a relay asks its upstream for a + /// track it does not hold yet and the answer waits on that. The Largest Location is + /// whatever the cache holds now: nothing waits for a fresher one. + async fn run_track_status_stream( + self, + mut stream: Stream, + msg: ietf::TrackStatus<'_>, + ) -> Result<(), Error> { + let request_id = msg.request_id; + let broadcast = match self + .serving_origin() + .await + .request_broadcast(&msg.track_namespace, None) + .await + { + Ok(broadcast) => broadcast, + Err(err) => return self.reject_track_status(stream, request_id, &err).await, + }; + let track = match broadcast.track(&msg.track_name) { + Ok(track) => track, + Err(err) => return self.reject_track_status(stream, request_id, &err).await, + }; + + // The requester abandoning the request is owed nothing. + let info = { + let query = track.query(); + let mut finished = false; + kio::wait(|waiter| { + let mut cx = waiter.context(); + if super::request_stream::poll_cancel(&mut stream, &mut finished, self.version, &mut cx).is_ready() { + return Poll::Ready(None); + } + query.poll_ok(waiter).map(Some) + }) + .await + }; + let info = match info { + Some(Ok(info)) => info, + Some(Err(err)) => return self.reject_track_status(stream, request_id, &err).await, + None => return Ok(()), + }; + + stream.writer.varint(ietf::TrackStatusOk::id(self.version)).await?; + stream + .writer + .encode(&ietf::TrackStatusOk { + request_id: match self.version { + Version::Draft14 | Version::Draft15 | Version::Draft16 => Some(request_id), + _ => None, + }, + largest: live_edge(&track).largest, + properties: track_properties(&info, msg.properties_wanted), + }) + .await?; + + // The answer is all this stream carries. See [`Self::reject_subscribe`] for why the + // close is not optional. + let _ = stream.writer.close().await; + Ok(()) + } + + async fn reject_track_status( + &self, + mut stream: Stream, + request_id: RequestId, + err: &Error, + ) -> Result<(), Error> { + let error_code = request::to_code(err, request::Kind::TrackStatus, self.version); + let reason_phrase = err.to_string().into(); if self.version == Version::Draft14 { - stream.writer.varint(0x0fu64).await?; // TRACK_STATUS_ERROR has the SUBSCRIBE_ERROR body. + // TRACK_STATUS_ERROR has the SUBSCRIBE_ERROR body. + stream.writer.varint(ietf::TRACK_STATUS_ERROR_14).await?; stream .writer .encode(&ietf::SubscribeError { request_id, error_code, - reason_phrase: "TRACK_STATUS is not supported".into(), + reason_phrase, }) .await?; } else { @@ -1669,7 +1746,7 @@ where .encode(&ietf::RequestError { request_id: matches!(self.version, Version::Draft15 | Version::Draft16).then_some(request_id), error_code, - reason_phrase: "TRACK_STATUS is not supported".into(), + reason_phrase, retry_interval: 0, }) .await?; @@ -3541,6 +3618,7 @@ mod serve_tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, } .encode_msg(&mut Encoder::new(&mut fetch_body, version.into()), version) .unwrap(); @@ -3875,68 +3953,125 @@ mod serve_tests { } } - /// TRACK_STATUS is recognized but not implemented, and must get a complete refusal. + const ALL_VERSIONS: [Version; 9] = [ + Version::Draft14, + Version::Draft15, + Version::Draft16, + Version::Draft17, + Version::Draft18, + Version::Draft19, + Version::Draft20, + Version::Draft21, + Version::Draft22, + ]; + + fn track_status(namespace: &str, wanted: bool, version: Version) -> Vec { + let msg = ietf::TrackStatus { + request_id: RequestId(REQUEST_ID), + track_namespace: crate::Path::new(namespace), + track_name: "video".into(), + properties_wanted: wanted, + }; + let mut body = Vec::new(); + msg.encode_msg(&mut Encoder::new(&mut body, version.into()), version) + .unwrap(); + body + } + + /// TRACK_STATUS gets what a SUBSCRIBE_OK would carry, as far as each draft's answer has + /// room, without subscribing to the track. INCLUDE_PROPERTIES=0 empties the properties. #[moq_net_sim::test] - async fn track_status_is_refused_on_every_draft() { - for version in [ - Version::Draft14, - Version::Draft15, - Version::Draft16, - Version::Draft17, - Version::Draft18, - Version::Draft19, - Version::Draft20, - Version::Draft21, - Version::Draft22, - ] { - let h = serve(version); - let stream = Stream::open(&mut h.session.clone(), version).await.unwrap(); - let msg = ietf::TrackStatus { - request_id: RequestId(REQUEST_ID), - track_namespace: crate::Path::new("live"), - track_name: "video".into(), - }; - let mut body = Vec::new(); - msg.encode_msg(&mut Encoder::new(&mut body, version.into()), version) - .unwrap(); - let mark = h.log.writes.lock().unwrap().len(); - h.publisher - .clone() - .handle_stream(ietf::TrackStatus::ID, ietf::Body(bytes::Bytes::from(body)), stream) - .unwrap() - .await; - let actual = h.log.writes.lock().unwrap()[mark..].to_vec(); - let expected = { - let log = crate::lite::test_transport::Log::default(); - let mut writer = - crate::coding::Writer::new(crate::lite::test_transport::SinkSend::new(log.clone()), version); - if version == Version::Draft14 { - writer.varint(0x0fu64).await.unwrap(); - writer - .encode(&ietf::SubscribeError { - request_id: RequestId(REQUEST_ID), - error_code: 0x3, - reason_phrase: "TRACK_STATUS is not supported".into(), - }) - .await - .unwrap(); - } else { - writer.varint(ietf::RequestError::ID).await.unwrap(); - writer - .encode(&ietf::RequestError { - request_id: matches!(version, Version::Draft15 | Version::Draft16) - .then_some(RequestId(REQUEST_ID)), - error_code: 0x3, - reason_phrase: "TRACK_STATUS is not supported".into(), - retry_interval: 0, - }) - .await - .unwrap(); - } - log.writes.lock().unwrap().clone() + async fn track_status_is_answered_on_every_draft() { + for version in ALL_VERSIONS { + for wanted in [true, false] { + let h = serve(version); + let mut group = h.track.create_group(group::Info { sequence: 4 }).unwrap(); + group.write_frame(timestamp(), b"a".as_slice()).unwrap(); + group.write_frame(timestamp(), b"b".as_slice()).unwrap(); + moq_net_sim::sleep(std::time::Duration::from_millis(1)).await; + + let stream = Stream::open(&mut h.session.clone(), version).await.unwrap(); + let mark = h.log.writes.lock().unwrap().len(); + let body = track_status("room", wanted, version); + h.publisher + .clone() + .handle_stream(ietf::TrackStatus::ID, ietf::Body(bytes::Bytes::from(body)), stream) + .unwrap() + .await; + assert!(h.log.resets().is_empty(), "{version}: the answer was reset"); + assert!(h.track.subscription().is_none(), "{version}: TRACK_STATUS subscribed"); + + let wire = h.log.writes.lock().unwrap()[mark..].to_vec(); + let mut buf = Decoder::new(&wire, version.into()); + assert_eq!(buf.varint().unwrap(), ietf::TrackStatusOk::id(version), "{version}"); + let ok = ietf::TrackStatusOk::decode(&mut buf, version).unwrap(); + assert!(buf.is_empty(), "{version}: trailing bytes"); + + let expected = match version { + // GROUP_ORDER is a field on draft-14 and a parameter on draft-15. + Version::Draft14 | Version::Draft15 => ietf::Properties { + group_order: Some(GroupOrder::Descending), + ..Default::default() + }, + Version::Draft16 | Version::Draft17 => ietf::Properties::default(), + // Only draft-20 can carry the opt-out. + _ if !wanted && Filter::is_draft20(version) => ietf::Properties::default(), + _ => track_properties(&track::Info::default(), true), + }; + assert_eq!( + ok, + ietf::TrackStatusOk { + request_id: matches!(version, Version::Draft14 | Version::Draft15 | Version::Draft16) + .then_some(RequestId(REQUEST_ID)), + largest: Some(Location { group: 4, object: 1 }), + properties: expected, + }, + "{version} wanted={wanted}" + ); + } + } + } + + /// A TRACK_STATUS for a broadcast we do not serve is refused as a SUBSCRIBE would be. + #[moq_net_sim::test] + async fn track_status_for_a_missing_broadcast_is_refused() { + for version in ALL_VERSIONS { + let code = refusal(version, ietf::TrackStatus::ID, track_status("absent", true, version)).await; + let expected = match version { + Version::Draft14 => 0x4, + _ => 0x10, }; - assert_eq!(actual, expected, "{version}: wrong TRACK_STATUS refusal bytes"); - assert!(h.log.resets().is_empty(), "{version}: refusal was reset"); + assert_eq!(code, expected, "{version}"); + } + } + + /// Opting out of SUBSCRIBE_OK's properties does not strip the objects' timestamps: + /// their units come from TRACK_STATUS instead. + #[moq_net_sim::test] + async fn the_properties_opt_out_keeps_object_timestamps() { + let version = Version::Draft20; + let stamp = crate::Timestamp::from_millis(123_456).unwrap(); + let mut properties = Vec::new(); + ietf::encode_object_time( + &mut Encoder::new(&mut properties, version.into()), + stamp, + track::Info::default().timescale.expect("a timed track"), + version, + ) + .unwrap(); + + for wanted in [true, false] { + let mut h = serve(version); + let mut group = h.track.create_group(group::Info { sequence: 0 }).unwrap(); + group.write_frame(stamp, b"stamped".as_slice()).unwrap(); + group.finish().unwrap(); + + let mut msg = subscribe(Filter::Unfiltered, None); + msg.properties_wanted = wanted; + run_live(&mut h, msg).await; + + assert_eq!(occurrences(&h.log, b"stamped"), 1, "wanted={wanted}"); + assert_eq!(occurrences(&h.log, &properties), 1, "wanted={wanted}: object unstamped"); } } @@ -4034,7 +4169,7 @@ mod serve_tests { ]; #[rustfmt::skip] - let cases: [(&str, u64, &[u8]); 3] = [ + let cases: [(&str, u64, &[u8]); 2] = [ ("SUBSCRIBE with a Range Filter", ietf::Subscribe::ID, &[ 0x02, // Number of Parameters 0x03, 0x03, 0x03, 0x00, 0xAA, // AUTHORIZATION TOKEN @@ -4046,11 +4181,6 @@ mod serve_tests { 0x17, 0x01, 0x01, // LOCATION_FILTER (0x21): from one group back 0x14, 0x01, // INCLUDE_PROPERTIES (0x35) = 1 ]), - ("TRACK_STATUS with parameters", ietf::TrackStatus::ID, &[ - 0x02, // Number of Parameters - 0x03, 0x03, 0x03, 0x00, 0xAA, // AUTHORIZATION TOKEN - 0x32, 0x00, // INCLUDE_PROPERTIES (0x35) = 0 - ]), ]; // A standalone FETCH we would otherwise serve, carrying FILL_TIMEOUT=0: cache only, @@ -4371,6 +4501,7 @@ mod serve_tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }, ) .await?; @@ -4849,6 +4980,7 @@ mod serve_tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }, ) .await @@ -5117,6 +5249,7 @@ mod serve_tests { fetch_type, range_filters: false, fill_timeout: false, + properties_wanted: true, }, ) .await @@ -5160,6 +5293,7 @@ mod serve_tests { }, range_filters: false, fill_timeout: false, + properties_wanted: true, }, ) .await @@ -7093,6 +7227,7 @@ mod tests { fetch_type, range_filters: false, fill_timeout: false, + properties_wanted: true, }, ) .await diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index c4b2ccdd81f..d4065fbad09 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -444,6 +444,23 @@ impl Target { } } +/// The model's info for a track the publisher described (SUBSCRIBE_OK or TRACK_STATUS_OK). +/// +/// The copy keeps microsecond timestamps whatever units the publisher declared: each +/// object converts on receipt. +fn accepted_info( + timescale: Option, + max_age: Option, + priority: Option, +) -> track::Info { + // Normalized to microseconds, but only for a track that declared a timescale: an + // untimed one must not gain a timeline downstream. + track::Info::default() + .with_timescale(timescale.map(|_| Timescale::MICRO)) + .with_max_age(max_age) + .with_priority(super::priority::from_wire(priority.unwrap_or(128))) +} + /// A copy with no subscription upstream, kept for fetches and a returning subscriber. struct Idle { track: track::Producer, @@ -1755,7 +1772,26 @@ where let track_name = request.name().to_owned(); // Group FETCHes for cache misses: standalone, so they outlive each subscription. let mut group_fetches = TaskSet::owned(); - let mut target = Target::Request(request); + + // Demand with nobody subscribing (fetches, or a TRACK_STATUS downstream) learns the + // track from TRACK_STATUS rather than a SUBSCRIBE: a finished track refuses one, and + // one made only to learn the track races the fetches it was for. Draft-17 still + // SUBSCRIBEs: its TRACK_STATUS answer cannot say whether the track is timed. + let mut target = match request.subscription() { + None if ietf::TrackStatusOk::describes_track(self.version) => { + let Some(idle) = self.track_status(&broadcast_path, &track_name, request).await else { + return; + }; + let Some(next) = self + .linger(&broadcast_path, &track_name, idle, &mut group_fetches) + .await + else { + return; + }; + Target::Resume(next) + } + _ => Target::Request(request), + }; loop { let Some(idle) = self .subscribe_once(&broadcast_path, &track_name, target, &mut group_fetches) @@ -1842,10 +1878,27 @@ where // Write Subscribe message. The aggregate is read now: a subscriber can join while // the request ID and stream were awaited, and nothing updates the priority after. let priority = target.subscription().map(|s| s.priority).unwrap_or(0); - if let Err(err) = self - .write_subscribe(&mut stream, request_id, broadcast_path, track_name, priority, join) - .await - { + let written = async { + stream.writer.varint(ietf::Subscribe::ID).await?; + stream + .writer + .encode(&ietf::Subscribe { + request_id, + track_namespace: broadcast_path.to_owned(), + track_name: track_name.into(), + subscriber_priority: super::priority::to_wire(priority), + group_order: GroupOrder::Descending, + filter: join.filter, + fill: join.fill, + // A resumed copy already knows the track, from the answer that accepted it. + properties_wanted: matches!(target, Target::Request(_)), + forward: true, + range_filters: false, + }) + .await + } + .await; + if let Err(err) = written { tracing::debug!(%err, "failed to write subscribe"); self.remove_subscribe(request_id); target.fail(err); @@ -1991,12 +2044,10 @@ where priority, largest, } = accepted; - // Normalized to microseconds, but only for a track that declared a timescale: an - // untimed one must not gain a timeline downstream. - let info = track::Info::default() - .with_timescale(timescale.map(|_| Timescale::MICRO)) - .with_max_age(max_age) - .with_priority(super::priority::from_wire(priority.unwrap_or(128))); + // A resumed copy opts out of the properties from draft-20, so it keeps the units it + // learned. Before that the answer declares them again. + let timescale = timescale.or(resumed.as_ref().and_then(|idle| idle.timescale)); + let info = accepted_info(timescale, max_age, priority); // The copy already holds the track: it is current again up to the answer's Largest // Location once the join's head lands or a newer group does, since leaving cut the // group it was in. @@ -2368,34 +2419,6 @@ where Ok(()) } - async fn write_subscribe( - &self, - stream: &mut Stream, - request_id: RequestId, - broadcast: &Path<'_>, - track_name: &str, - priority: u8, - join: Join, - ) -> Result<(), Error> { - stream.writer.varint(ietf::Subscribe::ID).await?; - stream - .writer - .encode(&ietf::Subscribe { - request_id, - track_namespace: broadcast.to_owned(), - track_name: track_name.into(), - subscriber_priority: super::priority::to_wire(priority), - group_order: GroupOrder::Descending, - filter: join.filter, - fill: join.fill, - properties_wanted: true, - forward: true, - range_filters: false, - }) - .await?; - Ok(()) - } - /// Send the joining FETCH that follows a pre-draft-20 SUBSCRIBE, and keep reading its /// answer until the subscription ends. /// @@ -2467,6 +2490,7 @@ where fetch_type, range_filters: false, fill_timeout: false, + properties_wanted: true, }) .await?; Ok::<(), Error>(()) @@ -2566,6 +2590,131 @@ where } } + /// Learn a track nobody subscribes to from TRACK_STATUS, and accept it as a copy with no + /// subscription, which [`Self::linger`] serves fetches from and subscribes once someone + /// does. `None` once the request was refused or abandoned: a publisher that refuses + /// TRACK_STATUS refuses the fetch-only request too. + async fn track_status( + &mut self, + broadcast_path: &Path<'_>, + track_name: &str, + request: track::Request, + ) -> Option { + let request_id = match self.control.next_request_id(&self.runtime).await { + Ok(id) => id, + Err(err) => { + request.reject(err); + return None; + } + }; + let mut stream = match Stream::open(&mut self.session.clone(), self.version).await { + Ok(stream) => stream, + Err(err) => { + request.reject(err); + return None; + } + }; + let written = async { + stream.writer.varint(ietf::TrackStatus::ID).await?; + stream + .writer + .encode(&ietf::TrackStatus { + request_id, + track_namespace: broadcast_path.to_owned(), + track_name: track_name.into(), + properties_wanted: true, + }) + .await + } + .await; + if let Err(err) = written { + request.reject(err); + return None; + } + + // Nobody wanting the track any more ends the wait on a peer that may never answer, + // and the session ending fails it with the session's reason. + let demand = request.demand(); + let mut closed = self.session.clone(); + let response = { + let mut response = std::pin::pin!(self.read_track_status_response(&mut stream)); + loop { + let res = kio::wait(|waiter| { + // An answer that already arrived wins over abandonment. + if let Poll::Ready(res) = waiter.poll_future(response.as_mut()) { + return Poll::Ready(Some(res)); + } + if let Poll::Ready(err) = closed.poll_closed(&mut waiter.context()) { + return Poll::Ready(Some(Err(Error::from_transport(err)))); + } + demand.poll_unused(waiter).map(|_| None) + }) + .await; + match res { + Some(res) => break Some(res), + None if request.reject_unused(Error::Cancel) => break None, + // Demand returned in the gap. + None => continue, + } + } + }; + + let Some(response) = response else { + stream.reader.abort(&Error::Cancel); + stream.writer.abort(&Error::Cancel); + return None; + }; + // The publisher has read the request, since it answered, and FINs after its answer: + // FIN our side too and let the stream drop. + let _ = stream.writer.finish(); + let ok = match response { + Ok(ok) => ok, + Err(err) => { + tracing::debug!(%err, broadcast = %self.origin.absolute(broadcast_path), track = %track_name, "track status refused"); + request.reject(err); + return None; + } + }; + + tracing::debug!(broadcast = %self.origin.absolute(broadcast_path), track = %track_name, "track status known"); + let dynamic = request.dynamic(); + let mut track = request.accept(accepted_info( + ok.properties.timescale, + ok.properties.max_cache_duration, + ok.properties.priority, + )); + // Nothing feeds this copy live: what it caches is fetched, never the live edge. + track.set_idle(); + Some(Idle { + demand: track.demand(), + track, + dynamic, + timescale: ok.properties.timescale, + }) + } + + /// Read the answer to a TRACK_STATUS: TRACK_STATUS_OK, or the publisher's refusal as an + /// error. + async fn read_track_status_response(&self, stream: &mut Stream) -> Result { + let type_id = stream.reader.varint().await?; + let body: ietf::Body = stream.reader.decode().await?; + let mut data = body.decoder(self.version); + + let code = match type_id { + id if id == ietf::TrackStatusOk::id(self.version) => { + return Ok(ietf::TrackStatusOk::decode_msg(&mut data, self.version)?); + } + ietf::TRACK_STATUS_ERROR_14 if self.version == Version::Draft14 => { + ietf::SubscribeError::decode_msg(&mut data, self.version)?.error_code + } + ietf::RequestError::ID if self.version != Version::Draft14 => { + ietf::RequestError::decode_msg(&mut data, self.version)?.error_code + } + _ => return Err(Error::UnexpectedMessage), + }; + Err(request::from_code(code, request::Kind::TrackStatus, self.version)) + } + pub async fn recv_group(&mut self, stream: &mut Reader) -> Result<(), Error> { let mut group: ietf::GroupHeader = stream.decode().await?; @@ -3467,6 +3616,9 @@ where }, range_filters: false, fill_timeout: false, + // The copy learned the track before any fetch, from SUBSCRIBE_OK or + // TRACK_STATUS_OK, so a FETCH_OK repeating it is waste. + properties_wanted: false, }) .await?; Ok::<_, Error>(()) @@ -3525,15 +3677,10 @@ where // the stream owes every object up to it. One past the group covers it whole. let end = (end.group == sequence).then_some(end.object); - // Only a track nothing has subscribed to yet takes this info, as SUBSCRIBE_OK would - // have set it. FETCH_OK carries the same Track Properties, so the retention window - // comes from it. - let info = track::Info::default() - .with_timescale(timescale.map(|_| Timescale::MICRO)) - .with_max_age(ok.properties.max_cache_duration); // Joined fetches still count until they pick the accepted group up from the cache. let joined = request.result.clone(); - let mut producer = match request.accept(info) { + // The track was accepted before any fetch was asked, so it keeps that info. + let mut producer = match request.accept(None) { Ok(producer) => producer, // Already served by a concurrent fetch, or the track closed. Err(err) => { @@ -5365,6 +5512,120 @@ mod tests { outstanding } + /// A copy that already knows its track, here from TRACK_STATUS_OK, opts out of the + /// properties when a subscriber returns, and keeps the units it learned rather than + /// taking the empty block the opted-out SUBSCRIBE_OK carries. + #[moq_net_sim::test] + async fn a_resumed_subscribe_opts_out_of_known_properties() { + use crate::coding::Encode as _; + fn message(id: u64, msg: &M, version: Version) -> Vec { + let mut buf = Vec::new(); + crate::coding::Encoder::new(&mut buf, version.into()) + .varint(id) + .unwrap(); + msg.encode(&mut crate::coding::Encoder::new(&mut buf, version.into()), version) + .unwrap(); + buf + } + + let declared = Timescale::new(90_000).unwrap(); + for version in [Version::Draft18, Version::Draft20, Version::Draft22] { + let opts_out = ietf::Filter::is_draft20(version); + let declares = |yes: bool| ietf::Properties { + timescale: yes.then_some(declared), + ..Default::default() + }; + let status_ok = message( + ietf::TrackStatusOk::ID, + &ietf::TrackStatusOk { + request_id: None, + largest: None, + properties: declares(true), + }, + version, + ); + let session = crate::lite::test_transport::ScriptedSession::per_stream(vec![ + status_ok, + // Empty when opted out. Before draft-20 SUBSCRIBE cannot opt out, so it declares + // the units again. + message( + ietf::SubscribeOk::ID, + &ietf::SubscribeOk { + request_id: None, + track_alias: 7, + largest: None, + properties: declares(!opts_out), + }, + version, + ), + ]); + let log = session.log.clone(); + let (tasks, _task_set) = crate::util::TaskSet::new(); + let mut subscriber = Subscriber::new( + crate::time::Clock::sim(), + session, + crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(), + Control::new(None, false), + None, + peer::PeerSetup::default(), + crate::Hop::new(1).unwrap(), + None, + version, + tasks, + Default::default(), + ); + + let producer = crate::broadcast::Info::default().produce(); + let mut dynamic = producer.dynamic(); + let consumer = producer.consume(); + let track = consumer.track("video").unwrap(); + // Demand with nobody subscribing learns the track from TRACK_STATUS. + let mut query = std::pin::pin!(track.query()); + assert!(futures::poll!(query.as_mut()).is_pending()); + let request = dynamic.requested_track().await.expect("no track requested"); + + let probe = subscriber.clone(); + let serving = moq_net_sim::spawn(async move { + subscriber.run_subscribe(Path::new("broadcast"), dynamic, request).await; + }); + settle().await; + let _subscription = track.subscribe(None); + settle().await; + + let writes = log.writes.lock().unwrap().clone(); + let mut buf = Decoder::new(&writes, version.into()); + let mut subscribes = Vec::new(); + while !buf.is_empty() { + let type_id = buf.varint().unwrap(); + let size = buf.u16().unwrap(); + let mut body = Decoder::new(buf.slice(size as usize).unwrap(), version.into()); + if type_id == ietf::Subscribe::ID { + subscribes.push(ietf::Subscribe::decode_msg(&mut body, version).unwrap()); + } + } + assert_eq!(subscribes.len(), 1, "{version}: the returning subscriber subscribes"); + assert_eq!( + subscribes[0].properties_wanted, !opts_out, + "{version}: TRACK_STATUS_OK already gave the properties" + ); + + let timescale = probe + .state + .lock() + .subscribes + .values() + .next() + .expect("the subscription is registered") + .timescale; + assert_eq!( + timescale, + Some(declared), + "{version}: the copy keeps the units it learned" + ); + serving.abort(); + } + } + /// The publisher opens no fetch stream for an empty range, so a fill against a track /// with no content is owed nothing. Leaving it outstanding would withhold every later /// group behind a head that is never coming. @@ -9330,70 +9591,6 @@ mod joining_fetch_tests { assert!(fetch.await.is_err(), "the accepted group was aborted"); } - /// A group FETCH for a track nothing subscribed to takes its retention window from - /// FETCH_OK's properties, as a subscription would from SUBSCRIBE_OK's. - #[moq_net_sim::test] - async fn a_group_fetch_takes_fetch_ok_max_age() { - const GROUP: u64 = 4; - const MAX_AGE: Duration = Duration::from_secs(12); - - for version in [Version::Draft17, Version::Draft19] { - let ok = message_bytes( - ietf::FetchOk::ID, - &ietf::FetchOk { - request_id: None, - group_order: GroupOrder::Ascending, - end_of_track: false, - end_location: ietf::Location { - group: GROUP + 1, - object: 0, - }, - properties: ietf::Properties { - max_cache_duration: Some(MAX_AGE), - ..Default::default() - }, - }, - version, - ); - - let session = ScriptedSession::per_stream_reset(vec![ok]); - let (tasks, _task_set) = crate::util::TaskSet::new(); - let subscriber = Subscriber::new( - crate::time::Clock::sim(), - session.clone(), - crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(), - Control::new(None, false), - None, - peer::PeerSetup::default(), - crate::Hop::new(1).unwrap(), - None, - version, - tasks, - Default::default(), - ); - - // A track the publisher has not accepted, so the fetch supplies its info. - let broadcast = crate::broadcast::Info::new().produce(); - let reserved = broadcast.reserve_track("video").unwrap(); - let dynamic = reserved.dynamic(); - let track = broadcast.consume().track("video").unwrap(); - let mut fetch = std::pin::pin!(track.fetch_group(GROUP, None)); - assert!(futures::poll!(fetch.as_mut()).is_pending()); - let request = dynamic.requested_group().await.expect("no group requested"); - - subscriber - .clone() - .run_group_fetch(Path::new("broadcast").to_owned(), "video".into(), request, None) - .await; - - let info = moq_net_sim::timeout(Duration::from_secs(1), track.query()) - .await - .expect("the fetch installed the track info") - .expect("info"); - assert_eq!(info.max_age, Some(MAX_AGE), "{version}"); - } - } - /// A cache miss for a group's tail asks upstream from the frame the reader wants and /// numbers what arrives from there, so a publisher that evicted the prefix can answer. #[moq_net_sim::test] diff --git a/rs/moq-net/src/ietf/track.rs b/rs/moq-net/src/ietf/track.rs index 616b5bf5466..243f06037b4 100644 --- a/rs/moq-net/src/ietf/track.rs +++ b/rs/moq-net/src/ietf/track.rs @@ -1,4 +1,4 @@ -//! IETF moq-transport track status messages (v14 + v15) +//! IETF moq-transport TRACK_STATUS and its answer. use std::borrow::Cow; @@ -7,7 +7,7 @@ use num_enum::{IntoPrimitive, TryFromPrimitive}; use crate::{ Path, coding::*, - ietf::{Filter, GroupOrder, RequestId, Subscribe}, + ietf::{Filter, GroupOrder, Location, Properties, RequestId, Subscribe, SubscribeOk}, }; use super::Message; @@ -23,6 +23,9 @@ pub struct TrackStatus<'a> { pub request_id: RequestId, pub track_namespace: Path<'a>, pub track_name: Cow<'a, str>, + /// Whether the requester wants Track Properties on the answer (INCLUDE_PROPERTIES, + /// draft-20). + pub properties_wanted: bool, } impl Message for TrackStatus<'_> { @@ -45,20 +48,154 @@ impl Message for TrackStatus<'_> { w.u8(0); // no parameters } _ => { - encode_params!(w, version,); + // Only the opt-out is worth bytes, and an older peer would read the + // parameter as unknown, which is a protocol violation. + let include_properties = (!self.properties_wanted && Filter::is_draft20(version)).then_some(false); + encode_params!(w, version, 0x35 => include_properties); } } Ok(()) } /// Every draft defines TRACK_STATUS as identical to SUBSCRIBE, so it decodes as one and - /// keeps only what names the track. We refuse the request, so the rest goes unread. + /// keeps what names the track and shapes the answer. The rest goes unread: there is no + /// subscription for it to configure. fn decode_msg(r: &mut Decoder<'_>, version: Version) -> Result { let subscribe = Subscribe::decode_msg(r, version)?; Ok(Self { request_id: subscribe.request_id, track_namespace: subscribe.track_namespace, track_name: subscribe.track_name, + properties_wanted: subscribe.properties_wanted, + }) + } +} + +/// TRACK_STATUS_OK: what a SUBSCRIBE_OK would say about the track, with no subscription. +/// +/// Draft-14 gives it its own type ([`Self::ID_14`]) and the SUBSCRIBE_OK body, with a +/// Track Alias of 0. Later drafts answer with a REQUEST_OK, which gained Track Properties +/// in draft-18; before that only the message parameters carry anything. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct TrackStatusOk { + /// Present on drafts 14-16 only. + pub request_id: Option, + + /// The largest Location in the track (LARGEST_OBJECT), absent while it has no content. + pub largest: Option, + + /// The Track Properties, on the drafts whose answer has room for them. + pub properties: Properties, +} + +/// TRACK_STATUS_ERROR on draft-14, with the SUBSCRIBE_ERROR body. Later drafts refuse +/// with REQUEST_ERROR. +pub const TRACK_STATUS_ERROR_14: u64 = 0x0F; + +impl TrackStatusOk { + /// The draft-14 message type. + pub const ID_14: u64 = 0x0E; + + /// The message type on `version`. + pub fn id(version: Version) -> u64 { + match version { + Version::Draft14 => Self::ID_14, + _ => Self::ID, + } + } + + /// Whether this answer describes the track as fully as SUBSCRIBE_OK on `version`. + /// Draft-17 is the gap: SUBSCRIBE_OK declares TIMESCALE, but REQUEST_OK has no + /// properties block to say whether the track is timed. + pub(crate) fn describes_track(version: Version) -> bool { + Self::has_properties(version) || !super::Properties::sends_timescale(version) + } + + /// Whether the answer carries a Track Properties block on `version`. + fn has_properties(version: Version) -> bool { + !matches!( + version, + Version::Draft14 | Version::Draft15 | Version::Draft16 | Version::Draft17 + ) + } +} + +impl Message for TrackStatusOk { + /// REQUEST_OK; see [`Self::id`]. + const ID: u64 = 0x07; + + fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { + if version == Version::Draft14 { + return SubscribeOk { + request_id: self.request_id, + track_alias: 0, + largest: self.largest, + properties: self.properties, + } + .encode_msg(w, version); + } + + if matches!(version, Version::Draft15 | Version::Draft16) { + self.request_id + .expect("request_id required for draft14-16") + .encode(w, version)?; + } else { + assert!(self.request_id.is_none(), "request_id must be None for draft17+"); + } + + // The parameters SUBSCRIBE_OK carries: GROUP_ORDER is one only on draft-15, and a + // Track Property after. + let group_order = match version { + Version::Draft15 => self.properties.group_order, + _ => None, + }; + encode_params!(w, version, + 0x09 => self.largest, + 0x22 => group_order, + ); + + // Track Properties are the final field, so nothing may follow. + if Self::has_properties(version) { + self.properties.encode(w, version)?; + } + Ok(()) + } + + fn decode_msg(r: &mut Decoder<'_>, version: Version) -> Result { + if version == Version::Draft14 { + let ok = SubscribeOk::decode_msg(r, version)?; + return Ok(Self { + request_id: ok.request_id, + largest: ok.largest, + properties: ok.properties, + }); + } + + let request_id = match version { + Version::Draft15 | Version::Draft16 => Some(RequestId::decode(r, version)?), + _ => None, + }; + // Read as SUBSCRIBE_OK's are: EXPIRES is ignored, and MAX_CACHE_DURATION is a + // parameter only on draft-15. + decode_params!(r, version, + 0x04 => max_cache_duration: Option where version == Version::Draft15, + 0x08 => _expires: Option, + 0x09 => largest: Option, + 0x22 => group_order: Option where version == Version::Draft15, + ); + let mut properties = match Self::has_properties(version) { + true => Properties::decode(r, version)?, + false => Properties::default(), + }; + properties.group_order = properties.group_order.or(group_order); + if version == Version::Draft15 { + properties.max_cache_duration = max_cache_duration.map(std::time::Duration::from_millis); + } + + Ok(Self { + request_id, + largest, + properties, }) } } @@ -96,6 +233,22 @@ mod tests { buf.to_vec() } + /// GROUP_ORDER and MAX_CACHE_DURATION are TRACK_STATUS_OK parameters only on draft-15, + /// as on SUBSCRIBE_OK. Draft-16 ignores a GROUP_ORDER it defines elsewhere, even with a + /// value no field could hold, and rejects 0x04, which it does not define at all. Draft-18 + /// rejects a known parameter on a message that does not define it. + #[test] + fn track_status_ok_parameters_follow_the_draft() { + // Request 1, then GROUP_ORDER as the varint 256. + let ok = decode_message::(&[0x01, 0x01, 0x22, 0x41, 0x00], Version::Draft16) + .expect("draft-16 ignores GROUP_ORDER"); + assert_eq!(ok.properties.group_order, None); + // Request 1, then MAX_CACHE_DURATION of 5. + assert!(decode_message::(&[0x01, 0x01, 0x04, 0x05], Version::Draft16).is_err()); + // GROUP_ORDER = Ascending, then an empty properties block. + assert!(decode_message::(&[0x01, 0x22, 0x01], Version::Draft18).is_err()); + } + fn decode_message(bytes: &[u8], version: Version) -> Result { let mut buf = bytes::Bytes::from(bytes.to_vec()); crate::coding::decode_buf(&mut buf, version, M::decode_msg) @@ -107,6 +260,7 @@ mod tests { request_id: RequestId(1), track_namespace: Path::new("test/ns"), track_name: "video".into(), + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft14); @@ -123,6 +277,7 @@ mod tests { request_id: RequestId(1), track_namespace: Path::new("test/ns"), track_name: "video".into(), + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft15); @@ -139,6 +294,7 @@ mod tests { request_id: RequestId(1), track_namespace: Path::new("test/ns"), track_name: "video".into(), + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft17); @@ -155,6 +311,7 @@ mod tests { request_id: RequestId(1), track_namespace: Path::new("test/ns"), track_name: "video".into(), + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft16); @@ -171,6 +328,7 @@ mod tests { request_id: RequestId(1), track_namespace: Path::new("test/ns"), track_name: "video".into(), + properties_wanted: true, }; let encoded = encode_message(&msg, Version::Draft18); @@ -181,9 +339,100 @@ mod tests { assert_eq!(decoded.track_name, "video"); } + const ALL: [Version; 9] = [ + Version::Draft14, + Version::Draft15, + Version::Draft16, + Version::Draft17, + Version::Draft18, + Version::Draft19, + Version::Draft20, + Version::Draft21, + Version::Draft22, + ]; + + /// INCLUDE_PROPERTIES only exists from draft-20, so the opt-out is dropped before it, + /// and the default (1) is never written. + #[test] + fn test_track_status_opts_out_of_properties_from_draft_20() { + for version in ALL { + for wanted in [true, false] { + let msg = TrackStatus { + request_id: RequestId(1), + track_namespace: Path::new("ns"), + track_name: "video".into(), + properties_wanted: wanted, + }; + let decoded: TrackStatus = decode_message(&encode_message(&msg, version), version) + .unwrap_or_else(|e| panic!("{version}: {e}")); + let expected = wanted || !Filter::is_draft20(version); + assert_eq!(decoded.properties_wanted, expected, "{version} wanted={wanted}"); + } + } + } + + /// TRACK_STATUS_OK carries what SUBSCRIBE_OK would, as far as each draft's answer + /// has room: draft-14 is a SUBSCRIBE_OK, drafts 15-17 a REQUEST_OK with only + /// parameters, and from draft-18 the REQUEST_OK ends with the Track Properties. + #[test] + fn test_track_status_ok_round_trip() { + let properties = Properties { + max_cache_duration: Some(std::time::Duration::from_secs(30)), + timescale: Some(crate::Timescale::MILLI), + priority: Some(7), + group_order: Some(GroupOrder::Descending), + }; + let largest = Some(Location { group: 5, object: 2 }); + + for version in ALL { + let msg = TrackStatusOk { + request_id: matches!(version, Version::Draft14 | Version::Draft15 | Version::Draft16) + .then_some(RequestId(3)), + largest, + properties, + }; + let decoded: TrackStatusOk = + decode_message(&encode_message(&msg, version), version).unwrap_or_else(|e| panic!("{version}: {e}")); + + let expected = match version { + Version::Draft14 | Version::Draft15 => Properties { + group_order: Some(GroupOrder::Descending), + ..Default::default() + }, + Version::Draft16 | Version::Draft17 => Properties::default(), + _ => properties, + }; + assert_eq!(decoded.request_id, msg.request_id, "{version}"); + assert_eq!(decoded.largest, largest, "{version}"); + assert_eq!(decoded.properties, expected, "{version}"); + } + } + + /// Draft-14's TRACK_STATUS_OK is a SUBSCRIBE_OK with Track Alias 0, under its own type. + #[test] + fn test_track_status_ok_draft_14_is_a_subscribe_ok() { + let msg = TrackStatusOk { + request_id: Some(RequestId(3)), + largest: None, + properties: Properties::default(), + }; + assert_eq!(TrackStatusOk::id(Version::Draft14), 0x0E); + assert_eq!(TrackStatusOk::id(Version::Draft15), 0x07); + let subscribe_ok = SubscribeOk { + request_id: Some(RequestId(3)), + track_alias: 0, + largest: None, + properties: Properties::default(), + }; + assert_eq!( + encode_message(&msg, Version::Draft14), + encode_message(&subscribe_ok, Version::Draft14) + ); + } + /// TRACK_STATUS is identical to SUBSCRIBE on every draft, so it carries whatever - /// fields and parameters a SUBSCRIBE can. A peer that sends them must still reach - /// our refusal rather than having its session closed. + /// fields and parameters a SUBSCRIBE can. A peer that sends them must still be + /// answered rather than having its session closed. #[test] fn test_track_status_carries_subscribe_fields() { // Request ID 1, Track Namespace ("live"), Track Name ("video"). @@ -219,6 +468,7 @@ mod tests { let msg = TrackStatus::decode_msg(&mut buf, version).unwrap_or_else(|e| panic!("{version}: {e}")); assert!(buf.is_empty(), "{version}: trailing bytes"); assert_eq!(msg.track_name, "video", "{version}"); + assert_eq!(msg.properties_wanted, version != Version::Draft20, "{version}"); } } } diff --git a/rs/moq-net/src/ietf/version.rs b/rs/moq-net/src/ietf/version.rs index 7f0e1b44d91..4e2af0633ad 100644 --- a/rs/moq-net/src/ietf/version.rs +++ b/rs/moq-net/src/ietf/version.rs @@ -137,6 +137,7 @@ mod tests { }, range_filters: false, fill_timeout: false, + properties_wanted: false, }; let group = GroupHeader { diff --git a/rs/moq-net/tests/fetch_only.rs b/rs/moq-net/tests/fetch_only.rs new file mode 100644 index 00000000000..60c991f483b --- /dev/null +++ b/rs/moq-net/tests/fetch_only.rs @@ -0,0 +1,217 @@ +//! Fetching a group from a moq-transport publisher never subscribes to its track. +//! +//! The subscriber learns the track from TRACK_STATUS instead, so a finished track is still +//! fetchable, and a relay's fetch-only demand puts no live subscription upstream to race +//! the fetches that report where the track ends. + +mod support; + +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::time::Duration; + +use moq_net::{Hop, Timestamp, Version}; +use support::harness::{MockConnectOptions, connect_mock}; + +const TIMEOUT: Duration = Duration::from_secs(5); + +fn produce_origin(hop: u64) -> moq_net::origin::Producer { + let (producer, driver) = moq_net::origin::Producer::new(moq_net::origin::Config::new(Hop::new(hop).unwrap())); + support::harness::spawn(driver); + producer +} + +async fn resolve(origin: &moq_net::origin::Producer) -> moq_net::broadcast::Consumer { + let consumer = origin.consume(); + moq_net_sim::timeout(TIMEOUT, consumer.routed("bcast")) + .await + .expect("announce timeout") + .expect("routed"); + moq_net_sim::timeout(TIMEOUT, consumer.request_broadcast("bcast", None)) + .await + .expect("resolve timeout") + .expect("broadcast resolves") +} + +/// Publish a finished track of three groups, and flag any subscription it ever gets. +fn publish_finished(origin: &moq_net::origin::Producer) -> (moq_net::broadcast::Producer, Arc) { + let broadcast = origin.create_broadcast("bcast").unwrap(); + let mut track = broadcast.create_track("video", None).unwrap(); + broadcast.announce(Default::default()).unwrap(); + for sequence in 0..3u64 { + let mut group = track.append_group().unwrap(); + for frame in 0..2 { + group + .write_frame(Timestamp::ZERO, format!("{sequence}-{frame}").into_bytes()) + .unwrap(); + } + group.finish().unwrap(); + } + track.finish().unwrap(); + + let subscribed = Arc::new(AtomicBool::new(false)); + let flag = subscribed.clone(); + drop(moq_net_sim::spawn(async move { + while let Ok(subscription) = track.subscription_changed().await { + if subscription.is_some() { + flag.store(true, Ordering::SeqCst); + } + } + })); + (broadcast, subscribed) +} + +async fn fetch(broadcast: &moq_net::broadcast::Consumer, sequence: u64) -> Result>, moq_net::Error> { + let mut group = moq_net_sim::timeout(TIMEOUT, async { + broadcast.track("video").unwrap().fetch_group(sequence, None).await + }) + .await + .expect("fetch timed out")?; + let mut frames = Vec::new(); + while let Some(frame) = moq_net_sim::timeout(TIMEOUT, group.read_frame()) + .await + .expect("read timed out")? + { + frames.push(frame.payload.to_vec()); + } + Ok(frames) +} + +fn frames(sequence: u64) -> Vec> { + (0..2).map(|frame| format!("{sequence}-{frame}").into_bytes()).collect() +} + +/// Drafts whose FETCH we serve, and the newest, whose FETCH we refuse until draft-20 +/// FETCH lands: it must still not subscribe. Draft-17 is absent: its TRACK_STATUS answer +/// cannot say whether the track is timed, so it still SUBSCRIBEs to learn the track. +const VERSIONS: &[(&str, bool)] = &[ + ("moq-transport-14", true), + ("moq-transport-15", true), + ("moq-transport-16", true), + ("moq-transport-18", true), + ("moq-transport-19", true), + ("moq-transport-22", false), +]; + +async fn direct(version: &str, served: bool) { + let version: Version = version.parse().unwrap(); + let publisher = produce_origin(1); + let (broadcast, subscribed) = publish_finished(&publisher); + + let client = produce_origin(2); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(client.clone()); + let pair = connect_mock(options).await; + + let remote = resolve(&client).await; + let fetched = fetch(&remote, 1).await; + match served { + true => assert_eq!(fetched.expect("fetch"), frames(1), "{version}"), + false => assert!(fetched.is_err(), "{version}: draft-20 FETCH is refused"), + } + assert!(!subscribed.load(Ordering::SeqCst), "{version}: the fetch subscribed"); + + drop((remote, pair, broadcast, client, publisher)); +} + +/// A finished track is fetched without a SUBSCRIBE, which it would refuse. +#[moq_net_sim::test] +async fn a_fetch_learns_the_track_without_subscribing() { + let mut failures = Vec::new(); + for (version, served) in VERSIONS { + if moq_net_sim::spawn(direct(version, *served)).await.is_err() { + failures.push(*version); + } + } + assert!(failures.is_empty(), "failed: {failures:?}"); +} + +async fn relayed(version: &str) { + let version: Version = version.parse().unwrap(); + let publisher = produce_origin(1); + let (broadcast, subscribed) = publish_finished(&publisher); + + let relay = produce_origin(2); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(relay.clone()); + let upstream = connect_mock(options).await; + + let client = produce_origin(3); + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(relay.consume()); + options.client_subscribe = Some(client.clone()); + let downstream = connect_mock(options).await; + + let remote = resolve(&client).await; + for sequence in [2, 0] { + assert_eq!( + fetch(&remote, sequence).await.expect("fetch"), + frames(sequence), + "{version}" + ); + } + assert!( + !subscribed.load(Ordering::SeqCst), + "{version}: the relay subscribed upstream" + ); + + drop((remote, downstream, upstream, broadcast, client, relay, publisher)); +} + +/// A subscriber arriving at a copy learned from TRACK_STATUS still gets timestamped +/// frames: the copy is timed from TRACK_STATUS_OK, and the resumed SUBSCRIBE keeps it so. +#[moq_net_sim::test] +async fn a_subscribe_after_track_status_keeps_timestamps() { + for version in ["moq-transport-18", "moq-transport-19", "moq-transport-22"] { + moq_net_sim::timeout(TIMEOUT, async { + let publisher = produce_origin(1); + let broadcast = publisher.create_broadcast("bcast").unwrap(); + let track = broadcast.create_track("video", None).unwrap(); + broadcast.announce(Default::default()).unwrap(); + + let client = produce_origin(2); + let mut options = MockConnectOptions::new(version.parse::().unwrap()); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(client.clone()); + let _pair = connect_mock(options).await; + + let remote = resolve(&client).await; + let copy = remote.track("video").unwrap(); + let info = copy.query().await.expect("TRACK_STATUS resolves the track"); + assert!(info.timescale.is_some(), "{version}: the copy is untimed"); + assert!(track.subscription().is_none(), "{version}: the query subscribed"); + + let mut sub = copy.subscribe(None).await.unwrap(); + track.demand().used().await.unwrap(); + // Far from any arrival time, so a frame stamped on arrival cannot pass. + let stamp = Timestamp::from_millis(123_456).unwrap(); + let mut group = track.append_group().unwrap(); + group.write_frame(stamp, b"frame".as_ref()).unwrap(); + group.finish().unwrap(); + + let mut group = sub.recv_group().await.unwrap().unwrap(); + let frame = group.read_frame().await.unwrap().unwrap(); + assert_eq!( + frame.timestamp.map(Timestamp::as_millis), + Some(stamp.as_millis()), + "{version}" + ); + }) + .await + .unwrap_or_else(|_| panic!("{version}: timed out")); + } +} + +/// A relay serves a downstream fetch with a fetch upstream, never a subscription. +#[moq_net_sim::test] +async fn a_relay_fetches_without_subscribing_upstream() { + let mut failures = Vec::new(); + for (version, served) in VERSIONS { + if *served && moq_net_sim::spawn(relayed(version)).await.is_err() { + failures.push(*version); + } + } + assert!(failures.is_empty(), "failed: {failures:?}"); +} diff --git a/rs/moq-net/tests/rejoin.rs b/rs/moq-net/tests/rejoin.rs index 6357236a995..eb56fe8f710 100644 --- a/rs/moq-net/tests/rejoin.rs +++ b/rs/moq-net/tests/rejoin.rs @@ -543,3 +543,58 @@ async fn leaving_after_the_cache_window_keeps_the_latest_group() { .unwrap_or_else(|_| panic!("{version}: timed out")); } } + +/// A rejoin keeps the source's timestamps. +/// +/// The returning subscription opts out of the track's properties from draft-20, since the +/// copy already learned them, so its units must come from that first answer rather than +/// the empty block the second one carries. +#[moq_net_sim::test] +async fn rejoin_keeps_source_timestamps() { + for version in ["moq-transport-19", "moq-transport-22"] { + moq_net_sim::timeout(TEST_TIMEOUT, async { + let publisher = produce_origin(1); + let client = produce_origin(2); + + let broadcast = publisher.create_broadcast("bench").unwrap(); + let track = broadcast.create_track("video", None).unwrap(); + broadcast.announce(Default::default()).unwrap(); + + let mut options = MockConnectOptions::new(version.parse::().unwrap()); + options.server_publish = Some(publisher.consume()); + options.client_subscribe = Some(client.clone()); + let _pair = connect_mock(options).await; + + let consumer = client.consume(); + consumer.routed("bench").await.unwrap(); + let remote = consumer.request_broadcast("bench", None).await.unwrap(); + // Far from any arrival time, so a frame stamped on arrival cannot pass. + let ts = |ms| Timestamp::from_millis(ms).unwrap(); + + for (sequence, ms) in [(0, 123_456), (1, 654_321)] { + let mut sub = remote.track("video").unwrap().subscribe(None).await.unwrap(); + track.demand().used().await.unwrap(); + let mut group = track.append_group().unwrap(); + group.write_frame(ts(ms), b"frame".as_ref()).unwrap(); + group.finish().unwrap(); + + let mut group = loop { + let group = sub.recv_group().await.unwrap().unwrap(); + if group.sequence == sequence { + break group; + } + }; + let frame = group.read_frame().await.unwrap().unwrap(); + assert_eq!( + frame.timestamp.map(Timestamp::as_millis), + Some(u128::from(ms)), + "{version} group {sequence}" + ); + drop((group, sub)); + track.demand().unused().await.unwrap(); + } + }) + .await + .unwrap_or_else(|_| panic!("{version}: timed out")); + } +}