Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/concept/standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ subset drops. What a peer actually observes against this implementation:
- **Pull.** Subscribers ask. Single-track `PUBLISH` offers are declined, unless they answer our `SUBSCRIBE_TRACKS` (below); announce a namespace and serve the resulting subscriptions. Announcements go out unsolicited, and we also ask for every prefix we may discover. The solicit `SETUP` option makes us wait to be asked. Without it, a draft 16+ peer also gets each match as a `NAMESPACE` on its `SUBSCRIBE_NAMESPACE` stream, so it hears the namespace twice. We send `PUBLISH` only to answer a `SUBSCRIBE_TRACKS` (below): on drafts 16 and 17 a `SUBSCRIBE_NAMESPACE` asking only for `PUBLISH` is refused, one asking for both gets only `NAMESPACE`, and any other Subscribe Options value closes the session.
- **`SUBSCRIBE_TRACKS` lists, it never pushes.** On draft 18 and later, it lists every track of every broadcast under its prefix, later ones included. With `FORWARD=0`, each track gets a paused `PUBLISH` carrying its Track Properties, and an error `PUBLISH_DONE` once the track is no longer listed, never `TRACK_ENDED`, since the track may go on elsewhere. A `REQUEST_UPDATE` with `FORWARD=1` resumes one, served from the cache like a `SUBSCRIBE` with the update's filter and answered with `LARGEST_OBJECT`; `FORWARD=0` pauses it again. Each paused `PUBLISH` holds a subscription slot, and past the cap a track gets `PUBLISH_SKIPPED` (`PUBLISH_BLOCKED` on draft 18) instead. A listing that omits `FORWARD` or sets it to 1 asks for every track's objects: from draft 19 each track gets `PUBLISH_SKIPPED`, and the subscriber `SUBSCRIBE`s to what it wants; draft 18 has no such skip, so the request is refused with `NOT_SUPPORTED`. A prefix overlapping another listing gets `PREFIX_OVERLAP`, a `TRACK_PROPERTY_FILTER` gets `NOT_SUPPORTED`, and a `REQUEST_UPDATE` on the listing itself ends it. On draft 18, a `SUBSCRIBE` to a track we `PUBLISH`ed gets `DUPLICATE_SUBSCRIPTION`. Cancelling the listing stops new `PUBLISH`es, while those sent live on. A broadcast is listed only while our publish grant covers it, from scratch once it does. Rust sends it upstream for a listing (below); JavaScript does not.
- **A Rust TRACKS listing sends `SUBSCRIBE_TRACKS` upstream, only on demand.** Listing a broadcast from a moq-transport peer on draft 18 and later sends `SUBSCRIBE_TRACKS` with `FORWARD=0` while someone lists it, never on connect, announce, or subscribe, so a peer that refuses, resets, or ignores it affects only that listing. Before draft 18 the listing is refused as unsupported. Each listed track's paused `PUBLISH` is its upstream copy, which a reader resumes rather than subscribing again. Only a `PUBLISH_DONE` with `TRACK_ENDED` removes a listed track; any other end, including the error our own answerer sends for an unlisted track, fails the listing, and readers list again.
- **History is one group.** In Rust, Object ID gaps must be marked by Prior Object ID Gap properties for a group to be fetchable. A `FETCH` returns one group from the cache, or on drafts 14 through 19 the saved prefix of the group a new subscription just joined. A range of groups is refused, as is a draft 20+ `LOCATION_FILTER` that ends at Largest Object rather than an absolute End Location. On draft 20 and later, `FETCH_OK`'s End Location is the requested end, capped at Largest Object: a range holding no objects gets an empty fetch stream, and only a start past Largest Object is refused, with `INVALID_RANGE`. A relay's copy with no upstream subscription can't know Largest Object short of the track's end, so it echoes the requested end instead. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed, and so does a reader whose publisher refuses `TRACK_STATUS` as `NOT_SUPPORTED`.
- **History is one group.** In Rust, Object ID gaps must be marked by Prior Object ID Gap properties for a group to be fetchable. A `FETCH` returns one group from the cache, or on drafts 14 through 19 the saved prefix of the group a new subscription just joined. A range of groups is refused, as is a draft 20+ `LOCATION_FILTER` that ends at Largest Object rather than an absolute End Location. On drafts 14 through 19, `FETCH_OK` End Object 0 covers the entire End Group, matching the request; Rust also accepts the legacy answer naming the next group with Object 0. On draft 20 and later, `FETCH_OK`'s End Location is the requested end, capped at Largest Object: a range holding no objects gets an empty fetch stream, and only a start past Largest Object is refused, with `INVALID_RANGE`. A relay's copy with no upstream subscription can't know Largest Object short of the track's end, so it echoes the requested end instead. JavaScript publishing refuses every `FETCH`. Datagrams are never fetchable. A Rust reader that only fetches learns the track from `TRACK_STATUS` instead of subscribing, so a finished track stays fetchable; draft 17 still subscribes, since its `TRACK_STATUS` answer cannot say whether the track is timed, and so does a reader whose publisher refuses `TRACK_STATUS` as `NOT_SUPPORTED`.
- **Compression.** Rust peers on draft 17 and later exchange compressed tracks when both offer the [Flate extension](/draft/moq-flate). Older peers receive one shared inflated copy. A publisher that seals the compression declaration makes that fallback unavailable: an incompatible peer is refused, preserving the sealed metadata. JavaScript does not offer the IETF extension.
- **Timing.** A track whose `SUBSCRIBE_OK` declares no `TIMESCALE` is untimed, as is every track on drafts 14 through 16. A Rust reader that never subscribed takes the units from `TRACK_STATUS` on draft 18 and later.
- **Strict SETUP and parameters.** Repeated unknown `SETUP` options, GREASE included, are accepted if well-formed; a repeated known option is rejected. On draft 17 and later, a `GROUP_ORDER` other than Ascending (1) or Descending (2) closes the session. Message parameters follow the negotiated draft's lists: drafts 14 and 15 ignore an unknown or misplaced parameter, draft 16 ignores one defined only for another message but closes the session on an unknown one, and draft 17 and later close it on both. Other malformed control input closes the session with the draft's code too, such as a server `SETUP` carrying `PATH` or `AUTHORITY`, a request cut short by FIN, or a second `GOAWAY` on one stream; a first `GOAWAY` on a request stream is legal.
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,6 @@ tracks, so per-group stats stop costing cluster announces.
- [More tests under load](/quest/m1/test-flakes-2/README.md) - the second round of load-only failures, one quest per flake, fixed at the cause
- [A public model](/quest/m1/public-model/README.md) - every model item a built-in session reaches is public, unnecessary, or marked deliberately private, so a custom publisher, subscriber, or relay builds from the public model alone
- [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break has its MIGRATE entry
- [FETCH_OK whole group](/quest/m1/ietf-fetch-end-location.md) - on drafts 14 to 19, FETCH_OK End Object 0 means the whole group in both directions, and a FETCH ending at the maximum group is refused as unsupported
- [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface
- [IETF fill timeout](/quest/m1/ietf-fill-timeout.md) - native FILL_TIMEOUT bounds upstream waiting on drafts 18-22, alongside local content-age enforcement
- [Rust identity enforcement](/quest/m1/catalog-identity-rs.md) - `moq-mux` refuses a redefined rendition, and every Rust publisher mints a new rendition name for a decoder identity change and updates everything else in place
Expand Down
45 changes: 0 additions & 45 deletions quest/m1/ietf-fetch-end-location.md

This file was deleted.

1 change: 0 additions & 1 deletion quest/m1/js-fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ claims that change.
## Required

- [JS ranges](/quest/m1/subscribe-ranges/js.md) - the on-demand request surface this dispatches onto
- [FETCH_OK whole group](/quest/m1/ietf-fetch-end-location.md) - the Rust FETCH_OK contract this copies reads End Object 0 as the whole group first

## Related

Expand Down
7 changes: 7 additions & 0 deletions rs/moq-net/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first.

## Unreleased

### IETF whole-group FETCH answers name the requested group ([#6199](https://github.com/moq-dev/moq/issues/6199))

On moq-transport drafts 14 through 19, a whole-group `FETCH_OK` now names `{g, 0}` instead of `{g+1, 0}`.
Readers accept both forms and set End Of Track to the sequence after the fetched group.
An older moq-net reader can reject the new answer; upgrade it before pairing it with a new publisher, or use draft 20 and later, whose inclusive End Location is unchanged.
No Rust API changes.

### Receive streams are read through leases (#5878)

`transport::poll::RecvStream` now has `poll_lease(cx, want)`, `set_priority`, and `stop`, plus the provided `poll_read_chunk` and `read_chunk`.
Expand Down
2 changes: 2 additions & 0 deletions rs/moq-net/src/ietf/fetch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,8 @@ pub struct FetchOk {
pub request_id: Option<RequestId>,
pub group_order: GroupOrder,
pub end_of_track: bool,
/// Drafts 14-19 use the last object plus one, or Object 0 for the whole End Group;
/// draft 20 and later use an inclusive End Location.
pub end_location: Location,
/// The track's properties, as SUBSCRIBE_OK carries them: MAX_CACHE_DURATION from the
/// draft-14/15 parameters, the Track Properties block from draft-16 on.
Expand Down
70 changes: 50 additions & 20 deletions rs/moq-net/src/ietf/publisher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2422,16 +2422,14 @@ where
.await;
}

// An End Object of 0 asks for the whole End Group.
let end = match end.object {
0 => end.group.checked_add(1).map(|group| Location { group, object: 0 }),
_ => Some(end),
};
let Some(end) = end.filter(|end| (start.group, start.object) < (end.group, end.object)) else {
// End Object 0 includes the entire End Group, even at the largest Group ID.
if start.group > end.group
|| (start.group == end.group && end.object != 0 && start.object >= end.object)
{
return self
.reject_fetch(stream, msg.request_id, &Error::InvalidRange, "empty range")
.await;
};
}

// The peer must have seen the announcement to name this namespace, so this
// resolves like a SUBSCRIBE does.
Expand Down Expand Up @@ -2518,11 +2516,7 @@ where

// One group per FETCH: on a relay, a range costs a serial upstream fetch per missing
// group, all buffered until FETCH_OK. Ranges wait for upstream fills by range.
let last = match end.object {
0 => end.group - 1,
_ => end.group,
};
if start.group != last {
if start.group != end.group {
return self
.reject_fetch(
stream,
Expand Down Expand Up @@ -2651,17 +2645,17 @@ where
}
(end, false)
} else {
// A group that ends the track ends the response at its last object; otherwise
// the response ends where it was asked to.
match end_of_track {
// Cap a bounded response at the track's last object. A whole-group response
// keeps Object 0 rather than claiming that the next group is covered.
match end_of_track && end.object > 0 {
true => (
Location {
group: group.sequence,
object: group.end(),
},
true,
),
false => (end, false),
false => (end, end_of_track),
Comment thread
kixelated marked this conversation as resolved.
Resolved
}
};
// Draft-20's End Location is inclusive: the requested end, capped at Largest Object
Expand Down Expand Up @@ -9822,8 +9816,8 @@ mod serve_tests {
Version::Draft22,
];

/// FETCH_OK's End Location for a response whose last object is `last`: one past it
/// before draft 20, and the object itself from then on.
/// A bounded FETCH_OK's End Location: one past `last` before draft 20,
/// and the object itself from then on. Whole-group answers use Object 0 before draft 20.
fn end_location(version: Version, group: u64, last: u64) -> Location {
let object = match Filter::is_draft20(version) {
true => last,
Expand Down Expand Up @@ -10024,7 +10018,7 @@ mod serve_tests {
let (ok, objects) = fetch_answer(buf, version);
let end = match Filter::is_draft20(version) {
true => Location { group: 2, object: 1 },
false => Location { group: 3, object: 0 },
false => Location { group: 2, object: 0 },
};
assert_eq!(ok.end_location, end, "{version}");
assert!(!ok.end_of_track, "{version}");
Expand Down Expand Up @@ -10566,7 +10560,11 @@ mod serve_tests {
)
.await;
let (ok, objects) = fetch_answer(buf, version);
assert_eq!(ok.end_location, end_location(version, 4, 1), "{version}");
let end = match Filter::is_draft20(version) {
true => end_location(version, 4, 1),
false => Location { group: 4, object: 0 },
};
assert_eq!(ok.end_location, end, "{version}");
assert!(ok.end_of_track, "{version}");
assert_eq!(objects, pairs([4])[1..].to_vec(), "{version}");
}
Expand Down Expand Up @@ -10649,6 +10647,38 @@ mod serve_tests {
}
}

/// A whole End Group at the wire's limit is valid: a multi-group range is unsupported,
/// and a missing single group is not found. Drafts 14-16 cannot encode u64::MAX.
#[moq_clock::test]
async fn a_standalone_fetch_through_the_maximum_group_is_unsupported() {
for version in FETCH_DRAFTS {
let maximum = match version {
Version::Draft14 | Version::Draft15 | Version::Draft16 => (1 << 62) - 1,
_ => u64::MAX,
};
let mut h = serve(version);
publish_pairs(&mut h, 1, None);
h.track.finish().unwrap();
settle().await;
for (start, code) in [(0, 0x3), (maximum, does_not_exist(version))] {
let buf = standalone_fetch(
&h,
Location {
group: start,
object: 0,
},
Location {
group: maximum,
object: 0,
},
GroupOrder::Ascending,
)
.await;
assert_eq!(fetch_refusal(buf, version), code, "{version}: start {start}");
}
}
}

/// Read a REQUEST_ERROR (or draft-14 FETCH_ERROR) off a refused FETCH, as its code.
fn fetch_refusal(wire: bytes::Bytes, version: Version) -> u64 {
let mut buf = Decoder::new(&wire, version.into());
Expand Down
Loading
Loading