Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ above them.
- [One max_age meaning](/quest/m1/cache-max-age.md) - a superseded group goes stale on wall clock since its successor arrived or on media time, whichever is first, in Rust and js/net; fixes the untimed failover stall
- [Quiet latest group test](/quest/m1/quiet-edge-test.md) - pins on `main` that a quiet track's latest group survives rejoin, expiry, and takeover
- [FETCH max-delay](/quest/m1/fetch-max-delay.md) - historical readers obey publisher max-age and an optional tighter reader budget, including while holding a group
- [Draft-20 FETCH edge cases](/quest/m1/fetch-edge-cases.md) - an exact-final range reports End of Track and a FETCH_OK past the requested group closes the session, on main then release
- [Draft-20 FETCH edge-case backport](/quest/m1/fetch-edge-cases.md) - backport the final-object and response-group fixes to a later 0.17 patch after main lands
- [A local close never throws](/quest/m1/subscribe-close-throw.md) - closing a `@moq/net` lite subscription never raises ProtocolViolation for the end of stream it caused, on main
- [Subscribe-close backport](/quest/m1/subscribe-close-backport.md) - the same fix and the `race` loser fix on `release`, so closing a subscription during setup leaves no unhandled rejection in a later 0.17 patch
- [Main live-feed check](/quest/m1/fetch-live-feed.md) - tests pin that a fetch-only or parked warm cache never supplies Largest Object on main, fixing it if it does
Expand Down
28 changes: 16 additions & 12 deletions quest/m1/fetch-edge-cases.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
# [S] Draft-20 FETCH edge cases, main first
# [S] Backport draft-20 FETCH edge cases to 0.17

## Goal

Two edge cases in the draft-20+ single-group FETCH from
[#4971](https://github.com/moq-dev/moq/pull/4971) are fixed on `main`, each
with a regression test that fails without its fix, and then backported to
`release` in a later 0.17 patch:
After the main fixes have merged, backport the two draft-20+ single-group
FETCH edge cases to `release` in a later 0.17 patch, preserving its published
API and carrying regression tests:

- A range ending exactly at the final object of a finished track answers
FETCH_OK with End of Track set, so a relay handling it finishes its copy.
Expand All @@ -20,13 +19,18 @@ Found by Codex on the release backport
[#5402](https://github.com/moq-dev/moq/pull/5402#discussion_r4260963190)); the
same code is on `main`.

- Publisher (`rs/moq-net/src/ietf/publisher.rs`): `read_fetch` stops at
`until` without seeing the group's end, so `group.complete` stays false.
Derive End of Track from the track's final sequence and the group's last
object rather than from having read the group's end.
- Subscriber (`rs/moq-net/src/ietf/subscriber.rs`, `run_group_fetch`): the
draft-20 check refuses an End Location before the start only. Also refuse
one whose group differs from the requested group.
The main fixes and regression tests landed in
[#5487](https://github.com/moq-dev/moq/pull/5487). Port them in a separate
backport PR targeting `release`:

- Publisher: a bounded FETCH reaches the end only if the group is finished
and the read cursor reached or passed its final frame count. A start past
the final object must still be refused INVALID_RANGE on draft 20+.
- Subscriber: validate End Location stays within the requested group before
accepting the group or applying End of Track.

The regression tests cover drafts 20 through 22, including a shorter-prefix
control and rejecting malformed responses both with and without End of Track.

Decided 2026-10-13 (maintainer): this does not gate the next 0.17 cut on
[the release line](/quest/m0/release-22/README.md); the backport ships in a
Expand Down
109 changes: 75 additions & 34 deletions rs/moq-net/src/ietf/publisher.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::{frame, group, origin, track};
use futures::FutureExt as _;
use std::{
collections::HashMap,
ops::Bound,
Expand Down Expand Up @@ -79,7 +80,7 @@ struct FetchedGroup {
/// The Object ID of the first frame.
first: u64,
frames: Vec<frame::Frame>,
/// The group ended within the range, so every frame it will ever hold was read.
/// The fetched range reached the finished group's final object.
complete: bool,
/// The track's info, which FETCH_OK describes and whose units stamp the objects.
info: track::Info,
Expand Down Expand Up @@ -125,19 +126,19 @@ async fn read_fetch(
// longer holds it.
let first = group.index();
let mut frames = Vec::new();
let mut complete = false;
// A long cached group is read a slice per poll, so the session's other tasks still run.
let mut budget = kio::coop::Budget::new(32);
while until.is_none_or(|until| first + (frames.len() as u64) < until) {
kio::wait(|waiter| budget.poll_yield(waiter)).await;
match group.read_frame().await? {
Some(frame) => frames.push(frame),
None => {
complete = true;
break;
}
}
let Some(frame) = group.read_frame().await? else {
break;
};
frames.push(frame);
}
// A bounded range can reach the group's end without reading its terminal None, and a
// start past the end is already there.
let complete =
matches!(group.finished().now_or_never(), Some(Ok(_))) && group.index() >= group.frame_count() as u64;

Ok(FetchedGroup {
sequence,
Expand Down Expand Up @@ -7176,6 +7177,8 @@ mod serve_tests {
Idle,
/// An idle copy that learned a later end of track: still unknown.
IdleEnded,
/// An idle copy whose track ends after its newest group: that group's last object.
IdleFinished,
/// A live feed whose newest group is a datagram the cache cannot read: unknown.
Datagram,
/// A live feed whose newest group was aborted mid-write: unknown.
Expand All @@ -7200,6 +7203,11 @@ mod serve_tests {
h.track.finish_at(track::Position::group(10)).unwrap();
None
}
Largest::IdleFinished => {
h.track.set_idle();
h.track.finish().unwrap();
None
}
Largest::Datagram => {
h.track.append_datagram(timestamp(), b"d".as_slice()).unwrap();
None
Expand All @@ -7214,7 +7222,8 @@ mod serve_tests {
}

/// On draft 20, a start past Largest Object is refused INVALID_RANGE (section 10.13),
/// wherever the publisher knows it. Where it does not, it answers empty.
/// wherever the publisher knows it. Where it does not, it answers empty. A start beyond
/// the final group's last object still finds the track's end there.
#[moq_net_sim::test]
async fn a_draft20_fetch_past_the_largest_object_is_refused() {
for version in [Version::Draft20, Version::Draft21, Version::Draft22] {
Expand All @@ -7223,34 +7232,40 @@ mod serve_tests {
Largest::Live,
Largest::Idle,
Largest::IdleEnded,
Largest::IdleFinished,
Largest::Datagram,
Largest::Aborted,
] {
let mut h = serve(version);
let _newer = publish_largest(&mut h, largest);
settle().await;
for start in [2, 3] {
let mut h = serve(version);
let _newer = publish_largest(&mut h, largest);
settle().await;

let buf = standalone_fetch(
&h,
Location { group: 4, object: 2 },
Location { group: 4, object: 6 },
GroupOrder::Ascending,
)
.await;
match largest {
Largest::Finished | Largest::Live => assert_eq!(
fetch_refusal(buf, version),
invalid_range(version),
"{version} {largest:?}"
),
_ => {
let (ok, objects) = fetch_answer(buf, version);
assert_eq!(
ok.end_location,
Location { group: 4, object: 5 },
"{version} {largest:?}"
);
assert_eq!(objects, Vec::new(), "{version} {largest:?}");
let buf = standalone_fetch(
&h,
Location {
group: 4,
object: start,
},
Location { group: 4, object: 6 },
GroupOrder::Ascending,
)
.await;
match largest {
Largest::Finished | Largest::Live | Largest::IdleFinished => assert_eq!(
fetch_refusal(buf, version),
invalid_range(version),
"{version} {largest:?} start={start}"
),
_ => {
let (ok, objects) = fetch_answer(buf, version);
assert_eq!(
ok.end_location,
Location { group: 4, object: 5 },
"{version} {largest:?} start={start}"
);
assert_eq!(objects, Vec::new(), "{version} {largest:?} start={start}");
}
}
}
}
Expand Down Expand Up @@ -7344,6 +7359,32 @@ mod serve_tests {
}
}

/// A bounded range ending at the last object still reports the track's end; a
/// shorter prefix must not claim that the track ends there.
#[moq_net_sim::test]
async fn a_fetch_ending_at_the_final_object_reports_the_end_of_track() {
for version in FETCH_DRAFTS {
for end in [1, 2] {
let mut h = serve(version);
publish_pairs(&mut h, 5, None);
h.track.finish().unwrap();
settle().await;

let buf = standalone_fetch(
&h,
Location { group: 4, object: 0 },
Location { group: 4, object: end },
GroupOrder::Ascending,
)
.await;
let (ok, objects) = fetch_answer(buf, version);
assert_eq!(ok.end_location, end_location(version, 4, end - 1), "{version}");
assert_eq!(ok.end_of_track, end == 2, "{version}: exclusive end {end}");
assert_eq!(objects, pairs([4])[..end as usize].to_vec(), "{version}");
}
}
}

/// A draft-20 filter bounded by Largest Object is refused, even when that bound would
/// land in the start group: the one-group rule needs it resolved first.
#[moq_net_sim::test]
Expand Down
131 changes: 76 additions & 55 deletions rs/moq-net/src/ietf/subscriber.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4000,11 +4000,15 @@ where
end.object = object;
}

// From draft-20 an empty answer still covers its start, so an End Location before it
// is malformed and closes the session (section 10.14), before it can mark the track's end.
if draft20 && (end.group, end.object) <= (sequence, start) {
// Draft-20's answer must cover its start within the requested group. Validate before
// accepting the group or letting End of Track mark where the track ends.
if draft20 && (end.group != sequence || end.object <= start) {
let err = Error::ProtocolViolation;
tracing::warn!(group = sequence, ?end, "FETCH_OK's End Location is before its start");
tracing::warn!(
group = sequence,
?end,
"FETCH_OK's End Location is outside the requested range"
);
self.session.clone().close(
super::error::to_session_code(&SessionError::from(&err)),
err.to_string().as_ref(),
Expand Down Expand Up @@ -10427,68 +10431,85 @@ mod joining_fetch_tests {
assert!(fetch.await.is_err(), "the accepted group was aborted");
}

/// From draft-20 an End Location before the FETCH's start is malformed (section 10.14):
/// From draft-20 an End Location outside the FETCH's range is malformed:
/// the session closes with PROTOCOL_VIOLATION rather than reading it as an empty answer.
#[moq_net_sim::test]
async fn a_draft20_end_location_before_the_start_closes_the_session() {
const VERSION: Version = Version::Draft20;
async fn a_draft20_end_location_outside_the_range_closes_the_session() {
const GROUP: u64 = 4;
const START: u64 = 3;

let ok = message_bytes(
ietf::FetchOk::ID,
&ietf::FetchOk {
request_id: None,
group_order: GroupOrder::Ascending,
end_of_track: false,
end_location: ietf::Location {
for version in [Version::Draft20, Version::Draft21, Version::Draft22] {
for end_location in [
ietf::Location {
group: GROUP,
object: START - 1,
},
properties: Default::default(),
},
VERSION,
);

let session = ScriptedSession::per_stream_eof(vec![ok]);
let (tasks, _task_set) = crate::util::TaskSet::new();
let subscriber = Subscriber::new(
crate::time::Clock::sim(),
session.clone(),
crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(),
Control::new(None, false),
None,
peer::PeerSetup::default(),
crate::Hop::new(1).unwrap(),
None,
VERSION,
tasks,
Default::default(),
);
ietf::Location {
group: GROUP + 1,
object: 0,
},
] {
for end_of_track in [false, true] {
let ok = message_bytes(
ietf::FetchOk::ID,
&ietf::FetchOk {
request_id: None,
group_order: GroupOrder::Ascending,
end_of_track,
end_location,
properties: Default::default(),
},
version,
);

let track = track::Producer::new(
std::sync::Arc::new(crate::broadcast::Info::default()),
"video",
track::Info::default().with_timescale(None),
);
let dynamic = track.dynamic();
let consumer = track.consume();
let mut fetch = std::pin::pin!(consumer.fetch_group(GROUP, group::Fetch::default().with_frame_start(START)));
assert!(futures::poll!(fetch.as_mut()).is_pending());
let request = dynamic.requested_group().await.expect("no group requested");
let session = ScriptedSession::per_stream_reset(vec![ok]);
let (tasks, _task_set) = crate::util::TaskSet::new();
let subscriber = Subscriber::new(
crate::time::Clock::sim(),
session.clone(),
crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(),
Control::new(None, false),
None,
peer::PeerSetup::default(),
crate::Hop::new(1).unwrap(),
None,
version,
tasks,
Default::default(),
);

subscriber
.clone()
.run_group_fetch(Path::new("broadcast").to_owned(), "video".into(), request, None)
.await;
let track = track::Producer::new(
std::sync::Arc::new(crate::broadcast::Info::default()),
"video",
track::Info::default().with_timescale(None),
);
let dynamic = track.dynamic();
let consumer = track.consume();
let mut fetch =
std::pin::pin!(consumer.fetch_group(GROUP, group::Fetch::default().with_frame_start(START)));
assert!(futures::poll!(fetch.as_mut()).is_pending());
let request = dynamic.requested_group().await.expect("no group requested");

subscriber
.clone()
.run_group_fetch(Path::new("broadcast").to_owned(), "video".into(), request, None)
.await;

assert!(matches!(fetch.await, Err(Error::ProtocolViolation)));
let violation = SessionError::from(&Error::ProtocolViolation).to_code();
assert!(
session.log.closes().iter().any(|(code, _)| *code == violation),
"{:?}",
session.log.closes()
);
assert_eq!(
consumer.final_sequence(),
None,
"{version}: rejected end changed the track"
);
assert!(matches!(fetch.await, Err(Error::ProtocolViolation)));
let violation = SessionError::from(&Error::ProtocolViolation).to_code();
assert!(
session.log.closes().iter().any(|(code, _)| *code == violation),
"{:?}",
session.log.closes()
);
}
}
}
}

/// A cache miss for a group's tail asks upstream from the frame the reader wants and
Expand Down
Loading