diff --git a/quest/m2/README.md b/quest/m2/README.md index 19b24ca7d95..331c25308bf 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -198,7 +198,6 @@ lite-07; bitrate enforcement is not part of this release. - [Auth session caps cost O(1)](/quest/m2/auth-session-index.md) - per-token and per-remote counts and a time-ordered expiry queue replace the scans, after #7211 - [H.264 parsing with maintained crates](/quest/m2/h264-reader.md) - moq-mux parses SPS VUI with h264-reader and shares one bit reader across codec config parsers - [moq-transport finish interop](/quest/m2/ietf-finish.md) - the interop `finish` case runs on every even moq-transport draft instead of being skipped -- [Join after a quick finish](/quest/m2/ietf-join-after-finish.md) - a pre-draft-22 joining FETCH that arrives after its subscription finished is still answered, within a bound the peer already negotiates - [JS joining FETCH](/quest/m2/ietf-js-joining-fetch.md) - the JS publisher answers a joining FETCH that arrives after a quick finish, mirroring the Rust bound - [ashpd uses maintained D-Bus crates](/quest/m2/ashpd-maintained.md) - wait for a stable zbus 6 port, then remove the portal dependency exceptions - [Askama uses a maintained TOML parser](/quest/m2/askama-maintained.md) - wait for a stable parser migration, then remove the generator dependency exception diff --git a/quest/m2/audit-ci.md b/quest/m2/audit-ci.md index 76c59213adf..844ab887f52 100644 --- a/quest/m2/audit-ci.md +++ b/quest/m2/audit-ci.md @@ -66,7 +66,7 @@ verdicts: reads a published broadcast` (`remote error: code=0`). Bisected to [moq-dev/moq#6348](https://github.com/moq-dev/moq/pull/6348): the harness finishes its track before the pre-draft-22 joining FETCH arrives, which - [join after a quick finish](/quest/m2/ietf-join-after-finish.md) owns. + [moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306) fixes on the Rust publisher. - Audio quality: two Chromium Opus `wide` rows exceed their `target_ms_p95` budget. The [audio jitter target](/quest/m1/audio-jitter-target/README.md) owns the budgets. @@ -89,8 +89,8 @@ verdicts: [#6317](https://github.com/moq-dev/moq/pull/6317)). The moq-mux large-enum and moq-net minimal-feature lints pass on main `f42b336f0e`. - Interop's finite-track tails fail before draft 22 (`Error: old`, a JS - `RESET_STREAM`); [join after a quick finish](/quest/m2/ietf-join-after-finish.md) - owns them. TS fails T-STD overflow and late audio starts; the TS reliability + `RESET_STREAM`); [moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306) fixes the Rust publisher's + half, and [JS joining FETCH](/quest/m2/ietf-js-joining-fetch.md) owns the JS one. TS fails T-STD overflow and late audio starts; the TS reliability quests own them. The relay-log `FAIL` lines in Interop's self-test are negative controls, not failures. - Infrastructure flakes in the sample: DNS failures installing Nix diff --git a/quest/m2/ietf-fill-refill.md b/quest/m2/ietf-fill-refill.md index c193dbbcb84..c8af7d782fd 100644 --- a/quest/m2/ietf-fill-refill.md +++ b/quest/m2/ietf-fill-refill.md @@ -21,7 +21,7 @@ Group stitching across two streams drew repeated fixes: [#6348](https://github.com/moq-dev/moq/pull/6348), [#6407](https://github.com/moq-dev/moq/pull/6407), plus [#7322](https://github.com/moq-dev/moq/pull/7322) (headless groups) and -[join after finish](/quest/m2/ietf-join-after-finish.md). The `Fill` enum +join after finish ([moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306)). The `Fill` enum and its handoff (`rs/moq-net/src/ietf/subscriber.rs` about lines 273-396 and 4712-4862) exist only because the model has one producer per group. diff --git a/quest/m2/ietf-finish.md b/quest/m2/ietf-finish.md index 9c214702090..9109cbd1bcd 100644 --- a/quest/m2/ietf-finish.md +++ b/quest/m2/ietf-finish.md @@ -17,8 +17,8 @@ Observed on 2026-10-22 with `just test interop --finish --all --versions ietf` - Draft 18: late readers see the finished catalog end empty, a short tail, or a refused joining FETCH (`publish done: internal error`). This is the pre-draft-22 join after a finish, which - [join after a quick finish](/quest/m2/ietf-join-after-finish.md) fixes; rerun - these drafts once it lands. + [moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306) fixes for Rust publishers; rerun + these drafts. - Draft 22: several readers time out after the withdrawal (C and C++ on the health broadcast), JS readers see `publish done: internal error`, and native JS refuses a "malformed request stream message" from the relay. @@ -29,7 +29,3 @@ every reader's acknowledgement, so judge each cell from its reader's log. Public API: none. Wire: none expected; any fix that changes framing updates its draft. - -## Required - -- [Join after a quick finish](/quest/m2/ietf-join-after-finish.md) - the pre-draft-22 half of this, without which draft 18 cannot pass diff --git a/quest/m2/ietf-join-after-finish.md b/quest/m2/ietf-join-after-finish.md deleted file mode 100644 index 94ca706ed1b..00000000000 --- a/quest/m2/ietf-join-after-finish.md +++ /dev/null @@ -1,84 +0,0 @@ -# [M] A quick finish keeps its pre-draft-22 join - -## Goal - -On draft 18, a joining FETCH that reaches our publisher after its -subscription finished cleanly is still answered from the subscription's -saved Largest group. A track that ends within about one RTT of SUBSCRIBE_OK -then finishes cleanly at the subscriber, head included, instead of losing -its last group. -What the publisher retains for this is bounded by design (the session, the -subscription's request, or a limit the peer already negotiates), never by a -timer constant. - -## Plan - -Every pre-draft-22 join is a SUBSCRIBE followed, after SUBSCRIBE_OK, by a -`RelativeJoining{0}` FETCH ([moq-dev/moq#6348](https://github.com/moq-dev/moq/pull/6348)). -A refused one costs the subscriber that group's head and nothing more -([moq-dev/moq#6407](https://github.com/moq-dev/moq/pull/6407), decided with -the maintainer on 2026-10-22): a stream starting mid-group is dropped and -the subscription goes on. - -State: `rs/moq-net/src/ietf/publisher.rs` registers a `Join` entry in -`Publisher::joins` when it dispatches the SUBSCRIBE, fills it with a `Joined` -snapshot as SUBSCRIBE_OK goes out, and drops it when the subscription ends, -before PUBLISH_DONE. A joining FETCH that arrives later finds no entry, waits -out the registration wait in `joined()`, and is refused as an unknown -subscription, so the subscriber loses the Largest group. Short or -already-finished tracks hit this often. - -It flakes the pinned-pass interop `tail` cells with a Rust publisher, about -one run in six (seen 2026-10-22 while landing -[moq-dev/moq#6407](https://github.com/moq-dev/moq/pull/6407)): the tail test -finishes its track before SUBSCRIBE_OK, so PUBLISH_DONE follows within -milliseconds. A FETCH that already had FETCH_OK being reset with its -subscription is not the cause: it has finished since -[moq-dev/moq#6999](https://github.com/moq-dev/moq/pull/6999), and neither -lifetime failed in six runs of the draft-18 Rust-publisher `tail` cells on -2026-10-28. The relay's settle grace (about a second) ends long before the -10 s registration wait refuses the FETCH, and the empty subscription stream -for the Largest group then becomes an empty group, which `rust -> rust` and -`rust -> js-native-*` read as "empty group". - -Fix it at the publisher: the subscriber cannot recover a head nobody serves, -and other publishers may refuse the same way. - -A finished entry holds a `track::Consumer`, which keeps the track's cache -alive, so retaining it costs memory a peer controls. Candidate bounds: - -- **The request.** Draft 18 gives each SUBSCRIBE its own request stream; - keep the entry until the subscriber closes its side, with our subscriber - holding it open until its joining FETCH resolves. A peer that never sends - the FETCH costs no more than an open subscription, which stream credit - already bounds. -- **The session alone.** Grows with every subscription a long-lived relay - session ever makes, so probably not bounded enough. - -Recommendation: the request close. Keep only entries that can be answered (`Joined::Group`); an -unsubscribed or refused subscription still drops its entry at once. The -entry must not outlive what authorized the subscription: when its request -token's grant expires or is revoked, or the session grant no longer covers -the track, drop it, since a late FETCH reads the cache on that authority. A head -evicted from the cache meanwhile stays a loud refusal. The existing wait in -`joined()` bounds a FETCH whose subscription has not arrived yet; do not -stretch it to cover this. - -Tests, on mocked time with no sleeps, on draft 18: - -- A track that finishes right after SUBSCRIBE_OK, with the joining FETCH - delivered an RTT after PUBLISH_DONE: the FETCH returns the Largest group's - head and the subscriber's track finishes cleanly with it. Today the FETCH - is refused and the group is lost. -- Closing the subscription's request drops its entry. -- A request token that expires (by mocked time) or is revoked after the - finish drops the entry, and the late FETCH is refused. -- An unsubscribed subscription's FETCH is refused, and session close frees - every entry. - -Public API: none. Wire: none; a late joining FETCH is answered where it was -refused. - -## Related - -- [JS joining FETCH](/quest/m2/ietf-js-joining-fetch.md) - the JS publisher mirrors the bound settled here diff --git a/quest/m2/ietf-js-joining-fetch.md b/quest/m2/ietf-js-joining-fetch.md index e8965b687a0..18d9ed2a675 100644 --- a/quest/m2/ietf-js-joining-fetch.md +++ b/quest/m2/ietf-js-joining-fetch.md @@ -16,9 +16,11 @@ in m2. A FETCH that resolves before PUBLISH_DONE already finishes its head. In the tail test the track is finished before SUBSCRIBE_OK goes out, so PUBLISH_DONE follows at once and the FETCH arrives to no subscription. Keep a -finished subscription's join answerable under the bound -[Join after a quick finish](/quest/m2/ietf-join-after-finish.md) settles for -Rust, and mirror it on the joinable-SUBSCRIBE registry js-fetch adds. +finished subscription's join answerable under the bound Rust settled in +[moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306), and mirror it on the joinable-SUBSCRIBE registry js-fetch adds: +only a track that ended cleanly keeps its join, until the subscriber closes the +SUBSCRIBE's request stream, the session closes, or the request token's grant +or the session grant stops covering the track. Tests: a unit test for a FETCH that arrives after PUBLISH_DONE, the bound's refusals mirrored from Rust, and the tail cells @@ -27,7 +29,3 @@ plus the browser publisher. Public API: none. Wire: none; a late joining FETCH is answered where it was refused. - -## Required - -- [Join after a quick finish](/quest/m2/ietf-join-after-finish.md) - settles the bound this reuses diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index 03ad691880e..42a3a59d8fb 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -891,7 +891,8 @@ enum Joined { }, } -/// One subscription's entry in [`Publisher::joins`], removed when the subscription ends. +/// One subscription's entry in [`Publisher::joins`], removed when the subscription ends, or +/// for a track that ended cleanly, once its [`Retain`] lets go. /// /// A fetch that arrives afterwards then finds nothing and is refused, rather than being /// answered from a group that is no longer the edge of anything. @@ -900,6 +901,44 @@ struct Join { request_id: RequestId, } +impl Join { + /// Whether a joining FETCH would be answered from this entry rather than refused. + fn answers(&self) -> bool { + matches!( + self.joins.read().get(&self.request_id), + Some(Some(Joined::Group { .. })) + ) + } +} + +/// A cleanly finished draft-18 subscription's join, kept past PUBLISH_DONE. +/// +/// The subscriber sends its joining FETCH once SUBSCRIBE_OK arrives, so on a track that ends +/// within about an RTT of it the FETCH reaches us after PUBLISH_DONE, and would lose the +/// Largest group's head. The entry holds the track's cache, which a peer controls, so it is +/// bounded by the request: kept until the subscriber closes its side (ours holds it open until +/// the FETCH's head lands or its settle grace ends), and no longer than what authorized the +/// subscription, since a late FETCH reads the cache on that authority. The open request still takes its stream +/// credit and subscription slot meanwhile. +struct Retain { + _join: Join>, + grant: Option, + gate: crate::auth::Gate, + _slot: crate::auth::Hold, +} + +impl Retain { + /// Ready once what authorized the subscription no longer covers it. + fn poll_lapsed(&mut self, waiter: &kio::Waiter) -> Poll<()> { + if let Some(grant) = &self.grant + && grant.poll_ended(waiter).is_ready() + { + return Poll::Ready(()); + } + self.gate.poll_denied(waiter) + } +} + impl Drop for Join { fn drop(&mut self) { self.joins.lock().remove(&self.request_id); @@ -1359,6 +1398,8 @@ impl Publisher { // releasing what it holds upstream, and only then is the peer told. `acked` holds the // group stream count once SUBSCRIBE_OK is buffered, which decides how. let mut acked: Option> = None; + // Set once the track ran to its end, which is all that leaves a join worth keeping. + let mut track_ended = false; let rendezvous = msg.rendezvous; let subscribed = { let serve = async { @@ -1551,6 +1592,7 @@ impl Publisher { let served = served.unwrap_or(Served::Track(Ok(()))); if completed && matches!(served, Served::Track(Ok(()))) { + track_ended = true; self.write_end_of_track(&mut stream, &mut track_serve, &mut request_finished, &mut goaways) .await; } @@ -1578,10 +1620,24 @@ impl Publisher { }) .await }; - // A FETCH joining this subscription is refused if it has not answered yet. - drop(_join); - // The subscription's demand is gone; only its credit may still wait on a verification. - drop((request_grant, _slot)); + // A track that ended cleanly keeps what a joining FETCH needs (see [`Retain`]); any + // other end drops the join, refusing a joining FETCH that has not answered yet. + let retain = match (&subscribed, _join) { + (Ok(Ok(Subscribed::Served(Served::Track(Ok(())), _))), Some(join)) if track_ended && join.answers() => { + Some(Retain { + _join: join, + grant: request_grant, + gate, + _slot, + }) + } + _ => { + // The subscription's demand is gone; only its credit may still wait on a + // verification. + drop((request_grant, _slot)); + None + } + }; let (served, streams) = match subscribed { Ok(Ok(Subscribed::Served(Served::RenewalRefused(refusal), streams))) => { @@ -1610,7 +1666,8 @@ impl Publisher { } }, }; - self.write_publish_done(stream, served, streams, Some(&credit)).await + self.write_publish_done(stream, served, streams, Some(&credit), retain) + .await } } @@ -1693,13 +1750,14 @@ impl Publisher { /// End a subscription on the wire with the PUBLISH_DONE `served` reports, after `streams` /// group streams, returning how serving ended. The stream closes only once no verification - /// holds `credit`'s request, if it has one. + /// holds `credit`'s request, if it has one, and once `retain` lets go of its join. async fn write_publish_done( &self, mut stream: Stream, served: Served, streams: u64, credit: Option<&kio::Producer<()>>, + retain: Option, ) -> Result<(), Error> { let (status, reason) = served.status(); let res = match served { @@ -1709,14 +1767,31 @@ impl Publisher { Served::Paused(_) => Ok(()), }; - let _ = stream - .writer - .encode_message(&ietf::PublishDone { + let mut retain = retain; + { + let done = ietf::PublishDone { status_code: status.code(), stream_count: streams, reason_phrase: reason.into(), + }; + // A peer holding back flow control parks PUBLISH_DONE, and the kept join must still + // go once the request closes or its authorization lapses meanwhile. + let mut encode = std::pin::pin!(stream.writer.encode_message(&done)); + let reader = &mut stream.reader; + let _ = kio::wait(|waiter| { + if let Some(kept) = &mut retain + && (reader.poll_discard(waiter).is_ready() || kept.poll_lapsed(waiter).is_ready()) + { + retain = None; + } + waiter.poll_future(encode.as_mut()) }) .await; + } + + if let Some(retain) = retain { + self.retain_join(&mut stream, retain).await; + } // The peer learns the subscription ended at once, but the stream holds its credit until // the application lets go of every token it took for it. @@ -1729,6 +1804,24 @@ impl Publisher { res } + /// Keep a finished subscription's join until the subscriber closes the request, the + /// session closes, or what authorized the subscription stops covering it. + async fn retain_join(&self, stream: &mut Stream, mut retain: Retain) { + let session = self.session.clone(); + kio::wait(|waiter| { + if session.poll_closed(waiter).is_ready() || stream.writer.poll_closed(waiter).is_ready() { + return Poll::Ready(()); + } + // An update that crossed PUBLISH_DONE is skipped, since nothing answers it now; the + // subscriber's FIN or a reset ends the request. + if stream.reader.poll_discard(waiter).is_ready() { + return Poll::Ready(()); + } + retain.poll_lapsed(waiter) + }) + .await; + } + async fn run_subscription( &self, stream: &mut Stream, @@ -6766,7 +6859,7 @@ mod serve_tests { use super::*; use crate::coding::{Decode, Decoder}; use crate::ietf::GroupOrder; - use crate::lite::test_transport::{Log, ScriptedSession, Sent, SinkSession}; + use crate::lite::test_transport::{Close, Log, ScriptedSession, Sent, SinkSession}; use crate::model::ProduceTest; #[kio::test] @@ -8449,7 +8542,25 @@ mod serve_tests { } h.track.finish().unwrap(); - serve.await.unwrap(); + served(h, serve).await; + h.session.reopen(); + } + + /// Drive `serving` to a clean end once its track has finished. A draft-18 subscription + /// whose track ended keeps its join until the subscriber closes the request, so close it + /// once PUBLISH_DONE is out. + async fn served(h: &Serve, mut serving: std::pin::Pin<&mut impl Future>>) { + while futures::poll!(serving.as_mut()).map(Result::unwrap).is_pending() { + if publish_done_sent(h) { + h.session.close(Close::Fin); + } + kio::clock::sim::yield_now().await; + } + } + + /// Whether the subscription's request stream carries PUBLISH_DONE yet. + fn publish_done_sent(h: &Serve) -> bool { + !h.log.trail().is_empty() && Wire::read(&h.log, h.publisher.version).publish_done.is_some() } /// A subscribe for a broadcast we do not serve is refused with the negotiated draft's own @@ -9742,8 +9853,13 @@ mod serve_tests { /// Drive a subscription until its snapshot is available, failing if it ends first. async fn registered(h: &Serve, serving: impl std::future::Future>) { + registered_as(h, RequestId(REQUEST_ID), serving).await + } + + /// [`registered`], for the subscription with request id `id`. + async fn registered_as(h: &Serve, id: RequestId, serving: impl std::future::Future>) { let joined = h.publisher.joins.wait(|joins| { - if matches!(joins.get(&RequestId(REQUEST_ID)), Some(Some(_))) { + if matches!(joins.get(&id), Some(Some(_))) { Poll::Ready(()) } else { Poll::Pending @@ -9814,7 +9930,7 @@ mod serve_tests { } frame.finish().unwrap().finish().unwrap(); h.track.finish().unwrap(); - serving.await.unwrap(); + served(&h, serving.as_mut()).await; assert_eq!(occurrences(&h.log, payload), 1, "{version} streamed={streamed}"); assert!(h.log.resets().is_empty(), "{version} streamed={streamed}"); @@ -10158,8 +10274,9 @@ mod serve_tests { assert!(h.log.resets().is_empty()); } - /// A joining FETCH that arrives after its subscription ended has no live edge to name, so - /// it is refused rather than answered from a group that is no longer the edge of anything. + /// A joining FETCH that arrives after its subscription ended and the subscriber closed the + /// request has no live edge to name, so it is refused rather than answered from a group + /// that is no longer the edge of anything. #[kio::test] async fn a_joining_fetch_without_its_subscription_is_refused() { for version in JOINING_DRAFTS { @@ -10182,6 +10299,334 @@ mod serve_tests { } } + /// The refusal code of the REQUEST_ERROR a joining FETCH is answered with, failing on a + /// FETCH_OK. + async fn joining_refusal(h: &Serve, version: Version) -> u64 { + let mark = h.log.writes.lock().unwrap().len(); + let mut buf = bytes::Bytes::from(joining_fetch(h, mark).await.unwrap()); + assert_eq!( + crate::coding::decode_varint(&mut buf, version).unwrap(), + ietf::RequestError::ID, + "{version}: the joining FETCH was answered" + ); + crate::coding::decode_buf(&mut buf, version, |r, v| { + Ok(ietf::RequestError::decode(r, v)?.error_code) + }) + .unwrap() + } + + /// Finish the track and drive `serving` until its PUBLISH_DONE is out, failing if the + /// request ends with it: a finished subscription keeps its join until the subscriber + /// closes the request. + async fn finish_and_retain(h: &Serve, mut serving: std::pin::Pin<&mut impl Future>>) { + h.track.finish().unwrap(); + for _ in 0..200 { + assert!( + futures::poll!(serving.as_mut()).is_pending(), + "the request ended with the subscription" + ); + if publish_done_sent(h) { + return; + } + kio::clock::sim::yield_now().await; + } + panic!("PUBLISH_DONE never went out"); + } + + /// A track that ends within an RTT of SUBSCRIBE_OK ends its subscription before the + /// joining FETCH sent on that SUBSCRIBE_OK arrives. The FETCH is still answered with the + /// Largest group's head, until the subscriber closes the request. + #[kio::test] + async fn a_joining_fetch_after_the_track_ends_is_answered() { + for version in JOINING_DRAFTS { + let mut h = serve(version); + publish_groups(&mut h, 4); + let mut group = h.track.create_group(group::Info { sequence: 5 }).unwrap(); + group.write_frame(timestamp(), b"the-head".as_slice()).unwrap(); + group.finish().unwrap(); + settle().await; + + let stream = Stream::open(&mut crate::driver::Transport::new(h.session.clone()), version) + .await + .unwrap(); + let mut serving = std::pin::pin!( + h.publisher + .clone() + .run_subscribe_stream(stream, subscribe(Filter::NextObject, None)) + ); + registered(&h, serving.as_mut()).await; + finish_and_retain(&h, serving.as_mut()).await; + + // The FETCH lands an RTT after PUBLISH_DONE. + kio::clock::sleep(Duration::from_millis(100)).await; + let mark = h.log.writes.lock().unwrap().len(); + let mut buf = bytes::Bytes::from(joining_fetch(&h, mark).await.unwrap()); + assert_eq!( + crate::coding::decode_varint(&mut buf, version).unwrap(), + ietf::FetchOk::ID, + "{version}: the joining FETCH was refused" + ); + let ok = crate::coding::decode_buf(&mut buf, version, |r, v| ietf::FetchOk::decode(r, v)).unwrap(); + assert_eq!(ok.end_location, Location { group: 5, object: 1 }, "{version}"); + assert_eq!( + occurrences(&h.log, b"the-head"), + 1, + "{version}: the head was not served" + ); + assert!(h.log.resets().is_empty(), "{version}"); + + // Closing the request drops the join, and a FETCH after that is refused. + assert!(futures::poll!(serving.as_mut()).is_pending(), "{version}"); + assert!(!h.publisher.joins.read().is_empty(), "{version}"); + h.session.close(Close::Fin); + serving.await.unwrap(); + assert!( + h.publisher.joins.read().is_empty(), + "{version}: the join outlived its request" + ); + h.session.reopen(); + assert_eq!(joining_refusal(&h, version).await, INVALID_JOINING_REQUEST_ID); + } + } + + /// A REQUEST_UPDATE that crossed PUBLISH_DONE is skipped, not taken as a protocol + /// violation, and the join stays until the subscriber's FIN. + #[kio::test] + async fn an_update_after_publish_done_keeps_the_join() { + for version in JOINING_DRAFTS { + // One script shared by every stream, so the update can arrive after PUBLISH_DONE. + let mut h = serve_session( + version, + crate::track::Info::timed(), + crate::origin::Route::default(), + ScriptedSession::new(Vec::new()), + ); + publish_groups(&mut h, 5); + settle().await; + let stream = Stream::open(&mut crate::driver::Transport::new(h.session.clone()), version) + .await + .unwrap(); + let mut serving = std::pin::pin!( + h.publisher + .clone() + .run_subscribe_stream(stream, subscribe(Filter::NextObject, None)) + ); + registered(&h, serving.as_mut()).await; + finish_and_retain(&h, serving.as_mut()).await; + + // REQUEST_UPDATE, Request ID 2, no parameters. + h.session.push(&[0x02, 0, 2, 2, 0]); + for _ in 0..10 { + assert!(futures::poll!(serving.as_mut()).is_pending(), "{version}"); + kio::clock::sim::yield_now().await; + } + assert!(h.log.closes().is_empty(), "{version}: the update closed the session"); + assert!( + !h.publisher.joins.read().is_empty(), + "{version}: the update dropped the join" + ); + + h.session.close(Close::Fin); + serving.await.unwrap(); + assert!(h.publisher.joins.read().is_empty(), "{version}"); + } + } + + /// A peer holding back flow control parks PUBLISH_DONE, and the kept join still goes + /// once the subscription's request token is revoked meanwhile. + #[kio::test] + async fn a_kept_join_ends_with_its_token_while_publish_done_waits() { + for version in JOINING_DRAFTS { + let origin = crate::origin::Config::new(crate::Hop::new(1).unwrap()).produce(); + let broadcast = origin.publish("room", crate::origin::Route::default()).unwrap(); + let track = broadcast.create_track("video", crate::track::Info::timed()).unwrap(); + let mut group = track.create_group(group::Info { sequence: 0 }).unwrap(); + group.write_frame(timestamp(), b"frame".to_vec()).unwrap(); + group.finish().unwrap(); + settle().await; + + let bi = kio::Producer::new(true); + let session = SinkSession::gated_bi(bi.consume()); + let peer_setup = peer::PeerSetup::default(); + peer_setup.set(peer::Peer::default()); + let auth = crate::auth::Link::new(false); + let mut requests = auth.handle().requests(); + // A limit of nothing, so only the token admits the subscription. + auth.handle().authorize(&crate::auth::Grant::default()); + let publisher = Publisher::new( + crate::time::sim(), + crate::driver::Transport::new(session.clone()), + origin.consume(), + Control::new(false), + None, + peer_setup, + version, + ) + .with_auth(auth.clone()); + + let stream = Stream::open(&mut crate::driver::Transport::new(session.clone()), version) + .await + .unwrap(); + let mut serving = std::pin::pin!(publisher.clone().run_subscribe_stream(stream, token_subscribe())); + let accepted = async { requests.next().await.expect("a request") }; + let request = match futures::future::select(std::pin::pin!(accepted), serving.as_mut()).await { + futures::future::Either::Left((request, _)) => request, + futures::future::Either::Right((res, _)) => panic!("{version}: ended unverified: {res:?}"), + }; + let enforced = request.accept(crate::auth::Grant::all()); + let joined = publisher.joins.wait(|joins| match joins.get(&RequestId(REQUEST_ID)) { + Some(Some(_)) => Poll::Ready(()), + _ => Poll::Pending, + }); + match futures::future::select(std::pin::pin!(joined), serving.as_mut()).await { + futures::future::Either::Left(_) => {} + futures::future::Either::Right((res, _)) => panic!("{version}: subscription ended: {res:?}"), + } + + // The request stream takes nothing more, so PUBLISH_DONE parks. + let Ok(mut writable) = bi.write() else { + panic!("request stream gate closed"); + }; + *writable = false; + drop(writable); + track.finish().unwrap(); + for _ in 0..50 { + assert!(futures::poll!(serving.as_mut()).is_pending(), "{version}"); + settle().await; + } + assert!( + !publisher.joins.read().is_empty(), + "{version}: the finished join was not kept" + ); + + enforced.revoke(crate::SessionError::Unauthorized, "revoked"); + for _ in 0..10 { + let _ = futures::poll!(serving.as_mut()); + settle().await; + } + assert!( + publisher.joins.read().is_empty(), + "{version}: the join outlived its grant while PUBLISH_DONE waited" + ); + } + } + + /// A subscription the subscriber cancels before the track ends keeps nothing for a later + /// joining FETCH. + #[kio::test] + async fn an_unsubscribed_join_is_not_kept() { + for version in JOINING_DRAFTS { + let mut h = serve(version); + publish_groups(&mut h, 5); + settle().await; + let stream = Stream::open(&mut crate::driver::Transport::new(h.session.clone()), version) + .await + .unwrap(); + let mut serving = std::pin::pin!( + h.publisher + .clone() + .run_subscribe_stream(stream, subscribe(Filter::NextObject, None)) + ); + registered(&h, serving.as_mut()).await; + h.session.close(Close::Fin); + serving.await.unwrap(); + assert!(h.publisher.joins.read().is_empty(), "{version}"); + h.session.reopen(); + assert_eq!(joining_refusal(&h, version).await, INVALID_JOINING_REQUEST_ID); + } + } + + /// The session closing frees every join a finished subscription kept. + #[kio::test] + async fn a_session_close_frees_every_kept_join() { + for version in JOINING_DRAFTS { + let mut h = serve(version); + publish_groups(&mut h, 5); + settle().await; + let mut servings = Vec::new(); + for id in [REQUEST_ID, REQUEST_ID + 2] { + let stream = Stream::open(&mut crate::driver::Transport::new(h.session.clone()), version) + .await + .unwrap(); + let mut msg = subscribe(Filter::NextObject, None); + msg.request_id = RequestId(id); + let mut serving = Box::pin(h.publisher.clone().run_subscribe_stream(stream, msg)); + registered_as(&h, RequestId(id), serving.as_mut()).await; + servings.push(serving); + } + h.track.finish().unwrap(); + for _ in 0..50 { + for serving in &mut servings { + assert!(futures::poll!(serving.as_mut()).is_pending(), "{version}"); + } + kio::clock::sim::yield_now().await; + } + assert_eq!( + h.publisher.joins.read().len(), + 2, + "{version}: the finished joins were not kept" + ); + + crate::transport::poll::Session::close(&h.session, 0, "done"); + for serving in servings { + serving.await.unwrap(); + } + assert!(h.publisher.joins.read().is_empty(), "{version}"); + } + } + + /// How a token subscription's grant ends after its track did. + #[derive(Clone, Copy, Debug)] + enum GrantEnd { + Expired, + Revoked, + } + + /// A join kept past the track's end reads the cache on the subscription's request token, + /// so it goes once that grant expires or is revoked, and a late FETCH is refused. + #[kio::test] + async fn a_kept_join_ends_with_its_request_token() { + const VERSION: Version = Version::Draft18; + for end in [GrantEnd::Expired, GrantEnd::Revoked] { + let (auth, mut requests, _cred) = auth_covering_other(); + let mut h = serve_with_auth(VERSION, auth, Vec::new()); + let rt = h.publisher.runtime.clone(); + publish_groups(&mut h, 5); + settle().await; + + let stream = Stream::open(&mut crate::driver::Transport::new(h.session.clone()), VERSION) + .await + .unwrap(); + let mut serving = std::pin::pin!(h.publisher.clone().run_subscribe_stream(stream, token_subscribe())); + let accepted = async { requests.next().await.expect("a request") }; + let request = match futures::future::select(std::pin::pin!(accepted), serving.as_mut()).await { + futures::future::Either::Left((request, _)) => request, + futures::future::Either::Right((res, _)) => panic!("{end:?}: ended unverified: {res:?}"), + }; + let enforced = request.accept(grant_all_expiring(&rt, Some(60))); + registered(&h, serving.as_mut()).await; + finish_and_retain(&h, serving.as_mut()).await; + + match end { + GrantEnd::Expired => kio::clock::sleep(Duration::from_secs(61)).await, + GrantEnd::Revoked => { + kio::clock::sleep(Duration::from_secs(1)).await; + assert!(futures::poll!(serving.as_mut()).is_pending(), "{end:?}"); + enforced.revoke(crate::SessionError::Unauthorized, "revoked"); + } + } + let _ = kio::clock::timeout(Duration::from_secs(1), serving.as_mut()) + .await + .unwrap_or_else(|_| panic!("{end:?}: the join outlived its grant")); + assert!(h.publisher.joins.read().is_empty(), "{end:?}"); + assert_eq!( + joining_refusal(&h, VERSION).await, + INVALID_JOINING_REQUEST_ID, + "{end:?}" + ); + } + } + #[kio::test] async fn a_joining_fetch_rejects_unsupported_filters() { for version in JOINING_DRAFTS { diff --git a/rs/moq-net/src/lite/test_transport.rs b/rs/moq-net/src/lite/test_transport.rs index 60455c595ed..231559b8777 100644 --- a/rs/moq-net/src/lite/test_transport.rs +++ b/rs/moq-net/src/lite/test_transport.rs @@ -1114,6 +1114,11 @@ impl ScriptedSession { } } + /// Undo [`Self::close`], so streams opened from here park once their script runs out. + pub fn reopen(&self) { + *self.close.lock().unwrap() = None; + } + /// Answer each stream from `scripts`, but only once the gate opens: a peer that /// replies normally after its transport header gets send capacity. pub fn gated_open(scripts: Vec>, gate: kio::Consumer) -> Self { diff --git a/rs/moq-net/tests/integration/track_tail.rs b/rs/moq-net/tests/integration/track_tail.rs index 35a6ad81d30..511472fd864 100644 --- a/rs/moq-net/tests/integration/track_tail.rs +++ b/rs/moq-net/tests/integration/track_tail.rs @@ -288,6 +288,63 @@ async fn lite07_empty_track_ends_without_the_grace() { assert!(outcome.elapsed < GRACE / 10, "ended after {:?}", outcome.elapsed); } +/// Before draft 22, a group written before SUBSCRIBE_OK sits below its Largest Object, so +/// its head arrives on the joining FETCH sent once SUBSCRIBE_OK lands. A track that ends as +/// the subscription starts has ended it by the time that FETCH reaches the publisher, an RTT +/// later, and still ends cleanly with the head. +#[kio::test] +async fn draft18_a_join_reaching_a_finished_track_keeps_its_head() { + const RTT: Duration = Duration::from_millis(100); + + let Pair { + pair, + track, + remote, + _keep, + } = connect("moq-transport-18").await; + pair.client_transport.set_latency(RTT / 2); + let mut group = track.append_group().unwrap(); + group.write_frame(Timestamp::ZERO, PAYLOAD).unwrap(); + group.finish().unwrap(); + + let reader = kio::clock::sim::spawn(async move { + let mut sub = remote + .track("video") + .unwrap() + .subscribe(moq_net::track::Subscription::default().with_range(0..)) + .await + .expect("subscribe"); + let mut frames = Vec::new(); + let err = loop { + let mut group = match sub.recv_group().await { + Ok(Some(group)) => group, + Ok(None) => break None, + Err(err) => break Some(err), + }; + while let Some(frame) = group.read_frame().await.expect("group failed") { + frames.push(frame.payload.to_vec()); + } + }; + (frames, err) + }); + + kio::clock::timeout(TIMEOUT, support::harness::subscribed(&track)) + .await + .expect("no subscriber appeared"); + // Kept alive, as a publisher draining its session does: a finished track whose producer + // is gone lets its cache reclaim the head, and that FETCH is refused. + track.finish().unwrap(); + let _track = track; + + let (frames, err) = kio::clock::timeout(TIMEOUT, reader) + .await + .expect("the subscription never ended") + .expect("reader panicked"); + assert!(err.is_none(), "{err:?}"); + assert_eq!(frames, [PAYLOAD], "the head was lost"); + drop((pair, _keep)); +} + /// IETF drafts with a joining FETCH (18) and with subscription fills (22). const IETF: &[&str] = &["moq-transport-18", "moq-transport-22"];