Repository navigation
fix(net): a subscribe update racing a finished track ends it cleanly #5833
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
db02953
e08fade
8296627
074066a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,7 +23,7 @@ import { Cost, type Hop, MAX_HOPS, type Route, routesEqual, UNKNOWN_HOP } from " | |
| import { groupBounds, hiddenBelow, scopeCaptures, scopeHead, scopeOverlaps } from "../internal.ts"; | ||
| import * as Path from "../path.ts"; | ||
| import { sendOrder } from "../priority.ts"; | ||
| import { type OpenOptions, type Reader, Stream } from "../stream.ts"; | ||
| import { type OpenOptions, type Reader, Stream, Writer } from "../stream.ts"; | ||
| import { TAIL_GRACE_MS, Tail } from "../tail.ts"; | ||
| import * as Time from "../time.ts"; | ||
| import type * as track from "../track.ts"; | ||
|
|
@@ -768,8 +768,8 @@ export class Subscriber { | |
|
|
||
| // Terminal conditions (stream end, track close, a failed subscription update) settle at most | ||
| // once; race them into one stable promise so the demand loop doesn't re-subscribe each pass. | ||
| // Updates stop quietly at the FIN, which can land before the responses ahead of it are | ||
| // decoded, so only their failure is terminal on its own. | ||
| // Updates stop quietly at the FIN or a failed write, either of which can land before the | ||
| // responses ahead of it are decoded, so only a refused update is terminal on its own. | ||
| const terminal: PromiseLike<unknown>[] = [closed, producer.closed]; | ||
| if (subscriptionUpdates !== undefined) terminal.push(subscriptionUpdates.then(() => closed)); | ||
| const done = race(terminal); | ||
|
|
@@ -1237,7 +1237,7 @@ export class Subscriber { | |
| /** | ||
| * Send SUBSCRIBE_UPDATE messages whenever the track's aggregate subscription changes. | ||
| * | ||
| * Resolves cleanly when the stream or track closes, so the caller can include | ||
| * Resolves cleanly when the stream or track closes, or a write fails, so the caller can include | ||
| * this in a race without leaving a dangling pending write that would | ||
| * become an unhandled rejection if the user calls update after close. | ||
| * | ||
|
|
@@ -1293,7 +1293,17 @@ export class Subscriber { | |
| startGroup: current.groups?.start === undefined ? undefined : bounds.start, | ||
| endGroup: inclusiveGroupEnd(bounds.end), | ||
| }); | ||
| await update.encode(stream.writer, this.version); | ||
| // Encoded apart from the write, so a bound the wire can't carry still fails the track. | ||
| const encoded: Uint8Array[] = []; | ||
| const buffer = new WritableStream<Uint8Array>({ write: (chunk) => void encoded.push(chunk.slice()) }); | ||
| await update.encode(new Writer(buffer, this.version), this.version); | ||
| // A failed write means the publisher stopped reading, as it does once it finishes the | ||
| // track. The responses decide how the track ends, cleanly at the FIN or with a reset's error. | ||
| try { | ||
| for (const chunk of encoded) await stream.writer.write(chunk); | ||
| } catch { | ||
| return; | ||
|
Comment on lines
+1302
to
+1305
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When the failed update lowers the start after Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Real, but I'm keeping it out of this PR. It predates the failed-write path: the owed range comes from the current demand, not what the publisher acknowledged. So a lowered-start update that crosses SUBSCRIBE_END on the wire, with a write that succeeds, already makes the tail wait out its grace the same way. It only affects lite-05/06 (lite-07 counts streams), and the track still ends cleanly, at worst one grace late. Before this PR, the same race failed the track outright. Rolling back state only on a failed write would cover one of the two crossings. The fix belongs in how the tail bounds its debt once the publisher has ended (demand made after its END owes nothing), in both languages. I've suggested that as a follow-up. (Written by Claude Opus 5.5) |
||
| } | ||
| lastSent = { ...current }; | ||
| console.debug(`subscribe update: id=${id} broadcast=${broadcast} track=${track.name}`); | ||
| } | ||
|
|
||
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2883,6 +2883,79 @@ mod tests { | |
| )); | ||
| } | ||
|
|
||
| /// A publisher that finishes the track stops reading its subscribe stream, so a | ||
| /// SUBSCRIBE_UPDATE written then fails. The track still ends with the publisher's | ||
| /// finish; a reset still hands it back with the reset's error. | ||
| #[moq_net_sim::test] | ||
| async fn an_update_the_publisher_stopped_reading_defers_to_its_end() { | ||
| use crate::transport::poll::{RecvStream as _, SendStream as _, Session as _}; | ||
| for reset in [false, true] { | ||
| let version = Version::Lite06; | ||
| let (session, mut peer) = crate::transport::memory::pair(Default::default()); | ||
| let subscriber = Subscriber::new(SubscriberConfig { | ||
| runtime: crate::time::Clock::sim(), | ||
| session, | ||
| origin: origin::Config::new(crate::Hop::new(1).unwrap()).produce(), | ||
| recv_bandwidth: None, | ||
| version, | ||
| peer_setup: crate::lite::PeerSetup::new(false, crate::transport::Binding::InBand), | ||
| peer_hop: None, | ||
| cost: None, | ||
| going_away: Default::default(), | ||
| auth: crate::auth::Handle::new(false), | ||
| }); | ||
| let serve = TrackServe { | ||
| validity: Default::default(), | ||
| subscriber, | ||
| path: Path::new("room").to_owned(), | ||
| epoch: None, | ||
| name: "video".into(), | ||
| }; | ||
| let broadcast = crate::broadcast::Info::new().produce(); | ||
| let mut serving = ServeLoop::new( | ||
| &serve, | ||
| broadcast.reserve_track("video").unwrap(), | ||
| Default::default(), | ||
| None, | ||
| ); | ||
| let mut reader = serving.serving.subscribe(None); | ||
| assert!(serving.poll(&serve, &kio::Waiter::noop()).is_pending()); | ||
| assert!(matches!(serving.sub, Sub::Active(_)), "the SUBSCRIBE is on the wire"); | ||
|
|
||
| // The publisher ends the subscription and stops reading the stream. | ||
| let (mut send, mut recv) = std::future::poll_fn(|cx| peer.poll_accept_bi(cx)).await.unwrap(); | ||
| recv.stop(crate::StreamError::Cancel.to_code()); | ||
| if reset { | ||
| send.reset(crate::StreamError::NotFound.to_code()); | ||
| } else { | ||
| let mut end = Vec::new(); | ||
| lite::SubscribeResponse::End(lite::SubscribeEnd { group: 0, streams: 0 }) | ||
| .encode(&mut crate::coding::Encoder::new(&mut end, version.into()), version) | ||
| .unwrap(); | ||
| std::future::poll_fn(|cx| send.poll_write(cx, &end)).await.unwrap(); | ||
| send.finish().unwrap(); | ||
| } | ||
|
|
||
| // The update races it, and its write fails. | ||
| reader.update(Subscription::default().with_priority(1)).unwrap(); | ||
| let gate = serve.gate(); | ||
| let mut running = TrackServeRun { | ||
| serve, | ||
| state: TrackRunState::Serve(serving), | ||
| gate, | ||
| }; | ||
| assert!(kio::Task::poll(&mut running, &kio::Waiter::noop()).is_ready()); | ||
| let end = reader.recv_group().now_or_never().expect("the track ended").map(|_| ()); | ||
| match reset { | ||
| false => assert!(matches!(end, Ok(())), "{end:?}"), | ||
| true => assert!( | ||
| matches!(end, Err(Error::Stream(crate::StreamError::NotFound))), | ||
| "{end:?}" | ||
| ), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /// Mid-group demand: a subscriber resuming at frame 3 of group 5, capped by a later | ||
| /// bound in the same group. | ||
| fn mid_group_demand() -> Subscription { | ||
|
|
@@ -5530,12 +5603,11 @@ impl<S: crate::transport::poll::Session> ServeLoop<S> { | |
| if let Sub::Active(active) = &mut self.sub { | ||
| match active.stream.writer.poll_flush(&mut cx) { | ||
| Poll::Ready(Ok(())) => {} | ||
| // The publisher stopped reading, as it does once it finishes the | ||
| // track. The update is dropped, and the responses decide how the | ||
| // subscription ends: cleanly at the FIN, or with a reset's error. | ||
| Poll::Ready(Err(err)) => { | ||
| // The stream is broken; drop it (the writer resets) and | ||
| // hand the track back. | ||
| serve.subscriber.remove_subscribe(active.id); | ||
| self.sub = Sub::None; | ||
| return Poll::Ready(ServeEnd::GiveBack(err)); | ||
| tracing::debug!(track = %serve.name, %err, "subscribe update dropped") | ||
| } | ||
|
Comment on lines
5609
to
5611
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a failed update lowered the floor after Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Real, but I'm keeping it out of this PR. It predates the failed-write path: the owed range comes from the current demand, not what the publisher acknowledged. So a lowered-start update that crosses SUBSCRIBE_END on the wire, with a write that succeeds, already makes the tail wait out its grace the same way. It only affects lite-05/06 (lite-07 counts streams), and the track still ends cleanly, at worst one grace late. Before this PR, the same race failed the track outright. Rolling back state only on a failed write would cover one of the two crossings. The fix belongs in how the tail bounds its debt once the publisher has ended (demand made after its END owes nothing), in both languages. I've suggested that as a follow-up. (Written by Claude Opus 5.5) |
||
| Poll::Pending => return Poll::Pending, | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On pre-Lite-06 connections, an update whose start group is
Number.MAX_SAFE_INTEGERreaches this catch as a localRangeError, not a stopped-stream error:groupBoundsaccepts that safe integer, butencodeStartGroupadds one for the older wire format andWriter.u53rejects the resulting value. Returning here leaves the track open while silently ignoring its requested bound; restrict suppression to transport write failures, or reject the unencodable bound before this catch, so codec errors still surface.AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, fixed in 074066a. The update is now encoded into memory before the stream write, and only a failed write defers to the responses. An unencodable bound (your lite-05
MAX_SAFE_INTEGERstart) fails the track again, covered byan update whose bound the wire can't carry fails the track. Rust already keepsbuffer_update(encode, propagated with?) apart from the flush, so it needed no change.(Written by Claude Opus 5.5)