diff --git a/quest/m1/explicit-state/README.md b/quest/m1/explicit-state/README.md index cb1a3ff9a6e..3e5d877b096 100644 --- a/quest/m1/explicit-state/README.md +++ b/quest/m1/explicit-state/README.md @@ -49,8 +49,9 @@ Decided 2026-11-07 (maintainer): REVIEW.md's State check and its AGENTS.md line). Not chosen: REVIEW.md only, or no rule change. - ✅ `quic::driver::Endpoint::rebind` becomes crate-private, since only - tests call it (see [Transport close](/quest/m1/explicit-state/transport-close.md)). - Not chosen: refusing to rebind after close. + tests call it (done: test-only, and unreleased, so it broke no published + surface and needed no m2 quest). Not chosen: refusing to rebind after + close. Every child reproduces each folded bug in a test before fixing it. A bug that doesn't reproduce is dropped from the child, and its PR says why. A @@ -69,7 +70,6 @@ which collapses that state and waits on [Drop set](/quest/m1/drop-set.md). ## Required - [moq-boy pause state](/quest/m1/explicit-state/boy-pause.md) - a new viewer always wakes a paused emulator -- [Transport close](/quest/m1/explicit-state/transport-close.md) - connection status ends with its error, and every Rust QUIC backend agrees when a connection is closed - [Video codec drains](/quest/m1/explicit-state/video-drain.md) - a hardware codec whose drain fails is ready for input again or terminally failed, never left stuck - [Publish capture phases](/quest/m1/explicit-state/publish-capture.md) - dismissing the screen-share picker turns sharing off - [js/qmux close](/quest/m1/explicit-state/qmux-close.md) - a graceful remote close ends qmux streams cleanly diff --git a/quest/m1/explicit-state/transport-close.md b/quest/m1/explicit-state/transport-close.md deleted file mode 100644 index 20fbd8c0a7d..00000000000 --- a/quest/m1/explicit-state/transport-close.md +++ /dev/null @@ -1,76 +0,0 @@ -# [L] Transport close - -## Goal - -The Rust connection and QUIC transport state each live in one phase value: - -- Once moq-tokio's reconnecting connection gives up, its status stream yields - the terminal error next, never a `Disconnected` first, even on a handle - that never connected. The bindings that expose status report the same - sequence. -- The moq-tokio, moq-uring, and qmux backends agree on when `closed()` - resolves for a local close. -- A closed or driverless endpoint can't be revived. - -## Plan - -The shared decisions are in the [epic](/quest/m1/explicit-state/README.md). -Decided 2026-11-07 (maintainer): ✅ `quic::driver::Endpoint::rebind` becomes -crate-private, since only tests call it. Not chosen: refusing to rebind after -close. This is breaking, so if the rest of this child stays additive, split -the `rebind` change into its own m2 quest. - -Bugs to reproduce and fix: - -- **Trailing `Disconnected`** (`rs/moq-tokio/src/connection.rs`): - - `Shared::end` calls `state.offline()`, which writes - `status = Some(Disconnected)` in the same revision that sets `ended = true`. - - `poll_status` matches only `status`, and kio's `Consumer::poll` runs the - closure before it reports a closed channel (`rs/kio/src/consumer.rs`). So - the caller gets `Ok(Disconnected)` first and only then `Err(terminal)`. - - That contradicts `poll_status`'s doc and `end`'s own comment. - - Recommended: `enum Phase { Connecting { failure }, Live { session, - version, transport, migrating: Option }, Offline { failure }, - Closed(Option) }`, with `Status` and `Observation` derived from it. -- **`rebind` after close** (`rs/moq-tokio/src/quic/driver/endpoint.rs`): - - `rebind` checks neither `driver_lost` nor the pending close, and - `slot.replace(socket)` refills a released slot. - - The result is a bound port that nothing reads. - - Recommended: `enum Lifecycle { Running, Closing { code, reason }, - Released, DriverLost }` on the endpoint state, checked by every entry - point. - -Instances to sweep: - -- `rs/moq-quic/src/connection/mux/mod.rs`, QMux `Connection`: - - `params_sent`, `peer`, `error`, `close`, `close_sent`, and a reused - `conn_state`, with `error.is_some()` checked at about 11 sites. - - `conn_state` starts as `Established` while handshaking. - - Recommended: `Phase { Handshake, Established(Params), Closing { error, - frame }, Closed(error) }`. -- `rs/moq-uring/src/quic/driver/connection.rs` and `rs/qmux/src/session.rs`: - - `closed`, `local_close`, `dead`, and `close_staged` beside the connection. - - moq-tokio resolves `poll_closed` on a local close at once, while the - other two wait for a staged latch. Settle on one behavior. -- `rs/moq-quic/src/connection/streams/recv.rs` and `lease.rs`: `stopped` plus - `evicted`. Also `send.rs`: the peer's STOP lives only in `stop_reason`, and - `fin_pending` and `released` sit beside `SendState`. -- `rs/moq-quic/src/connection/paths.rs`: `challenge` plus - `challenge_pending` plus `validated`. -- `rs/moq-quic/src/connection/mod.rs`: `close` and `draining_reported` beside - `State`. -- `rs/moq-tokio/src/quic/web/session.rs`: WebTransport and raw sessions share - one struct of `Option`s. `rs/moq-uring/src/quic/web.rs` already groups - most of it. -- `rs/moq-relay/src/cluster.rs` (`DialersPhase`) and - `rs/moq-relay/src/shutdown.rs` (`Serving`, whose `closed()` underflows if - it is called twice). -- `rs/qmux/src/driver.rs` (`Timer.closing` duplicates `Phase::Closing`) and - `rs/qmux/src/session.rs` (`SendStream` keeps `fin` and `reset`, and folds - STOP into `fin`). - -Keep the BBR congestion fields that mirror the draft spec as they are unless -a phase enum makes them clearer: they port a published algorithm. - -[moq-dev/moq#8139](https://github.com/moq-dev/moq/pull/8139) edits -`connection.rs`'s tests only. diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index cf1778ce298..118fc2cfc84 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -3102,7 +3102,7 @@ impl kio::Task for DatagramServe { self.state, DatagramState::Ending { .. } | DatagramState::Finish { .. } | DatagramState::Done ) && self.gate.poll_denied(waiter).is_ready() - && self.closed.poll_closed(waiter).is_pending() + && self.closed.poll_ending(waiter).is_pending() { tracing::info!(broadcast = %self.subscriber.log_path(&self.path), track = %self.name(), "datagrams no longer authorized"); self.abort(Error::Unauthorized); @@ -10856,7 +10856,7 @@ impl kio::Task for TrackServeRun { // its tracks end. if !matches!(self.state, TrackRunState::Ending { .. } | TrackRunState::Finish { .. }) && self.gate.poll_denied(waiter).is_ready() - && self.serve.subscriber.session.clone().poll_closed(waiter).is_pending() + && self.serve.subscriber.session.poll_ending(waiter).is_pending() { self.revoke(); return Poll::Ready(()); diff --git a/rs/moq-net/src/session.rs b/rs/moq-net/src/session.rs index de31bf97706..0c478f925f8 100644 --- a/rs/moq-net/src/session.rs +++ b/rs/moq-net/src/session.rs @@ -69,6 +69,14 @@ impl Closing { sent.get_or_insert_with(|| (code, cause.cloned())); } + /// Whether this side closed the transport. + pub(crate) fn is_sent(&self) -> bool { + self.sent + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .is_some() + } + /// Whether the handles closed the transport by dropping or draining. pub(crate) fn is_local(&self) -> bool { self.local.load(std::sync::atomic::Ordering::Relaxed) diff --git a/rs/moq-net/src/transport/boxed.rs b/rs/moq-net/src/transport/boxed.rs index b89739ae030..91350aaeabe 100644 --- a/rs/moq-net/src/transport/boxed.rs +++ b/rs/moq-net/src/transport/boxed.rs @@ -117,6 +117,17 @@ impl Session { &self.closing } + /// Resolve once the session is ending: this side closed the transport, or it closed. + /// + /// Unlike `poll_closed`, which a backend holds until its close is on the wire, this sees a + /// local close the moment it is sent. + pub(crate) fn poll_ending(&self, waiter: &kio::Waiter) -> Poll<()> { + if self.closing.is_sent() { + return Poll::Ready(()); + } + poll::Session::poll_closed(self, waiter).map(|_| ()) + } + /// The subscriptions and fetches the session's tasks carry over this transport. pub(crate) fn work(&self) -> &crate::session::Work { &self.work diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index c02ba252daa..2fd1328c900 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -68,7 +68,7 @@ mod packet_crypto; use packet_crypto::PrevCrypto; mod paths; -use paths::{PathData, PathResponses}; +use paths::{Challenge, PathData, PathResponses}; pub(crate) mod qlog; use qlog::QlogSink; @@ -191,6 +191,9 @@ pub struct Connection { retry_src_cid: Option, endpoint_events: VecDeque, /// Whether `EndpointEventInner::Draining` was emitted, which happens once. + /// + /// Not derived from `state`: a server's first Initial can carry a CONNECTION_CLOSE, which + /// reaches `Draining` without reporting it, since that packet skips `finish_packet`. draining_reported: bool, /// Whether the spin bit is in use for this connection spin_enabled: bool, @@ -234,7 +237,6 @@ pub struct Connection { // /// Responses to PATH_CHALLENGE frames path_responses: PathResponses, - close: bool, // // ACK frequency @@ -377,7 +379,6 @@ impl Connection { packet_number_filter: PacketNumberFilter::new(&mut rng), path_responses: PathResponses::default(), - close: false, ack_frequency: AckFrequencyState::new(get_max_ack_delay(&TransportParameters::default())), next_bundled_ack_time: None, @@ -491,7 +492,7 @@ impl Connection { pub fn streams(&mut self) -> Streams<'_> { Streams { state: &mut self.streams, - conn_state: &self.state, + closed: self.state.is_closed(), } } @@ -522,7 +523,7 @@ impl Connection { id, state: &mut self.streams, pending: &mut self.spaces[SpaceId::Data].pending, - conn_state: &self.state, + closed: self.state.is_closed(), } } @@ -584,7 +585,11 @@ impl Connection { // The peer's maximum UDP payload size can reduce the path MTU below our configured minimum. let min_mtu = Ord::min(self.config.min_mtu, self.path.current_mtu()); - if let Some(challenge) = self.send_path_challenge(now, buf) { + // A closing endpoint sends only CONNECTION_CLOSE (RFC 9000 §10.2.1), so a challenge the + // previous path still owes stays unsent. + if !self.state.is_closed() + && let Some(challenge) = self.send_path_challenge(now, buf) + { return Some(challenge); } @@ -610,10 +615,10 @@ impl Connection { self.app_limited = true; return None; } - State::Draining | State::Closed(_) => { - // self.close is only reset once the associated packet had been + State::Draining { send_close } | State::Closed(state::Closed { send_close, .. }) => { + // `send_close` is only reset once the associated packet had been // encoded successfully - if !self.close { + if !send_close { self.app_limited = true; return None; } @@ -1003,7 +1008,7 @@ impl Connection { } else if buf.len() + frame::ConnectionClose::SIZE_BOUND <= builder.max_size { let max_frame_size = builder.max_size - buf.len(); match &self.state { - State::Closed(state::Closed { reason }) => { + State::Closed(state::Closed { reason, .. }) => { if space_id == SpaceId::Data || reason.is_transport_layer() { reason.encode(buf, max_frame_size) } else { @@ -1015,7 +1020,7 @@ impl Connection { .encode(buf, max_frame_size) } } - State::Draining => frame::ConnectionClose { + State::Draining { .. } => frame::ConnectionClose { error_code: TransportErrorCode::NO_ERROR, frame_type: None, reason: String::new(), @@ -1026,7 +1031,7 @@ impl Connection { } if space_id == self.highest_space { // Don't send another close packet - self.close = false; + self.state.close_sent(); // `CONNECTION_CLOSE` is the final packet break; } else { @@ -1213,13 +1218,9 @@ impl Connection { /// Send PATH_CHALLENGE for a previous path if necessary fn send_path_challenge(&mut self, now: Instant, buf: &mut Vec) -> Option { let (prev_cid, prev_path) = self.prev_path.as_mut()?; - if !prev_path.challenge_pending { - return None; - } - prev_path.challenge_pending = false; - let token = prev_path - .challenge - .expect("previous path challenge pending without token"); + let challenge = prev_path.challenge.as_mut().filter(|challenge| challenge.pending)?; + challenge.pending = false; + let token = challenge.token; let remote = prev_path.remote; debug_assert_eq!( self.highest_space, @@ -1415,7 +1416,6 @@ impl Connection { self.set_loss_detection_timer(now); } self.path.challenge = None; - self.path.challenge_pending = false; } Timer::Pacing => trace!("pacing timer expired"), Timer::PushNewCid => { @@ -1482,8 +1482,10 @@ impl Connection { let unanswered = matches!(&self.state, State::Handshake(state) if !state.rem_cid_set); self.close_common(); self.set_close_timer(now); - self.close = true; - self.state = State::Closed(state::Closed { reason }); + self.state = State::Closed(state::Closed { + reason, + send_close: true, + }); if unanswered { self.report_draining(); } @@ -2740,7 +2742,7 @@ impl Connection { ) { // State transitions for error cases if let Err(Error::Reset) = result - && matches!(self.state, State::Closed(_) | State::Draining | State::Drained) + && matches!(self.state, State::Closed(_) | State::Draining { .. } | State::Drained) { // Preserve the original close reason once the connection has closed. self.state = State::Drained; @@ -2761,7 +2763,7 @@ impl Connection { debug!("closing connection due to transport error: {}", err); State::closed(err) } - Error::VersionMismatch => State::Draining, + Error::VersionMismatch => State::Draining { send_close: false }, Error::LocallyClosed { .. } => { unreachable!("LocallyClosed isn't generated by packet processing"); } @@ -2777,7 +2779,7 @@ impl Connection { self.set_close_timer(now); } } - if matches!(self.state, State::Draining | State::Drained) { + if matches!(self.state, State::Draining { .. } | State::Drained) { self.report_draining(); } if !was_drained && self.state.is_drained() { @@ -2788,8 +2790,9 @@ impl Connection { } // Transmit CONNECTION_CLOSE if necessary - if let State::Closed(_) = self.state { - self.close = remote == self.path.remote; + let on_path = remote == self.path.remote; + if let State::Closed(closed) = &mut self.state { + closed.send_close = on_path; } } @@ -2811,7 +2814,8 @@ impl Connection { } return Ok(()); } - State::Closed(_) => { + State::Closed(closed) => { + let send_close = closed.send_close; for result in frame::Iter::new(packet.payload.freeze())? { let frame = match result { Ok(frame) => frame, @@ -2829,13 +2833,14 @@ impl Connection { if let Frame::Close(_) = frame { trace!("draining"); - self.state = State::Draining; + // A close still due goes out once more, as the draining state's one reply. + self.state = State::Draining { send_close }; break; } } return Ok(()); } - State::Draining | State::Drained => return Ok(()), + State::Draining { .. } | State::Drained => return Ok(()), State::Handshake(state) => state, }; // Whether a server Initial was processed or a Retry accepted. Packets the client discards @@ -3050,7 +3055,7 @@ impl Connection { // through to the catch-all arm below. Frame::Close(reason @ Close::Connection(_)) => { self.lose(reason.into()); - self.state = State::Draining; + self.state = State::Draining { send_close: false }; return Ok(()); } _ => { @@ -3158,14 +3163,15 @@ impl Connection { } } Frame::PathResponse(token) => { - if self.path.challenge == Some(token) && remote == self.path.remote { + if self.path.challenge.is_some_and(|challenge| challenge.token == token) + && remote == self.path.remote + { trace!("new path validated"); self.timers.stop(Timer::PathValidation); self.path.challenge = None; self.path.validated = true; if let Some((_, prev_path)) = &mut self.prev_path { prev_path.challenge = None; - prev_path.challenge_pending = false; } } else { debug!(token, "ignoring invalid PATH_RESPONSE"); @@ -3368,8 +3374,8 @@ impl Connection { if let Some(reason) = close { self.lose(reason.into()); - self.state = State::Draining; - self.close = true; + // Reply with a close of our own, the one packet draining allows (RFC 9000 §10.2.2) + self.state = State::Draining { send_close: true }; } if remote != self.path.remote && !is_probing_packet && number == self.spaces[SpaceId::Data].rx_packet { @@ -3411,8 +3417,7 @@ impl Connection { ) }; new_path.congestion.on_probe_target(self.bandwidth_probe.target()); - new_path.challenge = Some(self.rng.random()); - new_path.challenge_pending = true; + new_path.challenge = Some(Challenge::new(self.rng.random())); let prev_pto = self.pto(SpaceId::Data); let mut prev = mem::replace(&mut self.path, new_path); @@ -3420,8 +3425,7 @@ impl Connection { // Don't clobber the original path if the previous one hasn't been validated yet if prev.challenge.is_none() { - prev.challenge = Some(self.rng.random()); - prev.challenge_pending = true; + prev.challenge = Some(Challenge::new(self.rng.random())); // We haven't updated the remote CID yet, this captures the remote CID we were using on // the previous path. self.prev_path = Some((self.rem_cids.active(), prev)); @@ -3566,9 +3570,10 @@ impl Connection { // PATH_CHALLENGE if buf.len() + 9 < max_size && space_id == SpaceId::Data { // Transmit challenges with every outgoing frame on an unvalidated path - if let Some(token) = self.path.challenge { + if let Some(challenge) = &mut self.path.challenge { // But only send a packet solely for that purpose at most once - self.path.challenge_pending = false; + challenge.pending = false; + let token = challenge.token; sent.non_retransmits = true; sent.requires_padding = true; trace!("PATH_CHALLENGE {:08x}", token); @@ -4079,8 +4084,11 @@ impl Connection { /// See also `self.space(SpaceId::Data).can_send()` fn can_send_1rtt(&self, max_size: usize) -> bool { self.streams.can_send_frames(max_size) - || self.path.challenge_pending - || self.prev_path.as_ref().is_some_and(|(_, x)| x.challenge_pending) + || self.path.challenge.is_some_and(|challenge| challenge.pending) + || self + .prev_path + .as_ref() + .is_some_and(|(_, prev)| prev.challenge.is_some_and(|challenge| challenge.pending)) || !self.path_responses.is_empty() } @@ -4360,14 +4368,28 @@ pub(crate) enum State { Handshake(state::Handshake), Established, Closed(state::Closed), - Draining, + Draining { + /// A packet with a CONNECTION_CLOSE is due + send_close: bool, + }, /// Waiting for application to call close so we can dispose of the resources Drained, } impl State { fn closed>(reason: R) -> Self { - Self::Closed(state::Closed { reason: reason.into() }) + Self::Closed(state::Closed { + reason: reason.into(), + send_close: false, + }) + } + + /// A CONNECTION_CLOSE went out, so none is due until another packet asks for one + fn close_sent(&mut self) { + match self { + Self::Closed(state::Closed { send_close, .. }) | Self::Draining { send_close } => *send_close = false, + Self::Handshake(_) | Self::Established | Self::Drained => {} + } } fn is_handshake(&self) -> bool { @@ -4379,7 +4401,7 @@ impl State { } fn is_closed(&self) -> bool { - matches!(*self, Self::Closed(_) | Self::Draining | Self::Drained) + matches!(*self, Self::Closed(_) | Self::Draining { .. } | Self::Drained) } fn is_drained(&self) -> bool { @@ -4409,6 +4431,9 @@ mod state { #[derive(Clone)] pub(crate) struct Closed { pub(super) reason: Close, + /// A packet with a CONNECTION_CLOSE is due: set on closing, and again by each packet the + /// peer sends on the current path meanwhile + pub(super) send_close: bool, } } @@ -4545,7 +4570,10 @@ mod tests { let mut buf = Vec::new(); assert!(conn.transmit(now, 1, &mut buf).is_some()); assert!(buf.len() <= 1200); - assert!(!conn.close); + assert!(matches!( + conn.state, + State::Closed(state::Closed { send_close: false, .. }) + )); let (packet, rest) = PartialDecode::new(buf.as_slice().into(), &FixedLengthConnectionIdParser::new(0), false).unwrap(); diff --git a/rs/moq-quic/src/connection/mux/mod.rs b/rs/moq-quic/src/connection/mux/mod.rs index 65045e24fc1..4b5d171c65c 100644 --- a/rs/moq-quic/src/connection/mux/mod.rs +++ b/rs/moq-quic/src/connection/mux/mod.rs @@ -18,7 +18,7 @@ use bytes::{Buf, Bytes, BytesMut}; use tracing::{debug, trace}; use super::{ - Error as ConnectionError, State as ConnectionState, ThinRetransmits, + Error as ConnectionError, ThinRetransmits, datagrams::DatagramState, spaces::Retransmits, stats::FrameStats, @@ -127,6 +127,19 @@ fn limit(limit: Option, default: u64) -> VarInt { VarInt::from_u64(limit.map_or(default, NonZeroU64::get)).unwrap_or(VarInt::MAX) } +/// How a [`Connection`] ended, if it has, during the handshake or after it +enum End { + /// Not ended + Open, + /// Ended with `error`; `frame` is the close still to write, if any + Closing { + error: ConnectionError, + frame: Option, + }, + /// Ended with `error`, having written everything it will + Closed(ConnectionError), +} + /// A QMux connection over a reliable, ordered transport pub struct Connection { side: Side, @@ -134,11 +147,14 @@ pub struct Connection { streams: StreamsState, /// Control frames waiting to be written pending: Retransmits, - conn_state: ConnectionState, stats: FrameStats, + /// Our transport parameters went out: the first record carries them, whether the peer's + /// arrived first or the connection already ended params_sent: bool, + /// The peer's transport parameters, once they arrive; they stay in force after the end peer: Option, + end: End, protocol: Option, timer: Timer, @@ -150,10 +166,6 @@ pub struct Connection { datagrams: DatagramState, - /// The close frame waiting to be written - close: Option, - close_sent: bool, - error: Option, /// Draft-00 frames already serialized, one per transport message queued: VecDeque, /// Where records are serialized. Each one handed out is split off, so the allocation is @@ -183,10 +195,10 @@ impl Connection { config, streams, pending: Retransmits::default(), - conn_state: ConnectionState::Established, stats: FrameStats::default(), params_sent: false, peer: None, + end: End::Open, protocol: None, timer, next_ping: 0, @@ -194,9 +206,6 @@ impl Connection { ping_request: false, ping_response: None, datagrams: DatagramState::default(), - close: None, - close_sent: false, - error: None, queued: VecDeque::new(), out: BytesMut::new(), } @@ -214,23 +223,29 @@ impl Connection { /// Whether the peer's transport parameters arrived and the connection is still up pub fn is_established(&self) -> bool { - self.peer.is_some() && self.error.is_none() + self.peer.is_some() && !self.is_closed() } /// Whether the connection ended pub fn is_closed(&self) -> bool { - self.error.is_some() + !matches!(self.end, End::Open) } /// Whether the connection ended and has nothing left to write, so the transport can be /// shut down pub fn is_drained(&self) -> bool { - self.error.is_some() && self.close.is_none() && self.queued.is_empty() + match self.end { + End::Closing { frame: None, .. } | End::Closed(_) => self.queued.is_empty(), + End::Open | End::Closing { frame: Some(_), .. } => false, + } } /// Why the connection ended, if it has pub fn error(&self) -> Option<&ConnectionError> { - self.error.as_ref() + match &self.end { + End::Closing { error, .. } | End::Closed(error) => Some(error), + End::Open => None, + } } /// Wait for the peer's transport parameters @@ -261,7 +276,7 @@ impl Connection { /// A protocol violation ends the connection with a CONNECTION_CLOSE from the next /// [`Self::poll_transmit`]. pub fn handle_record(&mut self, payload: Bytes, now: Instant) { - if self.error.is_some() { + if self.is_closed() { return; } self.timer.on_recv(now); @@ -273,17 +288,13 @@ impl Connection { /// The transport refused an incoming record larger than `max_record_size`, or a draft-00 /// frame larger than `max_frame_size`, rather than read it into memory pub fn handle_oversized(&mut self) { - if self.error.is_none() { - self.fail(self.oversized()); - } + self.fail(self.oversized()); } /// The transport found the peer violating its framing, such as overrunning a buffer it /// bounds itself, so close with `err` pub fn handle_violation(&mut self, err: TransportError) { - if self.error.is_none() { - self.fail(err); - } + self.fail(err); } /// The transport failed, or ended without a close frame @@ -302,17 +313,20 @@ impl Connection { /// The frame comes from the next [`Self::poll_transmit`]; flush it before shutting down /// the transport. pub fn close(&mut self, error_code: VarInt, reason: String) { - if self.error.is_some() { + if self.is_closed() { return; } - self.close = Some(Close::Application(ApplicationClose { + let frame = Close::Application(ApplicationClose { error_code, reason: reason.clone(), - })); - self.lost(ConnectionError::LocallyClosed { - code: error_code, - reason, }); + self.terminate( + ConnectionError::LocallyClosed { + code: error_code, + reason, + }, + Some(frame), + ); } /// The next record to write, without its Size field, or the next frame on draft-00, or wait @@ -345,15 +359,15 @@ impl Connection { /// The instant [`Self::handle_timeout`] should run next pub fn poll_timeout(&self) -> Option { - match self.error { - Some(_) => None, - None => self.timer.next(), + match self.is_closed() { + true => None, + false => self.timer.next(), } } /// Run the handshake, idle, keep-alive, and RTT timers pub fn handle_timeout(&mut self, now: Instant) { - if self.error.is_some() { + if self.is_closed() { return; } let due = self.timer.poll(now); @@ -372,8 +386,8 @@ impl Connection { /// Open and accept streams pub fn streams(&mut self) -> Streams<'_> { Streams { + closed: self.is_closed(), state: &mut self.streams, - conn_state: &self.conn_state, } } @@ -388,9 +402,9 @@ impl Connection { pub fn send_stream(&mut self, id: crate::StreamId) -> send::Stream<'_> { send::Stream { id, + closed: self.is_closed(), state: &mut self.streams, pending: &mut self.pending, - conn_state: &self.conn_state, } } @@ -491,7 +505,7 @@ impl Connection { /// Serialize the next batch of frames, each counted as acknowledged on the spot fn fill(&mut self, now: Instant) -> Option { - if self.close_sent { + if matches!(self.end, End::Closed(_)) { return None; } let budget = self.budget(); @@ -514,13 +528,13 @@ impl Connection { self.params_sent = true; } - if self.error.is_some() { + if let End::Closing { error, frame } = &mut self.end { // Write the close frame, if any, and go quiet - if let Some(close) = self.close.take() { + if let Some(close) = frame.take() { let room = budget - buf.len(); close.encode(buf, room); } - self.close_sent = true; + self.end = End::Closed(error.clone()); if buf.is_empty() { return false; } @@ -630,7 +644,7 @@ impl Connection { )); } self.on_frame(frame, allocation, frame_len, now)?; - if self.error.is_some() { + if self.is_closed() { // The peer closed; ignore the rest return Ok(()); } @@ -650,11 +664,15 @@ impl Connection { } } - /// End the connection over a violation, telling the peer with a CONNECTION_CLOSE + /// End the connection over a violation, telling the peer with a CONNECTION_CLOSE, unless it + /// already ended fn fail(&mut self, err: TransportError) { + if self.is_closed() { + return; + } debug!(%err, "qmux protocol error"); - self.close = Some(Close::from(err.clone())); - self.lost(ConnectionError::TransportError(err)); + let frame = Close::from(err.clone()); + self.terminate(ConnectionError::TransportError(err), Some(frame)); } fn on_frame( @@ -850,14 +868,19 @@ impl Connection { Ok(()) } - /// End the connection with `reason`, unless it already ended: the first close wins + /// End the connection with `reason` without a close frame, unless it already ended fn lost(&mut self, reason: ConnectionError) { - if self.error.is_some() { + self.terminate(reason, None); + } + + /// End the connection with `error`, writing `frame` if any, unless it already ended: the + /// first close wins + fn terminate(&mut self, error: ConnectionError, frame: Option) { + if self.is_closed() { return; } - self.error = Some(reason.clone()); - self.streams.waits.lose(reason.clone()); - self.conn_state = ConnectionState::Draining; + self.streams.waits.lose(error.clone()); + self.end = End::Closing { error, frame }; } } @@ -867,7 +890,7 @@ impl std::fmt::Debug for Connection { .field("side", &self.side) .field("version", &self.config.version) .field("established", &self.peer.is_some()) - .field("error", &self.error) + .field("error", &self.error()) .finish_non_exhaustive() } } diff --git a/rs/moq-quic/src/connection/paths.rs b/rs/moq-quic/src/connection/paths.rs index e97715cfa48..edee94416db 100644 --- a/rs/moq-quic/src/connection/paths.rs +++ b/rs/moq-quic/src/connection/paths.rs @@ -11,6 +11,21 @@ use crate::{Duration, Instant, TIMER_GRANULARITY, congestion, packet::SpaceId, t #[cfg(feature = "qlog")] use qlog::events::{ExData, quic::RecoveryMetricsUpdated}; +/// An unanswered PATH_CHALLENGE +#[derive(Clone, Copy)] +pub(super) struct Challenge { + pub(super) token: u64, + /// A packet of its own is still due for it; after that, it only rides along with the path's + /// other frames + pub(super) pending: bool, +} + +impl Challenge { + pub(super) fn new(token: u64) -> Self { + Self { token, pending: true } + } +} + /// Description of a particular network path pub(super) struct PathData { pub(super) remote: SocketAddr, @@ -19,9 +34,8 @@ pub(super) struct PathData { pub(super) sending_ecn: bool, /// Congestion controller state pub(super) congestion: Box, - /// Pacing state - pub(super) challenge: Option, - pub(super) challenge_pending: bool, + /// The PATH_CHALLENGE validating this path, until it is answered or abandoned + pub(super) challenge: Option, /// Whether we're certain the peer can both send and receive on this address /// /// Initially equal to `use_stateless_retry` for servers, and becomes false again on every @@ -89,7 +103,6 @@ impl PathData { // Sized for the MTU discovery starts at, which a known peer limit may have lowered congestion: build_congestion(now, mtud.current_mtu(), seed, config), challenge: None, - challenge_pending: false, validated: false, total_sent: 0, total_recvd: 0, @@ -110,7 +123,6 @@ impl PathData { sending_ecn: true, congestion: prev.congestion.clone_box(), challenge: None, - challenge_pending: false, validated: false, total_sent: 0, total_recvd: 0, diff --git a/rs/moq-quic/src/connection/streams/lease.rs b/rs/moq-quic/src/connection/streams/lease.rs index 2fbfafdd3ed..72df8afa3c5 100644 --- a/rs/moq-quic/src/connection/streams/lease.rs +++ b/rs/moq-quic/src/connection/streams/lease.rs @@ -8,7 +8,7 @@ use std::task::Poll; use bytes::{Buf, BufMut, Bytes, BytesMut}; use super::StreamId; -use super::recv::{Error, Stream}; +use super::recv::{Error, Reader, Stream}; use super::{StreamsState, state::get_or_insert_recv}; use crate::VarInt; use crate::connection::assembler::Assembler; @@ -41,9 +41,6 @@ pub(super) struct Leased { /// The chunks are out in a [`Lease`], whose reader may be about to free the window, which /// holds eviction off until it comes back. out: bool, - /// Discarded to free a stuck window, with the reader and the peer yet to hear of it from a - /// stop. - pub(super) evicted: bool, } /// A receive stream's place in the eviction order: lowest priority first, then the newest, which @@ -182,7 +179,6 @@ impl Leased { self.chunks.clear(); self.want = None; self.ready = false; - self.evicted = true; leased } @@ -351,7 +347,12 @@ impl Stream<'_> { match result { // A stuck window may have evicted this very stream, waking its readers before this // one parked. - Err(Error::Blocked) if self.state.lease_mut(self.id).is_some_and(|lease| lease.evicted) => { + Err(Error::Blocked) + if self + .state + .recv_mut(self.id) + .is_some_and(|recv| recv.reader == Reader::Evicted) => + { Poll::Ready(Err(Error::Evicted)) } Err(Error::Blocked) if self.state.waits.lost().is_some() => Poll::Ready(Err(Error::ConnectionLost)), @@ -392,11 +393,11 @@ impl Stream<'_> { /// Return the credit of every byte read out of a lease. fn release_credit(&mut self) -> bool { - let Some(lease) = self.state.lease_mut(self.id) else { + let Some(recv) = self.state.recv_mut(self.id) else { return false; }; - let freed = lease.release(); - if lease.evicted { + let freed = recv.lease.release(); + if recv.reader == Reader::Evicted { self.state.evicted_held -= freed; } let transmit = self.return_credit(freed); @@ -444,14 +445,15 @@ impl Stream<'_> { /// /// Dropped instead once the stream was stopped, which discarded its lease and its credit. fn put_back(&mut self, lease: Lease) { - let Some(state) = self.state.lease_mut(self.id) else { + let Some(recv) = self.state.recv_mut(self.id) else { return; }; + let state = &mut recv.lease; if !std::mem::take(&mut state.out) { return; } // An out lease holds eviction off, and nothing fills a lease while it is out. - debug_assert!(!state.evicted && state.chunks.is_empty()); + debug_assert!(recv.reader != Reader::Evicted && state.chunks.is_empty()); state.chunks = lease.chunks; state.len = lease.len as u64; } @@ -477,11 +479,10 @@ impl Stream<'_> { }; let recv = get_or_insert_recv(slot, self.state.initial_stream_receive_window); assert!(!recv.lease.out, "leased again before the last lease was handed back"); - if recv.lease.evicted { - return Err(Error::Evicted); - } - if recv.stopped { - return Err(Error::Closed); + match recv.reader { + Reader::Reading => {} + Reader::Evicted => return Err(Error::Evicted), + Reader::Stopped => return Err(Error::Closed), } self.state.unlooked.remove(&self.id); self.state.held += recv.lease.fill(&mut recv.assembler); @@ -530,12 +531,24 @@ impl StreamsState { pub(super) fn leased(&self, id: StreamId) -> Option<&Leased> { let recv = self.recv.get(&id)?.as_deref()?; let lease = &recv.lease; - (lease.filled && (!recv.stopped || lease.evicted)).then_some(lease) + (lease.filled && recv.reader != Reader::Stopped).then_some(lease) + } + + /// Whether a stuck window evicted `id`, and its reader has yet to stop it. + #[cfg(test)] + pub(super) fn is_evicted(&self, id: StreamId) -> bool { + self.recv + .get(&id) + .and_then(|slot| slot.as_deref()) + .is_some_and(|recv| recv.reader == Reader::Evicted) + } + + fn recv_mut(&mut self, id: StreamId) -> Option<&mut super::Recv> { + self.recv.get_mut(&id)?.as_deref_mut() } fn lease_mut(&mut self, id: StreamId) -> Option<&mut Leased> { - let recv = self.recv.get_mut(&id)?.as_deref_mut()?; - Some(&mut recv.lease) + Some(&mut self.recv_mut(id)?.lease) } /// Return `bytes` of credit withheld by a lease to the peer. @@ -550,13 +563,13 @@ impl StreamsState { } /// Take the lease of `id`, stopped or freed, out of the eviction order and return the credit - /// of the bytes it holds. - pub(super) fn discard_lease(&mut self, id: StreamId, lease: &mut Leased) -> bool { + /// of the bytes it holds, which `evicted` counts in [`Self::evicted_held`]. + pub(super) fn discard_lease(&mut self, id: StreamId, lease: &mut Leased, evicted: bool) -> bool { if std::mem::take(&mut lease.ranked) { self.ranks.remove(&lease.rank(id)); } let held = lease.discard(); - if std::mem::take(&mut lease.evicted) { + if evicted { self.evicted_held -= held; } self.release(held) @@ -614,7 +627,7 @@ impl StreamsState { .and_then(|slot| slot.as_deref_mut()) .expect("a freed stream leaves the ranks"); recv.lease.ranked = false; - if recv.stopped || recv.unread_credit() + recv.lease.held == 0 { + if recv.reader != Reader::Reading || recv.unread_credit() + recv.lease.held == 0 { continue; } debug_assert!( @@ -640,7 +653,7 @@ impl StreamsState { .recv .get(&id) .and_then(|slot| slot.as_deref()) - .filter(|recv| !recv.stopped && unlooked(recv.unread_credit(), &recv.lease)); + .filter(|recv| recv.reader == Reader::Reading && unlooked(recv.unread_credit(), &recv.lease)); if let Some(recv) = recv { // Nothing else reruns the pass once the lease comes back. self.stuck_waiting = recv.lease.out; @@ -670,6 +683,7 @@ impl StreamsState { self.evicted_held += recv.lease.held; // The peer hears of it when the application stops the stream with its own code. let (unread, _) = recv.stop().expect("a candidate is not stopped"); + recv.reader = Reader::Evicted; self.held -= leased; self.unlooked.remove(&id); self.evicted += 1; @@ -688,12 +702,12 @@ impl StreamsState { }; let lease = &recv.lease; held += lease.held; - if lease.evicted { + if recv.reader == Reader::Evicted { evicted_held += lease.held; } assert_eq!(lease.ranked, self.ranks.contains(&lease.rank(id)), "{id}"); ranked += usize::from(lease.ranked); - if lease.filled && !recv.stopped && recv.unread_credit() + lease.held > 0 { + if lease.filled && recv.reader == Reader::Reading && recv.unread_credit() + lease.held > 0 { assert!(lease.ranked, "{id} holds credit unranked"); } } diff --git a/rs/moq-quic/src/connection/streams/mod.rs b/rs/moq-quic/src/connection/streams/mod.rs index a11c24fe810..919011125de 100644 --- a/rs/moq-quic/src/connection/streams/mod.rs +++ b/rs/moq-quic/src/connection/streams/mod.rs @@ -7,7 +7,7 @@ use crate::{Dir, StreamId, connection::Error, frame}; mod lease; pub mod recv; -use recv::Recv; +use recv::{Reader, Recv}; #[cfg(test)] mod saturated; @@ -23,14 +23,15 @@ pub(crate) use state::StreamsState; /// Access to streams pub struct Streams<'a> { pub(super) state: &'a mut StreamsState, - pub(super) conn_state: &'a super::State, + /// The connection ended, so no stream opens + pub(super) closed: bool, } impl<'a> Streams<'a> { /// Drive a bare `StreamsState` without a connection, for the `streams` fuzz target #[cfg(any(test, feature = "fuzz"))] - pub(crate) fn new(state: &'a mut StreamsState, conn_state: &'a super::State) -> Self { - Self { state, conn_state } + pub(crate) fn new(state: &'a mut StreamsState, closed: bool) -> Self { + Self { state, closed } } /// Open a single stream if possible, its sending half scheduled by `options` @@ -41,7 +42,7 @@ impl<'a> Streams<'a> { /// - when `options.group` was opened on another connection pub fn open(&mut self, dir: Dir, options: send::Options<'_>) -> Option { let group = options.group.map(|g| g.index(self.state.tag)); - if self.conn_state.is_closed() { + if self.closed { return None; } diff --git a/rs/moq-quic/src/connection/streams/recv.rs b/rs/moq-quic/src/connection/streams/recv.rs index a5583815898..10c6c4a85ee 100644 --- a/rs/moq-quic/src/connection/streams/recv.rs +++ b/rs/moq-quic/src/connection/streams/recv.rs @@ -44,12 +44,13 @@ impl Stream<'_> { // An eviction already discarded the buffer, leaving the peer and the bytes the reader // took to this stop. - let (read_credits, stop_sending) = match stream.lease.evicted { - true => (0, stream.final_offset_unknown()), - false => { - let (credits, stop_sending) = stream.stop()?; - (credits, stop_sending) + let evicted = stream.reader == Reader::Evicted; + let (read_credits, stop_sending) = match evicted { + true => { + stream.reader = Reader::Stopped; + (0, stream.final_offset_unknown()) } + false => stream.stop()?, }; let mut lease = std::mem::take(&mut stream.lease); if stop_sending { @@ -69,7 +70,7 @@ impl Stream<'_> { self.state.queue_max_stream_id(self.pending); } - let leased = self.state.discard_lease(self.id, &mut lease); + let leased = self.state.discard_lease(self.id, &mut lease, evicted); let transmit = self.state.add_read_credits(read_credits); self.pending.max_data |= leased || transmit; // The credit returned may only have paid a shrunk window's debt. @@ -95,11 +96,10 @@ impl Stream<'_> { let Some(s) = entry.get().as_deref() else { return Ok(None); }; - if s.stopped { - return Err(match s.lease.evicted { - true => Error::Evicted, - false => Error::Closed, - }); + match s.reader { + Reader::Reading => {} + Reader::Evicted => return Err(Error::Evicted), + Reader::Stopped => return Err(Error::Closed), } let Some(code) = s.reset_code() else { return Ok(None); @@ -150,11 +150,23 @@ pub(super) struct Recv { /// next one starts. Unlike `end`, a reset's final size does not raise it. #[cfg(feature = "mux")] pub(super) data_end: u64, - pub(super) stopped: bool, + pub(super) reader: Reader, /// What a reader leased, which lives as long as the stream does. pub(super) lease: Leased, } +/// Whether the application still reads a receive stream +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub(super) enum Reader { + #[default] + Reading, + /// Discarded to free a stuck window, with the reader and the peer yet to hear of it from a + /// stop; it keeps no data meanwhile, but its lease keeps the credit of bytes the reader took + Evicted, + /// The application stopped the stream, which keeps no data + Stopped, +} + impl Recv { pub(super) fn new(initial_max_data: u64) -> Box { Box::new(Self { @@ -164,7 +176,7 @@ impl Recv { end: 0, #[cfg(feature = "mux")] data_end: 0, - stopped: false, + reader: Reader::Reading, lease: Leased::default(), }) } @@ -217,17 +229,17 @@ impl Recv { } // Don't bother storing data or releasing stream-level flow control credit if the stream's // already stopped - if !self.stopped && !frame.data.is_empty() { + if self.reader == Reader::Reading && !frame.data.is_empty() { self.assembler .insert(frame.offset, frame.data, payload_len) .map_err(|_| TransportError::INTERNAL_ERROR("too many gaps in stream buffer"))?; } - Ok((new_bytes, frame.fin && self.stopped)) + Ok((new_bytes, frame.fin && self.reader != Reader::Reading)) } pub(super) fn stop(&mut self) -> Result<(u64, bool), Error> { - if self.stopped { + if self.reader != Reader::Reading { return Err(Error::Closed); } @@ -237,7 +249,7 @@ impl Recv { // data that the peer might still be trying to retransmit, in which case a STOP_SENDING is // still useful. A stream read to its end, as a lease leaves it, sends none. let stop_sending = self.is_receiving() && !self.is_read(); - self.stopped = true; + self.reader = Reader::Stopped; self.assembler.clear(); Ok((read_credits, stop_sending)) } @@ -305,7 +317,7 @@ impl Recv { pub(super) fn can_send_flow_control(&self) -> bool { // Stream-level flow control is redundant if the sender has already sent the whole stream, // and moot if we no longer want data on this stream. - self.final_offset_unknown() && !self.stopped + self.final_offset_unknown() && self.reader == Reader::Reading } /// Whether data is still being accepted from the peer @@ -438,7 +450,7 @@ impl<'a> Chunks<'a> { }; let recv = get_or_insert_recv(entry.get_mut(), streams.initial_stream_receive_window); - if recv.stopped { + if recv.reader != Reader::Reading { return Err(Error::Closed); } let recv = entry.remove().unwrap(); // initialized above diff --git a/rs/moq-quic/src/connection/streams/saturated.rs b/rs/moq-quic/src/connection/streams/saturated.rs index f72fad558f9..50613a30c8e 100644 --- a/rs/moq-quic/src/connection/streams/saturated.rs +++ b/rs/moq-quic/src/connection/streams/saturated.rs @@ -10,7 +10,7 @@ use super::state::StreamsState; use super::{Streams, send}; use crate::{ Dir, Side, StreamId, VarInt, - connection::{State as ConnState, spaces::Retransmits}, + connection::spaces::Retransmits, frame::{self, Frame, StreamMeta}, transport_parameters::TransportParameters, }; @@ -36,7 +36,6 @@ const PING: u8 = 0x01; struct Sender { state: StreamsState, pending: Retransmits, - conn: ConnState, /// Payload bytes each stream got out. sent: HashMap, /// Bytes each stream's frames took on the wire, headers included, which is what @@ -74,7 +73,6 @@ impl Sender { Self { state, pending: Retransmits::default(), - conn: ConnState::Established, sent: HashMap::new(), wire: HashMap::new(), datagram_wire: HashMap::new(), @@ -96,7 +94,7 @@ impl Sender { id, state: &mut self.state, pending: &mut self.pending, - conn_state: &self.conn, + closed: false, } } @@ -104,7 +102,7 @@ impl Sender { fn open(&mut self, group: Option<&send::Group>, priority: i64, bytes: usize) -> StreamId { let id = Streams { state: &mut self.state, - conn_state: &self.conn, + closed: false, } .open(Dir::Uni, send::Options { group, priority }) .unwrap(); diff --git a/rs/moq-quic/src/connection/streams/send.rs b/rs/moq-quic/src/connection/streams/send.rs index 5c061c1230d..cefad9cb298 100644 --- a/rs/moq-quic/src/connection/streams/send.rs +++ b/rs/moq-quic/src/connection/streams/send.rs @@ -7,7 +7,7 @@ use thiserror::Error; use tracing::trace; use super::{StreamsState, schedule::Slot}; -use crate::connection::{ConnectionTag, State as ConnState, send_buffer::SendBuffer, spaces::Retransmits, waits::Wait}; +use crate::connection::{ConnectionTag, send_buffer::SendBuffer, spaces::Retransmits, waits::Wait}; use crate::{StreamId, VarInt, frame}; #[derive(Debug)] @@ -17,8 +17,6 @@ pub(super) struct Send { pub(super) pending: SendBuffer, pub(super) priority: i64, pub(super) slot: Slot, - /// Whether a frame containing a FIN bit must be transmitted, even if we don't have any new data - pub(super) fin_pending: bool, /// Value of `max_data` for which a `STREAM_DATA_BLOCKED` frame was most recently queued /// /// A new frame is only queued once the peer has raised the limit. @@ -39,7 +37,6 @@ impl Send { pending: SendBuffer::new(), priority: 0, slot: Slot::default(), - fin_pending: false, data_blocked_limit: None, stop_reason: None, notify: None, @@ -56,8 +53,10 @@ impl Send { if let Some(error_code) = self.stop_reason { Err(Error::Stopped(error_code)) } else if self.state == SendState::Ready { - self.state = SendState::DataSent { finish_acked: false }; - self.fin_pending = true; + self.state = SendState::DataSent { + finish_acked: false, + fin_pending: true, + }; Ok(()) } else { Err(Error::Closed) @@ -101,7 +100,7 @@ impl Send { } match self.state { SendState::ResetSent { error_code, .. } => Some(End::Reset(error_code)), - SendState::DataSent { finish_acked: true } if self.pending.is_fully_acked() => Some(End::Finished), + SendState::DataSent { finish_acked: true, .. } if self.pending.is_fully_acked() => Some(End::Finished), _ => None, } } @@ -120,7 +119,6 @@ impl Send { debug_assert!(reliable_size <= self.pending.offset()); let final_size = self.pending.offset(); self.pending.truncate(reliable_size); - self.fin_pending = false; let send = at || self.pending.is_fully_acked(); self.state = ResetSent { final_size, @@ -240,7 +238,7 @@ impl Send { pub(super) fn ack(&mut self, frame: frame::StreamMeta) -> bool { self.pending.ack(frame.offsets); match &mut self.state { - SendState::DataSent { finish_acked } => { + SendState::DataSent { finish_acked, .. } => { *finish_acked |= frame.fin; *finish_acked && self.pending.is_fully_acked() } @@ -295,7 +293,7 @@ impl Send { } pub(super) fn is_pending(&self) -> bool { - self.pending.has_unsent_data() || self.fin_pending + self.pending.has_unsent_data() || matches!(self.state, SendState::DataSent { fin_pending: true, .. }) } pub(super) fn is_writable(&self) -> bool { @@ -308,7 +306,11 @@ pub(super) enum SendState { /// Sending new data Ready, /// Stream was finished; now sending retransmits only - DataSent { finish_acked: bool }, + DataSent { + finish_acked: bool, + /// A frame with the FIN bit is due, even without new data + fin_pending: bool, + }, /// Reset by the application, while data below `reliable_size` is still delivered /// /// The peer gets RESET_STREAM_AT, or, without it, a RESET_STREAM `held` until it has @@ -331,23 +333,19 @@ pub struct Stream<'a> { pub(crate) id: StreamId, pub(crate) state: &'a mut StreamsState, pub(crate) pending: &'a mut Retransmits, - pub(crate) conn_state: &'a ConnState, + /// The connection ended, so nothing more is written + pub(crate) closed: bool, } impl<'a> Stream<'a> { /// Drive a bare `StreamsState` without a connection, for the `streams` fuzz target #[cfg(any(test, feature = "fuzz"))] - pub(crate) fn new( - id: StreamId, - state: &'a mut StreamsState, - pending: &'a mut Retransmits, - conn_state: &'a ConnState, - ) -> Self { + pub(crate) fn new(id: StreamId, state: &'a mut StreamsState, pending: &'a mut Retransmits, closed: bool) -> Self { Self { id, state, pending, - conn_state, + closed, } } @@ -357,7 +355,7 @@ impl<'a> Stream<'a> { /// splits off without copying. Fails with [`Error::Blocked`] when nothing fits; /// [`Self::poll_write`] waits for room instead. pub fn write(&mut self, data: &mut impl Buf) -> Result { - if self.conn_state.is_closed() { + if self.closed { trace!(%self.id, "write blocked; connection draining"); return Err(Error::Blocked); } @@ -543,7 +541,7 @@ impl<'a> Stream<'a> { }, }; match acked { - Ok(false) if self.conn_state.is_closed() => Err(Error::Closed), + Ok(false) if self.closed => Err(Error::Closed), acked => acked, } } diff --git a/rs/moq-quic/src/connection/streams/state.rs b/rs/moq-quic/src/connection/streams/state.rs index 2795a4c4354..df172424dcf 100644 --- a/rs/moq-quic/src/connection/streams/state.rs +++ b/rs/moq-quic/src/connection/streams/state.rs @@ -11,7 +11,7 @@ use rustc_hash::{FxBuildHasher, FxHashMap, FxHashSet}; use tracing::{debug, trace}; use super::{ - Next, Recv, Retransmits, Schedule, Send, SendState, StreamHalf, ThinRetransmits, + Next, Reader, Recv, Retransmits, Schedule, Send, SendState, StreamHalf, ThinRetransmits, lease::Rank, send::{End, Error}, }; @@ -267,7 +267,7 @@ impl StreamsState { } let (new_bytes, closed) = rs.ingest(frame, payload_len, self.data_recvd, self.local_max_data)?; - let (stopped, evicted) = (rs.stopped, rs.lease.evicted); + let (stopped, evicted) = (rs.reader != Reader::Reading, rs.reader == Reader::Evicted); let rank = (!stopped).then(|| rs.lease.arrived(id)).flatten(); self.spend(new_bytes); @@ -362,7 +362,7 @@ impl StreamsState { // Redundant reset return Ok(false); }; - let (stopped, evicted) = (rs.stopped, rs.lease.evicted); + let (stopped, evicted) = (rs.reader != Reader::Reading, rs.reader == Reader::Evicted); let rank = (!stopped).then(|| rs.lease.arrived(id)).flatten(); self.spend(new_bytes); let credit = if stopped { @@ -717,10 +717,14 @@ impl StreamsState { // are required to encode it. let max_buf_size = max_buf_size - buf.len() - 1 - VarInt::size(id.into()); let (offsets, encode_length) = stream.pending.poll_transmit(max_buf_size); - let fin = offsets.end == stream.pending.offset() && matches!(stream.state, SendState::DataSent { .. }); - if fin { - stream.fin_pending = false; - } + let fin = offsets.end == stream.pending.offset() + && match &mut stream.state { + SendState::DataSent { fin_pending, .. } => { + *fin_pending = false; + true + } + SendState::Ready | SendState::ResetSent { .. } => false, + }; let meta = frame::StreamMeta { id, offsets, fin }; trace!(id = %meta.id, off = meta.offsets.start, len = meta.offsets.end - meta.offsets.start, fin = meta.fin, "STREAM"); @@ -857,7 +861,11 @@ impl StreamsState { }; let was_pending = stream.is_pending(); // The Reliable Size of a reset stream supersedes any FIN - stream.fin_pending |= frame.fin && !stream.is_reset(); + if frame.fin + && let SendState::DataSent { fin_pending, .. } = &mut stream.state + { + *fin_pending = true; + } stream.pending.retransmit(frame.offsets); if !was_pending && stream.is_pending() { self.pending.push(frame.id, &mut stream.slot, stream.priority); @@ -1314,7 +1322,8 @@ impl StreamsState { /// /// Returns whether a `MAX_DATA` frame should be enqueued for that credit. pub(super) fn stream_recv_freed(&mut self, id: StreamId, mut recv: Box) -> bool { - let transmit = self.discard_lease(id, &mut recv.lease); + let evicted = recv.reader == Reader::Evicted; + let transmit = self.discard_lease(id, &mut recv.lease, evicted); self.unlooked.remove(&id); self.stream_freed(id, StreamHalf::Recv); transmit @@ -1348,7 +1357,6 @@ mod tests { use super::*; use crate::{ TransportErrorCode, - connection::State as ConnState, connection::Streams, connection::streams::{recv, send}, connection::waits::Counter, @@ -1376,12 +1384,12 @@ mod tests { for index in 0..127 { let id = StreamId::new(Side::Client, dir, index); data(&mut server, id, 0, 1, false); - assert_eq!(Streams::new(&mut server, &ConnState::Established).accept(dir), Some(id)); + assert_eq!(Streams::new(&mut server, false).accept(dir), Some(id)); } for index in 127..131 { let id = StreamId::new(Side::Client, dir, index); data(&mut server, id, 0, 1, true); - assert_eq!(Streams::new(&mut server, &ConnState::Established).accept(dir), Some(id)); + assert_eq!(Streams::new(&mut server, false).accept(dir), Some(id)); let mut stream = recv(&mut server, &mut pending, id); let mut chunks = stream.read().unwrap(); assert_eq!(chunks.next(1).unwrap().unwrap().bytes.len(), 1); @@ -1392,7 +1400,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &ConnState::Established, + closed: false, }; stream.finish().unwrap(); stream.release(); @@ -1435,7 +1443,7 @@ mod tests { for (&id, reader) in ids.iter().zip(&readers) { data(&mut server, id, 0, 0, false); server.waits.register(id, Wait::Read, reader); - assert_eq!(Streams::new(&mut server, &ConnState::Established).accept(dir), Some(id)); + assert_eq!(Streams::new(&mut server, false).accept(dir), Some(id)); } for packet in 0..packets { for &id in &ids { @@ -1473,7 +1481,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &ConnState::Established, + closed: false, }; stream.finish().unwrap(); stream.release(); @@ -1508,14 +1516,14 @@ mod tests { let mut pending = Retransmits::default(); let mut sent = Vec::new(); for index in 0..streams { - let id = Streams::new(&mut server, &ConnState::Established) + let id = Streams::new(&mut server, false) .open(Dir::Uni, Default::default()) .unwrap(); let mut stream = send::Stream { id, state: &mut server, pending: &mut pending, - conn_state: &ConnState::Established, + closed: false, }; assert_eq!(stream.write(&mut &b"x"[..]), Err(send::Error::Blocked)); stream.reset(0u32.into()).unwrap(); @@ -1556,7 +1564,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &ConnState::Established, + closed: false, } .release(); assert!(server.ends.capacity() <= server.ends.len() * 4); @@ -1580,11 +1588,11 @@ mod tests { initial_max_data: 0u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); for _ in 0..512 { let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1592,7 +1600,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; assert_eq!(stream.write(&mut &b"blocked"[..]), Err(send::Error::Blocked)); assert_eq!(stream.write(&mut &b"blocked"[..]), Err(send::Error::Blocked)); @@ -1618,10 +1626,10 @@ mod tests { initial_max_stream_data_uni: 1024u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1629,7 +1637,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; assert_eq!(stream.write(&mut &b"blocked"[..]), Err(send::Error::Blocked)); stream.finish().unwrap(); @@ -1655,10 +1663,10 @@ mod tests { initial_max_data: if peer_credit { 1u32.into() } else { 1024u32.into() }, ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let holder = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1667,14 +1675,14 @@ mod tests { id: holder, state: &mut server, pending: &mut pending, - conn_state: &state + closed: false } .write(&mut &b"x"[..]), Ok(1) ); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1682,7 +1690,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; let (write, end) = (Counter::new(), Counter::new()); assert!(stream.poll_write(&write, &mut &b"blocked"[..]).is_pending()); @@ -1710,14 +1718,14 @@ mod tests { initial_max_stream_data_uni: 1024u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let writers: Vec<_> = (0..4).map(|_| Counter::new()).collect(); let ids: Vec<_> = writers .iter() .map(|writer| { let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1725,7 +1733,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; assert_eq!(stream.write(&mut &b"x"[..]), Err(send::Error::Blocked)); assert!(stream.poll_write(writer, &mut &b"x"[..]).is_pending()); @@ -1737,7 +1745,7 @@ mod tests { id: ids[1], state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .reset(0u32.into()) .unwrap(); @@ -1745,7 +1753,7 @@ mod tests { id: ids[2], state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .finish() .unwrap(); @@ -1774,10 +1782,10 @@ mod tests { initial_max_stream_data_uni: 1000u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1785,7 +1793,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .write(&mut &[42; 24][..]) .unwrap(); @@ -1820,7 +1828,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .write(&mut &[7; 24][..]) .unwrap(); @@ -1838,10 +1846,10 @@ mod tests { initial_max_stream_data_uni: 1000u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1849,7 +1857,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .write(&mut &[42; 24][..]) .unwrap(); @@ -1875,10 +1883,10 @@ mod tests { initial_max_stream_data_uni: 1000u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -1886,7 +1894,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .write(&mut &[42; 24][..]) .unwrap(); @@ -1902,7 +1910,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .reset(0u32.into()) .unwrap(); @@ -2250,10 +2258,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -2262,7 +2270,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; let error_code = 0u32.into(); @@ -2312,10 +2320,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; let id_high = streams.open(Dir::Bi, Default::default()).unwrap(); @@ -2326,7 +2334,7 @@ mod tests { id: id_mid, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; mid.write(&mut &b"mid"[..]).unwrap(); @@ -2334,7 +2342,7 @@ mod tests { id: id_low, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; low.set_priority(-1).unwrap(); low.write(&mut &b"low"[..]).unwrap(); @@ -2343,7 +2351,7 @@ mod tests { id: id_high, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; high.set_priority(1).unwrap(); high.write(&mut &b"high"[..]).unwrap(); @@ -2369,10 +2377,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; let id_low = streams.open(Dir::Bi, Default::default()).unwrap(); let id_high = streams.open(Dir::Bi, Default::default()).unwrap(); @@ -2381,7 +2389,7 @@ mod tests { id: id_low, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; low.set_priority(-1).unwrap(); assert_eq!(low.write(&mut &b"low"[..]).unwrap(), 3); @@ -2403,7 +2411,7 @@ mod tests { id: id_high, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; high.set_priority(1).unwrap(); assert_eq!(high.write(&mut &b"high"[..]).unwrap(), 4); @@ -2430,10 +2438,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; let id_high = streams.open(Dir::Bi, Default::default()).unwrap(); @@ -2443,7 +2451,7 @@ mod tests { id: id_mid, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; assert_eq!(mid.write(&mut &b"mid"[..]).unwrap(), 3); assert_eq!(server.pending.len(), 1); @@ -2452,7 +2460,7 @@ mod tests { id: id_high, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; high.set_priority(1).unwrap(); assert_eq!(high.write(&mut &[0; 200][..]).unwrap(), 200); @@ -2463,7 +2471,7 @@ mod tests { id: id_high, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; high.set_priority(-1).unwrap(); assert_eq!(server.pending.len(), 2); @@ -2490,10 +2498,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; let ids: Vec<_> = (0..3) .map(|_| streams.open(Dir::Bi, Default::default()).unwrap()) @@ -2505,7 +2513,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.set_priority(priority).unwrap(); stream.write(&mut &[0; 200][..]).unwrap(); @@ -2522,7 +2530,7 @@ mod tests { id: starved, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, } .set_priority(10) .unwrap(); @@ -2542,10 +2550,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; // a, b and c all have the same priority @@ -2557,7 +2565,7 @@ mod tests { id: id_a, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_a.write(&mut &[b'a'; 100][..]).unwrap(); @@ -2565,7 +2573,7 @@ mod tests { id: id_b, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_b.write(&mut &[b'b'; 100][..]).unwrap(); @@ -2573,7 +2581,7 @@ mod tests { id: id_c, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_c.write(&mut &[b'c'; 100][..]).unwrap(); @@ -2616,10 +2624,10 @@ mod tests { ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; // a, and b have the same priority, c has higher priority @@ -2631,7 +2639,7 @@ mod tests { id: id_a, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_a.write(&mut &[b'a'; 100][..]).unwrap(); @@ -2639,7 +2647,7 @@ mod tests { id: id_b, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_b.write(&mut &[b'b'; 100][..]).unwrap(); @@ -2657,7 +2665,7 @@ mod tests { id: id_c, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream_c.set_priority(1).unwrap(); stream_c.write(&mut &[b'b'; 100][..]).unwrap(); @@ -2720,10 +2728,10 @@ mod tests { initial_max_stream_data_uni: 42u32.into(), ..TransportParameters::default() }); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let mut streams = Streams { state: &mut server, - conn_state: &state, + closed: false, }; let id = streams.open(Dir::Uni, Default::default()).unwrap(); @@ -2731,7 +2739,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello"[..]).unwrap(); stream.reset(0u32.into()).unwrap(); @@ -3176,11 +3184,10 @@ mod tests { assert_eq!(open.wakes(), 0, "no open was refused"); let mut retransmits = Retransmits::default(); - let conn_state = ConnState::Established; let stream_id = Streams { state: &mut server, - conn_state: &conn_state, + closed: false, } .open(Dir::Uni, Default::default()) .expect("should be able to open a stream"); @@ -3189,7 +3196,7 @@ mod tests { id: stream_id, state: &mut server, pending: &mut retransmits, - conn_state: &conn_state, + closed: false, }; // Check that the stream accepts `initial_send_window` bytes @@ -3248,11 +3255,10 @@ mod tests { assert_eq!(open.wakes(), 0, "no open was refused"); let mut retransmits = Retransmits::default(); - let conn_state = ConnState::Established; let stream_id = Streams { state: &mut server, - conn_state: &conn_state, + closed: false, } .open(Dir::Uni, Default::default()) .expect("should be able to open a stream"); @@ -3261,7 +3267,7 @@ mod tests { id: stream_id, state: &mut server, pending: &mut retransmits, - conn_state: &conn_state, + closed: false, }; let initial_send_len = initial_send_window as usize; @@ -3510,10 +3516,10 @@ mod tests { #[test] fn held_reset_waits_for_prefix_ack() { let mut server = server_with_reset_stream_at(false); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3521,7 +3527,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); let mut buf = Vec::new(); @@ -3554,10 +3560,10 @@ mod tests { #[test] fn stale_reset_ack_keeps_shrunk_reset() { let mut server = reset_at_server(); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3565,7 +3571,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); stream.reset_at(stream.offset().rewind(4), 7u32.into()).unwrap(); @@ -3583,10 +3589,10 @@ mod tests { #[test] fn acked_reset_shrunk_waits_for_replacement() { let mut server = reset_at_server(); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3594,7 +3600,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); let mut buf = Vec::new(); @@ -3624,10 +3630,10 @@ mod tests { fn stop_sending_abandons_reliable_prefix() { for reset_stream_at in [false, true] { let mut server = server_with_reset_stream_at(reset_stream_at); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3635,7 +3641,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); stream.reset_at(stream.offset().rewind(4), 7u32.into()).unwrap(); @@ -3657,10 +3663,10 @@ mod tests { #[test] fn reset_at_retains_prefix_until_acked() { let mut server = reset_at_server(); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3668,7 +3674,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); let mut buf = Vec::new(); @@ -3701,10 +3707,10 @@ mod tests { #[test] fn reset_at_after_stop_sending_drops_prefix() { let mut server = reset_at_server(); - let (mut pending, state) = (Retransmits::default(), ConnState::Established); + let mut pending = Retransmits::default(); let id = Streams { state: &mut server, - conn_state: &state, + closed: false, } .open(Dir::Uni, Default::default()) .unwrap(); @@ -3712,7 +3718,7 @@ mod tests { id, state: &mut server, pending: &mut pending, - conn_state: &state, + closed: false, }; stream.write(&mut &b"hello world"[..]).unwrap(); stream.state.received_stop_sending(id, 9u32.into()).unwrap(); @@ -3993,7 +3999,7 @@ mod tests { assert_eq!(client.credit_stats().evicted, 2); assert_eq!(readers.woken(&mut client), [ids[1], ids[2]]); assert_eq!(client.local_max_data, max_data + WINDOW / 2); - assert!(!client.leased(ids[0]).unwrap().evicted); + assert!(client.leased(ids[0]).is_some() && !client.is_evicted(ids[0])); } /// However its reader consumes tiny frames, and however many arrive, the lease holds copies @@ -4372,7 +4378,7 @@ mod tests { assert!(matches!(wait(&mut client, high, 2000), Err(recv::Error::Blocked))); assert!(matches!(wait(&mut client, low, 2000), Err(recv::Error::Blocked))); assert!( - client.leased(low).is_some_and(|lease| !lease.evicted), + client.leased(low).is_some() && !client.is_evicted(low), "a byte of credit left" ); assert_eq!(client.credit_stats().evicted, 0); @@ -4380,7 +4386,7 @@ mod tests { data(&mut client, low, WINDOW - 1, 1, false); // The arrival woke the reader of `low`, which may yet consume. assert_eq!(readers.woken(&mut client), [low]); - assert!(!client.leased(low).unwrap().evicted); + assert!(client.leased(low).is_some() && !client.is_evicted(low)); let blocked = wait_transmit(&mut client, low, 2000); assert_eq!(blocked, (Err(recv::Error::Blocked), true)); assert_eq!(readers.woken(&mut client), [low], "the evicted reader wakes"); @@ -4418,7 +4424,7 @@ mod tests { assert!(matches!(wait(&mut client, high, 2000), Err(recv::Error::Blocked))); assert!(matches!(wait(&mut client, low, 2000), Err(recv::Error::Blocked))); data(&mut client, low, WINDOW - 1, 1, false); - assert!(!client.leased(low).unwrap().evicted); + assert!(client.leased(low).is_some() && !client.is_evicted(low)); // This wait spends the last byte of credit and evicts its own stream. let waiter = Counter::new(); @@ -4590,7 +4596,7 @@ mod tests { recv(&mut client, &mut pending, waiting).peek(2 * WINDOW), (Err(recv::Error::Blocked), true) ); - assert!(client.leased(parked).unwrap().evicted); + assert!(client.leased(parked).is_some() && client.is_evicted(parked)); assert!(!pending.max_stream_id[Dir::Uni as usize], "its reader has yet to hear"); let pending = stop(&mut client, parked, 0); diff --git a/rs/moq-quic/src/fuzz.rs b/rs/moq-quic/src/fuzz.rs index 974ac7294a3..56792287e5b 100644 --- a/rs/moq-quic/src/fuzz.rs +++ b/rs/moq-quic/src/fuzz.rs @@ -25,7 +25,7 @@ use rand::{Rng, SeedableRng, rngs::StdRng}; use crate::{ ConnectionId, Dir, MAX_CID_SIZE, RESET_TOKEN_SIZE, Side, StreamId, VarInt, coding::BufMutExt, - connection::{Retransmits, State, Streams, StreamsState}, + connection::{Retransmits, Streams, StreamsState}, frame::{self, Frame}, packet::{FixedLengthConnectionIdParser, PartialDecode}, transport_parameters::{PreferredAddress, TransportParameters}, @@ -168,7 +168,6 @@ pub fn streams(data: &[u8]) -> bool { let side = if limits.server { Side::Server } else { Side::Client }; let dir = if limits.uni { Dir::Uni } else { Dir::Bi }; - let conn_state = State::Established; let mut pending = Retransmits::default(); let mut state = StreamsState::new( side, @@ -185,14 +184,14 @@ pub fn streams(data: &[u8]) -> bool { }; match op { StreamOp::Open => { - Streams::new(&mut state, &conn_state).open(dir, Default::default()); + Streams::new(&mut state, false).open(dir, Default::default()); } StreamOp::Accept { uni } => { let dir = if uni { Dir::Uni } else { Dir::Bi }; - Streams::new(&mut state, &conn_state).accept(dir); + Streams::new(&mut state, false).accept(dir); } StreamOp::Finish { id } => { - let _ = crate::send::Stream::new(StreamId(id), &mut state, &mut pending, &conn_state).finish(); + let _ = crate::send::Stream::new(StreamId(id), &mut state, &mut pending, false).finish(); } StreamOp::ReceivedStopSending { id, error_code } => { let _ = state.received_stop_sending(StreamId(id), varint(error_code)); @@ -211,8 +210,7 @@ pub fn streams(data: &[u8]) -> bool { }); } StreamOp::Reset { id } => { - let _ = - crate::send::Stream::new(StreamId(id), &mut state, &mut pending, &conn_state).reset(0u32.into()); + let _ = crate::send::Stream::new(StreamId(id), &mut state, &mut pending, false).reset(0u32.into()); } } } diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index f759c576cb5..350f62ccff8 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -3953,6 +3953,54 @@ fn a_peer_heard_on_a_new_path_brings_no_probe_forward() { assert_eq!(pair.server_conn_mut(server_ch).remote_address(), moved); } +/// A closing endpoint sends only CONNECTION_CLOSE (RFC 9000 §10.2.1), so the PATH_CHALLENGE a +/// migration queued for the previous path never goes out after a close, and the close is the +/// first transmit, as drivers that publish a local close once it is sent rely on. +#[test] +fn a_close_drops_a_pending_path_challenge() { + let _guard = subscribe(); + let mut pair = Pair::default(); + let (client_ch, server_ch) = pair.connect(); + pair.drive(); + + let moved = SocketAddr::new(Ipv4Addr::new(127, 0, 0, 1).into(), pair.next_port()); + pair.client.addr = moved; + pair.client_conn_mut(client_ch).ping(); + pair.drive_client(); + let (arrival, ecn, datagram) = pair.server.inbound.pop_front().unwrap(); + pair.time = arrival; + let mut buf = Vec::new(); + let Some(DatagramEvent::ConnectionEvent(ch, event)) = pair.server.endpoint.handle( + arrival, + Received { + remote: moved, + local_ip: None, + ecn, + data: datagram, + arrival: None, + }, + &mut buf, + ) else { + panic!("the PING did not reach the connection"); + }; + assert_eq!(ch, server_ch); + let conn = pair.server_conn_mut(server_ch); + conn.handle_event(event); + assert_eq!(conn.remote_address(), moved); + + let challenges = conn.stats().frame_tx.path_challenge; + conn.close(arrival, 7u32.into(), String::new()); + let mut buf = Vec::new(); + let close = conn.transmit(arrival, 1, &mut buf).expect("the close goes out"); + assert_eq!(close.remote, moved, "the challenge to the previous path went out"); + assert_eq!(conn.stats().frame_tx.path_challenge, challenges); + buf.clear(); + assert!( + conn.transmit(arrival, 1, &mut buf).is_none(), + "more than the close went out" + ); +} + /// A Version Negotiation packet carries no packet protection, so anyone on the path can send /// one: it is not the peer heard again, and brings no probe forward. #[test] diff --git a/rs/moq-relay/src/cluster.rs b/rs/moq-relay/src/cluster.rs index d9dc6239237..65c7c897346 100644 --- a/rs/moq-relay/src/cluster.rs +++ b/rs/moq-relay/src/cluster.rs @@ -586,12 +586,11 @@ struct Dialers { stop: Arc>, } -struct DialersPhase { - /// `None` once closed, which refuses any further dial. - open: Option, - /// The shutdown the first close started, which every close awaits, so none - /// returns before it is done. - closing: Option>>, +enum DialersPhase { + Open(DialersState), + /// Closed, which refuses any further dial. The shutdown the first close + /// started, which every close awaits, so none returns before it is done. + Closing(futures::future::Shared>), } struct DialersState { @@ -602,17 +601,20 @@ struct DialersState { closing: kio::Tasks>, } +impl DialersState { + fn new() -> Self { + Self { + clients: HashMap::new(), + next: 0, + closing: kio::Tasks::new(), + } + } +} + impl Dialers { fn new() -> Self { Self { - state: Arc::new(Mutex::new(DialersPhase { - open: Some(DialersState { - clients: HashMap::new(), - next: 0, - closing: kio::Tasks::new(), - }), - closing: None, - })), + state: Arc::new(Mutex::new(DialersPhase::Open(DialersState::new()))), stop: Arc::new(tokio::sync::watch::Sender::new(false)), } } @@ -624,7 +626,9 @@ impl Dialers { /// Keep `client` until [`Self::close`]. fn insert(&self, client: &moq_tokio::Client) -> anyhow::Result { let mut phase = self.lock(); - let state = phase.open.as_mut().context("cluster closed")?; + let DialersPhase::Open(state) = &mut *phase else { + anyhow::bail!("cluster closed"); + }; let id = state.next; state.next += 1; state.clients.insert(id, client.clone()); @@ -654,7 +658,7 @@ impl Dialers { /// Advance the closes of clients whose guard dropped. Never resolves the dials' /// lifetime: an empty set is only drained for now. fn poll_closing(&self, waiter: &kio::Waiter) { - if let Some(state) = self.lock().open.as_mut() { + if let DialersPhase::Open(state) = &mut *self.lock() { let _ = state.closing.poll_each(waiter, |_, res| { if let Err(err) = res { tracing::warn!(%err, "cluster client close failed"); @@ -668,14 +672,15 @@ impl Dialers { async fn close(&self) { let closing = { let mut phase = self.lock(); - let phase = &mut *phase; - phase - .closing - .get_or_insert_with(|| { - let state = phase.open.take().expect("open until the first close"); - Self::shutdown(state, self.stop.clone()).boxed().shared() - }) - .clone() + match &mut *phase { + DialersPhase::Closing(closing) => closing.clone(), + DialersPhase::Open(state) => { + let state = std::mem::replace(state, DialersState::new()); + let closing = Self::shutdown(state, self.stop.clone()).boxed().shared(); + *phase = DialersPhase::Closing(closing.clone()); + closing + } + } }; closing.await; } @@ -702,7 +707,7 @@ impl Drop for Held { fn drop(&mut self) { let mut phase = self.dialers.lock(); // Already taken by `Dialers::close`, which closes the client itself. - let Some(state) = phase.open.as_mut() else { + let DialersPhase::Open(state) = &mut *phase else { return; }; let Some(client) = state.clients.remove(&self.id) else { @@ -4024,7 +4029,10 @@ mod tests { assert!(format!("{err:#}").contains("cluster token"), "{err:#}"); std::fs::write(&token, "secret").expect("write token"); let _started = cluster.clone().start().await.expect("a corrected start binds the port"); - let registered = cluster.dialers.lock().open.as_ref().expect("open").clients.len(); + let registered = match &*cluster.dialers.lock() { + DialersPhase::Open(state) => state.clients.len(), + DialersPhase::Closing(_) => panic!("closed"), + }; assert_eq!(registered, 0, "a start registers its client only once it runs"); } @@ -4184,20 +4192,17 @@ mod tests { let dialers = Dialers::new(); // Stands in for a dropped guard's close, which holds the shutdown open. let (background, closed) = tokio::sync::oneshot::channel::<()>(); - dialers - .lock() - .open - .as_mut() - .expect("open") - .closing - .push(Child::new("held close", async move { + match &mut *dialers.lock() { + DialersPhase::Open(state) => state.closing.push(Child::new("held close", async move { let _ = closed.await; - })); + })), + DialersPhase::Closing(_) => panic!("closed"), + } let first = tokio::spawn({ let dialers = dialers.clone(); async move { dialers.close().await } }); - while dialers.lock().open.is_some() { + while matches!(*dialers.lock(), DialersPhase::Open(_)) { tokio::task::yield_now().await; } diff --git a/rs/moq-relay/src/shutdown.rs b/rs/moq-relay/src/shutdown.rs index 08551bd6957..ccd0f94017e 100644 --- a/rs/moq-relay/src/shutdown.rs +++ b/rs/moq-relay/src/shutdown.rs @@ -278,8 +278,10 @@ impl Observer { } Serving { _active: self.activity.enter(), - counted, - closed: None, + count: match counted { + true => Count::Open, + false => Count::Uncounted, + }, shutdown: self.clone(), } } @@ -417,46 +419,59 @@ pub struct Serving { // `None` only once the drain closed admission, which the connection's own // admission prevents while it is held before the SETUP reply. _active: Option, - /// False for a session served after the deadline, which is not counted. - counted: bool, - closed: Option, + count: Count, pub(crate) shutdown: Observer, } +/// Where a [`Serving`] session is counted in the [`Tally`]. +#[derive(Clone, Copy)] +enum Count { + /// Served after the deadline, so not counted at all. + Uncounted, + /// Counted as open. + Open, + /// Closed at this instant, and counted as ending until it drops. + Ending(Instant), +} + impl Serving { pub(crate) async fn drain(&self, session: &moq_net::Session) { - self.shutdown.drain(session, self.counted).await; + let counted = !matches!(self.count, Count::Uncounted); + self.shutdown.drain(session, counted).await; } + /// The session closed: count it as ending until this drops. Only the first call counts. pub(crate) fn closed(&mut self) { - self.closed = Some(Instant::now()); - if self.counted { - self.shutdown.state.send_modify(|state| { - state.tally.open -= 1; - state.tally.ending += 1; - }); + if !matches!(self.count, Count::Open) { + return; } + self.count = Count::Ending(Instant::now()); + self.shutdown.state.send_modify(|state| { + state.tally.open -= 1; + state.tally.ending += 1; + }); } } impl Drop for Serving { fn drop(&mut self) { - if self.counted { - let now = Instant::now(); - let ending = self - .closed - .zip(self.shutdown.deadline()) - .is_some_and(|(closed, deadline)| closed < deadline && now >= deadline); - self.shutdown.state.send_modify(|state| { - if self.closed.is_some() { - state.tally.ending -= 1; - } else { - state.tally.open -= 1; - } - state.ending += usize::from(ending); - state.last_left = Some(state.last_left.map_or(now, |last| last.max(now))); - }); - } + let closed = match self.count { + Count::Uncounted => return, + Count::Open => None, + Count::Ending(closed) => Some(closed), + }; + let now = Instant::now(); + let ending = closed + .zip(self.shutdown.deadline()) + .is_some_and(|(closed, deadline)| closed < deadline && now >= deadline); + self.shutdown.state.send_modify(|state| { + match closed { + Some(_) => state.tally.ending -= 1, + None => state.tally.open -= 1, + } + state.ending += usize::from(ending); + state.last_left = Some(state.last_left.map_or(now, |last| last.max(now))); + }); } } @@ -755,6 +770,19 @@ mod tests { .unwrap(); } + /// A session reported closed twice is counted as ending once, and leaves the tally + /// empty when it drops. + #[tokio::test(start_paused = true)] + async fn a_session_closed_twice_ends_once() { + let (_trigger, shutdown) = Observer::new(Duration::from_secs(10)); + let mut serving = shutdown.serve(); + serving.closed(); + serving.closed(); + assert_eq!(shutdown.tally(), Tally { open: 0, ending: 1 }); + drop(serving); + assert_eq!(shutdown.tally(), Tally::default()); + } + /// A drain whose admissions emptied before the deadline still waits for a session /// counted without one, rather than racing its cleanup. #[tokio::test(start_paused = true)] diff --git a/rs/moq-tokio/src/connection.rs b/rs/moq-tokio/src/connection.rs index 428c214e347..1b432b2f93e 100644 --- a/rs/moq-tokio/src/connection.rs +++ b/rs/moq-tokio/src/connection.rs @@ -377,41 +377,109 @@ impl Goaway { goaway } } + /// Shared reconnect state, observed by consumers through a [`kio`] channel. /// -/// The channel closing (all producers dropped) is the terminal signal; `error` -/// distinguishes a permanent give-up from a graceful close. +/// The channel closing (all producers dropped) follows [`Lifecycle::Closed`], which +/// every reader checks first. #[derive(Default)] struct State { /// Advances on every lifecycle update, including identical failed dials. revision: u64, - /// The loop has finished and cannot publish another state. - ended: bool, - /// Current connection status, or `None` before the first connect. - status: Option, + /// Where the loop is. Every status, observation, and live-session read derives from it. + lifecycle: Lifecycle, /// How many sessions have been live on this handle, counting the current one. - /// Advanced by `Shared::connected` with the same write that publishes `status`, + /// Advanced by `Shared::connected` with the same write that publishes the session, /// so a reader can't see `Connected` without its count. connects: u64, last_connected: Option, /// Cumulative connects and disconnects, bumped by the reconnect loop itself so a session that /// connects and drops before a consumer polls still counts. presence: moq_net::stats::Presence, - /// The negotiated MoQ version of the live session, or `None` when disconnected. - version: Option, - /// What the live session runs on, or `None` when disconnected. - transport: Option, - /// Set when the reconnect loop permanently gives up (reconnect timeout exceeded). - error: Option, - /// Why nothing is live while the loop retries: the last session's close reason or - /// the latest failed dial. Cleared by the next connect. +} + +/// The reconnect loop's lifecycle, as one value so a stopped loop can't also report a session or +/// a redial. +enum Lifecycle { + /// No session has connected yet. `failure` is the latest failed dial. + Connecting { failure: Option }, + /// A session serves. + Live(Box), + /// The last session ended and the loop redials. `failure` is why nothing serves: the last + /// session's close reason or the latest failed dial. + Offline { + failure: Option, + predecessor: Option, + }, + /// The loop stopped: the error it gave up with, or `None` when stopped locally. + Closed(Option), +} + +impl Default for Lifecycle { + fn default() -> Self { + Self::Connecting { failure: None } + } +} + +/// The session a [`Lifecycle::Live`] loop serves on. +struct Live { + /// Read by [`Monitor`] to snapshot live connection stats. + session: moq_net::Session, + transport: crate::Transport, + /// Set by a GOAWAY until the replacement connects: this session drains but still serves. + migrating: Option, + predecessor: Option, +} + +/// A GOAWAY handover in progress, from the old session's side. +#[derive(Default)] +struct Migrating { + /// The latest failed dial of the replacement. failure: Option, - /// The currently-connected session, or `None` while reconnecting. Read by - /// [`Monitor`] to snapshot live connection stats. - session: Option, - /// The policy's draining predecessor and its handover deadline, so - /// [`Connection::close`] drains it too without overstaying that window. - predecessor: Option<(moq_net::Session, kio::clock::Instant)>, +} + +/// The policy's draining predecessor and its handover deadline, so [`Connection::close`] drains it +/// too without overstaying that window. Boxed: only a migration holds one, and only briefly. +type Predecessor = Box<(moq_net::Session, kio::clock::Instant)>; + +impl Lifecycle { + /// What [`Connection::status`] reports: `None` before the first connect and once closed. + fn status(&self) -> Option { + match self { + Self::Live(live) if live.migrating.is_some() => Some(Status::Migrating), + Self::Live(_) => Some(Status::Connected), + Self::Offline { .. } => Some(Status::Disconnected), + Self::Connecting { .. } | Self::Closed(_) => None, + } + } + + fn live(&self) -> Option<&Live> { + match self { + Self::Live(live) => Some(live), + _ => None, + } + } + + fn is_closed(&self) -> bool { + matches!(self, Self::Closed(_)) + } + + /// Why the loop is retrying, while it is. + fn failure(&self) -> Option<&Error> { + match self { + Self::Connecting { failure } | Self::Offline { failure, .. } => failure.as_ref(), + Self::Live(live) => live.migrating.as_ref()?.failure.as_ref(), + Self::Closed(_) => None, + } + } + + fn predecessor(&self) -> Option<&Predecessor> { + match self { + Self::Live(live) => live.predecessor.as_ref(), + Self::Offline { predecessor, .. } => predecessor.as_ref(), + Self::Connecting { .. } | Self::Closed(_) => None, + } + } } /// The producer side of everything a [`Connection`] handle can observe. @@ -446,55 +514,88 @@ impl Shared { // reads its estimate, and the grants that divide it. let _ = self.send_bw.set(session.send_bandwidth().and_then(|bw| bw.peek())); if let Ok(mut state) = self.state.write() { - // A migration replaces a live session without a disconnect in between, so - // its end is counted here, keeping `started - ended` at 1 while connected. - if state.session.is_some() { - state.presence.sessions_ended += 1; - } + let state = &mut *state; + let predecessor = match &mut state.lifecycle { + Lifecycle::Connecting { .. } => None, + // A migration replaces a live session without a disconnect in between, so + // its end is counted here, keeping `started - ended` at 1 while connected. + Lifecycle::Live(live) => { + state.presence.sessions_ended += 1; + live.predecessor.take() + } + Lifecycle::Offline { predecessor, .. } => predecessor.take(), + Lifecycle::Closed(_) => unreachable!("the loop publishes nothing once it ends"), + }; state.revision += 1; - state.status = Some(Status::Connected); state.connects += 1; state.last_connected = Some(kio::clock::now()); state.presence.sessions_started += 1; - state.failure = None; - state.version = Some(session.version()); - state.transport = Some(transport); - state.session = Some(session.clone()); + state.lifecycle = Lifecycle::Live(Box::new(Live { + session: session.clone(), + transport, + migrating: None, + predecessor, + })); } } /// The session in [`State`] keeps serving while its replacement comes up, so /// only the status moves. fn migrating(&self) { - self.status(Status::Migrating); - } - - fn status(&self, status: Status) { if let Ok(mut state) = self.state.write() { state.revision += 1; - state.status = Some(status); + if let Lifecycle::Live(live) = &mut state.lifecycle { + live.migrating = Some(Migrating::default()); + } } } /// Mirror the policy's draining predecessor, for [`Connection::close`]. fn predecessor(&self, predecessor: Option<(&moq_net::Session, kio::clock::Instant)>) { // A write wakes every watcher, so skip one that changes nothing. - if predecessor.is_none() && self.state.read().predecessor.is_none() { + if predecessor.is_none() && self.state.read().lifecycle.predecessor().is_none() { return; } + let predecessor = predecessor.map(|(session, deadline)| Box::new((session.clone(), deadline))); if let Ok(mut state) = self.state.write() { - state.predecessor = predecessor.map(|(session, deadline)| (session.clone(), deadline)); + match &mut state.lifecycle { + Lifecycle::Live(live) => live.predecessor = predecessor, + Lifecycle::Offline { predecessor: slot, .. } => *slot = predecessor, + // Only a session that connected can drain, and nothing drains once the loop ends. + Lifecycle::Connecting { .. } | Lifecycle::Closed(_) => {} + } } } /// Nothing is live. Clears the session so a handle can't be handed a closed one, - /// and drops the estimates that belonged to it. + /// and drops the estimates that belonged to it. Without a `failure`, the reason + /// already recorded stands. fn offline(&self, failure: Option<&Error>) { if let Ok(mut state) = self.state.write() { - state.offline(); - if let Some(failure) = failure { - state.failure = Some(failure.clone()); - } + let state = &mut *state; + let failure = failure.cloned(); + state.lifecycle = match std::mem::take(&mut state.lifecycle) { + Lifecycle::Live(live) => { + state.presence.sessions_ended += 1; + Lifecycle::Offline { + failure: failure.or(live.migrating.and_then(|migrating| migrating.failure)), + predecessor: live.predecessor, + } + } + Lifecycle::Offline { + failure: kept, + predecessor, + } => Lifecycle::Offline { + failure: failure.or(kept), + predecessor, + }, + // Only a session that connected can go offline. + Lifecycle::Connecting { failure: kept } => Lifecycle::Connecting { + failure: failure.or(kept), + }, + closed @ Lifecycle::Closed(_) => closed, + }; + state.revision += 1; } let _ = self.send_bw.set(None); let _ = self.recv_bw.set(None); @@ -504,7 +605,16 @@ impl Shared { fn retrying(&self, err: &Error) { if let Ok(mut state) = self.state.write() { state.revision += 1; - state.failure = Some(err.clone()); + match &mut state.lifecycle { + Lifecycle::Connecting { failure } | Lifecycle::Offline { failure, .. } => *failure = Some(err.clone()), + // A live session retries only for its replacement. + Lifecycle::Live(live) => { + if let Some(migrating) = &mut live.migrating { + migrating.failure = Some(err.clone()); + } + } + Lifecycle::Closed(_) => {} + } } } @@ -523,17 +633,17 @@ impl Shared { (None, None) => moq_net::Error::Closed, }; if let Ok(mut state) = self.state.write() { - if state.ended { - return; + let state = &mut *state; + match state.lifecycle { + Lifecycle::Closed(_) => return, + Lifecycle::Live(_) => state.presence.sessions_ended += 1, + Lifecycle::Connecting { .. } | Lifecycle::Offline { .. } => {} } - state.offline(); - // Including when the task was aborted mid-handover: nothing enforces the - // predecessor's deadline anymore, so a parked clone would hold its transport open. - // A graceful close took its own clone before aborting. - state.predecessor = None; - state.ended = true; - state.failure = None; - state.error = err; + state.revision += 1; + // Drops the predecessor too, including when the task was aborted mid-handover: + // nothing enforces its deadline anymore, so a parked clone would hold its + // transport open. A graceful close took its own clones before aborting. + state.lifecycle = Lifecycle::Closed(err); } let _ = self.send_bw.set(None); let _ = self.recv_bw.set(None); @@ -602,39 +712,25 @@ pub struct Observation { } impl State { - /// Nothing is live: one lifecycle revision that drops the session and its version. - fn offline(&mut self) { - // Count one close per live session; a second call without an intervening - // connect (e.g. Drop after the loop already reported the close) must not - // move the counter again. - if self.session.is_some() { - self.presence.sessions_ended += 1; - } - self.revision += 1; - self.status = Some(Status::Disconnected); - self.version = None; - self.transport = None; - self.session = None; - } - fn observation(&self) -> Observation { + let live = self.lifecycle.live(); Observation { revision: self.revision, - phase: if self.ended { - Phase::Closed - } else { - match self.status { - None => Phase::Connecting, - Some(Status::Connected) => Phase::Connected, - Some(Status::Disconnected) => Phase::Disconnected, - Some(Status::Migrating) => Phase::Migrating, - } + phase: match &self.lifecycle { + Lifecycle::Connecting { .. } => Phase::Connecting, + Lifecycle::Live(live) if live.migrating.is_some() => Phase::Migrating, + Lifecycle::Live(_) => Phase::Connected, + Lifecycle::Offline { .. } => Phase::Disconnected, + Lifecycle::Closed(_) => Phase::Closed, }, connects: self.connects, last_connected: self.last_connected, - transport: self.transport, - failure: self.error.clone().or_else(|| self.failure.clone()), - version: self.version, + transport: live.map(|live| live.transport), + failure: match &self.lifecycle { + Lifecycle::Closed(err) => err.clone(), + lifecycle => lifecycle.failure().cloned(), + }, + version: live.map(|live| live.session.version()), } } } @@ -650,7 +746,7 @@ impl Monitor { match self.state.poll(waiter, |state| { if state.revision != after { Poll::Ready(Ok(state.observation())) - } else if state.ended { + } else if state.lifecycle.is_closed() { Poll::Ready(Err(terminal(state))) } else { Poll::Pending @@ -698,7 +794,7 @@ impl Monitor { /// Snapshot the current connection's stats, or `None` if not currently connected. pub fn stats(&self) -> Option { - self.state.read().session.as_ref().map(moq_net::Session::stats) + self.state.read().lifecycle.live().map(|live| live.session.stats()) } /// Why the loop is retrying: the error that ended the last session or failed the @@ -708,13 +804,13 @@ impl Monitor { /// that keeps failing never changes the status, so this is where its reason shows. /// The loop giving up reports through [`Connection::closed`] instead. pub fn failure(&self) -> Option { - self.state.read().failure.clone() + self.state.read().lifecycle.failure().cloned() } /// Snapshot statistics and protocol together, or `None` while disconnected. pub fn snapshot(&self) -> Option { let state = self.state.read(); - let session = state.session.as_ref()?; + let session = &state.lifecycle.live()?.session; Some(Snapshot { stats: session.stats(), version: session.version(), @@ -876,7 +972,7 @@ impl Connection { let session = { let mut closed = self.task.closed.lock().unwrap(); *closed = Some(err.clone()); - self.state.read().session.clone() + self.state.read().lifecycle.live().map(|live| live.session.clone()) }; if let Some(session) = session { session.abort(err); @@ -903,7 +999,8 @@ impl Connection { let mut closed = self.task.closed.lock().unwrap(); *closed = Some(moq_net::Error::Cancel); let state = self.state.read(); - (state.session.clone(), state.predecessor.clone()) + let session = state.lifecycle.live().map(|live| live.session.clone()); + (session, state.lifecycle.predecessor().cloned()) }; self.task.handle.abort(); let session = async move { @@ -913,7 +1010,7 @@ impl Connection { } }; let predecessor = async move { - let Some((session, deadline)) = predecessor else { + let Some((session, deadline)) = predecessor.map(|predecessor| *predecessor) else { return Ok(()); }; let abort = session.clone(); @@ -1067,11 +1164,12 @@ impl Connection { /// old session, which still serves), `Ready(Err)` once the loop has stopped, `Pending` /// otherwise. pub fn poll_established(&self, waiter: &kio::Waiter) -> Poll> { - match ready!(self.state.poll(waiter, |state| match (state.status, &state.session) { - (Some(Status::Connected | Status::Migrating), Some(_)) => Poll::Ready(()), - _ => Poll::Pending, + match ready!(self.state.poll(waiter, |state| match &state.lifecycle { + Lifecycle::Live(_) => Poll::Ready(Ok(())), + Lifecycle::Closed(_) => Poll::Ready(Err(terminal(state))), + Lifecycle::Connecting { .. } | Lifecycle::Offline { .. } => Poll::Pending, })) { - Ok(()) => Poll::Ready(Ok(())), + Ok(res) => Poll::Ready(res), Err(state) => Poll::Ready(Err(terminal(&state))), } } @@ -1094,6 +1192,7 @@ impl Connection { /// The connection status, once it changes from what this handle last reported, or the error /// once the loop has stopped (the give-up error, or a generic one when the handle is dropped). + /// The error comes next once the loop stops, even if a status change went unreported. /// /// The loop moves between `Connected`, `Disconnected`, and `Migrating` (a GOAWAY handover, where /// the old session is still serving), so successive calls report changes rather than a fixed @@ -1102,11 +1201,12 @@ impl Connection { #[kio::wrapper] pub fn poll_status(&mut self, waiter: &kio::Waiter) -> Poll> { let last = self.last_reported; - let status = match ready!(self.state.poll(waiter, |state| match state.status { - Some(status) if Some(status) != last => Poll::Ready(status), + let status = match ready!(self.state.poll(waiter, |state| match state.lifecycle.status() { + _ if state.lifecycle.is_closed() => Poll::Ready(Err(terminal(state))), + Some(status) if Some(status) != last => Poll::Ready(Ok(status)), _ => Poll::Pending, })) { - Ok(status) => status, + Ok(res) => res?, Err(state) => return Poll::Ready(Err(terminal(&state))), }; @@ -1119,7 +1219,7 @@ impl Connection { /// The synchronous read behind [`status`](Self::status), for callers that just want the current /// state rather than the next change. pub fn connected(&self) -> bool { - self.state.read().status == Some(Status::Connected) + self.state.read().lifecycle.status() == Some(Status::Connected) } /// How many sessions have been live on this handle: 1 after the first connect, @@ -1137,14 +1237,14 @@ impl Connection { /// The [`moq_net::Session::version`] counterpart; `Option` because a reconnecting handle can be /// between sessions. pub fn version(&self) -> Option { - self.state.read().version + self.state.read().lifecycle.live().map(|live| live.session.version()) } /// What the live session runs on, or `None` while disconnected. /// /// Fixed for a session's lifetime; a reconnect may land on a different one. pub fn transport(&self) -> Option { - self.state.read().transport + self.state.read().lifecycle.live().map(|live| live.transport) } /// Observe a GOAWAY from the peer, or `None` while between sessions. @@ -1158,7 +1258,7 @@ impl Connection { /// Scoped to the current session, since that's what a GOAWAY is about; a redial /// starts a fresh one. pub fn draining(&self) -> Option { - self.state.read().session.as_ref().map(moq_net::Session::draining) + self.state.read().lifecycle.live().map(|live| live.session.draining()) } /// A consumer for the live session's estimated send bitrate, mirroring @@ -1202,11 +1302,17 @@ impl Connection { /// stopped by dropping the handle. #[kio::wrapper] pub fn poll_closed(&self, waiter: &kio::Waiter) -> Poll> { - ready!(self.state.poll_closed(waiter)); - Poll::Ready(match &self.state.read().error { - Some(err) => Err(err.clone()), - None => Ok(()), - }) + let err = match ready!(self.state.poll(waiter, |state| match &state.lifecycle { + Lifecycle::Closed(err) => Poll::Ready(err.clone()), + _ => Poll::Pending, + })) { + Ok(err) => err, + Err(state) => match &state.lifecycle { + Lifecycle::Closed(err) => err.clone(), + _ => unreachable!("the loop closes its lifecycle before its channel"), + }, + }; + Poll::Ready(err.map_or(Ok(()), Err)) } /// The auth handle every session of this connection is built on: its tokens are @@ -1346,9 +1452,9 @@ fn poll_forward(bw: &mut Option, out: &BandwidthProducer, wai /// it is, and a watcher that only has an error to go on can still tell it apart /// from a connection that failed. fn terminal(state: &State) -> Error { - match &state.error { - Some(err) => err.clone(), - None => Error::Stopped, + match &state.lifecycle { + Lifecycle::Closed(Some(err)) => err.clone(), + _ => Error::Stopped, } } @@ -1957,7 +2063,7 @@ mod tests { let monitor = connection.monitor(); let failure = kio::wait(|waiter| { - monitor.state.poll(waiter, |state| match &state.failure { + monitor.state.poll(waiter, |state| match state.lifecycle.failure() { Some(err) => Poll::Ready(err.clone()), None => Poll::Pending, }) @@ -1999,7 +2105,7 @@ mod tests { let mut monitor = connection.monitor(); kio::wait(|waiter| { - monitor.state.poll(waiter, |state| match state.failure { + monitor.state.poll(waiter, |state| match state.lifecycle.failure() { Some(_) => Poll::Ready(()), None => Poll::Pending, }) @@ -2139,6 +2245,35 @@ mod tests { assert_eq!(woken.phases(), [Phase::Closed], "the loop is not redialing"); } + /// A loop that gives up reports its error next, never a `Disconnected` it will not redial + /// from, whether or not a session ever connected. + #[cfg(feature = "tcp")] + #[tokio::test] + async fn a_stopped_loop_reports_its_error_next() { + const WAIT: Duration = Duration::from_secs(10); + let once = || { + crate::connect::Config::default() + .init(Default::default()) + .unwrap() + .with_reconnect(false) + }; + + // The only dial fails, which ends the loop before anything connected. + let mut connection = once().connect(dead().await); + let status = kio::clock::timeout(WAIT, connection.status()).await.unwrap(); + assert!(status.is_err(), "{status:?}"); + + // The session ending in one-shot mode ends the loop. + let (addr, mut accepted, _task) = serve(crate::origin::spawn()).await; + let mut connection = once().connect(url(&format!("tcp://{addr}/"))); + let status = kio::clock::timeout(WAIT, connection.status()).await.unwrap(); + assert_eq!(status.unwrap(), Status::Connected); + let session = kio::clock::timeout(WAIT, accepted.recv()).await.unwrap().unwrap(); + session.abort(moq_net::Error::App(7)); + let status = kio::clock::timeout(WAIT, connection.status()).await.unwrap(); + assert!(status.is_err(), "{status:?}"); + } + /// A settled answer from one candidate ends the walk instead of being buried. /// /// The walk keeps only the last failure, so without this a `404` from the first diff --git a/rs/moq-tokio/src/quic/driver/connection.rs b/rs/moq-tokio/src/quic/driver/connection.rs index 3138fa4b741..325d7a3696e 100644 --- a/rs/moq-tokio/src/quic/driver/connection.rs +++ b/rs/moq-tokio/src/quic/driver/connection.rs @@ -204,11 +204,11 @@ impl Future for ConnectionDriver { this.conn.shared.wire.set_received_bytes(stats.udp_rx.bytes); } + // A local close still reaches a driver whose socket holds output back, which skips + // `poll_transmit` and so parks nothing there, to re-arm the timer for its closing + // period. Waiting on the close consumes no transmit. + conn.check_lost(waiter); if !conn.inner.is_drained() { - // A local close still reaches a driver whose socket holds output back, which skips - // `poll_transmit` and so parks nothing there, to re-arm the timer for its closing - // period. Waiting on the close consumes no transmit. - let _ = conn.inner.poll_closed(waiter); // Otherwise `poll_transmit` parked the waiter for the application's next call, and the // ingress channel, the timer, and the socket hold the context. if keep_going { @@ -217,27 +217,38 @@ impl Future for ConnectionDriver { } return Poll::Pending; } - if !conn.is_lost() { - unreachable!("drained connections always have an error"); - } + // A close moq-quic never managed to send (the server's anti-amplification limit can + // withhold it) is published here too. + let reason = conn.inner.error().cloned(); + conn.publish(reason.expect("drained connections always have an error")); Poll::Ready(Ok(())) } } impl Drop for ConnectionDriver { fn drop(&mut self) { - // A panic poisoned the state, and without a driver nothing will time the connection out: - // fail every waiter now, so it reaches the poisoned lock instead of hanging. - if let Some(mut conn) = self.conn.state.poisoned() { - let reason = TransportError::new( - TransportErrorCode::INTERNAL_ERROR, - "connection driver panicked".to_string(), - ); - conn.terminate(Error::TransportError(reason)); + // Nothing drives the connection anymore, so no close goes out and no timer fires: publish + // its end now rather than leave every waiter hanging. After a normal end it is already + // published, and this keeps it. + if let Some(mut conn) = self.conn.state.lock_for_drop() { + match conn.inner.error().cloned() { + Some(reason) => conn.publish(reason), + None => conn.terminate(internal("connection driver stopped")), + } + } else if let Some(mut conn) = self.conn.state.poisoned() { + // A panic poisoned the state: fail every waiter, so it reaches the poisoned lock. + conn.terminate(internal("connection driver panicked")); } } } +fn internal(reason: &str) -> Error { + Error::TransportError(TransportError::new( + TransportErrorCode::INTERNAL_ERROR, + reason.to_string(), + )) +} + /// A QUIC connection. /// /// If all references to a connection (including every clone of the `Connection` handle, streams of @@ -345,14 +356,20 @@ impl Connection { /// The error the connection closed with, for any reason. /// + /// A local close resolves this only once the driver has sent its CONNECTION_CLOSE, so a caller + /// may stop the runtime the moment it resolves and the peer still hears the close. + /// /// Despite the return type's name, closed connections are often not an error condition at the /// application layer. Cases that might be routine include [`Error::LocallyClosed`] /// and [`Error::ApplicationClosed`]. #[kio::wrapper] pub fn poll_closed(&self, waiter: &kio::Waiter) -> Poll { let mut state = self.0.state.lock(); - let err = ready!(state.inner.poll_closed(waiter)); - Poll::Ready(state.lost_as(err)) + if let Some(err) = &state.closed { + return Poll::Ready(err.clone()); + } + waiter.register(&mut state.closed_waiters); + Poll::Pending } /// Whether the connection is closed, and why. @@ -360,7 +377,8 @@ impl Connection { /// The close_reason is always set to `Some(Error)` when a socket is /// closed; whether it was closed manually by calling [`Connection::close()`] or due to /// an internal error (such as an idle timeout or the peer closing the - /// connection). + /// connection). Unlike [`closed`](Self::closed), a local close counts from the moment it + /// was asked for. /// /// Note: when the connection is closed, `connection.close_reason().is_some()` will always be true. pub fn close_reason(&self) -> Option { @@ -655,15 +673,20 @@ pub(crate) struct State { timer_deadline: Option, conn_events: mpsc::UnboundedReceiver, endpoint_events: mpsc::UnboundedSender<(ConnectionHandle, EndpointEvent)>, - /// Why the connection failed, when moq-quic did not see it fail: the endpoint driver went away - /// - /// moq-quic takes no foreign error, so this wins over its reason when handles report the loss. - external: Option, + /// The terminal error [`Connection::poll_closed`] reports, set once by the driver: moq-quic's + /// reason, a local close only once its CONNECTION_CLOSE is on the socket, or why the driver + /// stopped, which moq-quic cannot take and so wins over its reason everywhere. + /// + /// A latch of its own rather than moq-quic's `poll_closed`, which resolves a local close as + /// soon as it is asked for, before anything sent it. + closed: Option, + closed_waiters: kio::WaiterList, /// The endpoint's socket, sent through without keeping it open slot: Arc, send_buffer: Vec, - /// We buffer a transmit when the underlying I/O would block - buffered_transmit: Option, + /// We buffer a transmit when the underlying I/O would block, with whether it carries the + /// CONNECTION_CLOSE: a close that came after it leaves it an ordinary packet + buffered_transmit: Option<(moq_quic::Transmit, bool)>, } impl State { @@ -681,7 +704,8 @@ impl State { timer_deadline: None, conn_events, endpoint_events, - external: None, + closed: None, + closed_waiters: kio::WaiterList::new(), slot, send_buffer: Vec::new(), buffered_transmit: None, @@ -698,8 +722,8 @@ impl State { loop { // Retry the last transmit, or get a new one. - let t = match self.buffered_transmit.take() { - Some(t) => t, + let (t, closes) = match self.buffered_transmit.take() { + Some(buffered) => buffered, None => { self.send_buffer.clear(); self.send_buffer.reserve(self.inner.current_mtu() as usize); @@ -712,7 +736,8 @@ impl State { None => 1, Some(s) => t.size.div_ceil(s), // round up }; - t + // Once closed, moq-quic transmits nothing but the CONNECTION_CLOSE. + (t, self.inner.is_closed()) } Poll::Pending => break, } @@ -722,7 +747,7 @@ impl State { let len = t.size; match self.slot.poll_send(&udp_transmit(&t, &self.send_buffer[..len]), cx) { Poll::Pending => { - self.buffered_transmit = Some(t); + self.buffered_transmit = Some((t, closes)); return Ok(false); } // The datagram is lost, and moq-quic decides whether the connection survives it. @@ -733,6 +758,10 @@ impl State { Poll::Ready(Err(e)) => return Err(e), Poll::Ready(Ok(())) => {} } + // The socket now has the CONNECTION_CLOSE. + if closes && let Some(reason) = self.inner.error() { + self.publish(reason.clone()); + } if transmits >= MAX_TRANSMIT_DATAGRAMS { // TODO: What isn't ideal here yet is that if we don't poll all @@ -756,6 +785,7 @@ impl State { fn process_conn_events(&mut self, cx: &mut Context<'_>) -> Result<(), Error> { loop { match self.conn_events.poll_recv(cx) { + #[cfg(test)] Poll::Ready(Some(ConnectionEvent::Rebind)) => { self.inner.local_address_changed(); } @@ -817,19 +847,39 @@ impl State { true } - /// Why the connection was lost, if it was: the driver's external reason, or moq-quic's + /// Why the connection was lost, if it was: the published reason, or moq-quic's pub(crate) fn error(&self) -> Option { - self.external.clone().or_else(|| self.inner.error().cloned()) + self.closed.clone().or_else(|| self.inner.error().cloned()) } /// Whether the connection was lost, without cloning why pub(crate) fn is_lost(&self) -> bool { - self.external.is_some() || self.inner.error().is_some() + self.closed.is_some() || self.inner.error().is_some() } /// The error handles report for a loss moq-quic reported as `err` fn lost_as(&self, err: Error) -> Error { - self.external.clone().unwrap_or(err) + self.closed.clone().unwrap_or(err) + } + + /// Publish `reason` as the terminal error, unless one already is, waking every + /// [`Connection::poll_closed`]. + fn publish(&mut self, reason: Error) { + if self.closed.is_none() { + self.closed = Some(reason); + self.closed_waiters.wake(); + } + } + + /// Publish a loss moq-quic recorded, parking `waiter` until it records one. + /// + /// A local close waits for `drive_transmit` to put its CONNECTION_CLOSE on the socket. + fn check_lost(&mut self, waiter: &kio::Waiter) { + if let Poll::Ready(reason) = self.inner.poll_closed(waiter) + && !matches!(reason, Error::LocallyClosed { .. }) + { + self.publish(reason); + } } /// The error handles report for a loss moq-quic reported without a reason, as a stream's @@ -838,13 +888,15 @@ impl State { self.error().expect("moq-quic reported a loss without a reason") } - /// Fail the connection for `reason`, which moq-quic did not see: the endpoint driver went away + /// Fail the connection for `reason`, which moq-quic did not see: the endpoint driver or this + /// connection's driver went away /// - /// The reason is kept to win over moq-quic's, and a local close makes moq-quic wake every - /// waiter. The driver exits right after, so the close is never sent. + /// The reason is published to win over moq-quic's, unless a loss already was, and a local + /// close makes moq-quic wake every waiter. The driver exits right after, so the close is never + /// sent. fn terminate(&mut self, reason: Error) { let message = reason.to_string(); - self.external = Some(reason); + self.publish(reason); self.inner.close(Instant::now(), 0u32.into(), message); } diff --git a/rs/moq-tokio/src/quic/driver/endpoint.rs b/rs/moq-tokio/src/quic/driver/endpoint.rs index 599996b1b51..087ef733ceb 100644 --- a/rs/moq-tokio/src/quic/driver/endpoint.rs +++ b/rs/moq-tokio/src/quic/driver/endpoint.rs @@ -169,7 +169,8 @@ impl Endpoint { /// Get the next incoming connection attempt, or `None` once the endpoint closes. pub fn poll_accept(&self, waiter: &kio::Waiter) -> Poll> { let mut endpoint = self.inner.state.lock().unwrap(); - if endpoint.driver_lost { + // Leaving `Running` refuses or ignores every queued attempt, and queues no new one. + if !endpoint.recv_state.connections.lifecycle.is_running() { return Poll::Ready(None); } if let Some(incoming) = endpoint.recv_state.incoming.pop_front() { @@ -183,9 +184,6 @@ impl Endpoint { let incoming = Incoming::new(incoming, self.inner.clone()); return Poll::Ready(Some(incoming)); } - if endpoint.recv_state.connections.close.is_some() { - return Poll::Ready(None); - } waiter.register(&mut endpoint.incoming_waiters); Poll::Pending } @@ -205,7 +203,7 @@ impl Endpoint { server_name: &str, ) -> Result { let mut endpoint = self.inner.state.lock().unwrap(); - if endpoint.driver_lost || endpoint.recv_state.connections.close.is_some() { + if !endpoint.recv_state.connections.lifecycle.is_running() { return Err(ConnectError::EndpointStopping); } if addr.is_ipv6() && !endpoint.ipv6 { @@ -227,7 +225,7 @@ impl Endpoint { Ok(endpoint .recv_state .connections - .insert(ch, conn, slot, false, &self.inner.runtime)) + .insert(ch, conn, slot, &self.inner.runtime)) } /// Switch to a new UDP socket @@ -235,8 +233,10 @@ impl Endpoint { /// Allows the endpoint's address to be updated live, affecting all active connections. Incoming /// connections and connections to servers unreachable from the new address will be lost. /// - /// On error, the old UDP socket is retained. - pub fn rebind(&self, socket: impl Into) -> Result<(), Error> { + /// On error, the old UDP socket is retained. Only tests switch sockets: on a closed endpoint + /// this would bind a port that nothing reads. + #[cfg(test)] + pub(crate) fn rebind(&self, socket: impl Into) -> Result<(), Error> { let socket = socket.into().into_async()?; let addr = socket.local_addr()?; let mut inner = self.inner.state.lock().unwrap(); @@ -292,9 +292,18 @@ impl Endpoint { } /// Close every connection and refuse new ones, without waiting for the closes to go out + /// + /// The first close's code and reason stand, and a lost driver leaves nothing to close. fn close_now(&self, error_code: VarInt, reason: &str) { let endpoint = &mut *self.inner.state.lock().unwrap(); - endpoint.recv_state.connections.close = Some((error_code, reason.to_string())); + let connections = &mut endpoint.recv_state.connections; + if !connections.lifecycle.is_running() { + return; + } + connections.lifecycle = Lifecycle::Closed { + error_code, + reason: reason.to_string(), + }; for sender in endpoint.recv_state.connections.senders.values() { // Ignoring errors from dropped connections let _ = sender.send(ConnectionEvent::Close { @@ -432,7 +441,7 @@ impl Drop for EndpointDriver { let endpoint = &mut *self.0.state.lock().unwrap(); endpoint.prev_socket = None; endpoint.forward = None; - endpoint.driver_lost = true; + endpoint.recv_state.connections.lifecycle = Lifecycle::DriverLost; endpoint.incoming_waiters.wake(); // Drop all outgoing channels, signaling the termination of the endpoint to the associated // connections. @@ -498,11 +507,10 @@ impl EndpointInner { state.pending_accepts -= 1; let result = match state.inner.finish_accept(accepted, &mut response_buffer) { Ok((handle, conn)) => { - let driver_lost = state.driver_lost; Ok(state .recv_state .connections - .insert(handle, conn, self.slot.clone(), driver_lost, &self.runtime)) + .insert(handle, conn, self.slot.clone(), &self.runtime)) } Err(error) => { if let Some(transmit) = error.response { @@ -555,7 +563,6 @@ pub(crate) struct State { driver: Option, ipv6: bool, events: mpsc::UnboundedReceiver<(ConnectionHandle, EndpointEvent)>, - driver_lost: bool, /// Connections in the process of being accepted pending_accepts: usize, /// QUIC-LB forwarding to and from the other servers behind this one's address @@ -725,8 +732,26 @@ struct ConnectionSet { draining: FxHashSet, /// Stored to give out clones to new ConnectionInners sender: mpsc::UnboundedSender<(ConnectionHandle, EndpointEvent)>, - /// Set if the endpoint has been manually closed - close: Option<(VarInt, String)>, + /// Whether the endpoint still takes connections, which every entry point checks + lifecycle: Lifecycle, +} + +/// Where an endpoint is in its life. Nothing returns it to [`Running`](Self::Running). +#[derive(Debug)] +enum Lifecycle { + /// Accepting and dialing. + Running, + /// [`Endpoint::close`] ran: every connection closes with this code and reason, including one + /// accepted afterwards, and no attempt is dialed or queued again. + Closed { error_code: VarInt, reason: String }, + /// The driver is gone, so nothing can drive a connection again. + DriverLost, +} + +impl Lifecycle { + fn is_running(&self) -> bool { + matches!(self, Self::Running) + } } impl ConnectionSet { @@ -735,24 +760,24 @@ impl ConnectionSet { handle: ConnectionHandle, conn: moq_quic::connection::Connection, slot: Arc, - driver_lost: bool, runtime: &crate::runtime::Spawner, ) -> Connecting { let (send, recv) = mpsc::unbounded_channel(); - if let Some((error_code, reason)) = &self.close { - send.send(ConnectionEvent::Close { - error_code: *error_code, - reason: reason.clone(), - }) - .unwrap(); - } - match driver_lost { - // Close the event channel before spawning the connection driver, so it cannot - // transmit a handshake after the endpoint driver has stopped. - true => drop(send), - false => { + match &self.lifecycle { + Lifecycle::Running => { + self.senders.insert(handle, send); + } + Lifecycle::Closed { error_code, reason } => { + send.send(ConnectionEvent::Close { + error_code: *error_code, + reason: reason.clone(), + }) + .unwrap(); self.senders.insert(handle, send); } + // Close the event channel before spawning the connection driver, so it cannot + // transmit a handshake after the endpoint driver has stopped. + Lifecycle::DriverLost => drop(send), } Connecting::new(handle, conn, self.sender.clone(), recv, slot, runtime) } @@ -805,7 +830,6 @@ impl EndpointRef { ipv6: local_addr.is_ipv6(), events, driver: None, - driver_lost: false, recv_state, pending_accepts: 0, forward, @@ -871,7 +895,7 @@ impl RecvState { senders: FxHashMap::default(), draining: FxHashSet::default(), sender, - close: None, + lifecycle: Lifecycle::Running, }, incoming: VecDeque::new(), recv_buf: recv_buf.into(), @@ -998,7 +1022,7 @@ impl RecvState { let mut response_buffer = Vec::new(); match endpoint.handle(now, received, &mut response_buffer) { Some(DatagramEvent::NewConnection(incoming)) => { - if connections.close.is_none() { + if connections.lifecycle.is_running() { incoming_queue.push_back(incoming); } else { let transmit = endpoint.refuse(incoming, &mut response_buffer); diff --git a/rs/moq-tokio/src/quic/driver/mod.rs b/rs/moq-tokio/src/quic/driver/mod.rs index 48455f90b26..67db48abc5c 100644 --- a/rs/moq-tokio/src/quic/driver/mod.rs +++ b/rs/moq-tokio/src/quic/driver/mod.rs @@ -85,6 +85,7 @@ pub(crate) enum ConnectionEvent { }, Proto(moq_quic::ConnectionEvent), /// The endpoint switched sockets + #[cfg(test)] Rebind, } diff --git a/rs/moq-tokio/src/quic/driver/runtime/mod.rs b/rs/moq-tokio/src/quic/driver/runtime/mod.rs index 494172a4633..767c5c42880 100644 --- a/rs/moq-tokio/src/quic/driver/runtime/mod.rs +++ b/rs/moq-tokio/src/quic/driver/runtime/mod.rs @@ -122,6 +122,7 @@ impl Slot { } /// Send through `socket` from now on, returning the one it replaces + #[cfg(test)] pub(crate) fn replace(&self, socket: Box) -> Option> { let previous = self.socket.write().unwrap().replace(socket); // A sender blocked on the old socket would otherwise wait on it forever. diff --git a/rs/moq-tokio/src/quic/driver/tests.rs b/rs/moq-tokio/src/quic/driver/tests.rs index 3c86beb7f0f..68c07759c29 100755 --- a/rs/moq-tokio/src/quic/driver/tests.rs +++ b/rs/moq-tokio/src/quic/driver/tests.rs @@ -1259,6 +1259,47 @@ async fn refused_incoming_wakes_all_draining_waiter() { assert!(timeout(Duration::from_secs(5), connect).await.unwrap().is_err()); } +/// Nothing revives a closed endpoint or one whose driver is gone: neither dials nor accepts again. +#[tokio::test] +async fn a_closed_or_driverless_endpoint_stays_closed() { + let _guard = subscribe(); + let factory = EndpointFactory::new(); + let mut roots = RootCertStore::empty(); + roots.add(factory.cert.cert.der().clone()).unwrap(); + let config = client::Config::with_root_certificates(Arc::new(roots)).unwrap(); + + let closed = factory.endpoint(); + let addr = closed.local_addr(); + closed.close(0u32.into(), "done").await; + assert!(matches!( + closed.connect(&config, addr, "localhost"), + Err(endpoint::ConnectError::EndpointStopping) + )); + assert!(closed.accept().await.is_none()); + + let failure = Arc::new(SocketFailure::default()); + let socket: Box = Box::new(FailingSocket { + inner: localhost_socket(), + failure: failure.clone(), + }); + let driverless = Endpoint::new( + endpoint::Config::default().with_server(factory.server_config(transport::Config::default())), + socket, + ) + .unwrap(); + failure.trigger(); + assert!( + timeout(Duration::from_secs(5), driverless.accept()) + .await + .unwrap() + .is_none() + ); + assert!(matches!( + driverless.connect(&config, addr, "localhost"), + Err(endpoint::ConnectError::EndpointStopping) + )); +} + /// The endpoint driver may exit while an accept runs without the lock. The connection registered /// afterwards must still be told that the driver is gone, so that its `Connecting` fails promptly /// and `wait_idle` does not wait on a connection nobody drives. @@ -2582,11 +2623,100 @@ struct ControlledSocket { #[derive(Debug, Default)] struct ControlledOutput { blocked: bool, + /// Sends that still pass while blocked + allowed: usize, attempts: usize, writable: Option, parked: kio::WaiterList, } +#[cfg(feature = "sim")] +impl ControlledOutput { + /// Wait until a send parks on the blocked socket, past `attempts` earlier ones + async fn parked(output: &Mutex, attempts: usize) { + kio::wait(|waiter| { + let mut output = output.lock().unwrap(); + if output.writable.is_some() && output.attempts > attempts { + return Poll::Ready(()); + } + waiter.register(&mut output.parked); + Poll::Pending + }) + .await; + } + + /// Let `sends` more through, keeping the socket blocked once they are spent, or every one + /// with `None` + fn release(output: &Mutex, sends: Option) { + let waker = { + let mut output = output.lock().unwrap(); + match sends { + Some(sends) => output.allowed = sends, + None => output.blocked = false, + } + output.writable.take().unwrap() + }; + waker.wake(); + } +} + +/// A client connected to a server whose output the test holds, with the handshake settled +#[cfg(feature = "sim")] +struct Controlled { + output: Arc>, + _client: Endpoint, + server: Endpoint, + /// The client's end + conn: Connection, + /// The server's end, which sends through the held socket + peer: Connection, +} + +#[cfg(feature = "sim")] +impl Controlled { + async fn connect() -> Self { + let network = super::sim::Network::new(1); + let client_addr = SocketAddr::from(([10, 0, 0, 1], 4433)); + let server_addr = SocketAddr::from(([10, 0, 0, 2], 443)); + let factory = EndpointFactory::new(); + let mut transport = transport::Config::default(); + transport.idle_timeout = None; + let mut roots = RootCertStore::empty(); + roots.add(factory.cert.cert.der().clone()).unwrap(); + let mut config = client::Config::with_root_certificates(Arc::new(roots)).unwrap(); + config.transport = transport.clone(); + let output = Arc::new(Mutex::new(ControlledOutput::default())); + let socket: Box = Box::new(ControlledSocket { + inner: network.bind(server_addr), + output: output.clone(), + }); + let client = Endpoint::new(endpoint::Config::default(), network.bind(client_addr)).unwrap(); + let server = Endpoint::new( + endpoint::Config::default().with_server(factory.server_config(transport)), + socket, + ) + .unwrap(); + let (conn, peer) = tokio::try_join!(client.connect(&config, server_addr, "localhost").unwrap(), async { + server.accept().await.unwrap().accept().unwrap().await + }) + .unwrap(); + + // Settle the server's handshake flight and FIN before holding an ACK-only packet. + let warmup = peer.open_uni(Default::default()).await.unwrap().finish(); + assert_eq!(warmup.await, Ok(None)); + // Retire handshake keys on the simulated clock before blocking its next send. + sleep(Duration::from_secs(1)).await; + + Self { + output, + _client: client, + server, + conn, + peer, + } + } +} + #[cfg(feature = "sim")] impl AsyncUdpSocket for ControlledSocket { fn poll_recv( @@ -2604,15 +2734,21 @@ impl AsyncUdpSocket for ControlledSocket { } fn try_send(&self, transmit: &moq_sock::udp::Transmit<'_>) -> io::Result<()> { - if self.output.lock().unwrap().blocked { - return Err(io::ErrorKind::WouldBlock.into()); + { + let mut output = self.output.lock().unwrap(); + if output.blocked { + if output.allowed == 0 { + return Err(io::ErrorKind::WouldBlock.into()); + } + output.allowed -= 1; + } } self.inner.try_send(transmit) } fn poll_writable(&self, cx: &mut Context<'_>) -> Poll> { let mut output = self.output.lock().unwrap(); - if !output.blocked { + if !output.blocked || output.allowed > 0 { return Poll::Ready(Ok(())); } output.attempts += 1; @@ -2632,74 +2768,29 @@ impl AsyncUdpSocket for ControlledSocket { fn a_close_behind_a_blocked_socket_is_bounded() { for release in [false, true] { kio::clock::sim::run(async { - let network = super::sim::Network::new(1); - let client_addr = SocketAddr::from(([10, 0, 0, 1], 4433)); - let server_addr = SocketAddr::from(([10, 0, 0, 2], 443)); - let factory = EndpointFactory::new(); - let mut transport = transport::Config::default(); - transport.idle_timeout = None; - let mut roots = RootCertStore::empty(); - roots.add(factory.cert.cert.der().clone()).unwrap(); - let mut config = client::Config::with_root_certificates(Arc::new(roots)).unwrap(); - config.transport = transport.clone(); - let output = Arc::new(Mutex::new(ControlledOutput::default())); - let socket: Box = Box::new(ControlledSocket { - inner: network.bind(server_addr), - output: output.clone(), - }); - let client = Endpoint::new(endpoint::Config::default(), network.bind(client_addr)).unwrap(); - let server = Endpoint::new( - endpoint::Config::default().with_server(factory.server_config(transport)), - socket, - ) - .unwrap(); - let (conn, peer) = tokio::try_join!(client.connect(&config, server_addr, "localhost").unwrap(), async { - server.accept().await.unwrap().accept().unwrap().await - }) - .unwrap(); - - // Settle the server's handshake flight and FIN before holding an ACK-only packet. - let warmup = peer.open_uni(Default::default()).await.unwrap().finish(); - assert_eq!(warmup.await, Ok(None)); - // Retire handshake keys on the simulated clock before blocking its next send. - sleep(Duration::from_secs(1)).await; + let Controlled { + output, + server, + conn, + peer, + .. + } = Controlled::connect().await; output.lock().unwrap().blocked = true; let mut stream = conn.open_uni(Default::default()).await.unwrap(); stream.write_all(&b"held"[..]).await.unwrap(); let finished = stream.finish(); - kio::wait(|waiter| { - let mut output = output.lock().unwrap(); - if output.writable.is_some() { - return Poll::Ready(()); - } - waiter.register(&mut output.parked); - Poll::Pending - }) - .await; + ControlledOutput::parked(&output, 0).await; // Consume any transmit registration left from before the socket blocked. The driver // must park again without polling transmission, which would consume a packet. let attempts = output.lock().unwrap().attempts; peer.send_datagram(Bytes::from_static(b"queued"), Default::default()) .unwrap(); - kio::wait(|waiter| { - let mut output = output.lock().unwrap(); - if output.attempts > attempts { - return Poll::Ready(()); - } - waiter.register(&mut output.parked); - Poll::Pending - }) - .await; + ControlledOutput::parked(&output, attempts).await; if release { - let waker = { - let mut output = output.lock().unwrap(); - output.blocked = false; - output.writable.take().unwrap() - }; - waker.wake(); + ControlledOutput::release(&output, None); let mut received = peer.accept_uni().await.unwrap(); assert_eq!(received.read_to_end(4).await.unwrap(), b"held"); assert_eq!(finished.await, Ok(None)); @@ -2721,6 +2812,44 @@ fn a_close_behind_a_blocked_socket_is_bounded() { } } +/// A datagram the socket held from before a local close is not the CONNECTION_CLOSE, so +/// sending it leaves `closed` waiting for the close itself. +#[cfg(feature = "sim")] +#[test] +fn closed_waits_past_a_datagram_held_from_before_the_close() { + use futures::FutureExt; + + kio::clock::sim::run(async { + let Controlled { output, conn, peer, .. } = Controlled::connect().await; + + output.lock().unwrap().blocked = true; + peer.send_datagram(Bytes::from_static(b"held"), Default::default()) + .unwrap(); + ControlledOutput::parked(&output, 0).await; + peer.close(7u32.into(), "bye"); + + // Send only the held datagram, so the close behind it parks. + let attempts = output.lock().unwrap().attempts; + ControlledOutput::release(&output, Some(1)); + ControlledOutput::parked(&output, attempts).await; + assert_eq!(output.lock().unwrap().allowed, 0, "the held datagram was not sent"); + assert!( + peer.closed().now_or_never().is_none(), + "resolved before the close was sent" + ); + + ControlledOutput::release(&output, None); + assert!(matches!( + wait_closed(&peer).await, + crate::quic::driver::connection::Error::LocallyClosed { .. } + )); + assert!(matches!( + wait_closed(&conn).await, + crate::quic::driver::connection::Error::ApplicationClosed(_) + )); + }); +} + /// A finished handle keeps the terminal outcome when driven as a future or a caller-owned poll. #[tokio::test] async fn finished_waits_keep_the_terminal_outcome() { @@ -2746,3 +2875,29 @@ async fn finished_waits_keep_the_terminal_outcome() { } } } + +/// A local close resolves `closed` once the driver has sent the CONNECTION_CLOSE, as on the +/// other backends, so a caller that exits the moment it resolves still tells the peer. +#[tokio::test] +async fn closed_waits_for_the_close_to_be_sent() { + use futures::FutureExt; + + let _guard = subscribe(); + let endpoint = endpoint(); + let (conn, server) = connect_self(&endpoint).await; + let sent = conn.stats().udp_tx.datagrams; + conn.close(7u32.into(), "bye"); + assert!(conn.closed().now_or_never().is_none(), "nothing has sent the close yet"); + assert!(matches!( + wait_closed(&conn).await, + crate::quic::driver::connection::Error::LocallyClosed { .. } + )); + assert!( + conn.stats().udp_tx.datagrams > sent, + "resolved before the close was sent" + ); + assert!(matches!( + wait_closed(&server).await, + crate::quic::driver::connection::Error::ApplicationClosed(_) + )); +} diff --git a/rs/moq-tokio/src/quic/web/session.rs b/rs/moq-tokio/src/quic/web/session.rs index 70ea87e003c..55f9b0e9b2f 100644 --- a/rs/moq-tokio/src/quic/web/session.rs +++ b/rs/moq-tokio/src/quic/web/session.rs @@ -32,43 +32,50 @@ use crate::quic::web::{ #[derive(Clone)] pub struct Session { conn: crate::quic::driver::connection::Connection, + mode: Mode, - // The session ID, as determined by the stream ID of the connect request. - session_id: Option, + // Why the session closed: set once by a local close() or the background task + // when a remote CloseWebTransportSession capsule is received. + close: Arc, + + opening: Arc>, +} + +/// Whether a [`Session`] is WebTransport over HTTP/3 or a raw QUIC connection. +#[derive(Clone)] +enum Mode { + /// A raw QUIC connection, with the ALPN its handshake negotiated, copied out of the + /// connection's lock so `protocol()` can borrow it. + Raw { alpn: Option> }, + /// WebTransport over HTTP/3, shared by the session's clones. + Web(Arc), +} + +/// The WebTransport layering a raw QUIC session goes without. +struct Web { + /// The stream ID of the CONNECT request, which names the session. + session_id: VarInt, - // The accept logic is stateful, so use an Arc to share it. - accept: Option>>, + /// The accept logic is stateful, so the clones share it. + accept: Mutex, - // Cache the headers in front of each stream we open. + /// The headers in front of each stream and datagram we send. header_uni: Bytes, header_bi: Bytes, header_datagram: Bytes, - #[allow( - dead_code, - reason = "keep a reference to the settings and connect stream to avoid closing them until dropped" - )] - settings: Option>, - - // The send side of the CONNECT stream, used to write the CloseWebTransportSession capsule. - // Wrapped in Arc>> so close() can take it exactly once. - connect_send: Arc>>, - - // Why the session closed: set once by a local close() or the background task - // when a remote CloseWebTransportSession capsule is received. - close: Arc, + /// Held so the control and QPACK streams stay open until the last clone drops. + _settings: Settings, - // The request sent by the client, or None for a raw QUIC session. - request: Option, + /// The send side of the CONNECT stream, which the first close takes to write the + /// CloseWebTransportSession capsule. + connect_send: Mutex>, - // The response sent by the server, or None for a raw QUIC session. - response: Option, + /// The request sent by the client. + request: ConnectRequest, - // The ALPN the QUIC handshake negotiated, for raw QUIC sessions, copied out of the - // connection's lock so `protocol()` can borrow it. HTTP/3 sessions read the - // subprotocol out of the response instead. - alpn: Option>, - opening: Arc>, + /// The response sent by the server, with the subprotocol it selected. + response: ConnectResponse, } impl Session { @@ -98,19 +105,21 @@ impl Session { let early = std::mem::take(settings.early.get_mut().unwrap()); let accept = SessionAccept::new(conn.clone(), session_id, close.clone(), early); - let this = Self { - conn, - accept: Some(Arc::new(Mutex::new(accept))), - session_id: Some(session_id), + let web = Web { + session_id, + accept: Mutex::new(accept), header_uni: header_uni.into(), header_bi: header_bi.into(), header_datagram: header_datagram.into(), - settings: Some(Arc::new(settings)), - connect_send: Arc::new(Mutex::new(Some(connect.send))), + _settings: settings, + connect_send: Mutex::new(Some(connect.send)), + request: connect.request.clone(), + response: connect.response.clone(), + }; + let this = Self { + conn, + mode: Mode::Web(Arc::new(web)), close: close.clone(), - request: Some(connect.request.clone()), - response: Some(connect.response.clone()), - alpn: None, opening: Default::default(), }; @@ -215,8 +224,8 @@ impl Session { /// The next unidirectional stream, after its WebTransport header. #[kio::wrapper] pub fn poll_accept_uni(&self, waiter: &Waiter) -> Poll> { - if let Some(accept) = &self.accept { - poll_accept_uni_shared(accept, waiter).map_err(|e| self.map_error(e)) + if let Some(web) = self.web() { + poll_accept_uni_shared(&web.accept, waiter).map_err(|e| self.map_error(e)) } else { self.conn .poll_accept_uni(waiter) @@ -228,8 +237,8 @@ impl Session { /// The next bidirectional stream, after its WebTransport header. #[kio::wrapper] pub fn poll_accept_bi(&self, waiter: &Waiter) -> Poll> { - if let Some(accept) = &self.accept { - poll_accept_bi_shared(accept, waiter).map_err(|e| self.map_error(e)) + if let Some(web) = self.web() { + poll_accept_bi_shared(&web.accept, waiter).map_err(|e| self.map_error(e)) } else { self.conn .poll_accept_bi(waiter) @@ -256,7 +265,8 @@ impl Session { } )) .map_err(|e| self.map_error(e))?; - opening.uni = Some((send, self.header_uni.clone())); + let header = self.web().map_or_else(Bytes::new, |web| web.header_uni.clone()); + opening.uni = Some((send, header)); } let (send, header) = opening.uni.as_mut().unwrap(); let result = std::task::ready!(poll_header(send, opening.park.hold(cx), header)); @@ -264,7 +274,7 @@ impl Session { result.map_err(|e| self.map_error(e))?; send.set_priority(options.priority); send.set_group(options.group); - let header = self.session_id.map(|_| send.offset()); + let header = self.web().map(|_| send.offset()); Poll::Ready(Ok(SendStream::new(send, self.close.clone(), header))) }) } @@ -287,7 +297,8 @@ impl Session { )) .map_err(|e| self.map_error(e))?; send.set_priority(i64::MAX); - opening.bi = Some((send, recv, self.header_bi.clone())); + let header = self.web().map_or_else(Bytes::new, |web| web.header_bi.clone()); + opening.bi = Some((send, recv, header)); } let (send, _, header) = opening.bi.as_mut().unwrap(); let result = std::task::ready!(poll_header(send, opening.park.hold(cx), header)); @@ -295,7 +306,7 @@ impl Session { result.map_err(|e| self.map_error(e))?; send.set_priority(options.priority); send.set_group(options.group); - let header = self.session_id.map(|_| send.offset()); + let header = self.web().map(|_| send.offset()); Poll::Ready(Ok(( SendStream::new(send, self.close.clone(), header), RecvStream::new(recv, self.close.clone()), @@ -312,12 +323,12 @@ impl Session { loop { let mut datagram = std::task::ready!(self.conn.poll_recv_datagram(waiter)).map_err(|e| self.map_error(e))?; - let Some(session_id) = self.session_id else { + let Some(web) = self.web() else { return Poll::Ready(Ok(datagram)); }; let mut cursor = Cursor::new(&datagram); match VarInt::decode(&mut cursor) { - Ok(actual) if actual == session_id => { + Ok(actual) if actual == web.session_id => { return Poll::Ready(Ok(datagram.split_off(cursor.position() as usize))); } _ => tracing::debug!("dropping a datagram for an unknown session"), @@ -334,7 +345,8 @@ impl Session { pub fn send_datagram(&self, data: impl Buf, options: send::Options<'_>) -> Result<(), Error> { // Prepend the header naming the session, which is empty for a raw QUIC session. The // payload is chained rather than copied. - let datagram = self.header_datagram.clone().chain(data); + let header = self.web().map_or_else(Bytes::new, |web| web.header_datagram.clone()); + let datagram = header.chain(data); self.conn .send_datagram(datagram, options) .map_err(|e| self.map_error(e))?; @@ -356,7 +368,7 @@ impl Session { self.conn .max_datagram_size() .unwrap_or(0) - .saturating_sub(self.header_datagram.len()) + .saturating_sub(self.datagram_header_len()) } /// Returns the available buffer space for sending datagrams. @@ -365,7 +377,7 @@ impl Session { pub fn datagram_send_buffer_space(&self) -> usize { self.conn .datagram_send_buffer_space() - .saturating_sub(self.header_datagram.len()) + .saturating_sub(self.datagram_header_len()) } /// Close the session with an error code and reason. @@ -393,9 +405,9 @@ impl Session { return; } - if self.session_id.is_some() { + if let Some(web) = self.web() { // Take the send stream for the capsule write. - let send = self.connect_send.lock().unwrap().take(); + let send = web.connect_send.lock().unwrap().take(); if let Some(send) = send { let capsule = web_transport_proto::Capsule::CloseWebTransportSession { @@ -489,6 +501,19 @@ impl Session { self.close.map(e.into()) } + /// The WebTransport layering, or `None` on a raw QUIC session. + fn web(&self) -> Option<&Web> { + match &self.mode { + Mode::Web(web) => Some(web), + Mode::Raw { .. } => None, + } + } + + /// The bytes in front of each datagram we send, naming the session. + fn datagram_header_len(&self) -> usize { + self.web().map_or(0, |web| web.header_datagram.len()) + } + /// Create a new session from a raw QUIC connection. /// /// This is used to pretend like a QUIC connection is a WebTransport session, @@ -504,32 +529,23 @@ impl Session { .and_then(|data| String::from_utf8(data.alpn?).ok()) .map(Arc::from); Self { - alpn, - opening: Default::default(), conn, - session_id: None, - header_uni: Default::default(), - header_bi: Default::default(), - header_datagram: Default::default(), - accept: None, - settings: None, - connect_send: Arc::new(Mutex::new(None)), + mode: Mode::Raw { alpn }, close: Arc::new(CloseReason::new(true)), - request: None, - response: None, + opening: Default::default(), } } /// Returns the [`ConnectRequest`] if this session was established over HTTP/3, /// or `None` for a raw QUIC session. pub fn request(&self) -> Option<&ConnectRequest> { - self.request.as_ref() + self.web().map(|web| &web.request) } /// Returns the [`ConnectResponse`] if this session was established over HTTP/3, /// or `None` for a raw QUIC session. pub fn response(&self) -> Option<&ConnectResponse> { - self.response.as_ref() + self.web().map(|web| &web.response) } /// Returns the application protocol negotiated for this session. @@ -538,9 +554,9 @@ impl Session { /// `WT-Available-Protocols`; for a raw QUIC session it is the negotiated ALPN. /// `None` if neither was negotiated or the ALPN is not valid UTF-8. pub fn protocol(&self) -> Option<&str> { - match &self.response { - Some(response) => response.protocol.as_deref(), - None => self.alpn.as_deref(), + match &self.mode { + Mode::Web(web) => web.response.protocol.as_deref(), + Mode::Raw { alpn } => alpn.as_deref(), } } @@ -549,10 +565,9 @@ impl Session { /// If the peer sent a `CloseWebTransportSession` capsule, the returned error will be /// [`Error::Closed`] with the code and reason from the capsule. /// - /// Unlike [`crate::quic::driver::connection::Connection::closed`], this does **not** return early when - /// [`close()`](Self::close) has been called. It waits for the underlying QUIC - /// connection to shut down, ensuring the `CloseWebTransportSession` capsule has - /// been delivered. Use [`close_reason()`](Self::close_reason) for a non-blocking check. + /// After [`close()`](Self::close), this waits for the `CloseWebTransportSession` capsule + /// to be delivered and then for the QUIC connection to send its own close. Use + /// [`close_reason()`](Self::close_reason) for a non-blocking check. #[kio::wrapper] pub fn poll_closed(&self, waiter: &Waiter) -> Poll { self.conn.poll_closed(waiter).map(|e| self.map_error(e)) @@ -1021,10 +1036,9 @@ impl moq_net::transport::poll::Session for Session { Self::protocol(self) } fn address(&self) -> moq_net::transport::Address { - if self.session_id.is_some() { - moq_net::transport::Address::Uri - } else { - moq_net::transport::Address::Setup + match self.mode { + Mode::Web(_) => moq_net::transport::Address::Uri, + Mode::Raw { .. } => moq_net::transport::Address::Setup, } } fn close(&self, code: u32, reason: &str) { @@ -1059,11 +1073,11 @@ impl moq_net::transport::poll::Session for Session { } /// The connection's exporter on raw QUIC; on WebTransport, this session's own. fn export_keying_material(&self, label: &[u8], context: &[u8]) -> Option<[u8; 32]> { - match self.session_id { + match self.web() { None => self.conn.export_keying_material(label, context), - Some(id) => self.conn.export_keying_material( + Some(web) => self.conn.export_keying_material( WEBTRANSPORT_EXPORTER, - &webtransport_exporter_context(id.into_inner(), label, context)?, + &webtransport_exporter_context(web.session_id.into_inner(), label, context)?, ), } } @@ -1502,7 +1516,10 @@ mod tests { let client = client.unwrap(); let server = server.unwrap(); let conn = server.conn.clone(); - let settings = Arc::downgrade(server.settings.as_ref().unwrap()); + let Mode::Web(web) = &server.mode else { + panic!("an HTTP/3 session"); + }; + let settings = Arc::downgrade(web); tokio::time::pause(); gate.blocked.store(true, Ordering::SeqCst); server.close(2, "unauthorized"); @@ -1799,7 +1816,7 @@ mod tests { assert_eq!(recv.read_to_end(64).await.unwrap(), b"after"); // Closing the QPACK stream would end the peer's HTTP/3 connection, so the session holds it. - let accept = server.accept.as_ref().unwrap().lock().unwrap(); + let accept = server.web().expect("an HTTP/3 session").accept.lock().unwrap(); assert!(accept.qpack_encoder.is_some(), "the QPACK stream was dropped"); } diff --git a/rs/moq-uring/src/quic/driver/connection.rs b/rs/moq-uring/src/quic/driver/connection.rs index 163570a8a28..76785931860 100644 --- a/rs/moq-uring/src/quic/driver/connection.rs +++ b/rs/moq-uring/src/quic/driver/connection.rs @@ -60,26 +60,29 @@ pub(crate) struct State { /// they queue wakes the driver's `poll_transmit`. driver: kio::WaiterList, - /// The terminal error as the handle's `poll_closed` reports it, set exactly - /// once: moq-quic's reason, published by the driver (a local close only once - /// staged, see `Driver::publish_close`), or the driver's own failure, which - /// moq-quic cannot take and so overrides its reason everywhere. + /// How far the connection's end has reached the handles' `poll_closed`. /// /// A latch of its own rather than moq-quic's `poll_closed`, which resolves a /// local close as soon as it is asked for. Only the driver knows when the /// CONNECTION_CLOSE is staged on the socket, which moq-quic cannot see, and /// a caller may stop driving the worker the moment `poll_closed` resolves, /// so publishing earlier would leave the packet unsent. - closed: Option, + end: End, closed_waiters: kio::WaiterList, +} - /// The socket died under us: the driver stops where it stands rather than - /// waiting out a drain it could never transmit. - dead: bool, - - /// The close this side asked for, as moq-quic reported it, until the driver - /// has put it on the wire. - local_close: Option, +/// The connection's terminal error as its handles see it, set once. +enum End { + /// `poll_closed` pends: the connection is open, or this side closed it and the + /// driver has yet to stage the CONNECTION_CLOSE (see `Driver::publish_close`). + Open, + /// moq-quic's reason, published by the driver, which runs out the drain. + Closed(Error), + /// Failed from outside (the socket died, or the worker dropped the driver): the + /// driver stops where it stands rather than wait out a drain it could never + /// transmit. Its reason wins over moq-quic's, which cannot take it, unless one + /// was already published. + Dead(Error), } impl State { @@ -87,20 +90,42 @@ impl State { Self { active: None, driver: kio::WaiterList::new(), - closed: None, + end: End::Open, closed_waiters: kio::WaiterList::new(), - dead: false, - local_close: None, } } - /// Terminate with `err` (the first one wins) and wake the closed waiters and - /// the driver: an externally failed driver has to notice. - fn fail(&mut self, err: Error) { - self.active.take(); - if self.closed.is_none() { - self.closed = Some(err); + /// The published terminal error, if any. + fn published(&self) -> Option<&Error> { + match &self.end { + End::Open => None, + End::Closed(err) | End::Dead(err) => Some(err), } + } + + /// Publish `err` (the first one wins), waking the closed waiters and the driver. + /// + /// Only the first wakes anyone: the driver checks on every poll, and waking itself + /// again for an end already published would spin it through the drain. + fn publish(&mut self, err: Error) { + if let End::Open = self.end { + self.end = End::Closed(err); + self.ended(); + } + } + + /// Fail from outside with `err`, unless a reason was already published, and + /// stop the driver: an externally failed driver has to notice. + fn kill(&mut self, err: Error) { + self.end = match std::mem::replace(&mut self.end, End::Open) { + End::Open => End::Dead(err), + End::Closed(err) | End::Dead(err) => End::Dead(err), + }; + self.ended(); + } + + fn ended(&mut self) { + self.active.take(); self.driver.wake(); self.closed_waiters.wake(); } @@ -113,7 +138,7 @@ impl Inner { /// Unlike the handle's `poll_closed`, a local close counts from the moment it /// was asked for, as moq-quic's stream waits already fail by then. pub(crate) fn closed(&self) -> Option { - if let Some(err) = &self.state.borrow().closed { + if let Some(err) = self.state.borrow().published() { return Some(err.clone()); } self.conn.borrow().error().cloned().map(Error::from) @@ -152,11 +177,7 @@ impl Inner { /// every handle parked in it; they then report `err`, which wins. pub(crate) fn fail(&self, err: Error) { let reason = err.to_string(); - { - let mut state = self.state.borrow_mut(); - state.dead = true; - state.fail(err); - } + self.state.borrow_mut().kill(err); self.conn .borrow_mut() .close(self.owner.now(), VarInt::from_u32(0), reason); @@ -389,7 +410,6 @@ pub(crate) fn launch( endpoint, key, deadline: owner.timer(), - close_staged: false, turn: 0, trains: 0, }; @@ -592,7 +612,7 @@ fn recv_datagram(shared: &Shared, waiter: &kio::Waiter) -> Poll Poll { let mut state = shared.state.borrow_mut(); - if let Some(err) = &state.closed { + if let Some(err) = state.published() { return Poll::Ready(err.clone()); } waiter.register(&mut state.closed_waiters); @@ -667,10 +687,6 @@ struct Driver { endpoint: Weak, key: ConnectionHandle, deadline: crate::Timer, - /// A CONNECTION_CLOSE has been handed to the socket. Not implied by a - /// flush that came back empty: moq-quic paces the close like any other packet, - /// so it can sit behind a pacing timer after the application asked for it. - close_staged: bool, /// The worker turn `trains` counts in. turn: u64, /// Trains staged this turn. @@ -685,7 +701,8 @@ struct Driver { /// in its waits. impl Drop for Driver { fn drop(&mut self) { - if self.shared.state.borrow().closed.is_none() { + let open = matches!(self.shared.state.borrow().end, End::Open); + if open { self.shared .fail(Error::Io(crate::shared::Shared::gone_error().to_string())); } @@ -698,7 +715,7 @@ impl Driver { let mut state = self.shared.state.borrow_mut(); // The endpoint fails the connection from outside when its socket // dies; there is nothing left to drain it through. - if state.dead { + if let End::Dead(_) = state.end { return Poll::Ready(()); } // Register for ingress first: a datagram the endpoint feeds the @@ -710,7 +727,7 @@ impl Driver { self.endpoint_events(); self.check_lost(waiter); - if self.shared.state.borrow().dead { + if matches!(self.shared.state.borrow().end, End::Dead(_)) { return Poll::Ready(()); } // A closed connection still owes the peer its CONNECTION_CLOSE @@ -727,9 +744,6 @@ impl Driver { self.shared.fail(err); return Poll::Ready(()); } - if self.close_staged { - self.publish_close(); - } // Arm, *then* poll: the poll is what registers the waiter, so // polling before the set would leave the firing to wake nobody @@ -770,16 +784,10 @@ impl Driver { /// `poll_closed` resolves has at least handed the packet over. fn check_lost(&mut self, waiter: &kio::Waiter) { let lost = self.shared.conn.borrow_mut().poll_closed(waiter); - let Poll::Ready(reason) = lost else { - return; - }; - let mut state = self.shared.state.borrow_mut(); - if state.closed.is_some() || state.local_close.is_some() { - return; - } - match reason { - reason @ moq_quic::connection::Error::LocallyClosed { .. } => state.local_close = Some(reason.into()), - reason => state.fail(reason.into()), + if let Poll::Ready(reason) = lost + && !matches!(reason, moq_quic::connection::Error::LocallyClosed { .. }) + { + self.shared.state.borrow_mut().publish(reason.into()); } } @@ -791,9 +799,9 @@ impl Driver { /// fire-and-forget, so a worker torn down in the same breath can still /// take the packet with it and leave the peer to idle out. fn publish_close(&mut self) { - let mut state = self.shared.state.borrow_mut(); - if let Some(err) = state.local_close.take() { - state.fail(err); + let reason = self.shared.conn.borrow().error().cloned(); + if let Some(reason) = reason { + self.shared.state.borrow_mut().publish(reason.into()); } } @@ -877,7 +885,47 @@ impl Driver { if let Err(err) = tx.send(transmit) { return Poll::Ready(Err(Error::Io(err.to_string()))); } - self.close_staged |= closing; + // Staged, not implied by a flush that came back empty: moq-quic paces the close like + // any other packet, so it can sit behind a pacing timer after it was asked for. + if closing { + self.publish_close(); + } Poll::Ready(Ok(())) } } + +#[cfg(test)] +mod tests { + use std::sync::Arc; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::task::{Wake, Waker}; + + use super::{Error, State}; + + struct Wakes(AtomicUsize); + + impl Wake for Wakes { + fn wake(self: Arc) { + self.0.fetch_add(1, Ordering::Relaxed); + } + } + + /// The driver parks on the end and republishes it on every poll, so only the first publish + /// may wake it, or a remote close spins it through the drain. + #[test] + fn an_end_published_again_wakes_nobody() { + let wakes = Arc::new(Wakes(AtomicUsize::new(0))); + let mut state = State::new(); + + let driver = kio::Waiter::new(Waker::from(wakes.clone())); + driver.register(&mut state.driver); + state.publish(Error::TimedOut); + assert_eq!(wakes.0.load(Ordering::Relaxed), 1); + + let driver = kio::Waiter::new(Waker::from(wakes.clone())); + driver.register(&mut state.driver); + state.publish(Error::TimedOut); + assert_eq!(wakes.0.load(Ordering::Relaxed), 1, "woke the driver again"); + assert!(matches!(state.published(), Some(Error::TimedOut))); + } +} diff --git a/rs/moq-uring/src/quic/web.rs b/rs/moq-uring/src/quic/web.rs index faccff8045e..2ee216c4119 100644 --- a/rs/moq-uring/src/quic/web.rs +++ b/rs/moq-uring/src/quic/web.rs @@ -331,6 +331,9 @@ struct Web { /// `CloseWebTransportSession` capsule, or the local [`Session::close`]. /// First writer wins. closed: RefCell>, + request: proto::ConnectRequest, + /// The response sent, with the subprotocol it selected. + response: proto::ConnectResponse, } struct State { @@ -359,12 +362,8 @@ pub struct Session { opening: Rc>, handles: Rc, conn: Connection, + /// The WebTransport layering, or `None` for raw QUIC. web: Option>, - /// What [`protocol`](Self::protocol) - /// reports: the selected subprotocol in web mode, the ALPN in raw mode. - alpn: Option, - request: Option, - response: Option, } impl Session { @@ -407,15 +406,11 @@ impl Session { /// Wrap a raw-QUIC connection in a session, with no /// WebTransport layering: streams and datagrams pass through untouched. pub fn raw(conn: Connection) -> Self { - let alpn = conn.protocol().map(str::to_owned); Self { opening: Default::default(), handles: Handles::new(&conn, None), conn, web: None, - alpn, - request: None, - response: None, } } @@ -476,6 +471,8 @@ impl Session { connect_send: Some(send), }), closed: RefCell::new(None), + request, + response, }); // The peer signals session close with a capsule on the CONNECT @@ -490,9 +487,6 @@ impl Session { handles: Handles::new(&conn, Some(&web)), conn, web: Some(web), - alpn: None, - request: Some(request), - response: Some(response), } } @@ -509,9 +503,6 @@ impl Clone for Session { handles: self.handles.clone(), conn: self.conn.clone(), web: self.web.clone(), - alpn: self.alpn.clone(), - request: self.request.clone(), - response: self.response.clone(), } } } @@ -705,18 +696,18 @@ impl Session { } /// The CONNECT request, or `None` for raw QUIC. pub fn request(&self) -> Option<&proto::ConnectRequest> { - self.request.as_ref() + self.web.as_ref().map(|web| &web.request) } /// The CONNECT response, or `None` for raw QUIC. pub fn response(&self) -> Option<&proto::ConnectResponse> { - self.response.as_ref() + self.web.as_ref().map(|web| &web.response) } /// The negotiated WebTransport subprotocol, or raw QUIC ALPN. pub fn protocol(&self) -> Option<&str> { - self.response - .as_ref() - .and_then(|response| response.protocol.as_deref()) - .or(self.alpn.as_deref()) + match &self.web { + Some(web) => web.response.protocol.as_deref(), + None => self.conn.protocol(), + } } /// Close the session with an application code and reason. pub fn close(&self, code: u32, reason: &str) { diff --git a/rs/qmux/src/driver.rs b/rs/qmux/src/driver.rs index b647c59caee..365483962ee 100644 --- a/rs/qmux/src/driver.rs +++ b/rs/qmux/src/driver.rs @@ -158,12 +158,26 @@ impl std::fmt::Debug for Driver { enum Phase { /// Moving records both ways. Open, - /// The connection said everything; closing the transport until this deadline. + /// The connection ended, by a close from either side; still moving records both ways to + /// deliver what it has left to say, until this deadline. + Flushing(Instant), + /// The connection said everything; closing the transport until this deadline, the same one + /// [`Self::Flushing`] had. Closing(Instant), /// The transport is closed and the session's end published. Done, } +impl Phase { + /// When a closing session gives up, if it is closing. + fn deadline(&self) -> Option { + match self { + Self::Flushing(deadline) | Self::Closing(deadline) => Some(*deadline), + Self::Open | Self::Done => None, + } + } +} + /// What one round of reading and writing came to. enum Step { /// Did something; go again. @@ -209,7 +223,7 @@ impl Run { } self.finish(Ok(())); } - Phase::Open => match self.step(waiter, recv) { + Phase::Open | Phase::Flushing(_) => match self.step(waiter, recv) { Step::Progress => { rounds += 1; if rounds == ROUNDS { @@ -219,7 +233,11 @@ impl Run { } Step::Idle => return Poll::Pending, Step::Drained => { - let deadline = self.timer.closing.unwrap_or_else(|| kio::clock::now() + CLOSE_TIMEOUT); + // Drained means closed, so `step` already started the close bound. + let deadline = self + .phase + .deadline() + .unwrap_or_else(|| kio::clock::now() + CLOSE_TIMEOUT); self.phase = Phase::Closing(deadline); } Step::Failed(err) => self.finish(Err(err)), @@ -288,8 +306,8 @@ impl Run { // room for, so wait for the close with `poll_closed`; the transport wakes us for room. let _ = state.conn.poll_closed(waiter); state.check_lost(); - if state.conn.is_closed() && self.timer.closing.is_none() { - self.timer.closing = Some(now + CLOSE_TIMEOUT); + if state.conn.is_closed() && matches!(self.phase, Phase::Open) { + self.phase = Phase::Flushing(now + CLOSE_TIMEOUT); } let drained = state.conn.is_drained(); let deadline = state.conn.poll_timeout(); @@ -337,7 +355,8 @@ impl Run { if drained && self.outbound.is_empty() && self.room { return Step::Drained; } - if self.timer.closing.is_some_and(|at| now >= at) { + let closing = self.phase.deadline(); + if closing.is_some_and(|at| now >= at) { // The close never got through a stalled transport. return Step::Failed(Error::Closed); } @@ -347,7 +366,7 @@ impl Run { if more_to_read || (self.room && (gathered >= BATCH_BYTES || !had_room)) { return Step::Progress; } - let deadline = match (deadline, self.timer.closing) { + let deadline = match (deadline, closing) { (Some(a), Some(b)) => Some(a.min(b)), (a, b) => a.or(b), }; @@ -392,7 +411,7 @@ fn deliver(state: &mut State, record: Result, now: Instant) { // The transport's own error says more than the connection's reset, unless this side // is closing, when the peer hanging up is the expected answer and the driver // publishes the close it asked for. - if state.local_close.is_none() { + if !state.closing() { state.fail(err); } state.conn.transport_closed(); @@ -403,26 +422,20 @@ fn deliver(state: &mut State, record: Result, now: Instant) { /// The transport is done: publish a local close, or the failure that ended it. fn finish(inner: &Inner, result: Result<(), Error>) { let mut state = inner.lock(); - state.check_lost(); - if let Some(err) = state.local_close.take() { - state.fail(err); - } + // A close from either side comes first: the driver is done delivering it. + state.publish(); if let Err(err) = result { state.fail(err); } + // Otherwise the transport died under a live connection, which this ends. state.conn.transport_closed(); - state.check_lost(); - // A drained connection ended with a close from either side, already published; this only - // lands when the transport died under a live one. - state.fail(Error::Closed); + state.publish(); } -/// The driver's timers. +/// The driver's timer. #[derive(Default)] struct Timer { sleep: Option, - /// When a closing session gives up on delivering its close frame. - closing: Option, /// Writes are stalled on the transport. stalled: bool, } diff --git a/rs/qmux/src/session.rs b/rs/qmux/src/session.rs index 49d5e5782e3..829d79d8a50 100644 --- a/rs/qmux/src/session.rs +++ b/rs/qmux/src/session.rs @@ -37,17 +37,13 @@ pub(crate) struct State { /// /// The connection's `poll_closed` resolves the moment it is closed, but a close this side /// asked for is published here only once the driver delivered it and shut the transport, or - /// gave up after [`CLOSE_TIMEOUT`](crate::driver::CLOSE_TIMEOUT) (see `local_close`), so - /// [`Session`]'s `poll_closed` keeps this latch and its own list. + /// gave up after [`CLOSE_TIMEOUT`](crate::driver::CLOSE_TIMEOUT) (see [`Self::closing`]), + /// so [`Session`]'s `poll_closed` keeps this latch and its own list. /// Only the driver knows when the close frame reached the transport, which the sans-I/O /// connection cannot see, and a caller may exit the moment `poll_closed` resolves, so /// publishing earlier would leave the frame unwritten. closed: Option, closed_waiters: kio::WaiterList, - /// The error for a close this side asked for, published once the driver has delivered - /// the close frame or gave up on a stalled one, so a caller that exits the moment - /// `poll_closed` resolves already handed it to the transport. - pub(crate) local_close: Option, } impl State { @@ -66,22 +62,29 @@ impl State { /// Publish a loss the connection recorded since the last check: a remote close or a /// violation fails the session now, while a close this side asked for waits for the driver - /// to deliver its frame. + /// to deliver its frame and [`publish`](Self::publish) it. /// /// Called after every call that can lose the connection: a record, a timeout, a close. pub(crate) fn check_lost(&mut self) { - if self.closed.is_some() || self.local_close.is_some() { - return; + if self.closed.is_none() && !self.closing() { + self.publish(); } - let Some(reason) = self.conn.error() else { + } + + /// Whether this side closed the connection, which the session reports only once the driver + /// is done delivering the close. The connection keeps the first reason it ended with. + pub(crate) fn closing(&self) -> bool { + matches!(self.conn.error(), Some(ConnectionError::LocallyClosed { .. })) + } + + /// Publish why the connection ended, if it has, a close this side asked for included. + pub(crate) fn publish(&mut self) { + if self.closed.is_some() { return; - }; - let local = matches!(reason, ConnectionError::LocallyClosed { .. }); - let err = Error::lost(reason.clone()); - match local { - // The driver publishes a local close once its frame is out. - true => self.local_close = Some(err), - false => self.fail(err), + } + if let Some(reason) = self.conn.error() { + let err = Error::lost(reason.clone()); + self.fail(err); } } @@ -181,7 +184,6 @@ impl Session { conn, closed: None, closed_waiters: kio::WaiterList::new(), - local_close: None, }), wire: net::Wire::new(), socket_stats: transport.socket_stats(), @@ -489,15 +491,24 @@ impl std::fmt::Debug for SendGroup { } } -/// An outgoing stream. Dropping it unfinished resets it with code 0. +/// An outgoing stream. Dropping it unfinished resets it with code 0, and dropping one the peer +/// stopped echoes the stop's code. pub struct SendStream { inner: Arc, id: StreamId, - /// The FIN was queued; further writes are refused and `poll_closed` waits for it to go - /// out. - fin: bool, + end: SendEnd, +} + +/// How this side ended a [`SendStream`]. A peer's STOP_SENDING is not one: the connection keeps +/// it, and reports it to every call. +#[derive(Clone, Copy, PartialEq, Eq)] +enum SendEnd { + /// Still writable. + Open, + /// The FIN was queued; further writes are refused and `poll_closed` waits for it to go out. + Finished, /// We reset the stream; it is as closed as it will ever be. - reset: bool, + Reset, } impl SendStream { @@ -505,20 +516,19 @@ impl SendStream { Self { inner, id, - fin: false, - reset: false, + end: SendEnd::Open, } } fn reset_with(&mut self, code: u32) { - if self.reset { + if self.end == SendEnd::Reset { return; } let code = VarInt::from_u32(code); let mut state = self.inner.lock(); // Err means the stream is already gone, which is what we wanted. let _ = state.conn.send_stream(self.id).reset(code); - self.reset = true; + self.end = SendEnd::Reset; } } @@ -531,7 +541,7 @@ impl poll::SendStream for SendStream { } fn poll_write_buf(&mut self, waiter: &kio::Waiter, buf: &mut B) -> Poll> { - if self.fin || self.reset { + if self.end != SendEnd::Open { return Poll::Ready(Err(Error::StreamClosed)); } let mut state = self.inner.lock(); @@ -555,22 +565,19 @@ impl poll::SendStream for SendStream { } fn finish(&mut self) -> Result<(), Error> { - if self.fin || self.reset { + if self.end != SendEnd::Open { return Ok(()); } let mut state = self.inner.lock(); match state.conn.send_stream(self.id).finish() { Ok(()) => {} - // A STOP_SENDING beat us here; carry its code like `poll_write` does. The stream is - // over without our reset, so `poll_closed` reports the stop the connection recorded. - Err(send::Error::Stopped(code)) => { - self.fin = true; - return Err(Error::StreamStop(code.into_inner())); - } + // A STOP_SENDING beat us here; carry its code like `poll_write` does. No FIN was + // queued, so the stream stays open for the drop to answer the stop with a reset. + Err(send::Error::Stopped(code)) => return Err(Error::StreamStop(code.into_inner())), // Already finished or reset, so the FIN it wanted is out. Err(_) => {} } - self.fin = true; + self.end = SendEnd::Finished; Ok(()) } @@ -579,7 +586,7 @@ impl poll::SendStream for SendStream { } fn poll_closed(&self, waiter: &kio::Waiter) -> Poll> { - if self.reset { + if self.end == SendEnd::Reset { return Poll::Ready(Ok(())); } let mut state = self.inner.lock(); @@ -599,7 +606,7 @@ impl poll::SendStream for SendStream { /// Whether every byte written so far went to the transport, which acknowledges it on a /// reliable transport. fn acknowledged(&self) -> Option { - if self.reset { + if self.end == SendEnd::Reset { return Some(false); } let mut state = self.inner.lock(); @@ -612,11 +619,24 @@ impl poll::SendStream for SendStream { impl Drop for SendStream { fn drop(&mut self) { let mut state = self.inner.lock(); - // Reset before giving up the ID, which refuses every later call on it. - if !self.fin && !self.reset { - let _ = state.conn.send_stream(self.id).reset(VarInt::from_u32(0)); - } - state.conn.send_stream(self.id).release(); + let mut stream = state.conn.send_stream(self.id); + let stopped = match stream.end() { + Ok(Some(send::End::Stopped(code))) => Some(code), + _ => None, + }; + // Reset before giving up the ID, which refuses every later call on it. Err means the + // stream is already gone, which is what we wanted. + let _ = match (self.end, stopped) { + (SendEnd::Reset, _) => Ok(()), + // Echo the code, as RFC 9000 section 3.5 asks, even after `finish`: the peer reads + // nothing more, so whatever is still queued is wasted. A stream stopped before its + // FIN needs it most, or the peer never learns its final size and never returns its + // credit. + (_, Some(code)) => stream.reset(code), + (SendEnd::Finished, None) => Ok(()), + (SendEnd::Open, None) => stream.reset(VarInt::from_u32(0)), + }; + stream.release(); } } @@ -635,11 +655,30 @@ impl std::fmt::Debug for SendStream { pub struct RecvStream { inner: Arc, id: StreamId, + end: RecvEnd, +} + +/// How a [`RecvStream`] ended, once the connection no longer keeps its state. +#[derive(Clone, Copy, PartialEq, Eq)] +enum RecvEnd { + /// The connection keeps the stream, and answers for it. + Open, /// The peer reset the stream with this code. Observing it frees the connection's stream /// state, so later calls report it from here. - reset: Option, + Reset(u64), /// We stopped the stream; it is gone. - stopped: bool, + Stopped, +} + +impl RecvEnd { + /// What every call reports once the stream ended. + fn error(self) -> Option { + match self { + Self::Open => None, + Self::Reset(code) => Some(Error::StreamReset(code)), + Self::Stopped => Some(Error::StreamClosed), + } + } } impl RecvStream { @@ -647,19 +686,15 @@ impl RecvStream { Self { inner, id, - reset: None, - stopped: false, + end: RecvEnd::Open, } } /// Take the connection's lease, which [`RecvLease`] wraps once this returns: the borrow of /// `self` it holds cannot start inside a poll that may return `Pending`. fn poll_take(&mut self, waiter: &kio::Waiter, want: usize) -> Poll> { - if let Some(code) = self.reset { - return Poll::Ready(Err(Error::StreamReset(code))); - } - if self.stopped { - return Poll::Ready(Err(Error::StreamClosed)); + if let Some(err) = self.end.error() { + return Poll::Ready(Err(err)); } let mut state = self.inner.lock(); // moq-quic wakes the driver for any credit the lease frees, and the reader of any stream @@ -671,7 +706,7 @@ impl RecvStream { // Observing the reset frees the stream and the credit its lease withheld; only // then may later calls skip it. let _ = state.conn.recv_stream(self.id).received_reset(); - self.reset = Some(code.into_inner()); + self.end = RecvEnd::Reset(code.into_inner()); Poll::Ready(Err(Error::StreamReset(code.into_inner()))) } Err(recv::Error::Evicted) => Poll::Ready(Err(Error::Evicted)), @@ -697,35 +732,32 @@ impl poll::RecvStream for RecvStream { } fn set_priority(&mut self, priority: i64) { - if self.reset.is_some() || self.stopped { + if self.end != RecvEnd::Open { return; } self.inner.lock().conn.recv_stream(self.id).set_priority(priority); } fn stop(&mut self, code: u32) { - if self.stopped || self.reset.is_some() { + if self.end != RecvEnd::Open { return; } let mut state = self.inner.lock(); // Err means the stream is already gone, which is what we wanted. let _ = state.conn.recv_stream(self.id).stop(VarInt::from_u32(code)); - self.stopped = true; + self.end = RecvEnd::Stopped; } fn poll_reset(&mut self, waiter: &kio::Waiter) -> Poll { - if let Some(code) = self.reset { - return Poll::Ready(Error::StreamReset(code)); - } - if self.stopped { - return Poll::Ready(Error::StreamClosed); + if let Some(err) = self.end.error() { + return Poll::Ready(err); } let mut state = self.inner.lock(); match ready!(state.conn.recv_stream(self.id).poll_received_reset(waiter)) { Ok(code) => { // Observing it freed the stream, and moq-quic wakes the driver for any credit // that owes the peer. - self.reset = Some(code.into_inner()); + self.end = RecvEnd::Reset(code.into_inner()); Poll::Ready(Error::StreamReset(code.into_inner())) } // Evicted, so its bytes are gone: nothing frees the window until it is stopped. @@ -739,7 +771,7 @@ impl poll::RecvStream for RecvStream { impl Drop for RecvStream { fn drop(&mut self) { - if self.stopped || self.reset.is_some() { + if self.end != RecvEnd::Open { return; } let mut state = self.inner.lock(); diff --git a/rs/qmux/src/session/tests.rs b/rs/qmux/src/session/tests.rs index c71c4e603b3..7ab9d8e4558 100644 --- a/rs/qmux/src/session/tests.rs +++ b/rs/qmux/src/session/tests.rs @@ -409,6 +409,35 @@ async fn resets_and_stops_carry_their_codes() { assert!(matches!(send.closed().await, Err(Error::StreamStop(11)))); } +/// A stream the peer stopped answers with a reset, as RFC 9000 section 3.5 requires, even when +/// a `finish` came after the stop: otherwise the peer never learns its final size, and never +/// returns its stream credit. +#[tokio::test(start_paused = true)] +async fn a_stopped_stream_resets_even_after_finish() { + let mut server = config(Version::QMux02); + server.limits.max_streams = std::num::NonZeroU64::new(1); + let (client, server) = pair(config(Version::QMux02), server).await; + + let mut send = client.open_uni(SendOptions::default()).await.unwrap(); + write_all(&mut send, Bytes::from_static(b"refused")).await; + let mut recv = server.accept_uni().await.unwrap(); + recv.stop(11); + drop(recv); + kio::clock::sleep(Duration::from_millis(1)).await; + assert!(matches!(send.finish(), Err(Error::StreamStop(11)))); + assert!( + matches!(send.write(b"more").await, Err(Error::StreamStop(11))), + "still reads as the stop" + ); + drop(send); + + let opening = client.open_uni(SendOptions::default()); + kio::clock::timeout(Duration::from_secs(1), opening) + .await + .expect("credit returned once the stopped stream is reset") + .unwrap(); +} + /// A send stream dropped before `finish` resets with code 0, so its reader learns it ended. #[tokio::test(start_paused = true)] async fn dropping_an_unfinished_send_resets_it() {