diff --git a/quest/m1/quic/group-reset.md b/quest/m1/quic/group-reset.md index 1b851760e6a..2754f19bdea 100644 --- a/quest/m1/quic/group-reset.md +++ b/quest/m1/quic/group-reset.md @@ -14,9 +14,8 @@ browser, keep the grace. Split from the transport half, which landed in `moq-quic`, `moq_tokio::quic::web`, and moq-uring's WebTransport adapter (#5690). There, -`SendStream::checkpoint()` marks the data written so far and -`Checkpoint::reset(code)` sends `RESET_STREAM_AT` with that offset as the -Reliable Size. A peer without the extension instead gets a plain +`SendStream::reset_at(offset, code)` sends `RESET_STREAM_AT` with that +offset as the Reliable Size. A peer without the extension instead gets a plain `RESET_STREAM` once it has acknowledged the data before the checkpoint, a round trip later. The WebTransport adapters already keep their own stream header this way. @@ -29,9 +28,11 @@ read by the application, so check whether Chrome hands it to JS when a plain `RESET_STREAM` follows the ACK, and treat a peer without `reset_stream_at` as unable to keep the header unless it does. -- Give moq-net's `transport::poll::SendStream` a reset that keeps a - checkpoint, using `Checkpoint` rather than a byte count, so a size past the - written data cannot be asked for. Implement it in the moq-tokio and +- Give moq-net's `transport::poll::SendStream` a reset that keeps the first + `offset` bytes, matching `moq-quic`'s `reset_at(offset, code)`. Decided + 2026-10-17 on #5817: a byte offset rather than a `Checkpoint` handle; an + offset past the written data panics instead of being unrepresentable. + Implement it in the moq-tokio and moq-uring adapters. Backends that cannot do it (browser, qmux, iroh) report that rather than silently sending a plain reset. - The lite publisher checkpoints right after writing a group header and @@ -58,7 +59,5 @@ half did not reach. - [SUBSCRIBE_DROP accounts for every group](/quest/m1/subscribe-drop.md) - a DROP covers a reset whose header was lost; a reset that keeps its header acts as a one-group drop -- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - adds - `Checkpoint::acked()` to the same handle - [qmux on the QUIC stream state machine](/quest/m2/quic-qmux.md) - qmux reuses `moq-quic`'s reset state diff --git a/quest/m2/README.md b/quest/m2/README.md index 39219c30c5b..95fb3ae0078 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -111,8 +111,6 @@ ranked just below deterministic TS legs, which it requires. - [Drop web-transport-trait](/quest/m2/drop-web-transport-trait.md) - every backend implements moq-net's `transport::poll` traits, and the generic `web_transport_trait` adapters are deleted - [Measure ECN on the backbone](/quest/m2/quic-ecn-measure.md) - a written verdict on marking versus dropping, and whether Linode and OVH keep marks -- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - `moq-quic` reports - how far a send stream has been acknowledged and when - [poll_acked on moq-net's send stream](/quest/m2/quic-ack-hook.md) - the backend-neutral hook on moq-net's own transport trait that awaits an acknowledged stream offset, implemented by the moq-tokio and moq-uring adapters diff --git a/quest/m2/quic-ack-hook.md b/quest/m2/quic-ack-hook.md index e3706528728..9ca0cba8a9a 100644 --- a/quest/m2/quic-ack-hook.md +++ b/quest/m2/quic-ack-hook.md @@ -22,11 +22,14 @@ wakeups a latency sample needs. Add one method to `SendStream`, in the trait's poll style: ```rust -fn poll_acked(&mut self, cx: &mut Context, offset: u64) -> Poll, Self::Error>>; +fn poll_acked(&mut self, cx: &mut Context, offset: u64) -> Poll, Self::Error>>; ``` It is ready with `Some` once every byte below `offset` has been acknowledged. -`Acked` carries the ACK-delay-corrected receive instant from `moq-quic`. It resolves +It carries no instant: the caller records one when woken, which includes the +peer's delayed-ACK timer (up to `max_ack_delay`) as well as the return trip. +Decided 2026-10-17 on #5817, which dropped `moq-quic`'s ACK-delay-corrected +instant until a consumer shows it matters. It resolves with an error once the stream is reset by either side or the session closes, so a waiter never hangs on bytes the peer will never acknowledge. The default implementation is ready with `Ok(None)` immediately: `None` means the backend @@ -36,7 +39,15 @@ unsupported has to live in the return type rather than the error, and a consumer must treat `None` as unknown, never as delivered. Implement it in the moq-tokio and moq-uring adapters over `moq-quic`'s -accessor. Leave the browser transport on the default; the browser's `WebTransportSendStream.getStats()` is +`SendStream::acked(offset)`, which returns whether every byte before the +offset is acknowledged and fails once it never will be. A `false` asks +`moq-quic` to report `StreamEvent::Acked { id, offset }` when the stream's +acknowledged prefix reaches it, so the adapter wakes waiters at or below that +offset; moq-uring's `sweep_stream` currently ignores the event. `acked` fails +once the stream is freed, after `StreamEvent::Finished`, which means +everything was acknowledged, so the adapter answers from that. The trait +method takes a byte offset, like `moq-quic` (decided 2026-10-17 on #5817); +the starvation quest's writer counts the bytes it writes. Leave the browser transport on the default; the browser's `WebTransportSendStream.getStats()` is unimplemented in shipping Chrome and its `bytesAcknowledged` is at risk in the W3C draft. qmux over a reliable transport may treat serialization as acknowledgment only if the qmux quest decides that is honest; until then it @@ -51,11 +62,6 @@ STOP_SENDING by the receiver, and session close. Decided in the 2026-09-30 audit: moved to m2 with its only consumer, [frame-granularity starvation](/quest/m2/starvation-frames.md). -## Required - -- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - the - `moq-quic` accessor the adapter reads - ## Related - [Starvation at frame granularity](/quest/m2/starvation-frames.md) - the diff --git a/quest/m2/quic-ack-progress.md b/quest/m2/quic-ack-progress.md deleted file mode 100644 index ac73c222784..00000000000 --- a/quest/m2/quic-ack-progress.md +++ /dev/null @@ -1,51 +0,0 @@ -# [S] moq-quic reports per-stream acknowledgment progress - -## Goal - -`moq-quic` lets a sender ask how much of a send stream the peer has -acknowledged and when each acknowledgment was received, corrected for the -peer's reported ACK delay. Nothing MoQ builds on top needs private state. - -## Plan - -The work lives in `moq-quic`. - -`moq-quic`'s `SendBuffer` (`connection/send_buffer.rs`) already tracks most -of what is needed: the acknowledged range set (`acks`), the write offset, the -unacked length, and `is_fully_acked`; the contiguous acknowledged prefix -follows from them. `Connection` also -computes the peer's `ack_delay` per ACK frame when it updates the RTT -estimator. Expose that state through the public `SendStream` handle without -copying it: - -- an accessor for the contiguous acknowledged prefix of the stream; -- an event or waker registration that fires when that prefix crosses an - offset the caller names, so a caller can await "bytes below X are - acknowledged" without polling; -- the receive instant of the ACK that advanced the prefix, minus the peer's - reported ACK delay. That subtraction removes the delayed-ACK timer (up to - `max_ack_delay`, 25 ms by default) so a delivery-latency sample sees the - network path and not the receiver's batching. State clearly that the - instant still includes the return one-way delay. - -Match Quinn semantics for partial ACKs, retransmission of a range that was -already partly acknowledged, a stream that is reset by either side, and -stream teardown: the accessor must not return a prefix that includes bytes the -peer will never acknowledge, and a waiter for an offset beyond the final -size must resolve with the reset instead of hanging. - -Decided 2026-10-16 (maintainer, #5690): the waiter is `Checkpoint::acked()` -on the send-stream checkpoint that `RESET_STREAM_AT` already uses, so an -acknowledged offset is named the same way a reliable reset's is. - -The `web-transport-moq` half of [the ACK hook](/quest/m2/quic-ack-hook.md) -is in-tree too and can land in the same PR. - -Decided in the 2026-09-30 audit: moved to m2 with its consumers, the ACK -hook and [frame-granularity starvation](/quest/m2/starvation-frames.md). - -## Related - -- [poll_acked on moq-net's send stream](/quest/m2/quic-ack-hook.md) - the first - consumer of the accessor -- [noq#808](https://github.com/n0-computer/noq/issues/808) - the acked-offset ask to n0 diff --git a/quest/m2/starvation-frames.md b/quest/m2/starvation-frames.md index 4923e8c7794..4760ae8f2ab 100644 --- a/quest/m2/starvation-frames.md +++ b/quest/m2/starvation-frames.md @@ -31,10 +31,11 @@ span from the newest acknowledged frame to the newest written one, which is the exact media the viewer lost. Add a second byte-weighted cumulative histogram of delivery delay: -`acked.received - written_at` per frame, where `received` is the -ACK-delay-corrected instant the hook returns. Its low edge approaches the -path RTT and its spread is the jitter. Document that the value still includes -the return one-way delay and that a backend without the hook leaves the +`acked_at - written_at` per frame, where `acked_at` is when the serve task is +woken by `poll_acked`. Its low edge approaches the path RTT and its spread is +the jitter. Document that the value still includes the return one-way delay +and the peer's delayed-ACK timer, since the hook reports no corrected instant +(decided 2026-10-17 on #5817), and that a backend without the hook leaves the histogram absent, never zero. Backends that return unsupported from `poll_acked` keep the group-granularity @@ -52,8 +53,8 @@ is Required. Tests: the frontier tracking frame ends under a peer that acknowledges in bursts, with interval samples landing one bucket lower than group-granularity tracking of the same run; a reset mid-group attributing only the -unacknowledged span to `dropped_duration`; delivery delay under an injected ACK delay staying inside one bucket of the -true RTT; and fallback to group-granularity frontier tracking when the hook is +unacknowledged span to `dropped_duration`; delivery delay staying inside one bucket of the true RTT plus the peer's +`max_ack_delay`; and fallback to group-granularity frontier tracking when the hook is unsupported. ## Required diff --git a/rs/moq-quic/MIGRATE.md b/rs/moq-quic/MIGRATE.md index 0b5ffd1aeff..fd3da11282d 100644 --- a/rs/moq-quic/MIGRATE.md +++ b/rs/moq-quic/MIGRATE.md @@ -6,6 +6,16 @@ How to update your code for each breaking change, newest first. Each entry is a change from `quinn-proto`. +### `StreamEvent::Acked` (#5817) + +`StreamEvent` gained `Acked { id, offset }`, reported when the answer changes for the lowest offset a `SendStream::acked` call is waiting for: the stream's acknowledged prefix reached it, or `STOP_SENDING` or a reset made it fail. +An exhaustive `match` needs an arm for it. + +```rust +// After +StreamEvent::Acked { id, offset } => wake_ack_waiters(id, offset), +``` + ### Streams and datagrams take `SendOptions` (#5770, #5872) `Streams::open` and `Datagrams::send` take a `SendOptions { group, priority }`, diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index ed3886470f2..ba9ec36b6d0 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -70,7 +70,8 @@ Changes on top of the upstream commit, besides the renames: - **Send groups** (`streams/schedule.rs`, `Connection::send_groups`): streams outside every group go first, in quinn's order; groups are served strictly by priority, with equal priorities sharing bytes by deficit round robin. A stream's priority is an `i64` ordering it only within its group, and `Streams::open` takes a `SendOptions { group, priority }` to start it there, like W3C WebTransport's send options. The schedule also has scheduling domains that share bytes the same way, crate-private until a consumer needs them. This replaces quinn's single pending-stream heap, so a cherry-pick touching `PendingStreamsQueue` has to be redone against the schedule. moq-tokio's `quic::driver` and moq-uring's driver wrap a group in a shared `SendGroup` handle that closes on drop. - **Scheduled datagrams** (`Datagrams::send`): a datagram is queued by its `SendOptions`, in a send group (or ungrouped) at an `i64` priority, and scheduled with the group's streams, winning a tie with a stream of equal priority and spending the group's byte credit, instead of quinn's datagram-first FIFO in `populate_packet`. Sending never blocks: `datagram_send_buffer_size` is shared, and an overflow evicts the oldest datagram of the lowest-priority group in the domain holding the most datagram bytes. `SendDatagramError::Blocked` and `Event::DatagramsUnblocked` are gone, and a cherry-pick touching the `// DATAGRAM` loop has to be redone against `StreamsState::write_frames`. - **Sharded connection IDs** (`RandomConnectionIdGenerator::set_shard`): every CID names shard `index` of a `count`-endpoint group in its first byte, modulo `count`, and sets a marker in its fourth byte, so a `SO_REUSEPORT` steering filter delivers a packet to the endpoint that issued its CID. `validate` refuses a CID without this shard's prefix and marker, so a misrouted packet is dropped instead of answered with a stateless reset. Not offered upstream; it serves moq-sock's per-core sharding. -- **`RESET_STREAM_AT`** ([draft-ietf-quic-reliable-stream-reset](https://datatracker.ietf.org/doc/draft-ietf-quic-reliable-stream-reset/)): the `reset_stream_at` transport parameter is always advertised, `Checkpoint::reset` (from `SendStream::checkpoint`) keeps the data written before the checkpoint reliable, holding a plain `RESET_STREAM` until that data is acknowledged when the peer lacks the extension, and a received reset surfaces only once the reliable prefix has been read. Stopping a stream after it was reset no longer credits its unread data twice. +- **`RESET_STREAM_AT`** ([draft-ietf-quic-reliable-stream-reset](https://datatracker.ietf.org/doc/draft-ietf-quic-reliable-stream-reset/)): the `reset_stream_at` transport parameter is always advertised, `SendStream::reset_at` keeps the data written before an offset reliable, holding a plain `RESET_STREAM` until that data is acknowledged when the peer lacks the extension, and a received reset surfaces only once the reliable prefix has been read. Stopping a stream after it was reset no longer credits its unread data twice. +- **ACK progress** (`SendStream::offset`, `SendStream::acked`, `StreamEvent::Acked`): `acked` reports whether the peer acknowledged every byte before an offset, by comparing against the stream's contiguous acknowledged prefix. A `false` stores the lowest polled offset on the stream, and the ACK that advances the prefix past it, or a reset or `STOP_SENDING` that makes it fail, reports `StreamEvent::Acked` with the prefix, so a cherry-pick touching `StreamsState::received_ack_of`, `received_stop_sending`, or the reset path has to keep that report. Not offered upstream: quinn has no API for when sent data was acknowledged. - **Fuzzing** runs through the hidden `fuzz` module behind a `fuzz` feature, with one-line shims in [`fuzz/`](fuzz/README.md): upstream's `packet`, `params`, `streamid`, and `streams` targets, plus `frame` and a connection op sequence. Upstream's `cfg(fuzzing)` `fuzzing` module and `arbitrary` feature are gone, so its fuzz patches port by hand. - **Draining endpoint event**: `EndpointEvent::is_draining` fires once per connection when it stops being active, before the drained event, so the driver's `Endpoint::wait_all_draining` can end a shutdown without sitting out every closing period. - **Late stream limits**: peer transport parameters that raise a stream limit after an open was refused emit `StreamEvent::Available`, as `MAX_STREAMS` does. A server accepted before the rest of a multi-packet ClientHello (a post-quantum key share is enough) otherwise leaves a 0.5-RTT `open_uni` parked until the idle timeout. Not offered upstream yet. diff --git a/rs/moq-quic/fuzz/README.md b/rs/moq-quic/fuzz/README.md index 0b5e608809a..a24c0489b21 100644 --- a/rs/moq-quic/fuzz/README.md +++ b/rs/moq-quic/fuzz/README.md @@ -48,7 +48,7 @@ symbol stripping. Prefix direct calls with - A stream ID keeps the initiator, direction, and index it was built from. - ACK ranges walk downward without overlapping. - `connection` handshakes a client and server over TLS, then applies opens, accepts, - writes, finishes, resets (plain and from a checkpoint), stops, reads that hold or + writes, finishes, resets (plain and at an offset), acknowledgment queries, stops, reads that hold or release credit, runtime limit and window changes, datagrams, key updates, and time jumps, while dropping, duplicating, reordering, corrupting, and injecting datagrams. Every stream read is in order, never past what was written, byte-for-byte what was diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index 4e55a1179cb..01695cdbaf8 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -93,9 +93,8 @@ pub use stats::{ConnectionStats, CreditStats, FrameStats, PathStats, UdpStats}; mod streams; pub(crate) use streams::StreamsState; pub use streams::{ - Checkpoint, Chunks, ClosedStream, Credit, FinishError, ReadError, ReadableError, RecvStream, - SendGroup, SendGroups, SendOptions, SendStream, ShouldTransmit, StreamEvent, Streams, - WriteError, Written, + Chunks, ClosedStream, Credit, FinishError, ReadError, ReadableError, RecvStream, SendGroup, + SendGroups, SendOptions, SendStream, ShouldTransmit, StreamEvent, Streams, WriteError, Written, }; mod timer; diff --git a/rs/moq-quic/src/connection/send_buffer.rs b/rs/moq-quic/src/connection/send_buffer.rs index 97efcde5c20..9828a3c4a84 100644 --- a/rs/moq-quic/src/connection/send_buffer.rs +++ b/rs/moq-quic/src/connection/send_buffer.rs @@ -219,6 +219,11 @@ impl SendBuffer { self.offset } + /// The end of the contiguous acknowledged prefix + pub(super) fn acked(&self) -> u64 { + self.offset - self.unacked_len as u64 + } + /// Whether all sent data has been acknowledged pub(super) fn is_fully_acked(&self) -> bool { self.unacked_len == 0 diff --git a/rs/moq-quic/src/connection/streams/mod.rs b/rs/moq-quic/src/connection/streams/mod.rs index d97adbaae22..9143c0946b5 100644 --- a/rs/moq-quic/src/connection/streams/mod.rs +++ b/rs/moq-quic/src/connection/streams/mod.rs @@ -369,29 +369,68 @@ impl<'a> SendStream<'a> { /// # Panics /// - when applied to a receive stream pub fn reset(&mut self, error_code: VarInt) -> Result<(), ClosedStream> { - self.reset_to(error_code, 0) + self.reset_at(0, error_code) } - /// Mark the data written so far, so a later [`Checkpoint::reset`] still delivers it + /// How many bytes were written to the stream so far, the offset the next write starts at + /// + /// After a reset, the bytes the reset still delivers. Zero before the first write, or once + /// the stream is gone. /// /// # Panics /// - when applied to a receive stream - pub fn checkpoint(&self) -> Checkpoint { - let offset = match self.state.send.get(&self.id) { + pub fn offset(&self) -> u64 { + match self.state.send.get(&self.id) { Some(Some(stream)) => stream.offset(), - // Not written yet, or already closed, where the reset fails anyway _ => 0, - }; - Checkpoint { - id: self.id, - tag: self.state.tag, - generation: self.state.generation, - offset, } } - /// Reset the stream, still delivering its first `reliable_size` bytes - fn reset_to(&mut self, error_code: VarInt, reliable_size: u64) -> Result<(), ClosedStream> { + /// Whether the peer acknowledged every byte before `offset` + /// + /// A `false` notes to report a [`StreamEvent::Acked`] once the answer changes, so poll again + /// then. The stream holds only its lowest waiting offset, so every waiter on it polls again + /// on that event. Like a [`StreamEvent::Finished`] stream, an acknowledgment only means the + /// peer's QUIC stack holds the data, not that its application read it. + /// + /// Fails with [`ClosedStream`] once the peer will never acknowledge it, or no longer reads + /// it: a reset dropped the data before `offset`, the peer sent STOP_SENDING, or the + /// connection closed. It also fails once the stream is freed, after its + /// [`StreamEvent::Finished`], which means every byte was acknowledged. + /// + /// # Panics + /// - when applied to a receive stream + /// - when `offset` is past the data written to a stream that was not reset + pub fn acked(&mut self, offset: u64) -> Result { + if self.conn_state.is_closed() { + return Err(ClosedStream { _private: () }); + } + match self.state.send.get_mut(&self.id) { + Some(Some(stream)) => stream.acked(offset), + Some(None) => { + assert_eq!(offset, 0, "acked past the data written"); + Ok(true) + } + None => Err(ClosedStream { _private: () }), + } + } + + /// Abandon the stream, but still deliver its first `offset` bytes + /// + /// Nothing past `offset` is sent or retransmitted. A peer that accepts RESET_STREAM_AT gets + /// it at once, and reads the data before the error. Otherwise the data keeps being + /// retransmitted, and a plain RESET_STREAM follows once the peer has acknowledged all of it, a + /// round trip later. An acknowledgment only means the peer's QUIC stack holds the data: + /// RFC 9000 lets it discard data its application has not read yet on RESET_STREAM, so this + /// only reliably delivers data the peer parses on arrival, like a WebTransport stream header. + /// A peer that asked us to stop sending gets the reset at once, without the data. + /// + /// Fails with [`ClosedStream`] if the stream was already reset or is gone. + /// + /// # Panics + /// - when applied to a receive stream + /// - when `offset` is past the data written + pub fn reset_at(&mut self, offset: u64, error_code: VarInt) -> Result<(), ClosedStream> { let max_send_data = self.state.max_send_data(self.id); let at = self.state.peer_reset_stream_at; let stream = self @@ -405,12 +444,11 @@ impl<'a> SendStream<'a> { // Redundant reset call return Err(ClosedStream { _private: () }); } - // A checkpoint names its stream's connection and generation, so it never runs past the data - debug_assert!(reliable_size <= stream.offset()); + assert!(offset <= stream.offset(), "reset past the data written"); // The peer that sent STOP_SENDING will not read the prefix let reliable_size = match stream.stop_reason { Some(_) => 0, - None => reliable_size, + None => offset, }; // Restore the portion of the send window consumed by the data that we aren't about to @@ -419,6 +457,12 @@ impl<'a> SendStream<'a> { let buffered = stream.pending.buffered(); let send = stream.reset(error_code, reliable_size, at); self.state.buffered_data -= buffered - stream.pending.buffered(); + if let Some(offset) = stream.take_waiting() { + self.state.events.push_back(StreamEvent::Acked { + id: self.id, + offset, + }); + } // Only data below the Reliable Size is left to send if !stream.is_pending() { self.state.pending.remove(&mut stream.slot); @@ -490,51 +534,6 @@ impl<'a> SendStream<'a> { } } -/// The data written to a send stream up to some point, which a reset can still deliver -/// -/// Taken with [`SendStream::checkpoint`], and only valid on the connection it came from. -#[derive(Debug)] -pub struct Checkpoint { - id: StreamId, - tag: ConnectionTag, - generation: u64, - offset: u64, -} - -impl Checkpoint { - /// Abandon the stream, but still deliver everything written before the checkpoint - /// - /// Nothing past the checkpoint is sent or retransmitted. A peer that accepts RESET_STREAM_AT - /// gets it at once, and reads the data before the error. Otherwise the data keeps being - /// retransmitted, and a plain RESET_STREAM follows once the peer has acknowledged all of it, a - /// round trip later. An acknowledgment only means the peer's QUIC stack holds the data: - /// RFC 9000 lets it discard data its application has not read yet on RESET_STREAM, so this - /// only reliably delivers data the peer parses on arrival, like a WebTransport stream header. - /// A peer that asked us to stop sending gets the reset at once, without the data. - /// - /// Fails with [`ClosedStream`] if the stream was already reset or is gone, including one - /// rejected 0-RTT discarded, even if a new stream has since taken its id. - /// - /// # Panics - /// If the checkpoint came from another connection. - pub fn reset( - self, - conn: &mut crate::Connection, - error_code: VarInt, - ) -> Result<(), ClosedStream> { - let mut stream = conn.send_stream(self.id); - assert!( - self.tag == stream.state.tag, - "checkpoint used on another connection" - ); - // Rejected 0-RTT discarded the stream, and a new one may have its id - if self.generation != stream.state.generation { - return Err(ClosedStream { _private: () }); - } - stream.reset_to(error_code, self.offset) - } -} - /// Scheduling of outgoing streams between send groups /// /// A higher priority group preempts every lower one, and groups of equal priority get an @@ -751,6 +750,18 @@ pub enum StreamEvent { /// Which stream has been finished id: StreamId, }, + /// The answer changed for the lowest offset a [`SendStream::acked`] call is waiting for + /// + /// The stream's contiguous acknowledged prefix reached it, or a reset or STOP_SENDING may have + /// failed any waiter. Every waiter on the stream polls again: one at or below `offset` is + /// acknowledged, and one past it learns its error or registers again. Reported before a [`StreamEvent::Finished`] from the same ACK. + /// Not reported when the connection closes, which fails [`SendStream::acked`] instead. + Acked { + /// Which stream was acknowledged + id: StreamId, + /// The end of the stream's contiguous acknowledged prefix + offset: u64, + }, /// The peer asked us to stop sending on an outgoing stream Stopped { /// Which stream has been stopped diff --git a/rs/moq-quic/src/connection/streams/saturated.rs b/rs/moq-quic/src/connection/streams/saturated.rs index 140d3f916a1..6e1bbb30df5 100644 --- a/rs/moq-quic/src/connection/streams/saturated.rs +++ b/rs/moq-quic/src/connection/streams/saturated.rs @@ -523,7 +523,7 @@ fn reliable_reset_sends_its_prefix_then_frees_its_group() { sender.send(finished).finish().unwrap(); sender.groups().close(group); - sender.send(reset).reset_to(0u32.into(), 3_000).unwrap(); + sender.send(reset).reset_at(3_000, 0u32.into()).unwrap(); assert!(sender.state.can_send_stream_data()); sender.drain(100); assert_eq!(sender.bytes(&[reset]), 3_000, "only the reliable prefix"); @@ -555,7 +555,7 @@ fn stop_sending_drops_a_reliable_prefix_from_the_schedule() { let mut sender = Sender::new(); let group = sender.group(0); let reset = sender.open(Some(&group), 0, 10_000); - sender.send(reset).reset_to(0u32.into(), 3_000).unwrap(); + sender.send(reset).reset_at(3_000, 0u32.into()).unwrap(); assert!(sender.state.can_send_stream_data()); assert_eq!( diff --git a/rs/moq-quic/src/connection/streams/send.rs b/rs/moq-quic/src/connection/streams/send.rs index 0b59adabd1b..58ab5c77204 100644 --- a/rs/moq-quic/src/connection/streams/send.rs +++ b/rs/moq-quic/src/connection/streams/send.rs @@ -1,7 +1,7 @@ use bytes::Bytes; use thiserror::Error; -use super::schedule::Slot; +use super::{ClosedStream, schedule::Slot}; use crate::{StreamId, VarInt, connection::send_buffer::SendBuffer, frame}; #[derive(Debug)] @@ -21,6 +21,8 @@ pub(super) struct Send { pub(super) data_blocked_limit: Option, /// The reason the peer wants us to stop, if `STOP_SENDING` was received pub(super) stop_reason: Option, + /// The lowest offset a [`SendStream::acked`](super::SendStream::acked) call is waiting for + pub(super) notify: Option, } impl Send { @@ -35,6 +37,7 @@ impl Send { connection_blocked: false, data_blocked_limit: None, stop_reason: None, + notify: None, }) } @@ -243,6 +246,41 @@ impl Send { } } + /// Whether the peer acknowledged everything before `offset`, noting to report it if not + pub(super) fn acked(&mut self, offset: u64) -> Result { + // A reset cuts the buffer short, so only an unreset stream knows how much was written + assert!( + self.is_reset() || offset <= self.pending.offset(), + "acked past the data written" + ); + // An acknowledgment no longer means the peer reads the data, and a reset dropped the rest + if self.stop_reason.is_some() || offset > self.pending.offset() { + return Err(ClosedStream::default()); + } + if offset <= self.pending.acked() { + return Ok(true); + } + self.notify = Some(self.notify.map_or(offset, |notify| notify.min(offset))); + Ok(false) + } + + /// The acknowledged prefix to report once it reaches the offset a waiter is waiting for + pub(super) fn take_acked(&mut self) -> Option { + let acked = self.pending.acked(); + if self.notify.is_none_or(|notify| acked < notify) { + return None; + } + self.notify = None; + Some(acked) + } + + /// The acknowledged prefix to report to every waiter after a reset or STOP_SENDING, which may + /// fail any of them + pub(super) fn take_waiting(&mut self) -> Option { + self.notify.take()?; + Some(self.pending.acked()) + } + /// Handle increase to stream-level flow control limit /// /// Returns whether the stream was unblocked diff --git a/rs/moq-quic/src/connection/streams/state.rs b/rs/moq-quic/src/connection/streams/state.rs index e3126bd235c..d0e8bddbd41 100644 --- a/rs/moq-quic/src/connection/streams/state.rs +++ b/rs/moq-quic/src/connection/streams/state.rs @@ -113,7 +113,7 @@ pub(crate) struct StreamsState { /// Streams with outgoing data queued and outgoing datagrams, in the order to send them pub(super) pending: Schedule, - events: VecDeque, + pub(super) events: VecDeque, /// Streams blocked on connection-level flow control or stream window space /// /// Streams are only added to this list when a write fails. @@ -151,9 +151,6 @@ pub(crate) struct StreamsState { initial_max_stream_data_bidi_remote: VarInt, /// Whether the peer accepts RESET_STREAM_AT pub(super) peer_reset_stream_at: bool, - /// Bumped when rejected 0-RTT discards our streams and their ids are opened again, so a - /// [`Checkpoint`](super::Checkpoint) from before cannot reach the new stream of the same id - pub(super) generation: u64, /// The shrink to be applied to local_max_data when receive_window is shrunk receive_window_shrink_debt: u64, @@ -210,7 +207,6 @@ impl StreamsState { initial_max_stream_data_bidi_local: 0u32.into(), initial_max_stream_data_bidi_remote: 0u32.into(), peer_reset_stream_at: false, - generation: 0, receive_window_shrink_debt: 0, streams_blocked: [false, false], open_refused: [false, false], @@ -278,7 +274,6 @@ impl StreamsState { } } - self.generation += 1; self.send_streams = 0; self.data_sent = 0; self.buffered_data = 0; @@ -425,6 +420,9 @@ impl StreamsState { let buffered = stream.pending.buffered(); let reset = stream.abandon_reliable(); self.buffered_data -= buffered - stream.pending.buffered(); + if let Some(offset) = stream.take_waiting() { + self.events.push_back(StreamEvent::Acked { id, offset }); + } if !stream.is_pending() { self.pending.remove(&mut stream.slot); } @@ -772,6 +770,9 @@ impl StreamsState { let buffered = stream.pending.buffered(); let finished = stream.ack(frame); self.buffered_data -= buffered - stream.pending.buffered(); + if let Some(offset) = stream.take_acked() { + self.events.push_back(StreamEvent::Acked { id, offset }); + } if !finished { // The stream is unfinished or may still need retransmits return stream.take_held_reset(); @@ -2918,7 +2919,7 @@ mod tests { stream.write(b"hello world").unwrap(); let mut buf = Vec::new(); let sent = stream.state.write_frames(&mut buf, 0..100, true).streams; - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); assert!(pending.reset_stream.is_empty()); assert_eq!(server.buffered_data, 4); @@ -2960,7 +2961,7 @@ mod tests { conn_state: &state, }; stream.write(b"hello world").unwrap(); - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); assert_eq!(pending.reset_stream, &[(id, 4)]); assert_eq!(server.received_stop_sending(id, 9u32.into()), Some(0)); @@ -2991,7 +2992,7 @@ mod tests { stream.write(b"hello world").unwrap(); let mut buf = Vec::new(); let sent = stream.state.write_frames(&mut buf, 0..100, true).streams; - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); server.reset_acked(id, 4); assert_eq!(server.received_stop_sending(id, 9u32.into()), Some(0)); @@ -3032,7 +3033,7 @@ mod tests { conn_state: &state, }; stream.write(b"hello world").unwrap(); - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); assert_eq!(pending.reset_stream.is_empty(), !reset_stream_at); assert_eq!(server.received_stop_sending(id, 9u32.into()), Some(0)); @@ -3068,7 +3069,7 @@ mod tests { let mut buf = Vec::new(); let sent = stream.state.write_frames(&mut buf, 0..100, true).streams; assert_eq!(sent[0].offsets, 0..11); - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); assert_eq!(server.buffered_data, 4); // Only the lost prefix is retransmitted, without the FIN of a stream that was never finished @@ -3107,7 +3108,7 @@ mod tests { }; stream.write(b"hello world").unwrap(); stream.state.received_stop_sending(id, 9u32.into()); - stream.reset_to(7u32.into(), 4).unwrap(); + stream.reset_at(4, 7u32.into()).unwrap(); let send = server.send[&id].as_ref().unwrap(); let frame = send.reset_frame(id, true).unwrap(); diff --git a/rs/moq-quic/src/fuzz/connection.rs b/rs/moq-quic/src/fuzz/connection.rs index 3c32dc34e0d..8e650004e5c 100644 --- a/rs/moq-quic/src/fuzz/connection.rs +++ b/rs/moq-quic/src/fuzz/connection.rs @@ -69,7 +69,7 @@ enum Who { Server, } -/// One step of a run. Stream and checkpoint indexes wrap around what that side has. +/// One step of a run. Stream indexes wrap around what that side has. #[derive(Arbitrary, Debug)] enum Op { Open { @@ -94,14 +94,18 @@ enum Op { stream: u8, code: u32, }, - Checkpoint { + /// Reset a stream, still delivering a prefix of what was written + ResetAt { who: Who, stream: u8, + offset: u16, + code: u32, }, - ResetCheckpoint { + /// Ask whether a prefix of what was written is acknowledged + Acked { who: Who, - checkpoint: u8, - code: u32, + stream: u8, + offset: u16, }, Priority { who: Who, @@ -198,7 +202,6 @@ struct Peer { recv: Vec, /// How far each stream has been read, in order. read: HashMap, - checkpoints: Vec, held: Vec, /// The length of each datagram this side sent, indexed by the sequence number it carries. datagrams_sent: Vec, @@ -223,7 +226,6 @@ impl Peer { send: Vec::new(), recv: Vec::new(), read: HashMap::new(), - checkpoints: Vec::new(), held: Vec::new(), datagrams_sent: Vec::new(), datagrams_received: HashSet::new(), @@ -481,26 +483,27 @@ impl Pair { let _ = conn.send_stream(id).reset(code.into()); } } - Op::Checkpoint { who, stream } => { - let peer = self.peer(who); - if let Some((id, conn)) = send_stream(peer, stream) { - let checkpoint = conn.send_stream(id).checkpoint(); - peer.checkpoints.push(checkpoint); - } - } - Op::ResetCheckpoint { + Op::ResetAt { who, - checkpoint, + stream, + offset, code, } => { - let peer = self.peer(who); - if peer.checkpoints.is_empty() { - return; + if let Some((id, conn)) = send_stream(self.peer(who), stream) { + let mut stream = conn.send_stream(id); + let offset = u64::from(offset) % (stream.offset() + 1); + let _ = stream.reset_at(offset, code.into()); } - let index = checkpoint as usize % peer.checkpoints.len(); - let checkpoint = peer.checkpoints.swap_remove(index); - if let Some(conn) = peer.conn() { - let _ = checkpoint.reset(conn, code.into()); + } + Op::Acked { + who, + stream, + offset, + } => { + if let Some((id, conn)) = send_stream(self.peer(who), stream) { + let mut stream = conn.send_stream(id); + let offset = u64::from(offset) % (stream.offset() + 1); + let _ = stream.acked(offset); } } Op::Priority { diff --git a/rs/moq-quic/src/lib.rs b/rs/moq-quic/src/lib.rs index b6e978a01bb..7927e9b9fd4 100644 --- a/rs/moq-quic/src/lib.rs +++ b/rs/moq-quic/src/lib.rs @@ -53,9 +53,9 @@ pub use bloom_token_log::BloomTokenLog; mod connection; pub use crate::connection::{ - Checkpoint, Chunk, Chunks, ClosedStream, Connection, ConnectionError, ConnectionStats, Credit, - CreditStats, Datagrams, Event, FinishError, FrameStats, PathStats, ReadError, ReadableError, - RecvStream, RttEstimator, SendDatagramError, SendGroup, SendGroups, SendOptions, SendStream, + Chunk, Chunks, ClosedStream, Connection, ConnectionError, ConnectionStats, Credit, CreditStats, + Datagrams, Event, FinishError, FrameStats, PathStats, ReadError, ReadableError, RecvStream, + RttEstimator, SendDatagramError, SendGroup, SendGroups, SendOptions, SendStream, ShouldTransmit, StreamEvent, Streams, UdpStats, WriteError, Written, }; #[cfg(feature = "qlog")] diff --git a/rs/moq-quic/src/tests/acked.rs b/rs/moq-quic/src/tests/acked.rs new file mode 100644 index 00000000000..0b03c6a4e85 --- /dev/null +++ b/rs/moq-quic/src/tests/acked.rs @@ -0,0 +1,342 @@ +//! Tests for [`SendStream::acked`], which reports whether the peer acknowledged a send stream's prefix + +use super::*; + +const LATENCY: Duration = Duration::from_millis(10); + +/// A pair with a one-way delay, whose server does or does not advertise `reset_stream_at` +fn pair(reset_stream_at: bool) -> Pair { + let mut config = server_config(); + let mut transport = cubic_transport(); + transport.reset_stream_at(reset_stream_at); + config.transport = Arc::new(transport); + let mut pair = Pair::new(Default::default(), config); + pair.latency = LATENCY; + pair +} + +/// Open a client stream that the server has accepted, with nothing acknowledged yet +fn open(pair: &mut Pair, client_ch: ConnectionHandle, server_ch: ConnectionHandle) -> StreamId { + let s = pair + .client_streams(client_ch) + .open(Dir::Uni, Default::default()) + .unwrap(); + pair.client_send(client_ch, s).write(b"header").unwrap(); + pair.drive(); + assert_eq!(pair.server_streams(server_ch).accept(Dir::Uni), Some(s)); + s +} + +fn acked( + pair: &mut Pair, + ch: ConnectionHandle, + s: StreamId, + offset: u64, +) -> Result { + pair.client_send(ch, s).acked(offset) +} + +/// The prefix the client last reported acknowledged on `s`, draining its events +fn reported(pair: &mut Pair, ch: ConnectionHandle, s: StreamId) -> Option { + let mut reported = None; + while let Some(event) = pair.client_conn_mut(ch).poll() { + if let Event::Stream(StreamEvent::Acked { id, offset }) = event + && id == s + { + reported = Some(offset); + } + } + reported +} + +/// Transmit the client's pending packets and drop them +fn lose_client_packets(pair: &mut Pair) { + pair.client.drive(pair.time, pair.server.addr); + assert!(!pair.client.outbound.is_empty()); + pair.client.outbound.clear(); +} + +/// An offset is acknowledged only once every byte before it is, and the event that reports it +/// fires only for an offset someone polled. +#[test] +fn waits_for_contiguous_prefix() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + pair.drive(); + reported(&mut pair, client_ch, s); + + pair.client_send(client_ch, s).write(b"lost").unwrap(); + let lost = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + + // Acknowledged, but behind the lost packet: the middle of the stream, not its prefix + pair.client_send(client_ch, s).write(b"after").unwrap(); + let after = pair.client_send(client_ch, s).offset(); + let acks = pair.client_conn_mut(client_ch).stats().frame_rx.acks; + while pair.client_conn_mut(client_ch).stats().frame_rx.acks == acks { + assert!(pair.step(), "never acknowledged"); + } + assert_eq!(acked(&mut pair, client_ch, s, after), Ok(false)); + + // The retransmission fills the gap, which acknowledges both at once + pair.drive(); + assert_eq!( + reported(&mut pair, client_ch, s), + Some(b"headerlostafter".len() as u64) + ); + assert_eq!(acked(&mut pair, client_ch, s, lost), Ok(true)); + assert_eq!(acked(&mut pair, client_ch, s, after), Ok(true)); + assert!(pair.client_conn_mut(client_ch).stats().path.lost_packets > 0); +} + +/// A waiter past the offset an event reports polls again and registers anew, so it learns of +/// a later ACK too. +#[test] +fn later_waiter_registers_again() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + pair.drive(); + reported(&mut pair, client_ch, s); + + // The first packet is held, and the second lost + pair.client_send(client_ch, s).write(b"first").unwrap(); + let first = pair.client_send(client_ch, s).offset(); + pair.client.drive(pair.time, pair.server.addr); + pair.client.delay_outbound(); + pair.client_send(client_ch, s).write(b"second").unwrap(); + let second = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + assert_eq!(acked(&mut pair, client_ch, s, second), Ok(false)); + assert_eq!(acked(&mut pair, client_ch, s, first), Ok(false)); + + // Only the first arrives at first + pair.client.finish_delay(); + while acked(&mut pair, client_ch, s, first) != Ok(true) { + assert!(pair.step(), "never acknowledged"); + } + assert_eq!(reported(&mut pair, client_ch, s), Some(first)); + assert_eq!(acked(&mut pair, client_ch, s, second), Ok(false)); + + // The retransmission reports the second + pair.drive(); + assert_eq!(reported(&mut pair, client_ch, s), Some(second)); +} + +/// Without a poll waiting on it, an acknowledgment is not reported. +#[test] +fn unpolled_is_not_reported() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + + pair.client_send(client_ch, s).write(b"payload").unwrap(); + let offset = pair.client_send(client_ch, s).offset(); + pair.drive(); + assert_eq!(reported(&mut pair, client_ch, s), None); + assert_eq!(acked(&mut pair, client_ch, s, offset), Ok(true)); +} + +/// An offset taken before anything was written, or once everything was acknowledged, is +/// acknowledged at once. +#[test] +fn already_acked() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, _) = pair.connect(); + let s = pair + .client_streams(client_ch) + .open(Dir::Uni, Default::default()) + .unwrap(); + let empty = pair.client_send(client_ch, s).offset(); + assert_eq!(acked(&mut pair, client_ch, s, empty), Ok(true)); + + pair.client_send(client_ch, s).write(b"payload").unwrap(); + pair.drive(); + let all = pair.client_send(client_ch, s).offset(); + assert_eq!(acked(&mut pair, client_ch, s, all), Ok(true)); +} + +/// A finished stream is freed by the ACK that completes it, which reports a waiting offset +/// before the stream finishes. +#[test] +fn finished_stream_reports_before_freed() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + pair.drive(); + reported(&mut pair, client_ch, s); + + pair.client_send(client_ch, s).write(b"payload").unwrap(); + let offset = pair.client_send(client_ch, s).offset(); + assert_eq!(acked(&mut pair, client_ch, s, offset), Ok(false)); + pair.client_send(client_ch, s).finish().unwrap(); + pair.drive(); + + assert_eq!(pair.client_streams(client_ch).send_streams(), 0); + let mut events = Vec::new(); + while let Some(Event::Stream(event)) = pair.client_conn_mut(client_ch).poll() { + events.push(event); + } + let len = b"headerpayload".len() as u64; + assert_eq!( + events, + [ + StreamEvent::Acked { id: s, offset: len }, + StreamEvent::Finished { id: s } + ] + ); + assert_eq!( + acked(&mut pair, client_ch, s, offset), + Err(ClosedStream::default()) + ); +} + +/// A reset fails the offsets past its Reliable Size at once, while those before it are +/// still acknowledged once the prefix arrives. +#[test] +fn reset_fails_past_reliable_size() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + + pair.client_send(client_ch, s).write(b"kept").unwrap(); + let kept = pair.client_send(client_ch, s).offset(); + let reset = pair.client_send(client_ch, s).offset(); + pair.client_send(client_ch, s).write(b"dropped").unwrap(); + let dropped = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + reported(&mut pair, client_ch, s); + + pair.client_send(client_ch, s) + .reset_at(reset, VarInt(42)) + .unwrap(); + assert_eq!( + acked(&mut pair, client_ch, s, dropped), + Err(ClosedStream::default()) + ); + assert_eq!(acked(&mut pair, client_ch, s, kept), Ok(false)); + assert_eq!(reported(&mut pair, client_ch, s), None); + + // The retransmitted prefix is acknowledged before the reset frees the stream + pair.drive(); + assert_eq!(pair.client_streams(client_ch).send_streams(), 0); + assert_eq!( + reported(&mut pair, client_ch, s), + Some(b"headerkept".len() as u64) + ); +} + +/// A waiter whose offset a later reset drops learns so from the event, even beside a lower +/// waiter the reset keeps. +#[test] +fn reset_reports_a_dropped_waiter() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + + // A lower waiter the reset keeps doesn't hide the one it drops + pair.client_send(client_ch, s).write(b"kept").unwrap(); + let kept = pair.client_send(client_ch, s).offset(); + pair.client_send(client_ch, s).write(b"dropped").unwrap(); + let dropped = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + reported(&mut pair, client_ch, s); + assert_eq!(acked(&mut pair, client_ch, s, kept), Ok(false)); + assert_eq!(acked(&mut pair, client_ch, s, dropped), Ok(false)); + + pair.client_send(client_ch, s) + .reset_at(kept, VarInt(42)) + .unwrap(); + assert_eq!(pair.client_send(client_ch, s).offset(), kept); + assert!(reported(&mut pair, client_ch, s).is_some()); + assert_eq!( + acked(&mut pair, client_ch, s, dropped), + Err(ClosedStream::default()) + ); +} + +/// A peer that stopped reading will not read what it acknowledges, before or after the +/// offset was taken, and a waiter learns so from the event. +#[test] +fn stop_sending_fails() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + + pair.client_send(client_ch, s).write(b"unread").unwrap(); + let before = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + assert_eq!(acked(&mut pair, client_ch, s, before), Ok(false)); + + pair.server_recv(server_ch, s).stop(VarInt(7)).unwrap(); + pair.drive(); + assert_eq!( + pair.client_send(client_ch, s).stopped(), + Ok(Some(VarInt(7))) + ); + let after = pair.client_send(client_ch, s).offset(); + assert_eq!( + reported(&mut pair, client_ch, s), + Some(b"header".len() as u64) + ); + for offset in [before, after] { + assert_eq!( + acked(&mut pair, client_ch, s, offset), + Err(ClosedStream::default()) + ); + } +} + +/// Nothing is acknowledged on a closed connection. +#[test] +fn close_fails() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + + pair.client_send(client_ch, s).write(b"unacked").unwrap(); + let offset = pair.client_send(client_ch, s).offset(); + lose_client_packets(&mut pair); + + let now = pair.time; + pair.client_conn_mut(client_ch) + .close(now, VarInt(0), Bytes::new()); + assert_eq!( + acked(&mut pair, client_ch, s, offset), + Err(ClosedStream::default()) + ); +} + +/// Asking about data never written is a caller bug. +#[test] +#[should_panic(expected = "acked past the data written")] +fn acked_past_written_panics() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + let past = pair.client_send(client_ch, s).offset() + 1; + let _ = acked(&mut pair, client_ch, s, past); +} + +/// Resetting past the data written is a caller bug. +#[test] +#[should_panic(expected = "reset past the data written")] +fn reset_past_written_panics() { + let _guard = subscribe(); + let mut pair = pair(true); + let (client_ch, server_ch) = pair.connect(); + let s = open(&mut pair, client_ch, server_ch); + let past = pair.client_send(client_ch, s).offset() + 1; + let _ = pair.client_send(client_ch, s).reset_at(past, VarInt(42)); +} diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index b0ded60c966..2887be2dfe0 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -42,6 +42,7 @@ use crate::{ mod util; use util::*; +mod acked; mod characterize; mod reset_stream_at; mod sim; diff --git a/rs/moq-quic/src/tests/reset_stream_at.rs b/rs/moq-quic/src/tests/reset_stream_at.rs index 5db100f3672..fcf2cd36b54 100644 --- a/rs/moq-quic/src/tests/reset_stream_at.rs +++ b/rs/moq-quic/src/tests/reset_stream_at.rs @@ -36,21 +36,12 @@ fn accept(pair: &mut Pair, ch: ConnectionHandle) -> StreamId { pair.server_streams(ch).accept(Dir::Uni).unwrap() } -/// Write `MSG` to a client stream, with a checkpoint after its first `RELIABLE` bytes -fn write(pair: &mut Pair, ch: ConnectionHandle, s: StreamId) -> Checkpoint { +/// Write `MSG` to a client stream, returning the offset after its first `RELIABLE` bytes +fn write(pair: &mut Pair, ch: ConnectionHandle, s: StreamId) -> u64 { pair.client_send(ch, s).write(&MSG[..RELIABLE]).unwrap(); - let checkpoint = pair.client_send(ch, s).checkpoint(); + let offset = pair.client_send(ch, s).offset(); pair.client_send(ch, s).write(&MSG[RELIABLE..]).unwrap(); - checkpoint -} - -/// Reset a client stream, still delivering what was written before `checkpoint` -fn reset( - pair: &mut Pair, - ch: ConnectionHandle, - checkpoint: Checkpoint, -) -> Result<(), ClosedStream> { - checkpoint.reset(pair.client_conn_mut(ch), ERROR) + offset } /// Drive the pair to idle while the server reads stream `s` as its data arrives, returning the @@ -113,12 +104,14 @@ fn stop_sending_abandons_prefix(reset_stream_at: bool) { pair.client_send(client_ch, s) .write(&MSG[1..RELIABLE]) .unwrap(); - let checkpoint = pair.client_send(client_ch, s).checkpoint(); + let offset = pair.client_send(client_ch, s).offset(); pair.client_send(client_ch, s) .write(&MSG[RELIABLE..]) .unwrap(); lose_client_packets(&mut pair); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.client.drive(pair.time, pair.server.addr); pair.client.outbound.clear(); let resets = resets_sent(&mut pair, client_ch); @@ -155,8 +148,10 @@ fn delivers_prefix_before_error() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = write(&mut pair, client_ch, s); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -187,8 +182,10 @@ fn delivers_unsent_prefix() { pair.client_send(client_ch, s) .write(&MSG[..RELIABLE]) .unwrap(); - let checkpoint = pair.client_send(client_ch, s).checkpoint(); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = pair.client_send(client_ch, s).offset(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -208,9 +205,11 @@ fn retransmits_lost_prefix() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); lose_client_packets(&mut pair); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); // The reset arrives first, and withholds the error until the prefix is retransmitted pair.drive_client(); @@ -245,9 +244,11 @@ fn retransmits_lost_reset() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); pair.drive(); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); lose_client_packets(&mut pair); pair.drive(); @@ -270,8 +271,10 @@ fn received_reset_after_prefix() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = write(&mut pair, client_ch, s); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -304,12 +307,14 @@ fn after_finish() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let mut checkpoint = write(&mut pair, client_ch, s); + let mut offset = write(&mut pair, client_ch, s); if reliable == MSG.len() { - checkpoint = pair.client_send(client_ch, s).checkpoint(); + offset = pair.client_send(client_ch, s).offset(); } pair.client_send(client_ch, s).finish().unwrap(); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -339,7 +344,7 @@ fn flow_control_blocked() { pair.client_send(client_ch, s) .write(&MSG[..RELIABLE]) .unwrap(); - let checkpoint = pair.client_send(client_ch, s).checkpoint(); + let offset = pair.client_send(client_ch, s).offset(); assert_eq!( pair.client_send(client_ch, s).write(&MSG[RELIABLE..]), Ok(10 - RELIABLE) @@ -348,7 +353,9 @@ fn flow_control_blocked() { pair.client_send(client_ch, s).write(MSG), Err(WriteError::Blocked) ); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -360,7 +367,7 @@ fn flow_control_blocked() { } /// Without RESET_STREAM_AT, the plain RESET_STREAM waits until the peer has acknowledged the data -/// before the checkpoint, so the peer reads that data before the error. +/// before the offset, so the peer reads that data before the error. #[test] fn unsupported_holds_reset_until_acked() { let _guard = subscribe(); @@ -371,10 +378,12 @@ fn unsupported_holds_reset_until_acked() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = write(&mut pair, client_ch, s); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); - // The first flight carries only the data before the checkpoint + // The first flight carries only the data before the offset pair.drive_client(); assert_eq!(resets_sent(&mut pair, client_ch), 0); assert_eq!( @@ -396,9 +405,11 @@ fn unsupported_retransmits_lost_prefix() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); lose_client_packets(&mut pair); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); assert_eq!( drive_reading(&mut pair, server_ch, s), @@ -420,7 +431,7 @@ fn unsupported_resets_at_once_when_acked() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); assert_eq!( @@ -428,7 +439,9 @@ fn unsupported_resets_at_once_when_acked() { (MSG.to_vec(), Err(ReadError::Blocked)) ); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive_client(); assert_eq!(resets_sent(&mut pair, client_ch), 1); pair.drive(); @@ -449,9 +462,11 @@ fn unsupported_after_finish() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); pair.client_send(client_ch, s).finish().unwrap(); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); assert_eq!( pair.client_send(client_ch, s).finish(), Err(FinishError::ClosedStream) @@ -485,12 +500,14 @@ fn unsupported_repeated_reset_is_closed() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - let again = pair.client_send(client_ch, s).checkpoint(); + let offset = write(&mut pair, client_ch, s); + let again = pair.client_send(client_ch, s).offset(); lose_client_packets(&mut pair); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); assert_eq!( - reset(&mut pair, client_ch, again), + pair.client_send(client_ch, s).reset_at(again, ERROR), Err(ClosedStream::default()) ); assert!(pair.client_send(client_ch, s).reset(ERROR).is_err()); @@ -507,9 +524,11 @@ fn unsupported_close_while_held() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); + let offset = write(&mut pair, client_ch, s); lose_client_packets(&mut pair); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); let now = pair.time; pair.client_conn_mut(client_ch) .close(now, VarInt(0), Bytes::new()); @@ -520,7 +539,7 @@ fn unsupported_close_while_held() { assert!(pair.server_conn_mut(server_ch).is_closed()); } -/// The data abandoned past the checkpoint still counts against connection flow control until +/// The data abandoned past the offset still counts against connection flow control until /// the peer learns the final size, and then returns. #[test] fn unsupported_returns_flow_control() { @@ -537,8 +556,10 @@ fn unsupported_returns_flow_control() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = write(&mut pair, client_ch, s); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); let next = pair .client_streams(client_ch) .open(Dir::Uni, Default::default()) @@ -567,9 +588,11 @@ fn unsupported_allows_zero() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = pair.client_send(client_ch, s).checkpoint(); + let offset = pair.client_send(client_ch, s).offset(); pair.client_send(client_ch, s).write(MSG).unwrap(); - reset(&mut pair, client_ch, checkpoint).unwrap(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert_eq!(accept(&mut pair, server_ch), s); @@ -589,93 +612,16 @@ fn repeated_reset_is_closed() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - let again = pair.client_send(client_ch, s).checkpoint(); - reset(&mut pair, client_ch, checkpoint).unwrap(); - assert_eq!( - reset(&mut pair, client_ch, again), - Err(ClosedStream::default()) - ); - assert!(pair.client_send(client_ch, s).reset(ERROR).is_err()); -} - -/// A checkpoint taken in 0-RTT that the server rejects names a discarded stream, so it stays -/// closed even once the same stream id is opened again. -#[test] -fn zero_rtt_rejected_checkpoint_is_closed() { - let _guard = subscribe(); - let server_config = ServerConfig::with_crypto(Arc::new(server_crypto_with_alpn(vec![ - "foo".into(), - "bar".into(), - ]))); - let mut pair = Pair::new(Arc::new(EndpointConfig::default()), server_config); - let mut client_crypto = Arc::new(client_crypto_with_alpn(vec!["foo".into()])); - - let client_ch = pair.begin_connect(ClientConfig::new(client_crypto.clone())); - pair.drive(); - pair.server.assert_accept(); - let now = pair.time; - pair.client_conn_mut(client_ch) - .close(now, VarInt(0), Bytes::new()); - pair.drive(); - pair.client.connections.clear(); - pair.server.connections.clear(); - - // A changed ALPN makes the server reject the resumed session's 0-RTT - let this = Arc::get_mut(&mut client_crypto).expect("QuicClientConfig is shared"); - let inner = Arc::get_mut(&mut this.inner).expect("QuicClientConfig.inner is shared"); - inner.alpn_protocols = vec!["bar".into()]; - let client_ch = pair.begin_connect(ClientConfig::new(client_crypto)); - assert!(pair.client_conn_mut(client_ch).has_0rtt()); - let s = pair - .client_streams(client_ch) - .open(Dir::Uni, Default::default()) - .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - pair.drive(); - assert!(!pair.client_conn_mut(client_ch).accepted_0rtt()); - - let reopened = pair - .client_streams(client_ch) - .open(Dir::Uni, Default::default()) + let offset = write(&mut pair, client_ch, s); + let again = pair.client_send(client_ch, s).offset(); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) .unwrap(); - assert_eq!(reopened, s); - pair.client_send(client_ch, reopened).write(MSG).unwrap(); assert_eq!( - reset(&mut pair, client_ch, checkpoint), + pair.client_send(client_ch, s).reset_at(again, ERROR), Err(ClosedStream::default()) ); - pair.client_send(client_ch, reopened).finish().unwrap(); - pair.drive(); - - let server_ch = pair.server.assert_accept(); - assert_eq!(accept(&mut pair, server_ch), s); - assert_eq!(read(&mut pair, server_ch, s), (MSG.to_vec(), Ok(()))); -} - -/// A checkpoint from another connection panics, even where that connection has a stream of the -/// same id holding at least as much data. -#[test] -#[should_panic(expected = "checkpoint used on another connection")] -fn checkpoint_from_another_connection() { - let _guard = subscribe(); - let mut first = pair(true); - let (first_ch, _) = first.connect(); - let mut second = pair(true); - let (second_ch, _) = second.connect(); - - let s = first - .client_streams(first_ch) - .open(Dir::Uni, Default::default()) - .unwrap(); - let checkpoint = write(&mut first, first_ch, s); - let other = second - .client_streams(second_ch) - .open(Dir::Uni, Default::default()) - .unwrap(); - assert_eq!(other, s); - second.client_send(second_ch, other).write(MSG).unwrap(); - let _ = reset(&mut second, second_ch, checkpoint); + assert!(pair.client_send(client_ch, s).reset(ERROR).is_err()); } /// A client remembers the server's support for 0-RTT, and may use it there. @@ -703,8 +649,10 @@ fn zero_rtt() { .client_streams(client_ch) .open(Dir::Uni, Default::default()) .unwrap(); - let checkpoint = write(&mut pair, client_ch, s); - reset(&mut pair, client_ch, checkpoint).unwrap(); + let offset = write(&mut pair, client_ch, s); + pair.client_send(client_ch, s) + .reset_at(offset, ERROR) + .unwrap(); pair.drive(); assert!(pair.client_conn_mut(client_ch).accepted_0rtt()); diff --git a/rs/moq-tokio/src/quic/driver/mod.rs b/rs/moq-tokio/src/quic/driver/mod.rs index 029b1657c9c..5acc2455104 100644 --- a/rs/moq-tokio/src/quic/driver/mod.rs +++ b/rs/moq-tokio/src/quic/driver/mod.rs @@ -81,7 +81,7 @@ pub(crate) use crate::quic::driver::runtime::send_failed; pub(crate) use crate::quic::driver::runtime::wrap_udp_socket; pub use crate::quic::driver::runtime::{AsyncUdpSocket, UdpSender}; pub use crate::quic::driver::send_group::{SendGroup, SendOptions}; -pub use crate::quic::driver::send_stream::{Checkpoint, SendStream, StoppedError, WriteError}; +pub use crate::quic::driver::send_stream::{SendStream, StoppedError, WriteError}; #[cfg(test)] mod tests; diff --git a/rs/moq-tokio/src/quic/driver/send_stream.rs b/rs/moq-tokio/src/quic/driver/send_stream.rs index 63401220104..44010e1e1d5 100644 --- a/rs/moq-tokio/src/quic/driver/send_stream.rs +++ b/rs/moq-tokio/src/quic/driver/send_stream.rs @@ -214,15 +214,25 @@ impl SendStream { Ok(()) } - /// Mark the data written so far, so a later [`Checkpoint::reset()`] still delivers it. - pub fn checkpoint(&self) -> Checkpoint { - let mut conn = self.conn.state.lock("SendStream::checkpoint"); - let inner = conn.inner.send_stream(self.stream).checkpoint(); - Checkpoint { - conn: self.conn.clone(), - inner, - is_0rtt: self.is_0rtt, + /// Close the send stream immediately, but still deliver its first `offset` bytes. + /// + /// Like [`SendStream::reset()`], except the data before `offset` still reaches the peer. + /// A peer that accepts RESET_STREAM_AT reads it before the error. Otherwise the reset waits a + /// round trip until the peer acknowledges the data, and the peer's application may never read + /// it: see [`moq_quic::SendStream::reset_at`]. + /// + /// # Panics + /// When `offset` is past the data written. + pub fn reset_at(&mut self, offset: u64, error_code: VarInt) -> Result<(), ClosedStream> { + let mut conn = self.conn.state.lock("SendStream::reset_at"); + if self.is_0rtt && conn.check_0rtt().is_err() { + return Ok(()); } + conn.inner + .send_stream(self.stream) + .reset_at(offset, error_code)?; + conn.wake(); + Ok(()) } /// Set the priority within the stream's send group, or session-wide when ungrouped @@ -355,34 +365,6 @@ impl tokio::io::AsyncWrite for SendStream { } } -/// The data written to a [`SendStream`] up to some point, which a reset can still deliver -/// -/// Taken with [`SendStream::checkpoint()`]. It keeps the connection open, like the stream does. -#[derive(Debug)] -pub struct Checkpoint { - conn: ConnectionRef, - inner: moq_quic::Checkpoint, - is_0rtt: bool, -} - -impl Checkpoint { - /// Close the send stream immediately, but still deliver everything written before the checkpoint. - /// - /// Like [`SendStream::reset()`], except the data before the checkpoint still reaches the peer. - /// A peer that accepts RESET_STREAM_AT reads it before the error. Otherwise the reset waits a - /// round trip until the peer acknowledges the data, and the peer's application may never read - /// it: see [`moq_quic::Checkpoint::reset`]. - pub fn reset(self, error_code: VarInt) -> Result<(), ClosedStream> { - let mut conn = self.conn.state.lock("Checkpoint::reset"); - if self.is_0rtt && conn.check_0rtt().is_err() { - return Ok(()); - } - self.inner.reset(&mut conn.inner, error_code)?; - conn.wake(); - Ok(()) - } -} - impl Drop for SendStream { fn drop(&mut self) { let mut conn = self.conn.state.lock("SendStream::drop"); diff --git a/rs/moq-tokio/src/quic/web/send.rs b/rs/moq-tokio/src/quic/web/send.rs index 3c807c0fc7a..30aae19ba57 100644 --- a/rs/moq-tokio/src/quic/web/send.rs +++ b/rs/moq-tokio/src/quic/web/send.rs @@ -17,16 +17,12 @@ use crate::quic::web::{CloseReason, ClosedStream, SessionError, WriteError}; pub struct SendStream { stream: crate::quic::driver::SendStream, close: Arc, - /// Taken after the WebTransport header this side wrote first, which a reset must still deliver. - header: Option, + /// The end of the WebTransport header this side wrote first, which a reset must still deliver. + header: Option, } impl SendStream { - pub(crate) fn new( - stream: crate::quic::driver::SendStream, - close: Arc, - header: Option, - ) -> Self { + pub(crate) fn new(stream: crate::quic::driver::SendStream, close: Arc, header: Option) -> Self { Self { stream, close, header } } @@ -53,7 +49,7 @@ impl SendStream { pub fn reset(&mut self, code: u32) -> Result<(), ClosedStream> { let code = self.close.encode_stream_code(code); match self.header.take() { - Some(header) => header.reset(code).map_err(Into::into), + Some(header) => self.stream.reset_at(header, code).map_err(Into::into), None => self.stream.reset(code).map_err(Into::into), } } diff --git a/rs/moq-tokio/src/quic/web/session.rs b/rs/moq-tokio/src/quic/web/session.rs index d14ed86f191..b65d732209d 100644 --- a/rs/moq-tokio/src/quic/web/session.rs +++ b/rs/moq-tokio/src/quic/web/session.rs @@ -282,7 +282,7 @@ impl Session { if let Some(group) = options.group { send.set_group(Some(group)).ok(); } - let header = send.checkpoint(); + let header = self.header_uni.len() as u64; Ok(SendStream::new(send, self.close.clone(), Some(header))) } @@ -306,7 +306,7 @@ impl Session { if let Some(group) = options.group { send.set_group(Some(group)).ok(); } - let header = send.checkpoint(); + let header = self.header_bi.len() as u64; Ok(( SendStream::new(send, self.close.clone(), Some(header)), RecvStream::new(recv, self.close.clone()), diff --git a/rs/moq-uring/src/quic/driver/connection.rs b/rs/moq-uring/src/quic/driver/connection.rs index 5bc8070b419..8dcb786ef21 100644 --- a/rs/moq-uring/src/quic/driver/connection.rs +++ b/rs/moq-uring/src/quic/driver/connection.rs @@ -974,6 +974,8 @@ fn sweep_stream(state: &mut State, event: moq_quic::StreamEvent) { waiters.wake(); } } + // Nothing waits on a send stream's acknowledgment yet. + moq_quic::StreamEvent::Acked { .. } => {} } } diff --git a/rs/moq-uring/src/quic/driver/stream.rs b/rs/moq-uring/src/quic/driver/stream.rs index 36157e1cee6..b00ec6d9c34 100644 --- a/rs/moq-uring/src/quic/driver/stream.rs +++ b/rs/moq-uring/src/quic/driver/stream.rs @@ -8,7 +8,7 @@ use std::task::{Context, Poll, ready}; use bytes::{Buf, Bytes, BytesMut}; -use moq_quic::{Checkpoint, StreamId, VarInt}; +use moq_quic::{StreamId, VarInt}; use super::super::Error; use super::{End, Shared}; @@ -51,9 +51,9 @@ impl SendStream { self.fin || self.reset } - /// Mark the data written so far, for a later [`reset_code`](Self::reset_code) to deliver. - pub(crate) fn checkpoint(&self) -> Checkpoint { - self.shared.conn.borrow_mut().send_stream(self.id).checkpoint() + /// How many bytes were written so far, for a later [`reset_code`](Self::reset_code) to deliver. + pub(crate) fn offset(&self) -> u64 { + self.shared.conn.borrow_mut().send_stream(self.id).offset() } /// Set the priority within the stream's send group, or session-wide when ungrouped. @@ -80,16 +80,16 @@ impl SendStream { /// [`reset`](web_transport_trait::poll::SendStream::reset) with a /// full-width code, for the WebTransport HTTP/3 error mapping, that still - /// delivers the data before `checkpoint`, one this stream took. - pub(crate) fn reset_code(&mut self, code: u64, checkpoint: Option) { + /// delivers the first `offset` bytes. + pub(crate) fn reset_code(&mut self, code: u64, offset: Option) { if self.reset { return; } let code = VarInt::from_u64(code).unwrap_or(VarInt::MAX); let mut conn = self.shared.conn.borrow_mut(); // Err means the stream is already gone, which is what we wanted. - let _ = match checkpoint { - Some(checkpoint) => checkpoint.reset(&mut conn, code), + let _ = match offset { + Some(offset) => conn.send_stream(self.id).reset_at(offset, code), None => conn.send_stream(self.id).reset(code), }; drop(conn); diff --git a/rs/moq-uring/src/quic/web.rs b/rs/moq-uring/src/quic/web.rs index 9733fae3bce..e644043da0a 100644 --- a/rs/moq-uring/src/quic/web.rs +++ b/rs/moq-uring/src/quic/web.rs @@ -844,7 +844,7 @@ pub struct SendStream { inner: super::SendStream, /// The end of the WebTransport header on a stream we opened, for a reset /// to deliver. - header: Option, + header: Option, web: bool, } @@ -875,8 +875,8 @@ impl SendStream { self.inner.set_group(group); } - /// Write the WebTransport header of a stream we opened, checkpointing it - /// once it is all written. + /// Write the WebTransport header of a stream we opened, noting where it + /// ends once it is all written. fn poll_header(&mut self, cx: &mut Context<'_>, header: &mut Bytes) -> Poll> { if !self.web { return Poll::Ready(Ok(())); @@ -890,7 +890,7 @@ impl SendStream { .map_err(|err| self.map(err))?; header.advance(n); } - self.header = Some(self.inner.checkpoint()); + self.header = Some(self.inner.offset()); Poll::Ready(Ok(())) }