Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions quest/m3/qos/lag-histogram.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,10 @@ Decided 2026-10-26 (maintainer, on #6791), each with its reason:
`Billing` mints an offset whenever a write ends a frame and retires them
oldest first, so the frontier rides those offsets rather than minting its
own.
- **No hook, no fields.** A subscription whose backend reports `poll_acked`
unsupported (qmux, the browser transport) keeps no frontier and adds
- **No hook, no fields.** A subscription whose backend cannot see
acknowledgments (`poll_acked` answers `Unsupported`, as the browser
transport does, or `Sent`, as qmux does once its link took the bytes, which
says nothing of when the peer got them) keeps no frontier and adds
nothing to `lag` or `dropped`. A node serving only such viewers leaves both
absent. Document this in `doc/concept/stats.md`.
- **The lag edge is the newest group's.** On the branch, `Produced.newest` is
Expand Down
12 changes: 8 additions & 4 deletions quest/m4/quic-ack-hook.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,14 @@ Those methods, `poll_acked` on `send::Stream` and `send::Finished` and
`Finished::offset`, forward to `moq-quic`'s public `send::Stream::poll_acked`.
They fail with the peer's stop code (`send::Error::Stopped`) or the
connection's loss (`send::Error::Connection`); a reset consumes the handle, so
no handle ever waits past one. qmux and the browser transport stay on the
trait's default `Acked::Unsupported` (decided 2026-10-22 and 2026-10-26,
maintainer): writing into TCP is not delivery, and the browser's
`WebTransportSendStream.getStats()` is unimplemented in shipping Chrome.
no handle ever waits past one. qmux answers `Acked::Sent` once its link
takes the bytes, which billing counts and acknowledgment consumers ignore:
writing into TCP is still not delivery (decided 2026-10-22, maintainer;
`Sent` added 2026-11-17 in
[moq-dev/moq#8930](https://github.com/moq-dev/moq/pull/8930)). The browser
transport stays on the trait's default `Acked::Unsupported` (decided
2026-10-26): its `WebTransportSendStream.getStats()` is unimplemented in
shipping Chrome.

Decided 2026-11-15 (maintainer, /quest-complete): ✅ keep only the trap tests,
on the crate-private methods, and drop the public driver API this quest
Expand Down
26 changes: 21 additions & 5 deletions rs/moq-net/src/coding/writer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -339,8 +339,9 @@ impl<S: crate::transport::poll::SendStream, V: StreamCodes> Writer<S, V> {
}
}

/// A stream's payload billed as the transport acknowledges it, through offsets the writer asks
/// [`poll_acked`](crate::transport::poll::SendStream::poll_acked) about.
/// A stream's payload billed as the transport acknowledges it, or a reliable link takes it,
/// through offsets the writer asks [`poll_acked`](crate::transport::poll::SendStream::poll_acked)
/// about.
struct Billing<O> {
delivery: crate::stats::Delivery,
ack: Ack<O>,
Expand Down Expand Up @@ -414,7 +415,8 @@ impl<O> Billing<O> {
while let Some((offset, through)) = asked.front() {
match stream.poll_acked(waiter, offset) {
Poll::Pending => break,
Poll::Ready(Ok(Acked::Delivered)) => {
// Bytes a reliable link took bill too: a reset can no longer discard them.
Poll::Ready(Ok(Acked::Delivered | Acked::Sent)) => {
self.delivery.deliver(*through);
asked.pop_front();
}
Expand Down Expand Up @@ -630,6 +632,7 @@ mod tests {

/// A stream that takes every write, acknowledging a prefix the test moves, or that cannot
/// see acknowledgments. It counts the polls waiting on acknowledgment, which stay pending.
/// With `sent`, it answers for the prefix as a reliable link that took it.
#[derive(Clone, Default)]
struct Acking(std::sync::Arc<std::sync::Mutex<AckingState>>);

Expand All @@ -638,6 +641,7 @@ mod tests {
written: u64,
acked: u64,
unsupported: bool,
sent: bool,
reset: bool,
waits: usize,
}
Expand All @@ -649,6 +653,12 @@ mod tests {
stream
}

fn sent() -> Self {
let stream = Self::default();
stream.0.lock().unwrap().sent = true;
stream
}

fn ack(&self, through: u64) {
self.0.lock().unwrap().acked = through;
}
Expand All @@ -675,6 +685,7 @@ mod tests {
let mut state = self.0.lock().unwrap();
match () {
_ if state.unsupported => Poll::Ready(Ok(crate::transport::poll::Acked::Unsupported)),
_ if *offset <= state.acked && state.sent => Poll::Ready(Ok(crate::transport::poll::Acked::Sent)),
_ if *offset <= state.acked => Poll::Ready(Ok(crate::transport::poll::Acked::Delivered)),
_ if state.reset => Poll::Ready(Err(Stopped(0))),
_ => {
Expand Down Expand Up @@ -738,10 +749,15 @@ mod tests {
/// A frame bills once the transport acknowledges an offset past it, not as the stream takes
/// it: each write that ends a frame asks about its own offset, so a frame acknowledged before
/// a reset still bills, and nothing the reset discarded does. A park with nothing else to ask
/// about asks about a frame still being written, billing its acknowledged bytes.
/// about asks about a frame still being written, billing its acknowledged bytes. Bytes a
/// reliable link took bill the same way.
#[test]
fn egress_bills_acknowledged_payload() {
let stream = Acking::default();
egress_bills_payload(Acking::default());
egress_bills_payload(Acking::sent());
}

fn egress_bills_payload(stream: Acking) {
let (mut writer, registry) = billing(&stream);
let waiter = &kio::Waiter::noop();

Expand Down
3 changes: 2 additions & 1 deletion rs/moq-net/src/stats.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@
//! * `bytes` / `frames` / `groups`: cumulative payload counters bumped as
//! groups/frames are read (egress) or written (ingress) in the model. A
//! session's egress `bytes` / `frames` count only payload the peer
//! acknowledged (see [`SendStream::poll_acked`](crate::transport::poll::SendStream::poll_acked)),
//! acknowledged, or that a reliable link such as qmux's took (see
//! [`SendStream::poll_acked`](crate::transport::poll::SendStream::poll_acked)),
//! never what a reset or a lost session discarded: a frame's bytes once an
//! offset past them is acknowledged, and the frame with its last byte. A
//! transport that cannot see acknowledgments counts the bytes the stream
Expand Down
5 changes: 5 additions & 0 deletions rs/moq-net/src/transport.rs
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,10 @@ pub mod poll {
Delivered,
/// The transport cannot see acknowledgments, so delivery is unknown, never assumed.
Unsupported,
/// The transport handed every byte below the offset to a reliable link, such as qmux's
/// TCP, which delivers it unless the link fails. Not an acknowledgment: it says nothing of
/// when the peer got it, but a reset can no longer discard it.
Sent,
}

/// An outgoing stream with partial writes, priorities, and reset codes.
Expand All @@ -727,6 +731,7 @@ pub mod poll {
///
/// Errors once those bytes can never be acknowledged: the stream was reset by either
/// side or the session closed. Bytes already acknowledged stay so after a reset or stop.
/// A transport over a reliable link answers [`Acked::Sent`] once the link took them.
/// A transport that cannot see acknowledgments is ready with [`Acked::Unsupported`]
/// immediately, the default, on every stream, so a caller can decide once per session. An
/// offset minted by another stream may panic.
Expand Down
22 changes: 19 additions & 3 deletions rs/qmux/src/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -527,10 +527,26 @@ impl SendStream {
impl poll::SendStream for SendStream {
type Error = Error;
type Group = SendGroup;
// Writing into TCP is not delivery, so acknowledgments stay unsupported.
type Offset = ();
type Offset = send::Offset;

fn offset(&self) -> Self::Offset {}
fn offset(&self) -> send::Offset {
self.inner.lock().conn.send_stream(self.id).offset()
}

/// [`Acked::Sent`](poll::Acked::Sent) once the transport took the bytes, which is not the
/// peer acknowledging them. A reset or stop keeps the prefix that went out before it.
fn poll_acked(&self, waiter: &kio::Waiter, offset: &send::Offset) -> Poll<Result<poll::Acked, Error>> {
let mut state = self.inner.lock();
let mut stream = state.conn.send_stream(self.id);
if u64::from(*offset) <= stream.acked_len() {
return Poll::Ready(Ok(poll::Acked::Sent));
}
match ready!(stream.poll_acked(waiter, *offset)) {
Ok(()) => Poll::Ready(Ok(poll::Acked::Sent)),
Err(send::Error::ConnectionLost) => Poll::Ready(Err(state.lost())),
Err(_) => Poll::Ready(Err(Error::StreamClosed)),
}
}

fn poll_write(&mut self, waiter: &kio::Waiter, mut buf: &[u8]) -> Poll<Result<usize, Error>> {
self.poll_write_buf(waiter, &mut buf)
Expand Down
43 changes: 43 additions & 0 deletions rs/qmux/src/session/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,49 @@ async fn read_exact(recv: &mut RecvStream, want: usize) -> Result<Bytes, Error>
.await
}

/// What `send` answers about the bytes below `offset`, without waiting.
fn acked(send: &SendStream, offset: &moq_quic::send::Offset) -> Poll<Result<poll::Acked, Error>> {
send.poll_acked(&kio::Waiter::noop(), offset)
}

/// A stream's bytes count as sent once the transport takes them, which a stalled link holds
/// back. A reset discards what the link still holds, so only the prefix that went out stays sent,
/// and egress never bills the rest.
#[tokio::test(start_paused = true)]
async fn bytes_count_as_sent_once_the_link_takes_them() {
for &version in Version::ALL {
let (client, server, up, _down) = link();
let (client, server) = establish(client, server, config(version), config(version)).await;

let mut send = client.open_uni(SendOptions::default()).await.unwrap();
let first = payload(1000, 5);
write_all(&mut send, first.clone()).await;
let mut recv = server.accept_uni().await.unwrap();
assert_eq!(read_exact(&mut recv, first.len()).await.unwrap(), first, "{version:?}");
let went_out = send.offset();
assert!(
matches!(acked(&send, &went_out), Poll::Ready(Ok(poll::Acked::Sent))),
"{version:?}: the link took it"
);

up.send(Flow::Stall).unwrap();
// More than the pipe holds, within the peer's stream window.
write_all(&mut send, payload(2 * PIPE, 6)).await;
let held = send.offset();
assert!(acked(&send, &held).is_pending(), "{version:?}: the link holds the rest");

send.reset(1);
assert!(
matches!(acked(&send, &went_out), Poll::Ready(Ok(poll::Acked::Sent))),
"{version:?}: what went out stays sent"
);
assert!(
matches!(acked(&send, &held), Poll::Ready(Err(_))),
"{version:?}: the reset discarded the rest"
);
}
}

/// Leased bytes keep their credit, so incomplete messages fill the receive window rather than
/// memory outside it. Once no reader can make progress, the lowest priority holder is evicted,
/// and the credit it held lets the others carry on.
Expand Down