Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 0 additions & 50 deletions quest/m1/quic-handshake-loss.md

This file was deleted.

38 changes: 38 additions & 0 deletions quest/m1/relay-accept-deadline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# [S] A relay's accept deadline outlasts a client's handshake backoff

## Goal

A relay keeps a client on a bad network long enough to finish its QUIC
handshake under the QUIC Interop Runner's 30% loss, so `--listen-timeout`
does not shed the clients moq-quic's own handshake timeout now keeps.

## Plan

Found by [the handshake loss quest](https://github.com/moq-dev/moq/pull/8824),
which raised moq-quic's `handshake_timeout` to 30 s, an inactivity bound.
`moq_net::server::DEFAULT_TIMEOUT` (10 s), which `moq-tokio`'s
`listen.timeout` and so the relay's `--listen-timeout` default to, is a total
deadline from the connection's arrival. It covers the QUIC handshake, the
client's SETUP, and the caller's admission, and for moq-lite 01/02, whose
server SETUP waits on `ok()`, the reply too; every other version sends its
reply from the started session, outside the deadline. Under the QUIC Interop
Runner's 30% loss a client's probe gaps alone reach 13 s, so this deadline
still drops it.

Decided 2026-11-15 (maintainer, /quest-complete): raise the default to 30 s
(rejected: keep 10 s), the same as `moq-tokio`'s connect timeout on the client
side, and update `doc/bin/relay/config.md`, `doc/bin/cli.md`, and
`doc/lib/rs/moq-net.md`.

A total deadline is not an inactivity bound: a client that keeps making
progress but needs more than 30 s overall, such as one configured with a
handshake timeout above 30 s and hitting several long gaps, is still shed.
That is the intended trade: past 30 s the client's own connect timeout gives
up too.

Nothing else bounds a half-open session: `Admission` only counts active
connections for draining and caps none, so this deadline is the whole bound on
how long each one holds its slot, now up to three times longer.

Public API: the default of `moq_net::Server::with_timeout` and `--listen-timeout`,
a behavior change. Wire: none.
41 changes: 41 additions & 0 deletions quest/m4/quic-handshake-speedup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# [M] moq-quic re-sends a lost ServerHello before its PTO

## Goal

Under heavy loss or corruption, a moq-quic handshake whose ServerHello was
lost recovers within a round trip of the evidence, instead of waiting out
PTOs backed off past 16 s. The QUIC Interop Runner's `handshakecorruption`
case passes for the moq-quic self pair, and its `known` entry in
`test/quic-interop/cases.toml` is gone.

## Plan

Found by [the handshake loss quest](https://github.com/moq-dev/moq/pull/8824),
which fixed the handshake timeout and the server's CID switch. The self
pair still failed `handshakecorruption` in one of nine runs afterwards. In the
failing connection's pcap, the server's Handshake flight reached the client
before the ServerHello did, so the client discarded it; its own probes were
PINGs (its ClientHello was already acknowledged), which the server answered
with bare ACKs; and once the ServerHello arrived, the client's anti-deadlock
probe and the server's next Handshake retransmission were both 8 to 16 s out.
The server heard nothing for 30 s and gave up.

RFC 9002 section 6.2.3 allows sending unacknowledged CRYPTO data before the PTO,
a limited number of times per connection:

- A client that receives Handshake packets before it has Handshake keys
probes in Initial at once.
- A server that receives an ack-eliciting client Initial while its own
Initial CRYPTO is unacknowledged re-sends that CRYPTO at once, within
the anti-amplification limit.

Cover it on the seeded sim (`rs/moq-quic/src/tests/sim.rs`,
`handshake_under_loss`) with corruption, and rerun
`just test quic-interop --peers moq-quic --cases handshakeloss,handshakecorruption --roles server`
until it is stable.

Public API: none. Wire: none.

## Related

- [Public interop listing](/quest/m4/quic-interop-listing.md) - this cell is public there
3 changes: 2 additions & 1 deletion rs/moq-quic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,9 @@ Changes on top of the upstream commit, besides the renames:
- Discarding Initial or Handshake keys resets the PTO backoff, as RFC 9002 A.11 does, so a client's Initial backoff no longer delays the retransmission of a lost Finished. Also [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); not offered upstream yet.
- **Probes on the peer heard again**: a 1-RTT probe that times out with nothing heard from the peer since the packet it probes for went out lets the next packet heard send the next probe at once, rather than leave a path back from an outage idle for the rest of the backoff, which still doubles with each probe (RFC 9002 section 6.2.1), up to 2^16 times its base. A packet acknowledging anything new already resets the backoff; this covers one that does not, as after an outage that began with everything acknowledged. A peer heard while the probes still go unanswered says nothing of their path, so it leaves the backoff alone. Not offered upstream yet.
- **Keep-alive stays enabled**: zero uses the derived interval, and a connection with no idle timeout uses the default idle timeout to derive it. `keep_alive_interval` returns a duration. Not offered upstream: MoQ sessions keep quiet connections and their NAT bindings alive.
- `transport::Config::handshake_timeout` (default 10s, as msquic's `HandshakeIdleTimeoutMs`) governs the idle timer until the handshake completes, and the negotiated idle timeout takes over once established. A 2s idle timeout otherwise expires 3x the initial PTO in, just before the second Initial probe, leaving a client two flights. A server also drops an Initial left waiting for `accept` past it, and a deadline too far out to represent never fires instead of panicking. Also [moq-dev/noq#34](https://github.com/moq-dev/noq/pull/34), whose third change (the handshake idle timeout picks noq's idle-based PTO interval cap) has nothing to apply to here, since quinn's PTO backoff has no such cap.
- `transport::Config::handshake_timeout` (default 30s, the idle timeout's default, so a client's probes backed off past 10s under heavy loss still land) governs the idle timer until the handshake completes, and the negotiated idle timeout takes over once established. A 2s idle timeout otherwise expires 3x the initial PTO in, just before the second Initial probe, leaving a client two flights. A server also drops an Initial left waiting for `accept` past it, and a deadline too far out to represent never fires instead of panicking. Also [moq-dev/noq#34](https://github.com/moq-dev/noq/pull/34), whose third change (the handshake idle timeout picks noq's idle-based PTO interval cap) has nothing to apply to here, since quinn's PTO backoff has no such cap.
quinn's `handshake_timeout` test in its async layer expects the handshake to end at the idle timeout, so the driver's copy sets both timeouts.
- **In-order CID switch**: a server still on the client's initial CID moves to sequence 1 once it arrives, not to whichever NEW\_CONNECTION\_ID arrives first, since jumping retires the sequences it skipped while they are still in flight, and quiche's client then fails with `InvalidState` re-sending one it was told to retire.
- **Time on [`kio::clock`](../kio)**: the crate's `Instant` is `kio::clock::Instant`, so one clock steps every timer, tokio's paused clock and the simulator included. qlog stamps events relative to `QlogConfig::start_time` on that clock rather than the streamer's own `std::time::Instant`. Address-validation tokens and their log read `kio::clock::wall()` instead of quinn's `TimeSource`, for portable 1970-2554 wall times; the simulator steps wall time with its clock, but paused tokio does not, so token expiry there follows the host's wall clock.
- **Send groups** (`streams/schedule.rs`, `Connection::send_groups`): streams outside every group go first, in quinn's order; groups are served strictly by priority, with equal priorities sharing bytes by deficit round robin. A stream's priority is an `i64` ordering it only within its group, and `Streams::open` takes a `send::Options { group, priority }` to start it there, like W3C WebTransport's send options. This replaces quinn's single pending-stream heap, so a port touching `PendingStreamsQueue` has to be redone against the schedule. moq-tokio's `quic::driver` and moq-uring's driver wrap a group in a shared `send::Group` handle that closes on drop.
- **Scheduled datagrams** (`Datagrams::send`): a datagram is queued by its `send::Options`, in a send group (or ungrouped) at an `i64` priority, and scheduled with the group's streams, winning a tie with a stream of equal priority and spending the group's byte credit, instead of quinn's datagram-first FIFO in `populate_packet`. Sending never blocks: `datagram_send_buffer_size` is shared, and an overflow evicts the oldest datagram of the lowest-priority group. `SendDatagramError::Blocked` and `Event::DatagramsUnblocked` are gone, and a port touching the `// DATAGRAM` loop has to be redone against `StreamsState::write_frames`.
Expand Down
36 changes: 36 additions & 0 deletions rs/moq-quic/src/cid_queue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,20 @@ impl CidQueue {
Some((cid_data.1.unwrap(), orig_offset..self.offset))
}

/// Switch to the CID right after the active one, if known, returning its reset token and
/// the retired active sequence number
pub(crate) fn next_in_order(&mut self) -> Option<(ResetToken, Range<u64>)> {
let next = (self.cursor + 1) % Self::LEN;
let (_, token) = self.buffer[next]?;
self.buffer[self.cursor] = None;
self.cursor = next;
self.offset += 1;
Some((
token.expect("non-initial CID missing reset token"),
self.offset - 1..self.offset,
))
}

/// Iterate CIDs in CidQueue that are not `None`, including the active CID
fn iter(&self) -> impl Iterator<Item = (usize, CidData)> + '_ {
(0..Self::LEN).filter_map(move |step| {
Expand Down Expand Up @@ -178,6 +192,28 @@ mod tests {
assert!(q.next().is_none());
}

#[test]
fn next_in_order_waits_for_the_gap() {
let mut q = CidQueue::new(initial_cid());
q.insert(cid(4, 0)).unwrap();
q.insert(cid(2, 0)).unwrap();
assert!(q.next_in_order().is_none(), "sequence 1 is still in flight");
assert_eq!(q.active_seq(), 0);

q.insert(cid(1, 0)).unwrap();
assert_eq!(q.next_in_order().unwrap().1, 0..1);
assert_eq!(q.active_seq(), 1);
assert_eq!(q.next_in_order().unwrap().1, 1..2);
assert!(q.next_in_order().is_none(), "sequence 3 is still in flight");

// The CIDs that arrived early are still there to use.
q.insert(cid(3, 0)).unwrap();
assert_eq!(q.next_in_order().unwrap().1, 2..3);
assert_eq!(q.next_in_order().unwrap().1, 3..4);
assert_eq!(q.active_seq(), 4);
assert!(q.next_in_order().is_none());
}

#[test]
fn wrap() {
let mut q = CidQueue::new(initial_cid());
Expand Down
7 changes: 4 additions & 3 deletions rs/moq-quic/src/config/transport/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,9 @@ pub struct Config {
/// Used in place of [`idle_timeout`](Self::idle_timeout) until the handshake completes, so a
/// short idle timeout meant to detect a dead peer quickly does not also cut a handshake off
/// after a single probe. It applies whether the idle timeout is shorter, longer, or disabled,
/// and whatever idle timeout the peer advertises. Defaults to 10 seconds, as msquic's
/// `HandshakeIdleTimeoutMs`.
/// and whatever idle timeout the peer advertises. Defaults to 30 seconds, the default idle
/// timeout: under heavy loss a client backs its probes off past 10 seconds, and a half-open
/// handshake costs no more than an idle connection.
pub handshake_timeout: Duration,

/// Period of inactivity before sending a keep-alive packet, `None` (the default) to derive it
Expand Down Expand Up @@ -286,7 +287,7 @@ impl Default for Config {
Self {
limits: Limits::default(),
idle_timeout: Some(DEFAULT_IDLE_TIMEOUT),
handshake_timeout: Duration::from_secs(10),
handshake_timeout: DEFAULT_IDLE_TIMEOUT,
keep_alive: None,
send_fairness: true,

Expand Down
13 changes: 9 additions & 4 deletions rs/moq-quic/src/connection/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3312,10 +3312,15 @@ impl Connection {
}
};

if self.side.is_server() && self.rem_cids.active_seq() == 0 {
// We're a server still using the initial remote CID for the client, so
// let's switch immediately to enable clientside stateless resets.
self.update_rem_cid();
// A server still on the client's initial CID switches at once, to enable
// client-side stateless resets, but only to sequence 1: jumping to a later
// one would retire the CIDs before it, still in flight, without using them.
if self.side.is_server()
&& self.rem_cids.active_seq() == 0
&& let Some((reset_token, retired)) = self.rem_cids.next_in_order()
{
self.spaces[SpaceId::Data].pending.retire_cids.extend(retired);
self.set_reset_token(reset_token);
}
}
Frame::NewToken(NewToken { token }) => {
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-quic/src/tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1194,7 +1194,7 @@ fn deferred_accept_uses_handshake_idle_timeout() {
pair.time = start.saturating_add(Duration::from_secs(3));
assert!(pair.server.try_accept(delayed, pair.time).is_ok());

pair.time = start.saturating_add(Duration::from_secs(10));
pair.time = start.saturating_add(transport::Config::default().handshake_timeout);
assert_matches!(pair.server.try_accept(late, pair.time), Err(ConnectionError::TimedOut));
}

Expand Down
Loading