From 12a726c8df1e849eb44c43ebf3b7829178e4d760 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 04:56:29 -0800 Subject: [PATCH 1/4] chore(quest): claim quest/a2/quiche-delivery-rate.md Quest-Claim: 89b5c03b28a43481 From b684dc4870732e5c29c57e8f1775b3f5b19675fe Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 05:03:43 -0800 Subject: [PATCH 2/4] docs(quiche): audit send-rate controller semantics Document the upstream app-limited guards and retain BBR2's congestion bound. Complete the send-rate audit quest without changing runtime behavior. Co-Authored-By: GPT-6.1 Sol --- quest/a1/qmux-rate-filter.md | 2 +- quest/a2/README.md | 1 - quest/a2/quiche-delivery-rate.md | 13 ------- rs/web-transport-quiche/README.md | 39 ++++++++++++++++++++ rs/web-transport-quiche/src/connection.rs | 1 + rs/web-transport-quiche/src/error.rs | 2 +- rs/web-transport-quiche/src/ez/connection.rs | 17 +++++++-- 7 files changed, 56 insertions(+), 19 deletions(-) delete mode 100644 quest/a2/quiche-delivery-rate.md diff --git a/quest/a1/qmux-rate-filter.md b/quest/a1/qmux-rate-filter.md index b16f041a..b24619ea 100644 --- a/quest/a1/qmux-rate-filter.md +++ b/quest/a1/qmux-rate-filter.md @@ -13,7 +13,7 @@ Decided with the maintainer on 2026-11-06 (planning interview): - `TcpStats` gains a small lock around the filter. macOS still reports no rate. - Test the filter with controlled time: app-limited samples never lower the estimate, a real sample replaces expired maxima, and rejected samples retire nothing. - This quest lands on main. The `qmux-v0.5.x` backport is [its own quest](/quest/a1/qmux-rate-filter-0.5.md), which closes the issue, because published moq-relay (0.17.14) depends on `qmux ^0.5.3`. -- Other backends are audited separately in [the quiche audit](/quest/a2/quiche-delivery-rate.md). Quinn and Noq report the pacing rate or cwnd/rtt, which don't have this flaw. +- Quinn and Noq report the pacing rate or cwnd/rtt, which don't have this flaw. Decision prompts (✅ chosen): diff --git a/quest/a2/README.md b/quest/a2/README.md index dc5516cb..36e6b0b0 100644 --- a/quest/a2/README.md +++ b/quest/a2/README.md @@ -15,6 +15,5 @@ The Required list is ordered by priority. Independent children may proceed concu - [Native metric availability](/quest/a2/native-stats-upstream.md) - advance external accessor prerequisites - [QMux draft readiness](/quest/a2/qmux-draft-readiness.md) - establish strict-validation compatibility boundary - [Protocol buffering](/quest/a2/protocol-buffering.md) - measure and remove redundant copying -- [Quiche send-rate audit](/quest/a2/quiche-delivery-rate.md) - app-limited samples under legacy controllers - [Quinn provider-neutral rustls release](/quest/a2/quinn-provider-release.md) - waiting on quinn-rs/quinn#2341 - [Quinn adapter removal](/quest/a2/quinn-adapter-removal.md) - switch to upstream and delete the port diff --git a/quest/a2/quiche-delivery-rate.md b/quest/a2/quiche-delivery-rate.md deleted file mode 100644 index 86041506..00000000 --- a/quest/a2/quiche-delivery-rate.md +++ /dev/null @@ -1,13 +0,0 @@ -# [S] Audit quiche's send-rate estimate for app-limited samples - -## Goal - -web-transport-quiche's `estimated_send_rate` reports available bandwidth under every congestion controller it uses, or its docs say exactly where it doesn't. - -## Plan - -`ez/connection.rs` reports quiche's `PathStats::delivery_rate * 8` as-is. In quiche 0.30 that value is BBR2's filtered bandwidth estimate under gcongestion, but the latest raw rate sample under legacy CUBIC/Reno. That is the flaw [#460](https://github.com/moq-dev/web-transport/issues/460) found in qmux. - -Confirm which controllers web-transport-quiche configures by default and on request, and whether app-limited samples reach the reported value. If they do, prefer a filtered value quiche exposes publicly (such as `max_bandwidth` where it exists), or apply the same rule as [the qmux filter](/quest/a1/qmux-rate-filter.md) when quiche exposes the app-limited state. Otherwise document the limitation on the method. Use only quiche's public API, and don't fork its recovery code. - -Decided with the maintainer on 2026-11-06: a separate quest from the qmux fix, in a2. diff --git a/rs/web-transport-quiche/README.md b/rs/web-transport-quiche/README.md index 871922ce..506e7f1c 100644 --- a/rs/web-transport-quiche/README.md +++ b/rs/web-transport-quiche/README.md @@ -52,6 +52,45 @@ Call `Connection::close` before dropping to choose the close code and reason; final drop preserves an explicit close already in progress. Await `Connection::closed` when the application needs the driver to complete teardown. +## Send-rate statistics + +`Connection::stats()` returns `ez::ConnectionStats`; its `send_rate` field and +`web_transport_trait::Stats::estimated_send_rate()` expose the same estimate in +bits per second. The driver snapshots Quiche's public `PathStats::delivery_rate` +and converts bytes per second with saturating multiplication by eight. +`None` means the snapshot has no path; `Some(0)` is possible before a usable +sample. The estimate is not a guarantee of link capacity or application throughput, +and it can remain stale while the application sends too little to measure capacity. + +With tokio-quiche 0.20 and Quiche 0.30, both client and server use the +[default `cubic` controller][cc-default]. Pass `Settings` to either builder's +`with_settings` method to select another `cc_algorithm`: + +| `Settings::cc_algorithm` | Meaning of the reported estimate | +| --- | --- | +| `cubic` (default), `reno` | Latest accepted delivery-rate sample. A sample marked app-limited may raise the rate but cannot lower it. There is no windowed maximum or expiry of the retained estimate. | +| `bbr`, `bbr2`, `bbr2_gcongestion` | All select BBR2. Its maximum-bandwidth filter rejects lower app-limited samples; the reported estimate is also capped by BBR2's congestion-derived `bandwidth_lo` when present. | + +Quiche's [controller selector][cc-selector] accepts those five names and rejects +others. The [CUBIC/Reno sampler][legacy-rate] already checks the app-limited flag +before replacing its stored rate, so this value is not an unfiltered raw sample. +Its per-sample flag is internal, and `PathStats::max_bandwidth` is `None` for +these controllers. The public statistics therefore cannot support a separate +application-side windowed filter that distinguishes app-limited samples. + +For BBR2, [the network model][bbr-model] reports +`min(max_bandwidth, bandwidth_lo)` when the congestion bound exists, and +`max_bandwidth` otherwise. Using the public `PathStats::max_bandwidth` instead +would discard that bound. Keeping `delivery_rate` preserves both upstream sample +filtering and the controller's response to congestion. Quiche also requires +respecting pacing hints for accurate delivery-rate estimates; tokio-quiche drives +packet transmission here. + +[cc-default]: https://github.com/cloudflare/quiche/blob/925b225825e2188f8e58e359114150500557bf61/tokio-quiche/src/settings/quic.rs +[cc-selector]: https://github.com/cloudflare/quiche/blob/be47c5011215b9f13bad06bd7627d3ae49888a19/quiche/src/recovery/mod.rs +[legacy-rate]: https://github.com/cloudflare/quiche/blob/be47c5011215b9f13bad06bd7627d3ae49888a19/quiche/src/recovery/congestion/delivery_rate.rs +[bbr-model]: https://github.com/cloudflare/quiche/blob/be47c5011215b9f13bad06bd7627d3ae49888a19/quiche/src/recovery/gcongestion/bbr2/network_model.rs + ## Testing Run `cargo test -p web-transport-quiche` for the unit tests and socket integration diff --git a/rs/web-transport-quiche/src/connection.rs b/rs/web-transport-quiche/src/connection.rs index a3922a39..ba537a4d 100644 --- a/rs/web-transport-quiche/src/connection.rs +++ b/rs/web-transport-quiche/src/connection.rs @@ -507,6 +507,7 @@ impl web_transport_trait::Stats for ez::ConnectionStats { self.rtt } + /// Returns [`ez::ConnectionStats::send_rate`], with its controller-specific limits. fn estimated_send_rate(&self) -> Option { self.send_rate } diff --git a/rs/web-transport-quiche/src/error.rs b/rs/web-transport-quiche/src/error.rs index 8e7a6be5..1ffb266e 100644 --- a/rs/web-transport-quiche/src/error.rs +++ b/rs/web-transport-quiche/src/error.rs @@ -2,7 +2,7 @@ use web_transport_proto::error_from_http3; use crate::ez; -/// An error returned by [Connection], split based on whether they are underlying QUIC errors or WebTransport errors. +/// An error returned by [Connection](crate::Connection), split based on whether they are underlying QUIC errors or WebTransport errors. #[derive(Clone, thiserror::Error, Debug)] pub enum SessionError { #[error("remote closed: code={0} reason={1}")] diff --git a/rs/web-transport-quiche/src/ez/connection.rs b/rs/web-transport-quiche/src/ez/connection.rs index 19ea369f..9706e08b 100644 --- a/rs/web-transport-quiche/src/ez/connection.rs +++ b/rs/web-transport-quiche/src/ez/connection.rs @@ -38,8 +38,17 @@ pub struct ConnectionStats { pub packets_lost: u64, /// Smoothed round-trip time for the active path, if one is established. pub rtt: Option, - /// Estimated send rate in bits per second (from the congestion controller), - /// if a path is established. + /// Estimated available send bandwidth in bits per second, if a path exists. + /// + /// Uses Quiche 0.30's [`PathStats::delivery_rate`](quiche::PathStats::delivery_rate). + /// BBR2 reports its filtered bandwidth estimate, capped by its congestion + /// bound. CUBIC (the default) and Reno report the most recent accepted rate + /// sample: a lower app-limited sample cannot replace the previous rate, + /// but there is no windowed maximum or expiry for stale estimates. + /// + /// This is not a guarantee of link capacity or application throughput. + /// Estimates can remain stale while the sender is app-limited. `Some(0)` + /// is possible before a usable sample; `None` means the snapshot has no path. pub send_rate: Option, } @@ -57,7 +66,9 @@ impl ConnectionStats { packets_received: stats.recv as u64, packets_lost: stats.lost as u64, rtt: path.as_ref().map(|p| p.rtt), - // quiche reports the delivery rate in bytes/sec; the trait wants bits/sec. + // Both recovery implementations filter app-limited samples upstream. + // BBR2's delivery_rate also honors its congestion bound; max_bandwidth does not. + // Quiche reports bytes/sec; the trait wants bits/sec. send_rate: path.as_ref().map(|p| p.delivery_rate.saturating_mul(8)), } } From 89f94ad14e5561de6235d41783d2def589aba1d1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 05:11:25 -0800 Subject: [PATCH 3/4] docs(quiche): clarify rate audit limits after review Co-Authored-By: GPT-6.1 Sol --- rs/web-transport-quiche/README.md | 16 +++++++++------- rs/web-transport-quiche/src/ez/connection.rs | 5 ++--- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/rs/web-transport-quiche/README.md b/rs/web-transport-quiche/README.md index 506e7f1c..3a26566b 100644 --- a/rs/web-transport-quiche/README.md +++ b/rs/web-transport-quiche/README.md @@ -58,12 +58,12 @@ final drop preserves an explicit close already in progress. Await `web_transport_trait::Stats::estimated_send_rate()` expose the same estimate in bits per second. The driver snapshots Quiche's public `PathStats::delivery_rate` and converts bytes per second with saturating multiplication by eight. -`None` means the snapshot has no path; `Some(0)` is possible before a usable -sample. The estimate is not a guarantee of link capacity or application throughput, +`None` means no path statistics have been sampled yet; `Some(0)` is possible +before a usable sample. The estimate is not a guarantee of link capacity or application throughput, and it can remain stale while the application sends too little to measure capacity. -With tokio-quiche 0.20 and Quiche 0.30, both client and server use the -[default `cubic` controller][cc-default]. Pass `Settings` to either builder's +For the audited tokio-quiche 0.20.1 and Quiche 0.30.0 releases, both client and +server use the [default `cubic` controller][cc-default]. Pass `Settings` to either builder's `with_settings` method to select another `cc_algorithm`: | `Settings::cc_algorithm` | Meaning of the reported estimate | @@ -77,14 +77,16 @@ before replacing its stored rate, so this value is not an unfiltered raw sample. Its per-sample flag is internal, and `PathStats::max_bandwidth` is `None` for these controllers. The public statistics therefore cannot support a separate application-side windowed filter that distinguishes app-limited samples. +Deriving `cwnd / rtt` would give a send-window heuristic rather than a measured +delivery rate, and would not recover that missing flag. For BBR2, [the network model][bbr-model] reports `min(max_bandwidth, bandwidth_lo)` when the congestion bound exists, and `max_bandwidth` otherwise. Using the public `PathStats::max_bandwidth` instead would discard that bound. Keeping `delivery_rate` preserves both upstream sample -filtering and the controller's response to congestion. Quiche also requires -respecting pacing hints for accurate delivery-rate estimates; tokio-quiche drives -packet transmission here. +filtering and the controller's response to congestion. Delivery-rate accuracy +also depends on respecting Quiche's pacing hints: disabling pacing or ignoring +those hints in the transport or platform can distort the estimate. [cc-default]: https://github.com/cloudflare/quiche/blob/925b225825e2188f8e58e359114150500557bf61/tokio-quiche/src/settings/quic.rs [cc-selector]: https://github.com/cloudflare/quiche/blob/be47c5011215b9f13bad06bd7627d3ae49888a19/quiche/src/recovery/mod.rs diff --git a/rs/web-transport-quiche/src/ez/connection.rs b/rs/web-transport-quiche/src/ez/connection.rs index 9706e08b..2205e48e 100644 --- a/rs/web-transport-quiche/src/ez/connection.rs +++ b/rs/web-transport-quiche/src/ez/connection.rs @@ -48,7 +48,8 @@ pub struct ConnectionStats { /// /// This is not a guarantee of link capacity or application throughput. /// Estimates can remain stale while the sender is app-limited. `Some(0)` - /// is possible before a usable sample; `None` means the snapshot has no path. + /// is possible before a usable sample; `None` means no path statistics have + /// been sampled yet. pub send_rate: Option, } @@ -66,8 +67,6 @@ impl ConnectionStats { packets_received: stats.recv as u64, packets_lost: stats.lost as u64, rtt: path.as_ref().map(|p| p.rtt), - // Both recovery implementations filter app-limited samples upstream. - // BBR2's delivery_rate also honors its congestion bound; max_bandwidth does not. // Quiche reports bytes/sec; the trait wants bits/sec. send_rate: path.as_ref().map(|p| p.delivery_rate.saturating_mul(8)), } From ff897f28d04f28ab5044e287138fbfa02108d255 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 8 Nov 2026 09:05:06 -0800 Subject: [PATCH 4/4] docs(quiche): warn that client settings default to no verification Co-Authored-By: Claude Opus 5.5 --- rs/web-transport-quiche/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rs/web-transport-quiche/README.md b/rs/web-transport-quiche/README.md index 3a26566b..19da4bf3 100644 --- a/rs/web-transport-quiche/README.md +++ b/rs/web-transport-quiche/README.md @@ -71,6 +71,12 @@ server use the [default `cubic` controller][cc-default]. Pass `Settings` to eith | `cubic` (default), `reno` | Latest accepted delivery-rate sample. A sample marked app-limited may raise the rate but cannot lower it. There is no windowed maximum or expiry of the retained estimate. | | `bbr`, `bbr2`, `bbr2_gcongestion` | All select BBR2. Its maximum-bandwidth filter rejects lower app-limited samples; the reported estimate is also capped by BBR2's congestion-derived `bandwidth_lo` when present. | +The client builder's `with_settings` replaces its defaults, and +`Settings::default()` leaves `verify_peer` false. When starting a client from +`Settings::default()`, set `verify_peer = true` or configure +`with_root_certificates` or `with_server_certificate_hashes`; otherwise the +client skips certificate verification. + Quiche's [controller selector][cc-selector] accepts those five names and rejects others. The [CUBIC/Reno sampler][legacy-rate] already checks the app-limited flag before replacing its stored rate, so this value is not an unfiltered raw sample.