diff --git a/quest/m2/quic-realtime/README.md b/quest/m2/quic-realtime/README.md index 8335b87c102..27f22f94d06 100644 --- a/quest/m2/quic-realtime/README.md +++ b/quest/m2/quic-realtime/README.md @@ -56,6 +56,17 @@ spike assumed no consumer; teleoperation uplinks are one): upgrading `Delay`, so current users see no behavior change. - Docs: inline only. The congestion-control lines in `doc/bin/cli.md` and `doc/lib/*` gain the new variant; no new page. +- SCReAMv2 landed as `moq_congestion::Scream` + ([moq-dev/moq#7547](https://github.com/moq-dev/moq/pull/7547)). moq-quic's + `tests::scream` measures it on the sim beside BBR3, and moq-tokio's + `scream` integration tests over the real presets. Its estimate is a window + over the RTT rather than a delivery rate: under application-limited media, + each keyframe's burst raises it past the path's capacity, which matters + for re-enabling a rung. Its frame p95 on the sim rose under #7606's + corrected pacer, from 853 to 1327 ms on `cellular-step` and from 300 to + 1361 ms on `cellular-deep`: its keyframes (5x a delta frame) no longer + overshoot its pacing of 1.5x the target rate, and the draft's media-queue + terms that would answer that sender backlog were left out. The shared pacer's low-rate correction and BBR3's one-datagram minimum send quantum are implemented together in [moq-dev/moq#7606](https://github.com/moq-dev/moq/pull/7606). @@ -117,7 +128,6 @@ owns that feedback. ## Required - [BBR3 delay bound](/quest/m2/quic-realtime/bbr-delay-bound.md) - BBR3 with inflight capped when the forward delay gradient shows queue growth -- [SCReAMv2 candidate](/quest/m2/quic-realtime/scream.md) - the RFC 8298bis delay path ported to `moq-congestion` - [io_uring runs an application's controller](/quest/m2/quic-realtime/uring-controller.md) - moq-uring takes a controller factory, so the winner installs the same way on both runtimes - [moqsink encoder feedback](/quest/m2/quic-realtime/moqsink-feedback.md) - GStreamer publishers retune their encoder to the target rate diff --git a/quest/m2/quic-realtime/scream.md b/quest/m2/quic-realtime/scream.md deleted file mode 100644 index d5802af2e97..00000000000 --- a/quest/m2/quic-realtime/scream.md +++ /dev/null @@ -1,25 +0,0 @@ -# [L] SCReAMv2 candidate - -## Goal - -The SCReAMv2 (RFC 8298bis) delay-based path runs as a -controller in `moq-congestion`, fed by receive timestamps, and reports -its queuing delay, throughput, and reaction time on the shared moq-shaper -uplink presets. - -## Plan - -No Rust implementation exists; port from the specification, using Ericsson's -BSD-licensed reference (EricssonResearch/scream) to check behavior. - -- Port only the queue-delay target path. The L4S/ECN path waits on - [the ECN measurement](/quest/m2/quic-ecn-measure.md): an ECN path between - publishers and relays is unproven, and the shaper cannot mark. -- SCReAM's queue delay comes from one-way delay samples; use the receive-ts - forward delay with its basis offset removed (the spike's estimator), not - absolute time. -- Implement `Controller::probes` and report `probe_inflight` so moq-quic's - native probing still raises the estimate (decided 2026-10-25, maintainer; - see the [epic](/quest/m2/quic-realtime/README.md)). Measure that a stopped - rung recovers under it. -- Unit tests drive the controller with mocked time, not wall clock. diff --git a/rs/justfile b/rs/justfile index 5c6740d5aeb..3ab3fcd9517 100644 --- a/rs/justfile +++ b/rs/justfile @@ -329,7 +329,7 @@ doctest *args: miri *args: bash sh/rs/miri.sh "$@" -# Run the moq-quic simulation's random workloads from a fresh seed, printing the settings that replay it, then the bandwidth probe's acceptance suite and the receive-timestamps comparison. +# Run the moq-quic simulation's random workloads from a fresh seed, printing the settings that replay it, then the bandwidth probe's acceptance suite, the receive-timestamps comparison, and the realtime controllers' comparison. quic-sim: #!/usr/bin/env bash set -euo pipefail @@ -341,6 +341,9 @@ quic-sim: bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-quic --lib -E 'test(/^tests::sim::/)' # The bandwidth probe's acceptance suite, over fixed seeds: the merge gate runs only its short cases. bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-quic --lib --run-ignored only -E 'test(/^tests::probe::/)' + # SCReAMv2's queue, throughput and reaction on the cellular presets: on the sim beside BBR3, then over real sockets through moq-shaper, one preset at a time. + bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-quic --lib --run-ignored only --no-capture -E 'test(/^tests::scream::/)' + bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-tokio --test integration --run-ignored only --no-capture --test-threads 1 -E 'test(/^scream::/)' # Forward delay against RTT as a queue-growth signal: deterministic in the simulation, then over real sockets through each cellular moq-shaper preset, one at a time so they do not add to each other's noise. bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-quic --lib --run-ignored only --no-capture -E 'test(/^tests::receive_ts::/)' bash sh/rs/nextest.sh cargo nextest run --locked --release -p moq-tokio --test integration --run-ignored only --no-capture --test-threads 1 -E 'test(/^receive_ts::/)' diff --git a/rs/moq-congestion/README.md b/rs/moq-congestion/README.md index 5eed580d649..013502f48c5 100644 --- a/rs/moq-congestion/README.md +++ b/rs/moq-congestion/README.md @@ -1,14 +1,14 @@ # moq-congestion -BBR3, Cubic, GCC, and Unlimited congestion controllers, consumed by moq-quic and the native transport drivers. +BBR3, Cubic, GCC, SCReAMv2, and Unlimited congestion controllers, consumed by moq-quic and the native transport drivers. The controllers take explicit `kio::clock::Instant` values and never drive a runtime. The `Controller` trait names no transport types: `on_sent` returns an opaque per-packet state that the transport stores and hands back with each acknowledgement or loss, and the controller paces its own sends. -Construct controllers with `Factory::build` on an `Arc`, `Arc`, `Arc`, or `Arc`. `Bbr3`, `Cubic`, `Gcc`, `Unlimited`, and `Metrics` live at the crate root. +Construct controllers with `Factory::build` on an `Arc`, `Arc`, `Arc`, `Arc`, or `Arc`. `Bbr3`, `Cubic`, `Gcc`, `Scream`, `Unlimited`, and `Metrics` live at the crate root. For callers of the unpublished API, replace `Bbr3Config` with `bbr3::Config`, `CubicConfig` with `cubic::Config`, `ControllerFactory` with `Factory`, and `ControllerMetrics` with `Metrics`. Remove `Controller::into_any` from custom implementations; inspect the controller's behavior or metrics instead of downcasting. `clone_box` still copies controller state for path migration. The test-only `Bbr3::probing_rtt` and `min_rtt` accessors are removed; model internals are tested in this crate. -The code was extracted from [moq-quic](../moq-quic), under the same MIT or Apache-2.0 license. Cubic originated in quinn; BBR3 was ported from moq-dev/noq and carries the fixes listed in [moq-quic's fork history](../moq-quic/README.md). +The code was extracted from [moq-quic](../moq-quic), under the same MIT or Apache-2.0 license. Cubic originated in quinn; BBR3 was ported from moq-dev/noq and carries the fixes listed in [moq-quic's fork history](../moq-quic/README.md). SCReAMv2's delay-based path was ported from draft-ietf-ccwg-rfc8298bis-screamv2 and checked against Ericsson's [reference implementation](https://github.com/EricssonResearch/scream). GCC is libwebrtc's Google Congestion Control, a delay-based controller that reads the peer's receive timestamps (draft-ietf-quic-receive-ts). `src/gcc` vendors a Rust port of WebRTC's `goog_cc` under WebRTC's BSD license, kept in [`src/gcc/LICENSE`](src/gcc/LICENSE) with its `PATENTS` grant; its module docs record the WebRTC revision it tracks and what it trims. diff --git a/rs/moq-congestion/src/lib.rs b/rs/moq-congestion/src/lib.rs index f40a557e2f0..f41ef308b4e 100644 --- a/rs/moq-congestion/src/lib.rs +++ b/rs/moq-congestion/src/lib.rs @@ -17,12 +17,14 @@ pub mod cubic; pub mod gcc; mod pacer; mod rate; +pub mod scream; mod unlimited; pub use bbr3::Bbr3; pub use cubic::Cubic; pub use gcc::Gcc; pub use rate::Packet; +pub use scream::Scream; pub use unlimited::Unlimited; /// Common interface for different congestion controllers @@ -283,11 +285,12 @@ mod tests { /// The largest UDP payload (RFC 9000), whose two datagrams outgrow every 1200-based default const MTU: u16 = 65527; - fn factories() -> [(&'static str, Arc); 4] { + fn factories() -> [(&'static str, Arc); 5] { [ ("bbr3", Arc::new(bbr3::Config::default())), ("cubic", Arc::new(cubic::Config::default())), ("gcc", Arc::new(gcc::Config::default())), + ("scream", Arc::new(scream::Config::default())), ("unlimited", Arc::new(Unlimited)), ] } diff --git a/rs/moq-congestion/src/scream.rs b/rs/moq-congestion/src/scream.rs new file mode 100644 index 00000000000..7a2b5a34640 --- /dev/null +++ b/rs/moq-congestion/src/scream.rs @@ -0,0 +1,921 @@ +//! SCReAMv2's delay-based congestion control, ported from draft-ietf-ccwg-rfc8298bis-screamv2-01. +//! +//! +//! A reference window caps the bytes in flight. It grows with every byte acknowledged, and shrinks +//! on loss, on classic ECN-CE, and as the averaged queue delay passes half the target, in +//! proportion to how far past it is ("virtual L4S" marking). The queue delay is each packet's +//! one-way delay above the lowest seen recently, from the peer's receive timestamps +//! (draft-ietf-quic-receive-ts), so ACK-path delay never reads as a queue and the peer's unknown +//! clock basis cancels out. +//! +//! Only the queue-delay target path is ported. The L4S path waits on ECN being measured between +//! publishers and relays, and the competing-flow compensation that raises the target is left out: +//! the controller keeps the queue near empty and accepts a smaller share beside loss-based flows. +//! Checked against Ericsson's BSD-licensed reference, `ScreamV2Tx.cpp` in +//! . + +use std::sync::Arc; + +use crate::pacer::{Pacer, Rate}; +use crate::{Ack, Ce, Controller, Factory, Init, Loss, Metrics, Packet, Sent}; +use crate::{Duration, Instant}; + +/// equivalent to MIN_REF_WND: the smallest reference window, in bytes +const MIN_REF_WND: f64 = 3000.0; + +/// equivalent to BYTES_IN_FLIGHT_HEAD_ROOM: how far past the most bytes recently in flight the +/// reference window may grow, so an application-limited sender's window tracks what it sends +const BYTES_IN_FLIGHT_HEAD_ROOM: f64 = 1.5; + +/// equivalent to BETA_LOSS: the reference window's scale on a loss event +const BETA_LOSS: f64 = 0.7; + +/// equivalent to BETA_ECN: the reference window's scale on a classic ECN-CE event +const BETA_ECN: f64 = 0.8; + +/// equivalent to POST_CONGESTION_DELAY_RTT: how many round trips after congestion the window +/// grows only additively +const POST_CONGESTION_DELAY_RTTS: f64 = 100.0; + +/// equivalent to MUL_INCREASE_FACTOR: the multiplicative part of the window's growth per round +/// trip, as a fraction of the window +const MUL_INCREASE_FACTOR: f64 = 0.02; + +/// equivalent to VIRTUAL_RTT: the shortest interval between reactions to congestion, and the +/// round trip below which the window grows more slowly +const VIRTUAL_RTT: Duration = Duration::from_millis(25); + +/// equivalent to QDELAY_AVG_G: the gain of the slow-attack, fast-decay queue delay average +const QDELAY_AVG_G: f64 = 1.0 / 4.0; + +/// equivalent to QDELAY_MIN_MAX_AVG_G: the gain pulling the queue delay's tracked maximum down and +/// its minimum up, with [`Config::reduce_jitter`] +const QDELAY_MIN_MAX_AVG_G: f64 = 1.0 / 16.0; + +/// equivalent to QDELAY_DEV_AVG_G: the gain of the queue delay's averaged spread +const QDELAY_DEV_AVG_G: f64 = 1.0 / 32.0; + +/// equivalent to QDELAY_DEV_THRESHOLD: the spread at which jitter stops the window's growth +const QDELAY_DEV_THRESHOLD: Duration = Duration::from_millis(10); + +/// equivalent to REF_WND_OVERHEAD_MIN: how far bytes in flight may exceed the reference window +/// once the queue delay varies widely +const REF_WND_OVERHEAD_MIN: f64 = 1.5; + +/// equivalent to REF_WND_OVERHEAD_MAX: how far bytes in flight may exceed the reference window on +/// a steady path, so a large frame leaves without waiting on the window +const REF_WND_OVERHEAD_MAX: f64 = 3.0; + +/// equivalent to RATE_PACE_MIN: the lowest pacing rate, 50 kbit/s, in bytes per second +const RATE_PACE_MIN: f64 = 50_000.0 / 8.0; + +/// equivalent to PACKET_PACING_HEADROOM: how far past the target bitrate packets are paced +const PACKET_PACING_HEADROOM: f64 = 1.5; + +/// equivalent to PACKET_OVERHEAD: bytes of packetization each packet adds to the media +const PACKET_OVERHEAD: f64 = 20.0; + +/// How long one window of the base delay's history spans, as LEDBAT's minute-long windows do +/// over its ten-minute history, shortened as the reference implementation's are. +/// +const BASE_INTERVAL: Duration = Duration::from_secs(10); + +/// How many [`BASE_INTERVAL`] windows the base delay is the lowest of +/// +/// The receiver's clock drifts against the sender's by tens of parts per million, which a base +/// delay held too long reads as a queue that never drains. A hundred seconds holds that to a few +/// milliseconds, against the target's tens. +const BASE_HISTORY: usize = 10; + +/// Configuration for the [`Scream`] congestion controller +#[derive(Debug, Clone)] +pub struct Config { + qdelay_target: Duration, + reduce_jitter: bool, +} + +impl Config { + /// The queue delay the controller holds below: it backs off from half of it, harder the + /// closer the averaged delay comes to it. + /// + /// Equivalent to QDELAY_TARGET_LO. Defaults to 60ms. + pub fn qdelay_target(&mut self, value: Duration) -> &mut Self { + self.qdelay_target = value; + self + } + + /// Whether the window grows more slowly and runs closer to the reference window while the + /// queue delay varies, as on a path with scheduling jitter. + /// + /// Equivalent to REDUCE_JITTER. Defaults to off. + pub fn reduce_jitter(&mut self, value: bool) -> &mut Self { + self.reduce_jitter = value; + self + } +} + +impl Default for Config { + fn default() -> Self { + Self { + qdelay_target: Duration::from_millis(60), + reduce_jitter: false, + } + } +} + +impl Factory for Config { + fn build(self: Arc, init: Init) -> Box { + Box::new(Scream::new(self, init)) + } +} + +/// SCReAMv2's delay-based congestion control, which keeps the bottleneck queue near empty +/// +/// Needs the peer's receive timestamps: without them it sees no queue and backs off only on loss. +#[derive(Debug, Clone)] +pub struct Scream { + config: Arc, + /// equivalent to MSS: the path's datagram size + mss: f64, + /// When the controller was built, which every packet's send time is counted from, as the + /// peer counts its receive times from a basis of its own + epoch: Instant, + /// equivalent to s_rtt: the transport's smoothed RTT + srtt: Duration, + + /// equivalent to ref_wnd, in bytes + ref_wnd: f64, + /// equivalent to ref_wnd_i: the reference window when congestion last began, in bytes + ref_wnd_i: f64, + /// equivalent to ref_wnd_i_update_allowed + ref_wnd_i_update_allowed: bool, + /// equivalent to bytes_newly_acked + bytes_newly_acked: u64, + /// equivalent to max_bytes_in_flight: the most in flight this round trip + max_bytes_in_flight: u64, + /// equivalent to max_bytes_in_flight_prev: the most in flight the round trip before + max_bytes_in_flight_prev: u64, + /// When the current round trip of [`Self::max_bytes_in_flight`] began + round_start: Instant, + /// When the window was last updated + last_update: Instant, + + /// The lowest forward delay recently seen, which the queue delay is measured above + base: BaseDelay, + /// equivalent to qdelay: the latest queue delay sample + qdelay: Duration, + /// equivalent to qdelay_avg + qdelay_avg: Duration, + /// equivalent to last_update_qdelay_avg_time + last_qdelay_avg_update: Instant, + /// equivalent to qdelay_max_avg, with [`Config::reduce_jitter`] + qdelay_max_avg: Duration, + /// equivalent to qdelay_min_avg, with [`Config::reduce_jitter`] + qdelay_min_avg: Duration, + /// equivalent to qdelay_dev_avg, with [`Config::reduce_jitter`] + qdelay_dev_avg: Duration, + /// equivalent to ref_wnd_delay_scale: 1 on a steady path, falling to 0 as the queue delay + /// varies by [`QDELAY_DEV_THRESHOLD`] + ref_wnd_delay_scale: f64, + + /// Loss of a packet sent since the last reaction, not yet reacted to + loss_detected: bool, + /// ECN-CE on a packet sent since the last reaction, not yet reacted to + ce_detected: bool, + /// equivalent to last_congestion_detected_time + last_congestion_detected: Option, + /// equivalent to last_reaction_to_congestion_time + last_reaction: Option, + + /// equivalent to target_bitrate, in bytes per second + target_rate: f64, + /// Whether an application-limited sender may resend in-flight data to raise the window + probing: bool, + pacer: Pacer, +} + +impl Scream { + fn new(config: Arc, init: Init) -> Self { + let mut scream = Self { + mss: f64::from(init.mtu), + epoch: init.now, + srtt: init.rtt, + ref_wnd: MIN_REF_WND, + ref_wnd_i: 1.0, + ref_wnd_i_update_allowed: true, + bytes_newly_acked: 0, + max_bytes_in_flight: 0, + max_bytes_in_flight_prev: 0, + round_start: init.now, + last_update: init.now, + base: BaseDelay::new(init.now), + qdelay: Duration::ZERO, + qdelay_avg: Duration::ZERO, + last_qdelay_avg_update: init.now, + qdelay_max_avg: config.qdelay_target, + qdelay_min_avg: Duration::ZERO, + qdelay_dev_avg: Duration::ZERO, + ref_wnd_delay_scale: 1.0, + loss_detected: false, + ce_detected: false, + last_congestion_detected: None, + last_reaction: None, + target_rate: 0.0, + probing: false, + pacer: Pacer::new( + init.rtt, + (MIN_REF_WND as u64).max(2 * u64::from(init.mtu)), + init.mtu, + init.max_pacing_rate, + init.now, + ), + config, + }; + scream.update_target_rate(); + scream + } + + /// The bytes per second the media should be encoded at + fn target_rate(&self) -> u64 { + self.target_rate as u64 + } + + /// equivalent to the Media Rate Control's target_bitrate, without the media queue terms a + /// transport cannot see. + /// + fn update_target_rate(&mut self) { + let ratio = (self.mss / self.ref_wnd).min(1.0); + let mut scale = 1.0 - (ratio - 0.1).clamp(0.0, 0.2); + scale *= self.mss / (self.mss + PACKET_OVERHEAD); + scale /= 1.2; + self.target_rate = scale * self.ref_wnd / self.srtt.as_secs_f64().max(1e-6); + } + + /// The pacing rate, in bytes per second. + /// + fn pacing_rate(&self) -> u64 { + (self.target_rate.max(RATE_PACE_MIN) * PACKET_PACING_HEADROOM) as u64 + } + + /// The forward delay of a packet sent at `sent` and received at `received` on the peer's + /// clock, in nanoseconds, offset by the peer's unknown basis + fn forward_delay(&self, sent: Instant, received: Duration) -> i64 { + let sent = sent.saturating_duration_since(self.epoch); + nanos(received) - nanos(sent) + } + + /// equivalent to the qdelay_avg and ref_wnd_delay_scale updates, once per + /// min(VIRTUAL_RTT, s_rtt). + /// + fn update_qdelay_avg(&mut self, now: Instant) { + if now.saturating_duration_since(self.last_qdelay_avg_update) < self.interval() { + return; + } + self.last_qdelay_avg_update = now; + self.qdelay_avg = match self.qdelay < self.qdelay_avg { + true => self.qdelay, + false => self + .qdelay + .mul_f64(QDELAY_AVG_G) + .saturating_add(self.qdelay_avg.mul_f64(1.0 - QDELAY_AVG_G)), + }; + + if self.config.reduce_jitter { + self.qdelay_max_avg = self.qdelay_max_avg.mul_f64(1.0 - QDELAY_MIN_MAX_AVG_G); + self.qdelay_min_avg = self + .qdelay_min_avg + .mul_f64(1.0 - QDELAY_MIN_MAX_AVG_G) + .saturating_add(self.qdelay_max_avg.mul_f64(QDELAY_MIN_MAX_AVG_G)); + let spread = self.qdelay_max_avg.saturating_sub(self.qdelay_min_avg); + self.qdelay_dev_avg = self + .qdelay_dev_avg + .mul_f64(1.0 - QDELAY_DEV_AVG_G) + .saturating_add(spread.mul_f64(QDELAY_DEV_AVG_G)); + self.ref_wnd_delay_scale = (1.0 + - self.qdelay_dev_avg.as_secs_f64() / QDELAY_DEV_THRESHOLD.as_secs_f64()) + .clamp(0.0, 1.0); + } + } + + /// The shortest interval between reactions and window updates: min(VIRTUAL_RTT, s_rtt) + fn interval(&self) -> Duration { + Ord::min(VIRTUAL_RTT, self.srtt) + } + + /// Updates the reference window if one is due, or at once on loss or ECN-CE: the reduction, + /// then the increase, then the target rate. + /// + fn update(&mut self, now: Instant) { + let congested = self.loss_detected || self.ce_detected; + if !congested && now.saturating_duration_since(self.last_update) < self.interval() { + return; + } + self.last_update = now; + self.reduce(now); + self.increase(now); + self.update_target_rate(); + self.bytes_newly_acked = 0; + } + + /// equivalent to scl_t: 1 away from the window congestion last began at, falling to 0.1 near + /// it + /// + /// Taken after the update's reduction, as the draft's comment has it ("ref_wnd_i is updated + /// before this code"), though its pseudocode and the reference implementation take it + /// before. A reaction makes the window it began at the inflection, so the increase right + /// after a small backoff is held near it, as the draft intends, rather than measured against + /// an older inflection the window grew past. Taken before, the queue's p95 after + /// `follows_a_capacity_drop`'s drop rose from 58ms to 62ms, past the 60ms target. + fn scl(&self) -> f64 { + let scl = (self.ref_wnd - self.ref_wnd_i) / self.ref_wnd_i * 8.0; + (scl * scl).clamp(0.1, 1.0) + } + + /// equivalent to the Reference Window Reduction. + /// + fn reduce(&mut self, now: Instant) { + if self.loss_detected || self.ce_detected { + self.last_congestion_detected = Some(now); + } + let due = self + .last_reaction + .is_none_or(|last| now.saturating_duration_since(last) >= self.interval()); + if !due { + return; + } + + let target = self.config.qdelay_target.as_secs_f64(); + let qdelay_avg = self.qdelay_avg.as_secs_f64(); + let scale = if self.loss_detected { + BETA_LOSS + } else if self.ce_detected { + BETA_ECN + } else if qdelay_avg > target / 2.0 { + // Averaged, so a delay spike from a link-layer retransmission or a handover is not + // read as a queue + let alpha = ((qdelay_avg - target / 2.0) / (target / 2.0)).clamp(0.0, 1.0); + // Reactions come every min(VIRTUAL_RTT, s_rtt), so several a round trip on a long + // path: each takes its share + let backoff = + alpha / 2.0 / (self.srtt.as_secs_f64() / VIRTUAL_RTT.as_secs_f64()).max(1.0); + 1.0 - backoff + } else { + return; + }; + self.loss_detected = false; + self.ce_detected = false; + // A queue past half the target is congestion too, which the window's multiplicative + // growth waits out, as the reference implementation's virtual CE events do + self.last_congestion_detected = Some(now); + + if self.ref_wnd_i_update_allowed { + self.ref_wnd_i = self.ref_wnd; + self.ref_wnd_i_update_allowed = false; + } + self.ref_wnd = (self.ref_wnd * scale).max(MIN_REF_WND); + self.last_reaction = Some(now); + } + + /// equivalent to the Reference Window Increase. + /// + fn increase(&mut self, now: Instant) { + let srtt = self.srtt.as_secs_f64(); + let virtual_rtt = VIRTUAL_RTT.as_secs_f64(); + let post_congestion = self.last_congestion_detected.map_or(1.0, |last| { + let since = now.saturating_duration_since(last).as_secs_f64(); + (since / (POST_CONGESTION_DELAY_RTTS * srtt.max(virtual_rtt))).clamp(0.0, 1.0) + }); + let scl = self.scl(); + + let mut increment = self.bytes_newly_acked as f64 * (self.mss / self.ref_wnd).min(1.0); + increment *= (srtt / virtual_rtt).min(1.0); + increment *= scl.max(0.25); + increment *= self.ref_wnd_delay_scale.max(0.1); + let multiplier = 1.0 + MUL_INCREASE_FACTOR * self.ref_wnd / self.mss; + increment *= 1.0 + (multiplier - 1.0) * post_congestion * scl; + + // Grows only while enough is in flight: an application-limited sender's window stays + // within reach of what it sends, so it has little to give back on congestion + let in_flight = Ord::max(self.max_bytes_in_flight, self.max_bytes_in_flight_prev) as f64; + let max_allowed = self.mss + in_flight * BYTES_IN_FLIGHT_HEAD_ROOM; + if self.ref_wnd + increment <= max_allowed && increment > 0.0 { + self.ref_wnd += increment; + self.ref_wnd_i_update_allowed = true; + } + } +} + +impl Controller for Scream { + fn on_sent(&mut self, sent: Sent) -> Packet { + self.pacer.on_transmit(sent.bytes); + self.max_bytes_in_flight = Ord::max(self.max_bytes_in_flight, sent.in_flight); + Packet::default() + } + + fn on_ack(&mut self, ack: Ack<'_>) { + let now = ack.now; + self.srtt = ack.rtt; + if now.saturating_duration_since(self.round_start) >= self.srtt { + self.max_bytes_in_flight_prev = self.max_bytes_in_flight; + self.max_bytes_in_flight = ack.in_flight; + self.round_start = now; + } + + // One sample per ACK: the least delayed of its packets, which a link that delivers in + // bursts, as a cellular scheduler's grants do, delayed least past its queue + let mut sample = None; + for acked in ack.packets { + self.bytes_newly_acked += u64::from(acked.bytes); + if let Some(received) = acked.received { + let delay = self.forward_delay(acked.sent, received); + self.base.update(now, delay); + sample = Some(Ord::min(sample.unwrap_or(i64::MAX), delay)); + } + } + if let Some(delay) = sample { + let qdelay = delay.saturating_sub(self.base.get()); + self.qdelay = Duration::from_nanos(qdelay.max(0) as u64); + if self.config.reduce_jitter { + self.qdelay_max_avg = Ord::min( + self.config.qdelay_target, + Ord::max(self.qdelay, self.qdelay_max_avg), + ); + self.qdelay_min_avg = Ord::min(self.qdelay, self.qdelay_min_avg); + } + } + self.update_qdelay_avg(now); + self.update(now); + + // An application-limited sender may fill in for the media it lacks while no queue is + // building, so the window grows to what the path carries + let target = self.config.qdelay_target; + self.probing = self.qdelay_avg < target / 4 + && self + .last_reaction + .is_none_or(|last| now.saturating_duration_since(last) > self.srtt); + } + + fn on_loss(&mut self, loss: Loss<'_>) { + if loss.persistent { + self.ref_wnd = MIN_REF_WND; + self.last_congestion_detected = Some(loss.now); + self.last_reaction = Some(loss.now); + self.loss_detected = false; + self.update_target_rate(); + return; + } + // A packet sent before the last reaction was lost to the congestion it already answered + let newest = loss.packets.iter().map(|lost| lost.sent).max(); + if newest.is_some_and(|sent| self.last_reaction.is_none_or(|last| sent > last)) { + self.loss_detected = true; + self.update(loss.now); + } + } + + // The draft's increase leaves out CE-marked bytes. [`Ce`] carries no count of them, so they + // count here: classic ECN grows the window additively after its reaction, at most an MSS a + // round trip, so they cost a fraction of an MSS against the 20% backoff. The L4S path, which + // grows through marks every round trip, would need the count. + fn on_ce(&mut self, ce: Ce) { + if self.last_reaction.is_none_or(|last| ce.sent > last) { + self.ce_detected = true; + self.update(ce.now); + } + } + + fn on_mtu_update(&mut self, new_mtu: u16) { + self.mss = f64::from(new_mtu); + } + + fn window(&self) -> u64 { + let overhead = REF_WND_OVERHEAD_MIN + + (REF_WND_OVERHEAD_MAX - REF_WND_OVERHEAD_MIN) * self.ref_wnd_delay_scale; + // At least two datagrams of the path: the draft's 3000-byte minimum, sized for 1200 to + // 1500-byte packets, would block every send of a larger one + ((self.ref_wnd * overhead) as u64).max(2 * self.mss as u64) + } + + fn pace(&mut self, now: Instant, bytes: u64) -> Option { + let rate = Rate { + window: self.window(), + pacing_rate: Some(self.pacing_rate()), + }; + self.pacer + .delay(self.srtt, bytes, self.mss as u16, now, rate) + } + + fn metrics(&self) -> Metrics { + Metrics { + congestion_window: self.window(), + ssthresh: None, + pacing_rate: Some(self.pacing_rate()), + bandwidth_estimate: Some(self.target_rate()), + probe_inflight: self.probing.then(|| self.window()), + } + } + + fn probes(&self) -> bool { + true + } + + fn clone_box(&self) -> Box { + Box::new(self.clone()) + } +} + +/// The lowest forward delay over the last [`BASE_HISTORY`] windows of [`BASE_INTERVAL`], as +/// LEDBAT keeps its base delay. +/// +#[derive(Debug, Clone)] +struct BaseDelay { + /// The lowest of each finished window, the newest last, in nanoseconds + history: [i64; BASE_HISTORY], + /// The lowest of the window in progress + current: i64, + /// When the window in progress began + since: Instant, +} + +impl BaseDelay { + fn new(now: Instant) -> Self { + Self { + history: [i64::MAX; BASE_HISTORY], + current: i64::MAX, + since: now, + } + } + + fn update(&mut self, now: Instant, delay: i64) { + // Every window that ended since the last sample, which a quiet spell leaves empty, so a + // minimum from before it ages out as it would have without the gap + let ended = now.saturating_duration_since(self.since).as_nanos() / BASE_INTERVAL.as_nanos(); + if ended > 0 { + match usize::try_from(ended) { + Ok(ended) if ended <= BASE_HISTORY => { + self.history.rotate_left(ended); + self.history[BASE_HISTORY - ended..].fill(i64::MAX); + self.history[BASE_HISTORY - ended] = self.current; + } + _ => self.history.fill(i64::MAX), + } + self.current = i64::MAX; + self.since = now; + } + self.current = Ord::min(self.current, delay); + } + + fn get(&self) -> i64 { + self.history.iter().copied().fold(self.current, Ord::min) + } +} + +/// A duration in nanoseconds, saturating past the 292 years an `i64` holds +fn nanos(duration: Duration) -> i64 { + i64::try_from(duration.as_nanos()).unwrap_or(i64::MAX) +} + +#[cfg(test)] +mod tests { + use std::collections::VecDeque; + + use super::*; + use crate::{Acked, Lost}; + + const MTU: u16 = 1200; + const PROPAGATION: Duration = Duration::from_millis(20); + const TICK: Duration = Duration::from_millis(1); + + fn scream(now: Instant, config: Config) -> Scream { + Scream::new( + Arc::new(config), + Init { + now, + mtu: MTU, + rtt: Duration::from_millis(333), + max_pacing_rate: None, + }, + ) + } + + /// A packet on its way + struct Flight { + sent: Instant, + /// When it reaches the peer + arrival: Instant, + /// When its ACK reaches the sender + acked: Instant, + } + + /// A sender driving a [`Scream`] up through a FIFO bottleneck, with an ACK for every packet + /// coming straight back, on a mocked clock + struct Path { + scream: Scream, + now: Instant, + /// The bottleneck's rate in bytes per second + rate: f64, + /// When the bottleneck finishes serving what it holds + busy_until: Instant, + /// The peer's clock basis, which its receive times count from + basis: Instant, + /// How much faster the peer's clock runs, in parts per million + drift_ppm: f64, + /// The media's rate in bytes per second, or `None` for an encoder that follows the + /// controller's estimate + media: Option, + /// Media bytes written and not yet sent + backlog: f64, + flights: VecDeque, + in_flight: u64, + srtt: Option, + /// Each packet's queueing delay at the bottleneck + queue: Vec, + delivered: u64, + } + + impl Path { + fn new(rate: f64, media: Option, config: Config) -> Self { + let now = Instant::now(); + Self { + scream: scream(now, config), + now, + rate, + busy_until: now, + basis: now - Duration::from_secs(1), + drift_ppm: 0.0, + media, + backlog: 0.0, + flights: VecDeque::new(), + in_flight: 0, + srtt: None, + queue: Vec::new(), + delivered: 0, + } + } + + /// Runs for `duration`, returning the bytes delivered and each packet's queueing delay + fn run(&mut self, duration: Duration) -> (u64, Vec) { + let (delivered, queued) = (self.delivered, self.queue.len()); + let end = self.now + duration; + while self.now < end { + self.now += TICK; + self.deliver(); + self.send(); + } + (self.delivered - delivered, self.queue[queued..].to_vec()) + } + + fn send(&mut self) { + let media = self.media.unwrap_or(self.scream.target_rate); + self.backlog += media * TICK.as_secs_f64(); + let bytes = u64::from(MTU); + while self.backlog >= bytes as f64 + && self.in_flight + bytes <= self.scream.window() + && self.scream.pace(self.now, bytes).is_none() + { + self.backlog -= bytes as f64; + self.in_flight += bytes; + self.scream.on_sent(Sent { + now: self.now, + bytes: MTU, + in_flight: self.in_flight, + }); + let start = Ord::max(self.busy_until, self.now); + self.queue.push(start - self.now); + self.busy_until = start + Duration::from_secs_f64(bytes as f64 / self.rate); + let arrival = self.busy_until + PROPAGATION; + self.flights.push_back(Flight { + sent: self.now, + arrival, + acked: arrival + PROPAGATION, + }); + } + } + + fn deliver(&mut self) { + let mut acked = Vec::new(); + while self.flights.front().is_some_and(|f| f.acked <= self.now) { + let flight = self.flights.pop_front().unwrap(); + let elapsed = flight.arrival - self.basis; + acked.push(Acked { + sent: flight.sent, + bytes: MTU, + packet: Packet::default(), + received: Some(elapsed.mul_f64(1.0 + self.drift_ppm / 1e6)), + }); + let rtt = self.now - flight.sent; + self.srtt = Some(self.srtt.map_or(rtt, |srtt| (srtt * 7 + rtt) / 8)); + } + if acked.is_empty() { + return; + } + let bytes = acked.len() as u64 * u64::from(MTU); + self.in_flight -= bytes; + self.delivered += bytes; + self.scream.on_ack(Ack { + now: self.now, + packets: &acked, + in_flight: self.in_flight, + app_limited: false, + rtt: self.srtt.unwrap(), + }); + } + } + + fn percentile(delays: &mut [Duration], p: f64) -> Duration { + delays.sort(); + delays[((delays.len() as f64 * p) as usize).min(delays.len() - 1)] + } + + /// Media that follows the estimate fills a 2 Mbit/s path while its queue stays under the target + #[test] + fn fills_the_path_with_little_queue() { + let mut path = Path::new(250_000.0, None, Config::default()); + path.run(Duration::from_secs(10)); + let (delivered, mut queue) = path.run(Duration::from_secs(10)); + let throughput = delivered as f64 / 10.0; + let p95 = percentile(&mut queue, 0.95); + assert!( + throughput > 250_000.0 * 0.8, + "carried {throughput:.0} B/s of 250000" + ); + assert!(p95 < Duration::from_millis(60), "queue p95 {p95:?}"); + } + + /// When the path halves, the target rate follows within a second and the queue settles back + /// under the target + #[test] + fn follows_a_capacity_drop() { + let mut path = Path::new(250_000.0, None, Config::default()); + path.run(Duration::from_secs(10)); + path.rate = 125_000.0; + let mut fell = None; + for ms in 1..=5_000 { + path.run(TICK); + if fell.is_none() && path.scream.target_rate() < 125_000 { + fell = Some(ms); + } + } + let fell = fell.expect("the target rate never fell to the path's"); + assert!( + fell < 1_000, + "the target rate fell to the path's after {fell}ms" + ); + let (_, mut queue) = path.run(Duration::from_secs(5)); + let p95 = percentile(&mut queue, 0.95); + assert!( + p95 < Duration::from_millis(60), + "queue p95 {p95:?} after the drop" + ); + } + + /// The peer's clock basis is unknown and cancels out: any basis gives the same run + #[test] + fn ignores_the_peer_basis() { + let mut windows = Vec::new(); + for basis in [Duration::ZERO, Duration::from_secs(3600)] { + let mut path = Path::new(250_000.0, None, Config::default()); + path.basis = path.now - basis; + path.run(Duration::from_secs(10)); + windows.push(path.scream.window()); + } + assert_eq!(windows[0], windows[1]); + } + + /// A peer clock running 100ppm fast reads as a queue growing 0.1ms a second, which the base + /// delay's bounded history keeps from building into a backoff + #[test] + fn tolerates_clock_drift() { + let mut path = Path::new(1_000_000.0, Some(125_000.0), Config::default()); + path.drift_ppm = 100.0; + path.run(Duration::from_secs(10)); + let reacted = path.scream.last_reaction; + for _ in 0..300 { + path.run(Duration::from_secs(1)); + assert!( + path.scream.qdelay < Duration::from_millis(15), + "queue delay {:?} on an empty queue", + path.scream.qdelay + ); + } + assert_eq!( + path.scream.last_reaction, reacted, + "backed off an empty queue" + ); + } + + /// An application-limited sender's window stops growing once it covers what it sends, short + /// of the path, and it may probe while no queue builds + #[test] + fn app_limited_window_tracks_the_media() { + let mut path = Path::new(1_000_000.0, Some(100_000.0), Config::default()); + path.run(Duration::from_secs(10)); + let window = path.scream.ref_wnd; + path.run(Duration::from_secs(10)); + let scream = &path.scream; + assert_eq!( + scream.ref_wnd, window, + "the window grew past what the media needs" + ); + let estimate = scream.metrics().bandwidth_estimate.unwrap(); + assert!( + (100_000..500_000).contains(&estimate), + "estimate {estimate} for 100000 B/s of media on a 1000000 B/s path" + ); + assert!( + scream.metrics().probe_inflight.is_some(), + "not probing an idle path" + ); + } + + /// A quiet spell ages the base delay out as the time passing would have, so a minimum from + /// before it does not outlive the history; the minimum of windows still inside it holds + #[test] + fn base_delay_ages_across_a_quiet_spell() { + let now = Instant::now(); + let mut base = BaseDelay::new(now); + base.update(now, 1_000); + base.update(now + BASE_INTERVAL * 3, 5_000); + assert_eq!(base.get(), 1_000, "a minimum inside the history expired"); + base.update(now + BASE_INTERVAL * 3 + Duration::from_secs(600), 9_000); + assert_eq!( + base.get(), + 9_000, + "a minimum from before the quiet spell outlived it" + ); + } + + /// Probing stops once a queue builds, and the controller backs off + #[test] + fn stops_probing_on_a_queue() { + let mut path = Path::new(250_000.0, None, Config::default()); + path.run(Duration::from_secs(10)); + path.rate = 62_500.0; + path.run(Duration::from_millis(500)); + assert!(path.scream.metrics().probe_inflight.is_none()); + let reacted = path.scream.last_reaction.expect("never backed off"); + assert!(path.now - reacted < Duration::from_millis(500)); + } + + fn lose(scream: &mut Scream, now: Instant, sent: Instant) { + let packets = [Lost { + sent, + bytes: MTU, + packet: Packet::default(), + new_burst: true, + }]; + scream.on_loss(Loss { + now, + packets: &packets, + in_flight: 0, + persistent: false, + }); + } + + /// Loss backs off once, and a loss of a packet sent before that reaction does not again + #[test] + fn loss_backs_off_once() { + let ms = |ms| Duration::from_millis(ms); + let now = Instant::now(); + let mut scream = scream(now, Config::default()); + scream.ref_wnd = 100_000.0; + lose(&mut scream, now + ms(50), now + ms(1)); + assert_eq!(scream.ref_wnd, 100_000.0 * BETA_LOSS); + + lose(&mut scream, now + ms(200), now + ms(40)); + assert_eq!( + scream.ref_wnd, + 100_000.0 * BETA_LOSS, + "reacted to a loss it answered" + ); + + lose(&mut scream, now + ms(300), now + ms(100)); + assert_eq!(scream.ref_wnd, 100_000.0 * BETA_LOSS * BETA_LOSS); + } + + /// A small delay backoff holds the increase that follows near its new inflection, even when + /// the window had grown far past the old one + #[test] + fn backoff_holds_growth_at_the_new_inflection() { + let now = Instant::now(); + let mut scream = scream(now, Config::default()); + scream.ref_wnd = 100_000.0; + scream.ref_wnd_i = 50_000.0; + scream.max_bytes_in_flight = 100_000; + scream.bytes_newly_acked = 100_000; + // Just past half the target: alpha 0.1 + scream.qdelay_avg = Duration::from_millis(33); + scream.update(now + Duration::from_millis(50)); + + // At a 333ms RTT, a reaction every 25ms takes 1/13.3 of alpha / 2 + let reduced = 100_000.0 * (1.0 - 0.1 / 2.0 / (333.0 / 25.0)); + assert!((scream.ref_wnd_i - 100_000.0).abs() < 1e-6); + // 100 kB acked is about one MSS, scaled by max(0.25, scl) with scl at its floor; the + // multiplicative part is zero right after congestion + let expected = reduced + 100_000.0 * f64::from(MTU) / reduced * 0.25; + assert!( + (scream.ref_wnd - expected).abs() < 1e-3, + "ref_wnd {} != {expected}", + scream.ref_wnd + ); + } +} diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index e532ec525d1..70197995c1c 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -2,7 +2,7 @@ MoQ's sans-IO QUIC state machine, a hard fork of [quinn-proto](https://github.com/quinn-rs/quinn). -BBR3, Cubic, GCC, and Unlimited live in [moq-congestion](../moq-congestion); this crate stores each packet's controller state and hands it back on acknowledgement or loss, and the controller paces the sends. +BBR3, Cubic, GCC, SCReAMv2, and Unlimited live in [moq-congestion](../moq-congestion); this crate stores each packet's controller state and hands it back on acknowledgement or loss, and the controller paces the sends. All credit for the code goes to the quinn developers; it stays under quinn's MIT or Apache-2.0 license. diff --git a/rs/moq-quic/src/sim.rs b/rs/moq-quic/src/sim.rs index dc72fa5fc53..41cd82fec15 100644 --- a/rs/moq-quic/src/sim.rs +++ b/rs/moq-quic/src/sim.rs @@ -19,9 +19,14 @@ pub struct Bottleneck { max_queue: Duration, /// Whether packets are marked CE once the queue is half full, as a classic AQM does marks_ce: bool, + /// Whether the queue is bounded in time rather than bytes, as moq-shaper's is, so a rate + /// change keeps `max_queue` + timed: bool, /// When the link finishes serving everything queued so far busy_until: Option, pub stats: QueueStats, + /// Each queued packet's arrival and queueing delay, once set to `Some` + pub trace: Option>, } impl Bottleneck { @@ -33,18 +38,32 @@ impl Bottleneck { buffer_size, max_queue: buffer_size * time_per_byte, marks_ce, + timed: false, busy_until: None, stats: QueueStats::default(), + trace: None, } } - /// Serve at `bytes_per_second` from now on, keeping the queue's size in bytes + /// A link serving `bytes_per_second`, dropping what would wait in its queue past `max_queue` + /// at whatever rate it serves, as a moq-shaper rate limit does + pub fn timed(bytes_per_second: u64, max_queue: Duration) -> Self { + let mut bottleneck = Self::new(bytes_per_second, 0, false); + bottleneck.max_queue = max_queue; + bottleneck.timed = true; + bottleneck + } + + /// Serve at `bytes_per_second` from now on, keeping the queue's size in bytes, or in time + /// for a [`Self::timed`] queue /// /// Packets already queued still leave when they were due, as if the link stepped once they /// had been served. pub fn set_rate(&mut self, bytes_per_second: u64) { self.time_per_byte = Duration::from_nanos(1_000_000_000 / bytes_per_second); - self.max_queue = self.buffer_size * self.time_per_byte; + if !self.timed { + self.max_queue = self.buffer_size * self.time_per_byte; + } } /// Queues `bytes` arriving at `now`, returning when they leave the link and whether they @@ -57,6 +76,9 @@ impl Bottleneck { return None; } let service = bytes as u32 * self.time_per_byte; + if let Some(trace) = &mut self.trace { + trace.push((now, delay)); + } self.stats.queued += 1; self.stats.total_delay += delay; self.stats.max_delay = self.stats.max_delay.max(delay); @@ -89,8 +111,13 @@ pub struct Link { pub reorder: f64, /// The most a reordered arrival is held back pub reorder_delay: Duration, + /// Releases in-order arrivals only at the end of each interval this long, as a cellular + /// uplink's scheduler grants them, bunching what arrived within it + pub grant: Option, /// The latest in-order arrival, which jitter never moves another before last_arrival: Option, + /// When the first grant interval began + grant_epoch: Option, } impl Link { @@ -122,12 +149,23 @@ impl Link { return arrival + rng.random_range(Duration::ZERO..=self.reorder_delay); } let arrival = self.last_arrival.map_or(arrival, |last| last.max(arrival)); + let arrival = self.granted(arrival); self.last_arrival = Some(arrival); arrival }) .collect(); (marked, arrivals) } + + /// When the grant covering `arrival` releases it: the end of its interval + fn granted(&mut self, arrival: Instant) -> Instant { + let Some(grant) = self.grant else { + return arrival; + }; + let epoch = *self.grant_epoch.get_or_insert(arrival); + let intervals = (arrival - epoch).as_nanos().div_ceil(grant.as_nanos()); + epoch + Duration::from_nanos((intervals * grant.as_nanos()) as u64) + } } /// What a [`Bottleneck`] queue has seen diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 5de98e4bc1f..b54dff99709 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -46,6 +46,7 @@ mod probe; mod rate; mod receive_ts; mod reset_stream_at; +mod scream; mod sim; mod token; mod waits; diff --git a/rs/moq-quic/src/tests/scream.rs b/rs/moq-quic/src/tests/scream.rs new file mode 100644 index 00000000000..5dcead58748 --- /dev/null +++ b/rs/moq-quic/src/tests/scream.rs @@ -0,0 +1,625 @@ +//! SCReAMv2 ([`congestion::Scream`]) on the seeded sim: an encoder that follows the controller's +//! estimate sends up through a bottleneck shaped like each cellular moq-shaper preset, and the run +//! reports the queue it built, what it carried, and how fast it answered each step. +//! +//! The sim models each preset's rates, steps, delay, in-order jitter and timed queue, and the +//! bursty preset's batches as scheduler grants, as moq-quic's `tests::realtime` replicas do. +//! `rs/moq-tokio/tests/integration/scream.rs` runs the presets themselves over real sockets. The merge gate runs one preset and the probe's recovery +//! on one seed; the `#[ignore]`d [`compare`] table runs nightly with `just rs quic-sim`, beside +//! BBR3 on the same media for reference. + +use std::collections::HashMap; + +use super::*; +use crate::congestion::{bbr3, scream}; + +const FPS: u64 = 30; +/// Frames per group, each group on its own stream: a keyframe every two seconds +const GOP: u64 = 60; +/// A keyframe's size, in delta frames +const KEYFRAME: u64 = 5; +/// The encoder's bitrate range, 100 kbit/s to 10 Mbit/s +const MIN_RATE: u64 = 12_500; +const MAX_RATE: u64 = 1_250_000; +/// The most receive timestamps the sender asks for per ACK, past any ACK threshold here +const TIMESTAMPS: u32 = 32; +/// How often the run samples the estimate +const SAMPLE: Duration = Duration::from_millis(10); +/// Samples before this are the connection starting up, and count for nothing +const WARMUP: Duration = Duration::from_secs(3); +/// The queue delay a step's reaction drains back under: half SCReAMv2's default target, where it +/// starts backing off +const DRAINED: Duration = Duration::from_millis(30); + +/// The uplink a run sends through, after one of the cellular moq-shaper presets +#[derive(Debug, Clone)] +struct Uplink { + name: &'static str, + /// One-way propagation delay, each way + latency: Duration, + /// In-order jitter on the uplink + jitter: Duration, + /// The uplink scheduler's grant period + grant: Option, + /// The longest a datagram waits in the uplink queue before it is dropped + queue: Duration, + /// The uplink's rate in bytes per second from each instant on, the first at the start + rates: Vec<(Duration, u64)>, + duration: Duration, +} + +/// `cellular-step`: 2 Mbit/s behind a 500ms queue, down to 500 kbit/s at ten seconds and back at +/// twenty +fn cellular_step() -> Uplink { + Uplink { + name: "cellular-step", + latency: Duration::from_millis(20), + jitter: Duration::from_millis(3), + grant: None, + queue: Duration::from_millis(500), + rates: vec![ + (Duration::ZERO, 250_000), + (Duration::from_secs(10), 62_500), + (Duration::from_secs(20), 250_000), + ], + duration: Duration::from_secs(30), + } +} + +/// `cellular-bursty`: 2 Mbit/s arriving in 20ms grants, down to 1 Mbit/s at ten seconds and back +/// at twenty +fn cellular_bursty() -> Uplink { + Uplink { + name: "cellular-bursty", + jitter: Duration::from_millis(5), + grant: Some(Duration::from_millis(20)), + rates: vec![ + (Duration::ZERO, 250_000), + (Duration::from_secs(10), 125_000), + (Duration::from_secs(20), 250_000), + ], + ..cellular_step() + } +} + +/// `cellular-deep`: 1 Mbit/s behind a 2s queue, down to 300 kbit/s at ten seconds and back at +/// twenty +fn cellular_deep() -> Uplink { + Uplink { + name: "cellular-deep", + latency: Duration::from_millis(30), + jitter: Duration::from_millis(5), + grant: None, + queue: Duration::from_secs(2), + rates: vec![ + (Duration::ZERO, 125_000), + (Duration::from_secs(10), 37_500), + (Duration::from_secs(20), 125_000), + ], + duration: Duration::from_secs(30), + } +} + +/// What the publisher sends +#[derive(Debug, Clone, Copy)] +enum Media { + /// An encoder that retunes each frame to the controller's estimate + Adaptive, + /// One rung at a fixed bitrate in bytes per second, its upper rungs stopped + Fixed(u64), + /// The same rung at a constant bitrate, with no keyframes to burst + Constant(u64), +} + +/// One run's controller, media and path +#[derive(Clone)] +struct Scenario { + controller: Arc, + media: Media, + /// The rate the session probes up to; zero for none + probe_target: u64, + path: Uplink, + seed: u64, +} + +/// One sample of the sender's estimate +#[derive(Debug, Clone, Copy)] +struct Snapshot { + /// Since the media's start + at: Duration, + /// The bandwidth estimate, in bytes per second + estimate: u64, +} + +/// What a run measured +#[derive(Debug, Default)] +struct Outcome { + samples: Vec, + /// Each packet's arrival at the bottleneck, since the media's start, and its queueing delay + queue: Vec<(Duration, Duration)>, + /// Each delivered frame's delay from its write to its last byte read, sorted + frames: Vec, + /// Frames written that never arrived + missing: u64, + /// Media bytes read by the receiver, by when + read: Vec<(Duration, u64)>, + /// Media bytes written + unique: u64, + /// Bytes of probe packets + redundant: u64, + dropped: u64, +} + +/// A group the sender has opened +struct Group { + id: StreamId, + /// When each frame was written, and the stream offset its last byte ends at + frames: Vec<(Instant, u64)>, + /// Bytes written to the group that the stream had no room for yet + backlog: u64, + ended: bool, + finished: bool, +} + +fn run(scenario: &Scenario) -> Outcome { + let _guard = subscribe(); + entropy::reseed(scenario.seed); + let path = &scenario.path; + let mut server = server_config(); + server.transport.limits.max_streams = NonZeroU64::new(1000); + let mut pair = Pair::new(Default::default(), server); + pair.latency = path.latency; + let mut config = client_config(); + config.transport.congestion = scenario.controller.clone(); + config.transport.receive_timestamps = NonZeroU32::new(TIMESTAMPS); + let (client_ch, server_ch) = pair.connect_with(config); + + let (_, first) = path.rates[0]; + let mut bottleneck = Bottleneck::timed(first, path.queue); + bottleneck.trace = Some(Vec::new()); + pair.to_server.bottleneck = Some(bottleneck); + pair.to_server.jitter = path.jitter; + pair.to_server.grant = path.grant; + pair.client_conn_mut(client_ch).set_probe_target(scenario.probe_target); + + let start = pair.time; + let end = start + path.duration; + let frame_at = |n: u64| start + Duration::from_nanos(n * 1_000_000_000 / FPS); + let mut outcome = Outcome::default(); + let mut groups: Vec = Vec::new(); + let mut reading: HashMap = HashMap::new(); + let mut read = 0; + let mut next_frame = 0; + let mut next_sample = start; + let mut next_rate = 1; + loop { + let now = pair.time; + if let Some(&(at, rate)) = path.rates.get(next_rate) + && now >= start + at + { + pair.to_server.bottleneck.as_mut().unwrap().set_rate(rate); + next_rate += 1; + } + + while frame_at(next_frame) <= now && frame_at(next_frame) < end { + let rate = match scenario.media { + Media::Adaptive => { + let estimate = pair.client_conn_mut(client_ch).stats().path.bandwidth_estimate; + estimate.unwrap_or(MIN_RATE).clamp(MIN_RATE, MAX_RATE) + } + Media::Fixed(rate) | Media::Constant(rate) => rate, + }; + let keyframe = match scenario.media { + Media::Constant(_) => 1, + _ => KEYFRAME, + }; + let index = next_frame % GOP; + let delta = rate * GOP / FPS / (GOP - 1 + keyframe); + let size = if index == 0 { delta * keyframe } else { delta }; + if index == 0 { + if let Some(group) = groups.last_mut() { + group.ended = true; + } + let id = pair + .client_streams(client_ch) + .open( + Dir::Uni, + send::Options { + group: None, + priority: groups.len() as i64, + }, + ) + .expect("stream limit"); + groups.push(Group { + id, + frames: Vec::new(), + backlog: 0, + ended: false, + finished: false, + }); + } + let group = groups.last_mut().unwrap(); + let offset = group.frames.last().map_or(0, |&(_, end)| end) + size; + group.frames.push((now, offset)); + group.backlog += size; + outcome.unique += size; + next_frame += 1; + } + flush(&mut pair, client_ch, &mut groups); + + pair.drive_client(); + pair.drive_server(); + + let conn = pair.server_conn_mut(server_ch); + while let Some(id) = conn.streams().accept(Dir::Uni) { + reading.insert(id, (0, 0)); + } + for (id, (received, completed)) in reading.iter_mut() { + let mut stream = conn.recv_stream(*id); + let Ok(mut chunks) = stream.read() else { continue }; + while let Ok(Some(chunk)) = chunks.next(usize::MAX) { + *received += chunk.bytes.len() as u64; + read += chunk.bytes.len() as u64; + } + let _ = chunks.finalize(); + let frames = &groups[(id.index() - groups[0].id.index()) as usize].frames; + while let Some(&(written, end)) = frames.get(*completed) + && end <= *received + { + outcome.frames.push(now - written); + *completed += 1; + } + } + + if now >= next_sample { + let estimate = pair.client_conn_mut(client_ch).stats().path.bandwidth_estimate; + outcome.samples.push(Snapshot { + at: now - start, + estimate: estimate.unwrap_or(0), + }); + outcome.read.push((now - start, read)); + next_sample += SAMPLE; + } + if now >= end { + break; + } + + let wakeups = [ + pair.client.next_wakeup(), + pair.server.next_wakeup(), + Some(frame_at(next_frame)), + Some(next_sample), + path.rates.get(next_rate).map(|&(at, _)| start + at), + ]; + pair.time = pair.time.max(wakeups.into_iter().flatten().min().unwrap()); + } + + let written: u64 = groups.iter().map(|g| g.frames.len() as u64).sum(); + outcome.missing = written - outcome.frames.len() as u64; + outcome.frames.sort(); + outcome.redundant = pair.client_conn_mut(client_ch).stats().path.sent_bandwidth_probe_bytes; + let bottleneck = pair.to_server.bottleneck.take().unwrap(); + outcome.dropped = bottleneck.stats.dropped; + let trace = bottleneck.trace.unwrap(); + outcome.queue = trace + .into_iter() + .filter(|(at, _)| *at >= start) + .map(|(at, delay)| (at - start, delay)) + .collect(); + outcome +} + +/// Write what credit allows of each group's backlog, finishing every ended group once written +fn flush(pair: &mut Pair, ch: ConnectionHandle, groups: &mut [Group]) { + const ZEROS: [u8; 16_384] = [0; 16_384]; + for group in groups.iter_mut().filter(|g| !g.finished) { + let mut stream = pair.client_send(ch, group.id); + while group.backlog > 0 { + let chunk = &ZEROS[..group.backlog.min(ZEROS.len() as u64) as usize]; + match stream.write(&mut &chunk[..]) { + Ok(n) => group.backlog -= n as u64, + Err(send::Error::Blocked) => return, + Err(e) => panic!("write: {e}"), + } + } + if group.ended { + stream.finish().unwrap(); + group.finished = true; + } + } +} + +/// What a run's numbers came to over each of its rate phases +#[derive(Debug, Default, Clone)] +struct Report { + /// The bottleneck's queueing delay at the 50th, 95th and 99th percentile, past [`WARMUP`] + queue: [Duration; 3], + /// Frame delays at the 50th, 95th and 99th percentile + frame: [Duration; 3], + /// Each phase's goodput over its rate, from a second into it + utilization: Vec, + /// How soon after each step down the estimate fell to the new rate + fall: Vec>, + /// How soon after each step down the queue it built drained back under [`DRAINED`] + drain: Vec>, + /// How soon after each step up the estimate reached 80% of the new rate + rise: Vec>, +} + +impl Outcome { + fn report(&self, path: &Uplink) -> Report { + let mut queue: Vec = self + .queue + .iter() + .filter(|(at, _)| *at >= WARMUP) + .map(|(_, delay)| *delay) + .collect(); + queue.sort(); + let mut report = Report { + queue: [0.5, 0.95, 0.99].map(|p| percentile(&queue, p)), + frame: [0.5, 0.95, 0.99].map(|p| percentile(&self.frames, p)), + ..Default::default() + }; + for (i, &(from, rate)) in path.rates.iter().enumerate() { + let until = path.rates.get(i + 1).map_or(path.duration, |&(at, _)| at); + let read = |at: Duration| self.read.iter().take_while(|(t, _)| *t <= at).last().map_or(0, |r| r.1); + let from_settled = from.max(WARMUP) + Duration::from_secs(1); + let carried = read(until) - read(from_settled); + report + .utilization + .push(carried as f64 / (until - from_settled).as_secs_f64() / rate as f64); + let Some(&(_, before)) = i.checked_sub(1).map(|j| &path.rates[j]) else { + continue; + }; + let estimates: Vec<&Snapshot> = self.samples.iter().filter(|s| s.at >= from && s.at < until).collect(); + if rate < before { + report + .fall + .push(estimates.iter().find(|s| s.estimate <= rate).map(|s| s.at - from)); + let queue: Vec<&(Duration, Duration)> = + self.queue.iter().filter(|(at, _)| *at >= from && *at < until).collect(); + let built = queue.iter().position(|(_, delay)| *delay >= DRAINED).unwrap_or(0); + report.drain.push( + queue[built..] + .iter() + .find(|(_, delay)| *delay < DRAINED) + .map(|(at, _)| *at - from), + ); + } else { + report.rise.push( + estimates + .iter() + .find(|s| s.estimate >= rate * 4 / 5) + .map(|s| s.at - from), + ); + } + } + report + } +} + +fn percentile(sorted: &[Duration], p: f64) -> Duration { + sorted + .get(((sorted.len() as f64 * p) as usize).min(sorted.len().saturating_sub(1))) + .copied() + .unwrap_or_default() +} + +fn ms(d: Duration) -> f64 { + d.as_secs_f64() * 1e3 +} + +fn scream() -> Arc { + Arc::new(scream::Config::default()) +} + +/// The cellular-step preset with media that follows the estimate: SCReAMv2 holds a smaller queue +/// than BBR3 does on the same media, and its estimate follows the drop where BBR3's max filter +/// holds on to the old rate +#[test] +fn smaller_queue_than_bbr3() { + let _guard = subscribe(); + let path = cellular_step(); + let report = |controller| { + let scenario = Scenario { + controller, + media: Media::Adaptive, + probe_target: 0, + path: path.clone(), + seed: 1, + }; + run(&scenario).report(&path) + }; + let scream = report(scream()); + let bbr3 = report(Arc::new(bbr3::Config::default())); + info!(?scream, ?bbr3, "cellular-step"); + for (i, p) in ["p50", "p95"].into_iter().enumerate() { + assert!( + scream.queue[i] < bbr3.queue[i], + "queue {p} {:?}, {:?} under BBR3", + scream.queue[i], + bbr3.queue[i] + ); + } + assert!(scream.fall[0].is_some(), "the estimate never fell to the dropped rate"); +} + +/// A stopped rung recovers under native probing: a constant 500 kbit/s rung alone on a 4 Mbit/s +/// path raises the estimate to the 2.5 Mbit/s the stopped rungs need, where without a probe +/// target it stays near the rung's own rate. A rung with keyframes needs no probe: each keyframe's +/// burst raises the window on its own. +#[test] +fn probe_recovers_a_stopped_rung() { + let _guard = subscribe(); + const TARGET: u64 = 312_500; + let path = Uplink { + name: "4 Mbit/s", + rates: vec![(Duration::ZERO, 500_000)], + duration: Duration::from_secs(15), + ..cellular_step() + }; + let scenario = |probe_target| Scenario { + controller: scream(), + media: Media::Constant(62_500), + probe_target, + path: path.clone(), + seed: 1, + }; + let reached = |outcome: &Outcome| outcome.samples.iter().find(|s| s.estimate >= TARGET).map(|s| s.at); + let baseline = run(&scenario(0)); + assert_eq!( + reached(&baseline), + None, + "the estimate reached the target with no probe" + ); + let probing = run(&scenario(TARGET)); + let rise = reached(&probing).expect("the probe never raised the estimate to the target"); + info!(?rise, redundant = probing.redundant, unique = probing.unique, "probe"); +} + +/// The comparison table: each preset with adaptive media under SCReAMv2's variants and BBR3, and +/// the probe recovering a stopped rung on a path that opens up +#[test] +#[ignore = "SCReAMv2's measurement, run nightly by `just rs quic-sim`"] +fn compare() { + const SEEDS: ops::RangeInclusive = 1..=4; + let reduce_jitter = { + let mut config = scream::Config::default(); + config.reduce_jitter(true); + Arc::new(config) + }; + let target_30ms = { + let mut config = scream::Config::default(); + config.qdelay_target(Duration::from_millis(30)); + Arc::new(config) + }; + let controllers: [(&str, Arc); 4] = [ + ("scream", scream()), + ("scream reduce_jitter", reduce_jitter), + ("scream target 30ms", target_30ms), + ("bbr3", Arc::new(bbr3::Config::default())), + ]; + + println!( + "| preset | controller | queue p50/p95/p99 | frame p50/p95/p99 | utilization per phase | estimate fall | queue drain | estimate rise |" + ); + println!("|---|---|---|---|---|---|---|---|"); + for path in [cellular_step(), cellular_bursty(), cellular_deep()] { + for (name, controller) in &controllers { + let reports: Vec = SEEDS + .map(|seed| { + let scenario = Scenario { + controller: controller.clone(), + media: Media::Adaptive, + probe_target: 0, + path: path.clone(), + seed, + }; + run(&scenario).report(&path) + }) + .collect(); + let triple = |pick: fn(&Report) -> [Duration; 3]| { + let mean = |i: usize| reports.iter().map(|r| ms(pick(r)[i])).sum::() / reports.len() as f64; + format!("{:.0}/{:.0}/{:.0}ms", mean(0), mean(1), mean(2)) + }; + let utilization = (0..path.rates.len()) + .map(|i| { + let mean = reports.iter().map(|r| r.utilization[i]).sum::() / reports.len() as f64; + format!("{mean:.2}") + }) + .collect::>() + .join(" "); + println!( + "| {} | {name} | {} | {} | {utilization} | {} | {} | {} |", + path.name, + triple(|r| r.queue), + triple(|r| r.frame), + latency(reports.iter().flat_map(|r| r.fall.clone())), + latency(reports.iter().flat_map(|r| r.drain.clone())), + latency(reports.iter().flat_map(|r| r.rise.clone())), + ); + } + } + + // A stopped 500 kbit/s rung on a path too narrow for the stopped rungs' 2.5 Mbit/s until it + // opens up to 4 Mbit/s at 15s + const TARGET: u64 = 312_500; + let path = Uplink { + name: "1 then 4 Mbit/s", + rates: vec![(Duration::ZERO, 125_000), (Duration::from_secs(15), 500_000)], + duration: Duration::from_secs(30), + ..cellular_step() + }; + println!(); + println!( + "| media | probe | estimate before the path opened, mean | reached the target after it opened | overhead | queue p95 before/after |" + ); + println!("|---|---|---|---|---|---|"); + let opened = path.rates[1].0; + for (media, name) in [ + (Media::Fixed(62_500), "500 kbit/s rung"), + (Media::Constant(62_500), "same, no keyframes"), + ] { + for target in [TARGET, 0] { + let outcomes: Vec = SEEDS + .map(|seed| { + run(&Scenario { + controller: scream(), + media, + probe_target: target, + path: path.clone(), + seed, + }) + }) + .collect(); + let mean = |f: &dyn Fn(&Outcome) -> f64| outcomes.iter().map(f).sum::() / outcomes.len() as f64; + let before = mean(&|o| { + let estimates: Vec = o + .samples + .iter() + .filter(|s| s.at >= WARMUP && s.at < opened) + .map(|s| s.estimate as f64) + .collect(); + estimates.iter().sum::() / estimates.len() as f64 + }); + let reached = outcomes.iter().map(|o| { + o.samples + .iter() + .find(|s| s.at >= opened && s.estimate >= TARGET) + .map(|s| s.at - opened) + }); + let queue = |o: &Outcome, range: ops::Range| { + let mut queue: Vec = o + .queue + .iter() + .filter(|(at, _)| range.contains(at)) + .map(|(_, delay)| *delay) + .collect(); + queue.sort(); + ms(percentile(&queue, 0.95)) + }; + println!( + "| {name} | {} | {:.0} B/s | {} | {:.3} | {:.0}/{:.0}ms |", + if target > 0 { "on" } else { "off" }, + before, + latency(reached), + mean(&|o| o.redundant as f64 / o.unique as f64), + mean(&|o| queue(o, WARMUP..opened)), + mean(&|o| queue(o, opened..path.duration)), + ); + } + } +} + +/// The mean over the steps that happened, and how many never did +fn latency(values: impl Iterator>) -> String { + let values: Vec> = values.collect(); + let hits: Vec = values.iter().flatten().map(|d| ms(*d)).collect(); + let missed = values.len() - hits.len(); + let mean = hits.iter().sum::() / hits.len().max(1) as f64; + match (hits.is_empty(), missed) { + (true, _) => format!("never ({missed})"), + (false, 0) => format!("{mean:.0}ms"), + (false, _) => format!("{mean:.0}ms, never {missed}"), + } +} diff --git a/rs/moq-tokio/tests/integration/main.rs b/rs/moq-tokio/tests/integration/main.rs index f1b30103dc9..3186e724989 100644 --- a/rs/moq-tokio/tests/integration/main.rs +++ b/rs/moq-tokio/tests/integration/main.rs @@ -15,6 +15,7 @@ mod receive_ts; mod reconnect; mod reset_secret; mod runtime; +mod scream; mod sim; mod stream_code; mod subscription_end_integrity; diff --git a/rs/moq-tokio/tests/integration/scream.rs b/rs/moq-tokio/tests/integration/scream.rs new file mode 100644 index 00000000000..0f868ac6472 --- /dev/null +++ b/rs/moq-tokio/tests/integration/scream.rs @@ -0,0 +1,422 @@ +//! SCReAMv2 ([`congestion::Scream`]) over real sockets through the cellular moq-shaper presets. +//! +//! A media sender encodes 30 frames a second at the controller's estimate, a keyframe every two +//! seconds, each group on its own stream, up through the shaper's uplink. Each frame carries when +//! it was written, so the receiver times it to its last byte. The sender asks for receive +//! timestamps, and every acknowledged packet's forward delay above the run's lowest is its +//! queueing delay: sender and receiver share this host's clock. `near-zero` measures the noise +//! floor that delay carries on its own, from the shaper's userspace timing and this host's +//! scheduling. +//! +//! It runs on the wall clock for about a minute and a half, so it is ignored: `just rs quic-sim` +//! runs it nightly. moq-quic's `tests::scream` runs the same measurement on the seeded sim, beside +//! BBR3 on the same media. A busy host widens every number here, so read it beside the noise +//! floor. + +use std::{ + net::UdpSocket, + sync::{Arc, Mutex}, + time::Duration, +}; + +use bytes::{Buf, BufMut, Bytes, BytesMut}; +use kio::clock::Instant; +use moq_congestion::{self as congestion, Controller, Factory}; +use moq_tokio::quic::driver; + +const FPS: u64 = 30; +/// Frames per group: a keyframe every two seconds +const GOP: u64 = 60; +/// A keyframe's size, in delta frames +const KEYFRAME: u64 = 5; +/// The encoder's bitrate range, 100 kbit/s to 10 Mbit/s, in bytes per second +const MIN_RATE: u64 = 12_500; +const MAX_RATE: u64 = 1_250_000; +/// Each frame's header: when it was written, in microseconds since the run began, and its length +const HEADER: usize = 12; +/// Samples before this are the connection starting up, and count for nothing +const WARMUP: Duration = Duration::from_secs(3); +/// How long the sender runs past the last step +const AFTER: Duration = Duration::from_secs(2); +/// The queue delay a step's reaction drains back under: half SCReAMv2's default target +const DRAINED: Duration = Duration::from_millis(30); + +/// Delegates to a controller, recording when each acknowledged packet was sent and received +struct Recording { + inner: Box, + log: Arc>>, +} + +impl Controller for Recording { + fn on_sent(&mut self, sent: congestion::Sent) -> congestion::Packet { + self.inner.on_sent(sent) + } + + fn on_cwnd_limited(&mut self) { + self.inner.on_cwnd_limited() + } + + fn on_app_limited(&mut self, in_flight: u64) { + self.inner.on_app_limited(in_flight) + } + + fn on_probe_filled(&mut self, in_flight: u64) { + self.inner.on_probe_filled(in_flight) + } + + fn on_ack(&mut self, ack: congestion::Ack<'_>) { + let mut log = self.log.lock().unwrap(); + log.extend(ack.packets.iter().filter_map(|p| Some((p.sent, p.received?)))); + drop(log); + self.inner.on_ack(ack) + } + + fn on_loss(&mut self, loss: congestion::Loss<'_>) { + self.inner.on_loss(loss) + } + + fn on_ce(&mut self, ce: congestion::Ce) { + self.inner.on_ce(ce) + } + + fn on_spurious_loss(&mut self) { + self.inner.on_spurious_loss() + } + + fn on_mtu_update(&mut self, new_mtu: u16) { + self.inner.on_mtu_update(new_mtu) + } + + fn on_ack_frequency_update(&mut self, threshold: u64, max_ack_delay: Duration) { + self.inner.on_ack_frequency_update(threshold, max_ack_delay) + } + + fn window(&self) -> u64 { + self.inner.window() + } + + fn pace(&mut self, now: Instant, bytes: u64) -> Option { + self.inner.pace(now, bytes) + } + + fn burst(&self) -> Option { + self.inner.burst() + } + + fn metrics(&self) -> congestion::Metrics { + self.inner.metrics() + } + + fn probes(&self) -> bool { + self.inner.probes() + } + + fn paces_probe(&self) -> bool { + self.inner.paces_probe() + } + + fn on_probe_target(&mut self, bytes_per_second: u64) { + self.inner.on_probe_target(bytes_per_second) + } + + fn clone_box(&self) -> Box { + Box::new(Self { + inner: self.inner.clone_box(), + log: self.log.clone(), + }) + } +} + +struct Recorder { + inner: Arc, + log: Arc>>, +} + +impl Factory for Recorder { + fn build(self: Arc, init: congestion::Init) -> Box { + Box::new(Recording { + inner: self.inner.clone().build(init), + log: self.log.clone(), + }) + } +} + +/// What one run measured +struct Run { + /// Each acknowledged packet's queueing delay, by when it was sent + queue: Vec<(Duration, Duration)>, + /// Each delivered frame's delay from its write to its last byte read + frames: Vec, + /// The bandwidth estimate every 10ms + estimates: Vec<(Duration, u64)>, + /// Media bytes read, by when + read: Arc>>, +} + +async fn run(preset: &moq_shaper::Preset) -> Run { + let cert = rcgen::generate_simple_self_signed(vec!["localhost".into()]).unwrap(); + let key = rustls::pki_types::PrivatePkcs8KeyDer::from(cert.signing_key.serialize_der()); + let server_config = + driver::server::Config::with_single_cert(vec![cert.cert.into()], key.into()).expect("server config"); + let server = driver::endpoint::Endpoint::new( + driver::endpoint::Config::default().with_server(server_config), + UdpSocket::bind("127.0.0.1:0").unwrap(), + ) + .expect("server"); + let server_addr = server.local_addr(); + + let setup = preset.setup("127.0.0.1:0".parse().unwrap(), server_addr); + let duration = setup + .up + .steps + .last() + .map_or(Duration::from_secs(10), |last| last.at + AFTER); + let start = Instant::now(); + let shaper = moq_shaper::Shaper::bind(setup).await.expect("shaper"); + + let frames = Arc::new(Mutex::new(Vec::new())); + let read = Arc::new(Mutex::new(Vec::new())); + tokio::spawn({ + let frames = frames.clone(); + let read = read.clone(); + async move { + let conn = server.accept().await.unwrap().accept().unwrap().await.unwrap(); + let total = Arc::new(Mutex::new(0u64)); + while let Ok(mut stream) = conn.accept_uni().await { + let (frames, read, total) = (frames.clone(), read.clone(), total.clone()); + tokio::spawn(async move { + let mut buf = BytesMut::new(); + while let Ok(Some(chunk)) = stream.read_chunk(usize::MAX).await { + let now = start.elapsed(); + let mut total = total.lock().unwrap(); + *total += chunk.len() as u64; + read.lock().unwrap().push((now, *total)); + buf.extend_from_slice(&chunk); + while buf.len() >= HEADER { + let len = u32::from_be_bytes(buf[8..12].try_into().unwrap()) as usize; + if buf.len() < HEADER + len { + break; + } + let written = Duration::from_micros(buf.get_u64()); + buf.advance(4 + len); + frames.lock().unwrap().push(now - written); + } + } + }); + } + } + }); + + let log = Arc::new(Mutex::new(Vec::new())); + let tls = rustls::ClientConfig::builder() + .dangerous() + .with_custom_certificate_verifier(Arc::new(crate::receive_credit::AnyCert)) + .with_no_client_auth(); + let mut client = driver::client::Config::new(Arc::new(tls)); + client.transport.congestion = Arc::new(Recorder { + inner: Arc::new(congestion::scream::Config::default()), + log: log.clone(), + }); + client.transport.receive_timestamps = std::num::NonZeroU32::new(32); + let endpoint = + driver::endpoint::Endpoint::new(Default::default(), UdpSocket::bind("127.0.0.1:0").unwrap()).expect("client"); + let conn = endpoint + .connect(&client, shaper.addr(), "localhost") + .expect("connect") + .await + .expect("handshake"); + + let estimates = Arc::new(Mutex::new(Vec::new())); + let sampler = tokio::spawn({ + let conn = conn.clone(); + let estimates = estimates.clone(); + async move { + let mut tick = kio::clock::interval(Duration::from_millis(10)); + loop { + tick.tick().await; + let estimate = conn.stats().path.bandwidth_estimate.unwrap_or(0); + estimates.lock().unwrap().push((start.elapsed(), estimate)); + } + } + }); + + let mut tick = kio::clock::interval(Duration::from_nanos(1_000_000_000 / FPS)); + let mut stream = None; + let mut finished = Vec::new(); + for n in 0.. { + tick.tick().await; + let now = start.elapsed(); + if now >= duration { + break; + } + let rate = conn + .stats() + .path + .bandwidth_estimate + .unwrap_or(MIN_RATE) + .clamp(MIN_RATE, MAX_RATE); + let delta = rate * GOP / FPS / (GOP - 1 + KEYFRAME); + let size = if n % GOP == 0 { delta * KEYFRAME } else { delta } as usize; + if n % GOP == 0 { + if let Some(done) = stream.take() { + finished.push(driver::send::Stream::finish(done)); + } + stream = Some(conn.open_uni(Default::default()).await.unwrap()); + } + let mut frame = BytesMut::with_capacity(HEADER + size); + frame.put_u64(now.as_micros() as u64); + frame.put_u32(size as u32); + frame.put_bytes(0, size); + stream.as_mut().unwrap().write_all(Bytes::from(frame)).await.unwrap(); + } + sampler.abort(); + conn.close(0u32.into(), "done"); + shaper.verify().expect("the shaper acted"); + + let log = log.lock().unwrap(); + let forward: Vec<(Duration, i128)> = log + .iter() + .filter(|(sent, _)| *sent >= start) + .map(|&(sent, received)| { + let sent = sent - start; + (sent, received.as_nanos() as i128 - sent.as_nanos() as i128) + }) + .collect(); + let base = forward.iter().map(|(_, delay)| *delay).min().unwrap_or(0); + let queue = forward + .into_iter() + .map(|(sent, delay)| (sent, Duration::from_nanos((delay - base) as u64))) + .collect(); + let mut frames = frames.lock().unwrap().clone(); + frames.sort(); + let estimates = estimates.lock().unwrap().clone(); + Run { + queue, + frames, + estimates, + read, + } +} + +fn percentile(sorted: &[Duration], p: f64) -> Duration { + sorted + .get(((sorted.len() as f64 * p) as usize).min(sorted.len().saturating_sub(1))) + .copied() + .unwrap_or_default() +} + +fn ms(d: Duration) -> f64 { + d.as_secs_f64() * 1e3 +} + +fn triple(sorted: &[Duration]) -> String { + let [p50, p95, p99] = [0.5, 0.95, 0.99].map(|p| ms(percentile(sorted, p))); + format!("{p50:.0}/{p95:.0}/{p99:.0}ms") +} + +/// Runs `preset` and prints its row of the table, under the same header as the others +async fn report(name: &str) { + let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); + let preset = moq_shaper::Preset::load(name).expect("preset"); + let setup = preset.setup("127.0.0.1:0".parse().unwrap(), "127.0.0.1:1".parse().unwrap()); + // The uplink's rate over each phase, from its start; none on a path with no rate limit + let mut phases = Vec::new(); + if let Some(rate) = &setup.config.up.rate { + phases.push((Duration::ZERO, rate.bits_per_second / 8)); + for step in &setup.up.steps { + if let Some(bps) = step.bits_per_second { + phases.push((step.at, bps / 8)); + } + } + } + println!( + "| preset | queue p50/p95/p99 | frame p50/p95/p99 | utilization per phase | estimate fall | queue drain | estimate rise |" + ); + let run = run(&preset).await; + let end = run.estimates.last().map_or(Duration::ZERO, |(at, _)| *at); + let mut queue: Vec = run + .queue + .iter() + .filter(|(sent, _)| *sent >= WARMUP) + .map(|(_, q)| *q) + .collect(); + queue.sort(); + + let read = run.read.lock().unwrap(); + let read_at = |at: Duration| read.iter().take_while(|(t, _)| *t <= at).last().map_or(0, |r| r.1); + let mut utilization = Vec::new(); + let (mut fall, mut drain, mut rise) = (Vec::new(), Vec::new(), Vec::new()); + let show = |d: Option| d.map_or("never".to_string(), |d| format!("{:.0}ms", ms(d))); + for (i, &(from, rate)) in phases.iter().enumerate() { + let until = phases.get(i + 1).map_or(end, |&(at, _)| at); + let settled = from.max(WARMUP) + Duration::from_secs(1); + let carried = read_at(until).saturating_sub(read_at(settled)); + utilization.push(format!( + "{:.2}", + carried as f64 / (until - settled).as_secs_f64() / rate as f64 + )); + let Some(&(_, before)) = i.checked_sub(1).map(|j| &phases[j]) else { + continue; + }; + let estimates = run.estimates.iter().filter(|(at, _)| *at >= from && *at < until); + if rate < before { + fall.push(show( + estimates.clone().find(|(_, e)| *e <= rate).map(|(at, _)| *at - from), + )); + let phase: Vec<&(Duration, Duration)> = run + .queue + .iter() + .filter(|(sent, _)| *sent >= from && *sent < until) + .collect(); + let built = phase.iter().position(|(_, q)| *q >= DRAINED).unwrap_or(0); + drain.push(show( + phase[built..] + .iter() + .find(|(_, q)| *q < DRAINED) + .map(|(sent, _)| *sent - from), + )); + } else { + rise.push(show( + estimates + .clone() + .find(|(_, e)| *e >= rate * 4 / 5) + .map(|(at, _)| *at - from), + )); + } + } + println!( + "| {name} | {} | {} | {} | {} | {} | {} |", + triple(&queue), + triple(&run.frames), + utilization.join(" "), + fall.join(" "), + drain.join(" "), + rise.join(" "), + ); + assert!(!run.queue.is_empty(), "no acknowledged packet carried a receive time"); +} + +/// The noise floor: the shaper in the path, treating nothing +#[tokio::test(flavor = "multi_thread")] +#[ignore = "SCReAMv2's real-socket measurement, run nightly by `just rs quic-sim`"] +async fn near_zero() { + report("near-zero").await; +} + +#[tokio::test(flavor = "multi_thread")] +#[ignore = "SCReAMv2's real-socket measurement, run nightly by `just rs quic-sim`"] +async fn cellular_step() { + report("cellular-step").await; +} + +#[tokio::test(flavor = "multi_thread")] +#[ignore = "SCReAMv2's real-socket measurement, run nightly by `just rs quic-sim`"] +async fn cellular_bursty() { + report("cellular-bursty").await; +} + +#[tokio::test(flavor = "multi_thread")] +#[ignore = "SCReAMv2's real-socket measurement, run nightly by `just rs quic-sim`"] +async fn cellular_deep() { + report("cellular-deep").await; +}