diff --git a/quest/m1/release-perf/README.md b/quest/m1/release-perf/README.md index a35788fa4bb..930b09742e1 100644 --- a/quest/m1/release-perf/README.md +++ b/quest/m1/release-perf/README.md @@ -39,9 +39,13 @@ noise. [FETCH fan-out](/quest/m1/release-perf/fetch-fanout.md) owns it. - `stats/session-report` is accepted at O(live sessions), after a cheap fix. Per-label payload bytes ([#5731](https://github.com/moq-dev/moq/pull/5731)) - match payload billing per label, such as per signing key. The drain runs - once a second, using under 0.01% of a core at 10,000 sessions. Deleting - those bytes was rejected. + match payload billing per label, such as per signing key. After removing + redundant row locking and live-session CAS, three alternating measurements + give 52.2 us for 10,000 sessions without wires and 388.4 us with one real + transport wire each. A once-per-second drain uses about 0.04% of a core + with those wires; the earlier under-0.01% estimate omitted their walk. + This remaining per-session and per-connection sampling is accepted. Deleting + per-label bytes was rejected. ### Release as a baseline @@ -59,7 +63,7 @@ usually pays for a feature. | `track_gap_flush/4096` | 5.7 us | 551 us | 96x | [Sparse gaps](/quest/m1/release-perf/sparse-gaps.md) | | `track_gap_flush/512` | 1.8 us | 48 us | 27x | same | | `track_gap_flush/64` | 1.5 us | 6.2 us | 4.1x | same | -| `stats/session-report/1000x10` | 11.0 us | 86.8 us | 7.9x | [Stats report](/quest/m1/release-perf/stats-report.md) | +| `stats/session-report/1000x10` | 11.0 us | 86.8 us | 7.9x | accepted (above) | | `track_gc/idle/16384` | 11.6 ns | 73.7 ns | 6.3x | [Idle GC](/quest/m1/release-perf/gc-idle.md) | | `origin/resume_overlap/sequence/1r_64g_32c` | 93 us | 426 us | 4.6x | [Resume duplicates](/quest/m1/release-perf/resume-duplicates.md) | | `session_delivery_group_cost/moq-transport-22/.../viewers=64/.../fetch` | 1.19 ms | 3.86 ms | 3.2x | accepted (above) | @@ -96,7 +100,6 @@ one. - [Sparse gaps](/quest/m1/release-perf/sparse-gaps.md) - an arrival and a gap flush cost O(log gaps), not O(gaps) - [Resume duplicates](/quest/m1/release-perf/resume-duplicates.md) - an ordered resume skips copies of an already-delivered group without consuming them - [Idle GC](/quest/m1/release-perf/gc-idle.md) - an idle track GC skips the wall-clock pass when nothing is due -- [Stats report](/quest/m1/release-perf/stats-report.md) - the stats drain locks each row once and its bench counts the wire walk; per-label bytes stay, at an accepted O(live sessions) - [Re-check](/quest/m1/release-perf/recheck.md) - the screen-only moves are re-measured with instruction counts, and the ones that hold are fixed or accepted - [JS frames](/quest/m1/release-perf/js-frames.md) - the js/net frame path is no slower than release in a browser, or the gap is accepted - [Comparison harness](/quest/m1/release-perf/compare-harness.md) - `just bench ` pairs full Criterion IDs and keeps its report diff --git a/quest/m1/release-perf/stats-report.md b/quest/m1/release-perf/stats-report.md deleted file mode 100644 index a256acda04a..00000000000 --- a/quest/m1/release-perf/stats-report.md +++ /dev/null @@ -1,44 +0,0 @@ -# [S] Stats report - -## Goal - -`stats::Registry::report` takes each row's lock once and reads live sessions -without a compare-and-swap. The `stats/session-report` bench registers -transport `Wire`s, so the per-connection wire walk shows up in the measured -drain. The remaining O(live sessions) cost is accepted, and the reason is -recorded in the [epic](/quest/m1/release-perf/README.md). - -## Plan - -Decided 2026-10-21. `stats/session-report/1000x10` went from 11 us to 87 us -(7.9x, confirmed). The drain runs once a second, so at 10,000 live sessions -this is under 0.01% of a core. - -- Cause: [#5731](https://github.com/moq-dev/moq/pull/5731) added per-label - `sent_bytes`/`received_bytes`, kept by `SessionBytes`/`Arrival`. That keeps - the write path to one session-local atomic, and the cost moves to the drain: - `SessionBytes::read` does an `Arc::get_mut` per live session, and the row's - bytes mutex is taken twice. [#5659](https://github.com/moq-dev/moq/pull/5659) - (refusals, failures) and [#5773](https://github.com/moq-dev/moq/pull/5773) - (wire counters, `collect_wire` per report) add smaller fixed costs. -- Labels and per-label bytes stay. Labels are how an embedder buckets - sessions, such as per project or per signing key. Payload bytes per label - are what matches payload billing, and wire bytes can't give that. Nothing - reads them yet. Deleting them was rejected anyway: the cost is negligible, - and removing them would be a breaking change to the stats wire that would - have to be redone once a per-key or per-bucket view wants them. -- Cheap fix: in `read()`, use `Arc::strong_count(&arrival.traffic) == 1` plus - an `Acquire` fence instead of `Arc::get_mut`. This is valid only while no - `Weak` exists, so check that first. Return - `live.is_empty()` from the same guard as `read()`, instead of locking a - second time for the closed check. -- Bench: register a `Wire` per session in `session-report`. `collect_wire` - walks every live connection (a `Weak` upgrade plus two mutexes) on every - report and every relay `/metrics` scrape, which walks the registry a second - time for refusals. Wire counters are billed by embedders, so this walk is - needed by design. If it turns out significant at realistic scale, apply the - light fix: keep `WireInner.last` in atomics, and skip the `current` lock - unless the generation changed. A push design, where writers add straight to - the label row so the drain costs O(labels), needs its own `/quest-plan`. - -No API or wire change. diff --git a/rs/moq-net/benches/stats.rs b/rs/moq-net/benches/stats.rs index 422bfe5b5d7..255078ef208 100644 --- a/rs/moq-net/benches/stats.rs +++ b/rs/moq-net/benches/stats.rs @@ -1,12 +1,12 @@ //! Drain benchmark for the stats registry: [`stats::Registry::report`] refilling -//! one reused report, swept over broadcasts and tiers so a per-drain cost that -//! grows faster than the entries it reports shows up as a slope. +//! one reused report, swept over broadcasts, tiers, labels, sessions and connections +//! so a cost that grows faster than the entries it reports shows up as a slope. //! //! Run with `cargo bench -p moq-net --bench stats`. use criterion::{BenchmarkId, Criterion, Throughput, criterion_group, criterion_main}; use futures::FutureExt; -use moq_net::{announce, broadcast, fuzz, origin, stats}; +use moq_net::{announce, broadcast, fuzz, origin, stats, transport}; /// `(broadcasts, tiers)` shapes; every broadcast records under every tier. const SHAPES: [(usize, usize); 4] = [(100, 1), (1_000, 1), (100, 10), (1_000, 10)]; @@ -96,23 +96,46 @@ fn write(c: &mut Criterion) { c.bench_function("stats/write/session-bytes", |b| b.iter(|| session.bytes(1))); } +/// Include the live transport walk, independently swept over labels, sessions and connections. fn session_report(c: &mut Criterion) { let mut group = c.benchmark_group("stats/session-report"); for labels in [1, 100, 1_000] { for sessions in [1, 10] { - let registry = stats::Registry::new(stats::Config::new()); - let handle = registry.tier(stats::Tier::default()); - let contexts: Vec<_> = (0..labels) - .flat_map(|label| { - let handle = handle.clone(); - (0..sessions).map(move |_| handle.session(format!("project/key/{label}"))) - }) - .collect(); - let mut report = stats::Report::default(); - group.bench_function(BenchmarkId::from_parameter(format!("{labels}x{sessions}")), |b| { - b.iter(|| registry.report(&mut report)); - }); - std::hint::black_box(contexts); + for connections in [0, 1, 4] { + let registry = stats::Registry::new(stats::Config::new()); + let handle = registry.tier(stats::Tier::default()); + let contexts: Vec<_> = (0..labels) + .flat_map(|label| { + let handle = handle.clone(); + (0..sessions).map(move |_| handle.session(format!("project/key/{label}"))) + }) + .collect(); + let wires: Vec<_> = contexts + .iter() + .flat_map(|session| { + (0..connections).map(move |_| { + let counters = transport::Wire::new(); + counters.set_sent_bytes(100); + counters.set_received_bytes(200); + session.wire(counters) + }) + }) + .collect(); + let mut report = stats::Report::default(); + registry.report(&mut report); + assert_eq!(report.sessions.len(), labels); + for row in &report.sessions { + assert_eq!(row.presence.sessions_started, sessions as u64); + assert_eq!(row.presence.wire_tx, (sessions * connections * 100) as u64); + assert_eq!(row.presence.wire_rx, (sessions * connections * 200) as u64); + } + group.throughput(Throughput::Elements((labels * sessions) as u64)); + group.bench_function( + BenchmarkId::from_parameter(format!("{labels}x{sessions}/connections={connections}")), + |b| b.iter(|| registry.report(&mut report)), + ); + std::hint::black_box((contexts, wires)); + } } } group.finish(); diff --git a/rs/moq-net/src/stats.rs b/rs/moq-net/src/stats.rs index 1d003dd43e9..44d1845277c 100644 --- a/rs/moq-net/src/stats.rs +++ b/rs/moq-net/src/stats.rs @@ -137,7 +137,7 @@ use std::{ fmt, sync::{ Arc, Mutex, MutexGuard, PoisonError, Weak, - atomic::{AtomicU64, AtomicUsize, Ordering}, + atomic::{AtomicU64, AtomicUsize, Ordering, fence}, }, }; @@ -317,15 +317,19 @@ impl SessionCounters { } } - /// Read the gauge into a [`Presence`]. Ended is loaded with `Acquire` + /// Read the gauge and whether all its payload meters have ended. Ended is loaded with `Acquire` /// before started with `Relaxed`, the same pairing as [`Counters::snapshot`], /// so the readout never shows `ended > started`. - fn snapshot(&self) -> Presence { + fn snapshot(&self) -> (Presence, bool) { let sessions_failed = self.sessions_failed.snapshot(); let sessions_ended = self.sessions_ended.load(Ordering::Acquire); let sessions_started = self.sessions_started.load(Ordering::Relaxed); - let (sent_bytes, received_bytes) = self.bytes().read(); - Presence { + let (sent_bytes, received_bytes, empty) = { + let mut bytes = self.bytes(); + let (sent, received) = bytes.read(); + (sent, received, bytes.live.is_empty()) + }; + let presence = Presence { sessions_refused: self.sessions_refused.snapshot(), sessions_failed, requests: self.requests.load(Ordering::Relaxed), @@ -337,7 +341,8 @@ impl SessionCounters { received_bytes, announces_peak: self.announces_peak.load(Ordering::Relaxed), subscriptions_peak: self.subscriptions_peak.load(Ordering::Relaxed), - } + }; + (presence, empty) } fn bytes(&self) -> MutexGuard<'_, SessionBytes> { @@ -402,8 +407,12 @@ impl SessionBytes { let Self { sent, received, live } = self; let (mut live_sent, mut live_received) = (0, 0); live.retain(|_, arrival| { - // Unique means the session and every meter it handed out have dropped. - let ended = Arc::get_mut(&mut arrival.traffic).is_some(); + // No Weak exists, so a unique arrival cannot gain another writer. + let ended = Arc::strong_count(&arrival.traffic) == 1; + if ended { + // Pair with the last writer's Arc drop before loading its final relaxed counters. + fence(Ordering::Acquire); + } let (now_sent, now_received) = arrival.traffic.load(); let (delta_sent, delta_received) = (now_sent - arrival.sent, now_received - arrival.received); if ended { @@ -1204,7 +1213,7 @@ impl Registry { for (tier, labels) in sessions.iter() { let totals = snap.sessions.entry(tier.clone()).or_default(); for counters in labels.values() { - totals.add(counters.snapshot()); + totals.add(counters.snapshot().0); } } } @@ -1262,8 +1271,8 @@ impl Registry { // Nothing can join a row only the map holds (that needs the lock held here), // but a meter outliving its session still writes until its last clone drops. let unheld = Arc::strong_count(counters) == 1; - let presence = counters.snapshot(); - let closed = unheld && counters.bytes().live.is_empty(); + let (presence, empty) = counters.snapshot(); + let closed = unheld && empty; report.sessions.push(SessionEntry { tier: tier.clone(), label: label.clone(), @@ -2237,7 +2246,7 @@ mod tests { .sessions .lock() .get(tier) - .and_then(|labels| labels.get(&PathOwned::from(label.to_string())).map(|c| c.snapshot())) + .and_then(|labels| labels.get(&PathOwned::from(label.to_string())).map(|c| c.snapshot().0)) } fn test_stats() -> Registry { @@ -2572,13 +2581,17 @@ mod tests { } }); let mut previous = 0; + let mut previous_received = 0; while !worker.is_finished() { - let bytes: u64 = stats - .snapshot() - .traffic() - .iter() - .map(|(_, _, traffic)| traffic.bytes) - .sum(); + let snap = stats.snapshot(); + let received = snap.sessions().iter().map(|(_, row)| row.received_bytes).sum::(); + assert!( + received >= previous_received, + "retiring a session must retain its final payload bytes" + ); + assert!(received <= 256); + previous_received = received; + let bytes: u64 = snap.traffic().iter().map(|(_, _, traffic)| traffic.bytes).sum(); assert!( bytes >= previous, "retiring an entry must not double-count or lose its bytes" @@ -2594,6 +2607,7 @@ mod tests { ); assert_eq!(snap.sessions()[0].1.sessions_started, 256); assert_eq!(snap.sessions()[0].1.sessions_ended, 256); + assert_eq!(snap.sessions()[0].1.received_bytes, 256); assert!(stats.shared().entries.lock().is_empty()); assert!(stats.shared().sessions.lock().is_empty()); let retired = stats.shared().retired.lock(); @@ -2679,6 +2693,7 @@ mod tests { // check the line order. let body_start = src.find("fn snapshot(&self) -> Traffic").expect("snapshot fn present"); let body = &src[body_start..]; + let body = &body[..body.find("\n\t}\n").expect("snapshot fn end")]; let ended_pos = body.find("self.announces_ended.load").expect("announces_ended load"); let started_pos = body .find("self.announces_started.load") @@ -3077,9 +3092,10 @@ mod tests { // `sessions_ended > sessions_started` leak into a readout. let src = include_str!("stats.rs"); let body_start = src - .find("fn snapshot(&self) -> Presence") + .find("fn snapshot(&self) -> (Presence, bool)") .expect("SessionCounters::snapshot fn present"); let body = &src[body_start..]; + let body = &body[..body.find("\n\t}\n").expect("SessionCounters::snapshot fn end")]; let ended_pos = body.find("self.sessions_ended.load").expect("sessions_ended load"); let started_pos = body.find("self.sessions_started.load").expect("sessions_started load"); assert!(