Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions quest/m1/release-perf/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Main is no slower than release
# [S] Main is no slower than release

## Goal

Expand Down Expand Up @@ -266,11 +266,26 @@ Decided 2026-10-30 (maintainer), in

- The sparse rows are accepted as that per-group model cost.
- The fanout mode is accidental and is fixed before the cut, in
[Fanout lock contention](/quest/m1/release-perf/fanout-lock.md).
[moq-dev/moq#7493](https://github.com/moq-dev/moq/pull/7493).

That fix cut the shared track lock from about 155 acquisitions per delivered
group to 34 (release takes 31), counted on one worker. Most were the
logical-track reader re-reading every copy's end, the retention edge, and a
probe's fills on each of several reads per wake, plus a whole track walk on
every group machine's wake. Its own pairing, 14 alternating rounds at load 7
to 28, found no high-mode round on either side: relay CPU 0.066 to 0.090 cores
against release's 0.062 to 0.104, with mixed-sign paired deltas inside
release's own spread, and 5 to 36% fewer user-space instructions than release
in every round. The re-pair above, on a quieter host, is what remains.

Decided 2026-11-01 (maintainer), in
[moq-dev/moq#7493](https://github.com/moq-dev/moq/pull/7493): ✅ the new
`session_fanout_threads` Criterion case joins the release pairing through a
bench-only backport to `release`, which carries it once that PR lands. Bench-only
backports are allowed from now on.

## Required

- [Fanout lock contention](/quest/m1/release-perf/fanout-lock.md) - relay fanout subscribers stop contending one track lock, so main's relay CPU matches release's in every round
- [Relay CPU per frame since #5926](/quest/m1/release-perf/relay-cpu-bisect.md) - the 18-56% rise in relay instructions per delivered frame is bisected, attributed, and its avoidable cost fixed

## Related
Expand Down
61 changes: 0 additions & 61 deletions quest/m1/release-perf/fanout-lock.md

This file was deleted.

2 changes: 1 addition & 1 deletion quest/m1/release-perf/relay-cpu-bisect.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-a
- Bisect `cdff3c8f95..main` on chat over io_uring first (the largest rise and the tightest ranges), with a step size that keeps each build's cost bounded, then confirm the culprits on video and on the tokio workers. A rise that shows on io_uring but not on the tokio workers is attributed on its own, not folded into a shared culprit.
- Attribute each step with `perf record` at a fixed period, as #7505 did, before deciding fix or accept. Record accepted costs in the epic's Plan with the feature they pay for.
- Commit the driver this needs as a deterministic mode of `just bench-profile` or `just bench-runtime` (pinned cores, instructions per delivered frame, fixed-period samples) instead of another throwaway script; #7505's driver on `bench/relay.sh` was not committed.
- [Fanout lock contention](/quest/m1/release-perf/fanout-lock.md) covers the bimodal futex cost on the fanout workload; don't fix it twice. A bisect step that lands on the same lock goes there.
- [moq-dev/moq#7493](https://github.com/moq-dev/moq/pull/7493) fixed the bimodal futex cost on the fanout workload; don't fix it twice. A bisect step that lands on the same lock starts from that PR's lock counts.

Public API: none expected. Wire: none.

Expand Down
9 changes: 5 additions & 4 deletions rs/moq-net/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,13 @@ futures = { workspace = true }
kio = { path = "../kio", features = ["runtime", "sim"] }
proptest = { workspace = true }
serde_json = { workspace = true }
# The benches run on tokio. test-util (tokio::time::pause/advance) is test-only, so
# it must not leak into the normal dependency feature set.
tokio = { workspace = true, features = ["rt", "test-util", "time"] }
# The benches run on tokio, the session fanout bench across worker threads. test-util
# (tokio::time::pause/advance) is test-only, so it must not leak into the normal
# dependency feature set.
tokio = { workspace = true, features = ["rt", "rt-multi-thread", "test-util", "time"] }
tracing-test = { workspace = true }

# Optional per-thread CPU timing in the session benchmark.
# Thread and process CPU timing in the session benchmark.
[target.'cfg(unix)'.dev-dependencies]
libc = { workspace = true }

Expand Down
1 change: 1 addition & 0 deletions rs/moq-net/benches/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ criterion_group!(
session::fetch_range::<Model>,
session::range_updates::<Model>,
session::withdrawal::<Model>,
session::fanout_threads::<Model>,
session::idle
);
criterion_main!(benches);
107 changes: 88 additions & 19 deletions rs/moq-net/benches/session/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,13 @@
//! holds others open, swept over sessions and requests held per session, so a slope
//! here is per-request bookkeeping growing with the sessions or the requests.
//!
//! Everything runs on one current-thread runtime, so the only work measured is
//! `session_fanout_threads` serves one broadcast's one-frame groups through one relay to 64
//! viewers on multi-threaded runtimes, timed in process CPU. Each group wakes every
//! subscription's serve task at once, so a read path that makes them take one shared lock
//! many times pays for it in lock waits and wakes across workers: the `workers=16` rows
//! climb above the `workers=1` ones.
//!
//! Everything else runs on one current-thread runtime, so the only work measured is
//! protocol and relay work, never scheduling across threads.
//!
//! Run with `cargo bench -p moq-net --features fuzz --bench session`.
Expand Down Expand Up @@ -283,40 +289,64 @@ fn runtime() -> tokio::runtime::Runtime {
.unwrap()
}

/// One round's clock; the CPU variant excludes scheduling delays from other processes.
/// A multi-threaded runtime with `workers` threads, for the scenarios timed in process CPU.
fn threaded(workers: usize) -> tokio::runtime::Runtime {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(workers)
.enable_time()
.build()
.unwrap()
}

/// What a round is timed in.
#[derive(Clone, Copy)]
enum Clock {
Wall,
/// This thread's CPU time, excluding scheduling delays from other processes.
Thread,
/// Every thread's CPU time, for a multi-threaded runtime: lock waits and wakes across
/// its workers count too.
Process,
}

/// One round's clock.
enum MeasurementClock {
Wall(Instant),
#[cfg(unix)]
Cpu(Duration),
Cpu(libc::clockid_t, Duration),
}

impl MeasurementClock {
fn start(cpu: bool) -> Self {
if cpu {
#[cfg(unix)]
return Self::Cpu(Self::cpu());
#[cfg(not(unix))]
panic!("SESSION_CPU requires Unix thread CPU clocks");
fn start(clock: Clock) -> Self {
#[cfg(unix)]
let id = match clock {
Clock::Wall => return Self::Wall(Instant::now()),
Clock::Thread => libc::CLOCK_THREAD_CPUTIME_ID,
Clock::Process => libc::CLOCK_PROCESS_CPUTIME_ID,
};
#[cfg(unix)]
return Self::Cpu(id, Self::cpu(id));
#[cfg(not(unix))]
match clock {
Clock::Wall => Self::Wall(Instant::now()),
_ => panic!("CPU-timed rounds require Unix CPU clocks"),
}
Self::Wall(Instant::now())
}

fn elapsed(self) -> Duration {
match self {
Self::Wall(start) => start.elapsed(),
#[cfg(unix)]
Self::Cpu(start) => Self::cpu()
.checked_sub(start)
.expect("thread CPU clock moved backwards"),
Self::Cpu(id, start) => Self::cpu(id).checked_sub(start).expect("CPU clock moved backwards"),
}
}

#[cfg(unix)]
fn cpu() -> Duration {
fn cpu(id: libc::clockid_t) -> Duration {
let mut time = libc::timespec { tv_sec: 0, tv_nsec: 0 };
// SAFETY: `time` is a valid, initialized output pointer for this supported clock.
let result = unsafe { libc::clock_gettime(libc::CLOCK_THREAD_CPUTIME_ID, &mut time) };
assert_eq!(result, 0, "thread CPU clock: {}", std::io::Error::last_os_error());
let result = unsafe { libc::clock_gettime(id, &mut time) };
assert_eq!(result, 0, "CPU clock: {}", std::io::Error::last_os_error());
Duration::new(
u64::try_from(time.tv_sec).unwrap(),
u32::try_from(time.tv_nsec).unwrap(),
Expand Down Expand Up @@ -600,7 +630,7 @@ struct Room<B: Backend> {
/// The group each viewer's subscriber is reading, when paced, flattened in viewer order.
reading: Vec<Option<group::Consumer>>,
cache_now: moq_net::time::Instant,
cpu: bool,
clock: Clock,
}

impl<B: Backend> Room<B> {
Expand Down Expand Up @@ -667,7 +697,10 @@ impl<B: Backend> Room<B> {
writing: Vec::new(),
reading: Vec::new(),
cache_now: moq_net::time::Instant::now(),
cpu: std::env::var_os("SESSION_CPU").is_some(),
clock: match std::env::var_os("SESSION_CPU") {
Some(_) => Clock::Thread,
None => Clock::Wall,
},
};
room.writing.resize_with(room.watched.len(), || None);
room.reading.resize_with(shape.viewers * shape.watch, || None);
Expand Down Expand Up @@ -848,7 +881,7 @@ impl<B: Backend> Room<B> {
let mut next_report = Duration::from_secs(10);
for _ in 0..iters {
self.tick();
let start = MeasurementClock::start(self.cpu);
let start = MeasurementClock::start(self.clock);
self.round().await;
elapsed += start.elapsed();
if report && began.elapsed() >= next_report {
Expand Down Expand Up @@ -1415,6 +1448,42 @@ impl IdleRoom {
}
}

/// One relay serving one broadcast's one-frame groups to 64 viewers, as the relay load
/// harness's `fanout` workload does, on multi-threaded runtimes timed in process CPU; see
/// the module docs.
pub(crate) fn fanout_threads<B: Backend>(c: &mut Criterion) {
// Process CPU is read through a Unix clock.
if !cfg!(unix) || std::env::var_os("SESSION_ALLOCS").is_some() || std::env::var_os("SESSION_EXPIRY_ONLY").is_some()
{
return;
}
let shape = Shape {
publishers: 1,
viewers: 64,
frame: 400,
group_frames: 1,
..Shape::BASE
};
let mut group = c.benchmark_group(format!("{}session_fanout_threads", B::PREFIX));
// Groups delivered per round: one to each viewer.
group.throughput(Throughput::Elements(shape.viewers as u64));
for workers in [1, 16] {
let rt = threaded(workers);
for version in ["moq-lite-06", "moq-lite-07-wip"] {
let mut room = None;
group.bench_function(format!("{version}/workers={workers}"), |b| {
let room = room.get_or_insert_with(|| {
let mut room = rt.block_on(Room::<B>::new(version, shape));
room.clock = Clock::Process;
room
});
b.iter_custom(|iters| rt.block_on(room.measure(iters)))
});
}
}
group.finish();
}

pub(crate) fn idle(c: &mut Criterion) {
let mut group = c.benchmark_group("session_idle");
for version in ["moq-lite-06", "moq-transport-22"] {
Expand Down
34 changes: 34 additions & 0 deletions rs/moq-net/src/lite/publisher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ use crate::{
priority::{Priority, PriorityHandle, PriorityQueue},
track::Order,
transport::poll::{GroupOf, SendGroup as _, SendOptions},
util::Idle,
};

use super::Version;
Expand Down Expand Up @@ -3990,6 +3991,8 @@ struct TrackRun {
opening: Option<GroupServe>,
/// The groups with a stream, serving or awaiting the acknowledgement of their FIN.
children: kio::Tasks<GroupServe>,
/// The track side's last poll, if it parked; see [`Self::poll_step`].
idle: Option<Box<Idle>>,
}

impl TrackRun {
Expand Down Expand Up @@ -4029,6 +4032,7 @@ impl TrackRun {
datagrams,
opening: None,
children: kio::Tasks::new(),
idle: None,
}
}

Expand All @@ -4046,6 +4050,9 @@ impl TrackRun {
/// what lets relays pause an upstream subscription across consumer churn without
/// tearing it down. Lite-07 ends one whose every range is bounded once it is covered.
fn update(&mut self, upd: lite::SubscribeUpdate) {
// The update moves the cursor and the bounds the track side judges against, which
// notifies nothing it parked on.
Idle::disarm(&mut self.idle);
// Every update restates the wanted group. Past our largest it is a join, which
// repeats the pending ask; at or below it the group already exists.
if upd
Expand Down Expand Up @@ -4170,14 +4177,22 @@ impl TrackRun {
if let Some(opening) = &mut self.opening
&& let Poll::Ready(res) = opening.poll_open(&mut self.ctx.session, self.ctx.group.as_ref(), waiter)
{
// The track side takes groups again.
Idle::disarm(&mut self.idle);
let opening = self.opening.take().expect("polled above");
match res {
Ok(()) => self.children.push(opening),
Err(err) => opening.settle_failed(&err),
}
}
// Drive the in-flight group machines; completions just retire.
let running = self.children.len();
let _ = self.children.poll(waiter);
// A retired machine releases its group, and the track side waits on the machines
// to finish or cover the subscription, neither of which the track notifies.
if self.children.len() < running {
Idle::disarm(&mut self.idle);
}
// A group whose header does not fit this peer's varint wrote nothing. The
// reset is on a stream the subscriber never tied to this subscription, so
// forward it here or the reader waits forever.
Expand All @@ -4193,6 +4208,25 @@ impl TrackRun {
return Poll::Ready(Ok(ControlFlow::Continue(())));
}

// Most wakes are a group machine's frame or the writer's credit. Nothing the track
// side parked on fired then, so it would park again: skip it rather than lock the
// shared track state every subscriber of the track reads through.
if Idle::parked(&self.idle, (), waiter) {
return Poll::Pending;
}
let mut idle = Idle::start(&mut self.idle, waiter);
let polled = self.poll_track(writer, idle.waiter());
idle.finish(&mut self.idle, (), polled.is_pending());
polled
}

/// The track side of [`Self::poll_step`]: forward drops, resolve the start, and take the
/// next group, datagram, or boundary. Every wait here parks on the track subscriber.
fn poll_track(
&mut self,
writer: &mut Writer<crate::transport::boxed::SendStream, Version>,
waiter: &kio::Waiter,
) -> Poll<Result<ControlFlow<()>, Error>> {
if self.poll_forward_drop(writer, waiter)? {
return Poll::Ready(Ok(ControlFlow::Continue(())));
}
Expand Down
Loading