Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ nix develop --command just bench-runtime 5 16
```

Runtime comparison requires Linux because io\_uring and relay process metrics
come from Linux interfaces. The default worker count is the number of online
logical CPUs.
come from Linux interfaces. The default worker count is the number of logical
CPUs the run may use.

Sweep the relay's QUIC shard count against the connection count, for the
room (`video`: every connection publishes and subscribes) and `fanout` (one
Expand All @@ -78,7 +78,7 @@ nix develop --command just bench-shard
nix develop --command just bench-shard "0 1 8 24" "64 512"
```

Zero shards is the shared runtime with one Tokio thread per logical CPU; any
Zero shards is the shared runtime with one Tokio thread per usable CPU; any
other count is that many unpinned Tokio workers. The table reports CPU per
connection, so a steering, dispatch, or per-shard cost that grows with either
axis reads as a slope. Linux only, since nothing else steers a reuseport
Expand Down Expand Up @@ -145,6 +145,13 @@ work.
Benchmark output is informational and machine-specific. Crashes, zero delivery,
and invalid samples still fail the command.

On a hybrid CPU the same case measures far apart on performance and efficiency
cores, so `run.sh` pins itself, and every build and benchmark process it starts,
to the performance cores wherever the kernel lists them
(`/sys/devices/cpu_core/cpus` on Intel hybrid parts), within the CPUs it was
already allowed, so an outer `taskset` or cpuset still narrows it. The run prints the pinned cores beside its result and records
them as `affinity` in the metadata. A CPU with one core type runs unpinned.

BASE comparisons run paired rounds (default three, `MOQ_BENCH_RUNTIME_ROUNDS`),
alternating base/current order. Startup remains outside the final five-second
throughput window. `MOQ_BENCH_ARTIFACTS` chooses the evidence parent (default
Expand Down
35 changes: 33 additions & 2 deletions bench/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,34 @@ if [[ ! $ROUNDS =~ ^[0-9]+$ ]] || ((10#$ROUNDS < 1)); then
exit 1
fi
ROUNDS=$((10#$ROUNDS))

# One case measures far apart on performance and efficiency cores, so on a
# hybrid CPU pin this shell, and with it every build and benchmark process, to
# the performance cores it may already use. The kernel only lists them on Intel
# hybrid parts.
PINNED=
if [[ -r /sys/devices/cpu_core/cpus ]]; then
PINNED=$(
python3 - "$$" <<'EOF'
import os, sys
pid = int(sys.argv[1])
cores = set()
for part in open("/sys/devices/cpu_core/cpus").read().strip().split(","):
first, _, last = part.partition("-")
cores.update(range(int(first), int(last or first) + 1))
allowed = cores & os.sched_getaffinity(pid)
if not allowed:
sys.exit("cannot pin to the performance cores: none are in the allowed CPU set")
os.sched_setaffinity(pid, allowed)
print(",".join(map(str, sorted(allowed))))
EOF
)
fi

# nproc counts the allowed CPUs, unless OpenMP variables override it.
usable_cpus() {
env -u OMP_NUM_THREADS -u OMP_THREAD_LIMIT nproc
}
case $MODE in
--audio)
BASE=
Expand All @@ -43,7 +71,7 @@ case $MODE in
BASE=
RUNTIME_WORKERS=${MOQ_BENCH_RUNTIME_WORKERS:-}
if [[ -z $RUNTIME_WORKERS ]]; then
RUNTIME_WORKERS=$(getconf _NPROCESSORS_ONLN)
RUNTIME_WORKERS=$(usable_cpus)
if ((RUNTIME_WORKERS > 256)); then
RUNTIME_WORKERS=256
fi
Expand Down Expand Up @@ -115,6 +143,9 @@ cleanup() {
exec 3>&- 4>&-
fi
printf 'Benchmark evidence retained: %s\n' "$RUN"
if [[ -n $PINNED ]]; then
printf 'Pinned to performance cores: %s\n' "$PINNED"
fi
}
trap cleanup EXIT
trap 'exit 130' INT
Expand Down Expand Up @@ -522,7 +553,7 @@ run_shard_sweep() {
HOST_BIN=$CURRENT_TARGET/release/moq-bench-host

local cpus workload connections shards label
cpus=$(getconf _NPROCESSORS_ONLN)
cpus=$(usable_cpus)
for workload in video fanout; do
for connections in $SHARD_CONNECTIONS; do
for shards in $SHARD_COUNTS; do
Expand Down
1 change: 0 additions & 1 deletion quest/m1/quic/coverage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ and the harness defaults to Cubic while production runs BBR3.
- [Characterization tests](/quest/m1/quic/coverage/characterize.md) - golden scheduling order, and every uncovered frame, reset, limit, and endpoint path, on today's harness
- [Plaintext fuzz crypto](/quest/m1/quic/coverage/fuzz-crypto.md) - a fuzz-only plaintext session so the connection target skips TLS and its injected packets decrypt
- [Richer simulation workloads](/quest/m1/quic/coverage/sim-workloads.md) - resets, bidi streams, datagrams, and migration in the random workloads, each with its own invariant
- [Pin benchmark cores](/quest/m1/quic/coverage/bench-pin.md) - local base/head comparisons run on one core type on hybrid CPUs

## Related

Expand Down
20 changes: 0 additions & 20 deletions quest/m1/quic/coverage/bench-pin.md

This file was deleted.