Skip to content

perf(cyper-axum): avoid buffered TCP compatibility I/O - #70

Open
Xerxes-2 wants to merge 1 commit into
compio-rs:masterfrom
Xerxes-2:perf/websocket-io
Open

perf(cyper-axum): avoid buffered TCP compatibility I/O#70
Xerxes-2 wants to merge 1 commit into
compio-rs:masterfrom
Xerxes-2:perf/websocket-io

Conversation

@Xerxes-2

@Xerxes-2 Xerxes-2 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Uses readiness-based TCP I/O on every platform, where hyper writes to the socket directly instead of going through Compio's futures adapter and its intermediate buffer. Accepted sockets reuse their shared descriptor via to_poll_fd(); if that fails the buffered path is kept.

Relies on the vectored-write and half-close impls for PollFd from compio#993, released in compio-runtime 0.12.4. Cargo resolves that by default, but the compio facade still declares compio-runtime = "0.12.3", so the requirement cannot be expressed transitively — see the note below.

Closes #53.

Latency

Measured on Linux. Single connection, tokio-tungstenite client, 40 000 round trips per series (6 000 at 1 MiB).

latency distribution

Payload server p50 p90 p99 p99.9 max
64 B master 6.77 µs 7.38 µs 8.76 µs 11.30 µs 45.8 µs
this PR 5.99 µs 6.64 µs 7.71 µs 10.34 µs 43.0 µs
axum 5.96 µs 6.52 µs 7.07 µs 8.43 µs 20.9 µs
16 KiB master 12.28 µs 13.53 µs 16.88 µs 33.32 µs 53.8 µs
this PR 9.16 µs 10.15 µs 12.52 µs 21.73 µs 42.1 µs
axum 7.07 µs 7.60 µs 8.05 µs 10.55 µs 21.0 µs
1 MiB master 377.96 µs 391.71 µs 427.77 µs 539.86 µs 573.6 µs
this PR 149.29 µs 154.46 µs 203.44 µs 248.35 µs 286.4 µs
axum 186.43 µs 190.86 µs 197.92 µs 215.55 µs 367.3 µs

Every percentile improves on master, including max. At 1 MiB the whole distribution moves left and stays ahead of Axum out to the tail. At 64 B the median is now level with Axum; at 16 KiB a gap remains, and in both cases it is concentrated in the tail rather than the median — the same upward bend shows up at every payload size, which looks more like an occasional extra wakeup than per-byte cost. Worth a follow-up.

Concurrency

A separate measurement, so small differences from the table above are run-to-run variance.

Payload × conns round trips/conn master p50 this PR p50 axum p50 master MiB/s this PR MiB/s
64 B × 64 313 265 µs 224 µs 208 µs 29 35
64 B × 256 79 877 µs 304 µs 280 µs 24 28
64 B × 1024 20 396 µs 349 µs 342 µs 19 22
16 KiB × 64 125 494 µs 253 µs 233 µs 4022 7799
16 KiB × 256 32 2.21 ms 316 µs 1.09 ms 3515 6780
16 KiB × 1024 8 20.55 ms 385 µs 353 µs 1389 4325

Medians of 4 alternating runs per build. The buffered path degrades sharply past 256 connections at 16 KiB while this branch stays flat — at 16 KiB × 1024 master measures 20.2–21.5 ms against 374–398 µs here, consistent across every run on both sides. Note this compares two implementations, not two I/O models: master chunks every message through an 8 KiB buffer, and raising that buffer alone recovers much of the difference.

Only the 64-connection rows are safe to read as an axum comparison. Axum reproduces to within 1.00–1.02× there, but swings several-fold at 256, so its 1.09 ms is not meaningful. Two separate reasons: ws_latency splits a fixed round-trip budget across connections, leaving too few samples per connection to support a tail percentile at these depths (8 at 16 KiB × 1024, addressed in #71); and axum keeps swinging at 256 even with 500 samples each, which looks like real behaviour rather than a measurement artefact. The master-versus-branch gap is unaffected either way, since both sides see identical sampling. The compio client saturates around 1024 connections — 17–27 ms for both servers — so those rows say nothing about the server and are omitted.

HTTP

Vectored writes let hyper keep its Queue strategy instead of flattening the body into its own buffer, so plain HTTP benefits too: a 1 MiB keep-alive response body goes from 73.8 µs to 58.4 µs (median of 3 runs).

Benchmark environment
  • AMD Ryzen 9 9950X3D (16C/32T), 92 GiB RAM, performance governor
  • CachyOS, Linux 7.1.6-1-cachyos x86_64, io_uring enabled
  • rustc 1.97.1, release profile
  • Client and server on the same host over loopback, each single-threaded
  • Both pinned to one CCD (taskset -c 0-7). A mask spanning both CCDs makes round trips bimodal — at 16 KiB, ~9.6 µs when co-resident against ~20.7 µs across the fabric — which flips run-level p50. This affects master and this branch alike.
  • Closed loop, one request in flight per connection: service time at a given queue depth, not response time at a target rate.
  • Concurrency numbers come from the ws_latency benchmark in bench(cyper-axum): add websocket latency benchmark #69, with CELLS extended to 256 and 1024 connections; the single-connection distributions come from a standalone dumper so every round trip is recorded individually.

@Xerxes-2
Xerxes-2 force-pushed the perf/websocket-io branch 11 times, most recently from c464e9c to 65c61dd Compare August 4, 2026 13:44
Comment thread cyper-axum/Cargo.toml Outdated
@Xerxes-2
Xerxes-2 force-pushed the perf/websocket-io branch from 65c61dd to d5fecd6 Compare August 4, 2026 14:21
@Berrysoft

Copy link
Copy Markdown
Member

Oh, please... don't write response with LLM.

@Xerxes-2

Xerxes-2 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

My bad, Claude Opus is so eager to do this, deleted

@Berrysoft Berrysoft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make all targets use the PollFd, not only Linux and Android.

@Xerxes-2
Xerxes-2 force-pushed the perf/websocket-io branch from d5fecd6 to e612114 Compare August 4, 2026 14:30
@Xerxes-2

Xerxes-2 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Done, feature gate is removed for using PollFd

Reuse the accepted socket's descriptor through `PollFd` so hyper writes to it
directly instead of going through Compio's futures adapter and its intermediate
buffer, keeping vectored writes and TCP half-close.

Unix only: on Windows `PollFd` funnels every readiness kind through a single
`WSAEventSelect` registration, which does not survive hyper reading and writing
one socket concurrently.
@Xerxes-2
Xerxes-2 force-pushed the perf/websocket-io branch from e612114 to caa3348 Compare August 4, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cyper-axum: Performance regression(?) on WS with large payload

2 participants