Repository navigation
Description
Motivation
The first big customer plans to use MoQ as a message bus for chat. Egress per connection is tiny, so the cost driver is CPU per connection and CPU per message, not bandwidth. The current relay runs one work-stealing tokio runtime: every packet potentially crosses threads, every wakeup is a candidate context switch, and every UDP datagram is a syscall. The goal is a thread-per-core mode where each connection is pinned to one thread for its whole life, I/O is batched via io-uring, and incoming packets are steered to the owning thread in the kernel.
Media workloads (1:1 and 1:N) must not regress and should also benefit.
What the codebase already gives us (verified)
moq-netis already runtime-agnostic ondev. refactor(net)!: require the poll-based transport interface #2736 (refactor(net)!: require the poll-based transport interface) drives transports purely throughweb_transport_trait::poll::{Session, SendStream, RecvStream}(v0.4).moq-nethas no tokio dependency; every future is kio-based and a sessionDrivercan be stepped withkio::Waiterfrom any executor. This part of the plan is confirmed, not speculative.- The one remaining runtime tie is time.
kio::time::Deadline(andmoq-net's bandwidth sampling, control-stream timeout, and subscription linger) bottom out inweb_async::time, which is tokio's time driver on native and panics outside a tokio runtime. A non-tokio runtime needs a pluggable timer backend in kio. - Send bounds: on native,
moq-netboxes internal futures asSend(MaybeSendBox=BoxFuture); only wasm gets theLocalBoxFuturepath. Thread-per-core does not require!Send(pinnedSendtypes work; locks just become uncontended), but!Sendinternals (Rc, thread-local wakers) are off the table until that cfg split is widened. - A quiche backend already exists (
web-transport-quiche, thequichefeature ofmoq-native), including raw QUIC. quiche is sans-IO by design: the application owns sockets, timers, and packet pacing, which is exactly the contract an io-uring driver wants. moq-benchexists and itsF=0mode (lone JSON keyframe groups) is nearly the chat workload already. It has no CPU-side measurement yet.- No
SO_REUSEPORTanywhere today, onlySO_REUSEADDR.
Decisions taken (from design review)
- Success bar: baseline first. Milestone 0 measures where CPU actually goes today, then we commit to a concrete multiplier (e.g. Nx connections per core). No optimization is judged without a number from the rig.
- QUIC stack: quiche. Battle-tested at exactly this architecture (thread-per-core, SO_REUSEPORT, eBPF steering) at Cloudflare. noq-proto and quinn-proto remain possible later since everything goes through
web-transport-trait, but the io-uring driver is built around quiche. - Milestone 1 is thread-per-core tokio, not io-uring. N pinned
current_threadruntimes with per-threadSO_REUSEPORTsockets and eBPF steering validates the architecture, keeps timers and WebSocket working, and isolates how much win comes from pinning alone vs io-uring itself. - Scope: QUIC first, parity later. The new runtime owns QUIC on :443/UDP only. WebSocket/WSS, TCP, UDS, iroh, and cert reload stay on a tokio runtime in the same process. The plan commits to eventually porting the fallbacks so tokio can be dropped from the relay, but not in v1.
- Deployment floor: our own infra. Modern kernels (6.x), CAP_BPF available. The io-uring path hard-requires its features and refuses to start otherwise. Self-hosters keep the tokio relay; no fallback detection logic.
- Send story: decided by data. Ship Milestone 1 with
Sendtypes, profile atomic/lock overhead under the chat workload, and only widen the wasm!Sendpath to native (feature-gated) if it shows up as real CPU. - io-uring shape: direct implementation. Monoio is rejected for this work because its public UDP API does not expose the ancillary-data and batching primitives we require. Build the spike directly on the
io-uringcrate. - WebTransport layer:
web-transport-protoover quiche.web-transport-quicheis coupled totokio-quicheand spawns tokio tasks, so it is not the abstraction for the new runtime. Reuse the protocol state machine fromweb-transport-protoand implement its I/O against sans-I/O quiche. - UDP batching is required, not optional polish. The tokio/control implementation must support
recvmmsg; the io-uring implementation uses multishotrecvmsgwith provided buffers as its equivalent. Both must preserve ancillary data and UDP GRO. Egress must usesendmsgwithUDP_SEGMENTGSO. - Time is an M3 prerequisite. Replace the concrete tokio sleep inside
kio::timewith an executor-provided deadline backend. Tokio and wasm keep their existing behavior; the direct runtime drives deadlines with timeout SQEs or a measured local timer structure.
Architecture sketch
- N worker threads, each pinned to a core. Each binds its own UDP socket to :443 with
SO_REUSEPORT, giving N sockets in one reuseport group. - An
sk_reuseporteBPF program (SK_REUSEPORT+BPF_MAP_TYPE_REUSEPORT_SOCKARRAY) parses the QUIC header and steers by connection ID: server-chosen CIDs encode the owning socket index (a few plaintext bytes plus entropy, standard quiche-style routing). - Initial packets carry a client-chosen DCID we cannot decode, so they hash to an arbitrary socket. Whichever thread receives the Initial owns the connection forever and issues CIDs that encode its own index. No cross-thread handoff, no migration.
- Retransmitted Initials must reach the same thread before our CID takes effect: steer them by hashing the client-chosen DCID consistently in the eBPF program, so retries land on the thread that saw the first packet.
- Each thread runs: io-uring UDP rx (multishot
recvmsgwith a provided-buffer ring, the io-uring equivalent ofrecvmmsg, plus GRO) -> quiche packet processing ->moq-netDrivers polled on a local task queue -> quiche egress -> io-uring UDP tx (sendmsg+UDP_SEGMENTGSO, zerocopy only where it wins). Timers come from timeout SQEs or a measured local timer structure feeding quiche'son_timeoutand kio deadlines. - Fanout (1:N) across threads goes through the existing shared
Originmodel: cross-thread wakeups are memory traffic, not syscalls. Measured, not assumed, in the benchmarks. - The same process keeps a small tokio runtime for WebSocket/TCP fallback, stats, signal handling, and cert hot-reload.
Non-goals
- Routing Initials to threads with spare capacity. Accept hash placement; revisit only if the benchmarks show real imbalance.
- Customer/tenant affinity (SNI, token root, path). Invites hot keys; a busy tenant must be able to span cores.
- Full moq-native parity in the new crate for v1 (see decision 4).
- Fallback detection for hardened/exotic environments (see decision 5).
Milestones
Each milestone is a tracking issue with its own PRs; this issue is the epic. Every milestone ends with the same benchmark suite run on the same rig, so the deltas compose into one story.
M0: Measure. Chat + media benchmark suite and the tokio baseline
The whole project is gated on this: if profiling shows CPU goes to crypto or memcpy rather than syscalls and scheduling, the plan changes before we build anything.
- Extend
moq-benchwith a chat profile: thousands of connections, tiny frames, low per-connection rate,F=0style groups, plus fanout matrices for 1:1 and 1:N (N spanning small rooms to large ones, e.g. 10 / 1k / 100k subscribers). - Keep/refresh the media profiles (video-sized frames, keyframe cadence) for 1:1 and 1:N.
- Add CPU measurement to the harness: CPU per connection, CPU per message, context switches (
perf stat), syscall share, flame graphs of the relay under each profile. Results in a machine-readable format so runs are diffable. - A
justrecipe to run the matrix against a relay; document the reference hardware. - Exit criteria: baseline numbers for the current multi-thread tokio relay are written down, the dominant CPU sinks are identified, and the target multiplier for the project is committed.
M1: Thread-per-core tokio relay
- Relay mode: N pinned threads, each a
current_threadtokio runtime, each with its ownSO_REUSEPORTUDP socket on :443. - The
sk_reuseporteBPF steering program and its loader: CID-encoded socket index for established connections, consistent DCID hash for Initials. This program is runtime-agnostic and carries over unchanged to the io-uring milestones. - CID generation in the quiche backend encodes the thread index (quinn can follow later if we care).
- Wire it behind a relay config flag; the default stays the multi-thread runtime.
- Profile: how much did pinning + no work-stealing buy on the M0 suite? What share of remaining CPU is atomics/locks (decides the
!Sendquestion), syscalls (decides how much io-uring can still win), crypto? - Exit criteria: benchmark delta vs M0 recorded; Send/!Send decision made from the profile; go/no-go on the io-uring investment made from the syscall share.
M2: Crate split. moq-native becomes moq-tokio
- Rename
moq-nativetomoq-tokio(semver break, lands ondevper branch targeting rules). - Extract the runtime-neutral pieces the new runtime will also need (TLS/cert plumbing, config types, address resolution, the CID codec and eBPF loader from M1) into a shared crate so
moq-tokioand the future io-uring crate depend on common code instead of copying it. - Keep
moq-nativeas a deprecated re-export shim only if it is cheap; otherwise a clean break ondevwith migration notes in the PR. - Docs sweep per the cross-package sync table (
doc/lib/rs, examples, justfiles). - Exit criteria: tree builds with
moq-tokio; no behavior change; downstream crates in the workspace migrated.
M3: Prove the direct io-uring data path and runtime prerequisites
Time-boxed. This milestone answers whether the required Linux batching path works and improves the actual quiche workload before the relay integration grows around it. There is no Monoio candidate.
M3a: Replace the native kio::time backend
- Consolidate every
moq-nettimer and clock read behindkio::time. The initial audit found directweb_async::timeuse in bandwidth sampling, GOAWAY enforcement, and the lite PROBE interval in addition to existingDeadlineusers. - Replace
Deadline's concreteweb_async::time::Sleepwith an executor-selected timer registration. Selection must happen at runtime because tokio fallback work and io-uring QUIC coexist in one process. - Keep the public surface small: callers continue to own and poll a deadline. Do not pass a timer provider through every
moq-nettype. - Preserve tokio paused-clock tests and the wasm backend. Add a no-tokio test executor that proves a
moq-netdriver can arm, re-arm, cancel, and fire deadlines without entering a tokio runtime. - Compare timeout SQEs with a local heap/wheel for cancellation churn and pacing precision. The ring is not automatically the winner for thousands of short-lived timers.
- Exit criteria:
kio::time::Deadlineand a representativemoq-netdriver run under the local executor with no tokio runtime; timer precision and cancellation behavior are measured and documented.
M3b: UDP batching ablation benchmark
- Build the direct per-thread loop on the
io-uringcrate. Receive with multishotrecvmsgplus a provided-buffer ring, parse source/destination address, ECN, timestamps, andUDP_GRO, and re-arm cleanly afterIORING_CQE_F_MOREclears orENOBUFS. - Add a
recvmmsgreceive-batching control to the tokio path. Multishotrecvmsgis the io-uring equivalent, but the control is needed to separate receive batching from executor effects. - Send quiche packet batches with
sendmsgand aUDP_SEGMENTcontrol message. Only coalesce packets with the same destination, source, ECN, pacing time, and segment size. - Run an ablation matrix at a fixed offered load: single receive/single send, receive batching only, GSO only, receive batching + GSO, then the direct io-uring path. Toggle GRO separately. Record actual batch sizes, packets/syscall, ring submissions/completions, drops,
ENOBUFS, CPU/message, CPU/Gbps, and latency. - Test chat-sized datagrams and media-sized traffic. Treat zerocopy as a separate experiment because notification CQEs and buffer lifetime may cost more than the copy for small packets.
- Use a dedicated transport benchmark first, then repeat the winning configuration with the M0 workload. The M0 harness speaks MoQ and cannot directly drive a raw echo server without a transport mode/client.
- Exit criteria: all required socket features work on the deployment kernel and NIC; the ablation attributes the gain to receive batching and/or GSO; the direct path improves quiche CPU at the same offered load. If it does not improve end-to-end CPU, stop before M4.
M3c: quiche plus WebTransport proof
- Drive sans-I/O quiche directly from the M3b loop.
- Put
web-transport-protoon top for the HTTP/3 CONNECT handshake and WebTransport session state. Do not depend onweb-transport-quicheortokio-quichein the new path. - Accept a session from an existing client and run bidirectional stream and datagram echo through it. Feed quiche timeout and pacing deadlines through the M3a backend.
- Keep the adapter shaped toward
web_transport_trait::poll, but do not build the complete public transport crate in the spike. - Exit criteria: the same benchmark client completes the WebTransport handshake and echo workload on tokio and direct io-uring; CPU, latency, loss, and syscall/ring-operation deltas are recorded on the same hardware.
M4: Integrate the proven io-uring runtime
- New crate (name TBD in the M4 PR, e.g.
moq-uring): grow the M3 winner into the production runtime. It owns thread spawning/pinning, sockets, the ring, provided-buffer pools, the local executor, and the timer backend selected in M3a. - Extract the runtime-neutral TLS/cert configuration, CID codec, eBPF loader, and socket configuration that remain private in
moq-tokio. The M2 rename landed, but these shared pieces were not extracted. - Implement the WebTransport-over-quiche adapter from M3c as
web_transport_trait::pollsession and stream types, then drive realmoq-netDrivers. - Relay integration behind the same config flag as M1: io-uring threads own QUIC, while the tokio side keeps WebSocket/TCP/UDS, stats, signals, and cert reload. Cluster/upstream QUIC dials happen from the io-uring threads too.
- Exit criteria: relay passes the existing integration and smoke tests in io-uring mode; the full M0 suite records deltas against M1; the required receive batching and send GSO remain enabled and observable.
M5: Hardening and continuous benchmarks
- Nightly benchmark runs on a dedicated box (rig details TBD; cloud CI runners are too noisy for CPU numbers), trended over time, regressions visible per commit range.
- Soak: multi-hour chat + media mixed load, memory stability, CID rotation, connection churn.
- Operational story: qlog in io-uring mode,
moq-statsintegration, graceful drain/shutdown of pinned threads. - Rollout on our fleet behind the flag, then default for our deployment.
moq-tokioremains the default for everyone else. - If M1's profile said the
!Sendrefactor pays: land it here (feature-gatedLocalBoxFuturepath inmoq-net, Rc internals, thread-local kio wakers) and re-measure. - Exit criteria: the M0 target multiplier is met or the gap is explained with a profile; benchmarks run nightly; the chat customer's projected load fits the promised hardware.
Risks and open questions
- WebTransport adapter scope:
web-transport-protosupplies protocol state, but the new path still owns the quiche stream/datagram adapter and itsweb_transport_trait::polllifecycle. Prove the minimal boundary in M3c before designing the production public API. - Cross-thread fanout: 1:N where subscribers live on many threads keeps the shared-memory origin model on the hot path. If M0/M1 show contention there, group fanout may need per-thread egress queues. Measured before designed.
- io_uring security posture is fine for our fleet (decision 5) but permanently excludes some self-host environments; the tokio relay remains the answer there.
- eBPF program lifecycle: pinning, upgrade across relay restarts (reuseport group membership changes when threads restart), and observability when steering goes wrong.
- Timer precision and cancellation: quiche pacing wants fine-grained timers, while a timeout SQE per logical deadline may create excessive submission and cancellation traffic. M3a compares the ring with a local timer structure under realistic churn.
- Provided-buffer ownership: multishot receive depends on correct buffer-ring replenishment. Exhaustion, stale CQEs after cancellation, and GRO segment parsing need counters and stress tests.
- Benchmark interpretation: loopback UDP microbenchmarks prove feature support and give directional data, but only the fixed-load quiche and M0 runs decide whether the runtime is better.
- Benchmark rig: hardware not yet chosen (dedicated bare metal vs a fixed cloud metal instance). Owner input welcome on what we can dedicate.
- Fanout matrix ceilings (is 100k subscribers per broadcast a real chat shape, or is 10k the honest max?).
No wire format changes anywhere in this plan, so no drafts/ updates are expected. Relay config/docs updates ride each milestone PR per the cross-package sync table.
(Written by GPT-5.6)
Activity
coderabbitai commentedon Aug 15, 2026
🔗 Related PRs
#570 - Add WebSocket fallback support [closed]
#928 - (AI) Add support for quiche to moq-native [closed]
#1096 - Switch to qmux with ALPN negotiation and TLS 1.2 [closed]
#1810 - feat(relay): unauthenticated internal listener over qmux (tcp:// + unix://) [merged]
#2161 - feat(moq-native): add quic::Client/quic::Server transport config [merged]
#2423 - refactor!: pre-bump API polish across the release batch [merged]
#2490 - feat(net, relay): GOAWAY with graceful drain and cluster migration [open]
#2500 - fix(net): tear down idle upstream subscriptions [merged]
#2514 - feat(moq-native)!: expand quiche backend parity [merged]
#2556 - fix(net): prefer the newest route so a reconnect takes over immediately [merged]
#2629 - feat(net): implement the MoQ Cluster extension over moq-transport [merged]
#2647 - feat: fail-fast retries: jittered backoff bounded by time, not error type [merged]
#2748 - feat(net): announce namespaces unasked, with a SETUP opt-out [merged]
#2771 - fix(net): serve subscriptions in arrival order so reordered groups aren't dropped [merged]
📝 Issue Planner
Check the box below or use the
@coderabbitai plancommand to generate an implementation plan and prompts that you can use with your favorite coding assistant.🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!
feat(relay): serve QUIC from pinned per-core workers
kixelated commentedon Aug 21, 2026
Preliminary UDP batching benchmark
I ran the existing
noq-udp1.0.1 throughput benchmark as a kernel feasibility and directional check.Environment:
recvmmsgWhat this establishes:
sendmsgwithUDP_SEGMENTGSO works on this kernel and is the dominant low-level win in this test.recvmmsgadds another 14.7% over GSO alone.recvmmsgalone is neutral here because the benchmark sends once and immediately drains, leaving little queued work to batch.This is not the M3 decision benchmark. It measures UDP throughput, not quiche CPU, and loopback avoids the NIC. The next benchmark must hold offered load constant and compare CPU/message and CPU/Gbps for:
recvmsgtorecvmmsg,recvmsgwith provided buffers,The code audit also found that current
tokio-quichealready sends GSO batches withsendmsgand receives GRO with a singlerecvmsg. The new path needs to preserve that GSO behavior, add receive batching, and show an end-to-end quiche CPU improvement before M4.(Written by GPT-5.6)
kixelated commentedon Aug 22, 2026
Plan: starting moq-uring, and the moq-net/kio API changes it needs
Outcome of a design session on the runtime prerequisites (M3a plus the M4 crate shape), assuming #3003 lands. A few decisions amend the epic text; each amendment is called out.
Decisions
1. Explicit runtime injection, no ambient state. (Amends M3a's "executor-selected timer registration", which implied a thread-local or global driver.) Timers reach moq-net through a
Runtimetrait passed explicitly at session construction, quinn-style. No thread-locals, no globals, no link-time registration. moq-tokio, moq-uring, and moq-wasm each supply their runtime inside their own connect/accept helpers, so end users never see the parameter.2. kio sheds time entirely.
kio::timeandkio::tokioare deleted along with the web-async and tokio dependencies; kio 0.6 is pure waker/channel primitives. Time moves tomoq_net::runtime. (kio::time's only consumer is moq-net; the apparent sibling-crate matches were substring hits on "tokio::time".)3. The Runtime trait.
Instantisstd::time::Instanton native (wasm gets a same-API shim), so quiche interop is conversion-free and tokio arms viaInstant::from_std.Sendbound anywhere in the trait. The associatedTransporttype is what makes conditional Send work without feature flags or companion traits: each impl spawns a concrete machine type whose Send-ness it knows. moq-tokio's transport enum isSend, so plaintokio::spawnkeeps working (no LocalSets, no work-stealing sacrifice). moq-uring's machine is!Sendand goes on its pinned local queue. moq-wasm wraps the machine inkio::waitandspawn_locals it.Drivertype is removed.Client::connect(runtime, transport)spawns the machine and awaits the establishment signal;Server::acceptlikewise. Deliberate API break.Clonebecause subsystems store their own handle to mint deadlines; it's a ZST copy or anRcbump.4. No clock on the trait.
Runtimehas nonow(). Clock reads usestd::time::Instant::now()directly, under one discipline: deadline firings gate protocol behavior, clock reads only annotate (group activity stamps, cache epochs, stats, Timestamp anchors). The few sites that comparenow()against armed instants (ietf publisherRefused::offerable, linger idle checks) get refactored to key off deadline-fired state, so virtual-time tests stay coherent without virtualizing the clock.5. Deterministic test runtime.
moq_net::runtime::Testbehind atest-runtimefeature (moq-net dev-depends on itself with the feature enabled, becausecfg(test)items are invisible to thetests/directory's integration crates). Manual clock + deadline wheel + machine queue:advance(Duration)andadvance_to_deadline(), both explicit, no auto-advance. Replaces thetokio::time::pause()magic across moq-net's 67 paused tests; migration is incremental behind a shim.6. Timers are userspace; one park deadline; never per-deadline SQEs. The uring worker keeps a timer wheel/heap and parks with the earliest deadline via the 6.12 absolute-timeout enter argument (zero SQEs). This is what every serious runtime converged on: monoio explicitly rejected per-timer SQEs over cancel churn, glommio uses a
BTreeMap, and quiche hands you exactly onetimeout()per connection anyway. That survey settles the M3a "timeout SQEs vs local structure" comparison; the trait is SQE-agnostic if this ever needs revisiting.7. The !Send plan: sever, de-future, relax. No feature flags; Send-ness flows from types.
Deadlineis session-side. The model staysSend + Syncforever, which cross-thread fanout requires.Sessionstops holding the transport.close()becomes a kio flag the machine observes,closed()a kio consumer,stats()the machine's last sample (the bandwidth sampler already ticks at 100ms while consumed). The publicSessionhandle is then alwaysSend + Syncover any transport, so moq-ffi (UniFFI requires Send + Sync), the relay, and hang never notice the uring path exists. Mirror of the existing invariant: the machine holds no Session clone, and the Session holds no transport.asyncblocks (protocol setup, session maintenance, a few ietf select arms) become named poll state machines;MaybeSendBox/MaybeSendTaskand the lastcfg(wasm)split inutil.rsare deleted.SessionMachine<R>isSendexactly whenR::Transportis.transport::poll::SessiondropsMaybeSend + MaybeSync, keepsClone + 'static. Clone stays because concurrent protocol ops each need a handle, and the quiche adapter is structurallyRc<RefCell<Connection>>plus per-stream ids anyway:web_transport_trait::pollstreams are owned objects while quiche multiplexes every stream op through one&mut Connection, so interior sharing in the adapter is unavoidable.8. moq-uring worker, kernel floor 6.12, no fallback. (Amends the spike's 6.1 floor.) 6.12 buys
IOU_PBUF_RING_INC, the abs-timer park, min-timeout batched waits, and cheap MSG_RING into DEFER_TASKRUN rings, with one code path. Refuses to start on older kernels with a legible error (default Docker/containerd seccomp blocks io_uring entirely; self-hosters keep moq-tokio).SINGLE_ISSUER | DEFER_TASKRUN | COOP_TASKRUN.recvmsg+ provided buffer ring +UDP_GRO, withIOU_PBUF_RING_INCsizing (few huge buffers consumed incrementally, matching GRO's 100B-to-64KB completion variance).sendmsg+UDP_SEGMENTcmsg on every send (never rely on the socket default; cf. tokio-quiche: always set UDP_SEGMENT cmsg when calling sendmsg() cloudflare/quiche#2060). No zerocopy in v1: it only pays above ~3KB per send and had a GSO+ZC EFAULT history around 6.7. The TX pool owns buffers by id with explicit release, so SENDMSG_ZC's deferred-reclaim NOTIF model can be added later without reshaping the path.BufFactorysits above crypto and is not that pool; the adapter usesBufFactory<Buf = Bytes>+stream_send_zcso publish payloads flow app -> packetizer without a stream-buffer copy.futex(2)wake, needing no ring of its own, and the atomic fast path skips the syscall while the worker is awake). MSG_RING is a later prototype.Workerper thread that yields accepted sessions to a caller-owned loop, plus local spawn. The relay writes the thread loop and owns pinning and drain policy; no serve-callback runtime.web-transport-protostate machine over sans-IO quiche, private to moq-uring until the boundary is proven, asRc<RefCell>types implementingweb_transport_trait::poll(already!Send-ready and&mut selfby design).Sequencing (the breaking series targets
dev)dev): kio 0.6 (deletetime/tokio) +moq_net::runtime(trait,Instant,Test) + the moq-tokio Runtime impl.Driverremoved; connect/accept take the runtime.dev): severSession, de-future the protocol machines, relax the transport bounds. Landable step by step; sever first, it is independently valuable.Benchmark obligations are unchanged from the epic: every milestone runs the M0 suite, and the ablation matrix (recv batching / GSO / GRO toggles) rides PR 2's echo harness.
Follow-up ledger (post-v1, rough order of expected value)
io_uring_submit_and_wait_min_timeout, 6.12) under bursty load.poll_read_chunkserved from a per-thread pool viaBytes::from_owner).Mutex(cross-thread by design); session-local state can shedArc<Mutex>forRc<RefCell>once profiling justifies it.Context from the research pass: Cloudflare runs tokio-quiche on plain sendmsg+GSO at millions of rps with no io_uring in production, and the best published io_uring QUIC gain is ~25%. The pinning/steering architecture is expected to carry as much of the win as the ring itself, which the M1-vs-M3 split measures separately.
(Written by Claude Fable 5)
kixelated commentedon Aug 22, 2026
PR 1 of the plan above is up: #3007 (moq_net::runtime, explicit injection, Driver removed, tokio/wasm/Inline/Test runtimes).
Two amendments to the plan, found during implementation and detailed in the PR description:
now(). Relative arming (now + interval) and stamp-derived deadlines must read the virtual clock under a virtual-time test, or an advance leaves every later-armed instant in the past (the bandwidth sampler would fire on every poll). The default is the std clock; only the test runtime overrides it, so real runtimes never think about it.origin::Driver(likely by handing the runtime to the driver at spawn time soorigin::Producer::newkeeps its signature), then deleteskio::time/kio::tokio.Sequencing is otherwise unchanged: model migration + kio 0.6 next, then the moq-uring worker skeleton.
(Written by Claude Fable 5)
kixelated commentedon Aug 24, 2026
Stack update + a scope finding on the de-future step
The runtime series now runs four deep: #3007 (runtime injection) <- #3014 (uring worker) <- #3027 (sever) <- #3028 (kio Task + de-erasure).
Landed by the new PRs:
Sessionholds no transport; abort/last-drop,closed(), andstats()relay through a supervisor leg of the machine, and the handle is unconditionallySend + Sync(const-asserted).kio::TasksaTasktrait so named machines store unboxed, and converts every task set whose occupants were already hand-rolled machines (all of lite, plus ietf'sGroupServe).util::poll_taskis gone.The finding: the plan's "de-future the ~8 remaining async blocks" undercounts. For
Machine<R>to beSendexactly whenR::Transportis, the protocol future must be a nameable type all the way down: a singleasyncblock anywhere in the composition forces dyn boxing at the machine, and a box must pickSendor not at compile time per target, which is exactly what a native!Senduring transport cannot satisfy. I checked the alternatives before accepting this: a GAT (type Boxed<F>) cannot add a per-implF: Sendbound, and a hyper-styleExecutor<F>needs the future type nameable inconnect's signature (TAIT is unstable). Conditional-Send erasure is inexpressible in stable Rust, so full de-future is the only sound route, as the plan says, just bigger than it says.Measured scope: the ietf tree has ~40 non-test
async fns acrossadapter.rs(1.4k lines),publisher.rs(3.0k),subscriber.rs(2.8k), andsession.rs(1.2k), and the driver arms await subscriber/publisher/adapter futures, so conversion is forced bottom-up through the deepest code: adapter run loops -> subscriber/publisher bodies (run_broadcast,run_subscribe,handle_stream, namespace machinery) -> session driver arms ->Machineconcretization + deletingMaybeSendBox/MaybeSendTask. The lite path is already fully machine-based after #3028.Options for sequencing, in my preference order:
connect's signature, which is its own design question.Nothing below #3028 depends on this choice; the stack is green (
just check/just test, 3171 tests) through all four PRs.(Written by Claude Fable 5)
feat(uring): run moq-lite sessions over quiche on the worker
feat(uring): add the thread-per-core worker skeleton
feat(net)!: sever Session from the transport
feat(kio): let Tasks hold named machines, unboxing the protocol children
feat(net)!: run moq-lite machines unboxed over any transport
feat(uring): run moq-lite sessions over quiche on the worker
feat(net)!: run moq-lite machines unboxed over any transport
7 remaining items
feat(net)!: sever Session from the transport
feat(kio): let Tasks hold named machines, unboxing the protocol children
feat(net)!: run moq-lite machines unboxed over any transport
feat(uring): run moq-lite sessions over quiche on the worker
feat(net)!: sever Session from the transport
feat(uring): run moq-lite sessions over quiche on the worker
feat(kio): let Tasks hold named machines, unboxing the protocol children
feat(uring): run moq-lite sessions over quiche on the worker
feat(uring): run moq-lite sessions over quiche on the worker
feat(uring): run moq-lite sessions over quiche on the worker
kixelated commentedon Aug 26, 2026
M4 has started: the multi-connection endpoint is up
M3 closed out with the #3007 <- #3014 <- #3027 <- #3028 <- #3031 <- #3033 stack all merged: runtime injection, the worker, the session sever, unboxed lite machines, and real moq-lite sessions over quiche on the worker (lite-only per option 2 of the previous comment; the ietf de-future remains open and non-blocking).
#3073 is the first M4 PR:
quic::Endpoint, the multi-connection demux #3033 deferred. One worker socket now serves many connections (dials included), routed by fixed 8-byte connection ids, with id rotation for migration and version negotiation for unknown versions. Ingress stays zero-staging-copy: the demux task feedsquiche::Connection::recvstraight from the provided-buffer ring.client::connect/server::acceptbecame single-connection shorthands over it. One deliberate behavior change:accept()only yields successful handshakes (a listener must not die because one client failed), so the mTLS refusal test now asserts the client-side death instead.Remaining M4 sequence as I see it:
Workerscounterpart (thread spawn/pin, theSO_REUSEPORTgroup, thecid_prefixsteering byte feeding the endpoint's id generation). This is where the runtime-neutral bind/lock/steer pieces extract out of moq-tokio per the M4 plan, so both stacks share one group-formation implementation.origin::DrivererasesSendtimers, so origins stay on the main runtime initially.web-transport-protoover the same adapter (HTTP/3 CONNECT), or an explicit decision that uring mode serves raw-QUIC ALPNs only at first while WebTransport stays on the tokio listeners. Needs a call before the relay flag can default anywhere.Still on the ledger from #3073 itself: stateless retry / handshake rate limits (DoS posture), stateless resets,
SENDMSG_ZC, pacing hints.(Written by Claude Fable 5)
kixelated commentedon Aug 26, 2026
Stack update: #3073 (endpoint) <- #3078 (moq-sock extraction + steering).
#3073 picked up two fixes from review: a pending-handshake backlog (default 1024) bounding what an unauthenticated Initial can allocate, and a driver bug the backlog test exposed, where the connection driver armed its deadline after polling it, so a dial nobody answers hung forever instead of timing out (the endpoint made that path reachable for the first time).
#3078 is the shared-crate extraction the epic committed to:
moq-sockownsbind(moved wholesale, re-exported asmoq_tokio::bind),shard(the reuseport group bind, port lock, cBPF steering filter,Shard,cid_prefix), andcpu(pinning). moq-tokio's workers form their group from it unchanged; moq-uring is the second consumer viaendpoint::Config::shard, which puts the steering byte on every id the endpoint issues, dials included (the cluster shape). A steered two-worker group test proves it end to end on real threads.Next per the plan: the relay integration behind the worker-mode flag, which brings the uring group owner (thread lifecycle, the structural bind-in-order guarantee) and decides where origins run.
(Written by Claude Fable 5)
kixelated commentedon Aug 26, 2026
Stack update: #3073 (endpoint) <- #3078 (moq-sock + steering) <- #3081 (WebTransport).
#3081 closes the browser gap that stood between the uring runtime and the relay integration:
quic::web::Requestruns the HTTP/3 CONNECT handshake over the same quiche adapter (web-transport-proto state machines, subprotocol selection as the moq version carrier, capsule close so codes reachWebTransport.closed), andquic::web::Sessionbecomes the runtime's one transport type, wrapping raw QUIC and WebTransport in the same machinery per web-transport-quinn'sSession::rawpattern. Validated by interop tests against web-transport-quinn itself, including a full moq-lite session over WebTransport, which is the browser-to-relay path end to end.With that, every transport-side prerequisite for the relay integration is in place: multi-connection endpoints, handshake backlog, steered reuseport groups, and both peer flavors on one worker socket. The next PR is the relay itself: uring workers behind the existing worker-mode config flag (h3 + raw ALPN dispatch mirroring moq-tokio's), origins on the main runtime, cluster dials from the workers' endpoints. The remaining protocol scope note stands: uring sessions are lite-only until the ietf de-future lands, so ietf ALPNs stay off the uring listener's offer.
(Written by Claude Fable 5)
kixelated commentedon Aug 26, 2026
Stack update: #3073 <- #3078 <- #3081 <- #3082, the M4 relay integration.
--runtime-workers N --runtime-io-uringnow serves the relay's QUIC from io_uring workers: browsers (WebTransport) and native peers (raw QUIC) on one steered socket per pinned worker, moq-lite only, with authentication and session supervision on the shared runtime (the #3027 sever is what makes that handoff free). The end-to-end test runs the stock relay loop with a WebTransport publisher and mixed WT/raw subscribers across a two-worker group.Two things of note beyond the integration itself:
Requestlost its thread-affinity bounds. The relay needed the path-gating pause on a!Sendtransport, so the ietf handshake continuations are now boxed at construction (where the bounds hold) andaccept_request_litejoins the surface. EvenRequest::path()used to demand theBoxablebundle, which was an artifact.Relay::runbug fixed: with QUIC on a worker group (the tokio mode included) and no tcp/unix listeners, the stream-only server's accept loop reported "stopped accepting" instantly and took the relay down. The existing worker test dodged it by never callingrun().That completes the M4 exit criteria's integration half: the relay serves QUIC in io_uring mode behind the config flag, workers own the handshakes, the tokio side keeps HTTP/WebSocket/streams/auth/supervision. Still open before closing M4: cluster dials from the worker endpoints (they stay on tokio for now; the dial side of the endpoint already works), the full M0 benchmark deltas against M1 on the same rig, and running the integration/smoke suites in uring mode by default in CI. Ledger unchanged otherwise (mTLS on the uring listener, cert hot reload, stateless retry, SENDMSG_ZC).
(Written by Claude Fable 5)