Skip to content

perf(qmux): batch writer frames, flushing the transport once - #397

Merged
kixelated merged 1 commit into
moq-dev:mainfrom
pouriya:qmux-batch-writer-frames
Sep 27, 2026
Merged

kixelated merged 1 commit into
moq-dev:mainfrom
pouriya:qmux-batch-writer-frames

Conversation

@pouriya

@pouriya pouriya commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

The problem

Fanning out to ~2400 WebSocket viewers, 66% of the relay's sendto calls
carried 8–15 bytes, and 51% of outbound packets were under 64 bytes. Every
frame is flushed as it is written, so small writes never coalesce — each one
costs a syscall, a segment and an ACK.

The change

Writer gains feed and flush. The writer feeds the frames already queued —
up to 64 KiB or 256 frames, in the existing priority order — then flushes once.
Each qmux frame is still its own WebSocket message; only the flush is shared.

Both methods have defaults that keep today's behaviour, so an external
impl Writer compiles and behaves unchanged. No public signature changed.

Measured

FreeBSD 15.1, 8 vCPU. Same host, same publisher, same load, before and after:

Before After
sendto/s 150k 26k
Outbound packets/s 91k 28k
Packets carrying 1–63 bytes 51% 7%
Relay cores per Gbps ~5.1 ~2.5

Tests

New: queued_frames_are_fed_then_flushed_once queues a three-write media frame
and asserts it produces one flush instead of three, with payloads and stream
offsets unchanged. It fails against main.

Rebased onto #399 / #402

Since #402 signals SendStream::closed() success only once the FIN is written, a fed FIN is now signalled Finished after the batch's flush succeeds; a failed flush drops the signal. New test: failed_fin_flush_does_not_signal_finished. StreamWriter::send now delegates to feed + flush.

@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 8fadfd26-ac46-407f-9d8c-847da4f1126d

📥 Commits

Reviewing files that changed from the base of the PR and between f0c9607 and 3f3bc73.

📒 Files selected for processing (2)
  • rs/qmux/src/session.rs
  • rs/qmux/src/transport.rs

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


Walkthrough

PriorityQueue now supports synchronous removal of the next queued frame. The Writer trait now separates queuing data from flushing. StreamWriter buffers framed payloads and flushes them explicitly. WsWriter queues binary messages and flushes the sink explicitly. WriterState feeds bounded batches in priority order and flushes each batch. It signals pending FIN completion only after a successful flush.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 3f3bc

No identified batching issue remains to resolve before merging. Normal checks can proceed.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 3f3bc

Outbound writes now share a flush, but batching is bounded and a failed flush closes the affected session rather than reporting its stream as finished. No new security boundary or privilege was identified. Teardown behavior and custom writer implementations remain worth checking.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The identified failure and buffering effects are confined to the affected session writer and its transport; the reviewed path provides no evidence of a new privilege, datastore, or cross-service authority.

Trust Boundaries and Controls

  • observed — The session checks each encoded frame against the peer record limit before feeding it, and marks both feed and flush as backpressured while awaiting the transport.

Resilience and Maintainability Implications

  • observed — Teardown makes a best-effort attempt to feed queued control frames and flush them, but skips the explicit flush if a control-frame feed fails. Interruption of an in-flight write also existed before batching, so loss of a close frame is not established as a new security condition.

Hardening Proposals

  • proposed — Specify whether a custom Writer must flush pending fed control frames during close, and exercise teardown after a partial batch. This would clarify the close-delivery guarantee without treating the current uncertainty as a verified vulnerability.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely identifies the main change: batching writer frames and flushing the transport once.
Description check ✅ Passed The description directly explains the batching problem, the feed and flush changes, compatibility behavior, measurements, and tests.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The session writer flushed after every frame, so each small frame cost
one syscall and one TCP segment. It now feeds the frames already queued
and flushes once.

A fed FIN's SendStream is signalled Finished only after the flush
succeeds, so closed() still never reports success for a FIN the transport
failed to write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the qmux-batch-writer-frames branch from f0c9607 to 3f3bc73 Compare September 27, 2026 02:33
@kixelated

Copy link
Copy Markdown
Collaborator

Thanks @pouriya, nice numbers. Reviewed and rebased onto main (after #399 and #402):

  • The batch keeps strict priority order (try_next_outbound checks control, then datagrams, then the stream queue, the same as next_outbound). It flushes as soon as the queues drain, after at most one yield_now for a tiny stream batch. The default feed/flush keep existing Writer impls unchanged.
  • fix(qmux): report a finished send stream as closed cleanly #402 made SendStream::closed() succeed only once the FIN is written. With batching, transmit only feeds, so FIN signals are now held in WriterState::finished and sent Finished after the flush succeeds. A failed or abandoned flush drops them. New test: failed_fin_flush_does_not_signal_finished, which fails if the signal is sent at feed time.
  • StreamWriter::send now delegates to feed + flush instead of duplicating the record framing.

cargo test -p qmux --all-features and just check pass locally. I'm enabling auto-merge.

(Written by Opus 5.5)

@kixelated
kixelated enabled auto-merge (squash) September 27, 2026 03:03
@kixelated
kixelated merged commit d69d746 into moq-dev:main Sep 27, 2026
1 check passed
This was referenced Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants