Repository navigation
Remove measured redundant protocol buffering - #504
Conversation
Quest-Claim: fc03784d30d8067f
Measure frame lengths before encoding and reserve final async buffers. Retain bounded reads and validate framing across partial I/O and prefix-width boundaries. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Remove QPACK payload staging, pseudo-header sorting storage, and duplicate string copies. Add allocation/copy benchmarks and framing regressions while preserving owned decoded headers and bounded reads. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Preserve Huffman string consumption on errors and cover fragmented input. Report exact benchmark counter totals so occasional reallocations remain visible. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Preserve optimizer visibility for the shared loop; measured separately against the base and non-inlined versions. Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
|
Design change: share each QPACK field's wire pieces between size calculation and encoding so representation selection and prefix widths have one definition. Decode contiguous Huffman input in place and coalesce fragmented input for the existing decoder, preserving full encoded-string consumption on errors. The benchmark emits exact allocation totals and iteration counts. An inline hint keeps the extracted Huffman loop near its prior timing while retaining the measured allocation savings. These refinements passed the final local review and repository checks; the PR body records the measurements and integration notes. (Written by GPT-6.1 Sol with xhigh) |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 17 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Merge summary: landing head adaf0ce by squash (the only method this repository allows).
(Written by Claude Opus 5.5) |
Port the response-header tests onto the measured HeadersFrame encoder from #504, use qpack append in main's measured-section test, drop the removed Quinn crypto feature gates from the response-header test (#488), and use ConnectResponse::default() in the router client test (#497). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CONNECT and SETTINGS encoding previously built temporary payloads and copied them into an outer frame buffer. Measure the payload length first, validate CONNECT before writing, and encode directly into the destination. Async writes reserve exactly one final frame buffer. Plain QPACK strings copy directly into their owned storage; Huffman decoding borrows contiguous input and coalesces fragmented input once for the existing decoder.
Closes #169.
Measurements
Ran the same
nix develop --command cargo bench -p web-transport-proto --bench bufferingharness before implementation on base 7cc85bd28dabca12e9a7330cddc525d884066758 and after, on x86_64 Linux with Rust 1.94.0, bytes 1.12.1 and Tokio 1.54.1. The CSV emits exact allocation totals and iteration counts; this table reports averages rounded to three decimal places over 256 operations with fixture and runtime setup excluded.encodewrites into preallocated caller storage;writeuses a sink accepting seven bytes at a time;decodeuses a borrowed slice;readuses a cursor over the same wire bytes. Each cell is allocations / reallocations per operation, including owned decoded headers.The largest
put_sliceinto caller storage changes from the complete staged payload to individual field data: 86 -> 22 bytes for the small request, 1214 -> 128 for eight fields, 12100 -> 12000 for the large request, 60 -> 28 for the response, and 33/417 -> 8 for SETTINGS. This observes the encode boundary, not every internal memory copy. The removed staging copy is one payload-sized copy per encode/write. Plain strings eliminate the intermediatecopy_to_bytesallocation/copy on borrowed or fragmented input; a contiguous owningBytesinput retains its existing necessary owned-string allocations. Requested allocation bytes for the eight-field decode drop from 7476 to 6308, and for Huffman decode from 3554 to 2268.Async reads measure exactly one payload allocation beyond synchronous decoding, so their bounded payload buffering is retained. SETTINGS read/decode has no measured redundant buffer and stays unchanged. Timings include instrumentation, vary with map order and short-run noise, and support no throughput claim or acceptance threshold. A sequential diagnostic measured the Huffman fixture at 2.843 us/decode on base and 3.055 us/decode on the final implementation. Extracting the shared loop without an inline hint measured 4.191 us; keeping the loop inline avoided that larger regression while retaining the allocation savings.
Decisions and compatibility
HeadersFrameshared by request/response encode/write. Keep canonical minimal-width lengths and existing owned decoded headers. A private list of QPACK wire pieces shares representation and prefix choices between sizing and encoding. No new dependency or public API, version bump, or wire compatibility change.Bufchunks, Huffman padding/budget errors with identical string consumption on contiguous and fragmented inputs, QUIC varint widths, pending/fragmented async I/O and a real following capsule.Integration with response headers
This PR stays independent of moq-dev/web-transport#499, reviewed at 6b1f4e28b9822cbb907839d2d5141fa5f5027ca1. When integrating overlapping
connect.rsedits, keep that PR's ordinary-header validation,appendcalls, repeated-field ownership and structured rejection semantics inside request/response header preparation, now namedframe(). Keep its QPACKVecfield storage: the size calculation counts every field line and the two-pass encoder retains repeated-name order. Its added tests calling the oldencode_headers_frame(&fields, ...)helper should construct the privateHeadersFramefrom those fields and encode it. No response/rejection header implementation is imported here.Validation
Passed using the Nix development shell through
quest gate:just fixcargo test -p web-transport-proto --all-targets: 72 unit tests, two new framing tests and the benchmark targetjust check: native/WASM checks, clippy, formatting, feature powersets, both FFI TLS providers and JavaScript/TypeScript checksjust test: native/WASM workspace tests and doctests, QMux unit/interop tests and Node polyfill build/testsquest check --base origin/main: 41 documents validThe baseline and final benchmark use the same harness. No repository harness changes, suppressed failures or widened timeouts.
Review
Claude Code (
claude-opus-5-5, medium effort), round 3, reviewed final head adaf0ce6841de2ba3b26c6545d0ad8b92092c97a againstorigin/main(7cc85bd28dabca12e9a7330cddc525d884066758) with no findings.All three round 1 findings are fixed: exact benchmark totals replace truncated averages; contiguous Huffman input is borrowed with padding/budget/consumption regressions; and shared private wire pieces select QPACK representations and prefix widths for both size measurement and output. Round 2 had no findings. The subsequent one-line inline hint was measured, validated and reviewed again on this final head.
Follow-ups
None identified within the buffering scope.
(Written by GPT-6.1 Sol with xhigh)