Repository navigation
perf(kio): index task wakes so a poll skips capacity a burst left behind - #7513
Conversation
Quest-Claim: 18add28434d52af3
Tasks::poll swapped every wake-bit word the set ever grew, so a long-lived set paid for its historical peak on every owner poll. A three-level index (word bits, a per-chunk summary, one top word in the shared state) lets a poll swap only woken words: an idle poll is one swap at any peak. The snapshot is still taken in full before any task runs, in ascending slot order from the rotation point. The per-slot waker keeps only its slot index (8 B smaller). New tasks_capacity_* bench groups sweep historical peak against live and woken tasks, and the task_capacity bench reports retained bytes. The quest is rewritten to its remaining memory half. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The maintainer picked (a) on 2026-11-01: land the CPU half as is and keep the quest for the memory half. The merged toolchain flags the after_burst return type, which cannot be aliased on stable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 52 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 36e475bf81306e978a27c631a699fb9d20c4d549.
No actionable bugs found. Checked the hierarchical wake marking/clearing order, folded top-word collisions beyond 64 chunks, sparse snapshot ordering and rotation, stale-slot reuse, and the capacity benchmarks/tests. No prior published review findings to carry forward.
Direction: the small index addresses the historical-capacity scan at its source without changing public API, wire format, or one-pass scheduling. Keeping the fixed top word rather than adding a growable hierarchy is a reasonable complexity tradeoff; above 65,536 slots, active polls still visit the chunks sharing a top bit. Separating memory reclamation preserves the existing cheap reuse and stale-waker ownership model.
Verification: GitHub source inspection only; I did not run tests, concurrency tooling, or reproduce the performance measurements.
(Written by OpenAI)
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed the full diff at 36e475bf8, with a focus on the new wake-index algorithm and its concurrency guarantees.
rs/kio/src/task.rs— three-level wake index:Chunk.summarymarks nonzero words andShared.readymarks chunks with a marked summary; a wake marks a level above only when it makes that level nonzero, and a poll clears top → summary → word before reading down. Verified the lost-wake invariant by hand: a nonzero word implies its summary was set or a poll already owns the word (which will swap it), and a set summary implies the top bit is set or a poll owns the chunk. The> WORD_BITSsort threshold is exactly where a top bit aliases two chunks, sopartition_pointplus the wrap masks always see ascending word indices. Rotation and mid-pass snapshot semantics are preserved.SlotWakershrink: keeps only the absolute slot index and derives word/bit/chunk; the 8 B/slot saving and the extrafetch_ors are bounded and as described.- Tests:
wakes_past_64_chunks_run_in_slot_ordercovers aliasing past 64 chunks and rotated order;concurrent_wakes_are_never_loststresses cross-thread wakes against owner polls. I rancargo test -p kio --lib task::on the head — 17 passed, including both new tests. - Benches:
tasks_capacity_*groups and thetask_capacityretained-bytes table; the LIFO/reuse shapes match the quest's deferral of memory reclaim. - Quest docs:
task-capacity.mdis correctly rewritten to the memory half, and theREADME.md/uring-quiescence.mdcross-references are consistent.
No public API or wire impact. The deferred memory half and the optional worker-bench follow-up are recorded as a maintainer decision in the quest, not blockers.
deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Problem
Advances
quest/m2/performance-survey/task-capacity.md.kio::Tasks::pollswapped every wake-bit word the set had ever grown, then scanned them again to dispatch. A long-lived set paid for its historical peak on every owner poll, even with one task left. The io_uring worker polls its set every turn, and the session serving loops poll theirs on every wake. The 2026-10-22 CDN profile putTasks<GroupServe>::pollat 1.9-2.4% self-time, the largest moq-net site.Approach
rs/kio/src/task.rsnow indexes the wake bitset in three levels:AtomicU64per 1,024-slot chunk. Bitwmarks wordwas nonzero.Shared::ready, where bitc % 64marks chunkcas having a marked summary.A wake marks the level above only when it makes a level nonzero. A poll clears each level before it reads the level below. So a wake into a level that is still nonzero is either taken by that poll, or covered by the mark that level's first wake left for the next poll. An owner poll with nothing woken is now one
swap, whatever the peak.What stays the same:
(word, bits)pairs, so dispatch keeps the same rotated slot order fromnext.swapeach), and the pairs are sorted.SlotWakernow keeps only its slot index and derives its bits from it, which saves 8 B per slot.Benches, both picked up by the nightly kio leg (
criterion kio --no-fail-fast -p kio --bench '*'insh/gh/nightly.sh):tasks_capacity_{idle,survivor,sparse,full,churn}groups inrs/kio/benches/tasks.rs. They sweep historical peak (1, 1,024, 16,384, 65,536) against the tasks that are live and woken now.rs/kio/benches/task_capacity.rstable: retained bytes after a burst, allocations per owner poll, and bytes left after drop.Quests:
uring-quiescence.mdpreviously said whichever of the two quests landed second rebases on the other. It now says to baseline its pass budget on the indexed snapshot.Measurements
User-space instructions (
perf stat --no-scale -e cpu_core/instructions/u), pinned to P-core 2 of a Core Ultra 7 270K Plus. The host was shared, with a load average of about 30 on 24 threads.git archiveexport, outside the checkout, with theprofilingprofile (fat LTO) and Rust 1.98.1.9b823e2e16. Candidate: this branch'stask.rs. Later edits were formatting, docs and test attributes only.Owner poll after a burst, one task left (
kio::Tasksalone; instructions per poll):Putting the survivor in the highest slot instead of slot 0 changes nothing material (within about 60 instructions in either revision).
16 woken per poll (instructions per poll):
Reuse paths (instructions per poll):
peaktasks, per poll (two per burst)Steady polls allocate nothing in either revision.
Consumer: io_uring worker after a spawn burst. A
moq_uring::Workerspawnedpeaktasks that all finished, then ran one-datagram round trips (3 turns each). User instructions per round trip:Memory (unchanged in kind; see Decisions). Retained heap after a drained burst, from the
task_capacitybench with an 8-byte task:SlotWaker.Wall clock, from a Criterion smoke run on the loaded host (indicative only):
tasks_capacity_idletakes 15-20 ns at every peak. The quest's base measurement was 4,831 ns at 65,536.Validation
wakes_past_64_chunks_run_in_slot_order: 66 chunks of tasks. Woken slots that share top-word bits each run exactly once, in ascending rotated slot order. No other task runs, and an idle poll runs nothing.concurrent_wakes_are_never_lost: 4 threads wake 3 chunks of tasks while the owner polls. Every task's last poll must see its last wake.cfg(miri)it shrinks to 16 rounds, ahead of the Miri nightly in moq-dev/moq#7315. The 66-chunk test is ignored under Miri.nix develop -c quest gate just checkpasses on141960d652: 10,543 tests, lint and quest. Only the flake check was skipped: this host's Nix builder cannot create user namespaces.origin/main(bf4e726f66):cargo test -p kio --all-targets(182 tests) andjust rs check -p kiopass. The merged toolchain flaggedclippy::type_complexityon theafter_burstbench helper's opaque return type, which cannot be aliased on stable; it now carries an#[expect]with that reason.Impact
fetch_or(the chunk summary), and a chunk's first such wake does another (the top word). That is what the +6% on the sparse 1,024-slot shape pays for.Choices made without asking
chunk % 64past 64 chunks, rather than a growable top level. Wakers hold no pointer that would have to move, and sets up to 65,536 slots are exact.SlotWakerto its slot index, since it is the per-slot allocation this quest is about.Decisions
Settled 2026-11-01 by the maintainer.
What to do with the memory half of the quest. This PR fixes poll CPU. The set still keeps about 184 B per peak slot, plus
size_of::<T>(), after a burst drains.The quest's Plan records the same answer.
Review
Codex
gpt-6-astra, medium effort (quest review --deep --base origin/main), session01a1962d-68ba-7ff2-8bba-f1d1147046ff.0f9f840d7a: 0 findings.141960d652(quest text correction only): 0 findings.36e475bf81: local review skipped under the simple-review rule (moq-dev/quest#364). Since round 2 the branch has only a conflict-free merge oforigin/main, the recorded decision in the quest Plan, and one#[expect(clippy::type_complexity)]attribute on a bench helper. None touches behavior, public API, wire format, security code, orunsafe. No online-reviewer threads were open.Follow-ups
worker_turnsvariant after a spawn burst would give the consumer path its own nightly series. That is optional, since the kio groups already cover the mechanism.Harness:
rs/kio/examples/capacity.rs(uncommitted)Harness:
rs/moq-uring/examples/worker_burst.rs(uncommitted)Driver scripts
(Written by Claude Opus 5.5)
🤖 Generated with Claude Code