Repository navigation
perf(preprocessing): parallelize CPU setup passes - #1737
Merged
Merged
Conversation
Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
Benchmark comparison (crates) |
Conflict resolutions: - Cargo.toml: keep the a16z/dory#26 git pin (parallel prepared-point cache init); dory-pcs 0.4.1 on main does not contain that change (dory#26 is closed, 0.4.1 ships dory#27's routine ops only). - Cargo.lock: main's lock + minimal re-resolve for the dory git source. - tracer/src/execution_backend.rs: both sides are additive — keep the parallel Cycle->TraceRow conversion (this PR) alongside the chunked execution seam and its tests (#1728). mod tests keeps the unconditional expect(clippy::unwrap_used) for parallel_cycle_conversion_matches_serial.
Contributor
@@ added removed @@
+ actual code +77
- actual code -24
tests 0 0
docs 0 0
+ fixtures +2
helper scripts 0 0
----------------------------
+ total +79
- total -24 |
0xAndoroid
marked this pull request as ready for review
August 19, 2026 21:17
0xAndoroid
marked this pull request as draft
August 19, 2026 21:19
0xAndoroid
marked this pull request as ready for review
August 20, 2026 17:12
markosg04
approved these changes
Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Cycle→TraceRowwith Rayon above a 2^14-row threshold; small traces stay serial. Error capture stays out of band so indexed collection writes directly into one output allocation while preserving ordering, error propagation, and panic behavior.ProverConfig::derive's touched-address reduction through Rayon under the existing defaultparallelfeature; small/no-default builds stay serial.dory-pcs 0.4.1. Companion perf(cache): prepare pairing points in parallel dory#30 carries the prepared-point cache portion of this optimization; Jolt no longer pins an unreleased repository commit.CPU measurements
Originating locked 2^26 campaign pair, same proof bytes:
Cycle→TraceRowProverConfig::derivePrepared-point initialization (companion a16z/dory#30):
log_n=34, 131,072 points/groupThe campaign's 2^28 traces put serial conversion at ~30–43 s and serial prepared-point initialization above 30 s.
Scope
Excluded W7
PAD_INTERLEAVE: it is implemented inside the CUDA trace-store/store-early pipeline and has no standalone plain-CPU path to upstream here.Verification
-D warnings: host and host+zkAuthored with Claude Code.