Repository navigation
feat(memory)!: misaligned loads and stores behind a zicclsm feature (byte-addressable memory, Tier 2) - #2137
Open
moodlezoup wants to merge 2 commits into
Open
moodlezoup wants to merge 2 commits into
moodlezoup wants to merge 2 commits into
Conversation
…te-addressable memory, Tier 2) Tier 2 of #2013 (full Zicclsm). A top-level LH/LHU/LW/LWU/LD/SH/SW/SD expands twice, and the bytecode holds both sequences at the instruction's address: - The aligned sequence is Tier 1's (the native LD/SD row, 3-row loads, 6-row stores). Constraint 0 and RAM RAF evaluation still pin it to naturally aligned addresses. - The misaligned sequence reads the doublewords holding the first and last accessed bytes with byte-class floor rows and is correct at every offset, so no table pins it. The prover picks the entry PC. The honest tracer runs the misaligned sequence exactly when ea mod width != 0 (DualAccess::is_misaligned), and the x86 dispatch stub mirrors that rule. The protocol changes only by two lookup tables (57 tables, 464 bytecode lanes): - WindowMaskD(ea) = 2^64 - 2^(8*(ea mod 8)). - FunnelHigh(x, M), the left-shift half of ROTR. Loads add the Pext and FunnelHigh halves. Stores scale by 2^(8*(ea mod 8)) with MUL/MULHU and write the second doubleword first. The second doubleword is addressed at ea + width - 1, so the sequence touches only doublewords holding accessed bytes and needs no guard doubleword. Rows (aligned -> misaligned): | Ops | Aligned | Misaligned | |---|---|---| | LH/LHU | 3 | 8 | | LW/LWU | 3 | 7 | | LD | 1 | 6 | | SH/SW | 6 | 16 | | SD | 1 | 12 | Atomics keep aligned-only rows, so a misaligned atomic stays unprovable and still aborts the trace. - JoltInstructionRow::misaligned_variant and a two-run PcSlot keep the sequences' PCs apart. stamp_sequence_metadata writes the flag, and JoltTraceRow keeps it in control bit 7. - expand_instruction returns SourceExpansion::{Single, Dual}. - Tracer: - per-PC InlineExpansion with runtime selection; - native fast path for aligned LD/SD; - a non-dispatching row executor for sequence rows; - force_aligned_sequences replaces allow_misaligned_floor_accesses. - x86: a dispatch stub at dual group heads, with the misaligned rows out of line. - z3: each sequence is proved at all eight doubleword offsets, and the aligned sequence's address assumptions admit only aligned offsets. BREAKING CHANGE: the program-preprocessing digest domain moves to dory-whole-bytecode/v3 and akita-whole-bytecode/v2 (row and PcSlot layout), verifier fixtures use JVCF0009, and every program's bytecode gains the misaligned sequences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
📏 PR diffNote Actual code changed: 1,571 lines🟢 +1,386 added 🔴 −185 removed Tests, docs, fixtures, and helper scripts excluded.
Every changed line is classified once. |
Tier 2 costs every program bytecode (1.7-2.9x the rows, one more padded power of two) while only guests that make misaligned accesses gain from it. Put it behind a `zicclsm` Cargo feature: - Without it a build proves what Tier 1 does: loads and stores expand once, a misaligned access aborts the trace, and WindowMaskD/FunnelHigh and their virtual instructions do not exist (55 tables, 462 lanes). - With it, Tier 2 is unchanged (57 tables, 464 lanes). As with field-inline, every crate that expands the instruction-kind lists or matches on the gated kinds declares the feature (jolt-riscv, jolt-lookup-tables, jolt-program, tracer, jolt-tracer-x86, jolt-witness, z3-verifier), and jolt-claims, jolt-host, jolt-verifier, jolt-prover and jolt-sdk forward it. Only the semantic pieces are gated: the dual-expansion plumbing (row flag, PcSlot run, SourceExpansion, tracer selection, x86 dispatch) is shared and never sees a misaligned sequence without the feature. - zicclsm selects its own program-preprocessing digest domains (.../zicclsm/v3, .../zicclsm/v2), pinned by golden digests. - CommittedProgramPreprocessing records bytecode_lanes, and the verifier rejects a committed artifact whose lane count differs from its own. Both layouts pad to 512 lanes, so an artifact from the other build would otherwise be accepted and misread. - jolt-program's expansion fixture pins the single or aligned sequence in output_sha256 and a dual expansion's misaligned sequence in misaligned_sha256, checked under zicclsm; jolt-eval's parity invariant follows the same contract. - memory-ops: memory_ops is aligned-only again; misaligned_memory_ops runs every offset and joins the e2e matrix under zicclsm. - Verifier fixtures use JVCF0010 and cache zicclsm builds apart; the fuzz bundles are regenerated from the default build. - z3: without zicclsm, loads and stores are checked against the contained-lane semantics; the nightly builds with zicclsm. - CI: zicclsm clippy lanes, crate, tracer and x86 test steps, a zicclsm prover job (the full package in clear, ZK and Akita, plus the field-inline matrix) and a zicclsm verifier-fixture job. BREAKING CHANGE: default builds drop Tier 2's two tables and misaligned sequences; zicclsm builds use their own digest domains; committed program preprocessing gains bytecode_lanes, which changes committed program digests; verifier fixtures use JVCF0010. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2132 (Tier 1). Implements Tier 2 of #2013, full Zicclsm, behind a
zicclsmCargo feature: with it, misaligned loads and stores prove at every offset, including those that cross a doubleword boundary. Without it, a build proves exactly what Tier 1 does.The
zicclsmfeatureTier 2 costs every program bytecode (see Measurements), while only guests that make misaligned accesses gain from it, so they opt in.
zicclsm(default).WindowMaskD/FunnelHighand their virtual instructions do not exist: 55 tables and 462 bytecode lanes (472 with field-inline).zicclsm. Tier 2 as described below: 57 tables, 464 lanes (474 with field-inline).Plumbing follows
field-inline:jolt-sdk/zicclsm; guests need no change.PcSlotrun,SourceExpansion, tracer selection, x86 dispatch) is shared, and without the feature it never sees a misaligned sequence.It is a separate protocol profile:
zicclsmselects its own program-preprocessing digest domains (…/zicclsm/v3,…/zicclsm/v2).CommittedProgramPreprocessingrecordsbytecode_lanes, and the verifier rejects a committed artifact whose lane count differs from its own build's. Both layouts pad to 512 lanes, so without the check an artifact committed by the other build would be accepted and its lanes misread.Design (with
zicclsm)A top-level
LH/LHU/LW/LWU/LD/SH/SW/SDexpands twice, and the committed bytecode holds both sequences at the instruction's address:LD/SDrow, 3-row loads and 6-row stores. Constraint 0 and RAM RAF evaluation still pin it to naturally aligned addresses. An aligned program runs exactly the rows it ran under Tier 1.The prover picks the entry PC. The honest tracer runs the misaligned sequence exactly when
ea mod width ≠ 0, the ruleDualAccess::is_misalignedowns; the x86 dispatch stub mirrors it. R1CS is unchanged. The only protocol change is two lookup tables, both built from existing prefix and suffix families:WindowMaskD2^64 − 2^(8·(ea mod 8))FunnelHigh(x, M)VirtualROTR:(x mod 2^s) << (64 − s)onM = !0 << s, 0 ats = 0Rows per sequence:
LH/LHULW/LWULDSH/SWSDA misaligned load adds
Pext(lo, m)andFunnelHigh(hi, m). The halves are disjoint, so theADDis carry-free. A misaligned store scales by2^(8·(ea mod 8)) = −mwithMUL/MULHUand writes the second doubleword first.Where this departs from the #2013 sketch
Each departure removes a table family or a memory-layout change. The spec records all four under "Tier 2 as implemented".
ea + width − 1, so the sequence only touches doublewords that hold accessed bytes. That is a single doubleword when the access does not cross.Plumbing
JoltInstructionRow::misaligned_variantis prover metadata, not a bytecode lane. It is stamped bystamp_sequence_metadataand stored inJoltTraceRowcontrol bit 7.PcSlotgainsmisaligned_sequence_lengthand is still 8 bytes. PCs resolve by(address, vsr, misaligned_variant).expand_instructionreturnsSourceExpansion::{Single, Dual(DualExpansion)}. Only top-level instructions are dual. TheLW/SWinsideLR.W/SC.Wand theLD/SDrows of the.Datomics keep their aligned rows, so a misaligned atomic stays unprovable and still aborts the trace.InlineExpansion, and the sequence is chosen before any row runs.LD/SDkeep the native-row fast path.trace_row, so anLD/SDrow of an atomic never takes a misaligned sequence.force_aligned_sequencesreplacesallow_misaligned_floor_accesses.ea & (width − 1). The misaligned rows are emitted out of line and jump back to the next instruction. The jump table, pause checks and observations are unchanged.Breaking changes
dory-whole-bytecode/v3andakita-whole-bytecode/v2, because the row andPcSlotlayouts changed, andzicclsmbuilds use…/zicclsm/v3and…/zicclsm/v2.CommittedProgramPreprocessinggainsbytecode_lanes. All golden digests are updated.FIXTURE_MAGICis nowJVCF0010.zicclsmbuilds cache their verifier fixtures separately.zicclsm, every program's bytecode gains the misaligned sequences (see Measurements).Testing
All run locally on this branch, with and without
zicclsmunless noted.z3. With
zicclsm, each sequence of every dual load and store is proved at all eight doubleword offsets against a byte-level RISC-V model, and the aligned sequence's own address assumptions are shown to admit only the naturally aligned offsets. Planted recipe bugs are all caught. Without the feature, loads and stores are checked against the contained-lane semantics, as in Tier 1. The nightly builds withzicclsm, whose expansions contain every row a default build runs.Expansion. Without
zicclsm, every load and store expands once. The golden fixture pins each case's single or aligned sequence inoutput_sha256(identical in both builds) and, for the 84 dual cases, the misaligned sequence inmisaligned_sha256(checked withzicclsm).Tracer. A byte-model test covers all eight offsets × 11 kinds × {positive, negative immediate} × {plain, compressed, rd = x0}. With
zicclsmit checks registers, memory, the sequence chosen and the RAM records. Without it, a misaligned access must abort. Misaligned atomics abort in both builds.x86.
zicclsm, at every aligned offset without it.memory_opspasses the record-mode and fast-run equivalence tests in both builds.misaligned_memory_opsalso passes dense chunking withzicclsm, and a misaligned access past the memory plane faults on its second row.End to end. Without
zicclsm, memory-ops'memory_opsproves every aligned offset of every width. With it,misaligned_memory_opsproves every in-bounds offset of every width, by immediate and from one PC per kind with a walking base. Both run in clear, ZK and Akita modes. The field-inline matrix passes in both builds.Soundness.
misaligned_memory.rsforces the aligned sequence on misalignedLH/LHU/LW/LWU/SH/SW/LD/SD. In both builds each witness satisfies every R1CS row and is rejected by RAF evaluation. Tamper tests reject a committed artifact with a foreignbytecode_lanes(Dory clear and ZK, Akita).Suites.
zicclsmAlso passing:
zicclsm; clear, ZK and Akita with it);akita×field-inline×zicclsmcombinations, and the Fiat-Shamir inventory;-D warnings), fmt, taplo, the style-invariants check and the fuzz lockfile check.CI. New lanes cover:
zicclsmclippy;Measurements
Scale-20 profile inputs, Tier 1 → this PR with
zicclsm. Prover times are means of three interleaved runs of the optimized backend. Builds withoutzicclsmexpand exactly as Tier 1 does.btreemapsha2-chainsha3-chainfibonaccizicclsm.Notes
ram_Ksizing, existing and not fixed here.jolt-prover/src/config.rssizesram_Kfrom the highest touched index with no+1. An access to the very top of memory can therefore fall outside the domain, whether or not it is misaligned. It surfaces as the jolt-witness "beyond ram_k" error.misaligned-ldst-01test is not enabled (tests/arch-tests/jolt/sail.jsonstill declares misaligned accesses unsupported). It would need azicclsmbuild of the arch-test harness, so I left it for a follow-up.zicclsm, the entry PC is the aligned head. A program whose very first instruction is a misaligned dual access cannot prove that first execution. This affects liveness only.🤖 Generated with Claude Code