[quangvdao]

Stacked on #2110 (transcript/j7-sponge-port). Supersedes #2103.

Why a new PR instead of retargeting #2103. #2103 was built on J3 and edits the legacy facade, including legacy.rs, which #2110 deletes. The two conflict in 13 files. Moving #2103 onto #2110 would mean rewriting its history and force-pushing it. This branch ports the audited Poseidon design onto #2110's API. It now merges #2110's head 6cea4bc, which contains J6's adaptive grind search, main (through J2, 7931b22) and #2110's short scalar challenges. The update is an ordinary merge commit, with no rebase or force-push. #2103 is closed.

What changed

#2110 still ships the older byte-based Poseidon sponge. Its capacity gives only about 2^127 hash calls to break, below the 2^128 target. This PR replaces it with the width-5 sponge over BN254 Fr that the J4 audit approved, so the crate has exactly one Poseidon sponge.

• Sponge. PoseidonSponge is spongefish's overwrite-mode DuplexSponge, with FrUnit (one Fr) as its unit. The permutation is circomlib's width-5 Poseidon (x^5 S-box, 8 full and 60 partial rounds), built from the constants light-poseidon ships. Rate is 3 elements (slots 0–2) and capacity is 2 elements (slots 3–4), about 2^253 generic security.
• Messages on any sponge unit. refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 writes every message through a byte Encoding. A new sealed trait, SpongeUnit, maps a message's bytes into the sponge's units: for u8 it is the identity, and for FrUnit it packs the bytes as described below. A small internal adapter, Units<M>, feeds the result to spongefish's ProverState/VerifierState. The Transcript sealed core gains type Unit, and ProverTranscript::Sponge is tied to it (DuplexSpongeInterface<U = Self::Unit>).
• Challenges. FieldTranscript<F> now draws through one sealed trait, ChallengeSponge<F>. Keccak and Blake2b get it from their existing byte-decoding path (UniformBytes + ChallengeCodec), unchanged. Poseidon implements it only for Fr: a challenge is a squeezed element, used as is, with no byte decoding and no reduction bias. Asking Poseidon for challenges in any other field is a compile error. The challenge-size cap counts 32 bytes per element and is checked before anything is squeezed.
• Short challenges. refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110's challenge_short_scalar draws an integer of b bits (b from the step's loss factor, at most 192 bits for a u64 loss). It now reads that integer through SpongeUnit::short_word, so it runs on any sponge unit:
  • On Keccak and Blake2b it squeezes ceil(b / 8) bytes and clears the bits above b, exactly as refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 does.
  • On Poseidon it squeezes two elements x0 and x1 and keeps the low b bits of x0 + p * x1. That integer is uniform on [0, p^2), so its low b bits are within 2^b / p^2 <= 2^-253 of uniform. One element alone would be about 2^-64 from uniform at b = 192, which misses the 2^-160 target.
  • jolt-dory's label-based short Dory widths come from refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 unchanged, and run on Poseidon too.
• Grinding. Proof of work on Poseidon squeezes one Fr and requires the low bits bits of its canonical integer to be zero; for bits <= 25 those bits are within 2^-228 of uniform. The nonce is still sent one byte per prover message. On Poseidon each byte is packed as a message of its own (2 elements), and the MessageSearch probe absorbs exactly the same units, so the search tests what the verifier checks. Grinding still runs once per round.
• Verifier failures. spongefish's Transcript::check exists only for byte sponges. The verifier therefore poisons itself: on failure it reads a message type that never parses and absorbs nothing. Every rejection still makes check_eof fail: truncated input, failed proof of work, bad difficulty, oversized challenge request.
• Poseidon protocol build is back. jolt-sdk and the fibonacci and merkle-tree-save examples regain the transcript-poseidon feature, which sets ProtocolSponge = PoseidonSponge. Keccak stays the default. Enabling transcript-blake2b and transcript-poseidon together is a compile error.
• ScriptedTranscript stays byte-only (it wraps transcripts whose unit is u8), because its script checks bytes.
• BlindFold. The replay-divergence check in jolt-prover/src/blindfold.rs now compares 32 squeezed units of the transcript's own unit type, so it works on any sponge.
• Keccak and Blake2b transcripts are unchanged, byte for byte. Their known-answer tests pass without edits.

How bytes are packed on Poseidon

Every message, the session identifier and the transcript instance reach the Poseidon sponge as byte strings, and each one is packed on its own:

• A byte string of length n becomes the element n, followed by ceil(n / 31) elements. Each of those is the little-endian integer of the next 31 bytes, and the last chunk may be shorter. Every chunk is below 2^248, which is below the field modulus, so every chunk is a canonical element.
• The leading length fixes how many chunks follow and how long the last one is. The map is therefore injective, and no packed string is a prefix of another. A message boundary can never be moved or hidden by changing the bytes.
• Costs: a one-byte message (each nonce byte) is 2 elements, and a 32-byte field element is 3 elements.

Soundness audit

Merge with 6cea4bc. A second independent audit at xhigh effort covered the merge. Verdict: sound. It found the following:

• The Keccak and Blake2b short challenges are byte-for-byte and value-for-value those of refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110, on success and on failure.
• The Poseidon short challenge computes (x0 + p * x1) mod 2^256 correctly. The auditor checked this against exact integer arithmetic on 200,100 input pairs for every b from 1 to 253. Its exact distance from uniform is 2^-381.7 at b = 128, 2^-326.7 at b = 192, and at most 2^-256.8 for any b <= 253.
• The two elements are fresh rate slots, so they cannot alias other challenges.
• Nothing refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 added since the merge base was dropped.
• The new path cannot panic, and it poisons the verifier on refusal.
• Fixed from that audit: a dropped carry flag that failed -D warnings, and stale docs in challenge.rs, poseidon.rs, lib.rs and the two fuzz targets.

Original port. An independent auditor reviewed the port at xhigh effort. They covered capacity and rate, the permutation, packing injectivity and domain separation, byte-sponge equivalence with #2110, challenge uniformity and the cap, grinding, verifier poisoning and verifier panics. They found no exploitable issue.

• Recomputed independently (standalone Python, not the Rust code): the width-5 permutation against all three full-state vectors; the round constants and MDS regenerated with the Poseidon reference (Grain) generator, matching light-poseidon; the Poseidon known-answer vector from scratch; the unchanged Keccak known-answer vector; and a property check that packed messages always decode uniquely and none is a prefix of another.
• Fixed in this PR: the fuzz harnesses had dropped to observing 4 squeezed units, which is 4 bytes on Keccak and Blake2b, so a long fuzz run could raise a false crash. They observe 32 units again (16 per challenge), as on refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110.
• Notes, no action needed:
  • In spongefish's overwrite-mode sponge, how many challenges are squeezed within one rate block does not change later states. Soundness relies on a fixed protocol schedule, as it already does for Keccak. On Poseidon the per-message length prefix makes aliasing even less likely (about 2^-190).
  • Poseidon never grinds in production: Fr's challenge set is so large that grind_bits_for_loss returns 0. The path is correct and tested.
  • constants() has assert/expect calls on fixed library data. Input cannot reach them, and refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 already had expect on this path.

The capacity and permutation checks from the J4 audit were re-run on this branch:

• poseidon.rs unit tests pin all five input and output slots of the permutation against circomlibjs 0.1.7 poseidon(inputs, init, 5), and output slot 0 against light-poseidon.
• poseidon_tests replays a sample transcript on a bare sponge from the documented packing and checks that the challenges match, and pins a known-answer vector for two native challenges.

Validation

All runs were on a remote sandbox at this branch's head after the merge with 6cea4bc.

• cargo fmt --all --check; python3 scripts/check_style_invariants.py --base 6cea4bc85; typos.
• cargo clippy -p jolt-transcript --all-targets -- -D warnings with --all-features, default features, --no-default-features --features transcript-poseidon,prover-grinding, --features bn254,solinas,binary,prover-grinding, and --features test-utils.
• cargo nextest run -p jolt-transcript with --all-features, default features (Keccak only), and --no-default-features --features transcript-poseidon,prover-grinding. All pass.
• The Keccak and Blake2b known-answer tests on their own: cargo nextest run -p jolt-transcript --features bn254,transcript-blake2b --test keccak_tests --test blake2b_tests. They pass on the files refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 ships.
• jolt-dory: cargo clippy --all-targets -- -D warnings and cargo nextest run.
• jolt-sumcheck: cargo clippy -p jolt-sumcheck -p jolt-blindfold --all-targets -- -D warnings, cargo clippy -p jolt-sumcheck --all-targets --features committed,prover -- -D warnings, and cargo nextest run -p jolt-sumcheck --features committed,prover.
• jolt-prover: cargo clippy --all-targets -- -D warnings with default features and with --features prover-fixtures,zk; cargo nextest run -p jolt-prover (default features).
• refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110's Fiat-Shamir obligations and inventory test: cargo nextest run -p jolt-verifier --test fs_obligations --features fs-audit, with and without jolt-transcript/transcript-poseidon. The inventory needed no change from the merge.
• cargo clippy -p jolt-sdk --all-targets --features host,transcript-poseidon -- -D warnings; cargo clippy -p jolt-eval --all-targets -- -D warnings; cargo check of the jolt-transcript fuzz crate.
• Run before the merge, on the d04df1b base: cargo clippy --all --all-targets -- -D warnings with --features allocative,host, --features allocative,host,zk and --no-default-features, plus cargo check --workspace.
• Run before the first rebase, on 6691ec7: cargo check -p fibonacci --features transcript-poseidon, and a one-off run of the jolt-sumcheck clear-sumcheck example switched to PoseidonSponge over Fr, which proved and verified (not committed).
• cargo doc -p jolt-transcript --all-features --no-deps with -D warnings reports two broken intra-doc links that refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110 already has ([fail] in challenge.rs and [1600] in lib.rs); this PR adds none. CI does not run rustdoc.
• Not run:
  • The jolt-prover fixture tests (--features prover-fixtures[,zk]). They need the guest toolchain, so CI runs them.
  • A full Jolt proof with transcript-poseidon, which also needs the guest toolchain. CI's Keccak end-to-end jobs cover the shared code paths.

Test changes against #2110

Keccak and Blake2b known-answer tests are byte-identical. keccak_tests.rs and blake2b_tests.rs are not touched by this PR and pass unchanged, so those transcripts produce the same bytes as on #2110. challenge_tests.rs, narg_eof_tests.rs and native_traits_tests.rs are also untouched.

Against #2110's head d04df1b, no assertion on a byte sponge was removed or loosened. Every removed or rewritten assertion is listed below. #2110's search_is_independent_of_the_thread_count test in pow.rs is kept as #2110 has it. The search it checks is now generic over the sponge unit. #2110's short-challenge tests (test_challenge_short_scalar for Keccak and Blake2b, and short_scalar_refuses_a_field_without_room) run unchanged.

New for Poseidon short challenges:

• short_challenge_keeps_the_low_bits_of_two_elements checks b = 128, 130 and 192 against x0 + p * x1 computed with num-bigint. It also checks that the next challenge is the third squeezed element, and that the verifier draws the same value.
• short_challenge_refuses_zero_loss_and_poisons checks that a zero loss factor is refused before anything is squeezed, and that it poisons the verifier.

Removed or replaced (all Poseidon, whose sponge this PR replaces):

• poseidon_tests::test_poseidon_known_vector pinned 32 squeezed bytes from the old byte sponge. That vector cannot hold once the sponge changes. It is replaced by native_known_vector, which pins the new sponge's NARG bytes and two Fr challenges, and by a test that replays the transcript on a bare sponge from the documented packing.
• test_poseidon_determinism_and_order squeezed bytes, which Poseidon no longer offers. Its two checks move to challenges_bind_messages_session_and_instance: the same messages give the same challenges, and swapping two messages changes them.
• test_poseidon_prover_and_verifier_agree is covered by round_trips_byte_and_scalar_messages, which appends a public message on both sides and compares every challenge.
• The poseidon.rs unit tests of the hand-written byte sponge are gone with that sponge: empty versus zero absorb, clone resume, clone divergence and ratchet. The new sponge is spongefish's DuplexSponge, so its clone and ratchet code is spongefish's. New unit tests cover:
  • the permutation against circomlibjs on all five slots, and against light-poseidon on slot 0;
  • packing, where an empty message and [0] pack to different lengths;
  • divergence for distinct inputs (including [0] versus [0, 0]) and for distinct sessions;
  • associativity of absorb and squeeze;
  • clone resume in the middle of a rate block;
  • the proof-of-work low-bits read.

Rewritten without loss:

• pow_tests::sponge_checks and verify_nonce now run on any sponge unit. They still compare 16 squeezed units after the grind and 40 units for the probe, so on Keccak and Blake2b they compare the same bytes as before. One new assertion checks that the probe's check units pass and that its next 16 units equal the prover's challenge.
• The jolt-transcript fuzz harnesses observe 32 units for states and 16 per challenge, the same byte counts as on refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110.
• The jolt-eval transcript-symmetry invariant compares challenges as Fr values drawn through FieldTranscript, not as 32 raw squeezed bytes, so the same invariant also runs on Poseidon. On Keccak and Blake2b each Fr is reduced from at least 32 squeezed bytes, so the comparison still covers about 254 bits.
• pow.rs's layout test reads the check length from u8::check_len, which gives the same value as the removed word_len.

Open questions

• Field messages. feat(transcript): native Fr absorb and squeeze for the Poseidon transcript #2103 also had FieldMsg<F>, which absorbed a field element as one element. This PR drops it: field elements go through their byte encoding like every other message, costing 3 elements instead of 1. That keeps a single message path through refactor(transcript): run Jolt on spongefish ProverState/VerifierState and delete the legacy facade #2110's byte Encoding API. A native one-element path can come back if recursion cost needs it.
• Cost. Width 5 costs a little more per permutation than width 4: 68 rounds instead of 64, and a 5×5 MDS. Recursion circuits that verify this transcript will need the width-5 constants.
• Slot order. The capacity sits in slots 3–4, because spongefish's DuplexSponge puts the rate first. circomlib and the Poseidon reference keep the capacity in slot 0. The ideal-permutation bound does not depend on position, but a circuit verifier has to use the same order.

🤖 Generated with Claude Code

[github-actions]

📏 PR diff

Note

Actual code changed: 707 lines

🟢 +450 added    🔴 −257 removed

Tests, docs, fixtures, and helper scripts excluded.

Category	🟢 Added	🔴 Removed
⌨️ Actual code	+450	−257
🧪 Tests	+436	−145
📚 Docs	+209	−89
📦 Fixtures	0	0
🔧 Helper scripts	0	0
Total diff	+1,095	−491

_{Every changed line is classified once.}