Skip to content

feat(transcript): native Fr absorb and squeeze for the Poseidon transcript - #2103

Closed
quangvdao wants to merge 4 commits into
transcript/j3-sponge-powfrom
transcript/j4-poseidon-native-fr
Closed

quangvdao wants to merge 4 commits into
transcript/j3-sponge-powfrom
transcript/j4-poseidon-native-fr

Conversation

@quangvdao

Copy link
Copy Markdown
Collaborator

Stacked on #2099 (transcript/j3-sponge-pow).

What changed

The Poseidon transcript now works over BN254 Fr elements. Before this change it worked over bytes and turned each squeezed Fr into little-endian bytes. Now it absorbs and squeezes field elements directly.

  • Sponge. PoseidonSponge is spongefish's overwrite-mode DuplexSponge, with FrUnit (one Fr) as its unit. The permutation is circomlib's width-5 Poseidon (x^5 S-box, 8 full and 60 partial rounds), built from the constants light-poseidon ships. Rate is 3 elements (slots 0–2) and capacity is 2 elements (slots 3–4). A single element of capacity would give only about 2^127 hash calls, short of the 2^128 target. Two elements give about 2^253.
  • Challenges. A Poseidon challenge is a squeezed Fr element, used as is: no byte decoding and no reduction bias. Poseidon does not implement the sealed UniformBytes trait, so challenges decoded from bytes, and challenges in any other field, cannot be requested on Poseidon. Asking for them is a compile error. field_challenges now goes through a new sealed trait, ChallengeSponge<F>. Keccak and Blake2b get it from their existing byte-decoding path, and Poseidon implements it only for Fr. The challenge-size cap counts 32 bytes per element and is checked before anything is squeezed.
  • Field messages. New FieldMsg<F>. On Poseidon it is absorbed as one element. In the proof it is written as the element's canonical little-endian bytes, and the verifier rejects short or non-canonical bytes.
  • Grinding. Proof of work on Poseidon squeezes one Fr and requires the low bits bits of its canonical integer to be zero. For bits <= 25 those bits are within 2^-228 of uniform. Each nonce byte is absorbed as one element, both in prover and verifier messages and in the MessageSearch probe, so the search tests exactly what the verifier checks. Grinding still runs once per round. The unit-specific parts are in a sealed trait, PowUnit, implemented for u8 and FrUnit.
  • Verifier failures. spongefish's Transcript::check exists only for byte sponges. The verifier therefore poisons the transcript itself: on failure it reads a message that never parses, and it never absorbs or squeezes while doing so. Every rejection still makes check_eof fail: bad length, non-canonical element, truncated input, failed proof of work, oversized challenge request.
  • Legacy facade. PoseidonTranscript is now a plain struct (no field parameter). It packs the label and framed appends as described below, and challenge() returns a full-width Fr.
  • Keccak and Blake2b transcripts are unchanged, byte for byte. Their existing known-answer tests pass without edits.

How bytes are packed

Bytes enter the Poseidon sponge only in these forms:

  • A byte string of length n (a BytesMsg, the session identifier, or either part of the transcript instance) becomes the element n, followed by ceil(n / 31) elements. Each of those is the little-endian integer of the next 31 bytes, and the last chunk may be shorter. Every chunk is below 2^248, which is below the field modulus, so every chunk is a canonical element. The leading length fixes how many chunks follow and how long the last one is. The map is therefore injective, and no packed string is a prefix of another.
  • A single u8 is one element, the byte's value. The proof-of-work nonce is sent this way, one byte per message. A [u8; N] therefore costs N elements; use BytesMsg for byte strings.
  • A FieldMsg<Fr> is one element, the field element itself.

Soundness audit

An independent auditor reviewed the change at xhigh effort. They covered capacity and rate, permutation correctness, domain separation, packing injectivity, challenge uniformity, grinding, verifier poisoning, and verifier panics. They found no exploitable bug.

  • Capacity. The original width-4 design had one element of capacity, about 2^126.8, which falls short of the 2^128 target. This PR therefore uses width 5 with two elements of capacity, as the auditor recommended.
  • Permutation test. The auditor noted that the permutation was only checked on output slot 0. A new test pins all five input and output slots against circomlibjs 0.1.7 poseidon(inputs, init, 5).
  • Open notes, carried over from before this PR:
    • PROTOCOL_ID was not bumped, because that would change the Keccak and Blake2b bytes too.
    • The legacy facade and the native transcript can reach the same sponge state when a label equals a session string. The byte sponges already behave this way.

Validation

All runs were on a remote sandbox.

  • cargo fmt --all --check
  • cargo clippy -p jolt-transcript --all-targets -- -D warnings with --all-features, with --features bn254,solinas,binary,prover-grinding, with default features, and with --no-default-features --features transcript-poseidon,prover-grinding
  • cargo nextest run -p jolt-transcript with the same four feature sets
  • cargo check --workspace, cargo clippy -p jolt-eval --all-targets -- -D warnings, and cargo check -p jolt-sdk --features host,transcript-poseidon
  • cargo nextest run -p jolt-verifier --test fs_obligations --features fs-audit, with the absorb-site inventory regenerated for the new Poseidon code

New Poseidon tests:

  • round trips of native Fr challenges, byte messages and field messages;
  • grinding at 1, 2 and 9 bits, plus forged-nonce rejection;
  • rejection of truncated, non-canonical and trailing input, each followed by a failing check_eof;
  • rejection of oversized challenge requests, which also poisons the transcript;
  • a replay on a bare sponge from the documented packing;
  • known-answer vectors for the native and legacy layouts.

Open questions

  • Cost. Width 5 costs a little more per permutation than width 4: 68 rounds instead of 64, and a 5×5 MDS. Recursion circuits that verify this transcript will need the width-5 constants.
  • Slot order. The capacity sits in slots 3–4, because spongefish's DuplexSponge puts the rate first. circomlib and the Poseidon reference keep the capacity in slot 0. The ideal-permutation bound does not depend on position, but a circuit verifier has to use the same order.
  • Legacy challenge width. Legacy PoseidonTranscript::challenge() now returns a full 254-bit Fr, where it used to decode 16 squeezed bytes (about 128 bits). The prover multiplies by wider challenges as a result.

🤖 Generated with Claude Code

…cript

The Poseidon sponge now works over BN254 Fr elements: spongefish's
overwrite-mode DuplexSponge with the circom width-4 permutation (rate 3,
capacity 1). Byte strings pack as their length followed by 31-byte
little-endian chunks. Challenges on Poseidon are squeezed Fr elements;
byte-decoded challenges stay off Poseidon. Grinding squeezes one Fr and
tests its low bits.
Width 5, rate 3, capacity 2: a single element of capacity bounds the
sponge at about 2^127 calls, short of the 2^128 target. Pins the full
permutation state against circomlibjs and regenerates the verifier's
absorb-site inventory.
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

📏 PR diff

Note

Actual code changed: 703 lines

🟢 +451 added    🔴 −252 removed

Tests, docs, fixtures, and helper scripts excluded.

Category 🟢 Added 🔴 Removed
⌨️ Actual code +451 −252
🧪 Tests +420 −119
📚 Docs +188 −72
📦 Fixtures 0 0
🔧 Helper scripts 0 0
Total diff +1,059 −443

Every changed line is classified once.

… transcript/j4-poseidon-native-fr

# Conflicts:
#	crates/jolt-verifier/tests/fs_inventory/absorb-sites.inventory
@quangvdao
quangvdao added this pull request to stack #2098 October 21, 2026 15:49
@quangvdao
quangvdao marked this pull request as ready for review October 21, 2026 15:49
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark comparison (crates)

Same-runner A/B: base_run = cae3051 (merge target) vs pr_run = 0341b90 (PR head merged onto it). Bold = faster run; Δ colored when the difference exceeds the ±10% noise threshold.

2 of 81 benchmarks differ by more than ±10%:

Benchmark base_run (merge target) pr_run (this PR) Δ
append_bytes/Poseidon/256B 161.8±9.11µs 81.5±1.14µs ${\color{green}\mathbf{-49.6\%}}$
append_bytes/Poseidon/32B 50.5±0.90µs 27.2±0.39µs ${\color{green}\mathbf{-46.2\%}}$
All 81 benchmarks
Benchmark base_run (merge target) pr_run (this PR) Δ
EqPolynomial::evals/17 1.0±0.03ms 1.1±0.02ms +3.5%
EqPolynomial::evals/19 4.0±0.08ms 4.0±0.05ms +2.4%
EqPolynomial::evals/20 8.5±0.27ms 9.0±0.44ms +5.5%
EqPolynomial::evals/22 32.7±2.39ms 33.9±1.74ms +3.6%
EqPolynomial::evaluations/14 275.7±16.26µs 277.1±12.91µs +0.5%
EqPolynomial::evaluations/18 3.4±0.06ms 3.5±0.07ms +3.1%
EqPolynomial::evaluations/20 15.6±1.83ms 16.0±0.39ms +2.5%
F128/accumulator/deferred 1.6±0.09µs 1.7±0.03µs +2.5%
F128/accumulator/naive 2.9±0.04µs 2.9±0.06µs -0.7%
F128/mul 2.6±0.08ns 2.6±0.20ns -1.0%
F128/mul_slice 2.4±0.26µs 2.4±0.06µs +0.2%
F128/square 2.1±0.03ns 2.1±0.04ns +1.5%
F192/accumulator/deferred 3.1±0.07µs 3.1±0.15µs +0.4%
F192/accumulator/naive 5.6±0.12µs 5.6±0.12µs -0.4%
F192/mul 5.6±1.01ns 5.6±0.47ns -0.2%
F192/mul_slice 5.8±0.09µs 5.6±0.12µs -2.4%
F192/square 4.2±0.10ns 4.1±0.09ns -2.9%
F64/accumulator/deferred 718.4±24.58ns 716.2±34.84ns -0.3%
F64/accumulator/naive 1.5±0.29µs 1.4±0.09µs -6.0%
F64/mul 1.4±0.02ns 1.4±0.17ns +3.3%
F64/mul_slice 1.4±0.03µs 1.4±0.03µs +1.0%
F64/square 1.4±0.21ns 1.4±0.06ns -3.6%
Polynomial::bind/14 70.5±2.40µs 71.9±4.17µs +2.0%
Polynomial::bind/18 1.1±0.05ms 1.2±0.02ms +2.4%
Polynomial::bind/20 4.3±0.10ms 4.5±0.13ms +4.4%
Polynomial::evaluate/20 22.1±0.91ms 23.2±0.26ms +5.1%
UnivariatePoly::from_evals/16 34.7±1.70µs 34.8±0.61µs +0.4%
UnivariatePoly::from_evals/3 4.1±0.14µs 4.3±0.08µs +3.7%
UnivariatePoly::from_evals/4 6.1±0.14µs 6.4±0.12µs +4.6%
UnivariatePoly::from_evals/8 15.6±0.52µs 15.9±0.85µs +1.7%
append_bytes/Blake2b/256B 199.1±6.06ns 200.3±18.41ns +0.6%
append_bytes/Blake2b/32B 112.3±3.43ns 116.4±6.86ns +3.6%
append_bytes/Keccak/256B 530.7±16.88ns 533.0±5.29ns +0.4%
append_bytes/Keccak/32B 19.4±1.41ns 19.8±3.76ns +2.1%
append_bytes/Poseidon/256B 161.8±9.11µs 81.5±1.14µs ${\color{green}\mathbf{-49.6\%}}$
append_bytes/Poseidon/32B 50.5±0.90µs 27.2±0.39µs ${\color{green}\mathbf{-46.2\%}}$
challenge/Blake2b 393.0±8.43ns 396.0±15.66ns +0.8%
challenge/Keccak 264.6±2.67ns 256.8±3.85ns -3.0%
challenge/Poseidon 26.0±0.85µs 26.8±0.29µs +3.2%
field arithmetic/add/fp128_a7f7 3.7±0.08µs 3.8±0.03µs +2.6%
field arithmetic/add/fp32_c99 1.9±0.05µs 1.9±0.04µs -1.1%
field arithmetic/add/fp64_c59 1.9±0.11µs 1.9±0.03µs -1.0%
field arithmetic/mul/fp128_a7f7 9.6±0.68µs 9.4±0.19µs -2.1%
field arithmetic/mul/fp32_c99 3.6±0.09µs 3.5±0.04µs -2.0%
field arithmetic/mul/fp64_c59 4.7±0.09µs 4.6±0.08µs -1.3%
field arithmetic/square/fp128_a7f7 11.1±0.32µs 11.1±0.28µs -0.3%
field arithmetic/square/fp32_c99 3.8±0.15µs 3.7±0.04µs -1.7%
field arithmetic/square/fp64_c59 4.2±0.18µs 4.1±0.07µs -2.1%
field arithmetic/sub/fp128_a7f7 2.3±0.05µs 2.2±0.03µs -4.1%
field arithmetic/sub/fp32_c99 1.4±0.07µs 1.3±0.05µs -4.6%
field arithmetic/sub/fp64_c59 1.3±0.03µs 1.3±0.05µs +0.1%
fp64_prime63_offset259/ext2_packed_mul 13.7±2.74ns 13.0±0.15ns -5.0%
fp64_prime63_offset259/ext2_scalar_mul 6.4±0.55ns 6.6±0.07ns +2.3%
fp64_prime63_offset259/packed_mul 5.1±0.18ns 5.2±0.10ns +2.2%
fp64_prime63_offset259/scalar_mul 1.5±0.11ns 1.4±0.02ns -4.2%
g1_add 237.3±21.06ns 241.1±13.78ns +1.6%
g1_deserialize_bincode 5.0±0.30µs 5.0±0.09µs -0.9%
g1_double 123.1±7.30ns 124.4±1.73ns +1.0%
g1_msm/1024 7.5±0.38ms 7.5±0.06ms +0.1%
g1_msm/16 311.8±4.26µs 316.8±10.48µs +1.6%
g1_msm/256 2.5±0.15ms 2.5±0.10ms +2.2%
g1_msm/4 146.5±6.31µs 147.4±1.52µs +0.6%
g1_msm/64 852.8±45.01µs 887.1±44.71µs +4.0%
g1_scalar_mul 36.5±0.70µs 37.8±1.53µs +3.6%
g1_serialize_bincode 67.8±2.15ns 67.5±2.59ns -0.4%
g2_msm/16 1.1±0.03ms 1.1±0.04ms +2.9%
g2_msm/256 9.2±0.18ms 9.5±0.35ms +3.1%
g2_msm/4 425.4±11.91µs 447.5±41.94µs +5.2%
g2_msm/64 3.3±0.07ms 3.3±0.02ms +1.8%
g2_scalar_mul 187.2±9.46µs 187.7±3.20µs +0.3%
gt_scalar_mul 503.1±21.94µs 491.2±20.96µs -2.4%
multi_pairing/16 3.6±0.20ms 3.6±0.15ms +1.9%
multi_pairing/2 707.6±13.78µs 727.0±10.88µs +2.7%
multi_pairing/4 1.1±0.04ms 1.1±0.01ms +1.1%
multi_pairing/8 1.9±0.07ms 1.9±0.07ms +2.6%
pairing 478.9±29.42µs 492.5±10.90µs +2.8%
pedersen_commit/1024 7.5±0.53ms 7.7±0.10ms +1.7%
pedersen_commit/16 349.0±31.50µs 352.7±6.39µs +1.1%
pedersen_commit/256 2.5±0.19ms 2.6±0.09ms +1.3%
pedersen_commit/4 178.4±4.19µs 186.4±6.29µs +4.5%
pedersen_commit/64 917.6±61.41µs 934.4±16.60µs +1.8%

@quangvdao

Copy link
Copy Markdown
Collaborator Author

Superseded by #2113, which ports this Poseidon design onto #2110.

@quangvdao quangvdao closed this Oct 21, 2026
An error occurred while trying to automatically change base from transcript/j3-sponge-pow to main October 24, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant