Skip to content

feat(transcript): proof of work on the transcript sponge - #2099

Merged
0xAndoroid merged 17 commits into
mainfrom
transcript/j3-sponge-pow
Oct 24, 2026
Merged

0xAndoroid merged 17 commits into
mainfrom
transcript/j3-sponge-pow

Conversation

@quangvdao

@quangvdao quangvdao commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #2097 (J2), which is stacked on #2095 (J1). Third step of moving Jolt, then Akita, onto one Keccak duplex-sponge transcript.

What this adds

A proof-of-work step on the transcript's own sponge: ProverTranscript::grind(bits) and VerifierTranscript::check_grind(bits). It is a step of its own, not tied to one challenge call.

  • bits == 0: does nothing.
  • 1 ≤ bits ≤ 25 (MAX_POW_BITS):
    1. The prover sends a nonce below 2^(bits+7) in canonical LEB128, one byte per prover message.
    2. Both sides squeeze ⌈bits/8⌉ bytes and accept only if their low bits bits are zero.
      The prover sends the least nonce that passes, so honest proofs are deterministic.
  • Above 25 bits: both sides return VerificationError before touching the sponge. Nothing panics.
  • Rule for callers: the step protects the verifier messages drawn after it, up to the next prover message. A protocol draws the challenges it protects directly after the step. One step can protect several challenge messages, of different fields.

The layout is the one in arkworks-rs/spongefish#301 (PowTranscriptExt::sponge_pow). There is no separate proof-of-work hash. With Keccak, each search attempt costs one permutation.

Why a varint nonce

The search space is 2^(bits+7), so the honest search fails with probability at most e^-128. Most nonces are small, so LEB128 is short on average. For a run of about 400 grinding steps between 1 and 10 bits, the nonces take about 420 bytes in expectation, against about 820 for a fixed-width nonce and about 3,300 for a u64. The cap of 25 bits keeps the nonce in a u32; no protocol should grind anywhere near that.

Verifier behaviour

  • The verifier accepts only canonical encodings in range: no trailing zero byte, no continuation past the maximum length, no value at or above 2^(bits+7).
  • Every failure (unsupported difficulty, truncated, non-canonical or out-of-range nonce, or check bytes that are not zero) goes through spongefish::Transcript::check or a failed read, so the state is poisoned and check_eof fails afterwards too.
  • Any passing nonce in range is accepted, not only the least, so proof bytes are not unique. This is documented.

Prover search

  • MessageSearch (behind prover-grinding) copies the sponge and implements Probe: absorb a candidate message, squeeze some bytes, without touching the transcript.
  • It absorbs the candidate one byte at a time, as the real sends do. Jolt's PoseidonSponge absorbs a length with every call, so absorbing the whole nonce at once would give a different state than the transcript, and the verifier would reject.
  • parallel searches on all threads with rayon and returns the same least nonce as the sequential scan.

Dependencies: spongefish 0.8.0 only

  • The verifier needs nothing beyond the published 0.8.0. check_grind is always available.
  • The prover search copies the sponge state, which 0.8.0 exposes only under its yolocrypto feature. That feature makes internals public and changes no computation. grind and MessageSearch therefore sit behind a new prover-grinding feature, which turns on spongefish/yolocrypto.
  • cargo tree -e features shows no yolocrypto for default jolt-transcript or for jolt-prover, since no protocol grinds yet. The one yolocrypto in jolt-verifier's tree comes from Akita's own akita-transcript; moving Akita over removes it.
  • parallel (it implies prover-grinding) searches on all threads with rayon.
  • Once spongefish#301 is released, MessageSearch can defer to spongefish's search primitive and yolocrypto goes away.

Security note

For the Keccak duplex sponge, the reduction of Chiesa and Orrù prices each attempt per permutation call. The Blake2b hash bridge and Poseidon need their own argument; the docs say so.

Tests (tests/pow_tests.rs, plus unit tests in src/pow.rs)

  • Layout: the step is rebuilt by hand for bits ∈ {1, 7, 8, 9, 16} and challenge lengths {0, 1, 32, 200}. The test also checks that every smaller nonce fails, so the sent nonce really is the least one.
  • Round trips: one grind protecting two challenge messages (Fp128<2^128 − 275> and Fr, both orders) at bits 0, 1, 8, 9 and 16.
  • Every sponge: for Keccak, Blake2b and Poseidon at bits 1, 2 and 9, the search equals sending then squeezing, the honest nonce round-trips and is the least, a larger passing nonce in range is accepted (proofs are not unique), and a passing nonce out of range is rejected and poisons the verifier.
  • bits = 0: byte-identical to no step.
  • MessageSearch: probing matches sending then squeezing, and leaves the transcript untouched.
  • Bad input: smaller, non-canonical (trailing zero byte) and truncated nonces are rejected, and check_eof fails afterwards. 26 bits and u32::MAX bits are rejected with the transcript unchanged.
  • Unit tests: difficulty bounds, layout table, canonical LEB128 decoding, a parallel search that finds a lone accepted nonce anywhere in the space, and one that returns the least of several accepted nonces spread across threads.

Validation (Linux x86 sandbox)

  • cargo clippy -p jolt-transcript --all-targets -- -D warnings with --all-features, --features bn254,solinas,binary,prover-grinding, --no-default-features --features transcript-keccak, and default features
  • cargo nextest run -p jolt-transcript: 103 passed with --all-features, 103 with bn254,solinas,binary,prover-grinding, 10 with Keccak only

CI gains a clippy run and a test run for bn254,solinas,binary,prover-grinding, since --all-features turns on parallel and would skip the sequential search. Rebased onto the updated J2, which caps message sizes, skips empty squeezes and poisons the verifier when the rejection pool runs out.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

📏 PR diff

Note

Actual code changed: 424 lines

🟢 +422 added    🔴 −2 removed

Tests, docs, fixtures, and helper scripts excluded.

Category 🟢 Added 🔴 Removed
⌨️ Actual code +422 −2
🧪 Tests +681 0
📚 Docs +195 0
📦 Fixtures +2 0
🔧 Helper scripts +6 0
Total diff +1,306 −2

Every changed line is classified once.

@quangvdao
quangvdao added this pull request to stack #2098 October 20, 2026 18:34
@quangvdao
quangvdao force-pushed the transcript/j3-sponge-pow branch from 40aff20 to b918e64 Compare October 20, 2026 18:35
@quangvdao quangvdao changed the title feat(transcript): proof of work on the transcript sponge before a challenge message (J3) feat(transcript): proof of work on the transcript sponge Oct 20, 2026
Reject unsupported lengths and messages above MAX_CHALLENGE_BYTES before
touching the sponge; return early on empty messages; poison the verifier on
pool exhaustion; seal ChallengeCodec; correct the BN254 width table; lint and
test jolt-transcript with all features in CI.
@quangvdao
quangvdao force-pushed the transcript/j3-sponge-pow branch from 9411420 to eaefc6f Compare October 20, 2026 19:06
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Benchmark comparison (crates)

Same-runner A/B: base_run = 7931b22 (merge target) vs pr_run = f5a2197 (PR head merged onto it). Bold = faster run; Δ colored when the difference exceeds the ±10% noise threshold.

No differences above the ±10% noise threshold.

All 81 benchmarks
Benchmark base_run (merge target) pr_run (this PR) Δ
EqPolynomial::evals/17 1.8±0.04ms 1.8±0.02ms -0.3%
EqPolynomial::evals/19 6.9±0.09ms 6.9±0.11ms +0.5%
EqPolynomial::evals/20 14.4±0.10ms 14.7±0.19ms +2.6%
EqPolynomial::evals/22 54.9±0.78ms 54.7±0.59ms -0.4%
EqPolynomial::evaluations/14 498.8±11.61µs 496.7±10.11µs -0.4%
EqPolynomial::evaluations/18 6.4±0.15ms 6.4±0.13ms +0.0%
EqPolynomial::evaluations/20 30.4±0.43ms 31.0±0.41ms +2.0%
F128/accumulator/deferred 1.5±0.02µs 1.5±0.02µs +0.1%
F128/accumulator/naive 3.1±0.02µs 3.1±0.03µs +0.0%
F128/mul 2.3±0.12ns 2.3±0.10ns +0.2%
F128/mul_slice 2.2±0.01µs 2.2±0.02µs +0.0%
F128/square 3.1±0.06ns 3.1±0.04ns -0.1%
F192/accumulator/deferred 3.0±0.10µs 3.0±0.08µs +0.4%
F192/accumulator/naive 4.9±0.10µs 4.9±0.10µs +0.1%
F192/mul 3.8±0.03ns 3.8±0.03ns +0.0%
F192/mul_slice 3.6±0.02µs 3.7±0.07µs +0.2%
F192/square 2.6±0.02ns 2.6±0.05ns +0.2%
F64/accumulator/deferred 901.7±29.06ns 900.9±24.92ns -0.1%
F64/accumulator/naive 1.2±0.04µs 1.2±0.01µs -0.6%
F64/mul 1.0±0.03ns 1.0±0.04ns +0.0%
F64/mul_slice 891.3±17.61ns 887.2±3.45ns -0.5%
F64/square 0.9±0.00ns 0.9±0.00ns +0.1%
Polynomial::bind/14 140.9±10.62µs 137.6±4.85µs -2.3%
Polynomial::bind/18 1.8±0.02ms 1.8±0.02ms -0.5%
Polynomial::bind/20 7.2±0.08ms 7.1±0.03ms -0.5%
Polynomial::evaluate/20 42.5±0.47ms 42.2±0.36ms -0.7%
UnivariatePoly::from_evals/16 43.8±0.75µs 43.8±0.83µs +0.1%
UnivariatePoly::from_evals/3 5.3±0.06µs 5.3±0.11µs +0.2%
UnivariatePoly::from_evals/4 8.0±0.08µs 8.0±0.09µs +0.0%
UnivariatePoly::from_evals/8 19.6±0.19µs 19.6±0.36µs +0.2%
append_bytes/Blake2b/256B 348.6±8.56ns 347.8±5.09ns -0.2%
append_bytes/Blake2b/32B 199.9±6.79ns 199.9±7.20ns +0.0%
append_bytes/Keccak/256B 813.1±11.45ns 801.8±5.47ns -1.4%
append_bytes/Keccak/32B 41.5±4.12ns 38.1±3.02ns -8.3%
append_bytes/Poseidon/256B 207.6±4.84µs 207.2±5.17µs -0.2%
append_bytes/Poseidon/32B 65.5±1.16µs 66.0±1.86µs +0.8%
challenge/Blake2b 673.4±14.91ns 669.1±12.94ns -0.6%
challenge/Keccak 379.6±16.06ns 378.3±7.01ns -0.4%
challenge/Poseidon 32.8±0.87µs 32.5±0.54µs -0.8%
field arithmetic/add/fp128_a7f7 4.4±0.12µs 4.5±0.21µs +0.8%
field arithmetic/add/fp32_c99 3.3±0.14µs 3.2±0.07µs -0.7%
field arithmetic/add/fp64_c59 3.4±0.05µs 3.4±0.05µs +0.1%
field arithmetic/mul/fp128_a7f7 15.0±0.28µs 14.9±0.03µs -0.3%
field arithmetic/mul/fp32_c99 5.0±0.02µs 5.0±0.04µs +0.0%
field arithmetic/mul/fp64_c59 7.6±0.11µs 7.6±0.11µs -0.1%
field arithmetic/square/fp128_a7f7 18.2±0.22µs 18.2±0.13µs -0.1%
field arithmetic/square/fp32_c99 5.3±0.11µs 5.3±0.09µs -0.2%
field arithmetic/square/fp64_c59 6.5±0.21µs 6.5±0.21µs -0.3%
field arithmetic/sub/fp128_a7f7 4.0±0.11µs 4.1±0.13µs +0.6%
field arithmetic/sub/fp32_c99 2.4±0.01µs 2.4±0.04µs +0.1%
field arithmetic/sub/fp64_c59 2.5±0.06µs 2.5±0.02µs -0.2%
fp64_prime63_offset259/ext2_packed_mul 19.3±0.30ns 19.3±0.29ns +0.0%
fp64_prime63_offset259/ext2_scalar_mul 8.1±0.05ns 8.1±0.15ns +0.2%
fp64_prime63_offset259/packed_mul 7.8±0.15ns 7.8±0.04ns -0.1%
fp64_prime63_offset259/scalar_mul 1.9±0.01ns 1.9±0.03ns +0.1%
g1_add 375.5±5.22ns 376.9±10.92ns +0.4%
g1_deserialize_bincode 8.0±0.29µs 8.0±0.10µs -0.4%
g1_double 183.6±3.91ns 183.9±2.23ns +0.1%
g1_msm/1024 11.8±0.05ms 12.0±0.26ms +1.0%
g1_msm/16 513.5±11.44µs 517.7±9.13µs +0.8%
g1_msm/256 4.0±0.02ms 4.0±0.04ms +0.6%
g1_msm/4 229.7±0.86µs 232.7±5.72µs +1.3%
g1_msm/64 1.5±0.03ms 1.5±0.02ms +1.0%
g1_scalar_mul 58.2±1.21µs 58.0±0.53µs -0.4%
g1_serialize_bincode 107.1±3.62ns 105.9±0.87ns -1.1%
g2_msm/16 1.9±0.03ms 1.9±0.02ms +0.1%
g2_msm/256 14.9±0.25ms 14.9±0.18ms +0.0%
g2_msm/4 759.0±16.53µs 762.1±11.04µs +0.4%
g2_msm/64 5.4±0.04ms 5.4±0.12ms +0.7%
g2_scalar_mul 305.5±3.18µs 305.4±1.82µs +0.0%
gt_scalar_mul 885.7±9.15µs 879.6±11.50µs -0.7%
multi_pairing/16 5.6±0.02ms 5.6±0.02ms +0.0%
multi_pairing/2 1.2±0.02ms 1.2±0.03ms +0.0%
multi_pairing/4 1.8±0.02ms 1.8±0.03ms +0.0%
multi_pairing/8 3.1±0.01ms 3.1±0.01ms +0.0%
pairing 937.8±28.76µs 933.4±11.00µs -0.5%
pedersen_commit/1024 11.9±0.07ms 12.0±0.08ms +0.7%
pedersen_commit/16 589.6±8.88µs 590.8±9.03µs +0.2%
pedersen_commit/256 4.0±0.02ms 4.1±0.06ms +0.6%
pedersen_commit/4 292.2±2.84µs 293.2±8.64µs +0.4%
pedersen_commit/64 1.5±0.03ms 1.6±0.02ms +0.7%

Add grind / check_grind: a standalone proof-of-work step on the transcript
sponge that protects the challenges drawn after it, up to the next prover
message. The nonce is canonical LEB128 below 2^(bits+7), one byte per prover
message; the check squeezes ceil(bits/8) bytes and needs the low bits zero.
Difficulty is at most 25 bits. MessageSearch probes byte by byte so sponges
that absorb a length per call (Poseidon) replay the real transcript. Failed
checks poison the verifier. CI lints and tests the sequential search.
…ected lengths

PoseidonSponge squeezes the bytes of a BN254 scalar, so its output is far
from uniform and the codecs' TV bound fails there. Restrict field_challenges
to sponges with uniform output (sealed UniformBytes: Keccak, Blake2b512).
The verifier now also poisons on a rejected length.
…enge-codecs

# Conflicts:
#	.github/workflows/rust.yml
…m comments

- ChallengeCodec, UniformBytes and MAX_CHALLENGE_BYTES stay public because
  they bound the public field_challenges methods; their docs now state that
  contract and name Akita's transcript as the consumer.
- Remove the doc block on the crate-private message_bytes helper.
- Remove the test-file heading and a comment that restated its assertions.
- Correct the CI comment: default features already cover BN254 and every
  sponge; the all-features run adds the Solinas and binary codecs.
…ge-pow

# Conflicts:
#	.github/workflows/rust.yml
Base automatically changed from transcript/j2-challenge-codecs to main October 23, 2026 22:05

@0xandoroid-ai-agent 0xandoroid-ai-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One finding, inline: round_trips duplicates existing coverage. Nothing else needs to change at 8b49d91.

Comment thread crates/jolt-transcript/tests/pow_tests.rs Outdated

@0xandoroid-ai-agent 0xandoroid-ai-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One test-redundancy finding remains at 9b9cf9b: remove known_answers, whose cases are already covered by the integer-oracle test. The earlier round_trips finding is addressed. No production or soundness defect was found.

Comment thread crates/jolt-transcript/src/grind_bits.rs Outdated

@0xandoroid-ai-agent 0xandoroid-ai-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No code findings remain at f5a2197: round_trips and known_answers are gone, and both threads are addressed.

This can't land yet because Prove x86-64 baseline and BMI2 arithmetic (Fp64 formal verification) did not pass at this head. GitHub cancelled it at its 45-minute limit before the HOL Light proofs ran: https://github.com/a16z/jolt/actions/runs/40039630199/job/120707813828. In Install proof dependencies, apt-get install fetched 153 MB in 40 min 32 s (63.0 kB/s); when this job passed at 9b9cf9b, the same step fetched it in 3 s. A passing rerun at this head is needed before merge.

@0xandoroid-ai-agent 0xandoroid-ai-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prove x86-64 baseline and BMI2 arithmetic passed on rerun at f5a2197 (https://github.com/a16z/jolt/actions/runs/40039630199/job/120724532377), and all 67 checks are green. Its timeout was my previous review's only finding, so nothing remains to change at this head.

@0xAndoroid
0xAndoroid merged commit f3265b5 into main Oct 24, 2026
72 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants