feat(mega-evme): batch replay and verification tooling - #366
Open
RealiCZ wants to merge 123 commits into
Open
Conversation
Canonical flag is now --rpc.cu-per-sec; --rpc.rate-limit remains a visible alias. Clarify that the value is a CU/s budget, not RPS, and warn once at provider build when retries are on and the budget is <100.
Delete --rpc.cache-size and introduce --rpc.cache-max-entries (default 0 = never evict) so verification workloads keep large RPC caches. The cache layer is always installed; 0 maps to u32::MAX capacity. Capture/replay conflict lists and docs updated.
Alloy SharedCache preallocates its LRU hash table to full capacity, so mapping "unlimited" to u32::MAX caused multi-GB RSS on every default online run. Cap at 2^20 entries (cheap preallocation, covers far more than any observed corpus) and add a construction-reality unit test.
Replay many transactions in a single process: build one provider and one RPC cache, group targets by their containing block, and execute each block once while recording every target's result. Batch mode emits NDJSON with --json (one line per target, result or error entry), exits non-zero when a target hit an infrastructure failure, and rejects the flags that only have single-transaction semantics (fixture dump, overrides, forced spec, trace, state dump). The single-transaction path is unchanged; its output stays byte-identical. Command dispatch now maps errors instead of propagating with ?, so the error handler runs and diagnostics stay off stdout.
Share --rpc.cache-dir across processes via exclusive sidecar lock plus re-read-merge on persist (provider cache and capture envelopes). Add mega-evme cache merge for offline consolidation of both shapes.
Add `replay --verify-receipt`, which fetches each replayed transaction's on-chain receipt and compares it against the receipt the replay produced: success status, gas used, and the emitted logs (count plus each log's address, topics, and data). Works in single-transaction and batch mode. The comparison lives in a new `replay/verify` module as a pure function over the consensus facts of both receipts, so it is independent of how either receipt was obtained. The verdict is reported as a `verification` object in JSON (absent without the flag) and as one verdict line in human-readable output; a mismatch fails the run through a dedicated `VerificationMismatch` error. Anything that prevents the comparison from running is reported as an infrastructure failure rather than a mismatch: a receipt the endpoint cannot serve or has pruned, a receipt describing a different inclusion than the replayed block (reorg or divergent endpoint), and pending transactions.
Add batch sedimentation of self-validating EEST fixtures so a tx list or block can be swept into <DIR>/<tx_hash>.json in one run. Per-target fidelity/BLOCKHASH/unsupported-shape gates skip with a recorded reason instead of failing the run; write failures stay infrastructure errors.
…e output
Every failure now maps onto one of four documented exit codes through a single
module: 0 success, 1 execution/internal error, 2 receipt verification mismatch,
3 RPC/transport failure. The mapping matches the error enums exhaustively, and
`main` is the only place that turns a command result into a process status.
Batch replay aggregates its per-target failures into a structured error carrying
the counts by class, so the run-level precedence (execution before rpc before
mismatch) is resolved from data instead of a formatted message.
On failure a run reports once: an `error: <message>` line on stderr, plus — with
`--json` — a compact `{"error":{"code","kind","message"}}` object as the last
stdout line, so a machine-readable run never ends with empty stdout. Per-target
NDJSON lines and all success-path output are unchanged.
…n checks
Reject conflicting non-null external_env on envelope persist (same rule as
cache merge), validate rpc-cache-{id}.json chain identity on provider merge,
classify malformed --rpc URLs as InvalidInput (exit 1), and type envelope
re-read hard vs degradable without substring matching.
Report a failed fixture dump on the target's own result line instead of replacing it, so the receipt verification still runs and its mismatch is counted; the failed dump remains an execution-class failure. Classify targets swept up by a mid-block abort as unanswered (`rpc`) with a message naming the aborting cause, reserving `not_found` for the target the endpoint actually denied, and emit them in block transaction-index order so the stream stays ascending by (block, index). Route argument-parsing failures through the structured output, report a capture-persist failure on stderr next to the run error that owns the exit code, and stop mapping a `BatchFailed` with no counts to success. Classify a block execution error by the database failure behind it, so a state read that fails mid-execution exits 3 instead of 1; the pre-block system calls and the keyless-deploy sandbox render their cause into a message before it reaches the mapping and stay execution-class.
…c JSON Carry load-time external_env through capture persist so intentional --bucket-capacity refreshes win while true concurrent conflicts hard-error; canonicalize bucket lists; always report non-aborting swept targets as rpc; emit structured JSON on panic under --json.
Stop RpcCacheStore::new_envelope from re-reading the capture file after build_capture_provider already loaded it. A concurrent writer between the two reads could make C look like the load-time baseline so an A-derived persist silently overwrote C. Carry the first-load snapshot as a parameter.
Default 30s bounds hung endpoints so they surface as retryable transport errors (exit 3 after retries) instead of hanging the process forever.
Contributor
Claude review status
✅ Review clean Last reviewed: New this round: 0 finding(s), 0 question(s) · Resolved this round: 1 · Open questions: 0 |
🧬 Mutation testing — ✅ PASSNothing to test — no mutants were generated on the changed lines. |
Contributor
|
Documentation Impact This PR adds a new top-level Agent / Skill Files
This update can be included in this PR or in a follow-up. |
🧬 Mutation testingNo results at |
RealiCZ
added
comp:mega-evme
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d8c47fd128
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Defer fixture writes until block finish succeeds; classify construction failures as fixture errors rather than skips; validate parent-block hash linkage; reject receipts with a null blockHash as infrastructure errors; stamp receipt inner logs with block/tx identity and block-global indices.
The kernel exposed one untyped hook: a call between a target's execution and its commit whose return value it carried to the harvest. That is not enough for the single-transaction path to move onto it, and the parts it was missing are the parts where a mistake is silent — an artifact published from a block that never finished, an abort blamed on the wrong transaction. Type the whole participation as `TargetLifecycle`: - The block executor is built with the driver's inspector, and the driver declares whether execution routes through it, so a plain replay keeps the EVM's non-inspected path while a tracing driver arms one. - `before_target` turns the transaction the endpoint served into the transaction that runs, and the pre-execution nonce is read for whoever signs the result. The batch driver returns it unchanged. - `on_target_executed` receives the inspector and the whole `ResultAndState` alongside the pre-commit database, and may now fail: a failing hook aborts the body exactly like a failed fetch. - A draft leaves the run inside a `PendingDraft` that only yields its value against a `CleanRun` proof the kernel mints when the walk completes. Since a block that fails to finish drops its drafts inside the kernel, holding both means the block ran to a clean finish. A driver can still read a discarded draft to report it, but cannot take it, and publishing consumes it. The abort is now attributed where it happens: the loop records the transaction it was walking, so the driver no longer re-derives the attribution from the error. The introspection fallback that did so was already unreachable — every iteration names its transaction before any fallible step — and it could name a different transaction than the one that aborted, so it is retired along with its helpers. The inspector bound is quantified over a borrow that only exists inside the kernel, which is why the forked database is moved into `State` instead of borrowed: a second reference layer under the quantifier is more than the compiler can discharge for a real inspector. Behavior is unchanged: the committed equivalence goldens are untouched.
…rnel The single-transaction path walked its own block: it fetched every preceding transaction, executed them, then ran the target and harvested its receipt — the same sequence the kernel already performs for the batch driver, written a second time and drifting from it in small ways. Move that walk onto `execute_until_targets` by implementing `TargetLifecycle` for the driver. It is the first participant that uses the whole lifecycle: the target executes as `--override.*` rewrote it, an inspector is armed for it alone, and the trace, the state diff and the fixture draft are all read between its execution and its commit, coming back as one draft the driver can only redeem against the kernel's clean-run proof. The refusal to dump a fixture for a BLOCKHASH reader is now a failing hook, which aborts the body exactly as it used to abort the run. Three points where the two walks disagreed are settled rather than left to whichever code survived: - **First log index.** The kernel counts the logs of the receipts of the committed body transactions ahead of the target; the old fold counted every receipt but the target's own. The two agree today because no receipt precedes the first transaction, and the kernel's reading is the one that stays right if that ever changes: the chain numbers logs across the receipts of the block body, so a receipt with no transaction behind it is not part of that numbering. - **BLOCKHASH isolation.** The kernel clears the access record before every transaction where the old path cleared once after the preceding ones. The only reader is a target's own count, taken after its own clear, so the rhythms are indistinguishable. Written down as an invariant on the clear. - **Target re-fetch.** The kernel asks the endpoint for each body hash, including the target the driver already resolved. The extra ask is served by the in-memory transport cache in every mode, so the wire sequence is unchanged: a mined transaction's metadata is cached under the same key the first lookup filled. Pending targets keep their own adapter. Their single block fetch fills both the fork and the environment role, and their metadata is exactly what the online cache refuses to keep, so walking them through the kernel would turn one endpoint lookup into two. Behavior is unchanged: the committed equivalence goldens are untouched, and trace, state-dump, override and fixture output — which those goldens do not cover — was diffed byte for byte against the previous binary across four target transactions and nineteen argument shapes, leaving only the measured execution time.
The equivalence baseline pinned what a replay prints and what a fixture dump writes, and nothing else. Everything the single-transaction path gained by moving onto the shared kernel — the tracers, the state dump, the overrides, and what a broken block body costs — was covered only by a one-off byte-for-byte comparison against the previous binary, which is evidence, not a gate. Fifteen rows turn it into one. Eight rows cross the four tracers with the two destinations the flag family offers, so a trace that stopped being generated, started being generated from another moment of the block, or stopped reaching the file it was pointed at, fails here. Two more do the same for the state dump. Three drive the what-if knobs: a forced older spec, which prices the same transaction at less than half its own fork's gas; all three transaction overrides at once, where the value and the input each move the outcome on their own; and a gas limit below the call's cost, the only form of that override the EVM answers with a rejection rather than an identical result. The last two break a block body — once ahead of a single target, once between two batch targets — which pins the abort contract nothing else reached: the target that had already committed keeps the result and the receipt it earned, the one behind the break is reported as the abort. A traced replay prints hundreds of kilobytes, so a row may opt into recording its oversized sections as a content digest — byte count, line count, keccak-256 — instead of the body. The gate stays as strict; what is lost is the diff, which is why each digesting trace row has a sibling that writes its payload to a file and keeps the printed summary verbatim, and why the call tracer, small enough to hold whole, digests nothing. A negative control drives the same row with and without `--trace.opcode.disable-stack`: the reported output must not move while the digest does, so a digest computed over the wrong bytes, or a threshold that quietly stopped triggering, cannot pass. No committed golden is rewritten: the twenty-two existing rows produce the same bytes they did before.
Moving the mined single-transaction path onto the shared kernel left the pending adapter holding three expressions that were already answers. Its block-global log index folded over every receipt but the target's own. The block it builds holds exactly one transaction, so the fold has always summed to zero — but it is also the reading the kernel rejected when the two walks were reconciled: a receipt produced before the block's first transaction is not part of the numbering the chain assigns, and counting its logs would shift the index off that numbering the day such a receipt exists. Stated as zero instead, with the reason. Its transaction index read the length of the preceding-hash list, which `fetch_replay_context` only fills for a mined target and therefore leaves empty here. Stated as zero, with the same reason written down. Its trace was generated from a `ResultAndState` rebuilt out of clones of the execution result and the whole post-state, when the outcome already carries that pair — the kernel borrows it directly. Borrowed here too, which drops two clones of the state from the pending path. Behavior is unchanged: the equivalence goldens, including the trace and state-dump rows added alongside, are untouched.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9936c71a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Batch target resolution read placement metadata straight out of the served answer. A pending verdict exits planning immediately and never reaches the authenticated lookup inside the execution kernel, so an endpoint answering hash H with a different (pending-looking) transaction could stamp H as definitively pending — and capture mode would persist that answer for offline reuse. Resolution now authenticates every served transaction before classification; an identity failure is an inconsistent-endpoint rpc failure, matching the other fetch sites. The abort-before-finish dump test switches to --block: as a listed target its doctored index-2 object is now refused at planning, while the block walk still reaches it after the dump target has drafted.
The flag is only consulted while a fixture directory is being written; everywhere else it parsed successfully and silently did nothing, letting an invalid automation invocation exit 0 while ignoring a requested artifact policy. It now requires --dump-fixture-dir at parse time.
…e state boundary Two review acknowledgements made explicit instead of implicit: - The per-chain cache persists mined transactions, numbered blocks, receipts, and numbered state reads as final. That is MegaETH's single-sequencer reality; on an endpoint whose recent history can still change the assumption does not hold, and --rpc.clear-cache / --rpc.no-cache-file are the outs. - --override.spec synthesizes the schedule, not history: forked parent state stays as recorded, so a contract a later fork had already deployed remains reachable under a downgrade as plain bytecode without its spec-gated interception. Deleting it would fabricate a parent state that never existed on any chain.
The `hash` an endpoint reports beside a block header is a claim about the header, not a property of it, and every guard the replay applies to a block consumes that claim: the inclusion anchor, the parent/block linkage and the body membership all compare against it. An endpoint, or a tampered offline capture, could therefore rewrite the fields the replay actually executes under -- `timestamp`, `gasLimit`, `baseFeePerGas`, `parentHash` -- keep the original `hash`, pass every guard, and have the target replayed in a world it never ran in while the run exits 0. A block header is a consensus object whose hash is a function of its own fields, so the claim can be checked without asking the endpoint anything further. `coherence::authenticate_block_header` recomputes it and rejects a mismatch as an unanswered question (exit 3), naming both hashes. The hash is recomputed from the served fields rather than read back through `Header::hash()`, which returns the value being authenticated -- the same trap the transaction-level authentication meets with the envelope's cached hash. Both drivers now fetch every block they execute against through one helper each -- `batch::fetch_block` and `cmd::fetch_resolved_block` -- so the check sits ahead of every guard that reads the reported hash. It is a local recomputation, so the RPC call sequence is unchanged. Test support follows the split the check introduces: the doctor's `set_block_*` operations now produce unauthentic headers (the vehicle for testing the authentication), and new `reseal_block` / `reseal_block_keeping_references` operations rewrite a header and recompute its hash, with and without relinking the references to the previous one. The parent-linkage and mid-block-abort tests move to the resealing vehicles so they still reach the behavior they pin, and the mock endpoints seal the block headers they serve the same way they already computed authentic transaction identities.
…vers Each case rewrites one header field a replay executes under -- `timestamp`, `gasLimit`, `baseFeePerGas`, `parentHash` -- leaves the served `hash` alone, and requires the run to reject the block naming both the hash the fields produce and the hash the endpoint reported. The single-transaction driver, the `--tx-file` batch driver and whole-block mode are each driven over the whole table, and a stripped capture holding nothing but the forged block proves the verdict is reached without a further request.
There was a problem hiding this comment.
💡 Codex Review
mega-evm/bin/mega-evme/src/common/provider/mod.rs
Lines 308 to 309 in 7ba1eac
Cross-check offline chain metadata with the captured response
Offline replay trusts the envelope's top-level chain_id without comparing it with the captured eth_chainId response, even though capture mode always records that request before persisting the envelope. If the metadata is corrupted or edited while the cached responses remain unchanged, replay silently selects the wrong hardfork schedule and exposes the wrong CHAINID value to execution, so it can emit incorrect results or false receipt mismatches. Query the replay transport for eth_chainId during construction and reject disagreement with envelope.chain_id.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Authentication covers what a recomputable hash can prove — transaction bodies and block headers. The block body listing and state reads stay trusted: rebuilding the transactions root would need the trailing bodies the walk deliberately never fetches after the last target, and state reads carry no proof. --verify-receipt is the documented cross-check for both.
`authenticate_block_header` proved a served header hashes to the hash it was served under, but nothing proved the block answering a numbered fetch is the height that was asked for: every block is fetched by number and no guard reads the height back. An endpoint, or a tampered offline capture, could therefore answer `eth_getBlockByNumber(N)` with a real, self-consistent block M. Offset the parent fetch by the same distance and the parent/block linkage holds too -- whole-block mode then executed M's body under M's environment, forked the state at N-1, stamped every receipt with N, and exited 0. The shared judgment now takes the height that was asked for and rejects a mismatch as an unanswered question (exit 3), naming both heights. The hash is checked first: the height a header claims is one of the fields the hash covers, so it is worth reading only once the header is proven to be the one the endpoint vouches for. Both drivers' block fetches -- `batch::fetch_block` and `cmd::fetch_resolved_block` -- pass the height they requested, so no call site can wire one check without the other. Both checks are local, so the RPC call sequence is unchanged. The doctor gains `answer_block_with`, which moves one captured block's response onto another height's request without rewriting a byte of it, so the moved answer still authenticates and only its height gives it away. The single-height offset is a new cross-driver row of the coherence table; the matched pair, which the linkage and membership guards cannot see, is a whole-block case of its own.
`cache merge out.json new.json -o out.json` names the output as an input too, and inputs are read before the output lock is taken -- they are shape-checked first so a doomed merge leaves no sidecar behind. The output was therefore read twice: once as a pre-lock snapshot and once, correctly, under the lock. A concurrent writer landing an updated value while the merge waited then lost it: the union's inputs-win rule let the stale snapshot beat the locked read on the shared key, and the merge exited 0 with nothing on stdout to say an entry had been rolled back. That contradicts the whole point of taking the lock, which is that neither side's entries are lost. An input that names the output is no longer read as an input. It reaches the union through the locked read alone, so `cache merge out.json new.json -o out.json` means "fold new.json into whatever out.json holds now" -- and the inputs-win rule is unchanged for the files that really are inputs. Identity is by canonical path, so a symlink or a `..` path reaching the output is caught too. Naming nothing but the output stays a success: the file is re-read under the lock and written back, and only a missing or unreadable output with no other input left is refused, since there would be nothing to write in its place. The two-process harness gains the case the bug lives in: the writer updates a key the merge already read while the merge is blocked on the lock, and the merged file must carry the writer's value. Before the fix that assertion reports the pre-lock value.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b6485930ba
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
An input that names the merge output is folded in under the output lock instead of being read before it, so a pre-lock snapshot cannot overwrite entries a concurrent writer landed while the merge waited. Sameness was decided by comparing canonical paths, which sees through symlinks and `..` but not through a hard link: one file under two real paths, each canonicalizing to itself. Compare the file system's own identity instead — `(device, inode)` on Unix — falling back to the resolved path when no file stands behind the name yet, which is the usual case for the output. Windows keeps the path comparison alone, having no stable equivalent.
Authentication recomputed each fetched transaction's hash from its encoding and compared it against the requested hash, deliberately ignoring the response's own `hash` field. But an RPC deserialization seeds the envelope's cached hash from that field, and that cached value is what every later `tx_hash()` read returns: the name a dumped fixture is filed and keyed under, and the identity a result line carries. An honest payload paired with a `hash` field naming another transaction therefore passed, and the batch dump wrote the target's execution to `<other hash>.json` — replacing an unrelated target's fixture under `--overwrite` — while the result line reported the real hash. Make the served field a third invariant of `authenticate_transaction`. Every fetched transaction is admitted through that one seam, so `tx_hash()` on an authenticated transaction is a verified value and no consumer has to recompute it.
The state dump serialized the raw post-execution EvmState without reading
the accounts' status, so a contract created and destroyed in the same
transaction was printed as a live account, complete with the code and
storage the commit is about to erase. The prestate diff tracer, reading
the same execution, already omitted it from its post side.
Such an account is now written as `{"selfdestructed": true}` and nothing
else, and a prestate load treats an entry carrying that marker as an
address absent from the file, so a dump still round-trips through
`--prestate` and reproduces the world the transaction committed.
Accounts that survive the transaction serialize byte for byte as before.
Describe how a state dump reports an address erased by SELFDESTRUCT and how a prestate load reads that entry back.
An address the run only observed as nonexistent — read, or touched without ever gaining balance, nonce, or code — was printed as an existing empty account. No such account exists on either side of the commit, and a round-tripped prestate would answer EXTCODEHASH with the empty-code hash where the chain answers zero. Such addresses are now omitted; the self-destruct marker outranks the omission, so an account created and destroyed in one transaction still reports its tombstone. The two dump fixtures lose exactly the zero-fee beneficiary ghosts.
Contributor
|
No blocking findings from this pass. 34k lines / 137 files, all under Things I verified myself rather than taking from the PR body:
Also checked the two headline fixes for red-green discrimination and the mechanisms hold up: reverting the EIP-7702 fresh-authority One design decision worth calling out as right: when Non-blocking, processThis PR accumulated four rounds of "review-fix batch" in the body (22 commits, then 11, then 4 fixes) across 137 files. At this size it has stopped being a reviewable unit. For the next comparable piece of work, four PRs — batch replay / receipt verification / concurrent-safe caching / exit-code taxonomy — would each be independently reviewable; the coupling between them is low. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns
mega-evme replayinto a self-contained equivalence-verification tool: replay whole blocks or transaction lists in one process, verify every replay against its on-chain receipt, and persist reusable fixtures and RPC caches safely under concurrency.--block <N>/--tx-file <LIST>replay many transactions in a single process (one provider, one cache, each block executed once), emitting one NDJSON entry per target; single-transaction output stays byte-identical.--verify-receiptfetches each target's on-chain receipt and compares status / gasUsed / logs, reporting a structured diff; mismatches drive a dedicated exit code.--dump-fixture-dir <DIR>bulk-dumps self-validating state-test fixtures (bench corpus format) with per-target fidelity gating.--rpc.cache-dir;cache mergeconsolidates existing per-worker caches; capture envelopes get optimistic-concurrency protection for the external-env snapshot.--rpc.cache-sizeis replaced by--rpc.cache-max-entries(0= effectively unlimited), so long verification runs stop silently evicting early entries.--rpc.rate-limitis renamed to--rpc.cu-per-sec(old name kept as a visible alias), with a warning for self-throttling values.--rpc.request-timeout(default 30 s,0disables) bounds every HTTP request, so a stalled endpoint fails fast and retries instead of hanging the process.0success /1execution or input error /2verification mismatch /3RPC failure);--jsonruns always end with a machine-readable error object on failure.All changes are confined to
bin/mega-evmeanddocs/mega-evme; no consensus code is touched.Testing
cargo test -p mega-evme) plus envelope-gated offline batch integration tests.--verify-receipt(29/29 and 25/25 receipt matches on fresh blocks), two-process concurrent cache sharing with no lost entries, and fixture sweeps self-validated through the state-test runner.-D warnings, rustfmt, cargo-sort, Prettier on docs.Follow-up fixes (2026-08-11)
Rebased onto the current
cz/chore/upgrade-revm-40tip (repository rules forbid force-pushes, so the base sync landed as merge commit62e1dc0; zero conflicts), plus three fixes:36081e7): JSON-RPC cannot express "this account was never created" (eth_getBalance/eth_getTransactionCount/eth_getCodeall answer zero), so the forked backend materialized never-created accounts as existing empty accounts. That flipped the EIP-7702 per-authorization refund condition: a brand-new authority was judged already-in-trie and each replayed authorization refunded 12,500 gas the chain did not, making replayedgasUsedunder-report the on-chain receipt. The forked state now maps the all-zero answer back toNone. This is safe because an existing-but-empty account cannot occur post-EIP-161, which every chain this tool replays has had from genesis. Tests: DB-level boundary matrix (all-zero → nonexistent; balance-only / nonce-only / code-only → still existing) plus an execution-level mock-transport test asserting a type-4 transaction with a fresh authority costs exactly 12,500 gas more than with an existing authority (verified to fail without the fix).409de9a): online batch replay (--tx-file/--block) now engages the on-disk RPC cache only when--rpc.cache-diris passed explicitly, and otherwise behaves as--rpc.no-cache-file. Rationale: a linear history scan's request keys are block-scoped and essentially never repeat across runs, so a shared cache file buys almost no hits — while the clean-exit persist re-reads, merges, and rewrites the whole file under a cross-process lock, a tail that grows linearly with the file (minutes at multi-GB sizes) and serializes concurrent batch workers into hour-long queues. Defaulting batch to no persistence makes the exit cost zero and independent of cache size, which dominates the alternative (incremental/sharded persist) that would still pay the per-process load cost and add format complexity. Single-transaction replay,run/tx --fork, and capture mode keep their previous defaults;--rpc.no-cache-file,--tx-file/--block/--verify-receiptsemantics, and the exit-code taxonomy are unchanged. Tests: a batch run against a seeded default-path cache file leaves it byte-identical and creates no cache file anywhere (so a 26k-target batch exits with no cache tail at all), an explicit--rpc.cache-dirstill persists, and single-transaction replay still persists by default.tx --rawdecoding (a77ed86):DecodedRawTxheld the real envelope yet hand-mapped every variant into aTxEnv; it now derives the transaction through the upstreamFromTxWithEncodedimpl (which also fills the deposit parts and the enveloped bytes for L1 fee calculation), so new transaction types are picked up with the dependency instead of a manual mapping. The CLI-flag path (TxArgs) has no real envelope and is unchanged. Tests: EIP-155 signed-vector decode, deposit-envelope decode, and CLI-override behavior.Review-fix batch + coherent
--override.spec(2026-08-12)22 commits (16 changes + 2 test-hardening + 4 merges), all in
bin/mega-evme/anddocs/mega-evme/; workspace suite 1846 green, clippy/fmt/prettier/cargo-sort clean.(blockNumber, blockHash)resolution space: a mined answer without an inclusion hash and the contradictory null-number-with-hash shape both fail the target asrpcinstead of being queued or misread as pending; a block-body hash that resolves to null aborts as a typed RPC inconsistency (exit 3) that still names the vanished transaction for the abort sweep.replay --override.specis now a coherent what-if: the executor receives a schedule synthesized from the forced spec (activation from the spec, per-fork parameters delegated to the chain config), so pre-block predeploys, EIP-2935/4788 gating, block-level limits, and EVM semantics all derive from one spec; without an override the behavior is byte-identical (verified by a binary-output diff probe). Replaying an old block under a newer spec deliberately installs predeploys that never existed at that height. Note: mainnet/testnet schedules currently publish only the Rex5 registry parameters, so--override.spec Rex6+ on those chains fails closed with a message naming the missing config (previously it crashed with a code-hash mismatch).cache mergelocks its output and folds the current on-disk file in under the lock (two-process serialization tests), lock-acquisition failure fails closed instead of degrading to an unlocked write,--rpc.clear-cacheunlinks and loads inside one critical section, and the provider persist classifies the on-disk shape before writing so it can no longer replace a capture envelope. Safeguard diagnostics (chain-identity, output replacement) now reach stderr at default verbosity.result: nullanswers into fixtures — offline replay reports a cache miss naming the request instead of a frozen not-found.--json | head) ends with the documented exit 1 instead of a SIGABRT; the fixturepre-map's absent-means-nonexistent shape is pinned by tests; raw-tx decoding gains signed EIP-2930/1559/7702 vectors with full field assertions.Review threads fixed by this batch are resolved; the batch-mode
--rpc.clear-cachethread stays open pending a semantics decision.Exit-taxonomy unification + batch-cache opt-in (2026-08-12, second batch)
11 commits, all in
bin/mega-evme/anddocs/mega-evme/; workspace suite 1891 green, clippy/fmt/prettier/cargo-sort clean.--rpc.clear-cacheis a disk-cache opt-in for batch replay: the documented recovery flag now engages the cache (delete under the sidecar lock, start empty, persist on exit) instead of being silently dropped; the--rpc.no-cache-filecombination's actual behavior (clear does not run) is documented as is.--verify-receipt --dump-fixture-diris counted once while both result fields are emitted; NDJSON output follows the documented(block, tx_index)order with absent targets last; a block containing none of its targets is answered without being executed.verification.error/fixture.error, per the documented keep-your-result policy; the documented jq selectors cover both failure shapes.--block 0(invalid request, exit 1) vs endpoint-reported block-0 inclusion (contradictory endpoint data, exit 3) asymmetry is now documented rationale rather than an accident.All four previously open review threads are fixed by this batch and resolved.
Served-answer authentication + classification consistency (2026-08-12, third batch)
Four fixes from the Codex review round on
8a8cad0, each with red-green regression tests:eth_getTransactionByHashconsumers (batch loop, single-path preceding loop, single-path target) recompute the hash from the served consensus encoding and re-derive the sender from the signature, refusing mismatches as an inconsistent-endpoint failure (exit 3). The response's ownhash/fromfields are never trusted — notably, alloy'strie_hash()/tx_hash()return the cached server-supplied hash for RPC-deserialized transactions, so authentication hashes the encoding explicitly.BlockNotFound(exit 1) stays reserved for user-supplied heights.Docs: the exit-code taxonomy in
overview.mdnow records the authentication and resolved-block-null conventions;state-management.mdrecords the clear-cache failure class. All four review threads from this round are fixed and resolved.Refactor: consistency infrastructure + single-format RPC cache (2026-08-14, fourth batch)
Two refactor waves, every step gated by a committed golden-equivalence baseline that was never regenerated.
A-phase (consistency infrastructure):
tests/common/doctor.rs): every envelope-tampering operation carries an exactly-one-hit contract, parsed-field locators (a trap entry pins that raw-marker matching cannot come back), and a full contract matrix.tests/replay_equivalence.rs+tests/golden/): 22 argv rows × human/json over the committed captures, full-stderr pinning, exit-2/mixed-batch/keep-result rows, negative controls that run under bless too, and a bless flow that re-derives each snapshot from a second run before accepting it.src/replay/coherence.rs): the metadata four-shape classification, genesis guard, parent-linkage, inclusion-anchor, and body-membership checks now exist once as pure predicates with typed errors; both drivers adapt them locally, and every contract message is byte-identical to before.verify::ReceiptEvidence): dump+verify on the single path fetch/authenticate/anchor the receipt once instead of twice.B-phase (single-format RPC cache):
kind: "cache"marker; capture fixtures stay byte-identical), with a typed caching policy: JSON-RPC errors, null results,eth_blockNumber, moving-block-tag requests, and pending transaction metadata are never persisted — fixing the open review thread where a cached pending answer froze--verify-receiptuntil a manual--rpc.clear-cache.--rpc.cache-max-entriesno longer balloons memory),0keeps its published 1,048,576-entry ceiling.chain_idis verified against the endpoint (hard error on mismatch), files the tool cannot identify are never deleted or overwritten, and the retired provider-array format self-heals with one warning (it stores only hashed keys under a different algorithm, so no migration is possible).CacheLayerpersistence,SharedCache, dual merge paths, filename chain-id heuristics) is deleted;cache mergeis envelope-only, checks chain identity from the body, and refuses non-envelope input with actionable guidance. Its output carries nokindmarker, so consolidating worker caches into a seedable per-chain file is retired (documented incache.md).--helptext describe the single format and the real memory behavior.Refactor: shared mined-block execution kernel (2026-08-14, fifth batch)
The single-transaction and batch replay drivers previously each carried their own copy of the block execution walk. This batch extracts one kernel and migrates both onto it, in four equivalence-gated steps:
src/replay/kernel.rs): fork parent state → authenticated per-transaction walk with in-flight abort attribution → early stop at the last target → finish and receipt harvest. Guards, fetching, entry assembly, and error adaptation stay driver-side.TargetLifecycle(inspector as a construction generic, fallible transaction-transform and post-execution hooks) and a two-phase materialization guarantee enforced by the type system — a fixture draft can only be redeemed against aCleanRuntoken the kernel mints after a clean finish, so a draft cannot be published for a block that aborted.--override.*,--dump-fixture, and--verify-receiptall plug into the kernel's lifecycle points. Zero test modifications, zero golden regenerations; the surfaces the goldens did not cover were verified by a 239-file binary diff against the pre-migration build (only the execution-time line differs), and RPC wire-request counts are unchanged. The pending path deliberately keeps its dedicated adapter: its fetch-once-fill-both-roles anti-divergence semantics are a special case, not a kernel behavior.There is now exactly one body-walk implementation in the tool. Honest accounting:
src/replaygrew 13.5% (typed API surface, the now-standalone pending adapter, and reconciliation rationale in comments — about half the growth is prose); the structural goal, not the line count, was the point, and the same-fix-twice class of drift this PR repeatedly hit during review is structurally gone for the execution core.Two deliberate, non-behavioral deltas: the human-output execution-time span now includes fork and finish, and the fixture-refusal path now finishes the block (provably no state read, no RPC, same surfaced error).
Review response: block-header authentication + planning-time target authentication (2026-08-18, sixth batch)
Three fixes and two documented acknowledgements from the Codex round on the refactor batches:
hash_slow, not the served field) and compared against the endpoint's claim at every block-retrieval site in both drivers. Previously, a tampered capture or inconsistent backend could alter timestamp/gasLimit/baseFee while keeping the claimed hash and every guard passed — verified operationally against the pre-fix binary (forged gasLimit exited 0; now exits 3 naming both hashes). Mock chains in tests are now sealed bottom-up like real blocks; the capture doctor gained field-rewrite-and-reseal operations under its contract tests.--overwriterequires--dump-fixture-dirat parse time instead of parsing successfully and silently doing nothing.--rpc.clear-cache/--rpc.no-cache-fileare the outs on endpoints where that assumption fails), and--override.specsynthesizes the schedule but never rewrites forked parent state (contracts deployed by later forks remain reachable as plain bytecode under a downgrade).All five review threads from this round are addressed and resolved.
Review response: height authentication + in-place merge concurrency (2026-08-18, seventh batch)
cache mergecan no longer roll back a concurrent writer: an input that names the output is excluded from the pre-lock read and contributes only through the locked re-read, pinned by a two-process regression test that fails against the pre-fix binary.--verify-receiptas the cross-check for both.All three review threads from this round are addressed and resolved.
Review response: hard-link aliases + served-hash-field invariant (2026-08-18, eighth batch)
(dev, ino)rather than canonical path, closing the remaining alias route by which an in-placecache mergecould read a stale pre-lock copy of the output.hashfield joins the transaction authentication invariants: an honest payload carrying a lyinghashfield previously passed authentication (which deliberately recomputes) while the batch dump path filed the fixture under the lied name — result line reportingH, artifact written asX.json,--overwriteable to replace an unrelated target's fixture (verified against the pre-fix binary). Everytx_hash()read downstream of authentication is now a verified value.Both review threads from this round are addressed and resolved.
--dump commit semantics (2026-08-25, ninth batch)
--dumpserialized the rawEvmStatewithout interpreting revm's account status flags, so it described a world the commit never produces:SELFDESTRUCT(created and erased in one transaction, EIP-6780) was printed as a live account with full code and storage, contradicting the prestate-diff tracer's post side. It is now the bare marker{"selfdestructed": true}, and--prestatereads such an entry as "this address does not exist" — so a dump stays safe to feed straight back, and the round-tripped world equals the committed one.EXTCODEHASHwith the empty-code hash where the chain answers zero. Such addresses are now omitted; the tombstone outranks the omission.A class sweep found no further members: the fixture pre-state closure already maps a missing account to absence, the prestate-diff tracer is upstream-owned and correct, storage slots have no observable zero-vs-absent distinction, and the commit paths use revm's own
commit. The tombstone predicate is the same one revm'sCacheDB::commituses, so dump and commit cannot drift apart on it.Live accounts serialize byte-identically to before; the equivalence goldens are untouched.