Skip to content

ethdebug: Add the semantic debug info sidecar for the Yul pipeline - #16954

Draft
djolertrk wants to merge 4 commits into
argotorg:developfrom
walnuthq:feature/ethdebug-impl
Draft

djolertrk wants to merge 4 commits into
argotorg:developfrom
walnuthq:feature/ethdebug-impl

Conversation

@djolertrk

Copy link
Copy Markdown
Contributor

Introduce the compiler-internal semantic debug info that the ethdebug output is produced from, following docs/internals/ethdebug_internal_debug_info.rst.

@djolertrk

Copy link
Copy Markdown
Contributor Author

cc @nikola-matic @cameel

Comment thread Changelog.md
Comment on lines +11 to +22
* Commandline Interface: Add the experimental `--ir-ethdebug` output and the assembler-mode `--ethdebug-input` option, which carry the semantic debug info of the Yul IR across a textual Yul boundary.
* Commandline Interface: Selecting an ethdebug output no longer implicitly enables the `ethdebug` and `ast-id` components of `--debug-info`; they have to be selected explicitly.
* EVM-ASM Optimizer: Improve performance of block deduplicator.
* General: Improve performance throughout the compiler using Boost's flat versions of unordered set and map.
* General: Remove support for the experimental Generic Solidity prototype (`pragma experimental solidity`).
* SMTChecker: Emit a deprecation warning for the BMC engine.
* SMTChecker: Support `block.slotnum`.
* Standard JSON Interface: Add the experimental `irEthdebug` output and `auxiliaryInput.ethdebug` input, which carry the semantic debug info of the Yul IR across a textual Yul boundary.
* Standard JSON Interface: Selecting an ethdebug output no longer implicitly enables the `ethdebug` and `ast-id` components of `settings.debug.debugInfo`; they have to be selected explicitly.
* Standard JSON Interface: Fix the entire output being replaced by a `JSONError` ("Error writing output JSON.") when an error message quotes a long source line and truncating it splits a multi-byte character.
* Yul: Add the `@ast-id-instance` debug info annotation, which distinguishes Yul scopes cloned from the same Solidity definition.
* ethdebug: Populate `ethdebug/format/info/resources` with the type documents and the pointer templates of the state variables in storage and transient storage, and publish the state variables in a program-level context (experimental).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR is fairly large, and it's clear from the changelog that there are multiple things here. Can you split it a series of stacked PRs that can be reviewed separately? For example the filling of ethdebug/format/info/resources should be pretty self-contained and independent of anything that happens in the Yul and sidecar. The change to --debug-info that makes ethdebug independent of ast-id is also something small that we can get out of the way quickly.

@djolertrk djolertrk Aug 31, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…debug outputs

Selecting an ethdebug output no longer implicitly enables the ethdebug
and ast-id components of the debug info selection, just as --ir-optimized
does not imply --optimize: requesting a program ethdebug output without
ethdebug in --debug-info or settings.debug.debugInfo is now an error, in
both the CLI and Standard JSON.

Semantic debug info transfer depends on @ast-id comments, so ethdebug now
also depends on the ast-id component: selecting ethdebug without ast-id
is an error. The ethdebug.resources and ethdebug.compilation outputs are
derived from analysis results alone and still need no selection.

The optimizer and SSA CFG gates now apply whenever ethdebug is selected,
not only when a program output is requested.
…m-level context

Populate ethdebug/format/info/resources from analysis results: every
referenced type as an ethdebug/format/type document keyed by the
compiler's type identifier, and the pointer of every state variable in
storage and transient storage as a reusable template, derived from the
storage layout (packing, mappings, dynamic arrays, bytes/string
encodings, structs). Mapping keys become template parameters the
consumer binds.

The internal model behind this is the semantic debug info side table
specified in docs/internals/ethdebug_internal_debug_info.rst: scope
records keyed by (AST ID, instance) with variables carrying identity,
declaration range, type reference, phase and an ethdebug pointer, plus
a versioned serialization of the table checked strictly against the
public pointer schema. Numeric source IDs match the ethdebug.compilation
record.

State variables whose pointer is closed, i.e. reads no template
parameter and no generated Yul local, are also published in the
program-level context of the ethdebug program outputs, with their
declaration and type reference.
…t round trip

Attach the scope records of the side table to Yul AST nodes through
DebugData, keyed by (AST ID, instance), and keep them across the
Yul -> text -> Yul reparse the compiler performs after optimization:
records are collected from the AST into the retained table before
printing and reattached by their @ast-id annotations after parsing.

Attaching re-checks every pointer against the current code: a
materialized variable (or update) whose pointer reads a generated Yul
local that no longer exists becomes optimized-out instead of keeping a
stale pointer. The check is scoped per object.

Scope instances travel in the new @ast-id-instance annotation next to
@ast-id, so passes that clone code can give each copy its own records;
code generation only produces instance 0. Contract-level scopes attach
to the top-level block of each object, which now carries the contract's
AST ID. Blocks generated by the function grouper and hoister keep
locations only, so they do not inherit another node's identity.

CompilerStack stores the attached table on the contract; it pairs with
the unoptimized IR, whose generated locals it references. EthdebugTest
exposes it as Contract.semantic for the golden tests.
…l boundary

Serialize the side table as a versioned JSON sidecar and accept it back,
so a two-stage Solidity -> Yul and Yul -> bytecode compilation produces
the same debug output as a one-stage one, and other Yul-based frontends
can supply their own semantic debug info:

- Standard JSON: the irEthdebug output emits the sidecar of a contract's
  ir output; auxiliaryInput.ethdebug attaches one to Yul input.
- CLI: --ir-ethdebug emits it; --ethdebug-input <file> attaches one in
  strict assembly mode.

Attached records flow into the ethdebug outputs of the Yul pipeline:
resources are published from the attached table, and the program-level
context of the assembled objects is derived from it, under the sidecar's
contract name.
@djolertrk
djolertrk force-pushed the feature/ethdebug-impl branch from 317bfec to 08c975a Compare August 31, 2026 12:51
@cameel cameel self-assigned this Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants