ethdebug: Emit type documents and state variable pointer templates in resources by djolertrk · Pull Request #16990 · argotorg/solidity

djolertrk · 2026-09-09T10:44:48Z

Populate the type and pointer tables of ethdebug/format/info/resources. The schema's type documents and pointers are modelled as data structures in the ethdebug schema unit, so that what the compiler emits is checked by the type system rather than assembled as JSON.

For every contract, the types of its state variables and of the parameters and return variables of the functions and modifiers compiled into it get an ethdebug/format/type document, keyed by the compiler's type identifier; composed types reference their components by that identifier. Compile-time-only types, such as literals, have no document.

Every state variable in storage and transient storage gets a pointer template derived from the storage layout: packed value types, dynamic and static arrays, the compact bytes/string encoding, structs and mappings, whose keys become the template's expected parameters.

djolertrk · 2026-09-09T10:45:20Z

cc @nikola-matic @cameel

cameel

Here's some initial feedback.

cameel · 2026-09-15T01:04:33Z

+		[&](Literal const& _literal) { _json = _literal.value; },
+		[&](Variable const& _variable)
+		{
+			requireIdentifier(_variable.identifier, "Pointer expression variable");


Having these validations here implies that one can have a Variable with an invalid identifier. When would that happen from user's perspective? How is this error meant to be reported?

This looks like it should be an assert to me, though in that case why not give Variable a constructor and enforce it there?

This applies generally to all the stuff potentially throwing EthdebugException here.

Agreed, it is internal. Moved the checks into constructors as asserts, so the serializer no longer validates anything.

cameel · 2026-09-15T01:11:17Z

+// .resources.types.t_array$_t_uint256_$dyn_storage: {"kind": "array", "contains": {"type": {"id": "t_uint256"}}}
+// .resources.types.t_array$_t_array$_t_uint256_$2_storage_$dyn_storage.contains.type.id: t_array$_t_uint256_$2_storage
+// .resources.types.t_array$_t_uint256_$2_storage.count: 0x02
+// .resources.pointers | length: 5


Why are the pointers not shown? It seems that we're losing a lot of coverage that way.

I haven't looked at it in detail yet, but my first impression is that here's a lot of new code added, but few new tests covering the output. Shouldn't there be more of it?

Yeah, good point. The tests are now in ethdebugTests/resources/, one per layout family, and show the full tables. That even caught a crash on modifier parameters and a wrong offset direction for packed values, both fixed.

cameel · 2026-09-15T01:28:49Z

+    assert kinds >= {
+        "uint", "int", "bool", "bytes", "string", "address", "contract", "enum",
+        "alias", "array", "mapping", "struct", "function",
+    }
+
+    assert types["t_uint128"] == {"kind": "uint", "bits": 128}


One thing I strongly dislike about pytest is that it takes over assert and converts it into a testing tool without providing an alternative. This way we lose the distinction between checking assumptions before test (the test itself is broken if that fails) and things we actually test (the tested code is broken).

I think it would be good to convert this file to unittest now, before we accumulate too many tests here and it becomes impractical. This is the only file in the codebase that uses pytest and we consistently use unittest everywhere else.

Converted to unittest. It runs as a script with --solc-binary-path and CI and the README are updated.

Thanks. Can you extract the part changing it to unittest into a separate PR? We could merge that already and make this PR smaller.

@cameel #17055

cameel · 2026-09-15T01:39:47Z

Like I said in #16780 (comment), I think that we should start subsequent PRs from the high-level description that can be reviewed first and later serve as docs. Ideally that's where I'd stop and hand things off to @nikola-matic.

For this PR specifically we should have these bits:

User-facing documentation:

Structure of ethdebug outputs (can just refer to ethdebug schemas, but should describe any extra structure we add to it)

Documentation of compiler internals (aimed at solc developers/contributors):

How each language type maps to the type schema.

We need to also create the place to put this docs in. For now we have that implementation plan that's only a stub.

OK, I agree.

Added docs/ethdebug.rst (the outputs and what we add on top of the schemas: type ids, template names, region names, expect) and docs/internals/ethdebug.rst (type mapping table, pointer shape per type and location). Both are in the TOC, please advise me if you'd rather keep them orphaned for now.

I think we should introduce a separate TOC tree on a new page that's not reachable from the current TOC. That's where I'd link the page you added under internals and also the implementation spec from #16780.

Also, to clarify the "Internals" section vs the new TOC, I'd classify our docs into three levels:

Basic user docs (current TOC) - aimed at language users

Avanced user docs (current TOC, Internals section) - aimed at tooling authors (including debuggers) and advanced users

Compiler developer docs (new TOC) - aimed at compiler developers

(1) and (2) should not require any knowledge about the C++ implementation, just Solidity, Yul and the compiler interface. (3) is a deep dive into the compiler with full detail.

IMO docs/ethdebug.rst fits into (2) while docs/internals/ethdebug.rst should go into (3).

Some bits of docs/internals/ethdebug.rst could actually be moved to user docs. I'll comment on that later when I'll be doing another pass through the PR.

One more thing, if at any point you're blocked waiting for the next review pass, feel free to already start docs for the subsequent implementation stages (in a new PR). I might be able to give you some early comments there too.

I think we should introduce a separate TOC tree on a new page that's not reachable from the current TOC. That's where I'd link the page you added under internals and also the implementation spec from #16780.

Also, to clarify the "Internals" section vs the new TOC, I'd classify our docs into three levels:

Basic user docs (current TOC) - aimed at language users

Avanced user docs (current TOC, Internals section) - aimed at tooling authors (including debuggers) and advanced users

Compiler developer docs (new TOC) - aimed at compiler developers

(1) and (2) should not require any knowledge about the C++ implementation, just Solidity, Yul and the compiler interface. (3) is a deep dive into the compiler with full detail.

IMO docs/ethdebug.rst fits into (2) while docs/internals/ethdebug.rst should go into (3).

Some bits of docs/internals/ethdebug.rst could actually be moved to user docs. I'll comment on that later when I'll be doing another pass through the PR.

addressed now.

One more thing, if at any point you're blocked waiting for the next review pass, feel free to already start docs for the subsequent implementation stages (in a new PR). I might be able to give you some early comments there too.

sure. I have some of the branches in our fork that I keep up to date on top of this, so I may create PRs (at least inside our fork) for them so you can check those as well.

walnuthq#4 and walnuthq#5 goes on top of this branch.

cc @cameel

djolertrk · 2026-09-18T16:29:31Z

@cameel @nikola-matic I also added walnuthq/soldb#181, it is a draft PR in the consumer of these things, that reads the data solc is about to emit.

It covers also covers some extra things (https://github.com/walnuthq/solidity/tree/feature/ethdebug-program-context) such as program context, which will come after this PR here in solidity repo.

cameel · 2026-09-22T21:48:06Z

+        "storage_16_8": {
+            "expect": [],
+            "for": {"location": "storage", "name": "total", "slot": "0x00"}
+        },
+        "storage_16_11": {
+            "expect": [],
+            "for": {
+                "group": [
+                    {"location": "storage", "name": "origin-x", "slot": "0x01", "offset": "0x1f", "length": "0x01"},
+                    {"location": "storage", "name": "origin-y", "slot": "0x01", "offset": "0x1e", "length": "0x01"}
+                ]
+            }
+        },


Does the spec mandate the expect/for wrapper even for cases without template parameters? This is the most common case so I think we should omit it if we can to make things simpler.

Yes, the template schema requires both, and the pointers table is typed as templates: https://github.com/ethdebug/format/blob/937e3484c/schemas/pointer/template.schema.yaml. It's moot after the redesign anyway, every template expects slot.

cameel · 2026-09-22T21:49:14Z

+|                   | (per contract)                            |                                 |                                           |
+-------------------+-------------------------------------------+---------------------------------+-------------------------------------------+
+
+The compilation and the resources are available as soon as the analysis of the sources succeeded.


Producing compilation schema should not even require analysis. Is that not the case?

Right, only the code required it (an assert since #16565). Relaxed to just needing the sources, and fixed the sentence. The interfaces still emit it only on success and that's a separate change.

cameel · 2026-09-22T21:55:35Z

+.. code-block:: json
+
+    {
+        "compilation": { "id": "...", "compiler": { "name": "solc", "version": "..." }, "sources": [ "..." ] },


Why is compilation here? Wasn't is supposed to be separated from resources as a part of #16460?

#16460 defines ethdebug.compilation as "the compilation key from the resources schema", and #16565 kept it inside resources too, so develop already emits it in both places. Not from this PR. The schema has it as optional in resources, so we could drop it there, but that's a separate behaviour change. We can handle it but I think it should not be part of this PR.

cameel · 2026-09-22T22:26:53Z

+    }
+
+The resources of this contract contain these pointer templates, keyed by the AST IDs of ``C`` and of the three variables.
+The two members of ``origin`` share slot 1: a region's ``offset`` counts from the most significant byte of the slot, so the ``uint8`` in the least significant byte is at offset 31 and the one before it at offset 30.


Why does it count from the most significant byte?

Looking at the segment schema, the spec is actually ambiguous about it, only referring to the "start of the slot":

If unspecified, it has the default value of 0, indicating that the segment begins at the start of the specified slot.

In solc, as documented in the storage layout spec, we count from the least significant byte:

The first item in a storage slot is stored lower-order aligned.

I assumed that ethdebug does the same. If not, we should propose changing that.

well, I saw that's how the reference implementation reads it: packages/pointers/src/read.ts slices the word from byte 0 (the MSB). It also has to be that way for a length that runs past the slot to continue into the next one. But yeah, we can discuss these changes in an issue inside ethdebug/format repo.

Can you open that issue?

Here it is ethdebug/format#307

BTW, can you add a warning about this difference?

But the ethdebug/format#309 settled it. The offsets count from the most significant byte, which is what we emit. I bumped the schemas to it and the docs now cite the segment schema.

Do you still think we should emit a warning?

To clarify, I meant a warning box in the docs, not a warning emitted by the compiler. It's fine for the spec to do things differently than Solidity, I just think we should bring attention to the fact that this difference exists.

Hmm, yes, it is worth of mentioning, so I am adding it.

cameel · 2026-09-28T22:07:53Z

+
+Composed types reference their components by identifier, so that the table is closed:
+an array, a mapping or a user-defined value type refers to its element, key, value or underlying type as ``{"type": {"id": "t_uint256"}}`` and a struct lists its members with their ``name`` and such a reference.
+Function types are the exception: their parameters and return values are described as tuples inline.


Why? Doesn't it make more sense to reuse them too?

Well, I tried it, but the function schema doesn't allow it. The parameters has to wrap the tuple itself, not a reference ($defs/Parameters in type/complex/function.schema.yaml). The tuple's components are references though. Anyway, I added that to the doc.

Maybe something to change in the spec upstream? It would be a smaller change than reworking the whole schema so I bet we could get that in pretty quickly.

Hmm, let me double check this in the latest spec. It seems that it was addressed recently, so we will need to bump the schema dependency.

Yep, bumped the pinned schemas to upstream main and addressed this.

cameel · 2026-09-28T22:13:14Z

+
+The compilation document identifies one run of the compiler: an ``id`` derived from the sources, the ``compiler`` name and version and the list of ``sources``.
+Every source carries its ``id``, ``path``, ``contents`` and ``language``.
+The source ``id`` is the index of the source unit in the compilation, the same number the ``id`` of a source in the Standard JSON output and the source mappings use.


The document could use some more cross links to other parts of the docs. For example:

"source mappings" here could be a link to https://docs.soliditylang.org/en/develop/internals/source_mappings.html

"storageLayout output" under "Type Documents" could link to https://docs.soliditylang.org/en/develop/internals/layout_in_storage.html#storage-layout

The table under "Outputs" could link to the schemas it's referencing.

yes, good point

cameel · 2026-09-28T22:22:01Z

+--------------
+
+The ``types`` table maps type identifiers to type documents (schema ``ethdebug/format/type``).
+The identifiers are the compiler's canonical type identifiers, which the ``storageLayout`` output uses as well: ``t_uint256``, ``t_address_payable``, ``t_array$_t_uint8_$dyn_storage``, ``t_mapping$_t_address_$_t_uint256_$`` or ``t_struct$_Point_$6_storage``, where the number in the identifier of a user-defined type is the ID of its definition in the AST.


As mentioned in #16990 (comment), there are some details in this doc that describe implementation details. It's fine for them to be mentioned, but only in a way that makes it clear that it's how the compiler may behave, not that it has to.

Here are the ones I noticed in this pass:

The exact structure of type names.

That the number in the name is the AST ID.

That the data location of a reference type is part of its identifier.

Source IDs

That they are a sequential index.

We don't really guarantee much about these two. Just that they are unique and one is a string and the other a number.

Yes, I agree. Reworded all of those.

cameel · 2026-09-30T16:42:56Z

+The ``pointers`` table maps type identifiers to pointer templates (schema ``ethdebug/format/pointer/template``): for every struct, array and mapping type that a state variable in storage has or is composed of, how a value of the type is laid out from a base slot.
+The keys are the same identifiers the ``types`` table uses, so the template of a type sits next to its document.


So now, after #16990 (comment), I guess the names here need to contain the location?

Hmm, I would say not yet. Templates only exist for storage, and their keys already name it, e.g. t_struct$_Point_$6_storage. Once we add templates for other locations, each location gets its own key.

cameel · 2026-09-30T16:45:00Z

+Transient storage
+    Only value types can be declared ``transient``, so transient storage adds no templates; a transient state variable is a single region with the location ``transient``.


What about immutables and constants? Do we include types and pointer templates for those as well?

No. Only state variables in storage and transient storage are covered. Constants are inlined and immutables live in the code, so there's no storage to point to. Their types only show up if something else uses them, e.g. a function parameter.
And, describing immutables in code can come later, since it doesn't belong in resources (Edit: I will add it to my "future PRs" branch already).

cameel · 2026-09-30T16:46:42Z

+- the parts of a ``bytes`` or ``string`` ``length-flag``, ``long-length`` and ``data``.
+
+A template referencing the template of a member's or element's type prefixes the names that one produces with the member's name or ``item``, so that the ``x`` of the ``from`` member of a ``Line`` is ``from-x`` and that of an element of a ``Point[]`` is ``item-x``.
+Since ethdebug identifiers must not start with ``$`` but Solidity identifiers may, a member name starting with ``$`` is prefixed with ``_``.


Maybe we should at least ping @gnidan about this one? Prefixing with _ is fine, but since the spec is still in development, it's still an option to choose a symbol that can't be a part of a Solidity identifier.

Hm, it's possible that we don't even need that restriction at all? I'd have to check... if pointer expression operations are contextually self-evident, like if there's no risk of ambiguity around what's a Solidity identifier vs. what's an ethdebug/format primitive, then we can just remove that restriction entirely.

At the very least, yeah, we can change $ to something else. (This should probably be opened as an issue on the ethdebug side... I will try not to forget to do that today)

@gnidan Thanks! The only bare strings with a meaning in expressions are $wordsize and $this. The operators are object keys. So reserving those two would be enough, and Solidity names could be used as they are. It would also fix a collision in the _ workaround: _$x is a valid Solidity identifier too, so members $x and _$x both end up as _$x. I can open the issue if you haven't yet.

Hm, so thinking about this a bit, wouldn't there be downsides also if we switch to the approach to marking reserved words? Theoretically solc might emit a template named $wordsize or $this because these are also valid Solidity identifiers, so you're going to need to handle this validity check no matter what (either check against a list of reserved words, or do what you're doing, and check against the initial $)

I don't feel strongly about this, but it seems like not a bad option to have solc just change leading $ to _$? Or you could apply some kind of prefix in all cases, but that seems bad.

Anyway, if it's annoying (which it is), what other character should we use? What about %?

~~(Making the issue on the ethdebug/format side now)~~ ethdebug/format#310

@ might be fine too.

Or, alternatively, a way to escape the $.

solc can decide on its own escape mechanism here, I feel? But I'm looking at alternatives also. I just determined that % won't work because it would require escaping in YAML. But I've found a much better contender: ~.

It's invalid in identifiers for Solidity, Yul (and inline assembly), Vyper, and Fe, but it's fine in YAML as part of an unquoted string or as an object key. (~ alone has special meaning in YAML, but that doesn't matter here).

There is one downside, which is that ~ has special meaning in RFC 6901 JSON Pointer notation, it would require escaping in any tool that uses JSON Pointers, like, if it needs to reference paths to a subtree within an ethdebug pointer... e.g. /offset/~difference would need to be escaped as /offset/~0difference. This downside seems minor compared to the gains.

I'll go update the issue accordingly.

Addressed this here.

cameel · 2026-09-30T16:49:43Z

+   ethdebug support is :ref:`experimental <experimental-mode>`.
+
+The compiler can describe its output in the `ethdebug format <https://ethdebug.github.io/format/>`_, a JSON format for debugging information shared between compilers and debuggers.
+The format defines a `schema <https://github.com/ethdebug/format/tree/main/schemas>`_ for each kind of document.


I think the document should mention which ethdebug version we are targeting. Perhaps links should also go to a specific tag rather than main.

Yes, thanks!

The helpers locating the schemas and building the schema registry move unchanged into schema_helpers.py; conftest.py keeps thin fixtures that call them. This separates them from pytest ahead of converting the tests to unittest.

The suite was the only pytest user in the repository, and pytest turns `assert` into its testing tool, so a broken assumption of a test reads the same as a failure of the code under test. The tests are now methods of a unittest TestCase using its assertions, and check the same things as before. test/ethdebugSchemaTests.py replaces conftest.py, following test/pyscriptTests.py: it discovers every test module in test/ethdebugSchemaTests/, so a new one is not silently skipped, and gives each test case a configuration with the compiler path instead of reading it from a global.

… the resources Populate the type and pointer tables of ethdebug/format/info/resources. The schema's type documents and pointers are modelled as data structures in the ethdebug schema unit, so that what the compiler emits is checked by the type system rather than assembled as JSON. For every contract, the types of its state variables and of the parameters and return variables of the functions and modifiers compiled into it get an ethdebug/format/type document, keyed by the compiler's type identifier; composed types reference their components by that identifier. Compile-time-only types, such as literals, have no document. Every state variable in storage and transient storage gets a pointer template derived from the storage layout: packed value types, dynamic and static arrays, the compact bytes/string encoding, structs and mappings, whose keys become the template's expected parameters.

…Pointer The `type` and `pointer` namespaces added one level to names that are in use elsewhere in the compiler, so that they had to be spelled out as `solidity::evmasm::ethdebug::schema::type::Type`. The parts a type document and a pointer are made of are now nested in `schema::Type` and `schema::Pointer`, e.g. `schema::Type::Array` and `schema::Pointer::Region`.

…arts The serializer checked identifiers, widths, operand counts and the presence of sub-pointers while writing JSON, as if a malformed type or pointer could reach it from the outside. The only producer of these structures is the compiler itself, so a violation is an internal error. The parts now assert their constraints in their constructors and the serializer no longer checks anything a constructor guarantees.

The type and pointer table tests gated on `solc-at-least-0.8.38`, but `develop` now builds as `0.8.38-develop` without the tables, which only come with argotorg/solidity#16990, so the develop job ran them and they failed on empty tables. Probe for the tables the way the context tests already probe for the context: compile a contract with one `uint256` and look for its type in the resources. The tests now run with a compiler that emits the tables, whatever its version, and are skipped otherwise.

nikola-matic · 2026-10-09T08:10:39Z

+The resources of this contract contain these pointer templates.
+``total`` has a value type and needs none; the storage layout puts it at slot 0.
+The two members of ``Point`` share the base slot: a region's ``offset`` counts from the most significant byte of the slot, so the ``uint8`` in the least significant byte is at offset 31 and the one before it at offset 30.
+A debugger instantiates the ``Point`` template with ``slot`` bound to 1, where the layout puts ``origin``, and the mapping's template with ``slot`` bound to 2 and ``key`` to the address of the entry it wants to see.


I'm not sure if it's stated elsewhere, but shouldn't we mention that a key is bound to the word (32 bytes)? E.g. what happens if it's a bytes4 which we right pad, but ~wordsize left pads?

yes, added, thank you!

nikola-matic · 2026-10-09T08:21:20Z

+   * - ``bool``
+     - Bool
+     - bool
+     -


Why are some of these empty?

Hmm, but those types have no properties beyond their kind. I filled them with “none” to make that clear.

Just asking, though it was maybe a typo or something.

nikola-matic · 2026-10-09T08:23:48Z

+        "compilation": {"id": "...", "compiler": {"name": "solc", "version": "..."}, "sources": ["..."]},
+        "types": {"t_uint256": {"kind": "uint", "bits": 256}, "...": "..."},
+        "pointers": {"t_struct$_Point_$6_storage": {"expect": ["slot"], "for": {"group": ["..."]}}, "...": "..."}


You're missing "ethdebug": { ... }.

Added, thanks

nikola-matic · 2026-10-09T11:38:33Z

+//         "Blue"
+//     ]
+// }
+// .resources.types.t_function_external_nonpayable$_t_uint256_$returns$_t_bool_$: {


In the rst it says that a definition field should be present, but there's none here?

These function types are written as type names, the state variables internalFunction and externalFunction, so there’s no declaration to point to. Only the type of a declared function gets a definition. I clarified that in both docs pages.

nikola-matic · 2026-10-09T13:32:03Z

 * Standard JSON Interface: Remove the experimental `yulCFGJson` output.
 * Standard JSON Interface: Selecting an ethdebug output no longer implicitly enables the `ethdebug` and `ast-id` components of `settings.debug.debugInfo`; without `ethdebug` in the selection the `evm.bytecode.ethdebug` and `evm.deployedBytecode.ethdebug` outputs carry no semantic debug info.
 * Yul Optimizer: Improve performance of `DataFlowAnalyzer` using flat hash containers.
+* ethdebug: Populate the type and pointer tables of `ethdebug/format/info/resources` with the type documents of every type a contract's state variables, functions and modifiers reference and with a pointer template for every struct, array and mapping type of its state variables (experimental).


This needs to go above the other ethdebug entry as we require the changelog to be sorted in alphabetical order.

djolertrk · 2026-10-12T14:38:56Z

any comment on this? can we merge it? @cameel @nikola-matic

cameel · 2026-10-12T17:15:28Z

Not yet, but I'll get you more feedback soon.

cameel

Here's the next batch of comments. I only found minor stuff on this pass, though I'm not finished going through all of it so you can expect more.

cameel · 2026-10-15T05:06:26Z

The internals/ dir already contains files that are a part of the main TOC. I think we should not mix them. Let's create a compiler-guide/ directory, make compiler-developers.rst into its index.rst and move all the files it links to into that dir as well.

I'd also change the title to "Solidity Compiler Development Guide".

By the way, what happens if you add index entries in those documents? Do they end up in the main index?

Done. Moved everything to docs/compiler-guide/, with compiler-developers.rst as index.rst and the title “Solidity Compiler Development Guide”.

By the way, what happens if you add index entries in those documents? Do they end up in the main index?

Yes, they do. There’s only one genindex per build, so entries from these pages showed up next to the user-facing ones. I removed the index directives from the guide pages.

cameel · 2026-10-15T05:13:04Z

+   internals/ethdebug_internal_debug_info.rst
+   internals/ethdebug.rst


Let's add categories here. We don't have many entries now, but we'll have more with SSA CFG and I expect some for other features too. This will also better show what kind of documents we expect in this tree. I'd split it like this for now:

Feature guides

Implementation plans (or maybe just Blueprints?)

Also, since ethdebug_internal_debug_info.rst already exists on develop, you can put this new TOC and the second category into a separate small PR to be merged already.

Done. Added two categories, “Feature guides” and “Implementation plans”. I kept the longer name since it says more about what goes there, but I can switch to “Blueprints” if you prefer.

The TOC with ethdebug_internal_debug_info.rst under “Implementation plans” is in a separate PR: #17120. This PR adds “Feature guides” with the ethdebug implementation page on top of it.

cameel · 2026-10-15T05:16:02Z

+
+This page is aimed at compiler developers.
+It describes how the compiler derives the :ref:`ethdebug output <ethdebug>` from its analysis results: how each type of the language maps to the type schema and what the pointers of the state variables look like for each kind of type and location.
+The debug info that is passed from the Solidity frontend through Yul is specified in :doc:`ethdebug_internal_debug_info`.


I'd put this sentence in a note. It's fine to link to that document for now, but it's temporary and that information should eventually be added here (and/or to the ethdebug page in the main TOC).

cameel · 2026-10-15T05:25:46Z

+    uint256[3] words;
+    uint128[3] halves;
+    bytes32[2][2] grid;
+    Point[2] points;


I wanted to check if the slot for static arrays of custom types is calculated correctly, but I'm still not sure. Can you add one with a struct that takes more than one slot? I'd like to be sure that the calculation will include the size of the struct.

Hmm, ok, I added struct Wide { uint256 a; uint128 b; uint256 c; } and Wide[2] wides;. The element slot comes out as slot + index * 0x03, so the struct size is included. I guess this is enough

cameel · 2026-10-15T05:28:05Z

+
+    - for a value type narrower than a word, packed *k* to a slot from the least significant byte, the region at slot ``slot + index / k``, offset ``~wordsize - (index % k + 1) * size`` and length ``size``,
+    - for another value type the region at ``slot + index * slots``, with ``slots`` being the storage size of the element type, omitted when it is one,
+    - for a type with a template a reference to it with that slot bound, its regions prefixed with ``item``.


This bit makes it sound like the slot expression is not included for template references, but I checked the tests and it seems that it is.

Also for another value type: I guess you meant for any other value type? Though for a value type occupying the whole slot would be a better way to phrase it IMO.

OK, reworded both.

cameel · 2026-10-15T05:37:42Z

+
+A ``definition`` carries the ``name`` of the declaration, unless it is empty, and its source range as the ``location``.
+The source ``id`` of the location is the index of the source unit in ``CompilerStack::sourceIndices()``, which is also what the compilation record and the source mappings use; the offset and length are byte positions in the UTF-8 source, as ``SourceLocation`` holds them.
+Definitions without a name and a known location are omitted.


When exactly does this happen? You can have a type with no definition but I don't think you can you have a definition that's not located anywhere.

Well, I thought it is valid as in some other langs, and it is valid in DWARF, but yeah, you’re right

cameel · 2026-10-15T05:41:55Z

-	result.pointers = Json::object();
+	result.types = std::move(_types);
+	result.pointers = std::move(_pointers);
 	return result;


I was looking for the place where to_json() is invoked and then realized that apparently we have implicit JSON conversions enabled. I don't think that's a good thing - it's not a trivial operation so the fact that it's being invoked should stand out more. And there's also the possibility of these conversions being invoked unintentionally.

Apparently this can be disabled with JSON_USE_IMPLICIT_CONVERSIONS. I think we should do that on develop. Can you submit a PR?

cameel · 2026-10-15T05:44:40Z

+	std::variant<
+		UInt, Int, Bool, Bytes, String, UFixed, Fixed, Address, Contract, Enum,
+		Alias, Array, Mapping, Struct, Tuple, Function
+	> value;


This would be more readable with each type on its own line.

Same goes for enums - I'd avoid squashing them into a single line even if they only have one item. That makes them harder to recognize as enums at a glance.

And in a similar vein, I think that member initializers in constructors should always go on separate lines. They just blend with the rest of the line otherwise, especially when the constructor has arguments:

schema::Pointer::Arithmetic::Arithmetic(Operator _op, Operands _operands): op(_op), operands(std::move(_operands))

Something like this is much easier on the eyes:

schema::Pointer::Arithmetic::Arithmetic(Operator _op, Operands _operands): op(_op), operands(std::move(_operands))

cameel · 2026-10-15T05:53:42Z

+	switch (_region.location)
+	{
+	case Pointer::Location::Stack: location = "stack"; break;
+	case Pointer::Location::Storage: location = "storage"; break;
+	case Pointer::Location::Transient: location = "transient"; break;
+	case Pointer::Location::Memory: location = "memory"; break;
+	case Pointer::Location::Calldata: location = "calldata"; break;
+	case Pointer::Location::Returndata: location = "returndata"; break;
+	case Pointer::Location::Code: location = "code"; break;
+	}


We should create and expose static maps for any enums that need such conversions. They're generally useful outside of producing JSON: for debugging, listing available options in messages, etc. If you have to write it even once, it's usually already worth generalizing that way.

cameel · 2026-10-15T05:58:48Z

+	auto const definition = [&](std::optional<Type::Definition> const& _definition) {
+		if (_definition)
+			_json["definition"] = *_definition;
+	};


It's not great to have very similar names next to each other, like definition vs _definition. The name should be more descriptive, e.g. assignDefinition().

Though TBH I don't think this shortcut is even worth it. It saves only one very short line and forces you to look up the definition when reading the code.

Agreed, removed it

…ntents Add a separate table of contents for pages aimed at compiler developers, which the main one does not link, and move the specification of the ethdebug internal debug info there under the implementation plans. Its index entries are dropped, since Sphinx has one index for the whole documentation and they would show up next to the user-facing pages.

…ment guide The page describes how the compiler derives the ethdebug output and is aimed at compiler developers, so it belongs to the guide rather than to the internals of the main table of contents, as its first feature guide.

…nd version ethdebug/format 0.1.0-draft.1 adds an `ethdebug` stamp naming the schema an object conforms to and the version of the specification that defines it, which a program emitted on its own and the resources should carry. Stamp both, validate against the schemas of that release, and name it as the version the compiler targets. The command-line test runner, which replaces the ethdebug output of some tests with a placeholder, now leaves the stamps inside it alone.

ethdebug/format 0.1.0-draft.2 spells its own pointer expression terms with `~` instead of `$` (ethdebug/format#323), so that they cannot clash with source-language names, and lets identifiers start with `$` (ethdebug/format#325), which every Solidity identifier now satisfies. Emit `~sum`, `~read`, `~keccak256`, `~wordsized`, `~wordsize`, `~this` and the other terms, and publish member names as they are instead of prefixing a name that starts with `$` with `_`, which could clash with a member actually named so. Validate against the schemas of the release and name it in the stamp.

A key of value type is hashed as the word Solidity holds it in, so it has to be bound as that word: the template's zero-padding on the left gives the same word only for the types held in the low-order bytes, not for a `bytesN` or a negative signed integer. Show the `ethdebug` stamp in the example of the resources, and fill the properties of the types that have none in the type mapping.

Only the type of a declared function has a declaration to point to; a function type written as a type name, as for a variable, has none.

Every declaration has a source location, so a definition is never dropped. Put the reference to the internal debug info spec into a note, say that template references of static array elements carry the element slot too and stop calling the variants of pointers shapes.

… and enums one per line The string forms of the enums live in static maps next to them, so that readers can share them. Constructor initializers go on separate lines and the definition helper in the type serialization is inlined.

Sphinx has one index for the whole documentation, so the pages of the compiler development guide are indexed under specific subentries of ethdebug rather than as main entries.

github-actions Bot added the external contribution ⭐ label Sep 9, 2026

djolertrk mentioned this pull request Sep 9, 2026

ethdebug: Emit type documents and state variable pointers walnuthq/solidity#1

Closed

djolertrk force-pushed the feature/ethdebug-resources-schema branch 2 times, most recently from 043e2ae to 3cb412a Compare September 9, 2026 12:13

djolertrk marked this pull request as ready for review September 9, 2026 12:55

cameel added the ethdebug label Sep 9, 2026

cameel requested review from cameel and nikola-matic September 9, 2026 12:58

djolertrk force-pushed the feature/ethdebug-resources-schema branch from 3cb412a to 3e14741 Compare September 11, 2026 13:23

cameel assigned cameel and nikola-matic Sep 14, 2026

cameel reviewed Sep 15, 2026

View reviewed changes

djolertrk mentioned this pull request Sep 18, 2026

Read the ETHDebug type and pointer tables walnuthq/soldb#181

Draft

djolertrk force-pushed the feature/ethdebug-resources-schema branch from 3e14741 to 65fd625 Compare September 18, 2026 16:29

nikola-matic reviewed Sep 21, 2026

View reviewed changes

Comment thread libsolidity/interface/Ethdebug.cpp Outdated

cameel reviewed Sep 22, 2026

View reviewed changes

djolertrk force-pushed the feature/ethdebug-resources-schema branch from 65fd625 to 4891cbf Compare September 24, 2026 14:46

djolertrk mentioned this pull request Sep 25, 2026

Clarify the direction of offset in storage regions ethdebug/format#307

Closed

argotorg deleted a comment from rrsb9775858729 Sep 28, 2026

cameel reviewed Sep 28, 2026

View reviewed changes

djolertrk force-pushed the feature/ethdebug-resources-schema branch from a8cae05 to 96ed19d Compare September 30, 2026 11:36

djolertrk mentioned this pull request Sep 30, 2026

test: Convert the ethdebug schema conformity tests to unittest #17055

Open

cameel reviewed Sep 30, 2026

View reviewed changes

djolertrk added 5 commits October 1, 2026 12:54

test: Move the ethdebug schema helpers out of conftest.py

6a1e5cb

The helpers locating the schemas and building the schema registry move unchanged into schema_helpers.py; conftest.py keeps thin fixtures that call them. This separates them from pytest ahead of converting the tests to unittest.

This was referenced Oct 3, 2026

pointers: allow regions named like object properties ethdebug/format#316

Merged

format: say that a scope's variables are visible only within its in ethdebug/format#318

Merged

figtracer mentioned this pull request Oct 6, 2026

feat(debugger): read ETHDebug programs in soldb foundry-rs/foundry#17430

Draft

gnidan mentioned this pull request Oct 7, 2026

bugc: give a storage mapping's pointer a template for its entry ethdebug/format#371

Merged

nikola-matic reviewed Oct 9, 2026

View reviewed changes

djolertrk requested a review from nikola-matic October 9, 2026 10:56

nikola-matic reviewed Oct 9, 2026

View reviewed changes

figtracer mentioned this pull request Oct 12, 2026

Emit ETHDebug types and variable locations paradigmxyz/solar#1800

Open

cameel reviewed Oct 15, 2026

View reviewed changes

djolertrk added 11 commits October 15, 2026 13:40

docs: Say which function types carry a definition

76c9993

Only the type of a declared function has a declaration to point to; a function type written as a type name, as for a variable, has none.

changelog: move the item above

9a770e2

ethdebug: Convert the documents to JSON explicitly

505cbd8

ethdebug: Test a static array of a struct spanning several slots

050b2e8

This was referenced Oct 15, 2026

docs: Start a compiler development guide outside the main table of contents #17120

Open

Disable implicit conversions from JSON values #17121

Merged

djolertrk force-pushed the feature/ethdebug-resources-schema branch from e47546c to 050b2e8 Compare October 16, 2026 12:01

djolertrk requested review from cameel and nikola-matic October 19, 2026 06:50

docs: Index the compiler guide pages as ethdebug subentries

ff320a9

Sphinx has one index for the whole documentation, so the pages of the compiler development guide are indexed under specific subentries of ethdebug rather than as main entries.

		The ``pointers`` table maps type identifiers to pointer templates (schema ``ethdebug/format/pointer/template``): for every struct, array and mapping type that a state variable in storage has or is composed of, how a value of the type is laid out from a base slot.
		The keys are the same identifiers the ``types`` table uses, so the template of a type sits next to its document.

		Transient storage
		Only value types can be declared ``transient``, so transient storage adds no templates; a transient state variable is a single region with the location ``transient``.

		internals/ethdebug_internal_debug_info.rst
		internals/ethdebug.rst

Conversation

djolertrk commented Sep 9, 2026

Uh oh!

djolertrk commented Sep 9, 2026

Uh oh!

cameel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cameel Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cameel Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cameel Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

djolertrk commented Sep 18, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cameel Sep 15, 2026 •

edited

Loading

cameel Sep 15, 2026 •

edited

Loading

cameel Sep 15, 2026 •

edited

Loading

cameel Sep 30, 2026 •

edited

Loading

cameel Sep 28, 2026 •

edited

Loading