diff --git a/Changelog.md b/Changelog.md index 37962970efad..863590ed091b 100644 --- a/Changelog.md +++ b/Changelog.md @@ -7,6 +7,7 @@ Compiler Features: * Commandline Interface: Selecting an ethdebug output no longer implicitly enables the `ethdebug` and `ast-id` components of `--debug-info`; without `ethdebug` in the selection the `--ethdebug-program` and `--ethdebug-program-runtime` outputs carry no semantic debug info. * ethdebug: Stamp the `ethdebug/format/program` and `ethdebug/format/info/resources` outputs with their schema and the version of the ethdebug/format specification they follow (experimental). * ethdebug: Populate the type and pointer tables of `ethdebug/format/info/resources` with the type documents of every type a contract's state variables, functions and modifiers reference and with a pointer template for every struct, array and mapping type of its state variables (experimental). +* ethdebug: Publish the state variables in storage and transient storage and, in the deployed program, the immutable variables in the program-level context of the `ethdebug/format/program` outputs, with their declaration, type reference and pointer (experimental). * Standard JSON Interface: Remove the experimental `yulCFGJson` output. * Standard JSON Interface: Selecting an ethdebug output no longer implicitly enables the `ethdebug` and `ast-id` components of `settings.debug.debugInfo`; without `ethdebug` in the selection the `evm.bytecode.ethdebug` and `evm.deployedBytecode.ethdebug` outputs carry no semantic debug info. * Yul Optimizer: Improve performance of `DataFlowAnalyzer` using flat hash containers. diff --git a/docs/compiler-guide/ethdebug.rst b/docs/compiler-guide/ethdebug.rst index 2d911dcdec72..7c1e4166c313 100644 --- a/docs/compiler-guide/ethdebug.rst +++ b/docs/compiler-guide/ethdebug.rst @@ -24,7 +24,7 @@ Which Types Get a Document The resources of a contract contain a document for -- the types of its state variables in storage and transient storage, +- the types of its state variables in storage and transient storage and of its immutable variables, - the types of the parameters and return variables of every function and modifier compiled into the contract: the ones it defines, the ones it inherits, the free functions of the source units it references and the internal functions of the libraries in those source units, - and, transitively, every type these types are composed of. diff --git a/docs/ethdebug.rst b/docs/ethdebug.rst index 0ca771113ba1..bd1727aaaa19 100644 --- a/docs/ethdebug.rst +++ b/docs/ethdebug.rst @@ -163,3 +163,11 @@ The Programs A program (schema ``ethdebug/format/program``) describes one bytecode: the creation bytecode or the deployed bytecode of a contract. It names the ``contract`` and the source range of its definition, states the ``environment`` the bytecode runs in, ``create`` or ``call``, and lists its ``instructions``. Every instruction carries its byte ``offset`` in the bytecode, the ``operation`` with the ``mnemonic`` of the opcode and the ``arguments`` of a push, and, where the compiler knows it, a ``context`` with the source range of the code the instruction was generated from. + +The program of a contract with state variables in storage or transient storage also carries a program-level ``context`` listing them as ``variables``. +Every such variable names its ``identifier`` and the source range of its ``declaration``, refers to its ``type`` by identifier into the type table and carries its ``pointer``: a single region for a value type, a reference to the template of its type with ``slot`` bound to the variable's slot for a struct, an array, ``bytes`` or ``string``, and the region of its base slot for a mapping, whose entries the mapping's template locates once a key is bound. + +The deployed program lists the immutable variables of the contract as well, after the variables in storage and transient storage. +The deployed code holds a copy of the value of an immutable wherever the code reads it, so its pointer is a region of the ``code`` covering the value in one of those copies. +The value takes the trailing bytes of the 32-byte copy, or the leading ones for a ``bytesN`` value, the way it is aligned on the stack. +An immutable that the code never reads has no copy and is not listed, and neither is any immutable in the creation program, which holds the values in memory until it returns the deployed code. diff --git a/libevmasm/Ethdebug.cpp b/libevmasm/Ethdebug.cpp index aaffb7728ecc..1925f809fc7b 100644 --- a/libevmasm/Ethdebug.cpp +++ b/libevmasm/Ethdebug.cpp @@ -171,7 +171,13 @@ schema::materials::Compilation materialCompilation(std::vector const& _s } // anonymous namespace -Json ethdebug::program(std::string_view _name, unsigned _sourceID, Assembly const& _assembly, LinkerObject const& _linkerObject) +Json ethdebug::program( + std::string_view _name, + unsigned _sourceID, + Assembly const& _assembly, + LinkerObject const& _linkerObject, + std::optional _programContext +) { schema::Program const result{ .ethdebug = schema::data::Stamp{"ethdebug/format/program", std::string{SpecificationVersion}}, @@ -187,7 +193,7 @@ Json ethdebug::program(std::string_view _name, unsigned _sourceID, Assembly cons } }, .environment = _assembly.isCreation() ? schema::Program::Environment::CREATE : schema::Program::Environment::CALL, - .context = std::nullopt, + .context = std::move(_programContext), .instructions = programInstructions(_assembly, _linkerObject, _sourceID) }; return Json(result); diff --git a/libevmasm/Ethdebug.h b/libevmasm/Ethdebug.h index eb67d8771206..dfa7b9793f8b 100644 --- a/libevmasm/Ethdebug.h +++ b/libevmasm/Ethdebug.h @@ -25,6 +25,7 @@ #include #include +#include namespace solidity::evmasm::ethdebug { @@ -45,7 +46,13 @@ struct Source constexpr std::string_view SpecificationVersion = "0.1.0-draft.2"; // returns ethdebug/format/program, stamped with its schema and the specification version. -Json program(std::string_view _name, unsigned _sourceID, Assembly const& _assembly, LinkerObject const& _linkerObject); +Json program( + std::string_view _name, + unsigned _sourceID, + Assembly const& _assembly, + LinkerObject const& _linkerObject, + std::optional _programContext = std::nullopt +); // returns ethdebug/format/info/resources, stamped with its schema and the specification version. Json resources( diff --git a/liblangutil/EthdebugSchema.cpp b/liblangutil/EthdebugSchema.cpp index 2e4c8d837f3f..9fe25101cc2f 100644 --- a/liblangutil/EthdebugSchema.cpp +++ b/liblangutil/EthdebugSchema.cpp @@ -595,7 +595,9 @@ void schema::program::to_json(Json& _json, Context::Variable const& _contextVari { auto const numProperties = _contextVariable.identifier.has_value() + - _contextVariable.declaration.has_value(); + _contextVariable.declaration.has_value() + + _contextVariable.type.has_value() + + _contextVariable.pointer.has_value(); solRequire(numProperties >= 1, EthdebugException, "Context variable has no properties."); if (_contextVariable.identifier) { @@ -604,6 +606,13 @@ void schema::program::to_json(Json& _json, Context::Variable const& _contextVari } if (_contextVariable.declaration) _json["declaration"] = *_contextVariable.declaration; + if (_contextVariable.type) + _json["type"] = *_contextVariable.type; + if (_contextVariable.pointer) + { + solAssert(!hasInternalExpression(*_contextVariable.pointer), "Context variable pointer contains a compiler-internal expression."); + _json["pointer"] = *_contextVariable.pointer; + } } void schema::program::to_json(Json& _json, Context const& _context) @@ -1348,3 +1357,202 @@ std::map schema::info::pointersFromJson( } return pointers; } + +namespace +{ + +/// Collects the free names of pointers, resolving template references. The +/// free names of a table template are computed once, with the template's +/// own parameters bound, and cached. +class FreeNameCollector +{ +public: + explicit FreeNameCollector(std::map const& _templates): + m_templates(_templates) + {} + + std::set collect(schema::Pointer const& _pointer) + { + std::set free; + std::set bound; + LocalTemplates localTemplates; + visit(_pointer, bound, localTemplates, free); + return free; + } + +private: + using LocalTemplates = std::map; + + /// Binds names for the lifetime of the object, restoring the previous state afterwards. + class ScopedNames + { + public: + ScopedNames(std::set& _bound, std::vector const& _names): + m_bound(_bound) + { + for (std::string const& name: _names) + if (m_bound.insert(name).second) + m_added.emplace_back(name); + } + ~ScopedNames() + { + for (std::string const& name: m_added) + m_bound.erase(name); + } + + private: + std::set& m_bound; + std::vector m_added; + }; + + void visit(schema::Pointer::Expression const& _expression, std::set const& _bound, std::set& _free) + { + using Pointer = schema::Pointer; + auto const operands = [&](Pointer::Operands const& _operands) { + for (Pointer::Expression const& operand: _operands) + visit(operand, _bound, _free); + }; + std::visit(util::GenericVisitor{ + // A Yul local is a dependency on the current Yul code by + // definition; nothing binds it. An unbound variable is one too. + [&](Pointer::YulLocal const& _yulLocal) { _free.insert(_yulLocal.name); }, + [&](Pointer::Variable const& _variable) + { + if (!_bound.count(_variable.identifier)) + _free.insert(_variable.identifier); + }, + [&](Pointer::Arithmetic const& _arithmetic) { operands(_arithmetic.operands); }, + [&](Pointer::Keccak256 const& _keccak256) { operands(_keccak256.operands); }, + [&](Pointer::Concat const& _concat) { operands(_concat.operands); }, + [&](Pointer::Resize const& _resize) { visit(*_resize.operand, _bound, _free); }, + // Literals, constants, lookups and reads: the latter two name + // regions, which live in a separate namespace. + [](auto const&) {} + }, _expression.value); + } + + void visit( + std::optional const& _expression, + std::set const& _bound, + std::set& _free + ) + { + if (_expression) + visit(*_expression, _bound, _free); + } + + void visit( + schema::Pointer const& _pointer, + std::set& _bound, + LocalTemplates& _localTemplates, + std::set& _free + ) + { + using Pointer = schema::Pointer; + auto const sub = [&](std::shared_ptr const& _sub) { + if (_sub) + visit(*_sub, _bound, _localTemplates, _free); + }; + std::visit(util::GenericVisitor{ + [&](Pointer::Region const& _region) + { + visit(_region.slot, _bound, _free); + visit(_region.offset, _bound, _free); + visit(_region.length, _bound, _free); + }, + [&](Pointer::Group const& _group) + { + for (Pointer const& member: _group.members) + visit(member, _bound, _localTemplates, _free); + }, + [&](Pointer::List const& _list) + { + visit(_list.count, _bound, _free); + ScopedNames index{_bound, {_list.each}}; + sub(_list.is); + }, + [&](Pointer::Conditional const& _conditional) + { + visit(_conditional.condition, _bound, _free); + sub(_conditional.then); + sub(_conditional.otherwise); + }, + [&](Pointer::Scope const& _scope) + { + // Definitions are ordered: each may reference the earlier ones. + std::vector definedNames; + for (auto const& [name, value]: _scope.definitions) + { + visit(value, _bound, _free); + definedNames.emplace_back(name); + } + ScopedNames defined{_bound, definedNames}; + sub(_scope.in); + }, + [&](Pointer::TemplateReference const& _reference) + { + for (std::string const& name: freeNamesOfTemplate(_reference.name, _localTemplates)) + if (!_bound.count(name)) + _free.insert(name); + }, + [&](Pointer::Templates const& _templates) + { + // Locally defined templates are visible to references inside the target. + std::vector added; + for (auto const& [name, definition]: _templates.templates) + if (_localTemplates.emplace(name, &definition).second) + added.emplace_back(name); + sub(_templates.in); + for (std::string const& name: added) + _localTemplates.erase(name); + } + }, _pointer.value); + } + + /// The free names of a template, with its own parameters bound. Local + /// templates shadow the table's; results for the table's templates are + /// cached, results for local ones are recomputed per reference. + std::set freeNamesOfTemplate(std::string const& _name, LocalTemplates& _localTemplates) + { + if (m_resolving.count(_name)) + return {}; + + schema::Pointer::Template const* definition = nullptr; + bool fromTable = false; + if (auto const local = _localTemplates.find(_name); local != _localTemplates.end()) + definition = local->second; + else if (auto const it = m_templates.find(_name); it != m_templates.end()) + { + definition = &it->second; + fromTable = true; + if (auto const cached = m_tableTemplateFreeNames.find(_name); cached != m_tableTemplateFreeNames.end()) + return cached->second; + } + if (!definition) + return {}; + + m_resolving.insert(_name); + std::set free; + std::set bound; + { + ScopedNames parameters{bound, definition->expect}; + visit(*definition->body, bound, _localTemplates, free); + } + m_resolving.erase(_name); + + if (fromTable) + m_tableTemplateFreeNames.emplace(_name, free); + return free; + } + + std::map const& m_templates; + std::set m_resolving; + std::map> m_tableTemplateFreeNames; +}; + +} + +std::set schema::freeNames(Pointer const& _pointer, std::map const& _templates) +{ + return FreeNameCollector{_templates}.collect(_pointer); +} diff --git a/liblangutil/EthdebugSchema.h b/liblangutil/EthdebugSchema.h index a010722231a7..d493c0f62b05 100644 --- a/liblangutil/EthdebugSchema.h +++ b/liblangutil/EthdebugSchema.h @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -592,8 +593,11 @@ struct Context { std::optional identifier; std::optional declaration; - // TODO: type - // TODO: pointer according to ethdebug/format/spec/pointer + /// The variable's type, usually a reference into the type resources. + std::optional type; + /// Where the value lives. Omitted for a variable whose pointer is a + /// template expecting parameters, which stays in the pointer resources. + std::optional pointer; }; std::optional code; @@ -726,6 +730,14 @@ Pointer::Template templateFromJson(Json const& _json, std::string_view _path, Po /// contains a compiler-internal expression such as YulLocal. bool hasInternalExpression(Pointer const& _pointer); +/// The names @a _pointer reads and nothing inside it binds: Yul locals and +/// variables that no scope definition, list index or template parameter +/// binds. Template references are followed through the templates blocks +/// enclosing them and through @a _templates, with the template's own +/// parameters bound; a reference to an unknown template contributes +/// nothing, and neither does a template that references itself. +std::set freeNames(Pointer const& _pointer, std::map const& _templates); + namespace info { /// The type table of a resources object, keyed by type ID. diff --git a/libsolidity/interface/CompilerStack.cpp b/libsolidity/interface/CompilerStack.cpp index 75782d4e2b58..06854360d300 100644 --- a/libsolidity/interface/CompilerStack.cpp +++ b/libsolidity/interface/CompilerStack.cpp @@ -83,6 +83,7 @@ #include +#include #include #include @@ -1212,8 +1213,28 @@ Json CompilerStack::ethdebug(Contract const& _contract, bool _runtime) const if (!assembly) return {}; - solAssert(sourceIndices().contains(_contract.contract->sourceUnitName())); - return evmasm::ethdebug::program(_contract.contract->name(), sourceIndices()[_contract.contract->sourceUnitName()], *assembly, object); + std::map const sourceIndexMap = sourceIndices(); + solAssert(sourceIndexMap.contains(_contract.contract->sourceUnitName())); + + // The code generator names an immutable after the AST ID of its declaration; + // the creation code holds no copies. + std::map immutableOffsets; + if (_runtime) + for (VariableDeclaration const* immutable: ContractType(*_contract.contract).immutableVariables()) + for (auto const& [name, codeOffsets]: object.immutableReferences | ranges::views::values) + if (name == std::to_string(immutable->id()) && !codeOffsets.empty()) + immutableOffsets[immutable->id()] = *ranges::min_element(codeOffsets); + + return evmasm::ethdebug::program( + _contract.contract->name(), + sourceIndexMap.at(_contract.contract->sourceUnitName()), + *assembly, + object, + ethdebug::programContext( + ethdebug::stateVariableScope(*_contract.contract, sourceIndexMap, immutableOffsets), + ethdebug::resources(*_contract.contract, sourceIndexMap) + ) + ); } bytes CompilerStack::cborMetadata(std::string const& _contractName, bool _forIR) const diff --git a/libsolidity/interface/Ethdebug.cpp b/libsolidity/interface/Ethdebug.cpp index 0a4391e73c88..3756a428cc2b 100644 --- a/libsolidity/interface/Ethdebug.cpp +++ b/libsolidity/interface/Ethdebug.cpp @@ -22,6 +22,8 @@ #include #include +#include + #include #include @@ -733,6 +735,42 @@ class PointerTemplateRegistry std::set m_building; }; +/// The pointer of a state variable of @a _type at @a _slot, packed at +/// @a _layoutOffset, in @a _location, as the program-level context carries +/// it: a value type is a single region; a struct, an array, `bytes` or +/// `string` references the template of its type with the slot bound; a +/// mapping is the region of its base slot, from which the mapping's template +/// locates the entries once a key is bound. +Pointer stateVariablePointer( + Type const& _type, + schema::Pointer::Location _location, + u256 const& _slot, + unsigned _layoutOffset +) +{ + if (dynamic_cast(&_type)) + return region(_location, std::nullopt, literal(_slot)); + if (PointerTemplateRegistry::needsTemplate(_type)) + { + solAssert(_location == schema::Pointer::Location::Storage, "Only value types can be transient."); + return scope({{"slot", literal(_slot)}}, Pointer{schema::Pointer::TemplateReference{_type.identifier()}}); + } + return valueRegion(_type, _location, literal(_slot), _layoutOffset, std::nullopt); +} + +/// The region of the code holding the value of an immutable variable of type +/// @a _type whose copy starts at byte @a _codeOffset. A copy is a word holding +/// the value as it is on the stack, so the value takes the leading bytes of +/// the word if its type is left-aligned and the trailing ones otherwise. +Pointer immutableRegion(Type const& _type, size_t _codeOffset) +{ + solAssert(_type.isValueType() && _type.sizeOnStack() == 1, "An immutable is a single word."); + u256 const byteLength = _type.storageBytes(); + solAssert(byteLength > 0 && byteLength <= 32); + u256 const offset = _type.leftAligned() ? u256(_codeOffset) : u256(_codeOffset) + 32 - byteLength; + return {schema::Pointer::Region{std::nullopt, schema::Pointer::Location::Code, std::nullopt, literal(offset), literal(byteLength)}}; +} + void registerCallableTypes(TypeRegistry& _types, CallableDeclaration const& _callable) { for (ASTPointer const& parameter: _callable.parameters()) @@ -755,6 +793,89 @@ void ethdebug::Resources::merge(Resources _other) pointers.insert_or_assign(name, std::move(pointerTemplate)); } +langutil::SemanticDebugScope ethdebug::stateVariableScope( + ContractDefinition const& _contract, + std::map const& _sourceIndices, + std::map const& _immutableOffsets +) +{ + TypeRegistry types{_sourceIndices}; + auto const* typeType = dynamic_cast(_contract.type()); + solAssert(typeType, "Contract TypeType expected."); + auto const* contractType = dynamic_cast(typeType->actualType()); + solAssert(contractType, "Contract type expected."); + + langutil::SemanticDebugScope scope; + auto const addStateVariables = [&](DataLocation _dataLocation, schema::Pointer::Location _location) { + for (auto const& [variable, slot, offset]: contractType->linearizedStateVariables(_dataLocation)) + { + if (variable->name().empty()) + continue; + langutil::SemanticDebugVariable record; + record.identifier = variable->name(); + record.declarationASTID = variable->id(); + record.declarationSourceRange = types.sourceRange(variable->location()); + if (variable->annotation().type && types.registerType(*variable->annotation().type)) + record.typeID = variable->annotation().type->identifier(); + record.phase = langutil::SemanticDebugVariablePhase::Materialized; + record.pointer = stateVariablePointer(*variable->annotation().type, _location, slot, offset); + scope.variableDefinitions.emplace_back(std::move(record)); + } + }; + addStateVariables(DataLocation::Storage, schema::Pointer::Location::Storage); + addStateVariables(DataLocation::Transient, schema::Pointer::Location::Transient); + + // An immutable that the code never reads has no copy in it. + for (VariableDeclaration const* variable: contractType->immutableVariables()) + if (auto const codeOffset = _immutableOffsets.find(variable->id()); codeOffset != _immutableOffsets.end()) + { + langutil::SemanticDebugVariable record; + record.identifier = variable->name(); + record.declarationASTID = variable->id(); + record.declarationSourceRange = types.sourceRange(variable->location()); + if (variable->annotation().type && types.registerType(*variable->annotation().type)) + record.typeID = variable->annotation().type->identifier(); + record.phase = langutil::SemanticDebugVariablePhase::Materialized; + record.pointer = immutableRegion(*variable->annotation().type, codeOffset->second); + scope.variableDefinitions.emplace_back(std::move(record)); + } + return scope; +} + +std::optional ethdebug::programContext( + langutil::SemanticDebugScope const& _scope, + Resources const& _resources +) +{ + std::vector variables; + for (langutil::SemanticDebugVariable const& variable: _scope.variableDefinitions) + { + if (variable.phase != langutil::SemanticDebugVariablePhase::Materialized || !variable.pointer) + continue; + // A pointer that reads a generated Yul local, or a variable that + // nothing binds, is only meaningful once the Yul-to-EVM transform has + // resolved it; the program-level context carries closed pointers. The + // templates a pointer references are in the resources, their + // parameters bound by the pointer. + if (!schema::freeNames(*variable.pointer, _resources.pointers).empty()) + continue; + + schema::program::Context::Variable contextVariable; + contextVariable.identifier = variable.identifier; + contextVariable.declaration = variable.declarationSourceRange; + if (variable.typeID) + contextVariable.type = schema::Type::Specifier{schema::Type::Reference{schema::materials::ID{*variable.typeID}}}; + contextVariable.pointer = *variable.pointer; + variables.emplace_back(std::move(contextVariable)); + } + + if (variables.empty()) + return std::nullopt; + schema::program::Context context; + context.variables = std::move(variables); + return context; +} + ethdebug::Resources ethdebug::resources(ContractDefinition const& _contract, std::map const& _sourceIndices) { TypeRegistry types{_sourceIndices}; @@ -780,6 +901,9 @@ ethdebug::Resources ethdebug::resources(ContractDefinition const& _contract, std }; addStateVariables(DataLocation::Storage); addStateVariables(DataLocation::Transient); + // Immutables are of value types and need no template. + for (VariableDeclaration const* variable: contractType->immutableVariables()) + types.registerType(*variable->annotation().type); // Inherited functions and modifiers are compiled into the most derived // contract, so every linearized base contract contributes its types. diff --git a/libsolidity/interface/Ethdebug.h b/libsolidity/interface/Ethdebug.h index 402b234f7253..186bfea53d38 100644 --- a/libsolidity/interface/Ethdebug.h +++ b/libsolidity/interface/Ethdebug.h @@ -25,8 +25,10 @@ #include #include +#include #include +#include #include namespace solidity::frontend::ethdebug @@ -56,4 +58,30 @@ struct Resources /// the ethdebug compilation record, which definition locations refer to. Resources resources(ContractDefinition const& _contract, std::map const& _sourceIndices); +/// The scope record of @a _contract's state variables in storage and +/// transient storage: each materialized at its slot, a value type as a +/// region, a mapping as the region of its base slot and any other type +/// through the template of the type in resources(), with its declaration +/// and type reference. Followed by its immutable variables that have a copy +/// in the code, each materialized as the region of the code holding the value. +/// @param _immutableOffsets the byte offset in the code of one copy of the +/// value of each immutable variable, by the AST ID of its declaration. Empty +/// for the creation code, which holds none. +langutil::SemanticDebugScope stateVariableScope( + ContractDefinition const& _contract, + std::map const& _sourceIndices, + std::map const& _immutableOffsets = {} +); + +/// The program-level context of ethdebug/format/program: every materialized +/// variable of @a _scope whose pointer is closed, i.e. reads no name that is +/// not bound within it, with its declaration, type reference and pointer. +/// The templates the pointer references are in @a _resources, with their +/// parameters bound by the pointer. +/// @returns nothing when no variable qualifies. +std::optional programContext( + langutil::SemanticDebugScope const& _scope, + Resources const& _resources +); + } diff --git a/test/ethdebugSchemaTests/input_file.json b/test/ethdebugSchemaTests/input_file.json index 9f3d55051181..19606687cff9 100644 --- a/test/ethdebugSchemaTests/input_file.json +++ b/test/ethdebugSchemaTests/input_file.json @@ -22,6 +22,7 @@ "*": [ "evm.bytecode.ethdebug", "evm.deployedBytecode.ethdebug", + "evm.deployedBytecode.immutableReferences", "ethdebug.resources", "ethdebug.compilation", "storageLayout", diff --git a/test/ethdebugSchemaTests/sources/a.sol b/test/ethdebugSchemaTests/sources/a.sol index 8f61883fd5fb..a88ba8ac7f35 100644 --- a/test/ethdebugSchemaTests/sources/a.sol +++ b/test/ethdebugSchemaTests/sources/a.sol @@ -33,6 +33,9 @@ contract A1 { function (uint256) external returns (bool) externalFunction; uint256 transient temporary; uint256 constant CONSTANT = 1; + int24 immutable shift = -5; + bytes4 immutable tag = 0xdeadbeef; + uint64 immutable unread = 3; function a(uint x) public pure { assert(x > 0); @@ -41,6 +44,10 @@ contract A1 { function b(Point memory p, uint256[] calldata xs) public pure returns (Line memory, bytes memory) { assert(p.x > 0 && xs.length > 0); } + + function c() public view returns (int24, bytes4) { + return (shift, tag); + } } contract A2 { diff --git a/test/ethdebugSchemaTests/test_ethdebug_schema_conformity.py b/test/ethdebugSchemaTests/test_ethdebug_schema_conformity.py index 833bef4ac99c..da1b87d810a0 100755 --- a/test/ethdebugSchemaTests/test_ethdebug_schema_conformity.py +++ b/test/ethdebugSchemaTests/test_ethdebug_schema_conformity.py @@ -166,6 +166,26 @@ def pointer_expression_names(expression): return (variables, regions) +def immutable_names(solc_output): + """The names of the immutable variables by the AST IDs of their declarations, which + `immutableReferences` is keyed by.""" + names = {} + + def visit(node): + if isinstance(node, dict): + if node.get("nodeType") == "VariableDeclaration" and node.get("mutability") == "immutable": + names[str(node["id"])] = node["name"] + for value in node.values(): + visit(value) + elif isinstance(node, list): + for value in node: + visit(value) + + for source in solc_output["sources"].values(): + visit(source["ast"]) + return names + + def region_names(pointer): names = set() if isinstance(pointer, dict): @@ -415,6 +435,78 @@ def test_pointer_templates_describe_the_storage_types(self): composed = kind in ("struct", "array", "mapping") or dynamic self.assertEqual(type_id in pointers, composed) + def test_program_contexts_list_the_storage_variables(self): + """The program-level context names the storage variables of the contract with their + types and a closed pointer: a region for a value type and for a mapping's base slot, + the template of the type with the slot bound for any other type.""" + self.templates = self.resources["pointers"] + layouts = {"storage": "storageLayout", "transient": "transientStorageLayout"} + for output_selection in PROGRAM_OUTPUTS: + for (source_name, contract_name, program) in ethdebug_programs(self.solc_output, output_selection): + contract_output = self.solc_output["contracts"][source_name][contract_name] + layout_variables = { + variable["label"]: (location, variable) + for (location, layout_output) in layouts.items() + for variable in contract_output[layout_output]["storage"] + } + context_variables = { + variable["identifier"]: variable + for variable in program.get("context", {}).get("variables", []) + if variable["pointer"].get("location") != "code" + } + with self.subTest(output=output_selection, contract=contract_name): + self.assertEqual(set(context_variables), set(layout_variables)) + for (label, (location, layout_variable)) in layout_variables.items(): + variable = context_variables[label] + type_id = escaped_type_id(layout_variable["type"]) + self.assertEqual(variable["type"], {"id": type_id}) + self.assertIn(type_id, self.resources["types"]) + pointer = variable["pointer"] + self.assertPointerIsClosed(pointer, set(), region_names(pointer)) + kind = self.resources["types"][type_id]["kind"] + if kind != "mapping" and type_id in self.templates: + self.assertEqual(int(pointer["define"]["slot"], 16), int(layout_variable["slot"])) + self.assertEqual(pointer["in"], {"template": type_id}) + else: + # A value type, or a mapping represented by its base slot. + self.assertEqual(pointer["location"], location) + self.assertEqual(int(pointer["slot"], 16), int(layout_variable["slot"])) + self.assertRegionCoversLayoutOffset(pointer, layout_variable["offset"]) + + def test_runtime_program_contexts_list_the_read_immutables(self): + """The context of a runtime program names the immutables the code reads, each with + a region within one of the copies of its value in the code. The creation code holds + no copies.""" + names = immutable_names(self.solc_output) + self.assertGreater(len(names), 0) + for (output_selection, environment) in PROGRAM_OUTPUTS.items(): + for (source_name, contract_name, program) in ethdebug_programs(self.solc_output, output_selection): + contract_output = self.solc_output["contracts"][source_name][contract_name] + copies = {} + if environment == "call": + copies = { + names[ast_id]: references + for (ast_id, references) in contract_output["evm"]["deployedBytecode"]["immutableReferences"].items() + if ast_id in names + } + code_variables = { + variable["identifier"]: variable + for variable in program.get("context", {}).get("variables", []) + if variable["pointer"].get("location") == "code" + } + with self.subTest(output=output_selection, contract=contract_name): + self.assertEqual(set(code_variables), set(copies)) + for (name, references) in copies.items(): + variable = code_variables[name] + self.assertIn(variable["type"]["id"], self.resources["types"]) + pointer = variable["pointer"] + self.assertEqual(set(pointer), {"location", "offset", "length"}) + (start, length) = (int(pointer["offset"], 16), int(pointer["length"], 16)) + self.assertTrue(any( + reference["start"] <= start and start + length <= reference["start"] + reference["length"] + for reference in references + )) + class ResourcesTestSourcesTest(EthdebugTestCase): """The resources of the isoltest cases under ethdebugTests/resources/.""" diff --git a/test/libsolidity/ethdebugTests/context_immutables.sol b/test/libsolidity/ethdebugTests/context_immutables.sol new file mode 100644 index 000000000000..7f0f255a0ec4 --- /dev/null +++ b/test/libsolidity/ethdebugTests/context_immutables.sol @@ -0,0 +1,27 @@ +contract Base { + address immutable owner = msg.sender; + function getOwner() public view returns (address) { return owner; } +} + +contract C is Base { + uint256 counter; + int24 immutable shift; + bytes4 immutable tag; + uint64 immutable unread; + constructor() { shift = -5; tag = 0xdeadbeef; unread = 3; } + function f() public view returns (int24, bytes4) { return (shift + shift, tag); } +} +// ---- +// C.creation.context.variables | length: 1 +// C.creation.context.variables[0].identifier: counter +// C.runtime.context.variables | length: 4 +// C.runtime.context.variables[0].identifier: counter +// C.runtime.context.variables[1].identifier: owner +// C.runtime.context.variables[1].type: {"id":"t_address"} +// C.runtime.context.variables[1].pointer: {"length":"0x14","location":"code","offset":"0x023c"} +// C.runtime.context.variables[2].identifier: shift +// C.runtime.context.variables[2].type: {"id":"t_int24"} +// C.runtime.context.variables[2].pointer: {"length":"0x03","location":"code","offset":"0x01d3"} +// C.runtime.context.variables[3].identifier: tag +// C.runtime.context.variables[3].type: {"id":"t_bytes4"} +// C.runtime.context.variables[3].pointer: {"length":"0x04","location":"code","offset":"0x01ff"} diff --git a/test/libsolidity/ethdebugTests/context_state_variables.sol b/test/libsolidity/ethdebugTests/context_state_variables.sol new file mode 100644 index 000000000000..d4a6a2107b99 --- /dev/null +++ b/test/libsolidity/ethdebugTests/context_state_variables.sol @@ -0,0 +1,50 @@ +struct Point { uint8 x; uint8 y; } + +contract C { + uint256 counter; + mapping(address => uint256) balances; + uint8 packed; + bool flag; + Point origin; + uint256 transient temporary; + uint256 constant CONSTANT = 1; + function f(uint256 argument) public { counter += argument; } +} +// ==== +// EVMVersion: >=cancun +// ---- +// C.creation.context.variables | length: 6 +// C.creation.context.variables[0].identifier: counter +// C.creation.context.variables[0].declaration.source.id: 0 +// C.creation.context.variables[0].type: {"id":"t_uint256"} +// C.creation.context.variables[0].pointer: {"location":"storage","slot":"0x00"} +// C.creation.context.variables[1].identifier: balances +// C.creation.context.variables[1].type: {"id":"t_mapping$_t_address_$_t_uint256_$"} +// C.creation.context.variables[1].pointer: {"location":"storage","slot":"0x01"} +// C.creation.context.variables[2].identifier: packed +// C.creation.context.variables[2].pointer: {"length":"0x01","location":"storage","offset":"0x1f","slot":"0x02"} +// C.creation.context.variables[3].identifier: flag +// C.creation.context.variables[3].pointer: {"length":"0x01","location":"storage","offset":"0x1e","slot":"0x02"} +// C.creation.context.variables[4].identifier: origin +// C.creation.context.variables[4].type: {"id":"t_struct$_Point_$6_storage"} +// C.creation.context.variables[4].pointer: { +// "define": { +// "slot": "0x03" +// }, +// "in": { +// "template": "t_struct$_Point_$6_storage" +// } +// } +// C.creation.context.variables[5].identifier: temporary +// C.creation.context.variables[5].pointer: {"location":"transient","slot":"0x00"} +// C.runtime.context.variables | length: 6 +// C.runtime.context.variables[0].identifier: counter +// C.runtime.context.variables[4].pointer: { +// "define": { +// "slot": "0x03" +// }, +// "in": { +// "template": "t_struct$_Point_$6_storage" +// } +// } +// C.runtime.context.variables[5].identifier: temporary diff --git a/test/libsolidity/ethdebugTests/context_without_state_variables.sol b/test/libsolidity/ethdebugTests/context_without_state_variables.sol new file mode 100644 index 000000000000..e654d2fb5a56 --- /dev/null +++ b/test/libsolidity/ethdebugTests/context_without_state_variables.sol @@ -0,0 +1,6 @@ +contract C { + function f(uint256 argument) public pure returns (uint256) { return argument; } +} +// ---- +// C.creation.context: +// C.runtime.context: