Repository navigation
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
482d57f to
eaa68e6
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved critical and moderate findings remain.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 2
Open (5)
What changed in this PR
This PR modernizes BaseUInt construction and related protocol, RPC, caching, allocation, and test utilities.
Changes:
- Adds safer fixed-size and validated byte conversions.
- Modernizes counting, caching, spinlocks, slab allocation, and zero comparisons.
- Updates production code and extensive test coverage.
| File | Reviewed change |
|---|---|
src/xrpld/rpc/handlers/transaction/Simulate.cpp |
Modernizes integer formatting. |
src/xrpld/rpc/handlers/ledger/LedgerEntry.cpp |
Validates raw ledger keys. |
src/xrpld/rpc/handlers/ledger/LedgerData.cpp |
Validates ledger markers. |
src/xrpld/rpc/handlers/admin/status/GetCounts.h |
Updates count threshold types. |
src/xrpld/rpc/handlers/admin/status/GetCounts.cpp |
Adds structured counter reporting. |
src/xrpld/rpc/detail/TransactionSign.cpp |
Modernizes integer formatting. |
src/xrpld/rpc/detail/RPCLedgerHelpers.cpp |
Validates ledger hashes. |
src/xrpld/overlay/detail/PeerImp.h |
Modernizes integer formatting. |
src/xrpld/core/detail/Config.cpp |
Removes account-cache configuration. |
src/xrpld/core/Config.h |
Removes the account-cache setting. |
src/xrpld/app/misc/NegativeUNLVote.cpp |
Modernizes candidate selection. |
src/xrpld/app/misc/FeeVoteImpl.cpp |
Adds bounded fee-vote conversion. |
src/xrpld/app/misc/detail/AmendmentTable.cpp |
Updates regex match conversion. |
src/xrpld/app/main/Application.cpp |
Removes cache initialization. |
src/xrpld/app/ledger/OrderBookDBImpl.cpp |
Uses explicit tagged conversions. |
src/xrpld/app/ledger/detail/LedgerReplayMsgHandler.cpp |
Validates replay hashes and keys. |
src/xrpld/app/consensus/RCLValidations.cpp |
Uses zero-value initialization. |
src/tests/libxrpl/shamap/SHAMap.cpp |
Updates SHAMap key construction. |
src/tests/libxrpl/protocol_autogen/ledger_entries/XChainOwnedCreateAccountClaimIDTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/XChainOwnedClaimIDTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/VaultTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/TicketTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/SponsorshipTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/SignerListTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/RippleStateTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/PermissionedDomainTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/PayChannelTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/OracleTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/OfferTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/NFTokenPageTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/NFTokenOfferTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/NegativeUNLTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/MPTokenTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/MPTokenIssuanceTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/LoanTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/LoanBrokerTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/LedgerHashesTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/FeeSettingsTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/EscrowTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/DirectoryNodeTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/DIDTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/DepositPreauthTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/DelegateTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/CredentialTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/CheckTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/BridgeTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/AMMTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/AmendmentsTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/protocol_autogen/ledger_entries/AccountRootTests.cpp |
Updates generated test literals. |
src/tests/libxrpl/csf/Validation.h |
Uses zero-value initialization. |
src/tests/libxrpl/csf/Peer.h |
Uses zero-value comparison. |
src/tests/libxrpl/csf/ledgers.h |
Uses zero-value initialization. |
src/tests/libxrpl/consensus/Validations.cpp |
Updates zero-value test cases. |
src/test/rpc/Subscribe_test.cpp |
Updates account test generation. |
src/test/rpc/LedgerEntry_test.cpp |
Uses compile-time hash construction. |
src/test/rpc/GetCounts_test.cpp |
Tests registry counter output. |
src/test/rpc/GetAggregatePrice_test.cpp |
Updates hash construction. |
src/test/rpc/Feature_test.cpp |
Uses zero-value initialization. |
src/test/rpc/BookChanges_test.cpp |
Uses incrementing hash keys. |
src/test/protocol/STTx_test.cpp |
Updates hash construction. |
src/test/protocol/STObject_test.cpp |
Updates vector hash construction. |
src/test/protocol/STAmount_test.cpp |
Updates issue construction. |
src/test/protocol/STAccount_test.cpp |
Uses zero-value comparison. |
src/test/protocol/SecretKey_test.cpp |
Uses compile-time digest construction. |
src/test/protocol/Quality_test.cpp |
Updates issue construction. |
src/test/protocol/Issue_test.cpp |
Updates typed construction. |
src/test/overlay/reduce_relay_test.cpp |
Uses incrementing message hashes. |
src/test/overlay/compression_test.cpp |
Uses zero-value initialization. |
src/test/jtx/utility.h |
Cleans namespace closure. |
src/test/jtx/impl/Oracle.cpp |
Modernizes integer formatting. |
src/test/jtx/impl/Env.cpp |
Modernizes integer formatting. |
src/test/consensus/NegativeUNL_test.cpp |
Updates hash and node initialization. |
src/test/app/vault/VaultValidation_test.cpp |
Updates domain hash construction. |
src/test/app/vault/VaultLifecycle_test.cpp |
Updates domain hash construction. |
src/test/app/vault/VaultDomain_test.cpp |
Updates domain hash construction. |
src/test/app/Sponsor_test.cpp |
Updates hash construction. |
src/test/app/Regression_test.cpp |
Uses the counted-object registry. |
src/test/app/RCLValidations_test.cpp |
Uses zero-value comparison. |
src/test/app/PseudoTx_test.cpp |
Updates tagged hash construction. |
src/test/app/PermissionedDomains_test.cpp |
Updates hash construction. |
src/test/app/PermissionedDEX_test.cpp |
Reuses a shared test hash. |
src/test/app/Path_test.cpp |
Uses compile-time account IDs. |
src/test/app/OfferMPT_test.cpp |
Uses compile-time identifiers. |
src/test/app/MultiSign_test.cpp |
Uses compile-time signer tags. |
src/test/app/MPToken_test.cpp |
Updates domain and MPT identifiers. |
src/test/app/lending/LoanTestBase.h |
Updates hash construction. |
src/test/app/lending/LoanLifecycle_test.cpp |
Validates parsed loan IDs. |
src/test/app/lending/LoanBroker_test.cpp |
Updates hash construction. |
src/test/app/lending/LendingHelpers_test.cpp |
Updates hash construction. |
src/test/app/LedgerReplay_test.cpp |
Updates replay hash construction. |
src/test/app/LedgerHistory_test.cpp |
Reuses a dummy transaction hash. |
src/test/app/invariants/InvariantsEscrowNFT_test.cpp |
Uses compile-time NFT IDs. |
src/test/app/HashRouter_test.cpp |
Updates hash key construction. |
src/test/app/FeeVote_test.cpp |
Updates account construction. |
src/test/app/EscrowToken_test.cpp |
Updates MPT issue construction. |
src/test/app/Delegate_test.cpp |
Adds a utility header dependency. |
src/test/app/ConfidentialTransferExtended_test.cpp |
Updates credential test data. |
src/test/app/Batch_test.cpp |
Updates tagged hash construction. |
src/test/app/AMMMPT_test.cpp |
Updates MPT identifier construction. |
src/test/app/AMM_test.cpp |
Updates currency construction. |
src/libxrpl/tx/transactors/nft/NFTokenMint.cpp |
Uses fixed byte-range construction. |
src/libxrpl/tx/transactors/lending/LoanSet.cpp |
Uses the shared zero sentinel. |
src/libxrpl/tx/transactors/dex/AMMVote.cpp |
Uses default account initialization. |
src/libxrpl/shamap/SHAMapNodeID.cpp |
Uses span-based node decoding. |
src/libxrpl/protocol/UintTypes.cpp |
Modernizes currency helpers. |
src/libxrpl/protocol/STVector256.cpp |
Uses span-based vector decoding. |
src/libxrpl/protocol/STPathSet.cpp |
Uses explicit tagged conversions. |
src/libxrpl/protocol/STIssue.cpp |
Modernizes issue serialization. |
src/libxrpl/protocol/STCurrency.cpp |
Uses explicit currency construction. |
src/libxrpl/protocol/STAmount.cpp |
Uses explicit amount-field conversions. |
src/libxrpl/protocol/Seed.cpp |
Uses fixed-size seed construction. |
src/libxrpl/protocol/PublicKey.cpp |
Uses fixed-size node-ID construction. |
src/libxrpl/protocol/Indexes.cpp |
Uses compile-time constants and spans. |
src/libxrpl/protocol/AccountID.cpp |
Reworks AccountID caching. |
src/libxrpl/nodestore/DecodedBlob.cpp |
Stores typed node keys. |
src/libxrpl/nodestore/backend/RocksDBFactory.cpp |
Uses checked status conversions. |
src/libxrpl/ledger/Ledger.cpp |
Uses hash increment idioms. |
src/libxrpl/ledger/helpers/AccountRootHelpers.cpp |
Uses fixed-size account construction. |
src/libxrpl/ledger/BookDirs.cpp |
Uses the shared zero sentinel. |
src/libxrpl/basics/CountedObject.cpp |
Removes obsolete counter implementation. |
include/xrpl/shamap/SHAMapItem.h |
Reworks slab allocation. |
include/xrpl/protocol/UintTypes.h |
Adds constexpr protocol constants. |
include/xrpl/protocol/SystemParameters.h |
Modernizes system constants. |
include/xrpl/protocol/STBitString.h |
Adds tagged bit-string constructors. |
include/xrpl/protocol/Serializer.h |
Uses fixed-size bit-string reads. |
include/xrpl/protocol/nftPageMask.h |
Uses compile-time mask construction. |
include/xrpl/protocol/nft.h |
Uses span-based issuer extraction. |
include/xrpl/protocol/KnownFormats.h |
Replaces type-name utility usage. |
include/xrpl/protocol/jss.h |
Adds the counter maximum field. |
include/xrpl/protocol/digest.h |
Uses span-based digest conversion. |
include/xrpl/protocol/AccountID.h |
Adds constexpr account helpers. |
include/xrpl/nodestore/detail/DecodedBlob.h |
Adds typed key storage and compatibility construction. |
include/xrpl/beast/utility/Zero.h |
Reworks zero comparisons. |
include/xrpl/beast/type_name.h |
Removes the obsolete type-name utility. |
include/xrpl/basics/spinlock.h |
Reworks spinlock primitives. |
include/xrpl/basics/safe_cast.h |
Adds checked cast categories. |
include/xrpl/basics/CountedObject.h |
Adds the lock-free counter registry. |
include/xrpl/basics/contract.h |
Uses Boost type demangling. |
cmake/scripts/codegen/templates/LedgerEntryTests.cpp.mako |
Updates generated hash literals. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| ret[jss::counters] = [minObjectCount] { | ||
| json::Value ctrs = json::ValueType::Object; | ||
|
|
||
| for (auto const& [k, v] : objectCounts) | ||
| { | ||
| ret[k] = v; | ||
| } | ||
| for (auto const& c : gCountedObjects) | ||
| { | ||
| if (auto const count = c.count(); count >= minObjectCount) | ||
| { | ||
| json::Value obj(json::ValueType::Object); | ||
| obj[jss::current] = count; | ||
| obj[jss::maximum] = std::max(count, c.max()); | ||
| ctrs[c.name()] = std::move(obj); |
fc9e074 to
26d6ec7
Compare
26d6ec7 to
f38c95b
Compare
Replace Beast's demangling wrapper (originally by @HowardHinnant) with calls to `boost::core::demangle`, eliminating the low-level dependency and manual memory management from the codebase. Note that `boost::core::demangle` does not reconstruct top-level cv-qualifiers and references, whereas `typeName` would. Since no caller passed those 'decorated' types, this is not an issue, and there are no functional changes to existing callers.
- Replace the lazy singleton with a constinit registry of per-type static counters. The `getInstance` method is removed. - A concurrency bug that would corrupt the list of counters during insertion has been fixed. - Debug asserts can detect incorrect accounting that can result in counter under- or overflow. - `CountedObject` can only be used as a CRTP base for its own type parameter. - Track the current and maximum values of counter. This results in a change in the output of `get_counts`, with individual counters now reported as sub-objects of a `counters` object.
- Add function-based spinlock API - Improve comments - Reduce bouncing with contested locks
The previous cache guarded a configurably-sized vector with 64
packed spinlocks. Two aspects of that setup were not worth the
cost:
- Callers were forced to perform atomic RMW operations on a
single word which was shared by all 64 locks; this caused
the lock line to bounce between cores on every lookup.
- The configurable size and runtime initialization resulted
in complexity that did not yield a meaningful improvement
in performance.
This commit ditches the 64 packed spinlocks, replacing them
with per-entry sequence locks, allowing readers on the fast
path to avoid performing any stores at all.
Cache entries are now carefully sized and aligned to fit into
a typical cache line, which helps avoid false sharing and any
unnecessary coherence traffic.
The cache is now a fixed array of 65,536 entries. The size was
chosen to balance capacity against memory overhead and to keep
the indexing operation fast and simple: since AccountID values
are uniformly distributed, two bytes can serve directly as the
index without requiring additional hashing.
Note: The reader path will perform unsynchronized reads against
concurrent writes; the sequence lock protocol detects and
discards any data observed mid-update, so these races are
benign. A sanitizer like TSAN can still flag them and the
warnings are expected. A compile-time knob to disable the
cache is available, if necessary.
- Define size classes as template parameters, validated at compile time, allowing allocators to be constinit-constructed - Move slab metadata out of the data buffer into a cache-line-aligned global pool - Replace per-block mutexes with spinlocks, and add lock-free "parking" slots so most alloc/dealloc pairs avoid the lock entirely - Release a slab's buffer when it fully drains; cache one spare buffer to avoid thrashing around slab boundaries - Make the fallback path a compile-time policy (heap or none) - Add sized deallocate() overloads that route directly to the owning size class
The SafeToCast concept transposed its signedness clause: it was written in <Dest, Src> order but it declared <Src, Dest>. This introduced what can be best described as a polarity bug. As a result: * Signed-to-wider-unsigned casts were wrongly deemed safe. * Unsigned-to-wider-signed casts were wrongly deemed unsafe. The root cause was drift caused by the safety condition being repeated in the concept and again as a static_assert in safeCast. The checking is now done only in the concept and is expressed in terms of range coverage, and not via a sizeof/signedness proxy. This change also improves the handling of bool (which previously compiled despite truncating) and eliminates the platform-dependent accept/reject behavior for same-size types. Additional fixes: * The pointer form of safeDowncast now performs the same static_cast in all builds. The dynamic_cast check is now entirely contained in the XRPL_ASSERT. * Extended integer types wider than intmax_t (e.g. __int128) are now excluded from SafeToCast instead of silently misevaluating through wrapped bounds; they remain expressible via unsafeCast. Cleanups: * Remove the single-parameter enum overload of safeCast: it had an explicitly-specified template argument binding to Src, where all sibling overload bound to Dest, silently yielding the underlying type from expressions that read as casts *to* an enum. * Add checkedCast for conversions guarded by runtime bounds checks where safety varies per instantiation. * Constrain safeDowncast to polymorphic sources and genuine public unambiguous downcasts, rejecting upcasts and unrelated types. * Route enum conversions through std::to_underlying; clarify the unsafeCast/checkedCast documentation split.
- Replace the twelve comparison operators with two `constexpr`-capable replacements: `==` and `<=>` and allow the compiler to synthesize the rest. - Constrain `==` and `<=>` using concepts and mark them as conditionally noexcept). - Remove the detail::zero_helper indirection which did not do what its comment claimed.
This commit reworks the BaseUInt class, which was originally taken from Bitcoin and has since been heavily modified. The internal limb type now widens from 32 to 64 bits when the requested width is a multiple of 64. This halves the iteration count when working on limbs; coupled with the use of add-with-carry intrinsics, the result is better code generation and optimized use of processor resources. The constructors have been reshuffled, constraining how an instance can be initialized. As a result, many common initialization errors will now fail at compile time instead of run time. A `BaseUInt` can now be initialized with: - A byte sequence of fixed size, i.e an `std::array`, a C array, or a fixed-extent `std::span` of `unsigned char` or `std::byte`. - A string literal, formatted as a hexadecimal string with no leading 0x. Malformed or improperly sized inputs are a compile-time error. - An unsigned integer. This constructor is mostly used for tests, and must execute at compile time. - A byte sequence whose length is not known at compile time, via the fromRaw() function which returns a seated `std::optional` only if the input is byte sequence is properly sized. The runtime raw-pointer and `std::uint64_t` constructors are gone, as are the container assignment operator and the the `fromVoid` and `fromVoidChecked` functions. Other changes: - The SFINAE-based container trait is replaced by the ByteCopySource and FixedByteRange concepts. - operator== and operator<=> are now hidden friends. The workaround in the old operator<=> is gone. - Efficient tag conversion is now possible. - `BaseUInt` is now `noexcept` and (almost) fully `constexpr`: every operation except byte access, hashing and text output is usable in constant evaluation. Cleanups: - Most invocations of `stringIsUInt256Sized` in PeerImp are now gone and leverage `fromRaw`.
f38c95b to
349f817
Compare
|
This looks like it should be 8 separate smaller PRs tbh. |
There's no need to use AI. Each commit is separate (and separately reviewable), but I guess I can just create 8 separate PRs if that's the preference... it'll take all of 10 seconds. The only downside is that later commits depend on earlier ones, so it will just delay the review process, since PR X and can't go until PR X+1 is in. |


High Level Overview of Change
This PR modernizes several low-level utility types and data structures, with the largest change being a rework of
BaseUInt.BaseUIntnow uses 64-bit limbs when possible, supports safer compile-time construction, removes unsafe raw-pointer and unchecked runtime constructors, and makes most operationsconstexprandnoexcept. Call sites have been updated to use fixed-size byte ranges,fromRaw()validation, explicit tag conversions, and clearer zero/increment idioms.The PR also includes related modernization and cleanup work around zero comparisons, safe casting, spinlocks, slab allocation, counted-object tracking, AccountID base58 caching, and validation of raw hash/key inputs in peer/RPC/serialization paths.
Context of Change
BaseUIntis widely used for hashes, account IDs, currencies, ledger keys, and protocol identifiers, so unsafe construction paths and loosely checked byte copies were a recurring source of fragile code. This refactor moves many errors from runtime assertions or implicit behavior to compile-time failures or explicit optional-returning validation.The new implementation improves code generation by reducing limb iteration counts for widths divisible by 64 and by using carry-aware addition helpers. It also makes the type easier to use in constant evaluation, enabling more protocol constants and sentinel values to be expressed directly as compile-time values.
Several adjacent utility changes support this work:
beast::Zeronow uses C++20 comparisons,safeCastnow correctly models value-preserving casts, spinlock/slab/counting/cache infrastructure has been simplified or made more concurrency-friendly, and peer/RPC parsing paths now prefer explicit size-checked conversion.API Impact
libxrplchange (any change that may affectlibxrplor dependents oflibxrpl)Before / After
Before,
BaseUIntsupported several unchecked or weakly checked construction paths, including raw pointer construction and runtime integer construction, and many malformed-size byte inputs were caught only by assertions or later failures.After, fixed-size byte inputs are accepted only when their extent is known and correct, dynamic-size byte inputs go through
fromRaw()and return an emptystd::optionalon size mismatch, and string/integer literal construction is compile-time checked. Internal arithmetic and comparisons are now more constexpr-friendly and, where possible, operate over fewer limbs.For performance, this is an improvement to existing functionality. The impact is broad but most visible in code paths that manipulate hashes, ledger keys, AccountIDs, SHAMap keys, serialization bit strings, and peer/RPC hash parsing. Some of the supporting changes also affect concurrent processing by reducing lock contention in allocator/cache/spinlock internals.
Test Plan
Existing unit and integration tests were updated for the stricter construction rules and new validation behavior. The added/updated coverage exercises
BaseUIntconstruction, parsing, comparison, serialization/deserialization, raw byte validation, JSON parsing of UInt fields, ledger/replay message handling, SHAMap proofs, AccountID/base58 behavior, counted-object reporting, and slab/spinlock-related users.