Repository navigation
ENH: Replace library version gates in bounded-memory helpers #1768
Description
Summary
Replace library-version approval as the basis for allocation accounting in bounded-memory operations.
A bounded-memory operation accounts for participating storage against an allowance before allocating it.
The request covers simplnx and the companion SimplnxOoc repository.
This is separate work from the immediate compatibility repairs in #1764. Use a separate branch.
Do not remove existing guards before verifying a replacement.
At 2eb6b878f6ff408d1454754d4c0e63edd676d7e0, the cited public BoundedRead.hpp and BoundedChunkRead.cpp helpers are absent.
Their named representation checks also do not appear in src/simplnx.
The detailed implementation observations below describe the reported #1764 branch context, rather than confirmed current develop behavior.
The original public code inventory uses compatibility snapshot 3d7d59310e236887f38d8a51508517604e934b1b.
The companion repository is not inspected here.
The report describes guards that approve specific C++ standard-library versions before bounded reads or storage preparation can proceed.
Those guards estimate allocations for vectors, strings, paths, and error results from reviewed library implementations.
A compiler or library upgrade can therefore reject an otherwise working operation until its implementation is reviewed.
The reported rules require consistent behavior across both repositories.
The original report records these compatibility results. These historical runs are not independently verified here:
- The original Linux job rejects GCC 13 because it is outside the approved list.
- The corrected Linux run passes 3,290 tests with GCC 13 and Clang 16, including all 11 earlier failures.
- The macOS capture reports Apple libc++ 210106 (21.1.6), ABI 1, C++20, and expected-lite 0.8.0.
Typed new is inactive. The reported header review finds allocations within existing bounds. Compatibility correction and native verification remain separate from this redesign. - The original Linux and macOS failures each affect 11 tests. They include diagnostic construction, bounded extent reads, and compressed-chunk reads.
An unsupported result can precede ordinary validation or input/output.
Version checks record reviewed configurations. They do not enforce allocation limits directly.
Version identifiers do not fully identify vendor patches, build options, or the linked runtime.
The reported memory contract includes working buffers and participating metadata and diagnostics.
It excludes operating-system caches and opaque library or allocator bookkeeping. It does not cap total process memory.
Ordinary containers do not guarantee every allocation bound required by these estimates.
For example, vector::reserve(n) promises sufficient capacity, rather than exactly n elements of capacity.
String growth and path storage also depend on implementation.
See the C++ vector capacity requirements; this reference is retained from the report and is not independently retrieved here.
Deleting guards or accepting all newer versions does not prove the existing guarantee.
In the reported public helpers, BoundedRead::Diagnostic constructs text in fixed storage.
Its finish() conversion to Result<bool> reintroduces ordinary string and vector allocations.
The public code inventory names DiagnosticRepresentationSupported, FreshVectorBytes, Diagnostic::finish, and terminal or empty-result accounting in BoundedRead.hpp.
It also names Error, Warning, their collections, and the public interface in Result.hpp.
The reported bounded callers include DataStore.hpp and BoundedChunkRead.cpp.
For SimplnxOoc, retain the same user-visible admission, failure, and memory-budget guarantees without exposing its implementation here.
Inventory adjacent checks, such as filename conversion, separately. Actual platform or API differences can still require platform-specific code.
Evaluate allocation-time enforcement in place of predictions about library internals. The original report proposes these directions for evaluation:
- Use fixed storage where the contract already supplies a maximum, including diagnostic text and supported array rank.
- Use budget-controlled storage for variable working data and tracking lists. Include nested allocations and old/new overlap during growth.
- Reject allocations before exceeding the allowance. Do not silently fall back to ordinary heap storage.
- Avoid unnecessary path, name, and shape copies where safe, or use controlled storage.
Borrowed data must remain valid throughout the operation and follow existing concurrency rules. Do not truncate valid filenames to fit fixed buffers. - Define result handoff so diagnostics cannot bypass the budget or retain references to a destroyed temporary pool.
C++ memory-resource facilities are candidates, not a selected design.
One candidate uses a caller-owned fixed buffer and a refusing upstream resource.
Alignment, growth, reuse, nested allocations, and lifetime still need assessment.
The original references are monotonic buffer resources and the null resource.
They are not independently retrieved here.
The following design decisions remain open:
- Preserve the public
Resultinterface while removing dependence on library internals for bounded return and error storage.
Evaluate fixed internal results, caller-owned output, or allocator-aware representation. Map callers before selecting an approach. - Define the accounting boundary explicitly. Preserve the existing guarantee by default.
Moving result construction or bookkeeping outside that boundary changes the contract and requires explicit approval. - Match storage mechanisms to lifetimes. A pool that does not reclaim individual blocks can waste memory for long-lived tracking data.
- Determine whether public API or binary-interface changes are necessary. Plan migration without changing valid existing behavior.
This request does not replace every project container. Establish the helper and caller scope before estimating implementation work.
Acceptance requires all of the following:
- Preserve an explicit memory contract. Enforce admission before allocations, including nested allocations and storage overlap during growth.
- Remove expanding version lists for replaced allocation assumptions. Explain every remaining platform-specific check.
- Preserve error, warning, unavailable, cancellation, and allocation-failure behavior.
Failed admission must preserve destination data wherever the existing interface promises that behavior. - Preserve diagnostic and metadata ownership after return, including moves, exceptions, and rollback.
- Test tiny allowances, alignment, growth, long names, maximum rank, overflow, allocation failure, and cleanup.
Tests must verify the contract rather than mirror version lists. - Pass in-memory and actual out-of-core tests on supported Windows, Linux, and macOS configurations.
Keep native continuous integration assertions. Do not skip failures on new toolchains. - Measure runtime, allocation count, and peak participating memory in ordinary and constrained cases. Do not assume a pool is faster.
- Review CPU cost, memory, C++ correctness, design, defensive behavior, maintainability, and incorrect assumptions before replacing guards.
Capacity checks after allocation and startup probes can provide evidence. Neither proves every input respects a strict limit before allocation.
Do not silently substitute them for the agreed guarantee.
Keep immediate compatibility repairs until the separate replacement is implemented and verified.
The broad unit-test audit, historical revalidation, KMeans sampling, and unrelated cache redesign remain outside this request.
Activity