Repository navigation
Add atomic complete-inventory cleaning to daemon v2 - #888
Merged
Merged
Conversation
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2 Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2 Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2 Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ocation charges Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds opt-in daemon v2
text-blocks-v1requests that protect a complete ordered inventory atomically. A late rule declaration can protect an earlier block before any response is published.Every request fully re-cleans its inventory, and every
cache_hitis false. Bounded transactions use hash lookups and incremental manifest charges. Their staging budget reserves the live manifest and other sessions before a block can allocate new mappings. Each pass takes one manifest snapshot and clones only each block's matching token entries, so 600 one-value blocks fit the default limits. Sweep and restore matchers cache their allocation charges at construction. Sessions bind immutable projection and pipeline identities to unpredictable epochs. Sequenced clean, resync, strict restore, and close operations support exact retries. Manifest, inventory, response, and receipt changes commit together within explicit allocation limits. Legacy and v1 retain their wire contract.The protocol reference describes recovery and limits. Selective reuse is a separate change. No correctness axis is intentionally weakened.
Local gates: run before pushing
Local Rust verification candidate:
6c6f7c6bb78ca319e313bc32799f68b03ea5b3af.Named fast check passed on this clean head with
assessment.passing_and_current=true.CI-matching all-feature Clippy with warnings denied passed on this clean head.
All-feature workspace tests passed on this clean head.
Every feature-matrix command completed successfully across the initial run and one retry of its last step.
Current
cargo deny check: advisories, bans, licenses, and sources passed.The named matrix passed all stages through the all-feature workspace suite. macOS then held a test executable at
_dyld_start, beforemain, in the final warnings-denied default workspace suite. That process was terminated, and the exact remaining command passed once under the same build mutex. The original named matrix record remains interrupted; it is not claimed aspassing_and_current. No security settings changed. GitHub CI on this pushed head is pending.The four new mutation probes restore whole-snapshot token capacity, recompute sweep and restore allocation charges, and remove the JSON-spelling charge. Each mutant compiled and failed its executed named test. The 600-block test first reproduced
CacheCapacitybefore the fix. The running-charge test uses actual JSON escape bytes, verifies their recorded spelling, and compares the charge with a full recomputation after every mint. Earlier probes cover hash lookups, incremental charges, global reservation, audit atomicity, validation, and retries. Compile errors and zero-test runs receive no credit.The daemon validates and locks a prepared session commit, then commits the final pass's redaction and safety-net rows in one SQLite transaction. Session publication is infallible after the audit commit. An audit error rolls back both tables and preserves the revision, generation, manifest, block inventory, detection tally, and request sequence. Trigger tests refuse the first insert and a later insert in each table, then remove the trigger and retry the exact frame successfully without duplicate rows. A process crash between durable audit and memory publication can leave audit rows for a lost session; restart invalidates its epoch. The protocol reference documents this limit.
Capacity, convergence, and result-validation refusals write no rows. Buffered audit metadata counts against the aggregate transaction budget and refuses before an oversized redaction entry is cloned. A refused open preserves existing sessions and their restore mappings. V2 idle and LRU evictions emit the same audit metadata and tracing warning as v1.
The focused v2 suite has 31 tests. The focused session and combined daemon runs passed 76 and 49 tests respectively. The reversible-daemon integration suite passed 15 tests. The 20-turn test also re-cleans every returned block through v1 against the committed session and compares its bytes and manifest. The result-validation probe confirms the refusal test reaches
validate_resultthrough the full request path. A core test checks that dropping a prepared commit publishes nothing and a retry publishes once. Unreleased changelog entries describe the v2 operations, error variants, and routing ofversion: 2frames. The CI full-input table now includes the new protocol reference; the input audit and computed-reader checks pass after integration with main.Fixtures & PII hygiene
gaze cleanunchanged with zero entries.Benchmark evidence
No benchmark was run. The new behavior is confined to opt-in daemon v2 requests; the existing default clean path is unchanged. No benchmark or performance gain is claimed.
Default-path equality compares
mainat8e2ed551187e06d83e2556a04579e2d788da839ewith tested source commitb349adbbc2d6396d3fb59dd3cd14dc24948a2123using independently built release binaries and 1,072 hash-pinned inputs. The script and input manifest are committed asscripts/verify_clean_equality.pyandscripts/verification/clean-equality-inputs-v1.json. The two whole-file inputs are explicitly pinned as the drift corpus and README because the earlier manual review script was not retained.Result: zero differences in normalized clean text, entries, stats, leak reports, and exit codes. Both binaries succeeded on all 1,072 cases and produced 103 detections. Only emitted random session hex is normalized; owner-secret session blobs are excluded. The value-free equality report records commit IDs and the binary, input-manifest, and script hashes. The report-only commit follows the tested source commit.
DCO and commit discipline