Skip to content

Add atomic complete-inventory cleaning to daemon v2 - #888

Merged
Naoray merged 37 commits into
mainfrom
feat/daemon-text-blocks-v2
Oct 26, 2026
Merged

Naoray merged 37 commits into
mainfrom
feat/daemon-text-blocks-v2

Conversation

@Naoray

@Naoray Naoray commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

What & why

Adds opt-in daemon v2 text-blocks-v1 requests that protect a complete ordered inventory atomically. A late rule declaration can protect an earlier block before any response is published.

Every request fully re-cleans its inventory, and every cache_hit is false. Bounded transactions use hash lookups and incremental manifest charges. Their staging budget reserves the live manifest and other sessions before a block can allocate new mappings. Each pass takes one manifest snapshot and clones only each block's matching token entries, so 600 one-value blocks fit the default limits. Sweep and restore matchers cache their allocation charges at construction. Sessions bind immutable projection and pipeline identities to unpredictable epochs. Sequenced clean, resync, strict restore, and close operations support exact retries. Manifest, inventory, response, and receipt changes commit together within explicit allocation limits. Legacy and v1 retain their wire contract.

The protocol reference describes recovery and limits. Selective reuse is a separate change. No correctness axis is intentionally weakened.

Local gates: run before pushing

Local Rust verification candidate: 6c6f7c6bb78ca319e313bc32799f68b03ea5b3af.

  • Named fast check passed on this clean head with assessment.passing_and_current=true.

  • CI-matching all-feature Clippy with warnings denied passed on this clean head.

  • All-feature workspace tests passed on this clean head.

  • Every feature-matrix command completed successfully across the initial run and one retry of its last step.

  • Current cargo deny check: advisories, bans, licenses, and sources passed.

The named matrix passed all stages through the all-feature workspace suite. macOS then held a test executable at _dyld_start, before main, in the final warnings-denied default workspace suite. That process was terminated, and the exact remaining command passed once under the same build mutex. The original named matrix record remains interrupted; it is not claimed as passing_and_current. No security settings changed. GitHub CI on this pushed head is pending.

The four new mutation probes restore whole-snapshot token capacity, recompute sweep and restore allocation charges, and remove the JSON-spelling charge. Each mutant compiled and failed its executed named test. The 600-block test first reproduced CacheCapacity before the fix. The running-charge test uses actual JSON escape bytes, verifies their recorded spelling, and compares the charge with a full recomputation after every mint. Earlier probes cover hash lookups, incremental charges, global reservation, audit atomicity, validation, and retries. Compile errors and zero-test runs receive no credit.

The daemon validates and locks a prepared session commit, then commits the final pass's redaction and safety-net rows in one SQLite transaction. Session publication is infallible after the audit commit. An audit error rolls back both tables and preserves the revision, generation, manifest, block inventory, detection tally, and request sequence. Trigger tests refuse the first insert and a later insert in each table, then remove the trigger and retry the exact frame successfully without duplicate rows. A process crash between durable audit and memory publication can leave audit rows for a lost session; restart invalidates its epoch. The protocol reference documents this limit.

Capacity, convergence, and result-validation refusals write no rows. Buffered audit metadata counts against the aggregate transaction budget and refuses before an oversized redaction entry is cloned. A refused open preserves existing sessions and their restore mappings. V2 idle and LRU evictions emit the same audit metadata and tracing warning as v1.

The focused v2 suite has 31 tests. The focused session and combined daemon runs passed 76 and 49 tests respectively. The reversible-daemon integration suite passed 15 tests. The 20-turn test also re-cleans every returned block through v1 against the committed session and compares its bytes and manifest. The result-validation probe confirms the refusal test reaches validate_result through the full request path. A core test checks that dropping a prepared commit publishes nothing and a retry publishes once. Unreleased changelog entries describe the v2 operations, error variants, and routing of version: 2 frames. The CI full-input table now includes the new protocol reference; the input audit and computed-reader checks pass after integration with main.

Fixtures & PII hygiene

  • Synthetic fixtures only, with reserved email domains.
  • Fixture citation lint and the actual public-text scrub passed.
  • The final body passed installed gaze clean unchanged with zero entries.

Benchmark evidence

  • Not a detection change and not a benchmark change.
  • Changes the benchmark.

No benchmark was run. The new behavior is confined to opt-in daemon v2 requests; the existing default clean path is unchanged. No benchmark or performance gain is claimed.

Default-path equality compares main at 8e2ed551187e06d83e2556a04579e2d788da839e with tested source commit b349adbbc2d6396d3fb59dd3cd14dc24948a2123 using independently built release binaries and 1,072 hash-pinned inputs. The script and input manifest are committed as scripts/verify_clean_equality.py and scripts/verification/clean-equality-inputs-v1.json. The two whole-file inputs are explicitly pinned as the drift corpus and README because the earlier manual review script was not retained.

Result: zero differences in normalized clean text, entries, stats, leak reports, and exit codes. Both binaries succeeded on all 1,072 cases and produced 103 detections. Only emitted random session hex is normalized; owner-secret session blobs are excluded. The value-free equality report records commit IDs and the binary, input-manifest, and script hashes. The report-only commit follows the tested source commit.

DCO and commit discipline

  • All commits have signatures and DCO sign-offs. Files were staged by name; no amend, rebase, or force-push.

Naoray added 30 commits October 24, 2026 23:51
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2

Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2

Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ks-v2

Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
…ocation charges

Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
Signed-off-by: Krishan Koenig <krishan.koenig@googlemail.com>
@Naoray
Naoray added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 2fc435a Oct 26, 2026
26 checks passed
@Naoray
Naoray deleted the feat/daemon-text-blocks-v2 branch October 26, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant