fix(engine): journal a root-only debt when a dropped version's staged root is gone

[FSM1]

Summary

This change implements ADR 0059 D1.

• A drop whose staged root is gone, fails its own CID, or does not decode now journals a retire-ledger debt. The debt has the new origin "dropped root", under the existing version byte. The root CID comes from the op record, and the owed figure is the size that the op record carries.
• The settle fetches the root over the gateway ladder, reads the owing node as OwingRecord::Unconfirmed, and retires the whole version under the node's record.
• A root that no source serves keeps the entry as a TargetUnexpandable stall. Its figure stays in the pending-reclaim figure.
• The preserved-set trim dropped an entry whose root stopped opening and journaled nothing. It now journals the same debt.
• Decode reads a dropped-root entry that has a target tail as unwritten. The type of the origin carries no target set, so encode cannot write a tail.
• The previous release reads a dropped-root entry as unwritten (unknown origin), and the ledger never discards, so the entry waits. This release reads every entry shape that the previous release wrote.
• blueprint/engine.md "Retirement" and "Referenced equals kept" carry the rewords that ADR 0059 Consequences 1 and 2 name.

Test plan

• write_plane::a_discarded_version_whose_staged_root_is_gone_still_retires_its_rows: a parked version loses its staged root, the member discards it, and the settle retires every block under the node's record. This test fails without the fix.
• retire unit tests: a dropped root round-trips, and a tail reads as unwritten. The settle retires it off the fetched root under the node's record, as unconfirmed. A root that no source serves stalls as TargetUnexpandable at its quoted figure. Hand-framed bytes of a dropped-version entry from the previous release still decode.
• staging unit tests: a trimmed version whose root is gone owes its root alone. A root that fails its CID is released and owes a dropped-root debt.
• cargo fmt --all, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p cipherbox-engine, cargo check -p cipherbox-wasm --target wasm32-unknown-unknown, pnpm lint:md, pnpm lint:tracker-refs.

Closes #2065.

Summary by CodeRabbit

• Bug Fixes
  • Retirement tracking now accounts for dropped versions even when their staged content is missing, invalid, or unreadable.
  • When the root content is available, its associated blocks can be reclaimed. If it cannot be read or expanded, the unreclaimed amount remains visible in pending reclaim totals.
  • Dead-lettered versions with a readable staged root can be settled without fetching a gateway copy.

Note

Journal root-only DroppedRoot debt when a dropped version's staged root is gone

Previously, dropping a staged version whose root was missing, failed CID verification, could not be decoded, or did not expand would silently lose registry debt. Now such drops journal a root-only debt.

• Adds a DroppedRoot variant to DebtOrigin in retire_ledger.rs, encoded as a versioned ledger entry with no target-set tail. Entries with an unexpected tail are treated as unwritten.
• DroppedVersionDebts.drop_staged in staging.rs always journals a debt: readable roots still carry a full target set, others journal the root alone with the operation's plaintext size (or zero). reconcile_preserved_dead_letters also converts preserved entries with missing roots into debt.
• Settlement in retire.rs treats DroppedRoot debts as Unconfirmed, fetches the root before deriving its target set, and expands with the staged-root profile instead of the quoted manifest size. If no source serves the root, the debt stays owed at its quoted figure.
• Updates ADRs 0047, 0054, and 0059, and engine.md policy prose to match the new rules.
• Risk: DroppedRoot debts settle via the unconfirmed-record path, so a fetchable-but-wrong gateway root could retire a derived target set under the node's name; check net.retire.expand_owed's DroppedRoot branch.

^{Macroscope summarized 6788a4d.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: FSM1/cipher-box/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 54a80347-29ca-4cc2-a11e-8b77b779799c

📥 Commits

Reviewing files that changed from the base of the PR and between 8951916 and 6788a4d.

📒 Files selected for processing (8)

• blueprint/engine.md
• crates/engine/src/net/retire.rs
• crates/engine/src/seams/retire_ledger.rs
• crates/engine/src/sync/staging.rs
• crates/engine/tests/write_plane.rs
• decisions/0047-a-failed-or-abandoned-publish-retires-exactly-what-it-charged.md
• decisions/0054-a-dropped-versions-debt-carries-its-target-set-and-settles-above-the-acknowledged-sequence.md
• decisions/0059-a-dropped-version-whose-staged-root-does-not-read-journals-its-root-alone.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

---

Walkthrough

Dropped staged versions now journal root-only retirement debt when their staged root is unavailable or cannot be expanded. Reclaim processing fetches and expands that root; unexpandable debt remains pending. The retirement ledger, staging reconciliation, tests, and related documentation were updated.

Changes

Dropped-root retirement debt

Layer / File(s)	Summary
Debt format and origin crates/engine/src/seams/retire_ledger.rs, crates/engine/src/net/retire.rs	The ledger adds the DroppedRoot origin. Its encoded entry has no target-set tail, and decoding rejects entries with a tail.
Journaling during staging crates/engine/src/sync/staging.rs, blueprint/engine.md	When expansion succeeds, drop_staged records the targets and pinned-byte count. Otherwise, it records root-only debt priced at the op’s plaintext size. Reconciliation routes gone roots through this fallback.
Root-only reclaim and decision updates crates/engine/src/net/retire.rs, crates/engine/tests/write_plane.rs, decisions/0047-*, decisions/0054-*, decisions/0059-*, blueprint/engine.md	Reclaim fetches and expands the root for DroppedRoot debt. If expansion fails, the debt remains owed and its quoted figure is reported. Tests cover ledger encoding, reclaim behavior, and dead-letter discard. The decision records describe the implemented behavior and pending-reclaim figure.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant DroppedVersionDebts
  participant RetirementLedger
  participant expand_owed
  participant RootSource
  participant expand_staged_root
  participant RetireBatches
  DroppedVersionDebts->>RetirementLedger: journal DroppedRoot with root CID and quoted figure
  expand_owed->>RootSource: fetch root CID
  RootSource-->>expand_owed: return root block
  expand_owed->>expand_staged_root: expand root block
  expand_staged_root-->>expand_owed: return targets or unexpandable result
  expand_owed->>RetireBatches: send batches for expanded targets

Loading

Merge Risk: ⚪ Minimal · up to 6788a

The change preserves retirement debt when staged roots are unusable and settles it when the root is available. No merge-blocking issue was found; merge after normal checks pass.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 6788a

The change makes previously untracked retirement work recoverable while retaining record-scoped retirement and content validation. No introduced security vulnerability was established. Risk remains low rather than minimal because concurrent recovery behavior and server-side authorization were not fully established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The demonstrated new exposure is gateway activity and retirement of content named by an owner-local dropped-root debt. Client requests remain scoped to the owing node's record, not an account-wide retirement operation. Backend enforcement and maximum server-side impact were not established by the inspected client source.

Trust Boundaries and Controls

• observed — Untrusted staging bytes must open under the owner's bookkeeping seal, and the embedded CID must match the ledger key. Remote root bytes must satisfy the root-plane anchor, size cap, CID verification, decoding, and content-profile checks before becoming retirement targets.
• observed — DroppedRoot inherits the Unconfirmed identity gate: the production path reads the current node record without cache and refuses tied records, unreadable acknowledged state, or records at or below the acknowledged sequence. A target still live is deferred, and other live CIDs are removed from the retirement set.

Resilience and Maintainability Implications

• inferred — A failed dequeue can leave an operation both queued and preserved, but the inspected gone-root branch does not establish a new loss of a usable queued retry. Upload already requires the staged root before reading its resume mark, while root-only cleanup removes no unknown leaves. Opaque roots and store errors are not automatically classified as Gone.
• observed — Sequential replay preserves an already-readable debt at the owner-and-CID key. Unavailable or unexpandable roots retain their debt without issuing retirement. ADR 0059 explicitly accepts that roots lost before upload can leave previously uploaded leaves charged; this residual was already unreclaimable before the PR and is now represented in pending debt.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR implements the coding requirements in [#2065]. DroppedVersionDebts::drop_staged journals DebtOrigin::DroppedRoot when the staged root is missing or unusable. Settlement fetches and expands …
Out of Scope Changes check	✅ Passed	The changes stay within [#2065]. The retirement-ledger, staging, and settlement changes implement the required debt path. The regression and unit-test updates verify the path. The ADR and engine docum…
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 3 files. (5 skipped: 4…
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the primary change: journaling root-only debt when a dropped version's staged root is missing.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[FSM1]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.