Implement packed repository hard cutover and owned recovery - #34
Merged
Merged
Conversation
The SSH publication fixture selected a port by binding and immediately closing it, then asynchronously initialized a server before binding again. A competing listener at that real call site reproduces AddrInUse. Retain the original bound listener and hand it to the existing supervised startup API for initial startup and every restoration in this fixture family. Assert that another binder cannot claim the fixture port before handoff. All original refusal, disconnect and cold-preparation scenarios pass in the 105-case multi-server rerun. No bind retries or deadline changes.
Reuse the existing root purpose and ETag reservation in a bounded canopy-pack-v1 envelope. Reject unversioned or unknown remote formats before workspace reclamation, probes, identity writes or Cell activation. Reuse the owner/worker fences with the new managed runtime directory; reject and retain legacy local state. Include the boundary in release source hashing and update affected fixtures and harness paths. Local hard-cutover work only: the old production Git schema and command registry still require conversion with every producer and reader before this branch can be released. No legacy decoder or migration is added. Validation: 661 unique workspace Rust cases, 8 isolated RustFS cases, 96 Python harness cases, warnings-denied all-target Clippy, formatting, doctest completion, and server build. Original red format regressions and the earlier SSH suite failure are retained separately.
Reuse the bounded typed publication contract for actual Repository Cell migrations, activation and maintenance. Initialize private empty catalog/ref roots before Ready and authenticate retained initialization on cold load without new allocation. Production Git producer/reader conversion remains an unreleasable local cutover. Qualification reads published Cell roots through the sparse VFS, uses scoped provider keys for corruption injection and releases connection guards before awaits.
Retain the actual accepted Begin receipt in the existing logical request row, sharing the bounded authenticated record and lookup with staging. Startup recovers it before another Begin and checks current fence/custody separately. Explicit Claim can recover the authenticated original after operation reaping without restoring old custody or displacing a successor. Update initialization/compaction admission checks, command codecs and source hashing. Qualify both object formats, SDK expiry, fresh-owner restore, rollback, immutability and purpose separation. Preserve initial receipts in fixture state hashes and target fault injection at the actual publication update. Validation: 271 publication + 3 actual startup tests; all-target workspace Clippy with warnings denied; canopy binary build; fmt/diff and frozen sources. Local checkpoint only: remaining producer/reader cutover and full durable custody-command recovery are required before release.
Persist original SDK snapshots before Begin and retain positive and denied custody phases atomically with domain execution. Reuse the typed domain receivers, recorded receipts, namespace allocator and independent pins. Discover pending initialization and successor grants after cold restore. Unbind raw custody commands in production; explicit qualification fixtures retain domain receivers. Keep this partial cutover local pending service conversion, compact history archival and full capacity qualification.
Retain original custody and registrar commands in the staging service and fair preparation dispatcher. Preserve both identities through cancellation, registrar uncertainty and closed recovery; gate absent execution on the local fence and deadlines while returning recorded knowledge first. Remove caller-owned raw renewal APIs. Restore renewals with the existing shared session fence so a failed fresh observation also fences old resolvers. Charge bounded intent/body copies without raising admission or byte caps. Qualify 286 publication and nine startup/workspace tests, warnings-denied all-target Clippy, server build, frozen sources, dependency pins and protected checkout files. Document asynchronous file attribution and its native studies. This is an unpublished, unreleasable cutover checkpoint. Current-owner fencing for cold sessions, orphan lifecycle/history archival, complete production producer/reader/schema conversion and the remaining runtime/capacity gates are still required.
A fresh CheckPreparation can still return the previous owner's historical lease after actual owner restoration. Reopening that result incorrectly made the old token usable. Require a target-bound server-owned authority source and compare the admitted incarnation/epoch before and after fresh lease probes. Production reuses validated Cell Control and live node advertisements. Carry the source through preparation and staging handoff, base/frontier loading and standalone positive recovery. Failed observations permanently fence shared sessions; original known outcomes remain recoverable. Explicit registered Claim under the current owner can still restore a usable session. Validation: 289 publication and nine real startup/workspace tests on frozen source, workspace/all-target Clippy with warnings denied, canopy binary build, formatting, dependency/protection and documentation-link checks. New SHA-1/SHA-256 regressions cover cold old-owner renewal replay, current-owner takeover and missing/corrupt authority repair without un-fencing old sessions. This is a local unreleasable cutover checkpoint. Cold staging reconstruction, orphan intent lifecycle/history archival, production ingress/read conversion, retention and full-history/team capacity remain open.
Restore all seven original custody actions without changing identities or receipts. Retain positive and negative history before fresh owner/lease checks, including after closed-service recovery and SDK expiry. Wake bound callback supervisors from the shared permanent session fence; join cancellation and drop owned resources before returning worker credit. Validate 296 publication and nine real startup/workspace lifecycle tests, all-target workspace Clippy with warnings denied, server build and format. The production cutover and full-history/team capacity gates remain open.
Preserve exact original execution uncertainty in a separate authenticated stop fact. Reuse dispatcher admission for bounded keyset discovery and exact retirement recovery, and allow a stop beside its own pending preparation so its shared session can fence and drain. All 307 publication and nine real lifecycle tests pass, with workspace all-target Clippy, build, format and frozen-source/static checks. Full library qualification passes 585 of 590 cases; five legacy consumers still query the already-removed objects table. Production lifecycle wiring, reader/producer cutover and capacity qualification remain open. This is an unpublished implementation checkpoint, not a release.
Retire only the tracked transition's expired unresolved initialization head using the existing authenticated stop factory and exact completion. Preserve original outcomes and SDK identities; choose an explicit successor from a receipt-watermarked indexed observation and reject inconsistent state instead of treating it as absence. Seven new production-registry/schema regression families cover both object formats, real owner restoration after SQLite deletion, original history, automatic retirement, and authority/purpose/context refusal. Full library:592pass/5fail; the same five legacy readers still query the removed objects table. Nine real lifecycle cases, Clippy, build, fmt and frozen/static checks pass. The cutover remains unpublished and incomplete.
forhappy
marked this pull request as ready for review
October 4, 2026 16:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Packed repository cutover
The selected packed schema removes authoritative loose-object and graph tables. This PR selects that registry/schema, initializes repositories through certified empty roots, retains exact custody/publication recovery, and converts browser and native HTTP/SSH fetch reads to certified joint snapshots. Mutation publication and remaining authoritative consumers still need their production conversion.
Write-preparation caches now read one retained catalog/ref snapshot. Source descriptors stream through the immutable range tree; admitted disk deduplicates authenticated native pack/index pairs, and live refs stream into packed-refs. A writable sibling uses the baseline as an alternate, preventing existing catalog packs from becoming new receive inputs. Only requested ref expectations and exact post-receive names enter bounded batches. The retired SQL hydration cursor and shared loose-cache reuse/repack loop are removed; their regression coverage exercises the actual source cursor used by the cache constructor.
Staging contexts now retain the original worker/actor admission through detached physical work. Native receive, capture uploads, isolated physical verification, canonical extraction, edge writes and verifier cleanup carry that ownership through the existing process/file/spool structures. Bound callbacks receive the shared session and staging context. Custody remains checked independently; ownership cannot extend a lease or grant publication authority. Production physical download requires current-namespace staging custody, and the unowned entry point is qualification-only. Publication expiry fixtures apply the same one-second ceiling after Bind, preserving the original expiry, no-publication and credit assertions while separating setup latency from the tested phase.
CI is not fully green; this implementation is not ready to merge or deploy. The original read/cursor failures are resolved, including both actual preceding-head Linux library runs. The directory integration failure exposes unfinished production writes that still invoke retired loose-object ingestion. Complete the actual owned producers before converting their fixtures; no retired command/schema fallback or skipped case is added. Cold per-request write baselines, adopted old-owner input verification, and remaining request/metadata/policy physical ownership still need work. Large-history/team capacity is unqualified.
Validation
Frozen source at
d86f315, macOS / Rust 1.98.0:Registry("operation descriptor is unavailable")indirectory_reservations_recover_two_distinct_repository_cells. Total fresh Rust coverage is 730 unique cases: 729 pass, one fails; focused reruns and nested subprocess summaries are excluded. Later integration binaries/doctests and the latest-source full Linux/provider workflow remain unrun.6101d58Linux CI runs pass 693 server library cases and fail at the same directory integration case. New-head checks must report their own results.Remaining release requirements