feat(protocol): publish Git and Xet state through capsules

[forhappy]

Summary

• Publishes Git refs through v2 per-ref capsules with conditional root updates, immutable layered Git packs/checkpoints, and Xet xorbs/shards kept outside the capsule.
• Integrates capsule-aware push, fetch/clone, history, repack, fsck, GC, and recovery.
• Reuses published chunk-to-xorb mappings; new or unknown content follows the validated staging path.

Qualification status

• The 5,000-commit Kubernetes/RustFS replay passed exact-tip correctness; fetch p95 was 8.581 s.
• A 500-commit incremental fetch completed in 5.55 s and installed one new pack. It made 27 object-store requests; request count is diagnostic, not a gate.
• Push p95 missed the 1 s target in 5/10 windows; cold/warm clone took 53.724/20.008 s. No matched v1 performance comparison has run.
• The 100 GiB RustFS GA Xet run stopped with SIGBUS during historical hydration; cold cross-repository xorb discovery remains incomplete.

This PR is not release-qualified. Provider/product parity, full crash/recovery and maintenance coverage, the zero-error 100 GiB Xet run, push-tail and clone targets, and matched v1 performance remain open. Keep v1 supported until correctness, parity, and matched-or-better performance are demonstrated.

Evidence:

• Layered-pack design and qualification status
• Xorb/shard parity inventory
• 5,000-commit RustFS replay

[forhappy]

Follow-up qualification and hardening (commit 9393caf):

• Terminal upload-pack now retains delta bases external only when the request is unfiltered, non-shallow, non-deepen, advertises thin-pack, and every client have is authenticated in the pinned v2 visibility plan. OFS deltas are rewritten to REF_DELTA; the full dependency sort/materialization path remains for filtered, shallow, incomplete, and legacy requests.
• Catalog-selected objects retain authenticated physical pack order; legacy materialized selection keeps canonical OID ordering.
• Fresh release binary v2-tiny-external-final-20260917 on RustFS: two incremental pushes 304–327 ms at 8 requests each; two fetches 225–247 ms at 9 requests; final clone 501 ms at 18 requests; matching tip and strict fsck passed.
• Focused tests: crab-remote-git 133, crab-read 188, upload-pack wire 35; cargo check -p crab, release build, architecture gates, and format/diff checks pass.

The Kubernetes 5,000-commit stress artifact remains explicitly negative at the 500-commit fetch/repack boundary because the interrupted pre-fix repository state still requires a large historical pack; it is not being reported as parity proof. Hosted-provider, multipart, replica/tiering, mount/browser, S3 gateway, migration/recovery, and backup/restore rows remain release gates.

[forhappy]

Parity follow-up (commit d6431f8): removed the stale client rejection for protected pushes carrying a v2 mirror-plan ID. The plan ID is already authenticated in CapsuleTransaction::for_plan; the auth-server capsule publisher commits the same transaction-scoped capsule plan receipt. The protected capsule receive test now exercises that planned transaction, verifies the receipt, and retries successfully. cargo test -p crab-auth-server --lib (101), capsule-push tests (9), and the release build pass.

[forhappy]

Added external_thin_subset_pack_keeps_the_proven_base_outside_the_pack (commit 36f2ea6). It generates the public external-base thin-pack API from a real REF_DELTA fixture, verifies the one-object thin pack with strict git index-pack --fix-thin, and passes.

[forhappy]

Documentation follow-up (commit 0233c25): the main capsule publication design now records the authenticated external thin-base rule and protected mirror-plan receipt path alongside the parity inventory and RustFS evidence.

[forhappy]

V1 parity pass

Implemented and pushed in 6ad3b0b0e9a (rebased onto current main):

• Legacy read replicas now use the v1 manifest/index/object readiness contract only when v2/root is absent; a present or corrupt v2 root never downgrades. The resolver accepts verified legacy manifests, and tests cover both acceptance and corrupt-root rejection.
• migrate import, migrate export, and adopt --rewrite-history now use the built-in verified fast-export/fast-import engine with v2 Xet staging/hydration. Shared Git blobs are converted inline only for selected paths; ref rollback is attempted on post-import failures, checkout failures are surfaced, and staging is closed before success.
• Remote snapshot download/export, mount, hydrator, browser/HTTP, protected publication, and checkpoint readers carry one authenticated v2 view and immutable pointer catalog through reconstruction.
• Documentation now contains a complete v1 product-parity inventory, cross-surface contracts, closure order, and Level-3 acceptance gates.

Proof after rebase:

• cargo check -p crab --locked passed.
• cargo test -p crab --locked --lib: 4,250 passed, 0 failed, 3 ignored.
• cargo fmt --all -- --check, git diff --check, and python3 crab/scripts/check-architecture-gates.py passed.

Remaining release blockers are intentionally explicit: hosted-provider checksum/multipart and 5,000-commit current-format replay; managed replica failover/repair; tier/archive restore; mount range/cancellation/unmount; browser/HTTP load and fault matrix; S3 gateway operation/concurrency/restart matrix; lifecycle/workflow/admin inventory; backup/restore export inventory; and migration fault/resume/provider plus older-Git/interrupted/adversarial qualification. Full v1 production parity is not claimed until those Level-3 gates pass.

[forhappy]

Parity closure update (c2d87d8):

• Hydrate and remote mount now share one restore-availability adapter. It is built from the resolved physical store identity, so managed and replica read views target the bucket that owns the authenticated v2 catalog instead of reconstructing a provider from the logical crab:// URL.
• --no-restore does not construct a cloud restore client, but archived shard/xorb reads still fail closed with the typed archive-class admission error.
• A standalone crab:// mount now refuses to start if its authenticated v2 read context cannot be built; it no longer starts with stub readers and defers the failure until first pointer access. Local Git-native mount fallback is unchanged.
• Design matrix and closure notes are updated in crab/docs/design/capsule-xorbs-shards.md.

Local proof after this change:

• cargo test -p crab --locked --lib: 4,254 passed, 0 failed, 3 ignored.
• cargo test -p crab --locked --lib cmd::mount: 120 passed before the final guard, plus the two new fail-closed/fallback tests passed individually.
• cargo check -p crab --locked, cargo fmt --all -- --check, git diff --check, and python3 crab/scripts/check-architecture-gates.py all pass.

This closes the local wiring gap, but is not a claim of complete v1 production parity. Release gates remain: live S3/GCS/Azure lifecycle and restore behavior; replica readiness/failover/repair; restored-content verification; the full FUSE/NFS range/cache/cancellation matrix; browser and smart-HTTP load/fault coverage; S3 gateway restart/concurrency/request-count coverage; and migration, backup inventory, and delete/restore qualification on populated v1/v2 repositories.

[forhappy]

Follow-up test hardening: the focused mount module now passes 122/122. I also serialized the unmount test's HOME override through the existing test guard; this removes a process-global HOME race that could make local_pipeline_config_rejects_active_cache flaky when mount tests ran in parallel. This is test-only and does not alter local Git-native fallback behavior.

[forhappy]

Final local rerun after the test-only race fix: cargo test -p crab --locked --lib --quiet passed 4,254 tests (3 ignored) in 99.51s; cmd::mount passed 122/122. No working-tree changes are pending other than pre-existing generated Python __pycache__ directories, which were not added.

[forhappy]

Layered-pack qualification update (commit 1640af9):

• Fixed the ref-only layered-run regression: physical pack source validation now runs only for runs that carry Git members; ref-only runs still retain authenticated visibility and transition checks.
• Exact release build verified on local RustFS/Kubernetes fixture: 1.1 GiB cold git clone --no-checkout completed in 3.75 s; default checkout clone completed in 8.04 s; exact tip 71f0fc6e72d53d5caf50b1314ca4d754463117f0; connectivity fsck passed; 26,892 files checked out.
• Test proof: crab-read 200/200, remote-helper 139/139, metadata 210/210, release build, formatting, and diff checks pass.
• The earlier ~20 s shared-volume result is destination write contention (RustFS and destination sharing the qualification volume), not object-store request or connectivity amplification.

The 5,000-commit replay with 500-commit fetch/repack checkpoints and hosted-provider matrix remain explicit release gates; v1 is not being retired until those pass.

[forhappy]

Follow-up pushed in ee72375b4b09b3df74d90b15ffcecfcd5edd8868.

The first fresh PR-208 5,000-commit run found a correctness blocker before replay could continue: an append-only in-memory visibility dictionary could be non-canonical when serialized into the layered capsule (CRAB-E0020: layered visibility dictionary is not in canonical order). The fix canonicalizes the OID dictionary at the capsule boundary and remaps ref, transition, and history ordinals; it preserves the append-only runtime representation and keeps the decoder fail-closed.

Proof for the fix:

• cargo test -p crab-metadata --lib --locked: 211 passed
• cargo test -p crab-read --lib --locked: 200 passed
• cargo test -p crab --lib git::remote_helper --locked: 139 passed
• cargo fmt --all -- --check and git diff --check: passed

I am rebuilding the exact release binary from this commit and rerunning the fresh local-RustFS qualification. The earlier run remains recorded as a failed negative qualification at seed + replay 1; no 5,000-commit success is claimed until all replay, checkpoint fetch/repack, clone, and fsck gates pass.

[forhappy]

Layered-pack follow-up pushed in 4ed9664ab70 (on top of 042c129cd55):

• The writer now canonicalizes append-only visibility ordinals at the wire boundary and remaps refs, transitions, history closures, and authenticated member admission through the same old-to-new map. This closes the two fresh-replay correctness failures found after the ref-only fix.
• crab-metadata passes 212/212 and crab-read passes 200/200 on the patched tree.
• The local-RustFS smoke (seed + 10 replay pushes, fetch/repack checkpoints at 5 and 10) now passes every push, fetch, repack, tip, and fsck gate; incremental pushes remain about 1.04 s after the seed.
• The design doc records the remaining cold-clone blocker: a multi-member checkpoint correctly declines the one-source/one-member direct installer, then the normal path performed 357,313 uncoalesced range reads in 163 s before the run was intentionally stopped (1.89 GB read, 15.2 GB inflated). This is not being reported as a cold-clone or 5,000-commit qualification pass.

The PR is updated with the fixes and evidence, but v1 is not retired and the 5,000-commit/multi-pack cold-clone gates remain open until the authenticated multi-member install or equivalent whole-member union path is implemented and requalified.

[forhappy]

Post-push test completion for 4ed9664ab70:

• cargo test -p crab-metadata --lib --locked: 212 passed
• cargo test -p crab-read --lib --locked: 200 passed
• cargo test -p crab --lib git::remote_helper --locked: 139 passed
• cargo fmt --all -- --check and git diff --check: passed

The branch remains clean apart from the pre-existing untracked local crab.toml RustFS config, which is not part of the PR.

[forhappy]

Updated in commit 1cfdd63 (perf(fetch): preload layered locators for cold clones).\n\nWhat changed:\n- Complete layered views now coalesce and authenticate index, reverse-index, and kind-bearing locator sidecars, then expose inline locators to the planner.\n- Ordinary incremental haves stay on the footer/tip-bound path; cold, filtered, shallow, and tag requests promote only when complete visibility is required.\n- Multi-member cold clones no longer fall back to per-object visibility reads; direct one-pack installation remains fail-closed.\n- Design history and qualification evidence are recorded in crab/docs/design/capsule-layered-packs.md.\n\nVerification:\n- crab-read: 200 passed.\n- remote-helper: 139 passed.\n- upload-pack wire: 39 passed.\n- Local RustFS, Kubernetes-derived fixture: blob:none clone 14.50 s with 173 ms planning; cache-miss shallow blob:none clone 6.12 s with 1,095 planning reads and 372 terminal response-pack reads; unfiltered multi-member clone 85.94 s, exact source tip, native git fsck --full clean.\n- Full, filtered, and shallow clone tips all match the source.\n\nThe fresh 5,000-push/fetch/repack matrix, hosted-provider latency, and v1-retirement gates remain open; this update does not claim those are complete.

[forhappy]

CI follow-up: GitHub did not emit a synchronize run for the new head, so I manually dispatched the current commit (1cfdd63) against the repository workflows:\n- CI: https://github.com/crabbuild/crab/actions/runs/35675208869\n- Git protocol v2 partial-clone qualification: https://github.com/crabbuild/crab/actions/runs/35675210814\n- Large repository RustFS qualification: https://github.com/crabbuild/crab/actions/runs/35675212409\n\nAt this update they are queued, not yet green; the prior completed run was for an older head.

[forhappy]

Pushed dba2cb4 (fix(protocol-v2): retain negotiated fetch haves).

What changed:

• Accumulate and de-duplicate protocol-v2 haves across negotiation rounds before any tip-bound → complete-view promotion.
• Copy the complete negotiated have set into the terminal done request, so incremental planning remains wants-minus-haves instead of regenerating the repository.
• Added design evidence in crab/docs/design/capsule-layered-packs.md.

Proof:

• cargo test -p crab --lib upload_pack_wire --locked: 39 passed
• cargo test -p crab-read --lib --locked: 200 passed
• cargo test -p crab --lib remote_helper --locked: 140 passed
• Full-history K8s + local RustFS: incremental fetch after pushes 1 and 5 passed with exact tips and no missing objects; response packs were 49.9 MiB and 55.7 MiB instead of the prior 1.09 GiB complete response.

Qualification status remains honest: the replay later reached an existing 503,980,520-byte Crab/Xet pointer commit and stopped with CRAB-E0086 because the replay harness had not staged its local chunks. The 5,000-push/xorb qualification gate is still open; this is a staging-contract failure, not evidence that the haves fix is incorrect.

[forhappy]

Pushed follow-up commit 1696f53 to PR 208.

The fresh full-history GitHub-origin Kubernetes RustFS qualification (pr208-v2-fresh-github-smoke2-20260921, binary dba2cb4) passed seed publication, protocol-v2 incremental fetches at pushes 1/5/10, exact tip checks, cold and warm full clones, and native fsck. Measured incremental fetches were 33.457s / 15,494 storage range reads / 50.1MB response at push 1 and 24.227s / 16,999 reads / 55.5MB response at push 5. Cold full clone was 217.5s with 10 store requests; warm clone was 99.8s with zero store requests.

The run stopped only at the blobless-clone qualification assertion blob-none-ordinal-metadata-lookup: the exact ordinal-metadata lookup branch emitted no locator_lookup_mode event, so the harness observed zero metadata events even though the request completed through the catalog-filter plan. The new commit adds that trace at the crab-metadata reader boundary; no data-path or authorization behavior changes. Focused crab-metadata tests pass (212/212).

Fresh workflows have been dispatched at this new head:

• CI: https://github.com/crabbuild/crab/actions/runs/35684015947
• protocol-v2 partial clone: https://github.com/crabbuild/crab/actions/runs/35684017903
• large repository RustFS: https://github.com/crabbuild/crab/actions/runs/35684019821

I am not marking the PR green until those runs complete.

[forhappy]

Pushed f0db75ac994 to PR 208.

This fixes the v2 incremental-fetch regression caused by generation-owner checkpoint compaction: the control-only reader previously saw no live capsule transitions after compaction and fell back to a visibility traversal. Layered checkpoints now carry a bounded, authenticated recent per-ref transition suffix in the footer. Control-only fetches consume that suffix; older/incomplete have chains still fail closed to the existing catalog/traversal planner. The complete ordinal visibility body remains authoritative for cold/strict paths.

Proof:

• cargo fmt --all --check
• cargo test -p crab-metadata --lib --locked: 213 passed
• cargo test -p crab-read --lib --locked: 200 passed
• local RustFS tiny fixture, committed binary SHA matched source SHA; seed + 10 replay pushes, incremental fetches at 1/5/10, clones, and fsck reached exact tips. Incremental logs report strategy=tip_bound_transition, visibility_plan_ms=0, and exact-pack-member responses.
• Qualification report is intentionally not called fully green: the tiny fixture's blob-none-ordinal-metadata-lookup check is false because that small clone selected the valid catalog_filter path; the incremental correctness gates all passed.

Fresh manual workflows for this commit:

• CI: https://github.com/crabbuild/crab/actions/runs/35687962739
• protocol-v2 qualification: https://github.com/crabbuild/crab/actions/runs/35687958669
• large-repository RustFS qualification: https://github.com/crabbuild/crab/actions/runs/35687960517

[forhappy]

Qualification update for f0db75ac994:

• The local RustFS K8s replay is still running with the refreshed GitHub-origin source and 5,000 replay commits. It has completed 302 pushes so far with no push failure, missing-object error, or tip mismatch.
• Seed publication, cold clone, and incremental fetch gates at pushes 1, 10, and 100 have passed with exact tips. At push 10, the v2 transition planner took 0 ms, used 3 object-store reads for 149 objects, and generated the response pack in about 30 ms.
• This is not a green qualification result yet: the full replay, 500-push fetch/repack gate, final clone, and fsck are still pending.
• The three fresh workflows for this head remain pending/queued: CI, protocol-v2 qualification, and large-repository RustFS qualification.

[forhappy]

Update: pushed a1d745f41abc9234c18bdebfd4d092318fa913a8 (perf(metadata): bound visibility updates and fetch history).

• crab-metadata lib tests: 214 passed; targeted visibility tests: 17 passed.
• Release binary built from this commit.
• Local RustFS/Kubernetes-shaped K8s qualification is running with the exact binary, seed + 5,000 first-parent pushes, fetch/repack every 500, and end-to-end verification.
• At 500 pushes: 501/501 pushes succeeded; visibility planning used tip_bound_transition in 1 ms; incremental fetch completed in 6.95 s with 51 measured object-store requests (53 in the operation log). The old pre-change 500-fetch control was 69.7 s and 74,135 requests.
• The remaining fetch latency is Git pack/index materialization; the v2 server still generated a response pack for this protocol-v2 request. The 5,000-commit run is intentionally still in progress, so this is not being claimed as final green qualification yet.

Known local gate: workspace cargo clippy -p crab-metadata --lib -- -D warnings still reports six pre-existing/unrelated lint failures in adjacent capsule/visibility code; no lint suppression was added.

[forhappy]

Qualification update from the exact a1d745f binary: the 1,000-push checkpoint completed with 1,001/1,001 pushes successful. Owner/checkpoint maintenance was 608.9 s (two active packs, peak child RSS ~0.86 GiB). The 1,000-commit incremental fetch completed in 59.7 s, with 100 storage requests and 6.53 s pack generation; visibility planning remained 24 ms. This exposes the remaining v2 bottleneck clearly: the protocol-v2 server is still materializing response packs as the requested delta grows, so PR 208 is not yet a final sub-second/under-10-request qualification. The local 5,000 replay remains in progress.

[forhappy]

Focused regression gate from a1d745f: cargo test -p crab-read --lib capsule_protocol --locked -j2 passed 21 tests (179 filtered), covering layered member admission, authenticated pack identity, coalesced/split range windows, control-only checkpoint reads, and incremental transition retention.

[forhappy]

Qualification update: the 1,500-boundary checkpoint completed with 1,501/1,501 pushes successful. Owner maintenance was 511.8 s (peak child RSS ~1.69 GiB, two active packs). The incremental fetch then completed in 15.4 s with 146 storage requests and 17,445 logical objects; visibility planning was 1 ms. This reinforces the outstanding protocol-v2 response-pack scaling gap; correctness remains intact and the 5,000 replay is continuing.

[forhappy]

Qualification update: the 2,000-boundary checkpoint completed with 2,001/2,001 pushes successful. Owner maintenance was 659.3 s (peak child RSS ~0.79 GiB, two active packs). Incremental fetch completed in 5.88 s with 126 storage requests and 20,279 logical objects; visibility planning was 3 ms. Fetch wall time varied versus the 1,500 sample, but the request count remains dominated by protocol-v2 response-pack reads. The 5,000 replay continues with no correctness failure.

[forhappy]

Interim 100 GiB Xet qualification (not closeout)

Fresh local RustFS 1.0.0 GA run against PR head 03b3185e6a22 is in progress at xet-100g-head03b3185-ga20261003-r1 (isolated bucket crab-xet-100g-pr208-03b3185-20261003-r1).

• Seed crab add: 1,046,175 ms (17m26s).
• Seed push: 1,118,287 ms (18m38s), about 21 GiB stored for the 100 GiB logical / 20 GiB unique-content fixture.
• Capsule root/run publication and the checks so far pass. The v0 layered repack completed in 4,134 ms and preserved refs, xorb/shard inventories, and retained history.
• 74 checks are currently passing; no transport errors have been observed. The next full-corpus add is still running, so incremental push, clone/hydrate, recovery, and full-run outcomes are not yet available.

These seed timings are not evidence of acceptable performance or a matched v1 comparison. The current-head CI Compose/Cellule qualification failure and remaining provider/product parity gates are still open. Fetch request count is diagnostic only per acceptance; it is not a gate. Keep v1 supported until all correctness, parity, and matched-performance gates pass.

[forhappy]

Qualification progress: v1 incremental update

The full-run report now includes v1 on the 100 GiB fixture. This was a broad history commit (50 model files and 500 source files changed), not a simple one-file push:

• v1 add: 594,360 ms (9m54s); v1 push: 13,579 ms.
• Push meter: 132 object-store requests (63 PUT, 63 GET, 4 HEAD, 2 LIST), 80,240,856 request-body bytes.
• Inventory grew from 330 to 380 xorbs and from 1 to 2 shards; retained refs/history checks pass.
• Proxy transport exceptions remain zero. The aggregate records one 4xx response with no path trace, so I am not treating that status as harmless until it is classified.
• 128 checks currently pass. Concurrent-add qualification has started; clone/hydration and remaining recovery gates are still pending.

This is not a matched v1 performance comparison and does not pass the sub-second/simple-push or <10-request push gates. The full run remains in progress; v1 stays supported.

[forhappy]

v2 update push: repeated measurement

The third history version completed on the same 100 GiB fixture. The v1 and v2 updates each changed 50 model files plus 500 source files and had nearly flat push results:

Version	Add	Push	Object-store requests	PUT / GET / HEAD / LIST
v1	594,360 ms	13,579 ms	132	63 / 63 / 4 / 2
v2	627,645 ms	13,636 ms	132	63 / 63 / 4 / 2

Each update added 50 xorbs and one shard; push request bodies were about 80 MB. These are broad content updates, not the simple-push benchmark, and they do not pass the sub-second goal. Each push has one aggregated 4xx response with no path trace; proxy transport exceptions are zero, but the 4xx still needs classification.

The report currently has 178 checks and no failures. Consumer cross-repository reuse, cold clone/hydrate, retained-history restore, and full-run completion remain pending.

[forhappy]

Fresh cold cross-repository reuse result

The current-head 100 GiB run has now passed cold-consumer-reuses-shared-chunks: an independent cold consumer published a 537,919,488-byte file containing shared source bytes plus a unique tail, while adding just 1 xorb (1,433,575 bytes). Its cold cross-repository push completed in 8,232 ms. The consumer clone completed in 767 ms; full hydrate is still running.

This supersedes the earlier failed cold-cache fixtures for this tested overlap shape. It does not yet establish arbitrary partial-overlap discovery, all-provider behavior, or full v1 parity. The request meter does not isolate request counts for this individual consumer push. At the last report snapshot, all 234 checks passed and transport-proxy exceptions remained zero.

[forhappy]

Clone and hydration progress

The independent 537,919,488-byte consumer hydrate completed in 14,279 ms with exact byte identity. The full 100 GiB repository clone completed in 839 ms; this is pointer/Git metadata clone latency, not full content materialization.

Full-corpus cold hydration is still in progress. At the latest sample it had materialized 8 of 50 model files (16 GiB logical output) after 12m39s; RustFS transfer/block counters continued increasing, with no observed transport exceptions. All 238 checks remain passing. This phase must finish exact byte verification, a rehydrate cycle, and retained-history restore before the run can pass.

[forhappy]

Cold-hydrate midpoint

The exact-head 100 GiB cold hydrate has reached 25/50 model files (50 GiB logical output) at 24m39s. RustFS network counters have stayed flat since the shared Xet chunks were fetched; current progress is local reconstruction and file writes. Workspace free space is ~245 GiB, and the report remains at 238 passing checks with no failures.

The clone command itself was 839 ms; this slower phase is full Xet content materialization, not Git/capsule clone metadata. Final byte verification, the rehydrate cycle, and history restore are still pending.

[forhappy]

Qualification update for PR #208 at 03b3185.

The 500-commit incremental fetch passed correctness in 5.55 s and installed one pack. Its 27 object-store requests are diagnostic only, not an acceptance gate; fetch correctness and latency remain gated.

The fresh RustFS 1.0.0 GA 100 GiB run has 238 checks with no reported failures so far, including the large seed/repack, v1/v2 updates, Xet cross-repository chunk reuse, and byte-identical consumer hydration. Full cold hydrate is still running at about 30 minutes, actively writing large temporary outputs; workspace free space is about 231 GiB. The report has not advanced past the hydrate-capacity check. Transport totals are 1,197 requests with no proxy exceptions; 16 4xx responses still need path-level classification, so this is not a zero-error result.

This does not close v2 qualification. Remaining gates include completing and verifying cold hydrate plus dehydrate/rehydrate and historical restore, classifying the 4xx responses, current-head large replay and matched v1 performance, provider/product parity, and resolving the current-head CI failure in Build and inspect image (fallback-candidate/Cellule rotation assertion). Azure, GCS, NFS, and Kubernetes qualification jobs are skipped in the current workflow. V1 retirement remains deferred until correctness, parity, and matched performance are demonstrated.

[forhappy]

Qualification update for current PR head 2493d4cc5f609746c13c1922e5916ae54eb3952d.

Fetch decision: the 27 requests observed for a 500-commit incremental fetch are accepted as diagnostic, not a release gate. Keep the correctness/latency contract: exact tip, one installed pack, no fetch-triggered repack or stable-body reread, and p95 at or below 10 seconds. The completed October 3 replay meets that fetch contract at 8.581 s p95; request count is not a blocker.

The latest pushed change fixes the Compose qualification fixture while preserving the genuine non-member recovery assertion. Its current CI job is still in progress, so this does not yet count as a pass. Current PR checks: 5 passed, 10 running, 6 queued, and 2 skipped.

The fresh RustFS GA 100-GiB run is still active. Its report has 839 checks and zero reported failures; cold hydrate and dehydrate passed, and rehydrated hydrate is in progress after passing capacity admission (315,642,658,816 available bytes vs. 128,849,018,880 required). The transport artifact currently records 1,197 requests: no proxy exceptions, but 16 HTTP 4xx responses remain unclassified, so this is not yet a zero-error result.

Closeout remains blocked by push tails (five of ten 500-push windows miss the 1-second p95 target), clone throughput and missing matched-v1 comparison, completion of the 100-GiB Xet/history run, v2 add-time cross-repository chunk discovery parity, live crash/GC and eventual historical-restore qualification, hosted provider/platform and shipped-product parity, and green final-head CI. The request-count change closes none of those gates. Keep v1 supported; do not mark the capsule design or implementation complete yet.

Design/gate inventory: layered-pack closeout and Xorb/shard parity.

[forhappy]

Current-head update: guarded push visibility reuse

Current PR head: 822e88a079b997da54f5b126fc084f1091459b01.

This commit narrows one incremental-push path: for a verified, non-force, single-ref fast-forward whose incoming pack proves the new tip and excludes only the old tip, visibility uses the exact verified pack-member set instead of repeating a graph walk. Force pushes, multiple ref edits, uncertain ancestry, excluded/sibling refs, and oversized visibility sets retain the existing graph-walk path. The focused capsule-push suite passed (14 tests); cargo fmt -p crab -- --check and git diff --check passed.

Current-head CI run 37115777425 is still in progress. The image-build job has not reached Compose recovery yet; the previous head's inactive-log-rotation assertion therefore remains unresolved. Several hosted-provider and platform qualification jobs are skipped in this run.

The retained 100-GiB RustFS run is not current-head proof: its frozen Crab source revision is 03b3185e6a22ad5f85166ae995d5647f562f3274, not this PR head. Its report ended failed when v0 historical hydrate exited -10 after 245,444 ms. The command logs are empty and the signal source is unidentified. Transport recorded 1,960 requests, 31 HTTP 4xx responses, and zero proxy exceptions; the 4xx responses remain unclassified. Evidence and remote objects are preserved.

Closeout decision

The capsule design and implementation are not complete or ready to retire v1. The prior 5,000-push replay passed content correctness, but five of ten windows missed the sub-second push-p95 gate; this new optimization has only focused-test proof and needs a current-head replay plus a matched v1 comparison. The 500-commit fetch result remains accepted: 8.581-second p95 and 27 requests, with request count diagnostic rather than a gate.

Still open are a successful current-head 100-GiB Xet/history run with classified transport responses, general add-time partial-overlap dedup parity, the live crash/concurrency/GC and eventual-restore matrix, and complete provider/platform/product parity. The design inventories in crab/docs/design/capsule-layered-packs.md and crab/docs/design/capsule-xorbs-shards.md remain the acceptance checklist. Keep v1 supported until those correctness and matched-performance gates pass.

[forhappy]

Exact-head qualification closeout update (behavior HEAD db9286794b34282eb75d3a552655170f7f531c3a; documentation-only follow-up pushed as 2bb034eaac65a65e3e49ff199372e0a26cf536d6): design status and retained measurements.

• Local RustFS 1.0.0 GA run k8s-5000-headdb928-20261003-r1 completed all 5,000 replay pushes with no push command failures. All ten fetch/repack checkpoints at 500-commit intervals returned the exact expected tips; fetch times were 2.482–7.035 s. Fetch object-request counts are diagnostic under the current acceptance policy.
• Push mean was 342.8 ms and aggregate p95 710 ms, but one 500-push window failed the required 1 s p95 target: pushes 2,501–3,000 had p95 1.144 s (max push 9.731 s). The other nine windows passed that p95 threshold. Per-push object-store request counts are not instrumented in this run; report zeros are missing telemetry.
• Final full clone completed in 19.323 s cold / 15.704 s warm for a 1.315 GB ODB; strict Git fsck passed in 125.815 / 122.781 s. The blobless clone command exited 0, but the harness stopped on blob-none-ordinal-metadata-lookup (zero events). Final source/sample comparison, incremental clone fsck, and remote Crab fsck therefore did not run. This is a failed/incomplete qualification, not a correctness pass.
• Retained report SHA-256: dc02b49e5d1f880240e8b7ccac730646fea95ac48335eb9d0a903ff576a2b467. The prior exact-head CI run failed the Compose owner-loss fixture before the follower-only write; approval is pending for the scoped fixture correction and replacement of the blobless telemetry-only gate with stronger behavior checks. CI for docs head 2bb034e is queued.

Do not conclude the design or retire v1: the push window, instrumentation gap, final integrity checks, few-second clone goal, 100 GiB Xet, matched v1, provider/product parity, and recovery/concurrency/GC qualification remain open.

[forhappy]

Closeout decision: still not complete

Agreed: the 27 object-store requests on the 500-commit fetch are acceptable and diagnostic only. Fetch remains within the correctness/latency contract; request count is not a blocker.

The capsule design/implementation still has material open gates on the current behavior head. The exact-head 5,000-commit run completed all pushes and exact-tip fetch checkpoints, but one 500-push window missed the 1 s p95 target (1.144 s; maximum push 9.731 s), and per-push request telemetry was not measured. Full clones took 19.3/15.7 s and strict Git fsck took about 2 minutes. The blobless clone command exited successfully, but the harness stopped on a telemetry-only assertion before proving omitted-blob/lazy-hydration behavior and before final source/sample and Crab fsck checks. This is incomplete qualification, not a correctness pass.

CI for documentation head 2bb034eaac65a65e3e49ff199372e0a26cf536d6 is still running/pending; the previous behavior-head image job failed in the Compose fixture. The 100 GiB Xet, old Xorb-journal historical restore after quarantine, matched v1 performance, hosted provider/product parity, and full failure/concurrency/GC matrix also remain open. I have not changed the remaining Compose/blobless assertions because the requested approval for those replacements is still pending.

The design doc records the evidence and acceptance inventory: capsule layered-pack status. Keep v1 supported; do not mark v2 release-qualified or retire v1 yet.