Skip to content

perf(read): defer proven blob leaves during pack planning - #504

Draft
forhappy wants to merge 2 commits into
crab-v2from
codex/capsule-shallow-read-20261004
Draft

forhappy wants to merge 2 commits into
crab-v2from
codex/capsule-shallow-read-20261004

Conversation

@forhappy

@forhappy forhappy commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Avoid reconstructing proven blob leaves during shared upload-pack graph selection. The original planner fails the new minimized regression with a 2 MiB blob under a 64 KiB aggregate inflation budget. The retained Kubernetes depth-100 failure motivated this work, but its cold differential rerun is still pending.

  • Admit visibility before using authenticated, pinned locator kind proof.
  • Defer only non-root blobs under size-independent filters; preserve per-visit logical-object accounting, selection order, and deduplication.
  • Keep bounded content reads for explicit roots, missing kind proof, and size-dependent filters.
  • Preserve packed-entry verification, response limits, cancellation, and native Git installation checks. No budget increases, storage-format changes, or new configuration.

Ownership and scope

The change belongs in crab-read::plan_with_operation, shared by classic capsule fetch, upload-pack wire handling, and HTTP Git reads. Existing OperationContext::pinned_object_metadata provides snapshot-bound kind proof; it does not charge logical objects, so deferred visits are charged explicitly. Response generation remains responsible for packed-entry validation.

The regression publishes through the real capsule push path to an in-memory object store, plans a shallow response, installs its pack with native Git, verifies byte-identical blob content, and confirms explicit blob roots still hit the unchanged inflation limit. README documents the boundary.

Validation

  • Original source: new regression fails with the expected inflated-byte limit error.
  • Candidate: regression passes; 6 adjacent classic capsule tests and 30 shared planner tests pass (37 total).
  • cargo fmt --all --check and git diff --check pass.
  • cargo clippy --locked -p crab-read --all-targets -- -D warnings passes.

Container CI repair

The HTTP server image build failed before Rust compilation because Debian no longer publishes the pinned libpcre2-8-0=10.42-1+deb12u1 in the configured Bookworm repositories. With explicit user approval, both the server and Compose-example Dockerfiles now pin 10.42-1+deb12u2; image digests and security gates remain unchanged.

On the exact pinned Debian base, AMD64 (server package set) and ARM64 (Compose-example package set) both reproduced the obsolete-pin failure and installed the new exact version successfully. This is package-installation proof, not a full image/Compose qualification pass; current-head CI must supply that proof.

Still open

Draft pending current-candidate cold Kubernetes depth-100 verification, final exact-byte/integrity proof, and CI. This is not a claim that all shallow-clone amplification is eliminated. It does not waive push/fetch latency gates, establish matched-v1 performance, complete Xet/provider/recovery qualification, or retire v1. PR #208 is already merged; this is a follow-up against crab-v2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant