Skip to content

refactor(producer): add plan v2 artifact publisher#2792

Open
jrusso1020 wants to merge 1 commit into
feat/plan-protocol-v2-gcpfrom
feat/plan-v2-artifact-publisher
Open

refactor(producer): add plan v2 artifact publisher#2792
jrusso1020 wants to merge 1 commit into
feat/plan-protocol-v2-gcpfrom
feat/plan-v2-artifact-publisher

Conversation

@jrusso1020

@jrusso1020 jrusso1020 commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

What

Introduces a manifest-last PlanV2ArtifactPublisher seam and routes planV2() through a local publisher. The local implementation hard-links immutable blobs when staging and CAS share a filesystem, with an atomic copy fallback for cross-device or restricted filesystems.

The frozen v1 catalog and manifest construction are now storage-independent, duplicate digests publish once, and the existing synchronous createPlanV2FromV1() compatibility helper remains available.

Why

Plan v2 currently creates a complete v1 staging tree and then copies the retained bytes into a second local CAS tree before cloud upload. This PR is the first direct-emission foundation: it removes the second set of local data blocks on the normal same-filesystem path and establishes the manifest-last publisher contract needed by direct S3/GCS emission.

This does not yet remove the complete v1 staging tree. That remains the next stacked producer change; oversized-plan rollout stays disabled.

How

  • separates plan catalog/manifest construction from storage publication
  • publishes each unique digest before committing plan.json
  • aborts partial publication without exposing a manifest
  • hard-links local immutable blobs and atomically renames the completed transport
  • preserves manifest bytes, hashes, materialization, and worker execution

Test plan

  • Plan v2 focused tests: 11 passed
  • Producer typecheck
  • Changed-file oxlint and oxfmt
  • Fallow, tracked-artifact, and pre-commit gates
  • Same-filesystem inode-sharing assertion
  • Injected blob failure proves abort occurs and manifest is never committed
  • Execution plan updated with the remaining direct-emission stack and rollout gates

Copy link
Copy Markdown
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@jrusso1020
jrusso1020 force-pushed the feat/plan-protocol-v2-gcp branch from 5fc20bd to d5991e5 Compare July 25, 2026 23:33
@jrusso1020
jrusso1020 force-pushed the feat/plan-v2-artifact-publisher branch from b6f41f6 to 8c31919 Compare July 25, 2026 23:33
@jrusso1020
jrusso1020 force-pushed the feat/plan-v2-artifact-publisher branch from 8c31919 to 7ecae52 Compare July 25, 2026 23:37
@jrusso1020
jrusso1020 force-pushed the feat/plan-protocol-v2-gcp branch from d5991e5 to 33a06aa Compare July 25, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant