Repository navigation
feat(bundle): agent-env bundle register writes a bundle into the configured stores - #355
Merged
Merged
Conversation
…igured stores The bundle is planned as a run plans it, then each @Local id it writes takes a registry id: - an entry takes <prefix><name>, the prefix the folder's name and a dash by default; - a task's output takes <task id>-<name>; - a derived id keeps its suffix; - a shipped agent named by its alias takes the alias. Each task and eval names the version of what it references. Each image the bundle builds is built for linux/amd64, pushed under a tag of its sources and registered by digest with its build context. A ledger in the target store's bundle_registrations makes a rerun write only what changed. Problems are refused before anything is written: an id the store holds that the bundle didn't register (unless --adopt), an @Local id left in a document, or, for a shared store, a credential written out or a URL only this machine reaches. A step that leaves its agent unset stays unset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…her than write the next one Two registrations of one bundle on different machines aren't serialized by the local locks, so both wrote every id, the second at v2. The registration ledger now holds each write to the version its check found next (expecting_version): a put of that id then takes exactly that version, through insert_next, or raises ConcurrentModificationError, and a rerun reuses what the winner wrote. Only an agent an installed bundle comes with is registered under its alias: no store env may start on an installed env alias's id, so an installed env is refused before anything is written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pending row marks a write a run started, so the next run keeps the version an interrupted one wrote. One whose write raised wrote no version the run knows of, so the row is removed: a registration that lost a race left one behind in the shared store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed registration, recheck ownership at each write - A step naming no version took another step's pin of the same id; each now takes the version the bundle wrote, or the store's latest the plan read. - A registration killed after a write but before its ledger row was done couldn't resume: the rerun refused the id as unowned. A pending row of this bundle at the stored version now counts as its own, so the ledger adopts it. - Ownership is checked again as each write is checked, so an id another bundle registers after the first check is refused before it's written, not taken at its next version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bundle's own ledger row let it write an id whatever the store's latest was, so a version another tool or person wrote after the last registration was written over unseen. Now an id whose latest version isn't the one this bundle last registered is refused unless --adopt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hich nothing resolves The shared-store check let a secret-named env var through when its value was a secret: reference, as if the run would resolve it. Nothing resolves env: or secret: references in a step's env vars, so the run would get the reference itself. Such a value is now refused, whatever the variable's name, with a reason saying so. The other refusals now say what each one exposes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests that a bundle's identity drops a remote's credentials held URLs with user info written out, which the pre-commit secret scan flags as new secrets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… timeout A task and an artifact whose names make one id live in different stores, so they no longer refuse each other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead each pinned version one way - materialize takes the ledger instead of a Registering wrapper: Ledger.pins_references now also decides whether tasks and evals are written pinned, and Ledger.pushes_images whether built images are pushed. - Tasks and evals pin through one helper, reading the versions straight from what was written rather than a copy kept in step by hand. An eval ref whose store version the plan didn't read now floats instead of raising, as a task's does. - The kind-to-store table is gone: an EntityKind's value is its store's key. - The registry-id check no longer rewrites "__", which the id pattern already allows. - translated() builds its fields in _translations, beside _KEPT, the fields that hold no @Local id; a test fails when a Plan field is in neither. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
@greptileai review |
…and through one that's down Builds and pushes the env's image to the local registry, runs the registered task by its id to a passing score, and checks registering again writes nothing. A registry that can't be reached fails the registration before the image is built or named, and a rerun once it's back finishes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ration tests on the local registry main folded Plan's aliases and env_aliases into alias_uses, by kind, so a registered plan renames each of them there. The integration tests now use the local registry whatever AGENT_ENV_CONFIG names, so they never push to a hosted one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comments Outside DiffThese findings could not be posted inline.
|
…gister An id the store holds already now takes the bundle's next version, whoever wrote the store's latest. Two bundles taking one id is unlikely, and the check cost a refusal, its recheck at each write, and --adopt to get past it. The strict version each write lands on stays, so two registrations racing for one id still can't both write it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A bundle folder runs locally (
agent-env run) but nothing writes it where a service runs tasks.agent-env bundle registerwrites a folder into the configured stores under registry ids, so whatever runs the stores' tasks runs what the folder ran: the hub and its workers,agent-env task run, an eval at scale.What it does
agent-env bundle register FOLDER [--task NAME]… [--eval NAME]… [--prefix P] [--dry-run] [--yes]It plans the bundle as a run does, reusing parse, resolve, plan and materialize, then translates each
@localid the plan writes into a registry id:<prefix><name>, with the name lowercased and every character an id can't hold made-. The prefix defaults to the folder's name plus a dash. Two entries that land on one id are refused.echo-echo__env_image). A task's outputs take<task id>-<name>.claude-code) is registered under the alias when the store lacks it, with its published image pulled by digest. A step that leaves its agent unset stays unset, so whatever runs the task gives its own default; the planner gainsdefault_agents=Falsefor this.Each task and eval is written naming the version of each entity it references. That is the version register wrote or reused for the bundle's own entities, and the version the plan read for a store's. A registered version never changes meaning, and a new env version gives the tasks naming it new versions too. A field that takes no version is listed as floating in the dry run.
Images. Each image the bundle builds is built here for linux/amd64 with Docker and pushed to the configured image store under the tag
src-<sources>. A push the registry already serves is skipped. The image is registered by digest with its build context beside it. An image published for its sources (a shipped agent, the mail example) is registered as published, and nothing is built.Ledger. A ledger in the target store's
bundle_registrationscollection makes a rerun write only what changed. Each row names the bundle by its git remote, without credentials, plus its path in the repo (orfolder:<path>outside git), and records the commit, whether the folder was dirty, and the agent-env version. TheLedgerclass gains a collection, reference pinning and row fields, soRegistrationLedgeris a small subclass.Refused before anything is written:
@localid left in a document;An id the store already holds takes the bundle's next version, whoever wrote the store's latest.
For a store other than this machine's (anything but the local SQLite store), also refused:
env:orsecret:reference, since nothing resolves one in a step's env vars, so the run would get the reference itself;file:, loopback). Writing to such a store asks for confirmation first, unless--yes.Concurrency. Two registrations of a bundle on one machine wait for each other on the run's per-id locks, then the second reuses what the first wrote. Two on different machines aren't serialized by those locks, so the registration ledger holds each write to the version its check found next.
This uses
expecting_versionin the document store: inside it, a put of that id that names no version takes exactly the held version throughinsert_next, in bothVersionedEntityStore.putandArtifactStore.put_document. A registration that loses a race raisesConcurrentModificationErrorinstead of writing v2, and a rerun reuses what the winner wrote. A write that raises also leaves no pending ledger row.Repository names. An image's repository is its id with
__made-, because ECR refuses two separators in a row and a derived id holds__.Not in this PR
Verification
Unit tests:
tst/unit/bundle/register_test.py, 32 tests, covering:--prefix;@localleftover, credential, env-var reference and loopback refusals;--yesand nothing-to-write;push_context_imagepushing once.The full unit tier passes: 10,903.
e2e and chaos on an agent-environments-admin VM, against open-source stores: MongoDB 7, SeaweedFS (S3) and
registry:2with TLS and basic auth.echo-mcpregistered with its image built and pushed to the registry. The record holds a digest and the S3 context, and the task pins the env and check at v1.mail-mcpregistered with its published GHCR image, building nothing.agent-env task runon the local provider, score 1.0. Infra envs were put withenv service-db/gateway put.--prefixby an ownership check this PR has since dropped.)Chaos cases, each followed by a rerun and a check of every id's versions:
kill -9) during the image pushConcurrentModificationError, rerun has nothing to writeBefore the strict writes, the two-machines case wrote every id at v1 and v2.
Default local stores, with no config:
echo-mcpregistered into SQLite, the local object store and the local registry, then ran, score 1.0.Dev stage (agentenvhub-sdk 0.35.5,
--stage dev): run from a laptop, because the dev cluster isn't reachable from the admin VM.mail-mcpregistered intoagent_env_devand its dev bucket, and ran on a Modal VM, score 1.0.echo-mcpbuilt and pushed to the dev ECR registry, atagent-env/agent-images-dev/fd3965-e2e-echo-echo-env_image, then ran on a Modal VM, score 1.0.These ids stay in dev, since entities are append-only: 8 entities under
fd3965-e2e-, theirbundle_registrationsrows, and one ECR repository.🤖 Generated with Claude Code


Confidence Score: 5/5
The reviewed changes appear safe to merge, with no new blocking issues found.Summary
Adds
agent-env bundle registerto write bundle entities into configured stores, pin task and eval references, push built images, and reuse unchanged versions.Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Bundle folder] --> B[Parse, resolve, and plan] B --> C[Translate local ids] C --> D[Check references and shared-store restrictions] D --> E[Check registration ledger] E --> F[Reuse unchanged versions] E --> G[Build and push required images] G --> H[Write entities at expected versions] H --> I[Write pinned tasks and evals] I --> J[Record completed writes]Reviews (10) · Last reviewed commit: "refactor(bundle): drop the ownership che..." · Reviewed by Greptile