Skip to content

feat(bundle): agent-env bundle register writes a bundle into the configured stores - #355

Merged
earakely-scale merged 13 commits into
mainfrom
feat/bundle-register
Oct 28, 2026
Merged

earakely-scale merged 13 commits into
mainfrom
feat/bundle-register

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Why

A bundle folder runs locally (agent-env run) but nothing writes it where a service runs tasks. agent-env bundle register writes a folder into the configured stores under registry ids, so whatever runs the stores' tasks runs what the folder ran: the hub and its workers, agent-env task run, an eval at scale.

What it does

agent-env bundle register FOLDER [--task NAME]… [--eval NAME]… [--prefix P] [--dry-run] [--yes]

It plans the bundle as a run does, reusing parse, resolve, plan and materialize, then translates each @local id the plan writes into a registry id:

  • Entries: <prefix><name>, with the name lowercased and every character an id can't hold made -. The prefix defaults to the folder's name plus a dash. Two entries that land on one id are refused.
  • Derived ids keep their suffix after the base's registry id (echo-echo__env_image). A task's outputs take <task id>-<name>.
  • A shipped agent named by its alias (claude-code) is registered under the alias when the store lacks it, with its published image pulled by digest. A step that leaves its agent unset stays unset, so whatever runs the task gives its own default; the planner gains default_agents=False for this.

Each task and eval is written naming the version of each entity it references. That is the version register wrote or reused for the bundle's own entities, and the version the plan read for a store's. A registered version never changes meaning, and a new env version gives the tasks naming it new versions too. A field that takes no version is listed as floating in the dry run.

Images. Each image the bundle builds is built here for linux/amd64 with Docker and pushed to the configured image store under the tag src-<sources>. A push the registry already serves is skipped. The image is registered by digest with its build context beside it. An image published for its sources (a shipped agent, the mail example) is registered as published, and nothing is built.

Ledger. A ledger in the target store's bundle_registrations collection makes a rerun write only what changed. Each row names the bundle by its git remote, without credentials, plus its path in the repo (or folder:<path> outside git), and records the commit, whether the folder was dirty, and the agent-env version. The Ledger class gains a collection, reference pinning and row fields, so RegistrationLedger is a small subclass.

Refused before anything is written:

  • an @local id left in a document;
  • a store reference the store lacks.

An id the store already holds takes the bundle's next version, whoever wrote the store's latest.

For a store other than this machine's (anything but the local SQLite store), also refused:

  • a credential written out;
  • an env var set to an env: or secret: reference, since nothing resolves one in a step's env vars, so the run would get the reference itself;
  • a URL only this machine reaches (file:, loopback). Writing to such a store asks for confirmation first, unless --yes.

Concurrency. Two registrations of a bundle on one machine wait for each other on the run's per-id locks, then the second reuses what the first wrote. Two on different machines aren't serialized by those locks, so the registration ledger holds each write to the version its check found next.

This uses expecting_version in the document store: inside it, a put of that id that names no version takes exactly the held version through insert_next, in both VersionedEntityStore.put and ArtifactStore.put_document. A registration that loses a race raises ConcurrentModificationError instead of writing v2, and a rerun reuses what the winner wrote. A write that raises also leaves no pending ledger row.

Repository names. An image's repository is its id with __ made -, because ECR refuses two separators in a row and a derived id holds __.

Not in this PR

  • Registering an imported folder back to its source ids. That PR uses feat(bundle): record what an import wrote, so registering the folder back can tell an edit #353's record: unedited folders write nothing, and dropped user-scoped fields are re-grafted.
  • An installed bundle's env, which a task names by its alias, is refused before anything is written: no store env may start on an installed env alias's id. An installed agent named by its alias is registered under the alias.
  • The agentenvhub-sdk wrapper (project ids, hub task defaults, the prod path).

Verification

Unit tests: tst/unit/bundle/register_test.py, 32 tests, covering:

  • ids, slugs and collisions, and --prefix;
  • pins for bundle and store refs, and eval pins;
  • the ledger: reruns, an edit re-versioning only what it reaches, rows in the target store, and remotes without credentials;
  • an id the store already holds taking the bundle's next version, whoever wrote it;
  • outputs, the unset default agent, a shipped agent under its alias, and a pushed image;
  • the @local leftover, credential, env-var reference and loopback refusals;
  • the CLI's dry run, confirm, --yes and nothing-to-write;
  • a failed write, a lost race, a refused installed env, and a registration resumed after being killed between a write and its ledger row;
  • a step that names no version taking the latest even where another step pins one;
  • push_context_image pushing once.

The full unit tier passes: 10,903.

e2e and chaos on an agent-environments-admin VM, against open-source stores: MongoDB 7, SeaweedFS (S3) and registry:2 with TLS and basic auth.

  • echo-mcp registered with its image built and pushed to the registry. The record holds a digest and the S3 context, and the task pins the env and check at v1.
  • mail-mcp registered with its published GHCR image, building nothing.
  • Each registered task ran from those stores alone with agent-env task run on the local provider, score 1.0. Infra envs were put with env service-db/gateway put.
  • A rerun wrote nothing. An edited check re-versioned only the check and the task, and the new task ran, score 1.0.
  • A second clone of the repo is the same bundle, with nothing to write. (A copy outside git was refused until --prefix by an ownership check this PR has since dropped.)
  • No secret text in any results: the Mongo, S3 and registry passwords, and the token in the git remote.

Chaos cases, each followed by a rerun and a check of every id's versions:

Case Result
killed (kill -9) during the image push rerun completes, every id at v1
killed just after its first write rerun reuses it, every id at v1
4 registrations at once, one machine serialized by the locks, every id at v1
2 at once as from two machines (own state dirs) one writes everything at v1, the other fails with ConcurrentModificationError, rerun has nothing to write
registry stopped mid-push fails, rerun completes
Mongo down fails on connect, nothing written, rerun completes
S3 down during upload fails, rerun completes

Before the strict writes, the two-machines case wrote every id at v1 and v2.

Default local stores, with no config: echo-mcp registered into SQLite, the local object store and the local registry, then ran, score 1.0.

Dev stage (agentenvhub-sdk 0.35.5, --stage dev): run from a laptop, because the dev cluster isn't reachable from the admin VM.

  • mail-mcp registered into agent_env_dev and its dev bucket, and ran on a Modal VM, score 1.0.
  • echo-mcp built and pushed to the dev ECR registry, at agent-env/agent-images-dev/fd3965-e2e-echo-echo-env_image, then ran on a Modal VM, score 1.0.
  • A rerun wrote nothing. (An id another bundle registered was refused, by the ownership check since dropped.)

These ids stay in dev, since entities are append-only: 8 entities under fd3965-e2e-, their bundle_registrations rows, and one ECR repository.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The reviewed changes appear safe to merge, with no new blocking issues found.

Summary

Adds agent-env bundle register to write bundle entities into configured stores, pin task and eval references, push built images, and reuse unchanged versions.

  • Bundle folders can be registered in configured stores under registry IDs.
  • Registered tasks keep the versions they were written against.
  • Bundle-built images are pushed to the configured image store.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Bundle folder] --> B[Parse, resolve, and plan]
  B --> C[Translate local ids]
  C --> D[Check references and shared-store restrictions]
  D --> E[Check registration ledger]
  E --> F[Reuse unchanged versions]
  E --> G[Build and push required images]
  G --> H[Write entities at expected versions]
  H --> I[Write pinned tasks and evals]
  I --> J[Record completed writes]
Loading

Reviews (10) · Last reviewed commit: "refactor(bundle): drop the ownership che..." · Reviewed by Greptile

earakely-scale and others added 3 commits October 27, 2026 11:12
…igured stores

The bundle is planned as a run plans it, then each @Local id it writes takes a registry id:
- an entry takes <prefix><name>, the prefix the folder's name and a dash by default;
- a task's output takes <task id>-<name>;
- a derived id keeps its suffix;
- a shipped agent named by its alias takes the alias.

Each task and eval names the version of what it references. Each image the bundle builds is
built for linux/amd64, pushed under a tag of its sources and registered by digest with its
build context. A ledger in the target store's bundle_registrations makes a rerun write only
what changed. Problems are refused before anything is written: an id the store holds that the
bundle didn't register (unless --adopt), an @Local id left in a document, or, for a shared
store, a credential written out or a URL only this machine reaches. A step that leaves its
agent unset stays unset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…her than write the next one

Two registrations of one bundle on different machines aren't serialized by the local locks, so
both wrote every id, the second at v2. The registration ledger now holds each write to the
version its check found next (expecting_version): a put of that id then takes exactly that
version, through insert_next, or raises ConcurrentModificationError, and a rerun reuses what
the winner wrote. Only an agent an installed bundle comes with is registered under its alias:
no store env may start on an installed env alias's id, so an installed env is refused before
anything is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pending row marks a write a run started, so the next run keeps the version an interrupted one
wrote. One whose write raised wrote no version the run knows of, so the row is removed: a
registration that lost a race left one behind in the shared store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner October 27, 2026 19:10
Comment thread src/agent_env/bundle/materialize.py Outdated
Comment thread src/agent_env/bundle/register.py Outdated
Comment thread src/agent_env/bundle/register.py Outdated
earakely-scale and others added 2 commits October 27, 2026 12:49
…ed registration, recheck ownership at each write

- A step naming no version took another step's pin of the same id; each now takes the version
  the bundle wrote, or the store's latest the plan read.
- A registration killed after a write but before its ledger row was done couldn't resume: the
  rerun refused the id as unowned. A pending row of this bundle at the stored version now counts
  as its own, so the ledger adopts it.
- Ownership is checked again as each write is checked, so an id another bundle registers after
  the first check is refused before it's written, not taken at its next version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bundle's own ledger row let it write an id whatever the store's latest was, so a version
another tool or person wrote after the last registration was written over unseen. Now an id
whose latest version isn't the one this bundle last registered is refused unless --adopt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits October 27, 2026 15:04
…hich nothing resolves

The shared-store check let a secret-named env var through when its value was a secret:
reference, as if the run would resolve it. Nothing resolves env: or secret: references in a
step's env vars, so the run would get the reference itself. Such a value is now refused,
whatever the variable's name, with a reason saying so. The other refusals now say what
each one exposes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests that a bundle's identity drops a remote's credentials held URLs with user info
written out, which the pre-commit secret scan flags as new secrets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/register.py Outdated
earakely-scale and others added 2 commits October 27, 2026 15:58
… timeout

A task and an artifact whose names make one id live in different stores, so they no longer refuse each other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead each pinned version one way

- materialize takes the ledger instead of a Registering wrapper: Ledger.pins_references now also decides whether tasks
  and evals are written pinned, and Ledger.pushes_images whether built images are pushed.
- Tasks and evals pin through one helper, reading the versions straight from what was written rather than a copy kept
  in step by hand. An eval ref whose store version the plan didn't read now floats instead of raising, as a task's does.
- The kind-to-store table is gone: an EntityKind's value is its store's key.
- The registry-id check no longer rewrites "__", which the id pattern already allows.
- translated() builds its fields in _translations, beside _KEPT, the fields that hold no @Local id; a test fails when a
  Plan field is in neither.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/materialize.py
@earakely-scale

Copy link
Copy Markdown
Collaborator Author

@greptileai review

…and through one that's down

Builds and pushes the env's image to the local registry, runs the registered task by its id to a passing score, and
checks registering again writes nothing. A registry that can't be reached fails the registration before the image is
built or named, and a rerun once it's back finishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread tst/integration/cli/bundle_register_test.py
earakely-scale and others added 2 commits October 27, 2026 23:08
…ration tests on the local registry

main folded Plan's aliases and env_aliases into alias_uses, by kind, so a registered plan renames each of them there.
The integration tests now use the local registry whatever AGENT_ENV_CONFIG names, so they never push to a hosted one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/ledger.py
@greptile-apps

greptile-apps Bot commented Oct 28, 2026

Copy link
Copy Markdown

Comments Outside Diff

These findings could not be posted inline.

  • P2 Unchanged tasks gain extra versions src/agent_env/bundle/ledger.py:150 ▶

    Registering an unchanged task that names a missing shipped agent, such as claude-code, writes another task version on the next registration. The first registration records the agent in inputs["needs"]. Once the alias exists in the store, the second records the same id and version in inputs["store_refs"].

    Those different hashes rewrite the task and any eval that pins it, despite unchanged content. Compare the final pinned references consistently across both paths.

…gister

An id the store holds already now takes the bundle's next version, whoever wrote the store's latest. Two bundles taking
one id is unlikely, and the check cost a refusal, its recheck at each write, and --adopt to get past it. The strict
version each write lands on stays, so two registrations racing for one id still can't both write it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit d8476a3 into main Oct 28, 2026
25 of 26 checks passed
@earakely-scale
earakely-scale deleted the feat/bundle-register branch October 28, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant