Skip to content

feat(bundle): register a folder bundle init wrote back into the store it came from - #356

Open
earakely-scale wants to merge 13 commits into
feat/bundle-registerfrom
feat/bundle-register-imports
Open

earakely-scale wants to merge 13 commits into
feat/bundle-registerfrom
feat/bundle-register-imports

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Why

agent-env bundle init writes a stored task into a folder you can run and edit. Registering that folder back should keep the task what it was. The folder's entities should go back to the ids they came from, and an untouched folder should write nothing. An edit should re-version only what it reaches, without losing the fields the import left out.

Stacked on #355 (bundle register), and it includes #353's commit (the import record v2), which this is the reader of. If you'd rather review one PR for the import side, close #353 and merge this after #355.

What it does

When the folder's provenance.json records an import from the configured store (its store fingerprint matches):

  • Ids: each entity, image, task and eval takes the store id it was imported from. An imported task's outputs keep the names its file gives them, as the stored task did. Only ids register makes up are held to the registry id rule, so a source id like nightly/regression passes.

  • Untouched folders write nothing: a write whose files are as the import wrote them keeps the version it was imported at, as long as everything it needs is kept at the versions they were imported at too. The ledger records that version, so later runs compare digests as usual. The files compared are:

    • a task's or eval's file;
    • an image's build context;
    • an env's or agent's toml, or the whole folder for an env type written from its folder;
    • every file of an artifact or skill.

    This compares against the import's file hashes, not the stored entity's digests, because import prunes contexts and pins their installs.

  • Edits re-version only the edited write and what references it.

  • Re-grafting: each imported task's steps get back the user-scoped fields the import left out, such as the agent, env vars, sandbox type and model. They come from the stored version, matched by step id and type, wherever the folder doesn't set them. A shared store's credential and loopback-URL checks skip these fields, since the store holds them already.

  • A version someone else wrote since the folder was imported is refused, unless --adopt.

A folder imported from another store, or recorded by an older agent-env, registers as a new bundle with prefix ids, and the run says why.

Verification

Unit tests: tst/unit/bundle/register_imports_test.py, 8 tests, plus #353's import-record tests. They cover:

  • an unedited import writes nothing, keeps every version, and records rows;
  • an edit re-versions only the edited artifact and its task, and gives back the agent and env vars;
  • a re-grafted field isn't refused as a credential on a shared store;
  • an edit over a version the store wrote since is refused until --adopt;
  • an edited Dockerfile pushes a new image and rewrites its env;
  • another store's import, and an older record, register as new bundles;
  • an imported task's output keeps its name.

The full unit tier passes: 10,912 passed. One a2a staging timing test failed once under load (the run took 15 minutes instead of 2) and passes 3/3 alone; this PR doesn't touch it.

Open-source stores on an agent-environments-admin VM (Mongo, SeaweedFS S3, and a TLS registry with auth):

  • echo-mcp registered, then imported from those stores with bundle init. Registering the folder back, as a dry run and for real, kept every entity at its source id and v1, writing nothing.
  • A git clone of the imported folder on another path still registered as untouched.
  • Editing the check re-versioned only the check and the task, v2 ran, score 1.0.
  • An edited Dockerfile gave image, env and task new versions, and the new task ran, score 1.0.
  • After another write took the check to v3, an edit was refused, then written as v4 with --adopt.
  • No version was written twice, no pending rows were left, and no secret text appeared in any results.

Dev stage, from a laptop, since the dev cluster isn't reachable from the admin VM:

  • The fd3965-e2e-echo-smoke task feat(bundle): agent-env bundle register writes a bundle into the configured stores #355's e2e registered was imported and registered back untouched, writing nothing.
  • Its edit wrote only the check and the task, v2. The new version got back the sandbox_type: local the import had dropped, and ran on a Modal VM, score 1.0.
  • The real task-rocket-prospecting-christina-rivera-v1 (v7) was imported with --include-scope user and dry-run registered. All 21 writes showed as unchanged at their source versions, and nothing was written. The writes are 9 MCP-server envs and their images, the multi-env, the claude-code-cli agent (v74) and its image, the universe artifact, and the task.
  • A plain import of rocket fails in bundle init itself. Two of its stored build contexts hold several Dockerfiles and record none, which import already refuses without --include-scope user.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 4/5

Fix the masked URLs saved after filename edits before merging.

Fix All in CursorFindings

  1. P1 Renamed downloads lose credentials ▶
Fix with agent prompt
### Issue 1
src/agent_env/bundle/_imported.py:256-257
After importing with `--include-scope user`, changing `urls[].filename` prevents `_counterpart()` from finding the stored download-even when it is the list's only item. Registration then saves the masked `url` instead of restoring the original URL. Downloads that require those credentials stop working.

Match safely where possible, and refuse registration when a masked value cannot be restored rather than silently saving it.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR registers imported folders under their original store IDs and versions, restores omitted task settings, and reuses untouched writes.

  • Imported folders register under the store IDs they came from.
  • Imported task steps regain settings that the folder left out.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Imported folder] --> B{Same store and supported record?}
    B -->|No| C[Register with generated IDs]
    B -->|Yes| D[Keep source IDs and restore omitted settings]
    D --> E{Files and dependency versions unchanged?}
    E -->|Yes| F[Reuse imported versions]
    E -->|No| G[Check ownership and write changed versions]
Loading

Reviews (7) · Last reviewed commit: "fix(bundle): restore a masked value in a..." · Reviewed by Greptile

earakely-scale and others added 4 commits October 27, 2026 10:28
…back can tell an edit

provenance.json, schema 2, now also holds:
- the task map, each task file with the store's task id and version, and the eval's;
- a fingerprint of the configured document store's section, which tells one store
  from another without naming where either is;
- each file the import wrote, as its content hash (+x when executable).

The pinned store references move from "store" to "pinned". Nothing reads the file yet.
The file hash the ledger computes moves to _fs, so both use one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it came from

When the folder's provenance.json records an import from the configured store:
- each entity, image, task and eval takes the store id it was imported from, and an imported
  task's outputs the names its file gives them;
- a write the folder still holds as the import wrote it, with all it needs at the versions
  they were imported at, keeps the version it was imported at, so an unedited folder writes
  nothing, and the ledger records it;
- an edit re-versions only what it reaches;
- each imported task's steps get back the user-scoped fields the import left out (its agent,
  env vars, sandbox type...), from the stored version, by step id, where the folder doesn't
  set them. A shared store's credential and URL checks skip those fields, which it holds
  already.

A folder imported from another store, or recorded by an older agent-env, registers as a new
bundle, and says why. Registering over a version someone else wrote since, the bundle's own
or the imported one, is refused unless --adopt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	src/agent_env/bundle/register.py
@earakely-scale
earakely-scale requested a review from a team as a code owner October 27, 2026 21:43
Comment thread src/agent_env/bundle/_import_report.py Outdated
Comment thread src/agent_env/bundle/register.py
Comment thread src/agent_env/bundle/_imported.py Outdated
Comment thread src/agent_env/bundle/_imported.py Outdated
Comment thread src/agent_env/bundle/_imported.py
Comment thread src/agent_env/bundle/_imported.py
Comment thread src/agent_env/bundle/register.py Outdated
Comment thread src/agent_env/bundle/_import_report.py Outdated
# Conflicts:
#	src/agent_env/bundle/register.py
Comment thread src/agent_env/bundle/register.py Outdated
Comment thread src/agent_env/bundle/_imported.py Outdated
earakely-scale and others added 2 commits October 27, 2026 15:59
# Conflicts:
#	src/agent_env/bundle/register.py
…d id its own, when registering the folder

- A pin is given back only beside the reference it was stored with, not beside an entity the import wrote into the folder.
- An env var the import dropped and a credential it masked come back where the folder still holds what it left.
- A list item, such as a skill, takes back its left-out source from the stored item it was written from.
- Envs sharing one stored image register as imported; an edit to one copy is refused.
- An image whose toml names another Dockerfile is rebuilt.
- An installed bundle's entries take nothing from the folder's record, and an output under a folder keeps its name.
- The import numbers a name another kind holds for another id, and registering refuses one name taking two ids.
- The store fingerprint resolves env: references and ignores credentials, and registering reads only the files of
  what the import wrote.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/_import_report.py
Comment thread src/agent_env/bundle/_imported.py Outdated
# Conflicts:
#	src/agent_env/bundle/register.py
…urce the folder gives

- The import records each image it names by registry ref with its stored id and version, so registering an untouched
  folder reuses it rather than writing it, its env and its task anew.
- A list item's left-out fields come back together, and only when the item sets none of them, so a new object_url or
  a switch to body or skill_artifact_id is never undone by the stored s3_url. Matching by name no longer applies when
  the item sets a field its stored item doesn't.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/_import_report.py Outdated
Comment thread src/agent_env/bundle/_import_report.py Outdated
Comment thread src/agent_env/bundle/register.py Outdated
earakely-scale and others added 2 commits October 27, 2026 23:17
…n unchanged imported write once

- Registration compares each imported write with the files its writer reads: links inside the bundle are followed, so
  a linked folder in an image's context is an edit to it, and what the OS leaves behind, such as .DS_Store, isn't.
- An unchanged imported write is adopted only when the ledger has no row of this bundle for it at that version and
  digest, so registering again adds no rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/bundle/_imported.py Outdated
… it was masked from

While a list is as the import left it, each item takes back its own position's value. Once it's edited, an item takes
back the value of the one stored item whose scrubbed form it equals, so reordered downloads keep their own credentials,
and an item that matches none, or several, keeps what it holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment on lines +256 to +257
same = [other for other, item in enumerate(scrubbed) if item == value[index]]
return same[0] if len(same) == 1 else None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Renamed downloads lose credentials

After importing with --include-scope user, changing urls[].filename prevents _counterpart() from finding the stored download—even when it is the list's only item. Registration then saves the masked url instead of restoring the original URL. Downloads that require those credentials stop working.

Match safely where possible, and refuse registration when a masked value cannot be restored rather than silently saving it.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/bundle/_imported.py
Line: 256-257

Comment:
**Renamed downloads lose credentials**

After importing with `--include-scope user`, changing `urls[].filename` prevents `_counterpart()` from finding the stored download—even when it is the list's only item. Registration then saves the masked `url` instead of restoring the original URL. Downloads that require those credentials stop working.

Match safely where possible, and refuse registration when a masked value cannot be restored rather than silently saving it.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant