Skip to content

feat(computer): setup and check steps reach inside a computer, and computer app envs - #380

Open
earakely-scale wants to merge 10 commits into
mainfrom
edgararakelyan/fd-4366-m09-load_artifact-and-run_code-inside-the-computer
Open

earakely-scale wants to merge 10 commits into
mainfrom
edgararakelyan/fd-4366-m09-load_artifact-and-run_code-inside-the-computer

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

M0.9 and M1.1 of the CUA env epic (FD-4345): the framework's setup and check steps reach inside a computer, FD-4366, and the computer_app env type, deployable alone as a one-app computer, FD-4359. It follows the reviewed design, CUA Env M0.9 + M1.1: Code in the Computer, and Computer Apps, with its 15 choices confirmed. It builds on #369 (M0.7), now merged, and is rebased onto main (#374).

What's in it

M0.9: steps that reach inside the computer. Every setup and check step on a computer env now goes through its driver, as the session user, inside the desktop session; none runs on the sandbox's host or as root.

  • load_artifact of a file or file universe (choice 1a): a computer env overrides load_file_artifact_universe to send the files as data/add parts (env/envs/computer/inside.py).
    • Each part's bytes come through a read grant when the object store's grants reach the computer and its network allows everything, else inline, so a computer with an allowlist, or none, still loads (choice 2). On local Docker the local store's grant server serves them.
    • Parts go in batches of at most 64 MiB, since the driver holds a call's files in memory; a bigger file goes alone.
    • destination_path is relative to the home, Downloads by default; an absolute path is refused, and the record keeps home-relative paths (choice 3).
  • Environment artifacts (choice 4): load_artifact refuses one aimed at an env whose type has no loader for it (load_environment_artifact, load_environment_universe_artifact), at save and before it loads, so a computer env gets a clear refusal instead of an AttributeError at run. Env itself is unchanged (decided in review, rather than base defaults that refuse). How an environment universe maps onto a computer's apps is designed with M1.4 (FD-4369).
  • run_code with env_id on a computer (choices 1b, 5, 6, 7): a fourth target, chosen when the env's card serves computer-exec/v1 run, so a plugin driver works too. A card serving run but not data/add or data/get is refused, naming what's missing.
    • The runner and input are staged under ~/.agentenv/run_code/<step id>/ with data/add, run with run, the result read back with data/get after the same 50 MiB size check as every target, and the folder removed, since the agent can read the home.
    • input["env"] holds the driver on loopback, http://127.0.0.1:<port> and its /mcp, which code inside the computer reaches with no network. The deploy record's metadata.computer now keeps the driver's port for this.
    • A timeout_seconds that leaves the runner less than 10 s of the card's max_timeout_s is refused, so the runner always stops the script itself; the client waits for the command's timeout plus a margin.
    • Each command has a fresh request_id, sent again only when a call's connection drops, so a retried step really runs again.
  • run_container_unit_tests_verifier with env_id (choice 8): exclusive with sandbox_name/container_name, it reuses run_code's driver target. Commands run as the session user (user must be left unset), env_vars go into the run's env, and result_paths/reward_path are home-relative (an absolute one is refused) and read with data/get.
  • env_outcome_verifier (choice 9): a script whose verify takes an env_url keyword gets the env's address, which the protocol client takes as it is. The shipped examples stop stripping /mcp.

M1.1: computer apps.

  • ComputerAppEnvironment in agentenv_protocol.computer (choices 11, 13, 14): an AgentEnvEnvironment whose class declares what it is to a computer.
    • @computer_app(supports, permissions) declares what it is; its env.toml names it (environment_name) and the computer and driver it deploys alone on.
    • linux = Linux(files, python, services) is how a Linux computer installs it (feat(computer): the computer env type and the composer (container forms) #358's install format); launch = Launch(at, command, window, ready=Ready(...)); a skill as a file (skill = "skill.md") or an @agent_skill method, not both; @install_step(scope=...) and @ready_check.
    • A typed Settings (a SecretStr field is a secret), read as self.settings.
    • Launch, readiness, install steps and data hooks are declared and stored; M1.2, M1.4 and M1.5 act on them. permissions is stored, not checked.
    • The package now loads a driver's names (which need the computer extra) on first use, so an app's code imports with only the plain protocol package.
  • The computer env types share one package: the driver and app env types move into env/envs/computer/ as driver.py and app.py, beside env.py and the install, gate, composer and inside modules all three use (decided in review). The env type names don't change, and the plugin-API check reports no break.
  • _declare dispatches by base class: python -m agentenv_protocol.computer._declare reads a driver or an app, and says which; a computer_driver env naming an app's class, or the reverse, is refused.
  • The computer_app env type (env/envs/computer/app.py): its env.toml takes module, environment_name, and the computer and driver to deploy it alone on, which a bundle builds with it. put reads the class, checks its install, and stores its folder as a file universe.
  • Deploying an app alone (choice 10B): a ComputerAppEnv is a ComputerEnv, so with a computer and a driver it is checked by gate 1 (its platforms and arches against the base's and the driver's, an install for the base's OS), composed and deployed like one, as a one-app computer.
    • The composer installs it under /opt/agentenv/apps/<name>/, built like the driver's layer: its files, a venv of its requirements, and its services as supervisord programs run as the session user, with its ENVIRONMENT_NAME.
    • The deploy record is keyed by the app's env id, and its card is named after it.
    • No launch, ready checks or data plane run yet. A computer env that lists apps is still refused, until M1.3.
  • app_settings (choice 12): a new env_params key, app_settings: {<app>: {<name>: value}}, checked at preflight against the full JSON schema of each app's Settings (jsonschema, now declared; it already came with mcp), required fields included, a secret (a SecretStr, optional or not) only as a bare secret:/env: reference. At boot each reference is resolved and the values reach the app's services as one JSON variable, AGENTENV_APP_SETTINGS, keyed by app.
  • Secrets on a computer (choice 15): the computers README now says that code anywhere in a computer can read a secret setting, since everything there runs as one user; FD-4612 keeps agents off computer-exec.

Where this differs from the design

  1. How an app's code avoids the computer extra (choice 14): frames.py imports results.py, which imports mcp, so making av and PIL lazy wouldn't have been enough. Instead agentenv_protocol.computer loads the driver, card and recording names on first use, and an app's import loads only app.py. Same outcome; recording.py and frames.py are untouched.
  2. An env target's verifier takes no user (choice 8 allowed the session user's name too): leaving it unset is the one way to run as the session user, so the step needn't look the name up.
  3. The run folder is removed only on a computer: the host, sandbox and agent targets keep theirs, as before.
  4. @computer_app names no computer or driver (choice 11 had the decorator name defaults; decided in review of Greptile's finding): a bundle reads references from env.toml alone, so a computer or driver only the class named was never built. env.toml is the one place for them, as for the app's name.

Review

Greptile's review of ec434c1 raised six threads. Five are fixed in 4df9a68: the verifier on a computer stops when it can't clear the reward file; an optional SecretStr is a secret at save and at boot; an allowlist network loads inline rather than through grants it may not reach; run_code keeps the runner's 10 s inside the computer's command limit; and app_settings are validated against the full schema. The sixth, that a computer and driver named only by @computer_app can't be resolved by a bundle (which reads references from env.toml alone), is decided in 1e5def9: @computer_app no longer names them (difference 4).

Greptile's review of 4df9a68 raised three more, on secrets in that schema check, fixed in a46f63c: a secret is checked only as a reference (or null where it's optional), never against the constraints on its value, which therefore never reaches a message, and a required one left out is named. a46f63c also makes a data/add, data/get or run whose connection to the driver breaks fail naming the env, the call and the cause (the EC2 chaos run below found a bare ReadError: ), and fixes test_artifact_staging.py's import of LocalSandbox, which #374 brought in from before #369 moved it, so main's unit tier fails on it.

Testing

  • Unit: 11,507 passed, at 1e5def9; the fixture app's integration test passed again there. New tests cover the computer's loader (inline and grant parts, batching, a closed network, refused destinations), run_code on a computer (staging, loopback env, the timeout cap, a dropped call resent with its request_id while a new run gets its own, a failed script leaving no folder), the verifier's env_id target, env_url, the environment-artifact refusals, the app class and _declare, the app's bundle folder, its composition, its fit and its app_settings.
  • Integration (tst/integration/computer, laptop arm64, real Docker and the local registry), at 6ace0d5: 21 passed and 1 skipped (the agent task, which needs a model endpoint). New: the desktop example's files task loads a CSV into Documents, reads it with run_code inside the computer while asking the driver for its card on loopback, and checks it with the verifier's env_id target; a closed-network computer loads a file inline and runs code that reaches its driver; a fixture app, deployed alone through agent-env run, comes up with its card named after it and its service running from a plain-protocol venv on the settings the deploy gave it, a secret resolved at boot.
  • Chaos on the laptop (arm64), 10/10, with a harness for the design's chaos row (each scenario also tears down with nothing failed):
    • a load larger than one batch: 150 MiB in three files lands whole, one data/add per file, through read grants on an open network (0.9 s) and inline on a closed one (1.7 s);
    • the driver killed during data/add: the step fails with RemoteProtocolError: Server disconnected without sending a response; supervisord has the driver back in 3 s, a retry loads every file whole, and no .part file is left in the home;
    • the driver killed during a run_code run: the dropped call is sent again with its request_id, the restarted driver runs the script again, and the step succeeds; a retry runs and leaves no staging folder;
    • a run_code past its timeout: reported as timed out after 5 s, with no process of it left running and its folder removed.
  • On EC2 (amd64, c6i.2xlarge): at 4df9a68, tst/integration/computer 21 passed and 1 skipped, and chaos 9/10: the driver killed during data/add failed the step with ReadError: , naming no cause. a46f63c fixes that; at a46f63c, 21 passed and 1 skipped, and chaos 10/10.
  • The agent run (agent-env run desktop --task agent, laptop only, the model key held in-process), at 6ace0d5: passed in 53.4 s. load_artifact put the figures in ~/Downloads through a read grant; Claude Code, given the computer's skill, made Documents/Reports and moved the file into it through the file manager (the prompt took 40.3 s), and the check confirmed it through the driver with env_url. Two sandboxes torn down, nothing left.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 4/5

Fix the cleanup pattern before merging so one code step cannot stop another.

Fix All in CursorFindings

  1. P1 One step stops another ▶
Fix with agent prompt
### Issue 1
src/agent_env/task_step/task_steps/run_code.py:638
`reset_dir` passes the step ID straight into `pkill -f`, which treats it as a regular expression. If independent steps `read.v1` and `read-v1` run on the same computer, starting `read.v1` can kill the already-running `read-v1` runner because `.` matches `-`. The other step then fails despite using a separate folder.

Escape the path and match the runner argument exactly, or stop a tracked process group instead.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds setup and check steps inside computers, plus standalone computer app environments.

  • Task steps can load files and run scripts inside a computer.
  • Computer verifiers run commands and checks through the driver.
  • App authors can declare computer apps with the protocol package.
  • A computer app can be built and deployed alone with its settings.
  • Computer environment support moves into a shared package without changing its existing type names.

All supplied previous threads were rechecked and are addressed. None supplied a Comment number, so previousFindings is empty.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Step[Setup or check step] --> Driver[Computer driver]
    Driver --> Files[Files in the user's home]
    Driver --> Runner[Code as the session user]
    App[Computer app declaration] --> Compose[Compose app, driver and base]
    Compose --> Computer[Standalone app computer]
    Computer --> Driver
Loading

Reviews (8) · Last reviewed commit: "fix(computer): a run runs once, an app w..." · Reviewed by Greptile

Comment thread src/agent_env/task_step/task_steps/verifiers/run_container_unit_tests_verifier.py Outdated
Comment thread src/agent_env/env/envs/computer/gate.py Outdated
Comment thread src/agent_env/env/envs/computer_app.py Outdated
Comment thread src/agent_env/env/envs/computer/inside.py Outdated
Comment thread src/agent_env/task_step/task_steps/run_code.py Outdated
Comment thread src/agent_env/env/envs/computer/gate.py Outdated
Comment thread src/agent_env/env/envs/computer/gate.py Outdated
Comment thread src/agent_env/env/envs/computer/gate.py Outdated
Comment thread src/agent_env/env/envs/computer/app.py
earakely-scale and others added 5 commits October 28, 2026 18:41
…ach inside a computer

load_artifact writes files into a computer's home through its driver's data/add; run_code and
run_container_unit_tests_verifier run inside it as the session user through computer-exec/v1;
env_outcome_verifier hands scripts the env's base URL as env_url.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…app computer

ComputerAppEnvironment in agentenv_protocol.computer declares an app to a computer; a computer_app env
reads it at put through _declare, stores its folder, and deployed alone composes it onto its computer
and driver under /opt/agentenv/apps/<name>/, with env_params app_settings handed to it at boot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The unit-test verifier on a computer stops when it can't clear the reward file, so one an agent left
  behind a read-only folder isn't graded.
- An optional secret setting (SecretStr | None) is a secret too, at save and at boot.
- Loads read through grants only on a computer whose network allows everything; an allowlist loads inline.
- run_code keeps the runner's 10 s inside the computer's command limit, so a script at the limit is stopped
  by its runner.
- app_settings are checked against the full schema of each app's Settings, required fields included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l names its cause

- A secret app setting is checked only as a reference (or null, where it's optional), never against the
  constraints on its value, which its value then never reaches; a required one left out is named.
- A data/add, data/get or run whose connection to the driver breaks fails naming the env, the call and
  the cause, which httpx often leaves empty (found by the chaos run on EC2: a bare "ReadError: ").
- test_artifact_staging imports LocalSandbox from local_docker, where #369 moved it; main's unit tier has
  failed on it since #374.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale force-pushed the edgararakelyan/fd-4366-m09-load_artifact-and-run_code-inside-the-computer branch from 4df9a68 to a46f63c Compare October 29, 2026 01:47
…does, whoever runs the tests

CodeBuild runs the unit tier as root, which a read-only folder doesn't stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Collaborator Author

EC2 rerun at a46f63c (amd64, c6i.2xlarge): tst/integration/computer 21 passed and 1 skipped (the agent task, which needs a model endpoint); chaos 10/10. The driver killed during data/add now fails the step with "env '@local/agentenv-framework/desktop/desktop': the connection to its driver broke during data/add, so some of the files it carried may be written, each whole (…)", and a retry loads every file whole. c01da9e makes the new reward-file test independent of who runs it (CodeBuild runs the unit tier as root, which a read-only folder doesn't stop).

earakely-scale and others added 4 commits October 28, 2026 20:58
….toml alone

@computer_app declares what an app is (supports, permissions); its env.toml names it and the computer and
driver it deploys alone on, which a bundle builds with it. A name only the class gave was never built,
since a bundle reads references from env.toml (decided in review of Greptile's finding).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…puter/

computer_driver.py and computer_app.py move into the computer package as driver.py and app.py, beside
env.py and the install, gate, composer and inside modules all three share. The env types keep their names;
the plugin-API check reports no break.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eaving Env as it was

load_artifact refuses an environment artifact aimed at an env whose type has no loader for it, at save and
before it loads, instead of Env growing two methods that only refuse (decided in review). The CLI's hasattr
checks are main's again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… deploy, and old workers read user

- A computer-exec call whose connection breaks fails naming why and is never sent again: a driver that
  restarted would run it a second time beside the first, which outlives the driver that started it.
  run_code's staging ends whatever still runs from the step's folder first, so a retry runs once.
- The computer provider refuses an app that names no computer with its own refusal, not AttributeError.
- run_container_unit_tests_verifier writes user only when it's set, so a release before env_id targets
  still reads a container step's user as root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

async def reset_dir(self, path: str) -> None:
"""End whatever still runs from the folder, as a run whose driver died leaves going, then remove it."""
await self.run("pkill", "-f", f"{path}/runner.py")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 One step stops another

reset_dir passes the step ID straight into pkill -f, which treats it as a regular expression. If independent steps read.v1 and read-v1 run on the same computer, starting read.v1 can kill the already-running read-v1 runner because . matches -. The other step then fails despite using a separate folder.

Escape the path and match the runner argument exactly, or stop a tracked process group instead.

Knowledge Base Used: Task-step context and execution

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/task_step/task_steps/run_code.py
Line: 638

Comment:
**One step stops another**

`reset_dir` passes the step ID straight into `pkill -f`, which treats it as a regular expression. If independent steps `read.v1` and `read-v1` run on the same computer, starting `read.v1` can kill the already-running `read-v1` runner because `.` matches `-`. The other step then fails despite using a separate folder.

Escape the path and match the runner argument exactly, or stop a tracked process group instead.

**Knowledge Base Used:** [Task-step context and execution](https://app.greptile.com/scale-ai/-/custom-context/knowledge-base/scaleapi/agentenv-framework/-/docs/task-step-execution.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant