Skip to content

feat(openenv): serve every bundle task as an OpenEnv env - #230

Merged
earakely-scale merged 19 commits into
mainfrom
feat/openenv
Oct 26, 2026
Merged

earakely-scale merged 19 commits into
mainfrom
feat/openenv

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Every task in a bundle is now an OpenEnv env. agent_env.openenv.build_app(bundle) returns the OpenEnv server for a bundle's tasks. Any OpenEnv client (OpenEnv's own GenericEnvClient and MCPToolClient, verifiers, a TRL wrapper) resets an episode, acts through the env's MCP tools and gets the task's verifier score as its reward, with no per-task code.

from agent_env.openenv import build_app

app = build_app("portsim")  # a folder or an installed bundle, or {"train": "dock-v1-train/", "eval": "dock-v1-eval/"}

openenv>=0.9,<0.11 becomes a hard dependency, the way mcp and a2a-sdk are. That range is OpenEnv's two-minor deprecation window, moved up when OpenEnv 0.10.0 came out on 2026-10-22. Only agent_env/openenv/ imports it, so no other command pays the cost of loading it.

Two fixes outside openenv/ came out of testing on PortSim's packs:

  • Materializing a large bundle no longer crashes. On main, agent-env run crashes with EMFILE on a bundle of more than about 1,000 entries (Linux) or 250 (macOS), such as PortSim's generated 1,050-task pack.
  • The explorer extra is retired. OpenEnv's fastapi makes it install nothing core lacks.

Merged onto main after #270 (2026-10-26)

Main's Episode API now plays an outside agent through a ProxyAgent (#270): env tools are called by a proxy, served in this process for a task whose agent works in no sandbox. The adapter rides on it unchanged in shape:

  • Tool errors: an env tool's failure reaches the adapter as the proxy's error result, so it's an execution_error with the proxy's text (error: the tool call timed out after 600s, error: ConnectError: …). A timeout or lost connection between the adapter and the proxy is still timeout or transport_error, and an unknown tool tool_not_found.
  • EpisodeSpec.prompts reads each outside turn's system prompt as its agent gets it: from its deploy, or the last prompt that set one.
  • A cancelled episode is still recorded in its task's run scope, so an @local task's lands where its instance is.
  • Bundle preparation keeps main's --agent, --user-model and --judge-model overrides beside the played-agent skip, and one helper (sandbox_overrides) builds the sandbox overrides for agent-env run --sandbox and EpisodeSpec.start(sandbox=).
  • Docs: main's README now points at the docs site, so the README's OpenEnv section is one paragraph; the website page is a follow-up.
  • Sandbox tasks: the Episode API no longer refuses a task whose agent works in a sandbox, so build_app serves those tasks too: a proxy is deployed in the agent's place, and bash, first in the tools, runs as the agent in its folder. A new integration test covers it, and the chaos below runs it.
  • A reset cancelled while the last episode tears down (Greptile) now leaves that teardown tracked for the server's shutdown, and the next reset waits for it.

How a task maps onto OpenEnv

OpenEnv AgentEnv
Task API (/splits, /tasks, /task) The bundle's tasks: all, then a split per eval, or a mapping's keys. Each task listing carries {id, name, index, prompt, system_prompt, agent}, read without deploying.
reset(split, index) Starts an episode (EpisodeSpec) and returns once its first turn is open (deploy and setup done). The observation carries the prompt and system prompt. With neither argument, it plays the first task served, as OpenEnv's ORS and verifiers importers do.
step(ListToolsAction / CallToolAction) The turn's MCP tools, called as the agent's role. A result has MCP's own shape, as MCPEnvironment's does.
end_turn(response) (an added tool) Hands the reply back: either the next turn's prompt, or, after the last turn, the reward with done=True and every score in metadata["scores"].
One session One episode, which is one deployment. SUPPORTS_CONCURRENT_SESSIONS, and max_concurrent_envs (default 8) caps them.

The choices follow OpenEnv's own envs:

  • Reset waits for the deploy, as jupyter_env and terminus_env create their sandbox in reset.
  • A reset on a live session abandons the previous episode and waits for its teardown, and so does close().
  • Calls on one session take turns. OpenEnv's HTTP MCP sessions and its web playground can call one env concurrently, so a step waits for a reset or end_turn in flight.
  • Failures:
    • A failure that ends an episode is an observation: done, no reward, and metadata["status"] == "error", as terminus_env reports a sandbox that can't start.
    • A failed tool call is a ToolError, and the episode goes on.
    • A call cancelled while reset or end_turn waited for the turn to open ends the episode at the next step, in the same error shape.
  • Tools named reset, step, state, close or end_turn are refused, as MCPEnvironment refuses them.
  • Reward: a task that records one score has it as its reward. Several scores need reward=<step id> or a function of the scores, checked over every task at startup.
  • Startup prepares each bundle exactly as agent-env run does before its first task, except that the agent the client plays is never deployed, so it needn't exist.
  • Namespace routing is on from each startup of the app to its shutdown, so @local ids resolve while it serves.
  • Shutdown: the app's shutdown closes every open session and waits. OpenEnv's own shutdown doesn't, so without this, local Docker containers would outlive the server.

Changes outside agent_env/openenv/

  • agent_env.bundle.prepare_bundle(..., played=False): run_bundle's work before its first task, moved out unchanged, so agent-env run behaves exactly as before.
    • played=True drops the references of each task's only prompted agent's deploy_agent steps, keeping its envs, and preflight skips those deploys.
    • locate_bundle moves from cli/run.py into bundle/installed.py.
  • Episode API:
    • EpisodeSpec.start(sandbox=) sets user_overrides the way run_bundle does, through a shared task_step.context.sandbox_overrides.
    • EpisodeSpec.prompts gives each outside turn's prompt and system prompt before anything runs.
    • Episode.finished is added.
  • Episode API fix (a bug already on main): an abandoned episode's cancelled mark is now written in the task's run scope, as run_bundle's _mark_cancelled does. Before, an @local task's abandoned episode stayed running in the local store, because the mark went to the configured store.
  • holding_locks raises the soft open-file limit to fit its locks, as Go's runtime does.
    • On main, materializing a bundle holds one open lock file per entry. A bundle of more entries than the limit allows therefore crashed agent-env run with OSError: [Errno 24] Too many open files.
    • Reproduced with main's agent-env run on PortSim's generated dock-v1-train (1,050 tasks, Linux default limit 1024) and dock-v1-train-standard (266 tasks at macOS's default of 256).
    • The per-id locking is unchanged; lock striping would have made unrelated bundles wait on each other.
  • The explorer extra is gone.
    • It installed fastapi and uvicorn, which core now installs (uvicorn already was), so the CI check that every extra installs something core lacks failed.
    • agent-env up drops its check for them, and the README and AGENTS.md drop the extra.
    • pip install 'agentenv-framework[explorer]' still works, with pip's warning that the extra doesn't exist.
  • The installer tier's offline index names each project as PEP 503 normalizes it. jaraco.classes (needed through openenv's keyring) ships a wheel whose filename keeps the dot, so uv found no jaraco-classes.
  • THIRD_PARTY_NOTICES.md: lists the runtime dependencies uv.lock resolves for core and its documented extras, without explorer: main's entries, the new packages, and the versions that moved (openenv 0.10.0, hf-xet, huggingface_hub, pandas, brotli), and PyJWT and tensorlake, which already disagreed with main's lock.
  • Docs: a README paragraph and an AGENTS.md row.

Notes for review

  • A planning or preflight problem stops startup with agent-env run's own error. Only the tasks the Episode API refuses are left out (and logged), since planning attributes problems to the bundle, not to individual tasks. So a task the Episode API would refuse (say, one prompting two agents) must still pass agent-env run's checks: its agents must exist.
  • OpenEnv's stateless HTTP POST /reset deploys an episode and tears it down at once, as it does for every OpenEnv env that sets up in reset. Real episodes go over /ws sessions, which max_concurrent_envs caps.
  • A bundle name with a . is served with - (dock.v1 at /dock-v1/...). OpenEnv's web interface reads the env name as a module path, so a dotted name failed create_app under ENABLE_WEB_INTERFACE, which openenv push sets in a Space.
  • An eval named all is refused, since it would hide the split that holds every task.
  • OpenEnv's /mcp endpoint answers a plain Environment's tools/call with the whole observation, not the tool's data. That's true for every plain-Environment env, including the ORS importer's. So MCPToolClient.call_tool returns the observation for these envs.
  • Install footprint: OpenEnv brings gradio, which it uses only for its web UI, and pandas, numpy and Pillow through it: about 170 MB.

Verification

Tests: unit tests in tst/unit/openenv/ (25), plus extensions to the Episode API, bundle, ledger and installed-bundle tests. Integration tests in tst/integration/openenv/ (6) drive build_app on local Docker through OpenEnv's own GenericEnvClient and MCPToolClient; they passed twice on a Linux box.

After merging main (head 6d7fe038, macOS, local Docker):

  • Unit: 10,542 passed, 64 skipped; ruff clean. The OpenEnv unit tests (26) pass three runs in a row.
  • tst/integration/openenv/: 5 of 6 pass, twice, on OpenEnv 0.10.0. The sixth builds the echo agent for linux/amd64, which this Mac's Docker can't without buildx; CI runs it.

After the merge, on a Linux EC2 box (c6i.8xlarge, local Docker, the head with OpenEnv 0.10.0 and agentenv-portsim 0.5.5):

  • PortSim proof:
    • Startup: build_app({"train": dock-v1-train, "eval": dock-v1-eval}) serves 1,050 + 50 tasks in 7.3 s cold and 2.0 s warm.
    • Rewards: through GenericEnvClient, the optimal, naive and empty plans of the first standard train week score 1.0, 0.238456 and 0.0, exactly what the same plays score through the Episode API in process.
    • The eval split plays too.
    • Every record is completed, and nothing is left running.
  • Chaos on PortSim (the standard pack, the server in its own process). Every scenario passes:
Scenario Result
Steady load, optimal plans, 2 rounds each, then 64 episodes back to back 8 sessions in 7–9 s, 16 in 10–11 s, 32 in 18 s; all 176 rewards exactly the in-process scores; every record completed; nothing left
8 clients SIGKILLed mid-episode All torn down within 1.0 s, recorded cancelled
5 resets on one live session 4 cancelled, 1 completed; never more than one deployment up
The env's container killed mid-episode The tool call is an execution_error (error: ConnectError: …); end_turn is then the error observation
Server SIGTERM with 6 open episodes All 6 cancelled and torn down; stopped in 0.8 s
Server SIGKILL with 4 open episodes 4 left behind, as expected with no hook; removed
5 sessions against max_concurrent_envs=3 3 open, 2 refused with CAPACITY_REACHED
A client timeout shorter than the deploy The server tears the episode down within 2.4 s
In-process proxies, 4 batches of 32 concurrent episodes The server's threads hold at 34 after every batch (its asyncio pool at its cap); no proxy thread outlives its episode
  • Chaos on a sandbox task (the items env, a file loaded into the agent, verify_sandbox). Every scenario passes:
Scenario Result
2 rounds of 8 concurrent sessions All 16 score 1.0; bash runs as agent in /app and reads the loaded file; every record completed
The env's container killed Its tools fail as execution_error, bash still works, and the task still scores 1.0
The proxy's container killed bash is a transport_error; end_turn is the error observation once prompt_agent gives up on the agent (about 60 s)
4 clients SIGKILLed Each episode's proxy and env torn down within 0.9 s, recorded cancelled
Server SIGTERM with 4 open All cancelled and torn down
  • tst/integration/openenv/ passes 7 of 7 on the box, and server_env_local_test.py (the Episode API's container tests) passes too.
  • The box is terminated.

The reset and close chaos again, on the final head 11695a4c (a fresh box, after the cancelled-reset fix). Every scenario passes, two of them new:

Scenario Result
A client killed while its second reset tears down its first episode, at 0, 0.2, 0.5, 1 and 2 s The first episode recorded cancelled every time; never more than one deployment up; nothing left
The same kill, then the server SIGTERMed at once, at 0, 0.3 and 1 s Shutdown waits for the teardown in flight (2.2–3.2 s); nothing left; the first episode recorded cancelled
5 resets on one session; 8 SIGKILLed clients; a short client timeout; a killed env container; capacity; SIGTERM and SIGKILL Same results as above
Sandbox tasks: steady, killed env or proxy container, SIGKILLed clients, SIGTERM Same results as above

PortSim proof, before the merge (agentenv-portsim 0.5.5, a Linux EC2 box, local Docker):

  • Startup: build_app({"train": dock-v1-train, "eval": dock-v1-eval}) serves 1,050 + 50 tasks after 11 s cold, 5 s warm.
  • Rewards match exactly: the optimal, naive and empty plans, replayed through GenericEnvClient, score the same as the same episode run through the Episode API in process: 1.0, 0.200181, 0.0.
  • Eval split: served too.
  • Cleanup: every record ends terminal, and no container is left.

Chaos, before the merge (dock-v1-train-standard, 266 tasks, c6i.8xlarge). All 21 checks pass:

Scenario Result
Steady load, known-good plans, 2 rounds each 8 sessions in 5.0–5.2 s, 16 in 7.7–7.8 s, 32 in 13.5–14.1 s. Every reward exact, every record terminal, no container left.
8 clients SIGKILLed while holding episodes All 8 torn down within 2.1 s.
5 resets on one live session (TRL's pattern) The first four recorded cancelled, the last completed. Never more than one deployment.
The env's container killed mid-episode The tool call is a transport_error ToolError (since the merge, an execution_error: the proxy makes the call). end_turn is then the error observation (done, no reward, status: error).
Server SIGTERM with 6 open episodes All 6 recorded cancelled and torn down; stopped in 0.8 s.
Server SIGKILL with 4 open episodes 4 left behind, as expected with no hook to run; measured and removed.
5 sessions against max_concurrent_envs=3 2 refused with OpenEnv's CAPACITY_REACHED.
A client timeout shorter than the deploy The client times out; the server tears the episode down.

OpenEnv's own tools against the PortSim server:

  • openenv validate --url ... --level runtime: PASS, 6 of 6 criteria.
  • The web playground (ENABLE_WEB_INTERFACE=true): the page serves, and an eval task resets with its four tools; end_turn scores it.

Regression (main 705a178 and this branch on the same Linux box):

Suite main this branch
make unit-test 8,340 passed, 7 skipped, 1 failed 8,368 passed (+28 new), 7 skipped, the same 1 failed
make int-test-fast 167 passed 172 passed (+5 new)
make int-test-slow 66 passed, 490 skipped 67 passed (+1 new), 490 skipped
  • The unit failure is the same on both: run_docker_container_test.py::test_a_context_zip_from_a_non_s3_store_unpacks_in_a_real_local_sandbox.
  • Flaky CI test: local_sandbox_test.py::test_teardown_stops_what_a_command_left_running_and_only_that failed once in this PR's CI. It failed the same way on another PR's CI today, and passed in both full runs on the box.

Follow-ups (separate PRs)

  • Website: the getting-started page drops the explorer extra once this is released, and a page on serving tasks as OpenEnv envs is added.
  • Hub Spaces: publish a bundle's OpenEnv Space from agent-env hf publish, with envs deployed to a remote sandbox.
  • Trainers: a trainer-side adapter for TRL.
  • Upstream OpenEnv asks:
    • discovery of installed AgentEnv bundles through AutoEnv;
    • gradio as an optional extra;
    • closing open sessions on server shutdown.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge, with no new actionable issues found.

Summary

Adds build_app(bundle) to serve bundle tasks through OpenEnv clients, with task splits, tools, rewards, and session cleanup.

  • OpenEnv clients can play bundle tasks and receive their verifier scores.
  • Bundle preparation can leave out the agent played by an outside client.
  • Episodes expose prompts and sandbox settings for outside agents.
  • Large bundles can hold all their entry locks without hitting the file limit.
  • OpenEnv is a core dependency, and agent-env up no longer needs an explorer extra.

Diagram

sequenceDiagram
  participant Client as OpenEnv client
  participant App as AgentEnvEnvironment
  participant Episode as Episode API
  participant Proxy as Agent proxy
  Client->>App: reset(split, index)
  App->>Episode: Start task and await turn
  Episode-->>App: Prompt and tools
  App-->>Client: Observation
  Client->>App: Call tool or bash
  App->>Proxy: Run action
  Proxy-->>Client: Tool result through app
  Client->>App: end_turn(response)
  App->>Episode: End turn and await next
  Episode-->>App: Next turn or scores
  App-->>Client: Next prompt or final reward
  Note over App,Episode: Reset, close, and shutdown wait for teardown
Loading

Reviews (7) · Last reviewed commit: "test(openenv): a task whose agent works ..." · Reviewed by Greptile

earakely-scale and others added 9 commits October 15, 2026 16:56
build_app(bundle) returns the OpenEnv server for a bundle's tasks: each session plays one episode
through the Episode API, with the env's MCP tools plus an end_turn tool, and the task's verifier score
as the reward. Bundles are prepared as agent-env run prepares them, without the agent a client plays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…onment's

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e's loop

The playground calls the sync step in a worker thread; the episode's tool
sessions belong to the loop its reset ran on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e app's lifespan

OpenEnv's HTTP MCP sessions and web playground can call one env concurrently; a step
during a reset or end_turn dereferenced the turn's unopened tools, and a close racing a
turn's opening left its tool sessions open. Reset, step and close now take turns.

Routing is on from each startup to its shutdown, so a failed create_app no longer leaves
it on and a second run of the app has it. Reset defaults to the first split's first task
as OpenEnv's importers do, with all listed first; an eval named all is refused; a Path is
always a folder; a dotted bundle name is served dashed, since OpenEnv's web interface
reads the env name as a module path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The extra installed fastapi and uvicorn, which core now installs, so agent-env up no longer
checks for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e limit allows

Each entry's lock holds a file open, so agent-env run crashed with EMFILE on a bundle of
over about 1000 entries on Linux, or 250 on macOS, such as a generated task pack. The soft
limit is raised toward the hard one to fit them, as Go's runtime does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@socket-security

socket-security Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​openenv@​0.10.073100100100100

View full report

@socket-security

socket-security Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Priority Alert  (click "▶" to expand/collapse) Action
Critical priority
Critical CVE: Pillow: Heap buffer overflow in _tiffWriteProc when saving TIFF to BytesIO

CVE: GHSA-332p-7hrc-92wf Pillow: Heap buffer overflow in _tiffWriteProc when saving TIFF to BytesIO (CRITICAL)

Affected versions: >= 2.0.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a critical CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known critical CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: PNG eXIf chunk makes EXIF parsing run in quadratic time

CVE: GHSA-g4qv-2jxx-769h Pillow: PNG eXIf chunk makes EXIF parsing run in quadratic time (HIGH)

Affected versions: >= 11.0.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: Heap buffer overflow writing JPEG comment after ICC profile fills buffer

CVE: GHSA-g7cr-3p24-7wf7 Pillow: Heap buffer overflow writing JPEG comment after ICC profile fills buffer (HIGH)

Affected versions: >= 9.4.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: APNG background disposal allocates full canvas before decompression bomb check

CVE: GHSA-5xm2-6676-qcp9 Pillow: APNG background disposal allocates full canvas before decompression bomb check (HIGH)

Affected versions: >= 8.1.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: Memory disclosure and segfault in libtiff encoder via crafted TIFF tags

CVE: GHSA-xvh5-g6gv-559v Pillow: Memory disclosure and segfault in libtiff encoder via crafted TIFF tags (HIGH)

Affected versions: < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: NULL pointer dereference in Image.merge via an unvalidated band 0

CVE: GHSA-cp9v-662q-vvx9 Pillow: NULL pointer dereference in Image.merge via an unvalidated band 0 (HIGH)

Affected versions: >= 12.0.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: Infinite loop in MIC n_frames when the embedded TIFF

CVE: GHSA-22hg-r758-rrw9 Pillow: Infinite loop in MIC n_frames when the embedded TIFF (HIGH)

Affected versions: >= 7.2.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: OOB read in TiffDecode.c tile and RGBA paths leaks heap memory into pixels

CVE: GHSA-c6hw-5grf-ff96 Pillow: OOB read in TiffDecode.c tile and RGBA paths leaks heap memory into pixels (HIGH)

Affected versions: >= 8.1.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: TIFF encoder ColorMap plane-size mismatch causes a heap out-of-bounds read

CVE: GHSA-pchr-8j38-3x4p Pillow: TIFF encoder ColorMap plane-size mismatch causes a heap out-of-bounds read (HIGH)

Affected versions: < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: TiffDecode returns uninitialized memory as pixels for short JPEG strips

CVE: GHSA-q45j-58j2-45rg Pillow: TiffDecode returns uninitialized memory as pixels for short JPEG strips (HIGH)

Affected versions: >= 5.3.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
High priority
High CVE: Pillow: GIF seek keeps rejected size after DecompressionBombError, allowing DoS

CVE: GHSA-65qm-jvmh-6pqg Pillow: GIF seek keeps rejected size after DecompressionBombError, allowing DoS (HIGH)

Affected versions: >= 9.2.0 < 13.0.0

Patched version: 13.0.0

From: uv.lock → pypi/openenv@0.10.0 → pypi/pillow@12.3.0

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/pillow@12.3.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi gradio with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/gradio@6.30.0

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/gradio@6.30.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: pypi numpy with risk level "medium"

Location: Package overview

From: uv.lock → pypi/openenv@0.10.0 → pypi/numpy@2.4.6

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/numpy@2.4.6. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
See 63 more rows in the dashboard

View full report

Comment thread src/agent_env/openenv/environment.py Resolved
Comment thread src/agent_env/openenv/environment.py Resolved
earakely-scale and others added 8 commits October 15, 2026 18:47
…set skips an empty split

A call cancelled while end_turn or reset waited for the next turn left a live episode with
no open tools, so the next step dereferenced None; it now ends the episode as an error
observation. A bare reset plays the first task served even when the first split has none,
and the validator test asserts that OpenEnv's validation passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…alizes it

jaraco.classes, which openenv's keyring needs, ships a wheel whose filename keeps the dot,
so its index page was jaraco.classes and uv found no jaraco-classes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_bundle and EpisodeSpec.start built the same three-key user_overrides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… lifecycle helpers say what they do

Startup turns reward= into one function per task (a user's function, or the score at a step),
so the episode applies it in one place. _end/_close/_ended/_ended_episode/_fail become
_abandon/_abandon_when_free/_error_observation/_why_stopped/_end_with_error, and done lives
only in the episode's State. A missing score now reads as the reward's own error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's Episode API now plays an outside agent through a proxy (#270), so the OpenEnv adapter rides on it:
- an env tool's failure reaches the adapter as the proxy's error result, an execution error with the proxy's text;
  a timeout or a lost connection between the adapter and the proxy is still its own error type;
- EpisodeSpec.prompts reads each outside turn's system prompt as its agent gets it, from its deploy or the last
  prompt that set one;
- a cancelled episode is recorded in its task's run scope, as before.

Bundle preparation keeps main's --agent, --user-model and --judge-model overrides beside the played-agent skip, and
one helper builds the sandbox overrides for agent-env run --sandbox and EpisodeSpec.start(sandbox=). The explorer
extra stays retired (README, AGENTS.md, THIRD_PARTY_NOTICES.md), and the notices list the merged lock's runtime
dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… proxy reports it

Since the episode plays its agent through a proxy, an env tool call that can't reach its env fails in the proxy, which
answers with an error result; the adapter serves it as an execution error with the proxy's text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ce 0.10.0

OpenEnv 0.10.0 came out on 2026-10-22, past the range's top, so the range and the lock move together to the two
minors OpenEnv keeps working, and the notices list 0.10.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/openenv/environment.py Resolved
earakely-scale and others added 2 commits October 26, 2026 12:09
…ves that teardown tracked

The session's abandon awaited its tracked teardown directly, so a reset cancelled during that wait cancelled the
tracked task: it left Served.ending while the episode's teardown ran on, so the server's shutdown no longer waited
for it, and the next reset could deploy before it finished. The wait is shielded, and the session keeps the
teardown until it's done, so the next reset waits for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…played through bash

Since #270 the Episode API serves a task whose agent works in its sandbox, so build_app does too. The test serves one
that loads a file into its agent and checks what the agent left: through OpenEnv's own client, bash comes first in the
tools and runs as the agent in its folder, the check scores what it left, and nothing is left running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 94321a9 into main Oct 26, 2026
16 checks passed
@earakely-scale
earakely-scale deleted the feat/openenv branch October 26, 2026 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant