[earakely-scale]

What and why

OpenEnv's review of the agentenv importer (openenv/OpenEnv#1493) checked AgentEnvEnvironment against RFC 015's contract for imported envs, and found two gaps:

• A step before reset. RFC 015 has an imported env raise on a tool call before reset(), so an agent can't start an episode. OpenEnv's ORS wrapper raises RuntimeError("Call reset() before invoking ORS tools"). AgentEnvEnvironment answered with an ended error observation instead.
  • It now raises RuntimeError, for either action, from step_async and from the worker-thread step. Over OpenEnv's WebSocket the client gets an error message, and the session goes on.
  • After a reset, nothing changes: once an episode has ended, or a reset was refused, a step still answers with the ended observation.
• An end_turn the proxy refuses. Turn.end raises EpisodeError when the proxy no longer has the turn (a 404) or refuses the reply. _end_turn called it outside its error handler, so the error escaped step. It's inside the handler now, so the episode ends in an error observation and is torn down, like every other failure.

Teardown was already safe. Closing a turn's tools does nothing that can fail, and Episode.abandon waits for the run with asyncio.wait, which doesn't re-raise the run's error. A new test pins this down: a teardown that fails still answers with an observation and frees the session. The observation is unscored, because the run's result reports the teardown error.

Also, a flaky test. test_a_background_apps_output_is_drained_after_the_call_returns (protocol package, computer extension) failed in this PR's first CI run, on both the unit job and CodeBuild.

• It slept 1 s and then expected the drain set to be empty. A loaded runner can take longer to start the background child and finish its sleep 0.4.
• It now polls until the drain set is empty, failing after 10 s.
• Nothing else in this PR touches that code.

How it was tested

• pytest tst/unit/openenv tst/unit/test_optional_extras_stay_optional.py: 69 passed.
  • New tests: a step before reset (both actions, on the event loop and from a worker thread, and over OpenEnv's WebSocket in app_test), the refused end_turn, and the failing teardown.
  • Against the old adapter, the five tests that cover the change fail.
• pytest tst/integration/openenv on local Docker (macOS): 6 passed. The seventh failed building the echo agent image for linux/amd64 on this arm64 Mac, before the adapter runs; CI runs it on Linux.
• pytest packages/agentenv-protocol/tests -n 12, three times: 801 passed each time.
• ruff check passes.

🤖 Generated with Claude Code

Confidence Score: 4/5

The timing settings need descriptive names to satisfy the repository rule before merging.

Findings

1.  Test timing settings lack names ▶

Fix with agent prompt

### Issue 1
packages/agentenv-protocol/tests/test_computer_extensions.py:170-172
The drain test uses an inline polling delay of `0.05` and timeout of `10`. The repository requires magic numbers to be stored as class or instance variables with descriptive names. Name these settings so readers can tell what each controls. This repository requirement must be satisfied before merging.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR makes OpenEnv reject steps before the first reset and return an ended error observation when end_turn fails.

• OpenEnv sessions now reject steps until their first reset.
• Refused end-turn replies now end as error observations.
• The computer-output drain test waits for background output to finish.

_{Reviews (3) · Last reviewed commit: "test(protocol): bound the drain wait wit..." · Reviewed by Greptile}

[greptile-apps]

Test timing settings lack names

The drain test uses an inline polling delay of 0.05 and timeout of 10. The repository requires magic numbers to be stored as class or instance variables with descriptive names. Name these settings so readers can tell what each controls. This repository requirement must be satisfied before merging.

Rule Used: Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. (source)

Learned From
scaleapi/scaleapi#126388

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/agentenv-protocol/tests/test_computer_extensions.py
Line: 170-172

Comment:
**Test timing settings lack names**

The drain test uses an inline polling delay of `0.05` and timeout of `10`. The repository requires magic numbers to be stored as class or instance variables with descriptive names. Name these settings so readers can tell what each controls. This repository requirement must be satisfied before merging.

**Rule Used:** Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. ([source](https://app.greptile.com/scale-ai/-/custom-context?memory=002e0051-41ad-46c1-9098-47433c580150))

**Learned From**
[scaleapi/scaleapi#126388](https://github.com/scaleapi/scaleapi/pull/126388)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[earakely-scale]

Leaving these inline. This repo writes test timings inline, as this test already does two lines above (DRAIN_S patched to 0.1, the sleep 0.4) and as the package's other tests do (wait_for(..., timeout=1) in test_a2a_agent.py). The rule cited comes from another repository.

[greptile-apps]

You’re right. This package uses inline timing values, and the 0.05 polling interval and 10-second bound are consistent with that convention. I applied a shared magic-number instruction as though it were a requirement for this repo. I’m withdrawing the finding; no change is needed.