Skip to content

feat(computer): a computer serves its tools under a deploy's tool prefix (FD-4935) - #535

Merged
earakely-scale merged 6 commits into
mainfrom
edgararakelyan/fd-4935-tool-prefix
Nov 8, 2026
Merged

earakely-scale merged 6 commits into
mainfrom
edgararakelyan/fd-4935-tool-prefix

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

PR 2 of the four in the FD-4930 + FD-4935 design note, decisions 2b and 2c. Several computers will serve their tools side by side on one gateway, which keys a tool by its name, so each computer serves its tools under a prefix a deploy gives it: desktop_computer_screenshot, laptop_computer_screenshot. The computer names its tools that way wherever it names them. Without a prefix nothing changes, and nothing passes one yet: PR 3 has a multi env give each computer its own name.

agentenv-protocol

  • A handler hook, serve_tools_as, names and describes the tools it serves, after served_tools has chosen them by their own names. Registration, the card, tools/list and tool access all take the served names.
  • ComputerDriverEnvironment reads AGENTENV_TOOL_PREFIX at start (tool_prefix), serves every tool under it, its forwarded app tools included, and refuses a prefix a tool's name can't hold or longer than 16 characters, its separator included, so every prefixed name stays well within the 64 characters model APIs take.
  • The prefix is one class, ToolPrefix in computer/prefix.py: served and own names, the rewrite of the SDK's names in text, the rule sentence and the renamed tool list. The driver keeps it as _computer_prefix, which check_names reserves, so a driver can't replace it by accident; only the hooks the SDK calls (serve_tools_as, served_instructions) are on the class. Which tools are the SDK's is one definition, COMPUTER_TOOLS in computer/tools.py, read from the @tool marks on Tools; agent-env imports it in place of its own scan.
  • Its text rewrite changes only the SDK's own computer_* names in the computer's text, and computer_* families of them, as whole words: those names always mean this computer's tools. It applies to the computer_* tools' descriptions, the extensions' descriptions, the skill, the server instructions and the coordinate-space rewrite of the point tools' descriptions. The two errors that name a tool (computer_key, computer_dialog) use served_name.
  • A driver's own and forwarded tools keep their descriptions and schemas exactly as written, since another program's text can name the same word as a tool to call or as a value to pass (cua-driver's run_steps takes click as a value of steps[].tool). When a computer serves such tools under a prefix, its skill and its server instructions lead with one sentence: "This computer serves its tools under desktop_: a tool this text calls X is desktop_X, so list_windows is desktop_list_windows. Names inside a tool's arguments stay as written."
  • The declaration a computer_driver env stores keeps the tools' own names, whatever the environment holds.

agent-env

  • ComputerEnv.deploy and the computer provider take tool_prefix. The provider refuses an invalid one before anything boots, passes it to the driver at boot, and keeps it on the deploy record, so a reset passes it again.
  • The env's tools field (standard, driver, both) gives its agents the prefixed names.

Choices

  1. The prefix is internal: the computer provider sets it, through an environment variable. It is not a driver_settings field a task can set, since the multi env is what keeps names unique.
  2. Decided with Edgar after Greptile's first review: rewrite only the SDK's computer_* names, and state the rule for the rest (the "option a" above). An example with three cua-driver computers shows the full toolset an agent gets.

Test plan

  • SDK: tests/test_computer_tool_prefix.py. Every tool on the card and in tools/list is served under the prefix, its screenshot answers, and its own name isn't served. The stored declaration keeps the own names. The skill, extensions and notes name the served computer_* tools, with nothing prefixed twice. A forwarded tool keeps its own description and schema and still runs, and the skill and the server instructions lead with the prefix rule, which a computer serving only computer_* tools doesn't get. An error names the tool to use instead as served. Tool access works on the served names. The coordinate-space rewrite reaches the prefixed point tools. An invalid prefix is refused. The SDK suite passes: 1,243.
  • agent-env: in test_computer_env.py, a prefixed deploy boots the driver with AGENTENV_TOOL_PREFIX, records it, passes it again at a reset, and gives standard's tools by their prefixed names at deploy and after the reset. A deploy without a prefix gives the driver none. An invalid prefix is refused before anything boots.
  • Integration test, tst/integration/computer/tool_prefix_test.py (slow tier): the desktop example's computer deployed twice side by side, as desktop_ and as laptop_ with tools = "standard". Each serves its tools under its own prefix and takes screenshots by those names. The skill names the prefixed tools. laptop_ gives its agents only its laptop_computer_* tools, at deploy and after a reset.
  • VM e2e and chaos on an agent-environments-admin VM (c6i.2xlarge, us-west-2), at 3d6ead9, the final commit. The VM is torn down.
    • The integration test, twice: passed both times. The existing computer_env_test.py test of a deployed computer's routes passed as a check on unprefixed computers.
    • agent-env run: a task's computer, with the reference driver and with cua-driver, deployed with no prefix, serves its tools under their own names.
    • Chaos on prefixed computers. Three deployed at once (desktop_, laptop_ and none) each served its own 14 names, and a screenshot answered by its served name. The computer's container restarted with no reset: still prefixed. Killed, then reset_env: prefixed again. The coordinate space set to norm: desktop_computer_click's description rewritten. desktop_computer_click disabled, then enabled, through tool access: hidden, then listed.
    • A cua-driver computer on Ubuntu, deployed with the prefix ubuntu1_ beside one with none: all 73 of its tools served under the prefix. ubuntu1_run_steps kept cua-driver's description and schema byte for byte, click among its step values. The computer_* descriptions named the prefixed tools. The skill led with the prefix rule, cua-driver's own text after it. ubuntu1_get_screen_size answered, and ubuntu1_run_steps ran a move_cursor step. After reset_env: the same names, and the skill still led with the rule. The teardown left nothing.
  • The local unit run on macOS also failed in provider tests whose SDKs the local venv lacks (vercel, boxd, daytona and others), and in five tests (cua_driver_test, egress_hosts_test, env_gateway_provider_test) that fail the same way on main there.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

Computers can serve tools under a deploy-specific prefix.

  • Computers serve every tool under the prefix their deploy gives them.
  • A deployed computer keeps its tool prefix after a reset.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["ComputerEnv.deploy(tool_prefix)"] --> B["Provider checks prefix"]
  B --> C["Driver receives AGENTENV_TOOL_PREFIX"]
  C --> D["ToolPrefix names served tools"]
  B --> E["Deploy record stores prefix"]
  E --> F["Reset restores prefix"]
  F --> C
Loading

Reviews (9) · Last reviewed commit: "fix(computer): word the prefix rule so i..." · Reviewed by Greptile

earakely-scale and others added 2 commits November 7, 2026 15:42
…fix (FD-4935)

So several computers can serve their tools side by side on one gateway, a computer's driver
serves every tool under the prefix a deploy gives it (desktop_computer_screenshot), and names
its tools that way wherever it names them: the card, tools/list, its skill, its extensions'
descriptions, its errors and its tool-access rules. Without a prefix nothing changes.

- agentenv-protocol: a handler hook, serve_tools_as, names the tools it serves after
  served_tools has chosen them. ComputerDriverEnvironment reads AGENTENV_TOOL_PREFIX, serves its
  tools under it, and rewrites its own outgoing text to the served names: each tool's own name
  and a computer_* family of them, as a whole word. The declaration a driver env stores keeps
  the tools' own names.
- agent-env: ComputerEnv.deploy and the computer provider take tool_prefix, refuse one a tool's
  name can't hold, pass it to the driver at boot, and keep it on the record, so a reset passes
  it again. The env's tools field gives its agents the prefixed names. Nothing passes a prefix
  yet; a multi env's computers will.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oken

The skill is one of agent-env's own routes, which a call without the env's control token
is refused; the integration test now asks through the deploy record, which carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner November 7, 2026 23:55
earakely-scale and others added 4 commits November 7, 2026 16:12
… prefix rule for the rest

A driver that forwards another program's tools serves text that program wrote, where the same
word can name a tool to call or a value to pass: cua-driver's run_steps lists click and
double_click as values of steps[].tool. Rewriting the driver's tool names in that text broke
those values, and the server instructions weren't rewritten at all.

The computer now rewrites only the SDK's own computer_* names, which always mean its tools, in
its text and its computer_* tools' descriptions; a driver's own and forwarded tools keep their
text and schemas as written. When it serves tools other than computer_* under a prefix, its
skill and its server instructions lead with one sentence saying how they are named: a tool the
text calls X is <prefix>X, and names inside a tool's arguments stay as written.

Also drops a local launch log the SDK tests wrote, committed by mistake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tool's name is at most 64 characters in the model APIs that take it. Capping the prefix at
16, its separator included, keeps every prefixed name well within that: the longest computer_*
tool, computer_pointer_position, is 41 under the longest prefix. The driver and the computer
provider refuse a longer one, as they refuse one a tool's name can't hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…names defined once

The prefix logic was six public methods on ComputerDriverEnvironment, which check_names
doesn't protect, so a driver defining one of its own would have changed prefixing without a
word. It is now ToolPrefix, in computer/prefix.py: the served and own names, the rewrite of the
SDK's names in text, the rule sentence and the renamed tool list. The driver keeps it as
_computer_prefix, which check_names reserves, and only serve_tools_as and served_instructions,
the hooks the SDK calls, stay on the class.

Which tools are the SDK's is now one definition, COMPUTER_TOOLS in computer/tools.py, read from
the @tool marks on Tools. The prefix uses it, and agent-env imports it in place of its own scan
for names starting with computer_.

Also drops the local launch log the SDK tests wrote again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a served name again

The skill already names the SDK's tools as served (desktop_computer_screenshot), so "a tool
this text calls X is desktop_X" could be applied to them too, giving
desktop_desktop_computer_screenshot. The rule now speaks only of the names this text gives
without the prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 1df8f31 into main Nov 8, 2026
37 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/fd-4935-tool-prefix branch November 8, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant