Repository navigation
feat(computer): a computer serves its tools under a deploy's tool prefix (FD-4935) - #535
Merged
Merged
Conversation
…fix (FD-4935) So several computers can serve their tools side by side on one gateway, a computer's driver serves every tool under the prefix a deploy gives it (desktop_computer_screenshot), and names its tools that way wherever it names them: the card, tools/list, its skill, its extensions' descriptions, its errors and its tool-access rules. Without a prefix nothing changes. - agentenv-protocol: a handler hook, serve_tools_as, names the tools it serves after served_tools has chosen them. ComputerDriverEnvironment reads AGENTENV_TOOL_PREFIX, serves its tools under it, and rewrites its own outgoing text to the served names: each tool's own name and a computer_* family of them, as a whole word. The declaration a driver env stores keeps the tools' own names. - agent-env: ComputerEnv.deploy and the computer provider take tool_prefix, refuse one a tool's name can't hold, pass it to the driver at boot, and keep it on the record, so a reset passes it again. The env's tools field gives its agents the prefixed names. Nothing passes a prefix yet; a multi env's computers will. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oken The skill is one of agent-env's own routes, which a call without the env's control token is refused; the integration test now asks through the deploy record, which carries it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… prefix rule for the rest A driver that forwards another program's tools serves text that program wrote, where the same word can name a tool to call or a value to pass: cua-driver's run_steps lists click and double_click as values of steps[].tool. Rewriting the driver's tool names in that text broke those values, and the server instructions weren't rewritten at all. The computer now rewrites only the SDK's own computer_* names, which always mean its tools, in its text and its computer_* tools' descriptions; a driver's own and forwarded tools keep their text and schemas as written. When it serves tools other than computer_* under a prefix, its skill and its server instructions lead with one sentence saying how they are named: a tool the text calls X is <prefix>X, and names inside a tool's arguments stay as written. Also drops a local launch log the SDK tests wrote, committed by mistake. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tool's name is at most 64 characters in the model APIs that take it. Capping the prefix at 16, its separator included, keeps every prefixed name well within that: the longest computer_* tool, computer_pointer_position, is 41 under the longest prefix. The driver and the computer provider refuse a longer one, as they refuse one a tool's name can't hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…names defined once The prefix logic was six public methods on ComputerDriverEnvironment, which check_names doesn't protect, so a driver defining one of its own would have changed prefixing without a word. It is now ToolPrefix, in computer/prefix.py: the served and own names, the rewrite of the SDK's names in text, the rule sentence and the renamed tool list. The driver keeps it as _computer_prefix, which check_names reserves, and only serve_tools_as and served_instructions, the hooks the SDK calls, stay on the class. Which tools are the SDK's is now one definition, COMPUTER_TOOLS in computer/tools.py, read from the @tool marks on Tools. The prefix uses it, and agent-env imports it in place of its own scan for names starting with computer_. Also drops the local launch log the SDK tests wrote again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a served name again The skill already names the SDK's tools as served (desktop_computer_screenshot), so "a tool this text calls X is desktop_X" could be applied to them too, giving desktop_desktop_computer_screenshot. The rule now speaks only of the names this text gives without the prefix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 2 of the four in the FD-4930 + FD-4935 design note, decisions 2b and 2c. Several computers will serve their tools side by side on one gateway, which keys a tool by its name, so each computer serves its tools under a prefix a deploy gives it:
desktop_computer_screenshot,laptop_computer_screenshot. The computer names its tools that way wherever it names them. Without a prefix nothing changes, and nothing passes one yet: PR 3 has a multi env give each computer its own name.agentenv-protocol
serve_tools_as, names and describes the tools it serves, afterserved_toolshas chosen them by their own names. Registration, the card,tools/listand tool access all take the served names.ComputerDriverEnvironmentreadsAGENTENV_TOOL_PREFIXat start (tool_prefix), serves every tool under it, its forwarded app tools included, and refuses a prefix a tool's name can't hold or longer than 16 characters, its separator included, so every prefixed name stays well within the 64 characters model APIs take.ToolPrefixincomputer/prefix.py: served and own names, the rewrite of the SDK's names in text, the rule sentence and the renamed tool list. The driver keeps it as_computer_prefix, whichcheck_namesreserves, so a driver can't replace it by accident; only the hooks the SDK calls (serve_tools_as,served_instructions) are on the class. Which tools are the SDK's is one definition,COMPUTER_TOOLSincomputer/tools.py, read from the@toolmarks onTools; agent-env imports it in place of its own scan.computer_*names in the computer's text, andcomputer_*families of them, as whole words: those names always mean this computer's tools. It applies to thecomputer_*tools' descriptions, the extensions' descriptions, the skill, the server instructions and the coordinate-space rewrite of the point tools' descriptions. The two errors that name a tool (computer_key,computer_dialog) useserved_name.run_stepstakesclickas a value ofsteps[].tool). When a computer serves such tools under a prefix, its skill and its server instructions lead with one sentence: "This computer serves its tools underdesktop_: a tool this text callsXisdesktop_X, solist_windowsisdesktop_list_windows. Names inside a tool's arguments stay as written."computer_driverenv stores keeps the tools' own names, whatever the environment holds.agent-env
ComputerEnv.deployand the computer provider taketool_prefix. The provider refuses an invalid one before anything boots, passes it to the driver at boot, and keeps it on the deploy record, so a reset passes it again.toolsfield (standard,driver,both) gives its agents the prefixed names.Choices
driver_settingsfield a task can set, since the multi env is what keeps names unique.computer_*names, and state the rule for the rest (the "option a" above). An example with three cua-driver computers shows the full toolset an agent gets.Test plan
tests/test_computer_tool_prefix.py. Every tool on the card and intools/listis served under the prefix, its screenshot answers, and its own name isn't served. The stored declaration keeps the own names. The skill, extensions and notes name the servedcomputer_*tools, with nothing prefixed twice. A forwarded tool keeps its own description and schema and still runs, and the skill and the server instructions lead with the prefix rule, which a computer serving onlycomputer_*tools doesn't get. An error names the tool to use instead as served. Tool access works on the served names. The coordinate-space rewrite reaches the prefixed point tools. An invalid prefix is refused. The SDK suite passes: 1,243.test_computer_env.py, a prefixed deploy boots the driver withAGENTENV_TOOL_PREFIX, records it, passes it again at a reset, and givesstandard's tools by their prefixed names at deploy and after the reset. A deploy without a prefix gives the driver none. An invalid prefix is refused before anything boots.tst/integration/computer/tool_prefix_test.py(slow tier): the desktop example's computer deployed twice side by side, asdesktop_and aslaptop_withtools = "standard". Each serves its tools under its own prefix and takes screenshots by those names. The skill names the prefixed tools.laptop_gives its agents only itslaptop_computer_*tools, at deploy and after a reset.computer_env_test.pytest of a deployed computer's routes passed as a check on unprefixed computers.agent-env run: a task's computer, with the reference driver and with cua-driver, deployed with no prefix, serves its tools under their own names.desktop_,laptop_and none) each served its own 14 names, and a screenshot answered by its served name. The computer's container restarted with no reset: still prefixed. Killed, thenreset_env: prefixed again. The coordinate space set tonorm:desktop_computer_click's description rewritten.desktop_computer_clickdisabled, then enabled, through tool access: hidden, then listed.ubuntu1_beside one with none: all 73 of its tools served under the prefix.ubuntu1_run_stepskept cua-driver's description and schema byte for byte,clickamong its step values. Thecomputer_*descriptions named the prefixed tools. The skill led with the prefix rule, cua-driver's own text after it.ubuntu1_get_screen_sizeanswered, andubuntu1_run_stepsran amove_cursorstep. Afterreset_env: the same names, and the skill still led with the rule. The teardown left nothing.cua_driver_test,egress_hosts_test,env_gateway_provider_test) that fail the same way on main there.🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge.Summary
Computers can serve tools under a deploy-specific prefix.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A["ComputerEnv.deploy(tool_prefix)"] --> B["Provider checks prefix"] B --> C["Driver receives AGENTENV_TOOL_PREFIX"] C --> D["ToolPrefix names served tools"] B --> E["Deploy record stores prefix"] E --> F["Reset restores prefix"] F --> CReviews (9) · Last reviewed commit: "fix(computer): word the prefix rule so i..." · Reviewed by Greptile