[earakely-scale]

M2.9 of the CUA env epic (FD-4345): cua-driver as a driver you can name in place of the reference driver, FD-4357. It follows the design doc CUA Env M2.9: cua-driver as a Driver, whose seven choices were all decided on 2026-10-29.

An env names driver = "cua-driver" and its computer runs trycua's cua-driver. Agents get what cua-driver's own MCP server (cua-driver mcp) serves, its instructions, tools and skill resources, beside our computer_* tools and our skill. The framework keeps the same computer/v1 contract, built on cua-driver.

What's in it

The cua-driver driver (src/agent_env/drivers/envs/cua-driver/, alias cua-driver):

• Install: cua-driver 0.38.0 from PyPI, in requirements.txt. Its wheel carries the executable for both Linux arches.

• The daemon: driver.py, run as a script, starts cua-driver serve as a session-user service once X answers, exec'ing the executable computer-ready hashes. It starts in the user's home, listens on /run/agentenv/cua-driver.sock, bypasses approvals, and has telemetry, update checks and the overlay cursor off.

• What agents see, as cua-driver mcp 0.38.0 serves it (pinned in mcp.json):

  • its instructions when they connect;
  • its 66 tools with their output schemas, beside the seven computer_* tools. Each call is forwarded unchanged, and cua-driver's reply comes back as it sent it. An env narrows the tools with [tools];
  • its 11 skill resources at skill://cua-driver/…, which its instructions point to, served by one @resource method over mcp.json's list, their text read from cua-driver mcp itself;
  • a failed call answered as it answers one, {"code": "tool_invocation_failed", "exit_code": n}, which every output schema admits.

• computer/v1: cua-driver's desktop-scope calls (XTest), made in the framework's own session:

  • get_screen_size and get_desktop_state;
  • click, with 3 buttons, up to a triple click;
  • drag, from a start to an end point;
  • scroll, type_text and press_key.

  There's no pointer or keyboard hold group. Scrolling with keys held, and chords that hold a non-modifier key, are refused as unsupported.

• computer-ready checks the executable's sha256 against the value pinned for each arch, and that the daemon lists the tools mcp.json pins. Both are checked until they pass once.

• Sessions: each caller, told apart by MCP session, and the framework run in an unnamed cua-driver session of their own, as cua-driver's own MCP server gives each client. A caller's call that names a session runs in that one. Element tokens still belong to a window, not a session, so two agents reading one window invalidate each other's tokens for it.

• Connections: each call goes on a connection of its own, as cua-driver's own MCP server sends them. A call while the daemon restarts fails as unavailable, and input that couldn't be sent is "not applied". Input whose reply is lost fails as DeliveryFailed with its outcome unknown.

• The skill leads with cua-driver's own skill pack, written offline from the executable's embedded copy, with its 8 other files beside the SKILL.md. A "This computer" section of ours follows.

The protocol SDK:

• Forwarded tools: forwarded_tools() and ForwardedTool serve tools on another server's behalf. Each keeps its own schema, annotations and output schema, its arguments pass through untouched, and its CallToolResult is returned as it is. They're on the card, chosen by served_tools and under tool access, like @tool methods. They need mcp's FastMCP.
• @resource: marks a method that serves MCP resources; an env may mark several. A fixed URI is one resource; a template such as orders://{order_id} is an MCP resource template whose parts the method takes; a template with listed= serves only the resources listed returns, each read through the method. The cua-driver driver serves cua-driver's skill files with @resource("skill://cua-driver/{file}", listed=...).
• @server_instructions: marks the plain method that returns what the env's MCP server tells an agent when it connects (the instructions of its initialize result), as @agent_skill marks its skill. It's read when the SDK builds the app; an app passed to mount keeps its own.
• Skill files: @agent_skill may return a skill's other files as a third value, and skill/v1's get returns them as files.
• ComputerDriverEnvironment.bundled_skill(): the skill a driver's software ships. The computer skill leads with it.
• Keys in the skill: the computer skill says "any printable ASCII character" once, instead of listing 94 keys. PRINTABLE_KEYS is exported from agentenv_protocol.computer for drivers.

agent-env:

• add_skills sends an env skill that has files as a bundle, stored once per content under agentenv-env-skills/<digest>/ in the object store.
• The drivers bundle gains the cua-driver alias. A test bundle, tst/integration/computer/cua-driver/, names it.

Changed while building

1. Skills carry files. cua-driver's skill pack is nine linked files, which the one-file skill/v1 couldn't carry.
2. The overlay setting moved to M2.1, its first user. ComputerSettings is closed, and nothing uses the setting before the bench.
3. The daemon starts in the session user's home. Apps cua-driver launches inherit its working directory: Thunar first opened in the driver's install folder.
4. Each call on a connection of its own. Chaos found the first call after a daemon restart failing on a kept connection. A resend of reads fixed it first; one connection per call, as cua-driver's own MCP server does, replaced that.
5. A tool argument named name (launch_app's) collided with the forwarding method's own parameter. The tool name is positional-only now.
6. No fallback per operation. The probe passes at full on cua-driver alone.
7. A cua-driver session per caller (Greptile). All agents shared one session, which owns screenshot frames, the agent cursor, config overrides and recordings.
8. The framework's session is unnamed. cua-driver ends a named session after five idle minutes and refuses every later call in it. The framework's calls ran in the named session agentenv, so computer_* failed after five idle minutes. An unnamed session is started again on its next call.
9. Review fixes (Greptile): a cancelled request can't leave its reply for the next one; lost input replies are DeliveryFailed, outcome unknown; an env skill's files must stay in its folder; holding a key that isn't a modifier is refused as unsupported.
10. daemon.sh is gone. driver.py starts the daemon, so the socket and the no-telemetry settings are defined once. The drivers README has the steps for moving to another cua-driver version.
11. What cua-driver mcp sends besides tools. A side-by-side check against cua-driver mcp found three gaps: its instructions, its tools' output schemas and its skill resources. All three are served now. mcp.json replaces tools.json. The resources' text is read from cua-driver mcp rather than pinned, since that would add 266 KB of markdown to every install. A separate check showed the Linux pack mcp-config writes leaves two of the 11 out.

Testing

cua-driver ran only on two test VMs in agent-environments-admin: c6i.2xlarge (amd64) and c7g.2xlarge (arm64), each probing its own arch.

• Unit: tst/unit and the protocol tests, 11,800 passed and 66 skipped, with one Hermes test failing on macOS only, since it reads /proc/self/fd.
• New tests:
  • forwarded tools: their own schema, arguments as they came, annotations, a name clash, the refusal on the standalone fastmcp, and tool access by role;
  • skill files, a driver's bundled skill, and add_skills' stored bundle;
  • the driver against a fake daemon on a Unix socket:
    • the operations' desktop-scope calls, and refusals;
    • forwarding in the caller's own session, including an argument named name, and two callers in sessions of their own;
    • the pinned readiness checks;
    • calls while the daemon restarts, lost input replies and a cancelled request;
    • the skill pack written offline.
• Gate 3 at put: the probe at full passed 47 checks, with 1 not applicable, on both arches.
• Integration:
  • amd64: the whole computer tier gave 23 passed and 1 allowed skip, including the reference driver on the changed SDK.
  • arm64: the cua-driver tests passed 2 of 2.
  • An agent's session launched Thunar with launch_app, read its tree with get_window_state, and clicked "Open Parent" by element token.
  • computer-ready reported "66 tools, as pinned", and the skill came with its 8 files.
  • A second agent doesn't see the session the first started.
• Chaos, 8 of 8 on each arch, then 13 of 13 on arm64 with sessions per caller:
  • The daemon killed mid-run: it's back in 0.5 s, calls work again, and a token from before the restart is refused as stale.
  • The framework capturing between an agent's read and its click: the agent's token stays valid.
  • No network: the computer is ready, the tools answer, and the daemon's log shows nothing trying to reach out.
  • Every session idle for 330 s: the framework's calls and the agent's still work. Before the fix, every computer_* call failed with "session has ended".
  • Two agents: a session one starts isn't seen by the other. Before the fix, it wasn't kept apart.
• The cleanup round (one connection per call, the daemon from driver.py, the shared keys, the skill's opening): a smoke test on amd64, with the cua-driver tests 2 of 2 and the daemon killed mid-run 3 of 3.
• Against cua-driver mcp, on one amd64 computer: the same instructions; all 66 tools the same in description, inputSchema, annotations and outputSchema; the same 11 resources with the same text; and the same answer to a call in an ended session. Ours adds only the seven computer_* tools. The cua-driver tests passed 2 of 2, now also checking the instructions and a resource read.
• Not run yet:
  • an agent run with Claude Code, which needs the model endpoint;
  • Modal VM.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no new actionable issue was established.

Summary

Adds cua-driver as a computer driver while keeping the computer/v1 contract.

• Linux X11 environments can use cua-driver for computer control.
• Environment MCP servers can forward tools, instructions, and resources.
• Computer and environment skills can include companion files.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Agent[Agent MCP session] --> SDK[Protocol SDK]
  SDK --> Forwarded[cua-driver tools]
  SDK --> Computer[computer_* tools]
  SDK --> Resources[Skill resources]
  Forwarded --> Caller[Caller-specific session]
  Computer --> Framework[Framework session]
  Caller --> Daemon[cua-driver daemon]
  Framework --> Daemon
  Resources --> Upstream[cua-driver MCP server]
  Upstream --> Daemon

Loading

_{Reviews (7) · Last reviewed commit: "refactor(protocol): serve an env's MCP r..." · Reviewed by Greptile}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​cua-driver@​0.38.0

View full report