feat(acp): browser control via MCP-over-ACP + stdio bridge (Phase 2)

[brettchien]

What problem does this solve?

The ACP-over-WebSocket base (#1418) ships a 1:1 streaming chat surface at GET /acp: a browser side-panel extension connects as an ACP client and drives an OpenAB agent. It deliberately stops at chat — no tool calls, no way for the agent's LLM to act on the page it is talking about.

This PR delivers the base ADR §6 north-star: let the agent's LLM autonomously operate the user's real, logged-in Chrome (read the DOM, screenshot, navigate, click, type) by exposing the browser as MCP tools and routing them MCP-over-ACP over the /acp WebSocket the extension already holds. No sandbox VM, no second connection, no per-frontend adapter.

The design was recorded design-only in #1418 as acp-server-websocket-mcp-browser.md (Status: Proposed); this PR is its as-built implementation.

Closes # — None; tracked by the base ADR §6 "Critical path" roadmap, not a separate issue.

Discord Discussion URL: https://discord.com/channels/1491295327620169908/1527521703255605338

At a Glance

   REMOTE (user's real Chrome)          OPENAB POD (`openab run` — one process tree)
 ┌──────────────────┐        ┌────────────────────────────────────────────────────┐
 │ side-panel ext.  │        │  ┌─────────┐   ┌───────────┐   ┌──────────────────┐ │
 │  = MCP SERVER    │◀─/acp─▶│  │ gateway │──▶│  core     │──▶│  agent CLI       │ │
 │  5 browser tools │  WSS   │  │ /acp srv│   │ MCP proxy │   │  LLM (MCP client)│ │
 └──────────────────┘ (only  │  └─────────┘   └───────────┘   └──────────────────┘ │
                      remote  │       ▲ upstream: MCP-over-ACP    ▲ downstream:      │
                       hop)   │       └── mcp/message framing     └ HTTP MCP (proxy) │
                             │                                     or stdio (bridge) │
                             └────────────────────────────────────────────────────┘

 one tool call (LLM clicks a button); only the extension hop leaves the pod:
   LLM ─tools/call browser.click─▶ core(MCP proxy) ─▶ gateway ─MCP-over-ACP─▶ extension
   extension runs it in the active tab ─▶ result ─▶ gateway ─▶ core ─▶ LLM continues

Prior Art & Industry Research

The problem: give the agent's LLM tools that drive the user's own remote, logged-in browser, where the tool provider is an MV3 extension that cannot open a listening socket.

OpenClaw — browser control is either an OpenClaw-managed, isolated Chrome profile driven by a local control service inside the Gateway (Control UI, managed browser), or the Chrome DevTools MCP server against a locally-reachable Chrome (MCP docs). openclaw mcp serve is a stdio bridge that keeps a stdio MCP session open and forwards to a local/remote Gateway over WebSocket — but it exposes routed channel conversations as MCP, not a remote browser. In every case the MCP server is colocated with the browser (local CDP or an in-Gateway managed profile); OpenClaw does not route tools from a remote, can't-listen extension back to a colocated agent.

Hermes Agent — built-in browser tools work over accessibility-tree snapshots + stable refs + screenshots, with pluggable backends (Browserbase / Browser Use / Firecrawl cloud, or local Chromium / Camofox) (browser feature). hermes-computer-use is a pixel-level MCP server (screenshot + xdotool on an Xvfb display) that any MCP client connects to (repo). Here too the MCP server runs where the browser runs (a display/Xvfb the server owns, or a cloud backend); the agent is a straight MCP client of a colocated/hosted server, not of a remote user's extension.

Takeaway — both projects colocate the browser-tool MCP server with the browser (local CDP, managed profile, Xvfb, or cloud backend) and let the agent be a normal MCP client. Neither tunnels MCP from a remote user's own can't-listen extension back to a colocated agent over the client's existing protocol connection. That inversion — extension = MCP server over its outbound /acp WS; OpenAB = MCP proxy/aggregator; agent = in-pod MCP client — is the novel part and is what this PR builds. (ACP itself, and the general OpenClaw/Hermes ACP comparison, are covered in #1418.)

Proposed Solution

Two hops, one wire contract, behind the existing acp feature.

1. Protocol gap — agent→client REQUEST direction. The base only did client→agent prompts and agent→client notifications (streaming text). Browser control needs the agent to ask the client and await a result, so acp_server's dispatch loop gains a server-initiated request path (T1.2/1.3, T2.1). Wire types for the expanded surface are the generated serde-only v1 types already landed in #1418.

2. Upstream hop — MCP-over-ACP tunnel (extension ↔ gateway). A tunnel frame API multiplexes MCP tools/list / tools/call / results over the same /acp WS using the official mcp/message framing (T4, T4.1); the extension declares type:acp mcpServers on initialize and the gateway records + establishes a per-session TunnelHandle via an AcpTunnelRegistry, opened on session/new and session/resume and torn down on cleanup (T5.3). The wire contract is pinned in docs/mcp-over-acp-tunnel-contract.md.

3. Downstream hop — how the colocated agent sees the tools. Core hosts a normal in-process MCP server (the extension, not the agent, is the one that can't listen). Two delivery modes, selected by OPENAB_BROWSER_MODE:

• proxy (default, HTTP MCP) — a per-session loopback HTTP MCP server (mcp_proxy::start_session_server), bearer-gated (constant-time compare), that forwards tools/call down the tunnel via a BrowserTunnel trait + ProxyHandler. openab writes a fresh {url, headers} entry (openab-browser) into the CLI's native MCP config (.cursor/mcp.json, .kiro/settings/mcp.json) per session, 0600, stripped on evict. Suits HTTP-MCP CLIs.
• bridge (stdio MCP, "Option C") — a per-pod browser-bridge Unix-socket server plus an openab browser-bridge stdio-MCP relay subcommand. The CLI config becomes a static {"command":"openab","args":["browser-bridge"]} (no per-session url/token to mint), written once; the relay resolves its session channel by process ancestry (inherits the agent's OPENAB_BROWSER_CHANNEL) rather than a config value, so N windows → N agent processes → N bridges, each bound to its own browser with no clobber. Suits stdio-MCP CLIs.

Toolset — five DOM-semantic tools (browser.read_dom, browser.screenshot, browser.navigate, browser.click, browser.type). The ACP frame cap is raised 1→8 MiB to carry screenshot results (JPEG). Per-session isolation is enforced: one tunnel per session, fixing an earlier fan-out overwrite/orphan.

Why this approach?

• Extension-as-MCP-server over its own outbound WS — an MV3 extension cannot open a listening socket, and the user's browser is remote. Serving MCP over the /acp WS the extension already holds is the only way a can't-listen remote provider can be a full MCP server, and it reuses the one connection (no second transport, no inbound firewall hole).
• MCP tools, not a custom ACP ExtRequest — only tools appear in the LLM's tools/list, so only tools let the model autonomously act. An ExtRequest would fit OpenAB-driven ops but the LLM would never call it.
• DOM-semantic tools, not model-specific computer (pixel) tools — click(selector) / read_dom are cheaper, more reliable, and model-agnostic; screenshot+coords remain expressible if wanted.
• Two downstream modes — HTTP-MCP CLIs (Cursor, Kiro) take the proxy directly; stdio-MCP CLIs get the static bridge, which also removes per-session secret minting and makes multi-window isolation fall out of process ancestry. Trade-off: bridge adds a per-pod socket + a relay subcommand; proxy mints a port/token per session.

Known limitations — auto-written CLI config today covers Cursor + Kiro only (Claude Code / Codex / Gemini paths are documented in docs/browser-mcp-agent-setup.md but not yet auto-written); the browser tunnel binds to the ACP session, so the agent must be driven through the extension's ACP session (no Discord↔browser bridge).

Alternatives Considered

• Custom ExtRequest per browser action — rejected: not surfaced to the LLM as a tool, so the model can't autonomously call it.
• Extension hosts a standalone HTTP/SSE MCP server — rejected: MV3 extensions cannot open a listening socket.
• Anthropic-style computer tool (screenshot + pixel coords) — subsumed: expressible as screenshot+click(x,y), but DOM-semantic tools are cheaper and model-agnostic.
• Colocated/sandbox browser (OpenClaw-managed / Hermes Xvfb) — rejected: the goal is the user's real, logged-in Chrome, not a sandbox VM.

Validation

Rust changes, gated behind --features acp (openab-gateway/acp + openab-core/acp-mcp):

• cargo build --features acp — clean (only pre-existing dead_code warnings, unrelated)
• MCP-over-ACP tunnel e2e suite in scripts/acp-ws-smoke.py (producer section: tools/list / tools/call, fan-out + filtering)
• Handler-level + streaming tests from the base remain green
• Live manual test — full loop against a real deployment with an MV3 side-panel extension driving Chrome: all five tools (read_dom / screenshot / navigate / click / type), connection status pill, and stable reconnect (tunnel re-established on session/resume)
• cargo clippy --workspace -- -D warnings (default) and cargo clippy --workspace --features unified -- -D warnings (includes acp) — both clean
• bridge-mode two-window per-session isolation — manual acceptance checklist in the Option C validation notes (each window's agent sees only its own browser)

Review Contract

Goal

Let a colocated agent's LLM autonomously operate the user's real remote Chrome via five DOM-semantic MCP tools (read_dom/screenshot/navigate/click/type), routed MCP-over-ACP over the existing /acp WebSocket: extension = MCP server, OpenAB core = MCP proxy/aggregator, agent = in-pod MCP client. Ship both downstream delivery modes (proxy HTTP-MCP default; bridge stdio-MCP via OPENAB_BROWSER_MODE) with per-session tunnel isolation and the mcp/message wire contract.

Non-goals

• Auto-writing MCP config for Claude Code / Codex / Gemini CLIs (documented, not wired — Cursor + Kiro only today).
• Any Discord↔browser bridge: the tunnel is bound to the extension's ACP session by design.
• New browser tools beyond the five DOM-semantic ones; pixel/computer-style tooling.
• Backpressure / global connection limits beyond the base's per-connection caps.

Accepted Residual Risks

• Downstream MCP is loopback + bearer only — the per-session HTTP proxy binds 127.0.0.1 with a fresh constant-time-compared bearer, 0600 config stripped on evict; it trusts in-pod locality. A hostile in-pod process could reach it (same trust boundary as the agent subprocess itself).
• bridge mode resolves channel by process ancestry — correct for the normal agent→bridge spawn tree; a bridge started outside that tree would not inherit OPENAB_BROWSER_CHANNEL and would fail closed (no browser attached) rather than cross-wire sessions.
• 8 MiB frame cap — screenshots must be JPEG; a pathological DOM/screenshot over 8 MiB closes the frame rather than truncating.
• Tool execution trust — the extension executes DOM actions in the user's logged-in tab; safety relies on the user driving their own session and the tools being DOM-semantic (no arbitrary JS eval tool).

Acceptance Criteria

• cargo build --features acp green; base handler/streaming/conformance tests green; cargo clippy --features acp -- -D warnings clean before merge.
• scripts/acp-ws-smoke.py MCP-over-ACP producer section passes (tools/list, tools/call, fan-out + filtering).
• Live e2e: all five tools drive a real Chrome via the extension over /acp; tunnel re-establishes on session/resume.
• bridge mode: two windows on the same endpoint each drive their own browser with no clobber.

Follow-ups

• Auto-write MCP config for Claude Code / Codex (TOML) / Gemini variants.
• Progressive/streamed tool results and richer error surfaces.
• Bounded outbound / idle eviction inherited from the base's Follow-ups.
• CI: add a headless tunnel smoke to canary-tests so the producer path is gated, not just manual.