Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .grok-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
"description": "Let Grok see and operate desktop apps and browsers through a dedicated subagent, with a safety-checked MCP facade",
"category": "automation",
"version": "0.2.0",
"keywords": ["computer-use", "gui", "desktop", "browser", "testing"],
"homepage": "https://github.com/bo-516/computer-use",
"keywords": ["computer-use", "computer use", "grok computer", "grok-computer-mcp"],
"source": { "type": "local", "path": "./plugins/computer-use" }
}
]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**English** | [简体中文](README.zh-CN.md)

Let Grok Build see the screen, click and type: a Grok Build plugin (`computer-use`) plus a purpose-built MCP facade (`grok-computer-mcp`). The main use is checking UI inside the development loop (change code → launch the app → look at the UI → change again), and operating desktop and web tools that have no API.
Computer use for Grok Build: a Grok Build plugin (`computer-use`) plus a purpose-built MCP server (the `grok-computer-mcp` facade) that let Grok Build see the screen, click and type in desktop apps and browsers on macOS, Windows and Linux. The main use is checking UI inside the development loop (change code → launch the app → look at the UI → change again), and operating desktop and web tools that have no API.

Design document: [docs/goal.md](docs/goal.md). Installation and configuration: [docs/install.md](docs/install.md). The documents under `docs/` are written in Chinese.

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[English](README.md) | **简体中文**

让 Grok Build 能看屏幕、点鼠标、敲键盘:一个 Grok Build 插件(`computer-use`)加一个自研的门面 MCP 服务(`grok-computer-mcp`)。主要用途是开发闭环里的界面验证(改代码 → 启动应用 → 看界面 → 再改),以及操作没有 API 的桌面和网页工具。
为 Grok Build 提供 computer use(电脑操控)能力:一个 Grok Build 插件(`computer-use`)加一个自研的门面 MCP 服务(`grok-computer-mcp`),让 Grok Build 能在 macOS、Windows、Linux 的桌面应用和浏览器里看屏幕、点鼠标、敲键盘。主要用途是开发闭环里的界面验证(改代码 → 启动应用 → 看界面 → 再改),以及操作没有 API 的桌面和网页工具。

设计文档:[docs/goal.md](docs/goal.md)。安装与配置:[docs/install.md](docs/install.md)。

Expand Down
1 change: 1 addition & 0 deletions docs/goal.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ grok-computer/

- manifest 放在 `plugins/computer-use/.grok-plugin/plugin.json`,而不是插件根目录:官方 marketplace 的目录工具只读取 `.grok-plugin/plugin.json` 或 `.claude-plugin/plugin.json`,根目录的 `plugin.json` 不会进入组件目录。
- 本仓库的 `marketplace.json` 使用 `local` 源(插件与 marketplace 同仓库,随 marketplace 源的提交一起固定)。供其他目录(如 xai-org/plugin-marketplace)引用时,用 `uv run python scripts/plugin_index.py --entry <40 位 sha> --url <git 地址>` 生成固定 sha 的 `url` 源条目(含 `path: plugins/computer-use`);用户也可以直接安装固定提交:`grok plugin install your-org/grok-computer@<sha>#plugins/computer-use --trust`。
- marketplace 条目的 `keywords` 只放品牌/产品词(`computer-use`、`computer use`、`grok computer`、`grok-computer-mcp`),并带 `homepage`:Grok Build 用 `keywords` 和插件名按词边界匹配用户输入、主动推荐插件(CTA),`gui`、`browser`、`testing` 这类通用词会在无关请求上误触发,xai-org/plugin-marketplace 也会退回带通用词或缺 `homepage` 的提交。附录 A.1 中的 keywords 是设计阶段的旧值。
- `.grok-plugin/plugin-index.json` 由 `scripts/plugin_index.py` 生成,格式与官方目录一致(版本 + skills/commands/agents/mcpServers/hooks 组件清单),CI 用 `--check` 保证它与插件文件一致。

### 4.3 MCP 挂载(`.mcp.json`)
Expand Down
12 changes: 7 additions & 5 deletions servers/grok-computer-mcp/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# grok-computer-mcp

The desktop facade of the Grok Build `computer-use` plugin: an MCP server (stdio) that gives the
plugin's `computer` subagent nine small GUI tools on top of [Cua Driver](https://github.com/trycua/cua),
with one coordinate space, canonical screenshots, compact accessibility observations and
fail-closed safety rules.
The desktop facade of the Grok Build
[`computer-use` plugin](https://github.com/bo-516/computer-use): an MCP server (stdio) that gives
the plugin's `computer` subagent nine small GUI tools on top of
[Cua Driver](https://github.com/trycua/cua), with one coordinate space, canonical screenshots,
compact accessibility observations and fail-closed safety rules.

| Tool | Purpose |
|---|---|
Expand Down Expand Up @@ -35,6 +36,7 @@ uvx grok-computer-mcp@0.2.0 trace purge --all
machines without a system `python3`.

Configuration is through `GROK_COMPUTER_*` environment variables (backend, host or sandbox mode,
Cua Driver transport, grounding provider and tier); see `docs/install.md` in the repository.
Cua Driver transport, grounding provider and tier); see
[`docs/install.md`](https://github.com/bo-516/computer-use/blob/main/docs/install.md) in the repository.

License: Apache-2.0.
24 changes: 23 additions & 1 deletion servers/grok-computer-mcp/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,29 @@ readme = "README.md"
license = "Apache-2.0"
requires-python = ">=3.11"
authors = [{ name = "Grok Computer contributors" }]
keywords = ["mcp", "computer-use", "gui", "automation", "grok"]
keywords = [
"mcp",
"mcp-server",
"model-context-protocol",
"computer-use",
"gui-automation",
"desktop-automation",
"accessibility",
"ai-agent",
"grok",
"grok-build",
]
classifiers = [
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Operating System :: MacOS",
"Operating System :: Microsoft :: Windows",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"Topic :: Software Development :: Testing",
]
# Default dependencies must stay MIT / Apache-2.0 / BSD (AGENTS.md). mcp: MIT; pydantic: MIT;
# pillow: MIT-CMU (MIT family); httpx2 (grounding HTTP, also mcp's client): BSD-3-Clause.
Expand All @@ -23,6 +40,11 @@ dependencies = [
"pydantic>=2.11,<3",
]

[project.urls]
Homepage = "https://github.com/bo-516/computer-use"
Repository = "https://github.com/bo-516/computer-use"
Issues = "https://github.com/bo-516/computer-use/issues"

[project.scripts]
grok-computer-mcp = "grok_computer_mcp.cli:main"

Expand Down
Loading