diff --git a/README.md b/README.md index d0d7091c..827dad13 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # CoreCoder -**The nanoGPT of coding agents. A 1.3k-line engine inside 2,816 readable lines of pure Python: understand how a coding agent actually works, then fork your own.** +**The nanoGPT of coding agents. A 1.3k-line engine inside 2,948 readable lines of pure Python: understand how a coding agent actually works, then fork your own.** *learn from it · fork it · ship something better* @@ -25,7 +25,7 @@ | | CoreCoder | Claude Code | aider | nanoGPT | |---|---|---|---|---| -| Lines of code | ~1,322 engine / 2,816 total | hundreds of thousands (closed) | tens of thousands of Python | ~600 (two files) | +| Lines of code | ~1,322 engine / 2,948 total | hundreds of thousands (closed) | tens of thousands of Python | ~600 (two files) | | Time to read it all | one afternoon | can't (closed) | a few days of slogging | one afternoon | | Breakpoint, change, rerun? | yes, every line | no | yes, but there's a lot | yes | | What it's for | understand one, then fork your own | production coding assistant | terminal pair-programming | minimal GPT for teaching | @@ -36,7 +36,7 @@ The nanoGPT column is there as a reference point: minimal, readable, but it teac I've always felt coding agents get talked about as if they were arcane. Strip a tool like Claude Code or Cursor all the way down and the core is a `while` loop wrapped around a large model, plus seven or eight tools that let it actually do things. The hard part was never the loop; it's everything the loop has to cope with once it meets the real world. CoreCoder is the minimal version that writes that core out honestly. -The engine (loop, model interface, context, tools, sessions) is 1,322 lines once you drop blank lines and comments. Counting the outer CLI, config and packaging too, the whole package is 25 files: 2,816 physical lines, 2,276 net, every one short enough to read in a single sitting. The growth since the original 1,161-line snapshot went into visible features: plan mode, hooks and checkpoints, each documented below. +The engine (loop, model interface, context, tools, sessions) is 1,322 lines once you drop blank lines and comments. Counting the outer CLI, config and packaging too, the whole package is 26 files: 2,948 physical lines, 2,390 net, every one short enough to read in a single sitting. The growth since the original 1,161-line snapshot went into visible features: plan mode, hooks and checkpoints, each documented below. And it really runs: reads and writes files, executes shell, spawns sub-agents, compacts context in three tiers, and tells you the tokens and dollars a run burned whenever you ask. Anything that would mutate your disk or run a command stops for your consent first. 227 tests, all green. But the point of it running isn't to become your daily driver. It runs so the walkthrough can't lie: a reference that shows how an agent works has to actually work. @@ -94,7 +94,8 @@ corecoder/ ├── permissions.py consent for mutating tools 75 lines ├── hooks.py Pre/PostToolUse shell hooks 87 lines ├── shell.py POSIX shell routing (Git Bash on Windows) 61 lines -├── mcp.py MCP stdio client for external tools 237 lines +├── mcp.py shared MCP tools + stdio transport 248 lines +├── mcp_http.py Streamable HTTP transport 121 lines ├── prompt.py system prompt 41 lines ├── cli.py REPL + slash commands + one-shot 393 lines ├── config.py env-var config 55 lines @@ -266,7 +267,15 @@ Drop a `mcp.json` under `~/.corecoder` and tools from any MCP server join the ag } ``` -Each configured server starts as a subprocess at launch, handshakes, and lists its tools; every one is registered as `mcp____`, so hook matchers and the consent gate treat it exactly like a built-in. MCP tools stay out of the read-only set, meaning the agent asks before running one. The handshake gets fifteen seconds, a call gets sixty, and a server that dies or never answers fails that one call as an ordinary tool result instead of killing the loop. The REPL's `/mcp` lists every server with its state, and `/mcp reconnect ` starts a dead one fresh: new process, new handshake, and the tools the agent already holds work again with no restart. The client speaks the tools slice of the protocol (initialize, tools/list, tools/call) and nothing else, which keeps the whole thing inside `mcp.py` at about 240 lines. With no `mcp.json` there is no MCP and nothing changes. A runnable starter config lives in [`examples/mcp_filesystem.md`](examples/mcp_filesystem.md): the reference filesystem server, scoped to one directory, behind the same consent gate. +Each configured server starts as a subprocess at launch, handshakes, and lists its tools; every one is registered as `mcp____`, so hook matchers and the consent gate treat it exactly like a built-in. MCP tools stay out of the read-only set, meaning the agent asks before running one. The handshake gets fifteen seconds, a call gets sixty, and a server that dies or never answers fails that one call as an ordinary tool result instead of killing the loop. The REPL's `/mcp` lists every server with its state, and `/mcp reconnect ` starts a dead one fresh: new process, new handshake, and the tools the agent already holds work again with no restart. The client speaks the tools slice of the protocol (initialize, tools/list, tools/call) and nothing else, with the shared tools and stdio transport in `mcp.py` and HTTP transport in `mcp_http.py`. With no `mcp.json` there is no MCP and nothing changes. A runnable starter config lives in [`examples/mcp_filesystem.md`](examples/mcp_filesystem.md): the reference filesystem server, scoped to one directory, behind the same consent gate. + +Remote servers can use Streamable HTTP instead of launching a process: set +`"url": "https://your-server.example/mcp"` in the server entry, with optional +`"headers"` for static authentication. JSON and SSE responses, protocol negotiation, +session IDs and shutdown are supported. Remote tools keep the same consent gate, +hooks and `/mcp reconnect ` command. Failed calls are never automatically +replayed, since the server may already have executed them. See +[`examples/mcp_http.md`](examples/mcp_http.md) for configuration and supported scope. ## Related Projects diff --git a/README_CN.md b/README_CN.md index a0550c26..ba63fc07 100644 --- a/README_CN.md +++ b/README_CN.md @@ -2,7 +2,7 @@ # CoreCoder -**编程 agent 里的 nanoGPT。1.3k 行引擎、整包 2816 行纯 Python 全部一口气可读,读懂一个 coding agent 到底怎么运作,再 fork 出你自己的。** +**编程 agent 里的 nanoGPT。1.3k 行引擎、整包 2948 行纯 Python 全部一口气可读,读懂一个 coding agent 到底怎么运作,再 fork 出你自己的。** *learn from it · fork it · ship something better* @@ -25,7 +25,7 @@ | | CoreCoder | Claude Code | aider | nanoGPT | |---|---|---|---|---| -| 代码量 | 引擎约 1322 行 / 整包 2816 行 | 几十万行(闭源) | 数万行 Python | 约 600 行(两个文件) | +| 代码量 | 引擎约 1322 行 / 整包 2948 行 | 几十万行(闭源) | 数万行 Python | 约 600 行(两个文件) | | 读完要多久 | 一个下午 | 读不了(闭源) | 得啃几天 | 一个下午 | | 能不能下断点改了再跑 | 能,每一行 | 不能 | 能,但量大 | 能 | | 定位 | 读懂并 fork 出你自己的 agent | 生产级编程助手 | 终端结对编程 | 教学用最小 GPT | @@ -36,7 +36,7 @@ nanoGPT 那一列是拿来对照的:它最小、可读,但教的是训一个 我一直觉得 coding agent 被讲得太玄了。把 Claude Code、Cursor 这类工具扒到底,核心是一个 while 循环套着一个大模型,外加七八个让它能真正动手的工具。难的从来不是这个循环,而是循环跑进真实世界以后要兜的那些底。CoreCoder 就是把这个核心老老实实写出来的最小版本。 -引擎部分(循环、模型接口、上下文、工具、会话)去掉空行和注释是 1322 行。连最外层的 CLI、配置、打包一起算,整个包 25 个文件、物理 2816 行、净 2276 行,每个文件都短到能一口气读完。自 1161 行快照之后的增长都花在了看得见的功能上:plan mode、hooks、checkpoints,下文各有交代。 +引擎部分(循环、模型接口、上下文、工具、会话)去掉空行和注释是 1322 行。连最外层的 CLI、配置、打包一起算,整个包 26 个文件、物理 2948 行、净 2390 行,每个文件都短到能一口气读完。自 1161 行快照之后的增长都花在了看得见的功能上:plan mode、hooks、checkpoints,下文各有交代。 它真能跑:读写文件、执行 shell、派子 agent、分三层压上下文,还能随时把这趟烧掉的 token 和美元数报给你。任何要动你磁盘、要跑命令的调用,都会先停下来等你点头,227 个测试是绿的。但能跑不是为了劝你拿去日用,而是为了让这份「注释」不撒谎:一个解释 agent 怎么运作的范例,自己得真能运作。 @@ -94,7 +94,8 @@ corecoder/ ├── permissions.py 改动类工具的用户授权 75 行 ├── hooks.py 工具调用前后的用户 shell 钩子 87 行 ├── shell.py POSIX shell 路由(Windows 走 Git Bash) 61 行 -├── mcp.py MCP stdio 客户端,接外部工具 237 行 +├── mcp.py MCP 共用工具与 stdio 传输 248 行 +├── mcp_http.py Streamable HTTP 传输 121 行 ├── prompt.py 系统提示词 41 行 ├── cli.py REPL + 斜杠命令 + 一次性模式 393 行 ├── config.py 环境变量配置 55 行 @@ -265,7 +266,10 @@ REPL 里 `/plan` 开关计划模式。开着的时候,提示符变成 `(plan)` } ``` -每个配好的服务器在启动时拉起一个子进程,握手、列出工具;每件工具都注册成 `mcp__<服务器>__<工具>`,钩子匹配和授权闸对它和内建工具一视同仁。MCP 工具不在只读名单里,模型要调,得先问过你。握手给十五秒,一次调用给六十秒;服务器挂了或者迟迟不应,那一次调用就以普通工具结果的形式报错,循环照常往下走。REPL 里的 `/mcp` 列出每台服务器的状态,`/mcp reconnect <名字>` 把挂掉的重新拉起:新进程、重新握手,agent 手里已有的工具不用重启就恢复可用。客户端只实现协议里工具那一小片(initialize、tools/list、tools/call),别的一概不碰,所以整块实现收在 `mcp.py` 一个文件里,两百四十行上下。没有 `mcp.json` 就没有 MCP,一切照旧。想直接跑起来的话,[`examples/mcp_filesystem.md`](examples/mcp_filesystem.md) 给了一个最小配置:MCP 官方的参考文件系统服务器,只暴露一个目录,照样过授权闸。 +每个配好的服务器在启动时拉起一个子进程,握手、列出工具;每件工具都注册成 `mcp__<服务器>__<工具>`,钩子匹配和授权闸对它和内建工具一视同仁。MCP 工具不在只读名单里,模型要调,得先问过你。握手给十五秒,一次调用给六十秒;服务器挂了或者迟迟不应,那一次调用就以普通工具结果的形式报错,循环照常往下走。REPL 里的 `/mcp` 列出每台服务器的状态,`/mcp reconnect <名字>` 把挂掉的重新拉起:新进程、重新握手,agent 手里已有的工具不用重启就恢复可用。客户端只实现协议里工具那一小片(initialize、tools/list、tools/call),别的一概不碰,共用工具与 stdio 传输在 `mcp.py`,HTTP 传输在 `mcp_http.py`。没有 `mcp.json` 就没有 MCP,一切照旧。想直接跑起来的话,[`examples/mcp_filesystem.md`](examples/mcp_filesystem.md) 给了一个最小配置:MCP 官方的参考文件系统服务器,只暴露一个目录,照样过授权闸。 + + +远程服务器也可以通过 Streamable HTTP 连接:在服务器配置中设置 `"url": "https://your-server.example/mcp"`,并按需用 `"headers"` 对象配置静态认证。传输层支持 JSON 与服务器发送事件(SSE)响应、协议版本协商、会话 ID 和关闭时的会话清理。远程工具仍经过相同的用户授权与钩子,`/mcp reconnect <名字>` 也能重新连接。失败的工具调用不会自动重放,因为服务器可能已经执行了操作。配置示例及支持范围见 [`examples/mcp_http.md`](examples/mcp_http.md)。 ## 相关项目 diff --git a/article/00-index.md b/article/00-index.md index 9c554d8f..f323168c 100644 --- a/article/00-index.md +++ b/article/00-index.md @@ -29,7 +29,7 @@ CoreCoder 做的事,是把这套骨架压到一千三百行出头的纯 Python 5. [并行执行与子 agent](05-parallel-and-subagents.md)。模型一次返回多个工具调用时,CoreCoder 用线程池并发跑。这篇老实讲这个简化版相对 Claude Code 的流式执行器差在哪,并发又会引入什么新麻烦,以及子 agent 为什么不准递归。 6. [把它跑成一个真正的命令行工具](06-session-and-cli.md)。会话存盘、断点续聊、斜杠命令、一次性模式。`session.py` 里有个不起眼但很要命的安全细节:怎么防住用恶意会话名做路径穿越。 7. [Fork CoreCoder,搭一个你自己的 coding agent](07-build-your-own.md)。收尾的实操篇。从 clone 到换成你常用的模型,到加一个真正有用的自定义工具,到改系统提示词调教它的风格,到打包发布。读完前六篇你已经懂了原理,这篇让你真有一个东西。 -8. [不动主循环的三种加法:MCP、钩子与计划模式](08-extensibility.md)。给 agent 加东西的三条正路:`mcp.py`(237 行)把远程工具接成与内建同权,hooks 用退出码 2 否决一次调用却永远杀不死循环,计划模式用一个布尔量压住包括 `--yes` 在内的整个权限层。最后一节把三块收拢回同一条契约:工具同形、错误是普通返回值、循环不被外围杀死。 +8. [不动主循环的三种加法:MCP、钩子与计划模式](08-extensibility.md)。给 agent 加东西的三条正路:`mcp.py`(248 行)把远程工具接成与内建同权,hooks 用退出码 2 否决一次调用却永远杀不死循环,计划模式用一个布尔量压住包括 `--yes` 在内的整个权限层。最后一节把三块收拢回同一条契约:工具同形、错误是普通返回值、循环不被外围杀死。 不想按顺序也行。想搞懂「它凭什么敢自动跑命令」直接跳第二篇;想接自己的模型直接跳第三篇;只想赶紧 fork 出个能用的,第七篇是自洽的。 diff --git a/article/08-extensibility.md b/article/08-extensibility.md index 621714ad..f0432618 100644 --- a/article/08-extensibility.md +++ b/article/08-extensibility.md @@ -1,6 +1,6 @@ # 不动主循环的三种加法:MCP、钩子与计划模式 -第七篇结束的时候,你手里已经有一个能跑的 agent 了。接下来你几乎一定会想做三件事:给它接新工具(接进来一个文件系统服务、一个数据库服务),在工具调用前后插一段自己的逻辑(每次跑 bash 前先过一道检查),以及让它偶尔「只许看、不许动」(先出个方案再开工)。这三件事对应 v0.6.0 新增的三块代码:`mcp.py`(237 行)、`hooks.py`(87 行)、以及 `agent.py` 里一个五十行的布尔量。它们的共同点是:都不碰第一到第六篇讲的那个主循环。这一篇讲为什么「不碰主循环」不是克制,而是这三块功能能成立的前提。 +第七篇结束的时候,你手里已经有一个能跑的 agent 了。接下来你几乎一定会想做三件事:给它接新工具(接进来一个文件系统服务、一个数据库服务),在工具调用前后插一段自己的逻辑(每次跑 bash 前先过一道检查),以及让它偶尔「只许看、不许动」(先出个方案再开工)。这三件事对应 v0.6.0 新增的三块代码:`mcp.py`(248 行)、`hooks.py`(87 行)、以及 `agent.py` 里一个五十行的布尔量。它们的共同点是:都不碰第一到第六篇讲的那个主循环。这一篇讲为什么「不碰主循环」不是克制,而是这三块功能能成立的前提。 ## MCP:把「协议」裁成「能用的那一片」 diff --git a/corecoder/mcp.py b/corecoder/mcp.py index 3b37f8ac..2fc92a52 100644 --- a/corecoder/mcp.py +++ b/corecoder/mcp.py @@ -4,6 +4,9 @@ {"mcpServers": {"fs": {"command": "npx", "args": ["-y", "some-fs-server", "/tmp"]}}} +A server entry with a "url" instead of "command" uses Streamable HTTP +(mcp_http.py); optional "headers" configure static authentication. + Each server is a subprocess speaking JSON-RPC 2.0, one message per line over stdin/stdout. Startup handshakes (`initialize`), pulls `tools/list`, and every remote tool joins the agent as `mcp____`, so consent, hooks and @@ -73,18 +76,22 @@ def _start(self): self._dead = None # a fresh process starts clean self._responses.clear() try: - self._request("initialize", { - "protocolVersion": PROTOCOL_VERSION, - "capabilities": {}, - "clientInfo": {"name": "corecoder", "version": __version__}, - }, INIT_TIMEOUT) - self._notify("notifications/initialized") - listed = self._request("tools/list", {}, INIT_TIMEOUT) - self.tools = [MCPTool(self, t) for t in listed.get("tools", [])] + self._initialize() except BaseException: self.close() # a half-started server must not leak raise + def _initialize(self): + initialized = self._request("initialize", { + "protocolVersion": PROTOCOL_VERSION, + "capabilities": {}, + "clientInfo": {"name": "corecoder", "version": __version__}, + }, INIT_TIMEOUT) + self._protocol_version = initialized.get("protocolVersion", PROTOCOL_VERSION) + self._notify("notifications/initialized") + listed = self._request("tools/list", {}, INIT_TIMEOUT) + self.tools = [MCPTool(self, t) for t in listed.get("tools", [])] + @property def error(self) -> MCPError | None: """What killed the server, or None while it is healthy.""" @@ -222,7 +229,11 @@ def load_mcp_tools(path: Path = CONFIG_FILE) -> list[Tool]: tools: list[Tool] = [] for name, spec in (data.get("mcpServers") or {}).items(): try: - client = MCPClient(name, spec["command"], spec.get("args", []), spec.get("env")) + if "url" in spec: + from .mcp_http import HTTPMCPClient + client = HTTPMCPClient(name, spec["url"], spec.get("headers")) + else: + client = MCPClient(name, spec["command"], spec.get("args", []), spec.get("env")) except (MCPError, OSError, KeyError, TypeError) as e: log.warning("MCP server %r skipped: %s", name, e) continue diff --git a/corecoder/mcp_http.py b/corecoder/mcp_http.py new file mode 100644 index 00000000..b48bce74 --- /dev/null +++ b/corecoder/mcp_http.py @@ -0,0 +1,121 @@ +"""The tools-only MCP client over Streamable HTTP, without automatic call replay.""" + +import json +import time +from urllib.error import HTTPError, URLError +from urllib.parse import urlsplit +from urllib.request import HTTPRedirectHandler, Request, build_opener + +from .mcp import INIT_TIMEOUT, PROTOCOL_VERSION, MCPClient, MCPError + + +class _NoRedirect(HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None # Never forward configured credentials to a redirect target. + + +class HTTPMCPClient(MCPClient): + """POST JSON-RPC requests, accepting either JSON or SSE responses. + + Reuse the stdio client's tool wrappers and explicit reconnect. A lost + response is never retried: a remote tool may already have had side effects. + """ + + def __init__(self, name: str, url: str, headers: dict | None = None): + if not isinstance(url, str) or urlsplit(url).scheme not in {"http", "https"} or not urlsplit(url).netloc: + raise MCPError("MCP url must use http or https") + if headers is not None and (not isinstance(headers, dict) or + not all(isinstance(k, str) and isinstance(v, str) for k, v in headers.items())): + raise MCPError("MCP headers must map strings to strings") + self._url = url + self._headers = dict(headers or {}) + self._opener = build_opener(_NoRedirect) + super().__init__(name, "") + + def _start(self): + self._session_id = None + self._protocol_version = None + self._dead = None + try: + self._initialize() + except BaseException: + self.close() + raise + + def _http_headers(self): + headers = {**self._headers, "Content-Type": "application/json", "Accept": "application/json, text/event-stream"} + if self._session_id: + headers["Mcp-Session-Id"] = self._session_id + if self._protocol_version: + headers["MCP-Protocol-Version"] = self._protocol_version + return headers + + def _post(self, msg: dict, timeout: float): + request = Request(self._url, data=json.dumps(msg).encode("utf-8"), headers=self._http_headers(), method="POST") + try: + with self._opener.open(request, timeout=timeout) as response: + if "id" not in msg and response.status == 202: + return {} + if msg["method"] == "initialize": + self._session_id = response.headers.get("Mcp-Session-Id") + deadline = time.monotonic() + timeout + for reply in _messages(response, deadline): + if time.monotonic() > deadline: + raise MCPError("MCP HTTP response timed out") + if not isinstance(reply, dict): + raise MCPError("MCP HTTP response must be a JSON object") + if reply.get("id") == msg.get("id") and "id" in msg and ("result" in reply or "error" in reply): + if "error" in reply: + raise MCPError(f"MCP server {self.name!r} rejected {msg['method']}: {reply['error']}") + return reply.get("result", {}) + raise MCPError("MCP HTTP response ended without a matching reply") + except HTTPError as e: + self._dead = MCPError(f"MCP server {self.name!r}: HTTP {e.code}; use /mcp reconnect {self.name}") + e.close() + raise self._dead from e + except (URLError, OSError, ValueError) as e: + self._dead = MCPError(f"MCP server {self.name!r}: HTTP transport failed ({type(e).__name__})") + raise self._dead from e + + def _request(self, method: str, params: dict, timeout: float) -> dict: + with self._cond: + if self._dead is not None: + raise self._dead + self._next_id += 1 + req_id = self._next_id + result = self._post({"jsonrpc": "2.0", "id": req_id, "method": method, "params": params}, timeout) + if method == "initialize" and result.get("protocolVersion") not in {PROTOCOL_VERSION, "2025-03-26"}: + raise MCPError("MCP server negotiated an unsupported protocol version") + return result + + def _notify(self, method: str): + self._post({"jsonrpc": "2.0", "method": method, "params": {}}, INIT_TIMEOUT) + + def close(self): + session, self._session_id = self._session_id, None + self._dead = MCPError(f"MCP server {self.name!r} closed") + if session: + headers = {**self._http_headers(), "Mcp-Session-Id": session} + try: + with self._opener.open(Request(self._url, headers=headers, method="DELETE"), timeout=INIT_TIMEOUT): + pass + except (URLError, OSError): + pass # Termination is best effort; a server may return 405. + + +def _messages(response, deadline): + if response.headers.get_content_type() == "application/json": + yield json.load(response) + elif response.headers.get_content_type() == "text/event-stream": + data = [] + for raw in response: + if time.monotonic() > deadline: + raise MCPError("MCP HTTP response timed out") + line = raw.decode("utf-8").rstrip("\r\n") + if not line and data: + yield json.loads("\n".join(data)) + data = [] + elif line.startswith("data:"): + data.append(line[5:].lstrip(" ")) + else: + raise MCPError("MCP HTTP response has an unsupported Content-Type") diff --git a/examples/mcp_http.md b/examples/mcp_http.md new file mode 100644 index 00000000..ed8dbfe9 --- /dev/null +++ b/examples/mcp_http.md @@ -0,0 +1,64 @@ +# MCP example: a remote server over Streamable HTTP + +For an existing MCP Streamable HTTP endpoint, add a URL entry to +`~/.corecoder/mcp.json`. It can live alongside your existing stdio servers: + +```json +{ + "mcpServers": { + "remote": { + "url": "https://your-server.example/mcp", + "headers": {"Authorization": "Bearer YOUR_TOKEN"} + } + } +} +``` + +Replace the endpoint and token with your server's values. Omit `headers` when +the endpoint does not need authentication. A URL entry connects to an already +running server; it does not launch a subprocess. No new dependency is required. + +At startup, CoreCoder initializes the connection and lists the remote tools. +They appear as `mcp__remote__` and keep the same consent prompts, hooks +and plan-mode restrictions as stdio tools. `/mcp` shows their connection state. + +## Sessions and failures + +The client accepts JSON and SSE responses to POST requests, including +multi-line SSE data and interleaved server notifications. It carries the +server's session ID and negotiated protocol version on subsequent requests. +The supported versions are `2025-06-18` and `2025-03-26`. + +If the session expires (HTTP 404) or the connection fails, the tool call +reports an error. CoreCoder never retries a tool call automatically: a lost +response does not prove that the server did not execute it. Use +`/mcp reconnect remote` to initialize a fresh session, then decide whether to +run the tool again. Existing tool wrappers continue to use the reconnected +client. Shutdown sends DELETE for a stateful session; servers that do not +support DELETE may respond with 405. + +Redirects are not followed, so configured headers are not forwarded to another +endpoint. Configure the final MCP URL directly. Protocol headers are managed +by the client, independently of custom headers. + +## Deliberate scope + +This is the tools-only Streamable HTTP transport, following the +[MCP transport specification](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports). +It does not implement the legacy 2024 HTTP+SSE transport, OAuth discovery, +resources, prompts, unsolicited GET streams, or automatic stream resumption. +Static authentication headers are supported. + +## Verify without credentials + +From a development checkout: + +```bash +python -m pytest tests/test_mcp_http.py tests/test_mcp.py -q +``` + +The HTTP tests start a real server bound only to `127.0.0.1`. They exercise +JSON and SSE tool calls, concurrent requests, protocol/session headers, +timeouts, redirect refusal, session expiration, explicit reconnect, and +shutdown. They use a scripted LLM and synthetic tool data, so no provider key +or external MCP service is needed. diff --git a/tests/test_mcp_http.py b/tests/test_mcp_http.py new file mode 100644 index 00000000..2260caf5 --- /dev/null +++ b/tests/test_mcp_http.py @@ -0,0 +1,289 @@ +"""Exercise the HTTP transport against a real loopback server, without an LLM.""" + +import json +import logging +import threading +from concurrent.futures import ThreadPoolExecutor +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from types import SimpleNamespace + +import pytest + +from corecoder import mcp +from corecoder.agent import Agent +from corecoder.demo import ScriptedLLM +from corecoder.llm import LLMResponse, ToolCall +from corecoder.mcp import MCPError, load_mcp_tools +from corecoder.mcp_http import HTTPMCPClient +from corecoder.permissions import Permission + + +@pytest.fixture +def endpoint(): + state = SimpleNamespace(requests=[], sessions=0, sse=False, stateful=True, mode="ok", + delete_status=200, release=threading.Event(), redirects=0, + protocol="2025-03-26", notify_status=202) + + class Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + def log_message(self, *args): + pass + + def send(self, value, status=200, session=None, content_type="application/json"): + body = value.encode("utf-8") if isinstance(value, str) else json.dumps(value).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", content_type) + self.send_header("Content-Length", str(len(body))) + if session: + self.send_header("Mcp-Session-Id", session) + self.end_headers() + self.wfile.write(body) + + def do_DELETE(self): + state.requests.append(("DELETE", dict(self.headers), {})) + self.send({}, status=state.delete_status) + + def do_POST(self): + request = json.loads(self.rfile.read(int(self.headers["Content-Length"]))) + state.requests.append((request["method"], dict(self.headers), request)) + if self.path == "/redirected": + state.redirects += 1 + method = request["method"] + session = None + if method == "initialize": + state.sessions += 1 + session = f"session-{state.sessions}" if state.stateful else None + result = {"protocolVersion": state.protocol, "capabilities": {"tools": {}}, + "serverInfo": {"name": "remote", "version": "1"}} + elif method == "notifications/initialized": + self.send("", status=state.notify_status) + return + elif method == "tools/list": + result = {"tools": [{"name": "echo", "description": "Echo remotely", + "inputSchema": {"type": "object", "properties": {"text": {"type": "string"}}}}]} + else: + if state.mode == "redirect": + self.send_response(301) + self.send_header("Location", "/redirected") + self.send_header("Content-Length", "0") + self.end_headers() + return + if state.mode in {"expired", "server-error"}: + self.send({}, status=404 if state.mode == "expired" else 500) + return + if state.mode == "timeout": + state.release.wait(3) + return + if state.mode == "heartbeats": + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.end_headers() + try: + for _ in range(20): + self.wfile.write(b": keepalive\n\n") + self.wfile.flush() + if state.release.wait(0.01): + break + except (BrokenPipeError, ConnectionResetError): + pass + self.close_connection = True + return + if state.mode == "bad-json": + self.send("not json") + return + if state.mode == "bad-type": + self.send("hello", content_type="text/plain") + return + if state.mode == "not-object": + self.send([]) + return + if state.mode == "rpc-error": + self.send({"jsonrpc": "2.0", "id": request["id"], "error": {"code": -32602, "message": "bad arguments"}}) + return + result = {"content": [{"type": "text", "text": "echo: " + request["params"]["arguments"].get("text", "")}], + "isError": state.mode == "tool-error"} + reply = {"jsonrpc": "2.0", "id": request["id"], "result": result} + if state.mode == "wrong-id" and method == "tools/call": + reply["id"] += 100 + if not state.sse: + self.send(reply, session=session) + return + # Keep the connection open after the reply: a client must return as + # soon as it gets its response rather than waiting for EOF. + self.send_response(200) + self.send_header("Content-Type", "text/event-stream; charset=utf-8") + if session: + self.send_header("Mcp-Session-Id", session) + self.end_headers() + data = ": keepalive\r\nevent: message\r\ndata: {\"jsonrpc\":\"2.0\",\"method\":\"notifications/message\"}\r\n\r\n" + data += "\r\n".join("data: " + line for line in json.dumps(reply, indent=2, ensure_ascii=False).splitlines()) + "\r\n\r\n" + self.wfile.write(data.encode("utf-8")) + self.wfile.flush() + state.release.wait(3) + self.close_connection = True + + server = ThreadingHTTPServer(("127.0.0.1", 0), Handler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + try: + yield f"http://127.0.0.1:{server.server_port}/mcp", state + finally: + state.release.set() + for client in mcp._live_clients: + client.close() + mcp._live_clients.clear() + server.shutdown() + server.server_close() + thread.join(timeout=3) + + +def configured_tools(tmp_path, endpoint): + url, _ = endpoint + path = tmp_path / "mcp.json" + path.write_text(json.dumps({"mcpServers": {"remote": {"url": url, "headers": {"X-Client-Key": "example"}}}})) + return load_mcp_tools(path) + + +def test_http_handshake_and_agent_tool_round_trip(tmp_path, endpoint): + tools = configured_tools(tmp_path, endpoint) + assert [tool.name for tool in tools] == ["mcp__remote__echo"] + assert tools[0].description == "Echo remotely" + agent = Agent(llm=ScriptedLLM([ + LLMResponse(tool_calls=[ToolCall(id="call-1", name=tools[0].name, arguments={"text": "hello"})]), + LLMResponse(content="done"), + ]), tools=tools) + assert agent.chat("go") == "done" + assert agent.messages[2]["content"] == "echo: hello" + _, state = endpoint + assert [method for method, _, _ in state.requests] == ["initialize", "notifications/initialized", "tools/list", "tools/call"] + for method, headers, request in state.requests: + assert headers["X-Client-Key"] == "example" + assert headers["Accept"] == "application/json, text/event-stream" + assert request["jsonrpc"] == "2.0" + if method != "initialize": + assert headers["Mcp-Session-Id"] == "session-1" + assert headers["Mcp-Protocol-Version"] == "2025-03-26" + assert Permission().check(tools[0].name, {}) is not None + + +def test_sse_multiline_unicode_and_notifications_without_waiting_for_eof(tmp_path, endpoint): + endpoint[1].sse = True + with ThreadPoolExecutor() as executor: + tools = executor.submit(configured_tools, tmp_path, endpoint).result(timeout=2) + result = executor.submit(tools[0].execute, text="merhaba 世界").result(timeout=2) + assert result == "echo: merhaba 世界" + + +def test_parallel_http_calls_keep_their_responses(tmp_path, endpoint): + tool = configured_tools(tmp_path, endpoint)[0] + with ThreadPoolExecutor() as executor: + results = list(executor.map(lambda text: tool.execute(text=text), ["one", "two", "three"])) + assert results == ["echo: one", "echo: two", "echo: three"] + + +@pytest.mark.parametrize("mode, message", [ + ("rpc-error", "bad arguments"), ("tool-error", "echo:"), + ("bad-json", "transport failed"), ("bad-type", "Content-Type"), + ("not-object", "JSON object"), ("wrong-id", "matching reply"), ("server-error", "HTTP 500"), +]) +def test_http_failures_are_mcp_errors(tmp_path, endpoint, mode, message): + tool = configured_tools(tmp_path, endpoint)[0] + endpoint[1].mode = mode + with pytest.raises(MCPError, match=message): + tool.execute(text="hello") + + +def test_expired_session_never_replays_a_tool_and_explicit_reconnect_recovers(tmp_path, endpoint): + tool = configured_tools(tmp_path, endpoint)[0] + state = endpoint[1] + state.mode = "expired" + with pytest.raises(MCPError, match="HTTP 404"): + tool.execute(text="one") + with pytest.raises(MCPError, match="HTTP 404"): + tool.execute(text="two") + assert sum(method == "tools/call" for method, _, _ in state.requests) == 1 + assert state.sessions == 1 + state.mode = "ok" + tool._client.reconnect() + assert state.sessions == 2 + assert tool.execute(text="again") == "echo: again" + assert state.requests[-1][1]["Mcp-Session-Id"] == "session-2" + + +def test_http_call_read_timeout(tmp_path, endpoint): + tool = configured_tools(tmp_path, endpoint)[0] + endpoint[1].mode = "timeout" + tool._client.call_timeout = 0.05 + with pytest.raises(MCPError, match="transport failed"): + tool.execute(text="hello") + assert sum(method == "tools/call" for method, _, _ in endpoint[1].requests) == 1 + + +def test_sse_keepalives_cannot_extend_the_call_budget(tmp_path, endpoint): + tool = configured_tools(tmp_path, endpoint)[0] + endpoint[1].mode = "heartbeats" + tool._client.call_timeout = 0.05 + with pytest.raises(MCPError, match="timed out"): + tool.execute(text="hello") + + +@pytest.mark.parametrize("status", [200, 405]) +def test_http_close_terminates_session_once(tmp_path, endpoint, status): + tool = configured_tools(tmp_path, endpoint)[0] + endpoint[1].delete_status = status + tool._client.close() + tool._client.close() + deletes = [headers for method, headers, _ in endpoint[1].requests if method == "DELETE"] + assert len(deletes) == 1 + assert deletes[0]["Mcp-Session-Id"] == "session-1" + with pytest.raises(MCPError, match="closed"): + tool.execute(text="hello") + + +def test_stateless_server_needs_no_session_header_or_delete(tmp_path, endpoint): + endpoint[1].stateful = False + tool = configured_tools(tmp_path, endpoint)[0] + assert tool.execute(text="hello") == "echo: hello" + tool._client.close() + assert all("Mcp-Session-Id" not in headers for _, headers, _ in endpoint[1].requests) + assert all(method != "DELETE" for method, _, _ in endpoint[1].requests) + + +def test_http_redirect_is_not_followed_with_configured_headers(tmp_path, endpoint): + tool = configured_tools(tmp_path, endpoint)[0] + endpoint[1].mode = "redirect" + with pytest.raises(MCPError, match="HTTP 301"): + tool.execute(text="hello") + assert endpoint[1].redirects == 0 + + +@pytest.mark.parametrize("url, headers", [("file:///tmp/socket", None), ("http://", None), + ("https://example.test/mcp", "bad"), + ("https://example.test/mcp", {"X-Token": 1})]) +def test_invalid_http_config_is_skipped_with_a_warning(tmp_path, caplog, url, headers): + path = tmp_path / "mcp.json" + path.write_text(json.dumps({"mcpServers": {"remote": {"url": url, "headers": headers}}})) + with caplog.at_level(logging.WARNING): + assert load_mcp_tools(path) == [] + assert len(caplog.records) == 1 + assert "remote" in caplog.text + + +def test_unreachable_http_server_reports_transport_error(): + # Port zero is reserved and cannot have a listener. + with pytest.raises(MCPError, match="transport failed"): + HTTPMCPClient("offline", "http://127.0.0.1:0/mcp") + + +@pytest.mark.parametrize("failure", ["version", "notification"]) +def test_failed_initialization_terminates_its_partial_session(tmp_path, endpoint, failure): + state = endpoint[1] + if failure == "version": + state.protocol = "2099-01-01" + else: + state.notify_status = 400 + assert configured_tools(tmp_path, endpoint) == [] + assert mcp._live_clients == [] + assert state.requests[-1][0] == "DELETE" + assert state.requests[-1][1]["Mcp-Session-Id"] == "session-1"