haoshan98 · Sep 30, 2026 · Sep 30, 2026 · Sep 30, 2026
@@ -27,6 +27,8 @@ This repository is Rust-first under the `vllm-project` GitHub organization.
 ├── TERMINOLOGY.md              # Normative project vocabulary
 ├── crates/agentic-server/       # Axum binary, transport handlers, and configuration
 ├── crates/agentic-server-core/  # Protocol types, execution, tools, and persistence
+├── crates/agentic-llm-d/        # Split-execution state backend for the llm-d coordinator
+├── crates/agentic-cli-docs/     # Build-time generator for the CLI reference docs
 ├── crates/agentic-praxis/       # Praxis integration
 ├── python/agentic_api/          # Python distribution, diagnostics, and launcher
 ├── tests/python/                # Python package and CLI tests

@@ -574,7 +574,7 @@ call inference, run the tool loop, persist. `agentic-server` never reaches past
 
 `multi_agent.enabled` selects gateway orchestration; it is not forwarded to the
 upstream model. Enabled requests require `store: true`; the current validator also
-rejects `reasoning.summary` and `reasoning.generate_summary`. These are gateway
+rejects `max_tool_calls`, `reasoning.summary`, and `reasoning.generate_summary`. These are gateway
 validation rules, not a claim that every upstream provider has the same restrictions.
 New requests without multi-agent configuration use single-agent execution;
 continuations of a stored agent tree inherit its configuration.

@@ -81,23 +81,6 @@ All notable changes to Agentic API are documented here.
   `crawl_timeout`, and `boost_domains` arguments are ignored. Rejected credentials and HTTP 429 responses fail the
   `web_search_call` without an automatic retry, naming the key variable or the upstream `Retry-After` value and never
   echoing the secret. Each Brave `metadata[]` entry carries `"provider": "brave"`.
-- Added Tavily as a selectable backend for the gateway-owned `web_search` tool (#327, Phase 3 of #291). Select it
-  with `AGENTIC_WEB_SEARCH_PROVIDER=tavily` or `[web_search] provider = "tavily"` and supply `TAVILY_API_KEY`; the
-  endpoint defaults to `https://api.tavily.com` and can be overridden with `AGENTIC_WEB_SEARCH_BASE_URL` or
-  `[web_search] base_url`. Each query is one `POST /search` with a JSON body and a bearer token; the key is never
-  placed in the body. `allowed_domains` / `blocked_domains` and the model's `include_domains` / `exclude_domains` are
-  forwarded to Tavily's native `include_domains` / `exclude_domains` and re-checked client-side, `count` is clamped
-  to Tavily's maximum of 20, `freshness` maps to `time_range` or to `start_date` / `end_date` widened by one day on
-  each side because Tavily's bounds are exclusive and sent with `filter_by_published_date` so undated results cannot
-  bypass the requested recency window, `language` keeps Tavily's documented compound tags (`zh-cn`) and
-  otherwise reduces to its primary subtag, `safesearch` maps to the boolean `safe_search`, and `country` plus the
-  You.com-specific
-  `livecrawl`, `livecrawl_formats`, `crawl_timeout`, and `boost_domains` arguments are ignored. Results fill
-  `results.web` with `published_date` as `page_age`; `results.news` stays empty because a second news search per
-  query would double credit usage. Rejected credentials, HTTP 429, and Tavily's 432/433 plan-limit statuses fail the
-  `web_search_call` without an automatic retry, naming the key variable or the upstream `Retry-After` value and never
-  echoing the secret. Each Tavily `metadata[]` entry carries `"provider": "tavily"`, Tavily's `request_id` as
-  `search_uuid`, and its `response_time` as `latency`. Tavily inherits the gateway concurrency limit.
 - Added `[web_search] max_concurrent_queries` and `AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES` to cap concurrent
   provider requests inside one batched search. Brave defaults to `1` for its free-plan rate limit; You.com keeps
   inheriting `max_concurrent_gateway_calls`. The effective ceiling is the smallest of the gateway limit, this
@@ -140,11 +123,6 @@ All notable changes to Agentic API are documented here.
   uses it. With `provider` unset, You.com behavior, configuration, and model-facing output are unchanged; a generated
   `config.toml` now records `provider = "you"` and leaves `api_key_env` unset so provider switches select the matching
   default credential variable.
-- `WebSearchProviderKind` gains a `Tavily` variant; `WebSearchProviderKind::ALL` is now a `&'static [Self]` slice
-  listing all three providers, so adding a provider no longer changes its type; and
-  `WebSearchHandler::from_config` builds the Tavily provider for it (#327). The shared `null_as_default` and
-  `read_response_limited` helpers moved from `tool/web_search/mod.rs` to `tool/web_search/provider.rs`; both were and
-  remain crate-private, so no public API changed.
 
 ### Fixed
 

@@ -8,7 +8,7 @@
 
 **The stateful, agentic API layer for [vLLM](https://github.com/vllm-project/vllm), written in Rust 🦀**
 
-*Run OpenAI-grade agentic workloads (Responses API, server-side tools, Codex) on your own GPUs.*
+*Run OpenAI-grade agentic workloads (Responses and Messages APIs, server-side tools, Codex, Claude Code) on your own GPUs.*
 
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
 [![Rust](https://img.shields.io/badge/Rust-1.85%2B-orange.svg?logo=rust)](Cargo.toml)
@@ -27,14 +27,14 @@ vLLM gives you state-of-the-art inference throughput. But real agentic applicati
 
 ```mermaid
 flowchart LR
-    C(["🧑‍💻 Client<br/>Codex · SDKs · curl"]) -->|"📮 <code>POST /v1/responses</code><br/>🌐 HTTP&nbsp;&nbsp;📡 SSE&nbsp;&nbsp;🔌 WebSocket"| A
+    C(["🧑‍💻 Client<br/>Codex · Claude Code · SDKs · curl"]) -->|"📮 <code>POST /v1/responses</code> · <code>/v1/messages</code><br/>🌐 HTTP&nbsp;&nbsp;📡 SSE&nbsp;&nbsp;🔌 WebSocket"| A
     subgraph A ["⚡ Agentic API (Rust 🦀)"]
         direction TB
         S["🔄 State hydration<br/><code>previous_response_id</code>"]
-        T["🛠️ Server-side tools<br/>web search · functions"]
-        P["💾 Persistence<br/>SQLite response store"]
+        T["🛠️ Server-side tools<br/>web search · MCP · code interpreter"]
+        P["💾 Persistence<br/>SQLite · PostgreSQL"]
     end
-    A -->|"🚀 <code>POST /v1/responses</code><br/>⚙️ stateless&nbsp;&nbsp;🤝 OpenAI-compatible"| V(["🚀 vLLM core<br/>inference engine"])
+    A -->|"🚀 <code>POST /v1/responses</code> · <code>/v1/messages</code><br/>⚙️ stateless&nbsp;&nbsp;🤝 OpenAI/Anthropic-compatible"| V(["🚀 vLLM core<br/>inference engine"])
 
     classDef client fill:#FFE8B3,stroke:#F59E0B,stroke-width:2px,color:#7C2D12
     classDef inner fill:#E0E7FF,stroke:#6366F1,stroke-width:2px,color:#312E81
@@ -53,12 +53,14 @@ flowchart LR
 
 ## ✨ Key Features
 
-- 🔄 **Stateful conversations**: the server manages history via `previous_response_id`. No client-side message tracking, no replaying full transcripts.
-- 🛠️ **Server-side tool execution**: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via [You.com](https://you.com), [Brave Search](https://brave.com/search/api/), or [Tavily](https://tavily.com), and the model executes multi-step tool chains automatically.
-- 📡 **Every transport**: non-streaming HTTP, server-sent events for token streaming, and full **WebSocket** support for interactive clients.
-- 🧰 **Codex-ready**: accepts Codex-shaped Responses traffic out of the box, preserving the tool declarations and response item shapes Codex depends on.
-- 🏃 **Background execution**: fire-and-forget requests that keep processing server-side.
-- ✅ **Compatibility tested**: validated against the [Open Responses](https://www.openresponses.org/) compatibility suite, with replay-cassette tests for real OpenAI and vLLM traffic.
+- 🔄 **Stateful conversations**: the server manages history via `previous_response_id` or the [Conversations API](#-api-surface). No client-side message tracking, no replaying full transcripts.
+- 🛠️ **Server-side tool execution**: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via [You.com](https://you.com), [Brave Search](https://brave.com/search/api/), or [Tavily](https://tavily.com), alongside MCP tools and an opt-in [embedded code interpreter](#optional-embedded-code-interpreter); the model executes multi-step tool chains automatically.
+- 🤝 **Multi-agent orchestration**: a stored Responses request can spawn and coordinate subagents server-side over HTTP, returning their attributed work in one response ([details](#-multi-agent-responses)).
+- 📡 **Every transport**: non-streaming HTTP, server-sent events for token streaming, and full **WebSocket** support for interactive clients, with `stream_id` multiplexing.
+- 🧰 **Codex and Claude Code ready**: accepts Codex-shaped Responses traffic and Anthropic Messages traffic, preserving the tool declarations and item shapes each client depends on.
+- 🗜️ **Compaction**: automatic and explicit context compaction through `/v1/responses/compact` ([guide](docs/guides/responses-compaction.md)).
+- 🔭 **Observability**: opt-in OpenTelemetry traces and metrics over OTLP ([guide](docs/deploying/observability.md)).
+- ✅ **Compatibility tested**: validated against the [Open Responses](https://www.openresponses.org/) compatibility suite, with replay-cassette tests for real OpenAI, vLLM, SGLang, and NVIDIA Dynamo traffic.
 
 ## 🧭 API Surface
 
@@ -67,13 +69,18 @@ flowchart LR
 | `POST /v1/responses` | OpenAI-compatible Responses API with state, tools, and streaming | ✅ |
 | `GET /v1/responses/{response_id}` | Retrieve a locally stored response | ✅ |
 | `GET /v1/responses` | WebSocket transport for the Responses API | ✅ |
-| `POST /v1/conversations` | Conversation management | ✅ |
+| `POST /v1/responses/compact` | Compact direct input or a stored response chain | ✅ |
+| `/v1/conversations` · `/v1/conversations/{id}/items` | Conversations API: create, retrieve, update, and delete conversations and their items | ✅ |
+| `POST /v1/messages` · `POST /v1/messages/count_tokens` | Anthropic Messages API forwarded to the upstream, with a server-side loop for gateway-owned tools | ✅ |
 | `GET /v1/models` | Model listing proxied from vLLM | ✅ |
 | `POST /v1/chat/completions` · `POST /v1/completions` | Forwarded to the upstream verbatim, so clients on those endpoints keep working when the gateway is the entry point | ✅ |
 | `GET /health` · `GET /ready` | Liveness and readiness probes | ✅ |
-| Messages API | Anthropic-style stateful messages on shared primitives | 🚧 Planned |
+| `GET /swagger-ui` · `GET /openapi.json` | Generated OpenAPI spec and Swagger UI, served when `ENABLE_OPENAPI_DOCS=true` | ✅ |
+| Stateful Messages | Messages continuation on the shared persistence primitives | ⏳ Planned |
 | Interactions API | Higher-level agentic workflow surface | ⏳ Planned |
 
+See the [API reference](docs/api/index.md) for routing, authentication, and WebSocket details.
+
 Responses created with `store: true` retain a terminal snapshot for retrieval, including status, usage,
 and this turn's output. Retrieval does not call the upstream model. Unknown IDs return a JSON `404`;
 older records created before snapshot storage (or through history-only APIs) return `409` because their
@@ -329,8 +336,8 @@ parsed. Order of precedence is `--max-request-body-size-bytes`, then `AGENTIC_MA
 file setting.
 
 `api_key_env` names the process environment variable containing the web-search credential; it does not contain the
-credential itself. When it is unset, the selected provider's conventional variable is read (`YOU_API_KEY` or
-`BRAVE_API_KEY`). Newly generated files leave `api_key_env` unset so changing providers also changes the default
+credential itself. When it is unset, the selected provider's conventional variable is read (`YOU_API_KEY`,
+`BRAVE_API_KEY`, or `TAVILY_API_KEY`). Newly generated files leave `api_key_env` unset so changing providers also changes the default
 credential variable. `AGENTIC_WEB_SEARCH_PROVIDER`, `AGENTIC_WEB_SEARCH_BASE_URL`, `AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES`,
 `AGENTIC_MCP_ALLOWED_HOSTS`, `AGENTIC_MAX_REQUEST_BODY_SIZE_BYTES`, and `AGENTIC_MAX_CONCURRENT_GATEWAY_CALLS` can
 override their typed file settings; `YOU_API_BASE_URL` is still honored as the endpoint override when the provider is
@@ -592,19 +599,44 @@ Every tool call has exactly one execution path, so nothing runs by accident:
 
 | Ownership | Who executes it | Examples |
 | --- | --- | --- |
-| **Gateway-owned** | Agentic API executes it server-side and continues the loop | Web search, file search, MCP-backed tools |
-| **Client-owned** | Preserved and returned to the client | Codex shell / editor tools, your functions |
+| **Gateway-owned** | Agentic API executes it server-side and continues the loop | Web search, MCP-backed tools, code interpreter (opt-in) |
+| **Client-owned** | Preserved and returned to the client | Shell tool, Codex editor tools, your functions |
 | **Provider-owned** | Passed through to vLLM or an upstream provider | Provider-native tools |
 
 Unknown or ambiguous tool shapes are **never executed by default**. They are preserved and returned.
 
+## 🤝 Multi-agent Responses
+
+Set `multi_agent.enabled` on a stored Responses request and the gateway runs the [Responses multi-agent](https://developers.openai.com/api/docs/guides/responses-multi-agent) collaboration loop itself: the root agent spawns subagents, each with its own history and tool registry, and the response returns their attributed `multi_agent_call`, `multi_agent_call_output`, and `agent_message` items. Continue the whole agent tree with `previous_response_id`.
+
+```bash
+curl http://localhost:9000/v1/responses \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "Qwen/Qwen3-30B-A3B-FP8",
+    "store": true,
+    "multi_agent": {"enabled": true, "max_concurrent_subagents": 3},
+    "input": "Have one agent review correctness and another review test coverage, then summarize."
+  }'
+```
+
+- `store: true` is required; the gateway does not yet support stateless multi-agent requests.
+- `max_concurrent_subagents` limits active descendant turns (default `3`); it is not a total agent count.
+- Multi-agent execution is HTTP and SSE only. WebSocket requests with multi-agent enabled are rejected.
+- `max_tool_calls` and reasoning summaries are rejected while multi-agent is enabled.
+
+See [HTTP multi-agent execution](ARCHITECTURE.md#http-multi-agent-execution) for scheduling and persistence details.
+
 ## 🏗️ Repository Layout
 
 ```
 crates/
-├── agentic-server/       # Axum binary, transport handlers (HTTP/SSE/WS), configuration
+├── agentic-server/       # Axum binary, transport handlers (HTTP/SSE/WS), configuration, `agentic` CLI
 ├── agentic-server-core/  # Protocol types, executor, tool framework, persistence
+├── agentic-llm-d/        # Split-execution state backend for the llm-d coordinator
+├── agentic-cli-docs/     # Build-time generator for the CLI reference docs
 └── agentic-praxis/       # Praxis gateway integration
+python/agentic_api/       # Python distribution and launcher for the packaged gateway
 docs/                     # MkDocs documentation, ADRs, and design notes
 ```
 
@@ -631,10 +663,14 @@ Design and migration decisions are tracked as ADRs in [docs/adr/](docs/adr/), wi
 
 - [x] **Responses API hydration**: stateful continuation with `previous_response_id`
 - [x] **Codex support**: practical Codex sessions through the Responses API
-- [x] **Server-side tool execution**: explicit ownership, web search built in
-- [ ] **Messages API**: built on the same persistence and execution primitives
+- [x] **Server-side tool execution**: explicit ownership, web search, MCP, and code interpreter built in
+- [x] **Messages API**: Claude Code support with a server-side gateway tool loop
+- [x] **Conversations API**: OpenAI-compatible conversation and item management
+- [ ] **Multi-agent Responses**: HTTP execution ships today; WebSocket injection and stateless requests are next
+- [ ] **File search**: Files and Vector Stores APIs with a gateway-executed `file_search` tool
+- [ ] **Stateful Messages**: Messages continuation on the same persistence and execution primitives
 - [ ] **Interactions API**: durable, higher-level agentic workflows
-- [ ] **Production hardening**: storage backends, observability, cached-prefix continuation
+- [ ] **Production hardening**: tenant-scoped state, gateway metrics, cached-prefix continuation
 
 ______________________________________________________________________
 

@@ -84,6 +84,7 @@ Initial and expected tool areas include:
 
 - File search.
 - Web search.
+- Code interpreter.
 - Web fetch.
 - Computer use.
 - Shell tools.
@@ -94,6 +95,10 @@ Initial and expected tool areas include:
 
 The third major API goal is Messages.
 
+Status: `/v1/messages` forwards Anthropic Messages traffic to the upstream and
+runs a server-side loop for gateway-owned tools, which is enough for Claude
+Code. Stateful continuation and persistence for Messages remain the work below.
+
 Messages should build on the same persistence, rehydration, and execution
 foundations as Responses, while exposing the API surface expected by agentic
 clients that organize state around message objects.
@@ -191,9 +196,18 @@ independent WebSocket lanes, async calls, and steering have distinct semantics.
   ([#334](https://github.com/vllm-project/agentic-api/issues/334)).
 - **Multi-agent orchestration:** implement isolated agent contexts and shared
   scheduling through the existing tracker
-  ([#298](https://github.com/vllm-project/agentic-api/issues/298)), including HTTP
-  execution ([#299](https://github.com/vllm-project/agentic-api/issues/299)) and
-  WebSocket injection ([#300](https://github.com/vllm-project/agentic-api/issues/300)).
+  ([#298](https://github.com/vllm-project/agentic-api/issues/298)). HTTP execution
+  for stored requests has shipped
+  ([#299](https://github.com/vllm-project/agentic-api/issues/299)); WebSocket
+  injection ([#300](https://github.com/vllm-project/agentic-api/issues/300)) and
+  stateless requests ([#393](https://github.com/vllm-project/agentic-api/issues/393))
+  remain.
+- **File search:** add the Files and Vector Stores APIs, indexed retrieval, and a
+  gateway-executed `file_search` tool with OpenAI-compatible streaming events
+  ([#34](https://github.com/vllm-project/agentic-api/pull/34)). Resolve `file_id`
+  references in Responses input against the same file store
+  ([#356](https://github.com/vllm-project/agentic-api/issues/356)). Until then,
+  `file_search` declarations are accepted but not offered to the model.
 - **Computer use:** add typed computer calls, ordered actions, screenshots, and
   explicit execution-location handling through the existing feature issue
   ([#171](https://github.com/vllm-project/agentic-api/issues/171)).