diff --git a/.github/workflows/ai-fix.yml b/.github/workflows/ai-fix.yml index 8829306..bcb0d14 100644 --- a/.github/workflows/ai-fix.yml +++ b/.github/workflows/ai-fix.yml @@ -96,8 +96,8 @@ jobs: set -euo pipefail npm ci npm run build - # Pinned to track @opencode-ai/sdk ^1.18.29 used by this repo. - npm i -g opencode-ai@1.18.30 + # Pinned to match Dockerfile ARG OPENCODE_VERSION. + npm i -g opencode-ai@1.18.35 # No-auth localhost gateway: empty API_KEY = no auth; the gateway # spawns its own backend (anonymous out-of-box free quota). API_KEY= OPENCODE_PROXY_MANAGE_BACKEND=true OPENCODE_SERVER_PASSWORD=ci-local-only \ diff --git a/.github/workflows/ai-review.yml b/.github/workflows/ai-review.yml index abeed25..b4e9ae8 100644 --- a/.github/workflows/ai-review.yml +++ b/.github/workflows/ai-review.yml @@ -131,8 +131,8 @@ jobs: set -euo pipefail npm ci npm run build - # Pinned to track @opencode-ai/sdk ^1.18.29 used by this repo. - npm i -g opencode-ai@1.18.30 + # Pinned to match Dockerfile ARG OPENCODE_VERSION. + npm i -g opencode-ai@1.18.35 # No-auth localhost gateway: empty API_KEY = no auth; the gateway # spawns its own backend (anonymous out-of-box free quota). API_KEY= OPENCODE_PROXY_MANAGE_BACKEND=true OPENCODE_SERVER_PASSWORD=ci-local-only \ diff --git a/.github/workflows/ai-scan.yml b/.github/workflows/ai-scan.yml index c88ab15..7a208cd 100644 --- a/.github/workflows/ai-scan.yml +++ b/.github/workflows/ai-scan.yml @@ -87,8 +87,8 @@ jobs: set -euo pipefail npm ci npm run build - # Pinned to track @opencode-ai/sdk ^1.18.29 used by this repo. - npm i -g opencode-ai@1.18.30 + # Pinned to match Dockerfile ARG OPENCODE_VERSION. + npm i -g opencode-ai@1.18.35 # No-auth localhost gateway: empty API_KEY = no auth; the gateway # spawns its own backend (anonymous out-of-box free quota). API_KEY= OPENCODE_PROXY_MANAGE_BACKEND=true OPENCODE_SERVER_PASSWORD=ci-local-only \ diff --git a/CHANGELOG.md b/CHANGELOG.md index 2d6e221..8ac9103 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **Undici security upgrade (`undici@8.12.3`)**: Aligned with Node.js October 2026 Security Releases (October 21, 2026), upgrading `undici` from `^8.10.2` to `^8.12.3` to fix unbounded buffering SSE stream resource exhaustion DoS and TLS certificate verification bypass; runtime deployment recommended on Node.js `v24.22.1+` (LTS) / `v22.23.4+` (LTS) / `v26.12.1+` (fixing Brotli RCE CVE-2026-111497, TLS hostname verification bypass CVE-2026-106069 / CVE-2026-105483 / CVE-2026-106072, Keep-alive socket out-of-order response confusion CVE-2026-105481, and dns.resolve NUL truncation CVE-2026-105482). + - Undici 安全升级 (`undici@8.12.3`)：對齊 Node.js 2026 年 10 月安全通告（October 21, 2026 Security Releases），升級 `undici` 從 `^8.10.2` 至 `^8.12.3`（修復 SSE 串流未含換行符時的無限緩衝資源耗盡 DoS 漏洞與 TLS 憑證驗證旁路問題）。執行環境建議升級至修復版 Node.js `v24.22.1+` (LTS) / `v22.23.4+` (LTS) / `v26.12.1+`（修復 Brotli RCE CVE-2026-111497、TLS 主機名稱校驗旁路 CVE-2026-106069 / CVE-2026-105483 / CVE-2026-106072、Keep-alive socket 錯序回應混淆 CVE-2026-105481 與 dns.resolve NUL 截斷 CVE-2026-105482）。 - **独立维护声明**：README 致谢明确直接上游 `TiaraBasori/opencode2api`，LICENSE 追加上游归属，CHANGELOG 声明分叉点。 - **仓库去耦合**：文档克隆/Issue 链接与 Docker 镜像指向 `samson910022/OpenCode2API`，`package.json` 补全维护元数据。 - **Responses 示例模型**：`README`、`docs/api-reference.md`、`docs/getting-started.md` 的 `/v1/responses` 示例统一为 `opencode/muse-spark-1.3-contributor-free`（解析逻辑无需改动，裸名与带前缀均兼容）。 @@ -22,6 +24,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Scoped Idempotency-Key replay for continuations**: `/v1/responses` and both Interactions paths now scope keys by authenticated tenant, endpoint, parent ID, and key; the two Interactions aliases share a scope. Different payloads and concurrent in-progress duplicates return distinct 409 errors. Completed JSON and SSE responses replay exactly from the beginning (including the original wire response), with a 60-minute default TTL and a 30-minute minimum. Submitted but unconfirmed outcomes remain unknown and retries only reconcile by polling the recorded child/message—never resubmit the prompt. Continuations fork the full parent session into an isolated child, leaving the parent unchanged. The ledger is process-local and is not shared across workers or restarts. + - `/v1/responses` 与 Interactions 兩個路徑新增依租戶、端點、父 ID 與 key 隔離的 Idempotency-Key 重播；兩個 Interactions 別名共用 scope。不同 payload 與並行中的重複請求分別回傳 409。已完成的 JSON/SSE 會從頭精確重播（含原始線上回應），預設 TTL 為 60 分鐘、最短 30 分鐘。已送出但未確認的結果維持 unknown，重試僅輪詢已記錄的子 session/訊息進行調和，不會重新送出 prompt。續寫會完整 fork 父 session 至隔離子 session，不修改父 session。ledger 僅存於單一程序，不跨 worker 或重啟共享。 - **GitHub AI bot (gateway + CPA dual-channel)**: new `github_bot/` (dual-provider `llm_client` with `xhigh→high` CPA mapping, verbatim thinking/signature passthrough, `response_id` session echo, `_redact_secrets`/`sanitize_public_error_text`; `agent_orchestrator` triage/review/explain/scan/fix-plan pipelines; `repo_scan` deterministic pre-scan; `github_runner` sticky-marker publish + allowlist labels + fingerprint-dedupe auto-issue + opt-in draft PRs) with role-separated primaries (triage `gemini-3.8-flash-high`, safety `grok-4.6`, api-compat `claude-opus-4-6-thinking`, docs `muse-spark-1.3-contributor-free`, explainer `big-pickle`, scanner `grok-composer-2.5-fast`) and cross-channel fallbacks; new workflows `ai-review.yml` / `ai-scan.yml` (weekly cron) / `ai-fix.yml` (`/fix` plans; draft PRs only with opt-in, human review required), each starting its own no-auth in-job gateway (`API_KEY` empty, `OPENCODE_PROXY_MANAGE_BACKEND=true`, anonymous out-of-box quota; external `GATEWAY_BASE_URL` secret overrides) with CPA as optional fallback; `tests/test_ai_bot.py` (34 tests); `docs/AI_BOT.md`. 新增 GitHub AI Bot（自託管 gateway 免費通道 + CPA 通道，定時掃描自動開 issue，`/fix` 規劃；opt-in 才開 draft PR，需人工審查）。 - **N×N protocol translator registry + route wiring**: new pure `src/converters/` matrix (registry/pipeline/formats/translator-types + 12 request + 12 response directed pairs for chat, responses, messages, interactions; stream + non-stream; `fidelity.ts` ledger with 3 full forward edges out of chat and 9 text-core legs; TokenCount intentionally unregistered so counting stays in the collector). Wiring: registry initializes once in `createApp` and is exposed via `ctx.translators`; `src/converters/wire.ts` Safe wrappers enforce error-envelope bypass (wrapped `{error}`, `response.failed`, `error` events, bare `{message,type}` bodies, raw SSE error strings) and per-stream holder reuse; `POST /v1/messages` inbound conversion now goes through the registry (thin wrapper over the same `anthropic.ts` pure layer, adaptive thinking collapsed to the legacy fallback for strict parity). Behavior is otherwise unchanged: wire/SSE/`[DONE]`/error-exit shapes stay route-owned, translated tools still flow through the `external__*` bridge + allowlist, translated usage is client-facing only. New `src/converters/holder.ts` + `wire.ts`, `tests/translator-wire.test.js` (fail-fast, bypass, holder, parity) and `tests/translator-integration.test.js` (live route outputs translate cross-protocol). 新增 N×N 纯函数转换矩阵与 messages 入站接线，线形与工具政策保持不变。 - **Interactions API (Gemini-compatible thin layer)**: new `POST /v1beta/interactions` (alias `POST /v1/interactions`) mapping text + `google_search` grounding onto the Responses pipeline, with `system_instruction` / `previous_interaction_id` / `store:false` (ephemeral) / SSE (`interaction.created` → `step.delta` → `interaction.completed`, no `[DONE]`). 新增 `src/routes/interactions.ts` 与 `tests/interactions.test.js`. Client-executed function tools are rejected with 400 (use `/v1/responses` for those). @@ -33,6 +37,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Streaming external tool call reliability**: replaced lossy delta-only assembly with sequence-based per-part reconciliation (`deltasAfterSnapshot`) in `src/stream/collector.ts` between streaming deltas and authoritative snapshots (`part.text`, `info.parts`), resolving a single authoritative source across Chat, Messages, and Responses without unioning stale raw delta artifacts (`callsFromText` and `repairFromSnapshot`), so lossy deltas never falsely trigger `malformed_external_tool_call` or duplicate/parallel tool errors. Routes perform targeted single snapshot repair, derive visible text in one pass under external tool buffering to eliminate tool-call JSON leaks, preserve whitespace-only deltas in unbuffered responses streaming, reconcile hosted search output with authoritative snapshots when deltas are dropped, and prevent legacy co-occurring snapshot/delta duplication. Replaced invalid JSON `{...}` in router reminder with `{}`. + - 流式外部工具呼叫可靠性重構：修復分塊丟失與快照交錯導致的文本重複與工具呼叫誤報，單一權威來源徹底隔離過期分塊，緩衝模式一次性移除工具標記避免 JSON 洩漏至文字，保留 Responses 串流純空格分塊，補齊託管搜尋快照調和，避免舊版後端並發 delta 重複文字。 +- **SSE framing, keepalives, Anthropic error mapping, and block lifecycle**: keepalive comments across Chat, Messages, Responses, and Interactions now write single newline `: comment\n` (without `\n\n`) to eliminate phantom empty SSE events; added `X-Accel-Buffering: no` header across all streaming routes; open Anthropic `text` and `thinking` blocks are cleanly closed via `content_block_stop` before emitting `event: error`; mapped Messages 404 to Anthropic-compliant `not_found_error` and 415 to `invalid_request_error`; deduplicated Interactions routes and cleaned up client-close listeners; all outer catch handlers close open sockets via `res.end()` with wire-compliant framing; stop keepalive intervals cleanly on stream abort/finally. + - SSE 幀格式硬化：keepalive 修正為單換行避免產生空事件，新增 `X-Accel-Buffering: no`，Anthropic 區塊在報錯前確實關閉，Messages 404 對齊 `not_found_error` 且 415 對齊 `invalid_request_error`，Interactions 路由去重與監聽清理，外層 catch 確保斷開連線不卡死，串流結束或中斷時確實清理 keepalive 計時器。 +- **Body-parser error handling, Express hardening, and provider coalescing**: added pure HTTP error classifier (`src/errors/http.ts`) and terminal 4-argument Express error middleware to return protocol-shaped JSON (400 for bad JSON, 413 for oversized body, 415 for unsupported media type) instead of HTML stack traces; mounted auth middleware before body parsing with trailing slash normalization (allowing `/health/`, `/health/details/`, `/metrics/` probes to bypass auth) to block unauthenticated payload processing; hardened urlencoded body parser to 1MB with `extended: false` against prototype pollution; disabled `X-Powered-By` header; added `ENV NODE_ENV=production` in container runtime stage; coalesced in-flight `getProvidersList` requests to prevent cache stampedes; ensured `getProvidersList` invokes `ensureBackend` when `MANAGE_BACKEND: true` so cold-start or auto-restart is never blocked by pre-lock model resolution; exported `clearBackendReconnectListeners` and exposed `ctx.dispose()`; explicitly return 404 for unknown model resolution. + - 請求主體解析錯誤處理與安全加固：終端錯誤中介軟體統一回傳協定專用 JSON 格式（杜絕 HTML 堆疊洩漏），先鑑權再解析主體（支援結尾斜槓探測放行），收緊 urlencoded 解析器至 1MB 且關閉 extended，移除 X-Powered-By，provider 清單請求合併防擊穿並於冷啟動時自動調用 ensureBackend，提供重連監聽清理介面，未知模型顯式回傳 404。 +- **Streaming 503 retry gaps, continuation history retention, and proxy fallback**: added transient 503 retry loops on Responses stream, Messages stream, and Interactions before output starts, strictly failing closed after partial output; continuation turns (`previous_response_id` / `previous_interaction_id`) now fork the full saved parent session into an isolated child instead of mutating the parent, preserving parent history across retries without creating blank sessions; Responses proxy fallback properly resets attempt error state and polls snapshot output; in-attempt snapshot recovery catches transient 503 errors within the retry budget; strict 4xx filtering (recognizing HTTP 408 Request Timeout alongside 401, 402, and 429 as retryable) and anchored pattern in `isTransientUpstreamError`; continuation terminal fail-closed protection (failing closed when provider inference or prompt call throws/fails on continuation turns to prevent duplicate prompt writes in the isolated child session); re-synchronized `ensureActiveModel` on retry when proxy fallback switches `activeClient` across all routes; moved model resolution outside the process-global `lock()` in Chat and Messages so unknown models fail fast with 404 in milliseconds without waiting on concurrent streaming requests. + - 串流 503 重試填補與鎖分離：Responses 串流、Messages 串流與 Interactions 補齊 503 暫態重試；續答回合（`previous_response_id` / `previous_interaction_id`）現會將完整已保存的父 session fork 至隔離子 session，不再修改父 session，重試時保留父歷史且不建立空白 session；Responses 代理回退清理前次錯誤，上游暫態錯誤嚴格過濾 4xx（將 HTTP 408 請求超時與 401、402、429 共同納入可重試狀態碼）；續答回合在 prompt 調用或上游推論失敗時嚴格 fail-closed 終止，避免在隔離子 session 重複寫入 prompt；重試切換代理時跨路由重新同步 activeModel；模型解析移出全域請求鎖，未知模型毫秒級回傳 404 不再排隊阻塞。 +- **CI backend pins alignment (`opencode-ai@1.18.35`)**: aligned `.github/workflows/ai-fix.yml`, `ai-review.yml`, and `ai-scan.yml` to `opencode-ai@1.18.35` matching `Dockerfile` `ARG OPENCODE_VERSION=1.18.35`, with automated lock tests in `tests/dockerfile-pin.test.js`. CI 工作流與 Dockerfile 固定版本一致對齊至 1.18.35。 - **Zen free-tier `read` gate (`FreeTierError` 403)**: the default backend `read` permission is now an inert ordered gate (`{ '*': 'deny', [FREE_TIER_READ_GATE_PATH]: 'allow' }`, NUL-terminated sentinel) instead of plain `deny`, so OpenCode `Permission.disabled()`'s findLast-by-tool-name check keeps `read` in the upstream `tools[]` (the free tier requires it) while execution-time path matching still denies every real path; the NUL sentinel closes the `Wildcard.match` backslash-normalization bypass, `bash` remains `deny`, and explicit allowlist `read` still emits a real `allow`. The data-level contract is locked by `tests/free-tier-gate.test.js` + `tests/backend-permission.test.js`; source/live verification is still required when bumping the backend pin because unit tests cannot execute OpenCode's installed backend. 默认后端 `read` 权限改为有序惰性闸门（NUL 哨兵 allow 让 `read` 保留在上游 `tools[]`），真实路径仍全部拒绝，`bash` 默认仍为 deny，显式 allowlist `read` 仍是真 allow；该数据层契约由 `tests/free-tier-gate.test.js` 与 `tests/backend-permission.test.js` 锁定，升级后端固定版本时仍需源码与真实 Zen 验证（单测无法执行镜像内安装的后端）。 - **Backend tool-hang + per-request dispose fixes**: the proxy no longer `PUT config {activeModel}` on every request — native `ConfigHttpApi.update` disposes the whole backend project instance unconditionally (no changed-check), aborting in-flight sessions into the `Aborted` retry-loop storm; `ensureActiveModel` (`src/backend/active-model.ts`) PUTs only on actual model change per backend client (failures never cached, existing catch semantics preserved). Separately, the headless backend can no longer `ask` (nobody approves, so prompts hung until the 180s timeout) nor silently execute unlisted tools (the free-tier strip omits the prompt tools map, so the backend fell back to agent defaults = all on): backend `opencode.json` now carries a generated deny-all `permission` map (`src/backend/backend-permission.ts`, single source for `entrypoint.sh` via `dist` with static fallback and `src/backend/manager.ts` isolated path) allowing only explicitly configured internal tools, with `external_directory` always jailed to `/home/node/project/**`. 后端不再每请求 dispose（模型变更才 PUT activeModel）；headless 后端不再 `ask` 卡死、不再静默执行未授权工具（deny-all，仅放行明确配置的 internal 工具）。Locked by `tests/active-model.test.js` + `tests/backend-permission.test.js` + `tests/tool-protocol-parity.test.js`. - **Anonymous first-party backend posture + Zen free-tier gate handling**: spawned backend pins `OPENCODE_CLIENT` to a known first-party value (`cli` default; `desktop`/`acp`/`app` pass through), strips the `OPENCODE_API_KEY="public"` footgun (anonymous uses empty), and git-inits the jail workspace best-effort (5s timeout, warn-and-continue) as defense-in-depth for session-dir sensitivity. Prompt `tools` maps have `false` entries stripped (true-only sent; all-false omitted) only for likely-free `opencode` models (`-free` suffix, `big-pickle`, `union-alpha`) to dodge the `FreeTierError` 403 (any `false` gates); stripped tools fall back to agent defaults server-side while paid/other providers keep the exact hard-disable map. The gate (`...used from within OpenCode` / `FreeTierError`) now surfaces as 403 `permission_denied` fail-fast (no retry, no proxy fallback) with false-positive guards for plain quota text. Jail stdout no longer prints absolute paths; interactions route gains text-level tool-markup stripping parity with chat/responses/messages. New `tests/backend-identity.test.js` / `tests/first-party-gate.test.js` / `tests/prompt-tool-overrides.test.js` / `tests/session-ids.test.js`. No new proxy knob (no config matrix change). diff --git a/Dockerfile b/Dockerfile index bdad248..b9e6858 100644 --- a/Dockerfile +++ b/Dockerfile @@ -50,6 +50,8 @@ RUN npm ci --omit=dev COPY --from=builder /build/dist ./dist RUN chown -R node:node /home/node/project +ENV NODE_ENV=production + EXPOSE 10000 10001 # Config env defaults stay empty on purpose: a non-empty default would win over diff --git a/README.md b/README.md index ed3dde1..7b51eb8 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ Turn a local [OpenCode](https://opencode.ai) runtime into an OpenAI-, Anthropic- | 🔑 **Multi-key auth** | `API_KEY` + `OPENCODE_API_KEYS` merge; `Bearer` or `x-api-key`; empty skips auth | | 🔀 **Fallback proxy** | free-limit 429-only `OPENCODE_UPSTREAM_PROXIES` rotation | | 📡 **Streaming** | SSE for Chat, Responses, Messages, Interactions | +| 🔄 **Continuation idempotency** | Scoped `Idempotency-Key` replay for `/v1/responses` and `/v1/interactions` | | 🧠 **Reasoning control** | `reasoning_effort` / `reasoning: {"effort": "high"}` | | 🐳 **Docker deploy** | One-command with auto backend | | 🛡️ **Tool safety** | Disabled by default | @@ -141,13 +142,13 @@ Use each client's native endpoint; cross-protocol conversion is best-effort — ### Model name formats -Formats: `opencode/big-pickle`, `gpt5-nano` (auto-resolved) and `opencode/gpt5-nano` — see [API Reference](./docs/api-reference.md). +Formats: `opencode/big-pickle`, `gpt5-nano` (auto-resolved) and `opencode/gpt5-nano` — see [API Reference](./docs/api-reference.md). When a requested model is not found on the upstream provider, the proxy fails fast with HTTP 404 (`model_not_found`) and returns `available_models` without waiting on downstream request queues. --- ## 🔧 Troubleshooting -Hangs: try `OPENCODE_USE_ISOLATED_HOME=false`. Missing model: check `/v1/models`. No reasoning: use Responses API with `stream: true` + `reasoning.effort`. +Hangs: try `OPENCODE_USE_ISOLATED_HOME=false`. Missing model: check `/v1/models` (requests for unknown models fail fast with HTTP 404 `model_not_found`). No reasoning: use Responses API with `stream: true` + `reasoning.effort`. > 📖 Full guide: [Troubleshooting](./docs/troubleshooting.md) diff --git a/README.zh-CN.md b/README.zh-CN.md index 3742d66..8152a78 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -25,6 +25,7 @@ | 🔑 **多 Key 认证** | `API_KEY` 与 `OPENCODE_API_KEYS` 合并；`Bearer` 或 `x-api-key`；为空免认证 | | 🔀 **免费限流 fallback 代理** | 仅免费限流 429 时轮换 `OPENCODE_UPSTREAM_PROXIES` | | 📡 **流式输出** | Chat、Responses、Messages、Interactions 全支持 SSE | +| 🔄 **Continuation 幂等重播** | `/v1/responses` 与 `/v1/interactions` 支持作用域隔离的 `Idempotency-Key` | | 🧠 **推理控制** | `reasoning_effort` / `reasoning: {"effort": "high"}` | | 🐳 **Docker 部署** | 一键部署，自动启动后端 | | 🛡️ **工具安全** | 默认禁用工具调用 | @@ -141,13 +142,13 @@ curl -N -X POST http://127.0.0.1:10000/v1/responses \ ### 模型名称格式 -支持 `opencode/big-pickle`、`gpt5-nano`（自动解析）与 `opencode/gpt5-nano`；详见 [API 参考文档](./docs/api-reference.md)。 +支持 `opencode/big-pickle`、`gpt5-nano`（自动解析）与 `opencode/gpt5-nano`；详见 [API 参考文档](./docs/api-reference.md)。若请求的上游 Provider 中未找到指定模型，代理会毫秒级快速失败并返回 HTTP 404（`model_not_found`）及可用模型列表（`available_models`），避免排队阻塞后续请求。 --- ## 🔧 故障排查 -卡住：试 `OPENCODE_USE_ISOLATED_HOME=false`。模型缺失：检查 `/v1/models`。无推理：用 `stream: true` + `reasoning.effort` 的 Responses API。 +卡住：试 `OPENCODE_USE_ISOLATED_HOME=false`。模型缺失：检查 `/v1/models`（未知模型会快速返回 HTTP 404 `model_not_found`）。无推理：用 `stream: true` + `reasoning.effort` 的 Responses API。 > 📖 完整指南：[故障排查](./docs/troubleshooting.md) diff --git a/docs/api-reference.md b/docs/api-reference.md index b00f126..864b52c 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -205,7 +205,7 @@ curl -X POST http://127.0.0.1:10000/v1/responses \ }' ``` -非流式 `responses` 响应会在 `response.output` 中返回 `type: "function_call"` 项；流式模式会发送 function_call 生命周期和参数增量事件。`previous_response_id` 续写只保留最近 128 条工具调用元数据（超出时丢弃最旧、保留最新；按 response ID 链作用域保存，内存态、30min TTL + 60s 清扫，重启动失）；`function_call_output` 的 `call_id` 不在 state 且无 `name` 时返回 400 `unknown_tool_call_id`。 +非流式 `responses` 响应会在 `response.output` 中返回 `type: "function_call"` 项；流式模式会发送 function_call 生命周期和参数增量事件。`previous_response_id` 续写会将完整父 session fork 为隔离子 session，不会向父 session 追加本轮消息；工具调用元数据只保留最近 128 条（超出时丢弃最旧、保留最新；按 response ID 链作用域保存，内存态、30min TTL + 60s 清扫，重启动失）。`function_call_output` 的 `call_id` 不在 state 且无 `name` 时返回 400 `unknown_tool_call_id`。 **服务端联网搜索（`web_search`）:** @@ -237,10 +237,28 @@ POST /v1/interactions | `tools` | array | - | 仅支持 `{type: "google_search"}`（等价 `web_search`），其余 function 工具返回 400 | | `previous_interaction_id` | string | - | 续写上轮会话（等价 responses 的 `previous_response_id`） | | `stream` | boolean | - | SSE：`interaction.created` / `step.delta` / `interaction.completed`（无 `[DONE]`，错误为 `{type:'error'}` 事件；15s heartbeat） | -| `store` | boolean | - | `false` 则不持久化；仅删除本请求新建的会话，复用的父会话保留（注意：带 `previous_interaction_id` 的 `store:false` 轮次仍会追加到父会话历史，并非完全无痕） | +| `store` | boolean | - | `false` 则不持久化；仅删除本请求新建或 fork 的子会话，父会话保留且不会被本轮修改 | 响应为 `Interaction` 资源：`{id, status, model, output_text, steps[], usage: {grounding_tool_count}}`，其中 `steps` 含 `google_search_call{queries}`、`google_search_result{sources}`（标注为 opencode websearch 代理结果，非 Google 原生）、`model_output{text, annotations}`。文本往返与 `web_search` 接地逻辑复用 Responses 管线；限流错误的重试与代理 fallback 和其余路由一致（engage + 换 session，最多 `maxAttempts` 次），普通错误直接返回。 +### Continuation Idempotency and Retries / Continuation 幂等与重试 + +`POST /v1/responses`, `POST /v1beta/interactions`, and alias `POST /v1/interactions` support an optional `Idempotency-Key` request header. Clients should supply a unique key per logical operation and retry after timeouts or disconnections using the **same key and identical JSON request body**. The idempotency scope is isolated by authenticated tenant (a cryptographic digest of the API key; unauthenticated requests share an anonymous tenant when auth is disabled), target endpoint, parent continuation ID (`previous_response_id` / `previous_interaction_id`), and key. The two Interactions paths (`/v1beta/interactions` and `/v1/interactions`) map to the same endpoint and share idempotency records. Records are never shared between Responses and Interactions, across distinct parent IDs, or between different tenant API keys. + +- **Payload conflicts & in-progress handling**: Reusing an existing key in the same scope with a different payload returns HTTP `409` with error code `idempotency_key_reused`. Concurrent duplicate requests for an in-flight prompt return a retryable HTTP `409` with code `idempotency_in_progress`. Request bodies are compared via canonical JSON normalization, so object property order does not affect the payload hash. +- **Exact replay from beginning**: Confirmed completed JSON and SSE responses are cached and **replayed exactly from the beginning**, preserving the original HTTP status code, relevant headers, and the full response body. SSE retry replay is not resumable delta streaming—it replays the complete wire byte stream and terminal events from start to finish. +- **Unconfirmed prompt submission & reconciliation**: If a prompt was submitted to the upstream backend but the outcome has not yet been confirmed, the record is marked as unknown. Retrying with the same key will only poll and reconcile the recorded child session and message; it will never resubmit the prompt or spawn an unintended child session. If confirmation is still pending during a retry, the proxy returns HTTP `503 idempotency_reconciliation_pending`; clients should retry later with the same key and payload. +- **Isolated child session fork**: Continuation turns specifying a parent continuation ID fork the entire saved parent session into an isolated child session. Turns are appended to the child session while the parent session remains completely unmodified, allowing multiple independent conversational branches. +- **In-memory process-local ledger**: The idempotency ledger is held **strictly in process memory**. Completed or definitively failed replayable records are retained for a 60-minute default TTL (minimum retention floor is 30 minutes; currently not user-configurable). In-progress records do not expire via TTL; disconnections prior to prompt submission are abandoned, while disconnections after prompt submission transition to `unknown`. `unknown` entries never expire automatically and are freed only upon successful reconciliation or process restart. Persistent unreconciled entries may count against per-tenant capacity limits until restart. All records are discarded on server restart and are not shared across multi-process clusters or replicas. This mechanism prevents duplicate execution within a single server process lifecycle, but does not replace persistent distributed idempotency storage. + +`POST /v1/responses`、`POST /v1beta/interactions` 与别名 `POST /v1/interactions` 支持可选的 `Idempotency-Key` 请求头。建议每个逻辑操作使用唯一 key，并在超时或断线后用**相同 key 与相同 JSON 请求体**重试。作用域按认证租户（API key 的摘要；未启用鉴权时共用匿名租户）、端点、父 ID（`previous_response_id` / `previous_interaction_id`）及 key 隔离；两个 Interactions 路径是同一端点，故共享幂等记录。Responses 与 Interactions 之间、不同父 ID 之间、不同 API key 租户之间不会共享记录。 + +- 同一作用域内 key 对应不同 payload 时返回 `409`，错误码 `idempotency_key_reused`；同一请求仍在处理中时返回可重试的 `409 idempotency_in_progress`。请求体按 JSON 规范化后比较，对象属性顺序不影响 payload hash。 +- 已确认完成的 JSON 或 SSE 响应会缓存并**从头精确重播**：包括原 HTTP 状态、相关响应头与完整响应 body。SSE 重试不是从断点续传，而是重放原始流的全部字节及终止事件。 +- 如果 prompt 已提交但结果尚未确认，结果会标记为未知。使用相同 key 重试只会轮询/调和已记录的子 session 与消息；不会再次提交 prompt，也不会因为未确认而创建另一个子 session。尚未确认时可能收到 `503 idempotency_reconciliation_pending`，稍后继续用原 key 和原 payload 查询。 +- 带 continuation parent ID 的新操作会从完整父 session fork 一个隔离子 session；本轮写入 child，不修改 parent。这样允许多个分支独立续写。 +- Idempotency ledger 是**进程内存态**。已完成/确定失败的可重播记录目前固定保留 60 分钟（代码保留时长下限为 30 分钟，当前没有配置项）；未完成请求中的 `in_progress` 记录不会按 TTL 淘汰，断线且尚未提交 prompt 时会被放弃，已提交但结果未确认时则转为 `unknown`。`unknown` 记录不会自动过期或淘汰，只能在成功调和后释放，无法调和时会保留到服务重启，并可能使该租户后续新 key 遇到容量限制。服务重启会丢失全部记录；多进程/多副本之间不共享记录。因此它能防止同一进程生命周期内的重复执行，不能替代跨重启或分布式的持久化幂等存储。 + ### 🧭 推荐提示模板（OpenClaw / Claude Code） 在真实运行环境中，如果你希望第一跳**稳定先产出 tool call**，推荐把“调用工具”和“基于工具结果继续回答”拆成两步，而不是混在同一句里。 @@ -284,7 +302,13 @@ Great, now answer the original request using the tool result. > 这个两段式模式是当前 `opencode2api` 外部工具桥接的推荐集成方式。 -### 🌊 流式工具调用 +### 🌊 流式输出与反向代理缓冲 / Streaming & Reverse Proxy Buffering + +所有流式接口（`POST /v1/chat/completions`、`POST /v1/messages`、`POST /v1/responses`、`POST /v1/interactions` 及别名 `POST /v1beta/interactions`）在开启 `stream: true` 时，均会自动附加 `X-Accel-Buffering: no` 响应头。该响应头指示反向代理（如 Nginx、Cloudflare、Caddy 等）立即关闭响应缓冲（response buffering），防止中间代理积攒数据块，确保 Server-Sent Events (SSE) 分块数据与保活注释行（`: comment\n`）能够实时无延迟地推送至客户端。 + +All streaming routes (`POST /v1/chat/completions`, `POST /v1/messages`, `POST /v1/responses`, `POST /v1/interactions`, and alias `POST /v1beta/interactions`) automatically emit the `X-Accel-Buffering: no` HTTP header when `stream: true` is enabled. This header instructs reverse proxies (such as Nginx, Cloudflare, Caddy) to disable response buffering so Server-Sent Events (SSE) chunks and keepalive comments are delivered to clients immediately without intermediate buffering lag. + +#### 流式工具调用示例 / Streaming Tool Call Example ```bash curl -N -X POST http://127.0.0.1:10000/v1/responses \ @@ -318,6 +342,7 @@ curl -N -X POST http://127.0.0.1:10000/v1/responses \ - Chat Completions 会在 `chat.completion.chunk` 中返回 `delta.tool_calls` - Responses API 会返回 `response.output_item.added`、`response.function_call_arguments.delta`、`response.function_call_arguments.done`、`response.output_item.done` 等事件 - Messages API 会返回 `message_start`、`content_block_start`、`content_block_delta`（`text_delta`/`thinking_delta`/`input_json_delta`）、`content_block_stop`、`message_delta`、`message_stop`（无 `[DONE]`） +- Interactions API 会在 SSE 中返回 `interaction.created`、`step.delta`、`interaction.completed`（无 `[DONE]`） > 注意：代理内部会使用命名空间隔离同名工具，但这些内部名称不会作为公开 API 返回给客户端。 diff --git a/package-lock.json b/package-lock.json index 659c006..9d6780d 100644 --- a/package-lock.json +++ b/package-lock.json @@ -14,7 +14,7 @@ "body-parser": "^1.20.3", "cors": "^2.8.5", "express": "^4.21.2", - "undici": "^8.10.2" + "undici": "^8.12.3" }, "devDependencies": { "@swc/core": "^1.16.2", @@ -6956,9 +6956,9 @@ } }, "node_modules/undici": { - "version": "8.10.2", - "resolved": "https://registry.npmjs.org/undici/-/undici-8.10.2.tgz", - "integrity": "sha512-/y4/bH9YNU5hi9NIrpOuvGXFcxrj3CMrV+/AYpowAYTpHn8gX/XPFjNy766FPoYY0miQhdW977JFWKGNhBdwyQ==", + "version": "8.12.3", + "resolved": "https://registry.npmjs.org/undici/-/undici-8.12.3.tgz", + "integrity": "sha512-+fdndUTwBP86cVjihTp21hZHo7d6oj1c/47XPA3lL8mM0XHs+C4EkcjmgRsFnFk2f/UONGDy9ItBItmXsLOQHw==", "license": "MIT", "engines": { "node": ">=22.19.0" diff --git a/package.json b/package.json index 7b68f2a..e06ed5c 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "body-parser": "^1.20.3", "cors": "^2.8.5", "express": "^4.21.2", - "undici": "^8.10.2" + "undici": "^8.12.3" }, "devDependencies": { "@swc/core": "^1.16.2", diff --git a/src/backend/manager.ts b/src/backend/manager.ts index 0913358..d8eac82 100644 --- a/src/backend/manager.ts +++ b/src/backend/manager.ts @@ -354,6 +354,35 @@ if (process.platform !== 'win32') { // Backend management state (per-instance) export const backendState = new Map(); +type BackendReconnectListener = () => void; +const backendReconnectListeners = new Set(); + +export function onBackendReconnect(listener: BackendReconnectListener): () => void { + backendReconnectListeners.add(listener); + return () => { + backendReconnectListeners.delete(listener); + }; +} + +/** Exposed for narrow lifecycle assertions; listeners remain owned by their app instances. */ +export function getBackendReconnectListenerCount(): number { + return backendReconnectListeners.size; +} + +export function notifyBackendReconnect(): void { + for (const listener of backendReconnectListeners) { + try { + listener(); + } catch { + // ignore + } + } +} + +export function clearBackendReconnectListeners(): void { + backendReconnectListeners.clear(); +} + /** * Backend Lifecycle Management */ @@ -572,6 +601,7 @@ export async function ensureBackend(config: unknown): Promise { // A fresh backend reverts to its default model: drop cached // set-active-model state so the next request PUTs again. resetActiveModelCache(); + notifyBackendReconnect(); // Handle spawn errors state.process.on('error', (spawnErr: unknown) => { diff --git a/src/converters/anthropic.ts b/src/converters/anthropic.ts index d0aa754..74b7510 100644 --- a/src/converters/anthropic.ts +++ b/src/converters/anthropic.ts @@ -396,3 +396,15 @@ export function sseEvent(event: string, payload: unknown): string { export function estimateTokens(text: unknown): number { return Math.ceil(String(text || '').length / 4); } + +export function toAnthropicErrorType(statusCode: number, upstreamType?: string): string { + if (statusCode === 404 || upstreamType === 'model_not_found') return 'not_found_error'; + if (statusCode === 400 || upstreamType === 'invalid_request_error') return 'invalid_request_error'; + if (statusCode === 401 || upstreamType === 'invalid_api_key') return 'authentication_error'; + if (statusCode === 403 || upstreamType === 'permission_denied') return 'permission_error'; + if (statusCode === 413) return 'request_too_large'; + if (statusCode === 415) return 'invalid_request_error'; + if (statusCode === 429 || upstreamType === 'rate_limit_exceeded') return 'rate_limit_error'; + if (statusCode === 529) return 'overloaded_error'; + return 'api_error'; +} diff --git a/src/errors/http.ts b/src/errors/http.ts new file mode 100644 index 0000000..32944de --- /dev/null +++ b/src/errors/http.ts @@ -0,0 +1,107 @@ +// HTTP error classification and protocol-shaped error bodies for request/body-parser errors. +import { asRecord } from '../utils/guards.js'; + +export interface AnthropicHttpErrorBody { + type: 'error'; + error: { + type: 'invalid_request_error' | 'api_error' | 'request_too_large'; + message: string; + code?: string; + }; +} + +export interface OpenAiHttpErrorBody { + error: { + message: string; + type: 'invalid_request_error' | 'server_error'; + code?: string; + }; +} + +export interface ClassifiedHttpError { + status: number; + statusCode: number; + body: AnthropicHttpErrorBody | OpenAiHttpErrorBody; +} + +export function isAnthropicMessagesPath(path?: string): boolean { + if (!path || typeof path !== 'string') return false; + const normalized = (path.split('?')[0]?.replace(/\/+$/, '') ?? '').toLowerCase(); + return normalized === '/v1/messages'; +} + +/** + * Pure HTTP error classifier. + * Maps body-parser and request-level errors to safe HTTP statuses and protocol-shaped bodies. + * NEVER echoes err.body, err.message, stack traces, internal class names, or filesystem paths. + */ +export function classifyHttpError(err: unknown, path?: string): ClassifiedHttpError { + const record = asRecord(err); + const rawType = record['type']; + const errType = typeof rawType === 'string' ? rawType : ''; + const rawStatus = record['status'] ?? record['statusCode']; + const statusNum = + typeof rawStatus === 'number' + ? rawStatus + : typeof rawStatus === 'string' + ? Number(rawStatus) + : null; + const isSyntaxError = err instanceof SyntaxError || record['name'] === 'SyntaxError'; + + let status: number; + let message: string; + let code: string | undefined; + + if (errType === 'entity.parse.failed' || isSyntaxError || (statusNum === 400 && 'body' in record)) { + status = 400; + message = 'Malformed JSON in request body'; + code = 'invalid_json'; + } else if (errType === 'entity.too.large' || statusNum === 413) { + status = 413; + message = 'Request body exceeds the maximum permitted size'; + code = 'entity_too_large'; + } else if (errType === 'media.type.unsupported') { + status = 415; + message = 'Unsupported or missing Content-Type; expected a JSON media type'; + code = 'unsupported_media_type'; + } else if (errType === 'charset.unsupported' || errType === 'encoding.unsupported' || statusNum === 415) { + status = 415; + message = 'Unsupported body encoding or charset'; + code = 'unsupported_media_type'; + } else if (errType === 'entity.verify.failed') { + status = 400; + message = 'Request body verification failed'; + code = undefined; + } else { + status = 500; + message = 'Internal server error'; + code = undefined; + } + + const isAnthropic = isAnthropicMessagesPath(path); + const errorType = isAnthropic + ? (status === 413 ? 'request_too_large' : status >= 500 ? 'api_error' : 'invalid_request_error') + : (status >= 500 ? 'server_error' : 'invalid_request_error'); + const body: AnthropicHttpErrorBody | OpenAiHttpErrorBody = isAnthropic + ? { + type: 'error', + error: { + type: errorType as AnthropicHttpErrorBody['error']['type'], + message, + ...(code ? { code } : {}), + }, + } + : { + error: { + message, + type: errorType as OpenAiHttpErrorBody['error']['type'], + ...(code ? { code } : {}), + }, + }; + + return { + status, + statusCode: status, + body, + }; +} diff --git a/src/errors/upstream.ts b/src/errors/upstream.ts index ca13334..c70900f 100644 --- a/src/errors/upstream.ts +++ b/src/errors/upstream.ts @@ -8,6 +8,7 @@ function readStringField(record: Record, key: string): string | } function collectMessageParts(error: unknown): string { + if (typeof error === 'string') return error; const record = asRecord(error); const data = asRecord(record['data']); const parts: string[] = []; @@ -27,7 +28,7 @@ function readStatusCode(error: unknown): number | null { const n = Number(direct.trim()); if (Number.isFinite(n) && n > 0) return n; } - const candidates: unknown[] = [record['message'], data['message']]; + const candidates: unknown[] = [typeof error === 'string' ? error : null, record['message'], data['message']]; for (const candidate of candidates) { if (typeof candidate !== 'string') continue; const m = candidate.match(/^\s*(\d{3})\s*:/); @@ -149,11 +150,10 @@ export function isFreeUsageLimitError(error: unknown): boolean { export function isTransientUpstreamError(error: unknown): boolean { if (!error) return false; const message = collectMessageParts(error); - if (!message) return false; // Upstream opencode never retries context overflow (retry.ts); neither do we. // A 500 carrying a context message must surface immediately, not burn retries. - if (/context.?length|context.?overflow|context_length_exceeded/i.test(message)) return false; + if (message && /context.?length|context.?overflow|context_length_exceeded/i.test(message)) return false; // First-party gate is permanent: retrying (or proxy-fallbacking) can never // succeed, and a Python Traceback body might otherwise trip a generic @@ -161,20 +161,27 @@ export function isTransientUpstreamError(error: unknown): boolean { // "from within") so incidental phrasing like "originated from within the // model context" never matches; FreeTierError is the structured class seen // live in Zen response bodies (collectMessageParts covers name/code/type). - if (/from within opencode|used from within|freetiererror/i.test(message)) return false; + if (message && /from within opencode|used from within|freetiererror/i.test(message)) return false; - const record = asRecord(error); - const data = asRecord(record['data']); - // Provider-marked retryable (SDK ApiError.data.isRetryable) is transient, - // whatever the status code. Explicit false defers to the checks below. - if (record['isRetryable'] === true || data['isRetryable'] === true) return true; + const status = readStatusCode(error); + if (typeof status === 'number' && status >= 400 && status < 500) { + if (status !== 401 && status !== 402 && status !== 408 && status !== 429) { + return false; + } + } // Genuine auth failures must surface immediately: a misconfigured key // should not burn the full backoff. The issue-#5 mislabeled billing case // ("Insufficient balance"/CreditsError) never carries these strings, so - // it still retries. Placed after isRetryable so a provider-explicit - // retryable flag keeps winning (upstream-faithful). - if (/invalid[_\s]?api[_\s]?key|unauthorized|authentication (failed|error)|api key (expired|invalid|incorrect)/i.test(message)) return false; + // it still retries. + if (message && /invalid[_\s]?api[_\s]?key|unauthorized|authentication (failed|error)|api key (expired|invalid|incorrect)/i.test(message)) return false; + + const record = asRecord(error); + const data = asRecord(record['data']); + // Provider-marked retryable (SDK ApiError.data.isRetryable) is transient, + // whatever the status code (unless filtered out by 4xx / auth check above). + // Explicit false defers to the checks below. + if (record['isRetryable'] === true || data['isRetryable'] === true) return true; const transientSignatures: RegExp[] = [ /insufficient balance/i, @@ -206,31 +213,18 @@ export function isTransientUpstreamError(error: unknown): boolean { /retry your request/i, /resource exhausted/i, /(?:currently|temporarily) at capacity/i, + /^\s*endpoint is unavailable\.?\s*$/i, ]; - if (transientSignatures.some((re) => re.test(message))) return true; - // Upstream errors arrive as ": {json}" strings; SDK errors may also - // carry a numeric status on the object itself. Only accept a leading - // ":" prefix — a loose \b(\d{3}): match misfires on embedded - // numbers like "retry after 300: ...". - const recordStatus: unknown = record['statusCode'] ?? data['status']; - let status: number | null = typeof recordStatus === 'number' ? recordStatus : null; - if (typeof status !== 'number') { - const statusCandidates: string[] = []; - const msg: unknown = record['message']; - const dataMsg: unknown = data['message']; - if (typeof msg === 'string') statusCandidates.push(msg); - if (typeof dataMsg === 'string') statusCandidates.push(dataMsg); - for (const candidate of statusCandidates) { - const prefixMatch = candidate.match(/^\s*(\d{3})\s*:/); - if (prefixMatch?.[1]) { - status = Number(prefixMatch[1]); - break; - } - } - } + const msgParts: string[] = []; + if (message) msgParts.push(message); + if (typeof record['message'] === 'string' && record['message'] !== message) msgParts.push(record['message']); + if (typeof data['message'] === 'string' && data['message'] !== message) msgParts.push(data['message']); + + if (msgParts.some((text) => transientSignatures.some((re) => re.test(text)))) return true; + if (typeof status === 'number') { - if (status === 401 || status === 402 || status === 429) return true; + if (status === 401 || status === 402 || status === 408 || status === 429) return true; if (status >= 500) return true; } return false; diff --git a/src/idempotency/middleware.ts b/src/idempotency/middleware.ts new file mode 100644 index 0000000..90108c0 --- /dev/null +++ b/src/idempotency/middleware.ts @@ -0,0 +1,613 @@ +import type { NextFunction, Request, RequestHandler, Response } from 'express'; +import { + canonicalJsonHash, + CAPTURE_CHUNK_OVERHEAD_BYTES, + IdempotencyRegistry, + idempotencyDigest, + idempotencyHeaderStorageBytes, + MAX_CAPTURE_CHUNKS, + type IdempotencyRecord, + type IdempotencyWireResponse, +} from './registry.js'; + +const IDEMPOTENCY_HEADER = 'idempotency-key'; +const MAX_KEY_BYTES = 256; +const TARGET_ENDPOINTS = new Set(['/v1/responses', '/v1beta/interactions', '/v1/interactions']); +const ANONYMOUS_TENANT = 'anonymous'; + +export interface IdempotencyRequestControl { + readonly enabled: true; + readonly reconciling: boolean; + readonly submitted: boolean; + readonly resultConfirmed: boolean; + readonly confirmedAssistantMessageId: string | null; + readonly childSessionId: string | null; + readonly userMessageId: string | null; + readonly model: string | null; + readonly publicId: string | null; + markSubmitted(childSessionId: string, userMessageId: string, model?: string): void; + setModel(model: string): void; + setPublicId(publicId: string): void; + confirmResult(userMessageId: string, assistantMessageId: string): void; + onCompleted(callback: () => void | Promise): void; +} + +interface IdempotencyRequest extends Request { + idempotency?: IdempotencyRequestControl; + continuationTenantId?: string; +} + +interface IdempotencyMiddlewareOptions { + registry: IdempotencyRegistry; + configuredKeys: readonly string[]; + matchedKeyIndex: (req: Request) => number; +} + +function jsonError(res: Response, statusCode: number, code: string, message: string, retryable = false): void { + res.status(statusCode).json({ + error: { + type: 'idempotency_error', + code, + message, + ...(retryable ? { retryable: true } : {}), + }, + }); +} + +function idempotencyHeaderCount(req: Request): number { + const rawHeaders = (req as unknown as { rawHeaders?: unknown }).rawHeaders; + if (!Array.isArray(rawHeaders)) return 0; + let count = 0; + for (let index = 0; index + 1 < rawHeaders.length; index += 2) { + if (typeof rawHeaders[index] === 'string' && (rawHeaders[index] as string).toLowerCase() === IDEMPOTENCY_HEADER) { + count += 1; + } + } + return count; +} + +function requestKey(req: Request): { present: boolean; key: string | null; invalid: boolean } { + const count = idempotencyHeaderCount(req); + const header = req.headers[IDEMPOTENCY_HEADER]; + const present = count > 0 || header !== undefined; + if (!present) return { present: false, key: null, invalid: false }; + if (count > 1 || typeof header !== 'string') return { present: true, key: null, invalid: true }; + const key = header.trim(); + if (!key || Buffer.byteLength(key, 'utf8') > MAX_KEY_BYTES || /[\u0000-\u001f\u007f]/.test(key)) { + return { present: true, key: null, invalid: true }; + } + return { present: true, key, invalid: false }; +} + +function endpointPath(req: Request): string { + const pathname = req.path.replace(/\/+$/, '').toLowerCase() || '/'; + if (pathname === '/v1beta/interactions' || pathname === '/v1/interactions') return '/v1/interactions'; + return pathname; +} + +function parentIdFor(endpoint: string, body: unknown): string { + const record = body && typeof body === 'object' && !Array.isArray(body) ? (body as Record) : {}; + const value = endpoint === '/v1/responses' ? record['previous_response_id'] : record['previous_interaction_id']; + return typeof value === 'string' ? value : ''; +} + +function captureHeaders( + res: Response, + reserveHeaderBytes: (bytes: number) => boolean, +): Record | null { + const headers = res.getHeaders(); + const excluded = new Set([ + 'connection', + 'content-length', + 'date', + 'keep-alive', + 'transfer-encoding', + 'upgrade', + ]); + const selected = Object.create(null) as Record; + for (const [name, value] of Object.entries(headers)) { + if (excluded.has(name.toLowerCase()) || value === undefined) continue; + if (typeof value === 'string' || typeof value === 'number' || Array.isArray(value)) { + selected[name] = value as string | number | string[]; + } + } + if (!reserveHeaderBytes(idempotencyHeaderStorageBytes(selected))) return null; + const output = Object.create(null) as Record; + for (const [name, value] of Object.entries(selected)) output[name] = Array.isArray(value) ? [...value] : value; + return output; +} + +function chunkByteLength(chunk: unknown, encoding?: unknown): number | null { + if (typeof chunk === 'string') { + const selectedEncoding = typeof encoding === 'string' ? (encoding as BufferEncoding) : 'utf8'; + return Buffer.byteLength(chunk, selectedEncoding); + } + if (Buffer.isBuffer(chunk) || chunk instanceof Uint8Array) return chunk.byteLength; + return null; +} + +function copyChunkBytes(chunk: unknown, encoding?: unknown): Buffer | null { + if (typeof chunk === 'string') { + const selectedEncoding = typeof encoding === 'string' ? (encoding as BufferEncoding) : 'utf8'; + return Buffer.from(chunk, selectedEncoding); + } + if (Buffer.isBuffer(chunk)) return Buffer.from(chunk); + if (chunk instanceof Uint8Array) return Buffer.from(chunk); + return null; +} + +interface SseDataRange { + start: number; + end: number; +} + +interface SseDataEventRanges { + previous: SseDataRange | null; + latest: SseDataRange | null; +} + +const MAX_SSE_TERMINAL_DATA_BYTES = 64 * 1024; +const SSE_TERMINAL_VALIDATION_RESERVATION_MULTIPLIER = 16; + +/** Scan SSE line/event boundaries without materializing strings or event arrays. */ +export function scanSseDataEventRanges(body: Buffer): SseDataEventRanges { + let eventData: SseDataRange | null = null; + let previous: SseDataRange | null = null; + let latest: SseDataRange | null = null; + const finishEvent = (): void => { + if (eventData) { + previous = latest; + latest = eventData; + eventData = null; + } + }; + const scanLine = (lineStart: number, lineEnd: number): void => { + if (lineEnd > lineStart && body[lineEnd - 1] === 0x0d) lineEnd -= 1; + if (lineEnd === lineStart) { + finishEvent(); + return; + } + if ( + !eventData && + lineEnd - lineStart >= 5 && + body[lineStart] === 0x64 && + body[lineStart + 1] === 0x61 && + body[lineStart + 2] === 0x74 && + body[lineStart + 3] === 0x61 && + body[lineStart + 4] === 0x3a + ) { + eventData = { start: lineStart + 5, end: lineEnd }; + } + }; + + let lineStart = 0; + for (let index = 0; index < body.byteLength; index += 1) { + if (body[index] !== 0x0a) continue; + scanLine(lineStart, index); + lineStart = index + 1; + } + if (lineStart < body.byteLength) scanLine(lineStart, body.byteLength); + finishEvent(); + return { previous, latest }; +} + +function isDoneData(body: Buffer, range: SseDataRange): boolean { + let { start, end } = range; + while (start < end && isSseWhitespace(body[start])) start += 1; + while (end > start && isSseWhitespace(body[end - 1])) end -= 1; + return end - start === 6 && + body[start] === 0x5b && body[start + 1] === 0x44 && body[start + 2] === 0x4f && + body[start + 3] === 0x4e && body[start + 4] === 0x45 && body[start + 5] === 0x5d; +} + +function isSseWhitespace(byte: number): boolean { + return byte === 0x20 || byte === 0x09 || byte === 0x0a || byte === 0x0d || byte === 0x0b || byte === 0x0c; +} + +function parseSseTerminalEvent( + body: Buffer, + range: SseDataRange, + expectedType: string, + reserveTemporaryBytes: (bytes: number) => boolean, + releaseTemporaryBytes: (bytes: number) => void, +): boolean { + const dataBytes = range.end - range.start; + if (dataBytes < 1 || dataBytes > MAX_SSE_TERMINAL_DATA_BYTES) return false; + // Account for the temporary UTF-8 string and bounded JSON parse tree before + // allocating either. The multiplier is intentionally conservative. + const reservedBytes = dataBytes * SSE_TERMINAL_VALIDATION_RESERVATION_MULTIPLIER; + if (!reserveTemporaryBytes(reservedBytes)) return false; + try { + const eventData = body.toString('utf8', range.start, range.end).trim(); + return (JSON.parse(eventData) as { type?: unknown }).type === expectedType; + } catch { + return false; + } finally { + releaseTemporaryBytes(reservedBytes); + } +} + +function isSuccessfulResponse( + endpoint: string, + response: IdempotencyWireResponse, + reserveTemporaryBytes: (bytes: number) => boolean, + releaseTemporaryBytes: (bytes: number) => void, +): boolean { + if (response.statusCode < 200 || response.statusCode >= 300) return false; + const contentType = String(response.headers['content-type'] ?? '').toLowerCase(); + if (!contentType.includes('text/event-stream')) return true; + const dataEvents = scanSseDataEventRanges(response.body); + if (endpoint === '/v1/responses') { + return Boolean( + dataEvents.latest && + isDoneData(response.body, dataEvents.latest) && + dataEvents.previous && + parseSseTerminalEvent( + response.body, + dataEvents.previous, + 'response.completed', + reserveTemporaryBytes, + releaseTemporaryBytes, + ), + ); + } + return Boolean( + dataEvents.latest && + parseSseTerminalEvent( + response.body, + dataEvents.latest, + 'interaction.completed', + reserveTemporaryBytes, + releaseTemporaryBytes, + ), + ); +} + +function replay(res: Response, response: IdempotencyWireResponse): void { + res.statusCode = response.statusCode; + for (const [name, value] of Object.entries(response.headers)) res.setHeader(name, value); + res.setHeader('Content-Length', response.body.byteLength); + res.end(response.body); +} + +export function isDeterministicPreSubmitFailure(statusCode: number, body?: Buffer): boolean { + if (statusCode < 400 || statusCode >= 500) { + return false; + } + if (statusCode === 408 || statusCode === 429) { + return false; + } + if (body && body.length > 0) { + try { + const parsed = JSON.parse(body.toString('utf8')) as unknown; + if (parsed && typeof parsed === 'object') { + const record = parsed as Record; + const errObj = + record.error && typeof record.error === 'object' + ? (record.error as Record) + : null; + if ( + errObj?.retryable === true || + errObj?.type === 'transient_error' || + record.retryable === true || + record.type === 'transient_error' + ) { + return false; + } + } + } catch { + // Unparseable body falls through to deterministic 4xx return + } + } + return true; +} + +function instrumentResponse( + res: Response, + endpoint: string, + record: IdempotencyRecord, + generation: number, + registry: IdempotencyRegistry, + onCompletedCallbacks: Array<() => void | Promise>, +): void { + const writable = res as unknown as { + write: (...args: unknown[]) => boolean; + end: (...args: unknown[]) => unknown; + }; + const originalWrite = writable.write; + const originalEnd = writable.end; + const chunks: Buffer[] = []; + let capturedBytes = 0; + let capturedReservationBytes = 0; + let captureOverflow = false; + let captureReleased = false; + const capture = (chunk: unknown, encoding?: unknown): void => { + if (captureReleased || captureOverflow || chunk == null || typeof chunk === 'function') return; + if (chunks.length >= MAX_CAPTURE_CHUNKS) { + captureOverflow = true; + return; + } + let byteLength: number | null; + try { + byteLength = chunkByteLength(chunk, encoding); + } catch { + captureOverflow = true; + return; + } + if (byteLength === null) { + captureOverflow = true; + return; + } + if (!registry.reserveCaptureBytes(record, generation, byteLength, CAPTURE_CHUNK_OVERHEAD_BYTES)) { + captureOverflow = true; + return; + } + const reservationBytes = byteLength + CAPTURE_CHUNK_OVERHEAD_BYTES; + let bytes: Buffer | null; + try { + bytes = copyChunkBytes(chunk, encoding); + } catch { + registry.releaseCaptureBytes(record, generation, reservationBytes, byteLength); + captureOverflow = true; + return; + } + if (!bytes || bytes.byteLength !== byteLength) { + registry.releaseCaptureBytes(record, generation, reservationBytes, byteLength); + captureOverflow = true; + return; + } + chunks.push(bytes); + capturedBytes += byteLength; + capturedReservationBytes += reservationBytes; + }; + const releaseCapture = (): void => { + captureReleased = true; + chunks.length = 0; + capturedBytes = 0; + capturedReservationBytes = 0; + registry.releaseCapture(record, generation); + }; + const runCompletedCallbacks = (): void => { + void Promise.allSettled(onCompletedCallbacks.map((callback) => Promise.resolve().then(callback))); + }; + const abandonOrMarkUnknown = (): void => { + registry.releaseCapture(record, generation); + chunks.length = 0; + capturedBytes = 0; + capturedReservationBytes = 0; + if (record.submitted) { + registry.markUnknown(record, generation); + } else { + registry.abandon(record, generation); + runCompletedCallbacks(); + } + }; + + writable.write = function (...args: unknown[]): boolean { + capture(args[0], args[1]); + return originalWrite.apply(res, args); + }; + writable.end = function (...args: unknown[]): unknown { + capture(args[0], args[1]); + return originalEnd.apply(res, args); + }; + + res.once('finish', () => { + captureReleased = true; + if (captureOverflow) { + abandonOrMarkUnknown(); + return; + } + // Buffer.concat allocates an assembly buffer while the chunk buffers are + // still live. Reserve that peak before allocating, then release the chunk + // reservation after dropping those references. The assembled Buffer is + // transferred to the registry without another full-body copy. + const assemblyBytes = capturedBytes; + if (!registry.reserveAssemblyBytes(record, generation, assemblyBytes)) { + abandonOrMarkUnknown(); + return; + } + let body: Buffer; + try { + body = Buffer.concat(chunks, assemblyBytes); + } catch { + abandonOrMarkUnknown(); + return; + } + chunks.length = 0; + registry.releaseCaptureBytes(record, generation, capturedReservationBytes, capturedBytes); + capturedBytes = 0; + capturedReservationBytes = 0; + if (!registry.isCurrent(record, generation)) return; + const headers = captureHeaders( + res, + (bytes) => registry.reserveAssemblyBytes(record, generation, bytes), + ); + if (!headers) { + abandonOrMarkUnknown(); + return; + } + const response: IdempotencyWireResponse = { statusCode: res.statusCode, headers, body }; + if (!record.submitted) { + if (isDeterministicPreSubmitFailure(response.statusCode, response.body)) { + registry.completeCaptured(record, generation, 'failed', response); + runCompletedCallbacks(); + return; + } + if (response.statusCode >= 200 && response.statusCode < 300) { + if (isSuccessfulResponse( + endpoint, + response, + (bytes) => registry.reserveAssemblyBytes(record, generation, bytes), + (bytes) => registry.releaseCaptureBytes(record, generation, bytes), + )) { + const stored = registry.completeCaptured(record, generation, 'completed', response); + if (stored) runCompletedCallbacks(); + return; + } + } + abandonOrMarkUnknown(); + return; + } + const resultConfirmed = record.resultConfirmed && + typeof record.confirmedAssistantMessageId === 'string' && + record.confirmedAssistantMessageId.trim().length > 0; + if (response.statusCode >= 200 && response.statusCode < 300 && !resultConfirmed) { + registry.markUnknown(record, generation); + return; + } + if (isSuccessfulResponse( + endpoint, + response, + (bytes) => registry.reserveAssemblyBytes(record, generation, bytes), + (bytes) => registry.releaseCaptureBytes(record, generation, bytes), + )) { + const stored = registry.completeCaptured(record, generation, 'completed', response); + if (stored) runCompletedCallbacks(); + return; + } + if (resultConfirmed) { + const stored = registry.completeCaptured(record, generation, 'failed', response); + if (stored) runCompletedCallbacks(); + return; + } + // A submitted prompt with no complete success wire result is uncertain + // until a complete assistant result has been observed and confirmed. + registry.markUnknown(record, generation); + }); + + res.once('close', () => { + releaseCapture(); + const response = res as unknown as { writableFinished?: boolean }; + if (!response.writableFinished) { + if (record.submitted) registry.markUnknown(record, generation); + else registry.abandon(record, generation); + } + }); + +} + +/** + * Idempotency-Key middleware for the two stateful continuation APIs. It is + * intentionally installed after auth and JSON parsing, and does nothing at + * all when the header is absent. + */ +export function createIdempotencyMiddleware(options: IdempotencyMiddlewareOptions): RequestHandler { + const { registry, configuredKeys, matchedKeyIndex } = options; + return (req: Request, res: Response, next: NextFunction): void => { + const endpoint = endpointPath(req); + if (req.method !== 'POST' || !TARGET_ENDPOINTS.has(endpoint)) return next(); + + let tenant = ANONYMOUS_TENANT; + const keyIndex = configuredKeys.length > 0 ? matchedKeyIndex(req) : -1; + const configuredKey = keyIndex >= 0 ? configuredKeys[keyIndex] : undefined; + if (typeof configuredKey === 'string') tenant = `key:${idempotencyDigest(configuredKey)}`; + (req as IdempotencyRequest).continuationTenantId = tenant; + + const header = requestKey(req); + if (!header.present) return next(); + if (header.invalid || !header.key) { + jsonError(res, 400, 'invalid_idempotency_key', 'Idempotency-Key must be a single non-empty value no longer than 256 bytes'); + return; + } + + if (configuredKeys.length > 0) { + if (keyIndex < 0 || typeof configuredKey !== 'string') { + jsonError(res, 401, 'unauthorized', 'Unauthorized'); + return; + } + tenant = `key:${idempotencyDigest(configuredKey)}`; + } + + const parentId = parentIdFor(endpoint, (req as IdempotencyRequest).body); + const scopeId = idempotencyDigest(JSON.stringify([ + tenant, + endpoint, + parentId, + idempotencyDigest(header.key), + ])); + let payloadHash: string; + try { + payloadHash = canonicalJsonHash((req as IdempotencyRequest).body); + } catch { + jsonError(res, 400, 'invalid_idempotency_payload', 'The request payload cannot be safely canonicalized'); + return; + } + const prior = registry.get(scopeId); + const priorState = prior?.state ?? null; + const result = registry.begin(scopeId, payloadHash, tenant); + if (result.kind === 'full') { + jsonError( + res, + 503, + 'idempotency_capacity_exceeded', + 'Idempotency capacity for this tenant or the service is temporarily exhausted; retry later', + true, + ); + return; + } + if (result.kind === 'mismatch') { + jsonError(res, 409, 'idempotency_key_reused', 'This Idempotency-Key was already used with a different request payload'); + return; + } + + const record = result.record; + if (result.kind === 'existing' && priorState !== 'unknown') { + if (record.state === 'completed' || record.state === 'failed') { + if (record.wireResponse) { + replay(res, record.wireResponse); + return; + } + jsonError(res, 503, 'idempotency_result_unavailable', 'The original result is unavailable; retry reconciliation is required', true); + return; + } + jsonError(res, 409, 'idempotency_in_progress', 'A request with this Idempotency-Key is already in progress', true); + return; + } + + const reconciling = result.kind === 'existing' && priorState === 'unknown'; + if (reconciling && (!record.childSessionId || !record.userMessageId)) { + registry.markUnknown(record, record.generation); + jsonError(res, 503, 'idempotency_reconciliation_pending', 'The previous outcome is uncertain and cannot yet be reconciled', true); + return; + } + + const callbacks: Array<() => void | Promise> = []; + const generation = record.generation; + const control: IdempotencyRequestControl = { + enabled: true, + reconciling, + get submitted() { return record.submitted; }, + get resultConfirmed() { return record.generation === generation && record.resultConfirmed; }, + get confirmedAssistantMessageId() { + return record.generation === generation ? record.confirmedAssistantMessageId : null; + }, + get childSessionId() { return record.childSessionId; }, + get userMessageId() { return record.userMessageId; }, + get model() { return record.model; }, + get publicId() { return record.publicId; }, + markSubmitted: (childSessionId, userMessageId, model) => { + registry.markSubmitted(record, generation, childSessionId, userMessageId, model); + }, + setModel: (model) => registry.setModel(record, generation, model), + setPublicId: (publicId) => registry.setPublicId(record, generation, publicId), + confirmResult: (userMessageId, assistantMessageId) => + registry.confirmResult(record, generation, userMessageId, assistantMessageId), + onCompleted: (callback) => { + if (registry.isCurrent(record, generation)) callbacks.push(callback); + }, + }; + (req as IdempotencyRequest).idempotency = control; + instrumentResponse(res, endpoint, record, generation, registry, callbacks); + next(); + }; +} + +export function getIdempotencyControl(req: Request): IdempotencyRequestControl | null { + return (req as IdempotencyRequest).idempotency ?? null; +} + +/** Return the matched key's digest tenant, or the stable anonymous tenant when auth is disabled. */ +export function getContinuationTenantId(req: Request): string { + return (req as IdempotencyRequest).continuationTenantId ?? ANONYMOUS_TENANT; +} diff --git a/src/idempotency/registry.ts b/src/idempotency/registry.ts new file mode 100644 index 0000000..1ea580e --- /dev/null +++ b/src/idempotency/registry.ts @@ -0,0 +1,562 @@ +import { createHash } from 'node:crypto'; + +export const MIN_IDEMPOTENCY_TTL_MS = 30 * 60 * 1000; +export const DEFAULT_IDEMPOTENCY_TTL_MS = 60 * 60 * 1000; +export const DEFAULT_IDEMPOTENCY_MAX_ENTRIES = 2_000; +export const DEFAULT_IDEMPOTENCY_MAX_LIVE_ENTRIES_PER_TENANT = 100; +export const DEFAULT_IDEMPOTENCY_MAX_RETAINED_ENTRIES_PER_TENANT = 500; +export const DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES = 64 * 1024 * 1024; +export const DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES = 8 * 1024 * 1024; +export const MAX_CAPTURE_CHUNKS = 1_024; +export const CAPTURE_CHUNK_OVERHEAD_BYTES = 128; +const HEADER_ENTRY_OVERHEAD_BYTES = 64; +const HEADER_ARRAY_OVERHEAD_BYTES = 24; +const HEADER_ARRAY_ITEM_OVERHEAD_BYTES = 16; +export const DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES_PER_TENANT = + 2 * DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES + + MAX_CAPTURE_CHUNKS * CAPTURE_CHUNK_OVERHEAD_BYTES + + 2 * 1024 * 1024; + +export type IdempotencyState = 'in_progress' | 'completed' | 'failed' | 'unknown'; + +export interface IdempotencyWireResponse { + statusCode: number; + headers: Record; + body: Buffer; +} + +/** Conservative bounded estimate of the retained JS header storage in UTF-8 bytes. */ +export function idempotencyHeaderStorageBytes( + headers: Record, +): number { + let bytes = 0; + const add = (value: number): void => { + if (!Number.isSafeInteger(value) || value < 0 || bytes > Number.MAX_SAFE_INTEGER - value) { + bytes = Number.MAX_SAFE_INTEGER; + return; + } + bytes += value; + }; + for (const [name, value] of Object.entries(headers)) { + add(Buffer.byteLength(name, 'utf8') + HEADER_ENTRY_OVERHEAD_BYTES); + if (Array.isArray(value)) { + add(HEADER_ARRAY_OVERHEAD_BYTES); + for (const item of value) { + add(Buffer.byteLength(item, 'utf8') + HEADER_ARRAY_ITEM_OVERHEAD_BYTES); + } + } else { + add(Buffer.byteLength(String(value), 'utf8')); + } + } + return bytes; +} + +/** Body plus retained header storage, used for global and per-tenant cache budgets. */ +export function idempotencyResponseStorageBytes(response: IdempotencyWireResponse): number { + const bodyBytes = response.body.byteLength; + const headerBytes = idempotencyHeaderStorageBytes(response.headers); + return bodyBytes > Number.MAX_SAFE_INTEGER - headerBytes ? Number.MAX_SAFE_INTEGER : bodyBytes + headerBytes; +} + +function copyHeaders(headers: Record): Record { + const copy = Object.create(null) as Record; + for (const [name, value] of Object.entries(headers)) copy[name] = Array.isArray(value) ? [...value] : value; + return copy; +} + +export interface IdempotencyRecord { + readonly scopeId: string; + readonly tenantId: string; + readonly payloadHash: string; + state: IdempotencyState; + createdAt: number; + updatedAt: number; + expiresAt: number; + generation: number; + submitted: boolean; + resultConfirmed: boolean; + confirmedAssistantMessageId: string | null; + childSessionId: string | null; + userMessageId: string | null; + model: string | null; + publicId: string | null; + wireResponse: IdempotencyWireResponse | null; +} + +export type IdempotencyBeginResult = + | { kind: 'created'; record: IdempotencyRecord } + | { kind: 'existing'; record: IdempotencyRecord } + | { kind: 'mismatch'; record: IdempotencyRecord } + | { kind: 'full' }; + +export interface IdempotencyRegistryOptions { + ttlMs?: number; + maxEntries?: number; + maxLiveEntriesPerTenant?: number; + maxRetainedEntriesPerTenant?: number; + maxCachedBytes?: number; + maxCachedBytesPerTenant?: number; + maxResponseBytes?: number; + now?: () => number; +} + +const MAX_CANONICAL_DEPTH = 256; +const MAX_CANONICAL_NODES = 1_000_000; + +function canonicalize(value: unknown, state: { nodes: number }, depth: number): string { + state.nodes += 1; + if (state.nodes > MAX_CANONICAL_NODES || depth > MAX_CANONICAL_DEPTH) { + throw new RangeError('Idempotency payload exceeds canonicalization limits'); + } + if (typeof value === 'number' && !Number.isFinite(value)) { + throw new RangeError('Idempotency payload contains a non-finite number'); + } + if (value === null || typeof value !== 'object') return JSON.stringify(value) ?? 'null'; + if (Array.isArray(value)) return `[${value.map((item) => canonicalize(item, state, depth + 1)).join(',')}]`; + const record = value as Record; + const fields = Object.keys(record) + .sort() + .map((key) => `${JSON.stringify(key)}:${canonicalize(record[key], state, depth + 1)}`); + return `{${fields.join(',')}}`; +} + +/** Canonical JSON makes object property order irrelevant while preserving array order and JSON values. */ +export function canonicalJsonHash(payload: unknown): string { + return createHash('sha256').update(canonicalize(payload, { nodes: 0 }, 0), 'utf8').digest('hex'); +} + +/** Hash secrets and scope components before they are used as a registry key. */ +export function idempotencyDigest(value: string): string { + return createHash('sha256').update(value, 'utf8').digest('hex'); +} + +/** + * A bounded, per-app in-memory idempotency ledger. Active and unknown entries + * are never evicted: doing so could turn an uncertain prompt into a new prompt. + * Completed/failed replay records have a TTL clamped to at least 30 minutes. + */ +export class IdempotencyRegistry { + private readonly entries = new Map(); + private readonly now: () => number; + private readonly ttlMs: number; + private readonly maxEntries: number; + private readonly maxLiveEntriesPerTenant: number; + private readonly maxRetainedEntriesPerTenant: number; + private readonly maxCachedBytes: number; + private readonly maxCachedBytesPerTenant: number; + private readonly maxResponseBytes: number; + private cachedBytes = 0; + private pendingCaptureBytes = 0; + private readonly captureBytesByScope = new Map(); + private readonly responseBytesByScope = new Map(); + private readonly tenantEntrySlots = new Map(); + private readonly retainedEntriesByTenant = new Map(); + private readonly cachedBytesByTenant = new Map(); + private readonly pendingCaptureBytesByTenant = new Map(); + private readonly liveEntriesByTenant = new Map(); + + constructor(options: IdempotencyRegistryOptions = {}) { + this.now = options.now ?? Date.now; + this.ttlMs = Math.max(MIN_IDEMPOTENCY_TTL_MS, options.ttlMs ?? DEFAULT_IDEMPOTENCY_TTL_MS); + this.maxEntries = Math.max(1, Math.floor(options.maxEntries ?? DEFAULT_IDEMPOTENCY_MAX_ENTRIES)); + const perTenantEntryLimit = this.maxEntries > 1 ? this.maxEntries - 1 : this.maxEntries; + this.maxLiveEntriesPerTenant = Math.max( + 1, + Math.min( + perTenantEntryLimit, + Math.floor(options.maxLiveEntriesPerTenant ?? DEFAULT_IDEMPOTENCY_MAX_LIVE_ENTRIES_PER_TENANT), + ), + ); + this.maxRetainedEntriesPerTenant = Math.max( + 1, + Math.min( + perTenantEntryLimit, + Math.floor(options.maxRetainedEntriesPerTenant ?? DEFAULT_IDEMPOTENCY_MAX_RETAINED_ENTRIES_PER_TENANT), + ), + ); + this.maxCachedBytes = Math.max(0, options.maxCachedBytes ?? DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES); + this.maxCachedBytesPerTenant = Math.max( + 0, + Math.min( + this.maxCachedBytes > 1 ? this.maxCachedBytes - 1 : this.maxCachedBytes, + options.maxCachedBytesPerTenant ?? DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES_PER_TENANT, + ), + ); + this.maxResponseBytes = Math.max(0, options.maxResponseBytes ?? DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES); + } + + begin(scopeId: string, payloadHash: string, tenantId: string = 'anonymous'): IdempotencyBeginResult { + this.pruneExpired(); + const existing = this.entries.get(scopeId); + if (existing) { + existing.updatedAt = this.now(); + if (existing.payloadHash !== payloadHash) return { kind: 'mismatch', record: existing }; + if (existing.state === 'unknown') { + // Claim reconciliation synchronously so concurrent retries cannot poll + // or emit a second response for the same submitted prompt. + existing.state = 'in_progress'; + existing.generation += 1; + existing.resultConfirmed = false; + existing.confirmedAssistantMessageId = null; + existing.expiresAt = this.now() + this.ttlMs; + } + return { kind: 'existing', record: existing }; + } + // Reserve a per-tenant slot for every record. It moves from live to + // retained on completion. No record may be evicted before its TTL: a + // completed/failed record must remain replayable, and active/unknown + // records must not permit a duplicate prompt. + if ((this.liveEntriesByTenant.get(tenantId) ?? 0) >= this.maxLiveEntriesPerTenant) return { kind: 'full' }; + if ((this.tenantEntrySlots.get(tenantId) ?? 0) >= this.maxRetainedEntriesPerTenant) return { kind: 'full' }; + if (this.entries.size >= this.maxEntries) return { kind: 'full' }; + + const now = this.now(); + const record: IdempotencyRecord = { + scopeId, + tenantId, + payloadHash, + state: 'in_progress', + createdAt: now, + updatedAt: now, + expiresAt: now + this.ttlMs, + generation: 1, + submitted: false, + resultConfirmed: false, + confirmedAssistantMessageId: null, + childSessionId: null, + userMessageId: null, + model: null, + publicId: null, + wireResponse: null, + }; + this.entries.set(scopeId, record); + this.adjustCount(this.tenantEntrySlots, tenantId, 1); + this.liveEntriesByTenant.set(tenantId, (this.liveEntriesByTenant.get(tenantId) ?? 0) + 1); + return { kind: 'created', record }; + } + + isCurrent(record: IdempotencyRecord, generation: number): boolean { + return this.entries.get(record.scopeId) === record && record.generation === generation && record.state === 'in_progress'; + } + + markSubmitted( + record: IdempotencyRecord, + generation: number, + childSessionId: string, + userMessageId: string, + model?: string, + ): void { + if (!this.isCurrent(record, generation)) { + throw new Error('Cannot submit an abandoned or inactive idempotency request'); + } + record.submitted = true; + record.childSessionId = childSessionId; + record.userMessageId = userMessageId; + record.resultConfirmed = false; + record.confirmedAssistantMessageId = null; + if (model) record.model = model; + this.touch(record); + } + + /** Remove a disconnected request only if it has not submitted its prompt. */ + abandon(record: IdempotencyRecord, generation: number): void { + if (!this.isCurrent(record, generation) || record.submitted) return; + record.generation += 1; + this.entries.delete(record.scopeId); + this.adjustCount(this.tenantEntrySlots, record.tenantId, -1); + this.adjustTenantLiveCount(record.tenantId, -1); + const reservedBytes = this.captureBytesByScope.get(record.scopeId) ?? 0; + this.captureBytesByScope.delete(record.scopeId); + this.responseBytesByScope.delete(record.scopeId); + this.pendingCaptureBytes = Math.max(0, this.pendingCaptureBytes - reservedBytes); + this.adjustPendingTenantBytes(record.tenantId, -reservedBytes); + this.removeCachedResponse(record); + } + + setModel(record: IdempotencyRecord, generation: number, model: string): void { + if (!this.isCurrent(record, generation)) { + throw new Error('Cannot update an abandoned or inactive idempotency request'); + } + record.model = model; + this.touch(record); + } + + setPublicId(record: IdempotencyRecord, generation: number, publicId: string): void { + if (!this.isCurrent(record, generation)) { + throw new Error('Cannot update an abandoned or inactive idempotency request'); + } + record.publicId = publicId; + this.touch(record); + } + + confirmResult(record: IdempotencyRecord, generation: number, userMessageId: string, assistantMessageId: string): void { + if ( + !this.isCurrent(record, generation) || + !record.submitted || + record.userMessageId !== userMessageId || + typeof assistantMessageId !== 'string' || + assistantMessageId.trim().length === 0 + ) { + throw new Error('Cannot confirm an inactive or mismatched idempotency result'); + } + record.resultConfirmed = true; + record.confirmedAssistantMessageId = assistantMessageId; + this.touch(record); + } + + /** Reserve bounded memory for response capture while a route is streaming. */ + reserveCaptureBytes( + record: IdempotencyRecord, + generation: number, + bytes: number, + overheadBytes = 0, + ): boolean { + if ( + !this.isCurrent(record, generation) || + !Number.isFinite(bytes) || + bytes < 0 || + !Number.isFinite(overheadBytes) || + overheadBytes < 0 + ) return false; + const reservedBytes = bytes + overheadBytes; + const current = this.captureBytesByScope.get(record.scopeId) ?? 0; + const responseBytes = this.responseBytesByScope.get(record.scopeId) ?? 0; + if ( + responseBytes + bytes > this.maxResponseBytes || + this.cachedBytes + this.pendingCaptureBytes + reservedBytes > this.maxCachedBytes || + (this.cachedBytesByTenant.get(record.tenantId) ?? 0) + + (this.pendingCaptureBytesByTenant.get(record.tenantId) ?? 0) + reservedBytes > + this.maxCachedBytesPerTenant + ) return false; + this.captureBytesByScope.set(record.scopeId, current + reservedBytes); + this.responseBytesByScope.set(record.scopeId, responseBytes + bytes); + this.pendingCaptureBytes += reservedBytes; + this.adjustPendingTenantBytes(record.tenantId, reservedBytes); + return true; + } + + /** Reserve the second, temporary buffer needed to assemble captured chunks. */ + reserveAssemblyBytes(record: IdempotencyRecord, generation: number, bytes: number): boolean { + if (!this.isCurrent(record, generation) || !Number.isFinite(bytes) || bytes < 0) return false; + const current = this.captureBytesByScope.get(record.scopeId) ?? 0; + if ( + this.cachedBytes + this.pendingCaptureBytes + bytes > this.maxCachedBytes || + (this.cachedBytesByTenant.get(record.tenantId) ?? 0) + + (this.pendingCaptureBytesByTenant.get(record.tenantId) ?? 0) + bytes > + this.maxCachedBytesPerTenant + ) return false; + this.captureBytesByScope.set(record.scopeId, current + bytes); + this.pendingCaptureBytes += bytes; + this.adjustPendingTenantBytes(record.tenantId, bytes); + return true; + } + + releaseCaptureBytes( + record: IdempotencyRecord, + generation: number, + bytes: number, + responseBytes = 0, + ): void { + if ( + !this.isCurrent(record, generation) || + !Number.isFinite(bytes) || + bytes <= 0 || + !Number.isFinite(responseBytes) || + responseBytes < 0 + ) return; + const current = this.captureBytesByScope.get(record.scopeId) ?? 0; + const released = Math.min(current, bytes); + const remaining = current - released; + if (remaining === 0) this.captureBytesByScope.delete(record.scopeId); + else this.captureBytesByScope.set(record.scopeId, remaining); + const capturedResponseBytes = this.responseBytesByScope.get(record.scopeId) ?? 0; + // Body bytes, per-chunk overhead, and temporary validation/assembly bytes + // share the memory reservation but only body bytes count toward response + // size limits. + const releasedResponseBytes = Math.min(capturedResponseBytes, responseBytes, released); + const remainingResponseBytes = capturedResponseBytes - releasedResponseBytes; + if (remainingResponseBytes === 0) this.responseBytesByScope.delete(record.scopeId); + else this.responseBytesByScope.set(record.scopeId, remainingResponseBytes); + this.pendingCaptureBytes = Math.max(0, this.pendingCaptureBytes - released); + this.adjustPendingTenantBytes(record.tenantId, -released); + } + + releaseCapture(record: IdempotencyRecord, generation: number): void { + if (!this.isCurrent(record, generation)) return; + const bytes = this.captureBytesByScope.get(record.scopeId) ?? 0; + this.captureBytesByScope.delete(record.scopeId); + this.responseBytesByScope.delete(record.scopeId); + this.pendingCaptureBytes = Math.max(0, this.pendingCaptureBytes - bytes); + this.adjustPendingTenantBytes(record.tenantId, -bytes); + } + + markUnknown(record: IdempotencyRecord, generation: number): void { + if (!this.isCurrent(record, generation)) return; + this.releaseCapture(record, generation); + this.removeCachedResponse(record); + record.state = 'unknown'; + record.wireResponse = null; + record.resultConfirmed = false; + record.confirmedAssistantMessageId = null; + record.generation += 1; + this.touch(record); + } + + /** + * Store a caller-owned response without retaining an aliased Buffer. The + * registry reserves the clone before allocating it, so temporary storage is + * included in the configured memory bound. + */ + complete( + record: IdempotencyRecord, + generation: number, + state: 'completed' | 'failed', + response: IdempotencyWireResponse, + ): boolean { + if (!this.isCurrent(record, generation)) return false; + const responseBytes = response.body.byteLength; + const storageBytes = idempotencyResponseStorageBytes(response); + if (responseBytes > this.maxResponseBytes || !this.reserveAssemblyBytes(record, generation, storageBytes)) { + if (record.submitted) this.markUnknown(record, generation); + else this.abandon(record, generation); + return false; + } + let ownedResponse: IdempotencyWireResponse; + try { + ownedResponse = { ...response, headers: copyHeaders(response.headers), body: Buffer.from(response.body) }; + } catch { + if (record.submitted) this.markUnknown(record, generation); + else this.abandon(record, generation); + return false; + } + return this.commitComplete(record, generation, state, ownedResponse); + } + + /** Commit an assembled body whose allocation is already reserved by capture. */ + completeCaptured( + record: IdempotencyRecord, + generation: number, + state: 'completed' | 'failed', + response: IdempotencyWireResponse, + ): boolean { + return this.commitComplete(record, generation, state, response); + } + + private commitComplete( + record: IdempotencyRecord, + generation: number, + state: 'completed' | 'failed', + response: IdempotencyWireResponse, + ): boolean { + if (!this.isCurrent(record, generation)) return false; + const priorBytes = record.wireResponse ? idempotencyResponseStorageBytes(record.wireResponse) : 0; + const responseBytes = response.body.byteLength; + const storageBytes = idempotencyResponseStorageBytes(response); + const captureBytes = this.captureBytesByScope.get(record.scopeId) ?? 0; + if ( + responseBytes > this.maxResponseBytes || + this.cachedBytes - priorBytes + this.pendingCaptureBytes - captureBytes + storageBytes > this.maxCachedBytes || + (this.cachedBytesByTenant.get(record.tenantId) ?? 0) - priorBytes + + (this.pendingCaptureBytesByTenant.get(record.tenantId) ?? 0) - captureBytes + storageBytes > + this.maxCachedBytesPerTenant + ) { + if (record.submitted) this.markUnknown(record, generation); + else this.abandon(record, generation); + return false; + } + this.releaseCapture(record, generation); + this.cachedBytes += storageBytes - priorBytes; + this.adjustCachedTenantBytes(record.tenantId, storageBytes - priorBytes); + this.adjustTenantLiveCount(record.tenantId, -1); + this.adjustCount(this.retainedEntriesByTenant, record.tenantId, 1); + record.state = state; + record.wireResponse = response; + this.touch(record); + return true; + } + + get(scopeId: string): IdempotencyRecord | null { + this.pruneExpired(); + return this.entries.get(scopeId) ?? null; + } + + get size(): number { + this.pruneExpired(); + return this.entries.size; + } + + get retainedResponseBytes(): number { + return this.cachedBytes; + } + + get pendingResponseBytes(): number { + return this.pendingCaptureBytes; + } + + getTenantRetainedEntries(tenantId: string): number { + return this.retainedEntriesByTenant.get(tenantId) ?? 0; + } + + getTenantRetainedResponseBytes(tenantId: string): number { + return this.cachedBytesByTenant.get(tenantId) ?? 0; + } + + getTenantPendingResponseBytes(tenantId: string): number { + return this.pendingCaptureBytesByTenant.get(tenantId) ?? 0; + } + + /** A child backing a live or unknown submitted operation must remain pollable. */ + hasUnresolvedChildSession(sessionId: string): boolean { + for (const record of this.entries.values()) { + if ( + record.childSessionId === sessionId && + record.submitted && + (record.state === 'in_progress' || record.state === 'unknown') + ) return true; + } + return false; + } + + private touch(record: IdempotencyRecord): void { + record.updatedAt = this.now(); + record.expiresAt = this.now() + this.ttlMs; + } + + private adjustTenantLiveCount(tenantId: string, delta: number): void { + this.adjustCount(this.liveEntriesByTenant, tenantId, delta); + } + + private adjustCount(map: Map, tenantId: string, delta: number): void { + const count = Math.max(0, (map.get(tenantId) ?? 0) + delta); + if (count === 0) map.delete(tenantId); + else map.set(tenantId, count); + } + + private adjustPendingTenantBytes(tenantId: string, delta: number): void { + this.adjustCount(this.pendingCaptureBytesByTenant, tenantId, delta); + } + + private adjustCachedTenantBytes(tenantId: string, delta: number): void { + this.adjustCount(this.cachedBytesByTenant, tenantId, delta); + } + + private removeCachedResponse(record: IdempotencyRecord): void { + const bytes = record.wireResponse ? idempotencyResponseStorageBytes(record.wireResponse) : 0; + this.cachedBytes = Math.max(0, this.cachedBytes - bytes); + this.adjustCachedTenantBytes(record.tenantId, -bytes); + } + + private pruneExpired(): void { + const now = this.now(); + for (const [scopeId, record] of this.entries) { + // A hung live request must not expire and admit a second prompt. It is + // bounded by maxEntries and transitions when the response closes/ends. + if (record.state === 'in_progress' || record.state === 'unknown') continue; + if (record.expiresAt > now) continue; + this.removeCachedResponse(record); + this.adjustCount(this.retainedEntriesByTenant, record.tenantId, -1); + this.adjustCount(this.tenantEntrySlots, record.tenantId, -1); + this.entries.delete(scopeId); + } + } +} diff --git a/src/proxy.ts b/src/proxy.ts index c44c308..474c138 100644 --- a/src/proxy.ts +++ b/src/proxy.ts @@ -24,14 +24,19 @@ import { buildProxyConfig, DEFAULT_REQUEST_TIMEOUT_MS, } from './config/proxy-config.js'; -import { buildBackendAuthHeaders, ensureBackend, backendState } from './backend/manager.js'; +import { + buildBackendAuthHeaders, + ensureBackend, + backendState, + onBackendReconnect, +} from './backend/manager.js'; import { createCollector } from './stream/collector.js'; import { createProxyPool } from './upstream-proxy/pool.js'; import { registerChatRoutes } from './routes/chat.js'; import { registerResponsesRoutes } from './routes/responses.js'; import { registerMessagesRoutes } from './routes/messages.js'; import { registerInteractionsRoutes } from './routes/interactions.js'; -import { registerSystemRoutes, registerNotFoundRoute } from './routes/system.js'; +import { registerSystemRoutes, registerNotFoundRoute, registerErrorHandler } from './routes/system.js'; import type { ProxyConfig } from './types/config.js'; import type { AppContext, @@ -46,6 +51,10 @@ import { asRecord, toErrorMessage } from './utils/guards.js'; import { buildEffectiveApiKeys, createApiKeyVerifier } from './auth/keys.js'; import { defaultTranslatorRegistry } from './converters/registry.js'; import { ensureTranslatorsRegistered } from './converters/wire.js'; +import { classifyHttpError, isAnthropicMessagesPath } from './errors/http.js'; +import { createIdempotencyMiddleware } from './idempotency/middleware.js'; +import { IdempotencyRegistry } from './idempotency/registry.js'; +import { makeId } from './converters/json.js'; // P4: thin re-exports to preserve original import paths // (tests/env-alias.test.js, stream-hardening.test.js import these from '../src/proxy.js'). @@ -124,15 +133,14 @@ export function createApp(config: ProxyConfig): CreateAppResult { const maxAttempts = maxRetries + 1; const app: Application = express(); + app.disable('x-powered-by'); app.use( cors({ origin: '*', methods: ['GET', 'POST', 'OPTIONS'], - allowedHeaders: ['Content-Type', 'Authorization', 'x-api-key', 'anthropic-version'], + allowedHeaders: ['Content-Type', 'Authorization', 'x-api-key', 'anthropic-version', 'Idempotency-Key'], }), ); - app.use(bodyParser.json({ limit: '50mb' })); - app.use(bodyParser.urlencoded({ limit: '50mb', extended: true })); const clientHeaders = buildBackendAuthHeaders(OPENCODE_SERVER_PASSWORD); const rawClient: OpencodeClient = createOpencodeClient({ baseUrl: OPENCODE_SERVER_URL, headers: clientHeaders }); @@ -145,32 +153,200 @@ export function createApp(config: ProxyConfig): CreateAppResult { // instead of assuming a shared reference. const effectiveApiKeys: string[] = buildEffectiveApiKeys(API_KEY, API_KEYS); const apiKeyVerifier = createApiKeyVerifier(effectiveApiKeys); + const rejectRequestAndClose = (req: Request, res: Response, status: number, body: unknown): void => { + let cleanedUp = false; + const cleanup = (): void => { + if (cleanedUp) return; + cleanedUp = true; + req.off('error', ignoreRequestError); + req.off('close', cleanup); + }; + const ignoreRequestError = (): void => cleanup(); + req.once('error', ignoreRequestError); + req.once('close', cleanup); + res.setHeader('Connection', 'close'); + res.status(status).json(body); + req.resume(); + }; // Auth middleware (accepts Authorization: Bearer and x-api-key for Anthropic SDK compat) + // Runs before bodyParser so unauthenticated requests are rejected immediately without parsing. app.use((req: Request, res: Response, next: NextFunction): void => { - if (req.method === 'OPTIONS' || req.path === '/health' || req.path === '/' || req.path === '/health/details' || req.path === '/metrics') + const normalizedPath = req.path.replace(/\/+$/, '') || '/'; + if ( + req.method === 'OPTIONS' || + normalizedPath === '/health' || + normalizedPath === '/' || + normalizedPath === '/health/details' || + normalizedPath === '/metrics' + ) { return next(); + } if (apiKeyVerifier.keys.length > 0) { const authorized = apiKeyVerifier.isAuthorized( req as unknown as { headers: { authorization?: unknown; 'x-api-key'?: unknown } }, ); if (!authorized) { - if (req.path === '/v1/messages') { - res.status(401).json({ type: 'error', error: { type: 'authentication_error', message: 'Unauthorized' } }); + if (isAnthropicMessagesPath(req.originalUrl || req.path)) { + rejectRequestAndClose(req, res, 401, { + type: 'error', + error: { type: 'authentication_error', message: 'Unauthorized' }, + }); return; } - res.status(401).json({ error: { message: 'Unauthorized' } }); + rejectRequestAndClose(req, res, 401, { error: { message: 'Unauthorized' } }); return; } } next(); }); + // body-parser's JSON parser intentionally skips unsupported media types. Do + // this check after auth but before parsing so unsupported payloads receive a + // protocol-shaped 415 instead of reaching routes as an empty body. + const jsonApiPostPaths = new Set([ + '/v1/chat/completions', + '/v1/messages', + '/v1/responses', + '/v1beta/interactions', + '/v1/interactions', + ]); + const rejectUnsupportedContentType = (req: Request, res: Response): void => { + const classified = classifyHttpError({ type: 'media.type.unsupported' }, req.originalUrl || req.path); + rejectRequestAndClose(req, res, classified.status, classified.body); + }; + app.use((req: Request, res: Response, next: NextFunction): void => { + const normalizedPath = req.path.replace(/\/+$/, '').toLowerCase() || '/'; + if (req.method !== 'POST' || !jsonApiPostPaths.has(normalizedPath)) return next(); + + const rawContentType = req.headers['content-type']; + const contentType = Array.isArray(rawContentType) ? rawContentType.join(',') : rawContentType; + const mediaType = typeof contentType === 'string' ? contentType.split(';', 1)[0]?.trim().toLowerCase() : ''; + if (mediaType) { + if (mediaType === 'application/json') return next(); + } else { + const rawContentLength = req.headers['content-length']; + const contentLength = typeof rawContentLength === 'string' ? Number(rawContentLength) : 0; + if (Number.isFinite(contentLength) && contentLength > 0) { + // A declared non-empty body without a media type is unsupported. + } else if (req.headers['transfer-encoding'] != null) { + // Transfer-Encoding alone does not mean the chunked body contains data: + // an empty `0\r\n\r\n` request should reach normal route validation. + const cleanup = (): void => { + req.off('data', onData); + req.off('end', onEnd); + req.off('error', onError); + }; + const onData = (): void => { + cleanup(); + rejectUnsupportedContentType(req, res); + }; + const onEnd = (): void => { + cleanup(); + next(); + }; + const onError = (error: Error): void => { + cleanup(); + next(error); + }; + req.once('data', onData); + req.once('end', onEnd); + req.once('error', onError); + return; + } else { + return next(); + } + } + + rejectUnsupportedContentType(req, res); + }); + + app.use(bodyParser.json({ limit: '50mb' })); + app.use(bodyParser.urlencoded({ limit: '1mb', extended: false })); + + // Request bodies and the authenticated key identity are now available. The + // ledger is per app instance and stores only hashed scope/key material. + const idempotencyRegistry = new IdempotencyRegistry(); + app.use(createIdempotencyMiddleware({ + registry: idempotencyRegistry, + configuredKeys: apiKeyVerifier.keys, + matchedKeyIndex: (req) => apiKeyVerifier.matchedIndex( + req as unknown as { headers: { authorization?: unknown; 'x-api-key'?: unknown } }, + ), + })); + + const PROVIDERS_CACHE_TTL_MS = 30_000; + let cachedProvidersList: ProviderInfo[] | null = null; + let cachedProvidersAt = 0; + let pendingProvidersPromise: Promise | null = null; + let providersCacheEpoch = 0; + + const invalidateProvidersCache = (): void => { + providersCacheEpoch += 1; + cachedProvidersList = null; + cachedProvidersAt = 0; + pendingProvidersPromise = null; + }; + + const unsubscribeReconnect = onBackendReconnect(invalidateProvidersCache); + let reconnectListenerDisposed = false; + let responseStateSweepTimer: ReturnType | null = null; + let conversationCleanupTimeout: ReturnType | null = null; + let conversationCleanupTimer: ReturnType | null = null; + const disposeAppResources = (): void => { + if (reconnectListenerDisposed) return; + reconnectListenerDisposed = true; + unsubscribeReconnect(); + if (responseStateSweepTimer) { + clearInterval(responseStateSweepTimer); + responseStateSweepTimer = null; + } + if (conversationCleanupTimeout) { + clearTimeout(conversationCleanupTimeout); + conversationCleanupTimeout = null; + } + if (conversationCleanupTimer) { + clearInterval(conversationCleanupTimer); + conversationCleanupTimer = null; + } + }; + const getProvidersList = async (): Promise => { - const providersRes = (await client.config.providers()) as unknown; - const providersRaw: unknown = asRecord(asRecord(providersRes)['data'])['providers'] ?? []; - if (Array.isArray(providersRaw)) return providersRaw as ProviderInfo[]; - return Object.entries(asRecord(providersRaw)).map(([id, info]) => ({ ...(asRecord(info) as object), id }) as ProviderInfo); + const now = Date.now(); + if (cachedProvidersList && now - cachedProvidersAt < PROVIDERS_CACHE_TTL_MS) { + return cachedProvidersList; + } + if (pendingProvidersPromise) { + return pendingProvidersPromise; + } + const epoch = providersCacheEpoch; + pendingProvidersPromise = (async () => { + try { + if (config.MANAGE_BACKEND) { + await ensureBackend(config); + } + const providersRes = (await client.config.providers()) as unknown; + const providersRaw: unknown = asRecord(asRecord(providersRes)['data'])['providers'] ?? []; + const list = Array.isArray(providersRaw) + ? (providersRaw as ProviderInfo[]) + : Object.entries(asRecord(providersRaw)).map(([id, info]) => ({ ...(asRecord(info) as object), id }) as ProviderInfo); + if (providersCacheEpoch === epoch) { + cachedProvidersList = list; + cachedProvidersAt = Date.now(); + } + return list; + } catch (err: unknown) { + if (providersCacheEpoch === epoch) { + invalidateProvidersCache(); + } + throw err; + } finally { + if (providersCacheEpoch === epoch) { + pendingProvidersPromise = null; + } + } + })(); + return pendingProvidersPromise; }; const buildModelsList = (providersList: unknown): ModelInfo[] => { @@ -242,7 +418,7 @@ export function createApp(config: ProxyConfig): CreateAppResult { code?: string; availableModels?: string[]; }; - error.statusCode = 400; + error.statusCode = 404; error.code = 'model_not_found'; error.availableModels = models.map((m) => m.id); throw error; @@ -255,6 +431,7 @@ export function createApp(config: ProxyConfig): CreateAppResult { }; const responseState = new Map(); + const pendingExpiredResponseSessionIds = new Set(); const RESPONSE_STATE_TTL_MS = 30 * 60 * 1000; const RESPONSE_STATE_SWEEP_INTERVAL_MS = 60 * 1000; const RESPONSE_TOOL_CALL_STATE_LIMIT = 128; @@ -277,46 +454,68 @@ export function createApp(config: ProxyConfig): CreateAppResult { }); return calls.slice(-RESPONSE_TOOL_CALL_STATE_LIMIT); }; - const getResponseState = (responseId: unknown): ResponseStateEntry | null => { + const getResponseState = (responseId: unknown, tenantId: string): ResponseStateEntry | null => { if (typeof responseId !== 'string') return null; const state = responseState.get(responseId); if (!state) return null; + // Pre-tenant entries are legacy state and are claimable only by unauthenticated callers. + if (state.tenantId === undefined ? tenantId !== 'anonymous' : state.tenantId !== tenantId) return null; if (state.expiresAt <= Date.now()) { responseState.delete(responseId); + pendingExpiredResponseSessionIds.add(state.sessionId); return null; } return state; }; - const storeResponseState = (responseId: unknown, sessionId: unknown, model: unknown, toolCalls: unknown = []): void => { - if (!responseId || !sessionId || typeof responseId !== 'string' || typeof sessionId !== 'string') return; + const storeResponseState = ( + responseId: unknown, + sessionId: unknown, + model: unknown, + toolCalls: unknown, + checkpointId: unknown, + tenantId: string, + ): boolean => { + if ( + !responseId || + !sessionId || + typeof responseId !== 'string' || + typeof sessionId !== 'string' || + typeof checkpointId !== 'string' || + !checkpointId.trim() + ) return false; responseState.set(responseId, { sessionId, + tenantId, + checkpointId, model: typeof model === 'string' ? model : String(model ?? ''), expiresAt: Date.now() + RESPONSE_STATE_TTL_MS, toolCalls: normalizeResponseToolCallState(toolCalls), }); + return true; }; const sweepResponseState = async (): Promise => { const now = Date.now(); - const expired: ResponseStateEntry[] = []; for (const [id, state] of responseState.entries()) { if (state.expiresAt <= now) { - expired.push(state); responseState.delete(id); + pendingExpiredResponseSessionIds.add(state.sessionId); } } - if (!expired.length) return; - const liveSessionIds = new Set([...responseState.values()].map((s) => s.sessionId)); - for (const state of expired) { - if (liveSessionIds.has(state.sessionId)) continue; + for (const sessionId of pendingExpiredResponseSessionIds) { + const stillReferenced = [...responseState.values()].some( + (state) => state.sessionId === sessionId && state.expiresAt > Date.now(), + ); + if (stillReferenced) continue; + if (idempotencyRegistry.hasUnresolvedChildSession(sessionId)) continue; try { - await client.session.delete({ path: { id: state.sessionId } }); + await client.session.delete({ path: { id: sessionId } }); + pendingExpiredResponseSessionIds.delete(sessionId); } catch (e: unknown) { - logDebug('Failed to delete expired response session', { sessionId: state.sessionId, error: toErrorMessage(e) }); + logDebug('Failed to delete expired response session', { sessionId, error: toErrorMessage(e) }); } } }; - const responseStateSweepTimer = setInterval(() => { + responseStateSweepTimer = setInterval(() => { sweepResponseState().catch(() => {}); }, RESPONSE_STATE_SWEEP_INTERVAL_MS); if (typeof responseStateSweepTimer.unref === 'function') responseStateSweepTimer.unref(); @@ -927,13 +1126,14 @@ export function createApp(config: ProxyConfig): CreateAppResult { }; if (AUTO_CLEANUP_CONVERSATIONS) { - setTimeout(() => { + conversationCleanupTimeout = setTimeout(() => { + conversationCleanupTimeout = null; cleanupConversationFiles().catch((e: unknown) => logDebug('Cleanup run failed', { error: toErrorMessage(e) })); }, 3000); - const cleanupTimer = setInterval(() => { + conversationCleanupTimer = setInterval(() => { cleanupConversationFiles().catch((e: unknown) => logDebug('Cleanup run failed', { error: toErrorMessage(e) })); }, Number(CLEANUP_INTERVAL_MS)); - if (cleanupTimer.unref) cleanupTimer.unref(); + if (conversationCleanupTimer.unref) conversationCleanupTimer.unref(); } const createForcedToolCallRequester = @@ -948,10 +1148,19 @@ export function createApp(config: ProxyConfig): CreateAppResult { toolOverrides, requestTimeoutMs, forbidThinkBlock = false, + signal, + requireTerminalCompletion = false, } = options; return async (): Promise | null> => { if (mode !== 'required') return null; if (!requiredTool) return null; + const assertNotAborted = (): void => { + if (signal?.aborted) { + const error = new Error('Request aborted'); + error.name = 'AbortError'; + throw error; + } + }; const forcedPromptParams: { path: { id: string }; body: Record } = { path: { id: sessionId }, body: { @@ -970,8 +1179,35 @@ export function createApp(config: ProxyConfig): CreateAppResult { if (forcedToolOverrides) { forcedPromptParams.body['tools'] = forcedToolOverrides; } + assertNotAborted(); + options.assertCanPrompt?.(); + const forcedMessageId = options.beforePrompt ? makeId('msg_') : null; + if (forcedMessageId) { + forcedPromptParams.body['messageID'] = forcedMessageId; + options.beforePrompt?.(sessionId, forcedMessageId); + } + assertNotAborted(); + options.assertCanPrompt?.(); await promptWithTimeout(forcedPromptParams, requestTimeoutMs); - return pollForAssistantResponse(sessionId, requestTimeoutMs) as Promise>; + assertNotAborted(); + if (!forcedMessageId) { + return pollForAssistantResponse( + sessionId, + requestTimeoutMs, + undefined, + signal, + undefined, + requireTerminalCompletion, + ) as Promise>; + } + return pollForAssistantResponse( + sessionId, + requestTimeoutMs, + undefined, + signal, + forcedMessageId, + requireTerminalCompletion, + ) as Promise>; }; }; @@ -1039,6 +1275,7 @@ export function createApp(config: ProxyConfig): CreateAppResult { maxRetries, maxAttempts, getProvidersList, + invalidateProvidersCache, buildModelsList, normalizeModelID, resolveRequestedModel, @@ -1096,17 +1333,19 @@ export function createApp(config: ProxyConfig): CreateAppResult { getDiscoveryLastErrorAt: () => discoveryLastErrorAt, getDiscoveryLastError: () => discoveryLastError, translators: ensureTranslatorsRegistered(defaultTranslatorRegistry()), + dispose: disposeAppResources, }; - // P3: register routes (404 last to preserve catch-all order). + // P3: register routes (404 last to preserve catch-all order, error handler after). registerSystemRoutes(app, ctx); registerChatRoutes(app, ctx); registerResponsesRoutes(app, ctx); registerMessagesRoutes(app, ctx); registerInteractionsRoutes(app, ctx); registerNotFoundRoute(app); + registerErrorHandler(app); - return { app, client }; + return { app, client, ctx }; } /** @@ -1116,7 +1355,7 @@ export function startProxy(options: unknown = {}): StartProxyResult { // buildProxyConfig tolerates explicit null and preserves merge semantics verbatim. const config = buildProxyConfig((options ?? {}) as unknown); - const { app } = createApp(config); + const { app, ctx } = createApp(config); const server = app.listen(config.PORT, config.BIND_HOST, () => { void (async (): Promise => { @@ -1129,8 +1368,30 @@ export function startProxy(options: unknown = {}): StartProxyResult { })(); }); + let disposed = false; + let onServerError: (() => void) | null = null; + const dispose = (): void => { + if (disposed) return; + disposed = true; + ctx?.dispose?.(); + }; + onServerError = (): void => { + // A bind failure happens before `listening` and does not reliably produce + // the normal close lifecycle, so release the app-scoped listener here too. + if (!server.listening) { + dispose(); + if (onServerError) server.off('error', onServerError); + } + }; + server.on('error', onServerError); + server.once('close', () => { + dispose(); + if (onServerError) server.off('error', onServerError); + }); + return { server, + dispose, killBackend: () => { const state = backendState.get(config.OPENCODE_SERVER_URL); if (state?.process) { diff --git a/src/routes/chat.ts b/src/routes/chat.ts index a3f4cb0..bd7693a 100644 --- a/src/routes/chat.ts +++ b/src/routes/chat.ts @@ -13,6 +13,10 @@ import { createToolCallFilter, createExternalToolCallStreamParser, } from '../tool-runtime/parser.js'; +import { + isMalformedExternalToolCallError, + resolveAuthoritativeStreamSource, +} from '../tool-runtime/stream-source.js'; import { isTransientUpstreamError, normalizeBackendError, transformUpstreamError } from '../errors/upstream.js'; import { engageFallbackForFreeLimit } from '../upstream-proxy/fallback.js'; import { detectHostedSearchTools } from '../search/grounding.js'; @@ -27,6 +31,7 @@ import type { Application, Request, Response } from 'express'; import type { AppContext } from '../types/context.js'; import type { ExternalToolEntry } from '../tool-runtime/registry.js'; import type { FinalToolCall } from '../tool-runtime/parser.js'; +import type { AuthoritativeStreamSource } from '../tool-runtime/stream-source.js'; import { asRecord, toErrorMessage } from '../utils/guards.js'; function asArray(value: unknown): unknown[] { @@ -86,12 +91,94 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { app.post('/v1/chat/completions', (req: Request, res: Response): void => { void (async (): Promise => { try { + const body = asRecord((req as unknown as { body: unknown }).body); + const messages: unknown = body['messages']; + const model: unknown = body['model']; + const toolsRaw: unknown = body['tools']; + const tools: unknown[] = Array.isArray(toolsRaw) ? (toolsRaw as unknown[]) : []; + const tool_choice: unknown = body['tool_choice']; + const requestStream: unknown = body['stream']; + const temperature: unknown = body['temperature']; + const max_tokens: unknown = body['max_tokens']; + const top_p: unknown = body['top_p']; + const frequency_penalty: unknown = body['frequency_penalty']; + const presence_penalty: unknown = body['presence_penalty']; + const stop: unknown = body['stop']; + const reasoning_effort: unknown = body['reasoning_effort']; + const reasoning: unknown = body['reasoning']; + const requestOpencodeConfig: unknown = body['opencode']; + const parallelToolCalls: boolean = body['parallel_tool_calls'] !== false; + const stream = Boolean(requestStream); + if (!messages || !Array.isArray(messages) || (messages as unknown[]).length === 0) { + res.status(400).json({ error: { message: 'messages array is required' } }); + return; + } + if (detectHostedSearchTools(tools).requested) { + res.status(400).json({ + error: { + message: + 'web_search is not supported on /v1/chat/completions; use POST /v1/responses with tools:[{type:"web_search"}] or POST /v1beta/interactions with tools:[{type:"google_search"}]', + type: 'invalid_request_error', + }, + }); + return; + } + const unsupportedChatToolIndex = tools.findIndex((def) => { + const t = asRecord(def); + return t['type'] !== 'function'; + }); + if (unsupportedChatToolIndex >= 0) { + const rawType = asRecord(tools[unsupportedChatToolIndex])['type']; + const label = typeof rawType === 'string' && rawType.trim() ? rawType.trim() : 'missing'; + res.status(400).json({ + error: { + message: `Unsupported tool type "${label}" at tools[${unsupportedChatToolIndex}]. Only function tools are supported on this endpoint.`, + type: 'invalid_request_error', + code: 'unsupported_tool_type', + }, + }); + return; + } + + const reasoningLevel = normalizeReasoningEffort( + (reasoning_effort as string) || (asRecord(reasoning)['effort'] as string), + null, + ); + + const requestParams: Record = { + temperature: typeof temperature === 'number' ? temperature : 0.7, + max_tokens: typeof max_tokens === 'number' ? max_tokens : null, + top_p: typeof top_p === 'number' ? top_p : 1.0, + frequency_penalty: typeof frequency_penalty === 'number' ? frequency_penalty : 0, + presence_penalty: typeof presence_penalty === 'number' ? presence_penalty : 0, + stop: Array.isArray(stop) ? stop : stop ? [stop] : null, + reasoning_effort: reasoningLevel, + }; + + logDebug('Request params', { + temperature: requestParams['temperature'], + max_tokens: requestParams['max_tokens'], + top_p: requestParams['top_p'], + reasoning_effort: reasoningLevel, + }); + + let pID = 'opencode'; + let mID = 'kimi-k2.5-free'; + const resolvedModel = (await withTimeout( + resolveRequestedModel(model), + REQUEST_TIMEOUT_MS, + 'resolve model', + )) as unknown as { providerID: string; modelID: string; aliasFrom?: string; resolved?: string }; + const resolvedRecord = asRecord(resolvedModel); + pID = String(resolvedRecord['providerID'] ?? pID); + mID = String(resolvedRecord['modelID'] ?? mID); + if (resolvedRecord['aliasFrom']) { + logDebug('Resolved model alias', { from: resolvedRecord['aliasFrom'], to: resolvedRecord['resolved'] }); + } + await lock(async (): Promise => { let sessionId: string | null = null; let eventStream: { close?: unknown } | null = null; - let stream = false; - let pID = 'opencode'; - let mID = 'kimi-k2.5-free'; let id = `chatcmpl-${crypto.randomUUID()}`; let keepaliveInterval: ReturnType | null = null; const requestAbortController = new AbortController(); @@ -124,89 +211,6 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { if (proxyPool.isEngaged()) switchToProxyBundle(); try { - const body = asRecord((req as unknown as { body: unknown }).body); - const messages: unknown = body['messages']; - const model: unknown = body['model']; - const toolsRaw: unknown = body['tools']; - const tools: unknown[] = Array.isArray(toolsRaw) ? (toolsRaw as unknown[]) : []; - const tool_choice: unknown = body['tool_choice']; - const requestStream: unknown = body['stream']; - const temperature: unknown = body['temperature']; - const max_tokens: unknown = body['max_tokens']; - const top_p: unknown = body['top_p']; - const frequency_penalty: unknown = body['frequency_penalty']; - const presence_penalty: unknown = body['presence_penalty']; - const stop: unknown = body['stop']; - const reasoning_effort: unknown = body['reasoning_effort']; - const reasoning: unknown = body['reasoning']; - const requestOpencodeConfig: unknown = body['opencode']; - const parallelToolCalls: boolean = body['parallel_tool_calls'] !== false; - stream = Boolean(requestStream); - if (!messages || !Array.isArray(messages) || (messages as unknown[]).length === 0) { - res.status(400).json({ error: { message: 'messages array is required' } }); - return; - } - if (detectHostedSearchTools(tools).requested) { - res.status(400).json({ - error: { - message: - 'web_search is not supported on /v1/chat/completions; use POST /v1/responses with tools:[{type:"web_search"}] or POST /v1beta/interactions with tools:[{type:"google_search"}]', - type: 'invalid_request_error', - }, - }); - return; - } - const unsupportedChatToolIndex = tools.findIndex((def) => { - const t = asRecord(def); - return t['type'] !== 'function'; - }); - if (unsupportedChatToolIndex >= 0) { - const rawType = asRecord(tools[unsupportedChatToolIndex])['type']; - const label = typeof rawType === 'string' && rawType.trim() ? rawType.trim() : 'missing'; - res.status(400).json({ - error: { - message: `Unsupported tool type "${label}" at tools[${unsupportedChatToolIndex}]. Only function tools are supported on this endpoint.`, - type: 'invalid_request_error', - code: 'unsupported_tool_type', - }, - }); - return; - } - - const reasoningLevel = normalizeReasoningEffort( - (reasoning_effort as string) || (asRecord(reasoning)['effort'] as string), - null, - ); - - const requestParams: Record = { - temperature: typeof temperature === 'number' ? temperature : 0.7, - max_tokens: typeof max_tokens === 'number' ? max_tokens : null, - top_p: typeof top_p === 'number' ? top_p : 1.0, - frequency_penalty: typeof frequency_penalty === 'number' ? frequency_penalty : 0, - presence_penalty: typeof presence_penalty === 'number' ? presence_penalty : 0, - stop: Array.isArray(stop) ? stop : stop ? [stop] : null, - reasoning_effort: reasoningLevel, - }; - - logDebug('Request params', { - temperature: requestParams['temperature'], - max_tokens: requestParams['max_tokens'], - top_p: requestParams['top_p'], - reasoning_effort: reasoningLevel, - }); - - const resolvedModel = await withTimeout( - resolveRequestedModel(model), - REQUEST_TIMEOUT_MS, - 'resolve model', - ) as unknown as { providerID: string; modelID: string; aliasFrom?: string; resolved?: string }; - const resolvedRecord = asRecord(resolvedModel); - pID = String(resolvedRecord['providerID'] ?? pID); - mID = String(resolvedRecord['modelID'] ?? mID); - if (resolvedRecord['aliasFrom']) { - logDebug('Resolved model alias', { from: resolvedRecord['aliasFrom'], to: resolvedRecord['resolved'] }); - } - const normalizeMessageContent = (content: unknown): string => normalizeTextContent(content); const buildPromptParts = async ( @@ -471,6 +475,7 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); if (stream) { + res.setHeader('X-Accel-Buffering', 'no'); const flushHeaders = (res as unknown as { flushHeaders?: unknown }).flushHeaders; if (typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); const onClientClose = (): void => { @@ -515,8 +520,8 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { const ensureKeepalive = (): void => { if (!keepaliveInterval) { keepaliveInterval = setInterval(() => { - if (!res.destroyed) { - res.write(': keepalive\n\n'); + if (!res.destroyed && !res.writableEnded) { + res.write(': keepalive\n'); } }, 15000); } @@ -568,10 +573,16 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { let collected: Record | null = null; let lastStreamAttemptError: unknown = null; - let polledSnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null = null; + let polledSnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean } | null = null; let snapshotPolled = false; - let snapshotPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> | null = null; - const pollSnapshot = async (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> => { + let snapshotPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }> | null = null; + const hasRecoveryEvidence = (snapshot: { content: string; reasoning: string; toolParts?: unknown }): boolean => + Boolean( + snapshot.content || + snapshot.reasoning || + (Array.isArray(snapshot.toolParts) && snapshot.toolParts.length > 0), + ); + const pollSnapshot = async (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }> => { snapshotPolled = true; if (!snapshotPromise) { snapshotPromise = activePollForAssistantResponse( @@ -586,13 +597,14 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { }; // Recovery snapshots repeat what the event stream already delivered. Only the // suffix past the raw prefix may be replayed, otherwise every recovered turn - // duplicates text. A snapshot the raw prefix does not match is still sent in - // full rather than dropped. + // duplicates text. A divergent snapshot (raw "AC", snapshot "ABC") is not a + // safe replay: emitting the whole snapshot would produce "ACABC". Tool calls + // are still parsed from the snapshot group; visible text keeps the raw stream. const unsentSuffix = (full: unknown, raw: string): string => { const text = typeof full === 'string' ? full : ''; if (!text) return ''; if (!raw) return text; - return text.startsWith(raw) ? text.slice(raw.length) : text; + return text.startsWith(raw) ? text.slice(raw.length) : ''; }; const replaySnapshot = (snapshot: { content: string; reasoning: string; error: unknown }): void => { if (snapshot.error != null) throw snapshot.error; @@ -601,45 +613,31 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { if (remainingReasoning) sendDelta(remainingReasoning, true); if (remainingContent) sendDelta(remainingContent, false); }; - const hasPartialOutput = (): boolean => { - const current = asRecord(collected); - return Boolean( - rawStreamedContent || - rawStreamedReasoning || - streamedToolCalls.length > 0 || - current['content'] || - current['reasoning'], - ); - }; const recoverSnapshot = async ( fallbackError: unknown = null, - ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown } | null> => { + ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean } | null> => { if (clientDisconnected) return null; - let snapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown }; + let snapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }; try { snapshot = await pollSnapshot(); } catch (error: unknown) { if (clientDisconnected) return null; - if (hasPartialOutput()) { - logDebug('Ignoring recovery poll failure after partial stream output', { error: toErrorMessage(error) }); - return null; - } if (fallbackError != null) throw fallbackError; throw error; } - if (snapshot.error != null) throw snapshot.error; - if ( - fallbackError != null && - !snapshot.content && - !snapshot.reasoning && - !(Array.isArray(snapshot.toolParts) && snapshot.toolParts.length > 0) - ) throw fallbackError; - return snapshot; - + if (snapshot.error != null) throw snapshot.error; + if (snapshot.partial === true) throw new Error('Recovery poll returned incomplete response'); + if (!hasRecoveryEvidence(snapshot)) { + if (fallbackError != null) throw fallbackError; + return null; + } + return snapshot; }; for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { if (attempt > 1) { + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); try { await activeClient.session.delete({ path: { id: sessionId } }); } catch (e: unknown) { @@ -662,6 +660,7 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { polledSnapshot = null; snapshotPolled = false; snapshotPromise = null; + fallbackServed = false; filterContentDelta = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, @@ -684,8 +683,13 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { // through the proxy bundle instead of the event stream. // Deltas go out immediately so the post-loop poll is skipped. await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); - const pres = (await pollSnapshot()) as { content: string; reasoning: string; error: unknown }; - polledSnapshot = pres; + const pres = await pollSnapshot(); + if (pres.error != null) throw pres.error; + if (pres.partial === true) throw new Error('Recovery poll returned incomplete response'); + if (!hasRecoveryEvidence(pres) && externalToolChoice.mode !== 'required') { + throw new Error('Recovery poll returned empty response'); + } + polledSnapshot = pres; const presReasoning = unsentSuffix(pres.reasoning, rawStreamedReasoning); const presContent = unsentSuffix(pres.content, rawStreamedContent); if (presReasoning) sendDelta(presReasoning, true); @@ -696,7 +700,7 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { collected = { content: pres.content, reasoning: pres.reasoning, - ...(pres.error ? { error: pres.error } : {}), + toolParts: pres.toolParts, }; } else { const collectPromise = collectFromEvents( @@ -761,8 +765,8 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { polledSnapshot = snapshot; replaySnapshot(snapshot); } - } else if (collectedR['noData']) { - logDebug('Fallback to polling (stream)', { sessionId }); + } else if (collectedR['noData'] || collectedR['cut']) { + logDebug('Fallback to polling (stream)', { sessionId, cut: Boolean(collectedR['cut']) }); const snapshot = await recoverSnapshot(); if (snapshot) { polledSnapshot = snapshot; @@ -792,11 +796,14 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { !collectedR['__error'] && !collectedR['noData'] && !collectedR['idleTimeout'] && + !collectedR['cut'] && !collectedR['cancelled'] && + collectedR['finish'] !== 'tool' && + collectedR['finish'] !== 'tool-calls' && (rawStreamedContent || rawStreamedReasoning || streamedToolCalls.length > 0 || collectedR['content'] || collectedR['reasoning']), ); - if (!fallbackServed && !rawStreamedContent && !rawStreamedReasoning && !hasValidTerminal) { + if (!fallbackServed && !hasValidTerminal) { logDebug('SSE returned empty, falling back to polling', { sessionId }); const snapshot = await recoverSnapshot(); if (snapshot) { @@ -821,12 +828,53 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { } } + if (!hasValidTerminal && !polledSnapshot && !fallbackServed) { + if (streamedToolCalls.length === 0 && externalToolChoice.mode !== 'required') { + throw new Error('Stream ended without valid terminal'); + } + } + const flushedReasoningCalls = parseReasoningToolCalls.flush ? parseReasoningToolCalls.flush() : []; const flushedContentCalls = parseContentToolCalls.flush ? parseContentToolCalls.flush() : []; const flushedReasoningText = filterReasoningDelta.flush ? filterReasoningDelta.flush() : ''; const flushedContentText = filterContentDelta.flush ? filterContentDelta.flush() : ''; + const parseJoined = (reasoning: string, content: string): FinalToolCall[] => + externalToolRegistry.length > 0 + ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoning, content) + : []; + const callsFromText = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? parseJoined(source.reasoning, source.content) + : []; + const callsFromRaw = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? mergeToolCallArtifacts( + streamedToolCalls, + flushedReasoningCalls, + flushedContentCalls, + parseJoined(source.reasoning, source.content), + ) + : []; + // One authoritative source group. The snapshot (recovery poll or + // collector-authoritative full text) replaces the raw delta group; + // validating both independently fails complete snapshots whenever a + // delta was dropped. Raw is only used when no snapshot exists. + let authoritativeSource = resolveAuthoritativeStreamSource({ + collected: collectedR, + polledSnapshot, + rawContent: rawStreamedContent, + rawReasoning: rawStreamedReasoning, + }); + const streamSource = (): string[][] => [[authoritativeSource.reasoning, authoritativeSource.content]]; + if (!shouldBufferExternalStream) { + if (rawStreamedContent && authoritativeSource.content && !authoritativeSource.content.startsWith(rawStreamedContent)) { + throw new Error('Stream content diverged from authoritative snapshot'); + } + if (rawStreamedReasoning && authoritativeSource.reasoning && !authoritativeSource.reasoning.startsWith(rawStreamedReasoning)) { + throw new Error('Stream reasoning diverged from authoritative snapshot'); + } appendFilteredDelta(flushedReasoningText, true); appendFilteredDelta(flushedContentText, false); writeVisibleDelta(flushedReasoningText, true); @@ -834,49 +882,89 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { } else { const visible = stripExternalToolCallMarkupFromJoinedText( externalToolRegistry, - rawStreamedReasoning, - rawStreamedContent, + authoritativeSource.reasoning, + authoritativeSource.content, ); streamedReasoning = visible.reasoning; streamedContent = visible.content; } - const snapshotReasoning = polledSnapshot?.reasoning ?? (typeof collectedR['reasoning'] === 'string' ? collectedR['reasoning'] as string : rawStreamedReasoning); - const snapshotContent = polledSnapshot?.content ?? (typeof collectedR['content'] === 'string' ? collectedR['content'] as string : rawStreamedContent); - const parseJoined = (reasoning: string, content: string): FinalToolCall[] => - externalToolRegistry.length > 0 - ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoning, content) - : []; - const streamSource = [ - [snapshotReasoning, snapshotContent], - [rawStreamedReasoning, rawStreamedContent] - ]; - let parsedToolCalls: FinalToolCall[] = externalToolRegistry.length > 0 - ? mergeToolCallArtifacts( - streamedToolCalls, - flushedReasoningCalls, - flushedContentCalls, - parseJoined(snapshotReasoning, snapshotContent), - parseJoined(rawStreamedReasoning, rawStreamedContent), - ) - : []; + let parsedToolCalls: FinalToolCall[] = authoritativeSource.fromSnapshot + ? callsFromText(authoritativeSource) + : callsFromRaw(authoritativeSource); + const repairFromSnapshot = async (): Promise => { + if (externalToolRegistry.length === 0) return false; + // Reuse an already-polled snapshot (partial timeout snapshots are + // valid repair evidence when they contain a complete call). Only + // poll when no snapshot attempt has been made. + let snapshot = polledSnapshot; + if (!snapshot) { + if (snapshotPolled) return false; + try { + snapshot = await recoverSnapshot(); + } catch { + return false; + } + } + if (!snapshot || snapshot.error != null) return false; + const repairedSource: AuthoritativeStreamSource = { + content: String(snapshot.content ?? ''), + reasoning: String(snapshot.reasoning ?? ''), + kind: 'polled-snapshot', + fromSnapshot: true, + }; + const snapshotCalls = parseJoined(repairedSource.reasoning, repairedSource.content); + if (snapshotCalls.length === 0) return false; + try { + assertToolCallArtifactIntegrity(snapshotCalls, externalToolRegistry, [ + [repairedSource.reasoning, repairedSource.content], + ]); + } catch { + return false; + } + polledSnapshot = snapshot; + authoritativeSource = repairedSource; + parsedToolCalls = snapshotCalls; + if (shouldBufferExternalStream) { + const visible = stripExternalToolCallMarkupFromJoinedText( + externalToolRegistry, + authoritativeSource.reasoning, + authoritativeSource.content, + ); + streamedReasoning = visible.reasoning; + streamedContent = visible.content; + } + return true; + }; let finalStreamedToolCalls: FinalToolCall[]; try { - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource); + try { + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); + } catch (integrityError) { + if (!isMalformedExternalToolCallError(integrityError) || !(await repairFromSnapshot())) { + throw integrityError; + } + } if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required') { const forcedResponse = await requestForcedChatToolCall(); if (forcedResponse) { const forcedReasoning = typeof forcedResponse['reasoning'] === 'string' ? forcedResponse['reasoning'] as string : ''; const forcedContent = typeof forcedResponse['content'] === 'string' ? forcedResponse['content'] as string : ''; parsedToolCalls = mergeToolCallArtifacts(parsedToolCalls, parseJoined(forcedReasoning, forcedContent)); - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [forcedReasoning, forcedContent]); + authoritativeSource = { + content: `${authoritativeSource.content}${forcedContent}`, + reasoning: `${authoritativeSource.reasoning}${forcedReasoning}`, + kind: authoritativeSource.kind, + fromSnapshot: authoritativeSource.fromSnapshot, + }; + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); } } finalStreamedToolCalls = finalizeStreamToolCalls( parsedToolCalls, externalToolRegistry, externalToolChoice, - streamSource, + streamSource(), parallelToolCalls, ) as unknown as FinalToolCall[]; } catch (toolError) { @@ -989,6 +1077,8 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { return; } if (attempt > 1) { + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); try { await activeClient.session.delete({ path: { id: sessionId } }); } catch (e: unknown) { @@ -1016,9 +1106,18 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { requestAbortController.signal, ); - content = collected.content || ''; - reasoning = collected.reasoning || ''; - error = collected.error || null; + content = collected.content || ''; + reasoning = collected.reasoning || ''; + error = + collected.error || + (collected.partial === true + ? new Error('Polling returned incomplete response') + : externalToolChoice.mode !== 'required' && + !content && + !reasoning && + !(Array.isArray(collected.toolParts) && collected.toolParts.length > 0) + ? new Error('Polling returned empty response') + : null); } catch (promptError: unknown) { content = ''; reasoning = ''; @@ -1165,10 +1264,18 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { if (!res.headersSent) { const transformed = transformUpstreamError(error); res.status(transformed.statusCode).json(transformed.error); - } else if (!res.destroyed) { - const transformed = transformUpstreamError(error); - res.write(`data: ${JSON.stringify({ error: transformed.error })}\n\n`); - res.end(); + } else if (!res.writableEnded && !res.destroyed) { + try { + const transformed = transformUpstreamError(error); + res.write(`data: ${JSON.stringify({ error: transformed.error })}\n\n`); + } catch { + // ignore write failure on closed stream + } + try { + res.end(); + } catch { + // ignore end failure + } } if (sessionId) { try { @@ -1199,13 +1306,16 @@ export function registerChatRoutes(app: Application, ctx: AppContext): void { } catch (error: unknown) { console.error('[Proxy] Request Handler Error:', toErrorMessage(error)); if (!res.headersSent) { - const er = asRecord(error); - res.status(500).json({ - error: { - message: toErrorMessage(error), - type: error instanceof Error ? error.constructor.name : String(er['name'] ?? 'Error'), - }, - }); + const transformed = transformUpstreamError(error); + res.status(transformed.statusCode).json(transformed.error); + } else if (!res.writableEnded && !res.destroyed) { + try { + const transformed = transformUpstreamError(error); + res.write(`data: ${JSON.stringify({ error: transformed.error })}\n\n`); + } catch { + // ignore + } + res.end(); } } })(); diff --git a/src/routes/interactions.ts b/src/routes/interactions.ts index af7ea38..6978292 100644 --- a/src/routes/interactions.ts +++ b/src/routes/interactions.ts @@ -14,10 +14,11 @@ // throw immediately with no generic transient backoff. // - `store:false` skips persistence and deletes the session after responding. import crypto from 'crypto'; +import { makeId } from '../converters/json.js'; import { withTimeout } from '../config/proxy-config.js'; import { computeRetryDelay } from '../retry/policy.js'; import { ensureActiveModel } from '../backend/active-model.js'; -import { normalizeBackendError, transformUpstreamError } from '../errors/upstream.js'; +import { isTransientUpstreamError, normalizeBackendError, transformUpstreamError } from '../errors/upstream.js'; import { engageFallbackForFreeLimit } from '../upstream-proxy/fallback.js'; import { buildCitationAnnotations, @@ -30,6 +31,7 @@ import { ensureBackend, sleep } from '../backend/manager.js'; import type { Application, Request, Response } from 'express'; import type { AppContext } from '../types/context.js'; import { asRecord, toErrorMessage } from '../utils/guards.js'; +import { getContinuationTenantId, getIdempotencyControl } from '../idempotency/middleware.js'; function textOfContent(content: unknown): string { if (typeof content === 'string') return content; @@ -133,28 +135,99 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v } = ctx; const handle = async (req: Request, res: Response): Promise => { + const idempotency = getIdempotencyControl(req); + const hasConfirmedIdempotencyResult = (): boolean => Boolean( + idempotency?.resultConfirmed && + typeof idempotency.confirmedAssistantMessageId === 'string' && + idempotency.confirmedAssistantMessageId.trim(), + ); + const assertIdempotencyResultConfirmed = (): void => { + if (idempotency?.submitted && !hasConfirmedIdempotencyResult()) { + throw new Error('Idempotency result is not confirmed for the exact assistant turn'); + } + }; + const isExactTerminalPoll = (result: unknown): boolean => { + const poll = asRecord(result); + return poll['complete'] === true && + poll['partial'] !== true && + typeof poll['assistantMessageId'] === 'string' && + poll['assistantMessageId'].trim().length > 0 && + typeof idempotency?.userMessageId === 'string' && + poll['userMessageId'] === idempotency.userMessageId; + }; + const confirmExactTerminalPoll = (result: unknown): void => { + if (!idempotency?.submitted) return; + if (!isExactTerminalPoll(result)) { + throw new Error('Idempotency result is incomplete or does not identify the exact assistant turn'); + } + const poll = asRecord(result); + idempotency.confirmResult(idempotency.userMessageId as string, poll['assistantMessageId'] as string); + }; const requestAbortController = new AbortController(); let clientGone = false; - let ownedSessionId: string | null = null; + let interactionCreated = false; + const ownedSessionIds = new Set(); // P3 bundle (direct unless the pool is already engaged). let activeClient = client; let activePromptWithTimeout = promptWithTimeout; let activePollForAssistantResponse = pollForAssistantResponse; let fallbackToProxy = false; - if (proxyPool.isEngaged() && proxyClient) { - activeClient = proxyClient; - activePromptWithTimeout = proxyPromptWithTimeout; - activePollForAssistantResponse = proxyPollForAssistantResponse; - } - const engageProxyFallback = (err: unknown): boolean => { - if (!engageFallbackForFreeLimit(err, proxyPool) || !proxyClient) return false; + const switchToProxyBundle = (): boolean => { + if (!proxyClient) return false; activeClient = proxyClient; activePromptWithTimeout = proxyPromptWithTimeout; activePollForAssistantResponse = proxyPollForAssistantResponse; fallbackToProxy = true; - return true; + return true; + }; + const deleteOwnedSession = async (id: string): Promise => { + if (!ownedSessionIds.has(id)) return; + const maxDeleteAttempts = 2; + for (let attempt = 1; attempt <= maxDeleteAttempts; attempt += 1) { + try { + await activeClient.session.delete({ path: { id } }); + ownedSessionIds.delete(id); + return; + } catch (e: unknown) { + logDebug('Failed to delete owned interaction session', { + sessionId: id, + attempt, + error: toErrorMessage(e), + }); + if (attempt === maxDeleteAttempts || !isTransientUpstreamError(e)) return; + } + } + }; + const cleanupOwnedSessionsNow = async (exceptId?: string): Promise => { + for (const id of [...ownedSessionIds]) { + if (id !== exceptId) await deleteOwnedSession(id); + } + }; + const cleanupOwnedSessions = async (exceptId?: string): Promise => { + if (!idempotency?.submitted) { + await cleanupOwnedSessionsNow(exceptId); + return; + } + // Keep a possibly submitted child available for unknown-outcome + // reconciliation. Delete it only after a complete response is cached. + const retainedChild = idempotency.childSessionId; + for (const id of [...ownedSessionIds]) { + if (id !== exceptId && id !== retainedChild) await deleteOwnedSession(id); + } + idempotency.onCompleted(() => cleanupOwnedSessionsNow(exceptId)); + }; + const engageProxyFallback = (err: unknown): boolean => { + if (!engageFallbackForFreeLimit(err, proxyPool)) return false; + return switchToProxyBundle(); }; + if (proxyPool.isEngaged()) switchToProxyBundle(); + const onClientClose = (): void => { + if (res.writableEnded) return; + clientGone = true; + requestAbortController.abort(); + }; + res.once('close', onClientClose); try { const body = asRecord((req as unknown as { body: unknown }).body); const model: unknown = body['model']; @@ -163,14 +236,8 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v const systemInstruction: unknown = body['system_instruction']; const toolsRaw: unknown = body['tools']; const tools: unknown[] = Array.isArray(toolsRaw) ? (toolsRaw as unknown[]) : []; - const stream = body['stream'] === true; - const store = body['store'] !== false; - const onClientClose = (): void => { - if (res.writableEnded) return; - clientGone = true; - requestAbortController.abort(); - }; - res.once('close', onClientClose); + const stream = body['stream'] === true; + const store = body['store'] !== false; const previousInteractionId: unknown = body['previous_interaction_id']; @@ -206,22 +273,24 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v }), ); - const previousState = + const tenantId = getContinuationTenantId(req); + const previousState = typeof previousInteractionId === 'string' && previousInteractionId - ? getResponseState(previousInteractionId) + ? getResponseState(previousInteractionId, tenantId) : null; - if (previousInteractionId && !previousState) { - res.status(400).json({ error: { message: 'Invalid or expired previous_interaction_id', type: 'invalid_request_error' } }); + if (!idempotency?.reconciling && previousInteractionId && (!previousState || !previousState.checkpointId)) { + res.status(400).json({ error: { message: 'Invalid or expired previous_interaction_id', type: 'invalid_request_error', code: 'invalid_previous_interaction_id' } }); return; } const resolvedModel = (await withTimeout( - resolveRequestedModel(typeof model === 'string' && model ? model : (previousState?.model ?? '')), + resolveRequestedModel(typeof model === 'string' && model ? model : (idempotency?.model ?? previousState?.model ?? '')), REQUEST_TIMEOUT_MS, 'resolve model', )) as unknown as { providerID: string; modelID: string }; const pID = String(asRecord(resolvedModel)['providerID']); const mID = String(asRecord(resolvedModel)['modelID']); + idempotency?.setModel(`${pID}/${mID}`); await ensureBackend(config); @@ -283,17 +352,33 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v return; } - let sessionId: string | null = previousState?.sessionId || null; - // Only a session created by this request may be deleted by store:false; - // deleting a reused parent session would orphan its stored state. - let sessionNewlyCreated = false; - if (!sessionId) { - sessionId = - ((asRecord(asRecord(await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session'))['data'])['id'] as string | undefined) ?? null); - if (!sessionId) throw new Error('Failed to create OpenCode session'); - sessionNewlyCreated = true; - ownedSessionId = sessionId; - + let sessionId: string | null = idempotency?.reconciling ? idempotency.childSessionId : null; + // Both new sessions and continuation forks belong to this request until + // response state storage succeeds; the stored parent is never mutated. + if (idempotency?.reconciling) { + // Unknown retries reuse the exact recorded child and never fork again. + if (!sessionId) throw new Error('Idempotency reconciliation is missing its child session'); + ownedSessionIds.add(sessionId); + } else if (previousState) { + const forkRes = (await withTimeout( + activeClient.session.fork({ + path: { id: previousState.sessionId }, + }), + REQUEST_TIMEOUT_MS, + 'fork continuation session', + )) as unknown; + sessionId = (asRecord(asRecord(forkRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to fork OpenCode continuation session'); + ownedSessionIds.add(sessionId); + } else { + const sessionRes = (await withTimeout( + activeClient.session.create(), + REQUEST_TIMEOUT_MS, + 'create session', + )) as unknown; + sessionId = (asRecord(asRecord(sessionRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to create OpenCode session'); + ownedSessionIds.add(sessionId); } const parts: Record[] = []; @@ -323,91 +408,149 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v parts, }, }; + promptParams.body.messageID = idempotency?.reconciling + ? idempotency.userMessageId + : makeId('msg_'); + if (typeof promptParams.body.messageID !== 'string' || !promptParams.body.messageID) { + throw new Error('Idempotency reconciliation is missing its user message ID'); + } // Stage-5: strip false entries for free-tier suspects (any false gates; true-only sent, all-false omitted). const promptToolOverrides = selectPromptToolOverrides(toolOverrides, pID, mID); if (promptToolOverrides) { promptParams.body['tools'] = promptToolOverrides; } - // Thin retry: free-limit errors engage the proxy pool and rotate the - // session (bounded by maxAttempts like the other routes). Ordinary - // errors throw immediately — no generic transient backoff here. + // Upstream retry: free-limit errors engage the proxy pool and rotate the + // session; transient 5xx / rate-limit errors retry with exponential backoff + // (bounded by maxAttempts like the other routes). const parentSessionId = previousState?.sessionId ?? null; + let promptSubmitted = false; const promptAndPoll = async (): Promise<{ content: string; reasoning: string; - error: unknown; - toolParts?: unknown; - }> => { + error: unknown; + toolParts?: unknown; + assistantMessageId?: string; + userMessageId?: string; + partial?: boolean; + complete?: boolean; + }> => { + if (idempotency?.reconciling) { + const polled = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + requestAbortController.signal, + idempotency.userMessageId ?? undefined, + true, + ); + confirmExactTerminalPoll(polled); + if (polled.error != null) throw normalizeBackendError(polled.error); + return polled; + } let lastErr: unknown = null; for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { - if (res.destroyed || res.writableEnded) throw new Error('Client disconnected'); + let confirmedTerminalAssistantError = false; + if (requestAbortController.signal.aborted || res.destroyed || res.writableEnded) { + throw new Error('Client disconnected'); + } if (attempt > 1) { - const currentId = sessionId as string; - const shouldDelete = sessionNewlyCreated || (parentSessionId ? currentId !== parentSessionId : true); - if (shouldDelete) { - try { - await activeClient.session.delete({ path: { id: currentId } }); - if (ownedSessionId === currentId) ownedSessionId = null; - } catch (e: unknown) { - logDebug('Failed to delete retried interaction session', { error: toErrorMessage(e) }); - } + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); + if (!parentSessionId && !idempotency?.submitted) { + promptSubmitted = false; + const currentId = sessionId as string; + if (currentId) await deleteOwnedSession(currentId); + const retryRes = (await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session')) as unknown; + sessionId = (asRecord(asRecord(retryRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); + ownedSessionIds.add(sessionId); + promptParams.path.id = sessionId; } - const retryRes = (await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session')) as unknown; - sessionId = (asRecord(asRecord(retryRes)['data'])['id'] as string | undefined) ?? null; - if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); - sessionNewlyCreated = true; - ownedSessionId = sessionId; - promptParams.path.id = sessionId; - - await awaitWithAbort(sleep(computeRetryDelay(attempt - 1, lastErr)), requestAbortController.signal); + await awaitWithAbort(sleep(computeRetryDelay(attempt - 1, lastErr)), requestAbortController.signal); } try { - await awaitWithAbort(activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS), requestAbortController.signal); - const polled = await activePollForAssistantResponse( - sessionId as string, - REQUEST_TIMEOUT_MS, - undefined, - requestAbortController.signal, + if (!((parentSessionId || idempotency?.submitted) && promptSubmitted)) { + if (parentSessionId) { + promptSubmitted = true; + } + if (idempotency) { + idempotency.markSubmitted( + sessionId as string, + String(promptParams.body['messageID']), + `${pID}/${mID}`, + ); + } + promptSubmitted = true; + await awaitWithAbort(activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS), requestAbortController.signal); + promptSubmitted = true; + } + const polled = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + requestAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : parentSessionId ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), ); - if (polled.error && !polled.content && !polled.reasoning) throw normalizeBackendError(polled.error); - return polled; + if (idempotency?.submitted) confirmExactTerminalPoll(polled); + if (polled.error != null) { + if (idempotency?.submitted) confirmedTerminalAssistantError = true; + throw normalizeBackendError(polled.error); + } + if (polled.partial === true) throw new Error('Polling returned incomplete response'); + if ( + !polled.content && + !polled.reasoning && + !(Array.isArray(polled.toolParts) && polled.toolParts.length > 0) + ) { + throw new Error('Polling returned empty response'); + } + return polled; } catch (e: unknown) { lastErr = e; + if (confirmedTerminalAssistantError) throw normalizeBackendError(e); + if (parentSessionId) { + throw normalizeBackendError(e); + } if (attempt < maxAttempts && engageProxyFallback(e)) continue; + if (attempt < maxAttempts && isTransientUpstreamError(e)) { + console.warn( + `[Proxy] Transient upstream error in interactions (attempt ${attempt}/${maxAttempts}), retrying:`, + toErrorMessage(e), + ); + continue; + } throw normalizeBackendError(e); } } throw normalizeBackendError(lastErr ?? new Error('Upstream provider error')); }; - const interactionId = `intr_${crypto.randomUUID()}`; + const interactionId = idempotency?.publicId ?? `intr_${crypto.randomUUID()}`; + idempotency?.setPublicId(interactionId); const deleteEphemeralSession = async (force: boolean = false): Promise => { // Only sessions created by this request may be deleted; a reused // parent session stays alive for its stored continuation chain. - if ((!force && store) || !sessionNewlyCreated) return; - - try { - await activeClient.session.delete({ path: { id: sessionId as string } }); - ownedSessionId = null; - } catch (e: unknown) { - logDebug('Failed to delete unstored interaction session', { error: toErrorMessage(e) }); - - } + if (!force && store) return; + await cleanupOwnedSessions(); }; if (stream) { res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); - const flushHeaders = (res as unknown as { flushHeaders?: unknown }).flushHeaders; - if (typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); + res.setHeader('X-Accel-Buffering', 'no'); + const flushHeaders = (res as unknown as { flushHeaders?: unknown }).flushHeaders; + if (!idempotency?.reconciling && typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); // Stream contract: interaction.created → step.delta* → interaction.completed // (no [DONE]; errors arrive as {type:'error'} like /v1/messages). const heartbeat = setInterval(() => { - if (!res.destroyed && !res.writableEnded) res.write(': heartbeat\n\n'); + if (!idempotency?.reconciling && !res.destroyed && !res.writableEnded) res.write(': heartbeat\n'); }, 15000); if (typeof heartbeat.unref === 'function') heartbeat.unref(); const emit = (payload: unknown): void => { @@ -415,8 +558,16 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v res.write(`data: ${JSON.stringify(payload)}\n\n`); }; try { - emit({ type: 'interaction.created', interaction: { id: interactionId, status: 'in_progress', model: `${pID}/${mID}` } }); - const polled = await promptAndPoll(); + if (!idempotency?.reconciling) { + emit({ type: 'interaction.created', interaction: { id: interactionId, status: 'in_progress', model: `${pID}/${mID}` } }); + interactionCreated = true; + } + const polled = await promptAndPoll(); + assertIdempotencyResultConfirmed(); + if (idempotency?.reconciling) { + emit({ type: 'interaction.created', interaction: { id: interactionId, status: 'in_progress', model: `${pID}/${mID}` } }); + interactionCreated = true; + } if (clientGone) { await deleteEphemeralSession(true); return; @@ -426,8 +577,9 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v // DISABLE_TOOLS so a free-tier strip (no false entries in the tools map, // see selectPromptToolOverrides) cannot leak call syntax to clients. // stripFunctionCalls is a no-op unless DISABLE_TOOLS=true. - const text = stripFunctionCalls(polled.content || ''); - // Chunked deltas keep clients alive on long answers. + const text = stripFunctionCalls(polled.content || ''); + if (!text.trim()) throw new Error('Interactions response contained no visible text'); + // Chunked deltas keep clients alive on long answers. for (let i = 0; i < text.length; i += 500) { emit({ type: 'step.delta', interaction_id: interactionId, delta: text.slice(i, i + 500) }); } @@ -446,13 +598,16 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v }); } steps.push({ type: 'model_output', text, annotations }); - emit({ type: 'interaction.completed', interaction: { id: interactionId, status: 'completed', output_text: text, steps } }); - if (store) { - // Text+grounding only: no function tool calls to persist, default [] is intentional. - storeResponseState(interactionId, sessionId as string, `${pID}/${mID}`); - ownedSessionId = null; - } - await deleteEphemeralSession(); + if (store) { + const stateStored = polled.assistantMessageId + ? storeResponseState(interactionId, sessionId as string, `${pID}/${mID}`, [], polled.assistantMessageId, tenantId) + : false; + if (stateStored) ownedSessionIds.delete(sessionId as string); + await cleanupOwnedSessions(); + } + assertIdempotencyResultConfirmed(); + emit({ type: 'interaction.completed', interaction: { id: interactionId, status: 'completed', output_text: text, steps } }); + await deleteEphemeralSession(); } finally { clearInterval(heartbeat); @@ -461,9 +616,11 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v return; } - const polled = await promptAndPoll(); + const polled = await promptAndPoll(); + assertIdempotencyResultConfirmed(); // Same parity strip as the stream path above. const text = stripFunctionCalls(polled.content || ''); + if (!text.trim()) throw new Error('Interactions response contained no visible text'); const evidence = hostedSearch.requested ? extractSearchEvidence(polled.toolParts ?? []) : { queries: [] as string[], sources: [] as { url: string; title: string }[] }; const searchCalls = buildWebSearchCallItems(evidence); const annotations = buildCitationAnnotations(text, evidence.sources); @@ -480,14 +637,17 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v } steps.push({ type: 'model_output', text, annotations }); const groundingCount = searchCalls.length; - if (store) { - // Text+grounding only: no function tool calls to persist, default [] is intentional. - storeResponseState(interactionId, sessionId as string, `${pID}/${mID}`); - ownedSessionId = null; - } - await deleteEphemeralSession(); + if (store) { + const stateStored = polled.assistantMessageId + ? storeResponseState(interactionId, sessionId as string, `${pID}/${mID}`, [], polled.assistantMessageId, tenantId) + : false; + if (stateStored) ownedSessionIds.delete(sessionId as string); + await cleanupOwnedSessions(); + } + await deleteEphemeralSession(); - res.json({ + assertIdempotencyResultConfirmed(); + res.json({ id: interactionId, status: 'completed', model: `${pID}/${mID}`, @@ -496,17 +656,80 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v usage: { grounding_tool_count: [{ type: 'google_search', count: groundingCount }] }, }); return; - } catch (error: unknown) { - if (clientGone && ownedSessionId) { - try { - await activeClient.session.delete({ path: { id: ownedSessionId } }); - } catch (cleanupError: unknown) { - logDebug('Failed to cleanup disconnected interaction session', { error: toErrorMessage(cleanupError) }); - } - ownedSessionId = null; - return; - } - if (!fallbackToProxy) engageProxyFallback(error); + } catch (error: unknown) { + await cleanupOwnedSessions(); + if (clientGone) { + return; + } + const isUnconfirmedStreamingReconciliation = + idempotency?.reconciling && asRecord((req as unknown as { body: unknown }).body)['stream'] === true; + if ( + idempotency?.submitted && + !hasConfirmedIdempotencyResult() && + !res.headersSent && + !isUnconfirmedStreamingReconciliation + ) { + res.removeHeader('Content-Type'); + res.status(503).json({ + error: { + type: 'idempotency_error', + code: 'idempotency_reconciliation_pending', + message: 'The previous outcome is still uncertain; retry the same request to reconcile it', + retryable: true, + }, + }); + return; + } + if (isUnconfirmedStreamingReconciliation && !res.destroyed && !res.writableEnded) { + const transformed = transformUpstreamError(error); + console.error('[Proxy] Interactions API Error:', toErrorMessage(error)); + if (!res.headersSent) { + res.status(200); + res.setHeader('Content-Type', 'text/event-stream'); + res.setHeader('Cache-Control', 'no-cache'); + res.setHeader('Connection', 'keep-alive'); + res.setHeader('X-Accel-Buffering', 'no'); + } + if (!interactionCreated) { + res.write(`data: ${JSON.stringify({ + type: 'interaction.created', + interaction: { + id: idempotency?.publicId ?? `intr_${crypto.randomUUID()}`, + status: 'in_progress', + model: idempotency?.model ?? 'unknown', + }, + })}\n\n`); + interactionCreated = true; + } + res.write(`data: ${JSON.stringify({ type: 'error', error: transformed.error })}\n\n`); + res.end(); + return; + } + if ( + asRecord((req as unknown as { body: unknown }).body)['stream'] === true && + idempotency?.reconciling && + idempotency.resultConfirmed && + !res.destroyed && + !res.writableEnded + ) { + const transformed = transformUpstreamError(error); + console.error('[Proxy] Interactions API Error:', toErrorMessage(error)); + if (!interactionCreated) { + res.write(`data: ${JSON.stringify({ + type: 'interaction.created', + interaction: { + id: idempotency.publicId ?? `intr_${crypto.randomUUID()}`, + status: 'in_progress', + model: idempotency.model ?? 'unknown', + }, + })}\n\n`); + interactionCreated = true; + } + res.write(`data: ${JSON.stringify({ type: 'error', error: transformed.error })}\n\n`); + res.end(); + return; + } + if (!fallbackToProxy) engageProxyFallback(error); console.error('[Proxy] Interactions API Error:', toErrorMessage(error)); const transformed = transformUpstreamError(error); @@ -518,16 +741,29 @@ export function registerInteractionsRoutes(app: Application, ctx: AppContext): v } catch { // ignore } - res.end(); + if (!res.writableEnded && !res.destroyed) { + res.end(); + } } return; + } finally { + res.removeListener('close', onClientClose); } }; - app.post('/v1beta/interactions', (req: Request, res: Response): void => { - void handle(req, res); - }); - app.post('/v1/interactions', (req: Request, res: Response): void => { - void handle(req, res); + app.post(['/v1beta/interactions', '/v1/interactions'], (req: Request, res: Response): void => { + void handle(req, res).catch((err: unknown) => { + const transformed = transformUpstreamError(err); + if (!res.headersSent) { + res.status(transformed.statusCode).json(transformed.error); + } else if (!res.writableEnded && !res.destroyed) { + try { + res.write(`data: ${JSON.stringify({ type: 'error', error: transformed.error })}\n\n`); + } catch { + // ignore + } + res.end(); + } + }); }); } diff --git a/src/routes/messages.ts b/src/routes/messages.ts index c9aea3f..21e366e 100644 --- a/src/routes/messages.ts +++ b/src/routes/messages.ts @@ -11,6 +11,7 @@ import { buildAnthropicMessage, sseEvent, estimateTokens, + toAnthropicErrorType, } from '../converters/anthropic.js'; import { defaultTranslatorRegistry } from '../converters/registry.js'; import { ensureTranslatorsRegistered } from '../converters/wire.js'; @@ -23,6 +24,10 @@ import { createToolCallFilter, createExternalToolCallStreamParser, } from '../tool-runtime/parser.js'; +import { + isMalformedExternalToolCallError, + resolveAuthoritativeStreamSource, +} from '../tool-runtime/stream-source.js'; import { isTransientUpstreamError, normalizeBackendError, transformUpstreamError } from '../errors/upstream.js'; import { engageFallbackForFreeLimit } from '../upstream-proxy/fallback.js'; import { detectHostedSearchTools } from '../search/grounding.js'; @@ -36,8 +41,19 @@ import { getImageDataUri } from '../stream/collector.js'; import type { Application, Request, Response } from 'express'; import type { AppContext } from '../types/context.js'; import type { FinalToolCall } from '../tool-runtime/parser.js'; +import type { AuthoritativeStreamSource } from '../tool-runtime/stream-source.js'; import { asRecord, toErrorMessage } from '../utils/guards.js'; +function readDataMessage(e: unknown): string { + const r = asRecord(e); + const data = asRecord(r['data']); + const dm: unknown = data['message']; + if (typeof dm === 'string') return dm; + const m: unknown = r['message']; + if (typeof m === 'string') return m; + return 'unknown'; +} + export function registerMessagesRoutes(app: Application, ctx: AppContext): void { const { client, @@ -71,12 +87,132 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void const activeTranslators = translators ?? ensureTranslatorsRegistered(defaultTranslatorRegistry()); app.post('/v1/messages', async (req: Request, res: Response): Promise => { + let textBlockOpen = false; + let thinkingBlockOpen = false; + let textIndex: number | null = null; + let thinkingIndex: number | null = null; + const closeOpenBlocks = (): void => { + if (res.destroyed || res.writableEnded) return; + try { + if (thinkingBlockOpen && thinkingIndex !== null) { + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: thinkingIndex })); + thinkingBlockOpen = false; + } + if (textBlockOpen && textIndex !== null) { + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: textIndex })); + textBlockOpen = false; + } + } catch { + // ignore + } + }; try { + const rawBody: unknown = (req as unknown as { body: unknown }).body; + const validationError = validateMessagesRequest(rawBody); + if (validationError) { + res.status(validationError.statusCode).json(validationError.body); + return; + } + const body = asRecord(rawBody); + const model: unknown = body['model']; + const system: unknown = body['system']; + const messagesRaw: unknown = body['messages']; + const toolsRaw: unknown = body['tools']; + const tools: unknown[] = Array.isArray(toolsRaw) ? (toolsRaw as unknown[]) : []; + // Anthropic server-side web_search would be silently dropped by the + // function-tool bridge; fail loudly with a pointer instead. Uses the + // shared helper (covers versioned types like web_search_20260222) + // plus the bare `web_search` name for name-only defs. + const hasHostedSearch = + detectHostedSearchTools(tools).requested || + tools.some((def: unknown) => { + const r = asRecord(def); + return typeof r['name'] === 'string' && String(r['name']) === 'web_search'; + }); + if (hasHostedSearch) { + res.status(400).json({ + type: 'error', + error: { + type: 'invalid_request_error', + message: + 'web_search is not supported on /v1/messages; use POST /v1/responses with tools:[{type:"web_search"}] or POST /v1beta/interactions with tools:[{type:"google_search"}]', + }, + }); + return; + } + const tool_choice: unknown = body['tool_choice']; + const requestStream: unknown = body['stream']; + const temperature: unknown = body['temperature']; + const top_p: unknown = body['top_p']; + const top_k: unknown = body['top_k']; + const max_tokens: unknown = body['max_tokens']; + const stop_sequences: unknown = body['stop_sequences']; + const thinking: unknown = body['thinking']; + const stream = Boolean(requestStream); + const requestOpencodeConfig: unknown = body['opencode']; + const disableParallelRaw = body['disable_parallel_tool_use']; + const choiceDisableParallel = asRecord(tool_choice)['disable_parallel_tool_use']; + const parallelToolCalls: boolean = disableParallelRaw === true || choiceDisableParallel === true ? false : true; + // Phase 2 wiring: inbound claude.request -> chat.request via the wired + // N×N registry (thin wrapper over the same anthropic.ts pure layer, + // so shapes match). Direct registry call (not the Safe wrapper): + // requests are validated above and never error envelopes; an extra + // `type:'error'` field must not bypass conversion (mode confusion). + // Translated tools still flow through createRequestToolContext below + // (external__* bridge + server ∩ request), unchanged. + const translatedChatRequest = asRecord( + activeTranslators.translateRequest( + 'claude', + 'openai', + typeof model === 'string' ? model : '', + { + model, + system, + messages: messagesRaw, + tools, + tool_choice, + thinking, + max_tokens, + temperature, + top_p, + stop_sequences, + }, + stream, + ), + ); + const chatMessages = ( + Array.isArray(translatedChatRequest['messages']) ? (translatedChatRequest['messages'] as unknown[]) : [] + ) as unknown as Array<{ role?: unknown; content?: unknown }>; + const chatTools = Array.isArray(translatedChatRequest['tools']) ? (translatedChatRequest['tools'] as unknown[]) : []; + const chatToolChoice = translatedChatRequest['tool_choice']; + // Strict legacy parity via the testable gate (collapses the whole + // adaptive/auto family to the legacy fallback; translator itself + // stays Go-faithful). + const translatedEffort = translatedChatRequest['reasoning_effort']; + const reasoningLevel = resolveMessagesReasoningLevel(thinking, translatedEffort, normalizeReasoningEffort); + + const requestParams: Record = { + temperature: typeof temperature === 'number' ? temperature : 0.7, + max_tokens: typeof max_tokens === 'number' ? max_tokens : null, + top_p: typeof top_p === 'number' ? top_p : 1.0, + stop: Array.isArray(stop_sequences) ? stop_sequences : null, + reasoning_effort: reasoningLevel, + }; + if (top_k !== undefined) logDebug('Ignoring top_k (no opencode equivalent)', { top_k }); + + const resolvedModel = (await withTimeout(resolveRequestedModel(model), REQUEST_TIMEOUT_MS, 'resolve model')) as unknown as { + providerID: string; + modelID: string; + }; + const pID = String(asRecord(resolvedModel)['providerID']); + const mID = String(asRecord(resolvedModel)['modelID']); + const publicModel = `${pID}/${mID}`; + await lock(async (): Promise => { - let sessionId: string | null = null; - let keepaliveInterval: ReturnType | null = null; - const requestAbortController = new AbortController(); - let disconnectedSessionCleaned = false; + let sessionId: string | null = null; + let keepaliveInterval: ReturnType | null = null; + const requestAbortController = new AbortController(); + let disconnectedSessionCleaned = false; // P3 fallback bundle: direct by default; adopts the proxy bundle when // this request hits a free-limit error (or the pool is already engaged). @@ -84,6 +220,7 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void let activePromptWithTimeout = promptWithTimeout; let activePollForAssistantResponse = pollForAssistantResponse; let fallbackToProxy = false; + let fallbackServed = false; const switchToProxyBundle = (): boolean => { if (!proxyClient) return false; activeClient = proxyClient; @@ -98,107 +235,6 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void }; if (proxyPool.isEngaged()) switchToProxyBundle(); try { - const rawBody: unknown = (req as unknown as { body: unknown }).body; - const validationError = validateMessagesRequest(rawBody); - if (validationError) { - res.status(validationError.statusCode).json(validationError.body); - return; - } - const body = asRecord(rawBody); - const model: unknown = body['model']; - const system: unknown = body['system']; - const messagesRaw: unknown = body['messages']; - const toolsRaw: unknown = body['tools']; - const tools: unknown[] = Array.isArray(toolsRaw) ? (toolsRaw as unknown[]) : []; - // Anthropic server-side web_search would be silently dropped by the - // function-tool bridge; fail loudly with a pointer instead. Uses the - // shared helper (covers versioned types like web_search_20260222) - // plus the bare `web_search` name for name-only defs. - const hasHostedSearch = - detectHostedSearchTools(tools).requested || - tools.some((def: unknown) => { - const r = asRecord(def); - return typeof r['name'] === 'string' && String(r['name']) === 'web_search'; - }); - if (hasHostedSearch) { - res.status(400).json({ - type: 'error', - error: { - type: 'invalid_request_error', - message: - 'web_search is not supported on /v1/messages; use POST /v1/responses with tools:[{type:"web_search"}] or POST /v1beta/interactions with tools:[{type:"google_search"}]', - }, - }); - return; - } - const tool_choice: unknown = body['tool_choice']; - const requestStream: unknown = body['stream']; - const temperature: unknown = body['temperature']; - const top_p: unknown = body['top_p']; - const top_k: unknown = body['top_k']; - const max_tokens: unknown = body['max_tokens']; - const stop_sequences: unknown = body['stop_sequences']; - const thinking: unknown = body['thinking']; - const stream = Boolean(requestStream); - const requestOpencodeConfig: unknown = body['opencode']; - const disableParallelRaw = body['disable_parallel_tool_use']; - const choiceDisableParallel = asRecord(tool_choice)['disable_parallel_tool_use']; - const parallelToolCalls: boolean = disableParallelRaw === true || choiceDisableParallel === true ? false : true; - // Phase 2 wiring: inbound claude.request -> chat.request via the wired - // N×N registry (thin wrapper over the same anthropic.ts pure layer, - // so shapes match). Direct registry call (not the Safe wrapper): - // requests are validated above and never error envelopes; an extra - // `type:'error'` field must not bypass conversion (mode confusion). - // Translated tools still flow through createRequestToolContext below - // (external__* bridge + server ∩ request), unchanged. - const translatedChatRequest = asRecord( - activeTranslators.translateRequest( - 'claude', - 'openai', - typeof model === 'string' ? model : '', - { - model, - system, - messages: messagesRaw, - tools, - tool_choice, - thinking, - max_tokens, - temperature, - top_p, - stop_sequences, - }, - stream, - ), - ); - const chatMessages = ( - Array.isArray(translatedChatRequest['messages']) ? (translatedChatRequest['messages'] as unknown[]) : [] - ) as unknown as Array<{ role?: unknown; content?: unknown }>; - const chatTools = Array.isArray(translatedChatRequest['tools']) ? (translatedChatRequest['tools'] as unknown[]) : []; - const chatToolChoice = translatedChatRequest['tool_choice']; - // Strict legacy parity via the testable gate (collapses the whole - // adaptive/auto family to the legacy fallback; translator itself - // stays Go-faithful). - const translatedEffort = translatedChatRequest['reasoning_effort']; - const reasoningLevel = resolveMessagesReasoningLevel(thinking, translatedEffort, normalizeReasoningEffort); - - const requestParams: Record = { - temperature: typeof temperature === 'number' ? temperature : 0.7, - max_tokens: typeof max_tokens === 'number' ? max_tokens : null, - top_p: typeof top_p === 'number' ? top_p : 1.0, - stop: Array.isArray(stop_sequences) ? stop_sequences : null, - reasoning_effort: reasoningLevel, - }; - if (top_k !== undefined) logDebug('Ignoring top_k (no opencode equivalent)', { top_k }); - - const resolvedModel = (await withTimeout(resolveRequestedModel(model), REQUEST_TIMEOUT_MS, 'resolve model')) as unknown as { - providerID: string; - modelID: string; - }; - const pID = String(asRecord(resolvedModel)['providerID']); - const mID = String(asRecord(resolvedModel)['modelID']); - const publicModel = `${pID}/${mID}`; - const requestToolContext = createRequestToolContext(chatTools, chatToolChoice, requestOpencodeConfig); const toolMode: string = requestToolContext.mode; const externalToolContext = requestToolContext.external; @@ -418,6 +454,8 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void return; } if (attempt > 1) { + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); try { await activeClient.session.delete({ path: { id: sessionId } }); } catch { @@ -439,9 +477,18 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void requestAbortController.signal, ); - content = collected.content || ''; - reasoning = collected.reasoning || ''; - error = collected.error || null; + content = collected.content || ''; + reasoning = collected.reasoning || ''; + error = + collected.error || + (collected.partial === true + ? new Error('Polling returned incomplete response') + : externalToolChoice.mode !== 'required' && + !content && + !reasoning && + !(Array.isArray(collected.toolParts) && collected.toolParts.length > 0) + ? new Error('Polling returned empty response') + : null); } catch (promptError: unknown) { content = ''; reasoning = ''; @@ -454,10 +501,11 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void // Fail-closed: any backend error fails even with partial content (see chat.ts). if (error != null) { const t = transformUpstreamError(error); + const anthropicType = toAnthropicErrorType(t.statusCode, t.error.type); res.status(t.statusCode).json({ type: 'error', error: { - type: t.error.type || 'api_error', + type: anthropicType, message: t.error.message || 'Upstream error', ...(t.error.code ? { code: t.error.code } : {}), }, @@ -522,10 +570,11 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); + res.setHeader('X-Accel-Buffering', 'no'); const flushHeaders = (res as unknown as { flushHeaders?: unknown }).flushHeaders; if (typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); keepaliveInterval = setInterval(() => { - if (!res.destroyed && !res.writableEnded) res.write(': keep-alive\n\n'); + if (!res.destroyed && !res.writableEnded) res.write(': keep-alive\n'); }, 15000); const resClosed = new Promise((resolve) => res.once('close', () => { @@ -545,15 +594,15 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void // markup is never emitted as incremental `content_block_delta` before it // can be classified. Pure-text turns with tools declared also batch. const shouldBufferExternalStream = externalToolRegistry.length > 0; - const filterContent = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); - const filterReasoning = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); - const parseContent = createExternalToolCallStreamParser(externalToolRegistry); - const parseReason = createExternalToolCallStreamParser(externalToolRegistry); - let textBlockOpen = false; - let thinkingBlockOpen = false; - let nextBlockIndex = 0; - let textIndex: number | null = null; - let thinkingIndex: number | null = null; + let filterContent = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); + let filterReasoning = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); + let parseContent = createExternalToolCallStreamParser(externalToolRegistry); + let parseReason = createExternalToolCallStreamParser(externalToolRegistry); + textBlockOpen = false; + thinkingBlockOpen = false; + let nextBlockIndex = 0; + textIndex = null; + thinkingIndex = null; const ensureTextIndex = (): number => { if (textIndex === null) textIndex = nextBlockIndex++; return textIndex; @@ -570,6 +619,10 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void ); const sendTextDelta = (delta: string): void => { if (!delta) return; + if (thinkingBlockOpen && thinkingIndex !== null) { + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: thinkingIndex })); + thinkingBlockOpen = false; + } const idx = ensureTextIndex(); if (!textBlockOpen) { res.write(sseEvent('content_block_start', { type: 'content_block_start', index: idx, content_block: { type: 'text', text: '' } })); @@ -580,6 +633,10 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void }; const sendReasoningDelta = (delta: string): void => { if (!delta) return; + if (textBlockOpen && textIndex !== null) { + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: textIndex })); + textBlockOpen = false; + } const idx = ensureThinkingIndex(); if (!thinkingBlockOpen) { res.write(sseEvent('content_block_start', { type: 'content_block_start', index: idx, content_block: { type: 'thinking', thinking: '', signature: '' } })); @@ -603,10 +660,16 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void else sendTextDelta(filtered); }; let collected: Record | null = null; - let polledSnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null = null; + let polledSnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean } | null = null; let snapshotPolled = false; - let snapshotPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> | null = null; - const pollSnapshot = async (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> => { + let snapshotPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }> | null = null; + const hasRecoveryEvidence = (snapshot: { content: string; reasoning: string; toolParts?: unknown }): boolean => + Boolean( + snapshot.content || + snapshot.reasoning || + (Array.isArray(snapshot.toolParts) && snapshot.toolParts.length > 0), + ); + const pollSnapshot = async (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }> => { snapshotPolled = true; if (!snapshotPromise) { snapshotPromise = activePollForAssistantResponse( @@ -620,103 +683,202 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void return snapshotPromise; }; // Recovery snapshots repeat what the event stream already delivered, so only the - // suffix past the raw prefix may be replayed. A snapshot the raw prefix does not - // match is still sent in full rather than dropped. + // suffix past the raw prefix may be replayed. A divergent snapshot is not a safe + // replay (raw "AC" + snapshot "ABC" would render "ACABC"); tool calls still come + // from the snapshot while visible text keeps the raw stream. const unsentSuffix = (full: unknown, raw: string): string => { const text = typeof full === 'string' ? full : ''; if (!text) return ''; if (!raw) return text; - return text.startsWith(raw) ? text.slice(raw.length) : text; - }; - const hasPartialOutput = (): boolean => { - const current = asRecord(collected); - return Boolean(rawContent || rawReasoning || streamedToolCalls.length > 0 || current['content'] || current['reasoning']); + return text.startsWith(raw) ? text.slice(raw.length) : ''; }; const recoverSnapshot = async ( fallbackError: unknown = null, - ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown } | null> => { + ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean } | null> => { if (res.destroyed || res.writableEnded) return null; - let snapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown }; + let snapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown; partial?: boolean }; try { snapshot = await pollSnapshot(); } catch (error: unknown) { if (res.destroyed || res.writableEnded) return null; - if (hasPartialOutput()) { - logDebug('Ignoring messages recovery poll failure after partial stream output', { error: toErrorMessage(error) }); - return null; - } if (fallbackError != null) throw normalizeBackendError(fallbackError); throw normalizeBackendError(error); - } - if (snapshot.error != null) throw normalizeBackendError(snapshot.error); - if ( - fallbackError != null && - !snapshot.content && - !snapshot.reasoning && - !(Array.isArray(snapshot.toolParts) && snapshot.toolParts.length > 0) - ) throw normalizeBackendError(fallbackError); - return snapshot; + } + if (snapshot.error != null) throw snapshot.error; + if (snapshot.partial === true) throw new Error('Recovery poll returned incomplete response'); + if (!hasRecoveryEvidence(snapshot)) { + if (fallbackError != null) throw normalizeBackendError(fallbackError); + return null; + } + return snapshot; }; - if (fallbackToProxy) { - // SSE subscribe ignores custom fetch (SDK gap): prompt+poll - // through the proxy bundle instead of the event stream. + let lastStreamAttemptError: unknown = null; + let promptPromise: Promise | null = null; + let promptError: unknown = null; + for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { + collected = null; + promptPromise = null; + promptError = null; + if (attempt > 1) { + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); + try { + await activeClient.session.delete({ path: { id: sessionId } }); + } catch (e: unknown) { + logDebug('Failed to delete retried messages session', { sessionId, error: toErrorMessage(e) }); + } + const retrySessionRes = (await withTimeout( + activeClient.session.create(), + REQUEST_TIMEOUT_MS, + 'create session', + )) as unknown; + sessionId = (asRecord(asRecord(retrySessionRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); + promptParams.path.id = sessionId; + requestForcedMessagesToolCall = makeForcedMessagesToolCallRequester(); + streamedText = ''; + streamedReasoning = ''; + rawContent = ''; + rawReasoning = ''; + streamedToolCalls.length = 0; + filterContent = createToolCallFilter({ + disableTools: DISABLE_TOOLS, + forceStrip: shouldBufferExternalStream, + registry: externalToolRegistry, + }); + filterReasoning = createToolCallFilter({ + disableTools: DISABLE_TOOLS, + forceStrip: shouldBufferExternalStream, + registry: externalToolRegistry, + }); + parseContent = createExternalToolCallStreamParser(externalToolRegistry); + parseReason = createExternalToolCallStreamParser(externalToolRegistry); + polledSnapshot = null; + snapshotPolled = false; + snapshotPromise = null; + textBlockOpen = false; + thinkingBlockOpen = false; + nextBlockIndex = 0; + textIndex = null; + thinkingIndex = null; + fallbackServed = false; + await sleep(computeRetryDelay(attempt - 1, lastStreamAttemptError)); + } try { - await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); - const pres = await pollSnapshot(); - polledSnapshot = pres; - if (pres.error != null) throw normalizeBackendError(pres.error); - if (res.destroyed || res.writableEnded) { - if (keepaliveInterval) clearInterval(keepaliveInterval); - return; + if (fallbackToProxy) { + // SSE subscribe ignores custom fetch (SDK gap): prompt+poll + // through the proxy bundle instead of the event stream. + await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); + const pres = await pollSnapshot(); + if (pres.error != null) throw normalizeBackendError(pres.error); + if (pres.partial === true) throw new Error('Recovery poll returned incomplete response'); + if (!hasRecoveryEvidence(pres) && externalToolChoice.mode !== 'required') { + throw new Error('Recovery poll returned empty response'); + } + polledSnapshot = pres; + if (res.destroyed || res.writableEnded) { + if (keepaliveInterval) clearInterval(keepaliveInterval); + return; + } + // Raw text: sendDelta parses tool calls and filters display markup. + const presContent = unsentSuffix(pres.content, rawContent); + const presReasoning = unsentSuffix(pres.reasoning, rawReasoning); + if (presContent) sendDelta(presContent, false); + if (presReasoning) sendDelta(presReasoning, true); + if (pres.content || pres.reasoning) fallbackServed = true; + collected = { content: pres.content, reasoning: pres.reasoning, toolParts: pres.toolParts }; + } else { + const collectPromise = collectFromEvents( + sessionId as string, + REQUEST_TIMEOUT_MS, + sendDelta, + DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS, + DEFAULT_EVENT_IDLE_TIMEOUT_MS, + requestAbortController.signal, + ).catch((err: unknown) => ({ __error: err })); + + promptPromise = activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS) + .then(() => undefined) + .catch((err: unknown) => { + promptError = err; + logDebug('Prompt error:', toErrorMessage(err)); + }); + const raced = (await Promise.race([collectPromise, resClosed.then(() => ({ __cancelled: true }))])) as Record; + collected = raced; + if (raced['__cancelled'] || requestAbortController.signal.aborted) { + if (keepaliveInterval) clearInterval(keepaliveInterval); + try { + if (sessionId) { + await activeClient.session.delete({ path: { id: sessionId } }); + disconnectedSessionCleaned = true; + } + } catch (e: unknown) { + logDebug('Failed to cleanup cancelled messages session', { error: toErrorMessage(e) }); + } + try { + if (!res.destroyed) res.end(); + } catch { + // ignore + } + return; + } } - // Raw text: sendDelta parses tool calls and filters display markup. - const presContent = unsentSuffix(pres.content, rawContent); - const presReasoning = unsentSuffix(pres.reasoning, rawReasoning); - if (presContent) sendDelta(presContent, false); - if (presReasoning) sendDelta(presReasoning, true); - collected = { content: pres.content, reasoning: pres.reasoning }; } catch (e: unknown) { - throw normalizeBackendError(e); + collected = { __error: e }; } - } else { - const collectPromise = collectFromEvents( - sessionId as string, - REQUEST_TIMEOUT_MS, - sendDelta, - DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS, - DEFAULT_EVENT_IDLE_TIMEOUT_MS, - requestAbortController.signal, - ).catch((err: unknown) => ({ __error: err })); - activeClient.session.prompt(promptParams).catch((err: unknown) => logDebug('Prompt error:', toErrorMessage(err))); - const raced = (await Promise.race([collectPromise, resClosed.then(() => ({ __cancelled: true }))])) as Record; - collected = raced; - if (raced['__cancelled'] || requestAbortController.signal.aborted) { - if (keepaliveInterval) clearInterval(keepaliveInterval); - try { - if (sessionId) { - await activeClient.session.delete({ path: { id: sessionId } }); - disconnectedSessionCleaned = true; - } - } catch (e: unknown) { - logDebug('Failed to cleanup cancelled messages session', { error: toErrorMessage(e) }); - } - try { - if (!res.destroyed) res.end(); - } catch { - // ignore - } - return; - } - if (raced['error'] != null) throw normalizeBackendError(raced['error']); - const recoveryRecord = asRecord(collected); - const hasValidTerminal = Boolean( + const collectedRecord = asRecord(collected); + const nothingStreamed = !rawContent && !rawReasoning && streamedToolCalls.length === 0; + if (promptPromise && nothingStreamed) await promptPromise; + let attemptError: unknown = promptError ?? collectedRecord['error'] ?? collectedRecord['__error'] ?? null; + if (!attemptError && (collectedRecord['noData'] || collectedRecord['cut'] || collectedRecord['idleTimeout'])) { + try { + const recovery = await pollSnapshot(); + if (recovery.error != null) { + attemptError = recovery.error; + } else if (recovery.partial === true) { + attemptError = new Error('Recovery poll returned incomplete response'); + } else if (!hasRecoveryEvidence(recovery)) { + attemptError = new Error('Recovery poll returned empty response'); + } else { + polledSnapshot = recovery; + const presContent = unsentSuffix(recovery.content, rawContent); + const presReasoning = unsentSuffix(recovery.reasoning, rawReasoning); + if (presContent) sendDelta(presContent, false); + if (presReasoning) sendDelta(presReasoning, true); + collected = { content: recovery.content, reasoning: recovery.reasoning, toolParts: recovery.toolParts }; + } + } catch (pollErr: unknown) { + attemptError = pollErr; + } + } + if (attemptError && nothingStreamed && attempt < maxAttempts && engageProxyFallback(attemptError)) { + lastStreamAttemptError = attemptError; + continue; + } + if (attemptError && nothingStreamed && attempt < maxAttempts && isTransientUpstreamError(attemptError)) { + console.warn( + `[Proxy] Transient upstream error in messages stream (attempt ${attempt}/${maxAttempts}), retrying:`, + readDataMessage(attemptError), + ); + lastStreamAttemptError = attemptError; + continue; + } + break; + } + + const recoveryRecord = asRecord(collected); + if (recoveryRecord['error'] != null) throw normalizeBackendError(recoveryRecord['error']); + const hasValidTerminal = Boolean( collected && !recoveryRecord['__error'] && !recoveryRecord['noData'] && !recoveryRecord['idleTimeout'] && + !recoveryRecord['cut'] && !recoveryRecord['cancelled'] && + recoveryRecord['finish'] !== 'tool' && + recoveryRecord['finish'] !== 'tool-calls' && (rawContent || rawReasoning || streamedToolCalls.length > 0 || recoveryRecord['content'] || recoveryRecord['reasoning']), ); if (recoveryRecord['__error']) { @@ -730,7 +892,7 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void if (remainingContent) sendDelta(remainingContent, false); if (remainingReasoning) sendDelta(remainingReasoning, true); } - } else if (recoveryRecord['noData'] || recoveryRecord['idleTimeout']) { + } else if (recoveryRecord['noData'] || recoveryRecord['idleTimeout'] || recoveryRecord['cut'] || !hasValidTerminal) { const snapshot = await recoverSnapshot(); if (snapshot) { @@ -751,7 +913,11 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void if (remainingContent) sendDelta(remainingContent, false); } } - } + if (!hasValidTerminal && !polledSnapshot && !fallbackServed) { + if (streamedToolCalls.length === 0 && externalToolChoice.mode !== 'required') { + throw new Error('Stream ended without valid terminal'); + } + } if (requestAbortController.signal.aborted || res.destroyed || res.writableEnded) { if (sessionId && !disconnectedSessionCleaned) { try { @@ -768,59 +934,135 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void const flushedContentCalls = parseContent.flush ? parseContent.flush() : []; const flushedReasoningText = filterReasoning.flush ? filterReasoning.flush() : ''; const flushedContentText = filterContent.flush ? filterContent.flush() : ''; + const collectedR = asRecord(collected); + if (collectedR['error'] != null) throw normalizeBackendError(collectedR['error']); + // One authoritative source group: snapshot when available, otherwise + // the raw delta text. Never validate both groups independently. + let authoritativeSource = resolveAuthoritativeStreamSource({ + collected: collectedR, + polledSnapshot, + rawContent, + rawReasoning, + }); + const streamSource = (): string[][] => [[authoritativeSource.reasoning, authoritativeSource.content]]; + if (shouldBufferExternalStream) { const visible = stripExternalToolCallMarkupFromJoinedText( externalToolRegistry, - rawReasoning, - rawContent, + authoritativeSource.reasoning, + authoritativeSource.content, ); streamedReasoning = visible.reasoning; streamedText = visible.content; } else { + if (rawContent && authoritativeSource.content && !authoritativeSource.content.startsWith(rawContent)) { + throw new Error('Stream content diverged from authoritative snapshot'); + } + if (rawReasoning && authoritativeSource.reasoning && !authoritativeSource.reasoning.startsWith(rawReasoning)) { + throw new Error('Stream reasoning diverged from authoritative snapshot'); + } if (flushedReasoningText) sendReasoningDelta(flushedReasoningText); if (flushedContentText) sendTextDelta(flushedContentText); + if (!streamedText && !streamedReasoning) { + const collectedRemainingContent = unsentSuffix(authoritativeSource.content, rawContent); + const collectedRemainingReasoning = unsentSuffix(authoritativeSource.reasoning, rawReasoning); + if (collectedRemainingReasoning) sendReasoningDelta(collectedRemainingReasoning); + if (collectedRemainingContent) sendTextDelta(collectedRemainingContent); + } } - const collectedR = asRecord(collected); - if (collectedR['error'] != null) throw normalizeBackendError(collectedR['error']); - const snapshotReasoning = polledSnapshot?.reasoning ?? (typeof collectedR['reasoning'] === 'string' ? collectedR['reasoning'] as string : rawReasoning); - const snapshotContent = polledSnapshot?.content ?? (typeof collectedR['content'] === 'string' ? collectedR['content'] as string : rawContent); const parseJoined = (reasoningText: string, contentText: string): FinalToolCall[] => externalToolRegistry.length > 0 ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoningText, contentText) : []; - const streamSource = [ - [snapshotReasoning, snapshotContent], - [rawReasoning, rawContent] - ]; - let parsedToolCalls: FinalToolCall[] = externalToolRegistry.length > 0 - ? mergeToolCallArtifacts( - streamedToolCalls, - flushedReasoningCalls, - flushedContentCalls, - parseJoined(snapshotReasoning, snapshotContent), - parseJoined(rawReasoning, rawContent), - ) - : []; - let finalValidated: FinalToolCall[]; - try { - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource); - if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required') { - const forcedResponse = await requestForcedMessagesToolCall(); - if (forcedResponse) { - const forcedReasoning = typeof forcedResponse['reasoning'] === 'string' ? forcedResponse['reasoning'] as string : ''; - const forcedContent = typeof forcedResponse['content'] === 'string' ? forcedResponse['content'] as string : ''; - parsedToolCalls = mergeToolCallArtifacts(parsedToolCalls, parseJoined(forcedReasoning, forcedContent)); - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [forcedReasoning, forcedContent]); - } - } - finalValidated = finalizeStreamToolCalls( - parsedToolCalls, - externalToolRegistry, - externalToolChoice, - streamSource, - parallelToolCalls, - ) as unknown as FinalToolCall[]; + const callsFromText = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? parseJoined(source.reasoning, source.content) + : []; + const callsFromRaw = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? mergeToolCallArtifacts( + streamedToolCalls, + flushedReasoningCalls, + flushedContentCalls, + parseJoined(source.reasoning, source.content), + ) + : []; + let parsedToolCalls: FinalToolCall[] = authoritativeSource.fromSnapshot + ? callsFromText(authoritativeSource) + : callsFromRaw(authoritativeSource); + const repairFromSnapshot = async (): Promise => { + if (externalToolRegistry.length === 0) return false; + let snapshot = polledSnapshot; + if (!snapshot) { + if (snapshotPolled) return false; + try { + snapshot = await recoverSnapshot(); + } catch { + return false; + } + } + if (!snapshot || snapshot.error != null) return false; + const repairedSource: AuthoritativeStreamSource = { + content: String(snapshot.content ?? ''), + reasoning: String(snapshot.reasoning ?? ''), + kind: 'polled-snapshot', + fromSnapshot: true, + }; + const snapshotCalls = parseJoined(repairedSource.reasoning, repairedSource.content); + if (snapshotCalls.length === 0) return false; + try { + assertToolCallArtifactIntegrity(snapshotCalls, externalToolRegistry, [ + [repairedSource.reasoning, repairedSource.content], + ]); + } catch { + return false; + } + polledSnapshot = snapshot; + authoritativeSource = repairedSource; + parsedToolCalls = snapshotCalls; + if (shouldBufferExternalStream) { + const visible = stripExternalToolCallMarkupFromJoinedText( + externalToolRegistry, + authoritativeSource.reasoning, + authoritativeSource.content, + ); + streamedReasoning = visible.reasoning; + streamedText = visible.content; + } + return true; + }; + let finalValidated: FinalToolCall[]; + try { + try { + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); + } catch (integrityError) { + if (!isMalformedExternalToolCallError(integrityError) || !(await repairFromSnapshot())) { + throw integrityError; + } + } + if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required') { + const forcedResponse = await requestForcedMessagesToolCall(); + if (forcedResponse) { + const forcedReasoning = typeof forcedResponse['reasoning'] === 'string' ? forcedResponse['reasoning'] as string : ''; + const forcedContent = typeof forcedResponse['content'] === 'string' ? forcedResponse['content'] as string : ''; + parsedToolCalls = mergeToolCallArtifacts(parsedToolCalls, parseJoined(forcedReasoning, forcedContent)); + authoritativeSource = { + content: `${authoritativeSource.content}${forcedContent}`, + reasoning: `${authoritativeSource.reasoning}${forcedReasoning}`, + kind: authoritativeSource.kind, + fromSnapshot: authoritativeSource.fromSnapshot, + }; + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); + } + } + finalValidated = finalizeStreamToolCalls( + parsedToolCalls, + externalToolRegistry, + externalToolChoice, + streamSource(), + parallelToolCalls, + ) as unknown as FinalToolCall[]; } catch (toolError) { const toolCode = (toolError as Error & { code?: string }).code; const failClosedCodes = [ @@ -847,6 +1089,7 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void error: { type: 'api_error', message: toolMessage, code: toolCode }, }); } else if (!res.destroyed) { + closeOpenBlocks(); try { res.write( sseEvent('error', { @@ -874,6 +1117,8 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void thinkingBlockOpen = true; } res.write(sseEvent('content_block_delta', { type: 'content_block_delta', index: idx, delta: { type: 'thinking_delta', thinking: streamedReasoning } })); + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: idx })); + thinkingBlockOpen = false; } if (streamedText) { const idx = ensureTextIndex(); @@ -882,10 +1127,11 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void textBlockOpen = true; } res.write(sseEvent('content_block_delta', { type: 'content_block_delta', index: idx, delta: { type: 'text_delta', text: streamedText } })); + res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: idx })); + textBlockOpen = false; } } - if (textBlockOpen) res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: textIndex })); - if (thinkingBlockOpen) res.write(sseEvent('content_block_stop', { type: 'content_block_stop', index: thinkingIndex })); + closeOpenBlocks(); let toolBlockIndex = nextBlockIndex; for (const tc of toPublicToolCalls(finalValidated)) { const tcr = tc as unknown as Record; @@ -946,22 +1192,25 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void } if (!res.headersSent) { const t = transformUpstreamError(error); + const anthropicType = toAnthropicErrorType(t.statusCode, t.error.type); res.status(t.statusCode).json({ type: 'error', error: { - type: t.error.type || 'api_error', + type: anthropicType, message: t.error.message || toErrorMessage(error) || 'Upstream error', ...(t.error.code ? { code: t.error.code } : {}), }, }); return; } + closeOpenBlocks(); try { const t = transformUpstreamError(error); + const anthropicType = toAnthropicErrorType(t.statusCode, t.error.type); res.write( sseEvent('error', { type: 'error', - error: { type: t.error.type || 'api_error', message: t.error.message, ...(t.error.code ? { code: t.error.code } : {}) }, + error: { type: anthropicType, message: t.error.message, ...(t.error.code ? { code: t.error.code } : {}) }, }), ); } catch { @@ -972,8 +1221,33 @@ export function registerMessagesRoutes(app: Application, ctx: AppContext): void } }, REQUEST_TIMEOUT_MS + 20000); } catch (error: unknown) { - if (!res.headersSent) - res.status(500).json({ type: 'error', error: { type: 'api_error', message: toErrorMessage(error) } }); + if (!res.headersSent) { + const t = transformUpstreamError(error); + const anthropicType = toAnthropicErrorType(t.statusCode, t.error.type); + res.status(t.statusCode).json({ + type: 'error', + error: { + type: anthropicType, + message: t.error.message || toErrorMessage(error) || 'Upstream error', + ...(t.error.code ? { code: t.error.code } : {}), + }, + }); + } else if (!res.writableEnded && !res.destroyed) { + closeOpenBlocks(); + try { + const t = transformUpstreamError(error); + const anthropicType = toAnthropicErrorType(t.statusCode, t.error.type); + res.write( + sseEvent('error', { + type: 'error', + error: { type: anthropicType, message: t.error.message, ...(t.error.code ? { code: t.error.code } : {}) }, + }), + ); + } catch { + // ignore + } + res.end(); + } } }); } diff --git a/src/routes/responses.ts b/src/routes/responses.ts index ec66d1f..c553b7e 100644 --- a/src/routes/responses.ts +++ b/src/routes/responses.ts @@ -1,5 +1,6 @@ // P4 TS: POST /v1/responses (ported from P3 .js, behavior identical). import crypto from 'crypto'; +import { makeId } from '../converters/json.js'; import { ADDITIONAL_TOOLS_ITEM_TYPE, collectAdditionalToolSources, @@ -21,6 +22,10 @@ import { createToolCallFilter, createExternalToolCallStreamParser, } from '../tool-runtime/parser.js'; +import { + isMalformedExternalToolCallError, + resolveAuthoritativeStreamSource, +} from '../tool-runtime/stream-source.js'; import { isTransientUpstreamError, normalizeBackendError, transformUpstreamError, createInvalidRequestError, createToolDiscoveryUnavailableError } from '../errors/upstream.js'; import { engageFallbackForFreeLimit } from '../upstream-proxy/fallback.js'; import { @@ -41,7 +46,9 @@ import type { AppContext } from '../types/context.js'; import type { ExternalToolEntry } from '../tool-runtime/registry.js'; import type { ResponseToolCallState } from '../types/backend.js'; import type { FinalToolCall } from '../tool-runtime/parser.js'; +import type { AuthoritativeStreamSource } from '../tool-runtime/stream-source.js'; import { asRecord, toErrorMessage } from '../utils/guards.js'; +import { getContinuationTenantId, getIdempotencyControl } from '../idempotency/middleware.js'; function readDataMessage(e: unknown): string { const r = asRecord(e); @@ -214,9 +221,22 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void } = ctx; app.post('/v1/responses', async (req: Request, res: Response): Promise => { + const idempotency = getIdempotencyControl(req); let responsesKeepalive: ReturnType | null = null; let responsesResClosed: Promise | null = null; const responsesAbortController = new AbortController(); + responsesResClosed = new Promise((resolve) => + res.once('close', () => { + const disconnected = !(res as unknown as { writableFinished?: boolean }).writableFinished; + if (disconnected) responsesAbortController.abort(); + resolve(disconnected); + }), + ); + const assertResponsesCanPrompt = (): void => { + if (responsesAbortController.signal.aborted || res.destroyed || res.writableEnded) { + throw new Error('Client disconnected'); + } + }; const stopResponsesKeepalive = (): void => { if (responsesKeepalive) { @@ -231,18 +251,33 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void let activePromptWithTimeout = promptWithTimeout; let activePollForAssistantResponse = pollForAssistantResponse; let fallbackToProxy = false; + let fallbackServed = false; let ownedResponsesSessionId: string | null = null; + let responsesCleanupScheduled = false; let responsesStreamState: ResponsesStreamState | null = null; - const cleanupOwnedResponsesSession = async (): Promise => { + const cleanupOwnedResponsesSessionNow = async (): Promise => { const ownedSessionId = ownedResponsesSessionId; - ownedResponsesSessionId = null; if (!ownedSessionId) return; try { await activeClient.session.delete({ path: { id: ownedSessionId } }); + if (ownedResponsesSessionId === ownedSessionId) ownedResponsesSessionId = null; } catch (error: unknown) { logDebug('Failed to cleanup owned responses session', { sessionId: ownedSessionId, error: toErrorMessage(error) }); } }; + const cleanupOwnedResponsesSession = async (): Promise => { + if (!ownedResponsesSessionId) return; + if (idempotency?.submitted) { + // Keep a possibly submitted child available for unknown-outcome + // reconciliation. Delete only after the full wire response is cached. + if (!responsesCleanupScheduled) { + responsesCleanupScheduled = true; + idempotency.onCompleted(() => cleanupOwnedResponsesSessionNow()); + } + return; + } + await cleanupOwnedResponsesSessionNow(); + }; const switchToProxyBundle = (): boolean => { if (!proxyClient) return false; activeClient = proxyClient; @@ -268,7 +303,7 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void ? previousModel : 'unknown'; const state: ResponsesStreamState = { - id: `resp_${crypto.randomUUID()}`, + id: idempotency?.publicId ?? `resp_${crypto.randomUUID()}`, createdAt: Math.floor(Date.now() / 1000), model: initialModel, tools: requestTools, @@ -305,26 +340,22 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void }; state.adoptResolvedModel = (resolvedModel: string) => { state.model = resolvedModel; - state.emitCreated(); + if (!idempotency?.reconciling) state.emitCreated(); }; + idempotency?.setPublicId(state.id); responsesStreamState = state; res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); + res.setHeader('X-Accel-Buffering', 'no'); const flushHeaders = (res as unknown as { flushHeaders?: unknown }).flushHeaders; - if (typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); - responsesKeepalive = setInterval(() => { - if (!res.destroyed && !res.writableEnded) res.write(': heartbeat\n\n'); - }, 15000); - responsesResClosed = new Promise((resolve) => - res.once('close', () => { - if (!res.writableEnded) { - responsesAbortController.abort(); - resolve(true); - } - }), - ); - return state; + if (!idempotency?.reconciling && typeof flushHeaders === 'function') (flushHeaders as () => void).call(res); + if (!idempotency?.reconciling) { + responsesKeepalive = setInterval(() => { + if (!res.destroyed && !res.writableEnded) res.write(': heartbeat\n'); + }, 15000); + } + return state; }; const getResponsesResClosed = (): Promise | null => responsesResClosed; if (proxyPool.isEngaged()) switchToProxyBundle(); @@ -354,10 +385,12 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void const previousResponseId: unknown = body['previous_response_id']; const requestOpencodeConfig: unknown = body['opencode']; + const tenantId = getContinuationTenantId(req); + const store = body['store'] !== false; const previousState = - typeof previousResponseId === 'string' && previousResponseId ? getResponseState(previousResponseId) : null; + typeof previousResponseId === 'string' && previousResponseId ? getResponseState(previousResponseId, tenantId) : null; if (stream) responsesStreamState = initializeResponsesStream(model, previousState?.model, tools, parallelToolCalls); - if (previousResponseId && !previousState) { + if (!idempotency?.reconciling && previousResponseId && (!previousState || !previousState.checkpointId)) { rejectInvalidRequest('Invalid or expired previous_response_id', 'invalid_previous_response_id'); return; } @@ -633,12 +666,13 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void } const resolvedModel = (await withTimeout( - resolveRequestedModel(model ?? previousState?.model), + resolveRequestedModel(model ?? idempotency?.model ?? previousState?.model), REQUEST_TIMEOUT_MS, 'resolve model', )) as unknown as { providerID: string; modelID: string }; const pID = String(asRecord(resolvedModel)['providerID']); const mID = String(asRecord(resolvedModel)['modelID']); + idempotency?.setModel(`${pID}/${mID}`); if (responsesStreamState) responsesStreamState.adoptResolvedModel(`${pID}/${mID}`); await ensureBackend(config); @@ -650,8 +684,25 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void // ignore } - let sessionId: string | null = previousState?.sessionId || null; - if (!sessionId) { + let sessionId: string | null = idempotency?.reconciling ? idempotency.childSessionId : null; + if (idempotency?.reconciling) { + // Unknown retries reuse the exact recorded child and never fork again. + if (!sessionId) throw new Error('Idempotency reconciliation is missing its child session'); + // This is a request-owned child, not the parent continuation session. + // Keep it until the wire result is captured unless it becomes stored state. + ownedResponsesSessionId = sessionId; + } else if (previousState) { + const forkRes = (await withTimeout( + activeClient.session.fork({ + path: { id: previousState.sessionId }, + }), + REQUEST_TIMEOUT_MS, + 'fork continuation session', + )) as unknown; + sessionId = (asRecord(asRecord(forkRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to fork OpenCode continuation session'); + ownedResponsesSessionId = sessionId; + } else { const sessionRes = (await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session')) as unknown; sessionId = (asRecord(asRecord(sessionRes)['data'])['id'] as string | undefined) ?? null; if (!sessionId) { @@ -755,8 +806,42 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void toolOverrides, requestTimeoutMs: REQUEST_TIMEOUT_MS, forbidThinkBlock: false, + signal: responsesAbortController.signal, + requireTerminalCompletion: Boolean(idempotency), + assertCanPrompt: assertResponsesCanPrompt, + ...(idempotency + ? { + beforePrompt: (forcedSessionId: string, userMessageId: string): void => { + idempotency.markSubmitted(forcedSessionId, userMessageId, `${pID}/${mID}`); + }, + } + : {}), }); let requestForcedResponsesToolCall = makeForcedResponsesToolCallRequester(); + const confirmForcedResponsesToolCall = (forcedResponse: Record): void => { + if (!idempotency) return; + const assistantMessageId = forcedResponse['assistantMessageId']; + const userMessageId = forcedResponse['userMessageId']; + const hasActiveToolState = Array.isArray(forcedResponse['toolParts']) && + (forcedResponse['toolParts'] as unknown[]).some((part) => { + const partRecord = asRecord(part); + const state = asRecord(partRecord['state']); + const status = state['status'] ?? partRecord['status']; + return partRecord['type'] === 'tool' && (status === 'pending' || status === 'running'); + }); + if ( + forcedResponse['complete'] !== true || + typeof assistantMessageId !== 'string' || + !assistantMessageId || + typeof idempotency.userMessageId !== 'string' || + !idempotency.userMessageId || + userMessageId !== idempotency.userMessageId || + hasActiveToolState + ) { + throw new Error('Idempotency forced tool result is incomplete or does not identify the exact assistant turn'); + } + idempotency.confirmResult(idempotency.userMessageId, assistantMessageId); + }; const promptParams: { path: { id: string }; body: Record } = { path: { id: sessionId as string }, @@ -769,6 +854,39 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void ...(top_p !== undefined ? { top_p } : {}), }, }; + promptParams.body.messageID = idempotency?.reconciling + ? idempotency.userMessageId + : makeId('msg_'); + if (typeof promptParams.body.messageID !== 'string' || !promptParams.body.messageID) { + throw new Error('Idempotency reconciliation is missing its user message ID'); + } + const hasConfirmedIdempotencyResult = (): boolean => Boolean( + idempotency?.resultConfirmed && + typeof idempotency.confirmedAssistantMessageId === 'string' && + idempotency.confirmedAssistantMessageId.trim(), + ); + const assertIdempotencyResultConfirmed = (): void => { + if (idempotency?.submitted && !hasConfirmedIdempotencyResult()) { + throw new Error('Idempotency result is not confirmed for the exact assistant turn'); + } + }; + const isExactTerminalPoll = (result: unknown): boolean => { + const poll = asRecord(result); + return poll['complete'] === true && + poll['partial'] !== true && + typeof poll['assistantMessageId'] === 'string' && + poll['assistantMessageId'].trim().length > 0 && + typeof idempotency?.userMessageId === 'string' && + poll['userMessageId'] === idempotency.userMessageId; + }; + const confirmExactTerminalPoll = (result: unknown): void => { + if (!idempotency?.submitted) return; + if (!isExactTerminalPoll(result)) { + throw new Error('Idempotency result is incomplete or does not identify the exact assistant turn'); + } + const poll = asRecord(result); + idempotency.confirmResult(idempotency.userMessageId as string, poll['assistantMessageId'] as string); + }; // Stage-5: strip false entries for free-tier suspects (any false gates; true-only sent, all-false omitted). const promptToolOverrides = selectPromptToolOverrides(toolOverrides, pID, mID); if (promptToolOverrides) { @@ -1011,7 +1129,7 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void // Intentional: buffer while any external tool contract is declared so tool // markup is never streamed before classification. Pure-text turns also batch. const shouldBufferExternalStream = externalToolRegistry.length > 0; - const deferVisibleOutput = shouldBufferExternalStream || hostedSearch.requested; + const deferVisibleOutput = shouldBufferExternalStream || hostedSearch.requested || Boolean(idempotency); let filterContentDelta = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); let filterReasoningDelta = createToolCallFilter({ disableTools: DISABLE_TOOLS, forceStrip: shouldBufferExternalStream, registry: externalToolRegistry }); let parseContentToolCalls = createExternalToolCallStreamParser(externalToolRegistry); @@ -1098,8 +1216,9 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void summary_index: 0, delta: filtered, }); - } else if (filtered.trim()) { + } else { const entry = ensureOutputScaffold(); + announcedContent = true; emit({ type: 'response.output_text.delta', sequence_number: nextSeq(), @@ -1121,143 +1240,426 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void appendVisibleDelta(filtered, isReasoning); if (!deferVisibleOutput) writeVisibleDelta(filtered, isReasoning); }; + // Only the suffix past the raw prefix is safe to replay. A divergent + // snapshot (raw "AC", snapshot "ABC") would duplicate visible text if + // appended whole; tool calls can still be parsed from the snapshot. const unsentSuffix = (full: unknown, raw: string): string => { const text = typeof full === 'string' ? full : ''; if (!text) return ''; if (!raw) return text; - return text.startsWith(raw) ? text.slice(raw.length) : text; + return text.startsWith(raw) ? text.slice(raw.length) : ''; + }; + // Snapshot replay is display-only: tool calls are parsed once from the + // authoritative source. Feeding snapshot text through the delta parser + // would create streamed artifacts with empty arguments and mix stale + // partial calls into the final merge. + const replaySnapshotText = (snapshotContent: unknown, snapshotReasoning: unknown): void => { + const remainingContent = unsentSuffix(snapshotContent, rawContent); + const remainingReasoning = unsentSuffix(snapshotReasoning, rawReasoning); + if (!remainingContent && !remainingReasoning) return; + // Keep the buffered raw text in sync so the later static strip of + // `rawContent`/`rawReasoning` does not erase the replayed suffix. + if (remainingContent) rawContent += remainingContent; + if (remainingReasoning) rawReasoning += remainingReasoning; + const visible = stripExternalToolCallMarkupFromJoinedText( + externalToolRegistry, + remainingReasoning, + remainingContent, + ); + if (!visible.content && !visible.reasoning) return; + appendVisibleDelta(visible.reasoning, true); + appendVisibleDelta(visible.content, false); + if (!deferVisibleOutput) { + writeVisibleDelta(visible.reasoning, true); + writeVisibleDelta(visible.content, false); + } }; - let collected: Record | null = null; - try { - if (fallbackToProxy) { - // SSE subscribe ignores custom fetch (SDK gap): prompt through the - // proxy bundle; the poll below (line ~614) fills content/reasoning. - await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); - } else { - const collectPromise = collectFromEvents( + let streamSearchToolParts: unknown[] = []; + const keepStreamToolParts = (polled: { toolParts?: unknown }): void => { + if (hostedSearch.requested && Array.isArray(polled.toolParts) && polled.toolParts.length > 0) { + streamSearchToolParts = polled.toolParts; + } + }; + let recoverySnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean } | null = null; + let recoveryPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean }> | null = null; + let currentAssistantMessageId: string | null = null; + const isContinuation = Boolean(previousState?.sessionId); + const hasRecoveryEvidence = (snapshot: { content: string; reasoning: string; toolParts?: unknown }): boolean => + Boolean( + snapshot.content || + snapshot.reasoning || + (Array.isArray(snapshot.toolParts) && snapshot.toolParts.length > 0), + ); + const pollRecoverySnapshot = (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean }> => { + if (!recoveryPromise) { + recoveryPromise = activePollForAssistantResponse( sessionId as string, REQUEST_TIMEOUT_MS, - sendResponsesDelta, - DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS, - DEFAULT_EVENT_IDLE_TIMEOUT_MS, - responsesAbortController.signal, - ); - - const safeCollect = collectPromise.catch((err: unknown) => ({ __error: err })); - activeClient.session.prompt(promptParams).catch((err: unknown) => logDebug('Responses prompt error:', toErrorMessage(err))); - const closePromise = getResponsesResClosed(); - const raced = closePromise - ? await Promise.race([safeCollect, closePromise.then(() => ({ __cancelled: true }))]) - : await safeCollect; - const racedRecord = asRecord(raced); - if (racedRecord['__cancelled']) { - stopResponsesKeepalive(); - try { - if (!res.destroyed) res.end(); - } catch { - // ignore + undefined, + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isContinuation ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), + ); + } + return recoveryPromise; + }; + let collected: Record | null = null; + let lastStreamAttemptError: unknown = null; + let promptSubmitted = false; + let promptPromise: Promise | null = null; + + for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { + collected = null; + let promptError: unknown = null; + if (attempt > 1) { + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); + const retriedSessionId = sessionId; + if (!isContinuation && !idempotency?.submitted) { + promptSubmitted = false; + if (retriedSessionId && ownedResponsesSessionId === retriedSessionId) { + await cleanupOwnedResponsesSession(); + if (ownedResponsesSessionId === retriedSessionId) { + throw new Error('Failed to cleanup retried OpenCode session'); + } + } + const retrySessionRes = (await withTimeout( + activeClient.session.create(), + REQUEST_TIMEOUT_MS, + 'create session', + )) as unknown; + sessionId = (asRecord(asRecord(retrySessionRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); + ownedResponsesSessionId = sessionId; + promptParams.path.id = sessionId; + } else { + if (promptPromise) { + await promptPromise.catch(() => {}); } - return; } - collected = raced as Record; + requestForcedResponsesToolCall = makeForcedResponsesToolCallRequester(); + content = ''; + reasoning = ''; + rawContent = ''; + rawReasoning = ''; + streamedToolCalls.length = 0; + filterContentDelta = createToolCallFilter({ + disableTools: DISABLE_TOOLS, + forceStrip: shouldBufferExternalStream, + registry: externalToolRegistry, + }); + filterReasoningDelta = createToolCallFilter({ + disableTools: DISABLE_TOOLS, + forceStrip: shouldBufferExternalStream, + registry: externalToolRegistry, + }); + parseContentToolCalls = createExternalToolCallStreamParser(externalToolRegistry); + parseReasoningToolCalls = createExternalToolCallStreamParser(externalToolRegistry); + recoverySnapshot = null; + recoveryPromise = null; + fallbackServed = false; + await sleep(computeRetryDelay(attempt - 1, lastStreamAttemptError)); } - } catch (e: unknown) { - collected = { __error: e }; - } - - const collectedR = asRecord(collected); - if (responsesAbortController.signal.aborted || collectedR['cancelled']) { - stopResponsesKeepalive(); - return; + try { + if (idempotency?.reconciling) { + const reconciled = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + responsesAbortController.signal, + idempotency.userMessageId ?? undefined, + true, + ); + if (reconciled.error != null) { + if (isExactTerminalPoll(reconciled)) { + confirmExactTerminalPoll(reconciled); + throw normalizeBackendError(reconciled.error); + } + throw new Error('Idempotency reconciliation has no confirmed assistant result'); + } + if ( + !isExactTerminalPoll(reconciled) + ) { + throw new Error('Idempotency reconciliation has no confirmed assistant result'); + } + confirmExactTerminalPoll(reconciled); + recoverySnapshot = reconciled; + currentAssistantMessageId = reconciled.assistantMessageId ?? null; + keepStreamToolParts(reconciled); + responsesStreamState?.emitCreated(); + replaySnapshotText(reconciled.content, reconciled.reasoning); + collected = { + content: reconciled.content, + reasoning: reconciled.reasoning, + toolParts: reconciled.toolParts, + assistantMessageId: reconciled.assistantMessageId, + userMessageId: reconciled.userMessageId, + complete: reconciled.complete, + finish: 'stop', + }; + } else if (fallbackToProxy) { + // SSE subscribe ignores custom fetch (SDK gap): prompt through the + // proxy bundle; the poll below fills content/reasoning. + if (!((isContinuation || idempotency?.submitted) && promptSubmitted)) { + if (isContinuation) { + promptSubmitted = true; + } + if (idempotency) { + idempotency.markSubmitted( + sessionId as string, + String(promptParams.body['messageID']), + `${pID}/${mID}`, + ); + } + promptSubmitted = true; + assertResponsesCanPrompt(); + const promptResult = await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); + const promptInfo = asRecord(asRecord(asRecord(promptResult)['data'])['info']); + const promptMessageId = promptInfo['id'] ?? promptInfo['messageID'] ?? promptInfo['messageId']; + if (typeof promptMessageId === 'string' && promptMessageId) currentAssistantMessageId = promptMessageId; + promptSubmitted = true; + } + const pres = await pollRecoverySnapshot(); + if (idempotency?.submitted) confirmExactTerminalPoll(pres); + if (pres.error != null) { + collected = { error: pres.error }; + } else if (pres.partial === true) { + collected = { error: new Error('Recovery poll returned incomplete response') }; + } else if (!hasRecoveryEvidence(pres) && externalToolChoice.mode !== 'required') { + collected = { error: new Error('Recovery poll returned empty response') }; + } else { + recoverySnapshot = pres; + currentAssistantMessageId = pres.assistantMessageId ?? currentAssistantMessageId; + keepStreamToolParts(pres); + if (pres.content || pres.reasoning) { + replaySnapshotText(pres.content, pres.reasoning); + fallbackServed = true; + } + collected = { + content: pres.content, + reasoning: pres.reasoning, + toolParts: pres.toolParts, + assistantMessageId: pres.assistantMessageId, + userMessageId: pres.userMessageId, + complete: pres.complete, + }; + } + } else { + const collectPromise = collectFromEvents( + sessionId as string, + REQUEST_TIMEOUT_MS, + sendResponsesDelta, + DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS, + DEFAULT_EVENT_IDLE_TIMEOUT_MS, + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isContinuation ? String(promptParams.body['messageID']) : undefined, + ); + + const safeCollect = collectPromise.catch((err: unknown) => ({ __error: err })); + if ((isContinuation || idempotency?.submitted) && promptSubmitted) { + // Do NOT prompt again! + } else { + if (isContinuation || idempotency) { + promptSubmitted = true; + } + if (idempotency) { + idempotency.markSubmitted( + sessionId as string, + String(promptParams.body['messageID']), + `${pID}/${mID}`, + ); + } + assertResponsesCanPrompt(); + promptPromise = activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS) + .then((promptResult: unknown) => { + const promptInfo = asRecord(asRecord(asRecord(promptResult)['data'])['info']); + const promptMessageId = promptInfo['id'] ?? promptInfo['messageID'] ?? promptInfo['messageId']; + if (typeof promptMessageId === 'string' && promptMessageId) currentAssistantMessageId = promptMessageId; + promptSubmitted = true; + }) + .catch((err: unknown) => { + promptError = err; + logDebug('Responses prompt error:', toErrorMessage(err)); + }); + } + const closePromise = getResponsesResClosed(); + const raced = closePromise + ? await Promise.race([safeCollect, closePromise.then(() => ({ __cancelled: true }))]) + : await safeCollect; + const racedRecord = asRecord(raced); + if (racedRecord['__cancelled']) { + stopResponsesKeepalive(); + try { + if (!res.destroyed) res.end(); + } catch { + // ignore + } + return; + } + collected = raced as Record; + } + } catch (e: unknown) { + collected = { __error: e }; } - if (collectedR['error'] != null) { - throw normalizeBackendError(collectedR['error']); - } - let streamSearchToolParts: unknown[] = []; - const keepStreamToolParts = (polled: { toolParts?: unknown }): void => { - if (hostedSearch.requested && Array.isArray(polled.toolParts) && polled.toolParts.length > 0) { - streamSearchToolParts = polled.toolParts; + const collectedRecord = asRecord(collected); + if (typeof collectedRecord['assistantMessageId'] === 'string') { + currentAssistantMessageId = collectedRecord['assistantMessageId'] as string; } - }; - let recoverySnapshot: { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null = null; - let recoveryPromise: Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> | null = null; - const pollRecoverySnapshot = (): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown }> => { - if (!recoveryPromise) { - recoveryPromise = activePollForAssistantResponse( - sessionId as string, - REQUEST_TIMEOUT_MS, - undefined, - responsesAbortController.signal, - ); - } - return recoveryPromise; - }; - const hasPartialOutput = (): boolean => Boolean( - rawContent || rawReasoning || streamedToolCalls.length > 0 || content || reasoning, - ); - const hasValidTerminal = Boolean( - collected && - !collectedR['__error'] && + let attemptError: unknown = promptError ?? (collectedRecord['error'] ?? collectedRecord['__error'] ?? null); + if (!attemptError && (collectedRecord['noData'] || collectedRecord['cut'] || collectedRecord['idleTimeout'])) { + try { + const recovery = await pollRecoverySnapshot(); + if (recovery.error != null) { + attemptError = recovery.error; + } else if (recovery.partial === true) { + attemptError = new Error('Recovery poll returned incomplete response'); + } else if (!hasRecoveryEvidence(recovery)) { + if (streamedToolCalls.length === 0 && externalToolChoice.mode !== 'required') { + attemptError = new Error('Recovery poll returned empty response'); + } + } else { + recoverySnapshot = recovery; + keepStreamToolParts(recovery); + replaySnapshotText(recovery.content, recovery.reasoning); + currentAssistantMessageId = recovery.assistantMessageId ?? currentAssistantMessageId; + collected = { + content: recovery.content, + reasoning: recovery.reasoning, + toolParts: recovery.toolParts, + assistantMessageId: recovery.assistantMessageId, + }; + } + } catch (pollErr: unknown) { + attemptError = pollErr; + } + } + const nothingStreamed = !rawContent && !rawReasoning && streamedToolCalls.length === 0; + if (idempotency?.submitted && attemptError) { + const recovered = await pollRecoverySnapshot(); + confirmExactTerminalPoll(recovered); + if (recovered.error != null) throw normalizeBackendError(recovered.error); + recoverySnapshot = recovered; + currentAssistantMessageId = recovered.assistantMessageId ?? currentAssistantMessageId; + keepStreamToolParts(recovered); + collected = recovered; + attemptError = null; + } else if (isContinuation && attemptError) { + throw normalizeBackendError(attemptError); + } + if (attemptError && nothingStreamed && attempt < maxAttempts && engageProxyFallback(attemptError)) { + lastStreamAttemptError = attemptError; + continue; + } + if (attemptError && nothingStreamed && attempt < maxAttempts && isTransientUpstreamError(attemptError)) { + console.warn( + `[Proxy] Transient upstream error in responses stream (attempt ${attempt}/${maxAttempts}), retrying:`, + readDataMessage(attemptError), + ); + lastStreamAttemptError = attemptError; + continue; + } + break; + } + + const collectedR = asRecord(collected); + if (responsesAbortController.signal.aborted || collectedR['cancelled']) { + stopResponsesKeepalive(); + return; + } + if (collectedR['error'] != null) { + throw normalizeBackendError(collectedR['error']); + } + const hasValidEventTerminal = Boolean( + collected && + !collectedR['__error'] && !collectedR['noData'] && !collectedR['idleTimeout'] && + !collectedR['cut'] && !collectedR['cancelled'] && - (rawContent || rawReasoning || streamedToolCalls.length > 0 || content || reasoning), - ); - const recoverSnapshot = async ( - fallbackError: unknown = null, - ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown } | null> => { + collectedR['finish'] !== 'tool' && + collectedR['finish'] !== 'tool-calls' && + (rawContent || + rawReasoning || + streamedToolCalls.length > 0 || + content || + reasoning || + collectedR['content'] || + collectedR['reasoning']), + ); + const hasValidConfirmedEventTerminal = Boolean( + hasValidEventTerminal && + collectedR['finish'] === 'stop' && + typeof collectedR['assistantMessageId'] === 'string' && + collectedR['assistantMessageId'].trim().length > 0 && + collectedR['userMessageId'] === idempotency?.userMessageId, + ); + const hasValidTerminal = idempotency?.submitted + ? hasValidConfirmedEventTerminal || isExactTerminalPoll(collectedR) + : hasValidEventTerminal; + if (idempotency?.submitted && hasValidConfirmedEventTerminal) { + idempotency.confirmResult( + idempotency.userMessageId as string, + collectedR['assistantMessageId'] as string, + ); + } else if (idempotency?.submitted && isExactTerminalPoll(collectedR)) { + confirmExactTerminalPoll(collectedR); + } + const recoverSnapshot = async ( + fallbackError: unknown = null, + ): Promise<{ content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean } | null> => { if (responsesAbortController.signal.aborted) return null; - let polled: { content: string; reasoning: string; error: unknown; toolParts?: unknown }; + let polled: { content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean }; try { polled = await pollRecoverySnapshot(); } catch (error: unknown) { if (responsesAbortController.signal.aborted) return null; - if (hasPartialOutput()) { - logDebug('Ignoring responses recovery poll failure after partial stream output', { error: toErrorMessage(error) }); - return null; - } if (fallbackError != null) throw normalizeBackendError(fallbackError); throw normalizeBackendError(error); } - if (polled.error != null) throw normalizeBackendError(polled.error); - if ( - fallbackError != null && - !polled.content && - !polled.reasoning && - !(Array.isArray(polled.toolParts) && polled.toolParts.length > 0) - ) throw normalizeBackendError(fallbackError); - recoverySnapshot = polled; + if (idempotency?.submitted) confirmExactTerminalPoll(polled); + if (polled.error != null) { + throw polled.error; + } + if (polled.partial === true) throw new Error('Recovery poll returned incomplete response'); + if (!hasRecoveryEvidence(polled)) { + if (fallbackError != null) throw normalizeBackendError(fallbackError); + return null; + } + recoverySnapshot = polled; + currentAssistantMessageId = polled.assistantMessageId ?? currentAssistantMessageId; return polled; }; const needsRecovery = Boolean( - collectedR['__error'] || collectedR['noData'] || collectedR['idleTimeout'] || !hasValidTerminal, + collectedR['__error'] || collectedR['noData'] || collectedR['idleTimeout'] || collectedR['cut'] || !hasValidTerminal, ); if (needsRecovery) { if (collectedR['__error'] && !fallbackToProxy) engageProxyFallback(collectedR['__error']); const polled = await recoverSnapshot(collectedR['__error']); if (polled) { keepStreamToolParts(polled); - const remainingReasoning = unsentSuffix(polled.reasoning, rawReasoning); - const remainingContent = unsentSuffix(polled.content, rawContent); - if (remainingReasoning) sendResponsesDelta(remainingReasoning, true); - if (remainingContent) sendResponsesDelta(remainingContent, false); + replaySnapshotText(polled.content, polled.reasoning); } } else if (collected && ((collectedR['content'] as string) || (collectedR['reasoning'] as string))) { - const remainingReasoning = unsentSuffix(collectedR['reasoning'], rawReasoning); - const remainingContent = unsentSuffix(collectedR['content'], rawContent); - if (remainingReasoning) sendResponsesDelta(remainingReasoning, true); - if (remainingContent) sendResponsesDelta(remainingContent, false); + replaySnapshotText(collectedR['content'], collectedR['reasoning']); } if (hostedSearch.requested && streamSearchToolParts.length === 0) { try { const evPoll = await activePollForAssistantResponse( - sessionId as string, - Math.min(15000, REQUEST_TIMEOUT_MS), - undefined, - responsesAbortController.signal, + sessionId as string, + Math.min(15000, REQUEST_TIMEOUT_MS), + undefined, + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isContinuation ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), ); if (Array.isArray(evPoll.toolParts)) streamSearchToolParts = evPoll.toolParts; } catch { @@ -1266,20 +1668,23 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void if (responsesAbortController.signal.aborted) return; - let polledForToolCalls: { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null = null; + let polledForToolCalls: { content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; partial?: boolean } | null = null; if (externalToolRegistry.length > 0 && !hasValidTerminal && !recoveryPromise) { const polled = await recoverSnapshot(); if (polled) { polledForToolCalls = polled; if (shouldBufferExternalStream) { - const remainingReasoning = unsentSuffix(polled.reasoning, rawReasoning); - const remainingContent = unsentSuffix(polled.content, rawContent); - if (remainingReasoning) sendResponsesDelta(remainingReasoning, true); - if (remainingContent) sendResponsesDelta(remainingContent, false); + replaySnapshotText(polled.content, polled.reasoning); } } } + if (!hasValidTerminal && !recoverySnapshot && !fallbackServed) { + if (streamedToolCalls.length === 0 && externalToolChoice.mode !== 'required') { + throw new Error('Stream ended without valid terminal'); + } + } + const flushedReasoningCalls = parseReasoningToolCalls.flush ? parseReasoningToolCalls.flush() : []; const flushedContentCalls = parseContentToolCalls.flush ? parseContentToolCalls.flush() : []; @@ -1287,59 +1692,140 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void const flushedContentText = filterContentDelta.flush ? filterContentDelta.flush() : ''; appendVisibleDelta(flushedReasoningText, true); appendVisibleDelta(flushedContentText, false); + const recoveryForParse = recoverySnapshot as { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null; + const parseJoined = (reasoningText: string, contentText: string): FinalToolCall[] => + externalToolRegistry.length > 0 + ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoningText, contentText) + : []; + const callsFromText = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? parseJoined(source.reasoning, source.content) + : []; + const callsFromRaw = (source: AuthoritativeStreamSource): FinalToolCall[] => + externalToolRegistry.length > 0 + ? mergeToolCallArtifacts( + streamedToolCalls, + flushedReasoningCalls, + flushedContentCalls, + parseJoined(source.reasoning, source.content), + ) + : []; + // One authoritative source group: snapshot when available, otherwise + // the raw delta text. Never validate both groups independently. + let authoritativeSource = resolveAuthoritativeStreamSource({ + collected: collectedR, + polledSnapshot: polledForToolCalls ?? recoveryForParse, + rawContent, + rawReasoning, + }); + const streamSource = (): string[][] => [[authoritativeSource.reasoning, authoritativeSource.content]]; + if (shouldBufferExternalStream) { const visible = stripExternalToolCallMarkupFromJoinedText( externalToolRegistry, - rawReasoning, - rawContent, + authoritativeSource.reasoning, + authoritativeSource.content, ); reasoning = visible.reasoning; content = visible.content; - } else if (!deferVisibleOutput) { + } else if (deferVisibleOutput) { + reasoning = authoritativeSource.reasoning; + content = authoritativeSource.content; + } else { + if (rawContent && authoritativeSource.content && !authoritativeSource.content.startsWith(rawContent)) { + throw new Error('Stream content diverged from authoritative snapshot'); + } + if (rawReasoning && authoritativeSource.reasoning && !authoritativeSource.reasoning.startsWith(rawReasoning)) { + throw new Error('Stream reasoning diverged from authoritative snapshot'); + } writeVisibleDelta(flushedReasoningText, true); writeVisibleDelta(flushedContentText, false); } - const recoveryForParse = recoverySnapshot as { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null; - const snapshotReasoning = polledForToolCalls?.reasoning ?? recoveryForParse?.reasoning ?? (typeof collectedR['reasoning'] === 'string' ? collectedR['reasoning'] as string : rawReasoning); - const snapshotContent = polledForToolCalls?.content ?? recoveryForParse?.content ?? (typeof collectedR['content'] === 'string' ? collectedR['content'] as string : rawContent); - - const parseJoined = (reasoningText: string, contentText: string): FinalToolCall[] => - externalToolRegistry.length > 0 - ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoningText, contentText) - : []; - const streamSource = [ - [snapshotReasoning, snapshotContent], - [rawReasoning, rawContent] - ]; - let parsedToolCalls: FinalToolCall[] = externalToolRegistry.length > 0 - ? mergeToolCallArtifacts( - streamedToolCalls, - flushedReasoningCalls, - flushedContentCalls, - parseJoined(snapshotReasoning, snapshotContent), - parseJoined(rawReasoning, rawContent), - ) - : []; - let validatedStreamedToolCalls: FinalToolCall[]; - try { - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource); - if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required') { - const forcedResponse = await requestForcedResponsesToolCall(); - if (forcedResponse) { + let parsedToolCalls: FinalToolCall[] = authoritativeSource.fromSnapshot + ? callsFromText(authoritativeSource) + : callsFromRaw(authoritativeSource); + const repairFromSnapshot = async (): Promise => { + if (externalToolRegistry.length === 0) return false; + // Reuse the first recovery snapshot (partial timeout snapshots are + // valid repair evidence when they carry a complete call). Only poll + // when no recovery attempt has been made. + let snapshot = polledForToolCalls ?? recoverySnapshot; + if (!snapshot) { + if (recoveryPromise) return false; + try { + snapshot = await recoverSnapshot(); + } catch { + return false; + } + } + if (!snapshot || snapshot.error != null) return false; + const repairedSource: AuthoritativeStreamSource = { + content: String(snapshot.content ?? ''), + reasoning: String(snapshot.reasoning ?? ''), + kind: 'polled-snapshot', + fromSnapshot: true, + }; + const snapshotCalls = parseJoined(repairedSource.reasoning, repairedSource.content); + if (snapshotCalls.length === 0) return false; + try { + assertToolCallArtifactIntegrity(snapshotCalls, externalToolRegistry, [ + [repairedSource.reasoning, repairedSource.content], + ]); + } catch { + return false; + } + polledForToolCalls = snapshot; + authoritativeSource = repairedSource; + parsedToolCalls = snapshotCalls; + if (shouldBufferExternalStream) { + const visible = stripExternalToolCallMarkupFromJoinedText( + externalToolRegistry, + authoritativeSource.reasoning, + authoritativeSource.content, + ); + reasoning = visible.reasoning; + content = visible.content; + } + return true; + }; + let validatedStreamedToolCalls: FinalToolCall[]; + try { + try { + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); + } catch (integrityError) { + if (!isMalformedExternalToolCallError(integrityError) || !(await repairFromSnapshot())) { + throw integrityError; + } + } + if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required' && !idempotency?.reconciling) { + const forcedResponse = await requestForcedResponsesToolCall(); + if (forcedResponse) { + confirmForcedResponsesToolCall(forcedResponse); + currentAssistantMessageId = + typeof forcedResponse['assistantMessageId'] === 'string' + ? (forcedResponse['assistantMessageId'] as string) + : currentAssistantMessageId; + if (forcedResponse['error'] != null) throw normalizeBackendError(forcedResponse['error']); const forcedReasoning = typeof forcedResponse['reasoning'] === 'string' ? forcedResponse['reasoning'] as string : ''; const forcedContent = typeof forcedResponse['content'] === 'string' ? forcedResponse['content'] as string : ''; parsedToolCalls = mergeToolCallArtifacts(parsedToolCalls, parseJoined(forcedReasoning, forcedContent)); - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [forcedReasoning, forcedContent]); - } - } - validatedStreamedToolCalls = finalizeStreamToolCalls( - parsedToolCalls, - externalToolRegistry, - externalToolChoice, - streamSource, - parallelToolCalls, - ) as unknown as FinalToolCall[]; + authoritativeSource = { + content: `${authoritativeSource.content}${forcedContent}`, + reasoning: `${authoritativeSource.reasoning}${forcedReasoning}`, + kind: authoritativeSource.kind, + fromSnapshot: authoritativeSource.fromSnapshot, + }; + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, streamSource()); + } + } + validatedStreamedToolCalls = finalizeStreamToolCalls( + parsedToolCalls, + externalToolRegistry, + externalToolChoice, + streamSource(), + parallelToolCalls, + ) as unknown as FinalToolCall[]; } catch (toolError) { const toolCode = (toolError as Error & { code?: string }).code; const failClosedCodes = [ @@ -1401,10 +1887,11 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void ); const safeContent = joinedSafe.content; const safeReasoning = joinedSafe.reasoning; - if (!safeContent.trim() && !safeReasoning.trim() && validatedStreamedToolCalls.length === 0) { - throw new Error('Upstream returned no assistant data'); - } - const streamEvidence = hostedSearch.requested + if (!safeContent.trim() && !safeReasoning.trim() && validatedStreamedToolCalls.length === 0) { + throw new Error('Upstream returned no assistant data'); + } + assertIdempotencyResultConfirmed(); + const streamEvidence = hostedSearch.requested ? extractSearchEvidence(streamSearchToolParts) : { queries: [] as string[], sources: [] as { url: string; title: string }[] }; const streamSearchCallItems = buildWebSearchCallItems(streamEvidence); @@ -1440,6 +1927,7 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void streamSearchCallItems.forEach((item) => emitWebSearchCallItem(item)); if (deferVisibleOutput) { + assertIdempotencyResultConfirmed(); writeVisibleDelta(safeReasoning, true); writeVisibleDelta(safeContent, false); } @@ -1480,10 +1968,24 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void output_tokens_details: { reasoning_tokens: reasoningTokens }, }, }; - emit({ type: 'response.completed', sequence_number: nextSeq(), response }); - res.write('data: [DONE]\n\n'); - storeResponseState(responseId, sessionId, `${pID}/${mID}`, buildStoredResponseToolCalls(validatedStreamedToolCalls)); - if (ownedResponsesSessionId === sessionId) ownedResponsesSessionId = null; + const stateStored = store && currentAssistantMessageId + ? storeResponseState( + responseId, + sessionId, + `${pID}/${mID}`, + buildStoredResponseToolCalls(validatedStreamedToolCalls), + currentAssistantMessageId, + tenantId, + ) + : false; + if (stateStored) { + if (ownedResponsesSessionId === sessionId) ownedResponsesSessionId = null; + } else { + await cleanupOwnedResponsesSession(); + } + assertIdempotencyResultConfirmed(); + emit({ type: 'response.completed', sequence_number: nextSeq(), response }); + res.write('data: [DONE]\n\n'); stopResponsesKeepalive(); res.end(); return; @@ -1497,57 +1999,166 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void let polledFilled = false; let lastResponsesAttemptError: unknown = null; let searchToolParts: unknown[] = []; + const isNonStreamContinuation = Boolean(previousState?.sessionId); + let promptSubmitted = false; + let currentAssistantMessageId: string | null = null; for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { - if (res.writableEnded || res.destroyed) { + if (responsesAbortController.signal.aborted || res.writableEnded || res.destroyed) { logDebug('Client gone, stop retrying', { sessionId, attempt }); return; } if (attempt > 1) { - const retriedSessionId = sessionId; - if (ownedResponsesSessionId === retriedSessionId) ownedResponsesSessionId = null; - // A `previous_response_id` parent session is shared with earlier turns; only - // a session this request created may be discarded on retry. - if (retriedSessionId && retriedSessionId !== previousState?.sessionId) { - try { - await activeClient.session.delete({ path: { id: retriedSessionId } }); - } catch (e: unknown) { - logDebug('Failed to delete retried session', { sessionId, error: toErrorMessage(e) }); + if (fallbackToProxy) switchToProxyBundle(); + await ensureActiveModel(activeClient, pID, mID).catch(() => {}); + if (!isNonStreamContinuation && !idempotency?.submitted) { + promptSubmitted = false; + const retriedSessionId = sessionId; + if (retriedSessionId && ownedResponsesSessionId === retriedSessionId) { + await cleanupOwnedResponsesSession(); + if (ownedResponsesSessionId === retriedSessionId) { + throw new Error('Failed to cleanup retried OpenCode session'); + } } + const retrySessionRes = (await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session')) as unknown; + sessionId = (asRecord(asRecord(retrySessionRes)['data'])['id'] as string | undefined) ?? null; + if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); + ownedResponsesSessionId = sessionId; + promptParams.path.id = sessionId; } - const retrySessionRes = (await withTimeout(activeClient.session.create(), REQUEST_TIMEOUT_MS, 'create session')) as unknown; - sessionId = (asRecord(asRecord(retrySessionRes)['data'])['id'] as string | undefined) ?? null; - if (!sessionId) throw new Error('Failed to create OpenCode session for retry'); - ownedResponsesSessionId = sessionId; - promptParams.path.id = sessionId; requestForcedResponsesToolCall = makeForcedResponsesToolCallRequester(); await sleep(computeRetryDelay(attempt - 1, lastResponsesAttemptError)); } - let polledResponse: { content: string; reasoning: string; error: unknown; toolParts?: unknown } | null = null; + let polledResponse: { content: string; reasoning: string; error: unknown; toolParts?: unknown; assistantMessageId?: string; userMessageId?: string; partial?: boolean; complete?: boolean } | null = null; try { - responseRes = await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); - const dataParts: unknown = asRecord(asRecord(responseRes)['data'])['parts']; - responseParts = Array.isArray(dataParts) ? (dataParts as unknown[]) : []; - promptContent = responseParts - .filter((p) => asRecord(p)['type'] === 'text') - .map((p) => String(asRecord(p)['text'] ?? '')) - .join('\n'); - promptReasoning = responseParts - .filter((p) => asRecord(p)['type'] === 'reasoning') - .map((p) => String(asRecord(p)['text'] ?? '')) - .join('\n'); - promptParsedToolCalls = - externalToolRegistry.length > 0 - ? parseExternalToolCallsFromJoinedText(externalToolRegistry, promptReasoning, promptContent) - : []; - if (promptContent || promptReasoning) break; - polledResponse = await activePollForAssistantResponse( - sessionId as string, - REQUEST_TIMEOUT_MS, - undefined, - responsesAbortController.signal, - ); - + if (idempotency?.reconciling) { + polledResponse = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + responsesAbortController.signal, + idempotency.userMessageId ?? undefined, + true, + ); + if (polledResponse.error != null) { + if (isExactTerminalPoll(polledResponse)) { + confirmExactTerminalPoll(polledResponse); + throw normalizeBackendError(polledResponse.error); + } + throw new Error('Idempotency reconciliation has no confirmed assistant result'); + } + if ( + !isExactTerminalPoll(polledResponse) + ) { + throw new Error('Idempotency reconciliation has no confirmed assistant result'); + } + confirmExactTerminalPoll(polledResponse); + currentAssistantMessageId = polledResponse.assistantMessageId ?? null; + } else if ((isNonStreamContinuation || idempotency?.submitted) && promptSubmitted) { + polledResponse = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isNonStreamContinuation ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), + ); + currentAssistantMessageId = polledResponse.assistantMessageId ?? currentAssistantMessageId; + } else { + if (isNonStreamContinuation || idempotency) { + promptSubmitted = true; + } + if (idempotency) { + idempotency.markSubmitted( + sessionId as string, + String(promptParams.body['messageID']), + `${pID}/${mID}`, + ); + } + assertResponsesCanPrompt(); + responseRes = await activePromptWithTimeout(promptParams, REQUEST_TIMEOUT_MS); + promptSubmitted = true; + if (idempotency) assertResponsesCanPrompt(); + const promptData = asRecord(asRecord(responseRes)['data']); + const promptInfo = asRecord(promptData['info']); + const promptMessageId = promptInfo['id'] ?? promptInfo['messageID'] ?? promptInfo['messageId']; + if (typeof promptMessageId === 'string' && promptMessageId) currentAssistantMessageId = promptMessageId; + const dataParts: unknown = promptData['parts']; + responseParts = Array.isArray(dataParts) ? (dataParts as unknown[]) : []; + promptContent = responseParts + .filter((p) => asRecord(p)['type'] === 'text') + .map((p) => String(asRecord(p)['text'] ?? '')) + .join('\n'); + promptReasoning = responseParts + .filter((p) => asRecord(p)['type'] === 'reasoning') + .map((p) => String(asRecord(p)['text'] ?? '')) + .join('\n'); + promptParsedToolCalls = + externalToolRegistry.length > 0 + ? parseExternalToolCallsFromJoinedText(externalToolRegistry, promptReasoning, promptContent) + : []; + if (idempotency) { + const promptTime = asRecord(promptInfo['time']); + const promptFinish = promptInfo['finish']; + const hasActiveToolPart = responseParts.some((part) => { + const partRecord = asRecord(part); + const state = asRecord(partRecord['state']); + const status = state['status'] ?? partRecord['status']; + return partRecord['type'] === 'tool' && (status === 'pending' || status === 'running'); + }); + const finishIsIntermediate = promptFinish === 'tool' || promptFinish === 'tool-calls'; + const explicitTerminal = !finishIsIntermediate && + ((promptFinish != null) || Boolean(promptTime['completed'])); + const exactTurn = + promptInfo['role'] === 'assistant' && + promptInfo['parentID'] === idempotency.userMessageId && + typeof promptMessageId === 'string' && + promptMessageId.length > 0; + if (promptInfo['error'] != null || !explicitTerminal || !exactTurn || hasActiveToolPart) { + // Prompt return values may contain a still-growing snapshot. + // Reconcile only the exact submitted user turn before + // returning text when the snapshot is incomplete or errored. + responseParts = []; + promptContent = ''; + promptReasoning = ''; + promptParsedToolCalls = []; + polledResponse = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + responsesAbortController.signal, + idempotency.userMessageId ?? undefined, + true, + ); + currentAssistantMessageId = polledResponse.assistantMessageId ?? currentAssistantMessageId; + } + } + const needsRequiredToolFollowup = Boolean( + idempotency?.submitted && + externalToolChoice.mode === 'required' && + promptParsedToolCalls.length === 0 && + (promptContent || promptReasoning), + ); + if ((promptContent || promptReasoning) && (!idempotency?.submitted || needsRequiredToolFollowup)) break; + if (!polledResponse) { + polledResponse = await activePollForAssistantResponse( + sessionId as string, + REQUEST_TIMEOUT_MS, + undefined, + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isNonStreamContinuation ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), + ); + currentAssistantMessageId = polledResponse.assistantMessageId ?? currentAssistantMessageId; + } + } } catch (loopError: unknown) { + if (isNonStreamContinuation || idempotency?.submitted) { + throw normalizeBackendError(loopError); + } if (attempt < maxAttempts && engageProxyFallback(loopError)) { lastResponsesAttemptError = loopError; continue; @@ -1562,24 +2173,42 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void } throw normalizeBackendError(loopError); } - if (polledResponse && polledResponse.error != null) { - if (attempt < maxAttempts && engageProxyFallback(polledResponse.error)) { - lastResponsesAttemptError = polledResponse.error; + if (idempotency?.submitted && polledResponse) confirmExactTerminalPoll(polledResponse); + const polledEmpty = Boolean( + polledResponse && + !polledResponse.content && + !polledResponse.reasoning && + !(Array.isArray(polledResponse.toolParts) && polledResponse.toolParts.length > 0), + ); + const polledError = + polledResponse?.error ?? + (polledResponse?.partial === true + ? new Error('Polling returned incomplete response') + : polledEmpty && externalToolChoice.mode !== 'required' + ? new Error('Polling returned empty response') + : null); + if (polledResponse && polledError != null) { + if (isNonStreamContinuation || idempotency?.submitted) { + throw normalizeBackendError(polledError); + } + if (attempt < maxAttempts && engageProxyFallback(polledError)) { + lastResponsesAttemptError = polledError; continue; } - if (attempt < maxAttempts && isTransientUpstreamError(polledResponse.error)) { + if (attempt < maxAttempts && isTransientUpstreamError(polledError)) { console.warn( `[Proxy] Transient upstream error (attempt ${attempt}/${maxAttempts}), retrying:`, - readDataMessage(polledResponse.error), + readDataMessage(polledError), ); - lastResponsesAttemptError = polledResponse.error; + lastResponsesAttemptError = polledError; continue; } - throw normalizeBackendError(polledResponse.error); + throw normalizeBackendError(polledError); } if (polledResponse) { content = polledResponse.content || content; reasoning = polledResponse.reasoning || reasoning; + currentAssistantMessageId = polledResponse.assistantMessageId ?? currentAssistantMessageId; polledFilled = true; if (Array.isArray(polledResponse.toolParts) && polledResponse.toolParts.length > 0) { searchToolParts = polledResponse.toolParts; @@ -1597,8 +2226,12 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void sessionId as string, Math.min(15000, REQUEST_TIMEOUT_MS), undefined, - responsesAbortController.signal, - ); + responsesAbortController.signal, + idempotency?.submitted + ? (idempotency.userMessageId ?? undefined) + : isNonStreamContinuation ? String(promptParams.body['messageID']) : undefined, + Boolean(idempotency), + ); if (Array.isArray(evPoll.toolParts) && evPoll.toolParts.length > 0) { searchToolParts = evPoll.toolParts; @@ -1636,25 +2269,31 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void : externalToolRegistry.length > 0 ? parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoning, content) : []; - let validatedToolCalls: FinalToolCall[] = []; - try { - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [reasoning, content]); - if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required') { - const forcedResponse = await requestForcedResponsesToolCall(); - if (forcedResponse) { - content = String(forcedResponse['content'] ?? content); - reasoning = String(forcedResponse['reasoning'] ?? reasoning); - parsedToolCalls = parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoning, content); - assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [reasoning, content]); - } - } - validatedToolCalls = finalizeStreamToolCalls( - parsedToolCalls, - externalToolRegistry, - externalToolChoice, - [reasoning, content], - parallelToolCalls, - ) as unknown as FinalToolCall[]; + let validatedToolCalls: FinalToolCall[] = []; + try { + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [reasoning, content]); + if (parsedToolCalls.length === 0 && externalToolChoice.mode === 'required' && !idempotency?.reconciling) { + const forcedResponse = await requestForcedResponsesToolCall(); + if (forcedResponse) { + confirmForcedResponsesToolCall(forcedResponse); + currentAssistantMessageId = + typeof forcedResponse['assistantMessageId'] === 'string' + ? (forcedResponse['assistantMessageId'] as string) + : currentAssistantMessageId; + if (forcedResponse['error'] != null) throw normalizeBackendError(forcedResponse['error']); + content = String(forcedResponse['content'] ?? content); + reasoning = String(forcedResponse['reasoning'] ?? reasoning); + parsedToolCalls = parseExternalToolCallsFromJoinedText(externalToolRegistry, reasoning, content); + assertToolCallArtifactIntegrity(parsedToolCalls, externalToolRegistry, [reasoning, content]); + } + } + validatedToolCalls = finalizeStreamToolCalls( + parsedToolCalls, + externalToolRegistry, + externalToolChoice, + [reasoning, content], + parallelToolCalls, + ) as unknown as FinalToolCall[]; } catch (toolError) { const toolCode = (toolError as Error & { code?: string }).code; const failClosedCodes = [ @@ -1723,7 +2362,8 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void }); const orderedOutput = orderResponsesOutputItems(output); - const responseId = `resp_${crypto.randomUUID()}`; + const responseId = idempotency?.publicId ?? `resp_${crypto.randomUUID()}`; + idempotency?.setPublicId(responseId); const createdAt = Math.floor(Date.now() / 1000); const response = { ...buildResponsesResponseEnvelope({ @@ -1747,14 +2387,49 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void }, }; - storeResponseState(responseId, sessionId, `${pID}/${mID}`, buildStoredResponseToolCalls(validatedToolCalls)); - if (ownedResponsesSessionId === sessionId) ownedResponsesSessionId = null; + const stateStored = store && currentAssistantMessageId + ? storeResponseState( + responseId, + sessionId, + `${pID}/${mID}`, + buildStoredResponseToolCalls(validatedToolCalls), + currentAssistantMessageId, + tenantId, + ) + : false; + if (stateStored) { + if (ownedResponsesSessionId === sessionId) ownedResponsesSessionId = null; + } else { + await cleanupOwnedResponsesSession(); + } - res.json(response); + assertIdempotencyResultConfirmed(); + res.json(response); return; } catch (error: unknown) { stopResponsesKeepalive(); console.error('[Proxy] Responses API Error:', toErrorMessage(error)); + const isUnconfirmedStreamingReconciliation = + idempotency?.reconciling && Boolean(asRecord((req as unknown as { body: unknown }).body)['stream']); + if ( + idempotency?.submitted && + !idempotency.resultConfirmed && + !responsesAbortController.signal.aborted && + !res.headersSent && + !res.destroyed && + !isUnconfirmedStreamingReconciliation + ) { + res.removeHeader('Content-Type'); + res.status(503).json({ + error: { + type: 'idempotency_error', + code: 'idempotency_reconciliation_pending', + message: 'The previous outcome is still uncertain; retry the same request to reconcile it', + retryable: true, + }, + }); + return; + } if (!fallbackToProxy) engageProxyFallback(error); const transformed = transformUpstreamError(error); if (responsesStreamState) { @@ -1786,12 +2461,21 @@ export function registerResponsesRoutes(app: Application, ctx: AppContext): void } catch (writeError: unknown) { logDebug('Failed to report error on open response stream', { error: toErrorMessage(writeError) }); } - res.end(); + if (!res.writableEnded && !res.destroyed) { + res.end(); + } + return; + } + if (!res.headersSent) { + res.status(transformed.statusCode).json(transformed.error); return; } - res.status(transformed.statusCode).json(transformed.error); + if (!res.writableEnded && !res.destroyed) { + res.end(); + } return; } finally { + stopResponsesKeepalive(); await cleanupOwnedResponsesSession(); } }); diff --git a/src/routes/system.ts b/src/routes/system.ts index db80deb..b8d6263 100644 --- a/src/routes/system.ts +++ b/src/routes/system.ts @@ -1,7 +1,8 @@ // P4 TS: system routes (models/health/details/metrics/404) (ported from P3 .js, behavior identical). -import type { Application, Request, Response } from 'express'; +import type { Application, NextFunction, Request, Response } from 'express'; import type { AppContext } from '../types/context.js'; import { buildEffectiveApiKeys, createApiKeyVerifier } from '../auth/keys.js'; +import { classifyHttpError } from '../errors/http.js'; function toErrorMessage(e: unknown): string { return e instanceof Error ? e.message : String((e as Record)?.['message'] ?? e); @@ -226,3 +227,14 @@ export function registerNotFoundRoute(app: Application): void { }); }); } + +export function registerErrorHandler(app: Application): void { + app.use((err: unknown, req: Request, res: Response, next: NextFunction): void => { + if (res.headersSent) { + next(err); + return; + } + const { status, body } = classifyHttpError(err, req.originalUrl || req.path); + res.status(status).json(body); + }); +} diff --git a/src/stream/collector.ts b/src/stream/collector.ts index 5b6bd67..0559a83 100644 --- a/src/stream/collector.ts +++ b/src/stream/collector.ts @@ -32,6 +32,18 @@ interface BackendEvent { properties?: unknown; } +function readInfoMessageId(info: Record): string | null { + const value = info['id'] ?? info['messageID'] ?? info['messageId']; + if (value == null || typeof value === 'boolean') return null; + return String(value); +} + +export function readParentMessageId(record: Record): string | null { + const value = record['parentID'] ?? record['parentId'] ?? record['parent_id']; + if (value == null || typeof value === 'boolean') return null; + return String(value); +} + // --- Mutex Logic with Timeout --- export async function getImageDataUri(url: unknown): Promise { const urlStr = String(url ?? ''); @@ -215,7 +227,18 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { timeoutMs: number, intervalMs: number = DEFAULT_POLL_INTERVAL_MS, signal?: AbortSignal, - ): Promise<{ content: string; reasoning: string; error: unknown; toolParts: unknown[] }> { + expectedCurrentUserMessageId?: string, + requireTerminalCompletion: boolean = false, + ): Promise<{ + content: string; + reasoning: string; + error: unknown; + toolParts: unknown[]; + assistantMessageId?: string; + userMessageId?: string; + partial?: boolean; + complete?: boolean; + }> { // Drift signal only: backend-minted ses_* IDs should match the canonical // shape (see src/session/ids.ts). Never block on mismatch — the schema // itself accepts any startsWith("ses") value. @@ -227,7 +250,14 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { // the text part only afterwards, so returning on the first non-empty snapshot // truncates the answer to the reasoning alone. Keep the partial around purely as // a timeout fallback and otherwise wait for the message to actually finish. - let lastPartial: { content: string; reasoning: string; error: null; toolParts: unknown[] } | null = null; + let lastPartial: { + content: string; + reasoning: string; + error: unknown; + toolParts: unknown[]; + assistantMessageId?: string; + userMessageId?: string; + } | null = null; while (Date.now() - startedAt < timeoutMs) { if (signal?.aborted) throw createAbortError(); const remainingMs = Math.max(1, timeoutMs - (Date.now() - startedAt)); @@ -241,6 +271,10 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const entry = asRecord(messages[i]); const info = asRecord(entry['info']); if (info['role'] !== 'assistant') continue; + if ( + expectedCurrentUserMessageId !== undefined && + (readParentMessageId(info) ?? readParentMessageId(entry)) !== expectedCurrentUserMessageId + ) continue; const { content, reasoning, toolParts } = extractFromParts(entry['parts'] ?? []); const error: unknown = info['error'] ?? null; // finish === 'tool' and 'tool-calls' mark an intermediate turn that pauses for a @@ -251,7 +285,11 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const activeToolParts = toolParts.filter((part) => isActiveToolPart(part)); const awaitingTool = finish === 'tool' || finish === 'tool-calls' || activeToolParts.length > 0; const finished = info['finish'] != null && !awaitingTool; - const done = Boolean(finished || (!awaitingTool && time['completed']) || error); + const done = Boolean( + finished || + (!awaitingTool && time['completed']) || + (!requireTerminalCompletion && error), + ); if (toolParts.length > 0) { logDebug('Polling found tool parts', { @@ -283,10 +321,29 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { reasoningLen: reasoning.length, error: error ? (asRecord(error)['name'] as unknown) : null, }); - return { content, reasoning, error, toolParts }; + const assistantMessageId = readInfoMessageId(info); + const userMessageId = (readParentMessageId(info) ?? readParentMessageId(entry)) ?? undefined; + return { + content, + reasoning, + error, + toolParts, + ...(assistantMessageId ? { assistantMessageId } : {}), + ...(userMessageId ? { userMessageId } : {}), + complete: true, + }; } - if (content || reasoning) { - lastPartial = { content, reasoning, error: null, toolParts }; + if (content || reasoning || toolParts.length > 0 || error != null) { + const assistantMessageId = readInfoMessageId(info); + const userMessageId = (readParentMessageId(info) ?? readParentMessageId(entry)) ?? undefined; + lastPartial = { + content, + reasoning, + error, + toolParts, + ...(assistantMessageId ? { assistantMessageId } : {}), + ...(userMessageId ? { userMessageId } : {}), + }; } break; } @@ -302,7 +359,7 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { contentLen: lastPartial.content.length, reasoningLen: lastPartial.reasoning.length, }); - return lastPartial; + return { ...lastPartial, partial: true }; } logDebug('Polling timeout', { sessionId, ms: Date.now() - pollStart }); throw new Error(`Request timeout after ${timeoutMs}ms`); @@ -315,6 +372,7 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { firstDeltaTimeoutMs?: number | null, idleTimeoutMs?: number | null, signal?: AbortSignal, + expectedCurrentUserMessageId?: string, ): Promise> { if (!isValidSessionId(sessionId)) logDebug('Unexpected backend session ID format', { sessionId }); if (signal?.aborted) return { content: '', reasoning: '', cancelled: true }; @@ -322,6 +380,7 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const CANCELLED = Symbol('collector-cancelled'); let removeSubscribeAbortListener = (): void => {}; const subscribePromise = Promise.resolve().then(() => client.event.subscribe({ signal: controller.signal })); + subscribePromise.catch(() => {}); const subscribeAbortPromise = new Promise((resolve) => { const onAbort = (): void => { controller.abort(); @@ -341,8 +400,15 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const eventStream = asRecord(eventStreamResult)['stream'] as AsyncIterable; let finished = false; - let content = ''; - let reasoning = ''; + let terminalFinishSeen = false; + let lastFinish: string | null = null; + let content = ''; + let reasoning = ''; + let assistantMessageId: string | null = null; + let bufferedRoleUnknownDeltaSeen = false; + let sawUserParts = false; + let sawStaleMessageParts = false; + let sawUnverifiedAssistantParts = false; let receivedDelta = false; let deltaChars = 0; let firstDeltaAt: number | null = null; @@ -350,7 +416,15 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { // tool call is active, the stream must stay open even if no text deltas arrive // (the backend is executing the tool). Resolving early here is what previously // truncated streaming responses that relied on internal tool execution. - const activeToolCallIds = new Set(); + interface ToolActivityState { + messageID: string | null; + partID: string; + active: boolean; + sequence: number; + } + const toolActivityByOwnerPart = new Map(); + let toolActivitySequence = 0; + const ambiguousToolPartIds = new Set(); // ID-less backend tool parts carry no stable `id` (older servers / some // internal tools). A cumulative counter inflates here because // `message.part.updated` fires repeatedly for the same logical tool while @@ -370,19 +444,60 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const tool = pr['tool']; return typeof tool === 'string' && tool.trim() ? `noid:${tool.trim()}` : 'noid:unknown'; }; - const hasActiveTools = (): boolean => activeToolCallIds.size > 0 || activeNoidToolKeys.size > 0; + // Ownerless updates may be provisionally correlated to a sole known + // owner, but that inference must be revoked if a later event reuses the + // part ID. Keep each owner's latest state plus the ownerless state, then + // resolve activity dynamically instead of mutating one owner's bucket. + const activePartToolCount = (): number => { + const byPart = new Map(); + for (const state of toolActivityByOwnerPart.values()) { + const states = byPart.get(state.partID) ?? []; + states.push(state); + byPart.set(state.partID, states); + } + let count = 0; + for (const [partID, states] of byPart) { + const owners = new Set(states.map((state) => state.messageID).filter((owner) => owner != null)); + const ambiguous = ambiguousToolPartIds.has(partID) || owners.size > 1; + if (ambiguous) { + const latestByOwner = new Map(); + for (const state of states) { + if (state.messageID == null) continue; + const previous = latestByOwner.get(state.messageID); + if (!previous || state.sequence > previous.sequence) latestByOwner.set(state.messageID, state); + } + count += [...latestByOwner.values()].filter((state) => state.active).length; + continue; + } + const latest = states.reduce((current, state) => state.sequence > current.sequence ? state : current); + if (latest.active) count += 1; + } + return count; + }; + const hasActiveTools = (): boolean => activePartToolCount() > 0 || activeNoidToolKeys.size > 0; // Bucket count for debug logging only (not a tool count: same-name // concurrent ID-less tools share one bucket by design). - const activeToolCount = (): number => activeToolCallIds.size + activeNoidToolKeys.size; - const trackToolState = (part: unknown): void => { + const activeToolCount = (): number => activePartToolCount() + activeNoidToolKeys.size; + const trackToolState = (part: unknown, messageID: string | null = null): void => { // Unknown statuses are ignored (neither active nor terminal) to avoid // false idle exemption. const pr = asRecord(part); if (pr['type'] !== 'tool') return; const status = toolPartStatus(pr); - const id = typeof pr['id'] === 'string' ? (pr['id'] as string) : null; + const id = typeof pr['id'] === 'string' && pr['id'] ? (pr['id'] as string) : null; + const rawMessageID = pr['messageID'] ?? pr['messageId']; + const owner = rawMessageID == null || typeof rawMessageID === 'boolean' + ? messageID + : String(rawMessageID); if (status === 'pending' || status === 'running') { - if (id) activeToolCallIds.add(id); + if (id) { + toolActivityByOwnerPart.set(JSON.stringify([owner, id]), { + messageID: owner, + partID: id, + active: true, + sequence: ++toolActivitySequence, + }); + } else { const key = noidKeyFor(pr); // A re-run of the same tool after completion is a new logical tool. @@ -390,7 +505,14 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { activeNoidToolKeys.add(key); } } else if (status === 'completed' || status === 'error') { - if (id) activeToolCallIds.delete(id); + if (id) { + toolActivityByOwnerPart.set(JSON.stringify([owner, id]), { + messageID: owner, + partID: id, + active: false, + sequence: ++toolActivitySequence, + }); + } else { const key = noidKeyFor(pr); if (activeNoidToolKeys.has(key)) { @@ -422,6 +544,7 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { const timeoutId = setTimeout(() => { if (finished) return; finished = true; + clearAllTimers(); controller.abort(); reject(new Error(`Request timeout after ${timeoutMs}ms`)); }, timeoutMs); @@ -430,19 +553,31 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { ? setTimeout(() => { if (finished || receivedDelta) return; finished = true; + clearAllTimers(); controller.abort(); logDebug('No event data received', { sessionId, ms: Date.now() - startedAt }); - resolve({ content: '', reasoning: '', noData: true }); + const result = finalizeCollected(); + // A snapshot can produce content without any delta activity; only + // report noData when reconciliation is also empty. + if (!(result['content'] as string) && !(result['reasoning'] as string)) { + result['noData'] = true; + } + resolve(result); }, firstDeltaTimeoutMs) : null; let idleTimer: ReturnType | null = null; - const onSignalAbort = (): void => { - if (finished) return; - finished = true; + + const clearAllTimers = (): void => { clearTimeout(timeoutId); if (firstDeltaTimer) clearTimeout(firstDeltaTimer); if (idleTimer) clearTimeout(idleTimer); + }; + + const onSignalAbort = (): void => { + if (finished) return; + finished = true; + clearAllTimers(); controller.abort(); resolve({ content, reasoning, cancelled: true }); }; @@ -470,23 +605,22 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { return; } finished = true; + clearAllTimers(); controller.abort(); logDebug('Event idle timeout', { sessionId, ms: Date.now() - startedAt, deltaChars, }); - resolve({ - content, - reasoning, + resolve(finalizeCollected({ idleTimeout: true, receivedDelta, - }); + })); }, idleTimeoutMs); }; - const trackToolActivity = (part: unknown): void => { - trackToolState(part); + const trackToolActivity = (part: unknown, messageID: string | null = null): void => { + trackToolState(part, messageID); // Tool activity means the session is still working; treat it as progress @@ -496,20 +630,383 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { scheduleIdleTimer(); }; - // Newer OpenCode servers stream deltas as `message.part.delta` events that - // carry only a `partID` (no `part.type`). The part type is announced by the - // preceding `message.part.updated` event, so we key partID -> type here and - // resolve each delta against it. Without this, reasoning and answer text can - // never be told apart and the answer is mis-routed (or dropped) entirely. - const partTypeById = new Map(); - const rememberPartType = (part: unknown): void => { - const pr = asRecord(part); - if (pr['id'] != null && typeof pr['type'] === 'string') { - partTypeById.set(String(pr['id']), pr['type'] as string); + // Per-part reconciliation state. A turn can contain several text/reasoning + // parts, so snapshots are tracked per part (messageID + partID, numeric IDs + // stringified) instead of as one global joined snapshot. This prevents a + // missing snapshot in part 1 from truncating part 2, and it keeps a sticky + // `message.updated info.parts` snapshot from overriding a newer full + // `message.part.updated part.text` (both write the same part state; the + // later event wins). + type StreamPartType = 'text' | 'reasoning'; + interface StreamPartState { + type: StreamPartType; + deltaText: string; + snapshotText: string; + deltasAfterSnapshot: string; + unreleasedDeltas: { delta: string; sequence: number }[]; + order: number; + messageID: string | null; + partID: string | null; + snapshotSeq: number; + deltaSeq: number; + } + const partStates = new Map(); + // A `message.part.delta` can arrive before the part update that announces + // its type. Keep those deltas (and their first-seen order) until the part + // update arrives, then replay them into the now-typed part state. + const pendingDeltaParts = new Map(); + // Message roles are keyed by the assistant/user message id. A missing id + // uses the empty bucket, which covers legacy events that also omit + // `messageID`; unknown roles are accepted so old backends keep working. + const messageRoles = new Map(); + const staleMessageIds = new Set(); + const unverifiedAssistantMessageIds = new Set(); + const assistantMessageParents = new Map(); + const assistantPartParents = new Map(); + const partMessageOwners = new Map(); + const ambiguousPartMessageOwnerIds = new Set(); + let sawAmbiguousPartOwnership = false; + // Rejected and role-classified part IDs are scoped to their owner message. + // ID-less events may use these sets conservatively, but an explicit message + // identity always takes precedence when a backend reuses a part ID. + const rejectedPartOwners = new Map>(); + const rejectedPartIdsWithoutOwner = new Set(); + const userPartOwners = new Map>(); + const userPartIdsWithoutOwner = new Set(); + const assistantPartOwners = new Map>(); + const assistantPartIdsWithoutOwner = new Set(); + const quarantinedToolParts = new Map< + string, + { messageID: string | null; partID: string | null; part: unknown; sequence: number } + >(); + // Full message snapshots can lag behind explicit per-part updates. Keep + // the most recent tool part update so replayed snapshots cannot regress + // a running tool to a stale completed state. + interface LatestToolPartUpdate { + messageID: string | null; + partID: string | null; + part: unknown; + sequence: number; + } + const latestToolPartUpdates = new Map(); + let partOrderCounter = 0; + let eventSeq = 0; + + const readPartId = (value: unknown): string | null => { + if (value == null || typeof value === 'boolean') return null; + return String(value); + }; + const rememberPartMessageOwner = (messageID: string | null, partID: string | null): void => { + if (messageID == null || partID == null || ambiguousPartMessageOwnerIds.has(partID)) return; + const previousOwner = partMessageOwners.get(partID); + if (previousOwner == null) partMessageOwners.set(partID, messageID); + else if (previousOwner !== messageID) { + partMessageOwners.delete(partID); + ambiguousPartMessageOwnerIds.add(partID); + ambiguousToolPartIds.add(partID); + sawAmbiguousPartOwnership = true; + } + }; + const isAmbiguousOwnerlessPart = (messageID: string | null, partID: string | null): boolean => + messageID == null && partID != null && ambiguousPartMessageOwnerIds.has(partID); + const resolvePartMessageId = (messageID: string | null, partID: string | null): string | null => { + if (messageID != null) return messageID; + if (partID == null || ambiguousPartMessageOwnerIds.has(partID)) return null; + return partMessageOwners.get(partID) ?? null; + }; + const readPartMessageId = (record: Record): string | null => + readPartId(record['messageID'] ?? record['messageId']); + const idPartKey = (messageID: string | null, partID: string): string => + messageID == null ? `n\u0000${partID}` : `s${messageID.length}\u0000${messageID}${partID}`; + const toolPartKey = (messageID: string | null, part: unknown): string => { + const record = asRecord(part); + const owner = readPartMessageId(record) ?? messageID; + const partID = readPartId(record['id']); + const identity = partID ?? `noid:${noidKeyFor(record)}`; + return idPartKey(owner, identity); + }; + const latestToolUpdateFor = ( + messageID: string | null, + partID: string | null, + part: unknown, + ): [string, LatestToolPartUpdate] | null => { + const exactKey = toolPartKey(messageID, partID == null ? part : { ...asRecord(part), id: partID }); + if (partID == null) { + const exact = latestToolPartUpdates.get(exactKey); + return exact ? [exactKey, exact] : null; + } + let matching = [...latestToolPartUpdates.entries()].filter(([, update]) => update.partID === partID); + if (messageID != null) { + const canUseOwnerless = messageID.length > 0 && !ambiguousPartMessageOwnerIds.has(partID); + matching = matching.filter(([, update]) => + update.messageID === messageID || (canUseOwnerless && update.messageID == null), + ); + } else { + if (ambiguousPartMessageOwnerIds.has(partID)) return null; + const unowned = matching.filter(([, update]) => update.messageID == null); + if (unowned.length > 0) matching = unowned; + else if (new Set(matching.map(([, update]) => update.messageID)).size > 1) matching = []; + } + return matching.reduce<[string, LatestToolPartUpdate] | null>( + (current, candidate) => current == null || candidate[1].sequence > current[1].sequence ? candidate : current, + null, + ); + }; + const rememberToolPartUpdate = (messageID: string | null, part: unknown): void => { + if (asRecord(part)['type'] !== 'tool') return; + const record = asRecord(part); + const partID = readPartId(record['id']); + const owner = readPartMessageId(record) ?? messageID; + // Keep ownerless updates in their own slot. A later unique owner is + // only provisional evidence; the ID may be reused by another message. + latestToolPartUpdates.set(toolPartKey(owner, part), { messageID: owner, partID, part, sequence: ++eventSeq }); + }; + const latestToolPart = (messageID: string | null, partID: string | null, part: unknown): unknown => { + return latestToolUpdateFor(messageID, partID, part)?.[1].part ?? part; + }; + const trackLatestToolActivity = (messageID: string | null, partID: string | null, part: unknown): void => { + const latest = latestToolUpdateFor(messageID, partID, part); + if (latest) { + trackToolActivity(latest[1].part, latest[1].messageID); + return; } + trackToolActivity(part, readPartMessageId(asRecord(part)) ?? messageID); }; - const applyTextDelta = (partType: unknown, delta: unknown): void => { - const deltaStr = typeof delta === 'string' ? delta : String(delta ?? ''); + const isPartOutsideExpectedTurn = (part: Record): boolean => { + const explicitParent = readParentMessageId(part); + return expectedCurrentUserMessageId !== undefined && explicitParent != null && explicitParent !== expectedCurrentUserMessageId; + }; + const ownersFor = (owners: Map>, partID: string): Set => { + let result = owners.get(partID); + if (!result) { + result = new Set(); + owners.set(partID, result); + } + return result; + }; + const rememberPartOwner = (owners: Map>, noOwner: Set, messageID: string | null, partID: string): void => { + if (messageID == null) noOwner.add(partID); + else ownersFor(owners, partID).add(messageID); + }; + const hasPartOwner = (owners: Map>, noOwner: Set, messageID: string | null, partID: string): boolean => { + if (messageID == null) return noOwner.has(partID) || (owners.get(partID)?.size ?? 0) > 0; + return owners.get(partID)?.has(messageID) ?? false; + }; + const rememberRejectedPart = (messageID: string | null, partID: string): void => + rememberPartOwner(rejectedPartOwners, rejectedPartIdsWithoutOwner, messageID, partID); + const isRejectedPart = (messageID: string | null, partID: string | null): boolean => + partID != null && hasPartOwner(rejectedPartOwners, rejectedPartIdsWithoutOwner, messageID, partID); + const rememberUserPart = (messageID: string | null, partID: string): void => + rememberPartOwner(userPartOwners, userPartIdsWithoutOwner, messageID, partID); + const isUserPartId = (messageID: string | null, partID: string | null): boolean => + partID != null && hasPartOwner(userPartOwners, userPartIdsWithoutOwner, messageID, partID); + const isAssistantPartId = (messageID: string | null, partID: string | null): boolean => + partID != null && hasPartOwner(assistantPartOwners, assistantPartIdsWithoutOwner, messageID, partID); + const isPartOwnerOrUnassigned = ( + stateMessageID: string | null, + messageID: string | null, + partID?: string | null, + ): boolean => + stateMessageID == null + ? (partID == null || !ambiguousPartMessageOwnerIds.has(partID)) && + messageID == null + : messageID != null && stateMessageID === messageID; + const partKeyFor = (messageID: string | null, partID: string | null, type: StreamPartType): string => + partID != null ? `id\u0000${idPartKey(messageID, partID)}` : `noid\u0000${idPartKey(messageID, type)}`; + const resolveMessageRole = (messageID: string | null): string | undefined => { + if (messageID == null) return undefined; + const role = messageRoles.get(messageID); + // Only user and assistant have special semantics. Preserve unrecognized + // legacy role strings as role-unknown instead of treating them as a + // known non-assistant message. + return role === 'user' || role === 'assistant' ? role : undefined; + }; + const isUserPart = (messageID: string | null): boolean => resolveMessageRole(messageID) === 'user'; + const roleForPart = (messageID: string | null, partID: string | null): string | undefined => { + if (isUserPartId(messageID, partID)) return 'user'; + if (isAssistantPartId(messageID, partID)) return 'assistant'; + return resolveMessageRole(messageID); + }; + const isAssistantPartState = (state: StreamPartState): boolean => + !(state.messageID == null && state.partID != null) && + roleForPart(state.messageID, state.partID) !== 'user' && + (expectedCurrentUserMessageId === undefined + ? roleForPart(state.messageID, state.partID) === 'assistant' + : (state.messageID != null && assistantMessageParents.get(state.messageID) === expectedCurrentUserMessageId) || + (state.partID != null && assistantPartParents.get(idPartKey(state.messageID, state.partID)) === expectedCurrentUserMessageId)); + const isKnownNonAssistantPartState = (state: StreamPartState): boolean => { + const role = roleForPart(state.messageID, state.partID); + return role === 'user'; + }; + const findPartState = (messageID: string | null, partID: string | null): StreamPartState | undefined => { + if (partID == null) return undefined; + if (messageID == null && ambiguousPartMessageOwnerIds.has(partID)) return undefined; + const direct = partStates.get(partKeyFor(messageID, partID, 'text')); + if (direct) return direct; + // Ownerless state stays quarantined until finalization confirms a + // unique owner; it must not be streamed speculatively. + if (messageID != null) return undefined; + for (const state of partStates.values()) { + if (state.partID === partID && state.messageID == null) return state; + } + return undefined; + }; + const ensurePartState = ( + messageID: string | null, + partID: string | null, + type: StreamPartType, + order?: number, + ): StreamPartState => { + let state = findPartState(messageID, partID); + // ID-less parts fall back to one stable state per type. + if (!state && partID == null) state = partStates.get(partKeyFor(messageID, null, type)); + if (!state) { + state = { + type, + deltaText: '', + snapshotText: '', + deltasAfterSnapshot: '', + unreleasedDeltas: [], + order: order ?? ++partOrderCounter, + messageID, + partID, + snapshotSeq: 0, + deltaSeq: 0, + }; + partStates.set(partKeyFor(messageID, partID, type), state); + } else { + state.type = type; + if (state.messageID == null && messageID != null) { + const ownerlessKey = partKeyFor(null, state.partID, state.type); + if (partStates.get(ownerlessKey) === state) partStates.delete(ownerlessKey); + state.messageID = messageID; + partStates.set(partKeyFor(messageID, state.partID, state.type), state); + } + } + if (order !== undefined) state.order = order; + return state; + }; + const discardMessageParts = (messageID: string, partIds: Set = new Set()): void => { + staleMessageIds.add(messageID); + unverifiedAssistantMessageIds.delete(messageID); + sawStaleMessageParts = true; + discardQuarantinedToolParts(messageID, partIds); + for (const partID of partIds) rememberRejectedPart(messageID, partID); + for (const [key, state] of partStates) { + if (state.messageID === messageID || (state.messageID == null && state.partID != null && partIds.has(state.partID))) { + if (state.partID != null) rememberRejectedPart(messageID, state.partID); + partStates.delete(key); + } + } + for (const [key, pending] of pendingDeltaParts) { + if (pending.messageID === messageID || (pending.messageID == null && pending.partID != null && partIds.has(pending.partID))) { + if (pending.partID != null) rememberRejectedPart(messageID, pending.partID); + pendingDeltaParts.delete(key); + } + } + }; + const discardUnownedParts = (partIds: Set): void => { + sawStaleMessageParts = true; + discardQuarantinedToolParts(null, partIds); + for (const partID of partIds) rememberRejectedPart(null, partID); + for (const [key, state] of partStates) { + if (state.messageID == null && state.partID != null && partIds.has(state.partID)) partStates.delete(key); + } + for (const [key, pending] of pendingDeltaParts) { + if (pending.messageID == null && pending.partID != null && partIds.has(pending.partID)) pendingDeltaParts.delete(key); + } + }; + const discardPartData = (messageID: string | null, partID: string | null): void => { + if (partID == null) return; + rememberRejectedPart(messageID, partID); + for (const [key, state] of partStates) { + if (state.partID === partID && isPartOwnerOrUnassigned(state.messageID, messageID, partID)) partStates.delete(key); + } + for (const [key, pending] of pendingDeltaParts) { + if (pending.partID === partID && isPartOwnerOrUnassigned(pending.messageID, messageID, partID)) pendingDeltaParts.delete(key); + } + for (const [key, toolPart] of quarantinedToolParts) { + if (toolPart.partID === partID && isPartOwnerOrUnassigned(toolPart.messageID, messageID, partID)) quarantinedToolParts.delete(key); + } + }; + const parentForMessage = (messageID: string | null, parentID: string | null): string | undefined => + parentID ?? (messageID == null ? undefined : assistantMessageParents.get(messageID)); + // Use enclosing metadata to correlate legacy parts, but still validate an + // explicit part-level parent independently against the expected turn. + const parentForPart = (messageID: string | null, partID: string | null): string | undefined => { + const owner = resolvePartMessageId(messageID, partID); + return (owner == null ? undefined : assistantMessageParents.get(owner)) ?? + (partID == null ? undefined : assistantPartParents.get(idPartKey(owner, partID))); + }; + const quarantinedToolKey = (messageID: string | null, partID: string | null, part: unknown): string => { + const tool = asRecord(part)['tool']; + const identity = partID ?? `noid:${typeof tool === 'string' ? tool : 'unknown'}`; + return idPartKey(messageID, identity); + }; + const quarantineToolPart = (messageID: string | null, partID: string | null, part: unknown): void => { + const latestPart = latestToolPart(messageID, partID, part); + const owner = readPartMessageId(asRecord(latestPart)) ?? messageID; + if (partID != null && owner == null && ambiguousPartMessageOwnerIds.has(partID)) return; + let key = quarantinedToolKey(owner, partID, latestPart); + let retainedOwner = owner; + if (partID != null) { + const samePart = [...quarantinedToolParts.entries()].filter(([, queued]) => queued.partID === partID); + const compatible = samePart.filter(([, queued]) => + owner == null + ? queued.messageID == null + : queued.messageID === owner, + ); + const match = owner == null && compatible.length !== 1 + ? undefined + : compatible.sort((left, right) => right[1].sequence - left[1].sequence)[0]; + if (match) { + key = match[0]; + retainedOwner = owner ?? match[1].messageID; + } + } + const latestUpdate = latestToolUpdateFor(owner, partID, latestPart)?.[1]; + const sequence = latestUpdate?.sequence ?? ++eventSeq; + const existing = quarantinedToolParts.get(key); + if (existing && existing.sequence > sequence) return; + quarantinedToolParts.set(key, { messageID: retainedOwner, partID, part: latestPart, sequence }); + }; + const discardQuarantinedToolParts = (messageID: string | null, partIds: Set): void => { + for (const [key, toolPart] of quarantinedToolParts) { + if ( + (messageID != null && toolPart.messageID === messageID) || + (toolPart.messageID == null && toolPart.partID != null && partIds.has(toolPart.partID)) + ) { + quarantinedToolParts.delete(key); + } + } + }; + const releaseQuarantinedToolParts = (messageID: string | null, partIds: Set): void => { + for (const [key, toolPart] of quarantinedToolParts) { + const matchesMessage = messageID != null && toolPart.messageID === messageID; + const matchesPart = + toolPart.partID != null && + partIds.has(toolPart.partID) && + (toolPart.messageID === messageID || ( + toolPart.messageID == null && + messageID != null && + messageID.length > 0 && + !ambiguousPartMessageOwnerIds.has(toolPart.partID) + )); + if (!matchesMessage && !matchesPart) continue; + quarantinedToolParts.delete(key); + // This function is called only after the enclosing message's exact + // parent has been confirmed as the expected continuation turn. + trackLatestToolActivity(toolPart.messageID, toolPart.partID, toolPart.part); + } + }; + const isExpectedTurn = (messageID: string | null, parentID: string | null): boolean => + expectedCurrentUserMessageId === undefined || parentForMessage(messageID, parentID) === expectedCurrentUserMessageId; + const isOutsideExpectedTurn = (messageID: string | null, parentID: string | null): boolean => { + if (expectedCurrentUserMessageId === undefined) return false; + const knownParent = parentForMessage(messageID, parentID); + return knownParent !== undefined && knownParent !== expectedCurrentUserMessageId; + }; + const noteDeltaActivity = (partType: unknown): void => { receivedDelta = true; if (firstDeltaTimer) clearTimeout(firstDeltaTimer); scheduleIdleTimer(); @@ -521,6 +1018,8 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { type: partType, }); } + }; + const emitTextDelta = (partType: StreamPartType, deltaStr: string): void => { if (partType === 'reasoning') { reasoning += deltaStr; if (onDelta) onDelta(deltaStr, true); @@ -530,6 +1029,272 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { } deltaChars += deltaStr.length; }; + const recordPartDelta = (state: StreamPartState, delta: string, afterSnapshot = true): void => { + state.deltaText += delta; + if (afterSnapshot) state.deltasAfterSnapshot += delta; + state.deltaSeq = ++eventSeq; + if (isKnownNonAssistantPartState(state)) return; + // Continuation deltas do not count as progress until their message/part + // is positively correlated to the current user turn. Legacy non- + // continuation streams retain their role-unknown compatibility path. + if (expectedCurrentUserMessageId === undefined || isAssistantPartState(state)) { + noteDeltaActivity(state.type); + } + if (isAssistantPartState(state)) { + emitTextDelta(state.type, delta); + } else { + // Keep the collector's legacy fallback text internally while + // withholding it from the route callback until its role is known. + if (state.type === 'reasoning') reasoning += delta; + else content += delta; + deltaChars += delta.length; + if (onDelta) bufferedRoleUnknownDeltaSeen = true; + // Do not expose role-unknown legacy deltas to route callbacks. If a + // later message update identifies this as assistant, release them; + // if it identifies user, purge them. Truly legacy unknown-role text + // remains available only through final reconciliation. + state.unreleasedDeltas.push({ delta, sequence: state.deltaSeq }); + } + }; + const releaseAssistantDeltas = (messageID: string | null, partIds: Set = new Set()): void => { + const ready: { state: StreamPartState; delta: string; sequence: number }[] = []; + for (const state of partStates.values()) { + const matchesMessage = messageID != null && state.messageID === messageID; + const matchesPart = state.partID != null && partIds.has(state.partID) && isPartOwnerOrUnassigned(state.messageID, messageID, state.partID); + if (!matchesMessage && !matchesPart) continue; + if (!isAssistantPartState(state) || state.unreleasedDeltas.length === 0) continue; + for (const pending of state.unreleasedDeltas) ready.push({ state, ...pending }); + state.unreleasedDeltas = []; + } + ready.sort((left, right) => left.sequence - right.sequence); + for (const item of ready) { + // These deltas were quarantined before parent metadata arrived, so + // start the first-delta/idle activity clocks only when they are valid. + noteDeltaActivity(item.state.type); + if (onDelta) onDelta(item.delta, item.state.type === 'reasoning'); + } + }; + const appendPartDelta = (state: StreamPartState, delta: string): void => { + recordPartDelta(state, delta); + }; + // Delta events may omit `messageID` while the part update carries it (or + // vice versa), so pending lookups fall back to the part id alone. Numeric + // ids are already stringified by readPartId. + const findPendingKey = (messageID: string | null, partID: string): string | null => { + if (messageID == null && ambiguousPartMessageOwnerIds.has(partID)) return null; + const exact = idPartKey(messageID, partID); + const exactPending = pendingDeltaParts.get(exact); + if (exactPending && exactPending.messageID === messageID) return exact; + if (messageID != null) return null; + for (const [key, pending] of pendingDeltaParts) { + if (pending.partID === partID && pending.messageID == null) return key; + } + return null; + }; + const flushPendingDeltas = (state: StreamPartState): void => { + if (state.partID == null) return; + const key = findPendingKey(state.messageID, state.partID); + if (key == null) return; + const pending = pendingDeltaParts.get(key); + if (!pending) return; + pendingDeltaParts.delete(key); + pending.deltas.forEach((delta) => appendPartDelta(state, delta)); + }; + const releaseAssistantPendingDeltas = (messageID: string | null, partIds: Set): void => { + for (const pending of [...pendingDeltaParts.values()]) { + const matchesMessage = messageID != null && pending.messageID === messageID; + const matchesPart = pending.partID != null && partIds.has(pending.partID) && isPartOwnerOrUnassigned(pending.messageID, messageID, pending.partID); + if (!matchesMessage && !matchesPart) continue; + const state = ensurePartState(messageID ?? pending.messageID, pending.partID, 'text', pending.order); + flushPendingDeltas(state); + } + }; + const rememberPartRole = (messageID: string | null, role: string | null): void => { + if (role == null || messageID == null) return; + messageRoles.set(messageID, role); + }; + const isStaleAssistantSnapshot = (info: Record, role: string | null): boolean => { + if (role !== 'assistant') return false; + const created = asRecord(info['time'])['created']; + return typeof created === 'number' && created < startedAt - 1000; + }; + const isStalePart = (part: Record): boolean => { + const timeRecord = asRecord(part['time']); + const created = + typeof timeRecord['created'] === 'number' + ? timeRecord['created'] + : typeof part['time'] === 'number' + ? (part['time'] as number) + : null; + return typeof created === 'number' && created < startedAt - 1000; + }; + const applyPartSnapshot = ( + part: Record, + order?: number, + ): void => { + const type = part['type']; + if (type !== 'text' && type !== 'reasoning') return; + if (isStalePart(part)) return; + const partID = readPartId(part['id']); + rememberPartMessageOwner(readPartMessageId(part), partID); + if (isAmbiguousOwnerlessPart(readPartMessageId(part), partID)) return; + const messageID = readPartMessageId(part); + if (isPartOutsideExpectedTurn(part)) { + if (failOnConfirmedPartParentMismatch(messageID, part)) return; + discardPartData(messageID, partID); + return; + } + if (messageID && staleMessageIds.has(messageID)) return; + if (isRejectedPart(messageID, partID)) return; + rememberPartMessageOwner(messageID, partID); + const partRole = roleForPart(messageID, partID); + if (partRole === 'user') return; + const pendingKey = partID != null ? findPendingKey(messageID, partID) : null; + const pendingOrder = pendingKey != null ? pendingDeltaParts.get(pendingKey)?.order : undefined; + const state = ensurePartState(messageID, partID, type, order ?? pendingOrder); + flushPendingDeltas(state); + const text = part['text']; + // Only a newer non-empty full text replaces the snapshot. An omitted or + // empty `text` (status-only update) must not erase a complete snapshot. + if (typeof text === 'string' && text) { + state.snapshotText = text; + state.snapshotSeq = ++eventSeq; + state.deltasAfterSnapshot = ''; + } + }; + const choosePartText = (state: StreamPartState): string => { + const { deltaText, snapshotText, deltasAfterSnapshot, snapshotSeq, deltaSeq } = state; + if (snapshotText === '') return deltaText; + if (deltaText === '') return snapshotText; + if (snapshotText === deltaText) return snapshotText; + if (snapshotText.startsWith(deltaText)) return snapshotText; + if (deltaText.startsWith(snapshotText)) return deltaText; + // If deltas arrived after snapshotText was set, deltas were continuing: + if (deltaSeq > snapshotSeq) { + return `${snapshotText}${deltasAfterSnapshot}`; + } + // Snapshot arrived after deltas: snapshot is the authoritative replacement: + return snapshotText; + }; + const finalizeCollected = (extra: Record = {}): Record => { + let suppressedOwnerlessPartData = false; + if (pendingDeltaParts.size > 0) { + for (const [, pending] of pendingDeltaParts) { + if ( + pending.messageID == null && + pending.partID != null && + (ambiguousPartMessageOwnerIds.has(pending.partID) || partMessageOwners.has(pending.partID)) + ) { + suppressedOwnerlessPartData = true; + continue; + } + const pendingRole = roleForPart(pending.messageID, pending.partID); + if (pendingRole === 'user') continue; + if (pending.messageID && staleMessageIds.has(pending.messageID)) continue; + const state = ensurePartState(pending.messageID, pending.partID, 'text', pending.order); + pending.deltas.forEach((delta) => recordPartDelta(state, delta)); + } + pendingDeltaParts.clear(); + } + + const states = [...partStates.values()].filter( + (state) => { + if ( + state.messageID == null && + state.partID != null && + (ambiguousPartMessageOwnerIds.has(state.partID) || partMessageOwners.has(state.partID)) + ) { + suppressedOwnerlessPartData = true; + return false; + } + return !isKnownNonAssistantPartState(state) && + !(state.messageID && staleMessageIds.has(state.messageID)) && + !(state.messageID && unverifiedAssistantMessageIds.has(state.messageID)) && + !(expectedCurrentUserMessageId !== undefined && !isAssistantPartState(state)); + }, + ); + let finalContent = content; + let finalReasoning = reasoning; + let unresolvedRoleUnknownDeltas = false; + if (states.length > 0) { + const joinedFor = (type: StreamPartType): string => + states + .filter((state) => state.type === type) + .sort((left, right) => left.order - right.order) + .map((state) => choosePartText(state)) + .join(''); + finalContent = joinedFor('text'); + finalReasoning = joinedFor('reasoning'); + unresolvedRoleUnknownDeltas = states.some( + (state) => roleForPart(state.messageID, state.partID) == null && state.unreleasedDeltas.length > 0, + ); + } else if ( + sawAmbiguousPartOwnership || + suppressedOwnerlessPartData || + sawUserParts || + sawStaleMessageParts || + sawUnverifiedAssistantParts || + [...partStates.values()].some((state) => isKnownNonAssistantPartState(state)) + ) { + // If the only observed states were user messages, clear any leaked content + finalContent = ''; + finalReasoning = ''; + } + const result: Record = { + content: finalContent, + reasoning: finalReasoning, + ...(assistantMessageId ? { assistantMessageId } : {}), + ...(assistantMessageId && expectedCurrentUserMessageId !== undefined + ? { userMessageId: expectedCurrentUserMessageId } + : {}), + ...extra, + }; + // `authoritative` means reconciliation repaired/replaced live deltas, or + // finalization is the only safe point to expose role-unknown legacy text. + // Equal text (or empty/empty) keeps routes on the raw path so streamed + // tool artifacts stay available. + if ( + finalContent !== content || + finalReasoning !== reasoning || + (Boolean(onDelta) && bufferedRoleUnknownDeltaSeen && unresolvedRoleUnknownDeltas) + ) { + result['authoritative'] = true; + } + return result; + }; + + function failCollector(error: Error): void { + terminalFinishSeen = true; + finished = true; + clearAllTimers(); + controller.abort(); + // A correlation failure invalidates any previously reconciled raw text; + // do not let it escape as a successful partial answer. + content = ''; + reasoning = ''; + partStates.clear(); + pendingDeltaParts.clear(); + quarantinedToolParts.clear(); + resolve(finalizeCollected({ error })); + } + + const failOnConfirmedPartParentMismatch = ( + messageID: string | null, + part: Record, + ): boolean => { + const explicitParent = readParentMessageId(part); + if ( + expectedCurrentUserMessageId === undefined || + explicitParent == null || + explicitParent === expectedCurrentUserMessageId || + messageID == null || + assistantMessageParents.get(messageID) !== expectedCurrentUserMessageId + ) { + return false; + } + failCollector(new Error('Continuation correlation error: assistant part parent mismatch')); + return true; + }; (async (): Promise => { try { @@ -539,74 +1304,344 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { if (finished) break; if (evType === 'session.error' && (props['sessionID'] == null || props['sessionID'] === sessionId)) { const sessionError = readSessionEventError(props); + terminalFinishSeen = true; finished = true; - clearTimeout(timeoutId); - if (firstDeltaTimer) clearTimeout(firstDeltaTimer); - if (idleTimer) clearTimeout(idleTimer); + clearAllTimers(); controller.abort(); logDebug('SSE session error', { sessionId, ms: Date.now() - startedAt }); - resolve({ content, reasoning, error: sessionError }); + resolve(finalizeCollected({ error: sessionError })); break; } - if (evType === 'message.part.updated' && asRecord(props['part'])['sessionID'] === sessionId) { - - const part = asRecord(props['part']); - rememberPartType(part); - trackToolActivity(part); - // Older OpenCode servers carried the streaming delta directly on - // message.part.updated; newer servers emit message.part.delta. - const delta: unknown = props['delta']; - if ( - typeof delta === 'string' && - delta && - (part['type'] === 'text' || part['type'] === 'reasoning') - ) applyTextDelta(part['type'], delta); + if (evType === 'message.part.updated' && asRecord(props['part'])['sessionID'] === sessionId) { + const rawPart = asRecord(props['part']); + const explicitPartMessageID = readPartMessageId(rawPart) ?? readPartMessageId(props); + const partID = readPartId(rawPart['id']); + if (isAmbiguousOwnerlessPart(explicitPartMessageID, partID)) continue; + // Even a rejected out-of-turn event is ownership evidence: + // its part ID may have been reused and must invalidate any + // ownerless fallback already observed for another message. + rememberPartMessageOwner(explicitPartMessageID, partID); + const part = explicitPartMessageID != null && readPartMessageId(rawPart) == null + ? { ...rawPart, messageID: explicitPartMessageID } + : rawPart; + const partType = part['type']; + const resolvedPartMessageID = resolvePartMessageId(explicitPartMessageID, partID); + const partMessageID = explicitPartMessageID; + if (isPartOutsideExpectedTurn(part)) { + if (failOnConfirmedPartParentMismatch(resolvedPartMessageID, part)) break; + discardPartData(partMessageID, partID); + continue; + } + const knownParent = parentForPart(partMessageID, partID); + if (expectedCurrentUserMessageId !== undefined && knownParent != null && knownParent !== expectedCurrentUserMessageId) { + if (partMessageID != null) discardMessageParts(partMessageID, new Set(partID == null ? [] : [partID])); + else if (partID != null) discardUnownedParts(new Set([partID])); + continue; + } + const missingExpectedTurn = expectedCurrentUserMessageId !== undefined && knownParent !== expectedCurrentUserMessageId; + if (missingExpectedTurn) { + sawUnverifiedAssistantParts = true; + if (partMessageID != null) unverifiedAssistantMessageIds.add(partMessageID); + } + if (isRejectedPart(partMessageID, partID) || (partMessageID != null && staleMessageIds.has(partMessageID))) continue; + if (isStalePart(part) || (partMessageID != null && staleMessageIds.has(partMessageID))) { + if (partMessageID != null && staleMessageIds.has(partMessageID)) { + discardMessageParts(partMessageID, new Set(partID == null ? [] : [partID])); + } + if (isStalePart(part) && partID != null) rememberRejectedPart(partMessageID, partID); + continue; + } + if (isUserPart(partMessageID) && partID != null) rememberUserPart(partMessageID, partID); + if (isUserPart(partMessageID) || isUserPartId(partMessageID, partID)) { + sawUserParts = true; + for (const [key, state] of partStates) { + if ( + (partID != null && state.partID === partID && isPartOwnerOrUnassigned(state.messageID, partMessageID, partID)) || + (partMessageID != null && state.messageID === partMessageID) + ) { + partStates.delete(key); + } + } + for (const [key, pending] of pendingDeltaParts) { + if ( + (partID != null && pending.partID === partID && isPartOwnerOrUnassigned(pending.messageID, partMessageID, partID)) || + (partMessageID != null && pending.messageID === partMessageID) + ) { + pendingDeltaParts.delete(key); + } + } + } + // A user prompt's text/reasoning parts must never become assistant + // answer/tool content. When the role is still unknown the part is + // stored and filtered at finalize if the role later turns out user. + const partRole = roleForPart(partMessageID, partID); + if ((partType === 'text' || partType === 'reasoning') && partRole !== 'user') { + const textUpdated = typeof part['text'] === 'string' && Boolean(part['text']); + applyPartSnapshot(part); + if (finished) break; + // Older OpenCode servers carried the streaming delta directly on + // message.part.updated; newer servers emit message.part.delta. + const delta: unknown = props['delta']; + if (typeof delta === 'string' && delta) { + if (isRejectedPart(partMessageID, partID)) continue; + const state = ensurePartState(partMessageID, partID, partType); + recordPartDelta(state, delta, !textUpdated); + } + } + if (partRole !== 'user') { + rememberToolPartUpdate(partMessageID, part); + if (expectedCurrentUserMessageId === undefined || knownParent === expectedCurrentUserMessageId) { + trackLatestToolActivity(partMessageID, partID, part); + } else if (partType === 'tool') { + quarantineToolPart(partMessageID, partID, part); + } + } continue; } if (evType === 'message.part.delta' && props['sessionID'] === sessionId) { - const partID: unknown = props['partID']; + const partID = readPartId(props['partID']); const delta: unknown = props['delta']; const field: unknown = props['field']; // Text and reasoning deltas both stream through field === 'text'. // Tool-input deltas surface via message.part.updated tool state. - if (typeof delta === 'string' && field === 'text') { - const partType = typeof partID === 'string' ? partTypeById.get(partID) : undefined; - if (partType === 'reasoning' || partType === 'text') { - applyTextDelta(partType, delta); + if (typeof delta === 'string' && delta && field === 'text' && partID != null) { + const explicitPartMessageID = readPartMessageId(props); + if (isAmbiguousOwnerlessPart(explicitPartMessageID, partID)) continue; + rememberPartMessageOwner(explicitPartMessageID, partID); + const resolvedPartMessageID = resolvePartMessageId(explicitPartMessageID, partID); + const partMessageID = explicitPartMessageID; + const partRole = roleForPart(partMessageID, partID); + if (isPartOutsideExpectedTurn(props)) { + if (failOnConfirmedPartParentMismatch(resolvedPartMessageID, props)) break; + discardPartData(partMessageID, partID); + continue; + } + const knownParent = parentForPart(partMessageID, partID); + if (expectedCurrentUserMessageId !== undefined && knownParent != null && knownParent !== expectedCurrentUserMessageId) { + if (partMessageID != null) discardMessageParts(partMessageID, new Set([partID])); + else discardUnownedParts(new Set([partID])); + continue; + } + const missingExpectedTurn = expectedCurrentUserMessageId !== undefined && knownParent !== expectedCurrentUserMessageId; + if (isRejectedPart(partMessageID, partID) || partRole === 'user' || (partMessageID != null && staleMessageIds.has(partMessageID))) { + continue; + } + if (missingExpectedTurn) { + sawUnverifiedAssistantParts = true; + if (partMessageID != null) unverifiedAssistantMessageIds.add(partMessageID); + } + const state = findPartState(partMessageID, partID); + if (state) { + appendPartDelta(state, delta); + } else { + // Type not announced yet; keep the delta (numeric IDs already + // stringified) until the part update arrives. + if (!missingExpectedTurn) noteDeltaActivity(undefined); + const existingKey = findPendingKey(partMessageID, partID); + if (existingKey != null) { + pendingDeltaParts.get(existingKey)?.deltas.push(delta); + } else { + pendingDeltaParts.set(idPartKey(partMessageID, partID), { + deltas: [delta], + order: ++partOrderCounter, + messageID: partMessageID, + partID, + }); + } } } continue; } if (evType === 'message.updated' && asRecord(props['info'])['sessionID'] === sessionId) { - const info = asRecord(props['info']); - const finish: unknown = info['finish']; + const info = asRecord(props['info']); + const finish: unknown = info['finish']; + const messageID = readInfoMessageId(info); + const role = typeof info['role'] === 'string' ? (info['role'] as string) : null; + const parentID = readParentMessageId(info); + const previouslyConfirmedParent = messageID == null ? undefined : assistantMessageParents.get(messageID); + if ( + role !== 'user' && + expectedCurrentUserMessageId !== undefined && + previouslyConfirmedParent === expectedCurrentUserMessageId && + parentID != null && + parentID !== expectedCurrentUserMessageId + ) { + failCollector(new Error('Continuation correlation error: assistant message parent changed')); + break; + } + if (messageID != null && parentID != null) { + assistantMessageParents.set(messageID, parentID); + } + if (role !== 'user' && expectedCurrentUserMessageId !== undefined && isOutsideExpectedTurn(messageID, parentID)) { + const partIds = new Set( + (Array.isArray(info['parts']) ? info['parts'] as unknown[] : []) + .map((rawPart) => readPartId(asRecord(rawPart)['id'])) + .filter((id): id is string => id != null), + ); + if (messageID != null) { + for (const partID of partIds) rememberPartMessageOwner(messageID, partID); + discardMessageParts(messageID, partIds); + } else discardUnownedParts(partIds); + continue; + } + if (role !== 'user' && !isExpectedTurn(messageID, parentID)) { + sawUnverifiedAssistantParts = true; + if (messageID != null) unverifiedAssistantMessageIds.add(messageID); + rememberPartRole(messageID, role); + // Missing turn metadata is not evidence that this message + // belongs to the continuation. Keep its data quarantined, but + // do not let it emit or finish the expected turn. + if (Array.isArray(info['parts'])) { + (info['parts'] as unknown[]).forEach((rawPart) => { + const part = asRecord(rawPart); + const partID = readPartId(part['id']); + rememberPartMessageOwner(messageID, partID); + if (partID != null && role != null) { + rememberPartOwner(assistantPartOwners, assistantPartIdsWithoutOwner, messageID, partID); + } + const messagePart = messageID != null && readPartMessageId(part) == null + ? { ...part, messageID } + : part; + if (part['type'] === 'tool') { + quarantineToolPart(messageID, readPartId(part['id']), messagePart); + } + applyPartSnapshot(messagePart); + }); + } + continue; + } + if (isStaleAssistantSnapshot(info, role)) { + logDebug('Ignoring stale assistant message snapshot', { + sessionId, + messageID, + created: asRecord(info['time'])['created'], + }); + if (messageID != null) { + const partIds = new Set( + (Array.isArray(info['parts']) ? info['parts'] as unknown[] : []) + .map((rawPart) => readPartId(asRecord(rawPart)['id'])) + .filter((id): id is string => id != null), + ); + for (const partID of partIds) rememberPartMessageOwner(messageID, partID); + discardMessageParts(messageID, partIds); + } + continue; + } + if (role !== 'user' && messageID != null) { + unverifiedAssistantMessageIds.delete(messageID); + } + rememberPartRole(messageID, role); + const classifiedPartIds = new Set(); + if (role != null) { + for (const rawPart of Array.isArray(info['parts']) ? info['parts'] as unknown[] : []) { + const rawPartRecord = asRecord(rawPart); + const partID = readPartId(asRecord(rawPart)['id']); + if (partID == null) continue; + rememberPartMessageOwner(messageID, partID); + if (isPartOutsideExpectedTurn(rawPartRecord)) { + if (failOnConfirmedPartParentMismatch(messageID, rawPartRecord)) break; + discardPartData(messageID, partID); + continue; + } + classifiedPartIds.add(partID); + if (role === 'user') rememberUserPart(messageID, partID); + else { + rememberPartOwner(assistantPartOwners, assistantPartIdsWithoutOwner, messageID, partID); + if (parentID != null) assistantPartParents.set(idPartKey(messageID, partID), parentID); + } + } + } + if (finished) break; + // User prompt snapshots are never assistant output: skipping the + // whole update also prevents a user `finish` from ending the turn. + if (role === 'user') { + sawUserParts = true; + if (messageID != null) { + for (const [key, st] of partStates) { + if (st.messageID === messageID) { + if (st.partID != null) rememberUserPart(messageID, st.partID); + partStates.delete(key); + } + } + } + for (const partID of classifiedPartIds) { + for (const [key, st] of partStates) { + if (st.partID === partID && isPartOwnerOrUnassigned(st.messageID, messageID, partID)) partStates.delete(key); + } + } + for (const [key, pending] of pendingDeltaParts) { + const matchesMessage = messageID != null && pending.messageID === messageID; + const matchesPart = pending.partID != null && classifiedPartIds.has(pending.partID) && isPartOwnerOrUnassigned(pending.messageID, messageID, pending.partID); + if (matchesMessage || matchesPart) { + if (pending.partID != null) rememberUserPart(messageID, pending.partID); + pendingDeltaParts.delete(key); + } + } + continue; + } + if (role === 'assistant' || expectedCurrentUserMessageId !== undefined) { + releaseAssistantDeltas(messageID, classifiedPartIds); + releaseAssistantPendingDeltas(messageID, classifiedPartIds); + } + if (typeof finish === 'string') lastFinish = finish; // An aborted or failed message never produces another delta. Without // this, the collector waits out the whole first-delta window before // polling rediscovers the same error. const infoError: unknown = info['error']; if (infoError != null) { + terminalFinishSeen = true; finished = true; - clearTimeout(timeoutId); - if (firstDeltaTimer) clearTimeout(firstDeltaTimer); - if (idleTimer) clearTimeout(idleTimer); + clearAllTimers(); logDebug('SSE upstream message error', { sessionId, ms: Date.now() - startedAt, error: (asRecord(infoError)['name'] as unknown) || 'UnknownError', }); - resolve({ content, reasoning, error: infoError }); + resolve(finalizeCollected({ error: infoError })); break; } - // Reconcile active tool calls from the full message snapshot so we - // detect pending tools even when only message.updated fires. - if (Array.isArray(info['parts'])) { - for (const part of info['parts'] as unknown[]) { - rememberPartType(part); - trackToolState(part); - - - } + // Reconcile active tool calls and per-part text snapshots from the + // full message snapshot so we detect pending tools even when only + // message.updated fires. + if (Array.isArray(info['parts'])) { + const snapshotParts = info['parts'] as unknown[]; + snapshotParts.forEach((rawPart, index) => { + const part = asRecord(rawPart); + // `info.parts` is ordered; the array index is authoritative for + // joining multiple text/reasoning parts. + const messagePart = { + ...part, + ...(messageID != null && readPartMessageId(part) == null ? { messageID } : {}), + ...(parentID != null && readParentMessageId(part) == null ? { parentID } : {}), + }; + rememberPartMessageOwner(messageID, readPartId(part['id'])); + if (isPartOutsideExpectedTurn(messagePart)) { + if (failOnConfirmedPartParentMismatch(messageID, messagePart)) return; + discardPartData(messageID, readPartId(part['id'])); + return; + } + applyPartSnapshot(messagePart, index); + if (part['type'] === 'tool') { + const partID = readPartId(part['id']); + const toolMessageID = messageID; + if (expectedCurrentUserMessageId === undefined) { + trackLatestToolActivity(toolMessageID, partID, messagePart); + } else { + quarantineToolPart(toolMessageID, partID, messagePart); + } + } + }); + if (finished) break; + if (role === 'assistant' || expectedCurrentUserMessageId !== undefined) { + releaseAssistantDeltas(messageID, classifiedPartIds); + releaseAssistantPendingDeltas(messageID, classifiedPartIds); + } + // New parts observed after this snapshot must sort after it. + partOrderCounter = Math.max(partOrderCounter, snapshotParts.length); + } + if (expectedCurrentUserMessageId !== undefined) { + releaseQuarantinedToolParts(messageID, classifiedPartIds); } if (finish === 'tool' || finish === 'tool-calls') { // Assistant turn ended pending a tool call; keep waiting for the result. @@ -625,28 +1660,59 @@ export function createCollector(deps: CollectorDeps): CollectorHandle { }); continue; } + terminalFinishSeen = true; + if ( + (role === 'assistant' || resolveMessageRole(messageID) === 'assistant') && + messageID != null && + (expectedCurrentUserMessageId === undefined || parentForMessage(messageID, parentID) === expectedCurrentUserMessageId) + ) { + assistantMessageId = messageID; + } if (!finished) { finished = true; - clearTimeout(timeoutId); - if (firstDeltaTimer) clearTimeout(firstDeltaTimer); - if (idleTimer) clearTimeout(idleTimer); + clearAllTimers(); logDebug('SSE completed', { sessionId, ms: Date.now() - startedAt, deltaChars, }); - resolve(content || reasoning ? { content, reasoning } : { content, reasoning, noData: true }); + const result = finalizeCollected({ finish: 'stop' }); + if (!(result['content'] as string) && !(result['reasoning'] as string)) { + result['noData'] = true; + } + resolve(result); } break; } } } + if (!finished) { + finished = true; + clearAllTimers(); + logDebug('SSE eventStream ended naturally', { + sessionId, + ms: Date.now() - startedAt, + terminalFinishSeen, + deltaChars, + }); + if (!terminalFinishSeen) { + const result = finalizeCollected({ cut: true, ...(lastFinish ? { finish: lastFinish } : {}) }); + if (!(result['content'] as string) && !(result['reasoning'] as string)) { + result['noData'] = true; + } + resolve(result); + } else { + const result = finalizeCollected({ ...(lastFinish ? { finish: lastFinish } : {}) }); + if (!(result['content'] as string) && !(result['reasoning'] as string)) { + result['noData'] = true; + } + resolve(result); + } + } } catch (e: unknown) { if (!finished) { finished = true; - clearTimeout(timeoutId); - if (firstDeltaTimer) clearTimeout(firstDeltaTimer); - if (idleTimer) clearTimeout(idleTimer); + clearAllTimers(); reject(e); } } diff --git a/src/tool-runtime/router.ts b/src/tool-runtime/router.ts index 12c6e8d..f03cf83 100644 --- a/src/tool-runtime/router.ts +++ b/src/tool-runtime/router.ts @@ -244,7 +244,7 @@ export function buildExternalToolsReminder(registry: unknown, toolChoice: unknow const exampleName = normalizedChoice.requiredTool || first.namespacedName; return [ 'REMINDER: External tools are called by emitting markup, not through any native tool API.', - `To call one, your entire reply must be ONLY {"name":"${exampleName}","arguments":{...}}`, + `To call one, your entire reply must be ONLY {"name":"${exampleName}","arguments":{}}`, 'with no prose, no markdown and no block. Otherwise answer normally.', `Available names: ${list.map((tool) => tool.namespacedName).join(', ')}` ].join('\n'); diff --git a/src/tool-runtime/stream-source.ts b/src/tool-runtime/stream-source.ts new file mode 100644 index 0000000..0e6eb20 --- /dev/null +++ b/src/tool-runtime/stream-source.ts @@ -0,0 +1,93 @@ +// Authoritative stream source selection for external tool finalization. +// +// Streaming routes receive the same assistant text twice: incremental deltas +// (`message.part.delta` / legacy `message.part.updated` delta) and one or more +// full snapshots (a recovery poll, `message.updated` info.parts, or the final +// `part.text`). The two sources are not independent artifacts: a lossy delta +// stream can be missing a chunk while the snapshot is complete. Validating both +// groups independently therefore fails a turn that the snapshot could repair. +// +// Routes must pick exactly one group for parse/validate/finalize: the snapshot +// when one is available, otherwise the raw delta text. This helper centralizes +// that choice so all three public protocols behave identically. +import { asRecord } from '../utils/guards.js'; + +export interface StreamSnapshotLike { + content?: unknown; + reasoning?: unknown; + error?: unknown; + /** Timeout fallback: the backend message has not finished yet. */ + partial?: unknown; + /** Recovery poll observed a terminal message. */ + complete?: unknown; +} + +export type AuthoritativeStreamSourceKind = 'polled-snapshot' | 'collector-snapshot' | 'raw'; + +export interface AuthoritativeStreamSource { + content: string; + reasoning: string; + kind: AuthoritativeStreamSourceKind; + fromSnapshot: boolean; +} + +function asText(value: unknown): string { + if (typeof value === 'string') return value; + if (value == null) return ''; + return String(value); +} + +export function resolveAuthoritativeStreamSource(input: { + collected: unknown; + polledSnapshot?: StreamSnapshotLike | null; + rawContent: unknown; + rawReasoning: unknown; +}): AuthoritativeStreamSource { + const polled = input.polledSnapshot; + if (polled && polled.error == null && polled.partial !== true) { + const content = asText(polled.content); + const reasoning = asText(polled.reasoning); + // Empty snapshots (including terminal ones) are not evidence: preferring + // them would erase calls that were already streamed. A timeout partial is + // likewise not final and must not replace the raw/collector group. + if (content || reasoning) { + return { + content, + reasoning, + kind: 'polled-snapshot', + fromSnapshot: true, + }; + } + } + + const collected = asRecord(input.collected); + const rawContent = asText(input.rawContent); + const rawReasoning = asText(input.rawReasoning); + if (collected['authoritative'] === true && collected['error'] == null) { + const content = asText(collected['content']); + const reasoning = asText(collected['reasoning']); + // The collector only marks a real repair, but keep the non-empty guard here + // too: an empty authoritative repair must not hide streamed tool calls. Text + // equal to the raw group is not a repair either, so keep that path as well. + if ((content || reasoning) && (content !== rawContent || reasoning !== rawReasoning)) { + return { + content, + reasoning, + kind: 'collector-snapshot', + fromSnapshot: true, + }; + } + } + + return { + content: rawContent, + reasoning: rawReasoning, + kind: 'raw', + fromSnapshot: false, + }; +} + +/** True when the parser rejected partial/malformed tool markup. */ +export function isMalformedExternalToolCallError(error: unknown): boolean { + return asRecord(error)['code'] === 'malformed_external_tool_call'; +} diff --git a/src/types/backend.ts b/src/types/backend.ts index 542543b..806d7ff 100644 --- a/src/types/backend.ts +++ b/src/types/backend.ts @@ -15,6 +15,10 @@ export interface ResponseToolCallState { /** Stateful Responses continuation entry (previous_response_id). */ export interface SessionInfo { sessionId: string; + /** Hashed API-key tenant owner; absent only on legacy in-memory entries. */ + tenantId?: string; + /** Completed assistant message ID used as the fork checkpoint for continuations. */ + checkpointId?: string; model: string; expiresAt: number; toolCalls?: ResponseToolCallState[]; diff --git a/src/types/client.ts b/src/types/client.ts index 0de5383..c9085db 100644 --- a/src/types/client.ts +++ b/src/types/client.ts @@ -2,6 +2,7 @@ export interface SessionNamespace { create: (args?: unknown) => Promise; + fork: (args: { path: { id: string } }) => Promise; prompt: (args: unknown) => Promise; messages: (args: unknown) => Promise; delete: (args: unknown) => Promise; diff --git a/src/types/context.ts b/src/types/context.ts index dd519fb..bfc8a6c 100644 --- a/src/types/context.ts +++ b/src/types/context.ts @@ -48,6 +48,12 @@ export interface ForcedToolCallRequesterOptions { toolOverrides: Record | null; requestTimeoutMs: number; forbidThinkBlock?: boolean; + signal?: AbortSignal; + requireTerminalCompletion?: boolean; + /** Checked immediately before the forced prompt is submitted. */ + assertCanPrompt?: () => void; + /** Called immediately before the forced prompt can reach the backend. */ + beforePrompt?: (sessionId: string, userMessageId: string) => void; } export interface CollectorHandle { @@ -57,15 +63,27 @@ export interface CollectorHandle { sessionId: string, timeoutMs: number, intervalMs?: number, - signal?: AbortSignal - ) => Promise<{ content: string; reasoning: string; error: unknown; toolParts: unknown[] }>; + signal?: AbortSignal, + expectedCurrentUserMessageId?: string, + requireTerminalCompletion?: boolean, + ) => Promise<{ + content: string; + reasoning: string; + error: unknown; + toolParts: unknown[]; + assistantMessageId?: string; + userMessageId?: string; + partial?: boolean; + complete?: boolean; + }>; collectFromEvents: ( sessionId: string, timeoutMs: number, onDelta?: ((delta: string, isReasoning?: boolean) => void) | null, firstDeltaTimeoutMs?: number | null, idleTimeoutMs?: number | null, - signal?: AbortSignal + signal?: AbortSignal, + expectedCurrentUserMessageId?: string, ) => Promise>; } @@ -106,13 +124,14 @@ export interface AppContext { maxRetries: number; maxAttempts: number; getProvidersList: () => Promise; + invalidateProvidersCache: () => void; buildModelsList: (providersList: unknown) => ModelInfo[]; normalizeModelID: (modelID: unknown) => unknown; resolveRequestedModel: (requestedModel: unknown) => Promise; logDebug: (...args: unknown[]) => void; responseState: Map; - getResponseState: (responseId: unknown) => ResponseStateEntry | null; - storeResponseState: (responseId: unknown, sessionId: unknown, model: unknown, toolCalls?: unknown) => void; + getResponseState: (responseId: unknown, tenantId: string) => ResponseStateEntry | null; + storeResponseState: (responseId: unknown, sessionId: unknown, model: unknown, toolCalls: unknown, checkpointId: unknown, tenantId: string) => boolean; sweepResponseState: () => Promise; TOOL_MODE: ToolModeSet; TOOL_GUARD_MESSAGE: string; @@ -171,14 +190,19 @@ export interface AppContext { getDiscoveryLastError: () => string | null; /** Wired N×N translator registry (Phase 1 wiring; optional so manual test ctx still compiles). */ translators?: TranslatorRegistry; + /** Cleanup app-owned listeners, subscriptions, and timers. */ + dispose?: () => void; } export interface CreateAppResult { app: import('express').Application; client: ProxyClient; + ctx?: AppContext; } export interface StartProxyResult { server: import('node:http').Server; killBackend: () => void; + /** Idempotently dispose app-owned listeners; also called when server closes. */ + dispose: () => void; } diff --git a/tests/app.test.js b/tests/app.test.js index 1dc3886..b7e4031 100644 --- a/tests/app.test.js +++ b/tests/app.test.js @@ -39,6 +39,7 @@ const sdkMockDefaults = { sessionCreate: async () => ({ data: { id: 'test-session-id' } }), + sessionFork: async () => ({ data: { id: `test-fork-session-${++forkSequence}` } }), sessionPrompt: async (args) => { const promptText = args.body.prompt || args.body.parts?.map(part => part.text || '').join(' ') || ''; const parts = [{ type: 'text', text: 'Mock response' }]; @@ -47,41 +48,60 @@ const sdkMockDefaults = { parts.unshift({ type: 'reasoning', text: 'Thinking process...' }); } - return { data: { parts } }; + return { + data: { + info: { + id: `assistant-${args.path.id}`, + role: 'assistant', + parentID: args.body.messageID, + finish: 'stop' + }, + parts + } + }; }, - sessionMessages: async () => ([ + sessionMessages: async (args) => ([ { - info: { role: 'assistant', finish: 'stop' }, + info: { + id: `assistant-${args?.path?.id || 'test-session-id'}`, + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + finish: 'stop' + }, parts: [ { type: 'text', text: 'Mock response' } ] } ]), sessionDelete: async () => ({}), - eventSubscribe: async () => { - const sessionId = 'test-session-id'; - const mockEvents = [ - { type: 'message.part.updated', properties: { part: { type: 'reasoning', sessionID: sessionId }, delta: 'Thinking...' } }, - { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: sessionId }, delta: 'Mock' } }, - { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: sessionId }, delta: ' response' } }, - { type: 'message.updated', properties: { info: { sessionID: sessionId, finish: 'stop' } } } - ]; - - return { - stream: (async function* () { - for (const event of mockEvents) { - yield event; - } - })() - }; - } + eventSubscribe: async () => ({ + stream: (async function* () { + const prompt = sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]; + const sessionId = prompt?.path?.id || 'test-session-id'; + const mockEvents = [ + { type: 'message.part.updated', properties: { part: { type: 'reasoning', sessionID: sessionId }, delta: 'Thinking...' } }, + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: sessionId }, delta: 'Mock' } }, + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: sessionId }, delta: ' response' } }, + { type: 'message.updated', properties: { info: { + id: `assistant-${sessionId}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionId, + finish: 'stop' + } } } + ]; + for (const event of mockEvents) yield event; + })() + }) }; +let forkSequence = 0; const sdkMocks = { configProviders: jest.fn(sdkMockDefaults.configProviders), configUpdate: jest.fn(sdkMockDefaults.configUpdate), toolIds: jest.fn(sdkMockDefaults.toolIds), sessionCreate: jest.fn(sdkMockDefaults.sessionCreate), + sessionFork: jest.fn(sdkMockDefaults.sessionFork), sessionPrompt: jest.fn(sdkMockDefaults.sessionPrompt), sessionMessages: jest.fn(sdkMockDefaults.sessionMessages), sessionDelete: jest.fn(sdkMockDefaults.sessionDelete), @@ -139,6 +159,7 @@ jest.unstable_mockModule('@opencode-ai/sdk', () => ({ }, session: { create: sdkMocks.sessionCreate, + fork: sdkMocks.sessionFork, prompt: sdkMocks.sessionPrompt, messages: sdkMocks.sessionMessages, delete: sdkMocks.sessionDelete @@ -220,6 +241,17 @@ describe('Phase 1A tool policy and exact tool choice', () => { expect(preflightExternalToolChoice('required', [registry[1]])).toEqual({ ok: true, normalized: { mode: 'auto', requiredTool: null } }); expect(preflightExternalToolChoice({ type: 'any' }, [])).toEqual({ ok: true, normalized: { mode: 'auto', requiredTool: null } }); }); + + test('external tool reminder shows a valid JSON example', () => { + const registry = buildExternalToolRegistry([{ type: 'function', name: 'read' }]); + const { reminder } = buildToolExposure(registry); + + expect(reminder).not.toContain('{...}'); + const match = reminder.match(/(\{[\s\S]*?\})<\/function_calls>/); + expect(match).not.toBeNull(); + const example = JSON.parse(match[1]); + expect(example).toEqual({ name: 'external__read', arguments: {} }); + }); }); describe('Proxy OpenAI API', () => { @@ -1203,7 +1235,7 @@ describe('Proxy OpenAI API', () => { const mockEvents = [ { type: 'message.part.updated', - properties: { part: { type: 'text', sessionID: sessionId }, delta: 'I need to search. ' } + properties: { part: { id: 'p1', type: 'text', sessionID: sessionId }, delta: 'I need to search. ' } }, { type: 'message.updated', @@ -1212,15 +1244,15 @@ describe('Proxy OpenAI API', () => { sessionID: sessionId, finish: 'stop', parts: [ - { type: 'text', text: 'I need to search. ' }, - { type: 'tool', id: 'call_internal_1', tool: 'web_fetch', state: { status: 'pending', input: { url: 'https://example.com/weather' } } } + { id: 'p1', type: 'text', text: 'I need to search. ' }, + { id: 'call_internal_1', type: 'tool', tool: 'web_fetch', state: { status: 'pending', input: { url: 'https://example.com/weather' } } } ] } } }, { type: 'message.part.updated', - properties: { part: { type: 'text', sessionID: sessionId }, delta: 'The weather is 14C.' } + properties: { part: { id: 'p2', type: 'text', sessionID: sessionId }, delta: 'The weather is 14C.' } }, { type: 'message.updated', @@ -1229,8 +1261,8 @@ describe('Proxy OpenAI API', () => { sessionID: sessionId, finish: 'stop', parts: [ - { type: 'tool', id: 'call_internal_1', tool: 'web_fetch', state: { status: 'completed', input: { url: 'https://example.com/weather' }, output: '14C' } }, - { type: 'text', text: 'The weather is 14C.' } + { id: 'call_internal_1', type: 'tool', tool: 'web_fetch', state: { status: 'completed', input: { url: 'https://example.com/weather' }, output: '14C' } }, + { id: 'p2', type: 'text', text: 'The weather is 14C.' } ] } } @@ -1357,9 +1389,9 @@ describe('Proxy OpenAI API', () => { expect(streamed).toContain('Done: 42.'); }); - test('polling fallback returns the last partial snapshot when the timeout is reached', async () => { - // If the message never reports completion, the partial text still beats throwing a - // timeout error and losing everything the model produced. + test('polling fallback fails closed when timeout is reached with partial response', async () => { + // When the message never reports completion and poll reaches timeout, recovery poll + // returns partial: true and must fail closed rather than emitting success. sdkMocks.sessionMessages.mockImplementation(async () => ([{ info: { role: 'assistant' }, parts: [{ type: 'text', text: 'Partial answer that never completes' }] @@ -1387,7 +1419,9 @@ describe('Proxy OpenAI API', () => { }); expect(res.statusCode).toEqual(200); - expect(res.text).toContain('Partial answer that never completes'); + expect(res.text).toContain('Recovery poll returned incomplete response'); + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); }); test('streaming surfaces an upstream message error without waiting out the first-delta window', async () => { @@ -1525,6 +1559,15 @@ describe('Proxy OpenAI API', () => { }); test('POST /v1/chat/completions emits tool_calls finish_reason in streaming for external tools', async () => { + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [ + { type: 'reasoning', text: 'Thinking...' }, + { type: 'text', text: '[{"id":"call_weather_stream_1","name":"external__weather_lookup","arguments":{"city":"Tokyo","unit":"celsius"}}]' } + ] + } + ]); sdkMocks.eventSubscribe.mockResolvedValueOnce({ stream: (async function* () { const sessionId = 'test-session-id'; @@ -2530,13 +2573,14 @@ describe('Proxy OpenAI API', () => { }); test('non-streaming retries once on transient upstream CreditsError and recovers', async () => { - // Same mislabeled billing failure, polling path: first snapshot reports the - // CreditsError with no usable content, the retry succeeds. + // Same mislabeled billing failure, polling path: the first snapshot is + // explicitly terminal despite having no usable content, then the retry succeeds. sdkMocks.sessionMessages.mockReset(); sdkMocks.sessionMessages .mockResolvedValueOnce([{ info: { role: 'assistant', + finish: 'stop', error: { name: 'CreditsError', data: { message: '401: Insufficient balance', responseHeaders: { 'retry-after-ms': '10' } } } }, parts: [] @@ -2601,6 +2645,38 @@ describe('Proxy Responses API previous_response_id', () => { .flatMap((event) => event.choices || []) .flatMap((choice) => (choice.delta?.content ? [choice.delta.content] : [])); }; + const toolCallsFor = (path, text) => { + const frames = readSseData(text); + if (path === '/v1/responses') { + const argumentText = frames + .filter((event) => event.type === 'response.function_call_arguments.delta') + .map((event) => event.delta) + .join(''); + return frames + .filter((event) => event.type === 'response.output_item.added' && event.item?.type === 'function_call') + .map((event) => ({ + ...event.item, + arguments: argumentText || event.item.arguments, + })); + } + if (path === '/v1/messages') { + const inputJson = frames + .filter((event) => event.type === 'content_block_delta' && event.delta?.type === 'input_json_delta') + .map((event) => event.delta.partial_json) + .join(''); + return frames + .filter((event) => event.type === 'content_block_start' && event.content_block?.type === 'tool_use') + .map((event) => ({ ...event.content_block, arguments: inputJson })); + } + return frames + .flatMap((event) => event.choices || []) + .flatMap((choice) => choice.delta?.tool_calls || []) + .map((call) => ({ + id: call.id, + name: call.function?.name, + arguments: call.function?.arguments, + })); + }; test('non-stream parsing joins reasoning and content before extraction', async () => { sdkMocks.sessionMessages.mockResolvedValueOnce([{ @@ -2940,7 +3016,89 @@ describe('Proxy Responses API previous_response_id', () => { ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] - ])('%s preserves partial output when recovery polling fails', async (_label, path, payload) => { + ])('%s repairs a partial raw delta from the final part snapshot', async (label, path, payload) => { + const callId = `call_partial_${label}`; + const complete = `{"id":"${callId}","name":"read","arguments":{"path":"a.txt"}}`; + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { + type: 'message.part.updated', + properties: { + part: { id: 'part-text', type: 'text', sessionID: 'test-session-id' }, + delta: complete.slice(0, 25) + } + }, + { + type: 'message.part.updated', + properties: { + part: { id: 'part-text', type: 'text', text: complete, sessionID: 'test-session-id' } + } + }, + { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockRejectedValue(new Error('poll must not run')); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(res.text).toContain(callId); + expect(visibleTextFor(path, res.text).join('')).toBe(''); + expect(sdkMocks.sessionMessages).not.toHaveBeenCalled(); + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s repairs malformed raw markup from a recovery snapshot', async (label, path, payload) => { + const callId = `call_snapshot_${label}`; + const malformed = 'prefix {"name":"read" arguments suffix'; + const valid = `{"id":"${callId}","name":"read","arguments":{"path":"a.txt"}}`; + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: malformed } }, + { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockResolvedValueOnce([{ + info: { role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: valid }] + }]); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(res.text).toContain(callId); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s fails closed when malformed raw markup cannot be repaired', async (_label, path, payload) => { + const malformed = 'prefix {"name":"read" arguments suffix'; + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: malformed } }, + { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockRejectedValue(new Error('snapshot poll failed')); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(res.text).toContain('malformed_external_tool_call'); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s fails closed when recovery polling fails', async (label, path, payload) => { sdkMocks.eventSubscribe.mockImplementationOnce(eventStreamThenThrow([ { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: 'partial answer' } } ])); @@ -2951,8 +3109,103 @@ describe('Proxy Responses API previous_response_id', () => { .set('anthropic-version', '2023-06-01') .send(payload); expect(res.statusCode).toBe(200); - expect(visibleTextFor(path, res.text).join('')).toBe('partial answer'); - expect(res.text).not.toContain('poll failed'); + expect(res.text).toMatch(/event stream cut|poll failed/); + if (label === 'chat') { + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('finish_reason'); + } else if (label === 'messages') { + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('message_delta'); + } else if (label === 'responses') { + expect(res.text).not.toContain('response.completed'); + } + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s emits snapshot-only assistant content without polling', async (_label, path, payload) => { + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { type: 'message.part.updated', properties: { part: { id: 'part-snapshot-only', messageID: 'msg-assistant', type: 'text', sessionID: 'test-session-id' } } }, + { type: 'message.part.updated', properties: { part: { id: 'part-snapshot-only', messageID: 'msg-assistant', type: 'text', text: 'Snapshot-only answer', sessionID: 'test-session-id' } } }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', sessionID: 'test-session-id', role: 'assistant', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockRejectedValue(new Error('poll must not run')); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toBe('Snapshot-only answer'); + expect(sdkMocks.sessionMessages).not.toHaveBeenCalled(); + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s keeps divergent raw text visible while parsing a snapshot tool call', async (label, path, payload) => { + const callId = `call_divergent_${label}`; + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { type: 'message.part.updated', properties: { part: { id: 'part-content', messageID: 'msg-assistant', type: 'text', sessionID: 'test-session-id' }, delta: 'AC' } }, + { type: 'message.part.updated', properties: { part: { id: 'part-reasoning', messageID: 'msg-assistant', type: 'reasoning', text: `{"id":"${callId}","name":"read","arguments":{"path":"a.txt"}}`, sessionID: 'test-session-id' } } }, + { type: 'message.part.updated', properties: { part: { id: 'part-content', messageID: 'msg-assistant', type: 'text', text: 'ABC', sessionID: 'test-session-id' } } }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', sessionID: 'test-session-id', role: 'assistant', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockRejectedValue(new Error('poll must not run')); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toBe('ABC'); + expect(res.text).not.toContain('ACABC'); + const calls = toolCallsFor(path, res.text); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ name: 'read' }); + expect(JSON.stringify(calls[0].arguments ?? calls[0].input)).toContain('a.txt'); + expect(sdkMocks.sessionMessages).not.toHaveBeenCalled(); + }); + + test.each([ + ['chat', '/v1/chat/completions', { ...chatStreamPayload, tools: [readTool] }], + ['messages', '/v1/messages', { ...messagesStreamPayload, tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Read a file', stream: true, tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }] + ])('%s never emits or parses a user prompt part', async (_label, path, payload) => { + const prompt = 'SECRET PROMPT {"id":"call_prompt","name":"read","arguments":{"path":"secret.txt"}}'; + sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ + { + type: 'message.updated', + properties: { + info: { + id: 'msg-user', + sessionID: 'test-session-id', + role: 'user', + parts: [{ id: 'part-user', messageID: 'msg-user', type: 'text', text: prompt }] + } + } + }, + { type: 'message.part.updated', properties: { part: { id: 'part-user', messageID: 'msg-user', type: 'text', text: prompt, sessionID: 'test-session-id' } } }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', sessionID: 'test-session-id', role: 'assistant' } } }, + { type: 'message.part.updated', properties: { part: { id: 'part-assistant', messageID: 'msg-assistant', type: 'text', sessionID: 'test-session-id' }, delta: 'Visible answer' } }, + { type: 'message.part.updated', properties: { part: { id: 'part-assistant', messageID: 'msg-assistant', type: 'text', text: 'Visible answer', sessionID: 'test-session-id' } } }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', sessionID: 'test-session-id', role: 'assistant', finish: 'stop' } } } + ])); + sdkMocks.sessionMessages.mockRejectedValue(new Error('poll must not run')); + const res = await request(app) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toBe('Visible answer'); + expect(res.text).not.toContain('SECRET PROMPT'); + expect(res.text).not.toContain('call_prompt'); + expect(toolCallsFor(path, res.text)).toEqual([]); + expect(sdkMocks.sessionMessages).not.toHaveBeenCalled(); }); test('chat stream disconnect aborts collection and deletes its session', async () => { @@ -3180,10 +3433,24 @@ describe('Proxy Responses API previous_response_id', () => { }); test('responses stream transfers a newly created owned session to response state', async () => { - sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.sessionMessages.mockImplementation(async (args) => [{ + info: { + id: `assistant-${args?.path?.id}`, + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + finish: 'stop' + }, + parts: [] + }]); sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: 'done' } }, - { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + { type: 'message.updated', properties: { info: { + id: 'assistant-test-session-id', + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + sessionID: 'test-session-id', + finish: 'stop' + } } } ])); sdkMocks.sessionCreate.mockClear(); sdkMocks.sessionDelete.mockClear(); @@ -3211,11 +3478,22 @@ describe('Proxy Responses API previous_response_id', () => { .send({ input: 'And a follow-up', previous_response_id: completed.response.id }); expect(followUp.statusCode).toBe(200); expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionFork).toHaveBeenCalledWith({ + path: { id: 'test-session-id' } + }); expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); }); test('responses non-stream transfers a newly created owned session to response state', async () => { - sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.sessionMessages.mockImplementation(async (args) => [{ + info: { + id: `assistant-${args?.path?.id}`, + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + finish: 'stop' + }, + parts: [] + }]); sdkMocks.sessionCreate.mockClear(); sdkMocks.sessionDelete.mockClear(); const res = await request(app) @@ -3237,6 +3515,9 @@ describe('Proxy Responses API previous_response_id', () => { .send({ input: 'And a follow-up', previous_response_id: res.body.id }); expect(followUp.statusCode).toBe(200); expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionFork).toHaveBeenCalledWith({ + path: { id: 'test-session-id' } + }); expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); }); @@ -3246,11 +3527,19 @@ describe('Proxy Responses API previous_response_id', () => { .set('Authorization', 'Bearer test-key') .send({ model: 'opencode/kimi-k2.5', input: 'seed' }); expect(seed.statusCode).toBe(200); + const continuationSessionId = 'responses-error-fork'; + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: continuationSessionId } }); sdkMocks.sessionDelete.mockClear(); - sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.sessionMessages.mockImplementation(async () => { + const latestPrompt = sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]; + return [{ + info: { role: 'assistant', parentID: latestPrompt?.body?.messageID, finish: 'stop' }, + parts: [], + }]; + }); sdkMocks.eventSubscribe.mockImplementationOnce(eventStream([ - { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: '{"name":"read" arguments' } }, - { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: continuationSessionId }, delta: '{"name":"read" arguments' } }, + { type: 'message.updated', properties: { info: { sessionID: continuationSessionId, finish: 'stop' } } } ])); const res = await request(app) .post('/v1/responses') @@ -3262,7 +3551,8 @@ describe('Proxy Responses API previous_response_id', () => { tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] }); expect(res.text).toContain('response.failed'); - expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: continuationSessionId } }); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: 'test-session-id' } }); }); test.each([ @@ -3756,7 +4046,8 @@ describe('Proxy Responses API previous_response_id', () => { test('Phase 2A mid-stream failure finalizes the announced item and reports it as partial failed output', async () => { setToolStream([ - { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'test-session-id' }, delta: 'Partial answer' } }, + { type: 'message.updated', properties: { info: { id: 'assistant-partial', role: 'assistant', sessionID: 'test-session-id' } } }, + { type: 'message.part.updated', properties: { part: { id: 'partial-part', messageID: 'assistant-partial', type: 'text', sessionID: 'test-session-id' }, delta: 'Partial answer' } }, { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', error: { name: 'MessageAbortedError', data: { message: 'upstream aborted mid-turn' } } } } }, ]); const res = await request(app) @@ -4142,7 +4433,10 @@ describe('Proxy Responses API previous_response_id', () => { .set('Authorization', 'Bearer test-key') .send(payload); const replyWithToolCall = (callId, namespacedName, args) => sdkMocks.sessionPrompt.mockResolvedValueOnce({ - data: { parts: [{ type: 'text', text: `{"id":"${callId}","name":"${namespacedName}","arguments":${args}}` }] } + data: { + info: { id: 'assistant-test-session-id', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: `{"id":"${callId}","name":"${namespacedName}","arguments":${args}}` }] + } }); const streamedCallReply = (callId, namespacedName, args) => sdkMocks.eventSubscribe.mockImplementationOnce(eventStreamLite([ { @@ -4152,7 +4446,7 @@ describe('Proxy Responses API previous_response_id', () => { delta: `{"id":"${callId}","name":"${namespacedName}","arguments":${args}}` } }, - { type: 'message.updated', properties: { info: { sessionID: 'test-session-id', finish: 'stop' } } } + { type: 'message.updated', properties: { info: { id: 'assistant-test-session-id', role: 'assistant', sessionID: 'test-session-id', finish: 'stop' } } } ])); describe('declaration expansion', () => { @@ -5358,6 +5652,38 @@ describe('Proxy Responses API previous_response_id', () => { const first = await postResponses({ model: 'opencode/kimi-k2.5', input: 'Read a.txt', tools: [liteRead] }); expect(first.statusCode).toBe(200); + let continuationMessageID; + sdkMocks.sessionPrompt.mockImplementationOnce(async (params) => { + continuationMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.delta', + properties: { + sessionID: 'test-session-id', + messageID: 'assistant-stream-continuation', + partID: 'continuation-answer-part', + field: 'text', + delta: 'Continuation completed', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stream-continuation', + role: 'assistant', + parentID: continuationMessageID, + finish: 'stop', + sessionID: 'test-session-id', + }, + }, + }; + })(), + })); + const res = await postResponses({ previous_response_id: first.body.id, stream: true, @@ -5394,7 +5720,10 @@ describe('Proxy Responses API previous_response_id', () => { `{"id":"call_cap_${index}","name":"external__read","arguments":{"path":"a-${index}.txt"}}` ).join(','); sdkMocks.sessionPrompt.mockResolvedValueOnce({ - data: { parts: [{ type: 'text', text: `[${calls}]` }] } + data: { + info: { id: 'assistant-test-session-id', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: `[${calls}]` }] + } }); const first = await postResponses({ model: 'opencode/kimi-k2.5', @@ -5439,7 +5768,10 @@ describe('Proxy Responses API previous_response_id', () => { `{"id":"call_cap_${index}","name":"external__read","arguments":{"path":"a-${index}.txt"}}` ).join(','); sdkMocks.sessionPrompt.mockResolvedValueOnce({ - data: { parts: [{ type: 'text', text: `[${calls}]` }] } + data: { + info: { id: 'assistant-test-session-id', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: `[${calls}]` }] + } }); const res = await request(debugApp) .post('/v1/responses') @@ -5546,4 +5878,4 @@ describe('Proxy Responses API previous_response_id', () => { expect(res.body.error.message).toContain('previous_response_id'); }); }); -}); \ No newline at end of file +}); diff --git a/tests/backend-error-surface.test.js b/tests/backend-error-surface.test.js index 6e7d854..45119cb 100644 --- a/tests/backend-error-surface.test.js +++ b/tests/backend-error-surface.test.js @@ -203,7 +203,8 @@ describe('POST /v1/responses backend plain-object error', () => { .set('Authorization', 'Bearer test-key') .send({ model: 'opencode/kimi-k2.5', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }); expect(res.statusCode).toBe(402); - expect(res.body.error?.type).toBe('insufficient_quota'); + expect(res.body.error?.type).toBe('api_error'); + expect(res.body.error?.code).toBe('insufficient_quota'); expect(JSON.stringify(res.body)).toContain('Insufficient balance'); expect(JSON.stringify(res.body)).not.toContain('"Object"'); }); @@ -232,10 +233,10 @@ describe('POST /v1/responses backend plain-object error', () => { responseHeaders: { 'retry-after-ms': '10' } } }; - const buildApp = (retries) => createApp({ + const buildApp = (retries, requestTimeoutMs = 5000) => createApp({ PORT: 10000, API_KEY: 'test-key', OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', - REQUEST_TIMEOUT_MS: 5000, DISABLE_TOOLS: true, DEBUG: false, + REQUEST_TIMEOUT_MS: requestTimeoutMs, DISABLE_TOOLS: true, DEBUG: false, RETRY_MAX_RETRIES: retries }).app; @@ -266,6 +267,27 @@ describe('POST /v1/responses backend plain-object error', () => { expect(sdkMocks.sessionCreate.mock.calls.length).toBe(1); }); + test('chat retries a transient assistant error without finish immediately when no key is supplied', async () => { + sdkMocks.sessionMessages + .mockResolvedValueOnce([ + { info: { role: 'assistant', error: fastTransientError }, parts: [] } + ]) + .mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'recovered without waiting' }] } + ]); + const chatApp = buildApp(1, 500); + const startedAt = Date.now(); + const res = await request(chatApp).post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', messages: [{ role: 'user', content: 'hi' }] }); + + expect(res.statusCode).toBe(200); + expect(JSON.stringify(res.body)).toContain('recovered without waiting'); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionCreate.mock.calls.length).toBe(2); + expect(Date.now() - startedAt).toBeLessThan(400); + }); + test('RETRY_MAX_RETRIES clamps to upstream ceiling of 5', async () => { sdkMocks.sessionMessages.mockImplementation(async () => ([ { info: { role: 'assistant', finish: 'stop', error: fastTransientError }, parts: [] } diff --git a/tests/continuation-fork.test.js b/tests/continuation-fork.test.js new file mode 100644 index 0000000..c83f826 --- /dev/null +++ b/tests/continuation-fork.test.js @@ -0,0 +1,318 @@ +import request from 'supertest'; +import { jest } from '@jest/globals'; + +const simulatedTranscripts = new Map(); +const simulatedForkContexts = new Map(); +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { providers: [{ id: 'opencode', models: { 'kimi-k2.5': { name: 'Kimi' } } }] }, + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ({ data: ['websearch', 'webfetch'] })), + sessionCreate: jest.fn(async () => ({ data: { id: 'parent-session' } })), + sessionFork: jest.fn(async ({ path }) => { + const childId = `child-${sdkMocks.sessionFork.mock.calls.length}`; + const copiedContext = [...(simulatedTranscripts.get(path.id) ?? [])]; + simulatedTranscripts.set(childId, copiedContext); + simulatedForkContexts.set(childId, structuredClone(copiedContext)); + return { data: { id: childId } }; + }), + sessionPrompt: jest.fn(async ({ path, body }) => { + const transcript = simulatedTranscripts.get(path.id) ?? []; + for (const part of body.parts ?? []) { + if (typeof part.text === 'string') transcript.push({ role: 'user', text: part.text }); + } + transcript.push({ role: 'assistant', text: 'answer' }); + simulatedTranscripts.set(path.id, transcript); + return { + data: { + info: { + id: `assistant-${path.id === 'parent-session' ? 'parent' : path.id}-checkpoint`, + role: 'assistant', + parentID: body.messageID, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'answer' }], + }, + }; + }), + sessionMessages: jest.fn(async ({ path }) => { + const prompt = sdkMocks.sessionPrompt.mock.calls + .map(([args]) => args) + .findLast((args) => args?.path?.id === path.id); + return [{ + info: { + id: path.id === 'parent-session' ? 'assistant-parent-checkpoint' : `assistant-${path.id}-checkpoint`, + role: 'assistant', + parentID: prompt?.body?.messageID, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'answer' }], + }]; + }), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ stream: (async function* () {})() })), +}; + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { + create: sdkMocks.sessionCreate, + fork: sdkMocks.sessionFork, + prompt: sdkMocks.sessionPrompt, + messages: sdkMocks.sessionMessages, + delete: sdkMocks.sessionDelete, + }, + event: { subscribe: sdkMocks.eventSubscribe }, + })), +})); + +jest.unstable_mockModule('http', () => ({ + default: { + get: jest.fn((_url, options, callback) => { + const cb = typeof options === 'function' ? options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn(), setTimeout: jest.fn() }; + }), + }, +})); + +jest.unstable_mockModule('https', () => ({ + default: { + get: jest.fn((_url, options, callback) => { + const cb = typeof options === 'function' ? options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn() }; + }), + }, +})); + +const { createApp } = await import('../src/proxy.js'); + +function createTestApp(overrides = {}) { + return createApp({ + PORT: 10000, + API_KEY: '', + API_KEYS: [], + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 1000, + DISABLE_TOOLS: true, + DEBUG: false, + ...overrides, + }); +} + +describe('checkpoint-based continuation forks', () => { + beforeEach(() => { + jest.clearAllMocks(); + simulatedTranscripts.clear(); + simulatedForkContexts.clear(); + }); + + test('Responses forks every continuation from the same checkpoint and stores each child checkpoint', async () => { + const { app, ctx } = createTestApp(); + const seed = await request(app).post('/v1/responses').send({ model: 'opencode/kimi-k2.5', input: 'seed' }); + expect(seed.statusCode).toBe(200); + + const first = await request(app).post('/v1/responses').send({ + input: 'follow-up one', + previous_response_id: seed.body.id, + }); + const second = await request(app).post('/v1/responses').send({ + input: 'follow-up two', + previous_response_id: seed.body.id, + }); + + expect(first.statusCode).toBe(200); + expect(second.statusCode).toBe(200); + expect(sdkMocks.sessionFork.mock.calls).toEqual([ + [{ path: { id: 'parent-session' } }], + [{ path: { id: 'parent-session' } }], + ]); + expect(sdkMocks.sessionPrompt.mock.calls.slice(1).map(([args]) => args.path.id)).toEqual(['child-1', 'child-2']); + expect(sdkMocks.sessionPrompt.mock.calls.slice(1).map(([args]) => args.body.parts.map((part) => part.text))).toEqual([ + ['follow-up one'], + ['follow-up two'], + ]); + expect(simulatedForkContexts.get('child-1')).toEqual([ + { role: 'user', text: 'seed' }, + { role: 'assistant', text: 'answer' }, + ]); + expect(simulatedForkContexts.get('child-2')).toEqual(simulatedForkContexts.get('child-1')); + expect(ctx.responseState.get(seed.body.id).checkpointId).toBe('assistant-parent-checkpoint'); + expect(ctx.responseState.get(first.body.id).checkpointId).toBe('assistant-child-1-checkpoint'); + expect(ctx.responseState.get(second.body.id).checkpointId).toBe('assistant-child-2-checkpoint'); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + }); + + test('Interactions forks every continuation from the same checkpoint and stores each child checkpoint', async () => { + const { app, ctx } = createTestApp(); + const seed = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'seed' }); + expect(seed.statusCode).toBe(200); + + const first = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'follow-up one', + previous_interaction_id: seed.body.id, + }); + const second = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'follow-up two', + previous_interaction_id: seed.body.id, + }); + + expect(first.statusCode).toBe(200); + expect(second.statusCode).toBe(200); + expect(sdkMocks.sessionFork.mock.calls).toEqual([ + [{ path: { id: 'parent-session' } }], + [{ path: { id: 'parent-session' } }], + ]); + expect(sdkMocks.sessionPrompt.mock.calls.slice(1).map(([args]) => args.path.id)).toEqual(['child-1', 'child-2']); + expect(sdkMocks.sessionPrompt.mock.calls.slice(1).map(([args]) => args.body.parts.map((part) => part.text))).toEqual([ + ['follow-up one'], + ['follow-up two'], + ]); + expect(simulatedForkContexts.get('child-1')).toEqual([ + { role: 'user', text: 'seed' }, + { role: 'assistant', text: 'answer' }, + ]); + expect(simulatedForkContexts.get('child-2')).toEqual(simulatedForkContexts.get('child-1')); + expect(ctx.responseState.get(seed.body.id).checkpointId).toBe('assistant-parent-checkpoint'); + expect(ctx.responseState.get(first.body.id).checkpointId).toBe('assistant-child-1-checkpoint'); + expect(ctx.responseState.get(second.body.id).checkpointId).toBe('assistant-child-2-checkpoint'); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + }); + + test.each([ + ['/v1/responses', { model: 'opencode/kimi-k2.5', input: 'follow-up', previous_response_id: 'no-checkpoint' }], + ['/v1beta/interactions', { model: 'opencode/kimi-k2.5', input: 'follow-up', previous_interaction_id: 'no-checkpoint' }], + ])('%s fails closed when the stored checkpoint is missing', async (path, body) => { + const { app, ctx } = createTestApp(); + ctx.responseState.set('no-checkpoint', { + sessionId: 'parent-session', + model: 'opencode/kimi-k2.5', + expiresAt: Date.now() + 60_000, + }); + + const response = await request(app).post(path).send(body); + + expect(response.statusCode).toBe(400); + expect(sdkMocks.sessionFork).not.toHaveBeenCalled(); + expect(sdkMocks.sessionPrompt).not.toHaveBeenCalled(); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + }); + + test.each([ + ['/v1/responses', { model: 'opencode/kimi-k2.5', input: 'no assistant id' }], + ['/v1beta/interactions', { model: 'opencode/kimi-k2.5', input: 'no assistant id' }], + ])('%s does not retain a session when the completed turn has no assistant ID', async (path, body) => { + const { app, ctx } = createTestApp(); + if (path === '/v1/responses') { + sdkMocks.sessionPrompt.mockResolvedValueOnce({ + data: { parts: [{ type: 'text', text: 'answer without message metadata' }] }, + }); + } else { + sdkMocks.sessionMessages.mockResolvedValueOnce([{ + info: { role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'answer without message metadata' }], + }]); + } + + const response = await request(app).post(path).send(body); + + expect(response.statusCode).toBe(200); + expect(ctx.responseState.has(response.body.id)).toBe(false); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'parent-session' } }); + }); + + test('expired lookup queues deletion until no live response state references the session, then retries failures', async () => { + const { ctx } = createTestApp(); + ctx.responseState.set('expired-lookup', { + sessionId: 'shared-expired-session', + model: 'opencode/kimi-k2.5', + expiresAt: Date.now() - 1, + }); + ctx.responseState.set('live-reference', { + sessionId: 'shared-expired-session', + model: 'opencode/kimi-k2.5', + expiresAt: Date.now() + 60_000, + }); + sdkMocks.sessionDelete.mockRejectedValueOnce(new Error('temporary delete failure')); + + expect(ctx.getResponseState('expired-lookup', 'anonymous')).toBeNull(); + expect(ctx.responseState.has('expired-lookup')).toBe(false); + await ctx.sweepResponseState(); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + + ctx.responseState.get('live-reference').expiresAt = Date.now() - 1; + await ctx.sweepResponseState(); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(1, { path: { id: 'shared-expired-session' } }); + + await ctx.sweepResponseState(); + expect(sdkMocks.sessionDelete).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(2, { path: { id: 'shared-expired-session' } }); + }); + + test.each([ + ['/v1/responses', 'previous_response_id', { model: 'opencode/kimi-k2.5', input: 'tenant turn' }, 'invalid_previous_response_id'], + ['/v1beta/interactions', 'previous_interaction_id', { model: 'opencode/kimi-k2.5', input: 'tenant turn' }, 'invalid_previous_interaction_id'], + ])('%s continuation state is tenant-owned, while the owning tenant can continue it', async (path, previousKey, basePayload, invalidCode) => { + const { app, ctx } = createTestApp({ API_KEYS: ['tenant-a-secret', 'tenant-b-secret'] }); + const call = (tenant, body) => request(app).post(path) + .set('Authorization', `Bearer ${tenant}`) + .send(body); + const seeded = await call('tenant-a-secret', basePayload); + expect(seeded.statusCode).toBe(200); + const state = ctx.responseState.get(seeded.body.id); + expect(state.tenantId).toMatch(/^key:[a-f0-9]{64}$/); + expect(state.tenantId).not.toContain('tenant-a-secret'); + + const continueBody = { input: 'same-tenant follow-up', [previousKey]: seeded.body.id }; + if (path === '/v1beta/interactions') continueBody.model = 'opencode/kimi-k2.5'; + const crossTenant = await call('tenant-b-secret', continueBody); + expect(crossTenant.statusCode).toBe(400); + expect(crossTenant.body.error.code).toBe(invalidCode); + expect(sdkMocks.sessionFork).not.toHaveBeenCalled(); + + const sameTenant = await call('tenant-a-secret', continueBody); + expect(sameTenant.statusCode).toBe(200); + expect(sdkMocks.sessionFork).toHaveBeenCalledTimes(1); + }); + + test('legacy continuation state without tenant belongs only to anonymous callers', () => { + const { ctx } = createTestApp(); + const legacyState = { + sessionId: 'legacy-session', + model: 'opencode/kimi-k2.5', + expiresAt: Date.now() + 60_000, + }; + ctx.responseState.set('legacy-state', legacyState); + + expect(ctx.getResponseState('legacy-state', 'anonymous')).toBe(legacyState); + expect(ctx.getResponseState('legacy-state', 'key:authenticated-digest')).toBeNull(); + }); + + test('retries deleting a continuation child after its first cleanup attempt fails', async () => { + const { app } = createTestApp(); + const seed = await request(app).post('/v1/responses').send({ model: 'opencode/kimi-k2.5', input: 'seed' }); + expect(seed.statusCode).toBe(200); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: 'continuation-child' } }); + sdkMocks.sessionPrompt.mockResolvedValueOnce({ + data: { parts: [{ type: 'text', text: 'answer without message metadata' }] }, + }); + sdkMocks.sessionDelete.mockRejectedValueOnce(new Error('temporary delete failure')); + + const continuation = await request(app).post('/v1/responses').send({ + input: 'follow-up', + previous_response_id: seed.body.id, + }); + + expect(continuation.statusCode).toBe(200); + expect(sdkMocks.sessionDelete).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(1, { path: { id: 'continuation-child' } }); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(2, { path: { id: 'continuation-child' } }); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: 'parent-session' } }); + }); +}); diff --git a/tests/dockerfile-pin.test.js b/tests/dockerfile-pin.test.js index 523efec..3b27436 100644 --- a/tests/dockerfile-pin.test.js +++ b/tests/dockerfile-pin.test.js @@ -53,4 +53,15 @@ describe('Dockerfile OpenCode backend pin', () => { test('does not persist the pin as a runtime ENV', () => { expect(dockerfile).not.toMatch(/^\s*ENV\s+OPENCODE_VERSION=/m); }); + + test('workflow files match PINNED_VERSION', () => { + const rootDir = resolve(dirname(fileURLToPath(import.meta.url)), '..'); + const workflows = ['ai-fix.yml', 'ai-review.yml', 'ai-scan.yml']; + for (const file of workflows) { + const content = readFileSync(resolve(rootDir, '.github', 'workflows', file), 'utf8'); + const match = content.match(/npm\s+i\s+-g\s+opencode-ai@([\w.-]+)/); + expect(match).not.toBeNull(); + expect(match[1]).toBe(PINNED_VERSION); + } + }); }); diff --git a/tests/http-errors.test.js b/tests/http-errors.test.js new file mode 100644 index 0000000..88df86a --- /dev/null +++ b/tests/http-errors.test.js @@ -0,0 +1,732 @@ +import { jest } from '@jest/globals'; +import request from 'supertest'; +import express from 'express'; +import net from 'node:net'; + +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { + providers: [ + { + id: 'opencode', + models: { + 'kimi-k2.5': { id: 'kimi-k2.5', name: 'Kimi K2.5' }, + 'kimi-k2.5-free': { id: 'kimi-k2.5-free', name: 'Kimi K2.5 Free' } + } + } + ] + } + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ['bash', 'read']), + sessionCreate: jest.fn(async () => ({ id: 'mock-session-123' })), + sessionPrompt: jest.fn(async () => ({ parts: [] })), + sessionMessages: jest.fn(async () => []), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ + stream: (async function* () { })() + })) +}; + +jest.unstable_mockModule('http', () => ({ + default: { + get: jest.fn((url, options, callback) => { + const res = { statusCode: 200, headers: {}, on: jest.fn() }; + callback(res); + return { on: jest.fn(), destroy: jest.fn(), setTimeout: jest.fn() }; + }) + } +})); + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { create: sdkMocks.sessionCreate, prompt: sdkMocks.sessionPrompt, messages: sdkMocks.sessionMessages, delete: sdkMocks.sessionDelete }, + event: { subscribe: sdkMocks.eventSubscribe } + })) +})); + +const { createApp } = await import('../src/proxy.js'); +const { classifyHttpError } = await import('../src/errors/http.js'); +const { registerErrorHandler } = await import('../src/routes/system.js'); +const { toAnthropicErrorType } = await import('../src/converters/anthropic.js'); + +describe('Pure HTTP Error Classifier (classifyHttpError)', () => { + test('classifies entity.parse.failed as 400 invalid_json for OpenAI routes', () => { + const result = classifyHttpError({ type: 'entity.parse.failed' }, '/v1/chat/completions'); + expect(result.status).toBe(400); + expect(result.statusCode).toBe(400); + expect(result.body).toEqual({ + error: { + message: 'Malformed JSON in request body', + type: 'invalid_request_error', + code: 'invalid_json' + } + }); + }); + + test('classifies entity.parse.failed as 400 invalid_json for Anthropic /v1/messages', () => { + const result = classifyHttpError({ type: 'entity.parse.failed' }, '/v1/messages'); + expect(result.status).toBe(400); + expect(result.statusCode).toBe(400); + expect(result.body).toEqual({ + type: 'error', + error: { + type: 'invalid_request_error', + message: 'Malformed JSON in request body', + code: 'invalid_json' + } + }); + }); + + test('classifies SyntaxError with status 400 as 400 invalid_json', () => { + const err = new SyntaxError('Unexpected token'); + err.status = 400; + const result = classifyHttpError(err, '/v1/responses'); + expect(result.status).toBe(400); + expect(result.body.error.code).toBe('invalid_json'); + expect(result.body.error.message).toBe('Malformed JSON in request body'); + }); + + test('classifies entity.too.large as 413 entity_too_large', () => { + const openAiRes = classifyHttpError({ type: 'entity.too.large' }, '/v1/chat/completions'); + expect(openAiRes.status).toBe(413); + expect(openAiRes.body).toEqual({ + error: { + message: 'Request body exceeds the maximum permitted size', + type: 'invalid_request_error', + code: 'entity_too_large' + } + }); + + const anthropicRes = classifyHttpError({ type: 'entity.too.large', status: 413 }, '/v1/messages'); + expect(anthropicRes.status).toBe(413); + expect(anthropicRes.body.error.type).toBe('request_too_large'); + expect(anthropicRes.body).toEqual({ + type: 'error', + error: { + type: 'request_too_large', + message: 'Request body exceeds the maximum permitted size', + code: 'entity_too_large' + } + }); + }); + + test('classifies charset.unsupported and encoding.unsupported as 415 unsupported_media_type', () => { + const charsetRes = classifyHttpError({ type: 'charset.unsupported' }, '/v1/chat/completions'); + expect(charsetRes.status).toBe(415); + expect(charsetRes.body).toEqual({ + error: { + message: 'Unsupported body encoding or charset', + type: 'invalid_request_error', + code: 'unsupported_media_type' + } + }); + + const encodingRes = classifyHttpError({ type: 'encoding.unsupported' }, '/v1/messages'); + expect(encodingRes.status).toBe(415); + expect(encodingRes.body).toEqual({ + type: 'error', + error: { + type: 'invalid_request_error', + message: 'Unsupported body encoding or charset', + code: 'unsupported_media_type' + } + }); + }); + + test('classifies entity.verify.failed as 400 without code', () => { + const openAiRes = classifyHttpError({ type: 'entity.verify.failed' }, '/v1/chat/completions'); + expect(openAiRes.status).toBe(400); + expect(openAiRes.body).toEqual({ + error: { + message: 'Request body verification failed', + type: 'invalid_request_error' + } + }); + + const anthropicRes = classifyHttpError({ type: 'entity.verify.failed' }, '/v1/messages'); + expect(anthropicRes.status).toBe(400); + expect(anthropicRes.body).toEqual({ + type: 'error', + error: { + type: 'invalid_request_error', + message: 'Request body verification failed' + } + }); + }); + + test('generic fallback returns 500 without leaking error details', () => { + const secretError = new Error('Database password leak at /var/secrets/key.pem'); + secretError.body = 'sensitive raw snippet'; + secretError.stack = 'SecretError at /app/server.ts:42'; + + const openAiRes = classifyHttpError(secretError, '/v1/chat/completions'); + expect(openAiRes.status).toBe(500); + expect(openAiRes.body).toEqual({ + error: { + message: 'Internal server error', + type: 'server_error' + } + }); + const serializedOpenAi = JSON.stringify(openAiRes); + expect(serializedOpenAi).not.toContain('Database password'); + expect(serializedOpenAi).not.toContain('sensitive raw snippet'); + expect(serializedOpenAi).not.toContain('/var/secrets'); + expect(serializedOpenAi).not.toContain('SecretError'); + + const anthropicRes = classifyHttpError(secretError, '/v1/messages'); + expect(anthropicRes.status).toBe(500); + expect(anthropicRes.body).toEqual({ + type: 'error', + error: { + type: 'api_error', + message: 'Internal server error' + } + }); + const serializedAnthropic = JSON.stringify(anthropicRes); + expect(serializedAnthropic).not.toContain('Database password'); + expect(serializedAnthropic).not.toContain('sensitive raw snippet'); + expect(serializedAnthropic).not.toContain('/var/secrets'); + }); +}); + +describe('Anthropic Error Type Mapping (toAnthropicErrorType)', () => { + test('maps 415 to invalid_request_error', () => { + expect(toAnthropicErrorType(415)).toBe('invalid_request_error'); + }); +}); + +describe('Terminal Express Error Handler with Supertest', () => { + let app; + beforeEach(() => { + jest.clearAllMocks(); + app = createApp({ + PORT: 10000, + API_KEY: 'test-key', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 5000, + DISABLE_TOOLS: true, + DEBUG: false + }).app; + }); + + test('malformed JSON on /v1/chat/completions returns 400 JSON (never HTML or stack)', async () => { + const res = await request(app) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(res.status).toBe(400); + expect(res.headers['content-type']).toMatch(/application\/json/); + expect(res.body).toEqual({ + error: { + message: 'Malformed JSON in request body', + type: 'invalid_request_error', + code: 'invalid_json' + } + }); + expect(res.text).not.toContain(' { + const res = await request(app) + .post('/v1/messages') + .set('x-api-key', 'test-key') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(res.status).toBe(400); + expect(res.headers['content-type']).toMatch(/application\/json/); + expect(res.body).toEqual({ + type: 'error', + error: { + type: 'invalid_request_error', + message: 'Malformed JSON in request body', + code: 'invalid_json' + } + }); + expect(res.text).not.toContain(' { + const res = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(res.status).toBe(400); + expect(res.headers['content-type']).toMatch(/application\/json/); + expect(res.body).toEqual({ + error: { + message: 'Malformed JSON in request body', + type: 'invalid_request_error', + code: 'invalid_json' + } + }); + expect(res.text).not.toContain(' { + const res = await request(app) + .post('/v1/interactions') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(res.status).toBe(400); + expect(res.headers['content-type']).toMatch(/application\/json/); + expect(res.body).toEqual({ + error: { + message: 'Malformed JSON in request body', + type: 'invalid_request_error', + code: 'invalid_json' + } + }); + expect(res.text).not.toContain(' { + const call = request(app).post(path).set(authHeader, authValue) + .set('Content-Type', 'text/plain').send('not JSON'); + const res = await call; + + expect(res.status).toBe(415); + if (path === '/v1/messages') { + expect(res.body).toEqual({ + type: 'error', + error: { + type: 'invalid_request_error', + message: 'Unsupported or missing Content-Type; expected a JSON media type', + code: 'unsupported_media_type' + } + }); + } else { + expect(res.body).toEqual({ + error: { + message: 'Unsupported or missing Content-Type; expected a JSON media type', + type: 'invalid_request_error', + code: 'unsupported_media_type' + } + }); + } + }); + + test('nonempty body without Content-Type returns 415, but an empty body can reach normal validation', async () => { + const missingType = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', '') + .send('not JSON'); + expect(missingType.status).toBe(415); + expect(missingType.body.error.code).toBe('unsupported_media_type'); + + const missingBody = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', ''); + expect(missingBody.status).toBe(400); + expect(missingBody.body.error.type).toBe('invalid_request_error'); + }); + + test('empty chunked body without Content-Type reaches normal validation', async () => { + const server = app.listen(0, '127.0.0.1'); + try { + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + const port = typeof address === 'object' && address ? address.port : 0; + const rawResponse = await new Promise((resolve, reject) => { + const socket = net.createConnection({ host: '127.0.0.1', port }); + let response = ''; + socket.on('connect', () => { + socket.write([ + 'POST /v1/responses HTTP/1.1', + 'Host: 127.0.0.1', + 'Authorization: Bearer test-key', + 'Transfer-Encoding: chunked', + 'Connection: close', + '', + '0', + '', + '', + ].join('\r\n')); + }); + socket.on('data', (chunk) => { response += chunk.toString('utf8'); }); + socket.on('error', reject); + socket.on('end', () => resolve(response)); + }); + expect(rawResponse).toMatch(/^HTTP\/1\.1 400\b/); + expect(rawResponse).toContain('"type":"invalid_request_error"'); + expect(rawResponse).not.toContain('"code":"unsupported_media_type"'); + } finally { + await new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())); + } + }); + + test('nonempty chunked body without Content-Type returns 415', async () => { + const server = app.listen(0, '127.0.0.1'); + try { + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + const port = typeof address === 'object' && address ? address.port : 0; + const rawResponse = await new Promise((resolve, reject) => { + const socket = net.createConnection({ host: '127.0.0.1', port }); + let response = ''; + socket.on('connect', () => { + socket.write([ + 'POST /v1/responses HTTP/1.1', + 'Host: 127.0.0.1', + 'Authorization: Bearer test-key', + 'Transfer-Encoding: chunked', + 'Connection: close', + '', + '4', + 'nope', + '0', + '', + '', + ].join('\r\n')); + }); + socket.on('data', (chunk) => { response += chunk.toString('utf8'); }); + socket.on('error', reject); + socket.on('end', () => resolve(response)); + }); + expect(rawResponse).toMatch(/^HTTP\/1\.1 415\b/); + expect(rawResponse).toContain('"code":"unsupported_media_type"'); + } finally { + await new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())); + } + }); + + test('unsupported incomplete body is rejected and the connection is closed', async () => { + const server = app.listen(0, '127.0.0.1'); + try { + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + const port = typeof address === 'object' && address ? address.port : 0; + const rawResponse = await new Promise((resolve, reject) => { + const socket = net.createConnection({ host: '127.0.0.1', port }); + let response = ''; + socket.setTimeout(3000, () => socket.destroy(new Error('server did not close rejected request'))); + socket.on('connect', () => { + socket.write([ + 'POST /v1/responses HTTP/1.1', + 'Host: 127.0.0.1', + 'Authorization: Bearer test-key', + 'Content-Type: text/plain', + 'Content-Length: 1048576', + 'Connection: keep-alive', + '', + 'x'.repeat(64 * 1024), + ].join('\r\n')); + }); + socket.on('data', (chunk) => { response += chunk.toString('utf8'); }); + socket.on('error', reject); + socket.on('end', () => resolve(response)); + }); + expect(rawResponse).toMatch(/^HTTP\/1\.1 415\b/); + expect(rawResponse).toMatch(/Connection: close/i); + } finally { + await new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())); + } + }); + + test('unauthenticated malformed JSON returns 401 (auth before parsing)', async () => { + const chatRes = await request(app) + .post('/v1/chat/completions') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(chatRes.status).toBe(401); + expect(chatRes.body).toEqual({ + error: { message: 'Unauthorized' } + }); + + const messagesRes = await request(app) + .post('/v1/messages') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(messagesRes.status).toBe(401); + expect(messagesRes.body).toEqual({ + type: 'error', + error: { + type: 'authentication_error', + message: 'Unauthorized' + } + }); + }); + + test('unauthenticated unsupported media type still returns 401 before media validation', async () => { + const chatRes = await request(app) + .post('/v1/chat/completions') + .set('Content-Type', 'text/plain') + .send('not JSON'); + expect(chatRes.status).toBe(401); + expect(chatRes.body).toEqual({ error: { message: 'Unauthorized' } }); + + const messagesRes = await request(app) + .post('/v1/messages') + .set('Content-Type', 'text/plain') + .send('not JSON'); + expect(messagesRes.status).toBe(401); + expect(messagesRes.body).toEqual({ + type: 'error', + error: { type: 'authentication_error', message: 'Unauthorized' } + }); + }); + + test('unauthenticated incomplete upload is rejected and the connection is closed', async () => { + const server = app.listen(0, '127.0.0.1'); + try { + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + const port = typeof address === 'object' && address ? address.port : 0; + const rawResponse = await new Promise((resolve, reject) => { + const socket = net.createConnection({ host: '127.0.0.1', port }); + let response = ''; + socket.setTimeout(3000, () => socket.destroy(new Error('server did not close unauthorized request'))); + socket.on('connect', () => { + socket.write([ + 'POST /v1/responses HTTP/1.1', + 'Host: 127.0.0.1', + 'Content-Type: application/json', + 'Content-Length: 1048576', + 'Connection: keep-alive', + '', + 'x'.repeat(64 * 1024), + ].join('\r\n')); + }); + socket.on('data', (chunk) => { response += chunk.toString('utf8'); }); + socket.on('error', reject); + socket.on('end', () => resolve(response)); + }); + expect(rawResponse).toMatch(/^HTTP\/1\.1 401\b/); + expect(rawResponse).toMatch(/Connection: close/i); + } finally { + await new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())); + } + }); + + test('unauthorized request on /v1/messages/ with trailing slash returns Anthropic error format', async () => { + const res = await request(app) + .post('/v1/messages/') + .set('Content-Type', 'application/json') + .send('{bad json'); + + expect(res.status).toBe(401); + expect(res.body).toEqual({ + type: 'error', + error: { + type: 'authentication_error', + message: 'Unauthorized' + } + }); + }); + + test('oversized body (simulated 413) returns 413 JSON', async () => { + const testApp = express(); + testApp.disable('x-powered-by'); + testApp.post('/v1/chat/completions', (_req, _res, next) => { + const err = new Error('request entity too large'); + err.type = 'entity.too.large'; + err.status = 413; + next(err); + }); + testApp.post('/v1/messages', (_req, _res, next) => { + const err = new Error('request entity too large'); + err.type = 'entity.too.large'; + err.status = 413; + next(err); + }); + registerErrorHandler(testApp); + + const chatRes = await request(testApp) + .post('/v1/chat/completions') + .send({}); + expect(chatRes.status).toBe(413); + expect(chatRes.body).toEqual({ + error: { + message: 'Request body exceeds the maximum permitted size', + type: 'invalid_request_error', + code: 'entity_too_large' + } + }); + + const messagesRes = await request(testApp) + .post('/v1/messages') + .send({}); + expect(messagesRes.status).toBe(413); + expect(messagesRes.body.error.type).toBe('request_too_large'); + expect(messagesRes.body).toEqual({ + type: 'error', + error: { + type: 'request_too_large', + message: 'Request body exceeds the maximum permitted size', + code: 'entity_too_large' + } + }); + }); + + test('response headers do NOT include x-powered-by', async () => { + const healthRes = await request(app).get('/health'); + expect(healthRes.headers['x-powered-by']).toBeUndefined(); + + const modelsRes = await request(app).get('/v1/models'); + expect(modelsRes.headers['x-powered-by']).toBeUndefined(); + + const postRes = await request(app) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .set('Content-Type', 'application/json') + .send('{bad json'); + expect(postRes.headers['x-powered-by']).toBeUndefined(); + }); + + test('unknown model request returns 404 with model_not_found and available_models', async () => { + const res = await request(app) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/unknown-model-xyz', + messages: [{ role: 'user', content: 'hello' }] + }); + + expect(res.status).toBe(404); + expect(res.body.code).toBe('model_not_found'); + expect(res.body.type).toBe('model_not_found'); + expect(Array.isArray(res.body.available_models)).toBe(true); + expect(res.body.available_models).toContain('opencode/kimi-k2.5'); + expect(res.body.available_models).toContain('opencode/kimi-k2.5-free'); + }); + + test('unknown model request on /v1/messages returns 404', async () => { + const res = await request(app) + .post('/v1/messages') + .set('x-api-key', 'test-key') + .send({ + model: 'opencode/unknown-model-xyz', + max_tokens: 100, + messages: [{ role: 'user', content: 'hello' }] + }); + + expect(res.status).toBe(404); + expect(res.body.type).toBe('error'); + expect(res.body.error.code).toBe('model_not_found'); + expect(res.body.error.type).toBe('not_found_error'); + }); + + test('unknown model request on /v1/responses returns 404', async () => { + const res = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/unknown-model-xyz', + input: 'hello' + }); + + expect(res.status).toBe(404); + expect(res.body.code).toBe('model_not_found'); + expect(res.body.type).toBe('model_not_found'); + }); + + test('unknown model request on /v1/interactions returns 404', async () => { + const res = await request(app) + .post('/v1/interactions') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/unknown-model-xyz', + input: 'hello' + }); + + expect(res.status).toBe(404); + expect(res.body.code).toBe('model_not_found'); + expect(res.body.type).toBe('model_not_found'); + }); + + test('headersSent check in registerErrorHandler calls next(err)', async () => { + const testApp = express(); + let nextCalledWith = null; + testApp.get('/stream', (_req, res, next) => { + res.writeHead(200, { 'Content-Type': 'text/plain' }); + res.write('chunk'); + next(new Error('stream broke mid-flight')); + }); + registerErrorHandler(testApp); + // Custom error recorder after registerErrorHandler to verify next(err) was called + testApp.use((err, _req, res, _next) => { + nextCalledWith = err; + if (!res.writableEnded) res.end(); + }); + + await request(testApp).get('/stream'); + expect(nextCalledWith).toBeInstanceOf(Error); + expect(nextCalledWith.message).toBe('stream broke mid-flight'); + }); + + test('unexpected route errors use server_error and api_error in actual HTTP responses', async () => { + const testApp = express(); + testApp.post('/v1/responses', (_req, _res, next) => next(new Error('secret internal detail'))); + testApp.post('/v1/messages', (_req, _res, next) => next(new Error('secret internal detail'))); + registerErrorHandler(testApp); + + const openAiRes = await request(testApp).post('/v1/responses').send({}); + expect(openAiRes.status).toBe(500); + expect(openAiRes.body).toEqual({ error: { message: 'Internal server error', type: 'server_error' } }); + expect(openAiRes.text).not.toContain('secret internal detail'); + + const anthropicRes = await request(testApp).post('/v1/messages').send({}); + expect(anthropicRes.status).toBe(500); + expect(anthropicRes.body).toEqual({ + type: 'error', + error: { type: 'api_error', message: 'Internal server error' } + }); + expect(anthropicRes.text).not.toContain('secret internal detail'); + }); + + test('trailing slashes on bypass endpoints (/health/, /health/details/, /metrics/) bypass auth without 401 when API_KEY is configured', async () => { + const bypassApp = createApp({ + PORT: 10000, + API_KEY: 'test-key', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 5000, + DISABLE_TOOLS: true, + DEBUG: false, + HEALTH_DETAILS_ENABLED: true, + HEALTH_DETAILS_REQUIRE_AUTH: false, + METRICS_ENABLED: true, + METRICS_REQUIRE_AUTH: false + }).app; + + const healthRes = await request(bypassApp).get('/health/'); + expect(healthRes.status).toBe(200); + expect(healthRes.body.status).toBe('ok'); + + const healthDetailsRes = await request(bypassApp).get('/health/details/'); + expect(healthDetailsRes.status).toBe(200); + expect(healthDetailsRes.body.status).toBe('ok'); + + const metricsRes = await request(bypassApp).get('/metrics/'); + expect(metricsRes.status).toBe(200); + expect(metricsRes.text).toContain('opencode_internal_tool_mode_requests_total'); + }); + + test('GET /health/ with trailing slash bypasses auth without 401 on default app', async () => { + const res = await request(app).get('/health/'); + expect(res.status).toBe(200); + expect(res.body).toEqual({ status: 'ok', proxy: true }); + }); +}); diff --git a/tests/idempotency.test.js b/tests/idempotency.test.js new file mode 100644 index 0000000..0b5223a --- /dev/null +++ b/tests/idempotency.test.js @@ -0,0 +1,2767 @@ +import express from 'express'; +import net from 'node:net'; +import request from 'supertest'; +import { jest } from '@jest/globals'; + +import { + canonicalJsonHash, + IdempotencyRegistry, + idempotencyDigest, + idempotencyHeaderStorageBytes, + idempotencyResponseStorageBytes, + MIN_IDEMPOTENCY_TTL_MS, + DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES_PER_TENANT, + DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES, +} from '../src/idempotency/registry.js'; +import { + createIdempotencyMiddleware, + scanSseDataEventRanges, + isDeterministicPreSubmitFailure, +} from '../src/idempotency/middleware.js'; + +const promptBySession = new Map(); +let sessionSequence = 0; +let forkSequence = 0; +let assistantSequence = 0; +let toolObservationCount = 0; + +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { providers: [{ id: 'opencode', models: { 'kimi-k2.5': { name: 'Kimi' } } }] }, + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ({ data: ['websearch', 'webfetch'] })), + sessionCreate: jest.fn(async () => ({ data: { id: `idem-session-${++sessionSequence}` } })), + sessionFork: jest.fn(async () => ({ data: { id: `idem-fork-${++forkSequence}` } })), + sessionPrompt: jest.fn(async (args) => { + const sessionId = args?.path?.id; + promptBySession.set(sessionId, structuredClone(args)); + const assistantId = `assistant-${++assistantSequence}`; + return { + data: { + info: { id: assistantId, role: 'assistant', parentID: args?.body?.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'idempotent answer' }], + }, + }; + }), + sessionMessages: jest.fn(async ({ path }) => { + const prompt = promptBySession.get(path.id); + const messageID = prompt?.body?.messageID; + const assistantId = `assistant-poll-${path.id}`; + const parts = [{ type: 'text', text: 'idempotent answer' }]; + if (prompt?.body?.tools?.websearch === true) { + toolObservationCount += 1; + parts.push({ + type: 'tool', + id: 'search-tool-call', + tool: 'websearch', + state: { status: 'completed', input: { query: 'test query' }, output: 'tool result' }, + }); + } + return [{ + info: { + id: assistantId, + role: 'assistant', + parentID: messageID, + sessionID: path.id, + finish: 'stop', + }, + parts, + }]; + }), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ stream: (async function* () {})() })), +}; +const defaultSessionMessagesImplementation = sdkMocks.sessionMessages.getMockImplementation(); + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { + create: sdkMocks.sessionCreate, + fork: sdkMocks.sessionFork, + prompt: sdkMocks.sessionPrompt, + messages: sdkMocks.sessionMessages, + delete: sdkMocks.sessionDelete, + }, + event: { subscribe: sdkMocks.eventSubscribe }, + })), +})); + +jest.unstable_mockModule('http', () => ({ + default: { + get: jest.fn((_url, options, callback) => { + const cb = typeof options === 'function' ? options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn(), setTimeout: jest.fn() }; + }), + }, +})); + +jest.unstable_mockModule('https', () => ({ + default: { + get: jest.fn((_url, options, callback) => { + const cb = typeof options === 'function' ? options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn() }; + }), + }, +})); + +const { createApp } = await import('../src/proxy.js'); + +afterEach(() => { + jest.restoreAllMocks(); +}); + +function createRouteApp(overrides = {}) { + return createApp({ + PORT: 10000, + API_KEY: '', + API_KEYS: [], + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 1000, + RETRY_MAX_RETRIES: 0, + DISABLE_TOOLS: true, + DEBUG: false, + ...overrides, + }); +} + +function createMiddlewareApp({ keys = [], route, registryOptions = {} } = {}) { + const app = express(); + const registry = new IdempotencyRegistry(registryOptions); + app.use(express.json()); + app.use(createIdempotencyMiddleware({ + registry, + configuredKeys: keys, + matchedKeyIndex: (req) => { + const authorization = req.headers.authorization; + const token = typeof authorization === 'string' ? authorization.replace(/^Bearer\s+/i, '') : ''; + return keys.indexOf(token); + }, + })); + app.post(['/v1/responses', '/v1beta/interactions', '/v1/interactions'], route); + return { app, registry }; +} + +async function startAbortedRequest(app, path, key, body) { + const server = app.listen(0); + await new Promise((resolve) => server.once('listening', resolve)); + const address = server.address(); + const socket = net.createConnection(address.port, '127.0.0.1'); + await new Promise((resolve, reject) => { + socket.once('connect', resolve); + socket.once('error', reject); + }); + const payload = JSON.stringify(body); + socket.write( + `POST ${path} HTTP/1.1\r\n` + + 'Host: 127.0.0.1\r\n' + + `Idempotency-Key: ${key}\r\n` + + 'Content-Type: application/json\r\n' + + `Content-Length: ${Buffer.byteLength(payload)}\r\n` + + 'Connection: close\r\n\r\n' + + payload, + ); + return { + abort: () => socket.destroy(), + closeServer: () => new Promise((resolve, reject) => server.close((error) => error ? reject(error) : resolve())), + }; +} + +describe('idempotency canonicalization and bounded registry', () => { + test('canonical JSON hashes ignore object order and nested object order, but preserve array order', () => { + expect(canonicalJsonHash({ z: 2, a: { y: 1, x: 0 }, list: [1, 2] })) + .toBe(canonicalJsonHash({ list: [1, 2], a: { x: 0, y: 1 }, z: 2 })); + expect(canonicalJsonHash({ list: [1, 2] })).not.toBe(canonicalJsonHash({ list: [2, 1] })); + }); + + test('rejects non-finite numbers instead of canonicalizing them as null', () => { + expect(() => canonicalJsonHash({ max_output_tokens: Infinity })).toThrow(/non-finite number/); + expect(() => canonicalJsonHash({ max_output_tokens: -Infinity })).toThrow(/non-finite number/); + expect(() => canonicalJsonHash({ max_output_tokens: NaN })).toThrow(/non-finite number/); + }); + + test('same scope with a different payload hash conflicts and distinct scopes remain independent', () => { + const registry = new IdempotencyRegistry(); + const first = registry.begin('scope-a', canonicalJsonHash({ input: 'one' })); + expect(first.kind).toBe('created'); + expect(registry.begin('scope-a', canonicalJsonHash({ input: 'two' })).kind).toBe('mismatch'); + expect(registry.begin('scope-b', canonicalJsonHash({ input: 'two' })).kind).toBe('created'); + }); + + test('TTL is clamped to at least 30 minutes and expired records are removed', () => { + let now = 10_000; + const registry = new IdempotencyRegistry({ ttlMs: 1, now: () => now }); + const entry = registry.begin('scope', canonicalJsonHash({ input: 'x' })); + expect(entry.kind).toBe('created'); + const callerBody = Buffer.from('{"error":"bad request"}'); + registry.complete(entry.record, entry.record.generation, 'failed', { + statusCode: 400, + headers: { 'content-type': 'application/json' }, + body: callerBody, + }); + callerBody.fill(0); + expect(entry.record.wireResponse.body.toString()).toBe('{"error":"bad request"}'); + expect(registry.pendingResponseBytes).toBe(0); + expect(entry.record.expiresAt - now).toBeGreaterThanOrEqual(MIN_IDEMPOTENCY_TTL_MS); + now += MIN_IDEMPOTENCY_TTL_MS - 1; + expect(registry.get('scope')).not.toBeNull(); + now += 2; + expect(registry.get('scope')).toBeNull(); + }); + + test('retained header bytes are accounted and released with entry capacity after TTL', () => { + let now = 10_000; + const registry = new IdempotencyRegistry({ + maxEntries: 1, + ttlMs: MIN_IDEMPOTENCY_TTL_MS, + maxCachedBytes: 512, + maxCachedBytesPerTenant: 512, + now: () => now, + }); + const first = registry.begin('header-ttl-one', canonicalJsonHash({ input: 'one' }), 'tenant-a'); + expect(first.kind).toBe('created'); + const response = { + statusCode: 201, + headers: { 'x-retained-header': 'h'.repeat(80) }, + body: Buffer.from('x'), + }; + expect(registry.complete(first.record, first.record.generation, 'failed', response)).toBe(true); + const retainedBytes = idempotencyResponseStorageBytes(first.record.wireResponse); + expect(retainedBytes).toBeGreaterThan(response.body.byteLength); + expect(registry.retainedResponseBytes).toBe(retainedBytes); + expect(registry.getTenantRetainedResponseBytes('tenant-a')).toBe(retainedBytes); + expect(registry.begin('header-ttl-two', canonicalJsonHash({ input: 'two' }), 'tenant-a').kind).toBe('full'); + + now += MIN_IDEMPOTENCY_TTL_MS + 1; + expect(registry.size).toBe(0); + expect(registry.retainedResponseBytes).toBe(0); + expect(registry.getTenantRetainedResponseBytes('tenant-a')).toBe(0); + expect(registry.getTenantRetainedEntries('tenant-a')).toBe(0); + expect(registry.begin('header-ttl-two', canonicalJsonHash({ input: 'two' }), 'tenant-a').kind).toBe('created'); + }); + + test('own __proto__ response headers are estimated and cloned as ordinary headers', () => { + const registry = new IdempotencyRegistry(); + const entry = registry.begin('proto-header-clone', canonicalJsonHash({ input: 'same' })); + expect(entry.kind).toBe('created'); + const headerValues = ['first=one', 'second=two']; + const headers = Object.create(null); + Object.defineProperty(headers, '__proto__', { + value: headerValues, + enumerable: true, + configurable: true, + writable: true, + }); + + expect(idempotencyHeaderStorageBytes(headers)).toBeGreaterThan(0); + expect(registry.complete(entry.record, entry.record.generation, 'completed', { + statusCode: 201, + headers, + body: Buffer.from('stable'), + })).toBe(true); + headerValues[0] = 'mutated'; + + expect(Object.getPrototypeOf(entry.record.wireResponse.headers)).toBeNull(); + expect(Object.hasOwn(entry.record.wireResponse.headers, '__proto__')).toBe(true); + expect(entry.record.wireResponse.headers['__proto__']).toEqual(['first=one', 'second=two']); + expect(idempotencyResponseStorageBytes(entry.record.wireResponse)).toBeGreaterThan(Buffer.byteLength('stable')); + }); + + test('unknown entries remain tombstoned after their ordinary replay TTL', () => { + let now = 100; + const registry = new IdempotencyRegistry({ ttlMs: MIN_IDEMPOTENCY_TTL_MS, now: () => now }); + const entry = registry.begin('unknown-scope', canonicalJsonHash({ input: 'x' })); + expect(entry.kind).toBe('created'); + registry.markUnknown(entry.record, entry.record.generation); + now += MIN_IDEMPOTENCY_TTL_MS * 10; + expect(registry.get('unknown-scope')?.state).toBe('unknown'); + expect(registry.begin('different-scope', canonicalJsonHash({ input: 'y' })).kind).toBe('created'); + }); + + test('submitted child sessions stay pinned while their operation is unresolved', () => { + const registry = new IdempotencyRegistry(); + const entry = registry.begin('unresolved-child', canonicalJsonHash({ input: 'x' }), 'tenant-a'); + expect(entry.kind).toBe('created'); + registry.markSubmitted(entry.record, entry.record.generation, 'child-session', 'user-message'); + expect(registry.hasUnresolvedChildSession('child-session')).toBe(true); + + registry.markUnknown(entry.record, entry.record.generation); + expect(registry.hasUnresolvedChildSession('child-session')).toBe(true); + const retry = registry.begin('unresolved-child', canonicalJsonHash({ input: 'x' }), 'tenant-a'); + expect(retry.kind).toBe('existing'); + expect(registry.hasUnresolvedChildSession('child-session')).toBe(true); + + registry.confirmResult(retry.record, retry.record.generation, 'user-message', 'assistant-message'); + expect(registry.complete(retry.record, retry.record.generation, 'completed', { + statusCode: 200, + headers: { 'content-type': 'application/json' }, + body: Buffer.from('{"ok":true}'), + })).toBe(true); + expect(registry.hasUnresolvedChildSession('child-session')).toBe(false); + }); + + test('registry refuses new live entries at its bound instead of evicting uncertain records', () => { + const registry = new IdempotencyRegistry({ maxEntries: 1 }); + const first = registry.begin('scope-a', canonicalJsonHash({ a: 1 })); + expect(first.kind).toBe('created'); + registry.markUnknown(first.record, first.record.generation); + expect(registry.begin('scope-b', canonicalJsonHash({ b: 2 })).kind).toBe('full'); + expect(registry.get('scope-a')?.state).toBe('unknown'); + }); + + test('per-tenant live quota isolates unknown outcomes while the global cap remains available', async () => { + const { app, registry } = createMiddlewareApp({ + keys: ['tenant-a-secret', 'tenant-b-secret'], + registryOptions: { maxEntries: 3, maxLiveEntriesPerTenant: 2 }, + route: (req, res) => { + if (req.headers.authorization === 'Bearer tenant-a-secret') { + if (!req.idempotency.reconciling) req.idempotency.markSubmitted(`child-${req.idempotency.publicId}`, `message-${req.idempotency.publicId}`); + res.status(503).json({ error: 'still uncertain' }); + return; + } + res.json({ tenant: 'b' }); + }, + }); + const send = (tenant, key) => request(app).post('/v1/responses') + .set('Authorization', `Bearer ${tenant}`) + .set('Idempotency-Key', key) + .send({ input: key }); + + expect((await send('tenant-a-secret', 'unknown-one')).statusCode).toBe(503); + expect((await send('tenant-a-secret', 'unknown-two')).statusCode).toBe(503); + const quotaFull = await send('tenant-a-secret', 'unknown-three'); + expect(quotaFull.statusCode).toBe(503); + expect(quotaFull.body.error.code).toBe('idempotency_capacity_exceeded'); + expect(quotaFull.body.error.retryable).toBe(true); + + const tenantB = await send('tenant-b-secret', 'tenant-b-key'); + expect(tenantB.statusCode).toBe(200); + expect(tenantB.body.tenant).toBe('b'); + + const noReprompt = await send('tenant-a-secret', 'unknown-one'); + expect(noReprompt.statusCode).toBe(503); + expect(noReprompt.body.error).toBe('still uncertain'); + expect(registry.size).toBe(3); + }); + + test('retained quota preserves completed replays until TTL, reserves capacity for other tenants, and frees slots after expiry', async () => { + let now = 1_000; + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ + keys: ['tenant-a-secret', 'tenant-b-secret'], + registryOptions: { maxEntries: 2, ttlMs: MIN_IDEMPOTENCY_TTL_MS, now: () => now }, + route: (_req, res) => { + handlerCalls += 1; + res.json({ handlerCalls }); + }, + }); + const send = (tenant, key) => request(app).post('/v1/responses') + .set('Authorization', `Bearer ${tenant}`) + .set('Idempotency-Key', key) + .send({ input: key }); + const tenantAId = `key:${idempotencyDigest('tenant-a-secret')}`; + + const firstA = await send('tenant-a-secret', 'retained-a-one'); + expect(firstA.statusCode).toBe(200); + expect(registry.getTenantRetainedEntries(tenantAId)).toBe(1); + const replayA = await send('tenant-a-secret', 'retained-a-one'); + expect(replayA.text).toBe(firstA.text); + expect(handlerCalls).toBe(1); + + const sameTenantAtQuota = await send('tenant-a-secret', 'retained-a-two'); + expect(sameTenantAtQuota.statusCode).toBe(503); + expect(sameTenantAtQuota.body.error.code).toBe('idempotency_capacity_exceeded'); + expect(handlerCalls).toBe(1); + expect(registry.getTenantRetainedEntries(tenantAId)).toBe(1); + expect(registry.getTenantRetainedResponseBytes(tenantAId)).toBeGreaterThan(Buffer.byteLength(firstA.text)); + expect((await send('tenant-a-secret', 'retained-a-one')).text).toBe(firstA.text); + expect(handlerCalls).toBe(1); + + const tenantB = await send('tenant-b-secret', 'retained-b-one'); + expect(tenantB.statusCode).toBe(200); + expect(handlerCalls).toBe(2); + expect(registry.size).toBe(2); + + now += MIN_IDEMPOTENCY_TTL_MS + 1; + expect(registry.size).toBe(0); + const afterExpiry = await send('tenant-a-secret', 'retained-a-one'); + expect(afterExpiry.statusCode).toBe(200); + expect(afterExpiry.text).not.toBe(firstA.text); + expect(handlerCalls).toBe(3); + expect(registry.getTenantRetainedEntries(tenantAId)).toBe(1); + expect(registry.getTenantRetainedResponseBytes(tenantAId)).toBeGreaterThan(Buffer.byteLength(afterExpiry.text)); + const tenantBAfterExpiry = await send('tenant-b-secret', 'retained-b-after-expiry'); + expect(tenantBAfterExpiry.statusCode).toBe(200); + expect(handlerCalls).toBe(4); + expect(registry.size).toBe(2); + }); + + test('maxEntries=2 clamps a tenant to one retained entry and reserves the other slot', () => { + const registry = new IdempotencyRegistry({ maxEntries: 2 }); + const firstA = registry.begin('small-global-a-one', canonicalJsonHash({ a: 1 }), 'tenant-a'); + expect(firstA.kind).toBe('created'); + expect(registry.complete(firstA.record, firstA.record.generation, 'completed', { + statusCode: 200, + headers: {}, + body: Buffer.from('tenant a'), + })).toBe(true); + expect(registry.begin('small-global-a-two', canonicalJsonHash({ a: 2 }), 'tenant-a').kind).toBe('full'); + + const secondTenant = registry.begin('small-global-b-one', canonicalJsonHash({ b: 1 }), 'tenant-b'); + expect(secondTenant.kind).toBe('created'); + const replay = registry.begin('small-global-a-one', canonicalJsonHash({ a: 1 }), 'tenant-a'); + expect(replay.kind).toBe('existing'); + expect(replay.record).toBe(firstA.record); + expect(replay.record.wireResponse.body.toString()).toBe('tenant a'); + expect(registry.size).toBe(2); + }); + + test('small global byte limit reserves capacity for another tenant and default budget fits one maximum response', () => { + const registry = new IdempotencyRegistry({ maxCachedBytes: 10 }); + const tenantA = registry.begin('small-bytes-a', canonicalJsonHash({ a: 1 }), 'tenant-a'); + const tenantB = registry.begin('small-bytes-b', canonicalJsonHash({ b: 1 }), 'tenant-b'); + const tenantC = registry.begin('small-bytes-c', canonicalJsonHash({ c: 1 }), 'tenant-c'); + expect(tenantA.kind).toBe('created'); + expect(tenantB.kind).toBe('created'); + expect(tenantC.kind).toBe('created'); + + expect(registry.complete(tenantC.record, tenantC.record.generation, 'completed', { + statusCode: 200, + headers: {}, + body: Buffer.alloc(10, 0x63), + })).toBe(false); + expect(registry.retainedResponseBytes).toBe(0); + + expect(registry.complete(tenantA.record, tenantA.record.generation, 'completed', { + statusCode: 200, + headers: {}, + body: Buffer.alloc(9, 0x61), + })).toBe(true); + expect(registry.getTenantRetainedResponseBytes('tenant-a')).toBe(9); + expect(registry.complete(tenantB.record, tenantB.record.generation, 'completed', { + statusCode: 200, + headers: {}, + body: Buffer.alloc(1, 0x62), + })).toBe(true); + expect(registry.retainedResponseBytes).toBe(10); + expect(registry.getTenantRetainedResponseBytes('tenant-b')).toBe(1); + expect(DEFAULT_IDEMPOTENCY_MAX_CACHED_BYTES_PER_TENANT).toBeGreaterThanOrEqual( + 2 * DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES + 1_024 * 128 + 1024 * 1024, + ); + }); + + test('small global byte limit reserves pending capture capacity for another tenant', () => { + const registry = new IdempotencyRegistry({ maxCachedBytes: 10 }); + const tenantA = registry.begin('pending-bytes-a', canonicalJsonHash({ a: 1 }), 'tenant-a'); + const tenantB = registry.begin('pending-bytes-b', canonicalJsonHash({ b: 1 }), 'tenant-b'); + expect(tenantA.kind).toBe('created'); + expect(tenantB.kind).toBe('created'); + + expect(registry.reserveCaptureBytes(tenantA.record, tenantA.record.generation, 10)).toBe(false); + expect(registry.reserveCaptureBytes(tenantA.record, tenantA.record.generation, 9)).toBe(true); + expect(registry.reserveCaptureBytes(tenantB.record, tenantB.record.generation, 2)).toBe(false); + expect(registry.reserveCaptureBytes(tenantB.record, tenantB.record.generation, 1)).toBe(true); + expect(registry.pendingResponseBytes).toBe(10); + expect(registry.getTenantPendingResponseBytes('tenant-a')).toBe(9); + expect(registry.getTenantPendingResponseBytes('tenant-b')).toBe(1); + }); + + test('retained and pending response bytes are bounded independently per tenant', async () => { + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ + keys: ['tenant-a-secret', 'tenant-b-secret'], + registryOptions: { + maxCachedBytes: 1_000, + maxCachedBytesPerTenant: 350, + maxResponseBytes: 200, + }, + route: (_req, res) => { + handlerCalls += 1; + res.status(201).send('x'.repeat(50)); + }, + }); + const send = (tenant, key) => request(app).post('/v1/responses') + .set('Authorization', `Bearer ${tenant}`) + .set('Idempotency-Key', key) + .send({ input: key }); + const tenantAId = `key:${idempotencyDigest('tenant-a-secret')}`; + const tenantBId = `key:${idempotencyDigest('tenant-b-secret')}`; + + expect((await send('tenant-a-secret', 'bytes-a-one')).statusCode).toBe(201); + expect(registry.getTenantRetainedResponseBytes(tenantAId)).toBeGreaterThan(50); + // The response is still delivered, but its temporary assembly buffer + // would exceed tenant A's retained + pending byte budget, so it is not + // retained as a replay and the reservation is released. + expect((await send('tenant-a-secret', 'bytes-a-two')).statusCode).toBe(201); + const tenantABytes = registry.getTenantRetainedResponseBytes(tenantAId); + expect(tenantABytes).toBeGreaterThan(50); + expect(registry.getTenantPendingResponseBytes(tenantAId)).toBe(0); + + expect((await send('tenant-b-secret', 'bytes-b-one')).statusCode).toBe(201); + const tenantBBytes = registry.getTenantRetainedResponseBytes(tenantBId); + expect(tenantBBytes).toBeGreaterThan(50); + expect(registry.retainedResponseBytes).toBe(tenantABytes + tenantBBytes); + expect(registry.pendingResponseBytes).toBe(0); + expect(handlerCalls).toBe(3); + }); +}); + +describe('Idempotency-Key middleware', () => { + test('SSE scanner handles CRLF boundaries and large data without building event strings', () => { + const largeData = Buffer.alloc(4 * 1024 * 1024, 0x78); + const prefix = Buffer.from('event: message\r\ndata: '); + const separator = Buffer.from('\r\n\r\ndata: {"type":"interaction.completed"}\r\n\r\n'); + const body = Buffer.concat([prefix, largeData, separator]); + const ranges = scanSseDataEventRanges(body); + + expect(ranges.previous).toEqual({ start: prefix.byteLength - 1, end: prefix.byteLength + largeData.byteLength }); + expect(ranges.latest).not.toBeNull(); + expect(body.toString('utf8', ranges.latest.start, ranges.latest.end).trim()).toBe('{"type":"interaction.completed"}'); + }); + + test('replays completed JSON with status, headers, and body; canonical key order replays and payload mismatch returns 409', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + res.set('X-Origin', 'first').status(201).json({ handlerCalls }); + } }); + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'json-key') + .send({ payload: { z: 1, a: 2 }, input: 'hello' }); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'json-key') + .send({ input: 'hello', payload: { a: 2, z: 1 } }); + expect(replayed.statusCode).toBe(201); + expect(replayed.text).toBe(first.text); + expect(replayed.headers['x-origin']).toBe('first'); + expect(handlerCalls).toBe(1); + + const conflict = await request(app).post('/v1/responses').set('Idempotency-Key', 'json-key') + .send({ input: 'different' }); + expect(conflict.statusCode).toBe(409); + expect(conflict.body.error.code).toBe('idempotency_key_reused'); + expect(handlerCalls).toBe(1); + }); + + test('rejects an overflowed JSON number rather than replaying it as null', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.json({ ok: true }); + } }); + + const overflow = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'overflow-number') + .set('Content-Type', 'application/json') + .send('{"max_output_tokens":1e400}'); + + expect(overflow.statusCode).toBe(400); + expect(overflow.body.error.code).toBe('invalid_idempotency_payload'); + expect(handlerCalls).toBe(0); + }); + + test('case-insensitive Express paths share replay and conflict scopes', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.json({ handlerCalls }); + } }); + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'case-key') + .send({ input: 'same' }); + const replayed = await request(app).post('/V1/RESPONSES').set('Idempotency-Key', 'case-key') + .send({ input: 'same' }); + expect(replayed.statusCode).toBe(200); + expect(replayed.text).toBe(first.text); + expect(handlerCalls).toBe(1); + + const conflict = await request(app).post('/V1/RESPONSES').set('Idempotency-Key', 'case-key') + .send({ input: 'different' }); + expect(conflict.statusCode).toBe(409); + expect(conflict.body.error.code).toBe('idempotency_key_reused'); + expect(handlerCalls).toBe(1); + }); + + test('authenticated tenants and parent IDs are scoped; interaction aliases share one endpoint scope', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ keys: ['tenant-a-secret', 'tenant-b-secret'], route: (_req, res) => { + handlerCalls += 1; + res.json({ call: handlerCalls }); + } }); + const payload = { input: 'same' }; + const call = (path, tenant, body = payload) => request(app).post(path) + .set('Authorization', `Bearer ${tenant}`) + .set('Idempotency-Key', 'shared-key') + .send(body); + + const tenantA = await call('/v1beta/interactions', 'tenant-a-secret'); + const tenantAReplay = await call('/v1beta/interactions', 'tenant-a-secret'); + const tenantB = await call('/v1beta/interactions', 'tenant-b-secret'); + const alias = await call('/v1/interactions', 'tenant-a-secret'); + const responses = await call('/v1/responses', 'tenant-a-secret'); + const parentOne = await call('/v1beta/interactions', 'tenant-a-secret', { ...payload, previous_interaction_id: 'parent-one' }); + const parentTwo = await call('/v1beta/interactions', 'tenant-a-secret', { ...payload, previous_interaction_id: 'parent-two' }); + + expect(tenantAReplay.text).toBe(tenantA.text); + expect(tenantB.body.call).toBe(2); + expect(alias.text).toBe(tenantA.text); + expect(responses.body.call).toBe(3); + expect(parentOne.body.call).toBe(4); + expect(parentTwo.body.call).toBe(5); + expect(handlerCalls).toBe(5); + }); + + test('in-progress duplicates return retryable 409 without entering the route', async () => { + let handlerCalls = 0; + let release; + let signalStarted; + const pending = new Promise((resolve) => { release = resolve; }); + const started = new Promise((resolve) => { signalStarted = resolve; }); + const { app } = createMiddlewareApp({ route: async (_req, res) => { + handlerCalls += 1; + signalStarted(); + await pending; + res.json({ ok: true }); + } }); + const firstPromise = request(app).post('/v1/responses').set('Idempotency-Key', 'busy') + .send({ input: 'wait' }).then((response) => response); + await started; + const duplicate = await request(app).post('/v1/responses').set('Idempotency-Key', 'busy').send({ input: 'wait' }); + expect(duplicate.statusCode).toBe(409); + expect(duplicate.body.error.code).toBe('idempotency_in_progress'); + expect(duplicate.body.error.retryable).toBe(true); + expect(handlerCalls).toBe(1); + release(); + expect((await firstPromise).statusCode).toBe(200); + }); + + test('definite pre-submission failures replay the original error response', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.set('X-Failure', 'stable').status(422).json({ error: 'invalid input' }); + } }); + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'failed').send({ input: 'bad' }); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'failed').send({ input: 'bad' }); + expect(replayed.statusCode).toBe(422); + expect(replayed.text).toBe(first.text); + expect(replayed.headers['x-failure']).toBe('stable'); + expect(handlerCalls).toBe(1); + }); + + test('unknown requests never re-submit and reconcile only the recorded child/message', async () => { + let handlerCalls = 0; + const observed = []; + const { app } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + const control = req.idempotency; + observed.push({ + reconciling: control.reconciling, + child: control.childSessionId, + message: control.userMessageId, + publicId: control.publicId, + }); + if (handlerCalls === 1) { + control.markSubmitted('child-stable', 'message-stable', 'opencode/kimi-k2.5'); + control.setPublicId('public-id-exposed-before-disconnect'); + res.status(503).json({ error: 'upstream timed out' }); + return; + } + if (handlerCalls === 2) { + res.status(503).json({ error: 'not yet confirmed' }); + return; + } + control.confirmResult('message-stable', 'assistant-stable'); + res.json({ id: control.publicId, answer: 'reconciled' }); + } }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'unknown') + .send({ input: 'same' }); + expect((await send()).statusCode).toBe(503); + const unsuccessful = await send(); + expect(unsuccessful.statusCode).toBe(503); + expect(unsuccessful.body.error).toBe('not yet confirmed'); + const completed = await send(); + expect(completed.statusCode).toBe(200); + expect(completed.body.id).toBe('public-id-exposed-before-disconnect'); + expect((await send()).text).toBe(completed.text); + expect(observed).toEqual([ + { reconciling: false, child: null, message: null, publicId: null }, + { reconciling: true, child: 'child-stable', message: 'message-stable', publicId: 'public-id-exposed-before-disconnect' }, + { reconciling: true, child: 'child-stable', message: 'message-stable', publicId: 'public-id-exposed-before-disconnect' }, + ]); + expect(handlerCalls).toBe(3); + }); + + test.each([ + ['JSON', '/v1/responses'], + ['SSE', '/v1beta/interactions'], + ])('submitted unconfirmed %s 2xx responses stay unknown rather than being cached as completed', async (_label, path) => { + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + if (req.idempotency.reconciling) { + res.status(503).json({ error: { code: 'idempotency_reconciliation_pending' } }); + return; + } + req.idempotency.markSubmitted('unconfirmed-child', 'unconfirmed-user'); + if (path === '/v1/responses') { + res.status(200).json({ answer: 'unconfirmed' }); + return; + } + res.status(200).type('text/event-stream'); + res.write('data: {"type":"interaction.completed","interaction":{"status":"completed"}}\n\n'); + res.end(); + } }); + const send = () => request(app).post(path).set('Idempotency-Key', `unconfirmed-${_label}`) + .send({ input: 'same' }); + + const first = await send(); + expect(first.statusCode).toBe(200); + expect(registry.size).toBe(1); + const pending = await send(); + expect(pending.statusCode).toBe(503); + expect(pending.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(handlerCalls).toBe(2); + }); + + test('Set-Cookie header arrays are copied at finish and replay their original values', async () => { + let handlerCalls = 0; + const cookies = ['first=one; Path=/', 'second=two; Path=/']; + const { app, registry } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.setHeader('Set-Cookie', cookies); + res.once('finish', () => { + cookies[0] = 'mutated=after-finish; Path=/'; + cookies.push('late=mutation; Path=/'); + }); + res.status(201).send('stable body'); + } }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'mutable-set-cookie') + .send({ input: 'same' }); + + const first = await send(); + const scopeId = idempotencyDigest(JSON.stringify([ + 'anonymous', + '/v1/responses', + '', + idempotencyDigest('mutable-set-cookie'), + ])); + const record = registry.get(scopeId); + const replayed = await send(); + expect(first.statusCode).toBe(201); + expect(record.wireResponse.headers['set-cookie']).toEqual(['first=one; Path=/', 'second=two; Path=/']); + expect(replayed.headers['set-cookie']).toEqual(['first=one; Path=/', 'second=two; Path=/']); + expect(handlerCalls).toBe(1); + }); + + test('__proto__ response header arrays survive capture and replay', async () => { + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.setHeader('__proto__', ['first=one', 'second=two']); + res.status(201).send('stable body'); + } }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'proto-header-replay') + .send({ input: 'same' }); + const rawValues = (response) => { + const result = []; + const rawHeaders = response.res.rawHeaders; + for (let index = 0; index < rawHeaders.length; index += 2) { + if (rawHeaders[index].toLowerCase() === '__proto__') result.push(rawHeaders[index + 1]); + } + return result; + }; + + const first = await send(); + const scopeId = idempotencyDigest(JSON.stringify([ + 'anonymous', + '/v1/responses', + '', + idempotencyDigest('proto-header-replay'), + ])); + const record = registry.get(scopeId); + const replayed = await send(); + + expect(first.statusCode).toBe(201); + expect(rawValues(first)).toEqual(['first=one', 'second=two']); + expect(Object.hasOwn(record.wireResponse.headers, '__proto__')).toBe(true); + expect(record.wireResponse.headers['__proto__']).toEqual(['first=one', 'second=two']); + expect(rawValues(replayed)).toEqual(['first=one', 'second=two']); + expect(replayed.text).toBe(first.text); + expect(handlerCalls).toBe(1); + }); + + test('a tiny body with a huge response header is delivered but not retained past the cache budget', async () => { + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 1_024 }, + route: (_req, res) => { + handlerCalls += 1; + res.setHeader('X-Huge', 'h'.repeat(2_048)); + res.status(201).send('x'); + }, + }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'huge-header-budget') + .send({ input: 'same' }); + + expect((await send()).statusCode).toBe(201); + expect(registry.size).toBe(0); + expect(registry.retainedResponseBytes).toBe(0); + expect(registry.pendingResponseBytes).toBe(0); + expect((await send()).statusCode).toBe(201); + expect(handlerCalls).toBe(2); + }); + + test('an oversized __proto__ header array cannot evade capture accounting', async () => { + let handlerCalls = 0; + const headerValues = ['a'.repeat(1_024), 'b'.repeat(1_024)]; + const { app, registry } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 1_024 }, + route: (_req, res) => { + handlerCalls += 1; + res.setHeader('__proto__', headerValues); + res.status(201).send('x'); + }, + }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'proto-header-budget') + .send({ input: 'same' }); + const rawValues = (response) => { + const result = []; + const rawHeaders = response.res.rawHeaders; + for (let index = 0; index < rawHeaders.length; index += 2) { + if (rawHeaders[index].toLowerCase() === '__proto__') result.push(rawHeaders[index + 1]); + } + return result; + }; + + const first = await send(); + expect(first.statusCode).toBe(201); + expect(rawValues(first)).toEqual(headerValues); + expect(registry.size).toBe(0); + expect(registry.retainedResponseBytes).toBe(0); + expect(registry.pendingResponseBytes).toBe(0); + expect((await send()).statusCode).toBe(201); + expect(handlerCalls).toBe(2); + }); + + test('SSE completed replay preserves the exact event bytes and relevant headers', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.status(200).set('Content-Type', 'text/event-stream').set('X-Stream', 'stable'); + res.write('data: {"type":"interaction.created"}\n\n'); + res.write('data: {"type":"interaction.completed","interaction":{"id":"one"}}\n\n'); + res.end(); + } }); + const first = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'stream-key').send({ input: 'hello' }); + const replayed = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'stream-key').send({ input: 'hello' }); + expect(replayed.statusCode).toBe(200); + expect(replayed.text).toBe(first.text); + expect(replayed.headers['content-type']).toContain('text/event-stream'); + expect(replayed.headers['x-stream']).toBe('stable'); + expect(handlerCalls).toBe(1); + }); + + test('confirmed terminal Interactions SSE failures are retained as failed and replay byte-for-byte', async () => { + let handlerCalls = 0; + const key = 'confirmed-error-stream'; + const { app, registry } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + const control = req.idempotency; + if (!control.reconciling) control.markSubmitted('error-child', 'error-user-message'); + else control.confirmResult('error-user-message', 'error-assistant-message'); + res.status(200).set('Content-Type', 'text/event-stream'); + res.write('data: {"type":"interaction.created","interaction":{"id":"intr-error"}}\n\n'); + res.write('data: {"type":"error","error":{"message":"terminal failure"}}\n\n'); + res.end(); + } }); + const scopeId = idempotencyDigest(JSON.stringify([ + 'anonymous', + '/v1/interactions', + '', + idempotencyDigest(key), + ])); + const send = () => request(app).post('/v1beta/interactions').set('Idempotency-Key', key) + .send({ input: 'hello' }); + + const uncertain = await send(); + expect(registry.get(scopeId)?.state).toBe('unknown'); + const failed = await send(); + expect(failed.statusCode).toBe(200); + expect(failed.headers['content-type']).toContain('text/event-stream'); + expect(registry.get(scopeId)?.state).toBe('failed'); + expect(registry.get(scopeId)?.wireResponse?.body.toString()).toBe(failed.text); + const replayed = await send(); + expect(replayed.text).toBe(failed.text); + expect(uncertain.statusCode).toBe(200); + expect(handlerCalls).toBe(2); + }); + + test('a disconnected partial stream becomes unknown and retries do not launch another run', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + if (!req.idempotency.reconciling) { + req.idempotency.markSubmitted('stream-child', 'stream-message'); + res.set('Content-Type', 'text/event-stream'); + res.write('data: {"type":"response.created"}\n\n'); + res.socket.destroy(); + return; + } + res.status(503).json({ error: { code: 'idempotency_reconciliation_pending' } }); + } }); + await request(app).post('/v1/responses').set('Idempotency-Key', 'disconnected-stream') + .send({ input: 'partial' }).then(() => null, () => null); + + const retry = await request(app).post('/v1/responses').set('Idempotency-Key', 'disconnected-stream') + .send({ input: 'partial' }); + expect(retry.statusCode).toBe(503); + expect(retry.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(handlerCalls).toBe(2); + }); + + test('pre-submit disconnect abandons its record and delayed continuation cannot prompt', async () => { + let handlerCalls = 0; + let promptCalls = 0; + let refusedDelayedPrompt = false; + let signalStarted; + let signalClosed; + let signalContinued; + let resumeDelayedRoute; + const started = new Promise((resolve) => { signalStarted = resolve; }); + const closed = new Promise((resolve) => { signalClosed = resolve; }); + const continued = new Promise((resolve) => { signalContinued = resolve; }); + const delayedRoute = new Promise((resolve) => { resumeDelayedRoute = resolve; }); + const { app } = createMiddlewareApp({ route: async (req, res) => { + handlerCalls += 1; + if (handlerCalls === 1) { + res.once('close', signalClosed); + signalStarted(); + await delayedRoute; + try { + req.idempotency.markSubmitted('abandoned-child', 'abandoned-message'); + promptCalls += 1; + } catch { + refusedDelayedPrompt = true; + } + signalContinued(); + return; + } + req.idempotency.markSubmitted('fresh-child', 'fresh-message'); + promptCalls += 1; + res.json({ promptCalls }); + } }); + + const aborted = await startAbortedRequest(app, '/v1/responses', 'pre-submit-close', { input: 'same' }); + await started; + aborted.abort(); + await closed; + resumeDelayedRoute(); + await continued; + + expect(refusedDelayedPrompt).toBe(true); + expect(promptCalls).toBe(0); + const fresh = await request(app).post('/v1/responses').set('Idempotency-Key', 'pre-submit-close') + .send({ input: 'same' }); + expect(fresh.statusCode).toBe(200); + expect(fresh.body.promptCalls).toBe(1); + expect(handlerCalls).toBe(2); + expect(promptCalls).toBe(1); + await aborted.closeServer(); + }); + + test('submitted disconnect remains unknown and cannot submit again', async () => { + let handlerCalls = 0; + let promptCalls = 0; + let signalStarted; + let signalClosed; + const started = new Promise((resolve) => { signalStarted = resolve; }); + const closed = new Promise((resolve) => { signalClosed = resolve; }); + const { app } = createMiddlewareApp({ route: async (req, res) => { + handlerCalls += 1; + if (req.idempotency.reconciling) { + res.status(503).json({ error: { code: 'idempotency_reconciliation_pending' } }); + return; + } + req.idempotency.markSubmitted('submitted-child', 'submitted-message'); + promptCalls += 1; + res.once('close', signalClosed); + signalStarted(); + await new Promise((resolve) => res.once('close', resolve)); + } }); + + const aborted = await startAbortedRequest(app, '/v1/responses', 'submitted-close', { input: 'same' }); + await started; + aborted.abort(); + await closed; + + const retry = await request(app).post('/v1/responses').set('Idempotency-Key', 'submitted-close') + .send({ input: 'same' }); + expect(retry.statusCode).toBe(503); + expect(retry.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(handlerCalls).toBe(2); + expect(promptCalls).toBe(1); + await aborted.closeServer(); + }); + + test('a disconnected original lease cannot mutate or prompt after reconciliation is claimed', async () => { + let handlerCalls = 0; + let promptCalls = 0; + let signalOriginalStarted; + let signalOriginalClosed; + let signalRetryClaimed; + let signalOriginalContinued; + let resumeOriginal; + let finishRetry; + const originalStarted = new Promise((resolve) => { signalOriginalStarted = resolve; }); + const originalClosed = new Promise((resolve) => { signalOriginalClosed = resolve; }); + const retryClaimed = new Promise((resolve) => { signalRetryClaimed = resolve; }); + const originalContinued = new Promise((resolve) => { signalOriginalContinued = resolve; }); + const originalGate = new Promise((resolve) => { resumeOriginal = resolve; }); + const retryGate = new Promise((resolve) => { finishRetry = resolve; }); + let staleMutationRejected = false; + const { app } = createMiddlewareApp({ route: async (req, res) => { + handlerCalls += 1; + const control = req.idempotency; + if (handlerCalls === 1) { + control.markSubmitted('leased-child', 'leased-message'); + promptCalls += 1; + res.once('close', signalOriginalClosed); + signalOriginalStarted(); + await originalGate; + try { + control.setPublicId('stale-public-id'); + } catch { + staleMutationRejected = true; + } + try { + control.markSubmitted('stale-child', 'stale-message'); + promptCalls += 1; + } catch { + staleMutationRejected = true; + } + signalOriginalContinued(); + return; + } + expect(control.reconciling).toBe(true); + expect(control.publicId).toBeNull(); + signalRetryClaimed(); + await retryGate; + res.json({ answer: 'reconciled' }); + } }); + + const aborted = await startAbortedRequest(app, '/v1/responses', 'lease-race', { input: 'same' }); + await originalStarted; + aborted.abort(); + await originalClosed; + const retryPromise = request(app).post('/v1/responses').set('Idempotency-Key', 'lease-race') + .send({ input: 'same' }).then((response) => response); + await retryClaimed; + resumeOriginal(); + await originalContinued; + expect(staleMutationRejected).toBe(true); + expect(promptCalls).toBe(1); + finishRetry(); + expect((await retryPromise).statusCode).toBe(200); + expect(handlerCalls).toBe(2); + expect(promptCalls).toBe(1); + await aborted.closeServer(); + }); + + test('writes after close neither reserve capture bytes nor change the unknown lease', async () => { + let signalLateWrites; + const lateWrites = new Promise((resolve) => { signalLateWrites = resolve; }); + let handlerCalls = 0; + let pendingAfterLateWrites = -1; + const { app, registry } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + if (req.idempotency.reconciling) { + res.status(503).json({ error: 'still unknown' }); + return; + } + req.idempotency.markSubmitted('late-write-child', 'late-write-message'); + res.once('close', () => { + try { res.write('late write'); } catch {} + try { res.end('late end'); } catch {} + // A late finish signal must not assemble chunks retained by + // the closed response closure. + res.emit('finish'); + pendingAfterLateWrites = registry.pendingResponseBytes; + signalLateWrites(); + }); + res.write('partial'); + res.socket.destroy(); + } }); + const assemblyRequests = []; + const reserveAssemblyBytes = registry.reserveAssemblyBytes.bind(registry); + registry.reserveAssemblyBytes = (record, generation, bytes) => { + assemblyRequests.push(bytes); + return reserveAssemblyBytes(record, generation, bytes); + }; + + const aborted = await startAbortedRequest(app, '/v1/responses', 'late-capture', { input: 'same' }); + await lateWrites; + expect(pendingAfterLateWrites).toBe(0); + expect(assemblyRequests).toEqual([0]); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + const retry = await request(app).post('/v1/responses').set('Idempotency-Key', 'late-capture') + .send({ input: 'same' }); + expect(retry.statusCode).toBe(503); + expect(handlerCalls).toBe(2); + await aborted.closeServer(); + }); + + test('many tiny response writes hit the chunk bound and cannot be cached as a replay', async () => { + let handlerCalls = 0; + let promptCalls = 0; + let reconciled = false; + const { app, registry } = createMiddlewareApp({ route: (req, res) => { + handlerCalls += 1; + if (!req.idempotency.reconciling) { + promptCalls += 1; + req.idempotency.markSubmitted('many-chunks-child', 'many-chunks-message'); + for (let index = 0; index < 1_100; index += 1) res.write('x'); + res.end(); + return; + } + reconciled = req.idempotency.reconciling; + res.status(503).json({ error: 'reconciliation pending' }); + } }); + + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'many-tiny-chunks').send({ input: 'same' }); + expect((await send()).statusCode).toBe(200); + expect(registry.pendingResponseBytes).toBe(0); + + const retry = await send(); + expect(retry.statusCode).toBe(503); + expect(retry.body.error).toBe('reconciliation pending'); + expect(handlerCalls).toBe(2); + expect(promptCalls).toBe(1); + expect(reconciled).toBe(true); + }); + + test('one-byte chunks charge conservative object overhead and balance pending accounting on overflow', async () => { + let handlerCalls = 0; + const { app, registry } = createMiddlewareApp({ + registryOptions: { + maxCachedBytes: 1_024, + maxCachedBytesPerTenant: 1_024, + maxResponseBytes: 4_096, + }, + route: (req, res) => { + handlerCalls += 1; + if (req.idempotency.reconciling) { + res.status(503).json({ error: 'reconciliation pending' }); + return; + } + req.idempotency.markSubmitted('tiny-chunk-child', 'tiny-chunk-message'); + for (let index = 0; index < 100; index += 1) res.write('x'); + res.end(); + }, + }); + const reservations = []; + const reserveCaptureBytes = registry.reserveCaptureBytes.bind(registry); + registry.reserveCaptureBytes = (record, generation, bytes, overhead) => { + const accepted = reserveCaptureBytes(record, generation, bytes, overhead); + reservations.push({ bytes, overhead, accepted, pending: registry.pendingResponseBytes }); + return accepted; + }; + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'tiny-byte-budget') + .send({ input: 'same' }); + + expect((await send()).statusCode).toBe(200); + expect(reservations[0]).toMatchObject({ bytes: 1, overhead: 128, accepted: true, pending: 129 }); + expect(reservations.filter(({ accepted }) => accepted)).toHaveLength(7); + expect(reservations.find(({ accepted }) => !accepted)).toMatchObject({ bytes: 1, overhead: 128, pending: 903 }); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + expect(registry.retainedResponseBytes).toBe(0); + expect(registry.getTenantRetainedResponseBytes('anonymous')).toBe(0); + + const retry = await send(); + expect(retry.statusCode).toBe(503); + expect(handlerCalls).toBe(2); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + }); + + test.each([ + ['Responses', '/v1/responses', 'finish-tool', { finish: 'tool', parts: [] }], + ['Responses', '/v1/responses', 'active-tool', { + finish: 'stop', + parts: [{ type: 'tool', id: 'active-call', tool: 'websearch', state: { status: 'running' } }], + }], + ['Responses', '/v1/responses', 'incomplete-no-finish', { parts: [] }], + ['Interactions', '/v1beta/interactions', 'finish-tool-interaction', { finish: 'tool', parts: [] }], + ['Interactions', '/v1beta/interactions', 'active-tool-interaction', { + finish: 'stop', + parts: [{ type: 'tool', id: 'active-call', tool: 'websearch', state: { status: 'pending' } }], + }], + ['Interactions', '/v1beta/interactions', 'incomplete-no-finish-interaction', { parts: [] }], + ])('%s reconciliation keeps an assistant error pending while the exact-parent turn is incomplete (%s)', async (_name, path, key, snapshot) => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 100 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'uncertain tool turn' }; + const first = await request(app).post(path).set('Idempotency-Key', key).send(payload); + expect(first.statusCode).toBeGreaterThanOrEqual(400); + const promptCallsAfterInitialRequest = sdkMocks.sessionPrompt.mock.calls.length; + const sessionCreatesAfterInitialRequest = sdkMocks.sessionCreate.mock.calls.length; + + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: `assistant-awaiting-tool-${sessionPath.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + finish: snapshot.finish, + error: { name: 'ProviderError', message: 'tool turn still active' }, + }, + parts: snapshot.parts, + }]; + }); + sdkMocks.sessionMessages.mockClear(); + + const retry = () => request(app).post(path).set('Idempotency-Key', key).send(payload); + const pendingOne = await retry(); + const pendingTwo = await retry(); + expect(pendingOne.statusCode).toBe(503); + expect(pendingOne.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(pendingTwo.statusCode).toBe(503); + expect(pendingTwo.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(sdkMocks.sessionMessages).toHaveBeenCalled(); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(promptCallsAfterInitialRequest); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(sessionCreatesAfterInitialRequest); + }); + + test('capture accounting includes the temporary assembly buffer and retains the exact body', async () => { + const { app, registry } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 1_000, maxCachedBytesPerTenant: 1_000, maxResponseBytes: 11 }, + route: (_req, res) => res.status(201).send('01234567890'), + }); + const reservationPeaks = []; + const reserveCaptureBytes = registry.reserveCaptureBytes.bind(registry); + const reserveAssemblyBytes = registry.reserveAssemblyBytes.bind(registry); + registry.reserveCaptureBytes = (...args) => { + const accepted = reserveCaptureBytes(...args); + reservationPeaks.push(registry.pendingResponseBytes); + return accepted; + }; + registry.reserveAssemblyBytes = (...args) => { + const accepted = reserveAssemblyBytes(...args); + reservationPeaks.push(registry.pendingResponseBytes); + return accepted; + }; + + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'assembly-budget') + .send({ input: 'same' }); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'assembly-budget') + .send({ input: 'same' }); + expect(first.statusCode).toBe(201); + expect(first.text).toBe('01234567890'); + expect(replayed.text).toBe(first.text); + expect(reservationPeaks.slice(0, 2)).toEqual([139, 150]); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + expect(registry.retainedResponseBytes).toBeGreaterThan(11); + expect(registry.getTenantRetainedResponseBytes('anonymous')).toBe(registry.retainedResponseBytes); + }); + + test('capture overhead across multiple chunks releases exactly through assembly and completion', async () => { + const { app, registry } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 5_000, maxResponseBytes: 100 }, + route: (_req, res) => { + res.status(201); + for (let index = 0; index < 12; index += 1) res.write('x'); + res.end(); + }, + }); + const capturePeaks = []; + const assemblyPeaks = []; + const reserveCaptureBytes = registry.reserveCaptureBytes.bind(registry); + const reserveAssemblyBytes = registry.reserveAssemblyBytes.bind(registry); + registry.reserveCaptureBytes = (...args) => { + const accepted = reserveCaptureBytes(...args); + capturePeaks.push(registry.pendingResponseBytes); + return accepted; + }; + registry.reserveAssemblyBytes = (...args) => { + const accepted = reserveAssemblyBytes(...args); + assemblyPeaks.push(registry.pendingResponseBytes); + return accepted; + }; + + const response = await request(app).post('/v1/responses').set('Idempotency-Key', 'multi-chunk-overhead') + .send({ input: 'same' }); + expect(response.statusCode).toBe(201); + expect(response.text).toBe('x'.repeat(12)); + expect(capturePeaks).toEqual(Array.from({ length: 12 }, (_, index) => (index + 1) * 129)); + expect(assemblyPeaks).toHaveLength(2); + expect(assemblyPeaks[0]).toBe(12 * 129 + 12); + expect(assemblyPeaks[1]).toBeGreaterThan(0); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + expect(registry.retainedResponseBytes).toBeGreaterThan(12); + expect(registry.getTenantRetainedResponseBytes('anonymous')).toBe(registry.retainedResponseBytes); + }); + + test('pre-submit capture overflow abandons the key so a retry starts fresh', async () => { + let handlerCalls = 0; + const reconcilingStates = []; + const { app, registry } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 0 }, + route: (req, res) => { + handlerCalls += 1; + reconcilingStates.push(req.idempotency.reconciling); + if (handlerCalls === 1) { + res.status(422).json({ error: 'cannot capture this before submit' }); + return; + } + res.status(204).end(); + }, + }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'pre-submit-overflow') + .send({ input: 'same' }); + expect((await send()).statusCode).toBe(422); + expect(registry.pendingResponseBytes).toBe(0); + expect(registry.getTenantPendingResponseBytes('anonymous')).toBe(0); + expect((await send()).statusCode).toBe(204); + expect(handlerCalls).toBe(2); + expect(reconcilingStates).toEqual([false, false]); + }); + + test('submitted oversized responses remain unknown and cannot prompt again', async () => { + let handlerCalls = 0; + let promptCalls = 0; + const reconcilingStates = []; + const { app } = createMiddlewareApp({ + registryOptions: { maxCachedBytes: 128, maxResponseBytes: 4 }, + route: (req, res) => { + handlerCalls += 1; + reconcilingStates.push(req.idempotency.reconciling); + if (req.idempotency.reconciling) { + res.status(503).json({ error: 'reconciliation required' }); + return; + } + req.idempotency.markSubmitted('oversized-child', 'oversized-message'); + promptCalls += 1; + res.json({ answer: 'too large to capture' }); + }, + }); + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'submitted-overflow') + .send({ input: 'same' }); + expect((await send()).statusCode).toBe(200); + expect((await send()).statusCode).toBe(503); + expect(handlerCalls).toBe(2); + expect(promptCalls).toBe(1); + expect(reconcilingStates).toEqual([false, true]); + }); + + test('empty and oversized Idempotency-Key values are rejected safely', async () => { + let handlerCalls = 0; + const { app } = createMiddlewareApp({ route: (_req, res) => { + handlerCalls += 1; + res.json({ ok: true }); + } }); + const empty = await request(app).post('/v1/responses').set('Idempotency-Key', ' ').send({ input: 'x' }); + const oversized = await request(app).post('/v1/responses').set('Idempotency-Key', 'x'.repeat(257)).send({ input: 'x' }); + expect(empty.statusCode).toBe(400); + expect(oversized.statusCode).toBe(400); + expect(empty.body.error.code).toBe('invalid_idempotency_key'); + expect(handlerCalls).toBe(0); + }); +}); + +describe('Responses and Interactions idempotent routes', () => { + beforeEach(() => { + jest.clearAllMocks(); + promptBySession.clear(); + sessionSequence = 0; + forkSequence = 0; + assistantSequence = 0; + toolObservationCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => ({ data: { id: `idem-session-${++sessionSequence}` } })); + sdkMocks.sessionFork.mockImplementation(async () => ({ data: { id: `idem-fork-${++forkSequence}` } })); + sdkMocks.sessionDelete.mockImplementation(async () => ({})); + sdkMocks.sessionPrompt.mockImplementation(async (args) => { + const sessionId = args?.path?.id; + promptBySession.set(sessionId, structuredClone(args)); + const assistantId = `assistant-${++assistantSequence}`; + return { + data: { + info: { id: assistantId, role: 'assistant', parentID: args?.body?.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'idempotent answer' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => { + const prompt = promptBySession.get(path.id); + const parts = [{ type: 'text', text: 'idempotent answer' }]; + if (prompt?.body?.tools?.websearch === true) { + toolObservationCount += 1; + parts.push({ + type: 'tool', + id: 'search-tool-call', + tool: 'websearch', + state: { status: 'completed', input: { query: 'test query' }, output: 'tool result' }, + }); + } + return [{ + info: { + id: `assistant-poll-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts, + }]; + }); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ stream: (async function* () {})() })); + }); + + test('completed Responses JSON replays the original ID and result without prompting again', async () => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'answer this' }; + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'responses-json').send(payload); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'responses-json').send(payload); + expect(first.statusCode).toBe(200); + expect(replayed.statusCode).toBe(200); + expect(replayed.text).toBe(first.text); + expect(replayed.body.id).toBe(first.body.id); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test('expired response state does not delete a child pinned by an unknown idempotent outcome', async () => { + const { app, ctx } = createRouteApp(); + const largeText = 'x'.repeat(DEFAULT_IDEMPOTENCY_MAX_RESPONSE_BYTES + 1024); + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { + id: 'oversized-assistant', + role: 'assistant', + parentID: args.body.messageID, + finish: 'stop', + }, + parts: [{ type: 'text', text: largeText }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementationOnce(async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: 'oversized-assistant-poll', + role: 'assistant', + parentID: prompt.body.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: largeText }], + }]; + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'large response for retention test' }; + const first = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'oversized-child-retention') + .send(payload); + + expect(first.statusCode).toBe(200); + const state = ctx.responseState.get(first.body.id); + expect(state).toBeDefined(); + state.expiresAt = Date.now() - 1; + await ctx.sweepResponseState(); + expect(ctx.responseState.has(first.body.id)).toBe(false); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: state.sessionId } }); + + const reconciled = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'oversized-child-retention') + .send(payload); + expect(reconciled.statusCode).toBe(200); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(2); + + const reconciledState = ctx.responseState.get(reconciled.body.id); + expect(reconciledState).toBeDefined(); + reconciledState.expiresAt = Date.now() - 1; + await ctx.sweepResponseState(); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: state.sessionId } }); + }); + + test('keyed non-stream prompt with tool finish and completion time polls for the complete exact turn', async () => { + const app = createRouteApp().app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { + id: 'assistant-intermediate-tool-turn', + role: 'assistant', + parentID: args.body.messageID, + finish: 'tool', + time: { completed: Date.now() }, + }, + parts: [{ type: 'text', text: 'unsafe partial tool-turn text' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementationOnce(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-complete-after-tool', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'complete exact assistant result' }], + }]; + }); + + const response = await request(app).post('/v1/responses').set('Idempotency-Key', 'tool-finish-prompt') + .send({ model: 'opencode/kimi-k2.5', input: 'finish the tool turn' }); + expect(response.statusCode).toBe(200); + expect(JSON.stringify(response.body)).toContain('complete exact assistant result'); + expect(JSON.stringify(response.body)).not.toContain('unsafe partial tool-turn text'); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test('a keyed no-ID prompt snapshot cannot complete JSON before an exact terminal poll', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 50 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'unsafe no-ID prompt snapshot' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-poll-still-incomplete', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + }, + parts: [{ type: 'text', text: 'still incomplete' }], + }]; + }); + + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'no-id-prompt-snapshot') + .send({ model: 'opencode/kimi-k2.5', input: 'wait for an exact terminal turn' }); + const first = await send(); + expect(first.statusCode).toBe(503); + expect(first.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(first.text).not.toContain('unsafe no-ID prompt snapshot'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + const retry = await send(); + expect(retry.statusCode).toBe(503); + expect(retry.headers['content-type']).toContain('application/json'); + expect(retry.body.error).toMatchObject({ code: 'idempotency_reconciliation_pending', retryable: true }); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test('a complete-looking prompt snapshot is not confirmation without a complete exact-turn poll', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 50 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { + id: 'assistant-prompt-only-terminal', + role: 'assistant', + parentID: args.body.messageID, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'unpolled prompt snapshot answer' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-poll-incomplete', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + }, + parts: [{ type: 'text', text: 'unconfirmed poll snapshot' }], + }]; + }); + + const response = await request(app).post('/v1/responses').set('Idempotency-Key', 'prompt-snapshot-not-confirmed') + .send({ model: 'opencode/kimi-k2.5', input: 'confirm by polling' }); + expect(response.statusCode).toBe(503); + expect(response.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(response.text).not.toContain('unpolled prompt snapshot answer'); + expect(sdkMocks.sessionMessages).toHaveBeenCalled(); + }); + + test('keyed non-stream prompt text with an error and no finish stays unknown until an exact terminal poll', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 80 }).app; + let terminalVisible = false; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { + id: 'assistant-unconfirmed-prompt-result', + role: 'assistant', + parentID: args.body.messageID, + error: { name: 'ProviderError', message: 'prompt result is not terminal' }, + }, + parts: [{ type: 'text', text: 'unsafe partial prompt text' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + const exactTurn = { + info: { + id: terminalVisible ? 'assistant-later-terminal' : 'assistant-still-incomplete', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + ...(terminalVisible ? { finish: 'stop' } : {}), + }, + parts: [{ type: 'text', text: terminalVisible ? 'reconciled terminal answer' : 'still partial' }], + }; + // Put a different turn last so the poll must filter by the + // submitted user message ID rather than taking the newest assistant. + const unrelatedTurn = { + info: { + id: 'assistant-unrelated-terminal', + role: 'assistant', + parentID: 'another-user-message', + sessionID: sessionPath.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'wrong unrelated answer' }], + }; + return [exactTurn, unrelatedTurn]; + }); + + const payload = { model: 'opencode/kimi-k2.5', input: 'confirm this turn' }; + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'prompt-unconfirmed') + .send(payload); + const incomplete = await send(); + expect(incomplete.statusCode).toBe(503); + expect(incomplete.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(incomplete.text).not.toContain('unsafe partial prompt text'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + const submittedMessageId = sdkMocks.sessionPrompt.mock.calls[0][0].body.messageID; + expect(typeof submittedMessageId).toBe('string'); + + terminalVisible = true; + const reconciled = await send(); + expect(reconciled.statusCode).toBe(200); + expect(JSON.stringify(reconciled.body)).toContain('reconciled terminal answer'); + expect(JSON.stringify(reconciled.body)).not.toContain('unsafe partial prompt text'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledWith(expect.objectContaining({ path: { id: 'idem-session-1' } })); + + const replayed = await send(); + expect(replayed.text).toBe(reconciled.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test.each([ + ['Responses', '/v1/responses', 'terminal-response-error'], + ['Interactions', '/v1beta/interactions', 'terminal-interaction-error'], + ])('%s complete assistant errors are cached and replayed without another prompt', async (_name, path, key) => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'produce a terminal error' }; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { id: 'assistant-prompt-error', role: 'assistant', parentID: args.body.messageID }, + parts: [], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementationOnce(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-terminal-error', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + finish: 'stop', + error: { name: 'ProviderError', message: 'terminal assistant failure' }, + }, + parts: [], + }]; + }); + + const send = () => request(app).post(path).set('Idempotency-Key', key).send(payload); + const first = await send(); + const replayed = await send(); + expect(first.statusCode).toBeGreaterThanOrEqual(400); + expect(replayed.statusCode).toBe(first.statusCode); + expect(replayed.text).toBe(first.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test('same continuation key reuses one child while different keys fork separately from the same parent', async () => { + const app = createRouteApp().app; + const seed = await request(app).post('/v1/responses').send({ model: 'opencode/kimi-k2.5', input: 'seed' }); + const continuation = { input: 'follow up', previous_response_id: seed.body.id }; + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'follow-up-one').send(continuation); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'follow-up-one').send(continuation); + const separateKey = await request(app).post('/v1/responses').set('Idempotency-Key', 'follow-up-two').send(continuation); + + expect(first.statusCode).toBe(200); + expect(replayed.text).toBe(first.text); + expect(separateKey.statusCode).toBe(200); + expect(sdkMocks.sessionFork).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionFork.mock.calls.map(([args]) => args.path.id)).toEqual([ + 'idem-session-1', + 'idem-session-1', + ]); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(3); + const mismatch = await request(app).post('/v1/responses').set('Idempotency-Key', 'follow-up-one') + .send({ ...continuation, input: 'different' }); + expect(mismatch.statusCode).toBe(409); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(3); + }); + + test('Interactions SSE is replayed byte-for-byte through either endpoint alias', async () => { + const app = createRouteApp().app; + const payload = { + model: 'opencode/kimi-k2.5', + input: 'search this', + stream: true, + tools: [{ type: 'google_search' }], + }; + const first = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'interaction-stream').send(payload); + const replayed = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'interaction-stream').send(payload); + expect(first.statusCode).toBe(200); + expect(replayed.text).toBe(first.text); + expect(first.text).toContain('interaction.completed'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(toolObservationCount).toBe(1); + + const alias = await request(app).post('/v1/interactions').set('Idempotency-Key', 'interaction-stream').send(payload); + expect(alias.statusCode).toBe(200); + expect(alias.text).toBe(first.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(toolObservationCount).toBe(1); + }); + + test('keyed Responses streaming does not complete from a terminal event without an assistant ID', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 50 }).app; + sdkMocks.sessionPrompt.mockReset(); + sdkMocks.sessionPrompt.mockImplementation(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + return { + data: { + info: { id: 'assistant-prompt-snapshot', role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'unsafe prompt snapshot text' }], + }, + }; + }); + sdkMocks.sessionMessages.mockReset(); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-poll-incomplete-stream', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + }, + parts: [{ type: 'text', text: 'unsafe partial poll text' }], + }]; + }); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + await Promise.resolve(); + const [sessionID, prompt] = [...promptBySession.entries()][0] ?? []; + yield { + type: 'message.updated', + properties: { + info: { + role: 'assistant', + parentID: prompt?.body?.messageID, + finish: 'stop', + sessionID, + parts: [{ id: 'idless-terminal-part', type: 'text', text: 'unsafe idless event text' }], + }, + }, + }; + })(), + })); + + const response = await request(app).post('/v1/responses').set('Idempotency-Key', 'idless-stream-terminal') + .send({ model: 'opencode/kimi-k2.5', input: 'wait for stream confirmation', stream: true }); + expect(response.statusCode).toBe(200); + expect(response.text).toContain('response.failed'); + expect(response.text).not.toContain('response.completed'); + expect(response.text).not.toContain('unsafe prompt snapshot text'); + expect(response.text).not.toContain('unsafe partial poll text'); + expect(response.text).not.toContain('unsafe idless event text'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test('keyed Responses streaming reconciliation stays SSE-pending until a later exact poll completes', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 80 }).app; + let terminalVisible = false; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: terminalVisible ? 'assistant-later-responses-terminal' : 'assistant-incomplete-responses-poll', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + ...(terminalVisible ? { finish: 'stop' } : {}), + }, + parts: [{ type: 'text', text: terminalVisible ? 'reconciled Responses answer' : 'unsafe incomplete poll text' }], + }]; + }); + + const payload = { model: 'opencode/kimi-k2.5', input: 'reconcile this stream', stream: true }; + const send = () => request(app).post('/v1/responses').set('Idempotency-Key', 'responses-stream-pending') + .send(payload); + const parseData = (text) => text.split(/\r?\n/) + .filter((line) => line.startsWith('data: ')) + .map((line) => line.slice('data: '.length)) + .map((data) => data === '[DONE]' ? data : JSON.parse(data)); + + const initial = await send(); + expect(initial.statusCode).toBe(200); + expect(initial.headers['content-type']).toContain('text/event-stream'); + expect(initial.text).toContain('response.failed'); + expect(initial.text).toContain('data: [DONE]'); + + const pending = await send(); + const pendingEvents = parseData(pending.text); + expect(pending.statusCode).toBe(200); + expect(pending.headers['content-type']).toContain('text/event-stream'); + expect(pendingEvents.at(-1)).toBe('[DONE]'); + expect(pendingEvents.some((event) => event?.type === 'response.failed')).toBe(true); + expect(pendingEvents.some((event) => event?.type === 'response.completed')).toBe(false); + expect(pending.text).not.toContain('unsafe incomplete poll text'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + const pollsBeforeCompletion = sdkMocks.sessionMessages.mock.calls.length; + expect(pollsBeforeCompletion).toBeGreaterThan(0); + + terminalVisible = true; + const completed = await send(); + expect(completed.statusCode).toBe(200); + expect(completed.headers['content-type']).toContain('text/event-stream'); + expect(completed.text).toContain('response.completed'); + expect(completed.text).toContain('reconciled Responses answer'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages.mock.calls.length).toBeGreaterThan(pollsBeforeCompletion); + + const pollsAfterCompletion = sdkMocks.sessionMessages.mock.calls.length; + const replayed = await send(); + expect(replayed.text).toBe(completed.text); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(pollsAfterCompletion); + }); + + test('keyed Interactions streaming reconciliation stays SSE-pending until a later exact poll completes', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 80 }).app; + let terminalVisible = false; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: terminalVisible ? 'assistant-later-interactions-terminal' : 'assistant-incomplete-interactions-poll', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + ...(terminalVisible ? { finish: 'stop' } : {}), + }, + parts: [{ type: 'text', text: terminalVisible ? 'reconciled Interactions answer' : 'unsafe incomplete poll text' }], + }]; + }); + + const payload = { model: 'opencode/kimi-k2.5', input: 'reconcile this interaction', stream: true }; + const send = () => request(app).post('/v1beta/interactions').set('Idempotency-Key', 'interactions-stream-pending') + .send(payload); + const parseEvents = (text) => text.split(/\r?\n/) + .filter((line) => line.startsWith('data: ')) + .map((line) => JSON.parse(line.slice('data: '.length))); + + const initial = await send(); + const initialEvents = parseEvents(initial.text); + expect(initial.statusCode).toBe(200); + expect(initial.headers['content-type']).toContain('text/event-stream'); + expect(initialEvents.map((event) => event.type)).toEqual(['interaction.created', 'error']); + expect(initial.text).not.toContain('interaction.completed'); + + const pending = await send(); + const pendingEvents = parseEvents(pending.text); + expect(pending.statusCode).toBe(200); + expect(pending.headers['content-type']).toContain('text/event-stream'); + expect(pendingEvents.map((event) => event.type)).toEqual(['interaction.created', 'error']); + expect(pending.text).not.toContain('interaction.completed'); + expect(pending.text).not.toContain('unsafe incomplete poll text'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + const pollsBeforeCompletion = sdkMocks.sessionMessages.mock.calls.length; + + terminalVisible = true; + const completed = await send(); + expect(completed.statusCode).toBe(200); + expect(completed.headers['content-type']).toContain('text/event-stream'); + expect(parseEvents(completed.text).at(-1).type).toBe('interaction.completed'); + expect(completed.text).toContain('reconciled Interactions answer'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages.mock.calls.length).toBeGreaterThan(pollsBeforeCompletion); + + const pollCountAfterCompletion = sdkMocks.sessionMessages.mock.calls.length; + const replayed = await send(); + expect(replayed.text).toBe(completed.text); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(pollCountAfterCompletion); + }); + + test('keyed streaming Interactions reconciliation emits and replays a confirmed terminal assistant error as SSE', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 100 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + sdkMocks.sessionMessages.mockImplementationOnce(async ({ path: sessionPath }) => { + const prompt = promptBySession.get(sessionPath.id); + return [{ + info: { + id: 'assistant-terminal-stream-error', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: sessionPath.id, + finish: 'stop', + error: { name: 'ProviderError', data: { message: 'terminal stream failure' } }, + }, + parts: [], + }]; + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'produce a terminal error', stream: true }; + const send = () => request(app).post('/v1beta/interactions').set('Idempotency-Key', 'terminal-stream-error') + .send(payload); + + const first = await send(); + expect(first.statusCode).toBe(200); + const reconciled = await send(); + expect(reconciled.statusCode).toBe(200); + expect(reconciled.headers['content-type']).toContain('text/event-stream'); + const events = reconciled.text + .split(/\r?\n/) + .filter((line) => line.startsWith('data: ')) + .map((line) => JSON.parse(line.slice('data: '.length))); + expect(events.map((event) => event.type)).toEqual(['interaction.created', 'error']); + expect(events[0].interaction).toMatchObject({ id: expect.any(String), status: 'in_progress' }); + expect(events[1].error.message).toContain('terminal stream failure'); + expect(reconciled.text).not.toContain('interaction.completed'); + + const replayed = await send(); + expect(replayed.statusCode).toBe(reconciled.statusCode); + expect(replayed.text).toBe(reconciled.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test('unknown Responses outcome reconciles the recorded child/message without a second prompt', async () => { + const { app, ctx } = createRouteApp({ REQUEST_TIMEOUT_MS: 100 }); + sdkMocks.sessionDelete.mockClear(); + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'uncertain turn' }; + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'unknown-responses').send(payload); + expect(first.statusCode).toBeGreaterThanOrEqual(400); + + const reconciled = await request(app).post('/v1/responses').set('Idempotency-Key', 'unknown-responses').send(payload); + expect(reconciled.statusCode).toBe(200); + expect(reconciled.body.output[0].content[0].text).toBe('idempotent answer'); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledWith(expect.objectContaining({ path: { id: 'idem-session-1' } })); + expect(ctx.responseState.has(reconciled.body.id)).toBe(true); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + }); + + test('unknown Interactions outcome also polls its recorded child/message instead of prompting again', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 100 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'uncertain interaction' }; + const first = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'unknown-interaction').send(payload); + expect(first.statusCode).toBeGreaterThanOrEqual(400); + + const reconciled = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'unknown-interaction').send(payload); + expect(reconciled.statusCode).toBe(200); + expect(reconciled.body.status).toBe('completed'); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledWith(expect.objectContaining({ path: { id: 'idem-session-1' } })); + }); + + test('store:false reconciliation deletes its reused child only after caching, never the stored parent', async () => { + const app = createRouteApp().app; + const parent = await request(app).post('/v1beta/interactions') + .send({ model: 'opencode/kimi-k2.5', input: 'parent' }); + expect(parent.statusCode).toBe(200); + + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'ephemeral child', + previous_interaction_id: parent.body.id, + store: false, + }; + const first = await request(app).post('/v1beta/interactions').set('Idempotency-Key', 'ephemeral-child').send(payload); + expect(first.statusCode).toBeGreaterThanOrEqual(400); + + const deletedSessionIds = []; + sdkMocks.sessionDelete.mockImplementation(async ({ path }) => { + deletedSessionIds.push(path.id); + return {}; + }); + const reconciled = await request(app).post('/v1beta/interactions') + .set('Idempotency-Key', 'ephemeral-child').send(payload); + expect(reconciled.statusCode).toBe(200); + await new Promise((resolve) => setImmediate(resolve)); + expect(deletedSessionIds).toEqual(['idem-fork-1']); + }); + + test('Responses store:false reconciliation deletes its owned child after wire capture, never its parent', async () => { + const { app, ctx } = createRouteApp(); + const parent = await request(app).post('/v1/responses') + .send({ model: 'opencode/kimi-k2.5', input: 'stored parent' }); + expect(parent.statusCode).toBe(200); + const parentSessionId = ctx.responseState.get(parent.body.id).sessionId; + + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: 'responses-ephemeral-child' } }); + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'ephemeral response child', + previous_response_id: parent.body.id, + store: false, + }; + const failed = await request(app).post('/v1/responses').set('Idempotency-Key', 'responses-ephemeral-child-key').send(payload); + expect(failed.statusCode).toBeGreaterThanOrEqual(400); + + const deletedSessionIds = []; + sdkMocks.sessionDelete.mockImplementation(async ({ path }) => { + deletedSessionIds.push(path.id); + return {}; + }); + const reconciled = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'responses-ephemeral-child-key').send(payload); + expect(reconciled.statusCode).toBe(200); + expect(ctx.responseState.has(reconciled.body.id)).toBe(false); + await new Promise((resolve) => setImmediate(resolve)); + expect(deletedSessionIds).toEqual(['responses-ephemeral-child']); + expect(deletedSessionIds).not.toContain(parentSessionId); + + const replayed = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'responses-ephemeral-child-key').send(payload); + expect(replayed.text).toBe(reconciled.text); + expect(deletedSessionIds).toEqual(['responses-ephemeral-child']); + }); + + test('confirmed required-tool validation failure is cached and replayed without another prompt', async () => { + const app = createRouteApp().app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + await request(app).post('/v1/responses').set('Idempotency-Key', 'confirmed-required-failure').send(payload); + + const failed = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'confirmed-required-failure').send(payload); + expect(failed.statusCode).toBe(502); + expect(failed.body.error.code).toBe('external_tool_choice_required'); + const replayed = await request(app).post('/v1/responses') + .set('Idempotency-Key', 'confirmed-required-failure').send(payload); + expect(replayed.statusCode).toBe(failed.statusCode); + expect(replayed.text).toBe(failed.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test('complete terminal error from the required-tool prompt is cached and replayed without a second forced prompt', async () => { + const app = createRouteApp().app; + sdkMocks.sessionPrompt.mockClear(); + sdkMocks.sessionMessages.mockClear(); + let forcedPromptCalls = 0; + const promptWithoutTool = async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + if (args.body.parts?.some((part) => String(part.text).includes('previous reply did not emit the required external tool call'))) { + forcedPromptCalls += 1; + } + return { + data: { + info: { id: `assistant-${args.body.messageID}`, role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'No required tool call.' }], + }, + }; + }; + sdkMocks.sessionPrompt.mockImplementationOnce(promptWithoutTool).mockImplementationOnce(promptWithoutTool); + sdkMocks.sessionMessages.mockImplementationOnce(async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: 'assistant-forced-terminal-error', + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + finish: 'stop', + error: { name: 'ProviderError', data: { message: 'forced prompt terminal error' } }, + }, + parts: [], + }]; + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + + const failed = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-terminal-error').send(payload); + expect(failed.statusCode).toBe(500); + expect(failed.text).toContain('forced prompt terminal error'); + const replayed = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-terminal-error').send(payload); + expect(replayed.statusCode).toBe(failed.statusCode); + expect(replayed.text).toBe(failed.text); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(forcedPromptCalls).toBe(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test('Responses does not poll for a forced tool result when its prompt resolves after client abort', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 80 }).app; + let signalForcedPromptStarted; + let releaseForcedPrompt; + const forcedPromptStarted = new Promise((resolve) => { signalForcedPromptStarted = resolve; }); + const forcedPromptGate = new Promise((resolve) => { releaseForcedPrompt = resolve; }); + sdkMocks.sessionPrompt.mockImplementation(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + const isForced = args.body.parts?.some((part) => + String(part.text).includes('previous reply did not emit the required external tool call')); + if (isForced) { + signalForcedPromptStarted(); + await forcedPromptGate; + } + return { + data: { + info: { + id: `assistant-${args.body.messageID}`, + role: 'assistant', + parentID: args.body.messageID, + finish: 'stop', + }, + parts: [], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: `assistant-poll-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts: [], + }]; + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + const aborted = await startAbortedRequest(app, '/v1/responses', 'forced-prompt-abort', payload); + await forcedPromptStarted; + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + + aborted.abort(); + // Let the server's response-close listener abort the route signal + // while the forced prompt remains deliberately unresolved. + await new Promise((resolve) => setTimeout(resolve, 50)); + releaseForcedPrompt(); + await new Promise((resolve) => setTimeout(resolve, 100)); + await aborted.closeServer(); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionMessages).toHaveBeenCalledTimes(1); + }); + + test('incomplete required-tool prompt errors remain unknown and are not re-prompted', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 35 }).app; + sdkMocks.sessionPrompt.mockClear(); + sdkMocks.sessionMessages.mockClear(); + let forcedPromptCalls = 0; + const promptWithoutTool = async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + if (args.body.parts?.some((part) => String(part.text).includes('previous reply did not emit the required external tool call'))) { + forcedPromptCalls += 1; + } + return { + data: { + info: { id: `assistant-${args.body.messageID}`, role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'No required tool call.' }], + }, + }; + }; + sdkMocks.sessionPrompt.mockImplementationOnce(promptWithoutTool).mockImplementationOnce(promptWithoutTool); + const incompleteErrorSnapshot = async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: `assistant-incomplete-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + error: { name: 'ProviderError', data: { message: 'forced prompt still running' } }, + }, + parts: [], + }]; + }; + sdkMocks.sessionMessages.mockImplementation(incompleteErrorSnapshot); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-incomplete-error').send(payload); + expect(first.statusCode).toBe(503); + expect(first.body.error.code).toBe('idempotency_reconciliation_pending'); + const retry = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-incomplete-error').send(payload); + expect(retry.statusCode).toBe(503); + expect(retry.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(retry.body.error.retryable).toBe(true); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(forcedPromptCalls).toBe(1); + sdkMocks.sessionMessages.mockImplementation(defaultSessionMessagesImplementation); + }); + + test('keyed required-tool follow-up rejects parseable partial poll output and reconciles without prompting again', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 35 }).app; + // Clear any unconsumed one-shot mock from preceding route cases before + // this test installs its exact prompt/poll behavior. + sdkMocks.sessionPrompt.mockReset(); + sdkMocks.sessionMessages.mockReset(); + let forcedPromptCalls = 0; + sdkMocks.sessionPrompt.mockImplementation(async (args) => { + const sessionId = args.path.id; + promptBySession.set(sessionId, structuredClone(args)); + const forced = args.body.parts?.some((part) => + String(part.text).includes('previous reply did not emit the required external tool call')); + if (forced) forcedPromptCalls += 1; + return { + data: { + info: { + id: `assistant-${args.body.messageID}`, + role: 'assistant', + parentID: args.body.messageID, + finish: 'stop', + }, + parts: [{ type: 'text', text: forced ? 'Forced prompt submitted.' : 'No required tool call.' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: `assistant-partial-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + }, + parts: [{ + type: 'text', + text: '{"name":"external__lookup","arguments":{}}', + }], + }]; + }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + + const first = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-partial-call').send(payload); + expect(first.statusCode).toBe(503); + expect(first.body.error.code).toBe('idempotency_reconciliation_pending'); + const retry = await request(app).post('/v1/responses').set('Idempotency-Key', 'forced-partial-call').send(payload); + expect(retry.statusCode).toBe(503); + expect(retry.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(retry.body.error.retryable).toBe(true); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(forcedPromptCalls).toBe(1); + expect(sdkMocks.sessionMessages.mock.calls.length).toBeGreaterThanOrEqual(2); + sdkMocks.sessionMessages.mockImplementation(defaultSessionMessagesImplementation); + }); + + test('non-stream disconnect cannot launch a required-tool follow-up after reconciliation starts', async () => { + const app = createRouteApp().app; + let signalInitialPrompt; + let releaseInitialPrompt; + let signalRetryPoll; + let releaseRetryPoll; + let signalOriginalRouteCaught; + const initialPromptStarted = new Promise((resolve) => { signalInitialPrompt = resolve; }); + const initialPromptGate = new Promise((resolve) => { releaseInitialPrompt = resolve; }); + const retryPollStarted = new Promise((resolve) => { signalRetryPoll = resolve; }); + const retryPollGate = new Promise((resolve) => { releaseRetryPoll = resolve; }); + const originalRouteCaught = new Promise((resolve) => { signalOriginalRouteCaught = resolve; }); + const payload = { + model: 'opencode/kimi-k2.5', + input: 'call the required tool', + tools: [{ type: 'function', name: 'lookup', parameters: { type: 'object', properties: {} } }], + tool_choice: 'required', + }; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + signalInitialPrompt(); + await initialPromptGate; + return { + data: { + info: { id: 'assistant-initial', role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'No tool was emitted.' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => { + signalRetryPoll(); + await retryPollGate; + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: `assistant-poll-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'No tool was emitted.' }], + }]; + }); + const originalError = jest.spyOn(console, 'error').mockImplementation((...args) => { + if (args[0] === '[Proxy] Responses API Error:' && args[1] === 'Client disconnected') { + signalOriginalRouteCaught(); + } + }); + + const aborted = await startAbortedRequest(app, '/v1/responses', 'required-tool-close', payload); + await initialPromptStarted; + aborted.abort(); + + let retryPromise; + for (let attempt = 0; attempt < 20 && !retryPromise; attempt += 1) { + const candidate = request(app).post('/v1/responses') + .set('Idempotency-Key', 'required-tool-close').send(payload).then((response) => response); + const outcome = await Promise.race([ + candidate.then((response) => ({ response })), + retryPollStarted.then(() => ({ claimed: true })), + ]); + if (outcome.claimed) { + retryPromise = candidate; + } else { + expect(outcome.response.statusCode).toBe(409); + } + } + expect(retryPromise).toBeDefined(); + releaseInitialPrompt(); + await originalRouteCaught; + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + releaseRetryPoll(); + const reconciledFailure = await retryPromise; + expect(reconciledFailure.statusCode).toBe(502); + expect(reconciledFailure.body.error.code).toBe('external_tool_choice_required'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + originalError.mockRestore(); + await aborted.closeServer(); + }); + + test('unconfirmed reconciliation stays unknown, returns stable 503, and can later succeed', async () => { + const app = createRouteApp({ REQUEST_TIMEOUT_MS: 40 }).app; + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + throw Object.assign(new Error('upstream timed out after submit'), { statusCode: 503 }); + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'uncertain turn' }; + await request(app).post('/v1/responses').set('Idempotency-Key', 'unconfirmed').send(payload); + + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => [{ + info: { + id: 'assistant-partial', + role: 'assistant', + parentID: promptBySession.get(path.id)?.body?.messageID, + sessionID: path.id, + }, + parts: [{ type: 'text', text: 'partial answer' }], + }]); + const pending = await request(app).post('/v1/responses').set('Idempotency-Key', 'unconfirmed').send(payload); + expect(pending.statusCode).toBe(503); + expect(pending.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(pending.body.error.retryable).toBe(true); + + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => [{ + info: { + id: `assistant-poll-${path.id}`, + role: 'assistant', + parentID: promptBySession.get(path.id)?.body?.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'idempotent answer' }], + }]); + const reconciled = await request(app).post('/v1/responses').set('Idempotency-Key', 'unconfirmed').send(payload); + expect(reconciled.statusCode).toBe(200); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + }); +}); + +describe('pre-submit failure idempotency handling', () => { + describe('isDeterministicPreSubmitFailure helper', () => { + test('returns false for status < 400 or status >= 500', () => { + expect(isDeterministicPreSubmitFailure(200)).toBe(false); + expect(isDeterministicPreSubmitFailure(201)).toBe(false); + expect(isDeterministicPreSubmitFailure(204)).toBe(false); + expect(isDeterministicPreSubmitFailure(301)).toBe(false); + expect(isDeterministicPreSubmitFailure(302)).toBe(false); + expect(isDeterministicPreSubmitFailure(500)).toBe(false); + expect(isDeterministicPreSubmitFailure(502)).toBe(false); + expect(isDeterministicPreSubmitFailure(503)).toBe(false); + expect(isDeterministicPreSubmitFailure(504)).toBe(false); + }); + + test('returns false for transient timeout / rate limit 4xx (408 and 429)', () => { + expect(isDeterministicPreSubmitFailure(408)).toBe(false); + expect(isDeterministicPreSubmitFailure(429)).toBe(false); + }); + + test('returns false when error body specifies retryable: true or type: transient_error', () => { + const retryableJson = Buffer.from(JSON.stringify({ error: { message: 'temporary', retryable: true } })); + expect(isDeterministicPreSubmitFailure(400, retryableJson)).toBe(false); + + const transientTypeJson = Buffer.from(JSON.stringify({ error: { type: 'transient_error', message: 'rate limit' } })); + expect(isDeterministicPreSubmitFailure(400, transientTypeJson)).toBe(false); + + const topLevelRetryable = Buffer.from(JSON.stringify({ retryable: true })); + expect(isDeterministicPreSubmitFailure(422, topLevelRetryable)).toBe(false); + + const topLevelTransient = Buffer.from(JSON.stringify({ type: 'transient_error' })); + expect(isDeterministicPreSubmitFailure(422, topLevelTransient)).toBe(false); + }); + + test('returns true for deterministic 4xx client errors', () => { + expect(isDeterministicPreSubmitFailure(400)).toBe(true); + expect(isDeterministicPreSubmitFailure(404)).toBe(true); + expect(isDeterministicPreSubmitFailure(413)).toBe(true); + expect(isDeterministicPreSubmitFailure(415)).toBe(true); + expect(isDeterministicPreSubmitFailure(422)).toBe(true); + + const standardErrorJson = Buffer.from(JSON.stringify({ error: { code: 'invalid_request', message: 'Bad request' } })); + expect(isDeterministicPreSubmitFailure(400, standardErrorJson)).toBe(true); + + const nonRetryableJson = Buffer.from(JSON.stringify({ error: { retryable: false } })); + expect(isDeterministicPreSubmitFailure(422, nonRetryableJson)).toBe(true); + + const nonJsonBody = Buffer.from('plain error string'); + expect(isDeterministicPreSubmitFailure(400, nonJsonBody)).toBe(true); + }); + }); + + describe('route-level pre-submit and post-submit tests', () => { + beforeEach(() => { + jest.clearAllMocks(); + promptBySession.clear(); + sessionSequence = 0; + forkSequence = 0; + assistantSequence = 0; + toolObservationCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => ({ data: { id: `idem-session-${++sessionSequence}` } })); + sdkMocks.sessionFork.mockImplementation(async () => ({ data: { id: `idem-fork-${++forkSequence}` } })); + sdkMocks.sessionDelete.mockImplementation(async () => ({})); + sdkMocks.sessionPrompt.mockImplementation(async (args) => { + const sessionId = args?.path?.id; + promptBySession.set(sessionId, structuredClone(args)); + const assistantId = `assistant-${++assistantSequence}`; + return { + data: { + info: { id: assistantId, role: 'assistant', parentID: args?.body?.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'idempotent answer' }], + }, + }; + }); + sdkMocks.sessionMessages.mockImplementation(async ({ path }) => { + const prompt = promptBySession.get(path.id); + return [{ + info: { + id: `assistant-poll-${path.id}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID: path.id, + finish: 'stop', + }, + parts: [{ type: 'text', text: 'idempotent answer' }], + }]; + }); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ stream: (async function* () {})() })); + }); + + // Test A: Pre-submit transient 503 -> backend recovers -> second request with same Idempotency-Key and payload succeeds with 200, prompts exactly once! + test('Test A: pre-submit transient 503 on /v1/responses abandons entry so retry succeeds with 200 and prompts once', async () => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'test pre-submit transient failure' }; + const idempotencyKey = 'test-a-pre-submit-503'; + + // Simulate transient backend failure before prompt submit (e.g. session creation fails) + sdkMocks.sessionCreate.mockImplementationOnce(async () => { + const err = new Error('Backend temporarily unavailable'); + err.statusCode = 503; + throw err; + }); + + const first = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(first.statusCode).toBe(503); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(0); + + // Backend recovered: second request with same Idempotency-Key succeeds cleanly + const second = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(200); + expect(second.body.id).toBeDefined(); + // Prompted exactly once across both attempts + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + test('Test A (interactions): pre-submit transient 503 on /v1beta/interactions abandons entry so retry succeeds with 200 and prompts once', async () => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'test pre-submit transient interactions' }; + const idempotencyKey = 'test-a-pre-submit-503-interactions'; + + sdkMocks.sessionCreate.mockImplementationOnce(async () => { + const err = new Error('Interactions backend temporarily unavailable'); + err.statusCode = 503; + throw err; + }); + + const first = await request(app) + .post('/v1beta/interactions') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(first.statusCode).toBe(503); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(0); + + const second = await request(app) + .post('/v1beta/interactions') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(200); + expect(second.body.status).toBe('completed'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + + // Test B: Pre-submit deterministic 422 (or 400) -> second request with same Idempotency-Key replays the cached 422 without entering route/prompting. + test('Test B (400 route-level): pre-submit deterministic 400 replays cached response on retry without entering route/prompting', async () => { + const app = createRouteApp().app; + const payload = { + model: 'opencode/kimi-k2.5', + input: 'deterministic pre-submit validation error', + previous_response_id: 'nonexistent-checkpoint-id', + }; + const idempotencyKey = 'test-b-deterministic-400'; + + const first = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(first.statusCode).toBe(400); + expect(first.body.error.code).toBe('invalid_previous_response_id'); + expect(first.headers['x-idempotent-replay']).toBeUndefined(); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(0); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(0); + + // Second request with same idempotency key and payload replays cached 400 + const second = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(400); + expect(second.body.error.code).toBe('invalid_previous_response_id'); + expect(second.text).toBe(first.text); + // Route was not executed again; no backend calls made + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(0); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(0); + }); + + test('Test B (422 middleware-level): pre-submit deterministic 422 replays cached 422 without re-entering route handler', async () => { + let routeInvocations = 0; + const { app } = createMiddlewareApp({ + route: (_req, res) => { + routeInvocations += 1; + res.status(422).json({ + error: { + type: 'invalid_request_error', + code: 'unprocessable_entity', + message: 'Semantic validation failed', + }, + }); + }, + }); + const payload = { model: 'opencode/kimi-k2.5', input: 'bad semantic field' }; + const idempotencyKey = 'test-b-deterministic-422'; + + const first = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(first.statusCode).toBe(422); + expect(first.body.error.code).toBe('unprocessable_entity'); + expect(routeInvocations).toBe(1); + + const second = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(422); + expect(second.body.error.code).toBe('unprocessable_entity'); + expect(second.text).toBe(first.text); + // Route was not entered a second time + expect(routeInvocations).toBe(1); + }); + + // Test C: First prompt submitted (record.submitted === true), then 503 or disconnect -> second request reconciles, does not re-prompt. + test('Test C (503): submitted prompt followed by 503 reconciles on second request without re-prompting', async () => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'submitted then 503' }; + const idempotencyKey = 'test-c-submitted-503'; + + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + const err = new Error('Upstream timeout after submit'); + err.statusCode = 503; + throw err; + }); + + const first = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(first.statusCode).toBe(503); + expect(first.body.error.code).toBe('idempotency_reconciliation_pending'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + + // Second request with same idempotency key reconciles via sessionMessages + const second = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(200); + expect(second.body.id).toBeDefined(); + // sessionPrompt was called ONLY ONCE across both requests (not re-prompted) + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionMessages).toHaveBeenCalled(); + }); + + test('Test C (disconnect): submitted prompt followed by client disconnect reconciles on second request without re-prompting', async () => { + const app = createRouteApp().app; + const payload = { model: 'opencode/kimi-k2.5', input: 'submitted then disconnect' }; + const idempotencyKey = 'test-c-submitted-disconnect'; + + let signalPromptStarted; + let releasePrompt; + const promptStarted = new Promise((resolve) => { signalPromptStarted = resolve; }); + const promptGate = new Promise((resolve) => { releasePrompt = resolve; }); + + sdkMocks.sessionPrompt.mockImplementationOnce(async (args) => { + promptBySession.set(args.path.id, structuredClone(args)); + signalPromptStarted(); + await promptGate; + return { + data: { + info: { id: 'assistant-after-disconnect', role: 'assistant', parentID: args.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'reconciled text' }], + }, + }; + }); + + const aborted = await startAbortedRequest(app, '/v1/responses', idempotencyKey, payload); + await promptStarted; + aborted.abort(); + releasePrompt(); + await aborted.closeServer(); + + // Second request reconciles via polling and does NOT re-prompt + const second = await request(app) + .post('/v1/responses') + .set('Idempotency-Key', idempotencyKey) + .send(payload); + + expect(second.statusCode).toBe(200); + expect(second.body.id).toBeDefined(); + // Prompt was executed only once + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(1); + }); + }); +}); diff --git a/tests/interactions.test.js b/tests/interactions.test.js index a421425..58f983d 100644 --- a/tests/interactions.test.js +++ b/tests/interactions.test.js @@ -11,10 +11,16 @@ const sdkMocks = { configUpdate: jest.fn(async () => ({})), toolIds: jest.fn(async () => ({ data: ['websearch', 'webfetch'] })), sessionCreate: jest.fn(async () => ({ data: { id: 'intr-session' } })), + sessionFork: jest.fn(async () => ({ data: { id: `intr-fork-${sdkMocks.sessionFork.mock.calls.length}` } })), sessionPrompt: jest.fn(async () => ({ data: { parts: [] } })), - sessionMessages: jest.fn(async () => ([ + sessionMessages: jest.fn(async (args) => ([ { - info: { role: 'assistant', finish: 'stop' }, + info: { + id: `assistant-${args?.path?.id || 'intr-session'}`, + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + finish: 'stop', + }, parts: [ { type: 'text', text: ANSWER_WITH_URL }, { @@ -34,7 +40,7 @@ jest.unstable_mockModule('@opencode-ai/sdk', () => ({ createOpencodeClient: jest.fn(() => ({ config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, tool: { ids: sdkMocks.toolIds }, - session: { create: sdkMocks.sessionCreate, prompt: sdkMocks.sessionPrompt, messages: sdkMocks.sessionMessages, delete: sdkMocks.sessionDelete }, + session: { create: sdkMocks.sessionCreate, fork: sdkMocks.sessionFork, prompt: sdkMocks.sessionPrompt, messages: sdkMocks.sessionMessages, delete: sdkMocks.sessionDelete }, event: { subscribe: sdkMocks.eventSubscribe }, })), })); @@ -103,6 +109,7 @@ describe('POST /v1beta/interactions', () => { tools: [{ type: 'google_search' }], }); expect(res.statusCode).toBe(200); + expect(res.body.output_text).toBe(ANSWER_WITH_URL); const call = res.body.steps.find((s) => s.type === 'google_search_call'); expect(call).toEqual({ type: 'google_search_call', queries: ['latest PostgreSQL release'] }); const result = res.body.steps.find((s) => s.type === 'google_search_result'); @@ -121,7 +128,7 @@ describe('POST /v1beta/interactions', () => { expect(res.body.id).toMatch(/^intr_/); }); - test('previous_interaction_id continues the session; store:false skips persistence', async () => { + test('previous_interaction_id forks the stored checkpoint; store:false skips persistence', async () => { const app = createApp(baseConfig({})).app; const first = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi' }); const second = await request(app).post('/v1beta/interactions').send({ @@ -130,8 +137,11 @@ describe('POST /v1beta/interactions', () => { previous_interaction_id: first.body.id, }); expect(second.statusCode).toBe(200); - // stored session reused: only one session.create across both calls + // Continuations fork from the stored checkpoint without creating a blank session. expect(sdkMocks.sessionCreate.mock.calls.length).toBe(1); + expect(sdkMocks.sessionFork).toHaveBeenCalledWith({ + path: { id: 'intr-session' }, + }); const bad = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi', @@ -147,7 +157,7 @@ describe('POST /v1beta/interactions', () => { expect(sdkMocks.sessionDelete).toHaveBeenCalled(); }); - test('store:false with previous_interaction_id keeps the shared parent session', async () => { + test('store:false with previous_interaction_id deletes only its fork, not the parent', async () => { const app = createApp(baseConfig({})).app; const first = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi' }); expect(first.statusCode).toBe(200); @@ -159,8 +169,8 @@ describe('POST /v1beta/interactions', () => { store: false, }); expect(second.statusCode).toBe(200); - // Reused parent session must survive an ephemeral continuation. - expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'intr-fork-1' } }); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: 'intr-session' } }); expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); }); @@ -190,6 +200,55 @@ describe('POST /v1beta/interactions', () => { expect(sdkMocks.eventSubscribe).not.toHaveBeenCalled(); }); + test('non-stream rejects reasoning-only output even when search evidence exists', async () => { + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [ + { type: 'reasoning', text: 'Private reasoning only' }, + { + type: 'tool', + id: 'call_search_reasoning_only', + tool: 'websearch', + state: { status: 'completed', input: { query: 'release notes' }, output: SEARCH_OUTPUT }, + }, + ], + }, + ]); + const app = createApp(baseConfig({})).app; + const res = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'Latest release?', + tools: [{ type: 'google_search' }], + }); + + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.body.status).not.toBe('completed'); + expect(res.body.output_text).toBeUndefined(); + expect(JSON.stringify(res.body)).not.toContain('Private reasoning only'); + }); + + test('stream rejects reasoning-only output instead of completing with empty text', async () => { + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ type: 'reasoning', text: 'Private reasoning only' }], + }, + ]); + const app = createApp(baseConfig({})).app; + const res = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'Think about this', + stream: true, + }); + + expect(res.statusCode).toBe(200); + expect(res.text).toContain('interaction.created'); + expect(res.text).toContain('"type":"error"'); + expect(res.text).not.toContain('interaction.completed'); + expect(res.text).not.toContain('Private reasoning only'); + }); + test('non-stream strips tool-call markup from output_text and steps when DISABLE_TOOLS=true', async () => { sdkMocks.sessionMessages.mockResolvedValueOnce([ { @@ -322,6 +381,140 @@ describe('POST /v1beta/interactions', () => { expect(res.text).not.toContain('interaction.completed'); expect(res.text).not.toContain('[DONE]'); }); + test('cleans up a request-owned session on error but preserves a reused parent session', async () => { + const app = createApp(baseConfig({})).app; + sdkMocks.sessionPrompt.mockRejectedValueOnce(new Error('backend down')); + const failed = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi' }); + expect(failed.statusCode).toBeGreaterThanOrEqual(400); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'intr-session' } }); + + const stored = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi' }); + expect(stored.statusCode).toBe(200); + jest.clearAllMocks(); + sdkMocks.toolIds.mockResolvedValue({ data: ['websearch', 'webfetch'] }); + sdkMocks.sessionPrompt.mockRejectedValueOnce(new Error('backend down')); + const failedContinuation = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'follow-up', + previous_interaction_id: stored.body.id, + }); + expect(failedContinuation.statusCode).toBeGreaterThanOrEqual(400); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'intr-fork-1' } }); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: 'intr-session' } }); + expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); + }); + test('retry cleanup retains every owned session ID until deletion succeeds', async () => { + const app = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })).app; + sdkMocks.sessionCreate + .mockResolvedValueOnce({ data: { id: 'retry-owned-session-1' } }) + .mockResolvedValueOnce({ data: { id: 'retry-owned-session-2' } }); + sdkMocks.sessionPrompt + .mockRejectedValueOnce({ data: { statusCode: 503 } }) + .mockRejectedValueOnce(new Error('second prompt failure')); + const deleteAttempts = []; + sdkMocks.sessionDelete.mockImplementation(async ({ path: { id } }) => { + deleteAttempts.push(id); + if (id === 'retry-owned-session-1' && deleteAttempts.filter((attemptedId) => attemptedId === id).length === 1) { + throw new Error('first delete failed'); + } + return {}; + }); + + const response = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'retry cleanup', + }); + + expect(response.statusCode).toBeGreaterThanOrEqual(400); + expect(deleteAttempts).toEqual([ + 'retry-owned-session-1', + 'retry-owned-session-1', + 'retry-owned-session-2', + ]); + }); + test('store:false retries a transient owned-session delete after terminal success', async () => { + const app = createApp(baseConfig({})).app; + sdkMocks.sessionDelete.mockRejectedValueOnce({ data: { statusCode: 503 } }); + + const response = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'ephemeral success', + store: false, + }); + + expect(response.statusCode).toBe(200); + expect(sdkMocks.sessionDelete).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(1, { path: { id: 'intr-session' } }); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(2, { path: { id: 'intr-session' } }); + }); + test('store:true preserves the owned session after ordinary success', async () => { + const app = createApp(baseConfig({})).app; + + const response = await request(app).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'persisted success', + store: true, + }); + + expect(response.statusCode).toBe(200); + expect(sdkMocks.sessionCreate).toHaveBeenCalledTimes(1); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + }); + test('store:true client disconnect retries a transient owned-session delete before prompt resolves', async () => { + const app = createApp(baseConfig({})).app; + const server = app.listen(0); + let markPromptStarted; + const promptStarted = new Promise((resolve) => { markPromptStarted = resolve; }); + let resolvePrompt; + let promptResolved = false; + const pendingPrompt = new Promise((resolve) => { + resolvePrompt = () => { + promptResolved = true; + resolve({}); + }; + }); + sdkMocks.sessionPrompt.mockImplementationOnce(() => { + markPromptStarted(); + return pendingPrompt; + }); + let markDeleteAttempted; + const deleteAttempted = new Promise((resolve) => { markDeleteAttempted = resolve; }); + let markDeleteSucceeded; + const deleteSucceeded = new Promise((resolve) => { markDeleteSucceeded = resolve; }); + sdkMocks.sessionDelete + .mockImplementationOnce(async () => { + markDeleteAttempted(); + throw { data: { statusCode: 503 } }; + }) + .mockImplementationOnce(async () => { + markDeleteSucceeded(); + return {}; + }); + + try { + const clientRequest = request(server).post('/v1beta/interactions').send({ + model: 'opencode/kimi-k2.5', + input: 'disconnect cleanup', + store: true, + }); + // Observe the abort as a handled promise rejection; the cleanup path + // is asynchronous and is synchronized separately below. + void clientRequest.then(() => undefined, () => undefined); + await promptStarted; + clientRequest.abort(); + await deleteAttempted; + expect(promptResolved).toBe(false); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(1, { path: { id: 'intr-session' } }); + resolvePrompt({}); + await deleteSucceeded; + + expect(sdkMocks.sessionDelete).toHaveBeenCalledTimes(2); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(1, { path: { id: 'intr-session' } }); + expect(sdkMocks.sessionDelete).toHaveBeenNthCalledWith(2, { path: { id: 'intr-session' } }); + } finally { + await new Promise((resolve) => server.close(resolve)); + } + }); test('function tools stay rejected without touching backend', async () => { const app = createApp(baseConfig({})).app; jest.clearAllMocks(); @@ -334,7 +527,7 @@ describe('POST /v1beta/interactions', () => { expect(sdkMocks.sessionPrompt).not.toHaveBeenCalled(); expect(sdkMocks.sessionMessages).not.toHaveBeenCalled(); }); - test('stream previous_interaction_id reuses stored session', async () => { + test('stream previous_interaction_id forks the stored checkpoint', async () => { const app = createApp(baseConfig({})).app; const first = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'hi' }); expect(first.statusCode).toBe(200); @@ -346,7 +539,7 @@ describe('POST /v1beta/interactions', () => { expect(second.text).not.toContain('[DONE]'); expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); }); - test('stream store false deletes owned session but keeps shared parent', async () => { + test('stream store false deletes only its fork and keeps the shared parent', async () => { const app = createApp(baseConfig({})).app; jest.clearAllMocks(); sdkMocks.toolIds.mockResolvedValue({ data: ['websearch', 'webfetch'] }); @@ -359,7 +552,8 @@ describe('POST /v1beta/interactions', () => { sdkMocks.toolIds.mockResolvedValue({ data: ['websearch', 'webfetch'] }); const second = await request(app).post('/v1beta/interactions').send({ model: 'opencode/kimi-k2.5', input: 'follow-up', previous_interaction_id: first.body.id, stream: true, store: false }); expect(second.statusCode).toBe(200); - expect(sdkMocks.sessionDelete).not.toHaveBeenCalled(); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'intr-fork-1' } }); + expect(sdkMocks.sessionDelete).not.toHaveBeenCalledWith({ path: { id: 'intr-session' } }); expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); }); }); diff --git a/tests/messages-anthropic.test.js b/tests/messages-anthropic.test.js index da84577..6b0c7e7 100644 --- a/tests/messages-anthropic.test.js +++ b/tests/messages-anthropic.test.js @@ -170,6 +170,9 @@ describe('POST /v1/messages', () => { let app; beforeEach(() => { jest.clearAllMocks(); + sdkMocks.sessionMessages.mockReset().mockImplementation(async () => ([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Hello!' }] } + ])); const config = { PORT: 10000, API_KEY: 'test-key', OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', REQUEST_TIMEOUT_MS: 5000, DISABLE_TOOLS: true, DEBUG: false }; app = createApp(config).app; }); @@ -574,7 +577,7 @@ describe('POST /v1/messages', () => { expect(flat.indexOf('toolu_p_a')).toBeLessThan(flat.indexOf('toolu_p_b')); }); test('tool_choice any forces tool_use on non-stream when first turn is empty', async () => { - sdkMocks.sessionMessages.mockResolvedValueOnce([{ info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'hello no tool' }] }]); + sdkMocks.sessionMessages.mockResolvedValueOnce([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); sdkMocks.sessionMessages.mockResolvedValueOnce([{ info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: '[{"id":"toolu_req_1","name":"external__read","arguments":{"path":"a.txt"}}]' }] }]); const res = await request(app).post('/v1/messages') .set('Authorization', 'Bearer test-key') @@ -620,6 +623,16 @@ describe('POST /v1/messages', () => { expect(res.body.error.code).toBe('external_tool_choice_required'); expect(String(res.body.error.message)).toContain('required external tool call'); }); + test('tool_choice required on non-stream still forces a call after an empty completed poll', async () => { + sdkMocks.sessionMessages.mockResolvedValue([{ info: { id: 'empty-assistant', role: 'assistant', finish: 'stop' }, parts: [] }]); + const res = await request(app).post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/muse-spark-1.3-contributor-free', max_tokens: 100, messages: [{ role: 'user', content: 'Read a.txt' }], tools: [{ name: 'read', description: 'Read', input_schema: { type: 'object' } }], tool_choice: { type: 'any' } }); + + expect(res.statusCode).toBe(502); + expect(res.body.error.code).toBe('external_tool_choice_required'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + }); test('tool_choice required without tool on stream returns explicit contract', async () => { sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'hello no tool' }] }]); sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ stream: (async function* () { yield { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'msg-session' }, delta: 'hello no tool' } }; yield { type: 'message.updated', properties: { info: { sessionID: 'msg-session', finish: 'stop' } } }; })() })); @@ -637,4 +650,111 @@ describe('POST /v1/messages', () => { expect(res.text).not.toContain('[DONE]'); expect(res.text).not.toContain('external__'); }); + test('tool_choice required on stream still forces a call after an empty completed poll', async () => { + sdkMocks.sessionMessages.mockResolvedValue([{ info: { id: 'empty-assistant', role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ stream: (async function* () { + yield { type: 'message.updated', properties: { info: { id: 'empty-assistant', role: 'assistant', sessionID: 'msg-session', finish: 'stop', parts: [] } } }; + })() })); + + const res = await request(app).post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/muse-spark-1.3-contributor-free', max_tokens: 100, stream: true, messages: [{ role: 'user', content: 'Read a.txt' }], tools: [{ name: 'read', description: 'Read', input_schema: { type: 'object' } }], tool_choice: { type: 'any' } }); + const frames = parseMessagesSseFrames(res.text); + + expect(res.statusCode).toBe(200); + expect(frames.find((frame) => frame.event === 'error')?.data.error.code).toBe('external_tool_choice_required'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(res.text).not.toContain('[DONE]'); + }); + test('stream: reasoning followed by text emits content_block_stop for thinking before content_block_start for text', async () => { + sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { type: 'message.part.updated', properties: { part: { type: 'reasoning', sessionID: 'msg-session' }, delta: 'Thinking about this' } }; + yield { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'msg-session' }, delta: 'Here is the answer' } }; + yield { type: 'message.updated', properties: { info: { sessionID: 'msg-session', finish: 'stop' } } }; + })() + })); + const res = await request(app).post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/muse-spark-1.3-contributor-free', max_tokens: 100, stream: true, messages: [{ role: 'user', content: 'hello' }] }); + expect(res.statusCode).toBe(200); + const frames = parseMessagesSseFrames(res.text); + + const thinkStarts = messagesThinkingStarts(frames); + expect(thinkStarts).toHaveLength(1); + expect(thinkStarts[0].data.index).toBe(0); + + const thinkDeltas = messagesThinkingDeltas(frames); + expect(thinkDeltas).toHaveLength(1); + expect(thinkDeltas[0].data.delta.thinking).toBe('Thinking about this'); + + const textDeltas = messagesTextDeltas(frames); + expect(textDeltas).toHaveLength(1); + expect(textDeltas[0].data.delta.text).toBe('Here is the answer'); + + const textStarts = frames.filter((f) => f.event === 'content_block_start' && f.data?.content_block?.type === 'text'); + expect(textStarts).toHaveLength(1); + expect(textStarts[0].data.index).toBe(1); + + const thinkStopIdx = frames.findIndex((f) => f.event === 'content_block_stop' && f.data.index === 0); + const textStartIdx = frames.findIndex((f) => f.event === 'content_block_start' && f.data.index === 1); + const textStopIdx = frames.findIndex((f) => f.event === 'content_block_stop' && f.data.index === 1); + + expect(thinkStopIdx).toBeGreaterThan(-1); + expect(textStartIdx).toBeGreaterThan(-1); + expect(textStopIdx).toBeGreaterThan(-1); + expect(thinkStopIdx).toBeLessThan(textStartIdx); + expect(textStartIdx).toBeLessThan(textStopIdx); + + const stopFrames = frames.filter((f) => f.event === 'content_block_stop'); + expect(stopFrames).toHaveLength(2); + }); + test('shouldBufferExternalStream: thinking block is started and stopped before text block is started and stopped', async () => { + sdkMocks.sessionMessages.mockResolvedValue([{ info: { role: 'assistant', finish: 'stop' }, parts: [] }]); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { type: 'message.part.updated', properties: { part: { type: 'reasoning', sessionID: 'msg-session' }, delta: 'Plan: read file' } }; + yield { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'msg-session' }, delta: 'Calling tool now ' } }; + yield { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'msg-session' }, delta: '[{"id":"toolu_buf_1","name":"external__read","arguments":{"path":"file.txt"}}]' } }; + yield { type: 'message.updated', properties: { info: { sessionID: 'msg-session', finish: 'stop' } } }; + })() + })); + const res = await request(app).post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/muse-spark-1.3-contributor-free', + max_tokens: 100, + stream: true, + messages: [{ role: 'user', content: 'Read file.txt' }], + tools: [{ name: 'read', description: 'Read a file', input_schema: { type: 'object', properties: { path: { type: 'string' } } } }] + }); + expect(res.statusCode).toBe(200); + const frames = parseMessagesSseFrames(res.text); + + const thinkStartIdx = frames.findIndex((f) => f.event === 'content_block_start' && f.data?.content_block?.type === 'thinking'); + const thinkDeltaIdx = frames.findIndex((f) => f.event === 'content_block_delta' && f.data?.delta?.type === 'thinking_delta'); + const thinkStopIdx = frames.findIndex((f) => f.event === 'content_block_stop' && f.data?.index === 0); + + const textStartIdx = frames.findIndex((f) => f.event === 'content_block_start' && f.data?.content_block?.type === 'text'); + const textDeltaIdx = frames.findIndex((f) => f.event === 'content_block_delta' && f.data?.delta?.type === 'text_delta'); + const textStopIdx = frames.findIndex((f) => f.event === 'content_block_stop' && f.data?.index === 1); + + const toolStartIdx = frames.findIndex((f) => f.event === 'content_block_start' && f.data?.content_block?.type === 'tool_use'); + const toolDeltaIdx = frames.findIndex((f) => f.event === 'content_block_delta' && f.data?.delta?.type === 'input_json_delta'); + const toolStopIdx = frames.findIndex((f) => f.event === 'content_block_stop' && f.data?.index === 2); + + expect(thinkStartIdx).toBeLessThan(thinkDeltaIdx); + expect(thinkDeltaIdx).toBeLessThan(thinkStopIdx); + expect(thinkStopIdx).toBeLessThan(textStartIdx); + expect(textStartIdx).toBeLessThan(textDeltaIdx); + expect(textDeltaIdx).toBeLessThan(textStopIdx); + expect(textStopIdx).toBeLessThan(toolStartIdx); + expect(toolStartIdx).toBeLessThan(toolDeltaIdx); + expect(toolDeltaIdx).toBeLessThan(toolStopIdx); + + const stopFrames = frames.filter((f) => f.event === 'content_block_stop'); + expect(stopFrames).toHaveLength(3); + expect(stopFrames.map((f) => f.data.index)).toEqual([0, 1, 2]); + }); }); diff --git a/tests/stream-hardening.test.js b/tests/stream-hardening.test.js index c39cb1b..99a2dd5 100644 --- a/tests/stream-hardening.test.js +++ b/tests/stream-hardening.test.js @@ -210,6 +210,24 @@ describe('collector terminal state', () => { expect(result).toMatchObject({ content: '', reasoning: '', noData: true }); }); + test('first-delta timeout keeps snapshot-only text and skips noData', async () => { + const result = await run([ + { + type: 'message.updated', + properties: { + info: { + id: 'msg-1', + role: 'assistant', + sessionID: 'collector-session', + parts: [{ id: 'p1', messageID: 'msg-1', type: 'text', text: 'snapshot answer' }] + } + } + } + ], 20); + expect(result).toMatchObject({ content: 'snapshot answer', authoritative: true }); + expect(result.noData).toBeUndefined(); + }); + test.each([['tool'], ['tool-calls']])( 'keeps the stream open when a turn finishes with %s', async (finish) => { @@ -322,6 +340,63 @@ describe('collector ID-less tool tracking', () => { expect(result.noData).toBeUndefined(); }); + test('a stale full message snapshot cannot complete a newer running tool part', async () => { + let terminalUpdateObserved = false; + const toolPart = (status) => ({ + type: 'message.part.updated', + properties: { + part: { + id: 'tool-call-1', + messageID: 'assistant-1', + type: 'tool', + tool: 'bash', + state: { status }, + sessionID: 'collector-session', + }, + }, + }); + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield toolPart('running'); + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-1', + role: 'assistant', + finish: 'stop', + sessionID: 'collector-session', + parts: [{ + id: 'tool-call-1', + messageID: 'assistant-1', + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + }], + }, + }, + }; + await new Promise((resolve) => setTimeout(resolve, 30)); + terminalUpdateObserved = true; + yield toolPart('completed'); + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-1', role: 'assistant', finish: 'stop', sessionID: 'collector-session' } }, + }; + })(), + }), + }, + }; + + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents('collector-session', 1000); + + expect(terminalUpdateObserved).toBe(true); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + test('single-bucket mismatch clears without hanging (named pending, unnamed terminal)', async () => { const result = await run([ noidPart('web_fetch', 'pending'), @@ -408,36 +483,2088 @@ describe('collector event error and tool deltas', () => { }); }); -describe('collector polling finish classification', () => { - const run = (snapshots, timeoutMs = 1000) => { - let call = 0; +describe('collector authoritative snapshots', () => { + const run = (events, onDelta = null) => { const client = { - session: { - messages: async () => { - const next = snapshots[Math.min(call, snapshots.length - 1)]; - call += 1; - return next; - } + event: { + subscribe: async () => ({ + stream: (async function* () { + for (const event of events) yield event; + })() + }) } }; - return createCollector({ client, logDebug: () => {} }).pollForAssistantResponse('collector-session', timeoutMs, 1); + return createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, onDelta, null, null); }; - const assistant = (info, text) => [{ info: { role: 'assistant', ...info }, parts: [{ type: 'text', text }] }]; - - test.each([['tool'], ['tool-calls']])( - 'does not finish on time.completed while a turn awaits %s', - async (finish) => { - const result = await run([ - assistant({ finish, time: { completed: 1 } }, 'Calling a tool.'), - assistant({ finish: 'stop', time: { completed: 2 } }, 'Done: 42.') - ]); - expect(result.content).toBe('Done: 42.'); + const textDelta = (partID, delta, messageID) => ({ + type: 'message.part.delta', + properties: { sessionID: 'collector-session', partID, field: 'text', delta, ...(messageID ? { messageID } : {}) } + }); + const stop = (parts) => ({ + type: 'message.updated', + properties: { + info: { + sessionID: 'collector-session', + finish: 'stop', + ...(parts ? { parts } : {}) + } } - ); + }); - test('still finishes on time.completed for a terminal turn', async () => { - const result = await run([assistant({ time: { completed: 1 } }, 'Answer without a finish reason.')]); - expect(result.content).toBe('Answer without a finish reason.'); + test('final part.text repairs a delta that never arrived', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, delta: 'Hello ' } }, + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', text: 'Hello world', sessionID: 'collector-session' } } }, + stop() + ]); + expect(result).toMatchObject({ content: 'Hello world', authoritative: true }); + }); + + test('id-less final part.text still repairs a missing delta', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'collector-session' }, delta: 'Hello ' } }, + { type: 'message.part.updated', properties: { part: { type: 'text', text: 'Hello world', sessionID: 'collector-session' } } }, + stop() + ]); + expect(result).toMatchObject({ content: 'Hello world', authoritative: true }); + }); + + test('keeps multiple id-less text deltas in one stable bucket', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'collector-session' }, delta: 'Hello ' } }, + { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: 'collector-session' }, delta: 'world' } }, + stop() + ]); + expect(result.content).toBe('Hello world'); + expect(result.authoritative).toBeUndefined(); + }); + + test('message.updated info.parts snapshot replaces partial delta text', async () => { + const result = await run([ + textDelta('p1', 'partial '), + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', sessionID: 'collector-session' } } }, + textDelta('p1', 'answer'), + stop([ + { id: 'p1', type: 'text', text: 'complete answer' } + ]) + ]); + expect(result).toMatchObject({ content: 'complete answer', authoritative: true }); + expect(result.content).not.toContain('partial'); + }); + + test('keeps a delta that arrives before its part type is known', async () => { + const result = await run([ + textDelta('p1', 'early '), + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', sessionID: 'collector-session' } } }, + textDelta('p1', 'answer'), + stop() + ]); + expect(result).toMatchObject({ content: 'early answer' }); + expect(result.authoritative).toBeUndefined(); + }); + + test('a final snapshot wins over buffered deltas without duplicating text', async () => { + const result = await run([ + textDelta('p1', 'early '), + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', text: 'complete answer', sessionID: 'collector-session' } } }, + stop([{ id: 'p1', type: 'text', text: 'complete answer' }]) + ]); + expect(result).toMatchObject({ content: 'complete answer', authoritative: true }); + expect(result.content.match(/complete/g)).toHaveLength(1); + expect(result.content).not.toContain('early'); + }); + + test('does not mark equal delta and snapshot text as authoritative', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, delta: 'Hello world' } }, + stop([{ id: 'p1', type: 'text', text: 'Hello world' }]) + ]); + expect(result).toMatchObject({ content: 'Hello world' }); + expect(result.authoritative).toBeUndefined(); + expect(result.content.match(/Hello world/g)).toHaveLength(1); + }); + + test('ignores a user prompt snapshot and user text parts', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'msg-user', + sessionID: 'collector-session', + role: 'user', + parts: [{ id: 'user-text', messageID: 'msg-user', type: 'text', text: 'SECRET PROMPT' }] + } + } + }; + yield { + type: 'message.updated', + properties: { info: { id: 'msg-assistant', sessionID: 'collector-session', role: 'assistant' } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'user-text', messageID: 'msg-user', type: 'text', sessionID: 'collector-session', text: 'SECRET PROMPT' } } + }; + yield textDelta('assistant-text', 'answer', 'msg-assistant'); + yield { + type: 'message.part.updated', + properties: { part: { id: 'assistant-text', messageID: 'msg-assistant', type: 'text', text: 'answer', sessionID: 'collector-session' } } + }; + yield stop(); + })() + }) + } + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null); + expect(result.content).toBe('answer'); + expect(result.content).not.toContain('SECRET'); + expect(deltas.join('')).not.toContain('SECRET'); + }); + + test('a delayed user role filters a part that arrived before the role was known', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'user-text', messageID: 'msg-user', type: 'text', sessionID: 'collector-session' }, delta: 'SECRET PROMPT' } }, + { type: 'message.updated', properties: { info: { id: 'msg-user', sessionID: 'collector-session', role: 'user' } } }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', sessionID: 'collector-session', role: 'assistant' } } }, + { type: 'message.part.updated', properties: { part: { id: 'assistant-text', messageID: 'msg-assistant', type: 'text', sessionID: 'collector-session' } } }, + textDelta('assistant-text', 'answer', 'msg-assistant'), + stop() + ]); + expect(result.content).toBe('answer'); + expect(result.content).not.toContain('SECRET'); + expect(result.authoritative).toBe(true); + }); + + test('role-unknown legacy deltas are not called back and are purged when classified as user', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'user-part', messageID: 'msg-user-late', type: 'text', sessionID: 'collector-session' }, + delta: 'SECRET PROMPT', + }, + }; + expect(deltas).toEqual([]); + yield { + type: 'message.updated', + properties: { info: { id: 'msg-user-late', role: 'user', sessionID: 'collector-session' } }, + }; + expect(deltas).toEqual([]); + yield textDelta('user-part', 'SECRET PROMPT CONTINUATION'); + expect(deltas).toEqual([]); + yield { + type: 'message.updated', + properties: { info: { id: 'msg-assistant-late', role: 'assistant', sessionID: 'collector-session' } }, + }; + yield { + type: 'message.part.updated', + properties: { + part: { id: 'assistant-part', messageID: 'msg-assistant-late', type: 'text', sessionID: 'collector-session' }, + delta: 'Visible answer', + }, + }; + yield stop(); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null); + + expect(result.content).toBe('Visible answer'); + expect(result.content).not.toContain('SECRET PROMPT'); + expect(result.content).not.toContain('SECRET PROMPT CONTINUATION'); + expect(deltas.join('')).toBe('Visible answer'); + }); + + test('releases buffered legacy deltas to the callback when the role becomes assistant', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'assistant-part', messageID: 'msg-assistant-late', type: 'text', sessionID: 'collector-session' }, + delta: 'Early answer', + }, + }; + expect(deltas).toEqual([]); + yield { + type: 'message.updated', + properties: { info: { id: 'msg-assistant-late', role: 'assistant', sessionID: 'collector-session' } }, + }; + expect(deltas).toEqual(['Early answer']); + yield stop(); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null); + + expect(result.content).toBe('Early answer'); + expect(deltas).toEqual(['Early answer']); + }); + + test('accepts an unrecognized legacy role as role-unknown rather than filtering its text', async () => { + const deltas = []; + const result = await run([ + { + type: 'message.updated', + properties: { info: { id: 'legacy-message', role: 'developer', sessionID: 'collector-session' } }, + }, + { + type: 'message.part.updated', + properties: { + part: { id: 'legacy-part', messageID: 'legacy-message', type: 'text', sessionID: 'collector-session' }, + delta: 'Legacy role answer', + }, + }, + stop(), + ], (delta) => deltas.push(delta)); + + expect(result.content).toBe('Legacy role answer'); + expect(result.authoritative).toBe(true); + expect(deltas).toEqual([]); + }); + + test('keeps truly role-unknown legacy text only in the finalized result', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { type: 'message.part.updated', properties: { part: { id: 'legacy-part', messageID: 'role-never-known', type: 'text', sessionID: 'collector-session' }, delta: 'Legacy final text' } }; + expect(deltas).toEqual([]); + yield stop(); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null); + + expect(result.content).toBe('Legacy final text'); + expect(deltas).toEqual([]); + }); + + test('keeps every part when one part snapshot is missing', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', sessionID: 'collector-session' }, delta: 'Hello ' } }, + { type: 'message.part.updated', properties: { part: { id: 'p2', messageID: 'msg-1', type: 'text', sessionID: 'collector-session' }, delta: 'world' } }, + { type: 'message.part.updated', properties: { part: { id: 'p2', messageID: 'msg-1', type: 'text', text: 'world!', sessionID: 'collector-session' } } }, + stop() + ]); + expect(result.content).toBe('Hello world!'); + expect(result.authoritative).toBe(true); + }); + + test('a newer full part.text overrides a sticky info.parts snapshot', async () => { + const result = await run([ + { + type: 'message.updated', + properties: { + info: { + id: 'msg-1', + role: 'assistant', + sessionID: 'collector-session', + parts: [{ id: 'p1', messageID: 'msg-1', type: 'text', text: 'old snapshot' }] + } + } + }, + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'newer text', sessionID: 'collector-session' } } }, + stop() + ]); + expect(result.content).toBe('newer text'); + expect(result.content).not.toContain('old snapshot'); + }); + + test('does not attach an ownerless early delta to a later explicit part owner', async () => { + const result = await run([ + { type: 'message.part.delta', properties: { sessionID: 'collector-session', partID: 7, field: 'text', delta: 'early ' } }, + { type: 'message.part.updated', properties: { part: { id: 7, messageID: 'msg-1', type: 'text', sessionID: 'collector-session' } } }, + { type: 'message.part.delta', properties: { sessionID: 'collector-session', messageID: 'msg-1', partID: 7, field: 'text', delta: 'answer' } }, + stop() + ]); + expect(result.content).toBe('answer'); + expect(result.authoritative).toBeUndefined(); + }); + + test('snapshot-only final part.text becomes content with zero deltas', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'snapshot only', sessionID: 'collector-session' } } }, + stop() + ]); + expect(result.content).toBe('snapshot only'); + expect(result.authoritative).toBe(true); + }); + + test('combines snapshot prefix with subsequent deltas for the same part', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Hello', sessionID: 'collector-session' } } }, + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', sessionID: 'collector-session' }, delta: ' world' } }, + stop() + ]); + expect(result.content).toBe('Hello world'); + expect(result.authoritative).toBe(true); + }); + + test('interleaved snapshot -> delta -> newer snapshot -> delta does not duplicate delta text', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Hello', sessionID: 'collector-session' } } }, + textDelta('p1', ' world', 'msg-1'), + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Hello world', sessionID: 'collector-session' } } }, + textDelta('p1', '!', 'msg-1'), + stop() + ]); + expect(result.content).toBe('Hello world!'); + expect(result.content).not.toBe('Hello world world!'); + expect(result.content).not.toContain('world world'); + }); + + test('interleaved snapshot -> delta -> newer snapshot -> delta via message.part.updated delta', async () => { + const result = await run([ + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Hello', sessionID: 'collector-session' } } }, + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', sessionID: 'collector-session' }, delta: ' world' } }, + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Hello world', sessionID: 'collector-session' } } }, + { type: 'message.part.updated', properties: { part: { id: 'p1', messageID: 'msg-1', type: 'text', sessionID: 'collector-session' }, delta: '!' } }, + stop() + ]); + expect(result.content).toBe('Hello world!'); + expect(result.content).not.toBe('Hello world world!'); + expect(result.content).not.toContain('world world'); + }); + + test('co-occurring part.text and delta on legacy OpenCode server events does not duplicate delta text', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Initial', sessionID: 'collector-session' } + } + }, + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Initial reply', sessionID: 'collector-session' }, + delta: ' reply' + } + }, + stop() + ]); + expect(result.content).toBe('Initial reply'); + expect(result.content).not.toContain('reply reply'); + }); + + test('flushes orphaned deltas whose part update was dropped', async () => { + const result = await run([ + { type: 'message.part.delta', properties: { sessionID: 'collector-session', messageID: 'msg-1', partID: 'orphan', field: 'text', delta: 'orphan text' } }, + stop() + ]); + expect(result.content).toBe('orphan text'); + }); + + test('purges an ID-less pending delta when a user snapshot identifies its part', async () => { + const result = await run([ + textDelta('shared-part', 'SECRET USER TEXT'), + { + type: 'message.updated', + properties: { + info: { + id: 'msg-user', + role: 'user', + sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'shared-part', + messageID: 'msg-user', + type: 'text', + text: 'SECRET USER TEXT', + sessionID: 'collector-session', + }, + }, + }, + { type: 'message.updated', properties: { info: { id: 'msg-assistant', role: 'assistant', sessionID: 'collector-session' } } }, + textDelta('assistant-part', 'answer', 'msg-assistant'), + stop(), + ]); + expect(result.content).toBe('answer'); + expect(result.content).not.toContain('SECRET USER TEXT'); + }); + + test('remembers user part IDs for ID-less deltas without purging unrelated legacy deltas', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield textDelta('legacy-orphan', 'Legacy answer'); + yield { + type: 'message.updated', + properties: { + info: { + sessionID: 'collector-session', + role: 'user', + parts: [{ id: 'known-user-part', type: 'text', text: 'SECRET USER TEXT' }], + }, + }, + }; + yield textDelta('known-user-part', 'SECRET USER TEXT'); + yield stop(); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null); + + expect(result.content).toBe('Legacy answer'); + expect(result.content).not.toContain('SECRET USER TEXT'); + expect(deltas.join('')).not.toContain('SECRET USER TEXT'); + }); + + test('ignores a stale assistant part from a previous turn', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { + id: 'old-part', + messageID: 'msg-old', + type: 'text', + text: 'OLD TEXT', + time: { created: Date.now() - 60000 }, + sessionID: 'collector-session' + } + } + }, + { type: 'message.part.updated', properties: { part: { id: 'new-part', messageID: 'msg-new', type: 'text', sessionID: 'collector-session' }, delta: 'new answer' } }, + stop() + ]); + expect(result.content).toBe('new answer'); + expect(result.content).not.toContain('OLD TEXT'); + }); + + test('ignores a timestamp-less mismatched-parent assistant turn when parts arrive first', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'stale-part', + messageID: 'assistant-stale', + type: 'text', + sessionID: 'collector-session', + }, + delta: 'Stale answer', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stale', + role: 'assistant', + sessionID: 'collector-session', + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stale', + role: 'assistant', + parentID: 'previous-user', + finish: 'stop', + sessionID: 'collector-session', + }, + }, + }; + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null, undefined, 'current-user'); + + expect(result.content).toBe(''); + expect(result.noData).toBe(true); + expect(result.finish).toBeUndefined(); + expect(deltas).toEqual([]); + }); + + test.each([ + ['mismatched', { parentID: 'previous-user' }], + ['missing', {}], + ])('does not accept a timestamp-less completed assistant snapshot with %s parent metadata', async (_label, parent) => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-unverified', + role: 'assistant', + ...parent, + finish: 'stop', + sessionID: 'collector-session', + parts: [{ id: 'unverified-part', type: 'text', text: 'UNVERIFIED ANSWER' }], + }, + }, + }; + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null, undefined, 'current-user'); + + expect(result.content).toBe(''); + expect(result.noData).toBe(true); + expect(result.finish).toBeUndefined(); + expect(deltas).toEqual([]); + }); + + test('a reused part ID from an unverified message cannot overwrite the verified current turn', async () => { + const deltas = []; + const reusedPartID = 'reused-ownerless-part'; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { + id: reusedPartID, + type: 'text', + text: 'ownerless early snapshot', + sessionID: 'collector-session', + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'current-user', + sessionID: 'collector-session', + parts: [{ id: reusedPartID, type: 'text', text: 'verified current answer' }], + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-unverified', + role: 'assistant', + sessionID: 'collector-session', + parts: [{ id: reusedPartID, type: 'text', text: 'foreign overwrite' }], + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'current-user', + finish: 'stop', + sessionID: 'collector-session', + }, + }, + }; + })(), + }), + }, + }; + + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null, undefined, 'current-user'); + + expect(result.content).toBe('verified current answer'); + expect(result.content).not.toContain('foreign overwrite'); + expect(deltas).not.toContain('foreign overwrite'); + }); + + test('empty message IDs cannot alias ownerless part state or pending deltas', async () => { + const reusedPartID = 'empty-message-id-part'; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + partID: reusedPartID, + field: 'text', + delta: 'ownerless prefix', + }, + }; + yield { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + messageID: '', + partID: reusedPartID, + field: 'text', + delta: 'FOREIGN EMPTY-ID TEXT', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'current-user', + finish: 'stop', + sessionID: 'collector-session', + parts: [{ id: reusedPartID, type: 'text', text: 'verified answer' }], + }, + }, + }; + })(), + }), + }, + }; + + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, null, null, null, undefined, 'current-user'); + + expect(result.content).toBe('verified answer'); + expect(result.content).not.toContain('FOREIGN EMPTY-ID TEXT'); + }); + + test('remembers stale part IDs for ID-less deltas without blocking an explicitly different message', async () => { + const deltas = []; + const sharedPartID = 'shared-stale-part'; + const currentUserID = 'current-user'; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-old', + role: 'assistant', + parentID: 'previous-user', + finish: 'stop', + sessionID: 'collector-session', + parts: [{ id: sharedPartID, type: 'text', text: 'STALE SNAPSHOT' }], + }, + }, + }; + // The stale part metadata is already known; an ID-less + // delta cannot safely establish which message produced it. + yield textDelta(sharedPartID, ' STALE REINTRODUCTION'); + yield { + type: 'message.part.updated', + properties: { + part: { + id: sharedPartID, + messageID: 'assistant-current', + parentID: currentUserID, + type: 'text', + sessionID: 'collector-session', + }, + delta: 'VALID ANSWER', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + finish: 'stop', + sessionID: 'collector-session', + }, + }, + }; + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (delta) => deltas.push(delta), null, null, undefined, currentUserID); + + expect(result.content).toBe('VALID ANSWER'); + expect(result.content).not.toContain('STALE'); + expect(deltas).toEqual(['VALID ANSWER']); + }); +}); + +describe('collector continuation turn correlation', () => { + const currentUserID = 'current-user'; + const collect = (events, onDelta = null) => { + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + for (const event of events) yield event; + })(), + }), + }, + }; + return createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 1000, onDelta, null, null, undefined, currentUserID, + ); + }; + const stopMessage = (info) => ({ + type: 'message.updated', + properties: { info: { sessionID: 'collector-session', finish: 'stop', ...info } }, + }); + const delta = (messageID, partID, text, extra = {}) => ({ + type: 'message.part.delta', + properties: { sessionID: 'collector-session', messageID, partID, field: 'text', delta: text, ...extra }, + }); + + test('quarantines an answer delta until exact parent metadata, then releases it once', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield delta('assistant-current', 'answer-part', 'Answer'); + expect(deltas).toEqual([]); + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + expect(deltas).toEqual(['Answer']); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 1000, (text) => deltas.push(text), null, null, undefined, currentUserID, + ); + + expect(result).toMatchObject({ content: 'Answer', finish: 'stop' }); + expect(result.assistantMessageId).toBe('assistant-current'); + expect(deltas).toEqual(['Answer']); + }); + + test('also quarantines legacy message.part.updated deltas until parent confirmation', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'legacy-answer-part', messageID: 'assistant-current', type: 'text', sessionID: 'collector-session' }, + delta: 'Legacy answer', + }, + }; + expect(deltas).toEqual([]); + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 1000, (text) => deltas.push(text), null, null, undefined, currentUserID, + ); + + expect(result.content).toBe('Legacy answer'); + expect(deltas).toEqual(['Legacy answer']); + }); + + test('does not leak a part-parent-matched delta when message parent later mismatches', async () => { + const deltas = []; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: 'contradictory-part', + messageID: 'assistant-current', + parentID: currentUserID, + type: 'text', + sessionID: 'collector-session', + }, + delta: 'Must not leak', + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: 'previous-user' }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe(''); + expect(result.finish).toBeUndefined(); + expect(result.noData).toBe(true); + expect(deltas).toEqual([]); + }); + + test.each(['message.part.updated', 'message.part.delta'])( + 'rejects explicit mismatched part parent on %s despite a confirmed message parent', + async (eventType) => { + const deltas = []; + const mismatchedPart = eventType === 'message.part.updated' + ? { + type: eventType, + properties: { + part: { + id: 'wrong-parent-part', + messageID: 'assistant-current', + parentID: 'previous-user', + type: 'text', + sessionID: 'collector-session', + }, + delta: 'MUST NOT ESCAPE', + }, + } + : { + type: eventType, + properties: { + sessionID: 'collector-session', + messageID: 'assistant-current', + partID: 'wrong-parent-part', + parentID: 'previous-user', + field: 'text', + delta: 'MUST NOT ESCAPE', + }, + }; + const result = await collect([ + { + type: 'message.updated', + properties: { info: { id: 'assistant-current', role: 'assistant', parentID: currentUserID, sessionID: 'collector-session' } }, + }, + mismatchedPart, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe(''); + expect(result.content).not.toContain('MUST NOT ESCAPE'); + expect(deltas).toEqual([]); + expect(result.error).toBeInstanceOf(Error); + expect(result.error.message).toMatch(/correlation error/i); + }, + ); + + test.each(['message.part.updated', 'message.part.delta'])( + 'fails closed when ownerless %s contradicts the retained confirmed part owner', + async (eventType) => { + const conflictingPart = eventType === 'message.part.updated' + ? { + type: eventType, + properties: { + part: { + id: 'retained-owner-part', + parentID: 'previous-user', + type: 'text', + sessionID: 'collector-session', + }, + delta: 'MUST NOT SUCCEED', + }, + } + : { + type: eventType, + properties: { + sessionID: 'collector-session', + partID: 'retained-owner-part', + parentID: 'previous-user', + field: 'text', + delta: 'MUST NOT SUCCEED', + }, + }; + const result = await collect([ + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + sessionID: 'collector-session', + parts: [{ id: 'retained-owner-part', type: 'text', text: 'STALE RAW CONTENT' }], + }, + }, + }, + conflictingPart, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ]); + + expect(result.error).toBeInstanceOf(Error); + expect(result.error.message).toMatch(/correlation error/i); + expect(result.content).toBe(''); + expect(result.content).not.toContain('STALE RAW CONTENT'); + expect(result.finish).toBeUndefined(); + expect(result.cut).toBeUndefined(); + }, + ); + + test('fails closed when a confirmed message parent explicitly changes', async () => { + const result = await collect([ + { + type: 'message.updated', + properties: { info: { id: 'assistant-current', role: 'assistant', parentID: currentUserID, sessionID: 'collector-session' } }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: 'previous-user' }), + ]); + + expect(result.error).toBeInstanceOf(Error); + expect(result.error.message).toMatch(/parent changed/i); + expect(result.finish).toBeUndefined(); + expect(result.cut).toBeUndefined(); + }); + + test('keeps the confirmed message parent when a later sparse update omits it', async () => { + const result = await collect([ + { + type: 'message.updated', + properties: { info: { id: 'assistant-current', role: 'assistant', parentID: currentUserID, sessionID: 'collector-session' } }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant' }), + ]); + + expect(result.error).toBeUndefined(); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('a quarantined text delta does not clear the first-delta timeout', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'unverified-text-part', messageID: 'assistant-current', type: 'text', sessionID: 'collector-session' }, + delta: 'Unverified answer', + }, + }; + await new Promise((resolve) => setTimeout(resolve, 50)); + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 500, (text) => deltas.push(text), 20, null, undefined, currentUserID, + ); + + expect(result).toMatchObject({ content: '', noData: true }); + expect(deltas).toEqual([]); + }); + + test('releasing a verified delta clears the first-delta timeout', async () => { + const deltas = []; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'verified-text-part', messageID: 'assistant-current', type: 'text', sessionID: 'collector-session' }, + delta: 'Verified answer', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { id: 'assistant-current', role: 'assistant', parentID: currentUserID, sessionID: 'collector-session' }, + }, + }; + await new Promise((resolve) => setTimeout(resolve, 50)); + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 500, (text) => deltas.push(text), 20, null, undefined, currentUserID, + ); + + expect(result).toMatchObject({ content: 'Verified answer', finish: 'stop' }); + expect(deltas).toEqual(['Verified answer']); + }); + + test('an unverified tool part cannot block a valid current-turn terminal', async () => { + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: 'prior-tool-call', + messageID: 'assistant-prior', + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('a quarantined tool becomes active only after exact parent confirmation', async () => { + let continuedPastFirstStop = false; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'current-tool-call', + messageID: 'assistant-current', + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }; + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + continuedPastFirstStop = true; + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'current-tool-call', + messageID: 'assistant-current', + parentID: currentUserID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }; + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 1000, null, null, null, undefined, currentUserID, + ); + + expect(continuedPastFirstStop).toBe(true); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('latest quarantined tool state wins before matching message finish', async () => { + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'latest-tool-state', + messageID: 'assistant-current', + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }; + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'latest-tool-state', + messageID: 'assistant-current', + parentID: currentUserID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }; + yield stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }); + await new Promise(() => {}); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 150, null, null, 10, undefined, currentUserID, + ); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(result.idleTimeout).toBeUndefined(); + }); + + test('non-continuation tool lifecycle still waits for its terminal state', async () => { + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'ordinary-tool-state', + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }; + yield stopMessage({ id: 'ordinary-assistant', role: 'assistant' }); + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'ordinary-tool-state', + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }; + yield stopMessage({ id: 'ordinary-assistant', role: 'assistant' }); + })(), + }), + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', 1000, + ); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test.each([ + ['missing', {}], + ['mismatched', { parentID: 'previous-user' }], + ])('does not expose or finish an unknown-role message with %s parent metadata', async (_label, parent) => { + const deltas = []; + const result = await collect([ + delta('developer-message', 'developer-part', 'UNVERIFIED'), + stopMessage({ id: 'developer-message', role: 'developer', ...parent }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe(''); + expect(result.finish).toBeUndefined(); + expect(result.noData).toBe(true); + expect(deltas).toEqual([]); + }); + + test('accepts an unknown role only when its parent exactly matches the expected turn', async () => { + const deltas = []; + const result = await collect([ + delta('developer-message', 'developer-part', 'Developer answer'), + stopMessage({ id: 'developer-message', role: 'developer', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result).toMatchObject({ content: 'Developer answer', finish: 'stop' }); + expect(deltas).toEqual(['Developer answer']); + }); + + test('filters a legacy ID-less part delta after a stale snapshot classified its owner', async () => { + const deltas = []; + const result = await collect([ + stopMessage({ + id: 'assistant-old', + role: 'assistant', + parentID: 'previous-user', + parts: [{ id: 'stale-shared-part', type: 'text', text: 'STALE SNAPSHOT' }], + }), + { + type: 'message.part.updated', + properties: { + part: { id: 'stale-shared-part', type: 'text', sessionID: 'collector-session' }, + delta: 'STALE LEGACY DELTA', + }, + }, + { + type: 'message.part.updated', + properties: { + part: { id: 'current-part', messageID: 'assistant-current', type: 'text', sessionID: 'collector-session' }, + delta: 'Current answer', + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe('Current answer'); + expect(result.content).not.toContain('STALE'); + expect(deltas).toEqual(['Current answer']); + }); + + test('an explicitly different message owner takes precedence over a reused user part ID', async () => { + const result = await collect([ + { + type: 'message.updated', + properties: { + info: { + id: 'user-message', role: 'user', sessionID: 'collector-session', + parts: [{ id: 'reused-part', type: 'text', text: 'SECRET USER TEXT' }], + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { id: 'reused-part', messageID: 'assistant-message', type: 'text', sessionID: 'collector-session' }, + delta: 'Valid answer', + }, + }, + stopMessage({ id: 'assistant-message', role: 'assistant', parentID: currentUserID }), + ]); + + expect(result.content).toBe('Valid answer'); + expect(result.content).not.toContain('SECRET'); + }); + + test('ownerless text and tool events are ignored when a part ID has multiple explicit owners', async () => { + const deltas = []; + const result = await collect([ + { + type: 'message.updated', + properties: { + info: { + id: 'user-message', role: 'user', sessionID: 'collector-session', + parts: [{ id: 'reused-part', type: 'text', text: 'SECRET USER TEXT' }], + }, + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-message', role: 'assistant', parentID: currentUserID, + sessionID: 'collector-session', parts: [{ id: 'reused-part', type: 'text' }], + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'reused-part', messageID: 'assistant-message', type: 'text', + sessionID: 'collector-session', + }, + delta: 'Verified answer', + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'reused-part', type: 'tool', tool: 'bash', + state: { status: 'running' }, sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', partID: 'reused-part', + field: 'text', delta: 'STALE OWNERLESS TEXT', + }, + }, + stopMessage({ + id: 'assistant-message', role: 'assistant', parentID: currentUserID, + parts: [{ id: 'reused-part', type: 'text', text: 'Verified answer' }], + }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe('Verified answer'); + expect(result.content).not.toContain('SECRET'); + expect(result.content).not.toContain('STALE'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(deltas).toEqual(['Verified answer']); + }); + + test.each([ + ['out-of-turn', { parentID: 'previous-user' }], + ['timestamp-stale', { parentID: currentUserID, time: { created: Date.now() - 60_000 } }], + ])('does not attribute ownerless reused part events after a %s snapshot', async (_label, staleInfo) => { + const deltas = []; + const sharedPartID = 'reused-stale-part'; + const uniquePartID = 'unique-current-part'; + const result = await collect([ + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stale', + role: 'assistant', + sessionID: 'collector-session', + ...staleInfo, + parts: [{ id: sharedPartID, type: 'text', text: 'STALE SNAPSHOT' }], + }, + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + sessionID: 'collector-session', + parts: [ + { id: sharedPartID, type: 'text' }, + { id: uniquePartID, type: 'text' }, + ], + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: sharedPartID, + messageID: 'assistant-current', + type: 'text', + sessionID: 'collector-session', + }, + delta: 'Current explicit answer', + }, + }, + { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + partID: uniquePartID, + field: 'text', + delta: ' plus unique ownerless text', + }, + }, + { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + partID: sharedPartID, + field: 'text', + delta: 'LEAKED OWNERLESS TEXT', + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: sharedPartID, + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe('Current explicit answer'); + expect(result.content).not.toContain('STALE'); + expect(result.content).not.toContain('LEAKED'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(deltas).toEqual(['Current explicit answer']); + }); + + test.each(['text', 'reasoning'])('keeps ID-less %s snapshots isolated by message owner', async (type) => { + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + messageID: 'assistant-current', + parentID: currentUserID, + type, + text: 'verified current content', + sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + messageID: 'assistant-unverified', + type, + text: 'foreign owner overwrite', + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ]); + + expect(type === 'text' ? result.content : result.reasoning).toBe('verified current content'); + expect(result.content).not.toContain('foreign owner overwrite'); + expect(result.reasoning).not.toContain('foreign owner overwrite'); + }); + + test('an empty message ID cannot claim an ownerless ID-less text delta', async () => { + const deltas = []; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: 'collector-session' }, + delta: 'OWNERLESS TEXT MUST STAY QUARANTINED', + }, + }, + { + type: 'message.part.updated', + properties: { + part: { messageID: '', type: 'text', sessionID: 'collector-session' }, + }, + }, + stopMessage({ id: '', role: 'assistant', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe(''); + expect(result.content).not.toContain('OWNERLESS TEXT'); + expect(deltas).toEqual([]); + }); + + test('an empty message ID cannot inherit an ownerless running tool update', async () => { + const partID = 'empty-owner-tool'; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: '', + role: 'assistant', + parentID: currentUserID, + parts: [{ + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + }], + }), + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(result.idleTimeout).toBeUndefined(); + }); + + test('a reused tool part ID cannot inherit an ownerless update after ownership is ambiguous', async () => { + const partID = 'ambiguous-owner-tool'; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-stale', + role: 'assistant', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'running' } }], + }), + stopMessage({ + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'completed' } }], + }), + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(result.idleTimeout).toBeUndefined(); + }); + + test('an ambiguous part ID cannot attach or release an ownerless pending text delta', async () => { + const deltas = []; + const partID = 'ambiguous-pending-text'; + const result = await collect([ + { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + partID, + field: 'text', + delta: 'FOREIGN OWNERLESS PENDING TEXT', + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-stale', + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-current', + parentID: currentUserID, + type: 'text', + text: 'verified current answer', + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: partID, type: 'text', text: 'verified current answer' }], + }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe('verified current answer'); + expect(result.content).not.toContain('FOREIGN OWNERLESS PENDING TEXT'); + expect(deltas).not.toContain('FOREIGN OWNERLESS PENDING TEXT'); + }); + + test('does not stream an ownerless delta against a unique owner before later ID reuse is known', async () => { + const deltas = []; + const partID = 'late-reuse-ownerless-delta'; + const result = await collect([ + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + sessionID: 'collector-session', + parts: [{ id: partID, type: 'text', text: 'verified current answer' }], + }, + }, + }, + { + type: 'message.part.delta', + properties: { + sessionID: 'collector-session', + partID, + field: 'text', + delta: 'FOREIGN LATE OWNERLESS TEXT', + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stale', + role: 'assistant', + parentID: 'previous-user', + sessionID: 'collector-session', + parts: [{ id: partID, type: 'text', text: 'stale owner text' }], + }, + }, + }, + stopMessage({ id: 'assistant-current', role: 'assistant', parentID: currentUserID }), + ], (text) => deltas.push(text)); + + expect(result.content).toBe('verified current answer'); + expect(result.content).not.toContain('FOREIGN LATE OWNERLESS TEXT'); + expect(deltas).not.toContain('FOREIGN LATE OWNERLESS TEXT'); + }); + + test('an ambiguous text part ID cannot release an ownerless quarantined tool', async () => { + const partID = 'ambiguous-quarantined-tool'; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-stale', + role: 'assistant', + parts: [{ id: partID, type: 'text', text: 'stale unverified text' }], + }), + stopMessage({ + id: 'assistant-current', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: partID, type: 'text', text: 'verified answer' }], + }), + ]); + + expect(result.content).toBe('verified answer'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(result.idleTimeout).toBeUndefined(); + }); + + test('an ambiguous tool part is not released by a terminal snapshot with no message ID', async () => { + const partID = 'ambiguous-ownerless-terminal-tool'; + const result = await collect([ + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'running' }, + sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { id: partID, messageID: 'assistant-a', type: 'text', sessionID: 'collector-session' }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { id: partID, messageID: 'assistant-b', type: 'text', sessionID: 'collector-session' }, + }, + }, + stopMessage({ + role: 'assistant', + parentID: currentUserID, + parts: [{ id: partID, type: 'text', text: 'verified answer' }], + }), + ]); + + expect(result.content).toBe(''); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + expect(result.idleTimeout).toBeUndefined(); + }); + + test('a terminal tool update from another message cannot clear the active tool owner', async () => { + const partID = 'shared-active-tool-id'; + const result = await collect([ + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + finish: 'tool', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'running' } }], + }), + stopMessage({ + id: 'assistant-b', + role: 'assistant', + parentID: currentUserID, + finish: 'stop', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'completed' } }], + }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-a', + parentID: currentUserID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: 'assistant-a-answer', type: 'text', text: 'final answer after tool' }], + }), + ]); + + expect(result.content).toBe('final answer after tool'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('a later part-ID reuse revokes an ownerless terminal tool fallback', async () => { + const partID = 'ownerless-terminal-late-reuse'; + const result = await collect([ + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + finish: 'tool', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'running' } }], + }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-b', + role: 'assistant', + parentID: currentUserID, + finish: 'stop', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'completed' } }], + }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-a', + parentID: currentUserID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: 'assistant-a-answer', type: 'text', text: 'answer after A completes' }], + }), + ]); + + expect(result.content).toBe('answer after A completes'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('an out-of-turn reused part ID still revokes an ownerless tool fallback', async () => { + const partID = 'rejected-owner-reuse-tool'; + const result = await collect([ + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + finish: 'tool', + parts: [{ id: partID, type: 'tool', tool: 'bash', state: { status: 'running' } }], + }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ id: 'assistant-b', role: 'assistant', parentID: 'previous-user' }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-b', + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ id: 'assistant-a', role: 'assistant', parentID: currentUserID }), + { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID: 'assistant-a', + parentID: currentUserID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }, + stopMessage({ + id: 'assistant-a', + role: 'assistant', + parentID: currentUserID, + parts: [{ id: 'assistant-a-answer', type: 'text', text: 'answer after A completes' }], + }), + ]); + + expect(result.content).toBe('answer after A completes'); + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('non-continuation finalization discards ownerless pending deltas after ID ownership becomes ambiguous', async () => { + const partID = 'ambiguous-final-pending-text'; + const deltas = []; + const events = [ + { + type: 'message.part.delta', + properties: { sessionID: 'collector-session', partID, field: 'text', delta: 'FOREIGN PENDING TEXT' }, + }, + ...['assistant-a', 'assistant-b'].map((messageID) => ({ + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + })), + stopMessage({ id: 'assistant-a', role: 'assistant' }), + ]; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { yield* events; })(), + }), + }, + }; + + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, (text) => deltas.push(text), null, null); + + expect(result.content).toBe(''); + expect(result.content).not.toContain('FOREIGN PENDING TEXT'); + expect(deltas).toEqual([]); + }); + + test('non-continuation finalization discards ownerless text state after ID ownership becomes ambiguous', async () => { + const partID = 'ambiguous-final-text-state'; + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: partID, type: 'text', sessionID: 'collector-session' }, + delta: 'OWNERLESS TEXT STATE MUST NOT LEAK', + }, + }; + for (const messageID of ['assistant-a', 'assistant-b']) { + yield { + type: 'message.part.updated', + properties: { + part: { + id: partID, + messageID, + type: 'tool', + tool: 'bash', + state: { status: 'completed' }, + sessionID: 'collector-session', + }, + }, + }; + } + yield stopMessage({ id: 'assistant-a', role: 'assistant' }); + })(), + }), + }, + }; + + const result = await createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000); + + expect(result.content).toBe(''); + expect(result.content).not.toContain('OWNERLESS TEXT STATE'); + }); +}); + +describe('collector polling finish classification', () => { + const run = (snapshots, timeoutMs = 1000, expectedCurrentUserMessageId, requireTerminalCompletion = false) => { + let call = 0; + const client = { + session: { + messages: async () => { + const next = snapshots[Math.min(call, snapshots.length - 1)]; + call += 1; + return next; + } + } + }; + return createCollector({ client, logDebug: () => {} }).pollForAssistantResponse( + 'collector-session', timeoutMs, 1, undefined, expectedCurrentUserMessageId, requireTerminalCompletion, + ); + }; + const assistant = (info, text) => [{ info: { role: 'assistant', ...info }, parts: [{ type: 'text', text }] }]; + const priorTurnAndCurrentUser = (currentMessageID) => [ + { info: { id: 'assistant-old', role: 'assistant', parentID: 'msg-old-user', finish: 'stop' }, parts: [{ type: 'text', text: 'prior answer' }] }, + { info: { id: currentMessageID, role: 'user' }, parts: [{ id: 'user-part', type: 'text', text: 'current prompt' }] }, + ]; + + test.each([['tool'], ['tool-calls']])( + 'does not finish on time.completed while a turn awaits %s', + async (finish) => { + const result = await run([ + assistant({ finish, time: { completed: 1 } }, 'Calling a tool.'), + assistant({ finish: 'stop', time: { completed: 2 } }, 'Done: 42.') + ]); + expect(result.content).toBe('Done: 42.'); + expect(result.complete).toBe(true); + } + ); + + test('still finishes on time.completed for a terminal turn', async () => { + const result = await run([assistant({ time: { completed: 1 } }, 'Answer without a finish reason.')]); + expect(result.content).toBe('Answer without a finish reason.'); + expect(result.complete).toBe(true); + }); + + test('legacy polling returns an assistant error immediately without a finish reason', async () => { + let calls = 0; + const client = { + session: { + messages: async () => { + calls += 1; + return assistant({ error: { name: 'CreditsError', data: { message: '429: retry' } } }, ''); + }, + }, + }; + const result = await createCollector({ client, logDebug: () => {} }) + .pollForAssistantResponse('collector-session', 1000, 1); + + expect(result.error).toMatchObject({ name: 'CreditsError' }); + expect(result.complete).toBe(true); + expect(calls).toBe(1); + }); + + test('strict polling keeps an exact-turn assistant error incomplete without a finish reason', async () => { + const result = await run([ + assistant({ + id: 'assistant-incomplete-error', + parentID: 'msg-current-turn', + error: { name: 'ProviderError', message: 'still running' }, + }, 'partial answer'), + ], 30, 'msg-current-turn', true); + + expect(result).toMatchObject({ + assistantMessageId: 'assistant-incomplete-error', + userMessageId: 'msg-current-turn', + error: { name: 'ProviderError' }, + partial: true, + }); + expect(result.complete).toBeUndefined(); + }); + + test('returns a timeout partial with partial: true', async () => { + const result = await run([assistant({ finish: 'tool' }, 'Still working')], 100); + expect(result).toMatchObject({ content: 'Still working', partial: true }); + expect(result.complete).toBeUndefined(); + }); + + test('ignores a prior completed assistant turn until the current continuation assistant appears', async () => { + const currentMessageID = 'msg-current-turn'; + let call = 0; + const snapshots = [ + priorTurnAndCurrentUser(currentMessageID), + [ + ...priorTurnAndCurrentUser(currentMessageID), + { info: { id: 'assistant-current', role: 'assistant', parentID: currentMessageID, finish: 'stop' }, parts: [{ type: 'text', text: 'current answer' }] }, + ], + ]; + const client = { + session: { + messages: async () => snapshots[Math.min(call++, snapshots.length - 1)], + }, + }; + const result = await createCollector({ client, logDebug: () => {} }).pollForAssistantResponse( + 'collector-session', 1000, 1, undefined, currentMessageID, + ); + expect(result.content).toBe('current answer'); + expect(result.assistantMessageId).toBe('assistant-current'); + expect(call).toBeGreaterThan(1); + }); + + test('does not accept a prior completed assistant when the current assistant is absent', async () => { + await expect(run([priorTurnAndCurrentUser('msg-current-turn')], 30, 'msg-current-turn')) + .rejects.toThrow('Request timeout after 30ms'); + }); + + test('does not accept a timestamp-less completed assistant without a positively matching parent', async () => { + const currentMessageID = 'msg-current-turn'; + await expect(run([ + [ + { info: { id: 'assistant-no-parent', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'stale answer' }] }, + { info: { id: currentMessageID, role: 'user' }, parts: [] }, + ], + ], 30, currentMessageID)).rejects.toThrow('Request timeout after 30ms'); }); test.each(['pending', 'running'])('does not finish while a tool part is %s', async (status) => { diff --git a/tests/stream-phase2.test.js b/tests/stream-phase2.test.js new file mode 100644 index 0000000..bef12fc --- /dev/null +++ b/tests/stream-phase2.test.js @@ -0,0 +1,1174 @@ +import request from 'supertest'; +import { jest } from '@jest/globals'; +import fs from 'fs'; +import path from 'path'; + +const SESSION_ID = 'phase2-session'; + +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { + providers: [{ + id: 'opencode', + models: { + 'big-pickle': { name: 'Big Pickle' }, + 'kimi-k2.5': { name: 'Kimi' } + } + }] + } + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ({ data: ['web_fetch', 'filesystem', 'bash'] })), + sessionCreate: jest.fn(async () => ({ data: { id: SESSION_ID } })), + sessionPrompt: jest.fn(async () => ({ data: { parts: [{ type: 'text', text: 'Mock response' }] } })), + sessionMessages: jest.fn(async () => ([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Mock response' }] } + ])), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ + stream: (async function* () { + yield { type: 'message.part.updated', properties: { part: { type: 'text', sessionID: SESSION_ID }, delta: 'Hello' } }; + yield { type: 'message.updated', properties: { info: { sessionID: SESSION_ID, finish: 'stop' } } }; + })() + })) +}; + +jest.unstable_mockModule('https', () => ({ + default: { + get: jest.fn((url, options, callback) => { + const res = { statusCode: 200, headers: {}, on: jest.fn() }; + callback(res); + return { on: jest.fn(), destroy: jest.fn() }; + }) + } +})); + +jest.unstable_mockModule('http', () => ({ + default: { + get: jest.fn((url, options, callback) => { + const res = { statusCode: 200, headers: {}, on: jest.fn() }; + callback(res); + return { on: jest.fn(), destroy: jest.fn(), setTimeout: jest.fn() }; + }) + } +})); + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { + create: sdkMocks.sessionCreate, + prompt: sdkMocks.sessionPrompt, + messages: sdkMocks.sessionMessages, + delete: sdkMocks.sessionDelete + }, + event: { subscribe: sdkMocks.eventSubscribe } + })) +})); + +const { createApp } = await import('../src/proxy.js'); +const { createCollector } = await import('../src/stream/collector.js'); + +describe('Phase 2: SSE Framing, Keepalives, Headers, and Wire Integrity', () => { + beforeEach(() => { + jest.clearAllMocks(); + sdkMocks.sessionMessages.mockReset(); + sdkMocks.sessionMessages.mockImplementation(async () => ([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Mock response' }] } + ])); + sdkMocks.eventSubscribe.mockReset(); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { type: 'message.updated', properties: { info: { id: 'assistant-default', role: 'assistant', sessionID: SESSION_ID } } }; + yield { type: 'message.part.updated', properties: { part: { id: 'part-default', messageID: 'assistant-default', type: 'text', sessionID: SESSION_ID }, delta: 'Hello' } }; + yield { type: 'message.updated', properties: { info: { id: 'assistant-default', role: 'assistant', sessionID: SESSION_ID, finish: 'stop' } } }; + })() + })); + }); + + const buildApp = (overrides = {}) => createApp({ + PORT: 10000, + API_KEY: 'test-key', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 5000, + RETRY_MAX_RETRIES: 0, + DISABLE_TOOLS: true, + DEBUG: false, + ...overrides + }).app; + + describe('1. X-Accel-Buffering: no header', () => { + test('POST /v1/chat/completions carries X-Accel-Buffering: no when streaming', async () => { + const res = await request(buildApp()) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', messages: [{ role: 'user', content: 'hi' }], stream: true }); + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.headers['x-accel-buffering']).toBe('no'); + }); + + test('POST /v1/messages carries X-Accel-Buffering: no when streaming', async () => { + const res = await request(buildApp()) + .post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send({ model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }], stream: true }); + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.headers['x-accel-buffering']).toBe('no'); + }); + + test('POST /v1/responses carries X-Accel-Buffering: no when streaming', async () => { + const res = await request(buildApp()) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.headers['x-accel-buffering']).toBe('no'); + }); + + test('POST /v1/interactions carries X-Accel-Buffering: no when streaming', async () => { + const res = await request(buildApp()) + .post('/v1/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.headers['x-accel-buffering']).toBe('no'); + }); + + test('POST /v1beta/interactions carries X-Accel-Buffering: no when streaming', async () => { + const res = await request(buildApp()) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.headers['x-accel-buffering']).toBe('no'); + }); + }); + + describe('2. Keepalive framing', () => { + test('route files do not contain double-newline keepalives/heartbeats', () => { + const routeFiles = ['chat.ts', 'messages.ts', 'responses.ts', 'interactions.ts']; + for (const file of routeFiles) { + const content = fs.readFileSync(path.join(process.cwd(), 'src/routes', file), 'utf8'); + expect(content).not.toContain("': keepalive\\n\\n'"); + expect(content).not.toContain("': keep-alive\\n\\n'"); + expect(content).not.toContain("': heartbeat\\n\\n'"); + } + }); + }); + + describe('3. Anthropic Content Block Closing Before Error', () => { + test('mid-stream text block error emits content_block_stop before event: error and no [DONE]', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-message', role: 'assistant', sessionID: SESSION_ID } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'text', sessionID: SESSION_ID }, delta: 'Hello text' } + }; + yield { + type: 'session.error', + properties: { sessionID: SESSION_ID, error: { message: 'midstream error occurred' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send({ model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }], stream: true }); + + expect(res.text).toContain('content_block_start'); + expect(res.text).toContain('content_block_stop'); + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('[DONE]'); + + const blockStopPos = res.text.indexOf('event: content_block_stop'); + const errorPos = res.text.indexOf('event: error'); + expect(blockStopPos).toBeGreaterThan(-1); + expect(errorPos).toBeGreaterThan(-1); + expect(blockStopPos).toBeLessThan(errorPos); + }); + + test('mid-stream reasoning/thinking block error emits content_block_stop before event: error', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-message', role: 'assistant', sessionID: SESSION_ID } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'reasoning', sessionID: SESSION_ID }, delta: 'Thinking deep thoughts' } + }; + yield { + type: 'session.error', + properties: { sessionID: SESSION_ID, error: { message: 'thinking crash' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send({ model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }], stream: true }); + + expect(res.text).toContain('content_block_start'); + expect(res.text).toContain('"type":"thinking"'); + expect(res.text).toContain('content_block_stop'); + expect(res.text).toContain('event: error'); + + const blockStopPos = res.text.indexOf('event: content_block_stop'); + const errorPos = res.text.indexOf('event: error'); + expect(blockStopPos).toBeGreaterThan(-1); + expect(errorPos).toBeGreaterThan(-1); + expect(blockStopPos).toBeLessThan(errorPos); + }); + }); + + describe('4. Outer Catch Stream Closure & Wire Integrity', () => { + test('chat streaming error wire shape: data: {"error":{...}} then close without [DONE]', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'session.error', + properties: { sessionID: SESSION_ID, error: { message: 'Immediate chat backend error' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', messages: [{ role: 'user', content: 'hi' }], stream: true }); + + expect(res.text).toContain('data: {"error":'); + expect(res.text).not.toContain('data: [DONE]'); + }); + + test('chat streaming error ends response even if res.write fails', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'session.error', + properties: { sessionID: SESSION_ID, error: { message: 'Write fail error' } } + }; + })() + })); + + const app = buildApp(); + const server = (req, res) => { + const origWrite = res.write.bind(res); + res.write = function (chunk, ...args) { + if (typeof chunk === 'string' && chunk.includes('data: {"error"')) { + throw new Error('Broken pipe during error write'); + } + return origWrite(chunk, ...args); + }; + app(req, res); + }; + + const res = await request(server) + .post('/v1/chat/completions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', messages: [{ role: 'user', content: 'hi' }], stream: true }); + + expect(res.statusCode).toBe(200); + }); + + test('responses streaming error wire shape: response.failed then data: [DONE]', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'session.error', + properties: { sessionID: SESSION_ID, error: { message: 'Responses backend error' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.text).toContain('response.failed'); + expect(res.text).toContain('data: [DONE]'); + }); + + test('interactions streaming error wire shape: data: {"type":"error",...} then close without [DONE]', async () => { + sdkMocks.sessionPrompt.mockImplementationOnce(async () => { + throw new Error('Interactions failure'); + }); + + const res = await request(buildApp()) + .post('/v1/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.text).toContain('data: {"type":"error"'); + expect(res.text).not.toContain('[DONE]'); + }); + + test.each([ + ['chat', '/v1/chat/completions', { model: 'big-pickle', stream: true, messages: [{ role: 'user', content: 'hi' }] }], + ['messages', '/v1/messages', { model: 'big-pickle', stream: true, max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }], + ['responses', '/v1/responses', { model: 'big-pickle', stream: true, input: 'hi' }], + ['interactions', '/v1/interactions', { model: 'big-pickle', stream: true, input: 'hi' }] + ])('%s outer catch ends connection when backend throws error after headersSent', async (_name, endpoint, payload) => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => { + throw new Error('Fatal subscription failure after headersSent'); + }); + sdkMocks.sessionPrompt.mockImplementationOnce(async () => { + throw new Error('Fatal prompt failure after headersSent'); + }); + sdkMocks.sessionMessages.mockImplementationOnce(async () => { + throw new Error('Fatal recovery poll failure after headersSent'); + }); + const res = await request(buildApp()) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('error'); + }); + }); + + describe('5. Collector: Natural iterator completion finalizes and clears timers', () => { + const run = (events, firstDeltaTimeoutMs = null, idleTimeoutMs = null) => { + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + for (const event of events) { + yield event; + } + })() + }) + } + }; + return createCollector({ client, logDebug: () => {} }) + .collectFromEvents('collector-session', 1000, null, firstDeltaTimeoutMs, idleTimeoutMs); + }; + + test('resolves with cut: true and noData: true when stream iterator finishes with no events', async () => { + const result = await run([]); + expect(result).toMatchObject({ + content: '', + reasoning: '', + cut: true, + noData: true + }); + }); + + test('resolves with cut: true and partial content when stream iterator ends without terminal finish', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, + delta: 'Incomplete sentence without stop' + } + } + ]); + expect(result).toMatchObject({ + content: 'Incomplete sentence without stop', + cut: true + }); + expect(result.noData).toBeUndefined(); + }); + + test('resolves cleanly without cut when terminal stop is received', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, + delta: 'Complete sentence' + } + }, + { + type: 'message.updated', + properties: { + info: { sessionID: 'collector-session', finish: 'stop' } + } + } + ]); + expect(result).toMatchObject({ + content: 'Complete sentence', + finish: 'stop' + }); + expect(result.cut).toBeUndefined(); + }); + + test('resolves with finish: tool and cut: true when iterator ends after tool finish', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, + delta: 'Tool call output' + } + }, + { + type: 'message.updated', + properties: { + info: { sessionID: 'collector-session', finish: 'tool' } + } + } + ]); + expect(result).toMatchObject({ + content: 'Tool call output', + finish: 'tool', + cut: true + }); + expect(result.cut).toBe(true); + }); + + test('resolves with cut: true when iterator ends on intermediate tool-calls finish', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, + delta: 'Calling tools' + } + }, + { + type: 'message.updated', + properties: { + info: { sessionID: 'collector-session', finish: 'tool-calls' } + } + } + ]); + expect(result).toMatchObject({ + content: 'Calling tools', + finish: 'tool-calls', + cut: true + }); + expect(result.cut).toBe(true); + }); + + test('resolves with cut: true when iterator ends after intermediate stop while tools are active', async () => { + const result = await run([ + { + type: 'message.part.updated', + properties: { + part: { id: 'tool-1', type: 'tool', sessionID: 'collector-session', state: { status: 'running' } } + } + }, + { + type: 'message.updated', + properties: { + info: { sessionID: 'collector-session', finish: 'stop' } + } + } + ]); + expect(result.cut).toBe(true); + }); + + test('natural iterator completion clears timeoutId, firstDeltaTimer, and idleTimer', async () => { + const clearTimeoutSpy = jest.spyOn(global, 'clearTimeout'); + try { + await run([ + { + type: 'message.part.updated', + properties: { part: { id: 'p1', type: 'text', sessionID: 'collector-session' }, delta: 'chunk' } + } + ], 500, 500); + expect(clearTimeoutSpy).toHaveBeenCalled(); + } finally { + clearTimeoutSpy.mockRestore(); + } + }); + + const runContinuation = (events) => { + const client = { + event: { + subscribe: async () => ({ + stream: (async function* () { + for (const event of events) yield event; + })(), + }), + }, + }; + return createCollector({ client, logDebug: () => {} }).collectFromEvents( + 'collector-session', + 1000, + null, + null, + null, + undefined, + 'expected-user', + ); + }; + + test('correlates ownerless and owned tool updates by part ID and releases the newest terminal state', async () => { + const result = await runContinuation([ + { + type: 'message.part.updated', + properties: { + part: { id: 'logical-tool-part', type: 'tool', sessionID: 'collector-session', state: { status: 'running' } }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'logical-tool-part', + messageID: 'assistant-current', + type: 'tool', + sessionID: 'collector-session', + state: { status: 'completed' }, + }, + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'expected-user', + sessionID: 'collector-session', + finish: 'stop', + parts: [{ id: 'logical-tool-part', type: 'tool', state: { status: 'running' } }], + }, + }, + }, + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test("ownerless terminal tool update settles the confirmed owner's active state before a sparse stop", async () => { + const result = await runContinuation([ + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'expected-user', + sessionID: 'collector-session', + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'owned-tool-part', + messageID: 'assistant-current', + type: 'tool', + sessionID: 'collector-session', + state: { status: 'running' }, + }, + }, + }, + { + type: 'message.part.updated', + properties: { + part: { + id: 'owned-tool-part', + type: 'tool', + sessionID: 'collector-session', + state: { status: 'completed' }, + }, + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + sessionID: 'collector-session', + finish: 'stop', + }, + }, + }, + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBeUndefined(); + }); + + test('a stale completed snapshot cannot clear a live tool update before stop', async () => { + const result = await runContinuation([ + { + type: 'message.part.updated', + properties: { + part: { id: 'live-tool-part', type: 'tool', sessionID: 'collector-session', state: { status: 'running' } }, + }, + }, + { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: 'expected-user', + sessionID: 'collector-session', + finish: 'stop', + parts: [{ id: 'live-tool-part', type: 'tool', state: { status: 'completed' } }], + }, + }, + }, + ]); + + expect(result.finish).toBe('stop'); + expect(result.cut).toBe(true); + }); + }); + + describe('6. Premature cut and divergence fail closed (P1-1 & P1-2)', () => { + const routeCases = [ + ['chat', '/v1/chat/completions', { model: 'big-pickle', stream: true, messages: [{ role: 'user', content: 'hi' }] }], + ['messages', '/v1/messages', { model: 'big-pickle', stream: true, max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }], + ['responses', '/v1/responses', { model: 'big-pickle', stream: true, input: 'hi' }] + ]; + + test.each(routeCases)('%s delta -> cut -> recovery poll throws 503 fails closed without success terminal', async (name, endpoint, payload) => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', type: 'text', sessionID: SESSION_ID }, delta: 'Hello text' } + }; + })() + })); + sdkMocks.sessionMessages.mockImplementationOnce(async () => { + const err = new Error('Backend 503 Service Unavailable'); + err.status = 503; + throw err; + }); + + const res = await request(buildApp()) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.headers['content-type']).toContain('text/event-stream'); + if (name === 'chat') { + expect(res.text).toContain('error'); + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); + } else if (name === 'messages') { + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('end_turn'); + expect(res.text).not.toContain('message_delta'); + } else if (name === 'responses') { + expect(res.text).toContain('response.failed'); + expect(res.text).not.toContain('response.completed'); + } + }); + + test.each(routeCases)('%s delta -> cut -> recovery poll returns partial: true fails closed without success terminal', async (name, endpoint, payload) => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', type: 'text', sessionID: SESSION_ID }, delta: 'Hello text' } + }; + })() + })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { role: 'assistant' }, + parts: [{ type: 'text', text: 'Partial text that times out' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40 })) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('Recovery poll returned incomplete response'); + if (name === 'chat') { + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); + } else if (name === 'messages') { + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('end_turn'); + expect(res.text).not.toContain('message_delta'); + } else if (name === 'responses') { + expect(res.text).toContain('response.failed'); + expect(res.text).not.toContain('response.completed'); + } + }); + + test.each(routeCases)('%s raw = "AC" / snapshot = "ABC" divergence on unbuffered text stream fails closed with protocol error', async (name, endpoint, payload) => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-message', role: 'assistant', sessionID: SESSION_ID } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'text', sessionID: SESSION_ID }, delta: 'AC' } + }; + })() + })); + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'ABC' }] + } + ]); + + const res = await request(buildApp()) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('Stream content diverged from authoritative snapshot'); + if (name === 'chat') { + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); + } else if (name === 'messages') { + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('end_turn'); + expect(res.text).not.toContain('message_delta'); + } else if (name === 'responses') { + expect(res.text).toContain('response.failed'); + expect(res.text).not.toContain('response.completed'); + } + }); + }); + + describe('7. Poll completion validation and fail-closed behavior (P1-1)', () => { + const streamCases = [ + ['chat', '/v1/chat/completions', { model: 'big-pickle', stream: true, messages: [{ role: 'user', content: 'hi' }] }], + ['messages', '/v1/messages', { model: 'big-pickle', stream: true, max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }], + ['responses', '/v1/responses', { model: 'big-pickle', stream: true, input: 'hi' }] + ]; + + const nonStreamCases = [ + ['chat', '/v1/chat/completions', { model: 'big-pickle', messages: [{ role: 'user', content: 'hi' }] }], + ['messages', '/v1/messages', { model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }], + ['responses', '/v1/responses', { model: 'big-pickle', input: 'hi' }] + ]; + + function freeUsageError() { + return { + name: 'APIError', + data: { + message: 'Rate limit exceeded. Please try again later.', + statusCode: 429, + isRetryable: true, + responseBody: '{"type":"error","error":{"type":"FreeUsageLimitError","message":"Rate limit exceeded."}}', + }, + }; + } + + const buildProxyApp = (overrides = {}) => createApp({ + PORT: 10000, + API_KEY: 'test-key', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 40, + RETRY_MAX_RETRIES: 1, + DISABLE_TOOLS: true, + DEBUG: false, + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + ...overrides, + }).app; + + async function armProxyFallback(app) { + sdkMocks.sessionPrompt.mockRejectedValueOnce(freeUsageError()); + const armRes = await request(app) + .post('/v1/messages') + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send({ model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }); + expect(armRes.statusCode).toBe(200); + } + + test.each(streamCases)('%s proxy-mode stream when poll returns partial: true fails closed without success terminal', async (name, endpoint, payload) => { + const app = buildProxyApp(); + await armProxyFallback(app); + + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { role: 'assistant' }, + parts: [{ type: 'text', text: 'Partial text that times out' }] + }])); + + const res = await request(app) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.headers['content-type']).toContain('text/event-stream'); + if (name === 'chat') { + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); + } else if (name === 'messages') { + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('end_turn'); + } else if (name === 'responses') { + expect(res.text).toContain('response.failed'); + expect(res.text).not.toContain('response.completed'); + } + }); + + test.each(nonStreamCases)('%s non-stream when poll returns partial: true fails closed with error status', async (name, endpoint, payload) => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { role: 'assistant' }, + parts: [{ type: 'text', text: 'Partial text that times out' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.statusCode).not.toBe(200); + }); + + test.each([ + ['chat', '/v1/chat/completions', { model: 'big-pickle', messages: [{ role: 'user', content: 'hi' }] }], + ['messages', '/v1/messages', { model: 'big-pickle', max_tokens: 100, messages: [{ role: 'user', content: 'hi' }] }], + ])('%s non-stream fails closed when polling returns an empty completed assistant', async (_name, endpoint, payload) => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { id: 'empty-assistant', role: 'assistant', finish: 'stop' }, + parts: [], + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post(endpoint) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.statusCode).not.toBe(200); + expect(JSON.stringify(res.body)).toContain('Polling returned empty response'); + }); + + test.each([false, true])('interactions %s fails closed when polling returns an empty completed assistant', async (stream) => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { id: 'empty-assistant', role: 'assistant', finish: 'stop' }, + parts: [], + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', ...(stream ? { stream: true } : {}) }); + + if (stream) { + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('"type":"error"'); + expect(res.text).not.toContain('interaction.completed'); + } else { + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.statusCode).not.toBe(200); + expect(res.body?.status).not.toBe('completed'); + expect(JSON.stringify(res.body)).toContain('Polling returned empty response'); + } + }); + + test('interactions non-stream when poll returns partial: true fails closed with error status', async () => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { role: 'assistant' }, + parts: [{ type: 'text', text: 'Partial interaction text' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi' }); + + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.statusCode).not.toBe(200); + expect(res.body?.status).not.toBe('completed'); + }); + + test('interactions stream when poll returns partial: true fails closed without interaction.completed', async () => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { role: 'assistant' }, + parts: [{ type: 'text', text: 'Partial interaction text' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('"type":"error"'); + expect(res.text).not.toContain('interaction.completed'); + }); + + test('interactions non-stream when poll returns error != null with text fails closed with error status', async () => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { + role: 'assistant', + finish: 'stop', + error: { name: 'AssistantError', message: 'Failed mid-turn' } + }, + parts: [{ type: 'text', text: 'Some text generated before error' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi' }); + + expect(res.statusCode).toBeGreaterThanOrEqual(400); + expect(res.statusCode).not.toBe(200); + expect(res.body?.status).not.toBe('completed'); + }); + + test('interactions stream when poll returns error != null with text fails closed without interaction.completed', async () => { + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { + role: 'assistant', + finish: 'stop', + error: { name: 'AssistantError', message: 'Failed mid-turn' } + }, + parts: [{ type: 'text', text: 'Some text generated before error' }] + }])); + + const res = await request(buildApp({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })) + .post('/v1beta/interactions') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('"type":"error"'); + expect(res.text).not.toContain('interaction.completed'); + }); + }); + + describe('8. Responses streaming whitespace preservation and authoritative snapshot', () => { + test('POST /v1/responses streaming preserves whitespace deltas between words without dropping spaces', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-message', role: 'assistant', sessionID: SESSION_ID } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'text', sessionID: SESSION_ID }, delta: 'Hello' } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'text', sessionID: SESSION_ID }, delta: ' ' } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-message', type: 'text', sessionID: SESSION_ID }, delta: 'world' } + }; + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-message', role: 'assistant', sessionID: SESSION_ID, finish: 'stop' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.statusCode).toBe(200); + const lines = res.text.split('\n'); + const deltaEvents = lines + .filter((line) => line.startsWith('data: ') && !line.includes('[DONE]')) + .map((line) => JSON.parse(line.slice(6))) + .filter((event) => event.type === 'response.output_text.delta'); + + const deltas = deltaEvents.map((e) => e.delta); + expect(deltas).toContain(' '); + expect(deltas.join('')).toBe('Hello world'); + }); + + test('POST /v1/responses streaming with hosted search (deferVisibleOutput = true) updates content from authoritative snapshot when deltas are lost', async () => { + sdkMocks.toolIds.mockResolvedValueOnce({ data: ['websearch', 'web_fetch', 'filesystem', 'bash'] }); + sdkMocks.sessionPrompt.mockResolvedValueOnce({ data: { parts: [] } }); + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [ + { type: 'text', text: 'Authoritative search answer' }, + { + type: 'tool', + id: 'call_search_1', + tool: 'websearch', + state: { + status: 'completed', + input: { query: 'test query' }, + output: 'Search result: https://example.com/info' + } + } + ] + } + ]); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + // Deltas are lost: only terminal message update arrives with the complete part snapshot + yield { + type: 'message.updated', + properties: { + info: { + id: 'msg-1', + role: 'assistant', + sessionID: SESSION_ID, + finish: 'stop', + parts: [ + { id: 'p1', messageID: 'msg-1', type: 'text', text: 'Authoritative search answer' }, + { + id: 'call_search_1', + type: 'tool', + tool: 'websearch', + state: { + status: 'completed', + input: { query: 'test query' }, + output: 'Search result: https://example.com/info' + } + } + ] + } + } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'big-pickle', + input: 'search for something', + tools: [{ type: 'web_search' }], + stream: true + }); + + expect(res.statusCode).toBe(200); + expect(res.headers['content-type']).toContain('text/event-stream'); + expect(res.text).toContain('response.completed'); + expect(res.text).not.toContain('response.failed'); + + const lines = res.text.split('\n'); + const deltaEvents = lines + .filter((line) => line.startsWith('data: ') && !line.includes('[DONE]')) + .map((line) => JSON.parse(line.slice(6))) + .filter((event) => event.type === 'response.output_text.delta'); + + const deltas = deltaEvents.map((e) => e.delta); + expect(deltas.join('')).toContain('Authoritative search answer'); + + const completedLine = lines.find((line) => line.startsWith('data: ') && line.includes('response.completed')); + expect(completedLine).toBeDefined(); + const completedEvent = JSON.parse(completedLine.slice(6)); + const msgItem = completedEvent.response.output.find((item) => item.type === 'message'); + expect(msgItem).toBeDefined(); + expect(msgItem.content[0].text).toBe('Authoritative search answer'); + }); + + test('POST /v1/responses streaming preserves leading whitespace in output_text.delta', async () => { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-ws', role: 'assistant', sessionID: SESSION_ID } } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-ws', type: 'text', sessionID: SESSION_ID }, delta: '\n\n' } + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'p1', messageID: 'assistant-ws', type: 'text', sessionID: SESSION_ID }, delta: 'Hello world' } + }; + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-ws', role: 'assistant', sessionID: SESSION_ID, finish: 'stop' } } + }; + })() + })); + + const res = await request(buildApp()) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'big-pickle', input: 'hi', stream: true }); + + expect(res.statusCode).toBe(200); + const lines = res.text.split('\n'); + const deltaEvents = lines + .filter((line) => line.startsWith('data: ') && !line.includes('[DONE]')) + .map((line) => JSON.parse(line.slice(6))) + .filter((event) => event.type === 'response.output_text.delta'); + + const deltas = deltaEvents.map((e) => e.delta); + expect(deltas[0]).toBe('\n\n'); + expect(deltas.join('')).toBe('\n\nHello world'); + }); + }); + + describe('9. Collector: pollForAssistantResponse parentId/parent_id casing resilience', () => { + test('recognizes parentId casing for filtering and output', async () => { + const client = { + session: { + messages: async () => ([ + { info: { id: 'asst-1', role: 'assistant', parentId: 'user-msg-1', finish: 'stop' }, parts: [{ type: 'text', text: 'Answer 1' }] } + ]) + } + }; + const collector = createCollector({ client, logDebug: () => {} }); + const result = await collector.pollForAssistantResponse('collector-session', 1000, 1, undefined, 'user-msg-1'); + expect(result.content).toBe('Answer 1'); + expect(result.userMessageId).toBe('user-msg-1'); + expect(result.complete).toBe(true); + }); + + test('recognizes parent_id casing for filtering and output', async () => { + const client = { + session: { + messages: async () => ([ + { info: { id: 'asst-2', role: 'assistant', parent_id: 'user-msg-2', finish: 'stop' }, parts: [{ type: 'text', text: 'Answer 2' }] } + ]) + } + }; + const collector = createCollector({ client, logDebug: () => {} }); + const result = await collector.pollForAssistantResponse('collector-session', 1000, 1, undefined, 'user-msg-2'); + expect(result.content).toBe('Answer 2'); + expect(result.userMessageId).toBe('user-msg-2'); + expect(result.complete).toBe(true); + }); + }); +}); diff --git a/tests/stream-retries.test.js b/tests/stream-retries.test.js new file mode 100644 index 0000000..467b0b5 --- /dev/null +++ b/tests/stream-retries.test.js @@ -0,0 +1,2032 @@ +import request from 'supertest'; +import { jest } from '@jest/globals'; +import net from 'node:net'; + +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { + providers: [{ + id: 'opencode', + models: { + 'kimi-k2.5': { name: 'Kimi k2.5' } + } + }] + } + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ({ data: ['web_fetch', 'filesystem', 'bash'] })), + sessionCreate: jest.fn(async () => ({ data: { id: 'default-session' } })), + sessionPrompt: jest.fn(async () => ({ data: { parts: [] } })), + sessionFork: jest.fn(async () => ({ data: { id: `fork-session-${sdkMocks.sessionFork.mock.calls.length}` } })), + sessionMessages: jest.fn(async (args) => ([ + { + info: { + id: `assistant-${args?.path?.id || 'default-session'}`, + role: 'assistant', + parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, + finish: 'stop' + }, + parts: [{ type: 'text', text: 'Default response' }] + } + ])), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ + stream: (async function* () { + const prompt = sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]; + const sessionID = prompt?.path?.id || 'default-session'; + yield { type: 'message.updated', properties: { info: { + id: `assistant-${sessionID}`, + role: 'assistant', + parentID: prompt?.body?.messageID, + sessionID, + finish: 'stop' + } } }; + })() + })) +}; + +const httpMocks = { + get: jest.fn((_url, _options, callback) => { + const cb = typeof _options === 'function' ? _options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn(), setTimeout: jest.fn() }; + }) +}; + +jest.unstable_mockModule('http', () => ({ + default: { + get: httpMocks.get + } +})); + +jest.unstable_mockModule('https', () => ({ + default: { + get: jest.fn((_url, _options, callback) => { + const cb = typeof _options === 'function' ? _options : callback; + cb({ statusCode: 200, headers: {}, on: jest.fn() }); + return { on: jest.fn(), destroy: jest.fn() }; + }) + } +})); + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { + create: sdkMocks.sessionCreate, + fork: sdkMocks.sessionFork, + prompt: sdkMocks.sessionPrompt, + messages: sdkMocks.sessionMessages, + delete: sdkMocks.sessionDelete + }, + event: { subscribe: sdkMocks.eventSubscribe } + })) +})); + +const { isTransientUpstreamError } = await import('../src/errors/upstream.js'); +const { createApp, startProxy } = await import('../src/proxy.js'); +const { + lock, + notifyBackendReconnect, + clearBackendReconnectListeners, + getBackendReconnectListenerCount, +} = await import('../src/backend/manager.js'); + +function freeUsageError() { + return { + name: 'APIError', + data: { + message: 'Rate limit exceeded. Please try again later.', + statusCode: 429, + isRetryable: true, + responseBody: '{"type":"error","error":{"type":"FreeUsageLimitError","message":"Rate limit exceeded."}}', + }, + }; +} + +function baseConfig(overrides = {}) { + return { + PORT: 10000, + API_KEY: '', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 3000, + RETRY_MAX_RETRIES: 2, + DISABLE_TOOLS: true, + ...overrides + }; +} + +describe('isTransientUpstreamError', () => { + test('matches { data: { statusCode: 503 } }', () => { + expect(isTransientUpstreamError({ data: { statusCode: 503 } })).toBe(true); + }); + + test('matches "Endpoint is unavailable."', () => { + expect(isTransientUpstreamError({ message: 'Endpoint is unavailable.' })).toBe(true); + expect(isTransientUpstreamError({ data: { message: 'Endpoint is unavailable.' } })).toBe(true); + expect(isTransientUpstreamError(new Error('Endpoint is unavailable.'))).toBe(true); + expect(isTransientUpstreamError({ data: { statusCode: 503, message: 'Endpoint is unavailable.' } })).toBe(true); + }); + + test('does not match non-transient errors', () => { + expect(isTransientUpstreamError({ data: { statusCode: 400 } })).toBe(false); + expect(isTransientUpstreamError({ message: 'invalid request parameters' })).toBe(false); + expect(isTransientUpstreamError({ message: 'context length exceeded', data: { statusCode: 503 } })).toBe(false); + }); + + test('rejects HTTP 400, 403, 404 even if message has "endpoint is unavailable", but accepts 503 and exact "endpoint is unavailable"', () => { + expect(isTransientUpstreamError({ statusCode: 400, message: 'Endpoint is unavailable.' })).toBe(false); + expect(isTransientUpstreamError({ data: { statusCode: 400, message: 'Endpoint is unavailable.' } })).toBe(false); + expect(isTransientUpstreamError({ statusCode: 403, message: 'Endpoint is unavailable.' })).toBe(false); + expect(isTransientUpstreamError({ data: { statusCode: 403 }, message: 'Endpoint is unavailable.' })).toBe(false); + expect(isTransientUpstreamError({ statusCode: 404, message: 'Endpoint is unavailable.' })).toBe(false); + expect(isTransientUpstreamError({ data: { statusCode: 404 }, message: 'Endpoint is unavailable.' })).toBe(false); + expect(isTransientUpstreamError(new Error('400: Endpoint is unavailable.'))).toBe(false); + expect(isTransientUpstreamError(new Error('404: Endpoint is unavailable.'))).toBe(false); + expect(isTransientUpstreamError({ data: { statusCode: 503 } })).toBe(true); + expect(isTransientUpstreamError({ statusCode: 503 })).toBe(true); + expect(isTransientUpstreamError({ message: 'Endpoint is unavailable.' })).toBe(true); + expect(isTransientUpstreamError(new Error('Endpoint is unavailable.'))).toBe(true); + expect(isTransientUpstreamError({ message: 'Some other error: Endpoint is unavailable. Check endpoint' })).toBe(false); + }); + + test('enforces status code and auth filters before isRetryable flag', () => { + expect(isTransientUpstreamError({ statusCode: 400, isRetryable: true })).toBe(false); + expect(isTransientUpstreamError({ statusCode: 404, isRetryable: true })).toBe(false); + expect(isTransientUpstreamError({ statusCode: 401, message: 'Invalid API key', isRetryable: true })).toBe(false); + expect(isTransientUpstreamError({ statusCode: 408, isRetryable: true })).toBe(true); + expect(isTransientUpstreamError({ statusCode: 429, isRetryable: true })).toBe(true); + expect(isTransientUpstreamError({ statusCode: 503 })).toBe(true); + }); +}); + +describe('Streaming Retries on 503 Transient Error', () => { + let app; + let ctx; + + beforeEach(() => { + jest.clearAllMocks(); + const created = createApp(baseConfig({})); + app = created.app; + ctx = created.ctx; + }); + + test.each([ + ['Chat non-stream', '/v1/chat/completions', { model: 'opencode/kimi-k2.5', messages: [{ role: 'user', content: 'look up' }], tools: [{ type: 'function', function: { name: 'lookup', description: 'Lookup', parameters: { type: 'object', properties: {} } } }], tool_choice: 'required' }], + ['Chat stream', '/v1/chat/completions', { model: 'opencode/kimi-k2.5', messages: [{ role: 'user', content: 'look up' }], tools: [{ type: 'function', function: { name: 'lookup', description: 'Lookup', parameters: { type: 'object', properties: {} } } }], tool_choice: 'required', stream: true }], + ['Responses non-stream', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'look up', tools: [{ type: 'function', name: 'lookup', description: 'Lookup', parameters: { type: 'object', properties: {} } }], tool_choice: 'required' }], + ['Responses stream', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'look up', tools: [{ type: 'function', name: 'lookup', description: 'Lookup', parameters: { type: 'object', properties: {} } }], tool_choice: 'required', stream: true }], + ])('%s allows the required-tool fallback after an empty completed poll, then fails closed', async (label, path, body) => { + sdkMocks.sessionPrompt.mockResolvedValue({ data: { parts: [] } }); + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'empty-assistant', role: 'assistant', finish: 'stop' }, parts: [] }, + ]); + if (body.stream) { + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { id: 'empty-assistant', role: 'assistant', sessionID: 'default-session', finish: 'stop', parts: [] }, + }, + }; + })(), + })); + } + + const response = await request(app).post(path).send(body); + const output = response.text || JSON.stringify(response.body); + + expect(output).toContain('external_tool_choice_required'); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(output).not.toContain('response.completed'); + expect(response.body?.status).not.toBe('completed'); + }); + + test.each([ + ['Responses non-stream continuation', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_response_id: 'stored-response', stream: false }], + ['Responses stream continuation', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_response_id: 'stored-response', stream: true }], + ['Interactions non-stream continuation', '/v1beta/interactions', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_interaction_id: 'stored-interaction', stream: false }], + ['Interactions stream continuation', '/v1beta/interactions', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_interaction_id: 'stored-interaction', stream: true }], + ])('%s polls past the previous assistant until the current message parent is complete', async (_label, path, body) => { + const sessionId = 'continuation-route-session'; + const stateId = body.previous_response_id ?? body.previous_interaction_id; + ctx.storeResponseState(stateId, sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + let currentMessageID = null; + let pollCount = 0; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.sessionMessages.mockImplementation(async () => { + pollCount += 1; + const messages = [ + { + info: { id: 'assistant-prior', role: 'assistant', parentID: 'msg-prior-user', finish: 'stop' }, + parts: [{ type: 'text', text: 'prior turn answer' }], + }, + { + info: { id: currentMessageID, role: 'user' }, + parts: [{ id: 'current-user-part', type: 'text', text: 'second turn' }], + }, + ]; + if (pollCount > 1) { + messages.push({ + info: { id: 'assistant-current', role: 'assistant', parentID: currentMessageID, finish: 'stop' }, + parts: [{ type: 'text', text: 'current turn answer' }], + }); + } + return messages; + }); + + const response = await request(app).post(path).send(body); + const output = response.text || JSON.stringify(response.body); + expect(response.status).toBe(200); + expect(output).toContain('current turn answer'); + expect(output).not.toContain('prior turn answer'); + expect(pollCount).toBeGreaterThan(1); + expect(output).not.toMatch(/"type"\s*:\s*"(?:response\.failed|error)"/); + if (body.stream) { + if (path === '/v1/responses') { + const frames = response.text + .split('\n') + .filter((line) => line.startsWith('data: ') && line !== 'data: [DONE]') + .map((line) => JSON.parse(line.slice(6))); + expect(frames.at(-1)?.type).toBe('response.completed'); + expect(response.text).toContain('data: [DONE]'); + expect(response.text).not.toContain('response.failed'); + } else { + expect(response.text).toContain('"type":"interaction.completed"'); + expect(response.text).not.toContain('"type":"error"'); + expect(response.text).not.toContain('[DONE]'); + } + } else { + expect(response.body.status).toBe('completed'); + expect(response.body.error == null).toBe(true); + } + }); + + test.each([ + ['Responses non-stream continuation', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_response_id: 'empty-response-parent', stream: false }], + ['Interactions non-stream continuation', '/v1beta/interactions', { model: 'opencode/kimi-k2.5', input: 'second turn', previous_interaction_id: 'empty-interaction-parent', stream: false }], + ])('%s fails closed on an empty completed assistant poll snapshot', async (_label, path, body) => { + const sessionId = 'empty-continuation-session'; + const stateId = body.previous_response_id ?? body.previous_interaction_id; + ctx.storeResponseState(stateId, sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + let currentMessageID; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.sessionMessages.mockImplementation(async () => ([ + { info: { id: currentMessageID, role: 'user' }, parts: [] }, + { + info: { id: 'assistant-empty', role: 'assistant', parentID: currentMessageID, finish: 'stop' }, + parts: [], + }, + ])); + + const response = await request(app).post(path).send(body); + + expect(response.status).toBeGreaterThanOrEqual(400); + expect(response.body.status).not.toBe('completed'); + expect(response.body).not.toHaveProperty('output_text'); + expect(JSON.stringify(response.body)).toContain('Polling returned empty response'); + }); + + test('Responses non-stream fails closed on an ordinary empty completed poll snapshot', async () => { + sdkMocks.sessionPrompt.mockResolvedValue({ data: { parts: [] } }); + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'ordinary-empty-assistant', role: 'assistant', finish: 'stop' }, parts: [] }, + ]); + + const response = await request(app) + .post('/v1/responses') + .send({ model: 'opencode/kimi-k2.5', input: 'ordinary request', stream: false }); + + expect(response.status).toBeGreaterThanOrEqual(400); + expect(response.body.status).not.toBe('completed'); + expect(response.body).not.toHaveProperty('output_text'); + expect(JSON.stringify(response.body)).toContain('Polling returned empty response'); + }); + + test('Responses streaming continuation releases a pre-metadata text delta exactly once', async () => { + const sessionId = 'pre-metadata-stream-continuation-session'; + const childSessionId = `${sessionId}-child`; + ctx.storeResponseState('pre-metadata-stream-parent', sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: childSessionId } }); + let currentMessageID; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.delta', + properties: { + sessionID: childSessionId, + messageID: 'assistant-current', + partID: 'answer-part', + field: 'text', + delta: 'Continuation answer', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-current', + role: 'assistant', + parentID: currentMessageID, + finish: 'stop', + sessionID: childSessionId, + }, + }, + }; + })(), + })); + + const response = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'current turn', + stream: true, + previous_response_id: 'pre-metadata-stream-parent', + }); + + const frames = response.text + .split('\n') + .filter((line) => line.startsWith('data: ') && line !== 'data: [DONE]') + .map((line) => JSON.parse(line.slice(6))); + const textDeltas = frames + .filter((frame) => frame.type === 'response.output_text.delta') + .map((frame) => frame.delta); + + expect(response.status).toBe(200); + expect(textDeltas).toEqual(['Continuation answer']); + expect(frames.at(-1)?.type).toBe('response.completed'); + expect(response.text).toContain('data: [DONE]'); + expect(response.text).not.toContain('response.failed'); + }); + + test('Responses streaming continuation fails closed on a mismatched part parent after message confirmation', async () => { + const sessionId = 'part-parent-mismatch-session'; + const childSessionId = `${sessionId}-child`; + ctx.storeResponseState('part-parent-mismatch-parent', sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: childSessionId } }); + let currentMessageID; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-correlated', + role: 'assistant', + parentID: currentMessageID, + sessionID: childSessionId, + }, + }, + }; + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'mismatched-part', + messageID: 'assistant-correlated', + parentID: 'another-user-message', + type: 'text', + sessionID: childSessionId, + }, + delta: 'uncorrelated text', + }, + }; + })(), + })); + + const response = await request(app).post('/v1/responses').send({ + model: 'opencode/kimi-k2.5', + input: 'continuation', + stream: true, + previous_response_id: 'part-parent-mismatch-parent', + }); + + expect(response.text).toContain('response.failed'); + expect(response.text).toContain('Continuation correlation error'); + expect(response.text).not.toContain('response.completed'); + expect(response.text).not.toContain('uncorrelated text'); + }); + + test('Responses streaming continuation fails closed when a confirmed message parent explicitly changes', async () => { + const sessionId = 'message-parent-mismatch-session'; + const childSessionId = `${sessionId}-child`; + ctx.storeResponseState('message-parent-mismatch-parent', sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: childSessionId } }); + let currentMessageID; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-correlated', + role: 'assistant', + parentID: currentMessageID, + sessionID: childSessionId, + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-correlated', + role: 'assistant', + parentID: 'another-user-message', + sessionID: childSessionId, + finish: 'stop', + parts: [{ type: 'text', text: 'uncorrelated text' }], + }, + }, + }; + })(), + })); + + const response = await request(app).post('/v1/responses').send({ + model: 'opencode/kimi-k2.5', + input: 'continuation', + stream: true, + previous_response_id: 'message-parent-mismatch-parent', + }); + + expect(response.text).toContain('response.failed'); + expect(response.text).toContain('Continuation correlation error'); + expect(response.text).not.toContain('response.completed'); + }); + + test('Responses streaming continuation accepts a sparse update that omits its retained parent', async () => { + const sessionId = 'sparse-parent-update-session'; + const childSessionId = `${sessionId}-child`; + ctx.storeResponseState('sparse-parent-update-parent', sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: childSessionId } }); + let currentMessageID; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + currentMessageID = params.body.messageID; + return { data: { parts: [] } }; + }); + sdkMocks.eventSubscribe.mockImplementationOnce(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-sparse', + role: 'assistant', + parentID: currentMessageID, + sessionID: childSessionId, + }, + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-sparse', + role: 'assistant', + sessionID: childSessionId, + finish: 'stop', + parts: [{ type: 'text', text: 'sparse continuation answer' }], + }, + }, + }; + })(), + })); + + const response = await request(app).post('/v1/responses').send({ + model: 'opencode/kimi-k2.5', + input: 'continuation', + stream: true, + previous_response_id: 'sparse-parent-update-parent', + }); + + expect(response.text).toContain('response.completed'); + expect(response.text).toContain('sparse continuation answer'); + expect(response.text).not.toContain('response.failed'); + }); + + test('Responses streaming continuation does not succeed from a timestamp-less assistant with a mismatched parent', async () => { + const sessionId = 'stale-stream-continuation-session'; + const childSessionId = `${sessionId}-child`; + const created = createApp(baseConfig({ REQUEST_TIMEOUT_MS: 40, RETRY_MAX_RETRIES: 0 })); + app = created.app; + ctx = created.ctx; + ctx.storeResponseState('stale-stream-parent', sessionId, 'opencode/kimi-k2.5', [], 'assistant-parent-checkpoint'); + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: childSessionId } }); + + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + // Exercise ordering where deltas precede the assistant metadata + // that identifies this as a previous turn. + yield { + type: 'message.part.updated', + properties: { + part: { + id: 'stale-part', + messageID: 'assistant-stale', + type: 'text', + sessionID: childSessionId, + }, + delta: 'STALE ANSWER', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { + id: 'assistant-stale', + role: 'assistant', + parentID: 'previous-user-message', + finish: 'stop', + sessionID: childSessionId, + }, + }, + }; + })(), + })); + sdkMocks.sessionMessages.mockImplementation(async () => ([{ + info: { + id: 'assistant-stale', + role: 'assistant', + parentID: 'previous-user-message', + finish: 'stop', + }, + parts: [{ type: 'text', text: 'STALE ANSWER' }], + }])); + + const response = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'current turn', + stream: true, + previous_response_id: 'stale-stream-parent', + }); + + expect(response.headers['content-type']).toContain('text/event-stream'); + expect(response.text).toContain('response.failed'); + expect(response.text).not.toContain('STALE ANSWER'); + expect(response.text).not.toContain('response.completed'); + expect(response.text).not.toContain('response.output_text.delta'); + }); + + test('Responses stream retries 503 transient error on attempt 1 and succeeds on attempt 2 before any output', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `resp-session-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => { + const currentSession = `resp-session-${sessionCount}`; + return { + stream: (async function* () { + if (currentSession === 'resp-session-1') { + yield { + type: 'session.error', + properties: { + sessionID: 'resp-session-1', + error: { + name: 'APIError', + data: { + statusCode: 503, + message: 'Endpoint is unavailable.', + responseHeaders: { 'retry-after-ms': '5' } + } + } + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: 'resp-session-2' }, + delta: 'Hello from responses attempt 2' + } + }; + yield { + type: 'message.updated', + properties: { + info: { sessionID: 'resp-session-2', finish: 'stop' } + } + }; + } + })() + }; + }); + + const res = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'hello', + stream: true + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(res.text).toContain('Hello from responses attempt 2'); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'resp-session-1' } }); + }); + + test('Messages stream retries 503 transient error on attempt 1 and succeeds on attempt 2 before any output', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `msg-session-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => { + const currentSession = `msg-session-${sessionCount}`; + return { + stream: (async function* () { + if (currentSession === 'msg-session-1') { + yield { + type: 'session.error', + properties: { + sessionID: 'msg-session-1', + error: { + name: 'APIError', + data: { + statusCode: 503, + message: 'Endpoint is unavailable.', + responseHeaders: { 'retry-after-ms': '5' } + } + } + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: 'msg-session-2' }, + delta: 'Hello from messages attempt 2' + } + }; + yield { + type: 'message.updated', + properties: { + info: { sessionID: 'msg-session-2', finish: 'stop' } + } + }; + } + })() + }; + }); + + const res = await request(app) + .post('/v1/messages') + .send({ + model: 'opencode/kimi-k2.5', + max_tokens: 100, + messages: [{ role: 'user', content: 'hello' }], + stream: true + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(res.text).toContain('Hello from messages attempt 2'); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'msg-session-1' } }); + }); + + test('Messages stream retries a rejected prompt promise when no output was produced', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `msg-prompt-session-${sessionCount}` } }; + }); + const promptError = Object.assign(new Error('Endpoint is unavailable.'), { + data: { statusCode: 503, message: 'Endpoint is unavailable.' }, + }); + sdkMocks.sessionPrompt + .mockRejectedValueOnce(promptError) + .mockResolvedValueOnce({ data: { parts: [] } }); + sdkMocks.eventSubscribe.mockImplementation(async () => { + const currentSession = `msg-prompt-session-${sessionCount}`; + return { + stream: (async function* () { + if (currentSession !== 'msg-prompt-session-1') { + yield { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: currentSession }, + delta: 'Recovered after prompt rejection', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { sessionID: currentSession, finish: 'stop' }, + }, + }; + } + })(), + }; + }); + + const res = await request(app) + .post('/v1/messages') + .send({ + model: 'opencode/kimi-k2.5', + max_tokens: 100, + messages: [{ role: 'user', content: 'hello' }], + stream: true, + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(sdkMocks.sessionPrompt).toHaveBeenCalledTimes(2); + expect(res.text).toContain('Recovered after prompt rejection'); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'msg-prompt-session-1' } }); + }); + + test('Interactions retries 503 transient error on attempt 1 and succeeds on attempt 2', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `intr-session-${sessionCount}` } }; + }); + + sdkMocks.sessionPrompt.mockImplementation(async () => { + if (sessionCount === 1) { + const err = new Error('Endpoint is unavailable.'); + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + } + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockImplementation(async () => ([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'Hello from interactions attempt 2' }] + } + ])); + + const res = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'hello' + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(res.body.output_text).toBe('Hello from interactions attempt 2'); + expect(sdkMocks.sessionDelete).toHaveBeenCalledWith({ path: { id: 'intr-session-1' } }); + }); +}); + +describe('Fail Closed Immediately (No Retry) on Partial Stream Error', () => { + let app; + + beforeEach(() => { + jest.clearAllMocks(); + app = createApp(baseConfig({})).app; + }); + + test('Chat completions streaming fails closed without retry after partial output', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `chat-partial-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'chat-assistant', role: 'assistant', sessionID: 'chat-partial-1' } } + }; + yield { + type: 'message.part.updated', + properties: { + part: { id: 'chat-part', messageID: 'chat-assistant', type: 'text', sessionID: 'chat-partial-1' }, + delta: 'First chat chunk' + } + }; + yield { + type: 'session.error', + properties: { + sessionID: 'chat-partial-1', + error: { + name: 'APIError', + data: { statusCode: 503, message: 'Endpoint is unavailable.' } + } + } + }; + })() + })); + + const res = await request(app) + .post('/v1/chat/completions') + .send({ + model: 'opencode/kimi-k2.5', + messages: [{ role: 'user', content: 'hello' }], + stream: true + }); + + expect(sessionCount).toBe(1); + expect(res.text).toContain('First chat chunk'); + expect(res.text).toContain('error'); + }); + + test('Responses streaming fails closed without retry after partial output', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `resp-partial-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'responses-assistant', role: 'assistant', sessionID: 'resp-partial-1' } } + }; + yield { + type: 'message.part.updated', + properties: { + part: { id: 'responses-part', messageID: 'responses-assistant', type: 'text', sessionID: 'resp-partial-1' }, + delta: 'First responses chunk' + } + }; + yield { + type: 'session.error', + properties: { + sessionID: 'resp-partial-1', + error: { + name: 'APIError', + data: { statusCode: 503, message: 'Endpoint is unavailable.' } + } + } + }; + })() + })); + + const res = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'hello', + stream: true + }); + + expect(sessionCount).toBe(1); + expect(res.text).toContain('response.output_text.delta'); + expect(res.text).toContain('response.failed'); + expect(res.text).toContain('[DONE]'); + }); + + test('Messages streaming fails closed without retry after partial output', async () => { + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `msg-partial-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'messages-assistant', role: 'assistant', sessionID: 'msg-partial-1' } } + }; + yield { + type: 'message.part.updated', + properties: { + part: { id: 'messages-part', messageID: 'messages-assistant', type: 'text', sessionID: 'msg-partial-1' }, + delta: 'First messages chunk' + } + }; + yield { + type: 'session.error', + properties: { + sessionID: 'msg-partial-1', + error: { + name: 'APIError', + data: { statusCode: 503, message: 'Endpoint is unavailable.' } + } + } + }; + })() + })); + + const res = await request(app) + .post('/v1/messages') + .send({ + model: 'opencode/kimi-k2.5', + max_tokens: 100, + messages: [{ role: 'user', content: 'hello' }], + stream: true + }); + + expect(sessionCount).toBe(1); + expect(res.text).toContain('content_block_delta'); + expect(res.text).toContain('event: error'); + }); +}); + +describe('Model Resolution Fast-Fail & Lock Separation', () => { + let app; + + beforeEach(() => { + jest.clearAllMocks(); + app = createApp(baseConfig({})).app; + }); + + test('Unknown model on /v1/chat/completions fails fast with 404 without waiting for lock', async () => { + let releaseLock; + const lockHeldPromise = new Promise((resolve) => { + releaseLock = resolve; + }); + + let lockAcquired = false; + const lockHolder = lock(async () => { + lockAcquired = true; + await lockHeldPromise; + }); + + while (!lockAcquired) { + await new Promise((r) => setTimeout(r, 10)); + } + + try { + const start = Date.now(); + const res = await request(app) + .post('/v1/chat/completions') + .send({ + model: 'opencode/nonexistent-model-xyz', + messages: [{ role: 'user', content: 'hello' }] + }); + const elapsed = Date.now() - start; + + expect(res.status).toBe(404); + expect(res.body.code).toBe('model_not_found'); + expect(elapsed).toBeLessThan(150); + } finally { + releaseLock(); + await lockHolder; + } + }); + + test('Unknown model on /v1/messages fails fast with 404 without waiting for lock', async () => { + let releaseLock; + const lockHeldPromise = new Promise((resolve) => { + releaseLock = resolve; + }); + + let lockAcquired = false; + const lockHolder = lock(async () => { + lockAcquired = true; + await lockHeldPromise; + }); + + while (!lockAcquired) { + await new Promise((r) => setTimeout(r, 10)); + } + + try { + const start = Date.now(); + const res = await request(app) + .post('/v1/messages') + .send({ + model: 'opencode/nonexistent-model-xyz', + max_tokens: 100, + messages: [{ role: 'user', content: 'hello' }] + }); + const elapsed = Date.now() - start; + + expect(res.status).toBe(404); + expect(res.body.type).toBe('error'); + expect(res.body.error.type).toBe('not_found_error'); + expect(elapsed).toBeLessThan(150); + } finally { + releaseLock(); + await lockHolder; + } + }); +}); + +describe('Provider List Caching', () => { + test('Provider list is cached across requests within TTL and invalidated on reset/reconnect', async () => { + sdkMocks.configProviders.mockClear(); + const { app, ctx } = createApp(baseConfig({})); + + // First call fetches from backend + const list1 = await ctx.getProvidersList(); + expect(list1).toHaveLength(1); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + // Second call within TTL returns cached without calling backend again + const list2 = await ctx.getProvidersList(); + expect(list2).toEqual(list1); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + // HTTP GET /v1/models also uses cached list + const res = await request(app).get('/v1/models'); + expect(res.status).toBe(200); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + // Invalidate cache manually + ctx.invalidateProvidersCache(); + const list3 = await ctx.getProvidersList(); + expect(list3).toEqual(list1); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(2); + + // Reconnect notification also invalidates cache + notifyBackendReconnect(); + const list4 = await ctx.getProvidersList(); + expect(list4).toEqual(list1); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(3); + }); + + test('Concurrent calls to getProvidersList share the same pending promise', async () => { + sdkMocks.configProviders.mockClear(); + const { ctx } = createApp(baseConfig({})); + + const [listA, listB] = await Promise.all([ctx.getProvidersList(), ctx.getProvidersList()]); + expect(listA).toEqual(listB); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + }); + + test('Provider cache reconnect race: in-flight providers call does not restore invalidated cache on late resolution', async () => { + sdkMocks.configProviders.mockClear(); + const { ctx } = createApp(baseConfig({})); + + let resolveInFlight; + const inFlightBackendPromise = new Promise((resolve) => { + resolveInFlight = resolve; + }); + + let callCount = 0; + sdkMocks.configProviders.mockImplementation(async () => { + callCount += 1; + if (callCount === 1) { + return inFlightBackendPromise; + } + return { + data: { + providers: [{ + id: 'fresh-opencode', + models: { 'fresh-model': { name: 'Fresh Model' } } + }] + } + }; + }); + + try { + // 1. First fetch starts and remains in flight + const flight1Promise = ctx.getProvidersList(); + expect(callCount).toBe(1); + + // 2. Trigger backend reconnect/invalidation while fetch is in flight + notifyBackendReconnect(); + + // 3. The in-flight fetch resolves late with stale provider info + resolveInFlight({ + data: { + providers: [{ + id: 'stale-opencode', + models: { 'stale-model': { name: 'Stale Model' } } + }] + } + }); + + const flight1Result = await flight1Promise; + expect(flight1Result[0].id).toBe('stale-opencode'); + + // 4. Stale data must not overwrite the invalidated cache; subsequent call triggers fresh fetch + const freshResult = await ctx.getProvidersList(); + expect(freshResult[0].id).toBe('fresh-opencode'); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(2); + + // 5. Subsequent call hits the fresh cache + const cachedFresh = await ctx.getProvidersList(); + expect(cachedFresh[0].id).toBe('fresh-opencode'); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(2); + } finally { + sdkMocks.configProviders.mockImplementation(async () => ({ + data: { + providers: [{ + id: 'opencode', + models: { + 'kimi-k2.5': { name: 'Kimi k2.5' } + } + }] + } + })); + } + }); + + test('Responses continuation 503 fails closed, preserves parent session AND does not call session.create()', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1' }] } + ]); + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'parent-session-123' } }); + sdkMocks.sessionPrompt.mockResolvedValue({ data: { + info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'turn 1' }] + } }); + const seedRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'first turn' }); + expect(seedRes.status).toBe(200); + const prevId = seedRes.body.id; + + // 2. Continuation turn: attempt 1 fails with transient 503 error event, fails closed without creating new session or deleting parent session + let attempt = 0; + sdkMocks.eventSubscribe.mockImplementation(async () => { + attempt += 1; + const currentAttempt = attempt; + return { + stream: (async function* () { + if (currentAttempt === 1) { + yield { + type: 'session.error', + properties: { + sessionID: 'fork-session-1', + error: { + name: 'APIError', + data: { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } } + } + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { part: { type: 'text', sessionID: 'fork-session-1' }, delta: 'turn 2 success' } + }; + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-child-checkpoint', role: 'assistant', parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, sessionID: 'fork-session-1', finish: 'stop' } } + }; + } + })() + }; + }); + + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: 'fork-session-1' } }); + sdkMocks.sessionDelete.mockClear(); + sdkMocks.sessionCreate.mockClear(); + const contRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'second turn', stream: true, previous_response_id: prevId }); + + expect(contRes.status).toBe(200); + expect(contRes.text).toContain('response.failed'); + expect(contRes.text).toContain('Endpoint is unavailable.'); + expect(contRes.text).not.toContain('turn 2 success'); + + // Verify parent session was NOT deleted and session.create() was NOT called + expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); + const deletedIds = sdkMocks.sessionDelete.mock.calls.map(c => c[0]?.path?.id); + expect(deletedIds).not.toContain('parent-session-123'); + }); + + test('Responses continuation 503 fails closed and does not duplicate user prompt in stateful session transcript', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + const transcript = []; + let isTurn2 = false; + let sessionPromptTurn2Calls = 0; + let capturedMessageId = null; + + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + transcript.push({ role: 'user', content: params.body.parts }); + if (isTurn2) { + sessionPromptTurn2Calls += 1; + capturedMessageId = params.body?.messageID; + } + return { data: { + info: { id: isTurn2 ? 'assistant-child-checkpoint' : 'assistant-parent-checkpoint', role: 'assistant', parentID: params.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: isTurn2 ? 'turn 2' : 'turn 1' }] + } }; + }); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1' }] } + ]); + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'stateful-parent-session-123' } }); + + // Turn 1: seed parent session + const seedRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'first turn' }); + expect(seedRes.status).toBe(200); + const prevId = seedRes.body.id; + expect(transcript).toHaveLength(1); + + // Turn 2: continuation turn + // Attempt 1 succeeds in persisting user prompt, but SSE event stream emits 503 session.error + isTurn2 = true; + let attempt = 0; + sdkMocks.eventSubscribe.mockImplementation(async () => { + attempt += 1; + const currentAttempt = attempt; + return { + stream: (async function* () { + if (currentAttempt === 1) { + yield { + type: 'session.error', + properties: { + sessionID: 'fork-session-1', + error: { + name: 'APIError', + data: { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } } + } + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { part: { type: 'text', sessionID: 'fork-session-1' }, delta: 'turn 2 success' } + }; + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-child-checkpoint', role: 'assistant', parentID: sdkMocks.sessionPrompt.mock.calls.at(-1)?.[0]?.body?.messageID, sessionID: 'fork-session-1', finish: 'stop' } } + }; + } + })() + }; + }); + + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: 'fork-session-1' } }); + const contRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn', + stream: true, + previous_response_id: prevId, + }); + + expect(contRes.status).toBe(200); + expect(contRes.text).toContain('response.failed'); + expect(contRes.text).toContain('Endpoint is unavailable.'); + expect(contRes.text).not.toContain('turn 2 success'); + + // Assert sdkMocks.sessionPrompt is called only once for Turn 2 + expect(sessionPromptTurn2Calls).toBe(1); + expect(capturedMessageId).toMatch(/^msg_/); + // Assert session transcript has only one instance of Turn 2 user prompt (no duplicate) + const turn2Entries = transcript.filter((t) => + JSON.stringify(t.content).includes('second turn'), + ); + expect(turn2Entries).toHaveLength(1); + }); + + test('Responses non-stream continuation 503 fails closed without duplicate user prompt in stateful session transcript', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + const transcript = []; + let isTurn2 = false; + let sessionPromptTurn2Calls = 0; + let capturedMessageId = null; + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1 nonstream' }] } + ]); + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'stateful-parent-nonstream-789' } }); + + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + transcript.push({ role: 'user', content: params.body.parts }); + if (isTurn2) { + sessionPromptTurn2Calls += 1; + capturedMessageId = params.body?.messageID; + } + return { data: { + info: { id: isTurn2 ? 'assistant-child-checkpoint' : 'assistant-parent-checkpoint', role: 'assistant', parentID: params.body.messageID, finish: 'stop' }, + parts: [{ type: 'text', text: isTurn2 ? 'turn 2 nonstream' : 'turn 1 nonstream' }] + } }; + }); + + const seedRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'first turn nonstream' }); + expect(seedRes.status).toBe(200); + const prevId = seedRes.body.id; + expect(transcript).toHaveLength(1); + + isTurn2 = true; + let pollCount = 0; + // In continuation turn 2: sessionPrompt returns empty parts so it polls, and poll attempt 1 fails with 503 + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + transcript.push({ role: 'user', content: params.body.parts }); + if (isTurn2) { + sessionPromptTurn2Calls += 1; + capturedMessageId = params.body?.messageID; + } + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockImplementation(async () => { + pollCount += 1; + if (pollCount === 1) { + const err = new Error('Endpoint is unavailable.'); + err.statusCode = 503; + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + } + return [ + { info: { id: 'assistant-child-checkpoint', role: 'assistant', parentID: capturedMessageId, finish: 'stop' }, parts: [{ type: 'text', text: 'turn 2 nonstream success' }] } + ]; + }); + + const contRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn nonstream', + previous_response_id: prevId, + }); + + expect(contRes.status).toBe(503); + expect(contRes.body.message).toContain('Endpoint is unavailable.'); + expect(sessionPromptTurn2Calls).toBe(1); + expect(capturedMessageId).toMatch(/^msg_/); + const turn2Entries = transcript.filter((t) => + JSON.stringify(t.content).includes('second turn nonstream'), + ); + expect(turn2Entries).toHaveLength(1); + }); + + test('Interactions continuation 503 fails closed without duplicate user prompt in stateful session transcript', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + const transcript = []; + let isTurn2 = false; + let sessionPromptTurn2Calls = 0; + let capturedMessageId = null; + + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'intr-stateful-parent-456' } }); + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + transcript.push({ role: 'user', content: params.body.parts }); + if (isTurn2) { + sessionPromptTurn2Calls += 1; + capturedMessageId = params.body?.messageID; + } + return { data: { parts: [] } }; + }); + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { info: { id: 'assistant-intr-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1 intr' }] } + ]); + + const seedRes = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'first turn', + store: true, + }); + expect(seedRes.status).toBe(200); + const prevIntrId = seedRes.body.id; + expect(transcript).toHaveLength(1); + + isTurn2 = true; + let pollCount = 0; + sdkMocks.sessionMessages.mockImplementation(async () => { + pollCount += 1; + if (pollCount === 1) { + const err = new Error('Endpoint is unavailable.'); + err.statusCode = 503; + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + } + return [ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 2 intr success' }] } + ]; + }); + + const contRes = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn', + previous_interaction_id: prevIntrId, + }); + + expect(contRes.status).toBe(503); + expect(contRes.body.message).toContain('Endpoint is unavailable.'); + expect(sessionPromptTurn2Calls).toBe(1); + expect(capturedMessageId).toMatch(/^msg_/); + const turn2Entries = transcript.filter((t) => + JSON.stringify(t.content).includes('second turn'), + ); + expect(turn2Entries).toHaveLength(1); + }); + + test('Interactions continuation 503 prompt failure fails closed and preserves parent session', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'intr-parent-session-123' } }); + sdkMocks.sessionPrompt.mockResolvedValue({ data: { parts: [] } }); + sdkMocks.sessionMessages.mockResolvedValueOnce([ + { info: { id: 'assistant-intr-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1 intr' }] } + ]); + + const seedRes = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'first turn', + store: true + }); + expect(seedRes.status).toBe(200); + const prevIntrId = seedRes.body.id; + + let promptCount = 0; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + promptCount += 1; + expect(params.path.id).toBe('fork-session-1'); + if (promptCount === 1) { + const err = new Error('Endpoint is unavailable.'); + err.statusCode = 503; + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + } + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 2 intr success' }] } + ]); + + sdkMocks.sessionFork.mockResolvedValueOnce({ data: { id: 'fork-session-1' } }); + sdkMocks.sessionCreate.mockClear(); + sdkMocks.sessionDelete.mockClear(); + + const contRes = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn', + previous_interaction_id: prevIntrId + }); + + expect(contRes.status).toBe(503); + expect(contRes.body.message).toContain('Endpoint is unavailable.'); + expect(promptCount).toBe(1); + + // Verify parent session was NOT deleted and session.create() was NOT called on retry + expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); + const deletedIds = sdkMocks.sessionDelete.mock.calls.map(c => c[0]?.path?.id); + expect(deletedIds).not.toContain('intr-parent-session-123'); + }); + + test('Responses streaming continuation prompt failure fails closed, emits response.failed with 503 and prompts only once', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1' }] } + ]); + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'resp-stream-parent-456' } }); + sdkMocks.sessionPrompt.mockResolvedValue({ data: { + info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'turn 1' }] + } }); + const seedRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'first turn' }); + expect(seedRes.status).toBe(200); + const prevId = seedRes.body.id; + + let sessionPromptTurn2Calls = 0; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + sessionPromptTurn2Calls += 1; + const err = new Error('Endpoint is unavailable.'); + err.statusCode = 503; + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + }); + + sdkMocks.sessionDelete.mockClear(); + sdkMocks.sessionCreate.mockClear(); + + const contRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn', + stream: true, + previous_response_id: prevId, + }); + + expect(contRes.status).toBe(200); + expect(contRes.text).toContain('response.failed'); + expect(contRes.text).toContain('Endpoint is unavailable.'); + expect(contRes.text).not.toContain('response.completed'); + expect(sessionPromptTurn2Calls).toBe(1); + expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); + const deletedIds = sdkMocks.sessionDelete.mock.calls.map(c => c[0]?.path?.id); + expect(deletedIds).not.toContain('resp-stream-parent-456'); + }); + + test('Responses non-stream continuation prompt failure fails closed, returns 503 and prompts only once', async () => { + const { app } = createApp(baseConfig({ RETRY_MAX_RETRIES: 1 })); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'turn 1 nonstream' }] } + ]); + sdkMocks.sessionCreate.mockResolvedValueOnce({ data: { id: 'resp-nonstream-parent-789' } }); + sdkMocks.sessionPrompt.mockResolvedValue({ data: { + info: { id: 'assistant-parent-checkpoint', role: 'assistant', finish: 'stop' }, + parts: [{ type: 'text', text: 'turn 1 nonstream' }] + } }); + + const seedRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ model: 'opencode/kimi-k2.5', input: 'first turn nonstream' }); + expect(seedRes.status).toBe(200); + const prevId = seedRes.body.id; + + let sessionPromptTurn2Calls = 0; + sdkMocks.sessionPrompt.mockImplementation(async (params) => { + sessionPromptTurn2Calls += 1; + const err = new Error('Endpoint is unavailable.'); + err.statusCode = 503; + err.data = { statusCode: 503, message: 'Endpoint is unavailable.', responseHeaders: { 'retry-after-ms': '5' } }; + throw err; + }); + + sdkMocks.sessionDelete.mockClear(); + sdkMocks.sessionCreate.mockClear(); + + const contRes = await request(app) + .post('/v1/responses') + .set('Authorization', 'Bearer test-key') + .send({ + model: 'opencode/kimi-k2.5', + input: 'second turn nonstream', + previous_response_id: prevId, + }); + + expect(contRes.status).toBe(503); + expect(contRes.body.message).toContain('Endpoint is unavailable.'); + expect(sessionPromptTurn2Calls).toBe(1); + expect(sdkMocks.sessionCreate).not.toHaveBeenCalled(); + const deletedIds = sdkMocks.sessionDelete.mock.calls.map(c => c[0]?.path?.id); + expect(deletedIds).not.toContain('resp-nonstream-parent-789'); + }); + + test('Responses stream proxy fallback succeeds cleanly without repeating prompts or retaining Attempt 1 error', async () => { + const { app } = createApp(baseConfig({ + RETRY_MAX_RETRIES: 1, + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + })); + + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `resp-fallback-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => { + return { + stream: (async function* () { + yield { + type: 'session.error', + properties: { + sessionID: 'resp-fallback-1', + error: freeUsageError() + } + }; + })() + }; + }); + + let promptCalls = 0; + sdkMocks.sessionPrompt.mockImplementation(async () => { + promptCalls += 1; + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Hello from proxy fallback attempt 2' }] } + ]); + + const res = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'hello fallback', + stream: true + }); + + expect(res.status).toBe(200); + expect(res.text).toContain('Hello from proxy fallback attempt 2'); + expect(res.text).toContain('response.completed'); + expect(res.text).not.toContain('Rate limit exceeded'); + expect(promptCalls).toBe(2); + }); +}); + +describe('Backend Lifecycle & Active Model Retries', () => { + afterEach(() => { + clearBackendReconnectListeners(); + }); + + test('getProvidersList triggers ensureBackend when config.MANAGE_BACKEND: true', async () => { + sdkMocks.configProviders.mockClear(); + httpMocks.get.mockClear(); + + // With MANAGE_BACKEND: true, getProvidersList triggers ensureBackend (which calls checkHealth via http.get) + const { ctx: ctxManage } = createApp(baseConfig({ MANAGE_BACKEND: true })); + const list = await ctxManage.getProvidersList(); + expect(list).toHaveLength(1); + expect(httpMocks.get).toHaveBeenCalled(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + // With MANAGE_BACKEND: false, getProvidersList does not call ensureBackend + httpMocks.get.mockClear(); + sdkMocks.configProviders.mockClear(); + const { ctx: ctxNoManage } = createApp(baseConfig({ MANAGE_BACKEND: false })); + const listNoManage = await ctxNoManage.getProvidersList(); + expect(listNoManage).toHaveLength(1); + expect(httpMocks.get).not.toHaveBeenCalled(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + }); + + test('ensureActiveModel is called on retry when proxy fallback switches activeClient (Responses stream)', async () => { + sdkMocks.configUpdate.mockClear(); + const { app } = createApp(baseConfig({ + RETRY_MAX_RETRIES: 1, + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + })); + + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `resp-fallback-model-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => { + return { + stream: (async function* () { + if (sessionCount === 1) { + yield { + type: 'session.error', + properties: { + sessionID: 'resp-fallback-model-1', + error: freeUsageError() + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: 'resp-fallback-model-2' }, + delta: 'Hello from fallback attempt 2' + } + }; + yield { + type: 'message.updated', + properties: { + info: { sessionID: 'resp-fallback-model-2', finish: 'stop' } + } + }; + } + })() + }; + }); + + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Hello from fallback attempt 2' }] } + ]); + + const res = await request(app) + .post('/v1/responses') + .send({ + model: 'opencode/kimi-k2.5', + input: 'test active model retry', + stream: true + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + // Attempt 1 calls ensureActiveModel on primary client, Attempt 2 on fallback proxyClient + expect(sdkMocks.configUpdate).toHaveBeenCalledTimes(2); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(1, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(2, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + }); + + test('ensureActiveModel is called on retry when proxy fallback switches activeClient (Messages stream)', async () => { + sdkMocks.configUpdate.mockClear(); + const { app } = createApp(baseConfig({ + RETRY_MAX_RETRIES: 1, + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + })); + + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `msg-fallback-model-${sessionCount}` } }; + }); + + sdkMocks.eventSubscribe.mockImplementation(async () => { + return { + stream: (async function* () { + if (sessionCount === 1) { + yield { + type: 'session.error', + properties: { + sessionID: 'msg-fallback-model-1', + error: freeUsageError() + } + }; + } else { + yield { + type: 'message.part.updated', + properties: { + part: { type: 'text', sessionID: 'msg-fallback-model-2' }, + delta: 'Hello from messages fallback attempt 2' + } + }; + yield { + type: 'message.updated', + properties: { + info: { sessionID: 'msg-fallback-model-2', finish: 'stop' } + } + }; + } + })() + }; + }); + + sdkMocks.sessionPrompt.mockImplementation(async () => ({ data: { parts: [] } })); + + const res = await request(app) + .post('/v1/messages') + .send({ + model: 'opencode/kimi-k2.5', + max_tokens: 100, + messages: [{ role: 'user', content: 'test active model retry' }], + stream: true + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + // Attempt 1 on primary client, Attempt 2 on fallback proxyClient + expect(sdkMocks.configUpdate).toHaveBeenCalledTimes(2); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(1, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(2, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + }); + + test('ensureActiveModel is called on retry when proxy fallback switches activeClient (Interactions)', async () => { + sdkMocks.configUpdate.mockClear(); + const { app } = createApp(baseConfig({ + RETRY_MAX_RETRIES: 1, + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + })); + + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `intr-fallback-model-${sessionCount}` } }; + }); + + sdkMocks.sessionPrompt.mockImplementation(async () => { + if (sessionCount === 1) { + throw freeUsageError(); + } + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Hello from interactions fallback attempt 2' }] } + ]); + + const res = await request(app) + .post('/v1beta/interactions') + .send({ + model: 'opencode/kimi-k2.5', + input: 'test interactions active model retry', + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(res.body.output_text).toBe('Hello from interactions fallback attempt 2'); + expect(sdkMocks.configUpdate).toHaveBeenCalledTimes(2); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(1, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(2, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + }); + + test('Chat route ensures activeModel is updated on retry when proxy fallback switches activeClient', async () => { + sdkMocks.configUpdate.mockClear(); + const { app } = createApp(baseConfig({ + UPSTREAM_PROXIES: ['http://10.0.0.1:8080'], + UPSTREAM_PROXY_COOLDOWN_MS: 60000, + RETRY_MAX_RETRIES: 1, + })); + + let sessionCount = 0; + sdkMocks.sessionCreate.mockImplementation(async () => { + sessionCount += 1; + return { data: { id: `chat-fallback-session-${sessionCount}` } }; + }); + + sdkMocks.sessionPrompt.mockImplementation(async () => { + if (sessionCount === 1) { + throw freeUsageError(); + } + return { data: { parts: [] } }; + }); + + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [{ type: 'text', text: 'Hello from chat fallback attempt 2' }] } + ]); + + const res = await request(app) + .post('/v1/chat/completions') + .send({ + model: 'opencode/kimi-k2.5', + messages: [{ role: 'user', content: 'test chat active model retry' }], + }); + + expect(res.status).toBe(200); + expect(sessionCount).toBe(2); + expect(res.body.choices[0].message.content).toBe('Hello from chat fallback attempt 2'); + expect(sdkMocks.configUpdate).toHaveBeenCalledTimes(2); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(1, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + expect(sdkMocks.configUpdate).toHaveBeenNthCalledWith(2, { + body: { activeModel: { providerID: 'opencode', modelID: 'kimi-k2.5' } } + }); + }); + + test('clearBackendReconnectListeners and ctx.dispose() clean up reconnect listeners', async () => { + sdkMocks.configProviders.mockClear(); + + // 1. ctx.dispose() removes reconnect listener + const { ctx: ctx1 } = createApp(baseConfig({})); + await ctx1.getProvidersList(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + expect(typeof ctx1.dispose).toBe('function'); + ctx1.dispose(); + + // notifyBackendReconnect() does not invalidate disposed ctx cache + notifyBackendReconnect(); + await ctx1.getProvidersList(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + // 2. clearBackendReconnectListeners clears all listeners globally + sdkMocks.configProviders.mockClear(); + const { ctx: ctx2 } = createApp(baseConfig({})); + await ctx2.getProvidersList(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + + clearBackendReconnectListeners(); + notifyBackendReconnect(); + await ctx2.getProvidersList(); + expect(sdkMocks.configProviders).toHaveBeenCalledTimes(1); + }); + + test('ctx.dispose clears response-state and conversation-cleanup timers', () => { + jest.useFakeTimers(); + try { + const { ctx } = createApp(baseConfig({ + AUTO_CLEANUP_CONVERSATIONS: true, + CLEANUP_INTERVAL_MS: 60_000, + })); + expect(jest.getTimerCount()).toBeGreaterThanOrEqual(3); + ctx.dispose(); + ctx.dispose(); + expect(jest.getTimerCount()).toBe(0); + } finally { + jest.useRealTimers(); + } + }); + + test('startProxy unregisters its app reconnect listener on close and disposal is idempotent', async () => { + const getFreePort = async () => { + const probe = net.createServer(); + await new Promise((resolve, reject) => { + probe.once('error', reject); + probe.listen(0, '127.0.0.1', resolve); + }); + const address = probe.address(); + if (!address || typeof address === 'string') throw new Error('Failed to allocate a local test port'); + const port = address.port; + await new Promise((resolve, reject) => probe.close((error) => error ? reject(error) : resolve())); + return port; + }; + const close = (server) => new Promise((resolve, reject) => { + server.close((error) => error ? reject(error) : resolve()); + }); + const start = async () => { + const proxy = startProxy({ + PORT: await getFreePort(), + BIND_HOST: '127.0.0.1', + API_KEY: '', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + MANAGE_BACKEND: false, + }); + if (!proxy.server.listening) { + await new Promise((resolve, reject) => { + proxy.server.once('listening', resolve); + proxy.server.once('error', reject); + }); + } + return proxy; + }; + + const baseline = getBackendReconnectListenerCount(); + const closedByServer = await start(); + expect(getBackendReconnectListenerCount()).toBe(baseline + 1); + await close(closedByServer.server); + expect(getBackendReconnectListenerCount()).toBe(baseline); + expect(() => closedByServer.dispose()).not.toThrow(); + + const manuallyDisposed = await start(); + expect(getBackendReconnectListenerCount()).toBe(baseline + 1); + expect(() => manuallyDisposed.dispose()).not.toThrow(); + expect(getBackendReconnectListenerCount()).toBe(baseline); + expect(manuallyDisposed.server.listenerCount('error')).toBeGreaterThan(0); + const openServerErrorListeners = manuallyDisposed.server.listenerCount('error'); + await close(manuallyDisposed.server); + expect(getBackendReconnectListenerCount()).toBe(baseline); + expect(manuallyDisposed.server.listenerCount('error')).toBe(openServerErrorListeners - 1); + }); + + test('startProxy unregisters its app reconnect listener when bind fails', async () => { + const blocker = net.createServer(); + await new Promise((resolve, reject) => { + blocker.once('error', reject); + blocker.listen(0, '127.0.0.1', resolve); + }); + const address = blocker.address(); + if (!address || typeof address === 'string') throw new Error('Failed to reserve a local test port'); + + const baseline = getBackendReconnectListenerCount(); + try { + const proxy = startProxy({ + PORT: address.port, + BIND_HOST: '127.0.0.1', + API_KEY: '', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + MANAGE_BACKEND: false, + }); + expect(getBackendReconnectListenerCount()).toBe(baseline + 1); + await new Promise((resolve, reject) => { + proxy.server.once('listening', () => reject(new Error('Expected bind collision'))); + proxy.server.once('error', resolve); + }); + expect(proxy.server.listening).toBe(false); + expect(getBackendReconnectListenerCount()).toBe(baseline); + expect(proxy.server.listenerCount('error')).toBe(0); + expect(() => proxy.dispose()).not.toThrow(); + } finally { + await new Promise((resolve, reject) => blocker.close((error) => error ? reject(error) : resolve())); + } + }); +}); diff --git a/tests/stream-route-snapshots.test.js b/tests/stream-route-snapshots.test.js new file mode 100644 index 0000000..5bc7c7d --- /dev/null +++ b/tests/stream-route-snapshots.test.js @@ -0,0 +1,380 @@ +import request from 'supertest'; +import { jest } from '@jest/globals'; + +// Keep the collector idle window tiny so the idle-timeout recovery path is +// deterministic without a real backend. Set before importing the proxy because +// the collector reads these constants at module load. +process.env['OPENCODE2API_EVENT_IDLE_TIMEOUT_MS'] = '40'; +process.env['OPENCODE2API_EVENT_FIRST_DELTA_TIMEOUT_MS'] = '400'; + +const SESSION_ID = 'route-snapshot-session'; + +jest.unstable_mockModule('http', () => ({ + default: { + get: jest.fn((_url, _options, callback) => { + const response = { + statusCode: 200, + headers: {}, + on: jest.fn() + }; + callback(response); + return { + on: jest.fn(), + destroy: jest.fn(), + setTimeout: jest.fn() + }; + }) + } +})); + +const sdkMocks = { + configProviders: jest.fn(async () => ({ + data: { providers: [{ id: 'opencode', models: { 'kimi-k2.5': { name: 'Kimi k2.5' } } }] } + })), + configUpdate: jest.fn(async () => ({})), + toolIds: jest.fn(async () => ({ data: ['web_fetch', 'filesystem', 'bash'] })), + sessionCreate: jest.fn(async () => ({ data: { id: SESSION_ID } })), + sessionPrompt: jest.fn(async () => ({ data: { parts: [{ type: 'text', text: 'Mock response' }] } })), + sessionMessages: jest.fn(async () => ([ + { info: { role: 'assistant', finish: 'stop' }, parts: [] } + ])), + sessionDelete: jest.fn(async () => ({})), + eventSubscribe: jest.fn(async () => ({ + stream: (async function* () { + yield { type: 'message.updated', properties: { info: { sessionID: SESSION_ID, finish: 'stop' } } }; + })() + })) +}; + +jest.unstable_mockModule('@opencode-ai/sdk', () => ({ + createOpencodeClient: jest.fn(() => ({ + config: { providers: sdkMocks.configProviders, update: sdkMocks.configUpdate }, + tool: { ids: sdkMocks.toolIds }, + session: { + create: sdkMocks.sessionCreate, + prompt: sdkMocks.sessionPrompt, + messages: sdkMocks.sessionMessages, + delete: sdkMocks.sessionDelete + }, + event: { subscribe: sdkMocks.eventSubscribe } + })) +})); + +const { createApp } = await import('../src/proxy.js'); + +const readSseData = (text) => text + .split('\n') + .filter((line) => line.startsWith('data: ') && line !== 'data: [DONE]') + .map((line) => JSON.parse(line.slice(6))); + +const visibleTextFor = (path, text) => { + const frames = readSseData(text); + if (path === '/v1/responses') { + return frames.filter((event) => event.type === 'response.output_text.delta').map((event) => event.delta); + } + if (path === '/v1/messages') { + return frames + .filter((event) => event.type === 'content_block_delta' && event.delta?.type === 'text_delta') + .map((event) => event.delta.text); + } + return frames + .flatMap((event) => event.choices || []) + .flatMap((choice) => (choice.delta?.content ? [choice.delta.content] : [])); +}; + +const toolCallsFor = (path, text) => { + const frames = readSseData(text); + if (path === '/v1/responses') { + const args = frames + .filter((event) => event.type === 'response.function_call_arguments.delta') + .map((event) => event.delta) + .join(''); + return frames + .filter((event) => event.type === 'response.output_item.added' && event.item?.type === 'function_call') + .map((event) => ({ ...event.item, arguments: args })); + } + if (path === '/v1/messages') { + const input = frames + .filter((event) => event.type === 'content_block_delta' && event.delta?.type === 'input_json_delta') + .map((event) => event.delta.partial_json) + .join(''); + return frames + .filter((event) => event.type === 'content_block_start' && event.content_block?.type === 'tool_use') + .map((event) => ({ ...event.content_block, arguments: input })); + } + return frames + .flatMap((event) => event.choices || []) + .flatMap((choice) => choice.delta?.tool_calls || []) + .map((call) => ({ + id: call.id, + name: call.function?.name, + arguments: call.function?.arguments + })); +}; + +const buildApp = () => createApp({ + PORT: 10000, + API_KEY: 'test-key', + OPENCODE_SERVER_URL: 'http://127.0.0.1:10001', + REQUEST_TIMEOUT_MS: 120, + RETRY_MAX_RETRIES: 0, + DISABLE_TOOLS: true, + DEBUG: false +}).app; + +const readTool = { + type: 'function', + function: { + name: 'read', + description: 'Read a file', + parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] } + } +}; + +const payloads = [ + ['chat', '/v1/chat/completions', { model: 'opencode/kimi-k2.5', stream: true, messages: [{ role: 'user', content: 'Read a file' }], tools: [readTool] }], + ['messages', '/v1/messages', { + model: 'opencode/kimi-k2.5', + max_tokens: 100, + stream: true, + messages: [{ role: 'user', content: 'Read a file' }], + tools: [{ name: 'read', description: 'Read a file', input_schema: readTool.function.parameters }] + }], + ['responses', '/v1/responses', { + model: 'opencode/kimi-k2.5', + input: 'Read a file', + stream: true, + tools: [{ type: 'function', name: 'read', parameters: readTool.function.parameters }] + }] +]; + +const plainTextPayloads = [ + ['chat', '/v1/chat/completions', { model: 'opencode/kimi-k2.5', stream: true, messages: [{ role: 'user', content: 'Say hello' }] }], + ['messages', '/v1/messages', { model: 'opencode/kimi-k2.5', max_tokens: 100, stream: true, messages: [{ role: 'user', content: 'Say hello' }] }], + ['responses', '/v1/responses', { model: 'opencode/kimi-k2.5', input: 'Say hello', stream: true }], +]; + +describe('route snapshot source selection', () => { + beforeEach(() => { + jest.clearAllMocks(); + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.updated', + properties: { info: { id: 'assistant-current', role: 'assistant', sessionID: SESSION_ID } }, + }; + yield { + type: 'message.part.updated', + properties: { part: { id: 'raw-part', messageID: 'assistant-current', type: 'text', sessionID: SESSION_ID }, delta: '{"id":"call_raw_complete","name":"read","arguments":{"path":"a.txt"}}' } + }; + // finish=tool keeps the collector waiting for a terminal snapshot; + // the tiny idle window then forces the recovery-poll path. + yield { type: 'message.updated', properties: { info: { id: 'assistant-current', role: 'assistant', sessionID: SESSION_ID, finish: 'tool' } } }; + })() + })); + }); + + test.each(payloads)('%s keeps a complete raw tool call when the poll snapshot is empty', async (_label, path, payload) => { + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'stop' }, parts: [] } + ]); + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toBe(''); + const calls = toolCallsFor(path, res.text); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ name: 'read' }); + expect(JSON.stringify(calls[0].arguments)).toContain('a.txt'); + expect(sdkMocks.sessionMessages).toHaveBeenCalled(); + if (_label === 'chat') { + expect(res.text).toContain('"finish_reason":"tool_calls"'); + expect(res.text).toContain('data: [DONE]'); + expect(res.text).not.toMatch(/"error"\s*:\s*\{/); + } else if (_label === 'messages') { + const frames = readSseData(res.text); + const terminalDelta = frames.find((frame) => frame.type === 'message_delta'); + expect(terminalDelta?.delta?.stop_reason).toBe('tool_use'); + expect(res.text).toContain('event: message_stop'); + expect(res.text).not.toContain('[DONE]'); + expect(res.text).not.toContain('event: error'); + } else { + const frames = readSseData(res.text); + expect(frames.at(-1)?.type).toBe('response.completed'); + expect(res.text).toContain('data: [DONE]'); + expect(res.text).not.toContain('response.failed'); + expect(res.text).not.toMatch(/"error"\s*:\s*\{/); + } + }); + + test.each(plainTextPayloads)('%s does not succeed with partial raw text when recovery polling finds an empty completed snapshot', async (label, path, payload) => { + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { + part: { id: 'partial-text', messageID: 'assistant-current', type: 'text', sessionID: SESSION_ID }, + delta: 'partial answer', + }, + }; + yield { + type: 'message.updated', + properties: { + info: { id: 'assistant-current', role: 'assistant', sessionID: SESSION_ID, finish: 'tool' }, + }, + }; + })(), + })); + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { id: 'assistant-current', role: 'assistant', finish: 'stop' }, parts: [] }, + ]); + + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toContain('partial answer'); + if (label === 'chat') { + const frames = readSseData(res.text); + expect(frames.at(-1)).toHaveProperty('error'); + expect(res.text).toContain('data: {"error":'); + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('"finish_reason":"stop"'); + } else if (label === 'messages') { + const frames = readSseData(res.text); + expect(frames.at(-1)).toMatchObject({ type: 'error', error: expect.any(Object) }); + expect(res.text).toContain('event: error'); + expect(res.text).not.toContain('[DONE]'); + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('message_delta'); + } else { + const frames = readSseData(res.text); + expect(frames.at(-1)?.type).toBe('response.failed'); + expect(res.text).toContain('data: [DONE]'); + expect(res.text).not.toContain('response.completed'); + } + }); + + test.each(payloads)('%s fails closed when the poll snapshot is a timeout partial', async (label, path, payload) => { + // finish=tool means the polled message is never terminal, so the poll + // returns lastPartial with partial: true after the request timeout. + sdkMocks.sessionMessages.mockResolvedValue([ + { info: { role: 'assistant', finish: 'tool' }, parts: [{ type: 'text', text: 'partial answer' }] } + ]); + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(res.text).toContain('Recovery poll returned incomplete response'); + if (label === 'chat') { + expect(res.text).not.toContain('data: [DONE]'); + expect(res.text).not.toContain('finish_reason'); + } else if (label === 'messages') { + expect(res.text).not.toContain('message_stop'); + expect(res.text).not.toContain('message_delta'); + } else if (label === 'responses') { + expect(res.text).not.toContain('response.completed'); + } + }, 15000); + + test.each(payloads)('%s delivers only snapshot tool call (b.txt) and does not combine raw streamed call (a.txt)', async (_label, path, payload) => { + sdkMocks.sessionMessages.mockResolvedValue([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ + type: 'text', + text: '{"id":"call_snap_complete","name":"read","arguments":{"path":"b.txt"}}' + }] + } + ]); + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(visibleTextFor(path, res.text).join('')).toBe(''); + const calls = toolCallsFor(path, res.text); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ name: 'read' }); + expect(JSON.stringify(calls[0].arguments)).toContain('b.txt'); + expect(JSON.stringify(calls[0].arguments)).not.toContain('a.txt'); + expect(res.text).not.toContain('a.txt'); + expect(res.text).not.toContain('response.failed'); + expect(res.text).not.toMatch(/"error"\s*:\s*\{/); + }); + + test.each(payloads)('%s succeeds with 1 tool call under parallel_tool_calls: false without failing with parallel_external_tool_calls', async (_label, path, payload) => { + sdkMocks.sessionMessages.mockResolvedValue([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ + type: 'text', + text: '{"id":"call_snap_complete","name":"read","arguments":{"path":"b.txt"}}' + }] + } + ]); + const nonParallelPayload = { + ...payload, + parallel_tool_calls: false, + disable_parallel_tool_use: true + }; + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(nonParallelPayload); + expect(res.statusCode).toBe(200); + expect(res.text).not.toContain('parallel_external_tool_calls'); + const calls = toolCallsFor(path, res.text); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ name: 'read' }); + expect(JSON.stringify(calls[0].arguments)).toContain('b.txt'); + expect(JSON.stringify(calls[0].arguments)).not.toContain('a.txt'); + expect(res.text).not.toContain('response.failed'); + expect(res.text).not.toMatch(/"error"\s*:\s*\{/); + }); + + test.each(payloads)('%s cleanly delivers snapshot tool calls when repairing from malformed raw markup without combining raw deltas', async (_label, path, payload) => { + const malformedDelta = 'prefix {"name":"read" arguments suffix'; + sdkMocks.eventSubscribe.mockImplementation(async () => ({ + stream: (async function* () { + yield { + type: 'message.part.updated', + properties: { part: { type: 'text', sessionID: SESSION_ID }, delta: malformedDelta } + }; + yield { type: 'message.updated', properties: { info: { sessionID: SESSION_ID, finish: 'stop' } } }; + })() + })); + sdkMocks.sessionMessages.mockResolvedValue([ + { + info: { role: 'assistant', finish: 'stop' }, + parts: [{ + type: 'text', + text: '{"id":"call_snap_repaired","name":"read","arguments":{"path":"b.txt"}}' + }] + } + ]); + const res = await request(buildApp()) + .post(path) + .set('Authorization', 'Bearer test-key') + .set('anthropic-version', '2023-06-01') + .send(payload); + expect(res.statusCode).toBe(200); + expect(res.text).not.toContain('malformed_external_tool_call'); + const calls = toolCallsFor(path, res.text); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ name: 'read' }); + expect(JSON.stringify(calls[0].arguments)).toContain('b.txt'); + expect(res.text).not.toContain('response.failed'); + expect(res.text).not.toMatch(/"error"\s*:\s*\{/); + }); +}); diff --git a/tests/stream-source.test.js b/tests/stream-source.test.js new file mode 100644 index 0000000..9858eb2 --- /dev/null +++ b/tests/stream-source.test.js @@ -0,0 +1,97 @@ +import { resolveAuthoritativeStreamSource, isMalformedExternalToolCallError } from '../src/tool-runtime/stream-source.js'; + +const rawInput = (overrides = {}) => ({ + collected: { content: 'raw content', reasoning: 'raw reasoning', authoritative: false }, + rawContent: 'raw content', + rawReasoning: 'raw reasoning', + ...overrides +}); + +describe('resolveAuthoritativeStreamSource', () => { + test('an empty completed poll falls back to the collector/raw group', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + polledSnapshot: { content: '', reasoning: '', error: null, complete: true } + })); + expect(source).toMatchObject({ + kind: 'raw', + fromSnapshot: false, + content: 'raw content', + reasoning: 'raw reasoning' + }); + }); + + test('a partial poll never wins even when it has text', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + polledSnapshot: { content: 'partial text', reasoning: '', error: null, partial: true } + })); + expect(source).toMatchObject({ kind: 'raw', fromSnapshot: false, content: 'raw content' }); + }); + + test('a complete non-empty poll wins over raw and collector snapshots', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + collected: { content: 'collector repaired', reasoning: '', authoritative: true }, + polledSnapshot: { content: 'polled answer', reasoning: 'polled reasoning', error: null, complete: true } + })); + expect(source).toEqual({ + kind: 'polled-snapshot', + fromSnapshot: true, + content: 'polled answer', + reasoning: 'polled reasoning' + }); + }); + + test('a collector-authoritative repair wins when no poll snapshot exists', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + collected: { content: 'repaired answer', reasoning: 'repaired reasoning', authoritative: true } + })); + expect(source).toEqual({ + kind: 'collector-snapshot', + fromSnapshot: true, + content: 'repaired answer', + reasoning: 'repaired reasoning' + }); + }); + + test('equal or empty collector text is never authoritative', () => { + const equal = resolveAuthoritativeStreamSource(rawInput({ + collected: { content: 'raw content', reasoning: 'raw reasoning', authoritative: true } + })); + expect(equal).toMatchObject({ kind: 'raw', fromSnapshot: false, content: 'raw content' }); + + const empty = resolveAuthoritativeStreamSource(rawInput({ + collected: { content: '', reasoning: '', authoritative: true } + })); + expect(empty).toMatchObject({ kind: 'raw', fromSnapshot: false, content: 'raw content' }); + }); + + test('an error-bearing poll falls back to the collector/raw group', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + polledSnapshot: { content: 'ignored', reasoning: '', error: { name: 'Boom' }, complete: true } + })); + expect(source).toMatchObject({ kind: 'raw', fromSnapshot: false, content: 'raw content' }); + }); + + test('an error-bearing collector falls back to the raw group', () => { + const source = resolveAuthoritativeStreamSource(rawInput({ + collected: { content: 'ignored', reasoning: '', authoritative: true, error: { name: 'Boom' } } + })); + expect(source).toMatchObject({ kind: 'raw', fromSnapshot: false, content: 'raw content' }); + }); +}); + +describe('isMalformedExternalToolCallError', () => { + test('returns true for errors with code malformed_external_tool_call', () => { + const error = new Error('Malformed markup'); + error.code = 'malformed_external_tool_call'; + expect(isMalformedExternalToolCallError(error)).toBe(true); + expect(isMalformedExternalToolCallError({ code: 'malformed_external_tool_call' })).toBe(true); + }); + + test('returns false for other errors, strings, or null/undefined', () => { + expect(isMalformedExternalToolCallError(new Error('other error'))).toBe(false); + expect(isMalformedExternalToolCallError({ code: 'parallel_external_tool_calls' })).toBe(false); + expect(isMalformedExternalToolCallError(null)).toBe(false); + expect(isMalformedExternalToolCallError(undefined)).toBe(false); + expect(isMalformedExternalToolCallError('malformed_external_tool_call')).toBe(false); + }); +});