From 6bd72825ffed790d7161f7f77a5275162702d067 Mon Sep 17 00:00:00 2001 From: Khalil Gharbaoui Date: Sun, 15 Nov 2026 22:09:26 +0100 Subject: [PATCH] Wait 90 s before closing a silent turn Claude Code announces a retry after a stalled API connection at about the 60 s mark, and on the maintainer's machine 7 of 51 stream timeout closes came 0 to 2 s before that announcement could hold the watchdog. The default goes to 90 s; CLAUDE_CODE_RESULT_FALLBACK_MS still overrides it (h #g318). --- AGENTS.md | 4 +-- TODO.md | 17 +++---------- docs/agents-history.md | 31 +++++++++++++++++++++++ docs/configuration/environment.md | 2 +- docs/guides/tool-proxy.md | 2 +- docs/troubleshooting/known-limitations.md | 2 +- skills/claude-code-plugin/SKILL.md | 2 +- src/claude-session-bun.ts | 2 +- src/cli-events.ts | 6 ++--- src/turn-state.ts | 8 ++++-- test/result-fallback.test.ts | 21 +++++++++++++++ 11 files changed, 71 insertions(+), 26 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4fc4804..6c1972d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -75,7 +75,7 @@ changing, reverting or "simplifying" a rule. - **An abort that lands while a turn is being PREPARED must stop it, and the only thing that can see it is a watch created before the prologue's first await** (`watchTurnAbort`, `src/turn-abort.ts`): `addEventListener("abort")` on a signal that already aborted never fires, so the stream's own handler, which cannot exist until the prologue has finished, missed every stop in that window and the turn spawned, wrote and billed anyway (measured: a turn alive at 5,874 ms against a 5,000 ms signal). The handler is registered through `onAbort` in the **synchronous** part of `start()` now, never inside `setup()`, and `TurnState.cliAskedForWork` (set at exactly two places, the envelope write and the top of the tool-result branch) selects a new FIRST branch in front of the existing ones: close the stream, discard a proxy MCP server this turn created but never handed to a child, send **no** `interrupt` (a reused process's last turn is not this doStream's to stop) and leave an earlier step's pending proxy calls untouched, so #g26 and #g82 still own how they end. Three prologue checks, plus `stoppedBeforeWork()` on entry to `setup()`, at both spawn sites and after `interruptTurn`. **The raced `detectCliVersion` is left running on purpose**, because its answer is cached process-wide and cancelling it would be #g181 again; only the MCP connect wait takes the signal (`getRuntimeMcpStatus({ signal })`), because its poll loop is ours. `test/prologue-abort.test.ts`. (h #g182) - **A proxy deadline must not count the operator's time on a permission prompt.** Both timers go through `armProxyDeadline` and its `ProxyDeadlineGuard`; `isProxyCallStillServed` keeps a call only when `emitted` **and** the session reads `busy`, rechecking every `PROXY_DEADLINE_RECHECK_MS`. No guard, a throwing guard or `unknown` end it as before. `test/broker.test.ts`. (h #g27) **Live on opencode 2.0.22 with a real prompt in the way (h #g219)**: with the host config asking for `write`/`edit`, a proxied `write` was queued, the step drained, opencode raised a permission request, the call sat parked for the twelve seconds the operator took, and `POST /api/session/{id}/permission/{requestID}/reply {"decision":"once"}` resolved it in the same step (`resolved pending proxy call`) and the file was written. Every V2 probe before this ran with `--auto`, so the prompt path had never been exercised there. **Before (h #g253) the guard itself could never keep a call on V2**: its `busy` read had no route there, and `unknown` ends the call at its deadline. It reads the event-fed run state now, and live on 2.0.31 a proxied `write` with a 5-second deadline waited out a 15-second permission hold (`proxy call past its deadline, but opencode is still serving it; waiting`) and was written, where v0.58.0 timed it out twice and Claude told the operator the tool kept failing. `test/v2-session-events.test.ts`. - **Abort sends the CLI an `interrupt` control request** (`interruptTurn`), or the turn runs on and bills. `noteTurnStarted` marks in flight at every stdin write asking for work, and the terminal `result` clears it inside the `rl` handler, **never** a permanent `lineEmitter` listener. `test/session-manager.test.ts`. (h #g59) **Live end to end on Claude Code 2.1.288 + opencode 1.18.35 (h #g219)**: `POST /session/{id}/abort` mid-reply logged `abort signal received mid-turn, starting grace period`, the CLI answered 21 ms later with a `result` of `error_during_execution` carrying `costUsd: 0` and `outputTokens: 0`, `interrupt sent for aborted turn {idle: true}` followed, and **the CLI's own transcript recorded a `[Request interrupted by user]` user record** right after the truncated assistant record (`stop_reason: null`). The next turn **reused the same process and the same Claude session id** and answered normally, so the interrupt ends the turn without ending the conversation. -- **The wire-inactivity watchdog counts wall-clock time, sleep included, so a late fire re-arms and a closing fire interrupts the CLI** (`startResultFallback`, `RESULT_FALLBACK_LATE_SLACK_MS` 5 s, h #g316). Measured on 2026-11-14 from a subagent's own transcript and `pmset -g log`: the stream was still delivering thinking at 20:09:42Z, the Mac slept 20:10:23Z to 20:20:56Z with a dark wake at 20:15:28Z, and the 60 s timer fired at 20:15:31Z, 3 s into the dark wake, with at most 41 s of awake time behind it, so ten minutes of sleep read as ten minutes of a silent CLI. A fire more than the slack past its deadline means this process was not running (a sleep, or an event-loop stall with the CLI's output possibly unread in the pipe), so it waits a fresh full delay; the fire that does close the turn sends `interruptTurn`, exactly as an abort does (h #g59), because that CLI retried after the wake and ran on unattached for 28 minutes, its proxied calls each waiting out their 10-minute deadline. A proxied call queued with no stream attached is deliberately NOT rejected at once: that is the late-result recovery path (h #g82, h #g119), and the interrupt removes the cause. **The start watchdog gets the same late-fire rule** (a late fire re-arms instead of respawning), **and both watchdogs wait out a backoff the CLI announces** (`noteAnnouncedRetry`): measured on 2.1.322 against a local server answering 529 with `retry-after: 4`, the `system`/`api_retry` frame arrives 60 ms after the failed request with `retry_delay_ms: 4000` and the next request follows 4.0 s later, so whichever watchdog is armed waits that much longer (capped at 30 minutes; the CLI's own `max_retries` bounds how often). Without it a backoff longer than the window closed the turn (and, with the interrupt, would now stop the retrying CLI) or respawned a child that was working as designed. `api_retry` frames do not count as model progress. `test/result-fallback-sleep.test.ts` fakes the sleep through `_setResultFallbackClock`, never by blocking the loop, and jumps the clock only once the fake CLI holds the envelope, which is after the arm. +- **The wire-inactivity watchdog counts wall-clock time, sleep included, so a late fire re-arms and a closing fire interrupts the CLI** (`startResultFallback`, `RESULT_FALLBACK_LATE_SLACK_MS` 5 s, h #g316). Measured on 2026-11-14 from a subagent's own transcript and `pmset -g log`: the stream was still delivering thinking at 20:09:42Z, the Mac slept 20:10:23Z to 20:20:56Z with a dark wake at 20:15:28Z, and the 60 s timer fired at 20:15:31Z, 3 s into the dark wake, with at most 41 s of awake time behind it, so ten minutes of sleep read as ten minutes of a silent CLI. A fire more than the slack past its deadline means this process was not running (a sleep, or an event-loop stall with the CLI's output possibly unread in the pipe), so it waits a fresh full delay; the fire that does close the turn sends `interruptTurn`, exactly as an abort does (h #g59), because that CLI retried after the wake and ran on unattached for 28 minutes, its proxied calls each waiting out their 10-minute deadline. A proxied call queued with no stream attached is deliberately NOT rejected at once: that is the late-result recovery path (h #g82, h #g119), and the interrupt removes the cause. **The start watchdog gets the same late-fire rule** (a late fire re-arms instead of respawning), **The inactivity window is 90 s by default, not 60** (`resolveResultFallbackMs`, h #g318): opencode's database held 51 `stream timeout` closes from 2026-09 to 2026-11-15, and 7 of them came 0 to 2 s before Claude Code's own `api_error` record (`Connection error.`, 288 of 292 such records in both config dirs): the CLI gave up on a stalled connection at about the moment the 60 s window ran out, so the turn was closed before the retry it announces could hold the watchdog. Its own backoff is short (median 1 s, p90 10 s, max 40 s), so 30 s of margin covers it; the cost is a truly wedged CLI noticed 30 s later. `CLAUDE_CODE_RESULT_FALLBACK_MS` stays the override and the test seam, so no option. **and both watchdogs wait out a backoff the CLI announces** (`noteAnnouncedRetry`): measured on 2.1.322 against a local server answering 529 with `retry-after: 4`, the `system`/`api_retry` frame arrives 60 ms after the failed request with `retry_delay_ms: 4000` and the next request follows 4.0 s later, so whichever watchdog is armed waits that much longer (capped at 30 minutes; the CLI's own `max_retries` bounds how often). Without it a backoff longer than the window closed the turn (and, with the interrupt, would now stop the retrying CLI) or respawned a child that was working as designed. `api_retry` frames do not count as model progress. `test/result-fallback-sleep.test.ts` fakes the sleep through `_setResultFallbackClock`, never by blocking the loop, and jumps the clock only once the fake CLI holds the envelope, which is after the arm. - **`idleProcessTimeoutMs` is off unless set** (`resolveIdleProcessTimeoutMs`, so unset and `0` both arm nothing). Unref'd, cancelled by every get/set/detach/spawn/exit, deleting only the same process object; a process found `turnInFlight` is re-armed, never killed. `test/session-manager.test.ts`. (h #g60) - **The child's stdin needs its own `error` listener, and `proc.on("error")` is not it**: an `error` on a listener-less stream throws inside **opencode's** process. `spawnClaudeProcess` attaches one, logs at WARN, calls `settleTurn`, and deliberately does not end the turn. `test/session-manager.test.ts`. (h #g61) - **LRU eviction must never take a process that is mid-turn**: `evictIfNeeded` walks insertion order for the first with `turnInFlight !== true`, and when all are busy evicts **nothing** and warns. Do not "restore" the one-liner. Cap is **16**, and idle eviction stays off unless `idleProcessTimeoutMs` is set, so the cap is the only bound on a default install. `test/session-manager.test.ts`. (h #g62, #g179) @@ -404,7 +404,7 @@ Opt-in. `spawnInteractiveProcess` returns an `ActiveProcess`-shaped shim so doSt - This widens (h #g114) from "no CLI process" to **"no CLI inference"**, which is what it always protected: the version row has spawned via `detectCliVersion` since the beginning. Nothing in the doctor sends a prompt to a model. - **`/advisor` does not exist headless and `/reload-plugins` is an action, not a diagnostic, so neither has code.** Re-probed on 2.1.293 (h #g241): `claude -p "/advisor"` answers "/advisor isn't available in this environment" with zero turns and zero cost, although the CHANGELOG lists a text form for `-p`; it is gated. **`/context` DOES answer headless**, zero turns, zero cost, with a full per-category breakdown (system prompt, system tools, MCP tools and instructions, deferred tools, agents, memory, skills, messages), which is what a `/claude-code-doctor context` would print; whether a `--resume` of a live session's id writes to its transcript is unmeasured. `advisor` is `requires:{ink:true}` and gated behind first-party auth plus a server-side flag; the live headless probe found it in neither `slash_commands` nor `tools`. `reload-plugins` IS non-interactive and has a `{subtype:"reload_plugins", hold_on_cache_impact}` control request, but reading it changes the session, so it was recorded for whoever wants skill-bridge hot reload rather than wired. Same discipline as the `tombstone` verdict: a parser for a shape that never arrives is an invented shape. (h #g167) - **A hook the user configured that FAILED is otherwise completely silent, and the `SessionStart` family needs no flag to reach a plugin spawn.** The CLI's own gate is two-tier (`i3`): `["SessionStart","Setup"]` always emit `hook_started` / `hook_progress` / `hook_response`, every other hook event waits for `--include-hook-events`, which the plugin does not pass and should not. Measured on 2.1.280 in the plugin's own argv: three `SessionStart:startup` hooks produced three of each before `system`/`init`, and a deliberately broken one answered `{"exit_code":3,"outcome":"error"}` on a turn that still finished `{"subtype":"success","is_error":false}`. `reportHookEvent` WARNs once per `event:name:outcome:exit` (never per `hook_id`, a fresh uuid every spawn) and keeps the row for `/claude-code-doctor`; there is no transcript note, because a broken hook belongs to the session rather than to this answer. **`outcome` is the verdict, `cancelled` is not a failure** (an abort produces it), and **only `stderr` is ever read, capped at 200 chars**: `output` and `stdout` are what the CLI splices into the model's context. `test/cli-events.test.ts`, `test/cli-events-stream.test.ts`, `test/doctor.test.ts`. (h #g184, #g185) -- **`tool_progress` already does the only job it could do, so the watchdog was deliberately not touched.** Its heartbeat emitter is ungated (a 30 s `setInterval` around every main-conversation tool call; the `bash_progress` emitter next to it is the gated one, needing `CLAUDE_CODE_REMOTE` or `CLAUDE_CODE_CONTAINER_ID`), and `createLineHandler` calls `startResultFallback` on EVERY line before parsing. Measured on 2.1.280 over a CLI-executed `sleep 95`: frames at 30, 60 and 90 s, longest stdout gap over the whole 102 s turn 30.0 s, against a 60 s wire-inactivity watchdog. So `reportToolProgress` only logs, at INFO, naming the tool, its elapsed seconds and the **real** `tool_use_id`: a heartbeat's own `tool_use_id` is synthetic (`-heartbeat-N`) and the tool call is in `parent_tool_use_id`. It is the one reporter in `src/cli-events.ts` with no dedup, because its identity is `(id, elapsed)` and the CLI never repeats a pair; a `subagent_retry` does repeat and dedupes at NOTICE. (h #g184, #g185) +- **`tool_progress` already does the only job it could do, so the watchdog was deliberately not touched.** Its heartbeat emitter is ungated (a 30 s `setInterval` around every main-conversation tool call; the `bash_progress` emitter next to it is the gated one, needing `CLAUDE_CODE_REMOTE` or `CLAUDE_CODE_CONTAINER_ID`), and `createLineHandler` calls `startResultFallback` on EVERY line before parsing. Measured on 2.1.280 over a CLI-executed `sleep 95`: frames at 30, 60 and 90 s, longest stdout gap over the whole 102 s turn 30.0 s, against the then 60 s (now 90 s, h #g318) wire-inactivity watchdog. So `reportToolProgress` only logs, at INFO, naming the tool, its elapsed seconds and the **real** `tool_use_id`: a heartbeat's own `tool_use_id` is synthetic (`-heartbeat-N`) and the tool call is in `parent_tool_use_id`. It is the one reporter in `src/cli-events.ts` with no dedup, because its identity is `(id, elapsed)` and the CLI never repeats a pair; a `subagent_retry` does repeat and dedupes at NOTICE. (h #g184, #g185) - **`tool_use_summary`, `prompt_suggestion` and `command_lifecycle` are deliberately ignored, each for a measured reason, and none of them is reachable by accident.** `tool_use_summary` needs `CLAUDE_CODE_EMIT_TOOL_USE_SUMMARIES` in the env and costs a model call per tool batch to narrate tool rows opencode already renders. `prompt_suggestion` needs `--prompt-suggestions` plus a turn-origin check and a server-side gate, and produced zero frames even with the flag on; it would also be putting words in the operator's mouth. `command_lifecycle` needs a `uuid` on the inbound envelope, which `src/message-builder.ts` does not write, so the CLI's own schema says it can never fire here (confirmed both ways: zero without, `queued`/`started`/`completed` with). Being in the CLI's SDK allowlist is necessary and nowhere near sufficient. Do not re-derive these without re-running the probes in the history entry. (h #g184) - **`system`/`status`, `thinking_tokens`, `task_started` and `task_notification` DO all arrive in the plugin's mode, and all four are deliberately unparsed.** Measured live on 2.1.286 and 2.1.280 (`status` and `thinking_tokens` on an ordinary turn; the task pair needs a CLI-side background task, which a default `proxyTools` prevents). `status` is `"requesting"` / `"compacting"` and says only what the plugin just did; `thinking_tokens` is explicitly "approximate progress for spinners/pills, not the authoritative billed output_tokens", so adding it to any tally would double-count (the real number is `usage.output_tokens_details.thinking_tokens`); the task pair's one plugin-visible hazard is fixed at the replay instead (h #g189). 2.1.286 also adds ten `system` subtypes over 2.1.280, seven of them `ui_*` for the fullscreen surface, plus `per_turn_effort_changed`, `session_metadata` and `session_title_changed`: none is parsed either, and `init` gained `per_turn_effort_active`. (h #g191) - **A proxied tool costs one extra CLI API call per process, it is a `ToolSearch`, and it is inherent only for a tool that is not always loaded** (since h #g276, `bash`, `edit` and `write` are, see the proxy rule on `alwaysLoadProxyTools`). Claude Code defers MCP tools behind `ToolSearch`, so the model spends a request finding a proxied tool before calling it; built-ins are never deferred. Measured on 2.1.280: 3 API calls proxied against 2 unproxied for one `echo`, amortising to 4 against 3 for two calls (one `ToolSearch` either way), unchanged at 28 tools. **There is no post-tool "digest" turn in this architecture**, so do not port that claim from Meridian. `ENABLE_TOOL_SEARCH=0` removes the call and measured 2.5 to 4 times the total cost, so never set it and never accept "proxying is expensive" without checking it is unset. No code changed for this. (h #g166) diff --git a/TODO.md b/TODO.md index 7cfb8f8..cce1e45 100644 --- a/TODO.md +++ b/TODO.md @@ -182,20 +182,6 @@ them ran in this directory. Questions the maintainer still owes an answer on. Written here the turn they are raised, so they survive context compaction; removed when answered, done or dropped. -- 2026-11-14: watchdog follow-up: should the plugin wait longer after an API error than Claude Code - announces? Maintainer 2026-11-15: "maybe yes how much longer is needed is this something usefull for - all plugin users?" Measured 2026-11-15: opencode.db holds 51 real `▌ **stream timeout:**` closes - since 2026-09 (22 Sep, 19 Oct, 10 Nov, about four a week). 11 had a Claude Code `api_error` in the - same directory during the step, and 7 of those closed 0 to 2 s after it: Claude Code gave up on a - stalled connection (`Connection error.`, 288 of the 292 api_error records in both config dirs' - transcripts) at about the moment the 60 s inactivity watchdog fired, so the turn was closed just - before the retry was announced (v0.73.4 holds the watchdog only for a retry announced first). Claude - Code's own backoff: median 1 s, p90 10 s, max 40 s. Silence in the transcript before a Connection - error clusters at about 30 s (median 30.0 s) and at 300 s or more (29); Claude Code's own stall - timeout was not read off the binary (the shell redacted it). Recommendation: lengthen the inactivity - window from 60 s to 90 s by default for everyone (`CLAUDE_CODE_RESULT_FALLBACK_MS` stays the - override); cost: a truly hung CLI is noticed 30 s later. The other 40 closes have no retry behind - them (the sleep case is fixed in v0.73.4); their cause is not diagnosed. - 2026-11-10: what to build next from Future Features (doctor warnings for setups that fail silently, model aliases such as `opus`, a `task_send` to steer a background subagent, continuing a terminal `claude` session in opencode): pick any, or hold. @@ -254,6 +240,9 @@ Verification still owed, copied from the vault's `Future Features.md` "Next up" ## Done +- 2026-11-15: **done** (h #g318): the wire-inactivity watchdog defaults to 90 s instead of 60, because 7 + of 51 `stream timeout` closes on this machine came 0 to 2 s before Claude Code announced a retry after + a stalled connection. Still open: the other 40 closes had no retry behind them and are not diagnosed. - 2026-11-15: **done**, by the maintainer's own config: `AGENT_GUARD_PROMPT_GUARD_MODE=warn` set globally, so agent-guard warns instead of blocking a prompt with secret-like values. The stuck session `ses_e324dbf7bffeRHQIz10AIxm7dT` recovered: its 03:28:31Z replay (208,149 characters, the diff --git a/docs/agents-history.md b/docs/agents-history.md index 6c1254c..1653f94 100644 --- a/docs/agents-history.md +++ b/docs/agents-history.md @@ -13186,3 +13186,34 @@ Shipped as v0.73.5 on 2026-11-15 (PR #189), a patch: CI green on Linux and Windo `569ae0e`, on the merge commit `51bc536` before the tag and on the release commit `879d36b`, the publish confirmed by the `Publish package` step's tail and the version document's shasum (`0f49e6ca`), with a GitHub Release. + + +## The inactivity watchdog closed turns just before Claude Code announced a retry (2026-11-15) + +h #g318. The follow-up to h #g316, which made both watchdogs wait out a backoff the CLI announces +(`noteAnnouncedRetry`) but only when the `system`/`api_retry` frame arrives before the window runs out. +The maintainer asked whether the plugin should wait longer than Claude Code announces, and whether +that helps every user. + +Measured on this machine on 2026-11-15, read-only: + +- opencode.db: 51 assistant messages carry a text part that starts with `▌ **stream timeout:**` + (22 in September, 19 in October, 10 in November), about four a week. A plain `LIKE` found 108, the + rest being messages that quote the note. +- Both config dirs' Claude Code transcripts hold 292 `system`/`api_error` records: 288 + `Connection error.`, 3 expired OAuth tokens, 1 `Request timed out.` Claude Code's announced backoff + (`retryInMs`): median 1,003 ms, p90 9,887 ms, max 39,611 ms. +- 11 of the 51 closes had an `api_error` in the same directory during the step, and 7 of those came + 0 to 2 s after it: the CLI gave up on a stalled connection at about the moment the 60 s window ran + out, and the turn was closed before the retry could hold it. The other 4 closed 27 to 153 s after + their retry, so a retry was not their cause. +- The transcript silence before a `Connection error.` clusters at about 30 s (median 30.0 s, p75 + 30.6 s) and at 300 s or more (29 of 217). Claude Code's own stall timeout was not read off the + binary (the shell output redacted the surrounding code), so the tie is inferred from timing. + +Change: `resolveResultFallbackMs` defaults to 90,000 ms. That clears the tie with 30 s of margin over +the longest observed timing and over Claude Code's p90 backoff; the cost is that a CLI that is alive +but wedged is closed 30 s later. A network stall is not specific to this machine, so the default +changes for every user. `CLAUDE_CODE_RESULT_FALLBACK_MS` stays the override and the test seam, so +there is no option. The other 40 closes had no retry behind them and are not diagnosed (the sleep +case among them is h #g316). `test/result-fallback.test.ts` pins the default and the override. diff --git a/docs/configuration/environment.md b/docs/configuration/environment.md index 26f9995..c7d5fe5 100644 --- a/docs/configuration/environment.md +++ b/docs/configuration/environment.md @@ -16,7 +16,7 @@ Every variable the plugin itself reads, in one place. Config is read once at ope | `CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS` | Claude Code itself | Set to `0` for the interactive TUI unless you set it, so a long proxied call is not moved to the background after 120 s (headless sessions never do this). | | `CLAUDE_CODE_START_WATCHDOG_MS` | start watchdog | Milliseconds a `claude` process may stay completely silent on stdout after a turn is written, or after a proxy tool result should have resumed it, before the plugin acts. First expiry respawns the process and resumes the session; a second ends the turn with an error rather than hanging. Default `90000`; a positive integer is required and anything else falls back to that. A process that has not yet sent its first `system`/`init` line gets at least its `MCP_TIMEOUT` plus 30 s instead (150 s by default), because Claude Code sends that line only once its MCP servers have connected or timed out; from that line on, this value applies. Mainly a knob for reproducing the hang. | | `CLAUDE_CODE_LIVE_SWITCH_TIMEOUT_MS` | live model switch | Milliseconds one `set_model` or `apply_flag_settings` control request may take before a live model or effort change (`liveModelSwitch`) gives up, stops the process and falls back to starting a new one with `--resume`. Default `10000`; a positive integer is required and anything else falls back to that. Claude Code answered in 0.5 to 0.7 s for a model and well under a second for an effort when measured. | -| `CLAUDE_CODE_RESULT_FALLBACK_MS` | wire-inactivity watchdog | Milliseconds a `claude` process that has already produced output may stay silent on stdout before the turn is closed without a `result`. The close is announced in the reply as a `▌ **stream timeout:**` note. Default `60000`; a positive integer is required and anything else falls back to that. Like the start watchdog, mainly a knob for reproducing a hang. | +| `CLAUDE_CODE_RESULT_FALLBACK_MS` | wire-inactivity watchdog | Milliseconds a `claude` process that has already produced output may stay silent on stdout before the turn is closed without a `result`. The close is announced in the reply as a `▌ **stream timeout:**` note. Default `90000` (60000 before v0.73.6, which closed some turns just before Claude Code announced a retry after a stalled connection); a positive integer is required and anything else falls back to that. Like the start watchdog, mainly a knob for reproducing a hang. | | `OPENCODE_CLAUDE_CODE_LOG_FILE` | logger | `1` writes the log file, `0` forces it off even when `logging.file` is `true`. See [Logging](../configuration/logging.md). | | `OPENCODE_CLAUDE_CODE_LOG_DIR` | logger | Directory for the log file, overriding `logging.dir`. | | `OPENCODE_CLAUDE_CODE_MODEL_PROBE` | model catalog | `0` or `false` stops the background model probe for this environment (the test suite sets it). An answer already stored is still registered; `cliModelCatalog: false` is the config switch that ignores it too. | diff --git a/docs/guides/tool-proxy.md b/docs/guides/tool-proxy.md index f31997a..1a920d5 100644 --- a/docs/guides/tool-proxy.md +++ b/docs/guides/tool-proxy.md @@ -224,7 +224,7 @@ A deadline does not count time opencode is still spending on the call. When it p `question` keeps 30 minutes because it blocks on a human reading a form, and a form nobody answers is not an event. A positive `task` override restores a wall-clock backstop for operators who want one; if it fires, the error tells Claude not to "schedule a wake-up": that is a Claude Code affordance which cannot fire in this headless/proxy context, so deferring silently loses the work. -Two watchdogs are a different thing again and are unchanged: the start watchdog (90 s of complete silence after a turn is written, 150 s for a new process still waiting on its MCP servers, respawn then error, see `CLAUDE_CODE_START_WATCHDOG_MS`) and the wire-inactivity watchdog (60 s of silence after content, see `CLAUDE_CODE_RESULT_FALLBACK_MS`; when it fires the reply gets a `▌ **stream timeout:**` note so the turn does not just stop, and the CLI's turn is interrupted so it does not run on with nobody listening; a fire that comes well past its deadline because the machine slept waits a fresh 60 s instead). Those exist because a process that is alive but wedged emits no event to listen to, and a proxy call is never what they are waiting on: a CLI parked inside a proxied tool is producing nothing on purpose, and both watchdogs know that. +Two watchdogs are a different thing again and are unchanged: the start watchdog (90 s of complete silence after a turn is written, 150 s for a new process still waiting on its MCP servers, respawn then error, see `CLAUDE_CODE_START_WATCHDOG_MS`) and the wire-inactivity watchdog (90 s of silence after content, see `CLAUDE_CODE_RESULT_FALLBACK_MS`; when it fires the reply gets a `▌ **stream timeout:**` note so the turn does not just stop, and the CLI's turn is interrupted so it does not run on with nobody listening; a fire that comes well past its deadline because the machine slept waits a fresh 90 s instead). Those exist because a process that is alive but wedged emits no event to listen to, and a proxy call is never what they are waiting on: a CLI parked inside a proxied tool is producing nothing on purpose, and both watchdogs know that. If Claude nevertheless abandons the HTTP call, the plugin preserves narration emitted while opencode was running the tool, renders it on return, and delivers the late completion as a plain-text continuation naming the original call. It tells Claude not to run the tool again. A silent post-tool continuation gets one resumed-process retry, preserving the original model, account, effort, and proxy configuration; a second failure ends with an error rather than an indefinite hang. Buffered narration is capped at 500 lines and 2 MiB, with a warning if output was dropped. diff --git a/docs/troubleshooting/known-limitations.md b/docs/troubleshooting/known-limitations.md index 1ded397..d0ef552 100644 --- a/docs/troubleshooting/known-limitations.md +++ b/docs/troubleshooting/known-limitations.md @@ -17,7 +17,7 @@ sidebar: - **Empty text blocks are dropped.** Claude sometimes opens a `content_block_start` for text but never sends a delta. The plugin no longer emits the empty block (which was triggering Anthropic 400s like `cache_control cannot be set for empty text blocks`). - **Smart incomplete-turn continuation.** By default, the plugin keeps the current opencode stream open and feeds Claude CLI a small internal continuation message when Claude emits a `result` after reasoning/tool activity without a useful visible answer. It still stops normally on final-looking answers, questions, blockers, errors, aborts, or internal safety-budget exhaustion. It also resumes an answer the model was cut off mid-sentence: a `max_tokens` stop means truncation rather than completion, so the turn continues instead of ending on half a sentence, capped at 8 attempts and 10 minutes. Every other stop reason is taken at face value. Disable with `"autoContinueIncompleteTurns": false`. - **`AskUserQuestion`** from the CLI is converted into plain text content rather than forwarded as a tool call, unless `"Question"` is in `proxyTools`, in which case it is routed through opencode's native `question` tool (see [AskUserQuestion](../configuration/permissions.md#askuserquestion)). -- **Wire-inactivity watchdog.** Once the CLI has produced any content, the stream closes gracefully if stdout goes silent for 60 seconds without a `result` message arriving. Resets on every line received, so long mid-turn pauses (Sonnet between text-end and the next tool_use, for example) are tolerated. On a user-initiated abort, the watchdog shortens to 5 seconds. +- **Wire-inactivity watchdog.** Once the CLI has produced any content, the stream closes gracefully if stdout goes silent for 90 seconds without a `result` message arriving. Resets on every line received, so long mid-turn pauses (Sonnet between text-end and the next tool_use, for example) are tolerated. On a user-initiated abort, the watchdog shortens to 5 seconds. - **Context usage, not turn totals.** The CLI's `result` adds up every API call in a turn, and opencode reads a message's usage as how full the context is, so a tool-heavy turn looked several times its real size and triggered auto-compaction far below the window. The plugin reports the last API call's input and cache counts plus the turn's output instead. See [Per-turn stats](../guides/turn-stats.md) for what that does to opencode's cost figure. - **Lazy `cwd`.** The working directory is re-resolved at every request, so opencode's project-aware behavior works without restarting the plugin. - **Instruction files are sent once.** Claude Code 2.1.277 and later read a project's `AGENTS.md` themselves, and always read `CLAUDE.md`, while opencode sends its `AGENTS.md` in the system prompt, so the same file used to reach Claude twice (about 3,000 tokens for a 9 KB file, measured on Claude Code 2.1.322, on every new conversation and kept for its whole length). The plugin now tells Claude Code to skip its own copy of any `AGENTS.md` or `CLAUDE.md` whose whole text the system prompt already holds, including a `CLAUDE.md` that is a copy or a symlink of `AGENTS.md`. It never skips a file that imports another file, and when a conversation resumes it only skips a file whose current text the conversation already has, so an edit still reaches Claude. Your own `claudeMdExcludes` in Claude Code's settings still apply. Default transport only, not on Windows. diff --git a/skills/claude-code-plugin/SKILL.md b/skills/claude-code-plugin/SKILL.md index 5f0fac8..6f754e3 100644 --- a/skills/claude-code-plugin/SKILL.md +++ b/skills/claude-code-plugin/SKILL.md @@ -262,7 +262,7 @@ their secret values. Arbitrary MCP `{env:NAME}` placeholders are outside this li | `CLAUDE_CODE_MCP_AUTO_BACKGROUND_MS` | Set to `0` for the interactive TUI unless the operator set it, so a proxied call past 120 s stays in the foreground as on headless. `CLAUDE_CODE_DISABLE_MCP_TASK_BACKGROUND` is not set: it covers MCP-task servers only, which the proxy never is, and at `0` the CLI asks no server for a task. | | `CLAUDE_CODE_START_WATCHDOG_MS` | Positive integer ms before a headless start or proxy-result continuation is considered silent; default 90000 for missing/invalid/nonpositive values. First expiry respawns, second errors. Bookkeeping-only output is not progress. A process that has not sent its first `system`/`init` yet gets at least its `MCP_TIMEOUT` plus 30 s (150 s by default), since the CLI sends `init` only after its MCP servers connect or time out; the log's `beforeInit` says which delay fired. Keep within timer range; do not lower for routine config checks. | | `CLAUDE_CODE_LIVE_SWITCH_TIMEOUT_MS` | Positive integer ms one `set_model` or `apply_flag_settings` may take before a live model or effort switch gives up, stops the child and falls back to the respawn; default 10000 for missing/invalid/nonpositive values. Measured answers: 0.5 to 0.7 s and 20 to 140 ms. | -| `CLAUDE_CODE_RESULT_FALLBACK_MS` | Positive integer ms of stdout silence, after the CLI has produced output, before the turn is closed with no `result`; default 60000 for missing/invalid/nonpositive values. The close is announced in the reply as a `▌ **stream timeout:**` note, which is stripped from any rebuilt transcript, and interrupts the CLI's turn so it does not run on unattended. An aborted turn gets no note. A fire more than 5 s past its deadline (the machine slept, or the event loop stalled) waits a fresh full delay instead, because that silence was never observed, and an `api_retry` frame adds the `retry_delay_ms` the CLI announces (h #g316). The start watchdog (`CLAUDE_CODE_START_WATCHDOG_MS`) follows the same two rules. | +| `CLAUDE_CODE_RESULT_FALLBACK_MS` | Positive integer ms of stdout silence, after the CLI has produced output, before the turn is closed with no `result`; default 90000 for missing/invalid/nonpositive values (60000 before v0.73.6: Claude Code announces a retry after a stalled API connection at about 60 s, and the watchdog closed some of those turns first). The close is announced in the reply as a `▌ **stream timeout:**` note, which is stripped from any rebuilt transcript, and interrupts the CLI's turn so it does not run on unattended. An aborted turn gets no note. A fire more than 5 s past its deadline (the machine slept, or the event loop stalled) waits a fresh full delay instead, because that silence was never observed, and an `api_retry` frame adds the `retry_delay_ms` the CLI announces (h #g316). The start watchdog (`CLAUDE_CODE_START_WATCHDOG_MS`) follows the same two rules. | | `OPENCODE_CLAUDE_CODE_LOG_FILE` | Overrides `logging.file`: trimmed `0/false/no/off` are false; any other nonempty value is true; empty falls back to config. Prefer `1` or `0`. | | `OPENCODE_CLAUDE_CODE_LOG_DIR` | Overrides `logging.dir`. | | `OPENCODE_CLAUDE_CODE_MODEL_PROBE` | `0` or `false` stops the background model probe for this environment (the `npm test` script sets it); the stored answer is still registered. `cliModelCatalog: false` is the config switch and also ignores the stored answer. | diff --git a/src/claude-session-bun.ts b/src/claude-session-bun.ts index b28eb40..4b0afd2 100644 --- a/src/claude-session-bun.ts +++ b/src/claude-session-bun.ts @@ -1126,7 +1126,7 @@ export class ClaudeSession { submitMaxRetries: opts.submitMaxRetries ?? 8, debug: opts.debug ?? false, // The headless CLI's `tool_progress` heartbeat runs every 30 s, and the - // wire-inactivity watchdog it keeps quiet fires at 60 s. + // wire-inactivity watchdog it keeps quiet fires at 90 s. heartbeatMs: opts.heartbeatMs ?? 30_000, interruptGraceMs: opts.interruptGraceMs ?? 5_000, // Shorter than `bootQuietMs`, so a trust dialog is answered before boot diff --git a/src/cli-events.ts b/src/cli-events.ts index 89dca1b..b9a2954 100644 --- a/src/cli-events.ts +++ b/src/cli-events.ts @@ -1635,9 +1635,9 @@ export function _resetToolProgressReports(): void { * * The one job a progress frame could have done is already done for free. The * line handler calls `startResultFallback` on every line it receives, so a - * heartbeat resets the 60-second wire-inactivity watchdog like any other line, - * and the measured 30-second cadence keeps a long CLI-executed tool at half - * the deadline. What was missing was not the reset but the diagnostic: until + * heartbeat resets the 90-second wire-inactivity watchdog like any other line, + * and the measured 30-second cadence keeps a long CLI-executed tool at a third + * of the deadline. What was missing was not the reset but the diagnostic: until * now a turn stuck inside a ten-minute `Bash` wrote twenty `stream message * tool_progress` debug lines that named neither the tool nor how long it had * been going. That is what the INFO line below says. diff --git a/src/turn-state.ts b/src/turn-state.ts index 81f651c..412a99f 100644 --- a/src/turn-state.ts +++ b/src/turn-state.ts @@ -397,11 +397,15 @@ export interface TurnStateInit { /** * Wire-inactivity watchdog delay. Read per turn, because the regression test * sets the env var between runs. + * + * 90 s, not 60 (h #g318): Claude Code gives up on a stalled API connection and + * announces its retry at about the 60 s mark, and on this machine 7 turns were + * closed 0 to 2 s before that announcement could hold the watchdog. */ -function resolveResultFallbackMs(): number { +export function resolveResultFallbackMs(): number { const env = process.env.CLAUDE_CODE_RESULT_FALLBACK_MS const parsed = env ? Number.parseInt(env, 10) : NaN - return Number.isFinite(parsed) && parsed > 0 ? parsed : 60_000 + return Number.isFinite(parsed) && parsed > 0 ? parsed : 90_000 } /** Start watchdog delay, read per turn for the same reason. */ diff --git a/test/result-fallback.test.ts b/test/result-fallback.test.ts index d0b9e7b..906460b 100644 --- a/test/result-fallback.test.ts +++ b/test/result-fallback.test.ts @@ -19,6 +19,7 @@ import { STREAM_TIMEOUT_MARKER, formatStreamTimeoutNote } from "../src/cli-event import { createClaudeCode } from "../src/index.js" import { filterSideQuestionHistory } from "../src/message-builder.js" import { deleteActiveProcess, sessionKey } from "../src/session-manager.js" +import { resolveResultFallbackMs } from "../src/turn-state.js" /** Replays a fixed line sequence, then stays alive and silent forever. */ function createSilentFakeCli(lines: unknown[]) { @@ -143,3 +144,23 @@ test("the note rounds the silence to whole seconds, never to zero", () => { assert.match(formatStreamTimeoutNote(5_000), /5s/) assert.match(formatStreamTimeoutNote(10), /1s/) }) + +// 90 s, not 60 (h #g318): Claude Code announces a retry after a stalled API +// connection at about the 60 s mark, and a 60 s window closed those turns +// just before the announcement could hold it. +test("the inactivity window defaults to 90 s and the env var still overrides it", () => { + const saved = process.env.CLAUDE_CODE_RESULT_FALLBACK_MS + try { + delete process.env.CLAUDE_CODE_RESULT_FALLBACK_MS + assert.equal(resolveResultFallbackMs(), 90_000) + for (const bad of ["", "0", "-5", "soon"]) { + process.env.CLAUDE_CODE_RESULT_FALLBACK_MS = bad + assert.equal(resolveResultFallbackMs(), 90_000, JSON.stringify(bad)) + } + process.env.CLAUDE_CODE_RESULT_FALLBACK_MS = "60000" + assert.equal(resolveResultFallbackMs(), 60_000) + } finally { + if (saved === undefined) delete process.env.CLAUDE_CODE_RESULT_FALLBACK_MS + else process.env.CLAUDE_CODE_RESULT_FALLBACK_MS = saved + } +})