Skip to content

TD-5233: a reply that spends the output limit thinking is asked once more, and says so - #119

Open
captcrock wants to merge 2 commits into
mainfrom
td/5233-thinking-cap-retry
Open

captcrock wants to merge 2 commits into
mainfrom
td/5233-thinking-cap-retry

Conversation

@captcrock

@captcrock captcrock commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

What

A reasoning model on a tier with max_output_tokens: 4096 could spend the whole cap thinking. The stream ended finish_reason: length with no text and no tool call, and the turn failed as empty_completion. The chat told the user to start a new session even though the context was about a tenth full. Nothing retried, so the next turns failed the same way. The window had no copy for the code, so a toast also called it an unrecognised error.

Daemon:

  • stream_reply withdraws a reply that thought to the cap (assistant_retract, reason output_cap_thinking) and asks once more on its own counter. The new max_tokens is min(4× cap, max(32768, cap), room left in the window), sent only when that is at least twice the cap. provider_retry.max_retries: 0 turns this off. Later attempts of the same call keep the raised cap.
  • If the raised request is refused before any output (a 402 balance reservation, a 400, an overflow), the first result is kept so the real cause gets reported. A refusal of a later stall or guard retry is reported as that call's own error.
  • A turn that still has no answer fails as output_cap_thinking. The chat line names the config entry whose max_output_tokens applied, as presets.<preset>.<tier>, a backup's fallbacks[n], or the computer-use worker's preset.
  • A non-cap empty reply stays empty_completion, with accurate copy.
  • The "a new chat may help" sentence appears only when the prompt is at the compaction budget.
  • A Stop that arrives just as an empty reply comes back ends the turn as stopped, not failed.

UI:

  • Copy for both codes: a banner for output_cap_thinking, cleared once a later turn answers, live or replayed; a toast for empty_completion.
  • Activity title and preview for the new retract reason.
  • A reason-aware Working line.
  • The OS notification uses the copy's title.

Class B entry in DECISIONS.md. The protocol change is additive: one new error code and one new retract reason.

Tests

  • core/tests/test_thinking_cap_retry.py, 45 tests. With the retry switched off, 10 of them fail. Each fix that came out of review has its own test, and each of those fails without its fix.
  • UI: notifications.test.ts, chat-retract.test.ts, os-notify.test.ts, and the new chat-pane-retry.test.ts, which mounts the real ChatPane. The new behaviour tests fail on the old code.
  • Core:
    • 8450 passed and 1 failed, test_worktree_daemon::test_a_failed_checkout_moves_the_session_back_to_its_folder. It passes alone three times out of three; it's a timing flake under load. The baseline run on fe2810c3 had a different flake, test_workflow::test_stop_reaches_a_childs_shell, which also passes alone.
    • ruff, ruff format and mypy --strict are clean.
  • UI: vitest 2737 passed, svelte-check and tsc clean, vite build succeeds, Playwright walk 26 passed, and the protocol fixtures are unchanged.
  • Live against OpenRouter (deepseek/deepseek-v4.1-flash-1017), with this branch's daemon in a throwaway data dir and a tiny max_output_tokens. Three runs:
    • Cap 512, harder prompt: the first reply spent the cap thinking and was withdrawn with output_cap_thinking. The retry at 2,048 spent that too, and the turn failed with output_cap_thinking and the new text naming presets.<preset>.brain.
    • Cap 128, easier prompt: the first reply was withdrawn the same way. The retry at 512 answered correctly.
    • Cap 512, easier prompt: answered the first time, with no retry.

Not in this PR

These hit the same failure and are listed as follow-ups in the story: the delegate child, which still says "Worker finished with no summary.", the progress check, the validator review, distill, the Settings model probe, OpenRouter reasoning params, usage reasoning_tokens, and machine-aware copy for bridged sessions.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HUthoCrBcxKu4GVj429dwD

captcrock and others added 2 commits November 4, 2026 18:52
…more, and says so

A reasoning model on a tier with max_output_tokens 4096 could spend the
whole cap thinking: the stream ended finish_reason "length" with no text
and no tool call. The turn failed as empty_completion with advice to start
a new session (the context was a tenth full), nothing asked again, and the
window also showed an "unrecognised error" toast for the code.

Daemon:
- stream_reply withdraws a reply that thought to the cap (assistant_retract
  reason output_cap_thinking) and asks once more, on its own counter, with
  max_tokens = min(4x cap, max(32768, cap), room left in the window), only
  when that is at least twice the cap. provider_retry.max_retries 0 turns
  it off. Later attempts of the call keep the raised cap.
- A refusal of the raised request before any output (402 balance
  reservation, 400, overflow) keeps the first result so the real cause is
  reported; a refusal of a later stall or guard retry is its own error.
- A turn that still has no answer fails as output_cap_thinking and names
  the config entry whose max_output_tokens applied (a backup's fallbacks[n]
  or the computer-use worker's preset included). A non-cap empty reply
  stays empty_completion with accurate copy. The new-chat advice appears
  only at the compaction budget. Stop as an empty reply lands is a stop.
- StreamResult carries the max_tokens it sent, so a cut at the raised cap
  names that figure; the empty-reply log names model, cap, finish reason,
  prompt estimate and code.

UI: copy for both codes (banner, cleared once a later turn answers; toast),
Activity title and preview for the new retract reason, a reason-aware
Working line, and the OS notification uses the copy's title.

Class B (DECISIONS.md): new error code and retract reason, the raised-cap
retry and its refusal rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HUthoCrBcxKu4GVj429dwD
A reasoning model with a 128-token cap was withdrawn and answered on the
512 retry; with 512 it failed after the 2,048 retry with the line naming
the setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HUthoCrBcxKu4GVj429dwD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant