Repository navigation
fix(agent): stop replaying reasoning_content to strict openai-compatible providers - #2922
Dani Akash (DaniAkash) wants to merge 2 commits into
Conversation
…ble providers A reasoning model behind an openai-compatible gateway (for example GLM served through Mistral) returns reasoning that the @ai-sdk/openai-compatible serializer re-sends as a top-level reasoning_content field on the next request. reasoning_content is response-only in the chat-completions schema, so strict gateways (Mistral, Cerebras, Fireworks) reject the replay with 422 and every multi-step agent run fails on its second call. stripReasoningParts already cleans the pre-run history, but the assistant turn the multi-step tool loop generates mid-run never passes through that path, so strip reasoning from the model messages inside prepareStep for providers that serialize through openai-compatible. Native Anthropic, OpenAI, Google, Azure and Bedrock providers carry signed or encrypted reasoning that must be replayed, so they are left untouched.
✅ Tests passed: 1409/1412Ran 8 of 16 suites (8 not affected by this change).
|
 [Medium impact] Filters reasoning from replayed messages for some LLM providers. The PR appears safe to merge, with a non-blocking gap in coverage of the two-step agent flow. Findings
Fix with agent prompt### Issue 1
packages/browseros-agent/apps/server/tests/agent/message-normalization.test.ts:228
The new tests call `stripReasoningFromModelMessages` and `serializesReasoningContent` separately, so they would still pass if `prepareStep` stopped applying the filter. That leaves the reported second-call failure unprotected.
Add a two-step test that returns reasoning plus a tool call on step one, then checks that step two sends the tool call without reasoning for an OpenAI-compatible provider. Also check that a native provider keeps its reasoning.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryRemoves assistant reasoning from replayed requests on every agent step for the six providers that use
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Messages for next step] --> B[Normalize messages]
B --> C{Uses compatible serializer?}
C -->|Yes| D[Remove assistant reasoning]
C -->|No| E[Keep reasoning]
D --> F[Compact if needed]
E --> F
F --> G[Next model request]
Reviews (1) · Last reviewed commit: "fix(agent): stop replaying reasoning_con..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Strips assistant reasoning parts from the model messages replayed on each step of a multi-step tool loop, for providers that go through the @ai-sdk/openai-compatible serializer. That serializer turns reasoning into a top-level reasoning_content request field, which strict gateways such as Mistral reject with 400/422. The new stripReasoningFromModelMessages removes reasoning parts and drops assistant turns that would be left empty, while keeping text and tool calls. Reasoning stays in the stored, client-displayed history. A new serializesReasoningContent check in the provider factory picks the affected providers (OpenAI-compatible, LM Studio, Ollama, Moonshot, Qwen Code, GitHub Copilot), and prepareStep in the agent applies the strip only for those. Native Anthropic, OpenAI, Google, Azure, Bedrock and OpenRouter are left alone, since they replay signed or encrypted reasoning that must stay intact.
No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 81 functions depend on the 58 functions this change touches.
Health — grade A; 4 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):
createChatRoutes()— 6 callers, 14 callees (high)createApiRoutes()— 3 callers, 20 callees (high)createHttpServer()— 2 callers, 6 callees (medium)createLanguageModel()— 4 callers, 3 callees (medium)
Verification — 81 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 65 function(s) in the blast radius were not formally verified this run
… vs native providers Add a two-step pipeline test that reproduces the reported second-call failure: step one returns reasoning plus a tool call, and step two must send the tool call without reasoning to an openai-compatible provider while a native provider keeps its reasoning for replay. The unit tests already cover stripReasoningFromModelMessages and serializesReasoningContent in isolation; this exercises them composed the way prepareStep applies them on every step.
This branch has not been deployed
1 participant
Summary
Fixes #2921. A reasoning model behind an OpenAI-compatible gateway (for example GLM served through
api.mistral.ai) fails on the second LLM call of every agent run with HTTP 422extra_forbiddenatmessages[N].assistant.reasoning_content.Root cause
The agent runs on the AI SDK. For a provider of type "OpenAI Compatible" the request is serialized by
@ai-sdk/openai-compatible, whose assistant-message serializer re-emits a reasoning part as a top-levelreasoning_contentfield. That field is response-only in the chat-completions schema, and strict gateways (Mistral, Cerebras, Fireworks) validate withextra="forbid"and reject it. Step one succeeds; the assistant turn it produces (reasoning plus tool call) is replayed on the next step, which 422s. The existingstripReasoningPartsonly cleans the pre-run history copy, so it never touches the turn the multi-step tool loop generates mid-run.Fix
Strip reasoning from the model messages inside
prepareStep, which runs on every step, for the providers that serialize through@ai-sdk/openai-compatible(OpenAI Compatible, LM Studio, Ollama, Moonshot, Qwen Code, GitHub Copilot). Reasoning stays in the durable, client-displayed history; only the replayed request copy is cleaned, and the model generates fresh reasoning each step. Native Anthropic, OpenAI, Google, Azure and Bedrock providers carry signed or encrypted reasoning that must be replayed intact, so they are deliberately left untouched, as is OpenRouter (its own provider).Tests
bun run lint,turbo typecheck --affectedandbun fallowpass; the touched agent unit suites are green.Out of scope
The issue also notes the reasoning-effort dropdown offers "Medium" for a GLM model that only accepts low, high and max. That is a separate model-catalog and UI concern, and the effort value is not forwarded for openai-compatible providers anyway, so it is left for a follow-up.