Skip to content

fix(agent)!: preserve model replay with complete tool calls - #4034

Draft
WH-2099 wants to merge 6 commits into
mainfrom
fix/anthropic-agent-replay
Draft

WH-2099 wants to merge 6 commits into
mainfrom
fix/anthropic-agent-replay

Conversation

@WH-2099

@WH-2099 WH-2099 commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Closes #4032.

Agent tool continuations lose provider replay data when they rebuild assistant messages from visible text, and streamed tool fragments can be mistaken for executable calls.
Preserve complete assistant messages through Function Calling, ReAct, and Self Refine, including Anthropic signed/redacted thinking and provider/model-scoped model_history.
Adapt the providers below to the SDK complete-call contract: assemble native fragments inside each provider, emit each finished call once, preserve raw arguments and native IDs, and generate IDs only when the provider omits them.
Shared SDK collectors reject empty or duplicate IDs before appending a batch.

Function Calling now extracts calls after collection and adds the final usage snapshot once per model request.
Repeated usage updates replace earlier snapshots, including a final zero value.
Azure AI Studio, ZhipuAI, OCI, Tongyi, Volcengine, Gitee, and Vertex Claude reject unfinished streams, failed completion states, empty names, and invalid argument objects before emitting calls.
Vertex Claude uses the same completed stream for blocking and streaming requests, preserving native thinking, signatures, redacted blocks, citations, block order, and raw tool arguments in response-owned replay data.
Tool continuations preserve the requested thinking configuration, and HTTP responses close on both success and failure.
Agent image inputs, tool schemas, configured output/thinking budgets, and prompt-cache behavior remain covered by the replay regressions.

Release Notes

  • Preserve signed and redacted Claude thinking through tool continuations and stored history in Anthropic and Vertex AI.
  • Preserve complete assistant messages, image history, tool IDs, content metadata, and empty tool results in Agent strategies.
  • Assemble streamed tool calls before making them available to Agents, retaining the original argument text.
  • Report incomplete or malformed tool-call responses before tool execution.
  • Use the final token-usage snapshot for each model request when reporting Agent totals.

Change Type

  • LLM plugin
  • Non-LLM plugin (Agent strategies)

LLM Plugin Checklist

  • Message flow
  • Tool interaction flow
  • Multimodal input
  • Token consumption metrics
  • Reasoning, prompt caching, and output budgets

Versions and validation

All 19 projects declare SDK >=0.11,<0.12 and lock 0fa060a11fd774fe8ed70c915d7881e20a765b64 from SDK #390.
The Python runtime is 3.12, including the updated PerfxCloud and Upstage manifests.
The following suites passed in separate environments installed from each project's frozen lockfile, without a local SDK path override.

Plugin Manifest version Passed
Anthropic 0.3.40 169
Ollama 1.0.4 33
Vertex AI 0.0.71 187
OpenAI API Compatible 0.0.74 122
Llama API 0.0.11 13
PerfxCloud 0.1.4 11
Upstage 0.0.14 11
Fireworks 0.0.16 39
Tongyi 0.2.27 88
Cohere 0.0.21 18
Volcengine MaaS 0.0.58 39
ChatGLM 0.0.10 2
LocalAI 0.0.11 2
Gitee AI 0.1.13 28
Azure AI Studio 0.0.22 20
ZhipuAI 0.0.37 33
OCI 0.0.20 31
Agent Strategies 0.0.52 108
Self Refine 0.0.6 13

Total: 967 passed, with 18 existing Anthropic live cases skipped because credentials were absent.
All 19 final plugin sources were packaged with the official CLI, with source hashes, manifests, locks, and exported SDK requirements verified against the archives.
Relevant lint checks and git diff --check passed; existing provider lint findings were compared against HEAD.

Independent acceptance used real installed vendor SSE parsers for 36 scenarios across Azure AI Studio, ZhipuAI, OCI, Tongyi, Volcengine, and Gitee.
All 30 invalid scenarios raised errors before emitting any tool call; six valid controls preserved raw arguments, IDs, and token totals.
Vertex Claude also passed real-client SSE tests and an independent public-invocation review for both modes, including fresh-instance replay and errors received after the completion event.
Eight additional controlled Vertex/OpenAI transport scenarios exercised Graphon, Dify backward invocation serialization, real SDK Session HTTP parsing, and Function Calling/Self Refine in streaming and blocking modes.
They preserved parallel calls, replay metadata, and final usage through the next model request.

Release gates

  • Replace review Git sources with the published SDK release.
  • Publish Graphon 86f1e11b199bb8ad9111ed49d42edea82a16b7fa and Dify #44502 for legacy Agents, workflow history, and Agent v2.
  • Release daemon #828.
  • Coordinate OpenAI/endpoint #4035 with the same SDK contract.
  • Set plugin minimum Dify versions and record the daemon image once the joint release is selected.
  • Run live-provider acceptance before production release.

Keep this PR in draft until these release gates are met.
Current manifest minimum Dify versions do not define compatibility for this joint upgrade.
Controlled HTTP responses validate transport and replay equality; real vendor signatures and production deployment remain unverified.

@WH-2099 WH-2099 changed the title fix(anthropic): preserve replay state across agent tool turns fix(agent): preserve Anthropic replay through tool loops and history Oct 21, 2026
Assemble provider tool calls before emitting them, preserve raw arguments and native IDs, reject unfinished streams, and use the final usage snapshot across Agent loops.

BREAKING CHANGE: updated providers require the SDK 0.11 complete-call contract and coordinated Dify, Graphon, and daemon releases.
@WH-2099 WH-2099 changed the title fix(agent): preserve Anthropic replay through tool loops and history fix(agent)!: preserve model replay with complete tool calls Oct 28, 2026
Merge current main, preserve native Claude content through the shared Vertex stream collector, close responses on errors, and reserve plugin versions after upstream releases.

BREAKING CHANGE: these provider and Agent releases require the SDK 0.11 complete-call contract and the coordinated Dify, Graphon, and daemon changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(anthropic): preserve signed replay state in agent tool turns

1 participant