Repository navigation
Provider round-trip state (opaque_body) is dropped on assistant prompt messages and LLM results #260
Description
Delivery status — 2026-11-02
Implementation is complete through merged #261; this issue remains closed.
This dependency is tracked for the Dify 1.19.0 Agent replay rollout in langgenius/dify#41092 / #44502.
Refresh the host candidate and record the selected production Graphon dependency before rollout.
Background
Cross-ref: langgenius/dify#41092 — Claude extended thinking in Dify Agents fails with HTTP 400 (thinking or redacted_thinking blocks in the latest assistant message cannot be modified) because signed/encrypted thinking blocks are lost between LLM tool-use rounds.
The plugin SDK (dify_plugin >= 0.10.0) already defines opaque_body: JsonValue | None on AssistantPromptMessage (and on PromptMessageContent), and the plugin daemon passes the field through to Dify core. The official moonshot plugin already round-trips provider reasoning state through opaque_body in production.
Problem
Graphon's model runtime entities do not declare opaque_body, so provider state is silently dropped in two places:
- Entities —
AssistantPromptMessage(andPromptMessageContent) insrc/graphon/model_runtime/entities/message_entities.pyhave noopaque_bodyfield. The plugin daemon → core boundary deserializes plugin LLM results intoLLMResultChunk/LLMResult(whosemessageis typed asAssistantPromptMessage); Pydantic validation silently discards the field. - Result accumulators —
_LLMChunkAccumulatorand_StreamingInvokeAccumulatorinsrc/graphon/model_runtime/model_providers/base/large_language_model.pyrebuild a freshAssistantPromptMessagefrom content + tool calls only, dropping the field even when chunks carry it (affects non-stream normalization of stream-only models and callback bookkeeping results).
Expected
opaque_body: JsonValue | None = NoneonAssistantPromptMessageandPromptMessageContent, mirroring the plugin SDK field name/type/default exactly (full parity, including the content-block level used by providers such as Geminithought_signature).- Both accumulators preserve the provider state when assembling
LLMResult(complete snapshot semantics: last non-Nonechunk wins). - Backward compatible: payloads without the field still validate (default
None).
Prior art
- Closed langgenius/dify#31566 attempted the core-side half before the model runtime tree was extracted into this repo.
- Plugin-side contract: langgenius/dify-official-plugins#3715.
Activity
dosubot commentedon Aug 22, 2026
@hpiclaranet I've verified the current codebase and can confirm all four gaps you described:
AssistantPromptMessagehas noopaque_bodyfield — the class only declaresroleandtool_calls[1]. Anyopaque_bodyarriving from the plugin daemon is silently discarded during Pydantic validation. (PromptMessageContentsimilarly lacks the field [2].)_LLMChunkAccumulator.to_result()rebuildsAssistantPromptMessagefromcontent+tool_callsonly — no provider-state slot exists [3]._StreamingInvokeAccumulator.to_result()does the same —AssistantPromptMessage(content=self.message_content)with noopaque_body[4].SlimLLM's_CollectedLLMResultalso lacks the field [5], and theAssistantPromptMessagebuilt in_collect_llm_resultomits it [6].I found no open or merged PR in this repo that addresses
opaque_body— the recent merged PRs focus on prompt construction refactoring [7], runtime simplification, and structured output, none of which touch provider round-trip state. This issue is the right place to track it.The minimal fix would touch four locations:
opaque_body: JsonValue | None = NonetoAssistantPromptMessage(andPromptMessageContent) inmessage_entities.py.Noneopaque_bodyin_LLMChunkAccumulatorand thread it throughto_result()._StreamingInvokeAccumulator.opaque_bodyto_CollectedLLMResultand accumulate it in_accumulate_llm_chunk.All four changes are backward-compatible (default
None), self-contained within graphon, and would unblock the plugin-side (dify-official-plugins#3715) and core-side (dify#31566) counterparts.To reply, just mention @dosu.
Share context across your team and agents. Try Dosu.