Skip to content

fix(models)!: align provider usage with SDK 0.11 - #4156

Draft
WH-2099 wants to merge 1 commit into
mainfrom
fix/model-usage-fallback
Draft

WH-2099 wants to merge 1 commit into
mainfrom
fix/model-usage-fallback

Conversation

@WH-2099

@WH-2099 WH-2099 commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

Companion to langgenius/dify-plugin-sdks#411.

SDK 0.11 passes tool definitions through response hooks so missing usage can include the actual request and complete response.
Update the overrides in DeepSeek, GPUStack, LiteLLM, Moonshot, OpenAI API Compatible, OpenRouter, StepFun, Tokener, and Yi.
Use the shared usage calculator in custom response handlers, preserve explicit tool choices, and keep tool definitions available for estimation.
Yi now implements the SDK text counter signature and accepts literal special-token spellings.
Tokener adapts byte streams through the SDK parser, closes the underlying response, and supplies complete tool calls before usage is calculated.
Update the affected HTTP fixtures and assert complete tool arguments directly instead of deriving expected values from the previous SDK implementation.

Release dependency — 2026-11-02

langgenius/dify-plugin-sdks#411 merged on October 31.
Keep this PR as a draft until SDK 0.11.0 is published through langgenius/dify-plugin-sdks#413 and the final plugin artifacts are verified.
Current GitHub checks pass; the nine projects still pin review commit c378d255ea6155d5dcdd91f31ded9bd24ad9d446.
The additional langgenius/dify-plugin-sdks#412 response/schema fixes remain under review.

This PR is assigned to Dify 1.19.0 through a concrete dependency in #4034.
That candidate upgrades OpenAI API Compatible to SDK 0.11 while its non-stream response override still lacks the new tools argument; adopting the final SDK without this migration raises TypeError.
Reconcile the overlapping provider code and lockfiles before publishing #4034's complete-call rollout for langgenius/dify#44502.
The other providers can retain their own publication schedule once the required overlapping migration is delivered; a future split can narrow this milestone to that prerequisite.

Before Marketplace publication, replace Git dependencies with the released SDK and refresh the lockfiles.
Each affected plugin's top-level manifest version is incremented in the candidate and must be reconciled with current published versions during final packaging.

Release Notes

Correct estimated token usage when a model service omits usage counters.
Include the complete conversation and tool definitions and results, retain reported zero values, and support replies containing only tool calls.
Preserve explicit tool choices and reject incomplete or malformed Tokener streams.

Change Type

  • LLM plugin

Validation

Fresh Python 3.12 environments were created outside the plugin directories using each plugin's frozen lockfile and pytest.
All selected tests pass against the pinned SDK build:

Plugin Tests passed
DeepSeek 90
GPUStack 6
LiteLLM 22
Moonshot 25
OpenAI API Compatible 29
OpenRouter 33
StepFun 12
Tokener 72
Yi 31
Total 320

Tokener retains three existing expected failures for ambiguous tool fragments.
The targeted suites cover response hooks, token estimates, missing and partial usage, tool calls, stream termination, tool-choice forwarding, text blocks, and provider request behavior.
Ruff correctness checks on the changed Python files, dependency/lock consistency checks, and git diff --check pass.
The SDK commit also passes its Python 3.12/3.13 CI suites and coverage gates.

No live model inference, Marketplace publication, or installed Dify deployment verification was performed.

Code references for the SDK-hook dependency: SDK call with tools and the #4034 candidate override.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant