[WH-2099]

Summary

OpenAI-compatible calls could fail while estimating a valid tool schema, accept an unfinished stream with an empty finish reason, or send a tool reply that the daemon could not decode.
Count complete JSON tool schemas, require a non-empty finish reason or [DONE], and preserve an already received finish reason across later empty frames.
Parse response tools before validating missing content and constructing the message, so a raw list of unusable tools cannot turn a malformed response into an empty success.
Represent absent or null text in accepted non-streaming chat replies as an empty string.
Remove the obsolete schema field and enum walkers.

The shared GPT-2 estimator now tokenizes long inputs in bounded chunks instead of returning the character count at 100,000 characters.
This removes abrupt jumps in tool budgets, missing-usage estimates, and local price statistics.
Embedding truncation measures candidate prefixes against the configured token budget instead of assuming uniform token density across the input.
It retains fitting character and token estimates before refining the prefix, so token merges cannot discard an already verified candidate.

Refs #387.
Follow-up to #410 and #411.

Compatibility

This targets the unreleased SDK 0.11.0.
Public method signatures and the documented response-handler and token-count override signatures are preserved.
Local token and price estimates can change because tool schemas are counted in full and long inputs are tokenized.
Each tokenizer call processes at most 100,000 characters, using the existing direct or gevent thread execution path.
Counts across chunk boundaries remain approximate; inputs below that chunk size retain their existing GPT-2 counts.
Reported usage, including zero, still takes precedence.
Embedding inputs are shortened to a measured prefix within the estimated context budget, while retaining provider token-count overrides and Unicode character boundaries.
Non-streaming chat replies carry an empty text string for absent or null text on the daemon wire.
Missing content requires at least one parsed tool call; explicitly empty or null content retains its accepted behavior.
README documents these behaviors.

Validation

• Regression cases reproduced the original three failures, unusable tool replies, large-input estimate jumps, and embedding prefix overflows before their respective fixes.
• just check: passed.
• just test: 1803 passed.
• just build: passed.
• HTTP tests cover valid schema shapes, nested fields, the former 100,000-character boundary, missing and zero usage, Decimal prices, stream completion, modern/legacy tools, and invalid tools mixed with valid calls.
• Real-tokenizer embedding tests inspect the transmitted prefix for mixed token densities, Unicode, empty input, context boundaries, and non-monotonic token merges.
  Inputs already within the budget are retained in full.
  Existing override and batching tests remain covered.
• Direct and gevent tokenizer paths are tested with long text and literal special-token spellings.
• Independent response review exercised 506 cases against the original base, initial fix, and revised parser; valid inputs and response cleanup remained compatible.
• Independent integration validation exercised 106 HTTP scenarios through the actual SDK response writer, daemon 0.6.13 parser (626682bc), Dify (a79e2116), and Graphon (7b5ce132).
  Normal results preserved tool parameters, usage, and prices; replies with no content field and no parsed tool call were rejected before a success chunk.
• Independent embedding review compared actual HTTP payloads across revisions and exercised 1000 non-monotonic count overrides plus Unicode and large-input cases.
  Regression tests detect removal of either initial fitting candidate, loss of an already fitting full input, and reintroduction of truncation by average density.

Pull Request Checklist

Compatibility Check

• Backward compatibility impact has been checked.
• Forward compatibility impact has been checked.
• Compatibility impact and the target SDK version are documented in this description and README.
• README identifies the target as SDK 0.11.0 (unreleased).
• Breaking-change discussion with maintainers, if required; this follow-up preserves exported and documented provider override signatures.

Available Checks

• just check has passed.
• just test has passed.
• just build has passed.
• Relevant documentation has been updated.

Validation used local HTTP fixtures and the daemon parser, without a live model service or a full Dify deployment.
SDK release and verification in the original reporter's environment remain open under #387.

[Copilot]

🟢 Approval recommended

No active review issues are recorded for the response-handling and token-estimation changes.

No open issues 🎉

What changed in this PR

This follow-up to #410 and #411 strengthens OpenAI-compatible response handling and token estimates for SDK 0.11.0, while preserving public and documented provider override signatures.

Changes:

• Require non-empty stream completion markers, validate parsed tool calls, and normalize absent or null chat text for daemon compatibility.
• Count complete tool schemas, tokenize long inputs in bounded chunks, and truncate embedding inputs using measured prefix budgets.
• Add regression coverage and document estimate changes and response requirements.

File	Description
tests/​interfaces/​model/​openai_compatible/​test_model_configuration.py	Tests long-input estimates and nested tool-schema counting.
tests/​interfaces/​model/​openai_compatible/​test_llm.py	Tests long special-token text through direct and threaded tokenization.
tests/​interfaces/​model/​openai_compatible/​test_http_llm.py	Covers response validation, wire serialization, schema estimates, and pricing.
tests/​interfaces/​model/​openai_compatible/​test_http_embedding_rerank.py	Tests embedding prefix budgets, Unicode, and non-monotonic token merges.
src/​dify_plugin/​interfaces/​model/​openai_compatible/​text_embedding.py	Measures candidate prefixes and retains verified fitting estimates.
src/​dify_plugin/​interfaces/​model/​openai_compatible/​llm.py	Tightens response validation and replaces schema walkers with complete JSON counting.
src/​dify_plugin/​interfaces/​model/​ai_model.py	Tokenizes long inputs in chunks of at most 100,000 characters.
README.md	Documents updated estimation, embedding truncation, and response behavior.

_{Review effort: Balanced}