[shirumesu]

Summary

Qwen3.5 can prefill the opening <think> tag in its prompt, so generated content may contain reasoning followed by </think> without an opening tag.

References:

• Qwen chat template
• vLLM Qwen3 reasoning parser

Restore the missing opening tag so Question Classifier can remove the complete reasoning block before parsing JSON.

Release Notes

• Fix Question Classifier JSON parsing failures when an upstream chat template prefills the opening think tag.
• Upgrade version to 0.0.69

Change Type

• Documentation / non-plugin change
• Non-LLM plugin (tools, extensions, datasource, etc.)
• LLM plugin

Screenshots / Videos

Before	After
Question Classifier fails with an Extra data JSON parsing error when thinking is enabled.	The same classifier successfully parses the response with thinking enabled.

LLM Plugin Checklist

Areas affected by this change (check all that apply)

• Message flow (system messages, user ↔ assistant turn-taking)
• Tool interaction flow (multi-round usage, Agent App and Agent Node)
• Multimodal input (images, PDFs, audio, video, etc.)
• Multimodal output (images, audio, video, etc.)
• Structured output (JSON, XML, etc.)
• Token consumption metrics
• Other LLM functionality (reasoning, grounding, prompt caching, etc.)
• New models / model parameter fixes

Version

• Bumped top-level version in manifest.yaml (not the one under meta)
• dify_plugin>=0.3.0,<0.11.0 is declared in pyproject.toml and locked in uv.lock (or kept in requirements.txt for legacy plugins without uv.lock) — SDK docs

Testing

• Local deployment — Dify version: dify-ee 3.9.14 LTS
• SaaS (cloud.dify.ai)

[Copilot]

🟡 Changes recommended

The streaming path misses Qwen’s default-thinking case and buffers reasoning instead of streaming it incrementally.

3 open findings

• Normalize streams when thinking is implicitly enabled · New
• Stream prefixed content without buffering or quadratic concatenation · New
• Add coverage for stream transformer edge cases · New

What changed in this PR

Restores missing <think> opening tags in OpenAI-compatible responses and updates plugin dependencies/versioning.

Changes:

• Normalizes non-streaming and streaming reasoning output.
• Constrains the SDK dependency below 0.11.0.
• Bumps the plugin version to 0.0.69.

File	Description
models/​openai_api_compatible/​models/​llm/​llm.py	Adds reasoning-tag normalization.
models/​openai_api_compatible/​pyproject.toml	Adds the SDK upper bound.
models/​openai_api_compatible/​uv.lock	Locks the updated constraint.
models/​openai_api_compatible/​manifest.yaml	Bumps the plugin version.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The new response-normalization branch needs a regression test covering restoration and filtering.

1 open finding

• Add regression test for missing opening think tag handling · New

3 resolved since last review

• Stream prefixed content without buffering or quadratic concatenation
• Normalize streams when thinking is implicitly enabled
• Add coverage for stream transformer edge cases

🧠 Review effort: Balanced

[Copilot]

🟢 Approval recommended

The focused implementation matches the documented Qwen3.5 behavior and includes appropriate regression coverage.

1 open finding

• Add regression test for missing opening think tag handling

🧠 Review effort: Balanced

[Copilot]

🟡 Changes recommended

Unconditional tag restoration can corrupt valid JSON answers containing a literal closing think tag.

2 open issues

• Avoid corrupting ordinary JSON answers during reasoning restoration · New
• Add regression test for missing opening think tag handling

_{Review effort: Balanced}

[crazywoola]

Hi @shirumesu, thanks for fixing the prefilled think handling in openai_api_compatible!

Heads-up about models/openai_api_compatible/manifest.yaml: the consolidated dependency refresh (#4106–#4111) bumped it to 0.0.69 on main, and several PRs are currently open for this plugin from different authors (#3795, #3869, #4066, #4068, #4079). Please rebase on latest main and bump the version to the next unused one so the releases don't collide. If there's a related issue for this fix, linking it in the description would help too.

Thanks again! 🙏

[Copilot]

🟡 Changes recommended

The restoration heuristics can corrupt valid answers and fail to normalize streams with empty reasoning.

4 open issues

• Do not treat unmatched closing tags in prose as reasoning · New
• Recognize split closing tags in prefilled empty reasoning · New
• Avoid restoring reasoning for answer-only streams · New
• Add regression test for missing opening think tag handling

1 resolved since last review

• Avoid corrupting ordinary JSON answers during reasoning restoration

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Streaming restoration can prepend an unmatched reasoning tag to a valid content-only answer when thinking is enabled.

3 open issues

• Avoid restoring reasoning for answer-only streams
• Do not treat unmatched closing tags in prose as reasoning
• Add regression test for missing opening think tag handling

1 previously missed issue

In code that hasn't changed since last review

Avoid opening think tags without confirmed server-side thinking

models/​openai_api_compatible/​models/​llm/​llm.py:773

Enabling thinking locally does not guarantee visible reasoning, especially in strict compatibility mode, which does not send the thinking switch to the server. A content-only stream such as {"class":"x"} becomes <think>\n{"class":"x"} without a closing tag, incorrectly marking the answer as unfinished reasoning. Require an explicit prefilled-template capability or confirm an unmatched closing delimiter before inserting the opening tag, and add a content-only streaming regression case with enable_thinking=True.

1 resolved since last review

• Recognize split closing tags in prefilled empty reasoning

_{Review effort: Balanced}

[shirumesu]

Thanks @crazywoola! Rebased onto the latest main and bumped the version to 0.0.72.
Related issue: None. This came up in real-world usage; happy to open an issue with full details if that would help.

Summary of changes since the last review:

• Literal in normal answers: non-streaming restoration is now skipped when the whole response is valid JSON, so {"tag":"</think>"} is preserved in every configuration. I didn't gate it on enable_thinking, because in strict mode (the default) the flag is never sent, yet Qwen3.5 still prefills <think>, so gating would leave the all-defaults case broken.
• Streaming with the flag unset: streaming has to decide on the first chunk, before any </think> is visible. With the flag unset, the server default depends on the model (Qwen3.5 thinks by default, hybrid models like DeepSeek-V3.1 don't), so always restoring would wrap the latter's entire answer in an unclosed <think>. Streaming therefore restores only when thinking is explicitly enabled, which matches vLLM's Qwen3ReasoningParser.
• Tests: added the requested reasoning</think>\n{"class":"x"} case, plus all-defaults stripping, preserving a literal </think> across 3 configs, streaming restoration (including a tag split across chunks), and no false restoration in streaming. All 104 tests pass.

Known limitation: streaming with the flag unset (or when the server thinks but the plugin assumes it doesn't) is not covered, since fixing it would require buffering the full output and losing streaming for every default-config model. The recommended setup is still enabling a server-side reasoning parser

[Copilot]

🟡 Changes recommended

The thinking-tag heuristics can misclassify normal content as reasoning, deleting valid replies or marking content-only streams as unfinished reasoning.

2 open issues

• Avoid prepending think tag to content-only streams · New
• Avoid restoring reasoning for answer-only streams

2 resolved since last review

• Do not treat unmatched closing tags in prose as reasoning
• Add regression test for missing opening think tag handling

_{Review effort: Balanced}