Skip to content

fix(anthropic): stream tag duplication, 529 mapping, image fetch hardening - #3757

Closed
hpiclaranet wants to merge 1 commit into
langgenius:mainfrom
hpiclaranet:fix/anthropic-stream-errors-image-fetch
Closed

hpiclaranet wants to merge 1 commit into
langgenius:mainfrom
hpiclaranet:fix/anthropic-stream-errors-image-fetch

Conversation

@hpiclaranet

@hpiclaranet hpiclaranet commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Supersedes #3721 (closed by the maintainer with "Please resolve the conflicts"). Rebased onto current main (9e1fb9d5), squashed to a single commit; version 0.3.30 (main carries 0.3.29 via #3811). The llm.py changes merge cleanly on top of the Fable 5.1/SDK modernization (#3811) and earlier structured-output (#3720) / request-metadata (#3717) additions. Full anthropic suite: 94 passed, 15 skipped.

Three reliability fixes in the Anthropic provider, each confirmed by code-path analysis of the streaming state machine and the installed anthropic SDK (0.120.0 at analysis time; the plugin now locks 1.3.0 via #3811 and the full suite was re-verified on 1.3.0):

  1. Duplicate closing think tag in streams — when a thinking block was followed directly by a tool_use block, the input_json_delta events matched none of the block-type branches, so current_block_type stayed "thinking". The block transition had already emitted the closing tag once, and the MessageStopEvent handler emitted it a second time, corrupting the thought text Dify stores. Tool-use deltas now mark the block as "tool_use".
  2. HTTP 529 misclassified as a client error — anthropic.OverloadedError is a direct subclass of APIStatusError (not InternalServerError), so it fell through the APIError catch-all into InvokeBadRequestError. It is now mapped to InvokeServerUnavailableError (server-side capacity, retryable). The mapping entry is ordered before the catch-all, which is what the SDK's first-match lookup requires.
  3. Unbounded image URL fetch — requests.get(data) had no timeout (a hung URL could block the worker indefinitely), no size cap, accepted any URL scheme, and followed redirects. It now uses a (5s connect, 15s read) timeout, streamed reads bounded to a 10 MB cap (early rejection via Content-Length or during iter_content, so a hostile server cannot push unbounded bytes into the process), http(s)-only URLs (surrounding whitespace tolerated), and no redirect following with explicit 3xx rejection (a 3xx with an image-like body is rejected, not consumed).

Release Notes

  • Fixed duplicated "thinking" end markers in streamed agent responses when Claude calls a tool directly after thinking
  • Fixed Anthropic 529 (overloaded) errors being shown as user errors; they are now treated as transient server issues
  • Hardened image URL fetching: bounded timeouts, 10 MB size limit, http(s) only, redirects rejected

Change Type

  • Documentation / non-plugin change
  • Non-LLM plugin (tools, extensions, datasource, etc.)
  • LLM plugin

Screenshots / Videos

Not applicable — streaming state machine, error mapping, and fetch hardening, covered by unit tests.

LLM Plugin Checklist

Areas affected by this change (check all that apply)
  • Message flow (system messages, user ↔ assistant turn-taking)
  • Tool interaction flow (multi-round usage, Agent App and Agent Node)
  • Multimodal input (images, PDFs, audio, video, etc.)
  • Multimodal output (images, audio, video, etc.)
  • Structured output (JSON, XML, etc.)
  • Token consumption metrics
  • Other LLM functionality (reasoning, grounding, prompt caching, etc.)
  • New models / model parameter fixes

Version

Testing

  • Local deployment — clean uv venv aligned with manifest.yaml, pyproject.toml, and uv.lock: full anthropic suite 94 passed, 15 skipped on the rebased head (d639254f, main 9e1fb9d5) — 18 new tests built from real SDK stream event objects. The bug-target tests fail against the pre-fix code (verified by checking out the base commit): duplicate-tag, 529 mapping, timeout/no-redirect/stream kwargs, whitespace stripping, 3xx-with-valid-body, Content-Length cap, oversized body. ruff check shows zero new findings vs baseline.
  • SaaS (cloud.dify.ai)

Known limitations (pre-existing, out of scope here):

  • The fetch hardening does not restrict private-network hostnames (multi-tenant deployments should apply network-level egress controls); no upstream allowlist exists in this plugin's credential schema.
  • 503/504 mapping: ServiceUnavailableError/DeadlineExceededError are not explicitly mapped in this PR (pre-existing gap; generic 5xx map to InternalServerError, which is already mapped). With the SDK now at 1.3.0 (feat(models): add Claude Fable 5.1 and Gemini 3.8 Flash support #3811) those classes exist, so a 503/504 would be surfaced as a client error until explicitly mapped — out of scope here.
  • The redacted/omitted-thinking delta branch in the stream handler does not match the SDK event model (verified against both 0.120 and 1.3.0 — no redacted delta variant; pre-existing; encrypted thinking is intentionally not rendered).

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Aug 26, 2026
…ening

- Tool-use deltas now mark the block as "tool_use", fixing duplicate
  closing think tags when a thinking block is followed directly by a
  tool_use block in streams.
- Map anthropic.OverloadedError (HTTP 529) to InvokeServerUnavailableError
  (retryable), ordered before the generic APIError catch-all.
- Harden image URL fetching: (5s, 15s) timeout, 10 MB streamed cap with
  early Content-Length rejection, http(s)-only schemes, redirects rejected.

Rebased onto main (9e1fb9d); version 0.3.30 (main carries 0.3.29 via langgenius#3811).

@Mairuis Mairuis left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rebase onto current main and choose a fresh Anthropic manifest version: 0.3.30 is already on main, and #3788/#3725 now propose 0.3.31/0.3.32. The new image fetch path intentionally rejects all redirects and caps downloads at 10 MB; please call out those user-visible restrictions in Release Notes and include image-input verification for URL-based images that previously redirected. The thinking-tag and 529 fixes should remain covered after rebasing.

@hpiclaranet

Copy link
Copy Markdown
Contributor Author

Superseded by #3907 (rebase onto current main 4a9e83b, version 0.3.33, Release Notes with the image-fetch restrictions called out, URL-based image input verification tests added). Thanks for the review comments — all addressed there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants