Repository navigation
fix(bedrock): avoid unsupported inference profile JSON prefill - #4095
Conversation
|
Hi @Taranum01, thanks a lot for these Bedrock fixes! We noticed that #4094 and #4095 both update the Bedrock plugin and bump Could you please consolidate the changes from #4094 into this PR (or the other way around) so both fixes can ship together as a single release? We've closed #4094 in the meantime to keep the release queue clean — happy to reopen it if anything was missed. Thanks again for your contributions! 🙏 |
…e Bedrock fixes per maintainer request (langgenius#4094 closed)
… manifest to 0.0.92 (0.0.91 taken by langgenius#4097)
|
Hi @Taranum01, thanks for fixing the unsupported inference-profile JSON prefill on Bedrock! Heads-up: #3794 also targets Version note: the consolidated dependency refresh (#4106–#4111) bumped Thanks again! 🙏 |
main is now 0.0.92 (langgenius#4106, langgenius#4095), so bump to the next unused version. langgenius#4095 added a test that expected no Claude 5 response field paths for a Haiku 5.5 profile ID. With Haiku 5.5 registered as a Claude 5 model, profile IDs such as global.anthropic.claude-haiku-5-5 now request /stop_details like the other Claude 5 models; the application inference profile ARN case is unchanged. Also note in the README that Haiku 5.5 is now a predefined model.
… Claude 5 (#4112) * feat(bedrock): add thinking display option for Claude 5 models Add a `thinking_display` parameter (omitted / summarized) to the "Anthropic Claude 5" family. Claude 5 adaptive thinking defaults to `omitted`, so nothing is streamed while the model thinks. With `summarized`, reasoning summaries are streamed, which keeps the connection from going idle during long thinking. - Send `thinking.display = "summarized"` when selected; requests with `omitted` or no value are unchanged. - Allow `summarized` without Effort: send adaptive thinking only, with no output_config, and still drop sampling params. - Add tests, README notes, and bump the plugin version to 0.0.92. * feat(bedrock): add Claude Haiku 5.5 Add Claude Haiku 5.5 (anthropic.claude-haiku-5-5) to the "Anthropic Claude 5" model family. - model_ids: register 'Haiku 5.5' and its geo profile prefixes (us/eu/au/global; the jp. profile is not wired up yet, as for Opus 5.5) - anthropic-claude-5.yaml: add the model option and update the cross-region help text - llm.py: map 'Haiku 5.5' to the new pricing file - model_configurations/claude-5-5-haiku.yaml: 1M context, Global rates of $0.10 / $0.50 per 1M tokens (prompts up to 100K tokens) - cache_config: enable explicit prompt caching (min 512 tokens, 4 checkpoints) - README (en / zh_Hans): document the model, profiles and pricing - tests: add Haiku 5.5 to the Claude 5 model id, cache and llm param tests, plus regression tests for custom (inference profile) models - bump plugin version to 0.0.92 Because Haiku 5.5 profile IDs are now recognised as Claude 5 profiles, custom models using them get the Claude 5 parameter rules (no temperature/top_p/top_k) instead of the legacy Claude ones. Fixes #4044 * chore(bedrock): bump to 0.0.93 after merging main main is now 0.0.92 (#4106, #4095), so bump to the next unused version. #4095 added a test that expected no Claude 5 response field paths for a Haiku 5.5 profile ID. With Haiku 5.5 registered as a Claude 5 model, profile IDs such as global.anthropic.claude-haiku-5-5 now request /stop_details like the other Claude 5 models; the application inference profile ARN case is unchanged. Also note in the README that Haiku 5.5 is now a predefined model. --------- Co-authored-by: Y.A-SLC <217139054+Arai-Y-SLC@users.noreply.github.com>
Summary
Keep JSON formatting instructions in the system message without adding an unsupported assistant prefill for Claude 4.6+ inference profiles. Route the SDK code-block wrapper through the underlying-model capability check. Preserve prefill for older Claude profiles and keep native JSON-schema output precedence. Do not duplicate the first user message as system instructions.
Fixes #2824
Release Notes
Fix JSON response formatting for Claude inference profiles that reject assistant-message prefill.
Testing
python -m pytest -q tests: 283 passed.Change Type
Version