From 70511f36e44424e0cc98844c3f0325496b541805 Mon Sep 17 00:00:00 2001 From: GitHub Action Date: Fri, 9 Oct 2026 10:33:32 +0000 Subject: [PATCH 1/2] [Automated] Draft docs (agentgateway): llm: make Bedrock Converse honor tool_choice Signed-off-by: GitHub Action --- .../agw-docs/pages/agentgateway/llm/providers/bedrock.md | 8 ++++++++ .../docs/kubernetes/main/release-notes/release-notes.md | 2 ++ .../main/documentation/llm/api-types/messages.md | 2 ++ .../standalone/main/integrations/llm/providers/bedrock.md | 6 ++++++ .../docs/standalone/main/release-notes/release-notes.md | 2 ++ 5 files changed, 20 insertions(+) diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md index 7ab4b54b7..195a8f29a 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md @@ -277,6 +277,14 @@ Whether the preference changes the request format that a model accepts depends o > Requests to the Mantle endpoint are signed for the `bedrock-mantle` service rather than `bedrock`. If you scope an IAM policy by service name, grant both before you switch a route to Mantle. {{% /version %}} +{{% version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" %}} +## Tool choice + +When a chat request reaches Bedrock Runtime, the conversion sends the tool choice from the client request as the Bedrock Converse `toolChoice` field. OpenAI Chat Completions and Responses requests can use `tool_choice` values such as `auto`, `required`, or a named function. Anthropic Messages requests can use `tool_choice` values such as `auto`, `none`, `any`, or a named tool. + +The selected Bedrock model decides which tool-choice values are supported. If a model rejects forced tool use, the client receives the Bedrock error instead of a request that silently ignores the field. With extended thinking, the conversion keeps the `tool_choice` value from the client request. The `auto` value stays `auto`, and `none` or an omitted field is omitted. Forced choices go to Bedrock for the model to accept or reject. +{{% /version %}} + ## Prompt caching Prompt Caching is a performance, cost-optimization, and cost-reduction feature that allows the model to "remember" frequently used parts of your prompt, including long system instructions, reference documents, or tool definitions. This way, the model does not need to reprocess these parts every time you send a new prompt. diff --git a/content/docs/kubernetes/main/release-notes/release-notes.md b/content/docs/kubernetes/main/release-notes/release-notes.md index 63b458b31..6454f5278 100644 --- a/content/docs/kubernetes/main/release-notes/release-notes.md +++ b/content/docs/kubernetes/main/release-notes/release-notes.md @@ -186,6 +186,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `pre + **Traffic management** @@ -194,6 +195,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `pre **LLM** - Vertex AI catalog lookups resolve Anthropic model aliases such as `claude-sonnet-4-5-20250929`, `anthropic/claude-sonnet-4-5@20250929`, and `publishers/anthropic/models/claude-sonnet-4-5@20250929`. +- Bedrock Runtime requests now send client `tool_choice` settings as Converse `toolChoice`, so forced tool choices can take effect and models that reject them return Bedrock errors. **MCP** diff --git a/content/docs/standalone/main/documentation/llm/api-types/messages.md b/content/docs/standalone/main/documentation/llm/api-types/messages.md index 56311209e..5a738c38c 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/messages.md +++ b/content/docs/standalone/main/documentation/llm/api-types/messages.md @@ -148,6 +148,8 @@ Certain models, such as `gpt-5.3`, reject a Chat Completions request that sets b The Bedrock Converse conversion carries extended-thinking history in both directions, including encrypted reasoning. +The Bedrock Converse conversion preserves tool choice for Messages requests. The `auto` value becomes the Bedrock `toolChoice.auto` value, and `any` becomes `toolChoice.any`. A `tool` choice becomes `toolChoice.tool`. The `none` value or an omitted tool choice is omitted. Extended thinking does not rewrite the choice. A model that rejects forced tool use with thinking returns a provider error. + On the way out, a `thinking` block in the message history becomes a Bedrock `reasoningContent.reasoningText` block, with its signature. A `redacted_thinking` block becomes a Bedrock `reasoningContent.redactedContent` block, so that a client can replay encrypted reasoning that Bedrock returned on an earlier turn. Document, search-result, and server-tool content blocks are dropped. On the way back, Bedrock reasoning text becomes a `thinking` block. In a buffered reply, Bedrock encrypted reasoning becomes a `redacted_thinking` block that keeps the encrypted payload for the next turn. A streamed reply does not keep the payload. The encrypted reasoning arrives as a `thinking` block with the text `[REDACTED]`, which cannot be replayed. For the same behavior on other Bedrock endpoints, see [Encrypted reasoning]({{< link-hextra path="/integrations/llm/providers/bedrock/#encrypted-reasoning" >}}). diff --git a/content/docs/standalone/main/integrations/llm/providers/bedrock.md b/content/docs/standalone/main/integrations/llm/providers/bedrock.md index 7039fd006..119a26d2e 100644 --- a/content/docs/standalone/main/integrations/llm/providers/bedrock.md +++ b/content/docs/standalone/main/integrations/llm/providers/bedrock.md @@ -268,6 +268,12 @@ Whether the preference changes the request format that a model accepts depends o > [!NOTE] > Requests to the Mantle endpoint are signed for the `bedrock-mantle` service rather than `bedrock`. If you scope an IAM policy by service name, grant both before you switch a route to Mantle. +## Tool choice + +When a chat request reaches Bedrock Runtime, the conversion sends the tool choice from the client request as the Bedrock Converse `toolChoice` field. OpenAI Chat Completions and Responses requests can use `tool_choice` values such as `auto`, `required`, or a named function. Anthropic Messages requests can use `tool_choice` values such as `auto`, `none`, `any`, or a named tool. + +The selected Bedrock model decides which tool-choice values are supported. If a model rejects forced tool use, the client receives the Bedrock error instead of a request that silently ignores the field. With extended thinking, the conversion keeps the `tool_choice` value from the client request. The `auto` value stays `auto`, and `none` or an omitted field is omitted. Forced choices go to Bedrock for the model to accept or reject. + ## Token counting Bedrock supports token counting for Anthropic models via the `count_tokens` endpoint. diff --git a/content/docs/standalone/main/release-notes/release-notes.md b/content/docs/standalone/main/release-notes/release-notes.md index a8400e704..2739c7519 100644 --- a/content/docs/standalone/main/release-notes/release-notes.md +++ b/content/docs/standalone/main/release-notes/release-notes.md @@ -203,6 +203,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `fro + **Traffic management** @@ -213,6 +214,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `fro **LLM** - Vertex AI catalog lookups resolve Anthropic model aliases such as `claude-sonnet-4-5-20250929`, `anthropic/claude-sonnet-4-5@20250929`, and `publishers/anthropic/models/claude-sonnet-4-5@20250929`. +- Bedrock Runtime requests now send client `tool_choice` settings as Converse `toolChoice`, so forced tool choices can take effect and models that reject them return Bedrock errors. **MCP** From d21cd7e7792d63f7fc40088ec31470187f26682e Mon Sep 17 00:00:00 2001 From: Kristin Brown Date: Fri, 9 Oct 2026 12:28:37 -0400 Subject: [PATCH 2/2] Fixes Signed-off-by: Kristin Brown --- .../pages/agentgateway/llm/providers/bedrock.md | 11 +++++++++-- .../kubernetes/latest/release-notes/release-notes.md | 2 ++ .../standalone/latest/release-notes/release-notes.md | 9 +++++++++ .../main/documentation/llm/api-types/messages.md | 2 +- .../main/integrations/llm/providers/bedrock.md | 11 +++++++++-- 5 files changed, 30 insertions(+), 5 deletions(-) diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md index 195a8f29a..4e63577fc 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md @@ -280,9 +280,16 @@ Whether the preference changes the request format that a model accepts depends o {{% version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" %}} ## Tool choice -When a chat request reaches Bedrock Runtime, the conversion sends the tool choice from the client request as the Bedrock Converse `toolChoice` field. OpenAI Chat Completions and Responses requests can use `tool_choice` values such as `auto`, `required`, or a named function. Anthropic Messages requests can use `tool_choice` values such as `auto`, `none`, `any`, or a named tool. +When a chat request goes to Bedrock Runtime, the `tool_choice` value from the client request is converted to the Bedrock Converse `toolChoice` field. -The selected Bedrock model decides which tool-choice values are supported. If a model rejects forced tool use, the client receives the Bedrock error instead of a request that silently ignores the field. With extended thinking, the conversion keeps the `tool_choice` value from the client request. The `auto` value stays `auto`, and `none` or an omitted field is omitted. Forced choices go to Bedrock for the model to accept or reject. +| Client `tool_choice` | Bedrock `toolChoice` | +|---|---| +| `auto` | `auto` | +| `required` (Chat Completions, Responses) or `any` (Messages) | `any` | +| A named function or tool | `tool`, with that name | +| `none`, or omitted | Omitted. Bedrock Converse has no `none` option. The tools are still sent, so the model can still call them. | + +The Bedrock model decides which values it accepts. If the model rejects forced tool use, such as `any` or a named tool, the client receives the Bedrock error. For example, Anthropic models reject forced tool use when [extended thinking](https://platform.claude.com/docs/en/build-with-claude/thinking#thinking-with-tool-use) is enabled. The tool choice is not changed for requests that use extended thinking. {{% /version %}} ## Prompt caching diff --git a/content/docs/kubernetes/latest/release-notes/release-notes.md b/content/docs/kubernetes/latest/release-notes/release-notes.md index 904ad9dfa..63b458b31 100644 --- a/content/docs/kubernetes/latest/release-notes/release-notes.md +++ b/content/docs/kubernetes/latest/release-notes/release-notes.md @@ -185,6 +185,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `pre + **Traffic management** @@ -197,6 +198,7 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `pre **MCP** - Access-log CEL expressions can now read dynamic metadata that ExtMCP request-phase guardrails return through `mcpGuardrails`, including on resumed stateful MCP sessions. For more information, see [Log MCP guardrail metadata]({{< link-hextra path="/documentation/observability/access-logs/view/#mcp-guardrails" >}}). +- OpenAPI MCP tools return `image/*` responses as MCP image content instead of UTF-8-decoded text. **Security** diff --git a/content/docs/standalone/latest/release-notes/release-notes.md b/content/docs/standalone/latest/release-notes/release-notes.md index 15c66f8b9..8551e3c01 100644 --- a/content/docs/standalone/latest/release-notes/release-notes.md +++ b/content/docs/standalone/latest/release-notes/release-notes.md @@ -200,11 +200,15 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `fro + + + **Traffic management** - Connections with `maxConnectionDuration` close with up to 10% jitter, which reduces synchronized reconnects at the configured connection age. - Bare backend references in standalone HTTP routes, TCP routes, and MCP target backend hosts now resolve to top-level backends. A reference such as `backend: upstream` no longer fails at request time with `service not found` when the top-level backend is named `upstream`. +- Route path regexes now use full-path matching consistently, including patterns with alternation or lazy quantifiers. **LLM** @@ -213,3 +217,8 @@ You can now opt in to OpenTelemetry field names for stdout access logs. Set `fro **MCP** - Access-log CEL expressions can now read dynamic metadata that ExtMCP request-phase guardrails return through `mcpGuardrails`, including on resumed stateful MCP sessions. For more information, see [Log MCP guardrail metadata]({{< link-hextra path="/documentation/observability/access-logs/view/#mcp-guardrails" >}}). +- OpenAPI MCP tools return `image/*` responses as MCP image content instead of UTF-8-decoded text. + +**Operations** + +- Standalone Helm installs can set `podDisruptionBudget.maxUnavailable` without also rendering the default `podDisruptionBudget.minAvailable` value. diff --git a/content/docs/standalone/main/documentation/llm/api-types/messages.md b/content/docs/standalone/main/documentation/llm/api-types/messages.md index 5a738c38c..ae47f0eb5 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/messages.md +++ b/content/docs/standalone/main/documentation/llm/api-types/messages.md @@ -148,7 +148,7 @@ Certain models, such as `gpt-5.3`, reject a Chat Completions request that sets b The Bedrock Converse conversion carries extended-thinking history in both directions, including encrypted reasoning. -The Bedrock Converse conversion preserves tool choice for Messages requests. The `auto` value becomes the Bedrock `toolChoice.auto` value, and `any` becomes `toolChoice.any`. A `tool` choice becomes `toolChoice.tool`. The `none` value or an omitted tool choice is omitted. Extended thinking does not rewrite the choice. A model that rejects forced tool use with thinking returns a provider error. +The Bedrock Converse conversion preserves tool choice for Messages requests. The `auto` value becomes the Bedrock `toolChoice.auto` value, and `any` becomes `toolChoice.any`. A `tool` choice becomes `toolChoice.tool`. The `none` value or an omitted tool choice is omitted, and the tools are still sent, so the model can still call them. Extended thinking does not rewrite the choice. A model that rejects forced tool use with thinking returns a provider error. On the way out, a `thinking` block in the message history becomes a Bedrock `reasoningContent.reasoningText` block, with its signature. A `redacted_thinking` block becomes a Bedrock `reasoningContent.redactedContent` block, so that a client can replay encrypted reasoning that Bedrock returned on an earlier turn. Document, search-result, and server-tool content blocks are dropped. diff --git a/content/docs/standalone/main/integrations/llm/providers/bedrock.md b/content/docs/standalone/main/integrations/llm/providers/bedrock.md index 119a26d2e..e21549afa 100644 --- a/content/docs/standalone/main/integrations/llm/providers/bedrock.md +++ b/content/docs/standalone/main/integrations/llm/providers/bedrock.md @@ -270,9 +270,16 @@ Whether the preference changes the request format that a model accepts depends o ## Tool choice -When a chat request reaches Bedrock Runtime, the conversion sends the tool choice from the client request as the Bedrock Converse `toolChoice` field. OpenAI Chat Completions and Responses requests can use `tool_choice` values such as `auto`, `required`, or a named function. Anthropic Messages requests can use `tool_choice` values such as `auto`, `none`, `any`, or a named tool. +When a chat request goes to Bedrock Runtime, the `tool_choice` value from the client request is converted to the Bedrock Converse `toolChoice` field. -The selected Bedrock model decides which tool-choice values are supported. If a model rejects forced tool use, the client receives the Bedrock error instead of a request that silently ignores the field. With extended thinking, the conversion keeps the `tool_choice` value from the client request. The `auto` value stays `auto`, and `none` or an omitted field is omitted. Forced choices go to Bedrock for the model to accept or reject. +| Client `tool_choice` | Bedrock `toolChoice` | +|---|---| +| `auto` | `auto` | +| `required` (Chat Completions, Responses) or `any` (Messages) | `any` | +| A named function or tool | `tool`, with that name | +| `none`, or omitted | Omitted. Bedrock Converse has no `none` option. The tools are still sent, so the model can still call them. | + +The Bedrock model decides which values it accepts. If the model rejects forced tool use, such as `any` or a named tool, the client receives the Bedrock error. For example, Anthropic models reject forced tool use when [extended thinking](https://platform.claude.com/docs/en/build-with-claude/thinking#thinking-with-tool-use) is enabled. The tool choice is not changed for requests that use extended thinking. ## Token counting