diff --git a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md index d18ee08de..d08284ad7 100644 --- a/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md +++ b/assets/agw-docs/pages/agentgateway/llm/providers/bedrock.md @@ -465,7 +465,7 @@ Some Bedrock models return encrypted reasoning in a `reasoningContent.redactedCo | `/v1/responses` | A reasoning item with `encrypted_content`. | Send the reasoning item back in the input. The next Bedrock request includes it as `reasoningContent.redactedContent`. | | `/v1/chat/completions` | Omitted, because the Chat Completions format has no field for encrypted reasoning. Signed reasoning text still arrives in `reasoning_content` and `reasoning_signature`. | Not possible. | -A streamed reply does not keep the encrypted payload, so streamed encrypted reasoning cannot be replayed. A `/v1/messages` stream shows the encrypted reasoning as a `thinking` block with the text `[REDACTED]`. +A streamed reply does not keep the encrypted payload, so streamed encrypted reasoning cannot be replayed. A `/v1/messages` stream shows the encrypted reasoning as a `thinking` block with the text `[REDACTED]`.{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,1.6.x,2.2.x" >}} If redacted reasoning fragments in a streamed Bedrock Runtime reply exceed the stream buffer limit before the block ends, the stream emits an `api_error` event with the message `Stream processing error`. The stream then stops.{{< /version >}} {{% /version %}} ## Structured outputs diff --git a/content/docs/kubernetes/main/release-notes/release-notes.md b/content/docs/kubernetes/main/release-notes/release-notes.md index 9e5547e0c..747ee4830 100644 --- a/content/docs/kubernetes/main/release-notes/release-notes.md +++ b/content/docs/kubernetes/main/release-notes/release-notes.md @@ -37,6 +37,7 @@ Review the release notes for agentgateway on Kubernetes. + **Traffic management** @@ -48,6 +49,7 @@ Review the release notes for agentgateway on Kubernetes. - Messages requests that ask for reasoning no longer send reasoning parameters to older OpenAI chat models whose IDs start with `gpt-3.5`, `gpt-4`, or `chatgpt-4o`. - Bedrock conversion keeps prompt-cache markers when unsupported Messages blocks or blank Chat Completions text are dropped, and Anthropic cache TTLs reach Bedrock cache points. - Bedrock Runtime requests now send client `tool_choice` settings as Converse `toolChoice`, so forced tool choices can take effect and models that reject them return Bedrock errors. +- LLM streaming conversions now respect buffer limits: Bedrock redacted reasoning returns `Stream processing error` when buffered fragments exceed the stream limit, Responses namespace tool descriptions fail with `RequestTooLarge` above 8 MiB, and tool-call arguments are buffered only when tool-call logging is enabled. **MCP** diff --git a/content/docs/standalone/main/documentation/llm/api-types/messages.md b/content/docs/standalone/main/documentation/llm/api-types/messages.md index c7c0e9601..556255e2f 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/messages.md +++ b/content/docs/standalone/main/documentation/llm/api-types/messages.md @@ -172,7 +172,7 @@ Each `cache_control` marker becomes a Bedrock cache point after the content that Messages that are left empty after the conversion drops unsupported blocks are not sent to Bedrock. The conversion adds at most four cache points to a request, so a marker is dropped when four cache points are already set. -On the way back, Bedrock reasoning text becomes a `thinking` block. In a buffered reply, Bedrock encrypted reasoning becomes a `redacted_thinking` block that keeps the encrypted payload for the next turn. A streamed reply does not keep the payload. The encrypted reasoning arrives as a `thinking` block with the text `[REDACTED]`, which cannot be replayed. For the same behavior on other Bedrock endpoints, see [Encrypted reasoning]({{< link-hextra path="/integrations/llm/providers/bedrock/#encrypted-reasoning" >}}). +On the way back, Bedrock reasoning text becomes a `thinking` block. In a buffered reply, Bedrock encrypted reasoning becomes a `redacted_thinking` block that keeps the encrypted payload for the next turn. A streamed reply does not keep the payload. The encrypted reasoning arrives as a `thinking` block with the text `[REDACTED]`, which cannot be replayed. If redacted reasoning fragments in a streamed Bedrock Runtime reply exceed the stream buffer limit before the block ends, the stream emits an `api_error` event with the message `Stream processing error`. The stream then stops. For the same behavior on other Bedrock endpoints, see [Encrypted reasoning]({{< link-hextra path="/integrations/llm/providers/bedrock/#encrypted-reasoning" >}}). ## Using the API diff --git a/content/docs/standalone/main/documentation/llm/api-types/responses.md b/content/docs/standalone/main/documentation/llm/api-types/responses.md index db7409857..3aeeefe5b 100644 --- a/content/docs/standalone/main/documentation/llm/api-types/responses.md +++ b/content/docs/standalone/main/documentation/llm/api-types/responses.md @@ -19,6 +19,8 @@ Responses requests can include namespace tools. When the selected provider takes A function tool choice can use the bare member name only when that name is unique across the namespace tools. When more than one namespace has a function with the same name, set the tool choice function name to `namespace__function`. Chat Completions and Bedrock conversions cannot enforce `allowed_tools` choices or convert custom namespace members. Requests that use those features fail before they reach the provider. +During flattening, the conversion copies namespace descriptions onto each function tool. If the copied namespace descriptions exceed 8 MiB for one request, the request fails with `RequestTooLarge` before it reaches the provider. + ## Route type configuration In the simplified `llm` configuration, agentgateway automatically maps `/v1/responses` requests to the `responses` route type, so no explicit route configuration is required. diff --git a/content/docs/standalone/main/release-notes/release-notes.md b/content/docs/standalone/main/release-notes/release-notes.md index 4645e06f4..ffd42dc2c 100644 --- a/content/docs/standalone/main/release-notes/release-notes.md +++ b/content/docs/standalone/main/release-notes/release-notes.md @@ -32,6 +32,7 @@ Review the release notes for agentgateway standalone. + **Traffic management** @@ -43,6 +44,7 @@ Review the release notes for agentgateway standalone. - Messages requests that ask for reasoning no longer send reasoning parameters to older OpenAI chat models whose IDs start with `gpt-3.5`, `gpt-4`, or `chatgpt-4o`. - Bedrock conversion keeps prompt-cache markers when unsupported Messages blocks or blank Chat Completions text are dropped, and Anthropic cache TTLs reach Bedrock cache points. - Bedrock Runtime requests now send client `tool_choice` settings as Converse `toolChoice`, so forced tool choices can take effect and models that reject them return Bedrock errors. +- LLM streaming conversions now respect buffer limits: Bedrock redacted reasoning returns `Stream processing error` when buffered fragments exceed the stream limit, Responses namespace tool descriptions fail with `RequestTooLarge` above 8 MiB, and tool-call arguments are buffered only when tool-call logging is enabled. **MCP**