wso2 · Aakashwije · Aug 19, 2026
diff --git a/en/docs/ai-gateway/1.2.0/llm-proxy/multi-provider-routing.md b/en/docs/ai-gateway/1.2.0/llm-proxy/multi-provider-routing.md
@@ -8,15 +8,15 @@ tags:
   - llm
   - routing
 author: WSO2 API Platform Documentation Team
-last_updated: 2026-08-07
+last_updated: 2026-08-19
 content_type: "guide"
 ---
 
 # Multi-provider routing for LLM proxies
 
 ## Overview
 
-Multi-provider routing lets one large language model (LLM) proxy expose a single OpenAI-compatible endpoint while routing each request to a selected LLM provider. Applications continue to use the same endpoint and OpenAI-compatible request format when the upstream provider changes. Non-streaming responses are normalized where supported; streaming compatibility varies by provider.
+Multi-provider routing lets one large language model (LLM) proxy expose a single OpenAI-compatible endpoint while routing each request to a selected LLM provider. Applications continue to use the same endpoint and OpenAI-compatible request format when the upstream provider changes. Non-streaming and streaming Chat Completions responses are normalized to OpenAI-compatible shapes where supported.
 
 For example, an application can send all requests to `/openai-multi/chat/completions` and select OpenAI or Anthropic with the `x-provider` request header. The proxy can also distribute requests automatically across provider and model pairs by using round-robin or weighted round-robin routing.
 
@@ -142,7 +142,7 @@ OpenAI-compatible client request
       v
     Anthropic API
       |
-      | non-streaming response is transformed to OpenAI format
+      | response is transformed to OpenAI format
       v
 OpenAI-compatible client response
 ```
@@ -428,10 +428,31 @@ curl -X POST https://localhost:8443/openai-multi/chat/completions \
   }'
 ```
 
-The Anthropic transformer replaces the request's `model` value with the model configured under `transformer.params.model`. It also translates the request to the Anthropic Messages format and translates non-streaming responses back to the OpenAI response shape.
+The Anthropic transformer replaces the request's `model` value with the model configured under `transformer.params.model`. It also translates the request to the Anthropic Messages format and translates non-streaming and streaming responses back to the OpenAI response shape.
 
 Header names and mapped header values are matched case-insensitively. Leading and trailing whitespace in the header value is ignored. If the header is missing, empty, or does not match a mapping, the router selects `defaultProvider`.
 
+#### Stream from a selected provider
+
+Use the same OpenAI Chat Completions payload and set `stream` to `true`. The selected provider returns OpenAI-compatible SSE through the proxy. OpenAI, Azure OpenAI, and Mistral streams pass through in their OpenAI-compatible shape. Anthropic, Gemini, and AWS Bedrock streams are converted to OpenAI `chat.completion.chunk` events.
+
+```bash
+curl -N -X POST https://localhost:8443/openai-multi/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: ${PROXY_CONSUMER_KEY}" \
+  -H "x-provider: anthropic" \
+  -d '{
+    "model": "client-model-name",
+    "stream": true,
+    "messages": [
+      {
+        "role": "user",
+        "content": "Write a haiku about API gateways."
+      }
+    ]
+  }'
+```
+
 ### Add more providers
 
 Add each selectable provider under `additionalProviders`, then add a corresponding mapping under the LLM Header Router policy (`llm-header-router`).
@@ -442,10 +463,10 @@ Use a transformer when an additional provider does not accept and return the Ope
 
 | Target provider | Transformer type used in this guide | Purpose |
 |-----------------|-------------------------------------|---------|
-| Anthropic | `openai-to-anthropic-transformer` | Converts OpenAI-compatible requests to Anthropic Messages and converts non-streaming responses back to OpenAI format. |
+| Anthropic | `openai-to-anthropic-transformer` | Converts OpenAI-compatible requests to Anthropic Messages and converts responses, including streaming responses, back to OpenAI format. |
 | Azure OpenAI | `openai-to-azure-openai-transformer` | Adapts the request path for an Azure OpenAI deployment and API version. |
 | Mistral | `openai-to-mistral-transformer` | Normalizes OpenAI-compatible requests and responses for Mistral. |
-| Gemini | `openai-to-gemini-transformer` | Converts OpenAI-compatible requests and non-streaming responses for Gemini. |
+| Gemini | `openai-to-gemini-transformer` | Converts OpenAI-compatible requests and Gemini responses, including streaming responses, back to OpenAI format. |
 | AWS Bedrock | `openai-to-bedrock-transformer` | Converts OpenAI-compatible requests and Bedrock Converse responses, including streaming responses. |
 
 A transformer is not required when the selected provider already exposes an OpenAI-compatible API.
@@ -615,7 +636,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
     | Image input | Base64 and remote URL |
     | Function tools | Converted |
     | Non-streaming OpenAI response | Yes |
-    | Streaming response | Native Anthropic server-sent events (SSE) passthrough |
+    | Streaming response | Converted to OpenAI-compatible server-sent events (SSE) |
 
     Request conversion
 
@@ -644,7 +665,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
 
     Streaming
 
-    Streaming is supported. The transformer selects the Anthropic streaming endpoint and passes native Anthropic server-sent events (SSE) through unchanged. It does not convert the event payloads to OpenAI Chat Completions chunks, so streaming clients must handle Anthropic event payloads.
+    Streaming is supported. The transformer selects the Anthropic streaming endpoint and converts native Anthropic server-sent events (SSE) into OpenAI `chat.completion.chunk` events. It emits role, content, tool-call deltas, finish reasons, usage, and `data: [DONE]`. Anthropic events that have no OpenAI Chat Completions equivalent, such as `ping`, `content_block_stop`, `thinking_delta`, and `signature_delta`, are omitted.
 
     Tools and multimodal input
 
@@ -788,7 +809,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
     | Image input | Base64 and remote URL |
     | Function tools | Converted |
     | Non-streaming OpenAI response | Yes |
-    | Streaming response | Native Gemini SSE passthrough |
+    | Streaming response | Converted to OpenAI-compatible SSE |
 
     Request conversion
 
@@ -818,7 +839,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
 
     Streaming
 
-    Streaming is supported. The transformer selects `streamGenerateContent` and passes native Gemini SSE events through unchanged. It does not convert the event payloads to OpenAI Chat Completions chunks, so streaming clients must handle Gemini event payloads.
+    Streaming is supported. The transformer selects `streamGenerateContent` and converts native Gemini SSE events into OpenAI `chat.completion.chunk` events. It emits content, tool-call deltas, finish reasons, usage, and `data: [DONE]`. Gemini parts flagged as `thought: true` are omitted because OpenAI Chat Completions streaming has no equivalent visible field for them.
 
     Tools and multimodal input
 
@@ -907,7 +928,7 @@ The round-robin policies track failures per model across all requests. A failure
 ## Limitations
 
 - **Chat Completions only:** Cross-provider translation targets the OpenAI Chat Completions API.
-- No universal OpenAI streaming conversion: Only AWS Bedrock converts provider-specific streaming events into OpenAI Chat Completions chunk objects. Anthropic and Gemini return valid SSE streams. Their provider-native event payloads are passed through unchanged.
+- **Chat Completions streaming shape:** Supported provider transformers return OpenAI `chat.completion.chunk` SSE for streaming responses. Provider-native stream events that have no OpenAI Chat Completions equivalent are omitted.
 - **No automatic capability negotiation:** The gateway does not query the selected model for support for vision, tools, schemas, or individual generation parameters.
 - **No automatic routing validation:** Router mappings must match the primary provider ID or an additional provider's effective name.
 - **No request retry or immediate failover:** Suspension removes an unhealthy target from later rotations but does not retry the failing request.
@@ -955,9 +976,9 @@ If the mapping selects an additional provider that has no transformer, confirm t
 
 Model suspension does not retry the current request. Confirm the behavior with a later request after the first target returns `429` or `5xx`. Also confirm that `suspendDuration` is greater than `0` and that each model entry uses the correct effective provider name.
 
-### Streaming is not in OpenAI chunk format
+### Streaming chunks are missing provider-native details
 
-Anthropic and Gemini support streaming through provider-native SSE passthrough. If the client expects OpenAI Chat Completions chunks, adapt the provider-native event payloads in the client or choose a route that returns OpenAI-compatible chunks.
+Supported provider transformers convert streaming responses to OpenAI Chat Completions chunks. If provider-native events are missing, check whether those events have an OpenAI Chat Completions equivalent. Anthropic thinking events and Gemini thought parts are intentionally omitted.
 
 ### An image or tool request is rejected by the provider
 

diff --git a/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md b/en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md
@@ -8,15 +8,15 @@ tags:
   - llm
   - routing
 author: WSO2 API Platform Documentation Team
-last_updated: 2026-08-05
+last_updated: 2026-08-19
 content_type: "guide"
 ---
 
 # Multi-Provider Routing for LLM Proxies
 
 ## Overview
 
-Multi-provider routing lets one large language model (LLM) proxy expose a single OpenAI-compatible endpoint while routing each request to a selected LLM provider. Applications continue to use the same endpoint and OpenAI-compatible request format when the upstream provider changes. Non-streaming responses are normalized where supported; streaming compatibility varies by provider.
+Multi-provider routing lets one large language model (LLM) proxy expose a single OpenAI-compatible endpoint while routing each request to a selected LLM provider. Applications continue to use the same endpoint and OpenAI-compatible request format when the upstream provider changes. Non-streaming and streaming Chat Completions responses are normalized to OpenAI-compatible shapes where supported.
 
 For example, an application can send all requests to `/openai-multi/chat/completions` and select OpenAI or Anthropic with the `x-provider` request header. The proxy can also distribute requests automatically across provider and model pairs by using round-robin or weighted round-robin routing.
 
@@ -425,10 +425,31 @@ curl -k -X POST https://localhost:8443/openai-multi/chat/completions \
   }'
 ```
 
-The Anthropic transformer replaces the request's `model` value with the model configured under `transformer.params.model`. It also translates the request to the Anthropic Messages format and translates the response back to the OpenAI response shape.
+The Anthropic transformer replaces the request's `model` value with the model configured under `transformer.params.model`. It also translates the request to the Anthropic Messages format and translates non-streaming and streaming responses back to the OpenAI response shape.
 
 Header names and mapped header values are matched case-insensitively. Leading and trailing whitespace in the header value is ignored. If the header is missing, empty, or does not match a mapping, the router selects `defaultProvider`.
 
+#### Stream from a selected provider
+
+Use the same OpenAI Chat Completions payload and set `stream` to `true`. The selected provider returns OpenAI-compatible SSE through the proxy. OpenAI, Azure OpenAI, and Mistral streams pass through in their OpenAI-compatible shape. Anthropic, Gemini, and AWS Bedrock streams are converted to OpenAI `chat.completion.chunk` events.
+
+```bash
+curl -N -k -X POST https://localhost:8443/openai-multi/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: ${PROXY_CONSUMER_KEY}" \
+  -H "x-provider: anthropic" \
+  -d '{
+    "model": "client-model-name",
+    "stream": true,
+    "messages": [
+      {
+        "role": "user",
+        "content": "Write a haiku about API gateways."
+      }
+    ]
+  }'
+```
+
 ### Add more providers
 
 Add each selectable provider under `additionalProviders`, then add a corresponding mapping under the LLM Header Router policy (`llm-header-router`).
@@ -439,10 +460,10 @@ Use a transformer when an additional provider does not accept and return the Ope
 
 | Target provider | Transformer type used in this guide | Purpose |
 |-----------------|-------------------------------------|---------|
-| Anthropic | `openai-to-anthropic-transformer` | Converts OpenAI-compatible requests to Anthropic Messages and converts non-streaming responses back to OpenAI format. |
+| Anthropic | `openai-to-anthropic-transformer` | Converts OpenAI-compatible requests to Anthropic Messages and converts responses, including streaming responses, back to OpenAI format. |
 | Azure OpenAI | `openai-to-azure-openai-transformer` | Adapts the request path for an Azure OpenAI deployment and API version. |
 | Mistral | `openai-to-mistral-transformer` | Normalizes OpenAI-compatible requests and responses for Mistral. |
-| Gemini | `openai-to-gemini-transformer` | Converts OpenAI-compatible requests and non-streaming responses for Gemini. |
+| Gemini | `openai-to-gemini-transformer` | Converts OpenAI-compatible requests and Gemini responses, including streaming responses, back to OpenAI format. |
 | AWS Bedrock | `openai-to-bedrock-transformer` | Converts OpenAI-compatible requests and Bedrock Converse responses, including streaming responses. |
 
 A transformer is not required when the selected provider already exposes an OpenAI-compatible API.
@@ -612,7 +633,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
     | Image input | Base64 and remote URL |
     | Function tools | Converted |
     | Non-streaming OpenAI response | Yes |
-    | Streaming response | Native Anthropic SSE passthrough |
+    | Streaming response | Converted to OpenAI-compatible SSE |
 
     **Request conversion**
 
@@ -641,7 +662,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
 
     **Streaming**
 
-    Streaming is supported. The transformer selects the Anthropic streaming endpoint and passes native Anthropic server-sent events (SSE) through unchanged. It does not convert the event payloads to OpenAI Chat Completions chunks, so streaming clients must handle Anthropic event payloads.
+    Streaming is supported. The transformer selects the Anthropic streaming endpoint and converts native Anthropic server-sent events (SSE) into OpenAI `chat.completion.chunk` events. It emits role, content, tool-call deltas, finish reasons, usage, and `data: [DONE]`. Anthropic events that have no OpenAI Chat Completions equivalent, such as `ping`, `content_block_stop`, `thinking_delta`, and `signature_delta`, are omitted.
 
     **Tools and multimodal input**
 
@@ -783,7 +804,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
     | Image input | Base64 and remote URL |
     | Function tools | Converted |
     | Non-streaming OpenAI response | Yes |
-    | Streaming response | Native Gemini SSE passthrough |
+    | Streaming response | Converted to OpenAI-compatible SSE |
 
     **Request conversion**
 
@@ -813,7 +834,7 @@ Expand a provider to see its complete transformation behavior. `Converted` means
 
     **Streaming**
 
-    Streaming is supported. The transformer selects `streamGenerateContent` and passes native Gemini SSE events through unchanged. It does not convert the event payloads to OpenAI Chat Completions chunks, so streaming clients must handle Gemini event payloads.
+    Streaming is supported. The transformer selects `streamGenerateContent` and converts native Gemini SSE events into OpenAI `chat.completion.chunk` events. It emits content, tool-call deltas, finish reasons, usage, and `data: [DONE]`. Gemini parts flagged as `thought: true` are omitted because OpenAI Chat Completions streaming has no equivalent visible field for them.
 
     **Tools and multimodal input**
 
@@ -901,8 +922,8 @@ The round-robin policies track failures per provider/model pair. The same model
 
 ## Limitations
 
-- **Chat Completions only:** Cross-provider translation targets the OpenAI `/chat/completions` model.
-- No universal OpenAI streaming conversion: Only AWS Bedrock converts provider-specific streaming events into OpenAI Chat Completions chunk objects. Anthropic and Gemini return valid SSE streams. Their provider-native event payloads are passed through unchanged.
+- **Chat Completions only:** Cross-provider translation targets the OpenAI `/chat/completions` endpoint.
+- **Chat Completions streaming shape:** Supported provider transformers return OpenAI `chat.completion.chunk` SSE for streaming responses. Provider-native stream events that have no OpenAI Chat Completions equivalent are omitted.
 - **No automatic capability negotiation:** The gateway does not query the selected model for support for vision, tools, schemas, or individual generation parameters.
 - **No automatic routing validation:** Router mappings must match the primary provider ID or an additional provider's effective name.
 - **No request retry or immediate failover:** Suspension removes an unhealthy target from later rotations but does not retry the failing request.
@@ -950,9 +971,9 @@ If the mapping selects an additional provider that has no transformer, confirm t
 
 Model suspension does not retry the current request. Confirm the behavior with a later request after the first target returns `429` or `5xx`. Also confirm that `suspendDuration` is greater than `0` and that each model entry uses the correct effective provider name.
 
-### Streaming is not in OpenAI chunk format
+### Streaming chunks are missing provider-native details
 
-Anthropic and Gemini support streaming through provider-native SSE passthrough. If the client expects OpenAI Chat Completions chunks, adapt the provider-native event payloads in the client or choose a route that returns OpenAI-compatible chunks.
+Supported provider transformers convert streaming responses to OpenAI Chat Completions chunks. If provider-native events are missing, check whether those events have an OpenAI Chat Completions equivalent. Anthropic thinking events and Gemini thought parts are intentionally omitted.
 
 ### An image or tool request is rejected by the provider