[Muti Provider Routing] Add OpenAI-compatible multi-provider streaming

[Aakashwije]

Purpose

Update the AI Gateway multi-provider routing documentation to reflect the latest transformer behavior from wso2/gateway-controllers#270.

Previously, the docs stated that Anthropic and Gemini streaming responses were passed through as provider-native server-sent events (SSE). That is now outdated. Anthropic and Gemini streaming responses are converted to OpenAI chat.completion.chunk SSE, matching the OpenAI-compatible contract already supported by AWS Bedrock.

Related PRs:

• Resolves Add multi-provider LLM routing capabilities #404
• Related to Add OpenAI - compatible streaming for Anthropic and Gemini gateway-controllers#270

Checklist

• Verified that llms.txt (located at en/docs/llms.txt) is updated for AI readiness content.
• Ensured meaningful alt text for images and that the information contained in an image also appears in text so the relevant info exists in the body of the document.
• Added or updated frontmatter for the respective pages.

Goals

• Document that supported multi-provider LLM proxy streaming responses now preserve the OpenAI-compatible Chat Completions streaming shape.
• Replace outdated Anthropic and Gemini native SSE passthrough references.
• Add a streaming invocation example for selected providers.
• Clarify that provider-native stream events without OpenAI Chat Completions equivalents are omitted.

Approach

Updated the following pages:

• en/docs/ai-gateway/1.2.0/llm-proxy/multi-provider-routing.md
• en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md

The changes:

• Updated overview text to mention OpenAI-compatible normalization for non-streaming and streaming Chat Completions responses.
• Added a “Stream from a selected provider” example.
• Updated Anthropic and Gemini capability matrix entries.
• Updated limitations and troubleshooting sections to remove stale provider-native SSE passthrough guidance.
• Updated frontmatter last_updated values.

User stories

• As an application developer, I can use stream: true with a multi-provider LLM proxy and receive OpenAI-compatible streaming chunks.
• As a platform team, I can route streaming traffic across OpenAI, Azure OpenAI, Mistral, Anthropic, Gemini, and AWS Bedrock without changing OpenAI-compatible client code.
• As a docs reader, I can understand which provider-native stream details may be omitted when converted to OpenAI Chat Completions chunks.

Release note

Updated AI Gateway multi-provider routing documentation to describe OpenAI-compatible streaming support across supported providers, including Anthropic and Gemini streaming conversion.

Documentation

This PR updates product documentation:

• en/docs/ai-gateway/1.2.0/llm-proxy/multi-provider-routing.md
• en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md

Training

N/A - documentation clarification only.

Certification

N/A - this update clarifies documented behavior and does not introduce a new certification exam concept.

Marketing

N/A - documentation update only.

Automation tests

• Unit tests
  N/A - documentation-only change.

• Integration tests
  N/A - documentation-only change.

Validation performed:

git diff --check -- en/docs/ai-gateway/1.2.0/llm-proxy/multi-provider-routing.md en/docs/ai-gateway/next/llm-proxy/multi-provider-routing.md

cd en
python -m mkdocs build

Also verified the updated pages locally with the MkDocs build.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4ded1294-1d2e-46f6-9049-f3ea47a50ebb

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}