docs: add inference benchmarks for Kubernetes and standalone

assets/agw-docs/pages/agentgateway/benchmarking/README.md

@@ -0,0 +1,27 @@
+# Maintaining the inference benchmark pages
+
+The source of truth is `agentgateway/benchmarks`. This snapshot uses commit
+`522fc04a595faad40e6cb070d0f2275e1314f1f9` (PR #3) and campaign
+`optimized-baseline-v0230-gateway-refresh-20260817` under
+`inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817`.
+
+- `environment.md`, `results-notes.md`, and `methodology.md` share the test
+  environment, summary guidance, and metric definitions between both modes.
+- `latency-caveat.md` places the shared failure-rate and single-run qualifications
+  immediately after the charts.
+- `kubernetes.md` and `standalone.md` contain the respective published tables and
+  interpretation, reused by the `latest` and `main` page wrappers at
+  `documentation/llm/inference/benchmarking.md`. The inference overview at `/documentation/inference/` links to these
+  reports and the routing guide at `documentation/llm/inference/inference-routing/`.
+- The six PNGs under `assets/img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/`
+  are regenerated from the pinned metrics with the published plotting code,
+  grouped by mode. The baseline legend reads `Service load-balancing` instead
+  of `k8s service (RR)`; the plotted measurements are unchanged.
+
+To publish another campaign, review its manifest, provenance, reports, and CSVs
+at one pinned commit. Update the methodology, mode-specific tables and text,
+evidence links, and all charts together. Use a campaign-specific image directory
+and verify table values against the CSVs. Keep the tested component versions
+explicit even when they differ from the selected docs version. Build the site
+and check both modes in `latest` and `main`, including the inference section
+navigation, images, and links.

assets/agw-docs/pages/agentgateway/benchmarking/environment.md

@@ -0,0 +1,47 @@
+## Test environment
+
+This {{< gloss "Benchmark campaign" >}}campaign{{< /gloss >}} compares EPP-based agentgateway routing with a plain Kubernetes Service across eight vLLM model servers. The Kubernetes Service baseline uses Service load-balancing, with no [EPP](https://llm-d.ai/docs/architecture/core/router#llm-d-endpoint-picker-epp) or [scoring](https://llm-d.ai/docs/architecture/core/router/epp/configuration#plugins) to route inference requests to a model server endpoint.
+
+The `optimized-baseline-qwen3-32b-h100-v0.9` [reference profile](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/suites/llm-d-benchmark/references/optimized-baseline-qwen3-32b-h100-v0.9.yaml) is used for the campaign,
+based on llm-d's [Optimized Baseline](https://llm-d.ai/docs/well-lit-paths/foundations/optimized-baseline).
+The agentgateway {{< gloss "Benchmark treatment" >}}treatments{{< /gloss >}} use approximate prefix-cache affinity and token-load
+scoring to select model-server endpoints.
+
+If you are interested in benchmarks for other llm-d [well-lit paths](https://llm-d.ai/docs/well-lit-paths)
+or inference topologies, [create an issue](https://github.com/agentgateway/benchmarks/issues)
+that explains your use case.
+
+**Prefill/decode (P/D) disaggregation is not used.** Each vLLM replica handles both
+prefill and decode. The [comparison configuration](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/suites/llm-d-benchmark/scenarios/defaults/agentgateway-comparison.yaml)
+disables separate prefill replicas and the P/D routing sidecar, so all treatments
+use the same model-server topology. These results compare routing within that
+topology, rather than measuring the effects of P/D disaggregation.
+
+The table summarizes the published campaign. See the [campaign manifest](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-manifest.yaml)
+for configuration details and the [campaign provenance](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-provenance.yaml)
+for execution times and shared configuration hashes.
+
+| Setting | Value |
+| --- | --- |
+| Cluster provider | `gke` (Google Kubernetes Engine) |
+| Accelerator type and model | NVIDIA H100 GPU (`gpu`, `h100`) |
+| Total accelerators | 16 GPUs (8 replicas × 2 GPUs per replica) |
+| Backend type | `vllm` |
+| Model | `Qwen/Qwen3-32B` |
+
+> [!NOTE]
+> These are published results for agentgateway **v1.4.1**, regardless of the docs
+> version selected.
+
+### Workload
+
+The [workload configuration](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/suites/llm-d-benchmark/workloads/upstream-optimized-baseline.yaml.in)
+uses streaming completion requests with the following settings:
+
+- **Arrival pattern:** Randomized (Poisson).
+- **Request timeout:** 300 seconds.
+- **Shared-prefix groups:** 150 groups with five prompts per group.
+- **Prompt lengths:** 6,000 tokens for the shared system prompt and 1,200 tokens for the user prompt.
+- **Target output length:** 1,000 tokens.
+- **Multi-turn chat:** Disabled.
+- **Requested rates:** 3–60 requests/s across the measured stages, after warm-up.

assets/agw-docs/pages/agentgateway/benchmarking/kubernetes.md

@@ -0,0 +1,90 @@
+Compare the performance of agentgateway on Kubernetes versus a plain Kubernetes Service for inference routing.
+
+In `kubernetes` mode, an agentgateway proxy runs as the `Gateway` data plane.
+An `HTTPRoute` targets an `InferencePool`, and the [Endpoint Picker Extension (EPP)](https://llm-d.ai/docs/architecture/core/router/epp)
+selects the model-server endpoint.
+
+For setup instructions, see the [inference benchmark README](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/README.md).
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/environment.md" >}}
+
+## Results
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/results-notes.md" >}}
+
+| Metric | Kubernetes Service | agentgateway on Kubernetes | Change vs. baseline |
+| :--- | :--- | :--- | :--- |
+| Peak output tokens/s | 6,910 | 14,241 | +106.1% |
+| Achieved requests/s | 6.70 | 13.96 | +108.3% |
+| TTFT p50 (s) | 62.9 | 0.2 | −99.7% |
+| TTFT p90 (s) | 135.6 | 0.2 | −99.8% |
+| ITL p50 (ms) | 30.3 | 49.6 | +63.7% |
+
+### Token throughput
+
+The charts show input, output, and total token throughput in tokens per second
+across the tested request rates.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/kubernetes/throughput_vs_qps.png" alt="Input, output, and total token throughput versus requested QPS for agentgateway on Kubernetes and the Kubernetes Service baseline." >}}
+
+### Mean latency
+
+The charts show mean TTFT, ITL, and NTPOT in milliseconds. The summary table
+reports latency percentiles, with TTFT in seconds and ITL in milliseconds.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/kubernetes/latency_vs_qps.png" alt="Mean TTFT, ITL, and NTPOT in milliseconds versus requested QPS for agentgateway on Kubernetes and the Kubernetes Service baseline." >}}
+
+### Time to first token (p90)
+
+The chart shows p90 TTFT in milliseconds while the tables report it in seconds.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/kubernetes/ttft_p90_vs_qps.png" alt="Time to first token (p90) in milliseconds versus requested QPS for agentgateway on Kubernetes and the Kubernetes Service baseline." >}}
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/latency-caveat.md" >}}
+
+{{< details title="Per-rate breakdown" >}}
+
+Output throughput is in tokens/s (higher is better); TTFT is in seconds (lower
+is better).
+
+| Requests/s | Kubernetes Service Output | agentgateway on Kubernetes Output | Kubernetes Service TTFT p50 | agentgateway on Kubernetes TTFT p50 | Kubernetes Service TTFT p90 | agentgateway on Kubernetes TTFT p90 |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 3 | 1,570 | 1,520 | 0.5 | 0.1 | 0.5 | 0.2 |
+| 10 | 5,113 | 4,739 | 0.5 | 0.1 | 1.0 | 0.2 |
+| 15 | 4,634 | 6,547 | 0.6 | 0.1 | 1.8 | 0.2 |
+| 20 | 6,182 | 10,853 | 2.5 | 0.1 | 36.0 | 0.3 |
+| 22 | 6,255 | 11,530 | 3.9 | 0.1 | 36.8 | 0.2 |
+| 25 | 6,044 | 11,959 | 6.7 | 0.1 | 41.2 | 0.2 |
+| 30 | 6,296 | 12,532 | 7.0 | 0.1 | 42.6 | 0.2 |
+| 35 | 6,145 | 12,279 | 7.6 | 0.1 | 45.2 | 0.2 |
+| 40 | 6,910 | 14,241 | 78.0 | 0.2 | 124.5 | 0.2 |
+| 43 | 6,858 | 14,216 | 80.2 | 0.2 | 129.2 | 0.2 |
+| 46 | 6,800 | 14,163 | 65.3 | 0.2 | 132.0 | 0.2 |
+| 49 | 6,780 | 14,205 | 55.1 | 0.2 | 131.3 | 0.2 |
+| 52 | 6,893 | 14,116 | 72.0 | 0.2 | 133.6 | 0.2 |
+| 55 | 6,865 | 14,024 | 56.4 | 0.2 | 133.6 | 0.2 |
+| 57 | 6,840 | 13,988 | 55.2 | 0.2 | 134.3 | 0.2 |
+| 60 | 6,879 | 13,999 | 62.9 | 0.2 | 135.6 | 0.2 |
+
+{{< /details >}}
+
+## Interpreting ITL
+
+At a rate of 60 requests/s, agentgateway has higher p50 ITL (49.6 ms versus
+30.3 ms for the Kubernetes Service), but higher throughput and lower time to
+first token. Read ITL together with TTFT and throughput to assess this tradeoff.
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/methodology.md" >}}
+
+## Evidence and reproduction
+
+The [published report](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/service-vs-agentgateway-gateway/README.md),
+charts, and tables on this page come from campaign `optimized-baseline-v0230-gateway-refresh-20260817`.
+
+- [Normalized metrics (CSV)](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/service-vs-agentgateway-gateway/metrics.csv)
+- [Campaign manifest](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-manifest.yaml)
+- [Campaign provenance](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-provenance.yaml)
+- [Benchmark runner and reproduction instructions](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/README.md)
+
+Use the campaign manifest's versions and configuration when reproducing these
+results.

assets/agw-docs/pages/agentgateway/benchmarking/latency-caveat.md

@@ -0,0 +1,4 @@
+> [!NOTE]
+> The published metrics record zero failed requests across all measured stages
+> for the Kubernetes Service baseline and both agentgateway modes. Each treatment
+> has only one repetition, so these results do not quantify run-to-run variability.

assets/agw-docs/pages/agentgateway/benchmarking/methodology.md

@@ -0,0 +1,18 @@
+## Methodology and metric definitions
+
+Each treatment ran independently with the same workload and model-server
+topology. The provenance records the execution times and shared configuration
+hashes. These reports compare each agentgateway deployment mode with the Kubernetes Service baseline.
+
+- **Requested rate (QPS)** is the load offered by the generator.
+- **Requests/s** is the achieved request throughput, which can be lower under load.
+- **Output tokens/s** measures output throughput; higher is better. The summary
+  reports each treatment's peak across the tested request rates.
+- **TTFT** (time to first token) includes waiting for the first response token;
+  lower is better. p50 is the median, and p90 is the 90th percentile.
+- **ITL** (inter-token latency) measures token cadence after the first token;
+  lower is better, but it must be considered together with TTFT and throughput.
+- **NTPOT** (normalized time per output token), used in the latency chart, is
+  end-to-end latency divided by output tokens.
+
+Warm-up stages whose requested rate repeats in the measured ladder are excluded.

assets/agw-docs/pages/agentgateway/benchmarking/results-notes.md

@@ -0,0 +1,12 @@
+Peak output throughput is the maximum number of tokens-per-second across the tested request rates.
+All other summary metrics are at a rate of 60 requests-per-second.
+See [Metric definitions](#methodology-and-metric-definitions) for more detail.
+
+Higher throughput and lower latency are better. A positive percentage indicates
+an increase, which is favorable for throughput but unfavorable for latency.
+
+At 10 requests/s, the Service baseline has slightly higher output throughput;
+both agentgateway modes pull ahead at 15 requests/s, and the baseline's TTFT
+rises sharply at 20 requests/s as its throughput begins to plateau. The large
+TTFT reductions at 60 requests/s therefore describe performance against an
+overloaded baseline, rather than a uniform improvement across all loads.

assets/agw-docs/pages/agentgateway/benchmarking/standalone.md

@@ -0,0 +1,91 @@
+Compare the performance of agentgateway in standalone mode versus a plain Kubernetes Service for inference routing.
+
+In standalone request scheduler mode, agentgateway runs as a sidecar proxy
+with the [Endpoint Picker Extension (EPP)](https://llm-d.ai/docs/architecture/core/router/epp) and communicates with it over localhost.
+This benchmark runs that standalone configuration on Kubernetes, without a full
+Gateway API stack.
+
+For setup instructions, see the [inference benchmark README](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/README.md).
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/environment.md" >}}
+
+## Results
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/results-notes.md" >}}
+
+| Metric | Kubernetes Service | agentgateway standalone | Change vs. baseline |
+| :--- | :--- | :--- | :--- |
+| Peak output tokens/s | 6,910 | 16,178 | +134.1% |
+| Achieved requests/s | 6.70 | 16.52 | +146.5% |
+| TTFT p50 (s) | 62.9 | 0.1 | −99.8% |
+| TTFT p90 (s) | 135.6 | 0.2 | −99.8% |
+| ITL p50 (ms) | 30.3 | 52.7 | +73.7% |
+
+### Token throughput
+
+The charts show input, output, and total token throughput in tokens per second
+across the tested request rates.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/standalone/throughput_vs_qps.png" alt="Input, output, and total token throughput versus requested QPS for agentgateway standalone and the Kubernetes Service baseline." >}}
+
+### Mean latency
+
+The charts show mean TTFT, ITL, and NTPOT in milliseconds. The summary table
+reports latency percentiles, with TTFT in seconds and ITL in milliseconds.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/standalone/latency_vs_qps.png" alt="Mean TTFT, ITL, and NTPOT in milliseconds versus requested QPS for agentgateway standalone and the Kubernetes Service baseline." >}}
+
+### Time to first token (p90)
+
+The chart shows p90 TTFT in milliseconds while the tables report it in seconds.
+
+{{< benchmark-chart src="img/benchmarks/optimized-baseline-v0230-gateway-refresh-20260817/standalone/ttft_p90_vs_qps.png" alt="Time to first token (p90) in milliseconds versus requested QPS for agentgateway standalone and the Kubernetes Service baseline." >}}
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/latency-caveat.md" >}}
+
+{{< details title="Per-rate breakdown" >}}
+
+Output throughput is in tokens/s (higher is better); TTFT is in seconds (lower
+is better).
+
+| Requests/s | Kubernetes Service Output | agentgateway standalone Output | Kubernetes Service TTFT p50 | agentgateway standalone TTFT p50 | Kubernetes Service TTFT p90 | agentgateway standalone TTFT p90 |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 3 | 1,570 | 1,694 | 0.5 | 0.1 | 0.5 | 0.1 |
+| 10 | 5,113 | 4,723 | 0.5 | 0.1 | 1.0 | 0.2 |
+| 15 | 4,634 | 6,480 | 0.6 | 0.1 | 1.8 | 0.2 |
+| 20 | 6,182 | 10,974 | 2.5 | 0.2 | 36.0 | 0.3 |
+| 22 | 6,255 | 11,831 | 3.9 | 0.1 | 36.8 | 0.1 |
+| 25 | 6,044 | 12,227 | 6.7 | 0.1 | 41.2 | 0.1 |
+| 30 | 6,296 | 12,923 | 7.0 | 0.1 | 42.6 | 0.1 |
+| 35 | 6,145 | 13,032 | 7.6 | 0.1 | 45.2 | 0.1 |
+| 40 | 6,910 | 15,964 | 78.0 | 0.1 | 124.5 | 0.2 |
+| 43 | 6,858 | 15,781 | 80.2 | 0.1 | 129.2 | 0.2 |
+| 46 | 6,800 | 15,566 | 65.3 | 0.2 | 132.0 | 0.2 |
+| 49 | 6,780 | 15,834 | 55.1 | 0.1 | 131.3 | 0.2 |
+| 52 | 6,893 | 16,031 | 72.0 | 0.1 | 133.6 | 0.2 |
+| 55 | 6,865 | 15,766 | 56.4 | 0.1 | 133.6 | 0.2 |
+| 57 | 6,840 | 15,829 | 55.2 | 0.1 | 134.3 | 0.2 |
+| 60 | 6,879 | 16,178 | 62.9 | 0.1 | 135.6 | 0.2 |
+
+{{< /details >}}
+
+## Interpreting ITL
+
+At a rate of 60 requests/s, agentgateway has higher p50 ITL (52.7 ms versus
+30.3 ms for the Kubernetes Service), but higher throughput and lower time to
+first token. Read ITL together with TTFT and throughput to assess this tradeoff.
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/methodology.md" >}}
+
+## Evidence and reproduction
+
+The [published report](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/service-vs-agentgateway-standalone/README.md),
+charts, and tables on this page come from campaign `optimized-baseline-v0230-gateway-refresh-20260817`.
+
+- [Normalized metrics (CSV)](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/service-vs-agentgateway-standalone/metrics.csv)
+- [Campaign manifest](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-manifest.yaml)
+- [Campaign provenance](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-provenance.yaml)
+- [Benchmark runner and reproduction instructions](https://github.com/agentgateway/benchmarks/blob/522fc04a595faad40e6cb070d0f2275e1314f1f9/inference/README.md)
+
+Use the campaign manifest's versions and configuration when reproducing these
+results.

assets/agw-docs/pages/agentgateway/inference.md

@@ -1,6 +1,4 @@
-Use {{< reuse "agw-docs/snippets/kgateway.md" >}} with the Kubernetes Gateway
-API Inference Extension to route requests to Large Language Model (LLM)
-workloads in your Kubernetes environment.
+Use {{< reuse "agw-docs/snippets/kgateway.md" >}} with the [Kubernetes Gateway API Inference Extension](https://gateway-api-inference-extension.sigs.k8s.io/) and [llm-d Router](https://github.com/llm-d/llm-d-router) to route inference requests to Large Language Model (LLM) workloads in your Kubernetes environment.
 
 The Gateway API Inference Extension defines the `InferencePool` API and the
 protocol between gateways and endpoint pickers. The

content/docs/kubernetes/latest/documentation/llm/inference/benchmarking.md

@@ -0,0 +1,13 @@
+---
+title: Benchmarking
+weight: 30
+description: Explore performance benchmarks, methodology, and reproduction guidance for agentgateway inference routing.
+aliases:
+  - /docs/kubernetes/latest/documentation/llm/benchmarking/
+test: skip
+---
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/kubernetes.md" >}}
+
+For the other deployment mode, see the
+[standalone request scheduler benchmarks](/docs/standalone/latest/documentation/llm/inference/benchmarking/).

content/docs/kubernetes/main/documentation/llm/inference/benchmarking.md

@@ -0,0 +1,13 @@
+---
+title: Benchmarking
+weight: 30
+description: Explore performance benchmarks, methodology, and reproduction guidance for agentgateway inference routing.
+aliases:
+  - /docs/kubernetes/main/documentation/llm/benchmarking/
+test: skip
+---
+
+{{< reuse "agw-docs/pages/agentgateway/benchmarking/kubernetes.md" >}}
+
+For the other deployment mode, see the
+[standalone request scheduler benchmarks](/docs/standalone/main/documentation/llm/inference/benchmarking/).