Add top-level README and publish H100 benchmark reports

README.md

@@ -1,12 +1,48 @@
 # Benchmarks
 
-Community benchmarks for agentgateway.
+Community benchmarks for [agentgateway](https://github.com/agentgateway/agentgateway).
 
-- [`inference/`](inference/README.md) — inference benchmark execution, results, and publishable reports (plain Kubernetes Service vs. agentgateway standalone vs. agentgateway on Kubernetes, run across GKE campaigns)
+This repo holds the tooling and published results for measuring how agentgateway
+performs as an inference gateway - both as EPP's standalone sidecar and as a
+Gateway API data plane - compared to routing traffic through a plain Kubernetes
+Service with no gateway at all.
 
-PR #1 intentionally migrates the inference benchmark tooling only. Published
-benchmark reports and their supporting evidence will follow separately after
-the tooling migration is complete.
+## Layout
 
-Automated repository checks cover syntax and unit tests only; they do not
-provision infrastructure or execute benchmark campaigns.
+- [`inference/`](inference/README.md) - the benchmark runner itself: campaign-based
+  execution comparing `service`, `agentgateway-standalone`, and
+  `agentgateway-gateway` treatments, with automated GKE provisioning/teardown and
+  Markdown/PNG/CSV report generation.
+- [`inference/reports/`](inference/reports/README.md) - curated, published benchmark
+  results with their campaign manifests and provenance, safe to link to directly.
+
+## Quick start
+
+Run the `service` treatment locally on Kind:
+
+```bash
+make kind-create
+BENCHMARK_TREATMENT=service BENCHMARK_CAMPAIGN_ID=local-sim make benchmark
+```
+
+The `service` treatment runs fine on a laptop-sized Kind cluster.
+`agentgateway-standalone` and `agentgateway-gateway` currently need more CPU/memory
+than most laptops provide for the router pod - see
+[#2](https://github.com/agentgateway/benchmarks/issues/2).
+
+See [`inference/README.md`](inference/README.md) for the full set of treatments,
+GKE campaign instructions, and configuration reference.
+
+## Current results
+
+The most recent published campaign
+([`optimized-baseline-v0230-gateway-refresh-20260817`](inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/README.md))
+ran Qwen/Qwen3-32B on 16x H100 GPUs across 8 vLLM model servers. At the top of the
+request-rate ladder, agentgateway more than doubles peak throughput and cuts TTFT
+p90 by over 99% versus a plain Kubernetes Service doing round-robin, because
+round-robin has no way to know which backend pod is already overloaded and
+agentgateway's EPP-based routing does.
+
+Automated repository checks (`inference/suites/*/tests`, `inference/reporting/tests`)
+cover syntax and unit tests only - they don't provision infrastructure or execute
+benchmark campaigns.

inference/README.md

@@ -5,10 +5,6 @@ retains their native evidence in a shared campaign, and generates publishable
 Markdown, PNG, and CSV comparisons. Prism-compatible Benchmark Report v0.2
 documents remain available in each treatment's evidence bundle.
 
-PR #1 intentionally migrates tooling only. Curated reports from the original
-agentgateway repository change are not part of this migration and will be
-published separately with their supporting evidence.
-
 ## Layout
 
 ```text

inference/reports/README.md

@@ -1,14 +1,12 @@
-# Published inference reports
+# Published inference benchmark reports
 
-PR #1 intentionally migrates the inference benchmark tooling only. Published
-reports and their supporting evidence will be added separately after the
-tooling migration is complete.
-
-Curated reports belong under a suite and profile-specific path, for example:
+Curated benchmark reports may be committed below this directory using:
 
 ```text
-inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/
+<suite>/<benchmark-profile>/<campaign-id>/
 ```
 
-Generated local reports remain under the ignored campaign directory in
-`inference/results/` until they are reviewed and selected for publication.
+Raw and per-request evidence stays in the ignored `../results` tree or durable
+external storage. Every published report must retain its campaign manifest,
+normalized metrics, and provenance. Link native evidence only when it has a
+durable archive URL; do not commit links into the ignored local results tree.

inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/README.md

@@ -0,0 +1,8 @@
+# Inference Benchmark Campaign
+
+Campaign: `optimized-baseline-v0230-gateway-refresh-20260817`
+
+## Comparisons
+
+- [service vs agentgateway-standalone](service-vs-agentgateway-standalone/README.md)
+- [service vs agentgateway-gateway](service-vs-agentgateway-gateway/README.md)

inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-manifest.yaml

@@ -0,0 +1,135 @@
+schema_version: 1
+campaign_id: optimized-baseline-v0230-gateway-refresh-20260817
+suite: llm-d-benchmark
+identity:
+  scenario: optimized-baseline
+  cluster_provider: gke
+  accelerator_type: gpu
+  accelerator_model: h100
+  backend_type: vllm
+  model: Qwen/Qwen3-32B
+  replicas: 8
+  tensor_parallelism: 2
+  routing_policy: optimized-baseline
+  workload_variant: upstream
+  harness: inference-perf
+  workload: upstream-optimized-baseline.yaml
+  workload_sha256: b2ad162508976d18ed7ef21a8a0f8ed0431c92039e9c9b0ee43a2452882e63f7
+  scenario_source_sha256: 4ae7851be32a23cfd2272fe30d40032541261921e0cb18226537fa1cffb8246e
+  shared_config_sha256: d6a29f6eaa1360c24ad91c9f94bb914d33d28ec0ee6b146b11ba77f88ebcd2b4
+  llm_d_benchmark_ref: main
+  reference_profile: optimized-baseline-qwen3-32b-h100-v0.9
+  endpoint_path: internal
+  storage:
+    model:
+      profile: high-throughput-shared
+      strategy: filestore
+      class: premium-rwx
+      size: 2560Gi
+    workload:
+      profile: shared
+      class: standard-rwx
+      size: 100Gi
+  runtime_metrics:
+    enabled: true
+    interval_seconds: 5
+    gke_monitoring: auto
+  result_collection:
+    fast_collect: true
+  gpu_release:
+    policy: after-load
+    node_pool: gpu-h100
+treatments:
+  service:
+    repetitions:
+      '1':
+        config_sha256: df9d8e0b527a6f1b88fdeedf6c0d34a5ba7c925ee69834493b3fefa223e46158
+        gateway:
+          implementation: none
+          mode: none
+          image: not-applicable
+          agentgateway_version: not-applicable
+        storage:
+          model:
+            profile: high-throughput-shared
+            strategy: filestore
+            class: premium-rwx
+            size: 2560Gi
+          workload:
+            profile: shared
+            class: standard-rwx
+            size: 100Gi
+        router_chart_version: v0.9.0
+        router_chart_digest: not-applicable
+        standalone_invariant_sha256: not-applicable
+        runtime_metrics:
+          enabled: true
+          interval_seconds: 5
+          gke_monitoring: auto
+        result_collection:
+          fast_collect: true
+        gpu_release:
+          policy: after-load
+          node_pool: gpu-h100
+  agentgateway-standalone:
+    repetitions:
+      '1':
+        config_sha256: a3b689e6b16f29c395bc32784ca4cd24a1f3ea25e061921fc1905ee29250864c
+        gateway:
+          implementation: agentgateway
+          mode: standalone
+          image: cr.agentgateway.dev/agentgateway:v1.4.1
+          agentgateway_version: v1.4.1
+        storage:
+          model:
+            profile: high-throughput-shared
+            strategy: filestore
+            class: premium-rwx
+            size: 2560Gi
+          workload:
+            profile: shared
+            class: standard-rwx
+            size: 100Gi
+        router_chart_version: v0.9.0
+        router_chart_digest: sha256:8150c84045d59e6e91991daf810322140e116d5e02d244d96cbfc2b3000825cd
+        standalone_invariant_sha256: d27aa3658a2688014b3e7d88d84a9e22264b1b9eda9d1b60838bfa740d5886ee
+        runtime_metrics:
+          enabled: true
+          interval_seconds: 5
+          gke_monitoring: auto
+        result_collection:
+          fast_collect: true
+        gpu_release:
+          policy: after-load
+          node_pool: gpu-h100
+  agentgateway-gateway:
+    repetitions:
+      '1':
+        config_sha256: 63ee4f0ab209f078f7c81e60cf94ab6a1bb7bcf47a3632c48e6d9b817c4f0c6c
+        gateway:
+          implementation: agentgateway
+          mode: gateway
+          image: cr.agentgateway.dev/agentgateway:v1.4.1
+          agentgateway_version: v1.4.1
+        storage:
+          model:
+            profile: high-throughput-shared
+            strategy: filestore
+            class: premium-rwx
+            size: 2560Gi
+          workload:
+            profile: shared
+            class: standard-rwx
+            size: 100Gi
+        router_chart_version: v0.9.0
+        router_chart_digest: not-applicable
+        standalone_invariant_sha256: not-applicable
+        runtime_metrics:
+          enabled: true
+          interval_seconds: 5
+          gke_monitoring: auto
+        result_collection:
+          fast_collect: true
+        gpu_release:
+          policy: after-load
+          node_pool: gpu-h100

inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/campaign-provenance.yaml

@@ -0,0 +1,21 @@
+schema_version: 1
+campaign_id: optimized-baseline-v0230-gateway-refresh-20260817
+execution_model: independent-treatment-runs
+treatments:
+  service:
+    status: complete
+    harness_start: "2026-08-17T16:46:26+00:00"
+    harness_stop: "2026-08-17T18:10:01+00:00"
+  agentgateway-standalone:
+    status: complete
+    harness_start: "2026-08-17T21:10:17+00:00"
+    harness_stop: "2026-08-17T22:09:12+00:00"
+  agentgateway-gateway:
+    status: complete
+    harness_start: "2026-08-18T14:45:27+00:00"
+    harness_stop: "2026-08-18T15:45:43+00:00"
+shared_identity:
+  workload_sha256: b2ad162508976d18ed7ef21a8a0f8ed0431c92039e9c9b0ee43a2452882e63f7
+  scenario_source_sha256: 4ae7851be32a23cfd2272fe30d40032541261921e0cb18226537fa1cffb8246e
+  shared_config_sha256: d6a29f6eaa1360c24ad91c9f94bb914d33d28ec0ee6b146b11ba77f88ebcd2b4
+  inference_perf_revision: e28d9a0bf5cefa743910b73057b3b686f0a94b98

inference/reports/llm-d-benchmark/optimized-baseline-qwen3-32b-h100/optimized-baseline-v0230-gateway-refresh-20260817/service-vs-agentgateway-gateway/README.md

@@ -0,0 +1,79 @@
+# Benchmark Report
+
+The benchmark runs model Qwen/Qwen3-32B on 16 × H100 GPUs, distributed across 8 vLLM model servers (2 GPUs per server with TP=2). Workload is `upstream-optimized-baseline.yaml` driven across the configured request-rate ladder.
+
+> [!NOTE]
+> These results use the `optimized-baseline` routing policy. Latency is reported as median (p50) in the summary; NTPOT is end-to-end latency divided by output tokens. Missing latency histograms are shown as `n/a` and dropped from latency lines. Both treatments use the same model-server topology and workload.
+
+## Comparing agentgateway on Kubernetes to a Simple Kubernetes Service (vLLM)
+
+**agentgateway mode:** In Kubernetes Gateway API mode, agentgateway is the Gateway data plane; an HTTPRoute targets an InferencePool, whose EPP selects the model-server endpoint. [Learn more about Kubernetes inference routing](https://agentgateway.dev/docs/kubernetes/latest/llm/inference/inference-routing/).
+
+Graphs below compare agentgateway on Kubernetes routing to a stock Kubernetes Service that round-robins requests across the same 8 vLLM pods (no EPP, no scoring). Tables abbreviate this baseline as `k8s service (RR)`, where RR means round-robin.
+
+<img src="./throughput_vs_qps.png" width="900" alt="Throughput vs QPS">
+<img src="./latency_vs_qps.png" width="900" alt="Latency vs QPS">
+<img src="./ttft_p90_vs_qps.png" width="900" alt="TTFT p90 vs QPS">
+Summary (throughput is peak sustained; latencies at the top of the ladder, rate 60):
+
+| Metric | k8s service (RR) | agentgateway on Kubernetes | Δ% vs k8s |
+| :--- | :--- | :--- | :--- |
+| Peak output tokens/s | 6,910 | 14,241 | +106.1% |
+| Requests/sec (@ rate 60) | 6.70 | 13.96 | +108.3% |
+| TTFT p50 (s) | 62.9 | 0.2 | −99.7% |
+| TTFT p90 (s) | 135.6 | 0.2 | −99.8% |
+| ITL p50 (ms) | 30.3 | 49.6 | +63.7% |
+
+<details>
+<summary><b><i>Click</i></b> to view the per-rate breakdown across the full ladder</summary>
+
+Output tokens/sec — higher is better; TTFT in seconds — lower is better. `n/a` means the stage had no successful-request latency histogram.
+
+| Rate | k8s service (RR) Output | agentgateway on Kubernetes Output | k8s service (RR) TTFT p50 | agentgateway on Kubernetes TTFT p50 | k8s service (RR) TTFT p90 | agentgateway on Kubernetes TTFT p90 |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 3 | 1,570 | 1,520 | 0.5 | 0.1 | 0.5 | 0.2 |
+| 10 | 5,113 | 4,739 | 0.5 | 0.1 | 1.0 | 0.2 |
+| 15 | 4,634 | 6,547 | 0.6 | 0.1 | 1.8 | 0.2 |
+| 20 | 6,182 | 10,853 | 2.5 | 0.1 | 36.0 | 0.3 |
+| 22 | 6,255 | 11,530 | 3.9 | 0.1 | 36.8 | 0.2 |
+| 25 | 6,044 | 11,959 | 6.7 | 0.1 | 41.2 | 0.2 |
+| 30 | 6,296 | 12,532 | 7.0 | 0.1 | 42.6 | 0.2 |
+| 35 | 6,145 | 12,279 | 7.6 | 0.1 | 45.2 | 0.2 |
+| 40 | 6,910 | 14,241 | 78.0 | 0.2 | 124.5 | 0.2 |
+| 43 | 6,858 | 14,216 | 80.2 | 0.2 | 129.2 | 0.2 |
+| 46 | 6,800 | 14,163 | 65.3 | 0.2 | 132.0 | 0.2 |
+| 49 | 6,780 | 14,205 | 55.1 | 0.2 | 131.3 | 0.2 |
+| 52 | 6,893 | 14,116 | 72.0 | 0.2 | 133.6 | 0.2 |
+| 55 | 6,865 | 14,024 | 56.4 | 0.2 | 133.6 | 0.2 |
+| 57 | 6,840 | 13,988 | 55.2 | 0.2 | 134.3 | 0.2 |
+| 60 | 6,879 | 13,999 | 62.9 | 0.2 | 135.6 | 0.2 |
+</details>
+
+Latency must be interpreted together with failures. A treatment can show deceptively low latency under overload when slower requests time out and are excluded from successful-request latency distributions. Failure counts and rates are retained in `metrics.csv`.
+
+## Interpreting ITL
+
+At rate 60, k8s service (RR) has a lower p50 ITL (30.3 ms) than agentgateway on Kubernetes (49.6 ms). ITL measures token cadence only after the first response token; time waiting to enter prefill or decode appears in TTFT instead. Lower ITL therefore does not by itself mean better overall inference latency.
+
+Here, k8s service (RR) peaks at 6,910 output tokens/s and has 62.9 s TTFT p50, while agentgateway on Kubernetes reaches 14,241 output tokens/s with 0.2 s TTFT p50. This is consistent with the Service queueing longer while the routed treatment keeps more work decoding; vLLM continuous batching can trade higher per-stream ITL for greater throughput and lower TTFT. At 3 QPS, ITL is 16.1 ms versus 15.6 ms, indicating that the larger high-load gap is not a fixed proxy penalty.
+
+## Configuration
+
+- Campaign: `optimized-baseline-v0230-gateway-refresh-20260817`
+- Suite: `llm-d-benchmark`
+- Scenario: `optimized-baseline`
+- Model: `Qwen/Qwen3-32B`
+- Backend: `vllm`
+- Backend image/version: `docker.io/vllm/vllm-openai:v0.23.0`
+- Accelerator: `h100`
+- Model servers: 8 with TP=2
+- Workload: `upstream-optimized-baseline.yaml` (`upstream`)
+- Repetitions per treatment: 1
+- Candidate components: agentgateway cr.agentgateway.dev/agentgateway:v1.4.1
+
+Warm-up stages whose requested rate is repeated in the measured ladder are excluded. With multiple repetitions, each table and graph point is the median at that requested QPS.
+
+## Evidence
+
+- [Campaign manifest](../campaign-manifest.yaml)
+- [Normalized metrics](metrics.csv)