Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions assets/agw-docs/snippets/model-catalog-json-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,31 @@ Key points:
{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}- `perPage` is **USD per page** for document or OCR models that report page usage, not USD per 1,000,000 tokens.{{< /version >}}
- If a rate is omitted, that token type is not priced for the model.
- `tiers[]` is optional. Each tier selects alternate `rates` when the request context length is **over** the tier's `contextOver` value. Tiers must be ordered by strictly increasing `contextOver`.
{{< version exclude-if="1.3.x,1.4.x,1.5.x,1.6.x" >}}- To select rates by the service tier that the provider served, set `serviceTier` to `standard`, `flex`, `priority`, or `reserved`. Omit `serviceTier` to match any served tier. Omit `contextOver` to make the rule apply from zero context tokens. Within one `serviceTier`, order rules by increasing `contextOver`.{{< /version >}}
{{% version exclude-if="1.3.x,1.2.x,1.1.x,1.0.x,2.2.x" %}}
- `tags[]` is optional. Tags describe a model instead of pricing it, so a model entry can carry tags with no rates at all.{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,2.2.x" >}} For more information, see [Model tags](#model-tags).{{< /version >}}
{{% /version %}}

{{% version exclude-if="1.3.x,1.4.x,1.5.x,1.6.x" %}}
The following example adds a Flex tier that applies from zero context tokens, plus a second Flex tier for requests over 200,000 context tokens:

```json
{
"tiers": [
{
"serviceTier": "flex",
"rates": { "input": "1.0", "output": "6", "cacheRead": "0.1" }
},
{
"contextOver": 200000,
"serviceTier": "flex",
"rates": { "input": "2.0", "output": "9", "cacheRead": "0.2" }
}
]
}
```
{{% /version %}}

The following minimal example prices two OpenAI models and one tiered Gemini model:

```json
Expand Down
5 changes: 3 additions & 2 deletions assets/agw-docs/standalone/cost-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ For general LLM telemetry setup, see [Observe traffic]({{< link-hextra path="/do
include/exclude as a complete, matching pair — shortening either to "1.5.x"
would also match 1.4.x and older, and adding "main" would need an edit every
release. -->
Use `agctl {{< reuse "agw-docs/versions/agctl-catalog-cmd.md" >}} import` to generate a catalog file. {{< version include-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}The command reads from a supported pricing source, and the default source is `models.dev`.{{< /version >}}{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}The `--source` flag takes a comma-separated list, and the sources merge in the order that you list them, so a later source overlays an earlier one. The default is `models.dev,aws-bedrock-mantle`, which prices every provider that the proxy supports and then tags the Amazon Bedrock models.{{< /version >}}
Use `agctl {{< reuse "agw-docs/versions/agctl-catalog-cmd.md" >}} import` to generate a catalog file. {{< version include-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}The command reads from a supported pricing source, and the default source is `models.dev`.{{< /version >}}{{< version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" >}}The `--source` flag takes a comma-separated list, and the sources merge in the order that you list them, so a later source overlays an earlier one. The default is `models.dev,aws-bedrock-mantle`, which prices every provider that the proxy supports and then tags the Amazon Bedrock models.{{< /version >}}{{< version exclude-if="1.5.x,1.6.x" >}}The default also includes `vertex-pricing`, which adds Vertex AI service-tier prices.{{< /version >}}

{{% version exclude-if="1.0.x,1.1.x,1.2.x,1.3.x,1.4.x,1.5.x,2.2.x" %}}
A source is a catalog to import from, not an LLM provider: each source covers one or more providers and contributes rates, tags, or both. Use `--providers` to import a subset of the providers that a source covers.
Expand All @@ -73,7 +73,8 @@ A source is a catalog to import from, not an LLM provider: each source covers on
|--------|---------------------|
| `models.dev` | Rates for every provider that the proxy supports, from [models.dev](https://models.dev). |
| `aws-bedrock-mantle` | Tags for Amazon Bedrock models only, read from the AWS model cards. This source contributes no rates. The tags record which endpoint serves a model, `runtime` or `mantle`, and which request formats the Mantle endpoint accepts. |
| `github` | The curated catalog that the agentgateway project publishes at [agentgateway.dev/model-catalog](https://agentgateway.dev/model-catalog), which covers the models that the agentgateway project tracks rather than everything that models.dev lists. Not imported by default. |
| `github` | The curated catalog that the agentgateway project publishes at [agentgateway.dev/model-catalog](https://agentgateway.dev/model-catalog), which covers the models that the agentgateway project tracks rather than everything that models.dev lists. Not imported by default. |{{< version exclude-if="1.5.x,1.6.x" >}}
| `vertex-pricing` | Service-tier rates for Vertex AI Gemini models, read from Google Cloud pricing. This source augments models that earlier sources already added. |{{< /version >}}
{{% /version %}}

```sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,13 @@ Prepare a catalog by creating your own JSON file or using the `agctl catalog imp

Use `agctl catalog import` to generate a catalog JSON file, then load it into a ConfigMap.

1. Generate a catalog from one or more supported sources. The `--source` flag takes a comma-separated list, and the sources merge in the order that you list them, so a later source overlays an earlier one. By default, `agctl catalog import` imports `models.dev,aws-bedrock-mantle`, which prices every provider that the proxy supports and then tags the Amazon Bedrock models. A source is a catalog to import from, not an LLM provider: each source covers one or more providers and contributes rates, tags, or both. To import only a subset of the providers that a source covers, pass a comma-separated list to `--providers`.
1. Generate a catalog from one or more supported sources. The `--source` flag takes a comma-separated list, and the sources merge in the order that you list them, so a later source overlays an earlier one. By default, `agctl catalog import` imports `models.dev,aws-bedrock-mantle,vertex-pricing`, which prices every provider that the proxy supports, tags the Amazon Bedrock models, and adds Vertex AI service-tier prices. A source is a catalog to import from, not an LLM provider: each source covers one or more providers and contributes rates, tags, or both. To import only a subset of the providers that a source covers, pass a comma-separated list to `--providers`.

| Source | What it contributes |
|--------|---------------------|
| `models.dev` | Rates for every provider that the proxy supports, from [models.dev](https://models.dev). |
| `aws-bedrock-mantle` | Tags for Amazon Bedrock models only, read from the AWS model cards. This source contributes no rates. The tags record which endpoint serves a model, `runtime` or `mantle`, and which request formats the Mantle endpoint accepts. |
| `vertex-pricing` | Service-tier rates for Vertex AI Gemini models, read from Google Cloud pricing. This source augments models that earlier sources already added. |
| `github` | The curated catalog that the agentgateway project publishes at [agentgateway.dev/model-catalog](https://agentgateway.dev/model-catalog), which covers the models that the agentgateway project tracks rather than everything that models.dev lists. Not imported by default. |

```sh
Expand Down
6 changes: 6 additions & 0 deletions content/docs/kubernetes/main/release-notes/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ Review the release notes for agentgateway on Kubernetes.

- **Conditional virtual MCP targets**: Set `spec.mcp.targets[].condition` on an {{< reuse "agw-docs/snippets/backend.md" >}} to a CEL expression to select which targets take part in a request before agentgateway initializes or contacts them. A backend whose only target is a `static` target cannot set a condition. For more information, see [Select targets with conditions]({{< link-hextra path="/documentation/mcp/virtual/#target-conditions" >}}).

### LLM {#v17-llm}

<!-- ref: https://github.com/agentgateway/agentgateway/pull/3661 -->

- **Service-tier pricing in the model catalog**: The model catalog can price requests by normalized service tier, such as `standard`, `flex`, `priority`, or `reserved`. The `agctl catalog import` default sources now include `vertex-pricing`, which adds Vertex AI Gemini service-tier rates.

## 🐛 Fixes {#v17-fixes}

<!-- ref: https://github.com/agentgateway/agentgateway/pull/3744 -->
Expand Down
6 changes: 6 additions & 0 deletions content/docs/standalone/main/release-notes/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,12 @@ Review the release notes for agentgateway standalone.

- **Standalone Helm session key Secret**: Set `session.secretName` to load the AES-256-GCM session key from an existing Kubernetes Secret. The chart passes the Secret's `key` entry to the proxy as the `SESSION_KEY` environment variable, which takes precedence over `config.session.key`. For more information, see [Use an existing session key Secret]({{< link-hextra path="/documentation/setup/install/helm/#helm-session-secret" >}}).

### LLM {#v17-llm}

<!-- ref: https://github.com/agentgateway/agentgateway/pull/3661 -->

- **Service-tier pricing in the model catalog**: The model catalog can price requests by normalized service tier, such as `standard`, `flex`, `priority`, or `reserved`. The `agctl catalog import` default sources now include `vertex-pricing`, which adds Vertex AI Gemini service-tier rates.

## 🐛 Fixes {#v17-fixes}

<!-- ref: https://github.com/agentgateway/agentgateway/pull/3744 -->
Expand Down