Skip to content

Declare LLM usage extraction locations in LlmProviderTemplate #3275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Irash-Perera wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Declare LLM usage extraction locations in LlmProviderTemplate #3275

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
a6aca48
Add usage extraction fields to LLM provider templates
Irash-Perera Aug 20, 2026
58915c1
Preserve the chunk index when a policy rewrites the body
Irash-Perera Aug 20, 2026
1fa9d71
Clean up code formatting in chain executor and tests
Irash-Perera Aug 20, 2026
a41d4b4
Expose providerFields in the operator API and CRDs
Irash-Perera Aug 20, 2026
4f6684b
Add cachedTokens and reasoningTokens to OpenAI provider template
Irash-Perera Aug 20, 2026
deb6ce4
Add validation for providerFields in LLMProviderTemplate
Irash-Perera Aug 20, 2026
e607129
Merge remote-tracking branch 'upstream/main' into feat/usage-extracti…
Irash-Perera Aug 21, 2026
94a7387
Merge branch 'wso2:main' into feat/usage-extraction-core
Irash-Perera Aug 21, 2026
606663d
Merge branch 'wso2:main' into feat/usage-extraction-core
Irash-Perera Aug 23, 2026
a67504b
Read Anthropic thinking tokens
Irash-Perera Aug 23, 2026
806f5a5
Merge branch 'feat/usage-extraction-core' of https://github.com/Irash…
Irash-Perera Aug 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/rest-apis/gateway/llm-provider-template-management.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,11 +254,24 @@ Status Code **200**
|»»»» promptTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»» location|string|true|none|Where to find the token information|
|»»»»» identifier|string|true|none|JSONPath expression or header name to identify the token value|
|»»»»» fallbackIdentifiers|[string]|false|none|Additional locations to try when the primary identifier yields no<br>value, in order. Used where one provider returns several response<br>shapes on the same path.|
|»»»»» valueMap|object|false|none|Maps values this provider reports onto the vocabulary the gateway<br>bills on. A reported value that is not a key is used unchanged.<br>Used where providers name the same billing tier differently.|
|»»»»»» **additionalProperties**|string|false|none|none|
|»»»» completionTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» totalTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» remainingTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» requestModel|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» responseModel|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» cachedTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» cacheWriteTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» cacheWrite1hTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» reasoningTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» audioInputTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» audioOutputTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» serviceTier|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» providerFields|object|false|none|Named locations a provider-specific calculator needs but the closed<br>vocabulary does not cover. The value at each location is handed to<br>the calculator as-is; only its position is declared here.|
|»»»»» **additionalProperties**|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»» cacheAccounting|string|false|none|Whether the cached token count reported by the provider is already<br>part of the input token total, or additional to it. One of<br>'inclusive' or 'additive'. Defaults to inclusive when omitted.|
|»»»» resourceMappings|[LLMProviderTemplateResourceMappings](schemas.md#schemallmprovidertemplateresourcemappings)|false|none|none|
|»»»»» resources|[[LLMProviderTemplateResourceMapping](schemas.md#schemallmprovidertemplateresourcemapping)]|false|none|none|
|»»»»»» resource|string|true|none|Resource path pattern for this mapping|
Expand All @@ -268,6 +281,14 @@ Status Code **200**
|»»»»»» remainingTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» requestModel|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» responseModel|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» cachedTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» cacheWriteTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» cacheWrite1hTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» reasoningTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» audioInputTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» audioOutputTokens|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» serviceTier|[ExtractionIdentifier](schemas.md#schemaextractionidentifier)|false|none|none|
|»»»»»» cacheAccounting|string|false|none|Whether the cached token count reported by the provider is already<br>part of the input token total, or additional to it. One of<br>'inclusive' or 'additive'. Overrides the template-level setting for<br>this resource; when omitted, the template-level value is inherited.|

*and*

Expand Down
Loading
Loading