Repository navigation
Usage and cost: let the OTLP attribute names be configured, so gateways other than agentgateway can report #187
Description
Summary
The usage and cost feature from #160 only works with agentgateway: the controller's OTLP receiver reads a fixed set of attribute names, several of which are agentgateway's own. Could the receiver take its attribute names from configuration, with today's names as the defaults? That would let any gateway that can emit OTLP logs report usage, without changing anything for existing agentgateway users.
Background
agentdesktop itself is gateway-neutral. llmGateway.url is a plain URL, and we run enrolled devices against LiteLLM that way. The usage feature is the one part that is not. In crates/usage/src/otlp.rs the receiver reads these attributes:
| Used for | Attribute (fixed today) | agentgateway-specific? |
|---|---|---|
| provider | gen_ai.provider.name |
no (OTel GenAI semconv) |
| model | gen_ai.request.model, gen_ai.response.model |
no |
| tokens | gen_ai.usage.input_tokens, gen_ai.usage.output_tokens |
no |
| cache tokens | gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens |
no |
| cost | agw.ai.usage.cost.total |
yes |
| status / error | http.response.status_code, error |
no |
| user, device, tool, billing | agentdesktop.subject, .device, .client, .billing (added from the controller JWT) |
yes, needs the controller JWT in the request path |
| fallbacks | jwt.sub, user_agent.original |
agentgateway log fields |
A record without gen_ai.provider.name is dropped as "not an LLM request", so a gateway that names the provider differently reports nothing, with no error.
Why
A gateway that already follows the OTel GenAI conventions is most of the way there. From LiteLLM's OpenTelemetry docs (I have not run this against a live instance), it sets gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, so those would already match. The gaps are:
- provider:
gen_ai.systemby default, andgen_ai.provider.nameonly in semconv mode - cost:
gen_ai.cost.total_costrather thanagw.ai.usage.cost.total - identity: LiteLLM puts the caller in
metadata.user_api_key_*style attributes, and there is no device, because devices authenticate to the gateway with their own IdP token rather than a controller JWT - signal type: spans by default, with logs opt-in. The controller registers only the OTLP logs service.
Proposal
-
Configurable attribute names in the controller config, for example:
usage: tokenFile: /etc/agentdesktop/usage-token attributes: # all optional; defaults are today's names provider: gen_ai.system cost: gen_ai.cost.total_cost subject: metadata.user_api_key_user_id
Keys left out keep their current value, so existing configurations behave exactly as now.
-
Identity without a controller JWT. When
deviceis not present, store the record against the subject only (it already tolerates an empty device). Per-user numbers would then work for any gateway, and per-device ones only where the gateway can supply a device ID. -
(Optional, larger) Accept OTLP traces as well as logs, since many gateways export spans and not log events. I would treat this as a separate step and understand if it is out of scope.
Open questions
- Is a configurable mapping something you would want in the controller, or do you prefer to keep the receiver tied to agentgateway's record format?
- If yes: is a flat
usage.attributesmap the right shape, or would you rather have named presets (for examplepreset: litellm)? - Should the cost key also be allowed to carry a unit, since other gateways report USD in different forms?
Offer
I am happy to send a PR for point 1 and 2. I wanted to check the direction first so I do not guess at the config shape. I have not yet confirmed what a LiteLLM log event carries on the wire, so I would test against a real instance before claiming support for it.
cc @peterj
Activity
devmittal02 commentedon Oct 22, 2026
@peterj thoughts on this ?