Skip to content

Usage and cost: let the OTLP attribute names be configured, so gateways other than agentgateway can report #187

Open

Description

@devmittal02

Summary

The usage and cost feature from #160 only works with agentgateway: the controller's OTLP receiver reads a fixed set of attribute names, several of which are agentgateway's own. Could the receiver take its attribute names from configuration, with today's names as the defaults? That would let any gateway that can emit OTLP logs report usage, without changing anything for existing agentgateway users.

Background

agentdesktop itself is gateway-neutral. llmGateway.url is a plain URL, and we run enrolled devices against LiteLLM that way. The usage feature is the one part that is not. In crates/usage/src/otlp.rs the receiver reads these attributes:

Used for Attribute (fixed today) agentgateway-specific?
provider gen_ai.provider.name no (OTel GenAI semconv)
model gen_ai.request.model, gen_ai.response.model no
tokens gen_ai.usage.input_tokens, gen_ai.usage.output_tokens no
cache tokens gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens no
cost agw.ai.usage.cost.total yes
status / error http.response.status_code, error no
user, device, tool, billing agentdesktop.subject, .device, .client, .billing (added from the controller JWT) yes, needs the controller JWT in the request path
fallbacks jwt.sub, user_agent.original agentgateway log fields

A record without gen_ai.provider.name is dropped as "not an LLM request", so a gateway that names the provider differently reports nothing, with no error.

Why

A gateway that already follows the OTel GenAI conventions is most of the way there. From LiteLLM's OpenTelemetry docs (I have not run this against a live instance), it sets gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, so those would already match. The gaps are:

  • provider: gen_ai.system by default, and gen_ai.provider.name only in semconv mode
  • cost: gen_ai.cost.total_cost rather than agw.ai.usage.cost.total
  • identity: LiteLLM puts the caller in metadata.user_api_key_* style attributes, and there is no device, because devices authenticate to the gateway with their own IdP token rather than a controller JWT
  • signal type: spans by default, with logs opt-in. The controller registers only the OTLP logs service.

Proposal

  1. Configurable attribute names in the controller config, for example:

    usage:
      tokenFile: /etc/agentdesktop/usage-token
      attributes:            # all optional; defaults are today's names
        provider: gen_ai.system
        cost: gen_ai.cost.total_cost
        subject: metadata.user_api_key_user_id

    Keys left out keep their current value, so existing configurations behave exactly as now.

  2. Identity without a controller JWT. When device is not present, store the record against the subject only (it already tolerates an empty device). Per-user numbers would then work for any gateway, and per-device ones only where the gateway can supply a device ID.

  3. (Optional, larger) Accept OTLP traces as well as logs, since many gateways export spans and not log events. I would treat this as a separate step and understand if it is out of scope.

Open questions

  • Is a configurable mapping something you would want in the controller, or do you prefer to keep the receiver tied to agentgateway's record format?
  • If yes: is a flat usage.attributes map the right shape, or would you rather have named presets (for example preset: litellm)?
  • Should the cost key also be allowed to carry a unit, since other gateways report USD in different forms?

Offer

I am happy to send a PR for point 1 and 2. I wanted to check the direction first so I do not guess at the config shape. I have not yet confirmed what a LiteLLM log event carries on the wire, so I would test against a real instance before claiming support for it.

cc @peterj

Activity

  1. devmittal02 commented on Oct 22, 2026

    @devmittal02
    ContributorAuthor

    @peterj thoughts on this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions