Cost estimates ignore published long-context pricing tiers (OpenAI >272k, xAI >=200k) #908

Area

Dashboard

What are you trying to accomplish?

I want the ~$ column and /api/usage to be right for the requests that matter most — the big ones.

Several vendors reprice the entire request once the prompt crosses a token threshold. The two that affect me:

OpenAI publishes these as a separate "long context" row per model, not as a footnote — Sol is 5 / 0.50 / 30 short and 10 / 1.00 / 45 long.

Long requests are exactly the expensive ones, so an estimator that ignores the tier is wrong where being right is worth the most.

What prevents this today?

Cost4 is a flat four-tuple, and resolveMatchedPrice(provider, modelId) never sees a token count — so there is nowhere to express "this rate depends on how big the prompt is". Every request bills at the short rate regardless of size.

applyPriorityMultiplier() already establishes the shape for a conditional repricing step, but it keys off serviceTier, not off prompt size, so it cannot be reused as-is.

There is also a subtlety that makes this easy to get wrong: the threshold is measured on the raw usage.inputTokens the vendor counted, whereas normalizeCostTokens() subtracts cache read and write to produce billable input. A 280k-token prompt with a 200k cache read has 80k billable input but still crosses OpenAI's 272k threshold. Deciding the tier after normalization would silently under-bill exactly the cache-heavy long requests.

What should OpenCodex do?

When a request's raw input token count crosses a model's published threshold, the whole request should be costed at that model's long rate, and the estimate should say so — a flag on CostEstimate so the dashboard can distinguish "long" from "standard" rather than just showing a larger number.

Models with no published tier must be completely unaffected, and the tier must compose with the existing Fast-mode multiplier instead of replacing it.

Example usage or interface

Same request, openai/gpt-5.6-sol, 300,000 input tokens and 20,000 output tokens:

today:     5 * 0.300 + 30 * 0.020 = $2.10
should be: 10 * 0.300 + 45 * 0.020 = $3.90

What I run locally: tiers stored as multipliers rather than a second rate table, so a tier composes with whatever base rate resolved (jawcode bundle or overlay) instead of duplicating every price and drifting from it.

export interface ContextTier {
  thresholdInputTokens: number;
  /** true: `>= threshold` (xAI). false: `> threshold` (OpenAI). */
  inclusive: boolean;
  input: number; output: number; cacheRead: number; cacheWrite: number;
  source: string;
}

/** Exact (provider, modelId) key. No fuzzy matching. */
export const CONTEXT_TIERS: Readonly<Record<string, ContextTier>>;
export function findContextTier(provider: string, modelId: string): ContextTier | undefined;
export function isLongContext(tier: ContextTier, rawInputTokens: number): boolean;

Applied in estimateRequestCost() / estimateAttemptCost() before the priority multiplier, so a long Fast request bills long-rate × Fast-multiplier:

const [tieredCost4, contextTier] =
  applyContextTier(price.cost4, input.provider, input.model, input.usage.inputTokens);
const [effectiveCost4, multiplier] =
  applyPriorityMultiplier(tieredCost4, input.provider, input.model, input.serviceTier);
// ...
...(contextTier ? { contextTier } : {}),   // CostEstimate.contextTier?: "long"

estimateComboCost() propagates contextTier: "long" when any attempt crossed.

The inclusive flag is not over-engineering — xAI documents "once a prompt reaches 200k" and OpenAI documents "prompts above 272K", and a model sitting exactly on 272,000 bills differently under the two readings.

Measured on 30 days of my own usage.jsonl (same requests, same normalized tokens, only the rate table differs):

openai/gpt-5.6-sol    390/6283 requests over 272k   max prompt 337,247
                      short $76.47  ->  long $150.38
openai/gpt-5.6-luna     1/424  requests over 272k
                      short  $0.31  ->  long   $0.62
TOTAL  391 requests under-estimated by $74.22

Those 390 Sol requests were billed at roughly half what OpenAI charges. It is ~4% of my 30-day total, concentrated in <2% of requests.

Alternatives or workarounds

• A second overlay row per long variant (e.g. gpt-5.6-sol-long) — rejected: nothing in the log distinguishes them, so the resolver still could not pick between the two without a token count.
• Absolute long rates instead of multipliers — works, but doubles the table and lets the long rate drift from the short one when a vendor reprices. Multipliers stay correct through a price change; only the threshold is model-specific data.
• Correcting it after the fact in the dashboard — the tier has to be applied per request before summing, so it cannot be a display-layer adjustment.
• Leaving it — defensible while ~$ is explicitly a list-price estimate (Clarify that usage/logs \$ estimates are API list-price math, not actual charges (esp. subscription OAuth) #198), but the error is systematically one-directional and lands on the priciest requests.

Additional context

Related: I have just filed the stale gpt-5.6-terra / gpt-5.6-luna bundle rates separately. These interact — a gpt-5.6-luna prompt over 272k is currently costed with a 5×-too-high base rate and a 2×-too-low tier — but they are independent fixes and I did not want to bundle them.

Vendor sources: OpenAI, xAI, MiniMax.

I have been running the above against 2.10.0 with a test suite covering both threshold edges (199,999 vs 200,000 for xAI; 272,000 vs 272,001 for OpenAI), the raw-vs-normalized input case, models with no tier, and the interaction with the Fast multiplier. The existing tests/usage-cost.test.ts priority block needs its expectations updated, because its shared 1M-token fixture also crosses 272k — that is the change surfacing, not breaking, but it is worth knowing before review.

Happy to send it as a PR if the shape looks right to you. If you would rather keep Cost4 flat and put the threshold somewhere else entirely, say where and I will follow that instead.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.