Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions tools/cost_calculator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ component reference regions, processing scopes, and meter IDs.
The CSV also distinguishes managed-tier pricing from AOAI Global economic
references when a model has no published managed tier.
Changes are in memory only and reset on reload.
The pipeline comparison always shows **USD per minute**, using the full call
duration. The per-call/per-minute selector affects only the prompt-sensitivity
chart. CSV export retains both per-call and per-minute amounts.

## Deploy

Expand Down Expand Up @@ -125,13 +128,18 @@ Global, Data Zone, Batch, or fine-tuning meters.

## Assumptions and formulas

Defaults preserve the supplied **90-second call**, **4 user + 4 agent
turns**, **45 seconds of speech each**, **20 audio tokens/sec** in both
directions, **100% reused-context caching**, and **150 agent text tokens**
Defaults use the supplied **90-second call**, **4 user + 4 agent
turns**, **45 seconds of speech each**, **100% reused-context caching**,
and **150 agent text tokens**
for text-response pipelines. User transcript length was not supplied:
the calculator explicitly assumes an editable **150 user text tokens**.
Native output text and reasoning tokens default to zero and can be changed.
These defaults are modeling inputs, not measured usage.
Audio defaults follow
[Voice Live's Azure OpenAI estimates](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live#token-usage-and-cost-estimation):
**10 input audio tokens/sec** and **20 output audio tokens/sec**, giving
**450 input** and **900 output** audio tokens per default call. Both rates
remain editable for other model families or measured usage.

Each call starts cold; the system prompt costs `P * input_rate` on its
first turn. Its `P * (turns - 1)` reused tokens are charged at the selected
Expand All @@ -156,8 +164,7 @@ all token prices or context-accounting details and may use different rates.
This calculator recomputes costs transparently rather than fitting those totals.
Actual caching eligibility, audio token rates, silence billing, tool calls,
telephony, storage, network, tax, capacity charges, and model quality are not
captured. In particular, the documented OpenAI input estimate is typically
~10 audio tokens/sec; 20 is retained here because it was explicitly requested.
captured.

## Checks

Expand Down
3 changes: 1 addition & 2 deletions tools/cost_calculator/app.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,6 @@ function scenarios() {
}

function renderComparison(a) {
const metric = $("metric").value;
const heading = element("tr");
heading.append(element("th", "Pipeline / model"), ...PROMPTS.map(p => element("th", `${p.toLocaleString("en-US")} tokens`)));
$("comparison").tHead.replaceChildren(heading);
Expand All @@ -182,7 +181,7 @@ function renderComparison(a) {
row.append(title);
for (const prompt of PROMPTS) {
const result = calculate({ ...a, prompt }, pipeline, rates);
const cell = element("td", money(result[metric]), result[metric] === null ? "missing" : "");
const cell = element("td", money(result.listPerMinute), result.listPerMinute === null ? "missing" : "");
cell.title = result.missing.length ? `Missing list rates: ${result.missing.map(k => labels[k]).join(", ")}` :
`List price: ${money(result.list)}/call; ${money(result.listPerMinute)}/minute`;
row.append(cell);
Expand Down
2 changes: 1 addition & 1 deletion tools/cost_calculator/calculator.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export const DEFAULTS = Object.freeze({
duration: 90, turns: 4, humanSeconds: 45, agentSeconds: 45,
inputAudioRate: 20, outputAudioRate: 20,
inputAudioRate: 10, outputAudioRate: 20,
userText: 150, agentText: 150, nativeText: 0, reasoning: 0,
cachePercent: 100, prompt: 5000, calls: 10000,
});
Expand Down
30 changes: 22 additions & 8 deletions tools/cost_calculator/calculator.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ test("South India GPT-4.1 and realtime prices do not reuse managed tier rates",
close(book("gpt-realtime-2.1-mini", "byom").textCached, 0.06);
close(book("gpt-realtime", "byom").audioOutput, 64);
const result = calculate({ ...DEFAULTS, prompt: 0 }, "cascade", rates);
close(result.list, (150 * 2 + 450 * 0.5 + 150 * 8 + 900 * (12.5 + 23)) / 1e6);
close(result.list, (150 * 2 + 450 * 0.5 + 150 * 8 + 450 * 12.5 + 900 * 23) / 1e6);
});

test("unverified managed tiers use explicit Global model references, not invented tier rates", () => {
Expand Down Expand Up @@ -79,32 +79,46 @@ test("all selectable models have priced scenarios across both hosting modes", ()

test("default native call counts fresh audio and cached history without speech duplication", () => {
const result = calculate({ ...DEFAULTS, prompt: 0 }, "native", book("gpt-realtime-2.1"));
assert.equal(result.inputAudio, 900);
assert.equal(DEFAULTS.inputAudioRate, 10);
assert.equal(DEFAULTS.outputAudioRate, 20);
assert.equal(result.inputAudio, 450);
assert.equal(result.outputAudio, 900);
close(result.list, (900 * 32 + 900 * 64 + 2700 * 0.4) / 1e6);
close(result.list, (450 * 32 + 900 * 64 + 2025 * 0.4) / 1e6);
assert.ok(result.rows.every(r => !r.label.includes("Azure")));
});

test("input audio rate overrides do not change output audio usage", () => {
const rates = book("gpt-realtime-2.1");
const original = calculate({ ...DEFAULTS, prompt: 0, inputAudioRate: 20 }, "native", rates);
const updated = calculate({ ...DEFAULTS, prompt: 0 }, "native", rates);
assert.equal(original.inputAudio, 900);
assert.equal(updated.inputAudio, 450);
assert.equal(original.outputAudio, 900);
assert.equal(updated.outputAudio, 900);
close(original.list, (900 * 32 + 900 * 64 + 2700 * 0.4) / 1e6);
close(original.list - updated.list, (450 * 32 + 675 * 0.4) / 1e6);
});

test("system prompt is full price once, reused across the remaining turns", () => {
const rates = book("gpt-realtime-2.1");
const base = calculate({ ...DEFAULTS, prompt: 0 }, "native", rates);
const full = calculate(DEFAULTS, "native", rates);
close(full.list - base.list, 5000 * (4 + 3 * 0.4) / 1e6);
const noCache = calculate({ ...DEFAULTS, cachePercent: 0 }, "native", rates);
close(noCache.list, (20000 * 4 + 3600 * 32 + 900 * 64) / 1e6);
close(noCache.list, (20000 * 4 + 2475 * 32 + 900 * 64) / 1e6);
const halfCache = calculate({ ...DEFAULTS, cachePercent: 50 }, "native", rates);
close(halfCache.list, (noCache.list + full.list) / 2);
});

test("single turn does not charge replayed context", () => {
const result = calculate({ ...DEFAULTS, turns: 1 }, "hybrid", book("gpt-realtime-2.1"));
assert.ok(result.rows.every(r => !r.label.includes("/ cached")));
close(result.list, (5000 * 4 + 900 * 32 + 150 * 24 + 900 * 31) / 1e6);
close(result.list, (5000 * 4 + 450 * 32 + 150 * 24 + 900 * 31) / 1e6);
});

test("hybrid uses 150 text output tokens and HD list price, no native output audio charge", () => {
const result = calculate({ ...DEFAULTS, prompt: 0 }, "hybrid", book("gpt-realtime-2.1"));
const llm = (900 * 32 + 1350 * 0.4 + 225 * 0.4 + 150 * 24) / 1e6;
const llm = (450 * 32 + 675 * 0.4 + 225 * 0.4 + 150 * 24) / 1e6;
close(result.list, llm + 900 * 31 / 1e6);
assert.ok(result.rows.every(r => r.key !== "audioOutput"));
});
Expand All @@ -116,7 +130,7 @@ test("BYOM uses BYO speech meters, not managed tier prices", () => {
close(rates.textOutput, 15);
const result = calculate({ ...DEFAULTS, prompt: 0 }, "cascade", rates);
const llm = (150 * 2.5 + 450 * 0.25 + 150 * 15) / 1e6;
close(result.list, llm + 900 * (12.5 + 23) / 1e6);
close(result.list, llm + (450 * 12.5 + 900 * 23) / 1e6);
close(result.listPerMinute, result.list / 1.5);
});

Expand Down Expand Up @@ -201,7 +215,7 @@ test("zero-priced input is explicit and unused missing rates do not block a tota
const rates = { ...book("gpt-realtime"), textInput: null, textCached: null, audioOutput: 0 };
const result = calculate({ ...DEFAULTS, prompt: 0 }, "native", rates);
assert.deepEqual(result.missing, []);
close(result.list, (900 * 32 + 2700 * 0.4) / 1e6);
close(result.list, (450 * 32 + 2025 * 0.4) / 1e6);
});

test("all five prompt sizes grow monotonically for every priced model", () => {
Expand Down
8 changes: 4 additions & 4 deletions tools/cost_calculator/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ <h2><span>02</span> Conversation assumptions</h2>
<label>System prompt tokens<input name="prompt" type="number" min="0" step="any" required></label>
<label>Context cache hit (%)<input name="cachePercent" type="number" min="0" max="100" step="any" required></label>
</div>
<p class="hint">Defaults: 900 input + 900 output audio tokens. Text totals are independent estimates, not derived from audio. The 150-token user transcript is illustrative.</p>
<p class="hint">Defaults: 10 input and 20 output audio tokens/sec, giving 450 input + 900 output audio tokens. Text totals are independent estimates, not derived from audio. The 150-token user transcript is illustrative.</p>
</section>
<section>
<h2><span>03</span> Call volume</h2>
Expand Down Expand Up @@ -70,9 +70,9 @@ <h2><span>03</span> Call volume</h2>
<p class="hint">Same conversation, different prompt sizes. All values use list prices and the selected per-call or per-minute view.</p>
</section>
<section>
<div class="section-top"><div><span class="eyebrow">SIDE BY SIDE</span><h2>Pipeline comparison</h2></div><button id="export" type="button" class="secondary">Export CSV</button></div>
<p class="hint">Uses your assumptions and each model's own rates. Without a published managed tier, rows are explicitly labeled AOAI Global model-price + BYO speech references, not managed tariffs. Edits to the selected model apply to its rows. Missing prices never become zero.</p>
<div class="table-wrap"><table id="comparison"><thead></thead><tbody></tbody></table></div>
<div class="section-top"><div><span class="eyebrow">SIDE BY SIDE</span><h2>Pipeline comparison (USD / minute)</h2></div><button id="export" type="button" class="secondary">Export CSV</button></div>
<p class="hint">All costs are USD per minute, based on full call duration, your assumptions, and each model's own rates. Without a published managed tier, rows are explicitly labeled AOAI Global model-price + BYO speech references, not managed tariffs. Edits to the selected model apply to its rows. Missing prices never become zero.</p>
<div class="table-wrap"><table id="comparison" aria-label="Pipeline list-price comparison in USD per minute"><thead></thead><tbody></tbody></table></div>
</section>
<section>
<span class="eyebrow">FOLLOW THE TOKENS</span><h2>Selected call breakdown</h2>
Expand Down
Loading