Repository navigation
fix(sidecars): bill pre-rotation loop usage to the account that spent it - #7202
vadymhimself wants to merge 5 commits into
Conversation
|
@coderabbitai review |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
✅ Action performedReview finished.
|
|
✅ Deterministic PR hygiene checks passed. |
fce32f2 to
8580949
Compare
Each web-search turn runs several routed-model iterations (search-calling passes and, at the forced answer, one empty-answer retry), and every one re-sends the full context. Only the final iteration's terminal reached the bridge, so onUsage and the request log dropped N-1 full-context iterations per turn. Sum the usage of every discarded iteration with the shared mergeUsage from empty-completion-guard and add it to what onUsage reports: the final terminal's usage, or the sum alone when the loop ends with an in-stream or iteration-cap error. Token and cache counts add; contextTotalTokens stays the last iteration's. The client-facing terminal keeps the last iteration's usage. Codex and Claude Code size the context window from response.completed.usage (the Claude encoder derives input/cache_read/cache_creation from it), so a sum there would read as a context several times larger and trigger early auto-compaction. Error terminals still throw as before; per-iteration account attribution is unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
… termination too Co-Authored-By: Claude Code <noreply@anthropic.com>
…ge report An earlier iteration's contextTotalTokens and rawUsage no longer stand in for a final terminal that omits them; the shared mergeUsage keeps its fallback. Co-Authored-By: Claude Code <noreply@anthropic.com>
A web-search or image-bridge pass that ends in an adapter `error` event was billed, but parseStreamWithProgress failed on the event and dropped its usage, so the errored pass looked free in the request log. parseStreamWithProgress now hands that usage to an `onErrorUsage` callback before failing. Both loops add it to their hidden-iteration usage, which the existing error paths already report; the image loop's runTurn error branch does the same. The turn still fails with the same error. Co-Authored-By: Claude Code <noreply@anthropic.com>
A web-search or image loop reported its usage once, at the end, and the sidecar bound that total to whichever attempt was open then. After a 429 rotated the account mid-loop, the request split into one attempt row per account, but the first account's row carried no usage and the second carried both accounts' rounds, so per-account usage and quota telemetry were wrong exactly while accounts were being exhausted. Right after a rotation each loop now offers its hidden usage so far through onRotatedUsage. The sidecar accepts only when the next send will open a new attempt (the same attemptAccountChanged test that splits the row), binds it to the still-open attempt, and the loop drops it from what onUsage reports later, so nothing is counted twice. The image loop's client terminal still carries the whole turn. Co-Authored-By: Claude Code <noreply@anthropic.com>
8580949 to
1821716
Compare
|
Being carried into #7225 with your authorship preserved; this PR will be closed when the bundle lands. |
⏳ DRAFT
What to do
Review readiness checklist
0/4 boxes ticked. This PR stays in draft until every box above is ticked. |
…s and usage metering (#7194, #7201, #7176, #7177, #7178, #7179, #7199, #7202, #7189, #7190, #7200) (#7225) Carried from #7194, #7201, #7176, #7177, #7178, #7179, #7199, #7202, #7189, #7190, #7200 (one commit per source PR, original author preserved). #7205 held: replay can relocate server-tool exchanges into an earlier assistant turn. Remote validation (hosted GitHub Actions unreliable; local tests not run by maintainer rule): union candidate codex/prr-integ-8 = dev c03eafd + #7224 + #7223 + #7225 + #7217 + #6454 + #6594 plus a Gentle Shell import-cycle fix (integ-9 e1f289c: typecheck + full suite on mini (Windows) and lidge (Linux), compared with the same host's dev runs) and a test-only Windows path-escaping fix for the reasoning-retention routing test (integ-10 0d4cba7, tree a2b309a: that file re-run on both hosts). Co-authored-by: Vadym O <bolein95@gmail.com>
Summary
onUsage. The sidecar then binds that total to whichever attempt is open at that moment (bindKeyUsageFromBridge). Suppose a 429 rotates an OAuth account partway through the loop.noteProviderAttemptSendsplits the request into one attempt row per account, but every round's usage still lands on the last row. The first account's row shows no usage at all, and the second account's row is billed for both accounts' tokens. So per-account usage, quota and burn telemetry are wrong exactly when accounts are running out. Reproduced end to end with an xAI OAuth pool:on429rotation, each loop offers the hidden usage it has gathered so far through a new optionalonRotatedUsage(usage) => booleandep (insrc/web-search/loop.tsandsrc/images/loop.ts).src/server/responses/sidecar-execution.tsaccepts only when the next send will open a new attempt. It checks withattemptAccountChanged, the same predicatenoteProviderAttemptSenduses for the split. On accept, it binds the usage to the attempt that is still open, through the existingbindKeyUsageFromBridge. The next send then seals that attempt with it.onUsagereports later, so nothing is counted twice. When it is declined (the account did not change, or it is an API-key account, whichobserveUsagealready meters per physical response), behaviour is exactly as before.hiddenUsage. The image loop still puts the whole turn on the client terminal and the error event. Only the share it reports throughonUsageexcludes usage already billed.usage(logCtx.usage) is now the surviving attempt's own share, not the whole turn. Per-attempt rows sum to the whole turn, and usage summaries and cost already read the attempts when they exist. This matches what the API-key path already does after a key rotation (recordKeyAttemptUsage/bindKeyUsageFromBridge). It also matches the spend ledger's rule that the terminal usage belongs to the last send.src/server/request-log*.tsis not touched.allow429Recovery: !sidecarHasCommittedOutput()). So in practice this applies to generic OAuth pools (xAI, Kiro, Antigravity, Copilot and others) and to key/OAuth rotations that happen before any committed output.structure/runtime.mddocuments the rule in the loop-usage paragraph.Verification
tests/web-search/web-search-rotation-usage-attribution.test.ts, end to end throughhandleResponseswith the real web-search loop and sidecar, and a two-account xAI OAuth roster. It finalizes withaddFinalRequestLogand asserts two attempt rows with different account labels carrying[100,10]and[300,30].84e7728e5): rows were[undefined, undefined]and[400, 40].tests/images/loop.test.ts, newimage-loop rotated usage(2 cases): an image call (10/4), a 429 rotation, then an answer (30/3). AssertsonRotatedUsageis offered 10/4,onUsagereceives 30/3 when accepted and 40/7 when declined, and the client terminal carriesinput_tokens: 40.bun test --isolate tests/web-search tests/images tests/adapters/anthropic/anthropic-sidecar-account-failover.test.ts tests/oauth/generic-oauth-failover.test.ts tests/server/server-antigravity-web-search-validation.test.ts: 697 pass / 0 fail across 39 files.bun run test:changed(Bun 1.4.3): 14860 pass / 35 skip / 1 fail across 597 files. The one failure isconnected-client runtime probe scope > a rejected preferred runtime falls back without rewriting the saved selection(tests/cli/cli-connect-readiness.test.ts). It fails the same way on a cleanupstream/dev(60bf4c7e0) checkout and is unrelated.bun run typecheck: clean.bun run structure:check: passed.bun scripts/file-size-ratchet.ts: passed.bun run privacy:scan: passed. Layout guards (tests/test-layout.test.ts,tests/test-layout-tooling.test.ts): pass.bun run testsuite locally. CI covers it.Checklist
🤖 Generated with Claude Code
Summary by CodeRabbit
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
Required local validation passed; commands, results, and any full-suite exception are documented.
I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.