Skip to content

fix(sidecars): bill pre-rotation loop usage to the account that spent it - #7202

Closed
vadymhimself wants to merge 5 commits into
lidge-jun:devfrom
vadymhimself:fix/sidecar-usage-per-account
Closed

vadymhimself wants to merge 5 commits into
lidge-jun:devfrom
vadymhimself:fix/sidecar-usage-per-account

Conversation

@vadymhimself

@vadymhimself vadymhimself commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Depends on #7199, which depends on #7179. This branch is built on #7199's head (fix/loop-error-iteration-usage, 84e7728e5), so until those land the diff also includes their four commits. This PR's own change is the single commit fix(sidecars): bill pre-rotation loop usage to the account that spent it (fce32f24c). I will rebase once #7179 and #7199 merge. The new hunks stay clear of #7185's lines in sidecar-execution.ts (the recovery-kind ternary near line 421).

Summary

  • Bug. A web-search or image loop reports its usage once, at the end, through onUsage. The sidecar then binds that total to whichever attempt is open at that moment (bindKeyUsageFromBridge). Suppose a 429 rotates an OAuth account partway through the loop. noteProviderAttemptSend splits the request into one attempt row per account, but every round's usage still lands on the last row. The first account's row shows no usage at all, and the second account's row is billed for both accounts' tokens. So per-account usage, quota and burn telemetry are wrong exactly when accounts are running out. Reproduced end to end with an xAI OAuth pool:
    • Round 1 on account A (100/10) is a search call.
    • Round 2 gets a 429 on A and rotates to B, which answers (300/30).
    • Attempt rows today: A has no usage, B has 400/40.
  • Fix. Right after a successful on429 rotation, each loop offers the hidden usage it has gathered so far through a new optional onRotatedUsage(usage) => boolean dep (in src/web-search/loop.ts and src/images/loop.ts).
    • src/server/responses/sidecar-execution.ts accepts only when the next send will open a new attempt. It checks with attemptAccountChanged, the same predicate noteProviderAttemptSend uses for the split. On accept, it binds the usage to the attempt that is still open, through the existing bindKeyUsageFromBridge. The next send then seals that attempt with it.
    • When the offer is accepted, the loop drops that usage from what onUsage reports later, so nothing is counted twice. When it is declined (the account did not change, or it is an API-key account, which observeUsage already meters per physical response), behaviour is exactly as before.
    • After the fix: A has 100/10 and B has 300/30.
  • Client terminal is unchanged. Web search keeps fix(web-search): meter every sidecar iteration in the usage ledger, not only the last #7179's ledger-only hiddenUsage. The image loop still puts the whole turn on the client terminal and the error event. Only the share it reports through onUsage excludes usage already billed.
  • Requests with no rotation report exactly the same numbers as before.
  • Consequence. For a non-combo request that rotated mid-loop, the row-level usage (logCtx.usage) is now the surviving attempt's own share, not the whole turn. Per-attempt rows sum to the whole turn, and usage summaries and cost already read the attempts when they exist. This matches what the API-key path already does after a key rotation (recordKeyAttemptUsage/bindKeyUsageFromBridge). It also matches the spend ledger's rule that the terminal usage belongs to the last send. src/server/request-log*.ts is not touched.
  • Reachability. For Anthropic, a mid-loop rotation after a search or image call is already refused once sidecar output is committed (allow429Recovery: !sidecarHasCommittedOutput()). So in practice this applies to generic OAuth pools (xAI, Kiro, Antigravity, Copilot and others) and to key/OAuth rotations that happen before any committed output.
  • structure/runtime.md documents the rule in the loop-usage paragraph.

Verification

  • New tests/web-search/web-search-rotation-usage-attribution.test.ts, end to end through handleResponses with the real web-search loop and sidecar, and a two-account xAI OAuth roster. It finalizes with addFinalRequestLog and asserts two attempt rows with different account labels carrying [100,10] and [300,30].
  • tests/images/loop.test.ts, new image-loop rotated usage (2 cases): an image call (10/4), a 429 rotation, then an answer (30/3). Asserts onRotatedUsage is offered 10/4, onUsage receives 30/3 when accepted and 40/7 when declined, and the client terminal carries input_tokens: 40.
    • Red on the base (hook absent): 2 fail. Green here.
  • bun test --isolate tests/web-search tests/images tests/adapters/anthropic/anthropic-sidecar-account-failover.test.ts tests/oauth/generic-oauth-failover.test.ts tests/server/server-antigravity-web-search-validation.test.ts: 697 pass / 0 fail across 39 files.
  • bun run test:changed (Bun 1.4.3): 14860 pass / 35 skip / 1 fail across 597 files. The one failure is connected-client runtime probe scope > a rejected preferred runtime falls back without rewriting the saved selection (tests/cli/cli-connect-readiness.test.ts). It fails the same way on a clean upstream/dev (60bf4c7e0) checkout and is unrelated.
  • bun run typecheck: clean. bun run structure:check: passed. bun scripts/file-size-ratchet.ts: passed. bun run privacy:scan: passed. Layout guards (tests/test-layout.test.ts, tests/test-layout-tooling.test.ts): pass.
  • I did not run the full bun run test suite locally. CI covers it.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Usage Reporting
    • Web-search and image requests now include usage from earlier iterations and adapter errors in reported totals.
    • Client-facing usage continues to reflect the final iteration for web-search requests and the full turn for image requests.
  • Account Rotation
    • Usage incurred before a credential rotation is attributed to the departing account, helping prevent duplicate reporting.
  • Documentation
    • Clarified how usage is reported across hidden search iterations, failed requests, and account rotations.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

@vadymhimself

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: lidge-jun/opencodex/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 77c50296-0cd3-4e07-82bb-4dc9115578dd
📥 Commits

Reviewing files that changed from the base of the PR and between 60bf4c7 and fce32f2.

📒 Files selected for processing: 11
  • scripts/test-layout/layout.json
  • src/images/loop.ts
  • src/server/responses/sidecar-execution.ts
  • src/web-search/loop.ts
  • src/web-search/progress-stream.ts
  • structure/runtime.md
  • tests/fixtures/test-layout-expected.json
  • tests/images/loop.test.ts
  • tests/web-search/web-search-loop-error-usage.test.ts
  • tests/web-search/web-search-loop-iteration-usage.test.ts
  • tests/web-search/web-search-rotation-usage-attribution.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The web-search and image loops now collect usage from hidden iterations, include it in usage reporting, and support assigning usage to a departing account after credential rotation.

Changes: Hidden Usage Accounting

Layer / File(s) Summary
Collect and report hidden web-search usage
src/web-search/loop.ts, src/web-search/progress-stream.ts, tests/web-search/*usage.test.ts, scripts/test-layout/layout.json, tests/fixtures/test-layout-expected.json, structure/runtime.md
The web-search loop aggregates usage from search passes, discarded empty-answer passes, and adapter errors. Tests cover aggregate reporting, final-terminal usage, and error cases. The runtime documentation and test-layout mappings were updated.
Attribute usage after credential rotation
src/images/loop.ts, src/web-search/loop.ts, src/server/responses/sidecar-execution.ts, tests/images/loop.test.ts, tests/web-search/web-search-rotation-usage-attribution.test.ts
The loops offer accumulated usage after credential rotation. Sidecar execution binds usage to the departing attempt when its account differs from the current account. Tests cover accepted and declined attribution and per-account usage records.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant WebSearchLoop
  participant billRotatedUsage
  participant attemptAccountChanged
  participant ActiveAttempt
  WebSearchLoop->>billRotatedUsage: Offer accumulated usage after rotation
  billRotatedUsage->>attemptAccountChanged: Compare active and current accounts
  attemptAccountChanged-->>billRotatedUsage: Return whether the account changed
  billRotatedUsage->>ActiveAttempt: Bind usage when the account changed
  billRotatedUsage-->>WebSearchLoop: Return whether usage was billed
Loading

Suggested reviewers: lidge-jun

Merge Risk: ⚪ Minimal · up to fce32

The change attributes usage from hidden iterations to the account that spent it after an OAuth rotation. No concrete merge-blocking risk was identified in the supplied context.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 8 files. (3 skipped: 3… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: attributing usage collected before sidecar account rotation to the account that incurred it. This matches the updates in src/server/responses…
Full details: Docstring Coverage

Explanation

Docstring coverage is 57.14% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 8 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions github-actions Bot added the bug Something isn't working label Oct 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@vadymhimself
vadymhimself force-pushed the fix/sidecar-usage-per-account branch from fce32f2 to 8580949 Compare October 15, 2026 20:47
vadymhimself and others added 5 commits October 16, 2026 06:52
Each web-search turn runs several routed-model iterations (search-calling
passes and, at the forced answer, one empty-answer retry), and every one
re-sends the full context. Only the final iteration's terminal reached the
bridge, so onUsage and the request log dropped N-1 full-context iterations
per turn.

Sum the usage of every discarded iteration with the shared mergeUsage from
empty-completion-guard and add it to what onUsage reports: the final
terminal's usage, or the sum alone when the loop ends with an in-stream or
iteration-cap error. Token and cache counts add; contextTotalTokens stays
the last iteration's.

The client-facing terminal keeps the last iteration's usage. Codex and
Claude Code size the context window from response.completed.usage (the
Claude encoder derives input/cache_read/cache_creation from it), so a sum
there would read as a context several times larger and trigger early
auto-compaction. Error terminals still throw as before; per-iteration
account attribution is unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
… termination too

Co-Authored-By: Claude Code <noreply@anthropic.com>
…ge report

An earlier iteration's contextTotalTokens and rawUsage no longer stand in for a
final terminal that omits them; the shared mergeUsage keeps its fallback.

Co-Authored-By: Claude Code <noreply@anthropic.com>
A web-search or image-bridge pass that ends in an adapter `error` event was
billed, but parseStreamWithProgress failed on the event and dropped its usage,
so the errored pass looked free in the request log.

parseStreamWithProgress now hands that usage to an `onErrorUsage` callback
before failing. Both loops add it to their hidden-iteration usage, which the
existing error paths already report; the image loop's runTurn error branch
does the same. The turn still fails with the same error.

Co-Authored-By: Claude Code <noreply@anthropic.com>
A web-search or image loop reported its usage once, at the end, and the
sidecar bound that total to whichever attempt was open then. After a 429
rotated the account mid-loop, the request split into one attempt row per
account, but the first account's row carried no usage and the second carried
both accounts' rounds, so per-account usage and quota telemetry were wrong
exactly while accounts were being exhausted.

Right after a rotation each loop now offers its hidden usage so far through
onRotatedUsage. The sidecar accepts only when the next send will open a new
attempt (the same attemptAccountChanged test that splits the row), binds it
to the still-open attempt, and the loop drops it from what onUsage reports
later, so nothing is counted twice. The image loop's client terminal still
carries the whole turn.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Being carried into #7225 with your authorship preserved; this PR will be closed when the bundle lands.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

lidge-jun added a commit that referenced this pull request Oct 16, 2026
…s and usage metering (#7194, #7201, #7176, #7177, #7178, #7179, #7199, #7202, #7189, #7190, #7200) (#7225)

Carried from #7194, #7201, #7176, #7177, #7178, #7179, #7199, #7202, #7189, #7190, #7200 (one commit per source PR, original author preserved). #7205 held: replay can relocate server-tool exchanges into an earlier assistant turn.

Remote validation (hosted GitHub Actions unreliable; local tests not run by maintainer rule): union candidate codex/prr-integ-8 = dev c03eafd + #7224 + #7223 + #7225 + #7217 + #6454 + #6594 plus a Gentle Shell import-cycle fix (integ-9 e1f289c: typecheck + full suite on mini (Windows) and lidge (Linux), compared with the same host's dev runs) and a test-only Windows path-escaping fix for the reasoning-retention routing test (integ-10 0d4cba7, tree a2b309a: that file re-run on both hosts).

Co-authored-by: Vadym O <bolein95@gmail.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Landed on dev in 300e426 via #7225 (your change is its own commit in that bundle, and the squash keeps your authorship as Co-authored-by). Thank you!

@lidge-jun lidge-jun closed this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants