feat(providers): carry opt-in API key round-robin and per-key pacing (#6937)

[lidge-jun]

Summary

Carry of contributor PR #6937 by @GSL-R, retaining its commits through an ordinary merge of source head b8d48e7227a504586039f9a9a8a25779af42ce45 onto dev 7640a942b3. The feature starts at 636a7d2e3342d9f07141c8116bf29d3caae5a7d8; subsequent contributor and maintainer fixes are retained, including the final merge's failed-key refusal before turn readmission. Current dev does not contain this opt-in feature.

requestPacing.keySelection: "round-robin" allocates healthy API keys per request without persisting an active-key change. requestPacing.keys adds uniform per-credential spacing/concurrency gates alongside provider/model gates. Default active-key selection, OAuth and forwarded credentials retain their existing behavior. Queued stale selections rebuild and acquire replacement-key capacity before sending; terminal distributed 401/429 outcomes retain cooldowns. WebSocket paths fall back to HTTP/SSE when these controls require a guarded send.

Preserves fixes for all original CodeRabbit findings: translated invalid-limit feedback, Portuguese/Russian active-first copy, distributed 401 cooling, replacement-key admission, notification-error lease release, longest cooldown retention, and queued-key documentation. Ingwannu's review was a scoped compact/CI inspection and grants no broad security approval.

Images retains one non-idempotent upstream POST, distributed selection and cooldown recording. Its standalone relay remains outside inference pacing; per-key/provider pacing limits do not cover that relay. Counters/cursors are process-local and credential-reference-based, not project quota or cost balancing.

Maintainer security review is required before merge under MAINTAINERS.md because this changes credential selection and recovery. A GUI screenshot is still required before review readiness; no screenshot is committed to this branch. Independent transport audit found native Responses cooldown and CLI dispatch gaps; the follow-up fixes and regressions are included. Independent static re-reviews passed at 3e7270678a and final head 70d2473332; these are bounded code reviews, not maintainer security approval or independent test execution. No labels or changes to the source PR are requested by this carry.

Co-authored-by: GSL-R 174104911+GSL-R@users.noreply.github.com

Screenshots

Round-robin API-key selection and provider/per-key pacing limits.

Captured from this PR worktree in light theme using disposable homes and fake provider data. Fresh evidence for the screenshot requirement; earlier screenshot-pending notes below are historical.

Verification

All tests/checks ran in this worktree with Bun 1.4.3 and temporary HOME, OPENCODEX_HOME and CODEX_HOME via an ignored scratch wrapper. Installation used bun install and (cd gui && bun install) first; no dependency or lockfile changes. No live account/config/keychain access, service restart or provider preset edits.

bun test tests/adapters/key-failover.test.ts tests/providers/api-key-selection-cooldown.test.ts tests/usage/request-pacing.test.ts tests/usage/request-pacing-provenance.test.ts
# 90 pass, 0 fail; 387 assertions
bun test tests/server/server-key-failover-e2e.test.ts tests/responses/responses-run-turn-key-pacing.test.ts tests/responses/compact-key-selection.test.ts tests/server/distributed-key-401.test.ts tests/server/server-chat-native-key-selection.test.ts tests/claude-integration/messages-native-key-selection.test.ts tests/server/server-anthropic-billing.test.ts tests/server/management-provider-request-pacing.test.ts tests/server/fetch-header-timeout.test.ts tests/responses/responses-fetch-helpers-boundary.test.ts
# 210 pass, 0 fail; 1684 assertions
bun test tests/claude-integration/messages-native.test.ts tests/responses/chat-native-combo.test.ts tests/responses/responses-compaction-routing.test.ts tests/responses/compact-transient-alternate.test.ts tests/server/server-images.test.ts tests/lab/core-lab-boundary.test.ts
# 355 pass, 0 fail; 2285 assertions
bun test tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/file-size-ratchet.test.ts
# 36 pass, 0 fail; 653 assertions
bun test ./gui/tests/provider-settings-request-pacing.test.tsx
# 8 pass, 0 fail; 25 assertions; existing React act warnings
bun run typecheck                 # PASS, exit 0
bun run structure:check           # PASS, exit 0
bun run privacy:scan              # PASS, exit 0
bun scripts/file-size-ratchet.ts  # PASS, exit 0; no baseline increases
(cd gui && bun run lint:i18n)     # PASS, exit 0
(cd gui && bun run lint)          # PASS, exit 0
(cd gui && bun run build)         # PASS, exit 0; existing chunk-size warning
(cd docs-site && bun install --frozen-lockfile && bun run build)
# PASS, exit 0; 577 pages, 81200 internal links

691 focused runtime/guard cases plus 8 GUI cases passed. Structured comparisons retained every mapping from both parents in the shard-aware layout and expected fixture. git diff --check passed. No full local suite, prepush, whole tests directory or whole GUI suite was run: the maintainer explicitly prohibits heavy local suites on this shared host. Repository-wide/platform coverage is left to exact-head CI. PR and manual cross-platform run results will be recorded after pushing; no unrun CI is claimed as passed.

Follow-up validation at 3e7270678a482313cec547e8fc229daa838108e3

Native Responses now captures immutable credential/destination ownership after connection-policy rewrites and settles terminal distributed failures after configured same-key retries. Pending genuine 429 evidence survives a local replay-admission refusal; cancellation, synthetic responses, header overrides and stale revisions do not cool unused keys. CLI turns revalidate every admission and fence spawn after asynchronous tool setup. The passthrough catch also preserves pacing 429/Retry-After. Initial and replay queue-exhaustion/expiry end-to-end cases already returned 429 before this catch hardening; no reproduced 502 is claimed.

bun test tests/responses/responses-native-key-cooldown.test.ts tests/responses/responses-run-turn-key-readmission.test.ts tests/adapters/qoder-spawn-readmission.test.ts tests/responses/responses-run-turn-key-pacing.test.ts tests/server/server-key-failover-e2e.test.ts tests/usage/request-pacing-provenance.test.ts tests/adapters/anthropic/anthropic-billing-ownership.test.ts tests/server/server-anthropic-billing.test.ts tests/lab/core-lab-boundary.test.ts
# 247 pass, 0 fail; 1472 assertions
bun test tests/responses/responses-run-turn-key-readmission.test.ts tests/adapters/qoder-spawn-readmission.test.ts tests/responses/responses-run-turn-key-pacing.test.ts tests/providers/qoder-adapter.test.ts tests/providers/codebuddy-adapter.test.ts tests/providers/claude-cli-adapter.test.ts
# 197 pass, 0 fail; 764 assertions (implementation-lane execution)
bun test tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/file-size-ratchet.test.ts
# 36 pass, 0 fail; 653 assertions
bun run typecheck         # PASS
bun run structure:check   # PASS
bun run privacy:scan      # PASS

Regression evidence: native 401/429 cases failed before the fix (10 pass, 2 fail in the original 12-case batch); final native file is 15 pass, 0 fail. Disabling the admission/spawn fences produced 2 pass, 6 fail, and restored-code focused validation passed. New tests live in sibling files and are registered in both layout inventories. No ratchet baseline changed. GUI and public docs are unchanged by the follow-up, so their passing builds were not rerun. Hosted CI is still queued; final-head results remain pending.

Concurrent outcome validation at 70d24733328f6520463e109f84280cf016c1f164

Failure ownership now permits an already cooling, still-configured credential to extend its cooldown. The separate physical-send check continues to refuse that credential. A synthetic replay refusal retains preceding genuine rejection evidence. Both new native regression cases failed before their fixes and pass afterward (15 pass / 1 fail, then 16 pass / 1 fail in the respective red runs).

bun test tests/responses/responses-native-key-cooldown.test.ts tests/providers/api-key-selection-cooldown.test.ts tests/adapters/key-failover.test.ts tests/server/server-key-failover-e2e.test.ts
# 128 pass, 0 fail; 865 assertions
bun test tests/server/server-chat-native-key-selection.test.ts tests/claude-integration/messages-native-key-selection.test.ts tests/responses/compact-key-selection.test.ts tests/lab/core-lab-boundary.test.ts
# 68 pass, 0 fail; 768 assertions
bun test tests/test-layout.test.ts tests/test-layout-tooling.test.ts tests/ci-workflows/file-size-ratchet.test.ts
# 36 pass, 0 fail; 653 assertions
bun run typecheck         # PASS, exit 0
bun run structure:check   # PASS, exit 0
bun run privacy:scan      # PASS, exit 0

The prior independent focused re-review passed at 3e7270678a; this final concurrent-outcome delta passed a separate narrow independent static re-review at 70d2473332, with no remaining blockers. Hosted executable CI remains pending/queued; cancelled or older-head checks do not qualify as passing. ci.yml dispatch input is lane=all; the full cross-platform dispatch remains contingent on green PR CI, as requested. Screenshot and explicit maintainer security review remain required.

Exact-head hosted state

At 70d24733328f6520463e109f84280cf016c1f164, PR Cross-platform CI is pending; no executable job result is available. Catalog async contracts, React Doctor, Enforce PR target branch and PR Labeler are queued; PR hygiene is pending. These are not passing CI evidence. Full workflow_dispatch (lane=all) has not run because the requested green PR CI prerequisite is pending. GOAT rejected the implementation subagent's PR watch; the coordinator must own ongoing monitoring and dispatch after green.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Pending: explicit maintainer security review, GUI screenshot and exact-head CI.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Deterministic PR hygiene checks passed.

[github-actions]

⏳ DRAFT

• UI screenshot required.

What to do

• Add a screenshot of the UI change to the PR description.

This pull request was already a draft. Its draft status will be preserved after every issue above is resolved.

Hygiene

✅ Deterministic PR hygiene checks passed.