Repository navigation
[Feature]: Complete cross-layer send accounting for pre-output rate-limit replay #5122
Description
Area
Proxy and routing
What are you trying to accomplish?
Finish the explicitly deferred integration between adapter-local pre-output replay and OpenCodex request-wide send accounting, provider pacing and recovery telemetry.
What prevents this today?
Public PR #5041 explicitly lists physical inner replay integration with the shared send budget, provider fetch wrapper and physical-attempt accounting as remaining work. At the pinned source, src/server/responses/run-turn-execution.ts:162 passes the budget/fetch seam, while src/adapters/devin.ts:570 does not thread it into the reset-retry wrapper.
What should OpenCodex do?
Admit every actual inference send through the existing shared permit/observer path and count it once. Keep the zero-yielded-output replay boundary, cancellation and server-stated wait semantics unchanged. Reuse current spend reservation/refund and usage settlement semantics rather than adding another local counter.
Example usage or interface
With a fake reset response followed by another attempt, the request-level used-send count and recorded physical attempts match actual fake-upstream sends. A refused next permit emits the existing withheld-recovery reason and makes no further inference request.
Alternatives or workarounds
Removing safe replay discards the shipped recovery behavior. Incrementing budget.used directly misses reservation/observer/refund semantics. An adapter-only cap would preserve separate authorities.
Additional context
Source snapshot: 7864869c31c41cca9830d93540238f17df8faafb after fast-forwarding dev on 2026-09-19. This is source-grounded analysis, not a runtime test result.
This tracks a remaining item already disclosed in #5041, not a new broad retry system. #5044/#5056 handle attribution when recovery is withheld and do not complete this adapter integration. Long-wait keep-alives and a cross-transition scheduled-wait allowance remain separate work.
Implementation path
- Thread
IncomingMeta.sendBudgetandproviderFetchthroughsrc/adapters/devin.ts,src/adapters/devin/cloud-direct/stated-reset-retry.tsand the inference send insrc/adapters/devin/cloud-direct/chat.ts. Inventory each actual inference send before choosing permit placement. - Acquire/commit/refund through existing
reserveDispatchsemantics exactly once per physical send; do not double-charge the initial outer slot. - Extend
src/adapters/base.tsIncomingMeta callbacks only if needed to share the existing physical-send/recovery contract; wire caller, adapter, callback consumer and usage aggregation together. - Preserve auth, selected-account, pacing and no-replay-after-output boundaries.
- Extend
tests/providers/devin-stated-reset-retry.test.ts,tests/providers/devin-stated-reset-hardening.test.ts,tests/adapters/adapter-inner-send-budget.test.ts,tests/adapters/adapter-inner-send-budget-wiring.test.ts, andtests/responses/responses-send-budget-counts.test.ts; updatestructure/adapters/registry.md,structure/transports/responses.mdand the adapter reference.
Acceptance criteria and verification
- Shared cap, actual fake sends, per-attempt telemetry and spend settlement agree.
- Refused send performs no inference I/O and records why recovery was withheld.
- Aborted waits and post-output failures never replay.
- Initial send is neither omitted nor counted twice.
- Tests use fake send seams and isolated state; no live account calls.
Register new tests in both layout inventories, preserve size caps, and update the existing structure owners. User-facing policy changes need corresponding public documentation and non-contradictory translations. Run focused tests and exact-head hosted CI during implementation; no local suite was run for this analysis.
Checks
- I searched existing issues and documentation.
- This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
- I removed secrets and personal data.
Activity
lidge-jun commentedon Sep 19, 2026
Closed by #5152, squash-merged into
devas26d3a862d6.The gap this issue named was that Devin's pre-output replay of a server-stated rate-limit reset, shipped in #5041, made real inference requests that no shared accounting ever saw. A single turn could issue three physical sends while the request-wide send budget, the provider fetch wrapper and physical-attempt accounting each recorded one.
All three possible sends now run through the same shared physical-send primitive the rest of the request path uses: the initial
GetChatMessagePOST is reserved as ordinal 1, the two replays as ordinals 2 and 3. The permit is reserved before dispatch, confirmed at the wire boundary, and released in afinallyon every exit, so an admitted-but-unsent attempt is refunded rather than leaked. No new counter was introduced and nothing increments a budget field directly.Two behaviours were preserved deliberately. Reservation still happens after the stated wait rather than before it, so a provider asking the client to wait does not hold a spend booking open for that whole interval. And when the budget refuses a replay, no inference request is made, the reason for withholding recovery is recorded, and the caller still sees the provider's original 429 instead of a local budget error — which is what keeps the server-stated reset and any outer cooldown intact.
Evidence is exact-head hosted CI at
00666fea7c0a4cc0b23b3fd0eb1698026953856c:test 1/4through4/4andmacos 1/2,2/2all present and successful. No local suite was run in this lane.