Skip to content

fix(proxy): attach usage-recording listeners before streaming responses (#1222) - #1481

Open
lienhoatienphung-svg wants to merge 1 commit into
CalloraOrg:mainfrom
lienhoatienphung-svg:fix/1222-attach-usage-listener-before-streaming
Open

lienhoatienphung-svg wants to merge 1 commit into
CalloraOrg:mainfrom
lienhoatienphung-svg:fix/1222-attach-usage-listener-before-streaming

Conversation

@lienhoatienphung-svg

Copy link
Copy Markdown

Summary

Attaches the res.once('finish') and res.once('close') usage-recording listeners in src/routes/proxyRoutes.ts before streaming or sending the upstream response body so synchronous or fast-completing responses never miss the finish event and aborted streams are not billed.

Closes #1222

Key Changes

  1. Listener Ordering & Stream Error Guard (src/routes/proxyRoutes.ts):
    • Registered res.once('finish', recordUsageOnce) and res.once('close', ...) immediately after setting status/headers and before starting the body stream (await pump() / res.send(text)).
    • Guarded recordUsageOnce with !res.writableEnded || streamErrored so mid-stream upstream errors or prematurely closed client connections never record billable usage.
    • Logged billing.chargeUsage failures with full reconciliation context (userId, apiId, endpointId, requestId, amountUsdc).
  2. Streaming & Abort Tests (src/__tests__/usageMetering.test.ts):
    • Added tests verifying that multi-chunk streamed responses record usage exactly once and mid-stream aborted responses do not record usage.

Verification

  • Verified with npx jest src/__tests__/usageMetering.test.ts (13/13 passed).

Payout Wallet (Base Mainnet USDC)

0x3806cd592a508d1992Ed007C2F53de251C2E7452

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Attach usage-recording listener before streaming responses

1 participant