Skip to content

compaction: count opaque state by the provider's own measure - #1694

Merged
justrach merged 1 commit into
mainfrom
fix/opaque-state-accounting
Oct 20, 2026
Merged

justrach merged 1 commit into
mainfrom
fix/opaque-state-accounting

Conversation

@justrach

Copy link
Copy Markdown
Owner

Fixes #1686 (the accounting half; #1687 handled repeated compaction).

Root cause

context_tokens.estimatedTokensFromLen counted compaction state at bytes / 16. The real ratio varies widely: a 1.4 MB blob counted ~70k tokens, a 9.8 MB one ~80k. The second read as ~612k. The context meter is floored at the local estimate (agent_context.replaceContextTokens), so a history of one blob and a short reply sat over the compaction line and compacted every turn, while the provider reported ~82k input tokens.

Fix

  • New opaque_state_usage.learn, run before the meter reads the estimate. After a full (unchained) Responses request that carries compaction state, it computes the state's share of the reported input: the input less the estimate for everything else.
  • That share is kept by state size (context_tokens.calibrateState) and replaces /16 for that state.
  • Chained requests send only a delta and are skipped. Before the first count, or for a new state, /16 still applies.
  • Values are packed in atomics, so a subagent thread can't read a torn pair.

Tests

  • New: the issue's numbers (9.79 MB of state, 82k reported) recalibrate to ~74k, a second sample replaces the first, other sizes keep /16, the floor clamps, and an estimate of a value carrying that state follows the calibration.
  • Accounting and compaction unit tests: 327 pass.
  • Tier-1 pre-push gate is green.

Compaction state was always estimated at its bytes / 16. The real ratio
varies widely: a 1.4 MB blob counted ~70k tokens, a 9.8 MB one ~80k. The
second read as ~612k, and since the context meter is floored at the local
estimate, a history of one blob and a short reply sat over the compaction
line and compacted again every turn, while the provider reported ~82k.

After a full (unchained) Responses request that carries compaction state,
the state's share of the reported input (the input less the estimate for
everything else) is kept by size and replaces /16 for that state. Chained
requests send only a delta and are skipped. Before the first count, /16
still applies.

Fixes #1686
@justrach
justrach merged commit a9dc83d into main Oct 20, 2026
6 checks passed
@justrach
justrach deleted the fix/opaque-state-accounting branch October 20, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ACP turn repeatedly compacts and rereads unchanged files for 47 minutes

1 participant