fix(claude): keep the caller's 1h cache TTL on the translated lane

[vadymhimself]

Summary

On the translated Messages lane, opencodex ignores the caller's cache_control breakpoints and the anthropic adapter places its own 4. This lane covers combo and policy routes, and any route the native Messages lane declines. The adapter's TTL comes only from the global cacheRetention, which is 5m when unset (src/adapters/anthropic.ts:75-79, :1160-1171).

Claude Code on a subscription marks its breakpoints { "type": "ephemeral", "ttl": "1h" }. The translated lane downgraded those to 5m. Any pause longer than 5 minutes therefore misses the whole cached prefix, and the next turn re-writes it. That is normal human pacing, and it is the gap Claude Code's own 1h TTL is designed to cover.

Quota impact. Every pause of more than 5 minutes on a translated route paid a full prefix cache write again: system, tools and history. A 1h hit would have read it instead. On subscription accounts that is quota burned for no benefit. The size of the loss was measured offline only, with an in-process mock upstream and a local model of Anthropic's cache. No live API traffic was used, so the real-world magnitude is unmeasured.

The only workaround so far was cacheRetention: "long", which forces 1h on every breakpoint for every client.

Change. When cacheRetention is not set and any of the caller's own breakpoints asks for ttl: "1h", that one /v1/messages request runs with cacheRetention: "long".

• Precedence: explicit operator cacheRetention > caller hint > default (5m).
• src/claude/inbound.ts: new callerCacheRetention(raw) scans the caller's system, tools and message blocks. It is read out of body, because the Responses body has no field that would not leak to a native Responses forward.
• src/server/claude-messages.ts: the per-request config handed to handleResponses gets cacheRetention: config.cacheRetention ?? callerCacheRetention(anthropicBody). Only the anthropic adapter reads cacheRetention (src/adapters/registry.ts:101). GPT legs of a combo, and the native lane (which forwards the caller's breakpoints verbatim), are unaffected.
• Breakpoint positions are unchanged. The existing TTL-ordering pass still runs.
• "long" also makes the moving per-turn message breakpoint and the top-level automatic one 1h. This matches Claude Code's own 1h marking, which puts ttl: "1h" on its message breakpoint too. It also means those per-turn writes are billed at the 1h write rate, exactly as the caller asked.
• Docs: the cacheRetention row in reference/configuration/providers.md now describes the fallback (English only).

Verification

• Wire capture of real Claude Code 2.1.281 (CLAUDE_CODE_PROMPT_CACHE_TTL=1h) against a local capture server: breakpoints system[1], system[2] and the moving messages[1].content[0] are all {"type":"ephemeral","ttl":"1h"} — so mapping to "long" (1h on the moving message breakpoint too) mirrors what the caller asked for.

• Red/green. I reverted only the two src/ files to upstream/dev and kept the new test. bun test tests/claude-integration/claude-translated-cache-ttl.test.ts then gave 2 pass / 1 fail: caller ttl 1h keeps every adapter breakpoint at 1h received an untagged (5m) TTL. With the fix restored: 3 pass / 0 fail. The test drives handleClaudeMessages on a combo route with a mocked upstream and asserts the TTLs on the anthropic wire body in three cases: caller 1h gives 1h, caller with no TTL gives 5m, and caller 1h with operator "short" gives 5m.

• bun run typecheck: clean.

• Focused: bun test tests/claude-integration/claude-inbound.test.ts tests/claude-integration/claude-messages-endpoint.test.ts tests/claude-integration/claude-sidecar-override.test.ts tests/claude-integration/claude-inbound-cache-stabilize.test.ts: 168 pass / 0 fail.

• bun run test:changed (runner Bun 1.4.3): 10437 pass / 32 skip / 1 fail across 456 files. The one failure is tests/cli/cli-connect-readiness.test.ts, "connected-client runtime probe scope > a rejected preferred runtime falls back without rewriting the saved selection". It fails the same way on clean upstream/dev at 7640a942b (26 pass / 1 fail in isolation), so it is not caused by this change.

• bun test tests/test-layout.test.ts tests/test-layout-tooling.test.ts: everything passes except move end to end > domain verification launches isolate tests through the pinned runner. That case also fails on clean upstream/dev because the local default Bun is not 1.4.3. The new file is registered in scripts/test-layout/layout.json and tests/fixtures/test-layout-expected.json.

• bun run structure:check: passed. The docs-site build (cd docs-site && bun install --frozen-lockfile && bun run build) passed, as did docs-developer-role-policy and docs-429-failover-claims (23 pass).

• Offline harness. This drives handleClaudeMessages through 6 Claude Code-shaped turns, with the network blocked and the upstream mocked. Wire breakpoints at turn 2:

  • before, caller 1h on the combo route: tools[17] system[0] messages[2].user[0] messages[3].user[0]:auto (all 5m)
  • after, caller 1h on the combo route: tools[17]:1h system[0]:1h messages[2].user[0]:1h messages[3].user[0]:1h:auto
  • after, caller with no TTL: unchanged (all 5m). The native direct lane is unchanged and still forwards the caller's own system[2]:1h messages[2].user[1]:1h.
  • Breakpoint positions and the simulated turn-to-turn cache-read share (89.1% → 92.4%) are identical before and after.

• Not run: the full bun run test. The change is two src/ files on one request path, and test:changed covered its import graph. The full suite is left to CI.

• Maintainer fixes on top of the contributor commits (0feecfcbf, 33d522318, 26c38ea8a, 0e73693bf): callerCacheRetention now returns "long" only when every caller breakpoint asks for 1h, counting the top-level automatic cache_control and breakpoints nested at any depth in block containers (tool_result, search_result, document sources), while tool input / input_schema are skipped. Before this, a caller with a 1h prefix and a 5m tail had its moving per-turn breakpoint upgraded to 1h, so those writes were billed at the 1h rate (2x base input instead of 1.25x). New cases in claude-translated-cache-ttl.test.ts fail against each previous head and pass now (6 pass / 0 fail).

• Maintainer local evidence at 0e73693bf: bun test --isolate tests/claude-integration tests/adapters/anthropic 3255 pass / 1 fail. The failure is claude-desktop-remote-hub.test.ts "connected Desktop uses hub IDs across a cold restart", which fails the same way on a dev-equivalent tree. bun run test:changed at 0feecfcbf gave 10439 pass / 32 skip / 0 fail across 456 files; later commits only touch callerCacheRetention and its test. bun run typecheck clean, and the file-size ratchet passes.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Translated /v1/messages requests now use long cache retention when all caller-specified cache breakpoints use a 1-hour TTL.
  • Explicit cache-retention settings take precedence; mixed or 5-minute TTLs keep the existing short-retention default.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

• Required local validation passed; commands, results, and any full-suite exception are documented.

• I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

• I resolved all correct Codex and CodeRabbit findings.

• My PR is ready for review.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: lidge-jun/opencodex/.coderabbit.yaml
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 6fc402a0-1c22-4613-8873-63cd54f1563a

📥 Commits

Reviewing files that changed from the base of the PR and between 7640a94 and 0e73693.

📒 Files selected for processing: 6

• docs-site/src/content/docs/reference/configuration/providers.md
• scripts/test-layout/layout.json
• src/claude/inbound.ts
• src/server/claude-messages.ts
• tests/claude-integration/claude-translated-cache-ttl.test.ts
• tests/fixtures/test-layout-expected.json

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

📝 Walkthrough

Translated Claude Messages requests now derive long cache retention from caller breakpoints when every collected TTL is 1h and no operator setting is configured. The change also documents the precedence and adds integration coverage.

Changes: Translated Claude cache retention

Layer / File(s)	Summary
Detect caller cache TTLs src/claude/inbound.ts, tests/claude-integration/claude-translated-cache-ttl.test.ts	callerCacheRetention scans top-level cache control and nested blocks in system, tools, and messages. It ignores input and input_schema while traversing, stops beyond depth 16, and returns "long" only when it finds TTLs and all are "1h". Tests cover nested content and excluded fields.
Apply retention precedence and validate src/server/claude-messages.ts, tests/claude-integration/claude-translated-cache-ttl.test.ts, docs-site/src/content/docs/reference/configuration/providers.md, scripts/test-layout/layout.json, tests/fixtures/test-layout-expected.json	The route uses configured retention when present; otherwise it derives retention from the inbound request. Integration tests cover uniform, mixed, missing, and explicit retention cases. The configuration reference describes the defaults, and the test-layout mappings include the new test.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant ClaudeMessagesRoute
  participant callerCacheRetention
  participant handleResponses
  ClaudeMessagesRoute->>callerCacheRetention: Scan anthropicBody for caller TTLs
  callerCacheRetention-->>ClaudeMessagesRoute: Return "long" or undefined
  ClaudeMessagesRoute->>handleResponses: Pass configured retention or caller hint

Loading

Suggested reviewers: lidge-jun

Merge Risk: ⚪ Minimal · up to 0e736

The translated caching change is mergeable after normal checks; no actionable issue was established.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. (3 skipped: 3…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: preserving the caller's 1-hour cache TTL on the translated Claude lane.

Full details: Docstring Coverage

Explanation

Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. (3 skipped: 3 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[vadymhimself]

Live A/B on real Anthropic (Claude Code 2.1.281, CLAUDE_CODE_PROMPT_CACHE_TTL=1h, same scripted 6-turn session, combo/waterfall → claude-opus-5-5, 6 min idle before turn 5). Usage summed from the session transcript per turn:

build	turn 5 (after 6 min idle) cache write	cache read
upstream/dev @ c37161c (translated lane, adapter breakpoints 5m)	89,511	84,668
upstream/dev + this PR	7,255	164,987
upstream native Messages lane (direct route, reference)	5,817	170,092

Without the fix the prefix expired during the pause and the whole conversation was re-written to cache; with it the caller's 1h TTL holds and the turn reads from cache like the native lane. Turns before the pause were equivalent across builds.

[vadymhimself]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Pull request base or head changed.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[vadymhimself]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[github-actions]

⏳ DRAFT

• review readiness checklist open (0/4 boxes ticked).

What to do

• Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

• ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
• ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
• ⬜ I resolved all correct Codex and CodeRabbit findings.
• ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

[lidge-jun]

Being carried into #7225 with your authorship preserved; this PR will be closed when the bundle lands.

[lidge-jun]

Landed on dev in 300e426 via #7225 (your change is its own commit in that bundle, and the squash keeps your authorship as Co-authored-by). Thank you!