Skip to content

feat(chat): show the prompt-cache hit ratio in the context popover - #20033

Closed
javenciu wants to merge 1 commit into
mainfrom
feat/17985-context-cache-hit-ratio
Closed

javenciu wants to merge 1 commit into
mainfrom
feat/17985-context-cache-hit-ratio

Conversation

@javenciu

@javenciu javenciu commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Let a user see, in the composer's context popover, how much of the session's prompt the model served from its prompt cache.

The popover (ContextUsageControl, website/src/components/chat-input/ContextShelf.tsx:230) shows used, remaining and total tokens, the model and the auto-compact threshold. It does not say whether the prompt cache is working. Backends that report cache counts already send them: the crew log usage fold sums cache reads and writes (_usage_step, src/kiro_crew/crew_log/projection.py:1915) and the dashboard serves that fold, but nothing in the chat UI turns the counts into a ratio. A user who sees more token use than expected cannot tell whether the prompt prefix is reused or rebuilt every turn.

This is item 4 of #4316, split out as #17985.

Why it matters

A low hit ratio on a long session is the sign that something in the injected prompt changes every turn and defeats the cache, which costs tokens or credits. The context popover is where a user already checks what a turn costs in window terms, so the ratio belongs beside those numbers instead of on a page they have to navigate to.

Not a goal

  • A per-turn or historical view of the ratio (Overview or Telemetry). The row is the session total only.
  • Estimating a ratio for a backend that reports no cache counts. Those sessions show no row.
  • Changing how backends report cache tokens or how the usage fold accumulates them. This change reads the fold as it is.
  • The Activity "Context" tab (website/src/pages/ContextBreakdownPanel.tsx), which charts prompt block sizes and is a different surface.
  • A tooltip or explainer on the row.

What changed (motivation → approach → change)

Approach. The numbers already reach the browser. GET /api/sessions/{id}/crew-log/projections serves the usage fold, api.sessionCrewLogProjections (website/src/api/client/telemetry.ts:73) reads it, and a pushed session_projection frame updates that cache entry (applySessionProjection, website/src/hooks/websocket/sessionProjection.ts:62). The popover reads the same entry, so there is no backend change and no new transport. The fold's four token counts are disjoint (_usage_step adds each on its own, src/kiro_crew/crew_log/projection_usage.py:391, and _usage_render totals them, :655), so the ratio is cache_read / (cache_read + cache_write + input), where input is the uncached input.

Change.

  • cacheHitRatio (website/src/components/chat-input/cacheHitRatio.ts:25) is a pure function over the cached read. Each count is coerced to a finite, non-negative number, so a malformed value counts as zero and cannot poison the rest. reportsTokens (:40) says whether the backend has sent any token count at all, from the fold's own count of the turns that did (turns.tokens_reported, which counts a turn only when one of its four counts is above zero, src/kiro_crew/crew_log/projection_usage.py:394-:397).
  • useCacheHitRatio (:51) takes the slot and returns { ratio, failed, reported }, with the same 30 s staleness as the auto-compact threshold beside it. failed is true when the read errored with nothing earlier to show (isError && data === undefined), so a read that fails after a good one keeps the rate it had. It reaches the cache key through the useWebSocket facade's crewLogProjectionsKey (website/src/hooks/useWebSocket.ts:90).
  • CacheHitRate (website/src/components/chat-input/ContextShelf.tsx:208) draws a bordered group of its own below the window-token rows: the label Cache hit rate, the percentage, and under it Share of the prompt reused from cache (:230-:235). A failed read draws the shared inline ErrorNotice in that slot (:210-:220), the way the auto-compact load error below it does, with the hand-off off under a No hand-off comment because the composer's draft is unsaved. A backend that has sent token counts but no cache count draws one muted line (:222-:228). A backend that sends no token counts draws nothing. The popover mounts it (:298), so the read starts when the popover opens and the composer root holds no query.
  • website/src/components/ChatInput.tsx:1698 passes cacheSlot={sessionOnPeer ? null : activeSlot}, the same peer gate useAutoCompactThreshold uses. A peer session's slot is not this machine's, so it must not read the local slot's fold.
  • Four keys are added to every catalog (components.chatInput.cache_hit_rate, _basis, _load_failed, _no_data), with translator context in en.context.json; en-XA is regenerated with scripts/gen-pseudolocale.mjs.
  • website/scripts/capture-cache-hit-rate.mjs renders the stills under Screenshots from the built app, every API call stubbed, and asserts each frame's content before shooting it.

Which backends fill the row. The counts come from one place, the ACP prompt response (parse_prompt_token_usage, src/kiro_crew/acp/_dispatch.py:4065), which on_turn_completed writes into the crew log (src/kiro_crew/crew_log/emit.py:2755). The recorded responses in test/fixtures/acp_frames/ show who sends cache counts: claude (claude/session.jsonl:17), opencode (opencode/compact-live.jsonl:6) and codex (codex/session-live.jsonl:16, reads only). kiro-cli and KAS, the default (DEFAULT_ACP_BACKEND, src/kiro_crew/agent_sdk/backends.py:319), answer a prompt with a stop reason only (kiro/session.jsonl:12, kas/session.jsonl:12), so their fold holds four zeros and on the default backend the row never renders; the popover is unchanged there. goose sends token counts without cache fields (goose/turn-live.jsonl:10); pi and deepseek send neither. Each backend that sends cache counts sends inputTokens without them (claude's total is its four parts summed, 4 + 164 + 45107 + 39943 = 85218, and opencode's is 12219 + 3 + 1792 = 14014; the codex adapter subtracts cached tokens from its input count), so the denominator holds.

Three faces, never 0%. The ratio is null unless cache_read + cache_write > 0: input tokens without cache counts would compute to 0%, a miss that was never measured, and the fold cannot tell "reported zero cache tokens" from "did not report cache tokens". So the popover says what the backend has sent. Token counts without a cache count draw Cache hit rate: no data yet (a session that reports cache but has read and written nothing so far reads the same). A failed read draws Cache hit rate unavailable. Reopen to retry., and reopening the popover reads again. No token counts at all draw nothing, because nothing will ever fill the row there. The no-data line is muted text with no alert role or icon, because a backend that has sent no cache count has not failed. The failed read is an error, so it renders through the shared inline ErrorNotice (rule errors-use-error-notice), with the hand-off off because the composer's draft is unsaved.

Rejected alternatives.

  • ContextBreakdownPanel: it holds the per-turn block trace and no token counts, and Lessons Visibility & Management UI + Cache Metrics #4316 asks for the popover, "visible every turn without navigating away".
  • Carrying the counts on the context_usage frame: a backend and store change for numbers the dashboard already serves.
  • The latest turn's ratio: the first turn of a session is always cold, so it would read as a failure.
  • Calling the hook in ChatInput: the composer root re-renders on every keystroke, and the facade import then runs in each test that renders it against a minimal useWebSocket mock.
  • no data yet on every backend without a ratio: on kiro-cli and KAS it would never resolve.

Backwards compatibility

Compatible: one new optional prop (cacheSlot) and four new i18n keys. The group renders only when the data reports cache tokens; a muted line can appear where nothing was drawn before, for token counts without a cache count, and an inline error notice for a failed read. A backend that sends no token counts, the default included, draws what it drew before.

Removes nothing: no prop, key, route or stored value is removed or renamed.

Tests

In short: a red-then-green test per behavior, with mutants showing each rule is guarded: the rate and its basis, nothing for a backend with no token counts, a quiet no data yet line, an inline error notice for a failed read that reads again on reopen, the last rate kept, and no fold read on a peer session.

All in website/src/test/ChatInputCoverage.test.tsx, beside the existing popover tests:

  • shows the rate as a share of read, creation and uncached input tokens, with its basis (:918): 750 reads, 150 writes and 100 uncached input tokens draw Cache hit rate, 75% and Share of the prompt reused from cache in the context-cache-hits group.
  • draws nothing for %s, and shows the row once a later frame reports cache tokens (:928), for a backend that sends no token counts and for a fold with no token block: no group, no note, no failure line. A pushed reporting fold (what a session_projection frame does) then makes the group appear, which is the control: the absence comes from the data, not from a group that never renders.
  • says so quietly when input tokens arrive without a cache count, never a 0% row, and shows the row once a later frame reports cache tokens (:946): the no data yet line is muted text with no alert role, icon or button, no 0% is drawn, and the row replaces the line when cache tokens arrive.
  • reads no usage fold on a peer session, whose slot is not this machine's (:964): with sessionOnPeer there is no sessionCrewLogProjections call and no row.
  • shows the inline error notice and no row when the usage read fails, and keeps the other rows (:981): the read rejects, the cache-hit-rate-load-error notice reads Cache hit rate unavailable. Reopen to retry., is the popover's one alert, carries the warning icon and offers no button, and the window header and Used still render.
  • reads again when the popover is reopened after a failed read, which is the retry the line names (:1008): the first open fails, the reopen reads a second time and shows the rate.
  • keeps the last rate and shows no failure line when a later read fails (:1031).
  • reportsTokens (:1043) and cacheHitRatio (:1064) unit cases: malformed input, a fold with no reported turn, and counts that are strings, negative or infinite are all false or null, and one reported turn makes reportsTokens true.

Red on the previous head (5cd8ba0199), 3 of 3 runs of vitest run src/test/ChatInputCoverage.test.tsx -t "prompt-cache hit ratio|reportsTokens" with that head's ContextShelf.tsx put back: 1 failed and 16 passed of the 17 selected. The failure is the failed-read test (:981), Unable to find an accessible element with the role "alert", because that head drew a failed read as a muted div and the repo rule errors-use-error-notice asks for the shared notice. The 16 that pass on both sides pin behavior the change must keep.

Green on the fix, 3 of 3 runs of the whole ChatInputCoverage.test.tsx file (104 tests).

Mutants applied one at a time to the fix, then reverted, each fail only their own tests:

  • Dropping the peer gate in website/src/components/ChatInput.tsx:1698 fails only the peer test.
  • Counting any error as a failure (failed: isError) fails only keeps the last rate and shows no failure line when a later read fails.
  • Never reporting a failure fails the failure-line test and the reopen test.
  • Reporting tokens always fails the two draws nothing cases and the eight false cases of reportsTokens; never reporting them fails the no data yet test and the true case of reportsTokens.
  • Removing the caption fails only the first test.
  • Making the no data yet line loud (a non-muted text colour) fails only the no data yet test.

Every test file that names ChatInput, ContextShelf, ContextUsageControl, cacheHitRatio or crewLogProjectionsKey was run: 159 files, 2839 tests, all pass. tsc -p tsconfig.app.json --noEmit exits 0, eslint --max-warnings 0 on the touched source, test and script files exits 0, and npm run i18n:check with the base set to origin/main exits 0 (19 checks). The whole src/i18n suite (50 files, 1022 tests, including bnStyle.test.ts) passes, and vite build and the capture script run clean on this head.

Manual verification

The stills below come from the built app in Chromium with every API call stubbed. Still required against a live gateway: open the context popover on a session whose backend reports cache tokens (claude, opencode or codex) and read the Cache hit rate group, then open it on a kiro-cli session and confirm nothing is drawn in its place.

Screenshots / video

The popover on a session whose backend reports cache counts, dark and light:

The context popover with the Cache hit rate group, dark theme
The context popover with the Cache hit rate group, light theme

A backend that has sent token counts and no cache count yet: one muted line, no 0% and no alert:

The context popover with the no-data line

A backend that sends no token counts at all (kiro-cli and KAS, the default). Nothing is drawn and nothing is missing:

The context popover without a cache group

The crew-log read failing: the shared inline error notice, which names the next step, with the other rows unchanged:

The context popover with the cache hit rate unavailable line

The longest label of the set (Spanish) wraps to a second line at the popover's width, with the value kept right-aligned and the caption below:

The context popover with the group in Spanish

First-time reader's check of these stills:

  • The group is set apart from Used, Remaining and Total by a divider, in the same muted label and value style as its neighbours. The popover's rows carry no icons, so none is added.
  • The label names a rate, and the caption under the value says what the rate is of: Share of the prompt reused from cache. The number is not read as a share of the 200K window above it.
  • A missing cache count is a muted line and not an alert, because nothing has failed. A failed read is the shared red notice, with no hand-off over the composer's unsaved draft.

Related Issues

Fixes #17985

Open PRs edit the same files: #16906 changes the ContextUsageControl props and popover body and rewrites the <ContextUsageControl /> line in ChatInput.tsx, and #18704 changes the chip's click binding in ContextShelf.tsx. Neither adds a cache row, so the overlap is textual.

Pattern harvest

Not generalizable: this adds a display row and fixes no defect.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners October 14, 2026 01:03
@javenciu
javenciu requested a review from mbriones98 October 14, 2026 01:03
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening right after opening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 5abf57fdf5bae6acf6b906a6787806b12834e929 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the codex adapter's inputTokens really excludes cached reads, because the denominator only holds if the four token counts never overlap.

[DESIGN-REVIEWED] 5abf57f

Description read

[DESCRIPTION-READ] a929c126b019fd23a810136ddbd77a4db813849d8be7b352585efd3018b835ab

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — 🟡 CONCERNS

UX-level review of 5abf57fdf5bae6acf6b906a6787806b12834e929 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

The new row isn't understood: the blind reader couldn't tell what "Cache hit rate" measures, or whether 75% is good or bad.

Watch

  • The number has no meaning for a newcomer. The row shows Cache hit rate and Share of the prompt reused from cache with a bare percentage. The reader said "I don't know what 'cache', 'cache hit rate', 'prompt' or 'reused from cache' mean, or whether 75% is good or bad. Nothing tells me if I should do anything about it." The PR exists so users can spot a low ratio, and that can't happen without a sense of what "low" is. Severity: medium. Everyone who sees the row hits it, every time. Fix: rewrite the caption around what the user gets, e.g. "Higher means fewer tokens spent re-sending the same prompt".
  • The failed read looks more serious than it is. It renders as a red ErrorNotice with a triangle icon, but the translator note in en.context.json calls it "a quiet note about a minor figure, not an error alert". The reader also didn't know what "Reopen" referred to: "reopen what? This panel, the chat, or the whole app?" Severity: low. It's rare, but it alarms the user and the recovery step is unclear. Fix: change the copy to "Close and reopen this panel to retry", or put a Retry action in the notice.

Suggestions

  • In es.json, change cache_hit_rate_basis to "Parte de la petición reutilizada de la caché". The current text leaves "prompt" in English, as shot-04 shows.

[UX-REVIEWED] 5abf57f

Description read

[DESCRIPTION-READ] f8833123704fa16b2aa9d277efc2cb4dae034c17bad62b34638b378c4de20339

That sha256 names everything this verdict read: the PR description, plus the 9 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 5abf57fdf5bae6acf6b906a6787806b12834e929 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The "no data yet" line promises a figure that never comes on goose, the same reason the author gave for keeping it off kiro-cli.

Not justified as shipped

  • Item 2 — inherited: the description itself says the fold "cannot tell 'reported zero cache tokens' from 'did not report cache tokens'", and that "goose sends token counts without cache fields". So on goose Cache hit rate: no data yet stays on screen for good. That breaks the author's own rejected alternative: "no data yet on every backend without a ratio: on kiro-cli and KAS it would never resolve."
    Subtraction: delete the ratio == null && reported branch in CacheHitRate, reportsTokens (one consumer, useCacheHitRatio) and its reported field, and the cache_hit_rate_no_data key from all 16 catalogs plus en.context.json. Every no-ratio success then draws nothing, as it already does on kiro-cli.
    Clears when: the branch is deleted, or a recorded goose/opencode frame shows tokens_reported > 0 with zero cache counts on a backend that later sends them.

What this change ships

Inventory (5 items) — 4 justified

Intent: let a user see in the context popover how much of the session's prompt came from the prompt cache (#17985). ADDITION.

  1. Popover shows a "Cache hit rate" percentage with a basis caption when the backend reports cache counts — justified
  2. A muted "Cache hit rate: no data yet" line when tokens arrive without cache counts — inherited; never resolves on goose
  3. An inline error notice when the crew-log read fails; reopening the popover retries — justified
  4. Peer sessions skip the read (cacheSlot null) — justified
  5. Screenshot harness capture-cache-hit-rate.mjs and its stills — justified

[FIRST-PRINCIPLES-REVIEWED] 5abf57f

Description read

[DESCRIPTION-READ] 63cca73def7d7aa28abc60eb87f9361a27e32c0f426b1e5c70e698d6c0805db8

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 5abf57fdf5bae6acf6b906a6787806b12834e929 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 5abf57f

Verdict parsed from the review's SHA-scoped output markers for commit 5abf57fdf5bae6acf6b906a6787806b12834e929.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 5abf57fdf5bae6acf6b906a6787806b12834e929: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 5abf57fdf5bae6acf6b906a6787806b12834e929 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 5abf57f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 5abf57fdf5bae6acf6b906a6787806b12834e929: <one-sentence reason>

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Thanks for the reviews. One fix commit answers them, and each point is below.

  1. Design, no rendered evidence for the new group. Fixed. Five stills of the popover from this revision are committed under temp-screenshots/17985-cache-hits/ and linked in the description: the group on a backend that reports cache counts (dark and light), the group in Spanish, the popover without the group, and the load-failed notice. website/scripts/capture-cache-hit-rate.mjs produces them from the built app and asserts each frame's content before it is shot.

  2. UX, evidence gaps (the group, and the popover on a backend with no cache counts). Fixed by the same stills: with-row-dark.png and with-row-light.png show the group, without-row.png shows the popover with nothing missing and no 0% row.

  3. UX, the percentage reads as a window figure. Fixed. The rate is in a bordered group of its own below the window-token rows (website/src/components/chat-input/ContextShelf.tsx:205), and the value carries its basis in a caption, Share of input tokens.

  4. UX, relabel to Cache hit rate. Fixed in all 13 catalogs under components.chatInput.cache_hit_rate; the key moved with the wording.

  5. GPT, a failed cache-usage read looked like unsupported reporting (website/src/components/ChatInput.tsx). Fixed. useCacheHitRatio (website/src/components/chat-input/cacheHitRatio.ts:35) reports a read that errored with nothing earlier to show, and the popover renders it as an inline ErrorNotice with askAgent off and a No hand-off comment naming the unsaved composer draft. A read that fails after a good one keeps the rate it had. read-failed.png shows the notice.

  6. First Principles, whether every cache-reporting backend sends input without its cached tokens. Inside the fold the four counts are disjoint, since its total is their sum (src/kiro_crew/crew_log/projection.py:2181). I did not verify each backend's convention at the source: the only producer of the cache counts in this repository is the ACP prompt response (src/kiro_crew/acp/_dispatch.py:3864), and the fold keeps the fields as sent, so a backend that counted cache reads inside inputTokens would make the ratio read low. Changing what a backend reports is outside this change, so the question is put to a maintainer in its own disposition comment.

  7. CI, useWebSocket.ownership.test.ts (outside the owner directory, only the facade imports an owner). Fixed. cacheHitRatio.ts and ChatInputCoverage.test.tsx import crewLogProjectionsKey from the useWebSocket facade, which re-exports it as the owner's own binding (website/src/hooks/useWebSocket.ts:90). The hook now runs in the popover instead of the composer root, so the test files that render the composer against a minimal useWebSocket mock do not read the key.

Gates on the new head: the ownership test, ChatInputCoverage.test.tsx (3 of 3 runs), tsc -p tsconfig.app.json, eslint src/ --max-warnings 0, npm run i18n:check, npm run jscpd and the comment-history check exit 0. ChatInput.approval.test.tsx fails the same three tests on an unmodified origin/main checkout.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=10f93377d1b3

fixed in 45e011a: useCacheHitRatio reports failed for a read that errored with nothing earlier to show (website/src/components/chat-input/cacheHitRatio.ts:35), and the popover renders it as an inline ErrorNotice (website/src/components/chat-input/ContextShelf.tsx:205) with askAgent off and a No hand-off comment naming the unsaved composer draft.
website/src/test/ChatInputCoverage.test.tsx, shows an error notice and no row when the usage read fails, and keeps the other rows, fails on the previous head with Unable to find an element by: [data-testid="cache-hit-rate-load-error"] and passes on the fix, 3 of 3 runs each. The no-counts cases assert the notice stays absent, and keeps the last rate and shows no error notice when a later read fails pins that a read failing after a good one keeps the rate.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=808b297281d7

fixed in 45e011a: the rate has a bordered group of its own below the window-token rows, split off the way the model row is (website/src/components/chat-input/ContextShelf.tsx:205), and the value carries its basis in a caption, Share of input tokens, so the percentage is not read as a share of the context window.
The border was not left to carry the unit by itself: with-row-dark.png and with-row-light.png under temp-screenshots/17985-cache-hits/ show the group, and website/src/test/ChatInputCoverage.test.tsx asserts the caption beside the value.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=5841a5cffb70

fixed in 45e011a: the label reads Cache hit rate in every catalog. The key moved with the wording, components.chatInput.cache_hit_rate, and the caption cache_hit_rate_basis and the failure text cache_hit_rate_load_failed sit beside it, in all 13 catalogs.
npm run i18n:check, the src/i18n tests (including bnStyle.test.ts) and tsc -p tsconfig.app.json pass; en-XA is regenerated by scripts/gen-pseudolocale.mjs.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=dd5e471b76b6

fixed in 45e011a: five stills of the popover rendered from this revision are committed under temp-screenshots/17985-cache-hits/ and linked in the description: the group on a backend that reports cache counts (dark and light), the same group in Spanish, the popover without the group, and the load-failed notice.
They come from website/scripts/capture-cache-hit-rate.mjs, which runs the built app and asserts each frame's content before shooting it. The group sits under its own divider in the same label and value style as Used, Remaining and Total; the Spanish label Tasa de aciertos de caché wraps to a second line at the popover's width with the value kept right-aligned.

@javenciu

Copy link
Copy Markdown
Contributor Author

needs-a-decision: inside the fold the four counts are disjoint, because the fold's own total is their sum ("total": sum(tokens.values()), src/kiro_crew/crew_log/projection.py:2181). Whether that holds at the source depends on each backend reporting inputTokens without its cached tokens. The only producer of the cache counts in this repository is the ACP prompt response (parse_prompt_token_usage, src/kiro_crew/acp/_dispatch.py:3864), and the fold keeps the fields as sent. I did not verify the convention of every backend a user can attach, and one that counted cache reads inside inputTokens would make the ratio read low.
Changing how a backend reports tokens, or how the fold accumulates them, is outside this change (Not a goal). Question for a maintainer: should an adapter normalise inputTokens to exclude cached tokens, or should the fold carry the convention it was sent in?

@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu javenciu reopened this Oct 14, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 14, 2026
@javenciu
javenciu force-pushed the feat/17985-context-cache-hit-ratio branch from 45e011a to ea66ad0 Compare October 14, 2026 11:10
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 14, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

span=9dfc7258376d

  • When the cache-rate read fails, the error is shown in a hand-written text-muted div instead of ErrorNotice. fixed: the failed branch of CacheHitRate now renders <ErrorNotice variant="inline" testId="cache-hit-rate-load-error" message={i18nT('components.chatInput.cache_hit_rate_load_failed')} /> (website/src/components/chat-input/ContextShelf.tsx:210-:220), under the same No hand-off: the composer draft below is unsaved. comment the auto-compact load error carries (:213, :312), so the hand-off stays off over the unsaved draft. This is errors-use-error-notice, and the earlier muted-line wording is gone from the code, the tests and the description. The no-data line is not a failure and stays a muted div (:222-:228). website/src/test/ChatInputCoverage.test.tsx:981 now asserts the notice (the popover's one alert, with the warning icon and no button), and with the previous head's ContextShelf.tsx put back it fails with Unable to find an accessible element with the role "alert", 3 of 3 runs, as the only failure of the 17 tests selected by -t "prompt-cache hit ratio|reportsTokens". The whole ChatInputCoverage.test.tsx passes, 104 of 104, 3 of 3 runs. temp-screenshots/17985-cache-hits/read-failed.png shows the notice.

@javenciu

Copy link
Copy Markdown
Contributor Author
  • Check that the codex adapter removes cached tokens from inputTokens before the fold sees them. kept: checked, and it does. codex-acp 1.11.0 builds its count as inputTokens: usage.inputTokens - usage.cachedInputTokens under the comment "Codex includes cached input tokens in the input token count, so they are subtracted here" (toTokenCount), then sends that as inputTokens and the cached count as cachedReadTokens (toPromptUsage) (TokenCount.ts). Crew takes those two fields as they come: parse_prompt_token_usage returns inputTokens and cachedReadTokens (src/kiro_crew/acp/_dispatch.py:4065), apply_prompt_token_usage accumulates them (src/kiro_crew/acp/types.py:1369), and the crew log writes them as input and cache_read (src/kiro_crew/crew_log/emit.py:2749-:2755). So the denominator in cacheHitRatio (website/src/components/chat-input/cacheHitRatio.ts:25) is uncached input for codex, as it is for claude (test/fixtures/acp_frames/claude/session.jsonl:17, 4 + 164 + 45107 + 39943 = 85218) and opencode (opencode/compact-live.jsonl:6, 12219 + 3 + 1792 = 14014). The recorded codex response (codex/session-live.jsonl:16) has cachedReadTokens: 0 beside inputTokens: 4060, so the capture alone does not exercise the subtraction; the adapter source is the evidence. Nothing changes in this PR. Enforcing the convention for every adapter, instead of observing it, belongs in parse_prompt_token_usage as its own change.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=aec54426c392

  • Item 2: the no data yet line can never fill on goose. kept: the limit is real. goose sends token counts and no cache field in all five usage blocks of its recorded compaction session (test/fixtures/acp_frames/goose/compact-live.jsonl:7, :10, :13, :17, :20), so on goose the line stays for the whole session. It is kept because the state it names is not goose's alone: opencode answers with token counts and no cache field in one recording (opencode/session-live.jsonl:20, 6480 input and 15 output) and with cachedReadTokens: 1792 in another (opencode/compact-live.jsonl:6), and codex's recorded response carries cachedReadTokens: 0 beside 4060 input tokens (codex/session-live.jsonl:16). Those are the backends where the line shows until a response reports cache, and website/src/test/ChatInputCoverage.test.tsx:946 pins the swap to the row when a later frame does. No recording holds both responses of one session, so that swap is pinned by the test and not observed on a backend. The line costs one branch (website/src/components/chat-input/ContextShelf.tsx:222-:228) and one key. This needs a maintainer ruling: the UX review asked for this line at the previous head (finding 066c76c9bbe2 on 605a23fb) and this review asks to delete it. If the ruling is to delete, the change is that branch, reportsTokens, components.chatInput.cache_hit_rate_no_data in the 13 catalogs and en.context.json, and the no-data frame.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=2c00552b04f4

  • Item 6: reportsTokens duplicates turns.tokens_reported. fixed: reportsTokens no longer sums the four counts a second time. It reads the fold's own turns.tokens_reported (website/src/components/chat-input/cacheHitRatio.ts:40), the field the Crew Log tab already uses to decide whether a total was measured (website/src/pages/chat/CrewLogPanel.tsx:142, :358). The fold counts a turn there only when one of its four counts is above zero (src/kiro_crew/crew_log/projection_usage.py:394-:397), and a closer with four zeros writes no token block (src/kiro_crew/crew_log/emit.py:2755), so a backend that sends no counts, kiro-cli and KAS, still reads false. website/src/test/ChatInputCoverage.test.tsx:1043 pins the helper against a missing fold, a missing turns block, a 0, a string, a negative and an infinite count, and one reported turn. The usageRead helper (:896) now seeds tokens_reported the way the fold does, 0 for four zeros. Replacing the helper's body with return true fails 10 of the 17 selected tests and with return false fails 2, each time only tests that read it. The branch it feeds stays (Item 2).

@javenciu

Copy link
Copy Markdown
Contributor Author

span=19d6d1aaa926

  • The rate has no reference point. kept: the caption says what the number is a share of, Share of the prompt reused from cache (website/src/i18n/locales/en.json:4334), drawn under the value (website/src/components/chat-input/ContextShelf.tsx:233); that is the answer to the maintainer's earlier ask that the row explain itself. The suggested Higher is cheaper — reused prompt isn't re-billed is a different kind of text, a billing claim. The row reads token counts only (website/src/components/chat-input/cacheHitRatio.ts:25), and the fold's token counts carry no price, so the caption cannot promise that a reused token is not billed. An explainer on the row is on the Not a goal list and the Goal is frozen, so nothing changes here.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=ee348075987a

  • Fix: let the caption carry the meaning, e.g. Higher is cheaper — reused prompt isn't re-billed. kept: the caption already carries what the number measures, Share of the prompt reused from cache (website/src/i18n/locales/en.json:4334). The proposed sentence adds a billing claim, and the row's data is token counts with no price (website/src/components/chat-input/cacheHitRatio.ts:25), so it cannot show that a reused token is not billed. See the disposition for span 19d6d1aaa926.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=c00b2f1defc6

  • The rate is hard to tell apart from the other percentages. kept: the group is set apart by its own divider (website/src/components/chat-input/ContextShelf.tsx:231), its label names a rate and not a window figure (Cache hit rate, website/src/i18n/locales/en.json:4333), and the line under the value says what the percentage is a share of (ContextShelf.tsx:233, en.json:4334), so 75% is not read as a share of the window above it (temp-screenshots/17985-cache-hits/with-row-dark.png). The fix the review names is the caption change from the first item, which stays as it is for the reasons in the disposition for span 19d6d1aaa926.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=dc541a78ce73

  • Fix: the caption change above also covers this. kept: the caption change it points to is not made (see the disposition for span 19d6d1aaa926), and the current caption plus the group's divider already separate the rate from the window figures (website/src/components/chat-input/ContextShelf.tsx:231-:233; reasons in the disposition for span c00b2f1defc6).

@javenciu

Copy link
Copy Markdown
Contributor Author

span=7cccbb1a3792

  • "Reopen to retry" points to an action with no visible control. kept: reopening is the retry the popover has. CacheHitRate mounts with the popover (website/src/components/chat-input/ContextShelf.tsx:298), so the read starts on every open, and website/src/test/ChatInputCoverage.test.tsx:1008 pins it: the first open fails, the reopen reads a second time and shows the rate. The reader's own guess, closing and opening the pop-up, was the right one. A retry button would add a control and a handler that the Goal does not ask for. The failure renders through the shared inline ErrorNotice (ContextShelf.tsx:210-:220, rule errors-use-error-notice), and its sentence, Cache hit rate unavailable. Reopen to retry. (website/src/i18n/locales/en.json:4335), is the next step the maintainer asked the notice to name.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=f5bf984b9078

  • Fix: use Couldn't load cache hit rate. Close and reopen this panel to retry. kept: the notice already names the one action that retries, reopening (see the disposition for span 7cccbb1a3792). The suggested wording would be a text change in the 13 catalogs this PR touches, with the screenshot and the tests that read the English sentence, to say the same thing at more length. The message renders in the shared inline ErrorNotice (website/src/components/chat-input/ContextShelf.tsx:210-:220, rule errors-use-error-notice) and its text is website/src/i18n/locales/en.json:4335.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=69a8d27db6ab

  • The Spanish Compactación automática al70 % is missing a space, and (70%) drifts from 70 %. kept: both strings are older than this change and it does not touch them. components.chatInput.auto_compact_at is Compactación automática al (website/src/i18n/locales/es.json:6606) and following_global is Sigue el global ({{pct}}%) (:6610); they render at website/src/components/chat-input/ContextShelf.tsx:336 and :362, outside every hunk of this change. The change adds four lines to es.json, the cache_hit_rate keys. The spacing and the percent style belong to the auto-compact row, in a change of their own.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 16, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI runs on hosted runners: same-repo runs from this account queue on the fleet label without a runner (details on #19683).

@javenciu javenciu closed this Oct 16, 2026
@javenciu javenciu reopened this Oct 16, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 16, 2026
@javenciu
javenciu enabled auto-merge (squash) October 16, 2026 17:41
@javenciu

Copy link
Copy Markdown
Contributor Author

span=6d66bbc444b4

  • Item 2, inherited: on goose Cache hit rate: no data yet stays on screen for good. kept: the limit is real, and the previous disposition for this item named it (span aec54426c392). goose sends token counts and no cache field in all six usage blocks it recorded (test/fixtures/acp_frames/goose/compact-live.jsonl:7, :10, :13, :17, :20 and goose/turn-live.jsonl:10; grep -c 'cachedReadTokens\|cachedWriteTokens' finds 0 in both files), so on goose the line never gives way to a rate. The second clearing cannot be met from the repository: of the six recordings that hold a usage block, only goose's compaction session holds more than one response and none of its blocks carries a cache count, so no recording shows a backend that first reports tokens without cache and later reports cache. The swap from the line to the row is pinned by a test (website/src/test/ChatInputCoverage.test.tsx:946) and has not been observed on a backend. The first clearing is a ruling, not a defect. The UX review asked for this line at an earlier head (finding 066c76c9bbe2, head 605a23fb) and this review asks to delete it, so one of the two has to give way, and that call is the maintainers'. If the ruling is to delete, the change is the branch at website/src/components/chat-input/ContextShelf.tsx:222-:228, reportsTokens (website/src/components/chat-input/cacheHitRatio.ts:40) with the reported field that useCacheHitRatio returns (:51, :58), components.chatInput.cache_hit_rate_no_data in the 13 catalogs that carry it, its note at website/src/i18n/en.context.json:218, and the no-data frame. The review counts 16 catalogs; website/src/i18n/locales/ holds 13 that carry the key. Nothing is deleted at this head.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=71842400e86a

  • The number has no meaning for a newcomer. kept: the row says what the number is. The caption under the value reads Share of the prompt reused from cache (website/src/components/chat-input/ContextShelf.tsx:233, website/src/i18n/locales/en.json:4334), which is the answer to the earlier review's ask that the row explain itself. What this review adds is a reading of the number and a reason to act on it, and that is an explainer: the description's Not a goal list excludes A tooltip or explainer on the row. The sentence offered, Higher means fewer tokens spent re-sending the same prompt, claims a benefit the row's data cannot show. The usage fold the row reads has four token counts, input, output, cache_read and cache_write (src/kiro_crew/crew_log/projection_usage.py:70), with no price and no count of tokens saved or re-sent, so nothing on the row could back it. The reason to watch a low rate is written in the description, under Why it matters. This is the same answer as for spans 19d6d1aaa926 and ee348075987a on the previous head.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=d387cd1d48f1

  • The failed read looks more serious than it is. kept: a failed read is an error, and the rule errors-use-error-notice has an error render through ErrorNotice (website/docs/frontend-conventions.md:185), so the red text and the triangle belong to the shared inline notice (website/src/components/ErrorNotice.tsx:290, :294) and are not a choice made for this row. The previous head drew the failure as a muted line and was changed for that rule (spans 9b0949fe2799 and 9dfc7258376d); the rule outranks a quieter look. The notice names its next step, Cache hit rate unavailable. Reopen to retry. (website/src/i18n/locales/en.json:4335): the group mounts with the popover (website/src/components/chat-input/ContextShelf.tsx:298), so every open reads again, and website/src/test/ChatInputCoverage.test.tsx:1008 pins it (the first open fails, the reopen reads a second time and shows the rate). Reopen stays. The longer sentence suggested would change the text in 13 catalogs and in the screenshot and tests that read the English sentence to say the same thing, and a Retry control is a handler the description does not ask for. One point in the review is right: the translator note for this key, website/src/i18n/en.context.json:217, still says Muted status line and not an error alert, which no longer matches what is drawn, and the earlier answer that said the muted wording was gone missed it. The note changes no catalog string and nothing a user sees, and this revision has passed every check, so it is not pushed on its own. The corrected note (an inline error notice with a warning icon, in a plain register, with Reopen as the retry) goes in with the next revision, or before merge if a maintainer asks for it.

@javenciu

Copy link
Copy Markdown
Contributor Author

span=4847d267cf0d

  • In es.json, change cache_hit_rate_basis to Parte de la petición reutilizada de la caché. kept: prompt is the Spanish catalog's word for this term and petición is its word for a request. 75 of the values in website/src/i18n/locales/es.json contain prompt, for example Prompt adicional (:2741) and Optimizar el prompt (:6532), while petición names a request to a service (:625, :1138), so the suggested wording would make the caption a share of the request and not of the prompt. fr (fr.json:6502), de (de.json:6393), it (it.json:6502) and pt (pt.json:6502) keep the same loanword in this caption. Neither website/src/i18n/glossary.json nor website/src/i18n/style/es.md mentions the term, so no rule asks for a Spanish word. The current value is Parte del prompt reutilizada de la caché (es.json:6502).

@javenciu

Copy link
Copy Markdown
Contributor Author
  • Check that the codex adapter's inputTokens really excludes cached reads. kept: checked on the previous head, and the ratio's arithmetic has not changed since. codex-acp 1.11.0 builds inputTokens: usage.inputTokens - usage.cachedInputTokens and sends the cached count separately as cachedReadTokens (TokenCount.ts), so the four counts the row adds never overlap for codex. Since that check, this revision changed only how reportsTokens decides that a backend reports tokens at all (website/src/components/chat-input/cacheHitRatio.ts:40, now the fold's turns.tokens_reported), which decides whether the row shows and is not part of the denominator. The full answer, with the Crew side of the read (parse_prompt_token_usage, apply_prompt_token_usage and the crew-log write), is this comment.

@iamwhatever

Copy link
Copy Markdown
Contributor

Holding at 5abf57f on maintainer decisions. The code reads soundly.

The ratio stays null without cache evidence, the peer gate matches the auto-compact threshold, a failed read keeps an earlier good rate, and the newest test and build runs on this head succeed. Each current-head AI item has a disposition. Two of them hand the call to a maintainer instead of settling it.

What would clear the hold:

  1. First Principles item 2 versus the earlier UX ask. On goose, Cache hit rate: no data yet never resolves. A maintainer needs to decide whether to keep that line or delete the reported branch (disposition).
  2. The denominator assumes every adapter's inputTokens excludes cached tokens. The author asked for a maintainer ruling on that and has not had one.
  3. The PNGs under temp-screenshots/17985-cache-hits/ are still committed on a same-repo branch. Please attach them and drop the files before merge, as you offered.
  4. The translator note at
    "components.chatInput.cache_hit_rate_load_failed": "Muted status line in the context-usage popover when the cache hit rate could not be read. It is a quiet note about a minor figure, not an error alert. 'Reopen' means close the popover and open it again, which reads the figure again.",
    still describes a muted non-error line. Please fix it in the same commit, rather than deferring it with no tracking.
  5. Sensitive Change Review has been queued on this head since 2026-10-16 16:26Z. Auto-merge is armed and would not wait for it.

— crew-pr-reviewer (r108)

@javenciu
javenciu disabled auto-merge October 18, 2026 05:31
@javenciu

Copy link
Copy Markdown
Contributor Author

Closing without merging, to keep my open PRs to the ones close to landing.

The Oct 17 hold on this head (5abf57f) needs two maintainer rulings (keep or drop the "no data yet" line on goose, and whether every adapter's inputTokens excludes cached tokens), the screenshots attached instead of committed, and the translator note at en.context.json L217 fixed.

The cache hit ratio isn't on main as of 2e50054, and #17985 stays open. I've deleted the branch; it can be restored from this page at 5abf57f.

@javenciu javenciu closed this Oct 18, 2026
@javenciu
javenciu deleted the feat/17985-context-cache-hit-ratio branch October 18, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context popover: show the prompt-cache hit ratio for the session

2 participants