Skip to content

perf(dashboard): reuse unchanged rows in the slot-detail render - #20153

Merged
buluoray merged 2 commits into
mainfrom
perf/slot-detail-no-rehash
Oct 14, 2026
Merged

buluoray merged 2 commits into
mainfrom
perf/slot-detail-no-rehash

Conversation

@buluoray

@buluoray buluoray commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make session details faster to open without changing displayed content or redaction.

The slot-detail read (GET /api/chat/slots/{slot}) re-rendered every row on every request: a keyed HMAC per string for the display-redaction cache, the redaction pass, and a json.dumps of the whole history. The independent perf review measured this as the largest remaining GIL consumer (Figures 5.1 and 5.7: _display_redaction_cache_key over every message, plus the full-history dump). A reader re-reads the same page many times (tab switch, reconnect, the refetch after each turn), and almost every row is unchanged between reads.

Why it matters

Repeated reads of unchanged history spend CPU on the same redaction and serialization work, delaying session detail responses.

Not a goal

What changed (motivation → approach → change)

Repeated per-string hashing and whole-history serialization → reuse rendered row JSON under a bounded cache, with deferred filling.

  • New dashboard/wire_render.py keeps each rendered row's JSON text, keyed by a keyed BLAKE2b digest (random per-process key) of the exact source row plus everything else the render reads: the exempt-host set (policy hosts and the workspace's allowed hosts), the token-parameter exempt prefixes, and the live child id for mcp_oauth rows. An unchanged row costs one row encode and one hash; no per-string HMAC, no redaction, no second serialization.
  • The fill stays off the request path. Until a page (session, before cursor, whole-history flag) has been filled, every read of it is the unmemoized render, exactly as before. The first read schedules a fill that runs in a worker thread 1 s later (at most two at once). A fill that cannot start or fails leaves the page unfilled; its next read asks again.
  • slot_detail._render serializes the messages array separately and splices it between the fields before and after it. The body is byte-for-byte the previous json.dumps.
  • chat_utils.py is unchanged.

Security

  • The memo keeps only rendered output, which is already redacted and is what the response sends. Keys are keyed digests, so a kept key can't be checked offline against a guessed plaintext.
  • The key covers a superset of what the existing per-string cache keys on (text + host set + prefixes): every row field, any nesting, any key order, plus the live child for mcp_oauth.
  • Owner-view renders (credential_pass_bypassed()) neither read nor write the memo.
  • On a miss the row is rendered from json.loads of the exact bytes that were hashed, not from the live dict. A window row the event loop changes during the render therefore can't store its output under another row's key.
  • A row json can't encode is rendered as before, without the memo.

Relation to #19753 (redaction verdicts stored on write)

This PR leaves out every chat_utils.py hunk of #19753 (_display_redaction_cache_key, _redact_for_display). The only overlap is the _render closure in slot_detail.py, where #19753 wraps _prepare_messages in display_verdicts.session(verdict_key). Whichever lands second resolves that by wrapping both render branches (_prepare_messages and wire_render.prepare_messages_json) in the same session(...). Then a cold first read still uses the stored verdicts, and memo misses use them too. A memo hit never reaches _redact_for_display, so #19753's read scope counts fewer from_memory hits on such a render. That counter only feeds its cold-open heuristic, so after both land a mostly-hit render may load the sidecar a little earlier. Output is unaffected.

Measured

Synthetic corpus: 3 sessions × 3,000 rows (~3.1 MB each). The rows mix prose, code blocks, tool output and fake credentials. Isolated in-process gateway (aiohttp test server, real ConversationLog under a scratch dir). base = origin/main, new = this branch. Six alternating base/new runs; medians, ms.

Request latency:

read limit=100 base → new limit=500 base → new whole history base → new
1st read (page not yet filled) 15.5 → 16.3 45.1 → 47.4 224 → 227
2nd read, immediately (fill not run yet) 9.7 → 9.9 15.9 → 16.7 48.2 → 50.7
2nd read after fill 9.7 → 9.8 15.5 → 14.1 47.3 → 25.8
warm p50 9.9 → 8.7 16.5 → 12.1 46.3 → 27.7
warm p95 13.5 → 10.8 21.8 → 17.9 72.0 → 39.4

The first two rows run the same code on both sides (the unmemoized render). A second batch of six runs on a busier host put those rows anywhere from −10 % to +5 %, so I read them as noise.

Render time on its own (the work that holds the GIL in the render thread), medians over 15 interleaved iterations:

rows base new (filled) fill (background, once)
100 1.32 0.60 2.31
500 6.34 2.58 11.28
3,000 42.37 17.24 71.82

Memo memory, filled past its cap (16 MiB of charged output): 16.0 MB charged, 15.7 MB traced by tracemalloc, 22 MB of RSS freed when cleared.

Byte-for-byte oracle (turn_gen, which is random per process, masked):

  • Endpoint bodies, base vs new, cold and warm, for limit=100, limit=500 and the whole history over 3 sessions: 18 bodies, 22.6 MB, 0 differences.
  • In process: memoized render vs json.dumps(_prepare_messages(...)) over 4,000 edge rows. These cover every role, structured, None and empty content, lone surrogates, NUL, CJK, RTL, full-width digits, mcp_oauth with live/dead/no child and javascript: URLs, variants, blocked_links with and without the host allowed, unknown row fields, and chunk/done runs. Each was run under 3 live-child values × 2 host sets, cold, at the fill and warm: 18 renders, 0 differences.

Not measured: the review's own load mix (its load.py harness isn't available here), so its GIL-share figures are not reproduced. The numbers above come from the synthetic corpus described.

Tests

  • test/test_slot_detail_render_memo.py (new):
    • output equals json.dumps(_prepare_messages) cold and warm;
    • changing the exempt-host set, the workspace's allowed hosts, the token prefixes or the live child each re-renders;
    • owner view leaves the memo untouched;
    • byte and entry caps hold;
    • the page fill state machine, the fill delay and the fill limit;
    • the endpoint does not digest any message string on a repeat read. On origin/main this test fails: 376 digests for limit=100 and 2,626 for the whole history, against at most 1 (the title).
  • python3 scripts/local-gate.py: backend related tests and 362 frontend test files pass.
  • isort, flake8 and mypy are clean on the changed files.
  • test_flake_pattern_ratchet.py passes.
  • semgrep 1.78.0 with semgrep/ on the diff: 0 findings.
  • Local review ran as a self-review against the reviewer contracts. No subagents were used: sub-agents are not available in this environment.

Risks

  • Background CPU: each page's first read is followed by one extra render about 1 s later, on a worker thread. It costs about 1.7× the warm render once, then each repeat read saves about 60 % of its render time.
  • A refetch inside that 1 s window is the unmemoized render: no fill on the request path, so the cost is the same as before.
  • Memory: at most 16 MiB of charged output (about 22 MB RSS when full), plus up to 4,096 small page records.
  • After a rewind or rewrite the rows differ, so those reads miss and render as before.

Backwards compatibility

Compatible: response bodies and redaction behavior are unchanged.
Removes nothing: all existing fields, pagination and whole-history reads remain available.

Manual verification

The measured endpoint and render oracles above check the display byte for byte. The original review load harness was not available.

Related Issues

Related PR: #19753; the integration boundary is described above.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 14, 2026 06:17
@buluoray
buluoray requested a review from mbriones98 October 14, 2026 06:18
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of a4db64d664c99ce29d18e080a928f240f6964927 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Nothing to check.

[UX-REVIEWED] a4db64d

Description read

[DESCRIPTION-READ] e2e16202b384451f132f5100167ddde0c335637781cb56963ea37b7b73a0bac9

That sha256 names everything this verdict read: the PR description, plus the 13 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of a4db64d664c99ce29d18e080a928f240f6964927 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

The description says mcp_oauth rows are memoized under the live child id, but the code never memoizes them. Correct the description, or remove the leftover live_child key branch.

[DESIGN-REVIEWED] a4db64d

Description read

[DESCRIPTION-READ] 4a5db538768d00fce22251678f7f70c81e51e254183a4a913f061d294d2ec040

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of a4db64d664c99ce29d18e080a928f240f6964927 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The memo key carries an OAuth live-child slot that can never be filled, and the bench measures a smaller "hoisted" fix without reporting the result.

Not justified as shipped

  • Item 4 — zero consumers: the _memo_row key is live_child if m.get("role") == "mcp_oauth" else "". But prepare_messages_json routes role == "mcp_oauth" to _render_row before that line, so the third slot is always "". The Security section says "plus the live child for mcp_oauth", which describes a key that doesn't exist.
    Subtraction: drop the third tuple element from _memo keys and _remember's signature.
    Clears when: the element is deleted.
  • Item 6 — oversized: test/perf/slot_render_bench.py benchmarks a hoisted arm, which computes the per-string key suffix once per render, next to memo. The description reports only base vs memo. A process-global LRU, a scope state machine and delayed background fills ship with no shown margin over the much smaller hoist.
    Subtraction: replace the memo and fill machinery with the hoisted key if its margin is comparable.
    Clears when: the hoisted arm's medians are shown beside memo's and memo's added margin is material.

What this change ships

Inventory (7 items) — 4 justified

Intent: open session details faster without changing what they show or redact. This is an ADDITION (perf).

  1. Repeat reads of unchanged slot-detail pages reuse rendered rows — justified
  2. One extra render on a worker thread, 1 s after a page's first read — justified
  3. Response assembled from spliced JSON parts, bytes unchanged — justified
  4. OAuth live-child id in the memo key — zero consumers, never reached
  5. wire_render.py listed as non-egress in security_posture.py — justified
  6. Benchmark script test/perf/slot_render_bench.py with an unreported hoisted arm — undeclared; oversized
  7. history.md spec section — rides along; spec update the docs rule requires

[FIRST-PRINCIPLES-REVIEWED] a4db64d

Description read

[DESCRIPTION-READ] 5fc78e5f02bca1f26ed5d18dc44b9ba82a8efe90cda0a3589969ac64f064e0d6

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of a4db64d664c99ce29d18e080a928f240f6964927 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a4db64d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt a4db64d664c99ce29d18e080a928f240f6964927: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed a4db64d664c99ce29d18e080a928f240f6964927 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a4db64d

Verdict parsed from the review's SHA-scoped output markers for commit a4db64d664c99ce29d18e080a928f240f6964927.

False positive or not applicable? A repository writer can comment:
/ai-review override fable a4db64d664c99ce29d18e080a928f240f6964927: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@buluoray
buluoray force-pushed the perf/slot-detail-no-rehash branch from 6be14cf to c1c18ae Compare October 14, 2026 07:12
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 14, 2026
Reuse bounded rendered row JSON after a deferred page fill. Keep the
ordinary renderer for unfilled pages and detect changed source rows with
keyed digests. A hit avoids per-string redaction and output encoding,
while source encoding and hashing still occur.

Derive context from the existing string-cache key and distinguish the
workspace-only link restoration scope. Never memoize OAuth rows, so a
revoked endpoint approval is checked on every read. Owner view bypasses
the memo. Classify this internal projection under the existing dashboard
output sink without changing the security posture's egress count.

Add cache-context and OAuth revocation regressions, row-oracle tests,
and a reproducible interleaved render benchmark. Document cache limits
and leave chat_utils unchanged for compatibility with stored verdicts.
@buluoray
buluoray force-pushed the perf/slot-detail-no-rehash branch from c1c18ae to 7002f11 Compare October 14, 2026 07:35
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security_posture.py

  • @buluoray posted a complete reasoning template for a4db64d.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 14, 2026
…e role cannot 500

A hand-edited transcript row whose role is a list or dict made the memoized
render raise TypeError on the set-membership test once the page was filled.
The unmemoized path compares role with == and renders the row; do the same.
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: a4db64d
  • What rule changed: No enforcement rule changed. The security-posture inventory now classifies dashboard/wire_render.py as an internal projection under the existing dashboard response sink.
  • Before -> after (what was allowed/blocked, now): Commands, credentials, access controls and output redaction remain unchanged. The omission detector previously rejected the unclassified module; it now recognizes its documented non-egress classification.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Misclassifying an independent output boundary could hide a missing sink in the posture inventory. This helper only returns JSON fragments to the already registered dashboard response path; it creates no transport or listener.
  • Evidence checked (which sandbox result proves it): N/A to sandbox execution changes: this is inventory metadata only, and no sandbox implementation or policy changed. The omission detector was reproduced failing before the registration and passing afterward; source review verified the helper returns fragments through slot_detail rather than opening its own output path.
  • No regression:
    • Tools still work (which tools/commands were run, result): Targeted pytest for security posture and both slot-render suites passed all 73 cases. No live gateway or production tools were changed or exercised.
    • Backward compatible (old config / old data / old callers still work, how checked): No configuration, storage schema or API changes. Existing endpoint and byte-equivalence cases passed.
    • Existing tests (which suites passed on this head): a4db64d only changes the role comparison in wire_render.py (== instead of set membership) and adds an unhashable-role byte-equivalence test, which fails on 7002f11 and passes here; the five slot-detail suites passed (45 passed, 4 xfailed). On the previous head 7002f11, local-gate passed 3,470 backend cases (4 expected failures), plus 362 frontend files and 11,686 tests. The isolated render oracle found zero byte differences for 100 and 3,000 rows.
  • How to undo: Revert this PR's commit through a new reviewed revert commit. That removes the row memo and its inventory registration together; no stored data migration is required.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026

@mbriones98 mbriones98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks green and readiness passed at a4db64d; no behavior change.

@buluoray
buluoray merged commit dac0e7c into main Oct 14, 2026
103 of 104 checks passed
@buluoray
buluoray deleted the perf/slot-detail-no-rehash branch October 14, 2026 19:12
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants