Skip to content

perf(search): memoize transcript-search counts and snippets - #20127

Merged
buluoray merged 2 commits into
mainfrom
perf/transcript-search-python
Oct 14, 2026
Merged

buluoray merged 2 commits into
mainfrom
perf/transcript-search-python

Conversation

@buluoray

@buluoray buluoray commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make transcript search and the session list faster in pure Python, with identical results.

Every keystroke in the history search runs search_sessions over up to 500 recent sessions. On current origin/main, profiling one search pass (cProfile, 120 queries over the synthetic corpus below) showed where the time goes:

  • 48% building snippets: each returned row decompressed and JSON-parsed the session's whole stored text (zlib.decompress 6.1 s and json 3.7 s of 28.6 s).
  • 12% fetching each candidate's folded document from the index, plus str.count over it (1.9 s).
  • 20% list_sessions and per-session Path building, existence probes and stats (ConversationLog._path: 78,000 calls, 3.5 s).

Why it matters

Search holds the GIL for nearly all of this work, so while one search runs, other dashboard requests wait behind it. With 4 concurrent searches on main, p95 is 454 ms on distinct queries and 859 ms on repeated ones (below).

Not a goal

  • No Rust or other native code.
  • No change to ranking, matching, snippets, or which sessions are scanned.
  • No change to the slot-detail render or to other GIL consumers.

What changed (motivation → approach → change)

Most of the per-query work depends only on a session's text and the query. So the approach is to address each session by a digest of its content and remember what was derived from it, instead of re-deriving it.

  • The search index stores a 16-byte digest of each session's message list, and keeps the raw texts in compressed chunks of whole messages (index schema 4; existing indexes are rebuilt once on upgrade by the normal backfill).
  • Needle and phrase counts are memoized under digest + needle, and snippets under digest + query. Both memos are the existing _SearchTextCache with admission control (8 MiB and 2 MiB per ConversationLog); its refusal warning is off for them, because reaching the ceiling costs only recomputation.
    • A session is scored without fetching its document only when every count the query needs is already in the memo (_MemoGate.answered_by). Disqualifying a session stays in the scoring loop; the gate does not repeat that rule.
    • Otherwise the document is read before any memoized count is used, in a statement that also checks that the row still carries the digest, so a reindex during a search cannot file counts under the digest of text they did not come from.
    • A snippet is memoized only when built from stored chunks read under the same digest check.
    • Once a memo refuses admissions, entries whose digest no longer belongs to a window session are released.
  • A snippet decompresses stored chunks only up to its first match.
  • Index freshness stats use the session's file name directly instead of building and probing a Path per session.
  • list_sessions builds no Path per directory entry. A first line that is JSON but not metadata is cached as "no metadata", which _read_metadata already does, so later listings stop reopening those files.

Sessions the index does not vouch for take the existing scan path unchanged.

A full memo admits no new entries; it keeps the entries it already holds and only releases those whose text is gone. So once the memo is full, a new set of queries is not memoized and runs at the memo-off cost. Results stay correct either way. Replacing entries by query recency was not tried or measured here.

Measured (head 08efd29)

Base for every comparison: the merge base 70d2fa6aa0b11ac0caa44e0c77668f95b798f9ba, labelled "main" below.

The harness builds a deterministic synthetic corpus (300 sessions, mixed ASCII / CJK / Hangul / forge refs / case-folding cases), indexes it, and calls search_sessions in-process. Each tree gets its own copy of the corpus. Three rounds per workload, with the three variants interleaved inside each round; medians are shown. Load average at the start of each round ranged from 3.5 to 8.2 on 32 cores. The harness is not part of this PR; its source is in a comment below. "Memo off" sets both memo ceilings to zero, and the harness asserts that neither memo exists.

Two workloads:

  • distinct: one warm-up pass over the 425 parity queries, then 200 measured calls walking the same list. No query repeats inside a pass, but the measured pass repeats queries from the warm-up pass, so with the memo on, whatever fit in the memo during warm-up is answered from it. The memo-off row is the cold cost.
  • repeat: cycles 9 queries. This is the best case for the memo.

Distinct queries (ms per query):

variant 1 thread p50 1 thread p95 4 threads p50 4 threads p95 CPU / query (1 thread)
main 107.6 115.3 317.4 371.2 107.0
this PR, memo off 53.9 57.8 229.2 256.4 54.0
this PR, memo on 24.1 64.0 111.5 280.8 32.5

Repeated queries (ms per query):

variant 1 thread p50 1 thread p95 4 threads p50 4 threads p95 CPU / query (1 thread)
main 125.7 152.1 373.1 451.6 114.4
this PR, memo off 73.7 94.5 278.7 362.6 65.2
this PR, memo on 10.4 16.0 76.5 108.6 10.3

On the distinct workload the memo lowers p50 but its p95 is higher than memo off (64.0 vs 57.8 ms on 1 thread, 280.8 vs 256.4 ms on 4 threads), in all three rounds. The cause was not isolated. Both memo rows are still below main.

list_sessions over the same 300 files (distinct rounds): p50 6.96 ms → 2.37 ms, and CPU per call 5.93 ms → 2.35 ms.

Memo memory, measured with tracemalloc by clearing both memos after one pass over the 425 parity queries:

  • At the default ceilings, both memos filled: they charged 10,485,458 bytes (count 8,388,486 of 8,388,608; snippet 2,096,972 of 2,097,152), and clearing them released 8,670,624 bytes.
  • With the count memo's ceiling set to 1 MiB: they charged 3,145,438 bytes, and clearing them released 2,751,388 bytes.

Parity

The 425 parity queries were run on the merge base and on this head, with the memos on and with both memos off:

  • single words, prefixes and multi-word queries;
  • CJK runs, forge references and case-folding cases;
  • short terms and queries with no hits.

Each query was run twice (cold, then warm), and the list_sessions output was compared too. The JSON dumps are byte-identical: 0 differences across 18,669 result rows, including 18,667 snippets.

Backwards compatibility

Compatible: the session index is derived data. A schema-3 index is dropped and rebuilt by the existing backfill, and until a session is re-indexed, search scans it the way it does today.

Dependencies: none added, removed or changed. The diff touches only src/kiro_crew/history_cache.py, src/kiro_crew/history_index.py, src/kiro_crew/history_search.py, test/test_history_index.py and test/test_search_sessions_memo.py; no manifest or lockfile is in it. The Dependency License Gate on this head failed with "Dependency review is not supported on this repository" (repository setting: Dependency graph), before it compared any dependency.

The raw_texts column moves into the new session_text table. SessionSearchIndex.raw_texts is replaced by iter_raw_texts, and SessionSearchIndex.shortlist now returns an IndexRow per key instead of a bare rowid.

Reader: src/kiro_crew/history_search.py:SessionCatalogProjection._snippet_texts -- chat|app -- uses iter_raw_texts; test_early_snippet_match_decompresses_only_its_chunk
Reader: src/kiro_crew/history_search.py:SessionCatalogProjection._index_shortlist / _folded_for -- chat|app -- take IndexRow; test_session_search_index_equivalence.py
Reader: src/kiro_crew/history_index.py:SessionSearchIndex.fresh_keys -- subagent|cron (backfill) -- still returns key -> rowid; test_history_index.py fresh_keys tests
Reader: test/test_history_index.py raw_texts tests -- release -- read through iter_raw_texts

Tests

New test/test_search_sessions_memo.py (9 tests):

  • a repeat query reads no document;
  • a repeat query decompresses no snippet text;
  • memo-off keeps reading documents;
  • an early snippet match decompresses less than a quarter of the session;
  • chunked text round-trips;
  • a full memo stays within its budget and keeps its early hits;
  • a full memo drops entries for changed text (this fails with the prune disabled);
  • a reindex during a search does not poison an identical twin session (this fails on 0a40937, an earlier head of this PR);
  • a listing builds no Path per file.

On origin/main's src, 8 of the 9 fail. The twin test passes there, because main has no memo to poison.

Other tests run:

  • test/test_search_sessions_memo.py and test/test_history_index.py on this head: 48 passed.
  • python3 scripts/local-gate.py on this head: all selected gates passed.
  • CI on this head: every Backend Tests shard (Linux 3.12, Windows, macOS, IPv6, namespace sandbox, crew container), Backend Lint & Type Check, Coverage Gate and the E2E jobs passed.

Manual verification

The parity run and the benchmark described above, both on this head. The corpus is synthetic; nothing was measured on a real home.

Related Issues

None.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 14, 2026 04:53
@buluoray
buluoray requested a review from mbriones98 October 14, 2026 04:53
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 08efd293daf327342bc0ec0d286a8e67bfb2df9f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Check that the one-time search index rebuild after the schema-4 upgrade doesn't make history search look empty or slow while it runs.

[UX-REVIEWED] 08efd29

Description read

[DESCRIPTION-READ] 74ec1009dc876fba126792df5d71156929d8ee36913235664b346187264f314b

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 14, 2026
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of 08efd293daf327342bc0ec0d286a8e67bfb2df9f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The description promises a skip the code no longer makes: memoized counts that disqualify a session still trigger a document fetch.

Watch

  • The description claims behaviour the head commit removed. The description says "A session whose counts are all memoized, or whose memoized counts already disqualify it, is scored without fetching its document." At this head, _MemoGate.answered_by returns all(key in hits for key in self.memo_keys). It ignores its _title_folded argument, and the docstring says "Session disqualification belongs to the scoring loop, not this gate." That matches commit 08efd293d, "require complete memo hits before skipping reads". So a session with one memoized zero count for a required needle, and other counts not yet memoized, still pays search_index.document(...). That is the 12% fetch cost the PR says it removes. The benchmark was measured at 2bef257ca, so the numbers may not describe this head. Results are unaffected; the gap is that the claim and the measurements may not match the code being merged.
    Clears when: either remove the disqualification sentence (and the dead _title_folded parameter) and re-run the benchmark and parity at this head, or restore the disqualification check with a test that pins it.

[DESIGN-REVIEWED] 08efd29

Description read

[DESCRIPTION-READ] 77c96da798ef9aabec1c9f36ee89db4dbed901bd961d3c6e571c9ba39a10ce9b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ human override accepted

Human judgment by @buluoray overrides the GPT 6.1 finding for 08efd293daf327342bc0ec0d286a8e67bfb2df9f; the recorded reason is authoritative for this commit.

[GPT-OVERRIDE] 08efd29

This comment is updated in place on each push.

The model was not re-run because an authorized human decision supersedes it.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 08efd293daf327342bc0ec0d286a8e67bfb2df9f: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 08efd293daf327342bc0ec0d286a8e67bfb2df9f — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

_MemoGate.answered_by's _title_folded parameter does nothing, and the description still promises a skip-on-disqualify path that this head removed.

Not justified as shipped

  • Item 8 — zero consumers: the description says "or whose memoized counts already disqualify it, is scored without fetching its document". The diff's gate reads "Session disqualification belongs to the scoring loop, not this gate", and answered_by(self, hits, _title_folded) ignores its second argument. Grep answered_by|title_folded in history_search.py: 1 call site (:1139), 0 reads of the parameter.
    Subtraction: delete _title_folded from answered_by, and the title_folded parameter _ContentCounts.__init__ keeps only to forward it.
    Clears when: both parameters are deleted, so the gate's signature matches the all-keys-held rule it enforces.
  • Item 9 — undeclared: a stored chunk that fails to decode partway through a snippet now finishes from the transcript (_texts_or_file). The description never mentions it.

What this change ships

Inventory (9 items) — 7 justified

Intent: make history search and the session list faster while returning identical results. This is an ADDITION (a performance change).

  1. Repeated queries reuse per-session needle and phrase counts (8 MiB memo, on by default) — justified
  2. Repeated queries reuse snippets (2 MiB memo, on by default) — justified
  3. Index schema 4 stores texts in chunks, existing indexes rebuild once, and a snippet decompresses only up to its match — justified
  4. The index stores a content digest per session, used as the memo key — justified
  5. Freshness checks stat the file name directly instead of building a Path — justified
  6. list_sessions builds no Path per file — justified
  7. A non-metadata JSON first line is cached as "no metadata" — justified
  8. The memo gate takes a title argument it never reads — zero consumers, left over from the dropped disqualify path
  9. A snippet falls back to the transcript partway through when a chunk is unreadable — undeclared

[FIRST-PRINCIPLES-REVIEWED] 08efd29

Description read

[DESCRIPTION-READ] 5b74ce8cc1ccb7044c5f3f6095db3b898ffaff73eb25fca7b35bed433d1c7dfc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 08efd293daf327342bc0ec0d286a8e67bfb2df9f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 08efd29

Verdict parsed from the review's SHA-scoped output markers for commit 08efd293daf327342bc0ec0d286a8e67bfb2df9f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 08efd293daf327342bc0ec0d286a8e67bfb2df9f: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
…t digest

Transcript search spent most of each query re-deriving answers that only
depend on a session's text and the query: counting every needle over the
folded document, and building each snippet by decompressing and parsing
the session's whole stored text.

- The search index stores a 16-byte digest of each session's message list
  and keeps its raw texts in compressed chunks of whole messages
  (index schema 4; existing indexes rebuild once on upgrade).
- Needle and phrase counts are memoized under digest + needle, and
  snippets under digest + query, in _SearchTextCache memos with admission
  control (8 MiB and 2 MiB per ConversationLog). A query whose counts are
  all memoized, or that the memoized counts already disqualify, never
  fetches the folded document. Otherwise the document is read before any
  memoized count is used, and only if its row still carries the digest,
  so a reindex during a search cannot file counts under the digest of
  text they did not come from. Snippets are built for the memo only from
  stored chunks read under the same digest check.
- Once a memo refuses admissions, entries whose digest no longer belongs
  to a window session are released.
- A snippet decompresses stored chunks only up to its first match.
- Index freshness stats use the session's file name directly instead of
  building and probing a Path per session.
- list_sessions builds no Path per directory entry, and caches a first
  line that is JSON but not metadata as no metadata, as _read_metadata
  already does, so later listings stop reopening those files.
@buluoray
buluoray force-pushed the perf/transcript-search-python branch from 0a40937 to 2bef257 Compare October 14, 2026 06:11
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

Benchmark and parity harness used for the numbers in the description (not part of the PR). Run corpus, then parity / bench / memo with each tree's src first on PYTHONPATH.

#!/usr/bin/env python3
"""Repeatable benchmark and parity harness for transcript search.

Builds a deterministic synthetic session corpus, indexes it, and drives
``ConversationLog.search_sessions`` / ``list_sessions`` in-process. It never
touches a live Kiro Crew home: every path comes from ``--dir``.

Subcommands::

    corpus  --dir D                 build D/sessions (idempotent per seed)
    queries                         print the parity query set, one per line
    parity  --dir D --out F.json    run every parity query once, dump results
    bench   --dir D --out F.json    latency/CPU: 1 thread, then 4 threads
    memo    --dir D                 memo occupancy after a full parity pass

Run the same subcommand against two source trees by putting the tree's
``src`` first on ``PYTHONPATH``; the corpus directory can be shared.

``bench --memo off`` sets the count memo budget to zero when the tree has one,
so the memo's contribution is measured separately from the rest of the change.
``--workload repeat`` cycles 9 queries (memo-favourable); ``--workload distinct``
walks the parity set so no query repeats inside a pass (memo-hostile).
"""

from __future__ import annotations

import argparse
import gc
import json
import os
import random
import statistics
import sys
import threading
import time
from pathlib import Path

SEED = 20261013
N_SESSIONS = 300
#: Fixed "now" so recency scores are identical across runs and trees.
FIXED_NOW = 1_790_000_000.0

WORDS = (
    "deploy pipeline gateway session search index memory latency render "
    "snippet budget thread worker review commit branch rebase merge config "
    "token stream chunk frame socket timeout retry backoff cache eviction "
    "admission sqlite trigram tokenizer casefold unicode vector embedding "
    "model prompt agent crew member goal report dashboard sidebar folder "
    "widget button label toggle approval share server stub process pid "
    "heap arena malloc trim rss pss swap page fault kernel cgroup limit "
    "pytest fixture monkeypatch assert flaky shard ratchet coverage lint "
    "mypy flake8 isort black ruff vitest playwright screenshot pod port "
    "the a of to and in is it that for on with as was at by be this have "
    "from or one had not but what all were when we there can an your which"
).split()
# Written as escapes because the repository forbids literal Chinese characters.
CJK = (
    "\u641c\u7d22\u4f1a\u8bdd\u5185\u5b58\u5ef6\u8fdf\u6e32\u67d3\u7f13\u5b58\u7ebf\u7a0b"
    "\u6a21\u578b\u63d0\u793a\u4ee3\u7406\u6210\u5458\u76ee\u6807\u62a5\u544a\u4fa7\u8fb9"
    "\u680f\u6309\u94ae\u5171\u4eab\u670d\u52a1\u5668\u8fdb\u7a0b\u6d4b\u8bd5\u8986\u76d6\u7387"
)
HANGUL = "\uac80\uc0c9\uc138\uc158\uba54\ubaa8\ub9ac"


def _rng() -> random.Random:
    return random.Random(SEED)


def _sentence(r: random.Random, n: int) -> str:
    parts = []
    for _ in range(n):
        x = r.random()
        if x < 0.04:
            parts.append("".join(r.choice(CJK) for _ in range(r.randint(2, 6))))
        elif x < 0.06:
            parts.append(f"#{r.randint(100, 20999)}")
        elif x < 0.07:
            parts.append(f"https://github.com/kirodotdev/KiroCrew/pull/{r.randint(100, 20999)}")
        elif x < 0.09:
            parts.append(str(r.randint(0, 99999)))
        elif x < 0.095:
            parts.append("".join(r.choice(HANGUL) for _ in range(2)))
        elif x < 0.10:
            parts.append(r.choice(["Straße", "İstanbul", "ΣΊΣΥΦΟΣ", "naïve", "café"]))
        else:
            w = r.choice(WORDS)
            parts.append(w.capitalize() if r.random() < 0.1 else w)
    return " ".join(parts)


def build_corpus(root: Path) -> Path:
    sessions = root / "sessions"
    marker = root / f".corpus-{SEED}-{N_SESSIONS}"
    if marker.exists():
        return sessions
    sessions.mkdir(parents=True, exist_ok=True)
    r = _rng()
    for i in range(N_SESSIONS):
        key = f"dashboard_chat-{i}-{1700000000 + i}"
        # Heavy-tailed size: most sessions small, a few long.
        n_msgs = min(4000, int(r.lognormvariate(4.3, 1.1)) + 2)
        lines = []
        if r.random() < 0.8:
            lines.append(
                json.dumps(
                    {
                        "_type": "metadata",
                        "title": _sentence(r, r.randint(2, 7)),
                        "created_at": "2026-09-01T00:00:00",
                        "agent": r.choice(["kiro", "kirocrew-worker", "custom"]),
                    },
                    ensure_ascii=False,
                )
            )
        for m in range(n_msgs):
            role = "user" if m % 2 == 0 else "assistant"
            length = int(r.lognormvariate(3.6, 1.0)) + 1
            content = _sentence(r, min(length, 1500))
            rec = {"role": role, "content": content, "ts": 1700000000 + m}
            if role == "assistant" and r.random() < 0.3:
                rec["meta"] = {"tool": "read", "output": _sentence(r, r.randint(20, 400))}
            lines.append(json.dumps(rec, ensure_ascii=False))
        path = sessions / f"{key}.jsonl"
        path.write_text("\n".join(lines) + "\n", encoding="utf-8")
        mtime = FIXED_NOW - 86400 * 3 - i * 3600
        os.utime(path, (mtime, mtime))
    marker.write_text("ok")
    return sessions


def parity_queries() -> list[str]:
    r = random.Random(SEED + 1)
    out: list[str] = []
    seen: set[str] = set()

    def add(q: str) -> None:
        if q and q not in seen:
            seen.add(q)
            out.append(q)

    for w in WORDS:
        add(w)
        if len(w) > 4:
            add(w[:3])
            add(w[: len(w) - 1])
    while len(out) < 300:
        add(" ".join(r.choice(WORDS) for _ in range(r.randint(2, 4))))
    for _ in range(40):
        add("".join(r.choice(CJK) for _ in range(r.randint(1, 5))))
    for _ in range(25):
        n = r.randint(100, 20999)
        add(r.choice([f"#{n}", f"pr {n}", f"pull/{n}", str(n), f"{r.choice(WORDS)} {n}"]))
    for q in (
        "Straße",
        "STRASSE",
        "istanbul",
        "σίσυφος",
        "naive",
        "café",
        "\uac80\uc0c9",
        "5",
        "s3",
        "pr",
    ):
        add(q)
    for _ in range(30):
        add("zzq" + "".join(r.choice("abcdefghij") for _ in range(5)))  # no hits
    for _ in range(20):
        add(f"{r.choice(WORDS)} {''.join(r.choice(CJK) for _ in range(2))}")
    return out


REPEAT_QUERIES = [
    "deploy",
    "gateway timeout",
    "\u641c\u7d22",
    "#4411",
    "cache eviction",
    "sqlite",
    "review commit branch",
    "the",
    "pytest flaky",
]


def _open(root: Path, memo: str):
    from kiro_crew import history, history_search

    history_search._time.time = lambda: FIXED_NOW  # type: ignore[attr-defined]
    if memo == "off":
        for name in ("_SEARCH_COUNT_MEMO_BYTES",):
            if hasattr(history_search, name):
                setattr(history_search, name, 0)
    log = history.ConversationLog(base_dir=root / "sessions")
    # Index everything; the quiet window is measured against the real clock and
    # the corpus mtimes are days old, so nothing is deferred.
    while True:
        stats = log._catalog_projection.backfill_index(budget_secs=600.0)
        if not stats["remaining"]:
            break
    return log


def cmd_parity(args) -> None:
    root = Path(args.dir)
    log = _open(root, args.memo)
    results = {}
    for q in parity_queries():
        rows = log.search_sessions(q, 50)
        results[q] = rows
    # Second pass: warm memos must give the same answers as cold.
    for q in parity_queries():
        if log.search_sessions(q, 50) != results[q]:
            results[q + "  [warm-differs]"] = True
    results["__list_sessions__"] = log.list_sessions()
    Path(args.out).write_text(json.dumps(results, ensure_ascii=False, sort_keys=True, indent=0))
    print(f"parity: {len(parity_queries())} queries -> {args.out}")


def _pct(xs: list[float], p: float) -> float:
    xs = sorted(xs)
    if not xs:
        return float("nan")
    k = max(0, min(len(xs) - 1, int(round(p / 100 * (len(xs) - 1)))))
    return xs[k]


def _workload(kind: str) -> list[str]:
    return REPEAT_QUERIES if kind == "repeat" else parity_queries()


def _run(log, queries: list[str], n: int, threads: int) -> dict:
    lat: list[float] = []
    lock = threading.Lock()
    per_thread = n // threads

    def worker(offset: int) -> None:
        mine = []
        for i in range(per_thread):
            q = queries[(offset + i * threads) % len(queries)]
            t0 = time.perf_counter()
            log.search_sessions(q, 50)
            mine.append((time.perf_counter() - t0) * 1000)
        with lock:
            lat.extend(mine)

    cpu0 = time.process_time()
    wall0 = time.perf_counter()
    ts = [threading.Thread(target=worker, args=(k,)) for k in range(threads)]
    for t in ts:
        t.start()
    for t in ts:
        t.join()
    wall = time.perf_counter() - wall0
    cpu = time.process_time() - cpu0
    return {
        "threads": threads,
        "n": len(lat),
        "p50_ms": round(statistics.median(lat), 2),
        "p95_ms": round(_pct(lat, 95), 2),
        "max_ms": round(max(lat), 2),
        "wall_s": round(wall, 3),
        "cpu_s": round(cpu, 3),
        "cpu_ms_per_query": round(cpu * 1000 / len(lat), 2),
    }


def _list_bench(log, n: int) -> dict:
    lat = []
    cpu0 = time.process_time()
    for _ in range(n):
        t0 = time.perf_counter()
        log.list_sessions()
        lat.append((time.perf_counter() - t0) * 1000)
    cpu = time.process_time() - cpu0
    return {
        "n": n,
        "p50_ms": round(statistics.median(lat), 3),
        "p95_ms": round(_pct(lat, 95), 3),
        "cpu_ms_per_call": round(cpu * 1000 / n, 3),
    }


def cmd_bench(args) -> None:
    root = Path(args.dir)
    log = _open(root, args.memo)
    queries = _workload(args.workload)
    # One warm-up pass so both trees start from warm OS page cache and warm
    # process-level caches; memo state is whatever that pass leaves.
    for q in queries:
        log.search_sessions(q, 50)
    gc.collect()
    out = {
        "tree": args.label,
        "memo": args.memo,
        "workload": args.workload,
        "single": _run(log, queries, args.n, 1),
        "four": _run(log, queries, args.n, 4),
        "list_sessions": _list_bench(log, 200),
    }
    line = json.dumps(out)
    print(line)
    if args.out:
        with open(args.out, "a", encoding="utf-8") as f:
            f.write(line + "\n")


def cmd_memo(args) -> None:
    """Real memory held by the two memos, against what they charge.

    Fills the memos with one pass over the parity set, then measures with
    tracemalloc how much memory clearing them releases. ``--count-budget``
    overrides the count memo's ceiling, to show the bound holding.
    """
    import tracemalloc

    from kiro_crew import history_search

    if args.count_budget is not None:
        history_search._SEARCH_COUNT_MEMO_BYTES = args.count_budget
    root = Path(args.dir)
    log = _open(root, "on")
    proj = log._catalog_projection
    qs = parity_queries()
    tracemalloc.start()
    for q in qs:
        log.search_sessions(q, 50)
    gc.collect()
    before = tracemalloc.get_traced_memory()[0]
    counts = proj._count_memo.stats()
    snippets = proj._snippet_memo.stats()
    proj._count_memo.clear()
    proj._snippet_memo.clear()
    gc.collect()
    after = tracemalloc.get_traced_memory()[0]
    print(
        json.dumps(
            {
                "queries": len(qs),
                "count_memo": counts,
                "snippet_memo": snippets,
                "released_bytes": before - after,
            }
        )
    )


def main() -> None:
    ap = argparse.ArgumentParser()
    sub = ap.add_subparsers(dest="cmd", required=True)
    p = sub.add_parser("corpus")
    p.add_argument("--dir", required=True)
    sub.add_parser("queries")
    p = sub.add_parser("parity")
    p.add_argument("--dir", required=True)
    p.add_argument("--out", required=True)
    p.add_argument("--memo", choices=("on", "off"), default="on")
    p = sub.add_parser("bench")
    p.add_argument("--dir", required=True)
    p.add_argument("--out")
    p.add_argument("--label", default="")
    p.add_argument("--memo", choices=("on", "off"), default="on")
    p.add_argument("--workload", choices=("repeat", "distinct"), default="distinct")
    p.add_argument("-n", type=int, default=400)
    p = sub.add_parser("memo")
    p.add_argument("--dir", required=True)
    p.add_argument("--count-budget", type=int)
    args = ap.parse_args()
    if args.cmd == "corpus":
        s = build_corpus(Path(args.dir))
        files = list(s.glob("*.jsonl"))
        print(f"{len(files)} sessions, {sum(f.stat().st_size for f in files) / 1e6:.1f} MB")
    elif args.cmd == "queries":
        for q in parity_queries():
            print(q)
    elif args.cmd == "parity":
        cmd_parity(args)
    elif args.cmd == "bench":
        cmd_bench(args)
    elif args.cmd == "memo":
        cmd_memo(args)


if __name__ == "__main__":
    sys.exit(main())

@buluoray

Copy link
Copy Markdown
Contributor Author

Measurement correction for 2bef257: the published harness sets only _SEARCH_COUNT_MEMO_BYTES to zero for --memo off. It leaves the snippet memo enabled. The table rows labelled "memo off" are therefore count-memo-off measurements, NOT both-memos-off measurements; I withdraw the claim that they establish the fully uncached cost. The harness now disables both memos and asserts both are absent. All 425 parity queries remain byte-identical with both off. A fresh three-round, interleaved comparison is running, with continuous host-load samples and a full-hit-only ablation. The opening Why it matters paragraph also contains stale baseline values (454/859 ms); the existing measured table gives 437.2/560.9 ms. I will replace the stale figures with the corrected run.

First Principles advisory: the early-disqualify gate avoids fetching a document for a candidate that scoring will reject before reaching later needles. Those later counts are never populated by scoring, so a full-hit-only gate may keep fetching that document on every repeat. This explains the code path, but its isolated benefit was not measured in the published table. The new ablation changes only answered_by to require all memo keys; its measurements will decide whether retaining the extra rule is justified.

Opus advisory: saturation on unchanged content is real. Admission control intentionally preserves existing hits rather than letting a cyclic scan evict them before reuse, and the current regression test pins that behavior. Pruning only dead content digests does not adapt to a new query working set. Query-aware replacement has not been benchmarked here; this PR must not be read as making every new query cacheable after saturation. The cache remains bounded and results stay correct, but accepting that retention policy is a maintainer decision, not a dismissed finding.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

First Principles follow-up: fixed in 08efd29. _MemoGate.answered_by now checks only that every memo key is present; it no longer duplicates the scoring rejection rule. GPT 6.1 Sol and Opus 5.5 local read-only reviews of the working tree returned no findings. This push keeps two commits and leaves auto-merge disabled.

The attempted corrected-description file upload was blocked by host policy. I did not retry or send that file through another route. The owner must publish the prepared description; the current online description still describes the prior gate and retains the already-withdrawn memo-off labels. The correction comment above remains applicable.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 14, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 08efd29
  • What rule changed: No security, sandbox, secret, redaction or hook rule changed. This PR touches only src/kiro_crew/history_cache.py, history_index.py, history_search.py and two test files, and none of them matches a SENSITIVE_GLOBS entry in .github/scripts/sensitive_change_review.py. The failed Sensitive Change Review run on this head stopped at "Could not read the PR from the API" and never reached the path check.
  • Before -> after (what was allowed/blocked, now): Nothing newly allowed or blocked. Transcript search returns the same sessions, ranking and snippets as main; only the cost of computing them changes, through bounded per-ConversationLog memos keyed by a content digest.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): A wrong memo entry could make search show a stale count or snippet for a session whose text changed. It cannot run a command, expose a secret, or lock a user out. Memo entries are only stored after a digest-checked read, and test_reindex_during_a_search_does_not_poison_a_twin pins the reindex race.
  • Evidence checked (which sandbox result proves it): Running sensitive_files() from the gate script over this PR's five changed paths at 08efd29 returns an empty list. The 425-query parity run against the merge base 70d2fa6 gave byte-identical output with the memos on and with both memos off.
  • No regression:
    • Tools still work (which tools/commands were run, result): pytest test/test_search_sessions_memo.py test/test_history_index.py on this head: 48 passed. python3 scripts/local-gate.py: all selected gates passed.
    • Backward compatible (old config / old data / old callers still work, how checked): A schema-3 search index is dropped and rebuilt by the existing backfill, and until a session is re-indexed it takes main's scan path. No config key, dependency or user-facing setting is added or removed.
    • Existing tests (which suites passed on this head): The search, listing, index and search-cache test files pass locally on this head, and the backend CI suites on 08efd29 completed without a test failure.
  • How to undo: Revert the two commits. The next start rebuilds the search index at the previous schema through the same backfill, and no user data needs migrating.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 14, 2026
@buluoray
buluoray enabled auto-merge (squash) October 14, 2026 18:26
@buluoray

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt 08efd29: The flagged branch (src/kiro_crew/slack/events.py mismatch-command reply) is not in this PR's diff -- it landed on main in #18550 (a4c0852) and is absent from this head, which only touches history_cache/history_index/history_search and their tests; it runs no command, exposes nothing, and is logged by logger.warning, so the missing SEL entry belongs in a separate fix on main.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@buluoray marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 08efd293daf327342bc0ec0d286a8e67bfb2df9f.

The flagged branch (src/kiro_crew/slack/events.py mismatch-command reply) is not in this PR's diff -- it landed on main in #18550 (a4c0852) and is absent from this head, which only touches history_cache/history_index/history_search and their tests; it runs no command, exposes nothing, and is logged by logger.warning, so the missing SEL entry belongs in a separate fix on main.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Oct 14, 2026

@mbriones98 mbriones98 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks green and readiness passed at 08efd29; no behavior change.

@buluoray
buluoray merged commit 349e4df into main Oct 14, 2026
115 of 131 checks passed
@buluoray
buluoray deleted the perf/transcript-search-python branch October 14, 2026 19:47
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants