Skip to content

fix(context): use a new chat's early memory reads on the default store - #24394

Merged
bolichen97 merged 1 commit into
mainfrom
fix/g3-prefetch-store-key
Oct 26, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/g3-prefetch-store-key

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A new dashboard chat on the default memory store uses the memory it read early, beside its session start, instead of reading memory again on the send path.

The early memory reads #23269 added are never used for a chat on the default store. The turn reserves them under the store name "default"; the build looks them up under "". They miss every time, so the build reads memory again while the prompt waits.

Why it matters

Every new chat on the default store pays for its memory reads on the send path, after the session has started. That is the delay #23269 was meant to remove. On a short first message the wasted time is about 0.3–0.5 s per new chat (bench below).

Not a goal

  • Starting the reads for a session an eager spawn already started (R12 item 1). Measured here, it made those turns slower: the build comes ~0.1 s after the claim, so the early reads are never ready.
  • Starting the reads when the seed or first message arrives, before the turn starts. That is the change that would take a long seed's reads off the send path. It is future work.
  • Making the first message's embedding cheaper.

What changed (motivation → approach → change)

Two small fixes in src/kiro_crew/context_assembly/store_admission.py, the only code file this PR changes.

Before After
The prefetch table keys the raw store name. "default" (turn) and "" (build) miss. _prefetch_key keys the default store as "" whether it is spelled "default", "" or None. A named store keys as itself.
A long first message: the build waits for the running prefetch only until its own 1.0 s embed deadline. The prefetch still needs ~0.15 s of keyword reads, so the build reads again beside it. The build cuts the prefetch's embed deadline to its own, as before. Then it waits for the prefetch up to the 5 s build bound, since what is left is the read it would make itself. A prefetch with no deadline to cut still gets only the build's own deadline.

The memory-skills-hooks spec describes both.

Local bench, send to prompt at KAS, median seconds. Real _run_chat, real build_message, real embedder, on a copy of a live home. Each process ran 3 untimed turns first, so its pool threads were warm, as in a running gateway. The cold start is modelled as a 3 s claim. 9 runs per cell, interleaved.

Case main this PR early reads used
cold start, 120-char first message 3.70 3.25 0/9 → 9/9
cold start, 2,400-char first message 4.55 4.54 0/9 → 9/9
pre-started session, 120 chars 1.11 1.19 no early reads either way
pre-started session, 2,400 chars 1.54 1.53 no early reads either way

The pre-started rows run the same code on both sides. Their gap is a ~0.12 s pause before the build that shows up in both arms in some runs; the build itself took 1.02 s (main) and 1.03 s (this PR).

Known gap. In a fresh process with no warm-up, the second 2,400-char cold start is ~0.1 s slower than main: build 1.41–1.47 s vs 1.33 s (6 runs). Every later run is equal (1.33 s both). Median send to KAS over those warm runs: main 4.48 s, this PR 4.53 s. Left as is.

Backwards compatibility

Compatible: the prefetch table is process-local and never stored, and a named store keys exactly as before.

Removes nothing: no key, field, path or record is taken away; the default store's two spellings now find the same entry.

Tests

  • test_the_default_store_keys_one_entry_however_it_is_spelled: "default", "" and None key one entry; a named store keys apart.
  • test_a_cold_start_build_takes_its_prefetch_and_sends_the_same_context: a cold start through the real runner and the real build_message, whose session start returns only once the early reads have run. The build takes the prefetch and reads nothing itself, and the prompt sent is byte-for-byte the prompt without a prefetch.
  • test_a_prefetch_whose_embed_wait_is_cut_is_joined_past_the_build_deadline: once its deadline is cut, the join gets the 5 s bound.
  • Determinism: the three tests 20/20 at -n0, 10/10 at -n 2 --dist loadgroup with test_chat_runner_startup_recall.py and test_lesson_turn_injection.py as co-tenants (-n 2: the host's shared test limit), and the whole file in shuffled order, seed 64070, 39 passed.

Proof: PROVEN test/test_session_memory_prefetch.py::test_a_cold_start_build_takes_its_prefetch_and_sends_the_same_context (head 6d5caca)

Manual verification

The bench above, run locally. The bench script and raw output are outside the repo. Semgrep is not installed on this host, so the SAST rules run only in CI.

Related Issues

Closes #24354

Sibling sweep

Swept: git grep -nE "(workspace, memory_store|memory_store or "")|_vector_stores.get(" -- src/
Sibling: src/kiro_crew/context_assembly/store_admission.py:124 -- not affected -- private member stores are always named, never "default"
Sibling: src/kiro_crew/context.py:1375 -- not affected -- _target_key already folds "default" into the V1 store

Pattern harvest

Not generalizable: one process-local table keyed by a store name that two callers spell differently; the shared resolver _target_key already folds it everywhere else.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners October 26, 2026 08:06
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 6d5caca33e9204f767a13bedeb7bd0d130323375 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 6d5caca33e9204f767a13bedeb7bd0d130323375 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the fix is a key normalisation at the table's single owner, and the longer join waits only for the read the build would otherwise run again itself.

Notes

  • The second change can make the send path wait up to _PROMPT_BUILD_EMBED_TIMEOUT_SECS (5 s) for a prefetch whose embed deadline was cut, where before it waited about 1 s. The description's "Known gap" (~0.1 s slower in a fresh process) is the only measured cost of this. If keyword reads stall, the turn now waits up to 5 s before the build reads again on its own.

[DESIGN-REVIEWED] 6d5caca

Description read

[DESCRIPTION-READ] 94750c749149b6e07a56324e5cf924149409c7250c40a6fa4506a74da71d207a

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 6d5caca33e9204f767a13bedeb7bd0d130323375 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The key fix is aimed at the cause. Item 2, which lets the join wait up to 5 s, rests on a gain the author's own bench does not show.

Notes

  • Item 2 — oversized: the bench shows "cold start, 2,400-char first message | 4.55 | 4.54", and a fresh process is "~0.1 s slower than main". The worst-case wait goes from the build's own deadline to _PROMPT_BUILD_EMBED_TIMEOUT_SECS. The 120-char gain comes from item 1 alone. Subtraction: delete the else: split in _take_memory_prefetch and keep bound = min(bound, own.deadline - time.monotonic()) unconditional, along with test_a_prefetch_whose_embed_wait_is_cut_is_joined_past_the_build_deadline and its spec sentence. Clears when: the mechanism is removed, or a bench isolating item 2 shows a gain.

What this change ships

Inventory (3 items) — 2 justified

Intent: a new chat on the default memory store should use the memory it read early instead of reading it again while the prompt waits. FIX.

  1. A new default-store chat now uses its early memory reads; before, the store names "default" and "" never matched — justified
  2. When the early reads are still running, the build waits for them up to 5 s instead of only up to its own embed deadline — oversized
  3. The spec paragraph describes the shared key and the new wait — justified

[FIRST-PRINCIPLES-REVIEWED] 6d5caca

Description read

[DESCRIPTION-READ] 7b3dbc15b45f8ac5acff08181624cb889f79349b35caac8d14aa341839347c7f

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 6d5caca33e9204f767a13bedeb7bd0d130323375 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6d5caca

Verdict parsed from the review's SHA-scoped output markers for commit 6d5caca33e9204f767a13bedeb7bd0d130323375.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 6d5caca33e9204f767a13bedeb7bd0d130323375 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 6d5caca33e9204f767a13bedeb7bd0d130323375 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6d5caca

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 6d5caca33e9204f767a13bedeb7bd0d130323375 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 26, 2026
The early memory reads a dashboard cold start begins beside session/new were
never taken for a chat on the default memory store: the turn reserved them
under its bindings' store name "default", and the build looked them up under
the execution record's legacy name "". The prefetch table now keys the default
store under one name.

With the key matching, a long first message exposed a second gap: the build
joined a running prefetch only until its own embed deadline, so a prefetch
still doing its keyword reads past that deadline was read again beside it.
Once the build cuts the prefetch's embed deadline to its own, the join keeps
the long bound, since what is left is the read the build would make itself.

Closes #24354
@iamwhatever
iamwhatever force-pushed the fix/g3-prefetch-store-key branch from 17f1ce6 to 6d5caca Compare October 26, 2026 08:36
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 26, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 6d5caca. This looks good to ship. I did not approve yet because the readiness signal on this head is still pending. A maintainer approval remains once it reports passed.

The cause is clear: the turn reserves the prefetch under the store name "default" and the build takes it under "". _prefetch_key folds the default store to one spelling at the single table owner, and every reserve, discard, prefetch and take path now goes through it. A named store keys exactly as before.

What I checked:

  • DEFAULT_MEMORY_STORE is "default" at this head, so the fold matches the turn's bindings.
  • The prefetch runs only for non-member sessions, so a member store cannot share an entry with the default store.
  • The cold-start test runs the real runner and the real build_message, and asserts the sent prompt is byte-for-byte the no-prefetch prompt. On the old key the build reads memory itself, so the test goes red.
  • The join test asserts the bound is _PROMPT_BUILD_EMBED_TIMEOUT_SECS after the cut. The old code gives 0.05, so it goes red too.
  • Security: no change to auth, input parsing, trust boundaries, sandboxing, approval gates or secrets.

The longer join after the cut can wait up to 5 s if the keyword reads stall. A stalled read would also stall the build's own read on the same pool, so I accept the tradeoff as written.

This PR and #15953 both change store_admission.py and the memory spec. Merging this small fix first leaves #15953 to rebase.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 26, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at head 6d5caca. Checked on this head: readiness passed, all required checks green, no blocking AI review lane, no outstanding change requests, mergeable against main. The default store fix for early memory reads, its regression test and the spec update read correctly.

@bolichen97
bolichen97 enabled auto-merge (squash) October 26, 2026 10:09
@bolichen97
bolichen97 merged commit 78ebe6d into main Oct 26, 2026
111 checks passed
@bolichen97
bolichen97 deleted the fix/g3-prefetch-store-key branch October 26, 2026 10:56
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(context): a new chat's early memory reads are never used on the default store

2 participants