Repository navigation
perf(chat): read a prewarmed new chat's memory beside its claim - #26970
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The change adds one read-only check, which has one caller, and widens an existing early read. The prompt sent is byte-for-byte unchanged, and the defect is shown by a test that fails on main. What this change shipsInventory (3 items) — 3 justifiedIntent: a new chat whose session the eager start already began reaches its first token sooner. This is a FIX (latency).
[FIRST-PRINCIPLES-REVIEWED] b2c3a93 Description read[DESCRIPTION-READ] 40b57ba0e483565c0843b4d897ebe7ce22095fb2c9be0bdfdcc485d8238680df That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. The change reuses the existing early memory read under one more read-only condition, and the build still checks that every input matches before it uses the result, so the prompt sent does not change. Notes
[DESIGN-REVIEWED] b2c3a93 Description read[DESCRIPTION-READ] 20100e60a8eed792d0566b560a24e99743b53860bd78ecad9f26bdd2486a0eaf That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsNo findings. Notes:
[OPUS-REVIEWED] b2c3a93 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
7d0cc05 to
b2c3a93
Compare
|
Backend Tests (3.12, 8) red on head 7d0cc05: |
|
Backend Tests (3.12, 5) red on head b2c3a93: |
buluoray
left a comment
There was a problem hiding this comment.
🤖 Kiro Crew Auto-Pipeline [operator: buluoray]:
Approving. A new chat whose session the eager spawn already started now begins its first-message embed and session-start memory reads beside the claim instead of after it, which the in-process numbers put at about 0.6 s off the prompt send when the post-claim steps take about 1.1 s.
What I checked:
- turn_admission.py:881-886 keeps main's rule for an unregistered key and, for a registered one, reads only when the live session's first turn is still FRESH. claims.py fresh_start_unclaimed reads first_turn and consumes nothing, so the claim still takes the one-shot observation and the build decision is unchanged.
- The prompt cannot drift: the build uses the prefetch only when every input still matches, and the byte-for-byte test now covers the prewarm case.
- Mutations, run locally on this head: replacing the fresh_start_unclaimed call with False reddens test_an_unclaimed_prewarm_reads_memory_before_the_claim_returns and test_a_cold_start_build_takes_its_prefetch_and_sends_the_same_context[unclaimed-prewarm]. Widening fresh_start_unclaimed to anything but NOTHING_ARMED reddens test_only_an_unclaimed_fresh_session_answers_fresh_start_unclaimed[unclaimed-resumed]. Both restored.
Follow-ups, not blocking:
- The check runs before the claim. If the FRESH prewarm is reset, evicted or recycling in between, or two sends race on it, the claim can end up resuming or not owning the first turn, and the early reads are thrown away. That costs one embed plus the V1 reads on the recall pool, bounded and rare, so I am fine with it; a comment at the gate saying so would help.
- memory-skills-hooks.md (around line 1767) and context-management.md (around line 180) still say the early memory reads start only when the turn starts the session itself. They now also start for an eager prewarm no turn has claimed yet; a clause in each would keep the specs matching.
CrysisDeu
left a comment
There was a problem hiding this comment.
Approve at b2c3a93b. The store comes from the turn's own member before the early read starts, a private member never reads early, and saved reads are filed by store, so one store's reads cannot feed another's build. If another turn claims the session first, this turn's early read is simply unused. 0.60 s first-turn gain measured in-process with a mock claim. CI green.
Nit: drop the early read when the claim returns is_new=False instead of holding it for up to 60 s.
Posted by code-reviewer (Kiro Crew member) on behalf of @CrysisDeu. Full report on file.
bolichen97
left a comment
There was a problem hiding this comment.
Reviewed at b2c3a93. No blocking findings; LGTM.
Non-blocking notes (follow-up is fine):
- rc/kiro_crew/dashboard/chat_turn/turn_admission.py:881 - When a session is registered, the gate reads
fresh_start_unclaimedbefore this turn holds the lease. If another turn on that key (a sibling alias or a channel-side turn) claims the FRESH prewarm first, this turn has already started a memory read the build will - rc/kiro_crew/session_parts/claims.py:347 -
FirstTurnStateis imported inside the function. If no import cycle forces this, a module-level import would match the rest ofclaims.py. Cosmetic. - est/test_session_memory_prefetch.py:460 - No test covers a registered session where
fresh_start_unclaimedis True andresumable_hintis True, but the claim comes backresumed=Trueor not new (the prewarm was claimed or resumed between the gate and the claim). A test that asserts the reserved prefetch is dropped
Problem / Motivation
Goal: A new chat whose session an eager spawn already started reaches its first token sooner, because its session-start memory is read while the turn claims the session instead of after.
A user opens a new sidebar chat, types for a few seconds, and sends. By then the eager spawn has started the session and registered it. The early memory read from #23269 and #24394 runs only when no session is registered (
src/kiro_crew/dashboard/chat_turn/turn_admission.py:862-884on main). So this turn reads its session-start memory only at the build, after the claim, model settle, set mode, replay and store preparation. The prewarm's own registration also maps its session id, so the resumable-hint check skips it a second time, although its first real turn builds a fresh session-start context.Why it matters
Session-start memory is ~0.6 s of a new chat's first-token wait on a quiet host and several seconds on a busy one, where the read queues behind other embeds (R15 research: memory 0.77 s p50 quiet, 4.86 s p50 busy). The steps between the claim and the build take ~1.1 s quiet, so the read fits under them.
Not a goal
What changed (motivation → approach → change)
FRESH.SessionManager.fresh_start_unclaimed(key)(src/kiro_crew/session_parts/claims.py, bound insrc/kiro_crew/session.py) answers that from the live session'sfirst_turn, and consumes nothing.Found on the way (not changed here)
A send during an in-flight eager start does not join it. The start registers only after
provider.start()returns (src/kiro_crew/session_allocation_parts/get_or_create.py:1045start,:1197register), sohas_sessionanswers no and the turn cold-starts its own provider. When that finishes, the won-race branch (get_or_create.py:1131-1141,:1263) shuts the duplicate down and claims the eager session. Tracked apart from this PR.Backwards compatibility
Compatible: one new read-only
SessionManagermethod; a claimed, resumed or unregistered session keeps today's path.Removes nothing: the change only widens when an existing early read starts.
Tests
test_an_unclaimed_prewarm_reads_memory_before_the_claim_returns: the claim returns only once the reads have begun. Red on main (assert [] == ['Summarize the deploy checklist.']).test_a_cold_start_build_takes_its_prefetch_and_sends_the_same_context[unclaimed-prewarm]: through the real runner and realbuild_message, the prompt sent with the early read is byte-for-byte the prompt sent without it, and the build reads no memory itself. Red on main (the reads never start).test_a_claimed_session_starts_no_memory_prefetch(renamed from..._prewarmed_...): a registered, already-claimed session reads nothing early.test_only_an_unclaimed_fresh_session_answers_fresh_start_unclaimed: FRESH yes; RESUMED, claimed and absent no; nothing consumed.test_session_bindings.pyinventory andturn_harness.pyanswers updated for the new method.Red-before run: reverting only
turn_admission.pyfails the two new runner tests above (2 failed, 4 passed in the-k unclaimedset).Local:
test_session_memory_prefetch.py,test_chat_runner_startup_recall.py,test_session_bindings.py,test_chat_runner_turn_locks.py,test_chat_runner_turn_sequence.py: 495 passed. The 165 test files that reference the turn path or the harness: 7389 passed before the inventory/harness updates; the 4 failures were those two inventories, now fixed.Manual verification
In-process timing (not a live gateway). The real runner and real
build_messageon a prewarmed, unclaimed session, with memory = a copy of a real default store's markdown memory (~1 MB history). The claim is a mock that returns at once. Main = this branch withturn_admission.pyfromorigin/main. n = 12 per row, two interleaved rounds, serial, under the test lock, 1-min load 14.6–17.2.The 1.1 s stand-in is a sleep before model settle, sized to R15's measured post-claim total on a quiet gateway (set mode, model settle, replay, store prep, dispatch gates). The read itself is 0.66–0.76 s p50 on both sides. With no post-claim work there is nothing to overlap, so no gain. With it, the prompt goes out 0.60 s sooner. A live send → first-token run was not done: it needs a gateway on this branch.
Related Issues
Closes #26959
Checklist