Repository navigation
perf(chat): bound paginated transcript reads - #8235
Conversation
b5f14c1 to
155c6e3
Compare
155c6e3 to
c9b40b5
Compare
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Scroll-during-stream regresses: every append invalidates the sparse index, so pagination on an active session rebuilds O(total rows) per page, ~2× the old full-read latency. WatchPer the doc's own contract ("Any file-stamp or invalidation-generation change rebuilds the index from byte zero") and the PR's own numbers (index build 201.8 ms vs the 97.7 ms full-read baseline at 10k rows), a session that keeps appending while the user pages backward pays a full rebuild per page — the exact "repeats complete-history parsing as users load older pages" harm this PR names, now with a ~2× constant, and it lands precisely when the gateway is busiest. The transcript is append-only under the process's own writer, so the discarded checkpoints were still valid. [DESIGN-REVIEWED] 44f2cca |
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: make loading older chat-history pages stop scaling with total transcript length (FIX of the measured cost in #8234).
[FIRST-PRINCIPLES-REVIEWED] 44f2cca |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5 Review (fork) — ✅ no blocking findingsReviewed Review detailsA live-streaming slot never publishes the new bounded page, so paginated reads cost four discarded compositions plus the full reader they were meant to replace. FINDING — src/kiro_crew/dashboard/chat_handlers.py:2711 — the retry version keys on [OPUS-REVIEWED] 44f2cca |
c9b40b5 to
da2c5af
Compare
|
da2c5af to
21029b9
Compare
|
|
d336b03 to
cdccc6f
Compare
|
|
cixuuz
left a comment
There was a problem hiding this comment.
The sparse-index warmup is selected by off-loop execution rather than by a paging consumer, so every worker-thread full read (including consolidation, summaries, restore, backfill, and fork flows) pays the per-line offset-accounting cost even when it never pages. The PR’s own 10k-row numbers show roughly 97.7 ms to 141.7 ms (+45%) for this path. Please scope warming to paginated/dashboard keys, or measure and explicitly accept the overhead on the heaviest full-read consumer before merging.
bolichen97
left a comment
There was a problem hiding this comment.
Maintainer review (agent-assisted, on behalf of @bolichen97), head cdccc6f19 (includes the maintainer merge commit that resolved the history_projection.py import conflict). The page math is correct and verified: _read_message_range picks the largest checkpoint ≤ start, counts only decoded message rows and stops before end (no off-by-one at stride or page edges); _row_index checkpoints every 128th message row at its record start offset, matching _frames_with_offsets; the stale-index risk is closed by keying the cache on (mtime_ns, size, ino, dev) + generation, re-statting before publish, and re-verifying every chain member after the read. One blocker:
- Blocking —
_message_record(history_projection.py@+683):except ValueErroralso swallowsUnicodeDecodeError(aValueErrorsubclass —json.loads(bytes)decodes first), so an undecodable record is silently SKIPPED by the index and the range reader:totalis undercounted and the page shifts, which is exactly what the spec's new strict-posture paragraph says cannot happen, while the full reader (_read_messages_locked, strict utf-8handle.read()) fails closed on the same file.test_invalid_utf8_is_not_a_valid_truncated_pagepasses only because its fixture is ~200 bytes and the metadata text read trips first. CatchUnicodeDecodeErrorseparately and fail closed the way the full reader does; add a test with an invalid byte past the first page.
Please fold in: _read_messages_locked (@+1037..+1041) now re-encodes the whole corpus line by line (len(frame.encode("utf-8"))) to warm an index most sessions never page — by the body's own numbers the 10k-row full read goes from ~97.7 ms to 141.7 ms ("Full read + index", ~+45%), paid on restore/search/export, and the body does not name it as a regression; use len(frame) when frame.isascii() (the writer emits ASCII-only rows) or frame the bytes once and decode the buffer after. The bounded loop in chat_handlers.py (@+2299..+2306) retries four times on deterministic failures (OversizedRecord, a missing file, a genuine bug) at debug level, re-scanning the file each time — narrow to (TranscriptRevisionChanged, OversizedRecord, OSError, ValueError) or log at warning on the final attempt. The two index builders frame differently (str.splitlines vs the universal-newline byte framer) — unreachable today with ensure_ascii=True, worth an isascii() guard or a comment so a future writer cannot reopen it. Nits: except OSError: raise is a no-op; _TranscriptRowIndexEntry is inserted mid-sentence into the #: measurement comment in history_cache.py. Auto-Merge disarmed; will re-review on the next head.
|
Not approving for now: bolichen97's latest review is still CHANGES_REQUESTED, so the outstanding feedback on the bounded read path needs resolving first. |
|
A maintainer attempted a conflict-only rebase of this PR onto main What blocks a mechanical rebase:
Once rebased onto current main with the above reconciled, readiness should re-evaluate on the new head. |
|
Pushed bolichen97 -- blocker: bolichen97 #2 / cixuuz / Design Review -- restore-time warming taxes every full read. Resolved by removing restore-time warming entirely. bolichen97 #3 -- retry loop in bolichen97 #4 -- two index builders framing differently. Moot: the second builder was removed with warming; bolichen97 #5 -- nits. Removed the no-op Validation: 2,544 tests across history, JSONL, pagination, snapshot, chat-persistence and dashboard-chat suites; mypy, flake8, black, isort, docs-lint and comment-history gates clean; single commit on current main.
|
|
Pushed
The Backend shard-4 failures (Linux and Windows) and the cascading Coverage Gate on Validation: 2,545 tests across history, JSONL, pagination, snapshot, chat-persistence and dashboard-chat suites; mypy, flake8, black, isort and comment-history gates clean. |
|
Pushed |
bolichen97
left a comment
There was a problem hiding this comment.
Re-reviewed at head 51da907. The blocker is closed: _message_record re-raises UnicodeDecodeError ahead of the ValueError catch, the row-250-of-300 regression plus the endpoint 503 test pin it, restore-time index warming is gone so full reads are back to the upstream loop, OversizedRecord goes straight to the cap-free reader, and the framer offsets check out under an independent randomized walk (contiguous, seek-resume exact). 860 tests in test_history_pagination_projection.py + test_dashboard_chat.py pass locally; the branch merges cleanly onto current main.
One thing left from item #3 before I approve: a key with NO transcript file makes read_messages_chained_page raise TranscriptRevisionChanged on every path — the missing-file stamp is stored as (0, 0, 0, 0) (history_projection.py:715-719) but re-stat'ed as None (:843-845, :864-866), so "stable" can never be true — while read_messages_chained returns [] for the same key. Through api_chat_slot_detail that means every ?limit= request on a not-yet-flushed slot (a fresh tab; the embed SDK always paginates) burns four attempts and logs a WARNING with traceback before the full reader answers correctly — the deterministic case the earlier review asked not to retry, now at warning level. Please make a missing file a stable empty revision (add a no-file read_messages_chained_page test expecting total=0), and route the deterministic raw-vs-durable prefix mismatch (chat_handlers.py:2338) to the full reader like OversizedRecord (break, no retry, no warning).
Nits, non-blocking: json.loads(bytes) decodes with surrogatepass / BOM sniffing, so an encoded surrogate or BOM row is counted by the index but not by the strict full reader — raw.decode("utf-8") first makes the two identical; _read_messages_locked still carries newline="" despite the "byte-for-byte" note; the limit=1 total probe decodes a row it does not need; and the bound is window-sized (a session grown within one gateway lifetime has _disk_older_count == 0 until restore/trim) — worth one sentence in the body so the perf claim is not over-read. Will re-review on the next head.
|
@bolichen97 thanks for the re-review. Head Requested
Nits
Validation on the rebased tree: 2,062 tests across the history, JSONL and dashboard-chat suites; mypy, flake8, isort, black baseline gate, comment-history and docs lint clean. All four new regression tests were verified red on |
|
Pushed |
|
Pushed
The Backend Lint failure on Validation on the rebased tree: 2,064 tests across the history, JSONL and dashboard-chat suites; black baseline gate, mypy, flake8, isort, comment-history and docs lint clean. |
|
Pushed
Not this PR's, for the record: the Backend Lint black step's exit 143 ("runner has received a shutdown signal") is hitting every fork PR's lint job on 2,065 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8 and comment-history gates clean. |
|
Pushed
Not this PR's: Backend Lint's black step exit 143 on 2,067 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8, isort and comment-history gates clean. |
|
Pushed
Backend Lint's black exit 143 on 2,069 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8, isort, comment-history and docs lint clean. |
|
Pushed
The other seven failures on |
|
Pushed @bolichen97 both items from your 09-23 review are in this head, unchanged since
The nits landed too: strict The Opus 5 lane on
Validation on |
|
Pushed
Windows shard 1 on Validation on |
bolichen97
left a comment
There was a problem hiding this comment.
Re-reviewed at head 44f2ccab. Both items from the 2026-09-23 review are closed, verified against the source at this head rather than from the disposition notes:
Item 1 — a missing file is now a stable empty revision. _row_index returns _TranscriptRowIndexEntry(None, generation, 0, ()) when _file_stamp reports no file (history_projection.py:755-764), and ChainRevision is widened to tuple[tuple[str, FileStamp | None, int], ...] (:76), so the stability re-stat compares None == None and holds. The (0, 0, 0, 0) stand-in is gone from production and survives only as a deliberate mismatch fixture in a test. test_missing_transcript_file_is_a_stable_empty_revision pins total=0, messages=[], has_more=False, revision=((key, None, gen),), that the revision pins across a second read, and that read_messages_chained returns [] for the same key — the two-path parity the review asked for.
Item 2 — the deterministic prefix mismatch goes straight to the full reader. _DurablePrefixMismatch is raised at the guard (chat_handlers.py:2383-2386) and caught structurally identically to OversizedRecord / SplitlinesBoundaryRecord: logger.debug then break, no retry and no warning (:2760-2765 vs :2754-2759). Covered by test_prefix_counter_mismatch_uses_full_reader_without_retry_or_warning and test_never_flushed_slot_pages_without_retry_or_warning. _durable_prefix_counter returning -1 for a stand-in without the counter is a sensible extra: it fails the guard rather than passing it vacuously.
All four nits are addressed. raw.decode("utf-8") precedes any JSON parsing with the BOM / surrogatepass rationale in the docstring (history_projection.py:712); newline="" is gone; the total probe is limit=0 and decodes no rows (:2390); and the body now states the bound is window-sized, with the _disk_older_count == 0 caveat.
The 09-08 ordering note is moot — #8862 is closed unmerged, so there is no longer a stack to sequence behind this.
State at this head: PR Readiness success (2026-09-25T23:54:35Z), no failing, pending or cancelled check-run, mergeable=true. Approving; native auto-merge (squash) is already armed, so no admin override and no bypass of branch protection.
Problem / Motivation
Chat history is paginated at the API boundary but not at the storage boundary. Each bounded slot-detail request currently parses the complete chained JSONL history before selecting the requested rows.
This makes gateway work scale with total conversation length and repeats complete-history parsing as users load older pages.
Why it matters
Long agent conversations commonly contain thousands of messages and large tool results. Repeated complete-history parsing increases page latency, gateway CPU, and peak memory despite the browser requesting only a bounded window.
For a 10,000-row transcript, the existing full-read baseline was approximately 97.7 ms and 10.7 MiB per page request. With this change the first paginated read of a revision builds the sparse index in approximately 200 ms (one byte scan, off the event loop) and every later page completes in approximately 3.77 ms and 70 KiB. Full reads (restore, search, export, consolidation) are unchanged: they never build or touch the index.
What changed (motivation → approach → change)
This change keeps JSONL as the authoritative transcript while adding a gateway-owned sparse read projection:
UnicodeDecodeError(it is aValueErrorsubclass, so it is caught before the JSON-shape catch) and the endpoint answers503 history_corpus_unreadableinstead of silently dropping rows.ConversationLog.read_messages_chained_pagewith existingtotal,before,next_before, andhas_moresemantics.The sparse index is deliberately not persisted. It contains only row counts and byte offsets and never trusts agent-writable derived state.
This change is independent of #7916: that PR handles archive reachability and browser scroll stability, while this PR bounds gateway storage work for the current chained transcript. If archive pagination lands later, its archive segments should be incorporated into this bounded projection rather than returning to complete-corpus reads.
Tests
Added and updated coverage for:
before,next_before,has_more, and total semantics;503 history_corpus_unreadableresponse;Validation:
Manual verification
Reader-level profiling:
The full-read column is the upstream baseline (only measured at 10,000 rows); this PR does not modify that code path.
The first paginated read of a revision performs O(total rows) index construction. This PR does not claim constant-time process-cold opening. Restore-time warming was removed after review (bolichen97, cixuuz, Design Review): it taxed every full read by roughly 45% to save one index build.
The bound is window-sized, not row-sized: the indexed durable prefix is the part of the transcript already trimmed out of the resident window (
_disk_older_count). A session that grew within one gateway lifetime has_disk_older_count == 0until restore or trim, so its paginated reads still cover the whole resident window in memory; the disk-side saving above applies once the transcript has been restored or trimmed.Related Issues
Fixes #8234.
Checklist
Contribution License Agreement
The exact CLA wording will be supplied by OSPO before the first public PR.