Skip to content

perf(chat): bound paginated transcript reads - #8235

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/long-thread-loading
Sep 26, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/long-thread-loading

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Chat history is paginated at the API boundary but not at the storage boundary. Each bounded slot-detail request currently parses the complete chained JSONL history before selecting the requested rows.

This makes gateway work scale with total conversation length and repeats complete-history parsing as users load older pages.

Why it matters

Long agent conversations commonly contain thousands of messages and large tool results. Repeated complete-history parsing increases page latency, gateway CPU, and peak memory despite the browser requesting only a bounded window.

For a 10,000-row transcript, the existing full-read baseline was approximately 97.7 ms and 10.7 MiB per page request. With this change the first paginated read of a revision builds the sparse index in approximately 200 ms (one byte scan, off the event loop) and every later page completes in approximately 3.77 ms and 70 KiB. Full reads (restore, search, export, consolidation) are unchanged: they never build or touch the index.

What changed (motivation → approach → change)

This change keeps JSONL as the authoritative transcript while adding a gateway-owned sparse read projection:

  • Adds offset-preserving bounded JSONL framing.
  • Maintains trusted in-memory row checkpoints keyed by full file stamp and history generation.
  • Builds the index lazily on the first paginated read of a revision; authoritative full reads are byte-for-byte the upstream loop and pay nothing for the feature.
  • Fails closed on undecodable transcript bytes: the indexed decoder re-raises UnicodeDecodeError (it is a ValueError subclass, so it is caught before the JSON-shape catch) and the endpoint answers 503 history_corpus_unreadable instead of silently dropping rows.
  • Treats oversized rows as deterministic: the bounded path goes straight to the cap-free full reader without re-scanning; only revision churn and I/O errors are retried, with a warning on the final attempt.
  • Rebuilds the sparse index from byte zero on any file-stamp (mtime_ns, size, inode, device) or invalidation-generation change; only an exact revision reuses checkpoints.
  • Adds ConversationLog.read_messages_chained_page with existing total, before, next_before, and has_more semantics.
  • Pins all durable ranges composing one HTTP response to an exact ordered chain revision.
  • Retries complete page composition after concurrent mutation and falls back to the legacy full-reader oracle after repeated churn.
  • Reconciles bounded durable ranges with the existing event-loop-captured resident and unflushed tail.
  • Switches only explicitly paginated slot-detail requests to the bounded path.
  • Leaves unlimited reads and callers requiring complete history unchanged.
  • Keeps redaction at the existing response-preparation boundary.
  • Documents the index, invalidation, revision, fallback, and trust contracts.

The sparse index is deliberately not persisted. It contains only row counts and byte offsets and never trusts agent-writable derived state.

This change is independent of #7916: that PR handles archive reachability and browser scroll stability, while this PR bounds gateway storage work for the current chained transcript. If archive pagination lands later, its archive segments should be incorporated into this bounded projection rather than returning to complete-corpus reads.

Tests

Added and updated coverage for:

  • exact page equivalence across transcript chains;
  • boundary, before, next_before, has_more, and total semantics;
  • universal newline and byte-offset framing;
  • bounded warm reads over a 10,000-row transcript;
  • fail-closed handling of an invalid UTF-8 byte deep in the transcript (row 250 of 300) in both the indexed and full readers, and the endpoint's 503 history_corpus_unreadable response;
  • oversized rows routing to the full reader without a retry scan;
  • rebuild-from-zero after appends and after larger same-inode rewrites (no stale checkpoint reuse);
  • rebuilds after rewrites and invalidation;
  • concurrent revision changes between response ranges;
  • complete-response retry and legacy fallback;
  • live and unflushed tail ordering;
  • loop-captured tail snapshots;
  • shared cached-list identity;
  • fork, rewind, cache, and locking compatibility.

Validation:

  • 838 affected tests passed
  • Targeted mypy passed
  • Targeted Flake8 passed
  • Black and isort checks passed
  • Documentation lint passed
  • Subprocess-encoding, agent-SDK boundary, sync-I/O, lockdown, brand, harness, lock, testpath, changelog, focus, vendor-manifest, scrub-lint, and CloudFormation gates passed
  • Frontend scoped tests, TypeScript, ESLint, i18n, phantom-class, duplication, production build, and bundle-size gates passed
  • Electron gate: 1,558 passed with zero assertion failures; one unrelated gateway-stop timing test remained pending on this host and Node cancelled it plus 25 descendants
  • Full backend run: 83,756 passed; remaining failures were unrelated host-isolation, inherited command-environment, ownership-topology, and AF_UNIX path limitations. No changed history, JSONL, pagination, fork, or dashboard-chat test failed.

Manual verification

Reader-level profiling:

Rows Full read (unchanged path) First page of a revision (index build) Warm page
100 — 5.308 ms / 75 KiB 2.765 ms / 69 KiB
1,000 — 23.155 ms / 76 KiB 3.014 ms / 70 KiB
10,000 97.7 ms / 10.7 MiB 201.805 ms / 85 KiB 3.766 ms / 70 KiB

The full-read column is the upstream baseline (only measured at 10,000 rows); this PR does not modify that code path.
The first paginated read of a revision performs O(total rows) index construction. This PR does not claim constant-time process-cold opening. Restore-time warming was removed after review (bolichen97, cixuuz, Design Review): it taxed every full read by roughly 45% to save one index build.

The bound is window-sized, not row-sized: the indexed durable prefix is the part of the transcript already trimmed out of the resident window (_disk_older_count). A session that grew within one gateway lifetime has _disk_older_count == 0 until restore or trim, so its paginated reads still cover the whole resident window in memory; the disk-side saving above applies once the transcript has been restored or trimmed.

Related Issues

Fixes #8234.

Checklist

  • One commit with a Conventional Commits title
  • Existing affected tests pass and new tests cover the new behavior
  • Self-review completed; code follows project style guidelines
  • History-system documentation updated
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

The exact CLA wording will be supplied by OSPO before the first public PR.

@Pearcekieser
Pearcekieser requested review from a team and Zedmor September 3, 2026 18:22
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Sep 3, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/long-thread-loading branch 3 times, most recently from b5f14c1 to 155c6e3 Compare September 4, 2026 00:07
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 4, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/long-thread-loading branch from 155c6e3 to c9b40b5 Compare September 4, 2026 01:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of 44f2ccab3b7d89c8f1d15924757ce40a7d722e18 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Scroll-during-stream regresses: every append invalidates the sparse index, so pagination on an active session rebuilds O(total rows) per page, ~2× the old full-read latency.

Watch

Per the doc's own contract ("Any file-stamp or invalidation-generation change rebuilds the index from byte zero") and the PR's own numbers (index build 201.8 ms vs the 97.7 ms full-read baseline at 10k rows), a session that keeps appending while the user pages backward pays a full rebuild per page — the exact "repeats complete-history parsing as users load older pages" harm this PR names, now with a ~2× constant, and it lands precisely when the gateway is busiest. The transcript is append-only under the process's own writer, so the discarded checkpoints were still valid.
Clears when: the index extends across same-inode, size-grew appends instead of rebuilding from byte zero, or a measurement shows page latency during active streaming stays at or under the old full-read baseline.

[DESIGN-REVIEWED] 44f2cca

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of 44f2ccab3b7d89c8f1d15924757ce40a7d722e18 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

TranscriptPage.next_before/has_more ship with zero production consumers — the endpoint recomputes both itself, so the two fields are speculative surface.

Not justified as shipped

  • Item 4 — zero consumers: the description promises "next_before, and has_more semantics", but _bounded_slot_page reads only .messages, .total, .revision; I grepped the patch for .next_before/.has_more on TranscriptPage — 0 non-test reads (only test_history_pagination_projection.py uses them).
    Subtraction: drop next_before and has_more from TranscriptPage; the caller already derives both from start.
    Clears when: a production consumer of either field is counted, or the two fields are deleted.

What this change ships

Inventory (10 items) — 9 justified

Intent: make loading older chat-history pages stop scaling with total transcript length (FIX of the measured cost in #8234).

  1. Paginated slot-detail requests read a window plus a bounded index instead of the whole transcript — justified
  2. First paginated read of a revision pays a one-time full byte scan (~200 ms at 10k rows) — justified
  3. Gateway keeps a new bounded per-revision sparse index cache in memory — justified
  4. New facade method read_messages_chained_page returning TranscriptPage — zero consumers (its next_before/has_more fields; method itself has one)
  5. Corrupt rows on the paginated route answer the same 503 the full reader did — justified
  6. Deterministic unframeable rows (oversized, splitlines-only boundary) hand off to the full reader once, no retry — justified
  7. Rotated transcripts stay on the full reader, with an archive probe before and after each bounded page — justified
  8. JSONL framer now reports byte offsets via one shared implementation — justified
  9. Unflushed-tail reconciliation gains an offset-aware form; existing callers keep the old entry point — justified
  10. history.md documents the index/invalidation/revision contracts in the same commit — justified

[FIRST-PRINCIPLES-REVIEWED] 44f2cca

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 44f2ccab3b7d89c8f1d15924757ce40a7d722e18 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 44f2cca

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 44f2ccab3b7d89c8f1d15924757ce40a7d722e18 via the fork AI-review pipeline; updated in place on each push.

Review details

A live-streaming slot never publishes the new bounded page, so paginated reads cost four discarded compositions plus the full reader they were meant to replace.

FINDING — src/kiro_crew/dashboard/chat_handlers.py:2711 — the retry version keys on getattr(slot, "_dirty_gen", 0), which _ChatSlot.append bumps for every wire-only chunk row, so on a streaming slot current_version == version is false on all four attempts, bounded_page stays None and the request falls through to read_messages_chained_full after up to four full durable-suffix reads → Fix: drop _dirty_gen from both version tuples (2711 and 2781) and key the guard on the durable state the composition actually depends on — slot._disk_older_count, _durable_prefix_counter(slot) and slot.durable_row_count().

[OPUS-REVIEWED] 44f2cca

@Pearcekieser
Pearcekieser force-pushed the perf/long-thread-loading branch from c9b40b5 to da2c5af Compare September 4, 2026 02:36
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 4, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed span=8b3eaa01b062 — Fixed in da2c5af. The full-reader fallback now witnesses chain membership and each indexed file revision before reading, verifies the same chain and revisions afterward, retries boundedly, and raises on continued churn. A regression inserts a production-eligible chain member after the first full read and proves the returned messages, total, and revision come from the retried snapshot.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 4, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/long-thread-loading branch from da2c5af to 21029b9 Compare September 4, 2026 03:39
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed span=78d6e248f252 — Fixed in 21029b9. The loop now captures the durable-prefix counter alongside the raw window snapshot, includes it in the post-worker stability witness, and refuses bounded composition before any indexed read when the counters use different units. The endpoint then uses the established complete-reader reconciliation fallback. A regression trims a transient plus durable prefix and proves no bounded read occurs and the response has the exact durable display rows and total.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 4, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed span=8b3eaa01b062 — Fixed in 21029b9. Sparse-index scan OSError now propagates instead of becoming a stable zero-row entry, so endpoint retries and the complete-reader fallback remain authoritative. A regression injects a transient index scan failure, verifies it raises, and verifies no empty index is cached.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 4, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 10, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/long-thread-loading branch from d336b03 to cdccc6f Compare September 10, 2026 15:47
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed span=8b3eaa01b062 — Fixed in cdccc6f. Indexed reads (_row_index, _read_message_range) now use a strict framing posture, strict_raw_records_with_offsets, which raises OversizedRecord on a record over RECORD_CAP instead of skipping it; the slot handler's bounded composition then falls back to read_messages_chained_full, which has no per-record cap, so no row is undercounted or omitted. The skip-posture offsets reader was removed (no callers). Added test_oversized_row_aborts_indexed_page_instead_of_undercounting; spec updated in docs/system-specs/modules/history.md.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed span=8482c1b02fa9 — Fixed in cdccc6f. _frames is now a thin projection of _frames_with_offsets (drops the offsets), so there is one framing implementation and the two readers cannot drift; _frames_with_offsets tracks offsets by bytes consumed rather than tell() so readline-only handles frame identically. All existing test_jsonl_util.py framing tests pass unchanged against the unified implementation.

@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Sep 10, 2026

@cixuuz cixuuz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sparse-index warmup is selected by off-loop execution rather than by a paging consumer, so every worker-thread full read (including consolidation, summaries, restore, backfill, and fork flows) pays the per-line offset-accounting cost even when it never pages. The PR’s own 10k-row numbers show roughly 97.7 ms to 141.7 ms (+45%) for this path. Please scope warming to paginated/dashboard keys, or measure and explicitly accept the overhead on the heaviest full-read consumer before merging.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review (agent-assisted, on behalf of @bolichen97), head cdccc6f19 (includes the maintainer merge commit that resolved the history_projection.py import conflict). The page math is correct and verified: _read_message_range picks the largest checkpoint ≤ start, counts only decoded message rows and stops before end (no off-by-one at stride or page edges); _row_index checkpoints every 128th message row at its record start offset, matching _frames_with_offsets; the stale-index risk is closed by keying the cache on (mtime_ns, size, ino, dev) + generation, re-statting before publish, and re-verifying every chain member after the read. One blocker:

  • Blocking — _message_record (history_projection.py @+683): except ValueError also swallows UnicodeDecodeError (a ValueError subclass — json.loads(bytes) decodes first), so an undecodable record is silently SKIPPED by the index and the range reader: total is undercounted and the page shifts, which is exactly what the spec's new strict-posture paragraph says cannot happen, while the full reader (_read_messages_locked, strict utf-8 handle.read()) fails closed on the same file. test_invalid_utf8_is_not_a_valid_truncated_page passes only because its fixture is ~200 bytes and the metadata text read trips first. Catch UnicodeDecodeError separately and fail closed the way the full reader does; add a test with an invalid byte past the first page.

Please fold in: _read_messages_locked (@+1037..+1041) now re-encodes the whole corpus line by line (len(frame.encode("utf-8"))) to warm an index most sessions never page — by the body's own numbers the 10k-row full read goes from ~97.7 ms to 141.7 ms ("Full read + index", ~+45%), paid on restore/search/export, and the body does not name it as a regression; use len(frame) when frame.isascii() (the writer emits ASCII-only rows) or frame the bytes once and decode the buffer after. The bounded loop in chat_handlers.py (@+2299..+2306) retries four times on deterministic failures (OversizedRecord, a missing file, a genuine bug) at debug level, re-scanning the file each time — narrow to (TranscriptRevisionChanged, OversizedRecord, OSError, ValueError) or log at warning on the final attempt. The two index builders frame differently (str.splitlines vs the universal-newline byte framer) — unreachable today with ensure_ascii=True, worth an isascii() guard or a comment so a future writer cannot reopen it. Nits: except OSError: raise is a no-op; _TranscriptRowIndexEntry is inserted mid-sentence into the #: measurement comment in history_cache.py. Auto-Merge disarmed; will re-review on the next head.

@iamwhatever

Copy link
Copy Markdown
Contributor

Not approving for now: bolichen97's latest review is still CHANGES_REQUESTED, so the outstanding feedback on the bounded read path needs resolving first.

@bolichen97

Copy link
Copy Markdown
Contributor

A maintainer attempted a conflict-only rebase of this PR onto main cff7de54 (2026-09-19) but did not push, because the conflict is semantic rather than textual and resolving it needs changes outside the conflict hunks -- which is the author's call. Nothing on this branch was changed.

What blocks a mechanical rebase:

  • The import conflicts in chat_handlers.py / history_projection.py are trivial, but after the rebase test_invalid_utf8_is_not_a_valid_truncated_page flips from pass to fail: main now catches metadata UnicodeError, and this PR's page decoder then swallows invalid UTF-8 via except ValueError, silently dropping a transcript row. The fix is in non-conflicted decoder code.

Once rebased onto current main with the above reconciled, readiness should re-evaluate on the new head.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 0bfd47101efd (one commit, rebased onto main 0a80050938). This addresses @bolichen97's review, @cixuuz's comment, the Design Review watch item, and the Sep 19 rebase note.

bolichen97 -- blocker: _message_record swallowed UnicodeDecodeError. Fixed. UnicodeDecodeError is caught and re-raised before the ValueError catch in history_projection.py, so an undecodable row can no longer be skipped by the index or range reader. The endpoint maps it to 503 history_corpus_unreadable rather than retrying. New regression: a 300-row transcript with an invalid byte at row 250 (past the first page and past the metadata read) asserts both the indexed and full readers fail closed, plus an endpoint test for the 503. This is also the semantic conflict from the Sep 19 note: with main now catching metadata UnicodeError, the test fails on the old decoder and passes on this one.

bolichen97 #2 / cixuuz / Design Review -- restore-time warming taxes every full read. Resolved by removing restore-time warming entirely. _read_messages_locked is restored byte-for-byte to the upstream loop, so restore, search, export, consolidation and backfill pay nothing for this feature. The index is built lazily by the first paginated read of a revision (~200 ms for 10k rows, off-loop, once). Removing it also deletes the get_running_loop sniffing heuristic, the fstat-and-republish race guard, and the second (str.splitlines) index builder, which closes #4 as well. Spec and PR description updated; the 141.7 ms "full read + index" figure is gone.

bolichen97 #3 -- retry loop in chat_handlers.py. UnicodeDecodeError -> immediate 503, no retry. OversizedRecord -> straight to the cap-free full reader, no re-scan (regression asserts exactly one scan attempt). (TranscriptRevisionChanged, OSError, ValueError) -> bounded retry, with a warning on the final attempt instead of debug-only.

bolichen97 #4 -- two index builders framing differently. Moot: the second builder was removed with warming; _row_index is the only builder and uses the byte framer.

bolichen97 #5 -- nits. Removed the no-op except OSError: raise; moved _TranscriptRowIndexEntry out of the split #: measurement comment in history_cache.py.

Validation: 2,544 tests across history, JSONL, pagination, snapshot, chat-persistence and dashboard-chat suites; mypy, flake8, black, isort, docs-lint and comment-history gates clean; single commit on current main.

  • fixed span=warming -- Restore-time index warming removed in 0bfd471; full reads never build or touch the index.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 89cd3d6631 (one commit, rebased onto main 194f6d51da).

  • fixed span=chat_handlers.py:2663 -- Memory-only slot pagination raised HTTP 500: the bounded path ran _bounded_slot_page(None, ...) when state.conversation_log was unset, while only the fallback branch handled that case. The bounded branch is now gated on state.conversation_log and not requires_full_reader, so a memory-only gateway serves the page from the resident window through the existing fallback. Regression test_memory_only_slot_pagination_serves_the_resident_window (fails on the prior head, passes now): limit and before pages, total, and has_more for a slot with no log.

The Backend shard-4 failures (Linux and Windows) and the cascading Coverage Gate on 0bfd47101efd were test_pending_boundary_consumer_semantics.py::test_running_and_turn_running_slot_readers_are_enumerated, which this PR does not touch and which failed identically on a clean origin/main checkout at 0a80050938. It passes on the new base 194f6d51da.

Validation: 2,545 tests across history, JSONL, pagination, snapshot, chat-persistence and dashboard-chat suites; mypy, flake8, black, isort and comment-history gates clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 51da907769: the same single commit rebased onto main 151bfcbbaa, no code change. The prior head's three failures were external (Opus and Security Scope lanes hit the GitHub App rate limit at 06:22Z; Windows shard 8 flaked on test_decisions_tool_risk.py, untouched by this PR). Fresh head so all lanes rerun together; needs a maintainer's Approve and run.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at head 51da907. The blocker is closed: _message_record re-raises UnicodeDecodeError ahead of the ValueError catch, the row-250-of-300 regression plus the endpoint 503 test pin it, restore-time index warming is gone so full reads are back to the upstream loop, OversizedRecord goes straight to the cap-free reader, and the framer offsets check out under an independent randomized walk (contiguous, seek-resume exact). 860 tests in test_history_pagination_projection.py + test_dashboard_chat.py pass locally; the branch merges cleanly onto current main.

One thing left from item #3 before I approve: a key with NO transcript file makes read_messages_chained_page raise TranscriptRevisionChanged on every path — the missing-file stamp is stored as (0, 0, 0, 0) (history_projection.py:715-719) but re-stat'ed as None (:843-845, :864-866), so "stable" can never be true — while read_messages_chained returns [] for the same key. Through api_chat_slot_detail that means every ?limit= request on a not-yet-flushed slot (a fresh tab; the embed SDK always paginates) burns four attempts and logs a WARNING with traceback before the full reader answers correctly — the deterministic case the earlier review asked not to retry, now at warning level. Please make a missing file a stable empty revision (add a no-file read_messages_chained_page test expecting total=0), and route the deterministic raw-vs-durable prefix mismatch (chat_handlers.py:2338) to the full reader like OversizedRecord (break, no retry, no warning).

Nits, non-blocking: json.loads(bytes) decodes with surrogatepass / BOM sniffing, so an encoded surrogate or BOM row is counted by the index but not by the strict full reader — raw.decode("utf-8") first makes the two identical; _read_messages_locked still carries newline="" despite the "byte-for-byte" note; the limit=1 total probe decodes a row it does not need; and the bound is window-sized (a session grown within one gateway lifetime has _disk_older_count == 0 until restore/trim) — worth one sentence in the body so the perf claim is not over-read. Will re-review on the next head.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

@bolichen97 thanks for the re-review. Head 3754ffe40df0 (single commit, rebased onto e183a46e7, conflict in history.py resolved by keeping main's identity wording) addresses each point:

Requested

  1. Missing file is now a stable empty revision. _TranscriptRowIndexEntry.stamp is tuple | None; _row_index returns stamp=None for a key with no file, which is exactly what the stability re-stat returns, so read_messages_chained_page no longer raises TranscriptRevisionChanged on the no-file path. ChainRevision type widened to match. Tests: test_missing_transcript_file_is_a_stable_empty_revision (total=0, revision pins across reads, matches read_messages_chained() == []) and the endpoint test test_never_flushed_slot_pages_without_retry_or_warning (fresh tab, ?limit=, no "read failed" log at any level).
  2. Raw-vs-durable prefix mismatch goes straight to the full reader. _bounded_slot_page raises a dedicated _DurablePrefixMismatch; the handler catches it beside OversizedRecord with break, a debug line, no retry, no warning. Test: test_prefix_counter_mismatch_uses_full_reader_without_retry_or_warning asserts one call and zero WARNING records.

Nits

  • _message_record now does raw.decode("utf-8") before json.loads, so the index and the strict text-mode full reader agree on BOM and surrogatepass rows. Test: test_bom_and_surrogate_rows_count_like_the_strict_full_reader (an encoded lone surrogate raises UnicodeDecodeError from both readers).
  • newline="" dropped from _read_messages_locked; the open() call is now identical to upstream.
  • The total probe decodes no rows: read_messages_chained_page accepts limit=0 as a total/revision probe (indexes the chain, decodes nothing), and _bounded_slot_page uses it. Test: test_zero_limit_probe_reports_total_and_decodes_no_rows (counts _message_record calls on a warm 300-row index: zero). limit < 0 still raises.
  • PR body: added a paragraph stating the bound is window-sized (_disk_older_count == 0 until restore/trim for a session grown within one gateway lifetime).

Validation on the rebased tree: 2,062 tests across the history, JSONL and dashboard-chat suites; mypy, flake8, isort, black baseline gate, comment-history and docs lint clean. All four new regression tests were verified red on 51da907769c7.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed f0e58f5f9044: the same single commit rebased onto main 7e40e2175 to clear a merge conflict in chat_handlers.py (main added the CLIENT DEPENDENCY comment beside _collapse_wire_rows; carried onto this branch's fallback call). No code change versus 3754ffe40df0. This also reruns the seven lanes on that head that failed on the CI app's rate limit, a runner shutdown during the black step, and the test_diag_procs timing flake.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 0982bc7e23fb (single commit, rebased onto main c08b4e2b7, import-union conflict in history_projection.py resolved). It addresses the two blocking findings the fork GPT review raised on f0e58f5f9044 (read from the review run log; the PR comment never posted):

  • F1, history_projection.py _row_index, rule a-bound-bounds-every-field-it-retains: the sparse index kept one (row, byte_offset) checkpoint per 128 rows with no ceiling. Now _TRANSCRIPT_PAGE_INDEX_MAX_CHECKPOINTS = 1024; when a build would exceed it, every other checkpoint is dropped and the stride doubles (kept rows are multiples of the old stride at even positions, hence of the new stride), so an entry costs at most about 100 KiB for a transcript of any length and a warm page still decodes at most one stride of extra rows. Test: test_checkpoint_count_is_capped_and_pages_stay_exact (4-row stride, 8-entry cap, 200 rows; checks the cap, the power-of-two stride, and that walking pages equals the full read). Spec paragraph updated.
  • F2, _file_stamp: it caught every OSError and returned None, which after the previous head's "missing file is a stable empty revision" change meant a permission or I/O error on a transcript read as total=0. Now only FileNotFoundError maps to None; any other OSError propagates into the handler's existing retry / history_corpus_unreadable path. Test: test_unstattable_transcript_is_not_an_empty_transcript (absent file is None; PermissionError on stat propagates), red on the prior head.

The Backend Lint failure on f0e58f5f9044 (both attempts) was the black step ending in exit 143 with "The runner has received a shutdown signal" after ten minutes and no formatting diagnostic; scripts/ci_black_diagnostics.py scripts/check_black_formatting.py passes on this tree locally in 77 s. Other PRs' lint lanes in the same window passed, so I expect the new head to clear it.

Validation on the rebased tree: 2,064 tests across the history, JSONL and dashboard-chat suites; black baseline gate, mypy, flake8, isort, comment-history and docs lint clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 8865c6368bcd (same single commit, same base c08b4e2b7) for the fork GPT review's one blocking finding on 0982bc7e23fb (read from the review run log; F1 and F2 from the prior head are cleared):

  • chat_handlers.py, bounded slot page: a transcript row nested past the JSON parser's depth made _row_index raise RecursionError, which the narrowed except let escape as a 500 where the unlimited path answers 503. The handler now maps RecursionError to history_corpus_unreadable() beside the UnicodeDecodeError branch. Test: test_overdeep_json_row_fails_closed_on_bounded_page appends a 200,000-deep [ row and expects 503 history_corpus_unreadable; red on 0982bc7e23fb.

Not this PR's, for the record: the Backend Lint black step's exit 143 ("runner has received a shutdown signal") is hitting every fork PR's lint job on ubuntu-latest today (five other forks checked), while same-repo PRs pass on the CodeBuild instance-size:large runner forks do not get; and shard 7's test_aws_control_backup.py::...concurrent_failure_then_recovery is in a file this PR does not touch.

2,065 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8 and comment-history gates clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 3ed7194da658 (same single commit, same base c08b4e2b7, still clean against current main) for the fork GPT review's one blocking finding on 8865c6368bcd (the RecursionError finding from the prior head is cleared):

  • jsonl_util.py / history_projection.py: a record holding a raw U+2028, U+2029, U+0085 (or \x0b, \x0c, \x1c-\x1e) is one record to the byte framer but several fragments to the full reader's splitlines(), so the two disagreed on total (31 vs 30 in the new test). Rather than teach the framer splitlines semantics, the bounded reader now refuses such a record: _message_record calls raise_if_splitlines_boundary, which raises SplitlinesBoundaryRecord (an UnreadableRecord sibling of OversizedRecord), and api_chat_slot_detail routes it to the authoritative full reader exactly like OversizedRecord: break, no retry, debug log. The writer emits ASCII-only JSON, so this is unreachable through the normal path. Tests: test_splitlines_only_boundary_row_hands_the_read_to_the_full_reader (projection raises; read_messages_chained still answers) and test_splitlines_boundary_row_uses_full_reader_without_retrying (endpoint answers from the full reader in one attempt with matching totals; red on 8865c6368bcd).

Not this PR's: Backend Lint's black step exit 143 on ubuntu-latest is hitting every fork PR today; Opus "review incomplete" was the CI app's rate limit at 23:36 UTC; Windows shard 7 failed on test_runtime_cleanup_windows.py, a file this PR does not touch.

2,067 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8, isort and comment-history gates clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 18f453af160e (same single commit, same base c08b4e2b7, clean against current main) for the fork GPT review's two blocking findings on 3ed7194da658 (the splitlines finding from the prior head is cleared):

  • history_projection.py _message_record: str.strip removes Unicode whitespace (NBSP, ideographic space) that json.loads does not accept, and the full reader strips each line before parsing, so an NBSP-padded valid row counted there and not here. Now json.loads(text.strip()). Test: test_nbsp_padded_row_counts_like_the_full_reader (both readers report 6, page carries the padded row); red on 3ed7194da658.
  • _file_stamp: a writer outside this process can rewrite a transcript in place at the same size and restore mtime, and it never bumps the invalidation generation, so the cached checkpoints survived. The stamp now includes st_ctime_ns (FileStamp = (mtime_ns, ctime_ns, size, ino, dev) in history_cache.py), which user space cannot set; this is also the identity the module's other transcript memos already use. Test: test_same_size_rewrite_with_restored_mtime_misses_the_index_cache (in-place same-size corruption of one row plus os.utime restore, no ConversationLog call; total drops 40 to 39); red on 3ed7194da658. Spec line updated.

Backend Lint's black exit 143 on ubuntu-latest is still the repo-wide fork runner kill.

2,069 tests across the history, JSONL and dashboard-chat suites; black baseline, mypy, flake8, isort, comment-history and docs lint clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 23dc64720e7c (same single commit and base; test-only changes plus one docstring):

  • Windows shard 7 on 18f453af160e failed my new test_same_size_rewrite_with_restored_mtime_misses_the_index_cache: Windows reports creation time as st_ctime, so the ctime component of the stamp is inert there and the invalidation generation remains the in-process guard. The test is now skipif(sys.platform == "win32") and the _file_stamp docstring says so.
  • test_prefix_counter_mismatch_uses_full_reader_without_retry_or_warning asserted no WARNING records at all and intermittently caught an unrelated fixture diagnostic (the on-loop history: session mutation warning) under xdist; it now asserts only that the bounded slot history line from chat_handlers is at DEBUG.

The other seven failures on 18f453af160e were the CI app's rate limit (desktop matrix, readiness signal, GPT / Opus / Internal Content Scan "could not run", Security Scope with no legs) and the repo-wide fork Backend Lint runner kill. 2,069 tests locally, gates clean.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed c15296ac2 (one commit, rebased onto main 3f16157ba; merges clean).

@bolichen97 both items from your 09-23 review are in this head, unchanged since 3754ffe4:

  • Missing file is a stable empty revision. _TranscriptRowIndexEntry.stamp is FileStamp | None; _row_index returns stamp=None for a key with no file, which is the same value the stability re-stat produces, so read_messages_chained_page answers total=0 on the first attempt. Pinned by test_missing_transcript_file_is_a_stable_empty_revision (test_history_pagination_projection.py), which also pins the revision across a second pinned read.
  • Raw-vs-durable prefix mismatch goes to the full reader, no retry, no warning. _bounded_slot_page raises _DurablePrefixMismatch; api_chat_slot_detail catches it next to OversizedRecord, logs at DEBUG and breaks to the full reader. Pinned by test_prefix_counter_mismatch_uses_full_reader_without_retry_or_warning (test_dashboard_chat.py, one attempt, no WARNING record).

The nits landed too: strict raw.decode("utf-8") before json.loads, newline="" dropped, limit=0 total/revision probe, and a "bound is window-sized" paragraph in the body. Ready for your re-review when you have a moment.

The Opus 5 lane on 23dc6472 ended "review incomplete": discovery ran, validation died on a GitHub API rate limit (API rate limit exceeded for installation, 01:23:40Z), so no verdict or spans were published. I read the six discovery candidates from the run log and acted on each; the push above is the retrigger.

  • fixed chat_handlers.py:_bounded_slot_page — the pending_rewrite branch that dropped the on-disk suffix made the bounded path index a different corpus than the full reader during a rewrite, so a before cursor could cross index spaces. The special case is gone: the suffix is always read and the same reconciliation walk resolves the pre-rewind rows, exactly as read_messages_chained_full + _append_unflushed_tail does. The pending_rewrite parameter is removed with it. Docstring states the parity contract.
  • fixed test_dashboard_chat.py::test_bounded_request_never_calls_full_chained_reader — the deleted off-loop pin is back on the branch that gained the most disk I/O: the test records threading.get_ident() inside the archive probe, _bounded_slot_page, and the re-probe, and asserts the loop thread is in none of them (plus at least three recordings).
  • fixed history_projection.py:read_messages_chained_page — the primary stability loop now re-checks chain membership (self._chain_keys(key) == keys) like the fallback loop already did; _read_messages_chained_page_once returns the observed keys for it.
  • fixed chat_handlers.py — the getattr(slot, "_disk_older_durable_count", tail_snapshot[0]) default made the prefix guard x != x on a slot without the counter. _durable_prefix_counter() returns -1 for that case, which fails the guard and routes to the full reader. On the "prefix served raw" half: the guard compares the raw and durable counts of the frozen prefix, and every site that sets _disk_older_count sets _disk_older_durable_count beside it (chat_persistence.py:1721/1727, :2291/2295, channel_slots.py:663/668, chat_handlers.py:11329/11333; chat_rewind.py:647-648 restores both), so equal counters mean the prefix holds no wire-only row and collapsing it is a no-op. Production writers keep wire roles out of saves (chat_persistence.py: if role in ("chunk", "done", ...): return None); the test that appends done directly is the mismatch test, and it takes the full reader.
  • fixed history_projection.py:_read_messages_chained_page_once — a chain none of whose members yields a row now falls back to the key's own file, mirroring read_messages_chained's messages or self._log._read_messages(key). Pinned by test_rowless_chain_falls_back_to_the_key_like_the_full_reader (stale _tab_id_index pointing at an emptied member; paged total equals the full reader's 12).
  • fixed (bounded) history_projection.py:read_messages_chained_page — a read pinned to an expected_revision now makes one attempt instead of two: the stamp that broke the pin includes mtime_ns and size, so a second attempt cannot produce the expected revision. Per handler attempt the bounded path is now at most 1 probe (≤2 scans) + 2 pinned reads (1 scan each) before falling to the full reader, and _row_index publishes its index whenever the post-scan stamp matches, so consecutive attempts on a quiet file share one scan. The in-projection full-read fallback runs only for unpinned reads (the probe), once per handler attempt, same as before. Spec paragraph updated.

Validation on c15296ac2: 2,071 tests across the changed modules' suites (-n 8), black/isort/flake8 gates, mypy on both touched modules, comment-history gate, docs lint — all pass.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Pushed 44f2ccab3b (one commit, same base 3f16157ba) for the fork GPT review's one blocking finding on c15296ac2.

  • fixed F1 history_projection.py:_message_record — the catch was except ValueError, so an integer literal past sys.get_int_max_str_digits() (a plain ValueError, not JSONDecodeError) dropped the row from the index while the full reader, which skips only json.JSONDecodeError, raised on it. The catch is now except json.JSONDecodeError; any other ValueError propagates, and api_chat_slot_detail answers history_corpus_unreadable() for it directly (no retry: it is deterministic for the revision and the full reader answers the same 503). The retry arm is now (TranscriptRevisionChanged, OSError) only. Pinned by test_overlong_integer_row_fails_closed_on_bounded_page (test_dashboard_chat.py): appends a 4301-digit integer row, asserts the full reader raises ValueError, and asserts the bounded request returns 503 history_corpus_unreadable; red on c15296ac2 (200 with total=30), green here.

Windows shard 1 on c15296ac2 failed test_knowledge_handlers_off_loop.py::TestAuditRidesWithTheWrite::test_cancelling_the_caller_cannot_drop_a_committed_audit (14,567 passed otherwise), a cancellation-timing test in a module this PR does not touch.

Validation on 44f2ccab3b: 2,072 tests across the changed modules' suites (-n 8), black/isort/flake8 gates, mypy on both touched modules, comment-history gate, docs lint — all pass.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at head 44f2ccab. Both items from the 2026-09-23 review are closed, verified against the source at this head rather than from the disposition notes:

Item 1 — a missing file is now a stable empty revision. _row_index returns _TranscriptRowIndexEntry(None, generation, 0, ()) when _file_stamp reports no file (history_projection.py:755-764), and ChainRevision is widened to tuple[tuple[str, FileStamp | None, int], ...] (:76), so the stability re-stat compares None == None and holds. The (0, 0, 0, 0) stand-in is gone from production and survives only as a deliberate mismatch fixture in a test. test_missing_transcript_file_is_a_stable_empty_revision pins total=0, messages=[], has_more=False, revision=((key, None, gen),), that the revision pins across a second read, and that read_messages_chained returns [] for the same key — the two-path parity the review asked for.

Item 2 — the deterministic prefix mismatch goes straight to the full reader. _DurablePrefixMismatch is raised at the guard (chat_handlers.py:2383-2386) and caught structurally identically to OversizedRecord / SplitlinesBoundaryRecord: logger.debug then break, no retry and no warning (:2760-2765 vs :2754-2759). Covered by test_prefix_counter_mismatch_uses_full_reader_without_retry_or_warning and test_never_flushed_slot_pages_without_retry_or_warning. _durable_prefix_counter returning -1 for a stand-in without the counter is a sensible extra: it fails the guard rather than passing it vacuously.

All four nits are addressed. raw.decode("utf-8") precedes any JSON parsing with the BOM / surrogatepass rationale in the docstring (history_projection.py:712); newline="" is gone; the total probe is limit=0 and decodes no rows (:2390); and the body now states the bound is window-sized, with the _disk_older_count == 0 caveat.

The 09-08 ordering note is moot — #8862 is closed unmerged, so there is no longer a stack to sequence behind this.

State at this head: PR Readiness success (2026-09-25T23:54:35Z), no failing, pending or cancelled check-run, mergeable=true. Approving; native auto-merge (squash) is already armed, so no admin override and no bypass of branch protection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound backend transcript work for paginated chat history

6 participants