Repository navigation
docs(rfc): bounded-time restore for large sessions (#35875) - #35916
Conversation
Design for part B of #35875, with measurements on main: a tail window for eager tabs over 16 MB, focus-first restore order, one crew-log walk for all of a slot's folds, and a CI fixture at the threshold. Changes no code.
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This PR only adds a draft RFC and changes no code; the open choices go to reviewers under its "Decisions requested" section. Notes
[DESIGN-REVIEWED] a935074 Description read[DESCRIPTION-READ] a0f35738f6596db17d02d647b72941f3ccf8413e02a22a0720acf89a93d08eb6 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: A person should decideWhy: the decider raised 2 possible problems and estimates a 62% chance that this change introduces a defect; a person should check them. This PR adds a docs-only draft RFC. Its central sizing claim is wrong. §1.1 asks for approval of a 16 MB threshold because the worst case fits 'inside the 5 s target', but by the document's own numbers it comes to 5.4 to 5.9 s, because the restore reads tabs one after another. The verified main-side reading in the index row, and the bound built on it, also miss that main does a full read for every tab whose window cannot wait (non-person, crew, loop-driven, empty tail), not just the newest 8. I confirmed both points in restore_drivers.py. Both should be corrected before the RFC is used to ask for decisions. Things to check (possible problems the decider raised; at this confidence many turn out to be fine):
An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
Sections 1.1 and 1.2 are behaviour changes gated on review of this document; section 1.3 (pure performance) and 1.4 (tests) land as separate pull requests. Restate which tabs main reads in full (the newest 8 plus every tab whose window cannot wait), apply the tail window to that whole set, and correct the 5 s budget: it holds only with the focus-first order of section 1.2.
8fb1119 to
a935074
Compare
|
Re: Opus 5.5 Review (#35916 (comment)) Both findings are fixed in a935074.
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
This PR adds docs only: the bounded-restore RFC and one index row in docs/request-for-change/README.md. It changes no code and no blocking repository rule covers these paths. The review checked the restore symbols and line references the RFC cites against main, and they exist as described. The earlier arithmetic and full-read-set findings are fixed on this head. Every review lane on this head passed or was skipped for a stated reason, and PR Readiness is green.
Done well: measured and unverified claims are kept apart, the synthetic crew-log units are labelled as synthetic, and the 5 s bound is restated openly so it rests on the focus-first order and not on the threshold alone.
The design is signed off under the standing maintainer sign-off for changes by team members.
Optional:
- Main already keeps a 500-row window (
_WINDOW_ROWSin slot_builders.py) and has a page-index tail reader (slot_window_tail.read_window_tail, used by_PendingWindow.read) with scroll paging. §1.1 could shrink to routing the eager open-tab read through that reader. That would drop the 16 MB threshold, the new marker and behaviour change 1 in §3. - The 5 s argument counts only the build of the focused tab. Turn admission also waits on memory preparation (up to
MEMORY_ADMISSION_WAIT_SECONDS, and 9.4 s in the RFC's own table). Say whether that is in scope, or who owns it. - The in-flight work section lists #35840, #34887 and #35869 as open, but all three are now merged. The fresh-savepoint row is also credited to #34882, which does not touch the measured slot-memo path.
Problem / Motivation
Goal: Record an agreed plan that makes a large session accept a message within 5 s of a gateway restart, with a 200 MB transcript and 100+ open tabs.
Part B of #35875. This RFC measures where restore time still goes on current main and proposes a bounded-cost path. Docs only; it changes no code.
Why it matters
A large, long-lived session (the busiest conductor or the session a person works in all day) is slow to become usable after every restart, and a message sent in that window is refused or left pending. Users with small sessions see no change: every proposed path applies only above a 16 MB threshold or is pure performance with identical outputs.
Reviewer note: if accepted, the later implementation changes default behaviour for users with large sessions: a tab over 16 MB opens with its newest window and pages older rows in on scroll, and the restore builds the focused and visible tabs first. These are listed in the RFC's "Behaviour changes" section.
Not a goal
What changed (motivation → approach → change)
docs/request-for-change/rfc-large-session-bounded-restore.md: new draft RFC with the measurement tables, how it interacts with fix(dashboard): append a steady slot save's new rows instead of rewriting the file #35206, perf(dashboard): reuse written rows' entries when a slot save rewrites the file #35496, perf(crew-log): replay each unit's log from a watermark on boot #34882, perf(projection): fold a resumed checkpointed tail in place after one apply #35840, perf(work-ledger): parse only changed item files when listing a board #35869 and fix(dashboard): refuse a send to a tab the startup restore has not built #34887, and four proposals: a tail window for every fully read tab over 16 MB, focus-first restore order, one crew-log walk for all of a slot's folds, and a CI fixture at the threshold. Section 6 is the delivery plan: 1.1 and 1.2 are behaviour changes gated on review of this document; 1.3 (pure performance) and 1.4 (tests only) land as separate pull requests. The 5 s target holds only together with 1.2's focus-first order.docs/request-for-change/README.md: index row.Key numbers on main
029b6db029(130 tabs, 204 MB transcript): ready at 8.0 s warm and 14.1 s cold page cache; the biggest tab's full read is 1.2 s warm and 3.1 s cold. Synthetic 136 MB crew-log unit: 18 cold slot folds take 91 s, because each fold walks the unit separately; with a stale savepoint, 47 s on main and 42 s with #35840.Backwards compatibility
Compatible: docs only, nothing that works today changes.
Removes nothing: the diff adds one document and one index row.
Tests
Docs only.
scripts/docs-lint.sh: all documentation checks passed.Proof: not provable -- docs or config only
Manual verification
N/A -- docs only; the measurements were taken on a scratch copy of a data home and are recorded in the RFC.
Related Issues
Refs #35875
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.