Repository navigation
perf(crew-log): cut the fixed reads of a warm session fold pass - #38220
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. The open-count and home-resolution cuts are pinned by tests, the same-tick savepoint race fix is accounted for in the description, and the patch removes nothing. Notes
[DESIGN-REVIEWED] 802ae5b Description read[DESCRIPTION-READ] 5afeb4caa40d442b8daf17299ab23a892b856faeeda97d4dc500603466c65350 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The open and resolve cuts sit at mechanism level, with exact counts pinned in tests. The savepoint-race detector is outside the Goal but rests on a race reproduced on main. Notes
What this change shipsInventory (7 items) — 4 justifiedIntent: make the warm session fold pass (paid by every eager wake and panel read) cheaper in file opens and data-home resolutions. This is a FIX for #38210.
[FIRST-PRINCIPLES-REVIEWED] 802ae5b Description read[DESCRIPTION-READ] 568d221f260d8c753a4581e22fbb7c26df20ab7095f7fc1e3dff7127cb432732 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
a054167 to
f0e4e77
Compare
|
Update on head f0e4e77. Facade red on a054167. Savepoint race. Local lane dispositions (r4 and r5, GPT and Opus). No blocking findings in either round. Taken: the re-check moved under the lease after every other refusal, so only a write about to happen pays the re-read; a resumed fold that falls back to a cold walk spoils the span, so it writes no savepoint; the new tests patch through Declined, with reasons:
|
|
Thanks, the three read cuts match what I asked for in #38210. The handle now supplies the before-pass header ( On the
|
f0e4e77 to
a4300e4
Compare
|
Update on head a4300e4. GPT F1, fixed ( GPT F2, fixed ( Backend Tests (3.12, 7) on f0e4e77 was ours: Local Opus review, round 6. Its one gap is taken: the lease test now asserts the span check actually ran before it checks the lease was released. Savepoint race, root cause. The savepoint write compared only a stat (size, mtime) fingerprint, so a same-size rewrite inside one mtime tick got a savepoint written over it. @neilieva, on splitting the span check out: I've kept it here because this PR's read cuts are what made the existing race show up in CI, and the check, its pins and the test it fixes belong together. If a maintainer prefers a split, it separates cleanly as |
|
@neilieva, on your comment above, both points as of a4300e4:
The span check stays in this PR because this PR's faster pass is what turns the latent race from 2/30 into 11/30 failures, so the detector fix belongs with the change that exposes it. |
a4300e4 to
ad5f99d
Compare
The identity read before a pass takes created_at from the handle the same call just opened, the data home is resolved once per fold pass while the kind root is still lstat'd and resolved on every call, and the worker recap reads its three session folds from one warm pass. The savepoint write also re-checks a digest of the records the pass framed, so a same-size rewrite inside one mtime tick no longer gets a savepoint written over it. Fixes kirodotdev#38210
|
Rebased onto current main with no content change, so the base includes #39262, which fixes the main-side |
ad5f99d to
802ae5b
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Checked on this head: the cached data-home resolve still runs the kind-root lstat, exists and resolve checks on every call, and it refuses if the home moves mid-pass. fresh_log_identity reuses the header that the open just validated, and a recreation between the open and the stat makes the pass retry. Every way the savepoint span check can fail only skips a write. The recap handler gets the same three values from one pass. Reverting the read cuts brings main's counts back and turns test_crew_log_warm_pass_reads red. Turning off the span check turns 4 tests in test_crew_log_consumed_span red on 3 of 3 runs. The crew-log tests pass at head.
Done well: the same-tick rewrite tests restore the exact pre-edit mtime, so a race that failed now and then on main becomes a deterministic red.
Optional:
- Create
ConsumedSpanonly whenprefix_seenis set. A pass that cannot write a savepoint then skips the sha256 work. - Add a
spanrow to the read-budget table and a short note on the savepoint refusal in docs/system-specs/modules/crew-log-projection.md. Also say in the body that a cold pass that writes a savepoint reads the log twice. - In test_crew_log_warm_pass_reads.py, install the audit hook lazily and count only opens under
tmp_pathor on the measuring thread. Consider a test for the fallbackspoil()call too, because deleting it reds no test today.
Problem / Motivation
Goal: A warm session fold pass with nothing new to fold opens the crew log fewer times and resolves the data home once.
A warm
fold_session_warmpass with nothing new openedlog.jsonl5 times and ran_checked_crew_log_root10 times, each resolving the data home again. The recap route ran three such passes per read.Why it matters
Every eager wake, publisher backstop and panel read pays this fixed cost per pass. On a 15 s capture one 40 KB log was opened 230 times.
Not a goal
iter_fromor the stored resume point.What changed (motivation → approach → change)
Traced on main, nothing-new pass. Openers of
log.jsonl:CrewLog.open>_settle_tail>_scan_tail(tail scan)CrewLog.open>_read_header_line(header)_fold_attemptidentity before the pass:_unit_log_identity>unit_header_created_at>_proved_header(header again)held_still>log_originafter the pass (header a third time)extent_holdsin_fold_attemptCallers of
_checked_crew_log_root(10):segment_pathsfromCrewLog.exists,CrewLog.open,extent_holdsand both identity reads;_lock_pathin_settle_tail;_unit_header_objectdirectly and throughcrew_log_dir, once per identity read each.Changes:
fresh_log_identityinprojection_session_log.py: when_fold_attemptopened the handle itself, the identity before the pass takescreated_atfrom the headerCrewLog.openjust read, and still stats the file for device and inode. A handle passed in by the caller is older and still goes through_log_identity. A recreation between the open's header read and the stat mixes the old stamp with the new inode; the after-pass read then returns the new stamp, the pass failsheld_still, and nothing is kept.resolved_home_for_passinstore_paths.py, entered byfold_session:data_home().resolve()runs once per pass. The kind root'slstat,exists,resolve()and comparison still run on every call. A data home that moves mid-pass makes the root resolve away from the cached path, and the check fails closed. The memo is a context variable, so it never crosses threads or passes._session_foldsinsrc/kiro_crew/dashboard/handlers/crew_worker_recap.pyreadstimeline,approvalsandtoolsfrom onefold_session_warmcall while all three are eager, so the three values come from the same moment.The 4 opens left on the nothing-new path:
CrewLog.openCrewLog.opencreated_atheld_stillafter the passextent_holdsWith new entries the pass goes from 10 opens to 9, all from
iter_fromand the savepoint's extent read on top of the four above.Latent race
The savepoint write refuses a file that moved during the pass by comparing a stat-only (size, mtime) fingerprint. A same-size rewrite that lands inside one mtime tick leaves both equal, so a savepoint was written over records that had changed. Main carries this: a probe that puts the pre-edit mtime back fails 30/30 on main. The extent's fixed tail digest (
PrefixExtent.tail_sha) is not enough, because its own docstring says it cannot see an edit further back than its tail, and the edit intest_a_same_size_rewrite_during_a_pass_writes_no_savepointis about 190 KB back. This PR's faster pass made that test hit the race more often (alone, 30 runs: main 2/30 failing, this branch before the detector 11/30).ConsumedSpaninstore_segment_reads.pyhashes every record the walk frames, with its offsets, as the walk reads it. Just before the savepoint write,checkpoint.saveframes that same range from disk again and writes nothing on a mismatch (one debug line; the next pass re-folds from the older savepoint). The fingerprint stays the first check. Hashing on the way in is in memory over bytes the walk already read. The re-read runs under the savepoint lease, after every other refusal, so only a write that is about to happen pays it; it costs the bytes the pass consumed and is counted in the read budget. A pass whose resumed fold failed and fell back to a cold walk writes no savepoint, since its span does not describe that walk. After: the original test and the probe each 0/30 failing on this branch.Backwards compatibility
Compatible: what gets folded, the stored resume point and
iter_fromare unchanged; a savepoint is now skipped when the records its pass folded changed before the write, and the next pass writes it.Removes nothing: no field, key, route or function is removed; the root check keeps both of its conditions.
Tests
test/test_crew_log_warm_pass_reads.pycounts opens oflog.jsonlwith an audit hook and resolutions of the data home: exactly 4 opens and 1 resolution on a nothing-new pass (5 and 10 on main), and 9 opens and 1 resolution when new entries are present (10 and 14 on main). A third test checks the pass still folds the new entry.test/test_crew_log_consumed_span.py: a same-size rewrite with the old mtime put back is refused for the FIRST folded record (about 190 KB below the end) and for the LAST; a unit that only grew still gets its savepoint; a byte outside the framed range does not refuse and the re-read covers exactly the framed bytes. Skipping the re-hash by hand makes the first-record test fail 30/30 (the mutation proof; see the Proof line).test/test_crew_log_direct_segment_open_ratchet.pylists_holdsas a direct segment open that counts itself in the read budget, asraw_prefix_digestdoes.test/test_crew_worker_recap.pypins that_session_foldsruns one warm pass for its three folds.Proof: not provable -- other: prove.py was inconclusive because reverting every production hunk fails test collection on the new ConsumedSpan import; the mutation was done by hand: with the savepoint re-hash skipped, the same-tick probe fails 30/30
Manual verification
N/A — the counts are pinned by the tests above; macOS fs_usage was not re-run.
Related Issues
Fixes #38210
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)