Repository navigation
feat(crew-log): per-fold savepoint versions, and eager folds the reader no longer waits for - #14975
Conversation
|
Intent: a dashboard reading a crew log slot's work board or agent panel gets the current value straight away, because the fold was advanced when its entry committed instead of when the read asked -- and retiring one fold's stored meaning stops costing every other fold a refold. |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm a maintainer owns reversing the recorded "one shared savepoint version is deliberate" decision; the spec rewrite argues it well, but reversal is theirs to sanction. Suggestions
[DESIGN-REVIEWED] 7f48283 Description read[DESCRIPTION-READ] 83d564d583012968264133face4e2fd831baa61f8098a3857287444a164ec781 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The per-fold digest pin is now the only thing preventing a forgotten version bump — confirm a maintainer accepts reversing the recorded shared-number decision on that basis. What this change shipsInventory (10 items) — 10 justifiedIntent: dashboard reads of a slot's work board or agent panel get an already-current value instead of re-walking the log, and a fold fix stops retiring every other fold's savepoints — an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 7f48283 Description read[DESCRIPTION-READ] ff022f8cb57e083471f581a1f07c9c85bd1f07e2fd64214c687dd8291b6cc9af That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
1f3cbe7 to
ffb85ea
Compare
ffb85ea to
66e66ca
Compare
|
self-added: no fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes rebutted, with the scope documented in
|
|
self-added: no needs-a-decision.
|
ba26e0a to
b29d9bc
Compare
b29d9bc to
2630f4b
Compare
|
self-added: yes rebutted, with the harm named by measurement.
|
0e19e74 to
cdd3dd0
Compare
|
self-added: yes accept-and-defer -- the claim is mine and it is false; the head carrying the correction is not mine to push.
|
|
self-added: yes
|
|
self-added: yes accept-and-defer -- correct, and it corrects a rebuttal of mine from two rounds ago.
|
cdd3dd0 to
3b75ddf
Compare
3b75ddf to
329f38c
Compare
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
self-added: yes fixed in
|
|
Disposition -- Span: The finding is correct and it was a real defect, not a style point. Fix: Rejected alternative: lowering Verified on this head: 55 passed across |
|
self-added: no fixed in
|
|
self-added: no fixed in
|
…er no longer waits for A crew log fold had one shared savepoint version and one folding time. Both cost the readers this module exists for. PER-FOLD VERSION. Retiring the stored meaning of one fold retired all six, so a counting fix in `usage` refolded every fold of every long-lived unit -- the units savepoints exist for. `state_version` now lives on `_Fold`, a savepoint file records the version of the fold it holds, and a mismatch discards that fold's file alone. `FOLD_STATE_VERSION` stays as the maximum over the folds, because `checkpoint.py` publishes it as `CHECKPOINT_VERSION`. The digest pin that obliges the bump is per fold too, so a change to one fold demands exactly one version move instead of one that retires five valid savepoints to retire one stale one. EAGER FOLDING. A slot-keyed fold interprets one entry type and its reader walks every line of every unit the slot ran under to find it, so the first read of a cold cell is O(the slot's whole history) for a value that is a function of entries this process just wrote. `work` and `panel` are now `mode="eager"`: the append job hands `(unit, type, seq)` to a bounded queue with one `put_nowait`, one daemon thread coalesces the batch and continues the warm cell through the SAME read path the dashboard uses, and the value is published as one owner-only `crew_log_projection` frame under higher-seq-wins. `mode="eager"` requires `affects`, refused at import: a fold every entry moves would be woken for every message body in the log. Overflow drops and counts (`eager_dropped`) rather than waiting, because the fold is not the record -- a dropped wake leaves the memo behind the file and the next read carries it forward. `session/closed` drops the slot's memos. TWO THINGS THE MEASUREMENT DECIDED, NOT THE DESIGN. The warm memos are now bounded in BYTES (64 MiB, config-read, LRU) as well as in count: eager folding uncouples the retained set from what a reader asked for, and a measured `work` cell at `WORK_ITEM_LIMIT` is ~1.2 MB, so 64 of them is ~1.2x the budget. The byte ceiling is the one that binds. And the `work` fold's UNIT LIST is memoized, guarded by the conductor units' own stat fingerprints. Resolving it parses every entry of every conductor unit to find `bind` entries, which left a warm `work` read O(the log) even with the fold free. Measured: 41 of 41 entries parsed without the cache, 0 with it.
|
self-added: yes fixed in
|
buluoray
left a comment
There was a problem hiding this comment.
LGTM. I checked this at head 7f48283.
The append path cannot be blocked or failed by an eager wake.
- The append path gets one
put_nowait(crew_log/eager.py:251). queue.Fullis counted by_count_drop, and any other exception, including a failedThread.start, is caught (eager.py:205-209)._note_eagerruns only afterlog.appendreturns and is itself wrapped (emit.py:1022-1041,:2930-2935).- A worker crash is caught per batch, and
_ensure_workerrestarts a dead thread.
Nothing is broadcast, so there is no cross-owner surface. _advance calls read_slot_projection(slot, name) and discards the value. It only warms the same (data_home, slot, name) memo the reader uses (projection.py:3364), and owner resolution is unchanged.
A version mismatch retires only that one fold. A savepoint whose state_version does not match returns None (projection/checkpoint.py:266-269), and _resume_one cold-folds only that fold (crew_log/checkpoint.py:566-590). Forcing a stale version at :569 fails test_bumping_one_folds_version_retires_only_that_folds_savepoint.
Locking is safe. The lock order is always entry.lock then _slot_memo_guard, and I found no re-entry.
Tests: 261 pass across the crew-log eager, checkpoint, projection and slot-fold suites and the work-fold warm-read suite.
Non-blocking follow-ups:
- The squash commit body prepared for auto-merge says the value "is published as one owner-only
crew_log_projectionframe". The code and theeager.py:29docstring say an advanced fold is not broadcast. That paragraph is worth correcting before it lands in main's history. work_ledger.py:2881reads theworkmemo withalso_slots; the eager path reads it without. Their unit lists differ, so each misses the other's memo and refolds cold. This costs speed only:memo.units == unitskeeps the answers correct.
Problem / Motivation
Goal: a dashboard reading a crew log's work board or agent panel gets the current value straight away, instead of waiting while the whole session history is re-read to compute it.
Two separate costs, both paid by the readers these folds exist for.
A dashboard panel asks for a slot's
workorpanelvalue on a timer. Answering it means reading every line of every crew log the slot ran under, because the fold interprets one entry type and the file is mostly message bodies. The warm memo removed the repeat of that walk but never the first one, so every cold cell pays the whole history for a value that is a function of entries this same process just wrote.Separately, a savepoint carried one shared version number for all six session folds. Changing what one fold stores meant bumping that one number, which retired every fold's savepoints -- so a counting fix in
usagemade every long-lived unit refoldstatus,timeline,tools,approvalsandclassas well. The units that paid the six refolds are exactly the long-lived ones savepoints exist for.Why it matters
A panel that reads on a timer either shows a stale value or spends the read walking a log that grows all day. The work board is the surface a conductor watches while a fleet runs, so the read happens repeatedly and the log it walks is the longest one on the box.
The version half is smaller but compounds the same way: it puts a tax on fixing a fold. Correcting a count in one fold currently costs every session a full refold of five folds that did not change, which is a reason not to fix the count.
Not a goal
per-(slot, fold) revision shared by reads and frames, and there is no consumer to need
one. The frame belongs to the change that adds the reader.
rather than by unit, with its own admission rules about the unit vector it was folded
over. The eager cell stays the in-process memo, so a restart folds cold.
workfold's unit list. The resolved list depends on each boundworker's own unit tuple, so a worker slot gaining a unit on a session reset changes it
while no conductor mark moves; that needs its own change and its own argument.
60.8 MiB worst case, measured, so a byte ceiling on top of it never fires.
workandpanel, and no change to what any fold computes.Every fold's
start,stepandrenderare untouched.USAGE_TYPES,_usage_*,_timeline_step,entry_types.pyorterminal.py-- those belong to a sibling branch.What changed (motivation -> approach -> change)
Per-fold version.
src/kiro_crew/crew_log/projection.pymovesstate_versiononto_Foldand addsfold_state_version(name);src/kiro_crew/crew_log/checkpoint.pyreads iton the write and the resume, so a file records its own fold's version and a bump retires that
fold's files alone.
src/kiro_crew/projection/checkpoint.py's mapping table says per fold.FOLD_STATE_VERSIONandcheckpoint.CHECKPOINT_VERSIONare deleted:session_tree_projectiondeclares its own and imports neither. The digest pin is per fold, so a missed bump reddens CI.
Eager folding.
_Foldgainsmode;workandpanel, the two slot folds a dashboardreads, are eager.
timelinestays lazy: it looks like a live feed, butits value is a window nothing reads.
src/kiro_crew/crew_log/eager.pyholds the queue and the one daemon thread. It foldsthrough
read_slot_projection, the dashboard route's own call, rather than a second path: acontinuation rule a copy missed is a wrong record.
src/kiro_crew/crew_log/emit.pycallsnote_commitin the append job -- one frozensetlookup, then
put_nowait. The filter is literals, so rejecting the bulk of a log reaches noregistry; a test derives the same set from it, so divergence reddens CI.
The wake carries the BOARD from the entry, because a worker's
work/recordednames theconductor's board while its header names the worker's own, and it is enqueued AFTER the
emitter records unit order. A batch applies closers BEFORE advances, which is commit order:
an append after a
session/closedis accepted. A closer outranks a wake on two conditionstogether -- same unit, since a seq numbers one unit's file, and that unit's own board, since a
worker's report names the conductor's. A full queue drops and counts;
mode="eager"withoutaffectsis refused at import.One pass per (slot, fold).
fold_slot_warmholds a per-key lock for its whole body. Twowarm passes share the cell and the checkpoint pairs it with the caller's own
reached, sothe pass that read a shorter tail could return the other's newer state under its own older
seq -- the number a reader truncates against. The lock table holds only passes in flight: the
last holder out drops the entry.
Retention. Eviction is least recently stored or advanced, so a cell read without being
stored stays at the front and can be evicted.
stop_for_testsputs a sentinel on the queue,since a parked
Queue.getcannot see a flag and the join would wait out the poll. It raiseson an expired join and changes nothing, so the handle stays joinable.
retire_for_testsdrains the writer, FENCES, then stops -- the writer is its own thread, andan append landing during the stop would otherwise start a daemon outside the pinned home, so
worker creation re-reads the fence under its own lock and
_enqueuedrops a wake whose queuethe reset took.
Nothing is pushed. A slot fold's
last_seqis the newest unit's own seq and conductorunits fold first, so a conductor-side change on a board with a worker bound leaves it
unmoved, and a client ordering frames by it would discard the change. Section 5.1 of
docs/system-specs/modules/crew-log-projection.mdrecords this and the numbers behind it.Backwards compatibility
Compatible. No fold's stored state, rendered value or version number changes here, so every savepoint on disk resumes exactly as it did: each fold stands at the version the base already gave it (4 for the six session folds, after the credit-bucket change that arrived with #14966), and every file is written with and demanded back at
fold_state_version(name). The two deleted names,FOLD_STATE_VERSIONandcheckpoint.CHECKPOINT_VERSION, are read by nothing:session_tree_projectionhas its ownCHECKPOINT_VERSION, and the one test that reached for the alias asksfold_state_version("usage")instead -- which is the number ausagefile actually carries._Foldgains two fields with defaults, so a fold declared without them behaves as before.eageris a new module; with the crew log switched off nothing imports it. No environment variable is added and no WebSocket frame kind is added.Tests
test/test_crew_log_fold_eager.py, 35 tests, every one watched red before the code existed:mode="eager"with noaffectsis refused where the fold is built, with the same fold accepted once it declares one; the eager set is slot-keyed andtimelinestays lazy.work/recordedadvances theworkmemo with no reader; amessage/sentwakes nothing; a full queue drops and counts instead of blocking; a dropped wake is still recovered by the next read.drainanswers about folding, not about queue emptiness: a wake taken off the queue and not folded keeps it False.session/closeddrops the slot's memos; past the count ceiling the cell furthest from the back goes, and a cell advanced after an older neighbour outlives it and its read folds cold; the worst-case cell size the ceiling was decided on is re-derived from the folds' own caps rather than quoted, so a fold whose state outgrows that decision fails here.stop_for_testsraise instead of returning.getinstead of waiting out its poll (driven with the poll patched to 30s, so the unfixed path takes 31s against the fixed path's 1.5s); a wake inside the fence starts nothing and the resume hands the worker back; a wake landing mid-stop starts no replacement; and_ensure_workercalled under the fence hands back a queue while starting no daemon, which is the one window a thread race cannot be scheduled to drive.Existing pins updated rather than relaxed: the digest pin in
test_crew_log_checkpoint.pybecame per fold (_FOLD_STATE_PINScarries a digest AND a version per fold), and the kernel-definition test intest_crew_log_projection.pynow compares against the fold's own version instead of the module maximum.test/test_work_fold_warm_read.pysilences the eager folder in its fixture and is otherwise untouched: its three measurements stand exactly as written, and they are the reason the fixture needs it -- a background thread folding the same log lands its reads in a measurement of what a read costs.A 28-case mutation matrix was run (kept beside the worktree, not committed). Each mutation inverts one behaviour and every one is caught by a named test: the two entry-type filters individually and together, the queue bound, the closed-session drop, the count ceiling, the per-fold version on the write path, the import-time eager assertion, memo reuse,
drain's counters, the board resolution in both the folder and the emitter, the batch key, the per-key fold lock, the lock entry's release, the raise on a stop whose join expires, an eager fold's type left out of the append path's cheap filter, a re-stored memo appended without moving to the back, a batch that advances before it drops, a closer matched by board instead of by unit, a timed-out stop that drops its handle, each half of the closer predicate on its own, a retirement that leaves the gap between tests unfenced, a fence that is never lifted, a fence raised after the stop rather than before it, worker creation that skips its own fence re-read, an enqueue that trusts the fence and never checks the queue in its hand, and a closer that drops every fold of the slot rather than the eager ones. Two of those cases had no catcher when first run -- the fence re-read and the slot-wide drop -- and each is answered by a test written for it, which is what the matrix is for. The stale-queue case arrived with its own fix and its own catcher. A pre-flight aborts if the tree is not green before any mutation, a substitution that does not match aborts rather than reporting on a stale mutation, and every file any mutation touches is in the restore set -- a gap there is what made two cases read as surviving in an earlier run.Manual verification
Three measurements, because three claims in this PR turned on numbers. The machine's live crew log is not readable from an agent shell --
~/.kirocrew/crew-log/sessionsis an empty directory whose mtime is the session's start andfind -name '*.jsonl'returns 0 files -- so the logs were built and each script says exactly what it built.Which half of a read this change removes. A board with four bound workers, each with its own message-heavy log, plus a message-heavy conductor log; 1,813 entries across 5 units. A cold
read_slot_projection(slot, "work")parses 2,422 entries: 609 discovery, 1,813 fold. The warm read parses 609. So the change removes 74% of a cold read's parse work, and what remains is discovery, unchanged and paid on every read.What the retained memos can hold. Measured at each fold's declared caps, not extrapolated:
radaratRADAR_ITEM_LIMITis 995,342 bytes, so 64 of those is 60.8 MiB;workatWORK_ITEM_LIMITplusWORK_EVENT_LIMITis 138,067 bytes, 8.4 MiB for 64. This is what removed the byte ceiling: above 61 MiB it never fires, and below it the eviction order stops meaning least-recently-used.An earlier revision of this PR published 1.2 MB for a
workcell and ~79 MB for the table, from a per-item cost multiplied out. Both were wrong -- the per-item figure included the board-level event log that all 256 items share -- and the numbers above are direct measurements of the state at the caps. The body and the spec are corrected.Per-fold state sizes. Over a 4,158-entry session (1.0 MB on disk):
work30,779 bytes,timeline25,179,tools3,038,panel1,251, everything else under 600. With every declared cap pushed:tools369,531,approvals265,081,work60,068.Screenshots / video
N/A -- no user-visible frontend surface. No frame kind, route or rendered value is added, so a panel shows exactly what it showed; only what a read costs to produce changes.
Related Issues
no linked issue: this is a new capability rather than a reported defect, and the repository's policy is that only problems and bugs get a tracking issue -- the PR carries the whole context.
Two comment-only corrections ride this head rather than a follow-up: they answer the advisory findings on
span=c589740bbecfandspan=20b7f8b8cda7--_advance's docstring, which promised a publish this module does not do, and aconftest.pycomment naming a function that exists nowhere. They were held back while the head was otherwise settled, since a rebase for a comment re-rolls four review lanes; the blocking finding on_enqueuebought that rebase anyway, so they land with it.Checklist