Repository navigation
feat(work-ledger): the work fold forgets closed items, so a board never fills - #19068
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, check one cost: the cache now keeps every record, so What this change shipsInventory (10 items) — 10 justifiedIntent: let a long-running conductor board keep dispatching after its 256th item (FIX, under accepted
[FIRST-PRINCIPLES-REVIEWED] cbfe739 Description read[DESCRIPTION-READ] 09a3064fa64ba8492fe3abf7165b3e0fc017fc6b1fa7451546615adc6ee16e1c That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Only the fold is bounded now: the writer's cache keeps every record, so each create still costs more over the board's lifetime, and the cache disagrees with the fold for good. Watch
[DESIGN-REVIEWED] cbfe739 Description read[DESCRIPTION-READ] 9b602ca8a0e5777e2b89bde3eb754d50bd099a654ca288890b6f87a3aa3e8e42 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsFINDING -- test/test_work_ledger_projection.py:1489 -- An eager-folder lock collision makes on-loop False positive or not applicable? A repository writer can comment: |
71f6bb0 to
d3dea23
Compare
|
span=2d1b4f630a03
|
|
span=2d1b4f630a03
|
|
span=edc024cbb4a9
|
|
span=1ed8a0d06001
|
|
span=078d85b83cac
|
|
span=4a1d6dd514d9
|
|
span=400beb1272eb
|
|
span=9c3fdf6125fc
|
|
span=f6626ae6e86b
|
|
span=ab4416405989
|
|
span=c794979e0d5e
|
|
span=3e6cfcccfdf4
|
|
span=8b575b521776
|
|
span=06ae6337753a
|
|
span=1560236d6b9b
|
d3dea23 to
126588e
Compare
|
@buluoray thanks for the review at Blocking:
Non-blocking
Could you take another look when CI finishes? |
|
For whoever drives this PR, one gap at
Only a machine that ran the old head has such a file. Suggested fix: move the work fold to My claim above is withdrawn: |
bfacc27 to
d6eec1e
Compare
…, counts and binds them A forgotten item a later baseline can name -- one born from a baseline, or a created item whose recorded stamp failed, so its conductor bind and a worker's report after it carry baselines -- keeps its tombstone past the cap: the cap evicts other ids first. The worker's unit folds after the conductor's, and with the tombstone gone that report brought the item back as open and counted its create twice. The fold names those protected ids in its forgotten block, so a rebuild keeps a legacy record whose baseline the fold already counted as forgotten instead of adding it again as legacy. A recorded record older than the board's first surviving entry (its create's unit pruned) is kept as the cache holds it, never unlinked. A rebuild keeps the binding of every forgotten record it keeps, as it does for legacy ones, so a worker bound to such an item is not answered not_bound afterwards. The rebuilt create total sums pieces that can overlap without one holding the other (a pruned unit's create, a torn forgotten record), so it could come out short and let a goal at its cap create one more item. A trusted cache header's count is now a floor for it.
d6eec1e to
cbfe739
Compare
buluoray
left a comment
There was a problem hiding this comment.
Verdict: APPROVE at head cbfe739fa4.
What it fixes
A conductor board stopped dispatching after 256 lifetime items. Now the work fold forgets closed items once 32 newer closed ones sit behind them. It keeps their per-state counts in forgotten, and every record stays in the store. My previous review blocked one display regression: the board card's items tile and its "+N in round" line counted only the items the fold still held, so they disagreed with the whole-board total on the same card.
What I verified
pipeline_board_contract.py:589: theitemstile is now held +forgotten.items, the same sum asprogress.total(:540).pipeline_board_contract.py:518-520:added_sinceaddsforgotten.in_round.crew_log/projection.py:6450-6451:_work_forgetbumpsin_roundfor an item from the current round.:6466-6474_work_set_roundzeroes it when the round moves, and both round writers (:6499goal header,:6511baselineboard_round) go through it. A queue board that never bumps its round keeps the whole board in "in round 0", which is the pre-PR behaviour.- I ran
panel_card_dataon the 300/298/2 fixture. It now printsstat_items=300and299 of 300 settled · +300 in round 1. Before, the tile read 2. - The non-blocking points from last round are addressed: the RFC §4.2 implementation note,
implementation-prs: [19068], the README row (nowaccepted), and thegoal-conductor/SKILL.mdwording. - 526 tests pass across
test_pipeline_board_provider.py,test_pipeline_board_card_publisher.py,test_work_ledger_projection.py,test_work_ledger.pyandtest_work_ledger_board.py.
Mutations (one at a time, tree clean between them):
- Tile back to
len(view["items"]):test_the_items_tile_and_added_this_round_count_forgotten_itemsfails. - Dropping
added += forgotten.get("in_round", 0): the same test fails. - Dropping the
in_roundincrement in_work_forget:test_forgotten_items_count_toward_added_this_round_until_the_round_movesandtest_a_long_board_folds_to_its_open_items_plus_the_closed_windowfail. - Dropping the reset in
_work_set_round:test_forgotten_items_count_toward_added_this_round_until_the_round_movesfails.
Blocking
None.
Non-blocking follow-ups
crew-log-projection.mdaround line 947 describesforgottenonly as "its closing state". A one-line list of its keys (created,items, the per-state counts,in_round,baseline_ids) would help the next reader.- Carried over:
list_work_itemsstill reads every record file under the board lock. The RFC note now records this as an accepted cost, so a trim or an open-count index is the follow-up once boards reach thousands of items.
Not verified
- I did not open the HTML templates (
goal-board,work-kanban) in a browser. Their whole-board sums were checked from source only. - I did not check CI status on this head.
Problem / Motivation
Goal: A conductor board keeps dispatching for as long as it lives, instead of stopping for good after 256 items.
A board refused its 257th
createwithitem_store_full, even when only one item was still open. Closing items gave nothing back, and the only way out was a manualkirocrew ledger-sweep --purge, which skips a board that is still in use. A long-lived queue conductor therefore got stuck.Why it matters
Queue conductors (one item per ticket) are meant to run for weeks. Today each one stops after 256 dispatches and needs a person to clean it up, and a board that is still busy cannot be cleaned up at all.
Not a goal
item_cap, default 20) or how it is counted.ledger-sweep's purge rule.What changed (motivation → approach → change)
Decision recorded in rfc-work-fold-forget-closed.md (#18982).
Root cause: the crew log's
workfold kept every item it ever saw, closed ones included, so its memory grew with the board's lifetime. The fold capped itself at 256 items, and the writer capped lifetime creates at the same number so the fold never overflowed.Change: the fold (
crew_log/projection.py, window inwork_vocab.py) now forgets a closed item once 32 newer closed items sit behind it. It keeps the item's id (a late worker report for it is dropped, not parked) and counts it per closing state in a newforgottenblock. Creates and closes interleave in the conductor's log, so during a replay the fold holds only what is open plus that window. The writer drops the lifetime ceiling and theitem_store_fullcode.The store's cache keeps every record, so nothing on the board is deleted and every item stays listed. A rebuild keeps a closed record the fold does not hold instead of refusing (an open one the fold lacks is still refused), and sets
created_totalfrom the fold's count of every item it held. The crew status ("N of M items accepted") and the pipeline board (pipeline_board_contract.py) add the forgotten counts to every whole-board number: the progress total, theitemstile and "+N in round" (the fold counts forgotten items of the current round inforgotten.in_round, zeroed when the round moves). The columns show what the fold holds. The goal-conductor skill (builtin_skills/goal-conductor/SKILL.md) reads spend fromgoal_items_usedrather than counting items. Theworkfold's savepoint version moves, and the generated fold catalogue (dashboard-template/folds.json,FOLDS.md) is regenerated.Rebuild, two further changes in
work_ledger.py:_rebuild_locked. First, the pass over cache records the fold does not name reads each one withstrict=True: a present-but-momentarily-unreadable record (an I/O error) raises and the rebuild's undo restores every file, instead of reading the record as absent and unlinking it. Torn content still reads as absent and is dropped, except on a board whose fold has forgotten closed items: there an unparsable record may be a forgotten item whose only whole copy is the cache, so the rebuild leaves its record and events files in place and does not count it. Second, that pass tells a legacy record from a forgotten one byrecorded_at, not by stamp order alone: legacy means an UNRECORDED record from before the board's first recorded entry (the log never held its create, so it is added to the total); a recorded closed record the fold lacks is forgotten (the fold'sforgotten.createdalready counts it). Stamp order alone loses a closed legacy record or counts the board's first item twice. The pipeline board's column heads (pipeline_board_contract.py:panel_card_data) take their numerator from the progress segment, which includes forgotten items, so a column reads the same share the progress bar does.Docs:
src/kiro_crew/docs/work-ledger.md,docs/system-specs/modules/crew-log-projection.mdand the caps/errors lines ofdocs/request-for-change/rfc-conductor-work-ledger.mddescribe the open-bounded board. The RFC (docs/request-for-change/rfc-work-fold-forget-closed.md) and itsREADME.mdrow carry an implementation note: the cache keeps every record, where the RFC had planned to trim it.Backwards compatibility
Compatible: every create that worked before still works; the only refusal removed is
item_store_full, and nothing else is tightened.Reader: src/kiro_crew/dashboard/handlers/work_ledger.py:_CODE_STATUS -- chat -- the
item_store_fullmapping is gone with the code;test_every_store_code_maps_to_the_status_the_rfc_tabulatesReader: src/kiro_crew/pipeline_board_contract.py:_derive_numbers -- crew page -- progress and "+N in round" add
forgotten; columns show held items;test_forgotten_closed_items_join_the_progress_but_draw_no_cards,test_the_items_tile_and_added_this_round_count_forgotten_itemsReader: src/kiro_crew/pipeline_board_contract.py:_stats -- crew page -- the
itemstile addsforgotten.items;test_the_items_tile_and_added_this_round_count_forgotten_itemsReader: src/kiro_crew/crew_main_contract.py:_work_fields -- crew page -- totals add
forgotten;test_forgotten_closed_items_still_count_toward_the_board_totalsReader: src/kiro_crew/dashboard/handlers/agent_panel.py -- crew page -- reads the fold's items; recent closed items plus open ones
Reader: src/kiro_crew/work_ledger.py:rebuild_from_projection -- chat -- keeps closed records the fold forgot;
test_a_rebuild_keeps_a_closed_record_the_fold_forgotReader: src/kiro_crew/work_ledger.py:_rebuild_locked -- chat -- an unreadable record aborts the rebuild instead of being deleted; a closed legacy record is added to the total;
test_a_present_but_unreadable_record_is_not_deleted_by_a_rebuild,test_a_closed_unrecorded_legacy_record_is_added_not_folded_in_a_rebuild,test_a_recorded_first_item_the_fold_forgot_is_counted_onceReader: src/kiro_crew/pipeline_board_contract.py:panel_card_data -- crew page -- column head numerator is the state's full share;
test_the_column_head_shows_the_whole_state_share_not_just_held_cardsReader: src/kiro_crew/builtin_skills/goal-conductor/SKILL.md -- chat -- counts goal spend from
goal_items_usedReader: src/kiro_crew/dashboard_templates/builtin/goal-board/template.html:drawLoop,drawStates -- crew page -- the manifest binds
forgotten; accepted count, state bar and item count add it;test_the_whole_board_numbers_count_forgotten_itemsReader: src/kiro_crew/dashboard_templates/builtin/work-kanban/template.html:boardTally,boardTotal -- crew page -- the manifest binds
forgotten; headline, strip, column counts and item count add it (round rows stay per-round, held items only);TestForgottenItemsCountTests
test/fixtures/dashboard_templates/pod_folds.json: the capturedworkfold gains theforgottenblock (all zero,created3) the fold now renders, so the path-resolution test covers the new binding.In
test/test_work_ledger_projection.py,test/test_work_ledger.py,test/test_work_ledger_tools.py,test/test_work_ledger_doc.py,test/test_crew_main_contract.pyandtest/test_pipeline_board_provider.py:test_a_long_board_folds_to_its_open_items_plus_the_closed_window: 300 interleaved create/close fold withomitted == 0, peak held items ≤ window + open, correctforgottencounts.test_a_late_report_for_a_forgotten_item_is_dropped_not_parked/..._still_in_the_window_still_applies,test_a_create_reusing_a_forgotten_id_is_a_new_item,test_the_tombstone_set_is_capped.test_a_rebuild_keeps_a_closed_record_the_fold_forgot,test_a_rebuild_still_refuses_an_open_item_the_fold_does_not_hold,test_a_rebuild_sets_the_create_counter_to_what_the_record_holds,test_a_present_but_unreadable_record_is_not_deleted_by_a_rebuild,test_a_closed_unrecorded_legacy_record_is_added_not_folded_in_a_rebuild,test_a_recorded_first_item_the_fold_forgot_is_counted_once(each fails under one of the two stamp-order rules),test_a_torn_record_on_a_board_that_forgot_items_keeps_its_files.test_a_board_keeps_dispatching_past_256_lifetime_creates,test_forgotten_closed_items_still_count_toward_the_board_totals,test_forgotten_closed_items_join_the_progress_but_draw_no_cards.test_the_column_head_shows_the_whole_state_share_not_just_held_cards,test_the_items_tile_and_added_this_round_count_forgotten_items; fold counter:test_forgotten_items_count_toward_added_this_round_until_the_round_moves.item_store_fullrefusal.Determinism: no new test orders on a clock stamp, ids are fixed or come from the store, and nothing sleeps. The related-test set passes (25,043); every failure is environment-only on this host and fails the same way on clean main: a missing
markdown_itmodule in the local venv, andtest_a_worker_edit_of_its_own_item_file_is_refused_and_the_store_still_writes.Manual verification
N/A — the fold and rebuild paths are pure file/state logic covered end to end by the tests above.
Related Issues
Closes #18583
Supersedes #18584 (generation archive), which closes once this lands.
Checklist