Repository navigation
feat(crew-log): fold the session ledger from the session crew log - #12027
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Ledger recording now refuses by default: Watch
Suggestions
[DESIGN-REVIEWED] a1ab61e |
GPT 5.6 Review — ✅ no blocking findings (all downgraded on adjudication)GPT 5.6 flagged blocking issues on This comment is updated in place on each push. Review detailsBLOCKING -- src/kiro_crew/dashboard/handlers/session_ledger.py:181 -- A non-durable update is acknowledged as recorded Adjudication (Opus 4.8) — is blocking on each finding proportionate?The code returns F1 — Harm: LOW. Conditions confirmed: False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS Ledger recording now refuses on every default build (crew log off), a trade-off the description itself hands to the maintainer — and the delete-refusal subsystem is undeclared. Not justified as shipped
What this change shipsInventory (10 items) — 7 justifiedIntent: give a session's durable work state one authority — the session's append-only crew log — so the record and its history can never disagree. A re-architecture (ADDITION of the fold, removal of the document) whose provenance is the in-progress
[FIRST-PRINCIPLES-REVIEWED] a1ab61e |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsNo findings. Both candidates concern the ledger-as-projection conversion and both fail falsification. CANDIDATE 1 (dashboard CANDIDATE 2 ( No self-originated finding cleared 80. [OPUS-REVIEWED] a1ab61e Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
9f5c02d to
14a0a26
Compare
14a0a26 to
0fb8446
Compare
|
Every finding on GPT F1 -- existing durable ledgers disappear after upgrade. Fixed. The first update that finds a slot's folded record empty carries the pre-projection document into the log as one entry, then appends its own. Carried as an entry rather than kept as a read fallback, so the log stays the single authority and the document is consumed instead of consulted. My original trigger was wrong in a way the tests caught: a session's crew log is created on its first turn, so keying the carry on "no unit exists" would never have fired. It keys on the empty folded record. An entry holds one rejected approach, so the carry brings goal, phase, next, artifacts and the newest approach, and its event names how many earlier approaches and event lines it could not bring, which keeps that loss visible rather than silent. The phase rides with that event, so the carried entry keeps the invariant the record rests on. GPT F2 -- record success can precede durable persistence. Fixed, although adjudication downgraded it. GPT F3 -- an incomplete unit can poison the slot cache. Fixed. The finding is exact: Design review, third Watch item -- every read re-folds all entries from seq 1. Fixed. This was a real regression against a documented promise, and the suggested remedy was the right one. The checkpoint is now kept per slot and advanced over the entries that arrived since, through the same seq-anchored machinery a cold fold uses. A different unit list, a newest unit whose seq went backwards, or an empty cache each force a cold rebuild, because each would otherwise be a wrong answer rather than a slow one. A test asserts the warm fold equals a cold fold after every update, and it earned its place immediately: it caught the cache keyed on the slot alone, so two data homes in one process could share a checkpoint. The key now includes the data home, the same precaution the store's scan fingerprint takes. First principles, item 8 -- the First principles, item 7 -- the slot-keyed fold surface is generalized over one variant. Keeping it. First principles, item 9 -- two specs rewritten wholesale, deleting rationale for unchanged behavior. Correct, and worse than the wording suggests: the rewrite cut Design review, first Watch item, and first principles item 2 -- the ledger now depends on a default-off flag. Both reviews are right that this needs an owner rather than a note, and it is with the maintainer who owns scope for this work, with the alternatives written out: accept the gating, sequence behind the flag defaulting on, or keep the document as a flag-off fallback. I have not chosen on their behalf and have not used an override. |
0fb8446 to
19b3c83
Compare
19b3c83 to
0e40cc6
Compare
|
The spec condensation (first principles item 9) is addressed on head Both files were restored from base and edited additively instead of rewritten. What the two specs now add rather than replace: the slot-keyed |
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- src/kiro_crew/dashboard/handlers/session_ledger.py:175 -- a refused append still returns success BLOCKING -- src/kiro_crew/session_ledger.py:1256 -- unit IDs are retained without their declared byte bound [BLOCK-MERGE] e4208fc |
Both GPT blocking findings on
|
The shard-7 red on
|
|
Disposition for span=9bc7f518c977 (WATCH, recording refuses on a default build): ACCEPT AND DEFER to the maintainer who owns scope. The reading is accurate. On a gateway without Nothing in this change set can clear it, so it is recorded here rather than patched. It clears exactly as the lane says: a maintainer accepts the flag dependency on this head, or this lands with or after the release that turns the flag on by default. The refusal is loud and its remedy is one environment variable, which is why it ships as a watch item and not as a silent degradation. |
|
Disposition for span=c50a8e48b769 (WATCH, rollback then re-upgrade loses ledger state): ACCEPT AND DEFER, and the reason is that the proposed remedy reopens a hazard this PR spent several rounds closing. The mechanism is real as described. The carry marker reads That marker is doing load-bearing work, which is why it is not simply made re-armable. So the window is accepted and recorded, not fixed here. Downgrading across this change and then re-upgrading is not a supported path; the interim loss is bounded to updates written by the older build while it was in place, and the re-upgraded loop starts from the last carried record rather than from nothing. |
|
Disposition for span=7282b30edc84 (Item 2, default-availability change): ACCEPT AND DEFER to the maintainer who owns scope. Confirmed as stated. A capability that worked on every gateway now refuses on the default one, because This is the one premise in the change set that no code in the change set can settle, because the two ways to settle it are both release decisions: default the crew log on, or add a config key that lets recording proceed without it. Either is larger than this diff and changes behaviour for gateways that are not running this feature at all. Recorded here for the maintainer. The lane's own clearance applies: a repository writer records the decision on this head, or the flag dependency is confirmed as the RFC-intended shape. |
|
Disposition for span=72b714230e06 (Item 6, the new 409 refusal is undeclared): ACCEPTED as accurate, and the missing surface is declared here. The description says only that a permanent delete goes through the funnel that already removes the session, and that does omit a new way for it to refuse. Stating it properly: a permanent delete now writes the slot's ledger exclusion before it unlinks the transcript, and when that write fails the delete does not proceed. The single delete answers HTTP 409 with The refusal is the point rather than a side effect. If the exclusion cannot be written, deleting anyway would leave the slot's surviving units foldable by the next session on a recycled slot key, which is a cross-conversation state leak. Refusing costs the caller a retry; proceeding costs a leak that nothing reports. Declared in this comment rather than by editing the description, because a description edit fires |
|
Disposition for span=76fa355c31fc (Item 7, three new persisted control files are undeclared): ACCEPTED as accurate, and the surface is declared here. The change persists three per-slot files under
All three are control state rather than record content: a reader that cannot read them degrades to the header order and to the un-carried record, which is the same state a crash between the entry and the control write leaves. None of them is an authority over the appended entries, which remain the only record. |
|
Disposition for span=e08b122adb2b (Item 10, The count is right and I reproduced it: What makes it a follow-up rather than a free win is the prose, not the code. So the honest state is that this is a real YAGNI item with a measured zero, and the right place for it is a small follow-up that deletes the wrapper and renames the operation in both specs in one pass. |
|
Disposition for span=21910eb8fc84 (bulk clear double-counts a row when its ledger rollback fails): CONFIRMED REAL, recorded as a fast-follow rather than fixed on this head. I verified the mechanism in the code rather than reasoning about it. In The remedy needs no judgement, because the same function already decided it: one guarded helper used at all three bulk sites, logging what the Why it is not fixed on this head. The lane itself reports no blocking findings and files this as advisory. Reaching it needs an OSError writing a control file during the rollback of a delete that had already been skipped or had already failed, and the outcome is a wrong count in one response: the row is still reported as undeletable with its own code, and the underlying rollback failure is logged with the manual repair path. That is visible and recoverable, so on a head whose board is green across 90 lanes with PR Readiness passing, spending it re-rolls five non-deterministic review lanes for an error path inside an error path. The maintainer merging this should treat it as a small follow-up against the same function. |
|
Disposition for span=596c3fd926a1 (a refused append still follows the committed path): DOWNGRADED ON ADJUDICATION, no action, and I agree with the adjudicator's reading. The lane's own adjudication on this head returned The finding asks that a One part of the same theme was genuinely closeable and was closed on an earlier head: an update too large for a single entry can never become durable however many times it is retried, so size is now refused before the append with HTTP 413 |
iamwhatever
left a comment
There was a problem hiding this comment.
Approved on the current head: CI green, readiness passed, no security-sensitive path in the diff.
The session ledger's state record is a projection of the session's append-only crew log rather than a document beside it. session_ledger.record appends exactly one ledger/recorded entry per call, carrying the fields that call set and the event explaining them, and every reader folds those entries back with a new slot-keyed ledger fold. A slot owns one ACP session id at a time, so the fold joins every session crew log the slot ran under, oldest first, and a write only needs the unit it is in. state.json is neither written nor read; that directory is residue the ledger-sweep command collects, and permanent deletion of a live record removes the session's crew log unit through the existing funnel.
|
Disposition for span=6d658f756250 (WATCH, recording refuses on an unflagged gateway): ACCEPT AND DEFER to the maintainer who owns scope. The reading is accurate. Nothing in this change set can clear it, so it is recorded here rather than patched. The alternative is to give recording a path that does not depend on the flag, and that means either turning the crew log on by default or adding a second config key. Both are decisions about which release this lands in rather than about this diff. The refusal is loud and its remedy is one environment variable, which is why it ships as a watch item and not as a silent degradation. It clears exactly as the lane says: a maintainer accepts the flag dependency on this head, or this lands with or after the release that turns the flag on by default. This is the same item already dispositioned on head 164489c in comment 5748562751. That head was rebased onto current main to clear a merge conflict that blocked the merge button; the rebase re-rolled the design lane and the finding returned with its substance unchanged. |
|
Disposition for span=107bbf8dad7e (WATCH, the upgrade carry brings only the newest rejected approach): ACCEPT AND DEFER, and the loss is counted rather than silent. The mechanism is real, and I checked it against the code rather than against the description. In Two things bound it. The entry shape holds one It clears exactly as the lane says: the carry preserves the full rejected-approach list, or a maintainer accepts the counted loss. Widening the entry shape to hold a list of approaches is a larger change than a converged head should carry, which is why it is recorded here rather than patched. |
|
Disposition for span=8b19a9b816da (Item 2, default-availability change): ACCEPT AND DEFER to the maintainer who owns scope. Confirmed as stated, and re-checked on this head: This is the one premise in the change set that no code in the change set can settle, because the two ways to settle it are both release decisions: default the crew log on, or add a config key that lets recording proceed without it. Either is larger than this diff, and either changes behaviour for gateways that are not running this feature at all. The lane names a maintainer This item was dispositioned in the same terms on head 164489c in comment 5748563290. That head was rebased onto current main to clear a merge conflict that blocked the merge button; the rebase re-rolled this lane and the item returned under a new span id with its substance unchanged. |
|
Disposition for span=e78a16081eb8 (Item 4, the new 409 refusal on delete is undeclared): ACCEPTED as accurate, and the missing surface is declared here. The description says only that a permanent delete goes through the funnel that already removes the session, and that does omit a new way for it to refuse. Stating it properly: a permanent delete now writes the slot's ledger exclusion before it unlinks the transcript, and when that write fails the delete does not proceed. The single delete answers HTTP 409 with The refusal is the point rather than a side effect. If the exclusion cannot be written, deleting anyway would leave the slot's surviving units foldable by the next session on a recycled slot key, which is a cross-conversation state leak. Refusing costs the caller a retry; proceeding costs a leak that nothing reports. This item was dispositioned in the same terms on head 164489c in comment 5748563553. The rebase onto current main re-rolled this lane and the item returned under a new span id with its substance unchanged. |
GPT review upheld this on f2fbf02 (adjudication: harm-warrants-remedy). _note_unit_order ran before the flush and before `landed` was known, so a refused or still-queued append still moved its unit to the end of the order file. That rewrite is fsynced immediately, so no crash was needed for the wrong order to persist: a retired unit was pinned ahead of its successor, and a successor resuming with no ledger content of its own then folded the retired unit's stale goal and phase as current and acted on it. The call now runs only once this unit's own newest seq has moved. Gated on `landed` rather than `durable` deliberately: `durable` also requires `drained` and an unmoved refusal counter, and that counter is process-wide, so a concurrent session's refusal would suppress a precedence note this unit had genuinely earned. `landed` is exactly the fact the order file asserts. The sibling site that computes the same `landed` signal, the legacy carry, never published precedence and already refused on it, so there is one defective site and _note_unit_order has one caller. Test pins it and was mutation-verified: neutering the guard reddens test_an_append_that_did_not_land_does_not_publish_the_units_precedence with ('acp-2', 'acp-1') == ('acp-2',), the retired unit pinned last.
|
Disposition for span=6e119094988c (WATCH, recording refuses whenever the crew log is off): ACCEPT AND DEFER to the maintainer who owns scope. The reading is accurate and still holds on this head: Nothing in this change set can clear it, so it is recorded here rather than patched. The alternative is to give recording a path that does not depend on the flag, and that means either turning the crew log on by default or adding a second config key. Both are decisions about which release this lands in rather than about this diff, and both change behaviour for gateways that are not running this feature at all. The refusal is loud and its remedy is one environment variable, which is why it ships as a watch item and not as a silent degradation. It clears exactly as the lane says: a maintainer accepts the flag dependency, or the crew log becomes default-on first. This item has now been dispositioned in the same terms on two earlier heads, in comments 5748562751 and 5751494799. It recurs because each new head re-rolls the lane, not because the substance has changed. |
|
Disposition for span=1950a5454d2a (Item 2, default-availability change): ACCEPT AND DEFER to the maintainer who owns scope. Confirmed as stated, and re-checked on this head. A capability that worked on every gateway now refuses on a stock one, because This is the one premise in the change set that no code in the change set can settle, because the two ways to settle it are both release decisions: default the crew log on, or add a config key that lets recording proceed without it. Either is larger than this diff, and either changes behaviour for gateways that are not running this feature at all. The lane names a maintainer decision, and one route it offers is a review override. That is a maintainer action; it is deliberately not taken by the agent driving this branch, and is recorded here so the decision stays visible to whoever owns it. This item has now been dispositioned in the same terms on two earlier heads, in comments 5748563290 and 5751507102. It recurs because each new head re-rolls the lane, not because the substance has changed. |
|
Disposition for span=2f05529e5dae (Item 5, the control files and the new delete refusals are undeclared): ACCEPTED as accurate, and both surfaces are declared here. The description says only that a permanent delete goes through the funnel that already removes the session, and that does omit two things the diff adds. The persisted control files, all per-slot under The new refusal on delete. A permanent delete now writes the slot's ledger exclusion before it unlinks the transcript, and when that write fails the delete does not proceed: a single delete answers HTTP 409 with Both halves were dispositioned separately on an earlier head, in comments 5748563553 and 5748564164, and this head's lane raises them as one item. The substance is unchanged, and the |
|
Disposition for span=76faa62a45a5 (Item 9, one consumer generalized): MEASURED AND CONFIRMED, deferred to a follow-up rather than collapsed here. I reproduced every count rather than taking them from the description. One correction to what that implies: the fall-through is an unreachable BRANCH, not a dead stack. What makes it a follow-up rather than a free win is the prose. It clears exactly as the lane says: the singular form ships, or a second slot-keyed fold consumes the plural one. Recorded here so the next person does not re-measure it. |
|
Known deferred defect, recorded here so it is on the record at merge time rather than found later.
Mechanism: if any of those raises, evaluating the handler's exception type touches an unbound local, so Python raises Not fixed on this head deliberately: the review board has converged green, and a converged PR is the wrong place to land an unrelated change. Follow-up after merge. Scope: verified at head a1ab61e, and this is the only occurrence. An AST pass over all fourteen changed source files, looking for an except clause naming a module imported inside its own try, returned exactly one hit. |
|
Maintainer readiness-sweep record (agent-assisted, on behalf of @bolichen97), head a1ab61e: verified |
What is the problem?
A session's durable work state and a session's history are two records of the same
session, kept in two places. The session work ledger holds the state record -- goal,
phase, resumable next step, rejected approaches, artifact pointers, a bounded event
tail -- in a document of its own, replaced whole on every write. The session crew log
holds the append-only history beside it and says nothing about that state.
Two consequences follow from the duplication. The ledger's central rule, that a phase
never moves without a logged reason, is a property the writer enforces rather than one
the record guarantees; a reader has no way to tell which store is right when they
disagree. And the two are keyed differently -- the ledger by slot, the crew log by ACP
session id -- so nothing joins them, even though a slot's workstream and the sessions it
ran under are the same work.
Why this issue matters to the user
A long-horizon loop reads the ledger to recover its next step after compaction, and
treats it as authoritative over its memory of prior cycles. When a second store can
disagree with the log, the loop can resume on state the history contradicts and nothing
surfaces the conflict. Reading a session's history has the mirror problem: the phase
changes, the approaches already ruled out and the artifact pointers that history
produced live in a file the history never mentions, so the record a person most wants
while reading a session is the one they cannot see from it.
How our fix solves it
The symptom is two records of one session's state that can disagree. The root cause is
that the ledger owns a document, which makes "what this session's state is" a question
with two authorities. So the document goes and the crew log becomes the single one.
session_ledger.recordappends exactly oneledger/recordedentry per call, carryingonly the fields that call set plus the event explaining them, and every reader folds
those entries back with a new
ledgerfold registered increw_log/projection.py. Thephase rule stops being enforcement and becomes structure: a phase and its reason are the
same appended line, so no ordering exists in which a reader sees one without the other
and no crash can separate them.
The fold is keyed by SLOT and joins every session crew log the slot ran under, oldest
unit first. That join is required rather than convenient: a slot owns one ACP session id
at a time, not for its whole life, so a reset or an agent, model or provider change
spreads one workstream across several units. A seq is comparable only within one file,
so the guard
advanceapplies is re-based per unit -- the state carries forward while theseq restarts -- and the unit list comes from headers proved to belong to the store that
holds them.
Two costs of that shape are paid rather than accepted. The record is read on every loop
wake, so the fold checkpoint is kept per slot and advanced over the entries that arrived
since, instead of re-walking the whole log each time; a changed unit list, a newest unit
whose seq went backwards, or a cold cache each force a full rebuild, because each would
otherwise be a wrong answer rather than a slow one. And the append is waited on before
the call answers, so an acknowledgement is not merely a queued entry.
state.jsonis never written. It is read exactly once per slot, by an upgradecarry-forward: the first update that finds a slot's folded record empty carries that
document into the log as one entry and then appends its own, so a workstream in flight
across the upgrade keeps the state a resume needs. The carry is a consumption rather
than a fallback -- nothing consults the document to answer a read -- so the log remains
the single authority. An entry holds one rejected approach, so the carry brings the
goal, phase, next step, artifacts and the newest approach, and its event names how many
earlier approaches and event lines it could not bring.
Permanent deletion of a live record happens with the session's crew log unit, through
the funnel that already removes it, and
kirocrew ledger-sweepstill collects thelegacy directories. The sandbox and file-tool fences needed no edit, and that is worth
stating rather than passing over: the crew-log root is already a fenced leaf, so the new
home inherits the protection, and the old leaf is retained so residue on disk stays
unforgeable.
One consequence is deliberate and visible to a caller: recording requires the crew log
to be enabled and the session to have one. Otherwise
recordraisesLedgerUnavailableand the route answers HTTP 409
crew_log_unavailable, because a durable record mustnever report a write that went nowhere.
What tests we did
A new suite pins the properties the change has to keep: the record-to-fold round trip;
one call appending exactly one entry; a phase without an event refused with nothing
appended; a phase and its reason proved to be the same entry; the fold joining two units
of one slot with the older unit's goal carried across the boundary and the newer unit's
step winning; units of a neighbouring slot excluded; a unit whose header names no slot
attributed to none; the injected snapshot pinned line by line; a terminal phase
suppressing it and leaving one bringing it back; the field clamps, event tail and
artifact map re-applied on read against hand-written entries.
The review round added more, each pinning a defect it found: the upgrade carry-forward
and what it declines to carry; the entry proved on disk before the call returns; a unit
whose header lands after a scan becoming visible; a same-count unit swap inside one
mtime tick invalidating the slot map; a timestamp outside the datetime range folding to
an empty stamp instead of a 500; and the warm fold equalling a cold fold after every
update, which is what caught a cache keyed on the slot without the data home.
Two existing suites were rewritten where their fixtures exercised the removed write
path. The ledger suite keeps every property that still exists and builds legacy stores
directly on disk for the delete machinery, which still governs that directory. The sweep
suite is fixtures only.
Counts on the pushed head: 855 passed across the ledger, sweep, crew-log type, crew-log
projection, crew-log core, conductor-entry and reference-doc suites; 553 passed across
ten crew-log suites the change does not own, to check for collateral damage. The
repository's backend gate floor passes, including the black, isort, flake8, mypy,
docs-lint, feature-map, comment-history, sync-IO, loop-lock and per-file-coverage gates.
Two
platform_compatcases about/usr/binbeing root-owned fail in this environmentand fail identically on an untouched checkout that contains none of this change, so they
are an environment artifact rather than a regression.
Any other suggestions on the work
The flag dependency is the one real trade-off and a maintainer may want it decided
differently. The alternative was to make the ledger's append unconditional, which would
widen the emitter's own activation rule and create a session crew log for a gateway that
never asked for one -- including a unit retention cannot collect, since nothing would
write its close. Refusing loudly was chosen over degrading silently or widening quietly,
and the decision is with the maintainer who owns scope for this work.
Two follow-ups are worth noting rather than folding in here. Once a
previouspointerbetween a slot's successive units lands, the fold can follow that chain instead of
scanning the sessions root. And the
ledgerfold is deliberately not one of the fivepanel folds, so it is not pushed on growth; a client that wants it live would need a
slot-addressed push rather than the per-session frame.