Repository navigation
fix: order truncating rewrites against same-key slot takeover - #12276
Conversation
First Principles Review (Opus 5.5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Every item traces back to the reproduction test that fails on base. The size comes from the hand-over arm, which makes a lot of permanent surface but doesn't trigger a block. Notes
What this change shipsInventory (9 items) — 8 justifiedIntent: a tab reopened while its own edit is still saving never ends up with a mismatched (torn) history; either the edit applies or it is refused. FIX.
[FIRST-PRINCIPLES-REVIEWED] eac18f2 Description read[DESCRIPTION-READ] 674bc38478c971c12e8f14fa9982f2788a128d85bbb2fb4ca8b9cf0c324f99e9 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the fix targets the read-side race the base reproduction shows, every persisted change only adds something, and no website surface changes. Notes
[DESIGN-REVIEWED] eac18f2 Description read[DESCRIPTION-READ] d007663ab559712b289db53822f28f8d5a341607d1beef97236ff310dc58b87d That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findings (all downgraded on adjudication)Reviewed Opus 5.5 adjudication downgraded every blocking finding. Downgraded: the remedy each one needs is disproportionate to the harm it prevents. Adjudication downgraded all 1 blocking finding(s): the remedy each one requires is disproportionate to the harm it prevents. Review detailsBLOCKING -- src/kiro_crew/dashboard/slot_persistence/takeover_ordering.py:94 -- Post-close deliveries bypass an outstanding deferred dismissal
Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 is a narrow race with a visible outcome the user can undo, so I'm downgrading it. F1 — LOW harm. The failure needs four things at once. First, a close that defers its dismissal to an unpublished |
838275c to
1eefe98
Compare
|
Addressed the GPT 5.6 blocking finding (takeover before claim registration unordered) in The gap was real: a claim registered on the worker cannot be marked by a takeover that fires while the save is still in dispatch, and the unguarded pending-rewrite flush retry has no map recheck to fall back on. The fix follows the finding's suggestion — a per-key takeover generation:
New tests: |
1eefe98 to
f52fee3
Compare
|
Both GPT 5.6 blocking findings on Finding 1 (post-takeover basis authorizes a stale slot snapshot): correct — a basis read at the stale slot's turn is captured after the takeover and observes nothing. Fixed as prescribed: Finding 2 (claim refusal occurs after archive mutation): correct — a refusal after |
f52fee3 to
3116f7f
Compare
|
All three GPT 5.6 blocking findings on Post-takeover basis authorizes a stale slot snapshot: fixed as prescribed. The flush pass re-reads the live occupant after opening its watch and skips a snapshot entry whose key changed owner (the close flow that popped it owns its tail); the shutdown pass re-fetches the occupant per key and applies the same capture-then-verify, so it persists the replacement's state rather than a stale entry's. The same dispatch-time occupancy verify now guards Claim refusal occurs after archive mutation: fixed as prescribed. The commit gate moved ahead of Takeover generations retain every slot key forever: fixed as prescribed — the per-key counter is gone. Dispatchers open a refcounted takeover watch at their last synchronous instant and close it once the save's outcome is in hand; a takeover note bumps only an open watch (no save in flight means nobody is left to observe it), and the last holder out retires the entry, so the table is bounded by in-flight saves. Note on |
3116f7f to
157dc65
Compare
|
All three GPT 5.6 blocking findings on Archival callers treat takeover refusal as success: fixed as prescribed. The close and cleanup archive saves now check the result; a refusal whose teardown watch moved ( Cancellation retires the watch before its worker finishes: fixed. The worker's Takeovers before archival dispatch leave no ordering record: fixed as prescribed. Close and cleanup open one takeover watch before their pop and carry it through every teardown await and every save (archive, hand-over drains), so a resume beginning during the task-cancel wait — publication still pending behind its transcript read — is observed and the rewrite-shaped archive is born overtaken. Pinned end-to-end through the real close handler by Also note for the earlier shard red: |
157dc65 to
a4f1011
Compare
|
GPT 5.6 blocking finding on |
|
Tests ( |
bolichen97
left a comment
There was a problem hiding this comment.
Incremental review of a04f41c against the previously reviewed 57e7bdd.
The new head is a rebase onto current main plus one change on top of what was already judged ready: resume_slot_from_history now counts a resume as published only when the key it noted actually holds the slot it answered with (state._slots.get(noted_key) is outcome.slot). Before this, a resume answered by a live slot under a different key (the session identity dedup in _live_slot_for_resume) settled this key's owed dismissal even though nothing was published here, so the taker later retracted into nothing and the closed tab could come back on restart. The new condition routes that case through the retraction path, and the incarnation witness added in the previous round keeps the deferred marker off the other key's live transcript.
Both new tests pin the right edges: a dedup to another key leaves the dismissal owed and the marker lands only on A while B stays open; a dedup to this key's own live slot still settles. The range diff shows no other change to the PR's own patch beyond the rebase.
No new findings. The held fork CI runs on this head are now approved. Approval waits on the readiness check finishing green on this head.
|
Status on Not pushing — nothing in the diff can change this; it needs the gate fixed or relaxed on main, after which a re-run here should go green. @bolichen97 FYI since you said approval waits on readiness. |
bolichen97
left a comment
There was a problem hiding this comment.
Re-checked on head c866641. This head is a clean merge of main 3e39e7a into the branch. The PR's own patch is byte-identical to a04f41c (same 38 files, same stable patch id), so my earlier read still holds: good to ship on the incremental change.
Not approving yet. What is left:
- The fork CI runs for this head (20 workflows, including Fast Gate, CI and the AI review lanes) are held in action_required and need maintainer approval before they start.
- The readiness label is
readiness: action requiredand the PR Readiness check is red on this head, so readiness has not passed.
Once the runs are approved, go green and readiness reads passed on this head, I will approve and enable auto-merge. No new push is needed from you.
|
Fix. The record stays the witness through the write: Tests ( Push shape. Your merge commit |
bolichen97
left a comment
There was a problem hiding this comment.
Incremental review of 2427299 against the previously reviewed c866641.
This head is more than a merge of main. The branch is rebased onto current main as one commit, and on top of the patch already reviewed it adds a post-commit re-check in record_deferred_dismissal: a same-name create takes no transcript lock, so its put_slot could land between the guard's "still owed" and the rename and the rename would then stamp closed onto the line of a live tab. The new _recheck_after_commit reads ownership again after the rename, still under the transcript lock, and clears the stamp when a publication has won, reporting superseded. The spec paragraph in history.md is updated to match, and two tests pin both edges (create inside the window is un-stamped; no create leaves written with the marker on disk).
The logic is sound. ConversationLog._locked is reentrant (per-key RLock plus held-flock reuse), so calling clear_closed from inside the update_metadata_if callback does not deadlock, and any create that lands after the re-check is ordered after a committed marker, which is the case the existing reopen path already handles.
One 🟡 inline below: the callback now does file I/O, which update_metadata_if documents as forbidden for that hook, and a failure there leaves the stamp on a live tab with the dismissal already settled.
Not approving yet: the fork CI runs on this head are still held, so readiness is not green on this commit.
| if SlotRegistry.dismissal_still_owed(state, dismissal): | ||
| return | ||
| superseded_after_commit[0] = True | ||
| log.clear_closed(dismissal.transcript) |
There was a problem hiding this comment.
The un-stamp runs file I/O inside a hook documented as I/O-free, and its failure leaves a live tab stamped closed.
update_metadata_if says its after_commit_under_lock callback "must not perform I/O or mutate event-loop-affine state". _recheck_after_commit now calls log.clear_closed(...), which reads the file and does an atomic_write. It works because the lock is reentrant, but the contract no longer describes its callers. The failure path is the real gap: if clear_closed raises (disk full, permission, a transient read), the exception escapes update_metadata_if, the except returns "failed", and the next exit's retry sees dismissal_still_owed False (the publication already settled it), so nothing in this path clears the stamp. Unless a later save happens to rewrite that line, the newly created tab's line keeps closed: True and is hidden at the next restart, which is the exact outcome this change prevents.
def _recheck_after_commit() -> None:
if SlotRegistry.dismissal_still_owed(state, dismissal):
return
superseded_after_commit[0] = True
log.clear_closed(dismissal.transcript)Suggestion: either widen the update_metadata_if docstring to allow a same-key metadata rewrite in the hook, or set only the flag in the callback and do the un-stamp right after update_metadata_if returns (a compare-and-clear under its own lock, for example clear_closed(..., only_if_closed_before=...) keyed on dismissal.closed_at), logging and retrying on failure rather than reporting failed for a dismissal the registry has already settled. A test that makes clear_closed raise once would pin it.
Rule: a lock-held callback must honor the contract its host documents, and a recovery write must not fail into a state no retry revisits.
There was a problem hiding this comment.
Fixed in 96a23e120, your second option. The after_commit_under_lock callback now only records the verdict (_recheck_after_commit sets the flag; test_the_after_commit_hook_does_no_io pins by AST that its only call is dismissal_still_owed), and the un-stamp runs right after update_metadata_if returns, as a compare-and-clear under the transcript's own lock: clear_closed(..., only_if_closed_before=math.nextafter(our_closed_at, inf)), so exactly the stamp we wrote comes off and a later legitimate close of the created tab is never removed (test_an_unstamp_never_removes_a_newer_close_of_the_created_tab).
The failure path you traced is closed: the dismissal is settled on the registry regardless, so the outcome is superseded either way — never failed — and a clear that does not land is parked as an owed un-stamp on the key's watch (OwedUnstamp, SlotRegistry.park_owed_unstamp; it holds the watch open) and retried by every noter exit on the key (retry_owed_unstamp, called from both the resume and rehydrate wrappers) until it lands or the line provably carries a different stamp. Test: test_an_unstamp_that_fails_once_stays_owed_and_the_next_exit_clears_it — every clear of our stamp raises while the disk is "broken" (the superseded write's own attempt and its exit's immediate retry both fail), the stamp stays on and the clear stays owed, the disk recovers, the next exit on the key clears it. With the parking removed it fails on "the clear must stay owed".
|
|
bolichen97
left a comment
There was a problem hiding this comment.
Incremental review of 96a23e1 against the previously reviewed 2427299 (range diff of the PR's own commit across the rebase onto 512f13c).
The 🟡 from my last review is fixed. Outside the rebase, the only change is that one finding:
_recheck_after_commitnow only records the verdict. Its sole call isSlotRegistry.dismissal_still_owed, so theupdate_metadata_ifhook is I/O-free again, andtest_the_after_commit_hook_does_no_iopins that with an AST check.- The un-stamp runs after
update_metadata_ifreturns, inunstamp_superseded_marker. It is a compare-and-clear:clear_closed(..., only_if_closed_before=nextafter(closed_at))under the transcript's own lock, so a later legitimate close of the new tab is never removed (test_an_unstamp_never_removes_a_newer_close_of_the_created_tab). - A failed clear no longer reports
failedfor a settled dismissal. It is parked as anOwedUnstampon the key's watch and retried by the next resume or rehydrate exit (test_an_unstamp_that_fails_once_stays_owed_and_the_next_exit_clears_it). The parked record takes a plain holder, not a truncating one, sotakeover_ordering_pendingdoes not make readers wait on it. - The
history.mdparagraph is updated to match.
No new findings on this head. A small residual, not blocking: the owed un-stamp lives in memory, so if the clear fails and the gateway restarts before another exit on that key, the stamp stays. That needs a write failure in a narrow window, and it is the same durability every other in-memory owed record here has.
Not approving yet: readiness on this head is action required. Fork CI is still held for approval, most required checks are pending, and Sensitive Change Review and Dependency License Gate report failures. I will look again once readiness is green on this commit.
|
Status on
Not pushing — the diff is final. @bolichen97 when you look again: Sensitive Change Review needs the re-run click; Dependency License Gate needs either the repo's dependency graph enabled or the |
|
Status on What stands between this head and
@bolichen97 that's the set; no push needed from my side. |
|
Update on |
|
This branch conflicts with main, which has moved a lot in the dashboard persistence and chat API files this PR touches. Could you rebase onto current main when you have time? The fix for the linked issue is still wanted, and the Dependency License Gate red on this head should get a fresh run after the rebase. |
|
@bolichen97 — rebased as you asked. Head is now The rebase pushed Verified on |
bolichen97
left a comment
There was a problem hiding this comment.
Thanks for the rebase. Full review at 50b88d3.
One thing blocks the merge, and it is in the description, not the code. PR Hygiene fails with: "This diff takes something away: list each reader as 'Reader: : -- -- ' under '## Backwards compatibility'." PR Readiness is red with it. Could you add that list? For example, api_chat_slot_repair_images and api_chat_slot_reset_conversation moved out of chat_api/slot_lifecycle.py, and save_slot_off_loop now refuses some saves it used to commit.
Code review: no blocking findings. Coverage of the 42 changed files:
- All 19
src/files and both docs files were read hunk by hunk. - The code moved from
chat_api/slot_lifecycle.pyinto the newchat_api/slot_repair.pywas checked mechanically: every line ofslot_repair.pyexcept 17 header and import lines comes from the removed block. - The 17 edited test files were read in full. The 4 new test files (about 4,850 lines) were read line by line in a second pass.
Non-blocking notes:
history_append.appendwith a prebuiltentrykeeps the row's owntsand skipsmonotonic_transcript_ts, so a fallback append can land rows after newer ones with older timestamps. Worth a line if any reader orders byts.- An owed
closedmarker whose write fails keeps its watch open until the next resume or rehydrate on that key. If none comes, that watch stays, one per key at most. - The new-incarnation check compares
created_at(an ISO timestamp). A coarse clock could give a delete-and-recreate the same value, and_recreate_the_key_under_a_new_incarnationin the dismissal tests asserts they differ. - A few tests check less than their names say:
test_a_publication_that_noted_nothing_settles_the_dismissal_at_oncenever publishes in its second half, andtest_cleanup_restores_the_original_when_its_handover_to_a_takeover_failsdoes not check_takeover_watches == {}the way its single-tab sibling does.test_slot_close_recreation_race_takeover_retraction.pyalso has anassert chat_persistence is not Nonethat checks nothing.
CI at this head: apart from PR Hygiene and PR Readiness, the remaining reds are cancelled duplicate runs, and E2E, Integration and the desktop builds have passed. Backend test shards were still running. Once the description is updated and CI is green, I expect to approve.
|
@bolichen97 thanks for the full read. The description was updated at 15:19Z with five On the four non-blocking notes: all fair, and the three test ones are cheap (the dead |
|
|
|
@bolichen97 — updated onto Since the head had to move anyway, I folded in the four non-blocking notes from your review rather than re-pushing a bare rebase:
Also the two doc owner-name fixes for the The |
|
Pushed |
|
Tests ( |
|
|
|
Rebased onto |
|
Rebased onto |
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes at 3642a4c for one finding. This head is a rebase of c19d15d: the range-diff shows only the import logging line from main in history_append.py, and I read every other hunk in earlier rounds (44 files).
Blocking (the GPT lane's finding, which I checked in the code): a tied close time lets the unstamp clear a newer close.
unstamp_superseded_marker (takeover_ordering.py:497) calls clear_closed(record.transcript, only_if_closed_before=math.nextafter(record.closed_at, math.inf)). clear_closed keeps the stamp only when close_time >= only_if_closed_before (history_projection.py:615), so any stamp with closed_at equal to ours is cleared. The close time comes from time.time() (channel_slots.py:904). On Windows that clock can return the same value for two closes a few ms apart. So: a publication supersedes our deferred dismissal, the user then closes the new tab, and the unstamp runs after that, on the same tick. The new tab's legitimate closed marker is removed, and with Restore Sessions on, the conversation the user closed comes back after a restart.
A fix that keeps closed_at for activity comparisons: write a per-key close identity that is strictly increasing (or unique, like a nonce) next to closed_at, and have the unstamp clear only when the stored identity equals the one we wrote, checked under the transcript lock. A test that stamps two closes with the same closed_at and asserts the newer one survives the unstamp would pin it.
Other CI: 79 checks green. E2E (stub) and the Linux backend shards are still running.
A truncating history rewrite (rewind, edit-resend, regenerate, switch-variant) commits on a worker thread while a same-name close-and-recreate pops and republishes the slot on the event loop with no lock between the layers. The recreate-won map recheck at the commit boundary only sees replacements already published, and the replacement's own transcript read blocks behind the per-session lock until the save releases -- while a reopen whose read came FIRST, with the save still queued for the lock, publishes the pre-edit window over the truncated file: the tab shows turns the edit removed, the next turn's save writes them back, and the edit is silently undone. Introduce a publication-ordering claim (TruncationClaim): every truncating save registers one for the whole write, every same-key takeover marks it (SlotRegistry.put_slot and the start of a resume's transcript read), and a single mutex-guarded transition decides the winner before the rewrite's first mutation -- ahead of the dropped-line archive the payload composer performs, so a refusal leaves no sidecar recording still-live rows as dropped. A takeover that begins first makes the save refuse with nothing written; a save that commits first leaves the claim committed, and a later takeover knowingly resumes the post-rewrite transcript. A takeover firing while a save is still in dispatch has no claim to mark, so dispatchers open a refcounted takeover watch (TakeoverBasis) at their last synchronous instant and the claim registration compares its count, leaving the claim born overtaken when the key changed hands in the gap. The vouch is a dispatch-time occupancy verify for live-slot writers (save_slot_off_loop callers pinning expected_slot_name, plus the flush and shutdown passes, which re-read the live occupant and skip stale snapshot entries). The close/cleanup teardowns write popped slots by design, so they open one watch before their pop and carry it through every teardown save (the hand-over drain lives in chat_api/handover_drain.py and takes the basis as takeover_basis, handed to it only where the key names the drain's file, _teardown_drain_basis). Their archive result is read, not assumed: a refusal whose watch moved routes to the hand-over arms, a verified deletion (definitive evidence only; an unreadable probe is not a deletion) stays ignored, and any other refusal restores the tab or drains its tail and reports the close failed rather than discarding acknowledged rows. The takeover note is tombstone-gated: a noter whose start postdates the key's close tombstone is a reopen of a closed tab, not a takeover. It notes nothing, waits for the ordering to settle, and is then refused by the closed flag the close wrote, so the close commits the user's edit instead of yielding the key to a reopen that cannot survive it. A delivery rehydrate therefore never wins a close; only an explicit user resume that began before the close does. A noter that noted and then published nothing withdraws exactly the note it opened, through a handle bound to the watch that issued it (a retired watch's handle is a no-op against the fresh watch that replaced it). Accounting is per note and basis-relative, never a net counter: the claim registration and basis_moved both ask whether a note newer than the basis still stands, so one noter's refusal can neither hide nor fabricate another taker's note, and a close never yields the key to a taker that gave up: it commits, or takes the retry arm and restores the original with its tail owed; and a close that handed its key to a still-unpublished taker keeps its dismissal owed until that taker publishes or, as the last one standing, retracts and writes the closed marker the refused archive left unwritten. Claims and notes carry the transcript they write or declare, so a same-name publication resuming a different file leaves a linked tab's archive to commit. The arbitration's sync rules live in slot_persistence/takeover_ordering.py, bound on the chat_persistence facade. The read side honours the ordering: a resume whose takeover note overtook an in-flight save waits, on the loop and bounded by the save's own lock deadline, for the overtaken claims and the key's truncating watch holders to retire (a plain append holds the watch without holding readers; the waiter holds no executor thread), because a refused hand-over save lands the popped slot's unsaved rows append-safely under its lock before releasing and the transcript projection's cached, best-effort-locked reads cannot be ordered by the lock alone. The wait fails closed: on expiry the resume refuses with the retryable lock-contention error (HistoryLockTimeout; 503 takeover_ordering_unsettled at the endpoint) rather than hydrating a window whose missing tail a later rewrite deletes. The append-safe fallbacks persist the complete serialized rows: the history layer accepts a fully built entry alongside its dedup identity, so an appended tail carries the same ts, provenance, variants and meta the committed payload would have. Watch ownership survives cancellation: every executor save is shielded, the worker closes its own basis when it settles, and a caller-opened basis transfers to the worker on cancellation so the watch outlives every takeover the running write must still observe. Switch-variant restores the pre-switch row fields (redaction records included) when its durable write is refused. The append-safe fallbacks settle the transcript header before their first row, under the lock the appends take: ConversationLog.append mints a header with no memory_mode for an absent file and leaves a present one untouched, so a restricted slot's rows would otherwise land under a line every learning reader takes as persistent. The header is created with the slot's retained mode or tightened to it, the carried execution record folded to match and the store a restricted line must not name cleared, with the replacement's own fields kept -- the same ratchet the rows-only save applies. The settled mode is confirmed by re-reading the header under the lock; a header that cannot be read, or that still reads looser than the retained mode, refuses the append outright (the drain reports the rows and the close fails; the gate keeps the window owed). The gate's claim refusal keeps the window owed like every other retryable refusal, so a flush that lost its claim stays dirty; the drain's append runs as a shielded executor future whose basis transfers to its settlement on cancellation, and the rewind handler hands its watch to an unsettled worker the same way. Both replays judge "already persisted" against the whole durable history: append_if_absent gains include_rotated, which folds the size-rotation archives into its absence check, so a window reaching past the live file's rotation cut lands only the rows the transcript lacks rather than re-landing every archived one. A hand-over whose drain does not land restores the popped original while its key is still free, with the compensation a raised archive save already gets, so the tail's only holder is never dropped; and a teardown watch retains a bounded number of declared read targets, refusing an overflowing takeover note at admission. The hand-over drain prepares its rows as the full save does -- the window filtered through drop_notes_authorized_elsewhere, each row built by _build_message_entry -- and declared read targets are recorded and matched by transcript stem, so an alias spelling of one file is the same file. Both replays share one helper, and the archive lookup resolves the physical file stem rotation names its segments after, so a legacy bare-stem transcript's archived rows are not appended again. save_slot_off_loop opens a watch for every save its caller did not open one for: the worker reads _pending_rewrite itself, so an append-shaped dispatch can become a truncating rewrite in the gap, and a takeover landing there must still leave the claim born overtaken. The rewind handler's dispatch-verify refusal settles without a worker: the save task is awaited only on the branch that dispatched it, so a key that changed hands before dispatch audits as history_save_refused rather than as a raised save. Every interleaving resolves to one of two sequential histories. The mutex guards in-memory transitions only and is never held across the file replace, so the event loop never waits on the commit. slot_lifecycle.py crossed the 2,000-line cap on the rebase (main alone is 1,651); the repair-images and reset-conversation handlers move to chat_api/slot_repair.py unchanged, per the responsibility-split rule. Fixes kirodotdev#12090
|
Separately, the Backend shard-1/8 reds on 49 files, 1 commit, rebased on current |
bolichen97
left a comment
There was a problem hiding this comment.
Approving at eac18f2. This replaces my change request at 3642a4c.
My change request is fixed. Every close write now stamps a closed_id nonce beside closed_at: metadata_line.close_identity for slot saves, and the DeferredDismissal's own id for the owed marker. unstamp_superseded_marker calls clear_closed(only_if_closed_id=...), which compares the id under the transcript lock and keeps every other stamp. That includes a close of the new tab on the same clock tick, with an equal closed_at and its own id, and a stamp written without an id. test_an_unstamp_keeps_a_newer_close_stamped_on_the_same_clock_tick pins the equal-closed_at case, and the existing unstamp test now also checks the no-id stamp is kept. closed_id is slot-owned in history_schema, so a reopen drops it with the stamp.
The rest of the delta from 3642a4c: the two noter wrappers now read whether their slot held the key as the body returned, before the first exit await. They answer None / resume_session_deleted only when a slot that was under the key is gone afterwards, so a body that never registered its slot is handed back as built. The golden metadata line and the codec tables now carry closed_id.
Coverage: 44 files. I read every hunk of this PR across the earlier rounds, and each later head was a rebase or a delta I read in full.
CI: 86 checks green, including both AI review lanes, E2E (stub and hosted), the Windows E2E Gateway Boot, Integration and Native Redaction. The 8 cancelled entries in the rollup are older duplicate runs, and each has a newer passing run on this head. Only four Backend Tests (3.12) shards and PR Readiness are still running, and auto-merge waits for them.
|
Thanks @bolichen97 — holding this head as is; nothing further from my side until auto-merge lands. |
Problem / Motivation
Goal: A conversation reopened while one of its edits is still saving never ends up TORN — a tab showing the pre-edit window over a transcript the edit truncated, with the removed turns re-serialized behind the truncation on the next turn and the edit silently undone. The outcome is exactly one consistent history: the edit applied and the reopen reads it (the save committed first;
test_takeover_after_commit_reads_the_rewritten_transcript), or the edit refused — reported to its caller — and the reopen reads the pre-edit transcript intact (the reopen's read came first;test_resume_beginning_mid_write_makes_the_truncating_save_refuse). Never both.Base reproduction:
test_a_reopen_racing_an_edit_never_hydrates_the_truncation_with_the_removed_tail_behind_it(test_slot_close_recreation_race_takeover_ordering.py) fails onmain(verified on5494e55c6with the test file dropped in alone): a tab owing a truncating rewrite is closed, the user reopens it while the archive is queued for its lock, the reopen reads the four-row transcript, the archive then lands two rows plusclosed, and the reopen publishes its four-row window over them — close 200, tab shows four turns, disk holds two and is marked closed. The next turn's save re-serializes the four rows plus the new one (the edit is gone, with no error), and the marker the tab never saw makes it vanish at the next restart. Neither consistent history shows the user any of that; "the edit applied, then a reopen reads two rows" is the one the lane accepts, and this test accepts it too — it fails only on the torn state.Why the smaller alternative does not fix it. The Design lane proposes that the close teardown await any in-flight truncating save for the key before writing its tombstone. The torn case is a READ-side race, not a tombstone-side one: in the reproduction the close's archive is the truncating save, and the reopen's read is already done when it lands — awaiting the save on the close path cannot order a read that began between the pop and the commit, and the tombstone only decides whether a later delivery rehydrate is a reopen (it never gates the explicit resume endpoint, which exists to reopen closed tabs). The original #12090 report is the same shape with a rewind in place of the close: a resume reading while the rewind's save is queued, no close involved. What orders the read is the arbitration this PR adds — a takeover note that the save's commit gate consults, and a read that waits for a pending truncating save it did not overtake — which is also why
test_resume_beginning_mid_write_makes_the_truncating_save_refusepins the refused-edit history rather than the edit-applied one: a reopen that already read the pre-edit transcript must not have a truncation land under it.A truncating history rewrite (rewind, edit-resend, regenerate, switch-variant) can commit its truncated window onto a transcript that a same-name replacement slot has taken over. The user-visible symptom (the base reproduction above): close a tab while its rewrite is saving, reopen the same conversation, and the reopened tab shows the pre-edit window over a transcript the rewrite truncated and closed — the next turn writes the removed turns back and the edit is lost without a word; a restart loses the tab.
Why it matters
The replacement resumes the same conversation the user just had open, and ends up holding a window the transcript no longer matches: a rewrite that semantically belonged to the torn-down tab lands under it, and the replacement's next save overwrites the rewrite. The commit-boundary incarnation check (
expected_slot_name) cannot cover this: the save holds the per-session lock across its whole read-modify-write, so a replacement that begins mid-write is simply blocked from reading until the save releases — and then reads the truncated content. Shrinking the check-to-write gap changes nothing, because the harm is the read landing after the replace, not the publication landing before the check.Not a goal
state._slots.atomic_writepre-replace hook (313 call sites untouched) or holding a mutex across the commit instant.expected_history_key), which main already covers, or the slot-create transaction of fix(chat): build and file nothing for a create that is refused #15592 (a rebase neighbour, not a duplicate).closedflag as any such reopen is.What changed (motivation → approach → change)
Symptom → root cause: slot publication (event loop) and the persistence commit (worker thread) have no ordering contract. The cleanup pops
state._slots[name]and the facade republishes throughSlotRegistry.put_slot; neither takes the per-session lock, and no pre-commit map re-read can observe a replacement that does not exist yet.Approach: the issue sketches a claim that
put_slotconsults, with same-name publication deferred until the claim clears and the write refusing when a publication is pending. This PR lands the same contract with a strictly simpler arbitration: instead of deferring publication, a single mutex-guarded compare-and-set decides the winner, so publication never waits and there is no deferred-publication machinery to keep alive.Change:
TruncationClaim(slot_registry.py): every truncating save registers a claim on its map key for the whole write (opened before the per-session lock, so the lock wait is covered). Claims for one key are held as a list, because a popped slot's hand-over drain and its replacement's own rewrite can be in flight for one key at once.SlotRegistry.note_same_key_takeover: called byput_slotitself (publication) and at the start of a resume's transcript read (rehydrate_slot_from_history_async,chat_api/resume.py) — before the read is dispatched, which is what makes a mid-write takeover visible to the save at all. Fire-and-forget; callers never branch on it._slot_closed_before,chat_persistence.py, readingchannel_slots._RECENT_CLOSES): a noter whose start postdates the key's close tombstone is a reopen of a closed tab, not a takeover — it notes nothing, waits for the ordering to settle, and is then refused by theclosedflag the close wrote. A delivery rehydrate therefore never wins a close (one begun before the close is refused by its ownslot_closed_sincecheck); only an explicit user resume that began before the close does.DeferredDismissal,SlotRegistry.hand_basis_to_dismissal): the teardown watch outlives the close (non-truncating, so the taker's read is not held). It settles only where that is FINAL: a non-noting publication settles it at its insert; a noter's publication settles it at the noter's success exit (settle_dismissal_on_publish— both noters insert before their last refusal point, so the insert itself is not final); the last standing taker's retraction returns it to be written (takeover_ordering.record_deferred_dismissal, under the transcript lock, guarded at write time on the key still being unpublished so a reopen that landed meanwhile is never stamped closed, only onto an unclosed line of a still-existing session) and it is cleared from the watch only once that write is confirmed (settle_written) — a failed write leaves it owed and the next exit on the key (owed_dismissal) retries it. Both paths share that protocol: with nobody standing the arm writes the marker itself against the same registered record, and because no later exit is guaranteed to retry it, a write that does not land makes the close reporthistory_save_failed(the cleanup files the key underfailed) rather than a 200 the recent-session restore would contradict; the dismissal stays owed on its watch so any later taker's exit still writes it. So a taker that gives up after the close answered 200 — before or after registering its slot — leaves the tab dismissed rather than resurrected by the recent-session restore, and a dismissal is never lost to one failed write.SlotRegistry.open_same_key_takeover_notereturns a frozenTakeoverNote(watch_id, seq)— the admitting watch's process-unique id, never reused, plus the note's sequence on it;retract_same_key_takeoverwithdraws exactly that note fromTakeoverWatch.standingand only while the key's watch is still the issuing one, so a stale handle from a retired watch is a no-op against the fresh watch's same-numbered note; both noters retract in afinallywhen no slot was published). Accounting is per note and basis-relative, never a net counter:basis_movedand the claim registration both ask "does a note newer than this basis still stand?", so one noter's refusal — before or after the basis opened — can neither hide nor fabricate another taker's note. Without it a close whose archive lost to a noter later refused reported 200 with noclosedwritten and no session removed, leaving the key to nobody.TakeoverBasis) at their last synchronous instant —save_slot_off_loop, the periodic flush pass, the shutdown pass, and the rewind handler's dispatch — andbegin_truncation_claimcompares the watch's count against the snapshot: a moved count means the key changed hands in the gap and the claim is born already overtaken.save_slot_off_loopopens a basis for every save its caller did not open one for — the worker reads_pending_rewriteitself, so an append-shaped dispatch can become a truncating rewrite in that gap, and only a basis opened before it can observe a takeover that landed there. The vouch itself is a dispatch-time occupancy verify: guarded callers (expected_slot_name, wired into rewind and edit-resend as well) and the flush/shutdown passes re-read the live occupant and refuse or skip when the key is already someone else's. The watch table retains a key only while a basis is open for it, so it is bounded by in-flight saves.chat_api/slot_lifecycle.py) write POPPED slots by design, so they open one watch before their pop and carry it through every await and every save of the teardown; the hand-over drain they call (_persist_handover_tail,chat_api/handover_drain.py) takes it astakeover_basis, handed over only where the key names the drain's own file (_teardown_drain_basis, same module). A refusal is not a commit: the teardowns check the save's result, andbasis_movedroutes a takeover refusal to the hand-over arms (drain the tail, hand over the key-scoped markers, report the close honestly). A delete-won refusal is verified positively (session_deletion_confirmed,slot_persistence/write_guards.py: definitive evidence only, an unreadable probe is not a deletion) before being ignored; any other refusal restores the tab when the key is still free or drains its tail, and reports the close failed rather than discarding acknowledged rows. A hand-over drain carrying a_pending_rewriteretry whose key was taken over does not rebuild: it persists the window through the history layer's id-deduped appends (append_if_absentunderatomic_appends), so the unsaved tail lands and the lost truncation drops out.commit_truncation_claim) decidescommittedvsovertakenin one atomic transition, placed before the rewrite's first mutation — ahead of the payload composition whose dropped-line archive is that mutation, and ahead of the file replace, both under the per-session lock the save holds throughout. Losing refuses the whole save (False, nothing written). For the hand-over shape the refusal lands the popped slot's unsaved rows append-safely under that same lock, ahead of the takeover's queued read.503 takeover_ordering_unsettledatPOST /api/chat/slots/{slot}/resume(HistoryLockTimeoutfor a delivery rehydrate) rather than hydrate a window the refused save appends behind — the user sees the resume's error toast and reopens again. (2) A close whose archive lost the ordering does not report a success it did not commit: it hands the key over (drains the original's tail append-safely, reports 200) when a taker holds or is reading the key, and otherwise restores the tab and reports500 history_save_failed— the user sees the tab stay open with its unsaved tail dirty and closes it again. Both are recorded (SELhistory_save_refused; the close's own log line) because a refusal that is folded into a success is exactly thea-refusal-is-not-a-commitshape.get_metadata_status; unreadable refuses) and derivesdefer_clear/ the reserved publish /cleared_closed_atfrom that read — the save it waited for can be the user's own close committingclosed=True— while the pre-waitmetastays the identity witness the later checks compare against. The wait fails closed —HistoryLockTimeouton the facade, 503takeover_ordering_unsettledatPOST /api/chat/slots/{slot}/resume— rather than hydrating a window whose missing tail a later rewrite deletes.ConversationLog.append/append_if_absent(bodies inhistory_append.py, the owner modulehistory.pybinds them from) accept a fully builtentry, so the append-safe fallbacks persist the samets, provenance,variantsandmetathe committed payload would have.tighten_line_for_appends,slot_persistence/metadata_line.py):appendmints a mode-less header for an absent file and leaves a present one untouched, so a restricted slot's rows would otherwise land under a line every learning reader takes as persistent. The header is created with the slot's retainedmemory_modeor tightened to it, the carried execution record folded to match, the store a restricted line must not name cleared, and the replacement's own fields kept — the same ratchet the rows-only save applies. The settled mode is CONFIRMED by re-reading the header under the lock: a header that cannot be read (the merge defers on a transient read without running its guard), or that still reads looser than the retained mode, refuses the append outright (HeaderUnconfirmed,write_guards.py) — the drain reports the rows and the close fails, the commit-gate fallback keeps the window owed._keep_owed_after_refusal, like the sibling retryable refusals, so a periodic flush that lost its claim leaves the slot dirty; the dispatch-verify refusal insave_slot_off_loopdoes the same.write_guards.append_window_rows: header settled first, each row built by_build_message_entry, appended id-deduped — there is no second row pipeline.ConversationLog.append_if_absentgainsinclude_rotated, which folds the size-rotation archives (read_rotated_messages, the projection's cached corpus, resolved by the PHYSICAL file stem rotation names its segments after — a pre-migration Slack transcript addressed asslack:<ts>lives in the bare<ts>file and its archives are<ts>__*) into its absence check under the same lock, so a window reaching past the live file's rotation cut lands only the rows the transcript lacks instead of re-landing every archived row under a duplicate identity. Single-row writers (cron/workflow results) leave it off. An archive that cannot be enumerated raises into the fallbacks' refusal arms rather than reading as "absent".drop_notes_authorized_elsewhere(a note row stamped for a session the slot was rebound away from is dropped and audited asnote_save_drop, as every full save does) before_build_message_entrybuilds each entry — an append that skipped the filter would land such a row in the other session's transcript. Every transcript-identity comparison this change makes is by stem (transcript_stems, the identityreplacement_shares_transcriptcompares by):note_same_key_takeoverrecords a declared read target as its stems,takeover_noted_transcriptand_teardown_drain_basismatch by stem, so a resume declaringslack_<ts>is seen by a teardown writingslack:<ts>._restore_original_after_failed_close,chat_api/slot_lifecycle.py: restore while_slot_still_ours, the retired nudge loop and the app's dismissal taken back with the tab, the restricted marker re-derived); the close reportshistory_save_failed, the tab stays dirty for the flush, and the waiting resume dedups to the restored tab. A replacement already published keeps the key and the failure is reported, as main's published-replacement exit does. The idle sweep's takeover arm does the same._MAX_NOTED_TRANSCRIPTS_PER_WATCH(64) declared read targets (slot_registry.py): the name comes from the resume request body, so the bound is enforced at admission —note_same_key_takeoverreturnsFalseand records nothing (no count, no claim mark) for an overflowing note, and the resume behind it is refused (503takeover_note_refusedat the endpoint,HistoryLockTimeouton the facade) before its read runs. Publications and already-held names are always admitted._transfer_basis_to_worker, the hand-offsave_slot_off_loopalready makes), so a shutdown cancelling a close mid-drain neither drops the acknowledged tail's writer nor retires the watch under the running thread; the rewind handler makes the same hand-off for a rewrite that did not settle within its cancellation drain.save_slot_off_loopis shielded, the worker closes its own basis when it settles, and a caller-opened basis transfers to the worker on cancellation (_transfer_basis_to_worker).adopt_variant_texttouches (redaction records included) and restores them when its durable write is refused — before any answer goes out, the app-token path's anti-enumeration 404 included, so a takeover that pops the slot mid-save cannot have the drain persist a variant the caller was told failed._transcripts_meet): a publication notesslot_history_key(slot), a resume the key it reads, and only a matching (or unknown) pair marks a claim overtaken or leaves one born so — a same-name publication resuming a different transcript no longer fails a linked tab's closing archive.takeover_ordering_pendingcounts a watch only through its TRUNCATING holders (dispatchers state the shape atopen_takeover_basis(truncating=); a worker raises an append that turned truncating in its dispatch gap viamark_basis_truncatingbefore its claim); the read-side wait is loop-side (SlotRegistry.await_takeover_ordering_settled: a future registered under the claim mutex, resolved through its loop when the last pending basis or claim retires), so a burst of resumes holds no executor thread the saves need.session_was_deleted, protective;session_deletion_confirmed, destructive) map one shared verdict,write_guards._session_delete_evidence→unobserved | deleted | present | unverifiable.slot_persistence/takeover_ordering.py(_slot_closed_before,admit_rehydrate_takeover_note,_truncation_claim_scope,open_dispatch_basis,_transfer_basis_to_worker,refuse_overtaken_rewrite), bound on thechat_persistencefacade likewrite_guards. The save transaction, its on-loop entry point and the rehydrate drivers now live where upstream'schat_persistencesplit put them —chat_persistence_parts/save_transaction.py(save_slot_off_loopwith its basis plumbing,_save_slot_to_historywith the claim scope and commit gate, the shutdown pass's watch) andchat_persistence_parts/slot_builders.py(therehydrate_slot_from_history_asyncwrapper,_rehydrate_slot_from_history_noted, and the loop-side_await_takeover_ordering_settled) — composed onto the facade as before, so everychat_persistence.<name>patch still reaches them. That split also tookchat_persistence.pyto ~850 lines, under its line-cap baseline row.record_deferred_dismissalreportswritten | superseded | failed:supersededis a publication that took the key and settled the dismissal while the write was queued (a plain same-name create between the hand-off and the write) — nothing was owed and the tab is legitimately live, so both close arms treat it as a committed close; onlyfailed(the write raised, or the line is unreadable) is reported ashistory_save_failed/ underfailed.SlotRegistry.begin_resume_read/end_resume_read) and the write-time guard answers False while one is in flight, so a marker never commits between a reopen's header snapshot and its publish — the deferred holder is deliberately non-truncating (readers are never held behind a dismissal), so such a reopen records no note, waits on nothing and would otherwise publish against an open header with the marker on its line. The resume's own exit then settles the dismissal (publish) or writes it (refusal, or a cancelled resume that was the last reader); the pre-close-resume-wins / delivery-rehydrate-loses contract is unchanged. "Published" is an occupancy test — the slot the resume answered with must now hold the key the wrapper noted — because_live_slot_for_resumededups by session identity across slots and can answer a resume of key A with a live slot under key B, which publishes nothing on A; that answer takes the owed-dismissal path like a refusal. The rehydrate wrapper needs no such test: its body only ever returnsstate._slots[slot_name], the key it was asked for. The registration is a registry transition andget_metadataa cached file read, while the marker's guard runs under the transcript lock, so the two can interleave (guard sees no reader → resume registers and snapshots an open header → rename); to close that window the resume takes its close-metadata snapshot UNDER the transcript lock whenever the key carries a watch (takeover_ordering.close_metadata_under_lock, off-loop, no await or registry mutex held inside): a marker that landed before the read is seen anddefer_clearclears it before publishing; one that has not landed cannot land afterwards, because the guard now finds the read. The publish side is ordered too: a plain same-name create takes no lock, so itsput_slotcan settle the dismissal between the guard's "still owed" and the rename; the record is the witness through the write, and after the rename commits — still under the transcript lock (update_metadata_if'safter_commit_under_lock) — the ownership is read again — the hook only records the verdict, honouringupdate_metadata_if's I/O-free contract — and right after the call returns a marker a publication has since won against is un-stamped as a compare-and-clear under the transcript's own lock (unstamp_superseded_marker:clear_closed(only_if_closed_id=our closed_id), keyed on the close identity — every close write stamps aclosed_idnonce besideclosed_at(metadata_line.close_identityfor the slot saves,DeferredDismissal.closed_idfor the owed marker; a slot-owned key, dropped with the stamp on reopen) and the clear compares it under the transcript lock, so exactly our stamp comes off: a legitimate close of the new tab stamped on the same clock tick carries an equalclosed_at(Windows ticks ~15.6 ms) and its own id and is kept, and a stamp carrying no id — an older build's line — is never ours to clear), the outcomesuperseded. A clear that fails is parked as an owed un-stamp on the key's watch (OwedUnstamp,park_owed_unstamp) — the dismissal itself is settled, so it cannot ride the owed-dismissal retry — and every noter exit on the key retries it (retry_owed_unstamp) until it lands or the line provably carries a different stamp; a settled dismissal is never reported as a failed close.record_deferred_dismissalsettles a skipped write only on evidence —closedon the line, or the session gone — never on the line merely reading back: a transient read failure insideupdate_metadata_ifanswers False without writing, and the open line it leaves isfailed(still owed, retried by the next exit), notwritten.DeferredDismissal.created_atis the closed transcript'screated_atas the slot last observed it (_disk_meta_created_at, the same witness the delete probes use), and the write guard — inside theupdate_metadata_iflock — requires the line'screated_atto match. A session permanently deleted and the same name-derived key created again while the dismissal is owed is a different conversation: the marker is dropped and the dismissal settled as satisfied (superseded), never retried against the new line. A closed slot that recorded no identity (a legacy line) owes a marker nobody can place: it is not written and the dismissal settles rather than retrying forever — stamping a line one cannot identify is the harm the witness exists to prevent. Both close arms pass the witness.save_taskis awaited only on the branch that dispatched it, so a key that changed hands before dispatch settlessaved = Falsedirectly and audits ashistory_save_refusedrather than tripping anUnboundLocalErrorintohistory_save_exception.Rebased onto current
main(8c7417ea8): the contract sits inside theslot_persistence/andchat_api/owner splits (the commit gate precedestranscript_merge.compose_payload; every new name an owner function reads is bound on thechat_handlersfacade), composes with #13381's guarded-write registry andis_closingfence (the fence read, the dispatch verify and the registration share one synchronous block), and the two composition-contract pins and the #13381 ordering pins were updated to the current shapes.slot_lifecycle.pycrossed the 2,000-line cap on the rebase (main alone is 1,651); the two repair/reset endpoint handlersapi_chat_slot_repair_imagesandapi_chat_slot_reset_conversationmove tochat_api/slot_repair.pyunchanged, per the responsibility-split rule (newslot_repairowner row and_BASE_SURFACErow intest_chat_handlers_composition_contract.py; the path pins intest_json_object_body_guard.pyandtest_slot_running_readers.pyand the owner list inlearn-cron-dashboard.mdfollow it).Backwards compatibility
Compatible: the claim only ever refuses a truncating save when a same-key takeover began during that save's own write — a schedule in which landing the write was the defect. Saves with no concurrent takeover decide
committedand behave exactly as before;put_slotkeeps its signature and return;takeover_basisis an optional keyword on_save_slot_to_history/save_slot_off_loop/flush_slot_now;entryis an optional keyword onappend/append_if_absentandinclude_rotated(default off) onappend_if_absent; owners without a claim table (partial test doubles) fall back to the pre-contract shape via the same getattr idiom as_slots_under_construction.Every existing call shape, return value and error code is kept; the diff adds three refusal paths (503
takeover_ordering_unsettledand 503takeover_note_refusedon resume, and 500history_save_failedon a refused close archive) and widens no validator. The one thing that leaves a module is the responsibility split:api_chat_slot_repair_imagesandapi_chat_slot_reset_conversationmove fromchat_api/slot_lifecycle.pytochat_api/slot_repair.pywith their bodies unchanged, and every reader still resolves because none of them imported the owner module directly -- they all read thechat_handlersfacade, which composes the new owner.Reader: src/kiro_crew/dashboard/routes/chat.py:api_chat_slot_reset_conversation -- chat -- route registration reads
chat.api_chat_slot_reset_conversation, bound bydashboard/chat.pyfrom the facade;test_chat_slot_reset_conversation.py,test_teardown_routes_sibling_alias_busy.pyReader: src/kiro_crew/dashboard/routes/chat.py:api_chat_slot_repair_images -- chat -- route registration reads
chat.api_chat_slot_repair_imagesfrom the same facade binding;test_chat_slot_repair_images.py(12 tests)Reader: src/kiro_crew/dashboard/chat.py:api_chat_slot_repair_images -- chat -- re-export from
chat_handlers, which now imports both names fromchat_api.slot_repair(lines 77-78 unchanged);test_dashboard_structure_lock.py::test_reset_conversation_keeps_the_lock_and_keeps_the_unlockReader: src/kiro_crew/dashboard/chat_handlers.py:api_chat_slot_reset_conversation -- chat -- the facade composes
_owner_slot_repairand imports the two names from it, so patches ofchat_handlers.<name>still reach the moved bodies;test_chat_handlers_composition_contract.py(slot_repairowner row and_BASE_SURFACErow), path pins intest_json_object_body_guard.pyandtest_slot_running_readers.pyReader: docs/system-specs/modules/learn-cron-dashboard.md:slot_repair -- chat -- owner list updated in this diff;
docs/request-for-change/rfc-transcript-section-markers.mdanddocs/feature-map/README.mdnameslot_repair.pyas the handlers' file (docs only, nothing executes them)Tests
test/test_rewrite_publication_ordering.py(28 tests;test_append_safe_fallback_never_lands_a_row_that_sorts_before_the_one_it_follows(a prebuilt entry whose owntspredates the rows on disk is stamped one microsecond past the last of them byhistory_append.append, the rule a clock-stamped append already follows, so a reader ordering bytsnever places the fallback's row ahead of the row it followed; fails on the previous head),test_a_resume_behind_a_saturated_executor_still_settlesproves the loop-side wait on a two-thread default executor with a probe job,test_an_append_shaped_save_does_not_make_the_ordering_pending,test_a_dispatch_that_turns_truncating_raises_its_basis_before_its_claim), staged with the sibling close-race suite's fixtures and its bounded-seam parking idiom (the seam is the metadata-line build, which runs inside the lock after the map recheck and before the commit gate):test_resume_beginning_mid_write_makes_the_truncating_save_refuse/test_publication_mid_write_makes_the_truncating_save_refuse— the acceptance race from the issue, via a resume and viaput_slot: the save refuses, disk keeps all rows, no dropped-line archive is written.test_truncating_save_commits_when_no_takeover_interleaves/test_takeover_after_commit_reads_the_rewritten_transcript— the two sequential histories.test_takeover_during_dispatch_makes_an_unguarded_truncating_save_refuse,test_a_save_that_becomes_truncating_after_dispatch_still_loses_to_a_takeover(an append-shaped dispatch whose worker adopts_pending_rewritein the gap, with a takeover landing in the same gap: the save refuses, the transcript is untouched),test_a_stale_basis_leaves_the_claim_born_overtaken,test_takeovers_of_unwatched_keys_leave_no_record,test_flush_pass_skips_a_slot_whose_key_changed_owner_mid_pass,test_shutdown_saves_the_occupant_that_won_a_refused_key— watch semantics and the dispatch-time vouch.test_cancelled_dispatch_keeps_the_watch_open_for_its_running_worker,test_cancelled_caller_leaves_its_watch_open_for_the_running_worker— watch ownership across cancellation.test_refused_handover_save_lands_the_tail_ahead_of_the_takeover_read,test_drain_reports_a_mid_save_takeover_refusal_as_the_append_it_becomes,test_append_safe_fallback_persists_the_complete_row,test_commit_gate_fallback_lands_private_rows_under_a_restricted_line,test_a_refused_claim_keeps_the_window_owed_through_the_flush,test_commit_gate_fallback_does_not_duplicate_rows_rotated_into_archives(a realflush_slot_nowwhose save loses its claim and whose fallback cannot confirm the header: nothing appended,_dirtysurvives the pass) — the append-safe arm, theentrypassthrough, and the header ratchet at the commit gate (an incognito slot's refused hand-over save over a persistent replacement line: mode tightened, store cleared, carried execution folded, the replacement's title kept, the derivation seam withholds the file).test_resume_waits_for_a_save_still_in_dispatch_under_an_open_watch,test_resume_refuses_rather_than_reading_unsettled_history_on_timeout— the read-side wait and its fail-closed expiry.test_archive_dedup_resolves_the_physical_stem_of_a_legacy_transcript(a bare<ts>.jsonlwith<ts>__*archives, addressed asslack:<ts>: an archived identity is not appended again),test_takeover_note_marks_every_undecided_claim_and_only_those,test_a_watch_admits_a_bounded_number_of_declared_transcripts(the 65th distinct name is refused with nothing recorded; a held name and a publication stay admitted; the refused resume raises and publishes nothing),test_claim_helpers_tolerate_owners_without_a_claim_table.test/test_slot_close_recreation_race_takeover_ordering.py(new, 21 tests: the arbitration, the drain's arms, the base reproduction) andtest/test_slot_close_recreation_race_takeover_retraction.py(new, 27 tests: the tombstone-gated note,TakeoverNoteretraction and basis-relative accounting, the owed dismissal, the clock-frozen pre/post-close variants) — both split out oftest_slot_close_recreation_race.pyto respect the 2,000-line cap the way upstream's_handover_metadata/_privacy_ratchetparts were, sharingslot_close_recreation_race_helpers.py; the 4 pre-existing close-race tests whose save doubles now return a value stay intest_slot_close_recreation_race.py:test_a_resume_that_waited_out_a_committing_close_does_not_publish_over_its_marker(a reopen parked in its ordering wait while the close's archive commitsclosed=True: the reopen decisions come from the post-wait metadata read, so the resume refuses or clears the marker, never publishing over it),test_close_archive_yields_to_a_user_resume_begun_before_the_close(end-to-end through the real close handler and the real resume endpoint: the resume is observed in its ordering wait before the close's cancel completes, the close hands over with 200, the resume publishes the untruncated transcript),test_close_archive_commits_over_a_delivery_that_begins_mid_teardown(a rehydrate begun after the pop notes nothing: the archive commitsclosed, the session is removed, the rehydrate is refused, disk holds the rewound 2 rows),test_a_rehydrate_begun_after_the_close_does_not_make_it_yield,test_a_noter_refused_after_noting_does_not_make_the_close_yield(the regression: a noter refused after its note, the close takes its retry arm and restores the original instead of reporting a success it never committed),test_a_retracted_note_no_longer_moves_the_basis,test_a_retraction_before_the_basis_opened_does_not_hide_a_later_takeover/test_a_pre_basis_note_retracted_after_the_basis_does_not_hide_a_later_takeover(per-note accounting: a refused noter's retraction, before or after the basis opened, leaves a later real takeover visible to bothbasis_movedand the claim registration),test_a_stale_retraction_across_watch_recreation_does_not_withdraw_a_newer_note(the handle is bound to the issuing watch: a noter refused after its watch retired does not withdraw the fresh watch's first note),test_a_divergent_publication_does_not_overtake_a_linked_tabs_rewrite_archive(a same-name publication resuming another transcript leaves a linked tab's closing rewrite archive to commitclosed),test_a_reopen_racing_an_edit_never_hydrates_the_truncation_with_the_removed_tail_behind_it(the base reproduction, see Goal),test_a_taker_that_gives_up_after_the_close_returned_still_leaves_the_tab_dismissed(anadopt_closedrehydrate cancelled after the hand-over close returned:closedlands, no slot, watch retired),test_a_taker_refused_after_registering_its_slot_still_leaves_the_tab_dismissed[refused|cancelled](the taker's slot is registered and then its window load fails: the publication was not final, the retraction writes the marker),test_a_publication_that_noted_nothing_settles_the_dismissal_at_once,test_cleanup_keeps_its_dismissal_owed_when_the_taker_gives_up_later(the bulk cleanup arm),test_an_owed_marker_is_not_written_over_a_reopen_that_published_meanwhile(a reopen landing while the owed write is queued keeps its tab),test_an_owed_marker_whose_write_fails_stays_owed_and_lands_on_the_next_exit,test_a_taker_that_gives_up_during_the_drain_leaves_the_close_to_write_its_own_marker[single|cleanup](the immediate path: the taker retracts during the drain, the arm finds nobody standing and writes the registered record itself),test_a_close_whose_own_marker_write_fails_does_not_report_success[single|cleanup](the immediate write fails →500 history_save_failed/failed, dismissal stays owed, a later taker's exit writes it),test_a_plain_create_between_the_hand_off_and_the_queued_write_is_a_committed_close[single|cleanup](a publication lands while the close's own write is queued →superseded→ 200, no marker, the new tab live),test_an_owed_marker_yields_to_a_user_resume_that_read_the_header_before_it(the chain: close hands over → theadopt_closedtaker backs out while a user resume is between its header snapshot and its publish → no marker lands, the resume publishes, the live tab's line stays open),test_an_owed_marker_lands_when_the_resume_it_yielded_to_refuses,test_two_resumes_in_flight_and_a_cancelled_one_still_settle_the_owed_marker,test_a_resume_admitted_between_the_markers_guard_and_its_rename_reopens_the_tab(the owed write parked after its guard, before its rename; a resume admitted in that window reads the marker under the lock, clears it and publishes open),test_a_skipped_marker_write_whose_line_reads_open_stays_owed(a transient read failure inside the guarded update →failed, still owed, written by the next exit).test/test_slot_close_recreation_race_takeover_dismissal.py(new, 15 tests; the owed marker's incarnation witness, whose publication settles it, the un-stamp a late publication owes -- keyed on the close identity:test_an_unstamp_keeps_a_newer_close_stamped_on_the_same_clock_tickfreezeschannel_slots.time, lands the owed marker, closes a reopened tab on the same tick (equalclosed_at, its ownclosed_id) and asserts the un-stamp leaves that close in place; fails with the instant-keyed clear restored -- and the wrappers' post-exit ownership recheck:test_a_rehydrate_whose_slot_is_deleted_during_its_exit_work_returns_noneandtest_a_resume_whose_slot_is_deleted_during_its_exit_work_is_refusedpark the exit work's last worker hop (retry_owed_unstamp) with the slot published, retire it as a permanent deletion does, and assert the wrapper answersNone/resume_session_deletedrather than the retired slot; both fail with the recheck removed):test_an_owed_marker_is_not_written_onto_a_new_incarnation_of_the_key(close defers → transcript deleted and the key recreated → the pending reopen's retraction drops the old dismissal; the new conversation carries noclosed),test_an_owed_marker_lands_on_the_same_incarnation(the happy path still writes),test_an_owed_marker_without_an_identity_is_never_written,test_a_retried_owed_marker_also_checks_the_incarnation,test_the_dismissal_record_carries_the_closed_incarnation,test_a_resume_answered_by_another_keys_live_slot_does_not_settle_this_keys_dismissal(resume A by B's session key with B live → dedup to B → A's dismissal stays owed and lands when A's taker gives up; B untouched),test_a_resume_that_dedups_to_this_keys_own_live_slot_counts_as_published,test_a_plain_create_between_the_markers_guard_and_its_rename_is_unstamped(the write parked after its guard, before its rename; a same-name create lands; release → the marker is cleared again under the lock,superseded, the new tab's line open),test_an_owed_marker_with_no_publication_in_the_window_is_written,test_an_unstamp_that_fails_once_stays_owed_and_the_next_exit_clears_it(every clear of our stamp fails while the disk is "broken" →supersededstill, un-stamp owed, the next exit clears it once the disk recovers),test_an_unstamp_never_removes_a_newer_close_of_the_created_tab(compare-and-clear; idempotent),test_the_after_commit_hook_does_no_io(the hook's only call isdismissal_still_owed, pinned by AST);test_a_handed_over_dismissal_settles_on_publish_or_on_the_last_retraction,test_both_delete_witnesses_map_the_one_evidence_verdict,test_close_reports_failure_for_a_retryable_archive_refusal,test_deletion_is_only_confirmed_on_definitive_evidence,test_teardown_drain_basis_follows_the_transcript_not_the_key,test_handover_drain_saves_the_tail_a_lost_truncation_rode_with,test_divergent_takeover_does_not_relabel_an_unrelated_archive_refusal,test_a_delete_won_refusal_beside_a_takeover_does_not_resurrect_the_transcript,test_handover_append_arm_does_not_resurrect_a_deleted_transcript,test_handover_append_creates_the_header_under_the_restricted_mode(incognito original, no committed header, pending rewrite, takeover: the file is born withmemory_mode,transcript_withholds_derivationis true andderive_messagesraisesTranscriptWithheld),test_handover_append_tightens_a_persistent_replacement_line,test_handover_append_drops_notes_authorized_for_another_session(a note stamped for the slot's former session rides the window into a hand-over onto its channel transcript: not written,note_save_dropaudited),test_a_declared_transcript_matches_every_spelling_of_its_file(slack_123.456declared,slack:123.456written: the watch is matched and the drain keeps its basis),test_a_failed_handover_to_an_unpublished_takeover_restores_the_original(end to end through the close handler with every append-safe write raising ENOSPC: 500, the original back under its key and dirty, the waiting resume returns it),test_cleanup_restores_the_original_when_its_handover_to_a_takeover_fails,test_handover_append_does_not_duplicate_rows_rotated_into_archives(window past the rotation cut: the replay lands only the owed tail, identities stay unique, existing indices do not move),test_handover_append_refuses_when_the_header_cannot_be_confirmed(the tightening merge defers once on an incognito hand-over with no header: no row written, the hand-over reports failure, the tail stays owed),test_a_cancelled_handover_keeps_its_append_and_its_watch(the drain cancelled while its append is parked: the basis is transferred, the watch survives the caller's close and retires at the append's settlement, the rows land).Every parked test double in the ordering suite raises on an unreleased park, and the two tests that await a resume while a save or a close is parked observe the resume inside its ordering wait before releasing, so no test passes by waiting out a timeout.
test/test_dashboard_chat_rewind.py(+1):test_rewind_dispatch_verify_refusal_audits_as_a_refused_save— mints a same-name replacement during the sid flush (the last await before the verify) and asserts the SEL record readshistory_save_refusedwith no worker dispatched. Fails on the pre-fix shape withhistory_save_exception.test/test_chat_regenerate_cov80.py(+2):test_switch_variant_restores_the_row_when_the_save_is_refused,test_switch_variant_restores_the_row_before_an_app_is_answered_not_found.Updated pins:
test_chat_persistence_composition_contract.py(_SIGNATURESfor the new keyword; thetakeover_orderingowner),test_chat_persistence_parts_composition_contract.py(the two names added toslot_builders; the shutdown-save double acceptstakeover_basis),test_history_bindings.py(signature digests forappend/append_if_absentand the new_holds_identical_rowmember ofhistory_append),test_chat_regenerate_cov80_edit_resend_boundary.py(the replaced-slot edit-resend restaging followed its test out oftest_chat_regenerate_cov80.py),test_chat_handlers_composition_contract.py(thehandover_drainowner surface:_persist_handover_tailgainstakeover_basis,_teardown_drain_basisis added; andtest_conductor_wake.py(save/drain doubles return a value and accept the basis),test_truncating_save_recreate_order.py(zero awaits between the fence read and the registration; the periodic save carriestakeover_basis=Nonewhen called directly), plus thesave_slot_off_loopdoubles intest_slot_close_nudge_race.py,test_dashboard_chat_session_authority.pyandtest_resume_publishes_hydrated_slot.py.Manual verification
N/A — unit coverage sufficient: the race is staged deterministically in unit tests with real
ConversationLogfiles and there is no UI surface. Full local run on Linux/py3.12 of the 483 test files importing the touched modules, against a cleanmainworktree in the same environment: the failure-id sets are identical (main's local failures are allModuleNotFoundError: snowballstemmer/websockets), with 26 more tests passing on this branch.black,check_comment_history,docs-lint,isort,flake8andmypy src/kiro_crew/pass locally.Related Issues
Fixes #12090
Pattern harvest
Rule candidate: review-prompt
Pattern: a commit-boundary identity re-check guards a read-modify-write against actors that never take the writer's lock — any such check has an uncoverable check-to-commit gap and needs a two-sided arbitration token, not a later or duplicated check.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.