Repository navigation
fix(dashboard): settle undecidable strReplace before-snapshots at flush - #17367
Conversation
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Any turn that runs a shell tool skips the fix ( Not justified as shipped
What this change shipsInventory (5 items) — 4 justifiedIntent: a user wants the file-change card's line stats to show the real full-file before after a one-line
[FIRST-PRINCIPLES-REVIEWED] f45e62b Description read[DESCRIPTION-READ] 2132e04d6f4111514bd75f80948c49b60eea5c839ac76e251fc7a88be76671bb That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the deferred strReplace snapshot is taken before the write runs for kiro-cli's real tool_call/tool_call_update ordering; otherwise the fix rarely fires. [DESIGN-REVIEWED] f45e62b Description read[DESCRIPTION-READ] 4c7bf6a9b5a0498558d314670af8218d32f992187a09a926c24cb23cb7ace9d1 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
b0d8d2c to
efc3c4d
Compare
efc3c4d to
1ea2dd2
Compare
1ea2dd2 to
3a605bc
Compare
3a605bc to
e9ce896
Compare
e9ce896 to
5e98c66
Compare
A one-line strReplace could make the dashboard file-change card report the removed line against the entire post-edit file (`-1 +55` instead of `-1 +0`, issue #11387). The ACP diff block's `oldText` for strReplace is only the replaced fragment, so an edit whose on-disk state is valid as BOTH the pre- and post-write state could not be classified at snapshot time and fell back to the fragment, diffing one line against the whole file. Resolve the ambiguity at turn end instead of guessing. `_classify_str_replace_before` (renamed from `_reconstruct_str_replace_before`) returns the undecidable disk content alongside the proven before; the snapshot carries two `_MAX_SNAPSHOT`-capped hypotheses as `pending_str_replace` (`if_post_write`, `if_pre_write`), and `_flush_file_changes` settles them against the turn-end after-content via `_resolve_pending_str_replace`. Only the PRE-write hypothesis is settled: the turn-end file is compared to the forward substitution, and a match yields `if_post_write` — the content actually read from disk at snapshot time. The POST-write (append-style) direction is not settled, because proving it would require showing a reverse substitution as the before, a value never on disk; a shell command or a concurrent write from another session can restore the file to exactly the snapshot content and turn that branch into a fabricated before-diff. Keeping only the pre-write branch removes that fabrication hazard class outright. Because a resolved before is always captured disk content, no completion gate and no writer-count check are needed: a refused/failed/cancelled write leaves the file equal to `if_post_write` (the classifier only defers edits whose two hypotheses differ), so the pre-write comparison fails and the fragment stands on its own, and a second or concurrent writer only changes the turn-end disk, which cannot corrupt a captured before. `_record_turn_snapshot` and the flush both dedup by `canonical_path`, so two spellings of one file in a turn (`/work/f`, `/work/./f`) stay one entry and the first before-snapshot survives a revisit under either spelling. Ported onto the current `chat_turn/file_changes.py` layout (the file-change machinery moved out of `chat_runner.py` since the original PR). One unrelated one-line fix is declared, not a stale-base artifact: `test/test_source_pin_budget.py` fails closed on clean origin/main because the `test_session_control_owner_dm.py` source-pin budget sits at 3 while the module reads it once; lowered to 1 so this PR's backend shards can go green. The breakage is pre-existing and independent of the card fix. Tested: TestPendingStrReplaceResolution (resolve; unchanged/changed-again keep the fragment; the cross-slot/shell restore fail-safe; the append shape keeps the fragment; pending never reaches message meta), TestPendingStrReplaceThroughTheTurnLoop (snapshot + flush end to end through the real `_run_chat`), and a canonical-dedup case in TestTurnSnapshotAccumulator. Existing snapshot, chat_runner_coverage and composition-contract suites updated for the rename; 689 tests pass locally, flake8 / isort / mypy / black and the Fast Gate scripts clean. Not run on a real dashboard client — the turn-loop tests replay the dispatcher event order; no renderer change. Fixes #11387 Co-authored-by: Wediong Wang <wwd@amazon.com>
|
@buluoray Implemented option (a), the one you asked for, at head 3164a31 (force-pushed over 6137977). Blocking 1 — the writer-id gate is in, and it removes the wrong-before outcome.
Non-blocking follow-ups you listed: the self-contradicting spec and the Not touched (pre-existing, as you noted): WakaTime Local gate: 692 tests green, isort/flake8/mypy/docs-lint clean, local_review parity exit 0. Not resolving your thread — leaving it for your re-read. |
… write Review fix for #11387 (PR #17367). The deferred strReplace "before" could be wrong when the turn-end file is shaped by something other than the one tracked edit. Two vectors, each closed upstream of the resolver: - A second write TOOL repeating the identical forward substitution: _record_turn_snapshot stamps each snapshot with _tcid_identity_key(tool_call_id) and drops the deferred pending_str_replace when a second or unknown write tool touches the same canonical path (writer-id gate; fail closed on unknown id). The two _run_chat call sites pass the redacted event.tool_call_id. - A shell command (or any non-write tool) editing the file after a post-write snapshot (GPT 6.1's finding): a shell is opaque, can write any path, and never passes through _record_turn_snapshot, so it cannot be gated per path. _flush_file_changes now skips ALL pending resolution for a turn in which a shell tool ran (turn_had_shell, wired from the existing _shell_tool_calls set), keeping the fragment rather than trusting an unattributable turn-end read. Docstrings (_flush_file_changes, _resolve_pending_str_replace) and the learn-cron spec describe both guards; the earlier false claims about a _bind_pending_str_replace helper and a tool_call_id payload field are gone. Tests: test_second_identical_edit_by_another_writer_keeps_the_fragment, test_second_identical_edit_by_the_same_writer_still_resolves, test_shell_activity_in_the_turn_keeps_the_fragment (with a no-shell control), and the base-failing alias test test_alias_spelling_in_the_same_turn_stays_one_resolvable_entry. Composition-contract signature pin for _record_turn_snapshot updated.
3164a31 to
19c30d7
Compare
|
Addressing the GPT 6.1 BLOCKING finding (file_changes.py:264, shell write after a post-write snapshot) at head 19c30d7 — the finding is correct and is now FIXED in the diff, not overridden. The writer-id gate only sees write-TOOL snapshots (via Docstrings and the learn-cron spec were corrected — they no longer claim the write-tool gate covers the shell case; the shell case is attributed to Not overriding this finding — it is fixed in the code. |
|
@iamwhatever Follow-up at head 19c30d7: the GPT 6.1 lane surfaced a real gap in the gate — a shell command editing the file after a post-write snapshot is never seen by the write-tool gate (a shell doesn't pass through |
|
@buluoray Follow-up at head 19c30d7: the GPT 6.1 lane surfaced a real gap in the gate — a shell command editing the file after a post-write snapshot is never seen by the write-tool gate (a shell doesn't pass through |
|
Update on F1 (cross-slot, file_changes.py:271) at head 19c30d7. The shell vector is fixed (turn_had_shell). The residual you flagged — the writer-id gate is slot-local, so a byte-identical strReplace from ANOTHER concurrent session in the same turn isn't cleared — is being accepted as a known limitation rather than widening this PR, matching your own adjudication (downgraded to FLAG, likely edge case): it affects the saved chip's metadata only (the file on disk is correct, no fabrication), needs two concurrent sessions making the identical edit on a narrow deferred shape, and is not a regression (pre-PR, any concurrent cross-slot writer already mixed into another slot's turn-end diff). Tracked as a follow-up: #17619, and noted as a Known limitation in the PR body. A proper fix needs a cross-slot/same-path intervening-write signal, out of scope for #11387. |
|
On F1 (cross-slot, file_changes.py:271) at head 19c30d7 — lead decision: this stays out of scope for this PR and will not be widened to close it here. Rationale, consistent with the adjudication's own downgrade to FLAG:
The in-scope wrong-before vectors are fixed in this PR (the write-tool-repeat via the writer-id gate, and the shell vector via turn_had_shell). F1 is tracked as a follow-up in #17619. @iamwhatever @buluoray — since F1 is a fenced finding, clearing it needs a human /ai-review override. Could you decide on it during your re-review? If you agree it is an acceptable, out-of-scope residual, a maintainer override would unblock the lane; otherwise I'm happy to take direction on widening the PR. |
… write Review fix for #11387 (PR #17367). The deferred strReplace "before" could be wrong when the turn-end file is shaped by something other than the one tracked edit. Two vectors, each closed upstream of the resolver: - A second write TOOL repeating the identical forward substitution: _record_turn_snapshot stamps each snapshot with _tcid_identity_key(tool_call_id) and drops the deferred pending_str_replace when a second or unknown write tool touches the same canonical path (writer-id gate; fail closed on unknown id). The two _run_chat call sites pass the redacted event.tool_call_id. - A shell command (or any non-write tool) editing the file after a post-write snapshot (GPT 6.1's finding): a shell is opaque, can write any path, and never passes through _record_turn_snapshot, so it cannot be gated per path. _flush_file_changes now skips ALL pending resolution for a turn in which a shell tool ran (turn_had_shell, wired from the existing _shell_tool_calls set), keeping the fragment rather than trusting an unattributable turn-end read. Docstrings (_flush_file_changes, _resolve_pending_str_replace) and the learn-cron spec describe both guards; the earlier false claims about a _bind_pending_str_replace helper and a tool_call_id payload field are gone. Tests: test_second_identical_edit_by_another_writer_keeps_the_fragment, test_second_identical_edit_by_the_same_writer_still_resolves, test_shell_activity_in_the_turn_keeps_the_fragment (with a no-shell control), and the base-failing alias test test_alias_spelling_in_the_same_turn_stays_one_resolvable_entry. Composition-contract signature pin for _record_turn_snapshot updated.
19c30d7 to
f45e62b
Compare
|
Re-reviewed the new head (crew-pr-reviewer). My earlier point is addressed. The settle is now gated on a single known writer per path ( |
My points are addressed on the new head (see my latest comment).
buluoray
left a comment
There was a problem hiding this comment.
Re-review at f45e62bf90: approving.
What it brings: a one-line strReplace now shows its real diff against the full-file before, instead of counting the whole file as added; edits that cannot be attributed to a single write fall back to the fragment view rather than showing a wrong before.
Previous blocker: fixed
A second identical substitution in the same turn no longer makes the card show the wrong before. Two gates cover it:
- Writer-id gate in
_record_turn_snapshot(chat_turn/file_changes.py:539-542): a later write to the same canonical path dropspending_str_replacewhen it carries a different tool-call id, or when either id is unknown, so the flush keeps the fragment. The repeatedtool_call/tool_call_updatepair of one write shares an id and still resolves. - Shell guard in
_flush_file_changes(chat_runner.py:2034):turn_had_shellskips all pending resolution for a turn in which a shell tool ran, since shell edits are never recorded per path.
Turning the drop into a no-op reddens test_second_identical_edit_by_another_writer_keeps_the_fragment; removing and not turn_had_shell reddens test_shell_activity_in_the_turn_keeps_the_fragment. Docstrings and the spec now describe the code as it is.
Non-blocking follow-ups
- The call-site wiring is untested: passing
""aswriter_keyat both_record_turn_snapshotcall sites, or hard-codingturn_had_shell=Falseat both_flush_file_changescall sites, leaves all tests green. A turn-loop test driving one strReplace throughtool_call+tool_call_update, plus one with a shell call, would pin both. - The unknown-writer clause (
not writer_key or ...) is untested. _shell_tool_callsonly records shell calls that carry an id, so an id-less shell call does not setturn_had_shell. Narrow, but fail-open.- Carried over: the
after != if_post_writeclause is untested, and WakaTime's_turn_line_changesstill counts deferred snapshots from the fragment.
Upgrade impact / audience fit: none. writer_key lives only on the turn-local accumulator; persisted file_changes entries keep their keys, and every unattributable case behaves as on main.
CI on this head: 85 success, 13 skipped, 0 failures.
Problem / Motivation
Goal: show the true full-file before in the dashboard file-change card when a
one-line
strReplaceleaves the on-disk state ambiguous at snapshot time.After a one-line
strReplace(net: one line deleted), the dashboard file-changecard reported
-1 +N— one line removed and the entire post-edit filecounted as added — instead of
-1 +0(issue #11387).kiro-cli's ACP diff content block carries only the replaced fragment as
oldTextforstrReplace, not the whole file. When astrReplace's on-diskstate is valid as both the pre- and post-write state (a
newStr ⊂ oldStrdrop or
oldStr ⊂ newStrappend next to a kept line), the classifier could notdecide at snapshot time and fell back to the fragment — diffing one line
against the full-file after and counting the whole file as additions.
Why it matters
The card's stat badge is the first summary a user sees for an edit.
+55on adotfile reads as "the agent appended a big block to my config" even when the
underlying change was a one-line deletion. Misreported stats undermine trust in
the card even when the diff view itself is right.
Not a goal
The non-undecidable classifier branches keep the fragment
oldTextby design.The post-write (append-style) direction is not settled at all — resolving it
safely would need a reverse substitution against the turn-end disk, which a shell
or cross-slot restore can corrupt — so it keeps the fragment rather than being
guarded with a cross-slot write registry.
What changed (motivation → approach → change)
Resolve the ambiguity at turn end instead of guessing:
_classify_str_replace_before(was_reconstruct_str_replace_before) nowreturns
(before, undecidable_content). When the disk state is valid as both,the snapshot carries three
_MAX_SNAPSHOT-capped hypotheses aspending_str_replace(if_post_write,if_pre_write) — never the raw read,and no reverse substitution is stored.
_flush_file_changessettles the pending snapshot via_resolve_pending_str_replaceagainst the turn-end after-content: a match onthe forward substitution proves the snapshot was pre-write and yields the true
full-file before (
if_post_write, the captured disk content)._record_turn_snapshotdedups bycanonical_path(the same bucket key theflush uses), so two spellings of one file (
/work/f,/work/./f) stay oneentry and a revisit under either spelling cannot move the original
before-snapshot behind an alias.
Fail-safe resolution (pre-write only) — only the PRE-write hypothesis is
settled, so a resolved before is always content actually read from disk at
snapshot time (
if_post_write), never a reverse substitution against theturn-end disk. Proving the POST-write (append-style) direction would require
showing a reverse substitution as the before — a value never on disk — and the
turn-end read is not this slot's to trust: a shell command or a concurrent write
from another session can restore the file to exactly the snapshot content and
turn that branch into a fabricated before-diff. Dropping that branch removes the
fabrication hazard class outright (cross-slot and shell restores included); the
append direction keeps the fragment fallback.
Implementation lives in
src/kiro_crew/dashboard/chat_turn/file_changes.pywiththe turn-loop wiring in
chat_runner.py; the owning system spec is updated tomatch.
Backwards compatibility
Compatible: persisted
file_changesentries keep their{path, before, after}shape and truncation flags. The pending hypotheses and the canonical path are
turn-local and never reach message meta.
Tests
test/test_file_change_snapshots.py::TestPendingStrReplaceResolution— resolvethe undecidable snapshot; an unchanged file (refused write) and a changed-again
file keep the fragment; a restore to the snapshot content never fabricates a
before (the cross-slot/shell fail-safe); the
oldStr ⊂ newStrappend shapekeeps the fragment; pending never reaches message meta.
test/test_file_change_snapshots.py::TestPendingStrReplaceThroughTheTurnLoop—the snapshot + flush run end to end through the real
_run_chat, and the chipsurfaces without the internal payload.
TestTurnSnapshotAccumulatorgains a two-spellings-of-one-path case pinning thecanonical-path dedup.
test_chat_runner_coverage, and composition-contract suitesupdated for the rename and the new helpers. 689 tests pass locally;
flake8 / isort / mypy / black and the Fast Gate scripts clean.
Manual verification
N/A — the turn-loop tests replay the dispatcher event order and disk transitions
that produce the card metadata; no frontend renderer change.
Screenshots / video
N/A — this backend correction changes the accuracy of existing card data without
changing the rendered component or layout.
Related Issues
Fixes #11387
Builds on the approach in #11394 by @DcXiaoBing, ported onto the current
chat_turn/file_changes.pylayout and hardened to resolve only the pre-writehypothesis, so no shell or cross-slot restore can fabricate a before.
Credited via
Co-authored-by.Pattern harvest
Rule candidate: review-prompt
Pattern: before/after inference that defers a verdict to turn end must treat
every writer it cannot observe — a second write-tool call, an unidentified or
alias-spelled path, a shell command — as a reason to decline, never to resolve;
a deferred "before" is only safe when exactly one observed writer owns the path.
Known limitation
The writer-id gate is slot-local, so a byte-identical strReplace from another concurrent session in the same turn is not cleared and the saved file-change chip can show an intermediate before (chip metadata only; the file on disk is correct, not a regression). Tracked in #17619.