Skip to content

fix(restore): refuse --mode replace over a bundle's recorded omissions - #17636

Closed
buluoray wants to merge 2 commits into
mainfrom
fix/restore-replace-omissions-11499
Closed

buluoray wants to merge 2 commits into
mainfrom
fix/restore-replace-omissions-11499

Conversation

@buluoray

@buluoray buluoray commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: kirocrew restore --mode replace no longer removes live files that a snapshot recorded as omitted, unless the operator explicitly accepts that.

A snapshot that could not read an entry leaves it out and names it in MANIFEST.json skipped[] (for example unreadable_entry). Replace clears each component's live files before installing the bundle's copy. When the entry is readable again at restore time, the live file is removed. Its only copy ends up in pre-restore-<ts>/, and the run reports ✅ Replace complete. with exit 0. The ⚠️ Omitted lines in the manifest printout were the only reader of skipped[], and nothing acted on them (reproduced on main in the issue thread).

Why it matters

Snapshot tolerates omissions so that one protected file cannot block a backup. Without a matching check at restore time, every omission the snapshot tolerated becomes a silent removal from the live data home on replace. The operator's only protection is a manifest field that no decision reads.

Not a goal

  • Making replace "restore around" omitted paths. With the flag, replace keeps its clear-then-install meaning.
  • Changing merge, which clears nothing.
  • The dashboard import path (portability.apply_import_zip). It consumes the export zip format, whose manifest has no skipped field, so it is not a reader of this declaration.
  • The misleading "failed partway through" message when the entry is still unreadable at restore time. That is a separate defect noted in the issue thread.

What changed (motivation → approach → change)

  • Symptom → cause: replace never consulted skipped[], so a recorded omission was indistinguishable from a path the operator wanted gone.
  • Approach: the issue's option 2. Replace is refused by default, the omitted paths are printed, and --allow-omissions opts in.
    • Refusing with no override (option 1) strands a legitimate restore of a 99%-intact bundle.
    • Restoring around the omitted paths (option 3) redefines --mode replace.
  • Change:
    • snapshot_restore._replace_omissions(snap, components, mc) (re-exported by the snapshot facade, with its helper _live_path_absent) reads skipped[] and classes each reason through pinned_fs.omits_wanted_data, the predicate the snapshot prune guard already uses. symlink and not_regular are screened by design and do not count. Every other reason counts, including an unknown one.
    • It keeps only omissions under a component being restored, matched against COMPONENTS files and trees, with Windows separators normalised. An omitted directory that is an ancestor of a restored tree counts (a full backup records an unreadable workspace under that one path, and memory's subtrees live inside it).
    • It drops an omission when nothing exists at that path in the live data home, so a fresh-machine recovery restore is not refused. A path that cannot be placed under the data home (empty, absolute, drive-qualified, or containing ..) is never treated as absent.
    • A path no component claims, or a malformed skipped, counts as an omission (fails closed).
    • restore_main checks before --dry-run, so a preview gives the same answer as the real run. It prints the paths and the two ways forward, audits state_restore_rejected reason=bundle_has_omissions, and exits 1. With --allow-omissions it proceeds and prints how many omitted paths it may remove.
    • --allow-omissions is added to both the kirocrew restore parser in cli.py and the standalone parser.
    • docs/system-specs/modules/cli.md, the shipped user guide src/kiro_crew/docs/snapshot-and-restore.md, and the kirocrew-commands built-in skill document the rule and the flag.

Backwards compatibility

Breaking: kirocrew restore --mode replace (including the auto-selected replace on a home without memory.db) now exits 1 for a bundle whose skipped[] covers a restored component and names a path that exists in the live data home, where it used to succeed. Writer sweep on origin/main @ 0d885e6: the only callers of restore_main are cli.py (kirocrew restore) and tests. portability.py calls _do_replace directly on its own zip format and is unaffected. The escape is --allow-omissions or --mode merge.

Reader: src/kiro_crew/snapshot_archive.py:_print_manifest -- entry: release (CLI) -- still prints every skipped[] entry unchanged; this PR adds a second reader and removes none.

Tests

test/test_restore_recorded_omissions.py (27 cases; the extra one is a POSIX filename containing a literal backslash). test_snapshot_refactor_contracts.py pins the new --allow-omissions parser row, and test_snapshot_refactor_ownership.py declares the two new names under snapshot_restore reproduces the real chain. It runs the snapshot with the file's stat/lstat/open denied, lifts the denial, then restores:

  • replace is refused, the live file stays, and no pre-restore-* directory is created;
  • --dry-run gives the same refusal;
  • --allow-omissions replaces, the file is only in the rollback directory, and the warning is printed;
  • merge is unaffected;
  • a parametrised matrix over every reason class, Windows separators, an out-of-selection path, an unclaimed path, and malformed skipped values;
  • an omitted ancestor directory (workspace) refuses --components memory replace, while siblings workspace/notes.md and workspace/memoryx do not;
  • nothing live at the omitted path: replace succeeds without the flag. Unplaceable paths (../outside.md, /etc/x, C:/x) are still refused.

Mutations, run against the first head and this one:

  • stubbing _replace_omissions to return [] reddened 11 of the original 17 cases (the other 6 assert no refusal);
  • dropping the ancestor clause reddens the 2 ancestor cases;
  • making _live_path_absent always False reddens the fresh-home case;
  • removing its placement guard reddens the 3 unplaceable-path cases.

Run as PYTHONPATH=<worktree>/src python -m pytest -n0 test/test_restore_recorded_omissions.py. The shared venv imports the main checkout otherwise, which was checked.

Manual verification

N/A for Linux. Unit coverage drives restore_main end to end. Unverified on Windows and macOS: the path normalisation is covered by the matrix, but no restore was run on those hosts. I did not run kirocrew restore --help to see the flag rendered. The wiring was checked by reading cli.py (restore_main(parsed=args), dest="allow_omissions").

Related Issues

Closes #11499. Related: #11418 (introduced the snapshot tolerance), #11700.

Pattern harvest

Rule candidate: review-prompt
Pattern: "a manifest field written to declare a gap must have a consumer that acts on it; a tolerance on the writer needs a matching guard on every destructive reader"

Decision for the owner

The decision is what --mode replace guarantees when the bundle's manifest records entries the snapshot could not carry. This draft refuses by default and adds an explicit --allow-omissions opt-in, because that fails closed without stranding a legitimate restore, whereas restoring around the omitted paths would silently change what replace means.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality (new tests pass locally; the existing suite is left to CI)
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that the workspace ancestor rule refuses only bundles whose skipped[] actually names workspace, so ordinary --components memory replaces still run.

[DESIGN-REVIEWED] dacc6db

Description read

[DESCRIPTION-READ] f94b61e5d7a0c3b92a6352857a1d27c03b8619f1d33aac30bdf13235fc36d225

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The snapshot facade now re-exports _live_path_absent, and nothing reads it there. The fix itself is aimed at the cause.

Not justified as shipped

  • Item 9 — zero consumers: the hunk + _live_path_absent, adds it to the snapshot facade's re-exports. I grepped _live_path_absent across *.py and found 4 hits: the definition, one call inside snapshot_restore.py, this re-export, and the OWNERS table row. No code reads it through the facade, and no test patches it there. That leaves a facade name, plus an ownership row that has to be kept up, with no reader.
    Subtraction: delete _live_path_absent from the snapshot.py import list and from OWNERS in test_snapshot_refactor_ownership.py. Keep it private to snapshot_restore.
    Clears when: the re-export is removed, or a counted consumer reads snapshot._live_path_absent.

What this change ships

Inventory (10 items) — 9 justified

Intent: stop restore --mode replace from silently removing live files that the snapshot recorded as omitted. This is a FIX (#11499).

  1. Replace is refused when live files exist at omitted paths inside the components being restored — justified
  2. --dry-run gives the same refusal — justified
  3. New --allow-omissions flag on both restore parsers — justified
  4. With the flag, replace prints how many omitted paths it may remove — justified
  5. A refusal writes a state_restore_rejected reason=bundle_has_omissions audit entry — justified
  6. An omission with nothing live at that path is ignored, so a fresh-machine restore still runs — justified
  7. An unclaimed path or a malformed skipped field counts as an omission — justified
  8. An omitted ancestor directory, such as workspace, refuses a memory replace — justified
  9. _live_path_absent is re-exported from the snapshot facade — zero consumers
  10. The CLI spec, the user guide and the kirocrew-commands skill document the rule — justified

[FIRST-PRINCIPLES-REVIEWED] dacc6db

Description read

[DESCRIPTION-READ] 63cefc089f9ef200aa10164359f4466ec1c5d09f51b71cb2b9cc37645492ff3a

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] dacc6db

Verdict parsed from the review's SHA-scoped output markers for commit dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a: <one-sentence reason>

@buluoray
buluoray force-pushed the fix/restore-replace-omissions-11499 branch from 1c58c56 to cbebf65 Compare October 7, 2026 08:02
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] dacc6db

False positive or not applicable? A repository writer can comment:
/ai-review override gpt dacc6dbc2b8e991f83b576f5f4e2b5adacfadb3a: <one-sentence reason>

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@buluoray

buluoray commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

GPT 6.1 F1 (snapshot.py:318, omitted ancestor directory bypasses the guard): fixed in f332ff3.

_claimed now also matches an omitted path that is an ancestor of a restored tree (t.startswith(rel + "/")). So a skipped entry for workspace refuses --components memory replace, because memory's workspace/memory and workspace/knowledge sit under it. TestAnOmittedAncestorDirectoryCounts covers workspace and workspace/ (refused), plus the siblings workspace/notes.md and workspace/memoryx (not refused). Removing the new clause reddens exactly the two ancestor cases (2 failed / 19 passed).

@buluoray
buluoray force-pushed the fix/restore-replace-omissions-11499 branch from f332ff3 to e7625a6 Compare October 7, 2026 08:25
@buluoray

buluoray commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Design Review CONCERNS on f332ff3, both addressed in e7625a6:

  1. Refusal ignored whether a live file exists. _replace_omissions now drops an omission when nothing exists at that path in the live data home (_live_path_absent). A fresh-machine recovery restore therefore replaces normally. A manifest path that cannot be placed under the data home (empty, absolute, drive-qualified, or containing ..) is never treated as absent, so it still refuses. Two tests pin each half:
    • test_nothing_live_at_the_omitted_path_is_not_refused fails when the existence check is stubbed out.
    • test_a_path_that_cannot_be_placed_is_never_assumed_absent fails 3/4 when the placement guard is removed. The 4th case, an empty path, resolves to the home itself, which exists.
  2. Shipped docs. src/kiro_crew/docs/snapshot-and-restore.md adds the flag to its table and a short "Snapshots that left files out" section. builtin_skills/kirocrew-commands/SKILL.md lists --allow-omissions. The internal cli.md spec matches.

@buluoray
buluoray force-pushed the fix/restore-replace-omissions-11499 branch from e7625a6 to 5f1dc37 Compare October 7, 2026 08:28
@buluoray

buluoray commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

GPT 6.1 F1 on f332ff3 (test_restore_recorded_omissions.py:88, Windows separator in the fixture premise): fixed in 5f1dc37.

The expected manifest path is now os.path.join("skills", "my-skill", VICTIM), which matches how _record_skip writes it on each host. The refusal-message assertion keeps forward slashes on purpose, because _replace_omissions normalises backslashes before printing. That normalisation is pinned by the skills\\my-skill\\x matrix case.

Not run on Windows here; this host is Linux. The Windows backend lanes on this head are the check.

@buluoray
buluoray force-pushed the fix/restore-replace-omissions-11499 branch from 5f1dc37 to 7536415 Compare October 7, 2026 08:41
@buluoray

buluoray commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

GPT 6.1 F1 on 5f1dc37 (snapshot.py:336: backslash normalisation hid a POSIX filename containing a literal \): fixed in 7536415. I fixed it instead of asking for an override, even though the adjudicator flagged it as an edge case.

  • An omission is now treated as absent only when both the separator-normalised path and the literal recorded path are absent in the live data home.
  • The placement guard still runs on the normalised form, so a .. written with backslashes is still never treated as absent.
  • test_a_posix_filename_with_a_literal_backslash_still_counts (POSIX only) creates a live skills/my-skill/a\b.md and asserts that replace is refused. It fails when the literal-path check is replaced with True. 27/27 pass locally.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
@buluoray
buluoray force-pushed the fix/restore-replace-omissions-11499 branch from 7536415 to dacc6db Compare October 7, 2026 09:25
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@buluoray

buluoray commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

First Principles CONCERNS on dacc6db, item 9 (_live_path_absent re-exported from the snapshot facade with no reader): acknowledged; not pushed now.

The finding is correct. The re-export is unneeded, because only snapshot_restore._replace_omissions calls the helper. The fleet is under a GitHub API rate-limit hold that allows pushes only for real failures, and this change is advisory-only. Dropping it means removing the import line in snapshot.py and the OWNERS row in test_snapshot_refactor_ownership.py, and it can ride the next push that is needed for any other reason.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 16, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

🤖 Kiro Crew Auto-Pipeline [operator: buluoray#ce473fca]: closing this PR. The operator is pausing work on T3/T4 issues for now, so this stays unmerged. The branch is kept: reopen the PR or reuse the branch if the issue is picked up again.

@buluoray buluoray closed this Oct 25, 2026
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 25, 2026
@bolichen97
bolichen97 deleted the fix/restore-replace-omissions-11499 branch November 6, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

restore --mode replace with a bundle that has recorded omissions deletes the live files it omitted

1 participant