Repository navigation
fix(snapshot): scope the size estimate and skip entries the process cannot read - #11418
Conversation
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @iamwhatever overrides the GPT 5.6 finding for This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
|
Intent: Make |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS A permanently protected in-tree file disables Watch
[DESIGN-REVIEWED] 2c0a3ce |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify on a live data home that mid-walk What this change shipsInventory (10 items) — 10 justifiedIntent: Make
[FIRST-PRINCIPLES-REVIEWED] 2c0a3ce |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsNo findings. Both candidates hinge on the new retention guard (
[OPUS-REVIEWED] 2c0a3ce Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
c4008f4 to
833a6b7
Compare
|
self-added: yes
|
|
self-added: yes
|
833a6b7 to
503a469
Compare
|
self-added: yes
|
|
self-added: yes
|
503a469 to
d78c73e
Compare
|
self-added: yes
|
|
self-added: no
|
b05d36f to
5b0274f
Compare
|
self-added: yes
|
|
self-added: no
|
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
|
self-added: yes
|
|
self-added: no
|
|
self-added: no
|
|
self-added: no
|
|
/ai-review override gpt 2c0a3ce: Replace restore's phase-one rollback (_backup_tree_or_refuse) is fatal on any skipped entry and runs before all mutation, so an omitted-but-live file is either preserved in the retained rollback directory or the replace refuses outright; the loss is prevented or recoverable rather than silent, and the proposed pre-mutation guard is redundant with it. |
Human judgment recorded@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
Problem / Motivation
kirocrew snapshotdies with a traceback on a data home that holds a protected filethe process cannot
stat. No bundle is written, and--components config,memoryfails exactly the same way, so nothing can be backed up at all.
Why it matters
Snapshot is the sanctioned backup path. The docs tell people to take one before a
risky step, so this breaks the tool at the exact moment it is needed. Data homes
holding a platform-protected path are ordinary, not exotic, and there is no way to
work around it from the command line.
What changed (motivation → approach → change)
The size estimate measures the snapshot you asked for, staging no longer stops over an
entry it may not read, and a bundle that omits anything does not prune older backups.
MANIFEST.jsonMANIFEST.jsonMANIFEST.jsonMANIFEST.jsonMANIFEST.jsonEIO)--keepafter a run that omitted anything it was asked to carry--keepafter a run that only screened a symlinkidentity_changed, does not pruneEIO)🟩 added · 🟨 changed · 🟥 removed · 🟦 unchanged
The estimate exists for one "this may be large and slow" warning, and it walked the whole data
home before
--componentsnarrowed anything — so a protected file no component wants killed thecommand, and narrowing could not work around it. It now measures the selection, counts a refused
entry instead of raising, and says how many it could not read.
Staging a component TREE reads arbitrary user content, so a refusal there is a file the bundle
wanted and cannot have: skipped, printed, and recorded in
MANIFEST.json's existingskippedlist as
unreadable_entry, the same treatment a symlink already gets. Files, directories and atree's own root are covered on both traversals — the descriptor-pinned walk in
src/kiro_crew/pinned_fs.pyand the by-name walk a platform without directory descriptors takes.Every entry, and the loop's own tree root, is classified by an explicit stat FIRST, because
os.path.isdir,os.path.islinkandPath.is_dir()all answer False for a path they cannotstat — which reads a refusal as an ordinary absence. Only
ENOENTis absence;EIO,ESTALEand
ENOTDIRpropagate.Retention is what makes tolerating an omission safe.
--keepranks bundles by mtime alone,so an incomplete bundle displaced a complete one and reported success. A run that omitted
anything it was asked to carry now skips pruning and names the reasons. The guard asks a CLASS,
never a reason code:
pinned_fs.omits_wanted_data()is the one public question, backed by aprivate split kept next to the codes; a code the split does not know counts as incomplete, because a surplus bundle costs disk where a
wrong prune costs the last complete backup; and a test asserts the split covers every code, so
one added later cannot silently start pruning. Suppressing the prune on ANY recorded skip was
simpler and wrong —
symlinkis screened on every run, so any data home holding one would stoppruning forever. For the split to hold,
symlinkandnot_regularhave to mean a FIRST look:the pinned walk now hands the copy the inode it judged, and a link or FIFO found at that name
afterwards — the same-UID swap the module is built to refuse — is reported as
identity_changed, on the omission side, instead of under a screen code.Four places still fail hard, because failing is safer: a file a component declares, the
database re-copy pass at both ends, restore and merge, and any refusal to WRITE.
docs/system-specs/modules/cli.mddocuments the tolerance and its scope.Tests
test/test_snapshot_unreadable_entries.py, 66 cases. They denystat,open,scandirorlistdirfor one named entry, which is deterministic on Linux, macOS and Windows wherechmod 000is not — and the platform-protected class of file cannot be created by a test atall. Both traversals are exercised everywhere, including on a host that supports only one.
They lock in: a full and a
--components-scoped snapshot both succeed with an unreadable fileat the data-home root, which the estimate never looks at; the stderr count line, its wording
and its count when the selection overlaps; a refused file, a refused directory, a directory
whose stat is refused too, and a tree root refused at either its listing or its stat, each
skipped and recorded — while a tree root that is merely ABSENT is recorded as nothing and
still prunes normally, and a root on a failing disk raises instead of being skipped; a file whose
metadata reads but whose bytes are refused, reaching the refusal at a different syscall on each
traversal; an unreadable database never reaching the restage pass; an
EIOraised rather thancounted; a destination-side refusal propagating rather than being recorded as an unreadable
source; a prior complete archive surviving
--keep 1against a bundle with omissions, and anordinary run still pruning; and — the other half of the switch — that a default-argument walk
still RAISES and records nothing, so restore cannot silently become skip-and-drop.
Every guard was mutation-checked: reverting any one of thirty-one turns at least one test red,
with nothing skipped. The four for the root probe include both plausible WRONG fixes —
recording an absent tree as an omission, which passes the refusal tests and breaks
retention for every fresh data home, and widening the absent branch back to every
OSError, which hides a failing disk.Manual verification
N/A — unit coverage sufficient. The refusal is a platform behaviour that cannot be produced on
the CI hosts, so the tests deny the syscalls directly, which exercises both staging branches
instead of only the one the host supports.
Residual, filed not fixed.
restore --mode replaceclears a component tree and installsthe bundle's copy, so a file this change omitted is DELETED on restore — the manifest records
the omission and nothing on the restore path reads it. That is a change to what
--mode replacepromises rather than a repair to this fix, and restore is an explicit non-goal here, so it is
tracked as #11499 with the reproduction and three candidate fixes.
A second residual, pre-existing and unchanged: a narrowed
--componentsbundle also rotates inthe ordinary lane, so at
--keep 1a component-only bundle can displace a full one. Theselective path has always used the ordinary tarball name; closing it means deciding whether an
operator's own narrowing counts as incomplete.
Four pre-existing failures, none from this change. The related backend set is 41558
passed / 4 failed. One is
test_dev_fleet_app.py::test_trusted_bin_pins_the_resolved_target_not_the_symlink, whichreproduces on a pristine
origin/mainworktree. The other three aretest_pod_seed_scenarios.py::TestUpRefusesAnUnknownScenario, which cannot reach thesystemd user bus on this development host. None of the four touch
snapshot.pyorpinned_fs.py.Related Issues
no linked issue: reported directly, no public issue filed for the crash itself
Pattern harvest
Rule candidate: review-prompt
Pattern: a size or count walk over a whole store, run before the filter that would have
excluded the entry it dies on
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)