Repository navigation
feat(aws-control): stop the nightly re-uploading a tree that has not changed - #12381
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the fingerprint pass's runtime on the multi-GB sessions archive it cites — every claim was measured except the nightly decompress-and-hash cost itself. [DESIGN-REVIEWED] 2781c4b |
First Principles Review (Fable 5) — ✅ human override accepted@chenmingwei23 overrode this lane for |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsThe candidate discovery pass produced no candidates. I've read the full diff and worked through the core logic myself: No findings. [OPUS-REVIEWED] 2781c4b Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
e38a789 to
fb57e1f
Compare
|
Head New:
Two facts shaped how those tests had to be built, and both were found by a test
Both traps are now guarded inside the tests by an explicit "retention deleted nothing, One mutation deliberately left uncovered, disclosed rather than papered over. Scope unchanged: retention itself is untouched, no seal list is extended, and the |
fb57e1f to
5bef29a
Compare
Review findings dispositioned at
|
| Check | Result |
|---|---|
| flake8 / isort / mypy | 0 / 0 / 0 |
black, comment-history (base origin/main) |
0 / 0 |
| subprocess-encoding, sync-io, loop-bound-locks, brand | 0 / 0 / 0 / 0 |
5 owned test files, -n0 |
363 passed |
| Mutation runs on the three fixes | 7/7 killed |
| ASCII on added lines and commit message | 0 / 0 non-ASCII |
The mutation kills are AssertionErrors about values with non-uniform blast radii,
read rather than counted. Disabling the consent gate, moving it ahead of the local
early returns, dropping the version comparison, or accepting a missing version each
reddens its own tests.
One disclosure kept rather than papered over: the if uploaded: guard around
_merge_uploads is defensive, not load-bearing. Removing it changes nothing
observable today, because a skip passes the matched run's own key, fingerprint and
version, so the merge rewrites identical values. There is no test for it, and
writing one would have manufactured evidence for a provable no-op. The comment in
the source says so.
Scope
Unchanged from the description: retention stays out (it is already merged and owns
that half), and the conversation database stays out. The local build still runs
every night; only the upload is skipped.
5bef29a to
94b038e
Compare
|
|
The blocking finding on this head names a value that has a second instance
|
|
Head WithdrawalMy comment above quoted the failure on That is the behaviour assertion, not the premise. The test reached its subject and the skip was accepted when it had to be refused. I read the older annotation and did not check it against the head I was fixing, so the fix I shipped -- rewriting the test to construct the stamp by hand -- narrowed the test away from the defect instead of addressing it. Withdraw that reading. The real defect was a contract one
So both terms compare equal while the slot holds a different record, and a second skip overwrites the first. Windows produced both collisions at once. The docstring asserted a property the code did not have. The fix uses a token the module already writesEvery run record already carries
A record predating these fields carries neither, so the type guards refuse and a full copy uploads. That is the direction every other proof in this design fails toward, and it is why there is no fallback to comparing The test that was removed is restored
Evidence357 tests across the four owned backup test files pass with The two remaining survivors are the sequence type guards taken one at a time, and that is reported rather than papered over: each covers the other on a record with no sequence, so dropping either alone is not observable while dropping both accepts an absent sequence as proof of identity. The pair is pinned jointly, the code comment says so, and the joint mutation is the seventh above. The first mutation run was itself informative. It reported the sequence comparison as surviving, which was true and not a false alarm: on a fine clock the flake8, isort, mypy, the black gate, comment-history, brand, subprocess-encoding and docs-lint all report 0. Zero non-ASCII in added lines and in the commit message. One commit, five files, working tree clean at push. The owning spec and the commit message both state the identity the code now compares. |
|
Head Measured, on two archive shapes
The cost has two terms, not one rateSolving the two shapes together gives roughly: The large-file case is entirely byte-bound: 319.1 MB at 235 MB/s predicts 1.358 s against 1.356 s measured, leaving no room for per-member cost. The small-file case is the opposite: its bytes account for 0.35 s of the 1.113 s, so the remaining 0.76 s over 20000 members is the per-member overhead. One caveat worth more than the numberA packed-bytes rate does not carry across shapes and must not be extrapolated. The small-file tree compresses 91x, because every member is nearly identical, so its 0.8 MB/s packed rate would predict 50 minutes for a 2.5 GB packed archive when the real cost is bounded by uncompressed bytes and member count. Extrapolate from those two terms, never from a packed rate. For a 2.5 GB packed sessions archive at a realistic 4x ratio, roughly 10 GB uncompressed and say 50000 members, the model gives about 43 s of hashing plus about 2 s of per-member overhead. Tens of seconds, and stated as a model rather than as a measurement of that archive, because I do not have one that size to time. Why that is cheap here rather than merely smallThe archive is built on every run whether or not the upload is skipped, which this PR's body already discloses as a scope boundary. So building has already read and compressed the entire tree, and the fingerprint pass is one additional sequential read of a file written moments earlier, which is the case most likely to still be in page cache. What it buys on an unchanged night is not sending 10.1 GB over the network. The probe drove the real |
|
/ai-review override first-principles 2781c4b: Default accepted as the product decision: the nightly skips the upload when the tree is unchanged. rfc-s3-backup.md is the covering RFC and its draft status is a documentation lag, not an unmade decision. Fail-closed is preserved: any comparison that cannot be proven uploads anyway. Accepting the recorded-RFC gap. |
Human judgment recorded@chenmingwei23 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Branch frozen: the first-principles override recorded above is pinned to 2781c4b. Any further push voids it and the block returns. Do not push this branch. |
bolichen97
left a comment
There was a problem hiding this comment.
Approving at 2781c4b37a.
I read _tree_fingerprint, _archive_entries, _manifest_digest, _unchanged_baseline, _record_skip and the _record_run_locked CAS rather than taking the description.
The comparison is taken at the right level. A tar.gz embeds per-entry mtimes and a gzip stamp, so a body-level check would report "changed" every night and the skip could never fire. Digesting the entry set read back out of the packed payload is the stronger of the two options for the reason given: a second source walk would have to re-derive what snapshot.COMPONENTS and _add_tree pack, and the day the two disagreed the answer would be "unchanged" for a tree that had moved. It also sees a redacted upload, which a source walk would not.
Every uncertain branch uploads, and I checked each one. No tree fingerprint, no recorded tree, fingerprints differ, no recorded key, object absent, recorded version not provable, stored version not ours, length mismatch, HEAD unanswerable, archive unreadable — all return None. _is_provable_version_id rejecting "null" is the detail that matters most: on a suspended-versioning bucket both sides report "null", and a length-only check would let a co-writer's overwrite pass while restore_download would then refuse the foreign object as ORIGIN_UNVERIFIED with no automated path back. Declining to skip there is right.
The two normalizations are bounded and justified. Only created_at is dropped from the manifest, not the whole member, so a switch to an unpinned staging walk is still noticed — and the assumption about snapshot.py is pinned by its own two-real-bundles test rather than a comment, which is the part that keeps this from silently ceasing to fire.
The skip does not strand the loop or the baseline. Fresh at so due_for_nightly does not rebuild every wake; matched key/fingerprint/version copied forward so tomorrow has something provable and retention has a version to retire; (process, sequence) compare-and-set so a concurrent run's record is not replaced by a stale baseline; neither label publish nor retention runs on a skip.
mtime being out of the digest is correct — a restore or checkout bumps it without changing a byte and would spend the very upload this removes.
Advisory lanes are clean at this head and the first-principles override is an accepted RFC-recording gap with fail-closed behaviour preserved, so I am not re-litigating it. Landing this before #12190 so the SHA-pinned override is not voided by a rebase.
…e-checks
`Backend Lint & Type Check (3.12)` is red on main and therefore on every open
PR:
backup.py:2526: error: Missing named argument "payload_kind" for
"_authorize_upload" [call-arg]
Two merges eleven seconds apart did it. #12190 (619c10f, 06:11:07Z) made
`payload_kind` a required keyword-only parameter; #12381 (4e8af7b,
06:11:18Z) added this call site without it. The second PR's CI ran against a
base where the parameter did not yet exist, so it merged textually clean and
semantically stale -- neither PR is wrong on its own.
The value is not a mypy silencer. The parameter's own docstring says `None` is
a real answer meaning this write carries no kind's payload, and the baseline
probe is a HEAD of an archive already in the bucket: it writes nothing, so no
per-kind grant governs it. Line 2413 states `None` at the comparable site.
Naming a kind here would be a behaviour change, not a cosmetic one: it would
subject a read to that kind's grant, so a withdrawn snapshot consent would
refuse the PROBE -- which this module answers by uploading, making the nightly
re-upload an unchanged tree every night, the exact cost the baseline exists to
avoid.
Nothing covered that. With the wrong kind passed, all 357 tests across the four
aws-control backup suites still passed, so the fix arrives with a regression
test on the assertion that was missing: the probe's authorization carries
`payload_kind is None`. Negative control fires -- `assert 'snapshot' is None`.
511 tests pass across five aws-control suites; mypy, black and flake8 are clean
on both touched files.
## Pattern harvest
Rule candidate: none for the typo class; the type checker already catches it,
and it did.
Not generalizable: the missing argument itself is a one-site omission.
The RECURRENCE is generalizable and is the third time this session has met it:
two PRs merged seconds apart, each green against a base that lacked the other,
combining into a break neither one contains. A required status check cannot see
it, because both were checked before the other existed. The candidate fix is a
merge-time re-check rather than a rule in any source file: re-run the type
check on the merge result when a PR's base moved after its last green, which is
the only moment the combination first exists.
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length; the security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. _archive_is_readable walks every member AND requires the tar's end-of-archive marker. Walking alone is not enough: tarfile swallows a header truncated anywhere but the first byte and reports a normal end of archive, so a valid gzip wrapping an incomplete tar reads as fine. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length, applied with fullmatch: '$' also matches BEFORE a trailing newline, so an anchored pattern accepts "abc\n" and sends a whitespace-bearing id to the CLI. The security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. apps/aws-control/data is neither agent-readable nor agent-writable -- it sits behind the agent file-tool floor and is bind-masked from every agent sandbox -- so this check stands between a stored value and an argv element, not between an agent and the CLI. _archive_is_readable walks every member AND requires the tar's end-of-archive marker. Walking alone is not enough: tarfile swallows a header truncated anywhere but the first byte and reports a normal end of archive, so a valid gzip wrapping an incomplete tar reads as fine. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length, applied with fullmatch: '$' also matches BEFORE a trailing newline, so an anchored pattern accepts "abc\n" and sends a whitespace-bearing id to the CLI. The security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. apps/aws-control/data is neither agent-readable nor agent-writable -- it sits behind the agent file-tool floor and is bind-masked from every agent sandbox -- so this check stands between a stored value and an argv element, not between an agent and the CLI. A fingerprint match is the whole test, and nothing else is asked of the recovered bytes. Whether they still open as a tar.gz is not checked, because the current-version read does not check it either and the upload side pushes payloads it cannot read (_tree_fingerprint returns "" for an unreadable tar.gz), so an own archive can legitimately be malformed. Refusing one only on the recovery path would hand the operator their own file when nobody overwrote the key and a refusal when somebody did, for identical bytes, at the cost of decompressing a multi-gigabyte archive to get there. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length, applied with fullmatch: '$' also matches BEFORE a trailing newline, so an anchored pattern accepts "abc\n" and sends a whitespace-bearing id to the CLI. The security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. apps/aws-control/data is neither agent-readable nor agent-writable -- it sits behind the agent file-tool floor and is bind-masked from every agent sandbox -- so this check stands between a stored value and an argv element, not between an agent and the CLI. A fingerprint match is the whole test, and nothing else is asked of the recovered bytes. Whether they still open as a tar.gz is not checked, because the current-version read does not check it either and the upload side pushes payloads it cannot read (_tree_fingerprint returns "" for an unreadable tar.gz), so an own archive can legitimately be malformed. Refusing one only on the recovery path would hand the operator their own file when nobody overwrote the key and a refusal when somebody did, for identical bytes, at the cost of decompressing a multi-gigabyte archive to get there. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length, applied with fullmatch: '$' also matches BEFORE a trailing newline, so an anchored pattern accepts "abc\n" and sends a whitespace-bearing id to the CLI. The security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. apps/aws-control/data is neither agent-readable nor agent-writable -- it sits behind the agent file-tool floor and is bind-masked from every agent sandbox -- so this check stands between a stored value and an argv element, not between an agent and the CLI. A fingerprint match is the whole test, and nothing else is asked of the recovered bytes. Whether they still open as a tar.gz is not checked, because the current-version read does not check it either and the upload side pushes payloads it cannot read (_tree_fingerprint returns "" for an unreadable tar.gz), so an own archive can legitimately be malformed. Refusing one only on the recovery path would hand the operator their own file when nobody overwrote the key and a refusal when somebody did, for identical bytes, at the cost of decompressing a multi-gigabyte archive to get there. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
…y is overwritten One backup drive is reachable by every install pointed at the account, and versioning is on for that reason. When a co-writer overwrites a key this install recorded, our archive stays on the drive as a noncurrent version. The restore read named no version, so it fetched the overwriting object, failed the recorded body fingerprint and refused -- with our archive present and unnameable. storage.get_file now takes an optional version, and when an ORIGIN_SELF key fails the fingerprint backup._recover_recorded_version makes exactly one further read, of the VersionId uploaded_versions records for that key. It is accepted only on the same evidence the current-version path uses -- that fingerprint, re-taken over the bytes that arrive -- plus _archive_is_readable. Absent, deleted, malformed or mismatched returns the existing ORIGIN_UNVERIFIED refusal, so the recovery can only find bytes that already pass; it can never widen what a restore accepts. The recovery runs only where the mismatch would REFUSE. Under foreign_ok the caller has already said it will take whatever is current without proof, so there is no refusal to rescue, and reaching past the current object would hand that caller different bytes than it accepted, relabelled as a proven self archive. The reply carries recovered, because the alternative outcome for this case is a refusal, which is loud: an operator who silently receives a recovered archive would never learn a co-writer is overwriting their backup chain. Version ids are validated as data. They are opaque and drawn from the full base64 alphabet, so the shape is printable ASCII with no whitespace and a bounded length, applied with fullmatch: '$' also matches BEFORE a trailing newline, so an anchored pattern accepts "abc\n" and sends a whitespace-bearing id to the CLI. The security rule is that a leading '-' is refused, because get_file passes the id as a bare argv element and the AWS CLI's own option grammar would read one as another option. There is no shell in the path. apps/aws-control/data is neither agent-readable nor agent-writable -- it sits behind the agent file-tool floor and is bind-masked from every agent sandbox -- so this check stands between a stored value and an argv element, not between an agent and the CLI. A fingerprint match is the whole test, and nothing else is asked of the recovered bytes. Whether they still open as a tar.gz is not checked, because the current-version read does not check it either and the upload side pushes payloads it cannot read (_tree_fingerprint returns "" for an unreadable tar.gz), so an own archive can legitimately be malformed. Refusing one only on the recovery path would hand the operator their own file when nobody overwrote the key and a refusal when somebody did, for identical bytes, at the cost of decompressing a multi-gigabyte archive to get there. Retention is deliberately untouched. _current_version_is_ours stays a question about the current version, so a recoverable-but-noncurrent key still holds no keep slot -- which errs toward retaining more. Closes #12391 Refs #12381
Problem / Motivation
The aws-control nightly built a complete archive and pushed it under a new
timestamped key every run, with nothing comparing it against what the drive already
held. On one real install that produced 15 snapshot archives totalling 10.1 GB, one
per night, growing from 232 MB to 2.5 GB each. The growth was not the operator adding
data at that rate; it was the same tree being packed and sent again.
Retention (#12159, merged) bounds how many copies are KEPT. It does not stop a copy
being sent, so the transfer and the storage write happened every night regardless.
Two things that look like the fix and are not:
so writing the same key twice retains the previous version and keeps billing for it
behind a listing that no longer shows it.
tar.gzembeds per-entry mtimes and agzip header stamp, so two runs over a byte-identical tree produce different bytes.
Measured in this branch: identical input, differing archives, every time. An
archive-level check would report "changed" every night and the skip could never
fire.
Why it matters
The cost is a ratchet rather than a level. Today's figure is small in dollars and it
has no ceiling. Carrying transcripts in the nightly makes it sharply worse: the two
session trees measure 22 GB across 28,947 files on the same host, and a full
re-upload of that per night is a different order of cost from the memory snapshot.
What changed
A run now computes what its archive CARRIES and sends nothing when the drive already
holds it.
The comparison is over the entry set, not the artifact.
_tree_fingerprintdigests one row per archive member - path, kind, size and a SHA-256 of its bytes - in
sorted path order. That is the per-entry source manifest the decision needs, read
from the packed payload. Reading it from the packed copy rather than walking the
source a second time is deliberate and is the stronger of the two options:
packs, knowledge that lives in
snapshot.COMPONENTSand in_add_tree. The day thetwo disagreed, the check would answer "unchanged" for a tree whose content had
moved. A backup that silently stops backing up is a worse failure than a local
rebuild.
switch changes the bytes that leave while the source tree is untouched. Taken over
the payload this notices; taken over the source it would not.
Two normalizations, both measured rather than assumed. Building two bundles from
one untouched home and diffing every member shows exactly two things move: the
bundle's root directory is named
kirocrew-snapshot-<stamp>, and insideMANIFEST.jsonthe single fieldcreated_at. So the root segment is stripped for thesnapshot payload (
volatile_root=True; the sessions archive passesFalsebecauseits
crewandcliroots are meaningful), and the manifest is compared withcreated_atdropped. Only that one field:purpose,staging(pinned vs unpinned)and
versionare not derivable from the file set at all, so dropping the wholemember - the obvious shortcut - would silently stop noticing a bundle that switched to
an unpinned staging walk.
mtimeis excluded on purpose; a content change alreadymoves the content hash, while a restore or a checkout bumps mtime without changing a
byte and would spend the very upload this removes.
The skip requires the previous archive to be PROVEN present, at its recorded
length. A record proves only that this install once wrote that key. Retention
deletes by design, and this drive is reachable by several installs by design - the
same reason
_body_fingerprintexists - so a co-writer can overwrite a name._unchanged_baselinetherefore HEADs the recorded key and comparesContentLengthagainst the recorded size. Every uncertain branch uploads: no baseline, no recorded
fingerprint (an install upgrading into this feature has none, and unknown is not a
pass), a moved tree, no recorded key, a missing object, a length mismatch, a HEAD that
could not be answered at all, or an archive this cannot read.
A skip records itself honestly and does not strand the loop. The record carries
uploaded: falseplus the matched run's key, body fingerprint and version, so thebaseline survives for tomorrow's comparison and retention keeps the version it needs.
It takes a FRESH stamp because
due_for_nightlyreadsat: keeping the old stampwould read as due on the very next wake and rebuild the archive every few minutes,
turning the saving into a busy loop. Neither the label publish nor the retention sweep
runs on a skip; both exist to follow a push, and running retention here would let a
stretch of unchanged nights walk the keep window down and delete the archive the next
skip has to prove is present.
The module docstring's "the per-session incremental integration with the storage
inventory is future work" sentence is replaced by a section stating what the code now
does, and saying that per-session incremental transfer remains out of scope - the RFC
lists incremental and deduplicating transfer among its non-goals.
Backwards compatibility
Additive. Run records gain
treeanduploaded; a record written before this changecarries no
tree, which never matches, so an existing install uploads once and gets abaseline. The uploads and version maps are written only on a real upload, so retention
reads exactly what it read before. An archive that cannot be read as a
tar.gzyieldsno fingerprint and is pushed exactly as it was pushed before, so this adds no new
refusal path: validating the payload is a separate question from deciding whether to
send it. One extra
head-objectper run is the only added cloud call.Tests
test/test_aws_control_backup_unchanged.py, 35 cases in three layers:same tree packed twice yields differing bytes and an identical fingerprint - plus a
falsifying case for every exclusion, so each normalization is shown to ignore
exactly what it claims and nothing more.
created_atis ignored whilestagingisnot; the bundle root is ignored while content beneath it is not; a
MANIFEST.jsondeeper in the tree is ordinary content whose every byte counts.
snapshot.py, which this module does not own. Two realbundles are built from one unchanged tree and their fingerprints must match. The day
a second volatile field appears, this goes red instead of the skip quietly never
firing again.
_unchanged_baselinehas a case, alongside the one accepting case that stops themall passing on a function that returns
None; and each end-to-end skip test has asibling proving the same setup uploads when something really moved.
Mutation-verified, 15/15 killed, with non-uniform blast radii (1 to 10 failures)
rather than one number repeated - which is what distinguishes per-behaviour
assertions from a harness reporting the same breakage each time. Every kill was read
rather than counted: each is an
AssertionErrorabout a value, not an import or typeerror. Disabling either skip, making the baseline unconditional, dropping the
presence proof, the length proof, the
created_atexclusion, the rootnormalization, the fresh stamp, the non-file members, or recording a skip as an
upload each reddens its own tests.
Manual verification
Measured against the real snapshot engine on an isolated
KIROCREW_HOME:are equal, and exactly one member differs in content (
MANIFEST.json) in exactlyone field (
created_at).MANIFEST.json's isrewritten each build. This is why mtime is out of the digest.
storage.head_object_metaraises rather than returningNonefor a throttle, atimeout or an owner-pin refusal, so all three of present / absent / unanswerable
land on the safe side.
Related Issues
Closes #12239
Out of scope by design, not oversight:
it needs a table-scoped export rather than a file copy.
the build means deciding what to pack without packing it, which is the drift hazard
described above.
Pattern harvest
A check whose input is nondeterministic has to be taken one level in from the
artifact: the bytes of a
tar.gzcannot answer "did anything change", while the entryset can. And when a guard rests on an assumption about a module you do not own, pin
the assumption with its own test rather than a comment - here, that only
created_atis volatile. Prose cannot fail, so without that test the skip would one day stop
firing silently and the nightly would quietly go back to a full upload every night.