Repository navigation
fix(packaging): clean failed bundle setup - #13538
Conversation
|
Exact head |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify CI runs CPython 3.13, since the deferred-ELOOP resolve shape is exercised only via a monkeypatched simulation, not a real 3.13 interpreter. [DESIGN-REVIEWED] a6a0f50 |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the CPython 3.13 deferred-loop claim on a 3.13 runner; both shapes are normalized either way, so only the test premise rides on it. What this change shipsInventory (8 items) — 8 justifiedIntent: FIX — a transient filesystem fault or symlink loop during bundle setup no longer strands the staging tree/marker into a standing refusal or escapes as a traceback.
Provenance checked: new tests fail on the untouched base per description and exercise each claim; deleted [FIRST-PRINCIPLES-REVIEWED] a6a0f50 |
Opus 5 Review (fork) — ✅ no blocking findingsReviewed |
f8fcd7f to
773d3a7
Compare
773d3a7 to
ad216f0
Compare
|
Dispositions for every open review finding, on head
|
8056389 to
7fd942b
Compare
7fd942b to
95e1200
Compare
1f403fb to
f11a415
Compare
|
Disposition, GPT 5.6 finding When Regression: |
|
Disposition, Opus 5 finding The |
f11a415 to
ec81442
Compare
|
Head On The GPT Local on this head: crew packaging suite 538 passed, 1 skipped; isort, flake8, black and |
ec81442 to
2fc1764
Compare
|
Disposition, Opus 5 advisory on Confirmed the premise on Linux: Regression: Local on this head: crew packaging suite 539 passed, 1 skipped. mypy on |
2fc1764 to
84da367
Compare
84da367 to
a6a0f50
Compare
Problem / Motivation
build_bundlecreates its staging tree and ownership marker before reading the existing report, output directory, and carried plan. A raw filesystemOSErrorin that setup window escapes before the main cleanup transaction, leaving both staging paths behind so later builds refuse the stale claim. Separately,_open_dir_nofollow_pinnedletsPath.resolve()raiseRuntimeErrorfor a symlink loop past callers that handle onlyOSError.Why it matters
One transient permission, I/O, stale-handle, or long-path failure can become a standing refusal that requires manual deletion. A symlink loop bypasses the builder's fail-closed refusal boundary and surfaces as a traceback. The staging descriptor must also close before early cleanup so repeated failures cannot leak file descriptors.
What changed (motivation → approach → change)
OSErrorboundary. It keeps the retained staging descriptor open while cleanup captures the path, requires the captured tree's device/inode to match that descriptor, removes only this run's identity-owned staging tree and marker through existing guarded helpers, then closes the descriptor exactly once and raisesExportRefusedwith the original exception as its cause. The purge reports whether capture/verification/sweep completed: successful cleanup recommends a retry, while safe preservation of a swapped or unpinnable tree names the residue and tells the operator to move it before retrying._rmtree_pinnedsweep to delete, so that sweep IS the deletion. A swallowed sweep failure previously let_purge_staging_best_effortreturn success over a tree still sitting undeleted in the aside._dispose_via_private_asidenow records whether the sweep completed and carries it in its existingboolreturn:Trueonly when the rename captured the target,verifyandsettleran, and the sweep completed;Falsewhen the target vanished before capture (removed or moved by another process, so nothing was captured or verified) or when the sweep did not complete._purge_via_private_asideforwards that result. Only_purge_staging_best_effortreads it; the transaction-path and post-promotion<out>.previouspurges ignore it, so a failed sweep never turns a bundle that already landed into a refusal. No flag and no raise: the sweep status travels in the return the one reader already consumed. When the staging purge reportsFalse, the pre-transaction refusal says the tree could not be released safely and that it is still at the staging path or under a.smc-purge-*directory beside it.ExportRefusedremains outside theOSErrorcatch, preserving each existing specific message and cleanup path. Promotion and rollback ordering are unchanged._open_dir_nofollow_pinnednormalizes CPython 3.12 resolve-timeRuntimeErrorand CPython 3.13 component-walk loop errors toOSError(errno.ELOOP, ...)at one boundary. Confirmed symlink components that reportENOTDIRunder `O_NOFOLLOWThe cleanup path never deletes an unowned output or report. A prior bundle and report remain byte-identical after an injected fault, staging and its marker are absent, every retained descriptor is closed, and the same build succeeds when retried.
Backwards compatibility
No successful-build behavior or bundle format changes. Failures that previously escaped as raw
OSErroror symlink-loopRuntimeErrornow return the existing packaging refusal type after bounded cleanup. SpecificExportRefusedmessages continue unchanged.Tests
a6a0f5013sits on main3f16157ba(a clean rebase of2fc176474,git range-diffidentical:4926e50b3carries the ci(lint): run hosted black through bounded_black #13698 bounded-black fix for CI: Backend Lint's black step killed by runner shutdown (exit 143) on 4 consecutive fork-PR heads #13386, and84da367c7hit the CI flake: test_the_trim_and_count_execute_as_written dies with SIGPIPE (141) under pipefail + head -n1 #13360 SIGPIPE flake in Backend Tests shard 2).2fc176474answered the Opus advisory onec81442d1: when the no-follow component walk in_open_dir_nofollow_pinnedmeets a link, theELOOPrefusal now names the component and says it is a link where a directory was resolved (a loop, or a change since resolution), instead of calling every such case a "symlink loop" (regressiontest_a_component_swapped_for_a_link_is_named_not_called_a_loop).ec81442d1reworded one test docstring fromf11a41551to present tense (the Comment History Gate flagged "used to").f11a41551amended1f403fba7to answer the two advisory findings on that head: the unconfirmed-release refusal no longer claims the tree is "still there or under a .smc-purge-* directory" (a tree moved by another process before capture is elsewhere), and the_refuse_unless_this_build_wrote_itinventory verdict now records that its sole caller sits inside the OSError boundary.test_post_marker_fault_with_staging_moved_away_does_not_claim_where_it_ismoves staging away before the purge's capture rename during a post-marker fault and asserts the refusal says deletion was not confirmed and names "wherever another process moved it"; it fails against1f403fba7'sbuild.py.PYTHONPATH: 539 passed, 1 skipped.test_a_staging_tree_moved_away_before_capture_is_not_reported_as_purgedmoves staging away before the capture rename and asserts_purge_staging_best_effortreturnsFalse, leaves the moved tree untouched and sweeps the empty aside.test_a_failed_purge_sweep_is_not_reported_as_releaseddrives a failed private-aside sweep end to end through the pre-transaction refusal: the message says "could not be released safely", names the.smc-purge-*location, and never says "was released".test_hash_and_promotion_authority.py:test_a_failed_private_aside_sweep_reports_residue_not_successforces the sweep to fail after capture+verify and asserts_purge_staging_best_effortreturnsFalsewith the tree retained;test_MUTATION_swallowing_the_sweep_failure_reports_a_false_successsource-mutatesreturn not sweep_failedtoreturn Trueand shows the purge then reportsTrueover an undeleted tree, so the return value is load-bearing;test_a_failed_sweep_never_raises_so_post_promotion_sites_do_not_refuseasserts_purge_via_private_asidereturnsFalsewithout raising on a failed sweep and leaves residue. Against the prior head'sbuild.py, the changed tests fail (the default purge returnedTrueover the unswept tree; the refusal did not name the aside).f7d66984485d93a6d8cbd9e2ac1a38e7df8ca6c0: the setup rows escape rawOSError; the symlink-loop rows escapeRuntimeError; the non-vacuity controls pass.3f30b6cac782ccc541fc857ec1c7659f6698de2ebecause the planted bundle-shaped victim is recursively deleted. The repaired path preserves the victim and displaced original, removes this run's marker, and closes the retained descriptor.is_file, output-directoryexists, and carried-planis_file, and proves staging/marker cleanup, descriptor closure, prior artifact preservation, and successful retry.scripts/check_black_formatting.py), isort and flake8 on the changed files, and mypy onbuild.pypass. The diff contains no.githubchanges.Manual verification
N/A — deterministic filesystem fault injection and real symlink-loop tests exercise the affected transaction and no-follow boundaries.
Related Issues
Fixes #13382
Pattern harvest
Rule candidate: review-prompt
Pattern: Once a transaction publishes an ownership marker, every setup read before the main mutation must share the transaction's descriptor release, identity-safe cleanup, and typed refusal boundary. A best-effort teardown whose sweep IS the deletion must report whether the sweep completed, not swallow its failure and return success over an undeleted tree.
Checklist
.githubchanges in the diff