Repository navigation
[Bug]: native Windows cannot delete a smoke fixture that contains a git object store, so the canary premerge gate reports a passing check as failed and retains the fixture #6710
Description
Preflight
- I searched existing issues and discussions for this behavior.
- This report is not an unpatched security vulnerability.
- The report and attachments contain no credentials or private/internal material.
Issue origin
Observed or reproduced in a real environment
LoopX version or commit
fdf03ec (upstream main, 2026-10-17)
Host or runtime surface
Other (native Windows 11, CPython 3.14.3, Git for Windows 2.49.0)
LoopX area
Build, packaging, installer, or CI
Problem
On native Windows the canary smoke runner cannot delete a smoke's own fixture root
whenever the smoke created a git repository inside it, because git marks its loose
objects and packs read-only and Windows refuses to unlink a read-only file. The
runner then records the check as failed even though the smoke's own process exited
0, keeps the tree (fixture cleanup failed; retained ...), and never retries the
removal. The same Windows behaviour makes three shipped public issue-fix smokes and
the shipped repo-branch-fixture command abort with an unhandled PermissionError.
Who is hurt and how much:
- Every control-plane diff on Windows gets at least one false failure from the
pre-merge gate thatAGENTS.mdandCONTRIBUTING.mdrequire contributors to run.
loopx/canary/premerge.pyattaches thecanary-runnerrisk profile to any
control_planepath (if "control_plane" in surfaces and "canary-runner" not in risk_profiles), and that profile always selects
examples/canary/catalog-planner-smoke.py, which creates a git fixture. A
whitespace-only change toloopx/control_plane/runtime/time.pyalready returns
ok=Falsewith acanary-runnerfailure whose ownreturncodeis0. - The reported verdict is indistinguishable from a real regression:
status: "failed",ok: falseandfailure_count: 1, with the real cause only in a free
textreason. A contributor who trusts the payload cannot tell whether their
change broke a check or the harness failed to clean up. - The project's own validation artifacts are red on Windows for this reason:
examples/canary/pytest-smoke-suite-facade-smoke.pyfails because
tests/test_smoke_suite.py::test_smoke_suite_script[canary__catalog-planner-smoke.py]
assertsresult["ok"] is Truefor a check that exited 0, and
examples/issue-fix-workflow-e2e-smoke.pydies withPermissionErroron
.git\objects\.... - Retained trees are never removed, so every Windows canary run leaks one temp tree
per affected check.
Measured on this host, loopx canary smoke-suite --suite full-public was run over
the 33 of 514 public checks that invoke a git object write (a git add/commit/...
in the script itself). Four of them carry the runner's own
fixture cleanup failed ... [WinError 5] ... .git\objects\... reason and are counted
as failures; one of those four exited 0. This is a separate root cause from
#6642 / #6652 (WinError 32, a live child pinning the directory): here nothing holds
the file, the read-only attribute alone blocks unlink.
Minimal reproduction
From a source checkout on native Windows, on commit fdf03ec61:
# 1. The smoke itself succeeds.
python examples/canary/catalog-planner-smoke.py # prints "catalog-canary-planner-smoke ok", exit 0
# 2. The same smoke through the canary runner is reported failed.
loopx canary smoke-suite --suite full-public \
--script examples/canary/catalog-planner-smoke.py --format jsonThe payload contains one selected check with "returncode":0 and
"status":"failed", "ok":false, "failure_count":1, and
"reason":"fixture cleanup failed; retained <TEMP>\\loopx-smoke-XXXX: [WinError 5] Access is denied.: '<TEMP>\\loopx-smoke-XXXX\\loopx-catalog-canary-smoke-XXXX\\ selector-repo\\.git\\objects\\24\\396a2d80...'".
The underlying host behaviour, with no LoopX code involved:
import os, shutil, stat, subprocess, tempfile, pathlib
root = pathlib.Path(tempfile.mkdtemp())
repo = root / "fixture"; repo.mkdir()
subprocess.run(["git", "init", "-q", str(repo)], check=True)
(repo / "a.txt").write_text("hello", encoding="utf-8")
subprocess.run(["git", "-C", str(repo), "add", "a.txt"], check=True)
read_only = [p for p in repo.rglob("*")
if p.is_file() and not p.stat().st_mode & stat.S_IWRITE]
print(len(read_only), "read-only object files")
shutil.rmtree(root) # PermissionError: [WinError 5] ...\.git\objects\4f\34e2...Or through a public smoke:
python examples/issue-fix-workflow-e2e-smoke.pyExpected behavior
A check whose command exited 0 is reported as passed, and the runner's own fixture
root is removed. A tree the runner owns should be removable even when a member
carries the Windows read-only attribute, the way tempfile.TemporaryDirectory
already handles it (its _rmtree clears the write bit and retries). Cleanup that
genuinely cannot complete must still fail closed and retain the tree.
Actual behavior
shutil.rmtree(root) in loopx/canary/runner.py (the finally block of
_run_check) raises PermissionError: [WinError 5] and the check is marked
failed, retains the fixture, and reports the cleanup error as the check's reason.
loopx/capabilities/issue_fix/acceptance_loop.py
(_remove_temporary_git_workspace) retries five times with sleeps and then raises
the same PermissionError, aborting build_issue_fix_repo_branch_fixture_packet
and the repo-branch-fixture command from its finally block.
Sanitized diagnostics
$ loopx canary smoke-suite --suite full-public --script examples/canary/catalog-planner-smoke.py --format json
{"ok":false, ... ,"failure_count":1, ... ,
"selected_checks":[{"source":"suite","command":"python3 examples/canary/catalog-planner-smoke.py",
"reason":"fixture cleanup failed; retained <TEMP>\\loopx-smoke-<id>: [WinError 5] Access is denied.: '<TEMP>\\loopx-smoke-<id>\\loopx-catalog-canary-smoke-<id>\\selector-repo\\.git\\objects\\24\\396a2d80dac5ee96f7f8e1b120f2358ae0fcda'",
"status":"failed","ok":false,"returncode":0,"duration_seconds":19.069,
"stdout_tail":"catalog-canary-planner-smoke ok\n","stderr_tail":""}]}
$ python examples/issue-fix-workflow-e2e-smoke.py
File "<repo>/loopx/capabilities/issue_fix/acceptance_loop.py", line 515, in build_issue_fix_repo_branch_fixture_packet
_remove_temporary_git_workspace(Path(tmpdir))
File "<repo>/loopx/capabilities/issue_fix/acceptance_loop.py", line 261, in _remove_temporary_git_workspace
shutil.rmtree(workspace)
PermissionError: [WinError 5] Access is denied.: '<TEMP>\\loopx-issue-fix-git-<id>\\.git\\objects\\32\\667d2c685aa01495f7571c1a13fe83ad1525f3'
Absolute local paths are shown as <TEMP> / <repo>; nothing else was changed.
Additional context
Nearest related work, none of which fixes this:
- [Bug]: Windows background helpers (detached usage sender, effect-runtime host) pin the caller's directory so it cannot be renamed or removed (WinError 32) #6642 / fix(runtime): stop long-lived helpers pinning the caller's directory on Windows #6652 fix
WinError 32from a live child pinning a directory; here the
file is idle and only its read-only attribute blocksunlink. - [Bug]: native Windows canary premerge/smoke-suite renders repository-relative script paths with the host separator and fails the gate's own contract smokes #6450 / fix(canary): render repository-relative plan paths in canonical form #6464 fixed the canary plan's path rendering, [Bug]: native Windows fails the canary premerge gate's own contract smoke (POSIX scripts/loopx shim exec) #6556 / fix(canary): run the premerge contract smoke through the host launcher #6560 fixed the
pre-merge contract smoke's POSIX wrapper exec, fix(canary): stop owned descendants before smoke fixture cleanup #5501 / fix(runtime): recover Windows lock creation and exact fixture cleanup #6517 stop owned processes
before fixture cleanup. The cleanup's own file removal was left untouched. - The pre-merge failures caused by this are reproducible at the base commit and were
already noted as pre-existing in PR fix(runtime): stop long-lived helpers pinning the caller's directory on Windows #6652 and PR fix(task-lease): discover machine identity on native Windows #6653.
loopx/capabilities/integration_branch/core.py has the same shape with
shutil.rmtree(temporary_root, ignore_errors=True) for a git worktree root, so on
Windows that cleanup silently leaves the tree behind instead of failing.
Activity
coordination: agent-x5p2 claim for Windows read-only git fixture tear…
coordination: agent-x5p2 backfill issue loopx-project#6710 and PR loo…
fix(canary): keep a passing smoke's verdict when its fixture holds re…
test: pin Windows read-only fixture teardown at both owners
ci(windows): run the read-only fixture teardown regressions on Windows
fix(canary): keep a passing smoke's verdict when its fixture holds re…
test: pin Windows read-only fixture teardown at both owners
ci(windows): run the read-only fixture teardown regressions on Windows
fix(canary): keep a passing smoke's verdict when its fixture holds re…
test: pin Windows read-only fixture teardown at both owners
ci(windows): run the read-only fixture teardown regressions on Windows
fix(canary): keep a passing smoke's verdict when its fixture holds re…
test: pin Windows read-only fixture teardown at both owners
ci(windows): run the read-only fixture teardown regressions on Windows