Skip to content

fix(acp): orphan alias-lock sweep deletes under its own handle on Windows - #15633

Merged
bolichen97 merged 2 commits into
mainfrom
fix/skillproj-win-lock-sweep
Oct 1, 2026
Merged

bolichen97 merged 2 commits into
mainfrom
fix/skillproj-win-lock-sweep

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Make two main-side CI breakages green on current main by folding both root fixes into one PR: the orphaned skill-view alias-lock sweep now reclaims residue on Windows, and the member event-log service releases its file handles deterministically on retirement.

Both are breakages that landed on main unseen because main does not run the Windows (or full) shards on push, and both fail Backend Tests shards on every PR rebased onto main:

  1. Skill-projection alias-lock sweep (Backend Tests (Windows) shard 2). test/test_native_skill_projection.py::test_leftover_alias_lock_files_are_swept_only_when_provably_residue and ::test_a_lock_file_a_launcher_still_holds_is_never_swept fail. From e3309cd29 (fix(acp): make skill-view publication visible and bounded #15150).
  2. Member event-log handle leak (Backend Tests (Windows) shard 8). test/test_members_dm_thread.py::TestMemberActivityRoute::test_returns_recorded_entries_newest_first_with_allowlist_fields fails: record_activity(...) returns False on the first write of a fresh test after earlier tests ran on the same xdist worker. Flaked on shard 8 across multiple PR boards after test: reset the member eventlog service singleton between tests #15534 merged, on both Linux and Windows. This fix is folded in from fix(eventlog): release member log handles on service retirement to unflake dm-thread shard 8 #15613 (commit f8138d04a7), which will be closed after this PR merges.

Why it matters

  • (1) The sweep is what keeps an agents directory from accumulating empty <alias>.lock residue (one observed host carried 1,661 of them beside 23 aliases). On Windows it currently removes nothing, so residue grows unbounded, and the red shard blocks CI on every unrelated PR.
  • (2) The handle leak makes test/test_members_dm_thread.py flake red on shard 8 across many PR boards; left unfixed it also means a retired member event-log service holds its lease file descriptors until the cyclic GC runs, which on Windows pins a torn-down home directory and fails the next writer.

Not a goal

  • No change to the alias-lock sweep's policy (which locks are residue; the age/empty/beside-alias rules; the bounded walk; the held-lock exemption). Only the Windows file-sharing mode that defeated the unlink.
  • No change to the event-log service's behaviour in production (the service is rebuilt only when the crew-log root moves, which every test does and production never does); the fix only makes retirement release handles now instead of at GC.

What changed (motivation → approach → change)

Fix 1 — alias-lock sweep (platform_compat.py, acp/skill_projection.py).
Symptom: on Windows the sweep leaves genuine residue in place (removed == 0 where the tests expect 1). Root cause: _unlink_unheld_lock_file removes a lock file only while holding its own exclusive lock on it — the held lock proves no launcher owns the inode, so the unlink happens without first releasing (releasing before the unlink reopens the race the proof closes). On POSIX an open fd never blocks an unlink. On Windows a CRT descriptor (os.open) opens with FILE_SHARE_READ | FILE_SHARE_WRITE and omits FILE_SHARE_DELETE, so while the sweep's own verification handle lives, os.unlink of that same name fails with a sharing violation (PermissionError / WinError 32); the sweep catches it and declines to remove residue it should have removed. Change: a new platform_compat.open_lock_file_for_sweep(path) opens the lock file RDWR and, on Windows, uses CreateFileW with FILE_SHARE_READ_WRITE_DELETE so the unlink lands while the handle (and its msvcrt byte-range lock) is still held — preserving the hold-across-unlink guarantee identically on both platforms. OPEN_EXISTING never creates; no OPEN_REPARSE_POINT (the caller already refuses a reparse point at the name and re-checks (dev, ino) identity under the lock). POSIX keeps O_RDWR | O_NOFOLLOW.

Fix 2 — event-log handle release (crew_log/store.py, eventlog/log.py, eventlog/service.py).
Symptom: record_activity(...) returns False on the first write of a fresh test. Root cause: get_service() memoises one MemberEventLogService per process in a reference cycle (its projection registry holds set_on_change(service._on_change), a bound method back to the service). set_service(None) at each test boundary drops the external reference but the cycle keeps the service — and its cached MemberLog/CrewLog — alive until the cyclic collector runs; each CrewLog releases its write lease through a weakref.finalize that fires only then and holds its fd until then. On POSIX the deferral is invisible; on Windows the still-open descriptor keeps a mandatory lock that pins the torn-down home directory, so the next writer on the inheriting worker fails. #15534 reset the singleton reference at both ends — necessary but not sufficient, because the cycle keeps the handles alive regardless. Change: retirement releases the handles now — CrewLog.release_ownership() runs its lease finalizer explicitly (idempotent, atomic via weakref.finalize) and detaches it; MemberLog.close() releases its cached CrewLog; MemberEventLogService.close() closes every cached log, clears the caches, and severs the registry cycle; set_service()/get_service() close the outgoing/previous service on replace or home-rebuild (never mid-write — the home moves only at a test boundary).

Backwards compatibility

Compatible: no public API, config, or on-disk format changes. Fix 1 adds one internal platform_compat helper and leaves the sweep's observable policy unchanged — the only behaviour change is that the Windows sweep now actually deletes the residue it was always meant to. Fix 2 adds idempotent close()/release_ownership() methods and makes retirement deterministic; production never retires a live service, so no production path changes.

Tests

  • Fix 1: test/test_platform_compat.py::TestOpenLockFileForSweep (new) — POSIX RDWR unlink-while-open and POSIX symlink refusal (both skipped off POSIX, where their real os.unlink-while-open / os.symlink semantics do not hold), plus a faked-WinDLL test (runs everywhere) asserting the Windows CreateFileW requests FILE_SHARE_DELETE + GENERIC_READ|WRITE and that a wrap failure closes the native handle. The two shard-2 tests exercise the real sweep on POSIX and continue to pass.
  • Fix 1 ratchet: the extraction moved the sweep's by-name open behind open_lock_file_for_sweep, one hop past the link-screen gate's one-hop resolver reach — which would silently drop _sweep_orphan_alias_locks from the ratchet. open_lock_file_for_sweep is added to the gate's PRIMITIVES (exactly as open_lock_file already is for the same reason) and test/link_screen_sites.py is regenerated so the site stays declared.
  • Fix 2: test/test_members_dm_thread.py::TestEventlogServiceRetirementReleasesHandles (3) — clears-logs-and-breaks-cycle (proven to fail without the set_service close), home-rebuild-retires-previous, close-idempotent-reopens. The originally-failing TestMemberActivityRoute::test_returns_recorded_entries_newest_first_with_allowlist_fields passes.
  • Local, worktree src, -n0: test_members_dm_thread.py 182 passed; test/eventlog + crew_log core/concurrency/checkpoint + test_native_skill_projection.py + test_link_screen_hold_pin.py 582 passed; TestOpenLockFileForSweep 4 passed. black(py312)/isort/flake8/mypy clean.

Manual verification

N/A — unit coverage sufficient. The Windows-specific share mode is pinned by the faked-WinDLL test; both fixes' real Windows behaviour is validated by Backend Tests (Windows) shards 2 and 8 on this PR.

Related Issues

Folds in the fix from #15613 (commit f8138d04a7); that PR will be closed after this one merges. No tracking issue for the main-side CI breakage.

Pattern harvest

Rule candidate: review-prompt / agents-md
Pattern (fix 1): a cross-platform file helper that unlinks (or os.replaces) a path while holding its own open descriptor must open with FILE_SHARE_DELETE on Windows (via CreateFileW), because a CRT os.open omits it and a self-held handle then blocks the delete with a sharing violation — invisible on POSIX, red only on Windows CI.
Pattern (fix 2): a process-memoised singleton that sits in a reference cycle is not freed by dropping its external reference; any OS handle it holds through a weakref.finalize lives until the cyclic GC runs, so a boundary that must release handles deterministically has to close() them explicitly and sever the cycle, not just null the reference.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable) — N/A
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested a review from a team as a code owner October 1, 2026 00:42
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 73117544a8eba651f22429aafc6b49369333aefd — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify Backend Tests (Windows) shards 2 and 8 ran green on this PR — the faked-WinDLL test pins arguments, not real kernel delete-sharing behaviour.

[DESIGN-REVIEWED] 7311754

Description read

[DESCRIPTION-READ] dc9643a2697acb31ac39de573ab25dd465914b4bc51ae659a49f305a3898a68e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 73117544a8eba651f22429aafc6b49369333aefd — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Confirm Backend Tests (Windows) shards 2 and 8 go green on this PR — the faked-WinDLL test pins arguments, not real CreateFileW semantics.

What this change ships

Inventory (9 items) — 9 justified

Intent: FIX — make two Windows/full-shard CI breakages on main green: the orphan alias-lock sweep's unlink blocked by its own handle, and event-log lease handles released only at GC after service retirement.

  1. The Windows sweep now actually deletes orphaned <alias>.lock residue it previously left behind — justified
  2. New internal helper open_lock_file_for_sweep (one real consumer, _unlink_unheld_lock_file; reuses the existing _WIN_FILE_SHARE_READ_WRITE_DELETE constant; the deliberately separate-per-purpose helper shape the tail helper's own comment mandates) — justified
  3. The link-screen ratchet's PRIMITIVES names the new helper so the sweep site stays counted — justified
  4. CrewLog.release_ownership() releases the write lease now instead of at a GC pass — justified
  5. MemberLog.close() releases its cached CrewLog — justified
  6. MemberEventLogService.close() closes cached logs and severs the registry cycle (set_on_change(self._on_change), verified at service.py:613) — justified
  7. set_service() now retires the outgoing service — justified
  8. get_service() retires the replaced service on a home rebuild — justified
  9. New tests pin the delete-sharing open and the deterministic retirement contract — justified

Both defects have checkable provenance: the failing tests exist on base unmodified (test_native_skill_projection.py:4050,4259; test_members_dm_thread.py:2320), and both fixes sit at cause level (the share mode that blocks the unlink; the cycle that defers the finalizer).

[FIRST-PRINCIPLES-REVIEWED] 7311754

Description read

[DESCRIPTION-READ] 1bc8f182c7e3d29d7828a0519ecaa97756b4f740dbdecc8536e560dd5d080a84

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 73117544a8eba651f22429aafc6b49369333aefd and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 7311754

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 73117544a8eba651f22429aafc6b49369333aefd: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 73117544a8eba651f22429aafc6b49369333aefd — this comment is updated in place on each push.

Review details

No blocking defect: both survivors are advisory — the retirement rationale contradicts the module's own release timing, and the owning specs are not updated in this commit.

FINDING — src/kiro_crew/eventlog/log.py:194 — the added rationale that the cached handle "owns the member's write lease from its first append until it is dropped" (and that a "still-open descriptor keeps a mandatory lock") contradicts this module's own _append_through_contention at log.py:396 — "this class releases within the append that took it, because the reload below replaces the handle the claim was bound to" — and CrewLog.__slots__, which holds no descriptor: MemberLog.append ends in self._loaded = False; self._ensure_loaded(), rebinding self._crew_log to a fresh lease-less handle and freeing the lease-holding one at refcount zero, so at retirement release_ownership() finds _lease_finalizer is None and does nothing (store.py:2745's cycle-pinned-handle premise holds for the MemberLog, not for the CrewLog it caches) → Fix: state what retirement actually releases (the _logs/_names caches and the registry→bound-method→service cycle) in the three new docstrings, or cite the path on which a cached handle still holds a lease after its append returned.

FINDING — docs/system-specs/modules/crew-log-core.md:486 — the owning specs are not updated in the same commit as the contracts they document: crew-log-core.md still states "Release is bound to the HANDLE being dropped rather than to an explicit call" (and names the eager-release-at-eviction hazard) while store.py now exposes release_ownership() for exactly a cache-eviction caller, and platform-compat.md's "helper for each call" table — which already carries rows for open_log_file_for_tail and open_lock_file — gains no row for open_lock_file_for_sweep, the sweep-side open that deliberately permits FILE_SHARE_DELETE → Fix: rewrite the crew-log-core.md write-ownership paragraph to state the explicit-release contract and why the queued-write hazard does not apply to it, and add the open_lock_file_for_sweep row contrasting it with open_lock_file's omission pin.

[OPUS-REVIEWED] 7311754

Verdict parsed from the review's SHA-scoped output markers for commit 73117544a8eba651f22429aafc6b49369333aefd.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 73117544a8eba651f22429aafc6b49369333aefd: <one-sentence reason>

@chenmingwei23
chenmingwei23 force-pushed the fix/skillproj-win-lock-sweep branch from ca4c0a6 to f65019c Compare October 1, 2026 01:08
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 1, 2026
…dows

The orphan-lock sweep removes an empty <alias>.lock only while holding its
own exclusive lock on it: the held lock is the proof no launcher owns the
inode, so the unlink happens WITHOUT first releasing (releasing before the
unlink reopens the race the proof closes). On POSIX an open fd never blocks
an unlink, so this is correct and the two sweep tests pass.

On Windows a CRT descriptor (os.open) opens with FILE_SHARE_READ |
FILE_SHARE_WRITE and OMITS FILE_SHARE_DELETE, so while the sweep's own
verification handle lives, os.unlink of that same name fails with a sharing
violation (PermissionError / WinError 32). The sweep then declines to remove
genuine residue -- its own handle is what blocks the delete -- so
test_leftover_alias_lock_files_are_swept_only_when_provably_residue (the
residue file never unlinks) and test_a_lock_file_a_launcher_still_holds_
is_never_swept (the free sibling never unlinks, removed == 0 != 1) fail
Backend Tests (Windows) shard 2. main does not run Windows shards on push,
so this landed on main (e3309cd, #15150) and only surfaced on PRs rebased
onto it.

Fix at the root with a pin on both platforms: a new
platform_compat.open_lock_file_for_sweep opens the lock file RDWR, and on
Windows uses CreateFileW with FILE_SHARE_READ_WRITE_DELETE so the unlink
lands while the handle (and its msvcrt byte-range lock) is still held --
preserving the hold-across-unlink guarantee identically on both platforms.
OPEN_EXISTING never creates (the sweep only ever removes names already
present); no OPEN_REPARSE_POINT (the caller already refuses a reparse point
at the name and re-checks identity under the lock). POSIX keeps O_RDWR |
O_NOFOLLOW, exactly the open it was doing, named.

Tests + gates:
- TestOpenLockFileForSweep in test_platform_compat.py pins the POSIX
  unlink-while-open path and the POSIX symlink refusal (both skipped off
  POSIX, where their real os.unlink-while-open / os.symlink semantics do not
  hold), and -- via a faked WinDLL that runs everywhere -- that the Windows
  CreateFileW requests delete-sharing + GENERIC_READ|WRITE and that a wrap
  failure closes the native handle. The two shard-2 tests exercise the real
  sweep on POSIX and continue to pass.
- The extraction moved the sweep's by-name open behind
  open_lock_file_for_sweep, one hop past the link-screen gate's one-hop
  resolver reach -- which would silently drop _sweep_orphan_alias_locks from
  the ratchet (the hazard uncured, merely unseen). open_lock_file_for_sweep
  is added to the gate's PRIMITIVES (exactly as open_lock_file already is for
  the same reason), and test/link_screen_sites.py is regenerated so the site
  stays declared.
@chenmingwei23
chenmingwei23 force-pushed the fix/skillproj-win-lock-sweep branch from f65019c to d28c202 Compare October 1, 2026 01:24
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 1, 2026
…e retirement

test/test_members_dm_thread.py flakes on shard 8 across multiple PR boards
after #15534 merged, on both Linux and Windows. The CI traceback anchors it:

    test_returns_recorded_entries_newest_first_with_allowlist_fields
    assert record_activity(CREW, 'dashboard_chat-1', 'persistent', via='chat')
    AssertionError: assert False

The write side returns False on the FIRST call in a fresh test, after earlier
tests ran on the same xdist worker.

Root cause. get_service() memoises one MemberEventLogService per process and
rebuilds it only when the crew-log root moves (which every test does and
production never does). The service sits in a REFERENCE CYCLE: its projection
registry holds set_on_change(service._on_change), a bound method back to the
service. So set_service(None) -- which the autouse _reset_member_eventlog_singleton
floor calls at each boundary -- drops the last external reference but does not
free the graph; the cycle keeps it alive until the cyclic collector runs. Each
cached MemberLog's CrewLog releases its write lease through a weakref.finalize
that fires only then, and holds its file descriptor until then.

On POSIX that deferral is invisible (an advisory lock on a torn-down directory
harms nothing, and the next acquire against a fresh path just works), which is
why the leak could not be reproduced on Linux by running the file, targeted
interleaves, or the full real shard-8 selection. On Windows the still-open
descriptor keeps a mandatory lock that pins the (torn-down) home directory, so
the directory cannot be removed and the next writer on the inheriting worker
fails -- record_activity returns False. #15534 reset the singleton REFERENCE at
both ends, which was necessary but not sufficient: the reference-cycle keeps the
handles alive regardless.

Fix (root, deterministic). Retirement now releases the handles NOW instead of
waiting for GC:

* CrewLog.release_ownership() runs its lease finalizer explicitly (idempotent,
  atomic via weakref.finalize) and detaches it.
* MemberLog.close() releases its cached CrewLog and unloads.
* MemberEventLogService.close() closes every cached log, clears the caches, and
  severs the registry -> bound-method -> service cycle.
* set_service() closes the outgoing service when it is replaced; get_service()
  closes the previous service on a home-change rebuild. Neither happens while an
  append is in flight on the same instance (the home moves only at a test
  boundary), so nothing races a writer that still needs the lease.

Tests (whole-file guard, POSIX-provable). TestEventlogServiceRetirementReleasesHandles
pins the CAUSE rather than the Windows-only symptom: after set_service(None) the
retired service holds no cached logs and its registry cycle is severed; a
home-change rebuild retires the previous service; close() is idempotent and reads
re-open on demand. Verified to FAIL without the set_service close wiring and pass
with it.

Verification: full test_members_dm_thread.py + eventlog/test_eventlog.py +
test_members.py + test_members_eventlog_wiring.py + test_eventlog_hooks.py +
test_crew_log_member_unit_reclaim.py + test_members_roster_recency.py = 576
passed / 1 skipped (-n0). test_members_dm_thread.py stressed at -n4 x8 = 182
passed each. black (py312) / isort / flake8 / mypy clean on the changed modules.
@bolichen97
bolichen97 merged commit 7460284 into main Oct 1, 2026
95 of 105 checks passed
@bolichen97
bolichen97 deleted the fix/skillproj-win-lock-sweep branch October 1, 2026 02:30
bolichen97 pushed a commit that referenced this pull request Oct 1, 2026
…dows (#15633)

* fix(acp): orphan alias-lock sweep deletes under its own handle on Windows

The orphan-lock sweep removes an empty <alias>.lock only while holding its
own exclusive lock on it: the held lock is the proof no launcher owns the
inode, so the unlink happens WITHOUT first releasing (releasing before the
unlink reopens the race the proof closes). On POSIX an open fd never blocks
an unlink, so this is correct and the two sweep tests pass.

On Windows a CRT descriptor (os.open) opens with FILE_SHARE_READ |
FILE_SHARE_WRITE and OMITS FILE_SHARE_DELETE, so while the sweep's own
verification handle lives, os.unlink of that same name fails with a sharing
violation (PermissionError / WinError 32). The sweep then declines to remove
genuine residue -- its own handle is what blocks the delete -- so
test_leftover_alias_lock_files_are_swept_only_when_provably_residue (the
residue file never unlinks) and test_a_lock_file_a_launcher_still_holds_
is_never_swept (the free sibling never unlinks, removed == 0 != 1) fail
Backend Tests (Windows) shard 2. main does not run Windows shards on push,
so this landed on main (e3309cd, #15150) and only surfaced on PRs rebased
onto it.

Fix at the root with a pin on both platforms: a new
platform_compat.open_lock_file_for_sweep opens the lock file RDWR, and on
Windows uses CreateFileW with FILE_SHARE_READ_WRITE_DELETE so the unlink
lands while the handle (and its msvcrt byte-range lock) is still held --
preserving the hold-across-unlink guarantee identically on both platforms.
OPEN_EXISTING never creates (the sweep only ever removes names already
present); no OPEN_REPARSE_POINT (the caller already refuses a reparse point
at the name and re-checks identity under the lock). POSIX keeps O_RDWR |
O_NOFOLLOW, exactly the open it was doing, named.

Tests + gates:
- TestOpenLockFileForSweep in test_platform_compat.py pins the POSIX
  unlink-while-open path and the POSIX symlink refusal (both skipped off
  POSIX, where their real os.unlink-while-open / os.symlink semantics do not
  hold), and -- via a faked WinDLL that runs everywhere -- that the Windows
  CreateFileW requests delete-sharing + GENERIC_READ|WRITE and that a wrap
  failure closes the native handle. The two shard-2 tests exercise the real
  sweep on POSIX and continue to pass.
- The extraction moved the sweep's by-name open behind
  open_lock_file_for_sweep, one hop past the link-screen gate's one-hop
  resolver reach -- which would silently drop _sweep_orphan_alias_locks from
  the ratchet (the hazard uncured, merely unseen). open_lock_file_for_sweep
  is added to the gate's PRIMITIVES (exactly as open_lock_file already is for
  the same reason), and test/link_screen_sites.py is regenerated so the site
  stays declared.

* fix(eventlog): release member log handles deterministically on service retirement

test/test_members_dm_thread.py flakes on shard 8 across multiple PR boards
after #15534 merged, on both Linux and Windows. The CI traceback anchors it:

    test_returns_recorded_entries_newest_first_with_allowlist_fields
    assert record_activity(CREW, 'dashboard_chat-1', 'persistent', via='chat')
    AssertionError: assert False

The write side returns False on the FIRST call in a fresh test, after earlier
tests ran on the same xdist worker.

Root cause. get_service() memoises one MemberEventLogService per process and
rebuilds it only when the crew-log root moves (which every test does and
production never does). The service sits in a REFERENCE CYCLE: its projection
registry holds set_on_change(service._on_change), a bound method back to the
service. So set_service(None) -- which the autouse _reset_member_eventlog_singleton
floor calls at each boundary -- drops the last external reference but does not
free the graph; the cycle keeps it alive until the cyclic collector runs. Each
cached MemberLog's CrewLog releases its write lease through a weakref.finalize
that fires only then, and holds its file descriptor until then.

On POSIX that deferral is invisible (an advisory lock on a torn-down directory
harms nothing, and the next acquire against a fresh path just works), which is
why the leak could not be reproduced on Linux by running the file, targeted
interleaves, or the full real shard-8 selection. On Windows the still-open
descriptor keeps a mandatory lock that pins the (torn-down) home directory, so
the directory cannot be removed and the next writer on the inheriting worker
fails -- record_activity returns False. #15534 reset the singleton REFERENCE at
both ends, which was necessary but not sufficient: the reference-cycle keeps the
handles alive regardless.

Fix (root, deterministic). Retirement now releases the handles NOW instead of
waiting for GC:

* CrewLog.release_ownership() runs its lease finalizer explicitly (idempotent,
  atomic via weakref.finalize) and detaches it.
* MemberLog.close() releases its cached CrewLog and unloads.
* MemberEventLogService.close() closes every cached log, clears the caches, and
  severs the registry -> bound-method -> service cycle.
* set_service() closes the outgoing service when it is replaced; get_service()
  closes the previous service on a home-change rebuild. Neither happens while an
  append is in flight on the same instance (the home moves only at a test
  boundary), so nothing races a writer that still needs the lease.

Tests (whole-file guard, POSIX-provable). TestEventlogServiceRetirementReleasesHandles
pins the CAUSE rather than the Windows-only symptom: after set_service(None) the
retired service holds no cached logs and its registry cycle is severed; a
home-change rebuild retires the previous service; close() is idempotent and reads
re-open on demand. Verified to FAIL without the set_service close wiring and pass
with it.

Verification: full test_members_dm_thread.py + eventlog/test_eventlog.py +
test_members.py + test_members_eventlog_wiring.py + test_eventlog_hooks.py +
test_crew_log_member_unit_reclaim.py + test_members_roster_recency.py = 576
passed / 1 skipped (-n0). test_members_dm_thread.py stressed at -n4 x8 = 182
passed each. black (py312) / isort / flake8 / mypy clean on the changed modules.

---------

Co-authored-by: kirocrew-agent <agent@kirocrew.local>
(cherry picked from commit 7460284)
bolichen97 added a commit that referenced this pull request Oct 1, 2026
…dows (#15633) (#15699)

* fix(acp): orphan alias-lock sweep deletes under its own handle on Windows

The orphan-lock sweep removes an empty <alias>.lock only while holding its
own exclusive lock on it: the held lock is the proof no launcher owns the
inode, so the unlink happens WITHOUT first releasing (releasing before the
unlink reopens the race the proof closes). On POSIX an open fd never blocks
an unlink, so this is correct and the two sweep tests pass.

On Windows a CRT descriptor (os.open) opens with FILE_SHARE_READ |
FILE_SHARE_WRITE and OMITS FILE_SHARE_DELETE, so while the sweep's own
verification handle lives, os.unlink of that same name fails with a sharing
violation (PermissionError / WinError 32). The sweep then declines to remove
genuine residue -- its own handle is what blocks the delete -- so
test_leftover_alias_lock_files_are_swept_only_when_provably_residue (the
residue file never unlinks) and test_a_lock_file_a_launcher_still_holds_
is_never_swept (the free sibling never unlinks, removed == 0 != 1) fail
Backend Tests (Windows) shard 2. main does not run Windows shards on push,
so this landed on main (e3309cd, #15150) and only surfaced on PRs rebased
onto it.

Fix at the root with a pin on both platforms: a new
platform_compat.open_lock_file_for_sweep opens the lock file RDWR, and on
Windows uses CreateFileW with FILE_SHARE_READ_WRITE_DELETE so the unlink
lands while the handle (and its msvcrt byte-range lock) is still held --
preserving the hold-across-unlink guarantee identically on both platforms.
OPEN_EXISTING never creates (the sweep only ever removes names already
present); no OPEN_REPARSE_POINT (the caller already refuses a reparse point
at the name and re-checks identity under the lock). POSIX keeps O_RDWR |
O_NOFOLLOW, exactly the open it was doing, named.

Tests + gates:
- TestOpenLockFileForSweep in test_platform_compat.py pins the POSIX
  unlink-while-open path and the POSIX symlink refusal (both skipped off
  POSIX, where their real os.unlink-while-open / os.symlink semantics do not
  hold), and -- via a faked WinDLL that runs everywhere -- that the Windows
  CreateFileW requests delete-sharing + GENERIC_READ|WRITE and that a wrap
  failure closes the native handle. The two shard-2 tests exercise the real
  sweep on POSIX and continue to pass.
- The extraction moved the sweep's by-name open behind
  open_lock_file_for_sweep, one hop past the link-screen gate's one-hop
  resolver reach -- which would silently drop _sweep_orphan_alias_locks from
  the ratchet (the hazard uncured, merely unseen). open_lock_file_for_sweep
  is added to the gate's PRIMITIVES (exactly as open_lock_file already is for
  the same reason), and test/link_screen_sites.py is regenerated so the site
  stays declared.

* fix(eventlog): release member log handles deterministically on service retirement

test/test_members_dm_thread.py flakes on shard 8 across multiple PR boards
after #15534 merged, on both Linux and Windows. The CI traceback anchors it:

    test_returns_recorded_entries_newest_first_with_allowlist_fields
    assert record_activity(CREW, 'dashboard_chat-1', 'persistent', via='chat')
    AssertionError: assert False

The write side returns False on the FIRST call in a fresh test, after earlier
tests ran on the same xdist worker.

Root cause. get_service() memoises one MemberEventLogService per process and
rebuilds it only when the crew-log root moves (which every test does and
production never does). The service sits in a REFERENCE CYCLE: its projection
registry holds set_on_change(service._on_change), a bound method back to the
service. So set_service(None) -- which the autouse _reset_member_eventlog_singleton
floor calls at each boundary -- drops the last external reference but does not
free the graph; the cycle keeps it alive until the cyclic collector runs. Each
cached MemberLog's CrewLog releases its write lease through a weakref.finalize
that fires only then, and holds its file descriptor until then.

On POSIX that deferral is invisible (an advisory lock on a torn-down directory
harms nothing, and the next acquire against a fresh path just works), which is
why the leak could not be reproduced on Linux by running the file, targeted
interleaves, or the full real shard-8 selection. On Windows the still-open
descriptor keeps a mandatory lock that pins the (torn-down) home directory, so
the directory cannot be removed and the next writer on the inheriting worker
fails -- record_activity returns False. #15534 reset the singleton REFERENCE at
both ends, which was necessary but not sufficient: the reference-cycle keeps the
handles alive regardless.

Fix (root, deterministic). Retirement now releases the handles NOW instead of
waiting for GC:

* CrewLog.release_ownership() runs its lease finalizer explicitly (idempotent,
  atomic via weakref.finalize) and detaches it.
* MemberLog.close() releases its cached CrewLog and unloads.
* MemberEventLogService.close() closes every cached log, clears the caches, and
  severs the registry -> bound-method -> service cycle.
* set_service() closes the outgoing service when it is replaced; get_service()
  closes the previous service on a home-change rebuild. Neither happens while an
  append is in flight on the same instance (the home moves only at a test
  boundary), so nothing races a writer that still needs the lease.

Tests (whole-file guard, POSIX-provable). TestEventlogServiceRetirementReleasesHandles
pins the CAUSE rather than the Windows-only symptom: after set_service(None) the
retired service holds no cached logs and its registry cycle is severed; a
home-change rebuild retires the previous service; close() is idempotent and reads
re-open on demand. Verified to FAIL without the set_service close wiring and pass
with it.

Verification: full test_members_dm_thread.py + eventlog/test_eventlog.py +
test_members.py + test_members_eventlog_wiring.py + test_eventlog_hooks.py +
test_crew_log_member_unit_reclaim.py + test_members_roster_recency.py = 576
passed / 1 skipped (-n0). test_members_dm_thread.py stressed at -n4 x8 = 182
passed each. black (py312) / isort / flake8 / mypy clean on the changed modules.

---------


(cherry picked from commit 7460284)

Co-authored-by: Raymond Chen <45910466+chenmingwei23@users.noreply.github.com>
Co-authored-by: kirocrew-agent <agent@kirocrew.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: checking Automated validation is still running

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants