Repository navigation
fix: preserve concurrent first log appends on macOS - #11987
Conversation
|
Intent: Preserve every cooperating writer's decision-log record when multiple callers first create the same daily file on macOS. Keep pinned-directory, no-follow, regular/single-link, bounded lock and rollback guarantees. This is the separately authorized production repair for #11951, not an extension of its test-only diff. Verified at
The |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify the three native macOS shards are green at this head — Linux CI cannot exercise the Darwin create race this fixes. [DESIGN-REVIEWED] e06aed2 |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the three native Mac shards pass on this head — the author's own checklist leaves fresh remote CI unchecked. What this change shipsInventory (6 items) — 6 justifiedIntent: FIX — stop concurrent first appends from silently losing decision-log rows on macOS.
Defect provenance is checkable: new tests fail on base (author reports 9 failures against unchanged main) and a linked native-Mac CI run. Grep of [FIRST-PRINCIPLES-REVIEWED] e06aed2 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsI've independently traced the production change. The rewrite replaces a single non-exclusive
The candidate list contained no candidates, and I could not ground any defect (a/b/c at 80+) on the changed lines. No findings. [OPUS-REVIEWED] e06aed2 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Prior openat diagnosis. All probe files were temporary and removed when the experiment ended. |
|
Current head: The requester authorized backups plus SHA-leased force-pushes to this original PR branch. Local backups:
Change since Local verification on native macOS at
Remote CI on
Base freshness: main is 1 commit ahead ( No subagents, live gateway changes, merge or auto-merge. |
0bbfc65 to
301eef0
Compare
|
301eef0 to
4273fb9
Compare
Create the log exclusively and open an existing leaf under the same directory pin, so Darwin's nonexclusive-create race cannot drop a writer's first record. A leaf lost between EEXIST and the open is one more lost interleaving: the existing bounded directory-create retry redoes the whole pinned, exclusive, no-follow sequence, so the row is still written. Test descriptor cleanup, the recreated leaf's flags and mode, and native concurrent writes without weakening path checks.
4273fb9 to
e06aed2
Compare
|
Problem / Motivation
Concurrent first appends can lose decision-log rows on macOS before the file lock is acquired. The failure was reproduced without pytest or mocks while investigating PR #11951's native Mac CI.
Why it matters
The caller catches the open error and warns, but the record never reaches disk. This is log data loss, not evidence of a memory leak.
What changed (motivation → approach → change)
src/kiro_crew/platform_log_append.pycreates a new log exclusively. If the name already exists, it opens it without creation flags under the same pinned directory. This prevents Darwin's descriptor-relative nonexclusive-create race from dropping a writer's first record.If that existing log disappears again before the second open, the
ENOENTreaches the existing bounded directory-create retry like any other lost interleaving. The retry redoes the whole pinned sequence, so the row is still written, through the same exclusive no-follow create under the same resolved anchor. No private exception type, extra veto or new retry loop is added.The resolve-once anchor, no-follow flags, regular/single-link checks, mode bits, file lock and Windows handle path are unchanged. No new lock, worker or setting is added.
test/test_platform_log_append.pycovers first creation, existing files, link/hardlink refusal, parent-directory replacement, and a real existing log removed during handoff—with and without its directory. That case now asserts the row lands, the recreated leaf is created withO_CREAT | O_EXCL | O_NOFOLLOWat mode 0600, and every descriptor of the losing attempt is closed. The link-refusal test distinguishes one creation attempt from its two open calls.docs/system-specs/common/platform-compat.mdstates the handoff outcome.Backwards compatibility
Compatible: valid existing logs retain their bytes, modes and serialization. Existing-file appends on POSIX cost one failed exclusive-create call. An existing leaf deleted by another actor during handoff is recreated by the bounded retry, which is what the base
O_CREATopen did in that interleaving. Windows behavior is unchanged. This remains best-effort logging, not a crash-durable journal.Tests
Verified on native macOS at
e06aed2ccd5e3d0816719c5708e73840bd78039e, based on8c6b3b272c3a(current main):prove.py --base origin/main: PROVEN; restoring the main implementation trips assertions.setup.py build_pypassed, missing SPA assets reported.Manual verification
Fresh stdlib-only native probe on this head: 100 rounds, four synchronized writers per fresh file, zero failed rounds; all four records retained each time. Files were fixture-owned and removed afterward.
Historical before/after on the original implementation: 40/100 failed rounds before, 0/100 after. That is historical evidence, not a fresh old-version run.
No live gateway, real log, production configuration or permission changed. Direct self-review only, as requested; no subagents or independent local model review claimed.
Review note
The requester authorized backing up each published head and SHA-leased rewrites to keep one commit within the two-commit CI limit. Local backups:
keep/pr11987-before-rewrite-20260919at0bbfc6564c85d9d9f01d7087e50937cb2ef1db8bandkeep/pr11987-head-301eef0a1-20260919at301eef0a11590eb03a410f92eb2e69ddee81ec5e.First Principles on the previous head (span
1dced0e1bd62) found that refusing to recreate a disappeared existing leaf dropped the current row without a demonstrated harm from safe recreation. The requester ruled to allow safe recreation. The guard and its private exception are deleted; the previous head's test asserting non-recreation is replaced by one asserting exclusive, pinned recreation. That is the only behavior change from the previous head.Current main includes theme repair #11978, the BSD-stat fixture repair, directory-create retry #12039 and two
aws_controlbackup commits that import this module but do not touch it. No unrelated production fix is included.The previous head's Windows shard 3 failed
test_throttle_parks_two_runs_on_one_scope_and_wakes_by_capacityon a superseded run before that; it is not waived, and no scheduling code or assertion was changed. Fresh ordinary CI, current-head reviews, PR Readiness and all three native Mac shards are required. Rolling verification record.Related Issues
Separate production repair for the log-loss blocker found during PR #11951. Its memory-reclamation tests are not included here.
Pattern harvest
Rule candidate: review-prompt
Pattern: a lock acquired after open cannot protect first creation. Test native concurrent writers on a fresh leaf, and let a lost handoff re-enter the existing bounded retry rather than adding a second failure path for the same interleaving.
Checklist