Repository navigation
fix(apps): create dependency locks exclusively - #10464
Conversation
Use exclusive creation before opening an existing dependency lock. Concurrent macOS openat calls could otherwise fail with ENOENT before reaching flock. Preserve directory pinning and no-follow flags, and fail closed if the lock disappears before reopening. Force overlapping first creation in the regression test and verify that the second installer reuses the first install's stamp.
|
Intent: Make concurrent first-time app dependency provisioning acquire the same lock reliably on macOS, so the second installer waits and reuses the first install's stamp. Validation: 42 dependency-install tests ran serially on head |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS A reproduced platform failure gets a strictly-safer protocol change — creator election, no truncation, fail-closed on vanish — pinned by tests at both openers and self-healing on retry. Suggestions
[DESIGN-REVIEWED] 99ab31c |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The macOS ENOENT-on-concurrent- Not justified as shipped
What this change shipsInventory (6 items) — 4 justifiedIntent: stop concurrent first-time app dependency provisioning from failing on macOS before the per-app lock is even acquired — a FIX.
Watch
[FIRST-PRINCIPLES-REVIEWED] 99ab31c |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
self-added: no
|
Use the same exclusive-create-then-open discipline for the uninstall side of the shared dependency lock. Preserve directory pinning, no-follow flags and data rollback on a vanished lock. Include current main without rewriting published history. The five lock-fix files are identical to the reviewed local 9dc90e41 commit; 62 targeted tests run serially and the backend wheel build pass.
|
self-added: no
|
|
Native reproduction evidence for This is a standalone stdlib-only POSIX probe, not an application/test-suite run. I syntax-checked it and executed it once under a 15-second parent-process timeout. Cases and pairs run serially; only the two opens in each pair race, using one two-worker pool. Each pair starts with an absent lock in a new directory. The directory fd stays open until both calls return, and its device/inode are compared with the named directory before cleanup. There is no unlink/rename or application import during a pair. Actual stdout: {"system": "Darwin", "release": "25.6.0", "machine": "arm64", "python": "3.12.13", "pairs_per_case": 16}
{"relative": true, "exclusive": false, "outcomes": {"errno=2": 16, "ok": 16}, "eexist_reopens": 0, "parent_identity_stable": true}
{"relative": true, "exclusive": true, "outcomes": {"ok": 32}, "eexist_reopens": 16, "parent_identity_stable": true}
{"relative": false, "exclusive": false, "outcomes": {"ok": 32}, "eexist_reopens": 0, "parent_identity_stable": true}
{"relative": false, "exclusive": true, "outcomes": {"ok": 32}, "eexist_reopens": 16, "parent_identity_stable": true}
Reproduction source (save as """Bounded native lock-create probe; no application imports or test framework."""
import collections
import concurrent.futures
import json
import os
import platform
import tempfile
import threading
from pathlib import Path
ROOT = Path(__file__).parent
PAIRS = 16
FLAGS = os.O_RDWR | getattr(os, "O_NOFOLLOW", 0)
def attempt(parent, dir_fd, relative, exclusive, barrier):
path = "lock" if relative else str(parent / "lock")
kwargs = {"dir_fd": dir_fd} if relative else {}
barrier.wait(timeout=2)
elected = False
try:
if exclusive:
try:
fd = os.open(path, FLAGS | os.O_CREAT | os.O_EXCL, 0o644, **kwargs)
except FileExistsError:
elected = True
fd = os.open(path, FLAGS, **kwargs)
else:
fd = os.open(path, FLAGS | os.O_CREAT, 0o644, **kwargs)
os.close(fd)
return "ok", elected
except OSError as exc:
return "errno=" + str(exc.errno), elected
def main():
print(json.dumps({"system": platform.system(), "release": platform.release(),
"machine": platform.machine(), "python": platform.python_version(),
"pairs_per_case": PAIRS}), flush=True)
# Cases and pairs are serial; only the two opens within one pair race.
with tempfile.TemporaryDirectory(prefix="lock-evidence-", dir=ROOT) as tmp:
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as pool:
for relative in (True, False):
for exclusive in (False, True):
counts = collections.Counter()
eexist = 0
stable = True
for i in range(PAIRS):
parent = Path(tmp) / f"{relative}-{exclusive}-{i}"
parent.mkdir()
dfd = os.open(parent, os.O_RDONLY | os.O_DIRECTORY)
before = os.fstat(dfd)
try:
barrier = threading.Barrier(2)
futures = [pool.submit(attempt, parent, dfd, relative,
exclusive, barrier) for _ in range(2)]
for future in futures:
result, elected = future.result(timeout=2)
counts[result] += 1
eexist += elected
after = os.fstat(dfd)
named = parent.stat()
stable &= ((before.st_dev, before.st_ino)
== (after.st_dev, after.st_ino)
== (named.st_dev, named.st_ino))
finally:
os.close(dfd)
print(json.dumps({"relative": relative, "exclusive": exclusive,
"outcomes": dict(counts), "eexist_reopens": eexist,
"parent_identity_stable": stable}), flush=True)
if __name__ == "__main__":
main()The invocation used a stdlib parent wrapper with |
|
self-added: no
|
|
self-added: no
|
|
self-added: no
|
|
self-added: no
|
|
self-added: no
|
cixuuz
left a comment
There was a problem hiding this comment.
Reviewed the full exact-head diff and tests. The exclusive creator election preserves the pinned-dir/no-follow contract, avoids truncation, fails closed if the lock vanishes, and is covered at both lock owners plus the concurrent path.
Use exclusive creation before opening an existing dependency lock. Concurrent macOS openat calls could otherwise fail with ENOENT before reaching flock. Preserve directory pinning and no-follow flags, and fail closed if the lock disappears before reopening. Force overlapping first creation in the regression test and verify that the second installer reuses the first install's stamp.
Problem / Motivation
Concurrent first-time app dependency provisioning was observed to fail on macOS while creating
.kirocrew-deps.lock: one directory-relativeopenat(O_CREAT)returnedENOENTbefore reaching the file lock, although the parent directory remained valid. The data-preserving uninstall path opens the same lock and had the same creation pattern.Why it matters
The per-app lock should serialize dependency work, not reject an otherwise valid app start before locking. Both provisioning and uninstall must use compatible lock creation so neither retains the observed first-create failure mode.
What changed (motivation → approach → change)
Both lock openers now elect a creator with
O_CREAT | O_EXCL. OnlyFileExistsErrorpermits a second open, with no creation or truncation flags. Both attempts retainO_RDWR,O_NOFOLLOWwhere supported, and the same pinned directory descriptor (or the existing path-based fallback).If the lock disappears before reopening, the operation fails closed rather than recreating it. The existing file-lock, install-stamp, uninstall rollback and trust-handling policies are unchanged. No broad retry, lock deletion, dependency-policy change or new shared abstraction is introduced.
ENOENTbefore locking on macOSO_NOFOLLOWplatform🟩 added · 🟨 changed · 🟥 removed · 🟦 unchanged
Every opener of this dependency lock now uses the same creator-election discipline.
Tests
os.supports_dir_fdcapability detection.Validated locally with 62 related install/uninstall tests run serially (
-n 0), flake8, isort, Python compilation and a backend wheel build. After incorporating the base-only frontend update, the entire PR patch was verified byte-for-byte identical to the reviewed patch; the same 62 serial tests and wheel build passed again. Full matrices run in CI, not on the local Mac.Manual verification
The original production-path failure reproduced on clean main. Two independent native probes reproduced the concurrent directory-relative creation failure with valid descriptors for the same directory and no intervening deletion; exclusive creation followed by a creation-free reopen passed those probes. These observations are from one macOS host, not a claim about all Darwin releases or a linked upstream kernel defect. The deterministic tests check the lock-opening protocol independently of whether the scheduling race fires.
The original provisioning patch passed local GPT/Opus review. The uninstall follow-up passed a separate read-only verifier; the base synchronization did not change any PR patch bytes. No production services or live app installations were changed. Published-head cross-platform validation remains with CI.
Screenshots / video
N/A — no rendered UI change in this PR.
Related Issues
no linked issue: discovered as a mainline test failure while validating PR #10276; kept separate from that PR's path-resolution fix.
Pattern harvest
Rule candidate: review-prompt
Pattern: when fixing lock-file creation, inspect every opener of that inode; use exclusive creator election followed by a creation-free reopen without weakening pinned-directory or no-follow checks.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)