Repository navigation
fix(acp): reclaim per-run work dirs and report refused alias unlinks - #13974
Conversation
|
Intent: Stop subagent and stateless-cron run directories under |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm the doctor's manual remedy suffices for existing hosts: the motivating 17k-dir legacy piles are counted, never auto-drained. [DESIGN-REVIEWED] 45e07ef |
c29a0a9 to
716c614
Compare
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @bolichen97 overrides the GPT 5.6 finding for [GPT-OVERRIDE] 45e07ef This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the "one gateway per data home ( What this change shipsInventory (10 items) — 10 justifiedIntent: stop per-run work directories and refused-unlink silence from accumulating undiagnosable garbage that slows session start — a FIX (field-counted: 2387/17221 dirs; tests added in this PR).
List capped at 10; the [FIRST-PRINCIPLES-REVIEWED] 45e07ef |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
716c614 to
ee31e34
Compare
|
|
|
|
|
|
|
|
ee31e34 to
cc774f0
Compare
|
|
|
/ai-review override gpt ecdb2cb: The adjudicator flagged the path extreme (swallowed ledger-append failure + crash + orphan surviving the boot reap + residue-only for an hour), and the residue-only/rmdir gate bounds any loss to Crew's own regenerable cli.json, lock sidecar and marker -- no run or user data. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
AI-review override not recorded: |
|
|
|
|
|
Rebased: main moved in src/kiro_crew/acp/skill_projection.py and test/test_native_skill_projection.py (#12605 bounded the lease/prune scans) plus 35 other commits (694f619 -> 31c291d); conflicts resolved by taking main's scan rewrite and keeping this PR's unlink-warning wiring (the lease call-site |
|
/ai-review override gpt 0f22bd0: The adjudicator flagged the path extreme (swallowed ledger-append failure + crash + orphan surviving the boot reap + residue-only for an hour), and the residue-only/rmdir gate bounds any loss to Crew's own regenerable cli.json, lock sidecar and marker -- no run or user data. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for |
|
Rebased onto |
|
|
/ai-review override gpt 9dd7edd: The adjudicator flagged the path extreme (swallowed ledger-append failure + crash + orphan surviving the boot reap + residue-only for an hour), and the residue-only/rmdir gate bounds any loss to Crew's own regenerable cli.json, lock sidecar and marker -- no run or user data. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Rebased onto |
|
|
/ai-review override gpt 45e07ef: The adjudicator flagged the path extreme (swallowed ledger-append failure + crash + orphan surviving the boot reap + residue-only for an hour), and the residue-only/rmdir gate bounds any loss to Crew's own regenerable cli.json, lock sidecar and marker -- no run or user data. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Review-ready at head
Not merging; awaiting maintainer review. |
Problem / Motivation
~/.kiro/agents/keeps growing withkirocrew-skill-view-*.jsonfiles until kiro-cli'ssession/newtakes over 90 seconds. #13225 stopped the prune from keying on "does the work dir still exist"; #13282 madekirocrew doctorcount the pile; #13431 bounded the prune's critical section. #13502 (merged) takes the alias backlog itself (one copy per agent, drained at boot). This PR takes the layer none of them touch, plus one silence that made the whole thing hard to diagnose:workspace_root()/subagent_<id>orcron_<job>_<run>, wrote one file into it (.kiro/settings/cli.json), and left it. Field counts: 2387subagent_*dirs on one machine, 17221cron_*on another (9901 from one job).Falsewith no log, exactly like a deliberate keep.Why it matters
Every derived-cwd spawn re-enumerates the workspace root, and thousands of one-file directories are inode and listing cost that nothing ever pays back. The silent unlink failure sent an outside report chasing the wrong root cause (the sandbox's read-only
~/.kiro/agentsseal, which never applied to the gateway process).What changed (motivation → approach → change)
Chosen approach: delete at run end, not one shared directory per job. A shared directory would change every run's cwd and the skill-view alias hash; deleting the run's own directory changes nothing a live run observes. Safe by shape:
session_work_dir.pyunlinks only Crew's own names (cli.json, its lock sidecar, the marker) and removes directories withrmdir, so a run that wrote anything keeps its whole directory.The invariant. A reclaim decision uses only evidence the deciding process owns: its session registry for its own runs, and its own data home's pid ledger (
kiro_session_pids.txt, reaped at boot bycleanup_orphaned_sessions) for a dead predecessor gateway. It never probes the liveness of a pid another process wrote to decide a deletion, and it never judges a directory marked by another data home. Five review rounds each added a guard to the sweep's inference about foreign processes; the retrospective deleted the inference instead.One shape table in
session_work_dir.pygenerates both the key predicate and the name filter: generated run ids only --subagent:<16 hex>(eight hex on shipped builds) and statelesscron:<8 hex>:<8 hex>. Reusedcron:<job>andcron:<job>:<agent>keys hold the effort overlay and are never disposable. The factory inconfig/loader.pymarks a work dir only when it DERIVED it from such a key, never an explicitcwd.AcpProvider.start(providers/acp.py) writes a.kirocrew-run-dirmarker --<data_home_id>(sha256 of the realconfig_dir()path) and<gateway_pid>-- before any other writer: idempotent for this gateway, replacing a same-home predecessor's (one gateway per data home holds theGatewayLock), never touching another home's, malformed left alone. Shutdown reclaim (_reclaim_work_dir) needs three fail-closed facts: the client'sprocess_tree_confirmed_dead(new property inacp/client.py,acp/runtime.py,acp/session_provider.py); a marker that names this home and this pid, or none (the flag is provenance); and registry ownership -- at registrationsession_allocation.pyinstalls a claim (LLMProvider.set_work_dir_claim_probe,providers/base.py) holding the registry lock from the no-other-holder check through the off-loop removal (a cancellation mid-reclaim waits for the worker before the lock is released), so a successor cannot register in between; arms that already know a provider lost its key calldisown_work_dirfirst (session_compaction.py).sweep_predecessor_work_dirs(slack/gateway.py, boot right after the ledger reap and before any session writer, and the hourly wake) reclaims a marked, residue-only, >1h-idle directory only when the marker names this data home and a gateway pid that is not ours and has no entry left in the ledger (session_pid.py's new read-onlyretained_gateway_pids); anything the ledger still retains, another home's marker, or a malformed one is kept. Streamed, bounded per wake (5000 entries, 20 s). Unmarked directories from older builds are never deleted;kirocrew doctor(cli_doctor.py) reports unmarked legacy directories and marked directories this home cannot reclaim in one line, viaworkspace_root(create=False).Warning.
pinned_fs.py'sunlink_verifiedgains anon_errorcallback fired only when the unlink raises;acp/skill_projection.py's alias and lease unlinks report through one rate-limited reporter naming file kind, operation and errno, saying "not writable" only forEACCES/EPERM/EROFS.The system spec
docs/system-specs/modules/acp-client.mddescribes all of it. Nothing insandbox.pychanges.Backwards compatibility
Compatible: no config key, CLI flag or file format changes.
unlink_verified'son_error,workspace_root'screateandLLMProvider.disown_work_dirare additive with defaults that keep every existing caller's behaviour. A run directory that holds anything besides Crew's overlay is left exactly as before.Tests
test/test_session_work_dir.py(new): residue-only directories are removed on both the pinned and the by-name form; planted content of five kinds keeps the whole tree byte-for-byte; grace window and mtime renewal; links at every level refused; the marker is the only provenance the sweep accepts;mark_run_dircreates, is idempotent for this gateway, replaces a same-home predecessor, refuses another home's and a malformed one; the sweep reclaims a dead predecessor with no ledger entry and keeps one the ledger retains, another home's, own-pid, young, live, unmarked-legacy and person-named entries; no pid liveness is probed anywhere (pinned); the sweep is bounded, streams its listing and finishes over later wakes; the doctor's two figures are exact and report their floor; only the generated run-id key shapes are disposable and the name filter agrees; the factory marks generated keys and never dashboard keys, persistent or agent-sequencecron:keys or an explicitcwd; shutdown reclaims a confirmed-dead tree with a matching marker or none, keeps a surviving or unknown tree, a foreign marker, and a run that wrote a file, and never fails on a reclaim error; no claim or a raising claim keeps the directory; a successor that registers during the predecessor's shutdown keeps its directory; the registry leaves a live sibling's directory in the lost-race, recycled-successor and bootstrap arms, while a recycle with no successor still reclaims; the boot sweep follows the ledger reap and precedes every session writer.test/test_native_skill_projection.py(added): a refused alias unlink and a refused lease unlink report once per interval through the same throttle with the count; permission-class errnos are diagnosed as unwritable, others worded neutrally; end to end through a write-protected agents directory.test/test_cli_doctor.py(added): the doctor does not create a missing workspace root and still counts an existing one.test/test_session.py,test/test_session_more_coverage.py,test/test_session_core_audit.py: provider doubles gain the synchronousdisown_work_dir.Manual verification
Ran the resolved prepare-pr backend gate floor locally: black (fresh cache), isort, flake8, mypy, comment-history, brand, harness-parity, feature-map and the related backend test set (34,486 passed; the 4 failures reproduce identically on pristine
origin/mainin this environment and touch no changed file). Frontend gates not run locally: the diff touches nowebsite/file.Related Issues
no linked issue: follow-up to #13225 / #13282 / #13431 and a sibling of #13502, which owns the alias backlog; no open tracking issue names the work-dir layer.
Pattern harvest
Rule candidate: review-prompt
Pattern: a cleanup that returns
FalseonOSErrorwith no log makes a permission failure indistinguishable from a deliberate keep; every "refuse to delete" path that can be hit by an environment fault should say so once.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)