Repository navigation
fix(scratch): keep only the work in the scratch quarantine - #22435
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check one thing before merge: a tree whose only repositories sit in What this change shipsInventory (5 items) — 5 justifiedIntent: stop the hourly scratch sweep from keeping gigabytes of rebuildable bytes in
[FIRST-PRINCIPLES-REVIEWED] 1c47cce Description read[DESCRIPTION-READ] 41d924e8d448f6a51986566a122ad8bfc051b69bbeceb4a07d689cb6186ed1c0 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that the strip's [DESIGN-REVIEWED] 1c47cce Description read[DESCRIPTION-READ] c5d61b8b8f04f63e87bc086f9bf02d345a0f8b2d0b456258051dbf3ab552b48c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
4049031 to
53cc374
Compare
|
self-added: yes
|
|
|
Backend Tests (Windows) (3) red is main's own: test/test_chat_follow_default_slot_kinds.py::TestSlotCreatorRegistry (test_no_stale_entry, test_every_creator_is_classified) fails on a clean origin/main 6432cad checkout too, and names slack/interactions*.py, which this diff does not touch. Fix is in #22433; this PR rebases onto it once it lands. |
|
|
53cc374 to
daae3c4
Compare
|
self-added: yes
|
daae3c4 to
eba03af
Compare
|
|
|
|
|
|
The scratch sweep moves every dead, idle tree that holds a git repository into .quarantine for 14 days. Two things made that fill the disk: - pytest basetemps under the scratch dir hold the throwaway repositories test suites git init, so nearly every tree counted as holding a repo; - a quarantined clone kept node_modules, venvs and caches, which are most of its size and are rebuilt from the work. holds_git_repository no longer searches regenerable subtrees (node_modules, __pycache__, .pytest_cache, .mypy_cache, .ruff_cache, .tox, .nox, any dir holding a regular pyvenv.cfg, and a top-level pytest-of-*). A dir that itself holds .git or is a git dir is never regenerable. A tree that still holds a repo is moved as before, then stripped of those subtrees outside the sweep lock; trees already in the quarantine are stripped on a later sweep. Nothing inside a .git dir or a bare repository is touched, and a link with a regenerable name is removed as a link only.
|
Backend Tests (Windows) (8) on eba03af was this PR's own test: test_nested_venvs_are_searched_once cloned a repo 8 venvs deep and git hit Windows' 260-char path limit ('Filename too long'). The test now marks the repo with a .git file instead of a clone (still red on the exponential version, green on the fix). Pushed as 1c47cce; rebased on main. |
eba03af to
1c47cce
Compare
|
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Every review lane on the head is clean (only runner notices and PR Scope size warnings); held by the triage cron on a category, path or P0-area rule, not a finding. Approved on the watch holder's request.
Problem / Motivation
Goal: The scratch sweep stops keeping throwaway bytes in
.quarantine: pytest basetemps and rebuildable directories are no longer kept for 14 days.Since #19897 the hourly sweep moves every dead, idle scratch tree that holds a git repository into
<scratch root>/.quarantineand keeps it 14 days. On one host.quarantinereached 28 GB over 24 trees in about a day, and the 80 GB disk hit 99%.Two causes. Pytest puts
pytest-of-<user>/at the top of the scratch dir (it isTMPDIR), and about 112 test filesgit initthere, so nearly every tree that ran tests "holds a repository". And a kept KiroCrew clone (~4.2 GB) keepsnode_modules(0.95 GB) and.venv(0.63 GB), which tools rebuild.Why it matters
Every agent session that runs tests or installs dependencies in its scratch dir parks gigabytes for two weeks. A busy host fills its disk in a day, and then everything on it fails.
Not a goal
.quarantine..gititself is never trimmed.What changed (motivation → approach → change)
The quarantine now keeps the work, not what tools rebuild from it.
.quarantine?node_modules,__pycache__,.pytest_cache,.mypy_cache,.ruff_cache,.tox,.noxpyvenv.cfg(a venv) and no repo below itpytest-of-*at the top of a scratch tree.gitor is a git dir.gitdir or a bare repoholds_git_repositoryinsrc/kiro_crew/agent_scratch.pyskips those directories, so a tree whose only repositories sit in them is deleted at once, as any repo-free tree was. A tree that still holds a repository is moved as before. The quarantine pass of the same sweep then deletes those directories from every not-yet-expired tree in.quarantine, outside the sweep lock, on every hourly sweep. That covers a tree just moved and trees moved before this change, and a second run changes nothing. A link or junction is never removed or followed, whatever its name: it holds no bytes. Each entry fails open. One INFO line per tree names how many directories went.src/kiro_crew/docs/troubleshooting.mdsays what the quarantine drops and that a restored tree needsnpm cior a new venv.Backwards compatibility
Compatible: the sweep's inputs, names, expiry and log line for a move are unchanged; only which bytes stay in
.quarantinechanges.Reader: src/kiro_crew/agent_scratch.py:_expire_quarantine -- entry: release (gateway hourly sweep) -- deletes stamped trees after the TTL whatever they hold; test_a_quarantined_tree_is_deleted_only_after_its_ttl
Reader: src/kiro_crew/agent_scratch.py:sweep_dead_scratch -- entry: release (gateway hourly sweep) -- never reads inside
.quarantineexcept through_expire_quarantine; test_entries_the_sweep_did_not_write_are_left_in_the_quarantineReader: a person restoring a quarantined tree (src/kiro_crew/docs/troubleshooting.md) -- entry: chat -- the repository, its history and every other file stay byte-identical; installs and venvs are rebuilt, as the doc now says; test_the_quarantine_keeps_the_work_and_drops_what_tools_rebuild
Tests
New in
test/test_agent_scratch.py,TestGitTreesAreQuarantined(snapshot style):test_repositories_only_in_rebuildable_dirs_are_not_work: repos only inpytest-of-*andnode_modules→ tree deleted, nothing quarantined.test_a_basetemp_name_below_the_top_is_searched:pytest-of-*deeper down is not a basetemp.test_the_quarantine_keeps_the_work_and_drops_what_tools_rebuild: a clone loses installs, venvs, caches and the basetemp; every other path is byte-identical, including a.venvwith nopyvenv.cfg, a linkedpyvenv.cfg, and a clone namednode_modules.test_a_venv_holding_a_repository_is_work: a venv with a clone in itssrcis kept whole.test_nested_venvs_are_searched_once: nested venvs are listed once each, so the search stays linear.test_a_basetemp_name_inside_a_venv_is_workandtest_a_quarantined_venv_keeps_a_repository_below_a_basetemp_name:pytest-of-*counts as a basetemp only at the top of the scratch tree.test_nothing_inside_a_git_directory_is_stripped: regenerable names inside.gitand a bare repo stay.test_a_linked_install_is_never_followed:node_modules/pytest-of-*links stay as links; in-tree and outside targets stay byte-identical.test_a_tree_already_in_the_quarantine_is_stripped_on_a_later_sweep: strip on a later sweep, idempotent, repo still works.Proof:
prove.py --base origin/main: PROVEN.-n0, all green.-n 4 --dist loadgroup --max-worker-restart=2withtest/test_agents_janitor.pyandtest/test_subagent_shared_scratch.py, all green.Manual verification
N/A — unit coverage sufficient: the change is a filesystem walk the tests drive against real git repos, links and venvs.
Related Issues
Closes #22431
Pattern harvest
Not generalizable: the defect is this sweep's own definition of "holds a repository", which had no notion of tool-owned directories.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)