Repository navigation
ci(ratchet-audit): carry Black's cache between main pushes - #30280
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing to check. What this change shipsInventory (6 items) — 6 justifiedIntent: get main's ratchet verdict sooner by not re-running Black on files that haven't changed since the last main push. This is a FIX for a slow step, measured in #30259.
[FIRST-PRINCIPLES-REVIEWED] 4878e5f Description read[DESCRIPTION-READ] 1692133b6d34d97a7524067301127ebd7b96c08c110ff543e8092203191b307d That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns. Reuse is decided entirely by Black's own content hash, mode key and version check. A missing or rejected cache just means a cold run with the same verdict, and the scratch-branch runs show a warm run that still catches the two misformatted files. Notes
[DESIGN-REVIEWED] 4878e5f Description read[DESCRIPTION-READ] b57ba54e9331f67eab3bdc549cb9913b1fb0f056597e66d3cae45cd4c90025d9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
a558349 to
37d5415
Compare
The whole-tree black step in main-ratchet-audit took a median 17.5 of 23.2 minutes over 16 sampled runs. Almost all of it is bounded_black's one-process-per-file start-up on an empty Black cache. Black's own cache keys entries on absolute paths and the CodeBuild checkout root changes every build, so the cache is exported as repository-relative JSON on main pushes and re-rooted before the gate runs. Black's verdict is unchanged: entries are reused only on a matching content hash, and the mode key and Black version are part of the cache's identity. Refs #30259
37d5415 to
4878e5f
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
Approve at 4878e5f3286cb0505d450f9f2a39d65198a7a530.
The gate's verdict still comes from Black. restore hands Black only files whose current bytes hash to the recorded SHA-256 (black_cache_portable.py L209). It refuses a cache from another Black version or mode (L198-L201), and every failure is a warning followed by a cold run. Only a main push exports and saves the cache (workflow L196-L223), so a merge group can restore main's entry but cannot write one. Workflow permissions stay contents: read, no secrets are added, and actions/cache is pinned to the same SHA that ci.yml uses. The 40-minute cap is kept for cold runs. Checks are green.
AI findings judged non-blocking
- Design Review (Notes): each main push saves a new cache entry keyed by its SHA. The entries are small JSON files that
actions/cacheevicts, and the restore-keys prefix picks the newest, so correctness is unaffected.
— crew-pr-reviewer (r146)
Problem / Motivation
Goal: Main gets its ratchet verdict sooner after each merge, because the whole-tree black check in Main Ratchet Audit stops re-checking files that have not changed since the last push.
The
Check formatting (black, baselined)step takes a median 17.5 minutes (13.2 to 21.9) of a 23.2-minute job, measured over 16 push runs sampled evenly across 2026-11-04 09:37Z to 2026-11-05 06:28Z (#30259). Install takes 0.1 minutes; no other step takes more than about a minute.Why it matters
Main Ratchet Audit renders main's ratchet verdict for every push, and the merge queue's required check waits on it. Every minute in this step is a minute before drift on main shows up, or before a merge group can land.
Not a goal
scripts/bounded_black.py's one-file-per-worker recycling. It bounds the compiled wheel's memory retention (CI: Backend Lint's black step killed by runner shutdown (exit 143) on 4 consecutive fork-PR heads #13386, ci(lint): run hosted black through bounded_black #13698), and that trade stays as it is.ci.yml'sbackend-lintblack step, which is already scoped to a pull request's own files.What changed (motivation → approach → change)
Black is not the slow part.
bounded_black.pystarts a fresh interpreter and imports Black once per file, about 10,940 times a run, on a Black cache that is empty every time. On the same tree locally, the wrapper took 671 s with an empty cache, nativeblack -W 8took 142 s, and the wrapper with Black's cache warm took 67 s. A warm run only re-checks files whose content changed, plus the 724 baselined files, which Black never caches because it only caches files it found formatted.Black's cache cannot be saved as-is. It keys each entry on the file's absolute path, and the CodeBuild runner checks out under a new
/codebuild/output/src<random>/root on every build, so a restored cache would match nothing.So
scripts/black_cache_portable.pycarries it in a portable form, and Black reads and writes its own file throughout:exportreads the cache with Black'sblack.cache.Cache.readand writes JSON: the cache file's name, and each formatted file's repository-relative path with the SHA-256 Black recorded for it.restorehashes each listed file in the current checkout withCache.hash_digest, and passes only the files whose bytes still match toCache.write, which records each file's stat and hash itself.The script contains no
pickleand parses nothing but JSON. Theblack.Modeit handsCache.readis the gate's own: theTARGET_VERSIONthatcheck_black_formatting.pypasses on the command line, plus[tool.black] line-lengthread with Black'sparse_pyproject_toml. Any other[tool.black]key that could change the mode turns the cache off, because this script would not reproduce it.The workflow restores the JSON (at the relative path
.cache/black-gate-cache.json, for the same changing-root reason) withactions/cache/restorebefore the black step and saves it after, from main pushes only (the samegithub.event_name == 'push' && github.ref == 'refs/heads/main'condition asci.yml's mypy cache). Any cache problem is a warning and a cold run, never a failed step. That includes a Black release whoseblack.cacheAPI is missing, renamed or takes different arguments.The verdict is the cold run's because the rules that decide reuse are Black's own: a file is handed back only when its current SHA-256 matches the content Black found formatted, the cache file name carries Black's mode key (target versions, line length, preview features) and a saved cache for another mode is refused, and a cache saved by another Black version is refused. The
actions/cachekey also includeshashFiles('pyproject.toml'), where the Black pin and[tool.black]live, so a pin or config change on main restores nothing. That cost one cold run in the last week (pyproject.tomlchanged once in 2,551 main commits).The 40-minute job timeout stays, since a cold cache still pays the full step. Its comment now says so.
Backwards compatibility
Compatible: the gate's command, scope and verdict are unchanged, and a missing or rejected cache only makes the run as slow as it is today.
Removes nothing: the diff adds a cache around an existing step and changes no gate, file or key.
Tests
test/test_black_cache_portable.py, which seeds and reads Black's cache throughblack.cache.Cacheas the script does:py310, line length 100);line-lengthchange), restores nothing;[tool.black]setting such aspreviewturns the cache off;.., backslash and empty paths are dropped;black.cachewith noread, areadtaking different arguments, a renamedwrite, a non-dictfile_data, or noblack.cachemodule at all: restore and export both exit 0 with an empty cache, and the gate step is untouched;pickle;BLACK_CACHE_DIRthe restore and export steps use, in that order.On the final head, the new file with
test/test_main_ratchet_audit.pyandtest/test_black_gate_contract.py: 81 passed.Proof: not provable -- other: CI workflow speed change; the new script has no previous version to revert to
Manual verification
CI, on the CodeBuild runner, via
workflow_dispatchon a scratch branch (since deleted). The branch carried head 6d24bad plus a one-line change letting the dispatch save the cache. The script and workflow on it are the same as this PR's; only the test file changed after.src/kiro_crew/recovery/(last changed 2026-09-25) and misformatting the cached, cleansrc/kiro_crew/providers/mirrors/codex.py(changed today)The third run restored 10,245 entries, not 10,246:
codex.py's bytes no longer hashed to the recorded value, so it was not handed back to Black and Black re-checked it.An earlier version saved the JSON under
${{ runner.temp }}, and the next run missed the exact key it had just saved, becauseactions/cachefolds the path string into the entry's version andrunner.tempexpands to the build's own root. The relative, gitignored.cache/black-gate-cache.jsonfixes that, and a test pins it.Local, black 26.10.1: a real cache for this tree exported 10,219 entries through
Cache.read, andrestorehanded 10,037 back to Black; the other 182 files had changed since that cache was written. The two weakened-guard checks were run by hand: dropping the path check, or thefile_datatype check, makes the matching tests fail (4 failures).Related Issues
Fixes #30259
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.