Repository navigation
fix: match git status paths literally instead of as pathspecs (#921) - #922
gustav-fff wants to merge 1 commit into
Conversation
git_status_for_paths fed every changed file to libgit2 as a wildmatch pathspec, making status O(entries * paths). A throttled FSEvents Rescan on the base dir re-indexes the whole tree and enqueues all files, which pinned fff-git-status for minutes on an 80k-file repo. Disabling pathspec matching switches libgit2 to its sorted pathlist filter. Closes #921
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
Closes #921
Root cause
GitStatusCache::git_status_for_paths(crates/fff-core/src/git.rs:141) adds every changed file as a libgit2 wildmatch pathspec. libgit2 then callsgit_pathspec__matchfor every index/workdir entry against every pathspec, which is O(entries × paths). The trigger: FSEvents sendsRescanfor the base dir during an ignored-only write burst (pnpm install). The full rescan is throttled, so the base dir lands innew_dirs_to_watch(crates/fff-core/src/watcher/background_watcher.rs:492).index_new_directoryre-walks the tree and enqueues all 80k files (:899), and the status pass then runs 80k × 80k wildmatches.Fix
status_options.disable_pathspec_match(true). Callers always pass exact file paths, so libgit2 can use its sorted pathlist instead (bsearch and merge walk in the iterators, with directory pruning). Side effect: filenames containing glob characters (app/[id]/page.tsx) no longer match unrelated siblings (app/i/page.tsx).Not changed: a throttled base-dir
Rescanstill re-walks the whole tree viaindex_new_directory(fall-through added in #751). That cost is now bounded (~0.25 s walk + ~0.6 s status per event) and stops when events stop. @dmtrKovalenko, your call whether a base-dirRescanshould skip the fall-through.Steps to reproduce
macOS (FSEvents). Run on pre-fix
main:Expected: CPU returns to ~0 shortly after the burst.
Actual (pre-fix):
Indexed new 80401 files from new directory /private/tmp/r/wt/. CPU stays at 99.6-99.8% andsampleshowsfff-git-statuswith 1843/1843 samples inupdate_git_status_for_paths -> read_status -> git_status_list_new -> git_diff__from_iterators -> git_pathspec__match -> dowild. Thegit_status_for_paths{paths_count=80401}span had not closed when I killed the process ~35 s later.How verified
git_status_for_paths{paths_count=80401}:read_status close time.busy=539ms..622ms. CPU went128 -> 149 -> 141 -> 38 -> 0.0%and stayed at 0.0%.git::tests::status_for_paths_treats_paths_literallycovers a literal[id]path, WT_DELETED, and WT_NEW in an untracked nested dir. It fails without the fix (app/i/page.tsxglob-matched) and passes with it.make test-rust: all pass (fff-search lib 217 passed).make lint-rust: clean.cargo fmt --check: clean.Automated triage via Gustav. Honk-Honk 🪿
Summary by CodeRabbit