Repository navigation
High CPU: macOS watcher walks the whole tree on unrelated events and rehashes watch-cap overflow #748
Description
Problem
The macOS MCP watcher can consume excessive CPU after indexing has finished, even when indexed files are unchanged. Multiple MCP servers repeat the work independently.
Root cause
Source references below are for release 0.2.5854, commit 4eaca89:
- Unconditional whole-tree reconciliation:
src/watcher.zig:2406-2414,2483-2487runs reconciliation after each event batch or polling timeout, including empty batches.incrementalDiffDirtyrebuilds the parent index and walks from the root (2578-2587); even its unchanged-directory branch opens and recursively visits clean directories (2740-2758,2778-2787). - Polling candidates are treated as actual changes:
max_watcheddefaults to 1024 (src/config.zig:22). Directory watches consume the budget before file watches (src/watcher.zig:1616-1640).addUnwatchedinserts every overflow path into the dirty set on every cycle (1592-1596,2484-2485).dirtyForcesFileReadthen bypasses the unchanged-metadata shortcut inapplyKnownFile, forcing content reads and hashing (2720-2722,2625-2655). - Events accelerate this work: the two-second kqueue timeout is a maximum wait, not a minimum interval or debounce (
1983-2002). Repeated directory events therefore repeatedly trigger the traversal and overflow reads. Skipping a file during indexing does not prevent its parent-directory event from triggering this work. - Per-process amplification: each MCP server starts its own watcher (
src/commands.zig:487-535). The project singleton lock applies tocli-daemon, not MCP (src/main.zig:451-458).
Reproduction
- On macOS, create a synthetic repository with enough directories and text files to exceed the default watch budget.
- Start an explicit-root MCP server, complete initialization, and wait for
codedb_statusto reportscan: ready. - With no retrieval requests or edits to indexed files, repeatedly create and delete a root-level
.lockfile. This extension is excluded byskip_extensions(src/watcher.zig:2904-2912). - Compare CPU usage with the quiet repository; check that the indexed file count and sequence remain unchanged.
This reproduces excessive CPU with indexing ready and no indexed-file changes. Increasing the watch budget reduces overflow work but does not eliminate whole-tree traversal on events.
Fix direction
Keep the descriptor bound. Separate overflow polling candidates from actual invalidations so unchanged metadata can avoid rereads; avoid traversing clean subtrees on empty or unrelated events; coalesce event batches. Preserve change detection and path-safety checks. Add regression checks for bounded filesystem work with watch overflow and skipped-file churn. Raising the descriptor cap alone does not fix the cause.
Activity
[-]Multiple codedb processes consume excessive CPU concurrently[/-][+]High CPU: macOS watcher walks the whole tree on unrelated events and rehashes watch-cap overflow[/+]yxlyx commentedon Sep 10, 2026
@justrach — the issue now includes a reproducible watcher CPU problem, the responsible code paths, and a proposed fix direction. Could you take a look?
yousafsabir commentedon Sep 11, 2026
Experiencing this issue myself as well
justrach commentedon Sep 11, 2026
@yxlyx @yousafsabir — thanks for reporting and confirming this. The watcher CPU fix was merged into
mainin PR #749 and shipped in v0.2.5855.The fix coalesces watcher events, limits reconciliation to affected paths, and keeps overflow content checks on a separate two-second schedule so unrelated file churn cannot continually trigger them. In our macOS reproduction, average CPU during skipped-file churn fell from about 100% to 24%. Both macOS architectures passed the MCP checks, and the published binaries are signed and notarized.
Please update with
codedb updateand restart your MCP clients so they use the new binary. Closing as fixed in v0.2.5855; please let us know if you still see this after updating.justrach commentedon Sep 11, 2026
@yxlyx @yousafsabir — follow-up after stronger stress testing: v0.2.5855 reduces the event-driven work, but periodic hashing of overflow files still leaves substantial CPU cost on larger trees. We have an additional fix under review in draft PR #750; it is not in the published release yet.
The new watcher uses bounded recursive FSEvents on local APFS/HFS, with conservative kqueue/polling fallback for unsupported or unsafe cases. It retains periodic ignore-policy audits and full content verification when event history is lost. The watcher change is separately committed at 0c93dc3.
In the final 4,800-source-file synthetic fixture, three MCP clients used 1.80%, 1.70% and 1.75% of one core during excluded-file churn (5.25% combined, 20-second windows). Both recursive events and forced fallback passed all 122 named mutation cases per backend, including three-client runs. The checks cover edits with restored mtime, additions, renames, deletion and new directories; unit tests also cover metadata collisions and ignore-rule changes. These short synthetic results are not a general CPU guarantee or a like-for-like comparison with the earlier 9,600-file stress fixture.
Measurements, reproduction commands and limits are now available. The current branch also passes the unit build and 76/76 MCP checks. PR #750 remains a draft because its separately committed retrieval changes still have ranking and latency review items. No new release or notarized follow-up binary has been shipped;
codedb updatecurrently delivers v0.2.5855, not this additional watcher change.