Repository navigation
Conversation
Recursive FSEvents avoids per-file overflow polling, but its initial two-second full-tree audit still spent CPU on quiet trees and excluded-file bursts could force rescans. Probe a rotating bounded set of known files, audit directories every 60 seconds, and preserve immediate policy-event invalidation plus conservative fallback on unsupported paths. Keep this watcher fix separate from draft PR #750's ranking and transport changes. Cover moved-in file-like directories and sensitive-path boundaries on the dispatch path; redact local paths from historical measurement receipts. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
The raw per-mutation JSON overwhelms PR review without changing the release verdict. Preserve the historical binary hash, per-client CPU samples, stability flags, mutation totals, and failure counts in compact summaries, and document how to rerun the harness for full output. The values are derived exactly from the original receipts. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Owner
Author
|
Status: exact-head 8b36f31 passed bench and macos-resource-regression; local Zig, MCP, and watcher checks are in the PR body. An attempt to mark this PR ready was blocked by Codegraff publication preflight (128 KiB committed-source / 256 KiB review-packet limit), despite a 71 KiB diff and green exact-head CI. This limitation is tracked in justrach/codegraff#1304. I am keeping the PR draft rather than bypassing the gate; no new release or tag has been created. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
This extracts the macOS watcher work from draft PR #750 onto
release/0.2.5856without its ranking, hosted-transport, or installer changes. Local APFS/HFS projects use recursive FSEvents; aliases, foreign volumes, unsupported filesystems, and unavailable streams keep the bounded kqueue/polling fallback. A healthy stream verifies at most 64 known files every two seconds and audits all directory metadata every 60 seconds. Excluded file events are filtered before the 2,048-path cap, while directory, symlink, unknown-type, and ignore-policy events remain visible. Dropped history still forces recovery.Why
The initial recursive stream still walked the whole tree every two seconds, defeating its quiet-tree CPU goal for #756; excluded-file bursts could also force unnecessary full rescans. Simply removing audits would miss lost events, while shrinking retrieval candidates would trade accuracy for speed. Bounded rotating probes plus a slower full audit retain recovery without touching default hybrid retrieval, hosted Jina, or the 4× ANN candidate pool. Exact-metadata collisions may take longer to detect in large trees; the fallback watcher retains its two-second audit.
Compact summaries of historical watcher stress receipts are included for context. They omit local paths and per-mutation logs, and they predate this repair; current-head results below are separate.
Verification — exact head
8b36f31a4c41ae58b4f85a5f8699b782c374e32fThe current head adds only the receipt-summary documentation commit after the tested watcher-code head
7bb2f6a79146361a6c166c58089cdfb9262701ef; source and binary are unchanged.zig build test --summary all— 33/33 steps succeeded; 1,657/1,670 tests passed, 13 skipped.zig build -Doptimize=ReleaseFastpassed; the copied arm64 candidate SHA-256 is7723f984ec66f81a0102946da2630ea455184bab034128dd434f08d178d9b7d4.zig build -Dtarget=x86_64-macos -Doptimize=ReleaseFastand the equivalent x86_64/aarch64 Linux and x86_64 Windows target builds all passed;fileidentified the four outputs as Mach-O, ELF, ELF, and PE32+ respectively. These are local unsigned cross-builds, not release assets.python3 scripts/e2e_mcp_test.py --binary .graff/verify/codedb-watcher-only --project "$PWD"— 76/76 passed. The lazy MCP suite passed all six scenarios. Live stress in recursive and forced-fallback modes passed multi-client churn, mutations, renames, deletion, and nested-directory discovery with no failures. A moved-ingenerated.lock/main.pybecame searchable in 178 ms whilesecret.key/private.pystayed absent.python3 scripts/repro_watcher_748.py --binary .graff/verify/codedb-watcher-only --out <report> --seconds 12 --directories 1200 --files-per-dir 4— 1.17% of one core quiet and 0.92% during excluded churn. The checksum-verified publishedv0.2.5856binary measured 7.57% and 33.70% in the same synthetic fixture. Both arms held 4,802 indexed files at sequence 4,803 throughout; this short sample is diagnostic, not a general CPU guarantee.8b36f31passed:bench(8m17s) andmacos-resource-regression(3m26s); the guard job was skipped. The prior watcher-code head7bb2f6aalso passed both jobs.Merging this PR only updates the release branch; it does not create a new tag, signed/notarized artifact, or deployment. The published
v0.2.5856tag remains unchanged. Release publication also requires installer hardening (#759) and resolving the current hosted-Jina rejection; this watcher-only branch does not change either lane.