Repository navigation
fix(daemon): stream the dirty check past any number of no-op rows - #491
Merged
yifanxuaaa merged 1 commit intoOct 28, 2026
Merged
Conversation
…hemeral-AI-Lab#489) The checked Unmount read at most 4096 changed names and 4096 changed inodes. Reaching either cap set `truncated`, so a Workspace whose overlay held only no-op rows past the cap was refused `Dirty: 0 uncommitted change(s)` while Commit answered `UpToDate`. Both indexed windows are now streamed to the end, one bounded page at a time, and only the changes are retained: - A changed name binding is a change a Commit keeps, so the name scan stops after more than `PATHS` rows; those paths cannot all be listed anyway. - Changed inodes are scanned to exhaustion. A locally created inode nothing binds, and a base directory whose mode equals the base, are still no change, and however many of them exist they neither refuse the Unmount nor mark the report truncated. - Base-directory modes are compared one window of serials per batch (`TreeTable::values`, the same `lookup_many` read the walk uses) instead of one lookup per directory. - `truncated` now means only that real changes exist that the report does not list: more than `PATHS` of them, or a path the bounded walk could not name. A Workspace with real changes still stops early: the inode scan ends once `PATHS` changes are found. `WALK` is unchanged; reaching it only leaves a path unnamed. Tests in engine/crates/layerfs-api/tests/unmount_dirty.rs: - 4100 create+unlink pairs unmount without `discard`; - the same plus one real change refuses and names only that change; - exactly 4096 no-op rows plus a real change is not truncated; - a directory chmod'd away and back to its base mode unmounts cleanly; - removing a base directory (after unlinking its children) refuses and names `/.git`, `/.git/index` and `/.git/link`; - 300 real changes name 256 paths and set `truncated`. The first two fail on the parent with `paths: [], truncated: true`; deleted: none. Production LOC: 102763 -> 102766 (delta +3) first-parent comparison (upstream/main 582bc78 -> this commit); engine layerfs-daemon +3. Scopes: api 2659 -> 2659 (0), core 38395 -> 38395 (0), engine 61709 -> 61712 (+3); reference absent (retired R9c; last counted 65,417 at tree 498dd19). Counted with tools/production_loc.py (blob 3341e4d) --root on a checkout of the parent commit and on the staged tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #489.
Problem
The checked Unmount read at most 4096 changed names and 4096 changed inodes. Reaching either cap set
truncated, so a Workspace holding only no-op rows past the cap was refusedDirty: 0 uncommitted change(s)whileworkspace.commitansweredUpToDate.Fix
engine/crates/layerfs-daemon/src/control/dirty.rsonly. Both indexed windows are now streamed to the end in bounded pages, and only changes are retained:PATHSbindings: each row is a change a Commit keeps and names one path, so more than 256 of them cannot be listed anyway.TreeTable::values, the samelookup_manyreadlocateuses) instead of onetree.value+read_portableper directory.truncatednow means only that real changes exist that the report does not list: more thanPATHS, or a path the bounded walk could not name. The old row-cap truncation is gone.A Workspace with real changes still stops early: the inode scan ends once more than
PATHSchanges are found.WALKis unchanged, and reaching it only leaves a path unnamed.Resident state stays bounded: at most a page over
PATHSbindings, and at most a page overPATHSchanged inode serials, are held at once; no rows are collected for no-op windows.Tests
engine/crates/layerfs-api/tests/unmount_dirty.rs:no_op_rows_past_the_old_cap_do_not_refuse_an_unmount: 4100 create+unlink pairs unmount withoutdiscard.a_change_after_many_no_op_rows_is_named: the same plus one real change refuses and names only/kept, not truncated.exactly_the_old_cap_of_no_op_rows_is_not_truncated: 4096 no-op rows plus a real change is named withtruncated: false.a_directory_restored_to_its_base_mode_does_not_refuse: chmod.gitaway and back unmounts cleanly.removing_a_base_directory_refuses_and_names_it: after unlinking its children,rmdir .gitrefuses and names/.git,/.git/indexand/.git/link.more_changes_than_the_report_names_set_truncated: 300 created files name 256 paths and settruncated.On the parent,
no_op_rows_past_the_old_cap_do_not_refuse_an_unmountfails withpaths: [], truncated: true.Checks (macOS, Rust 1.85.1)
cargo test -p layerfs-api --test unmount_dirty— 8 passed in 0.8 s (sotools/ci_tests.py'sSLOWlist is unchanged).cargo test --locked --all-targets --no-run— builds.cargo clippy --locked --all-targets -- -D warnings— clean.cargo fmt --all -- --check— clean.python3 -B tools/check_product_boundary.py— PASS (956 files).python3 -B -m unittest discover -s tools -p 'test_*.py'— 155 passed.Gap: this host's system SQLite is 3.54.0, which the overlay profile does not qualify, so the fixture panics before any test body. For the local run above I temporarily enabled rusqlite's
bundledfeature (SQLite 3.53.2, already allowlisted) and did not commit it. Not run: Linux FUSE/mount targets.Production LOC: 102,763 -> 102,766 (delta +3),
layerfs-daemononly; tests and comments are not counted.