Skip to content

Unmount dirty check caps at 4096 changed rows: no-op rows above the cap still refuse "Dirty: 0 uncommitted change(s)" #489

Closed

Description

@yifanxuaaa

Version: main at 582bc786f9 (after #488).
Area: engine/crates/layerfs-daemon/src/control/dirty.rs, the check before an Unmount without discard.
Follows: #462, #488.

Summary

Before a checked Unmount without discard, the dirty check reads the Workspace's changed overlay rows and, since #488, skips rows a Commit would drop: a locally created inode that no name binds any more, and a base directory whose mode equals the base. It reads at most ROWS = 4096 changed names and 4096 changed inodes. When it reaches either cap it sets truncated and, since it has not seen every row, cannot prove the Workspace clean, so it refuses Dirty.

So a Workspace with 4096 or more changed-inode rows that are all no-op is still refused, with the #462 symptom at a larger size:

Dirty: 0 uncommitted change(s); unmount with discard to drop them
detail: { changed: [], pendingMerge: false, truncated: true }

while workspace.commit answers UpToDate. Typical causes: a build, install or test run that creates and removes thousands of temporary files in the Workspace, or a command that touches many directories.

This was the behaviour on main before #488 too (main refused any changed inode at all); #488 fixed the case below the cap and left this one unchanged.

Smaller related points

  • rows counts every inode row read, including the skipped no-op ones, so the cap is reached by rows that are no change.
  • When exactly ROWS rows exist and all were read, >= still marks the report truncated.
  • Since fix(daemon): name every change a dirty Unmount refusal counts #488 the row counter includes directory rows (main counted only non-directory rows), so with many changed directories fewer file paths are named. truncated is set, so the report is honest; only naming is affected.
  • Each changed base directory costs one TreeTable::value plus one read_portable, one at a time. Commit reads the same records in batches (FilesystemRead::lookup_inodes).

Proposed change

The check needs a bound on memory, not on total rows read:

  1. Page through active_inodes and active_directory_entries window by window until they are exhausted, each window one short overlay job as now (no lock held across Store reads).
  2. Do not collect every row. Keep at most PATHS = 256 named paths and drop each window after it is processed, so resident state stays bounded however many rows exist.
  3. Stop early once PATHS changes are found: the Workspace is dirty and truncated: true.
  4. Reaching the end with no change a Commit would keep means clean: the Unmount proceeds.
  5. Batch the base-directory mode comparison per window instead of one lookup per directory.

WALK = 4096 (the base walk that names a path) can stay: reaching it only leaves a path unnamed and sets truncated; it does not decide clean or dirty.

Cost: a Workspace made only of no-op rows is scanned to the end before Unmount, the same order of work a Commit does over those rows. A Workspace with real changes stops early.

Expected

  • A Workspace whose overlay holds no change a Commit would keep unmounts without discard, whatever the number of rows.
  • truncated is set only when changes exist that are not listed.
  • Memory used by the check stays bounded.

Tests

Extend engine/crates/layerfs-api/tests/unmount_dirty.rs:

  • more than 4096 create+unlink pairs unmount without discard;
  • the same plus one real change refuses and names that change;
  • exactly 4096 no-op rows is not truncated;
  • chmod on a directory and back to its base mode unmounts cleanly;
  • removing a base directory refuses and names it.

Workaround until then

workspace.commit before workspace.unmount: it answers UpToDate, settles the overlay, and the Unmount then succeeds.

Activity

  1. added a commit that references this issue on Oct 28, 2026
    26b295c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions