Repository navigation
fix: report actual st_blocks for directories (XFS inline dirs) - #694
Open
shivampwm2020 wants to merge 1 commit into
Open
shivampwm2020 wants to merge 1 commit into
shivampwm2020 wants to merge 1 commit into
Conversation
gdu charged every directory a fixed one-block (4096) usage instead of reading what the filesystem actually allocated. On XFS a small directory is stored inline in the inode, so stat reports Blocks: 0 and du reports 0, but gdu still printed 4.0 KiB. The fixed 4096 was later replaced by charging directories nothing at all, which swings the error the other way: on filesystems that do allocate a block, the directory's own usage disappeared from the total. Both come from the same place: gdu never read a directory's st_blocks. Read it, once, in the shared platform hook, and fold it into the directory's usage in resolveDirStats so every analyzer agrees. Verified against du on ext4: a directory holding two files now reports the same total as du, an inline directory reports 0, and --summarize, --db (sqlite) and the top-dir fast path all match.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #694 +/- ##
=======================================
Coverage 90.73% 90.74%
=======================================
Files 76 76
Lines 6843 6859 +16
=======================================
+ Hits 6209 6224 +15
- Misses 633 634 +1
Partials 1 1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
gdu charges a directory a fixed one-block (4096 B) of disk usage instead of reading what the filesystem actually allocated to it.
On XFS a small directory is stored inline in the inode:
statreportsBlocks: 0,xfs_bmapreports no extents, anddureports0. gdu reported4.0 KiBfor such a directory (issue #393).Root cause
gdu never read a directory's own
st_blocks.totalSize/totalUsage = 4096for every directory, which is exactly the reported4.0 KiBon an inline XFS directory.EmptyDirSize(a 512 B apparent-size sentinel) and set directory usage to0. That removes the wrong number but not the mistake: directories are now charged nothing at all, so on a filesystem that does allocate a block, the directory's own usage is dropped from the total. On ext4 a directory holding two files reports0 Bwheredureports4096 B.Both directions come from the same place: a directory's usage was assumed, never measured.
Fix
Measure it, once, in the shared platform hook, and fold it into the directory's usage where every analyzer already converges:
dirOwnUsage(path)readsst_blocks * 512for the directory itself, per platform (0where the platform exposes no block information, as before).setDirPlatformSpecificAttrsstores it asDir.OwnUsage.dirTotalsgainsownUsage, andresolveDirStatsadds it to the reported usage in both branches.resolveDirStatsremains the single definition of the rule, so the parallel, sequential, top-dir, sqlite and stored analyzers all agree.processSubDir), which aggregates separately, does the same.A directory is now charged exactly its
st_blocks:0for an XFS inline directory, the real block count elsewhere — the same ruleduuses.Testing
Verified against
duon ext4 (-n,--summarize, and--db/sqlite all agree):du--apparent-size(unchanged)A multi-block XFS directory reduces to the inline case:
st_blocksis0, so the reported usage is0, matchingdu.New tests:
TestDirOwnBlocksAreCounted— a directory's reported usage includes its ownst_blocks, checked againststat, not an assumed block count.TestEmptyDirOwnBlocksAreCounted— the gdu assumes directory takes up 1 block when it doesn't (on XFS) #393 case: a directory with zero blocks reports zero.Existing tests that encoded the old "directories cost nothing" rule (
TestEmptyDirStatsAcrossAnalyzers,TestSqliteAnalyzerIgnoreFileType,TestSqliteAnalyzerTimeFilter,TestTopDirAnalyzerCancellationStopsActiveSubdirectory) now take their expected usage fromstat, so they pass on any block size and on any filesystem.go test ./...— passgolangci-lint run -c .golangci.yml— 0 issuesgofmt— cleanFixes #393
— Shivam Maurya