-
-
Notifications
You must be signed in to change notification settings - Fork 15.4k
30% cargo target directory size regression between rust 1.96(.1) and 1.97(.1) #160659
Copy link
Copy link
Open
Labels
A-name-manglingArea: Name mangling / decoration aka symbol manglingArea: Name mangling / decoration aka symbol manglingC-bugCategory: This is a bug.Category: This is a bug.I-compiler-nominatedNominated for discussion during a compiler team meeting.Nominated for discussion during a compiler team meeting.I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}Issue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.Relevant to the compiler team, which will review and decide on the PR/issue.needs-triageThis issue may need triage. Remove it if it has been sufficiently triaged.This issue may need triage. Remove it if it has been sufficiently triaged.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.Performance or correctness regression from one stable version to another.
Description
Metadata
Metadata
Assignees
Labels
A-name-manglingArea: Name mangling / decoration aka symbol manglingArea: Name mangling / decoration aka symbol manglingC-bugCategory: This is a bug.Category: This is a bug.I-compiler-nominatedNominated for discussion during a compiler team meeting.Nominated for discussion during a compiler team meeting.I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}Issue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.Relevant to the compiler team, which will review and decide on the PR/issue.needs-triageThis issue may need triage. Remove it if it has been sufficiently triaged.This issue may need triage. Remove it if it has been sufficiently triaged.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.Performance or correctness regression from one stable version to another.
On our codebase (~1M LoC in a workspace with ~700 crates), we noticed a significant cargo target directory size regression after upgrading to Rust 1.97.1.
Examples of what we're observing:
cargo build --bins --releasewith default optimization settings (there are no[profile.release]section in our Cargo.toml) and latest mold linker: increase from ~6GB to ~7GB.cargo nextest archive(which as I understand simply builds everything with all features in debug profile), withCARGO_INCREMENTALenvironment variable set to1andRUSTFLAGSenvironment variable set to-C strip=debuginfoand necessary mold flags, and withdebug = 0in[profile.dev]: increase from ~62GB to ~79GB.cargo clippy --all-targetsorcargo clippy --binswithCARGO_INCREMENTAL=1and mold inRUSTFLAGS: no difference between 1.96.1 and 1.97.1 (at ~19GB and ~11GB respectively).The results are very reproducible; just changing channel in
rust-toolchain.tomlbetween 1.96.1 and 1.97.1 (and cleaning cargo target directory so that it's empty) and re-running the command is enough to see the old or the new size. This also reproduces in a clean container based onrust:1.96.1-bookwormorrust:1.97.1-bookwormimages (with regular libraries necessary for build installed from regular debian package repository).Comparing two cargo target directories (for
cargo nextest archiveexample) does not produce any insights; the directory structures are the same, and it's just that every file is suddely 20-50% larger on 1.97.1.debugdirectory is larger, everything indebug/depsis larger, and everything indebug/incremental/<crate_id>/<id>is larger (includingdep-graph.binandquery-cache.bin).Interestingly, archiving and compressing the target directory with
ZSTD_CLEVEL=12 tar --zstd -cf ...produces archives of more or less the same size (~16.5GB), hinting that all that extra disk space used does not actually contain any significant amounts of new information; that the extra data is somewhat redundant from zstd point of view.So I suspect that maybe this was caused by the change in #151994 . Maybe with the new mangling scheme, somehow our identifiers are so large (and there are so many of them in the output) that on non-incremental release builds they contribute 15% to the size of cargo target dir, and on incremental builds, 30%?
I'm sorry that this is not a very detailed or specific information.
Version it worked on
It most recently worked on: 1.96.1
Version with regression
rustc --version --verbose:@rustbot modify labels: +regression-from-stable-to-stable -regression-untriaged