Repository navigation
build: keep the CCCL submodule out of the sdist - #3138
Merged
Merged
Conversation
Follow-up to NVIDIA#2940. That change vendored CCCL as the submodule cuda_core/third_party/cccl and grafted its libcudacxx/include tree into the sdist through cuda_core/MANIFEST.in. The setuptools-scm 10 git file finder lists the files of an initialized submodule, so building an sdist from a checkout that has the submodule checked out now packs the entire CCCL tree (cub, thrust, docs, tests, ...) into the sdist, although cuda.core only needs the libcu++ headers that MANIFEST.in already adds explicitly. The same listing is also walked on every egg_info run, which slows down wheel builds. Mark the submodule export-ignore, which the finder honors, so it is skipped. The MANIFEST.in graft still provides the headers and licenses.
juenglin
added
packaging
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Summary
Priority: ➖ Normal Change: Bug fix Merge Risk: ⚪ Minimal · up to
|
Contributor
Author
|
/ok to test f40cbb1 |
Contributor
|
juenglin
enabled auto-merge (squash)
October 16, 2026 18:12
jpascucci-nv
approved these changes
Oct 16, 2026
Member
|
Seems like a surgical change! Thanks. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #2940.
Motivation: the
cuda.coresdist became ~6x larger than intended#2940 vendored CCCL as the submodule
cuda_core/third_party/cccland grafted only itslibcudacxx/includetree (plus the licenses) into the sdist throughcuda_core/MANIFEST.in. The comment there states the intent: "setuptools-scm only records the submodule gitlink, so graft the libcu++ headers needed for wheel-from-sdist builds".That no longer holds with setuptools-scm 10. Its git file finder lists the files of a submodule that is checked out. Building the
cuda.coresdist from a checkout with the submodule initialized, whichcuda.coreneeds in order to compile, therefore packs the entire CCCL tree into the sdist: cub, thrust, cudax, docs, tests, and so on.cuda.corecompiles against none of that.Measured with
python setup.py sdistincuda_core/, with the submodule initialized, swapping only.gitattributes:mainThat is 83% smaller, and nothing outside
third_party/cccl/changes. The same finder also walks the submodule on everyegg_inforun (twice per run), which slows down each wheel build as a side effect. On my workstationegg_infoforcuda_coregoes from about 600 ms to about 190 ms.Change
Mark the submodule
export-ignorein.gitattributes. The setuptools-scm finder honors the attribute for submodules and skips the path. TheMANIFEST.ingraft is matched on the file system, so it still adds whatcuda.coreneeds. With this change, theMANIFEST.incomment above holds again.What stays in the sdist from CCCL: all 1,584 files of
libcudacxx/include,third_party/cccl/LICENSEandthird_party/cccl/libcudacxx/LICENSE.TXT. What goes away is everything thatcuda.coredoes not compile against, including the license files of the components that are no longer shipped (for examplecub/LICENSE.TXTandcudax/LICENSE.TXT).Verification
mainand this branch only by files belowthird_party/cccl/. No file is added, and nolibcudacxx/includeheader is lost.build_hooks._get_cccl_include_dirs()(libcudacxx/include/cuda/tmais present).git check-attr export-ignore -- cuda_core/third_party/ccclreportsset..gitattributesmakescompute_ci_plan.pyselect the full CI, sotest-sdist-linuxandtest-sdist-windowsrun. They build thecuda.coresdist and then a wheel from it.Checklist
test-sdist-*jobs build thecuda.coresdist and a wheel from it.)