Skip to content

[DO NOT MERGE] Reproduce flaky spectral clustering test - #2788

Closed
bdice wants to merge 5 commits into
NVIDIA:mainfrom
bdice:fix-flaky-spectral-clustering
Closed

bdice wants to merge 5 commits into
NVIDIA:mainfrom
bdice:fix-flaky-spectral-clustering

Conversation

@bdice

@bdice bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Reproducer for the flaky SpectralClusteringTestF.Result/7 failure on arm64 A100 (#2094). Tracking: #2783.

CI is reduced to the arm64 / CUDA 12 / A100 C++ test job, which repeats the spectral clustering tests and runs extra diagnostics.

@copy-pr-bot

copy-pr-bot Bot commented Oct 10, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@bdice

bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 13b8d0d

@bdice

bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 224cb6f

bdice added 2 commits October 10, 2026 16:14
The expanded L2 distance epilogues clamp round-off of self-distances to zero
when the two norms are equal and the squared distance is below an absolute
tolerance. For data with a small magnitude, this also zeroes the distances
between distinct points with equal norms. Spectral embeddings of graphs with
several connected components are such data, which made k-means (and spectral
clustering) assign points to the wrong clusters on GPUs that use the CUTLASS
fused 1-NN kernel.

Scale the tolerance with the norms when they are smaller than one.
@bdice

bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 9e373d5

@bdice

bdice commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

Closing: this was a reproducer/diagnostic PR for workstream A of #2783. The fix is in #2794.

@bdice bdice closed this Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant