Skip to content

[DO NOT MERGE] Reproduce flaky TestCuVSRandomizedHNSWVectorSearch - #2819

Closed
bdice wants to merge 3 commits into
NVIDIA:mainfrom
bdice:flaky-lucene-randomized-hnsw
Closed

bdice wants to merge 3 commits into
NVIDIA:mainfrom
bdice:flaky-lucene-randomized-hnsw

Conversation

@bdice

@bdice bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Reproducer for the TestCuVSRandomizedHNSWVectorSearch.testVectorSearch flake ("Result returned was not in topK*3") found by the stress run in #2793. Workstream O of #2783.

CI is reduced to the cpp, java and lucene builds plus 4 lucene test jobs (2x amd64 12.9, 1x amd64 13.x, 1x arm64 12.9). Each job replays the failing seed, runs a recall sweep that compares the accelerated format against Lucene's CPU HNSW on the same segments, and loops the test with random seeds.

@copy-pr-bot

copy-pr-bot Bot commented Oct 10, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@bdice

bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test d1ee012

@bdice

bdice commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 8258a94

@bdice

bdice commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 50b16960e4b3f1ef9dfdac8aff2dd40a4cabda10

@bdice

bdice commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 50b1696

@bdice

bdice commented Oct 11, 2026

Copy link
Copy Markdown
Contributor Author

Closing: this was a reproducer/diagnostic PR for workstream O of #2783. The fix is in #2824.

@bdice bdice closed this Oct 11, 2026
rapids-bot Bot pushed a commit that referenced this pull request Oct 14, 2026
`TestCuVSRandomizedHNSWVectorSearch.testVectorSearch` required every hit of one query to be among the closest `3 * topK` vectors. The search is approximate: on this test's data (uniform random vectors with up to 2048 dimensions), a topK = 1 to 3 search sometimes returns a valid neighbor just outside that window, typically about 1% farther away than the nearest neighbor.

The test now checks all the queries and allows 2 such misses in total. It keeps hard checks that every result is a distinct document with a vector, scored by its own vector, in score order, with `min(topK, vectors)` results. This reuses the `assertSearchFindsNearestNeighbors` helper from #2775, extended for indexes where some documents have no vector. `TOP_K_LIMIT` is raised to 128, since the search does not fail beyond 64.

Measurements and the reproducer are in #2819. Part of #2783.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Igor Motov (https://github.com/imotov)

URL: #2824
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant