Skip to content

Reject non-finite brute-force search distances - #690

Open
shoaib0657 wants to merge 1 commit into
nmslib:developfrom
shoaib0657:fix/bf-l2-overflow
Open

shoaib0657 wants to merge 1 commit into
nmslib:developfrom
shoaib0657:fix/bf-l2-overflow

Conversation

@shoaib0657

Copy link
Copy Markdown

Addresses #686.

When squared L2 distances overflow float32, BFIndex can return the wrong neighbor order. In the reported example, both distances become infinity and the labels break the tie.

This reports an error for non-finite float/double distances before they reach the brute-force result heap. The C++ nonthrowing API returns a Status; Python and the throwing C++ API raise a runtime error. Labels excluded by a filter do not cause an error, and k=0 returns empty results without evaluating distances. Other distance types keep their existing behavior.

The check uses exponent bits because fast-math can optimize away std::isfinite. BFIndex query buffers now retain temporary ownership so failed queries free both arrays.

Tests

The overflow regression fails on upstream and passes with this change. Tests cover finite controls, filters, float/double infinity and NaN, zero-k, and parallel L2/cosine errors followed by successful queries.

Local Ubuntu WSL2 checks:

  • 31 Python tests and all five examples passed.
  • 17 C++ tests passed with exceptions ON and again with OFF.
  • The new regression passed ASan/UBSan with leak detection, scalar GCC/Clang fast-math builds, and forced SSE/AVX/AVX-512 Python runs.

@shoaib0657
shoaib0657 marked this pull request as ready for review October 2, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant