Skip to content

perf(dv): avoid dense expansion for sparse masks - #1824

Merged
zeroshade merged 3 commits into
apache:mainfrom
fallintoplace:perf/dv-sparse-mask
Aug 26, 2026
Merged

perf(dv): avoid dense expansion for sparse masks#1824
zeroshade merged 3 commits into
apache:mainfrom
fallintoplace:perf/dv-sparse-mask

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Clear sparse deletion positions directly in the keep mask.
  • Keep the dense word path for dense and run-heavy vectors.
  • Skip out-of-range buckets without expanding them.
  • Add sparse and dense benchmarks plus a high-position correctness case.

Benchmark

Apple M1 Pro, arm64. The mask covers 1 MiB of rows:

Case Before After Result
10 sparse deletes 95.2 us, 254 KB, 3 allocs 25.0 us, 131 KB, 2 allocs 3.8x faster, 48% fewer bytes
1,000 sparse deletes 108.1 us, 262 KB, 3 allocs 27.5 us, 131 KB, 2 allocs 3.9x faster, 50% fewer bytes
Dense range 115.0 us, 262 KB, 3 allocs 112.4 us, 262 KB, 3 allocs essentially unchanged
Out-of-range delete 208.3 us, 2.1 MB, 3 allocs 126.6 ns, 120 B, 2 allocs no dense temporary

Tests

  • go test ./...
  • go vet .

@zeroshade

Copy link
Copy Markdown
Member

Since this is a draft, i'll hold off on further review until it's marked ready and the tests are fixed

@fallintoplace
fallintoplace marked this pull request as ready for review August 21, 2026 22:30

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zeroshade
zeroshade merged commit ce8563f into apache:main Aug 26, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants