Skip to content

feat(index): read and write optional RaBitQ plane-row files - #9837

Draft
BubbleCal wants to merge 4 commits into
yang/layered-rq-cache-bodiesfrom
yang/layered-rq-plane-rows
Draft

BubbleCal wants to merge 4 commits into
yang/layered-rq-cache-bodiesfrom
yang/layered-rq-plane-rows

Conversation

@BubbleCal

@BubbleCal BubbleCal commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Read, build, convert, append and remap optional plane-row IVF_RQ files for native and layered codes. Validate fixed-width rows, rotated dimensions and exact buffer sizes; reject merges that mix row layouts. Include the cache-weight and lazy-query counters used by the column/plane-row equivalence checks.

Dependencies

Stacked directly on #9836.

Integrated Lance main b3c8619886b9 and the updated 3–9-bit layered prerequisites. Standalone benchmark/POC tools remain excluded.

Preserve main's streaming row-ID remapping and fused RQ benchmark metadata while integrating the plane-row writer.

Validation

This PR tree passed formatting and full-workspace clippy (--all --tests --benches -- -D warnings, Rust 1.98.1, Linux x86_64). The integrated composition passed 670 BQ tests and 22 layered integration tests on x86_64, plus 22 layered and 111 lazy/resident integration tests on macOS arm64. Production and Rust test source equality was verified after the final dependency refresh; the only later source change adds Java test coverage.

No new performance measurement is claimed. CI is not awaited.

Tracking

OSS-2565, OSS-2354

@github-actions github-actions Bot added A-python Python bindings A-index Vector index, linalg, tokenizer A-java Java bindings + JNI A-deps Dependency updates enhancement New feature or request labels Oct 9, 2026
@BubbleCal
BubbleCal force-pushed the yang/layered-rq-plane-rows branch from c1a8b97 to f899069 Compare October 9, 2026 16:48
@BubbleCal
BubbleCal force-pushed the yang/layered-rq-cache-bodies branch 2 times, most recently from 90835ec to 75a9468 Compare October 9, 2026 18:55
@BubbleCal
BubbleCal force-pushed the yang/layered-rq-plane-rows branch from f899069 to 4399615 Compare October 9, 2026 18:55
@BubbleCal
BubbleCal force-pushed the yang/layered-rq-cache-bodies branch from c392524 to bc2cc33 Compare October 10, 2026 16:03
@BubbleCal
BubbleCal force-pushed the yang/layered-rq-plane-rows branch from 53ee247 to 156f013 Compare October 10, 2026 16:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-deps Dependency updates A-index Vector index, linalg, tokenizer A-java Java bindings + JNI A-python Python bindings enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant