Repository navigation
Conversation
github-actions
Bot
added
A-index
Specify an opt-in plane-row layout for IVF_RQ auxiliary files: one fixed-width row column per plane (__rq_sign_rows, __rq_bounds_rows, __rq_high_rows and __rq_low_rows for layered indexes, __rq_rows for native ones) that packs each row's codes and factors little-endian, so a whole plane or a run of rows is one contiguous byte range. The layout is recorded as row_layout in the RabitQ storage metadata and in the new RabitQuantization.row_layout proto field, and plane-row segments are committed at index version 3. Every writer still produces the column layout, so the struct literals set row_layout to COLUMNS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mension Review F7-2. Optimize merges plane-row segments and keeps them plane rows at version 3; only the distributed merger rejects plane-row inputs, a single one included. Merged segments must share both the row layout and the layered flag. Define d as the rotated dimension, which older matrix-rotation indexes take from the rotation matrix when code_dim is 0, and note that readers reject null packed items because Lance file schemas do not keep item nullability. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BubbleCal
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
|
Important Format specification voteThis PR modifies the Lance format specification, so it requires 3 binding +1 votes from PMC members (excluding the proposer), at least one of them on the latest commit, and a minimum 72-hour voting period, weekends excluded, before it can merge. Vote by approving this PR (+1) or requesting changes (−1, a veto). See the voting process. Approvals carry over across pushes, so a rebase or a typo fix does not send everyone back to re-vote. Whoever approves the latest commit is vouching that nothing substantive has changed since the earlier approvals; if something has, ask for fresh votes. Status: ❌ Blocked — 0 of 3 required approvals
Updated automatically by the format-spec vote gate, which re-checks every 15 minutes — just voted? Re-check now (press Run workflow; leave the input blank to re-check every open format PR). A PMC member may apply the |
![lance-gatekeeper[bot]](https://avatars.githubusercontent.com/in/4433141?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Gate recommendation: approve.
Shared full-width quantization makes the stored planes consistent prefixes without introducing a second encoding, and existing default writes remain unchanged. This format contract can proceed separately from runtime recall/performance acceptance in #9490; optional plane-row reader/writer support belongs to #9837.
Please mark this PR with the breaking-change label.
lance-gatekeeper
Bot
added
the
K-approved
This branch has not been deployed
1 participant
Specify opt-in layered RaBitQ prefix planes for every supported width from 3 through 9 bits, plus the optional plane-row representation. Layered storage defaults to false; the existing column layout remains the default.
The high/low splits are 1+1, 2+1, 2+2, 4+1, 4+2, 4+3 and 4+4, with one additional sign bit. Full queries preserve native full-width encoding and pruning semantics. The proposal defines required metadata, per-level factors, auxiliary files, fixed-width plane rows and consistent layouts across merged segments.
Implementation is reviewed in #9490. This remains a draft proposal; the format vote is separate from implementation and performance acceptance.
Validation
Documentation-only update. All Rust, protobuf, manifests and lockfiles match the previously validated proposal exactly. Diff whitespace check passed; width table matches the tested core mapping. The proposal remains draft and unadopted.
Tracking
OSS-2357