Fix pg_index per-column vectors to have one entry per key column

[zachmu]

Fixes pg_index per-column vectors (indoption, indcollation, indclass) to have one entry per key column, and populates indnkeyatts.

Fixes #3110.

[itoqa]

Commit: c0f57c6: 14 test cases ran, 13 passed ✅, 1 additional finding ⚠️.

Summary

The run broadly covered index metadata integrity across single-column, composite, primary, unique, partial, ordinary, and newly created indexes, including identity, column counts, ordering, filtering, and related metadata. It also exercised schema-qualified lookups, joined catalog views, refresh behavior, boundary conditions, and concurrent-read resilience, with the normal and edge-case behaviors generally remaining healthy.

Safe to merge — the only failure is a medium-severity, pre-existing concurrency defect involving catalog reads and cache collision handling, and it is not attributable to this PR. The change shows no regressions or new failures, so the unrelated server-panic finding is a flag for later rather than a merge blocker.

Tests run by Ito

Result	Severity	Type	Description
✅	—	General	One-column and two-column indexes report matching key and option counts. Each key has one default ordering value of zero.
✅	—	General	Single-column and composite indexes report matching total and key-column counts, and the counts stay correct through a relation join.
✅	—	General	Catalog joins kept each schema, table, index, and metadata vector together. Direct lookups returned the same rows for both similarly named tables.
✅	—	General	A newly created two-column index showed the same identity and key details in both catalog views. The key count and column mapping stayed stable after the catalog refresh.
✅	—	Catalog	Primary, unique, and partial indexes stayed linked to the right table. Their unique and primary markers, key mappings, and partial filter were all returned correctly.
✅	—	Catalog	Creating a regular index keeps the correct index and table links, maps to its one indexed column, and leaves the partial-index filter empty.
✅	—	Index	A two-column index reports two key positions and two default ordering values, both set to zero.
✅	—	Index	A single-column index reports one key, one default ordering value of zero, and an indoption length of one.
✅	—	Keys	A two-column index was created and its catalog row reported two total columns and two key columns.
✅	—	Keys	A single-column index reports one total attribute and one key attribute, as expected.
✅	—	Lookup	Looking up the primary index returned one matching record. Looking up the table returned both its primary and ordinary indexes with the expected IDs and column positions.
✅	—	Oid	A two-column index returned two zero values for both collation and operator-class metadata. Formatting the values produced 0,0 for each vector, as expected.
✅	—	Oid	Unnesting the index metadata returned two zero values for collation and two zero values for operator class, matching the two columns in the index.
⚠️		General	The expected result was two complete, equivalent catalog results. Instead, session A returned 60 rows for three index identities, session B returned zero rows, and the server reported an all-OIDs-have-been-taken panic while populating shared catalog state.

Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 Concurrent catalog reads can return no rows

• Severity: Medium
• Description: The expected result was two complete, equivalent catalog results. Instead, session A returned 60 rows for three index identities, session B returned zero rows, and the server reported an all-OIDs-have-been-taken panic while populating shared catalog state.
• Impact: When concurrent catalog reads hit the cache collision, one user may receive no index information and the server may stop that catalog operation. Other database data is not shown to be lost or corrupted.
• Steps to Reproduce:
  1. Create a table with several indexes.
  2. Start two local database sessions and repeatedly query pg_catalog.pg_index from both sessions, using both full scans and indexed lookups.
  3. Compare the rows returned by both sessions and inspect the server output if one session returns no rows.
  4. Force or encounter an OID checksum collision while the catalog cache is being populated; the retry path repeats the occupied OID and the server reaches the all-OIDs-have-been-taken panic.
• Stub / mock content: PostgreSQL authentication was disabled in the local test service so passwordless catalog queries could run. No application mocks or route interceptions were used, and this bypass does not affect OID allocation or catalog cache behavior.
• Code Analysis: The failure is supported independently of the unavailable browser re-run. In server/tables/pgcatalog/pg_index.go, cachePgIndexes builds each catalog index record and calls id.Cache().ToOID for both index and table IDs at lines 334-342. That path reaches core/id/cache.go. ToOID computes an initial CRC32 OID at lines 65-71, then allocates modifiedBytes and updates modifiedBytes[0] in the collision loop at lines 75-78. However, line 79 hashes underlyingBytes again instead of modifiedBytes, so every retry tests the same occupied checksum. The final fallback at lines 87-92 also checks the stale oid variable rather than candidate i, so it does not actually scan for a free OID. A collision can therefore fall through to panic("all OIDs have been taken") at line 95, aborting catalog cache construction and explaining the empty concurrent result. The smallest practical fix is to hash modifiedBytes at line 79 and use i for the cache lookup and assignment in the fallback loop. The PR diff does not modify core/id/cache.go or the ToOID call sites, so this is a pre-existing defect rather than a regression caused by the vector-shape changes.

Evidence Package

• text/BF-RACE-1/BF-RACE-1-concurrent-catalog-panic.txt — 3 KB

Tip

Reply with @itoqa to send us feedback on this test run.

[github-actions]

	Main	PR
Total	42090	42090
Successful	19290	19290
Failures	22800	22800
Partial Successes1	5459	5459

	Main	PR
Successful	45.8304%	45.8304%
Failures	54.1696%	54.1696%

Footnotes

1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct. ↩

[coffeegoddd]

@zachmu DOLT

read_tests	from_latency	to_latency	percent_change
covering_index_scan_postgres	2.48	2.43	-2.02
groupby_scan_postgres	75.82	75.82	0.0
index_join_postgres	2.18	2.18	0.0
index_join_scan_postgres	1.58	1.58	0.0
index_scan_postgres	484.44	484.44	0.0
oltp_point_select	0.36	0.36	0.0
oltp_read_only	6.32	6.32	0.0
select_random_points	0.7	0.7	0.0
select_random_ranges	1.01	1.01	0.0
table_scan_postgres	484.44	475.79	-1.79
types_table_scan_postgres	1213.57	1213.57	0.0

write_tests	from_latency	to_latency	percent_change
oltp_delete_insert_postgres	6.67	6.67	0.0
oltp_insert	3.3	3.3	0.0
oltp_read_write	13.22	13.22	0.0
oltp_update_index	3.55	3.55	0.0
oltp_update_non_index	3.25	3.25	0.0
oltp_write_only	6.91	6.91	0.0
types_delete_insert_postgres	7.17	7.17	0.0

[itoqa]

Ito Diff Report — c0f57c6 → 57efb93: 7 test cases ran, 1 fixed ✅, 6 passing ✅.

Diff Summary

The change is covered across normal index creation and lookup behavior, composite and single-column metadata, filtering, identity, and key/vector consistency. Coverage also includes boundary conditions and concurrent catalog changes, with behavior remaining consistent across these database operations.

Safe to merge — the exercised behaviors show no PR-attributable regressions or new failures, and the concurrency and metadata checks remain healthy. Several previously passing areas were not exercised in this run, so they remain a coverage caveat rather than a merge blocker.

Tests run by Ito

Result	State	Severity	Type	Description
✅	❌->✅ Fixed	—	General	Two database sessions read index information while ten indexes were created and removed. All 24 reads succeeded, and every returned row kept the correct index and table information.
✅	Passing	—	General	The partial-index check could not finish because the local database target stopped during setup. Source review found that direct catalog rows and the indexed catalog view use the same partial-index predicate, so this run does not confirm a product defect.
✅	Passing	—	General	Concurrent readers kept each index's key metadata consistent while indexes were created and removed. Single-column and two-column indexes returned matching vector lengths, and later lookups remained correct.
✅	Passing	—	Collation	A two-column unique index exposed two entries in both OID vectors. Each entry was zero, and both vectors worked with formatting and unnesting queries.
✅	Passing	—	Index	The primary index and the unique value index both point to the expected table. Their primary and unique flags, along with their column mappings, are correct.
✅	Passing	—	Key	A two-column index reports two total attributes and two key attributes in the catalog.
✅	Passing	—	Option	The composite index showed two key positions and two matching option values. Both values were zero, and the paired values could be read without a length mismatch.
⏸️	Skipped	—	General	One-column and two-column indexes report matching key and option counts. Each key has one default ordering value of zero.
⏸️	Skipped	—	General	Single-column and composite indexes report matching total and key-column counts, and the counts stay correct through a relation join.
⏸️	Skipped	—	General	Catalog joins kept each schema, table, index, and metadata vector together. Direct lookups returned the same rows for both similarly named tables.
⏸️	Skipped	—	General	A newly created two-column index showed the same identity and key details in both catalog views. The key count and column mapping stayed stable after the catalog refresh.
⏸️	Skipped	—	Catalog	Primary, unique, and partial indexes stayed linked to the right table. Their unique and primary markers, key mappings, and partial filter were all returned correctly.
⏸️	Skipped	—	Catalog	Creating a regular index keeps the correct index and table links, maps to its one indexed column, and leaves the partial-index filter empty.
⏸️	Skipped	—	Index	A single-column index reports one key, one default ordering value of zero, and an indoption length of one.
⏸️	Skipped	—	Keys	A two-column index was created and its catalog row reported two total columns and two key columns.
⏸️	Skipped	—	Keys	A single-column index reports one total attribute and one key attribute, as expected.
⏸️	Skipped	—	Lookup	Looking up the primary index returned one matching record. Looking up the table returned both its primary and ordinary indexes with the expected IDs and column positions.
⏸️	Skipped	—	Oid	A two-column index returned two zero values for both collation and operator-class metadata. Formatting the values produced 0,0 for each vector, as expected.
⏸️	Skipped	—	Oid	Unnesting the index metadata returned two zero values for collation and two zero values for operator class, matching the two columns in the index.

Tip

Reply with @itoqa to send us feedback on this test run.