Skip to content

Per-collection "required features" so older versions refuse collections they cannot maintain (full-text indexes) #187

Closed

Description

@CaffeinatedCoder

Follow-up from #177 (full-text search, #70). Must be merged before any release that ships full-text indexes.

Problem

A full-text index is stored as IndexType.FullText = 6 in a regular B-Tree. The page format version is unchanged, so an older BLite opens such a database and writes to it without knowing about the index. #177 detects and repairs some of the damage: foreign keys trigger a rebuild on open, and every hit is re-checked against its document. These cases it can't detect:

  • Older DynamicCollection: it doesn't load type-6 indexes, so it never writes postings for new or changed documents and leaves no foreign keys behind. Its PersistIndexMetadata also rewrites the index list without the full-text index, which drops the index from the metadata and leaks its pages.
  • Older typed version that only deletes: it leaves stale postings. The re-check hides them from results, but they still inflate document frequency, which skews BM25 scores until a rebuild.

Proposal

  • Add a per-collection required-features value (bit flags) to CollectionMetadata, persisted the same backward-compatible way as IndexMetadata.KeyFormat.
  • Set a FullText bit when a collection gets a full-text index, and clear it when the last one is dropped.
  • On open, a version that finds a bit it doesn't know refuses the collection for writing (or entirely), with an error naming the feature and the minimum version.
  • Keep the mechanism general so later features (new index types, key layouts) can reuse it.

Tests

  • Round-trip of the value; old metadata without it reads as "no features".
  • A collection with an unknown bit is refused, typed and dynamic, with a clear message.
  • Dropping the last full-text index clears the bit.
  • Compatibility: metadata written by the previous release still opens; metadata with the bit set is refused by a build that predates the bit (simulated).

Releases before this lands can't honour the flag, so the "don't open with BLite < 5.3 for writing" note in the release notes stays.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    • Status
      Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions