Skip to content

fix: Preserve exact source values across legacy rebuilds - #805

Draft
buildingjoshbetter wants to merge 1 commit into
fix/rebuild-source-streaming-756from
fix/rebuild-value-exactness-756
Draft

buildingjoshbetter wants to merge 1 commit into
fix/rebuild-source-streaming-756from
fix/rebuild-value-exactness-756

Conversation

@buildingjoshbetter

Copy link
Copy Markdown
Owner

Scope

Follow-up for #756, stacked on #802. Prevent legacy-schema source changes from escaping the streamed rebuild publication fence. This does not activate tier switching or close the broader issue.

Reproduced defect

SQLite formats REAL values as text when casting them to BLOB. Adjacent representable numbers can therefore have identical casts while producing different separation input strings. The previous byte/type predicate could allow a rebuild to publish vectors after such a source edit. SQL numeric equality also cannot distinguish retained positive and negative REAL zero.

Changes

  • Supplement the existing storage-type and byte comparisons with explicit BINARY value comparison.
  • Conservatively invalidate covered REAL-zero source assignments, including same-zero writes.
  • For ordinary source columns, restrict the update trigger to covered fields and unshadowed rowid aliases so unrelated derived-only updates remain clean.
  • Keep all-update tracking for generated source columns and existing conservative UNIQUE schemas.
  • Repair changed trigger definitions through the existing writer-owned epoch reset. Earlier manifests cannot resume across that repair.

Validation

  • 14 new focused tests passed, including 529 storage-value pairs, collation/type controls, rowid aliases, generated fields, trigger repair and caller rollback.
  • Independent review reproduced six incorrect successful publications with the old builder. The corrected builder rejected all six before vector publication and retained an incomplete zero-progress manifest.
  • Root independently reran the focused suite and configured Ruff gate. Python 3.10 syntax, whitespace, staged-content and commit-identity scans passed.
  • Exact-source GPUBox suite with actual sqlite-vec enabled: 2,411 passed, 22 skipped, 3 deselected; 1,058 subtests passed.
  • Full-suite log SHA256: aca318473f556e269b99960a18512c5b65dfeb9a3930ab1489cffa3e9116eaad.
  • Hosted checks are pending.

Limits

The zero predicate deliberately favors safe invalidation over retaining a certificate for a same-zero source assignment. Generated fields and custom UNIQUE schemas retain conservative behavior. Raw numeric content is not claimed as a supported embedding input; the publication regressions use valid separation strings.

Canonical maintenance/FTS comparison and untracked schema-interval trust are separate follow-ups. No native performance, whole-process memory or MPS/thermal improvement is established here. Models, rerankers, precision, dimensions and retrieval limits are unchanged. Test inputs are generated; no personal memories, transcripts or credentials are included. No main merge, release or issue closure.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant