Skip to content

fix: Attest bounded paired tier rebuild sources - #823

Draft
buildingjoshbetter wants to merge 1 commit into
fix/tier-rebuild-oom-publish-297-v1from
fix/tier-source-adapter-publish-756-v1
Draft

buildingjoshbetter wants to merge 1 commit into
fix/tier-rebuild-oom-publish-297-v1from
fix/tier-source-adapter-publish-756-v1

Conversation

@buildingjoshbetter

Copy link
Copy Markdown
Owner

Tier rebuild progress currently relies on whole-source lists and registry/MAX-row heuristics. This prerequisite adds a model-free adapter for bounded source pages and certified completion/separation pairs. It publishes both vector blobs, the consumed-input cursor and a chained receipt atomically, retaining the uncommitted page suffix after a reduced batch.

Restart planning verifies exact pair bytes and source coverage in bounded pages. Malformed manifests, changed rows or schemas, TEMP shadows, and unreceipted existing contents are refused. Initialization requires an already empty inactive pair and never clears serving tables. Completion distinguishes the captured range from the current source snapshot.

Stacked on #821. Related to #756 and #795; neither issue is closed. No production manager/worker routing changes in this PR. Existing models, rerankers and retrieval policy remain unchanged.

Limits are explicit: restart verification scans O(N) rows in one read snapshot and may retain WAL; the conservative connection/data-version fence requires replanning even after unrelated writes. The caller must establish inactive-pair ownership, target identity/schema, tracking and database identity. Receipts do not protect against a writer that forges both data and metadata. No native memory reduction or whole-process memory bound is claimed.

Validation:

  • Sequential offline GPUBox installed-wheel builds and full suites passed. Publication: 2,877 tests and 1,734 subtests. Combined with certified target preparation: 3,137 tests and 2,696 subtests. Both had 17 skips and 3 deselections, real sqlite-vec enabled, no timeout and no OOM kill.
  • 26 independent probes passed. Coverage includes paired/metadata/commit failures, acknowledgement loss, uncertain rollback, negative/zero IDs, retained suffixes, corrections/deletions/lower-ID inserts, receipt/vector tampering and source/schema/connection fences.
  • Review reproduced integer/real page substitution because Python considers 1 and 1.0 equal. Exact SQLite types and IEEE-754 values now guard pending pages; signed-zero and boolean substitutions also fail. Failed evidence is preserved. Existing production dependencies and test assertions are unchanged.
  • Ruff, Python 3.10 parsing, exact three-file publication mapping, source-hash, privacy and public commit-identity checks passed.

Full log SHA256: publication ace26783d6332258020d0840f3dce32f272cdd8d76a3f24431387fff53ff803d; combined 38ebaa315fc2174be52f85047a843fd52f5ceb03bda8981e1b268f65d83464fc.

Draft only. Nothing is merged or released. The original Mac memory incident remains unproven.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant