Skip to content

fix: Report clustering availability and publication health - #810

Draft
buildingjoshbetter wants to merge 1 commit into
fix/maintenance-engine-routing-publish-753from
fix/clustering-health-publish-720
Draft

buildingjoshbetter wants to merge 1 commit into
fix/maintenance-engine-routing-publish-753from
fix/clustering-health-publish-720

Conversation

@buildingjoshbetter

Copy link
Copy Markdown
Owner

Scope

Truthful clustering availability and maintenance health for #720, stacked on #808. Optional clustering packaging is unchanged. This adds observations and categorical warnings; health reads do not open a database, load a model or schedule maintenance.

Changes

  • Add clustering to engine stats and MCP health/status while preserving the existing five health entries and their binary ok/degraded status contract.
  • Report availability, last attempted outcome, successful output count, source freshness, coverage, pending caller commit and bounded error categories separately. Empty success is distinct from missing dependencies, pending work and a failed attempt.
  • Inspect only top-level module availability, package metadata and bounded existing SQLite checkpoint/dependency state. Installed packages alone cannot certify usable native code or current vector coverage.
  • Give installation guidance only when the optional HDBSCAN package is actually absent. Busy foreground/model ownership returns a deferred observation without waiting or triggering work.
  • Emit a warning after an actual unavailable/failed clustering attempt, deduplicated per canonical database and failure transition. Health polling is silent. Warning text contains no exception message or source content.
  • Retain prior process failure evidence after a successful borrowed savepoint. Caller rollback cannot make failed clustering look recovered; a later committed successful maintenance pass clears that evidence.
  • Catch clustering inspection failures at the MCP boundary so the five existing health entries remain available. Failed owned inspection rollback is reported categorically and engine ownership is released.

Validation

  • 24 focused stdlib/SQLite tests passed, including missing packages, metadata failures, disconnected/busy state, pending/empty/failed/current generations, unverified coverage, committed recovery, caller rollback and warning deduplication.
  • The existing binary health-contract test body passed unchanged. Three original-production negative controls failed their intended assertions with zero setup errors.
  • Both borrowed-transaction regression cases (zero and nonzero output) failed before the correction, then passed through outer rollback and later committed recovery.
  • Integration-based candidate with actual sqlite-vec: 2,766 passed, 22 skipped, 3 deselected; 1,865 subtests passed. Log SHA256: 18e52b104aeff0e5145bb49e42b3012a699032a61df8795501d4e88e2de20df0.
  • Exact isolated publication branch with actual sqlite-vec: 2,650 passed, 22 skipped, 3 deselected; 1,251 subtests passed. Its execution-log hash is recorded in the maintenance checkpoint.
  • Configured Ruff, Python 3.10 syntax, whitespace, new-content privacy and commit-identity checks passed. Production Python and focused test bytes match the reviewed integration candidate. Hosted checks are pending.

Limits

There is no outer-commit callback. A prior process failure therefore remains conservatively visible until a subsequent committed successful maintenance pass. This does not certify arbitrary later caller commits, full vector-generation coverage or direct external database edits.

The existing bulk-ingestion result may still contain its original raw exception message; this draft does not claim to sanitize every pre-existing error payload. Models, rerankers, inference precision, dimensions and retrieval limits are unchanged. No latency, process-memory, MPS or thermal improvement is claimed. Only generated synthetic test data was used; no personal memories, transcripts or credentials. No main merge, release or issue closure.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant