fix: Add cancellable style maintenance with verified coverage

docs/mcp-tools.md

@@ -93,6 +93,31 @@ No parameters.
 
 **Returns:** version, tier, message count, health status, and capabilities.
 
+`health.clustering` reports dependency availability, the last maintenance outcome,
+source freshness, coverage and cluster count. Zero clusters can be a successful
+empty or all-noise result. A missing dependency is reported separately.
+
+All health entries retain `status: "ok" | "degraded"`. Clustering also has a
+`state`: `unknown`, `pending`, `deferred`, `degraded` or `ready`. A disconnected
+database is unknown; busy foreground/model ownership is deferred. Pending
+caller writes and unverified vector-generation coverage cannot report ready.
+`last_error`, `dependency_error` and `process_error` contain categories only.
+The last process failure clears after committed successful clustering work.
+Success pending a caller's outer commit preserves that evidence; a later
+committed maintenance pass can clear it.
+
+Clustering remains optional. Installation guidance is included only when the
+HDBSCAN module is confirmed missing; a failed native import alone does not
+produce a missing-package recommendation.
+
+## truememory_status
+
+Inspect rebuild progress and subsystem health. `status_id` is optional and
+defaults to `0`, the most recent tier-switch rebuild. Clustering appears under
+`degradation.clustering`, with the same fields as stats. Its inspection uses an
+existing connection when available and never constructs a memory instance,
+opens a database, loads a native package/model, or starts maintenance.
+
 ---
 
 ## truememory_configure

docs/resource-budgets.md

@@ -1082,3 +1082,207 @@ continue. This checkpoint adds no engine routing, automatic rebuild, model
 load, corpus scan on open, or user-facing maintenance status. Synthetic
 arithmetic and transaction checks do not establish retrieval quality or native
 performance acceptance.
+
+
+### Foreground model admission before writer ownership
+
+Owned foreground add and incremental vector publication now acquire the model
+identity fence before opening the SQLite write transaction. Add first owns its
+engine write lock, so another mutation on that same engine cannot hold the
+engine lock while waiting for a model fence held by the queued add. The model
+fence validates the captured generation and remains held through commit or
+rollback. There is one model acquisition, with no nested acquisition of the
+nonreentrant model lock.
+
+This permits an owned foreground call to wait for a concurrent cluster source
+snapshot without holding SQLite's writer or failing a valid vector publication.
+Model loading and inference finish before owned writer admission. A model
+generation change during loading, encoding or admission still rejects stale
+publication; add and update propagate those typed errors instead of silently
+falling back to a source-only write. Update retains its existing distinction:
+capture failure precedes its source write, while a later vector-publication
+failure can occur after the source update committed.
+
+Caller transactions use nonblocking model capture and publication admission.
+Add and update classify the connection during model capture under their engine
+write lock, so another same-engine mutation's transaction cannot be mistaken
+for borrowed caller work. They release that lock before encoding.
+Capture accepts only a cached model observed together with its identity under
+the model lock. Busy ownership or an unloaded model produces a typed retry
+error before any additional source write; the caller's transaction remains
+intact. Cached-model inference can still run inside a caller-held transaction.
+The publication fence protects the nested savepoint release, not a later outer
+commit controlled by that caller. Streamed rebuilds retain nonblocking model
+fences after writer admission and use the same cached-only borrowed capture.
+
+SQLite and event regressions cover actual clustering snapshot contention,
+same-engine mutation ordering, stale generations, borrowed sentinels, atomic
+add publication, and model release after admission/publication/commit failures.
+This checkpoint changes admission ordering rather than bounding wait duration:
+cluster snapshots may still hold the model lock while scanning the corpus.
+Native integration and latency remain separate acceptance checks.
+
+### Engine maintenance routing
+
+Engine startup, committed mutations and later safe API boundaries now use the
+same durable eight-layer planner. The default append trigger remains
+`insert_count - attempted_insert_count >= 25`; 24 committed appends do not
+qualify and the 25th does, including across engine instances and reopen.
+`TRUEMEMORY_AUTO_CONSOLIDATE_EVERY` retains its positive-integer override.
+Initial historical or empty input attempts once. Corrections, source epochs and
+dependency changes follow their separate eligibility rules. A successful empty
+cluster result no longer starts every layer again on each open. Source edits
+committed through another handle become visible at the next safe API boundary;
+there is no idle polling timer. Uncommitted caller changes do not schedule work.
+
+Foreground scheduling reads eight checkpoint records, source counters and small
+dependency metadata. It does not scan source/vector rows, import native
+frameworks, wait for the model lock or perform consolidation in `add()`. The
+foreground write lock is acquired nonblocking for this probe. Package versions
+are cached in three records and refreshed on connection setup and worker/manual
+setup. One extension-failure record is bound to the probed model/table identity
+and version snapshot. Actual worker extension loading supplies that evidence;
+metadata alone is not proof of availability. Epoch checks prevent a late worker
+from overwriting newer shared capability evidence, while that worker still uses
+its own actual failure for its current attempt. A transient model-busy probe
+cannot erase an existing extension-failure baseline.
+
+Each canonical file has at most one active owner and one pending notification
+generation. A notification stores only a generation and threshold, not an
+engine, caller connection or source content. Multiple foreground notifications
+replace that single pending slot. A same-process owner's release services an
+existing pending notification; it creates no notification of its own. A pending
+coordinator behind a local owner is retained only until that exact owner's
+release or cancellation, so closing the last engine does not lose the wake.
+Cross-process contention reports busy/pending and needs a later API boundary;
+this checkpoint has no automatic cross-process release notification. Deferred
+model ownership and failed attempts do not schedule their own retry loops.
+
+Workers create and close their own file connection. `engine.close()` detaches
+and closes only the foreground connection; it does not cancel or join a shared
+worker. Cancellation clears queued notifications but retains ownership until
+active computation reaches a cooperative boundary and its connection closes.
+Thread-start failure closes an unclaimed owner descriptor; interruption after
+the target started leaves teardown with that exact worker and cannot cancel a
+later successor. Child processes discard inherited owner descriptors, pending
+references and registry state. Daemon threads do not guarantee completion when
+the process exits; durable abandoned-attempt recovery remains necessary.
+
+Explicit `consolidate()` forces one pass with the same eight result keys plus
+the existing nonpersisted preference extraction result. A busy owner produces
+nine explicit `BUSY` entries and does not queue a hidden forced pass. A file
+call without an existing transaction uses a dedicated connection. An existing
+caller transaction uses the original connection's protected savepoints and
+reports `pending caller commit`; its writes and maintenance output remain
+rollbackable together. `:memory:` automatic work reports pending without opening
+an empty replacement database or creating a worker. Explicit manual maintenance
+uses the original in-memory handle. Caller-held writer and later outer-commit
+limitations from the borrowed-runner contract still apply.
+
+Successful manual summary/cluster publication restores the calling engine's
+corresponding capability flags, including successful empty output. Failed or
+unavailable layers never enable those flags or clear a previously usable
+capability. The existing automatic-consolidation opt-out remains until explicit
+manual repair. Cluster retrieval still requires live vector availability.
+`get_stats()` exposes bounded coordinator and per-layer outcome, freshness,
+coverage and categorical error state; it does not expose source text, paths or
+exception messages. Unverified vector-generation and legacy-contact coverage
+remain explicit limitations, rather than complete success claims.
+
+This routing checkpoint preserves builder algorithms and the A17 foreground
+vector-publication guards. Ingest database replacement, profile/style getter
+freshness and custom-schema source tracking remain separate work. Synthetic
+SQLite and event tests establish scheduling and ownership behavior; native
+latency, throughput and memory acceptance still require measurement.
+
+### Clustering health visibility (#720)
+
+Clustering health reuses the scheduler's dependency identity and persisted
+attempt/provenance row. It reports availability, outcome, freshness, coverage,
+output count and categorical errors separately. A successful empty or all-noise
+result remains `success_empty` with zero output; a missing dependency cannot
+become an empty success. Existing unverified vector-generation coverage remains
+visible and does not certify complete health.
+
+The engine accessor does not connect or initialize a coordinator. It attempts
+the foreground lock without waiting, then reads the existing connection under
+one read snapshot. Model ownership is also tried without waiting. Foreground
+and model contention report deferred observation; a disconnected engine reports
+unknown readiness. An existing caller transaction is preserved and reported as
+pending caller commit. Source mutations, pending work and failed attempts retain
+their distinct outcomes and prior published counts.
+
+Package inspection uses at most six top-level spec lookups and three distribution
+records per observation. It imports no native module and loads no model.
+Distribution metadata
+failure can report dependency unavailability, but installation guidance requires
+a confirmed absent HDBSCAN module. A cached worker extension failure is reused
+only while its dependency identity matches. Health reads do not change durable
+attempts, capability epochs or scheduling, and do not create a worker or database.
+
+Actual maintenance and bulk-ingest failures emit a categorical warning once per
+failure transition in the shared process coordinator. Repeated status reads and
+foreground notifications do not log warnings. Committed successful clustering,
+including empty output, clears that process failure. A borrowed savepoint release
+does not prove the caller's outer commit and preserves prior process failures
+through rollback. There is no outer-commit callback; a subsequent committed
+successful maintenance pass clears the retained evidence. A process failure is not a durable
+cross-process diagnostic; persisted scheduler outcomes remain independently
+visible. The optional clustering extra, eight persisted layers, nine manual
+result keys, builder algorithms and preference stage are unchanged.
+
+### Private style maintenance preparation (#747)
+
+Explicit style maintenance can enroll a private `style_vectors` checkpoint.
+Ordinary database initialization and spec construction do not enroll it or
+scan profiles. The eight public adapters and nine consolidation result keys
+remain unchanged; explicit enrollment adds a ninth physical checkpoint row.
+Fixed output INSERT/UPDATE/DELETE triggers invalidate only that private row
+and clear its attempt baseline. Trigger repair invalidates provenance using
+metadata without scanning messages or profiles. Caller rollback also rolls
+back enrollment, trigger installation and invalidation.
+
+Tracked rebuilds retain the existing 256-dimensional character n-gram
+arithmetic, sender ordering, case merging and ordinary-message filter. They
+capture source before computation, acquire writer ownership before checking
+source and future accumulator versions, then publish profiles, raw sums,
+hash version 2 and complete provenance in the runner's final transaction.
+Unsupported future formats are retained. Valid legacy getter output remains
+readable; getters never rebuild. Low-level standalone builders retain their
+existing SQLite-fenced contract and do not gain global coordination or the
+tracked future-format guard.
+
+Style work checks cooperative cancellation during capture, computation and
+publication. SQLite progress-handler installation additionally requires the
+explicit `connection_owned=True` opt-in and an owned runner transaction; a
+clean transaction alone does not establish ownership of the connection. That
+handler is disabled before rollback or commit. Other connections keep their
+caller's unknown handler, and borrowed transactions retain final commit
+ownership. Verified stale source, cancellation and SQLite writer
+contention defer without establishing a failure baseline after confirmed
+rollback; unexpected SQL failures remain failures. Failed publication retains
+the previous output and successful provenance.
+
+Initial enrollment and trigger repair can also defer on lock contention:
+the preparation boundary must establish that no writes started or confirm
+its owned rollback or caller savepoint rollback. Caller work stays pending;
+unknown errors and an unconfirmed transaction end do not become deferral.
+Read-only baseline probes before a running diagnostic, including the repeated
+layer probe and cancelled-result probe, can defer only for a known lock error
+with unchanged transaction ownership. Unavailable observations report an
+unknown count and unverified coverage; no attempt baseline is advanced.
+Lock contention before the running diagnostic leaves its prior checkpoint
+unchanged. If a writer instead blocks restoration after a deferred build, the
+running diagnostic remains persisted; a later owner can recover it after the
+writer releases. The deferred result does not claim that restoration succeeded.
+An owned SQLite interruption can attest automatic rollback only when this
+operation's installed callback returned cancellation and SQLite reported the
+interrupt code, or the exact legacy interrupt message on Python 3.10. The
+callback is cleared before cleanup. Merely observing an ended transaction
+does not establish rollback. Error categories also support Python 3.10's
+missing SQLite code attributes without hiding unexpected SQL errors.
+
+This checkpoint does not activate engine routing, automatic migration,
+incremental coverage, style status or coordinator style-only jobs. Those
+remaining migration steps are required before release. Synthetic transaction
+proofs do not establish native performance or retrieval quality.