Skip to content

fix: Adopt guarded tier authority across public operations - #828

Draft
buildingjoshbetter wants to merge 11 commits into
fix/tier-runtime-writer-publish-v2from
fix/tier-public-publish-v1
Draft

buildingjoshbetter wants to merge 11 commits into
fix/tier-runtime-writer-publish-v2from
fix/tier-public-publish-v1

Conversation

@buildingjoshbetter

@buildingjoshbetter buildingjoshbetter commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Public tier changes use bounded rebuild jobs and committed database authority across configure/status, Engine, MCP, CLI, maintenance and ingestion. Partially activated changes report requested, served and pending state separately. Stacked on draft #827; refs #795, #756, #753 and #684.

Changes

  • Add guarded policy journals and config projection. Same-space changes preserve the embedding space; absent/empty first-run setup selects lazy model identities without constructing models or claiming readiness.
  • Hold serving authority through public operations and each ingestion fact's encoding, duplicate decision and storage. Reject stale writers and propagate typed authority failures. Controlled vector clustering remains explicitly deferred.
  • Complete cold initialization before capturing vector tables. Validate bounded initialization evidence for reconnects and live automatic-worker handoffs. Eligible public opens share one 10-second owner-admission budget. Already-running native calls are not hard-preempted. Sibling MCP connection openings serialize before concurrent searches; maintenance starts after the outer lease exits.
  • Let foreground add/update/embed_single publish matching metadata within their model/writer fences. General migrations and metadata changes retain maintenance ownership. Preserve borrowed-connection FTS, invalid-input error precedence and ordinary optional legacy Deep dependency fallback; typed identity/admission/cancellation failures still propagate.
  • Add explicit interrupted-job cancellation with matching positive status ID and lowercase 32-character job ID. It does not take over active owners, delete source/vector rows or silently restart work. Retry remains explicit.

The reviewed platform corrections remain in the stack: portable fixture cleanup and exact supplied-path expectations, plus bounded FTS-only automatic reconnect evidence when SQLite's extension-loading API is unavailable. A callable API still requires vector initialization. Original Python 3.10, Windows and macOS failure evidence remains preserved.

Cached-handle reconnect correction

Engine annotates a maintenance refusal only after proving that the exact cached handle is closed or has a qualifying I/O failure. The error identifies the exact Engine and failed handle. Public admission rechecks both identities before and after waiting, uses existing strict automatic-owner receipt validation and shares the original 10,000 ms budget. A replaced handle or stale error cannot grant waiting. Strict maintenance admission source is unchanged; no waits are added inside admission.

Current-thread writer refusal runs before the cached opener, avoiding a nonreentrant self-acquire. Healthy unowned reuse, nested operations and borrowed SQL with an unowned Engine writer remain covered. Manual owners, missing receipts, serving/already-admitted callers, replacement handles and owned writers retain refusal.

The historical macOS closed-handle failure and old #829 Windows I/O reconnect failure reached maintenance-owner refusal, but their logs did not identify the actual owner or receipt. Duplicate hosted jobs passed. Separate deterministic actual-function controls reproduce refusal under a verified automatic owner and validate this narrow correction. Historical owner identity remains unproven; all original logs remain preserved.

Regression evidence and preserved failures

The reconnect suite passes 26 regression methods, including actual lock/read controls and three explicit gc.isenabled() assertions. Independent reconnect controls pass 42/42. All 101 public-boundary methods and 550 unittest assertion/fail call AST nodes remain unchanged through fixture binding corrections; the complete memory-only run passes 101/101. Original 21 missing-import errors and nine inherited setup errors remain preserved. Fixtures bind the actual production exception, actual storage budget and a strict unexpected-owner-wait guard.

The original combined full run at 5cd23db000fd2a7c98e93395be33cf8638641db1 failed ten MCP fixture tests while 3,725 tests and 3,223 subtests passed. Nine failures were missing time in the actual opener's AST namespace; one parallel-wait assertion was the resulting cascade. Original source, container and full log remain preserved; log SHA256 2a706b32240afa808f582e1751beeea1a24302f1a6f86f3eb3eddec5fa85befd.

The final append changes only the MCP fixture. It binds actual stdlib time, actual writer-lock/maintenance-error class ASTs, the actual storage budget and explicit inert import routes. Unexpected imports and owner waiting raise. All 11 test methods and 52 unittest/Mock assertion/fail call AST nodes remain unchanged; the safe suite passes 11/11 without SQLite connects, and independently frozen MCP controls pass 35/35. Production, model choices, rerankers, dimensions, retrieval formulas, limits and GC behavior remain unchanged by this fixture repair. These AST inventories are source counts, not runtime assertion execution totals.

Exact-source validation

Prepared publication head: 5873e5a4978f37bcd31772d6840717fb217df713. Proposed remote refresh destination: fix/tier-public-publish-v1. Base: draft #827, fix/tier-runtime-writer-publish-v2 at 48217a73f10ad7fa80029ad800660d76ef912de7. The complete PR delta is 55 paths: 20 production, 34 tests and one documentation file.

Gate Exact source Passing tests / subtests Runner elapsed Full-log SHA256
Expanded focused 3a9360b1ba6bf210a1b54e05287117923fbbaae5 833 / 633 39.36 s e1c203f112a3dedc0411874785f00a528c6bd3d34809657f1903df46da7ee0df
Combined full 3a9360b1ba6bf210a1b54e05287117923fbbaae5 3,735 / 3,223 412.643 s a750f39645d1c866e5e68cddaa9ad49fe3616f3500be6180dadbbb4344dbeaed
Exact Unit C full 5873e5a4978f37bcd31772d6840717fb217df713 3,531 / 2,396 411.415 s 13376c51a4174b4b2dc8fda569911d0e625a2cba9912e4bfe8d75bba3f0bd3bc

Both full gates exited 0 with 17 skips and three network deselections, no timeout/OOM, and matching actual source/build/log hashes. Tests ran sequentially in offline installed-wheel images with eight CPUs, 16 GiB, read-only source/models/fixtures and enabled native SQLite flags. No GC-disabling override was used; three selected regression assertions observed GC enabled at their checkpoints. Full-run runtime GC state was not recorded. Installation/build evidence does not measure every test's installed import origin; that stronger measurement remains absent.

Receipts: .work/reports/native-integrations-focused-validation-v6.json, .work/reports/native-integrations-combined-full-validation-v2.json and .work/reports/tier-public-publication-validation-v4.json, with corresponding actual container-config receipts. New-head hosted CI remains pending. Validated private integration remains 7bd70e9fa4812e9925b86652dbe5c3a5c92bafe1; adoption of the prepared combination remains separate.

Serving admission is process-local and arbitrary raw SQL is not fenced. Synthetic model/native SQLite controls do not establish model or reranker quality, native host/model-visible context acceptance or the original 26.24 GB Mac memory fix. Complete-archive Deep Search and durable source citations remain future scope. Broader issue acceptance stays open. Draft only; no merge or release.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant