Index locked passages so context queries skip unpacking archives

[admin-raintree]

docpull query now reads a full-text index built at sync time instead of unpacking and scanning the whole archive on every call. Results are identical either way.

Changes

• docpull sync writes .docpull/search/<artifact-sha256>.sqlite, an SQLite FTS5 index of the passages query_context already produces. It is built from the same verified records, written to a temp file, and then moved into place. Building is best effort: a failure logs a warning and does not fail the sync.
• Queries still hash the pinned archive on every call, so a missing or tampered artifact fails closed as before.
• A query uses the index only when its schema version and stored lock facts match the current lock: artifact, run, project, and content hashes.
• The index is opened with mode=ro&immutable=1, so queries never write project files.
• A missing, corrupt, or stale index falls back to scanning the archive. Frozen installs from other artifact directories still scan.
• FTS only selects candidate passages. Exact term matching, scoring, deduplication, and budgeting are unchanged, so the CLI, SDK, and MCP payloads are unchanged. Unicode spellings using combining marks might rank differently; the code notes this.

Benchmark (synthetic project, 3,000 documents, 120k passages)

Query	Indexed	Scan
Rare term (1234)	0.007s	0.76s
Common terms (retry backoff)	2.6s	3.3s

The common-term query is dominated by the existing budget loop, which re-serializes the whole payload for every candidate passage. That loop is not changed here.

Checks

• New tests in tests/test_context_query.py:
  • indexed and scanned results are equal
  • a valid index answers without unpacking the artifact
  • a corrupt or stale index falls back to scanning
  • queries leave project files unchanged
• bun run validate:full passes: 1,054 Python tests pass and 18 are skipped. Lint, mypy, the JS SDK, and the reproducible build also pass.

🤖 Generated with Claude Code