Skip to content

Rank coherent knowledge, harden ingestion, simplify MCP - #3

Merged
sgathrid merged 8 commits into
mainfrom
feat/knowledge-coherence-mcp-contract
Jul 30, 2026
Merged

sgathrid merged 8 commits into
mainfrom
feat/knowledge-coherence-mcp-contract

Conversation

@sgathrid

Copy link
Copy Markdown
Owner

Summary

Makes Brian's knowledge engine rank what is actually about the query, refuse wiki updates that quietly worsen retrieval, and expose a shorter MCP surface agents can follow without path bookkeeping.

Three product outcomes:

  1. Search — two-term natural-language queries stop promoting pages that only match terms across unrelated fields.
  2. Ingestion — every update preview reports page/link deltas and fails closed if cold-start ranks regress.
  3. MCP / agent UX — shorter tool names, flexible page refs, digest-only apply, proactive grounding guidance.

Why these decisions

Decision Chose Rejected Why
Incoherent 2-term matches Demote + annotate provenance Hard-filter Keeps fallback recall when nothing coherent exists; still guarantees coherent hits rank first.
Conversational scaffolding (who is, our, …) Strip before scoring Leave query intact Generic possessives/pronouns diluted scores without adding retrieval signal.
Ingest change visibility Structured deltas on the result Log-only / git diff Agents and reviewers can verify what changed without reconstructing the graph.
Retrieval safety on update Cold-start rank check in preview Post-apply only Blocks silent quality loss before any write; returns needs_revision with diagnostics.
Source registry updates Derive from {{SOURCE_PATH}} provenance Manual double-entry Removes a common mechanical failure mode for agents updating pages.
MCP tool names Short verbs (query_knowledge, …) Keep *_company_* Less token noise; same semantics. Breaking rename — no dual aliases.
read_page input path / URI / title / [[wikilink]] / stem Paths only Graph edges from query hits are titles/wikilinks; agents should not invent paths.
Apply gate approval_digest only Keep deprecated confirmed One canonical preview → approve → apply path.
Agent querying posture Proactive when freshness/specificity/citation needs exceed context; else don't ritual-query Always-query or never-query Cuts empty tool loops without going silent on consequential answers.

Important

Breaking MCP rename. Clients must call query_knowledge, read_page, list_sources, inspect_source, and update_knowledge. The old *_company_* / apply_knowledge_update names and the confirmed flag are gone. CLI subcommands (wiki knowledge …) are unchanged.

Note

Live field-coherence holdouts are adapted to Brian's seed wiki (single overview page). Synthetic multi-page fixtures still lock the ranking rule itself. Private-corpus partner pages are intentionally not shipped here.

What changed

Search (wikicli/core/resolve.py)

  • Normalize curly apostrophes; strip conversational prefixes.
  • For exactly two content terms: require co-occurrence in one discovery field, an exact body phrase, or a primary title/name anchor.
  • Incoherent hits stay visible but demoted, with field provenance: … and incoherent evidence demoted.

Ingestion (wikicli/core/ingest.py)

  • Result carries created_pages, updated_pages, metadata_changes, links_added, links_removed.
  • Preview compares cold-start ranks before/after the candidate tree; rank drops → needs_revision + retrieval_regressions.
  • Source mappings reconcile from page provenance citations.

MCP + knowledge API

Old New
query_company_knowledge query_knowledge
read_company_page(path=…) read_page(ref=…)
list_company_sources list_sources
inspect_company_source inspect_source
apply_knowledge_update (+ optional confirmed) update_knowledge (digest-only)

Agent surfaces

  • Session-start hook and Gemini/wiki-context skills describe proactive grounding and abstention when the wiki does not support an answer.

Review guide

Prefer reviewing behavior and trade-offs, not only line diffs:

  1. Search — Does demotion (vs filter) match the product goal? Are holdout queries on the seed wiki enough signal for OSS?
  2. Ingest gates — Is blocking on cold-start rank drops the right severity, or should some regressions warn?
  3. MCP break — Is a clean rename acceptable without compatibility aliases?
  4. Agent instructions — Is "proactive but not ritualistic" specific enough to change tool-use for the better?

Test plan

  • pytest — 433 passed
  • Companion contract via wiki-sync-engine candidate validation
  • Dogfood MCP stdio + knowledge CLI adapter parity on Brian corpus
  • Field-coherence unit + seed holdouts
  • Manual: wiki knowledge query "What is Brian?" still tops Brian Overview
  • Manual: MCP client lists only the five new tools; read_page accepts [[Brian Overview]]
  • Manual: update preview without digest → ready/needs_revision; apply only with digest from that ready preview

Commits

  1. feat(search): demote incoherent two-term hits and strip query scaffolding
  2. feat(ingest): expose page deltas and block retrieval regressions
  3. feat(mcp): rename tools, drop confirmed, and accept flexible page refs
  4. feat(agents): teach proactive grounding without ritualistic querying
  5. test: align Brian companion suites with the new MCP contract
  6. chore: fold incidental formatting from the engine sync

sgathrid and others added 6 commits July 30, 2026 09:00
…ding

Require field coherence for two-term queries so scattered term matches cannot
outrank same-field phrases, body phrases, or title anchors. Normalize curly
apostrophes and strip conversational prefixes that diluted scoring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Return created/updated page and link/metadata deltas from knowledge updates,
reconcile sources.json from provenance citations, and fail previews that drop
cold-start benchmark ranks before any write.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expose query_knowledge/read_page/list_sources/inspect_source/update_knowledge.
read_page resolves paths, URIs, titles, wikilinks, and stems. Preview/apply is
digest-only with no deprecated confirmed flag.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Session-start hooks and wiki skills now direct agents to query when freshness,
specificity, or citation needs exceed injected context, and to abstain when
curated pages do not support an answer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update dogfood stdio and knowledge CLI adapters for renamed tools, read_page
refs, and Brian seed-corpus expectations so engine sync validation can pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep config/status formatting aligned with upstream engine style without
behavior changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sgathrid sgathrid added the enhancement New feature or request label Jul 30, 2026
sgathrid and others added 2 commits July 30, 2026 09:11
Clarify that wiki knowledge and the five MCP tools share one engine, document
flexible read refs in CLI help and agent surfaces, and add adapter parity for
source inspect, read --help, and rejected legacy update fields.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Migrated from KOS: rank regressions now force needs_revision, clear the
approval digest, and emit RETRIEVAL_REGRESSION diagnostics so preview
cannot be approved and apply cannot write a regressing update.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sgathrid
sgathrid merged commit e13677a into main Jul 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant