Repository navigation
Conversation
Swap only the live retrieve() step from SQLite FTS5 MATCH to a Haystack 2.x InMemoryBM25Retriever Pipeline over latest captured versions. Citation validation, claims, approval, and the recorded Pages case stay unchanged. Co-authored-by: Jordi Lluis <coder058@users.noreply.github.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 58b7fe2fbf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| + where + " ORDER BY rank, c.id LIMIT 6", params).fetchall()] | ||
| WHERE d.source_id IN (""" + ",".join("?" for _ in source_ids) + ")" | ||
| + where + " ORDER BY c.id", params).fetchall()] | ||
| return retrieve_passages(question, rows) |
There was a problem hiding this comment.
Pass the filtered terms to the BM25 query
When an unanswerable question contains common words, such as What is the snowfall depth?, this forwards the raw question even though the preceding code computes stopword-filtered tokens. Haystack can therefore return passages matching only words such as what or the, causing LiveService to call the model with irrelevant evidence instead of taking its no-evidence path. Pass the normalized tokens to the retriever, or apply equivalent filtering inside the Haystack pipeline.
Useful? React with 👍 / 👎.
Expose search_sources, get_claim, and propose_write over stdio and the existing FastAPI app. Retrieval still uses Haystack; propose_write cannot approve itself or persist a note. Co-authored-by: Jordi Lluis <coder058@users.noreply.github.com>
Keep the live allowlist only; recorded claim IDs still come from CLAIMS. Co-authored-by: Jordi Lluis <coder058@users.noreply.github.com>
Live ask/retrieval ranks passages with a small Haystack 2.x Pipeline. This PR also adds a local MCP surface so the same desk can be attached in Cursor. The recorded Pages case is unchanged.
Haystack
LiveStore.retrieve()still reads latest captured versions from SQLite, then builds HaystackDocuments and runsInMemoryBM25Retriever(haystack-ai==2.31.0).approve(), and allowlisted connectors are untouched.MCP (local/stdio + existing FastAPI)
search_sources(Haystack live retrieval, no new corpus),get_claim(one recorded claim cell + exact sentence),propose_write(pending draft only).propose_writecannot approve itself. A separate humanstore.approve()/POST /api/drafts/{id}/decidewrites the note.GET /api/mcp/toolslists schemas (curl -s http://127.0.0.1:8000/api/mcp/tools).POST /api/mcp/tools/{name}invokes one tool.python -m infodesk.mcp_server(mcp==2.2.0). README has a Cursormcp.jsonsnippet and a checklist (auth, schema, approve, 429, harness).What did not change