Repository navigation
fix(knowledge): count knowledge_list_sources items in linear time - #22194
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] d0a7741 Description read[DESCRIPTION-READ] 04d66aa13b37e0bde11fffe2ca85c345aad071d4dbae04d2749e013248b53dc9 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing to check. What this change shipsInventory (3 items) — 3 justifiedIntent: let someone list what is in a large knowledge library from chat without the call hanging and taking knowledge search down for everyone on the shared MCP backend. This is a FIX (Closes #21705).
[FIRST-PRINCIPLES-REVIEWED] d0a7741 Description read[DESCRIPTION-READ] 5d51d00ccda90e4fb7a75343023e9675719c05d2e5c5cb5c96ed850d0d6399f0 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
6e99762 to
d0a7741
Compare
|
Rebased onto main carrying the Harness Parity fix (#22178); CI re-running. |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: replaces the correlated subquery in the knowledge_list_sources per-source count with a UNION join, so the count runs in linear time; the new tests check the result equals the old query on random libraries and that the query plan has no correlated subquery.
Problem / Motivation
Goal:
knowledge_list_sourcesanswers quickly on a large knowledge library instead of hanging and taking the shared MCP backend down with it.On a library with 21 sources and about 76k items, the tool never returned.
kirocrew-coreis a single-worker MCP backend, so every call queued behind it failed too (includinglocal_knowledge_search), until the gateway recycled the backend after 900 s as "wedged". Retrying did the same again.Why it matters
The tool is meant to be the cheap way to see what is in the knowledge base and to find a
source_id. On a large library it is the most expensive MCP call, and it breaks knowledge search from chat for everyone sharing the backend.Not a goal
/api/knowledge/sources,/api/knowledge/source-counts); they already use a linear query.What changed (motivation → approach → change)
Symptom: the per-source count query never finishes on a big library.
Cause: it put
OR i.id IN (SELECT ... WHERE sl.source_id = s.id)inside the JOIN. That subquery depends on the outer row, so SQLite runs it again for every (source, item) pair.Change: in
src/kiro_crew/mcp_tools/knowledge.py, the query (now_SOURCE_COUNTS_SQL) builds (source, item) pairs from the two ways an item belongs to a source (it owns it, or it is a location of it), joins them withUNIONso an item held both ways counts once, and joins the result tosourcesone time. The counts and the output text stay the same. This is the same shape the dashboard's/source-countshandler already uses. No other caller insrc/uses the correlated form.Backwards compatibility
Compatible: same rows, same counts, same output text; only the SQL shape changes.
Removes nothing: the query returns the same columns and rows; no path, key or record is taken away.
Tests
In
test/test_mcp_knowledge_search.py, classTestKnowledgeListSourcesCountQuery:test_item_owned_and_located_in_same_source_counts_once: an item owned by a source and also listed as its location counts as 1.test_matches_correlated_query_on_random_libraries(12 seeds): random libraries with deleted items, sourceless items and cross-source locations give the same rows from the new query and from the old correlated one.test_query_plan_has_no_correlated_subquery:EXPLAIN QUERY PLANof the old shape saysCORRELATED(a probe that this SQLite reports it), and the shipped query's plan does not. Swapping the old query back in makes this test fail (checked locally).Determinism: the 18
ListSourcestests passed 3 times in a row with-n0, and once more in a shuffled order (seed 7). The random libraries use a fixedrandom.Random(seed).Manual verification
Synthetic library, 21 sources, every item also listed as a location (script run locally, not committed):
Both queries gave the same total at 5,000 items.
Related Issues
Closes #21705
Pattern harvest
Rule candidate: semgrep
Pattern: correlated
IN (SELECT ... WHERE x = outer.col)inside aJOIN ... ONclause in a SQL string literal.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)