Skip to content

fix(search): scope rerank cache fallback to the scoring query - #1043

Open
andrewloux wants to merge 1 commit into
tobi:mainfrom
andrewloux:andrewloux/fix-rerank-intent-cache
Open

andrewloux wants to merge 1 commit into
tobi:mainfrom
andrewloux:andrewloux/fix-rerank-intent-cache

Conversation

@andrewloux

@andrewloux andrewloux commented Oct 8, 2026 •

Copy link
Copy Markdown

QMD prepends caller-supplied intent to the query before reranking. The compatibility fallback can reuse a score for the bare query, bypassing scoring with that intent.

For example, a caller reranks a passage from doc.md with query cache and intent configuration commands. QMD's scoring input is:

configuration commands

cache

Assume a previous file-scoped entry holds a score for doc.md, this passage, the same model, and bare cache. The current key has no entry for the input above.

  • Before: QMD returns the bare-query score. The model receives zero calls, so the supplied intent never reaches scoring.
  • After: QMD skips that fallback, scores the passage with the input above, and caches the new score. Repeating the request reuses that score.

The fallback now requires the formatted query to equal the bare query. Missing and empty intents retain compatibility reuse; current-key entries retain priority. formatRerankQuery() extracts the existing formatting; production rerank() calls it to build the scoring input.

The regression installs LegacyCacheReranker, a model fake that records the already-formatted query and returns 0.99. The test seeds a file-scoped score of 0.11, then checks one intent-aware model call, repeat reuse, and bare/empty-intent compatibility. All 324 store tests pass on Node and Bun (13 integration tests skipped per runtime). Types, lint, and compiled-package smoke pass.

bun run test:node test/store.test.ts
bun run test:bun test/store.test.ts
bun run test:types
bun run lint
bun run test:package

An intent-prefixed query receives its own score when the legacy cache
contains a score for the bare query. Bare and empty-intent queries retain
legacy cache reuse; current-format keys keep priority.
@andrewloux andrewloux changed the title fix(search): scope legacy rerank cache to the scoring query fix(search): scope rerank cache fallback to the scoring query Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant