Research semantic / vector retrieval

[SaitejaKommi]

Priority

P2

Area

Retrieval Research

Problem

Dense vector search is widely discussed, but its benefits for grocery search (exact brands, barcodes, numeric nutrient filters) versus BM25 are unproven on Open Food Facts data. SearchDocument.semantic_document text is generated during ingestion but only indexed as standard text. We need empirical research comparing BM25 vs Vector vs Hybrid before modifying production search.

Why This Matters

Vector search introduces significant RAM overhead, inference latency, and hardware costs. Furthermore, vector search often degrades exact keyword searches (barcodes, exact brand names) that BM25 handles perfectly. Objective research prevents premature architectural bloat.

Current Behaviour

• AskOFF operates purely on lexical BM25 retrieval with function scoring.
• backend/models/search_document.py includes semantic_document: str, but it is indexed as standard text in backend/search/mappings.py.
• No dense vector embeddings (knn_vector) exist in OpenSearch.
• No empirical benchmark compares lexical vs vector retrieval on AskOFF queries.

Expected Behaviour

• This is RESEARCH ONLY. Vector retrieval must NOT be described as part of the active production engine.
• A representative evaluation set is established.
• Baseline BM25 results are measured.
• Candidate embedding/vector approaches are evaluated.
• Computational and storage costs are documented.
• Failure cases are analyzed.
• A recommendation is produced for whether hybrid/vector retrieval is worthwhile.
• No production vector search should be introduced without maintainer review.

Proposed Implementation

1. Offline Research Harness: Build an offline benchmark script backend/evaluation/research_vector_retrieval.py using sentence-transformers (e.g. all-MiniLM-L6-v2) over a sample of products (5k-10k items).
2. Evaluate 3 Strategies:
   • Strategy A: Lexical BM25 only (current baseline)
   • Strategy B: Dense vector retrieval only
   • Strategy C: Hybrid retrieval (Reciprocal Rank Fusion / linear combination of BM25 + dense score)
3. Quantitative Metrics: Compare Precision@5, NDCG@10, query latency (p50/p95), and RAM/CPU footprint.
4. Produce Research Report: Write docs/research/VECTOR_RETRIEVAL_EXPERIMENT.md with objective empirical findings.

Acceptance Criteria

• A representative evaluation set is established.
• Baseline BM25 results are measured.
• Candidate embedding/vector approaches are evaluated.
• Results are compared using appropriate retrieval metrics (P@5, NDCG@10, latency).
• Computational/storage costs are documented.
• Failure cases are analyzed.
• A recommendation is produced for whether hybrid/vector retrieval is worthwhile.
• No changes are made to the live OpenSearch production index or serving API.

Implementation Prompt

First inspect the existing implementation and tests before making changes. Understand the current behaviour and identify the smallest appropriate change. Implement the requested functionality without unrelated refactoring. Add or update regression tests. Run the relevant tests, linting/type checks, and verification commands. Do not modify unrelated components.

Inspect `backend/models/search_document.py`, `backend/builders/search_document_builder.py`, `backend/evaluation/evaluate.py`, and `backend/evaluation/benchmark_queries.json`.
Conduct empirical research comparing lexical BM25 against dense vector and hybrid retrieval:
1. In `backend/evaluation/research_vector_retrieval.py`, build an offline benchmark using sentence-transformers (`all-MiniLM-L6-v2`) over a sample of products.
2. Evaluate and compare three modes across the benchmark queries:
   - BM25 lexical retrieval
   - Dense vector retrieval
   - Hybrid retrieval (BM25 + Vector)
3. Compute Precision@5, NDCG@10, and average query latency for each mode.
4. Document empirical results, strengths, weaknesses, and hardware costs in `docs/research/VECTOR_RETRIEVAL_EXPERIMENT.md`.
Do not modify the live OpenSearch mappings or serving search API.

Verification Plan

Run offline research evaluation:

python backend/evaluation/research_vector_retrieval.py --sample-size 2000
cat docs/research/VECTOR_RETRIEVAL_EXPERIMENT.md

Dependencies

Backlog ID 3

Maintainer Decision Required

A maintainer decision is required on whether dense vector retrieval provides sufficient measurable quality lift to justify the computational and memory overhead in production.

Out of Scope

• Replacing OpenSearch BM25 with a vector database in production.
• Introducing required PyTorch/Transformers dependencies into backend/requirements.txt.

Relevant Files

• backend/models/search_document.py
• backend/builders/search_document_builder.py
• backend/evaluation/evaluate.py
• backend/evaluation/benchmark_queries.json
• backend/search/mappings.py
• docs/research/VECTOR_RETRIEVAL_EXPERIMENT.md