Define multi-country indexing strategy

[SaitejaKommi]

Priority

P2

Area

Architecture Research / ADR

Problem

When serving multiple countries, how should OpenSearch indices be structured? Option A: Separate physical index per country (askoff_products_ca, askoff_products_in). Option B: Unified global index with country_code filtering. Option C: Sharded / routing key architecture. No Architecture Decision Record (ADR) currently documents this.

Why This Matters

Index topology decisions dictate cluster memory overhead, language analyzer isolation, re-indexing operational complexity, and query routing latency. Making this decision without structured evaluation risks expensive architectural migrations later.

Current Behaviour

• backend/search/index_lifecycle.py creates timestamped indices like askoff_products_YYYYMMDD... and swaps alias askoff_products.
• Single-country Canadian catalog assumption is embedded in index naming and alias management.
• No documented ADR exists analyzing multi-country index trade-offs.

Expected Behaviour

• Trade-offs between separate indices, unified index, and hybrid approaches are documented.
• Query routing implications and language analyzer isolation are analyzed.
• Storage, shard sizing, and operational complexity are evaluated.
• A clear recommendation is presented as an Architecture Decision Record (ADR).
• No implementation begins until maintainers review the decision.

Proposed Implementation

1. Author Architecture Decision Record: Create docs/adr/0001-multi-country-indexing-strategy.md following standard ADR format (Status, Context, Decision Drivers, Options Considered, Trade-Off Matrix, Recommendation).
2. Evaluate Dimensions:
   • Language Analyzers: Language-specific analyzers per index vs multi-fields in a global index.
   • Operational Independence: Re-indexing one country without taking down others.
   • Shard Overhead: Lucene memory footprint of multiple small indices vs one large index.
   • Cross-Border Search: Feasibility of multi-country queries.
3. Present Recommendation: Deliver an objective recommendation for maintainer approval.

Acceptance Criteria

• Trade-offs between isolated indices, unified global index, and routing options are documented.
• Query routing implications are analyzed.
• Storage, shard count, and operational complexity are evaluated.
• Country isolation and data quality concerns are addressed.
• A clear recommendation is presented as an Architecture Decision Record.
• No implementation begins until maintainers review the decision.

Implementation Prompt

First inspect the existing implementation and tests before making changes. Understand the current behaviour and identify the smallest appropriate change. Implement the requested functionality without unrelated refactoring. Add or update regression tests. Run the relevant tests, linting/type checks, and verification commands. Do not modify unrelated components.

Inspect `backend/search/mappings.py`, `backend/search/index_lifecycle.py`, `backend/repositories/opensearch_repository.py`, and `docs/DEPLOYMENT.md`.
Research and document the multi-country indexing strategy:
1. Create `docs/adr/0001-multi-country-indexing-strategy.md` following standard ADR format (Status, Context, Decision Drivers, Options Considered, Evaluation Matrix, Recommendation).
2. Deeply analyze Option A (isolated index per country, e.g. `askoff_products_ca`) vs Option B (unified index with `country_code` filter).
3. Evaluate specific technical trade-offs: custom language analyzers, alias promotion independence, Lucene shard overhead, and query latency.
4. Present a clear, actionable recommendation for Open Food Facts maintainers.
Do not modify runtime code or index mappings.

Verification Plan

Review ADR document with project maintainers:

# Verify ADR file exists and adheres to standard structure
cat docs/adr/0001-multi-country-indexing-strategy.md

Dependencies

Backlog ID 18

Maintainer Decision Required

A maintainer decision is required to formally approve the multi-country index strategy before any production implementation begins.

Out of Scope

• Modifying production OpenSearch cluster settings before maintainer consensus.
• Deleting existing Canadian index structures.

Relevant Files

• backend/search/mappings.py
• backend/search/index_lifecycle.py
• backend/repositories/opensearch_repository.py
• docs/DEPLOYMENT.md
• docs/adr/0001-multi-country-indexing-strategy.md