[Zhi-Xiang-Guo]

Fixes #45223

Summary

Two workspaces can configure the same embedding provider/model name with different endpoints. Previously, both the persistent document cache and Redis query cache could return the other workspace's vectors. With real CLIP (512 dimensions) in workspace A and BGE (384 dimensions) in B, this caused retrieval to fail with expected dim: 512, got 384 or the inverse error.

Scope cache identities by workspace for text and image embedding paths. The same API reproduction now returns HTTP 200 for both previously failing requests, with nonempty retrieval records; the fresh-query control also remains successful.

Changes Made

• api/core/rag/embedding/cached_embedding.py:
  • Include the model configuration's tenant ID and input modality in the versioned document-cache hash, preserving the existing fixed-width column and unique constraint.
  • Require tenant context for query-cache reads and use a new Redis namespace for text/image queries. Serialize provider/model components before hashing to avoid ambiguous concatenation.
  • Read and write the same workspace-scoped identities, with no fallback to legacy unscoped entries. Same-workspace cache reuse and the existing query TTL remain intact.
• api/core/rag/datasource/vdb/vector_factory.py:
  • Pass the dataset's tenant ID to the lazy query-cache lookup, preserving the optimization that skips model resolution on a cache hit.
• api/core/rag/rerank/weight_rerank.py:
  • Pass tenant context to weighted reranking's direct query-cache lookup.

Compatibility and recovery: no database schema migration is required. Upgrade the API and indexing workers together. The new document hashes and Redis namespace do not read legacy unscoped entries, so recovery does not depend on their expiry. Existing document-cache entries remain for the normal cleanup job; Redis TTL is sliding and refreshes on hits, so 600 seconds is an idle interval, not a guaranteed lifetime since creation.

This prevents new cross-workspace contamination but does not automatically repair vectors already stored in knowledge-base collections. On the old code, waiting for Redis expiry or merely re-running indexing can reuse the incompatible persistent cache. After upgrading all relevant processes, affected indexes/collections must be rebuilt from source with the intended model. A collection created with the wrong dimensionality may need recreation, rather than only replacing document points. The deployed validation below uses newly created datasets and does not claim an end-to-end repair of existing contaminated collections. Changing an embedding endpoint/configuration within one workspace and model load balancing remain outside this patch's scope.

Unit Tests

• Added 12 regressions in test_workspace_cache.py, covering text/image document and query isolation, same-dimensional different vectors, different dimensions, same-workspace reuse, and rejection of legacy unscoped cache entries. All 12 fail on the original production code and pass with the fix. Persistent-cache tests use real SQLite; model invocations and Redis are mocked in these unit tests.
• Updated existing cache, lazy-lookup and weighted-rerank tests to supply and verify tenant context.
• Related suite: 143 passed, 32 deselected. The 32 vector-backend registration cases were checked separately: the original full selection produced 31 failures for uninstalled backends and one pass for Qdrant. The same 31 failures/one pass were reproduced with the clean tracked baseline in the same environment; this is not a full-suite pass.
• Actual deployed API + Celery + official compatible plugin + real CLIP/BGE inference + PostgreSQL/Redis/Qdrant: Before, both collision cases returned HTTP 400 and the fresh-query control returned 200. After, all three B requests returned HTTP 200 with nonempty records. Models, input texts and settings were unchanged; each replay uses a fresh shared model alias and new disposable datasets.
• Changed-file Ruff lint/format, targeted production-file Pyrefly, four applicable repository guards, and git diff --check passed.
• Baseline: main@5336037aa91f07188baaf03957effd57a364f7df; tested local commit: 674687a46bf29db11ffb147551f6d997b83debb4. Remote head 5e3990a7728d025b425ecd77dabcf60b3aa78298 has the exact same Git tree (0a877b1e748d68ce491bd43eac480132ed7a1ab3); commit metadata differs because the branch was uploaded through the GitHub connector.

Checklist

• Associated issue and contribution guidance reviewed.
• Meaningful regression tests added and relevant local validation completed.
• Cache compatibility and already-contaminated indexes documented above.
• Full repository make lint && make type-check / full test suite completed. Only the focused checks listed above were run; no frontend changes.
• GitHub Actions completed successfully. The initial workflows report action_required; no remote test-pass result is claimed.