[Zhi-Xiang-Guo]

Summary

Fixes #45227

A text query against a multimodal knowledge base could recall an image, then lose it when a vision-capable reranker was enabled: the text-query branch filtered image candidates before scoring. With real CLIP inference and Qdrant, the same HTTP retrieval returned one image without reranking and zero with reranking.

When prepared candidates include an image, invoke the multimodal reranker with a TEXT query and the mixed candidate list. Text-only candidate sets retain the existing text invocation, and non-vision models keep their current behavior. The same API reproduction now returns one image with reranking enabled, and the model endpoint logs confirm actual image inference.

Replace the fallback-encoding test with five parameterized behavior cases covering image preservation and result identity, duplicate suppression, threshold/top-k filtering, vision-capable models with text-only candidates, and non-vision models. Existing image-query tests remain in the suite.

This change does not address the independent missing-upload index-alignment problem in #41480; no missing UploadFile rows are used here. No schema, configuration or dependency changes.

Testing

Baseline: 170dbaeb34789365131a7723f1f33063c4e58558; patch commit: 0ae37f3ecd3fdfb90fec78e55a8a11975eab4ae3.

• New regression matrix on original production code: 3 failed, 2 passed; the failures are the vision + image-candidate cases.
• UV_NO_SYNC=1 make test TARGET_TESTS='./api/tests/unit_tests/core/rag/rerank --no-cov -q': 53 passed on the fix.
• UV_NO_SYNC=1 make lint: passed, including 78 import contracts; Ruff and git diff --check also passed. The sparse checkout's tracked web/.env.example was materialized to complete the lint target.
• Focused production Pyrefly check passed. The strict test-file check has 125 errors versus 126 on the unmodified baseline, with no added error diagnostics. Full make type-check was attempted and failed with missing optional VDB/tracing dependencies and other diagnostics; it is not reported as passing. The full failure set has not been isolated against a clean baseline.
• Actual authenticated API verification with official compatible plugin 0.0.73, actual CLIP text/image inference and Qdrant 1.12.6, using both the existing fixture and a newly created fixture. The issue contains the runnable adapter and setup/replay scripts. Fresh setup creates its own models, dataset, document and uploaded PNG through supported HTTP APIs. After setup, before/after use the same dataset and request settings.
• Unit tests isolate storage/model I/O; those mocks are separate from the real HTTP validation above. No full repository unit suite or full pre-commit run was performed. Remote workflows currently report action_required and await maintainer approval; no CI pass is claimed.

Screenshots

Backend API change; reproducible request/response and model logs instead of UI screenshots.

Same text query, threshold disabled, top_k=10	Before	After
Reranking disabled	HTTP 200, 1 image	HTTP 200, 1 image
Vision reranking enabled	HTTP 200, 0 images	HTTP 200, 1 image
Candidates received by rerank endpoint	3 strings	4 structured candidates, including the image

Model log after the fix:

Real CLIP inference kind=image count=1 dimensions=512
REAL_CLIP_RERANK query_type=str document_types=['dict', 'dict', 'dict', 'dict']

The query remains text at the provider endpoint, as intended; the image is now present among the candidates. This does not claim every commercial provider's behavior or final LLM answer-quality improvements.

Checklist

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
• I've updated the documentation accordingly.
• I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

No frontend changes. Lint passed; the final checklist item remains unchecked because full type-check did not pass. AI assistance was used for investigation, implementation, test execution and drafting; the recorded results are actual runs, not a claim of human review.