[Bug]: Same-named embedding models in different workspaces reuse incompatible cached vectors #45223

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.18.0; source main@4c521a3c0a1f60607c8a7bcf90792f34adcf18d0

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

Deployment topology and workspace provisioning

This was reproduced in a LOCAL SELF-HOSTED SOURCE/DEVELOPMENT environment at main@4c521a3c0a1f60607c8a7bcf90792f34adcf18d0 (version endpoint: 1.18.0). Both workspaces belong to the SAME Dify deployment: one source API/worker installation, the same PostgreSQL database (including the embeddings table), and the same Redis cache. Separate Dify instances with separate cache stores are not the tested scenario.

Environment: macOS arm64, API Python 3.13.15, PostgreSQL 15, Redis 6, Qdrant 1.12.6, official langgenius/openai_api_compatible plugin 0.0.73. No production source changes were applied.

Workspace A was an existing disposable test workspace. Workspace B and its owner account were created through the repository's built-in administrative CLI, from the api directory with the same application/database configuration as the running API:

ALLOW_REGISTER=true ALLOW_CREATE_WORKSPACE=true FLASK_APP=app   uv run --project . --no-sync flask create-tenant   --email dify-rag-c@example.com --name 'Embedding isolation C' --language en-US

The registration/workspace-creation flags were enabled only for this CLI process; they were not a change to the running deployment's global settings. The CLI creates the account/workspace through application services. We did not insert tenant/model/cache rows directly into the database. The CLI prints a generated test password; it is not included here.

This establishes a development reproduction using the built-in provisioning path. It does NOT establish that every default Community deployment offers a multi-workspace creation UI, and this was not a Cloud or licensed Enterprise test.

Independently configured models

Log in as the owner of each workspace and configure its model separately through the Console model-configuration API. Use the SAME fresh registration name in both workspaces, for example rag-isolation-UNIQUE_SUFFIX, under provider langgenius/openai_api_compatible/openai_api_compatible. Choose a new suffix for each replay to avoid older cache entries.

The local HTTP adapters load actual model weights and expose real inference outputs through compatible endpoints, not fixed/random vectors. Set each workspace's own configured model as its default embedding model and explicitly use it for the datasets below. Both model-credential registrations were accepted through normal Console HTTP requests. A's successful retrieval and B's fresh-query control below independently demonstrate usable inference/retrieval; registration success alone is not the control.

Case 1: persistent document cache reuses A's vectors in B

1. In A, create a high-quality knowledge base using A's model and import exactly: RAG isolation fixture about capybara scientific taxonomy. Wait until document indexing_status is completed.
2. In A, retrieve the query What is the capybara scientific taxonomy? using the request shape below. Observe HTTP 200.
3. In B, create a NEW high-quality knowledge base using B's model and import the EXACT SAME text. Wait until indexing_status is completed.
4. Retrieve from this B dataset using a different query: capybara habitat new B query. Observe HTTP 400: expected dim 512, got 384.

Case 2: Redis query cache reuses A's query vector in B

5. In B, create another NEW high-quality knowledge base with B's model and UNIQUE text: Independent tenant B document explaining the biology of capybaras. Wait until indexing_status is completed.
6. Within 600 seconds of A's successful query, retrieve from this second B dataset with A's exact query: What is the capybara scientific taxonomy? Observe HTTP 400: expected dim 384, got 512. If setup takes longer, repeat A's query immediately before this step to warm/refresh its query cache.
7. Control: query the SAME second B dataset with a fresh query not used in A: A fresh independent capybara question for workspace B. Observe HTTP 200.

Retrieval request

Use the service API key belonging to the workspace being queried; DATASET_ID must also belong to that workspace. Substitute QUERY with each exact query above. DIFY_URL is the single shared deployment URL (http://127.0.0.1:15011 in this run).

curl -sS -i -X POST "$DIFY_URL/v1/datasets/$DATASET_ID/retrieve"   -H "Authorization: Bearer $WORKSPACE_SERVICE_API_KEY"   -H 'Content-Type: application/json'   --data '{"query":"QUERY","retrieval_model":{"search_method":"semantic_search","reranking_enable":false,"top_k":3,"score_threshold_enabled":false}}'

Dataset creation, document import (/v1/datasets/{id}/document/create-by-text), indexing-status polling and retrieval were executed through authenticated Dify HTTP APIs, with real Celery indexing and Qdrant storage. No existing dataset changes its embedding model, and no direct database writes manufacture a cache collision.

Observed errors and control

A indexes/queries: success
B same text, new query: HTTP 400
Wrong input: Vector dimension error: expected dim: 512, got 384
B unique text, A's cached query: HTTP 400
Wrong input: Vector dimension error: expected dim: 384, got 512
B unique text, fresh query: HTTP 200

Why using the same name is a reasonable trigger

The model configurations belong to DIFFERENT workspaces and are accepted through the supported model-configuration API. We may expose different private deployments under the same conventional name. A workspace-local name alone does not establish equivalent vector spaces across endpoints. If global name uniqueness is intended as a prerequisite, it needs an explicit constraint; silently substituting another workspace's computed vectors breaks accepted configurations.

The dimension difference makes the observed wrong reuse visible. Equal dimensions alone would not establish vector-space equivalence, but same-dimensional recall degradation has not been measured here. This report establishes incompatible cached-vector reuse and retrieval failure, not disclosure of another workspace's source documents.

Source-level identity boundary

At the tested commit, model-name uniqueness is explicitly workspace-scoped:

# api/models/provider.py: ProviderModel.__table_args__
sa.UniqueConstraint(
    "tenant_id", "provider_name", "model_name", "model_type",
    name="unique_provider_model_name",
)

The persistent embedding cache instead uses:

# api/models/dataset.py: Embedding.__table_args__
sa.UniqueConstraint("model_name", "hash", "provider_name", name="embedding_hash_idx")

The cache lookup uses the latter fields, and the Redis query key also omits workspace/configuration identity. Workspace-scoped configuration uniqueness therefore does not establish globally equivalent model outputs. These source constraints support the independently configured same-name scenario; they do not by themselves establish which deployment editions offer workspace creation UI.

Reproducible local model endpoints

The endpoint addresses in the table refer to the same local adapter below, using two routes. In the executed Docker plugin environment, host.docker.internal resolves to the host running port 15013. For a native plugin runtime use 127.0.0.1; on other Docker setups use a host address reachable FROM THE PLUGIN container. The earlier embedding-a/embedding-b addresses were illustrative names, not additional services to deploy.

Use Python 3.12 (the inference environment used 3.12.14). Save the code below as model_server.py in a separate test directory, then:

python3.12 -m venv .venv-model
.venv-model/bin/python -m pip install torch==2.15.0 transformers==5.22.1 fastapi==0.162.0 uvicorn==0.54.0 pillow==13.0.0 numpy==2.5.4 tokenizers==0.23.3 huggingface-hub==2.4.0 safetensors==0.8.0
.venv-model/bin/python -m uvicorn model_server:app --host 0.0.0.0 --port 15013

These are the direct dependency versions recorded from the environment that produced the evidence, not a full transitive lockfile. Initial startup/inference downloads the public CLIP/BGE weights, so allow model-download access and wait for completion. The adapter is a disposable local test fixture with no authentication; local-test is a dummy provider key. Do not deploy it as an exposed production model service.

Configure BOTH registered model credentials with context_size=77 and max_chunks=16; use endpoint_url from the table and api_key=local-test. Use the same fresh registered model name in BOTH workspaces. The adapter ignores the request's model alias and routes by URL to the actual weights shown above.

Optional direct-inference check, after the server starts (this check does not warm Dify's caches):

.venv-model/bin/python - <<'CHECK'
import json, urllib.request
for route, expected in [("/v1/embeddings", 512), ("/bge/v1/embeddings", 384)]:
    request = urllib.request.Request(
        "http://127.0.0.1:15013" + route,
        data=json.dumps({"model": "dimension-check", "input": ["capybara"]}).encode(),
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(request, timeout=600) as response:
        result = json.load(response)
    actual = len(result["data"][0]["embedding"])
    assert actual == expected, (route, actual)
    print(route, actual)
CHECK

The complete fixture used for inference is included inline. Its BGE response's informational model label is BAAI/bge-small-en; the loaded weights are explicitly BAAI/bge-small-en-v1.5. Cache identity in Dify comes from the registered model configuration, not that response label. The image/rerank routes are retained from the original fixture but are not needed for this text-cache reproduction.

"""Real CLIP inference behind the OpenAI-compatible embedding contract."""
import base64,io,logging
import torch
from PIL import Image
from transformers import CLIPModel,CLIPProcessor
from fastapi import FastAPI
model_id='openai/clip-vit-base-patch32'
model=CLIPModel.from_pretrained(model_id)
processor=CLIPProcessor.from_pretrained(model_id)
model.eval()
app=FastAPI()
@app.post('/v1/embeddings')
def embeddings(body:dict):
    if 'messages' in body:
        items=[c for m in body['messages'] if m['role']=='user' for c in m['content']]
        images=[Image.open(io.BytesIO(base64.b64decode(c['image_url']['url'].split(',',1)[1]))).convert('RGB') for c in items if c['type']=='image_url']
        with torch.inference_mode():
            result=model.get_image_features(**processor(images=images,return_tensors='pt'))
        kind='image'
    else:
        texts=body['input'];texts=[texts] if isinstance(texts,str) else texts
        with torch.inference_mode():
            result=model.get_text_features(**processor(text=texts,return_tensors='pt',padding=True,truncation=True,max_length=77))
        kind='text'
    if not isinstance(result,torch.Tensor):result=result.pooler_output
    result=result/result.norm(dim=-1,keepdim=True)
    logging.warning('Real CLIP inference kind=%s count=%d dimensions=%d',kind,result.shape[0],result.shape[1])
    return {'object':'list','model':model_id,'data':[{'object':'embedding','index':i,'embedding':v} for i,v in enumerate(result.tolist())],'usage':{'prompt_tokens':1,'total_tokens':1}}

@app.post('/v1/rerank')
def rerank(body:dict):
    def encode(value):
        if isinstance(value,str):return embeddings({'input':[value]})['data'][0]['embedding']
        content=value['content']
        images=[c for c in content if c['type']=='image_url']
        if images:return embeddings({'messages':[{'role':'user','content':images}]})['data'][0]['embedding']
        return encode(' '.join(c.get('text','') for c in content))
    query=torch.tensor(encode(body['query']))
    docs=body['documents']
    results=[{'index':i,'relevance_score':float(torch.dot(query,torch.tensor(encode(d)))), 'document':{'text':d if isinstance(d,str) else 'image'}} for i,d in enumerate(docs)]
    logging.warning('REAL_CLIP_RERANK query_type=%s document_types=%s',type(body['query']).__name__,[type(d).__name__ for d in docs])
    return {'results':sorted(results,key=lambda x:x['relevance_score'],reverse=True)[:body.get('top_n',len(docs))]}

bge_model=None
bge_tokenizer=None
@app.post('/bge/v1/embeddings')
def bge_embeddings(body:dict):
    global bge_model,bge_tokenizer
    from transformers import AutoModel,AutoTokenizer
    if bge_model is None:
        bge_tokenizer=AutoTokenizer.from_pretrained('BAAI/bge-small-en-v1.5')
        bge_model=AutoModel.from_pretrained('BAAI/bge-small-en-v1.5').eval()
    texts=body['input'];texts=[texts] if isinstance(texts,str) else texts
    with torch.inference_mode():
        out=bge_model(**bge_tokenizer(texts,return_tensors='pt',padding=True,truncation=True,max_length=128)).last_hidden_state[:,0]
    out=out/out.norm(dim=-1,keepdim=True)
    logging.warning('REAL_BGE_EMBED count=%d dimensions=%d',len(texts),out.shape[1])
    return {'object':'list','model':'BAAI/bge-small-en','data':[{'object':'embedding','index':i,'embedding':v} for i,v in enumerate(out.tolist())],'usage':{'prompt_tokens':1,'total_tokens':1}}

✔️ Expected Behavior

Cached embeddings must be interchangeable with outputs of the requesting workspace's effective model configuration. B should consistently index and query using its own 384-dimensional model. Sharing requires equivalent vector-space identity, not merely matching provider and model names.

❌ Actual Behavior

A indexes/queries successfully. B importing A's text fails retrieval with expected dim 512, got 384. B importing unique text but repeating A's query fails with expected dim 384, got 512. B's fresh-query control succeeds (HTTP 200).

The first failure exposes persistent document-cache reuse; the second exposes Redis query-cache reuse. The successful control confirms B's model and retrieval path work.

Source investigation: api/core/rag/embedding/cached_embedding.py looks up persistent Embedding rows by model_name/hash/provider_name; _query_cache_key uses provider/model_name/query_hash. Neither includes workspace/effective configuration identity. _LazyEmbeddings in vector_factory.py can read this query cache before resolving the workspace's model instance.

Related work: #41481/#41989 address NaN/input alignment; #42673 adds a lazy query-cache path. They do not cover this cross-workspace identity boundary. Discovery was source-first followed by targeted deduplication; historical coverage is bounded.

Observed impact is wrong-vector reuse and retrieval failure. This report does not establish disclosure of another workspace's source documents. Same-dimensional recall degradation was not measured. No fix/after-fix result is claimed. AI-assisted investigation and local API validation.