Skip to content

Eagerly load embedding model on FastAPI startup - #1

Open
railway-app[bot] wants to merge 1 commit into
mainfrom
railway/code-change-R1A5em
Open

railway-app[bot] wants to merge 1 commit into
mainfrom
railway/code-change-R1A5em

Conversation

@railway-app

@railway-app railway-app Bot commented Jul 25, 2026

Copy link
Copy Markdown

Problem

The Embedder in backend/indexing/embedder.py lazily loads the all-MiniLM-L6-v2 model via a @Property, deferring the download/initialization until the first request. Since this can take 5-15 minutes, the first requests after deploy hit the backend while it's mid-download, resulting in 502 Bad Gateway errors even though the Dockerfile pre-seeds data via python -m backend.cli ingest sample_docs/.

Solution

Added an explicit load() method to Embedder that forces the lazy model property to resolve. In backend/main.py's lifespan startup handler, we now call app.state.orchestrator.embedder.load() right after constructing the Orchestrator, so the model is fully downloaded and initialized before the FastAPI app finishes startup and begins accepting HTTP requests.

Changes

  • Modified backend/indexing/embedder.py
  • Modified backend/main.py

Generated by Railway

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants