shifted to pgvector, added docker, CI/CD and improved indexing

Author: devmehtaa

.dockerignore

@@ -0,0 +1,7 @@
+.git
+backend/.venv
+backend/storage
+frontend/.next
+frontend/node_modules
+**/__pycache__
+**/*.pyc

.github/workflows/ci.yml

@@ -0,0 +1,59 @@
+name: CI
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+jobs:
+  backend:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: pgvector/pgvector:pg16
+        env:
+          POSTGRES_DB: codeassist
+          POSTGRES_USER: codeassist
+          POSTGRES_PASSWORD: codeassist
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U codeassist -d codeassist"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+    env:
+      DATABASE_URL: postgresql://codeassist:codeassist@localhost:5432/codeassist
+      STORAGE_DIR: storage
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install backend dependencies
+        working-directory: backend
+        run: pip install -r requirements.txt
+      - name: Compile backend
+        run: python -m compileall backend/app
+
+  frontend:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "22"
+          cache: npm
+          cache-dependency-path: frontend/package-lock.json
+      - name: Install frontend dependencies
+        working-directory: frontend
+        run: npm ci
+      - name: Lint frontend
+        working-directory: frontend
+        run: npm run lint
+      - name: Build frontend
+        working-directory: frontend
+        env:
+          NEXT_PUBLIC_API_BASE_URL: http://localhost:8000
+        run: npm run build

.github/workflows/deploy.yml

@@ -0,0 +1,42 @@
+name: Deploy
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  id-token: write
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: aws-actions/configure-aws-credentials@v4
+        with:
+          role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
+          aws-region: ${{ vars.AWS_REGION }}
+      - uses: aws-actions/amazon-ecr-login@v2
+        id: ecr-login
+
+      - name: Build and push backend image
+        env:
+          REGISTRY: ${{ steps.ecr-login.outputs.registry }}
+          REPOSITORY: ${{ vars.ECR_BACKEND_REPOSITORY }}
+          IMAGE_TAG: ${{ github.sha }}
+        run: |
+          docker build -t "$REGISTRY/$REPOSITORY:$IMAGE_TAG" ./backend
+          docker push "$REGISTRY/$REPOSITORY:$IMAGE_TAG"
+
+      - name: Build and push frontend image
+        env:
+          REGISTRY: ${{ steps.ecr-login.outputs.registry }}
+          REPOSITORY: ${{ vars.ECR_FRONTEND_REPOSITORY }}
+          IMAGE_TAG: ${{ github.sha }}
+          NEXT_PUBLIC_API_BASE_URL: ${{ vars.NEXT_PUBLIC_API_BASE_URL }}
+        run: |
+          docker build \
+            --build-arg NEXT_PUBLIC_API_BASE_URL="$NEXT_PUBLIC_API_BASE_URL" \
+            -t "$REGISTRY/$REPOSITORY:$IMAGE_TAG" \
+            ./frontend
+          docker push "$REGISTRY/$REPOSITORY:$IMAGE_TAG"

README.md

@@ -7,7 +7,7 @@ A GitHub repository understanding assistant. Paste a repo URL, index the source
 - Frontend: Next.js, Tailwind CSS
 - Backend: FastAPI
 - RAG: LangChain, OpenAI
-- Vector store: ChromaDB for the MVP
+- Vector store: PostgreSQL + pgvector
 - Repository processing: GitPython
 
 ## Project Layout
@@ -33,6 +33,12 @@ codebase-assistant/
 
 ## Backend Setup
 
+Start Postgres with pgvector first:
+
+```bash
+docker compose up -d postgres
+```
+
 ```bash
 cd backend
 python -m venv .venv
@@ -59,6 +65,8 @@ OPENAI_CHAT_MODEL=gpt-4o-mini
 OLLAMA_BASE_URL=http://localhost:11434
 LLAMA_CHAT_MODEL=llama3.1 or llama3.2:1b
 OLLAMA_TIMEOUT_SECONDS=300
+DATABASE_URL=postgresql://codeassist:codeassist@localhost:5432/codeassist
+STORAGE_DIR=storage
 ```
 
 If OpenAI is not configured, OpenAI mode falls back to local hash embeddings and returns relevant chunks instead of a synthesized answer. If Local Llama is selected but Ollama is not running, the app also returns the relevant chunks with setup guidance.
@@ -90,6 +98,20 @@ Frontend runs at:
 http://localhost:3000
 ```
 
+## Docker Compose
+
+Run the full app with PostgreSQL/pgvector:
+
+```bash
+docker compose up --build
+```
+
+Services:
+
+- Frontend: `http://localhost:3000`
+- Backend: `http://localhost:8000`
+- Postgres/pgvector: `localhost:5432`
+
 ## API
 
 ### Index a repository
@@ -158,7 +180,6 @@ Chunk metadata:
 
 ## Notes
 
-- ChromaDB is used for the MVP because it is quick to run locally.
-- The vector database is persisted under `backend/storage/chroma`.
-- Cloned repositories are stored under `backend/storage/repos`.
-- Tree-sitter and pgvector are natural next steps once the MVP behavior is proven.
+- PostgreSQL with pgvector stores repos, files, chunks, metadata, and vectors.
+- Cloned repositories are stored under `STORAGE_DIR/repos`.
+- Tree-sitter chunking, background jobs, private repo auth, evals, and CI/CD deployment are natural next steps.

backend/.env.example

@@ -1,3 +1,8 @@
 OPENAI_API_KEY=
 OPENAI_EMBEDDING_MODEL=text-embedding-3-small
 OPENAI_CHAT_MODEL=gpt-4o-mini
+DATABASE_URL=postgresql://codeassist:codeassist@localhost:5432/codeassist
+STORAGE_DIR=storage
+OLLAMA_BASE_URL=http://localhost:11434
+LLAMA_CHAT_MODEL=llama3.1
+OLLAMA_TIMEOUT_SECONDS=300

backend/Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends git \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY app ./app
+
+EXPOSE 8000
+
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

backend/README.md

@@ -1,6 +1,6 @@
 # Backend
 
-FastAPI service for cloning repositories, chunking code, embedding chunks, storing vectors in ChromaDB, retrieving relevant chunks, and generating answers.
+FastAPI service for cloning repositories, chunking code, embedding chunks, storing vectors in PostgreSQL/pgvector, retrieving relevant chunks, and generating answers.
 
 The API supports two AI providers:
 
@@ -19,6 +19,7 @@ If local answers time out on slower machines, set `OLLAMA_TIMEOUT_SECONDS=300` o
 Run from this folder:
 
 ```bash
+docker compose up -d postgres
 python -m venv .venv
 .venv\Scripts\activate
 pip install -r requirements.txt

backend/app/api/repo_routes.py

@@ -20,6 +20,7 @@ def index_repository(payload: RepoIndexRequest) -> RepoIndexResponse:
         chunks = chunk_files(repo_id, files)
         VectorStore(embedding_service=EmbeddingService(provider=payload.ai_provider)).replace_repo_chunks(
             repo_id,
+            repo_url,
             chunks,
         )
     except Exception as exc:

backend/app/core/__init__.py

@@ -0,0 +1 @@
+

backend/app/core/settings.py

@@ -0,0 +1,10 @@
+import os
+from pathlib import Path
+
+
+DATABASE_URL = os.getenv(
+    "DATABASE_URL",
+    "postgresql://codeassist:codeassist@localhost:5432/codeassist",
+)
+STORAGE_DIR = Path(os.getenv("STORAGE_DIR", "storage"))
+REPO_STORAGE_DIR = STORAGE_DIR / "repos"