GlyGen AI Chatbot

A citation-first Retrieval-Augmented Generation (RAG) chatbot for glycobiology students, built over Essentials of Glycobiology, Fourth Edition. Students ask questions in a Streamlit chat UI; the UI talks to a FastAPI backend over HTTP. Answers are grounded in the textbook with page-level citations — the system prefers refusal over hallucination when evidence is weak.

Version: 0.2.0 (Dockerized prototype)

Overview
Quick Start
Get API Keys
Prerequisites
Install From Scratch (Local)
Docker Installation and Run
How the Application Works
Features
Architecture
Tech Stack
Project Structure
Configuration Reference
Choosing the LLM
Using the Streamlit UI
How It Works Internally
Data and Git
What's Implemented vs Planned
Troubleshooting

Overview

GlyGen AI Chatbot answers glycobiology questions using only retrieved textbook passages (not general LLM knowledge).

Layer	What runs
UI	Streamlit → HTTP client (`GLYGEN_API_URL`)
Backend	FastAPI + `ChatOrchestrator` + RAG + SQLite
Vectors	ChromaDB (embedded locally, or HTTP server in Docker)
LLM	Groq (free default) or OpenAI, selected by `LLM_OPTION`
Embeddings / rerank	HuggingFace MiniLM + BGE reranker

Four Docker containers (optional):

ID	Service	Role
C1	`ingestion`	One-shot PDF indexing
C2	`backend`	FastAPI + RAG + sessions
C3	`chromadb`	Vector database server
C4	`streamlit`	Chat UI

Quick Start

Option A — Docker (recommended for full stack)

Install Docker Desktop and start it.
Get API keys (below).
Place the textbook PDF at the project root.
Create .env and run:

cp .env.example .env
# Edit .env with HF_TOKEN, GROQ_API_KEY, and optionally OPENAI_API_KEY / LLM_OPTION

docker compose -f docker-compose.ingest.yml up --build
docker compose -f docker-compose.app.yml up --build

Open http://localhost:8501

Option B — Local (no Docker)

pip install -r apps/backend/requirements.txt
pip install -r apps/ingestion/requirements.txt
pip install -r apps/frontend/requirements.txt
cp .env.example .env   # add keys
# Place PDF at project root or text under data/converted/

# Ingest
$env:PYTHONPATH = "$PWD\packages;$PWD\apps\ingestion\src"
python -m ingestion.pipeline

# Terminal 1 — backend
$env:PYTHONPATH = "$PWD\packages;$PWD\apps\backend\src"
python -m api.server

# Terminal 2 — UI
cd apps/frontend/src
streamlit run app.py

Get API Keys

You need a HuggingFace token and a chat LLM key. The default chat model is Groq (LLM_OPTION=free). OpenAI is optional.

1. HuggingFace token (`HF_TOKEN`)

Used to download embedding and reranker models.

Step	Link / action
Sign up / log in	https://huggingface.co/join
Create access token	https://huggingface.co/settings/tokens
Token type	Read access is enough for model download
Put in `.env`	`HF_TOKEN=hf_...`

2. Groq API key (`GROQ_API_KEY`)

Used for the default (free) LLM answer generation (openai/gpt-oss-20b).

Step	Link / action
Sign up / log in	https://console.groq.com/
Create API key	https://console.groq.com/keys
Put in `.env`	`GROQ_API_KEY=gsk_...`

Never commit .env. It is listed in .gitignore.

3. OpenAI API key (`OPENAI_API_KEY`) — only if `LLM_OPTION=openai`

Step	Link / action
Sign up / log in	https://platform.openai.com/
Create API key	https://platform.openai.com/api-keys
Put in `.env`	`OPENAI_API_KEY=sk-...`
Select model	`LLM_OPTION=openai`

Prerequisites

Local run

Python 3.11+
Git
HF_TOKEN and GROQ_API_KEY (see above). Add OPENAI_API_KEY only if you set LLM_OPTION=openai.
Textbook PDF at project root:
Essential_of_Glycobiology_4E_EPUB_V5_InterVenn.pdf
(not in the GitHub repo — you must add it yourself)

Docker run

Everything above, plus:
Docker Desktop installed and running
Enough disk/RAM for model downloads (first build can take several minutes)

Verify Docker:

docker --version
docker compose version

Install From Scratch (Local)

Follow these steps on a clean machine.

Step 1 — Clone the repository

git clone <your-repo-url>
cd Glygen-AI-CHatbot

Step 2 — Create a virtual environment

python -m venv .venv

# Windows (PowerShell / CMD)
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

Step 3 — Install Python dependencies

pip install -r apps/backend/requirements.txt
pip install -r apps/ingestion/requirements.txt
pip install -r apps/frontend/requirements.txt

Step 4 — Get API keys and create `.env`

Create tokens at:
- HuggingFace: https://huggingface.co/settings/tokens
- Groq: https://console.groq.com/keys
Copy the example env file:

# Windows
copy .env.example .env

# macOS / Linux
cp .env.example .env

Edit .env:

HF_TOKEN=hf_your_token_here
GROQ_API_KEY=gsk_your_groq_key_here
OPENAI_API_KEY=sk_your_openai_key_here
LLM_OPTION=free
GLYGEN_API_URL=http://127.0.0.1:8000

Step 5 — Add the textbook PDF

Place this file in the project root (same folder as README.md):

Glygen-AI-CHatbot/Essential_of_Glycobiology_4E_EPUB_V5_InterVenn.pdf

PDFs and data/ are gitignored and are not pushed to GitHub.

Step 6 — Ingest the textbook (one-time)

From the project root, with the venv active:

# Windows PowerShell
$env:PYTHONPATH = "$PWD\packages;$PWD\apps\ingestion\src"
python -m ingestion.pipeline

This loads the PDF or data/converted/ text, chunks it (~1000 chars / 200 overlap), embeds with MiniLM, and stores vectors in data/chroma/ (collection glyco_corpus). First run downloads HuggingFace models.

Step 7 — Start the backend

$env:PYTHONPATH = "$PWD\packages;$PWD\apps\backend\src"
python -m api.server

Step 8 — Start the Streamlit UI

Open a second terminal, activate .venv, then:

cd apps/frontend/src
streamlit run app.py

Open http://localhost:8501

Streamlit calls the backend at GLYGEN_API_URL (default http://127.0.0.1:8000).

Docker Installation and Run

Use this path if you want the full 4-container setup.

What gets started

Compose file	Containers	When to run
`docker-compose.ingest.yml`	C3 ChromaDB + C1 Ingestion	Once (or when PDF changes)
`docker-compose.app.yml`	C3 ChromaDB + C2 Backend + C4 Streamlit	Every time you use the app

Both share the named volume glygen_chroma_data so the app can read the index built by ingest.

Step 1 — Install and start Docker Desktop

Download: https://www.docker.com/products/docker-desktop/
Install and open Docker Desktop
Wait until it says Docker is running
Open a new terminal and confirm:

docker --version
docker compose version

Step 2 — Clone, keys, PDF

git clone <your-repo-url>
cd Glygen-AI-CHatbot
copy .env.example .env   # or: cp .env.example .env

Edit .env with keys from:

Place the PDF at project root:

Essential_of_Glycobiology_4E_EPUB_V5_InterVenn.pdf

Step 3 — Build and run ingestion (C1 + C3)

From the project root:

docker compose -f docker-compose.ingest.yml up --build

What this does:

Starts ChromaDB on host port 8001
Mounts your PDF into the ingestion container
Runs python -m Ingestion.pipeline once
Writes embeddings into volume glygen_chroma_data
Ingestion container exits when finished (restart: "no")

First run may take a long time (image build + model download + ~1300 pages).

Step 4 — Build and run the app (C2 + C3 + C4)

docker compose -f docker-compose.app.yml up --build

URL	Service
http://localhost:8501	Streamlit UI
http://localhost:8000	FastAPI backend
http://localhost:8000/docs	Interactive API docs
http://localhost:8001	ChromaDB (debug)

Streamlit is configured with GLYGEN_API_URL=http://backend:8000 inside Docker.

Step 5 — Re-ingest after PDF updates

docker compose -f docker-compose.ingest.yml up ingestion --build

Step 6 — Stop containers

docker compose -f docker-compose.app.yml down
docker compose -f docker-compose.ingest.yml down

Volumes (glygen_chroma_data, chat_data, hf_cache) persist so you do not lose the index or chat DB unless you remove volumes.

To wipe volumes as well:

docker compose -f docker-compose.app.yml down -v

Docker file map

File	Purpose
`docker-compose.ingest.yml`	ChromaDB + ingestion job
`docker-compose.app.yml`	ChromaDB + backend + Streamlit
`apps/ingestion/Dockerfile`	Ingest image (load → embed → Chroma)
`apps/backend/Dockerfile`	FastAPI + RAG image
`apps/frontend/Dockerfile`	Streamlit UI image
`.dockerignore`	Keeps builds smaller

How the Application Works

Student (Browser)
       │
       ▼
┌──────────────────────┐
│   Streamlit UI       │  streamlit_app.py
│   - Chat input       │
│   - Session sidebar  │
└──────────┬───────────┘
           │  HTTP (GLYGEN_API_URL)
           ▼
┌──────────────────────┐
│  FastAPI Backend     │  api/server.py
│  ChatOrchestrator    │
└──────────┬───────────┘
           │
     ┌─────┴─────┐
     ▼           ▼
┌─────────┐  ┌──────────────┐
│RAG      │  │SessionStore  │
│Pipeline │  │(SQLite)      │
└────┬────┘  └──────────────┘
     │
     ▼
┌─────────┐
│ChromaDB │
│(vectors)│
└─────────┘

Each chat message: Streamlit → POST /api/v1/chat → classify → RAG or local handler → save to SQLite → return answer with page citation.

Features

Streamlit UI over HTTP to FastAPI
PDF ingest → Chroma (glyco_corpus)
Retrieve top-20 → BGE rerank top-5 → confidence gate
Groq grounded JSON answers + page citations (or OpenAI when LLM_OPTION=openai)
Session memory (name, recap, follow-ups) in SQLite
Question routing: textbook / name / recap / out-of-scope
Docker: 4 containers, 2 compose files
Chroma HTTP mode (Docker) or embedded mode (local)

Architecture

Component	File	Role
Streamlit UI	`apps/frontend/src/app.py`	Chat + sidebar
HTTP client	`apps/frontend/src/client.py`	Calls backend REST API
FastAPI	`apps/backend/src/api/server.py`	REST entry point
Orchestrator	`apps/backend/src/api/orchestrator.py`	Route + persist
RAG	`apps/backend/src/rag/pipeline.py`	Retrieve → rerank → gate → generate
Chat LLM	`packages/llm/` + `config/llms.json`	Groq or OpenAI from `LLM_OPTION`
Chroma client	`packages/glyco_chroma/`	HTTP or embedded (shared)
Ingestion	`apps/ingestion/src/ingestion/pipeline.py`	PDF/txt → Chroma
Sessions	`apps/backend/src/session/store.py`	SQLite

API endpoints

Method	Path	Purpose
`GET`	`/health`	Chroma + DB readiness
`GET`	`/api/v1/status`	Model / collection info
`POST`	`/api/v1/sessions`	Create session
`GET`	`/api/v1/sessions`	List sessions
`GET`	`/api/v1/sessions/{id}/messages`	History
`POST`	`/api/v1/chat`	Ask a question

Tech Stack

Layer	Technology
UI	Streamlit
API	FastAPI + Uvicorn
LLM	Groq `openai/gpt-oss-20b` (default) or OpenAI `gpt-4o-mini`
Embeddings	`sentence-transformers/all-MiniLM-L6-v2`
Reranker	`BAAI/bge-reranker-base`
Vector DB	ChromaDB
Sessions	SQLite
PDF	PyPDF / LangChain
Containers	Docker Compose

Project Structure

glycochatbot/
├── apps/
│   ├── frontend/                 # C4 Streamlit (HTTP client only)
│   ├── backend/                  # C2 FastAPI + RAG + SQLite
│   └── ingestion/                # C1 one-shot corpus indexer
├── packages/
│   ├── glyco_chroma/             # shared Chroma HTTP/embedded client
│   └── llm/                      # Groq / OpenAI chat clients + JSON loader
├── config/
│   └── llms.json                 # LLM options selected by LLM_OPTION
├── prompts/
│   ├── free/                     # Groq system/human prompts
│   └── openai/                   # OpenAI system/human prompts
├── docs/
│   └── choose-llm.md             # how to switch models
├── docker-compose.ingest.yml
├── docker-compose.app.yml
├── data/                         # local only (gitignored except .gitkeep)
├── .env.example
└── README.md

Configuration Reference

Variable	Required	Used by	Description
`HF_TOKEN`	Yes	Ingest, Backend	HuggingFace — create token
`GROQ_API_KEY`	Yes for `LLM_OPTION=free`	Backend	Groq — create key
`OPENAI_API_KEY`	Yes for `LLM_OPTION=openai`	Backend	OpenAI — create key
`LLM_OPTION`	No (default `free`)	Backend	Key in `config/llms.json`: `free` or `openai`
`GLYGEN_API_URL`	Streamlit	UI	Backend URL (`http://127.0.0.1:8000` local; `http://backend:8000` in Docker)
`CHROMA_HOST` / `CHROMA_PORT`	Docker	Ingest, Backend	Set by compose; omit for local embedded Chroma
`PDF_PATH`	Docker ingest	Ingestion	PDF path inside container
`DATA_DIR`	Docker app	Backend	SQLite / data directory

See docs/choose-llm.md for the JSON catalog and prompt folders.

Choosing the LLM

Set LLM_OPTION in .env. If it is missing, the app uses free.

`LLM_OPTION`	Chat model	Used for
`free`	Groq `openai/gpt-oss-20b`	Response generation
`openai`	OpenAI `gpt-4o-mini`	Response generation

Ingest embeddings always use MiniLM, even when LLM_OPTION=openai. Ranking stays BGE (not an LLM).

Change models, temperatures, or prompt paths in config/llms.json without editing Python.

Using the Streamlit UI

Chat input — ask glycobiology questions
New chat — new SQLite session
Sidebar — switch between recent chats

You say	Behavior
"My name is Alex"	Saves name (no LLM)
"What is glycosylation?"	Full RAG + page citation
"Tell me more about that"	Follow-up with prior context
"What did I ask?"	Session recap
"What's the weather?"	Out-of-scope refusal

How It Works Internally

Ingestion

Load PDF → chunk (1000/200) → embed → store in glyco_corpus.

RAG

Normalize → similarity k=20 → BGE top-5 → gate (min score -2.0) → selected LLM JSON → validate citations.

Question types

textbook_rag | session_name | session_recap | out_of_scope

Default models

Setting	Value
Embedding	`all-MiniLM-L6-v2` (always; not switched by `LLM_OPTION`)
Reranker	`bge-reranker-base` (always BGE)
LLM (default)	Groq `openai/gpt-oss-20b` when `LLM_OPTION` is unset or `free`
LLM (paid)	OpenAI `gpt-4o-mini` when `LLM_OPTION=openai`
Temperature	`0.0`

Data and Git

These are not committed to GitHub (see .gitignore):

.env (secrets)
*.pdf (textbook)
data/ contents (Chroma index, chat.db, converted text)

Only data/.gitkeep is kept so the folder exists after clone. You must add the PDF and run ingest yourself (local or Docker).

What's Implemented vs Planned

Done

FastAPI + Streamlit (HTTP)
RAG with gate and citations
SQLite sessions
Docker 4-container / 2-compose setup
Chroma HTTP + embedded modes
Configurable chat LLM (LLM_OPTION + config/llms.json + prompts/)
README for local and Docker run

Local vs Docker

Mode	Chroma	Backend	UI
Local	`data/chroma/` embedded	`uvicorn`	`streamlit`
Docker	Chroma container	`backend`	`streamlit` container

Still planned

React UI + SSE streaming
NCBI chapter/section metadata
Evaluation question bank
Rate limiting / CI/CD / cloud pilot

Troubleshooting

Problem	Fix
`docker` not recognized	Install/start Docker Desktop, open a new terminal
Missing `HF_TOKEN` / `GROQ_API_KEY`	Create keys at HuggingFace and Groq links above; put them in `.env`
Missing `OPENAI_API_KEY` with `LLM_OPTION=openai`	Set `OPENAI_API_KEY` or switch back to `LLM_OPTION=free`
PDF not found	Place `Essential_of_Glycobiology_4E_EPUB_V5_InterVenn.pdf` at project root
Streamlit can't reach backend	Local: start uvicorn first; set `GLYGEN_API_URL=http://127.0.0.1:8000`
Chroma empty in Docker app	Run `docker-compose.ingest.yml` before `docker-compose.app.yml`
First Docker build is slow	Normal — downloads base image + HF models into `hf_cache`

Acknowledgments

Built for glycomics education using Essentials of Glycobiology. Powered by Groq, OpenAI, HuggingFace, LangChain, ChromaDB, FastAPI, Streamlit, and Docker.

Name		Name	Last commit message	Last commit date
apps		apps
config		config
data		data
docs		docs
packages		packages
prompts		prompts
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.python-version		.python-version
README.md		README.md
docker-compose.app.yml		docker-compose.app.yml
docker-compose.ingest.yml		docker-compose.ingest.yml
pyproject.toml		pyproject.toml

Latest commit

History

Folders and files

Repository files navigation