🍲 Ragoût

Your vehicle's service manual, now with answers.

Got a stack of service manuals catching dust and no time to dig through them for one answer? Let Ragoût cook something up for you — an expert car-mechanic assistant powered by a multimodal RAG agent grounded in your specific vehicle's service manuals and technical service bulletins (TSBs).

PDFs are parsed with Docling (text, tables, and diagrams). Diagrams are exported as images and described by a local vision model. Everything is embedded into a persistent ChromaDB vector store via LlamaIndex. A tool-calling agent then answers questions in a browser chat, citing its sources and showing the relevant diagrams inline — served over AG-UI with a CopilotKit frontend.

All models run locally through LM Studio, so it works offline and for free.

How it works

source/*.pdf  ──parse.py──▶  Docling  ──▶  text/table chunks ─┐
                                  └──▶ diagrams ─▶ VLM describe ┤
                                                                ├─▶ embed ─▶ ChromaDB (data/chroma)
                                                                └─▶ docstore (data/storage)
                                                                              │
        utils/doc_index.py ──▶ per-document catalogue + routing index         │
                                                                              │
   browser ◀── web/ (CopilotKit) ◀── AG-UI ◀── serve.py ◀── agent ◀── vector + summary tools

• Image → text-summary RAG. Each diagram's VLM description is stored as a normal text node with the PNG path in its metadata, so diagrams are retrieved by the same text query as everything else (only a text embedding model is needed, no CLIP/multimodal embeddings).
• Two-pass retrieval. Users ask in symptom language ("creaking noise") while figure descriptions are written in figure language ("exploded view"), so a single search buries the diagrams. One pass answers the question; a second pass finds the figures on the pages that answer actually cited.
• Source authority. utils/doc_index.py catalogues each document (title, type, date, what it supersedes) with one LLM call. Bulletins correct the manual and newer bulletins correct older ones, so citations carry that context and the agent is told to prefer the authoritative source instead of averaging conflicting specs.
• Document routing. Broad questions are matched against document summaries first, so cost stays flat as the library grows instead of reading every document on every question.
• Incremental & idempotent. Each PDF is tracked by SHA-256 in data/manifest.json; re-running parse.py skips unchanged files and adds new ones to the existing database. Changes to the embedding model are also tracked and automatically rebuild the database.

Project structure

Path	What
source/	original service manuals and technical service bulletins as PDFs
data/images/	extracted diagram PNGs (data/images/<pdf>/picture-N.png)
data/chroma/	persistent vector index + document-summary routing index
data/manifest.json	tracks PDFs already ingested
data/documents.json	per-document catalogue (title, type, date, summary)
data/storage/	LlamaIndex node docstore (powers the summary tool)
web/	React chat frontend (CopilotKit over AG-UI)
.env	config
.env.example	template for .env
utils/config.py	shared env + model + store wiring
parse.py	script to ingest PDFs into the knowledge base
utils/doc_index.py	catalogues documents and builds the routing index
utils/rag_tools.py	the agent's retrieval tools (structured results)
serve.py	web server: AG-UI endpoint, diagrams, and the built UI
utils/logbook.py	append-only JSONL execution log (ragout.jsonl)
ragout.jsonl	questions, replies and tool calls; gitignored, quotes manual content

Neither source/ nor data/ is committed. Everything under data/ is derived from the PDFs — chunk text, VLM diagram descriptions, document summaries — and carries the same copyright restrictions as the manuals themselves (see NOTICE), so a clone starts empty and needs its own PDFs and a parse.py run.

Setup

1. Python 3.12:

   python3.12 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt

   On macOS: brew install python@3.12.

2. [Optional] Git LFS — only if you drop the data/ entry from .gitignore to commit the vector DB and images. .gitattributes already routes data/chroma/** and data/images/** through LFS for that case; with the default ignore rules you can skip this step.

   brew install git-lfs   # if needed
   git lfs install

3. Node.js 20+ (for the web frontend):

   cd web && npm install && npm run build && cd ..

4. LM Studio: install, download the relevant models and start the server:

   • a text embedding model
   • a chat model (drives the agent)
   • a vision model (describes diagrams)

   The chat model must support native (OpenAI-style) function calling — the agent reaches the manuals through tools, and there is no prompt-based fallback. google/gemma-4-26b-a4b is verified working. If you swap it, serve.py probes the model at startup and tells you if it does not qualify. Qwen models are also generally strong at tool calling.

5. Configure the project:

   cp .env.example .env
   # edit .env: set the model identifiers exactly as they appear in LM Studio,
   # and set VEHICULE_YEAR_BRAND_MODEL (e.g. "1996 Porsche 911 Targa").

Usage

If using a new terminal, make sure to use source .venv/bin/activate first

Ingest PDFs

# drop manuals / bulletins into source/, then:
python parse.py            # parse new/changed PDFs into the knowledge base
python parse.py --force    # re-ingest everything (ignores the manifest)

The first run downloads Docling's layout/table models (one-time). Diagram description is the slow part as it calls the vision model once per figure.

parse.py catalogues documents and rebuilds the routing index at the end of every run, so there is nothing else to run. Cataloguing is incremental and self-healing: a document whose catalogue failed (LM Studio down, say) is picked up by the next parse.py.

Re-cataloguing every document

Only needed after changing the cataloguing prompt or the chat model — the existing summaries stay as they were until you ask for them to be redone. It reads the docstore, so no PDF is re-parsed and the vision model never runs:

python -m utils.doc_index --force

Chat

python serve.py

Then open http://localhost:8000. One command serves the API, the diagram images, and the UI on a single port.

Ask questions about the vehicle. Answers stream as they are generated, and each search shows its sources inline — document, page, and the document's type and date — along with any relevant diagrams.

Working on the frontend

python serve.py          # terminal 1: the API
cd web && npm run dev    # terminal 2: http://localhost:5173, hot reload

The dev server proxies /run, /health and /images to serve.py, so it runs same-origin exactly like production. Run npm run build when you are done so python serve.py picks up the changes.

License

The source code of this project is licensed under the Apache License 2.0.

The license covers the code only. It does not cover any document you place in source/ or any artifact derived from it (everything under data/). See NOTICE for details.