TakoRAG

TakoRAG is my first serious RAG and agentic AI project. I built it as an experiment to move from a simple chatbot toward a system that can understand a user request, decide whether it needs retrieved knowledge, call the retrieval pipeline, and return an answer grounded in documents.

The first version was developed mostly during 2024, then imported and cleaned up for this repository as an initial private GitHub release.

Why I Built It

My goal was not just to connect an LLM to a PDF. I wanted to explore the idea of making a knowledge assistant more agentic:

• The user asks a question in natural language.
• The system builds a structured prompt for an agent-like decision step.
• The agent chooses a command such as direct answer, clarification, or RAG answer.
• The RAG layer retrieves relevant document chunks from a vector database.
• The final answer includes retrieved context, used documents, and basic cost tracking.

This project helped me learn how RAG systems behave in practice: document parsing, embeddings, vector search, authentication, streaming responses, prompt design, and API structure.

What It Does

• Provides a Flask API for chat, AGPT-style runs, document upload, streaming, authentication, and health checks.
• Uses OpenAI and Gemini model integrations.
• Stores and searches vector embeddings with local Qdrant.
• Reads PDF, DOCX, and TXT content for knowledge-base indexing.
• Supports JWT and API-key based access control.
• Tracks runs and responses in local SQLite databases.
• Streams long-running agent/RAG responses through server-sent events.
• Includes a simple web interface for chat and knowledge-base management.

Agentic RAG Flow

flowchart LR
    User["User Question"] --> API["Flask API"]
    API --> Agent["Agent Prompt + Decision"]
    Agent --> Direct["Direct Answer"]
    Agent --> Clarify["Clarify Request"]
    Agent --> RAG["RAG Command"]
    RAG --> Retrieve["Qdrant Retrieval"]
    Retrieve --> Rank["Ranking + Keyword Scoring"]
    Rank --> Answer["Grounded Answer"]
    Direct --> Response["Final Response"]
    Clarify --> Response
    Answer --> Response

Loading

Main Modules

Area	Purpose
main.py	Flask application entry point and route registration
API/	Authentication and API route handlers
AutoGPT/	Agent-style orchestration and command execution
Plugins/rag_system.py	Retrieval, ranking, answer generation, and cost calculation
Plugins/qdrant_system.py	Local Qdrant vector database integration
Models/	OpenAI and Gemini wrappers
Config/DataBases/	SQLite helpers for runs, embeddings, and document metadata
Web/	Chat and knowledge-base UI

For a more technical overview, see docs/ARCHITECTURE.md.

Tech Stack

• Python
• Flask
• OpenAI API
• Google Gemini API
• Qdrant
• SQLite
• JWT authentication
• Server-sent events
• HTML/CSS/JavaScript

Implementation Story

One of the funniest and most useful parts of this project was that the real problems did not show up in the clean "it works on my machine" moment. They showed up while we were presenting the idea to a company.

For example, the user was asking in Persian, the documents were also Persian, but because of the way I had written the system prompt, the model sometimes answered in English. Technically the retrieval worked, the context was there, the model understood the content, and then the final answer casually ignored the language expectation. That was a very good reminder that in RAG, prompt design is not decoration. It is part of the system.

There were a lot of these small but important lessons:

• The model needed clearer instructions about output language.
• The agent prompt had to separate "thinking about what to do" from "answering the user".
• Retrieved Persian context needed to stay readable and not get over-compressed.
• A demo can expose edge cases much faster than private testing.

It was a little stressful in the middle of the presentation, but honestly it made the project much more interesting for me. The whole path of building this, hitting those issues, and improving the system step by step was one of the best learning parts of the project.

Another big challenge came from the actual files I was preparing tests for. The company documents were not cleanly separated by topic. In some cases, one PDF had nearly the same structure for four or five different departments, and only a few keywords or section-specific terms were changed. That made the final answer harder than a normal RAG demo, because the vector search could retrieve something that was semantically close but still belonged to the wrong department or process.

That was the point where I realized I should not rely only on plain RAG retrieval. I started adding extra retrieval and ranking logic so the answer could be closer to the user's exact question:

• keyword and entity weighting,
• organization/document filtering,
• score combination instead of using only vector similarity,
• adjacent-page context,
• minimum score and content-length checks,
• clearer tracking of which documents were used in the final response.

This part was one of the most valuable lessons for me: real RAG quality is not only about embeddings. It is also about retrieval strategy, metadata, ranking, and understanding the shape of the documents.

Setup

Create a virtual environment, install dependencies, and copy the environment template:

python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
copy .env.example .env

Fill in .env with your own keys:

OPENAI_API_KEY=
GENAI_API_KEY=
ADMIN_API_KEY=
JWT_SECRET_KEY=

Run the API:

python main.py

The API starts on:

http://127.0.0.1:8000

Health check:

GET /health

Run smoke tests:

pytest

What I Learned

This project was my first hands-on attempt at building a real RAG backend instead of only using ready-made chat interfaces. The most important things I learned were:

• How to split an AI system into API, model, retrieval, database, and UI layers.
• Why prompt design matters when the model is expected to output structured commands.
• How vector search alone is not always enough, and why ranking, metadata, and keyword signals are useful.
• How authentication and permissions become important even in AI prototypes.
• How much project hygiene matters when a prototype becomes something I want to show in a portfolio.

Current Status

This is an early but functional version. It is intentionally kept as an initial project snapshot, with small cleanup improvements added for GitHub:

• Added .gitignore to keep secrets and runtime data out of version control.
• Added .env.example.
• Added dependency list in requirements.txt.
• Fixed an OpenAI chat wrapper return-path issue.
• Fixed the AGPT runner to work with the updated OpenAI wrapper.
• Moved runtime paths and CORS settings into environment-driven configuration.
• Removed hardcoded browser API keys and masked API keys in logs/admin output.
• Added SECURITY.md with a public-release checklist.
• Added docs/ARCHITECTURE.md to explain the system design and hybrid retrieval strategy.
• Added smoke tests for the health endpoint and protected route boundaries.

Next Improvements

If I continue this project, the main improvements I would add are:

• Docker setup for easier deployment.
• Better test coverage for the API and RAG pipeline.
• A cleaner admin dashboard for document management.
• Configurable remote Qdrant support.
• Better error handling around model responses and malformed JSON.
• More structured observability for costs, latency, and retrieval quality.
• A production-ready frontend instead of the current prototype UI.

Repository Notes

Runtime databases, local Qdrant storage, logs, uploaded files, local sample data, and .env are intentionally excluded from git.

Author

Built by Abolfazl Poryaei as an early RAG and agentic AI project.