by Andrei Goldenberg, Irena Kochtov

Customer Service Dataset Agent

This project implements an agentic customer service dataset analyst using LangGraph, LangChain tools, persistent memory, MCP, and a Streamlit UI.

The agent analyzes the Bitext Customer Support dataset and can answer structured questions, summarize qualitative patterns, remember conversation context, maintain a persistent user profile, expose tools through MCP, and recommend follow-up queries.

---

Model Choice

All LLM calls use Nebius Token Factory.

Model used: meta-llama/Llama-3.3-70B-Instruct

The API endpoint used is: https://api.tokenfactory.nebius.com/v1/

Tools

The agent defines the following dataset tools:

• list_categories: returns all dataset categories.
• list_intents: returns all intents, optionally filtered by category.
• count_rows: counts rows filtered by category and/or intent.
• show_examples: returns example dataset rows, with support for offset-based pagination.
• intent_distribution_by_category: returns intent counts for a specific category.
• get_text_samples: returns representative text samples for qualitative summaries and pattern analysis.

These tools are wrapped as LangChain StructuredTool objects with Pydantic input schemas in agent_tools.py.

The same underlying dataset operations are also exposed through the FastMCP server in mcp_server.py.

Features

Task 1 — LangGraph ReAct Agent

Implemented an explicit LangGraph ReAct workflow:

User Query
   ↓
Memory Router
   ↓
Dataset Router
   ├── structured
   ├── unstructured
   └── out_of_scope
   ↓
Model Node ↔ Tool Node
   ↓
Final Answer

The agent supports:

• Structured dataset questions
• Unstructured summarization questions
• Out-of-scope rejection
• Tool calls with visible observations
• Multi-step reasoning
• Max-iteration fallback

Example:

User: How many refund requests did we get?

Tool call: list_intents(category="REFUND")
Observation: ["check_refund_policy", "get_refund", "track_refund"]

Tool call: count_rows(intent="get_refund")
Observation: 997

Agent: There are 997 refund request records in the dataset.

---

Task 2a — Conversation Memory / Episodic Memory

Conversation history is persisted using LangGraph checkpoints with SQLite.

Each conversation is identified by a session_id.

Run:

python main.py --session memory_test --user andrei

The same session ID restores the same conversation even after restarting the app.

Example flow:

User: Show me 3 examples from the REFUND category.
Agent: [shows first 3 examples]

User: Show me 3 more.
Agent: [continues with offset=3]

Restart app with the same session.

User: Show me 3 more.
Agent: [continues with offset=6]

The agent also handles numeric follow-ups:

User: How many complaints did we get?
Agent: 1000

User: What about refunds?
Agent: 2992

User: What is the total count of the last two?
Agent: 3992

---

Task 2b — User Profile Memory

The agent maintains a persistent user profile separately from conversation history.

Profiles are stored as JSON files under:

profiles/<user_id>.json

The profile stores distilled long-term facts, not full conversation replay.

Tracked profile fields:

{
  "name": null,
  "preferences": [],
  "topics_of_interest": [],
  "notes": []
}

Example:

User: My name is Andrei.
Agent: Thanks — I’ll remember that in your profile.

User: I prefer concise technical explanations.
Agent: Thanks — I’ll remember that in your profile.

User: What do you remember about me?
Agent:
- Your name is Andrei.
- Preferences: concise technical explanations.

The profile persists across restarts and across different sessions as long as the same --user value is used.

---

Task 3 — MCP Server

The project includes an MCP server built with FastMCP.

The server exposes dataset tools as MCP tools.

Exposed MCP tools:

• get_categories
• get_intents
• get_record_count
• get_examples
• get_intent_distribution

Start the MCP server:

python mcp_server.py

MCP Client Example

Run the included client test:

python mcp_client_test.py

The client connects to mcp_server.py, lists available tools, and calls:

await client.call_tool(
    "get_record_count",
    {"intent": "get_refund"}
)

Expected result:

997

The main CLI agent uses the local Python tool layer directly. Separately, the same dataset operations are exposed through FastMCP so external MCP clients can call them.

---

Bonus A — Streamlit UI

The project includes a Streamlit chat UI that wraps the same agent.

Run:

streamlit run streamlit_app.py

The UI supports:

• Chat-style interaction
• Sidebar API key input
• Sidebar session ID input
• Sidebar user ID input
• Visible reasoning steps
• Tool calls and observations
• Persistent conversation memory
• Persistent user profile memory

Example Streamlit usage:

Session ID: streamlit_test
User ID: andrei

User: How many refund requests did we get?

The UI displays both the final answer and the reasoning steps.

---

Bonus B — Query Recommender

The agent includes an interactive query recommendation feature.

When the user asks:

What should I query next?

The agent looks at:

• Recent conversation history
• Persistent user profile

Then it suggests a relevant follow-up query without executing it immediately.

Example flow:

User: What should I query next?

Agent:
Suggested query: `What is the distribution of intents in the REFUND category?`
Should I go ahead?

User: I'd rather see examples instead.

Agent:
Suggested query: `Show 5 examples of get_refund requests.`
Should I go ahead?

User: Yes, do it.

Agent:
[executes show_examples(intent="get_refund", n=5)]

The query is executed only after explicit user confirmation.

---

Setup

1. Create a virtual environment

python -m venv .venv

Activate it on Windows PowerShell:

.\.venv\Scripts\Activate.ps1

If PowerShell blocks activation, run:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned
.\.venv\Scripts\Activate.ps1

---

2. Install dependencies

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

---

3. API key

The Nebius API key is not stored in code.

The CLI asks for it securely at runtime:

Enter your Nebius API key:

The Streamlit UI accepts it through a password field in the sidebar.

---

Running the CLI Agent

python main.py --session demo_session --user demo_user

Then ask questions such as:

What categories exist in the dataset?

How many refund requests did we get?

Summarize the FEEDBACK category.

Show me 3 examples from the REFUND category.

Show me 3 more.

What do you remember about me?

Exit with:

quit

or:

exit

---

Running the Streamlit App

streamlit run streamlit_app.py

Use the sidebar to provide:

• Nebius API key
• Session ID
• User ID

---

Running the MCP Server

Start the server:

python mcp_server.py

In another terminal, run the client:

python mcp_client_test.py

---

Project Files

main.py

CLI entry point.

graph_agent.py

Main LangGraph workflow, including memory routing, dataset routing, ReAct loops, profile memory, and query recommender.

agent_tools.py

LangChain StructuredTool wrappers and Pydantic schemas.

data_tools.py

Dataset loading and pure Python data operations.

profile_store.py

Persistent user profile storage using JSON files.

mcp_server.py

FastMCP server exposing dataset tools.

mcp_client_test.py

Example MCP client that lists tools and calls one tool.

streamlit_app.py

Streamlit chat UI wrapper around the agent.

requirements.txt

Project dependencies.

.gitignore

Ignores virtual environments, runtime memory files, cache files, and local profile data.

---

Architecture Summary

CLI / Streamlit
   ↓
LangGraph App
   ↓
SQLite Checkpointer
   ↓
Load User Profile
   ↓
Memory Router
   ├── profile_question
   ├── profile_update
   ├── query_recommendation
   ├── recommendation_refinement
   ├── recommendation_confirmation
   └── normal_query
           ↓
       Dataset Router
       ├── structured
       ├── unstructured
       └── out_of_scope
           ↓
       Explicit ReAct Loop
       Model Node ↔ Tool Node
           ↓
       Final Answer

---

Runtime Files

The following files and folders are created during runtime and should not be submitted as source files:

conversation_checkpoints*.sqlite
profiles/
__pycache__/
.venv/
.venv_test/

They are excluded through .gitignore.

---

Notes

• The agent uses the Bitext Customer Support dataset from Hugging Face.
• The Nebius API key is entered at runtime and is not stored in the repository.
• Conversation memory and user profile memory are intentionally separated:
  • --session controls episodic conversation memory.
  • --user controls persistent user profile memory.