Jarvis

A local-first, voice-driven AI assistant with a futuristic HUD interface.

Status

Phase 0 — Foundation. Modular service architecture, event-driven backbone, WebSocket gateway, and a Jarvis-inspired UI shell are in place. Speech, LLM, TTS, automation, tasks and memory live as wired-in stubs and are being filled in across phases 1–5.

Architecture

                 ┌────────────────────────────────────────────────┐
   Browser  ◄────┤  React HUD  (Vite + Tailwind v4 + Framer)      │
                 └──────────────┬─────────────────────────────────┘
                                │  REST  +  WebSocket  (ws://…/ws)
                 ┌──────────────┴─────────────────────────────────┐
                 │  FastAPI Gateway  (api/routes.py · api/ws.py)  │
                 ├────────────────────────────────────────────────┤
                 │  ServiceRegistry  ⟵ Lifecycle (Topo-sorted)    │
                 ├────────────────────────────────────────────────┤
                 │           Async EventBus (pub/sub)             │
                 ├──────┬──────┬──────┬──────┬──────┬──────┬─────┤
                 │ wake │ stt  │ llm  │ tts  │ auto │ task │ mem │
                 │ word │      │      │      │      │      │ ory │
                 └──────┴──────┴──────┴──────┴──────┴──────┴─────┘

Every service:

• inherits core.service.Service
• declares name and depends_on
• emits domain events (domain.events) onto the bus
• subscribes to events from other services — never imports them

The WebSocket gateway forwards a curated subset of events to all connected clients; the HUD reflects them in real time.

Tech stack

Layer	Tech
Backend	Python 3.12+, FastAPI, Uvicorn, structlog
Config	pydantic-settings v2
Wake word	Porcupine (Phase 1)
STT	faster-whisper (Phase 1)
LLM	Ollama (Qwen2.5 / Llama 3) (Phase 2)
TTS	Piper (Phase 3)
Automation	pyautogui + per-platform adapters (Phase 4)
Memory	SQLite → pgvector (Phase 5)
Frontend	React 19, Vite 6, TypeScript, Tailwind v4
Animation	Framer Motion, Canvas2D audio visualizer
State	Zustand
Container	Docker Compose (backend + Ollama)

Run it

# Backend
cd backend
py -3.13 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e ".[dev]"
jarvis        # or: python -m jarvis

# Frontend (in a second terminal)
cd frontend
npm install
npm run dev

Open http://localhost:5173.

Roadmap

• Phase 1 — Mic input · Wake word · STT
• Phase 2 — LLM integration · streaming · short-term memory
• Phase 3 — Real-time TTS
• Phase 4 — Tool calling · desktop automation · calendar · todos
• Phase 5 — Long-term memory · vector search · proactive planning

Layout

backend/                Python service runtime
  src/jarvis/
    core/               EventBus, Service, Registry, Lifecycle
    domain/             Typed domain events
    api/                FastAPI routes + WebSocket gateway
    services/           One subpackage per domain (wake, stt, llm, …)
  tests/
frontend/               React HUD
  src/
    api/                REST + WebSocket clients
    store/              Zustand state
    components/         HUD, AudioVisualizer, StatusPanel, …

License

MIT License — Copyright (c) 2026 Mas0n1x. See LICENSE for the full text.