Homer

Autonomous AI Production Director

Homer is a local-first production control system for two camera nodes. It receives live browser/WebRTC feeds, evaluates camera readiness and active-speaker evidence, recommends an explainable shot, protects continuity, and preserves immediate operator control.

This repository is the working pre-AdventureX base. It is intentionally useful before the final CV model, Pocket control experiment, MAX2 virtual PTZ, motor adapter, and OBS adapter are complete.

What works now

• production session creation and joining;
• exactly two camera nodes;
• browser camera and microphone capture;
• peer-to-peer camera-to-director WebRTC transport;
• room-based FastAPI WebSocket signaling;
• professional program/preview/multiview console;
• manual TAKE and quick-cut controls;
• live audio-level telemetry;
• explainable deterministic director scoring;
• 2.5-second continuity guard and switch margin;
• AI-service health/fallback state;
• optional ONNX provider interface;
• decision timeline and confidence display;
• SQLite session and decision foundation;
• Dockerfiles and Compose configuration;
• safe motion/production adapter interfaces.

What is intentionally left for AdventureX

• Pocket 4 hardware validation and native tracking control;
• MAX2 Open GoPro adapter and 360 proxy;
• real face/person tracking observations;
• DJI Mic channel-to-person calibration;
• physical motor-controller implementation;
• OBS WebSocket adapter;
• trained/selected ONNX model;
• recording, replay, and highlight export.

See ADVENTUREX_BUILD_GUIDE.md for the exact implementation order and ownership.

Architecture

Camera A browser -------- WebRTC --------+
  telemetry ----+                       |
                +--> FastAPI signaling  +--> Director console
Camera B browser -------- WebRTC --------+         |
  telemetry ----+                                 |
                                                    +--> AI director service
                                                    +--> manual override
                                                    +--> future OBS adapter

AI provider: heuristic-v1 (default) OR onnx-v1 (optional)
Hardware: adapter contracts; no unguarded motor commands in the base

Repository

packages/
  frontend/       Next.js camera node and director console
  backend/        FastAPI sessions, signaling, decisions, adapters
  ai-service/     Explainable shot scoring and optional ONNX provider
ADVENTUREX_BUILD_GUIDE.md
docker-compose.yml
.env.example

Local development

Requirements:

• Node.js 20+
• pnpm 11+
• Python 3.10+
• Poetry

Copy configuration:

cp .env.example .env

Terminal 1 - backend:

cd packages/backend
poetry install
poetry run uvicorn app.main:app --reload --port 8000

Terminal 2 - AI service:

cd packages/ai-service
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python -m src.main

Terminal 3 - frontend:

pnpm install
pnpm dev

After the first install, Windows users can start all three terminals with:

.\start-homer.ps1

Open:

• Home: http://localhost:3000
• Director: http://localhost:3000/director?room_id=FRAME26
• Camera A: http://localhost:3000/camera?room_id=FRAME26&node=a
• Camera B: http://localhost:3000/camera?room_id=FRAME26&node=b
• Backend docs: http://localhost:8000/docs
• AI docs: http://localhost:8001/docs

For a first test, use three browser windows or devices. Camera pages require localhost or HTTPS because browsers protect camera access.

Docker

docker compose up --build

Browser camera access from another device requires an HTTPS deployment. Plain HTTP works only on localhost in most browsers.

AI strategy

The default heuristic-v1 provider is not fake random switching. It scores:

• active-speaker audio evidence;
• composition/framing readiness;
• stability;
• subject visibility;
• observation freshness.

It then applies a switch margin and minimum shot duration. This gives the team a reliable baseline and generates labelled decision data.

To activate an ONNX model:

pip install numpy onnxruntime
export HOMER_AI_PROVIDER=onnx
export HOMER_ONNX_MODEL=/absolute/path/director.onnx

Model contract:

• input tensor: [N, 6] float32;
• features: audio, framing, stability, visibility, face-visible, freshness;
• output: one shot score per camera.

The CV pipeline should update observations; it should not directly move motors or switch OBS.

Verification

pnpm build
PYTHONPATH=packages/ai-service python -m unittest discover packages/ai-service/tests
python -m compileall packages/backend packages/ai-service

Safety

• automation starts disarmed;
• the operator can always cut manually;
• future motion adapters must enforce travel, velocity, watchdog, and emergency-stop limits;
• an LLM must never send raw motor or OBS commands;
• local camera recording should begin before automation during the event.

License

GPL-3.0. See LICENSE.