Local LLMs for Neuroergonomics

Hands-on workshop code: run a local LLM (via LM Studio) as a bounded, schema-constrained component inside neuroscience / human-factors experiments — including a live EEG → LLM → smart-home agentic loop.

The thesis: the LLM does not decode the brain. A cheap feature extractor produces a low-dimensional state + confidence; the LLM interprets that state in context and emits one allow-listed action. Everything else is guardrails.

What's here

demos/
  core/                 the taught sequence (00 → 07)
  psychopy_showcase/    standalone PsychoPy demos (zero extra dependencies)
capstone_smart_home/    the EEG → LLM → cozy-room agentic loop (its own Python 3.12 project)
docs/                   design notes, application ideas

Quick start

Full install instructions, model recommendations and troubleshooting: SETUP.md.

1. Install LM Studio, download a small instruct model, and start the local server (Developer/Server tab). Loading a model in the chat UI is not enough.
2. Install the demo dependencies into your PsychoPy interpreter:

   python -m pip install -r requirements.txt

3. Check it:

   cd demos/core
   python 00_check_server.py

4. Work through the sequence:

Script	Teaches
00_check_server.py	reaching the OpenAI-compatible endpoint
01_lmstudio_hello.py	first chat completion; latency and tokens
02_structured_guardrails.py	JSON-schema output + allow-list + confidence gate
03_psychopy_adaptive.py	a PsychoPy task where the LLM writes feedback and sets difficulty
04_capstone_agentic_loop.py	EEG (LSL) → alpha → LLM → action, with a threshold-rule baseline
05_llm_experimenter.py	the LLM as a closed-loop experimenter (hypothesis → design → notebook)
06_capstone_smart_home.py	the modular capstone
07_vlm_camera.py	a vision-language model reading a webcam (bonus modality)

Set the endpoint with LLM_BASE_URL (default http://localhost:1234/v1) and optionally pin LLM_MODEL.

PsychoPy Builder experiments (.psyexp)

Real Builder experiments — open them with File ▸ Open in PsychoPy Builder and edit them like any other: participant-info dialog, conditions files, TrialHandler loops, proper data saving.

File	What it shows
demos/core/builder_01_number_judge.psyexp	The simplest structured-output demo: a number appears, the LLM must answer with schema-constrained JSON {level, confidence, justification}. Watch the confidence fall for numbers near the middle of the range.
demos/core/builder_02_chat.psyexp	A chat: type a message, the model replies, 8 turns. Shows the background-thread pattern (the window never freezes).
demos/core/builder_03_flanker.psyexp	A full Eriksen flanker experiment. Randomised trials from conditions_flanker.csv, nested in a blocks loop, with an LLM coach between blocks.

Keep each .psyexp next to llm_config.py and its conditions_*.csv — the Code Components import from the same folder.

The guardrail worth pointing at, in builder_03: the LLM proposes the next difficulty, but a plain Python staircase decides it. Both are written to the data file (llm_proposed_difficulty, rule_difficulty, llm_agreed_with_rule), and the rule wins. A scalar adaptation doesn't need an LLM — a rule is simpler and more defensible — so the model is used for the language, not the arithmetic. When a small model disagrees with the rule, you can watch the guardrail override it.

The capstone

cd capstone_smart_home
cp .env.example .env          # then edit LLM_MODEL / LLM_BASE_URL
./run.sh                      # or .\run.ps1 on Windows

Starts a simulated 8-channel EEG stream, a live EEG scope, the room, and the agent. Press A in the room to "close the participant's eyes" — posterior alpha rises (the Berger effect), the agent notices, and acts. Press N to return to the noise baseline. Press H for controls and live LLM/EEG connectivity checks.

The scientifically interesting bit: run the agent with --gate and give it a noise-only signal. It abstains instead of confidently narrating a story over a coin-flip — the failure mode this whole design exists to prevent.

Requirements

• demos/ — Python 3.10+ with PsychoPy (its standalone interpreter is easiest), plus openai and pylsl. The psychopy_showcase/ track needs only requests.
• capstone_smart_home/ — its own Python 3.12 venv; see its README. The launcher scripts create/find it for you.

Guardrails, in layers

1. Schema-constrained JSON — the model cannot emit an off-spec action.
2. Allow-list + range clamping — an action must be valid for the object being faced.
3. Confidence gating on the SIGNAL (not the model's self-reported number — small models just say 0.9). If the EEG window is not reliably above baseline noise, the agent abstains.
4. Hysteresis / act-once semantics — it acts once per genuine change of state, not on every tick.
5. The LLM is off the timing-critical path, and every decision is logged with full provenance.

Attribution

Room artwork: LimeZu, "Modern Interiors" (CC-BY) — see capstone_smart_home/assets/limezu/README.md. Keep the attribution with the art.

License

See LICENSE. The artwork is licensed separately (CC-BY, as above).