A voice interface for your coding agents. Built at Ship Systems, shared as open source.
Speak to native Codex and Claude Code sessions on your Mac. The local service provides speech input/output, a session dashboard, and terminal/MCP speech tools.
Dashboard · Repository · Issues · Validation and limitations
An open-source developer preview for macOS, installed from source. Deepgram speech and native Codex continuity have live validation; other provider and session limits are described below. See validation and known limitations before relying on an unverified workflow. No packaged release is available yet.
Requires macOS, Git, Node 22.18 or newer, and an installed/authenticated native
codex or claude CLI for the agent you want to use. Tested locally with Node 26.5.1.
Bring your own speech-provider API key; speech usage and the native agent's account
requirements are separate. Provider charges apply.
git clone https://github.com/nolanmak/Agentflow.git
cd Agentflow
npm ci
npm run build
mkdir -p "$HOME/.local/bin"
ln -sf "$PWD/dist/src/cli.js" "$HOME/.local/bin/agentflow"
node dist/src/cli.js service install
node dist/src/cli.js openAdd ~/.local/bin to your shell PATH to use agentflow. The install command enables login startup. The microphone remains off until explicitly started.
In the dashboard, open Speech & settings, add your provider key, select speech input/output providers, and save your preferences. Deepgram is the currently live-tested speech option. Start a managed session with one of the commands below; approve native agent trust and permission prompts in its terminal. Keep this checkout in place: the CLI symlink and login service reference its built files.
agentflow run codex # native terminal shared with the dashboard
agentflow run claude
agentflow run codex --resume NATIVE_SESSION_UUID
agentflow speak --text "The tests passed."
agentflow speak --speed 2 --text "Say this twice as fast."
printf 'The tests passed.\n' | agentflow speak
agentflow service status
agentflow service restart
agentflow service uninstallCtrl+] detaches an Agentflow terminal view and leaves its native process running. Service restart/stop terminates processes that service owns, so detach does not mean restarting the service is safe for an active turn.
Agentflow-managed sessions have one native CLI process with terminal and browser views. User transcripts go unchanged into that process; no separate inference model or voice persona is inserted. Native tools, settings, cwd, and approvals remain in the native CLI. Approve native permission/trust prompts in its terminal panel.
Already-running external Codex sessions accept voice through native codex queue; the existing terminal stays open. This was tested live. Raw screen mirroring is available for Agentflow-managed terminals; external sessions synchronize conversation history.
Already-running external Claude sessions do not yet have a verified equivalent user-input attachment. Claude's private peer inbox adds a peer envelope, so it is deliberately excluded from production voice input. It does not satisfy exact user-message parity. The existing process is never silently stopped or forked. Claude launched with agentflow run claude uses its actual native terminal input. Claude's local weekly usage limit prevented a successful model-response test.
The transcript preview shows the last 40 readable native messages with original timestamps. It is not fed back as a replacement for native context. Native compaction/context limits still apply.
With a Deepgram key configured, Speech & settings provides 41 named Aura-2 English voices, a Preview voice button, and playback-speed choices from 0.75× through 2×. Select a voice and speed, preview them without saving, then Save preferences. Helena is the default for new configurations; saved preferences are retained. Voice catalogs for other providers are deferred.
Speech & settings selects STT and TTS independently: Deepgram, OpenAI, ElevenLabs, or 9Router. Keys are stored in macOS Keychain (com.agentflow.speech); the browser receives configured/not-configured status only. Keys may also be supplied through explicitly configured server environment variables. No keys belong in this repository.
Deepgram STT/TTS were tested live using the authorized local key. OpenAI, ElevenLabs, and 9Router pass mocked request/response tests; working live credentials/routes were not established. 9Router's inspected speech catalogs were empty. Agentflow does not start its Docker container automatically.
Hands-free browser mode records an utterance, transcribes it, sends it to the native agent, plays the reply, then listens again. It is half-duplex; use Interrupt to stop speech. Physical microphone/echo acceptance remains outstanding.
The MCP server provides speak, current_time, and session_history. Register it with each native CLI using absolute executable/file paths:
codex mcp add agentflow -- /absolute/path/to/node /absolute/path/to/Agentflow/dist/src/cli.js mcp
claude mcp add --scope user agentflow -- /absolute/path/to/node /absolute/path/to/Agentflow/dist/src/cli.js mcpOfficial Codex MCP configuration. Refresh tools using the native client's supported flow when adding tools to an already-running session; Agentflow never closes it automatically.
Ask: “Use Agentflow to read your answer aloud at 2× speed.” The speak tool plays through the Mac's audio device even with the dashboard closed. Its optional speed argument applies to that response only; use 2 for twice normal playback. No skill is required. Keep using your existing dictation for input. If the running agent has not loaded the MCP tool yet, ask it to run agentflow speak --speed 2 using its shell tool. Issue #17 scopes this minimal CLI/tool path with objective tests; custom terminal microphone capture and automatic speech modes are deferred.
npm test
npm run typecheck
npm run test:e2e # starts isolated test server; mocked speech/agent API
node dist/scripts/live-pipeline.js # opt-in paid Deepgram + disposable native Codex
npm run test:live # opt-in real speech and native Codex queue checksnpx playwright install chromium installs the browser test runtime. Live checks use synthetic prompts and disposable sessions, not personal conversations. See the plan, architecture, TDD policy, and backlog.
Handwritten application, browser, scripts, and tests are strict TypeScript. npm run build emits JavaScript and source maps under ignored dist/; Node runs those compiled files without TypeScript tooling. npm run dev builds and starts the service. CI uses Node 26.5.1 and runs typecheck, compiled unit/integration tests, and Playwright.
For existing installations, see the compiled-entrypoint migration. Agentflow is released under the MIT License. The public-release privacy review and the owner-accepted historical identity metadata are documented in PRIVACY.md.
Bug reports, documentation improvements, and pull requests are welcome. Start with CONTRIBUTING.md and the issue tracker. Use synthetic sessions in reproductions and remove private data from logs and screenshots. For sensitive reports, follow SECURITY.md.
Agentflow is licensed under the MIT License.
Select Local Whisper under Speech & settings → Listening. It runs speech recognition on your Mac with no API key or 9Router. Speaking still uses the independently selected TTS provider, and Codex/Claude retain their own model routing; local transcription does not make an agent conversation fully offline.
Install ffmpeg, Python 3, and the pinned whisper-server runtime described in local runtime setup. Then explicitly prepare and start the engine:
agentflow local-whisper setup
agentflow local-whisper start
agentflow local-whisper statusThe default URL is http://127.0.0.1:17864. To use an existing engine, enter its loopback HTTP origin in Local Whisper URL. The model setting is a label for the server's loaded model; changing it does not load a different model. Models are never downloaded while recording or during login. agentflow local-whisper stop only stops the engine owned by Agentflow. The helper accepts import-model PATH --sha256 HASH for an explicitly trusted local model; use --help for options. Agentflow's helper state lives beneath $AGENTFLOW_HOME/local-whisper (default ~/.config/agentflow/local-whisper).
Browser WebM/MP4/WAV recordings are converted in memory by ffmpeg to 16 kHz mono PCM WAV, with a 90-second duration limit, 20 MB input limit and 15-second conversion deadline. Set AGENTFLOW_FFMPEG to an absolute executable path if needed. No conversion files are retained. The local endpoint cannot contain remote hosts, credentials, paths, queries or redirects, and its transport bypasses global HTTP proxies. Silent/near-silent recordings do not reach Whisper. Very quiet speech may need a closer microphone.
Each recording captures its listening settings before microphone permission is requested. Canceling or changing sessions discards late transcription and cannot submit it to the agent. Canceling closes local transport and terminates conversion, but whisper-server may continue inference internally; this is not a claim that HTTP disconnect preempts GPU computation. Inference has a 60-second timeout; check agentflow local-whisper status, the server's loaded model and the configured URL after engine errors.
Tests and TDD evidence: local transcription validation. Real microphone permission, audible playback and sleep/wake still require device acceptance.
Use Local Kokoro for keyless, offline voice output, independently of local Whisper input. Explicit setup installs the verified model and CPU runtime; select Local Kokoro in Speaking settings. Native agent reasoning remains independently configured.