[nolanmak]

Agentflow can now use local Whisper for speech input without a provider key or 9Router. The new provider converts browser audio locally, sends WAV only to a literal loopback endpoint, and preserves TTS and native-agent routing. Recording settings are snapshotted; canceled or superseded results cannot submit a turn.

Includes explicit checksum-verified model setup/import and owned engine lifecycle commands. Tests reproduce and fix malformed-HTTP crashes, overlapping transcription, out-of-order microphone permissions, and stale permission failures. Browser recording stops at 89 seconds to leave margin below the 90-second decode limit.

Validation: 53 Node tests, 14 Playwright tests, nine runtime tests, typecheck/build, and real WAV/WebM/MP4/noisy/silent adapter runs with external networking denied. A 90-second fixture transcribed locally; oversized audio failed clearly. Reproducible synthetic quality/latency evidence is in docs/local-whisper-qa.md.

Physical microphone/device acceptance remains tracked in #23. This is local STT; TTS and native Codex/Claude reasoning remain independently configured. Local TTS remains follow-up #24. HTTP cancellation cannot guarantee immediate whisper-server compute interruption.

The managed supervisor receives its control token through a bounded private pipe, verified against the actual process list. Successful-engine fixtures allow cold runner startup while deliberate timeout tests retain their short failure deadline.

Closes #21
Closes #22
Refs #23, #24

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 5561b164-cfb5-4c02-92c2-0703256c5b48

📥 Commits

Reviewing files that changed from the base of the PR and between fce5c15 and f9adf83.

📒 Files selected for processing: 26

• .github/workflows/ci.yml
• README.md
• docs/local-whisper-qa.json
• docs/local-whisper-qa.md
• docs/local-whisper-runtime.md
• docs/local-whisper-validation.md
• public/app.ts
• public/contracts.ts
• public/dom.ts
• public/index.html
• scripts/benchmark-local-whisper.py
• scripts/build-assets.ts
• scripts/local-whisper-corpus.json
• scripts/local-whisper-offline.sb
• scripts/local-whisper.py
• scripts/test_local_whisper.py
• src/cli.ts
• src/local-audio.ts
• src/local-http.ts
• src/server.ts
• src/speech.ts
• src/state.ts
• src/store.ts
• test/browser/dashboard.spec.ts
• test/build.test.ts
• test/local-whisper.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

This change adds Local Whisper as a transcription-only provider, with loopback endpoint validation, local audio conversion, and a managed whisper.cpp runtime. Browser requests capture STT settings per recording and ignore stale results. New tests, benchmark materials, QA reports, and setup documentation cover the implementation.

Changes: Local Whisper

Layer / File(s)	Summary
Managed runtime setup and lifecycle scripts/local-whisper.py, scripts/test_local_whisper.py, src/cli.ts, scripts/build-assets.ts, docs/local-whisper-runtime.md, test/build.test.ts	Adds model setup and checksum-verified imports, plus start, status, and stop commands for an owned whisper.cpp runtime. The CLI bundles and launches the Python manager. Runtime tests cover model integrity, readiness, lifecycle, and port conflicts.
Provider settings and selection public/contracts.ts, src/state.ts, src/store.ts, public/app.ts, public/dom.ts, public/index.html, test/local-whisper.test.ts, test/browser/dashboard.spec.ts	Adds Local Whisper provider settings and loopback URL validation. The settings UI saves its STT URL and excludes the provider from TTS and API-key controls. Tests cover persistence and provider restrictions.
Local transcription and recording isolation src/local-audio.ts, src/local-http.ts, src/speech.ts, src/server.ts, public/app.ts, test/local-whisper.test.ts, test/browser/dashboard.spec.ts	Normalizes audio to bounded WAV data and sends inference requests through direct HTTP transport. The server accepts validated STT snapshots. Recording requests use captured settings and cancellation state; tests cover stale requests, microphone races, and transport behavior.
Benchmark, QA, and integration documentation scripts/benchmark-local-whisper.py, scripts/local-whisper-corpus.json, scripts/local-whisper-offline.sb, docs/local-whisper-qa.*, docs/local-whisper-validation.md, README.md, .github/workflows/ci.yml	Adds a synthetic audio benchmark and corpus, QA results and limitations, validation notes, setup instructions, and CI coverage for the runtime tests.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Browser as Browser recording
  participant API as Agentflow transcription API
  participant Normalizer as Local audio normalizer
  participant Engine as Local whisper.cpp server
  Browser->>API: Submit audio with captured STT settings
  API->>Normalizer: Convert audio to bounded WAV
  Normalizer-->>API: Return WAV or silence result
  API->>Engine: POST WAV to loopback /inference
  Engine-->>API: Return transcription response
  API-->>Browser: Return transcript or error

Loading

Merge Risk: ⚪ Minimal · up to f9adf

The supported tests and managed-engine startup are not blocked by the two reported issues. Normal validation remains appropriate before merging.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 1.89% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 53 functions across 17 files. (9 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Linked Issues check		The PR addresses the coding requirements in [#21] and [#22]. It adds local-whisper contracts, loopback-only /inference requests, local WebM/MP4/WAV normalization, per-recording settings snapshots, c…
Out of Scope Changes check		The changes remain within [#21] and [#22]. The added UI, server contracts, audio conversion, local HTTP transport, CLI manager, runtime tests, benchmark, fixtures, and QA documentation support local S…
Title check		The title clearly identifies the main changes: local Whisper speech input and managed model setup.
Description check		The description covers the outcome, linked issues, acceptance evidence, validation results, and limitations. It does not use every template heading explicitly, but it provides the required information…

Full details: Docstring Coverage

Explanation

Docstring coverage is 1.89% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 53 functions across 17 files. (9 skipped: 9 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}