Skip to content

Local Whisper STT provider with browser audio normalization and session-safe submission #21

Closed

Description

@nolanmak

Outcome

Select local-whisper independently of TTS and transcribe browser utterances locally without keys or 9Router. Native agent model routing remains unchanged.

Implementation

Add local-whisper to shared contracts, settings and speech dispatch. Use whisper.cpp /inference (not the OpenAI /v1/audio/transcriptions path). Convert supported browser WebM/MP4 and WAV audio to engine-compatible WAV locally with bounded, cancelable processing.

Acceptance criteria

  • Local STT selection/endpoint persist and require no API key or 9Router.
  • Actual browser recording formats and WAV normalize correctly and yield real transcripts.
  • Local endpoints are loopback-only; redirects and cloud fallback are prohibited.
  • Switching STT does not change TTS, native session ID, agent model or permissions; selection is snapshotted per utterance.
  • Success submits exactly once; silence, canceled/stale inference and failure submit nothing. Retry never duplicates native turns.
  • Timeout, missing engine/model, malformed output and conversion failure have actionable recoverable errors.
  • No audio/transcript/credential leakage into logs or retained conversion files.
  • Existing cloud speech and native-session contracts remain green; relevant browser E2E passes.

Required TDD workflow

For each behavior: write and run a focused failing test; confirm the failure is for missing behavior; implement minimally; refactor while green. Record actual red/green commands and outcomes. Use deterministic fixture transports/processes for contracts plus a real pinned engine/model smoke test. No personal sessions or recordings as test fixtures.

Completion evidence

Record commit/PR, test commands, engine/model versions and checksums, Mac hardware, real-engine results, and any unverified manual-device gates. Do not mark unverified acceptance criteria complete.

Implementation evidence

Implemented in #25. Automated acceptance, real offline engine results, and reproducible corpus are recorded in docs/local-whisper-qa.md and the PR. Native physical-device acceptance remains in the separate QA issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions