Skip to content

Repair contextual dictation and mobile speech flow - #601

Merged
witbrock merged 1 commit into
mainfrom
fix/JVNAUTOSCI-888-contextual-dictation
Sep 10, 2026
Merged

witbrock merged 1 commit into
mainfrom
fix/JVNAUTOSCI-888-contextual-dictation

Conversation

@witbrock

@witbrock witbrock commented Sep 10, 2026 •

Copy link
Copy Markdown
Member

Merge decision: ready — recorded dictation now supplies conversation vocabulary, preserves drafts through retry/cancellation, and avoids mobile recognition accumulation and playback workarounds.

User outcome

Repairs the repeated progressively longer prefixes and replayed previous dictation observed in the DGX “Von Task Smoke Test”. A microphone recording produces an editable transcript; Send finishes transcription first, failures retain retryable audio, and late results cannot alter another conversation.

Material changes

  • Add authenticated MP4/WebM transcription using an enabled OpenAI audio model and bounded visible conversation/draft/concept-name context.
  • Add shared composer capture/transcribe/retry/cancel states; use browser recognition explicitly and handle its results as revised snapshots.
  • Exclude supported transcription models from chat fallback/selection candidates.
  • Restrict Chromium TTS pause/resume to desktop and keep mobile playback within the tap gesture.
  • Document the runtime contract; current streaming/turn-taking plan is in JVNAUTOSCI-812, bounded delivery in JVNAUTOSCI-888.

Evidence

  • 13 speech/frontend tests passed; neighbouring chat/presenter checks passed (30 tests).
  • 13 targeted backend tests passed with VON_USE_MOCK_DB=1: actual upload handler, normal server session binding, model denial, MP4/WebM/context/language transport, safe errors, and three chat-model candidate boundaries.
  • A broader backend run reached 36 passes before being interrupted on unrelated database I/O; it is not claimed complete.
  • Frontend static lint and diff checks passed.
  • Chrome desktop and 390 × 844 viewport exercised the actual composer/template and controller with simulated microphone/provider: failed upload → edit draft → retry preserved the edit and inserted one transcript; cancellation preserved the draft; no mobile horizontal overflow.

Ship boundary

Minimum ship criteria are met for this bounded transport/lifecycle repair. Lost or duplicated draft, stuck microphone, cross-actor context access, or audio-only models entering chat fallback would block delivery. Physical iOS/Android, acoustic accuracy and runtime deployment are separate receipts, not established by browser emulation. Raw DGX microphone audio and recognition events were not retained; the stored transcript and code establish the accumulation failure class.

Streaming hands-free conversation, server TTS, wake words, paired devices and persistent raw-audio retention are outside this PR. Existing primary models and credentials are preserved; a transcription model must be added as an additional authorised model before runtime use.

@witbrock
witbrock merged commit 702530d into main Sep 10, 2026
2 checks passed
@witbrock
witbrock deleted the fix/JVNAUTOSCI-888-contextual-dictation branch September 10, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant