💡 Warm-Start Voice Turns: Persistent Agent Session and Turn Telemetry to Hit the 3-Second Voice Target

[github-actions[bot]]

Summary

Every TalkTerm turn currently calls sdk.query() again in single-message mode, which starts a fresh Claude Code process for each message. The SDK vendor's recommended alternative is streaming input mode: one long-lived session that accepts queued messages and real-time interruption. This idea moves TalkTerm to that mode, pre-warms the agent process during app launch, and adds turn-latency telemetry (speech end → first text event → first audio). NFR1a and NFR1b are then measured, not assumed.

Market Signal

• The vendor recommends it: "Streaming input mode is the preferred way to use the Claude Agent SDK… It allows the agent to operate as a long lived process that takes in user input, handles interruptions…", and single-message mode "does not support… real-time interruption" (SDK docs: Streaming Input).
• New warm-start primitives: TS SDK 0.3.282 added prewarm() and SpareProcess.claim() (alpha) plus a smaller /core entry point. The docs show startup() to pre-warm before the first query() (TS SDK CHANGELOG, DeepWiki: performance).
• Cold start is a real problem: a community report describes an ~80-second first message on Windows with query() (issue).
• Measurement is now cheap: Claude Platform launched OpenTelemetry span export from its SDKs on Oct 14, 2026 (release notes).
• Competitors set the bar: ChatGPT desktop voice runs on full-duplex GPT-Live (community post), and Rime Prism reports 0.57s to first audio (Voice AI News W44). A voice-first product that feels slow loses to that bar.

User Signal

• PRD NFR1a: avatar acknowledgement within 1s, "as measured by in-app telemetry". NFR1b: first spoken segment within 3s at p95. NFR4: avatar ready within 3s of cold launch. No telemetry for these exists yet.
• claude-sdk-backend.ts builds new options and calls sdk.query({ prompt: message, options }) inside a retry loop on every sendMessage.
• Existing voice ideas (💡 Sub-40ms Conversational Voice via Cartesia Sonic: Perceptually Instant Avatar Speech #432 Cartesia, 💡 Intelligent Turn-Taking: Breath-and-Pause-Aware Voice Pipeline for Natural Conversation Flow #502 turn-taking, 💡 Full-Duplex Voice Architecture: Simultaneous Listen-and-Speak Pipeline as GPT-Live Competitive Response #522 full-duplex) optimize what happens after the model starts streaming. The agent's cold start comes before all of them.

Technical Opportunity

• No port change: AgentBackend.sendMessage() stays the same. Inside ClaudeSdkBackend, a per-session AsyncIterable<SDKUserMessage> queue feeds one live query(). cancelCurrentAction() becomes interrupt() on that query, which TalkTerm already calls at line ~306.
• Pre-warm: start the session while the avatar greeting plays (after workspace selection), so the first user utterance lands on a warm process.
• Telemetry: timestamp voice:speech-ended, the first text AgentEvent and the first TTS onstart. Write p50/p95 through the audit/telemetry repository, and optionally export OTel spans for dev builds.
• Synergy: streaming input is the prerequisite for voice steering during execution (💡 Voice-Steered Agent Execution: Real-Time Conversational Redirection During Multi-Step Workflows #493) and for image input (screenshots, 💡 Screenshot Context: Point-and-Tell Visual Interaction Mode #246).

Assessment

Dimension	Score	Rationale
Feasibility	med	Streaming input works in the current SDK. prewarm() needs the 0.3.x upgrade. The process lifecycle needs careful tests with FakeAgentBackend.
Impact	high	Response speed is the core promise of a voice-first avatar and the PRD's top NFR.
Urgency	med	No user reports yet, but competitors are setting expectations now and Windows cold start is documented.

Adversarial Review

Strongest objection: This is premature optimization. Model time to first token probably dominates, and a long-lived process adds new failure modes: crashes, stale permission state, memory growth across long sessions.

Rebuttal: Phase 1 is measurement only. If process spawn and initialization are under 10% of time to first audio on Windows and macOS, stop and focus on TTS. Streaming input is not a niche hack. It is the vendor's recommended mode and unlocks interruption and image input, so the lifecycle work pays off several times. Crash handling reuses the existing resumeSession path with the persisted SDK session ID.

Suggested Next Step

1. Add the three timestamps to the voice pipeline and collect p50/p95 over 20 scripted turns on a baseline Windows laptop and an M-series Mac.
2. If spawn time is significant, spike streaming-input mode in ClaudeSdkBackend behind the existing contract tests, then evaluate prewarm() as part of the SDK upgrade.