kokoro_tts: stop throwing on phonemes the vocab lacks, and allow a supplied phoneme stream

[christopherthompson81]

Review copy on the fork, not a PR to upstream. Related issue upstream: 0xShug0#556.

Two commits, separable — the first is a bug fix, the second a small feature. They can go upstream independently.

---

1. kokoro_tts: skip phonemes the vocab has no id for, as KModel does

encode_input_ids_and_count threw on any symbol missing from Kokoro's 114-entry vocab — including symbols the model's own G2P had just produced, which aborted the whole request.

The reference implementation does not. hexgrad/Kokoro's KModel tokenizes with input_ids = list(filter(None, map(vocab.get, phonemes))) — it drops phonemes with no token id. Dropping gives a correct reading; throwing gives no audio at all.

What it actually costs, measured

40 real sentences per language through the release kokoro_82m_q8_0, before the fix:

language	result	symbol
en-us / en-gb	40 ok / 0 threw	—
es	35 ok / 5 threw	« U+00AB
fr-fr	34 ok / 6 threw	« U+00AB
it	39 ok / 1 threw	̪ U+032A
hi / pt-br	40 ok / 0 threw	—

So ~12% of Spanish and ~15% of French prose failed outright. English is affected too but rarely — eSpeak glottalises /t/ before a syllabic nasal, so button, kitten, written, forgotten produce a U+0329 the vocab lacks (100 of the first 40,000 dictionary words, but one anywhere aborts the request).

Malformed UTF-8 above still throws — that is a real error. An unknown but well-formed phoneme is not.

A debug trace records each skip (kokoro.skipped_phoneme) so the information is kept without failing the request.

---

2. kokoro_tts: accept a supplied phoneme stream, bypassing the built-in G2P

Adds a phonemes request option. When set, the string is synthesized as-is instead of running eSpeak-ng over the text.

The built-in G2P is one opinion about pronunciation, and a caller may have a better one for its material: a lexicon the engine does not carry, a language it does not cover, a domain vocabulary, or a pronunciation the application has already shown its user and must now speak the same way. Today there is no way to express any of that — phonemize_text() is unconditional and the option validator rejects anything undeclared, so the door is bolted rather than merely shut.

Text is still required and its language must still agree with the voice; only the phonemization is replaced.

Three details worth review

• Not chunked. Chunking splits the text, and nothing in the engine knows where the matching cut points in a caller's phoneme stream are — only their G2P does. A supplied stream runs whole, and the existing 510-symbol guard tells the caller to split it, with a message that distinguishes the two cases rather than reporting a framework limitation to someone who can simply send less.
• The run cache is keyed on the supplied stream. It keys on text today; without this, two requests with the same text and different phonemes hit the same entry and the second is spoken as the first.
• prepare() sizes the graph on the unchunked request when phonemes are supplied, since that is what run() will feed it in a single pass.

⚠ The declared option set is embedded in the GGUF at conversion time, so existing model files need --model-spec-override (or re-conversion) before they will accept it. A package converted with prepare_kokoro_gguf.py after this change declares it natively — verified.

---

Verification

Built and run against kokoro_82m_q8_0 plus a locally converted fully-bundled multilingual package.

Every previously-failing case now renders, with durations in line with words that always worked (button 1.00 s beside hidden 1.07 s), and 114/114 distinct English golden sentences render.

Existing suites, against the patched engine:

gate	result
C# bindings model test	ran=4 skipped=1 failures=0
audiocpp_c_api_path / _exports	Passed
audiocpp_c_api_model	Passed (146 s)
audiocpp_c_api_parity	PARITY OK across 5 families

kokoro_tts audio MATCH cli=79800@24000Hz peak=0.3383 api=79800@24000Hz peak=0.3383

79800 frames at 24 kHz is 3.325 s — byte-identical to what the unpatched engine produced for that sentence, and identical between the CLI and the C API. The fix is inert on text that never had an out-of-vocab symbol.

Supplied phonemes, with an external G2P's stream for the same text:

declared request options: language, seed, phonemes, text_chunk_size
supplied phonemes -> 2.77s     built-in G2P -> 2.48s
cache check: same text, two streams -> 34800 vs 35400 samples, identical=False
overlong: Kokoro phoneme string exceeds 510 symbols; supplied phonemes are not
          chunked, so split them across requests

Notes for the reviewer

• These commits were developed against pin 3eccab5 and cherry-picked onto this fork's main (b46fe6b). All four touched files, and every dependent header (options.h, trace.h, chunking.h, session.h), are byte-identical between the two bases, so the build and test results above carry over unchanged.
• The per-language numbers were measured by temporarily restoring the throw in a patched build and running real corpus text — not by diffing espeak-ng --ipa output against the vocab. That proxy was misleading in both directions: it predicted 507 hyphen failures in French that never occur, and missed the guillemets that actually fail. audio.cpp drives eSpeak through its own caret-tied IPA integration, so the CLI's output is not what the engine encodes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EkxqpYvUbjCpRDnFiNiVfx

[christopherthompson81]

Superseded by a split, per review: the two changes have very different review surfaces.

• kokoro_tts: skip phonemes the vocab has no id for, as KModel does #8 — the bug fix alone (skip phonemes the vocab has no id for). No ABI change, no new option; it only stops the encoder throwing on symbols its own G2P produced.
• a follow-up PR — the external-phonemizer option, which will propose a proper array-valued entry point in the C ABI rather than the delimiter-encoded string this PR carried.

The delimiter version worked and is verified (one call with caller-chosen chunks produced byte-identical samples to three separate calls concatenated), but encoding an array inside a string was a workaround for the option channel being string -> string at every layer, not the design worth sending upstream.