kokoro_tts: stop throwing on phonemes the vocab lacks, and let a caller supply its own

[christopherthompson81]

Review copy on the fork, not a PR to upstream. Upstream issue for the first commit: 0xShug0#556.

Four commits: a bug fix, an ABI addition that uses it, a versioning rule, and a correction to the first commit's scope that review surfaced.

---

1. kokoro_tts: skip phonemes the vocab has no id for, as KModel does

encode_input_ids_and_count threw on any symbol missing from Kokoro's 114-entry vocab — including symbols the model's own G2P had just produced — aborting the whole request.

The reference implementation does not. hexgrad/Kokoro's KModel tokenizes with input_ids = list(filter(None, map(vocab.get, phonemes))): it drops phonemes with no token id. Dropping gives a correct reading; throwing gives no audio at all. Malformed UTF-8 above still throws — that is a real error; an unknown but well-formed phoneme is not. Each skip is traced (kokoro.skipped_phoneme).

⚠ This leniency applies to the engine's OWN G2P output only — see commit 4, which was a correction found in review.

What it costs today, measured

40 real corpus sentences per language through the release kokoro_82m_q8_0, throw still in place:

language	result	symbol
en-us / en-gb	40 ok / 0 threw	—
es	35 ok / 5 threw	« U+00AB
fr-fr	34 ok / 6 threw	« U+00AB
it	39 ok / 1 threw	̪ U+032A
hi / pt-br	40 ok / 0 threw	—

~12% of Spanish and ~15% of French prose failed outright — the vocab carries ( and ) but not the guillemets. English is affected more rarely: eSpeak glottalises /t/ before a syllabic nasal, so button, kitten, written, forgotten produce a U+0329 the vocab lacks.

---

2. capi: list-valued request options, and Kokoro's external phoneme stream

The ABI half completes something already half-designed

The model spec has declared string_list / float_list / path_list / audio_path_list since schema v1 and validates their declarations — but nothing carries one at run time. Options are string -> string from the ABI down to TaskRequest::options, so a family wanting a list had no transport, and no spec has ever used the types.

AUDIOCPP_API audiocpp_status audiocpp_request_set_option_array(audiocpp_request * request,
                                                               const char * key,
                                                               const char * const * values,
                                                               size_t count);

ABI minor 1 → 2: additive, so a caller built against 0.1 keeps working.

• Values are copied; neither the array nor the strings need outlive the call.
• A second call with the same key replaces rather than appends, matching set_option's assignment semantics.
• count == 0 sets an empty list — distinct from never setting the key.
• Null elements are rejected before anything is written, so a bad element cannot leave the option half-assigned.
• Lists live in their own map (TaskRequest::option_arrays), so a family reading single-valued options cannot see a half-formed list and vice versa; which one it reads is declared by the type in its spec.
• validate_spec_backed_request_options gains an overload — an undeclared list key is rejected exactly as an undeclared scalar one is, and every existing call site is untouched.

The first user: Kokoro's phonemes

The built-in G2P is one opinion about pronunciation, and a caller may have a better one for its material: a lexicon the engine does not carry, a language it does not cover, a domain vocabulary, or a pronunciation the application has already shown its user and must now speak the same way.

The list is the point, not a convenience. Chunking splits the text, and nothing in the engine knows where the matching cut points in someone else's phoneme stream are — only their G2P does. So the caller supplies the chunks, one per entry, rendered in order and merged exactly as text chunks are. A caller whose document exceeds the 510-symbol limit still makes one call and gets one buffer back.

Two details worth review:

• The run cache is keyed on the entry. It keys on text today, and with a list every chunk shares one text — so that key is the only thing telling them apart.
• prepare() sizes the graph on the largest entry, since with the caller's chunking the text boundaries are not the ones run() will use.

⚠ The declared option set is embedded in the GGUF at conversion time, so an existing model file needs --model-spec-override before it will accept the option. A package converted with prepare_kokoro_gguf.py after this change declares it natively — verified.

---

3. capi: say what the minor version means, and make it mean it

audiocpp_abi_version() has always returned a triple, but only major had a documented meaning — and minor had never moved, including across 0xShug0#544, which added four entry points. A three-field version where two never change cannot answer the question a caller has.

Minor now increments when entry points are added; nothing is removed or changed, so a caller built against a lower minor keeps working, and one that needs a newer entry point can require a minimum. Patch is behaviour-only and must not be gated on.

A C caller can of course resolve the symbol and test for NULL, which is exact and needs no version. The field is for callers that cannot: a binding declaring its imports up front — C#, JNA, ctypes with prototypes — binds on first use and raises a missing-symbol error from inside the call.

⚠ Recorded rather than papered over: 0.1 covers two different surfaces, because 0xShug0#544's four entry points shipped without a bump. From 0.2 the minor is the answer.

4. kokoro_tts: validate a caller's phonemes, keep dropping our own

Found in review, and it reverses part of commit 1's reasoning. "Drop what the vocabulary lacks" is right for the engine's own output and wrong for a caller's.

Canonical IPA writes a diphthong as two symbols where Kokoro writes one, so an off-glide (U+1DA6, U+1DB7) is not in the 114-symbol vocabulary. Dropping it renders lˈaᶦk as lˈak:

canonical IPA : aᶦ lˈaᶦk ðə pɹˈaᶦs ʌv ðə wˈaᶦt ɹˈaᶦs .
glides removed: a  lˈak  ðə pɹˈas  ʌv ðə wˈat  ɹˈas .
IDENTICAL -> True   (55800 samples both)

"I like the price of the white rice" becomes "a lack the pras uv the wat ras", byte-identical to having written it that way, with no error.

So the rule depends on who produced the stream, because you can only demand a correction from someone able to make one:

source	behaviour	why
the engine's own G2P	drop, as KModel does	eSpeak emits a syllabic mark for button that the vocab has no id for; nothing downstream can fix it
a caller's supplied stream	refuse, naming the symbol and the likely cause	they can fix it, and silence costs them wrong words instead of an error

canonical IPA -> refused: Kokoro vocab is missing phoneme symbol: ᶦ; supplied phonemes must
                 be in Kokoro's own 114-symbol vocabulary, which is not canonical IPA -- a
                 diphthong is one symbol there and two in IPA, so an off-glide is a common cause
kokoro-vocab  -> OK 2.30s
built-in 'button' -> OK 1.43s (own G2P still lenient)
mixed list    -> refused on the bad entry

Only the offending entry is named, so a caller supplying a list knows which one to correct. Tests cover both directions, since the asymmetry is exactly what a later reader tidies into consistency.

Verification

ABI version = 0x200  (major 0, minor 2)
one call, 3 list entries -> 9.00s
three calls, summed      -> 9.00s
identical samples?       -> True
newline inside an entry  -> 5.17s (dropped as out-of-vocab, not split)
overlong entry           -> Kokoro phoneme chunk exceeds 510 symbols; split the supplied
                            phonemes across more list entries
empty list               -> accepted (no chunks -> falls back to text)
undeclared key           -> rejected: unknown Kokoro TTS request option: not_a_real_option
built-in G2P untouched   -> 2.35s

One call with three caller-chosen entries is byte-identical to three separate calls concatenated. An earlier draft encoded the list as a newline-delimited string; that worked, but a real array means a newline inside an entry is plain data rather than a boundary, which the fourth line above demonstrates.

ctest, against the patched engine:

gate	result
audiocpp_c_api_path / _exports	Passed
audiocpp_c_api_model	Passed (146 s)
audiocpp_c_api_parity	PARITY OK across 5 families

kokoro_tts audio MATCH cli=79800@24000Hz peak=0.3383 api=79800@24000Hz peak=0.3383

79800 frames at 24 kHz is 3.325 s — byte-identical to what the unpatched engine produced for that sentence, and identical between the CLI and the C API. Commit 1 is inert on text that never had an out-of-vocab symbol.

Every previously-failing word now renders, with durations in line with words that always worked (button 1.00 s beside hidden 1.07 s), and 114/114 distinct English golden sentences render.

Notes for the reviewer

• Developed against pin 3eccab5, cherry-picked onto b46fe6b. All touched Kokoro files and every dependent header are byte-identical between the two bases; audiocpp.h / audiocpp.cpp do differ upstream, so those hunks were applied and built against this base rather than copied.
• The per-language numbers came from restoring the throw in a patched build and running real corpus text — not from diffing espeak-ng --ipa against the vocab. That proxy misleads in both directions: it predicts 507 hyphen failures in French that never occur (espeak_text() ends with replace(out, "-", "")) and misses the guillemets that actually fail.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EkxqpYvUbjCpRDnFiNiVfx

[christopherthompson81]

Split into three, since upstream prefers small PRs:

• kokoro_tts: skip phonemes the vocab has no id for, as KModel does #10 — the bug fix alone. No ABI change, one function, and it repairs ~12% of Spanish and ~15% of French prose today. Nothing should make it wait on the rest.
• capi: list-valued request options #11 — the ABI addition. Framework only; no model family is touched. Reviewable by someone who does not care about TTS.
• kokoro_tts: accept a supplied phoneme stream, bypassing the built-in G2P #12 — Kokoro uses it. Stacked on both, and labelled as motivated by my own use case rather than dressed up as a general request.

The split was checked rather than assumed: the three branches together are byte-identical to the branch verified here (git diff against it is empty), so nothing was lost or altered in the reshuffle. #12's diff also excludes #10's hunk now, so a reviewer sees only its own work.

One commit gained since this PR: build_preparation_request was dropping the new option map, so a session prepared itself against a different request than the one run() was handed. For Kokoro that meant falling back to the built-in G2P at preparation time, and a Japanese request failing for want of UniDic even though phonemes had been supplied precisely so the G2P would never be consulted. Chinese hid it by passing — its table IS bundled, so the fallback succeeded, did the work twice, and sized the graph from the wrong count. It is fixed in #11.