Skip to content

fix(higgs_audio_tts): share one sampling seed across chunks of a request - #522

Open
mwzkhalil wants to merge 1 commit into
0xShug0:mainfrom
mwzkhalil:ci/higgs-tts-chunk-seed-consistency
Open

fix(higgs_audio_tts): share one sampling seed across chunks of a request#522
mwzkhalil wants to merge 1 commit into
0xShug0:mainfrom
mwzkhalil:ci/higgs-tts-chunk-seed-consistency

Conversation

@mwzkhalil

@mwzkhalil mwzkhalil commented Sep 12, 2026

Copy link
Copy Markdown

What this fixes

Reported in #471: Higgs TTS voice cloning "switches to a different voice" partway through longer text, and separately, generation can hit Higgs TTS generation reached max_tokens before EOC with no indication of how to fix it.

Root cause of the voice drift: HiggsTTSSession::run() splits long text into chunks (chunk_text_request) and calls generator_->generate() once per chunk. When a caller does not pass --seed, each of those calls independently resolves its own random seed:

sampling.seed = request.options.seed.value_or(runtime::random_u64_seed());

So an unseeded multi-chunk request samples every chunk against the same reference-voice conditioning but with unrelated entropy each time. That's consistent with what's reported: it gets worse with a smaller --text-chunk-size (more chunk boundaries = more independent re-rolls).

The fix

HiggsTTSSession::run() now resolves one seed up front when the request didn't supply one, and copies it onto every chunk's options before generation — the same as what already happens when a caller passes --seed explicitly (chunks already shared it via the copied options map in that case). This only changes behavior for the previously-undefined unseeded path; explicit --seed behavior is bit-for-bit unchanged.

I don't have a way to run the actual 4B GGUF weights in this environment to confirm the audio-level fix end to end, so I'd appreciate it if someone who can reproduce #471 tries this branch against the same Arabic long-form + voice-ref case. The mechanism is sound (verified by reading generator.cpp's seed resolution and session.cpp's per-chunk request construction), but I want to flag that the perceptual result is unverified on my end.

Also included

The max_tokens / EOC failure now names the configured value and points at --max-tokens / --text-chunk-size (or their server request-option equivalents) instead of leaving the caller to guess:

Higgs TTS generation reached max_tokens (2048) before EOC for this text chunk; raise it
with --max-tokens on the CLI or the "max_tokens" request option on the server, or lower
--text-chunk-size / "text_chunk_size" so each chunk needs fewer generated frames

This is a message-only change (same failure condition, no behavior change) — it was previously easy to hit with default settings on dense scripts (Arabic in the report) with no clue what to adjust.

Testing

Ref #471

Long text is split into chunks (session.cpp chunk_text_request) and
generator_->generate() runs once per chunk. When a caller does not pass
--seed, each of those calls independently draws a fresh random seed
(generator.cpp: request.options.seed.value_or(random_u64_seed())), so an
unseeded multi-chunk request sampled every chunk against the same
reference-voice conditioning but with unrelated entropy. That is a
plausible contributor to the voice/timbre drift between chunks reported
in 0xShug0#471 (Arabic voice cloning switching speakers partway through, worse
with smaller --text-chunk-size since that means more chunk boundaries).

HiggsTTSSession::run() now resolves one seed up front when the request
did not supply one, and copies it onto every chunk before generation, the
same as when a caller already passes --seed explicitly. This only changes
behavior for the previously-undefined unseeded case; an explicit --seed
still works exactly as before.

Also makes the "reached max_tokens before EOC" failure actionable: it now
names the configured max_tokens value and points at --max-tokens /
--text-chunk-size (or their server request-option equivalents) instead of
leaving the caller to guess, which was the other half of 0xShug0#471.

Ref 0xShug0#471
@0xShug0

0xShug0 commented Sep 12, 2026

Copy link
Copy Markdown
Owner

@mwzkhalil Unless I’m missing something, setting the seed won’t fix this issue. Please provide wavs generated by higgs tts on CPU with your PR applied using the test case provided in the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants