Skip to content

Add native Bark TTS model family - #527

Open
DrewThomasson wants to merge 6 commits into
0xShug0:mainfrom
DrewThomasson:feature/suno-bark-small
Open

Add native Bark TTS model family#527
DrewThomasson wants to merge 6 commits into
0xShug0:mainfrom
DrewThomasson:feature/suno-bark-small

Conversation

@DrewThomasson

Copy link
Copy Markdown
Contributor

Summary

  • add native bark_tts support for Suno Bark Small
  • implement semantic, coarse, and fine Bark transformers
  • implement the 24 kHz EnCodec decoder with parity coverage
  • add BERT WordPiece tokenizer and all 261 upstream speaker histories
  • add F16 and Q8_0 model package specs, converter, docs, and tests

Validation

  • bark_tokenizer_test
  • bark_codec_parity: cosine similarity 0.99999976 vs Hugging Face on 32 frames
  • semantic transformer top-token parity against Hugging Face
  • end-to-end audiocpp_cli synthesis at 24 kHz
  • python3 tools/check_loader_catalog_sync.py

The generated demo is attached in the companion model PR and linked below.

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

Bark Small audio.cpp demo

Generated by the native audiocpp_cli Q8_0 package at 24 kHz using v2/en_speaker_6 and the text “Hello from Bark.”

Download/play the MP4 demo

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

Fixed Bark full-sentence demo (generated after semantic history sequence fix): download MP4

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

Correction: the playable raw MP4 URL is https://raw.githubusercontent.com/DrewThomasson/audio.cpp/feature/suno-bark-small/demos/bark-small-fixed-full-sentence.mp4 (the download link above is correct).

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

still garbadge
gota fix further

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

Verified Bark fix (commit c917430). This MP4 was generated by the native audio.cpp CPU path using the corrected 703 MB quality-safe hybrid Q8 package. Input: “Hello, this is Bark.” Independent local Whisper Small English transcription: “Hello. This is bark.”

Download/play MP4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant