Skip to content

Add native Bark TTS model family - #527

Open
DrewThomasson wants to merge 6 commits into
0xShug0:mainfrom
DrewThomasson:feature/suno-bark-small
Open

DrewThomasson wants to merge 6 commits into
0xShug0:mainfrom
DrewThomasson:feature/suno-bark-small

Conversation

@DrewThomasson

Copy link
Copy Markdown
Contributor

Summary

  • add native bark_tts support for Suno Bark Small
  • implement semantic, coarse, and fine Bark transformers
  • implement the 24 kHz EnCodec decoder with parity coverage
  • add BERT WordPiece tokenizer and all 261 upstream speaker histories
  • add F16 and Q8_0 model package specs, converter, docs, and tests

Validation

  • bark_tokenizer_test
  • bark_codec_parity: cosine similarity 0.99999976 vs Hugging Face on 32 frames
  • semantic transformer top-token parity against Hugging Face
  • end-to-end audiocpp_cli synthesis at 24 kHz
  • python3 tools/check_loader_catalog_sync.py

The generated demo is attached in the companion model PR and linked below.

@DrewThomasson

DrewThomasson commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

garbage Bark Small audio.cpp demo

Generated by the native audiocpp_cli Q8_0 package at 24 kHz using v2/en_speaker_6 and the text “Hello from Bark.”

Download/play the MP4 demo

@DrewThomasson

DrewThomasson commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Still garbage (generated after semantic history sequence fix): download MP4

@DrewThomasson

DrewThomasson commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

still garbadge
gota fix further

@DrewThomasson

Copy link
Copy Markdown
Contributor Author

Verified Bark fix (commit c917430). This MP4 was generated by the native audio.cpp CPU path using the corrected 703 MB quality-safe hybrid Q8 package. Input: “Hello, this is Bark.” Independent local Whisper Small English transcription: “Hello. This is bark.”

Download/play MP4

@0xShug0

0xShug0 commented Sep 13, 2026

Copy link
Copy Markdown
Owner

@DrewThomasson I'd like to foucs on this PR first. The current PR doesn’t follow the established patterns used by other models. Simply reusing graphs and managing their allocation lifecycle correctly makes the model 1.44× faster. Further adding the framework’s AR runtime with managed KV caching brings the overall speedup to 10.75× compared with the original PR (RTF 2.2 to 0.2). This speedup can be achieved without extensive code-level optimization. The framework components are designed to be reusable and handle complex runtime logic, such as graph lifecycle and KV cache management, so you can focus on the model-specific implementation/optimization.

Stage Median gen time RTF
Original PR (A) 26.57s 2.200
with Graph/allocation reuse (B) 18.40s 1.523
with Framework AR runtime and managed KV decoding (C) 2.47s 0.205

A and B produce bit-identical output. A and C produce different waveforms, but this drift is expected with cached decoding.

Would you like to give this approach a try? It could be a good learning experience. I’m also happy to take over the PR if you prefer. Let me know!

Also please resolve the conflicts in the HF repo.

clean-A-1.wav
clean-B-1.wav
clean-C-1.wav

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants