Add native Bark TTS model family - #527
DrewThomasson wants to merge 6 commits into
Conversation
garbage Bark Small audio.cpp demoGenerated by the native |
|
Still garbage (generated after semantic history sequence fix): download MP4 |
|
still garbadge |
|
Verified Bark fix (commit |
|
@DrewThomasson I'd like to foucs on this PR first. The current PR doesn’t follow the established patterns used by other models. Simply reusing graphs and managing their allocation lifecycle correctly makes the model 1.44× faster. Further adding the framework’s AR runtime with managed KV caching brings the overall speedup to 10.75× compared with the original PR (RTF 2.2 to 0.2). This speedup can be achieved without extensive code-level optimization. The framework components are designed to be reusable and handle complex runtime logic, such as graph lifecycle and KV cache management, so you can focus on the model-specific implementation/optimization.
A and B produce bit-identical output. A and C produce different waveforms, but this drift is expected with cached decoding. Would you like to give this approach a try? It could be a good learning experience. I’m also happy to take over the PR if you prefer. Let me know! Also please resolve the conflicts in the HF repo. |
Summary
bark_ttssupport for Suno Bark SmallValidation
bark_tokenizer_testbark_codec_parity: cosine similarity 0.99999976 vs Hugging Face on 32 framesaudiocpp_clisynthesis at 24 kHzpython3 tools/check_loader_catalog_sync.pyThe generated demo is attached in the companion model PR and linked below.