docs: add Tesla T4 (Turing) hardware notes - #157
Open
moduvoice wants to merge 2 commits into
Open
Conversation
- Fix Manual Install Option 2/2b pip commands to include the s3tokenizer==0.3.0 and onnx==1.16.0 pins plus the protobuf upgrade, matching what start.py already installs automatically. Without these, the base ChatterboxTTS import fails with ModuleNotFoundError: No module named 's3tokenizer'. - Add predefined_voice_id to the /tts curl example (and a note explaining why it's required for the default predefined voice mode) since the example as written returns HTTP 400. - Add AGENTS.md documenting that TTS_BF16 should stay off (default) on T4/Turing (sm_75) GPUs, since is_bf16_supported() returns True there but bf16 is measured ~1.6x slower (RTF 0.535 vs 0.335) with no real hardware acceleration, and noting that the default Turbo engine ignores exaggeration/cfg_weight.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
While verifying this project's inference pipeline on a real NVIDIA Tesla T4 (16GB, Turing/sm_75 —
a common budget/free-tier GPU, e.g. Google Colab's free tier), I found some hardware-specific
behavior and a couple of install-doc gaps that aren't currently documented. This PR adds that
documentation only — no code or behavior changes.
Changes
s3tokenizer==0.3.0and
onnx==1.16.0plus the protobuf upgrade — the same commandstart.py's automatic installeralready runs (
install_chatterbox_no_deps()), but the manual doc block was missing these twopackages, so following it verbatim fails at import time with
ModuleNotFoundError: No module named 's3tokenizer'.predefined_voice_idto the/ttsstreaming curl example (plus a short note on why it'srequired), since the example as written returns
HTTP 400 {"detail":"Missing 'predefined_voice_id' for 'predefined' voice mode."}—voice_modedefaults to"predefined"and the server requiresthe id in that mode.
AGENTS.mddocumenting thatTTS_BF16should stay off (the default) on T4/Turing(sm_75) GPUs:
torch.cuda.is_bf16_supported()reportsTrueon T4, soTTS_BF16=autosilentlyenables bf16, but Turing has no bf16 tensor cores and it measures ~1.6x slower than the fp32
default. Also notes that the default Turbo engine ignores
exaggeration/cfg_weighteven thoughthe API/config expose them as tunable.
Testing
Chatterbox-Turbo engine, commit
915ae28.pip install -r requirements-nvidia.txt && pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master→ baseChatterboxTTSimport fails with
ModuleNotFoundError: No module named 's3tokenizer'. Addings3tokenizer==0.3.0 onnx==1.16.0+pip install --no-deps --force-reinstall "protobuf>=4.25.0"resolves it (
ChatterboxTTS OK/Turbo OK).HTTP 400; adding"predefined_voice_id":"Emily.wav"→HTTP 200(159KB wav, custom text/seed applied correctly).engine.synthesize()directly (seed=1234, 5 warm runs each,<0.5% run-to-run variance): warm RTF 0.335 (fp32/default) vs 0.535 (
TTS_BF16=on/auto) —bf16 is ~1.6x slower wall-clock per second of audio. Confirmed the bf16 path was actually active
at runtime (
t3dtype =torch.bfloat16,torch.is_autocast_enabled()=True) rather than ano-op. Peak VRAM: 4.91GB (fp32) vs 3.92GB (bf16) — well within the 16GB budget either way, so the
VRAM savings aren't needed on this card.