Skip to content

docs: add Tesla T4 (Turing) hardware notes - #157

Open
moduvoice wants to merge 2 commits into
devnen:mainfrom
moduvoice:docs/t4-turing-notes
Open

docs: add Tesla T4 (Turing) hardware notes#157
moduvoice wants to merge 2 commits into
devnen:mainfrom
moduvoice:docs/t4-turing-notes

Conversation

@moduvoice

Copy link
Copy Markdown

Motivation

While verifying this project's inference pipeline on a real NVIDIA Tesla T4 (16GB, Turing/sm_75 —
a common budget/free-tier GPU, e.g. Google Colab's free tier), I found some hardware-specific
behavior and a couple of install-doc gaps that aren't currently documented. This PR adds that
documentation only — no code or behavior changes.

Changes

  • Fix Manual Install Option 2 and Option 2b pip commands to include s3tokenizer==0.3.0
    and onnx==1.16.0 plus the protobuf upgrade — the same command start.py's automatic installer
    already runs (install_chatterbox_no_deps()), but the manual doc block was missing these two
    packages, so following it verbatim fails at import time with
    ModuleNotFoundError: No module named 's3tokenizer'.
  • Add predefined_voice_id to the /tts streaming curl example (plus a short note on why it's
    required), since the example as written returns HTTP 400 {"detail":"Missing 'predefined_voice_id' for 'predefined' voice mode."}voice_mode defaults to "predefined" and the server requires
    the id in that mode.
  • Add a new AGENTS.md documenting that TTS_BF16 should stay off (the default) on T4/Turing
    (sm_75) GPUs: torch.cuda.is_bf16_supported() reports True on T4, so TTS_BF16=auto silently
    enables bf16, but Turing has no bf16 tensor cores and it measures ~1.6x slower than the fp32
    default. Also notes that the default Turbo engine ignores exaggeration/cfg_weight even though
    the API/config expose them as tunable.

Testing

  • Measured on a real Tesla T4 16GB (Turing, sm_75), driver 550.163.01, torch 2.5.1+cu121, default
    Chatterbox-Turbo engine, commit 915ae28.
  • Reproduced the manual-install failure verbatim: pip install -r requirements-nvidia.txt && pip install --no-deps git+https://github.com/devnen/chatterbox-v2.git@master → base ChatterboxTTS
    import fails with ModuleNotFoundError: No module named 's3tokenizer'. Adding
    s3tokenizer==0.3.0 onnx==1.16.0 + pip install --no-deps --force-reinstall "protobuf>=4.25.0"
    resolves it (ChatterboxTTS OK / Turbo OK).
  • Reproduced the curl example verbatim → HTTP 400; adding "predefined_voice_id":"Emily.wav"
    HTTP 200 (159KB wav, custom text/seed applied correctly).
  • bf16 vs fp32 on T4, measured via engine.synthesize() directly (seed=1234, 5 warm runs each,
    <0.5% run-to-run variance): warm RTF 0.335 (fp32/default) vs 0.535 (TTS_BF16=on/auto)
    bf16 is ~1.6x slower wall-clock per second of audio. Confirmed the bf16 path was actually active
    at runtime (t3 dtype = torch.bfloat16, torch.is_autocast_enabled()=True) rather than a
    no-op. Peak VRAM: 4.91GB (fp32) vs 3.92GB (bf16) — well within the 16GB budget either way, so the
    VRAM savings aren't needed on this card.

- Fix Manual Install Option 2/2b pip commands to include the
  s3tokenizer==0.3.0 and onnx==1.16.0 pins plus the protobuf upgrade,
  matching what start.py already installs automatically. Without
  these, the base ChatterboxTTS import fails with
  ModuleNotFoundError: No module named 's3tokenizer'.
- Add predefined_voice_id to the /tts curl example (and a note
  explaining why it's required for the default predefined voice
  mode) since the example as written returns HTTP 400.
- Add AGENTS.md documenting that TTS_BF16 should stay off (default)
  on T4/Turing (sm_75) GPUs, since is_bf16_supported() returns True
  there but bf16 is measured ~1.6x slower (RTF 0.535 vs 0.335) with
  no real hardware acceleration, and noting that the default Turbo
  engine ignores exaggeration/cfg_weight.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant