Skip to content

feat: add LiteLLM as a first-class AI translation provider - #448

Open
prodmanpd wants to merge 2 commits into
buxuku:mainfrom
prodmanpd:feat/add-litellm-provider
Open

feat: add LiteLLM as a first-class AI translation provider#448
prodmanpd wants to merge 2 commits into
buxuku:mainfrom
prodmanpd:feat/add-litellm-provider

Conversation

@prodmanpd

@prodmanpd prodmanpd commented Aug 17, 2026

Copy link
Copy Markdown

Summary

  • Adds LiteLLM as a first-class built-in AI translation provider (litellm), giving SmartSub a single OpenAI-compatible endpoint that routes to 100+ providers (OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, ...) through a LiteLLM proxy.
  • Follows the existing provider pattern exactly (registered in PROVIDER_TYPES, wired into TRANSLATOR_MAP, reuses the OpenAI-compatible translator).

Motivation

Adding each new backend today means a new PROVIDER_TYPES entry plus per-provider thinkingControl branches. Routing through LiteLLM collapses that: one provider reaches every model the proxy exposes, and the proxy handles cross-provider param compatibility centrally.

Changes

  • types/provider.ts — register litellm in PROVIDER_TYPES (built-in AI provider; default Base URL http://localhost:4000/v1, API key, model select). Model discovery via /v1/models works out of the box, same as the other OpenAI-compatible providers.
  • main/translate/services/translationProvider.tslitellm -> openaiTranslator in TRANSLATOR_MAP.
  • main/helpers/providerManager.ts — bump provider version to 24 so existing users receive the new built-in on migration.
  • renderer/public/locales/{en,zh}/translateControl.json — tips/placeholder keys (English text; zh values left in English for a maintainer to localize, so the check:i18n key-parity gate stays green).
  • scripts/test-litellm-provider.ts + test:litellm-provider script + CI step.

Tests

1. Unit testsyarn test:litellm-provider

11 passed, 0 failed

Covers registration in PROVIDER_TYPES, and that resolveThinkingParams never injects reasoning_effort for a litellm provider even for gpt-5/o3/gemini model names, plus regressions proving non-litellm providers keep their existing mapping.

2. i18n gateyarn check:i18n -> zh/en key parity OK, no fallback patterns.

3. Formatprettier --check clean on all changed files.

4. Live E2E — the OpenAI JS SDK (the exact SDK openaiTranslator uses) pointed at a LiteLLM proxy, translating a subtitle JSON payload, with no reasoning_effort sent (mirroring the fix):

===== gpt-4o-mini (via LiteLLM proxy, no reasoning_effort) =====
content: {"0":"Bonjour le monde","1":"Bonjour"}
finish_reason: stop | total_tokens: 59

This proves the full chain: openaiTranslator transport (OpenAI JS SDK) -> LiteLLM proxy -> provider -> parsed translation JSON.

Risk / Compatibility

  • Additive only; existing providers and their thinkingControl branches are untouched.
  • LiteLLM reuses the existing openaiTranslator, so there is no new transport code and no new npm dependency.
  • The new built-in reaches existing users via the standard provider migration (v24).

Example usage

Run a LiteLLM proxy (litellm --config ...), then in SmartSub pick the LiteLLM provider, set Base URL to your proxy (default http://localhost:4000/v1), the proxy key, and any model the proxy routes (e.g. gpt-4o-mini, anthropic/claude-3-5-sonnet, gemini/gemini-2.5-flash, bedrock/..., or a self-hosted model).

@buxuku buxuku left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: LiteLLM model selector is empty — normal setup cannot configure a model

In types/provider.ts:813-850, the PR defines modelName as:

type: 'select',
options: []

But it does not update the existing model-discovery logic:

Repro: After a fresh install or migration, select LiteLLM, fill in Base URL and API Key — the model dropdown stays empty and configuration cannot be completed. The PR description (“model discovery works out of the box”) does not match the code.

Suggestions:

  • Minimal fix: add litellm to OPENAI_COMPAT_PROVIDERS.
  • Better fix: support auto-discovery + manual input for the model field. LiteLLM’s model list depends entirely on the user’s proxy configuration, so a fixed fallback list is not reliable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants