Skip to content

fix(litellm): edge rate limit + HSTS on the public LiteLLM route - #13

Open
Mushtaq-BGA wants to merge 1 commit into
mainfrom
feat/litellm-edge-hardening
Open

Mushtaq-BGA wants to merge 1 commit into
mainfrom
feat/litellm-edge-hardening

Conversation

@Mushtaq-BGA

Copy link
Copy Markdown
Contributor

What

In litellm mode the litellm.<domain> route publishes the LiteLLM UI and its admin API to the internet — and the UI must stay reachable so users can log in. Path-restricting the route can't reduce that surface (the UI is a client of the admin API), so this PR hardens the route with two edge controls instead:

  • HSTS — Strict-Transport-Security response header (via a Gateway API ResponseHeaderModifier filter on the route) so a browser/SDK never sends the UI login password or a virtual key over plaintext :80 before the 301 to HTTPS.
  • Local per-client-IP rate limit — an Envoy Gateway BackendTrafficPolicy (type: Local, no Redis / external ratelimit service) with a Distinct match on x-forwarded-for, giving each source IP its own counter. Blunts automated brute-force against the login/admin API and caps runaway abuse.

Notes

  • Both are toggleable and tunable via roles/litellm/defaults/main.yaml (litellm_hsts_*, litellm_rate_limit_*).
  • Idempotent / self-healing: HSTS is built as a fact so disabling it re-renders the route with no filter; the rate-limit policy has a paired state: absent task when disabled, and is removed on teardown.
  • Coarse by design. Local rate limit is per-gateway-replica, and clients behind a shared NAT/egress IP share one counter. Per-identity limits + proper multi-user login (OIDC/SSO) come with the in-progress auth-capability work.
  • Also corrects stale /key/verify extAuth references in the role docs — superseded by the litellm-mode redesign (gateway does TLS+routing only; LiteLLM is the sole auth boundary).

Validation

  • YAML parses; HSTS filter fact renders correctly for enabled/disabled.
  • Rendered HTTPRoute (with filter) and BackendTrafficPolicy both pass kubectl apply --dry-run=server against the live Envoy Gateway v1.7.1 CRDs.

Companion to the litellm-mode auth-scoping PRs (this repo #12; landing #17).

The litellm.<domain> route exposes the LiteLLM UI and its admin API to the
internet (the UI must stay reachable for user login). Harden that route with
two edge controls:

- HSTS response header so the UI login password / virtual key is never sent
  over plaintext :80 before the HTTPS redirect.
- Local per-client-IP rate limit (Distinct match on x-forwarded-for, no Redis
  or external ratelimit service) to blunt brute-force against the login/admin
  API and cap runaway abuse.

Both are toggleable/tunable via role defaults, self-heal on disable, and are
cleaned up on teardown. Also corrects stale /key/verify extAuth references in
the role docs (superseded: the gateway does TLS+routing only, LiteLLM is the
sole auth boundary).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant