fix(litellm): edge rate limit + HSTS on the public LiteLLM route - #13
Open
Mushtaq-BGA wants to merge 1 commit into
Open
Mushtaq-BGA wants to merge 1 commit into
Mushtaq-BGA wants to merge 1 commit into
Conversation
The litellm.<domain> route exposes the LiteLLM UI and its admin API to the internet (the UI must stay reachable for user login). Harden that route with two edge controls: - HSTS response header so the UI login password / virtual key is never sent over plaintext :80 before the HTTPS redirect. - Local per-client-IP rate limit (Distinct match on x-forwarded-for, no Redis or external ratelimit service) to blunt brute-force against the login/admin API and cap runaway abuse. Both are toggleable/tunable via role defaults, self-heal on disable, and are cleaned up on teardown. Also corrects stale /key/verify extAuth references in the role docs (superseded: the gateway does TLS+routing only, LiteLLM is the sole auth boundary).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
In litellm mode the
litellm.<domain>route publishes the LiteLLM UI and its admin API to the internet — and the UI must stay reachable so users can log in. Path-restricting the route can't reduce that surface (the UI is a client of the admin API), so this PR hardens the route with two edge controls instead:Strict-Transport-Securityresponse header (via a Gateway APIResponseHeaderModifierfilter on the route) so a browser/SDK never sends the UI login password or a virtual key over plaintext:80before the 301 to HTTPS.BackendTrafficPolicy(type: Local, no Redis / external ratelimit service) with aDistinctmatch onx-forwarded-for, giving each source IP its own counter. Blunts automated brute-force against the login/admin API and caps runaway abuse.Notes
roles/litellm/defaults/main.yaml(litellm_hsts_*,litellm_rate_limit_*).state: absenttask when disabled, and is removed on teardown./key/verifyextAuth references in the role docs — superseded by the litellm-mode redesign (gateway does TLS+routing only; LiteLLM is the sole auth boundary).Validation
HTTPRoute(with filter) andBackendTrafficPolicyboth passkubectl apply --dry-run=serveragainst the live Envoy Gateway v1.7.1 CRDs.Companion to the litellm-mode auth-scoping PRs (this repo #12; landing #17).