Client-IP rate-limit scope and burst allowance

[janiussyafiq]

Description

Rate limits key only on authenticated identities: api_key, model, team, member, team_member. There is no client-IP scope, no arbitrary header/XFF key, and no trusted-proxy / XFF-depth configuration for rate limiting. Counters are fixed-window with no burst allowance on top of a sustained rate.

Requested: an IP scope with trusted-proxy configuration, and an optional burst allowance layered over the sustained rate.

Why

Rate limiting is an abuse and denial-of-service control as much as a quota control, and identity-only keying cannot throttle an unauthenticated or pre-auth flood at the gateway.

Bursts matter because real client traffic is spiky: with a hard fixed window, operators must over-provision the limit to avoid false rejects, which defeats the point of setting one.

An edge/LB tier in front covers IP throttling in many deployments, which is why this is scored Low.

Priority

Low.

---

Prior art

Product	Has it	Reference
LiteLLM	No	no IP scope
Kong AI Gateway	Yes	config.identifier: ip + trusted_ips at gateway level
Cloudflare AI Gateway	No	uniform, not IP-keyed
Portkey	No	no IP scope
Envoy AI Gateway	Partial	IP-keyed at the Envoy layer, not the AI layer
Bifrost	No	no IP scope
Helicone / TrueFoundry / Databricks AI Gateway / OpenRouter	Unknown	not researched

Kong is the clearest prior art — note it keeps XFF trust at gateway level rather than per-limiter.

_{Surveyed 2026-07-15; every claim rests on a fetched docs/source page.}