Rate limiting is enforced per-instance, multiplying the documented quota by replica count

[fejilaup-cloud]

Complexity rank: 8 of 15
Priority: High
Selection criteria satisfied: 2 (correctness under real deployment)
Scope tier: Standard

Description:
docs/api-reference.md:1101-1127 documents a production rate-limit policy (global/IP/user tiers, backoff) with no caveat about deployment topology. api-server/src/rate_limit.rs:137-153 holds Store { ips, users, violations, user_tiers } in Arc<Mutex<Store>> owned per RateLimitMiddleware instance — the doc comment at :147-148 even frames per-instance independence as intentional ("Instances are application-owned, making tests and multiple server instances independent"). Behind a load balancer running N replicas, a client whose requests are distributed round-robin effectively gets N× the documented per-IP or per-user quota, since each replica enforces its own bucket with no shared counter. The existing test concurrent_requests_cannot_overspend_bucket (rate_limit.rs:512) only proves thread-safety within a single instance — it cannot and does not prove cross-instance enforcement.

Tasks:

• Replace the per-process Store (or add an alternate backend) with a shared counter store (e.g. Redis-backed token buckets using atomic INCR/Lua scripts) so limits are enforced across all replicas consistently.
• Preserve the existing token-bucket algorithm's semantics (burst allowance, refill rate, tiering) — only the counter storage location changes.
• Keep the current in-process store available as a fallback/test-only mode, clearly labeled as not safe for multi-instance deployment.
• Add a test proving two independent RateLimitMiddleware instances pointed at the same shared backend correctly enforce one combined quota for the same client, not two independent ones.
• Add a test proving the existing single-instance behavior (thread-safety, tiering, backoff) is unchanged.
• Preserve set_user_tier and all other public API surface.
• Confirm docs/api-reference.md:1101-1127 explicitly states the deployment assumption (shared backend required for multi-instance correctness) rather than leaving it implicit.

[zaraahmedev-glitch]

I've done this exact migration before — in-process limiter → shared backend without touching the caller-facing API — so this maps cleanly to a plan:

Approach:

Extract a RateLimitStore trait (check_and_consume(key, tier) -> Allowed/Denied, with TTL-aware bucket state) and have the current Arc<Mutex> implement it as InProcessStore. RateLimitMiddleware holds Box instead of owning Store directly — set_user_tier and the rest of the public surface stay unchanged, just delegating through the trait.
Add a RedisStore backend using a single Lua script executed via EVAL that does refill + check + decrement atomically per key (tier-scoped, e.g. ratelimit:{tier}:{ip_or_user}). That avoids a check/consume race across replicas and keeps the burst-allowance/refill-rate math identical — only the storage location moves.
Keep InProcessStore as the default for tests and single-instance runs, selected via a RateLimitBackend::InProcess | RateLimitBackend::Redis(url) config, and rewrite the :147-148 doc comment to state plainly it's not safe for multi-instance deployment (currently it frames the opposite).
For the cross-instance test: spin up two RateLimitMiddleware instances pointed at the same Redis (or a testcontainers-managed instance) in one test process, fire concurrent requests from the same client id across both, and assert the combined accepted count equals the single bucket's capacity, not 2×. Keep concurrent_requests_cannot_overspend_bucket untouched to prove single-instance behavior still holds independently.
Update docs/api-reference.md:1101-1127 with an explicit line that multi-instance correctness requires the shared (Redis) backend.

I've hit the usual gotchas with this pattern before — EVALSHA/script-reload fallback, clock-skew in refill timestamp calculations, connection pool sizing under burst load — so I'd bake tests for those into step 4 as well rather than treating them as follow-up work.

[grantfox-oss]

🦊 GrantFox — @zaraahmedev-glitch has been assigned to this issue as part of the Third Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #787)
2. Your PR will be reviewed by the AtomicIP maintainers

Good luck! Track your progress on GrantFox.

[grantfox-oss]

🎉 This issue has been marked as completed on GrantFox as part of the Third Campaign campaign!

@zaraahmedev-glitch's PR #801 was approved and merged by @fejilaup-cloud.

🏆 @zaraahmedev-glitch: You earned 35 FoxPoints for this contribution! Your current tier: Explorer (121 total points). Track your full progress on GrantFox.

👏 Great work, @zaraahmedev-glitch! Keep contributing to AtomicIP.