Enforce rate limits across replicas with a shared Redis backend - #801
Merged
fejilaup-cloud merged 2 commits intoAug 25, 2026
Merged
Conversation
RateLimitMiddleware previously kept token-bucket counters in an Arc<Mutex<Store>> owned per instance. Behind a load balancer running N replicas, a client distributed round-robin effectively got N times the documented per-IP/per-user quota, since each replica enforced its own bucket independently. RateLimitStore is now a trait with two implementations: - InProcessStore: the previous in-memory behavior, kept as the default for tests and single-instance deployments (not safe for multiple replicas). - RedisStore: refills and consumes tokens for all buckets (global, IP, user) in one atomic Lua script per request, so a client can't pass the ip bucket on one replica and the user bucket on another. Time comes from Redis's own TIME command rather than the caller's clock, avoiding clock-skew between replicas. Fails open (with a warning log) if Redis is unreachable, rather than taking the API down. Violation backoff, tier assignment, and cardinality bounding stay process-local under both backends, since they're memory-protection heuristics for that replica rather than part of the enforced quota. set_user_tier and the rest of the public API are unchanged. docs/api-reference.md now documents the deployment-topology requirement explicitly. Closes AtomicIP#787
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RateLimitMiddlewarekept token-bucket counters in anArc<Mutex<Store>>owned per instance. Behind a load balancer running N replicas, a client whose requests were distributed round-robin effectively received N times the documented per-IP/per-user quota, since each replica enforced its own bucket independently with no shared counter.RateLimitStoretrait behindRateLimitMiddleware, with two implementations selected viaRateLimitConfig::backend:RateLimitBackend::InProcess(default) — the previous in-memory behavior. Kept for tests and single-instance deployments; documented as not safe for multiple replicas.RateLimitBackend::Redis(url)— refills and consumes tokens for every bucket in a request (global, IP, user) in a single atomic Lua script, so a request can't pass the IP bucket on one replica and the user bucket on another. Time comes from Redis's ownTIMEcommand rather than the caller's clock, so refill accounting can't be skewed by clock drift between replicas. Fails open (allows the request, logs a warning) if Redis is unreachable, so a Redis outage degrades to unlimited-but-available rather than taking the API down with false429s.set_user_tier), and tracked-identity cardinality bounding stay process-local under both backends — they're memory-protection heuristics for that replica, not part of the documented quota, so they don't need to be distributed for the quota fix to be correct.set_user_tierand the rest of the public API are unchanged.docs/api-reference.mdnow has a "Deployment topology" section stating explicitly that multi-instance correctness requires the Redis backend.New tests:
redis_backend_enforces_one_combined_quota_across_instances— spins up a real Redis container (viatestcontainers) and proves two independentRateLimitMiddlewareinstances pointed at it enforce one combined quota for the same client, not two independent ones.in_process_backend_does_not_share_quota_across_instances— documents, as a test, that the in-process backend does not share quota (matching the new docs caveat).asyncsince the store trait'srefill_and_consumeis async.Verification
Ran from
api-server/:rustfmt --check src/rate_limit.rs— clean.cargo clippy --lib— no warnings inrate_limit.rs(added#[derive(Default)]/#[default]forRateLimitBackendto satisfyderivable_impls).Pre-existing, unrelated breakage found while verifying (not touched by this PR):
api-servercurrently does not build as a whole onmain— independent of this change:src/audit.rsdoesn't compile (tokio::sync::RwLock::read()/write()treated as fallible; it isn't).src/distributed_tracing.rsdoesn't compile (opentelemetry::trace::BoxedSpan— the type actually lives atopentelemetry::global::BoxedSpan).api-serverbinary target fails with "multiple different versions of crateaxumin the dependency graph."deduplication::tests::test_concurrent_deduplicator_duplicate_waitshangs indefinitely.circuit_breaker::tests::*(5 tests) andvalidation::tests::test_validate_stellar_address_valid/test_validation_rule_addressfail with assertion panics.This is consistent with
api-serverbeing commented out of the root Cargo workspace ("temporarily excluded: async-graphql version conflict") and therefore never actually built by.github/workflows/ci.yml'scargo build/test --workspace. None of these are touched by this diff; I verified this PR's change by temporarily patching those two unrelated compile errors locally (never committed), confirmed the fullcargo test --librun passes aside from the pre-existing failures/hang listed above (which are unrelated to rate limiting by file and by inspection), then reverted those files before committing so this diff stays scoped to #787. Recommend a separate issue for the pre-existingapi-serverbuild breakage.Closes #787