← sources

One customer sends 50,000 req/min against a 100/min quota across 20 instances. How do you build the rate limiter?

source: reddit (reddit:developersIndia)published 2026-10-02 04:41ingested 2026-10-03 02:37

retained text

I'm a backend dev. We practice system design a lot on whiteboards, but rarely on something that looks like a real incident. Here's a production-style puzzle.

Your public API serves 50,000 paying customers, each allowed 100 requests/minute. There is currently no rate limiting at all. A bug in one customer's integration starts sending 50,000 requests/minute to `/search` (500x its quota), and latency degrades for everyone else.

**Constraints**

* 20 stateless API instances behind a round-robin load balancer, no sticky sessions
* A Redis cluster already exists (used for caching elsewhere)
* The limit must hold globally per customer. An instance-local counter lets one customer reach 100 × 20 = 2,000 req/min
* Under 5ms added to the request path at p99
* Rolling reset: no bursting 100 requests either side of a minute boundary
* Rate limiting must never be what takes the API down

**What you observe**

* The flood spreads evenly across all 20 instances, about 2,500/min each, and every instance sees a slice that looks locally plausible
* p99 latency for other customers rises in the same window as the flood
* A read-then-increment limiter passes every single-instance test, but quota is still exceeded once two instances overlap
* The limiter's Redis call is \~0.3-1ms p99 when healthy, but p99 in the seconds when Redis is degraded
* At full quota across all customers, the limiter makes \~83,000 calls/sec

**The question:** Design the limiter end to end. Where does the counter live? What makes check-and-increment indivisible? What happens when Redis is down? What changes when limits are per-endpoint?

I'd love to hear how you'd explain your design in a review, and especially what you'd do when Redis is slow or down, and why.

document fetched 2026-10-03 00:25 · canonical https://reddit.com/r/developersIndia/comments/1wvkscx/one_customer_sends_50000_reqmin_against_a_100min