Learn › System Design Playbook › Design a Rate Limiter

Design a Rate Limiter

2 min read

A rate limiter caps how many requests a client can make in a window — protecting your system from abuse, runaway clients and cost overruns.

Limit requests per client (by API key / user / IP), e.g. 100 requests/minute.
Low latency (it's on every request), accurate, and works across a distributed fleet.
Return 429 Too Many Requests with a Retry-After header when exceeded.

Algorithm	Idea	Trade-off
Token bucket	Bucket refills at a fixed rate; each request takes a token	Allows bursts; simple; very common
Leaky bucket	Requests queue and drain at a fixed rate	Smooths bursts; can add latency
Fixed window	Count per calendar minute	Simple, but allows 2× burst at the boundary
Sliding window log	Timestamps of each request	Accurate, more memory
Sliding window counter	Weighted blend of two windows	Good accuracy/memory balance

Token bucket is the go-to default: capacity tokens, refilled r per second. A request is allowed if a token is available.

With many app servers, the counter must be shared — otherwise each server allows the full limit.

Request ─▶ App ─▶ Redis (INCR key, check limit) ─▶ allow / 429

Store counters in Redis (fast, atomic INCR/Lua scripts).
Use a Lua script to make "check + increment + set TTL" atomic, avoiding race conditions.

key   = "rl:{user}:{window}"
count = INCR key
if count == 1: EXPIRE key 60
if count > limit: return 429

Where it lives: at the API gateway / edge so bad traffic is rejected before hitting services.
Trade-off: a central Redis adds a network hop and is itself a dependency — cache token buckets locally and sync, or shard Redis for scale.
Fairness: separate limits per tier (free vs paid); return current limit/remaining in response headers.

Takeaway: token bucket + atomic Redis counter is the standard, interview-ready rate limiter.