Rate Limiters & Traffic Shaping

Every public and internal API is vulnerable to resource exhaustion. Whether from malicious DDoS attacks, credential-stuffing scrapers, runaway customer scripts, or internal services caught in retry storms, unconstrained traffic inevitably saturates database connection pools and takes down infrastructure. Distributed rate limiting enforces admission control at the ingress gateway, protecting downstream systems and guaranteeing tenant fairness.

A common production vulnerability is implementing rate limiting using separate Redis GET and SET/INCR operations across distributed API gateway nodes.

1
2
3
4
5
6
7
8
9

NAIVE CHECK-THEN-ACT RACE CONDITION (Client limit: 10 req/min, current count: 9):
Gateway 1: GET count (=9) --------------------------> INCR count (=10) -> ALLOWED
Gateway 2: ------> GET count (=9) -> INCR count (=11) -------------------> ALLOWED (BURST VIOLATION!)

ATOMIC REDIS LUA EXECUTION (Single-Threaded Engine):
Gateway 1 & 2 -> Redis EVAL (Lua Script)
                 |-- Script runs to completion without interruption
                 |-- Request 1: Tokens decremented from 1 to 0 -> ALLOWED (HTTP 200)
                 +-- Request 2: Tokens = 0 -> REJECTED (HTTP 429)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

-- Production-Grade Atomic Token Bucket in Redis Lua
-- KEYS[1]: ratelimit:{user_id}:tokens, KEYS[2]: ratelimit:{user_id}:last_updated
-- ARGV[1]: capacity (B), ARGV[2]: refill_rate_per_sec (R), ARGV[3]: now_seconds, ARGV[4]: requested (1)

local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])

local last_tokens = tonumber(redis.call('get', KEYS[1]))
local last_updated = tonumber(redis.call('get', KEYS[2]))

if last_tokens == nil then
    last_tokens = capacity
    last_updated = now
else
    local elapsed = math.max(0, now - last_updated)
    last_tokens = math.min(capacity, last_tokens + elapsed * refill_rate)
    last_updated = now
end

if last_tokens >= requested then
    last_tokens = last_tokens - requested
    redis.call('set', KEYS[1], last_tokens)
    redis.call('set', KEYS[2], last_updated)
    local ttl = math.ceil((capacity - last_tokens) / refill_rate) + 60
    redis.call('expire', KEYS[1], ttl)
    redis.call('expire', KEYS[2], ttl)
    return {1, math.floor(last_tokens)} -- 1 = Allowed, remaining tokens
else
    return {0, math.floor(last_tokens)} -- 0 = Denied (HTTP 429)
end

1. 01Ingress Interception: Client sends POST /v1/charges. Gateway extracts IP address and authenticated API key.
2. 02Atomic Lua Evaluation: Gateway executes Redis EVAL with token bucket parameters. Redis computes remaining capacity in ~0.5ms.
3. 03Successful Acceptance: If tokens remain, gateway attaches standard IETF headers: X-RateLimit-Limit: 100, X-RateLimit-Remaining: 42, X-RateLimit-Reset: 1698421000. Request forwards to downstream services.
4. 04Rejection (HTTP 429): If tokens < 1, gateway rejects request immediately with HTTP 429 Too Many Requests and Retry-After: 4. Zero load reaches internal microservices.

1. 01The Fail-Open vs. Fail-Closed Dilemma: What happens when the centralized Redis rate-limiter cluster crashes or becomes unreachable?
2. 02Fail-Open: Allow all requests through without rate limiting. Preserves availability for legitimate paying users, but risks overwhelming downstream databases during attacks.
3. 03Fail-Closed: Reject all requests with HTTP 500/429. Protects downstream databases, but converts a cache blip into a total outage for 100% of legitimate users.
4. 04Production Best Practice: Fail-Open with Emergency Local Fallback: If Redis times out (> 10ms), fail open into a conservative local in-memory token bucket on each gateway node to cap catastrophic bursts.