Planetary Distributed Rate Limiter (Cloudflare & Stripe)

A distributed API rate limiter protects downstream microservice fleets from traffic spikes, denial-of-service (DoS) attacks, brute-force credential stuffing, abusive web scrapers, and cascading system outages caused by misconfigured client retries. Operating across distributed edge gateways and application tiers, the limiter must evaluate incoming requests against complex per-tenant quotas with sub-2 millisecond latency overhead.

The rate limiter must operate with extreme precision, protecting diverse internal microservice APIs while maintaining transparent communication contracts with clients.

Quantitative analysis evaluates memory footprints and compute saturation across 50,000 requests/sec sustained load.

A production rate limiting topology employs a multi-tiered defense in depth to absorb traffic without single points of failure.

1. 01Tier 1: Edge Scrubbing (Cloudflare / AWS CloudFront): BGP Anycast filters absorb L3/L4 volumetric DDoS attacks. Coarse IP rate limits (e.g. max 2,000 req/min per IP) drop aggressive scrapers before reaching the datacenter.
2. 02Tier 2: API Gateway Fleet (Envoy / Kong): Intercepts authenticated traffic, extracts tenant ID and API key, and evaluates in-memory L1 Caffeine cache (Token Leasing).
3. 03Tier 3: Distributed State Store (Redis Cluster): Sharded Redis cluster executes atomic Lua scripts to evaluate sliding window counters and token refills across all gateway nodes.
4. 04Tier 4: Control Plane & Config Engine: Watches quota updates in the Postgres billing database, pushing updated rate-limit configurations to gateways and Redis via Redis Pub/Sub.
5. 05Tier 5: Audit & Telemetry Pipeline: When a request is rejected with HTTP 429, the gateway produces an asynchronous metric event to Kafka/Prometheus to feed security SIEM dashboards.

State storage requires precise key design to guarantee that multi-key operations map to identical hash slots in a distributed Redis cluster.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

-- Relational Policy Configuration Schema (PostgreSQL Control Plane)
CREATE TABLE rate_limit_policies (
    policy_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tier_name VARCHAR(50) NOT NULL UNIQUE,       -- 'free', 'developer', 'enterprise'
    route_pattern VARCHAR(255) NOT NULL,         -- '/api/v1/checkout' or '*'
    window_seconds INT NOT NULL DEFAULT 60,      -- Rolling window size
    max_requests INT NOT NULL DEFAULT 1000,      -- Allowed threshold per window
    burst_capacity INT NOT NULL DEFAULT 1500,    -- Peak burst ceiling
    fail_mode VARCHAR(10) NOT NULL DEFAULT 'OPEN', -- 'OPEN' or 'CLOSED'
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE client_api_keys (
    api_key VARCHAR(64) PRIMARY KEY,
    tenant_id UUID NOT NULL,
    policy_id UUID NOT NULL REFERENCES rate_limit_policies(policy_id),
    is_active BOOLEAN NOT NULL DEFAULT TRUE
);

Below is the battle-tested Redis Lua implementation of the Sliding Window Counter algorithm with O(1) time and memory complexity.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

-- Production Redis Lua Script: Sliding Window Counter
-- KEYS[1]: Current window key, e.g., {rl:user_123}:28400101
-- KEYS[2]: Previous window key, e.g., {rl:user_123}:28400100
-- ARGV[1]: Max requests allowed in window
-- ARGV[2]: Window size in seconds (e.g., 60)
-- ARGV[3]: Current epoch timestamp with millisecond precision
-- ARGV[4]: Cost of current request (typically 1)

local curr_key = KEYS[1]
local prev_key = KEYS[2]
local max_limit = tonumber(ARGV[1])
local window_size = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

-- Calculate offset within the current window (0.0 to 1.0)
local current_window_bucket = math.floor(now / window_size)
local offset = (now % window_size) / window_size
local weight = 1.0 - offset

-- Retrieve current and previous counts
local curr_count = tonumber(redis.call('get', curr_key) or '0')
local prev_count = tonumber(redis.call('get', prev_key) or '0')

-- Estimate total requests in rolling window
local estimated_count = math.floor(prev_count * weight + curr_count)

if estimated_count + cost > max_limit then
    -- Exceeded limit: return rejection + headers metadata
    local reset_time = (current_window_bucket + 1) * window_size
    local retry_after = math.max(1, math.ceil(reset_time - now))
    return {0, max_limit - estimated_count, retry_after, reset_time}
else
    -- Allowed: increment current window counter atomically
    local new_count = redis.call('incrby', curr_key, cost)
    if new_count == cost then
        -- Set TTL to 2x window to allow previous window calculation
        redis.call('expire', curr_key, window_size * 2)
    end
    local remaining = math.max(0, max_limit - (estimated_count + cost))
    local reset_time = (current_window_bucket + 1) * window_size
    return {1, remaining, 0, reset_time}
end

1. 01Step 1: Request Ingress: Client sends GET /api/v1/orders with Authorization: Bearer key_sec_99.
2. 02Step 2: Key Extraction: Gateway extracts API key, hashes it, and queries local in-memory policy cache to determine user tier (100 req/min).
3. 03Step 3: Atomic Evaluation: Gateway executes the Sliding Window Lua script against the regional Redis Cluster. Network RTT is ~0.8ms.
4. 04Step 4a (Traffic Admitted): If script returns 1, gateway sets response headers: X-RateLimit-Limit: 100, X-RateLimit-Remaining: 41, X-RateLimit-Reset: 1718000060, and forwards request upstream.
5. 05Step 4b (Rate Limit Exceeded): If script returns 0, gateway terminates the request immediately with HTTP 429 Too Many Requests. Response payload includes {"error": "rate_limit_exceeded", "retry_after": 14} and header Retry-After: 14. Zero compute touches backend microservices.