Intermediate · Accelerate

Prevent Key Leaks, Stampedes & Hotspots

Handle tenant isolation, concurrent misses, hot keys, negative caching, and bounded memory.

Primary source: Spring Boot 3.5 Reference: Caching

Written and reviewed by Yash Patel 40 minBoot 3.5.16 · Java 21· Reviewed 2026-09-07

Lesson 3 of 4

1d streak

The Architectural Question

Hot keys, synchronized expirations, and tenant-blind namespaces turn one popular product into a database stampede or a cross-tenant data leak.

What this guide proves

Namespace by environment and tenant
Use jittered TTLs or single-flight loading where needed
Never cache authorization decisions across the wrong identity boundary

The Anatomy of a Cache Stampede (Dogpile Effect)

A cache stampede (also known as the dogpile effect or thundering herd) occurs when a heavily requested hot cache key expires or is evicted. At that exact millisecond, hundreds or thousands of concurrent incoming HTTP requests experience a cache miss simultaneously.

Each thread attempts to read from the authoritative backend (database, external API, or complex compute engine) and write back to the cache. The sudden surge in concurrent database queries exhausts the HikariCP connection pool, saturates CPU, spikes p99 latency into seconds, and frequently causes cascade failure across the entire application cluster.

Hot Key Expiry in High-Traffic Systems

If a homepage configuration key receiving 10,000 requests/sec expires without stampede protection, all 10,000 threads will immediately issue the same SQL query to PostgreSQL, crashing database workers.

Mitigation via @Cacheable(sync = true) and Distributed Locks

Spring Cache provides built-in local stampede protection via the sync = true attribute on @Cacheable. When sync = true is enabled, the underlying cache provider synchronizes concurrent threads on the local JVM so that only one thread executes the underlying method while all other threads block until the cache is populated.

src/main/java/com/example/catalog/HotItemCatalogService.javajava

package com.example.catalog;

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Service
public class HotItemCatalogService {

    private final ProductRepository repository;

    public HotItemCatalogService(ProductRepository repository) {
        this.repository = repository;
    }

    // sync = true serializes concurrent thread lookups for the same key on this JVM
    @Cacheable(cacheNames = "hot-deals", key = "#dealId", sync = true)
    public Deal findHotDeal(String dealId) {
        // Only ONE thread executes this database fetch on miss; others block and read the cached result
        return repository.queryExpensiveDeal(dealId);
    }
}

Limitation of sync = true in Clustered Deployments

The sync = true attribute synchronizes threads only within a single JVM process. Across a Kubernetes cluster with 20 pods, up to 20 database queries can still fire concurrently (1 per pod). For global cluster-wide single-flighting, use a distributed Redis lock.

src/main/java/com/example/catalog/DistributedSingleFlightService.javajava

package com.example.catalog;

import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.stereotype.Service;
import java.time.Duration;
import java.util.List;
import java.util.UUID;

@Service
public class DistributedSingleFlightService {

    private final StringRedisTemplate redisTemplate;
    private final ProductRepository repository;
    private final ProductCache cache; // typed cache abstraction, omitted for brevity

    private static final DefaultRedisScript<Long> UNLOCK = new DefaultRedisScript<>(
        "if redis.call('get', KEYS[1]) == ARGV[1] " +
        "then return redis.call('del', KEYS[1]) else return 0 end", Long.class);

    public DistributedSingleFlightService(
            StringRedisTemplate redisTemplate,
            ProductRepository repository,
            ProductCache cache) {
        this.redisTemplate = redisTemplate;
        this.repository = repository;
        this.cache = cache;
    }

    public Product getWithDistributedLock(String sku) {
        Product cached = cache.get(sku);
        if (cached != null) return cached;

        String lockKey = "lock:rebuild:product:" + sku;
        String token = UUID.randomUUID().toString();
        Boolean acquired = redisTemplate.opsForValue()
            .setIfAbsent(lockKey, token, Duration.ofSeconds(5));

        if (Boolean.TRUE.equals(acquired)) {
            try {
                Product fresh = repository.fetchFromDatabase(sku);
                cache.put(sku, fresh, Duration.ofMinutes(10));
                return fresh;
            } finally {
                // Delete only if this process still owns the lock.
                redisTemplate.execute(UNLOCK, List.of(lockKey), token);
            }
        }

        for (int attempt = 0; attempt < 10; attempt++) {
            try {
                Thread.sleep(50L);
            } catch (InterruptedException interrupted) {
                Thread.currentThread().interrupt();
                throw new IllegalStateException("Interrupted while waiting for cache rebuild", interrupted);
            }
            cached = cache.get(sku);
            if (cached != null) return cached;
        }

        // A bounded fallback prevents an unbounded wait; apply normal DB timeouts.
        return repository.fetchFromDatabase(sku);
    }
}

TTL Jittering & Probabilistic Early Expiration

When batch jobs populate millions of cache entries with an identical TTL (e.g. exactly 60 minutes), all millions of keys will expire at the exact same second in the future, creating a synchronized system-wide cache avalanche.

Always introduce randomized jitter into expiration calculations to stagger key expiration smoothly over a time window:

src/main/java/com/example/util/CacheTtlJitter.javajava

package com.example.util;

import java.time.Duration;
import java.util.concurrent.ThreadLocalRandom;

public final class CacheTtlJitter {

    private CacheTtlJitter() {}

    public static Duration jitter(Duration baseTtl, double jitterFactor) {
        long baseSeconds = baseTtl.getSeconds();
        long variance = (long) (baseSeconds * jitterFactor);
        long jitteredSeconds = baseSeconds + ThreadLocalRandom.current().nextLong(-variance, variance + 1);
        return Duration.ofSeconds(Math.max(1, jitteredSeconds));
    }
}

Concurrent Load Testing and Stampede Verification

Verify stampede defense using Apache Bench (ab) or hey to send 500 concurrent requests during a key expiration event:

test-stampede.shbash

# 1. Flush the hot deal key from Redis
redis-cli DEL "app:v1:hot-deals::DEAL-SUMMER-2026"

# 2. Fire 500 concurrent requests to the endpoint
hey -n 500 -c 50 http://localhost:8080/api/deals/DEAL-SUMMER-2026

# 3. Check application logs for database query count:
# EXPECTED: Exactly 1 SQL query executed when sync = true is enabled.
# WITHOUT sync = true: 50+ concurrent SQL queries executed and Hikari pool exhaustion.

Active Recall Checkpoint

How does TTL jitter prevent cache avalanches?

Hint: Consider the effect of 100,000 keys expiring at 03:00:00 vs spread between 02:50:00 and 03:10:00.

Show answer & explanation

By adding a randomized offset to the expiration timestamp, keys generated in bulk expire smoothly across a distribution curve rather than simultaneously.

Working Implementation

Code Walkthrough

Load-test one hot product and compare uncached, naive-cache, and protected-cache behavior.

Java

CacheStampedeDefenseLab.javaJava

package com.example.cache;

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

import java.time.Duration;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.atomic.AtomicInteger;

@Service
public class CacheStampedeDefenseLab {

    private final AtomicInteger loaderInvocations = new AtomicInteger();

    public record FlashDeal(String dealId, String title, int discountPercent) {}

    // sync = true serializes concurrent thread lookups for the same key on this JVM, preventing thundering herd
    @Cacheable(cacheNames = "deals", key = "#dealId", sync = true)
    public FlashDeal getDealWithSync(String dealId) {
        loaderInvocations.incrementAndGet();
        try {
            // Simulate heavy database aggregation (50ms)
            Thread.sleep(50);
        } catch (InterruptedException ignored) {}
        return new FlashDeal(dealId, "Mega Deal", 50);
    }

    // Introduce randomized jitter to stagger expiration and avoid simultaneous mass invalidation
    public static Duration jitteredTtl(Duration baseTtl, double jitterFactor) {
        long baseSec = baseTtl.getSeconds();
        long variance = (long) (baseSec * jitterFactor);
        long randomized = baseSec + ThreadLocalRandom.current().nextLong(-variance, variance + 1);
        return Duration.ofSeconds(Math.max(1, randomized));
    }

    public int getLoaderInvocations() {
        return loaderInvocations.get();
    }
}

@Cacheable(sync = true) forces concurrent threads requesting an expired key to block while one thread executes the loader.

Randomized TTL jitter prevents thousands of batch-cached keys from expiring at the exact same second.

For multi-pod Kubernetes clusters, combine local sync with a Redis distributed lock (SET NX PX) for global single-flighting.

Synthesis & Production Verification

Production Acceptance Criteria

Namespace keys by environment and tenant and prove isolation
Absorb a hot-key miss without a thundering herd on the source
Never reuse cached authorization decisions across identity boundaries

Architectural & Technical Review

Core Interview Questions

Q01What causes a cache stampede (dogpile effect) and what is its impact on database connection pools?Show answer

When a hot key expires under heavy traffic, thousands of concurrent requests miss simultaneously. Each thread queries the database and tries to repopulate the cache, exhausting the HikariCP connection pool and spiking CPU.

Q02How does @Cacheable(sync = true) prevent local thundering herds, and what is its limitation in a Kubernetes cluster?Show answer

sync = true serializes concurrent thread lookups for the same key on a single JVM so only one thread executes the loader while others wait. Across multiple Kubernetes pods, up to one query per pod can still fire concurrently unless backed by a distributed lock.

Q03How does adding randomized jitter to TTL calculations prevent synchronized cache avalanches?Show answer

Adding a randomized offset to the base TTL staggers expiration timestamps across a normal distribution curve, preventing thousands of keys generated in batch from expiring simultaneously.

Production Pitfalls

Critical Anti-Patterns

Pitfall: Assuming @Cacheable without sync=true protects backends during peak cache expiry — All concurrent misses fire duplicate SQL queries simultaneously, saturating connection pools.
Pitfall: Setting identical static TTLs for millions of bulk-loaded database records — Guarantees that all records expire at the exact same future second, creating a massive stampede.

Primary Documentation & Specifications

Lesson Primary References

Spring Boot 3.5 Reference: Caching Spring Framework 6.2 Cache Abstraction

Supplementary Module Resources

Cache abstraction Redis support