Component Architecture · Phase 2: Core Infrastructure
Content Delivery Networks (CDNs) & Anycast Routing
Learn how BGP Anycast and Geo-DNS route users to edge points of presence (POPs), static vs dynamic asset caching, origin shielding, and stale-while-revalidate headers.
Primary specification: Cloudflare Architecture: How Anycast IP Works
Lesson 12 of 33
1d streakThe Architectural Question
High latency for international users and sudden traffic spikes directly overwhelm origin web servers despite serving mostly static media and cacheable API responses.
Core Engineering Objectives
- Edge routing mechanics: BGP Anycast routing vs Geo-DNS resolution
- Dynamic content acceleration: TLS termination at edge, TCP window optimization, connection reuse
- Origin Shielding and tag-based surrogate key cache invalidation
Interactive system trace
Watch the architecture make its key decision
Switch scenarios, predict the next active component, and step through the request or failure path.
Fast path
Content Delivery Network
A user requests /hero.webp while a fresh copy exists at the Mumbai edge.
Edge object
Fresh
Origin fetches
0
Response path
In progress
DNS returns the address of a nearby point of presence.
Why it matters: Shorter physical distance reduces round-trip time.
A CDN places a cacheable serving layer near the network edge
A content delivery network operates points of presence in many networks. DNS policy or BGP anycast directs a request to an available edge based on routing and provider policy. The selected edge is often latency-near, but it is not guaranteed to be the geographically closest building.
| Mechanism | Decision point | Important limitation |
|---|---|---|
| Geo-aware DNS | DNS returns an edge address using resolver location and health policy | Resolver location and cached answers may differ from the user location |
| Anycast | Internet routing selects a path to one site advertising the same IP prefix | BGP chooses policy-compatible paths, not literal geographic distance |
| Origin shield | A regional cache sits between edge sites and the origin | It reduces duplicate misses but adds a hop on a shield miss |
Cache key
The normalized identity used to decide whether two requests may share a cached response, commonly including host and path plus selected query parameters or headers.
Freshness lifetime
How long a stored response may be served without contacting the origin, derived from directives such as max-age or s-maxage.
Purge
An explicit invalidation request for one object or a tagged group of objects before normal freshness expiry.
Follow a cache hit, miss, and revalidation
- 01The client resolves the hostname and connects to the edge selected by DNS and network routing.
- 02The edge normalizes the request and builds a cache key. Authentication, cookies, or unsafe methods may make the response private or uncacheable.
- 03On a fresh hit, the edge returns the stored response without contacting the origin.
- 04On a miss, one request should fetch through the origin shield while equivalent concurrent requests are collapsed instead of creating a stampede.
- 05The edge stores the response only when its status, headers, and policy permit caching, then serves waiting clients.
- 06When stale-while-revalidate is allowed, the edge may serve a stale object within the stated window while one background request refreshes it.
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=30
Vary: Accept-Encoding
Surrogate-Key: article-482 category-system-designmax-age controls browser freshness, while shared caches may use s-maxage. Vary changes the cache key and can destroy hit ratio when it includes high-cardinality headers. Surrogate keys are provider-specific but allow related objects to be purged together.
HTTP caching is a response contract, not a CDN toggle
A shared cache may store and reuse a response only when the HTTP method, status, request directives, response directives, authorization rules, and cache key allow it. Freshness says how long reuse is allowed without validation. After freshness expires, a validator such as ETag can turn a full transfer into a conditional request and a 304 Not Modified response.
| Directive or header | Meaning at a shared edge | Common mistake |
|---|---|---|
| public | Response may be stored by a shared cache | Assuming it makes private user data safe to share |
| private | Response is intended for a private cache, such as a browser | Treating it as “encrypt this response” |
| s-maxage=600 | Shared-cache freshness is 600 seconds | Forgetting that it overrides max-age for shared caches |
| no-store | Do not store the response | Using it when revalidation, rather than no storage, is intended |
| must-revalidate | Once stale, reuse requires successful validation | Combining it with an assumed offline stale fallback |
| ETag | Opaque validator for a selected representation | Generating unstable validators that change without content changes |
GET /catalog/42 HTTP/1.1
If-None-Match: "catalog-42-v7"
HTTP/1.1 304 Not Modified
Cache-Control: public, s-maxage=600
ETag: "catalog-42-v7"Quantify origin offload before promising CDN savings
At 100,000 requests per second with a 200 KB average response, client egress is about 100,000 × 200,000 × 8 = 160 Gbps. With a 95% byte hit ratio, the simplified origin payload drops near 8 Gbps. Request hit ratio alone is insufficient: missing a few large videos may cost more origin bandwidth than hitting millions of tiny icons.
| Metric | Definition | Why it matters |
|---|---|---|
| Request hit ratio | Requests served without an origin fetch / cacheable requests | Explains origin request and compute offload |
| Byte hit ratio | Bytes served from cache / cacheable bytes served | Explains bandwidth and egress offload |
| Origin request rate | Misses plus revalidations reaching origin per second | Sizes origin connections and compute |
| Fill latency | Time to retrieve and store an object after a miss | Determines miss-path tail latency |
| Eviction rate | Objects removed before normal expiry | Exposes cache pressure and poor object mix |
State every simplification
Collapse misses and tier caches to protect the origin
A global CDN can turn one expired hot object into thousands of simultaneous origin requests because every edge observes its own miss. Request collapsing elects one fill per cache key at an edge. A regional or upper-tier cache then reduces the number of edge locations allowed to contact the origin.
- 01Edge A receives the first miss for version
v8and becomes the local fill owner. - 02Equivalent requests at Edge A wait, receive stale content when policy permits, or fail quickly rather than opening duplicate fills.
- 03The lower tier asks its assigned upper tier. Other lower tiers may converge on the same upper-tier object.
- 04Only an upper-tier miss reaches the origin, where an application-level single-flight guard can provide a final protection layer.
- 05The response is admitted only if size, status, directives, and policy allow it; waiting requests receive the same validated representation.
Active Recall Checkpoint
Why can a CDN with thousands of edge locations increase origin load during a synchronized expiry?
Hint: A cache miss is local unless another tier coordinates it.
Show answer & explanation
Each edge can independently miss and fetch the same hot object. Local request collapsing limits duplicate fills within one edge, while tiered caching or an origin shield coalesces misses across many edges. TTL jitter and stale-while-revalidate reduce synchronization further when the freshness contract permits stale service.
Keep cache safety separate from transport acceleration
Architectural Boundary · Do Not Confuse
Cache acceleration reduces origin work by reusing an already generated representation under an explicit cache contract.
Transport acceleration terminates connections near users and may reuse optimized, warm connections from edge to origin even when a response is not cacheable.
Dynamic and personalized traffic may benefit from edge TLS termination and connection reuse without ever entering a shared cache. Do not weaken cache isolation merely to claim a higher hit ratio.
TLS 1.3 early data is replayable
| Failure | Visible symptom | First checks |
|---|---|---|
| Bad cache key | Wrong variant or cross-user response | key dimensions, Vary, cookies, authorization |
| Purge lag | Regions serve different versions | purge propagation, object version, Age |
| Shield overload | Global miss latency rises together | upper-tier saturation, origin health, bypass rate |
| Routing shift | Hit ratio drops in one geography | BGP/DNS changes, edge selection, cold cache |
Treat cache policy and origin protection as correctness decisions
A cache key can leak data
Use it when
Cache versioned static assets and public read-heavy responses whose acceptable staleness is explicit. Use an origin shield and request collapsing when many edges may miss the same hot object.
Reconsider when
Bypass shared caching for sensitive personalized data, unsafe operations, rapidly changing responses without a staleness budget, or requests whose cache key cannot be bounded safely.
Active Recall Checkpoint
Why can purging every edge immediately be less reliable than versioned asset URLs?
Hint: Think about partial propagation and old references.
Show answer & explanation
A purge is a distributed control-plane operation and can be delayed or partially fail. A content-addressed or versioned URL is immutable, so new pages request the new object while old references continue to retrieve the old one safely. Purging remains useful for emergency removal and mutable public responses.
Production Postmortem & Verification
Production Incident Postmortem
- Trigger / Setup: A global media site releases breaking news; 5 million worldwide visitors hit origin servers directly for un-cached images.
- Production Symptom: Origin egress bandwidth saturates at 100 Gbps, dropping connection handshakes and elevating origin response latency beyond 10 seconds.
- Root Cause & Fix: Leverage CDN Anycast edge POPs with Origin Shielding; the shield absorbs edge cache misses and presents a single request to origin.
Acceptance Criteria & Outcomes
- Explain how BGP Anycast selects a policy-compatible network path to an edge, which is not guaranteed to be geographically closest.
- Differentiate static media edge caching from dynamic API acceleration (e.g. edge TLS termination).
- Formulate targeted cache purge strategies using surrogate keys (Cache-Tag headers).
Architectural & Technical Interview Review
Core FAANG Interview Questions
Q01How do you justify introducing Content Delivery Networks (CDNs) & Anycast Routing into a high-scale production architecture?Click to reveal model answer →Hide answer ↑Requirements & Design Brief
Introduce Content Delivery Networks (CDNs) & Anycast Routing only when measured scale, data size, or failure boundaries exceed single-node capacity. State the exact bottleneck, evaluate operational overhead, and verify that the trade-offs justify the added moving parts.
Q02What is the primary trade-off and hidden operational cost of Content Delivery Networks (CDNs) & Anycast Routing?Click to reveal model answer →Hide answer ↑Architecture Trade-offs
It provides Low global latency and reduced origin load, but costs Cache invalidation and regional consistency become operational concerns. Always name what the design spends immediately after what it buys.
Q03How do you handle catastrophic failure or dependency degradation in Content Delivery Networks (CDNs) & Anycast Routing?Click to reveal model answer →Hide answer ↑Failure Review & Recovery
Guard against An origin outage prevents cache misses from being filled.. Mitigate through bounded queues, circuit breakers with graceful fallback, and load shedding under extreme traffic.
Production Pitfalls & Anti-Patterns
Critical Failure Anti-Patterns
An origin outage prevents cache misses from being filled.
Scattering distributed components before identifying concrete write, read, or storage limits creates accidental points of failure and operational debt.
Production Mitigation: Start with the simplest modular design that satisfies requirements, evolving it only as measured constraints dictate.
Incorrect cache headers expose private content.
When a dependency degrades, immediate synchronized retries multiply downstream traffic, turning a minor slowdown into a total fleet outage.
Production Mitigation: Enforce strict retry budgets, exponential backoff, and full randomized jitter on all client calls.
Neglecting Observability and Metrics
Deploying architecture without distributed tracing or p99 tail latency alerts leaves engineers blind when partial partitions occur.
Production Mitigation: Export metrics for request rate, error rate, p95/p99 duration, and saturation at every service boundary.