Advanced · Distribute

API Gateways & Service Discovery

Route external traffic through one gateway and let services find each other through discovery instead of hard-coded addresses.

Primary source: Spring Cloud 2024 Reference Documentation

40 min

Lesson 2 of 4

1d streak

The Architectural Question

Hard-coded host:port pairs break on every reschedule, and edge logic scattered across services means every team re-implements rate limiting and TLS differently.

What this guide proves

  • Centralize edge concerns (routing, TLS, rate limits) in Spring Cloud Gateway
  • Compare platform DNS, client-side registries, and platform service meshes
  • Keep service-to-service calls resolvable across replicas and restarts

Spring Cloud Gateway: Architecture, Predicates & Filters

Spring Cloud Gateway acts as the single public entry point into a microservice ecosystem. Built on Spring WebFlux and Project Reactor, it routes incoming client requests asynchronously to downstream services based on route predicates and filters.

Gateway components:

1. Route Predicates: Evaluate HTTP request attributes (Path, Header, Method, Query, Host). If matched, the request is dispatched to the target URI.

2. Gateway Filters: Modify incoming HTTP requests before routing (e.g. stripping path prefixes, injecting JWT authentication headers) or modify responses before returning to the client.

src/main/resources/application.yml
spring:
  cloud:
    gateway:
      routes:
        - id: order-service-route
          uri: lb://order-service
          predicates:
            - Path=/api/orders/**
          filters:
            - StripPrefix=1
            - AddRequestHeader=X-Gateway-Forwarded, true

Distributed Rate Limiting with Redis & Token Bucket

To protect downstream services from distributed denial-of-service (DDoS) spikes and brute-force traffic, Spring Cloud Gateway integrates with Redis via the RequestRateLimiter filter factory using the token bucket algorithm.

src/main/java/com/example/gateway/RateLimiterConfig.java
package com.example.gateway;

import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import reactor.core.publisher.Mono;

import java.security.Principal;

@Configuration(proxyBeanMethods = false)
public class RateLimiterConfig {

    @Bean
    public KeyResolver userKeyResolver() {
        // Resolve rate limit key by authenticated user ID or client IP
        return exchange -> exchange.getPrincipal()
            .map(Principal::getName)
            .defaultIfEmpty(exchange.getRequest().getRemoteAddress().getAddress().getHostAddress());
    }
}

Service Discovery: Eureka vs Kubernetes CoreDNS

Historically, Spring Cloud relied on Netflix Eureka for application-level service discovery where microservice instances register their dynamic IPs at startup. Today, in Kubernetes environments, client-side discovery with Eureka is redundant.

FeatureNetflix Eureka / ConsulKubernetes CoreDNS
ArchitectureClient-side library registrationServer-side cloud-native DNS
MaintenanceRequires dedicated Eureka discovery server clusterBuilt-in Kubernetes infrastructure
Polyglot SupportRequires Spring Cloud / JVM client integrationUniversal across Java, Go, Node, Python
RecommendationUse for legacy on-premise deploymentsRecommended for all Kubernetes deployments

Testing Gateway Routing and Rate Limit Rejections

Verify gateway routing, path stripping, and rate-limiting responses using curl:

snippet.bash
# 1. Send request through Gateway to routed service
curl -i http://localhost:8080/api/orders/ORD-42
# HTTP/1.1 200 OK -> Gateway routes to lb://order-service and strips /api prefix

# 2. Exceed token bucket rate limit
for i in {1..20}; do curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/api/orders/ORD-42; done
# Output: 200 ... 200 ... 429 (Too Many Requests returned by Gateway)

Active Recall Checkpoint

Why is Kubernetes CoreDNS preferred over Eureka in modern cloud deployments?

Hint: Consider infrastructure simplicity vs application-level service discovery.

Show answer & explanation

Kubernetes CoreDNS is built into the infrastructure, works universally for all programming languages, and avoids the operational burden of deploying and maintaining dedicated Eureka server clusters.

Runtime Architecture & Flow

Dependency direction

Boundaries follow ownership

A microservice boundary buys independent ownership and deployment by introducing network, consistency, and operational failure modes.

Centralize edge concerns (routing, TLS, rate limits) in Spring Cloud Gateway

B1

Compare platform DNS, client-side registries, and platform service meshes

B2

Keep service-to-service calls resolvable across replicas and restarts

B3

Read these as design constraints. The arrows belong in the implementation only after the dependency rule is chosen.

Working Implementation

Code Walkthrough

Put the order and catalog services behind a gateway, register them with Eureka or DNS, and call catalog only by logical name.

GatewayRoutingLab.java
package com.example.gateway;

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration(proxyBeanMethods = false)
public class GatewayRoutingLab {

    @Bean
    public RouteLocator customRoutes(RouteLocatorBuilder builder) {
        return builder.routes()
            .route("catalog-route", r -> r
                .path("/api/catalog/**")
                .filters(f -> f
                    .stripPrefix(1)
                    .addRequestHeader("X-Gateway-Forwarded", "true"))
                .uri("lb://catalog-service"))
            .route("order-route", r -> r
                .path("/api/orders/**")
                .filters(f -> f
                    .stripPrefix(1)
                    .retry(config -> config.setRetries(2)))
                .uri("lb://order-service"))
            .build();
    }
}
Spring Cloud Gateway routes requests asynchronously using WebFlux and Netty for non-blocking I/O.
Filters strip path prefixes and inject correlation headers before forwarding to downstream services.
In Kubernetes environments, cloud-native CoreDNS replaces legacy client-side Eureka registration.

Synthesis & Production Verification

Deliberate Failure Condition

  1. Break: Point ordering at catalog via a fixed localhost port in configuration
  2. Observe: The second replica is unreachable and deploys fail when ports collide
  3. Repair: Discovery turns a logical name into live instances; the gateway gives external clients exactly one stable entry point.

Production Acceptance Criteria

  • Reach both services through one gateway route
  • Scale catalog to two replicas and observe load spreading
  • Explain why Kubernetes apps often skip a registry entirely

Architectural & Technical Review

Core Interview Questions

Q01How do Route Predicates and Gateway Filters coordinate request routing in Spring Cloud Gateway?Show answer

Predicates test incoming HTTP request conditions (Path, Header, Host). When matched, the request passes through an ordered filter chain (StripPrefix, AddRequestHeader, RateLimiter) before being dispatched to the downstream URI.

Q02How does the Redis RequestRateLimiter filter enforce the token bucket algorithm in Spring Cloud Gateway?Show answer

It evaluates replenishRate and burstCapacity in Redis. Each incoming request consumes a token from the bucket; when tokens are exhausted, the Gateway immediately rejects requests with HTTP 429 Too Many Requests.

Q03Why is cloud-native Kubernetes CoreDNS preferred over Eureka in modern deployments?Show answer

Kubernetes CoreDNS is built into the container runtime, requires zero JVM-level client dependencies, works universally across all programming languages, and eliminates the operational overhead of managing Eureka clusters.

Production Pitfalls

Critical Anti-Patterns

  • Pitfall: Executing blocking database or synchronous HTTP calls inside a Spring Cloud Gateway filter — Blocks the underlying Netty event loop, freezing gateway throughput for all routed microservices.
  • Pitfall: Omitting distributed rate limiting on public API Gateway entry points — Leaves backend microservices vulnerable to DDoS surges, brute force attacks, and cascading resource exhaustion.

Primary Documentation & Specifications

Supplementary Module Resources