Spring Boot

Load Balancing

Load balancing distributes incoming requests across multiple instances of a service to maximise throughput, minimise latency, and avoid overloading any single instance. In Spring Cloud microservices, client-side load balancing is handled by Spring Cloud LoadBalancer, which resolves a logical service name to a list of live instances from Eureka and selects one using a configurable algorithm.

Client-Side vs Server-Side Load Balancing

In server-side load balancing a dedicated infrastructure component (AWS ALB, nginx, HAProxy) sits between the client and the services and distributes traffic. The client sends every request to the same address and the balancer decides where it goes. In client-side load balancing the client itself holds a list of available instances (fetched from a registry such as Eureka) and picks one before making the call. Spring Cloud uses client-side load balancing by default.

Java

// ── Server-side load balancing: ──────────────────────────────────────
//
//  OrderService
//      │
//      └──▶  Load Balancer (nginx / AWS ALB)
//                  │
//                  ├──▶  UserService instance 1  172.18.0.2:8081
//                  ├──▶  UserService instance 2  172.18.0.3:8081
//                  └──▶  UserService instance 3  172.18.0.4:8081
//
//  Client always calls one address (the balancer).
//  The balancer decides which instance gets the request.
//  Client has no knowledge of individual instances.
//
//  Pros: simple client, centralised control
//  Cons: extra network hop, single point of failure if balancer is down

// ── Client-side load balancing (Spring Cloud default): ───────────────
//
//  Eureka Registry
//    user-service:
//      172.18.0.2:8081  ✓
//      172.18.0.3:8081  ✓
//      172.18.0.4:8081  ✓
//
//  OrderService (has local registry cache)
//      │
//      ├── 1. Fetch instances from Eureka cache
//      │        [172.18.0.2, 172.18.0.3, 172.18.0.4]
//      │
//      ├── 2. LoadBalancer picks one (round-robin)
//      │        → selected: 172.18.0.3:8081
//      │
//      └── 3. Call 172.18.0.3:8081/api/users/1 directly (no middle hop)
//
//  Pros: no extra hop, no single point of failure
//  Cons: every client must include balancer logic + registry dependency

// ── Spring Cloud LoadBalancer replaces Netflix Ribbon: ───────────────
// Spring Cloud Netflix Ribbon was the original client-side LB.
// It was deprecated in Spring Cloud 2020.0 and removed in 2022.0.
// Spring Cloud LoadBalancer is the current replacement — it is
// included automatically with spring-cloud-starter-netflix-eureka-client.

Setting Up @LoadBalanced RestTemplate

Annotating a RestTemplate bean with @LoadBalanced tells Spring Cloud LoadBalancer to intercept every outbound call made by that template. When the URL uses a logical service name (e.g. http://user-service/...) the interceptor resolves it to a real host:port using the Eureka registry cache and the configured load-balancing algorithm.

Java

// ── Configuration: ───────────────────────────────────────────────────
@Configuration
public class AppConfig {

    @Bean
    @LoadBalanced   // ← activates Spring Cloud LoadBalancer interceptor
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

// ── Usage in a service: ───────────────────────────────────────────────
@Service
@RequiredArgsConstructor
public class OrderService {

    private final RestTemplate restTemplate;

    public UserResponse getUser(Long userId) {
        // "user-service" is resolved at runtime via Eureka.
        // LoadBalancer picks an instance on every call.
        return restTemplate.getForObject(
            "http://user-service/api/users/" + userId,
            UserResponse.class
        );
    }

    public OrderResponse createOrder(CreateOrderRequest request) {
        UserResponse user = restTemplate.getForObject(
            "http://user-service/api/users/" + request.getUserId(),
            UserResponse.class
        );

        // Call another service — also load-balanced:
        InventoryResponse inventory = restTemplate.getForObject(
            "http://inventory-service/api/inventory/" +
                request.getProductId(),
            InventoryResponse.class
        );

        return orderRepository.save(Order.from(request, user, inventory));
    }
}

// ── How the interceptor works internally: ─────────────────────────────
// 1. restTemplate.getForObject("http://user-service/api/users/1", ...)
// 2. LoadBalancerInterceptor.intercept() fires
// 3. Calls LoadBalancerClient.choose("user-service")
// 4. ReactorLoadBalancer fetches instance list from ServiceInstanceListSupplier
//    (backed by Eureka's local cache — no network call to Eureka)
// 5. RoundRobinLoadBalancer picks: ServiceInstance{host=172.18.0.3, port=8081}
// 6. URL rewritten: http://172.18.0.3:8081/api/users/1
// 7. Actual HTTP call made to 172.18.0.3:8081

Setting Up @LoadBalanced WebClient

WebClient is the non-blocking reactive alternative to RestTemplate. Annotating the WebClient.Builder bean with @LoadBalanced enables the same service-name resolution and load balancing without blocking the calling thread. It is the preferred choice for reactive services or high-concurrency scenarios.

Java

// ── Configuration: ───────────────────────────────────────────────────
@Configuration
public class WebClientConfig {

    @Bean
    @LoadBalanced   // ← same annotation, works on WebClient.Builder
    public WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }
}

// ── Blocking usage (equivalent to RestTemplate): ──────────────────────
@Service
@RequiredArgsConstructor
public class OrderService {

    private final WebClient.Builder webClientBuilder;

    public UserResponse getUser(Long userId) {
        return webClientBuilder.build()
            .get()
            .uri("http://user-service/api/users/" + userId)
            .retrieve()
            .bodyToMono(UserResponse.class)
            .block();   // block() makes it synchronous
    }
}

// ── Non-blocking reactive usage: ─────────────────────────────────────
@Service
@RequiredArgsConstructor
public class ReactiveOrderService {

    private final WebClient.Builder webClientBuilder;

    public Mono<OrderResponse> placeOrder(CreateOrderRequest request) {
        Mono<UserResponse> userMono = webClientBuilder.build()
            .get()
            .uri("http://user-service/api/users/" + request.getUserId())
            .retrieve()
            .bodyToMono(UserResponse.class);

        Mono<InventoryResponse> inventoryMono = webClientBuilder.build()
            .get()
            .uri("http://inventory-service/api/inventory/" +
                request.getProductId())
            .retrieve()
            .bodyToMono(InventoryResponse.class);

        // Make both calls in parallel, combine results:
        return Mono.zip(userMono, inventoryMono)
            .flatMap(tuple -> {
                UserResponse user          = tuple.getT1();
                InventoryResponse inventory = tuple.getT2();
                Order order = Order.from(request, user, inventory);
                return Mono.just(orderRepository.save(order));
            })
            .map(OrderResponse::from);
    }
}

Load Balancing Algorithms

Spring Cloud LoadBalancer ships with two built-in algorithms: RoundRobinLoadBalancer (default) and RandomLoadBalancer. Custom algorithms can be implemented by extending ReactorServiceInstanceLoadBalancer. The algorithm can be overridden globally or per target service using @LoadBalancerClient.

Java

// ── Default: RoundRobinLoadBalancer ──────────────────────────────────
// Cycles through instances in order using an AtomicInteger counter.
// Request 1 → instance[0]
// Request 2 → instance[1]
// Request 3 → instance[2]
// Request 4 → instance[0]  (wraps around)

// ── Override per service: ─────────────────────────────────────────────
@Configuration
@LoadBalancerClient(
    name = "user-service",              // applies only to user-service
    configuration = RandomLBConfig.class
)
public class UserServiceLBConfig { }

// Must be a plain @Configuration, NOT a @SpringBootApplication class:
public class RandomLBConfig {

    @Bean
    ReactorLoadBalancer<ServiceInstance> randomLoadBalancer(
            Environment env,
            LoadBalancerClientFactory factory) {
        String name = env.getProperty(
            LoadBalancerClientFactory.PROPERTY_NAME);
        return new RandomLoadBalancer(
            factory.getLazyProvider(name,
                ServiceInstanceListSupplier.class),
            name
        );
    }
}

// ── Override globally (all services): ────────────────────────────────
@Configuration
@LoadBalancerClients(defaultConfiguration = RandomLBConfig.class)
public class GlobalLoadBalancerConfig { }

// ── Custom algorithm — weighted round-robin: ──────────────────────────
public class WeightedLoadBalancer implements
        ReactorServiceInstanceLoadBalancer {

    private final String serviceId;
    private final ObjectProvider<ServiceInstanceListSupplier> supplier;

    @Override
    public Mono<Response<ServiceInstance>> choose(Request request) {
        return supplier
            .getIfAvailable(NoopServiceInstanceListSupplier::new)
            .get(request)
            .next()
            .map(instances -> {
                // Read "weight" from instance metadata (set in application.yml):
                // eureka.instance.metadata-map.weight=80
                List<ServiceInstance> weighted = new ArrayList<>();
                for (ServiceInstance instance : instances) {
                    int weight = Integer.parseInt(
                        instance.getMetadata()
                            .getOrDefault("weight", "1"));
                    for (int i = 0; i < weight; i++) {
                        weighted.add(instance);  // add proportionally
                    }
                }
                if (weighted.isEmpty()) {
                    return new EmptyResponse();
                }
                ServiceInstance chosen = weighted.get(
                    ThreadLocalRandom.current().nextInt(weighted.size()));
                return new DefaultResponse(chosen);
            });
    }
}

Health-Aware Load Balancing

By default, Spring Cloud LoadBalancer includes all registered Eureka instances regardless of health status. Enabling health-check filtering configures the ServiceInstanceListSupplier to exclude instances whose Actuator health endpoint returns DOWN or OUT_OF_SERVICE, so traffic is never routed to a degraded instance.

Java

// ── Enable health-check aware instance filtering: ────────────────────

// application.yml:
// spring:
//   cloud:
//     loadbalancer:
//       health-check:
//         initial-delay: 0
//         interval: 25s         # re-check instance health every 25 seconds
//       configurations: health-check  # activate HealthCheckServiceInstanceListSupplier

// ── Programmatic health-check supplier configuration: ────────────────
public class HealthCheckLBConfig {

    @Bean
    public ServiceInstanceListSupplier serviceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
            .withDiscoveryClient()       // fetch from Eureka
            .withHealthChecks()          // filter unhealthy instances
            .build(context);
    }
}

@Configuration
@LoadBalancerClients(defaultConfiguration = HealthCheckLBConfig.class)
public class GlobalLBConfig { }

// ── Zone-preference supplier (prefer same-zone instances): ────────────
public class ZonePreferenceLBConfig {

    @Bean
    public ServiceInstanceListSupplier serviceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
            .withDiscoveryClient()
            .withZonePreference()    // prefer instances in the same zone
            .withHealthChecks()      // still filter unhealthy ones
            .build(context);
    }
}

// application.yml — declare current zone so the supplier can prefer it:
// spring:
//   cloud:
//     loadbalancer:
//       zone: us-east-1a
//
// eureka:
//   instance:
//     metadata-map:
//       zone: us-east-1a   # instances tag themselves with their zone

Caching and Tuning

Spring Cloud LoadBalancer caches the instance list fetched from Eureka to avoid a registry lookup on every single request. The default cache TTL is 35 seconds. In highly dynamic environments reduce the TTL; in stable environments increase it to reduce Eureka load. Caching can also be disabled entirely for testing.

yaml

// application.yml — cache tuning: ────────────────────────────────────
// spring:
//   cloud:
//     loadbalancer:
//       cache:
//         enabled: true          # default: true
//         ttl: 35s               # evict and refresh after 35 seconds
//         capacity: 256          # max number of services cached

// ── Disable cache entirely (useful in tests or highly dynamic envs): ──
// spring:
//   cloud:
//     loadbalancer:
//       cache:
//         enabled: false
// WARNING: disabling cache means a Eureka lookup on EVERY request.
// Only do this in development or low-traffic scenarios.

// ── Retry on different instance after failure: ────────────────────────
// spring:
//   cloud:
//     loadbalancer:
//       retry:
//         enabled: true
//         max-retries-on-same-service-instance: 0   # don't retry same instance
//         max-retries-on-next-service-instance: 2   # try 2 other instances
//         retry-on-all-operations: false            # only retry GET/HEAD
//         retryable-status-codes: 500, 502, 503

// ── Verify LoadBalancer resolved correctly (debug logging): ───────────
// logging:
//   level:
//     org.springframework.cloud.loadbalancer: DEBUG
//
// Debug output example:
// [RoundRobinLoadBalancer] Selected instance: {serviceId='user-service',
//   host='172.18.0.3', port=8081, secure=false, metadata={zone=us-east-1a}}

Spring Cloud Gateway

OpenFeign