DevKit Market
Theme
Explore Products
All lessons

Day 01 · System Design

Load Balancing (Layer 4 vs. Layer 7)

How load balancers spread traffic across servers, and when to route at the transport layer versus the application layer.

01Overview & core concept

A load balancer (LB) acts as a reverse proxy that distributes incoming network traffic across multiple backend servers, so no single server becomes a single point of failure (SPOF) or a bottleneck.

  • Layer 4 (Transport Layer)

    Routes traffic based on network-level attributes (IP addresses and TCP/UDP ports) without inspecting the application-level payload.

  • Layer 7 (Application Layer)

    Inspects application-level data (HTTP headers, cookies, URL paths, message payloads) to make intelligent routing decisions.

02Architecture & request flow

                     ┌──────────────────┐
                     │   Client App     │
                     └────────┬─────────┘
                              │ HTTPS Request
                              ▼
                     ┌──────────────────┐
                     │ DNS / Anycast IP │
                     └────────┬─────────┘
                              │
                              ▼
                     ┌──────────────────┐
                     │  Layer 4 LB      │ (e.g., AWS NLB / HAProxy TCP)
                     │ (TCP Termination)│
                     └────────┬─────────┘
                              │
                              ▼
                     ┌──────────────────┐
                     │  Layer 7 LB      │ (e.g., NGINX / Envoy / AWS ALB)
                     │ (TLS Term, Path) │
                     └────┬────────┬────┘
           /api/v1/auth   │        │   /api/v1/stream
         ┌────────────────┘        └────────────────┐
         ▼                                          ▼
┌─────────────────┐                        ┌─────────────────┐
│  Auth Services  │                        │ Video Streaming │
│ (Auth Cluster)  │                        │ (Stream Cluster)│
└─────────────────┘                        └─────────────────┘

A client request resolves through DNS or an Anycast IP, hits a Layer 4 load balancer for fast TCP-level distribution, then a Layer 7 gateway that terminates TLS and routes by URL path to the right backend cluster.

Routing algorithms

  • Round Robin / Weighted Round Robin

    Distributes requests sequentially across backends, optionally weighted by server capacity.

  • Least Connections

    Directs traffic to the server with the fewest active connections.

  • Consistent / IP Hashing

    Maps client IP hashes to specific servers (useful for sticky sessions or local in-memory caches).

03Real-world example: Netflix Edge Routing

  • Edge Layer (L4/L7)

    Netflix uses Amazon Route 53 with latency-based routing to direct global users to the nearest AWS region.

  • Zuul (L7 API Gateway)

    Once traffic reaches a region, Zuul inspects incoming HTTP request headers and paths to route requests dynamically to microservices (e.g., authentication, recommendation engine, playback metadata).

  • TLS Termination & Header Manipulation

    Layer 7 proxies terminate TLS at the edge, strip sensitive internal headers, and inject tracking IDs (e.g., distributed trace tokens).

04Architectural trade-offs

DimensionLayer 4 (Transport)Layer 7 (Application)
Performance & LatencyExtremely fast; handles millions of packets/sec with minimal CPU overhead.Higher latency and CPU usage due to packet parsing and TLS decryption/re-encryption.
Routing FlexibilityLimited to IP and port numbers. Cannot route by URL path or cookies.High; supports path-based routing (/api/v1/users vs /static), cookie-based stickiness, and header inspection.
Security & FeaturesBasic DDoS mitigation (SYN flood filtering).Advanced WAF integration, rate limiting, token verification, and content caching.
Connection HandlingSimple TCP/UDP proxying.Supports HTTP/2 multiplexing, gRPC proxying, and WebSocket upgrades.

Failure modes & mitigations

  • Load Balancer Failure

    Deploy load balancers in an active-passive or active-active pair using VRRP (Virtual Router Redundancy Protocol) or DNS Anycast.

  • Backend Server Death

    Implement proactive active health checks (e.g., polling /healthz endpoints) and passive health checks (monitoring error rates).

05Interview question & approach

How would you design a load balancing layer for a high-throughput video streaming service that serves both static video chunks and dynamic user authentication requests?

  • Tiered Architecture

    Separate the concerns by placing a high-throughput Layer 4 load balancer (or CDN/Anycast layer) at the edge, followed by a Layer 7 gateway (like Envoy or NGINX).

  • Path-Based Routing (L7)

    Direct requests matching /static/* or media chunks (.m3u8, .ts) to caching layers or CDN edge nodes, while routing /auth/* and /api/* to stateless backend compute clusters.

  • Session Persistence vs. Statelessness

    Emphasize stateless services using JWTs over session stickiness to allow even traffic distribution.

  • Health Checking & Autoscaling

    Explain how L7 health checks detect application-level deadlocks (e.g., 500 errors vs. simple TCP connection acceptance) and trigger auto-scaling events.