Random

The load balancer sends each request to a server picked at random. It keeps no turn order and no counts, so it needs no shared state, and any number of load balancers can do it at once without coordinating. Even on average, it is uneven over short stretches. Below, compare it with round robin on exactly the same requests.

Users463 req/sLoad balancerrandom1234last sent toServer 1Server 2Server 3Server 4

Each server drawn stands for one of your servers, and each dot for 100 real requests. Only the speed is slowed, so you can follow single requests. The random picks come from their own seeded stream, so switching the algorithm replays exactly the same requests, and Reset replays the same picks.

Sizing the fleet
x average

Busiest hour vs the daily average. The book's estimate uses 2x.

req/s

Measured by load testing one server until it cannot go faster.

% busy

Headroom for spikes and for random unevenness. 50 to 70% is common.

x planned peak

  1. 1. Average requests per second

    DAU x requests per user per day / 86,400
    = 1,000,000 x 50 / 86,400 = 579 req/s

  2. 2. Peak requests per second

    average x peak factor = 579 x 2 = 1,157 req/s

  3. 3. Servers needed n

    n = peak / (one server x target load), rounded up = 10

  4. 4. Load on each server, on average

    traffic now / (n x one server) = 58%

  5. 5. How uneven random is

    each server gets m = traffic now / n = 116 req/s
    its count in any one second typically swings by about sqrt((1 - 1/n) / m) = ±9%, and over a minute by ±1.1%

  6. 6. What that means for headroom

    at 58% average, a typical busy second asks a server for about 63%
    Round robin has no such swing in how requests are dealt out; the work in them still varies.
  7. Random is close to even for busy services, where each server gets hundreds of requests a second. For small services, or long-running requests such as open connections, its unevenness lasts long enough to matter.
Where these formulas come from

Steps 1 and 2 are the estimation method from the back-of-the-envelope chapter of Alex Xu's System Design Interview, Vol. 1, including its 2x peak. Step 5 is standard probability: the number of requests a server receives in a fixed time, when each lands on it with chance 1 / n, has a standard deviation of sqrt(m (1 - 1/n)), so the relative swing is sqrt((1 - 1/n) / m): close to 1 / sqrt(m) with many servers, and zero with one.

Limits of random

Even on average, not in every moment

Random spreads requests evenly only in the long run. Over short stretches some servers get several in a row. With little traffic per server, those bursts become queues.

Blind to the servers

Like round robin, it ignores how busy or how fast each server is. Weights help with known size differences; for everything else it needs a look at the servers, which is what power of two choices adds cheaply.

No shared state is its strength

Round robin needs a turn counter, least connections needs counts. Random needs nothing, so many load balancers, or every client of a service, can pick independently and still spread the load.

Failures are not inherited

Envoy notes that random avoids a bias round robin can have: when one host fails, the host after it in the order can end up with extra traffic. A random pick has no "after".

How Envoy, NGINX and AWS use random

All three offer it, two of them with a twist.

Envoy: random

"The random load balancer selects a random available host." Envoy's documentation adds that it "generally performs better than round robin if no health checking policy is configured", and that "Random selection avoids bias towards the host in the set that comes after a failed host."

NGINX: random, optionally two

With random, "a request is passed to a randomly selected server, taking into account weights of servers." Its optional two parameter picks two servers at random and sends the request to the better one, by default the one with fewer active connections: power of two choices, the next page.

AWS: weighted random

An Application Load Balancer's weighted random algorithm "routes requests evenly across healthy targets in the target group, in a random order." It is the only algorithm that supports AWS's Automatic Target Weights, which routes traffic away from targets that behave anomalously.
Sources and credibility

All are the projects' and vendor's own documentation (primary sources). Quoted phrases are short attributed quotations; everything else is described in our words. Check the pages for current behaviour.