Weighted Least Connections

Least connections with a weight for each server. The balancer divides each server's count of requests in progress by its weight and picks the lowest, so a bigger server is trusted with more at once. It combines what weighted round robin knows in advance with what least connections sees as it runs. Below, compare it with both on the same traffic.

Users579 req/sLoad balancerweighted least conn.123last sent toServer 13x sizeServer 21x sizeServer 31x size

The three servers drawn are one of your 2 groups of one big and two small servers, and each dot stands for 100 real requests. A bigger server has more workers, as a bigger machine has more cores. "In progress" under the balancer is what least connections compares. Switching the algorithm restarts the run with exactly the same requests.

Sizing the fleet and its weights
x average

Busiest hour vs the daily average. The book's estimate uses 2x.

req/s

Measured by load testing. A big server handles its size times this.

% busy

Headroom for spikes, slow requests and a failed server. 50 to 70% is common.

x planned peak

  1. 1. Average requests per second

    DAU x requests per user per day / 86,400
    = 1,000,000 x 50 / 86,400 = 579 req/s

  2. 2. Peak requests per second

    average x peak factor = 579 x 2 = 1,157 req/s

  3. 3. Servers needed

    9.65 small servers' worth, in groups of one 3x and two 1x servers = 2 big + 4 small

  4. 4. With the weight matching the size

    weight 3: every server at traffic now / total capacity = 58%

  5. 5. Without the weight

    while queues are short, equal counts in progress mean roughly equal shares of requests, as with round robin: big server 32%, small servers 96%

  6. 6. If the big server slows to half speed

    a fixed share (weighted round robin) asks it for 116%
    least connections moves work off it; a perfect split puts every server at 83%
  7. Set weights from each server's size, and let the counts in progress handle everything the weights cannot know about. Any value in red above is over 100%.
Where these formulas come from

Steps 1 and 2 are the estimation method from the back-of-the-envelope chapter of Alex Xu's System Design Interview, Vol. 1, including its 2x peak. The rest is standard capacity planning. Step 5 is approximate: once queues form, least connections starts sending the big server more, so the real gap is smaller than shown. The perfect split in step 6 spreads the same traffic over the reduced total capacity.

Limits of weighted least connections

Weights still have to be right

The weight says how much more a server can hold at once. Set it from the server's cores or measured capacity; if it is too high, that server is pushed harder than the rest. Move Server 1 weight away from its size to see it.

Speed and size are different things

A server with more cores holds more requests at once at the same speed per request: that needs a weight. A server that is simply faster finishes sooner, so it already holds fewer, and plain least connections gives it more traffic on its own.

Errors still look like free capacity

Like plain least connections, a server failing every request at once has nothing in progress and attracts more traffic. Health checks are still needed; see the Least Connections page.

A little more work per request

Dividing by a weight costs a little more than comparing counts. The Linux Virtual Server project keeps both versions for that reason, using the plain one when all servers are the same size.

How Linux Virtual Server and NGINX use it

One makes it the default; the other builds weights into its least connections method.

Linux Virtual Server: the default scheduler

Linux Virtual Server, the load balancer built into the Linux kernel (IPVS), documents weighted least-connection as picking the server j with "Cj/Wj = min { Ci/Wi }": connections divided by weight, smallest wins. Its documentation says plain least-connection "cannot get load well balanced among servers with various processing capacities", and the ipvsadm manual lists wlc as the scheduler that is used by default.

NGINX: least_conn with weights

NGINX's least_conn passes each request to "the server with the least number of active connections, taking into account weights of servers." The weights are the same weight= settings used by its default weighted round robin, so switching methods keeps them.
Sources and credibility

The Linux Virtual Server pages are the project's own documentation (the scheduling page is dated 2011; the manual page is a mirrored copy of ipvsadm's). The NGINX page is its official documentation. Quoted phrases are short attributed quotations; everything else is described in our words.