Load Balancing on the Cheap

A few months ago, we started to look at alternate load balancing methods for our site. At the time, we had a pretty basic setup using Nginx. While this worked for the most part, our setup lacked some of the nicer features of a traditional load balancer such as health checks and automated removal of poorly responding backend servers. This meant that every time a backend server went down, there would be some, however minor, user visible impact until we it could be removed from rotation.

Thus, began the search for a new solution. Our requirements were fairly simple:
- easy to setup and understand
- ability to health check and automatically remove failing servers
- low cost

From here, additional features would be a bonus, but not required for our immediate goals. After looking at several solutions, we settled on using HAProxy, an opensource and free solution.

The benefits were noticeable almost instantaneously. One of the first observations was that hosts behind HAProxy were able to serve the same amount of traffic, but with a lower load average. What we surmised is that there was less setup/teardown of tcp connections which freed up system resources to actually serve user requests. Once we finished our full migration, we actually saw the loads on our webservers drop nearly in half, which allowed for more growth on our existing infrastructure.

In the weeks that followed, we started to take advantage of additional features provided by HAproxy. For example, the status page also has a csv output option. By parsing this and doing some small calculations, we are able to see the various states of our servers and get an overall percentage of our serving capacity. It's well known that computers go down, it's just the nature of things. By utilizing cluster wide checks, we don't have to get alerted for a single host down, but rather if a large amount of capacity is down.

Enough of talking about some of the features, let's take a look at a configuration example.

Let's say you have 5 web servers: web1, web2, web3, web4, web5. A basic HAproxy configuration would look like this:

global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
maxconn 1024
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
#debug
#quiet

defaults
log global
mode http
option httplog

listen webservers :80
mode http
server web1 web1:80 weight 1 check inter 15s
server web2 web2:80 weight 1 check inter 15s
server web3 web3:80 weight 1 check inter 15s
server web4 web4:80 weight 1 check inter 15s
server web5 web5:80 weight 1 check inter 15s

All done! You now have a load balancer that distributes requests among 5 servers and checks every 15 seconds to see if their ports are responsive.

For more information on HAProxy and configuration options, check out their website.