There comes a time for many websites when their single hosting solution just doesn’t provide enough power to handle the amount of traffic they are receiving.
One solution is to simply get a more powerful server. However this is generally expensive, and doesn’t allow you to scale your solution nicely when you reach your next plateau of performance. This is an example of vertical scaling. Typically you want to avoid scaling out in this way as it is usually less cost effective, and harder to continue to scale.
The alternative and preferred method is horizontal scaling. Rather than add more hardware to your single server, you instead add additional servers. To do this you will need one server (or two or more if you want to be redundant) to act as a load balancer, and then several other servers to host the content (lets call these servers the application servers). In this way as you continue to grow your website and you need more power, you simply add another application server, and modify the load balancers configuration. It is that simple.
Configuring Load Balancing with HAProxy
HAProxy is a lightweight, high performance TCP/HTTP load balancer. It can be run on a budget server without any problems. Under extreme traffic you are more likely to saturate your network before your server running HAProxy runs out of memory or hits a performance wall.
Setting up HAProxy is pretty easy. Under most current Linux distributions you can simply install the application via the package manager (apt, yum, etc). If you have to compile HAProxy it is a simple ./configure, make, make install.
After installing the application you will want to edit your configuration file. The configuration file can be broken up into 4 parts. Global settings, Default settings, Frontend settings and Backend settings.
In your global settings you are going to setup options such as the user and group to run the application as, where to log to, whether to run as a daemon or not, and so on. Here is a typical global section that I use:
global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 50000 user haproxy group haproxy daemon chroot /var/chroot/haproxy
The first 2 lines are setting up logging to the syslog daemon. The next line sets “Sets the maximum per-process number of concurrent connections”. It is important to set this number high so that users don’t go unserviced. The next two lines are setting the user and group. Finally the last two lines say to run the server as a daemon, and to chroot the server to /var/chroot/haproxy for security reasons.
A “defaults” section sets default parameters for all other sections following
its declaration. So you can use this section to setup common configurations before fine tuning your frontends and backends. Here is a typical defaults section I use:
defaults log global mode tcp option httplog option forwardfor retries 2 redispatch maxconn 50000 timeout connect 10000 timeout client 30000 timeout server 60000 stats uri /ha_stats stats realm Global\ statistics stats auth myusername:mypassword
The first line tells HAProxy that all logging will be the same as the global settings. This ensures that all requests will get logged.
The next line sets the default connection mode to be tcp.
The option httplog will give you more in depth logging compared to the very sparse logs that are produced otherwise.
The option forwardfor will enable the X-Forwarded-For header to get sent to your application servers. This is important if you want to log the remote ip address of your requester on your application servers.
The retries option sets the number of attempts HAProxy will try when connecting to an application server. Best to keep this to a low number of 2 or 3.
The redispatch option will allow the final connection attempt to be made to a different application server. This is helpful if one of your application servers is under high load, as the request could then be sent to another server.
The maxconn line is the same as the global section.
The next 3 lines specify timeouts. timeout connect is the amount of time before making the initial connection to the application server times out, timeout client is the time for a client to make a response before they get timed out, and finally the timeout server is the time for the application server to respond before it is timed out. You may want to set a higher timeout server if you are running longer requests from your server. For example I know in drupal, running the drupal cron can take a long amount of time before the server sends any response back data, so to ensure the request doesn’t time out a high value here is needed.
Finally the last 3 lines specify a web accessible area where you can view HAProxy statistics. The uri states what uri to access the stats at (you want this to be unique so that it won’t collide with any webpages you are load balancing), the realm is simply the realm of the htpasswd that will be asked, and finally the auth specifies the user and password.
The frontend settings are used to describe sets of listening sockets that will accept connections from a client. A typical frontend section I like to use is:
frontend www *:80 maxconn 40000 mode http default_backend www_farm
The initial declaration gives the frontend a name (www) and sets it to listen on all ip’s on port 80.
The maxconn option is similar to above. As with the global setting you will want to set this setting extremely high as you won’t want clients to timeout while trying to connect.
The mode I am now setting to http. By setting the mode to http additional checks are made to make sure the request is a valid http request.
Finally the default_backend specifies what backend set of application servers will have this traffic load balanced to. Note that a single backend section can have multiple application servers.
The backend settings are used to specify a set of application servers that will handle the requests being load balanced through the frontend. Here is a typical backend section:
backend www_farm mode http balance roundrobin server backend-server-1 10.1.1.1:80 maxconn 250 check server backend-server-2 10.1.1.2:80 maxconn 250 check
The backend option determines the name of the backend you are defining. Note you use this name in the frontend section.
The next line is once again setting the mode to http.
The balance option sets the type of load balancing you want to do. I find roundrobin to be the most effective. It will send each new request to a different application server in order. There are many different balancing types you can set, and it is best to view the documentation to find the one that suits you best.
Finally the server options are used to specify each application server in this backend that will receive traffic. You give the server a name, then specify the IP and port, finally you can add some optional settings. I like to set a maxconn which is the maximum number of connections the application server can receive at a given time (this will help prevent your application servers from getting overrun with requests). I also like to add the check setting. The check setting will periodically check the application server to see if it is up and healthy. If the application fails this check 3 times it is removed from the rotation of load balancing. This is also helpful for allowing applications to cool down when you are getting hit by a lot of requests.
The Full Config
As you can see HAProxy is really easy to setup and configure. It has a lot of features that I didn’t talk about, and I suggest checking out the documentation on their website at: http://haproxy.1wt.eu/download/1.4/doc/configuration.txt. Here is the full config from above:
global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 50000 user haproxy group haproxy daemon chroot /var/chroot/haproxy defaults log global mode tcp option httplog option forwardfor retries 2 redispatch maxconn 50000 timeout connect 10000 timeout client 30000 timeout server 60000 stats uri /ha_stats stats realm Global\ statistics stats auth myusername:mypassword frontend www *:80 maxconn 40000 mode http default_backend www_farm backend www_farm mode http balance roundrobin server backend-server-1 10.1.1.1:80 maxconn 250 check server backend-server-2 10.1.1.2:80 maxconn 250 check
Advanced Features of HAProxy
The above configuration is a pretty basic configuration for HAProxy and is enough to get a pretty solid load balancer up and running. There are a few additional settings I find useful however, which I will outline below.
The errorfile option in HAProxy allows you to specify an html page to display to a client when an application server returns an error code. If you have ever seen those Twitter whale messages saying their servers are over capacity then you will know what I am talking about. An error file can be defined in any section of the HAProxy config file. I typically set it up under the default section unless I want different error pages for different sites. You can setup custom messages for the following codes: 400, 403, 408, 500, 502, 503, and 504. Basically what the directive looks like is:
errorfile 400 /path/on/haproxy/server/400.http
Where 400 is the error code, and the 400.http is the http response file to load. It is important to note that the file should contain the full http response back to the user. For example here is a sample 400.http file:
HTTP/1.0 400 Bad request^M Cache-Control: no-cache^M Connection: close^M Content-Type: text/html^M ^M
400 Bad requestYour browser sent an invalid request.
It is important that you include the proper carriage returns for an HTTP response. Also it is highly important not to display images that are hosted on your load balancer! The reason is, if you have an over capacity error for example, and all your application servers are at very high load, if you display an image that is hosted on your load balancer, then you will be sending another request to your already over capacity application servers, causing even more load. That is why it is important to host these images off site, such as on a CDN or a free file sharing site like imageshack.
ACL’s are what make HAProxy so versatile as a load balancer. With ACL’s you can define tests and send traffic to different backends based on those tests. For example you may have a set of application servers running Apache/PHP which are hosting a set of websites. Then you have another application server running Ruby on Rails for a different set of websites. Rather than have an application server running both Apache/PHP and Ruby on Rails, you could have two separate groups of application servers. 1 group running Apache/PHP the other running Ruby on Rails. Finally rather than having 2 load balancers to balance the traffic to these applications, you could setup an ACL in your HAProxy config to send traffic to the correct servers, and have all the traffic go through your single HAProxy server.
So lets pretend we have 4 servers. 2 servers are running Apache/PHP with the IP’s of 10.1.1.1 and 10.1.1.2. The 2 other servers are running Ruby on Rails with IP’s of 10.1.1.3 and 10.1.1.4. We will setup 2 backends in our configuration for these servers. It will look something like this:
backend apache_php_farm mode http balance roundrobin server apache_server0 10.1.1.1:80 maxconn 250 check server apache_server1 10.1.1.2:80 maxconn 250 check backend ruby_on_rails_farm mode http balance roundrobin server ruby_server0 10.1.1.1:80 maxconn 250 check server ruby_server1 10.1.1.2:80 maxconn 250 check
We now have our backend’s defined. We now need to setup an ACL to send the traffic to the correct backend. Lets say in our example that you have 2 website that are on Ruby on Rails called myrubywebsite.com, and iloverubyonrails.com. Lets assume the rest of your websites are running on Apache/PHP. What we can do is setup an ACL for the ruby domain, and then send all other traffic to the Apache/PHP servers. We would do this in the frontend section. It would look something like the following:
frontend www *:80 maxconn 40000 mode http acl ruby hdr_sub(host) myrubywebsite.com iloverubyonrails.com use_backend ruby_on_rails_farm if ruby default_backend apache_php_farm
With 3 lines in the config we have now separated out our Ruby traffic from our Apache/PHP traffic. Lets dissect the configuration lines.
The acl line (under the mode http) starts by giving the acl a name, we gave our acl a name of ruby. The next option sets the criteria for matching. In our case we used hdr_sub(host). This criteria will return true if the host being accessed contains one of the strings that follow it. Finally we list all the domains that we want to match the host with.
On the next line in the config we tell the load balancer to use the Ruby on Rails backend if our acl matched true. Finally we set the default backend to be our Apache/PHP servers.
There are a lot of different criteria you can use for ACL’s so I highly advise looking at the HAProxy Documentation for for further clarification.
HAProxy is an easy load balancer to setup and configure. It is fast, lightweight, and extremely powerful. I will show you in a future blog post how with a simple cron script you can auto-generate your HAProxy configuration such that you can auto-scale your application using Amazon Webservices.