Load Balancing and Django

NOTE: THIS DOCUMENT IS INCOMPLETE. It is currently undergoing heavy editing, and I make no account for its accuracy as of this moment. -James Crasta

It all started with this IRC conversation on 6/30:

13:55 < mattmcc> There isn't really much about load balancing that's Django-specific.
13:55 < brantley> Mike: I don't think so.
13:55 < Crast> mikearagua: no, but that's not really a django-specific topic anyway, it's more a webserver-config and DNS 
13:56 < brantley> Personally, if I were to do that, I'd setup a specific box as the database-box, everything else would pretty much fall into place.
13:56 < mikearagua> yeah, i know, i just assume there would be particularities when doing it with a django app
13:56 < mattmcc> Not really.
13:56 < Crast> I have loadbalanced django using fastcgi and a single lighttpd in front, no proxying, and it worked... but I had no reason for it except for as proof-of-concept
13:57 < Crast> lighttpd does a decent job of load-balancing fcgi
13:57 < brantley> Crast, was that on multiple machines?
13:57 < Crast> brantley: yessuh, though admittedley one of them was the lighttpd machine itself
13:58 < Crast> I took that one out of the loop and substituted a laptop just to see how things would change
13:58 < brantley> Hrm, I don't understand, exactly...  Lighttpd sent the requests to fastcgi's on different servers?
13:58 < Crast> and the database server just ran on the lighttpd machine
13:59 < Crast> brantley: yep.  you can specify multiple fastCGI servers to a single path using lighttpd
13:59 < brantley> Damn, I didn't know that.
13:59 < brantley> That's pretty sweet.
13:59 < brantley> Yeah, Mike.  Do that.
13:59 < benbangert> Crast: lighty has some issues dealing with dead FastCGI processes
13:59 < Crast> therefore you don't get all the ugliness of HTTP proxying
13:59 < aurynn> yay
13:59 < Crast> benbangert: I don't use lighty to spawn the processes though, since they're not even running on the same machine
13:59 < mikearagua> i was mostly thinking about issues with session state
14:00 < benbangert> ie, when a Fast CGI process stops responding, it takes awhile before lightty will shut it down and restart it
14:00 < aurynn> automatic cpu underclocking, as well as load-based rampup fills me with complete joy.
14:00 < Crast> benbangert: however, you are correct
14:00 < benbangert> Crast: yea, but it doesn't seem to do a very good job noticing when a FCGI connection has stopped answering requests
14:00 < brantley> Err, I think the sessions are handled in the database with cookies, it should be fine.
14:00 < benbangert> I heard pound is really slick
14:00 < benbangert> http://www.apsis.ch/pound/
14:01 < Crast> benbangert: that's due to the natural timeout of the protocol though, it doesn't "not notice"  it just allows sufficient time to complete the request
14:01 < mikearagua> brantley: that's exactly what i wanted to know
14:01 < mikearagua> thanks.  
14:01 < brantley> I can't give you an exact affirmative, you should probably ask on django-users.
14:01 < brantley> But I'm pretty sure that's how it works.
14:01 < Crast> yeah, sessions are stored in the DB
14:02 < Crast> serialized python dicts more or less
14:02 < brantley> And hooked up through cookies.
14:02 < brantley> And as long as the browser thinks it's the same server, it'll work fine.
14:02 < Crast> yep
14:02 < mikearagua> it seemed that way to me but i could have been missing something
14:02 < brantley> Also: I would create a specific server that is dedicated to serving media.
14:03 < mikearagua> maybe i should start messing with lighttpd.  i use apache mostly
14:03 < brantley> With big fast hard-drives.
14:03 < Crast> well you do have a possible concurrency issue if two processes are using a session at the same time, whicever one writes second writes the finished session, however that'll happen even without load balancing, and it's really a non-issue
14:03 < brantley> That's really the database that needs to care about that, it seems to me.
14:03 < brantley> The first "load-balancing" should be separating the application server from the media serving.
14:04 < brantley> Then you go from there.
14:04 < Crast> I don't mind writing up a doc of my experimental results
14:04 < mikearagua> that would be cool
14:04 < brantley> That would be very progressive for enterprise Django.
14:04 < mikearagua> i could add to the doc as i go.  i would like to do some testing with load balancing before i deploy my app
14:05 < Crast> so I'll put the doc on the wiki then
14:05 < mattmcc> Serving the static media on a separate box is definitely the first step.  Optimizations for that server will be quite different from the ones serving up python.
14:05 < mikearagua> yeah, that's fairly easy though
14:06 < mikearagua> there are lots of docs describing how to serve static files fast
14:07 < mikearagua> especially with apache.  there are some issues with having dojo on a separate domain name though that i've run into so for the time being i'm running both off the same machine
14:07 < Crast> benbangert: there's a "disable-time" option to fastcgi servers, perhaps that needs to be tweaked for loadbalancing to be effective
14:07 < Crast> (on lighttpd, that is)

There are a few approaches to load balancing, and in many cases, they can be combined (some large websites use more than one of these strategies)

1. Round Robin DNS: Involves setting up multiple DNS records for the same hosting domain.
   • Advantages: Very simple to set up, mostly does not require any special server setup at all. Machines serving the site can be as geographically dispersed as you wish. You can also round-robin more than just web requests, so even things like FTP, IRC, and many other services can be load-balanced this way.
   • Disadvantages: The round-robin DNS is picked by your local DNS server serving you recursion results (usually your ISP for most end-users). There's no nice way to "switch" users down to another server if one server is overloaded or down, and making changes to the DNS can take days to propagate across DNS caches. It's very difficult to track if users are unable to access your site, since they may be sending requests nowhere.
2. Reverse Proxying Using a front-end server or dedicated hardware device in front of multiple web serving.
   • Advantages: This allows a web site to be served by multiple independent web servers. Most load balancer solutions include tools for monitoring server load and appropriately sending requests to the least-busy servers, and performing proper failover. With appropriate hardware this can scale to very large proportions, and achieve very good uptime / consistent performance.
   • Disadvantages: Set-up is tricky, and load-balancer hardware is often expensive. In most set-ups, the static files have to be mirrored to all the web-servers, and every machine has to be running a full webserver stack. Some issues pop up with things like some HTTP meta-information being lost due to the proxying.

3. distributing dynamic content-generating processes
   • Advantages:
   • Disadvantages:

Example "traditional" Load-balanced network layout

Obligatory ascii art diagram follows:

   ----------
((( INTERNET )))
   ----+-----
       |
       |
       |
  +-----------+
  |  Gateway  |-----+
  +-----------+     |
                    |
               +-----------------+  +-------------------+    +----------------+    
               |  Load Balancer  |  |  Database Server  |    |  Web Server 1  |
               +-----------------+  +-------------------+    +----------------+
                      |                      |                       |
Internal network -->  +----------------------+-----------------------+---------
                      |                      |
               +----------------+     +----------------+
               |  Web Server 2  |     |  Web Server 3  |     (more web servers)
               +----------------+     +----------------+

About this document

This document will describe in detail one possible method of achieving load balancing with django, including discussion on solving the "static file problem." This solution involves using the LigHTTPd web-server, FastCGI, and anywhere between two and infinity web servers. Also, a potential solution for reverse proxying using Apache and mod_python will also be discussed, but in lesser detail. Some performance numbers will be put out based on my independent study I have conducted using a variety of my own machines encompassing multiple platforms and operating systems, giving a general idea of the performance of load balancing.

This document will also discuss potential pitfalls of using a load-balanced setup, how they occur and how to avoid them. It is by no means comprehensive, but usually your issues will fall into one of a few categories, most of which can be solved rather simply.