Apache on Windows x64 as Load Balancer for Mongrel

Rails, Windows

Its been awhile since I’ve posted any rails stuff on here but the drudgery with Windows and its 64 bits continues.

Cool things I’ve done with Rails is the past month:

  • PDF to HTML converter with multiple file upload (javascript/flash/ruby working together)
  • HTML to CSV converter (only works for one report format but its from one of the largest call center operators in the US so…you know whatever)
  • FTP downloader (not quite working yet)
  • cygwin sort of working (this gets me access to several useful Unix commands from Rails like
    cat

    and

    pdftohtml
  • Google maps with route making ability (javascript and ruby working together)
  • Google maps routing on custom routes

Ok so I’m awesome for those things lets continue.

Since I needed Unix commands cygwin had to work….BUT Pen, the load balancer I’ve been using, also needs a cygwin dll file in its directory work. I don’t like Pen b/c I can’t get it to start up when or if my server reboots – won’t install as a service in x64 land so its not a good option.

I decided to take the dive into Apache. Once you get past some of the scary config options (not that they are really scary its just daunting looking at config files you’ve never seen before and trying to get the damn thing to work). I followed a great tutorial on getting load balancing working in Ubuntu from UrbanPuddle – and then applied that to windows. The name-based virtual host thing is pretty easy to set up – tell Apache where to listen – port 80 – and then give it a list of possible names that could come in, then redirect that name to the right mongrel processes. We can also tell it to serve static stuff (images, stylesheets, and javascripts) and not bother mongrel for them.

Another problem I’ve had is long running processes. Like – we need to read a CSV file that is 26,000 lines long and do processing on each line and assign sales data. Ok, sure, well the whole thing takes like 8 minutes to run (I have it setup so it will tell you what row your on in the CSV file while it loads and gives you the average load time for each file based on previous runs). I had to do all this b/c the you get an hourglass for 8 minutes – thanks to load balancing other users can still access other parts of the app (you can even pop open a new tab to complete some other task if you want). But the real rub is that Spawn, from Tom Anderson (fellow Denver rubyist) for some reason won’t work on my x64 machine – works awesome in Unix, but no go on windows. It looks like I can get the long running process to fork but any ActiveRecord calls don’t get called after spawn renders my “waiting for data to load” page. If I ask it to wait, then it will wait for the process to finish before rendering a page but that really defeats the purpose b/c I still get 8 minutes of hourglass. Tom added Threading support, which requires I set concurrency to true in my environment file but when I tried this again this morning all I got was unrendered HTML output (looks like apache didn’t like the new settings) and if I hit a mongrel port directly everything renders but I have to reload each page I navigate to for the rendering to take place. Shitty.

I revisited spawn b/c I was getting:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /call_data/load_data.

Reason: Error reading from remote server

Nice…WTF I say. Turnsout this is a timeout problem. Apache defaults to 300 second timeout – awesome for a general website but my shit is an internal app that can take up to 15 minutes to load stuff. So I said, cool lets try Spawn again and see if I can get around not setting the timeout to like 20 minutes (so not safe omg). And whalaa…Spawny no worky.

So I had to set the timeout directive in apache to 1200 seconds. You can do this (at least in my setup) by uncommenting (remove the #)
Include conf/extra/httpd-default.conf
at the bottom of http.conf and then set
Timeout 1200
in httpd-default.conf (in the extra directory) then restart apache – worked for me

At least apache works and it will boot up when the server restarts – I tested it – so I have a persistent ruby on rails application that does not require anyone to administer the server on a crash (like how it should be) only took 10 months to figure out.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s