Pearware Blog

In this post, I’ll be talking about a solution I developed at my day job to improve the performance of our web site by allowing the browser to cache JavaScript, CSS, and image files. We were noticing that much of our traffic was from requests for these assets, rather than our page content. Since these asset files rarely change (once per production deployment), we wanted to have the user’s browser cache them until the next build.

At the beginning of March, I moved from Columbus, OH to Denver, CO. Nikki and I have a lot of friends here and, so far, we’ve had a blast hanging out, playing hockey, and hiking.

I’m writing an application that downloads access logs from our production servers and runs the AWStats package against them to create the statistics web pages. This process is setup as a Rake task that uses the Net::SFTP library used by Capistrano, written by Jamis Buck. There is also a front-end Rails application to manage each of the applications to be retrieved. Everything was working great until I tried to grab a 550MB file from one of our servers. Net::SFTP chocked as it ran out of memory.

It turns out that the command:

    sftp.get_file log_file, local_file

ends up putting the whole file into memory, which is fine for small files, but not the large one that I was trying to download. Luckily it wasn’t too bad to refactor my class. Here’s the new code to achieve the same effect as the above sftp.get_file command.

        stat = sftp.stat( log_file )
        offset = 0
        file_length = stat.size
        length = 64 * 1024 * 1024
        File.open(local_file, File::CREAT|File::TRUNC|File::RDWR, 0644) do |f|
          while (offset < file_length)
            sftp.open_handle(log_file) do |handle|
              data = sftp.read(handle, :length => length, :offset => offset)
              f.write(data)
              offset += data.length
            end
          end
        end

This downloads the file in 64MB increments, using only that much memory at any time.

RadiantOnRails is a Radiant extension I created to allow a Rails application to co-exist with Radiant, giving the developer the best of both (dynamic and static) worlds. You can now visit the new project page on RubyForge.

This extension will be a major piece for the website I’m currently working on, RealIdaho.com. Most of the pages for the site just display static content about cities, but there are other portions of the site that will be fully-dynamic Rails pages. This extension allows me to combine both portions of the site into one application so we can develop the Rails pages for displaying the data-driven pages and still leverage the wonderful user interface created by the Radiant team. This will allow the realtor to make content changes without me having to make code changes and redeploy.

RadiantOnRails currently allows Radiant snippets to be inserted into Rails views and the next step is to allow the Rails views to use the Radiant layouts so that the look ‘n’ feel of the site is consistent, while keeping the views DRY. I’ll also be working with Loren Johnson to make Radiant available as a plugin which will make integrating Radiant with Rails even easier.

I just switched some Ruby on Rails apps I’m running from Pound to Nginx based on the results from some articles I’ve read online. The two biggest advantages of Nginx are 1. It’s raw performance, and 2. It can serve up static files, which is great for running Capistrano’s disable_web command to show a maintenance page when redeploying an application.

At work, we’re working on a standard Ruby on Rails setup and are currently investigating two options. The first is a Mongrel cluster running behind Nginx, as I described above. The second option is fronting the Mongrel cluster with Lighttpd. Since the 1.4.x series of Lighttpd is known to have some issues with it’s mod_proxy implementation, we would use Pen until a stable 1.5 version is released.

I’ll post another article once we have finished our evaluation and chosen which option we’re going to deploy at work. Stay tuned…

I wrote an article awhile back about using the Ruby/LDAP library to handle LDAP authentication in Ruby on Rails. I just finished swapping out the LDAP client library in that application from Ruby/LDAP to ruby-net-ldap. The problems with Ruby/LDAP are that it isn’t a GEM, so installation is a bit more difficult, and it relies on a common LDAP library, like OpenLDAP, to already be installed on the system. The ruby-net-ldap library is written in pure Ruby, so no other library needs to be installed on the system.

Here is the new code that performs the authentication:

require "net/ldap" 

class User < ActiveRecord::Base
  def self.authenticate(login, password, host, port)
    if login.to_s.length > 0 and password.to_s.length > 0
      ldap = Net::LDAP.new
      ldap.host = host
      ldap.port = port
      ldap.auth = "cn=#{login},cn=users,o=xyz...", password
      if ldap.bin
        return find(:first, :conditions => ['username=?', login])
      else
        return false
      end
    end
  end
end

The site I am working on, RealIdaho.com, is hosted by site5. There are some forms that, when filled out by the user, send confirmation emails to the user, to us internally, and to the realtor’s cellphone. We started getting some reports that the confirmation page wouldn’t come up, and users were resubmitting the form many times. I discovered a few things while investigating that may be useful to others.

Yesterday I sold the first new car I ever purchased, a silver 2002 Honda Accord to CarMax. It’s always a little sad to lose something like that, but it isn’t bothering me. I hadn’t even driven it once over the past month. I was just sitting in my parent’s driveway doing its best impression of a bird’s outhouse. It was so unused, that I even had to jumpstart the car twice yesterday! Once at the house to get it going, and then again after I got gas (Maybe I shouldn’t have left such little gas in the tank). I even told the appraiser to keep the car running for fear of it not starting back up!

It had been at my parent’s house because nikki and I live downtown (Columbus, Ohio), and we only have one parking spot, which we use for our other car, a 2003 Honda Civic. It’s not bad only having one parking spot, because we only need one car. And that’s why I could sell the Accord. Now that I’m working from home and have a scooter, we just don’t need a second car.

Downsizing was one of the advantages that we saw when deciding on living downtown. For most of my errands, taking the scooter, bicycle, or even walking suits my needs. It’s nice to not have to deal with the maintenance, gasoline, car insurance, registration, etc. It’s really amazing how much goes into owning a car. Oh yeah… it was also nice to get some money for it!

I’m flying back to Phoenix today, as part of my telecommuting agreement. Due to the fact that I’ll be flying frequently, I feel that I should do something to balance the impact on the environment of my flights. Flying burns a lot of fuel, and flying from Columbus, OH to Phoenix, AZ is certainly no exception. So, today, I’m purchasing a Flight TerraPass to offset the carbon emissions of my flights.

This is what my dog does while I work all day…