Building an Internal Gem Infrastructure: A Crash Course

Rails, Ruby

I gave this short talk as part of RailsConf 2014. LivingSocial graciously sponsored my trip. We’re hiring awesome engineers if you didn’t already know. Hit me up on twitter @ubermajestix if you’re interested in working with a great team.



Being able to build, release, and host your own gems is really helpful. This post will introduce the basics of doing all three of these things. This is more of a motivation post as there are tutorials out there with more depth and step by step instruction than I will provide here.


Internal Gem Infrastructure.002

Moving code into gems really helps when you’re splitting up monolithic apps or building a service oriented architecture. Sharing code with gems can reduce copy/paste which reduces the mayhem maintenance costs it imposes.

Internal Gem Infrastructure.003

Building a gem infrastructure can help foster an “internal open source” culture where anyone can contribute. Gems get their own
repo and readme. You get to reap all the benefits of pull requests and open contribution within your company.

Internal Gem Infrastructure.004

We’ve heard experts say “go ahead, make a mess” but it’s often hard to figure out where to put your mess in a Rails project. A gem is a great place to make a mess of small classes with single responsibilities. Putting your code into a gem forces you to draw boundaries around your code. You get your own tests! You can separate concerns.

Internal Gem Infrastructure.005

There are a bunch of benefits to isolating your code. First, your code is really easy to delete when it’s no longer needed. You don’t even have to delete your gem, you can just remove the calls to your gem from other gems or apps that use it; its documentation and tests live on. It might even be resurrected in the future. This is so much easier than searching through old commits for “that code that did that one thing”.

Also, you have to think about how your code will interact with Rails. Does it need database access? Where should logs go? Do you even need Rails for this code to do its job? This has helped me constrain my code to its simplest form, use the smallest possible pieces of Rails, mock and stub boundaries in tests, and write good docs.

An Example

I recently was tasked with automating the generation of a daily report which needed to be uploaded to a third party’s FTP. The processing results are then emailed back to us as a plain text file. Sometimes I need to parse the email if anything went wrong. As a coworker said, 1997 called, it wants its architecture back. But those are the constraints we have to work in.

I could have plunked down the CSV and FTP code right in the database model, maybe thrown the email parsing code in a helper,  and called it a day. We all know that’s gross. We’ve all done it and justified it, “Gets the job done right? Fat models bro!”


But you’ve also been on the other side a week later, a month later, or even years later, “Ah man, why is all this code here?” and “Do we even use this anymore?”, “What does this even do?”


So I decided to cut a gem, and put all this domain specific code in one place. It’s a pretty simple gem, and that’s ok. It’s easy to describe and document, but it also let us have the conversation of where the code should be run from, instead of just throwing it in the monolith and moving on.


Building A Gem

Ok this is a crash course inside the crash course on bulding your own gem. This may be review for some of you or it may be new to you. If you haven’t written a gem, I highly recommend trying as it’s a great addition to your Ruby tool belt.

There are a bunch of ways to build a gem, but the easiest one I’ve used is Bundler’s gem command. The best part is it’s probably already installed on your machine. Just run:

bundle gem ls-super_awesomeness

Here’s what it creates for you:


  • The most important file is the gemspec there at the bottom. Without that file this isn’t a gem. You can readup on the details online here and here, but the version Bundler generates is pretty straight forward.
  • A Gemfile
  • A License (MIT)
  • A Rakefile which will help you build, install, and release
  • Your code goes in lib
  • If your namespacing you’ll have a namespace directory with your files in it
  • Finally your version file deep down in there.

Namespacing is something I think is really important and bundler will help you do this. Here’s how it works. You just shorten your organization’s name into two letters. It can be more or less, but I’ve always seen two characters in the wild. It’s important because it tells your reader that this is internal code and not an open source implementation.


Ok, so now a little bit about versioning, again this might be a refresher, but I feel like I explain this more than I have to. We liberally use the semantic versioning spec to control version numbers at LivingSocial. Checkout for the full spec.


The bugfix number is the number furthest to the left, it should be incremented when bugs are fixed or really small changes happen that don’t affect your public API. The minor version number is the middle number and should be incremented when you’ve added a new feature or additional functionality but your haven’t removed or broken the public API. Consumers of your code will not be negatively affected by upgrading. The major version number should be incremented when a breaking change is made. This usually means you’ve removed some functionality, extracted it to another gem, or have changed the public API. You should have a very good reason for breaking the public API as all consumers are affected and will have to upgrade their code. Think about how hard it is to upgrade from Rails 2 to 3 and from 3 to 4.

If a gem is used in production, is stable and reliable, then it should be at 1.0.0. I’m wary of deploying gems that aren’t 1.0.0, if you’re in production and the API is stable and your version is 0.0.1237, you’re doing it wrong.


Letting The World Behold Your Awesomeness

Ok so you’ve built your gem and got your version number figured out. Now, we need to let the world behold your awesomeness, or at least others in your company.

It should be EASY for anyone to release their code. Having a bunch of commands to release code is messy and error prone, even just three commands.If you use bundler to create your gem, `rake release` will do all this for you but we’ll need to modify its behavior to so you…


But be careful! You can accidentally open source your code. It happens! You might have sensitive information in your gem and there are bots that mirror gems as they hit, so once that code is out there, it’s not coming back.

It should also be easy to add a version tag in git. This helps people figure out what commits went into a release and Github makes version tags really easy to use. Bundler’s `rake release` task will use the version number from your VERSION constant in your version.rb file, so make sure you update your version number before you release.

So how do we get `rake release` to not open source your code?

We monkeypatch subclass bundler’s gem_tasks. If you look in the Rakefile bundler generated for you, you’ll see require ‘bundler/gem_tasks’. This is how you get the build, install, and release rake tasks. At LivingSocial is we built our own gem that all other gems use to get their default rake tasks. If you subclass bundler you can add your own rake tasks and most importantly keep developers from pushing their code to the wrong place and help them push their code to the right place.

Here’s the magic code. Notice how we disable the rubygems push and raise an error just in case. Then, we add a method that pushes to our geminabox server. From here we just replace the bundler/gem_tasks with your customized gem_tasks in your Rakefiles. Boom, releasing and tagging made easy.



Hosting Your Gems

Now you’ll need a place to store all these gems. Here are the options that I would recommend you use.


We use geminabox at LivingSocial. It gets the job done. It hosts our gems and provides a web interface to manage and view gems. It will support authentication through custom middleware (their wiki page has examples). It can be setup to do pull through mirroring of rubygems. Mirroring is useful when you don’t want to depend on when you deploy and you don’t want to vendor all your gems. It also provides a CLI by patching your system gem command with inabox. You can use that command to push gems and configure your machine to talk to your geminabox instance.

I’ve also used stickler at a previous company. It supports hosting and has a nice web interface. It supports auth in a similar way to geminabox. It will mirror gems from but you have to tell it which gems to mirror, but it does support mirroring everything in your Gemfile.lock. It has a really nice CLI and the author wanted me to mention that it will soon support the new bundler API. There’s a great blog post about setting it up over at copiousfreetime’s site.

Both stickler and geminabox are pretty easy to deploy and have lots of documentation online to help you out. In my experience stickler is faster and more reliable than geminabox, but your mileage may vary.

If you want to go the hosted route, appears to be the only solution out there. It supports auth, but doesn’t appear to support mirroring. It comes with a CLI and also supports nodejs and python packages in addition to ruby gems.

There you have it, a crash course with almost everything you need to setup a gem infrastructure for your company.

Rescuing Exceptions… err… I mean StandardError in Ruby

Linux, Ruby

I’ve heard from people how or how not to rescue things that might break in ruby.  And I was confused.
Your basic rescue frame of mind is something like this:

15306 2014 GitHub
Uploaded with plasq‘s Skitch!

This won’t give you a very good description of what you rescued so you can rescue specific errors and exceptions like this:

15307 2014 GitHub
Uploaded with plasq‘s Skitch!

But! I want to catch EVERY POSSIBLE ERROR I MIGHT RUN INTO! So why not just rescue the whole Exception class? Yeah that sounds like a great idea.

15307 2014 GitHub
Uploaded with plasq‘s Skitch!

Ok so that catches anything we might run into BUT NO!!! Its not a great idea. Let’s illustrate:

Uploaded with plasq‘s Skitch!

Now let’s say we have a script that is taking FOREVER to do anything like the one above. Let’s send it a kill signal via ctl+c or you could call kill with the process id from the command line. So ready set… kill! Die die die!

Um wait nothing happend…

What did the logger say?

Uh wait what!?!?!

Terminal 2014 bash 2014 142�0
Uploaded with plasq‘s Skitch!

Ah so rescuing Exception rescues Interrupt (ctl+c) and SignalException (kill).  In this case I had to call kill -9 to get the damn thing to die. The kill -9 command is kind of a silver bullet but you can end up with a mess so don’t use it when you don’t have to, like my associate who crashed a whole cluster of servers that ran a large rental video chain’s backend because kill -9 on that weird flavor of linux did a lot more than just kill the one process, it killed everything that didn’t belong to his user (i.e. goodbye oracle db processes)!

What we should do is rescue StandardError unless you really want to rescue things like ScriptErrors (when you’re eval-ing ruby code) or rescue SignalExceptions (like do something before the thing dies).

This should help:

The Ruby Programming Language
Uploaded with plasq‘s Skitch! now has RESTful search!

Rails, Ruby now has RESTful search.  Here’s how it works:

You can now just fill in the url to search for properties. Every option available in the search form has been put into a RESTful url that are built in the following format:

You can add “none” for min_price, max_price, or bedrooms, to say you don’t have a preference.

You can search for pets by adding /cats or /dogs, or /cats/dogs for both.

Some examples

/search/1850/none/3br/dogs will search for properties with a monthly rent of at least $1850, with no maximum price, that has three bedrooms and allows dogs.

/search/none/2650/none will search for properties with no minimum price but a monthly rent capped at $2650 and with no preference on bedrooms.

/search/1850/2650/4br/dogs/cats will search for properties between $1850 to $2650 per month that is four bedrooms and allows cats and/or dogs.

Currently you will have to pick a city to search in before these url’s will work.

I am working on a RESTful url search that will let you target specific markets.  Using the same schema from above I would only have to add the city’s name to the url.

/search/boulder/1850/none/3br would search for houses in Boulder with at least a monthly rent of $1850 that has three bedrooms.

This feature will shortly be extended into a “link to these results” generator, making sharing of results much easier, ala “link to this map” in google maps. more things coming soon…

Linux, Rails, Ruby

So I did some analysis on my results.  Looks like I get on average about 84% of postings to successfully geocode and get posted to rentmappr from craigslist.  I do have an address correction algorithm in place but I haven’t analyzed its results for success.  From watching the geocoder do its work though, I estimate I get about 20% of the postings that have a “bad” or non-geocodeable address to successfully geocode.

I will shortly be adding bedroom search to further narrow results.  I did push dog and cat search options to production last week.

I’m also pleased to announce that my roommate successfully found, and rented, an apartment using rentmappr!  Her glowing testimonial to follow.

Memcache Sessions in Rails

Linux, Rails, Ruby


Does rails scale? No! But memcached does!

Ok ok, you aren’t probably going to need memcache for caching huge queries yet.  But it is useful for storing your user’s session data server side and not having to worry about filling up cookies or clearing out files in your tmp directory or your database.

So I’ve been designing a single sign on system for the interanl applications at CI.  And I need apps on all kinds of different servers to talk to the same memcache instance(s) for their sessions.  I did follow this article on err.the_blog to get started but I only got so far and ran into some issues that got hard to debug.

The default memcached session store hits localhost:11211 which is generally fine for most applications but I needed a clustered approach and could not for the life of me figure it out.  Turns out like most things with rails it was really easy.

1. Memcached installed on your machine or whatever machine you want to use for sessions
1a. Top Funky has a really simple shell script that will work on os x and another that works on ubuntu
2. You will need the memcache-client gem, you know what to do.

In environment.rb in my rails app I needed to do a few things:

1. Setup the connection to the memcache server

2. Tell rails to use memcache for sessions

3. Setup rails to drop all its sessions stuff into the memcache server we setup

Alright so 1:
require 'memcache'
CACHE = => "your_app")
CACHE.servers = 'some_ip_address:some_port', 'another_ip_address_if_you_need_it:some_port'

and then 2(pretty easy)
config.action_controller.session_store = :mem_cache_store

and then 3 (this is where you can set the session timeout and then pass in the memcached object CACHE)

config.action_controller.session = {
:session_key => '_your_app_session',
:secret      => 'someotherkindofsecretthatnooneknows',
:cache       => CACHE,
:expires=>900  }

I think if memcache doesn’t find the session on one server it’ll look on the others you put in the CACHE.servers list.  However if your memcache instance goes down your rails app is hosed and starts throwing up 500 errors all over itself.

You can actually throw alot of this config into a gem if you are going to have several apps that are all going to use the same config (setup the CACHE stuff in your gem and then just do step 3 in your apps)  That way if you need to switch or add servers to your memcache setup you don’t have to make changes in 20 places.

Gmail with Rails…just so I don’t forget how to do it.

Rails, Ruby

Smtp settings in config/environment.rb:

ActionMailer::Base.smtp_settings = {
:address => “”,
:port => 587,
:domain => “”,
:authentication => :plain,
:user_name => “”,
:password => “*******”

Get code from this very helpful post.  Make the folders and files

  • action_mailer_tls
    • lib
      • smtp_tls.rb
    • init.rb

and copy/paste his code into the correct files you just created

LightMate Gedit Theme and now for TextMate too!

Design, Linux, Rails, Ruby

Ok kids, finally got the themes available up on my website:

Check the screenshot below for the gedit theme or hit up my website for screenshots of Textmate.

I really hate working in dark environments…well lights off yes, but a bright computing environment is a must. Thus when customizing gedit to work like TextMate for rails work I really hated the DarkMate theme – and the default lightish theme I didn’t like either.

So I made my own based on some colors from DarkMate. Take a look:

  • rhtml:

rhtml file

  • ruby:

ruby file

UPDATE: Theme is for GTKsourceview-2.0 and later (this is installed with Gutsy Gibbon). This theme will not work with Feisty or earlier releases.

download the theme here!

Upgraded to Ubuntu Gutsy Gibbon

Linux, Rails, Ruby

I went through with it on my dev machine at work b/c I heard there were lots of great new things in gedit and openoffice might stop crashing (gotta love the java apps).

I updated all the components update manager told me to BEFORE I ran the upgrade. I heard there were problems if all Feisty updates were not applied, so I updated. That took about 45 minutes.

Then I ran the upgrade package – I had to delete some custom software repositories in System=>Admin=>SoftwareSources because they were no longer live and the upgrade was trying to load the Gutsy source from them…I just switched the Download From: to Main Server and unchecked the repositories giving me issues under Third Party.

After that the upgrade went smooth…I was still trying to work while doing the upgrade so it took about 2 hours to complete. I lost one text document I tried to save when the installer was doing its thang (it locked the local filesystem) but thats the only other trouble i ran into.

Rebooted and it all looked fine except for gedit. This needed to be re-customized for rails since they change a whole bunch of crap in gedit. I followed the instructions here:

And whala everything in gedit is working…i did sudo apt-get install gedit-plugins

to get some extra fun stuff like a color picker, bracket matcher…I think I like Gemini better b/c I can select text and then wrap it with quotes and brackets…but it does completion of “<” those guys which I hate because I use “<<” all the time to add an element to an array so Gemini makes it into “<><“…anyway, the default color scheme is a little different…actually I don’t like it but I also don’t like Darkmate style which everyone raves about…so I have to find one that works.

I also installed IE6 under Wine…I followed the instructions here:

And this dude has some interesting/useful plugins for gedit also: