How We Upgraded A Very Large App from Rails 3 to Rails 4

For small projects, upgrading Rails versions is simple and can be done in a matter of hours or days. On the other hand, upgrading large projects can be quite a headache and take weeks or months. There are plenty of blog posts and upgrade guides out there explaining the mechanics of upgrading from one Rails version to another, but few of them provide the planning, organization, and implementation practices needed to upgrade a large project.

Invoca upgraded our multi-million line Rails project from version 2 to 3 several years ago. At that time we created a branch from our mainline, upgraded to version 3, and started fixing bugs while the rest of the team continued to develop. The bug fixing took months, meanwhile the mainline diverged and created a continual stream of merge conflicts to resolve.

For our recent upgrade from 3 to 4, we wanted a strategy that kept the upgraded code regularly merged back into the mainline and kept running our test suite for both Rails versions. Throughout this post we’ll share some of the techniques we used to streamline our Rails version upgrade.

Keeping it all together

Our main focus when upgrading from Rails 3 to Rails 4 was ensuring that we kept functionality equivalent between versions and minimized the differences. To do this we were able to simultaneously run both versions together in the same repo. Here are a few of the practices we used to combine Rails versions and keep our conflicts at bay.

Working with different Rails versions simultaneously

Bundler Gemfile Option

  • In order to deal with Rails 4 gem dependency issues, we created two separate gemfiles, “Gemfile” and “Gemfile_rails4”. We left “Gemfile” as-is and we upgraded the Rails version and the many dependencies in the “Gemfile_rails4”.
  • We then passed an option to Bundler telling it which environment to use. This allowed us to have both versions of Rails within the same repo.
  • To specify which gemfile to use, we would prepend the BUNDLE_GEMFILE option to `bundle exec`
BUNDLE_GEMFILE=Gemfile_rails4 bundle exec <COMMAND>
  • The same option is used to install the upgraded gems from “Gemfile_rails4”
BUNDLE_GEMFILE=Gemfile_rails4 bundle install
  • If the BUNDLE_GEMFILE option is not set, it will default to “Gemfile”, allowing the rest of the developers to continue working with Rails 3 without needing to change their workflow.
  • To make working with the Bundler option easier, we implemented a Bash script to automatically prepend the option for us. (Those that were consistently working on resolving bugs also implemented a simpler “r4” Bash alias on their local machine)
    • Instead of needing to use:
BUNDLE_GEMFILE=Gemfile_rails4 bundle exec rails server
  • We instead used:
script/r4 rails server

script/r4

#!/bin/bash
# To make your life easier, add this alias to your .bashrc or .bash_profile
# alias r4="BUNDLE_GEMFILE=Gemfile_rails4 bundle exec"

COMMAND="BUNDLE_GEMFILE=`pwd`/Gemfile_rails4 bundle exec $*"
echo $COMMAND
eval $COMMAND

Static Helper Method

  • When implementing code changes for Rails 4, we first went through and implemented all the changes that were backwards compatible. Then, to simplify differences in code specifically for Rails 4, we implemented our own helper method. This method takes two lambdas and executes the first in Rails 4 and the second in Rails 3.
    • Class StaticHelpers
       def self.rails_4_or_3(rails_4_lambda, rails_3_lambda = -> {})
         if Rails::VERSION::MAJOR == 4
           rails_4_lambda.call
         elsif Rails::VERSION::MAJOR == 3
           rails_3_lambda.call
         else
           raise "Rails Version #{Rails::VERSION::MAJOR} not supported."
         end
       end
      end
  • This became our defacto method to check the Rails version and is utilized to execute different sets of code, define modules for specific Rails versions, return specific values, etc.
    • StaticHelpers.rails_4_or_3(
       -> { 
         # Snippet to execute for Rails 4 only
        },
       -> {
         # Snippet for Rails 3 only
       }
      )
      
      StaticHelpers.rails_4_or_3(-> { include Rails4OnlyModule })

 

Automated Testing

  • For every branch, we configured our automated unit testing suite to produce two test builds, one for each Rails version. This allowed us to quickly troubleshoot both versions in parallel to know whether a fix made for Rails 4 created any adverse effects to Rails 3.
  • The results of the first test run where disheartening, over 15,000 test failures! But, we took a methodical approach and started knocking them down. Many, of course, were common failures. In the beginning it was not uncommon for a single change to a test helper to fix hundreds if not thousands of tests. However, toward the end, the fixes started coming slower and resolved less failures. Make sure you dedicated adequate resources and time: we had a team of four working on the project for five months.

Deploy It: One Piece at a Time

  • When our total test count reached a reasonable size, we began smoke testing by running our automated QA test suite against servers running in Rails 4 mode.
    • Along the way we were able to identify and resolve important feature errors that weren’t caught by our automated unit test suite.
  • Once we resolved our errors and failures, we began to switch server groups one at a time. We began with low priority servers such as background job processing servers, and gradually increased priority until we finally switched over our front end, customer-facing servers.
  • By switching dedicated groups singularly, we could focus our attention on the expected behavior of that group and could react quickly if unexpected errors occurred. It also gave us the option to quickly revert the system for that group back to its original working version to give us time to debug.

Good luck!

Hopefully these tips are as beneficial for you as they were for us throughout our upgrade. There are many other important facets to a Rails upgrade that must be determined by how the app works and will need their own specific implementations, and so for that reason every Rails upgrade is its own special snowflake. You’ll most likely end up chasing a plethora of test cases and wonder to yourself how it was possible for your app to become a perpetual Rube Goldberg machine. Despite the trouble that comes with it, the rewarding feeling of upgrading your system will give you plenty of motivation to work through whatever problems that arise. Good luck!

– Omeed Rabani

7 thoughts on “How We Upgraded A Very Large App from Rails 3 to Rails 4

  1. Brian

    Holy smokes, a team of four working on the project for five months? I am considering embarking on upgrading a very large Rails project from 3.0 to a higher version. I’ve upgraded smaller Rails apps a number of times and knew this upgrade would be bigger, but this stat is pretty shocking and eye opening.

    Reply
    1. Omeed Rabani Post author

      That line might be a little misleading. For a good amount of time in the beginning I was working on the upgrade on my own with guidance from a senior software dev. Once we were able to parallelize the debugging process we began to add other developers and they would devote portions of their time to the upgrade along with the other tasks they were working on.

      Reply
  2. Chris Kottom

    Thanks for sharing your experiences! I can relate to a lot of what I read here, but I’m having a difficult time understanding the decision to support both the old and new versions of Rails in the same branch. It seems like a lot of hassle to avoid having to deal with merges which, in my experience, are going to be a part of any actively developed project. Could you go into some more detail about the motivations behind that decision and whether it turned out to be worthwhile for you in the end – e.g. calendar time and effort to upgrade, etc?

    Reply
    1. Omeed Rabani Post author

      Our decision to support both the old and new version of Rails in the same branch was done for a few reasons
      1. It allowed us to keep our merge conflicts small in size, and this in turn made code review much easier.
      2. It allowed other developers to continue working with the old version while the team working on the upgrade squashed bugs with the new version.
      3. It allowed us to keep track of whether a developer introduced a feature or piece of code that was compatible with the old version, but not compatible with the new, so we could go and apply a fix that was compatible for both.

      Points 1 and 2 were most important for us because
      1 – a giant version upgrade merge conflict would not have been able to be reviewed effectively
      2 – developers working on other tasks would not have to port their work to two separate branches, and instead would only work in the main branch and run tests for both versions

      In regards to calendar time, I can only assume that the incremental reviews prevented many potential future bugs and as a result decreased the amount of time necessary for the project. In regards to upgrade effort, the small amount overhead in setting up both versions in the same branch and testing both versions for each build was well worth the parallelization we were able to achieve with other developers’ work.

      Reply
    2. Colin Kelley

      Hi Chris Kottom: when we migrated from Rails 2 to Rails 3 we followed the fork-then-merge strategy you mention. That’s how we knew we needed a new approach! 😉

      Here was the quandary we found ourselves in for that previous migration, from Rails 2 to 3:

      At the time we had a team of about 12 developers total. We couldn’t stop all customer feature development while migrating Rails, and that wouldn’t have made sense anyway since the Rails migration project couldn’t keep more than a couple developers busy at the start. So we assigned a developer and an intern and they got started. Every month or so they merged in the master branch. That was a giant hassle and it introduced many merge bugs because the master code branch was moving so fast. We tried merging more frequently but that quickly hit diminishing returns where the migration developers were spending nearly all their time merging–often the same pattern repeated–and not making enough progress on discovering new patterns. (Ultimately we went the other way and had them merge less frequently in order to make big leaps in the migration.) The last straw was that we found the mainline developers were introducing more Rails 2-incompatible code all the time! It started feeling like Groundhog Day.

      The Rails 3 – 4 approach that Omeed describes was much more successful. By using a common branch, we were constantly converging the entire team around best practices to work in Rails 4. We reviewed and approved Pull Requests from Rails 4 on a weekly cadence, so we didn’t have a giant review at the end, and so we could widely share new learning about best practices. We monitored the failure count of automated builds for Rails 4 so we got immediate feedback if anyone on the mainline development team just committed code that caused a decline in Rails 4 readiness.

      As Omeed says, the Rails 3 – 4 approach was an improvement in both calendar time and total effort. It made it seamless for us to add in additional developers on the migration, when they had a free hour or day, since everyone could see the Rails 4 updates in the master branch. (These Rails 4 updates were about a 50/50 split: half were best practices to write our code so it worked in both versions of Rails, and half of were conditional using the `rails_4_or_3` helper.)

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


× three = 24