Changing the wheels on the bus at 80 mph Andrew Bloomgarden and Julian Giuca RailsConf 2013 Long-running branches are painful, but upgrading to Rails 3 requires one if you can't stop development, right? Wrong! At New Relic, we worked on upgrading to Rails 3 on master while letting development continue in Rails 2. We patched Bundler, built a backwards-compatible boot sequence, and punched ActiveScaffold in the face. Other developers, meanwhile, released 1400 commits worth of work without noticing any changes. We talk about what we did, why we did it, and why we think this approach can help developers get over the hurdle into the Rails 3 promised land.

1. UPGRADINGTO RAILS 3
Changing the wheels on the bus at 80mph
(or 128.784 kph for everyone else)
Andrew Bloomgarden Julian Giuca

2. Andrew introduction
Andrew Bloomgarden
@aughr
2

3. Julian Giuca
@juliangiuca
3

4. Julian Giuca
@juliangiuca
3

5. Julian Giuca
@juliangiuca
3

6. Julian Giuca
@juliangiuca
3

7. 4

8. Overview
5
You can dual boot
Rails 2 and Rails 3

9. Do try this at home!
6

10. Let’s talk about numbers...
7

11. Rails versions in the wild
2.x.x 3.x.x
8
Numberofaccounts

12. 2.2.2# 2.2.3# 2.3.2# 2.3.4# 2.3.5# 2.3.8# 2.3.10# 2.3.11# 2.3.12# 2.3.14# 2.3.15# 3.0.1# 3.0.3# 3.0.5# 3.0.11# 3.0.15# 3.0.18# 3.0.19# 3.1.1# 3.2.1# 3.2.8# 3.2.11# 3.2.13#
Rails versions with >1 server
2.x.x 3.x.x
Numberofaccounts

13. Rails version vs number of server
10

14. Upgrading to Rails 3
1. Procrastinate
2. Goals
3. Feature Branch
4. Fix tests
5. Merge
6. Push
7. Cross fingers
8. Fix Production
11

15. Putting it off
Rearchitecting the app first
Do it all in a big spike!
Have a ruby_19 branch
STUPID THINGS WE TRIED
12

16. Rails 3? Why now?
13

17. Rails 3? Why now?
Someone will make upgrading
easier soon!
13

18. Rails 3? Why now?
Someone will make upgrading
easier soon!
Oh man, this is going to
be awful.
13

19. Gems are leaving us behind
14

20. Gems are leaving us behind
14

21. Come and work on
our Rails 2.3 app!
15

22. 16

23. Upgrading to Rails 3
1. Procrastinate
2. Goals
3. Feature Branch
4. Fix tests
5. Merge
6. Push
7. Cross fingers
8. Fix Production
17

24. A bit of history
18

25. A bit of history
Move to Rails 2.0.1
18

26. A bit of history
18

27. A bit of history
Rails 2.3.14
70,000 application LOC
59,000 test LOC
18

28. Don’t break the world
19

29. Keep everyone happy
not this
thisthis
this
20

30. Keep ourselves happy
CONFLICT (content): …
CONFLICT (content): …
CONFLICT (content): …
⋮
CONFLICT (content): …
Automatic merge failed; fix conflicts
and then commit the result.
21

31. Upgrading to Rails 3
1. Procrastinate
2. Goals
3. Feature Branch
4. Fix tests
5. Merge
6. Push
7. Cross fingers
8. Fix Production
22

32. 23

33. 24

34. I’m looking at you Rails 1.2.2 people
25

35. I’m looking at you Rails 1.2.2 people
25

36. Gemfile
platforms :ruby_18 do
gem 'ruby-debug'
end
platforms :ruby_19 do
gem 'debugger'
end
26

37. 27

38. Monkey patched!
27

39. In the Gemfile
if ENV.include?("USE_RAILS_3")
class Bundler::Dsl
if !self.method_defined?(:to_definition_without_rails3_lockfile)
alias_method :to_definition_without_rails3_lockfile, :to_definition
end
def to_definition(old_lockfile, unlock)
current = File.expand_path(Dir.pwd)
filename = File.join(current, "Gemfile_rails3.lock")
lockfile = Pathname.new(filename)
to_definition_without_rails3_lockfile(lockfile, unlock)
end
end
module Bundler::SharedHelpers
def default_lockfile
current = File.expand_path(Dir.pwd)
filename = File.join(current, "Gemfile_rails3.lock")
lockfile = Pathname.new(filename)
Pathname.new(lockfile)
end
end
end
28

40. Two Gemfile.lock files
29

41. Gemfile
if is_rails3
gem 'rails', '3.0.19'
gem 'lighthouse-api', '~>2.0.0'
gem 'dalli', '~> 2.3.0'
gem 'jquery-rails', '~>2.1'
gem 'active_reload'
else
gem 'rails', '2.3.15'
gem 'lighthouse-api', '~>1.1.0'
gem 'dalli', '~> 1.0.4'
end
30

42. # Booting in Rails 3 mode
$ export USE_RAILS_3=true
$ bundle install
$ rails server
31

43. Upgrading to Rails 3
1. Procrastinate
2. Goals
3. Feature Branch
4. Fix tests
5. Merge
6. Push
7. Cross fingers
8. Fix Production
32

44. Upgrading to Rails 3
1. Procrastinate
2. Goals
3. Feature Branch
4. Fix tests
5. Merge
6. Push
7. Cross fingers
8. Fix Production
Improved!
32

45. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
33

46. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
34

47. Make it easy for everyone
# Rails 2
$ script/server
# Rails 3
$ script/server3
35

48. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
36

49. Onto master
1. Boot Rails 3
2. Dual boot
3. Unbreak Rails 2
4. Merge upstream
37

50. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2

51. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3

52. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3

53. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3

54. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3
Dual boot

55. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3
Dual boot

56. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3
Dual boot

57. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3
Dual boot
Restore Rails 2

58. Unbreak Rails 2
BootsRails2RunsRails2BootsRails3RunsRails3
Rails 2
Boot Rails 3
Dual boot
Restore Rails 2

59. Onto master
1. Boot Rails 3
2. Dual boot
3. Unbreak Rails 2
4. Merge upstream
39

60. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
40

61. Dual-booting that works
Rails 2.3 Rails 3
config/boot.rb requires Rails requires Bundler
config/environment.rb
loads, configures,
and initializes the
app
initializes the app
config/application.rb doesn’t exist
loads and
configures the app
41

62. Dual-booting that works
Rails 2.3 Rails 3
config/boot.rb requires Rails requires Bundler
config/environment.rb
loads, configures,
and initializes the
app
initializes the app
config/application.rb doesn’t exist
loads and
configures the app
41

63. Just make it work
42

64. config/boot.rb
if ENV.include?("USE_RAILS_3")
require 'rubygems'
# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||=
File.expand_path('../../Gemfile', __FILE__)
require 'bundler/setup'
43
else
# 123 lines that load Rails
# All that for this:
Rails.boot!
end

65. config/boot.rb
if ENV.include?("USE_RAILS_3")
require 'rubygems'
# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||=
File.expand_path('../../Gemfile', __FILE__)
require 'bundler/setup'
43
else
# 123 lines that load Rails
# All that for this:
Rails.boot!
end

66. config/boot.rb
if ENV.include?("USE_RAILS_3")
require 'rubygems'
# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||=
File.expand_path('../../Gemfile', __FILE__)
require 'bundler/setup'
43
else
# 123 lines that load Rails
# All that for this:
Rails.boot!
end

67. config/environment.rb
# Load the rails application
require_relative "application"
if CoreAppConfig.is_rails3?
# Initialize the rails application
RpmSite::Application.initialize!
end
44

68. config/application.rb
module NewRelicApplicationConfig
def self.rails3_config(config)
# Do Rails 3 stuff
common_config(config)
end
def self.rails2_config(config)
# Do Rails 2 stuff
common_config(config)
end
def self.common_config(config)
# ...
end
end
45

69. config/application.rb
46
if is_rails3
require 'rails/all'
Bundler.require(:default, Rails.env)
module RpmSite
class Application < Rails::Application
::NewRelicApplicationConfig.rails3_config(config)
end
end
else
Rails::Initializer.run do |config|
::NewRelicApplicationConfig.rails2_config(config)
end
end

70. config/application.rb
46
if is_rails3
require 'rails/all'
Bundler.require(:default, Rails.env)
module RpmSite
class Application < Rails::Application
::NewRelicApplicationConfig.rails3_config(config)
end
end
else
Rails::Initializer.run do |config|
::NewRelicApplicationConfig.rails2_config(config)
end
end

71. config/application.rb
46
if is_rails3
require 'rails/all'
Bundler.require(:default, Rails.env)
module RpmSite
class Application < Rails::Application
::NewRelicApplicationConfig.rails3_config(config)
end
end
else
Rails::Initializer.run do |config|
::NewRelicApplicationConfig.rails2_config(config)
end
end

72. Have tests you trust
47

73. Use CI
48

74. Upgrade dependencies on master
49

75. BackgroundJob to Resque
Bj.submit 'script/background/do_something.rb',
:email => DEV_TEAM
50

76. Never again
def getopts opts = {}
if RUBY_VERSION.to_f >= 1.9
# In 1.9 this throw/catch/getopts nonsense breaks
lambda do |*args|
keys, default, ignored = args
[keys].flatten.each do |key|
[key, key.to_s, key.to_s.intern].each do |key|
return opts[key] if opts.has_key?(key)
end
end
return default
end
else # In 1.8 you can't return across threads
lambda do |*args|
keys, default, ignored = args
catch('opt') do
[keys].flatten.each do |key|
[key, key.to_s, key.to_s.intern].each do |key|
throw 'opt', opts[key] if opts.has_key?(key)
end
end
default
end
end
end
end
51

77. Add an abstraction layer
Bj.submit 'script/background/do_something.rb',
:email => DEV_TEAM
52

78. Add an abstraction layer
Async::Command.new(
'script/background/do_something.rb'
).notify(DEV_TEAM).enqueue
52

79. Use Resque or BackgroundJob
class Async::Command < Async::Job
if CoreAppConfig.use_resque?
include Async::ResqueJob
extend Resque::Plugins::History
else
include Async::BjJob
end
# ...
end
53

80. Finally, use Resque
54

81. Branch by abstraction
55
The problem being with feature
branches is that the current state of
any one of them might be unable to
be deployed for a number of weeks
while the team gets it right.Those
branches just end up running and
running ….
– Paul Hammant

82. Branch by abstraction
55
Branch by abstraction: a pattern
for making large-scale changes to your
application incrementally on mainline.
– Jez Humble

83. Punt when you can56

84. ActiveSupport::Deprecation.silenced = true
57

85. Your old routes still work
RpmSite::Application.routes.draw do |map|
Jammit::Routes.draw(map)
# ...
end
58

86. Your old routes still work
RpmSite::Application.routes.draw do |map|
Jammit::Routes.draw(map)
# ...
end
58

87. Your old mailers still work
class AccountMailer < ApplicationMailer
def new_account(account, user, subscription)
headers['Errors-to'] = BOUNCE_ACCOUNT
from BILLING_ACCOUNT
subject "New Relic invoice account request"
recipients INVOICE_RECIPIENT
@account = account
@user = user
@subscription = subscription
end
end
59

88. Your old mailers still work
class AccountMailer < ApplicationMailer
def new_account(account, user, subscription)
headers['Errors-to'] = BOUNCE_ACCOUNT
from BILLING_ACCOUNT
subject "New Relic invoice account request"
recipients INVOICE_RECIPIENT
@account = account
@user = user
@subscription = subscription
end
end
59

89. Your old models (can) still work
self.store_full_sti_class = false
60

90. Your old views (can) still work
helper PrototypeHelper
clear_helpers
61

91. Your error handling (can) still work
# included into ApplicationController
def rescue_with_handler(e)
if super # ActiveSupport::Rescuable
true
else
rescue_action_in_public e
true
end
end
62

92. Temporarily ugly
63
if CoreAppConfig.is_rails3?
# do something
else
# do something else
end

93. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
64

94. We wantYOU to test!
65
To run in Rails 3, just run
script/server3

95. Extensive manual testing
66
DatabaseData collection
UI
Rails 3 UI

96. Make everyone a tester
67

97. Turn on Rails 3 by default
-is_rails3 = ENV.include?("USE_RAILS_3")
+is_rails3 = !ENV.include?("USE_RAILS_2")
68

98. Incompatible sessions
69
ActionDispatch::Session::SessionRestoreError
(Original exception: uninitialized constant
ActionController::Flash::FlashHash)

99. Incompatible sessions
69

100. Deploy a canary
70

101. Pause the world
71

102. Deploy all the servers
72

103. Oops
73

104. 74

105. And we’re done!
75

106. And we’re done!
75

107. A few very tiny fires
76

108. Breaking the audit trail
ActionController::Base.class_eval do
cache_sweeper :audit_sweeper
end
Audit.add_observer(AuditSweeper.instance)
class AuditSweeper < ActionController::Caching::Sweeper
observe Audit
end
77

109. Rushed Resque rollout
78

110. Punching ActiveScaffold in the face
.../bundle/ruby/1.9.1/gems/
active_scaffold-3.0.26/lib/
active_scaffold/bridges/date_picker/
lib/datepicker_bridge.rb:127:in
`binread':
No such file or directory - .../public/
javascripts/active_scaffold/default/
date_picker_bridge.js (Errno::ENOENT)
79

111. Lost BackgroundJob instrumentation
This page left intentionally blank
80

112. Tagging green build off of Rails 2 tests
81

113. Lots of cleanup
82

114. From an idea to production
1. Procrastinate
2. Goals
3. Onto master
4. Getting it to work
5. Rollout
6. Lessons learned
83

115. Lessons learned
1. Preorder the donuts your on-call engineers like
2. Deprecate ActiveScaffold
3. Schedule lots of time for firefighting and cleanup
84

116. •Get it into master as early as possible
•Incremental everything
•CI is your bestest childhood friend
(and you secretly have a crush on them)
The special magic pony sauce
85

117. newrelic.com/rails3_upgrade@aughr | @juliangiuca
86
We’re done (and we’re hiring)