Practical Chef and Capistrano
For Your Rails Application
Dan Ivovich
SLS Internal Dec 2012
12/17/12

Dan Ivovich
SmartLogic Solutions
http://smartlogicsolutions.com
Twitter - @danivovich
Dan Ivovich on GitHub

What is the goal?
● Build a machine that can run the application
quickly and repeatedly
● Make deploying the app, upgrading
components, adding components, etc seamless
and easy

Who does what?
● Chef
○ User / SSH Keys
○ Web server
○ Database
○ Postfix
○ Redis / Memcached
○ Monit
○ NewRelic Server monitoring
○ /etc/hosts
○ rbenv & Ruby
○ Application binary dependencies (i.e.
Sphinx)

Who does what?
● Capistrano
○ Virtual Hosts
○ Unicorn init.d script
○ Unicorn.rb
○ Monit process monitors
○ Normal Capistrano Stuff

Why both?
● Use each for what it is best at
● Chef is for infrastructure
● Capistrano is for the app
● Could have more than one Capistrano app with
the same Chef config
● Chef config changes infrequently, Capistrano
config could change more frequently

How? - Chef
● Standard Recipes
● Custom Recipes
● Recipes assigned to Roles
● Roles assigned to Nodes
● Nodes with attributes to tailor the install

How? - Capistrano
● Standard Tasks
● Custom Tasks
● Templating files

Chef - Getting started
● Gemfile
● knife kitchen chef-repo
○ Create the folder structure you need
● Solo specific stuff (chef-repo/.chef/knife.rb)
● knife cookbook site install nginx
○ Get the nginx cookbook and anything it
needs

Custom cookbook
● knife cookbook create your_app_custom
● Edit:
chef-repo/cookbooks/your_app_custom/recipes/default.rb

package "logrotate"
rbenv_ruby node['your_app']['ruby_version']
rbenv_gem "bundler" do
ruby_version node['your_app']['ruby_version']
end
template "/etc/hosts" do
source "hosts.erb"
mode "0644"
owner "root"
group "root"
end

directory "#{node['your_app']['cap_base']}" do
action :create
owner 'deploy'
group 'deploy'
mode '0755'
end
directory "#{node['your_app']['deploy_to']}/shared" do
action :create
owner 'deploy'
group 'deploy'
mode '0755'
end
template "#{node['your_app']['deploy_to']}/shared/database.yml"
do
source 'database.yml.erb'
owner 'deploy'
group 'deploy'
mode '0644'
end

Recipe Templates
● chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb
<%= node['your_app']['environment'] %>:
adapter: <%= node['your_app']['adapter'] %>
database: <%= node['your_app']['database'] %>
username: <%= node['your_app']['database_user'] %>
<% if node['your_app']['database_password'] %>
password: <%= node['your_app']['database_password'] %>
<% end %>
host: <%= node['your_app']['database_host'] %>
encoding: utf8
min_messages: warning

Node Attributes
● Application namespace
○ chef-repo/cookbooks/your_app_custom/attributes/default.rb
default['your_app']['cap_base'] = '/home/deploy/apps'
default['your_app']['deploy_to'] = '/home/deploy/apps/your_app'
default['your_app']['environment'] = 'production'
default['your_app']['database'] = 'your_app'
default['your_app']['adapter'] = 'postgresql'
default['your_app']['database_user'] = 'postgres'
default['your_app']['database_password'] =
(node['postgresql']['password']['postgres'] rescue nil)
default['your_app']['database_host'] = 'localhost'
default['your_app']['ruby_version'] = '1.9.2-p320'

Node Attributes
● For your Node configuration
"your_app" : {
"environment" : "production",
"database" : "your_app",
"database_user" : "your_app_db_user",
"database_host" : "db1",
"hosts" : {
"db1" : "nn.nn.nn.nn"
}
},

Define Roles
chef-repo/roles/web_server.rb
name "web_server"
description "web server setup"
run_list [
"recipe[build-essential]", "recipe[annoyances]", "recipe[openssl]",
"recipe[openssh]", "recipe[sudo]", "recipe[postgresql::client]",
"recipe[users_solo::admins]", "recipe[sphinx]", "recipe[imagemagick]",
"recipe[nginx]", "recipe[rbenv]", "recipe[postfix]",
"recipe[monit]", "recipe[your_app_custom]"
]
default_attributes 'build-essential' => { 'compiletime' => true }

Node Configuration
{
"openssh" : { "permit_root_login" : "no", "password_authentication": "no" },
"authorization" : {
"sudo" : { "groups" : [ "admin", "sudo" ], "passwordless" : true }
},
"rbenv" : { "group_users" : [ "deploy" ] },
"sphinx" : { "use_mysql" : false, "use_postgres" : true },
"your_app" : {
"environment" : "production",
"database" : "your_app",
"database_user" : "your_app_db_user",
"database_host" : "db1",
"hosts" : { "db1" : "nn.nn.nn.nn" }
},
"run_list": [
"role[web_server]"
]
}

Not so bad!

Go!
● bundle exec knife bootstrap -x super_user node_name
--template-file=ubuntu-12.04-lts.erb
● bundle exec knife cook super_user@node_name
● Relax!

Capistrano - Getting Started
● Add capistrano and capistrano-ext
● Capify
● deploy.rb

Capistrano - deploy.rb
require 'bundler/capistrano'
require 'capistrano/ext/multistage'
load 'config/recipes/base'
load 'config/recipes/nginx'
load 'config/recipes/unicorn'
load 'config/recipes/monit'
set :default_environment, {
'PATH' => "/opt/rbenv/shims:/opt/rbenv/bin:$PATH",
'RBENV_ROOT' => "/opt/rbenv"
}
set :bundle_flags, "--deployment --quiet --binstubs --shebang ruby-local-exec"
set :use_sudo, false
set :application, 'your_app'
set :repository, 'git@github.com:you/your_app.git'
set :deploy_to, '/home/deploy/apps/your_app'
set :deploy_via, :remote_cache

Capistrano - deploy.rb
set :branch, 'master'
set :scm, :git
set :target_os, :ubuntu
set :maintenance_template_path, File.expand_path("..
/recipes/templates/maintenance.html.erb", __FILE__)
default_run_options[:pty] = true
ssh_options[:forward_agent] = true
namespace :custom do
desc 'Create the .rbenv-version file'
task :rbenv_version, :roles => :app do
run "cd #{release_path} && rbenv local 1.9.2-p320"
end
end
before 'bundle:install', 'custom:rbenv_version'

Capistrano - recipes/base.rb
def template(from, to)
erb = File.read(File.expand_path("../templates/#{from}", __FILE__))
put ERB.new(erb).result(binding), to
end
def set_default(name, *args, &block)
set(name, *args, &block) unless exists?(name)
end

Capistrano - recipes/monit.rb
set_default(:alert_email, "dan@smartlogicsolutions.com")
namespace :monit do
desc "Setup all Monit configuration"
task :setup do
unicorn
syntax
restart
end
after "deploy:setup", "monit:setup"
task(:unicorn, roles: :app) { monit_config "unicorn" }
%w[start stop restart syntax].each do |command|
desc "Run Monit #{command} script"
task command do
with_user "deploy" do
sudo "service monit #{command}"
end
end
end
end

Capistrano - recipes/monit.rb
def monit_config(name, destination = nil)
destination ||= "/etc/monit/conf.d/#{name}.conf"
template "monit/#{name}.erb", "/tmp/monit_#{name}"
with_user "deploy" do
sudo "mv /tmp/monit_#{name} #{destination}"
sudo "chown root #{destination}"
sudo "chmod 600 #{destination}"
end
end

Capistrano - recipes/nginx.rb
namespace :nginx do
desc "Setup nginx configuration for this application"
task :setup, roles: :web do
template "nginx_unicorn.erb", "/tmp/nginx_conf"
sudo "mv /tmp/nginx_conf /etc/nginx/sites-enabled/#{application}"
sudo "rm -f /etc/nginx/sites-enabled/default"
restart
end
after "deploy:setup", "nginx:setup"
%w[start stop restart].each do |command|
desc "#{command} nginx"
task command, roles: :web do
sudo "service nginx #{command}"
end
end
end

Capistrano - templates/nginx_unicorn.erb
upstream unicorn {
server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;
}
server {
listen 80 default deferred;
server_name your_app_domain.com;
root <%= current_path %>/public;
if (-f $document_root/system/maintenance.html) {
return 503;
}
error_page 503 @maintenance;
location @maintenance {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri @unicorn;
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 /500.html;
error_page 504 /504.html;
client_max_body_size 4G;
keepalive_timeout 10;
server_tokens off;
}

Capistrano - recipes/unicorn.rb
set_default(:unicorn_user) { user }
set_default(:unicorn_pid) { "#{current_path}/tmp/pids/unicorn.pid" }
set_default(:unicorn_config) { "#{shared_path}/config/unicorn.rb" }
set_default(:unicorn_log) { "#{shared_path}/log/unicorn.log" }
set_default(:unicorn_workers) {
if rails_env == "production"
10
else
3
end
}
set_default(:unicorn_timeout, 30)

Capistrano - recipes/unicorn.rb
namespace :unicorn do
desc "Setup Unicorn initializer and app configuration"
task :setup, roles: :app do
run "mkdir -p #{shared_path}/config"
template "unicorn.rb.erb", unicorn_config
template "unicorn_init.erb", "/tmp/unicorn_init"
run "chmod +x /tmp/unicorn_init"
sudo "mv /tmp/unicorn_init /etc/init.d/unicorn_#{application}"
sudo "update-rc.d -f unicorn_#{application} defaults"
end
after "deploy:setup", "unicorn:setup"
%w[start stop restart].each do |command|
desc "#{command} unicorn"
task command, roles: :app do
sudo "service unicorn_#{application} #{command}"
end
after "deploy:#{command}", "unicorn:#{command}"
end
end

Capistrano - templates/unicorn.rb.erb
root = "<%= current_path %>"
working_directory root
pid "#{root}/tmp/pids/unicorn.pid"
stderr_path "#{root}/log/unicorn.log"
stdout_path "#{root}/log/unicorn.log"
listen "/tmp/unicorn.<%= application %>.sock"
worker_processes <%= unicorn_workers %>
timeout <%= unicorn_timeout %>
preload_app true
before_exec { |server| ENV['BUNDLE_GEMFILE'] = "#{root}/Gemfile" }

Capistrano - templates/unicorn.rb.erb
before_fork do |server, worker|
# Disconnect since the database connection will not carry over
if defined? ActiveRecord::Base
ActiveRecord::Base.connection.disconnect!
end
# Quit the old unicorn process
old_pid = "#{server.config[:pid]}.oldbin"
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
end

Capistrano - templates/unicorn.rb.erb
after_fork do |server, worker|
# Start up the database connection again in the worker
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
end
child_pid = server.config[:pid].sub(".pid", ".#{worker.nr}.pid")
system("echo #{Process.pid} > #{child_pid}")
end

Capistrano - t/monit/unicorn.erb
check process <%= application %>_unicorn with pidfile <%= unicorn_pid %>
start program = "/etc/init.d/unicorn_<%= application %> start"
stop program = "/etc/init.d/unicorn_<%= application %> stop"
<% unicorn_workers.times do |n| %>
<% pid = unicorn_pid.sub(".pid", ".#{n}.pid") %>
check process <%= application %>_unicorn_worker_<%= n %> with pidfile <%= pid %>
start program = "/bin/true"
stop program = "/usr/bin/test -s <%= pid %> && /bin/kill -QUIT `cat <%= pid %>`"
if mem > 200.0 MB for 1 cycles then restart
if cpu > 50% for 3 cycles then restart
if 5 restarts within 5 cycles then timeout
alert <%= alert_email %> only on { pid }
if changed pid 2 times within 60 cycles then alert
<% end %>

Whoa!

But really, it is just a
bunch of Erb for files you
already have

Did you see the trick?
● after "deploy:setup", "nginx:setup"
So we can...
● cap staging deploy:setup deploy:migrations

From the top!

Ready?!? Here we go!
1. New VM at my_web_app in your .ssh/config
2. Create chef-repo/nodes/my_web_app.json
3. In chef-repo:
bundle exec knife bootstrap node_name
--template-file=ubuntu-12.04-lts.erb
4. bundle exec knife cook root@my_web_app
5. In app directory:
create/edit config/deploy/staging.rb
6. cap staging deploy:setup deploy:migrations
7. Hit the bars

Thoughts....
● Vagrant and VMs are you friend. Rinse and repeat
● It is ok to tweak your Chef stuff and re-cook, but I always
like to restart with a fresh VM once I think I'm done
● Capistrano tweaks should be easy to apply, especially with
tasks like nginx:setup, unicorn:setup etc.
● Chef issues are harder to debug and more frustrating than
Capistrano issues, another reason to put more app specific
custom stuff in Capistrano and do standard things in Chef

Questions?
http://smartlogicsolutions.com
http://twitter.com/smartlogic
http://github.com/smartlogic
http://fb.me/smartlogic