1.
Practical Chef and Capistrano For Your Rails ApplicationDan IvovichSLS Internal Dec 201212/17/12

2.
Dan Ivovich SmartLogic Solutionshttp://smartlogicsolutions.comTwitter - @danivovichDan Ivovich on GitHub

3.
What is the goal?● Build a machine that can run the application quickly and repeatedly● Make deploying the app, upgrading components, adding components, etc seamless and easy

4.
Who does what?● Chef ○ User / SSH Keys ○ Web server ○ Database ○ Postfix ○ Redis / Memcached ○ Monit ○ NewRelic Server monitoring ○ /etc/hosts ○ rbenv & Ruby ○ Application binary dependencies (i.e. Sphinx)

5.
Who does what?● Capistrano ○ Virtual Hosts ○ Unicorn init.d script ○ Unicorn.rb ○ Monit process monitors ○ Normal Capistrano Stuff

6.
Why both?● Use each for what it is best at● Chef is for infrastructure● Capistrano is for the app● Could have more than one Capistrano app with the same Chef config● Chef config changes infrequently, Capistrano config could change more frequently

7.
How? - Chef● Standard Recipes● Custom Recipes● Recipes assigned to Roles● Roles assigned to Nodes● Nodes with attributes to tailor the install

8.
How? - Capistrano● Standard Tasks● Custom Tasks● Templating files

9.
Chef - Getting started● Gemfile● knife kitchen chef-repo ○ Create the folder structure you need● Solo specific stuff (chef-repo/.chef/knife.rb)● knife cookbook site install nginx ○ Get the nginx cookbook and anything it needs

10.
Custom cookbook● knife cookbook create your_app_custom● Edit: chef-repo/cookbooks/your_app_custom/recipes/default.rb

11.
package "logrotate"rbenv_ruby node[your_app][ruby_version]rbenv_gem "bundler" do ruby_version node[your_app][ruby_version]endtemplate "/etc/hosts" do source "hosts.erb" mode "0644" owner "root" group "root"end

12.
directory "#{node[your_app][cap_base]}" do action :create owner deploy group deploy mode 0755enddirectory "#{node[your_app][deploy_to]}/shared" do action :create owner deploy group deploy mode 0755endtemplate "#{node[your_app][deploy_to]}/shared/database.yml"do source database.yml.erb owner deploy group deploy mode 0644end

13.
Recipe Templates● chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb<%= node[your_app][environment] %>: adapter: <%= node[your_app][adapter] %> database: <%= node[your_app][database] %> username: <%= node[your_app][database_user] %><% if node[your_app][database_password] %> password: <%= node[your_app][database_password] %><% end %> host: <%= node[your_app][database_host] %> encoding: utf8 min_messages: warning

14.
Node Attributes● Application namespace ○ chef-repo/cookbooks/your_app_custom/attributes/default.rbdefault[your_app][cap_base] = /home/deploy/appsdefault[your_app][deploy_to] = /home/deploy/apps/your_appdefault[your_app][environment] = productiondefault[your_app][database] = your_appdefault[your_app][adapter] = postgresqldefault[your_app][database_user] = postgresdefault[your_app][database_password] = (node[postgresql][password][postgres] rescue nil)default[your_app][database_host] = localhostdefault[your_app][ruby_version] = 1.9.2-p320

15.
Node Attributes● For your Node configuration "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } },

16.
Define Roles chef-repo/roles/web_server.rbname "web_server"description "web server setup"run_list [ "recipe[build-essential]", "recipe[annoyances]", "recipe[openssl]", "recipe[openssh]", "recipe[sudo]", "recipe[postgresql::client]", "recipe[users_solo::admins]", "recipe[sphinx]", "recipe[imagemagick]", "recipe[nginx]", "recipe[rbenv]", "recipe[postfix]", "recipe[monit]", "recipe[your_app_custom]"]default_attributes build-essential => { compiletime => true }

17.
Node Configuration{ "openssh" : { "permit_root_login" : "no", "password_authentication": "no" }, "authorization" : { "sudo" : { "groups" : [ "admin", "sudo" ], "passwordless" : true } }, "rbenv" : { "group_users" : [ "deploy" ] }, "sphinx" : { "use_mysql" : false, "use_postgres" : true }, "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } }, "run_list": [ "role[web_server]" ]}

18.
Not so bad!

19.
Go!● bundle exec knife bootstrap -x super_user node_name --template-file=ubuntu-12.04-lts.erb● bundle exec knife cook super_user@node_name● Relax!

20.
Capistrano - Getting Started● Add capistrano and capistrano-ext● Capify● deploy.rb

21.
Capistrano - deploy.rbrequire bundler/capistranorequire capistrano/ext/multistageload config/recipes/baseload config/recipes/nginxload config/recipes/unicornload config/recipes/monitset :default_environment, { PATH => "/opt/rbenv/shims:/opt/rbenv/bin:$PATH", RBENV_ROOT => "/opt/rbenv"}set :bundle_flags, "--deployment --quiet --binstubs --shebang ruby-local-exec"set :use_sudo, falseset :application, your_appset :repository, git@github.com:you/your_app.gitset :deploy_to, /home/deploy/apps/your_appset :deploy_via, :remote_cache

22.
Capistrano - deploy.rbset :branch, masterset :scm, :gitset :target_os, :ubuntuset :maintenance_template_path, File.expand_path("../recipes/templates/maintenance.html.erb", __FILE__)default_run_options[:pty] = truessh_options[:forward_agent] = truenamespace :custom do desc Create the .rbenv-version file task :rbenv_version, :roles => :app do run "cd #{release_path} && rbenv local 1.9.2-p320" endendbefore bundle:install, custom:rbenv_version

23.
Capistrano - recipes/base.rbdef template(from, to) erb = File.read(File.expand_path("../templates/#{from}", __FILE__)) put ERB.new(erb).result(binding), toenddef set_default(name, *args, &block) set(name, *args, &block) unless exists?(name)end

24.
Capistrano - recipes/monit.rbset_default(:alert_email, "dan@smartlogicsolutions.com")namespace :monit do desc "Setup all Monit configuration" task :setup do unicorn syntax restart end after "deploy:setup", "monit:setup" task(:unicorn, roles: :app) { monit_config "unicorn" } %w[start stop restart syntax].each do |command| desc "Run Monit #{command} script" task command do with_user "deploy" do sudo "service monit #{command}" end end endend

25.
Capistrano - recipes/monit.rbdef monit_config(name, destination = nil) destination ||= "/etc/monit/conf.d/#{name}.conf" template "monit/#{name}.erb", "/tmp/monit_#{name}" with_user "deploy" do sudo "mv /tmp/monit_#{name} #{destination}" sudo "chown root #{destination}" sudo "chmod 600 #{destination}" endend

26.
Capistrano - recipes/nginx.rbnamespace :nginx do desc "Setup nginx configuration for this application" task :setup, roles: :web do template "nginx_unicorn.erb", "/tmp/nginx_conf" sudo "mv /tmp/nginx_conf /etc/nginx/sites-enabled/#{application}" sudo "rm -f /etc/nginx/sites-enabled/default" restart end after "deploy:setup", "nginx:setup" %w[start stop restart].each do |command| desc "#{command} nginx" task command, roles: :web do sudo "service nginx #{command}" end endend

27.
Capistrano - templates/nginx_unicorn.erbupstream unicorn { server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;}server { listen 80 default deferred; server_name your_app_domain.com; root <%= current_path %>/public; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } location ^~ /assets/ { gzip_static on; expires max; add_header Cache-Control public; } try_files $uri/index.html $uri @unicorn; location @unicorn { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; proxy_pass http://unicorn; } error_page 500 502 /500.html; error_page 504 /504.html; client_max_body_size 4G; keepalive_timeout 10; server_tokens off;}

28.
Capistrano - recipes/unicorn.rbset_default(:unicorn_user) { user }set_default(:unicorn_pid) { "#{current_path}/tmp/pids/unicorn.pid" }set_default(:unicorn_config) { "#{shared_path}/config/unicorn.rb" }set_default(:unicorn_log) { "#{shared_path}/log/unicorn.log" }set_default(:unicorn_workers) { if rails_env == "production" 10 else 3 end}set_default(:unicorn_timeout, 30)

29.
Capistrano - recipes/unicorn.rbnamespace :unicorn do desc "Setup Unicorn initializer and app configuration" task :setup, roles: :app do run "mkdir -p #{shared_path}/config" template "unicorn.rb.erb", unicorn_config template "unicorn_init.erb", "/tmp/unicorn_init" run "chmod +x /tmp/unicorn_init" sudo "mv /tmp/unicorn_init /etc/init.d/unicorn_#{application}" sudo "update-rc.d -f unicorn_#{application} defaults" end after "deploy:setup", "unicorn:setup" %w[start stop restart].each do |command| desc "#{command} unicorn" task command, roles: :app do sudo "service unicorn_#{application} #{command}" end after "deploy:#{command}", "unicorn:#{command}" endend

30.
Capistrano - templates/unicorn.rb.erbroot = "<%= current_path %>"working_directory rootpid "#{root}/tmp/pids/unicorn.pid"stderr_path "#{root}/log/unicorn.log"stdout_path "#{root}/log/unicorn.log"listen "/tmp/unicorn.<%= application %>.sock"worker_processes <%= unicorn_workers %>timeout <%= unicorn_timeout %>preload_app truebefore_exec { |server| ENV[BUNDLE_GEMFILE] = "#{root}/Gemfile" }

31.
Capistrano - templates/unicorn.rb.erbbefore_fork do |server, worker| # Disconnect since the database connection will not carry over if defined? ActiveRecord::Base ActiveRecord::Base.connection.disconnect! end # Quit the old unicorn process old_pid = "#{server.config[:pid]}.oldbin" if File.exists?(old_pid) && server.pid != old_pid begin Process.kill("QUIT", File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH # someone else did our job for us end endend

32.
Capistrano - templates/unicorn.rb.erbafter_fork do |server, worker| # Start up the database connection again in the worker if defined?(ActiveRecord::Base) ActiveRecord::Base.establish_connection end child_pid = server.config[:pid].sub(".pid", ".#{worker.nr}.pid") system("echo #{Process.pid} > #{child_pid}")end

33.
Capistrano - t/monit/unicorn.erbcheck process <%= application %>_unicorn with pidfile <%= unicorn_pid %> start program = "/etc/init.d/unicorn_<%= application %> start" stop program = "/etc/init.d/unicorn_<%= application %> stop"<% unicorn_workers.times do |n| %> <% pid = unicorn_pid.sub(".pid", ".#{n}.pid") %> check process <%= application %>_unicorn_worker_<%= n %> with pidfile <%= pid %> start program = "/bin/true" stop program = "/usr/bin/test -s <%= pid %> && /bin/kill -QUIT `cat <%= pid %>`" if mem > 200.0 MB for 1 cycles then restart if cpu > 50% for 3 cycles then restart if 5 restarts within 5 cycles then timeout alert <%= alert_email %> only on { pid } if changed pid 2 times within 60 cycles then alert<% end %>

34.
Whoa!

35.
But really, it is just abunch of Erb for files you already have

36.
Did you see the trick?● after "deploy:setup", "nginx:setup" So we can...● cap staging deploy:setup deploy:migrations

37.
From the top!

38.
Ready?!? Here we go!1. New VM at my_web_app in your .ssh/config2. Create chef-repo/nodes/my_web_app.json3. In chef-repo: bundle exec knife bootstrap node_name --template-file=ubuntu-12.04-lts.erb4. bundle exec knife cook root@my_web_app5. In app directory: create/edit config/deploy/staging.rb6. cap staging deploy:setup deploy:migrations7. Hit the bars

39.
Thoughts....● Vagrant and VMs are you friend. Rinse and repeat● It is ok to tweak your Chef stuff and re-cook, but I always like to restart with a fresh VM once I think Im done● Capistrano tweaks should be easy to apply, especially with tasks like nginx:setup, unicorn:setup etc.● Chef issues are harder to debug and more frustrating than Capistrano issues, another reason to put more app specific custom stuff in Capistrano and do standard things in Chef

40.
Questions?http://smartlogicsolutions.comhttp://twitter.com/smartlogichttp://github.com/smartlogic http://fb.me/smartlogic