Practical Chef and Capistrano              For Your Rails ApplicationDan IvovichSLS Internal Dec 201212/17/12
Dan Ivovich          SmartLogic Solutionshttp://smartlogicsolutions.comTwitter - @danivovichDan Ivovich on GitHub
What is the goal?● Build a machine that can run the application  quickly and repeatedly● Make deploying the app, upgrading...
Who does what?● Chef  ○ User / SSH Keys  ○ Web server  ○ Database  ○ Postfix  ○ Redis / Memcached  ○ Monit  ○ NewRelic Ser...
Who does what?● Capistrano  ○ Virtual Hosts  ○ Unicorn init.d script  ○ Unicorn.rb  ○ Monit process monitors  ○ Normal Cap...
Why both?● Use each for what it is best at● Chef is for infrastructure● Capistrano is for the app● Could have more than on...
How? - Chef●   Standard Recipes●   Custom Recipes●   Recipes assigned to Roles●   Roles assigned to Nodes●   Nodes with at...
How? - Capistrano● Standard Tasks● Custom Tasks● Templating files
Chef - Getting started● Gemfile● knife kitchen chef-repo  ○ Create the folder structure you need● Solo specific stuff (che...
Custom cookbook● knife cookbook create your_app_custom● Edit:     chef-repo/cookbooks/your_app_custom/recipes/default.rb
package "logrotate"rbenv_ruby node[your_app][ruby_version]rbenv_gem "bundler" do ruby_version node[your_app][ruby_version]...
directory "#{node[your_app][cap_base]}" do action :create owner deploy group deploy mode 0755enddirectory "#{node[your_app...
Recipe Templates●   chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb<%= node[your_app][environment] ...
Node Attributes● Application namespace  ○ chef-repo/cookbooks/your_app_custom/attributes/default.rbdefault[your_app][cap_b...
Node Attributes● For your Node configuration  "your_app" : {    "environment" : "production",    "database" : "your_app", ...
Define Roles  chef-repo/roles/web_server.rbname "web_server"description "web server setup"run_list [  "recipe[build-essent...
Node Configuration{    "openssh" : { "permit_root_login" : "no", "password_authentication": "no" },    "authorization" : {...
Not so bad!
Go!●   bundle exec knife bootstrap -x super_user node_name         --template-file=ubuntu-12.04-lts.erb●   bundle exec kni...
Capistrano - Getting Started● Add capistrano and capistrano-ext● Capify● deploy.rb
Capistrano - deploy.rbrequire bundler/capistranorequire capistrano/ext/multistageload config/recipes/baseload config/recip...
Capistrano - deploy.rbset :branch, masterset :scm, :gitset :target_os, :ubuntuset :maintenance_template_path, File.expand_...
Capistrano - recipes/base.rbdef template(from, to) erb = File.read(File.expand_path("../templates/#{from}", __FILE__)) put...
Capistrano - recipes/monit.rbset_default(:alert_email, "dan@smartlogicsolutions.com")namespace :monit do desc "Setup all M...
Capistrano - recipes/monit.rbdef monit_config(name, destination = nil) destination ||= "/etc/monit/conf.d/#{name}.conf" te...
Capistrano - recipes/nginx.rbnamespace :nginx do desc "Setup nginx configuration for this application" task :setup, roles:...
Capistrano - templates/nginx_unicorn.erbupstream unicorn {  server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=...
Capistrano - recipes/unicorn.rbset_default(:unicorn_user) { user }set_default(:unicorn_pid) { "#{current_path}/tmp/pids/un...
Capistrano - recipes/unicorn.rbnamespace :unicorn do desc "Setup Unicorn initializer and app configuration" task :setup, r...
Capistrano - templates/unicorn.rb.erbroot = "<%= current_path %>"working_directory rootpid "#{root}/tmp/pids/unicorn.pid"s...
Capistrano - templates/unicorn.rb.erbbefore_fork do |server, worker| # Disconnect since the database connection will not c...
Capistrano - templates/unicorn.rb.erbafter_fork do |server, worker| # Start up the database connection again in the worker...
Capistrano - t/monit/unicorn.erbcheck process <%= application %>_unicorn with pidfile <%= unicorn_pid %> start program = "...
Whoa!
But really, it is just abunch of Erb for files you      already have
Did you see the trick?● after "deploy:setup", "nginx:setup"                  So we can...● cap staging deploy:setup deploy...
From the top!
Ready?!? Here we go!1. New VM at my_web_app in your .ssh/config2. Create chef-repo/nodes/my_web_app.json3. In chef-repo:  ...
Thoughts....● Vagrant and VMs are you friend. Rinse and repeat● It is ok to tweak your Chef stuff and re-cook, but I alway...
Questions?http://smartlogicsolutions.comhttp://twitter.com/smartlogichttp://github.com/smartlogic http://fb.me/smartlogic
Upcoming SlideShare
Loading in...5
×

Practical Chef and Capistrano for Your Rails App

10,800

Published on

Dan Ivovich walks through how to use Chef and Capistrano for your Rails application.

Published in: Technology

Practical Chef and Capistrano for Your Rails App

  1. 1. Practical Chef and Capistrano For Your Rails ApplicationDan IvovichSLS Internal Dec 201212/17/12
  2. 2. Dan Ivovich SmartLogic Solutionshttp://smartlogicsolutions.comTwitter - @danivovichDan Ivovich on GitHub
  3. 3. What is the goal?● Build a machine that can run the application quickly and repeatedly● Make deploying the app, upgrading components, adding components, etc seamless and easy
  4. 4. Who does what?● Chef ○ User / SSH Keys ○ Web server ○ Database ○ Postfix ○ Redis / Memcached ○ Monit ○ NewRelic Server monitoring ○ /etc/hosts ○ rbenv & Ruby ○ Application binary dependencies (i.e. Sphinx)
  5. 5. Who does what?● Capistrano ○ Virtual Hosts ○ Unicorn init.d script ○ Unicorn.rb ○ Monit process monitors ○ Normal Capistrano Stuff
  6. 6. Why both?● Use each for what it is best at● Chef is for infrastructure● Capistrano is for the app● Could have more than one Capistrano app with the same Chef config● Chef config changes infrequently, Capistrano config could change more frequently
  7. 7. How? - Chef● Standard Recipes● Custom Recipes● Recipes assigned to Roles● Roles assigned to Nodes● Nodes with attributes to tailor the install
  8. 8. How? - Capistrano● Standard Tasks● Custom Tasks● Templating files
  9. 9. Chef - Getting started● Gemfile● knife kitchen chef-repo ○ Create the folder structure you need● Solo specific stuff (chef-repo/.chef/knife.rb)● knife cookbook site install nginx ○ Get the nginx cookbook and anything it needs
  10. 10. Custom cookbook● knife cookbook create your_app_custom● Edit: chef-repo/cookbooks/your_app_custom/recipes/default.rb
  11. 11. package "logrotate"rbenv_ruby node[your_app][ruby_version]rbenv_gem "bundler" do ruby_version node[your_app][ruby_version]endtemplate "/etc/hosts" do source "hosts.erb" mode "0644" owner "root" group "root"end
  12. 12. directory "#{node[your_app][cap_base]}" do action :create owner deploy group deploy mode 0755enddirectory "#{node[your_app][deploy_to]}/shared" do action :create owner deploy group deploy mode 0755endtemplate "#{node[your_app][deploy_to]}/shared/database.yml"do source database.yml.erb owner deploy group deploy mode 0644end
  13. 13. Recipe Templates● chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb<%= node[your_app][environment] %>: adapter: <%= node[your_app][adapter] %> database: <%= node[your_app][database] %> username: <%= node[your_app][database_user] %><% if node[your_app][database_password] %> password: <%= node[your_app][database_password] %><% end %> host: <%= node[your_app][database_host] %> encoding: utf8 min_messages: warning
  14. 14. Node Attributes● Application namespace ○ chef-repo/cookbooks/your_app_custom/attributes/default.rbdefault[your_app][cap_base] = /home/deploy/appsdefault[your_app][deploy_to] = /home/deploy/apps/your_appdefault[your_app][environment] = productiondefault[your_app][database] = your_appdefault[your_app][adapter] = postgresqldefault[your_app][database_user] = postgresdefault[your_app][database_password] = (node[postgresql][password][postgres] rescue nil)default[your_app][database_host] = localhostdefault[your_app][ruby_version] = 1.9.2-p320
  15. 15. Node Attributes● For your Node configuration "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } },
  16. 16. Define Roles chef-repo/roles/web_server.rbname "web_server"description "web server setup"run_list [ "recipe[build-essential]", "recipe[annoyances]", "recipe[openssl]", "recipe[openssh]", "recipe[sudo]", "recipe[postgresql::client]", "recipe[users_solo::admins]", "recipe[sphinx]", "recipe[imagemagick]", "recipe[nginx]", "recipe[rbenv]", "recipe[postfix]", "recipe[monit]", "recipe[your_app_custom]"]default_attributes build-essential => { compiletime => true }
  17. 17. Node Configuration{ "openssh" : { "permit_root_login" : "no", "password_authentication": "no" }, "authorization" : { "sudo" : { "groups" : [ "admin", "sudo" ], "passwordless" : true } }, "rbenv" : { "group_users" : [ "deploy" ] }, "sphinx" : { "use_mysql" : false, "use_postgres" : true }, "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } }, "run_list": [ "role[web_server]" ]}
  18. 18. Not so bad!
  19. 19. Go!● bundle exec knife bootstrap -x super_user node_name --template-file=ubuntu-12.04-lts.erb● bundle exec knife cook super_user@node_name● Relax!
  20. 20. Capistrano - Getting Started● Add capistrano and capistrano-ext● Capify● deploy.rb
  21. 21. Capistrano - deploy.rbrequire bundler/capistranorequire capistrano/ext/multistageload config/recipes/baseload config/recipes/nginxload config/recipes/unicornload config/recipes/monitset :default_environment, { PATH => "/opt/rbenv/shims:/opt/rbenv/bin:$PATH", RBENV_ROOT => "/opt/rbenv"}set :bundle_flags, "--deployment --quiet --binstubs --shebang ruby-local-exec"set :use_sudo, falseset :application, your_appset :repository, git@github.com:you/your_app.gitset :deploy_to, /home/deploy/apps/your_appset :deploy_via, :remote_cache
  22. 22. Capistrano - deploy.rbset :branch, masterset :scm, :gitset :target_os, :ubuntuset :maintenance_template_path, File.expand_path("../recipes/templates/maintenance.html.erb", __FILE__)default_run_options[:pty] = truessh_options[:forward_agent] = truenamespace :custom do desc Create the .rbenv-version file task :rbenv_version, :roles => :app do run "cd #{release_path} && rbenv local 1.9.2-p320" endendbefore bundle:install, custom:rbenv_version
  23. 23. Capistrano - recipes/base.rbdef template(from, to) erb = File.read(File.expand_path("../templates/#{from}", __FILE__)) put ERB.new(erb).result(binding), toenddef set_default(name, *args, &block) set(name, *args, &block) unless exists?(name)end
  24. 24. Capistrano - recipes/monit.rbset_default(:alert_email, "dan@smartlogicsolutions.com")namespace :monit do desc "Setup all Monit configuration" task :setup do unicorn syntax restart end after "deploy:setup", "monit:setup" task(:unicorn, roles: :app) { monit_config "unicorn" } %w[start stop restart syntax].each do |command| desc "Run Monit #{command} script" task command do with_user "deploy" do sudo "service monit #{command}" end end endend
  25. 25. Capistrano - recipes/monit.rbdef monit_config(name, destination = nil) destination ||= "/etc/monit/conf.d/#{name}.conf" template "monit/#{name}.erb", "/tmp/monit_#{name}" with_user "deploy" do sudo "mv /tmp/monit_#{name} #{destination}" sudo "chown root #{destination}" sudo "chmod 600 #{destination}" endend
  26. 26. Capistrano - recipes/nginx.rbnamespace :nginx do desc "Setup nginx configuration for this application" task :setup, roles: :web do template "nginx_unicorn.erb", "/tmp/nginx_conf" sudo "mv /tmp/nginx_conf /etc/nginx/sites-enabled/#{application}" sudo "rm -f /etc/nginx/sites-enabled/default" restart end after "deploy:setup", "nginx:setup" %w[start stop restart].each do |command| desc "#{command} nginx" task command, roles: :web do sudo "service nginx #{command}" end endend
  27. 27. Capistrano - templates/nginx_unicorn.erbupstream unicorn { server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;}server { listen 80 default deferred; server_name your_app_domain.com; root <%= current_path %>/public; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } location ^~ /assets/ { gzip_static on; expires max; add_header Cache-Control public; } try_files $uri/index.html $uri @unicorn; location @unicorn { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; proxy_pass http://unicorn; } error_page 500 502 /500.html; error_page 504 /504.html; client_max_body_size 4G; keepalive_timeout 10; server_tokens off;}
  28. 28. Capistrano - recipes/unicorn.rbset_default(:unicorn_user) { user }set_default(:unicorn_pid) { "#{current_path}/tmp/pids/unicorn.pid" }set_default(:unicorn_config) { "#{shared_path}/config/unicorn.rb" }set_default(:unicorn_log) { "#{shared_path}/log/unicorn.log" }set_default(:unicorn_workers) { if rails_env == "production" 10 else 3 end}set_default(:unicorn_timeout, 30)
  29. 29. Capistrano - recipes/unicorn.rbnamespace :unicorn do desc "Setup Unicorn initializer and app configuration" task :setup, roles: :app do run "mkdir -p #{shared_path}/config" template "unicorn.rb.erb", unicorn_config template "unicorn_init.erb", "/tmp/unicorn_init" run "chmod +x /tmp/unicorn_init" sudo "mv /tmp/unicorn_init /etc/init.d/unicorn_#{application}" sudo "update-rc.d -f unicorn_#{application} defaults" end after "deploy:setup", "unicorn:setup" %w[start stop restart].each do |command| desc "#{command} unicorn" task command, roles: :app do sudo "service unicorn_#{application} #{command}" end after "deploy:#{command}", "unicorn:#{command}" endend
  30. 30. Capistrano - templates/unicorn.rb.erbroot = "<%= current_path %>"working_directory rootpid "#{root}/tmp/pids/unicorn.pid"stderr_path "#{root}/log/unicorn.log"stdout_path "#{root}/log/unicorn.log"listen "/tmp/unicorn.<%= application %>.sock"worker_processes <%= unicorn_workers %>timeout <%= unicorn_timeout %>preload_app truebefore_exec { |server| ENV[BUNDLE_GEMFILE] = "#{root}/Gemfile" }
  31. 31. Capistrano - templates/unicorn.rb.erbbefore_fork do |server, worker| # Disconnect since the database connection will not carry over if defined? ActiveRecord::Base ActiveRecord::Base.connection.disconnect! end # Quit the old unicorn process old_pid = "#{server.config[:pid]}.oldbin" if File.exists?(old_pid) && server.pid != old_pid begin Process.kill("QUIT", File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH # someone else did our job for us end endend
  32. 32. Capistrano - templates/unicorn.rb.erbafter_fork do |server, worker| # Start up the database connection again in the worker if defined?(ActiveRecord::Base) ActiveRecord::Base.establish_connection end child_pid = server.config[:pid].sub(".pid", ".#{worker.nr}.pid") system("echo #{Process.pid} > #{child_pid}")end
  33. 33. Capistrano - t/monit/unicorn.erbcheck process <%= application %>_unicorn with pidfile <%= unicorn_pid %> start program = "/etc/init.d/unicorn_<%= application %> start" stop program = "/etc/init.d/unicorn_<%= application %> stop"<% unicorn_workers.times do |n| %> <% pid = unicorn_pid.sub(".pid", ".#{n}.pid") %> check process <%= application %>_unicorn_worker_<%= n %> with pidfile <%= pid %> start program = "/bin/true" stop program = "/usr/bin/test -s <%= pid %> && /bin/kill -QUIT `cat <%= pid %>`" if mem > 200.0 MB for 1 cycles then restart if cpu > 50% for 3 cycles then restart if 5 restarts within 5 cycles then timeout alert <%= alert_email %> only on { pid } if changed pid 2 times within 60 cycles then alert<% end %>
  34. 34. Whoa!
  35. 35. But really, it is just abunch of Erb for files you already have
  36. 36. Did you see the trick?● after "deploy:setup", "nginx:setup" So we can...● cap staging deploy:setup deploy:migrations
  37. 37. From the top!
  38. 38. Ready?!? Here we go!1. New VM at my_web_app in your .ssh/config2. Create chef-repo/nodes/my_web_app.json3. In chef-repo: bundle exec knife bootstrap node_name --template-file=ubuntu-12.04-lts.erb4. bundle exec knife cook root@my_web_app5. In app directory: create/edit config/deploy/staging.rb6. cap staging deploy:setup deploy:migrations7. Hit the bars
  39. 39. Thoughts....● Vagrant and VMs are you friend. Rinse and repeat● It is ok to tweak your Chef stuff and re-cook, but I always like to restart with a fresh VM once I think Im done● Capistrano tweaks should be easy to apply, especially with tasks like nginx:setup, unicorn:setup etc.● Chef issues are harder to debug and more frustrating than Capistrano issues, another reason to put more app specific custom stuff in Capistrano and do standard things in Chef
  40. 40. Questions?http://smartlogicsolutions.comhttp://twitter.com/smartlogichttp://github.com/smartlogic http://fb.me/smartlogic
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×