Advertisement
Advertisement

More Related Content

Advertisement
Advertisement

Practical Chef and Capistrano for Your Rails App

  1. Practical Chef and Capistrano For Your Rails Application Dan Ivovich SLS Internal Dec 2012 12/17/12
  2. Dan Ivovich SmartLogic Solutions http://smartlogicsolutions.com Twitter - @danivovich Dan Ivovich on GitHub
  3. What is the goal? ● Build a machine that can run the application quickly and repeatedly ● Make deploying the app, upgrading components, adding components, etc seamless and easy
  4. Who does what? ● Chef ○ User / SSH Keys ○ Web server ○ Database ○ Postfix ○ Redis / Memcached ○ Monit ○ NewRelic Server monitoring ○ /etc/hosts ○ rbenv & Ruby ○ Application binary dependencies (i.e. Sphinx)
  5. Who does what? ● Capistrano ○ Virtual Hosts ○ Unicorn init.d script ○ Unicorn.rb ○ Monit process monitors ○ Normal Capistrano Stuff
  6. Why both? ● Use each for what it is best at ● Chef is for infrastructure ● Capistrano is for the app ● Could have more than one Capistrano app with the same Chef config ● Chef config changes infrequently, Capistrano config could change more frequently
  7. How? - Chef ● Standard Recipes ● Custom Recipes ● Recipes assigned to Roles ● Roles assigned to Nodes ● Nodes with attributes to tailor the install
  8. How? - Capistrano ● Standard Tasks ● Custom Tasks ● Templating files
  9. Chef - Getting started ● Gemfile ● knife kitchen chef-repo ○ Create the folder structure you need ● Solo specific stuff (chef-repo/.chef/knife.rb) ● knife cookbook site install nginx ○ Get the nginx cookbook and anything it needs
  10. Custom cookbook ● knife cookbook create your_app_custom ● Edit: chef-repo/cookbooks/your_app_custom/recipes/default.rb
  11. package "logrotate" rbenv_ruby node['your_app']['ruby_version'] rbenv_gem "bundler" do ruby_version node['your_app']['ruby_version'] end template "/etc/hosts" do source "hosts.erb" mode "0644" owner "root" group "root" end
  12. directory "#{node['your_app']['cap_base']}" do action :create owner 'deploy' group 'deploy' mode '0755' end directory "#{node['your_app']['deploy_to']}/shared" do action :create owner 'deploy' group 'deploy' mode '0755' end template "#{node['your_app']['deploy_to']}/shared/database.yml" do source 'database.yml.erb' owner 'deploy' group 'deploy' mode '0644' end
  13. Recipe Templates ● chef-repo/cookbooks/your_app_custom/templates/default/database.yml.erb <%= node['your_app']['environment'] %>: adapter: <%= node['your_app']['adapter'] %> database: <%= node['your_app']['database'] %> username: <%= node['your_app']['database_user'] %> <% if node['your_app']['database_password'] %> password: <%= node['your_app']['database_password'] %> <% end %> host: <%= node['your_app']['database_host'] %> encoding: utf8 min_messages: warning
  14. Node Attributes ● Application namespace ○ chef-repo/cookbooks/your_app_custom/attributes/default.rb default['your_app']['cap_base'] = '/home/deploy/apps' default['your_app']['deploy_to'] = '/home/deploy/apps/your_app' default['your_app']['environment'] = 'production' default['your_app']['database'] = 'your_app' default['your_app']['adapter'] = 'postgresql' default['your_app']['database_user'] = 'postgres' default['your_app']['database_password'] = (node['postgresql']['password']['postgres'] rescue nil) default['your_app']['database_host'] = 'localhost' default['your_app']['ruby_version'] = '1.9.2-p320'
  15. Node Attributes ● For your Node configuration "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } },
  16. Define Roles chef-repo/roles/web_server.rb name "web_server" description "web server setup" run_list [ "recipe[build-essential]", "recipe[annoyances]", "recipe[openssl]", "recipe[openssh]", "recipe[sudo]", "recipe[postgresql::client]", "recipe[users_solo::admins]", "recipe[sphinx]", "recipe[imagemagick]", "recipe[nginx]", "recipe[rbenv]", "recipe[postfix]", "recipe[monit]", "recipe[your_app_custom]" ] default_attributes 'build-essential' => { 'compiletime' => true }
  17. Node Configuration { "openssh" : { "permit_root_login" : "no", "password_authentication": "no" }, "authorization" : { "sudo" : { "groups" : [ "admin", "sudo" ], "passwordless" : true } }, "rbenv" : { "group_users" : [ "deploy" ] }, "sphinx" : { "use_mysql" : false, "use_postgres" : true }, "your_app" : { "environment" : "production", "database" : "your_app", "database_user" : "your_app_db_user", "database_host" : "db1", "hosts" : { "db1" : "nn.nn.nn.nn" } }, "run_list": [ "role[web_server]" ] }
  18. Not so bad!
  19. Go! ● bundle exec knife bootstrap -x super_user node_name --template-file=ubuntu-12.04-lts.erb ● bundle exec knife cook super_user@node_name ● Relax!
  20. Capistrano - Getting Started ● Add capistrano and capistrano-ext ● Capify ● deploy.rb
  21. Capistrano - deploy.rb require 'bundler/capistrano' require 'capistrano/ext/multistage' load 'config/recipes/base' load 'config/recipes/nginx' load 'config/recipes/unicorn' load 'config/recipes/monit' set :default_environment, { 'PATH' => "/opt/rbenv/shims:/opt/rbenv/bin:$PATH", 'RBENV_ROOT' => "/opt/rbenv" } set :bundle_flags, "--deployment --quiet --binstubs --shebang ruby-local-exec" set :use_sudo, false set :application, 'your_app' set :repository, 'git@github.com:you/your_app.git' set :deploy_to, '/home/deploy/apps/your_app' set :deploy_via, :remote_cache
  22. Capistrano - deploy.rb set :branch, 'master' set :scm, :git set :target_os, :ubuntu set :maintenance_template_path, File.expand_path(".. /recipes/templates/maintenance.html.erb", __FILE__) default_run_options[:pty] = true ssh_options[:forward_agent] = true namespace :custom do desc 'Create the .rbenv-version file' task :rbenv_version, :roles => :app do run "cd #{release_path} && rbenv local 1.9.2-p320" end end before 'bundle:install', 'custom:rbenv_version'
  23. Capistrano - recipes/base.rb def template(from, to) erb = File.read(File.expand_path("../templates/#{from}", __FILE__)) put ERB.new(erb).result(binding), to end def set_default(name, *args, &block) set(name, *args, &block) unless exists?(name) end
  24. Capistrano - recipes/monit.rb set_default(:alert_email, "dan@smartlogicsolutions.com") namespace :monit do desc "Setup all Monit configuration" task :setup do unicorn syntax restart end after "deploy:setup", "monit:setup" task(:unicorn, roles: :app) { monit_config "unicorn" } %w[start stop restart syntax].each do |command| desc "Run Monit #{command} script" task command do with_user "deploy" do sudo "service monit #{command}" end end end end
  25. Capistrano - recipes/monit.rb def monit_config(name, destination = nil) destination ||= "/etc/monit/conf.d/#{name}.conf" template "monit/#{name}.erb", "/tmp/monit_#{name}" with_user "deploy" do sudo "mv /tmp/monit_#{name} #{destination}" sudo "chown root #{destination}" sudo "chmod 600 #{destination}" end end
  26. Capistrano - recipes/nginx.rb namespace :nginx do desc "Setup nginx configuration for this application" task :setup, roles: :web do template "nginx_unicorn.erb", "/tmp/nginx_conf" sudo "mv /tmp/nginx_conf /etc/nginx/sites-enabled/#{application}" sudo "rm -f /etc/nginx/sites-enabled/default" restart end after "deploy:setup", "nginx:setup" %w[start stop restart].each do |command| desc "#{command} nginx" task command, roles: :web do sudo "service nginx #{command}" end end end
  27. Capistrano - templates/nginx_unicorn.erb upstream unicorn { server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0; } server { listen 80 default deferred; server_name your_app_domain.com; root <%= current_path %>/public; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } location ^~ /assets/ { gzip_static on; expires max; add_header Cache-Control public; } try_files $uri/index.html $uri @unicorn; location @unicorn { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; proxy_pass http://unicorn; } error_page 500 502 /500.html; error_page 504 /504.html; client_max_body_size 4G; keepalive_timeout 10; server_tokens off; }
  28. Capistrano - recipes/unicorn.rb set_default(:unicorn_user) { user } set_default(:unicorn_pid) { "#{current_path}/tmp/pids/unicorn.pid" } set_default(:unicorn_config) { "#{shared_path}/config/unicorn.rb" } set_default(:unicorn_log) { "#{shared_path}/log/unicorn.log" } set_default(:unicorn_workers) { if rails_env == "production" 10 else 3 end } set_default(:unicorn_timeout, 30)
  29. Capistrano - recipes/unicorn.rb namespace :unicorn do desc "Setup Unicorn initializer and app configuration" task :setup, roles: :app do run "mkdir -p #{shared_path}/config" template "unicorn.rb.erb", unicorn_config template "unicorn_init.erb", "/tmp/unicorn_init" run "chmod +x /tmp/unicorn_init" sudo "mv /tmp/unicorn_init /etc/init.d/unicorn_#{application}" sudo "update-rc.d -f unicorn_#{application} defaults" end after "deploy:setup", "unicorn:setup" %w[start stop restart].each do |command| desc "#{command} unicorn" task command, roles: :app do sudo "service unicorn_#{application} #{command}" end after "deploy:#{command}", "unicorn:#{command}" end end
  30. Capistrano - templates/unicorn.rb.erb root = "<%= current_path %>" working_directory root pid "#{root}/tmp/pids/unicorn.pid" stderr_path "#{root}/log/unicorn.log" stdout_path "#{root}/log/unicorn.log" listen "/tmp/unicorn.<%= application %>.sock" worker_processes <%= unicorn_workers %> timeout <%= unicorn_timeout %> preload_app true before_exec { |server| ENV['BUNDLE_GEMFILE'] = "#{root}/Gemfile" }
  31. Capistrano - templates/unicorn.rb.erb before_fork do |server, worker| # Disconnect since the database connection will not carry over if defined? ActiveRecord::Base ActiveRecord::Base.connection.disconnect! end # Quit the old unicorn process old_pid = "#{server.config[:pid]}.oldbin" if File.exists?(old_pid) && server.pid != old_pid begin Process.kill("QUIT", File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH # someone else did our job for us end end end
  32. Capistrano - templates/unicorn.rb.erb after_fork do |server, worker| # Start up the database connection again in the worker if defined?(ActiveRecord::Base) ActiveRecord::Base.establish_connection end child_pid = server.config[:pid].sub(".pid", ".#{worker.nr}.pid") system("echo #{Process.pid} > #{child_pid}") end
  33. Capistrano - t/monit/unicorn.erb check process <%= application %>_unicorn with pidfile <%= unicorn_pid %> start program = "/etc/init.d/unicorn_<%= application %> start" stop program = "/etc/init.d/unicorn_<%= application %> stop" <% unicorn_workers.times do |n| %> <% pid = unicorn_pid.sub(".pid", ".#{n}.pid") %> check process <%= application %>_unicorn_worker_<%= n %> with pidfile <%= pid %> start program = "/bin/true" stop program = "/usr/bin/test -s <%= pid %> && /bin/kill -QUIT `cat <%= pid %>`" if mem > 200.0 MB for 1 cycles then restart if cpu > 50% for 3 cycles then restart if 5 restarts within 5 cycles then timeout alert <%= alert_email %> only on { pid } if changed pid 2 times within 60 cycles then alert <% end %>
  34. Whoa!
  35. But really, it is just a bunch of Erb for files you already have
  36. Did you see the trick? ● after "deploy:setup", "nginx:setup" So we can... ● cap staging deploy:setup deploy:migrations
  37. From the top!
  38. Ready?!? Here we go! 1. New VM at my_web_app in your .ssh/config 2. Create chef-repo/nodes/my_web_app.json 3. In chef-repo: bundle exec knife bootstrap node_name --template-file=ubuntu-12.04-lts.erb 4. bundle exec knife cook root@my_web_app 5. In app directory: create/edit config/deploy/staging.rb 6. cap staging deploy:setup deploy:migrations 7. Hit the bars
  39. Thoughts.... ● Vagrant and VMs are you friend. Rinse and repeat ● It is ok to tweak your Chef stuff and re-cook, but I always like to restart with a fresh VM once I think I'm done ● Capistrano tweaks should be easy to apply, especially with tasks like nginx:setup, unicorn:setup etc. ● Chef issues are harder to debug and more frustrating than Capistrano issues, another reason to put more app specific custom stuff in Capistrano and do standard things in Chef
  40. Questions? http://smartlogicsolutions.com http://twitter.com/smartlogic http://github.com/smartlogic   http://fb.me/smartlogic
Advertisement