Railsconf2011 deployment tips_for_slideshare

  • 3,098 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,098
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
164
Comments
0
Likes
15

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • tunnel from port 4567 to port 80 on the VM\ntunnel from port 2222 to port 22 on the VM\nfolder in which vagrantfile exists is shared\n
  • tunnel from port 4567 to port 80 on the VM\ntunnel from port 2222 to port 22 on the VM\nfolder in which vagrantfile exists is shared\n
  • \n
  • - Use case for testing system configuration\n- Provisioning script development (Puppet, Chef, etc.)\n- Networked so they can talk\n
  • \n
  • \n
  • lib/deploy/\nhooks in config/deploy.rb\n\n\n
  • \n
  • submodule or gem? 50/50, maybe submodule\n
  • \n
  • \n
  • many servers, only one accepting connections\n
  • cap 2.6.0 released may 4\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • deploy to multiple environments\nintegration/qa/production\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • if you have qa/performance blocks in your gemfile, set :bundle_without\n
  • if you have qa/performance blocks in your gemfile, set :bundle_without\n
  • - show of hands - who's using hoptoad?\n - exception_notifier - pros: easy, simple, can modify to put in DB or on msg queue cons: flood o' email\n - can use new relic - pros: good display, no email flood, (? how does dev get notified?) cons: some delay, $$\n - hoptoad\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • individual accounts: logs get wrong permissions, daemons get started as wrong users, you need a generic account for init scripts anyway\ngoal is to avoid situation where people have to ssh in\n
  • when task fails, capistrano checks for rollback hooks\n
  • “cap deploy” consists of update and restart\nupdate has two subtasks which it runs in a txn\n
  • so you can see how you can write your own tasks that use txns and clean up after themselves\n
  • so you can see how you can write your own tasks that use txns and clean up after themselves\n
  • tinder - campfire notify, but don’t notify to ‘human’ chatrooms\ncap_gun - email, includes diff\n\n\n
  • \n
  • can also set : newrelic_revision, :newrelic_changelog\non the newrelic site appears as a line in the graph, also a list of recent deployments\n
  • \n
  • \n
  • \n
  • not just because of size, but also sensitive data\nautomate it\ndump from slave, normalize sensitive data (passwords)\ndecrease size by keeping recent or by partitioning\n
  • apache-specific\n
  • can use either time interval or size\nwith nginx, mv log, kill -USR1 4242, sleep 5, zip log or whatever\n
  • monitoring passenger memory usage\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • private keys with passphrases without hassle\nlaunchd - org.openbsd.ssh-agent.plist\n“ForwardAgent yes” in your .ssh/config\n
  • too many keys in ssh-agent results in server rejection (MaxAuthTries 6, maximum number of authentication attempts permitted per connection)\nssh-add -l to list, ssh-add -d filename to remove\n
  • \n
  • \n
  • \n

Transcript

  • 1. 21 DeploymentTips in 50 Minutes Anthony Burns & Tom Copeland
  • 2. Vagrant +• Created by Mitchell Hashimoto• http://vagrantup.com• Ruby DSL for provisioning VirtualBox VMs
  • 3. Vagrant::Config.run do |config| config.vm.box = dealz-ubuntu-lucid config.vm.forward_port(web, 80, 4567) config.vm.host_name(dealz) config.vm.network(13.37.33.30) config.vm.customize do |vm| vm.name = dealz endend
  • 4. config.network(‘13.37.33.33’)$ ping 13.37.33.31PING 13.37.33.31 (13.37.33.31): 56 data bytes64 bytes from 13.37.33.31: icmp_seq=0 ttl=64 time=0.337 ms64 bytes from 13.37.33.31: icmp_seq=1 ttl=64 time=0.290 ms64 bytes from 13.37.33.31: icmp_seq=2 ttl=64 time=0.320 ms
  • 5. Sadly, Slideshare can’t convert our videos
  • 6. Vagrant Multi-VM
  • 7. Vagrant::Config.run do |config| config.vm.define :web do |web_config| web_config.vm.box = dealz-ubuntu-lucid web_config.vm.host_name = dealz-web web_config.vm.network(13.37.33.31) web_config.vm.customize do |vm| vm.name = dealz-web end end config.vm.define :app do |app_config| app_config.vm.box = dealz-ubuntu-lucid app_config.vm.host_name = dealz-app app_config.vm.network(13.37.33.32) app_config.vm.customize do |vm| vm.name = dealz-app end end config.vm.define :db do |db_config| db_config.vm.box = dealz-ubuntu-lucid db_config.vm.host_name = dealz-db db_config.vm.network(13.37.33.33) db_config.vm.customize do |vm| vm.name = dealz-db end endend
  • 8. Sadly, Slideshare can’t convert our videos
  • 9. Capistrano OrganizationPittCaleb (flickr)
  • 10. Sharing Taskscourosa (flickr)
  • 11. Deploying with RVM• Created by Wayne Seguin• http://rvm.beginrescueend.com• Easy to have multiple Rubies• Not just for development
  • 12. Deploying with RVM$:.unshift File.expand_path(ENV[‘RVM_PATH’])require ‘rvm/capistrano’set :rvm_ruby_string, ‘ruby-1.9.2-p180’set :rvm_type, :system
  • 13. Capistrano gatewaymwschaff (Flickr)
  • 14. set :gateway, “myserver:4242”ORset ssh_options[:4242]ANDCapistrano 2.6.0 - multiple gateways!
  • 15. Speeding up with Git• Technique published by Chris Wanstrath in 2009• https://github.com/blog/470-deployment- script-spring-cleaning• Replacement for Capistrano’s deployment strategies
  • 16. Speeding up with Git• Technique published by Chris Wanstrath in 2009• https://github.com/blog/470-deployment- script-spring-cleaning• Replacement for Capistrano’s deployment strategies
  • 17. desc "Set up the current stage for Git-based deployment" task :setup, :except => { :no_release => true } do setup_command = ["rm -fr #{current_path}"] setup_command << "git clone #{repository} #{current_path}" setup_command << shared_children.map { |dir| "mkdir -p#{shared_path}/#{dir}" } setup_command << "rvm rvmrc trust #{current_path}" setup_command << "gem install bundler --no-rdoc --no-ri" run setup_command.join( && ) end
  • 18. desc "Update the current stage to the latest revision" task :update_code, :except => { :no_release => true } do update_command = ["git fetch origin && git reset --hard#{branch}"] update_command << "echo #{branch} > #{current_path}/BRANCH" update_command << "git rev-parse --verify HEAD --short >#{current_path}/REVISION" run "cd #{current_path} && #{update_command.join( && )}" end
  • 19. namespace :rollback do task :default do code end task :code, :except => { :no_release => true } do set :branch, HEAD^ deploy endend
  • 20. Speeding up symlinks• Capistrano defaults to using separate runs for each symlink• Each run is a separate SSH connection• Consolidate symlinks for a speed boost
  • 21. desc "Create symlinks to stage-specific configuration files andshared assets" task :symlink, :except => { :no_release => true } do command = cleanup_targets.map { |target| "rm -fr #{current_path}/#{target}" } command += release_directories.map { |directory| "mkdir -p#{directory}" } command += shared_symlinks.map { |from, to| "rm -fr#{current_path}/#{to} && ln -sf #{shared_path}/#{from}#{current_path}/#{to}" } run "cd #{current_path} && #{command.join(" && ")}" end
  • 22. desc "Create symlinks to stage-specific configuration files andshared assets" task :symlink, :except => { :no_release => true } do command = cleanup_targets.map { |target| "rm -fr #{current_path}/#{target}" } command += release_directories.map { |directory| "mkdir -p#{directory}" } command += shared_symlinks.map { |from, to| "rm -fr#{current_path}/#{to} && ln -sf #{shared_path}/#{from}#{current_path}/#{to}" } run "cd #{current_path} && #{command.join(" && ")}" end
  • 23. set :cleanup_targets, %w(log public/system tmp)set :release_directories, %w(log tmp)set :shared_symlinks, { config/database.yml => config/database.yml, config/hoptoad.yml => config/hoptoad.yml, config/newrelic.yml => config/newrelic.yml, log => log, pids => tmp/pids, sockets => tmp/sockets, system => public/system}
  • 24. Capistrano Multistagemzn37 (flickr)
  • 25. config/deploy.rbrequire capistrano/ext/multistageset :default_stage, qaset :application, dealzset :user, dealzset :group, dealzset :use_sudo, falseset :scm, :gitset :repository, git@github.com:dirtyalpaca/dealz.gitset :deploy_to, "/var/#{application}"set :deploy_via, :remote_cache
  • 26. config/deploy/production.rbset :rails_env, :productionrole :web, dealz-webrole :app, dealz-approle :db, dealz-db, :primary => true
  • 27. config/deploy/qa.rbset :rails_env, :qaserver dealz-qa, :web, :app, :db, :primary => true
  • 28. cap deploycap qa deploycap production deploy
  • 29. Autoloading from config/deployset :stage_dir, “config/stages”set :default_stage, “staging”
  • 30. +Capistrano
  • 31. bundler/capistrano.rbafter "deploy:update_code", "bundle:install"set :rake, bundle exec rake
  • 32. config/deploy.rbset :bundle_cmd # bundle (/opt/ree/bin/bundle)set :bundle_without, [:development, :test]
  • 33. Exception Tracking• exception_notification• newrelic_rpm• hoptoad
  • 34. exception_notification• Free and open source• https://github.com/rails/exception_notification• Simple email notification• Modifiable to put in database or message queue• Flood of emails
  • 35. newrelic_rpm• Hosted service (newrelic.com)• No email flood• Free and paid plans• Some delay
  • 36. Hoptoad• Hosted service (hoptoadapp.com)• Specifically for exception tracking• Free and paid plans• Excellent UI
  • 37. Capturing Output with Capistrano capture(command) stream(command)
  • 38. Capturing Outputnamespace :log do task :head, :roles => [:app] do capture(“tail -n 100 #{shared_path}/log/#{stage}.log”) end task :tail, :roles => [:app] do stream(“tail -f #{shared_path}/log/#{stage}.log”) endend
  • 39. Generic Deploy User
  • 40. Transactions and Rollbacks
  • 41. task :update do transaction do update_code symlink endend
  • 42. task :update_code, :except => { :no_release => true } do on_rollback { run "rm -rf #{release_path}; true" } [...]end
  • 43. task :symlink, :except => { :no_release => true } do on_rollback do if previous_release run
"rm
‐f
#{current_path};
ln
‐s
#{previous_release}
#{current_path};
true" [ ... ] end [...] end [...]end
  • 44. Capistrano Notificationswhatwhat (flickr)
  • 45. cap_gunrequire vendor/plugins/cap_gun/lib/cap_gunset :cap_gun_action_mailer_config, { :address => "smtp.gmail.com", :port => 587, :user_name => "[YOUR_USERNAME]@gmail.com", :password => "[YOUR_PASSWORD]", :authentication => :plain}set :cap_gun_email_envelope, { :from => "project.deployer@example.com", :recipients => %w[joe@example.com, jane@example.com]}after "deploy:restart", "cap_gun:email"
  • 46. Notifying NewRelic RPMrequire new_relic/recipes[ ... ]after "deploy:update", "newrelic:notice_deployment"
  • 47. Coming Soon Pagescap coming_soon:enablecap coming_soon:disable
  • 48. Coming Soon Pagesnamespace :coming_soon do desc "Redirect all requests to the coming soon page" task :enable, roles: :web, except: { no_release: true } do require erb on_rollback { run "rm -f #{shared_path}/system/coming_soon.html" } template = File.read(File.expand_path(../../../templates/coming_soon.html.erb, __FILE__)) result = ERB.new(template).result(binding) put result, "#{shared_path}/system/coming_soon.html", :mode => 0644 end desc "Disable application-wide coming soon page" task :disable, :roles => :web, :except => { :no_release => true } do run "rm -f #{shared_path}/system/coming_soon.html" endend
  • 49. Coming Soon Pagesif (-f $document_root/system/coming_soon.html) { rewrite ^(.*)$ /system/maintenance.html break;}
  • 50. Development Database Dumpsj_m_c (flickr)
  • 51. Log rotationdecade_null (flickr)
  • 52. CustomLog"|/usr/bin/rotatelogs/var/log/access_log 86400"common
  • 53. MonitoringPassenger
  • 54. require rubygemsgem passengerrequire phusion_passengerrequire phusion_passenger/platform_inforequire phusion_passenger/admin_tools/memory_statsprocs = AdminTools::MemoryStats.new.passenger_processesraise “ISSUE!” if procs.detect { |p| p.rss > 200*1000 }
  • 55. Capistrano + Unicornset :unicorn_command, bundle exec unicornset :unicorn_config, "#{current_path}/config/unicorn.rb"set :unicorn_pid, "#{current_path}/tmp/pids/unicorn.pid"namespace :deploy do task :start, :roles => :app, :except => { :no_release => true } do run "cd #{current_path} && #{unicorn_command} -c #{unicorn_config} -E#{rails_env} -D" end task :stop, :roles => :app, :except => { :no_release => true } do run "kill `cat #{unicorn_pid}`" end task :graceful_stop, :roles => :app, :except => { :no_release => true } do run "kill -s QUIT `cat #{unicorn_pid}`" end task :reload, :roles => :app, :except => { :no_release => true } do run "kill -s USR2 `cat #{unicorn_pid}`" end task :restart, :roles => :app, :except => { :no_release => true } do stop start endend
  • 56. Managing Your Schedule with Whenever• Written by Javan Makhmali• http://github.com/javan/whenever• Ruby DSL for specifying cron jobs• Integrates nicely with Capistrano
  • 57. Managing Your Schedule with Whenevergem ‘whenever’, :require => false
  • 58. Managing Your Schedule with Whenever wheneverize .
  • 59. Managing Your Schedule with Wheneverevery 3.hours do runner "MyModel.some_process" rake "my:rake:task" command "/usr/bin/my_great_command"endevery 1.day, :at => 4:30 am do runner "MyModel.task_to_run_at_four_thirty_in_the_morning"endevery :hour do # Many shortcutsavailable: :hour, :day, :month, :year, :reboot runner "SomeModel.ladeeda"endevery :sunday, :at => 12pm do # Use any day of the weekor :weekend, :weekday runner "Task.do_something_great"endevery 0 0 27-31 * * do command "echo you can use raw cron sytax too"end (from the GitHub wiki)
  • 60. Managing Your Schedule with Wheneverset :whenever_command, ‘bundle exec whenever’require ‘whenever/capistrano’ (from the GitHub wiki)
  • 61. ssh-agentandertoons-cartoons(flickr)
  • 62. A Gotchakayaker1204 (flickr)
  • 63. Shameless Plug
  • 64. Questions
  • 65. tom.copeland@livingsocial.comanthony.burns@livingsocial.com @tcopeland @dirtyalpaca