Fosdem chef-101-app-deploy

7,737 views

Published on

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,737
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
110
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Fosdem chef-101-app-deploy

  1. Deploying Apps with Chef Speaker: Joshua Timberman Sr. Technical Evangelist ‣ joshua@opscode.com ‣ @jtimberman ‣ www.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 1Sunday, February 6, 2011
  2. Chef 101 Speaker: Joshua Timberman Sr. Technical Evangelist ‣ joshua@opscode.com ‣ @jtimberman ‣ www.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 2Sunday, February 6, 2011
  3. Copyright © 2010 Opscode, Inc - All Rights Reserved 3Sunday, February 6, 2011
  4. Copyright © 2010 Opscode, Inc - All Rights Reserved 4Sunday, February 6, 2011
  5. (They make computers?) Copyright © 2010 Opscode, Inc - All Rights Reserved 5Sunday, February 6, 2011
  6. (Security training/certification) Copyright © 2010 Opscode, Inc - All Rights Reserved 6Sunday, February 6, 2011
  7. Copyright © 2010 Opscode, Inc - All Rights Reserved 7Sunday, February 6, 2011
  8. Copyright © 2010 Opscode, Inc - All Rights Reserved 8Sunday, February 6, 2011
  9. Copyright © 2010 Opscode, Inc - All Rights Reserved 9Sunday, February 6, 2011
  10. Copyright © 2010 Opscode, Inc - All Rights Reserved 10Sunday, February 6, 2011
  11. Copyright © 2010 Opscode, Inc - All Rights Reserved 11Sunday, February 6, 2011
  12. Copyright © 2010 Opscode, Inc - All Rights Reserved 12Sunday, February 6, 2011
  13. Copyright © 2010 Opscode, Inc - All Rights Reserved 13Sunday, February 6, 2011
  14. Copyright © 2010 Opscode, Inc - All Rights Reserved 14Sunday, February 6, 2011
  15. Copyright © 2010 Opscode, Inc - All Rights Reserved 15Sunday, February 6, 2011
  16. Copyright © 2010 Opscode, Inc - All Rights Reserved 16Sunday, February 6, 2011
  17. Copyright © 2010 Opscode, Inc - All Rights Reserved 17Sunday, February 6, 2011
  18. Developers? System administrators? “Business” people? http://www.flickr.com/photos/timyates/2854357446/sizes/l/ Copyright © 2010 Opscode, Inc - All Rights Reserved 18Sunday, February 6, 2011
  19. Copyright © 2010 Opscode, Inc - All Rights Reserved 19Sunday, February 6, 2011
  20. At a High Level... ‣ A library for configuration management ‣ A configuration management system ‣ A systems integration platform ‣ An API for your entire Infrastructure http://www.flickr.com/photos/asten/2159525309/sizes/l/Sunday, February 6, 2011
  21. Principles Idempotent Data-driven Sane defaults Hackability TMTOWTDI Copyright © 2010 Opscode, Inc - All Rights Reserved 21Sunday, February 6, 2011
  22. Multiple applications of an operation do not change the result Copyright © 2010 Opscode, Inc - All Rights Reserved 22Sunday, February 6, 2011
  23. We start with APIs, you supply data Copyright © 2010 Opscode, Inc - All Rights Reserved 23Sunday, February 6, 2011
  24. option :json_attribs, :short => "-j JSON_ATTRIBS", :long => "--json-attributes JSON_ATTRIBS", :description => "Load attributes from aJSON file or URL", :proc => nil option :node_name, :short => "-N NODE_NAME", :long => "--node-name NODE_NAME", :description => "The node name for thisclient", Defaults are sane, but :proc => nil easily changed Copyright © 2010 Opscode, Inc - All Rights Reserved 24Sunday, February 6, 2011
  25. Open source and community Copyright © 2010 Opscode, Inc - All Rights Reserved 25Sunday, February 6, 2011
  26. Copyright © 2010 Opscode, Inc - All Rights Reserved 26Sunday, February 6, 2011
  27. TIMTOWTDI is a Perl motto Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/lidarose/225156612 27Sunday, February 6, 2011
  28. A Tour of Chef Copyright © 2010 Opscode, Inc - All Rights Reserved 28Sunday, February 6, 2011
  29. Chef Client runs on your systems Copyright © 2010 Opscode, Inc - All Rights Reserved 29Sunday, February 6, 2011
  30. Clients talk to a Chef Server Copyright © 2010 Opscode, Inc - All Rights Reserved 30Sunday, February 6, 2011
  31. Clients authenticate with RSA keys Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/debbcollins/3401944550/ 31Sunday, February 6, 2011
  32. The Opscode Platform is a Chef Server Copyright © 2010 Opscode, Inc - All Rights Reserved 32Sunday, February 6, 2011
  33. Command-line API utility, Knife http://www.flickr.com/photos/myklroventine/3474391066/ Copyright © 2010 Opscode, Inc - All Rights Reserved 33Sunday, February 6, 2011
  34. We call each system you configure a Node Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.flickr.com/photos/peterrosbjerg/3913766224/ 34Sunday, February 6, 2011
  35. Nodes have Attributes Kernel info!{ "kernel": { "machine": "x86_64", "name": "Darwin", "os": "Darwin", "version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;root:xnu-1504.7.4~1/RELEASE_I386", "release": "10.4.0" }, "platform_version": "10.6.4", "platform": "mac_os_x", "platform_build": "10F569", "domain": "local", Platform info! "os": "darwin", "current_user": "jtimberman", "ohai_time": 1278602661.60043, "os_version": "10.4.0", "uptime": "18 days 17 hours 49 minutes 18 seconds", "ipaddress": "10.13.37.116", "hostname": "cider", "fqdn": "cider.local", Hostname and IP! "uptime_seconds": 1619358} Copyright © 2010 Opscode, Inc - All Rights Reserved 35Sunday, February 6, 2011
  36. Attributes are Searchable $ knife search node ‘platform:mac_os_x’ search(:node, ‘platform:mac_os_x’) Copyright © 2010 Opscode, Inc - All Rights Reserved 36Sunday, February 6, 2011
  37. Nodes have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 37Sunday, February 6, 2011
  38. Nodes have a Run List { "run_list": [ "role[production]", "role[webserver]" ] } Copyright © 2010 Opscode, Inc - All Rights Reserved 38Sunday, February 6, 2011
  39. Nodes have Roles Copyright © 2010 Opscode, Inc - All Rights Reserved 39Sunday, February 6, 2011
  40. Roles have a Run List What Roles or Recipes to apply in Order Copyright © 2010 Opscode, Inc - All Rights Reserved 40Sunday, February 6, 2011
  41. name "webserver" description "Systems that serve HTTP traffic" run_list( "recipe[apache2]", "recipe[apache2::mod_ssl]" ) default_attributes( "apache" => { "listen_ports" => [ "80", "443" ] } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 41Sunday, February 6, 2011
  42. Roles are Searchable $ knife search role ‘listen_ports:80’ search(:role, ‘listen_ports:80’) Copyright © 2010 Opscode, Inc - All Rights Reserved 42Sunday, February 6, 2011
  43. Chef manages Resources on Nodes Copyright © 2010 Opscode, Inc - All Rights Reserved 43Sunday, February 6, 2011
  44. Resources... Declare a description of the state a part of the node should be in ‣ Have a type package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Have a name end template "/etc/apache2/apache2.conf" do ‣ Have parameters source "apache2.conf.erb" owner "root" ‣ Take action to put the group "root" mode 0644 resource in the action :create declared state endSunday, February 6, 2011
  45. Resources take action through Providers Copyright © 2010 Opscode, Inc - All Rights Reserved 45Sunday, February 6, 2011
  46. Providers... Know how to actually perform the actions specified by a resource. Apt, Yum, Rubygems, Multiple providers per resource type. Portage, Macports, FreeBSD Ports, etc.Sunday, February 6, 2011
  47. Resources Platform ProviderSunday, February 6, 2011
  48. Chef::Platform:ubuntu => { :default => { :package => Chef::Provider::Package::Apt, :service => Chef::Provider::Service::Debian, :cron => Chef::Provider::Cron, :mdadm => Chef::Provider::Mdadm }}, Copyright © 2010 Opscode, Inc - All Rights Reserved 48Sunday, February 6, 2011
  49. Recipes are lists of Resources Copyright © 2010 Opscode, Inc - All Rights Reserved 49Sunday, February 6, 2011
  50. Recipes... Apply resources in the order they are specified 1 package "apache2" do version "2.2.11-2ubuntu2.6" action :install ‣ Evaluates resources in end the order they appear template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" ‣ Adds each resource to owner "root" the Resource Collection group "root" mode 0644 action :create 2 endSunday, February 6, 2011
  51. Recipes are just Ruby! extra_packages = case node[:platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end Copyright © 2010 Opscode, Inc - All Rights Reserved 51Sunday, February 6, 2011
  52. Cookbooks are packages for Recipes Copyright © 2010 Opscode, Inc - All Rights Reserved 52Sunday, February 6, 2011
  53. Cookbooks are shareable! cookbooks.opscode.com Copyright © 2010 Opscode, Inc - All Rights Reserved 53Sunday, February 6, 2011
  54. Data bags store arbitrary data Copyright © 2010 Opscode, Inc - All Rights Reserved 54Sunday, February 6, 2011
  55. A user data bag item... % knife data bag show users jtimberman { "comment": "Joshua Timberman", "groups": "sysadmin", "ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider", "files": { ".zshrc": { "mode": "0644", "source": "dot-zshrc" }, ".vimrc": { "mode": "0644", "source": "dot-vimrc" } }, "id": "jtimberman", "uid": 7004, "shell": "/usr/bin/zsh", "openid": "http://jtimberman.myopenid.com/" } Copyright © 2010 Opscode, Inc - All Rights ReservedSunday, February 6, 2011
  56. Data Bags are Searchable $ knife search users ‘shell:/bin/bash’ search(:users, ‘/bin/bash’) Copyright © 2010 Opscode, Inc - All Rights Reserved 56Sunday, February 6, 2011
  57. bash_users = search(:users, shell:/bin/bash) bash_users.each do |u| user u[id] do uid u[id] shell "/usr/bin/zsh" comment u[comment] supports :manage_home => true Data bags make recipes home "/home/#{u[id]}" end awesome-r (that’s directory "/home/#{u[id]}/.ssh" do owner u[id] group u[id] mode 0700 totally a word) end template "/home/#{u[id]}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u[id] group u[id] mode 0600 variables :ssh_keys => u[ssh_keys] end end Copyright © 2010 Opscode, Inc - All Rights Reserved 57Sunday, February 6, 2011
  58. I can has applications? Copyright © 2010 Opscode, Inc - All Rights Reserved 58Sunday, February 6, 2011
  59. Copyright © 2010 Opscode, Inc - All Rights Reserved 59Sunday, February 6, 2011
  60. Copyright © 2010 Opscode, Inc - All Rights Reserved 60Sunday, February 6, 2011
  61. Copyright © 2010 Opscode, Inc - All Rights Reserved 61Sunday, February 6, 2011
  62. Copyright © 2010 Opscode, Inc - All Rights Reserved 62Sunday, February 6, 2011
  63. Copyright © 2010 Opscode, Inc - All Rights Reserved 63Sunday, February 6, 2011
  64. Application Deployment Copyright © 2010 Opscode, Inc - All Rights Reserved 64Sunday, February 6, 2011
  65. Application Deployment tar -x -C /app -f app.tar rsync ~/dev/app www:/app cap deploy Copyright © 2010 Opscode, Inc - All Rights Reserved 65Sunday, February 6, 2011
  66. Server Configuration Copyright © 2010 Opscode, Inc - All Rights Reserved 66Sunday, February 6, 2011
  67. Server Configuration Web Servers Load Balancers Database Servers Copyright © 2010 Opscode, Inc - All Rights Reserved 67Sunday, February 6, 2011
  68. Server Configuration % vi /etc/mysql/my.cnf #!/bin/bash Capfile Copyright © 2010 Opscode, Inc - All Rights Reserved 68Sunday, February 6, 2011
  69. Capistrano anyone? def install_package(pkg) if pkg.kind_of?(Array) run("apt-get -y install #{pkg.join( )}") else run("apt-get -y install #{pkg}") end end packages = [ "build-essential", "ruby", "ruby1.8-dev", "libopenssl-ruby", "rake", "irb", "zlib1g-dev", "libssl-dev", "git-core" ] logger.info("Installing baseline packages: #{packages.join( )}") install_package(packages) Copyright © 2010 Opscode, Inc - All Rights Reserved 69Sunday, February 6, 2011
  70. Application Deployment Application Repository ‣ Source ‣ CI / Build Chef Repository cider:~/dev/rails-quick-start (ruby-1.9.2-p0) master ✔ % ls -l total 16 -rw-r--r-- 1 jtimberman staff 3521 Nov 5 13:09 README.md -rw-r--r-- 1 jtimberman staff 2171 Nov 5 13:09 Rakefile drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 certificates/ drwxr-xr-x 3 jtimberman staff 102 Nov 5 13:09 config/ drwxr-xr-x 26 jtimberman staff 884 Nov 12 08:16 cookbooks/ drwxr-xr-x 4 jtimberman staff 136 Nov 5 13:25 data_bags/ drwxr-xr-x 9 jtimberman staff 306 Nov 12 08:16 roles/ Copyright © 2010 Opscode, Inc - All Rights Reserved 70Sunday, February 6, 2011
  71. Chef Repository Roles Cookbooks Application Information ‣ Data Bag! Copyright © 2010 Opscode, Inc - All Rights Reserved 71Sunday, February 6, 2011
  72. Application Information Data Bag JSON Predefined structure Copyright © 2010 Opscode, Inc - All Rights Reserved 72Sunday, February 6, 2011
  73. Walkthrough Copyright © 2010 Opscode, Inc - All Rights Reserved 73Sunday, February 6, 2011
  74. knife ec2 server create role[production] role[base] role[radiant_database_master] role[radiant] role[radiant_run_migrations] recipe[radiant::db_bootstrap] -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu -G default -i ami-a403f7cd -f m1.small Copyright © 2010 Opscode, Inc - All Rights Reserved 74Sunday, February 6, 2011
  75. knife ec2 server create role[production] role[base] role[radiant_database_master] -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu -G default -i ami-a403f7cd -f m1.smallknife ec2 server create role[production] role[base] role[radiant] role[radiant_run_migrations] recipe[radiant::db_bootstrap] -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu -G default -i ami-a403f7cd -f m1.smallknife ec2 server create role[production] role[base] role[radiant] -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu -G default -i ami-a403f7cd -f m1.smallknife ec2 server create role[production] role[base] role[radiant_load_balancer] -S rails-quick-start -I ~/.ssh/rails-quick-start.pem -x ubuntu -G default -i ami-a403f7cd -f m1.small Copyright © 2010 Opscode, Inc - All Rights Reserved 75Sunday, February 6, 2011
  76. { "id": "radiant", "server_roles": [ "radiant" ], "type": { "radiant": [ "rails", "unicorn" ] }, "database_master_role": [ "radiant_database_master" ], "repository": "git://github.com/radiant/radiant.git", "revision": { "production": "0.9.1" }, Copyright © 2010 Opscode, Inc - All Rights Reserved 76Sunday, February 6, 2011
  77. Server Roles base.rb production.rb radiant.rb radiant_database_master.rb radiant_load_balancer.rb radiant_run_migrations.rb Copyright © 2010 Opscode, Inc - All Rights Reserved 77Sunday, February 6, 2011
  78. Base Role name "base" description "Base role applied to all nodes." run_list( "recipe[apt]", "recipe[git]", "recipe[build-essential]", "recipe[ruby]" ) Copyright © 2010 Opscode, Inc - All Rights Reserved 78Sunday, February 6, 2011
  79. Production Role name "production" description "Nodes in the production environment." default_attributes( "app_environment" => "production" ) Copyright © 2010 Opscode, Inc - All Rights Reserved 79Sunday, February 6, 2011
  80. Radiant Role name "radiant" description "radiant front end application server." run_list( "recipe[mysql::client]", "recipe[application]" ) Copyright © 2010 Opscode, Inc - All Rights Reserved 80Sunday, February 6, 2011
  81. Application Recipe search(:apps) do |app| (app["server_roles"] & node.run_list.roles).each do |app_role| app["type"][app_role].each do |thing| node.run_state[:current_app] = app include_recipe "application::#{thing}" end end end node.run_state.delete(:current_app) Copyright © 2010 Opscode, Inc - All Rights Reserved 81Sunday, February 6, 2011
  82. Application Rails Recipeapp[gems].each do |gem,ver| gem_package gem do action :install version ver if ver && ver.length > 0 endenddeploy_revision app[id] do revision app[revision][node.app_environment] repository app[repository] user app[owner] group app[group] deploy_to app[deploy_to] environment RAILS_ENV => node.app_environment action app[force][node.app_environment] ? :force_deploy : :deploy ... Copyright © 2010 Opscode, Inc - All Rights Reserved 82Sunday, February 6, 2011
  83. Radiant Database Master Rolename "radiant_database_master"description "Database master for the radiantapplication."run_list( "recipe[database::master]") Copyright © 2010 Opscode, Inc - All Rights Reserved 83Sunday, February 6, 2011
  84. Database Master Recipe search(:apps) do |app| (app[database_master_role] & node.run_list.roles).each do | dbm_role| app[databases].each do |env,db| if env =~ /#{node[:app_environment]}/ root_pw = node["mysql"]["server_root_password"] mysql_database "create #{db[database]}" do host "localhost" username "root" password root_pw database db[database] action [:create_db] end end end end end Copyright © 2010 Opscode, Inc - All Rights Reserved 84Sunday, February 6, 2011
  85. Radiant Load Balancer Role name "radiant_load_balancer" description "radiant load balancer" run_list( "recipe[haproxy::app_lb]" ) override_attributes( "haproxy" => { "app_server_role" => "radiant" } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 85Sunday, February 6, 2011
  86. Haproxy App Load Balancer Recipepool_members = search("node", "role:#{node[haproxy][app_server_role]} AND app_environment:#{node[app_environment]}")|| []template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members notifies :restart, resources(:service => "haproxy")end Copyright © 2010 Opscode, Inc - All Rights Reserved 86Sunday, February 6, 2011
  87. Control Deployment knife data bag edit apps radiant "force": { "production": false }, "force": { "production": true }, Copyright © 2010 Opscode, Inc - All Rights Reserved 87Sunday, February 6, 2011
  88. Control Migrations data bag item has migrate setting need an attribute set as well "migrate": { "production": true } name "radiant_run_migrations" description "Run db:migrate on demand for radiant" override_attributes( "apps" => { "radiant" => { "production" => { "run_migrations" => true } } } ) Copyright © 2010 Opscode, Inc - All Rights Reserved 88Sunday, February 6, 2011
  89. Add the role and migrations will be run knife node run list add NODE ‘role[radiant_run_migrations]’ Migrations will run, and the role is removed by Chef automatically. ruby_block "remove_run_migrations" do block do if node.role?("#{app[id]}_run_migrations") Chef::Log.info("Migrations were run, removing role[#{app[id]}_run_migrations]") node.run_list.remove("role[#{app[id]}_run_migrations]") end end end Copyright © 2010 Opscode, Inc - All Rights Reserved 89Sunday, February 6, 2011
  90. Your Application Your application is different than Radiant. But not a unique snowflake, right? Mostly, you will just need to modify the data and create application specific roles... But wait, I’m using Rails 3! Copyright © 2010 Opscode, Inc - All Rights Reserved 90Sunday, February 6, 2011
  91. Use the Gems data Use bundler or bundler08 in the gems hash of the application data "gems": { "gems": { "bundler": "1.0.9" "bundler08": "0.8.5" }, }, before_migrate do if app[gems].has_key?(bundler) execute "bundle install" do ignore_failure true cwd release_path end elsif app[gems].has_key?(bundler08) execute "gem bundle" do ignore_failure true cwd release_path end Copyright © 2010 Opscode, Inc - All Rights Reserved 91Sunday, February 6, 2011
  92. How does it scale? Copyright © 2010 Opscode, Inc - All Rights Reserved 92Sunday, February 6, 2011
  93. Quick FAQ Copyright © 2010 Opscode, Inc - All Rights Reserved 93Sunday, February 6, 2011
  94. Testing Copyright © 2010 Opscode, Inc - All Rights Reserved 94Sunday, February 6, 2011
  95. Reporting Copyright © 2010 Opscode, Inc - All Rights Reserved 95Sunday, February 6, 2011
  96. vs [Other tool] Copyright © 2010 Opscode, Inc - All Rights Reserved 96Sunday, February 6, 2011
  97. These slides will be posted Copyright © 2010 Opscode, Inc - All Rights Reserved 97Sunday, February 6, 2011
  98. Resources/Questions www.opscode.com/chef IRC and Mailing lists ‣ irc.freenode.net #chef ‣ lists.opscode.com Twitter: ‣ @opscode, #opschef ‣ @jtimberman Questions? Copyright © 2010 Opscode, Inc - All Rights Reserved 98Sunday, February 6, 2011

×