Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

vBACD - Introduction to Opscode Chef - 2/29

2,692 views

Published on

The shift to cloud computing means that organizations are undergoing a major shift as they develop scale-out infrastructure that can respond to apace of business change faster than ever before. Opscode Chef® is an open-source systems integration framework build specifically for
automating the cloud by making it easy to deploy and scale servers and applications throughout your infrastructure. Join us for this session
containing an introduction to Chef including:

An Overview of Chef
The Chef Architecture
Cookbook Components
System Integration
Live demo launching a Java Stack on Amazon EC2, Rackspace, Ubuntu, and
CentOS


[Presented as part of the Open Source Build a Cloud program on 2/29/2012 - http://cloudstack.org/about-cloudstack/cloudstack-events.html?categoryid=6]

Published in: Technology
  • Be the first to comment

vBACD - Introduction to Opscode Chef - 2/29

  1. 1. Automating Cloud Deployments with Chef [email_address] www.opscode.com Twitter/IRC/GitHub: mattray
  2. 2. <ul><li>U has a cloud </li></ul><ul><li>Now what? </li></ul>http://www.flickr.com/photos/ian_munroe/4758240536/ Congratulations!!!
  3. 3. But then what? http://www.flickr.com/photos/doctorow/2698336843
  4. 4. You need to configure them
  5. 6. APIs are awesome! <ul><li>You can provision compute resources in seconds </li></ul><ul><li>You can provision storage resources in seconds </li></ul><ul><li>That’ s cool. </li></ul>http://www.flickr.com/photos/jdhancock/3634246981/
  6. 7. Chef can help with that <ul><li>knife ec2 server create </li></ul><ul><li>knife rackspace server create </li></ul><ul><li>knife terremark server create </li></ul><ul><li>knife voxel server create </li></ul><ul><li>knife cloudstack server create </li></ul><ul><li>knife kvm server create </li></ul><ul><li>knife vsphere server create </li></ul><ul><li>knife eucalyptus server create </li></ul><ul><li>knife openstack server create </li></ul>http://www.flickr.com/photos/kyz/3122499444/
  7. 8. See Node Application Server
  8. 9. See Nodes Application Server Application Database
  9. 10. See Nodes Grow Application Server Application Databases
  10. 11. Application Servers Application Databases See Nodes Grow
  11. 12. Application Servers Application Databases Load Balancer See Nodes Grow
  12. 13. See Nodes Grow Application Servers Application Databases Load Balancers
  13. 14. See Nodes Grow Application Servers Application Database Cache Load Balancers Application Databases
  14. 15. Tied together with Config Application Servers Application Database Cache Load Balancers Application Databases
  15. 16. Infrastructure is a Snowflake Application Servers Application Database Cache Load Balancers Floating IP? Application Databases
  16. 17. Evolving Complexity Load Balancers Application Servers NoSQL Database Slaves Application Cache Database Cache Database
  17. 18. Complexity Grows Quickly DC1 DC3 DC2
  18. 19. http://www.flickr.com/photos/ 16339684 @N00/2681435235/ And it Continues to Evolve
  19. 20. Golden Images are not the answer <ul><li>Gold is heavy </li></ul><ul><li>Hard to transport </li></ul><ul><li>Hard to mold </li></ul><ul><li>Easy to lose configuration detail </li></ul>http://www.flickr.com/photos/garysoup/2977173063/
  20. 21. Typical Boring Infrastructure Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite
  21. 22. <ul><li>Move SSH off port 22 </li></ul><ul><li>Lets put it on 2022 </li></ul>New Compliance Mandate Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite
  22. 23. <ul><li>edit /etc/ssh/sshd_config </li></ul>6 Golden Image Updates Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 3 4 5 6
  23. 24. <ul><li>Delete, launch </li></ul><ul><li>Repeat </li></ul><ul><li>Typically manually </li></ul>8 12 Instance Replacements Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 3 4 5 6 7 9 10 11 12
  24. 25. <ul><li>Don't break anything! </li></ul>In a Maintenance Window 5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3
  25. 26. <ul><li>Invalid Configs </li></ul><ul><li>Bob just got fired :( </li></ul>With Different IP Addresses? Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite
  26. 27. http://www.flickr.com/photos/francoforeshock/5716969942/ Configuration Desperation
  27. 28. Configuration Management and Automated Systems Integration is the Answer http://www.flickr.com/photos/philliecasablanca/3354734116/
  28. 29. Chef Solves This Problem <ul><li>But you already guessed that, didn ’t you? </li></ul>
  29. 30. Collections of Resources <ul><li>Networking </li></ul><ul><li>Files </li></ul><ul><li>Directories </li></ul><ul><li>Symlinks </li></ul><ul><li>Mounts </li></ul><ul><li>Routes </li></ul><ul><li>Users </li></ul><ul><li>Groups </li></ul><ul><li>Tasks </li></ul><ul><li>Packages </li></ul><ul><li>Software </li></ul><ul><li>Services </li></ul><ul><li>Configurations </li></ul><ul><li>Other Stuff </li></ul>http://www.flickr.com/photos/stevekeys/3123167585/
  30. 31. http://www.flickr.com/photos/glowjangles/4081048126/ Acting in Concert
  31. 32. http://www.flickr.com/photos/28309157@N08/3743455858/ To Provide a Service
  32. 33. Chef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ <ul><li>Programmatically provision and configure </li></ul><ul><li>Treat like any other code base </li></ul><ul><li>Reconstruct business from code repository, data backup, and bare metal resources. </li></ul>
  33. 34. Declarative Interface to Resources <ul><li>Define policy </li></ul><ul><li>Say what, not how </li></ul><ul><li>Pull not Push </li></ul>http://www.flickr.com/photos/bixentro/2591838509/
  34. 35. That looks like this extra_packages = case node['platform'] when &quot;ubuntu&quot;,&quot;debian&quot; %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end
  35. 36. Or this search(:users, '*:*') do |u| user u['id'] do uid u['uid'] shell u['shell'] home &quot;/home/#{u['id']}&quot; end directory &quot;#{home_dir}/.ssh&quot; do owner u['id'] group u['gid'] mode &quot;0700&quot; end template &quot;#{home_dir}/.ssh/authorized_keys&quot; do source &quot;authorized_keys.erb&quot; owner u['id'] group u['id'] mode &quot;0600&quot; variables :ssh_keys => u['ssh_keys'] end end
  36. 37. Recipes and Cookbooks <ul><li>Recipes are collections of Resources </li></ul><ul><li>Cookbooks contain recipes, templates, files, custom resources, etc </li></ul><ul><li>Code re-use and modularity </li></ul><ul><li>Hundreds already on Community.opscode.com </li></ul>http://www.flickr.com/photos/shutterhacks/4474421855/
  37. 38. <ul><li>Chef-Client generates configurations directly on nodes from their run list </li></ul><ul><li>Reduce management complexity through abstraction </li></ul><ul><li>Store the configuration of your programs in version control </li></ul>http://www.flickr.com/photos/ssoosay/5126146763/ Nodes
  38. 39. Upload your infrastructure knife cookbook upload chef-client knife cookbook upload java knife cookbook upload jpackage knife cookbook upload ntp knife cookbook upload sudo knife cookbook upload tomcat knife cookbook upload users knife cookbook upload sample knife role from file base.rb knife role from file tc.rb knife role from file sample.rb knife data bag create users knife data bag from file users mray.json
  39. 40. Build it somewhere #EC2 knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce -f m1.small -d omnibus -r 'role[base],role[tc],role[sample] ’ #Rackspace knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample] ’ #CloudStack knife cs server create -S &quot;small instance&quot; -T &quot;CentOS 5.5(64-bit) no GUI (KVM)&quot; -i ~/.ssh/mray.pem -d omnibus -r 'role[base],role[tc],role[sample] ’ #Ubuntu Linux VM knife bootstrap test.lab -i ~/.ssh/mray.pem -x ubuntu --sudo -d omnibus -r 'role[base],role[tc],role[sample]'
  40. 41. Tomcat stack deployed ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Chef Run complete in 125.548799554 seconds ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Running report handlers ec2-107-21-179-169.compute-1.amazonaws.com [Thu, 23 Feb 2012 03:16:27 +0000] INFO: Report handlers complete Instance ID: i-ee18148b Flavor: m1.small Image: ami-0c6ebd65 Region: us-east-1 Availability Zone: us-east-1b Security Groups: default SSH Key: mray Root Device Type: instance-store Public DNS Name: ec2-107-21-179-169.compute-1.amazonaws.com Public IP Address: 107.21.179.169 Private DNS Name: ip-10-120-255-91.ec2.internal Private IP Address: 10.120.255.91 Environment: _default Run List: role[base], role[tc], role[sample]
  41. 42. Tomcat stack deployed
  42. 43. Update the sample cookbook $ knife cookbook upload sample Uploading sample [0.0.2] upload complete
  43. 44. Update the nodes $ knife ssh &quot;role:base&quot; &quot;sudo chef-client&quot; -i ~/.ssh/mray.pem -x ubuntu -a cloud.public_ipv4 107.21.179.169 [Thu, 23 Feb 2012 03:30:22 +0000] INFO: *** Chef 0.10.8 *** 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List is [role[base], role[tc], role[sample]] 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Run List expands to [chef-client, ntp, sudo, users::sysadmins, java, tomcat, sample] 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Starting Chef Run for i-ee18148b 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Running start handlers 107.21.179.169 [Thu, 23 Feb 2012 03:30:30 +0000] INFO: Start handlers complete. 107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Loading cookbooks [chef-client, java, jpackage, ntp, sample, sudo, tomcat, users] 107.21.179.169 [Thu, 23 Feb 2012 03:30:36 +0000] INFO: Storing updated cookbooks/sample/recipes/default.rb in the cache. <SNIP> 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] updated content 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: template[/var/lib/tomcat6/webapps/sample/index.html] sending restart action to service[tomcat] (immediate) 107.21.179.169 [Thu, 23 Feb 2012 03:30:37 +0000] INFO: Processing service[tomcat] action restart (tomcat::default line 42) 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: service[tomcat] restarted 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Chef Run complete in 15.170097638 seconds 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Running report handlers 107.21.179.169 [Thu, 23 Feb 2012 03:30:45 +0000] INFO: Report handlers complete
  44. 45. Tomcat stack updated
  45. 46. http://www.flickr.com/photos/kathycsus/2686772625 <ul><li>IP addresses </li></ul><ul><li>Hostnames </li></ul><ul><li>FQDNs </li></ul><ul><li>Search for nodes with Roles </li></ul><ul><li>Find configuration data </li></ul>Search
  46. 47. pool_members = search(&quot;node&quot;,&quot;role:webserver ”) template &quot;/etc/haproxy/haproxy.cfg&quot; do source &quot;haproxy-app_lb.cfg.erb ” owner &quot;root&quot; group &quot;root ” mode 0644 variables :pool_members => pool_members.uniq notifies :restart, &quot;service[haproxy] ” end Pass results into Templates
  47. 48. # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |memb| -%> server <%= memb[:hostname] %> <%= memb[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node[&quot;haproxy&quot;][&quot;enable_admin&quot;] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates
  48. 49. Jboss App Memcache Postgres Slaves Postgres Master So when this Nagios Graphite
  49. 50. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite Becomes this
  50. 51. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite Updates can be automatic
  51. 52. Nagios Graphite Count the resources Jboss App Memcache Postgres Slaves <ul><li>Load balancer config </li></ul><ul><li>Nagios host ping </li></ul><ul><li>Nagios host ssh </li></ul><ul><li>Nagios host HTTP </li></ul><ul><li>Nagios host app health </li></ul><ul><li>Graphite CPU </li></ul><ul><li>Graphite Memory </li></ul><ul><li>Graphite Disk </li></ul><ul><li>Graphite SNMP </li></ul><ul><li>Memcache firewall </li></ul><ul><li>Postgres firewall </li></ul><ul><li>Postgres authZ config </li></ul>12+ resource changes for 1 node addition
  52. 53. http://www.flickr.com/photos/evelynishere/2798236471/ CLONING CANNOT COPE WITH THIS <ul><li>Chef can. </li></ul>
  53. 54. Build anything <ul><li>Simple internal applications </li></ul><ul><li>Complex external applications </li></ul><ul><li>Workstations </li></ul><ul><li>Hadoop clusters </li></ul><ul><li>IaaS infrastructure </li></ul><ul><li>PaaS infrastructure </li></ul><ul><li>SaaS applications </li></ul><ul><li>Storage systems </li></ul><ul><li>You name it </li></ul>http://www.flickr.com/photos/hyku/245010680/
  54. 55. And manage it simply http://www.flickr.com/photos/helico/404640681/ <ul><li>Automatically reconfigure everything </li></ul><ul><li>Linux, Windows, Unixes, BSDs </li></ul><ul><li>Load balancers </li></ul><ul><li>Metrics collection systems </li></ul><ul><li>Monitoring systems </li></ul><ul><li>Cloud migrations become trivial </li></ul>
  55. 56. The Chef Community <ul><li>Apache License, Version 2.0 </li></ul><ul><li>550+ Individual contributors </li></ul><ul><li>100+ Corporate contributors </li></ul><ul><ul><li>Dell, Rackspace,VMware, RightScale, Heroku, and many more </li></ul></ul><ul><li>400+ Community cookbooks </li></ul><ul><li>http://community.opscode.com </li></ul>
  56. 57. Questions? http://www.flickr.com/photos/mrchippy/443960682/ Questions?
  57. 58. Thanks! [email_address] www.opscode.com Twitter/IRC/GitHub: mattray

×