Your SlideShare is downloading. ×
0
Introduction to ChefSCALE Build a Cloud Day     matt@opscode.com     www.opscode.com         @mattray
Congratulations!!!       • U has a cloud       • Now what?      http://www.flickr.com/photos/ian_munroe/4758240536/
APIs are awesome!• You can provision  compute resources in  seconds• You can provision  storage resources in  seconds• Tha...
Chef can help with that• knife ec2 server                            • knife cloudstack        create                     ...
But then what?http://www.flickr.com/photos/doctorow/2698336843
You need to configure them
See NodeApplication Server
See NodesApplication ServerApplication Database
See Nodes GrowApplication ServerApplication Databases
See Nodes GrowApplication ServersApplication Databases
See Nodes GrowLoad BalancerApplication ServersApplication Databases
See Nodes GrowLoad Balancers        Application ServersApplication Databases
See Nodes GrowLoad Balancers        Application ServersApplication Database CacheApplication Databases
Tied together with Config Load Balancers         Application Servers Application Database Cache Application Databases
Infrastructure is a Snowflake   Load Balancers            Application Servers   Application Database Cache   Floating IP? ...
Evolving Complexity        Load Balancers              ApplicationCache                   Application ServersNoSQL        ...
Complexity Grows Quickly        DC2DC1                   DC3
And it Continues to Evolve              http://www.flickr.com/photos/16339684@N00/2681435235/
Golden Images are not the answer•   Gold is heavy•   Hard to transport•   Hard to mold•   Easy to lose    configuration de...
Typical Boring InfrastructureGraphite     Nagios                               Jboss App             Memcache             ...
New Compliance Mandate                         Graphite    Nagios                                                       Jb...
6 Golden Image Updates                       Graphite     1   2   Nagios                              3                   ...
12 Instance Replacements                       Graphite   1    2    Nagios                             3    4    5       6...
In a Maintenance Window                          Graphite   1    2    Nagios                              3      4    5   ...
With Different IP Addresses?                          Graphite     Nagios                                                 ...
Configuration Desperation                 http://www.flickr.com/photos/francoforeshock/5716969942/
Configuration Management andAutomated Systems Integration        is the Answer                 http://www.flickr.com/photos...
Chef Solves This Problem           • But you already             guessed that, didn’t             you?
Collections of Resources• Networking                                        • Routes                                      ...
Acting in Concert      http://www.flickr.com/photos/glowjangles/4081048126/
To Provide a Service        http://www.flickr.com/photos/28309157@N08/3743455858/
Chef is Infrastructure as Code             •     Programmatically                   provision and configure             • ...
Declarative Interface to Resources•    Define policy•    Say what, not how•    Pull not Pushhttp://www.flickr.com/photos/bi...
That looks like thisextra_packages = case node[platform]  when "ubuntu","debian"    %w{      ruby1.8      ruby1.8-dev     ...
Or thissearch(:users, *:*) do |u|  user u[id] do      uid u[uid]      shell u[shell]      home "/home/#{u[id]}"  end  dire...
Recipes and Cookbooks•   Recipes are collections of    Resources•   Cookbooks contain    recipes, templates, files,    cus...
Nodes  • Chef-Client generates       configurations directly       on nodes  • Reduce management       complexity through ...
Upload your infrastructureknife cookbook upload aptknife cookbook upload chef-clientknife cookbook upload javaknife cookbo...
Build it somewhere#EC2knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce-f m1.small -...
Search•   Search for nodes    with Roles•   Find configuration    data•   IP addresses•   Hostnames•   FQDNs       http://...
Pass results into Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do  source "h...
Pass results into Templates# Set up application listeners here.listen application 0.0.0.0:80  balance roundrobin  <% @pool...
So when thisGraphite   Nagios                             Jboss App           Memcache           Postgres Slaves          ...
Becomes thisGraphite   Nagios                             Jboss App           Memcache           Postgres Slaves          ...
Updates can be automaticGraphite         Nagios                                   Jboss App                 Memcache      ...
Count the resources                                               •   Load balancer config              Graphite          ...
CLONING CANNOT COPE WITH THIS                              •       Chef can.                    http://www.flickr.com/photo...
Build anything       • Simple internal applications       • Complex external applications       • Workstations       • Had...
And manage it simply• Automatically  reconfigure  everything• Linux, Windows,  Unixes, BSDs• Load balancers• Metrics colle...
The Chef Community•   Apache License, Version 2.0•   550+ Individual contributors•   100+ Corporate contributors    •   De...
Questions?  http://www.flickr.com/photos/mrchippy/443960682/
Thanks!    matt@opscode.com    www.opscode.com
SCALE 10x Build a Cloud Day
Upcoming SlideShare
Loading in...5
×

SCALE 10x Build a Cloud Day

163,948

Published on

Matt Ray's Introduction to Chef talk from the 2012 Southern California Linux Expo's Build a Cloud Day.

Published in: Technology

Transcript of "SCALE 10x Build a Cloud Day"

  1. 1. Introduction to ChefSCALE Build a Cloud Day matt@opscode.com www.opscode.com @mattray
  2. 2. Congratulations!!! • U has a cloud • Now what? http://www.flickr.com/photos/ian_munroe/4758240536/
  3. 3. APIs are awesome!• You can provision compute resources in seconds• You can provision storage resources in seconds• That’s cool. http://www.flickr.com/photos/jdhancock/3634246981/
  4. 4. Chef can help with that• knife ec2 server • knife cloudstack create server create• knife rackspace • knife kvm server server create create• knife terremark • knife vsphere server create server create• knife voxel • knife eucalyptus server create server create• knife gandi • knife openstack server create server createhttp://www.flickr.com/photos/kyz/3122499444/
  5. 5. But then what?http://www.flickr.com/photos/doctorow/2698336843
  6. 6. You need to configure them
  7. 7. See NodeApplication Server
  8. 8. See NodesApplication ServerApplication Database
  9. 9. See Nodes GrowApplication ServerApplication Databases
  10. 10. See Nodes GrowApplication ServersApplication Databases
  11. 11. See Nodes GrowLoad BalancerApplication ServersApplication Databases
  12. 12. See Nodes GrowLoad Balancers Application ServersApplication Databases
  13. 13. See Nodes GrowLoad Balancers Application ServersApplication Database CacheApplication Databases
  14. 14. Tied together with Config Load Balancers Application Servers Application Database Cache Application Databases
  15. 15. Infrastructure is a Snowflake Load Balancers Application Servers Application Database Cache Floating IP? Application Databases
  16. 16. Evolving Complexity Load Balancers ApplicationCache Application ServersNoSQL Database Cache Database Slaves Database
  17. 17. Complexity Grows Quickly DC2DC1 DC3
  18. 18. And it Continues to Evolve http://www.flickr.com/photos/16339684@N00/2681435235/
  19. 19. Golden Images are not the answer• Gold is heavy• Hard to transport• Hard to mold• Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
  20. 20. Typical Boring InfrastructureGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  21. 21. New Compliance Mandate Graphite Nagios Jboss App• Move SSH off port 22• Lets put it on 2022 Memcache Postgres Slaves Postgres Master
  22. 22. 6 Golden Image Updates Graphite 1 2 Nagios 3 Jboss App• edit /etc/ssh/sshd_config 4 Memcache 5 Postgres Slaves 6 Postgres Master
  23. 23. 12 Instance Replacements Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Delete, launch 8• Repeat 9 Memcache• Typically manually 10 11 Postgres Slaves 12 Postgres Master
  24. 24. In a Maintenance Window Graphite 1 2 Nagios 3 4 5 6 7 Jboss App• Dont break anything! 8 9 Memcache 5 10 11 Postgres Slaves 12 Postgres Master
  25. 25. With Different IP Addresses? Graphite Nagios Jboss App• Invalid Configs• Bob just got fired :( Memcache Postgres Slaves Postgres Master
  26. 26. Configuration Desperation http://www.flickr.com/photos/francoforeshock/5716969942/
  27. 27. Configuration Management andAutomated Systems Integration is the Answer http://www.flickr.com/photos/philliecasablanca/3354734116/
  28. 28. Chef Solves This Problem • But you already guessed that, didn’t you?
  29. 29. Collections of Resources• Networking • Routes • Users• Files • Groups• Directories • Tasks• Symlinks • Packages• Mounts • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
  30. 30. Acting in Concert http://www.flickr.com/photos/glowjangles/4081048126/
  31. 31. To Provide a Service http://www.flickr.com/photos/28309157@N08/3743455858/
  32. 32. Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
  33. 33. Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.flickr.com/photos/bixentro/2591838509/
  34. 34. That looks like thisextra_packages = case node[platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
  35. 35. Or thissearch(:users, *:*) do |u| user u[id] do uid u[uid] shell u[shell] home "/home/#{u[id]}" end directory "#{home_dir}/.ssh" do owner u[id] group u[gid] mode "0700" end template "#{home_dir}/.ssh/authorized_keys" do source "authorized_keys.erb" owner u[id] group u[id] mode "0600" variables :ssh_keys => u[ssh_keys] endend
  36. 36. Recipes and Cookbooks• Recipes are collections of Resources• Cookbooks contain recipes, templates, files, custom resources, etc• Code re-use and modularity• Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
  37. 37. Nodes • Chef-Client generates configurations directly on nodes • Reduce management complexity through abstraction • Store the configuration of your programs in version controlhttp://www.flickr.com/photos/ssoosay/5126146763/
  38. 38. Upload your infrastructureknife cookbook upload aptknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload jpackageknife cookbook upload ntpknife cookbook upload sudoknife cookbook upload tomcatknife cookbook upload usersknife cookbook upload sampleknife role from file base-cloud.rbknife role from file tc.rbknife role from file sample.rbknife data bag create usersknife data bag from file users mray.json
  39. 39. Build it somewhere#EC2knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce-f m1.small -d omnibus -r role[base-cloud],role[tc],role[sample]#Rackspaceknife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -rrole[base-cloud],role[tc],role[sample]#CloudStackknife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i~/.ssh/mray.pem -d omnibus -r role[base-cloud],role[tc],role[sample]#Ubuntu Linuxknife bootstrap test.lab -r role[webserver] -i ~/.ssh/mray.pem -x ubuntu --sudo -domnibus -r role[base-cloud],role[tc],role[sample]
  40. 40. Search• Search for nodes with Roles• Find configuration data• IP addresses• Hostnames• FQDNs http://www.flickr.com/photos/kathycsus/2686772625
  41. 41. Pass results into Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end
  42. 42. Pass results into Templates# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>
  43. 43. So when thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  44. 44. Becomes thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  45. 45. Updates can be automaticGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  46. 46. Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall• 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config
  47. 47. CLONING CANNOT COPE WITH THIS • Chef can. http://www.flickr.com/photos/evelynishere/2798236471/
  48. 48. Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name ithttp://www.flickr.com/photos/hyku/245010680/
  49. 49. And manage it simply• Automatically reconfigure everything• Linux, Windows, Unixes, BSDs• Load balancers• Metrics collection systems• Monitoring systems• Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
  50. 50. The Chef Community• Apache License, Version 2.0• 550+ Individual contributors• 100+ Corporate contributors • Dell, Rackspace,VMware, RightScale, Heroku, and many more• Nearly 400 cookbooks• http://community.opscode.com
  51. 51. Questions? http://www.flickr.com/photos/mrchippy/443960682/
  52. 52. Thanks! matt@opscode.com www.opscode.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×