Your SlideShare is downloading. ×
0
OpenStack Deployments with Chef    OpenStack Conference        Spring 2012            Matt Ray      matt@opscode.com IRC/T...
Deploying and ManagingOpenStack is not simple.
Chef makes it easier.
See NodeApplication Server
See NodesApplication ServerApplication Database
See Nodes GrowApplication ServerApplication Databases
See Nodes GrowApplication ServersApplication Databases
See Nodes GrowLoad BalancerApplication ServersApplication Databases
See Nodes GrowLoad Balancers        Application ServersApplication Databases
See Nodes GrowLoad Balancers        Application ServersApplication Database CacheApplication Databases
Tied together with ConfigLoad Balancers        Application ServersApplication Database CacheApplication Databases
Infrastructure is a SnowflakeLoad Balancers         Application ServersApplication Database CacheFloating IP?Application D...
Evolving Complexity        Load Balancers              ApplicationCache                   Application ServersNoSQL        ...
Complexity Grows Quickly      DC2DC1                      DC3
And it Continues to Evolve          http://www.flickr.com/photos/16339684@N00/2681435235/
Chef is Infrastructure as Code        •     Programmatically              provision and configure        •     Treat like ...
Nodes  • Chef-Client generates        configurations directly        on nodes from their        run list  • Reduce managem...
Collections of Resources• Networking                                        • Routes                                      ...
Declarative Interface to Resources•    Define policy•    Say what, not how•    Pull not Pushhttp://www.flickr.com/photos/bi...
Ruby!extra_packages = case node[platform]  when "ubuntu","debian"    %w{      ruby1.8      ruby1.8-dev      rdoc1.8      r...
Recipes and Cookbooks•   Recipes are collections of    Resources•   Cookbooks contain    recipes, templates, files,    cus...
Search•   Search for nodes    with Roles•   Find configuration    data•   IP addresses•   Hostnames•   FQDNs       http://...
Pass Results to Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do  source "hap...
Pass Results to Templates# Set up application listeners here.listen application 0.0.0.0:80  balance roundrobin  <% @pool_m...
So when thisGraphite   Nagios                             Jboss App           Memcache           Postgres Slaves          ...
Becomes thisGraphite   Nagios                             Jboss App           Memcache           Postgres Slaves          ...
Updates can be automaticGraphite   Nagios                             Jboss App           Memcache           Postgres Slav...
Count the resources                                               •   Load balancer config              Graphite          ...
Build anything       • Simple internal applications       • Complex external applications       • Workstations       • Had...
And manage it simply• Automatically  reconfigure  everything• Linux, Windows,  Unixes, BSDs• Load balancers• Metrics colle...
The Chef Community•   Apache License, Version 2.0•   675+ Individual contributors•   125+ Corporate contributors    •   De...
Whats Out There?openstack-cookbooks   openstack-chef     Resources   chef-openstack      crowbar
Opscode•   Bexar/Cactus, now deprecated•   original source for many other efforts    •   Crowbar    •   Mercado Libre    •...
TryStack•   TryStack.org, NTT & others•   Smokestack testing•   Xen & PostgreSQL•   Apache 2 license•   #trystack on irc.f...
Rackspace Cloud Builders•   Rackspace, DreamHost, Opscode & others•   KVM & MySQL on Ubuntu 12.04 (others)•   Apache 2 lic...
Crowbar•   Datacenter installer from Dell    •   Extension of the Chef server    •   "Barclamps" contain cookbooks + more•...
Deploying OpenStack•   Chef ties it all together automatically•   Scaling changes how we deploy•   Interchangeable compone...
knife openstack
knife openstack$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMA...
knife openstack flavor list$ knife openstack flavor listID Name        Virtual CPUs RAM          Disk1   m1.tiny    1     ...
knife openstack image list$ knife openstack image listID Name13 natty-server-cloudimg-amd6412 natty-server-cloudimg-amd64-...
knife openstack server createknife openstack server create --node-name ko1 --flavor 1 --image 13 -S trystack
$ ssh -i ~/.ssh/trystack.pem ubuntu@8.21.28.24The authenticity of host 8.21.28.24 (8.21.28.24) cant be established.RSA key...
Chef for Infrastructure Portability•   knife ec2•   knife rackspace•   knife hp•   knife openstack•   ... and many others
Whats Next?  http://www.flickr.com/photos/felixmorgner/4347750467/
knife-openstack Roadmap•   http://wiki.opscode.com/display/chef/    OpenStack+Bootstrap+Fast+Start+Guide•   Merges & publi...
Chef Roadmap•   Chef 0.10.10    •   Full Stack installer    •   Linux, BSDs, OSX, Solaris (Illumos),        Windows•   Che...
OpenStack Cookbooks Roadmap•   Additional Hypervisors•   More Databases•   Operations Support•   HA Configuration•   Addit...
Opscodes OpenStack Roadmap•   Collaborate with the Community•   Reference cookbooks for Essex•   OpenStack Quick Start tut...
Thanks!             Matt Ray       matt@opscode.com  IRC/Twitter/LP/GitHub: mattray       www.opscode.com
OpenStack Deployments with Chef
Upcoming SlideShare
Loading in...5
×

OpenStack Deployments with Chef

8,448

Published on

Chef is an open source configuration management and service integration automation tool that has been integral to a number of large successful OpenStack deployments. This talk will provide a brief introduction to Chef and why it frequently the configuration tool of choice for large deployments and discuss the use of Chef within the OpenStack ecosystem (development, testing, deploying and managing the installation). Chef also provides the ability to manage the instances running on top of Nova through the knife-openstack plugin.

Published in: Technology

Transcript of "OpenStack Deployments with Chef"

  1. 1. OpenStack Deployments with Chef OpenStack Conference Spring 2012 Matt Ray matt@opscode.com IRC/Twitter/LP/GitHub: mattray www.opscode.com
  2. 2. Deploying and ManagingOpenStack is not simple.
  3. 3. Chef makes it easier.
  4. 4. See NodeApplication Server
  5. 5. See NodesApplication ServerApplication Database
  6. 6. See Nodes GrowApplication ServerApplication Databases
  7. 7. See Nodes GrowApplication ServersApplication Databases
  8. 8. See Nodes GrowLoad BalancerApplication ServersApplication Databases
  9. 9. See Nodes GrowLoad Balancers Application ServersApplication Databases
  10. 10. See Nodes GrowLoad Balancers Application ServersApplication Database CacheApplication Databases
  11. 11. Tied together with ConfigLoad Balancers Application ServersApplication Database CacheApplication Databases
  12. 12. Infrastructure is a SnowflakeLoad Balancers Application ServersApplication Database CacheFloating IP?Application Databases
  13. 13. Evolving Complexity Load Balancers ApplicationCache Application ServersNoSQL Database Cache Database Slaves Database
  14. 14. Complexity Grows Quickly DC2DC1 DC3
  15. 15. And it Continues to Evolve http://www.flickr.com/photos/16339684@N00/2681435235/
  16. 16. Chef is Infrastructure as Code • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources. http://www.flickr.com/photos/louisb/4555295187/
  17. 17. Nodes • Chef-Client generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version controlhttp://www.flickr.com/photos/ssoosay/5126146763/
  18. 18. Collections of Resources• Networking • Routes • Users• Files • Groups• Directories • Tasks• Symlinks • Packages• Mounts • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
  19. 19. Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.flickr.com/photos/bixentro/2591838509/
  20. 20. Ruby!extra_packages = case node[platform] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } endextra_packages.each do |pkg| package pkg do action :install endend
  21. 21. Recipes and Cookbooks• Recipes are collections of Resources• Cookbooks contain recipes, templates, files, custom resources, etc• Code re-use and modularity• Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
  22. 22. Search• Search for nodes with Roles• Find configuration data• IP addresses• Hostnames• FQDNs http://www.flickr.com/photos/kathycsus/2686772625
  23. 23. Pass Results to Templatespool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end
  24. 24. Pass Results to Templates# Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%>
  25. 25. So when thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  26. 26. Becomes thisGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  27. 27. Updates can be automaticGraphite Nagios Jboss App Memcache Postgres Slaves Postgres Master
  28. 28. Count the resources • Load balancer config Graphite Nagios • Nagios host ping • Nagios host ssh Jboss App • Nagios host HTTP • Nagios host app health Memcache • Graphite CPU • Graphite Memory Postgres Slaves • Graphite Disk • Graphite SNMP • Memcache firewall• 12+ resource changes for 1 node addition • Postgres firewall Postgres authZ config
  29. 29. Build anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name ithttp://www.flickr.com/photos/hyku/245010680/
  30. 30. And manage it simply• Automatically reconfigure everything• Linux, Windows, Unixes, BSDs• Load balancers• Metrics collection systems• Monitoring systems• Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
  31. 31. The Chef Community• Apache License, Version 2.0• 675+ Individual contributors• 125+ Corporate contributors • Dell, Rackspace, VMware, DreamHost, Joyent, Heroku, RightScale, Deutsche Telekom and many more• 450+ cookbooks• http://community.opscode.com
  32. 32. Whats Out There?openstack-cookbooks openstack-chef Resources chef-openstack crowbar
  33. 33. Opscode• Bexar/Cactus, now deprecated• original source for many other efforts • Crowbar • Mercado Libre • HP• Apache 2 license• http://wiki.opscode.com/display/chef/ Deploying+OpenStack+with+Chef• github.com/mattray/openstack-cookbooks
  34. 34. TryStack• TryStack.org, NTT & others• Smokestack testing• Xen & PostgreSQL• Apache 2 license• #trystack on irc.freenode.org & Facebook• github.com/openstack/openstack-chef
  35. 35. Rackspace Cloud Builders• Rackspace, DreamHost, Opscode & others• KVM & MySQL on Ubuntu 12.04 (others)• Apache 2 license• #openstack-chef on irc.freenode.net• github.com/osops/ • monitoring, logging, operations tools• github.com/rcbops/chef-cookbooks
  36. 36. Crowbar• Datacenter installer from Dell • Extension of the Chef server • "Barclamps" contain cookbooks + more• Dell released under the Apache 2 license• Dell, enStratus, Opscode & many others• Mailing list and Skype• github.com/dellcloudedge/crowbar
  37. 37. Deploying OpenStack• Chef ties it all together automatically• Scaling changes how we deploy• Interchangeable components• Configurations shared, supported & documented• Licensing makes it available to everyone
  38. 38. knife openstack
  39. 39. knife openstack$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)
  40. 40. knife openstack flavor list$ knife openstack flavor listID Name Virtual CPUs RAM Disk1 m1.tiny 1 512 MB 0 GB2 m1.small 1 2048 MB 20 GB3 m1.medium 2 4096 MB 40 GB4 m1.large 4 8192 MB 80 GB5 m1.xlarge 8 16384 MB 160 GB
  41. 41. knife openstack image list$ knife openstack image listID Name13 natty-server-cloudimg-amd6412 natty-server-cloudimg-amd64-kernel15 oneiric-server-cloudimg-amd6414 oneiric-server-cloudimg-amd64-kernel
  42. 42. knife openstack server createknife openstack server create --node-name ko1 --flavor 1 --image 13 -S trystack
  43. 43. $ ssh -i ~/.ssh/trystack.pem ubuntu@8.21.28.24The authenticity of host 8.21.28.24 (8.21.28.24) cant be established.RSA key fingerprint is 0c:d8:3e:34:d1:de:c4:ee:5f:bc:b5:89:11:0d:73:e0.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 8.21.28.24 (RSA) to the list of known hosts.Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-13-virtual x86_64) * Documentation: https://help.ubuntu.com/ System information as of Thu Feb 16 23:43:29 UTC 2012 System load: 0.08 Processes: 63 Usage of /: 40.8% of 1.35GB Users logged in: 0 Memory usage: 6% IP address for eth0: 8.21.28.24 Swap usage: 0%---------------------------------------------------------------------<snip>Get cloud support with Ubuntu Advantage Cloud Guest http://www.ubuntu.com/business/services/cloudThe programs included with the Ubuntu system are free software;the exact distribution terms for each program are described in theindividual files in /usr/share/doc/*/copyright.Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted byapplicable law.To run a command as administrator (user "root"), use "sudo <command>".See "man sudo_root" for details.ubuntu@ko1:~$
  44. 44. Chef for Infrastructure Portability• knife ec2• knife rackspace• knife hp• knife openstack• ... and many others
  45. 45. Whats Next? http://www.flickr.com/photos/felixmorgner/4347750467/
  46. 46. knife-openstack Roadmap• http://wiki.opscode.com/display/chef/ OpenStack+Bootstrap+Fast+Start+Guide• Merges & publish on rubygems.org• Testing with Essex• tickets.opscode.com/browse/ KNIFE_OPENSTACK
  47. 47. Chef Roadmap• Chef 0.10.10 • Full Stack installer • Linux, BSDs, OSX, Solaris (Illumos), Windows• Chef 11 • Reporting • Dry Run
  48. 48. OpenStack Cookbooks Roadmap• Additional Hypervisors• More Databases• Operations Support• HA Configuration• Additional Operating Systems• Folsom (Quantum, Melange, etc.)
  49. 49. Opscodes OpenStack Roadmap• Collaborate with the Community• Reference cookbooks for Essex• OpenStack Quick Start tutorial• Refresh Deploying OpenStack with Chef• Knife OpenStack enhancements
  50. 50. Thanks! Matt Ray matt@opscode.com IRC/Twitter/LP/GitHub: mattray www.opscode.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×