Chef 11 Preview/Chef for OpenStack

5,264 views

Published on

Presentation at the combined Boston Chef and OpenStack Meetups on January 22, 2013. Overview of the new features and changes coming in the upcoming Chef 11 release, as well as a quick state of the union for Chef for OpenStack.

Published in: Technology

Chef 11 Preview/Chef for OpenStack

  1. 1. Chef 11 Preview/Chef for OpenStack Boston 1/22/2013 Matt Ray matt@opscode.com @mattrayTuesday, January 22, 13
  2. 2. Whats New Since Chef 0.10? • Windows support • why-run • Librarian • output formatters • Spiceweasel • Test Kitchen • Foodcritic • Solaris support • Cookbooks in separate • Chef Developer Summit(2!) repositories • Berkshelf • Private Chef • ChefConf • Food Fight podcast • Fauxhai • knife plugins for • docs.opscode.com openstack, hp, azure, gce • 0.10 renumbered to Chef 10 • Hangouts for reviews • full-stack client • nyan-cat output formatterTuesday, January 22, 13Vaguely chronological, so much good stuff in the Chef Community
  3. 3. What is NOT in Chef 11 • CouchDB • Ruby-based chef-server-api • depsolver, gecode, treetop • Merb • OpenID support in Web UI • Migration tooling (coming soon) http://trainweb.org/carl/Pullman150/IMG_3380.jpgTuesday, January 22, 13
  4. 4. What’s NEW in Chef 11 • PostgreSQL • erchef • nginx • Rails • bookshelf • omnibus-chef server • chef-apply • partial search • users with key pairs • partials in templates • knife-essentialsTuesday, January 22, 13
  5. 5. Chef Server Architecture nginx erchef bookshelf chef-webui RabbitMQ chef-expander solr filesystem (lucene) cookbook PostgreSQL store filesystem search indexTuesday, January 22, 13Postgres has replaced Couch (who knew SQL was so good?)Erlang-based erchef has replaced Ruby-based chef-server-apichef-webui has been upgraded to Rails 3 from MerbBookshelf is our S3-compatibilish storage service
  6. 6. erchef Architecture erchef chef_wm chef_objects chef_authn chef_index chef_db bookshelf solr RabbitMQ PostgreSQLTuesday, January 22, 13Erlang applications, each on GitHub
  7. 7. CPU Usage on Chef ServerTuesday, January 22, 13These are old graphs from last year. Code is actually much faster and more efficient now that weve had a year of tuning.This is with erchef and Ruby code on the same box3 CPU vm graph is aggregate
  8. 8. CouchDB UptimeTuesday, January 22, 13the smaller spikes were when we chrond restarts
  9. 9. Database CPU CouchDB MySQLTuesday, January 22, 13Postgres is about the same performance, but weve found were able to focus optimizations on a single rdbms and put time thatwe would otherwise have to spend on ensuring both backend dbs work properly into other areas of the code for fixes, enhances,features.
  10. 10. Database Memory CouchDB MySQLTuesday, January 22, 13Nice and flat
  11. 11. Database Load Average CouchDB MySQLTuesday, January 22, 13Licensing is much better for us with Postgres
  12. 12. API Average LatencyTuesday, January 22, 13And things have gotten better from here with Private Chef and Open Source Chef. Not everything has been ported to OpscodeHosted Chef yet. Work has started to complete the migration of OHC to the OPC code base, things with OHC will get far better.
  13. 13. omnibus-chef server • full-stack for the Chef server, everything you need • installs to /opt/chef-server • RPMs, DEBs for now • build your own packages • chef-server-ctl • status • start/stop • tail http://apod.nasa.gov/apod/astropix.htmlTuesday, January 22, 13Much of the same tooling OPC
  14. 14. chef-apply • run a single recipe file • without modifying the nodes run_list • included with Chef gem, /usr/bin/chef-apply • chef-apply /path/to/recipe_file • chef-apply "content of a recipe file" http://www.flickr.com/photos/albill/sets/72157628046395000/Tuesday, January 22, 13http://tickets.opscode.com/browse/CHEF-3571
  15. 15. partial search partial_search(:node,  role:web,      :keys  =>  {  name  =>  [  name  ],                            ip      =>  [  ipaddress  ],                            kernel_version  =>  [  kernel,  version  ]                        } ).each  do  |result|    puts  result[name]    puts  result[ip]    puts  result[kernel_version] end • instead of entire node, just the pieces you want • massive reduction in bandwidth and memory • http://community.opscode.com/cookbooks/partial_search • to use with Chef 10 and Hosted Chef today http://www.flickr.com/photos/albill/sets/72157628046395000/Tuesday, January 22, 13
  16. 16. users with keypairs • users can have key pairs, just like clients • knife actions as a user instead of a client • post a public key when you create a user/client http://www.flickr.com/photos/albill/sets/72157628046395000/Tuesday, January 22, 13For users in Chef 11, the key point is:Before: users were just a concept of the webui. If you wanted a knife setup, you needed a client.After: users can have key pairs just like clients. So you can do knife actions as your user.
  17. 17. partials in templates <?xml version=1.0 encoding=UTF-8?> <server xmlns="urn:jboss:domain:1.2"> <extensions> <% if @infinispan -%> <extension module="org.jboss.as.clustering.infinispan"/> <% end -%> <extension module="org.jboss.as.web"/> <% if @webservices -%> <extension module="org.jboss.as.webservices"/> <% end -%> <extension module="org.jboss.as.weld"/> </extensions> <% if @infinispan include_template "infinispan.xml.erb" end -%> <% if @webservices include_template "webservices.xml.erb" end -%> <socket-binding-group name="standard-sockets" > ...... </socket-binding-group> </server> • Thanks to Andrea Campi! http://www.flickr.com/photos/modern_fred/2095565021/Tuesday, January 22, 13
  18. 18. knife-essentials • Unified commands that work on everything • https://github.com/jkeiser/knife-essentials • knife download [pattern1 pattern2 ...] • knife download roles data_bags cookbooks/emacs • knife diff cookbooks/*apache* • knife show *base* • knife upload apache* • knife list data_bags/users • knife deps roles/base.json http://photography.nationalgeographic.com/photography/photo-of-the-day/Tuesday, January 22, 13knife downloadknife diffknife showknife listknife upload
  19. 19. Chef 11 Breaking Changes • http://wiki.opscode.com/display/chef/Breaking +Changes+in+Chef+11 • chef-shell (formerly shef) • no implicits node attributes (no more node[blah] = foo) • attribute files may access role and environment attributes • delayed notifications run after failed converge • encrypted data bag item format change • chef-client lock so safe from simultaneous runs http://www.flickr.com/photos/modern_fred/2096352938/Tuesday, January 22, 13More changes on the wiki page
  20. 20. Chef 11 Server Preview • http://wiki.opscode.com/display/chef/Chef+11+Server +Preview • Nightly Builds • Centos 5 & 6 • Ubuntu 10.04, 11.04 and 12.04 • Everything is on GitHub • More testing means better releases! • Client is in beta: http://lists.opscode.com/sympa/arc/ chef-dev/2013-01/msg00028.html http://www.brandonbird.com/lazysunday.htmlTuesday, January 22, 13Coming soon!
  21. 21. Overview & Status of Chef for OpenStackTuesday, January 22, 13
  22. 22. Chef for OpenStack: WhoTuesday, January 22, 13These companies are currently involved to some extent
  23. 23. Chef for OpenStack: Why • Community for the automated deployment and management of OpenStack • Reduce fragmentation and encourage collaboration • Deploying OpenStack is not "secret sauce" • Project not a product • Apache 2 licenseTuesday, January 22, 13
  24. 24. Chef for OpenStack: What • Chef Repository for Deploying OpenStack • Documentation for Chef for OpenStack • Cookbooks • Keystone • Glance • Nova • Horizon • Swift • Quantum • Cinder • knife-openstackTuesday, January 22, 13
  25. 25. Chef for OpenStack: Where • groups.google.com/group/opscode-chef- openstack • #openstack-chef on irc.freenode.net • github.com/opscode/openstack-chef-repo • github.com/mattray/openstack-chef-docs • github.com/opscode-cookbooks/ • keystone, glance, nova, horizon, swift,quantum,cinder • github.com/opscode/knife-openstack • @chefopenstackTuesday, January 22, 13
  26. 26. Chef for OpenStack: When (Today) • Chef repo for Essex (2012.1.1 tag) • Operating Systems (Ubuntu 12.04) • Hypervisors (KVM, LXC) • Databases (MySQL) • FlatDHCP & VLAN networking • manual floating IPs documented • Test Kitchen integrationTuesday, January 22, 13
  27. 27. Chef for OpenStack: When (Tomorrow) • Documentation (docs.opscode.com) • Folsom is under active development • AT&T, DreamHost and Rackspace have active branches • Move to openstack-common • Cinder (lvm, Netapp) • Quantum (Nicira with Open vSwitch)Tuesday, January 22, 13
  28. 28. Chef for OpenStack When: (Roadmap) • Documentation (docs.opscode.com) • Continuous Integration testing • Grizzly & trunk(!?) • Cinder (Ceph) • Quantum (Midokura) • Hypervisors (Hyper-V, bare metal) • Databases (PostgreSQL) • Operating Systems (RHEL, Debian, SUSE) •Tuesday, January 22, 13 HA Configurations
  29. 29. Chef for OpenStack: How github.com/mattray/openstack-chef-docs moving to github.com/opscode/chef-docsTuesday, January 22, 13Working on a permanent URLHTML, PDF, Epub for your iPad
  30. 30. Questions? Boston 1/22/2013 Matt Ray matt@opscode.com @mattrayTuesday, January 22, 13ChefConf 2013 April 2013 "OPSCODE-MEETUP"Metarepo and CI work

×