OSDC 2013 | Introduction into Chef by Andy Hawkins
•
0 likes•28 views
This presentation will give an overview about what Chef is and how to access it. It will describe the typical use cases and architecture as well as Cookbooks, data bags and other concepts and will explain how to implement your CM solution. Finally it will show how to drive a successful Chef project.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to OSDC 2013 | Introduction into Chef by Andy Hawkins
Similar to OSDC 2013 | Introduction into Chef by Andy Hawkins (20)
Recently uploaded
Recently uploaded (20)
OSDC 2013 | Introduction into Chef by Andy Hawkins
- 1. Introduction to ChefIntroduction to Chef Andy Hawkins andy@opscode.com
- 2. Me :-)Me :-) • Solution Architect • Early cloud platforms • Roll-my -own automation • DCA • Chef! • Before that; system integration for major US outsourcer
- 3. Chef is an automation platform for developers & systems engineers to continuously define, build, and manage infrastructure. CHEF USES: • Recipes and Cookbooks that describe Infrastructure as Code. •Chef enables people to easily build & manage complex & dynamic applications at massive scale •New model for describing infrastructure that promotes reuse •Programmatically provision and configure •Reconstruct business from code repository, data backup, and bare metal resources ChefChef
- 4. When should I use an automation framework?When should I use an automation framework? • Now? • It is always earlier than you think :-) • Before you suffer configuration drift! • After you outgrow Heroku? • When you need to set-up the “third machine”
- 5. Evolving towards an automation frameworkEvolving towards an automation framework • Just build it • Keep notes in server.txt • Move notes to the wiki • Custom scripts (in scm?!) • Build from template / Golden Images • Automation framework
- 6. Typical Use CasesTypical Use Cases
- 7. See NodeSee Node Application Server
- 8. See NodesSee Nodes Application Server Application Database
- 9. See Nodes GrowSee Nodes Grow Application Server Application Databases
- 10. Application Servers Application Databases See Nodes GrowSee Nodes Grow
- 11. Application Servers Application Databases Load Balancer See Nodes GrowSee Nodes Grow
- 12. See Nodes GrowSee Nodes Grow Application Servers Application Databases Load Balancers
- 13. See Nodes GrowSee Nodes Grow Application Servers Application Database Cache Load Balancers Application Databases
- 14. Tied together with ConfigTied together with Config Application Servers Application Database Cache Load Balancers Application Databases
- 15. Infrastructure is a SnowflakeInfrastructure is a Snowflake Application Servers Application Database Cache Load Balancers Floating IP? Application Databases
- 16. Evolving ComplexityEvolving Complexity Load Balancers Application Servers NoSQL Database Slaves ApplicationCache Database Cache Database
- 17. Complexity Grows QuicklyComplexity Grows Quickly
- 19. Golden Images are not the answerGolden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
- 20. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite Typical InfrastructureTypical Infrastructure
- 21. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Move SSH off port 22 • Lets put it on 2022 New Compliance Mandate!New Compliance Mandate!
- 22. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6 6 Golden Image Updates6 Golden Image Updates
- 23. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually 12 Instance Replacements12 Instance Replacements
- 24. • Don’t break anything! • Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite 1 2 4 5 6 7 8 9 10 11 12 3 Done in Maintenance WindowsDone in Maintenance Windows
- 25. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Invalid configs! Different IP Addresses?Different IP Addresses?
- 26. Configuration DesperationConfiguration Desperation Code Sample http://www.flickr.com/photos/francoforeshock/5716969942
- 27. • But you already guessed that, didn’t you? Chef Solves this ProblemChef Solves this Problem
- 28. Chef is Infrastructure as CodeChef is Infrastructure as Code http://www.flickr.com/photos/louisb/4555295187/ • Programmatically provision and configure • Treat like any other code base • Reconstruct business from code repository, data backup, and bare metal resources.
- 29. • Chef-Client generates configurations directly on nodes from their run list • Reduce management complexity through abstraction • Store the configuration of your programs in version control http://www.flickr.com/photos/ssoosay/5126146763/ NodesNodes
- 30. Collections of ResourcesCollections of Resources • Networking • Files • Directories • Symlinks • Mounts • Routes • Users • Groups • Tasks • Packages • Software • Services • Configurations • Other Stuff http://www.flickr.com/photos/stevekeys/3123167585/
- 31. Declarative Interface to ResourcesDeclarative Interface to Resources • Define policy • Say what, not how • Pull not Push http://www.flickr.com/photos/bixentro/2591838509/
- 32. Ruby!Ruby! extra_packages = case node['platform'] when "ubuntu","debian" %w{ ruby1.8 ruby1.8-dev rdoc1.8 ri1.8 libopenssl-ruby } end extra_packages.each do |pkg| package pkg do action :install end end
- 33. chef-client runs on your systems (nodes) chef abstracts nodes into roles to help you scale efficiently chef environments help you to manage the lifecycle cookbooks are packages for chef and run lists attach them to nodes everything is indexed for search knife is your command line to control chef chef-client runs on your systems (nodes) chef abstracts nodes into roles to help you scale efficiently chef environments help you to manage the lifecycle cookbooks are packages for chef and run lists attach them to nodes everything is indexed for search knife is your command line to control chef Key ConceptsKey Concepts
- 34. Recipes and CookbooksRecipes and Cookbooks •Recipes are collections of Resources •Cookbooks contain recipes, templates, files, custom resources, etc •Code re-use and modularity •Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
- 35. Recipes and CookbooksRecipes and Cookbooks •Recipes are collections of Resources •Cookbooks contain recipes, templates, files, custom resources, etc •Code re-use and modularity •Hundreds already on Community.opscode.com http://www.flickr.com/photos/shutterhacks/4474421855/
- 38. http://www.flickr.com/photos/kathycsus/2686772625 • IP addresses • Hostnames • FQDNs • Search for when static config isn’t enough • data-driven power SearchSearch
- 39. pool_members = search("node","role:webserver”) template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]"end Pass Results to TemplatesPass Results to Templates
- 40. # Set up application listeners here.listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002 mode http stats uri /<% end -%> Pass Results to TemplatesPass Results to Templates
- 41. Jboss App Memcache Postgres Slaves Postgres Master So when thisSo when this NagiosGraphite
- 42. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite Becomes thisBecomes this
- 43. Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite Updates can be automaticUpdates can be automatic
- 44. NagiosGraphite Count the resourcesCount the resources Jboss App Memcache Postgres Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • 12+ resource changes for 1 node addition
- 45. Build anythingBuild anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/
- 46. And manage it simplyAnd manage it simply http://www.flickr.com/photos/helico/404640681/ • Automatically reconfigure everything • Linux,Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial
- 47. Code Sample Landscape of Chef-managed InfrastructureLandscape of Chef-managed Infrastructure
- 48. knifeknife
- 49. Code Sample Knife is the command-line tool used by ChefsKnife is the command-line tool used by Chefs
- 50. knife with the Chef Serverknife with the Chef Server • knife node • create/delete/edit • list • knife cookbook ... • knife role ... • knife environment ...
- 51. knife searchknife search • What operating systems are running? • What version of ruby is running? • How much memory do you have on each machine?
- 52. Ohai!Ohai!
- 53. "memory": { "swap": { "cached": "0kB", "total": "4128760kB", "free": "4128760kB" }, "total": "2055676kB", "free": "1646524kB", "buffers": "35032kB", "cached": "210276kB", "active": "125336kB", "inactive": "142884kB", "dirty": "8kB", "writeback": "0kB", "anon_pages": "22976kB", "mapped": "8416kB", "slab": "121512kB", "slab_reclaimable": "41148kB", "slab_unreclaim": "80364kB", "page_tables": "1784kB", "nfs_unstable": "0kB", "bounce": "0kB", "commit_limit": "5156596kB", "committed_as": "74980kB", "vmalloc_total": "34359738367kB", "vmalloc_used": "274512kB", "vmalloc_chunk": "34359449936kB" }, Ohai!Ohai! "block_device": { "ram0": { "size": "32768", "removable": "0" }, "ram1": { "size": "32768", "removable": "0" }, "ram2": { "size": "32768", "removable": "0" }, "hostname": "server-1", "fqdn": "server- 1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, "fe80::20c:29ff:fe43:26c5": { "family": "inet6", "prefixlen": "64", "scope": "Link" } },
- 55. knife searchknife search knife search “*:*” -a platform knife search “*:*” -a languages.ruby.version knife search “*:*” -a memory.total
- 56. knife sshknife ssh• $ knife ssh "roles:rails-web" "sudo chef-client"
- 57. knife bootstrapknife bootstrap knife bootstrap SERVER -r 'role[webserver]' -i ~/.ssh/id_rsa • SSH to the machine given existing credentials • Install the Chef Client • Register with the Chef Server • Run the initial Run List • Now managed with Chef!
- 58. knife ec2knife ec2 $ knife ec2 Available ec2 subcommands: (for details, knife SUB-COMMAND --help) ** EC2 COMMANDS ** knife ec2 flavor list (options) knife ec2 instance data (options) knife ec2 server create (options) knife ec2 server delete SERVER [SERVER] (options) knife ec2 server list (options) $ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I ami-4721882e -f m1.small -r 'role[webserver]'
- 59. knife openstackknife openstack $ knife openstack Available openstack subcommands: (for details, knife SUB-COMMAND --help) ** OPENSTACK COMMANDS ** knife openstack flavor list (options) knife openstack image list (options) knife openstack server create (options) knife openstack server delete SERVER [SERVER] (options) knife openstack server list (options) $ knife openstack server create -S keypair -i ~/.ssh/id_rsa -x ubuntu -I 1231 -f standard.small -r 'role[webserver]'
- 60. Chef for Infrastructure PortabilityChef for Infrastructure Portability • knife ec2 • knife rackspace • knife hp • knife google • knife azure • knife cloudstack • knife openstack • knife vsphere • ... and many others
- 62. The Chef CommunityThe Chef Community • Apache License, Version 2.0 • 1200+ Individual contributors • 200+ Corporate contributors • Google, HP, Dell, Rackspace, VMware, Joyent, Calxeda, Heroku, SUSE and many more • 800+ cookbooks • http://community.opscode.com
- 63. How Do I StartHow Do I Start
- 65. Your first projectYour first project • Identify the right project • a fragile artifact?, something that causes you issues, a new development? • be prepared to think outside the box • challenge the way you do things • identify what success is (and measure) • faster, better, lower effort .....
- 66. It’s all about peopleIt’s all about people • Find a sponsor • Assemble the right team; cross- department, compatible skill-sets? • Make change but measure the outcomes • Collaborate; the job is not done until it is done • Learn and improve
- 69. Yep, we’re hiring!Yep, we’re hiring!
- 70. Thanks! Any Questions?Thanks! Any Questions? Andy Hawkins andy@opscode.com