Chef - Configuration Management for the Cloud

1,343 views
1,196 views

Published on

Presentation given at PDX Cloud meeting on Chef

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,343
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Chef - Configuration Management for the Cloud

  1. 1. Configuration Management in the Cloudwith ChefPDX CloudJames Caseyjames@opscode.comTwitter: jamesc_000GitHub: jamescwww.opscode.comTuesday, April 30, 13
  2. 2. • Instant infrastructure• Unlimited capacity• Autoscaling• No commitment• Immediate replacementClouds are great !Tuesday, April 30, 13
  3. 3. APIs are awesome!• You can provisioncompute resources inseconds• You can provisionstorage resources inseconds• You can provisionnetwork resources inseconds• That’s cool.http://www.flickr.com/photos/jdhancock/3634246981/Tuesday, April 30, 13
  4. 4. The Dark Sideof the CloudTuesday, April 30, 13
  5. 5. • Performance• Security• Price• ReliabilityWhy not the Cloud?Tuesday, April 30, 13
  6. 6. See NodeApplication ServerTuesday, April 30, 13
  7. 7. See NodesApplication ServerApplication DatabaseTuesday, April 30, 13
  8. 8. See Nodes GrowApplication ServerApplication DatabasesTuesday, April 30, 13
  9. 9. Application ServersApplication DatabasesSee Nodes GrowTuesday, April 30, 13
  10. 10. Application ServersApplication DatabasesLoad BalancerSee Nodes GrowTuesday, April 30, 13
  11. 11. See Nodes GrowApplication ServersApplication DatabasesLoad BalancersTuesday, April 30, 13
  12. 12. See Nodes GrowApplication ServersApplication Database CacheLoad BalancersApplication DatabasesTuesday, April 30, 13
  13. 13. Tied together with ConfigApplication ServersApplication Database CacheLoad BalancersApplication DatabasesTuesday, April 30, 13
  14. 14. Infrastructure is a SnowflakeApplication ServersApplication Database CacheLoad BalancersFloating IP?Application DatabasesTuesday, April 30, 13
  15. 15. Evolving ComplexityLoad BalancersApplication ServersNoSQLDatabase SlavesApplicationCacheDatabase CacheDatabaseTuesday, April 30, 13
  16. 16. Complexity Grows QuicklyDC1DC3DC2Tuesday, April 30, 13
  17. 17. http://www.flickr.com/photos/16339684@N00/2681435235/And it Continues to EvolveTuesday, April 30, 13
  18. 18. http://www.flickr.com/photos/16339684@N00/2681435235/And it Continues to EvolveOk, so I’ve got a problem.What’s the solution ?Tuesday, April 30, 13
  19. 19. Golden Images are not the answer• Gold is heavy• Hard to transport• Hard to mold• Easy to loseconfiguration detailhttp://www.flickr.com/photos/garysoup/2977173063/Tuesday, April 30, 13
  20. 20. Configuration Management andAutomated Systems Integrationis the Answerhttp://www.flickr.com/photos/philliecasablanca/3354734116/Tuesday, April 30, 13
  21. 21. • Turn code and hardware intoinfrastructure• From bare metal to services inproduction• Scale applications as needed• Conform to policy• Align to business goalsConfiguration Management ?Tuesday, April 30, 13
  22. 22. Chef - Infrastructure as Codehttp://www.flickr.com/photos/louisb/4555295187/• Programmaticallyprovision and configure• Treat like any other codebase• Reconstruct business fromcode repository, databackup, and bare metalresources.Tuesday, April 30, 13
  23. 23. • Chef-Client generatesconfigurations directlyon nodes from theirrun list• Reduce managementcomplexity throughabstraction• Store the configurationof your programs inversion controlhttp://www.flickr.com/photos/ssoosay/5126146763/NodesTuesday, April 30, 13
  24. 24. Collections of Resources• Networking• Files• Directories• Symlinks• Mounts• Routes• Users• Groups• Tasks• Packages• Software• Services• Configurations• Other Stuffhttp://www.flickr.com/photos/stevekeys/3123167585/Tuesday, April 30, 13
  25. 25. Declarative Interface to Resources• Define policy• Say what, not how• Pull not Pushhttp://www.flickr.com/photos/bixentro/2591838509/Tuesday, April 30, 13
  26. 26. Ruby!extra_packages = case node[platform]when "ubuntu","debian"%w{ruby1.8ruby1.8-devrdoc1.8ri1.8libopenssl-ruby}endextra_packages.each do |pkg|package pkg doaction :installendendTuesday, April 30, 13
  27. 27. Or thissearch(:users, *:*) do |u|user u[id] douid u[uid]shell u[shell]home "/home/#{u[id]}"enddirectory "#{home_dir}/.ssh" doowner u[id]group u[gid]mode "0700"endtemplate "#{home_dir}/.ssh/authorized_keys" dosource "authorized_keys.erb"owner u[id]group u[id]mode "0600"variables :ssh_keys => u[ssh_keys]endendTuesday, April 30, 13
  28. 28. Recipes and Cookbooks• Recipes are collections ofResources• Cookbooks containrecipes, templates, files,custom resources, etc• Code re-use andmodularity• Hundreds already onCommunity.opscode.comhttp://www.flickr.com/photos/shutterhacks/4474421855/Tuesday, April 30, 13
  29. 29. http://www.flickr.com/photos/kathycsus/2686772625• IP addresses• Hostnames• FQDNs• Search for nodeswith Roles• Find configurationdataSearchTuesday, April 30, 13
  30. 30. pool_members = search("node","role:webserver”)template "/etc/haproxy/haproxy.cfg" dosource "haproxy-app_lb.cfg.erb"owner "root"group "root"mode 0644variables :pool_members => pool_members.uniqnotifies :restart, "service[haproxy]"endPass Results to TemplatesTuesday, April 30, 13
  31. 31. # Set up application listeners here.listen application 0.0.0.0:80balance roundrobin<% @pool_members.each do |member| -%>server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check<% end -%><% if node["haproxy"]["enable_admin"] -%>listen admin 0.0.0.0:22002mode httpstats uri /<% end -%>Pass Results to TemplatesTuesday, April 30, 13
  32. 32. Jboss AppMemcachePostgres SlavesPostgres MasterSo when thisNagiosGraphiteTuesday, April 30, 13
  33. 33. Jboss AppMemcachePostgres SlavesPostgres MasterNagiosGraphiteBecomes thisTuesday, April 30, 13
  34. 34. Jboss AppMemcachePostgres SlavesPostgres MasterNagiosGraphiteUpdates can be automaticTuesday, April 30, 13
  35. 35. NagiosGraphiteCount the resourcesJboss AppMemcachePostgres Slaves• Load balancer config• Nagios host ping• Nagios host ssh• Nagios host HTTP• Nagios host app health• Graphite CPU• Graphite Memory• Graphite Disk• Graphite SNMP• Memcache firewall• Postgres firewall• Postgres authZ config• 12+ resource changes for 1 node additionTuesday, April 30, 13
  36. 36. http://www.flickr.com/photos/evelynishere/2798236471/CLONING CANNOT COPE WITH THIS• Chef can.Tuesday, April 30, 13
  37. 37. Build anything• Simple internal applications• Complex external applications• Workstations• Hadoop clusters• IaaS infrastructure• PaaS infrastructure• SaaS applications• Storage systems• You name ithttp://www.flickr.com/photos/hyku/245010680/Tuesday, April 30, 13
  38. 38. And manage it simplyhttp://www.flickr.com/photos/helico/404640681/• Automaticallyreconfigureeverything• Linux,Windows,Unixes, BSDs• Load balancers• Metrics collectionsystems• Monitoring systems• Cloud migrationsbecome trivialTuesday, April 30, 13
  39. 39. knifeTuesday, April 30, 13
  40. 40. Upload your infrastructureknife cookbook upload aptknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload jpackageknife cookbook upload ntpknife cookbook upload sudoknife cookbook upload tomcatknife cookbook upload usersknife cookbook upload sampleknife role from file base-cloud.rbknife role from file tc.rbknife role from file sample.rbknife data bag create usersknife data bag from file users jamesc.jsonTuesday, April 30, 13
  41. 41. Build it somewhere#EC2knife ec2 server create -S jamesc -i ~/.ssh/jamesc.pem -x ubuntu -G default -I ami-a7a97dce -f m1.small -d omnibus -r role[base-cloud],role[tc],role[sample]#Rackspaceknife rackspace server create --image 110 --flavor 2 -i ~/.ssh/jamesc.pem -d omnibus -rrole[base-cloud],role[tc],role[sample]#CloudStackknife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i~/.ssh/jamesc.pem -d omnibus -r role[base-cloud],role[tc],role[sample]#Ubuntu Linuxknife bootstrap test.lab -r role[webserver] -i ~/.ssh/jamesc.pem -x ubuntu --sudo -domnibus -r role[base-cloud],role[tc],role[sample]Tuesday, April 30, 13
  42. 42. knife ec2$ knife ec2Available ec2 subcommands: (for details, knife SUB-COMMAND --help)** EC2 COMMANDS **knife ec2 flavor list (options)knife ec2 instance data (options)knife ec2 server create (options)knife ec2 server delete SERVER [SERVER] (options)knife ec2 server list (options)$ knife ec2 server create -S keypair -i ~/.ssh/id_rsa -x ubuntu-I ami-4721882e -f m1.small -r role[webserver]Tuesday, April 30, 13
  43. 43. knife openstack$ knife openstackAvailable openstack subcommands: (for details, knife SUB-COMMAND --help)** OPENSTACK COMMANDS **knife openstack flavor list (options)knife openstack image list (options)knife openstack server create (options)knife openstack server delete SERVER [SERVER] (options)knife openstack server list (options)$ knife openstack server create -S keypair -i ~/.ssh/id_rsa-x ubuntu -I 1231 -f standard.small -r role[webserver]Tuesday, April 30, 13
  44. 44. Chef for Infrastructure Portability• knife ec2• knife rackspace• knife hp• knife google• knife azure• knife cloudstack• knife openstack• knife vsphere• ... and manyothersTuesday, April 30, 13
  45. 45. The Chef Community• Apache License,Version 2.0• 850+ Individual contributors• 150+ Corporate contributors• HP, Dell, Rackspace,VMware, Joyent,Calxeda, Heroku, SUSE and many more• 550+ cookbooks• http://community.opscode.comTuesday, April 30, 13
  46. 46. Summary• Every infrastructure is a unique snowflake• You need tools to let you do what youwant• You need the power to grow yourinfrastructure• You need the ability to change yourcloud provider• Automated Configuration Management isthe solutionTuesday, April 30, 13
  47. 47. Questions?http://www.flickr.com/photos/mrchippy/443960682/Questions?Tuesday, April 30, 13
  48. 48. Thanks!James Caseyjames@opscode.comTwitter: jamesc_000Github: jamescwww.opscode.comTuesday, April 30, 13

×