Infrastructure as Code with Chef / Puppet

2,640 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,640
On SlideShare
0
From Embeds
0
Number of Embeds
320
Actions
Shares
0
Downloads
84
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Infrastructure as Code with Chef / Puppet

  1. 1. Infrastructure as Code with Chef / Puppet Edmund Haselwanter (e.haselwanter@cloudbau.de) Samstag, 16. November 13
  2. 2. Introduction Edmund Haselwanter ●Co-Founder and CSO cloudbau GmbH ●Dell Professional Services (Openstack/Crowbar/ Automation) ●Software Development Background ●10 Years Infrastructure Automation ●@ehaselwanter on twitter/github/irc/... cloudbau Samstag, 16. November 13
  3. 3. Business-Agility needs IT-Agility For most companies, IT functions as the nervous system and provides an increasing amount of the organizational muscle mass. Most critical business functions are entirely automated within IT, and 95% of all capital projects depend on IT to get done. Today, nearly every business decision will result in at least one IT change. „IT REVOLUTION MANIFESTO“, http://itrevolution.com Theory of Constraints → → E.M. Goldratt, 1984 cloudbau Samstag, 16. November 13 Gene Kim, Kevin Behr, George Spafford, 2013
  4. 4. IT-Agility and the „Risk of Change“ Request Approve Develop Test WaterScrum Deploy - Fall The core, chronic conflict that every IT leader faces is the need to simultaneously enable faster time to market (i.e., make as many changes as you can), while providing stable, secure and reliable IT services (i.e., make as few changes as you can).  Gene Kim „Lowering risk of change through tools and culture“ John Allspaw, Paul Hammond, 2009 small changes, often reproducible Samstag, 16. November 13 cooperative culture process standards cloudbau expect failure metrics
  5. 5. Tools and Culture Config-DB CPUVirt. Binary-Repo StorageVirt. SDN Orchestration Prod Env Programmable Infrastructure Description in Code Automatic Build DevOps Cross-Functional Teams Process and Culture cloudbau Samstag, 16. November 13 Stage Env Anti-Fragile Organizations Continuos Delivery Description Dev Env
  6. 6. Configuration Management cloudbau Samstag, 16. November 13
  7. 7. Evolving towards Configuration Management ●Just build it ●Keep notes in server.txt ●Move notes to the wiki ●Custom scripts (in scm?!) ●Snapshot & Clone cloudbau Samstag, 16. November 13
  8. 8. Applications http://www.flickr.com/photos/steffenz/ 337700069/ http://www.flickr.com/photos/kky/ 704056791/ cloudbau Samstag, 16. November 13
  9. 9. Infrastructure cloudbau http://www.flickr.com/photos/sbh/462754460/ Samstag, 16. November 13
  10. 10. Collection of Resources • • • • • Networking Files Directories Symlinks Mounts • • • • • • • • • cloudbau http://www.flickr.com/photos/philliecasablanca/3354734116/ Samstag, 16. November 13 Routes Users Groups Tasks Packages Software Services Configuration Other Stuff
  11. 11. Acting in Concert cloudbau Samstag, 16. November 13 http://www.flickr.com/photos/glowjangles/4081048126/
  12. 12. To Provide a Service cloudbau Samstag, 16. November 13 http://www.flickr.com/photos/28309157@N08/3743455858/
  13. 13. And it Evolves cloudbau Samstag, 16. November 13 http://www.flickr.com/photos/16339684@N00/2681435235/
  14. 14. See Node Application Server cloudbau Samstag, 16. November 13
  15. 15. See Nodes Application Server Application Database cloudbau Samstag, 16. November 13
  16. 16. See Nodes Grow Application Server Application Databases cloudbau Samstag, 16. November 13
  17. 17. See Nodes Grow Application Servers Application Databases cloudbau Samstag, 16. November 13
  18. 18. See Nodes Grow Load Balancer Application Servers Application Databases cloudbau Samstag, 16. November 13
  19. 19. See Nodes Grow Load Balancers Application Servers Application Databases cloudbau Samstag, 16. November 13
  20. 20. See Nodes Grow Load Balancers Application Servers Application Database Cache Application Databases cloudbau Samstag, 16. November 13
  21. 21. Tied Together with Configuration Load Balancers Application Servers Application Database Cache Application Databases cloudbau Samstag, 16. November 13
  22. 22. So when this Graphite Nagios Jboss App Memcache Postgres Slaves cloudbau Samstag, 16. November 13 Postgres Master
  23. 23. Becomes This Graphite Nagios Jboss App Memcache Postgres Slaves cloudbau Samstag, 16. November 13 Postgres Master
  24. 24. This can happen automagically Graphite Nagios Jboss App Memcache Postgres Slaves cloudbau Samstag, 16. November 13 Postgres Master
  25. 25. Count the Resources Graphite Nagios Jboss App Memcache Postgres Slaves • • • • • • • • • • • • • Load balancer config Nagios host ping Nagios host ssh Nagios host HTTP Nagios host app health Graphite CPU Graphite Memory Graphite Disk Graphite SNMP Memcache firewall Postgres firewall Postgres authZ config 12+ resource changes for 1 node addition cloudbau Samstag, 16. November 13
  26. 26. Focus Today: Chef and Puppet Series A: 2,5 M$ (2009) Series B: 11 M$ (2010) Series C: 19,5 M$ (2012) Technologie cloudbau Samstag, 16. November 13 Series A: 2 M$ (2009) Series B: 5 M$ (2010) Series C: 8,5 M$ (2011) Series D: 30 M$ (2013) Technologie *alle Angaben von www.crunchbase.com
  27. 27. Chef & Puppet since 2005 since 2009 Declarative DSL Ruby DSL Customers: Customers: • CERN • Facebook • Zynga • SAP •VMware • IBM (Smartcloud) • Samstag, 16. November 13 AWS (OpsWorks) • Twitter cloudbau • Microsoft
  28. 28. What is Chef? Chef is an IT automation platform for developers & systems engineers to continuously define, build, and manage infrastructure. CHEF USES: Recipes and Cookbooks that describe and deliver code. Chef enables people to easily build & manage complex & dynamic applications at massive scale. •  New model for describing infrastructure that promotes reuse •  Programmatically provision and configure •  Reconstruct business from code repository, data backup, and bare metal resources cloudbau Samstag, 16. November 13 “! ”!
  29. 29. cloudbau Samstag, 16. November 13
  30. 30. cloudbau Samstag, 16. November 13
  31. 31. What is Puppet? Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. How Puppet Works Puppet uses a declarative, model-based approach to IT automation. 1. Define the desired state of the infrastructure's configuration using Puppet's declarative configuration language. 2. Simulate configuration changes before enforcing them. 3. Enforce the deployed desired state automatically, correcting any configuration drift. 4. Report on the differences between actual and desired states and any changes made enforcing the desired state. cloudbau Samstag, 16. November 13
  32. 32. cloudbau Samstag, 16. November 13
  33. 33. cloudbau Samstag, 16. November 13
  34. 34. Concepts cloudbau Samstag, 16. November 13
  35. 35. Declarative Language Package A Template 1 File 1 Package B cloudbau Samstag, 16. November 13 Service I
  36. 36. Convergence to desired State State X State Y “run” Package A Package A Template 1 File 1 File 1 Package B cloudbau Samstag, 16. November 13 Service I
  37. 37. Idempotence State Y State Y “run” Package A Template 1 File 1 Package B cloudbau Samstag, 16. November 13 Service I Package A Template 1 File 1 Package B Service I
  38. 38. The Implementations Puppet and Chef cloudbau Samstag, 16. November 13
  39. 39. Terminology Java Puppet Chef * .java manifests * .pp recipes * .rb package module cookbook cloudbau Samstag, 16. November 13
  40. 40. Terminology Java Puppet Chef singleton parametrized classes recipes with attributes defines, classes definitions, libraries class cloudbau Samstag, 16. November 13
  41. 41. Syntax Puppet Chef Limited DSL (by design) DSL + full ruby Extended by Definitions Custom Resource providers ERB & Custom Libraries cloudbau Samstag, 16. November 13 Definitions, Resource/Providers, Light Weight Resource Definitions, Resource/Providers, Light Weight Resource Providers, Libraries
  42. 42. Syntax Examples Puppet class myapache { package “apache2” service “apache2”: ensure => “running”, require => Package[“apache2”] } cloudbau Samstag, 16. November 13 Chef package “apache2” service “apache2 do action [:enable , :start] end
  43. 43. Variables Puppet Chef #special syntax #native ruby $bla = “aa” bla = “aa” #string interpolation #string interpolation $bla_string = “${bla}” bla_string = #{bla} cloudbau Samstag, 16. November 13
  44. 44. Loops Puppet Chef pass array of elements erb template for loops create_resource (:type, hash) cloudbau Samstag, 16. November 13 do while , repeat etc. available through native ruby
  45. 45. Conditionals Puppet (since 2.7) Chef if case (execute block) selectors (get value) cloudbau Samstag, 16. November 13 if, unless, case etc. available through native ruby
  46. 46. Project Structure Puppet Modules Chef attributes/ definitions/ files/ lib/ files/ libraries/ README.rdoc metadata.rb manifests/ providers/ templates/ README.rdoc recipes/ resources/ cloudbau Samstag, 16. November 13 templates/
  47. 47. Execution Model Puppet Chef Compile catalog (directed graph model) Then Execute Top Down execution only code for state transition is visited every line of code is visited all the time cloudbau Samstag, 16. November 13
  48. 48. Execution Model Puppet Chef standalone standalone client - server cloudbau Samstag, 16. November 13 client - server hosted
  49. 49. System Information Puppet/Factor => Flat Key/Value Chef/Ohai => Nested Hash "hostname": "server-1", "fqdn": "server-1.example.com", "domain": "example.com", "network": { "interfaces": { "eth0": { "type": "eth", "number": "0", "encapsulation": "Ethernet", "addresses": { "00:0C:29:43:26:C5": { "family": "lladdr" }, "192.168.177.138": { "family": "inet", "broadcast": "192.168.177.255", "netmask": "255.255.255.0" }, cloudbau Samstag, 16. November 13
  50. 50. System Information ●Selects Package Provider ●Selects Service Provider ●.... cloudbau Samstag, 16. November 13
  51. 51. System Information (Chef Example) execute "load sysctl" do command "/sbin/sysctl -p" action :nothing end bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3, pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048 # adjust shared memory and semaphores template "/etc/sysctl.conf" do source "sysctl.conf.erb" variables( :shmmax_in_bytes => bytes, :shmall_in_pages => pages ) notifies :run, "execute[load sysctl]", :immediately end cloudbau Samstag, 16. November 13
  52. 52. Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> cloudbau Samstag, 16. November 13
  53. 53. Lint Tools http://acrmp.github.io/foodcritic/ http://puppet-lint.com/ cloudbau Samstag, 16. November 13
  54. 54. Dependency Management https://github.com/applicationsonline/librarian-chef http://berkshelf.com http://librarian-puppet.com cloudbau Samstag, 16. November 13
  55. 55. Reports cloudbau Samstag, 16. November 13
  56. 56. Web Interface cloudbau Samstag, 16. November 13
  57. 57. Tools ●Razor for bare metal provisioning ●mcollectiv for remote execution/ orchestration ●test-kitchen for infrastructure testing ●vagrant for rapid development ● .... cloudbau Samstag, 16. November 13
  58. 58. Networking ●Switch Config ●Driven by SDN Adoption ●Lots of Vendors (Arista ..) cloudbau Samstag, 16. November 13
  59. 59. Community cloudbau Samstag, 16. November 13
  60. 60. Chef Community Community Overview! 25,000+ Community Members! ! 1,000+ Community " Cookbooks! ! 250,000+ Cookbook Downloads! ! 400+ Public Training Attendees in the last year! ! 30+ Meetup Groups! ! cloudbau Samstag, 16. November 13 Over 200 Corporate Contributors
  61. 61. http://community.opscode.com/cookbooks approx 1200 co0kbooks 134 maintained by Opscode cloudbau Samstag, 16. November 13
  62. 62. http://forge.puppetlabs.com approx 1700 modules 72 maintained by Puppetlabs cloudbau Samstag, 16. November 13
  63. 63. The Price Tag cloudbau Samstag, 16. November 13
  64. 64. Puppet: How to Buy Cumulative # of Per Node Per Node Premium Nodes Standard Support Support* 1-10 Download FREE NA 11-99 $ 99 Contact Sales 100-249 $ 93 Contact Sales 250-499 $ 88 $ 152 500-999 $ 83 $ 119 1000-2499 $ 79 $ 99 2500+ Contact Sales Contact Sales cloudbau Samstag, 16. November 13 https://puppetlabs.com/puppet/how-to-buy
  65. 65. Puppet Enterprise vs. OpenSource cloudbau Samstag, 16. November 13
  66. 66. Chef: How to Buy Free Launch Free $ 120 $ 300 $ 600 Nodes 5 20 50 100 Standard Support — ✔ ✔ ✔ Price per Month Standard Premium http://www.opscode.com/enterprise-chef/#plans-pricing cloudbau Samstag, 16. November 13
  67. 67. Enterprise Chef vs. OpenSource Chef cloudbau Samstag, 16. November 13
  68. 68. Danke! cloudbau Edmund Haselwanter @ehaselwanter e.haselwanter@cloudbau.de +49 30 57701800 www.cloudbau.de Samstag, 16. November 13 cloudbau GmbH Körnerstr. 7-10 10785 Berlin
  69. 69. Credits: • Patrick Debois (http://www.jedi.be/blog/) for some of his Puppet/Chef Slides • Opscode (http://www.opscode.com) for product info and some slides • Puppetlabs (http://puppetlabs.com) for product info Samstag, 16. November 13

×