Code but now as we know it - Devoxx edition

3,843 views

Published on

Comparing config management systems with 'regular' code. Contains a lot of tools and hightlights known and some more obscure existing tools in the space.

Published in: Technology
  • Be the first to comment

Code but now as we know it - Devoxx edition

  1. 1. Infrastructure as Code Patrick Debois http://www.tumblr.com/tagged/star-trek-quotesThursday 15 November 12
  2. 2. Organizer Since 2009 Europe Organizer 2010 First Europe Training 2010 Speaker Monitoring Chapter Technical Reviewer Co-Author Veewee / Sahara / Mccloud Libvirt - Fog Freelance consultant Engineer http://github.com/jedi4ever Blog: http://jedi.be/blog - Twitter: @patrickdebois - #devopsThursday 15 November 12
  3. 3. http://cfengine.com/ http://ansible.cc/ http://puppetlabs.com/ http://palletops.com/ http://opscode.com/chef/ CONFIGURATION MANAGEMENT SYSTEMSThursday 15 November 12
  4. 4. Chef & Puppet focused based on my experience http://www.nerdnirvana.org/wp-content/uploads/2010/12/apology-form.jpgThursday 15 November 12
  5. 5. DECLARATIVE LANGUAGE PACKAGE FILE1 SERVICE FILE2Thursday 15 November 12
  6. 6. “KNOWN STATE” STATE X CODE RUN STATE Y FILE1 PACKAGE FILE1 SERVICE FILE2 convergenceThursday 15 November 12
  7. 7. “IDEMPOTENCE” STATE Y CODE RUN STATE Y PACKAGE PACKAGE FILE1 SERVICE FILE1 SERVICE FILE2 FILE2Thursday 15 November 12
  8. 8. GIVE ME THE CODE: “LANGUAGE”Thursday 15 November 12
  9. 9. EXAMPLE SYNTAX Puppet Chef class myapache { package “apache2” package “apache2” service “apache2 do action [:enable , :start] service “apache2”: end ensure => “running”, require => Package[“apache2”] }Thursday 15 November 12
  10. 10. TERMINOLOGY Java Puppet Chef manifests recipes *.java *.pp *.rb package module cookbookThursday 15 November 12
  11. 11. TERMINOLOGY (2) Java Puppet Chef parametrized recipes with singleton classes attributes defines, definitions, class classes librariesThursday 15 November 12
  12. 12. SYNTAX Puppet Chef Limited DSL DSL + full ruby (by design) Extended by Light Weight Definitions Resource Providers / Custom Resource Definitions Providers, Light providers Weight Resource ERB & Custom Providers Libraries LibrariesThursday 15 November 12
  13. 13. LOOPING CONSTRUCTS Puppet Chef pass array of elements erb template for loops do while , repeat etc. create_resource available through (:type, hash) native rubyThursday 15 November 12
  14. 14. VARIABLES Puppet Chef special syntax native ruby $bla = “aa” bla = “aa” string interpolation string interpolation $bla_string = “${bla}” bla_string = #{bla}Thursday 15 November 12
  15. 15. SINGLE CLASS INHERITANCE Puppet Chef override via ‘module override via path’ ‘cookbook’ path +> operator “cookbook” attributeThursday 15 November 12
  16. 16. PROJECT STRUCTURE Puppet Module Chef Cookbook attributes/ definitions/ files/ files/ libraries/ lib/ metadata.rb README.rdoc providers/ manifests/ README.rdoc templates/ recipes/ resources/ templates/Thursday 15 November 12
  17. 17. EXECUTION MODEL Puppet Chef Compile catalog (directed graph Top Down execution model) Then ExecuteThursday 15 November 12
  18. 18. PLUGINS Puppet Chef KNIFE FACES PLUGINS REPORT REPORTS HANDLERS http://puppetlabs.com/blog/puppet-faces-what-the-heck-are-faces/ http://wiki.opscode.com/display/chef/Knife+PluginsThursday 15 November 12
  19. 19. CODE EDITORThursday 15 November 12
  20. 20. LEXERS http://redmine.djagios.org/projects/puppet-lexer https://github.com/rodjek/puppet-pygments-lexerThursday 15 November 12
  21. 21. VIM SYNTAX HIGHLIGHTIING https://github.com/rodjek/vim-puppet https://github.com/t9md/vim-chef https://github.com/vim-scripts/Cfengine-version-3-syntaxThursday 15 November 12
  22. 22. GEPPETTO - ECLIPSE IDE https://github.com/cloudsmith/geppetto http://cloudsmith.github.com/geppetto/Thursday 15 November 12
  23. 23. COMING SOON JETBRAINS RUBYMINE (PUPPET) http://youtrack.jetbrains.com/issue/RUBY-10832 https://github.com/janschumann/intellij-lang-puppet REACTOR8 http://beta.r8network.com/Thursday 15 November 12
  24. 24. REVERSE ENGINEERING Existing Server -> Manifest/Recipes https://github.com/devstructure/blueprintThursday 15 November 12
  25. 25. CODING STYLEThursday 15 November 12
  26. 26. Puppet Chef http://puppet-lint.com/ http://acrmp.github.com/ foodcritic/ • FC001: Use strings in preference to symbols to access node attributes • Must use two-space soft tabs. • FC002: Avoid string interpolation where not required • Must not use literal tab characters. • FC003: Check whether you are running with chef server before using server- specific features • Must not contain trailing white space. • FC004: Use a service resource to start and stop services • Should not exceed an 80 character line width • FC005: Avoid repetition of resource declarations An exception has been made for source  =>  puppet://... lines as • FC006: Mode should be quoted or fully specified when setting file permissions splitting these over multiple lines decreases the readability of the • FC007: Ensure recipe dependencies are reflected in cookbook metadata manifests. • FC008: Generated cookbook metadata needs updating • Should align arrows (=>) within blocks of attributes. • FC009: Resource attribute not recognised • FC010: Invalid search syntax • .... • FC011: Missing README in markdown format • FC012: Use Markdown for README rather than RDoc • • FC013: Use file_cache_path rather than hard-coding tmp paths • https://github.com/ampledata/cookbook-style-guide http://docs.puppetlabs.com/guides/style_guide.htmlThursday 15 November 12
  27. 27. http://www.slideshare.net/PuppetLabs/modern-module- development-ken-barber-2012-edinburgh-puppet-campThursday 15 November 12
  28. 28. http://projects.puppetlabs.com/projects/hiera https://github.com/crayfishx/hiera-gpg CODE VS CONFIG (THINK “PROFILES”) http://wiki.opscode.com/display/chef/Data+BagsThursday 15 November 12
  29. 29. “STOP THE FORK(ING)”http://devopsanywhere.blogspot.it/2012/11/how-to-write-reusable-chef-cookbooks.htmlhttp://realityforge.org/code/2012/11/12/reusable-cookbooks-revisited.htmlhttps://github.com/jedi4ever/stop-the-forkThursday 15 November 12
  30. 30. PRE-COMMIT CHECKSThursday 15 November 12
  31. 31. DEPENDENCY MGMT Git Submodules à la Pom File SubtreeThursday 15 November 12
  32. 32. UPSTREAM REPOS http://community.opscode.com/ http://forge.puppetlabs.com/Thursday 15 November 12
  33. 33. Puppet Chef https://github.com/rodjek/ https://github.com/ librarian-puppet applicationsonline/librarian http://berkshelf.com/ site  "http://community.opscode.com/api/v1" cookbook  "ntp" cookbook  "timezone",  "0.0.1" cookbook  "rvm",    :git  =>  "https://github.com/fnichol/chef-­‐rvm",    :ref  =>  "v0.7.1" cookbook  "cloudera",    :path  =>  "vendor/cookbooks/cloudera-­‐cookbook"Thursday 15 November 12
  34. 34. CHEF METADATAThursday 15 November 12
  35. 35. Dependency Graphshttp://bitfieldconsulting.com/puppet-dependency-graphs https://github.com/miketheman/knife-role-spaghettiThursday 15 November 12
  36. 36. Online Servicehttp://wham.cloudsmith.comThursday 15 November 12
  37. 37. DEBUGGINGThursday 15 November 12
  38. 38. LOGGING WITH DIFFERENT LEVELS Info only, no exception Chef::Log.info(Some  useful  info) Fatal, raising exception Chef::Log.fatal(Something  bad)Thursday 15 November 12
  39. 39. REPL - STYLE (READ EVAL PRINT) https://github.com/pry/pry http://wiki.opscode.com/display/chef/ShefThursday 15 November 12
  40. 40. https://github.com/vimeo/simple-black-boxThursday 15 November 12
  41. 41. DEV ENVIRONMENTThursday 15 November 12
  42. 42. Virtual Machine (1) Virtual Machine ... N PACKAGE FILE1 SERVICE FILE2 Just Enough Operating System + Config Management Tool virtualization Your laptop (Linux,Thursday 15 November 12 Wind
  43. 43. http://vagrantup.com/ Virtual Machine (1) PACKAGE FILE1 SERVICE FILE2 Just Enough Operating System + Config Management Tool virtualization Virtualbox + (Fusion Soon) Your Laptop Linux & WindowsThursday 15 November 12
  44. 44. USEFUL VAGRANT PLUGINS https://github.com/dotless-de/vagrant-vbguest https://github.com/folken-laeneck/vagrant-bindfs https://github.com/BIAINC/vagrant-windows https://github.com/BerlinVagrant/vagrant-dns https://github.com/dwt/vagrant-hostsThursday 15 November 12
  45. 45. VAGRANT ECLIPSE INTEGRATION http://marketplace.eclipse.org/node/519961#.UKISB-Oe-d4Thursday 15 November 12
  46. 46. Vagrant ‘clones’ EC2, KVM, FUSION, https://github.com/jedi4ever/mccloud/ HOSTS, Vagrant https://github.com/lstoll/tenderloin FUSION https://github.com/neerolyte/vagueant LXC https://github.com/rtyler/blimpy EC2Thursday 15 November 12
  47. 47. VAGRANT - CLOJURE https://github.com/tbatchelli/vmfestThursday 15 November 12
  48. 48. VAGRANT - JAVA https://github.com/guigarage/vagrant-bindingThursday 15 November 12
  49. 49. JEOS CREATION https://github.com/jedi4ever/veewee/ Centos, Ubuntu, Debian, Solaris, Gentoo, Suse, Oracle Linux, .... Just Enough Operating System + Config Management Tool And windows now too! Virtualbox , Fusion virtualization KVM, Parallels Your Laptop Linux & WindowsThursday 15 November 12
  50. 50. CREATING BASEBOX ‘EN MASS’ https://github.com/atkm/origami https://github.com/atkm/seisan-line uses #veewee https://github.com/opscode/bento https://github.com/opscode/test-kitchenThursday 15 November 12
  51. 51. SHARE YOUR SETUP http://www.vagrantbox.es/ http://vagrantfil.es/http://www.morethanseven.net/2011/05/08/Vagrant-plugin-for-interacting-with-vagrantboxes/http://www.nodescription.net/ideation-into-the-void-vagrant-plugin-for-vagThursday 15 November 12
  52. 52. UNIT TESTINGThursday 15 November 12
  53. 53. https://github.com/nistude/cucumber-puppetThursday 15 November 12
  54. 54. https://github.com/rodjek/rspec-puppet https://github.com/acrmp/chefspec https://github.com/calavera/rspec-chefThursday 15 November 12
  55. 55. DON’T TEST YOUR CONFIG MANAGEMENT TEST YOUR LOGIC NOT YES Was the If (OS=’X’ ) and (file package exists) ... installed did it give an error?Thursday 15 November 12
  56. 56. Thursday 15 November 12 FAKE THE FACTS
  57. 57. https://github.com/customink/fauxhaiThursday 15 November 12
  58. 58. CODE AUTO RUN EDIT FEEDBACK AUTO-TRIGGER TESTS RUN https://github.com/guard/guard-rspec https://github.com/johnbintz/guard-puppet https://github.com/alister/guard-puppet-lint https://github.com/guard/guard-chefThursday 15 November 12
  59. 59. LOCAL VM TESTING http://www.cucumber-chef.org/ EC2+LXC Faster testing via LXC spinning up a vm is expensive unless in container Vagrant+LXC https://github.com/exceedhl/toftThursday 15 November 12
  60. 60. CI INTEGRATIONThursday 15 November 12
  61. 61. Junit processor for Ruby https://github.com/nicksieger/ci_reporterThursday 15 November 12
  62. 62. SYNTAX CHECKS UNIT TESTS NO VM COMPILE CHECKS NEEDED SMOKE TESTS REAL VMThursday 15 November 12
  63. 63. STATE A STATE B Create VM CLEAN Code Run Code Run Tests VM Applied SNAPSHOT SMOKE TESTING (ON REAL MACHINE)Thursday 15 November 12
  64. 64. http://auxesis.github.com/cucumber-nagios/Thursday 15 November 12
  65. 65. RE-USABLE CUCUMBER STEPS http://www.cucumber-chef.org/ https://github.com/hedgehog/cuken https://github.com/cucumber/arubaThursday 15 November 12
  66. 66. Validate “inside” the provision-run cycle https://github.com/calavera/minitest-chef-handler https://github.com/jedi4ever/puppet-assertThursday 15 November 12
  67. 67. VM CREATION IS EXPENSIVE STATE A STATE B Create VM CLEAN Code Run Code Run Tests VM Applied SNAPSHOT ROLLBACK https://github.com/t9md/vagrant-snap https://github.com/jedi4ever/saharaThursday 15 November 12
  68. 68. https://github.com/rtyler/vagrant-pluginThursday 15 November 12
  69. 69. Uses vagrant & veewee https://travis-ci.org/Thursday 15 November 12
  70. 70. Windows https://github.com/tknerr/bills-kitchenThursday 15 November 12
  71. 71. TARGETED TESTING IMPACT OF CHANGE MODULEA Real VM testing ROLE1 is expensive MODULEB ROLE2 Only test what’s needed MODULEC ROLEN https://github.com/jonlives/knife-preflight MODULE... https://github.com/jedi4ever/puppet-cicThursday 15 November 12
  72. 72. PERFORMANCE/METRICSThursday 15 November 12
  73. 73. https://github.com/rodjek/puppet-profilerThursday 15 November 12
  74. 74. execution time number of classes https://github.com/krux/puppet-module-graphite-report https://github.com/imeyer/chef-handler-graphite https://github.com/joemiller/puppet-graphite_eventThursday 15 November 12
  75. 75. Infrastructure as Code Patrick Debois http://www.tumblr.com/tagged/star-trek-quotesThursday 15 November 12
  76. 76. Thursday 15 November 12
  77. 77. THANK YOU!Thursday 15 November 12
  78. 78. QUESTIONS?Thursday 15 November 12
  79. 79. http://foodfightshow.org/Thursday 15 November 12
  80. 80. Thursday 15 November 12
  81. 81. Thursday 15 November 12
  82. 82. http://wiki.opscode.com/display/chef/Whyrun+TestingThursday 15 November 12
  83. 83. https://github.com/lak/puppet-memcachedThursday 15 November 12
  84. 84. https://github.com/lak/puppet-static-compilerThursday 15 November 12
  85. 85. http://puppetlabs.com/blog/what-is-user-experience-in- puppet/Thursday 15 November 12
  86. 86. AUDITING http://www.fusioninventory.org/ http://www.normation.com/en/solutions/rudderThursday 15 November 12
  87. 87. http://comodit.github.com/synapse-agent/Thursday 15 November 12
  88. 88. http://saltstack.org/Thursday 15 November 12
  89. 89. https://github.com/tobami/littlechefThursday 15 November 12
  90. 90. https://github.com/dwt/vagrant-hostsThursday 15 November 12
  91. 91. https://github.com/opscode/omnibusThursday 15 November 12
  92. 92. https://github.com/puppetlabs/puppet-module-toolThursday 15 November 12
  93. 93. http://wiki.opscode.com/display/chef/Recipes#Recipes- ExceptionsandLoggingThursday 15 November 12
  94. 94. http://projects.puppetlabs.com/projects/puppet/wiki/ Anchor_PatternThursday 15 November 12
  95. 95. VENDORINGThursday 15 November 12
  96. 96. https://github.com/railsmachine/rumpThursday 15 November 12
  97. 97. http://www.example42.com/?q=Puppi_A_Puppet_module_for_Deployment_AutomationThursday 15 November 12
  98. 98. http://rundeck.org/Thursday 15 November 12
  99. 99. http://www.sonian.com/cloud-monitoring-sensu/Thursday 15 November 12
  100. 100. http://docs.puppetlabs.com/guides/parameterized_classes.htmlThursday 15 November 12
  101. 101. https://github.com/rerun/rerunThursday 15 November 12
  102. 102. AUTHENTICATIONThursday 15 November 12
  103. 103. DASHBOARDSThursday 15 November 12
  104. 104. http://puppetlabs.com/puppet/related-projects/dashboard/Thursday 15 November 12
  105. 105. http://blog.mornati.net/2012/01/29/kermit-a-webui-for- mcollective/Thursday 15 November 12
  106. 106. http://www.opscode.com/hosted-chef/Thursday 15 November 12
  107. 107. http://theforeman.org/Thursday 15 November 12
  108. 108. https://github.com/chapmanb/cloudbiolinuxThursday 15 November 12
  109. 109. https://github.com/aetherical/nimblestratusThursday 15 November 12
  110. 110. https://github.com/fiddyspence/puppet-mconotifyThursday 15 November 12
  111. 111. Thursday 15 November 12

×