Infrastructure as Code   Patrick Debois
                                                        http://www.tumblr.com/tagged/star-trek-quotes

Wednesday 17 October 12
Organizer Since 2009        Europe Organizer 2010     First Europe Training 2010            Speaker




        Monitoring Chapter           Technical Reviewer               Co-Author           Veewee / Sahara / Mccloud




             Libvirt - Fog    Freelance consultant         Engineer                       http://github.com/jedi4ever



       Blog: http://jedi.be/blog - Twitter: @patrickdebois - #devops
Wednesday 17 October 12
http://cfengine.com/                                                          http://ansible.cc/
                http://puppetlabs.com/                  http://palletops.com/
                                     http://opscode.com/chef/




                    CONFIGURATION
                  MANAGEMENT SYSTEMS
Wednesday 17 October 12
Chef & Puppet focused
                          based on my experience

                                                   http://www.nerdnirvana.org/wp-content/uploads/2010/12/apology-form.jpg

Wednesday 17 October 12
GIVE ME THE CODE:
                             “LANGUAGE”



Wednesday 17 October 12
EXAMPLE SYNTAX
                          Puppet                             Chef

    class myapache {                        package “apache2”

           package “apache2”                service “apache2 do
                                              action [:enable , :start]
           service “apache2”:               end
            ensure => “running”,
            require => Package[“apache2”]

    }




Wednesday 17 October 12
TERMINOLOGY


                           Java      Puppet       Chef


                                    manifests    recipes
                           *.java
                                      *.pp         *.rb


                          package    module     cookbook


Wednesday 17 October 12
TERMINOLOGY (2)


                          Java      Puppet          Chef


                                  parametrized   recipes with
                     singleton
                                     classes      attributes

                                    defines,      definitions,
                          class
                                    classes       libraries


Wednesday 17 October 12
SYNTAX
                          Puppet             Chef
                      Limited DSL
                                        DSL + full ruby
                       (by design)
                      Extended by        Light Weight
                       Definitions     Resource Providers /
                    Custom Resource       Definitions
                                        Providers, Light
                        providers      Weight Resource
                     ERB & Custom          Providers
                                           Libraries
                        Libraries


Wednesday 17 October 12
LOOPING CONSTRUCTS

                          Puppet             Chef



             pass array of elements
             erb template for loops do while , repeat etc.
                create_resource      available through
                  (:type, hash)         native ruby




Wednesday 17 October 12
VARIABLES

                             Puppet               Chef



                          special syntax       native ruby
                           $bla = “aa”          bla = “aa”

                string interpolation       string interpolation
               $bla_string = “${bla}”       bla_string = #{bla}


Wednesday 17 October 12
SINGLE CLASS
                               INHERITANCE
                            Puppet             Chef


                override via ‘module        override via
                       path’              ‘cookbook’ path


                          +> operator   “cookbook” attribute



Wednesday 17 October 12
PROJECT STRUCTURE
                          Puppet Module      Chef Cookbook

                                          attributes/
                                          definitions/
                                          files/
                              files/
                                          libraries/
                               lib/
                                          metadata.rb
                           README.rdoc
                                          providers/
                            manifests/
                                          README.rdoc
                            templates/
                                          recipes/
                                          resources/
                                          templates/


Wednesday 17 October 12
EXECUTION MODEL

                              Puppet              Chef



                          Compile catalog
                          (directed graph
                                            Top Down execution
                               model)
                           Then Execute



Wednesday 17 October 12
CODE EDITOR




Wednesday 17 October 12
VIM SYNTAX HIGHLIGHTIING

                              https://github.com/rodjek/vim-puppet
                                  https://github.com/t9md/vim-chef
           https://github.com/vim-scripts/Cfengine-version-3-syntax

Wednesday 17 October 12
GEPPETTO - ECLIPSE IDE




                          https://github.com/cloudsmith/geppetto
                          http://cloudsmith.github.com/geppetto/
Wednesday 17 October 12
WYSIWYG CODING

                             REACTOR8


                                 http://beta.r8network.com/
Wednesday 17 October 12
http://redmine.djagios.org/projects/puppet-lexer
                https://github.com/rodjek/puppet-pygments-lexer
Wednesday 17 October 12
CODE GENERATION




Wednesday 17 October 12
REVERSE ENGINEERING


                            Existing Server -> Manifest/Recipes




                          https://github.com/devstructure/blueprint
Wednesday 17 October 12
DSL ON TOP OF PUPPET
                         ~GRAILS


                                                 Note: Not vastly used
                          https://github.com/jnewland/shadow_puppet
Wednesday 17 October 12
CODING STYLE




Wednesday 17 October 12
Puppet                                                                           Chef

               http://puppet-lint.com/                                               http://acrmp.github.com/
                                                                                             foodcritic/
                                                                                 •   FC001: Use strings in preference to symbols to access node attributes
    •   Must use two-space soft tabs.                                            •   FC002: Avoid string interpolation where not required
    •   Must not use literal tab characters.                                     •   FC003: Check whether you are running with chef server before using server-
                                                                                     specific features
    •   Must not contain trailing white space.
                                                                                 •   FC004: Use a service resource to start and stop services
    •   Should not exceed an 80 character line width                             •   FC005: Avoid repetition of resource declarations
        An exception has been made for source	
  =>	
  'puppet://...' lines as   •   FC006: Mode should be quoted or fully specified when setting file
                                                                                     permissions
        splitting these over multiple lines decreases the readability of the
                                                                                 •   FC007: Ensure recipe dependencies are reflected in cookbook metadata
        manifests.                                                               •   FC008: Generated cookbook metadata needs updating
    •   Should align arrows (=>) within blocks of attributes.                    •   FC009: Resource attribute not recognised
                                                                                 •   FC010: Invalid search syntax
    •   ....
                                                                                 •   FC011: Missing README in markdown format
                                                                                 •   FC012: Use Markdown for README rather than RDoc
    •                                                                            •   FC013: Use file_cache_path rather than hard-coding tmp paths
                                                                                 •




                 https://github.com/ampledata/cookbook-style-guide
                http://docs.puppetlabs.com/guides/style_guide.html
Wednesday 17 October 12
http://www.slideshare.net/PuppetLabs/modern-module-
        development-ken-barber-2012-edinburgh-puppet-camp
Wednesday 17 October 12
http://projects.puppetlabs.com/projects/hiera
                      https://github.com/crayfishx/hiera-gpg




                           CODE VS CONFIG
                          (THINK “PROFILES”)



                     http://wiki.opscode.com/display/chef/Data+Bags
Wednesday 17 October 12
“STOP THE FORK(ING)”



                          https://github.com/jedi4ever/stop-the-fork
Wednesday 17 October 12
PRE-COMMIT CHECKS




Wednesday 17 October 12
DEPENDENCY MGMT




             Git
         Submodules                  à la Pom File
           Subtree

Wednesday 17 October 12
UPSTREAM REPOS




                                http://community.opscode.com/
                                   http://forge.puppetlabs.com/


Wednesday 17 October 12
Puppet                                        Chef

         https://github.com/rodjek/    https://github.com/
              librarian-puppet      applicationsonline/librarian

                                                            http://berkshelf.com/

                           site	
  "http://community.opscode.com/api/v1"

                           cookbook	
  "ntp"
                           cookbook	
  "timezone",	
  "0.0.1"

                           cookbook	
  "rvm",
                           	
  	
  :git	
  =>	
  "https://github.com/fnichol/chef-­‐rvm",
                           	
  	
  :ref	
  =>	
  "v0.7.1"

                           cookbook	
  "cloudera",
                           	
  	
  :path	
  =>	
  "vendor/cookbooks/cloudera-­‐cookbook"


Wednesday 17 October 12
CHEF METADATA




Wednesday 17 October 12
Puppet Dependency Graphs




             http://bitfieldconsulting.com/puppet-dependency-graphs
Wednesday 17 October 12
http://wham.cloudsmith.com
Wednesday 17 October 12
DEBUGGING




Wednesday 17 October 12
LOGGING WITH
                          DIFFERENT LEVELS
        Info only, no exception
        Chef::Log.info('Some	
  useful	
  info')

        Fatal, raising exception
        Chef::Log.fatal('Something	
  bad')
Wednesday 17 October 12
REPL - STYLE
                          (READ EVAL PRINT)


                                           https://github.com/pry/pry
                             http://wiki.opscode.com/display/chef/Shef
Wednesday 17 October 12
https://github.com/vimeo/simple-black-box
Wednesday 17 October 12
DEV ENVIRONMENT




Wednesday 17 October 12
Virtualbox + (Fusion Soon)
   http://vagrantup.com/




    Virtualbox, Fusion,
      KVM,Parallels,        https://github.com/jedi4ever/veewee/
Wednesday 17 October 12
https://github.com/jedi4ever/mccloud/
                           EC2, KVM, FUSION, HOSTS, Vagrant




                                     Vagrant alike




                                            EC2
                              https://github.com/rtyler/blimpy
Wednesday 17 October 12
UNIT TESTING




Wednesday 17 October 12
https://github.com/nistude/cucumber-puppet
Wednesday 17 October 12
https://github.com/rodjek/rspec-puppet
                              https://github.com/acrmp/chefspec
                           https://github.com/calavera/rspec-chef
Wednesday 17 October 12
Wednesday 17 October 12
MOCKING




Wednesday 17 October 12
https://github.com/customink/fauxhai
Wednesday 17 October 12
AUTO-TRIGGER TESTS RUN

                               https://github.com/guard/guard-rspec
                          https://github.com/johnbintz/guard-puppet
                          https://github.com/alister/guard-puppet-lint
                                  https://github.com/guard/guard-chef
Wednesday 17 October 12
http://www.cucumber-chef.org/         EC2+LXC




                          Faster testing via LXC




       Vagrant+LXC                https://github.com/exceedhl/toft
Wednesday 17 October 12
CI INTEGRATION




Wednesday 17 October 12
https://github.com/nicksieger/ci_reporter
Wednesday 17 October 12
https://github.com/rtyler/vagrant-plugin
Wednesday 17 October 12
https://travis-ci.org/
Wednesday 17 October 12
CROSS OS-TESTING



                                   https://github.com/opscode/bento
                             https://github.com/opscode/test-kitchen
Wednesday 17 October 12
Windows




  https://github.com/tknerr/bills-kitchen
Wednesday 17 October 12
INTEGRATION TESTING
                      (ON REAL MACHINE)



Wednesday 17 October 12
http://auxesis.github.com/cucumber-nagios/
Wednesday 17 October 12
RE-USABLE CUCUMBER STEPS


                          http://www.cucumber-chef.org/
                          https://github.com/hedgehog/cuken
                          https://github.com/cucumber/aruba
Wednesday 17 October 12
Validate “inside” the provision-run cycle
                          https://github.com/calavera/minitest-chef-handler
                              https://github.com/jedi4ever/puppet-assert
Wednesday 17 October 12
PERFORMANCE/METRICS




Wednesday 17 October 12
https://github.com/rodjek/puppet-profiler
Wednesday 17 October 12
execution time
                           number of classes
         https://github.com/krux/puppet-module-graphite-report
                  https://github.com/imeyer/chef-handler-graphite
              https://github.com/joemiller/puppet-graphite_event
Wednesday 17 October 12
TARGETED TESTING
                          IMPACT OF CHANGE


                              https://github.com/jonlives/knife-preflight
                              https://github.com/jedi4ever/puppet-cic
Wednesday 17 October 12
Infrastructure as Code   Patrick Debois
                                                        http://www.tumblr.com/tagged/star-trek-quotes

Wednesday 17 October 12
Wednesday 17 October 12
http://foodfightshow.org/




Wednesday 17 October 12
Wednesday 17 October 12
http://wiki.opscode.com/display/chef/Whyrun+Testing
Wednesday 17 October 12
https://github.com/tbatchelli/vmfest
Wednesday 17 October 12
Wednesday 17 October 12
CACHING - AMI, SNAPSHOTS




Wednesday 17 October 12
https://github.com/lak/puppet-memcached
Wednesday 17 October 12
https://github.com/lak/puppet-static-compiler
Wednesday 17 October 12
http://puppetlabs.com/blog/what-is-user-experience-in-
                               puppet/
Wednesday 17 October 12
AUDITING



                                        http://www.fusioninventory.org/
                          http://www.normation.com/en/solutions/rudder
Wednesday 17 October 12
http://comodit.github.com/synapse-agent/
Wednesday 17 October 12
http://saltstack.org/
Wednesday 17 October 12
https://github.com/tobami/littlechef
Wednesday 17 October 12
https://github.com/dwt/vagrant-hosts
Wednesday 17 October 12
https://github.com/opscode/omnibus
Wednesday 17 October 12
https://github.com/puppetlabs/puppet-module-tool
Wednesday 17 October 12
http://wiki.opscode.com/display/chef/Recipes#Recipes-
                       ExceptionsandLogging
Wednesday 17 October 12
http://projects.puppetlabs.com/projects/puppet/wiki/
                                   Anchor_Pattern
Wednesday 17 October 12
VENDORING




Wednesday 17 October 12
https://github.com/railsmachine/rump
Wednesday 17 October 12
http://www.example42.com/?
q=Puppi_A_Puppet_module_for_Deployment_Automation
Wednesday 17 October 12
PLUGINS




Wednesday 17 October 12
http://wiki.opscode.com/display/chef/Knife+Plugins
Wednesday 17 October 12
FACES


 http://puppetlabs.com/blog/puppet-faces-what-the-heck-are-
                            faces/
Wednesday 17 October 12
TESTING




Wednesday 17 October 12
http://rundeck.org/
Wednesday 17 October 12
http://www.sonian.com/cloud-monitoring-sensu/
Wednesday 17 October 12
http://docs.puppetlabs.com/guides/parameterized_classes.html
Wednesday 17 October 12
INHERITANCE




Wednesday 17 October 12
NAMESPACING




Wednesday 17 October 12
https://github.com/rerun/rerun
Wednesday 17 October 12
AUTHENTICATION




Wednesday 17 October 12
DASHBOARDS




Wednesday 17 October 12
http://puppetlabs.com/puppet/related-projects/dashboard/
Wednesday 17 October 12
http://blog.mornati.net/2012/01/29/kermit-a-webui-for-
                              mcollective/
Wednesday 17 October 12
http://www.opscode.com/hosted-chef/
Wednesday 17 October 12
http://theforeman.org/
Wednesday 17 October 12

It's code but not as we know: Infrastructure as Code - Patrick Debois