My talk at RMLL 2013

1. ;
Testing your puppet codeTesting your puppet code
Julien PivottoJulien Pivotto
Libre Software Meeting 2013Libre Software Meeting 2013
July 15, 2013July 15, 2013

2. ;
Introduction
Testing tools
Jenkins
Conclusion
1 Introduction
Automation
Vagrant
Puppet in a large scale
Puppet code
2 Testing tools
Style and linting
Catalogs
rspec-puppet
3 Jenkins
4 Conclusion
Homework
Conclusion
Julien Pivotto Testing your puppet code

3. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Julien PivottoJulien Pivotto
• sysadmin @ inuitssysadmin @ inuits
• open-source defender for 7+ yearsopen-source defender for 7+ years
• devops believerdevops believer
• @roidelapluie on twitter/github@roidelapluie on twitter/github
Julien Pivotto Testing your puppet code

4. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Infrastructure as Code
• Keep your environments under SCMKeep your environments under SCM
• Overview of complete environmentsOverview of complete environments
• Reduce the deployment timeReduce the deployment time
Julien Pivotto Testing your puppet code

5. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Keep all environments the same
http://www.flickr.com/photos/bobvietnam/4828291896/
Julien Pivotto Testing your puppet code

6. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Packaging with FPM
• Ruby gemRuby gem
• package a directory (and much more)package a directory (and much more)
• Support .deb, .rpmSupport .deb, .rpm
• Package the code with several prefixesPackage the code with several prefixes
• /etc/puppet/environments/infradev/etc/puppet/environments/infradev
• /etc/puppet/environments/uat/etc/puppet/environments/uat
Julien Pivotto Testing your puppet code

7. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Vagrant
• Create virtual machinesCreate virtual machines
• Provision themProvision them
• Destroy & recreateDestroy & recreate
Julien Pivotto Testing your puppet code

8. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Vagrant
• Chef, scripts, puppet, . . .Chef, scripts, puppet, . . .
• Backend: Virtualbox, KVM, . . .Backend: Virtualbox, KVM, . . .
• A lot of baseboxes availableA lot of baseboxes available
• http://vagrantup.comhttp://vagrantup.com
Julien Pivotto Testing your puppet code

9. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Vagrant
• Local testingLocal testing
• The same environment as the targetThe same environment as the target
Julien Pivotto Testing your puppet code

10. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Puppet environments
• Multiple environmentsMultiple environments
• The same tree for all the environmentsThe same tree for all the environments
• Pushing changes to UAT/prod on-demandPushing changes to UAT/prod on-demand
• Small changes vs big releasesSmall changes vs big releases
Julien Pivotto Testing your puppet code

11. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Hiera
• Storing the data in Hiera(-gpg)Storing the data in Hiera(-gpg)
• Usernames, password, IP addressesUsernames, password, IP addresses
• Hiera is made to be structuredHiera is made to be structured
• Using one hiera repo for all the environmentsUsing one hiera repo for all the environments
• Using Hiera in your manifests, not in your modulesUsing Hiera in your manifests, not in your modules
Julien Pivotto Testing your puppet code

12. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Hiera tree
• %{environment}/%{hostname}%{environment}/%{hostname}
• %{environment}/common%{environment}/common
• infradev/www45.yamlinfradev/www45.yaml
• infradev/common.yamlinfradev/common.yaml
Julien Pivotto Testing your puppet code

13. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Keeping clean puppet modules
http://www.flickr.com/photos/aurelie_solenne/8340968061/
• Make them readableMake them readable
• Make them reusable and sharableMake them reusable and sharable
• Don’t puppetize everythingDon’t puppetize everything
• User generated content is not puppetizedUser generated content is not puppetized
Julien Pivotto Testing your puppet code

14. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Use the right structure for your modules
• Package, config, servicePackage, config, service
• module::package, module::config, module::servicemodule::package, module::config, module::service
• Parameterized classesParameterized classes
http://www.slideshare.net/PuppetLabs/modern-module-development-ken-barber-2012-edinburgh-
puppet-camp
Julien Pivotto Testing your puppet code

15. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Distribution-agnostic puppet modules
• You don’t have to support all the distrosYou don’t have to support all the distros
• Adding support for another distro should be easyAdding support for another distro should be easy
$config_dir = $configroot ? {
undef => $::operatingsystem ? {
/Debian|Ubuntu/ => ’/etc/apache2’,
/CentOS|RedHat/ => ’/etc/httpd’,
default => ’/etc/httpd’,
},
default => $configroot,
}
Julien Pivotto Testing your puppet code

16. ;
Introduction
Testing tools
Jenkins
Conclusion
Automation
Vagrant
Puppet in a large scale
Puppet code
Puppet function
• The fail function prevents catalog to be appliedThe fail function prevents catalog to be applied
• The notify function prints a warningThe notify function prints a warning
if (!$leftsubnet) and (!$leftsubnets) {
fail(’$leftsubnets and $leftsubnet both empty’)
}
Julien Pivotto Testing your puppet code

17. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
Puppet parser
• Included in puppetIncluded in puppet
• Validating the syntaxValidating the syntax
• puppet parser validate init.pppuppet parser validate init.pp
• find . -name ’*.pp’ -exec puppet parser validate
+ ;
Julien Pivotto Testing your puppet code

18. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
Puppet lint
http://www.flickr.com/photos/voyages-provence/8127668094/
• Follow the puppet style guideFollow the puppet style guide
• Two-space soft tabTwo-space soft tab
• align fat comma arrows (=>) within blocks of attributesalign fat comma arrows (=>) within blocks of attributes
• http://docs.puppetlabs.com/guides/style_guide.htmlhttp://docs.puppetlabs.com/guides/style_guide.html
Julien Pivotto Testing your puppet code

19. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
Cucumber puppet
• Write scenariosWrite scenarios
• Easy to read (full sentences)Easy to read (full sentences)
• Use your manifestsUse your manifests
• Need some tricks to work with Puppet 3Need some tricks to work with Puppet 3
• DiscontinuedDiscontinued
Julien Pivotto Testing your puppet code

20. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
Cucumber example
Cucumber
Feature: General catalog policy
In order to ensure applicability of a host’s catalog
As a manifest developer
I want all catalogs to obey some general rules
Scenario Outline: Compile and verify catalog
Given a node specified by "features/yaml/<hostname>."
When I compile its catalog
Then compilation should succeed
And all resource dependencies should resolve
Examples:
| hostname |
| localhost |
Julien Pivotto Testing your puppet code

21. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
rspec-puppet
• Check what is the behaviour of puppetCheck what is the behaviour of puppet
• Separate tests per modulesSeparate tests per modules
• Add context, facts, . . .Add context, facts, . . .
• Test custom functions, hosts, manifests, . . .Test custom functions, hosts, manifests, . . .
Julien Pivotto Testing your puppet code

22. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
rspec-puppet
Start with rspec puppet
gem install rspec-puppet
gem install puppet
cd my-module
rspec-puppet-init
Julien Pivotto Testing your puppet code

23. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
rspec-puppet
spec/defines/connection_spec.rb
require ’spec_helper’
describe ’openswan::connection’ do
describe ’should require rightsubnet or rightsubnets’ do
let(:title) { ’foobar’ }
let (:params) { {
:ike => ’aes256-sha1;modp1024’,
:esp => ’aes256-sha1;modp1024’,
:leftsubnet => ’8.8.5.5’,
:right => ’84.54.105.5’,
:left => ’68.65.98.6’,
:foreignip => ’45.25.5.5’,
:localtestip => ’82.8.8.8’, } }
it do
expect {
should contain_file("/etc/ipsec.d/foobar.conf")
}.to raise_error(Puppet::Error, /$rightsubnets and $rightsubnet cannot be both empty/)
end
end
end
Julien Pivotto Testing your puppet code

24. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
rspec-puppet
Second example
require ’spec_helper’
describe ’apache’, :type => :class do
let (:facts) { {
:operatingsystem => ’CentOS’,
:osfamily => ’RedHat’,
} }
describe ’without parameters’ do
it { should create_class(’apache’) }
it { should include_class(’apache::service’) }
it { should contain_apache__listen(’80’) }
it { should contain_apache__namevhost(’80’) }
end
end
Julien Pivotto Testing your puppet code

25. ;
Introduction
Testing tools
Jenkins
Conclusion
Style and linting
Catalogs
rspec-puppet
rspec-puppet
• should, should_notshould, should_not
• should contain_packageshould contain_package
• contain_foo__bar(’baz’) (for foo::bar)contain_foo__bar(’baz’) (for foo::bar)
Julien Pivotto Testing your puppet code

26. ;
Introduction
Testing tools
Jenkins
Conclusion
Integration with jenkins
• Pulling, testing and deploymentsPulling, testing and deployments
• Push-Test-Package-DeployPush-Test-Package-Deploy
• Continuous integrationContinuous integration
• Continuous deliveryContinuous delivery
Julien Pivotto Testing your puppet code

27. ;
Introduction
Testing tools
Jenkins
Conclusion
Jenkins pipelines
• Build pipelinesBuild pipelines
• Overview of what happensOverview of what happens
• Getting notified about what failedGetting notified about what failed
• Promoted build pluginPromoted build plugin
Julien Pivotto Testing your puppet code

28. ;
Introduction
Testing tools
Jenkins
Conclusion
Jenkins pipelines
Julien Pivotto Testing your puppet code

29. ;
Introduction
Testing tools
Jenkins
Conclusion
Advantages of CI
• You trust your codeYou trust your code
• ReproducabilityReproducability
• You get metrics: number of warning, . . .You get metrics: number of warning, . . .
• You have a backlogYou have a backlog
• It is easy!It is easy!
Julien Pivotto Testing your puppet code

30. ;
Introduction
Testing tools
Jenkins
Conclusion
Promotions
• Provides buttons you can clickProvides buttons you can click
• Trigger actionsTrigger actions
• deploy to other environmentsdeploy to other environments
• Get a mail with the changesGet a mail with the changes
• Have a log of who deployedHave a log of who deployed
Julien Pivotto Testing your puppet code

31. ;
Introduction
Testing tools
Jenkins
Conclusion
Promotions
Julien Pivotto Testing your puppet code

32. ;
Introduction
Testing tools
Jenkins
Conclusion
Homework
Conclusion
Homework
• Integrating tests with git hooksIntegrating tests with git hooks
• Integrating tests with VIIntegrating tests with VI
• github.com/philandstuff/fizzgiggithub.com/philandstuff/fizzgig
Julien Pivotto Testing your puppet code

33. ;
Introduction
Testing tools
Jenkins
Conclusion
Homework
Conclusion
Conclusion
• Use nice & simple Puppet modulesUse nice & simple Puppet modules
• Continuous integrationContinuous integration
• Multiple environmentsMultiple environments
• Readability & reusabilityReadability & reusability
• Tools exist and work togetherTools exist and work together
Julien Pivotto Testing your puppet code

34. ;
Introduction
Testing tools
Jenkins
Conclusion
Homework
Conclusion
Contact
Julien PivottoJulien Pivotto
julien@inuits.eujulien@inuits.eu
@roidelapluie@roidelapluie
INUITS bvbaINUITS bvba
Duboisstraat 50Duboisstraat 50
2060 Antwerp2060 Antwerp
BelgiumBelgium
+32 473 441 636+32 473 441 636
https://inuits.euhttps://inuits.eu
Julien Pivotto Testing your puppet code