This document discusses testing driven development of Puppet modules. It covers using puppet-lint to check style, puppet-syntax to validate code, and rspec-puppet to test catalogs and verify relationships. Packer is used to build consistent virtual machine images, Vagrant manages virtual environments for testing, and Beaker runs acceptance tests across multiple platforms. The goal is to create a virtuous cycle of testing, validation, and development feedback to continuously improve module quality.

1. Testing Driven Puppet Modules
[Nan Liu @sesshin]
Copyright 2014
1 / 45

2. Overview
2 / 45

3. Overview
Who
This talk is not for people who:
Write perfect code (manifests)
Never upgrade Puppet
Don't have SLA
Used Puppet since 0.2x
3 / 45

4. Overview
Who
Why
Testing Puppet on Production is __
4 / 45

5. Overview
Who
Why
Positive feedback loop:
5 / 45

6. Overview
Who
Why
Creating a virtuous cycle:
6 / 45

7. Overview
Who
Why
Creating a virtuous cycle:
7 / 45

8. Overview
Who
Why
What
Developing Puppet Modules
Development feedback
puppet-lint
puppet-syntax
rspec-puppet
Testing
Packer
Vagrant
beaker
8 / 45

9. Development Feedback
9 / 45

10. Development
puppet-lint
Manifest lint check:
$ puppet lint manifests/**/*
WARNING: top-scope variable being used without an explicit namespace on line 4
WARNING: class inheriting from params class on line 26
WARNING: class not documented on line 1
WARNING: line has more than 80 characters on line 52
10 / 45

11. Development
puppet-lint
Manifest lint cleanup:
$ puppet-lint -f demo.pp
FIXED: string containing only a variable on line 1
FIXED: variable not enclosed in {} on line 5
FIXED: indentation of => is not properly aligned on line 2
FIXED: indentation of => is not properly aligned on line 3
FIXED: indentation of => is not properly aligned on line 5
WARNING: ensure found on line but it's not the first attribute on line 4
11 / 45

12. Development
puppet-lint
Configure puppet-lint behavior
PuppetLint.configuration.disable_80chars
PuppetLint.configuration.disable_arrow_alignment
PuppetLint.configuration.disable_class_inherits_from_params_class
PuppetLint.configuration.disable_class_parameter_defaults
PuppetLint.configuration.fail_on_warnings = true
PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp']
12 / 45

13. Development
puppet-lint
puppet-syntax
Manifest validation:
$ puppet parser validate manifests/bad.pp
Error: Could not parse for environment production: Syntax error at 'demo';
expected '}' at /Users/nan/src/puppet-demo/manifests/bad.pp:4
13 / 45

14. Development
puppet-lint
puppet-syntax
Rakefile
require 'puppet-syntax/tasks/puppet-syntax'
PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"]
PuppetSyntax.future_parser = true
14 / 45

15. Development
puppet-lint
puppet-syntax
Puppet module folder:
manifests/
tests/
Rakefile
require 'puppetlabs_spec_helper/rake_tasks'
PuppetLint.configuration.disable_80chars
PuppetLint.configuration.disable_arrow_alignment
PuppetLint.configuration.disable_class_inherits_from_params_class
PuppetLint.configuration.disable_class_parameter_defaults
PuppetLint.configuration.fail_on_warnings = true
PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp']
require 'puppet-syntax/tasks/puppet-syntax'
PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"]
PuppetSyntax.future_parser = true
15 / 45

16. Development
puppet-lint
puppet-syntax
rspec-puppet
RSpec-Puppet
Compile catalog
module dependency
system fact
class parameters
Verifies catalog
Class/Resource specification
Relationships
Expectation
16 / 45

17. Development
puppet-lint
puppet-syntax
rspec-puppet
.fixtures.yml
fixtures:
repositories:
apt: "https://github.com/puppetlabs/puppetlabs-apt.git"
stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git"
firewall: "https://github.com/puppetlabs/puppetlabs-firewall.git"
concat: "https://github.com/puppetlabs/puppetlabs-concat.git"
symlinks:
postgresql: "#{source_dir}"
17 / 45

18. Development
puppet-lint
puppet-syntax
rspec-puppet
Supply system facts and class parameters:
let(:facts) {{ :osfamily => 'redhat' }}
let(:params) {{
:keys_enable => true,
:keys_file => '/etc/ntp/ntp.keys',
:keys_trusted => ['1', '2', '3'],
:keys_controlkey => '2',
:keys_requestkey => '3',
}}
18 / 45

19. Development
puppet-lint
puppet-syntax
rspec-puppet
require 'spec_helper'
describe 'ntp' do
Dir.glob('tests/*.pp').each do |file|
let(:facts) {{ :osfamily => 'redhat' }}
context file do
let(:pre_condition) { File.read(file) }
it{ should compile }
end
end
end
19 / 45

20. Development
puppet-lint
puppet-syntax
rspec-puppet
Puppet Class:
describe 'ntp' do
let(:facts) {{ :osfamily => 'RedHat' }}
let(:params) {{
:iburst_enable => true,
}}
it do
should contain_file('/etc/ntp.conf').with({
'content' => /iburstn/,
})
end
end
20 / 45

21. Development
puppet-lint
puppet-syntax
rspec-puppet
Define Type
describe 'mysql::db', :type => :define do
let(:facts) {{ :osfamily => 'RedHat' }}
let(:title) { 'test_db' }
let(:params) {
{ 'user' => 'testuser',
'password' => 'testpass',
}
}
...
end
21 / 45

22. Development
puppet-lint
puppet-syntax
rspec-puppet
Puppet module folder:
.fixtures.yml
Rakefile
manifests/
tests/
spec/classes/*
spec/defines/*
22 / 45

23. Development
puppet-lint
puppet-syntax
rspec-puppet
Gemfile
source "https://rubygems.org"
group :test do
gem "rake"
gem "puppet", ENV['GEM_PUPPET_VERSION']
gem "puppet-lint"
gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git'
gem "puppet-syntax"
gem "puppetlabs_spec_helper"
end
23 / 45

24. Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
How many modules do you have?
modulesync
24 / 45

25. Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
Gem::Specification.new do |s|
...
if facter_version = ENV['GEM_FACTER_VERSION']
s.add_runtime_dependency 'facter', facter_version
else
s.add_runtime_dependency 'facter'
end
if puppet_version = ENV['GEM_PUPPET_VERSION'] || ENV['PUPPET_GEM_VERSION']
s.add_runtime_dependency 'puppet', puppet_version
else
s.add_runtime_dependency 'puppet'
end
s.add_runtime_dependency 'rake'
s.add_runtime_dependency 'rspec', '~> 2.11.0'
s.add_runtime_dependency 'mocha', '~> 0.10.5'
s.add_runtime_dependency 'puppetlabs_spec_helper', '0.7'
s.add_runtime_dependency 'rspec-puppet'
s.add_runtime_dependency 'puppet-lint', '~> 1.0'
s.add_runtime_dependency 'puppet-syntax'
s.files = Dir.glob('lib/**/*') + %w(LICENSE)
s.require_path = 'lib'
end
25 / 45

26. Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
Gemfile
group :development, :test do
gem 'bodeco_module_helper', :git => 'https://github.com/bodeco/bodeco_module_helper.git'
end
Rakefile
require 'bodeco_module_helper/rake_tasks'
26 / 45

27. Development
puppet-lint
puppet-syntax
rspec-puppet
Customize
$ rake -T
rake beaker # Run beaker acceptance tests
rake beaker_nodes # List available beaker nodesets
rake build # Build puppet module package
rake clean # Clean a built module package
rake coverage # Generate code coverage inform...
rake help # Display the list of available...
rake lint # Check puppet manifests with p...
rake spec # Run spec tests in a clean fix...
rake spec_clean # Clean up the fixtures directory
rake spec_prep # Create the fixtures directory
rake spec_standalone # Run spec tests on an existing...
rake syntax # Syntax check Puppet manifests...
rake syntax:hiera # Syntax check Hiera config files
rake syntax:manifests # Syntax check Puppet manifests
rake syntax:templates # Syntax check Puppet templates
rake validate # Check syntax of Ruby files an...
27 / 45

28. Development
puppet-lint
puppet-syntax
rspec-puppet
Summary
puppet-lint: style enforcer
puppet-syntax: code parse
rspec-puppet: catalog verify
pin your versions
28 / 45

29. Testing
29 / 45

30. Testing
Packer
Say no to mystery boxes
Virtualbox or VMware fusion/workstation
Amazon, Digital Ocean, GCE, ...
30 / 45

31. Testing
Packer
"builders": [{
"vm_name": "centos70",
"type": "vmware-iso",
"guest_os_type": "centos-64",
"http_directory": "http",
"iso_url": "{{ user `iso_url` }}",
"iso_checksum": "{{ user `iso_checksum` }}",
"tools_upload_flavor": "linux",
"boot_command": [
"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort}}/ks7.cfg<enter>"
],
"disk_size": 10140,
"vmx_data": {
"memsize": "512",
"numvcpus": "1",
"cpuid.coresPerSocket": "1"
}
}]
31 / 45

32. Testing
Packer
"provisioners": [{
"type": "shell",
"environment_vars": [
"CM={{user `cm`}}",
"CM_VERSION={{user `cm_version`}}",
"CM_SET_PATH={{user `cm_set_path`}}",
"CLEANUP_PAUSE={{user `cleanup_pause`}}"
],
"execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S bash '{{.Path}}'",
"scripts": [
"script/fix-slow-dns.sh",
"script/sshd.sh",
"script/vagrant.sh",
"script/vmtool.sh",
"script/cmtool.sh",
"script/cleanup.sh"
]
}],
32 / 45

33. Testing
Packer
https://github.com/puppetlabs/puppet-vagrant-boxes
https://github.com/mitchellh/veewee-to-packer
https://github.com/box-cutter
https://github.com/hashicorp/puppet-bootstrap
33 / 45

34. Testing
Packer
Vagrant
Vagrant.configure('2') do |conf|
conf.vm.define 'demo' do |mod|
mod.vm.box = 'centos64.box'
mod.vm.synced_folder './modules', "/opt/puppet/share/puppet/modules"
mod.vm.synced_folder './manifests', "/etc/puppetlabs/puppet/manifests"
mod.vm.synced_folder './data', "/etc/puppetlabs/puppet/data"
mod.vm.provision :puppet do |p|
p.module_path = 'spec/fixtures/modules'
p.manifests_path = 'manifests'
p.manifest_file = ENV['VAGRANT_MANIFEST'] || 'init.pp'
p.options = '--verbose'
end
end
end
34 / 45

35. Testing
Packer
Vagrant
https://github.com/adrienthebo/vagrant-config_builder
$ vagrant plugin install vagrant-config_builder
├── config
│ ├── roles.yaml
│ └── vms.yaml
└── Vagrantfile
---
vms:
-
name: db
private_networks: [ {ip: '10.20.1.2'} ]
box: centos-5-i386
hostname: db.puppetlabs.vm
synced_folders:
- host_path: '.'
guest_path: '/vagrant'
disabled: true
provisioners:
- type: puppet
manifests_path: 'tests'
module_path: 'spec/fixtures/modules'
manifest_file: <%= ENV['VAGRANT_MANIFEST'] || 'init.pp' %>
35 / 45

36. Testing
Packer
Vagrant
def vm(opt)
module_name = opt.fetch(:module).to_s || raise(ArgumentError, 'Must provide puppet module hostname = opt.fetch(:hostname, module_name).to_s
memory = opt.fetch(:memory, 512)
cpu = opt.fetch(:cpu, 1)
os_type = opt.fetch(:type, :linux)
Vagrant.configure('2') do |conf|
conf.vm.network(:forwarded_port, guest: port, host: port, auto_correct: true) if port
if os_type == :windows
conf.ssh.username = 'vagrant'
conf.winrm.username = 'vagrant'
conf.winrm.password = 'vagrant'
end
...
36 / 45

37. Testing
Packer
Vagrant
vm(
:hostname => 'oel',
:module => 'application',
:memory => 8096,
:box => 'oracle65-pe3.2.3',
:port => 8080
)
37 / 45

38. Testing
Packer
Vagrant
$ vagrant up
$ vagrant provision
$ vagrant ssh
$ vagrant destroy
38 / 45

39. Testing
Packer
Vagrant
Beaker
Puppet Labs testing framework
spec/acceptance
├── class_spec.rb
├── disable_monitoring_spec.rb
├── nodesets
│ ├── centos-59-x64.yml
│ ├── centos-64-x64-pe.yml
│ ├── centos-64-x64.yml
│ ├── centos-65-x64.yml
│ ├── default.yml
│ ├── fedora-18-x64.yml
│ ├── sles-11-x64.yml
│ ├── ubuntu-server-10044-x64.yml
│ ├── ubuntu-server-12042-x64.yml
│ └── ubuntu-server-1404-x64.yml
├── ntp_config_spec.rb
├── ntp_install_spec.rb
├── ntp_parameters_spec.rb
├── ntp_service_spec.rb
├── preferred_servers_spec.rb
├── restrict_spec.rb
└── unsupported_spec.rb
39 / 45

40. Testing
Packer
Vagrant
Beaker
HOSTS:
centos-64-x64:
roles:
- master
platform: el-6-x86_64
box : centos-64-x64-vbox4210-nocm
box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box
hypervisor : vagrant
CONFIG:
type: foss
40 / 45

41. Testing
Packer
Vagrant
Beaker
it 'should run successfully' do
pp = "class { 'ntp': }"
# Apply twice to ensure no errors the second time.
apply_manifest(pp, :catch_failures => true) do |r|
expect(r.stderr).not_to match(/error/i)
end
apply_manifest(pp, :catch_failures => true) do |r|
expect(r.stderr).not_to eq(/error/i)
expect(r.exit_code).to be_zero
end
end
41 / 45

42. Testing
Packer
Vagrant
Beaker
it 'starts the service' do
pp = <<-EOS
class { 'ntp':
service_enable => true,
service_ensure => running,
service_manage => true,
service_name => '#{servicename}'
}
EOS
apply_manifest(pp, :catch_failures => true)
end
describe service(servicename) do
it { should be_running }
it { should be_enabled }
end
42 / 45

43. Testing
Packer
Vagrant
Beaker
Summary
Packer: VM build
Vagrant: VM clone and testing
Beaker: Automated testing
43 / 45

44. Questions
44 / 45

45. Thank You!
[nan@bodeco.io]
45 / 45