Puppet Module
Reusability
Learning from shipping to the Forge
Gareth Rushgrove
Who
(Who is this person?)
@garethr
UK Government
Digital Service
Last code
I wrote
7. Gareth Rushgrove
Last puppet
module I wrote
The Problem
(Sharing modules is hard)
Not invented here
syndrome
1
A search for
puppet-apache
145
Gareth Rushgrove
145Gareth Rushgrove
Puppetlabs
apache
67
contributors
Gareth Rushgrove
281
forks
Gareth Rushgrove
281Gareth Rushgrove
Vendor everything
pattern
2
Not publishing to the
Forge
3
Puppet
Forge
Undocumented
modules
4
Official docs
guidelines
(Too) opinionated
configuration
5
Sometimes I just
want to write nginx
configuration
Gareth Rushgrove
Duplicate resources
6
Package { 'build-essential’:
ensure => installed,
}
Gareth Rushgrove
Stdlib
(Start here)
Standard
Library from
Puppetlabs
Fail fast
Gareth Rushgrove
Validation
Gareth Rushgrove
Useful error
messages
Gareth Rushgrove
fail("${::operatingsystem}
not supported")
Gareth Rushgrove
validate_string
Gareth Rushgrove
validate_string($version)
Gareth Rushgrove
<Puppet::Error: true is
not a string.
Gareth Rushgrove
validate_slength
Gareth Rushgrove
validate_re
Gareth Rushgrove
validate_hash
Gareth Rushgrove
Gareth Rushgrove
validate_cmd
validate_bool
Gareth Rushgrove
Avoid duplicate
resources
Gareth Rushgrove
Package { 'build-essential’:
ensure => installed,
}
Gareth Rushgrove
include 'gcc'
Gareth Rushgrove
package{[
'python-pip',
'python-ldap',
]:
ensure => installed
}
Gareth Rushgrove
ensure_packages
Gareth Rushgrove
ensure_packages([
'python-pip',
'python-ldap',
])
Gareth Rushgrove
ensure_resource
Gareth Rushgrove
Nicer interfaces
Gareth Rushgrove
str2bool
Gareth Rushgrove
any2array
Gareth Rushgrove
And much more
Gareth Rushgrove
Dependency
management is
everywhere
(Else)
NPM, Bundler, Pip,
Mix, Leiningen
Gareth Rushgrove
Librarian
Puppet
r10k
echo 'modules' >> .gitignore
Gareth Rushgrove
dependency 'puppetlabs/stdlib'
dependency 'garethr/erlang'
dependency 'maestrodev/wget'
Gareth Rushgrove
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/ruby'
mod 'puppetlabs/ntp'
mod 'puppetlabs/git'
mod 'puppetlabs/vcsrepo'
mod 'puppetlabs/apt'
mod 'puppetlabs/gcc'
Gareth Rushgrove
librarian-puppet install
Gareth Rushgrove
A Nice
Module Pattern
(Structuring modules for reuse)
R.I.Pienaar
install, config,
service, params
Gareth Rushgrove
manifests
!"" config.pp
!"" init.pp
!"" install.pp
!"" params.pp
#"" service.pp
Gareth Rushgrove
class sample (
) inherits sample::params {
class { 'sample::install': } ->
class { 'sample::config': } ~>
class { 'sample::service': } ->
Class['sample']
}
Gareth Rushgrove
anchor { 'sample::begin': } ->
class { 'sample::install': } ->
class { 'sample::config': }
class { 'sample::service': } ->
anchor { 'sample::end': }
Gareth Rushgrove
Anchor['sample::begin'] ~>
Class['sample::service']
Class['sample::install'] ~>
Class['sample::service']
Class['sample::config'] ~>
Class['sample::service']
Gareth Rushgrove
Puppet module tool
Gareth Rushgrove
puppet module generate name
Gareth Rushgrove
Default module
skeleton
Gareth Rushgrove
In puppet
source code
!"" manifests
!"" spec
!"" tests
!"" Modulefile
!"" .gitignore
#"" README
Gareth Rushgrove
Creating your own
module skeleton
Gareth Rushgrove
~/.puppet/var/puppet-module/skeleton
Gareth Rushgrove
puppet module
skeleton
!"" manifests
!"" spec
!"" tests
!"" templates
!"" files
!"" lib
!"" Gemfile
Gareth Rushgrove
!"" Rakefile
!"" .nodeset.yml
!"" .fixtures.yml
!"" .travis.yml
!"" Modulefile
!"" .gitignore
#"" README.md
Gareth Rushgrove
Creates install,
config, service,
params classes
Gareth Rushgrove
Dependency
management with
Bundler
Gareth Rushgrove
Better unit testing
setup using rspec-
puppet-helper
Gareth Rushgrove
Adds syntax and lint
checking
Gareth Rushgrove
Adds Travis CI
configuration
Gareth Rushgrove
Adds integration tests
with rspec-system
Gareth Rushgrove
Adds module
management with
maestrodev
blacksmith
Gareth Rushgrove
Focus on the
interface
Gareth Rushgrove
Minimize number of
entry points
Gareth Rushgrove
Allow overriding
provided templates
Gareth Rushgrove
Multi-OS Support
(Where to abstract the differences)
Vary default
parameters
Gareth Rushgrove
Keep logic in one
place
Gareth Rushgrove
case $::osfamily {
'Debian': {
  }
  'RedHat', 'Amazon': {
}
default: {
fail("${::operatingsystem} not su
}
Gareth Rushgrove
ARM-9 Data
in Modules
garethr-erlang
0.0.x supported
Ubuntu only
Gareth Rushgrove
0.1.x supports
Ubuntu, Debian,
Redhat, Suse
Gareth Rushgrove
Gareth Rushgrove
Pull requests to
the rescue
Insist on tests with
open source
contributions
Gareth Rushgrove
Module Testing
(Code should have tests)
Why test a
declarative
language?
Gareth Rushgrove
Modules increasingly
contain logic
Gareth Rushgrove
Modules increasingly
take arguments
Gareth Rushgrove
Modules increasingly
have interfaces with
other modules
Gareth Rushgrove
Modules increasingly
used in many Ruby
and Puppet version
combinations
Gareth Rushgrove
Good news. The tools
got better
Gareth Rushgrove
puppet-lint
Gareth Rushgrove
Puppet
style guide
Available
as a gem
puppet-lint --with-filename /etc/puppet/modules
foo/manifests/bar.pp: trailing whitespace found
on line 1
apache/manifests/server.pp: variable not enclosed
in {} on line 56
Gareth Rushgrove
puppet-syntax
Gareth Rushgrove
Validate Puppet
and ERB syntax
require 'puppet-syntax/tasks/puppet-syntax'
Gareth Rushgrove
bundle exec rake syntax
---> syntax:manifests
---> syntax:templates
Gareth Rushgrove
rspec-puppet
Gareth Rushgrove
Unit testing
for Puppet
context "epel enabled" do
let(:params) {{ :epel_enable => true }}
it { should contain_class('epel') }
end
Gareth Rushgrove
context "epel disabled" do
let(:params) {{ :epel_enable => false }}
it { should_not contain_class('epel') }
end
Gareth Rushgrove
Fixtures,
matchers
Test the interface not
the implementation
Gareth Rushgrove
All of the above
Gareth Rushgrove
task :test => [
:syntax,
:lint,
:spec,
]
Gareth Rushgrove
bundle exec rake test
Gareth Rushgrove
Gareth Rushgrove
Nice continuous
integration
Test pull request
branches too
---
language: ruby
before_install: rm Gemfile.lock || true
rvm:
- 1.8.7
- 1.9.3
script: bundle exec rake test
env:
matrix:
- PUPPET_VERSION="~> 2.7.0"
- PUPPET_VERSION="~> 3.1.0"
- PUPPET_VERSION="~> 3.2.0"
Gareth Rushgrove
A matrix of
possibilities
Gareth Rushgrove
rspec-system
Gareth Rushgrove
Integration
test against
real systems
Puppet
helpers for
rspec-system
Vagrant
driver
VSphere
provider
Test that Puppet runs
without errors
Gareth Rushgrove
it 'should run without errors' do
pp = "class { 'sample': }"
puppet_apply(pp) do |r|
r.exit_code.should == 2
end
end
Gareth Rushgrove
Test runs are
idempotent
Gareth Rushgrove
it 'should run without errors' do
pp = "class { 'sample': }"
puppet_apply(pp) do |r|
r.exit_code.should == 2
r.refresh
r.exit_code.should be_zero
end
end
Gareth Rushgrove
Test that the module
installs relevant
binaries
Gareth Rushgrove
it 'should install the erl binary' do
shell 'which erl' do |r|
r.stdout.should =~ //usr/bin/e
r.stderr.should be_empty
r.exit_code.should be_zero
end
end
Gareth Rushgrove
Test against different
operating systems
Gareth Rushgrove
default_set: 'centos-64-x64'
sets:
'centos-64-x64':
nodes:
"main.foo.vm":
prefab: 'centos-64-x64'
'fedora-18-x64':
nodes:
"main.foo.vm":
prefab: 'fedora-18-x64'
'debian-607-x64':
nodes:
"main.foo.vm":
Gareth Rushgrove
Room for
improvements
(Making reuse easier in Puppet)
Run your own Forge
1
A more secure Forge
2
Bless a dependency
management tool
3
Optional
dependencies
4
dependency 'puppetlabs/stdlib'
optional_dependency 'puppetlabs/apache'
Gareth Rushgrove
Private classes
5
private class elixir::install {
Gareth Rushgrove
Parameter naming
conventions
6
example42
Proposal
Interfaces in Puppet
7
interface packageandservice {
$version
$enable
$package_name
$template
}
Gareth Rushgrove
class diamond (
$version,
$enable,
$package_name,
$template,
) {
implements packageandservice
Gareth Rushgrove
Questions?
(And thanks for listening)

Puppet Module Reusability - What I Learned from Shipping to the Forge