1.
How to Create a Next-Gen
Example42 Puppet Module
in 5 Minutes
Prepare for an info overload rush
and an Ignite's verbosity world record
Alessandro Franceschi
DevOps Days Rome 2012

2.
Example42 Next-Gen modules...
Support for different OS (Default: RedHat / Debian)
Fully parametrized classes
Data separation with Hiera/ENC parameters lookup
Extreme customization options
Module behavior customization options
Full and partial decommissioning support
Optional automatic Monitoring / Firewalling
Optional Puppi integration
Optional Debug and Auditing

3.
Fetch the sources
git clone --recursive
git://github.com/example42/
puppet-modules-nextgen.git
Cloning into 'puppet-modules-nextgen'...
remote: Counting objects: 608, done.
remote: Compressing objects: 100% (317/317), done.
remote: Total 608 (delta 281), reused 578 (delta 252)
Receiving objects: 100% (608/608), 102.59 KiB | 118 KiB/s, done.
Resolving deltas: 100% (281/281), done.
Submodule 'Example42-documentation' (git://github.com/example42/
Example42-documentation.git) registered for path 'Example42-
documentation'
Submodule 'Example42-tools' (git://github.com/example42/Example42-
tools.git) registered for path 'Example42-tools'
[...]

4.
Enjoy modules' variety
cd puppet-modules-nextgen ; ls
Example42-documentation! maven! ! ! ! puppi
Example42-tools! ! ! mcollective! ! ! rclocal
Gemfile!! ! ! ! monitor! ! ! ! redis
README.rdoc! ! ! ! munin! ! ! ! resolver
Rakefile!
! ! ! ! mysql! ! ! ! rsync
activemq!
! ! ! ! nagios! ! ! ! rsyncssh
apache! ! ! ! ! nginx! ! ! ! rvm
concat! ! ! ! ! nrpe!! ! ! ! solr
example42! ! ! ! ntp! ! ! ! ! splunk
firewall!
! ! ! ! openntpd!! ! ! stdlib42
foo!! ! ! ! ! openssh! ! ! ! sudo
foo_webapp! ! ! ! openvpn! ! ! ! tartarus
foreman!! ! ! ! orientdb!! ! ! tftp
haproxy!! ! ! ! pentaho! ! ! ! tomcat
icinga! ! ! ! ! php! ! ! ! ! vagrant
iptables!
! ! ! ! postfix! ! ! ! vsftpd
java! ! ! ! ! postgresql! ! ! wget
jboss! ! ! ! ! pupmod-concat! ! wordpress
jenkins!! ! ! ! puppet! ! ! ! xinetd
libvirt!! ! ! ! puppetdashboard! ! yum
logstash!
! ! ! ! puppetdb

5.
Create a new module from foo
Example42-tools/module_clone.sh
This script creates a skeleton for a new module based on different Example42 foo module templates.
[...]
Enter the name of the new module based on foo: lighttpd
COPYING MODULE
building file list ... done
Modulefile
README.rdoc
Rakefile
manifests/
manifests/init.pp
manifests/params.pp
manifests/spec.pp
spec/
spec/spec_helper.rb
spec/classes/
spec/classes/foo_spec.rb
[...]
RENAMING FILES
Renamed lighttpd/spec/classes/foo_spec.rb to lighttpd/spec/classes/lighttpd_spec.rb
---------------------------------------------------
CHANGING FILE CONTENTS
Changed lighttpd/manifests/init.pp
Changed lighttpd/manifests/params.pp
Changed lighttpd/manifests/spec.pp
Changed lighttpd/Modulefile
Changed lighttpd/README.rdoc
Changed lighttpd/spec/classes/lighttpd_spec.rb
Changed lighttpd/spec/spec_helper.rb
Module lighttpd created
Start to edit lighttpd/manifests/params.pp to customize it

6.
Explore module's contents
find lighttpd/
lighttpd/ git init
git add .
git commit -m "Example42-tools/module_clone.sh lighttpd"
lighttpd/spec
lighttpd/spec/spec_helper.rb
lighttpd/spec/classes
lighttpd/spec/classes/lighttpd_spec.rb
lighttpd/Rakefile
lighttpd/Modulefile
lighttpd/manifests
lighttpd/manifests/init.pp
lighttpd/manifests/params.pp
lighttpd/manifests/spec.pp
lighttpd/README.rdoc
lighttpd/templates
lighttpd/templates/spec.erb

7.
Edit params.pp 1/2
vi lighttpd/manifests/params.pp
class lighttpd::params {
$package = $::operatingsystem ? {
default => 'lighttpd',
}
$service = $::operatingsystem ? {
default => 'lighttpd',
}
[...]
$config_dir = $::operatingsystem ? {
/(?i:Debian|Ubuntu|Mint)/ => '/etc/lighttpd/conf-enabled',
default => '/etc/lighttpd/conf.d',
}
$config_file_init = $::operatingsystem ? {
/(?i:Debian|Ubuntu|Mint)/ => '/etc/default/lighttpd',
default => '/etc/sysconfig/lighttpd',
}
$data_dir = $::operatingsystem ? {
/(?i:Debian|Ubuntu|Mint)/ => '/var/run/lighttpd',
default => '/var/www',
}
$log_dir = $::operatingsystem ? {
default => '/var/log/lighttpd',
}
$log_file = $::operatingsystem ? {
default => '/var/log/lighttpd.log',
}
$port = '80'
$protocol = 'tcp'

8.
Edit params.pp 2/2
vi lighttpd/manifests/params.pp
# You should not need to change the following ones
class lighttpd::params {
[...]
# General Settings
$my_class = ''
$source = ''
$source_dir = ''
$source_dir_purge = false
$template = ''
$options = ''
$service_autorestart = true
$version = 'present'
$absent = false
$disable = false
$disableboot = false
### General module variables that can have a site or per module default
$monitor = false
$monitor_tool = ''
git status
$monitor_target = $::ipaddress
$firewall = false
#! modified: manifests/params.pp
$firewall_tool = ''
$firewall_src = '0.0.0.0/0' git add .
$firewall_dst = $::ipaddress git commit -m "Basic support for Ubuntu/RedHat families"
$puppi = false
$puppi_helper = 'standard'
$debug = false
$audit_only = false

9.
Review and sign module
vi lighttpd/README.rdoc
= Puppet module: lighttpd
This is a Puppet module for lighttpd based on the second generation layout ("NextGen") of Example42 Puppet Modules.
Made by Alvagante / Example42
Official site: http://www.example42.com
Official git repository: http://github.com/example42/puppet-lighttpd
Released under the terms of Apache 2 License.
== USAGE - Basic management
* Install lighttpd with default settings
class { 'lighttpd': }
* Use custom source directory for the whole configuration dir
class { 'lighttpd':
source_dir => 'puppet:///modules/example42/lighttpd/conf/',
}
* Use custom template for main config file. Note that template and source arguments are alternative.
class { 'lighttpd':
template => 'example42/lighttpd/lighttpd.conf.erb',
}
* Automatically include a custom subclass git status
class { 'lighttpd': #! modified: README.rdoc
my_class => 'example42::my_lighttpd',
} git add .
git commit -m "Changed module's author to ... me"
* Activate automatic monitoring
class { 'lighttpd':
monitor => true,
monitor_tool => [ 'nagios' , 'monit' , 'munin' ],
}

10.
Explore the module's logic 1/4
less lighttpd/manifests/init.pp
# = Class: lighttpd
# == Parameters
#
# [*my_class*]
# Name of a custom class to autoload to manage module's customizations
# If defined, lighttpd class will automatically "include $my_class"
# Can be defined also by the (top scope) variable $lighttpd_myclass
#
# [*source*]
# Sets the content of source parameter for main configuration file
# If defined, lighttpd main config file will have the param: source => $source
# Can be defined also by the (top scope) variable $lighttpd_source
[...]
class lighttpd (
$my_class = params_lookup( 'my_class' ),
$source = params_lookup( 'source' ),
$source_dir = params_lookup( 'source_dir' ),
$source_dir_purge = params_lookup( 'source_dir_purge' ),
$template = params_lookup( 'template' ),
$service_autorestart = params_lookup( 'service_autorestart' , 'global' ),
$options = params_lookup( 'options' ),
$version = params_lookup( 'version' ),
$disable = params_lookup( 'disable' ),
[...]
$data_dir = params_lookup( 'data_dir' ),
$log_dir = params_lookup( 'log_dir' ),
$log_file = params_lookup( 'log_file' ),
$port = params_lookup( 'port' ),
$protocol = params_lookup( 'protocol' )
) inherits lighttpd::params {

11.
Explore the module's logic 2/4
less lighttpd/manifests/init.pp
class lighttpd (
[...]
) inherits lighttpd::params {
$bool_source_dir_purge=any2bool($source_dir_purge)
$bool_service_autorestart=any2bool($service_autorestart)
$bool_absent=any2bool($absent)
$bool_disable=any2bool($disable)
$bool_disableboot=any2bool($disableboot)
$bool_monitor=any2bool($monitor)
[...]
### Definition of some variables used in the module
$manage_package = $lighttpd::bool_absent ? {
true => 'absent',
false => $lighttpd::version,
}
$manage_service_enable = $lighttpd::bool_disableboot ? {
true => false,
default => $lighttpd::bool_disable ? {
true => false,
default => $lighttpd::bool_absent ? {
true => false,
false => true,
},
},
}
[...]

12.
Explore the module's logic 3/4
less lighttpd/manifests/init.pp
class lighttpd (
[...]
) inherits lighttpd::params {
$manage_package = $lighttpd::bool_absent ? {
true => 'absent',
false => $lighttpd::version,
<=[...]
}
package { 'lighttpd':
ensure => $lighttpd::manage_package,
$manage_service_enable = $lighttpd::bool_disableboot ? {
name => $lighttpd::package,
true => false,
}
default => $lighttpd::bool_disable ? {
true => false,
service { 'lighttpd':
default => $lighttpd::bool_absent ? {
ensure => $lighttpd::manage_service_ensure,
true => false,
name => $lighttpd::service,
false => true,
enable => $lighttpd::manage_service_enable,
},
hasstatus => $lighttpd::service_status,
},
pattern => $lighttpd::process,
}
require => Package['lighttpd'],
[...]
}
$manage_file_source = $lighttpd::source ? {
'' => undef,
file { 'lighttpd.conf':
default => $lighttpd::source,
ensure => $lighttpd::manage_file,
}
path => $lighttpd::config_file, [...]
notify => $lighttpd::manage_service_autorestart,
$manage_file_content = $lighttpd::template ? {
source => $lighttpd::manage_file_source,
'' => undef,
content => $lighttpd::manage_file_content,
default => template($lighttpd::template),
replace => $lighttpd::manage_file_replace,
}
}
[...]=>

13.
Explore the module's logic 4/4
less lighttpd/manifests/init.pp
[...]
# The whole lighttpd configuration directory can be recursively overriden
if $lighttpd::source_dir {
file { 'lighttpd.dir':
ensure => directory,
path => $lighttpd::config_dir,
require => Package['lighttpd'],
notify => $lighttpd::manage_service_autorestart,
source => $lighttpd::source_dir,
recurse => true,
purge => $lighttpd::bool_source_dir_purge,
replace => $lighttpd::manage_file_replace,
audit => $lighttpd::manage_audit,
}
}
### Include custom class if $my_class is set
if $lighttpd::my_class {
# With
include $lighttpd::my_class class { 'lighttpd':
} my_class => 'example42::my_lighttpd',
}
### Provide puppi data, if puppi is enabled
if $lighttpd::bool_puppi == true { # You can autoload example42/manifests/my_lighttpd.pp:
$classvars=get_class_args() class example42::my_lighttpd {
puppi::ze { 'lighttpd': # My extra custom resources
ensure => $lighttpd::manage_file,
}
variables => $classvars,
helper => $lighttpd::puppi_helper,
}
}

14.
Add a parameter to the class
vi lighttpd/manifests/params.pp
### Module specific parameters
$use_ssl = false
vi lighttpd/manifests/init.pp
# == Parameters
# [*use_ssl*]
# Set to true to activate ssl.
# In order to use this option you need to use a template that honours it:
# template => 'site/lighttpd/lighttpd.conf.erb',
# (cp lighttpd/templates/lighttpd.conf.erb site/templates/lighttpd/lighttpd.conf.erb)
[...]
class lighttpd (
$use_ssl = params_lookup( 'use_ssl' ),
) inherits lighttpd::params {
[...]
$bool_use_ssl=any2bool($use_ssl)
[...]
### Include ssl subclass (if it were of any use)
# if $lighttpd::bool_use_ssl == true {
# include $lighttpd::ssl
# fi
git status
#! modified: manifests/init.pp
#! modified: manifests/params.pp
git add .
git commit -m "Added (mostly useless) use_ssl parameter"

15.
Create a sample template
vi lighttpd/templates/lighttpd.conf.erb
# File Managed by Puppet
#
# Sample /etc/lighttpd/lighttpd.conf based on Centos6 layout
#
var.log_root = "<%= scope.lookupvar('lighttpd::log_dir') %>"
var.server_root = "<%= scope.lookupvar('lighttpd::data_dir') %>"
var.conf_dir = "<%= scope.lookupvar('lighttpd::conf_dir') %>"
[...]
<% if bool_use_ssl == true %>
ssl.engine = "enable"
ssl.pemfile = "/path/to/server.pem"
<% end %>
[...]
server.max-connections = <%= scope.function_options_lookup(['server.max-connections',‘1024’]) %>
server.server.max-keep-alive-idle = <%= scope.function_options_lookup(['server.max-keep-alive-idle',‘5’]) %>
server.max-request-size = <%= scope.function_options_lookup(['server.max-request-size',‘0’]) %>
server.max-read-idle = <%= scope.function_options_lookup(['server.max-read-idle',‘60’]) %>
server.max-write-idle = <%= scope.function_options_lookup(['server.max-write-idle',‘360’]) %>
server.kbytes-per-second = <%= scope.function_options_lookup(['server.kbytes-per-second',‘128’]) %>
server.connection.kbytes-per-second = <%= scope.function_options_lookup(['connection.kbytes-per-second',‘32’]) %>
git status
#! modified: templates/lighttpd.conf.erb
git add .
git commit -m "Added sample lighttpd.conf template (not used by default)"

16.
Add a define 1/2
vi lighttpd/manifests/dotconf.pp
define lighttpd::dotconf (
$source = '' ,
$template = '' ,
$options = '',
$ensure = present ) {
$manage_file_source = $source ? {
'' => undef,
default => $source,
}
$manage_file_content = $template ? {
'' => undef,
default => template($template),
}
file { "Lighttpd_$name.conf":
ensure => $ensure,
path => "${lighttpd::config_dir}/${name}.conf",
mode => $lighttpd::config_file_mode,
owner => $lighttpd::config_file_owner,
group => $lighttpd::config_file_group,
require => Package['lighttpd'],
notify => $lighttpd::manage_service_autorestart,
source => $manage_file_source,
content => $manage_file_content,
audit => $lighttpd::manage_audit,
}
git status
}
#! modified: manifests/dotconf.pp

17.
Add rspec test for a define
vi spec/defines/lighttpd_dotconf_spec.rb
require "#{File.join(File.dirname(__FILE__),'..','spec_helper.rb')}"
describe 'lighttpd::dotconf' do
let(:title) { 'lighttpd::dotconf' }
let(:node) { 'rspec.example42.com' }
let(:facts) { { :arch => 'i386' , :operatingsystem => 'redhat' } }
let(:params) {
{ 'ensure' => 'present',
'name' => 'www.example42.com',
'source' => 'puppet:///modules/site/lighttpd/www.example42.com.conf',
}
}
describe 'Test lighttpd::dotconf' do
it 'should create a lighttpd::dotconf' do
should contain_file('Lighttpd_www.example42.com.conf').with_ensure('present')
end
end
describe 'Test lighttpd::dotconf source parameter' do
it 'should create a lighttpd::dotconf' do
content = catalogue.resource('file', 'Lighttpd_www.example42.com.conf').send(:parameters)[:source]
content.should == "puppet:///modules/site/lighttpd/www.example42.com.conf"
end
end
describe 'Test lighttpd::virtualhost decommissioning' do
let(:facts) { { :arch => 'i386' , :operatingsystem => 'ubuntu' } }
let(:params) {
{ 'ensure' => 'absent' git status
} #! modified: spec/defines/lighttpd_dotconf_spec.rb
}
it 'should remove a lighttpd::dotconf file with ensure => absent' do
git add .
should contain_file('Lighttpd_www.example42.com.conf').with_ensure('absent')
end git commit -m "Added lighttpd::dotconf spec tests"
end

18.
Run Tests
cd lighttpd;./Example42-tools/check-module.sh
############################
### Executing rake tasks ###
############################
/usr/local/rvm/rubies/ruby-1.8.7-head/bin/ruby -S rspec spec/classes/lighttpd_spec.rb --format doc --color
Run options: exclude {:broken=>true}
lighttpd
Test standard installation
should contain Package[lighttpd] with ensure => "present"
should contain Service[lighttpd] with ensure => "running"
should contain Service[lighttpd] with enable => "true"
should contain File[lighttpd.conf] with ensure => "present"
Test installation of a specific version
should contain Package[lighttpd] with ensure => "1.0.42"
Test standard installation with monitoring and firewalling
should contain Package[lighttpd] with ensure => "present"
should contain Service[lighttpd] with ensure => "running"
should contain Service[lighttpd] with enable => "true"
should contain File[lighttpd.conf] with ensure => "present"
should monitor the process
should place a firewall rule
[...]
############################
### Executing puppetlint ###
############################
### ./manifests/init.pp
WARNING: line has more than 80 characters on line 108
WARNING: line has more than 80 characters on line 437
### ./manifests/params.pp
### ./manifests/spec.pp

19.
Job Done
Module is ready to use:
node test {
class { 'lighttpd':
# Custom configuration provided as a template created in:
# MODULEPATH/site/templates/lighttpd/lighttpd.conf.erb
template => 'site/lighttpd/lighttpd.conf.erb'
# Custom options used in template:
options => {
'server.kbytes-per-second' => 254,
'server.max-keep-alive-idle' => 3,
}
}
}
git remote add origin https://github.com/example42/puppet-lighttpd.git
git push -u origin master

20.
Was it too fast?
Get the module:
http://github.com/example42/
puppet-lighttpd.git
Review these slides:
http://slideshare.net/alvagante
Follow Up on Twitter:
@alvagante