How to make an Example42 nextgen module in 5 minutes

Alessandro Franceschi
Alessandro FranceschiFounder @ Lab42
How to Create a Next-Gen
Example42 Puppet Module
      in 5 Minutes
          Prepare for an info overload rush
        and an Ignite's verbosity world record




       Alessandro Franceschi
      DevOps Days Rome 2012
Example42 Next-Gen modules...
Support for different OS (Default: RedHat / Debian)
Fully parametrized classes
Data separation with Hiera/ENC parameters lookup
Extreme customization options
Module behavior customization options
Full and partial decommissioning support
Optional automatic Monitoring / Firewalling
Optional Puppi integration
Optional Debug and Auditing
Fetch the sources
git clone --recursive
git://github.com/example42/
puppet-modules-nextgen.git
Cloning into 'puppet-modules-nextgen'...
remote: Counting objects: 608, done.
remote: Compressing objects: 100% (317/317), done.
remote: Total 608 (delta 281), reused 578 (delta 252)
Receiving objects: 100% (608/608), 102.59 KiB | 118 KiB/s, done.
Resolving deltas: 100% (281/281), done.
Submodule 'Example42-documentation' (git://github.com/example42/
Example42-documentation.git) registered for path 'Example42-
documentation'
Submodule 'Example42-tools' (git://github.com/example42/Example42-
tools.git) registered for path 'Example42-tools'
[...]
Enjoy modules' variety
cd puppet-modules-nextgen ; ls
Example42-documentation!   maven!   !   !     !   puppi
Example42-tools! !   !     mcollective! !     !   rclocal
Gemfile!!    !   !   !     monitor! !   !     !   redis
README.rdoc! !   !   !     munin!   !   !     !   resolver
Rakefile!
        !    !   !   !     mysql!   !   !     !   rsync
activemq!
        !    !   !   !     nagios! !    !     !   rsyncssh
apache! !    !   !   !     nginx!   !   !     !   rvm
concat! !    !   !   !     nrpe!!   !   !     !   solr
example42! !     !   !     ntp! !   !   !     !   splunk
firewall!
        !    !   !   !     openntpd!!   !     !   stdlib42
foo!!   !    !   !   !     openssh! !   !     !   sudo
foo_webapp! !    !   !     openvpn! !   !     !   tartarus
foreman!!    !   !   !     orientdb!!   !     !   tftp
haproxy!!    !   !   !     pentaho! !   !     !   tomcat
icinga! !    !   !   !     php! !   !   !     !   vagrant
iptables!
        !    !   !   !     postfix! !   !     !   vsftpd
java!   !    !   !   !     postgresql! !      !   wget
jboss! !     !   !   !     pupmod-concat!     !   wordpress
jenkins!!    !   !   !     puppet! !    !     !   xinetd
libvirt!!    !   !   !     puppetdashboard!   !   yum
logstash!
        !    !   !   !     puppetdb
Create a new module from foo
Example42-tools/module_clone.sh
This script creates a skeleton for a new module based on different Example42 foo module templates.
[...]

Enter the name of the new module based on foo:   lighttpd
COPYING MODULE
building file list ... done
Modulefile
README.rdoc
Rakefile
manifests/
manifests/init.pp
manifests/params.pp
manifests/spec.pp
spec/
spec/spec_helper.rb
spec/classes/
spec/classes/foo_spec.rb
[...]
RENAMING FILES
Renamed lighttpd/spec/classes/foo_spec.rb to lighttpd/spec/classes/lighttpd_spec.rb
---------------------------------------------------
CHANGING FILE CONTENTS
Changed lighttpd/manifests/init.pp
Changed lighttpd/manifests/params.pp
Changed lighttpd/manifests/spec.pp
Changed lighttpd/Modulefile
Changed lighttpd/README.rdoc
Changed lighttpd/spec/classes/lighttpd_spec.rb
Changed lighttpd/spec/spec_helper.rb
Module lighttpd created
Start to edit lighttpd/manifests/params.pp to customize it
Explore module's contents
find lighttpd/
lighttpd/              git init
                       git add .
                       git commit -m "Example42-tools/module_clone.sh lighttpd"
lighttpd/spec
lighttpd/spec/spec_helper.rb
lighttpd/spec/classes
lighttpd/spec/classes/lighttpd_spec.rb
lighttpd/Rakefile

lighttpd/Modulefile

lighttpd/manifests
lighttpd/manifests/init.pp
lighttpd/manifests/params.pp
lighttpd/manifests/spec.pp

lighttpd/README.rdoc

lighttpd/templates
lighttpd/templates/spec.erb
Edit params.pp 1/2
vi lighttpd/manifests/params.pp
class lighttpd::params {
  $package = $::operatingsystem ? {
    default => 'lighttpd',
  }
  $service = $::operatingsystem ? {
    default => 'lighttpd',
  }
[...]
  $config_dir = $::operatingsystem ? {
    /(?i:Debian|Ubuntu|Mint)/ => '/etc/lighttpd/conf-enabled',
    default                   => '/etc/lighttpd/conf.d',
  }
  $config_file_init = $::operatingsystem ? {
    /(?i:Debian|Ubuntu|Mint)/ => '/etc/default/lighttpd',
    default                   => '/etc/sysconfig/lighttpd',
  }

  $data_dir = $::operatingsystem ? {
    /(?i:Debian|Ubuntu|Mint)/ => '/var/run/lighttpd',
    default => '/var/www',
  }

 $log_dir = $::operatingsystem ? {
   default => '/var/log/lighttpd',
 }

 $log_file = $::operatingsystem ? {
   default => '/var/log/lighttpd.log',
 }
 $port = '80'
 $protocol = 'tcp'
Edit params.pp 2/2
vi lighttpd/manifests/params.pp
# You should not need to change the following ones
class lighttpd::params {
[...]
  # General Settings
  $my_class = ''
  $source = ''
  $source_dir = ''
  $source_dir_purge = false
  $template = ''
  $options = ''
  $service_autorestart = true
  $version = 'present'
  $absent = false
  $disable = false
  $disableboot = false

  ### General module variables that can have a site or per module default
  $monitor = false
  $monitor_tool = ''
                                      git status
  $monitor_target = $::ipaddress
  $firewall = false
                                      #! modified:         manifests/params.pp
  $firewall_tool = ''
  $firewall_src = '0.0.0.0/0'         git add .
  $firewall_dst = $::ipaddress        git commit -m "Basic support for Ubuntu/RedHat   families"
  $puppi = false
  $puppi_helper = 'standard'
  $debug = false
  $audit_only = false
Review and sign module
vi lighttpd/README.rdoc
= Puppet module: lighttpd
This is a Puppet module for lighttpd based on the second generation layout ("NextGen") of Example42 Puppet Modules.
Made by Alvagante / Example42
Official site: http://www.example42.com
Official git repository: http://github.com/example42/puppet-lighttpd
Released under the terms of Apache 2 License.

== USAGE - Basic management
* Install lighttpd with default settings
        class { 'lighttpd': }

* Use custom source directory for the whole configuration dir
        class { 'lighttpd':
          source_dir        => 'puppet:///modules/example42/lighttpd/conf/',
        }

* Use custom template for main config file. Note that template and source arguments are alternative.
        class { 'lighttpd':
          template => 'example42/lighttpd/lighttpd.conf.erb',
        }

* Automatically include a custom subclass            git status
        class { 'lighttpd':                          #! modified:          README.rdoc
          my_class => 'example42::my_lighttpd',
        }                                            git add .
                                                     git commit -m "Changed module's author to ... me"
* Activate automatic monitoring
        class { 'lighttpd':
          monitor      => true,
          monitor_tool => [ 'nagios' , 'monit' , 'munin' ],
        }
Explore the module's logic 1/4
less lighttpd/manifests/init.pp
# = Class: lighttpd
# == Parameters
#
# [*my_class*]
#   Name of a custom class to autoload to manage module's customizations
#   If defined, lighttpd class will automatically "include $my_class"
#   Can be defined also by the (top scope) variable $lighttpd_myclass
#
# [*source*]
#   Sets the content of source parameter for main configuration file
#   If defined, lighttpd main config file will have the param: source => $source
#   Can be defined also by the (top scope) variable $lighttpd_source
[...]
class lighttpd (
  $my_class            = params_lookup( 'my_class' ),
  $source              = params_lookup( 'source' ),
  $source_dir          = params_lookup( 'source_dir' ),
  $source_dir_purge    = params_lookup( 'source_dir_purge' ),
  $template            = params_lookup( 'template' ),
  $service_autorestart = params_lookup( 'service_autorestart' , 'global' ),
  $options             = params_lookup( 'options' ),
  $version             = params_lookup( 'version' ),
  $disable             = params_lookup( 'disable' ),
[...]
  $data_dir            = params_lookup( 'data_dir' ),
  $log_dir             = params_lookup( 'log_dir' ),
  $log_file            = params_lookup( 'log_file' ),
  $port                = params_lookup( 'port' ),
  $protocol            = params_lookup( 'protocol' )
  ) inherits lighttpd::params {
Explore the module's logic 2/4
less lighttpd/manifests/init.pp
class lighttpd (
[...]
  ) inherits lighttpd::params {

  $bool_source_dir_purge=any2bool($source_dir_purge)
  $bool_service_autorestart=any2bool($service_autorestart)
  $bool_absent=any2bool($absent)
  $bool_disable=any2bool($disable)
  $bool_disableboot=any2bool($disableboot)
  $bool_monitor=any2bool($monitor)
  [...]

  ### Definition of some variables used in the module
  $manage_package = $lighttpd::bool_absent ? {
    true => 'absent',
    false => $lighttpd::version,
  }

  $manage_service_enable = $lighttpd::bool_disableboot ? {
    true      => false,
    default => $lighttpd::bool_disable ? {
       true     => false,
       default => $lighttpd::bool_absent ? {
          true => false,
          false => true,
       },
    },
  }

  [...]
Explore the module's logic 3/4
less lighttpd/manifests/init.pp
class lighttpd (
[...]
  ) inherits lighttpd::params {

  $manage_package = $lighttpd::bool_absent ? {
    true => 'absent',
    false => $lighttpd::version,
                                                             <=[...]
  }
                                                             package { 'lighttpd':
                                                               ensure => $lighttpd::manage_package,
  $manage_service_enable = $lighttpd::bool_disableboot ? {
                                                               name   => $lighttpd::package,
    true      => false,
                                                             }
    default => $lighttpd::bool_disable ? {
       true     => false,
                                                             service { 'lighttpd':
       default => $lighttpd::bool_absent ? {
                                                               ensure     => $lighttpd::manage_service_ensure,
          true => false,
                                                               name       => $lighttpd::service,
          false => true,
                                                               enable     => $lighttpd::manage_service_enable,
       },
                                                               hasstatus => $lighttpd::service_status,
    },
                                                               pattern    => $lighttpd::process,
  }
                                                               require    => Package['lighttpd'],
  [...]
                                                             }
  $manage_file_source = $lighttpd::source ? {
    ''          => undef,
                                                             file { 'lighttpd.conf':
    default     => $lighttpd::source,
                                                               ensure => $lighttpd::manage_file,
  }
                                                               path    => $lighttpd::config_file, [...]
                                                               notify => $lighttpd::manage_service_autorestart,
  $manage_file_content = $lighttpd::template ? {
                                                               source => $lighttpd::manage_file_source,
    ''        => undef,
                                                               content => $lighttpd::manage_file_content,
    default   => template($lighttpd::template),
                                                               replace => $lighttpd::manage_file_replace,
  }
                                                             }
  [...]=>
Explore the module's logic 4/4
less lighttpd/manifests/init.pp
[...]
  # The whole lighttpd configuration directory can be recursively overriden
  if $lighttpd::source_dir {
    file { 'lighttpd.dir':
      ensure => directory,
      path    => $lighttpd::config_dir,
      require => Package['lighttpd'],
      notify => $lighttpd::manage_service_autorestart,
      source => $lighttpd::source_dir,
      recurse => true,
      purge   => $lighttpd::bool_source_dir_purge,
      replace => $lighttpd::manage_file_replace,
      audit   => $lighttpd::manage_audit,
    }
  }

  ### Include custom class if $my_class is set
  if $lighttpd::my_class {
                                                 # With
    include $lighttpd::my_class                  class { 'lighttpd':
  }                                                my_class => 'example42::my_lighttpd',
                                                 }
  ### Provide puppi data, if puppi is enabled
  if $lighttpd::bool_puppi == true {             # You can autoload example42/manifests/my_lighttpd.pp:
    $classvars=get_class_args()                  class example42::my_lighttpd {
    puppi::ze { 'lighttpd':                        # My extra custom resources
      ensure    => $lighttpd::manage_file,
                                                 }
      variables => $classvars,
      helper    => $lighttpd::puppi_helper,
    }
  }
Add a parameter to the class
vi lighttpd/manifests/params.pp
### Module specific parameters
$use_ssl = false


vi lighttpd/manifests/init.pp
# == Parameters
# [*use_ssl*]
#   Set to true to activate ssl.
#   In order to use this option you need to use a template that honours it:
#   template => 'site/lighttpd/lighttpd.conf.erb',
#   (cp lighttpd/templates/lighttpd.conf.erb site/templates/lighttpd/lighttpd.conf.erb)
[...]

class lighttpd (
  $use_ssl            = params_lookup( 'use_ssl' ),
  ) inherits lighttpd::params {
[...]

  $bool_use_ssl=any2bool($use_ssl)
[...]

  ### Include ssl subclass (if it were of any use)
  # if $lighttpd::bool_use_ssl == true {
  # include $lighttpd::ssl
  # fi
                                          git status
                                          #! modified:         manifests/init.pp
                                          #! modified:         manifests/params.pp

                                          git add .
                                          git commit -m "Added (mostly useless) use_ssl parameter"
Create a sample template
vi lighttpd/templates/lighttpd.conf.erb
# File Managed by Puppet
#
# Sample /etc/lighttpd/lighttpd.conf based on Centos6 layout
#
var.log_root    = "<%= scope.lookupvar('lighttpd::log_dir') %>"
var.server_root = "<%= scope.lookupvar('lighttpd::data_dir') %>"
var.conf_dir    = "<%= scope.lookupvar('lighttpd::conf_dir') %>"
[...]

<% if bool_use_ssl == true %>
  ssl.engine = "enable"
  ssl.pemfile = "/path/to/server.pem"
<% end %>
[...]

server.max-connections = <%= scope.function_options_lookup(['server.max-connections',‘1024’]) %>
server.server.max-keep-alive-idle = <%= scope.function_options_lookup(['server.max-keep-alive-idle',‘5’]) %>
server.max-request-size = <%= scope.function_options_lookup(['server.max-request-size',‘0’]) %>
server.max-read-idle = <%= scope.function_options_lookup(['server.max-read-idle',‘60’]) %>
server.max-write-idle = <%= scope.function_options_lookup(['server.max-write-idle',‘360’]) %>
server.kbytes-per-second = <%= scope.function_options_lookup(['server.kbytes-per-second',‘128’]) %>
server.connection.kbytes-per-second = <%= scope.function_options_lookup(['connection.kbytes-per-second',‘32’]) %>




                    git status
                    #! modified:         templates/lighttpd.conf.erb

                    git add .
                    git commit -m "Added sample lighttpd.conf template (not used by default)"
Add a define 1/2
vi lighttpd/manifests/dotconf.pp
define lighttpd::dotconf (
  $source   = '' ,
  $template = '' ,
  $options = '',
  $ensure   = present ) {

    $manage_file_source = $source ? {
      ''        => undef,
      default   => $source,
    }

    $manage_file_content = $template ? {
      ''        => undef,
      default   => template($template),
    }

    file { "Lighttpd_$name.conf":
      ensure => $ensure,
      path    => "${lighttpd::config_dir}/${name}.conf",
      mode    => $lighttpd::config_file_mode,
      owner   => $lighttpd::config_file_owner,
      group   => $lighttpd::config_file_group,
      require => Package['lighttpd'],
      notify => $lighttpd::manage_service_autorestart,
      source => $manage_file_source,
      content => $manage_file_content,
      audit   => $lighttpd::manage_audit,
    }

                       git status
}
                       #! modified:         manifests/dotconf.pp
Add rspec test for a define
vi spec/defines/lighttpd_dotconf_spec.rb
require "#{File.join(File.dirname(__FILE__),'..','spec_helper.rb')}"
describe 'lighttpd::dotconf' do
  let(:title) { 'lighttpd::dotconf' }
  let(:node) { 'rspec.example42.com' }
  let(:facts) { { :arch => 'i386' , :operatingsystem => 'redhat' } }
  let(:params) {
    { 'ensure'       => 'present',
      'name'         => 'www.example42.com',
      'source'       => 'puppet:///modules/site/lighttpd/www.example42.com.conf',
    }
  }
  describe 'Test lighttpd::dotconf' do
    it 'should create a lighttpd::dotconf' do
      should contain_file('Lighttpd_www.example42.com.conf').with_ensure('present')
    end
  end
  describe 'Test lighttpd::dotconf source parameter' do
    it 'should create a lighttpd::dotconf' do
      content = catalogue.resource('file', 'Lighttpd_www.example42.com.conf').send(:parameters)[:source]
      content.should == "puppet:///modules/site/lighttpd/www.example42.com.conf"
    end
  end
  describe 'Test lighttpd::virtualhost decommissioning' do
    let(:facts) { { :arch => 'i386' , :operatingsystem => 'ubuntu' } }
    let(:params) {
      { 'ensure'       => 'absent'        git status
      }                                   #! modified:        spec/defines/lighttpd_dotconf_spec.rb
    }
    it 'should remove a lighttpd::dotconf file with ensure => absent' do
                                          git add .
      should contain_file('Lighttpd_www.example42.com.conf').with_ensure('absent')
    end                                   git commit -m "Added lighttpd::dotconf spec tests"
  end
Run Tests
cd lighttpd;./Example42-tools/check-module.sh
############################
### Executing rake tasks ###
############################
/usr/local/rvm/rubies/ruby-1.8.7-head/bin/ruby -S rspec spec/classes/lighttpd_spec.rb --format doc --color
Run options: exclude {:broken=>true}
lighttpd
  Test standard installation
    should contain Package[lighttpd] with ensure => "present"
    should contain Service[lighttpd] with ensure => "running"
    should contain Service[lighttpd] with enable => "true"
    should contain File[lighttpd.conf] with ensure => "present"
  Test installation of a specific version
    should contain Package[lighttpd] with ensure => "1.0.42"
  Test standard installation with monitoring and firewalling
    should contain Package[lighttpd] with ensure => "present"
    should contain Service[lighttpd] with ensure => "running"
    should contain Service[lighttpd] with enable => "true"
    should contain File[lighttpd.conf] with ensure => "present"
    should monitor the process
    should place a firewall rule
[...]

############################
### Executing puppetlint ###
############################
### ./manifests/init.pp
WARNING: line has more than 80 characters on line 108
WARNING: line has more than 80 characters on line 437
### ./manifests/params.pp
### ./manifests/spec.pp
Job Done
Module is ready to use:
node test {
  class { 'lighttpd':
    # Custom configuration provided as a template created in:
    # MODULEPATH/site/templates/lighttpd/lighttpd.conf.erb
    template => 'site/lighttpd/lighttpd.conf.erb'
    # Custom options used in template:
    options => {
      'server.kbytes-per-second'   => 254,
      'server.max-keep-alive-idle' => 3,
    }
  }
}



            git remote add origin https://github.com/example42/puppet-lighttpd.git

            git push -u origin master
Was it too fast?
Get the module:
http://github.com/example42/
puppet-lighttpd.git


Review these slides:
http://slideshare.net/alvagante

Follow Up on Twitter:
@alvagante
1 of 20

More Related Content

More from Alessandro Franceschi(19)

ReUse Your (Puppet) Modules!ReUse Your (Puppet) Modules!
ReUse Your (Puppet) Modules!
Alessandro Franceschi991 views
Ten years of [Puppet] installations. What now?Ten years of [Puppet] installations. What now?
Ten years of [Puppet] installations. What now?
Alessandro Franceschi585 views
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction Kit
Alessandro Franceschi1.6K views
Puppet Continuous Integration with PE and GitLabPuppet Continuous Integration with PE and GitLab
Puppet Continuous Integration with PE and GitLab
Alessandro Franceschi1.1K views
Puppet control-repo 
to the next levelPuppet control-repo 
to the next level
Puppet control-repo 
to the next level
Alessandro Franceschi2.1K views
Tp install anythingTp install anything
Tp install anything
Alessandro Franceschi1.6K views
Puppet: From 0 to 100 in 30 minutesPuppet: From 0 to 100 in 30 minutes
Puppet: From 0 to 100 in 30 minutes
Alessandro Franceschi1.6K views
Puppet evolutionsPuppet evolutions
Puppet evolutions
Alessandro Franceschi10.3K views
Essential applications management with Tiny PuppetEssential applications management with Tiny Puppet
Essential applications management with Tiny Puppet
Alessandro Franceschi2.1K views
Raise the bar! ReloadedRaise the bar! Reloaded
Raise the bar! Reloaded
Alessandro Franceschi3.2K views
Raise the bar!Raise the bar!
Raise the bar!
Alessandro Franceschi1.4K views
Anatomy of a reusable moduleAnatomy of a reusable module
Anatomy of a reusable module
Alessandro Franceschi3.3K views
Puppet modules for Fun and ProfitPuppet modules for Fun and Profit
Puppet modules for Fun and Profit
Alessandro Franceschi3.7K views
Puppet modules: A Holistic Approach - GenevaPuppet modules: A Holistic Approach - Geneva
Puppet modules: A Holistic Approach - Geneva
Alessandro Franceschi1.4K views
Puppet modules: An Holistic ApproachPuppet modules: An Holistic Approach
Puppet modules: An Holistic Approach
Alessandro Franceschi3.1K views
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with Puppet
Alessandro Franceschi4.2K views
Spaghetti devopsSpaghetti devops
Spaghetti devops
Alessandro Franceschi3.7K views
Puppi. Puppet strings to the shellPuppi. Puppet strings to the shell
Puppi. Puppet strings to the shell
Alessandro Franceschi5.8K views
Puppet @ SeatPuppet @ Seat
Puppet @ Seat
Alessandro Franceschi5.9K views

Recently uploaded(20)

[2023] Putting the R! in R&D.pdf[2023] Putting the R! in R&D.pdf
[2023] Putting the R! in R&D.pdf
Eleanor McHugh31 views

How to make an Example42 nextgen module in 5 minutes

  • 1. How to Create a Next-Gen Example42 Puppet Module in 5 Minutes Prepare for an info overload rush and an Ignite's verbosity world record Alessandro Franceschi DevOps Days Rome 2012
  • 2. Example42 Next-Gen modules... Support for different OS (Default: RedHat / Debian) Fully parametrized classes Data separation with Hiera/ENC parameters lookup Extreme customization options Module behavior customization options Full and partial decommissioning support Optional automatic Monitoring / Firewalling Optional Puppi integration Optional Debug and Auditing
  • 3. Fetch the sources git clone --recursive git://github.com/example42/ puppet-modules-nextgen.git Cloning into 'puppet-modules-nextgen'... remote: Counting objects: 608, done. remote: Compressing objects: 100% (317/317), done. remote: Total 608 (delta 281), reused 578 (delta 252) Receiving objects: 100% (608/608), 102.59 KiB | 118 KiB/s, done. Resolving deltas: 100% (281/281), done. Submodule 'Example42-documentation' (git://github.com/example42/ Example42-documentation.git) registered for path 'Example42- documentation' Submodule 'Example42-tools' (git://github.com/example42/Example42- tools.git) registered for path 'Example42-tools' [...]
  • 4. Enjoy modules' variety cd puppet-modules-nextgen ; ls Example42-documentation! maven! ! ! ! puppi Example42-tools! ! ! mcollective! ! ! rclocal Gemfile!! ! ! ! monitor! ! ! ! redis README.rdoc! ! ! ! munin! ! ! ! resolver Rakefile! ! ! ! ! mysql! ! ! ! rsync activemq! ! ! ! ! nagios! ! ! ! rsyncssh apache! ! ! ! ! nginx! ! ! ! rvm concat! ! ! ! ! nrpe!! ! ! ! solr example42! ! ! ! ntp! ! ! ! ! splunk firewall! ! ! ! ! openntpd!! ! ! stdlib42 foo!! ! ! ! ! openssh! ! ! ! sudo foo_webapp! ! ! ! openvpn! ! ! ! tartarus foreman!! ! ! ! orientdb!! ! ! tftp haproxy!! ! ! ! pentaho! ! ! ! tomcat icinga! ! ! ! ! php! ! ! ! ! vagrant iptables! ! ! ! ! postfix! ! ! ! vsftpd java! ! ! ! ! postgresql! ! ! wget jboss! ! ! ! ! pupmod-concat! ! wordpress jenkins!! ! ! ! puppet! ! ! ! xinetd libvirt!! ! ! ! puppetdashboard! ! yum logstash! ! ! ! ! puppetdb
  • 5. Create a new module from foo Example42-tools/module_clone.sh This script creates a skeleton for a new module based on different Example42 foo module templates. [...] Enter the name of the new module based on foo: lighttpd COPYING MODULE building file list ... done Modulefile README.rdoc Rakefile manifests/ manifests/init.pp manifests/params.pp manifests/spec.pp spec/ spec/spec_helper.rb spec/classes/ spec/classes/foo_spec.rb [...] RENAMING FILES Renamed lighttpd/spec/classes/foo_spec.rb to lighttpd/spec/classes/lighttpd_spec.rb --------------------------------------------------- CHANGING FILE CONTENTS Changed lighttpd/manifests/init.pp Changed lighttpd/manifests/params.pp Changed lighttpd/manifests/spec.pp Changed lighttpd/Modulefile Changed lighttpd/README.rdoc Changed lighttpd/spec/classes/lighttpd_spec.rb Changed lighttpd/spec/spec_helper.rb Module lighttpd created Start to edit lighttpd/manifests/params.pp to customize it
  • 6. Explore module's contents find lighttpd/ lighttpd/ git init git add . git commit -m "Example42-tools/module_clone.sh lighttpd" lighttpd/spec lighttpd/spec/spec_helper.rb lighttpd/spec/classes lighttpd/spec/classes/lighttpd_spec.rb lighttpd/Rakefile lighttpd/Modulefile lighttpd/manifests lighttpd/manifests/init.pp lighttpd/manifests/params.pp lighttpd/manifests/spec.pp lighttpd/README.rdoc lighttpd/templates lighttpd/templates/spec.erb
  • 7. Edit params.pp 1/2 vi lighttpd/manifests/params.pp class lighttpd::params { $package = $::operatingsystem ? { default => 'lighttpd', } $service = $::operatingsystem ? { default => 'lighttpd', } [...] $config_dir = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/etc/lighttpd/conf-enabled', default => '/etc/lighttpd/conf.d', } $config_file_init = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/etc/default/lighttpd', default => '/etc/sysconfig/lighttpd', } $data_dir = $::operatingsystem ? { /(?i:Debian|Ubuntu|Mint)/ => '/var/run/lighttpd', default => '/var/www', } $log_dir = $::operatingsystem ? { default => '/var/log/lighttpd', } $log_file = $::operatingsystem ? { default => '/var/log/lighttpd.log', } $port = '80' $protocol = 'tcp'
  • 8. Edit params.pp 2/2 vi lighttpd/manifests/params.pp # You should not need to change the following ones class lighttpd::params { [...] # General Settings $my_class = '' $source = '' $source_dir = '' $source_dir_purge = false $template = '' $options = '' $service_autorestart = true $version = 'present' $absent = false $disable = false $disableboot = false ### General module variables that can have a site or per module default $monitor = false $monitor_tool = '' git status $monitor_target = $::ipaddress $firewall = false #! modified: manifests/params.pp $firewall_tool = '' $firewall_src = '0.0.0.0/0' git add . $firewall_dst = $::ipaddress git commit -m "Basic support for Ubuntu/RedHat families" $puppi = false $puppi_helper = 'standard' $debug = false $audit_only = false
  • 9. Review and sign module vi lighttpd/README.rdoc = Puppet module: lighttpd This is a Puppet module for lighttpd based on the second generation layout ("NextGen") of Example42 Puppet Modules. Made by Alvagante / Example42 Official site: http://www.example42.com Official git repository: http://github.com/example42/puppet-lighttpd Released under the terms of Apache 2 License. == USAGE - Basic management * Install lighttpd with default settings class { 'lighttpd': } * Use custom source directory for the whole configuration dir class { 'lighttpd': source_dir => 'puppet:///modules/example42/lighttpd/conf/', } * Use custom template for main config file. Note that template and source arguments are alternative. class { 'lighttpd': template => 'example42/lighttpd/lighttpd.conf.erb', } * Automatically include a custom subclass git status class { 'lighttpd': #! modified: README.rdoc my_class => 'example42::my_lighttpd', } git add . git commit -m "Changed module's author to ... me" * Activate automatic monitoring class { 'lighttpd': monitor => true, monitor_tool => [ 'nagios' , 'monit' , 'munin' ], }
  • 10. Explore the module's logic 1/4 less lighttpd/manifests/init.pp # = Class: lighttpd # == Parameters # # [*my_class*] # Name of a custom class to autoload to manage module's customizations # If defined, lighttpd class will automatically "include $my_class" # Can be defined also by the (top scope) variable $lighttpd_myclass # # [*source*] # Sets the content of source parameter for main configuration file # If defined, lighttpd main config file will have the param: source => $source # Can be defined also by the (top scope) variable $lighttpd_source [...] class lighttpd ( $my_class = params_lookup( 'my_class' ), $source = params_lookup( 'source' ), $source_dir = params_lookup( 'source_dir' ), $source_dir_purge = params_lookup( 'source_dir_purge' ), $template = params_lookup( 'template' ), $service_autorestart = params_lookup( 'service_autorestart' , 'global' ), $options = params_lookup( 'options' ), $version = params_lookup( 'version' ), $disable = params_lookup( 'disable' ), [...] $data_dir = params_lookup( 'data_dir' ), $log_dir = params_lookup( 'log_dir' ), $log_file = params_lookup( 'log_file' ), $port = params_lookup( 'port' ), $protocol = params_lookup( 'protocol' ) ) inherits lighttpd::params {
  • 11. Explore the module's logic 2/4 less lighttpd/manifests/init.pp class lighttpd ( [...] ) inherits lighttpd::params { $bool_source_dir_purge=any2bool($source_dir_purge) $bool_service_autorestart=any2bool($service_autorestart) $bool_absent=any2bool($absent) $bool_disable=any2bool($disable) $bool_disableboot=any2bool($disableboot) $bool_monitor=any2bool($monitor) [...] ### Definition of some variables used in the module $manage_package = $lighttpd::bool_absent ? { true => 'absent', false => $lighttpd::version, } $manage_service_enable = $lighttpd::bool_disableboot ? { true => false, default => $lighttpd::bool_disable ? { true => false, default => $lighttpd::bool_absent ? { true => false, false => true, }, }, } [...]
  • 12. Explore the module's logic 3/4 less lighttpd/manifests/init.pp class lighttpd ( [...] ) inherits lighttpd::params { $manage_package = $lighttpd::bool_absent ? { true => 'absent', false => $lighttpd::version, <=[...] } package { 'lighttpd': ensure => $lighttpd::manage_package, $manage_service_enable = $lighttpd::bool_disableboot ? { name => $lighttpd::package, true => false, } default => $lighttpd::bool_disable ? { true => false, service { 'lighttpd': default => $lighttpd::bool_absent ? { ensure => $lighttpd::manage_service_ensure, true => false, name => $lighttpd::service, false => true, enable => $lighttpd::manage_service_enable, }, hasstatus => $lighttpd::service_status, }, pattern => $lighttpd::process, } require => Package['lighttpd'], [...] } $manage_file_source = $lighttpd::source ? { '' => undef, file { 'lighttpd.conf': default => $lighttpd::source, ensure => $lighttpd::manage_file, } path => $lighttpd::config_file, [...] notify => $lighttpd::manage_service_autorestart, $manage_file_content = $lighttpd::template ? { source => $lighttpd::manage_file_source, '' => undef, content => $lighttpd::manage_file_content, default => template($lighttpd::template), replace => $lighttpd::manage_file_replace, } } [...]=>
  • 13. Explore the module's logic 4/4 less lighttpd/manifests/init.pp [...] # The whole lighttpd configuration directory can be recursively overriden if $lighttpd::source_dir { file { 'lighttpd.dir': ensure => directory, path => $lighttpd::config_dir, require => Package['lighttpd'], notify => $lighttpd::manage_service_autorestart, source => $lighttpd::source_dir, recurse => true, purge => $lighttpd::bool_source_dir_purge, replace => $lighttpd::manage_file_replace, audit => $lighttpd::manage_audit, } } ### Include custom class if $my_class is set if $lighttpd::my_class { # With include $lighttpd::my_class class { 'lighttpd': } my_class => 'example42::my_lighttpd', } ### Provide puppi data, if puppi is enabled if $lighttpd::bool_puppi == true { # You can autoload example42/manifests/my_lighttpd.pp: $classvars=get_class_args() class example42::my_lighttpd { puppi::ze { 'lighttpd': # My extra custom resources ensure => $lighttpd::manage_file, } variables => $classvars, helper => $lighttpd::puppi_helper, } }
  • 14. Add a parameter to the class vi lighttpd/manifests/params.pp ### Module specific parameters $use_ssl = false vi lighttpd/manifests/init.pp # == Parameters # [*use_ssl*] # Set to true to activate ssl. # In order to use this option you need to use a template that honours it: # template => 'site/lighttpd/lighttpd.conf.erb', # (cp lighttpd/templates/lighttpd.conf.erb site/templates/lighttpd/lighttpd.conf.erb) [...] class lighttpd ( $use_ssl = params_lookup( 'use_ssl' ), ) inherits lighttpd::params { [...] $bool_use_ssl=any2bool($use_ssl) [...] ### Include ssl subclass (if it were of any use) # if $lighttpd::bool_use_ssl == true { # include $lighttpd::ssl # fi git status #! modified: manifests/init.pp #! modified: manifests/params.pp git add . git commit -m "Added (mostly useless) use_ssl parameter"
  • 15. Create a sample template vi lighttpd/templates/lighttpd.conf.erb # File Managed by Puppet # # Sample /etc/lighttpd/lighttpd.conf based on Centos6 layout # var.log_root = "<%= scope.lookupvar('lighttpd::log_dir') %>" var.server_root = "<%= scope.lookupvar('lighttpd::data_dir') %>" var.conf_dir = "<%= scope.lookupvar('lighttpd::conf_dir') %>" [...] <% if bool_use_ssl == true %> ssl.engine = "enable" ssl.pemfile = "/path/to/server.pem" <% end %> [...] server.max-connections = <%= scope.function_options_lookup(['server.max-connections',‘1024’]) %> server.server.max-keep-alive-idle = <%= scope.function_options_lookup(['server.max-keep-alive-idle',‘5’]) %> server.max-request-size = <%= scope.function_options_lookup(['server.max-request-size',‘0’]) %> server.max-read-idle = <%= scope.function_options_lookup(['server.max-read-idle',‘60’]) %> server.max-write-idle = <%= scope.function_options_lookup(['server.max-write-idle',‘360’]) %> server.kbytes-per-second = <%= scope.function_options_lookup(['server.kbytes-per-second',‘128’]) %> server.connection.kbytes-per-second = <%= scope.function_options_lookup(['connection.kbytes-per-second',‘32’]) %> git status #! modified: templates/lighttpd.conf.erb git add . git commit -m "Added sample lighttpd.conf template (not used by default)"
  • 16. Add a define 1/2 vi lighttpd/manifests/dotconf.pp define lighttpd::dotconf ( $source = '' , $template = '' , $options = '', $ensure = present ) { $manage_file_source = $source ? { '' => undef, default => $source, } $manage_file_content = $template ? { '' => undef, default => template($template), } file { "Lighttpd_$name.conf": ensure => $ensure, path => "${lighttpd::config_dir}/${name}.conf", mode => $lighttpd::config_file_mode, owner => $lighttpd::config_file_owner, group => $lighttpd::config_file_group, require => Package['lighttpd'], notify => $lighttpd::manage_service_autorestart, source => $manage_file_source, content => $manage_file_content, audit => $lighttpd::manage_audit, } git status } #! modified: manifests/dotconf.pp
  • 17. Add rspec test for a define vi spec/defines/lighttpd_dotconf_spec.rb require "#{File.join(File.dirname(__FILE__),'..','spec_helper.rb')}" describe 'lighttpd::dotconf' do let(:title) { 'lighttpd::dotconf' } let(:node) { 'rspec.example42.com' } let(:facts) { { :arch => 'i386' , :operatingsystem => 'redhat' } } let(:params) { { 'ensure' => 'present', 'name' => 'www.example42.com', 'source' => 'puppet:///modules/site/lighttpd/www.example42.com.conf', } } describe 'Test lighttpd::dotconf' do it 'should create a lighttpd::dotconf' do should contain_file('Lighttpd_www.example42.com.conf').with_ensure('present') end end describe 'Test lighttpd::dotconf source parameter' do it 'should create a lighttpd::dotconf' do content = catalogue.resource('file', 'Lighttpd_www.example42.com.conf').send(:parameters)[:source] content.should == "puppet:///modules/site/lighttpd/www.example42.com.conf" end end describe 'Test lighttpd::virtualhost decommissioning' do let(:facts) { { :arch => 'i386' , :operatingsystem => 'ubuntu' } } let(:params) { { 'ensure' => 'absent' git status } #! modified: spec/defines/lighttpd_dotconf_spec.rb } it 'should remove a lighttpd::dotconf file with ensure => absent' do git add . should contain_file('Lighttpd_www.example42.com.conf').with_ensure('absent') end git commit -m "Added lighttpd::dotconf spec tests" end
  • 18. Run Tests cd lighttpd;./Example42-tools/check-module.sh ############################ ### Executing rake tasks ### ############################ /usr/local/rvm/rubies/ruby-1.8.7-head/bin/ruby -S rspec spec/classes/lighttpd_spec.rb --format doc --color Run options: exclude {:broken=>true} lighttpd Test standard installation should contain Package[lighttpd] with ensure => "present" should contain Service[lighttpd] with ensure => "running" should contain Service[lighttpd] with enable => "true" should contain File[lighttpd.conf] with ensure => "present" Test installation of a specific version should contain Package[lighttpd] with ensure => "1.0.42" Test standard installation with monitoring and firewalling should contain Package[lighttpd] with ensure => "present" should contain Service[lighttpd] with ensure => "running" should contain Service[lighttpd] with enable => "true" should contain File[lighttpd.conf] with ensure => "present" should monitor the process should place a firewall rule [...] ############################ ### Executing puppetlint ### ############################ ### ./manifests/init.pp WARNING: line has more than 80 characters on line 108 WARNING: line has more than 80 characters on line 437 ### ./manifests/params.pp ### ./manifests/spec.pp
  • 19. Job Done Module is ready to use: node test { class { 'lighttpd': # Custom configuration provided as a template created in: # MODULEPATH/site/templates/lighttpd/lighttpd.conf.erb template => 'site/lighttpd/lighttpd.conf.erb' # Custom options used in template: options => { 'server.kbytes-per-second' => 254, 'server.max-keep-alive-idle' => 3, } } } git remote add origin https://github.com/example42/puppet-lighttpd.git git push -u origin master
  • 20. Was it too fast? Get the module: http://github.com/example42/ puppet-lighttpd.git Review these slides: http://slideshare.net/alvagante Follow Up on Twitter: @alvagante