1. PuppetCamp Europe 2011
27/28 April Amsterdam

2. What’s Puppi?
A Puppet Module
A Bash Command
A tool to automate deployments
A SysAdmin friend

3. puppi

4. puppi
Usage: puppi <command> [project|topic] [options]
Available commands:
check [project] - Run puppi checks host-wide or for project
log [topic] [-i] - Show system and application specific logs
info [topic] [-i] - Show informations about the system
init <project> - First time project initialization and setup
deploy <project> - Deploy the specified project
rollback <project> - Rollback the specified project.
Available options:
-f - Force puppi commands execution flow also on CRITICAL errors
-i - Interactively ask confirmation for every step
-t - Test mode. Just show the commands that should be executed
-d <yes|full> - Debug mode. Show debug of what is done.
-o "parameter=value parameter2=value2" - Set manual options to override defaults
Available projects:
abnormalia.net git.example42.com openskills.info openskills.info_sql
www.example42.com www.example42.com_sql www.lab42.it
Available info topics:
apache! disks hardware mcollective munin mysql network nrpe ntp! openssh
packages perf postfix puppi rsync! users
Available log topics:
abnormalia.net! auth git.example42.com mail mcollective munin mysql
openskills.info rsync system www.example42.com www.lab42.it

5. puppi check
Instant
systems
health check

6. puppi check
# Run all local checks
puppi check
# Run checks related to myapp
puppi check myapp
# Checks can be on:
# - Running services
# - Listening ports
# - Pattern match on specific URLs
# - General system’s status
# - Remote services used by the host
#
# - Whatever a Nagios plugin can check

7. puppi check
# Each check is a Puppet define
puppi::check { "NTP_Sync":
command => "check_ntp -H ${puppi::params::ntp_server}" ,
priority => "20" ,
hostwide => "yes" ,
}
puppi::check { "Port_exim_$port":
command => "check_tcp -H ${fqdn} -p ${exim::params::port}" ,
}
puppi::check { "Url_$name":
enable => $enable,
hostwide => no,
project => “myapp”,
command => "check_http -I '${target}' -p '${port}' -u '$
{url}' -s '${pattern}'" ,
}

8. puppi info
Quick
and focused
info from the
system

9. puppi info
# Show all the info available
puppi info
# Interactive. Select the topics to show
puppi info -i
# Check local resources
puppi info network
puppi info perf
# Module based info sources
puppi info openssh
puppi info apache
# Company and node specific info
puppi info mycompany

10. puppi info
puppi::info { "network":
description => "Network settings and stats" ,
run => [ "ifconfig”,“route”,“cat /etc resolv.conf”,
“netstat -natup|grep LISTEN" ],
}
puppi::info::module { "openssh":
packagename => "${openssh::params::packagename}",
servicename => "${openssh::params::servicename}",
processname => "${openssh::params::processname}",
configfile => "${openssh::params::configfile}",
datadir => "${openssh::params::datadir}",
logdir => "${openssh::params::logdir}",
protocol => "${openssh::params::protocol}",
port => "${openssh::params::port}",
description => "What Puppet knows about openssh" ,
run => "ls -la ~/.ssh/",
}
puppi::info::readme { "mycompany": }

11. puppi log
All logs
in a single
command

12. puppi log
# tail -f of all the known logs
puppi log
# Interactive. CHoose logs to show
puppi log -i
# Tail of logs related to myapp
puppi log myapp
Troubleshoot in the quick way

13. puppi log
class puppi::logs {
puppi::log { "auth":
description => "Users and authentication" ,
log => $operatingsystem ? {
Debian,Ubuntu => [ "/var/log/user.log” ,
“/var/log/auth.log" ],
RedHat,CentOS => "/var/log/secure",
}
}
puppi::log { "mail":
description => "Mail messages" ,
log => $operatingsystem ? {
Debian,Ubuntu => "/var/log/mail.log",
RedHat,CentOS => "/var/log/maillog",
}
}
[...]
}

14. puppi deploy
Automating
deployment
procedures

15. puppi deploy
# To make this work:
puppi deploy www.lab42.it
# You write something like:
puppi::project::builder { "www.lab42.it":
source => "rsync://deploy.${domain}/deploy/www.lab42.it/",
init_source => "rsync://deploy.${domain}/init/www.lab42.it",
source_type => "dir",
deploy_root => "${apache::params::documentroot}/www.lab42.it/",
user => "root",
disable_services => “apache”,
run_checks => “true”,
backup => “full”,
report_email => "roots@lab42.it",
enable => "true",
}

16. puppi deploy
# Default sample deploy procedures (can be customized)
# Check puppi/manifests/project/*.pp
puppi::project::builder # General purpose scenario.
# Includes most of the cases below
puppi::project::war # Deploy a simple war
puppi::project::tar # Deploy a tar.gz file
puppi::project::maven # Deploy Maven artifacts published on a
# Nexus repository
puppi::project::mysql # Retrieve and imports a .sql file
puppi::project::files # Deploy the files defined in a list

17. puppi deploy
# SOME options available in puppi::project::builder
# Use them to adapt the default procedures to custom needs
define puppi::project::builder (
$source, # URI of source files: http://, ssh://, rsync://...
$source_type, # Type of source: tarball, zip, war, dir, maven...
$deploy_root, # Destination directory
$init_source="", # Source for init command
$user="root", # User that makes the deploy
$predeploy_customcommand="", # Optional pre-deploy command
$postdeploy_customcommand="", # Optional post-deploy command
$disable_services="", # Services to stop during deploy.
$firewall_src_ip="", # Load balancer IP
$report_email="", # Email(s) to notify at the end of the run
$backup="full", # Backup method for archiving old data
$run_checks="true", # If pre and post deploy checks are run
[...] ) {

18. puppi deploy
# A deploy procedure contains basic puppi defines:
# puppi::deploy, init, project, rollback, report
# A sample fragment:
puppi::deploy {
"${name}-Retrieve_SourceFile":
priority => "20" , command => "get_file.sh" ,
arguments => "-s $source -t $real_source_type" ,
user => "root" , project => "$name" , enable => $enable ;
"${name}-Deploy":
priority => "40" , command => "deploy.sh" ,
arguments => "$deploy_root" ,
user => "$user" , project => "$name" , enable => $enable;
}

19. puppi deploy
# The commands executed can be in any language
# By default Puppi provides some native commands for general uses:
get_file.sh # Retrieve a file using different protocols:
# http://, ssh://, file://, svn://, rsync:// ...
archive.sh # Backup and recovery data with various options
deploy.sh # Copy files to the deploy directory
wait.sh # Wait for events (file presence, content check, time...)
predeploy.sh # Prepare files to deploy
get_metadata.sh # Extract metadata from various sources
database.sh # Run database queries
# These and other scripts are placed in /etc/puppi/scripts and can
# be used during the deploy procedure
# All the native scripts use and can write to a runtime
# configuration file where are stored parameters related
# to the deployment.

20. puppi paths
/usr/sbin/puppi # The puppi main command
/etc/puppi/ # All puppi configs and scripts
/etc/puppi/scripts/ # Where commands are placed
/etc/puppi/checks/ # Where checks are defined (Nagios plugins)
/etc/puppi/info/ # Where are placed info topic scripts
/etc/puppi/logs/ # Where are placed log topic paths
/etc/puppi/projects/ # Where are stored deploy projects dirs
/etc/puppi/projects/<project_name>/deploy/ # Commands executed
# when you type: puppi deploy <project_name>
/tmp/puppi/<project_name>/ # Temporary dir used during a deploy
/var/lib/puppi/<project_name>/ # Where backups are stored
/var/log/puppi/<project_name>/ # Where logs are stored

21. puppi
rollback
If something
can go wrong...
One command solves

22. puppi rollback
[root@pg01 ~]# puppi rollback www.lab42.it
Puppi setup: 00-www.lab42.it-RuntimeConfig-Initialization [ OK ]
Choose deploy to rollback:
total 52
drwxr-xr-x 2 root root 4096 Mar 29 01:21 20110329-012108
drwxr-xr-x 2 root root 4096 Mar 29 02:59 20110329-025956
drwxr-xr-x 2 root root 4096 Apr 10 22:05 20110410-215942
drwxr-xr-x 2 root root 4096 Apr 19 23:55 20110419-235528
drwxr-xr-x 2 root root 4096 Apr 20 02:41 20110420-024115
drwxr-xr-x 2 root root 4096 Apr 20 02:56 20110420-025621
lrwxrwxrwx 1 root root 51 Apr 20 02:56 latest -> /var/lib/puppi/
archive/www.lab42.it/20110420-025621
Rollback operations require user’s interaction

23. puppi init
Automating
first time
deployments

24. puppi init
[root@pg02 ~]# puppi init www.lab42.it
Puppi setup: 00-www.lab42.it-RuntimeConfig-Initialization [ OK ]
pg02 Init: 40-www.lab42.it-Deploy_Files [ OK ]
Reporting: 20-www.lab42.it-Mail_Notification [ OK ]
REPORT FOR PUPPI - STATUS OK
Summary of operations is: /var/log/puppi/www.lab42.it/
20110423-005555/summary
Details are in: /var/log/puppi/www.lab42.it/20110423-005555/
Temporary workdir has been: /tmp/puppi/www.lab42.it/ (Will be
rewritten at the next puppi run)
Runtime config file is: /tmp/puppi/www.lab42.it/config
Files have been archived in: /var/lib/puppi/archive/www.lab42.it/
20110423-005555

25. Job done.
Notification plugins

26. mail notify
# Usage in a puppi::project define
report_email => "roots@lab42.it al@lab42.it",
# The actual code that makes it
puppi::report {
"${name}-Mail_Notification":
command => "report_mail.sh" ,
arguments => "$report_email" ,
project => "$name" ,
}

27. mc-puppi
Expanding
to a wider
world

28. mc-puppi
# Some examples
# Distributed real time check of the whole Infrastructure
mc-puppi check
# Gather network info of all nodes
mc-puppi info network
# Deploy myapp on all the nodes of the myapp-fe role
mc-puppi -F role=myapp-fe deploy myapp
# Instant check on the nodes where you deployed
mc-puppi -F role=myapp-fe check
# Realtime info on relevant services
mc-puppi -F role=myapp-fe info apache
# Check last log entries
mc-puppi -F role=myapp-fe log apache
Bringing puppi commands to MCollective space

29. mc-puppi

31. More notification methods
Wider OS support
Web Frontend
Orchestra

32. Dowload from:
www.example42.com
github.com/example42

33. Graphics by Tatlin
www.tatlin.net