Puppi. Puppet strings to the shell

PuppetCamp Europe 2011 27/28 April Amsterdam

What’s Puppi?A Puppet ModuleA Bash CommandA tool to automate deploymentsA SysAdmin friend

puppiUsage: puppi <command> [project|topic] [options]Available commands:check [project] - Run puppi checks host-wide or for projectlog [topic] [-i] - Show system and application specific logsinfo [topic] [-i] - Show informations about the systeminit <project> - First time project initialization and setupdeploy <project> - Deploy the specified projectrollback <project> - Rollback the specified project.Available options:-f - Force puppi commands execution flow also on CRITICAL errors-i - Interactively ask confirmation for every step-t - Test mode. Just show the commands that should be executed-d <yes|full> - Debug mode. Show debug of what is done.-o "parameter=value parameter2=value2" - Set manual options to override defaultsAvailable projects:abnormalia.net git.example42.com openskills.info openskills.info_sqlwww.example42.com www.example42.com_sql www.lab42.itAvailable info topics:apache! disks hardware mcollective munin mysql network nrpe ntp! opensshpackages perf postfix puppi rsync! usersAvailable log topics:abnormalia.net! auth git.example42.com mail mcollective munin mysqlopenskills.info rsync system www.example42.com www.lab42.it

puppi checkInstantsystemshealth check

puppi check# Run all local checkspuppi check# Run checks related to myapppuppi check myapp# Checks can be on:# - Running services# - Listening ports# - Pattern match on specific URLs# - General system’s status# - Remote services used by the host## - Whatever a Nagios plugin can check

puppi check# Each check is a Puppet definepuppi::check { "NTP_Sync": command => "check_ntp -H ${puppi::params::ntp_server}" , priority => "20" , hostwide => "yes" ,}puppi::check { "Port_exim_$port": command => "check_tcp -H ${fqdn} -p ${exim::params::port}" ,}puppi::check { "Url_$name": enable => $enable, hostwide => no, project => “myapp”, command => "check_http -I ${target} -p ${port} -u ${url} -s ${pattern}" ,}

puppi infoQuickand focusedinfo from thesystem

puppi info# Show all the info availablepuppi info# Interactive. Select the topics to showpuppi info -i# Check local resourcespuppi info networkpuppi info perf# Module based info sourcespuppi info opensshpuppi info apache# Company and node specific infopuppi info mycompany

puppi infopuppi::info { "network": description => "Network settings and stats" , run => [ "ifconfig”,“route”,“cat /etc resolv.conf”, “netstat -natup|grep LISTEN" ],}puppi::info::module { "openssh": packagename => "${openssh::params::packagename}", servicename => "${openssh::params::servicename}", processname => "${openssh::params::processname}", configfile => "${openssh::params::configfile}", datadir => "${openssh::params::datadir}", logdir => "${openssh::params::logdir}", protocol => "${openssh::params::protocol}", port => "${openssh::params::port}", description => "What Puppet knows about openssh" , run => "ls -la ~/.ssh/",}puppi::info::readme { "mycompany": }

puppi logAll logsin a singlecommand

puppi log# tail -f of all the known logspuppi log# Interactive. CHoose logs to showpuppi log -i# Tail of logs related to myapppuppi log myapp Troubleshoot in the quick way

puppi logclass puppi::logs { puppi::log { "auth": description => "Users and authentication" , log => $operatingsystem ? { Debian,Ubuntu => [ "/var/log/user.log” , “/var/log/auth.log" ], RedHat,CentOS => "/var/log/secure", } } puppi::log { "mail": description => "Mail messages" , log => $operatingsystem ? { Debian,Ubuntu => "/var/log/mail.log", RedHat,CentOS => "/var/log/maillog", } } [...]}

puppi deployAutomatingdeploymentprocedures

puppi deploy# To make this work:puppi deploy www.lab42.it# You write something like:puppi::project::builder { "www.lab42.it": source => "rsync://deploy.${domain}/deploy/www.lab42.it/", init_source => "rsync://deploy.${domain}/init/www.lab42.it", source_type => "dir", deploy_root => "${apache::params::documentroot}/www.lab42.it/", user => "root", disable_services => “apache”, run_checks => “true”, backup => “full”, report_email => "roots@lab42.it", enable => "true",}

puppi deploy# Default sample deploy procedures (can be customized)# Check puppi/manifests/project/*.pppuppi::project::builder # General purpose scenario. # Includes most of the cases belowpuppi::project::war # Deploy a simple warpuppi::project::tar # Deploy a tar.gz filepuppi::project::maven # Deploy Maven artifacts published on a # Nexus repositorypuppi::project::mysql # Retrieve and imports a .sql filepuppi::project::files # Deploy the files defined in a list

puppi deploy# SOME options available in puppi::project::builder# Use them to adapt the default procedures to custom needsdefine puppi::project::builder ( $source, # URI of source files: http://, ssh://, rsync://... $source_type, # Type of source: tarball, zip, war, dir, maven... $deploy_root, # Destination directory $init_source="", # Source for init command $user="root", # User that makes the deploy $predeploy_customcommand="", # Optional pre-deploy command $postdeploy_customcommand="", # Optional post-deploy command $disable_services="", # Services to stop during deploy. $firewall_src_ip="", # Load balancer IP $report_email="", # Email(s) to notify at the end of the run $backup="full", # Backup method for archiving old data $run_checks="true", # If pre and post deploy checks are run [...] ) {

puppi deploy# A deploy procedure contains basic puppi defines:# puppi::deploy, init, project, rollback, report# A sample fragment:puppi::deploy { "${name}-Retrieve_SourceFile": priority => "20" , command => "get_file.sh" , arguments => "-s $source -t $real_source_type" , user => "root" , project => "$name" , enable => $enable ; "${name}-Deploy": priority => "40" , command => "deploy.sh" , arguments => "$deploy_root" , user => "$user" , project => "$name" , enable => $enable;}

puppi deploy# The commands executed can be in any language# By default Puppi provides some native commands for general uses:get_file.sh # Retrieve a file using different protocols: # http://, ssh://, file://, svn://, rsync:// ...archive.sh # Backup and recovery data with various optionsdeploy.sh # Copy files to the deploy directorywait.sh # Wait for events (file presence, content check, time...)predeploy.sh # Prepare files to deployget_metadata.sh # Extract metadata from various sourcesdatabase.sh # Run database queries# These and other scripts are placed in /etc/puppi/scripts and can# be used during the deploy procedure# All the native scripts use and can write to a runtime# configuration file where are stored parameters related# to the deployment.

puppi paths/usr/sbin/puppi # The puppi main command/etc/puppi/ # All puppi configs and scripts/etc/puppi/scripts/ # Where commands are placed/etc/puppi/checks/ # Where checks are defined (Nagios plugins)/etc/puppi/info/ # Where are placed info topic scripts/etc/puppi/logs/ # Where are placed log topic paths/etc/puppi/projects/ # Where are stored deploy projects dirs/etc/puppi/projects/<project_name>/deploy/ # Commands executed # when you type: puppi deploy <project_name>/tmp/puppi/<project_name>/ # Temporary dir used during a deploy/var/lib/puppi/<project_name>/ # Where backups are stored/var/log/puppi/<project_name>/ # Where logs are stored

puppirollbackIf somethingcan go wrong... One command solves

puppi rollback[root@pg01 ~]# puppi rollback www.lab42.itPuppi setup: 00-www.lab42.it-RuntimeConfig-Initialization [ OK ]Choose deploy to rollback:total 52drwxr-xr-x 2 root root 4096 Mar 29 01:21 20110329-012108drwxr-xr-x 2 root root 4096 Mar 29 02:59 20110329-025956drwxr-xr-x 2 root root 4096 Apr 10 22:05 20110410-215942drwxr-xr-x 2 root root 4096 Apr 19 23:55 20110419-235528drwxr-xr-x 2 root root 4096 Apr 20 02:41 20110420-024115drwxr-xr-x 2 root root 4096 Apr 20 02:56 20110420-025621lrwxrwxrwx 1 root root 51 Apr 20 02:56 latest -> /var/lib/puppi/archive/www.lab42.it/20110420-025621 Rollback operations require user’s interaction

puppi initAutomatingfirst timedeployments

puppi init[root@pg02 ~]# puppi init www.lab42.itPuppi setup: 00-www.lab42.it-RuntimeConfig-Initialization [ OK ]pg02 Init: 40-www.lab42.it-Deploy_Files [ OK ]Reporting: 20-www.lab42.it-Mail_Notification [ OK ]REPORT FOR PUPPI - STATUS OKSummary of operations is: /var/log/puppi/www.lab42.it/20110423-005555/summaryDetails are in: /var/log/puppi/www.lab42.it/20110423-005555/Temporary workdir has been: /tmp/puppi/www.lab42.it/ (Will berewritten at the next puppi run)Runtime config file is: /tmp/puppi/www.lab42.it/configFiles have been archived in: /var/lib/puppi/archive/www.lab42.it/20110423-005555

Job done.Notiﬁcation plugins

mail notify# Usage in a puppi::project define report_email => "roots@lab42.it al@lab42.it",# The actual code that makes it puppi::report { "${name}-Mail_Notification": command => "report_mail.sh" , arguments => "$report_email" , project => "$name" , }

mc-puppiExpandingto a widerworld

mc-puppi# Some examples# Distributed real time check of the whole Infrastructuremc-puppi check# Gather network info of all nodesmc-puppi info network# Deploy myapp on all the nodes of the myapp-fe rolemc-puppi -F role=myapp-fe deploy myapp# Instant check on the nodes where you deployedmc-puppi -F role=myapp-fe check# Realtime info on relevant servicesmc-puppi -F role=myapp-fe info apache# Check last log entriesmc-puppi -F role=myapp-fe log apache Bringing puppi commands to MCollective space

mc-puppi

More notification methodsWider OS supportWeb FrontendOrchestra

Dowload from:www.example42.comgithub.com/example42

Graphics by Tatlin www.tatlin.net

http://www.example42.com	958
http://example42.com	486
http://lanyrd.com	92
http://coderwall.com	20
http://www.planetpuppet.org	9
http://wiki.roar.net	9
http://www.slideshare.net	4
http://planetpuppet.org	3
url_unknown	2
http://translate.googleusercontent.com	1
https://lanyrd.com	1
http://www.linkedin.com	1

Puppi. Puppet strings to the shell

by Alessandro Franceschi on Apr 29, 2011

Accessibility

Categories

Upload Details

Usage Rights

12 Embeds 1,586

Statistics

Puppi. Puppet strings to the shell Presentation Transcript