Automating Monitoring with Puppet
Upcoming SlideShare
Loading in...5

Automating Monitoring with Puppet



Talk on using puppet to configure icinga monitoring

Talk on using puppet to configure icinga monitoring



Total Views
Views on SlideShare
Embed Views



7 Embeds 318 298 7 7 2 2 1 1



Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Hello again,
    About my previous (most recent) comment (question!): Here's a ref to a post by a fellow with the same concern/question:
    Kind Regards,
    Ian James Gordon
    Are you sure you want to
    Your message goes here
  • Hi,
    Are your hostgroups empty before Puppet collects nagios_hosts? If so, how do you deal with nagios failing to start because there is a service defined for the hostgroup while the hostgroup has no members? If not, what host(s) populate(s) your hostgroup(s) before puppet collects nagios_hosts?
    Kind Regards,
    Ian James Gordon
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Automating Monitoring with Puppet Automating Monitoring with Puppet Presentation Transcript

  • Automating Monitoring with PuppetChris MagueMoovwebMay 23, 2012 1
  • Where I Want to be..... + 2
  • What Ill Settle For..... 3 View slide
  • RequirementsRock solid stabilityAutomated node addition (discovery)Scales horizontallyService dependency modelsEasy to write pluginsPromotes sane workflowsUnified front end viewFlexible configuration 4 View slide
  • Tool Stack 5
  • What????but #monitoringsucks and #ihatenagios How could you? 6
  • In defense of Nagios Been around since 1996 Has Service dependencies Easy to write plugins Easy-ish to troubleshoot ROCK SOLID 7
  • Valid attacks on NagiosNo automated discoveryIts complicated to setupText files – really?Front end wont win any beauty contestsDevelopment is slowStats collection is a PITA 8
  • SolutionsUse Icinga!Use Puppet to auto configureStats – leave it to graphite. Its really good at thatBig boys and girls learn their tools 9
  • IcingaFork of NagiosConfigurations are compatibleMore solid architecture ( core, API, Web, IDODB )Nice front end, nice mobile front endCan use NRPE 10
  • High Level View 11
  • Configure Icinga Serversusing Puppet Standard Types 12
  • Things to configure with Standard Typesicinga.cfg (file) => icinga main config fileApache icinga.conf (file) => http access to each servercgiauth.cfg (file) => cgi accesscgi.cfg (file) => options, userstemplates.cfg (file) got lazy => use for basic classesidomod.cfg (template) => template for hostname to DB 13
  • Configure Icinga using Nagios Types 14
  • Puppet Nagios Typesnagios_command nagios_hostgroupnagios_contact nagios_servicenagios_contactgroup nagios_servicedependencynagios_host nagios_serviceescalationnagios_hostdependency nagios_serviceextinfonagios_hostescalation nagios_servicegroupnagios_hostextinfo nagios_timeperiod 15
  • Configuring Hosts 16
  • Overview 17
  • Detailed Overview 18
  • Store ConfigsStore puppet info in a DBRetrieve information from DBShare info across nodesUse thin_storeconfigsSet up on puppet master 19
  • Exporting Nagios_host ResourcesExport = Save to DBUse facter for dynamic dataPRO TIP: use ENCPRO TIP: use targetsPRO TIP: hostgroupsPRO TIP: use tags 20
  • PRO TIP: Use your ENC 21
  • PRO TIP: use targetsUse cfg_dir in icinga.cfgCreate a unique file per host or serviceAddition and removal are now super easyAlso default dirs are in a horrible place /etc/nagios 22
  • PRO TIP: hostgroupsAdd machines to a hostgroupAdd services to a hostgroupNew machines inherit all of the services associated with a hostgroup 23
  • PRO TIP: use tagsTags allow you to filter resources so that you only realize those resources that you need 24
  • Configuring Services/Commands 25
  • Icinga ServicesOR Stuff I want to monitorAssociate with a hostgroupUse a target 26
  • Icinga CommandsOR What actually gets runUse Macros to set paths in resource.cfg 27
  • Dependencies 28
  • PRO TIP: DependenciesUnreliable servicesCut down on the number of alertsTell me whats really wrongRoute alerts accordingly 29
  • Nagios_servicedependencies 30
  • NRPE 31
  • NRPERuns on clientSecured via SSLHas ACLsRuns as nobodyCan run commandsUseful for other things... 32
  • Configuring NRPE 33
  • NRPE Checks 34
  • Plugins 35
  • 36
  • Writing PluginsWrite in any languageOutput 1 line to stdoutNRPE/Icinga/Nagios all use exit codes to determine statusRun by hand to check 37
  • Workflows 38
  • Watching Monitoring 39
  • Scheduling Downtime 40
  • Filtering 41
  • Alerting 42
  • #monitoringisawesomeREMOVE unreliable checksJust MONITOR – dont bolt on - especially statsTIER your monitoringUse timeperiods for sanityDelegate responsesUse dependencies to pin down problems quicklyWork smart 43
  • 44
  • ResourcesIcingahttp://icinga.orgPuppet, #icinga,, @maguec, #gaijin (freenode), http://blog.mague.comThanks 45