0
Automating Monitoring                   with PuppetChris MagueMoovwebMay 23, 2012                           1
Where I Want to be.....         +                          2
What Ill Settle For.....                            3
RequirementsRock solid stabilityAutomated node addition (discovery)Scales horizontallyService dependency modelsEasy to wri...
Tool Stack             5
What????but #monitoringsucks and #ihatenagios          How could you?                                        6
In defense of Nagios          Been around since 1996          Has Service dependencies          Easy to write plugins     ...
Valid attacks on NagiosNo automated discoveryIts complicated to setupText files – really?Front end wont win any beauty con...
SolutionsUse Icinga!Use Puppet to auto configureStats – leave it to graphite. Its really good at thatBig boys and girls le...
IcingaFork of NagiosConfigurations are compatibleMore solid architecture ( core, API, Web, IDODB )Nice front end, nice mob...
High Level View                  11
Configure Icinga Serversusing Puppet Standard Types                              12
Things to configure with Standard Typesicinga.cfg (file)           => icinga main config fileApache icinga.conf (file)    ...
Configure Icinga using Nagios Types                                      14
Puppet Nagios Typesnagios_command            nagios_hostgroupnagios_contact            nagios_servicenagios_contactgroup  ...
Configuring Hosts                    16
Overview           17
Detailed Overview                    18
Store ConfigsStore puppet info in a DBRetrieve information from DBShare info across nodesUse thin_storeconfigsSet up on pu...
Exporting Nagios_host ResourcesExport = Save to DBUse facter for dynamic dataPRO TIP: use ENCPRO TIP: use targetsPRO TIP: ...
PRO TIP: Use your ENC                        21
PRO TIP: use targetsUse cfg_dir in icinga.cfgCreate a unique file per host or serviceAddition and removal are now super ea...
PRO TIP: hostgroupsAdd machines to a hostgroupAdd services to a hostgroupNew machines inherit all of the services associat...
PRO TIP: use tagsTags allow you to filter resources so that you only realize those resources that you need                ...
Configuring Services/Commands                                25
Icinga ServicesOR Stuff I want to monitorAssociate with a hostgroupUse a target                                           26
Icinga CommandsOR What actually gets runUse Macros to set paths in resource.cfg                                          27
Dependencies               28
PRO TIP: DependenciesUnreliable servicesCut down on the number of alertsTell me whats really wrongRoute alerts accordingly...
Nagios_servicedependencies                             30
NRPE       31
NRPERuns on clientSecured via SSLHas ACLsRuns as nobodyCan run commandsUseful for other things...                         ...
Configuring NRPE                   33
NRPE Checks              34
Plugins          35
exchange.nagios.org                      36
Writing PluginsWrite in any languageOutput 1 line to stdoutNRPE/Icinga/Nagios all use exit codes to determine statusRun by...
Workflows            38
Watching Monitoring                      39
Scheduling Downtime                      40
Filtering            41
Alerting           42
#monitoringisawesomeREMOVE unreliable checksJust MONITOR – dont bolt on - especially statsTIER your monitoringUse timeperi...
44
ResourcesIcingahttp://icinga.orgPuppet http://docs.puppetlabs.com/references/latest/type.html#nagioscommandNRPEhttp://nagi...
Upcoming SlideShare
Loading in...5
×

Automating Monitoring with Puppet

11,413

Published on

Talk on using puppet to configure icinga monitoring

Published in: Technology, Design
2 Comments
21 Likes
Statistics
Notes
  • Hello again,
    About my previous (most recent) comment (question!): Here's a ref to a post by a fellow with the same concern/question: http://marc.info/?l=nagios-devel&m=120794059325327.
    Kind Regards,
    Ian James Gordon
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi,
    Are your hostgroups empty before Puppet collects nagios_hosts? If so, how do you deal with nagios failing to start because there is a service defined for the hostgroup while the hostgroup has no members? If not, what host(s) populate(s) your hostgroup(s) before puppet collects nagios_hosts?
    Kind Regards,
    Ian James Gordon
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
11,413
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
161
Comments
2
Likes
21
Embeds 0
No embeds

No notes for slide

Transcript of "Automating Monitoring with Puppet"

  1. 1. Automating Monitoring with PuppetChris MagueMoovwebMay 23, 2012 1
  2. 2. Where I Want to be..... + 2
  3. 3. What Ill Settle For..... 3
  4. 4. RequirementsRock solid stabilityAutomated node addition (discovery)Scales horizontallyService dependency modelsEasy to write pluginsPromotes sane workflowsUnified front end viewFlexible configuration 4
  5. 5. Tool Stack 5
  6. 6. What????but #monitoringsucks and #ihatenagios How could you? 6
  7. 7. In defense of Nagios Been around since 1996 Has Service dependencies Easy to write plugins Easy-ish to troubleshoot ROCK SOLID 7
  8. 8. Valid attacks on NagiosNo automated discoveryIts complicated to setupText files – really?Front end wont win any beauty contestsDevelopment is slowStats collection is a PITA 8
  9. 9. SolutionsUse Icinga!Use Puppet to auto configureStats – leave it to graphite. Its really good at thatBig boys and girls learn their tools 9
  10. 10. IcingaFork of NagiosConfigurations are compatibleMore solid architecture ( core, API, Web, IDODB )Nice front end, nice mobile front endCan use NRPE 10
  11. 11. High Level View 11
  12. 12. Configure Icinga Serversusing Puppet Standard Types 12
  13. 13. Things to configure with Standard Typesicinga.cfg (file) => icinga main config fileApache icinga.conf (file) => http access to each servercgiauth.cfg (file) => cgi accesscgi.cfg (file) => options, userstemplates.cfg (file) got lazy => use for basic classesidomod.cfg (template) => template for hostname to DB 13
  14. 14. Configure Icinga using Nagios Types 14
  15. 15. Puppet Nagios Typesnagios_command nagios_hostgroupnagios_contact nagios_servicenagios_contactgroup nagios_servicedependencynagios_host nagios_serviceescalationnagios_hostdependency nagios_serviceextinfonagios_hostescalation nagios_servicegroupnagios_hostextinfo nagios_timeperiod 15
  16. 16. Configuring Hosts 16
  17. 17. Overview 17
  18. 18. Detailed Overview 18
  19. 19. Store ConfigsStore puppet info in a DBRetrieve information from DBShare info across nodesUse thin_storeconfigsSet up on puppet master 19
  20. 20. Exporting Nagios_host ResourcesExport = Save to DBUse facter for dynamic dataPRO TIP: use ENCPRO TIP: use targetsPRO TIP: hostgroupsPRO TIP: use tags 20
  21. 21. PRO TIP: Use your ENC 21
  22. 22. PRO TIP: use targetsUse cfg_dir in icinga.cfgCreate a unique file per host or serviceAddition and removal are now super easyAlso default dirs are in a horrible place /etc/nagios 22
  23. 23. PRO TIP: hostgroupsAdd machines to a hostgroupAdd services to a hostgroupNew machines inherit all of the services associated with a hostgroup 23
  24. 24. PRO TIP: use tagsTags allow you to filter resources so that you only realize those resources that you need 24
  25. 25. Configuring Services/Commands 25
  26. 26. Icinga ServicesOR Stuff I want to monitorAssociate with a hostgroupUse a target 26
  27. 27. Icinga CommandsOR What actually gets runUse Macros to set paths in resource.cfg 27
  28. 28. Dependencies 28
  29. 29. PRO TIP: DependenciesUnreliable servicesCut down on the number of alertsTell me whats really wrongRoute alerts accordingly 29
  30. 30. Nagios_servicedependencies 30
  31. 31. NRPE 31
  32. 32. NRPERuns on clientSecured via SSLHas ACLsRuns as nobodyCan run commandsUseful for other things... 32
  33. 33. Configuring NRPE 33
  34. 34. NRPE Checks 34
  35. 35. Plugins 35
  36. 36. exchange.nagios.org 36
  37. 37. Writing PluginsWrite in any languageOutput 1 line to stdoutNRPE/Icinga/Nagios all use exit codes to determine statusRun by hand to check 37
  38. 38. Workflows 38
  39. 39. Watching Monitoring 39
  40. 40. Scheduling Downtime 40
  41. 41. Filtering 41
  42. 42. Alerting 42
  43. 43. #monitoringisawesomeREMOVE unreliable checksJust MONITOR – dont bolt on - especially statsTIER your monitoringUse timeperiods for sanityDelegate responsesUse dependencies to pin down problems quicklyWork smart 43
  44. 44. 44
  45. 45. ResourcesIcingahttp://icinga.orgPuppet http://docs.puppetlabs.com/references/latest/type.html#nagioscommandNRPEhttp://nagios.sourceforge.net/docs/3_0/addons.htmlIRC##infra-talk, #icinga, #puppetContactchristianmague@gmail.com, @maguec, #gaijin (freenode), http://blog.mague.comThanks 45
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×