This document discusses Nagios notification plugins and best practices for programming notifications. It describes various notification methods that can be used both at the office and on the go, such as email, phone, SMS, RSS, instant messaging, and support queues. When programming notifications, it is important to separate content generation from transport, handle timeouts properly without getting killed, think in parallel using locks or Nan, and use standardized macros and environment variables to pass information to the notification script. Notifications should have only one chance to succeed and return value is not used, but returning 0 indicates success.
2. About me
• CAPSiDE: Spanish based consulting company
• Monitoring Managed Services and Hosted
solutions
• Using Opsview
– Frontend for Nagios
– Altinity Partner
– Contributed too ☺
• Small team, highly specialized
3. Notifications are pluggable
• As pluggable as service checks
• Same interface
• Lots of work already done
– Ready to go
– No solution fits your needs
– But you have a good starting point
4. Plugins
• Email
• Telephone
• SMS
• RSS
• Support Queue
• IM
• Twitter (Jaiku)
• SMB
• Sound
• IRC
• Others
• Nagios Exchange (nagiosexchange.org)
• Google them too…
5. Not Plugins
• Windows tray
– Nagios Tray Monitor
– NTray
– NTrayC
– Nestor
• KTLight
• NagiosChecker
8. • Well known
• Simple
• Reliable
• Robust
• Almost no configuration
• Integrations: Your mail
server should be
pluggable too…
• Mail clients poll
@ the office
good old email
18. @ the office
Support Queue
• Integrate events into
your workflow
– RT
– OTRS
• Integrate your support
system with Nagios
– Acknowledge an alert
when a ticket is taken
– Close a ticket if the
alert clears out
• Good for big teams
28. On the go
Calling
• Asterisk
– Mobiles are not always
reachable
• Try many times
• Make sure that a
human answered (voice
mails answer your
phone)
• Have a backup scheme
– Don’t flood your poor
operator
– Call the right person
35. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
36. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
38. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Timeout
Get lock
Has he been called
Less than 15 mins ago?
Write notification succeded
Release lock
Pressed key
39. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Get lock
Has he been called
Less than 15 mins ago?
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeeded
Release lock
40. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Get lock
Has he been called
Less than 15 mins ago?
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeeded
Release lock
41. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Get lock
Has he been called
Less than 15 mins ago?
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeded
Release lock
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Pressed keyTimeout
Timeout
Write notification succeeded
Release lock
42. Programming Notifications
Call number
Make him
Press a key
Next number
Notification
succeeded
Pick up
Timeout
Get lock
Has he been called
Less than 15 mins ago?
Write notification succeded
Release lock
Pressed key
Return to Nagios
43. Programming Notifications
• Separate content generation from the transport
– Think UNIX style
– generate_content | transport_the_message
• echo “$MACRO$ on $MACRO$” | sendmail
• You only have one chance, so do it right
– Nagios will “timeout” your notification script (be
prepared)
• If you should not be killed, detach the process
• Think in parallel (serialize with locks or look at
Nan)
• You cannot return failure (or success)
44. Programming Notifications
• Language neutral
• Info about the notification
– Command Line options (macros) vs Env
Variables
• http://nagios.sourceforge.net/docs/2_0/macros.html
• http://nagios.sourceforge.net/docs/3_0/macrolist.html
• Return value
– Not used (but try to return a 0 if all goes well)