After it was mentioned on Twitter for the first time, it took the #monitoringsucks hashtag only a short time to get widely adopted. And as a long-time Nagios user, I certainly understand why. But at DevOps Days Rome, Ulf Månsson started a counter movement: #monitoringlove. This was after he introduced the Sensu monitoring framework at his company with great success. In my talk, I´m going to explain the basics and details of Sensu and how to get a complete Sensu monitoring system running in only a few hours.

1. #monitoringlove
with Sensu
OSMC 2014, Nuremberg,
Germany

2. http://sensuapp.org

3. Why Sensu?
• Written in clean Ruby
• Scalable architecture
• Plugins in any language
• Can use Nagios checks
• Collects both checks and metrics
• Great community

4. Jochen Lillich
http://freistil.it
@geewiz

5. Sensu Core

6. Sensu Enterprise

7. Installation
• Omnibus packaging
• Configuration in JSON files
• Sensu cookbook for Chef
• Puppet module

10. • Connects all Sensu components
• Asynchronous communication

11. Sensu Server
• Orchestrates check execution
• Processes check results
• Triggers event handlers

12. Sensu Client
• Registers automatically with the Server
• Sends keepalive information
• Receives check execution requests
• Schedules checks locally
• Executes checks
• Publishes check results
• Publishes external events

13. API
• get event data
• get agent data
• trigger check execution
• resolve events
• silence checks

14. Dashboard
• Uchiwa
• sensu-admin

16. Scheduling
• Standard checks (server)
• Standalone checks (client)
• Manual checks (API)

17. {
"checks": {
"disk_free": {
"type": "status",
"subscribers": [ "all" ],
"handlers": [ "default" ],
"command": "/usr/lib/nagios/plugins/check_disk
-w :::disk_warn::: -c :::disk_crit:::
-A -x /dev/shm -X nfs -i /boot",
"interval": 60
}
}
}

18. Checks in Chef
sensu_check 'mysql_server' do
command "/usr/lib/nagios/plugins/check_mysql " +
"-u 'monitoring' " +
"-p '#{node['mysql']['server_mon_password']}'"
handlers ['default']
standalone true
interval 30
end

19. Metrics check
{
"checks": {
"load_metrics": {
"type": "metric",
"command": "load-metrics.rb",
"subscribers": [
"production"
],
"interval": 10
}
}
}

20. Metrics output
$ ruby load-metrics.rb
srv3.local.load_avg.one 0.89 1365270842
srv3.local.load_avg.five 1.01 1365270842
srv3.local.load_avg.fifteen 1.06 1365270842
$ echo $?
0

21. External events
echo '{ "name": "my_check",
"output": "some output",
"status": 0 }'
> /dev/tcp/localhost/3030
Useful: https://github.com/solarkennedy/sensu-shell-helper

23. Handler types
• Pipe
• TCP
• UDP
• Transport
• Sets

24. Common event handlers
• Email
• PagerDuty
• Graphite
• IRC
• Slack

26. Example handler code
#!/usr/bin/env ruby
require 'rubygems'
require 'json'
# Read event data
event = JSON.parse(STDIN.read, :symbolize_names => true)
# Write the event data to a file
file_name = "/tmp/sensu_#{event[:client][:name]}_" +
"#{event[:check][:name]}"
File.open(file_name, 'w') do |file|
file.write(JSON.pretty_generate(event))
end

27. Example handler configuration
{
"handlers": {
"file": {
"type": "pipe",
"command": "/etc/sensu/handlers/file.rb"
}
}
}

28. Sensu CLI
https://github.com/agent462/sensu-cli
• sensu-cli resolve srv3 apache_http
• sensu-cli client delete srv3
• sensu-cli silence srv3 --reason "Shut up already"
--expire 3600

29. #chatops
https://github.com/sensu/sensu-hubot
• sensu events summarize
• sensu events filter severity critical
• sensu events filter subscription webservers

30. Monitoring your monitoring
• Check RabbitMQ ready queue!

31. Scaling Sensu

32. Scaling a single site
• Sensu Server
• Sensu API
• RabbitMQ
• Redis

33. Multi-DC operation

37. HA
• High availability monitoring with Sensu

38. References

39. Community plugins
https://github.com/sensu/sensu-community-plugins
• Over 600 plugins
• 80 contributors

40. Support
• #sensu on FreeNode IRC
• sensu-users mailing list
• Commercial support from HeavyWater

41. Thank you!
@geewiz
jochen@freistil.it

42. Credits
• Samuel Beckett Bridge by Miguel Mendez https://flic.kr/p/
dyn2FU