0
Monitoring with syslog-ng, Riemann and Kibana
@algernoone @balabit
syslog-ng
syslog-ng
● Open source event processor and swiss army knife
syslog-ng
● Open source event processor and swiss army knife
● Developed since 1998, LGPL + GPL
syslog-ng
● Open source event processor and swiss army knife
● Developed since 1998, LGPL + GPL
– (Commercial offering sin...
syslog-ng
● Open source event processor and swiss army knife
● Developed since 1998, LGPL + GPL
– (Commercial offering sin...
syslog-ng
● Open source event processor and swiss army knife
● Developed since 1998, LGPL + GPL
– (Commercial offering sin...
syslog-ng
● Open source event processor and swiss army knife
● Developed since 1998, LGPL + GPL
– (Commercial offering sin...
Riemann
Riemann
● Riemann monitors distributed systems
Riemann
● Riemann monitors distributed systems
● Event aggregator with a powerful stream processing
language
Riemann
● Riemann monitors distributed systems
● Event aggregator with a powerful stream processing
language
● Provides a ...
Kibana
Kibana
● Visualize logs and time-stamped data
Kibana
● Visualize logs and time-stamped data
● Powerful search syntax
Kibana
● Visualize logs and time-stamped data
● Powerful search syntax
● Flexible, powerful, yet intuitive interface
Kibana
Monitoring
Monitoring
What
Monitoring
● System state
What
Monitoring
● System state
● Application state
What
Monitoring
● System state
● Application state
● Exceptions
What
Monitoring
● System state
● Application state
● Exceptions
● Activity
What
Monitoring
Tools
Monitoring
● Nagios
● Collectd
● Munin
Tools
Monitoring
● Nagios
● Collectd
● Munin
● Riemann
Tools
Logging
Logging
Logging
● Persisting application state
Logging
● Persisting application state
● Format is usually application specific
Logging
● Persisting application state
● Format is usually application specific
● Structured vs unstructured
Logging
● Persisting application state
● Format is usually application specific
● Structured vs unstructured
● Great sourc...
Monitoring + Logging
Monitoring + Logging
● What do we already have?
Monitoring + Logging
● What do we already have?
● What can we add?
Monitoring + Logging
● What do we already have?
● What can we add?
● How will we benefit from all of this?
Q & A
Thanks!
● http://www.syslog-ng.org/
● https://github.com/balabit/syslog-ng-incubator
● https://talien.blogs.balabit.com/
●...
Upcoming SlideShare
Loading in...5
×

OSDC: Gergely Nagy: Monitoring with syslog-ng, Riemann and Kibana

531

Published on

In any data center, one will have a lot of machines, and even more applications, plenty of them legacy applications with little to no built-in monitoring capabilities. But even when monitoring is built in, quite often, it just provides basic building blocks.
In this talk, it will be shown how to tie a syslog-ng based logging solution to the Riemann monitoring system, and use Kibana to make sense of both logging and monitoring data. The presentation will suggest solutions for extracting data from various applications, ways to transform them into useful metrics, and will - of course - also touch the subject of what exactly useful metrics are to begin with. A live demo of all things discussed will be shown at the end.

Published in: Software, Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
531
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "OSDC: Gergely Nagy: Monitoring with syslog-ng, Riemann and Kibana"

  1. 1. Monitoring with syslog-ng, Riemann and Kibana @algernoone @balabit
  2. 2. syslog-ng
  3. 3. syslog-ng ● Open source event processor and swiss army knife
  4. 4. syslog-ng ● Open source event processor and swiss army knife ● Developed since 1998, LGPL + GPL
  5. 5. syslog-ng ● Open source event processor and swiss army knife ● Developed since 1998, LGPL + GPL – (Commercial offering since 2007)
  6. 6. syslog-ng ● Open source event processor and swiss army knife ● Developed since 1998, LGPL + GPL – (Commercial offering since 2007) ● Collects, parses, filters, transforms, transfers events
  7. 7. syslog-ng ● Open source event processor and swiss army knife ● Developed since 1998, LGPL + GPL – (Commercial offering since 2007) ● Collects, parses, filters, transforms, transfers events ● Wide variety of plugins
  8. 8. syslog-ng ● Open source event processor and swiss army knife ● Developed since 1998, LGPL + GPL – (Commercial offering since 2007) ● Collects, parses, filters, transforms, transfers events ● Wide variety of plugins ● A sizable, helpful and very inclusive community
  9. 9. Riemann
  10. 10. Riemann ● Riemann monitors distributed systems
  11. 11. Riemann ● Riemann monitors distributed systems ● Event aggregator with a powerful stream processing language
  12. 12. Riemann ● Riemann monitors distributed systems ● Event aggregator with a powerful stream processing language ● Provides a low-latency, transient shared state
  13. 13. Kibana
  14. 14. Kibana ● Visualize logs and time-stamped data
  15. 15. Kibana ● Visualize logs and time-stamped data ● Powerful search syntax
  16. 16. Kibana ● Visualize logs and time-stamped data ● Powerful search syntax ● Flexible, powerful, yet intuitive interface
  17. 17. Kibana
  18. 18. Monitoring
  19. 19. Monitoring What
  20. 20. Monitoring ● System state What
  21. 21. Monitoring ● System state ● Application state What
  22. 22. Monitoring ● System state ● Application state ● Exceptions What
  23. 23. Monitoring ● System state ● Application state ● Exceptions ● Activity What
  24. 24. Monitoring Tools
  25. 25. Monitoring ● Nagios ● Collectd ● Munin Tools
  26. 26. Monitoring ● Nagios ● Collectd ● Munin ● Riemann Tools
  27. 27. Logging
  28. 28. Logging
  29. 29. Logging ● Persisting application state
  30. 30. Logging ● Persisting application state ● Format is usually application specific
  31. 31. Logging ● Persisting application state ● Format is usually application specific ● Structured vs unstructured
  32. 32. Logging ● Persisting application state ● Format is usually application specific ● Structured vs unstructured ● Great source for monitoring too!
  33. 33. Monitoring + Logging
  34. 34. Monitoring + Logging ● What do we already have?
  35. 35. Monitoring + Logging ● What do we already have? ● What can we add?
  36. 36. Monitoring + Logging ● What do we already have? ● What can we add? ● How will we benefit from all of this?
  37. 37. Q & A
  38. 38. Thanks! ● http://www.syslog-ng.org/ ● https://github.com/balabit/syslog-ng-incubator ● https://talien.blogs.balabit.com/ ● https://algernon.blogs.balabit.com/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×