1. Rails exception tracking with Graylog2 Manage your logs in the dark and have lasers going and make it look like you're from space Lennart Koopmann RubyAndRails EU Amsterdam, 2010

2. About me Lennart Koopmann 22 years old Living in Hamburg, working at Jimdo lennartkoopmann.net / @_lennart

3. Log server with web interface.

4. phpLogCon … let's not talk about that Graylog1: August 2009 Graylog2: August 2010

6. Syslog daemon (TCP/UDP)

7. GELF (UDP)

8. Graylog extended log format UDP - Max (GZIP) 8192 byte per message (Chunking is supported) Libraries for Ruby and PHP available

9. { }

10. { ' message ':'Exception: Something went wrong.' }

11. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars' }

12. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19' }

13. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19', ' file ':'/var/www/index.php' }

14. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19', ' file ':'/var/www/index.php', ' line ':2638 }

15. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19', ' file ':'/var/www/index.php', ' line ':2638, ' level ':1 }

16. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19', ' file ':'/var/www/index.php', ' line ':2638, ' level ':1, ' something ':'foo' }

17. { ' message ':'Exception: Something went wrong.', ' full_message ':'Stacktrace.Some env vars', ' host ':'www19', ' file ':'/var/www/index.php', ' line ':2638, ' level ':1, ' something ':'foo', ' something_else ':'bar' }

18. Chunking

19. Use Cases

20. Plain syslog Collect everything of your /var/log from all servers, aggregate, analyze and get a warning if something goes wrong

21. GELF Embed it into the logging class of your application and trigger with every error. Include stacktrace and env vars in full message. Later filter by file:line to get statistics.

22. Aggregate, analyze and get a warning if something goes wrong

23. Notifications Currently only Nagios hook (rake task) – Triggered at too many messages in last X minutes

24. Average page generation time Graylog1 / MySQL: ~12000ms

25. Average page generation time Graylog1 / MySQL: ~12000ms Graylog2 / MongoDB: ~250ms

26. Exception tracking with Rails Rack middleware available as gem

27. lennart ~$ sudo gem install graylog2_exceptions Successfully installed gelf-0.9.1 Successfully installed graylog2_exceptions-0.5.3

28. # config/environment.rb config.middleware.use " Graylog2Exceptions ", { :host => ' 127.0.0.1 ', :port => ' 12201 ', :local_app_name => ' mama ' }

29. Roadmap

30. Incident management Define chained rules that explain an incident. Trigger several actions when incident occurs.

31. Fine grained notifications Be informed via Email, XMPP, SMS by Graylog2 directly without Nagios hook

32. More analyzing

33. More search features

34. Live demo. www.graylog2.org / @Graylog2 www.lennartkoopmann.net / @_lennart (Slides are on Slideshare)