Your SlideShare is downloading. ×
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
J-Day Kraków: Listen to the sounds of your application
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

J-Day Kraków: Listen to the sounds of your application

1,521

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,521
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Listen to the sounds of your application
  • 2. Krzysztof Ciesielski softwaremill.com @kpciesielski Lublin Software Craftsmen !2
  • 3. Maciej Biłas softwaremill.com @maciejb jbison.com !3
  • 4. Agenda • Monitoring, huh? • Introducing Graphite • Log analysis – the whys • Logstash architecture & use cases • Exploring logs with Kibana !4
  • 5. Monitoring, huh? !5
  • 6. source: codeascraft.com !6
  • 7. Types of measurements !7
  • 8. Types of measurements Network !7
  • 9. Types of measurements Network Machine !7
  • 10. Types of measurements Network Machine Application !7
  • 11. source: codeascraft.com !8
  • 12. measurement > prediction !9
  • 13. !10
  • 14. measurement >> prediction !11
  • 15. Our stack !12
  • 16. Our stack Server App Server … App !12
  • 17. Our stack Server App Graphite Server … App Logstash !12
  • 18. Our stack Server App Graphite Server … App Logstash !12
  • 19. Our stack Server App Graphite Server … App Logstash !12
  • 20. Our stack Server App Graphite Server … App Logstash !12
  • 21. Our stack Server App Graphite Server … App Logstash !12
  • 22. Our stack Server App Server … App Yammer Metrics Graphite Logstash !12
  • 23. Introducing Graphite !13
  • 24. Graphite !14
  • 25. PORT=2003 SERVER=graphite.your.org echo "local.random.diceroll 4 `date +%s`" | nc ${SERVER} ${PORT};  !15
  • 26. successful.login.attempt 1 1384471287 successful.login.attempt 1 1384471297 !16
  • 27. successful.login.attempt 1 1384471287 successful.login.attempt 1 1384471297 => successful.login.attempt = 1 !16
  • 28. Yammer Metrics !17
  • 29. Yammer Metrics private final Meter successfulLogins = metrics.meter(name(LoginHandler.class, "successful")); ! public void login(String user, String password) { if (canLogin(user,password)) { successfulLogins.mark(); // ... } else { // ... } } !17
  • 30. Metrics !18
  • 31. Metrics Types: !18
  • 32. Metrics Types: • Gauges !18
  • 33. Metrics Types: • Gauges • Counters !18
  • 34. Metrics Types: • Gauges • Counters • Meters !18
  • 35. Metrics Types: • Gauges • Counters • Meters • Histograms !18
  • 36. Metrics Types: • Gauges • Counters • Meters • Histograms • Timers !18
  • 37. Metrics Types: • Gauges • Counters • Meters • Histograms • Timers • Health Checks !18
  • 38. Metrics Types: Reporters: • Gauges • STDOUT • Counters • CSV • Meters • SLF4J • Histograms • JMX • Timers • Graphite • Health Checks • Ganglia !18
  • 39. Counters, meters… vs. Gauges !19
  • 40. Aggregation !20
  • 41. source: codeascraft.com !21
  • 42. You can also check out • collectd https://collectd.org • StatsD https://github.com/etsy/statsd/ • Riemman http://riemann.io/ • Twitter’s Ostrich https://github.com/twitter/ostrich • Ganglia http://ganglia.sourceforge.net/ • Dashboards:
 http://shopify.github.io/dashing/
 https://github.com/obfuscurity/descartes
 https://github.com/obfuscurity/dusk !22
  • 43. Event Sourcing Tracing and storing all the events Current app state: replaying the stream Great for data mining and analysis !23
  • 44. Log aggregation Less invasive way to build an “event stream” Logs are data with plenty of value !24
  • 45. Log aggregation Log source Log source Central storage Query engine Web view Log source !25
  • 46. Logstash Open Source (written in JRuby) !26
  • 47. Logstash Open Source (written in JRuby) Gathers logs from various inputs !26
  • 48. Logstash Open Source (written in JRuby) Gathers logs from various inputs Parses and extracts metadata !26
  • 49. Logstash Open Source (written in JRuby) Gathers logs from various inputs Parses and extracts metadata Writes to various outputs !26
  • 50. Logstash Open Source (written in JRuby) Gathers logs from various inputs Parses and extracts metadata Writes to various outputs Similar tools: ! Splunk, Graylog, Fluentd, log.io !26
  • 51. Logstash centralized architecture !27
  • 52. Logstash grok 2013-11-10 19:41:25.321 [main] INFO o.a.camel.impl.DefaultCamelContext ! - Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds !28
  • 53. Logstash grok 2013-11-10 19:41:25.321 [main] INFO o.a.camel.impl.DefaultCamelContext ! - Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds time 2013-11-10 19:41:25.321 thread main loglevel INFO source o.a.camel.impl.DefaultCamelContext message Apache Camel 2.11.1 (CamelContext: camel-1) started in 4.450 seconds !28
  • 54. Logstash simple use cases Grok: Exceptions Security issues Rare and business-meaningful events Output: e-mail alert !29
  • 55. Kibana !30
  • 56. Kibana checking general app activity !31
  • 57. Kibana Spotting anomalies !32
  • 58. Kibana Checking regular events !33
  • 59. Kibana How often is this feature used? !34
  • 60. Kibana Displaying multiple query results Query: exception Type: applog Query: index Type: mongolog !35
  • 61. References • Graphite http://graphite.readthedocs.org/en/latest/ • Yammer Metrics http://metrics.codahale.com/ • Logstash http://logstash.net/ • Surfing the event stream by Sam Newman at Geecon
 http://www.slideshare.net/spnewman/surfing-the-event-stream • Lessons from Building and Scaling LinkedIn by Jay Kreps
 http://www.infoq.com/presentations/linkedin-architecture-stack • Code as Craft http://codeascraft.com/ !36
  • 62. !37
  • 63. Thank You Reach us at: @maciejb and @kpciesielski softwaremill.com !38

×