Keep an eye on your app with logging and graphs - PHPBenelux 2014

3,657 views

Published on

Logging, metrics and graphs are often an afterthought. This usually becomes painfully clear when something goes wrong and you feel like you're running in the dark with a blindfold on. You need to know what's going on in your application! We're going to have a deeper look at tools like logstash, statsd and friends to see how they can help developers and the business as a whole. You'll fix problems quicker and will gain more insights in how your application is doing.

Published in: Technology, Business

Keep an eye on your app with logging and graphs - PHPBenelux 2014

  1. 1. So, logging! !1
  2. 2. http://www.egeniq.com info@egeniq.com @egeniq Keep an eye on your app with logging and graphs Felix De Vliegher PHPBenelux 2014
  3. 3. About me • Back-end developer • Sysadmin • <3 APIs • Travelling • @felixdv !3
  4. 4. About me !4
  5. 5. About me !5
  6. 6. About me !6
  7. 7. Terminology Logging shows what exactly happens in your app. ! Metrics show the impact !7
  8. 8. Terminology Logging shows what exactly happens in your app. ! Metrics show the impact !8
  9. 9. Both are important (but for different reasons) !9
  10. 10. What is this talk about? Knowing what is going on in your application! !10
  11. 11. Questionnaire • Who uses logging in their app? • Aggregation, filtering, manipulation? • Who collects metrics? • Who visualises this data in a dashboard? • Metric correlations? !11
  12. 12. Logging !12
  13. 13. Ye olden days • Logs per server • Requires SSH for inspection • Decentralised (for once not an advantage) • Hard to search !13
  14. 14. Goals Make logs searchable !14
  15. 15. Goals !15
  16. 16. Goals Collect and manipulate logs !16
  17. 17. Goals !17
  18. 18. Goals Extract metrics !18
  19. 19. Enter Mr. logstash (hipster mo’ included) !19
  20. 20. Logstash • Inputs • Filters • Outputs !20
  21. 21. Inputs • File • Redis • syslog • varnishlog • rabbitmq • … !21
  22. 22. Filters Modify the input data ! • • • • • • grok mutate dns json geoip … !22
  23. 23. Grok !23
  24. 24. Grok !24
  25. 25. Mutate !25
  26. 26. Json !26
  27. 27. Json !27
  28. 28. Outputs • • • • • • • • email graphite irc mongodb pagerduty sns elasticsearch … !28
  29. 29. Setup !29
  30. 30. Example shipper config !30
  31. 31. Example indexer config !31
  32. 32. Example indexer config (contd.) !32
  33. 33. Logstash !33
  34. 34. Elasticsearch !34
  35. 35. Elasticsearch • Distributed search and analytics • Based on Apache lucene (but better) • Document-oriented DB • Easy scaling • Restful API • Great for storing logs! !35
  36. 36. Elasticsearch ! Getting data in and out: !36
  37. 37. Elasticsearch ! • Logstash defaults to using daily indexes • Easy retention = delete indexes after x days • Can be configured in other ways !37
  38. 38. Kibana !38
  39. 39. Kibana • Search your logs • The v3 rewrite is pretty awesome :-) • HTML + JS only • Elasticsearch project • Almost no-config setup !39
  40. 40. Kibana !40
  41. 41. Kibana !41
  42. 42. Kibana !42
  43. 43. Kibana !43
  44. 44. Kibana !44
  45. 45. StatsD • Metrics aggregation framework • Made by Etsy • Node.js • UDP • Dead easy to use !46
  46. 46. Metrics “Track everything that moves” !47
  47. 47. Metrics !48
  48. 48. Running StatsD !49
  49. 49. StatsD config !50
  50. 50. StatsD config !51
  51. 51. Console backend !52
  52. 52. StatsD Metric types • Counters • Timers • Gauges !53
  53. 53. Counters !54
  54. 54. Timers !55
  55. 55. Gauges Arbitrary values: !56
  56. 56. Sampling data !57
  57. 57. Example: ZF Controller plugin !58
  58. 58. StatsD naming !59
  59. 59. But… StatsD is for collection and aggregation only !60
  60. 60. We want pretty graphs !61
  61. 61. Coffee is important too !62
  62. 62. StatsD config !63
  63. 63. Graphite • Store and visualise time-series data • Carbon, whisper and graphite UI • Rendering graphs • Powerful function library !64
  64. 64. Graphite UI !65
  65. 65. StatsD => Graphite production.api.httpstatus.200 !66
  66. 66. StatsD => Graphite !67
  67. 67. The render API !68
  68. 68. Json output Add &format=json to the render API !69
  69. 69. Functions • • • • • • cumulative(‘server*.api.logins’) asPercent(‘connections.failed’, ‘connections.attempted’) drawAsInfinite(‘app.deploys’) summarize(‘app.errors’, ‘1hour’) stacked() movingAverage() ! • http://graphite.readthedocs.org/en/latest/functions.html !70
  70. 70. StatsD !71
  71. 71. Functions • • • • • • cumulative(‘server*.api.logins’) asPercent(‘connections.failed’, ‘connections.attempted’) drawAsInfinite(‘app.deploys’) summarize(‘app.errors’, ‘1hour’) stacked() movingAverage() ! • http://graphite.readthedocs.org/en/latest/functions.html !72
  72. 72. Alternative dashboards !73
  73. 73. Dashing !74
  74. 74. Tasseo !75
  75. 75. Descartes !76
  76. 76. Conclusion • No excuse not to log • Manage your logs with logstash and kibana • Track everything using statsd • Visualise with graphite dashboards • Get insight in your application! !77
  77. 77. http://www.egeniq.com info@egeniq.com @egeniq Thank you! Questions? Feedback: https://joind.in/10276 Contact: @felixdv

×