You can't optimize what you cannot measure - PHP UK

3,406 views

Published on

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,406
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
18
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

You can't optimize what you cannot measure - PHP UK

  1. 1. You Cant Optimise What You Cant MeasureJuozas Kaziukėnas // juokaz.com // @juokaz
  2. 2. Juozas Kaziukėnas, LithuanianYou can call me JoeMore info http://juokaz.com
  3. 3. Why?
  4. 4. Data
  5. 5. Looking for lies
  6. 6. Looking for lies
  7. 7. Looking for lies
  8. 8. Data• Metrics• Removes subjective decisions• Can be aggregated and related• If it’s 0, it is 0• Tesla was probably right
  9. 9. Debugging production
  10. 10. Debugging production• Behavioral patterns• When something changes - something is not right• You better notice it• Facebook deployment process• What caused it?
  11. 11. What happened?
  12. 12. What happened?• Previous and current state• Events• Correlated events• State information
  13. 13. What happened here?
  14. 14. What happened here?
  15. 15. Nothing happened here
  16. 16. Something happened here
  17. 17. Logs suck
  18. 18. Logs suck• Someone needs to be checking them• Need to be aggregated• Need to be vizualized• High I/O to write• Distributed logs?
  19. 19. I want to sleep
  20. 20. I want to sleep• Call me when things go wrong• Otherwise everything is working• Things don’t break silently anymore
  21. 21. Business problems
  22. 22. Business problems• Detecting when business tools stop working• No PHP errors, no database errors• Failures of APIs, empty responses, invalid data• Things stop working silently
  23. 23. Counting and timing
  24. 24. Counting and timing• Record when something happens• Record how long it takes for something to happen• Use this to know how many things are happening
  25. 25. The solution
  26. 26. Statsd
  27. 27. Statsd• Counters and timing• No need to initialize or set up counters• Non-blocking writes• Originally written by Etsy.com• Just works• https://github.com/etsy/statsd/
  28. 28. StatsD::increment("phpuk.visitors");
  29. 29. $start = microtime(true);attend_conference();$spent = (microtime(true) - $start) * 1000;StatsD::timing("phpuk.timespent", $spent);
  30. 30. How it works
  31. 31. Graphite
  32. 32. Graphite• Real-time charts• Data collection• Data aggregation• Specialized database• http://graphite.wikidot.com/
  33. 33. How it works
  34. 34. Lobster
  35. 35. Logster
  36. 36. Logster• Parse log files• Send data to graphite• Integration with existing applications easier• Also from Etsy.com• https://github.com/etsy/logster
  37. 37. DataDogHQ.com
  38. 38. DataDogHQ.com
  39. 39. DataDogHQ.com• Hosted solution• Collect data from statsd• Store and aggregate from multiple servers• Chart combining any data• Real time charts• Alerts
  40. 40. Amazon outage
  41. 41. Amazon outage
  42. 42. Amazon outage
  43. 43. Amazon outage
  44. 44. How I use this
  45. 45. Web spiders
  46. 46. Web spiders• ~250 nodes• Couple thousand requests per second• Increasing throughput - main goal• Increasing reliability - secondary goal• Metrics for: request time, error rate, error types, proxy failures, unknown responses, etc.
  47. 47. Web spiders• Performance increased 1000% in 3 months• Reliability increased to being 24/7 stable• I can sleep
  48. 48. Wrapping up
  49. 49. Wrapping up• Measure things• Use statsd to collect data• Graph it• Sleep
  50. 50. THANKS!Juozas Kaziukėnas @juokaz

×