Measure everything                             Ruby Underground, August 2012                                       @arikfr...
Today’s plan                     • Why measure                     • How to measure                     • Graphite / Stats...
Questions to answer                     • How fast is my system?                     • Is it faster than last month?      ...
More questions                     • How many errors do we have a day?                     • How many failed logins?      ...
And more questions!                     • How many orders did we have today?                     • How many orders did we ...
To answer all of this,                            you need a way to                             track different           ...
So now we know the                                   why.                                But how...?Tuesday, February 12, 13
Tuesday, February 12, 13
But that’s not enough                     • Not real time enough                     • Hard to control what’s being collec...
The Alternative                     • Graphite (Whisper, Carbon, Graphite Web)                     • StatsD               ...
Benefits                     • Easy to install                     • Highly scalable                     • Practically zero...
Whisper                     • Default settings:                      • 6 hours of 10 second data                      • 1 ...
Types of data                     • Counters - number of orders per sec                     • Gauges - total orders today ...
Sending a number to                                Graphite                     • metric number timestamp                 ...
Sending a number to                                 StatsD                               ./statsd-client.sh my_metric:100|...
Naming Convention                     • Whatever makes sense to you, just                           remember that it’s a t...
Ruby ExamplesTuesday, February 12, 13
$ gem install statsd-ruby                           (https://github.com/reinh/statsd)Tuesday, February 12, 13
Track DeploymentsTuesday, February 12, 13
Measure controllersTuesday, February 12, 13
Measure Specific stuffTuesday, February 12, 13
Getting Data outTuesday, February 12, 13
Graphite webTuesday, February 12, 13
TasseoTuesday, February 12, 13
GdashTuesday, February 12, 13
GrapheneTuesday, February 12, 13
Down sides                     • Hard to track user analytics                      • You can tell how many orders were don...
Questions?Tuesday, February 12, 13
Upcoming SlideShare
Loading in...5
×

Measure Everything

353

Published on

A talk about monitoring using Graphite and StatsD I gave at a local Ruby meetup.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
353
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Measure Everything"

  1. 1. Measure everything Ruby Underground, August 2012 @arikfrTuesday, February 12, 13
  2. 2. Today’s plan • Why measure • How to measure • Graphite / StatsD • Using Graphite/StatsD with Ruby and RailsTuesday, February 12, 13
  3. 3. Questions to answer • How fast is my system? • Is it faster than last month? • Did our last deploy affect database performance? • How much time do we spend calling external web services?Tuesday, February 12, 13
  4. 4. More questions • How many errors do we have a day? • How many failed logins? • How many successful logins? • How many orders without house number?Tuesday, February 12, 13
  5. 5. And more questions! • How many orders did we have today? • How many orders did we have today from Android version 2.056? • How many rejected orders did we have? • How many rejected orders due to lack of coverage vs. lack of taxis?Tuesday, February 12, 13
  6. 6. To answer all of this, you need a way to track different numbers.Tuesday, February 12, 13
  7. 7. So now we know the why. But how...?Tuesday, February 12, 13
  8. 8. Tuesday, February 12, 13
  9. 9. But that’s not enough • Not real time enough • Hard to control what’s being collected • Pricey for big deploymentsTuesday, February 12, 13
  10. 10. The Alternative • Graphite (Whisper, Carbon, Graphite Web) • StatsD • CollectD • (there are other options -- OpenTSDB, Liberato, home grown)Tuesday, February 12, 13
  11. 11. Benefits • Easy to install • Highly scalable • Practically zero cost to measure anything: • efficient storage • UDP packets to send data in • EcosystemTuesday, February 12, 13
  12. 12. Whisper • Default settings: • 6 hours of 10 second data • 1 week of 1 minute data • 5 years of 10 minute data • That’s amounts to ~3.2MB per metric. • Configurable.Tuesday, February 12, 13
  13. 13. Types of data • Counters - number of orders per sec • Gauges - total orders today • Timers - time to make an order • (with additional values, such as: count, mean, 90th percentile, max, min, etc)Tuesday, February 12, 13
  14. 14. Sending a number to Graphite • metric number timestamp • example.ruby.under_ground 20 1346075634 • echo "example.ruby_underground 20 `date +%s`" | nc graphite.yourcorp.com 2003Tuesday, February 12, 13
  15. 15. Sending a number to StatsD ./statsd-client.sh my_metric:100|gTuesday, February 12, 13
  16. 16. Naming Convention • Whatever makes sense to you, just remember that it’s a tree. • We use: • {env}.{metric}.{region}.{hostname} • You can use globs, when querying: • app.orders.daily.completed.israel.* • app.orders.daily.completed.*.*Tuesday, February 12, 13
  17. 17. Ruby ExamplesTuesday, February 12, 13
  18. 18. $ gem install statsd-ruby (https://github.com/reinh/statsd)Tuesday, February 12, 13
  19. 19. Track DeploymentsTuesday, February 12, 13
  20. 20. Measure controllersTuesday, February 12, 13
  21. 21. Measure Specific stuffTuesday, February 12, 13
  22. 22. Getting Data outTuesday, February 12, 13
  23. 23. Graphite webTuesday, February 12, 13
  24. 24. TasseoTuesday, February 12, 13
  25. 25. GdashTuesday, February 12, 13
  26. 26. GrapheneTuesday, February 12, 13
  27. 27. Down sides • Hard to track user analytics • You can tell how many orders were done today • You can’t tell (easily) how many unique users did those orders • The tree of metrics is sometimes annoyingTuesday, February 12, 13
  28. 28. Questions?Tuesday, February 12, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×