Measure Everything
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Measure Everything

  • 591 views
Uploaded on

A talk about monitoring using Graphite and StatsD I gave at a local Ruby meetup.

A talk about monitoring using Graphite and StatsD I gave at a local Ruby meetup.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
591
On Slideshare
574
From Embeds
17
Number of Embeds
2

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 17

http://www.linkedin.com 11
https://www.linkedin.com 6

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Measure everything Ruby Underground, August 2012 @arikfrTuesday, February 12, 13
  • 2. Today’s plan • Why measure • How to measure • Graphite / StatsD • Using Graphite/StatsD with Ruby and RailsTuesday, February 12, 13
  • 3. Questions to answer • How fast is my system? • Is it faster than last month? • Did our last deploy affect database performance? • How much time do we spend calling external web services?Tuesday, February 12, 13
  • 4. More questions • How many errors do we have a day? • How many failed logins? • How many successful logins? • How many orders without house number?Tuesday, February 12, 13
  • 5. And more questions! • How many orders did we have today? • How many orders did we have today from Android version 2.056? • How many rejected orders did we have? • How many rejected orders due to lack of coverage vs. lack of taxis?Tuesday, February 12, 13
  • 6. To answer all of this, you need a way to track different numbers.Tuesday, February 12, 13
  • 7. So now we know the why. But how...?Tuesday, February 12, 13
  • 8. Tuesday, February 12, 13
  • 9. But that’s not enough • Not real time enough • Hard to control what’s being collected • Pricey for big deploymentsTuesday, February 12, 13
  • 10. The Alternative • Graphite (Whisper, Carbon, Graphite Web) • StatsD • CollectD • (there are other options -- OpenTSDB, Liberato, home grown)Tuesday, February 12, 13
  • 11. Benefits • Easy to install • Highly scalable • Practically zero cost to measure anything: • efficient storage • UDP packets to send data in • EcosystemTuesday, February 12, 13
  • 12. Whisper • Default settings: • 6 hours of 10 second data • 1 week of 1 minute data • 5 years of 10 minute data • That’s amounts to ~3.2MB per metric. • Configurable.Tuesday, February 12, 13
  • 13. Types of data • Counters - number of orders per sec • Gauges - total orders today • Timers - time to make an order • (with additional values, such as: count, mean, 90th percentile, max, min, etc)Tuesday, February 12, 13
  • 14. Sending a number to Graphite • metric number timestamp • example.ruby.under_ground 20 1346075634 • echo "example.ruby_underground 20 `date +%s`" | nc graphite.yourcorp.com 2003Tuesday, February 12, 13
  • 15. Sending a number to StatsD ./statsd-client.sh my_metric:100|gTuesday, February 12, 13
  • 16. Naming Convention • Whatever makes sense to you, just remember that it’s a tree. • We use: • {env}.{metric}.{region}.{hostname} • You can use globs, when querying: • app.orders.daily.completed.israel.* • app.orders.daily.completed.*.*Tuesday, February 12, 13
  • 17. Ruby ExamplesTuesday, February 12, 13
  • 18. $ gem install statsd-ruby (https://github.com/reinh/statsd)Tuesday, February 12, 13
  • 19. Track DeploymentsTuesday, February 12, 13
  • 20. Measure controllersTuesday, February 12, 13
  • 21. Measure Specific stuffTuesday, February 12, 13
  • 22. Getting Data outTuesday, February 12, 13
  • 23. Graphite webTuesday, February 12, 13
  • 24. TasseoTuesday, February 12, 13
  • 25. GdashTuesday, February 12, 13
  • 26. GrapheneTuesday, February 12, 13
  • 27. Down sides • Hard to track user analytics • You can tell how many orders were done today • You can’t tell (easily) how many unique users did those orders • The tree of metrics is sometimes annoyingTuesday, February 12, 13
  • 28. Questions?Tuesday, February 12, 13