Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Measure Anything,
Measure Everything
StatsD & Graphite
By Avi Revivo / 2015
New development checklist
 Code standards
 Comments
 Logic
 Error handling
 Testable code
 Logging
 What is missing?
What about?
 Feature usage
 Performance
 Business KPIs
So why is it not there?
2 main reasons
#1 Complexity
It’s just not simple enough
#2 Understanding the Developer’s Role
“Dear Engineers,
Your job is not to write code.
I know. You think you were hired to ...
Let’s focus on complexity
How simple should it be? Dead simple!
As simple as writing a log line…
Meet StatsD
A network service that runs on the Node.js that listens
for statistics, like counters and timers, sent over UD...
Example: Sending a Metric
echo "foo:1|c" | nc -u -w0 127.0.0.1 8125
<metricname>:<value>|<type>
Key Concepts
 Buckets - Each stat is in its own "bucket". They are not
predefined anywhere.
 Values - Each stat will hav...
Available Metric Types
Counting
gorets:1|c
Timing
glork:320|ms|@0.1
Gauges
gaugor:333|g
Sets
uniques:765|s
Sampling suppor...
Meet Graphite
An all-in-one solution for storing and visualizing realtime time-
series data in an efficient manner.
Graphite Architecture
Graphite Web
A powerful
dashboard for our
metrics and a
powerful plotting API.
Carbon
The core of Gr...
Understanding Carbon
Carbon's job is to make that data available for real-time graphing
immediately and try to get it stor...
Carbon Retention Rules
 For a timespan of 6 hours I want to store my data in intervals of 10 seconds.
 For a timespan of...
Carbon Aggregation Rules
[sum]
Pattern= .count$
aggregationMethod = sum
stats.app.users.logins.count
stats.app.emails.sent...
DEMO
StasD & Graphite - Measure anything, Measure Everything
StasD & Graphite - Measure anything, Measure Everything
Upcoming SlideShare
Loading in …5
×

StasD & Graphite - Measure anything, Measure Everything

280 views

Published on

An overview on StatsD and Graphite

Published in: Software
  • Be the first to comment

StasD & Graphite - Measure anything, Measure Everything

  1. 1. Measure Anything, Measure Everything StatsD & Graphite By Avi Revivo / 2015
  2. 2. New development checklist  Code standards  Comments  Logic  Error handling  Testable code  Logging  What is missing?
  3. 3. What about?  Feature usage  Performance  Business KPIs
  4. 4. So why is it not there? 2 main reasons
  5. 5. #1 Complexity It’s just not simple enough
  6. 6. #2 Understanding the Developer’s Role “Dear Engineers, Your job is not to write code. I know. You think you were hired to write code. In fact, your entire interview process centered around how well you could write code. And I’m sure you do it really well. But it’s not your job. Your job is to improve our product for our users.” Blog post Link
  7. 7. Let’s focus on complexity How simple should it be? Dead simple! As simple as writing a log line…
  8. 8. Meet StatsD A network service that runs on the Node.js that listens for statistics, like counters and timers, sent over UDP or TCP and sends aggregates to one or more pluggable backend services.
  9. 9. Example: Sending a Metric echo "foo:1|c" | nc -u -w0 127.0.0.1 8125 <metricname>:<value>|<type>
  10. 10. Key Concepts  Buckets - Each stat is in its own "bucket". They are not predefined anywhere.  Values - Each stat will have a value.  Flush - After the flush interval timeout, stats are aggregated and sent to an upstream backend service.
  11. 11. Available Metric Types Counting gorets:1|c Timing glork:320|ms|@0.1 Gauges gaugor:333|g Sets uniques:765|s Sampling supported
  12. 12. Meet Graphite An all-in-one solution for storing and visualizing realtime time- series data in an efficient manner.
  13. 13. Graphite Architecture Graphite Web A powerful dashboard for our metrics and a powerful plotting API. Carbon The core of Graphite. It consists of set of three processes that take care of receiving incoming data, aggregating it and persisting it to disk. Whisper An efficient time- series based database.
  14. 14. Understanding Carbon Carbon's job is to make that data available for real-time graphing immediately and try to get it stored on disk as fast as possible. It support configurable data persistency rules and full control over the way data is aggregated.
  15. 15. Carbon Retention Rules  For a timespan of 6 hours I want to store my data in intervals of 10 seconds.  For a timespan of 7 days I want to store my data in intervals of 1 minute.  For a timespan of 1 year I want to store my data in intervals of 1 hour.  For a timespan of 5 year I want to store my data in intervals of 24 hours. [counts] Pattern = .count$ retentions = 10s:6h;1m:7d;1h:1y,1d:5y
  16. 16. Carbon Aggregation Rules [sum] Pattern= .count$ aggregationMethod = sum stats.app.users.logins.count stats.app.emails.sent.count Avergage, Min, Max etc.
  17. 17. DEMO

×