Collecting metrics with Graphite and StatsD


Published on

Do you gather metrics from your application? Can you combine them and easily generate custom graphs out of them? Can your developers measure whatever they want at any point of your application without breaking it or making it slower?

In our next itnig friday, Víctor Martínez will show us how easy it is to roll on your own Graphite installation and how to use Etsy's statsd collector to flush your metrics. You will learn what Graphite is, how all of its components work, how to get your real time&historic metrics into Carbon, Graphite's database, and how to plot them in different manners. Víctor will show us some Graphite dashboards, alternative statds implementations, detailed common Graphite configuration gotchas, design limitations and how to deal with them.

<a>Visit details</a>

Published in: Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Collecting metrics with Graphite and StatsD

  1. 1. Collecting metrics with Graphite and StatsD Víctor Martínez @knoopx
  2. 2. What is Graphite?
  3. 3. What is Graphite?An all-in-one solution for storing and visualizingrealtime time-series data in an efficient manner. Key features: Efficient storage and ultra-fast retrieval.
  4. 4. Graphite ArchitectureGraphite Web Carbon WhisperThe frontend of The core of Graphite. It The current (stable)Graphite. It provides a consists of set of three data persistencedashboard for retrieval processes that take storage; an efficientand visualization of our care of receiving time-series basedmetrics and a powerful incoming data, database.plotting API. aggregating it and * Theres a new persistence persisting it to disk. layer called Ceres in the works
  5. 5. Feeding in your data Name PORT=2003 echo " 1 `date +%s`" | nc ${SERVER} ${PORT};schema.application_name.namespace.metric_name.metric_type Value PORT=2003 echo " 1 `date +%s`" | nc ${SERVER} ${PORT}; Timestamp PORT=2003 echo " 1 `date +%s`" | nc ${SERVER} ${PORT};
  6. 6. Understanding how Carbon works Key configuration filesstorage-schemas.conf storage-aggregation.confAllows you to define how your data Allows you to define how yourshould be persisted (retention rules) metrics should be aggregated.and in which order (priority).
  7. 7. Understanding how Carbon works storage-schemas.confWhat does retention mean? In a timespan of X time, in how much granularity Iwant to store my data
  8. 8. Understanding how Carbon works storage-schemas.conf● For a timespan of 6 hours I want to store my data in intervals of 10 seconds.● For a timespan of 7 days I want to store my data in intervals of 1 minute.● For a timespan of 1 year I want to store my data in intervals of 1 hour.● For a timespan of 5 year I want to store my data in intervals of 24 hours.
  9. 9. Understanding how Carbon works storage-schemas.conf
  10. 10. Understanding how Carbon works storage-aggregation.conf Some examples:
  11. 11. Understanding how Carbon works storage-aggregation.conf
  12. 12. Understanding how Carbon works Carbon limitationsWhat happens if I send two metrics at the same timespan (1h)? 1 new user – 6 minutes ago 2 new users – 5 minutes ago
  13. 13. Understanding how Carbon works Carbon limitations Carbon discards everything but the last one 2 users
  14. 14. Understanding how Carbon works Carbon limitations Possible Solutions:● Increase schema retention: 1s:12h,[...] and never, ever, push metrics older than 12 hours/push more than once per second● Pre-aggregate your data before pushing● Use StatsD
  15. 15. Understanding how Carbon works Pre-aggregate your data
  16. 16. Understanding how Carbon works Use StatsDWhat is StatdD?A network daemon that runs on the Node.js platformand listens for statistics, like counters and timers, sentover UDP and sends aggregates to one or morepluggable backend services (e.g., Graphite).Key features: Aggregates the data for us, fire andforget communication protocol (UDP)
  17. 17. Understanding how Carbon works StatsD Instrument Gem StatsD.increment(revenue, 23.99) StatsD.measure(worker.time) do Worker.perform(value) endImportant: Carefully read StatsD documentation and ensure that your Carbon configuration explicitly matches StatsD naming conventions
  18. 18. Visualizing your metricsUsing the Graphite dashboard Common functions: ● summarize ● timeShift ● keepLastValue ● integral ● drawAsInfinite ● sumSeries ● alias ● aliasByNodeFull Documentation:
  19. 19. Visualizing your metrics Using the Graphite dashboard Time-shifting and aliasing:alias(timeShift(camaloon.sidekiq.workers.success.count,"24h"),"Yesterday")alias(camaloon.sidekiq.workers.success.count,"Today")
  20. 20. Visualizing your metrics Using the Graphite dashboard Summarizing data:summarize(camaloon.sidekiq.workers.success.count,"1h") 1h
  21. 21. Visualizing your metrics Using the Graphite dashboard Aliasing and combining series:aliasByNode(camaloon.sidekiq.workers.*.count, 3)
  22. 22. Visualizing your metricsTasseo
  23. 23. Visualizing your metricsGraphene
  24. 24. Visualizing your metrics Your own solution/render/?target=camaloon.users.count&format=json We internally use Highchart Highstock
  25. 25. Other resourcesDeployment
  26. 26. Questions?