Collecting metrics with Prometheus and VictoriaMetrics - Aliaksandr Valialkin (RUS) | Ruby Meditation 26

Ruby Meditation
Ruby MeditationRuby Meditation
Collecting metrics with
Prometheus and
VictoriaMetric
Aliaksandr Valialkin, VictoriaMetrics
Let me introduce myself
● VictoriaMetrics founder and core developer
● Fasthttp creator
● Fond of performance optimizations
What is Prometheus?
● The most popular monitoring system
● Metrics’ collector
● Time Series DB
● Rules evaluator and alerter
What Prometheus can do?
● Collecting metrics from pre-configured targets
● Storing metrics in the local and remote DBs
● Executing queries against metrics stored in the DB
● Sending alerts on pre-configured rules
● Helping to build nice dashboards in Grafana
Collecting metrics with Prometheus and VictoriaMetrics - Aliaksandr Valialkin  (RUS) | Ruby Meditation 26
Ruby + Prometheus
● https://github.com/reinh/statsd +
https://github.com/prometheus/statsd_exporter
● https://github.com/prometheus/client_ruby - hard to use properly due
to support pre-fork (unicorn) servers issue
https://github.com/reinh/statsd usage
# Set up a global Statsd client for a server on localhost:9125
$statsd = Statsd.new 'localhost', 9125
# Send some stats
$statsd.increment 'garets'
$statsd.timing 'glork', 320
$statsd.gauge 'bork', 100
# Use {#time} to time the execution of a block
$statsd.time('account.activate') { @account.activate! }
# Create a namespaced statsd client and increment 'account.activate'
statsd = Statsd.new('localhost').tap{|sd| sd.namespace = 'account'}
statsd.increment 'activate'
Local DB issues in Prometheus
● Doesn’t scale outside the node
● Has limited size and retention (15 days)
Remote DB to rescue
● Remote DB may transparently scale to many nodes
● Remote DB may deal better with big retentions
● Remote DB may collect and query data from multiple Prometheus
instances
● Remote storages for Prometheus -
https://prometheus.io/docs/operating/integrations/#remote-endpoints-
and-storage
Collecting metrics with Prometheus and VictoriaMetrics - Aliaksandr Valialkin  (RUS) | Ruby Meditation 26
What is VictoriaMetrics?
● Long-term remote storage for Prometheus
● Single-node version is available at
https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Single-server-VictoriaMetrics
● SaaS is in progress at https://victoriametrics.com
● There is SaaS playground at https://play.victoriametrics.com/signIn
VictoriaMetrics features
● Fast and cost-effective - see articles at https://medium.com/@valyala
● Easy to operate - just a single executable without dependencies
● Supports PromQL - native query language for Prometheus
● Accepts metrics in InfluxDB and Graphite formats
Questions?
1 of 13

Recommended

Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30 by
Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30
Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30Ruby Meditation
207 views22 slides
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky... by
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...Ruby Meditation
462 views141 slides
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29 by
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29Ruby Meditation
210 views49 slides
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ... by
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...Ruby Meditation
1.6K views59 slides
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 by
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 Ruby Meditation
366 views23 slides
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28 by
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28Ruby Meditation
459 views20 slides

More Related Content

More from Ruby Meditation

Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio... by
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Ruby Meditation
320 views80 slides
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or... by
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...Ruby Meditation
285 views49 slides
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27 by
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27Ruby Meditation
1.1K views96 slides
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26 by
New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26Ruby Meditation
577 views56 slides
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26 by
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Ruby Meditation
299 views51 slides
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (... by
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Ruby Meditation
455 views78 slides

More from Ruby Meditation(20)

Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio... by Ruby Meditation
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Ruby Meditation320 views
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or... by Ruby Meditation
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
Ruby Meditation285 views
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27 by Ruby Meditation
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
Ruby Meditation1.1K views
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26 by Ruby Meditation
New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26
Ruby Meditation577 views
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26 by Ruby Meditation
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Ruby Meditation299 views
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (... by Ruby Meditation
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Ruby Meditation455 views
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26 by Ruby Meditation
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Ruby Meditation204 views
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25 by Ruby Meditation
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Ruby Meditation577 views
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita... by Ruby Meditation
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Ruby Meditation511 views
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me... by Ruby Meditation
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Ruby Meditation299 views
Rails App performance at the limit - Bogdan Gusiev by Ruby Meditation
Rails App performance at the limit - Bogdan GusievRails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan Gusiev
Ruby Meditation418 views
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23 by Ruby Meditation
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
Ruby Meditation179 views
Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton... by Ruby Meditation
Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton...Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton...
Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton...
Ruby Meditation2.7K views
Database Sharding in Rails Applications – Vitalik Danchenko | Ruby Meditatio... by Ruby Meditation
 Database Sharding in Rails Applications – Vitalik Danchenko | Ruby Meditatio... Database Sharding in Rails Applications – Vitalik Danchenko | Ruby Meditatio...
Database Sharding in Rails Applications – Vitalik Danchenko | Ruby Meditatio...
Ruby Meditation823 views
Building MVP from business owner’s perspective – Piotr Latoszek | Ruby Medita... by Ruby Meditation
Building MVP from business owner’s perspective – Piotr Latoszek | Ruby Medita...Building MVP from business owner’s perspective – Piotr Latoszek | Ruby Medita...
Building MVP from business owner’s perspective – Piotr Latoszek | Ruby Medita...
Ruby Meditation340 views
Growing Rails Apps - Dmitry Zhlobo | Ruby Meditation #23 by Ruby Meditation
Growing Rails Apps - Dmitry Zhlobo | Ruby Meditation #23Growing Rails Apps - Dmitry Zhlobo | Ruby Meditation #23
Growing Rails Apps - Dmitry Zhlobo | Ruby Meditation #23
Ruby Meditation202 views
Residence permit EU - Gennadii Miroshnychenko by Ruby Meditation
Residence permit EU - Gennadii MiroshnychenkoResidence permit EU - Gennadii Miroshnychenko
Residence permit EU - Gennadii Miroshnychenko
Ruby Meditation157 views
Practical SOLID with Rails - Andrii Savchenko by Ruby Meditation
Practical SOLID with Rails - Andrii SavchenkoPractical SOLID with Rails - Andrii Savchenko
Practical SOLID with Rails - Andrii Savchenko
Ruby Meditation139 views
Inside Out Ruby: Using MRI as a C library - Artur Pyrogovskyi by Ruby Meditation
Inside Out Ruby: Using MRI as a C library - Artur PyrogovskyiInside Out Ruby: Using MRI as a C library - Artur Pyrogovskyi
Inside Out Ruby: Using MRI as a C library - Artur Pyrogovskyi
Ruby Meditation151 views

Recently uploaded

PRODUCT LISTING.pptx by
PRODUCT LISTING.pptxPRODUCT LISTING.pptx
PRODUCT LISTING.pptxangelicacueva6
18 views1 slide
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院IttrainingIttraining
69 views8 slides
Data Integrity for Banking and Financial Services by
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial ServicesPrecisely
29 views26 slides
Zero to Automated in Under a Year by
Zero to Automated in Under a YearZero to Automated in Under a Year
Zero to Automated in Under a YearNetwork Automation Forum
22 views23 slides
Melek BEN MAHMOUD.pdf by
Melek BEN MAHMOUD.pdfMelek BEN MAHMOUD.pdf
Melek BEN MAHMOUD.pdfMelekBenMahmoud
17 views1 slide
Special_edition_innovator_2023.pdf by
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdfWillDavies22
18 views6 slides

Recently uploaded(20)

【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
Data Integrity for Banking and Financial Services by Precisely
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial Services
Precisely29 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2218 views
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software317 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn26 views
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by Jasper Oosterveld
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors by sugiuralab
TouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective SensorsTouchLog: Finger Micro Gesture Recognition  Using Photo-Reflective Sensors
TouchLog: Finger Micro Gesture Recognition Using Photo-Reflective Sensors
sugiuralab23 views
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec15 views
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays24 views

Collecting metrics with Prometheus and VictoriaMetrics - Aliaksandr Valialkin (RUS) | Ruby Meditation 26

  • 1. Collecting metrics with Prometheus and VictoriaMetric Aliaksandr Valialkin, VictoriaMetrics
  • 2. Let me introduce myself ● VictoriaMetrics founder and core developer ● Fasthttp creator ● Fond of performance optimizations
  • 3. What is Prometheus? ● The most popular monitoring system ● Metrics’ collector ● Time Series DB ● Rules evaluator and alerter
  • 4. What Prometheus can do? ● Collecting metrics from pre-configured targets ● Storing metrics in the local and remote DBs ● Executing queries against metrics stored in the DB ● Sending alerts on pre-configured rules ● Helping to build nice dashboards in Grafana
  • 6. Ruby + Prometheus ● https://github.com/reinh/statsd + https://github.com/prometheus/statsd_exporter ● https://github.com/prometheus/client_ruby - hard to use properly due to support pre-fork (unicorn) servers issue
  • 7. https://github.com/reinh/statsd usage # Set up a global Statsd client for a server on localhost:9125 $statsd = Statsd.new 'localhost', 9125 # Send some stats $statsd.increment 'garets' $statsd.timing 'glork', 320 $statsd.gauge 'bork', 100 # Use {#time} to time the execution of a block $statsd.time('account.activate') { @account.activate! } # Create a namespaced statsd client and increment 'account.activate' statsd = Statsd.new('localhost').tap{|sd| sd.namespace = 'account'} statsd.increment 'activate'
  • 8. Local DB issues in Prometheus ● Doesn’t scale outside the node ● Has limited size and retention (15 days)
  • 9. Remote DB to rescue ● Remote DB may transparently scale to many nodes ● Remote DB may deal better with big retentions ● Remote DB may collect and query data from multiple Prometheus instances ● Remote storages for Prometheus - https://prometheus.io/docs/operating/integrations/#remote-endpoints- and-storage
  • 11. What is VictoriaMetrics? ● Long-term remote storage for Prometheus ● Single-node version is available at https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Single-server-VictoriaMetrics ● SaaS is in progress at https://victoriametrics.com ● There is SaaS playground at https://play.victoriametrics.com/signIn
  • 12. VictoriaMetrics features ● Fast and cost-effective - see articles at https://medium.com/@valyala ● Easy to operate - just a single executable without dependencies ● Supports PromQL - native query language for Prometheus ● Accepts metrics in InfluxDB and Graphite formats