Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Monitoring MySQL with
Prometheus & Grafana
Julien Pivotto (@roidelapluie)
Percona University Belgium
June 22nd, 2017
SELECT USER();
Julien "roidelapluie" Pivotto
@roidelapluie
Sysadmin at inuits
Automation, monitoring, HA
MySQL/MariaDB use...
inuits
Monitoring
Creative Commons Attribution 2.0 https://www.flickr.com/photos/kevandotorg/13829348104/
DevOps
4 keys principles
Culture
Automation
Metrics
Sharing
Damon Edwards and John Willis, 2010
Metrics matters
Find when a service is unavailable (before your
users)
Understand Failures (post-mortem)
Learn from your i...
Collecting Metrics
In general, you should collect as many metrics
as you can
Frequently (5 min collection are not enough)
...
How to collect metrics?
Many many solutions, including:
Graphite
Elasticsearch beats
Zabbix
and .. Prometheus
Out of scope today
Alerting
Prometheus and Grafana installation
Prometheus Fine Tuning
But I'm happy to discuss that after...
Prometheus
https://prometheus.io/
Prometheus
Prometheus is a Cloud-Native Data-Centric Open-
Source Performant Simple metrics collection,
analysis and alert...
Cloud Native
Easy to configure, deploy, maintain
Designed in multiple services
Container ready
Orchestration ready (dynami...
Data Centric
A Metric in Prometheus has metadata:
myql_global_status_handlers_total{handler="tmp_write"} 1122
And lots of ...
Open Source
Apache 2.0
Go
Support for multiple OS
Many "exporters":
https://github.com/prometheus/prometheus/wiki/Defau
lt...
Performance
Prometheus is designed to fetch data in an
interval measured in SECONDS
You can fine tune its memory usage and...
How does it work?
How does it work?
How does it work?
How does it work?
How does it work?
Exporters
Exporters expose metrics with an HTTP API
Bindings available for many languages
Exporters do not save data ; the...
Exploring Metrics
Exploring Metrics
Exploring Metrics
Exploring Metrics
PromQL
mysql_global_status_commands_total
PromQL
mysql_global_status_commands_total{command="select"}
PromQL
mysql_global_status_commands_total
{command=~"select|set_options"}
PromQL
mysql_global_status_commands_total{command=~"select|set
_options"}
PromQL
deriv(mysql_global_status_connections[5m])
PromQL
{__name__=~".+innodb.+cache.*"}
predict_linear(mysql_heartbeat_lag_seconds[5m], 60*2)
sum(rate(mysql_global_status_...
Prometheus + mysqld_exporter
Collect a LOT of data
(e.g. global variables & status)
Embeds basic visualisation console
By ...
A word about
Prometheus vs Graphite
Prometheus does not see a metric as an "event".
Metrics are current value until they a...
One tool does one job...
Prometheus will collect data
Exporters will expose data
Grafana will graph data
Grafana
Open Source (Apache 2.0)
Web app
Specialized in visualization
Pluggable
Multiple datasources: prometheus, graphite...
History of Grafana
Grafana is a fork of Kibana 3 ; used to be JS-
Driven.
Now fully featured, requires a database, multi-
...
Grafana and Prometheus
Prometheus shipped its own consoles
Now it recommends Grafana and deprecated
its own consoles
Grafana Dashboards
Grafana Dashboards
Time Picker
Configure Prometheus in
Grafana
Configure Prometheus in
Grafana
Prometheus Dashboard
Creating Grafana Dashboards
Takes time
Requires deep knowledge of the tools
Improved over time
Easy to share (json + onlin...
Percona Grafana Dashboard
Percona Open Sourced Grafana Dashboards
Covering MySQL, Mongo and Linux monitoring
Part of a big...
Installing Percona Graphes
Method 1 (RO dashboards)
Enable File dashboards in Grafana
Clone grafana-dashboards to the conf...
MySQL Setup
You'll need mysqld_exporter, with a user
MySQL 5.1+
Performance Schema for full set of metrics
mysqld_exporter...
node_exporter setup
node_exporter
-collectors.enabled=
"diskstats,filefd,filesystem,loadavg,
meminfo,netdev,stat,time,unam...
Prometheus (static file)
scrape_configs:
  ­ job_name: prometheus
    static_configs:
      ­ targets: ['localhost:9090']
...
Dashboards
Dashboards
Dashboards
We don't need all of them?
Because Grafana is just viz, you can import
only the one you want (e.g. exclude Mongo)
You can ...
MySQL Overview
MySQL Overview
MySQL Overview
InnoDB
InnoDB
InnoDB
Replication
Conclusions
Prometheus and Grafana are first-class
monitoring tools
Totally different approach than other tools
Embeddable...
Julien Pivotto
roidelapluie
roidelapluie@inuits.eu
Inuits
https://inuits.eu
info@inuits.eu
Contact
Monitoring MySQL with Prometheus, Grafana and Percona Dashboards
Upcoming SlideShare
Loading in …5
×

of

Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 1 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 2 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 3 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 4 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 5 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 6 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 7 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 8 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 9 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 10 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 11 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 12 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 13 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 14 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 15 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 16 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 17 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 18 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 19 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 20 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 21 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 22 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 23 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 24 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 25 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 26 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 27 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 28 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 29 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 30 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 31 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 32 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 33 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 34 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 35 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 36 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 37 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 38 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 39 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 40 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 41 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 42 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 43 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 44 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 45 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 46 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 47 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 48 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 49 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 50 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 51 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 52 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 53 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 54 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 55 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 56 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 57 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 58 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 59 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 60 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 61 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 62 Monitoring MySQL with Prometheus, Grafana and Percona Dashboards Slide 63
Upcoming SlideShare
Getting Started Monitoring with Prometheus and Grafana
Next
Download to read offline and view in fullscreen.

4 Likes

Share

Download to read offline

Monitoring MySQL with Prometheus, Grafana and Percona Dashboards

Download to read offline

A talk that explains how monitoring mysql with prometheus works, together with grafana and Percona graphes. Given at percona university belgium.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Monitoring MySQL with Prometheus, Grafana and Percona Dashboards

  1. 1. Monitoring MySQL with Prometheus & Grafana Julien Pivotto (@roidelapluie) Percona University Belgium June 22nd, 2017
  2. 2. SELECT USER(); Julien "roidelapluie" Pivotto @roidelapluie Sysadmin at inuits Automation, monitoring, HA MySQL/MariaDB user/admin/contributor Grafana and Prometheus user/contributor
  3. 3. inuits
  4. 4. Monitoring Creative Commons Attribution 2.0 https://www.flickr.com/photos/kevandotorg/13829348104/
  5. 5. DevOps 4 keys principles Culture Automation Metrics Sharing Damon Edwards and John Willis, 2010
  6. 6. Metrics matters Find when a service is unavailable (before your users) Understand Failures (post-mortem) Learn from your infrastructure Anticipate
  7. 7. Collecting Metrics In general, you should collect as many metrics as you can Frequently (5 min collection are not enough) Store the metrics in a sane way (avoid averaging old metrics, etc)
  8. 8. How to collect metrics? Many many solutions, including: Graphite Elasticsearch beats Zabbix and .. Prometheus
  9. 9. Out of scope today Alerting Prometheus and Grafana installation Prometheus Fine Tuning But I'm happy to discuss that after the talks!
  10. 10. Prometheus https://prometheus.io/
  11. 11. Prometheus Prometheus is a Cloud-Native Data-Centric Open- Source Performant Simple metrics collection, analysis and alerting tool. Nothing more.
  12. 12. Cloud Native Easy to configure, deploy, maintain Designed in multiple services Container ready Orchestration ready (dynamic config) Fuzziness
  13. 13. Data Centric A Metric in Prometheus has metadata: myql_global_status_handlers_total{handler="tmp_write"} 1122 And lots of function to filter, change, remove... those metadata while fetching them.
  14. 14. Open Source Apache 2.0 Go Support for multiple OS Many "exporters": https://github.com/prometheus/prometheus/wiki/Defau lt-port-allocations
  15. 15. Performance Prometheus is designed to fetch data in an interval measured in SECONDS You can fine tune its memory usage and when it flushes to disk It can also adapt its scraping frequency dynamically
  16. 16. How does it work?
  17. 17. How does it work?
  18. 18. How does it work?
  19. 19. How does it work?
  20. 20. How does it work?
  21. 21. Exporters Exporters expose metrics with an HTTP API Bindings available for many languages Exporters do not save data ; they are not "proxies" and don't "cache" anything
  22. 22. Exploring Metrics
  23. 23. Exploring Metrics
  24. 24. Exploring Metrics
  25. 25. Exploring Metrics
  26. 26. PromQL mysql_global_status_commands_total
  27. 27. PromQL mysql_global_status_commands_total{command="select"}
  28. 28. PromQL mysql_global_status_commands_total {command=~"select|set_options"}
  29. 29. PromQL mysql_global_status_commands_total{command=~"select|set _options"}
  30. 30. PromQL deriv(mysql_global_status_connections[5m])
  31. 31. PromQL {__name__=~".+innodb.+cache.*"} predict_linear(mysql_heartbeat_lag_seconds[5m], 60*2) sum(rate(mysql_global_status_commands_total{command=~" (commit|rollback)"}[5m])) without (command)
  32. 32. Prometheus + mysqld_exporter Collect a LOT of data (e.g. global variables & status) Embeds basic visualisation console By default scraps every 15s
  33. 33. A word about Prometheus vs Graphite Prometheus does not see a metric as an "event". Metrics are current value until they are replaced. You can not see when a metric has been included in Prometheus. For Events, Prometheus refers to Elasticsearch.
  34. 34. One tool does one job... Prometheus will collect data Exporters will expose data Grafana will graph data
  35. 35. Grafana Open Source (Apache 2.0) Web app Specialized in visualization Pluggable Multiple datasources: prometheus, graphite, influxdb... Has an API!
  36. 36. History of Grafana Grafana is a fork of Kibana 3 ; used to be JS- Driven. Now fully featured, requires a database, multi- projects/users support, etc...
  37. 37. Grafana and Prometheus Prometheus shipped its own consoles Now it recommends Grafana and deprecated its own consoles
  38. 38. Grafana Dashboards
  39. 39. Grafana Dashboards
  40. 40. Time Picker
  41. 41. Configure Prometheus in Grafana
  42. 42. Configure Prometheus in Grafana
  43. 43. Prometheus Dashboard
  44. 44. Creating Grafana Dashboards Takes time Requires deep knowledge of the tools Improved over time Easy to share (json + online library)
  45. 45. Percona Grafana Dashboard Percona Open Sourced Grafana Dashboards Covering MySQL, Mongo and Linux monitoring Part of a bigger picture, PMM, but usable standalone Open Source (AGPL!) https://github.com/percona/grafana- dashboards
  46. 46. Installing Percona Graphes Method 1 (RO dashboards) Enable File dashboards in Grafana Clone grafana-dashboards to the configured location (or make a package) Method 2 (RW dashboards) Use the Grafana API to upload the JSON's.
  47. 47. MySQL Setup You'll need mysqld_exporter, with a user MySQL 5.1+ Performance Schema for full set of metrics mysqld_exporter -collect.binlog_size=true -collect.info_schema.processlist=true`
  48. 48. node_exporter setup node_exporter -collectors.enabled= "diskstats,filefd,filesystem,loadavg, meminfo,netdev,stat,time,uname,vmstat"
  49. 49. Prometheus (static file) scrape_configs:   ­ job_name: prometheus     static_configs:       ­ targets: ['localhost:9090']         labels:           instance: prometheus   ­ job_name: linux     static_configs:       ­ targets: ['10.0.98.43:9100']         labels:           instance: db1   ­ job_name: mysql     static_configs:       ­ targets: ['10.0.98.43:9104']         labels:           instance: db1
  50. 50. Dashboards
  51. 51. Dashboards
  52. 52. Dashboards
  53. 53. We don't need all of them? Because Grafana is just viz, you can import only the one you want (e.g. exclude Mongo) You can import later any extra dashboard you need
  54. 54. MySQL Overview
  55. 55. MySQL Overview
  56. 56. MySQL Overview
  57. 57. InnoDB
  58. 58. InnoDB
  59. 59. InnoDB
  60. 60. Replication
  61. 61. Conclusions Prometheus and Grafana are first-class monitoring tools Totally different approach than other tools Embeddable into your apps Percona Dashboards gets your graphes ready in no-time with minimal efforts
  62. 62. Julien Pivotto roidelapluie roidelapluie@inuits.eu Inuits https://inuits.eu info@inuits.eu Contact
  • MrEntrySKY

    Oct. 20, 2019
  • ssuser534d60

    May. 3, 2018
  • ssuserdf2a60

    Feb. 11, 2018
  • 3908282

    Jun. 26, 2017

A talk that explains how monitoring mysql with prometheus works, together with grafana and Percona graphes. Given at percona university belgium.

Views

Total views

3,820

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

109

Shares

0

Comments

0

Likes

4

×