Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Prometheus

262 views

Published on

A general introduction and demo of the prometheus monitoring solution and ecosystem with a live demo, given at FLOSSUK 2018.

Published in: Technology
  • Be the first to comment

Introduction to Prometheus

  1. 1. Prometheus Julien Pivotto (@roidelapluie) FLOSS UK Spring Conference April 26th, 2018
  2. 2. user{name="roidelapluie"} 1 I like Open Source I like monitoring I like automation ... and all of that is my daily job at inuits
  3. 3. Traditional Monitoring It's up No it's down It is degraded I don't know
  4. 4. Questions that come after: It's up but it is performant? It's down but for everyone? Its is degraded but are the users impacted? Is it even relevant?
  5. 5. Metrics Monitoring Gather fine grained data at frequent interval Make them useful by labelling them ; store them Analyze them to understand what is going on
  6. 6. We are in the cloud era. Here are some buzzwords for you cloud, API, openstack, devops, docker, bimodal, stateless, kubernetes, orchestration, automation, serverless, docker, humanops, ansible, continuous deployment, cri-o, jenkins, agile, docker, red hat, containers, virtualization, provisionning, monitoring, observability...
  7. 7. What is the cloud Scale Velocity Change
  8. 8. On Premise looks like the cloud Nowadays you have no choice. Scale Velocity Change
  9. 9. What are the needs Automation Scalability
  10. 10. Bye bye all-in-one tools tools that don't scale tools you can not automate
  11. 11. We need deserve better tools Our customers ask us to respond fast, in seconds We make hundreds of operations per second What is your monitoring frequency... 5 minutes?
  12. 12. Time to get better tools / protocols
  13. 13. Prometheus https://prometheus.io/
  14. 14. Cloud Native Easy to configure, deploy, maintain Designed in multiple services Container ready Orchestration ready (dynamic config) Fuzziness
  15. 15. Data Centric A Metric in Prometheus has metadata: myql_global_status_handlers_total{handler="tmp_write"} 1122 And lots of function to filter, change, remove... those metadata while fetching them.
  16. 16. Open Source Apache 2.0 Go Support for multiple OS Many "exporters": https://github.com/prometheus/prometheus/wiki/Defa ult-port-allocations
  17. 17. Efficient Prometheus is designed to fetch data in an interval measured in seconds Millions of datapoints Big improvements in prometheus 2.0
  18. 18. Storage Prometheus 2.x uses prometheus/tsdb 2 hours blocks ; later compacted in up to 10 days blocks 15000 metrics/s ends up ad 800 MiB/day
  19. 19. How does it work?
  20. 20. How does it work?
  21. 21. How does it work?
  22. 22. How does it work?
  23. 23. How does it work?
  24. 24. Exporters Exporters expose metrics with an HTTP API Bindings available for many languages Exporters do not save data ; they are not "proxies" and don't "cache" anything
  25. 25. Exploring Metrics
  26. 26. Exploring Metrics
  27. 27. Exploring Metrics
  28. 28. Exploring Metrics
  29. 29. PromQL mysql_global_status_commands_total
  30. 30. PromQL mysql_global_status_commands_total{command="select"}
  31. 31. PromQL mysql_global_status_commands_total {command=~"select|set_options"}
  32. 32. PromQL mysql_global_status_commands_total{command=~"select|se t_options"}
  33. 33. PromQL deriv(mysql_global_status_connections[5m])
  34. 34. PromQL {__name__=~".+innodb.+cache.*"} predict_linear(mysql_heartbeat_lag_seconds[5m],�60*2) sum(rate(mysql_global_status_commands_total{command=~" (commit|rollback)"}[5m]))�without�(command)
  35. 35. Demo node_exporter grok_exporter blackbox_exporter snmp_exporter service discovery
  36. 36. Security Prometheus supports TLS client (also with authentication) Exporter side is your business We use it with traefik (reverse proxy in go with native metrics) We manage certs with ansible
  37. 37. Alerting Prometheus has recording rules Record frequent queries or alerting Alert base on any PromQL queries.
  38. 38. One tool does one job... Prometheus collects data Exporters expose data Grafana graphes data Alertmanager dispatches alerts
  39. 39. Alertmanager Open Source Same developers as Prometheus Second service
  40. 40. What is the Alertmanager doing? Receives alerts Group them Inhibits them Dispatches them Deals with HA
  41. 41. How to alerts? Email Some vendors: Slack, Hipchat, VictorOps, pagerduty, ... Generic Wehbook -> Plugin anything you want
  42. 42. High Availability 2 prometheus servers do exact the same job They send alerts to Alertmanagers Alertmanagers are clustered not to send the same notification twice
  43. 43. Grafana Open Source (Apache 2.0) Web app Specialized in visualization Pluggable Multiple datasources: prometheus, graphite, influxdb... Has an API!
  44. 44. History of Grafana Grafana is a fork of Kibana 3 ; used to be JS- Driven. Now fully featured, requires a database, multi- projects/users support, etc...
  45. 45. Grafana and Prometheus Prometheus shipped its own consoles Now it recommends Grafana and deprecated its own consoles
  46. 46. Grafana Dashboards
  47. 47. Grafana Dashboards
  48. 48. Time Picker
  49. 49. Configure Prometheus in Grafana
  50. 50. Configure Prometheus in Grafana
  51. 51. Prometheus Dashboard
  52. 52. Creating Grafana Dashboards Takes time Requires deep knowledge of the tools Improved over time Easy to share (json + online library)
  53. 53. Conclusion Open Source Flexible and dynamic Multiple small pieces Low on resources Rich ecosystem For Cloud and On Premise
  54. 54. Julien Pivotto roidelapluie roidelapluie@inuits.eu Inuits https://inuits.eu info@inuits.eu Contact I will give a prometheus workshop at OSMC.de!

×