Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Prometheus kubernetes tech talk

Application monitoring on Kubernetes

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Prometheus kubernetes tech talk

  1. 1. Application monitoring on Kubernetes using Prometheus Chandresh Pancholi Engineer at AI
  2. 2. Kubernetes at Arvind Internet ● Our Infra is deployed on AWS ● Kubernetes minions are running on m4.xlarge instances ● Kubernetes version 1.7.5 in QA/Prod, 1.8.3 on Pre-prod ● QA/Dev, Pre-Prod & Production running on Kubernetes ● Total Pods ⇒ More than 350 (QA/Dev, Prod) ● Total services ⇒ More than 200 (QA/Dev, Prod) ● Running Mongo, MySQL, Redis, Hazelcast in Kubernetes in QA/Dev
  3. 3. Monitoring at AI (earlier) EC2 Sensu Kubernetes µServices
  4. 4. Cons 1. Multiple monitoring system 2. Difficulty in Troubleshooting 3. Additional Infrastructure cost to support three monitoring system 4. Graphite doesn’t provide pod level Application metrics 5. Infra team need to understand Sensu, Prometheus alerting 6. Application metrics are single dimension Ex. (a.b.c.d.99) 7. Grafana alerting for Application metrics
  5. 5. Prometheus ● It developed at SoundCloud by ex-Googlers ● Prometheus is a close cousin of Kubernetes ● A multi-dimensional data model with time series data identified by metric name and key/value pairs ● Alerting and graphing are unified, using the same language. ● Time series collection happens via a pull model over HTTP ● Targets are discovered via service discovery or static configuration ● Provides multiple exporters to send EC2, Kafka, Mongo, Cassandra, RMQ, Redis metrics
  6. 6. Sample metrics Image source: google
  7. 7. Alerting
  8. 8. Approach #1 - Prometheus on EC2 EC2 Kubernetes node ex µServices AWS EC2
  9. 9. #1. Getting EC2 server metrics is quite easy and straightforward. Prometheus provides EC2 discovery. #2. Getting Kubernetes and Application metrics is very complex. It has 300+ lines of configuration to support just Kubernetes metrics
  10. 10. Approach #2. Use Prometheus operator
  11. 11. What is Prometheus operator? The Prometheus Operator creates, configures, and manages Prometheus monitoring instances. Automatically generates monitoring target configurations based on familiar Kubernetes label queries.
  12. 12. Service monitor Custom Resource Definition(CRD)
  13. 13. Prometheus Custom Resource Definition (CRD)
  14. 14. Future work ● High availability of Prometheus ● Alert integration with Slack ● Sending frameworks(Kafka, Redis, Mongo) metrics to Prometheus

×