slide for spike Prometheus @ Proteus Operations

1. Prometheus
prometheus.io

2. What is Prometheus?
- Open-source monitoring & alerting system
- Built at SoundCloud
- Stores as time-series collection
- Written in Go language

3. Service Workflow

4. Components
● Prometheus server
● Metrics exporter
● PromDash (Grafana also supported)
● Alertmanager
● pushgateway (optional)

5. Data Model
- Stores data as time-series
- Structure as key-value pairs:
ex. api_http_requests_total {
method="POST",
handler="/messages" }

6. Metrics
- Value extracted from client
- 4 types of metric:
- Counter = numerical value that only goes up.
- Gauge = numerical value that can go up & down.
- Histogram
- Summary
more info on last 2 metrics: http://prometheus.io/docs/practices/histograms/

7. Exporter
- Pre-built application to exporting metrics
from various system.
- Lot of ready-to-deploy exporter includes:
- Linux system exporter
- Docker exporter
- MySQL exporter
- Django exporter
- SNMP exporter
- and many more...

8. Making your own exporter
Official client library support includes:
- Go
- Python
- Ruby
- Java
- Scala
- and many more unofficial support...

9. Query language
- Query metrics from server using expression
ex. rate(http_requests_total[5m]):
total HTTP request as measured over the
last 5 minutes.

10. PromDash
- Frontend GUI for Prometheus server
- using query language to visualize metrics
- Using Ruby on Rail framework

11. PromDash

12. Alerting
- using Alertmanager to handle all alerting
- divide into 2 parts:
- Alerting rules on Prometheus server
- Alertmanager handle the alert and sending
notification

13. Alerting rules
- Define what to alert on Prometheus server
- Prometheus send the alert to alertmanager
ex.
ALERT <alert name>
IF <expression>
[FOR <duration>]
[WITH <label set>]
SUMMARY "<summary>"
DESCRIPTION "<description>"

14. Alertmanager
- managing alert from Prometheus server
- Sending notification to various system:
- Email
- PagerDuty
- PushOver
- HipChat
- Slack
- Flowdock
- Generic Webhook API

15. The End
Thank you for listening!
Credit: http://prometheus.io
https://developers.soundcloud.com/blog/prometheus-monitoring-at-soundcloud
http://www.slideshare.net/brianbrazil/devops-ireland-systems-monitoring-with-prometheus