demo talk on service discovery mechanisms in prometheus. held at infracoders vienna

1. Service Discovery in Prometheus
Oliver Moser

2. The Guy
2
- TU Wien PhD
- Fellow @ TAG
- Cloud / Automation / SRE

3. The Talk
3
- Setup infrastructure
- Why Service Discovery
- Showcase SD options
- Static
- DNS
- Consul

4. The Tech
4
- Terraform
- Cloud-Init
- Prometheus
- Grafana
- DNS
- Consul
- Spring Boot

5. The Setup
5
prom-master-0
----------------
- Prometheus
- Grafana
- Consul
prom-node-0
----------------
- node_exporter
- prom-boot
prom-node-1
----------------
- node_exporter
- prom-boot

6. Demo
> Terraforming

7. Motivation
7
- Locating nearby services
- Not a new concept (cf. mDNS)
- In Prometheus: Learn scrape targets
- Static approach might work small scale
- Does not working large scale
- Microservices (of course!)
- Cluster schedulers like Kubernetes

8. Service Discovery
Options
8
- azure_sd_config
- consul_sd_config
- dns_sd_config
- ec2_sd_config
- openstack_sd_config
- file_sd_config
- gce_sd_config
- kubernetes_sd_config
- marathon_sd_config
- nerve_sd_config
- serverset_sd_config
- triton_sd_config
- static_config

9. Service Discovery
Options
9
- azure_sd_config
- consul_sd_config
- dns_sd_config
- ec2_sd_config
- openstack_sd_config
- file_sd_config
- gce_sd_config
- kubernetes_sd_config
- marathon_sd_config
- nerve_sd_config
- serverset_sd_config
- triton_sd_config
- static_config

10. static_configs
- Canonical way to declare static scrape targets
- Not really a service discovery mechanism
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['node-0:9090','node-1:9090']

11. dns_sd_configs
- Queries list of DNS names to learn scrape targets
- Supports A, AAAA and SRV records
scrape_configs:
- job_name: 'prometheus'
dns_sd_configs:
- names:
- 'prometheus-sd.prometheus.exokube.eu'

12. consul_sd_configs
- Queries Consul catalog API to learn scrape targets
- Consul metadata can be used via relabeling
scrape_configs:
- job_name: 'prometheus'
consul_sd_configs:
- server: 'localhost:8500'

13. The Targets
13
prom-master-0
----------------
- Prometheus
- Grafana
- Consul
prom-node-0
----------------
- node_exporter (9100)
- prom-boot (8080,18080)
prom-node-1
----------------
- node_exporter (9100)
- prom-boot (8080,18080)

14. prom-boot
- Minimal Spring Boot microservice
- Registers with Consul (unless disabled)
sudo docker run -d
-e SPRING_CLOUD_CONSUL_DISCOVERY_INSTANCE_ID="prom-boot-0"
-e SPRING_CLOUD_CONSUL_DISCOVERY_HOSTNAME=$(hostname -f)
-e SERVER_PORT=18080
-e SPRING_CLOUD_CONSUL_HOST=prom-master-0.prometheus.exokube.eu
-p 18080:18080 omoser/prom-boot
sudo docker run
-e SPRING_CLOUD_CONSUL_DISCOVERY_ENABLED=false
-p 8080:8080 omoser/prom-boot

15. Demo
> Service Discovery

16. The Code
16
https://github.com/olmoser/infracoders-reloaded

17. The End
17
Thank You