Successfully reported this slideshow.
Your SlideShare is downloading. ×

5 tips for Prometheus Service Discovery

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 11 Ad
Advertisement

More Related Content

More from Julien Pivotto (20)

Recently uploaded (20)

Advertisement

5 tips for Prometheus Service Discovery

  1. 1. PromCon Online Julien Pivotto @roidelapluie 5 tips to better use Prometheus Service Discovery July 15th, 2020
  2. 2. • Used to discover targets • Listens to events from target (e.g. consul) • Queries targets at a certain interval (e.g. DNS) Service Discovery in Prometheus @roidelapluie
  3. 3. scrape_configs: - job_name: node digitalocean_sd_configs: - bearer_token: "<replace with a Personal Access Token>" relabel_configs: - source_labels: [__meta_digitalocean_tags] regex: '.*,monitoring,.*' action: keep - source_labels: [__meta_digitalocean_public_ipv6] target_label: __address__ replacement: '[$1]:9100' Classic pattern: Service discovery + Relabel con g. Relabel con g = Change labels found by SD to alert scrape con g. Classic exemple @roidelapluie
  4. 4. alerting: alertmanagers: digitalocean_sd_configs: - bearer_token: "<replace with a Personal Access Token>" relabel_configs: - source_labels: [__meta_digitalocean_tags] regex: '.*,monitoring,.*' action: keep Automatically discover alertmanager instances. Tip 1: Alerting uses service discovery @roidelapluie
  5. 5. scrape_configs: - job_name: node digitalocean_sd_configs: - bearer_token: "<Personal Access Token1>" - bearer_token: "<Personal Access Token2>" - bearer_token: "<Personal Access Token3>" relabel_configs: - source_labels: [__meta_digitalocean_tags] regex: '.*,monitoring,.*' action: keep - source_labels: [__meta_digitalocean_public_ipv6] target_label: __address__ replacement: '[$1]:9100' You can use multiple sources in one job. Tip 2: SD con gs are lists @roidelapluie
  6. 6. scrape_configs: - job_name: node static_configs: - targets: [127.0.0.1:9090] gce_sd_configs: - zone: europe-west1-b project: myproject digitalocean_sd_configs: - bearer_token: "<Personal Access Token1>" - bearer_token: "<Personal Access Token2>" You can use multiple kinds of SD for one job. Tip 3: Multiple SD for one job @roidelapluie
  7. 7. - job_name: node digitalocean_sd_configs: - bearer_token: "<Personal Access Token1>" relabel_configs: - source_labels: [__meta_digitalocean_private_ipv4] target_label: __address__ replacement: '[$1]:9100' - job_name: cadvisor digitalocean_sd_configs: - bearer_token: "<Personal Access Token1>" relabel_configs: - source_labels: [__meta_digitalocean_private_ipv4] target_label: __address__ replacement: '[$1]:8080' Tip 4: Reuse SD for multiple jobs @roidelapluie
  8. 8. Prometheus will REUSE the same API calls/connection if the SD con g is exactly identical. Check in DEBUG logs: caller=manager.go:226 component="discovery manager scrape" msg="Starting provider" provider=*digitalocean.SDConfig/0 subs="[node cadvisor]" Tip 4: Reuse SD for multiple jobs (cont) @roidelapluie
  9. 9. scrape_configs: - job_name: node file_sd_configs: - files: - targets.yml Tip 5: le_sd les can contain con g labels @roidelapluie
  10. 10. targets.yml: - labels: __scheme__: https __metrics_path__: /metrics2 __param_foo: bar targets: [localhost:9100] - labels: __param_foo: bar2 targets: [localhost:9101] https://localhost:9100/metrics2?foo=bar http://localhost:9101/metrics?foo=bar2 Tip 5: le_sd les can contain con g labels (cont) @roidelapluie
  11. 11. Julien Pivotto @roidelapluie roidelapluie@inuits.eu Essensteenweg 31 2930 Brasschaat Belgium Contact: info@inuits.eu +32-3-8082105

×