SlideShare a Scribd company logo
1 of 51
Download to read offline
Incident Resolution as Code
Julien Pivotto (@roidelapluie)
Config Management Camp
February 4th, 2019
user{name="roidelapluie"} 1
I like Open Source
I like monitoring
I like automation
... and all of that is my daily job at inuits
Monitoring
Creative Commons Attribution 2.0 https://www.flickr.com/photos/24375810@N06/3719090065
Creative Commons Attribution ShareAlike 2.0 https://www.flickr.com/photos/grendelkhan/400428874
Creative Commons Public Domain https://pxhere.com/en/photo/265717
Creative Commons Attribution 2.0 https://www.flickr.com/photos/51809988@N06/5229933669
Traditional Monitoring
It works - OK
It does not work - CRITICAL
It kinda works - WARNING
I don't know - UNKNOWN
Creative Commons Public Domain https://pxhere.com/fr/photo/952999
Creative Commons Attribution 2.0 https://www.flickr.com/photos/wwarby/2460655511
Creative Commons Attribution-Share Alike 3.0 Unported
https://commons.wikimedia.org/wiki/File:CUPE3903-picketLine-20180504.jpg
Real world
It works ; it does not work ; it kinda works ; it maybe
works ; no one uses it ; it is broken ; some things
are broken ; it should work but it does not ; where
are my users? help me...
The Technical bias
By looking at technical service, we miss the
actual point
Are we serving our users correctly?
Just looking at the traffic light will not tell you
about the traffic jams.
Observability
Creative Commons Attribution 2.0 https://www.flickr.com/photos/24375810@N06/3719090065
Metrics
Creative Commons Attribution-Share Alike 2.0 https://www.flickr.com/photos/tillwe/11892564676/
Metric
Name
Labels (Key-Value Pairs)
Value (Number)
Timestamp
Fetched at a high frequency
Business Metrics
Creative Commons Attribution 2.0 https://www.flickr.com/photos/89228431@N06/11323330056
CPU usage is no money
Creative Commons Attribution-ShareAlike 2.0
https://www.flickr.com/photos/nox_noctis_silentium/3960497840
What are business metrics?
how you fullfil your customers' requests
quality and level of business service
Where are we?
Creative Commons Attribution-ShareAlike 2.0 https://www.flickr.com/photos/hernanpc/6259950189
What do we have?
metrics that tell us if business works
DB, Frontends, balancers, queing systems...
They don't come from the troublesome
component!
Recap on High Availability
Traditional High Availability
LinuxHA, pacemaker, corosync
"health check" script
Restart, promote, balance traffic elsewhere
High Availability Nowadays
Multiple workers
Health exposed by the app
Load balancer balances to healthy nodes
Unhealty nodes are restarted automatically
When HA is not enough...
the processes are not "really" crashing
the component that has issues does not really
know about them (metrics available from DB ,
frontends, clients..)
there is no HA in place... (but still need 24x7
availability)
Async cases
Backlog can be delayed by a few hours
But not a complete week-end ...
Alerting
alert is fired = someone to take action
Runbooks to follow, depending on the alert
knowledge is built, then runbook == (ansible)
playbook
An ideal world
Creative Commons Attribution 2.0 https://www.flickr.com/photos/athomeinscottsdale/3247600886/
Ideally...
Memory leaks are fixed (quickly)
Multi master, redundant, in service discovery
You build it, you run it
Full control over 3rd parties (and their bugs..)
Ironically
Developers often just payed for features
Ops not working closely with devs
No "bugfix money" for stuff that do not happen
really often
Code base is 20y old and "it will be
decommisioned soon"
What's next?
Creative Commons Attribution 2.0 https://www.flickr.com/photos/janitors/15795816662/
if Frontends says backend responds slowly
then Restart the backend
if Lots of write errors towards NFS
then Balance traffic to another datacenter
Challenges
How to prevent concurrent ansible runs?
How to avoid large scale failure? e.g. errors in
playbook?
How to make sure it works?
In pratice
Prometheus
Time series collection database
Computing rules, creating alerts
View on all the components
Open Source
Alertmanager
Receives alerts from prometheus
Propagates the alerts via webhooks
Open Source (part of Prometheus)
Ansible
Open Source orchestration/cfgmgmt
Acts on multiple servers
Knows your infra
Webhook that call Ansible
How/where to get the credentials? Where to
run Ansible?
Duration of the ansible run?
Which server to act upon?
Concurrent playbooks?
Queues
Alerts as JSON
Labels: env, playbook, limit
alert2amqp
AMQP protocol
Apache Artemis message broker (activemq
family)
Consumer
amqp2jenkins
Reads from queue
Launch a Jenkins job that will run ansible
Stop processing if a Jenkins job is failing
Ansible jobs take 2 input
Recap
In an ideal world we do not need this. Bugs are
fixed, techno is up to date, infra and apps are
reduntant.
Achievement
Metrics from 1, 2, X sources generate alerts that
are triggering automated resolution within minutes
towards different systems.
Common incidents get solved more quickly than
with people intervention.
People are woken up less often for known issue
with clear runbook.
Safeties
Needs monitoring to be up
Needs the last ansible run to be green
Whitelist upfrond
Discards "old alerts"
No concurrent execution
Alerts someone if not resolved in time
Use Cases
Must happen infrequently
Must not be predictable
Must not do more harm
Must impact daily work or on call
Code
https://github.com/roidelapluie/alert2amqp
https://github.com/roidelapluie/amqp2jenkins
Julien Pivotto
roidelapluie
roidelapluie@inuits.eu
Inuits
https://inuits.eu
info@inuits.eu
Contact

More Related Content

What's hot

Monitoring MySQL with Prometheus, Grafana and Percona Dashboards
Monitoring MySQL with Prometheus, Grafana and Percona DashboardsMonitoring MySQL with Prometheus, Grafana and Percona Dashboards
Monitoring MySQL with Prometheus, Grafana and Percona DashboardsJulien Pivotto
 
An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)Brian Brazil
 
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Brian Brazil
 
How to monitor your micro-service with Prometheus?
How to monitor your micro-service with Prometheus?How to monitor your micro-service with Prometheus?
How to monitor your micro-service with Prometheus?Wojciech Barczyński
 
Real world functional reactive programming
Real world functional reactive programmingReal world functional reactive programming
Real world functional reactive programmingEric Polerecky
 
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...Brian Brazil
 
Monitoring with prometheus
Monitoring with prometheusMonitoring with prometheus
Monitoring with prometheusKasper Nissen
 
Understanding and Extending Prometheus AlertManager
Understanding and Extending Prometheus AlertManagerUnderstanding and Extending Prometheus AlertManager
Understanding and Extending Prometheus AlertManagerLee Calcote
 
Hands-on monitoring with Prometheus
Hands-on monitoring with PrometheusHands-on monitoring with Prometheus
Hands-on monitoring with PrometheusBrice Fernandes
 
Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)SOASTA
 
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
SaltConf 2015: Salt stack at web scale: Better, Stronger, FasterSaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
SaltConf 2015: Salt stack at web scale: Better, Stronger, FasterThomas Jackson
 
Lock? We don't need no stinkin' locks!
Lock? We don't need no stinkin' locks!Lock? We don't need no stinkin' locks!
Lock? We don't need no stinkin' locks!Michael Barker
 
Practical Operation Automation with StackStorm
Practical Operation Automation with StackStormPractical Operation Automation with StackStorm
Practical Operation Automation with StackStormShu Sugimoto
 
We Are All Testers Now: The Testing Pyramid and Front-End Development
We Are All Testers Now: The Testing Pyramid and Front-End DevelopmentWe Are All Testers Now: The Testing Pyramid and Front-End Development
We Are All Testers Now: The Testing Pyramid and Front-End DevelopmentAll Things Open
 
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SP
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SPPrimeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SP
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SPZabbix BR
 
Puppet Camp Austin 2015: Getting Started with Puppet
Puppet Camp Austin 2015: Getting Started with PuppetPuppet Camp Austin 2015: Getting Started with Puppet
Puppet Camp Austin 2015: Getting Started with PuppetPuppet
 
Serverless in production, an experience report (codemotion milan)
Serverless in production, an experience report (codemotion milan)Serverless in production, an experience report (codemotion milan)
Serverless in production, an experience report (codemotion milan)Yan Cui
 
Jenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJulien Pivotto
 
Monitoring kubernetes with prometheus
Monitoring kubernetes with prometheusMonitoring kubernetes with prometheus
Monitoring kubernetes with prometheusBrice Fernandes
 

What's hot (20)

Monitoring MySQL with Prometheus, Grafana and Percona Dashboards
Monitoring MySQL with Prometheus, Grafana and Percona DashboardsMonitoring MySQL with Prometheus, Grafana and Percona Dashboards
Monitoring MySQL with Prometheus, Grafana and Percona Dashboards
 
An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)
 
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
 
How to monitor your micro-service with Prometheus?
How to monitor your micro-service with Prometheus?How to monitor your micro-service with Prometheus?
How to monitor your micro-service with Prometheus?
 
Prometheus course
Prometheus coursePrometheus course
Prometheus course
 
Real world functional reactive programming
Real world functional reactive programmingReal world functional reactive programming
Real world functional reactive programming
 
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...
Better Monitoring for Python: Inclusive Monitoring with Prometheus (Pycon Ire...
 
Monitoring with prometheus
Monitoring with prometheusMonitoring with prometheus
Monitoring with prometheus
 
Understanding and Extending Prometheus AlertManager
Understanding and Extending Prometheus AlertManagerUnderstanding and Extending Prometheus AlertManager
Understanding and Extending Prometheus AlertManager
 
Hands-on monitoring with Prometheus
Hands-on monitoring with PrometheusHands-on monitoring with Prometheus
Hands-on monitoring with Prometheus
 
Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)Velocity 2015 building self healing systems (slide share version)
Velocity 2015 building self healing systems (slide share version)
 
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
SaltConf 2015: Salt stack at web scale: Better, Stronger, FasterSaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
 
Lock? We don't need no stinkin' locks!
Lock? We don't need no stinkin' locks!Lock? We don't need no stinkin' locks!
Lock? We don't need no stinkin' locks!
 
Practical Operation Automation with StackStorm
Practical Operation Automation with StackStormPractical Operation Automation with StackStorm
Practical Operation Automation with StackStorm
 
We Are All Testers Now: The Testing Pyramid and Front-End Development
We Are All Testers Now: The Testing Pyramid and Front-End DevelopmentWe Are All Testers Now: The Testing Pyramid and Front-End Development
We Are All Testers Now: The Testing Pyramid and Front-End Development
 
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SP
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SPPrimeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SP
Primeiros Passos na API do Zabbix com Python - 2º ZABBIX MEETUP DO INTERIOR-SP
 
Puppet Camp Austin 2015: Getting Started with Puppet
Puppet Camp Austin 2015: Getting Started with PuppetPuppet Camp Austin 2015: Getting Started with Puppet
Puppet Camp Austin 2015: Getting Started with Puppet
 
Serverless in production, an experience report (codemotion milan)
Serverless in production, an experience report (codemotion milan)Serverless in production, an experience report (codemotion milan)
Serverless in production, an experience report (codemotion milan)
 
Jenkins Shared Libraries Workshop
Jenkins Shared Libraries WorkshopJenkins Shared Libraries Workshop
Jenkins Shared Libraries Workshop
 
Monitoring kubernetes with prometheus
Monitoring kubernetes with prometheusMonitoring kubernetes with prometheus
Monitoring kubernetes with prometheus
 

Similar to Incident Resolution as Code

StackStrom: If-This-Than-That for Devops Automation
StackStrom: If-This-Than-That for Devops AutomationStackStrom: If-This-Than-That for Devops Automation
StackStrom: If-This-Than-That for Devops AutomationDmitri Zimine
 
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)VMware Tanzu
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys AdminsPuppet
 
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSkynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSylvain Kalache
 
Velocity 2015 Amsterdam: Alerts overload
Velocity 2015 Amsterdam: Alerts overloadVelocity 2015 Amsterdam: Alerts overload
Velocity 2015 Amsterdam: Alerts overloadsarahjwells
 
Velocity 2015: Building Self-Healing Systems
Velocity 2015: Building Self-Healing SystemsVelocity 2015: Building Self-Healing Systems
Velocity 2015: Building Self-Healing SystemsSOASTA
 
Chaos Engineering Without Observability ... Is Just Chaos
Chaos Engineering Without Observability ... Is Just ChaosChaos Engineering Without Observability ... Is Just Chaos
Chaos Engineering Without Observability ... Is Just ChaosCharity Majors
 
MySQL Monitoring Shoot Out
MySQL Monitoring Shoot OutMySQL Monitoring Shoot Out
MySQL Monitoring Shoot OutKris Buytaert
 
Security Automation Simplified - BSides Austin 2019
Security Automation Simplified - BSides Austin 2019Security Automation Simplified - BSides Austin 2019
Security Automation Simplified - BSides Austin 2019Moses Schwartz
 
Monitoring - deeper dive
Monitoring  - deeper diveMonitoring  - deeper dive
Monitoring - deeper diveRobert Kubiś
 
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...Puppet
 
Monitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaMonitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaJulien Pivotto
 
OSMC 2017 | Monitoring MySQL with Prometheus and Grafana by Julien Pivotto
OSMC 2017 | Monitoring  MySQL with Prometheus and Grafana by Julien PivottoOSMC 2017 | Monitoring  MySQL with Prometheus and Grafana by Julien Pivotto
OSMC 2017 | Monitoring MySQL with Prometheus and Grafana by Julien PivottoNETWAYS
 
Open Source Monitoring Tools Shootout
Open Source Monitoring Tools ShootoutOpen Source Monitoring Tools Shootout
Open Source Monitoring Tools Shootouttomdc
 
Making Security Agile - Oleg Gryb
Making Security Agile - Oleg GrybMaking Security Agile - Oleg Gryb
Making Security Agile - Oleg GrybSeniorStoryteller
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUGslandelle
 
Monitoring shootout loadays
Monitoring shootout loadaysMonitoring shootout loadays
Monitoring shootout loadaystomdc
 
Process control daemon
Process control daemonProcess control daemon
Process control daemonhaish
 

Similar to Incident Resolution as Code (20)

StackStrom: If-This-Than-That for Devops Automation
StackStrom: If-This-Than-That for Devops AutomationStackStrom: If-This-Than-That for Devops Automation
StackStrom: If-This-Than-That for Devops Automation
 
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)
Spring Boot & Spring Cloud on PAS- Nate Schutta (2/2)
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the CloudSkynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
 
Velocity 2015 Amsterdam: Alerts overload
Velocity 2015 Amsterdam: Alerts overloadVelocity 2015 Amsterdam: Alerts overload
Velocity 2015 Amsterdam: Alerts overload
 
Velocity 2015: Building Self-Healing Systems
Velocity 2015: Building Self-Healing SystemsVelocity 2015: Building Self-Healing Systems
Velocity 2015: Building Self-Healing Systems
 
Chaos Engineering Without Observability ... Is Just Chaos
Chaos Engineering Without Observability ... Is Just ChaosChaos Engineering Without Observability ... Is Just Chaos
Chaos Engineering Without Observability ... Is Just Chaos
 
MySQL Monitoring Shoot Out
MySQL Monitoring Shoot OutMySQL Monitoring Shoot Out
MySQL Monitoring Shoot Out
 
Security Automation Simplified - BSides Austin 2019
Security Automation Simplified - BSides Austin 2019Security Automation Simplified - BSides Austin 2019
Security Automation Simplified - BSides Austin 2019
 
Monitoring - deeper dive
Monitoring  - deeper diveMonitoring  - deeper dive
Monitoring - deeper dive
 
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...
PuppetConf 2017: Deploying is Only Half the Battle! Operationalizing Applicat...
 
Monitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaMonitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and Grafana
 
OSMC 2017 | Monitoring MySQL with Prometheus and Grafana by Julien Pivotto
OSMC 2017 | Monitoring  MySQL with Prometheus and Grafana by Julien PivottoOSMC 2017 | Monitoring  MySQL with Prometheus and Grafana by Julien Pivotto
OSMC 2017 | Monitoring MySQL with Prometheus and Grafana by Julien Pivotto
 
Open Source Monitoring Tools Shootout
Open Source Monitoring Tools ShootoutOpen Source Monitoring Tools Shootout
Open Source Monitoring Tools Shootout
 
Making Security Agile - Oleg Gryb
Making Security Agile - Oleg GrybMaking Security Agile - Oleg Gryb
Making Security Agile - Oleg Gryb
 
Techno-Fest-15nov16
Techno-Fest-15nov16Techno-Fest-15nov16
Techno-Fest-15nov16
 
It Works On Dev
It Works On DevIt Works On Dev
It Works On Dev
 
Gatling - Bordeaux JUG
Gatling - Bordeaux JUGGatling - Bordeaux JUG
Gatling - Bordeaux JUG
 
Monitoring shootout loadays
Monitoring shootout loadaysMonitoring shootout loadays
Monitoring shootout loadays
 
Process control daemon
Process control daemonProcess control daemon
Process control daemon
 

More from Julien Pivotto

What's New in Prometheus and Its Ecosystem
What's New in Prometheus and Its EcosystemWhat's New in Prometheus and Its Ecosystem
What's New in Prometheus and Its EcosystemJulien Pivotto
 
Prometheus: What is is, what is new, what is coming
Prometheus: What is is, what is new, what is comingPrometheus: What is is, what is new, what is coming
Prometheus: What is is, what is new, what is comingJulien Pivotto
 
What's new in Prometheus?
What's new in Prometheus?What's new in Prometheus?
What's new in Prometheus?Julien Pivotto
 
Introduction to Grafana Loki
Introduction to Grafana LokiIntroduction to Grafana Loki
Introduction to Grafana LokiJulien Pivotto
 
Why you should revisit mgmt
Why you should revisit mgmtWhy you should revisit mgmt
Why you should revisit mgmtJulien Pivotto
 
Observing the HashiCorp Ecosystem From Prometheus
Observing the HashiCorp Ecosystem From PrometheusObserving the HashiCorp Ecosystem From Prometheus
Observing the HashiCorp Ecosystem From PrometheusJulien Pivotto
 
Monitoring in a fast-changing world with Prometheus
Monitoring in a fast-changing world with PrometheusMonitoring in a fast-changing world with Prometheus
Monitoring in a fast-changing world with PrometheusJulien Pivotto
 
5 tips for Prometheus Service Discovery
5 tips for Prometheus Service Discovery5 tips for Prometheus Service Discovery
5 tips for Prometheus Service DiscoveryJulien Pivotto
 
Prometheus and TLS - an Introduction
Prometheus and TLS - an IntroductionPrometheus and TLS - an Introduction
Prometheus and TLS - an IntroductionJulien Pivotto
 
Powerful graphs in Grafana
Powerful graphs in GrafanaPowerful graphs in Grafana
Powerful graphs in GrafanaJulien Pivotto
 
Improved alerting with Prometheus and Alertmanager
Improved alerting with Prometheus and AlertmanagerImproved alerting with Prometheus and Alertmanager
Improved alerting with Prometheus and AlertmanagerJulien Pivotto
 
SIngle Sign On with Keycloak
SIngle Sign On with KeycloakSIngle Sign On with Keycloak
SIngle Sign On with KeycloakJulien Pivotto
 
An introduction to Ansible
An introduction to AnsibleAn introduction to Ansible
An introduction to AnsibleJulien Pivotto
 
Taking advantage of Prometheus relabeling
Taking advantage of Prometheus relabelingTaking advantage of Prometheus relabeling
Taking advantage of Prometheus relabelingJulien Pivotto
 
Grafonnet, grafana dashboards as code
Grafonnet, grafana dashboards as codeGrafonnet, grafana dashboards as code
Grafonnet, grafana dashboards as codeJulien Pivotto
 
Don't be the bottleneck of your open source project!
Don't be the bottleneck of your open source project!Don't be the bottleneck of your open source project!
Don't be the bottleneck of your open source project!Julien Pivotto
 
Monitoring MySQL Replication lag with Prometheus & pt-heartbeat
Monitoring MySQL Replication lag with Prometheus & pt-heartbeatMonitoring MySQL Replication lag with Prometheus & pt-heartbeat
Monitoring MySQL Replication lag with Prometheus & pt-heartbeatJulien Pivotto
 

More from Julien Pivotto (20)

The O11y Toolkit
The O11y ToolkitThe O11y Toolkit
The O11y Toolkit
 
What's New in Prometheus and Its Ecosystem
What's New in Prometheus and Its EcosystemWhat's New in Prometheus and Its Ecosystem
What's New in Prometheus and Its Ecosystem
 
Prometheus: What is is, what is new, what is coming
Prometheus: What is is, what is new, what is comingPrometheus: What is is, what is new, what is coming
Prometheus: What is is, what is new, what is coming
 
What's new in Prometheus?
What's new in Prometheus?What's new in Prometheus?
What's new in Prometheus?
 
Introduction to Grafana Loki
Introduction to Grafana LokiIntroduction to Grafana Loki
Introduction to Grafana Loki
 
Why you should revisit mgmt
Why you should revisit mgmtWhy you should revisit mgmt
Why you should revisit mgmt
 
Observing the HashiCorp Ecosystem From Prometheus
Observing the HashiCorp Ecosystem From PrometheusObserving the HashiCorp Ecosystem From Prometheus
Observing the HashiCorp Ecosystem From Prometheus
 
Monitoring in a fast-changing world with Prometheus
Monitoring in a fast-changing world with PrometheusMonitoring in a fast-changing world with Prometheus
Monitoring in a fast-changing world with Prometheus
 
5 tips for Prometheus Service Discovery
5 tips for Prometheus Service Discovery5 tips for Prometheus Service Discovery
5 tips for Prometheus Service Discovery
 
Prometheus and TLS - an Introduction
Prometheus and TLS - an IntroductionPrometheus and TLS - an Introduction
Prometheus and TLS - an Introduction
 
Powerful graphs in Grafana
Powerful graphs in GrafanaPowerful graphs in Grafana
Powerful graphs in Grafana
 
YAML Magic
YAML MagicYAML Magic
YAML Magic
 
Improved alerting with Prometheus and Alertmanager
Improved alerting with Prometheus and AlertmanagerImproved alerting with Prometheus and Alertmanager
Improved alerting with Prometheus and Alertmanager
 
SIngle Sign On with Keycloak
SIngle Sign On with KeycloakSIngle Sign On with Keycloak
SIngle Sign On with Keycloak
 
An introduction to Ansible
An introduction to AnsibleAn introduction to Ansible
An introduction to Ansible
 
Jsonnet
JsonnetJsonnet
Jsonnet
 
Taking advantage of Prometheus relabeling
Taking advantage of Prometheus relabelingTaking advantage of Prometheus relabeling
Taking advantage of Prometheus relabeling
 
Grafonnet, grafana dashboards as code
Grafonnet, grafana dashboards as codeGrafonnet, grafana dashboards as code
Grafonnet, grafana dashboards as code
 
Don't be the bottleneck of your open source project!
Don't be the bottleneck of your open source project!Don't be the bottleneck of your open source project!
Don't be the bottleneck of your open source project!
 
Monitoring MySQL Replication lag with Prometheus & pt-heartbeat
Monitoring MySQL Replication lag with Prometheus & pt-heartbeatMonitoring MySQL Replication lag with Prometheus & pt-heartbeat
Monitoring MySQL Replication lag with Prometheus & pt-heartbeat
 

Recently uploaded

Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireExakis Nelite
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch TuesdayIvanti
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuidePixlogix Infotech
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPTiSEO AI
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligencePrecisely
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxFIDO Alliance
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 

Recently uploaded (20)

Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 

Incident Resolution as Code