SlideShare a Scribd company logo
From #MonitoringSucks toFrom #MonitoringSucks to
#MonitoringLove#MonitoringLove
Open Source Monitoring in 2018-2019Open Source Monitoring in 2018-2019
@KrisBuytaert
Devops Meetup, Brno
Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● Chief Twitter Ofcer and Open SourceChief Twitter Ofcer and Open Source
Consultant @inuits.euConsultant @inuits.eu
● Everything is an efng DNS ProblemEverything is an efng DNS Problem
● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore
● Organising ConferencesOrganising Conferences
● Evangelizing devopsEvangelizing devops
An opinionated talk about the Open SourceAn opinionated talk about the Open Source
Monitoring tooling landscapeMonitoring tooling landscape
In which I hope to learn from YOUIn which I hope to learn from YOU
#devops=~C(L)AMS#devops=~C(L)AMS
● CultureCulture
● (Lean)(Lean)
● AutomationAutomation
● Monitoring and MeasurementMonitoring and Measurement
● SharingSharing
Damon Edwards and John WillisDamon Edwards and John Willis
Gene KimGene Kim
Monitoring is usually anMonitoring is usually an
aftertoughtaftertought
ENOBUDGET, ENOTIMEENOBUDGET, ENOTIME
AnAn 20082008 OLS PaperOLS Paper
● We have bloated Java toolsWe have bloated Java tools
● Some open Core stufSome open Core stuf
● DYI folks want traditional NagiosDYI folks want traditional Nagios
● DBA RequiredDBA Required
#monitoringsucks#monitoringsucks
● John Vincent (@lusis), june 2011John Vincent (@lusis), june 2011
● A sub #devops movementA sub #devops movement
● https://github.com/monitoringsucks/https://github.com/monitoringsucks/
Why #monitoringsucksWhy #monitoringsucks
● Manual confg (gui)Manual confg (gui)
● Not in sync with realityNot in sync with reality
● Hosts onlyHosts only
● Services sometimesServices sometimes
● Application neverApplication never
● Chaos or out of sync with realityChaos or out of sync with reality
● Alert FatigueAlert Fatigue
#monitoringlove#monitoringlove
•
•
Ulf Mansson #devopsdays Rome 2011Ulf Mansson #devopsdays Rome 2011
•
A new era of toolingA new era of tooling
• #monitoringlove hacksessions @inuits#monitoringlove hacksessions @inuits
•
#monitorama#monitorama
What we wantWhat we want
● Small , well suited componentsSmall , well suited components
•
CollectCollect
•
Transport / MangleTransport / Mangle
•
StoreStore
•
AnalyseAnalyse
•
Act / AlertAct / Alert
•
VisualizeVisualize
Open Source Monitoring in 2019
Open Source Monitoring in 2019
The love was : SensuThe love was : Sensu
● Awesome for non staticAwesome for non static
environmentsenvironments
● Scaling a clustered RabbitMQ ?Scaling a clustered RabbitMQ ?
● Looking more and more likeLooking more and more like
PrometheusPrometheus
● This is Europe, U no do cloudThis is Europe, U no do cloud
Automation ofAutomation of
#monitoring#monitoring
brought backbrought back
thethe #love#love
There is no such thingThere is no such thing
as “Service” discoveryas “Service” discovery
Monitoring aMonitoring a serviceservice
vsvs
Monitoring aMonitoring a ServiceService
AutomationAutomation
defnition of done:defnition of done:
monitored and in productionmonitored and in production
A software project is not doneA software project is not done
until your last end user is deaduntil your last end user is dead
Culture,Culture,
Automation,Automation,
Measurement :Measurement :
measure all the thingsmeasure all the things
SharingSharing
Collection :Collection :
● Collectd,Collectd,
● DiamondDiamond
● From applicationFrom application
● Custom ExportersCustom Exporters
● LogsLogs
● LogsLogs
NetDataNetData
● GranularityGranularity
● Debug vsDebug vs
constant prod ?constant prod ?
●
Transport / Ship / Mangle:Transport / Ship / Mangle:
● Collectd / Diamond / TelegrafCollectd / Diamond / Telegraf
● (r)syslog, Beats, logstash(r)syslog, Beats, logstash
● Q , Nats, ActiveMQ, RabbitMQQ , Nats, ActiveMQ, RabbitMQ
● Collect fromCollect from
anywhereanywhere
● FilterFilter
● Send anywhereSend anywhere
Store :Store :
● TSDB : Time Series DBTSDB : Time Series DB
● Optimized DB for Time SeriesOptimized DB for Time Series
● Graphite/ Infux / OpenTSDB / ....Graphite/ Infux / OpenTSDB / ....
● ElasticElastic
● Long Term vs Short Term StorageLong Term vs Short Term Storage
Oldschool graphiteOldschool graphite
PrometheusPrometheus
● Started 2012Started 2012
● SoundCloudSoundCloud
● Metrics BasedMetrics Based
● ScrapesScrapes
EndpointsEndpoints
•
ExistingExisting
endpoints forendpoints for
limited toolslimited tools
● GraphiteGraphite
ExporterExporter
● Push GatewayPush Gateway
● Great AlertingGreat Alerting
PrometheusPrometheus
● Mostly for Short TermMostly for Short Term
● Still Ship longterm metrics to otherStill Ship longterm metrics to other
TSDBTSDB
● Nginx gw’s all over the placeNginx gw’s all over the place
•
(ssl fun)(ssl fun)
Infnite Diskspace ?Infnite Diskspace ?
● Logstash outputLogstash output
•
Statsd => GraphiteStatsd => Graphite
•
Keep patterns around,Keep patterns around,
•
Selectively purge dataSelectively purge data
● Prometheus for Short TermPrometheus for Short Term
•
Graphite for Long termGraphite for Long term
Log AlternativesLog Alternatives
● Graylog2Graylog2
● ELSA (Enterprise Log Search andELSA (Enterprise Log Search and
Archive)Archive)
● ELK StackELK Stack
● FluentdFluentd
Prometheus ?Prometheus ?
● Only For Containers ?Only For Containers ?
● Also for other setups !Also for other setups !
● Is this sufcient ?Is this sufcient ?
Act / Alert:Act / Alert:
Checking for FailureChecking for Failure
● IcingaIcinga
•
Automated confg generationAutomated confg generation
● SensuSensu
•
CloudstyleCloudstyle
● PrometheusPrometheus
•
AlertManagerAlertManager
Waking you up at nightWaking you up at night
● FlapjackFlapjack
fapjack.iofapjack.io
monitoring notifcation routing +monitoring notifcation routing +
event processing systemevent processing system
● OpenDutyOpenDuty
github.com/szechuen/OpenDutygithub.com/szechuen/OpenDuty
Duty managementDuty management
Waking you up at nightWaking you up at night
● AnagAnag
● Custom written stufCustom written stuf
Analyse:Analyse:
Basic SearchBasic Search
Graphs to KnowledgeGraphs to Knowledge
SkylineSkyline
•
OculusOculus
•
Creating Information out of this dataCreating Information out of this data
•
Big dataBig data
•
Machine LearningMachine Learning
•
Hastic.ioHastic.io
Hastic.ioHastic.io
● Open Source Pattern DetectionOpen Source Pattern Detection
● Label patterns → Wait for learning toLabel patterns → Wait for learning to
complete → Get detectionscomplete → Get detections
● Hastic Server + Grafana AppHastic Server + Grafana App
LogIslandsLogIslands
● Complex event processing &Complex event processing &
patterns mining at scalepatterns mining at scale
● Kafka, Nif, Spark, HadoopKafka, Nif, Spark, Hadoop
Visualize:Visualize:
KibanaKibana
GrafanaGrafana
ChallengeChallenge
● *ana as code*ana as code
● Template your ...Template your ...
● e.g grafonnet-libe.g grafonnet-lib
•
A jsonnet lib to generate GrafanaA jsonnet lib to generate Grafana
dashboards ...dashboards ...
AggregatingAggregating
● ThrukThruk
● GrafanaGrafana
● DashingDashing
Lack of change ?Lack of change ?
● Limited # new toolsLimited # new tools
● Feature Complete ?Feature Complete ?
Is prometheus the newIs prometheus the new
Docker ?Docker ?
APMAPM
Application Performance MonitoringApplication Performance Monitoring
But what about my apps ?But what about my apps ?
● agent required that ties to codeagent required that ties to code
● Code modifcationsCode modifcations
Old PacketBeatOld PacketBeat
Open Source “APM”Open Source “APM”
● Scouter
● Jaeger
● Kamon
● Zipkin
● Beats ...
● Performance Co Pilot
● Kamon
● Pinpoint
● Micrometer
● StageMonitor
● SkyWalking
● Kieker
=> Huge focus on the Java Ecosystem , little
options for PHP/ Python / Ruby shops.
OpenAPM.ioOpenAPM.io
OpenTracing 101OpenTracing 101
● The problem : It was not reasonable to ask all OSS services and all OSSThe problem : It was not reasonable to ask all OSS services and all OSS
packages and all application-specifc code to use a single tracingpackages and all application-specifc code to use a single tracing
vendor => Open Ttracingvendor => Open Ttracing
● Distributed Tracing StandardDistributed Tracing Standard
● CNCFCNCF
● Dapper inside GoogleDapper inside Google
● ““OpenTracing is not a download or a program. Distributed tracingOpenTracing is not a download or a program. Distributed tracing
requires that software developers add instrumentation to the code ofrequires that software developers add instrumentation to the code of
an application, or to the frameworks used in the application”an application, or to the frameworks used in the application”
Complexity is the EnemyComplexity is the Enemy
of Reliabilityof Reliability
I love where Monitoring is headingI love where Monitoring is heading
““Wait , was I oncall last week ?”Wait , was I oncall last week ?”
True words said by one of our oncall engineersTrue words said by one of our oncall engineers
OpservabilityOpservability
ContactContact
Kris Buytaert kris.buytaert@inuits.euKris Buytaert kris.buytaert@inuits.eu
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.eu/http://www.inuits.eu/
Find Inuits inFind Inuits in
Brasschaat,Ghent,Brasschaat,Ghent,
Rotterdam,Prague,Rotterdam,Prague,
Kiev,BrnoKiev,Brno

More Related Content

What's hot

Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
Kris Buytaert
 
From MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 EditionFrom MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 Edition
Kris Buytaert
 
Pipeline as Code
Pipeline as CodePipeline as Code
Pipeline as Code
Kris Buytaert
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
Kris Buytaert
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
Kris Buytaert
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
Kris Buytaert
 
Nightmare on Docker street
Nightmare on Docker streetNightmare on Docker street
Nightmare on Docker street
Kris Buytaert
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
Kris Buytaert
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
Kris Buytaert
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops efforts
Kris Buytaert
 
Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?
Kris Buytaert
 
Devops is dead, Long Live Devops
Devops is dead, Long Live DevopsDevops is dead, Long Live Devops
Devops is dead, Long Live Devops
Kris Buytaert
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with Puppet
Kris Buytaert
 
From devoops to devops
From devoops to devopsFrom devoops to devops
From devoops to devops
Kris Buytaert
 
Groovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineGroovy there's a docker in my application pipeline
Groovy there's a docker in my application pipeline
Kris Buytaert
 
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMSDevopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Kris Buytaert
 
The Return of the Dull Stack Engineer
The Return of the Dull Stack EngineerThe Return of the Dull Stack Engineer
The Return of the Dull Stack Engineer
Kris Buytaert
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
Kris Buytaert
 
No, we can't do continuous delivery
No, we can't do continuous deliveryNo, we can't do continuous delivery
No, we can't do continuous delivery
Kris Buytaert
 
On the Importance of Infrastructure as Code
On the Importance of Infrastructure as CodeOn the Importance of Infrastructure as Code
On the Importance of Infrastructure as Code
Kris Buytaert
 

What's hot (20)

Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
 
From MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 EditionFrom MonitoringSucks to Monitoring Love , 2016 Edition
From MonitoringSucks to Monitoring Love , 2016 Edition
 
Pipeline as Code
Pipeline as CodePipeline as Code
Pipeline as Code
 
Is there a future for devops ?
Is there a future for devops ?Is there a future for devops ?
Is there a future for devops ?
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
 
Nightmare on Docker street
Nightmare on Docker streetNightmare on Docker street
Nightmare on Docker street
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
 
Pipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as CodePipeline as code for your infrastructure as Code
Pipeline as code for your infrastructure as Code
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops efforts
 
Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?Dev secops opsec, devsec, devops ?
Dev secops opsec, devsec, devops ?
 
Devops is dead, Long Live Devops
Devops is dead, Long Live DevopsDevops is dead, Long Live Devops
Devops is dead, Long Live Devops
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with Puppet
 
From devoops to devops
From devoops to devopsFrom devoops to devops
From devoops to devops
 
Groovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineGroovy there's a docker in my application pipeline
Groovy there's a docker in my application pipeline
 
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMSDevopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
Devopsdays Amsterdam 2017 Keynote, looking back at 5 years of AMS
 
The Return of the Dull Stack Engineer
The Return of the Dull Stack EngineerThe Return of the Dull Stack Engineer
The Return of the Dull Stack Engineer
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
 
No, we can't do continuous delivery
No, we can't do continuous deliveryNo, we can't do continuous delivery
No, we can't do continuous delivery
 
On the Importance of Infrastructure as Code
On the Importance of Infrastructure as CodeOn the Importance of Infrastructure as Code
On the Importance of Infrastructure as Code
 

Similar to Open Source Monitoring in 2019

Open Source Monitoring in 2015
Open Source Monitoring in 2015Open Source Monitoring in 2015
Open Source Monitoring in 2015
Kris Buytaert
 
Another 7 tools for your #devops stack
Another 7 tools for your #devops stackAnother 7 tools for your #devops stack
Another 7 tools for your #devops stack
Kris Buytaert
 
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris BuytaertOSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
NETWAYS
 
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code AgeMonitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Puppet
 
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertOSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
NETWAYS
 
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code AgeMonitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Kris Buytaert
 
Continous Delivery of your Infrastructure
Continous Delivery of your InfrastructureContinous Delivery of your Infrastructure
Continous Delivery of your Infrastructure
Kris Buytaert
 
Monitoring Drupal In an Infrastructure as Code Age
Monitoring Drupal In an Infrastructure as Code AgeMonitoring Drupal In an Infrastructure as Code Age
Monitoring Drupal In an Infrastructure as Code Age
Kris Buytaert
 
Run stuff, Deploy Stuff, Jax London 2017 Edition
Run stuff, Deploy Stuff, Jax London 2017 EditionRun stuff, Deploy Stuff, Jax London 2017 Edition
Run stuff, Deploy Stuff, Jax London 2017 Edition
Kris Buytaert
 
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertOSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
NETWAYS
 
When traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needsWhen traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needs
Kris Buytaert
 
Continuous Delivery of (y)our infrastructure.
Continuous Delivery of (y)our infrastructure.Continuous Delivery of (y)our infrastructure.
Continuous Delivery of (y)our infrastructure.
Kris Buytaert
 
From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove
Kris Buytaert
 
Run stuff, Deploy Stuff
Run stuff, Deploy StuffRun stuff, Deploy Stuff
Run stuff, Deploy Stuff
Kris Buytaert
 
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLoveOSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
NETWAYS
 
Icinga Camp Amsterdam - Infrastructure as Code
Icinga Camp Amsterdam - Infrastructure as CodeIcinga Camp Amsterdam - Infrastructure as Code
Icinga Camp Amsterdam - Infrastructure as Code
Icinga
 
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
NETWAYS
 
The devops approach to monitoring, Open Source and Infrastructure as Code Style
The devops approach to monitoring, Open Source and Infrastructure as Code StyleThe devops approach to monitoring, Open Source and Infrastructure as Code Style
The devops approach to monitoring, Open Source and Infrastructure as Code Style
Julien Pivotto
 
OSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris BuyaertOSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris Buyaert
NETWAYS
 
OSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris BuytaertOSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris Buytaert
NETWAYS
 

Similar to Open Source Monitoring in 2019 (20)

Open Source Monitoring in 2015
Open Source Monitoring in 2015Open Source Monitoring in 2015
Open Source Monitoring in 2015
 
Another 7 tools for your #devops stack
Another 7 tools for your #devops stackAnother 7 tools for your #devops stack
Another 7 tools for your #devops stack
 
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris BuytaertOSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
OSDC 2016 - Another 7 Tools for your #devops Stack by Kris Buytaert
 
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code AgeMonitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
 
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris BuytaertOSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
OSMC 2014: From monitoringsucks to monitoringlove (and back) | Kris Buytaert
 
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code AgeMonitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
 
Continous Delivery of your Infrastructure
Continous Delivery of your InfrastructureContinous Delivery of your Infrastructure
Continous Delivery of your Infrastructure
 
Monitoring Drupal In an Infrastructure as Code Age
Monitoring Drupal In an Infrastructure as Code AgeMonitoring Drupal In an Infrastructure as Code Age
Monitoring Drupal In an Infrastructure as Code Age
 
Run stuff, Deploy Stuff, Jax London 2017 Edition
Run stuff, Deploy Stuff, Jax London 2017 EditionRun stuff, Deploy Stuff, Jax London 2017 Edition
Run stuff, Deploy Stuff, Jax London 2017 Edition
 
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris BuytaertOSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
OSMC 2014 | From monitoringsucks to monitoringlove, and back by Kris Buytaert
 
When traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needsWhen traditional configuration management is to slow for your needs
When traditional configuration management is to slow for your needs
 
Continuous Delivery of (y)our infrastructure.
Continuous Delivery of (y)our infrastructure.Continuous Delivery of (y)our infrastructure.
Continuous Delivery of (y)our infrastructure.
 
From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove From Config Management Sucks to #cfgmgmtlove
From Config Management Sucks to #cfgmgmtlove
 
Run stuff, Deploy Stuff
Run stuff, Deploy StuffRun stuff, Deploy Stuff
Run stuff, Deploy Stuff
 
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLoveOSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLove
 
Icinga Camp Amsterdam - Infrastructure as Code
Icinga Camp Amsterdam - Infrastructure as CodeIcinga Camp Amsterdam - Infrastructure as Code
Icinga Camp Amsterdam - Infrastructure as Code
 
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
 
The devops approach to monitoring, Open Source and Infrastructure as Code Style
The devops approach to monitoring, Open Source and Infrastructure as Code StyleThe devops approach to monitoring, Open Source and Infrastructure as Code Style
The devops approach to monitoring, Open Source and Infrastructure as Code Style
 
OSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris BuyaertOSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris Buyaert
 
OSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris BuytaertOSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris Buytaert
 

More from Kris Buytaert

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
Kris Buytaert
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
Kris Buytaert
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
Kris Buytaert
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
Kris Buytaert
 
Pipeline all the Dashboards as Code
Pipeline all the Dashboards as CodePipeline all the Dashboards as Code
Pipeline all the Dashboards as Code
Kris Buytaert
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
Kris Buytaert
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
Kris Buytaert
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
Kris Buytaert
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
Kris Buytaert
 
Looking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDXLooking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDX
Kris Buytaert
 

More from Kris Buytaert (10)

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
 
Pipeline all the Dashboards as Code
Pipeline all the Dashboards as CodePipeline all the Dashboards as Code
Pipeline all the Dashboards as Code
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
 
Looking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcampLooking back at 5 years of #cfgmgmtcamp
Looking back at 5 years of #cfgmgmtcamp
 
Looking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDXLooking back at 7.5 years of Devopsdays , DOd PDX
Looking back at 7.5 years of Devopsdays , DOd PDX
 

Recently uploaded

Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
Emerging Tech
 
Empowering Businesses in the Digital Age
Empowering Businesses in the Digital AgeEmpowering Businesses in the Digital Age
Empowering Businesses in the Digital Age
Bert Blevins
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Chris Swan
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
Andrey Yasko
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 
Password Rotation in 2024 is still Relevant
Password Rotation in 2024 is still RelevantPassword Rotation in 2024 is still Relevant
Password Rotation in 2024 is still Relevant
Bert Blevins
 
The Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdfThe Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdf
paysquare consultancy
 
Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...
chetankumar9855
 
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
Toru Tamaki
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Mydbops
 
Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
CEPTES Software Inc
 
Advanced Techniques for Cyber Security Analysis and Anomaly Detection
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionAdvanced Techniques for Cyber Security Analysis and Anomaly Detection
Advanced Techniques for Cyber Security Analysis and Anomaly Detection
Bert Blevins
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdfARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
Inglês no Mundo Digital
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Bert Blevins
 
ScrumGathering New Orleans 2024 Catherine Louis.pdf
ScrumGathering New Orleans 2024  Catherine Louis.pdfScrumGathering New Orleans 2024  Catherine Louis.pdf
ScrumGathering New Orleans 2024 Catherine Louis.pdf
Global Agile Consulting- CLL-Group, LLC
 
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdfWhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
ArgaBisma
 

Recently uploaded (20)

Implementations of Fused Deposition Modeling in real world
Implementations of Fused Deposition Modeling  in real worldImplementations of Fused Deposition Modeling  in real world
Implementations of Fused Deposition Modeling in real world
 
Empowering Businesses in the Digital Age
Empowering Businesses in the Digital AgeEmpowering Businesses in the Digital Age
Empowering Businesses in the Digital Age
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...
 
Comparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdfComparison Table of DiskWarrior Alternatives.pdf
Comparison Table of DiskWarrior Alternatives.pdf
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 
Password Rotation in 2024 is still Relevant
Password Rotation in 2024 is still RelevantPassword Rotation in 2024 is still Relevant
Password Rotation in 2024 is still Relevant
 
The Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdfThe Role of Technology in Payroll Statutory Compliance (1).pdf
The Role of Technology in Payroll Statutory Compliance (1).pdf
 
Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...Amul milk launches in US: Key details of its new products ...
Amul milk launches in US: Key details of its new products ...
 
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...
 
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsScaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - Mydbops
 
Salesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot WorkshopSalesforce AI & Einstein Copilot Workshop
Salesforce AI & Einstein Copilot Workshop
 
Advanced Techniques for Cyber Security Analysis and Anomaly Detection
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionAdvanced Techniques for Cyber Security Analysis and Anomaly Detection
Advanced Techniques for Cyber Security Analysis and Anomaly Detection
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdfARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
ARTIFICIAL INTELLIGENCE (AI) IN MUSIC.pdf
 
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...
 
ScrumGathering New Orleans 2024 Catherine Louis.pdf
ScrumGathering New Orleans 2024  Catherine Louis.pdfScrumGathering New Orleans 2024  Catherine Louis.pdf
ScrumGathering New Orleans 2024 Catherine Louis.pdf
 
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdfWhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
WhatsApp Image 2024-03-27 at 08.19.52_bfd93109.pdf
 

Open Source Monitoring in 2019

  • 1. From #MonitoringSucks toFrom #MonitoringSucks to #MonitoringLove#MonitoringLove Open Source Monitoring in 2018-2019Open Source Monitoring in 2018-2019 @KrisBuytaert Devops Meetup, Brno
  • 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Twitter Ofcer and Open SourceChief Twitter Ofcer and Open Source Consultant @inuits.euConsultant @inuits.eu ● Everything is an efng DNS ProblemEverything is an efng DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Organising ConferencesOrganising Conferences ● Evangelizing devopsEvangelizing devops
  • 3. An opinionated talk about the Open SourceAn opinionated talk about the Open Source Monitoring tooling landscapeMonitoring tooling landscape In which I hope to learn from YOUIn which I hope to learn from YOU
  • 4. #devops=~C(L)AMS#devops=~C(L)AMS ● CultureCulture ● (Lean)(Lean) ● AutomationAutomation ● Monitoring and MeasurementMonitoring and Measurement ● SharingSharing Damon Edwards and John WillisDamon Edwards and John Willis Gene KimGene Kim
  • 5. Monitoring is usually anMonitoring is usually an aftertoughtaftertought ENOBUDGET, ENOTIMEENOBUDGET, ENOTIME
  • 6. AnAn 20082008 OLS PaperOLS Paper ● We have bloated Java toolsWe have bloated Java tools ● Some open Core stufSome open Core stuf ● DYI folks want traditional NagiosDYI folks want traditional Nagios ● DBA RequiredDBA Required
  • 7. #monitoringsucks#monitoringsucks ● John Vincent (@lusis), june 2011John Vincent (@lusis), june 2011 ● A sub #devops movementA sub #devops movement ● https://github.com/monitoringsucks/https://github.com/monitoringsucks/
  • 8. Why #monitoringsucksWhy #monitoringsucks ● Manual confg (gui)Manual confg (gui) ● Not in sync with realityNot in sync with reality ● Hosts onlyHosts only ● Services sometimesServices sometimes ● Application neverApplication never ● Chaos or out of sync with realityChaos or out of sync with reality ● Alert FatigueAlert Fatigue
  • 9. #monitoringlove#monitoringlove • • Ulf Mansson #devopsdays Rome 2011Ulf Mansson #devopsdays Rome 2011 • A new era of toolingA new era of tooling • #monitoringlove hacksessions @inuits#monitoringlove hacksessions @inuits • #monitorama#monitorama
  • 10. What we wantWhat we want ● Small , well suited componentsSmall , well suited components • CollectCollect • Transport / MangleTransport / Mangle • StoreStore • AnalyseAnalyse • Act / AlertAct / Alert • VisualizeVisualize
  • 13. The love was : SensuThe love was : Sensu ● Awesome for non staticAwesome for non static environmentsenvironments ● Scaling a clustered RabbitMQ ?Scaling a clustered RabbitMQ ? ● Looking more and more likeLooking more and more like PrometheusPrometheus ● This is Europe, U no do cloudThis is Europe, U no do cloud
  • 14. Automation ofAutomation of #monitoring#monitoring brought backbrought back thethe #love#love
  • 15. There is no such thingThere is no such thing as “Service” discoveryas “Service” discovery
  • 16. Monitoring aMonitoring a serviceservice vsvs Monitoring aMonitoring a ServiceService
  • 18. defnition of done:defnition of done: monitored and in productionmonitored and in production
  • 19. A software project is not doneA software project is not done until your last end user is deaduntil your last end user is dead
  • 20. Culture,Culture, Automation,Automation, Measurement :Measurement : measure all the thingsmeasure all the things SharingSharing
  • 21. Collection :Collection : ● Collectd,Collectd, ● DiamondDiamond ● From applicationFrom application ● Custom ExportersCustom Exporters ● LogsLogs ● LogsLogs
  • 22. NetDataNetData ● GranularityGranularity ● Debug vsDebug vs constant prod ?constant prod ? ●
  • 23. Transport / Ship / Mangle:Transport / Ship / Mangle: ● Collectd / Diamond / TelegrafCollectd / Diamond / Telegraf ● (r)syslog, Beats, logstash(r)syslog, Beats, logstash ● Q , Nats, ActiveMQ, RabbitMQQ , Nats, ActiveMQ, RabbitMQ
  • 24. ● Collect fromCollect from anywhereanywhere ● FilterFilter ● Send anywhereSend anywhere
  • 25. Store :Store : ● TSDB : Time Series DBTSDB : Time Series DB ● Optimized DB for Time SeriesOptimized DB for Time Series ● Graphite/ Infux / OpenTSDB / ....Graphite/ Infux / OpenTSDB / .... ● ElasticElastic ● Long Term vs Short Term StorageLong Term vs Short Term Storage
  • 27. PrometheusPrometheus ● Started 2012Started 2012 ● SoundCloudSoundCloud ● Metrics BasedMetrics Based ● ScrapesScrapes EndpointsEndpoints • ExistingExisting endpoints forendpoints for limited toolslimited tools ● GraphiteGraphite ExporterExporter ● Push GatewayPush Gateway ● Great AlertingGreat Alerting
  • 28. PrometheusPrometheus ● Mostly for Short TermMostly for Short Term ● Still Ship longterm metrics to otherStill Ship longterm metrics to other TSDBTSDB ● Nginx gw’s all over the placeNginx gw’s all over the place • (ssl fun)(ssl fun)
  • 29. Infnite Diskspace ?Infnite Diskspace ? ● Logstash outputLogstash output • Statsd => GraphiteStatsd => Graphite • Keep patterns around,Keep patterns around, • Selectively purge dataSelectively purge data ● Prometheus for Short TermPrometheus for Short Term • Graphite for Long termGraphite for Long term
  • 30. Log AlternativesLog Alternatives ● Graylog2Graylog2 ● ELSA (Enterprise Log Search andELSA (Enterprise Log Search and Archive)Archive) ● ELK StackELK Stack ● FluentdFluentd
  • 31. Prometheus ?Prometheus ? ● Only For Containers ?Only For Containers ? ● Also for other setups !Also for other setups ! ● Is this sufcient ?Is this sufcient ?
  • 32. Act / Alert:Act / Alert:
  • 33. Checking for FailureChecking for Failure ● IcingaIcinga • Automated confg generationAutomated confg generation ● SensuSensu • CloudstyleCloudstyle ● PrometheusPrometheus • AlertManagerAlertManager
  • 34. Waking you up at nightWaking you up at night ● FlapjackFlapjack fapjack.iofapjack.io monitoring notifcation routing +monitoring notifcation routing + event processing systemevent processing system ● OpenDutyOpenDuty github.com/szechuen/OpenDutygithub.com/szechuen/OpenDuty Duty managementDuty management
  • 35. Waking you up at nightWaking you up at night ● AnagAnag ● Custom written stufCustom written stuf
  • 38. Graphs to KnowledgeGraphs to Knowledge SkylineSkyline • OculusOculus • Creating Information out of this dataCreating Information out of this data • Big dataBig data • Machine LearningMachine Learning • Hastic.ioHastic.io
  • 39. Hastic.ioHastic.io ● Open Source Pattern DetectionOpen Source Pattern Detection ● Label patterns → Wait for learning toLabel patterns → Wait for learning to complete → Get detectionscomplete → Get detections ● Hastic Server + Grafana AppHastic Server + Grafana App
  • 40. LogIslandsLogIslands ● Complex event processing &Complex event processing & patterns mining at scalepatterns mining at scale ● Kafka, Nif, Spark, HadoopKafka, Nif, Spark, Hadoop
  • 44. ChallengeChallenge ● *ana as code*ana as code ● Template your ...Template your ... ● e.g grafonnet-libe.g grafonnet-lib • A jsonnet lib to generate GrafanaA jsonnet lib to generate Grafana dashboards ...dashboards ...
  • 46. Lack of change ?Lack of change ? ● Limited # new toolsLimited # new tools ● Feature Complete ?Feature Complete ?
  • 47. Is prometheus the newIs prometheus the new Docker ?Docker ?
  • 48. APMAPM Application Performance MonitoringApplication Performance Monitoring But what about my apps ?But what about my apps ? ● agent required that ties to codeagent required that ties to code ● Code modifcationsCode modifcations
  • 50. Open Source “APM”Open Source “APM” ● Scouter ● Jaeger ● Kamon ● Zipkin ● Beats ... ● Performance Co Pilot ● Kamon ● Pinpoint ● Micrometer ● StageMonitor ● SkyWalking ● Kieker => Huge focus on the Java Ecosystem , little options for PHP/ Python / Ruby shops.
  • 52. OpenTracing 101OpenTracing 101 ● The problem : It was not reasonable to ask all OSS services and all OSSThe problem : It was not reasonable to ask all OSS services and all OSS packages and all application-specifc code to use a single tracingpackages and all application-specifc code to use a single tracing vendor => Open Ttracingvendor => Open Ttracing ● Distributed Tracing StandardDistributed Tracing Standard ● CNCFCNCF ● Dapper inside GoogleDapper inside Google ● ““OpenTracing is not a download or a program. Distributed tracingOpenTracing is not a download or a program. Distributed tracing requires that software developers add instrumentation to the code ofrequires that software developers add instrumentation to the code of an application, or to the frameworks used in the application”an application, or to the frameworks used in the application”
  • 53. Complexity is the EnemyComplexity is the Enemy of Reliabilityof Reliability
  • 54. I love where Monitoring is headingI love where Monitoring is heading ““Wait , was I oncall last week ?”Wait , was I oncall last week ?” True words said by one of our oncall engineersTrue words said by one of our oncall engineers
  • 56. ContactContact Kris Buytaert kris.buytaert@inuits.euKris Buytaert kris.buytaert@inuits.eu Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.eu/http://www.inuits.eu/ Find Inuits inFind Inuits in Brasschaat,Ghent,Brasschaat,Ghent, Rotterdam,Prague,Rotterdam,Prague, Kiev,BrnoKiev,Brno