Your SlideShare is downloading. ×
0
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Monitoring in an Infrastructure as Code Age
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Monitoring in an Infrastructure as Code Age

5,167

Published on

Monitoring an infrastructure is still one of the most complex tasks at hand. Yet in this age of Infrastructure as Code a lot of people are still stuck using tools that haven't adapted. This talk will …

Monitoring an infrastructure is still one of the most complex tasks at hand. Yet in this age of Infrastructure as Code a lot of people are still stuck using tools that haven't adapted. This talk will explain you how to look at monitoring your infrastructure from another angle, where you can benefit from using Puppet to deploy and provision your monitoring platforms. This talk will explain you which monitoring tools are suitable for monitoring at scale and Puppet driven provisioning and which ones will only make your life harder than it should be. Where do tools like Icinga, Sensu, Graphite or even CheckMK fit into the picture ?

Kris Buytaert
Chief Travel Officer, Inuits.eu
Kris Buytaert is a long time Linux and Open Source Consultant. He's one of instigators of the devops movement, currently working for Inuits He is frequently speaking at, or organizing different international conferences and has written about the same subjects in different Books, Papers and Articles He spends most of his time working on bridging the gap between developers and operations with a strong focus on High Availability, Scalability , Virtualisation and Large Infrastructure Management projects hence trying to build infrastructures that can survive the 10th floor test, better known today as the cloud while actively promoting the devops idea ! His blog titled "Everything is a Freaking DNS Problem" can be found at http://www.krisbuytaert.be/blog/

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,167
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
77
Comments
0
Likes
10
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Monitoring in an IAC AgeMonitoring in an IAC Age PuppetConf 2013 Kris Buytaert
  • 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● Chief Trolling Officer and Open SourceChief Trolling Officer and Open Source Consultant @inuits.euConsultant @inuits.eu ● Everything is an effing DNS ProblemEverything is an effing DNS Problem ● Building Clouds since before the bookstoreBuilding Clouds since before the bookstore ● Some books, some papers, some blogsSome books, some papers, some blogs ● Evangelizing devopsEvangelizing devops
  • 3. devops = clamsdevops = clams ● CultureCulture ● (Lean)(Lean) ● Automate all the things ...Automate all the things ... • Build AutomationBuild Automation • Test AutomationTest Automation • IACIAC ● Monitoring , Metrics ...Monitoring , Metrics ... ● SharingSharing
  • 4. Monitoring is usually anMonitoring is usually an aftertoughtaftertought ENOBUDGET, ENOTIMEENOBUDGET, ENOTIME
  • 5. #monitoringsucks#monitoringsucks ● John Vincent (@lusis)John Vincent (@lusis) ● A sub movementA sub movement ● https://github.com/monitoringsucks/https://github.com/monitoringsucks/
  • 6. #monitoringlove#monitoringlove • #monitoringlove hacksessions#monitoringlove hacksessions • #monitorama#monitorama
  • 7. Infrastructure as CodeInfrastructure as Code ● Model our infrastructureModel our infrastructure ● A fast reproducable platformA fast reproducable platform ● Disaster discovery for “free”Disaster discovery for “free”
  • 8. For years we've tolerated humans to to makeFor years we've tolerated humans to to make structural manual changes to the infrastructurestructural manual changes to the infrastructure our critical applications are running on.our critical applications are running on. Whilst at the same time demanding those criticalWhilst at the same time demanding those critical applications to go trough rigid test scenarios.applications to go trough rigid test scenarios. Who let this happen ?Who let this happen ?
  • 9. Infrastructure as CodeInfrastructure as Code ● Code = CodeCode = Code ● Version ControlVersion Control ● Quality ChecksQuality Checks ● TestingTesting ● Continuous IntegrationContinuous Integration ● Continous DeliveryContinous Delivery
  • 10. Infrastructure as CodeInfrastructure as Code ● Core InfrastructureCore Infrastructure ● Middleware deployment andMiddleware deployment and integrationintegration ● Automated continuous applicationAutomated continuous application deploymentdeployment ● Integrated Security enforcementIntegrated Security enforcement ● Host, Service and ApplicationHost, Service and Application Monitoring configuredMonitoring configured
  • 11. Where to monitor ?Where to monitor ? ● DevDev ● AcceptanceAcceptance ● ProdProd
  • 12. Why #monitoringsucksWhy #monitoringsucks ● Manual config (gui)Manual config (gui) ● Not in sync with realityNot in sync with reality ● Hosts onlyHosts only ● Services sometimesServices sometimes ● Appliccation neverAppliccation never ● ChaosChaos
  • 13. Let's forget aboutLet's forget about ● Tools with no (stable) APITools with no (stable) API ● Tools with strong focus on GUITools with strong focus on GUI ● Unless you are an SME with < 100 nodesUnless you are an SME with < 100 nodes ● Zabixx, Zenoss, Hyperic, GroundWork, ....Zabixx, Zenoss, Hyperic, GroundWork, ....
  • 14. What we wantWhat we want ● Small , wel suited componentsSmall , wel suited components • CollectCollect • Transport / MangleTransport / Mangle • Analyse / ActAnalyse / Act • VisualizeVisualize
  • 15. Monitoring BaselineMonitoring Baseline ● Deploy a host,Deploy a host, ● Add it to the monitoringAdd it to the monitoring ● Add collection toolsAdd collection tools ● Add check definitionsAdd check definitions ● Update the monitoring tool configUpdate the monitoring tool config
  • 16. Configuring a ServiceConfiguring a Service ● PackagePackage ● ConfigConfig ● ServiceService <- we pray status is correct<- we pray status is correct
  • 17. Apache Example:Apache Example:
  • 18. Icinga ?Icinga ? • Isn't nagios dead ?Isn't nagios dead ? • Vibrant CommunityVibrant Community • Throw great parties in NurnbergThrow great parties in Nurnberg • Nobody can pronounce it anyhowNobody can pronounce it anyhow • https://github.com/Inuits/puppet-icinga/https://github.com/Inuits/puppet-icinga/
  • 19. Stored ConfigsStored Configs
  • 20. Collection and ExportCollection and Export Export :Export : @@resource {@@resource { ... }... } Collect:Collect: Resource <<|Resource <<| query |>>query |>> Clean out nodes that dissapearClean out nodes that dissapear puppet node cleanpuppet node clean
  • 21. Exporting and CollectingExporting and Collecting
  • 22. Monitoring a VhostMonitoring a Vhost
  • 23. ● AutodetectionAutodetection ● MultiplexingMultiplexing ● Trend ForecastingTrend Forecasting I love CheckMKI love CheckMK
  • 24. • Autodetection ?Autodetection ? • Service,Service, • FunctionalitiesFunctionalities • eg. vhosts etceg. vhosts etc • Single Source of TruthSingle Source of Truth I hate CheckMKI hate CheckMK
  • 25. Monitoring a service vs Monitoring a serviceMonitoring a service vs Monitoring a service
  • 26. Definition of Done:Definition of Done: monitored and in productionmonitored and in production
  • 27. A software project is not doneA software project is not done untill your last end user is deaduntill your last end user is dead
  • 28. Exit DODExit DOD Measure Application UsageMeasure Application Usage
  • 29. But , err how do I ?But , err how do I ?
  • 30. Culture,Culture, Automation,Automation, Measurement :Measurement : measure all the thingsmeasure all the things SharingSharing
  • 31. Deploy StatisticsDeploy Statistics ● Time To DeployTime To Deploy ● DeployDeploy FrequencyFrequency ● LifecycleLifecycle frequencyfrequency ● Map toMap to
  • 32. Application MetricsApplication Metrics ● Number of current usersNumber of current users ● Number of sign upsNumber of sign ups ● Response timesResponse times ● TroughputTroughput ● XYZ UsageXYZ Usage ● # restarts# restarts ● Insert your specific valuable stuffInsert your specific valuable stuff
  • 33. Graphite APIGraphite API
  • 34. Triggers on GraphsTriggers on Graphs ● Export Java MetricsExport Java Metrics ● JMXTransJMXTrans ● Export JMXConfigsExport JMXConfigs ● Configure NRPEConfigure NRPE CheckCheck ● Export NagiosCheckExport NagiosCheck ● Collect JMX ExportsCollect JMX Exports on JMXTransNodeon JMXTransNode ● Graph EmGraph Em Collect NagiosCollect Nagios Configs on NagiosConfigs on Nagios ServerServer
  • 35. Triggers on GraphsTriggers on Graphs
  • 36. Triggers on GraphsTriggers on Graphs
  • 37. Self ServiceSelf Service Gdash based pipelinesGdash based pipelines Puppetized Templates (wip)Puppetized Templates (wip)
  • 38. Up Next:Up Next: • Creating Information out of this dataCreating Information out of this data • Big dataBig data • Machine LearningMachine Learning
  • 39. HomeworkHomework SkylineSkyline OculusOculus DuskDusk RiemannRiemann EsperEsper Puppetdb externalPuppetdb external NaginatorNaginator
  • 40. ContactContact Kris.Buytaert@inuits.euKris.Buytaert@inuits.eu Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.eu/http://www.inuits.eu/ InuitsInuits Duboistraat 50Duboistraat 50 2060 Antwerpen2060 Antwerpen BelgiumBelgium 891.514.231891.514.231 +32 475 961221+32 475 961221

×