Successfully reported this slideshow.

Monitoring in the cloud with Puppet

13

Share

Upcoming SlideShare
Future of Sysadmin 2014
Future of Sysadmin 2014
Loading in …3
×
1 of 48
1 of 48

More Related Content

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Monitoring in the cloud with Puppet

  1. Monitoring in the Cloud with Puppet PuppetCamp Ghent 2013 Kris Buytaert
  2. Kris Buytaert ● I used to be a Dev, ● Then Became an Op ● Chief Trolling Officer and Open Source Consultant @inuits.eu ● Everything is an effing DNS Problem ● Building Clouds since before the bookstore ● Some books, some papers, some blogs ● Evangelizing devops
  3. Do you know what your children do at 5 am in the morning ? ● Sleeping ? ● Crashing a Party ? ● Why are the cops at your front door ? ● What happened ? ● How long have they been gone ?
  4. devops ● Culture ● (Lean) ● Automation ● Measurement ● Sharing Damon Edwards and John Willis Gene Kim
  5. Monitoring is usually an aftertought ENOBUDGET, ENOTIME
  6. A sub movement ● #monitoringsucks trending ● https://github.com/monitoringsucks/ ● 2008 Study :Nagios + Friends ● 2011 Conclusion : Nagios/Icinga are the only automatable alternatives
  7. #monitoringsucks Monitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love: ● Having my hands tied with the model of host and service bindings. ● Having to set up "fake" hosts just to group arbitrary metrics together ● Having to either collect metrics twice - once for alerting and another for trending ● Only being able to see my metrics in 5 minute intervals ● Having to chose between shitty interface but great monitoring or shitty monitoring but great interface ● Dealing with a monitoring system that thinks IT is the system of truth for my environment ● Not actually having any real choices John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why- monitoring-sucks.html
  8. Let's forget about ● GroundWork ● Cacti ● Zenoss ● Munin ● Zabbix ● BigBrother ● Hyperic
  9. What's different in the cloud ? ● Scale ● Velocity ● Change
  10. Challenges ● Reproducability ● Speed ● Auditing ● Keeping stuff in sync
  11. Thank god Luke for Puppet
  12. And Mark for Cfengine
  13. } #monitoringsucks 2013
  14. What we want ● Small , wel suited components • Collect • Transport / Mangle • Analyse / Act • Visualize
  15. Collectd ● Collects ● Zillion Plugins • Nginx,apache, mysql, disk ● Graphite Carbon Plugin ● Send metrics to graphite ● https://github.com/KrisBuytaert/pupp et-collectd
  16. ● Collect from anywhere ● Filter ● Send anywhere
  17. Puppet and Logstash ● https://github.com/KrisBuytaert/vagrant-puppet-logstash ● Includes Logstash / Kibana / Grok / ES modules as submodules ! git clone ● git submodule init ● git submodule update ● vagrant up
  18. Graphite ● Graphing at Scale ● Graphing at Ease ● Any metric is a graph ● echo "somestring $somevalue $timestamp" | nc <%= graphitehost %> 2003
  19. Graphite Composer x
  20. Graphite API
  21. Gdash In action
  22. Puppet and Graphite ● https://github.com/KrisBuytaert/vagrant-graphite/ ● Includes Graphite / Gdash / Jmxtrans / Logster / Collectd / Statsd / Tattle and more modules as submodules ! ● git clone ● git submodule init ● git submodule update ● vagrant up
  23. Collectd & Graphite
  24. Configuring a Service ● Package ● Config ● Service <- we pray status is correct
  25. Apache Example:
  26. Definition of Done: Done means : being monitored
  27. Icinga ? • Isn't nagios dead ? • Vibrant Community • Throw great parties in Nurnberg • Nobody can pronounce it anyhow • https://github.com/Inuits/puppet-icinga/
  28. Monitoring Baseline ● Deploy a host, ● Add it to the monitoring ● Add collection tools ● Add check definitions ● Update the monitoring tool config
  29. Stored Configs
  30. Collection and Export Export : Collect: @@resource { Resource <<| query |>> ... } Clean out nodes that dissapear puppet node clean
  31. Exporting and Collecting
  32. Monitoring a Vhost
  33. It is not in production if it is not monitored
  34. Triggers on Graphs ● Export Java Metrics ● Collect JMX Exports on JMXTransNode ● JMXTrans ● Graph Em ● Export JMXConfigs Collect Nagios Configs ● Configure NRPE Check on Nagios Server ● Export NagiosCheck
  35. Triggers on Graphs
  36. Triggers on Graphs
  37. Conclusion: ● Reproducable monitoring setup ● Dynamically generated monitoring config ● Code is available at github.com/KrisBuytaert/snippets
  38. But, (pronounced in Ozzy dialect)
  39. http://www.kartar.net/2013/01/monitoring-sucks---a-rant/
  40. #monitoringlove
  41. Homework Sensu Riemann Extrememon Esper ...
  42. Contact Kris.Buytaert@inuits.eu Further Reading @krisbuytaert http://www.krisbuytaert.be/blog/ http://www.inuits.eu/ Inuits Duboistraat 50 2060 Antwerpen Belgium 891.514.231 +32 475 961221
  43. http://loadays.org

×