SlideShare a Scribd company logo
1 of 49
Download to read offline
7 Tools for your devops
          stack
          Kris Buytaert
     European Tour April 2012
Kris Buytaert
•   I used to be a Dev,
•   Then Became an Op
•   Chief Trolling Officer and Open Source
    Consultant @inuits.eu
•   Everything is an effing DNS Problem
•   Building Clouds since before the bookstore
•   Some books, some papers, some blogs
•   But mostly, trying to be good at my job
What's this devops thing
       anyhow ?
World , 200X-2009
 Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz
Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, Luke
                 Kanies, John Allspaw and lots of others ..

                     Gent , October 2009
                Mountain View , June 2010
                  Hamburg , October 2010
                     Boston, March 2011
                 Mountain View, June 2011
                   Bangalore, Melbourne,
                  Goteborg , October 2011
                                  ....
●   Devops is a growing professional and cultural
    movement
●   We don't have all the answers yet
●   We are reaching out to different communities
●   We will point out problems we see..
●   Only the name is new




         While we are still working out the solutions
Devops, a definition:
●   Adopt the new philosophy. We are in a new economic age. Western management
    must awaken to the challenge, must learn their responsibilities, and take on
    leadership for change.
●   Cease dependence on inspection to achieve quality. Eliminate the need for massive
    inspection by building quality into the product in the first place.
●   Improve constantly and forever the system of production and service, to improve
    quality and productivity, and thus constantly decrease costs.
●   Institute training on the job.
●   Institute leadership The aim of supervision should be to help people and machines
    and gadgets do a better job.
●   Drive out fear, so that everyone may work effectively for the company.
●   Break down barriers between departments. People in research, design, sales, and
    production must work as a team, in order to foresee problems of production and
    usage that may be encountered with the product or service.
●   Eliminate slogans, exhortations, and targets for the work force asking for zero defects
    and new levels of productivity. Such exhortations only create adversarial
    relationships, as the bulk of the causes of low quality and low productivity belong to
    the system and thus lie beyond the power of the work force.

              ●   Eliminate management by objective. Eliminate management by numbers
                  and numerical goals. Instead substitute with leadership.

              ●   Remove barriers that rob the hourly worker of his right to pride of
                  workmanship. The responsibility of supervisors must be changed from
                  sheer numbers to quality.

              ●  Remove barriers that rob people in management and in engineering of
                 their right to pride of workmanship.
●   Institute a vigorous program of education and self-improvement.
●   Put everybody in the company to work to accomplish the transformation. The
    transformation is everybody's job.
William Edwards
          Deming
1986, Out of the Crisis.



  http://en.wikipedia.org/wiki/W._Edwards_Deming
“DevOps is a cultural and
professional movement”
                            Adam Jacob
CAMS
●   Culture
●   Automation
●   Measurement
●   Sharing


              Damon Edwards and John Willis
Culture,
   automation,


  Measturement,


     sharing
Devs vs Ops
Breaking the Silos




Devs    Ops   Getting Along
What tool?
Culture,




Automation,
    Measurement,


      Sharing
“Solved problems”
●   Deployment automation :
         ●   Fai , kickstart, preseed, cobbler,
●   Configuration Management
         ●   Puppet, chef, cfengine, bcfg2
●   Build Automation
         ●   Jenkins, maven, ant, cruisecontrol,
             hudson
“challenges”
•   orchestration
    •   Mcollective
    •   Rundeck
    •   Noah
    Package & Repository management
    •   Pulp, sponge
    •   Fpm
The Marionette Collective
●   Distributed ssh ++
●   What version of ssh do I have installed on my
    servers ?
●   On what servers is XYZ running ?
●   Clean al my ssl certs ?
●   Restart apache on all servers with fact X
mc-service
mc-service httpd status -W /uat/

  *
[ =================================================
===========> ] 80 / 80

 bleh.poc.com                status=stopped
 xs03.ta.google.com              status=stopped
 elasticsearch02.uat.com           status=stopped
 dbs01-.poc.com                status=stopped
 dino.uat.com                status=stopped
 dsp01.uat.com                 status=stopped
 webserver01.uat.com               status=running
 dbs01-old.uat.com              status=stopped
 alexdb02-hdp.poc.com              status=stopped
 postfix.uat.com              status=stopped
 ns01.uat.com                 status=stopped
 devtools03.uat.com              status=running
mc-package
mc-package -W /dev/ status jdk
 * [ ============================================================> ] 33 / 33

servicesdb01.dev.com             version = -absent
services.dev.google.com           version = jdk-1.6.0_13-fcs
drbdtest02.dev.google.com           version = -absent
services3.dev.google.com           version = jdk-1.6.0_20-fcs
um.dev.google.com               version = jdk-1.5.0_19-fcs
devtools03.uat.com              version = jdk-1.6.0_29-fcs
alexandria02.dev.google.com          version = -absent
weblink01.dev.com               version = -absent
wikitest.dev.google.com          version = jdk-1.6.0_24-fcs
payment.dev.google.com              version = jdk-1.5.0_17-fcs
tiff2pdf01.dev.com             version = -absent
devdoos.dev.com                version = jdk-1.6.0_30-fcs
wiki.dev.google.com             version = jdk-1.6.0_24-fcs
reporting01.dev.com             version = -absent
devtools01-dev.uat.com            version = jdk-1.6.0_23-fcs
devtools02.uat.com              version = jdk-1.6.0_29-fcs
drbdtest01.dev.google.com           version = -absent

---- package agent summary ----
        Nodes: 33/33
            Versions: 1 * 1.5.0_17-fcs, 1 * 1.5.0_19-fcs, 1 * 1.6.0_13-fcs, 1 * 1.6.0_20-fcs, 1 * 1.6.0_23-fcs, 2 * 1.6.0_24-fcs, 2 * 1.6.0
               Elapsed Time: 1.73 s
fpm
●   Yes the F stands for Effing

●   #packagingsucks

             ●   Missing upstream

             ●   Ancient upstream

             ●   Unneeded dependencies

             ●   Broken upstream

             ●   Distro Policies

●   “Maximum RPM” ?

●   Packaging is needed

             ●   Dependencies, tooling, repositories

●   Anger driven development
fpm
fpm
fpm -t rpm -s dir -n hornetq -v 2.2.5 hornetq
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.nNkVwh
+ umask 022
+ cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.yUd4MV
+ umask 022
+ cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ tar -zxf /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/data.tar.gz
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.jkpqeA
+ umask 022
+ cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/brp-strip
+ /usr/lib/rpm/brp-strip-static-archive
+ /usr/lib/rpm/brp-strip-comment-note
Processing files: hornetq-2.2.5-1.x86_64
Checking for unpackaged file(s): /usr/lib/rpm/check-files /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
Wrote: /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/SRPMS/hornetq-2.2.5-1.src.rpm
Wrote: /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/RPMS/x86_64/hornetq-2.2.5-1.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.z2UL3B
+ umask 022
+ cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ rm -rf /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD
+ exit 0
Created /usr/local/hornetq-2.2.5.x86_64.rpm
fpm in action
●   https://github.com/KrisBuytaert/build-gems
●   Fork, pull
●   Jenkins pulls , builds , pushes to repo
Culture,


     Automation,




Measurement,
       Sharing
#monitoringsucks
    Monitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love:

●   Having my hands tied with the model of host and service bindings.

●   Having to set up "fake" hosts just to group arbitrary metrics together

●   Having to either collect metrics twice - once for alerting and another for trending

●   Only being able to see my metrics in 5 minute intervals

●   Having to chose between shitty interface but great monitoring or shitty monitoring
    but great interface

●    Dealing with a monitoring system that thinks IT is the system of truth for my
    environment

●   Not actually having any real choices

               John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why-
                                                                      monitoring-sucks.html
A sub movement
●   #monitoringsucks trending
●   https://github.com/monitoringsucks/
●   2008 Study :Nagios + Friends
●   2011 Conclusion : Nagios/Icinga are the only
    automatable alternatives
●   Monitoring and trending at scale , new kids
    Graphite, Icinga, flapjack, etc
●   Old Cool Kids, Ganglia
Logstash
●   Not your average centralized logging tool
●   Elasticsearch backed




●   Shipper
●   Indexer
●   Web
Logstash

    ●   Collect from anywhere
    ●   Filter
    ●   Send anywhere




    ●   Queuing support
Logstash in Action
Kibana in Action
Graphite
●   Graphing at Scale
●   Graphing at Ease
●   Any metric is a graph


●   echo "somestring $somevalue $timestamp" |
    nc <%= graphitehost %> 2003
Graphite Composer

   x
Graphite API
Friends of Graphite
●   Collection :
          ●   Statsd,
          ●   Collectd,
          ●   Jmxtrans
          ●   Logster
●   Dashboards
          ●   Tattle
          ●   Gdash
          ●   ...
Culture,


  Automation,


  Measurement,




Sharing
●   Open Source
●   Github
●   Talk about Experiences
●   Open Spaces
Sharing environments

●   Build identical environments
●   Share code
●   Shared ownership of content , code and
    configuration
Vagrant
●   Abstraction layer for VirtualBox
●   Integrates well with Puppet/Chef
●   Project =
         ●   Vagrantfile
         ●   Manifests / Cookbooks
●   Portable, Small , Versionable
●   Use veewee to build your boxen
Vagrantfile
Vagrant::Config.run do |config|
 # All Vagrant configuration is done here. The most common configuration
 # options are documented and commented below. For a complete reference,
 # please see the online documentation at vagrantup.com.

 config.vm.define :mongo1 do |mongo1_config|
    mongo1_config.ssh.max_tries = 100
    mongo1_config.vm.box = "MyCentOS2"
    mongo1_config.vm.network("192.168.99.101")
    mongo1_config.vm.host_name = "mongo1"
    mongo1_config.vm.provision :puppet do |mongo1_puppet|
         mongo1_puppet.pp_path = "/tmp/vagrant-puppet"
     mongo1_puppet.manifests_path = "manifests"
     mongo1_puppet.module_path = "modules"
     mongo1_puppet.manifest_file = "site.pp"
    end
  end
  config.vm.define :mongo2 do |mongo2_config|
   mongo2_config.ssh.max_tries = 100
    mongo2_config.vm.box = "MyCentOS2"
    mongo2_config.vm.network("192.168.99.102")
    mongo2_config.vm.host_name = "mongo2"
    mongo2_config.vm.provision :puppet do |mongo2_puppet|
     mongo2_puppet.pp_path = "/tmp/vagrant-puppet"
     mongo2_puppet.manifests_path = "manifests"
     mongo2_puppet.module_path = "modules"
     mongo2_puppet.manifest_file = "site.pp"
    end
  end
Vagrant Rocks
●   Vagrant init
●   Vagrant up
●   Vagrant provision
●   Vagrant down
●   Vagrant destroy
Sushi
Quiz Time :
Which tool did I forget ?
You
Links
http://puppetlabs.com/mcollective/introduction/
https://github.com/jordansissel/fpm
http://logstash.net/
https://github.com/rashidkpc/Kibana
http://graphite.wikidot.com/
https://github.com/wayfair/Graphite-Tattle
http://vagrantup.com/
https://github.com/KrisBuytaert
Contact
Kris Buytaert
Kris.Buytaert@inuits.be

Further Reading
@krisbuytaert
http://www.krisbuytaert.be/blog/
http://www.inuits.be/




                             Inuits

                             Duboistraat 50
                             2060 Antwerpen
                             Belgium
                             891.514.231

                             +32 475 961221

More Related Content

Similar to Devops Tools for Culture, Automation, Measurement and Sharing

OSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris BuyaertOSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris BuyaertNETWAYS
 
OSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris BuytaertOSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris BuytaertNETWAYS
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at ScaleKris Buytaert
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at FrosconKris Buytaert
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Kris Buytaert
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Kris Buytaert
 
Using SaltStack to DevOps the enterprise
Using SaltStack to DevOps the enterpriseUsing SaltStack to DevOps the enterprise
Using SaltStack to DevOps the enterpriseChristian McHugh
 
Devops at Startup Weekend BXL
Devops at Startup Weekend BXLDevops at Startup Weekend BXL
Devops at Startup Weekend BXLKris Buytaert
 
Devops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetDevops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetKris Buytaert
 
Towards Continuous Deployment with Django
Towards Continuous Deployment with DjangoTowards Continuous Deployment with Django
Towards Continuous Deployment with DjangoRoger Barnes
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with PuppetKris Buytaert
 
Release management with NuGet/Chocolatey/JIRA
Release management with NuGet/Chocolatey/JIRARelease management with NuGet/Chocolatey/JIRA
Release management with NuGet/Chocolatey/JIRAYaroslav Serhieiev
 
The UNE VLE Implementation: an initial overview
The UNE VLE Implementation: an initial overviewThe UNE VLE Implementation: an initial overview
The UNE VLE Implementation: an initial overviewmylescarrick
 
Continuous delivery - tools and techniques
Continuous delivery - tools and techniquesContinuous delivery - tools and techniques
Continuous delivery - tools and techniquesMike McGarr
 
Software Engineering For Startups
Software Engineering For StartupsSoftware Engineering For Startups
Software Engineering For StartupsRightScale
 
Groovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineGroovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineKris Buytaert
 
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert NETWAYS
 

Similar to Devops Tools for Culture, Automation, Measurement and Sharing (20)

OSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris BuyaertOSDC 2012 | Devops and Open Source by Kris Buyaert
OSDC 2012 | Devops and Open Source by Kris Buyaert
 
OSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris BuytaertOSDC 2012 | Devops and Open Source by Kris Buytaert
OSDC 2012 | Devops and Open Source by Kris Buytaert
 
Deploying software at Scale
Deploying software at ScaleDeploying software at Scale
Deploying software at Scale
 
Devops for drupal
Devops for  drupalDevops for  drupal
Devops for drupal
 
Devops Devops Devops, at Froscon
Devops Devops Devops, at FrosconDevops Devops Devops, at Froscon
Devops Devops Devops, at Froscon
 
Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.Devops, the future is here, it's just not evenly distributed yet.
Devops, the future is here, it's just not evenly distributed yet.
 
Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?Devops, Secops, Opsec, DevSec *ops *.* ?
Devops, Secops, Opsec, DevSec *ops *.* ?
 
Devops For Drupal
Devops  For DrupalDevops  For Drupal
Devops For Drupal
 
Using SaltStack to DevOps the enterprise
Using SaltStack to DevOps the enterpriseUsing SaltStack to DevOps the enterprise
Using SaltStack to DevOps the enterprise
 
Devops at Startup Weekend BXL
Devops at Startup Weekend BXLDevops at Startup Weekend BXL
Devops at Startup Weekend BXL
 
Devops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yetDevops, the future is here it's not evenly distributed yet
Devops, the future is here it's not evenly distributed yet
 
Towards Continuous Deployment with Django
Towards Continuous Deployment with DjangoTowards Continuous Deployment with Django
Towards Continuous Deployment with Django
 
Automating MySQL operations with Puppet
Automating MySQL operations with PuppetAutomating MySQL operations with Puppet
Automating MySQL operations with Puppet
 
Release management with NuGet/Chocolatey/JIRA
Release management with NuGet/Chocolatey/JIRARelease management with NuGet/Chocolatey/JIRA
Release management with NuGet/Chocolatey/JIRA
 
The UNE VLE Implementation: an initial overview
The UNE VLE Implementation: an initial overviewThe UNE VLE Implementation: an initial overview
The UNE VLE Implementation: an initial overview
 
Continuous delivery - tools and techniques
Continuous delivery - tools and techniquesContinuous delivery - tools and techniques
Continuous delivery - tools and techniques
 
Java User Group Cologne
Java User Group CologneJava User Group Cologne
Java User Group Cologne
 
Software Engineering For Startups
Software Engineering For StartupsSoftware Engineering For Startups
Software Engineering For Startups
 
Groovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineGroovy there's a docker in my application pipeline
Groovy there's a docker in my application pipeline
 
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
 

More from Kris Buytaert

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsKris Buytaert
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteKris Buytaert
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code PatternsKris Buytaert
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningKris Buytaert
 
Pipeline all the Dashboards as Code
Pipeline all the Dashboards as CodePipeline all the Dashboards as Code
Pipeline all the Dashboards as CodeKris Buytaert
 
Help , My Datacenter is on fire
Help , My Datacenter is on fireHelp , My Datacenter is on fire
Help , My Datacenter is on fireKris Buytaert
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live DevopsKris Buytaert
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? Kris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure FirstKris Buytaert
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ? Kris Buytaert
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdnessKris Buytaert
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of DevopsKris Buytaert
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?Kris Buytaert
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite EditionKris Buytaert
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure FirstKris Buytaert
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019 Kris Buytaert
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5Kris Buytaert
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as CodeKris Buytaert
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security RequirementKris Buytaert
 

More from Kris Buytaert (20)

Years of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoopsYears of (not) learning , from devops to devoops
Years of (not) learning , from devops to devoops
 
Observability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,IgniteObservability will not fix your Broken Monitoring ,Ignite
Observability will not fix your Broken Monitoring ,Ignite
 
Infrastructure as Code Patterns
Infrastructure as Code PatternsInfrastructure as Code Patterns
Infrastructure as Code Patterns
 
From devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learningFrom devoops to devops 13 years of (not) learning
From devoops to devops 13 years of (not) learning
 
Pipeline all the Dashboards as Code
Pipeline all the Dashboards as CodePipeline all the Dashboards as Code
Pipeline all the Dashboards as Code
 
Help , My Datacenter is on fire
Help , My Datacenter is on fireHelp , My Datacenter is on fire
Help , My Datacenter is on fire
 
GitOps , done Right
GitOps , done RightGitOps , done Right
GitOps , done Right
 
Devops is Dead, Long live Devops
Devops is Dead, Long live DevopsDevops is Dead, Long live Devops
Devops is Dead, Long live Devops
 
10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ? 10 years of #devopsdays, but what have we really learned ?
10 years of #devopsdays, but what have we really learned ?
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Is there a Future for devops ?
Is there a Future for devops   ? Is there a Future for devops   ?
Is there a Future for devops ?
 
10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness10 Years of #devopsdays weirdness
10 Years of #devopsdays weirdness
 
ADDO 2019: Looking back at over 10 years of Devops
ADDO 2019:    Looking back at over 10 years of DevopsADDO 2019:    Looking back at over 10 years of Devops
ADDO 2019: Looking back at over 10 years of Devops
 
Can we fix dev-oops ?
Can we fix dev-oops ?Can we fix dev-oops ?
Can we fix dev-oops ?
 
Continuous Infrastructure First Ignite Edition
Continuous Infrastructure First  Ignite EditionContinuous Infrastructure First  Ignite Edition
Continuous Infrastructure First Ignite Edition
 
Continuous Infrastructure First
Continuous Infrastructure FirstContinuous Infrastructure First
Continuous Infrastructure First
 
Open Source Monitoring in 2019
Open Source Monitoring in 2019 Open Source Monitoring in 2019
Open Source Monitoring in 2019
 
Migrating to Puppet 5
Migrating to Puppet 5Migrating to Puppet 5
Migrating to Puppet 5
 
Repositories as Code
Repositories as CodeRepositories as Code
Repositories as Code
 
Devops is a Security Requirement
Devops is a Security RequirementDevops is a Security Requirement
Devops is a Security Requirement
 

Recently uploaded

Tecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureTecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureAntonio de Llamas
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2DianaGray10
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxKunal Gupta
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdfHulkTheDevil
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxmprakaash5
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactivestartupro
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 

Recently uploaded (20)

Tecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for RotogravureTecnogravura, Cylinder Engraving for Rotogravure
Tecnogravura, Cylinder Engraving for Rotogravure
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
Efficiencies in RPA with UiPath and CyberArk Technologies - Session 2
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptx
 
full stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdffull stack practical assignment msc cs.pdf
full stack practical assignment msc cs.pdf
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptx
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactive
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 

Devops Tools for Culture, Automation, Measurement and Sharing

  • 1. 7 Tools for your devops stack Kris Buytaert European Tour April 2012
  • 2. Kris Buytaert • I used to be a Dev, • Then Became an Op • Chief Trolling Officer and Open Source Consultant @inuits.eu • Everything is an effing DNS Problem • Building Clouds since before the bookstore • Some books, some papers, some blogs • But mostly, trying to be good at my job
  • 3. What's this devops thing anyhow ?
  • 4. World , 200X-2009 Patrick Debois, Gildas Le Nadan, Andrew Clay Shafer, Kris Buytaert, Jezz Humble, Lindsay Holmwood, John Willis, Chris Read, Julian Simpson, Luke Kanies, John Allspaw and lots of others .. Gent , October 2009 Mountain View , June 2010 Hamburg , October 2010 Boston, March 2011 Mountain View, June 2011 Bangalore, Melbourne, Goteborg , October 2011 ....
  • 5. Devops is a growing professional and cultural movement ● We don't have all the answers yet ● We are reaching out to different communities ● We will point out problems we see.. ● Only the name is new While we are still working out the solutions
  • 7. Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change. ● Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place. ● Improve constantly and forever the system of production and service, to improve quality and productivity, and thus constantly decrease costs. ● Institute training on the job. ● Institute leadership The aim of supervision should be to help people and machines and gadgets do a better job. ● Drive out fear, so that everyone may work effectively for the company. ● Break down barriers between departments. People in research, design, sales, and production must work as a team, in order to foresee problems of production and usage that may be encountered with the product or service. ● Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force. ● Eliminate management by objective. Eliminate management by numbers and numerical goals. Instead substitute with leadership. ● Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality. ● Remove barriers that rob people in management and in engineering of their right to pride of workmanship. ● Institute a vigorous program of education and self-improvement. ● Put everybody in the company to work to accomplish the transformation. The transformation is everybody's job.
  • 8. William Edwards Deming 1986, Out of the Crisis. http://en.wikipedia.org/wiki/W._Edwards_Deming
  • 9. “DevOps is a cultural and professional movement” Adam Jacob
  • 10. CAMS ● Culture ● Automation ● Measurement ● Sharing Damon Edwards and John Willis
  • 11. Culture, automation, Measturement, sharing
  • 13.
  • 14. Breaking the Silos Devs Ops Getting Along
  • 16.
  • 17. Culture, Automation, Measurement, Sharing
  • 18. “Solved problems” ● Deployment automation : ● Fai , kickstart, preseed, cobbler, ● Configuration Management ● Puppet, chef, cfengine, bcfg2 ● Build Automation ● Jenkins, maven, ant, cruisecontrol, hudson
  • 19. “challenges” • orchestration • Mcollective • Rundeck • Noah Package & Repository management • Pulp, sponge • Fpm
  • 20. The Marionette Collective ● Distributed ssh ++ ● What version of ssh do I have installed on my servers ? ● On what servers is XYZ running ? ● Clean al my ssl certs ? ● Restart apache on all servers with fact X
  • 21. mc-service mc-service httpd status -W /uat/ * [ ================================================= ===========> ] 80 / 80 bleh.poc.com status=stopped xs03.ta.google.com status=stopped elasticsearch02.uat.com status=stopped dbs01-.poc.com status=stopped dino.uat.com status=stopped dsp01.uat.com status=stopped webserver01.uat.com status=running dbs01-old.uat.com status=stopped alexdb02-hdp.poc.com status=stopped postfix.uat.com status=stopped ns01.uat.com status=stopped devtools03.uat.com status=running
  • 22. mc-package mc-package -W /dev/ status jdk * [ ============================================================> ] 33 / 33 servicesdb01.dev.com version = -absent services.dev.google.com version = jdk-1.6.0_13-fcs drbdtest02.dev.google.com version = -absent services3.dev.google.com version = jdk-1.6.0_20-fcs um.dev.google.com version = jdk-1.5.0_19-fcs devtools03.uat.com version = jdk-1.6.0_29-fcs alexandria02.dev.google.com version = -absent weblink01.dev.com version = -absent wikitest.dev.google.com version = jdk-1.6.0_24-fcs payment.dev.google.com version = jdk-1.5.0_17-fcs tiff2pdf01.dev.com version = -absent devdoos.dev.com version = jdk-1.6.0_30-fcs wiki.dev.google.com version = jdk-1.6.0_24-fcs reporting01.dev.com version = -absent devtools01-dev.uat.com version = jdk-1.6.0_23-fcs devtools02.uat.com version = jdk-1.6.0_29-fcs drbdtest01.dev.google.com version = -absent ---- package agent summary ---- Nodes: 33/33 Versions: 1 * 1.5.0_17-fcs, 1 * 1.5.0_19-fcs, 1 * 1.6.0_13-fcs, 1 * 1.6.0_20-fcs, 1 * 1.6.0_23-fcs, 2 * 1.6.0_24-fcs, 2 * 1.6.0 Elapsed Time: 1.73 s
  • 23. fpm ● Yes the F stands for Effing ● #packagingsucks ● Missing upstream ● Ancient upstream ● Unneeded dependencies ● Broken upstream ● Distro Policies ● “Maximum RPM” ? ● Packaging is needed ● Dependencies, tooling, repositories ● Anger driven development
  • 24. fpm
  • 25. fpm fpm -t rpm -s dir -n hornetq -v 2.2.5 hornetq Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.nNkVwh + umask 022 + cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + exit 0 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.yUd4MV + umask 022 + cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + tar -zxf /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/data.tar.gz + exit 0 Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.jkpqeA + umask 022 + cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + /usr/lib/rpm/brp-compress + /usr/lib/rpm/brp-strip + /usr/lib/rpm/brp-strip-static-archive + /usr/lib/rpm/brp-strip-comment-note Processing files: hornetq-2.2.5-1.x86_64 Checking for unpackaged file(s): /usr/lib/rpm/check-files /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD Wrote: /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/SRPMS/hornetq-2.2.5-1.src.rpm Wrote: /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/RPMS/x86_64/hornetq-2.2.5-1.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.z2UL3B + umask 022 + cd /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + rm -rf /usr/local/build-rpm-hornetq-2.2.5.x86_64.rpm/BUILD + exit 0 Created /usr/local/hornetq-2.2.5.x86_64.rpm
  • 26. fpm in action ● https://github.com/KrisBuytaert/build-gems ● Fork, pull ● Jenkins pulls , builds , pushes to repo
  • 27. Culture, Automation, Measurement, Sharing
  • 28. #monitoringsucks Monitoring is AWESOME. Metrics are AWESOME. I love it. Here's what I don't love: ● Having my hands tied with the model of host and service bindings. ● Having to set up "fake" hosts just to group arbitrary metrics together ● Having to either collect metrics twice - once for alerting and another for trending ● Only being able to see my metrics in 5 minute intervals ● Having to chose between shitty interface but great monitoring or shitty monitoring but great interface ● Dealing with a monitoring system that thinks IT is the system of truth for my environment ● Not actually having any real choices John Vincent (@lusis) on his blog http://lusislog.blogspot.com/2011/06/why- monitoring-sucks.html
  • 29. A sub movement ● #monitoringsucks trending ● https://github.com/monitoringsucks/ ● 2008 Study :Nagios + Friends ● 2011 Conclusion : Nagios/Icinga are the only automatable alternatives ● Monitoring and trending at scale , new kids Graphite, Icinga, flapjack, etc ● Old Cool Kids, Ganglia
  • 30.
  • 31. Logstash ● Not your average centralized logging tool ● Elasticsearch backed ● Shipper ● Indexer ● Web
  • 32. Logstash ● Collect from anywhere ● Filter ● Send anywhere ● Queuing support
  • 35. Graphite ● Graphing at Scale ● Graphing at Ease ● Any metric is a graph ● echo "somestring $somevalue $timestamp" | nc <%= graphitehost %> 2003
  • 38. Friends of Graphite ● Collection : ● Statsd, ● Collectd, ● Jmxtrans ● Logster ● Dashboards ● Tattle ● Gdash ● ...
  • 39. Culture, Automation, Measurement, Sharing
  • 40. Open Source ● Github ● Talk about Experiences ● Open Spaces
  • 41. Sharing environments ● Build identical environments ● Share code ● Shared ownership of content , code and configuration
  • 42. Vagrant ● Abstraction layer for VirtualBox ● Integrates well with Puppet/Chef ● Project = ● Vagrantfile ● Manifests / Cookbooks ● Portable, Small , Versionable ● Use veewee to build your boxen
  • 43. Vagrantfile Vagrant::Config.run do |config| # All Vagrant configuration is done here. The most common configuration # options are documented and commented below. For a complete reference, # please see the online documentation at vagrantup.com. config.vm.define :mongo1 do |mongo1_config| mongo1_config.ssh.max_tries = 100 mongo1_config.vm.box = "MyCentOS2" mongo1_config.vm.network("192.168.99.101") mongo1_config.vm.host_name = "mongo1" mongo1_config.vm.provision :puppet do |mongo1_puppet| mongo1_puppet.pp_path = "/tmp/vagrant-puppet" mongo1_puppet.manifests_path = "manifests" mongo1_puppet.module_path = "modules" mongo1_puppet.manifest_file = "site.pp" end end config.vm.define :mongo2 do |mongo2_config| mongo2_config.ssh.max_tries = 100 mongo2_config.vm.box = "MyCentOS2" mongo2_config.vm.network("192.168.99.102") mongo2_config.vm.host_name = "mongo2" mongo2_config.vm.provision :puppet do |mongo2_puppet| mongo2_puppet.pp_path = "/tmp/vagrant-puppet" mongo2_puppet.manifests_path = "manifests" mongo2_puppet.module_path = "modules" mongo2_puppet.manifest_file = "site.pp" end end
  • 44. Vagrant Rocks ● Vagrant init ● Vagrant up ● Vagrant provision ● Vagrant down ● Vagrant destroy
  • 45. Sushi
  • 46. Quiz Time : Which tool did I forget ?
  • 47. You