Johann-Peter Hartmann / Mayflower GmbH


Practical DevOps for
Developers
Hi!
Nice to meet You!
I am Johann.
Who are You?
Developers?
    Ops?
Anything else?
Developers:
    PHP?
  Python?
   Ruby?
    Java?
Anything else?
What do you expect to hear today?



    (if you got a good explanation
i‘ll leave out the motivation slides)
Motivation

                        or

Why it‘s more fun to do DevOps as a Dev right now.
Hype!
Googles CIO Ben Fried 11
        days ago:

   „Cult Of DevOps“
Actually:
Because we need it.
Development in the dark
       ages ...
Golden Image
VMware /KVM
It wasn‘t reliable anymore.
And our applications changed
Browser




Apache with PHP             MySQL
Browser




                             Hip
Apache with PHP   MySQL     NoSQL
                            Server
Browser




Apache with PHP                 MySQL




     Hip
                                Memcache
    NoSQL
                            Performance FTW!
    Server
Browser




                                             Hip
Apache with PHP         MySQL               NoSQL
                                            Server



          Memcache, oops,
                                  Gearman
            Redis now
Browser




                                    Hip
Apache with PHP      MySQL         NoSQL
                                   Server



Memcache, oops,   Gearman, ahum,
                                   eJabberD
  Redis now         ActiveMQ
Browser




                                                  Hip
Apache with   Apache with
                                       MySQL     NoSQL
   PHP           PHP
                                                 Server



Memcache,     Memcache,               Gearman,
  oops,         oops,                  ahum,     eJabberD
Redis now     Redis now               ActiveMQ
Browser




                                                  Hip
Apache with   Apache with
                                       MySQL     NoSQL
   PHP           PHP
                                                 Server



Memcache,     Memcache,               Gearman,
  oops,         oops,                  ahum,     eJabberD
Redis now     Redis now               ActiveMQ
Apa    Apa    MyS     Hip        Apa          Apa      MyS          Hip

Me     Me     Gea     eJa
                                     Me       Me       Gea          eJa


       Development                        Continuous Integration




 Apa    Apa     Apa    Apa     MyS          MyS       Hip          Hip

 Me      Me      Me     Me     Gea           Gea      eJa      eJa

                        Production
Version X+2                 Version X+1
   Development                  Continuous Integration




                 Version X
                   Production
New! With Redis!                Not so new with Memcached


Version X+2                  Version X+1
    Development                     Continuous Integration




         Still without proper Caching :-(

                  Version X
                    Production
10th Floor
   Test
    http://www.flickr.com/photos/75905404@N00/
Collection of Fails
                                Failsafety
  Simplicity
         Fast Setup Time
Repeatability Self-Service

Consistency
       Version Management
                  http://www.flickr.com/photos/turtlemom_nancy/
?
DevOps for Devs in
3 (actually not so easy) steps
1. Manage your Dev-Setup
 with Vagrant and VeeWee

 (Install VirtualBox and Ruby first)
Vagrant


          Virtualbox based
           automatic creation
           and management of
           VMs based on
           Puppet / Chef
Better use rvm, see http://www.jedi.be/blog/2011/03/28/
using-vagrant-as-a-team/


~# gem install vagrant

~# gem install veewee

~# vagrant basebox templates

vagrant basebox define '<boxname>' 'archlinux-i386'
... (40 baseboxes)
vagrant basebox define '<boxname>' 'windows-2008R2-amd64'

~# vagrant   basebox define 'natty' 'ubuntu-11.04-server-
amd64'
~# vagrant   basebox build 'natty'
~# vagrant   basebox export   natty
~# vagrant   box add 'natty' 'natty.box'
http://vagrantbox.es
~# vagrant init 'natty'

~# vagrant up

~# vagrant ssh
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-server x86_64)

 * Documentation:   http://www.ubuntu.com/server/doc

vagrant@natty:~$
~# vagrant
Tasks:
  vagrant basebox      # Commands to manage baseboxes
  vagrant box          # Commands to manage system boxes
  vagrant destroy      # Destroy the environment, deleting the created virtual
machines
  vagrant halt         # Halt the running VMs in the environment
  vagrant help [TASK] # Describe available tasks or one specific task
  vagrant init [box_name] [box_url] # Initializes the current folder for
Vagrant usage
  vagrant package      # Package a Vagrant environment for distribution
  vagrant provision    # Rerun the provisioning scripts on a running VM
  vagrant reload       # Reload the environment, halting it then restarting
it.
  vagrant resume       # Resume a suspended Vagrant environment.
  vagrant ssh          # SSH into the currently running Vagrant environment.
  vagrant ssh_config   # outputs .ssh/config valid syntax for connecting to
this environment via ssh
  vagrant status       # Shows the status of the current Vagrant environment.
  vagrant suspend      # Suspend a running Vagrant environment.
  vagrant up           # Creates the Vagrant environment
  vagrant version      # Prints the Vagrant version information
2. Manage your configuration
Similarities
• Configuration as (Ruby-)Code
• Client-only or Client-server Setup
• there is a series-b funded company in the background
• both are officially supported by amazon
• there are a lot of BIG customers using the tool
• good documentation
• good, vibrant communities
• both know the current configuration (ohai and facter)
• You define your nodes (Servers)
• using a lot of pre-existing resources
• and a lot of default community-built cookbooks / modules
• it‘s easy to extend using ruby
• use configuration file templates
• use providers as platform abstractions (for packaging, ...)
• Chef is actually Ruby, Puppet provides a DSL
• puppet has the bigger community
• puppet has more documentation
• but chef is growing fast in both regards
• puppet = europe, chef = usa
• chef is more flexible because of native ruby
• chef is more flexible because of clever data structures
There is no „better“ tool.
There is no „better“ tool.

But we prefer Puppet. Less Ruby :-)
user { 'johann':
      ensure       =>   present,
      uid          =>   '507',
      gid          =>   'admin',
      shell        =>   '/bin/bash',
      home         =>   '/home/johann',
      managehome   =>   true,
}
user "johann" do
    username "johann"
    password "$1$P$WXmqrQEVj88fVTHevErxq."
    shell "/bin/bash"
    system true
    supports :manage_home => true
end
Back to your setup ...
Vagrant::Config.run do |config|
  config.vm.box = "natty"
end
~# cat Vagrantfile
Vagrant::Config.run do |config|
  config.vm.provision :puppet, :module_path => "modules" do
|puppet|
    puppet.manifests_path = "manifests"
    puppet.manifest_file = "development.pp"
  end

  config.vm.define :web do |web_config|
    web_config.vm.box = "natty"
    web_config.vm.host_name = "webserver01"
    # web_config.vm.boot_mode = :gui
    web_config.vm.network "33.33.33.10"
    web_config.vm.forward_port "http", 80, 8080
    web_config.vm.forward_port "ssh", 22, 20022
    web_config.vm.share_folder "v-data", "/srv/www", "../
silex-demo"
  end
end
~# cat manifests/development.pp

import "classes/*"
node "webserver01" {
  include web
}
node "dbserver01" {
  include db
}

node "ciserver01" {
  include ci
}
~# cat manifests/classes/web.pp

class web inherits basenode {
  include apache
  include apache::php
  apache::vhost { 'silex-demo.local':
    port => '80',
    docroot => '/srv/www/docroot',
  }

  package { ["mysql-client", "php5-cli", "phpunit", "php5-
curl", "php5-dev", "php5-gd", "php5-imagick", "php5-mcrypt",
"php5-mysql", "php5-xdebug","php5-suhosin", "php-pear",
"php-codesniffer" ]:
    ensure => present,
  }
}
~# cat manifests/classes/ci.pp

class ci inherits basenode {
  include apache
  include apache::php
  exec { "pear_autodiscover":
      command => "/usr/bin/pear config-set auto_discover 1",
  }
  package { ["pear.phpunit.de/PHP_CodeBrowser",
"pear.phpunit.de/PHPUnit_MockObject", "pear.phpunit.de/
PHPUnit_Selenium", "pear.phpunit.de/PHP_CodeCoverage",
"pear.phpunit.de/PHP_Timer", "pear.phpunit.de/phpcpd",
"pear.phpunit.de/phploc"]:
    ensure => latest,
    provider => "pear",
    require => Exec["pear_autodiscover"]
  }
}
3. Make the configuration
 part of your sourcecode
• application
• data
• docs
• library
• public
• scripts
     • jobs
     • build
     • configuration
         • VagrantFile
         • manifests
         • modules
• temp
Ok, that was a lot of work.

 Why did i do that again?
Collection of Wins
                               Failsafety
  Simplicity
         Fast Setup Time
Repeatability Self-Service

Consistency
       Version Management
                 http://www.flickr.com/photos/turtlemom_nancy/
There is no golden Image
         anymore
There is just one directory in
        your source ...
„vagrant up“

„vagrant provision“
Even more vagrant fun ...
Fail Safety with Vagrant-Snap


vagrant snap take -d “snap1“
vagrant snap list
vagrant snap go “snap1“
What if i need to
simulate 20
McCloud


Wrapper likeVagrant around Fog

Transparent local & cloud usage

Supports EC2, OpenStack, KVM, etc
What if i need to
simulate 2000
machines?
mCollective


dssh/ssh-for-loop on steroids

fast management for loads of servers

uses puppet/facter or chef/ohai, MQ-
 based
$     mc-package -W "architecture=x86" status apache

    * [ ============================================================> ] 10 / 10

host01.example.com                   version   =   apache-2.2.9-7
host02.example.com                   version   =   apache-2.2.9-7
host03.example.com                   version   =   apache-2.2.9-7
host04.example.com                   version   =   apache-2.2.9-7
host05.example.com                   version   =   apache-2.2.9-7
host06.example.com                   version   =   apache-2.2.9-7
host07.example.com                   version   =   apache-2.2.9-7
host08.example.com                   version   =   apache-2.2.9-7
host09.example.com                   version   =   apache-2.2.9-7
host10.example.com                   version   =   apache-2.2.9-7

---- package agent summary ----
           Nodes: 10 / 10
        Versions: 10 * 0.25.5-1.el5
    Elapsed Time: 1.03 s
The Foreman - Machine Life Cycle
Management
Acts as a web front end for Puppet

Shows You the system inventory

Creates new machines and takes care
 of provisioning
Knowing everything about your
server configuration is great!
Create new servers
Your (possible)
   Todolist
1. Install Vagrant,
    Veewee etc
2. Add an
configuration
folder to your
3. Configure Your
 Vagrant images
   using chef or
      puppet
4. Setup a chef-
or puppet-server
    using this
  configuration
5. Move your CI,
   Staging and
Production-Setup
 to chef/puppet,
       too
6. Create a self-
service-plattform
      for the
development team
More cool stuff
 you can do:
Test-Driven
  infrastructure
   using chef/
puppet-cucumber
Automated
monitoring using
nagios or munin
Automated
reporting using
   graphite
Thanks!
Johann-Peter Hartmann
johann__ @ freenode       I hope you enjoyed it!
Mail / Jabber:
johann-peter.hartmann@mayflower.de

Further reading:
http://www.planetdevops.net
http://dzone.com
http://twitter.com/#!/DEVOPS_BORAT
http://github.com/johannhartmann/
Vegetable Test Driven
Infrastructure
             Kontinuierliche Entiwcklung - und dann?   I   Mayflower GmbH   I   28. Oktober 2010   I 87
actually a behavior driven design tool
used for test driven infrastructure

cucumber-puppet
chef-cucumber
Feature: Manualsearch
  In order to find an article
  As an developer
  I want to use the search function
  Scenario: Search for bdd and check resulting page
    Given I go to "http://it-republik.de/php/"
    When I fill in "search_itr" with "bdd"
    And I click "search2"
    Then I should see "Suche"
Given /^I go to "([^"]*)"$/ do |url|
  visit url
end

When /^I fill in "([^"]*)" with "([^"]*)"$/ do |field, value|
  fill_in field, :with => value
end

When /^I click "([^"]*)"$/ do |button|
    click_button(button)
end

Then /^I should see "([^"]*)"$/ do |text|
  response_body.should include(text)
end
johann$ cucumber
Feature: Manualsearch
  In order to find an article
  As an developer
  I want to use the search function

  Scenario: Search for bdd and check resulting page # features/
search.feature:5
    Given I go to "http://it-republik.de/php/"      # features/
step_definitions/search_steps.rb:1
    When I fill in "search_itr" with "bdd"          # features/
step_definitions/search_steps.rb:5
    And I click "search2"                           # features/
step_definitions/search_steps.rb:9
    Then I should see "Suche"                       # features/
step_definitions/search_steps.rb:13

1 scenario (1 passed)
4 steps (4 passed)
0m1.615s
Feature: Install inetd
  In order to serve the web
  the httpd service
  must be installed

  Scenario: Setup httpd
    Given a node of class “web“
    When I compile the catalog
    Then package “httpd“ should be “installed“
    Then there should be a resource “Service[httpd]“
    And the service should have “enable“ set to “true“
    And State should be “running“
    And the service should require “Package[httpd]“




                               Kontinuierliche Entiwcklung - und dann?   I   Mayflower GmbH   I   28. Oktober 2010   I 92
Hey, i understand
Operations - and i‘m
 working in marketing
Unit Tests
Acceptance Tests
Metrics
Coding Style

... you get the
idea.
... and even
more ...

Packaging
Infrastructure
Updates
Releases
Deployment
Wetware
Refactoring
Work together
Own together
Eat together




         http://www.flickr.com/photos/vilavelosa/
         3815032524/
Plan
together




           Kontinuierliche Entiwcklung - und dann?   I   Mayflower GmbH   I   28. Oktober 2010   I 103
Mayflower
DevOpsification
Wetware - Mayflower


 1-2 Ops per team
   • Admin & Development tasks
   • full time working within the
     team
Wetware - Mayflower



Close cooperation with central operations

Full root access for any development-infrastructre
Wetware - Mayflower


1+n Puppet-Master
  • central company master
  • Team puppet master per team / project
  • company master is starting point for the project
    puppet configuration
Wetware - Mayflower


Example Setup:
  •   local Developer VM
  •   CI-Deployment-Server in the DMZ
  •   Staging in the private Cloud
  •   Beta happens in the Amazon-Cloud
  •   Production in AWS, too
Wetware - Mayflower


local GIT- / Gitorious-Server

Eucalyptus-Cloud in the DMZ
 - as Self-Service!
Wetware - Mayflower


    Vagrant for Development
       Scrum => KanBan
        Puppet Nagios

Dev ops for developers

  • 1.
    Johann-Peter Hartmann /Mayflower GmbH Practical DevOps for Developers
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
    Developers? Ops? Anything else?
  • 7.
    Developers: PHP? Python? Ruby? Java? Anything else?
  • 8.
    What do youexpect to hear today? (if you got a good explanation i‘ll leave out the motivation slides)
  • 9.
    Motivation or Why it‘s more fun to do DevOps as a Dev right now.
  • 10.
    Hype! Googles CIO BenFried 11 days ago: „Cult Of DevOps“
  • 11.
  • 12.
    Development in thedark ages ...
  • 13.
  • 17.
  • 18.
  • 19.
  • 20.
    Browser Hip Apache with PHP MySQL NoSQL Server
  • 21.
    Browser Apache with PHP MySQL Hip Memcache NoSQL Performance FTW! Server
  • 22.
    Browser Hip Apache with PHP MySQL NoSQL Server Memcache, oops, Gearman Redis now
  • 23.
    Browser Hip Apache with PHP MySQL NoSQL Server Memcache, oops, Gearman, ahum, eJabberD Redis now ActiveMQ
  • 24.
    Browser Hip Apache with Apache with MySQL NoSQL PHP PHP Server Memcache, Memcache, Gearman, oops, oops, ahum, eJabberD Redis now Redis now ActiveMQ
  • 25.
    Browser Hip Apache with Apache with MySQL NoSQL PHP PHP Server Memcache, Memcache, Gearman, oops, oops, ahum, eJabberD Redis now Redis now ActiveMQ
  • 26.
    Apa Apa MyS Hip Apa Apa MyS Hip Me Me Gea eJa Me Me Gea eJa Development Continuous Integration Apa Apa Apa Apa MyS MyS Hip Hip Me Me Me Me Gea Gea eJa eJa Production
  • 27.
    Version X+2 Version X+1 Development Continuous Integration Version X Production
  • 28.
    New! With Redis! Not so new with Memcached Version X+2 Version X+1 Development Continuous Integration Still without proper Caching :-( Version X Production
  • 30.
    10th Floor Test http://www.flickr.com/photos/75905404@N00/
  • 31.
    Collection of Fails Failsafety Simplicity Fast Setup Time Repeatability Self-Service Consistency Version Management http://www.flickr.com/photos/turtlemom_nancy/
  • 32.
  • 34.
    DevOps for Devsin 3 (actually not so easy) steps
  • 35.
    1. Manage yourDev-Setup with Vagrant and VeeWee (Install VirtualBox and Ruby first)
  • 36.
    Vagrant Virtualbox based automatic creation and management of VMs based on Puppet / Chef
  • 37.
    Better use rvm,see http://www.jedi.be/blog/2011/03/28/ using-vagrant-as-a-team/ ~# gem install vagrant ~# gem install veewee ~# vagrant basebox templates vagrant basebox define '<boxname>' 'archlinux-i386' ... (40 baseboxes) vagrant basebox define '<boxname>' 'windows-2008R2-amd64' ~# vagrant basebox define 'natty' 'ubuntu-11.04-server- amd64' ~# vagrant basebox build 'natty' ~# vagrant basebox export natty ~# vagrant box add 'natty' 'natty.box'
  • 38.
  • 39.
    ~# vagrant init'natty' ~# vagrant up ~# vagrant ssh Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-server x86_64) * Documentation: http://www.ubuntu.com/server/doc vagrant@natty:~$
  • 40.
    ~# vagrant Tasks: vagrant basebox # Commands to manage baseboxes vagrant box # Commands to manage system boxes vagrant destroy # Destroy the environment, deleting the created virtual machines vagrant halt # Halt the running VMs in the environment vagrant help [TASK] # Describe available tasks or one specific task vagrant init [box_name] [box_url] # Initializes the current folder for Vagrant usage vagrant package # Package a Vagrant environment for distribution vagrant provision # Rerun the provisioning scripts on a running VM vagrant reload # Reload the environment, halting it then restarting it. vagrant resume # Resume a suspended Vagrant environment. vagrant ssh # SSH into the currently running Vagrant environment. vagrant ssh_config # outputs .ssh/config valid syntax for connecting to this environment via ssh vagrant status # Shows the status of the current Vagrant environment. vagrant suspend # Suspend a running Vagrant environment. vagrant up # Creates the Vagrant environment vagrant version # Prints the Vagrant version information
  • 41.
    2. Manage yourconfiguration
  • 43.
  • 44.
    • Configuration as(Ruby-)Code • Client-only or Client-server Setup • there is a series-b funded company in the background • both are officially supported by amazon • there are a lot of BIG customers using the tool • good documentation • good, vibrant communities
  • 45.
    • both knowthe current configuration (ohai and facter) • You define your nodes (Servers) • using a lot of pre-existing resources • and a lot of default community-built cookbooks / modules • it‘s easy to extend using ruby • use configuration file templates • use providers as platform abstractions (for packaging, ...)
  • 46.
    • Chef isactually Ruby, Puppet provides a DSL • puppet has the bigger community • puppet has more documentation • but chef is growing fast in both regards • puppet = europe, chef = usa • chef is more flexible because of native ruby • chef is more flexible because of clever data structures
  • 47.
    There is no„better“ tool.
  • 48.
    There is no„better“ tool. But we prefer Puppet. Less Ruby :-)
  • 49.
    user { 'johann': ensure => present, uid => '507', gid => 'admin', shell => '/bin/bash', home => '/home/johann', managehome => true, }
  • 50.
    user "johann" do username "johann" password "$1$P$WXmqrQEVj88fVTHevErxq." shell "/bin/bash" system true supports :manage_home => true end
  • 51.
    Back to yoursetup ...
  • 52.
    Vagrant::Config.run do |config| config.vm.box = "natty" end
  • 53.
    ~# cat Vagrantfile Vagrant::Config.rundo |config| config.vm.provision :puppet, :module_path => "modules" do |puppet| puppet.manifests_path = "manifests" puppet.manifest_file = "development.pp" end config.vm.define :web do |web_config| web_config.vm.box = "natty" web_config.vm.host_name = "webserver01" # web_config.vm.boot_mode = :gui web_config.vm.network "33.33.33.10" web_config.vm.forward_port "http", 80, 8080 web_config.vm.forward_port "ssh", 22, 20022 web_config.vm.share_folder "v-data", "/srv/www", "../ silex-demo" end end
  • 54.
    ~# cat manifests/development.pp import"classes/*" node "webserver01" { include web } node "dbserver01" { include db } node "ciserver01" { include ci }
  • 55.
    ~# cat manifests/classes/web.pp classweb inherits basenode { include apache include apache::php apache::vhost { 'silex-demo.local': port => '80', docroot => '/srv/www/docroot', } package { ["mysql-client", "php5-cli", "phpunit", "php5- curl", "php5-dev", "php5-gd", "php5-imagick", "php5-mcrypt", "php5-mysql", "php5-xdebug","php5-suhosin", "php-pear", "php-codesniffer" ]: ensure => present, } }
  • 56.
    ~# cat manifests/classes/ci.pp classci inherits basenode { include apache include apache::php exec { "pear_autodiscover": command => "/usr/bin/pear config-set auto_discover 1", } package { ["pear.phpunit.de/PHP_CodeBrowser", "pear.phpunit.de/PHPUnit_MockObject", "pear.phpunit.de/ PHPUnit_Selenium", "pear.phpunit.de/PHP_CodeCoverage", "pear.phpunit.de/PHP_Timer", "pear.phpunit.de/phpcpd", "pear.phpunit.de/phploc"]: ensure => latest, provider => "pear", require => Exec["pear_autodiscover"] } }
  • 57.
    3. Make theconfiguration part of your sourcecode
  • 58.
    • application • data •docs • library • public • scripts • jobs • build • configuration • VagrantFile • manifests • modules • temp
  • 59.
    Ok, that wasa lot of work. Why did i do that again?
  • 60.
    Collection of Wins Failsafety Simplicity Fast Setup Time Repeatability Self-Service Consistency Version Management http://www.flickr.com/photos/turtlemom_nancy/
  • 61.
    There is nogolden Image anymore
  • 62.
    There is justone directory in your source ...
  • 63.
  • 64.
  • 65.
    Fail Safety withVagrant-Snap vagrant snap take -d “snap1“ vagrant snap list vagrant snap go “snap1“
  • 66.
    What if ineed to simulate 20
  • 67.
    McCloud Wrapper likeVagrant aroundFog Transparent local & cloud usage Supports EC2, OpenStack, KVM, etc
  • 68.
    What if ineed to simulate 2000 machines?
  • 69.
    mCollective dssh/ssh-for-loop on steroids fastmanagement for loads of servers uses puppet/facter or chef/ohai, MQ- based
  • 70.
    $ mc-package -W "architecture=x86" status apache * [ ============================================================> ] 10 / 10 host01.example.com version = apache-2.2.9-7 host02.example.com version = apache-2.2.9-7 host03.example.com version = apache-2.2.9-7 host04.example.com version = apache-2.2.9-7 host05.example.com version = apache-2.2.9-7 host06.example.com version = apache-2.2.9-7 host07.example.com version = apache-2.2.9-7 host08.example.com version = apache-2.2.9-7 host09.example.com version = apache-2.2.9-7 host10.example.com version = apache-2.2.9-7 ---- package agent summary ---- Nodes: 10 / 10 Versions: 10 * 0.25.5-1.el5 Elapsed Time: 1.03 s
  • 71.
    The Foreman -Machine Life Cycle Management Acts as a web front end for Puppet Shows You the system inventory Creates new machines and takes care of provisioning
  • 72.
    Knowing everything aboutyour server configuration is great!
  • 73.
  • 74.
  • 75.
  • 76.
  • 77.
    3. Configure Your Vagrant images using chef or puppet
  • 78.
    4. Setup achef- or puppet-server using this configuration
  • 79.
    5. Move yourCI, Staging and Production-Setup to chef/puppet, too
  • 81.
    6. Create aself- service-plattform for the development team
  • 83.
    More cool stuff you can do:
  • 84.
    Test-Driven infrastructure using chef/ puppet-cucumber
  • 85.
  • 86.
  • 87.
    Thanks! Johann-Peter Hartmann johann__ @freenode I hope you enjoyed it! Mail / Jabber: johann-peter.hartmann@mayflower.de Further reading: http://www.planetdevops.net http://dzone.com http://twitter.com/#!/DEVOPS_BORAT http://github.com/johannhartmann/
  • 88.
    Vegetable Test Driven Infrastructure Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 87
  • 89.
    actually a behaviordriven design tool used for test driven infrastructure cucumber-puppet chef-cucumber
  • 90.
    Feature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page Given I go to "http://it-republik.de/php/" When I fill in "search_itr" with "bdd" And I click "search2" Then I should see "Suche"
  • 91.
    Given /^I goto "([^"]*)"$/ do |url| visit url end When /^I fill in "([^"]*)" with "([^"]*)"$/ do |field, value| fill_in field, :with => value end When /^I click "([^"]*)"$/ do |button| click_button(button) end Then /^I should see "([^"]*)"$/ do |text| response_body.should include(text) end
  • 92.
    johann$ cucumber Feature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page # features/ search.feature:5 Given I go to "http://it-republik.de/php/" # features/ step_definitions/search_steps.rb:1 When I fill in "search_itr" with "bdd" # features/ step_definitions/search_steps.rb:5 And I click "search2" # features/ step_definitions/search_steps.rb:9 Then I should see "Suche" # features/ step_definitions/search_steps.rb:13 1 scenario (1 passed) 4 steps (4 passed) 0m1.615s
  • 93.
    Feature: Install inetd In order to serve the web the httpd service must be installed Scenario: Setup httpd Given a node of class “web“ When I compile the catalog Then package “httpd“ should be “installed“ Then there should be a resource “Service[httpd]“ And the service should have “enable“ set to “true“ And State should be “running“ And the service should require “Package[httpd]“ Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 92
  • 95.
    Hey, i understand Operations- and i‘m working in marketing
  • 96.
    Unit Tests Acceptance Tests Metrics CodingStyle ... you get the idea.
  • 97.
    ... and even more... Packaging Infrastructure Updates Releases Deployment
  • 100.
  • 101.
  • 102.
  • 103.
    Eat together http://www.flickr.com/photos/vilavelosa/ 3815032524/
  • 104.
    Plan together Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 103
  • 105.
  • 106.
    Wetware - Mayflower 1-2 Ops per team • Admin & Development tasks • full time working within the team
  • 107.
    Wetware - Mayflower Closecooperation with central operations Full root access for any development-infrastructre
  • 108.
    Wetware - Mayflower 1+nPuppet-Master • central company master • Team puppet master per team / project • company master is starting point for the project puppet configuration
  • 109.
    Wetware - Mayflower ExampleSetup: • local Developer VM • CI-Deployment-Server in the DMZ • Staging in the private Cloud • Beta happens in the Amazon-Cloud • Production in AWS, too
  • 110.
    Wetware - Mayflower localGIT- / Gitorious-Server Eucalyptus-Cloud in the DMZ - as Self-Service!
  • 111.
    Wetware - Mayflower Vagrant for Development Scrum => KanBan Puppet Nagios

Editor's Notes

  • #2 \n
  • #3 \n
  • #4 \n
  • #5 I am johann. a few people already know me. a warm welcome to you, too! I&amp;#x2018;ll leave the boring company stuff out.\n
  • #6 Because you are more interesting.\n
  • #7 Who is a developer? \nWho is a administrator? Should you wear a beard?\nWho is neither a developer nor a system administrator? \nHey, nice, what are you? And why are you here? \n\n
  • #8 Let&amp;#x2018;s see what the development background is. How many of you are php developers? You can tell the truth, i am a PHP developer, too. sounds a bit like the alcoholics anonymous - &amp;#x201E;Hi, i am johann, and i am a PHP developer&amp;#x201C;.\nA java developer? Are there still jobs around for java developers? \n
  • #9 \n
  • #10 \n
  • #11 Obviously You should do it because it&amp;#x2018;s cool. DevOps is much of a hype right now, and you can be part of it! It&amp;#x2018;s like the Google Wave developer hype without the disappointment later\n
  • #12 The truth is a lot more boring - it&amp;#x2018;s because we need it. Let me tell you a story about the dark age.\n
  • #13 Do You remember the dark ages of development? How did development happen that days? \n(by the way: he does look a bit like benjamin eberlei, doesn&amp;#x2018;t he?)\n
  • #14 We used an basic vmware image. it was downloadable at some local fileserver, several gigabytes big and everything you needed for development was already installed. \n
  • #15 This golden image always looked good in the beginning, but your application started to change. stuff was added, some kind of improvements were made. a default database was supplied, too. there was a default user used by everybody. \n
  • #16 - but changes were needed - bash scripts to change the configuration\n- database update scripts - versioned database update scripts\n- a lot of bugs were solved by &amp;#x201E;you need to run the update script&amp;#x201C;\n- and the same amount of bugs were created by running the update script. \n- from time to time a new golden image was needed and some of the devs used it.\n
  • #17 - after a while every developer had his own improved version of the image\n- incompatible, different versions, only the local version management sandbox was up to date\n\n
  • #18 \n
  • #19 \n
  • #20 And in the good old days our application infrastructure was simple 3-tier\nweb server and database server were happening on one host.\n
  • #21 Suddenly we had to add stuff. Like a hip NoSQL Server\n
  • #22 And a memcache server, for Caching.\n
  • #23 memcache became unhip, so it was replaced by redis\nan asynchronous messagequeue like gearman was introduced\n
  • #24 Gearman wasn&amp;#x2018;t so enterprisey in the end, so it was replaced by ActiveMQ. \nAn eJabberD was introduced for browser-side pubsub.\n
  • #25 And actually it was 4 Servers now. \n
  • #26 Ending up in 4 different bash scripted setup routines based on a set of 3 golden images.\n
  • #27 On the other hand side, there wasn&amp;#x2018;t just development, there was continious integration and production as well. sometimes with a different deployment mechanism.\n
  • #28 And there were different Versions deployed, anyway. \n
  • #29 With different tools, and software versions to work with different tools. Your application version happens in your version management system, your configuration in some adminstrators bash script. both are not in sync. \n
  • #30 This wasn&amp;#x2018;t any fun anymore. the number of wtf/minute was constantly increasing. We did not like it a lot.\n
  • #31 10th floor test: throw a random computer out of the windows and wait how long it takes everything is up &amp; running again. We did not actually do it, since we are in a 5 store building. If your building is higher, try it out, it&amp;#x2018;s a good benchmark.\n
  • #32 That&amp;#x2018;s our collection of fails. No simplicity, no failsafety - if a configuration is screwed it&amp;#x2018;s screwed. \n
  • #33 But how do we get there?\n
  • #34 DevOps for the win!\n
  • #35 (Danger: Code ahead)\n
  • #36 (Danger: Code ahead). It works good on any linux, bsd etc. including Mac. Windows, especially with 64 bits is a bit hard to do, you have to use jruby. \n
  • #37 With a cool logo!\n
  • #38 First install vagrant and veewee. this is done using the default ruby gem install. \nlist baseboxes, choose yours and use it as your default box.\n\n
  • #39 \n
  • #40 \n
  • #41 \n
  • #42 First thing to know: configuration is code. it&amp;#x2018;s not a setup anymore. \n
  • #43 That are the two main players. like linux and freebsd, like gnome and kde everything opensource gets better when there are two of a kind. Does anyone still know cfengine?\n
  • #44 On first sight chef and puppet look like twins. (Those are my sons, btw, sorry to show you, you know how proud parents are :-) )\n\n
  • #45 \n
  • #46 \n
  • #47 \n
  • #48 There are several tests and comparisons available online. half of the time puppet wins, half of the time chef does. there is no winner. have a look at it and take the tool you like. if you are an experenced ruby developer, chef is the better choice, if not, puppet can be. \n
  • #49 That&amp;#x2018;s how the puppet DSL looks like. You&amp;#x2018;ll see some more examples later.\n
  • #50 And that&amp;#x2018;s how chef syntax looks like. The difference is:\nThis is ruby code. You have the full flexibility of the language available.\n
  • #51 \n
  • #52 That is the first Vagrantfile generated by vagrant init\n
  • #53 Here we are talking vagrant. \n- puppet as a machine provisioner, with a link to the puppet directory and the default manifest for this machine - and more machines are possible\n- name of the server, network configuration, port forwarding and mount points.\n\n
  • #54 This is the puppet configuration file for my nodes ( servers). i can include directores, and i can include other classes in my classes. \n
  • #55 That&amp;#x2018;s the included definition of the web class. see, there is inheritance.\nThe apache-include is a puppet module and provides for example the vhost configuration\nThe package is a resource wrapper for apt here, since this is an ubuntu natty setup.\n
  • #56 This is an example for a custom package provider for pear packages.\n
  • #57 \n
  • #58 This is an example based on the zend framework default directory layout. Two parts are going to change - the configs will contain the server setup as well, and there is a new vms folder within the scripts directory, containing a Vagrantfile. let&amp;#x2018;s cd into scripts/configuration and start to work\n
  • #59 \n
  • #60 That&amp;#x2018;s our collection of fails. No simplicity, no failsafety - if a configuration is screwed it&amp;#x2018;s screwed. \n
  • #61 \n
  • #62 \n
  • #63 And all the developer has to do is a vagrant up to get his vms from the source\nif there have been configuration changes just do a vagrant provision\nno more. \nNO NEED TO SAVE YOUR VM ANYMORE!\n
  • #64 \n
  • #65 this is an additional module for vagrant to give you a chance to screw your vms.\n
  • #66 \n
  • #67 \n
  • #68 \n
  • #69 \n
  • #70 \n
  • #71 Right now libvirt-based, in future ec2-support is going to happen\n
  • #72 \n
  • #73 That&amp;#x2018;s how you create a new machine. Or your developers do in self service. \nDo you remember how long this took before? \n\n
  • #74 \n
  • #75 \n
  • #76 \n
  • #77 \n
  • #78 \n
  • #79 \n
  • #80 \n
  • #81 \n
  • #82 \n
  • #83 \n
  • #84 \n
  • #85 \n
  • #86 \n
  • #87 \n
  • #88 \n
  • #89 \n
  • #90 \n
  • #91 \n
  • #92 \n
  • #93 \n
  • #94 Danke an Jimdo f&amp;#xFC;r das Beispiel.. \n
  • #95 \n
  • #96 Wer setzt Jenkins ein? (sonst erkl&amp;#xE4;ren)\n
  • #97 \n
  • #98 Sebastian wird hier&amp;#xFC;ber noch mehr erz&amp;#xE4;hlen. \n
  • #99 \n
  • #100 \n
  • #101 - gemeinsame Standups\n- gegenseitige Teilnahme an den Sprint Plannings &amp; Retros\n- gleiche R&amp;#xE4;ume, wenn m&amp;#xF6;glich\n
  • #102 Der Code geh&amp;#xF6;rt auch den Admins, die Konfiguration und die Verl&amp;#xE4;sslichkeit auch den Developern.\n
  • #103 Wie bekommt man Respekt hin?\n- Soziale Interaktion, Feiern, Teambuilding\nWenn ich jemand pers&amp;#xF6;nliche kenne nehme ich auf seine Interessen R&amp;#xFC;cksicht\n
  • #104 Die langfristige Planung wird gemeinsam gemacht. Es werden gemeinsame Ziele definiert, und die L&amp;#xF6;sungsstrategien gemeinsam erstellt.\n
  • #105 \n
  • #106 \n
  • #107 \n
  • #108 \n
  • #109 \n
  • #110 \n
  • #111 \n