Virtualize and automate your development
environment for fun and profit



Andreas Heim - May 23 2011 – Roots Conference, Bergen
Vagrant

Virtualize and automate your development
      environment for fun and profit!



          Andreas Heim - BEKK Consulting
                      @heim
What defines a good
development environment?
What defines a good
development environment?

• Identical to production
What defines a good
development environment?

• Identical to production

• Identical for all developers
What defines a good
development environment?

• Identical to production

• Identical for all developers

• Independent of other systems
What defines a good
development environment?

• Identical to production

• Identical for all developers

• Independent of other systems

• You should be able to work on a plane
Not this plane
Not this plane

    (it has wi-fi)
What defines a good
development environment II
What defines a good
development environment II

• Lucid
What defines a good
development environment II

• Lucid

• Well documented
What defines a good
development environment II

• Lucid

• Well documented

• Fast on-boarding of new team members
What defines a good
development environment II

• Lucid

• Well documented

• Fast on-boarding of new team members

• Versioned
challenges
team members might develop on two different operating
                     systems
challenges
team members might develop on two different operating
                     systems



                  and deploy to a third
challenges
developers might have different versions of dependencies
challenges
developers might have different versions of dependencies
                database
challenges
developers might have different versions of dependencies
                database
                     build tools
challenges
developers might have different versions of dependencies
                database
                     build tools
                          app server
challenges
developers might not have taken the time to install
challenges
developers might not have taken the time to install
        enterprise database
challenges
developers might not have taken the time to install
        enterprise database

                 enterprise app server
challenges
  developers might not have taken the time to install
          enterprise database

                   enterprise app server


because normally “enterprise” means slow
            and complex
contamination

if you got multiple applications accessing the same
            database or central service...
contamination

 if you got multiple applications accessing the same
             database or central service...


how quick can you reset your development environment?
contamination

 if you got multiple applications accessing the same
             database or central service...


how quick can you reset your development environment?


  can one application contaminate the data of the other?
automate and virtualize
distribute
virtual machine that contains all of the projects dependencies
distribute
virtual machine that contains all of the projects dependencies



     app server, database, stubbed external services
so, what are we actually talking about?
developer 1                          developer 2

OS: snow leopard                      OS: Windows 7
some db v. 1.02                       some db v 1.03
app server. v. 9.2.1                 app server. v. 9.3.1
 apple java 1.6                       sun java 1.6


                       production machine
                       OS: Ubuntu

                   “enterprise db”
                   “scalable app server”
                       openjdk 1.5
script the installation of this machine
virtual linux macine
    enterprise db
enterprise app server
correct java version
virtualized environment inside your machine
virtualized environment inside your machine



         develop using your existing tools
virtualized environment inside your machine



         develop using your existing tools

           run your code on the virtual machine
developer 1                    developer 2

           editor                          editor



 Virtual   shared                Virtual   shared
machine     folder              machine     folder




                           production machine
script the setup of this   OS: Ubuntu
        machine
vagrant
   vagrantup.com
ruby application
(ruby power)

          built on top of
ruby application
(ruby power)

          built on top of
 • sun virtualbox
ruby application
(ruby power)

          built on top of
 • sun virtualbox

 • chef
ruby application
(ruby power)

          built on top of
 • sun virtualbox

 • chef

 • puppet
vagrant lets you
vagrant lets you
      automate creation
       and provisioning
      of virtual machines
replicate
replicate
       and
             rebuild
replicate
       and
             rebuild
              instantly!
built on top of rake
      easy to extend
create your virtual machine with
         $ vagrant up
create your virtual machine with
         $ vagrant up


       shut it down with
        $ vagrant halt
create your virtual machine with
         $ vagrant up


       shut it down with
        $ vagrant halt


        destroy it with
      $ vagrant destroy
and rebuild it instantly with
      $ vagrant up
easy configuration

   Vagrant::Config.run do |config|
     config.vm.box = "base"
     config.vm.box_url = "http://files.vagrantup.com/lucid32.box"
   end




     lots of boxes on http://vagrantbox.es
port forwarding

    Vagrant::Config.run do |config|
      config.vm.forward_port "http", 80, 8080
    end




forwards port 8080 on the host os to port 80 on the vm
multiple vms


Vagrant::Config.run do |config|
  config.vm.define :web do |web_config|
    web_config.vm.box = "web"
    web_config.vm.forward_port("http", 80, 8080)
  end

  config.vm.define :db do |db_config|
    db_config.vm.box = "db"
    db_config.vm.forward_port("db", 3306, 3306)
  end
end
provisioning
provisioning


the act of preparing and equipping a VM to run
                your application
provisioning


the act of preparing and equipping a VM to run
                your application




  in practice, installing software and configuring
                     the machine
alternatives

 chef




puppet
alternatives
               pure ruby, flexible, big ecosystem
 chef




puppet
alternatives
               pure ruby, flexible, big ecosystem
 chef
               complicated, lots of files




puppet
alternatives
               pure ruby, flexible, big ecosystem
 chef
               complicated, lots of files




          supported by google, flexible
puppet
alternatives
               pure ruby, flexible, big ecosystem
 chef
               complicated, lots of files




          supported by google, flexible
puppet
                 hard to get started
or use sprinkle, for simplicity
or use sprinkle, for simplicity

   sprinkle is a really nice ruby dsl
                                    github.com/crafterm/sprinkle
or use sprinkle, for simplicity

    sprinkle is a really nice ruby dsl
                                            github.com/crafterm/sprinkle

package :git, :provides => :scm do
  description 'Git Distributed Version Control'
  apt "git-core"
   verify do
    has_executable "git"
  end
end




find this code on http://github.com/heim/vagrant-java-example
and roll your own provisioner
class SprinkleProvisioner < Vagrant::Provisioners::Base
  def prepare
    
  end

  def provision!
    vm.ssh.execute do |ssh|
     ssh.exec!('gem list | grep "i18n (0.5.0)" ;if [ $? == "1" ]; then sudo gem install i18n --version "0.5.0"; fi;')
     ssh.exec!('gem list | grep "sprinkle (0.3.3)" ;if [ $? == "1" ]; then sudo gem install sprinkle --version "0.3.3"; fi;')
     ssh.exec!("sudo sprinkle -v -c -s /vagrant/sprinkle/install.rb")
    end
  end
end




       find this code on http://github.com/heim/vagrant-java-example
config your provisioner for use with vagrant
config your provisioner for use with vagrant


   Vagrant::Config.run do |config|
     config.vm.provision SprinkleProvisioner
   end
config your provisioner for use with vagrant


   Vagrant::Config.run do |config|
     config.vm.provision SprinkleProvisioner
   end



   Vagrant::Config.run do |config|
     config.vm.provision :chef_solo
   end
config your provisioner for use with vagrant


   Vagrant::Config.run do |config|
     config.vm.provision SprinkleProvisioner
   end



   Vagrant::Config.run do |config|
     config.vm.provision :chef_solo
   end


  Vagrant::Config.run do |config|
    config.vm.provision :shell, :path => "test.sh"
  end
but, why?
run code on production like environment
run code on production like environment
    instant feedback
run code on production like environment
    instant feedback
        catches bugs early
run code on production like environment
    instant feedback
        catches bugs early
            even before they hit scm
or if your setup is complicated
or if your setup is complicated
  do it quick
or if your setup is complicated
  do it quick
    reset data quickly
or if your setup is complicated
  do it quick
    reset data quickly
      everything in one package
or if your setup is complicated
   do it quick
     reset data quickly
       everything in one package
and you’re probably better off using sprinkle
for teams
for teams

  identical environments
for teams

  identical environments

     single responsibility principle
for teams

  identical environments

     single responsibility principle

      fast onboarding of new team members
for teams

  identical environments

     single responsibility principle

      fast onboarding of new team members

             increase in maintainability
for operations   (or devops)
for operations          (or devops)

 test your provisioning scripts
for operations          (or devops)

 test your provisioning scripts

   test your deploy scripts
for operations          (or devops)

 test your provisioning scripts

   test your deploy scripts

     test your clustering setup with multi-vms
for operations          (or devops)

 test your provisioning scripts

   test your deploy scripts

     test your clustering setup with multi-vms

        test load balancing and fallback mechanisms
for operations          (or devops)

 test your provisioning scripts

   test your deploy scripts

     test your clustering setup with multi-vms

        test load balancing and fallback mechanisms




       (this can even be part of your CI-run)
use vagrant if you
use vagrant if you
 deploy to the cloud
use vagrant if you
 deploy to the cloud

    already use chef or puppet
use vagrant if you
  deploy to the cloud

     already use chef or puppet


 have a large amount of dependencies, and a complicated
                   environment setup
your development environment is documented
your development environment is documented
    because it is declarative
your development environment is documented
    because it is declarative
       and it is in the source repository
your development environment is documented
     because it is declarative
         and it is in the source repository

it is easy to replicate
your development environment is documented
     because it is declarative
         and it is in the source repository

it is easy to replicate

    because its automated
your development environment is documented
     because it is declarative
         and it is in the source repository

it is easy to replicate

    because its automated
it is independent of other systems
your development environment is documented
     because it is declarative
         and it is in the source repository

it is easy to replicate

    because its automated
it is independent of other systems
    because its virtualized
thanks!



spkr8.com/heim
   @heim
resources:
http://vagrantup.com
https://github.com/heim/vagrant-java-example
https://github.com/crafterm/sprinkle
http://vagrantbox.es
ANDREAS HEIM
                          CONSULTANT
                          +47 959 39 833
                      andreas.heim@bekk.no



                             BEKK CONSULTING AS
SKUR 39, VIPPETANGEN. P.O. BOX 134 SENTRUM, 0102 OSLO, NORWAY. WWW.BEKK.NO

Virtualize and automate your development environment for fun and profit

  • 1.
    Virtualize and automateyour development environment for fun and profit Andreas Heim - May 23 2011 – Roots Conference, Bergen
  • 2.
    Vagrant Virtualize and automateyour development environment for fun and profit! Andreas Heim - BEKK Consulting @heim
  • 3.
    What defines agood development environment?
  • 4.
    What defines agood development environment? • Identical to production
  • 5.
    What defines agood development environment? • Identical to production • Identical for all developers
  • 6.
    What defines agood development environment? • Identical to production • Identical for all developers • Independent of other systems
  • 7.
    What defines agood development environment? • Identical to production • Identical for all developers • Independent of other systems • You should be able to work on a plane
  • 8.
  • 9.
    Not this plane (it has wi-fi)
  • 10.
    What defines agood development environment II
  • 11.
    What defines agood development environment II • Lucid
  • 12.
    What defines agood development environment II • Lucid • Well documented
  • 13.
    What defines agood development environment II • Lucid • Well documented • Fast on-boarding of new team members
  • 14.
    What defines agood development environment II • Lucid • Well documented • Fast on-boarding of new team members • Versioned
  • 15.
    challenges team members mightdevelop on two different operating systems
  • 16.
    challenges team members mightdevelop on two different operating systems and deploy to a third
  • 17.
    challenges developers might havedifferent versions of dependencies
  • 18.
    challenges developers might havedifferent versions of dependencies database
  • 19.
    challenges developers might havedifferent versions of dependencies database build tools
  • 20.
    challenges developers might havedifferent versions of dependencies database build tools app server
  • 21.
    challenges developers might nothave taken the time to install
  • 22.
    challenges developers might nothave taken the time to install enterprise database
  • 23.
    challenges developers might nothave taken the time to install enterprise database enterprise app server
  • 24.
    challenges developersmight not have taken the time to install enterprise database enterprise app server because normally “enterprise” means slow and complex
  • 25.
    contamination if you gotmultiple applications accessing the same database or central service...
  • 26.
    contamination if yougot multiple applications accessing the same database or central service... how quick can you reset your development environment?
  • 27.
    contamination if yougot multiple applications accessing the same database or central service... how quick can you reset your development environment? can one application contaminate the data of the other?
  • 28.
  • 29.
    distribute virtual machine thatcontains all of the projects dependencies
  • 30.
    distribute virtual machine thatcontains all of the projects dependencies app server, database, stubbed external services
  • 31.
    so, what arewe actually talking about?
  • 32.
    developer 1 developer 2 OS: snow leopard OS: Windows 7 some db v. 1.02 some db v 1.03 app server. v. 9.2.1 app server. v. 9.3.1 apple java 1.6 sun java 1.6 production machine OS: Ubuntu “enterprise db” “scalable app server” openjdk 1.5
  • 33.
    script the installationof this machine virtual linux macine enterprise db enterprise app server correct java version
  • 34.
  • 35.
    virtualized environment insideyour machine develop using your existing tools
  • 36.
    virtualized environment insideyour machine develop using your existing tools run your code on the virtual machine
  • 37.
    developer 1 developer 2 editor editor Virtual shared Virtual shared machine folder machine folder production machine script the setup of this OS: Ubuntu machine
  • 38.
    vagrant vagrantup.com
  • 39.
  • 40.
    ruby application (ruby power) built on top of • sun virtualbox
  • 41.
    ruby application (ruby power) built on top of • sun virtualbox • chef
  • 42.
    ruby application (ruby power) built on top of • sun virtualbox • chef • puppet
  • 43.
  • 44.
    vagrant lets you automate creation and provisioning of virtual machines
  • 45.
  • 46.
    replicate and rebuild
  • 47.
    replicate and rebuild instantly!
  • 48.
    built on topof rake easy to extend
  • 49.
    create your virtualmachine with $ vagrant up
  • 50.
    create your virtualmachine with $ vagrant up shut it down with $ vagrant halt
  • 51.
    create your virtualmachine with $ vagrant up shut it down with $ vagrant halt destroy it with $ vagrant destroy
  • 52.
    and rebuild itinstantly with $ vagrant up
  • 53.
    easy configuration Vagrant::Config.run do |config|   config.vm.box = "base"   config.vm.box_url = "http://files.vagrantup.com/lucid32.box" end lots of boxes on http://vagrantbox.es
  • 54.
    port forwarding Vagrant::Config.run do |config|   config.vm.forward_port "http", 80, 8080 end forwards port 8080 on the host os to port 80 on the vm
  • 55.
    multiple vms Vagrant::Config.run do|config| config.vm.define :web do |web_config| web_config.vm.box = "web" web_config.vm.forward_port("http", 80, 8080) end config.vm.define :db do |db_config| db_config.vm.box = "db" db_config.vm.forward_port("db", 3306, 3306) end end
  • 56.
  • 57.
    provisioning the act ofpreparing and equipping a VM to run your application
  • 58.
    provisioning the act ofpreparing and equipping a VM to run your application in practice, installing software and configuring the machine
  • 59.
  • 60.
    alternatives pure ruby, flexible, big ecosystem chef puppet
  • 61.
    alternatives pure ruby, flexible, big ecosystem chef complicated, lots of files puppet
  • 62.
    alternatives pure ruby, flexible, big ecosystem chef complicated, lots of files supported by google, flexible puppet
  • 63.
    alternatives pure ruby, flexible, big ecosystem chef complicated, lots of files supported by google, flexible puppet hard to get started
  • 64.
    or use sprinkle,for simplicity
  • 65.
    or use sprinkle,for simplicity sprinkle is a really nice ruby dsl github.com/crafterm/sprinkle
  • 66.
    or use sprinkle,for simplicity sprinkle is a really nice ruby dsl github.com/crafterm/sprinkle package :git, :provides => :scm do   description 'Git Distributed Version Control'   apt "git-core"    verify do     has_executable "git"   end end find this code on http://github.com/heim/vagrant-java-example
  • 67.
    and roll yourown provisioner class SprinkleProvisioner < Vagrant::Provisioners::Base   def prepare        end   def provision!     vm.ssh.execute do |ssh|      ssh.exec!('gem list | grep "i18n (0.5.0)" ;if [ $? == "1" ]; then sudo gem install i18n --version "0.5.0"; fi;')      ssh.exec!('gem list | grep "sprinkle (0.3.3)" ;if [ $? == "1" ]; then sudo gem install sprinkle --version "0.3.3"; fi;')      ssh.exec!("sudo sprinkle -v -c -s /vagrant/sprinkle/install.rb")     end   end end find this code on http://github.com/heim/vagrant-java-example
  • 68.
    config your provisionerfor use with vagrant
  • 69.
    config your provisionerfor use with vagrant Vagrant::Config.run do |config|   config.vm.provision SprinkleProvisioner end
  • 70.
    config your provisionerfor use with vagrant Vagrant::Config.run do |config|   config.vm.provision SprinkleProvisioner end Vagrant::Config.run do |config|   config.vm.provision :chef_solo end
  • 71.
    config your provisionerfor use with vagrant Vagrant::Config.run do |config|   config.vm.provision SprinkleProvisioner end Vagrant::Config.run do |config|   config.vm.provision :chef_solo end Vagrant::Config.run do |config| config.vm.provision :shell, :path => "test.sh" end
  • 72.
  • 73.
    run code onproduction like environment
  • 74.
    run code onproduction like environment instant feedback
  • 75.
    run code onproduction like environment instant feedback catches bugs early
  • 76.
    run code onproduction like environment instant feedback catches bugs early even before they hit scm
  • 77.
    or if yoursetup is complicated
  • 78.
    or if yoursetup is complicated do it quick
  • 79.
    or if yoursetup is complicated do it quick reset data quickly
  • 80.
    or if yoursetup is complicated do it quick reset data quickly everything in one package
  • 81.
    or if yoursetup is complicated do it quick reset data quickly everything in one package and you’re probably better off using sprinkle
  • 82.
  • 83.
    for teams identical environments
  • 84.
    for teams identical environments single responsibility principle
  • 85.
    for teams identical environments single responsibility principle fast onboarding of new team members
  • 86.
    for teams identical environments single responsibility principle fast onboarding of new team members increase in maintainability
  • 87.
    for operations (or devops)
  • 88.
    for operations (or devops) test your provisioning scripts
  • 89.
    for operations (or devops) test your provisioning scripts test your deploy scripts
  • 90.
    for operations (or devops) test your provisioning scripts test your deploy scripts test your clustering setup with multi-vms
  • 91.
    for operations (or devops) test your provisioning scripts test your deploy scripts test your clustering setup with multi-vms test load balancing and fallback mechanisms
  • 92.
    for operations (or devops) test your provisioning scripts test your deploy scripts test your clustering setup with multi-vms test load balancing and fallback mechanisms (this can even be part of your CI-run)
  • 93.
  • 94.
    use vagrant ifyou deploy to the cloud
  • 95.
    use vagrant ifyou deploy to the cloud already use chef or puppet
  • 96.
    use vagrant ifyou deploy to the cloud already use chef or puppet have a large amount of dependencies, and a complicated environment setup
  • 98.
  • 99.
    your development environmentis documented because it is declarative
  • 100.
    your development environmentis documented because it is declarative and it is in the source repository
  • 101.
    your development environmentis documented because it is declarative and it is in the source repository it is easy to replicate
  • 102.
    your development environmentis documented because it is declarative and it is in the source repository it is easy to replicate because its automated
  • 103.
    your development environmentis documented because it is declarative and it is in the source repository it is easy to replicate because its automated it is independent of other systems
  • 104.
    your development environmentis documented because it is declarative and it is in the source repository it is easy to replicate because its automated it is independent of other systems because its virtualized
  • 105.
  • 106.
  • 107.
    ANDREAS HEIM CONSULTANT +47 959 39 833 andreas.heim@bekk.no BEKK CONSULTING AS SKUR 39, VIPPETANGEN. P.O. BOX 134 SENTRUM, 0102 OSLO, NORWAY. WWW.BEKK.NO