Prepared For: Acquia                                            October 4, 2012                             Title: Harmoni...
Achieve Internet Overview   7+ years developing media and entertainment web    properties Over 60,000 development hours ...
Key Clients: Media & Entertainment             Media
Key Clients: Other      Other
Pop Quiz<?php/** * Transform upper camel case to lower camel case. * * @param string $word * Upper camel case word. * * @r...
Agenda+        =
Agenda (Cont.)   What’s all the fuss?   What is Virtualization?   Vagrant – Creating (and destroying) environments on t...
What We Are Not Covering Performance Tuning  These are examples only Version Control  All provisioning can be placed i...
What’s all the fuss?“Write once, run anywhere” -- Sun Microsystems“Write Once, Debug Everywhere” -- Anonymous / Reality“In...
The Good Old Days
Today….
How It ComparesOld Approach                      Old Approach… Today
The Drupal Ecosystem
Site Building Roles
How It All Fits     Virtual Box                         Vagrant(Virtual Environment)           (Virtual Machine Builder)  ...
Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/)...
Virtual Box     Virtual Box                                          Vagrant(Virtual Environment)                         ...
What is Virtualization? Technical Definition…. A logical representation of a computer in software. By decoupling the phy...
VirtualBox Open Source virtualization software for Windows, Mac, and Linux Can host multiple operating systems at once (...
Vagrant     Virtual Box                                       Vagrant(Virtual Environment)                              (V...
Vagrant Manages your virtual machines Works with VirtualBox Extremely customizable Over 50+ pre-built base boxes Free!
Why Vagrant? Creates consistent, reproducible environments Rapid setup – Start development in a few quick steps Reduce ...
Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the...
Using VagrantObjective:Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM)One Time Steps:List available boxes:$ vagrant ...
Using VagrantLet’s Rock!!Start up a virtual machine:$ vagrant upDelete a virtual machine:$ vagrant removeConnect to virtua...
Using VagrantOther Useful CommandsGo to lunch:$ vagrant suspend # Save state of virtual machine$ vagrant resume # Start vi...
Example 1$ mkdir <vm_dir>/myproject$ vagrant init hardyCustomize our VM Instance (Excerpts of Vagrantfile)config.vm.networ...
Example 1Scenario: Admin for a DayWonder what this does… (WARNING)*$ sudo chmod 444 /etc/sudoers* Warning, doing this on a...
Vagrant++ Instant development environments! No physical hardware required. Creates a safe environment for configuration...
Puppet     Virtual Box                                      Vagrant(Virtual Environment)                             (Virt...
Puppet   Provisioning = preparing a device/server for usage, aka configuration   Allows enforceable system configuration...
Puppet: Miscellaneous InfoFacter: Inspects operating system “facts” that can be used in modules.$ facter… id => vagrantint...
Puppet: Miscellaneous Info (Cont.)Describes a resource, use -s to summarize$ puppet describe <resource>Inspect individual ...
Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the...
Puppet: Important NotesCommon Pattern:Package, File, ServiceResources are processed asynchronously! Use dependencyrelation...
Puppet Main Manifestmyproject.ppclass myproject {                                     Site specific          ensure => abs...
Example 2: Puppet Module/modules/apache/init.pp (Header)# Class: apache                                                   ...
Puppet Sample Module/modules/apache/init.pp (Code)class apache {                                                          ...
Puppet Sample Module/modules/apache/init.pp (Code) # Define new type: mod define mod ($ensure = present) {                ...
Puppet Sample ModuleERB Templates: Create files on the fly./modules/apache/templates/site.conf.debian.erb# Puppet generate...
Puppet++ Construct fully functioning development environments on the fly! Bring new team members up to speed in a few si...
Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the...
Veewee     Virtual Box                                      Vagrant(Virtual Environment)                             (Virt...
Next Steps: VeeweeWant your own custom Vagrant Box or one doesn’t exist?Veewee has predefined definitions for creating a v...
Next Steps: VeeweeVeewee Installation (Mac OS X)$ rvm install 1.9.2$ git clone https://github.com/jedi4ever/veewee.git$ cd...
Next Steps: VeeweeVeewee Base Box Creation (Mac OS X)In veewee directory:- Copy new iso into [currentDir]/iso   (optional,...
Next Steps: Puppet Enterprise Allow for remote management of puppet nodes.  “One server to rule them all.” Clone your c...
Gotchas Be mindful of Puppets asyncronous execution order:  Make sure dependecies are properly set. Starbucks syndrome:...
Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/)...
Further Resources Prebulit Vagrant Base Boxes (http://www.vagrantbox.es/) Puppet Forge (http://forge.puppetlabs.com/) L...
Conclusion   …makes the "works on my machine" excuse a relic of the past. -- Vagrant   Created a disposable test bed to ...
Achieve Internet Pillars of Expertise                  Technical Foundation Built On the                  Following Core A...
QA
Thank You        Michael Lee       Senior Architectmichael.lee@achieveinternet.com    www.achieveinternet.com           ml...
Upcoming SlideShare
Loading in...5
×

Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

2,377

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,377
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

  1. 1. Prepared For: Acquia October 4, 2012 Title: Harmonious Development:Standardizing The Deployment Process via Vagrant and Puppet Achieve – Quality on Time 1
  2. 2. Achieve Internet Overview 7+ years developing media and entertainment web properties Over 60,000 development hours developing Drupal web & mobile-optimized solutions Deep expertise in UX (theming), Globalization, Integration, and Performance Company commitment to quality, transparency, continuity, reliability and client experience
  3. 3. Key Clients: Media & Entertainment Media
  4. 4. Key Clients: Other Other
  5. 5. Pop Quiz<?php/** * Transform upper camel case to lower camel case. * * @param string $word * Upper camel case word. * * @return string * $word with first letter in lower case. * * @see http://www.php.net/manual/en/function.lcfirst.php */function my_module_camel_case($word) { return lcfirst($word);}echo my_module_camel_case(HelloWorld);
  6. 6. Agenda+ =
  7. 7. Agenda (Cont.) What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly Provisioning (via Puppet) Examples Future Directions Q&A
  8. 8. What We Are Not Covering Performance Tuning  These are examples only Version Control  All provisioning can be placed into VCS Puppet vs. Chef
  9. 9. What’s all the fuss?“Write once, run anywhere” -- Sun Microsystems“Write Once, Debug Everywhere” -- Anonymous / Reality“In many distributed computing environments, failures arereported in a way that violates even the simplest notions ofconsistency.” -- IEEE Software, Bradford Glade. K. Birman
  10. 10. The Good Old Days
  11. 11. Today….
  12. 12. How It ComparesOld Approach Old Approach… Today
  13. 13. The Drupal Ecosystem
  14. 14. Site Building Roles
  15. 15. How It All Fits Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  16. 16. Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/) Puppet v2.7.19 Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee) Ubuntu v8.04 Hardy Heron
  17. 17. Virtual Box Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  18. 18. What is Virtualization? Technical Definition…. A logical representation of a computer in software. By decoupling the physical hardware from the operating system, virtualization provides more operational flexibility and increases the utilization rate of the underlying physical hardware.1 In a Nutshell…. Putting a small computer [guest/virtual machine] (operating system, memory, video display, peripherals, etc.) into the confines of another bigger physical system [host].
  19. 19. VirtualBox Open Source virtualization software for Windows, Mac, and Linux Can host multiple operating systems at once (within memory and CPU resources) Free!
  20. 20. Vagrant Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  21. 21. Vagrant Manages your virtual machines Works with VirtualBox Extremely customizable Over 50+ pre-built base boxes Free!
  22. 22. Why Vagrant? Creates consistent, reproducible environments Rapid setup – Start development in a few quick steps Reduce duplicated effort Increased reliability Allows you to tinker around, more on this later… Cost savings!
  23. 23. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  24. 24. Using VagrantObjective:Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM)One Time Steps:List available boxes:$ vagrant box listAdd a new Vagrant base box to inventory if not available:(@see http://www.vagrantbox.es/)Usage: vagrant box add <alias> <location>$ vagrant box add hardy /path/to/hardy.boxCreate a directory for your Vagrant VMs$ mkdir <vm_dir>/myprojectCreate a Vagrant file (do this in <vm_dir>/myproject)Usage: vagrant init <alias>$ vagrant init hardy
  25. 25. Using VagrantLet’s Rock!!Start up a virtual machine:$ vagrant upDelete a virtual machine:$ vagrant removeConnect to virtual machine:$ vagrant sshSuccess! Instant VM. Do the happy dance!
  26. 26. Using VagrantOther Useful CommandsGo to lunch:$ vagrant suspend # Save state of virtual machine$ vagrant resume # Start virtual machine up againReboot:$ vagrant halt # Shutdown the VM$ vagrant up –no-provision # Restart VM, skip provisioning** = Online documentation says no reprovisioning will occur…Reload configuration (automatically does a provisioning)$ vagrant reloadReprovision a system:$ vagrant reprovisionRepackage running system:$ vagrant package
  27. 27. Example 1$ mkdir <vm_dir>/myproject$ vagrant init hardyCustomize our VM Instance (Excerpts of Vagrantfile)config.vm.network :hostonly, "192.168.33.10”# Host: <vm_dir>/<project_name> == Guest: /vagrantconfig.vm.share_folder "v-root", "/vagrant", ".", :owner => www-data, :group => www-data’# Configure the VM with puppet.config.vm.provision :puppet, :module_path => "modules", :options => ["--environment", "local"] do |puppet| puppet.manifests_path = "manifests” puppet.manifest_file = ”myproject.pp”End$ vagrant up
  28. 28. Example 1Scenario: Admin for a DayWonder what this does… (WARNING)*$ sudo chmod 444 /etc/sudoers* Warning, doing this on a Production system may have an adverse effect onyour career. This will remove any ability to sudo.Solution (Assuming all configuration is provisioned):$ vagrant destroy$ vagrant up
  29. 29. Vagrant++ Instant development environments! No physical hardware required. Creates a safe environment for configuration optimization.
  30. 30. Puppet Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  31. 31. Puppet Provisioning = preparing a device/server for usage, aka configuration Allows enforceable system configuration Similar to Microsoft’s System Center Configuration Manager/SMS or Apple’s Profile Manager Uses a Resource Abstraction Layer (RAL) to interact with the host. Core Resource Types include: notify, file, package, service, exec, cron, user, group RAL can be used to read and modify resources Configurations can be store in manifests. See resources for prebuilt modules.
  32. 32. Puppet: Miscellaneous InfoFacter: Inspects operating system “facts” that can be used in modules.$ facter… id => vagrantinterfaces => eth0,eth1,loipaddress => 10.0.2.15ipaddress_eth0 => 10.0.2.15ipaddress_eth1 => 192.168.33.10ipaddress_lo => 127.0.0.1is_virtual => truekernel => Linuxkernelmajversion => 2.6kernelrelease => 2.6.24-26-serverkernelversion => 2.6.24lsbdistcodename => hardylsbdistdescription => Ubuntu 8.04.4 LTSlsbdistid => Ubuntulsbdistrelease => 8.04lsbmajdistrelease => 8memoryfree => 1.85 GBmemorysize => 1.98 GBmemorytotal => 1.98 GBnetmask => 255.255.255.0netmask_lo => 255.0.0.0network_eth0 => 10.0.2.0network_eth1 => 192.168.33.0network_lo => 127.0.0.0operatingsystem => Ubuntuoperatingsystemrelease => 8.04…
  33. 33. Puppet: Miscellaneous Info (Cont.)Describes a resource, use -s to summarize$ puppet describe <resource>Inspect individual resource in “Puppet-speak”$ puppet resource service apache2service { apache2: ensure => stopped, enable => true,}Test a manifest on the local system$ puppet apply --modulepath=<module_path> <manifest>.pp
  34. 34. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  35. 35. Puppet: Important NotesCommon Pattern:Package, File, ServiceResources are processed asynchronously! Use dependencyrelationships (require/before) if any ordering is necessary.@see http://forge.puppetlabs.com/ for prebuilt Puppetmodules.
  36. 36. Puppet Main Manifestmyproject.ppclass myproject { Site specific ensure => absent; $site = ’myproject $site: ensure => present; Site/DB # Vagrant specific setup, set up hostname, hosts file } class { vagrant: apache::mod { site => $site, rewrite: ensure => present } } # Cool stuff mysql::schema { $site: class { vim: } user => $site, password => $site, # LAMP stack } class { linux: } class { apache: } Base LAMP class { drupal::prepare: class { mysql: } site => $site, class { php: admin_pw => admin, error_reporting => E_ALL & ~(E_WARNING|E_NOTICE), } } } class { phpmyadmin: } class { myproject: } # Drupal class { drupal: server_type => dev, } # Solr class { tomcat: } class { solr: } Search # Site specific stuff apache::site { default: vhost => default,
  37. 37. Example 2: Puppet Module/modules/apache/init.pp (Header)# Class: apache # vhost => default,# # ensure => absent;# This module installs apache and sets up the virtual.# # <site_name_1>: # ensure => present; Advanced usage# Mike Lee <michael.lee@achieveinternet.com> # <site_name_2>:# 2012-09-07# # docroot => /custom/path, # ensure => present; Enable host/mod# Tested platforms: #}# - Ubuntu 8.04 Hardy Heron ## # Enable/disable modules.# Parameters: # apache::mod {# N/A # rewrite: ensure => present’# #}# Actions:# Installs and configures apache. Will automatically create the site file.## Requires:# N/A## Sample Usage:## Make sure apache is present.# class { apache: }# Base usage# Enable/disable sites.# <site_name> = Simple site name. The vagrant domain will be addedautomatically.# i.e. <site_name> = "example" creates a site named "example.vagrant".# <docroot> = Docroot. Defaults to /vagrant/<site_name>/docroot.# If this is overridden, this must be the ABSOLUTE path to the docroot.## apache::site {# default:
  38. 38. Puppet Sample Module/modules/apache/init.pp (Code)class apache { case $ensure { $apache2_sites = /etc/apache2/sites present: { $apache2_mods = /etc/apache2/mods file { "/etc/apache2/sites-available/$site_name": ensure => file, # OS Specific settings case $::operatingsystem { OS Specific require => Package[$apache::package_name], content => template("apache/${apache::conf_template}"), ubuntu: { } $package_name = apache2 $conf_template = site.conf.debian.erb exec { "/usr/sbin/a2ensite $site_name": } unless => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", default: { notify => Exec[force-reload-apache], notify { "${module_name}_unsupported": require => Package[$apache::package_name], message => "The ${module_name} module is not supported on }${::operatingsystem}.", } } absent: { error("OS support for ${::operatingsystem} needs to be configured") exec { "/usr/sbin/a2dissite $site_name": } #onlyif => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", # } readlink not returning 0 # Define new type: site Define type: site notify => Exec[force-reload-apache], require => Package[$apache::package_name], define site ($ensure = present, $vhost = undef, $docroot = undef) { } if $vhost { } $site_name = $vhost default: { } err("Unknown ensure value: $ensure") else { } $site_name = "${title}.vagrant" } } } if $docroot { $path = $docroot } else { $path = "/vagrant/${title}/docroot" }
  39. 39. Puppet Sample Module/modules/apache/init.pp (Code) # Define new type: mod define mod ($ensure = present) { service { $package_name: case $ensure { present: { ensure => running, hasstatus => true, Package/Service exec { "/usr/sbin/a2enmod $title": hasrestart => true, unless => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", require => Package[$package_name], notify => Exec[force-reload-apache], } require => Package[$apache::package_name], } } } absent: { Define type: mod exec { "/usr/sbin/a2dismod $title": #onlyif => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", #readlink not returning 0 notify => Exec[force-reload-apache], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: $ensure") } } } # Force reload all the time, doesnt take that much more in resources. exec { force-reload-apache: command => /etc/init.d/apache2 force-reload, refreshonly => true, } package { $package_name: ensure => installed, }
  40. 40. Puppet Sample ModuleERB Templates: Create files on the fly./modules/apache/templates/site.conf.debian.erb# Puppet generated file. DO NOT EDIT!# Managed by Class[apache]<VirtualHost *:80> ServerAdmin webmaster@achieveinternet.com DocumentRoot <%= path %> ServerName <%= site_name %> <Directory <%= path %>> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory></VirtualHost>
  41. 41. Puppet++ Construct fully functioning development environments on the fly! Bring new team members up to speed in a few simple commands. CONSISTENCY! CONSISTENCY! CONSISTENCY! Everyone is working with the same virtual machine.
  42. 42. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  43. 43. Veewee Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  44. 44. Next Steps: VeeweeWant your own custom Vagrant Box or one doesn’t exist?Veewee has predefined definitions for creating a variety of Vagrant base boxes.Installation Requirements (Mac OS X)- Install RVM (Ruby Version Manager) $ curl -L https://get.rvm.io | bash -s stable- Install Xcode, Command Line Tools for Xcode (required by Homebrew)- Install Homebrew $ ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)- Install apple-gcc42 GCC Required! $ brew update $ brew tap homebrew/dupes $ brew install autoconf automake apple-gcc42 $ rvm pkg install openssl
  45. 45. Next Steps: VeeweeVeewee Installation (Mac OS X)$ rvm install 1.9.2$ git clone https://github.com/jedi4ever/veewee.git$ cd veewee(official – Never worked for me…)$ gem install bundler$ bundle install(unoffical)$ gem build veewee.gemspec$ gem install veewee-*.gem (run as sudo)$ unalias veewee
  46. 46. Next Steps: VeeweeVeewee Base Box Creation (Mac OS X)In veewee directory:- Copy new iso into [currentDir]/iso (optional, saves veewee from finding)Create VM definition off of template:Usage: veewee vbox define <box_name> <template>$ veewee vbox define hardy ubuntu-8.04.4-server-i386Customize VM Definition:- Edit <veewee_dir>/definitions/<box_name>/definition.rbBuild virtual box (This should start VirtualBox)Usage: veewee build <box_name>$ veewee vbox build hardy # go get a coffeeUsage: veewee vbox validate <box_name>$ veewee vbox validate hardyBuild Vagrant base box:Usage: vagrant basebox export <box_name>$ vagrant basebox export hardy
  47. 47. Next Steps: Puppet Enterprise Allow for remote management of puppet nodes.  “One server to rule them all.” Clone your configurations to EC2. Audit your configurations.
  48. 48. Gotchas Be mindful of Puppets asyncronous execution order:  Make sure dependecies are properly set. Starbucks syndrome:  Switching networks can sometimes cause your VM’s networking to get confused. Halt and restart to correct. Provision everything… or as much as you can:  This will make resetting your environment go much more smoothly.
  49. 49. Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/) Puppet v2.7.19 Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee) Ubuntu v8.04 Hardy Heron
  50. 50. Further Resources Prebulit Vagrant Base Boxes (http://www.vagrantbox.es/) Puppet Forge (http://forge.puppetlabs.com/) Learn Puppet (http://docs.puppetlabs.com/learning/ral.html)
  51. 51. Conclusion …makes the "works on my machine" excuse a relic of the past. -- Vagrant Created a disposable test bed to play with configuration changes Created reusable modules to easily build customized setups Provided canned environment for those who simply want to jump into a project. Allows team members to focus on core competencies. Allows you to quickly and efficiently archive projects and come back to them later  CONSISTENCY - REDUCED ERRORS  EASY & INEXPENSIVE  INSTANT DEVELOPMENT - SAVE $$$
  52. 52. Achieve Internet Pillars of Expertise Technical Foundation Built On the Following Core Areas of Expertise and Knowledge: 1. User Experience (UX) / Theming 2. Globalization/Localization 3. Performance & Deployment 4. Integration
  53. 53. QA
  54. 54. Thank You Michael Lee Senior Architectmichael.lee@achieveinternet.com www.achieveinternet.com mlee11111 @ach_mikelee /pub/michael-lee/4/297/a6b Direct: 858-453-5760
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×