Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

  • 2,157 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,157
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
23
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Prepared For: Acquia October 4, 2012 Title: Harmonious Development:Standardizing The Deployment Process via Vagrant and Puppet Achieve – Quality on Time 1
  • 2. Achieve Internet Overview 7+ years developing media and entertainment web properties Over 60,000 development hours developing Drupal web & mobile-optimized solutions Deep expertise in UX (theming), Globalization, Integration, and Performance Company commitment to quality, transparency, continuity, reliability and client experience
  • 3. Key Clients: Media & Entertainment Media
  • 4. Key Clients: Other Other
  • 5. Pop Quiz<?php/** * Transform upper camel case to lower camel case. * * @param string $word * Upper camel case word. * * @return string * $word with first letter in lower case. * * @see http://www.php.net/manual/en/function.lcfirst.php */function my_module_camel_case($word) { return lcfirst($word);}echo my_module_camel_case(HelloWorld);
  • 6. Agenda+ =
  • 7. Agenda (Cont.) What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly Provisioning (via Puppet) Examples Future Directions Q&A
  • 8. What We Are Not Covering Performance Tuning  These are examples only Version Control  All provisioning can be placed into VCS Puppet vs. Chef
  • 9. What’s all the fuss?“Write once, run anywhere” -- Sun Microsystems“Write Once, Debug Everywhere” -- Anonymous / Reality“In many distributed computing environments, failures arereported in a way that violates even the simplest notions ofconsistency.” -- IEEE Software, Bradford Glade. K. Birman
  • 10. The Good Old Days
  • 11. Today….
  • 12. How It ComparesOld Approach Old Approach… Today
  • 13. The Drupal Ecosystem
  • 14. Site Building Roles
  • 15. How It All Fits Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  • 16. Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/) Puppet v2.7.19 Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee) Ubuntu v8.04 Hardy Heron
  • 17. Virtual Box Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  • 18. What is Virtualization? Technical Definition…. A logical representation of a computer in software. By decoupling the physical hardware from the operating system, virtualization provides more operational flexibility and increases the utilization rate of the underlying physical hardware.1 In a Nutshell…. Putting a small computer [guest/virtual machine] (operating system, memory, video display, peripherals, etc.) into the confines of another bigger physical system [host].
  • 19. VirtualBox Open Source virtualization software for Windows, Mac, and Linux Can host multiple operating systems at once (within memory and CPU resources) Free!
  • 20. Vagrant Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  • 21. Vagrant Manages your virtual machines Works with VirtualBox Extremely customizable Over 50+ pre-built base boxes Free!
  • 22. Why Vagrant? Creates consistent, reproducible environments Rapid setup – Start development in a few quick steps Reduce duplicated effort Increased reliability Allows you to tinker around, more on this later… Cost savings!
  • 23. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  • 24. Using VagrantObjective:Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM)One Time Steps:List available boxes:$ vagrant box listAdd a new Vagrant base box to inventory if not available:(@see http://www.vagrantbox.es/)Usage: vagrant box add <alias> <location>$ vagrant box add hardy /path/to/hardy.boxCreate a directory for your Vagrant VMs$ mkdir <vm_dir>/myprojectCreate a Vagrant file (do this in <vm_dir>/myproject)Usage: vagrant init <alias>$ vagrant init hardy
  • 25. Using VagrantLet’s Rock!!Start up a virtual machine:$ vagrant upDelete a virtual machine:$ vagrant removeConnect to virtual machine:$ vagrant sshSuccess! Instant VM. Do the happy dance!
  • 26. Using VagrantOther Useful CommandsGo to lunch:$ vagrant suspend # Save state of virtual machine$ vagrant resume # Start virtual machine up againReboot:$ vagrant halt # Shutdown the VM$ vagrant up –no-provision # Restart VM, skip provisioning** = Online documentation says no reprovisioning will occur…Reload configuration (automatically does a provisioning)$ vagrant reloadReprovision a system:$ vagrant reprovisionRepackage running system:$ vagrant package
  • 27. Example 1$ mkdir <vm_dir>/myproject$ vagrant init hardyCustomize our VM Instance (Excerpts of Vagrantfile)config.vm.network :hostonly, "192.168.33.10”# Host: <vm_dir>/<project_name> == Guest: /vagrantconfig.vm.share_folder "v-root", "/vagrant", ".", :owner => www-data, :group => www-data’# Configure the VM with puppet.config.vm.provision :puppet, :module_path => "modules", :options => ["--environment", "local"] do |puppet| puppet.manifests_path = "manifests” puppet.manifest_file = ”myproject.pp”End$ vagrant up
  • 28. Example 1Scenario: Admin for a DayWonder what this does… (WARNING)*$ sudo chmod 444 /etc/sudoers* Warning, doing this on a Production system may have an adverse effect onyour career. This will remove any ability to sudo.Solution (Assuming all configuration is provisioned):$ vagrant destroy$ vagrant up
  • 29. Vagrant++ Instant development environments! No physical hardware required. Creates a safe environment for configuration optimization.
  • 30. Puppet Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  • 31. Puppet Provisioning = preparing a device/server for usage, aka configuration Allows enforceable system configuration Similar to Microsoft’s System Center Configuration Manager/SMS or Apple’s Profile Manager Uses a Resource Abstraction Layer (RAL) to interact with the host. Core Resource Types include: notify, file, package, service, exec, cron, user, group RAL can be used to read and modify resources Configurations can be store in manifests. See resources for prebuilt modules.
  • 32. Puppet: Miscellaneous InfoFacter: Inspects operating system “facts” that can be used in modules.$ facter… id => vagrantinterfaces => eth0,eth1,loipaddress => 10.0.2.15ipaddress_eth0 => 10.0.2.15ipaddress_eth1 => 192.168.33.10ipaddress_lo => 127.0.0.1is_virtual => truekernel => Linuxkernelmajversion => 2.6kernelrelease => 2.6.24-26-serverkernelversion => 2.6.24lsbdistcodename => hardylsbdistdescription => Ubuntu 8.04.4 LTSlsbdistid => Ubuntulsbdistrelease => 8.04lsbmajdistrelease => 8memoryfree => 1.85 GBmemorysize => 1.98 GBmemorytotal => 1.98 GBnetmask => 255.255.255.0netmask_lo => 255.0.0.0network_eth0 => 10.0.2.0network_eth1 => 192.168.33.0network_lo => 127.0.0.0operatingsystem => Ubuntuoperatingsystemrelease => 8.04…
  • 33. Puppet: Miscellaneous Info (Cont.)Describes a resource, use -s to summarize$ puppet describe <resource>Inspect individual resource in “Puppet-speak”$ puppet resource service apache2service { apache2: ensure => stopped, enable => true,}Test a manifest on the local system$ puppet apply --modulepath=<module_path> <manifest>.pp
  • 34. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  • 35. Puppet: Important NotesCommon Pattern:Package, File, ServiceResources are processed asynchronously! Use dependencyrelationships (require/before) if any ordering is necessary.@see http://forge.puppetlabs.com/ for prebuilt Puppetmodules.
  • 36. Puppet Main Manifestmyproject.ppclass myproject { Site specific ensure => absent; $site = ’myproject $site: ensure => present; Site/DB # Vagrant specific setup, set up hostname, hosts file } class { vagrant: apache::mod { site => $site, rewrite: ensure => present } } # Cool stuff mysql::schema { $site: class { vim: } user => $site, password => $site, # LAMP stack } class { linux: } class { apache: } Base LAMP class { drupal::prepare: class { mysql: } site => $site, class { php: admin_pw => admin, error_reporting => E_ALL & ~(E_WARNING|E_NOTICE), } } } class { phpmyadmin: } class { myproject: } # Drupal class { drupal: server_type => dev, } # Solr class { tomcat: } class { solr: } Search # Site specific stuff apache::site { default: vhost => default,
  • 37. Example 2: Puppet Module/modules/apache/init.pp (Header)# Class: apache # vhost => default,# # ensure => absent;# This module installs apache and sets up the virtual.# # <site_name_1>: # ensure => present; Advanced usage# Mike Lee <michael.lee@achieveinternet.com> # <site_name_2>:# 2012-09-07# # docroot => /custom/path, # ensure => present; Enable host/mod# Tested platforms: #}# - Ubuntu 8.04 Hardy Heron ## # Enable/disable modules.# Parameters: # apache::mod {# N/A # rewrite: ensure => present’# #}# Actions:# Installs and configures apache. Will automatically create the site file.## Requires:# N/A## Sample Usage:## Make sure apache is present.# class { apache: }# Base usage# Enable/disable sites.# <site_name> = Simple site name. The vagrant domain will be addedautomatically.# i.e. <site_name> = "example" creates a site named "example.vagrant".# <docroot> = Docroot. Defaults to /vagrant/<site_name>/docroot.# If this is overridden, this must be the ABSOLUTE path to the docroot.## apache::site {# default:
  • 38. Puppet Sample Module/modules/apache/init.pp (Code)class apache { case $ensure { $apache2_sites = /etc/apache2/sites present: { $apache2_mods = /etc/apache2/mods file { "/etc/apache2/sites-available/$site_name": ensure => file, # OS Specific settings case $::operatingsystem { OS Specific require => Package[$apache::package_name], content => template("apache/${apache::conf_template}"), ubuntu: { } $package_name = apache2 $conf_template = site.conf.debian.erb exec { "/usr/sbin/a2ensite $site_name": } unless => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", default: { notify => Exec[force-reload-apache], notify { "${module_name}_unsupported": require => Package[$apache::package_name], message => "The ${module_name} module is not supported on }${::operatingsystem}.", } } absent: { error("OS support for ${::operatingsystem} needs to be configured") exec { "/usr/sbin/a2dissite $site_name": } #onlyif => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", # } readlink not returning 0 # Define new type: site Define type: site notify => Exec[force-reload-apache], require => Package[$apache::package_name], define site ($ensure = present, $vhost = undef, $docroot = undef) { } if $vhost { } $site_name = $vhost default: { } err("Unknown ensure value: $ensure") else { } $site_name = "${title}.vagrant" } } } if $docroot { $path = $docroot } else { $path = "/vagrant/${title}/docroot" }
  • 39. Puppet Sample Module/modules/apache/init.pp (Code) # Define new type: mod define mod ($ensure = present) { service { $package_name: case $ensure { present: { ensure => running, hasstatus => true, Package/Service exec { "/usr/sbin/a2enmod $title": hasrestart => true, unless => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", require => Package[$package_name], notify => Exec[force-reload-apache], } require => Package[$apache::package_name], } } } absent: { Define type: mod exec { "/usr/sbin/a2dismod $title": #onlyif => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", #readlink not returning 0 notify => Exec[force-reload-apache], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: $ensure") } } } # Force reload all the time, doesnt take that much more in resources. exec { force-reload-apache: command => /etc/init.d/apache2 force-reload, refreshonly => true, } package { $package_name: ensure => installed, }
  • 40. Puppet Sample ModuleERB Templates: Create files on the fly./modules/apache/templates/site.conf.debian.erb# Puppet generated file. DO NOT EDIT!# Managed by Class[apache]<VirtualHost *:80> ServerAdmin webmaster@achieveinternet.com DocumentRoot <%= path %> ServerName <%= site_name %> <Directory <%= path %>> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory></VirtualHost>
  • 41. Puppet++ Construct fully functioning development environments on the fly! Bring new team members up to speed in a few simple commands. CONSISTENCY! CONSISTENCY! CONSISTENCY! Everyone is working with the same virtual machine.
  • 42. Agenda - Revisited What’s all the fuss? What is Virtualization? Vagrant – Creating (and destroying) environments on the fly. Provisioning (via Puppet) Examples Future DirectionsQ&A
  • 43. Veewee Virtual Box Vagrant(Virtual Environment) (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
  • 44. Next Steps: VeeweeWant your own custom Vagrant Box or one doesn’t exist?Veewee has predefined definitions for creating a variety of Vagrant base boxes.Installation Requirements (Mac OS X)- Install RVM (Ruby Version Manager) $ curl -L https://get.rvm.io | bash -s stable- Install Xcode, Command Line Tools for Xcode (required by Homebrew)- Install Homebrew $ ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)- Install apple-gcc42 GCC Required! $ brew update $ brew tap homebrew/dupes $ brew install autoconf automake apple-gcc42 $ rvm pkg install openssl
  • 45. Next Steps: VeeweeVeewee Installation (Mac OS X)$ rvm install 1.9.2$ git clone https://github.com/jedi4ever/veewee.git$ cd veewee(official – Never worked for me…)$ gem install bundler$ bundle install(unoffical)$ gem build veewee.gemspec$ gem install veewee-*.gem (run as sudo)$ unalias veewee
  • 46. Next Steps: VeeweeVeewee Base Box Creation (Mac OS X)In veewee directory:- Copy new iso into [currentDir]/iso (optional, saves veewee from finding)Create VM definition off of template:Usage: veewee vbox define <box_name> <template>$ veewee vbox define hardy ubuntu-8.04.4-server-i386Customize VM Definition:- Edit <veewee_dir>/definitions/<box_name>/definition.rbBuild virtual box (This should start VirtualBox)Usage: veewee build <box_name>$ veewee vbox build hardy # go get a coffeeUsage: veewee vbox validate <box_name>$ veewee vbox validate hardyBuild Vagrant base box:Usage: vagrant basebox export <box_name>$ vagrant basebox export hardy
  • 47. Next Steps: Puppet Enterprise Allow for remote management of puppet nodes.  “One server to rule them all.” Clone your configurations to EC2. Audit your configurations.
  • 48. Gotchas Be mindful of Puppets asyncronous execution order:  Make sure dependecies are properly set. Starbucks syndrome:  Switching networks can sometimes cause your VM’s networking to get confused. Halt and restart to correct. Provision everything… or as much as you can:  This will make resetting your environment go much more smoothly.
  • 49. Versions VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads) Vagrant v1.0.3 (http://downloads.vagrantup.com/) Puppet v2.7.19 Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee) Ubuntu v8.04 Hardy Heron
  • 50. Further Resources Prebulit Vagrant Base Boxes (http://www.vagrantbox.es/) Puppet Forge (http://forge.puppetlabs.com/) Learn Puppet (http://docs.puppetlabs.com/learning/ral.html)
  • 51. Conclusion …makes the "works on my machine" excuse a relic of the past. -- Vagrant Created a disposable test bed to play with configuration changes Created reusable modules to easily build customized setups Provided canned environment for those who simply want to jump into a project. Allows team members to focus on core competencies. Allows you to quickly and efficiently archive projects and come back to them later  CONSISTENCY - REDUCED ERRORS  EASY & INEXPENSIVE  INSTANT DEVELOPMENT - SAVE $$$
  • 52. Achieve Internet Pillars of Expertise Technical Foundation Built On the Following Core Areas of Expertise and Knowledge: 1. User Experience (UX) / Theming 2. Globalization/Localization 3. Performance & Deployment 4. Integration
  • 53. QA
  • 54. Thank You Michael Lee Senior Architectmichael.lee@achieveinternet.com www.achieveinternet.com mlee11111 @ach_mikelee /pub/michael-lee/4/297/a6b Direct: 858-453-5760