• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Harmonious Development: Via Vagrant and Puppet

on

  • 2,313 views

Through the magic of virtualization technology (Vagrant) and Puppet, a companion Enterprise grade provisioning technology, we explore how to make the complex configuration game a walk in the park. ...

Through the magic of virtualization technology (Vagrant) and Puppet, a companion Enterprise grade provisioning technology, we explore how to make the complex configuration game a walk in the park. Bring new team members up to speed in minutes, eliminate variances in configurations, and make integration issues a thing of the past.

Welcome to the new age of team development!

Statistics

Views

Total Views
2,313
Views on SlideShare
2,313
Embed Views
0

Actions

Likes
1
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Harmonious Development: Via Vagrant and Puppet Harmonious Development: Via Vagrant and Puppet Presentation Transcript

    • Prepared For: Acquia October 4, 2012 Title: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet Achieve – Quality on Time 1
    • Achieve Internet Overview  7+ years developing media and entertainment web properties  Over 60,000 development hours developing Drupal web & mobile-optimized solutions  Deep expertise in UX (theming), Globalization, Integration, and Performance  Company commitment to quality, transparency, continuity, reliability and client experience 
    • Key Clients: Media & Entertainment Media
    • Key Clients: Other Other
    • Pop Quiz <?php /** * Transform upper camel case to lower camel case. * * @param string $word * Upper camel case word. * * @return string * $word with first letter in lower case. * * @see http://www.php.net/manual/en/function.lcfirst.php */ function my_module_camel_case($word) { return lcfirst($word); } echo my_module_camel_case('HelloWorld');
    • Agenda + =
    • Agenda (Cont.)  What’s all the fuss?  What is Virtualization?  Vagrant – Creating (and destroying) environments on the fly  Provisioning (via Puppet)  Examples  Future Directions  Q&A
    • What We Are Not Covering  Performance Tuning  These are examples only  Version Control  All provisioning can be placed into VCS  Puppet vs. Chef
    • What’s all the fuss? “Write once, run anywhere” -- Sun Microsystems “Write Once, Debug Everywhere” -- Anonymous / Reality “In many distributed computing environments, failures are reported in a way that violates even the simplest notions of consistency.” -- IEEE Software, Bradford Glade. K. Birman
    • The Good Old Days
    • Today….
    • How It Compares Old Approach Old Approach… Today
    • The Drupal Ecosystem
    • Site Building Roles
    • How It All Fits Virtual Box (Virtual Environment) Vagrant (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
    • Versions  VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads)  Vagrant v1.0.3 (http://downloads.vagrantup.com/)  Puppet v2.7.19  Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee)  Ubuntu v8.04 Hardy Heron
    • Virtual Box Virtual Box (Virtual Environment) Vagrant (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
    • What is Virtualization?  Technical Definition….  A logical representation of a computer in software. By decoupling the physical hardware from the operating system, virtualization provides more operational flexibility and increases the utilization rate of the underlying physical hardware.1  In a Nutshell….  Putting a small computer [guest/virtual machine] (operating system, memory, video display, peripherals, etc.) into the confines of another bigger physical system [host].
    • VirtualBox  Open Source virtualization software for Windows, Mac, and Linux  Can host multiple operating systems at once (within memory and CPU resources)  Free!
    • Vagrant Virtual Box (Virtual Environment) Vagrant (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
    • Vagrant  Manages your virtual machines  Works with VirtualBox  Extremely customizable  Over 50+ pre-built base boxes  Free!
    • Why Vagrant?  Creates consistent, reproducible environments  Rapid setup – Start development in a few quick steps  Reduce duplicated effort  Increased reliability  Allows you to tinker around, more on this later…  Cost savings!
    • Agenda - Revisited  What’s all the fuss?  What is Virtualization?  Vagrant – Creating (and destroying) environments on the fly.  Provisioning (via Puppet)  Examples  Future Directions Q&A
    • Using Vagrant Objective: Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM) One Time Steps: List available boxes: $ vagrant box list Add a new Vagrant base box to inventory if not available: (@see http://www.vagrantbox.es/) Usage: vagrant box add <alias> <location> $ vagrant box add hardy /path/to/hardy.box Create a directory for your Vagrant VMs $ mkdir <vm_dir>/myproject Create a Vagrant file (do this in <vm_dir>/myproject) Usage: vagrant init <alias> $ vagrant init hardy
    • Using Vagrant Let’s Rock!! Start up a virtual machine: $ vagrant up Delete a virtual machine: $ vagrant remove Connect to virtual machine: $ vagrant ssh Success! Instant VM. Do the happy dance!
    • Using Vagrant Other Useful Commands Go to lunch: $ vagrant suspend # Save state of virtual machine $ vagrant resume # Start virtual machine up again Reboot: $ vagrant halt # Shutdown the VM $ vagrant up –no-provision # Restart VM, skip provisioning* * = Online documentation says no reprovisioning will occur… Reload configuration (automatically does a provisioning) $ vagrant reload Reprovision a system: $ vagrant reprovision Repackage running system: $ vagrant package
    • Example 1 $ mkdir <vm_dir>/myproject $ vagrant init hardy Customize our VM Instance (Excerpts of Vagrantfile) config.vm.network :hostonly, "192.168.33.10” # Host: <vm_dir>/<project_name> == Guest: /vagrant config.vm.share_folder "v-root", "/vagrant", ".", :owner => 'www-data', :group => 'wwwdata’ # Configure the VM with puppet. config.vm.provision :puppet, :module_path => "modules", :options => ["--environment", "local"] do |puppet| puppet.manifests_path = "manifests” puppet.manifest_file = ”myproject.pp” End $ vagrant up
    • Example 1 Scenario: Admin for a Day Wonder what this does… (WARNING)* $ sudo chmod 444 /etc/sudoers * Warning, doing this on a Production system may have an adverse effect on your career. This will remove any ability to sudo. Solution (Assuming all configuration is provisioned): $ vagrant destroy $ vagrant up
    • Vagrant++  Instant development environments!  No physical hardware required.  Creates a safe environment for configuration optimization.
    • Puppet Virtual Box (Virtual Environment) Vagrant (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
    • Puppet  Provisioning = preparing a device/server for usage, aka configuration  Allows enforceable system configuration  Similar to Microsoft’s System Center Configuration Manager/SMS or Apple’s Profile Manager  Uses a Resource Abstraction Layer (RAL) to interact with the host. Core Resource Types include: notify, file, package, service, exec, cron, user, group  RAL can be used to read and modify resources  Configurations can be store in manifests. See resources for prebuilt modules.
    • Puppet: Miscellaneous Info Facter: Inspects operating system “facts” that can be used in modules. $ facter … id => vagrant interfaces => eth0,eth1,lo ipaddress => 10.0.2.15 ipaddress_eth0 => 10.0.2.15 ipaddress_eth1 => 192.168.33.10 ipaddress_lo => 127.0.0.1 is_virtual => true kernel => Linux kernelmajversion => 2.6 kernelrelease => 2.6.24-26-server kernelversion => 2.6.24 lsbdistcodename => hardy lsbdistdescription => Ubuntu 8.04.4 LTS lsbdistid => Ubuntu lsbdistrelease => 8.04 lsbmajdistrelease => 8 memoryfree => 1.85 GB memorysize => 1.98 GB memorytotal => 1.98 GB netmask => 255.255.255.0 netmask_lo => 255.0.0.0 network_eth0 => 10.0.2.0 network_eth1 => 192.168.33.0 network_lo => 127.0.0.0 operatingsystem => Ubuntu operatingsystemrelease => 8.04 …
    • Puppet: Miscellaneous Info (Cont.) Describes a resource, use -s to summarize $ puppet describe <resource> Inspect individual resource in “Puppet-speak” $ puppet resource service apache2 service { 'apache2': ensure => 'stopped', enable => 'true', } Test a manifest on the local system $ puppet apply --modulepath=<module_path> <manifest>.pp
    • Agenda - Revisited  What’s all the fuss?  What is Virtualization?  Vagrant – Creating (and destroying) environments on the fly.  Provisioning (via Puppet)  Examples  Future Directions Q&A
    • Puppet: Important Notes Common Pattern: Package, File, Service Resources are processed asynchronously! Use dependency relationships (require/before) if any ordering is necessary. @see http://forge.puppetlabs.com/ for prebuilt Puppet modules.
    • Puppet Main Manifest myproject.pp class myproject { $site = ’myproject' Site specific ensure => 'absent'; $site: ensure => 'present'; # Vagrant specific setup, set up hostname, hosts file class { 'vagrant': site => $site, } } apache::mod { 'rewrite': ensure => 'present' } # Cool stuff class { 'vim': } mysql::schema { $site: user => $site, password => $site, } # LAMP stack class { 'linux': } class { 'apache': } class { 'mysql': } class { 'php': error_reporting => 'E_ALL & ~(E_WARNING|E_NOTICE)', } class { 'phpmyadmin': } Base LAMP class { 'drupal::prepare': site => $site, admin_pw => 'admin', } } class { 'myproject': } # Drupal class { 'drupal': server_type => 'dev', } # Solr class { 'tomcat': } class { 'solr': } # Site specific stuff apache::site { 'default': vhost => 'default', Search Site/DB
    • Example 2: Puppet Module /modules/apache/init.pp (Header) # Class: apache # # This module installs apache and sets up the virtual. # # Mike Lee <michael.lee@achieveinternet.com> # 2012-09-07 # # Tested platforms: # - Ubuntu 8.04 Hardy Heron # # Parameters: # N/A # # Actions: # Installs and configures apache. Will automatically create the site file. # # Requires: # N/A # # Sample Usage: # # Make sure apache is present. # class { 'apache': } # # Enable/disable sites. # <site_name> = Simple site name. The vagrant domain will be added automatically. # i.e. <site_name> = "example" creates a site named "example.vagrant". # <docroot> = Docroot. Defaults to /vagrant/<site_name>/docroot. # If this is overridden, this must be the ABSOLUTE path to the docroot. # # apache::site { # 'default': Base usage # vhost => 'default', # ensure => 'absent'; # '<site_name_1>': # ensure => 'present'; # '<site_name_2>': # docroot => '/custom/path', # ensure => 'present'; #} # # Enable/disable modules. # apache::mod { # 'rewrite': ensure => 'present’ #} Advanced usage Enable host/mod
    • Puppet Sample Module /modules/apache/init.pp (Code) class apache { $apache2_sites = '/etc/apache2/sites' $apache2_mods = '/etc/apache2/mods' OS Specific # OS Specific settings case $::operatingsystem { ubuntu: { $package_name = 'apache2' $conf_template = 'site.conf.debian.erb' } default: { notify { "${module_name}_unsupported": message => "The ${module_name} module is not supported on ${::operatingsystem}.", } error("OS support for ${::operatingsystem} needs to be configured") } } Define type: site # Define new type: site define site ($ensure = 'present', $vhost = undef, $docroot = undef) { if $vhost { $site_name = $vhost } else { $site_name = "${title}.vagrant" } if $docroot { $path = $docroot } else { $path = "/vagrant/${title}/docroot" } case $ensure { present: { file { "/etc/apache2/sites-available/$site_name": ensure => file, require => Package[$apache::package_name], content => template("apache/${apache::conf_template}"), } exec { "/usr/sbin/a2ensite $site_name": unless => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } absent: { exec { "/usr/sbin/a2dissite $site_name": #onlyif => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", # readlink not returning 0 notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: '$ensure'") } } }
    • Puppet Sample Module /modules/apache/init.pp (Code) # Define new type: mod define mod ($ensure = 'present') { case $ensure { present: { exec { "/usr/sbin/a2enmod $title": unless => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } absent: { exec { "/usr/sbin/a2dismod $title": #onlyif => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", # readlink not returning 0 notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: '$ensure'") } } } Define type: mod # Force reload all the time, doesn't take that much more in resources. exec { 'force-reload-apache': command => '/etc/init.d/apache2 force-reload', refreshonly => true, } package { $package_name: ensure => installed, } service { $package_name: ensure => running, hasstatus => true, hasrestart => true, require => Package[$package_name], } Package/Service }
    • Puppet Sample Module ERB Templates: Create files on the fly. /modules/apache/templates/site.conf.debian.erb # Puppet generated file. DO NOT EDIT! # Managed by Class['apache'] <VirtualHost *:80> ServerAdmin webmaster@achieveinternet.com DocumentRoot <%= path %> ServerName <%= site_name %> <Directory <%= path %>> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory> </VirtualHost>
    • Puppet++  Construct fully functioning development environments on the fly!  Bring new team members up to speed in a few simple commands.  CONSISTENCY! CONSISTENCY! CONSISTENCY! Everyone is working with the same virtual machine.
    • Agenda - Revisited  What’s all the fuss?  What is Virtualization?  Vagrant – Creating (and destroying) environments on the fly.  Provisioning (via Puppet)  Examples  Future Directions Q&A
    • Veewee Virtual Box (Virtual Environment) Vagrant (Virtual Machine Builder) VeeWee (Vagrant Box Builder) Puppet (Provisioner)
    • Next Steps: Veewee Want your own custom Vagrant Box or one doesn’t exist? Veewee has predefined definitions for creating a variety of Vagrant base boxes. Installation Requirements (Mac OS X) - Install RVM (Ruby Version Manager) $ curl -L https://get.rvm.io | bash -s stable - Install Xcode, Command Line Tools for Xcode (required by Homebrew) - Install Homebrew $ ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go) - Install apple-gcc42 GCC Required! $ brew update $ brew tap homebrew/dupes $ brew install autoconf automake apple-gcc42 $ rvm pkg install openssl
    • Next Steps: Veewee Veewee Installation (Mac OS X) $ rvm install 1.9.2 $ git clone https://github.com/jedi4ever/veewee.git $ cd veewee (official – Never worked for me…) $ gem install bundler $ bundle install (unoffical) $ gem build veewee.gemspec $ gem install veewee-*.gem (run as sudo) $ unalias veewee
    • Next Steps: Veewee Veewee Base Box Creation (Mac OS X) In veewee directory: - Copy new iso into [currentDir]/iso (optional, saves veewee from finding) Create VM definition off of template: Usage: veewee vbox define <box_name> <template> $ veewee vbox define hardy ubuntu-8.04.4-server-i386 Customize VM Definition: - Edit <veewee_dir>/definitions/<box_name>/definition.rb Build virtual box (This should start VirtualBox) Usage: veewee build <box_name> $ veewee vbox build hardy # go get a coffee Usage: veewee vbox validate <box_name> $ veewee vbox validate hardy Build Vagrant base box: Usage: vagrant basebox export <box_name> $ vagrant basebox export hardy
    • Next Steps: Puppet Enterprise  Allow for remote management of puppet nodes.  “One server to rule them all.”  Clone your configurations to EC2.  Audit your configurations.
    • Gotchas  Be mindful of Puppets asyncronous execution order:  Make sure dependecies are properly set.  Starbucks syndrome:  Switching networks can sometimes cause your VM’s networking to get confused. Halt and restart to correct.  Provision everything… or as much as you can:  This will make resetting your environment go much more smoothly.
    • Versions  VirtualBox v4.1.18 (https://www.virtualbox.org/wiki/Downloads)  Vagrant v1.0.3 (http://downloads.vagrantup.com/)  Puppet v2.7.19  Veewee v0.3.0.beta1 (https://github.com/jedi4ever/veewee)  Ubuntu v8.04 Hardy Heron
    • Further Resources  Prebulit Vagrant Base Boxes (http://www.vagrantbox.es/)  Puppet Forge (http://forge.puppetlabs.com/)  Learn Puppet (http://docs.puppetlabs.com/learning/ral.html)
    • Conclusion  …makes the "works on my machine" excuse a relic of the past. -- Vagrant  Created a disposable test bed to play with configuration changes  Created reusable modules to easily build customized setups  Provided canned environment for those who simply want to jump into a project. Allows team members to focus on core competencies.  Allows you to quickly and efficiently archive projects and come back to them later  CONSISTENCY - REDUCED ERRORS  EASY & INEXPENSIVE  INSTANT DEVELOPMENT - SAVE $$$
    • Achieve Internet Pillars of Expertise Technical Foundation Built On the Following Core Areas of Expertise and Knowledge: 1. User Experience (UX) / Theming 2. Globalization/Localization 3. Performance & Deployment 4. Integration
    • QA
    • Thank You Michael Lee Senior Architect michael.lee@achieveinternet.com www.achieveinternet.com mlee11111 @ach_mikelee /pub/michael-lee/4/297/a6b Direct: 858-453-5760