Virtualized development environments phpne - may 2012

Virtualized Development
Environments with Vagrant

Ian Chilton
PHP North East - 14th May 2012

Introduction

Ian Chilton
Better Brand Agency
http://www.betterbrandagency.com
http://www.bettercommunities.co.uk

Twitter: @ichilton
Web: http://www.ichilton.co.uk
LinkedIn: http://uk.linkedin.com/in/ichilton

Contents

- Introduction to:

- Development Environments

- Virtualization

- Why Virtualize your development environment?

- Introduction to Vagrant

- Using Vagrant

- Vagrant Plugins

- Automated Provisioning (Conﬁguration Management)

Development Environments

We don’t want to be working on live sites.



So we setup a development environment.




DEV Staging

VCS LIVE




DEV
DEV Staging Staging
VCS DEV
VCS LIVE LIVE
DEV




DEV
DEV Staging Staging
VCS DEV
VCS LIVE LIVE
DEV
We work on development, push/pull
to/from version control and deploy to
staging and live.




DEV
DEV Staging Staging
VCS DEV
VCS LIVE LIVE
DEV
We work on development, push/pull
to/from version control and deploy to
staging and live.
It’s DEV we’re going to be talking about here.


Number of (valid) ways:



- Copy of site (on live server)
- Can easily affect the live site.



- Copy of site (on live server)
- Can easily affect the live site.

- Shared development server - Local / Remote / Dedicated / Cloud / VPS

- Cost
- Remote = latency / connection failure
- Setup / management / maintenance
- All developers and all projects sharing same hardware,
software, versions etc.


Number of (valid) ways: - continued



- Develop Locally:
- MAMP (mac) or WAMP (windows)
- Homebrew (mac), apt-get etc (linux) to install services locally



- Develop Locally:
- MAMP (mac) or WAMP (windows)
- Homebrew (mac), apt-get etc (linux) to install services locally

- Self sufﬁcient - no remote latency / connection failure

- However...

- Clutter on workstation
- Still shared between projects, set versions etc...
- Different OS / platform / architecture


Consider these scenarios (just some examples of many...)



The methods on the previous slides could install a different (much
older or newer) version of a service like MySQL or PHP than the
production box.

You could then inadvertently code with something only added in a
later version, or use something which had bugs in earlier
versions.

You’d there for have code which works in development but not
when deployed to production.



Both Windows and OS X use case insensitive filesystems.

Imagine writing some code which used MyFile.txt.

You could call this file myfile.txt and it would work on Windows/Mac.

On Linux that would either error or be a completely different file.

Virtualization

Such bugs can be very difﬁcult and time consuming to track down.

Virtualization

Such bugs can be very difﬁcult and time consuming to track down.

The lesson...

We really need to develop on the same operating
system, architecture, package versions and
conﬁguration as production.

Virtualization

Virtualization is a technology where by a number of “virtual
machines” can be hosted on top of a single machine.

Virtualization


The software running on the virtual machines (guests) is separated
from the software running on the “host”.

Virtualization



The software used to make this possible is called a “hypervisor”.

Virtualization



The software used to make this possible is called a “hypervisor”.

This allows us to run different software on different virtual machines -
for example you could run Windows and Linux virtual machines on a
Mac.

Types of Virtualization

Full Virtualization

Almost complete simulation of the actual hardware.

This allows the guest operating systems to run unmodiﬁed.

Eg:

- Oracle Virtualbox
- Free virtualization product for the desktop
- Cross platform - Windows, OS X, Linux
- Lots of features

- VMWare
- Commercial (££)
- Workstation and Server products


Paravirtualization

The hardware environment is not simulated.

The guest operating system is specially modiﬁed to allow the
hypervisor to run the guests in their own isolated domain.

Eg:

- Xen
which is used by:

- Amazon EC2
- Rackspace Cloud
- Some VPS providers


Hardware Assisted Virtualization

Modern Intel or AMD CPU’s contain native virtualization functionality
on the CPU - Intel VT-x and AMD-V.

Eg:

- Linux KVM (Kernel Virtual Machine)
which is used by:
- Bytemark
- Brightbox
- VPS providers

- Xen, Virtualbox & VMWare also support this as well, for hosts that
have the hardware support.


Operating System Level Virtualization (aka. containers)

The virtualization is implemented in software, on the host operating
system/kernel.

The guest operating systems share the kernel of the host, but run
the guest operating system independently of the host.

This allows isolated virtual machines and different distributions, but
is generally limited to Unix based operating systems.

Eg:
- OpenVZ
- LXC on Linux
- FreeBSD jails

Often used by budget VPS providers as they can over-sell.

Advantages of Using Virtualization

Advantages of Using Virtualization

- We can run the exact same operating system and setup as the
production boxes, on top of any other machine, be it Mac, Windows
or Linux.

- We can install the same software / packages / versions.

- Everything is run in a contained sandbox and resources can be
managed.

- We can create distributable images to share.

- If we run the VM’s locally, we can develop ofﬂine.

Disadvantages of Using Virtualization


- Many of the products we’ve mentioned require Linux (or another
UNIX operating system) and don’t run on Mac or Windows.

- Require searching around for operating system images or installing
the operating system on the VM manually.

- Can be complicated/time consuming to learn, setup and conﬁgure.


- Many of the products we’ve mentioned require Linux (or another
UNIX operating system) and don’t run on Mac or Windows.

- Require searching around for operating system images or installing
the operating system on the VM manually.

- Can be complicated/time consuming to learn, setup and conﬁgure.

What we need is a simple and quick way of
bringing up virtual machines with minimal fuss.

Vagrant

Created by Mitchell Hashimoto and John Bender

Command line interface/wrapper around Oracle’s free VirtualBox
software (support for other hypervisors are in the works).

Allows virtual machines to be created, inside of a minute, from pre-
provided images, with a few simple commands.

http://vagrantup.com

Vagrant

We can use vagrant machines for:

- Development
- Testing
- Experimenting new/different software/settings
- Testing conﬁguration management systems (explained later).

http://vagrantup.com

Apprehensions

- Won’t it be slow?

- Headless Linux VM’s aren’t as resource hungry as a Windows
virtualized.

- I want to use my local graphical editor...

- Vagrant automatically creates a shared directory on the host.

Install Vagrant

- Install Oracle Virtualbox (free from: http://virtualbox.org) - versions
for Windows, Mac OS X or Linux.

• Plus the extension pack.

- Install Vagrant (free from: http://downloads.vagrantup.com)

Still available as a Ruby gem but as of the recent v1.0, installers are
provided for Windows, Mac and Linux (deb and rpm).

$ vagrant -v
Vagrant version 1.0.3

Base Boxes

Pre-packaged operating system images.

Ofﬁcial base boxes - Ubuntu LTS (long term support) releases.

Ubuntu 10.04 (Lucid Lynx)

$ vagrant box add lucid32 http://files.vagrantup.com/lucid32.box

$ vagrant box add lucid64 http://files.vagrantup.com/lucid64.box

Ubuntu 12.04 (Precise Pangolin):

$ vagrant box add precise32 http://files.vagrantup.com/precise32.box

$ vagrant box add precise64 http://files.vagrantup.com/precise64.box

Basic Vagrant Usage
$ mkdir vagrant
$ cd vagrant

$ mkdir mydev
$ cd mydev

Create a new conﬁg ﬁle (VagrantFile) for this instance:

$ vagrant init lucid32

Boot the machine (create from base box the 1st time):

$ vagrant up

SSH in:

$ vagrant ssh

Basic Vagrant Usage
Vagrant automatically maps /vagrant inside the VM to the directory
on the host:
vagrant@vagrant:~$ ls /vagrant
index.html Vagrantfile

Ctrl-d exits the SSH session and takes us back to the host.

Suspend the machine:
$ vagrant suspend
$ vagrant resume

Shut down the machine:
$ vagrant halt

Destroy the machine:
$ vagrant destroy

Vagrant Conﬁguration

The “vagrant init” created a VagrantFile for this VM.

Vagrant::Config.run do |config|
config.vm.box = "lucid32"
end

We can edit this to add additional options.


GUI or Headless?:


# Boot with a GUI so you can see the screen.
# (Default is headless)

# config.vm.boot_mode = :gui
end


Port Forwarding:


# Map port 8080 on the host to port 80 in the VM:
config.vm.forward_port 80, 8080
end


Control VirtualBox:


# Make the VM have 1GB RAM:
config.vm.customize ["modifyvm", :id, "--memory", 1024]
end


Multiple Machines:

# Web Server:
config.vm.define :web do |web_config|
web_config.vm.box = "web"
web_config.vm.forward_port 80, 8080
end

# Database Server:
config.vm.define :db do |db_config|
db_config.vm.box = "db"
db_config.vm.forward_port 3306, 3306
end
end


Networking:

- Host Only - local communication between different VM’s.

config.vm.network :hostonly, “192.168.50.1”
end

- Bridged - bridge the VM onto the host’s network so we can access anything
on the network, and they can access the VM as if it was a normal machine on
the network.

config.vm.network :bridged
end


Packaging:

Use as a starting point, or distribute instances.

$ vagrant package --vagrantfile Vagrantfile.pkg

The --vagrantfile Vagrantfile.pkg option tells Vagrant to take the options placed
in Vagrantfile.pkg and add them into the box so things like port forwarding will
be done automatically.

We can then use this as normal:

$ vagrant box add my_box /path/to/the/package.box
$ vagrant init my_box
$ vagrant up

Vagrant Plugins

Vagrant Box.es - Gareth Rushgrove has created a site at: http://vagrantbox.es
which is a list of links to base boxes that people have built and made public
(currently 49 boxes).

He has also created a vagrant plugin to search, view and install them:
https://github.com/garethr/ruby-vagrantboxes

Sahara - allow vagrant to run in a sandbox so any changes can be rolled back:
https://github.com/jedi4ever/sahara

Vagrant Snap - allow snapshots and supports a multi vm environment:
https://github.com/t9md/vagrant-snap

Vagrant Hosts - automatically update a dns server with vagrant hostnames:
https://github.com/BerlinVagrant/vagrant-dns

Vagrant plugin for Jenkins:
http://unethicalblogger.com/2012/03/13/introducing-the-vagrant-plugin.html

Vagrant Plugins

VeeWee - https://github.com/jedi4ever/veewee

Build custom base boxes from scratch.

Includes templates for popular distributions - CentOS, Debian, Fedora,
ArchLinux, Gentoo, openSUSE, Ubuntu and even FreeBSD.

Vagrant Plugins


$ veewee vbox define 'myubuntubox' 'ubuntu-12.04-server-i386'

The basebox 'myubuntubox' has been succesfully created from
the template 'ubuntu-12.04-server-i386'

You can now edit the definition files stored in definitions/
myubuntubox or build the box with:
veewee vbox build 'myubuntubox'

$ ls definitions/myubuntubox
definition.rb postinstall.sh preseed.cfg

Vagrant Plugins


Here, we can customise the build.

- definition.rb - contains the parameters for the box: cpu’s, memory, disk, O/S, ISO file
and url, boot sequence, SSH details etc.

- preseed.cfg - This is the file the Debian/Ubuntu installer uses to automate the
installation.

- postinstall.sh - This is a shell script which is ran on the VM after the installation. In this
case, it will:

•Upgrade the O/S (apt-get update && apt-get dist-upgrade)
•Install the dependancies for Ruby.
•Install (or build from source) Ruby.
•Install Puppet and Chef
•Create a vagrant user and add in the standard Vagrant SSH public key into ~/.ssh/
authorized_keys.

Vagrant Plugins


We now build the box:

$ veewee vbox build 'myubuntubox'

- Download’s ISO the first time.

- Starts a mini web server to serve the preseed.cfg file.

- Creates a VirtualBox VM and boots the ISO.

- At the boot prompt, it enters a custom boot prompt which points it to the url
for the preseed.cfg.

- The install complete’s automatically (using the preseed.cfg file) and veewee
just waits until it can successfully SSH to the VM.

Vagrant Plugins


Once the build is complete (takes some time), we can run some automated
Cucumber tests against the box:

$ veewee vbox validate 'myubuntubox'
...
7 scenarios (7 passed)
21 steps (21 passed)
0m14.810s

This checks the box is working and that Ruby, Ruby Gems, Chef, Puppet etc
is responding as intended.

If so, we can then build the box and add it into Vagrant:

$ vagrant basebox export 'myubuntubox'

Automated Provisioning (Conﬁguration Management)


We’ve now got the ability to quickly spin up virtual machines as we require
them.

However, that only gets us a clean install - we’ve still got to start installing
packages and conﬁguring things how we like them.

If we are looking to share environments with colleagues etc, the problem with
pre-packaged boxes/images is that they get out of date so they either have to
be periodically updated or extra steps added in whenever they are used.

This also doesn’t just apply to development - what if we have tens or hundreds of
production servers?

- How can we ensure that every time we setup a new server, all of the correct setup steps
are followed?

- How can we ensure that the development, staging and production servers are all setup
the same.

- What if we have a failure and it takes someone hours of running manual commands to
build a new server? - what if a whole platform of tens or hundreds of servers fail?
(connectivity or business)

- How boring is it going to get for someone manually setting up servers?

- We could document it, but will it be kept up-to-date?

- What about maintaining the boxes on an ongoing basis? - how do we keep them up to
date? when someone new starts, how long will it take to go round a lot of servers and add
their user and ssh key or to apply a ﬁrewall change to all servers?

This is where something called “Infrastructure as code” comes in.
What if we were to do our server setup and management as we do
development:

- Write code (which also acts as documentation)

- Put the code into version control so we can:
- See and analyse changes, who made them and when.
- Have branches and tags.
- Easily roll back to old versions.

- Run automated tests with Rspec and Cucumber to ensure everything is
working correctly without manually checking each server.

- TDD - Test Driven Development

This ensures that server setup is consistent, reproducible, fast, mistake free
and automatable.

In the past, people would write shell scripts to do this. However, this is difficult
to manage and maintain.

There are now a number of “configuration management” tools written
specifically for this purpose.

The most popular and widely used of these is Puppet and Chef.

Puppet:

Puppet Labs - founded by Luke Kanies in 2005.

Uses a declarative language to describe system conﬁguration.

Widely used and trusted by a number of large enterprise users -
Wikimedia, Dell, Rackspace, Zynga, Twitter, New York Stock Exchange,
Disney, Citrix, Oracle, Stanford University, Google and GovUK.

James Turnbull (works for Puppet Labs) has written two books about
Puppet which are published through Apress:

- Pulling strings with Puppet
- Pro Puppet

-“Puppet 2.7 Cookbook” by John Arundel (Packtpub)

Chef:
Opscode (2009)

Recipes written in pure Ruby.

Opscode provides a “Hosted Chef” service to avoid running a local Chef server (like
Github is to Git).

Cookbooks - sharable
! ! (although Puppet now has “Puppet Forge”)

Knife - feature rich command line tool.

Documentation improving:
- Wiki
- Stephen Nelson Smith, author of “Test Driven Infrastructure with Chef” is working
on “Chef: The Deﬁnitive Guide” which is due to be published in September.
- Podcast: http://www.foodﬁghtshow.org/2012/03/episode-5-getting-started-with-
chef.html

Both products can run in both a standalone mode or a client-server model -
Vagrant supports both of these.

In the client/server model, the client will automatically “phone home” every so
often to check for changes.

Idempotence:

Run multiple times without affecting the result.

i.e:

- Instead of saying “Install Apache”, we say “Apache should be installed”.

- Instead of “chown root.root myﬁle” we say “myﬁle should have this owner /
group and these permissions.

Simple Puppet Example:

$ mkdir manifests

In the VagrantFile:

config.vm.provision :puppet

OR

config.vm.provision :puppet do |puppet|
puppet.manifests_path = "manifests"
puppet.manifest_file = "my_manifest.pp"
end
end


In manifests/default.pp:

class apache {

package { "apache2":
ensure => present,
}

service { "apache2":
ensure => running,
require => Package["apache2"],
}
}

include apache


file { "/etc/apache/apache.conf":
owner => root,
group => root,
mode => 644,
source => "puppet:////vagrant/puppet/files/apache.conf",
notify => Service["apache2"]
}

file { "/etc/apache2/sites-available/vagrantsite":
owner => root,
group => root,
mode => 644,
source => "puppet:////vagrant/puppet/files/vhost.conf",
}


file { "/etc/apache2/sites-enabled/vagrantsite":
ensure => symlink,
target => "/etc/apache2/sites-available/vagrantsite",
}

file { "/etc/apache2/sites-enabled/default":
ensure => absent,
}

More Information
Vagrant:

http://www.vagrantup.com

Google Group: http://groups.google.com/group/vagrant-up

#vagrant on irc.freenode.net

Puppet:

http://puppetlabs.com
#puppet on irc.freenode.net

http://www.puppetcookbook.com

Chef:

http://www.opscode.com
#chef on irc.feenode.net

Other:

http://devopsweekly.com

Virtualized development environments phpne - may 2012

More Related Content

What's hot

Similar to Virtualized development environments phpne - may 2012

Recently uploaded

Virtualized development environments phpne - may 2012

Editor's Notes