This document discusses assembling an open source tool chain for a hybrid cloud environment. It describes using Packer to build machine images for multiple platforms like AWS, VMware, and VirtualBox from a single blueprint. It also discusses using Vagrant and Ansible for automation, configuration management, and provisioning virtual machines across different cloud providers in a standardized way.

1. Assembling an Open Source Tool
Chain for the Hybrid Cloud
Isaac Christoffersen
Bit Herder & Cloud Wrangler
@ichristo

2. Bit Herder, Cloud Wrangler
● Geek,
Insomniac,
Tinkerer
● Wearer
of
Many
Hats
● Delivering
Open
Source
Solutions
for
10+
years
● Solution
Architect
at
Vizuri

3. Why
the
Hybrid
Cloud?

4. Our
App
is
going
to
change
the
world
as
we
know
it!
“The
best
way
to
predict
the
future
is
to
invent
it”
-­‐-­‐
Ed
Catmull

5. Unleash the System Administrators!
source
-­‐
http://commons.wikimedia.org/wiki/

6. Word of Mouth Spreads. Popularity Rises.

7. source
-­‐
http://commons.wikimedia.org/wiki/File:Ford_assembly_line_-­‐_1913.jpg
More Sys Admins and a Little Automation

8. SlashDot / Oprah / Twitter Effect ...

9. We Need More DevOps!

10. Your App Has Gone Global!

11. All Hail the Fail Whale!
source
-­‐
http://www.whatisfailwhale.info/

12. But We Had DevOps?!

13. source
-­‐
http://upload.wikimedia.org/wikipedia/commons/e/e0/Nagasakibomb.jpg
Cloud Bursting?

14. Assembling
the
Hybrid
Cloud
Tool
Chain

16. Automatically create
machine images for
multiple platforms
from a single blueprint

17. AWS: AMI
VMware: VMX + disks
VirtualBox: OVF + disks
DigitalOcean: Snapshots
and many more …
http://www.packer.io/docs/templates/builders.html

18. BoxCutter (github.com/boxcutter)
• Build VirtualBox, VMWare, and Parallels Images
• Choose your desired CM tool (Puppet, Chef,
Ansible)
Bento (https://github.com/chef/bento)
• Used by Chef Release Team for Testing
• Chef specific Configuration Managment

19. Automation
No human interaction. Great for
Continuous Integration / Deployment
Standardization
Use Puppet, Chef, Ansible, Bash to
configure the image
Repeatability
Template goes into version control
Image creation knowledge is now in code
Anyone can build / rebuild the base images

20. EXAMPLE
CentOS Image in both AWS
& Digital Ocean

21. {
"builders": [
{
"type"
"access_key"
"secret_key"
"region"
"source_ami"
"security_group_id"
"instance_type"
"ssh_username"
"ssh_timeout"
"ami_name"
},
…
: "amazon-ebs",
: "{{user `aws_access_key`}}",
: "{{user `aws_secret_key`}}",
: "us-east-1",
: "ami-8997afe0",
: "sg-8f7e24e4",
: "t1.micro",
: "ec2-user",
: "5m",
: "centos-baseline {{timestamp}}"

22. …
{
"type"
"api_key"
"client_id"
"image_id"
: "digitalocean",
: "{{user `do_api_key`}}",
: "{{user `do_client_id`}}",
: "562354",
"snapshot_name" : "centos-baseline {{timestamp}}"
}
]
…
}

23. $ packer validate base-image.json
Template validated successfully.
$ packer build base-image.json
amazon-ebs output will be in this color.
digitalocean output will be in this color.
…
==> amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
Build 'digitalocean' finished.
==> Builds finished. The artifacts of successful builds are:
--> digitalocean: A snapshot was created: 'centos-baseline
1396457723' in region 'New York 1'

24. EXAMPLE
BoxCutter Provisioning of
Centos to VMWare

25. Isaacs-MBP-2:centos-vm isaac$
make list
Prepend 'vmware/', 'virtualbox/',
or 'parallels/' to build only one
target platform:
make vmware/centos66
Targets:
…
centos66-desktop
centos66-docker
centos66-i386
centos66
centos70-desktop
centos70-docker
centos70
Isaacs-MBP-2:centos-vm isaac$

26. isaac$ make vmware/centos70-docker
rm -rf output-vmware-iso
mkdir -p box/vmware
packer build -only=vmware-iso -var 'cm=nocm’ -var
'headless=' -var 'update=' -var 'version=1.0.10' -var
'ssh_username=vagrant' -var 'ssh_password=vagrant' -
var 'install_vagrant_key=true' -var "iso_url=http://
mirrors.sonic.net/centos/7/isos/x86_64/CentOS-7-x86_64-
DVD-1503-01.iso" centos70-docker.json
vmware-iso output will be in this color.
…

27. …
==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: http://
mirrors.sonic.net/centos/7/isos/x86_64/CentOS-7-x86_64-
DVD-1503-01.iso
==> vmware-iso: Creating floppy disk...
vmware-iso: Copying: floppy/
vmware9.compat_dcache.h.patch
==> vmware-iso: Creating virtual machine disk
==> vmware-iso: Building and writing VMX file
==> vmware-iso: Starting HTTP server on port 8597
==> vmware-iso: Starting virtual machine...
==> vmware-iso: Waiting 10s for boot...
==> vmware-iso: Connecting to VM via VNC
==> vmware-iso: Typing the boot command over VNC...
==> vmware-iso: Waiting for SSH to become available..
…

28. +

29. Mature, stable, proven. Development
since Jan 2010. Used by thousands
of companies.

30. Deploy to Multiple Providers
vagrant up --provider=foo
AWS, DigitalOcean, HP Cloud,
Joyent, KVM, libvirt, lxc,
OpenStack, Rackspace,
Vmware, VirtualBox

31. Leverage your own
Packer Images
• Base image as starting point.
• Integrates with multiple
“provisioners” including
Puppet, Chef, Ansible, Bash

32. Automation
No human interaction. Great for
Continuous Integration / Deployment
Standardization
Use Puppet, Chef, Ansible, Bash to
configure the image
Repeatability
Template goes into version control
Image creation knowledge is now in code
Anyone can build / rebuild the base
images

33. Vagrant.configure("2") do |config|
config.vm.box = "dummy"
config.vm.box_url = "https://github.com/mitchellh/
vagrant-aws/raw/master/dummy.box"
config.vm.provider :aws do |aws, override|
aws.access_key_id = "YOUR KEY"
aws.secret_access_key = "YOUR SECRET KEY"
aws.keypair_name = "KEYPAIR NAME” aws.ami
= ”ami-9baa9cf2”
override.ssh.username = “ec2-user”
override.ssh.forward_agent = true
override.ssh.private_key_path = “YOUR PRIVATE
KEY”
end

34. EXAMPLE
Multiple CentOS Images
created in AWS from base AMI

35. $ vagrant up --provider=aws
Use `vagrant plugin` commands to manage plugins. This warning
will be removed in the next version of Vagrant.
Bringing machine 'test-broker' up with 'aws' provider...
Bringing machine 'test-node-01' up with 'aws' provider...
Bringing machine 'test-node-02' up with 'aws' provider…

37. • Configuration Management tool
like Puppet, Chef, CFEngine
• Quick to get started
• Builds on familiar tools
• Run commands over SSH. No
additional agents required

38. …
config.vm.provision :ansible do |ansible|
ansible.sudo = true
ansible.playbook = "provisioning/ansible/playbook.yml”
ansible.verbose = true
end
- hosts: all
tasks:
- name: ensure ntpd is at the latest version
yum: pkg=ntp state=latest
notify:
- restart ntpd
handlers:
- name: restart ntpd
service: name=ntpd state=restarted

39. EXAMPLE
Provision a CentOS LAMP
Stack in AWS

40. Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
boxes.each do |box|
config.vm.define box[:name], primary: box[:primary] do |config|
config.vm.box = "aws-centos"
config.vm.box_url =
https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
…
config.vm.provision :shell, :privileged => false, :inline => "sudo
yum -y install screen"
config.vm.provision :ansible do |ansible|
ansible.sudo
ansible.playbook
lamp.yml"
ansible.verbose
end
…
= true
= "provisioning/ansible/playbook-
= true

41. - name : Install LAMP Stack
user: ec2-user
hosts: all
tasks:
- name: Install mysql
yum: name=mysql-server state=latest
- name: install httpd
yum: name=httpd
- name: Install php for
mysql yum: name=$item
with_items:
- php
- php-mysql
- mysql-server

42. Only
Scratching
the
Surface

43. So many categories, so little time …
● Cloud Management (Scalr, ManageIQ)
● Log Aggregation (ELK)
● Monitoring Tools (nagios, zenoss)
● Version Control (Git, Subversion)
● CI Servers (Jenkins, TravisCI)
● … and many more

44. OSS DevOps Tools Ecosystem
Check
Out
-­‐
http://www.devopsbookmarks.com/

45. Thank You
Isaac Christoffersen
www.vizuri.com
ichristoffersen@vizuri.com
@ichristo
vagrantup.com packer.io ansible.com