Ansible is one of the new breed of tools that encompasses configuration management, orchestration and software defined infrastructure. Find out how many companies are spinning up entire environments from source code including vm's, networks, dns, firewalls, load balancers etc. Video: https://www.youtube.com/watch?v=unPVe2pcego Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London Follow DOXLON on twitter http://www.twitter.com/doxlon

1. Push
Infrastructure
with
Ansible
Paul
Angus
Cloud
Architect
&
Chief
Technology
Strategist
ShapeBlue
paul.angus@shapeblue.com
Twitter:
@CloudyAngus

2. @CloudyAngus
v How
we
use
Ansible
to
push
out
entire
cloud
infrastructures
v A
bit
of
background
v How
we
use
Ansible
v Work
flow
v Code
snippets
Push
Infrastructure
with
Ansible

3. @CloudyAngus
v Cloud
Architect
&
Chief
Technology
Strategist
for
ShapeBlue
v Apache
CloudStack
Committer
v Specialise
in….
v Designing
and
deploying
enterprise
and
public
clouds
v Helping
organisations
use
their
cloud
v Involved
with
CloudStack
before
donation
to
Apache
Foundation
v Designed
clouds
for
Orange,
TomTom,
PaddyPower,
Ascenty,
BSkyB
About
Me
Unofficial
Ansible
Evangelist/Cheerleader

4. @CloudyAngus
“ShapeBlue
are
expert
builders
of
public
&
private
clouds.
They
are
the
leading
global
Apache
CloudStack
integrator
&
consultancy”
About
ShapeBlue

6. @CloudyAngus
Push
Infrastructure
with
Ansible
What
is
CloudStack?

7. @CloudyAngus
CloudStack
is
a
open
source
IaaS
platform.
Hypervisor
agnostic
-­‐
KVM,
vSphere,
XenServer,
LXC,
OVM,
Baremetal
CloudStack
orchestrates
hypervisors
and
network
appliances
to
give
simple
control
to
complex
tasks
through
API
or
web
GUI.
(Yes,
it’s
like
OpenStack)
What
is
CloudStack

8. @CloudyAngus
Think
‘your
own
Amazon
Web
Services’
Public
clouds
(SPs/MSPs)
General
public
can
create
or
log
into
instances
themselves
Private
Clouds
(Enterprises)
Anyone
who
wants
to
be
able
to
orchestrate
their
environment
Hybrid
Clouds
(Enterprises)
Balance/share
load
between
their
own
DC
and
a
Public
Cloud
Who
uses
CloudStack

9. @CloudyAngus
This
is
a
UI
(for
Linux
guys
who’ve
forgotten).
What
is
CloudStack

10. @CloudyAngus
It’s
OK
it
has
an
API
cloudmonkey create network name=DemoNet displaytext=DemoNet zoneid=$ZONEID
networkofferingid=$NEWORKOFFERINGID account=$ACCOUNTNAME domainid=$DOMAINID
cloudmonkey list networks domainid="$DOMAINID" filter=id,name | jq '.network[] |
select(.name=="DemoNet") | .id‘
cloudmonkey deploy virtualmachine networkids="$NETWORKID"
serviceofferingid="$SERVICEOFFERINGID" templateid="$TEMPLATEID" zoneid="$ZONEID"
account="$ACCOUNTNAME" domainid="$DOMAINID" startvm=true displayname=HTTPServer1
displayvm=true name=HTTPServer1
What
is
CloudStack

11. @CloudyAngus
Push
Infrastructure
with
Ansible
Why?
(Duuurrr)

12. @CloudyAngus
CEO
Why
“Building
CloudStack
environments
using
Ansible?
Are
you
just
having
fun
with
Ansible?”

13. @CloudyAngus
Why
Talented
Cloud
Architect
“Noooo,
if
we
can
automate
the
building
of
environments
using
a
powerful,
simple
and
agentless
technology
we
can
make
building
at
scale
easy
while
ensuring
that
our
results
are
consistent
and
repeatable.”
“Building
CloudStack
environments
using
Ansible?
Are
you
just
having
fun
with
Ansible?”

14. @CloudyAngus
Why
CEO
“That
would
be
excellent.
Go
ahead.
Oh,
and
here’s
a
pay
rise”
“Noooo,
if
we
can
automate
the
building
of
environments
using
a
powerful,
simple
and
agentless
technology
we
can
make
building
at
scale
easy
while
ensuring
that
our
results
are
consistent
and
repeatable.”

15. @CloudyAngus
Some
of
that
might
actually
have
happened.
Disclaimer
Why

16. @CloudyAngus
CSForge™
v CSForge
delivers
the
rapid
deployment
of
a
standardised
CloudStack
powered
IaaS
cloud
for
small
production
deployments,
or
medium
scale
POCs
or
pilots.
The
framework
can
be
used
as
a
basis
for
public
cloud
or
enterprise
private
cloud
deployments
Production
v Cloud-­‐scale
environments
initial
deployment
typically
24
–
100s
of
hosts
v Often
multi-­‐hypervisor
Why

17. @CloudyAngus
Test/Dev
Need
to
be
able
to
create
full
environments
to
test:
v CloudStack
release
candidates
v CloudStack
features
v ShapeBlue
patches
Why

18. @CloudyAngus
Why
Ansible
v Technical:
v Client/Server
architecture
not
required
v Only
SSH
connectivity
required
(password
or
public/private
keys)
v …making
it
easier
to
build
virgin
environments
v Modules
can
be
in
any
language
capable
of
returning
JSON
or
key=value
text
pairs
v Has
an
API
v User:
v Much
shallower
learning
curve
v Don’t
need
to
learn
a
programming
language
(i.e.
Ruby)
v Not
as
many
pre-­‐existing
playbooks
(recipes/manifests)
about,
but
improving
with
Ansible
Galaxy

19. @CloudyAngus
Typical
Logical
Production
Topology
Management (1Gb)
CIMC (1GB)
CIMC (1GB)
Management (1GB LACP Bond)
WWW
Compute Hosts
Storage Nodes
Guest
Public
Storage
(10GB LACP Bond)
ManagementHosts
Management
1GB Active/Passive Bond
Load Balancers
DNS/NTP Servers
MySQL Master & Slave
ACS Management Servers
Deployment
Server
Management
(1GB Active/Passive Bond)
Storage
(10GB LACP Bond)
iDRAC (1GB)
Storage (10GB LACP Bond)
Public
Cloud Compute Hosts
Storage Link to ACS Managers

20. @CloudyAngus
v 3
zones
v 2
geographic
locations
v Upgrade
done,
then
tests
run
for
a
week.
Then
VRs
restarted
Client
Test
Environment
CCP3.0.7B
MySQL
CPBM 2.2
MySQL
ESXi 1a
ESXi 1b
ESXi 1c
vCenter
Appliance
ESXi 2a
ESXi 2b
ESXi 2c
vCenter
Appliance
Zone 1 (local)
Zone 2 (local)
Zone 3 (remote)
NFS
NFS
VPN VPN
ESXi 2a
ESXi 2b
ESXi 2c
vCenter
Appliance
NFS

21. @CloudyAngus
Using
Ansible
to
Deploy
CloudStack
Environments
How?
(variables,
I
Love
‘em)

22. @CloudyAngus
#
Copyright
(C)
ShapeBlue
Ltd
-­‐
All
Rights
Reserved
#
Unauthorized
copying
of
this
file,
via
any
medium
is
strictly
prohibited
#
Proprietary
and
confidential
#
Released
by
ShapeBlue
<info@shapeblue.com>,
April
2014
-­‐-­‐-­‐
-­‐
name:
Ensure
selinux
python
bindings
are
installed
yum:
name=libselinux-­‐python
state=present
-­‐
name:
Ensure
the
Apache
Cloudstack
Repo
file
is
configured
template:
src=cloudstack.repo.j2
dest=/etc/yum.repos.d/cloudstack.repo
-­‐
name:
Ensure
selinux
is
to
permissive
command:
setenforce
permissive
changed_when:
false
-­‐
name:
Ensure
selinux
is
set
permanently
selinux:
policy=targeted
state=permissive
-­‐
name:
Ensure
CloudStack
packages
are
installed
yum:
name=cloudstack-­‐management
state=present
-­‐
name:
Ensure
MySQL
Client
is
present
yum:
name=mysql
state=present
-­‐
name:
Ensure
vhd-­‐util
is
present
get_url:
url="{{
vhdutil_url
}}"
dest=/usr/share/cloudstack-­‐common/
scripts/vm/hypervisor/xenserver/vhd-­‐util
mode=0755
-­‐
name:
Ensure
CloudStack
Usage
Service
is
installed
yum:
name=cloudstack-­‐usage
state=present
-­‐
name:
Ensure
CloudStack
Usage
Service
is
started
service:
name=cloudstack-­‐usage
state=started
-­‐
include:
./../galera-­‐cluster/tasks/main.yml
when:
"{{
db_type
}}
==
'galera'"
-­‐
include:
./setupdb.yml
CloudStack
Management
Server
Role

23. @CloudyAngus
#
Copyright
(C)
ShapeBlue
Ltd
-­‐
All
Rights
Reserved
#
Unauthorized
copying
of
this
file,
via
any
medium
is
strictly
prohibited
#
Proprietary
and
confidential
#
Released
by
ShapeBlue
<info@shapeblue.com>,
April
2014
[cloudstack]
name=cloudstack
baseurl=http://{{
acs_build_repo
}}/{{
acs_build_path
}}
enabled=1
gpgcheck=1
cloudstack.repo.j2

24. @CloudyAngus
# ansible-­‐playbook management_vms_build.yml -­‐i acs_demo_hosts -­‐-­‐
extra-­‐vars "acs_build_path=tescng/centos/master"
Playbook
Arguments

25. @CloudyAngus
Create
bare
VM
+
yum
install
git
git
clone
mega
repo
with
roles
etc.
Install
&
configure
Ansible
(included
in
repo)
Update
hosts
and
group_vars
Create
Deployment
server
(from
role)
locally
PXE
boot
hosts/
mgmt
VMs
to
bare
OS
Push
application
configuration
to
VMs
and
Hosts
Building
Environments

26. @CloudyAngus
Distributed
Architecture
TEST STAGE PROD
Deployment
Srvr
with
cloned
repo
Deployment
Srvr
with
cloned
repo
Deployment
Srvr
with
cloned
repo

27. @CloudyAngus
v PXE
VM
against
Ansible
created
KickStart
file
v DNS
Servers
(Bind
9)
v Galera
Cluster
(Maria
DB10
–
any
size)
v MySQL
Master/Slave
v Clustered
CloudStack
Management
Servers
v HA
Proxy
(HA
pair
with
keepalived)
Management
Infrastructure
Roles

28. @CloudyAngus
v PXE
host
against
Ansible
created
KickStart
or
Answer
file
v KVM
v Straight
configuration
of
networking
and
packages
v XenServer
v XE
commands
via
shell
v Use
‘facts’
to
determine
installed
patches
v vSphere
v 99%
configured
through
kickstart
file
v SSH
enabled
(through
KickStart)
Hypervisor
Host
Roles

29. @CloudyAngus
Using
Ansible
to
Deploy
CloudStack
Environments
Snippets

30. @CloudyAngus
[management_hosts:children]
cloudstack_manager_hosts
loadbalancer_hosts
deployment_hosts
db_hosts
[kvm_hosts:children]
pod1_cluster1
pod1_cluster2
[pod1_cluster1]
10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-­‐p1-­‐c1-­‐1 mgmt_ip=10.101.1.13
mgmt_netmask=255.255.255.0
Host
File

31. @CloudyAngus
Host
File
[mysql_master_host]
10.101.1.6 pxemac=00:50:56:8E:19:20 hostname=mysqlmasterhost mgmt_ip=10.101.1.6
mgmt_netmask=255.255.255.0 serverid=1
[pod1_cluster1]
10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-­‐p1-­‐c1-­‐1 mgmt_ip=10.101.1.13
10.101.1.14 pxemac=00:1c:c0:ec:b5:43 hostname=kvm-­‐p1-­‐c1-­‐2 mgmt_ip=10.101.1.14
Creating
PXE
files

32. @CloudyAngus
File
format:
AA:BB:CC:DD:EE:FF
must
be
01-­‐aa-­‐bb-­‐cc-­‐dd-­‐ee-­‐ff
-­‐ name: create centos pxeconfig files
template: src=pxelinuxcfg-­‐centos6.j2 dest=/ptpboot/pxelinux.cfg/01-­‐{{ hostvars[item]['pxemac'] | lower |
replace(":","-­‐") }}
with_items:
-­‐ "{{ groups.management_hosts }}“
-­‐ name: create KVM pxeconfig files
template: src=pxelinuxcfg-­‐kvm.j2 dest=/ptpboot/pxelinux.cfg/01-­‐{{ hostvars[item]['pxemac'] | lower |
replace(":","-­‐") }}
with_items:
-­‐ "{{ groups.kvm_hosts }}"
Creating
PXE
files

33. @CloudyAngus
# Ansible built pxeconfig script
DEFAULT menu.c32
menu ctle ShapeBlue CSForge Deployment
cmeout 10
LABEL {{ centos_iso_version }}-­‐KVM
kernel htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}/
isolinux/vmlinuz
append initrd=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/
{{ centos_iso_version }}/isolinux/initrd.img ramdisk_size=15491 ip=dhcp ksdevice={{ ks_device }}
ks=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ pxe_files_dir }}/{{ kvm_ks_file }}
MENU LABEL {{ centos_iso_version }} for KVM
pxelinuxcfg-­‐kvm.j2

34. @CloudyAngus
# Specifies the keyboard layout
keyboard {{ keyboard_lang }}
# Used with an HTTP install to specify where the install files are located
url -­‐-­‐url htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}
# Assign a stacc IP address upon first boot & set the hostname
network -­‐-­‐onboot yes -­‐-­‐device {{ ks_device }} -­‐-­‐bootproto dhcp -­‐-­‐noipv6
# Set the root password
rootpw {{ mgmt_root_password }}
mkdir /root/.ssh
curl htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ publickey_file_name }} >> /root/.ssh/authorized_keys
kickstart.j2

35. @CloudyAngus
opcon subnet-­‐mask {{ dhcp_netmask }};
opcon routers {{ dhcp_router }};
opcon broadcast-­‐address {{ dhcp_broadcast }};
opcon domain-­‐name-­‐servers {{ int_dns_servers | join(", ") }};
opcon domain-­‐name "{{ dns_domain }}";
subnet {{ dhcp_subnet }} netmask {{ dhcp_netmask }} {
range {{ dhcp_range_starcp }} {{ dhcp_range_endip }};
# KVM Hosts
{% for host in groups['kvm_hosts'] %}
host {{ hostvars[host]['hostname'] }} {
hardware ethernet {{ hostvars[host]['pxemac'] | lower }};
fixed-­‐address {{ hostvars[host]['mgmt_ip'] }};
}
{% endfor %}
Creating
dhcpd.conf

36. @CloudyAngus
v Reverse Zone – Format:
2 IN PTR blah1.domain.com.
; Address Records
{% for host in groups['management_hosts'] %}
{{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}.
{% endfor %}
{% for host in groups['xenserver_hosts'] %}
{{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}.
{% endfor %}
DNS
Entries
into
Zone
File
–
Jinja2

37. @CloudyAngus
v Custom
Filter
to
Split
string
import re
def split_string(string, seperator=' '):
return string.split(seperator)
def split_regex(string, seperator_patern):
return re.split(seperator_patern, string)
class FilterModule(object):
''' A filter to split a string into a list. '''
def filters(self):
return {
'split' : split_string,
'split_regex' : split_regex,
}
Custom
Filter

38. @CloudyAngus
v ansible.cfg
# set plugin path directories here, separate with colons
accon_plugins = /usr/share/ansible_plugins/accon_plugins
callback_plugins = /usr/share/ansible_plugins/callback_plugins
conneccon_plugins = /usr/share/ansible_plugins/conneccon_plugins
lookup_plugins = /usr/share/ansible_plugins/lookup_plugins
vars_plugins = /usr/share/ansible_plugins/vars_plugins
filter_plugins = /usr/share/ansible_plugins/filter_plugins:/CSForge/custom_plugins/filter_plugins
Add
path
if
required

39. @CloudyAngus
v ACS_env_group_vars.yml
db_type: galera
# opcons: mysql, mysql-­‐masterslave, galera
v build_db_servers.yml
-­‐-­‐-­‐
-­‐ hosts: db_hosts
roles:
-­‐ { role: galera-­‐cluster, when: "'{{ db_type }}' == 'galera'" }
-­‐ { role: mysql, when: "'{{ db_type }}' != 'galera'" }
Conditional
Roles

40. Using
Ansible
to
Deploy
CloudStack
Environments
Paul
Angus
Cloud
Architect
ShapeBlue
paul.angus@shapeblue.com
Twitter:
@CloudyAngus