CloudStack EU user group - Trillian

The Cloud Specialists
THE HITCH-HIKERS GUIDE
TO CI/CD
ShapeBlue.com • @ShapeBlue
Paul Angus, VP Technology • @CloudyAngus

The Cloud Specialists ShapeBlue.com @ShapeBlue
“ShapeBlue are expert builders of public
& private clouds. They are the leading
independent global CloudStack services
company”
A b o u t S h a p e B l u e

C l i c k t o e d i t
Paul Angus – VP Technology
• Global authority on CloudStack & cloud infrastructure design.
• 10+ years C-Level experience.
• Apache CloudStack project committer
• Specialising in deployment of CloudStack
and surrounding infrastructure
• USP, Georgian Ministry of Justice, Orange, TomTom,
PaddyPower, Ascenty, BSkyB
A b o u t M e

S h a p e B l u e c u s t o m e r s

Tr i l l i a n
Why?

W h y
“Building CloudStack
environments using CloudStack
and Ansible? Aren’t you just
having fun?”
CEO

W h y
“Building CloudStack environments
using CloudStack?
Are you just having fun?”
Talented Cloud Architect
“If we can automate the
building of environments
including the networking, and
generate any topology we want
quickly, we can achieve the tasks
that I’ve highlighted in these
slides that I have with me, just
in case you asked.”

W h y
building of environments
including the networking, and
generate any topology we want
quickly, we can achieve the tasks
that I’ve highlighted in these
slides that I have with me, just
in case you asked.”
CEO
“That would be excellent.
Go ahead.
Oh, and here’s a pay rise”

We need to test:
• New features
• Hotfixes/patches
• Full Versions
• Physical integrations
• New hypervisors
• Upgrades
• Against a broad range of deployment architectures
• Against a broad range of hypervisor types and
versions
W h y

Many, many, MANY permutations of a
CloudStack deployment….
• Basic / Advanced
• Local / shared / mixed storage
• More than 8 common hypervisor types/versions
• 4 or 5 Management server OS possiblilities
• That’s 144 combinations only looking the
basics.
• Previous attempts at test automation haven’t
been able to get coverage or capacity.
W h y

Discovery projects:
• New Features
• CloudStack integrations (ie Ceph)
• CloudStack ecosystem components (ie ELK
stack)
W h y

Lab v1.0
• Templates to build nested hypervisor hosts
• Cloudmonkey scripts to configure zone
• Manual creation of storage mounts
Simple but pretty effective
H u m b l e B e g i n n i n g s

Lab v1.0
• A lot of small edits scripts to fit use cases.
• Human error still a big factor
• Can’t ‘set and forget’
• A lot of individual knowledge required
• Loose definition of ‘testing’.
H u m b l e B e g i n n i n g s

Trillian Tenets

Fully automated deployment of CloudStack
environments from any CloudStack package repo
Tr i l l i a n Te n e n t s

• Ability to support multiple concurrent environments
• No hardcoding to a specific (our) lab/environment
• As few separate components as possible
• Separation of ‘code’ and ‘configuration’
• Scripted command-line use fully supported
• Ability to support as many hypervisors as possible
• As flexible as possible
• Best ‘performance’ possible
• Best utilization of hardware resources possible
Tr i l l i a n Te n a n t s

Meet The Family

T h e C I F a m i l y
Trillian
Marvin
Winston
Jenkins
Blue
Orangutan

Trillian
Ansible based environment builder
• Multi-tenant (and multiple concurrent environments)
• CloudStack builds the Virtual Infrastructure
• Hypervisor hosts are CloudStack templates
• Supports multiple and mixed hypervisors
• Supports Advanced and Basic networks
• Open source

Trillian
Installs & Configures:
• CloudStack
• MySQL
• XenServer (configures)
• vCenter (configures)
• ESXi (configures)
• KVM
• Marvin hosts and Marvin test suites

Marvin
Python based test framework based on ‘nose’ test platform.
Configuration files define environment to be tested.
• Integral part of the CloudStack codebase
• Recently given its own install package
• Smoke and Component tests
• 53 smoke test suites
• 154 component test suites
• Additional tests for plugins such as Nuage and SolidFire

Jenkins
CI Platform
• Runs build jobs to produce RPMs from source code
• Presents an integration point to wrap up the creation of
environments using Trillian and the running of tests using
Marvin.
• Gives a UI to Trillian.

Blue Orangutan
Build-bot
• Automation endpoint to send build jobs to Jenkins for CentOS
6/7 and Debian packages from CloudStack source
• Provides an integrated way to create packages from pull
requests in Github.
• Can kick-off Jenkins jobs to build and test environments based
on packages which it builds.

Winston
Test Orchestrator (Future project)
• Receives ‘offers’ of test type/capacity from remote test labs
• Takes request from Blue Orangutan to test packages and
distributes tests to suitable remote labs.
Offers might include:
• Use of local storage
• Use of basic vs advanced networks
• Use of particular hypervisors/versions

How its used

P r o c e s s u s e d i n A p a c h e C l o u d S t a c k P r o j e c t

• Test Pull Requests to add XenServer 7 support
• XenServer won’t run on ESXi 5.5 
• Create XenServer 7 VMs on stand-alone ESXi 6.0 host
and ‘point’ Trillian at them. 
• vSphere 6.5 – Create vCenter and ESXi templates and
run tests. 
X e n S e r v e r 7 & v S p h e r e 6 . 5

• Build base CloudStack environment quickly
• Leverage CloudStack to create other infrastructure
components just as quickly.
ie
• HostBill server
• Cloudian manager server and storage nodes
• Leverage CloudStack networking to give external access
to environments – Static NAT, port forwarding or VPN.
C l o u d i a n & H o s t B i l l

• Build required configuration with required components
• Specific versions of CloudStack
• Specific hypervisor types
• Specific hypervisor versions
• Manual creation of additional components still allowed
• ie second zone with different hypervisor type or version
S o l i d F i r e P l u g i n

We can do all of these things
AT THE SAME TIME
C o u p D e G r â c e

Trillian

https://github.com/shapeblue/Trillian
Tr i l l i a n
• Single zone
• Use pre-set or custom Repos
• Local or shared storage
• ESXi, KVM (C6,C7,U14.04),
XenServer hosts
• Customisable number of primary
and secondary storage pools
• Multiple Management servers
• Selectable Management server
OS (C6.x, C7, U14.04)
• Multiple hypervisors
• dvSwitch (public/guest traffic)
• Configurable Java versions
• Option to use pre-built hosts
• Multiple MySQL configurations

A n s i b l e i n Tr i l l i a n
• All driven from Ansible
• A global group_vars file stores all
local configuration data such as:
• Passwords
• All defaults ie basic vs
advanced or network naming
• EVERYTHING can be customised at
runtime.
(except that)
(and that)

ansible-playbook generate-
cloudconfig.yml -i localhost --extra-vars
"env_name='localstoragetest‘
env_version=cs49
mgmt=1
db=0
hvtype=x
xs_ver=xs65sp1
hv=2
env_accounts=all
pri=2
sec=1
build_marvin=yes
mgmt_os=6
use_local_storage=true
use_shared_storage_systemvms=true
use_shared_storage=false"
ansible-playbook deployvms.yml -i
./hosts_localstoragetest
A n s i b l e i n Tr i l l i a n

Tr i l l i a n i n J e n k i n s
• Different Jenkins’ jobs with specific
‘assumptions’.
• Pull Requests receive URL from
Blue Orangutan and automatically
Run test suites, then clean up.
• Reference_Trillian is a reference
implementation of Jenkins and has
more corner-case options exposed.

• Allows user to force a version of
Java
• Allows use of ‘pre-built’
hypervisors and hypervisors which
cannot themselves be virtualised.

• If environment requires the
hypervisor to use local storage,
then CloudStack adds a data
volume to hypervisor and
configures hypervisor to use it for
local storage. This keeps template
size down
• Mixed local and shared storage is
supported

Internal
ShapeBlue
summary
page UI
plugin.

Tr i l l i a n & J e n k i n s i n C l o u d S t a c k
• UI gives overall status/health of
‘Lab’ and environments
• UI links to a job’s Jenkins output as
well as UI of built environment and
SSH session onto specific VMs

I n S u m m a r y

CloudStack EU user group - Trillian

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to CloudStack EU user group - Trillian

Similar to CloudStack EU user group - Trillian (20)

More from ShapeBlue

More from ShapeBlue (20)

Recently uploaded

Recently uploaded (20)

CloudStack EU user group - Trillian

Editor's Notes