The document discusses CloudStack test automation and continuous integration using Jenkins. It describes using the Marvin testing framework to automate deploying CloudStack infrastructure and running tests. The continuous integration process involves building CloudStack, deploying it to hypervisors and storage, then using Marvin to run integration tests on the deployed environment. Jenkins is used as the continuous integration server to trigger builds, deployments, and tests on a schedule or with each code change. The goal is to automate testing to speed up the process and catch issues early in development.

1. 2013 Trend Micro
25th Anniversary
TCloud Computing, Inc.
CloudStack
test automation and CI

2. 2013 Trend Micro
25th Anniversary
Agenda
CloudStack Test Automation
– Deploy and Test, Testing Framework Marvin
CloudStack Continuous Integration by Jenkins
– High Level Architecture
– Build Process
– Deploy Process
– Test Process

3. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation - Marvin

4. 2013 Trend Micro
25th Anniversary
What to test Apache CloudStack
What to Test and Verify
• RESTful API
• Virtual Machine
• Hypervisors
• Storage Pool
• Virtual Network
• Database
Management
Server
MySQL
DB
Cloud Infrastructure
(Hypervisors and
Storages)
User API
Admin API
RESTful API
User Interface
Virtual Machine Database
Hypervisors and Storages

5. 2013 Trend Micro
25th Anniversary
How to test Apache CloudStack
How to Test and Verify ?
(Manual Test)
• Web Browser
• XenCenter
• SSH client tools
• mysql client tools
Cloud we speed up the testing?
Management
Server
MySQL
DB
Cloud Infrastructure
(Hypervisors and
Storages)
User API
Admin API
Web Browser SSH Client MySQL Client
XenCenter
SSH Client

6. 2013 Trend Micro
25th Anniversary
CloudStack project - Marvin
Automation Test
Log Deployment

7. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation - marvin
Apache CloudStack Test Automation – marvin
• testing framework, Python 2.7, unittest2, paramiko, mysql …
Marvin Source:
• <CloudStack Source>/tools/marvin
Marvin features to verify:
• Binding to CloudStack API (synchronous and asynchronous job)
• Support Deployment and Testing
• Support Log and Report
• Support SSH Client
• Support Database Client

8. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin structure
Marvin
Test Client
Management
Server MySQL
DB
Cloud Infrastructure
(Hypervisors and
Storages)
API Client Verify
Database Client Verify
SSH Client Verify

9. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin process
Marvin
Test Client
Load Configuration (datacenterCfg)
Run Test Suite
Execute Deployment
Generate logs and reports
1.Load JSON format
- define cloud infrastructure
- define database location
- define Management Server
2. Create api/secret key
1. Deploy Cloud Infrastructure
by cloudstack API automatically
2. Include Zone, Pod, Cluster,
Host, Storage, Network
1.Load unittest2 module and the
Test Suite you define to verify.
2. Can discover all Test folder
3. Start to Test!
1. Create Test Client log for all
the execution.
2. Create Test Case Result

10. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Test Automation marvin requirement:
• CentOS 6.3/Ubuntu
• Python 2.7
• python-pip
cd <CloudStack_Source>
To build marvin: (Be sure you can build CS first)
- ant build-marvin
To packe marvin:
- ant package-marvin
To install marvin
- ant install-marvin

11. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Step 1
Configure datacenterCfg in JSON format
• Zone
• Pod
• Cluster
• Host
• Primary Storage
• Secondary Storage
• Network Information
• Database information
• Management Server Information
datacenterCfg
{
"zones": [
{
"name": "zone01",
"guestcidraddress": "10.1.1.0/24",
"dns2": "8.8.8.8",
"dns1": “4.4.4.4",
...
},
"dbSvr": {
"dbSvr": "10.1.191.251",
"passwd": "cloud",
"db": "cloud",
"port": 3306,
"user": "cloud"
},
"mgtSvr": [
{
"mgtSvrIp": "10.1.191.251",
"port": 8096
}
]
}

12. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Step 2
Write Your Own Test Case by unittest2
== test_SampleCode.py ==
class Test_SampleCode(cloudstackTestCase):
def setUp(self):
self.apiclient = self.testClient.getApiClient()
def tearDown(self):
pass
def test_apiclient(self)
listZones_parms = listZones.listZonesCmd()
res = self.apiClient.listZones(listZones_parms)
if not res.id:
self.fail('zone is empty')
# cat /tmp/results.log
test_apiclient (Test_SampleCode.Test_SampleCode) ... Ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK

13. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Three Types of testClient (apiClient/sshClient/dbClient)
• CloudStack API client
listZones_parms = listZones.listZonesCmd()
res = self.apiClient.listZones(listZones_parms)
zone_id= self.testClient.executeSql('select id from datacenter')
sshconnect = remoteSSHClient('10.1.191.251', 22, 'root', ‘password')
sshconnect.execute('ls')
• SSH client
• Database client

14. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
The debug method for log
• debug method.
sysvms = listSystemVms.listSystemVmsCmd()
sysvms.zoneid = z.id
self.debug("looking for system VMs in zone: %s, %s"%(z.id, z.name))
2013-03-01 17:15:08,394 - DEBUG - testclient.testcase.TestSetupSuccess - looking for
system VMs in zone: cbf16757-eb92-43f2-bb4b-8a46f3955bf9, zone01
2013-03-01 17:15:08,394 - DEBUG - testclient.testcase.TestSetupSuccess - looking for
system VMs in zone: cbf16757-eb92-43f2-bb4b-8a46f3955bf9, zone01
• debug messages.

15. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Step 3:
To run the Test
# cd <CloudStack_Source>/tools/marvin/marvin
Deployment and Automation test:
# python deployAndRun -c datacenterCfg -t /tmp/testcase.log -r /tmp/results.log -f sandbox/testSetupSuccess.py
option purpose
-c points to the configuration file defining our deployment
-r test results log where the summary report is written to
-t testcase log where all the logs we wrote in our tests is output for debugging purposes
-d directory containing all the test suites
-l only load the configuration, do not deploy the environment
-f Run tests in the given file

16. 2013 Trend Micro
25th Anniversary
CloudStack Test Automation – marvin
Step 4:
Check the Testing Result:
test_01_create_disk_offering (test_disk_offerings.TestCreateDiskOffering)
Test to create disk offering ... ok
test_02_edit_disk_offering (test_disk_offerings.TestDiskOfferings)
Test to update existing disk offering ... ok
test_03_delete_disk_offering (test_disk_offerings.TestDiskOfferings)
Test to delete disk offering ... ok
test_deploy_vm (test_vm_life_cycle.TestDeployVM)
Test Deploy Virtual Machine ... Ok
----------------------------------------------------------------------
Ran 4 tests in 127.529s

17. 2013 Trend Micro
25th Anniversary
CloudStack provides ..
<CloudStack_Source>testintegration
• Smoke Test
• Integration Test

18. 2013 Trend Micro
25th Anniversary
Summary - Marvin
It can help you:
• Automate and speed up the test
• Fast deployment for Zone/Pod/Cluster/Host
• Generate the test result and test log
Development purpose:
• Automation Test
• Continuous Integration
IT and operation purpose:
• Automated operation
• Proof of concept
• Fast deployment

19. 2013 Trend Micro
25th Anniversary
Cloudstack Continuous Integration by jenkins

20. 2013 Trend Micro
25th Anniversary
CloudStack on Continuous Integration
Advantages:
• Automate the build
– daily and every commit
• Automate the deployment and self-testing
• Keep the build fast and stable
• Release Purpose
CloudStack Continuous Integration Process
• Build Process
• Deploy Process
• Test Process
• Release Process

21. 2013 Trend Micro
25th Anniversary
Jenkins
Simple Continuous Integration flow
Source
Version
Control
(GIT/SVN..)
Job Get Source
Get, Build, Run and Get ...
Job Job
Job Job Job
Trigger the process by
• Schedule
• Poll SCM

22. 2013 Trend Micro
25th Anniversary
Master/Slave Architecture
Jenkins Slave
Job Job Job
Jenkins
Master
Job Job Job
Jenkins Slave
Job Job Job
Job Job Job
Subversion
Version
Control
(GIT/SVN..)
Allocate Jobs
Poll SCM
Jenkins Slave
Job Job Job
Job Job Job

23. 2013 Trend Micro
25th Anniversary
CloudStack on Jenkins CI: Term and Roles
Jenkins Roles:
• Master: Jenkins Manager
• Slave: Execute Jenkins Agent, It can execute jobs of the project
The Roles of CloudStack
• Jenkins Master: The Central Manager.
• Jenkins Slave: CloudStack Management Servers
• Hypervisors: XenServers
• Storage: Primary/Secondary Storages
• Test Clients: marvin

24. 2013 Trend Micro
25th Anniversary
CloudStack on Jenkins CI
Jenkins Master
(Jenkins Slave)
CloudStack
Management
Server
Deployment
Module
Subversion
Version
Control
Cloud Infrastructure
XenServer
XenServer
XenServer
XenServer
XenServer
XenServer
XenServer
Storage
3. Trigger Deployment
Test Client Test Client
1. Allocate Jobs
2. Get Source and Build
4. Deploy Cloud Infra.
5. Run Test Clients
…

25. 2013 Trend Micro
25th Anniversary
CloudStack Continuous Integration
CloudStack Continuous Integration Cycle:
Build
Process
Deploy
Process
Test
Process
Release
Process

26. 2013 Trend Micro
25th Anniversary
Cloudstack Continuous Integration
Build Process
• Retrieve the source code
• System environment setup
• Development environment setup
• Build and deploy to apache-tomcat
• Start the CloudStack Service
Jenkins
Master
(Jenkins Slave)
CloudStack
Management
Server
Subversion
Version
Control
Build
Process
Deploy
Process
Test
Process
Release
Process

27. 2013 Trend Micro
25th Anniversary
Cloudstack Continuous Integration
Deploy Process
• Deployment Module:
– cobbler/puppy:
• Install/Configure Host/Storages
– Hypervisors on Virtual Machine (nested VM)
• Take Snapshot and Rollback for each run
• CloudStack deployment:
– Use testing framework marvin to deploy CS
– Zone/Pod/Cluster/Host/Hypervisors/Storages
– Make sure SystemVMs start correctly Deployment
Module
Cloud Infrastructure
XenServer
XenServer
XenServer
XenServer
XenServer
XenServer
XenServer
Storage

28. 2013 Trend Micro
25th Anniversary
Cloudstack continuous integration
Test Process
• Run Test Suite
• Testing Framework marvin
– Test and verify cloudstack API
– Test by SSH Client
– Test by Database Client
• Generate Test Results and Reports
(Jenkins Slave)
CloudStack
Management
Server
Test Client Test Client
Cloud
Infrastructure
…

29. 2013 Trend Micro
25th Anniversary
Cloudstack continuous integration
Release Process
• Integrate the Documents and Reports
• Run the release procedure to build install package
• Notification.
Build
Process
Deploy
Process
Test
Process
Release
Process

30. 2013 Trend Micro
25th Anniversary

31. 2013 Trend Micro
25th Anniversary

32. 2013 Trend Micro
25th Anniversary

33. 2013 Trend Micro
25th Anniversary
Q & A