Towards an automated testing
environment
Amogh Vasekar
amogh.vasekar@citrix.com
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Motivation
• Master is invariably unstable
• Unchecked commits
• No provision for developer to test feature branch
• Testi...
Just for laughs
Cloudstack CommunityWho are we
What do we want Testing and Code
Quality
How do we want do it ………
Goals
• Continuously running BVTs
• Enable community testing
• Keep master stable
• Isolate test-runs from each other, and...
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Architecture
Architecture
Phase 1 : Set-up infrastructure
•Provision management server from scratch
•Provision hypervisor from scratch
...
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Implementation
• Jenkins for integration We all like Jenkins
• Glue to hold all together
• Reporting, alerting etc.
• Sepa...
Implementation
Provisioning management server & hypervisors
• Need a way to deploy an OS
• Need a way to configure the sys...
Implementation
Quick overview of technologies involved
• Cobbler
• IPMI tool
• Puppet
• DNSMASQ
• Squid
Cobbler
• Open source OS installation server
• Provides baremetal provisioning via PXE
• Components
 Distro – the OS to p...
Others…
Puppet
• Management server configuration
• Hypervisor configuration like KVM agent etc.
IPMI tool
• To bootstrap b...
Implementation
• Put all these and related technologies on a single VM
• May contain additional storage (NFS server)
• See...
Implementation
Implementation
Minimum Requirements
• XenServer to host VM and management server
• Hosts for hypervisors
• Isolated networ...
Implementation
Phase 2
• Run Marvin tests
• Track commits from previous stable build to current build
• If fails, one of t...
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Replication
Setting up from scratch requires
• Familiarity with CI tools
• Familiarity with CloudStack specifics
• Lots of...
Replication
Propose to push VM like DevCloud (QACloud?)
Thus, steps become
• Load VM on XenServer
• Provision network usin...
Replication
To set-up from scratch
https://cwiki.apache.org/confluence/display/CLOUDSTACK/QA+Infra+fr
om+scratch
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Proposal
To keep master stable
• Maintain a staging branch
• Developer commits go to staging
• If all BVTs pass, push comm...
Agenda
• Motivation
• Architecture
• Implementation
• Replication
• Proposal
• Enhancements
Enhancements
1. Better integration of VMWare hypervisor
2. Pull latest systemvm templates
3. Make VM size manageable
4. Sc...
Only the first step
Lets all keep master stable!!
Lets all test together!!
Big Thanks to AlexH for guiding 
Upcoming SlideShare
Loading in...5
×

Towards automated testing - CloudStack Collab Conference

163
-1

Published on

Prototype design and implementation of replicable automation framework

Published in: Engineering
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
163
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Towards automated testing - CloudStack Collab Conference

  1. 1. Towards an automated testing environment Amogh Vasekar amogh.vasekar@citrix.com
  2. 2. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  3. 3. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  4. 4. Motivation • Master is invariably unstable • Unchecked commits • No provision for developer to test feature branch • Testing is not yet a community effort • Not easy to test plug-ins privately • CloudStack set-up is involved – Management server, hypervisor, network, storage
  5. 5. Just for laughs Cloudstack CommunityWho are we What do we want Testing and Code Quality How do we want do it ………
  6. 6. Goals • Continuously running BVTs • Enable community testing • Keep master stable • Isolate test-runs from each other, and from other infrastructures • Easy to replicate and set-up • Provide commits to cherry pick / buggy commits
  7. 7. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  8. 8. Architecture
  9. 9. Architecture Phase 1 : Set-up infrastructure •Provision management server from scratch •Provision hypervisor from scratch •Storage and networking set-up Phase 2 : Run Marvin tests •Track bad commits •Cherry-pick good commits Need to isolate test-runs Based on Prasanna’s effort (Big Thanks!!)
  10. 10. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  11. 11. Implementation • Jenkins for integration We all like Jenkins • Glue to hold all together • Reporting, alerting etc. • Separate jobs for phases 1 and 2
  12. 12. Implementation Provisioning management server & hypervisors • Need a way to deploy an OS • Need a way to configure the systems • Need a way to provision storage and networking in an isolated manner
  13. 13. Implementation Quick overview of technologies involved • Cobbler • IPMI tool • Puppet • DNSMASQ • Squid
  14. 14. Cobbler • Open source OS installation server • Provides baremetal provisioning via PXE • Components  Distro – the OS to provision  Profile – associate kickstart / configuration with OS  System – MAC to profile mapping. Additional configs • Snippets to run post install scripts etc • Manages TFTP and PXE seamlessly
  15. 15. Others… Puppet • Management server configuration • Hypervisor configuration like KVM agent etc. IPMI tool • To bootstrap baremetal provisioning Squid • HTTP proxy server (needed for isolated network requirement) DNSMASQ • DHCP / DNS management underneath Cobbler
  16. 16. Implementation • Put all these and related technologies on a single VM • May contain additional storage (NFS server) • Seed system VMs • Python code acting as launcher script
  17. 17. Implementation
  18. 18. Implementation Minimum Requirements • XenServer to host VM and management server • Hosts for hypervisors • Isolated network • IPMI network
  19. 19. Implementation Phase 2 • Run Marvin tests • Track commits from previous stable build to current build • If fails, one of the commits broke BVT • If pass, auto cherry-pick commits to (stable) master • All code in Python
  20. 20. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  21. 21. Replication Setting up from scratch requires • Familiarity with CI tools • Familiarity with CloudStack specifics • Lots of time • Many DHCP / network configs
  22. 22. Replication Propose to push VM like DevCloud (QACloud?) Thus, steps become • Load VM on XenServer • Provision network using conf provided • Add IPMI network • Change MAC addresses for Cobbler (has easy to use interface) • Add to Jenkins master • Optionally add NFS volumes • Optionally change domain name for hosts
  23. 23. Replication To set-up from scratch https://cwiki.apache.org/confluence/display/CLOUDSTACK/QA+Infra+fr om+scratch
  24. 24. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  25. 25. Proposal To keep master stable • Maintain a staging branch • Developer commits go to staging • If all BVTs pass, push commits • May need more BVTs • Reduce BVT running time (Simulator?)
  26. 26. Agenda • Motivation • Architecture • Implementation • Replication • Proposal • Enhancements
  27. 27. Enhancements 1. Better integration of VMWare hypervisor 2. Pull latest systemvm templates 3. Make VM size manageable 4. Scheduling mechanism for resource pools 5. Throw open to developers via a service (Thanks to Bharath for 2,3,4,5)
  28. 28. Only the first step Lets all keep master stable!! Lets all test together!! Big Thanks to AlexH for guiding 
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×