Jim Trentadue describes how his organization first used the cloud for its non-production needs including development, testing, training, and production support. Jim begins by describing the components of a cloud environment and how it differs from a traditional physical server structure. To prove the cloud concept, he used a risk-based model for determining which servers would be migrated. The result was a win for the organization from a time-to-market and cost savings perspective. Jim shares his do’s and don’ts for moving to the cloud. Do’s include ensure you identify all costs associated with the new cloud infrastructure, implement a risk-based approach to cloud migration, define a governance model, and define Service Level Agreements for your cloud vendor. Jim warns against creating an open-ended environment without a charge-back model to allocate costs and failing to continuously monitor the overall environment.

1. T11
Cloud Testing
5/2/2013 11:15:00 AM
A Year of Testing in the Cloud: Lessons
Learned
Presented by:
Jim Trentadue
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jim Trentadue
Jim Trentadue has more than fourteen years of experience as a coordinator/manager in the software
testing field. Jim’s various roles in testing have focused on test execution, automation, management,
environment management, standards deployment, and test tool implementation. In the area of offshore
testing, he has worked with multiple large firms on developing and coordinating cohesive relationships. As
a speaker, Jim has presented at numerous industry conferences, chapter meetings, and at the University
of South Florida's software testing class, where he mentors students on the testing industry and discusses
trends for establishing future job searches and continued training.

3. 4/16/2013
A Year of Testing in the Cloud:
Lessons Learned
Jim Trentadue
Jim_Trentadue@hotmail.com
May 2nd, 2013
What is the ‘Cloud’?
Defining the terminology behind the Cloud and listing its components
“Cloud” is a new consumption and delivery model inspired by consumer Internet services.
Enabled by Virtualization, (Service) Automation, Standardization
Cloud enables:
Self-service
Cloud Services
Sourcing options
Economies-of-scale
Cloud Computing
Model
Multiple Types of Clouds will co-exist:
Private, Public and Hybrid
Workload and / or Programming Model Specific
1

4. 4/16/2013
Regular stats and environment reports
Leveraging an automated Cloud Management solution enables the following
Managing Cloud Services
Deploying Cloud Services
Secure User Centric SelfService Portal, Automation
Engine and Catalog
Automated Provisioning and
Image Management
Monitoring and Metering
2

5. 4/16/2013
Overview for a ‘Cloud’ test environment
A high-level overview of the constructs of the new virtual test configuration
1) Image Library
2) Software Stack
3) Server Specifications
Operating System
Software components
Hardware requirements
•Red Hat Linux 5.3
•Oracle 10g
•Number of CPU Processors
•Red Hat Linux 5.3Oracle 10gR2
•Oracle 10g R2
•Amount of memory
•Windows 2003 server
•Oracle 11g
•Storage size (through SAN)
•Windows 2008 server – 32bit
•SQL Server 2008
•Windows 2008 server – 32bit
SQL Server 2008
•JBOSS v
(plus potentially component)
•Windows 2008 server – 32bit
Application
Verify your OS is supported by
the Cloud provider
Must be available via a silent
install method
This can be a standard amount
or can be custom on demand
Benefits of a virtual test environment – why move from physical?
Compare and contrast of the two different environment structures is done below
Physical
environment
structure
Virtual
environment
structure (Cloud)
Key attributes
Key attributes
• Significant monetary investment upfront
• Blade technology that can be flexible for the various
layers (Application, Middle-Tier, Database)
• Fixed CPU Processor, Memory, and Storage
• Support costs are charged by the server
• Maintenance would need to occur on each of
the servers for O/S or software upgrades
• Concept of storing one master image library based
on O/S, adding in software stack, then request for
Processor, Memory and Storage requested
• Support costs are centralized, especially for
licensing, patches and upgrades
3

6. 4/16/2013
Why Development and Test Clouds?
High costs and poor utilization of non-production environments creates the need to consider alternatives
●
Very low utilization of development and test servers—usually less than 15%
●
Having to dedicate a substantial number of servers within a typical IT environment to test –
sometimes 50% or more
●
Finding instant access to available IT infrastructure resources (tools and platform) to perform tests
●
Provisioning of new environments is a manual process that can take up to 6 weeks or more
●
Very long testing backlogs, usually the single largest factor in the delay of new application deployments
●
Test environments are often seen as expensive and providing little ‘real’ business value
●
Inability to follow best practices due to expense of additional IT resources required.
●
High risk of defects caused by wrongly configured test environments
New opportunities for testing
Implementation of a Cloud computing model can open other test avenues
UAT End User Training environment
Functional Test Automation environment
Performance Test Automation environment
Multiple System Test environments if needed
Prototype environment for Architects Business Analysts Technical Leads
Pre-production Staging environment for deployment tests
Production Support environment for immediate break fixes
Release-1 environment for a specified duration
IT User Training environment for new associates to the application
Upgraded environment for OS or vendor patch level upgrades
4

7. 4/16/2013
Investment in the test environment build
Building a business case for a test environment project with the right-level of support
Business Case
Risk
Reliability testing of developed code is often
compromised with multiple versions of a same
module.
Return on investment
Need to increase the
number of test
environments to align
with parallel project
initiatives
Investment in additional hardware or
virtualization technology.
Additionally, project schedules are often at risk
with defects opened due to environment issues.
Need to limit the rights
to which users have
privileges across the
various environments
Often there are times when random associates
(IT or Business), may have access to a
particular region, manipulating critical data for a
series of tests.
Investment in a source code repository.
Need to have a DBA
Service Level
Agreement agreed
upon and adhered to
The ability to refresh the data and monitor the
performance of an environment on a regular
basis is critical. Manipulated test data and a
long running query may skew results if
inadequate measures are in place.
Increased reliability and sustainability of
projects, thus expediting timelines and
deployments. Environment-related defects
should decrease and application bugs found
are worked sooner.
Need to review the buy
vs. build concept for
procuring, or running
as a service
If called upon to procure new hardware for a
new request of an additional environment capital, maintenance and support costs will
continually be present. Additionally, the risk is
high is if one particular server crashes.
Investigate opportunities for a virtual
environment or Cloud computing model to
avoid repeatedly purchasing servers, storage,
processors, memory and support.
Leveraging a well-suited server can help
streamline additional RDBMS license costs by
storing multiple environments on one server.
Limiting the capability of who has access to
environments further in migration cycles (Dev,
Test, Prod).
Scope of initial environment setup
Below is the list of the type of applications scoped for virtualization of a test environment
Started with seven applications in scope with the following test environment needs
1 new application with no non-production
environments built yet
3 existing applications with faltering physical
test environments
2 existing applications with additional test
environment needs apart from the physical
servers
1 existing application looking to leverage a
virtual test environment instead of a physical
landscape
5

8. 4/16/2013
Initial landscape after setup
The first build out of the environment looked like the following:
Started with seven applications in scope with the following test environment needs
1 new application with no non-production
environments built yet
24 servers built
3 existing applications with faltering physical
test environments
9 servers built
2 existing applications with additional test
environment needs apart from the physical
servers
6 servers built
1 existing application looking to leverage a
virtual test environment instead of a physical
landscape
1 server evaluation
Cloud test environment start-up challenges
A list of items that needs to be verified before the first server build out
Network ports, connectivity, IP subnets, etc.
Data refresh strategy laid out as far as personnel supporting this
Full blade server outages for patches, upgrades, etc., need a thorough checklist to
ensure
Patches either replicated or not across existing servers, not just on the image
User access tests to all necessary servers
6

9. 4/16/2013
If it could be done over again, what would be done different?
Requirements following the Cloud infrastructure implementation
Some items to have advance planning on prior to creating the first virtual server
With hindsight being 20-20, these items are must have to start again
Full list of all costs
associated with the Cloud
Understand ALL of the costs like:
• Cost for server uptime
• Cost for support for the environment with various infrastructure groups
• Cost of creating images, creating servers
Pay-as –you need usage
model
Outline the costs for your customers:
• Shared cost for application teams using servers
• Shared cost for creating images
• Run rate cost for continued server maintenance
Governance model
established
Ensure there are rules of engagement for usage:
• If additional storage, CPU and Memory is required, there should be a
nominal charge
• If the servers have been idle for a defined period of time, they are
subject for deactivation
SLA defined for Cloud
support
Understanding this is a non-production environment:
• Should there be any infrastructure issues, define an SLA knowing
respective to the event and impact
7

10. 4/16/2013
Session recap
Recapping lessons learned after the first year of Cloud implementation
Before recommending, make sure you understand the Cloud infrastructure
components and how it would be deployed and managed in your environment
Build a strong business case for investment with industry stats on test environment
usage; expanding on improving what you have and venturing into new areas for
opportunities
Outline the scope and landscape for an initial rollout, based on a risk-adverse method
Research and study what others have had as start-up challenges, to try and avoid
these pitfalls when starting
Host a lessons learned immediately following the initial scope and landscape with the
groups using the environment to establish best practices for usage
QUESTIONS?
8

11. 4/16/2013
THANK YOU!
Jim Trentadue
Jim_Trentadue@hotmail.com
9