The document discusses how cloud computing can provide new tools for innovation in quality assurance and testing. It provides an overview of cloud computing topologies and implications of testing in the cloud. Key benefits of cloud computing include flexible pricing models, elastic scaling, rapid provisioning, and increased efficiency. While some workloads are well-suited for cloud delivery, others may not be ready due to security, regulatory compliance, or customization needs. Case studies demonstrate significant cost savings and returns on investment from cloud adoption.

1. Cloud Computing for QA Professionals Software Association of Oregon May 26, 2009 Dan Carr Executive IT Architect – Cloud Computing Engagements

2. What you should take away from today’s talk… Gain a better understanding of Cloud Computing with an overview of different topology options Have a clear view where Cloud Computing technologies can provide new tools for innovation in QA and test Understand some of implications of testing in a Cloud environment with a better understanding of the impact of key non-functional requirements 05/26/09

3. Cloud Computing Primer A Tool for QA Innovation Testing in the Cloud Final Thoughts     05/26/09

4. A Crisis of Complexity. The Need for Progress is Clear. 85% idle In distributed computing environments, up to 85% of computing capacity sits idle. Explosion of information driving 54% growth in storage shipments every year. 1.5x 05/26/09 70¢ per $1 70% on average is spent on maintaining current IT infrastructures versus adding new capabilities.

5. IT Analysts: Variable pricing No long term commitments Hosted, on demand provisioning Massive, elastic scaling Standard Internet technology Abstracted infrastructure Service-oriented Financial Analysts: Utility pricing Multi-core chips Hosted, a-a-s provisioning Parallel, on demand processing Scalable Virtualized, efficient infrastructure Flexible IT Customers: Flexible pricing Outsourced, on demand provisioning Unlimited scaling SW developer platform Flexible Press: Pay by consumption Lower costs On demand provisioning Grid and SaaS (Service as Software) combination Massive scaling Efficient infrastructure Simple and easy Common Attributes of Clouds Flexible pricing Elastic scaling Rapid provisioning Advanced virtualization There are varying opinions on the definition of Cloud Computing, but some common attributes are emerging… Standardized offerings 05/26/09

6. Operational definition for cloud computing A user experience and a business model Cloud computing is an emerging style of IT delivery in which applications, data, and IT resources are rapidly provisioned and provided as standardized offerings to users over the web in a flexible pricing model. An infrastructure management and services delivery methodology Cloud computing is a way of managing large numbers of highly virtualized resources such that, from a management perspective, they resemble a single large resource. This can then be used to deliver services with elastic scaling . Monitor & Manage Services & Resources Cloud Administrator Datacenter Infrastructure Service Catalog, Component Library Service Consumers Component Vendors/ Software Publishers Publish & Update Components, Service Templates IT Cloud Access Services 05/26/09

7. Evolution of Cloud Computing Solving large problems with parallel computing Network-based subscriptions to applications Offering computing resources as a metered service Anytime, anywhere access to virtualized IT resources delivered dynamically as a service. Software as a Service Utility Computing Cloud Computing Grid Computing Abstraction –> Simplification -> Capability Aggregation Infrastructure Provisioning 05/26/09

8. The layers of IT-as-a -Service Infrastructure as a Service Servers Networking Storage Data Center Fabric Shared virtualized, dynamic provisioning 05/26/09 Software as a Service Collaboration Business Processes CRM/ERP/HR Industry Applications Platform as a Service High Volume Transactions Middleware Database Web 2.0 Application Runtime Language Runtime Development Tooling

9. Some examples of product offerings mapped to layers 05/26/09 Software as a Service Infrastructure as a Service Platform as a Service

10. The future: Three co-existing delivery models Enterprise Service Consumers Service Integration Service Integration Traditional Enterprise IT Private Cloud Services Services Service Integration Public Clouds Services Over time, IT workloads will move to Cloud delivery models as applicable the industry and business environment Mission Critical Packaged Apps High Compliancy Test Systems Developer Systems Storage Cloud Test tools as a Service Compute as a Service Storage as a Service Software as a Service Examples: Hybrid Cloud 05/26/09

11. 05/26/09 Multiple tenants using the same public Cloud services Administrator Public, off-premise Cloud Compute Services Storage Services Platform Services Application Services Messaging Services User (Company A) User (Company B) User (Company C) User (Company D)

12. Private, on-premise Cloud 05/26/09 All resources are local and dedicated. All cloud management is local. User User User Administrator Compute Services Storage Services Platform Services Application Services Messaging Services

13. Private, off-premise Cloud 05/26/09 Resources are dedicated, but off-premise. Enterprise administrator can manage the service catalog and policies. Cloud provider operates and manages the cloud infrastructure and resource pool. V Enterprise Administrator User User VPN, MPLS VPLS Cloud Provider Private Cloud

14. Hybrid Cloud, Private and Public Clouds 05/26/09 Both Private and Public clouds are managed by the Enterprise administrator with a unified systems management view of both infrastructures. Provides for increased resiliency and allows workloads to be isolated geographically. VPN MPLS VPLS Public Cloud Private Cloud

15. 05/26/09 Keynote Systems Mercury SOASTA SkyTap Aptna Loadstorm Collabnet Dynamsoft

16. Cloud Architecture Model End User Requests & Operators … Service Request & Operations Design & Build Image Library (Store) Deployment Operational Lifecycle of Images IT Infrastructure & Application Provider Service Creation & Deployment Virtual Image Management Service Catalog Request UI Operational UI 05/26/09 Optimized Middleware (image deployment, integrated security, workload mgmt., high-availability) Service Oriented Architecture Information Architecture User Request Management/Self Service Portal Security: Identity, Integrity, Isolation, Compliance Usage Accounting License Mgmt. Image Lifecycle Mgmt. Provisioning Performance Mgmt. Availability/Backup/ Restore Service Lifecycle Management Service Management Virtual Resources & Aggregations SMP Servers Network Hardware Storage Servers System Resources Blades Storage Virtualized Infrastructure Server Virt. Storage Virt. Network Virt.

17. Basic Request Driven Cloud Provisioning Workflow Service Request Portal 05/26/09 Service Automation Manager Image Library CMDB Machine Image Asset Library Source, Config Binaries Retrieve and configure VM Image Retrieve App, Middleware, Config Configure and Deploy Software 1 2 3 5 4 6 Resource Pool Hypervisor Configured Machine Image Virtual Machine Virtual Machine User requests and schedules resource Reserve and allocate from resource pool Provision the image

18. What workloads are moving to Cloud delivery? 05/26/09 Test and Pre-production systems 1 Single virtual appliance workloads 2 Mature packaged offerings, like e-mail and collaboration (see http://www.lotuslive.com) 3 Software development environments 4 Batch processing jobs with limited security requirements 5 Isolated workloads where latency between components is not an issue 6 Storage Solutions/Storage as a Service 7 Backup Solutions/Backup & Restore as a Service 8 Some data intensive workloads if the provider has a cloud storage offering tied to the cloud compute offering 9

19. What workloads may not be ready for Cloud delivery today? Workloads which depend on sensitive data normally restricted to the Enterprise Employee Information - Most companies are not ready to move their LDAP server into a public cloud because of the sensitivity of the data Health Care Records - May not be ready to move until the security of the cloud provider is well established Workloads composed of multiple, co-dependent services High throughput online transaction processing Workloads requiring a high level of auditability, accountability Workloads subject to Sarbanes-Oxley, for example Workloads based on 3 rd party software which does not have a virtualization or cloud aware licensing strategy Workloads requiring detailed chargeback or utilization measurement as required for capacity planning or departmental level billing Workloads requiring customization (e.g. customized SaaS)‏ 05/26/09 1 2 3 4 5 6

20. So what’s different about Cloud? Legacy environments Cloud enabled enterprise Capability From To 05/26/09 Server/Storage Utilization 10-20% Self service None Test Provisioning Weeks Change Management Months Release Management Weeks Metering/Billing Fixed cost model Payback period for new services Years 70-90% Unlimited Minutes Days/Hours Minutes Granular Months

21. IBM Technical Adoption Program (TAP)—ROI Analysis Current IT Spend Strategic Change Capacity Hardware, labor & power savings reduced annual cost of operation by 84% 100% Without Cloud With Cloud Labor Costs ( Operations and Maintenance ) Power Costs (- 89%) Power Costs Software Costs Software Costs Reduced Capital Expenditure Reduced Operations Expenditure Additional Benefits Reduced risk, less idle time, more efficient use of energy, acceleration of innovation projects, enhanced customer service Business Case Results: Annual savings: $3.3M (84%) from $3.9M to $0.6M Payback Period: 73 days Net Present Value (NPV): $7.5M Internal Rate of Return (IRR): 496% Return On Investment (ROI): 1039% 05/26/09 Hardware Costs ( - 89%) Labor Costs ( - 81%) Deployment (1-time) Liberated funding for new development, transformation investment or direct saving Hardware Costs ( annualized ) New Development

22. Cloud Computing Primer A Tool for QA Innovation Testing in the Cloud Final Thoughts     05/26/09

23. There are number of emerging service offerings and internal infrastructure options available today Virtual Test Lab Services On Demand Testing & Build Tools Crowd Sourcing Test Services Private Cloud Options 05/26/09

24. Virtual Test Lab Services Immediate access to full virtual lab solution over the Web. Development and QA teams can scale up and down lab resources as needed from pool of virtualized infrastructure Machines deployed in the lab can be accessed through a Web-based or client console and Skytap's API Available as an on demand service in the cloud or as a hardware appliance, SOASTA offers integration of test design, monitoring, and reporting for Web applications. Support for Amazon EC2 - turns the Amazon cloud into a Virtual Test Lab for load and performance testing of their applications. 05/26/09 05/26/09

25. Cloud-Based Load Testing Services Use a Cloud infrastructure to provide browser-based load testing, Uses Selenium for recording and playback of web traffic. Uses real browsers for load test simulation. Provides easy to use interface along with sophisticated query and reporting tools Alliance with uTest Uses a Cloud infrastructure for rapid ramp up and down of compute resources Provides Load, Stress, Regression, and Endurance test tools 05/26/09

26. Continuous Integration and Crowd Sourced Testing Services Crowd Source model for usability testing Pre-screened users who meet specified demographics. Website owners can request that one or more users attempt to complete a task on their website. Per unit price for video feedback and written report Uses Crowd Source model to bring together a marketplace for software testing services with global community of 16,000+ QA professionals from more than 150 countries Functional, Load and User Testing Alliance with BrowserMob for Load Testing Built on top of Amazon EC2 Focused on Continuous Integration and distributed builds across multiple platforms (Linux and Windows) Large number of collaboration and reporting features 05/26/09

27. Current IT Test Provisioning processes are slow and inefficient Multiple Disparate Processes Human Intervention and Initiation required Lack of integration between Operating System provisioning and Application provisioning Lack of integration between Application Build/Package and Deploy Multiple manual steps to provision a test infrastructure High amount of human error Long time to create test infrastructure 05/26/09 Human Intervention and Initiation Disparate Processes Human Intervention and Initiation

28. Typical Test Environments 30% to 50% of all servers within a typical IT environment are dedicated to test Most test servers run at less than 10% utilization, if they are running at all! IT staff report a top challenge is finding available resources to perform tests in order to move new applications into production 30% of all defects are caused by wrongly configured test environments Testing backlog is often very long and single largest factor in the delay new application deployments Test environments are seen as expensive and providing little real business value. “ Industry Developments and Models – Global Testing Services: Coming of Age,” IDC, 2008 and IBM Internal Reports 05/26/09

29. Automation Control Visibility Management of a Virtualized Test Environment requires Service Management Virtualization without good management is more dangerous than not using virtualization in the first place,” –Gartner 05/26/09 User Request Interface and IT Operational Views Enable end users to request services Enable end users and IT staff to monitor status of requests Enable IT operations staff to manage the delivery of the services View monitoring statistics of cloud deployments, including breached thresholds Orchestration of People, Processes, Data, Technology Enable the fulfillment of user requests based on best practices for request types & conformance to organizational processes Consistently enforce operational policies for service delivery Provide usage and accounting management for cloud services Provide security for cloud services Enable green initiatives through optimized utilization of resources Automate Requests and Operational Tasks to Improve Efficiency and Effectiveness Automate the orchestration of technology to fulfill user requests for cloud services Automate processes and repetitive IT administration tasks for optimized utilization of resources Provide the ability to automatically meet higher scalability demands Provide process automation capabilities, enabling the value of change, release, and configuration management disciplines in a cloud computing environment

30. Cloud based Test Environment Management System (TEMS) 05/26/09

31. Cloud Computing Primer A Tool for QA Innovation Testing in the Cloud Question & Answer     05/26/09

32. “ The more things change, the more they stay the same” - Alphonse Karr 05/26/09

33. Non-Functional requirements describe the qualities of a system or solution These qualities don’t change with Cloud-based technologies, however new concerns enter the picture and change based whether we are building developing or using a Cloud service. Availability Capacity Backup & Recovery Configuration & Release Management Manageability Disaster Recovery Systems Management Reliability Scalability Failure Management Legal and Compliance Security Performance Service Level Agreements Data Portability 05/26/09

34. Non-Functional Requirements – Cloud Computing Implications 05/26/09 Quality Implications Availability, Reliability & Failure Management Can my provider meet my availability requirements? Are the penalties meaningful for not meeting my requirements? What if I can’t get to the service? Do I have adequate network redundancy? Does my application/infrastructure design support failover? Across geographies? Have I designed for failure? How will I test it? Capacity How do I plan for capacity? What’s my capacity management plan? What’s my provider’s? How will I provide capacity management metrics to my end users? How do I correlate the cost of application resources that I am providing to the infrastructure resources I am using? How do correlate my application workload to my infrastructure consumption Backup and Recovery How do I request a recovery of a file, database or table? How much time will it take? How will I provide backup and recovery services? File level, database level, an entire application?

35. Non-Functional Requirements – Cloud Computing Implications (continued) 05/26/09 Quality Implications Disaster Recovery Does my provider have a disaster recovery plan? If so, do they meet the business continuity plan? Does my architecture and/or the Cloud provider’s support a complete loss of a data center? How will you test it annually? How much will it cost? Performance & Scalability Do my performance characteristics match what the cloud service can provide? Have I selected the right design patterns for a cloud-deployed application? Have I adequately captured and developed a performance model (static, dynamic…) How will I test my application at Internet scale? How will I monitor performance? Can I integrate my vendor’s tools with my systems management tools? Legal and Compliance Is my workload appropriate for the Cloud topology selected? Does my Cloud provider have the correct controls and auditing features in place to meet my compliance and legal requirements (e.g.,SOX, HIPPA, PHI, PCI, etc..)

36. Non-Functional Requirements – Cloud Computing Implications (continued) 05/26/09 Quality Implications Configuration & Release Management How does my Cloud provider handle new versions and releases? Will I be notified? How far in advance? How will I test upgrades and affects on upstream and downstream systems? How will I discover new nodes in my infrastructure (servers, routers, switches, firewalls)? How will I track the software that is deployed on each of these nodes? How will I do patch management? Security Does my cloud provider have a documented security and privacy policy? How will I be notified of breaches and intrusions? How do I conduct forensics? Will I be able to integrate my Cloud vendor’s security monitoring with my own? Will I have to build my own PKI? Does the vendor provide PKI services? How will I integrate my identify management/directory services with my cloud provider? Do I understand my cloud provider’s tenancy model? Is it adequate given my business, legal and regulatory requirements? How is data and VM Image destruction handled? Data Portability What services will you provide to for importing or converting data into your application? Customer wants to get the data both in and out of your application / system. How will you get your data into the application. How long will it take?

37. Cloud Computing Primer A Tool for QA Innovation Testing in the Cloud Final Thoughts     05/26/09

38. 5 Step approach to adopting a Cloud Computing model 05/26/09 Start with a transformation roadmap 1 Adopt a reference architecture that supports a Cloud Computing model 2 Conduct a detailed analysis of your current and future workloads 3 Decide upon the right mix of workloads (Cloud and Non-Cloud) 4 Backup up your decisions with detailed ROI analysis 5

39. Selecting a pilot project 05/26/09 Address a well understood IT project 1 Incorporate aspects of self-service requests and provisioning automation 2 Incorporate Line of business and IT objectives into the project 3 Consider test and development environments as entry points 4 Require an achievable stretch beyond current capabilities to address gaps (skills, processes etc.) 5 Target an area that will eventually make it into production 6

40. Test Cloud Adoption is Iterative and Incremental Test Cloud Assess and address capability gaps Execute Review result Select (next) project scope … with each project delivering immediate and long-term value 05/26/09

41. In summary… Cloud computing is a new way to deliver IT services 3 categories of Clouds: IaaS, SaaS, PaaS Several deployment options: Private, Public and Hybrid Service Management and Virtualization make it all possible Test and Development environments are good candidates for Cloud deployments Not all workloads are appropriate for Cloud Cloud Service and Infrastructure Options for QA & Test Emerging marketplace for virtual test labs provide easier way to rapidly build and tear down test environments Cloud-based services can provide a viable alternative to traditional approaches to functional, load, and usability testing For many organizations, a private test Cloud can increase the utilization of test hardware and decrease provisioning time Testing The Cloud Non-Functional requirement categories don’t change, but new concerns enter the picture Focus on the Business Benefits Its about business results. Talk to the business about business drivers and benefits of Cloud not Cloud technology Architecture and planning are critical. We still need to design for non-functional requirements Don’t forget about governance Applying the technology to broken and inefficient processes will only cause more problems 05/26/09

42. IBM Cloud Computing Service and Software Offerings Cloud Consulting A portfolio of leadership products and services for optimizing with cloud computing that continues to grow to support customers with cloud building and cloud delivered offerings. Cloud Implementation Cloud Delivered Infrastructure Consulting Services for Cloud Computing Business Cloud Consulting Services Security and Resiliency Consulting Services for Cloud Resiliency Certification for Cloud Computing Service Management for Cloud Computing Test and Developer Cloud Services Managed Security Services for Cloud Computing End User Cloud Services Scale out File Services LotusLive Computing on Demand Information Protection Services Managed Data Protection for desktops and laptops DeveloperWorks on Amazon EC2 Service Management Automaton software 05/26/09

43. Thank you! For more information, please visit: ibm.com/cloud Or contact me at: [email_address] 05/26/09