- SOA performance testing is challenging due to the distributed nature of SOA architectures and the need to test multiple applications, databases, technologies, and protocols. - A key part of SOA performance testing is breaking applications into their smallest components like individual services, systems, databases, and technologies and testing each component. - The methodology involves discovery of services and components, test planning, automation for load generation, monitoring of performance under increasing load, and analysis of results. - Automated testing tools are needed to simulate traffic at scale and monitor performance across the SOA infrastructure under varying loads.

2. PDQ, SOA, SLA SOA Performance is a Critical Success Factor Ralph Decker Head of Performance Serviceline 7 th May 2008

3. Agenda What is SOA Service Level Agreements (SLA’s) for SOA Key to SOA Performance Testing SOA Performance Testing Challenges Overcoming Challenges Methodology and Approach Discovery Test Planning Automation and Tools Test Execution Monitoring Measurement and Analysis Summary

4. What is SOA Service Oriented Architecture Architectural paradigm (pattern/model) Variety of heterogeneous systems (dissimilar) Different locations and owners Web Services? SOA provides benefits in four basic categories: Reduces expensive integration Allows for more asset reuse Increased business agility Reduces business risk

5. SLAs for SOA A service-level agreement (SLA) is a formal contract between a service provider and a consumer Service availability Performance Traffic levels Messages / queries per hour / minute / second Response time Rejected transactions Errors … Poor Response Time Missed SLA Noncompliance with industry and government regulations

6. Key to SOA Performance Testing The key to successful performance testing in general require: Understanding the application and the infrastructure Understanding the user/stakeholders of the application Generate accurate anticipated volumes of traffic Investigate the impact of the traffic on the application and systems under test SOA Adds Complexity to Performance Testing Wide range of technologies Many different applications and usages Different hardware / infrastructure Knowledge of the application and the technologies Replicating traffic patterns

7. SOA Performance Testing Challenges SOA being a distributed environment, finding right skill sets who possess in-depth knowledge of involved platforms, applications, databases and any middleware. Assign appropriately skilled team of performance engineers to the test effort with knowledge of the systems to monitor and analyze the impact of testing An in-depth understanding of the service is required for adequate testing and evaluation A significant increase in testing activities and test assets (performance testing suites that include sophisticated harnesses and stubs) will be required at a service level Predicting the future usage of services to assist with performance, load, stress, scalability Test strategy differs from traditional testing and generally has to encompass many internal political factors e.g. ownership and responsibility

8. Overcoming Challenges To simplify performance testing for SOA applications break them down into the smallest components possible: Individual Service Systems Databases Technology Protocols Messaging Functionality Evaluate and analyze the performance of individual services based on components

9. Methodology and Approach Discovery: Narrow the testing event to the smallest element/service and understand the transaction, application service and the environment/systems Test Plan: Document the testing approach and the expected deliverables Automation: Develop automation to replicate transactions Test Execution: Conduct testing generating traffic increasing the traffic to pre-defined levels Monitoring: Monitor the response time for the requests sent under varying traffic levels and the impact of the traffic on the application and infrastructure under varying traffic levels Measurement and Analysis: Analyze the traffic patters to with the traffic/load

10. Discovery Identify the services and components to be included as part of the testing efforts Identify lower level and/or external services called by the services under test Identify the infrastructure hosting the services to be tested Identify the messages sent to and received from the service that will be automated for generating load Narrow the “functionality” to a subset that will be used for the testing Define data that will be used and how it will be validated Define pass/fail criteria (usually response time or transactions per second) Identify skills required for the testing effort Determine SLA or load/transactions for the testing Prioritize testing

11. Test Planning Test Plan - Document the Following: Services and sub services Infrastructure Transaction Automation Data SLA requirements (load/transactions) Pass / fail criteria

12. Automation and Tools Automation Subset of the functionality of the service Prepare data and mechanism to validate Tools to generate the transaction Proprietary SOA Custom Test Harness Enterprise Tools Loadrunner 9.1 Parasoft SOAtest Green Hat GH Tester Borland SilkPerformer SOA OpenSource Tools

13. Test Execution Load Testing (up to defined SLA transactions per second or other) Stress Testing (up to service failure) Volume Testing (introducing large amounts of data into system) Reliability Testing (high levels of load over long periods of time)

14. Monitoring Load Size (transactions per second/other) Throughput Response Time Hardware OS Disk Web (for Web Services) Application specific counters Database

15. Measurement and Analysis Size of message Response time Throughput under load CPU memory disk Application specific counters

16. Summary NEED 6 BULLET SUMMARY HERE

17. Questions Thank You