Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Soa Performance Is A Critical Success Factor From AppLabs


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Soa Performance Is A Critical Success Factor From AppLabs

  1. 2. PDQ, SOA, SLA SOA Performance is a Critical Success Factor Ralph Decker Head of Performance Serviceline 7 th May 2008
  2. 3. Agenda <ul><li>What is SOA </li></ul><ul><li>Service Level Agreements (SLA’s) for SOA </li></ul><ul><li>Key to SOA Performance Testing </li></ul><ul><li>SOA Performance Testing Challenges </li></ul><ul><li>Overcoming Challenges </li></ul><ul><li>Methodology and Approach </li></ul><ul><ul><li>Discovery </li></ul></ul><ul><ul><li>Test Planning </li></ul></ul><ul><ul><li>Automation and Tools </li></ul></ul><ul><ul><li>Test Execution </li></ul></ul><ul><ul><li>Monitoring </li></ul></ul><ul><ul><li>Measurement and Analysis </li></ul></ul><ul><li>Summary </li></ul>
  3. 4. What is SOA <ul><li>Service Oriented Architecture </li></ul><ul><ul><li>Architectural paradigm (pattern/model) </li></ul></ul><ul><ul><li>Variety of heterogeneous systems (dissimilar) </li></ul></ul><ul><ul><li>Different locations and owners </li></ul></ul><ul><ul><li>Web Services? </li></ul></ul><ul><li>SOA provides benefits in four basic categories: </li></ul><ul><ul><li>Reduces expensive integration </li></ul></ul><ul><ul><li>Allows for more asset reuse </li></ul></ul><ul><ul><li>Increased business agility </li></ul></ul><ul><ul><li>Reduces business risk </li></ul></ul>
  4. 5. SLAs for SOA <ul><li>A service-level agreement (SLA) is a formal contract between a service provider and a consumer </li></ul><ul><ul><li>Service availability </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Traffic levels </li></ul></ul><ul><ul><li>Messages / queries per hour / minute / second </li></ul></ul><ul><ul><li>Response time </li></ul></ul><ul><ul><li>Rejected transactions </li></ul></ul><ul><ul><li>Errors </li></ul></ul><ul><ul><li>… </li></ul></ul>Poor Response Time Missed SLA Noncompliance with industry and government regulations
  5. 6. Key to SOA Performance Testing <ul><li>The key to successful performance testing in general require: </li></ul><ul><ul><li>Understanding the application and the infrastructure </li></ul></ul><ul><ul><li>Understanding the user/stakeholders of the application </li></ul></ul><ul><ul><li>Generate accurate anticipated volumes of traffic </li></ul></ul><ul><ul><li>Investigate the impact of the traffic on the application and systems under test </li></ul></ul><ul><li>SOA Adds Complexity to Performance Testing </li></ul><ul><ul><li>Wide range of technologies </li></ul></ul><ul><ul><li>Many different applications and usages </li></ul></ul><ul><ul><li>Different hardware / infrastructure </li></ul></ul><ul><ul><li>Knowledge of the application and the technologies </li></ul></ul><ul><ul><li>Replicating traffic patterns </li></ul></ul>
  6. 7. SOA Performance Testing Challenges <ul><li>SOA being a distributed environment, finding right skill sets who possess in-depth knowledge of involved platforms, applications, databases and any middleware. </li></ul><ul><li>Assign appropriately skilled team of performance engineers to the test effort with knowledge of the systems to monitor and analyze the impact of testing </li></ul><ul><li>An in-depth understanding of the service is required for adequate testing and evaluation </li></ul><ul><li>A significant increase in testing activities and test assets (performance testing suites that include sophisticated harnesses and stubs) will be required at a service level </li></ul><ul><li>Predicting the future usage of services to assist with performance, load, stress, scalability </li></ul><ul><li>Test strategy differs from traditional testing and generally has to encompass many internal political factors e.g. ownership and responsibility </li></ul>
  7. 8. Overcoming Challenges <ul><li>To simplify performance testing for SOA applications break them down into the smallest components possible: </li></ul><ul><ul><li>Individual Service </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Databases </li></ul></ul><ul><ul><li>Technology </li></ul></ul><ul><ul><li>Protocols </li></ul></ul><ul><ul><li>Messaging </li></ul></ul><ul><ul><li>Functionality </li></ul></ul><ul><li>Evaluate and analyze the performance of individual services based on components </li></ul>
  8. 9. Methodology and Approach <ul><li>Discovery: Narrow the testing event to the smallest element/service and understand the transaction, application service and the environment/systems </li></ul><ul><li>Test Plan: Document the testing approach and the expected deliverables </li></ul><ul><li>Automation: Develop automation to replicate transactions </li></ul><ul><li>Test Execution: Conduct testing generating traffic increasing the traffic to pre-defined levels </li></ul><ul><li>Monitoring: Monitor the response time for the requests sent under varying traffic levels and the impact of the traffic on the application and infrastructure under varying traffic levels </li></ul><ul><li>Measurement and Analysis: Analyze the traffic patters to with the traffic/load </li></ul>
  9. 10. Discovery <ul><li>Identify the services and components to be included as part of the testing efforts </li></ul><ul><li>Identify lower level and/or external services called by the services under test </li></ul><ul><li>Identify the infrastructure hosting the services to be tested </li></ul><ul><li>Identify the messages sent to and received from the service that will be automated for generating load </li></ul><ul><li>Narrow the “functionality” to a subset that will be used for the testing </li></ul><ul><li>Define data that will be used and how it will be validated </li></ul><ul><li>Define pass/fail criteria (usually response time or transactions per second) </li></ul><ul><li>Identify skills required for the testing effort </li></ul><ul><li>Determine SLA or load/transactions for the testing </li></ul><ul><li>Prioritize testing </li></ul>
  10. 11. Test Planning <ul><li>Test Plan - Document the Following: </li></ul><ul><ul><li>Services and sub services </li></ul></ul><ul><ul><li>Infrastructure </li></ul></ul><ul><ul><li>Transaction </li></ul></ul><ul><ul><li>Automation </li></ul></ul><ul><ul><li>Data </li></ul></ul><ul><ul><li>SLA requirements (load/transactions) </li></ul></ul><ul><ul><li>Pass / fail criteria </li></ul></ul>
  11. 12. Automation and Tools <ul><li>Automation </li></ul><ul><ul><li>Subset of the functionality of the service </li></ul></ul><ul><ul><li>Prepare data and mechanism to validate </li></ul></ul><ul><ul><li>Tools to generate the transaction </li></ul></ul><ul><li>Proprietary SOA </li></ul><ul><ul><li>Custom Test Harness </li></ul></ul><ul><li>Enterprise Tools </li></ul><ul><ul><li>Loadrunner 9.1 </li></ul></ul><ul><ul><li>Parasoft SOAtest </li></ul></ul><ul><ul><li>Green Hat GH Tester </li></ul></ul><ul><ul><li>Borland SilkPerformer SOA </li></ul></ul><ul><li>OpenSource Tools </li></ul>
  12. 13. Test Execution <ul><li>Load Testing (up to defined SLA transactions per second or other) </li></ul><ul><li>Stress Testing (up to service failure) </li></ul><ul><li>Volume Testing (introducing large amounts of data into system) </li></ul><ul><li>Reliability Testing (high levels of load over long periods of time) </li></ul>
  13. 14. Monitoring <ul><li>Load Size (transactions per second/other) </li></ul><ul><li>Throughput </li></ul><ul><li>Response Time </li></ul><ul><li>Hardware </li></ul><ul><li>OS </li></ul><ul><li>Disk </li></ul><ul><li>Web (for Web Services) </li></ul><ul><li>Application specific counters </li></ul><ul><li>Database </li></ul>
  14. 15. Measurement and Analysis <ul><li>Size of message </li></ul><ul><li>Response time </li></ul><ul><li>Throughput under load </li></ul><ul><li>CPU memory disk </li></ul><ul><li>Application specific counters </li></ul>
  15. 16. Summary <ul><li>NEED 6 BULLET SUMMARY HERE </li></ul>
  16. 17. Questions <ul><li>Thank You </li></ul>