Soa Performance Is A Critical Success Factor From AppLabs


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Soa Performance Is A Critical Success Factor From AppLabs

    1. 2. PDQ, SOA, SLA SOA Performance is a Critical Success Factor Ralph Decker Head of Performance Serviceline 7 th May 2008
    2. 3. Agenda <ul><li>What is SOA </li></ul><ul><li>Service Level Agreements (SLA’s) for SOA </li></ul><ul><li>Key to SOA Performance Testing </li></ul><ul><li>SOA Performance Testing Challenges </li></ul><ul><li>Overcoming Challenges </li></ul><ul><li>Methodology and Approach </li></ul><ul><ul><li>Discovery </li></ul></ul><ul><ul><li>Test Planning </li></ul></ul><ul><ul><li>Automation and Tools </li></ul></ul><ul><ul><li>Test Execution </li></ul></ul><ul><ul><li>Monitoring </li></ul></ul><ul><ul><li>Measurement and Analysis </li></ul></ul><ul><li>Summary </li></ul>
    3. 4. What is SOA <ul><li>Service Oriented Architecture </li></ul><ul><ul><li>Architectural paradigm (pattern/model) </li></ul></ul><ul><ul><li>Variety of heterogeneous systems (dissimilar) </li></ul></ul><ul><ul><li>Different locations and owners </li></ul></ul><ul><ul><li>Web Services? </li></ul></ul><ul><li>SOA provides benefits in four basic categories: </li></ul><ul><ul><li>Reduces expensive integration </li></ul></ul><ul><ul><li>Allows for more asset reuse </li></ul></ul><ul><ul><li>Increased business agility </li></ul></ul><ul><ul><li>Reduces business risk </li></ul></ul>
    4. 5. SLAs for SOA <ul><li>A service-level agreement (SLA) is a formal contract between a service provider and a consumer </li></ul><ul><ul><li>Service availability </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Traffic levels </li></ul></ul><ul><ul><li>Messages / queries per hour / minute / second </li></ul></ul><ul><ul><li>Response time </li></ul></ul><ul><ul><li>Rejected transactions </li></ul></ul><ul><ul><li>Errors </li></ul></ul><ul><ul><li>… </li></ul></ul>Poor Response Time Missed SLA Noncompliance with industry and government regulations
    5. 6. Key to SOA Performance Testing <ul><li>The key to successful performance testing in general require: </li></ul><ul><ul><li>Understanding the application and the infrastructure </li></ul></ul><ul><ul><li>Understanding the user/stakeholders of the application </li></ul></ul><ul><ul><li>Generate accurate anticipated volumes of traffic </li></ul></ul><ul><ul><li>Investigate the impact of the traffic on the application and systems under test </li></ul></ul><ul><li>SOA Adds Complexity to Performance Testing </li></ul><ul><ul><li>Wide range of technologies </li></ul></ul><ul><ul><li>Many different applications and usages </li></ul></ul><ul><ul><li>Different hardware / infrastructure </li></ul></ul><ul><ul><li>Knowledge of the application and the technologies </li></ul></ul><ul><ul><li>Replicating traffic patterns </li></ul></ul>
    6. 7. SOA Performance Testing Challenges <ul><li>SOA being a distributed environment, finding right skill sets who possess in-depth knowledge of involved platforms, applications, databases and any middleware. </li></ul><ul><li>Assign appropriately skilled team of performance engineers to the test effort with knowledge of the systems to monitor and analyze the impact of testing </li></ul><ul><li>An in-depth understanding of the service is required for adequate testing and evaluation </li></ul><ul><li>A significant increase in testing activities and test assets (performance testing suites that include sophisticated harnesses and stubs) will be required at a service level </li></ul><ul><li>Predicting the future usage of services to assist with performance, load, stress, scalability </li></ul><ul><li>Test strategy differs from traditional testing and generally has to encompass many internal political factors e.g. ownership and responsibility </li></ul>
    7. 8. Overcoming Challenges <ul><li>To simplify performance testing for SOA applications break them down into the smallest components possible: </li></ul><ul><ul><li>Individual Service </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Databases </li></ul></ul><ul><ul><li>Technology </li></ul></ul><ul><ul><li>Protocols </li></ul></ul><ul><ul><li>Messaging </li></ul></ul><ul><ul><li>Functionality </li></ul></ul><ul><li>Evaluate and analyze the performance of individual services based on components </li></ul>
    8. 9. Methodology and Approach <ul><li>Discovery: Narrow the testing event to the smallest element/service and understand the transaction, application service and the environment/systems </li></ul><ul><li>Test Plan: Document the testing approach and the expected deliverables </li></ul><ul><li>Automation: Develop automation to replicate transactions </li></ul><ul><li>Test Execution: Conduct testing generating traffic increasing the traffic to pre-defined levels </li></ul><ul><li>Monitoring: Monitor the response time for the requests sent under varying traffic levels and the impact of the traffic on the application and infrastructure under varying traffic levels </li></ul><ul><li>Measurement and Analysis: Analyze the traffic patters to with the traffic/load </li></ul>
    9. 10. Discovery <ul><li>Identify the services and components to be included as part of the testing efforts </li></ul><ul><li>Identify lower level and/or external services called by the services under test </li></ul><ul><li>Identify the infrastructure hosting the services to be tested </li></ul><ul><li>Identify the messages sent to and received from the service that will be automated for generating load </li></ul><ul><li>Narrow the “functionality” to a subset that will be used for the testing </li></ul><ul><li>Define data that will be used and how it will be validated </li></ul><ul><li>Define pass/fail criteria (usually response time or transactions per second) </li></ul><ul><li>Identify skills required for the testing effort </li></ul><ul><li>Determine SLA or load/transactions for the testing </li></ul><ul><li>Prioritize testing </li></ul>
    10. 11. Test Planning <ul><li>Test Plan - Document the Following: </li></ul><ul><ul><li>Services and sub services </li></ul></ul><ul><ul><li>Infrastructure </li></ul></ul><ul><ul><li>Transaction </li></ul></ul><ul><ul><li>Automation </li></ul></ul><ul><ul><li>Data </li></ul></ul><ul><ul><li>SLA requirements (load/transactions) </li></ul></ul><ul><ul><li>Pass / fail criteria </li></ul></ul>
    11. 12. Automation and Tools <ul><li>Automation </li></ul><ul><ul><li>Subset of the functionality of the service </li></ul></ul><ul><ul><li>Prepare data and mechanism to validate </li></ul></ul><ul><ul><li>Tools to generate the transaction </li></ul></ul><ul><li>Proprietary SOA </li></ul><ul><ul><li>Custom Test Harness </li></ul></ul><ul><li>Enterprise Tools </li></ul><ul><ul><li>Loadrunner 9.1 </li></ul></ul><ul><ul><li>Parasoft SOAtest </li></ul></ul><ul><ul><li>Green Hat GH Tester </li></ul></ul><ul><ul><li>Borland SilkPerformer SOA </li></ul></ul><ul><li>OpenSource Tools </li></ul>
    12. 13. Test Execution <ul><li>Load Testing (up to defined SLA transactions per second or other) </li></ul><ul><li>Stress Testing (up to service failure) </li></ul><ul><li>Volume Testing (introducing large amounts of data into system) </li></ul><ul><li>Reliability Testing (high levels of load over long periods of time) </li></ul>
    13. 14. Monitoring <ul><li>Load Size (transactions per second/other) </li></ul><ul><li>Throughput </li></ul><ul><li>Response Time </li></ul><ul><li>Hardware </li></ul><ul><li>OS </li></ul><ul><li>Disk </li></ul><ul><li>Web (for Web Services) </li></ul><ul><li>Application specific counters </li></ul><ul><li>Database </li></ul>
    14. 15. Measurement and Analysis <ul><li>Size of message </li></ul><ul><li>Response time </li></ul><ul><li>Throughput under load </li></ul><ul><li>CPU memory disk </li></ul><ul><li>Application specific counters </li></ul>
    15. 16. Summary <ul><li>NEED 6 BULLET SUMMARY HERE </li></ul>
    16. 17. Questions <ul><li>Thank You </li></ul>