Testing SOA Architectures Ensuring Quality In Next-Generation ...


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing SOA Architectures Ensuring Quality In Next-Generation ...

  1. 1. Testing SOA Architectures Ensuring Quality In Next-Generation Applications Lee Copeland, Software Quality Engineering Consultant John Michelsen, iTKO, Inc. Chief Scientist and Co-Founder Thank you for joining us. We’ll be starting shortly.
  2. 2. Today’s Presenters Lee Copeland Consultant Software Quality Engineering John Michelsen Chief Scientist & Co-Founder iTKO, Inc.
  3. 3. Service Oriented Architecture Promises and Testing Challenges Lee Copeland [email_address] © SQE 2006
  4. 4. Service Oriented Architecture Promises © SQE 2006
  5. 5. The World As It Is Today <ul><li>Many of today’s applications are tightly coupled collections of data and function. </li></ul><ul><li>Reuse, redesign, and replacement of component parts can be very difficult. </li></ul>© SQE 2006 Function E Function D Database Function C Function B1 Function A User Interface Function B2 Database Application A
  6. 6. Innovations <ul><li>A key enabler of innovation in the IT industry is the myriad of available choices in hardware, software, and process technologies. </li></ul>© SQE 2006
  7. 7. Effects Of Emerging Technologies <ul><li>But, these new technologies emerge so rapidly that often only one application is built with a specific set of technologies. </li></ul>© SQE 2006 Application A Application B Application C
  8. 8. Incompatible Applications <ul><li>And so, over time, we build a set of successful applications that are incompatible with each other. </li></ul>© SQE 2006 Can’t reuse functionality Can’t share data Can’t communicate
  9. 9. The Concept Of SOA <ul><li>Service Oriented Architecture (SOA) is a holistic approach to implement business systems across a loosely coupled set of technologies. </li></ul><ul><li>Rather than continuing to build monolithic applications … </li></ul>© SQE 2006
  10. 10. The Concept Of SOA <ul><li>SOA is “a way of designing and implementing enterprise applications that deals with the intercommunication of loosely coupled, coarse grained (business level), reusable artifacts (services) that are accessed through well-defined, platform independent, interface contracts.&quot; </li></ul><ul><li>- Steve Wilkes </li></ul>© SQE 2006
  11. 11. SOA Application Design © SQE 2006 Function A Function C Function D Function B Function E User Interface Application A Business Level Function Component (Service)
  12. 12. The Concept Of SOA <ul><li>In fact, these services can from many sources and be distributed over heterogeneous environments. </li></ul>© SQE 2006 Developed Purchased Purchased 3 rd Party Shared Developed
  13. 13. Service Oriented Architecture Testing Challenges © SQE 2006
  14. 14. Testing Challenges <ul><li>Since SOA applications are composed of loosely coupled, business-level services, distributed over a network, we must test the application from end-to-end. </li></ul><ul><li>But, before we begin end-to-end testing, we should test the individual services. </li></ul>© SQE 2006
  15. 15. Testing SOA Services <ul><li>SOA services should be tested in the areas of: </li></ul><ul><ul><li>Functionality </li></ul></ul><ul><ul><li>API (Application Programming Interface) </li></ul></ul><ul><ul><ul><li>Parameter selection </li></ul></ul></ul><ul><ul><ul><li>Combinations </li></ul></ul></ul><ul><ul><ul><li>Sequencing </li></ul></ul></ul>© SQE 2006 Functionality (Service) API
  16. 16. Testing Functionality <ul><li>In a real sense, the overall functionality of SOA applications is easier to test. </li></ul><ul><li>That’s because ( but only if ) we thoroughly test the application’s components (services) before we assemble them to create the application. </li></ul><ul><li>Starting with lower-defect components means a smoother testing process BUT … </li></ul>© SQE 2006
  17. 17. Testing Functionality <ul><li>SOA applications generally have an increased number of: </li></ul><ul><ul><li>APIs (one for each service) </li></ul></ul><ul><ul><li>Communication paths between those services </li></ul></ul><ul><li>There is now an increased level of integration testing that must be performed. </li></ul>© SQE 2006
  18. 18. API Parameter Selection <ul><li>Consider a function </li></ul><ul><li>determineEmploymentStatus (int age) </li></ul><ul><li>that is to implement the following rules: </li></ul>© SQE 2006 Age 0-16 Don’t hire Age 16-18 Part time Age 18-55 Full time Age 55 or > Fire ‘em 0 16 18 55 What would you think about these test points? 20, 23, 25, 27, 30, 32, 37, 38, 40, 42, 50, 51
  19. 19. API Parameter Selection <ul><li>Services should be tested directly through the API to avoid GUI “filtering”. </li></ul>© SQE 2006 determineEmploymentStatus (DWORD age) Age
  20. 20. API Parameter Combinations © SQE 2006 <ul><li>Now let’s consider the function </li></ul><ul><li>doSomethingReallyUseful(parm1, parm2, parm3) </li></ul><ul><li>where: </li></ul><ul><li>parm1 has 6 equivalence classes </li></ul><ul><li>parm2 has 6 equivalence classes </li></ul><ul><li>parm3 has 6 equivalence classes </li></ul><ul><li>Choosing all combinations of parameters yields 216 different test cases – more than I want to do. </li></ul>
  21. 21. All-Pairs Testing <ul><li>An alternative to all-combinations testing is all-pairs testing in which pairs of parameters are combined for testing. </li></ul><ul><li>The reasoning behind all-pairs testing is this: the simplest bugs in a program are generally triggered by a single input parameter. The next most common category of bugs are those dependent on interactions between pairs of parameters, which can be discovered through all-pairs testing. </li></ul>© SQE 2006 A B C
  22. 22. All-Pairs Testing <ul><li>All-pairs testing can result in a significant reduction in the number of test cases required but still provide excellent defect find rates. </li></ul>© SQE 2006 5-20% of test cases 70-90% of defects
  23. 23. Service Invocation Sequencing <ul><li>SOA services are stateless. This means that they know nothing about functions that have executed before, and they require nothing of functions that will execute after. </li></ul><ul><li>But real applications have state, that is, they must guarantee that services are executed in a well-defined order. </li></ul><ul><li>A classic example is the Input-Process-Output sequence of most applications. </li></ul>© SQE 2006
  24. 24. Service Invocation Sequencing <ul><li>Well designed applications remember their state, allow valid transitions, and prevent invalid ones. </li></ul><ul><li>State-transition diagrams are a valuable tool in documenting the required temporal relationships. </li></ul>© SQE 2006
  25. 25. Service Invocation Sequencing <ul><li>Test cases are then derived from these state-transition diagrams. </li></ul><ul><li>Test cases should verify all services have been exercised, all transitions have been followed, and, if possible, all paths have been tested. </li></ul>© SQE 2006
  26. 26. Additional Testing Challenges <ul><li>Let’s now consider: </li></ul><ul><ul><li>Publish, Find, and Bind </li></ul></ul><ul><ul><li>Security </li></ul></ul><ul><ul><li>Interoperability </li></ul></ul><ul><ul><li>Performance </li></ul></ul>© SQE 2006
  27. 27. Publish, Find, And Bind <ul><li>Service providers must “advertise” their existence to brokers (automated catalogs). </li></ul><ul><li>Service requestors (your SOA application) must find providers and connect (bind) to them to use their services. </li></ul><ul><li>Service brokers must accept registrations and then provide information through search functions. </li></ul>© SQE 2006
  28. 28. Publish, Find, And Bind <ul><li>You didn’t have to test anything like this before because all the application’s functionality was there, right inside the application. </li></ul><ul><li>Are services your organization is providing able to properly register themselves? </li></ul><ul><li>Can your SOA application find and properly bind with services? </li></ul>© SQE 2006
  29. 29. Security © SQE 2006 <ul><li>An SOA application is a collection of independent services, collaborating to provide valuable functionality. </li></ul><ul><li>“ Valuable” often suggests the need for security, for the authentication of users before access. </li></ul><ul><li>Consider an application in which each service requires a different authentication approach and enforces different security policies – it’s a design, development, and testing nightmare. </li></ul>
  30. 30. Security <ul><li>Your organization needs a centralized SOA security management approach. </li></ul><ul><li>Various strategies include: </li></ul><ul><ul><li>Ignore the problem </li></ul></ul><ul><ul><li>Hide within a private network </li></ul></ul><ul><ul><li>Whip up a home-brewed solution </li></ul></ul><ul><ul><li>Buy this functionality from an experienced vendor </li></ul></ul><ul><li>and it all has to be tested. </li></ul>© SQE 2006
  31. 31. Interoperability <ul><li>Do you remember how Java’s </li></ul><ul><li>“ Write once, run anywhere.” quickly became “Write once, test everywhere.” </li></ul><ul><li>because of incompatible implementations? </li></ul><ul><li>Well, we’re seeing the same thing in the SOA world. In fact, it’s so problematic that the Web Services Interoperability Organization (WS-I) was created to deal with the problems. </li></ul>© SQE 2006
  32. 32. Interoperability <ul><li>Remember, just because it looks good in the glossy brochure </li></ul><ul><li>And in the vendor’s demo, </li></ul><ul><li>That doesn’t mean it will work for you. </li></ul>© SQE 2006
  33. 33. Performance <ul><li>All this loosely-coupled, platform-independent stuff is not free. </li></ul><ul><li>Major performance problems are often due to: </li></ul><ul><ul><li>Layer upon layer; abstraction upon abstraction </li></ul></ul><ul><ul><li>Small services with large overhead </li></ul></ul><ul><ul><li>Large services that are under supported by hardware </li></ul></ul><ul><ul><li>Services distributed across a network with its associated latency </li></ul></ul>© SQE 2006 It’s performance testing Jim, but not as we know it.
  34. 34. Performance <ul><li>You’ll need a tool that provides an integrated look across servers, networks, and layers. </li></ul>© SQE 2006
  35. 35. Testing Acquired Services <ul><li>What is the trustworthiness of acquired services? </li></ul><ul><ul><li>Functionality </li></ul></ul><ul><ul><li>APIs </li></ul></ul><ul><ul><li>Interface definition </li></ul></ul><ul><ul><li>Publish, find, and bind </li></ul></ul><ul><ul><li>Security </li></ul></ul><ul><ul><li>Interoperability </li></ul></ul><ul><ul><li>Performance </li></ul></ul>© SQE 2006 Or Are You
  36. 36. Testing Acquired Services <ul><li>How well did your vendor test? How do you know? </li></ul><ul><li>“ Classical” Acceptance Testing is required here (with all the work that it entails). </li></ul>© SQE 2006
  37. 37. Summary Of Testing Challenges <ul><li>Today we’ve discussed the promises and testing challenges in SOA applications: </li></ul><ul><ul><li>Functionality </li></ul></ul><ul><ul><li>APIs </li></ul></ul><ul><ul><li>Interface definition </li></ul></ul><ul><ul><li>Publish, Find, and Bind </li></ul></ul><ul><ul><li>Security </li></ul></ul><ul><ul><li>Interoperability </li></ul></ul><ul><ul><li>Performance </li></ul></ul>© SQE 2006
  38. 38. Thanks <ul><li>Thanks for joining with us today. </li></ul><ul><li>If I can be of assistance, or if you’d just like to chat, please contact me at </li></ul><ul><li>[email_address] </li></ul><ul><li>And now, I’m pleased to introduce John Michelsen … </li></ul>© SQE 2006