Testing SOA Architectures Ensuring Quality In Next-Generation ...
Upcoming SlideShare
Loading in...5
×
 

Testing SOA Architectures Ensuring Quality In Next-Generation ...

on

  • 435 views

 

Statistics

Views

Total Views
435
Views on SlideShare
435
Embed Views
0

Actions

Likes
0
Downloads
10
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Testing SOA Architectures Ensuring Quality In Next-Generation ... Testing SOA Architectures Ensuring Quality In Next-Generation ... Presentation Transcript

  • Testing SOA Architectures Ensuring Quality In Next-Generation Applications Lee Copeland, Software Quality Engineering Consultant John Michelsen, iTKO, Inc. Chief Scientist and Co-Founder Thank you for joining us. We’ll be starting shortly.
  • Today’s Presenters Lee Copeland Consultant Software Quality Engineering John Michelsen Chief Scientist & Co-Founder iTKO, Inc.
  • Service Oriented Architecture Promises and Testing Challenges Lee Copeland [email_address] © SQE 2006
  • Service Oriented Architecture Promises © SQE 2006
  • The World As It Is Today
    • Many of today’s applications are tightly coupled collections of data and function.
    • Reuse, redesign, and replacement of component parts can be very difficult.
    © SQE 2006 Function E Function D Database Function C Function B1 Function A User Interface Function B2 Database Application A
  • Innovations
    • A key enabler of innovation in the IT industry is the myriad of available choices in hardware, software, and process technologies.
    © SQE 2006
  • Effects Of Emerging Technologies
    • But, these new technologies emerge so rapidly that often only one application is built with a specific set of technologies.
    © SQE 2006 Application A Application B Application C
  • Incompatible Applications
    • And so, over time, we build a set of successful applications that are incompatible with each other.
    © SQE 2006 Can’t reuse functionality Can’t share data Can’t communicate
  • The Concept Of SOA
    • Service Oriented Architecture (SOA) is a holistic approach to implement business systems across a loosely coupled set of technologies.
    • Rather than continuing to build monolithic applications …
    © SQE 2006
  • The Concept Of SOA
    • SOA is “a way of designing and implementing enterprise applications that deals with the intercommunication of loosely coupled, coarse grained (business level), reusable artifacts (services) that are accessed through well-defined, platform independent, interface contracts."
    • - Steve Wilkes
    © SQE 2006
  • SOA Application Design © SQE 2006 Function A Function C Function D Function B Function E User Interface Application A Business Level Function Component (Service)
  • The Concept Of SOA
    • In fact, these services can from many sources and be distributed over heterogeneous environments.
    © SQE 2006 Developed Purchased Purchased 3 rd Party Shared Developed
  • Service Oriented Architecture Testing Challenges © SQE 2006
  • Testing Challenges
    • Since SOA applications are composed of loosely coupled, business-level services, distributed over a network, we must test the application from end-to-end.
    • But, before we begin end-to-end testing, we should test the individual services.
    © SQE 2006
  • Testing SOA Services
    • SOA services should be tested in the areas of:
      • Functionality
      • API (Application Programming Interface)
        • Parameter selection
        • Combinations
        • Sequencing
    © SQE 2006 Functionality (Service) API
  • Testing Functionality
    • In a real sense, the overall functionality of SOA applications is easier to test.
    • That’s because ( but only if ) we thoroughly test the application’s components (services) before we assemble them to create the application.
    • Starting with lower-defect components means a smoother testing process BUT …
    © SQE 2006
  • Testing Functionality
    • SOA applications generally have an increased number of:
      • APIs (one for each service)
      • Communication paths between those services
    • There is now an increased level of integration testing that must be performed.
    © SQE 2006
  • API Parameter Selection
    • Consider a function
    • determineEmploymentStatus (int age)
    • that is to implement the following rules:
    © SQE 2006 Age 0-16 Don’t hire Age 16-18 Part time Age 18-55 Full time Age 55 or > Fire ‘em 0 16 18 55 What would you think about these test points? 20, 23, 25, 27, 30, 32, 37, 38, 40, 42, 50, 51
  • API Parameter Selection
    • Services should be tested directly through the API to avoid GUI “filtering”.
    © SQE 2006 determineEmploymentStatus (DWORD age) Age
  • API Parameter Combinations © SQE 2006
    • Now let’s consider the function
    • doSomethingReallyUseful(parm1, parm2, parm3)
    • where:
    • parm1 has 6 equivalence classes
    • parm2 has 6 equivalence classes
    • parm3 has 6 equivalence classes
    • Choosing all combinations of parameters yields 216 different test cases – more than I want to do.
  • All-Pairs Testing
    • An alternative to all-combinations testing is all-pairs testing in which pairs of parameters are combined for testing.
    • The reasoning behind all-pairs testing is this: the simplest bugs in a program are generally triggered by a single input parameter. The next most common category of bugs are those dependent on interactions between pairs of parameters, which can be discovered through all-pairs testing.
    © SQE 2006 A B C
  • All-Pairs Testing
    • All-pairs testing can result in a significant reduction in the number of test cases required but still provide excellent defect find rates.
    © SQE 2006 5-20% of test cases 70-90% of defects
  • Service Invocation Sequencing
    • SOA services are stateless. This means that they know nothing about functions that have executed before, and they require nothing of functions that will execute after.
    • But real applications have state, that is, they must guarantee that services are executed in a well-defined order.
    • A classic example is the Input-Process-Output sequence of most applications.
    © SQE 2006
  • Service Invocation Sequencing
    • Well designed applications remember their state, allow valid transitions, and prevent invalid ones.
    • State-transition diagrams are a valuable tool in documenting the required temporal relationships.
    © SQE 2006
  • Service Invocation Sequencing
    • Test cases are then derived from these state-transition diagrams.
    • Test cases should verify all services have been exercised, all transitions have been followed, and, if possible, all paths have been tested.
    © SQE 2006
  • Additional Testing Challenges
    • Let’s now consider:
      • Publish, Find, and Bind
      • Security
      • Interoperability
      • Performance
    © SQE 2006
  • Publish, Find, And Bind
    • Service providers must “advertise” their existence to brokers (automated catalogs).
    • Service requestors (your SOA application) must find providers and connect (bind) to them to use their services.
    • Service brokers must accept registrations and then provide information through search functions.
    © SQE 2006
  • Publish, Find, And Bind
    • You didn’t have to test anything like this before because all the application’s functionality was there, right inside the application.
    • Are services your organization is providing able to properly register themselves?
    • Can your SOA application find and properly bind with services?
    © SQE 2006
  • Security © SQE 2006
    • An SOA application is a collection of independent services, collaborating to provide valuable functionality.
    • “ Valuable” often suggests the need for security, for the authentication of users before access.
    • Consider an application in which each service requires a different authentication approach and enforces different security policies – it’s a design, development, and testing nightmare.
  • Security
    • Your organization needs a centralized SOA security management approach.
    • Various strategies include:
      • Ignore the problem
      • Hide within a private network
      • Whip up a home-brewed solution
      • Buy this functionality from an experienced vendor
    • and it all has to be tested.
    © SQE 2006
  • Interoperability
    • Do you remember how Java’s
    • “ Write once, run anywhere.” quickly became “Write once, test everywhere.”
    • because of incompatible implementations?
    • Well, we’re seeing the same thing in the SOA world. In fact, it’s so problematic that the Web Services Interoperability Organization (WS-I) was created to deal with the problems.
    © SQE 2006
  • Interoperability
    • Remember, just because it looks good in the glossy brochure
    • And in the vendor’s demo,
    • That doesn’t mean it will work for you.
    © SQE 2006
  • Performance
    • All this loosely-coupled, platform-independent stuff is not free.
    • Major performance problems are often due to:
      • Layer upon layer; abstraction upon abstraction
      • Small services with large overhead
      • Large services that are under supported by hardware
      • Services distributed across a network with its associated latency
    © SQE 2006 It’s performance testing Jim, but not as we know it.
  • Performance
    • You’ll need a tool that provides an integrated look across servers, networks, and layers.
    © SQE 2006
  • Testing Acquired Services
    • What is the trustworthiness of acquired services?
      • Functionality
      • APIs
      • Interface definition
      • Publish, find, and bind
      • Security
      • Interoperability
      • Performance
    © SQE 2006 Or Are You
  • Testing Acquired Services
    • How well did your vendor test? How do you know?
    • “ Classical” Acceptance Testing is required here (with all the work that it entails).
    © SQE 2006
  • Summary Of Testing Challenges
    • Today we’ve discussed the promises and testing challenges in SOA applications:
      • Functionality
      • APIs
      • Interface definition
      • Publish, Find, and Bind
      • Security
      • Interoperability
      • Performance
    © SQE 2006
  • Thanks
    • Thanks for joining with us today.
    • If I can be of assistance, or if you’d just like to chat, please contact me at
    • [email_address]
    • And now, I’m pleased to introduce John Michelsen …
    © SQE 2006