Service-Oriented Architecture Testing Design and Practices


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Service-Oriented Architecture Testing Design and Practices

  1. 1. Service-Oriented Architecture Testing Design and Practices Alexandre A. dos Santos Júnior André Elia Assad Luiz Felipe S. L. Guimarães C.E.S.A.R / Motorola C.E.S.A.R / Motorola C.E.S.A.R / Motorola Abstract The proposed SOA testing techniques were designed during the JSR-232 TCK development at C.E.S.A.R. On Testing is an indispensable part of software this project the JCP group worked together with the development, and the testing techniques must keep always Mobile Expert Group (MEG) of the OSGi™ Alliance to evolving to address the new behaviors of software standardize the set of mobile services to run on top of applications. The procedural and object-oriented OSGi™ framework. Motorola (as a member of the OSGi paradigms have been driving the software development Alliance) actively contributed with MEG specifications, for a long time. A new paradigm is been progressively and C.E.S.A.R. was responsible for doing the compliance explored: the service-oriented. This new model is testing of these new services. composed by three entities that have the following functionalities: to publish a service (the service provider); 1.1. Scope to find, to invoke a service (the service requester); to bind, and to store available services (the service registry). The concept of service-orientation, which is the This paper introduces the concepts of service-oriented foundation of this work, is discussed in the next section. architecture, and presents some innovative testing design Section 3 presents a unit test approach for SOA and practices for verifying and validating service- applications. Section 4 describes techniques for testing oriented applications. services contracts. Section 5 shows how to make automatic API testing by identifying some patterns in the Keywords: Service-oriented, unit test, reflection, tested methods. Section 6 presents our conclusion about service API testing, automatic testing, test patterns. SOA testing design and practices. References are given at section 7. 1. Introduction 2. Service-orientation Software applications are gradually evolving from the In a service-oriented environment applications are object-orientation to service-orientation paradigm, this is assembled from reusable building blocks [1] called clearly noted with the success and adoption of web services. On this environment services are described and services in a wide variety of enterprise applications. organized to support their dynamic, automated discovery Today there are a number of application frameworks, such and use. In this model services can be used by arbitrary as J2EE, ASP .NET, OSGi™, RPG and others, that uses clients, and the service providers are not tied to any Service-Oriented Architecture (SOA) to provide users' particular client, instead, the service providers are applications a collaborative environment where business interchangeable [2]. parties can export and import services. In a service-oriented environment there are always Even though SOA is been gradually integrated on three entities: service provider, service requester, and the enterprises applications there are a lack of testing registry. The services provider publishes a service techniques designed for SOA applications in the software description and provides the implementation for the engineering literature. On this light, new techniques are service; service requesters finds the service description necessary to verify and validate applications based on in the service registry, binds, and invokes the service; the SOA concepts. This paper proposes test cases design and service registry is a directory that contains available practices to be used as a testing pattern in any automatic services, this entity accepts and stores contracts from testing process that aims to evaluate the behavior of SOA services providers and provides these contracts to applications. These patterns were designed for Java interested service requesters [3]. The contract is the language; however they are somehow agnostic, and can be specification of the way a requester of a service will implemented using other programming languages. This interact with the service provider. It specifies the format work focuses on unit tests, services contracts tests, and of the request and response from the service. Figure 1, automatic API testing. shows the architectural design of SOA applications.
  2. 2. test optional services features or not. Figure 2 indicates Service Provider Bind Service Requester that test cases should avoid users’ interactions in order to keep the integrity of the test results. Contract Publish Discover Test Case Service Registry Figure 1 - Service-oriented design 2.1. Service-oriented test guidelines Avoid user Get configuration When developing a component, it is indispensable to interaction inputs from files test the correctness of the code. Therefore, planning and executing test cases are an excellent way to assure a well- Figure 2 - Configuration design done work. However, it is recommended that the tests be done along with the product being developed. In that way, the errors can be detected as early as possible (even in On the other hand the tests may disregard some good requirements and design phases), avoiding rework. practices, such as when the test is executed in an Testing is not the final hurdle to release the code [4], it environment where the developer can share static must be treated as a continuous activity. variables with the services under test. Sometimes this is There are many thoughts to keep in mind when writing the only way a test case can evaluate the behavior of a test cases for a SOA application, which is quite different service after removal of the service reference. For user from a user oriented application. The table below shows a applications it is not recommended to share static comparison of these two kinds of applications testing: variables among threads and classes because it is not trivial to keep the synchronization of the variable value. However for testing purposes this approach overcomes the Table 1 - User Oriented x Service-Oriented problem of getting key values from the service under test User Oriented Service-Oriented after its removal from the registry. Figure 3 shows the Bases tests on requirements Bases tests on services design of a test case that shares static variables with contracts services under test in order to get valuable data for the test Tested API is never Normally extends services results. changed interfaces for testing Business process is usually Business process is a set of Test environment an entity services Static Variable Assumes components will Assumes components have Uses be always present a dynamic availability Test Case Share Service under Test However there are also some good practices of non- functional requirements of software engineering that may be considered and sometimes disregarded [5] when writing test cases for SOA applications. We will focus on Figure 3 - Static variable sharing robustness, performance, ease of use, and coding. 2.1.2. Performance 2.1.1. Robustness Performance is not a primary concern when writing test Test case developers must always try to perform cases, actually there is no non-functional requirement that automatic testing when possible. It is not a good practice is more important than generating a good quality test to ask users for inputs or any kind of interaction, it may result. For instance test case developers must do all efforts lead to unexpected test results. Users may provide a to cleanup a test case state before the execution of another wrong input to the test case, and this would be very test. Otherwise the unwanted state left on the application difficult to identify. A separate test case properties file may lead the test suit to produce a wrong test result. could be used to overcome this problem, this way test cases could get all the input values, and decide whether to
  3. 3. For example, in a SOA environment this means if a test When writing a test suite for many different services, case may have to register a service for testing, it must be you need to consider whether these services are optional unregistered after the test case execution; otherwise the or not. The mandatory services must always be tested: if service registry will contain unwanted services references. they do not exist in the service registry, the test case must fail. On the other hand, optional services may not exist 2.1.3. Ease of use and, even so, the test must not fail. It should be shown in the test result as a warning or something similar. It also In a SOA environment there are many events that may impacts integration tests, because in a SOA environment occur asynchronously. These events may also have to be the components have a dynamic availability. validated in some test suits. Therefore test case developers should never care about leaving test executers “waiting 3.1. Unit test pattern forever” for asynchronous events, yet another definition for do not handle timeout. When writing unit test a good pattern is to separate If the tests are to be executed in an embedded device it each method of the tested API in different classes giving is not predictable how much time an event will take to the class the same name of the method under test, and occur. So, it is recommended to leave the responsibility of each functional aspect in a different test case. Doing this stopping the execution, due to a timeout failure, to the test is better than putting many different tests in only one test framework. case, because it improves the readability of the code, facilitating the understanding of what the test case really 2.1.4. Coding does. For each test case you can write a specific documentation, associated with a unique identification The test cases must validate the services interfaces, number, hence that code can be analyzed by everyone. not the services implementations. In other words, the test Even the test results should have this identification, cases developers should not read the services avoiding waste of time when finding which test case implementation source code in order to write the test failed, debugging or anything similar. cases. One of the key principles of service-orientation is to hide the implementation details, so the only interface between implementers and test case developers are the services specification. In case of a Java services, the JavaDoc of the services interfaces is the main source for writing the test cases. 3. Unit tests in a service-oriented architecture A unit test is a code that tests individual units of software or hardware. It is done by executing a method and checking if the result was the expected. In case of SOA it can be an interface or a method [6]. Normally, the developer who has created the code writes the unit test while or even before writing the program itself [7]. There Figure 4 - Unit test pattern are some exceptions, for example, when testing standardized technologies. In this case, it is necessary to have a testing team just for evaluating the compliance of The Figure 4 - Unit test pattern above illustrates this the standards implementation. pattern. There is an interface called IntefaceUnderTest For SOA test cases, the validations are based on the (the service being tested) that has two methods, methodA behavior of a specified service (contracts and use cases), and methodB. In this case would exist a package called defined by an interface. Thus, you do not know how it is IntefaceUnderTest and two classes (called MethodA and implemented (black-box approach), all you know is the MethodB), each one tests its own method. Supposing that behavior and features of that service [8]. Therefore you you find out three different tests needed to be done for the cannot ensure anything about that code, even if it seems first method, they would be inside the respective class obvious. MethodA and would be called from testMethodA001 to Testing services is a bit different from testing objects testMethodA003 following the pattern test[class with the because you cannot directly instantiate them, you need to same method name][test index with three digits]. get the instances from the service registry before using There can be some utility methods to be used them. Consequently the services must be registered before commonly by these test methods preventing code being gotten. What you get is defined by the interface, and replication. All of these test methods should be private to its implementation is not visible. its respective class. The unit tests for a given class should
  4. 4. be executed by calling a public void method, called run. public some handy methods that will expose key values of To facilitate the readability of the testing code this run the service execution. Therefore instead of registering a method should be called from the TestControl class in a ServiceInterface implementation in the service registry, method which follows the pattern: test[interface under the test case developer must now register a test][class with the same method name]. In our example it ServiceInterfaceTestImpl as the implementation for the would be called testInterfaceUnderTestMethodA and ServiceInterface contract. The Figure 6 - Services testInterfaceUnderTestMethodB. contracts testing pattern shows a pattern that can be used to verify the services contracts implementations. 4. Services contracts testing In a service-oriented architecture an interface contract is a published agreement between a service provider and a service consumer, the specification of the way the requester will interact with the publisher. During this find and execute process the requester must fulfill a set of preconditions and post conditions, in order to be in conformity with the contract specification. Suppose a service Publisher has a known interface by the service Requester; however every time the Requester access the Publisher, the latter must invoke the Contract 1, which will do some important actions for the application. Although the execution of Contract 1 is hidden from the Requester its execution must not fail, otherwise the whole invocation of Publisher service will fail. Figure 5 - Services contract design – shows an overview of a services contract design. Figure 6 - Services contracts testing pattern Component A Component B For instance, consider a cash-machine service. The Publisher Requester service requires that a client validation request contain the account number, amount, and a special system code [9]. The service uses the system code to determine in which internal database to find the account. There is no reason to Contract 1 Contract 2 expose the system code outside the component that validates the client, because the service should identify the database itself based on the functional data passed into it. This information is only necessary for implementation, but Figure 5 - Services contract design the execution will fail if the system code is wrongly generated by the validation component. The extended testing interface that tests the client validation should have 4.1. Services contracts test pattern a method that returns the hidden system code. Thus, if test case developers use the proposed pattern, the problem of Although the contracts are necessarily hidden from writing a unit test to validate system code generations in other components, for testing purposes it must be client validations services components of cash-machines, transformed from an internal structure to an external can be overcome. structure. Exposing functionalities is strictly bound to public interfaces, so to test the services contracts we must use a tricky technique that uses interfaces inheritance 5. Services API testing with reflection extending the service contracts to expose key values for later evaluation of the service execution. Firstly, it is necessary to understand what reflection is Considering the contract has a published and its advantages before understanding its application in ServiceInterface with methods that will do something for services API testing [10]. the component. In order to evaluate the execution of each Reflection in a programming language context refers to public method a test case developer could extend the the ability to observe and/or manipulate the inner ServiceInterface to a ServiceInterfaceTest, which will workings of the environment programmatically [11]. For have a ServiceInterfaceTestImpl implementation, to make example, it is possible to discover the methods of a given
  5. 5. class and invoke any method as necessary. Note that you their exception. It is important to note that each group of do not have to know the methods names, and you can also method will be tested in a separated test method. discover how many parameters each method has. Observe that is not common to test methods that have Reflection is a powerful tool that can improve the team the same actions but are in different classes, it is productivity, the code legibility and as result, the test suit recommended to group only the methods that are in a user get an effectiveness tests result that is definitely the common class, otherwise, it will be difficult to identify goal of a test suit. which test has failed, if it was on class A or class B for There are some myths about reflection that must be example. clarified before using it [11]. Two cases are described The other methods that have a specific behavior may below: continue being tested normally, without using reflection. It is important to remember to only use reflection when the • “Reflection is too complex for use in test class or interface has many methods with the same applications”. This assumption is false. It behavior. Figure 8 shows the class design of a test case requires a skill that is easily mastered, and at that uses reflection in order to automatically test the end, you will have reduced the footprint of services’ interfaces methods. your test suit. • “Reflection reduces applications performance” This assumption is also false. It depends of the test design. It can actually increase the performance of the code. It can also simplify the source code design and greatly expand the capabilities of the test suit. Besides performance is not a primary requirement of the test cases. At this point, you know what reflection is, so the concepts and patterns of API testing can now be Figure 8 - API testing with reflection introduced. As said before, there are many advantages of using reflection in test cases design but be careful when Additionally, that there are some cases, in which the choosing what techniques will be used and what will not reflection use will not be so clear, such as when there are be used. When using reflection it is hard to forget to test a only two or three methods which have the same behavior. method but there is a problem when finding and grouping In these cases you can choose to use reflection or not, it is the methods, reflection API will try to return the first your choice, it depends on your project. For example, if instance of a method found dynamically [12]. Therefore, the class has three methods but in a near future is possible if the methods are overloaded, it is difficult to find the to have others methods added to this class, you may exact method with the right arguments. choose to use it. The most important thing is to identify which methods It is not recommended to use many control flow could be tested. It is necessary to distinguish which statements (such as switch, if/else) inside a reflection test methods have the same behavior and group them by what method, if you are using it, you probably should revise they have in common, for example group all the methods your code verifying if you can test them without using that throw the same exception in a specific class (See reflection or if you have to split them in two or three Figure 7 - Grouping methods). distinct groups. 7. Conclusion Even though application testers have decades of experience in software testing, there are many mysteries yet to solve, in order to verify and validate complex software. As complexity grows, researchers find more innovative ways to address the problem of software testing. SOA is the latest trend to be solved; it addresses Figure 7 - Grouping methods the problem of applications integration, system availability, reliability, scalability, and modularity. As enterprises gradually integrate SOA solutions into Get all the methods that receive one parameter and their suit of applications, the complexity of these solutions throw NullPointerException and use a test method to test grows, as well as the quality assurance process.
  6. 6. Since testing frameworks and practices driven to SOA environments are missing in the software engineering literature, this paper contributes with new testing techniques to evaluate the behavior of SOA applications, such a contribution are the best practices used during the development of the JSR-232 TCK. A part from that, we found no other work related to SOA testing, which made it difficult to evaluate, and compare our solution with other works. 8. References [1] H. Cervantes and R.S. Hall, “Autonomous Adaptation to Dynamic Availability Using a Service-Oriented Component Model,” Proceedings of the International Conference on Software Engineering, May 2004; [2] H. Cervantes and R.S. Hall, “Automating Service Dependency Management in a Service-Oriented Component Model,” Proceedings of the Sixth Component-Based Software Engineering Workshop, pp. 91-96, May 2003; [3] Ali Arsanjani. “Service-oriented modeling and architecture”, http://www- design1/, November 2004; [4] Jon Thomas, Matthew Young, Kyle Brown, Andrew Glover, “Java Testing Patterns”, Wiley, Indianapolis, Indiana, October 2004; [5] Sean Shubin, “Test First Guidelines”, XP Magazine,, January 2002; [6] Bridge Information Technologies, “Software Quality Assurance Processes”, QA-PROCESS.pdf, 2003; [7] Per Madsen, "Unit testing using design by contract and equivalence partitions", XP2003, Genova, 2003; [8] Tanja Toroi. "How to Test Software". University of Kuopio, March 2002; [9] James McGovern, Sameer Tyagi, Michael Stevens, and Sunil Mathew, “Java Web Services Architecture”, Chapter 2: Service-Oriented Architecture, Morgan Kaufmann Publishers, May 2003; [10] D. Green, “The Java Tutorial: The Reflection API”, Sun Microsystems, Inc,, 2005; [11] Michael T. Portwood, “Using Java™ Technology Reflection to improve Design”, Java One Conference, San Francisco, 2000; [12] Matthew A. Brown, "Validation Strategy", 9h Conference on Pattern Language of Programs, Illinois, 2002.