An Assertion Mechanism for Software Unit Testing to Remain ...


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

An Assertion Mechanism for Software Unit Testing to Remain ...

  1. 1. An Assertion Mechanism for Software Unit Testing to Remain Unaffected by Program Modification The Mechanism to Eliminate Dependency from/to Unnecessary Object Masanori SAKAKIBARA a,1 , Kohei SAKURAI b and Seiichi KOMIYA a a Graduate School of Engineering, Shibaura Institute of Technology b Graduate School of Arts and Sciences, the University of Tokyo Abstract. Assertions are used for unit testing in Object-Oriented Programming. With assertions, a test programmer compares with an expected value the result of a test execution which is obtained by referring to the object containing its result in the object being tested. Therefore, there is a problem that test classes described as testing code depend strongly on the unnecessary objects which are not of concern to unit testing. In this work we developed an assertion mechanism which traverses objects automatically in order to eliminate the dependency from/to such unneces- sary objects. Furthermore, we performed experiments on open source products, and confirmed that using this assertion mechanism decreases the coupling between a test class and other classes. Keywords. Unit testing, assertion, Object-Oriented Programming Introduction In the software development using object oriented programming (OOP), unit testing en- sure the software reliability. In unit testing, a test programmer write multiple test cases for each module with which the program being tested is configured. Therefore, running a test case with a test driver, unit testing are performed. In the software development using OOP language, like Java, the JUnit testing frame- work [6] are often used as a test driver. With JUnit the test case is described as a class (hereafter called a test class), and running test cases is automated. A test class targeted a certain class and its methods for testing and mainly the fol- lowing description is made. First, the specific test input for the method of a test, and the object of the class for the test is supplied. Next the method to be tested are executed with the supplied input. 1 Shibaura Institute of Technology toyosu 3-7-5, etoku, Tokyo, 135-8548 Japan; E-mail:
  2. 2. Finally the output of the method or the values of changed state are compared with the expected results using assertions. If the comparison check of the assertion fails, it means there are some defects in the method to be tested. The value used for comparison of assertion are obtained through reference of other related objects from the object of the class being tested. In OOP, by changing the program being tested by refactoring etc. the definition of the class or the method of the class, not being tested, get changed in many cases. As a result, the test programmer has to change the test class manually. Such changes need to be avoided by improving the robustness of the test classes related to the changes in the programs being tested. In this work, regarding the unit test, from the objects being tested at the start to the value (executed results) of the objects, which can be reached, when comparing with assertions, the problem of the robustness of changes to the programs is discussed. The objects of the class not to be tested, are called unnecessary objects. The dependency between the object being tested and such unnecessary objects is pointed as decreasing the robustness of the test class. And in order to eliminate the dependency between unnecessary objects, we pro- pose TraversalAssert, a new assertion mechanism, developed with the mechanism Deme- ter [4], which uses traversal. The Demeter has the functions to traverse the graph (object graph) by the reference relation of an object, indirectly related to another object, and obtaining its relation references. In order to use the traversal easily in unit testing, the authors have developed an assertion mechanism as a library, to traverse the object graph on the basis of the object of the class for testing. In this paper, this assertion mechanism is applied to the actual open source test pro- gram, and it explains that the dependency between unnecessary objects can be eliminated from a test class. Furthermore, the verification experiment was conducted for the several open source software products, and quantitative evaluation was performed using the degree of cou- pling between the classes [3], which are well known software matrics. The configuration after this paper is, as follows. Section 1, explains the problems regarding the robustness of unit test by a concrete example. Section 2, explains about an assertion mechanism to be proposed. Section 3, explains the implementation of the proposed assertion mechanism. Section 4, explains the validity of the proposed assertion mechanism by experiment and check of the results. Section 5, explains about the related work. Section 6, gives the conclusions and the future work. 1. Motivation Example In this section, the problem of the dependency between the unnecessary objects in the test class for unit testing are explained by using an actual example. Commons-Betwixt2 currently being developed by Jakarta Apache Project as open source software is taken up. Commons-Betwixt is a library to inter-relate Java beans and XML document. Henceforth, in Section 1.1, an actual test class is taken from Common-Betwixt and explained. In Section 1.2, problems are explained. 2
  3. 3. 1 public class TestXMLBeanInfoDigester ... 2 public void testDigester() { 3 XMLBeanInfoDigester digester =... 4 InputStream in =... 5 XMLBeanInfo info= 6 (XMLBeanInfo) digester.parse(in); 7 ElementDescriptor descriptor = 8 info.getElementDescriptor(); 9 .../*check null of descriptor*/ 10 ElementDescriptor[] elements = 11 descriptor.getElementDescriptors(); 12 .../*check null of elements*/ 13 assertEquals("channel", 14 elements[0].getLocalName()); 15 } 16 } Figure 1. A sample test class. 1.1. Example: Test Class for Syntax Analysis Program In Fig. 1, TestXMLBeanInfoDigester class is taken in parts from the definition of a test class for XMLBeanInfoDigester to be tested. In general, unit testing is performed by executing such test classes after the class XMLBeanInfoDigester and other related classes are implemented. The method testDigester is targeted for testing syntax analysis of XML docu- ment with the method parse. Actually, the object of class XMLBeanInfoDigester and the parameters of class InputStream are instantiated and set up (lines 3-4). After calling the method parse with parameter in together (line 6), the returned value of the method is checked (lines 7-14). In order to check the returned value of the method parse, the test programmer gives the expected value which serves as the element name in an XML document (in Fig. 1, line 13 "channel") to the test class, and the value has to be compared with the actual element name, called by the assert statement (assertEquals)(lines 13-14). In order to get the actual value, from the object of type XMLBeanInfo (lines 7-8), the object of type ElementDescriptor (lines 10-11) is acquired in order. 1.2. The Dependency on Unnecessary Objects The code which calls methods one after the another in an object which has a reference relation with a certain object is called Law of Demeter(LoD)[5], and is against the style of calling program. Here, the reference relation of an object calls the method from a certain object, and by referring to the fields or referring to the table having those results can acquire the references, and points to the relation with other objects. The programs which do not obey LoD, have a difficult structure and to maintain. LoD must use only the module of the limited object closely related to the module. Although LoD is the rules of a programming style in developing software program, we consider applying the definition of LoD to the test class. The limited object which is
  4. 4. XMLBeanInfoDigester digester XMLBeanInfo info ElementDescriptor descriptor ElementDescriptor elements[0] "cannel" Figure 2. Partial object graph. closely connected with a test class, 1) the object for test, 2) the input object for the test, 3 and 3) the object of the values for the check of output results or state of changes used in assertion. The class and method of these object groups can be called the limited modules, which are closely related to test class. All other objects, which have reference relation to the test related objects used in test class are unnecessary. Unnecessary object is used mainly in order to acquire the actual value of the result to compare with expected value. Fig. 1 checks the return value of method by sending method to the object of XMLBeanInfo and ElementDescriptor in order, including the violation code of LoD. Fig. 1 is the partial object graph made from the reference relation of the object when the code of Fig. 1 is executed. The ellipses in the figure show the objects and the arrows show the reference relation between the objects. • The top object of the Fig. 2 shows a object stored in the variable digester of XMLBeanInfoDigester, which is the target class for testing in Fig. 1. • The "channel" at the bottom of Fig. 2 is the object of type String, which should be checked in the test. It is referred when the method of line 14 is called in Fig. 1. • In the middle of Fig. 2, 3 objects of types XMLBeanInfo and ElementDescriptor are said as the unnecessary objects, which causes violation of LoD. These unnec- essary objects are referred in lines 7-13 of Fig. 1. The objects stored in the vari- able info of type XMLBeanInfo, second from top, are referred as the result of when method parse is called for top ranking objects. The robustness of a test class is decreased by LoD violation of a test class, be- cause the presence of the unnecessary objects, which have reference relation from the object of the class for a test, can receive the changes effects of the unnecessary ob- jects. For example, when the name of the method of the class ElementDescriptor, which is the type of the unnecessary objects used in Fig. 1, is changed the codes of the test class also need to coincide with the name. Moreover, when the definitions of class 3 The classes of objects for inputs are prepared for each test class by using a stub or Mock Object etc. So, it can be said that they are closely connected with a test class.
  5. 5. 1 public class TestXMLBeanInfoDigester ... 2 public void testDigester() { 3 XMLBeanInfoDigester digester =... 4 InputStream in =... 5 from(digester.parse(in)) 6 .to("channel").verify(); 7 } 8 } Figure 3. Test class rewritten with TraversalAssert. ElementDescriptor are deleted, a test programmer have to rewrite the procedure, which acquires the actual element name. 2. Assertion Mechanism for Elminating The Dependency on Unnecessary Objects By using the mechanism of Demeter[4] for traversal the problems of violation of LoD regarding the test class, explained in Section 1.2, is resolved by developing assertion mechanism for unit testing. First in Section 2.1, below, the explanation of Demeter to be used in the develop- ment of assertion mechanism is given. Then Section 2.2 outlines an example of use of assertion mechanism, and advantages. 2.1. Demeter: Mechanism for Traverse Demeter is the mechanism proposed in order to realize LoD. By automatically traversing the objects graph with Demeter, it becomes unneeded to acquire reference of objects other than the objects proposed at the top of object graph. As a result, codes, which call methods one after another for the objects in the reference relation are not required, and LoD can be realized. Demeter traverses the graphs of reference relation of the starting objects by the given traversal strategy, and executes procedure to the reached object. Traversal strategy is the requirement of the path to object graph. The strings about the type of objects are defined by declaration of valid expressions. For example, about the object graph at the top of Fig. 2, to arrive at the end object "channel", the path to object String, which is ob- tained by the reference relation of from the starting object XMLBeanInfoDigester to the object XMLBeanInfo and 2 objects ElementDescriptor, is given as traver- sal strategy. DJ[8] is developed as a processor of Demeter. DJ can be used as a library of a Java language without any extension of the language. DJ describes literal a character string of Java as traversal strategy. The visitor object is given as the executing procedure for the reached object. 2.2. Outlines of Assertion Mechanism The Assertion library TraversalAssert for testing was developed in order to describe Demeter by assert statement, without extending Java language. In order to generalize
  6. 6. the processing of the existing Demeter, such as DJ, when used in test classes includes many lengthy or duplicate descriptions in the traversal strategy or the procedures. The developed assertion mechanism applied beforehand the traversal strategy and procedure, which are often used to check of the values of test class results, as application program- ming interface (API). TraversalAssert as described in Section 2.1, is if the comparison of actual results and expected values is prepared for test input, it can be used any time for the usual assert statement assertEquals and also for the test related class by using API. Fig. 3 is the test class, rewritten by the foregoing example using the TraversalAssert library. In lines 5-6, it explains that from the object XMLBeanInfoDigester it can reach to the String "channel". When a test class of Fig. 3 is executed, In TraversalAssert the object graph included Fig. 2 is automatically traversed by the visitor in the assertion mechanism. If the object of the type String "channel", which is the target object, can be reached, the assertion is satisfied and the test becomes successful. If it is not reached, the test fails and an error message is outputted, as follows. not XMLBeanInfoDigester -> "channel" The test class using TraversalAssert does not require the description of acquir- ing the reference of unnecessary objects manually, their dependency is not required and LoD is realized. In Fig. 3, there is no need to describe the codes acquired by reference of XMLBeanInfo objects and ElementDescriptor objects from XMLBeanInfoDigester objects by comparing with Fig. 1. Therefore, it is hard to get the changes effects of the classes of XMLBeanInfo or ElementDescriptor and so increase the robustness regarding changes. For exam- ple, even if the method name of ElementDescriptor changes, or class definitions are deleted, by implementing the test class without re-writing, the unit test can be done. 3. Implementation of Assertion Mechanism This section briefly explains an implementation of TraversalAssert. The authors imple- mented a prototype of TraversalAssert as a Pure Java library by using DJ[8], which real- izes LoD through traversal, with about 1100 lines of code. In DJ the class graph, expressed by the class relation of the class in the specified package is generated. For example, by loading the code of Fig. 3, the class graph shown in Fig. 4 is generated. Regarding the class graph, which class is to be used as traversal source, and through which class, and to traverse till which class, defined traversal strategy can be specified. In TraversalAssert, by using traversal strategy and the object visitor to perform traversal, successful functional achievement of assertion mechanism is realized. In order to check that performance, the time required to carry the test regarding 19 test classes of Commons-Betwixt, is measured. The measured environments were with Intel CPU, Celeron 3.2GHz, memory 512MB, Linux (kernel 2.6.9-42) has JavaVM version 1.5.0. As a result, according to TraversalAssert, required more than 200 seconds after rewriting as compared to 2 seconds before rewriting.
  7. 7. NodeDescriptor String XMLBeanInfo ElementDescriptor Figure 4. Partial class graph. The time difference before and after rewriting is the time to traverse automatically the graph of the objects of origin for testing. In other words, before rewriting, from the objects being tested through the related objects, the results of a test execution obtained manually. And after rewriting, at the time of test execution, by traversing the results from the object being tested, it was found that the manual job is done automatically. In future, it is necessary to optimize the efficiency of traversal. 4. Verification Experiment In order to examine the validity of developed TraversalAssert, the verification experiment by comparison of assert statement of JUint and the case when TraversalAssert is applied, was done. The program for the experiment selected the software product of Jakarta project Commons-Betwixt4 , Commons-Digester5 , and Commons-Scxml6 , HttpCore7 . A test class exists for these products. In order to evaluate, Coupling between Object Classes (CBO) [3], which is one of the matrices of CK is used. CBO is the number of other classes which a class is coupled to. The coupling here means, a certain class is referring to the methods or argument instances of other class. The class of high CBO value depends on many other classes, and it is easy to get influenced by changes. So the robustness of the test class is lowered. By experiment, CBO value of the test classes existing in each product and the rewrit- ten test class by TraversalAssert, was measured. But the classes included in the standard library, arranged type objects, primitive types, and the classes in the JUnit and Traver- salAssert library with no changes are not counted as the classes included in CBO value. Further the test classes as they are not counted, but the internal classes are counted. The result of measurement is summarized as follows. • Table 1 shows the total number of test classes for each product, and test class number with which CBO value was lowered after rewriting. • Table 2 shows the total number of packages for each product, and the pack- ages number, including the test classes with which CBO value was lowered after rewriting. • Table 3 shows the low, high, average comparative ratio of CBO value before rewriting, and lower value after rewriting. 4 5 6 7
  8. 8. Table 1. The number of test class with lowered CBO value. Product Total Lowered number Ratio Commons-Betxixt 103 19 0.18 Commons-Digester 32 7 0.22 Commons-Scxml 32 15 0.47 HttpCore 61 9 0.15 Table 2. The number of package which include test class with lowered CBO value. Product Total Included number Ratio Commons-Betxixt 19 9 0.47 Commons-Digester 4 2 0.50 Commons-Scxml 10 5 0.50 HttpCore 12 7 0.58 Table 3. Lowered rate of CBO value. Product Min Max Average Commons-Betxixt 0.077 0.667 0.240 Commons-Digester 0.143 0.250 0.179 Commons-Scxml 0.100 0.500 0.262 HttpCore 0.083 0.500 0.219 With TraversalAssert library 20 percent test classes of the total test classes the value of CBO was revised, lowered. The packages containing the test classes, which revised (lowered) CBO value, are about 50 percent of the total packages, and were greater than the ratio of test classes. This shows that the dependency of test classes between unnec- essary objects are scattered over many packages. If the packages design is based on the functional unit of products, not only for a special package, but TraversalAssert library can be used for various packages. Moreover, the number of dependent classes of the unnecessary objects, were able to be eliminated about 1-3 times. Because of this, CBO value of test classes, were able to be lowered by 20 percent as a result. The maximum test classes of the ratio, which lowered CBO value, the number of objects required for the setup regarding the test was low. On the other hand, the minimum test classes of the ratio, which lowered CBO value, the number of objects required for the setup regarding the testing was high. How much the TraversalAssert library can lower the value of CBO depends upon the number of necessary objects required for setup of testing. 5. Related Work In unit testing, to guarantee the quality for each module, the technology and the problems for are described. 5.1. Mock object Mock Objects [9] are the techniques to support the test-drive development [1] of object- oriented programs. In the test- drive development, the development process is advanced
  9. 9. in steps through unit tests. Mock objects play the role of imitating the functions of the objects which the object being tested collaborates with. Objects which the object being tested are actually related to are replaced with mock objects, So the objects being tested do not depend on other related objects. As a result, the test programs are structured highly, and the test related programs preserve the encapsulation and maintainability is improved. However, mock objects regarding the objects being tested are arranged for input of each test class, and have close connection with a test class. Therefore, by refactoring etc. if mock objects are changed then test class must also be updated. By using the mech- anism of traversal of the graph of an object dynamically, TraversalAssert eliminate the dependency on mock objects and improves the robustness of the test class. 5.2. Design by Contract The method of guaranteeing the quality of an object-oriented program there is Design by Contract (DbC) [7] proposed by Meyer. In DbC, the general conditions, which a program should fulfill for every routine as a method, are described as assertion. By using the assertion although it is possible to develop the software which are reliable by pinpointing defective parts, but it is not easy to describe the general conditions to fulfill. In TraversalAssert of this work, there are not general conditions for main part of the program and the separated test class, but more specific conditions are described as assertion. The actual expected values regarding the execution result of a certain input, by comparing in the form of the actual execution result being described as conditions of the test, the description of assertion is very clear. Moreover, by the module specifications describing the assertion of DbC as per (Java Modeling Language) JML, there are arrangements to generate the test classes automati- cally [2]. If the test classes can be generated automatically, it becomes unnecessary for a test programmer to describe a lot of test classes, and reduce the labors of update of the test classes by changes in a test. But the general specifications using JML becomes too much descriptions, and become weak to changes of modules for the test and be- come necessary to update the description of the specifications. Moreover knowledge of specifications description is also required. In the TraversalAssert library, regarding the executions results of modules for unit testing and the input data to these modules only the actual expected values are described. For that reason, the description become brief and different from the specifications by JML, and become strong to the changes of the modules for the test. 6. Conclusion In this paper, in the software development by OOP language, the dependency between unnecessary objects of unit test, the problems of loosening the strictness of test classes are shown. As a solution of the problem, by using the mechanism Demeter for traversal, as- sertion mechanism TraversalAssert was developed to elminate the dependency between unnecessary objects. TraversalAssert by using DJ processor of Demeter implemented the easy use of traversal in unit tests.
  10. 10. In order to verify the validity of developed TraversalAssert, a verification experiment is conducted for a number of software, for which the test classes are announced as open source, and improvement in robustness of test classes was checked. In future, for reducing the time required for automatic traversal by TraversalAssert, the optimization of traversal strategy, improvement of robustness, specifying the execu- tion results for a test, and the side effects of the changes to the state of the objects by traversing stack, etc. are the subjects. References [1] Beck, K.: Test Driven Development: By Example, Addison-Wesley Professional; 1st edition, November 8, 2002. [2] Cheon, Y. and Leavens, G. T.: A Simple and Practical Approach to Unit Testing:The JML and JUnit Way, the European Conference on Object-Oriented Programming (ECOOP), Iowa 50011-1040, USA, Springer Verlag, June 10-14, 2002. [3] Chidamber, S. R. and Kemerer, C. F.: A Metrics Suite for Object Oriented Design, IEEE Traansactions on Software Engineering, VOL. 20, NO.6 , June 1994. [4] Lieberherr, K. J.: Adaptive Object-Oriented Software: The Demeter Method with Propagation Patterns, PWS Publishing Company, Boston, 1996. ISBN 0-534-94602-X. [5] Lieberherr, K. J., Holland, I., and Riel, A. J.: Object-oriented programming: An objective sense of style, No. 11, San Diego, CA, September 1988, pp. 323–334. A short version of this paper appears in IEEE Computer Magazine, June 1988, Open Channel section, pages 78-79. [6] Louridas., P.: Junit: Unit testing and coding in tandem, IEEE Software, 22(4):12 - 15, July-Aug 2005. [7] Meyer, B.: Applying Design by Contract, IEEE Computer, 25(10):40-51, October 1992. [8] Orleans, D. and Lieberherr, K.: DJ: Dynamic Adaptive Programming in Java, Reflection 2001: Meta-level Architectures and Separation of Crosscutting Concerns , Kyoto, Japan, Springer Verlag, September 2001. 8 pages. [9] Steve Freeman, Tim Mackinnon, N. P. J. W.: Mock Roles, Not Objects, Proceedings of the ACM SIGPLAN 2004 Conference on Object-Oriented Programming, Systems, Languages, and Applications, Berkshire House, 168-173 High Holborn, ACM Press, OOPSLA2004.