Patterns: from system design to software testing

573 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
573
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Patterns: from system design to software testing

  1. 1. Innovations Syst Softw Eng (2008) 4:71–85 DOI 10.1007/s11334-007-0042-z ORIGINAL PAPER Patterns: from system design to software testing Neelam Soundarajan · Jason O. Hallstrom · Guoqiang Shu · Adem Delibas Received: 16 December 2007 / Accepted: 19 December 2007 / Published online: 12 January 2008 © Springer-Verlag London Limited 2008 Abstract Design patterns are used extensively in the design of the particular system by specifying a set of specializa- of software systems. Patterns codify effective solutions for tion rules that are designed to reflect the scenarios in which recurring design problems and allow software engineers to the defects codified in this set of PTCTs are likely to man- reuse these solutions, tailoring them appropriately to their ifest themselves in the particular system. We illustrate the particular applications, rather than reinventing them from approach using the Observer pattern. scratch. In this paper, we consider the following question: How can system designers and implementers test whether Keywords Design patterns · Contracts · Testing their systems, as implemented, are faithful to the require- ments of the patterns used in their design? A key considera- 1 Introduction tion underlying our work is that the testing approach should enable us, in testing whether a particular pattern P has been Design patterns [4,11,24] have had a profound impact on correctly implemented in different systems designed using how software systems are built. This is not surprising since P, to reuse the common parts of this effort rather than hav- patterns capture the distilled wisdom of the software com- ing to do it from scratch for each system. Thus in the approach munity, and provide effective solutions to recurring design we present, corresponding to each pattern P, there is a set problems. They allow software engineers to reuse these solu- of pattern test case templates (PTCTs). A PTCT codifies a tions, tailoring them to the needs of their particular systems, reusable test case structure designed to identify defects asso- rather than reinventing them from scratch. Patterns are used ciated with applications of P in all systems designed using primarily during the design phase of software systems. In P. Next we present a process using which, given a system this paper, we focus on later stages of the software life-cycle, designed using P, the system tester can generate a test suite the testing and maintenance phases. Specifically, the ques- from the PTCTs for P that can be used to test the particular tion we are interested in is, how can system designers and system for bugs in the implementation of P in that system. implementers test whether their systems, as implemented, are This allows the tester to tailor the PTCTs for P to the needs faithful to the requirements of the underlying patterns? This question is especially important for large systems since such N. Soundarajan (B) · G. Shu · A. Delibas Computer Science and Engineering, Ohio State University, systems tend to have correspondingly large software teams Columbus, OH 43210, USA and hence a correspondingly greater likelihood that differ- e-mail: neelam@cse.ohio-state.edu ent members of the team may have subtly different inter- G. Shu pretations of the patterns underlying the system’s design. e-mail: shug@cse.ohio-state.edu Hence the team members may implement their respective A. Delibas parts of the system in ways such that when these parts are put e-mail: delibas@cse.ohio-state.edu together, the resulting overall system is not compatible with the patterns’ intent. Design-related bugs of this kind may not J. O. Hallstrom School of Computing, Clemson University, manifest themselves in test cases that are developed using Clemson, SC 29634, USA standard approaches such as testing against the functional e-mail: jasonoh@cs.clemson.edu requirements or even code-based criteria such as statement, 123
  2. 2. 72 N. Soundarajan et al. or branch, or path coverage [3]. The testing techniques and only create the subcontract that specifies how the pattern is supporting tools discussed in this paper will be of value in specialized for use in a particular application. reducing the likelihood of such bugs going undetected, and A similar consideration is a primary motivation for the will aid in their localization. approach we present in this paper. That is, a key goal of our The problem of ensuring that a system is faithful to the work is that the testing approach should enable us, in test- patterns underlying its design is even more serious during ing whether a particular pattern P has been correctly imple- the maintenance phase since system maintenance may be mented in different systems designed using P, to reuse the the responsibility of team members who were not involved common parts of this effort rather than having to implement it with the original design or implementation of the system— from scratch for each of these systems. Thus in the approach and consequently have only a superficial understanding of we develop, corresponding to each pattern P, there is a set its design. As a result, over time, the design integrity of the of what we will call pattern test case templates (PTCTs). A system is likely to erode. The approach we develop in this PTCT codifies a reusable test case structure designed to iden- paper can help address this aspect of the problem as well. tify defects associated with applications of P in all systems This is because the test suite used to test whether the system designed using P. Next we present a process using which, is faithful to its underlying patterns can be included as a given any system designed using P, the system tester can key part of the system’s design documentation in the same generate a test suite from the PTCTs for P that can be used manner as standard unit tests or other functionality tests. As to test the particular system for bugs in the implementation the system evolves during maintenance, the design test suite of P. The process allows the system tester to tailor the set will help system maintainers identify and localize changes of PTCTs corresponding to P to the needs of the particular that may conflict with the original design. If one of these tests system by specifying a set of specialization rules that are were to fail, that doesn’t necessarily mean that the changes designed to reflect the structure and the scenarios in which in question are flawed. It may be that the design needs to the defects codified in the set of PTCTs for P are likely to be modified; but this decision should be made consciously manifest themselves. with the design documentation being appropriately updated, An example will help illustrate our approach. Consider rather than simply modifying the code. We will return to this the classic Observer pattern which we will use as our pri- point in the final section of the paper. mary case study throughout the paper. According to the stan- In standard software testing, we test software against its dard description [4,11] of the pattern, participating objects specifications. These specifications must be formal since, play one of two roles: Subject or Observer. The intent of otherwise, there is no way to conclude whether a test was suc- the pattern is to keep multiple observers consistent with cessful or not1 . Similarly, to test whether a system is faithful the state of a single subject. When an object wishes to to the intent of the patterns underlying its design, we need become an observer of a subject, it invokes the subject’s formal specifications of the patterns. Attach() method. When it is no longer interested, it invokes In previous work [12,26,31], we presented an approach to the Detach() method. In addition, Subject includes a providing precise specifications for patterns and their appli- Notify() method which, according to the intended usage of cations in the form of pattern contracts and subcontracts. It the pattern, is required to be called whenever the state of the is against these contracts and subcontracts that we wish to subject is modified. Notify() is required to invoke Update() test individual systems to validate their design correctness. on each attached observer which must, in turn, update the An important aspect of this approach to pattern speci- state of that observer to make it consistent2 with the current fication is that the requirements and behavioral guarantees subject state. specified in a pattern contract apply to all uses of the pattern; The standard UML diagram for the pattern appears in a subcontract characterizes how the pattern is specialized in Fig. 1. The subject maintains a set of references to the a particular application. Thus the wisdom of the community currently attached observers in the variable _observers. captured in a design pattern is reflected in the corresponding According to the standard description, when Attach() is pattern contract; it can be reused by every team that uses the invoked, it adds a reference to the attaching observer to pattern in the design of its system. The individual team need _observers. But if this was all Attach() did, the intent of the pattern would be violated; this is because the newly attached observer’s state may not be consistent with the current state 1 When is a test successful? One definition in the testing literature [3] of the subject – it will remain inconsistent until the next says that a test is successful when it shows that the software does not meet its specification, since the purpose of the test is to find flaws in the software. Others [20] use the convention that a test succeeds, or that 2 What precisely terms such as “modified” and “consistent” mean is not the software “passes” the test if, during the test execution, the software clear from such informal descriptions. The formal contract for Observer behaves according to its specification. We will use this latter convention which we will see in the next section will resolve this and other ambi- in this paper. guities in the informal description. 123
  3. 3. Patterns: from system design to software testing 73 Subject Observer Paper organization. Section 2 summarizes our approach to _observers pattern contracts and their relation to PTCTs. Section 3 devel- +Attach(in Observer) +Update() ops the essential ideas of PTCTs and how they are defined 1 * +Notify() based on pattern contracts. A number of PTCTs for Observer +Detach(in Observer) are presented based on the contract for the pattern. Section 4 considers how test cases, for a particular system designed using a given pattern, may be generated from the PTCTs for for all o in _observers the pattern; a simple system, the simulation of a Hospital, is o.Update() used for illustration. Issues of coverage (and the associated metrics) in the context of pattern-centric testing are also dis- ConcreteSubject cussed. Section 5 briefly surveys elements of related work. ConcreteObserver −subjectState * Section 6 concludes with a summary of the approach, and its 1 −observerState relation to test-driven design (TDD). +Update() 2 Pattern contracts Fig. 1 Observer pattern The contract for a design pattern P consists of a set of role- notification cycle. To address this problem, Attach() must contracts, one corresponding to each role of P, and a portion invoke Update() on the newly attached observer. Failure to that corresponds to the pattern as a whole. The role-contract do this is a common bug that appears in systems designed for a role R lists the state components of R, and specifies, in using the Observer pattern. It is also a bug that can be missed standard pre-/post-condition format, requirements that must during standard (functional) testing since the inconsistency be satisfied by the various methods of R. In a system designed of the newly attached observer may last only for a short using P, a class C playing the role R will typically pro- time, depending on when exactly the next modification in vide other methods in addition to those “named” in R. If the subject state takes place, resulting in a notification cycle these “other” methods were to behave arbitrarily, then the which will remove the inconsistency. intent of the pattern would be violated even if the methods In our approach, given that this problem is common to sys- corresponding to the named methods behaved according to tems designed using the Observer pattern, one of the PTCTs their specifications in the role-contract. In order to elimi- for this pattern, as we will see, will correspond to this poten- nate this possibility, R’s role-contract will also include an tial bug. When testing a system designed using Observer, others specification that must be satisfied by all methods the system tester will specialize this PTCT appropriately to of C, except those corresponding to the ones named in the generate a set of test cases that will help test the particular role-contract. The portion of the contract that corresponds system for the presence of this bug. These tests will confirm to the pattern as a whole consists of an invariant over the that when a new observer in this system attaches to the sub- (role) states of the various objects enrolled, at runtime, in an ject, it is appropriately updated. In other words, these tests instance of the pattern. The invariant will be satisfied when- will help ensure that the system is faithful to (this particu- ever control is not inside any of the methods of any of the lar aspect of) the intent of the Observer pattern; and if the participating objects. system were to be modified during system evolution, the test One potential problem with formalizing a design pattern cases and the associated documentation will help the system is that its flexibility might be reduced or even eliminated. maintainers test for this bug and to identify any violations A number of features of our formalism help guard against of the intent of the underlying pattern that may be inserted this, the most important of these being the notion of an aux- during this phase. Thus, the approach will help preserve the iliary concept. An auxiliary concept is a relation over one design integrity of the system as it evolves. or more states of one or more objects interacting with each Although some approaches to formalizing patterns have other according to the pattern. Auxiliary concepts are used been proposed [8,10,19,23] (in addition to our work cited in the role-contracts and the pattern invariant, but their defin- above), we are not aware of any other work focused on ition are not part of the pattern contract. Instead, definitions the question of testing a system to see whether it correctly tailored to particular systems, are provided in the subcon- implements the patterns underlying its design. Given the tracts corresponding to systems. The subcontract for a widely recognized importance of design patterns and their particular system also includes a set of role-maps, one use in designing and implementing large software systems, corresponding to each class C playing a role R in the pattern, we believe that this is an important concern and believe that as applied in this system. The C-R-role-map specifies how the approach we present will prove valuable for this task. the state, i.e., variables, of C map to the state of R (listed in its 123
  4. 4. 74 N. Soundarajan et al. role-contract), which methods of C correspond to each of the named methods of R, etc. As noted in Sect. 1, the pattern contract applies to all systems designed using the pattern; the subcontract speci- fies how the pattern is specialized for use in a given system. Correspondingly, the pattern test case templates that we will develop in the next section and will apply to all systems designed using the pattern, will be based on the pattern’s con- tract. The test cases for a given system will be obtained by specializing the templates using the information in the sub- contract and accounting for additional application-dependent factors, as we will see later. Consider the Observer pattern. The intent of the pattern, as noted earlier, is to ensure that when the state of the subject is modified, the observers “attached” to the subject are appro- priately updated so that their states become consistent with the current state of the subject. “Modified” does not, how- ever, mean a change in any arbitrary bit or byte of the sub- ject state; rather, some modifications are important enough to require updates of the attached observers and others are not. At the same time, which modifications are important enough in this sense and which are not varies from system to system. Hence, we use an auxiliary concept, Modified(), a relation over two states of the subject, to distinguish “important” changes in the subject state from “unimportant” ones. Sim- ilarly, what it means for an observer state to be “consistent” with the subject state varies from system to system; and, Fig. 2 Observer pattern contract (partial) indeed, from one kind of observer that may be attached to the subject to another that may be attached to the same sub- interactions are then expressed in the form of appropriate ject. Hence we use a second auxiliary concept, Consistent(), conditions on the trace variables. As we will see in the next a relation over an observer state and a subject state, to rep- section, PTCTs will be designed to test whether the sequence resent this notion. The contract for Observer, a portion of of interactions that occur in a system designed using the pat- which appears in Fig. 2, is written using these concepts and tern satisfy these conditions. specifies the requirements that apply to all systems designed The contract for Observer starts by listing the auxiliary using this pattern. concepts. Next (lines 6–8), we have a constraint that must Many patterns are concerned with the sequences of inter- be satisfied by the definitions of these concepts in any sub- actions between various objects, i.e., the sequences of method contract of Observer; we will consider this after consider- calls invoked on the various objects involved in the pattern. ing the rest of the contract. Next (lines 10–11), we have Thus, as we noted in Sect. 1, the Notify() method of the the pattern invariant, the reason (or the “reward” [26]) for Subject role is required to call Update() on each of the using the pattern, that is guaranteed provided we meet all the observers currently attached to the subject. Such require- requirements specified in the pattern contract. players[] is ments are expressed in the pattern contract using the trace τ , the array of objects enrolled in the pattern instance in ques- a ghost variable [15] provided by the formalism. In effect, tion and is another ghost variable of the contract formalism. when a method m() starts execution, a corresponding trace In the case of an instance of Observer, the first player to τ , initialized to the empty sequence, is automatically cre- enroll3 in any instance of the pattern is the subject; hence ated. Each call that m() makes to a named method during its execution is recorded as an element of the sequence, and 3 The complete pattern contract will also include clauses that specify includes information about the object on which the method how an object enrolls to play a given role in a pattern instance, how was invoked, the name of the method, and information about a pattern instance is created, etc. We omit the details of these here; the associated arguments and return values. A number of see [12,26]. When testing a system designed using the pattern, it is, of course, necessary that the requirements specified in these clauses mathematical functions, some of which we will introduce of the pattern contract (specialized for the particular system) are satis- as needed, simplify trace manipulation, access to individual fied. Hence appropriate PTCTs must be defined, or these tests must be trace elements, etc. Contract requirements concerning object included as part of other PTCTs. 123
  5. 5. Patterns: from system design to software testing 75 (a reference to) this object will be in players[0]. The rest in a particular application, that s1, s2 are two states of the of the objects in players[] will be the objects that enroll to subject, and o1 is the state of an observer. Suppose that the play the Observer role (by invoking the Attach() method, definitions of Modified() and Consistent() for this application this being the action required for this purpose as specified are such that each of the following is true: in the (elided from Fig. 2) enrollment clause for this role). Consistent (s1,o1), ¬Modified (s1,s2), and Thus the invariant (lines 10–11) states that the state of the ¬Consistent (s2,o1) first object in players[], i.e., the state of the subject in any Suppose, at some point in the execution of this system, the given instance of this pattern, is Consistent() with the states subject is in the state s1 and the observer in the state o1. of each of the observers (currently) enrolled in this instance. Suppose, finally, that the subject state changes from s1 to Next we have (part of) the Subject role contract. First we s2. Then, according to the others specification in the Sub- specify the state of the subject as consisting of obs, used to ject role, the Notify() method will not be invoked and hence store references to the currently attached observers. Next we Update() will not be invoked on this observer and its state have the specification of Attach(), one of the named methods would remain as o1. At this point, the current state, o1, of the of this role. The pre-condition requires that the attaching observer will be inconsistent with the current state, s2, of observer not already be attached, i.e., not have a reference the subject, and the invariant that the pattern was intended to it in obs. The post-condition states4 that a reference to the to guarantee will be violated although the individual meth- object is added to obs; that the subject itself is not modified, ods in the individual roles, as actually implemented in the in the sense of the auxiliary concept Modified(), and that one particular application, satisfy their respective specifications. call to a named method of the pattern has been made during The problem is that we have conflicting notions of what a the execution of Attach(), this being to the Update() method sufficiently serious modification in the subject state is that invoked on the attaching observer. requires the observers to be updated on the one hand, versus The others specification (lines 20–23) states that any what it means for the subject state to be consistent with the other method of the class playing the Subject role should observer state, on the other. The constraint specified (lines make no change in obs, and should either not modify (again 6–8) in the contract ensures that this problem doesn’t arise. in the sense of Modified()) the subject state, or must invoke Thus while the designers of a particular system are free to the Notify() method (on the current subject). The (elided) tailor the definitions of the auxiliary concepts to their partic- specification of Notify() states that this method invokes ular needs, they must ensure that these definitions satisfy the Update() on each observer in obs. constraint specified in the contract. The Observer role contract states that the state of this How realistic is this problem? That is, how likely is it role consists of the variable sub (that holds a reference to that the software team responsible for an actual application the subject to which this observer is attached); that the designed using the Observer pattern would base their sys- Update() method does not change the value of sub, and tem on such mutually conflicting notions of Modified() and makes the state of the observer to be consistent (in the sense Consistent()? It depends on the size of the team in question; of the auxiliary concept Consistent()) with that of the sub- as the team grows larger, the likelihood of such problems ject. The others specification also states that it not modify seem to also grow. But even for relatively small systems, the sub, and leave the observer in a state that is consistent with problem can creep in during system evolution; we will see that of the subject5 . an example of this later in the paper. Let us now consider the constraint specified in the con- Unfortunately, however, specific PTCTs (or, more pre- tract (lines 6–8). Although the definitions of the auxiliary cisely, test cases obtained by specializing them for a partic- concepts, tailored to the needs of a particular application, will ular application) cannot test these constraints because they be provided by the corresponding subcontract, these defini- are general conditions typically involving universal quanti- tions cannot be completely arbitrary. For example, suppose, fiers over the states of the various objects enrolled in the pattern instance. Instead, violations of these constraints may 4 We use the “#” notation in the post-condition to refer to the pre- show up as violations of, for example, the pattern invariant— condition value of the variable, i.e., its value when the method started although the individual methods invoked as part of the test execution. case may satisfy their own specifications. We will see this in 5 Standard informal descriptions [11,25] of the pattern seem to suggest the discussion of the example. that the other methods of this role should not make any changes in the state of the observer. But this is unnecessarily restrictive. As the pattern contract states, all we need is that any changes in the observer 3 Pattern test case templates (PTCTs) be such that they leave the observer state consistent with the sub- ject state. This is an example [12,26] of how our pattern contracts can identify dimensions of flexibility that may be missing in the standard Given the requirements captured in a pattern contract, how informal descriptions. do we develop appropriate PTCTs that can be used to test 123
  6. 6. 76 N. Soundarajan et al. not the contract requirements are satisfied in those common situations. These templates must then be specialized to obtain the actual test cases corresponding to a particular system. The system tester does this specialization, as we will see in the next section, with help from JDUnit6 , a testing framework/- tool that we are in the process of building, by combining the information in the contract, in the subcontract and the PTCTs, and taking account of any relevant details about the particular system. The language for defining PTCTs is intended to mirror standard Java programming syntax to support rapid practi- tioner adoption. Before looking at specific PTCTs, we sum- marize the syntax of the structure of a PTCT. A PTCT is represented as a segment of stylized Java code with inter- spersed asserts that, based on the pattern contract that the PTCT corresponds to, are expected to be satisfied at those points. The asserts will be based on the pre- and post- conditions of the named methods as well as the other spec- ifications in the various role contracts of the pattern contract. In addition, some of the asserts may also be based on Fig. 3 Pattern-centric testing the pattern invariant. Since the PTCT is intended to apply to all applications of the corresponding pattern, the construc- tor and method calls used in its definition must conform to whether systems implemented using the pattern satisfy these the signatures specified in the relevant role contracts. When requirements? It is important to stress that, given a pattern the actual test cases corresponding to a given application are contract, construction of the corresponding PTCTs is not a generated from the PTCT, information in the corresponding mechanical activity. The purpose of the PTCTs is to help test subcontract about the mapping of named methods of the for common mistakes in implementing the particular pattern. roles to particular methods of the corresponding classes and Clearly, which mistakes are common and which are not is a the mappings of the role-method signatures to the class-level question that can be answered only on the basis of experience method signatures will be used to translate the statements in with systems designed using the pattern. In this section, we the PTCT to appropriate code in the test cases. will present three PTCTs, the first two corresponding to the A PTCT begins with an initialization portion that creates Observer pattern and the last one corresponding to the Com- instances of one or more role types. named methods may posite pattern. These are based on our assessment, based on be invoked to bring the new player objects into states appro- our own experience with implementing systems and based priate for the test case family being developed. The PTCT on the reports in the literature, of the most common mis- developer may optionally include calls to other methods, takes in implementing this pattern. PTCTs corresponding to which serve as placeholders for class methods, which may other patterns will similarly be based on an assessment of be introduced at the point of test case generation. Since the commonly reported mistakes made in implementing them. precise number of other method calls required to bring an But these are, of course, not cast in stone. Indeed, with addi- object into the state appropriate for testing a particular system tional information about such mistakes, these PTCTs will be varies, the PTCT syntax allows developers to specify that a refined and additional PTCTs introduced. The main contribu- single other method may be instantiated by multiple class tion of this paper, then, is the approach presented for testing methods, supplied at the point of test case generation. The systems to ensure the correct implementation of their under- initialization section typically concludes with a PTCT pre- lying patterns—not the particular PTCTs presented. We will condition. This assertion, expressed as a standard boolean return to this point in the final section. expression, captures properties assumed by the PTCT body. Figure 3 schematically illustrates our approach to test- The check ensures that the class-level constructor and method ing systems against the patterns underlying their designs. The pattern contract formally captures the requirements of the design pattern, the subcontract specifies how the pat- 6 The name of the framework, currently under construction, is intended tern is specialized in the particular system. The PTCTs are to draw a parallel to JUnit, the popular unit testing framework for Java; designed based on the most common mistakes that are made the additional D emphasizes that the framework is used to test units of in applying the pattern and are designed to check whether or design. 123
  7. 7. Patterns: from system design to software testing 77 calls introduced at the point of PTCT instantiation satisfy this pre-condition. Next, a PTCT defines a body. The body portion gener- ally follows a standard testing idiom: First, local variables are used to store the pre-conditional states of the players participating in the PTCT. Next, invocations are placed to the methods of interest, i.e., those relevant to the design pattern requirements being tested. Finally, the PTCT con- cludes with one or more assertions used to check that the relevant pattern requirements are satisfied. These assertions, as noted above, are based on the method and invariant spec- ifications in the pattern contract and generally involve the post-conditional states of the participating players, as well Fig. 4 PTCT for subject.attach() as the pre-conditional states saved at the start of the body. There is, however, one important deviation from the stan- that JDUnit provides appropriate getter methods for each dard testing idiom: Since key parts of the pattern contract role field, including private fields.) Next, we add the attach- involve the trace τ of method calls, the assertions contained ing observer to pre_obs (line 4) to simplify the expression in a PTCT will correspondingly include conditions on a spe- of the assertion check corresponding to the post-condition of cial variable tau, used to record the trace of invocations orig- Attach() (lines 9–10). We additionally store the current state inating from the PTCT. Note that the PTCT is not allowed of s (line 5), since this is required to check that Attach() not to contain any code that directly updates this variable; the modify s according to the applicable definition of Modified(). management of tau and updating it to reflect the method A couple of points should be noted here. The getter meth- invocations as they happen is the responsibility of JDUnit. ods are more involved than simply returning the current value The one exception to this rule is that a PTCT may clear tau of the field in question. In an actual system designed using the when the earlier invocations recorded in it are no longer of pattern, the class C playing a particular role R may provide interest. In addition, JDUnit provides a number of helper a different set of fields from those specified in R’s role con- functions that we may use in a PTCT; these functions allow tract. As we will see in the next section, the subcontract for us to obtain information about the current contents of tau. the system will specify the mapping from C’s fields to those We will see several of these in the PTCTs that we discuss of R. The getter methods provided by JDUnit will make use next. of this mapping to translate the values of C’s fields to the There are some important differences in detail between corresponding values of R’s fields. Similarly, the clone() the trace τ used in the specifications in the pattern contract operation applied to s (line 5) will create an object of what- and the trace tau maintained by JDUnit. The primary reason ever class plays the Subject role in the particular application. for these differences is to enable us to construct more useful Finally, JDUnit will use the auxiliary concept definitions PTCTs. In particular, suppose we have a PTCT that includes provided in the subcontract to evaluate assertions involving a call to a method m1() followed by a call to method m2(). concept references, such as the one involving Modified() Thus the invocation of m1() completes before the invocation (line 9). We will return to these points in the next section. of m2() begins; i.e., the execution of neither is nested inside Let us now consider contract requirements involving the other. Suppose the trace of method calls that takes place traces. As we saw, in the contract formalism, each method during the execution of m1() is t1 and that during m2() is m() has an associated trace τ , initialized to the empty t2. The trace tau maintained by JDUnit allows us to write an sequence when the method starts execution. Information assert following the completion of m2() that, for example, about calls made by m() to named methods are recorded in imposes conditions on t2 based on the value of t1 or vice- τ , with m()’s post-condition imposing necessary conditions versa. We will see these details shortly. on τ . In those cases where a PTCT includes calls to methods The main portion of a PTCT intended to test the behav- with trace requirements, the PTCT will include assert state- ior of the Attach() operation of the Subject role of the ments that check relevant conditions on the associated trace Observer pattern appears in Fig. 4. As discussed earlier, a variables. Indeed, a PTCT may include assert statements common problem in using this pattern is not updating an that require particular relations to hold across multiple trace observer when it initially attaches to a subject. In the PTCT, variables, each associated with methods preceding the asser- we begin by creating a subject s (line 1), and an observer o tion check. To enable easy expression of such asserts, we use (line 2). Next, we use a “getter” method provided by JDUnit, the following approach. When the instantiated PTCT begins to retrieve the pre-conditional value of s.obs before the call to execution, the testing framework creates a trace object tau, Attach() (line 7), and to store this value in pre_obs. Note initialized to the empty sequence. Each named method 123
  8. 8. 78 N. Soundarajan et al. invoked during the execution of the PTCT is recorded as an element of tau, and includes information about the tar- get object, the identity of the method invoked, etc. In addi- tion, and this is the key difference from the use of τ in the contract formalism, each element of tau itself includes the trace of methods executed during the associated call’s exe- cution. Thus, while there is only a single tau object, it main- tains a branching structure corresponding to the computation tree rooted at the PTCT. The traces contained in the individ- Fig. 5 PTCT for subject.other() ual elements of τ may be extracted using a simple accessor (“tr()”) function provide by JDUnit. Immediately prior to the call to Attach(), we clear tau sions will be based primarily on the common mistakes that (line 6). This removes the trace entries associated with the practitioners make in using the given pattern. preceding constructor calls (lines 1–2), as well as any calls to Note also the use of appropriate temporary objects to named methods introduced during the specialization process capture pre-conditional values referenced in various post- in going from the PTCT to actual test cases; in the next conditions; this is common in specification-based testing section we will see how the specialization may introduce given that post-conditions refer to both the state at the end of such calls. Hence, when control returns following the call to the method and at its start. The pattern invariant (lines 6–7, Attach(), tau contains a single entry, corresponding to Fig. 2) could also have been checked as part of the assert this call. The trace of method calls that occurred during the statement. This would amount to checking that Attach() not execution of this Attach() is extracted and stored in t1 only invokes Update() on the attaching observer, but also (line 8). The assert that follows imposes appropriate condi- that the latter method appropriately updates the observer’s tions on this trace, based on the requirements specified in the state to be Consistent() with the subject. pattern contract. The first two clauses require that s._obs be Next consider the PTCT shown in Fig. 5, intended to test appropriately updated (line 9), and that s not be Modified() the behavior of Subject’s other methods. We begin by cre- (line 10). The last three clauses (lines 11–13) address trace ating a subject (s) and two observers (o1, o2) (lines 1–3), conditions imposed on Attach(). Together, these clauses and attach both observers to the subject (line 4). We then require that Attach() invoke exactly one named method, save the pre-conditional value of s.obs (line 5) and the value and that this call be to the Update() method of the attach- of s as a whole (line 6). tau is then cleared, and some other ing observer7 . method is invoked on s (line 7). Finally, the trace associ- The assert statement is, of course, based on the specifi- ated with the other call is saved (line 8), and the behavior cation of Attach() included in the Subject role contract of the method is checked against the requirements specified shown in Fig. 2. The effort involved in writing the PTCTs in the pattern contract (lines 9–10). The assert statement corresponding to a pattern is primarily in identifying the requires that s.obs be unchanged, and that both o1 and o2 common mistakes that are made in applying the pattern and be Consistent() with s. coming up with appropriate test case templates based on this. In generating test cases for a particular system from this Once that has been done, the appropriate asserts to include PTCT, the s.other() call will have to be replaced by calls to are determined by the pattern contract. But even here, the appropriate methods of the class playing the Subject role. PTCT designer may choose not to include all of the clauses Some of these methods may require additional arguments. In specified in the role contract as the post-condition of the the next section, we will consider the problem of generating particular method. Even more importantly, at what points suitable values for these arguments. At this point, however, to include checks of the pattern invariant and assertions that the more important issue is the mismatch between the assert require particular relations between the traces associated with statement and the requirements specified in the Observer con- two or more method calls appearing earlier in the PTCT, are tract. According to the contract (Fig. 2), when s.other() all decisions the PTCT designer must make. And these deci- terminates, either the state of s must not be Modified(), or the Notify() method must have been invoked. As we have 7 We obtain the identify of the method invoked and the target object already seen, this method will in turn invoke Update() on by using the helper functions mt() and ob() provided by the JDUnit each attached observer, bringing the objects into states con- framework. These functions may be applied to individual elements of a sistent with the new state of the subject. Hence, the assert trace in which case they return the identify of the method/target object included in the PTCT is a test of the expected net behavior involved in that element. Or they may be applied to a trace, in which case of the participating objects, i.e., it is a more “global” test of they return the sequence of method identities/target objects involved in the various elements of the trace. We such usage in the pattern contract the system. There is, however, a risk in using such a PTCT, in the last section. especially if these were the only ones used. If, for instance, 123
  9. 9. Patterns: from system design to software testing 79 an other method were to modify the state of the subject, neglect a call to Notify(), but by chance leave s in a state Consistent() with the states of o1 and o2, the design defect would go undetected. In [27], we present a scenario in which this problem mani- fests itself during system evolution; we summarize the exam- ple here. A class S1 plays the Subject role, and provides two fields, f1 and f2. A change in either field is considered to be a modification of the subject according to the definition of Modified() supplied in the system’s subcontract. A class O1 plays the Observer role. In an initial version of the system, O1 is interested only in the value of f1; changes in f2 are Fig. 6 Composite role contract (partial) ignored. The Update() method of O1 uses an appropriate getter method to retrieve the value of f1, and then updates Patterns such as Composite, although classified as structural, the observer’s state to become Consistent() with the state of also have behavioral aspects to them. Indeed, the main pur- the subject. The Consistent() concept is defined suitably in pose of the Composite pattern is to ensure that each operation the subcontract. The S1 class includes a bug that omits a call of a composite object is implemented by invoking the cor- to Notify() when a change in f2 occurs. A PTCT similar to responding operation on each of its components and com- the one above will not detect this defect since each observer bining the results returned by these invocations appropriately will remain Consistent() with the subject when the other to obtain the result of the original operation invocation. method terminates — even if f2 has changed without a cor- A small portion of the Composite pattern contract, in par- responding call to Notify(). ticular a part of the Composite role contract, appears in During system evolution, O1 is modified so that the value Fig. 6. The specification of any operation() of this role of f2 becomes significant. Instances of O1 record the cur- states that when this method finishes, the set of children rent value of f2 associated with their corresponding subject. is unchanged; that during the execution of this method, there Hence, the definition of Consistent() is suitably modified, as should have been a call to the corresponding method on each is the implementation of O1’s Update() method. The new child c; and that the result returned by this method should be implementation retrieves the values of both f1 and f2, and equal to the value obtained by appropriately combining the updates the state of the observer appropriately. When the test results of these method calls invoked on the children. Since case derived from the PTCT shown in Fig. 5 is executed, the the details of precisely how these results are combined to assert statement will generate an error; the clause involv- obtain the result of the Operation() applied to the com- ing the Consistent() concept will be violated. The natural posite will vary from one application designed using this assumption is that the fault lies in an area affected by sys- pattern to another, we use an auxiliary concept, Glue() to tem evolution. In fact the defect lies in the original S1 class represent this. Thus the final clause of the post-condition — in particular, the failure of its other() method to invoke of Operation() requires that the result returned by Notify() when the value of f2 changes. the original call is equal to the value of Glue() applied to The solution is to revise the PTCT to fully test the interac- the sequence of results in the method calls recorded in τ , tion requirements specified in the pattern contract, rather than this being obtained by means of the helper function, testing for net effects. More precisely, the last two clauses of resultSeq(). the assert in Fig. 5 (line 10) should be replaced with the Figure 7 presents a PTCT to test that the requirements following conditions: specified in Fig. 6 are satisfied. We create two composites, c1 and c2, a leaf l, and create a suitable composite structure. We 8 a s s e r t ( … clauses from Fig. 5 … 9 && (! M o d i f i e d ( pre_s , s ) || then invoke operation() on the top level composite struc- 10 (( t1 . l e n g t h ( ) = = 1 ) & & ( t1 . m ( ) = = " N o t i f y ")) ture and, assert, after the call returns that the trace created during the execution of this method is equal to the number of Before concluding this section, we will consider another children of this structure, and that the result returned matches PTCT, this one for the Composite pattern. Composite is a that obtained by applying the auxiliary concept Glue() on structural pattern [11] intended to compose objects into a the results returned by the calls recorded in that trace. The tree structure to represent part-whole hierarchies and to allow definition of Glue() will, of course, be given in the sub- clients to treat individual objects and composite objects in a contract corresponding to a particular system, for which we uniform manner. The pattern has three roles, Component, will specialize this PTCT to produce actual test cases. It may Leaf, and Composite. Component is an abstract role with be worth noting that this PTCT does not completely test the Leaf and Composite being the two kinds of components. specification in Fig. 6; in particular, while it does test that the 123
  10. 10. 80 N. Soundarajan et al. Fig. 7 PTCT for composite.operation()) number of method calls recorded in the trace t1 is what it should be according to the specification, it does not test which methods were invoked in these calls, nor on what objects. It is straightforward to modify the PTCT to include these checks and we omit the details. 4 Generating test cases Given PTCTs such as those in the last section, how do we generate actual test cases corresponding to a given system S designed using a particular pattern? Consider, for example, line 7 of Fig. 4. The method Attach() that is being invoked Fig. 8 Hospital simulation code in this line may have an entirely different name in S. Indeed, even the classes playing the Subject and Observer roles will have their own names appropriate for the application. The role map for Patient as Subject specifies how the To illustrate these and other issues involved in generating _obs of this patient viewed as as subject may be obtained actual test cases, we will use a simple Hospital Simulation from the state of the patient. Next, the method maps specify system, parts of which appear in Fig. 8, designed using the that addNurse() and setDoctor play the role of Attach() Observer pattern. and that notify() plays the role of Notify(). The role map The Hospital system consists of three main classes, for Nurse as Observer illustrates a technical problem: since Patient, Nurse and Doctor. Instances of the Patient class a nurse may be observing multiple patients, each of which play the Subject role; instances of the other two play the is the subject in a distinct instance of Observer, how do Observer role. Zero or one doctor object and zero or more we identify the particular patient involved with the current nurse objects are assigned to observe patient. The variables pattern instance? The contract formalism [12,26] provides _nurses and _doctor in the Patient class are used to keep the notion of lead object to address this. The lead object in track of the assigned doctor/nurses. A nurse object tracks a pattern instance is the first object that enrolls in a pattern the temperature (_temp) of the patient it is observing; a instance, i.e., the one that is in players[0]. The method map doctor object tracks the heart-rate (_hrtRt). The checkVi- for Nurse maps its update() (applied to the lead object) to tals() method updates _temp and _hrtRt as needed, and Update() of the Observer role. The role map for Doctor invokes notify() if needed. The Nurse and Doctor classes as Observer is analogous. include an update() method to help keep track of the data Next we have the definitions of the auxiliary concepts. about the patients being observed. They also provide a get- According to the definition of Modified() (lines 25–27), the Status() method that returns information about the patients. state of a patient is considered to be modified if either _temp Before considering how test cases corresponding to the value or the _hrtRt value has changed. The definition in lines Hospital system may be generated from the PTCTs of the 28–29 state that a nurse n’s state is consistent with that of the last section, we have to consider the subcontract that specifies patient p if n has the correct information about p._temp. how the Observer pattern is specialized in this system; the The definition (lines 30–31) of consistency of a doctor d main part of this subcontract appears in Fig. 9. and patient p is similar. 123
  11. 11. Patterns: from system design to software testing 81 suitably specialize the PTCT by defining appropriate test case specializers. A key purpose of a test case specializer (TCS) is to limit the set of generated test cases, i.e., to specialize the PTCT in such a manner that the test cases that are generated are the ones most appropriate to the system under test. There are a number of dimensions in which this specialization has to be performed. First, the objects constructed in setting up the scenario for a PTCT are role objects. If R is a role of this pattern and, in the PTCT, we construct an object ob of type R, and if, according to the subcontract for the system in question the class C1 and C2 play the role R, then in an actual test case, ob may be either a C1 object or a C2 object; the TCS has to specify which it is. Second, calls in the PTCT to named role methods (such as Attach()) have to be replaced by calls to the corresponding class methods, as specified in the appropriate role maps of the subcontract. This process, by itself, does not require any additional information from the TCS – unless more than one class method plays the role method in question or, as is more frequently the case, the class method in question requires additional arguments beyond the ones expected by the role method. These values will, of course, have to be provided by the TCS. There are two important considerations to account for here. First, the application class may specify, as part of Fig. 9 Hospital-observer subcontract the pre-condition of the corresponding class method, some conditions that must be satisfied by these argument values. If some set of argument values generated according to the spe- Let us now turn to the main task of this section, generat- cialization in the TCS do not satisfy these conditions, that ing, from the PTCTs for a pattern, test cases for a particu- test case cannot be used. For example, in the Hospital sys- lar system designed using the pattern. For any such PTCT, tem, we are not allowed to attach two doctor objects at the a (large) number of test cases may be generated. Each test same time to a given patient object. In other words, an case would be obtained by replacing each role object in the attempt to call addDoctor() on a patient object that PTCT, such as s, o1, and o2 in the case of the PTCT in already has a doctor will violate the pre-condition of this Fig. 5, by corresponding application objects, as determined method and hence the method may behave in whatever way by the subcontract, such as a patient, a nurse, and a doc- it chooses and this is not a bug in the system, despite the fact tor, respectively, in the case of the Hospital system; replacing that, because of such behavior, an assert in the test case the calls to role methods in the PTCT, such as s.Attach(o1) may fail. At the same time, we cannot always eliminate this and s.Attach(o2), by calls to the appropriate application type of problem when the test case is generated because some methods, again as determined by the subcontract, such as of these assertions may be checkable only during execution s.addNurse(o1) and s.setDoctor(o2), respectively; etc. of the test case. In order to address this type of problem, While that much is straightforward, there are some impor- JDUnit inserts additional asserts in the test cases it gen- tant additional issues to consider. For example, it may be that erates immediately prior to calls to any methods. Each of the constructor functions (or other methods, for that matter) these asserts checks that the pre-condition of the method of some of the application classes require additional argu- being called is satisfied. If, when a test case is executed, ments. What values do we supply for these? The PTCT, of such an assertion is not satisfied, that is recorded in the log course, imposes no requirements on these. As far as the PTCT maintained by the system; but the assertion failure does not and the pattern are concerned, any values may be used for indicate a failure of the test case8 . The second issue that the these additional arguments. This means that the number of possible test cases we can generate will be very large. Which 8 This is a distinction from the JUnit framework where every assertion of these test cases will actually be useful will depend on violation is reported as a failure. In order to allow a different treat- the system, in particular its implementation details. Hence ment for pre-condition violations, in our design of JDUnit, we use an the human tester using our approach will be required to additional method, preAssert, to specify such assertions. 123
  12. 12. 82 N. Soundarajan et al. tester designing the TCS has to be concerned about in decid- 8 A t t a c h ( o ) { @7 } = ing on the argument values is producing a large enough set of 9 { D o c t o r ( o ): s e t D o c t o r ( o , /* addnl args */ ) test cases with different values for these arguments to ensure 10 || N u r s e ( o ): a d d N u r s e ( o , /* addnl args */ )}; adequate coverage of the system under test. Strictly speaking, it should not be necessary for the tester to Next, each other() call that appears in the PTCT must provide this information since it is available from the sub- be converted to a call to one of the methods of the appro- contract (Fig. 9). But in those cases where multiple methods priate application class, other than the ones mapped to the of the same class may play the role of a named method, the various named role methods. In addition, as in the case con- TCS will have to specify which of those must be used. This sidered above, if the method in question requires additional is, of course, very much needed in dealing with other() calls arguments, the TCS must provide these as well. And, as in that appear in the PTCT since, in general, there will be many the case considered above, we must account for any pre- class methods that qualify as other methods. In the case of conditions that may be imposed by the application class in other*() calls, we must provide the sequence of method question. calls that should replace it and the arguments for each call The final dimension of specialization of the PTCT that the in the sequence. For example, suppose the other() call in TCS must account for is concerned with a construct that may line 5 of the PTCT in Fig. 5 was an other*() call. Then we appear in a PTCT but, as it turns out, was not not used in may have the following: any of the PTCTs in the last section. This is the “other*();” construct, which denotes a sequence of zero or more calls 8 o t h e r *() { @7 } = to other methods. This construct may appear in a PTCT 9 { c h e c k V i t a l s (); c h e c k V i t a l s ();} at points where the PTCT designer wants to allow for the Here the other*() call is replaced by a sequence of two calls possibility that a number of additional calls to other methods to checkVitals(). We can also use the “||” notation to allow may be needed in order to get the various objects into suitable for alternative sequences of calls. states for continuing the test. The TCS will have to specify The detailed syntax for TCS is very much a work in the sequence of method calls that must be made in place of progress. For example, in the situation we just considered, the other*() construct. if the tester wanted to test a range of different sequences Let us now see how these considerations play out when of other calls, listing them all explicitly would be unde- applied to some of the PTCTs (for the Observer pattern) when sirable. It would be more convenient if we could use regu- we specialize them to test the Hospital system. Consider the lar expression-like notations for such situations. We intend PTCT in Fig. 4. Line 2 of this PTCT requires an observer to apply our approach to a number of patterns and systems object to be created. Suppose the TCS designer decides that designed using the patterns to determine the exact notational this object should be a doctor. In the notation for TCS that facilities that TCS must include. In addition, practitioner- we are designing concomitantly with the design of JDUnit, friendliness will be a key consideration since it is the testers this may be expressed as: who are part of individual software teams who will be respon- 8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ ); sible for defining the TCSs for their particular systems. On the other hand, given that both Doctor and Nurse play Before we conclude this section, one final point about the the Observer role in this application, the TCS designer may Hospital system is worth briefly discussing. Suppose, dur- want to create two test cases, one corresponding to each of ing system evolution, it is decided that the nurse objects these possibilities. This is expressed in our notation as: observing a given patient object should keep track also of 8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ ) the medicine-level, i.e., the value of the _medLvl variable 9 || N u r s e ( /* args */ ); of the Patient class. It would be easy enough to modify the Nurse class to do so, perhaps by introducing a new variable, Let us consider another statement that appears in the same _medLevels, similar to the Nurse._vitals; and to revise PTCT, the call to Attach() in line 7 of the PTCT in Fig. 4. Nurse.update() so that the value of _medLevels as well The following specializes that call so that it is replaced by a as _vitals is appropriately updated. call to the setDoctor() method: While this seems simple and reasonable, if this is all we 8 A t t a c h ( o ) { @7 } = do, there will be a bug in this system! The problem is that, as 9 { s e t D o c t o r (o , /* a d d i t i o n a l args */ )} it is implemented, the Patient class methods do not invoke If, as we considered above, the observer that was con- notify() if only the _medLvl value changes. So in those sit- structed earlier (line 2 of the PTCT) could be either a doctor uations where only this variable changes, the attached nurse or a nurse, our notation for TCS allows the tester to spec- objects will have incorrect information about medLvl (until ify which method must be used when replacing the call to the next time the patient’s _temp or _hrtRt changes, trig- Attach() in line 7: gering a call to notify()). 123
  13. 13. Patterns: from system design to software testing 83 Unfortunately, however, the test cases generated using the can the work described here be new? The standard meaning of PTCTs we have seen will not catch this error. The problem testing pattern refers to a pattern used to address a recurring is that, according to the definition of the auxiliary concepts problem in the context of software testing. Indeed, a test(ing) in Fig. 6, if only the value of _medLvl changes, the state pattern is simply a special type of design pattern [17], one of the patient is not considered to be modified; moreover, intended to guide the design of testing code. Our work, how- the state of nurse is considered to be consistent with that ever, is focused on validating the correct usage of design of the patient as long as the former has the correct infor- patterns through software testing. We note that other authors mation about the value of the _temp of the latter. Thus any have also considered applying design patterns as part of the test cases generated from the PTCT in Fig. 5 will succeed system design process to improve testability [1,6]. Again, when executed. If we were to change the definition of one however, our focus has been on ensuring the correct applica- of these concepts but not the other, the constraint specified tion of patterns. in the original pattern contract (lines 6–8 of Fig. 2) will be It is important to note that McGregor [17,18] extends the violated. Since such a problem could easily arise during sys- traditional notion of a test(ing) pattern to patterns intended tem evolution, it is important to ensure that our PTCTs check for use in validating design pattern implementations. The that constraints on the auxiliary concepts are satisfied. At the concept is similar to our use of PTCTs in that each test same time, the fact that these constraints are typically uni- pattern specifies a particular scenario of application objects versally quantified over the states of various roles makes this that interact in ways intended to check pattern requirements. a challenge. This is one of the questions we intend to address However, the scenarios are not based on precise pattern spec- in future work. ifications. Equally important, McGregor does not consider test suite specialization, nor automated generation. Still, the similarities are important. 5 Related work Finally, it may be useful to mention the SABER system described by Reimer et al. [22]. The system is designed to The PTCT testing approach builds upon our prior work in detect defects in large Java applications using static analy- pattern formalization [12,26,31]. The key conceptual ele- sis; SABER operates on systems designed using particular ments of this work were summarized in Section 2. Several frameworks. Correct use of these frameworks requires that other groups have also considered improving the precision of certain methods be invoked in certain orders, that others not pattern descriptions. Some have described techniques based be invoked at certain points, etc. SABER attempts to iden- on diagrammatic notations that extend standard UML(-like) tify violations of such requirements based on call sequence syntax [7,21,32]. Closer to our specification approach is that rules specified using an XML dialect. Given the importance of Eden et al. [9,10]; the authors describe an approach to cap- of call sequence requirements in pattern-based systems, turing the structural properties of patterns using a logic nota- elements of SABER may be useful in detecting pattern tion. Patterns are expressed as logic formulae that specify violations statically, prior to pattern testing. We intend to participating classes, methods, inheritance hierarchies, and explore this possibility as part of future work. the structural relations among them. Mikkonen [19] describes a behavioral specification approach based on DisCo [14], a UNITY-like [5] action system notation. Data tuples represent 6 Conclusion objects, and guarded actions model their interactions. Helm et al. [13] describe a contract formalism that bears similarity We began with the observation that design patterns play to ours. In particular, the formalism has constructs analogous a central role in software practice, influencing the design to our role contracts, auxiliary concepts, and a limited form of of most major software systems and class libraries. Conse- trace conditions. The formalism additionally provides some quently, techniques for ensuring the correctness of pattern support for contract specialization and system-level reason- implementations are likely to have a profound impact on the ing. However, despite the effort invested in documenting pat- reliability of real-world systems. To this end, we focused terns precisely, there appears to be relatively little prior work on an approach to testing pattern implementations based in testing the correctness of systems constructed according on precise pattern specifications. Whereas our prior work to particular pattern descriptions, either formal or informal. provides the specification foundation, the work reported By contrast, this has been the focus of our work. here provides the foundation for specification-based testing Some readers may take note of the fact that the phrase of pattern implementations. “test(ing) patterns” appears throughout the literature. In developing the testing approach, we observed that pat- Indeed, there is at least one workshop series devoted to testing tern implementations follow a similar structure. This is not patterns [28–30], a corresponding online repository [16], and surprising. After all, for a designer to claim that a particu- widely referenced books that detail the subject [3,18]. So how lar group of classes have been designed according to a given 123
  14. 14. 84 N. Soundarajan et al. pattern, she must have satisfied certain requirements— during testing, use PCT to generate appropriate test cases. namely, those common to all applications of the pattern in Hence, PCT and TDD are fully compatible, but orthogonal question. It then follows that the test cases used to vali- testing principles. date the pattern’s implementation share a common structure across systems designed using the pattern. The central moti- vation of our testing approach is that this common structure, References applicable to all systems designed using the pattern, should be reusable. In defining the test cases for a particular system, 1. Baudry B, Le Sunyé Y, Jézéquel J (2001) Towards a ‘safe’ use of testers should only be required to invest the incremental effort design patterns to improve OO software testability. In: The 12th necessary to specialize the generalized structure as appropri- international symposium on software reliability engineering, IEEE Computer Society, Washington, DC, pp 324–329 ate to the system under test. Moreover, the ability to define 2. Beck K, Gamma E (1998) Test infected: programmers love writing reusable test case structures enables the software engineer- tests. Java Rep 3(7): 37–50 ing community to encode test case families that are likely to 3. Binder R (1999) Testing object-oriented systems. Addison- reveal the most common pattern implementation errors. Wesley, Menlo Park 4. Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M We described a testing approach that satisfies these goals, (1996) Pattern-oriented software architecture: a system of patterns. and the design of a software tool used to assist in test case gen- Wiley, New York eration. In our approach, pattern test case templates (PTCTs) 5. Chandy K, Misra J (1988) Parallel program design. Addison- are used to define generalized test case structures reusable Wesley, Menlo Park 6. Dasiewicz P (2005) Design patterns and object-oriented software across applications of particular patterns. Each template is testing. In: The 2005 Canadian conference on electrical and com- expressed using a Java-like syntax amenable to practitioner puter engineering, IEEE Canada, Dundas adoption; the code is written in terms of the role types and role 7. Dong J (2002) UML extenstions for design pattern compositions. methods specified by the corresponding pattern specification. In: Mingins C (ed) Proceedings of TOOLS, in special issue of journal of object technology, vol 1, issue 3, pp 149–161 To test the behavior of a system implemented using a partic- 8. Dong J, Alencar P, Cowan D (2001) A behavioral analysis ular pattern, the corresponding PTCTs must be instantiated approach to pattern-based composition. In: Proceedings of the 7th to generate executable test cases. The instantiation process international conference on object-oriented information systems, is guided by a specification of the mappings between pat- Springer, pp 540–549 9. Eden A (2001) Formal specification of object-oriented design. In: tern roles and system classes, as well as rules that govern Proceedings of the international conference on multidisciplinary system-specific specializations. JDUnit assists in the instan- design in engineering tiation process by guiding developers in selecting the appro- 10. Eden A (2002) LePUS: a visual formalism for object-oriented priate PTCTs and pattern specifications, and in introducing architectures. In: Proceedings of the 6th world conference on inte- grated design and process technology, IEEE Computer Society, application-specific objects, method calls, and arguments. pp 149–159 The output of the tool is a set of JUnit test cases used to 11. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: validate the correctness of the relevant patterns as applied in elements of Reusable OO Software. Addison-Wesley, Menlo Park the system under test9 . 12. Hallstrom J, Soundarajan N, Tyler B (2006) Amplifying the ben- efits of design patterns. In: Aagedal J, Baresi L (eds) Proceedings We conclude with a short comment on the relationship of the 9th international conference on fundamental approaches to between our approach to pattern-centric testing (PCT), and software engineering (FASE), Springer, pp 214–229 test-driven-development (TDD) [2]. In TDD, test cases that 13. Helm R, Holland I, Gangopadhyay D (1990) Contracts: specifying capture the expected behavior of a component are written behavioral compositions in object-oriented systems. In: OOPSLA- ECOOP, pp 169–180 before the component is implemented. Hence, TDD dictates 14. Järvinen H, Kurki-Suonio R (1991) Disco specification language: when test cases must be designed. On the other hand, PCT is marriage of actions and objects. In: Proceedings of the 11th inter- concerned with what must be tested. One could, as dictated by national conference on distributed computing systems, IEEE Com- TDD, develop our test cases before a system is implemented. puter Society, Los Alamitos, pp 142–151 15. Jones C (1990) Systematic software development using VDM. (Of course, relevant portions of the design must be complete Prentice-Hall, Englewood Cliffs, New York since otherwise we would not know which PTCTs to instan- 16. Marick B (2007 (date of last access)) Test patterns repository/ tiate.) Alternatively, following a more conventional process, software testing patterns. http://www.testing.com/test-patterns/ we may design and implement the system first, and then, patterns/ 17. McGregor J (1999) Testpatterns: please stand by. J Object-oriented Program 12:14–19 9 A key guideline for using the JUnit framework is that the tests should 18. McGregor J, Sykes D (2001) A practical guide to testing object- be designed to test behavior rather than individual methods [20]. This oriented software. Addison-Wesley, Menlo Park also applies to our approach, except that the behaviors in question are 19. Mikkonen T (1998) Formalizing design patterns. In: Proceedings those corresponding to the use of a particular pattern. Hence, the PTCTs of 20th ICSE, IEEE Computer Society Press, pp 115–124 typically involve multiple roles. Consequently, the instantiated test cases 20. Rainsberger J (2005) JUnit recipes. Manning involve multiple system classes, by contrast to more traditional unit 21. Reenskaug T (1996) Working with objects. Prentice-Hall, Engle- tests. wood Cliffs 123
  15. 15. Patterns: from system design to software testing 85 22. Reimer D, Schonberg E, Srinivas K, Srinivasan H, Alpern B, 27. Soundarajan N, Hallstrom J (2006) Pattern-based system evolu- Johnson R, Kershenbaum A, Koved L (2004) “saber”: smart analy- tion: a case-study. In: Zhang K, Spanoudakis G, Visaggio G (eds) sis based error reduction. In: Proceedings of “ISSTA ’04”, ACM Proceedings of 18th international conference on software engineer- Press, pp 243–251 ing and knowledge engineering (SEKE 2006), Knowledge Systems 23. Riehle D (1997) Composite design patterns. In: Proceedings of Institute, pp 321–326 OOPSLA, ACM, pp 218–228 28. Testing (2001) Patterns of software testing 1 24. Schmidt D, Stal M, Rohnert H, Buschmann F (1996) Pattern- 29. Testing (2001) Patterns of software testing 2 oriented software architecture: patterns for concurrent and net- 30. Testing (2001) Patterns of software testing 3 worked objects. Wiley, New York 31. Tyler B, Hallstrom J, Soundarajan N (2006) A comparative study of 25. Shalloway A, Trott J (2002) Design patterns explained. Addison- monitoring tools for pattern-centric behavior. In: Hinchey M (ed) Wesley, Menlo Park Procedings of 30th IEEE/NASA sofware engineering workshop 26. Soundarajan N, Hallstrom J (2004) Responsibilities and rewards: (SEW-30), IEEE-Computer Society specifying design patterns. In: Finkelstein A, Estublier J, 32. Vlissides J (1998) Notation, notation, notation. C++ Report Rosenblum D (eds) Proceedings of 26th international conference on software engineering (ICSE), IEEE Computer Society, pp 666– 675 123

×